Data Handling and Storage Policy
Data handling and storage policy
All data accessed, stored, communicated or transferred on UiB's HPC systems (e.g. Hexagon), must be handled in compliance to legal and regulatory requirements.
In addition, all data has to be directly related to the work effectuated and/or the research project(s) the user is participating.
User area: /home
User's private data (such as keys, sessions, e-mail, etc.) may reside in their home directory ($HOME). $HOME is not a shared area and all data stored there has to be treated as being private, regardless of it's content.
To limit access to $HOME only to the user and designated system administrators, the default directory permissions are set to 0700. Permissions are regularly controlled and in case of mismatch reset to defaults.
Scratch area: /work
/work/users/$USER directory is used as a main storage during data processing. All data after processing must be moved out of the machine or deleted after use.
Deletion policy:
- Automatic deletion starts when file system usage is above 70%.
- Deletion depends on modification, access time and the total usage in the file system.
- The oldest files will be deleted first.
- The newest files which can be deleted without user notification are 3 weeks old. All three parameters, like access, modification and change counts.
Project data is private to the project and shared between the project members. The project leader (PL) has sole discretion over project members, thus access to the project area(s).
Projects has their own directory created under /shared/projects with permissions set to 2770 to set the global group ID. Group ownership is regularly controlled for each project directory and reset in case needed. This is required for storage accounting purposes.
Project collaborations
In special cases there might be a need for sharing data between projects for collaboration and possibly preventing data duplication.
If such a need is justified, a meta-group and it's according directory is created. Access to the shared project area is at the PL's sole discretion. For accounting purposes, the group ownerships are regularly controlled and in case needed reset.
Back-up policy
Incremental backups (only modified files) of user home directories (/home) are made every night. All versions of a file for the last 90 days are available from backup, a deleted file remains in backup for 365 days before it expires.
The following files are excluded from backup:
- contents of directories (and all their subdirectories) named tmp, TMP, temp, TEMP, scratch, SCRATCH or Scratch. Do not put any valuable data in such directories.
Backups are NOT taken for directories on different file systems (e.g. /work or /shared file systems).
