See: http://inside.mines.edu/mio/newpolicy.html for the HTML version of this note.
We are deploying a new file system for Mio and RA. The file system is up but not yet available for use. We are building directory structures and let you know when you can move your data. The file system on BlueM will have the same policies as outlined here.
Along with the new install we are implementing new policies to prevent many of the issues we have seen in recent time with the current file system. This policy is discussed here.
Each user will have three base directories:
In addition a group may have a $SETS directory which is designed for semipermanent data sets that will be used repeatedly by the group. $SETS can contain things like equations of state or velocity fields. It may also contain programs used by multiple members of a group. $SETS will be readable on the compute nodes.
$HOME - Should be kept very small, having only start up scripts and other simple scripts. Output from parallel jobs can not be directed to $HOME. It will be read only from compute nodes..
$DATA - Should contain programs users have built for personal use and small data sets and run scripts. Output from parallel jobs can not be directed to $DATA It will be read only from compute nodes..
$SCRATCH - The main area for running applications. Output from parallel runs should be done to this directory.
$HOME and $DATA will be bigger than they have been in the past but still much smaller that $SCRATCH.
$HOME and $DATA will have quotas:
$SCRATCH will not have quotas. However, data in $SCRATCH should be cleaned after runs to prevent the the file system from filling.
Users have two options for keeping $SCRATCH clean. Users will be placed in one of two classes: Automatic or Manual. The default class will be Manual. A group's PI can be asked people to be placed in either class by sending email to email@example.com.
Monitoring for the users in the Manual class will consist of the system periodically running a command to determine the amount of space being used. Usage will be deemed excessive if on two successive usage checks one of the following occurs:
When a person leaves CSM the PI is responsible for that person must designate a recipient of the persons data. The recipient will be "given" ownership of the data. They will be free to move it or delete it. After 3 months the directories which belonged to the person who has left will be removed.