Table of Contents
In the Aeolus environment there are multiple storage locations available. Each has a a different intended use, speed (read & write throughput), and cost associated to it.
Home directories are the landing place for an account that logs in to the system. This typically would be where people modify their environment to be unique to their needs, develop code, keep a diary, test various things, et cetera. To maintain a simple standard of data security on the server, ~/ directories should NOT be readable/executable by anyone but the owner.
All users are provided with a 100GB of home directory storage, thanks to generous contributions of the Voiland College of Engineering and Architecture as well as other initial investors. ~/ home directories will have a 100 GB cap to them and exist on a slower (1 Gb) network. When this cap gets implemented, you will no longer have write access to your ~/ directory until it is cleaned up.
Each invested lab, group, or investor in Aeolus has a /data/ location for shared binaries, project data, and group collaborative [stuff]. As an example of /data/* hierarchy:
/data/LAB - a data directory with access granted by those within that group
/data/LAB/BIN/ - a shared directory for custom binaries/modules
/data/LAB/PROJECTS/ - a directories for various projects within the group
/data/LAB/USERS/ - user specific data directories and projects
/data/LAB/ - anything else the group wants
Once a /data/LAB/{bin,projects,users} mount point has been set up, groups are allowed to create directories within that structure on their own.
Per the request & coordination of the PI(s), each /data/ storage group can set user quotas. To view an assigned user quota, each users can run the myquota command:
[user@aeolus ~]$ myquota --------------- Quotas --------------- /data/group: 123.45GiB used of 234.56GiB -------------------------------------- |
Fastscratch is high speed, volatile (short lived) storage. The expected lifetime of data on /fastscratch/ is 2 weeks. If you want data to stay alive on /fastscratch/ longer than that, you will be required to touch the file(s) to indicate they are being used. The data structure hierarchy of /fastscratch/ is flat. This means, all users have the ability to create their own /fastscratch/MYDIR directory.
Fastscratch is currently built from a raid 10 of 24 SSDs.
Archive storage is slow speed storage that sits behind a 1Gb network and is available for groups to have a cheap alternative to keep data in the HPC environment without needing write access to it.
Backup storage is currently searchable upon request. As new storage is purchased and larger backup storage is built, we will be providing a navigable, read only, path /backup/ for convenient access.
Scratch storage is node-local (only available on that one compute node) temporary SSD storage. Scratch storage is to be used only for temporary use. Files are to be deleted at the end of jobs/runs. Files that persist on scratch storage longer than 7 days will be deleted.
/data-failing/ is the resulting leftover data mounted in read-only, for groups to access while we migrate data to new storage systems.