Description

The GCC cluster @ UMCG is a 480 core PBS cluster described in detail here: ​GCC Cluster

Access to the default gcc queue is available to all GoNL members. If you do not have an account yet, you can request one via Morris who keeps the list of all users that have full data access.
For details: ​Request an Account

Usage

First of all here are a few important things to know about the cluster and using it efficiently:

• Storage: The block size on the storage is 6MB, which means that each file -regardless of its real size- will occupy at least 6MB on the file system. This means that data should rather be kept in big files rather than a multitude of small files whenever possible. Typically things like logs, old submit scripts, etc. should be compressed into 1 file for archiving.
• I/O: While 10Gb network connection per node is fast, typical GoNL jobs use large files and consumes lots of I/O. Therefore, I/O should be kept minimal and if a job can be parallelized on multiple cores (i.e. load data once in memory, process it on multiple cores, push it back), it is typically preferred as having separate processes all loading the same data in memory.
• Local Storage: In order to reduce I/O, temporary files (and eventually other heavily used resources) should be stored directly on the local node; the local storage on each node is mounted in /local. Note that:
  • Any data on the local storage that you want to keep after the job terminates should be copied to the general storage as the local storage is periodically cleaned and any data that is not in use by currently running job will be deleted.
  • Even if the local storage is periodically cleaned, if you store large files on a node while running a job you should clean afterwards. Small temp files are fine.
• Data Management: Please read thoroughly the Data Management section of this wiki and respect the structure and conventions described there when using data outside your home directory.