Access to the resources

The use of Garnatxa is open to I2SysBio’s groups and external companies or private/public institutions. The access policies of each entity are described below.

• I2SysBio’s members

  All research groups belonging to I2SysBio have free access to the Garnatxa cluster. In addition, the groups will be able to contribute resources from their projects to improve or expand it and will receive the compensation described below. If your group would like to make a contribution by purchasing resources (storage or CPU) then contact us via the ticket platform.

• External companies, laboratories and public institutions

  If you belong to a private company or public institution and you are interested in using our computational resources, there are two ways to do so.

  • Economic contribution:

    • You must make an initial financial contribution of the amount you deem appropriate, which will translate into computation hours and storage capacity.

    • Each month, you will receive a report with a breakdown of your expenditure in terms of CPU hours and storage capacity used.To calculate the cost of use, we take into account the public rates. approved annually by the CSIC.

    • Once the initial financial contribution has been used up, you can make new financial contributions that will allow you to extend your use of Garnatxa or, conversely, to cease using it. If you do not wish to continue using Garnatxa, you will be given a reasonable period of time to transfer your data to your local storage system.

  • Purchase of equipment:

    • The minimum contribution is one node, which must have at least 64 cores and 100 TB of storage.

    • The contributed nodes will be added to the total number of nodes available at that time to Garnatxa and may be used by all users.

    • Your organization becomes a regular user of Garnatxa, with the same privileges and restrictions as internal I2Sysbio users.

    • The 70% of the purchased storage capacity is guaranteed for use by the entity based on the disk quota system. 95% of the CPUs are guaranteed for use by the entity.

    • The computational resources contributed will be maintained and managed by I2SysBio’s IT staff in the same way as the rest of the infrastructure.

    • The contribution of equipment by an external organization will be reflected in a document signed by both parties specifying the conditions of use of the Garnatxa System.

    • These conditions will be valid for 5 years from the date of receipt of the equipment at the I2SysBio facilities.

    • I2SysBio shall guarantee the availability of and access to the resources provided for a period of 5 years, three of which must be covered by the equipment warranty.

Groups and users management

• The creation of a new research group to work with Garnatxa must be requested through the ticket platform (https://garnatxadoc.uv.es/support) under the topic “Cluster GarnatxaHPC / General Support”.

• The creation of users is requested personally by the main researcher (IP) or their delegate by opening a ticket through the I2SysBio ticketing platform (https://garnatxadoc.uv.es/support) under the topic ‘Cluster Garnatxa HPC / New Account Request’. The documentation relating to support is available at https://garnatxadoc.uv.es/contact/contact.html.

• The IP must provide contact details about new user, such as their official email address, and notify us of any changes. The user’s email address will be added to a distribution list for Garnatxa users, which is used to send important communications.

• User accounts that have been inactive for more than one year are blocked and the IP is notified. The group leader must inform Garnatxa administrators about the reactivation or permanent deletion of the account and its associated data.

Resource management and limits

• Garnatxa uses a resource management system called SLURM. A set of partitions and global queues are configured that all users can access, subject to certain limits.

• The system defines two partitions: interactive and global. The queues short, medium, long, and long-mem queues are limited by the number of CPUs per user, memory per user and execution time. The tape queue is used to send jobs that require data transfer to tape.

• In addition to the individual limits per queue type a global limit per user is established that the set of jobs launched to the system cannot exceed ( see detailed limits ). If these limits are exceeded, the jobs remain in a queued state awaiting the release of resources.

• Queued jobs are reordered by the SLURM system based on the score obtained from these four parameters:

  • Queue type: Queues with shorter execution times have higher scores.

  • Age: time spent in the queued state. The longer the time, the higher the score.

  • Fairshare: overall time a user has been running in the last 24-hour window. The longer a user uses the cluster, the lower the score for this parameter.

  • Job size: Jobs with fewer CPUs and less memory requested receive a higher score.

• The reordering system is dynamic, so the execution order of a queued job will change over time depending on the age and fairshare parameters.

• Garnatxa is a computing system available to I2SysBio researchers, who can access it free of charge. In addition, groups can contribute resources from their projects to improve or expand it, and will receive compensation as described below. This proposal for managing computational resources aims to encourage the incorporation of new resources into the system.

  • It is necessary to propose a mechanism that allows the resources contributed by a group to be made available primarily to that group. Each computing team that joins must allocate a portion of its resources to maintaining the globally distributed file system. The amount contributed is set at 5% of the CPU acquired by the group.

  • A queue associated with the contributed node is defined. I2SysBio members may use this queue (in addition to the rest) freely for a maximum of 24 hours. Members belonging to the contributed node may use this queue (in addition to the rest) freely without any time limit. They will also have priority in the execution of jobs on this queue.

Storage management

Garnatxa has a distributed online primary storage system and a secondary tape archiving storage system.

• Secondary storage:

  • Data that is obsolete or has not been used for a long period of time can be moved to magnetic tapes, reducing the storage space used by the group in the primary storage system.

  • Each research group may voluntarily purchase the tape units it may need for its projects. The procedure for acquiring and using the tape device is explained here.

• Primary storage:

  • Garnatxa has a distributed storage system with high performance and capacity.

  • Each group has an individual directory where it can store its project data: /storage/<GROUP_NAME>.

  • Each group user has an individual directory where they can store their personal data: /storage/<GROUP_NAME/home_members/<USER_NAME>

  • The full capacity of the primary storage system will be available to all Garnatxa users, as long as the critical point of 80% is not reached. If the global use reach this limit the system applies a dynamic quota system, which is described in detail here.

  • Quotas are calculated based on the storage voluntarily contributed by each research group.

  • No backups are made of data stored in primary storage. The user is responsible for making regular backups on their local devices.

Computing resources

All CPU resources in Garnatxa are available to all the cluster users, respecting the limits of each queue and user described here.

However, a number of CPUs were voluntarily contributed by I2SysBio groups and external collaborators. In these cases, specific queues are configured for each of those groups with the following usage policy:

• The specific group queue has a maximum execution time equivalent to the short queue (24 hours) for users not belonging to the owner group. That is, all users can use this group queue for jobs with a maximum execution time of 24 hours.

• For users belonging to the group that owns the queue, the maximum execution time per job in this queue is unlimited. Members of the group that owns the queue can launch jobs to this queue by adding this parameter: sbatch -q <group_name>. Consult with the administrator if you have questions about how to use these group queues.

• Thus, it is possible for all users to use the CPUs in the group queues while they are free, for a maximum of 24 hours per job.

• Only if the queue is full and an user is belonging to the group queue, they must wait a maximum of 24 hours (at worst) for their job to start running.

Virtualization

Garnatxa has a service virtualization platform (IaaS). It is currently used to provide documentation pages, ticketing, Gitlab repositories, online courses and web hosting services.

• Any I2SysBio group can request the creation of a virtual machine and access to the network. The request will be approved based on the resources requested (CPU, memory, and disk space).

• Once the request has been approved, the CPD staff will start up the virtual machine and provide the requesting group with the access credentials.

• The group is responsible for managing and updating the virtualised service.

Acknowledging

When you use Garnatxa for calculations which are published (for instance: in peer-reviewed literature or at conferences), we require that you acknowledge use of Garnatxa as follows:

The computations/simulations/[OR SIMILAR] were performed on the HPC cluster Garnatxa at Institute for Integrative Systems Biology (I2SysBio), I2SysBio is a mixed research centre formed by University of Valencia (UV) and Spanish National Research Council (CSIC).

Also, please send a copy of your publication to i2sysbiohpc@uv.es