Server Usage Policy

The SSCC makes a large number of powerful servers available, but we also have over three thousand users. Many programs are now capable of claiming all of a server's resources for a single user. The servers are a shared resource, so it's essential that all users share them properly.

We therefore ask all our members to adhere to the following:

  1. Only run one CPU-intensive job at a time on Winstat, or one job on each of the Linstat servers. Jobs like text editors, web browsers, etc. use very little processor power and you are free to run as many of these as you wish. Most statistical software and user-written programs will fully utilize at least one core while carrying out calculations, others may use multiple cores. If you need to run multiple jobs, use HTCondor.
  2. Do not use more than 8 cores on Winstat or 36 cores on Linstat. Software-specific instructions for tracking and setting the number of cores used are given below.
  3. Do not use more than 32 gigabytes of RAM on Winstat. Instructions for tracking RAM usage are given below.
  4. You may submit up to 15 jobs to the SSCC HTCondor flock at any time. You may be able to submit additional jobs depending on how long your jobs will take to run and how many slots are unclaimed at the time you submit them. Use condor_status to find out how many slots are unclaimed.
    Time your jobs will take to run Total Number of jobs you may submit
    is the maximum of 15 or...
    < 3 hours The number of unclaimed slots
    < 1 day 3/4 of the number of unclaimed slots
    > 1 day 1/2 of the number of unclaimed slots

Exceptions to these policies may be made under rare circumstances. Contact the Help Desk if you believe your case merits an exception.

SSCC staff regularly monitor the load on each of the servers. If we notice jobs that are in violation of these policies, we will contact the job owner, but let the job finish as long as it is not causing problems for other users. However, if the job disrupts usage for other users, it may be terminated without prior notice.

This will not keep the servers from getting busy on occasion, but it does ensure that they will not be overloaded by a single user or very small group of users.

Controlling the Cores Used by Popular Programs at the SSCC

Matlab

If you are doing parallel processing, run parpool(8) (Winstat) or parpool(36) (Linstat) . Alternatively, click the arrow by Parallel on the Home tab, then Parallel Preferences, and set Preferred number of workers in a parallel pool to 8 (Winstat) or 36 (Linstat).

MPlus

Include the following line in your program if you plan to run it on Winstat:

analysis: process = 8;

If you're running Mplus on Linstat replace 8 with 36.

Stata

The number of cores used by Stata depends on which version you run. The default version on Winstat is Stata/SE, which uses one core, but you can use eight cores if you run Stata/MP. On Linstat the default is Stata/MP, which on Linstat will use 32 cores if they are available. Running Stata/MP at the SSCC has details.

R

R will only use 1 core unless you are using a package designed for parallel processing. If you are using such a package, consult its documentation to see how to control the number of cores it uses. For example, if you are using parLapply you should run makeCluster(8) (Winstat) or makeCluster(36) (Linux).

SAS

You do not need to set the number of cores used by SAS. SAS is set to use 4 cores on Winstat and 16 cores on Linstat.

SPSS

You do not need to set the number of cores used by SPSS. It will use 4 cores on Winstat.

User-written programs

If you are using OpenMP with FORTRAN, you can set the number of cores to use with:

!$ call omp_set_num_threads(36)

If you are using MPI such that your job can take advantage of multiple machines, consider asking for an account on the SSCC's High Performance Computing Cluster.

Tracking RAM Usage

Most statistical software loads the data it is working on into RAM, and the amount of RAM used will be slightly larger than the size of the data used. Be aware of the size of your data set. If your data set takes a long time to load consider that a warning sign that you need to start paying attention to RAM.

Some programs will tell you how much RAM they are using. For example, Stata lists current memory usage among the properties of the data set.

To see how much memory any program is using on Winstat, right click on the gray bar at the bottom of the screen, choose Start Task Manager, go to the Processes tab, identify your job, and check the Memory column.