Server Usage Policy

The SSCC makes a large number of powerful servers available, but we also have over three thousand users. Many programs are now capable of claiming all of a server's resources for a single user. The servers are a shared resource, so it's essential that all users share them properly.

We therefore ask all our members to adhere to the following:

  1. Only run one CPU-intensive job at a time on Winstat, or one job on each of the Linstat servers. Jobs like text editors, web browsers, etc. use very little processor power and you are free to run as many of these as you wish. Most statistical software and user-written programs will fully utilize at least one core while carrying out calculations. Some will use multiple cores. Only run one of these programs at a time. If you need to run multiple jobs, use HTCondor.
  2. If you are using programs that can use more than one core, do not use more than 8 cores on Winstat or 16 cores on Long Job Winstat or Linstat. Software-specific instructions for tracking and setting the number of cores used are given below.
  3. Do not use more than 16 gigabytes of RAM on Winstat, or 64 gigabytes of RAM on Linstat. Instructions for tracking RAM usage are given below.
  4. You may submit up to 15 jobs to the SSCC HTCondor flock at any time. You may be able to submit additional jobs depending on how long your jobs will take to run and how many slots are unclaimed at the time you submit them. Use condor_status to find out how many slots are unclaimed.
    Time your jobs will take to run Total Number of jobs you may submit
    is the maximum of 15 or...
    < 3 hours The number of unclaimed slots
    < 1 day 3/4 of the number of unclaimed slots
    > 1 day 1/2 of the number of unclaimed slots

Exceptions to these policies may be made under unusual circumstances or for users with unusual needs. Contact the Help Desk if you believe your case merits an exception.

SSCC staff regularly monitor the load on each of the servers. If we notice jobs that are in violation of this policy, we will let them finish as long as they are not causing problems for other users. But if the jobs are causing problems for other users we may need to terminate them without prior notice.

This will not keep the servers from getting busy on occasion. But it does ensure that they will not be overloaded by a single user or very small group of users.

Controlling the Cores Used by Popular Programs at the SSCC

Matlab

If you are doing parallel processing, run parpool(8) (Winstat) or parpool(16) (Long Job Winstat or Linstat) . Alternatively, click the arrow by Parallel on the Home tab, then Parallel Preferences, and set Preferred number of workers in a parallel pool to 8 (Winstat) or 16 (Linstat).

MPlus

Include the following line in your program if you plan to run it on Winstat:

analysis: process = 8;

On Long Job Winstat, or if you're running Mplus on Linstat replace 8 with 16.

Stata

The number of cores used by Stata depends on which version you run. The default version on Winstat is Stata/MP2, which uses two cores, but you can use eight cores if you run Stata/MP16. On Long Job Winstat, Stata/MP16 will use all 16 cores. On Linstat the default is Stata/MP16, which on Linstat will use a full sixteen cores if they are available. Running Stata MP16 at the SSCC has details.

R

R will only use 1 core unless you are using a package designed for parallel processing. If you are using such a package, consult its documentation to see how to control the number of cores it uses. For example, if you are using parLapply you should run makeCluster(8) (Winstat) or makeCluster(16) (Long Job Winstat or Linux).

SAS

You do not need to set the number of cores used by SAS. SAS is set to use 4 cores on Winstat and 16 cores on Linstat.

SPSS

You do not need to set the number of cores used by SPSS. It will use 4 cores on Winstat.

User-written programs

If you are already using MPI to handle parallelization, consider asking for an account on the SSCC's High Performance Computing Cluster.

Tracking RAM Usage

Most statistical software loads the data it is working on into RAM, and the amount of RAM used will be slightly larger than the size of the data used. Be aware of the size of your data set. If your data set takes a long time to load consider that a warning sign that you need to start paying attention to RAM.

Some programs will tell you how much RAM they are using. For example, Stata lists current memory usage among the properties of the data set.

On Winstat, to see how much memory any program is using right click on the gray bar at the bottom of the screen, choose Start Task Manager, go to the Processes tab, identify your job, and check the Memory column.

On Linstat, to see how much memory any program is using type top. Identify your job and check the VIRT (virtual memory) column.