NCI Specific Functions#

These functions are specifically for working with the compute and data resources at NCI, and may not work at other sites.

climtas.nci#

NCI Specific functions and utilities

climtas.nci.Client(threads=1, malloc_trim_threshold=None)[source]#

Start a Dask client at NCI

An appropriate client will be started for the current system

Parameters

threads – Number of threads per worker process. The total number of workers will be ncpus/threads, so that each thread gets its own CPU
malloc_trim_threshold – Threshold for automatic memory trimming. Can be either a string e.g. ‘64kib’ or a number of bytes e.g. 65536. Smaller values may reduce out of memory errors at the cost of running slower

climtas.nci.GadiClient(threads=1, malloc_trim_threshold=None)[source]#

Start a Dask client on Gadi

If run on a compute node it will check the PBS resources to know how many CPUs and the amount of memory that is available.

If run on a login node it will ask for 2 workers each with a 1GB memory limit

Parameters

threads – Number of threads per worker process. The total number of workers will be $PBS_NCPUS/threads, so that each thread gets its own CPU
malloc_trim_threshold – Threshold for automatic memory trimming. Can be either a string e.g. ‘64kib’ or a number of bytes e.g. 65536. Smaller values may reduce out of memory errors at the cost of running slower

climtas.nci.OODClient(threads=1, malloc_trim_threshold=None)[source]#

Start a Dask client on OOD

This function is mostly to be consistent with the Gadi version

Parameters

threads – Number of threads per worker process. The total number of workers will be ncpus/threads, so that each thread gets its own CPU
malloc_trim_threshold – Threshold for automatic memory trimming. Can be either a string e.g. ‘64kib’ or a number of bytes e.g. 65536. Smaller values may reduce out of memory errors at the cost of running slower