geopm_agent_frequency_map(7) – agent for running regions at user selected frequencies

Description

The frequency map agent adjusts CPU frequency in response to changes in the application region being executed on each CPU. The agent policy can specify a map from a GEOPM region hash to a CPU frequency; see geopm_hash(3) for more information about the GEOPM region name hash function. When a CPU is executing a region with a hash provided by the policy, the CPU will operate at the mapped frequency. The policy also specifies a default CPU frequency which is applied to all CPUs running the application while not executing a region described by the policy map. In addition, the policy can specify a default GPU frequency which is applied to all GPUs at the board level.

Applying any control algorithm that changes CPU frequency on short intervals can result in undesirable interactions with the hardware control algorithm that dynamically selects uncore frequency. For this reason, the frequency map agent policy can specify an option to fix the uncore frequency to a single value while executing the application. If this option is not specified, the uncore frequency is not controlled by the agent at all.

Note

This agent can be used to control the application CPUs, optionally any GPUs, and optionally the uncore to run at fixed frequencies if no region frequency map is provided.

Agent Name

The agent described in this manual is selected in many geopm interfaces with the "frequency_map" agent name. This name can be passed to geopmlaunch(1) as the argument to the --geopm-agent option, or the GEOPM_AGENT environment variable can be set to this name (see geopm(7)). This name can also be passed to the geopmagent(1) as the argument to the '-a' option.

Policy Parameters

FREQ_CPU_DEFAULT:
The operating frequency in units of Hz set by the agent on CPUs executing the application while the application is not within a policy-mapped region. The cores not associated with the application (including the one running the GEOPM controller) will run at the default frequency. This policy parameter is required and must be specified with a CPU frequency that is allowed by the system. Failure to provide this parameter, setting it to an out of range value or specifying NAN will result in a runtime error.

FREQ_GPU_DEFAULT:
The operating frequency in units of Hz is set by the agent on GPUs. Note that the GPUs not associated with the application will also have their frequency adjusted. This policy parameter is optional. Setting the parameter to an out of range value or setting it on a system with no GPUs will result in an error.

FREQ_CPU_UNCORE:
The operating frequency for the uncore clock in units of Hz. If specified, the uncore clock will operate at the fixed frequency provided. If the parameter is not specified, or the value provided is NAN, then the system default range of uncore frequency will be allowed.

HASH_0:
The GEOPM hash of a region name with a mapped CPU frequency given by the FREQ_0 policy parameter. The region hash values for each application region are provided in the GEOPM report, and can be generated as described in geopm_hash(3). In binary code, the policy values are stored the double precision representation of the 32-bit integer generated by the hash function. When using the geopmagent(1) interface to generate a policy file, the region name can be provided in the comma separated list for the -p option rather than the hash of that string. See EXAMPLES section below for details.

FREQ_0:
The CPU frequency in units of Hz that will be set when a CPU is executing the region associated with the HASH_0 region hash. When using the geopmagent(1) command line interface to generate a policy file, the mapped frequency will always follow the region name or hash in the comma separated list for the -p option.

HASH_1:
The next region hash in the map after HASH_0.

FREQ_1:
The next CPU frequency in the map after FREQ_0.

…

HASH_30:
There are at most 31 values in the map provided by the policy.

FREQ_30:
The CPU frequency mapped by HASH_30.

Policy Requirements

The frequency map index values can be in the range of 0 to 30, inclusive. The order of index values does not matter, and gaps in index values are permitted. Multiple definitions of an index are not permitted, and multiple mappings of a region are not permitted. If a CPU, GPU, or uncore frequency specified in the policy is not allowed by the system at runtime, an error will occur and an exception will be raised.

Report Extensions

The assigned frequency is added to each region’s section of the report if that region was explicitly assigned a frequency in the policy map.

Control Loop Rate

The agent gates the control loop to sample the region hash and control CPU frequency at 2 millisecond intervals. If the uncore and/or GPU frequency is specified in the policy, then it is set at agent start-up time.

Examples

Besides the default CPU frequency, default GPU frequency, and uncore frequency parameters, the policy consists of the map of per-region frequencies given as pairs of region hash and CPU frequency values. The recommended way to generate a policy file for this agent is to use the geopmagent(1) command line tool.

To create a policy with a default frequency of 3 GHz, an uncore frequency of 2 GHz, and a GPU frequency of 1.2 GHz with a map of "region_1" to 1.5 GHz and "region_2" to 2.5 GHz you can specify:

geopmagent -a frequency_map -p '3e9,2e9,1.2e9,region_1,1.5e9,region_2,2.5e9'

Alternatively you can specify the regions by their hashed values:

geopmagent -a frequency_map -p '3e9,2e9,1.2e9,0xee12ee15,1.5e9,0xfd421de1,2.5e9'

Although the geopmagent(1) is the recommended tool for creating the json policy string, there is one thing to note about creating the json string without this tool. Since json does not support hex literals, the json must include the hashes in their decimal form. For example, the above policy json is:

{"FREQ_CPU_DEFAULT": 3e9,
 "FREQ_CPU_UNCORE": 2e9,
 "FREQ_GPU_DEFAULT" : 1.2e9,
 "HASH_0": 3994218005,
 "FREQ_0": 1.5e9,
 "HASH_1": 4248968673,
 "FREQ_1": 2.5e9}

The geopmread(1) command line tool can be useful for learning the bounds of these system parameters. The minimum, sticker, and maximum CPU frequencies are queried as below:

$ geopmread CPU_FREQUENCY_MIN_AVAIL board 0
1000000000

$ geopmread CPU_FREQUENCY_STICKER board 0
2100000000

$ geopmread CPU_FREQUENCY_MAX_AVAIL board 0
3700000000

The minimum and maximum GPU frequencies are queried as below:

$ geopmread GPU_CORE_FREQUENCY_MIN_AVAIL board 0
1000000000

$ geopmread GPU_CORE_FREQUENCY_MAX_AVAIL board 0
3700000000

The minimum, and maximum uncore frequencies are queried as below:

$ geopmread MSR::UNCORE_RATIO_LIMIT:MIN_RATIO board 0
1200000000

$ geopmread MSR::UNCORE_RATIO_LIMIT:MAX_RATIO board 0
2400000000

The geopmagent and geopmread command line tools can be used together to generate policies. For example, to create a policy with the default CPU frequency set to two steps below maximum, the GPU and the uncore frequency to the maximum, the following commands can be used:

$ freq_cpu_default=$(($(geopmread CPU_FREQUENCY_MAX_AVAIL board 0) - \
                  2 * $(geopmread CPU_FREQUENCY_STEP board 0)))
$ freq_cpu_uncore=$(geopmread MSR::UNCORE_RATIO_LIMIT:MAX_RATIO board 0)
$ freq_gpu=$(geopmread GPU_CORE_FREQUENCY_MAX_AVAIL board 0)
$ geopmagent -a frequency_map -p$freq_cpu_default,$freq_cpu_uncore,$freq_gpu
  {"FREQ_CPU_DEFAULT": 3500000000, "FREQ_CPU_UNCORE": 2400000000, "FREQ_GPU_DEFAULT":1500000000}