1. geopm(7)
  2. geopm(7)

NAME

geopm - global extensible open power manager

DESCRIPTION

The Global Extensible Open Power Manager (GEOPM) is a framework for exploring power and energy optimizations targeting high performance computing. The GEOPM package provides many built-in features. A simple use case is reading hardware counters and setting hardware controls with platform independant syntax using a command line tool on a particular compute node. An advanced use case is dynamically coordinating hardware settings across all compute nodes used by an application is response to the application's behavior and requests from the resource manager. The dynamic coordination is implemented as a hierarchical control system for scalable communication and decentralized control. The hierarchical control system can optimize for various objective functions including maximizing global application performance within a power bound or minimizing energy consumption with marginal degradation of application performance. The root of the control hierarchy tree can communicate with the system resource manager to extend the hierarchy above the individual MPI application and enable the management of system power resources for multiple MPI jobs and multiple users by the system resource manager.

JOB LAUNCH

geopmlaunch(1): Application launch wrapper

APPLICATION PROFILING

geopm_prof_c(3): Application profiling interfaces

geopm_fortran(3): GEOPM Fortran interfaces

ANALYSIS TOOLS

geopmread(1): Query platform information

geopmwrite(1): Modify platform state

geopm_topo_c(3): Query platform component topology

geopm_pio_c(3): Interfaces to query and modify platform

geopmbench(1): Synthetic benchmark application

BUILT-IN AGENTS

geopm_agent_monitor(7): Agent implementation that enforces no policies

geopm_agent_energy_efficient(7): Agent for saving energy, also finds optimal region frequencies

geopm_agent_frequency_map(7): Agent for running regions at user selected frequencies

geopm_agent_power_balancer(7): Agent that optimizes performance under a power cap

geopm_agent_power_governor(7): Agent that enforces a power cap

RUNTIME CONTROL

geopm_ctl_c(3): GEOPM runtime control thread

geopmctl(1): GEOPM runtime control application

geopm_agent_c(3): Query information about available agents

geopmagent(1): Query agent information and create static policies

MISC

geopm_error(3): Error code descriptions

geopm_version(3): GEOPM library version

geopm_sched(3): Interface with Linux scheduler

geopm_time(3): Time related helper functions

geopm_hash(3): Numerical encoding helper functions

PLUGIN EXTENSION

geopm::PluginFactory(3): Plugin developer guide

geopm::PlatformIO(3): High level platform abstraction

geopm::IOGroup(3): Plugin interface for platform

geopm::Agent(3): Plugin interface for monitor/control

INTEGRATION WITH PMPI

Linking to libgeopm will define symbols that intercept the MPI interface through PMPI. This can be disabled with the configure time option --disable-pmpi, but is enabled by default. See LD_DYNAMIC_WEAK environment variable description below for the runtime requirements of the PMPI design. When using the GEOPM PMPI interposition other profilers which use the same method will be in conflict. The GEOPM runtime can create an application performance profile report and a trace of the application runtime. As such, GEOPM serves the role of an application profiler in addition to management of power resources. The report and trace generation are controlled by the environment variables GEOPM_REPORT and GEOPM_TRACE; see description below.

INTEGRATION WITH OMPT

Unless the GEOPM runtime is configured to disable OpenMP, the library is compiled against the OpenMP runtime. If the OpenMP implementation that GEOPM is compiled against supports the OMPT callbacks, then GEOPM will use the OMPT callbacks to wrap OpenMP parallel regions with calls to geopm_prof_enter() and geopm_prof_exit(). In this way, any OpenMP parallel region not within another application-defined region will be reported to the GEOPM runtime. This will appear in the report as a region name beginning with "[OMPT]" and referencing the object file and function name containing the OpenMP parallel region e.g.

[OMPT]geopmbench:geopm::StreamModelRegion::run()

To expressly enable this feature, pass the --enable-ompt configure flag at GEOPM configure time. This will build and install the LLVM OpenMP runtime configured to support OMPT if the default OpenMP runtime does not support the OMPT callbacks. Note that your compiler must be compatible with the LLVM OpenMP ABI for extending it in this way.

LAUNCHING THE RUNTIME

The recommended method for launching the GEOPM runtime is the job launch wrapper script geopmlaunch(1). See this man page for details about the command line interface. If your system does not support aprun or srun for job launch, please make a change request for support of the job launch method used on your system at the github issues page:

https://github.com/geopm/geopm/issues

Also, consider porting your job launch command into the geopmpy.launcher module and submitting a change request as described in CONTRIBUTING.md.

If the job launch application is not supported by the geopmpy.launcher the recommended method is to use the environment variables described in this man page including the GEOPM_CTL environment variable. If using the "application" launch method then the geopmctl(1) application should be launched in parallel.

There are legacy methods for launching the runtime programmatically. These are documented in geopm_ctl_c(3), but are deprecated as an application-facing interface because their use within an application is incompatible with the GEOPM launcher script.

INTERPRETING THE REPORT

If the GEOPM_REPORT environment variable is set then a report will be generated. There is one report file generated for each run. This file has a header that describes the GEOPM version, job start time, profile name (job description), and agent that were used during the run. This is followed by a breakdown first by compute node host name and then by each compute region. The report file gives information about runtime and energy use for each identified code region. Additionally, time spent in calls to MPI is reported. There may be Agent specific report modifications see geopm::Agent(3) man page for more information.

It is important to note that per-region accounting in the report includes only time spent in the region by all MPI ranks concurrently on each compute node. During the time when two ranks on a compute node are not in the same marked region, the data collected is attributed to the "unmarked" code region. See the GEOPM_REGION_BARRIER environment variable for more information about debugging issues related region synchronicity. In the future the scope of this requirement may change from all ranks on a node to all ranks running within the same domain of control.

The hint that is used when a user marked region is created may influence the accounting of time spent in MPI. For more information, please see see geopm_prof_c(3).

INTERPRETING THE TRACE

If the GEOPM_TRACE environment variable is set (see below) then a trace file with time ordered information about the application runtime is generated. A separate trace file is generated for each compute node and each file is a pipe (the | character) delimited ASCII table. The file begins with a header that is marked by lines that start with the # character. The header contains information about the GEOPM version, job start time, profile name (job description), and agent that were used during the run.

The first row following the header gives a description of each field. A simple method for selecting fields from the trace file is with the awk command:

$ grep -v '^#' geopm.trace-host0 | awk -F\| '{print $1, $2, $11}'

will print a subset of the fields in the trace file called "geopm.trace-host0".

ENVIRONMENT

When using the launcher wrapper script geopmlaunch(1), the interface to the GEOPM runtime is controlled exclusively by the launcher command line options. The launcher script sets the environment variables described in this section according to the options specified on the command line. Direct use of these environment variables is only recommended when launching the GEOPM runtime without geopmlaunch(1). If launching the GEOPM controller in application mode without geopmlaunch(1), the environment variables documented below must be set to the same values in the contexts where geopmctl(1) and the compute application are executed.

LD_DYNAMIC_WEAK

When dynamically linking an application to libgeopm for any features supported by the PMPI profiling of the MPI runtime it may be required that the LD_DYNAMIC_WEAK environment variable be set at runtime as is documented in the ld.so(8) man page. When dynamically linking an application, if care is taken to link the libgeopm library before linking the library providing the weak MPI symbols, e.g. "-lgeopm -lmpi", linking order precedence will enforce the required override of the MPI interface symbols and the LD_DYNAMIC_WEAK environment variable is not required at runtime.

GEOPM_REPORT

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-report.

GEOPM_REPORT_SIGNALS

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-report-signals.

GEOPM_TRACE

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-trace.

GEOPM_TRACE_SIGNALS

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-trace-signals.

GEOPM_PROFILE

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-profile.

GEOPM_CTL

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-ctl.

GEOPM_AGENT

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-agent.

GEOPM_POLICY

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-policy.

GEOPM_ENDPOINT

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-endpoint.

GEOPM_SHMKEY

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-shmkey.

GEOPM_TIMEOUT

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-timeout.

GEOPM_PLUGIN_PATH

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-plugin-path.

GEOPM_DEBUG_ATTACH

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-debug-attach.

GEOPM_REGION_BARRIER

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-region-barrier.

GEOPM_DISABLE_HYPERTHREADS

See documentation for equivalent command line option to geopmlaunch(1) called --geopm-hyperthreads-disable.

Copyright (c) 2015, 2016, 2017, 2018, 2019, Intel Corporation. All rights reserved.

SEE ALSO

geopmpy(7), geopm_agent_energy_efficient(7), geopm_agent_frequency_map(7), geopm_agent_monitor(7), geopm_agent_power_balancer(7), geopm_agent_power_governor(7), geopm_agent_c(3), geopm_ctl_c(3), geopm_error(3), geopm_fortran(3), geopm_hash(3), geopm_prof_c(3), geopm_sched(3), geopm_time(3), geopm_version(3), geopm::Agent(3), geopm::Agg(3), geopm::CircularBuffer(3), geopm::CpuinfoIOGroup(3), geopm::EnergyEfficientAgent(3), geopm::EnergyEfficientRegion(3), geopm::Exception(3), geopm::Helper(3), geopm::IOGroup(3), geopm::MSR(3), geopm::MSRIO(3), geopm::MSRIOGroup(3), geopm::MonitorAgent(3), geopm::PlatformIO(3), geopm::PlatformTopo(3), geopm::PluginFactory(3), geopm::PowerBalancer(3), geopm::PowerBalancerAgent(3), geopm::PowerGovernor(3), geopm::PowerGovernorAgent(3), geopm::ProfileIOGroup(3), geopm::ProfileIOSample(3), geopm::RegionAggregator(3), geopm::SharedMemory(3), geopm::TimeIOGroup(3), geopmagent(1), geopmbench(1), geopmctl(1), geopmlaunch(1), geopmread(1), geopmwrite(1), ld.so(8)

  1. Intel Corporation
  2. April 2019
  3. geopm(7)