|
Linux Kernel Power Management 29 April 2003 Patrick Mochel Abstract Power management is the process by which the overall consumption of power by a computer is limited based on user requirements and policy. Power management has become a hot topic in the computer world in recent years, as laptops have become more commonplace and users have become more conscious of the environmental and financial effects of limited power resources. While there is no such thing as perfect power management, since all computers must use some amount of power to run, there have been many advances in system and software architectures to conserve the amount of power being used. Exploiting these features is key to providing good system- and device-level power management. This paper discusses recent advances in the power management infrastructure of the Linux kernel that will allow Linux to fully exploit the power management capabilities of the various platforms that it runs on. These advances will allow the kernel to provide equally great power management, using a simple interface, regardless of the underlying archtitecture. This paper covers the two broad areas of power management - System Power Management (SPM) and Device Power Management (DPM). It describes the major concepts behind both subjects and describes the new kernel infrastructure for implement both. It also discusses the mechanism for implementing hibernation, otherwise known as suspend-to-disk, support for Linux. Overview Benefits of Power Management A sane power management infrastructure provides many benefits to the kernel, and in not only the obvious areas. Battery-powered devices, such as embedded devices, handhelds, and laptops reap most of the rewards of power management, since the more conservative the draw on the battery is, the longer it will last. System power management decreases boot time of a system, by restoring previously saved state instead of reinitializing the entire system. This conserves battery life on mobile devices the annoying wait for the computer to boot into a useable state. Recently, power management concepts have begun to filter into less obvious places, like the enterprise. In a rack of servers, some servers may power down during idle times, and power back up when needed again to fulfill network requests. While the power consumption of a single server is but a drop in the water, being able to conserve the power draw of dozens or hundreds of computers could save a company a significant amount of money. Also, at the lower-level, power management may be used to provide emergency reaction to a critical system state, such as crossing a pre-defined thermal threshold or reaching a critically low battery state. The same concept can be applied when triggering a critical software state, like an Oops or a BUG() in the kernel. System and Device Power Management There are two types of power management that the OS must handle - System Power Management and Device Power Management. Device Power Management deals with the process of placing individual devices into low-power states while the system is running. This allows a user to conserve power on devices that are not currently being used, such as the sound device in my laptop while I write this paper. Individual device power management may be invoked explicitly on devices, or may happen automatically after a device has been idle for a set of amount of time. Not all devices support run-time power management, but those that do must export some mechanism for controlling it in order to execute the user's policy decisions. System Power Management is the process by which the entire system is placed into a low-power state. There are several power states that a system may enter, depending on the platform it is running on. Many are similar across platforms, and will be discussed in detail later. The general concept is that the state of the running system is saved before the system is powered down, and restored once the system has regained power. This prevents the system from performing an entire shutdown and startup sequence. System power management may be invoked for a number of reasons. It may automatically enter a low-power state after it has been idle for some amount of time, after a user closes a lid on a laptop, or when some critical state has been reached. These are also policy decisions that are up to the user to configure and require some global mechanism for controlling. Device Power Management Device power management in the kernel is made possible by the new driver model in the 2.5 kernel. In fact, the driver model was inspired by the requirement to implement decent power management in the kernel. The new driver model allows generic kernel to communicate with every device in the system, regardless of the bus the device resides on, or the class it belongs to. The driver model also provides a hierarchical representation of the devices in the system. This is key to power management, since the kernel cannot power down a device that another device, that isn't powered down, relies on for power. For example, the system cannot power down a parent device whose children are still powered up and depend on their parent for power. In its simplest form, device power management consists of a description of the state a device is in, and a mechanism for controlling those states. Device power states are described as 'D' states, and consist of states D0-D3, inclusive. This device state representation is inspired by the PCI device specification and the ACPI specification [ACPI]. Though not all device types define power states in this way, this representation can map on to all known device types. Each D state represents a tradeoff between the amount of power a device is consuming and how functional a device is. In a lower power state (represented by a higher digit following D), some amount of power to a device is lost. This means that some of the device's operating state is lost, and must be restored by its driver when returning to the D0 state. D0 represents the state when the device is fully powered on and ready for, or in, use. This state is implicitly supported by every device, since every device may be powered on at some point while the system is running. In this state, all units of a device are powered on, and no device state is lost. D3 represents the state when the device is off. This state is also implicitly supported by every device, since every device is implicitly powered off when the system is powered off. In this state, all device context is lost and must be restored before using the device again. This usually means the device must also be completely reinitialized. The PCI Power Management spec goes on to define D3hot as a D3 state that is entered via driver control and D3cold that is entered when the entire system is powered down. In D3hot, the device may not lose all operating power, requiring less restoration that must take place. This is however, device-dependent. The kernel does not distinguish between the two, though a driver theoretically could take extra steps to do so. D1 and D2 are intermediate power states that are optionally supported by a device. In each case, the device is not functional, but not entirely powered off. In order to bring the device back to an operating state, less work is required than reviving the device from D3. In D1, more power is consumed than in D2, but more device context is preserved. A device's power management information is stored in struct device_pm: struct device_pm { #ifdef CONFIG_PM dev_power_t power_state; u8 * saved_state; atomic_t depend; atomic_t disable; struct kobject kobj; #endif }; struct device contains a statically allocated device_pm object. The configuration dependency on CONFIG_PM guarantees the overhead for the structure is nil when power management support is not compiled in. The kernel defines the following power states in include/linux/pm.h: typedef enum { DEVICE_PM_ON, DEVICE_PM_INT1, DEVICE_PM_INT2, DEVICE_PM_OFF, DEVICE_PM_UNKNOWN, } dev_power_t; When a device is registered, it's initial power state is set to DEVICE_PM_UNKOWN. The device driver may query the device and initialize the known power state using void device_pm_init_power_state(struct device * dev, dev_power_t state); Controlling a Device's State A device's power state may be controlled by the suspend() and resume() methods in struct device_driver: int (*suspend) (struct device * dev, u32 state, u32 level); int (*resume) (struct device * dev, u32 level); These methods may be initialized by the low-level device driver, though they are typically initialized at registration time by the bus driver that the driver belongs to. The bus's functions should forward power management requests to the bus-specific driver, modifying the semantics where necessary. This model is used to provide the easiest route when converting to the new driver model. However, a device driver's explicit initialization of these methods will be honored. The same methods are called during individual device power management transitions and system power management transitions. There are two steps to suspending a device and two steps to resume it. In order to suspend a device, two separate calls are made to the suspend() method - one to save state, and another to power the device down. Conversely, one call is made to the resume() method to power the device up, and another to restore device state. These steps are encoded thusly: enum { SUSPEND_SAVE_STATE, SUSPEND_POWER_DOWN, }; enum { RESUME_POWER_ON, RESUME_RESTORE_STATE, }; and are passed as the 'level' parameter to each method. During the SUSPEND_SAVE_STATE call, the driver is expected to stop all device requests and save all relevant device context based on the state the device is entering. This call is made in process context, so the driver may sleep and allocate memory to save state. However during system suspend, backing swap devices may have already been powered down, so drivers should use GFP_ATOMIC when allocating memory. SUSPEND_POWER_DOWN is used only to physically power the device down. This call has some caveats, and drivers must be aware of them. Interrupts will be disabled when this device is called. However, during run-time device power management, interrupts will be re-enabled once the call returns. Some devices are known to cause problems once they are powered down and interrupts reenabled - e.g. flooding the system with interrupts. Drivers should be careful not to service power management requests for devices known to be buggy. During system power management, interrupts are disabled and remain disabled while powering down all devices in the system. The resume sequence is identical, though reversed, from the suspened sequence. The RESUME_POWER_ON stage is performed first, with interrupts disabled. The driver is expected to power the device on. Interrupts are then enabled and the RESUME_RESTORE_STATE is performed, and the driver is expected to restore device state and free memory that was previously allocated. A driver may use the struct device_pm::state field to store a pointer to device state when the device is powered down. n Power Dependencies Devices that are children of other devices (e.g. devices behind a PCI bridge) depend on their parent devices to be powered up to either provide power to them and/or provide I/O transactions. The system must respect the power dependencies of devices and must not attempt to power down a device which another device depends on being on. Put another way, all children devices must be powered down before their parent can be powered down. Conversely, the parent device must be powered up before any children devices may be accessed. Expressing this type of dependency is simple, since it is easy to determine whether or not a device has any children or not. But, there are more interesting power dependencies that are more difficult to express. On a PCI Hotplug system, the hotplug controller that controls power to a range of slots may reside on the primary PCI bus. However, the slots it controls may reside behind a PCI-PCI bridge that is a peer of the hotplug controller. The devices in the slots depend on the hotplug controller being on to operate, but it is not the devices' parent. There are similar transversal relationships on some embedded platforms in which some I/O controller resides near the system root that some PCI devices, several layers deep, may depend on to communicate properly. Both types of power dependencies are represented using the struct device_pm::depend field. Implicit dependencies, like parent-child relationships, are handled by the depend count being incremented when a child is registered with the PM core. When that child device is powered down or removed, its parent's depend count is decremented. Only when a device's depend count is 0 may it be powered down. Explicit power dependencies can be imposed on devices using int device_pm_get(struct device *); void device_pm_put(struct device *); device_pm_get() will increment a device's dependency count, and device_pm_put() will decrement it. It is up to the driver to properly manage the dependency counts on device discovery, removal, and power management requests. Disabling Power Management There are circumstances in which a driver must refuse a power management request. This is usually because the driver author does not know the proper reinitialization sequence, or because the user is performing an uninterruptible operation like burning a CD. It is valid for a driver to return an error from a suspend() method call. Although, a driver may know a priori that it can't handle the request. This works to the system's benefit, since the PM core can check if any devices have disabled power management before starting a suspend transition. To disable power management, a device may call int device_pm_disable(struct device *); void device_pm_enable(struct device *); The former increments the struct device_pm::disable count, and the lattr decrements it. If the count is positive, system power management will be disabled completely, and device power management on that device. This calls should be used judiciously, since they have a global impact on system power management. System Power Management System power management is the concept of putting an entire computer into a state in which it is consuming a relatively small amount of power while maintaining a relatively low response latency. In a system power management (SPM) state, the system is not running and no processes are being executed. Typically, all of the devices in the system are also in a low-power state that corresponds to the tradeoff between power connsumption and response latency of the entire system, which will be explained shortly. SPM details are dependent on the under-laying platform. The amount of power the system consumes, the response latency, and even the canonical name for the states a system can enter are dependent on the architecture; sometimes even the generation of the architecture. The new power management subsystem defines an abstract interface to control SPM states. It provides an interface via sysfs from which a user can trigger SPM transitions. Internally, the PM infrastructure performs platform-agnostic actions to quiesce the system, then calls down to a dynamically registered PM driver, which performs the platform-specific steps to transition the hardware into a low- power state. Power Transitions PM States The power states that a platform can enter are defined by the underlying hardware and the firmware that runs on the hardware. Though the name of each state, and the mechanism for entering each state is different across each platform, most platforms support three nearly identical states. To the PM subsystem, a power state is an abstract object, defined by struct pm_state. struct pm_state includes both the power state the system is to enter as well as the lowest power state every device in the system is to enter. PM Drivers The PM subsystem defines a simple object registration model that platform-specific code can use to register objects that can communicate with the hardware. Other Power Management Options So far, a lot of talk has been dedicated to describing the internals of the new power management subsystem, but little has been given to describe how the new infrastructure interacts with current power management options. This section describes those relationships, and although it focuses on options specific to ia32 platforms, the relationships should be extendable to other platforms. ACPI In terms of system power management, fits nicely into the new PM infrastructure. It behaves as a PM driver, and provides platform-specific hooks to transition the system into a low-power state. At the basic SPM level, this is all that is required, though ACPI offers a potentially much more powerful solution, since it it exposes intimate knowledge of the platform power requirements than has ever been available on ia32 platforms (e.g. response latencies, power consumption etc.). Exploiting this knowledge is up to the ACPI platform driver to expose these attributes via sysfs. ACPI offers similar potential for device power management. Devices that appear in the firmware's DSDT (Differentiated System Description Table) may expose a very fine-grained level of detail about the devices' power requirements and capabilities. ACPI also stress the capabilities of device Performance States. A performance state is a power state that describes a trade-off between the capabilities of a device against the power consumption of the device. In each performance state that a device supports, the device is fully running, but different functional hardware units may be powered off to conserve power. The driver model does not explicitly recognize performance states, though the new PM extensions to the driver model provide a framework that could easily be extended to recognize performance states. APM APM power management does not appear on very many new systems, but the current Linux install base includes a large number of APM-capable computers. The new PM model was not developed with APM, or any firmware-driven PM model, in mind. However, care was taken to ensure that it conceptually made sense to use such mechanisms as low-level platform drivers for the PM model. No work has been done, however, to convert APM to act as a PM driver for the new model. pm_* infratructure The original PM infrastucture was developed by Andrew Henroid and was very ground-breaking, since nothing like it had been done for the Linux kernel before. It exists in its entirety in: kernel/pm.c include/linux/pm.h The general idea is that drivers can declare and register an object with the pm infrastructure that is accessed during a power state transition. The idea is very similar to what we have now, though the registration now is implicit when a device is registered with the system. And, based on the implementation, we can guarantee that each device is notified in proper ancestral order, which the old model cannot do. Because the new model is far superior the old-style pm infrastructure, it is declared deprecated. All drivers that implement pm callbacks should be converted to use the hooks provided by the new driver model. swsusp swsusp is a mechanism for doing suspend-to-disk by saving kernel state to unused swap space. It was also a ground-breaking feature, as it was the first true suspend-to-disk implementation for Linux. There are some questionable characteristics of swsusp that many people have that the maintainers of swsusp counter are frivilous concerns, and it currently exists as an alternative to the new PM model. However, the author has revoked any philosophical issues with swsusp. It can be, and should be ported to be, used as a backend driver for the generic Hibernate mechanism. The current code base could be reduced to a fraction of its current complexity. Acknowledgements Many people have contributed to this document, both explicitly and implicitly. First, Linus deserves a mention for encouraging me look at implementing ACPI suspend-to-ram as my first kernel project. Andy Grover and Paul Diefenbaugh of Intel for many things - contributing ACPI to the kernel, for talking with me, for always arguing with and motivating me internally to do things better, and for pushing me over the edge to write the finest OS driver model in existence. Andy Henroid for writing the first open-source power management model and providing a great base -- despite its shortcomings -- to learn and build from. Pavel Machek for constantlyl providing code and being energetic about the project. All the swsusp people for doing it in the first place and keeping it up, no matter how much I gripe about it. |
