iPDUs: a critical step for next generation efficiency (2024)

Measuring the amount of power consumed by individual IT systems will be essential for advanced on-premises data center performance, efficiency and sustainability initiatives.

Historically this type of data was under the purview of IT teams; facilities staff were focused on optimizing power and cooling delivery to the server rack, so had little need to collect IT systems data. As a result, traditional facilities management tools such as building management systems (BMS) and data center infrastructure management (DCIM) software lack the capabilities to collect power consumption data directly from IT devices.

If facilities teams require this info, they have two options: rely on complex and costly software integration with existing IT management tools and systems; or deploy intelligent metered-by-outlet power distribution units (iPDUs). iPDUs are capable of collecting device-level data from enterprise server, storage and network equipment fleets that combine equipment from multiple vendors, across multiple generations of hardware.

This update focuses primarily on tracking server power consumption with iPDUs. Future updates will discuss two alternative options for collecting this data: integration with tools such as IT operations management (ITOM) software; and data collection through the baseboard management controller (BMC).

Why track device-level power consumption?

Device-level power consumption is one of several operational data points that enable organizations to identify strains on power subsystem capacity, servers that are periodically exceeding their power budget, and underutilized and zombie servers. In addition to providing accurate metrics for calculating server energy and carbon emissions footprints, this information can help capacity planning, and discovering and repurposing stranded data center space, power and cooling.

Despite these benefits, the Uptime Institute IT and Power Efficiency Survey 2023 shows that more than half (52%) of data center operators do not track average power demand for IT equipment (Figure 1). Only 19% of operators say their facilities are equipped with tools that can calculate average power demand at the individual server level, and only 13% actually use those tools (see Tools to watch and improve power use by IT are underused).

Figure 1. Most operators do not track server power demand

This cannot continue: increased public and regulatory scrutiny of data center resource consumption means more pressure on tracking device-level power demand. Future regulations — such as the expected update to the European Union’s Energy Efficiency Directive (EED) and copycat regulations in other jurisdictions — are likely to require organizations to collect average utilization and power demand data for individual servers and storage equipment. They will then need to match this data to their IT equipment inventories. This data will be key to calculating any potential work-per-energy metric, which is the ultimate yardstick of data center efficiency.

Although it is the IT teams that use the larger share of data center power, it is increasingly the facilities teams that are expected to take responsibility for tracking, compiling and reporting power consumption data for regulatory purposes. Forward-looking organizations will begin evaluating their options for data collection now, before the regulators force their hand.

Collecting server power data: the options

There are three common approaches to collecting server-level power consumption data.

Leveraging BMC capabilities

The least costly and disruptive approach is to leverage the capabilities of the BMC, a specialized processor that sits on virtually every server motherboard to provide remote monitoring and management capabilities.

The Redfish standard (a suite of specifications that describe a RESTful interface for the management of servers, storage, networking and converged infrastructure) enables system administrators to address the BMC to retrieve information on power consumption, thermal performance, and processor utilization. AMI DCM (formerly Intel DCM) is an example of a commercial product that derives much of its functionality from interactions with the BMC. Some vendors have developed their own versions of the BMC that offer advanced server management features, such as integrated Dell remote access controller (iDRAC) and integrated lights-out (iLO) by HPE.

Despite the advantages of this approach, data from the BMC is rarely used by data center operators since it requires a high level of familiarity with IT administration tools.

Software and IT management tool integration

This second approach involves integration between data center software, such as DCIM, and IT management tools. Some of these management tools can be provided by server vendors (e.g., HPE OneView), while some are vendor-agnostic (e.g., VMware vSphere). While this approach requires no changes on the facility operations side, the cost and complexity of integration can be high. Requirements include the installation of additional software ”agents” on all target systems, which might not suit some applications.

Power management at the outlet

The third approach is deploying metered-by-outlet iPDUs with networking capabilities that can report power consumption metrics via the manufacturer’s own lightweight software or forward this data to the operator’s management / analytics software of choice.

iPDUs are not suitable for (and not necessary in) some types of data centers: large colocation facilities typically employ PDUs that track the power to the customer’s rack, rather than an individual piece of equipment. Meanwhile, hyperscale cloud facilities are more concerned about tracking power consumed by virtual machines, and therefore rely on software rather than hardware. This leaves enterprise data centers as the largest users of iPDUs.

Branch circuit monitoring solutions (typically installed at the power distribution panel) are not suitable for collecting server-level power consumption data. The same applies to ”smart” power cables that use wireless protocols to transmit power metering data to an Ethernet-connected gateway, that then forwards it to a management system. Table 1 shows the key differences between the three approaches to collecting device-level power consumption data.

Table 1. Three common approaches to collecting server power data

What makes an iPDU?

iPDUs are referred to as intelligent PDUs, smart PDUs and monitored PDUs, but they all have the same features (although the terminology can vary by vendor):

• They continuously monitor current, voltage, real power, apparent power, power factor and kilowatt-hours (kWh) at rack level and / or outlet level (by phase).
• They are equipped with circuit breakers at currents above 20 A to protect the IT equipment.
• They can be managed either over the local network or remotely.
• They provide alarm capabilities to alert users of breaches in user-defined power thresholds.
• They can report IT equipment power consumption data to a management system.

Rack PDUs started out as little more than simple power strips, but have evolved into complex operational technology devices with a wide array of capabilities — iPDUs represent the apex of this evolution.

What makes these PDUs “intelligent” is a network management card (NMC): a small computer responsible for collecting, storing, analyzing and forwarding power consumption data. Typical power measurement accuracy for high-end iPDUs ranges from +/- 0.5% to -/+1%.

Virtually all NMCs designed for data center use have built-in displays. The NMC modules themselves are replaceable and can be hot-swapped without interrupting power delivery to the IT equipment. This also means that the iPDU will continue providing power if the NMC suffers a failure or needs to be taken out of operation for a firmware upgrade.

Every NMC is equipped with at least one Ethernet port that connects the iPDU to the IT network. This supports remote access and enables the devices to provide alerts and notifications using protocols, such as email, SMS or simple network management protocol (SNMP) traps. If present, the second Ethernet port can be used for fault-tolerant daisy-chaining. This means iPDUs can maintain connectivity even if there is an issue with one of the network switches or cables.

Switched PDUs are similar to iPDUs, but have the added ability to remotely turn power on and off at each outlet. This makes them especially useful in data centers that need to limit power usage to avoid overloads, and in remote data centers where they can be used to power cycle unresponsive IT equipment. The ability to turn off servers remotely is sometimes perceived as a cyber security risk and some organizations avoid switched PDUs for this reason.

iPDUs are typically accompanied by the manufacturer’s management software, which is presented through a simple web interface. Integration with third-party software, including DCIM, is achieved with the use of management information base (MIB) files. These MIB files are databases that describe the properties of a networked device, and how to interpret its messages.

MIB files are also used to deploy routers, switches, printers, and tape libraries, and can typically be downloaded from the device manufacturer’s website. Despite the relative simplicity of the process, some DCIM vendors charge a fee when integrating iPDUs.

Since iPDUs are connected to the IT network, they enjoy the security features that are used to protect IT systems: they support user authentication, password policies and the use of encryption.

Customize to fit requirements

Over the past five years, the design of iPDUs has been standardized to the point that any unit from a major manufacturer will look similar and deliver the same core capabilities. Any differences between product ranges will mostly come down to optional or custom features.

A typical iPDU lifecycle lasts five to seven years —even longer if these devices are refurbished. Therefore, it is important to choose iPDUs depending on the optional features they provide, dependent on the future needs of the data center. Optional features include (but are not limited to):

• Color-coded alternating outlets. These minimize errors in circuit / phase balancing and cable management.
• Combination outlets. Examples include 2-in-1 (C13 / C19) and 4-in-1 (C13 / C15 / C19 / C21).
• Outlet-locking mechanisms. These mechanisms secure the physical connection between the IT equipment and the PDU to ensure power cords are not accidentally pulled out of the outlet. Uptime Institute consultants note that loosely fitting plugs have been observed to disconnect during routine site maintenance, leading to incidents.
• Internet protocol (IP) aggregation. IP addresses and switch ports are expensive, and operators can reduce the cost of deploying iPDUs by utilizing units with IP aggregation capabilities. The number of units that can be aggregated on a single IP address depends on the manufacturer and will vary from two to 50 devices. Features included with IP aggregation include self-configuration of downstream devices.
• Environmental monitoring. Some iPDUs incorporate sockets for plug-in environmental sensors that eliminate the need to deploy a separate monitoring solution.
• Residual current metering type B (RCM-B). This functionality continuously monitors the fault current of the power distribution within the IT rack. Alarms provide immediate notifications of changes in residual current leakage allowing operators to respond to a fault condition immediately to avoid equipment shutdown, electric shock or electrical fires.
• USB connectivity. A USB port is used to upload firmware, backup or restore device configuration, expand logging capacity via USB storage and support the addition of security cameras or wireless USB adapters.
• Out-of-band communication. If the primary network to the iPDU goes down, some models provide redundant communications through out-of-band management devices, such as serial consoles or KVM (keyboard, video, mouse) switches. Some models feature visible light communication that enables wireless data transmission to mobile devices.

Consider cost, lead time and manufacturer

An iPDU is at least twice as expensive as a comparable PDU without power metering features, with prices starting at around $2,000 for a vertical unit with 42 outlets.

While iPDUs are considered to be expensive, their cost is negligible when considering the overall cost of the data center, and the ongoing cost of powering IT equipment. When used proactively, iPDUs provide a return on investment quickly through the savings that arise from identifying underloaded or overloaded circuits. They also provide data to support workload consolidation initiatives that involve migration of workloads to fewer servers and the elimination of ”zombie” machines.

Historically, major PDU vendors have stocked devices to be available for immediate shipping. However, the 2021 to 2023 supply chain crisis impacted parts of the market, placing PDUs and their components among the top three types of equipment affected by delays, along with computer room air conditioning units and UPS systems. Today, lead times for iPDUs quoted by major manufacturers range from two to nine weeks, but these should be verified with specific organizations.

Major operational technology vendors that offer global shipping on a full range of PDUs, including metered-by-outlet models, include (but are not limited to): Aten (Taiwan), Chatsworth Products (US), Eaton (US), Legrand (France), nVent (US), Panduit (US), Rittal (Germany), Schneider Electric (France) and Vertiv (US).

Shifting the focus to IT

It is becoming harder to improve the efficiency of data center facilities, exemplified by industry average PUE indicators which have remained level since 2018 (see Global PUEs — are they going anywhere?). The current generation of air-cooling equipment is reaching the limits of what is physically and practically possible – hence, the excitement about liquid cooling – and enhancing UPS power conversion efficiency would require a major technological breakthrough.

At the same time, average server utilization rates across the industry remain low: estimates vary, but most put the rate between 10% and 20%. The amount of stranded space, power and cooling in enterprise data centers is even harder to estimate, but is likely to be embarrassingly high.

Future data center efficiency gains will not stem from improvements to facilities, but from more prudent approaches to running IT — and tracking individual server power consumption is a critical step on this journey.