Aug 5, 2021

How accurate is the self-reported energy consumption of connected devices?

By Michael Poplawski and Anay Waghale

Connected devices—such as lights, sensors, power strips and other equipment in buildings—can bring greater functionality and services, which often entails operating modes beyond simple “on” and “off.” Device energy consumption can vary depending on how much data they are producing or analyzing, and devices may operate in low-power or stand-by modes when they are not active. This makes it more difficult to estimate energy consumption, because the energy performance of connected devices depends on what operating modes they use and how much time they spend in each mode. The uncertainty about energy performance can be largely mitigated if connected systems can accurately report their own energy consumption, and many emerging connected lighting systems already offer some form of this capability.

But how accurate is this energy self-reporting? To find out, Pacific Northwest National Laboratory (PNNL), on behalf of the U.S. Department of Energy (DOE), conducted a study that explored the energy-reporting accuracy of market-available connected devices, in this case connected electrical outlets. The study, conducted at PNNL’s Connected Lighting Test Bed, was intended to generate awareness of building systems that are capable of reporting their own energy consumption, and to further interest in the value of energy data for a variety of uses. The findings draw attention to how the accuracy of reported metrics can be characterized and quantify the performance variation found in market-available products.

THE STUDY CONSIDERED two residential-market products and three commercial-market products with the ability to report the power drawn and/or energy consumed by devices plugged in to their connected electrical outlets. Energy-consumption data reported by the devices were compared to measurements taken by a reference meter under 10 test conditions. The residential products reported power draw but not cumulative energy consumption; the commercial products reported both power draw and cumulative energy consumption.

Analysis of the results revealed variations across test conditions and units, as summarized in Figures 1A and B. The average relative reporting error (RRE) for the two residential products, derived from the reported power draw, was −0.02% and −1.20%. The average RRE for two of the three commercial products, derived from reported power draw, was greater than those of the residential products (-2.40%, -2.72%, -0.36%). The average commercial-product RRE derived from reported energy consumption should be very consistent and better than performance based on reported power draw. However, the average RREs derived from reported energy consumption varied significantly (17%, -10.78%, -1.5%) across the three makes of commercial-market products and were uniformly less accurate than performance based on reported power draw. Subsequent analysis identified a number of root causes for this decrease in performance, most of which were related to reporting resolution.

Figure 1A
Figure 1A
Figure 1B
Figure 1B

The power draw required to provide the energy-reporting functionality can vary significantly with the connected load. While all makes and models showed very little dependence on the device being tested, the power draw of the commercial units varied considerably across load conditions, as shown in Figure 2. The average power draws for the three commercial units (9.76 watts, 8.10 watts, 21.76 watts) were significantly higher than those of the residential units (1.21 watts, 2.11 watts), as were their average no-load power draws (5.13 watts, 5.72 watts and 20.50 watts for the commercial units, vs. 1.31 watts and 1.95 watts for the residential units).

Figure 2
Figure 2

THE RESULTS OF THIS STUDY and subsequent related work may be relevant to industry specification and standards development organizations. The methods this study employs could inform test and measurement procedures and performance classifications for connected outlets, lighting products and other building systems capable of reporting their own energy consumption.

For example, there is a need for application-specific performance classifications that end users can understand and relate to their energy-data use needs (e.g., 2% accuracy class for utility streetlight energy billing needs, or 10% accuracy class for ESCO performance verification needs). Additional recommendations from the study include development of internal power-draw limits for energy-reporting devices and systems under well-defined operating conditions, and test methods to verify whether energy-reporting devices and systems comply with established internal power-draw limits and application-specific performance classifications.

Owners, operators and specifiers of energy reporting can support the development of relevant industry standards and specifications by analyzing the dependency of current and planned energy-data use cases on accuracy—noting in particular the degree of dependence on relative vs. absolute accuracy and on trueness vs. precision (i.e., repeatability)—and communicating use-case needs to industry-standards and specification organizations.

In the meantime, manufacturers developing products that report energy consumption should:

The accuracy of self-reported energy consumption in connected devices is an important topic, but it’s also a complex one. For additional details— including a normality analysis that attempted to separate systematic and random error sources—download the full report at www.energy.gov.

Contributor(s)

Michael Poplaswki

Michael Poplawski

Michael Poplawski is a senior engineer at PNNL, where he supports the U.S. Department of Energy Solid-State Lighting program, primarily in the areas of connected lighting system technology evaluation and demonstration, standards and specification development, and the estimation... More info »
Anay Waghale

Anay Waghale

Anay Waghale is a lighting research engineer at PNNL, where he contributes to several projects focused on performance of connected lighting systems. His current research efforts supported by the DOE Lighting R&D Program focus on estimation of lighting energy end-use consumption,... More info »