Mar 26, 2020

Updates from a Living Lab

By Ruth Taylor

The Next Generation Lighting Systems (NGLS) evaluations of indoor connected lighting systems, which we reported on two years ago (“Connected Lighting: Lessons from a Living Lab,” LD+A, May 2018), continue to yield new lessons and insights. Importantly, we continue to learn about system maintenance, performance and user experience.

Organized by the U.S. Department of Energy’s (DOE) Lighting R&D Program in partnership with the IES and the International Association of Lighting Designers, NGLS evaluates today’s connected lighting systems in real-world installations. Through this research, we look to identify challenges in installation and operation that hinder the adoption of energy-saving connected lighting systems. By doing so, we hope to reveal needed product improvements and articulate best practices that can reduce configuration complexity and enable system performance to meet expectations.

The fundamental idea of NGLS is simple: by considering human factors—that is, by observing and evaluating the interaction of people and lighting control systems—we can find real “pressure points” limiting market penetration. In this way, NGLS provides a valuable complement to conventional demonstrations, mock-ups and testing laboratories.

Click images for larger view

Two years in, NGLS has installed 14 connected lighting systems in classrooms at The New School, Parsons School of Design, in New York City, including two installed last summer. These systems are being continually evaluated in what amounts to a “living lab,” with the feedback from installers, evaluators and classroom users being widely shared.

We have several important findings to report since our last article. The two systems installed most recently use Bluetooth mesh technology for connectivity, whereas most of the previous installations used a Zigbee, or similar IEE 802.15.4- based, protocol. Like our first installations, these latest ones also encountered problems during installation and setup, but there were some encouraging developments in terms of simplifying the configuration process. We’ve also collected considerable feedback as facility staff and electrical teams have faced a variety of maintenance issues. Additionally, we’re conducting extensive performance testing on both occupancy detection and daylight harvesting.

As we reported previously, installation and maintenance “ain’t so simple.” While the installed systems are all marketed as easy to install, configure and operate, both installers and users have found a variety of challenges in getting the systems to work as expected. Although NGLS required contractors to install and configure the systems, relying only on manufacturers’ published instructions (print or visual), all but two of the systems ultimately required factory assistance during installation or following setup. Most often this assistance involved a phone call—sometimes just a simple reference to a video or a brief explanation, although at other times, calls were lengthy, or intensive site-based intervention was needed.

Maintenance issues have arisen in the Living Lab as replacements for faulty or missing components need to be integrated into already installed systems. Often a specific sequence is required to delete or add new equipment, and any deviation from the sequence can throw off the operation of the system. Firmware and software incompatibility can also be more than a nuisance. And since local distributors do not stock system components, repairs and upgrades have typically proven time-consuming.

Language continues to be a challenge in both initial installation and ongoing maintenance: designers, installers, manufacturers and those operating the system all use a wide range of terms to describe the same thing. Even the terminology of the various manufacturers differs enough to confuse many contractors who install a variety of systems. Consistent vocabulary; visual, app-based instructions; and readily available phone support will certainly help alleviate many of these problems.

Regarding the user experience, the diversity of wall controls continues to frustrate users who are unfamiliar with the configuration, logic and mechanics of the different devices. Simpler controls are learned with just a few repetitions
or a brief explanation. Wall controls with more capabilities—and complexity—often leave users unaware of what they can do and how to activate those functions. Anecdotal evidence suggests that once users master the on/off, they may not dig deeper.

In terms of performance evaluation, NGLS requires systems to:

All systems can dim manually, many to below 5% as measured at several typical locations. Some systems exhibit flicker, determined visually. Reconfiguring the zones (as might be required to maintain performance when spatial needs change) varies from easy to moderately difficult. Presence detection is proving more problematic, at least for some of the systems.

For most of the systems, basic vacancy functionality worked well, including a sensitivity test detecting minor hand motion. However, when using the same hand motion to assess the extent of sensor coverage, several of the systems turned off while the space was occupied. When manufacturers provide coverage patterns in their literature, that information can inform better sensor placement to avoid some of these problems, if it’s followed during layout and installation. Many people report that sensors end up disabled if users dislike their operation. We will monitor the persistence of the sensors as the lab continues.

The opposite problem—lights staying on in an empty space—also occurs in the lab, with a loss of energy savings. Here the problem may be a defective device or incorrect programming. Determining which is the culprit is a nagging maintenance issue— although once solved, systems typically perform as expected.

Assessing daylight harvesting quantitatively is a work in progress. The NGLS team is refining its measurement protocols to assess both whether systems dim automatically and how efficiently they do so. So stay tuned.

The NGLS indoor evaluations live on. Additional daylighting evaluations are on the agenda. We’ll also install new systems to evaluate some enhanced system capabilities, such as energy monitoring, control of multiple spaces and integration with different types of luminaires. And we’re planning a focused study of tunable white lighting control systems, in collaboration with DOE’s Commercial Buildings Integration Program.

For more information on the NGLS, visit


Ruth Taylor

Ruth Taylor

Ruth Taylor currently serves as a program manager on the Advanced Lighting Team at Pacific Northwest National Laboratory, where she manages the Next Generation Lighting Systems evaluations and contributes to several other projects focused on the application and development of... More info »