Heating and cooling are the largest consumer of energy in American homes and commercial buildings. There’s a need for smarter, more flexible climate control that keeps us comfortable without heating and cooling entire empty buildings.

Researchers at the University of Michigan have developed a solution that could do away with the wall-mounted thermostats we’re accustomed to. They describe Human Embodied Autonomous Thermostat, or “HEAT,” in a study in the journal Building and Environment.

The system pairs thermal cameras with three-dimensional video cameras to measure whether occupants are hot or cold by tracking their facial temperature. It then feeds the temperature data to a predictive model, which compares it with information about occupants’ thermal preferences.

Finally, the system determines the temperature that will keep the largest number of occupants comfortable with minimum energy expenditure. The new study shows how the system can effectively and efficiently maintain the comfort of 10 occupants in a lab setting.

“COVID presents a variety of new climate control challenges, as buildings are occupied less consistently and people struggle to stay comfortable while wearing masks and other protective gear,” says project principal investigator and study coauthor Carol Menassa, associate professor of civil and environmental engineering.

“HEAT could provide an unobtrusive way to maximize comfort while using less energy. The key innovation here is that we’re able to measure comfort without requiring users to wear any detection devices and without the need for a separate camera for each occupant.”

HEAT works a bit like today’s internet-enabled learning thermostats. When it’s newly installed, occupants teach the system about their preferences by periodically giving it feedback from their smartphones on a three-point scale: “too hot,” “too cold,” or “comfortable.” After a few days, HEAT learns their preferences and operates independently.

The research team is working with power company Southern Power to begin testing HEAT in its Alabama offices, where test cameras will be mounted on tripods in the corners of rooms. Menassa explains that cameras would be less obtrusive in a permanent installation. The cameras collect temperature data without identifying individuals, and all footage deletes immediately after processing, usually within a few seconds.

A second test, also with Southern Power, will place the system in an Alabama community of newly constructed smart homes. The team estimates that they could have a residential system on the market within the next five years.

Facial temperature is a good predictor of comfort, Menassa says. When we’re too hot, the blood vessels expand to radiate additional heat, raising facial temperature; when we’re too cold, they constrict, cooling the face. While earlier iterations of the system also used body temperature to predict comfort, they required users to wear wristbands that measured body temperature directly, and to provide frequent feedback about their comfort level.

“The cameras we’re using are common and inexpensive, and the model works very well in a residential context,” says study coauthor Vineet Kamat, professor of civil and environmental engineering, and electrical engineering and computer science. “Internet-enabled thermostats that detect you and learn from you have sort of built a platform for the next phase, where there’s no visible thermostat at all.”

Coauthor Eunshin Byon, an associate professor of industrial operations and engineering, built HEAT’s predictive model. She believes that tweaks to the model could make the system useful in applications beyond homes and offices—in hospitals, for example, where care providers struggle to stay comfortable under masks and other protective equipment.

“The COVID-19 pandemic requires nurses and other hospital workers to wear a lot of protective gear, and they’ve struggled to stay comfortable in the fast-faced hospital environment,” Byon says. “The HEAT system could be adapted to help them stay comfortable by adjusting room temperature or even by signaling to them when they need to take a break.”

In partnership with the school of nursing, Menassa’s research group has already conducted a pilot study that explored how the system can provide personalized thermal comfort for nurses working in healthcare environments such as chemotherapy administration units.