Introduction

Human Factors Engineering (HFE) is a vital subject of my undergraduate program—Industrial engineering. This project integrated modern AI and IoT technologies into HFE control theory, aiming at recognising and actively intervening users’ fatigue in office. The system consists of 4 layers.

• Perception Layer: while sensors in the office monitor microclimate data, cameras and wearable devices monitor users’ facial features and heart rate data in real time.
• Processing Layer: the office LAN gateway collects monitoring data from sensors and then sends it to the cloud server. Subsequently, the cloud server calculates users’ fatigue degree and sends back control commands.
• Device Layer: according to the control commands transmitted by the gateway, the office microclimate is regulated in terms of sound, light, electricity and temperature.
• Display Layer: real-time display of sensors, server status and microclimate control strategy.

This project won third prize in the 2th Sunon Smart Service Innovation Competition.

My Contribution

• I was one of the most important contributors of the ideals in this project
• I designed the control strategy
• The fatigue recognizing module (facial feature part) and the GUI software were developed by me alone.

The following video shows a demo of my work.

*Unfortunately, Chinese is still the only supported language of the UI.

In this video, a volunteer was invited to test this module. Control panel and visualized information can be seen on the right. This software captures users’ eye and mouth contour lines and quantifies them as eye aspect ratio (EAR) and mouth aspect ratio (MAR), respectively.

$$EAR=\frac{|p37\ -\ p41|\ +\ |p38\ -\ p40|}{2|p36\ -\ p39|}$$ $$MAR=\frac{|p2\ -\ p6|\ +\ |p3\ -\ p5|}{2|p1\ -\ p4|}$$