Publication: Integration of human detection model and computer vision room segmentation method with lighting control system
Loading...
Date
2023-07
Authors
Khor, Huai Sheng
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
To develop a computer vision-based lighting control system in indoor environment, there is two major components to this project. The first component is the
computer vision component, one of the objectives of this project is to use the minimum amount of camera needed in one room and the detection of human to be fast enough to be run in real time since it is part of the light automation process. To satisfy both of this requirement has proved to be a very difficult task since most of the computer vision model has trade off being slow to have a higher accuracy or sacrificing speed for more precise prediction. To overcome this limitation, the first step is to find a starting point, the model chosen to be a starting point will both be CNN based architecture which is YoloV7 model, and an SSD based model which is PeopleNet which is a pre-trained model provided by NVIDIA TAO toolkit exclusively for detecting human. First is to obtain a benchmark performance by testing the model using a public dataset. Then, the model will be tested using manually annotated dataset that is simulating the environment of a manufacturing floor. The model will be tested in various environment that would reflect the manufacturing floor. After testing the performance of both YoloV7 and PeopleNet, a better performing one will be chosen to use as a starting point for transfer learning starting point. The second component for this project is the integration part, the final goal of this project is to develop a lighting control system based on the segment of the room, therefore knowing the position of the human detected is essential, the main limitation for the segmentation process is the camera angle which does not allow a frame in the video to be segmented regularly and the human detection model generally struggle to detect human lower body part, to overcome that the room will be segmented into overlapping region of interest and the ROI is calculated using nvdsanalytics plugin. Then, the hardware integration will be done by using the Jetson Xavier NX GPIO pin to send signal to LED that represent the light in manufacturing floor which is simulate sending a signal to the relay that control the switch of the lighting to turn on or off the light based on which region has person
detected.