[1]李佩娴,曹大千,戴鹏飞,等.基于红外热图与深度学习的建筑室内人脸属性分类研究[J].西安建筑科技大学学报(自然科学版),2022,54(03):441-449.[doi:10.15986/j.1006-7930.2022.03.015]
　LI Peixian,CAO Daqian,DAI Pengfei,et al.Deep learning-based facial attribute classification from indoor thermal images[J].J. Xi'an Univ. of Arch. & Tech.(Natural Science Edition),2022,54(03):441-449.[doi:10.15986/j.1006-7930.2022.03.015]
点击复制

基于红外热图与深度学习的建筑室内人脸属性分类研究()
分享到：

西安建筑科技大学学报（自然科学版）[ISSN:1006-7930/CN:61-1295/TU]

卷:
54
期数:
2022年03期
页码:
441-449
栏目:
出版日期:
2022-06-28

文章信息/Info

Title:
Deep learning-based facial attribute classification from indoor thermal images
文章编号:
1006-7930(2022)03-0441-09
作者:
李佩娴; 曹大千; 戴鹏飞; 卢昱杰; 刘 博
(同济大学 土木工程学院,上海 200092)
Author(s):
LI Peixian;  CAO Daqian;  DAI Pengfei;  LU Yujie;  LIU Bo
(College of Civil Engineering, Tongji University, Shanghai 200092, China)
关键词:
热舒适;  计算机视觉;  红外热成像;  年龄识别;  性别识别
Keywords:
thermal comfort;  computer vision;  thermography camera;  age classification;  gender recognition
分类号:
TU246.2
DOI:
10.15986/j.1006-7930.2022.03.015
文献标志码:
A
摘要:
利用热成像相机预测个体热舒适是无干扰温控的一种途径,有助于建筑节能.而人体热舒适范围在不同年龄与性别之间差异较大,现有文献尚缺乏红外热图中年龄与性别差异的研究.为探究利用深度学习从红外热图中自动识别性别与年龄的可行性,本文建立了红外热图和可见光的人脸数据集,对比了ResNet-50、DenseNet-121、DenseNet-201、Inception-V3四种卷积神经网络的效果,实验结果表明:男女红外热图差异明显,用Inception-V3可达到98.7%的识别准确率; 中青年红外热图差异较小,中老年红外热图差异明显,在分三类时,ResNet-50可获得80.0%的年龄识别准确率; 性别与年龄识别准确率均高于现有文献记载.同时,本文研究了红外滤镜和人脸裁剪对准确率的影响,提出了有助于提高识别精度的人脸红外热图数据采集与处理方法.
Abstract:
Thermal camera is a non-invasive method to predict individual thermal comfort which helps saving HVAC energy. While thermal comfort differs a lot between different groups of age and gender, existing literature lack research on the detection of age and gender using thermal cameras for accurate prediction of human thermal comfort in the built environment. To explore the feasibility of using deep learning to automatically recognize gender and age from thermal images, we establish a dataset of thermal images and visible-light images, study the impacts of algorithms(four convolutional neural networks: ResNet-50, DenseNet-121, DenseNet-201, and Inception-V3), thermal image filters, and image cropping on the recognition accuracy, and compare the recognition performances using thermal images and visible-light images. The results show that the gender classification accuracy can reach 98.7% using Inception-V3, meaning that there is a significant difference between male and female thermal images. The highest age classification accuracy(80.0%)is achieved using ResNet-50 when the dataset is divided into three classes—young, middle-aged, and old. It is noticed that there is little difference between young and middle-aged thermal images but a more obvious difference between the middle-aged and the old ones. The achieved accuracies are higher than or comparable to those in the literature. This study demonstrates that convolutional neural network is suitable for gender and age recognition from thermal images.

参考文献/References:

[1]GRAHAM L T, PARKINSON T, SCHIAVON S. Lessons learned from 20 years of CBE's occupant surveys[J]. Buildings and Cities, 2021, 2(1): 166-184.
[2]于竞宇, 於蓉, 张琦等. 基于机器学习的养老机构室内环境质量满意度评价模型[J]. 西安建筑科技大学学报(自然科学版), 2020, 52(4): 587-593,609.
YU Jingyu, YU Rong, ZHANG Qi, et al.Evaluation model of indoor environment quality satisfaction for nursing homes based on machine learning[J]. J. of Xi'an Univ. of Arch. & Tech.(Natural Science Edition), 2020, 52(4): 587-593,609.
[3]KIM J, DE DEAR R. Nonlinear relationships between individual IEQ factors and overall workspace satisfaction[J]. Building and Environment, 2012, 49(1): 33-40.
[4]World Green Building Council. Health, wellbeing & productivity in offices: The next chapter for green building[J].The Architrcts' Journal,2014(11):48.
[5]朱颖心. 如何营造健康舒适的建筑热环境——建筑环境与人体舒适及健康关系的探索[J]. 世界建筑, 2021, 3: 42-46.
ZHU Yingxin. How to create a healthy and comfortable indoor thermal environment: Exploration on the relationship between the built environment and human comfort and health[J]. World Architecture, 2021, 3: 42-46.
[6]徐畅, 李念平, 伍志斌, 等. 夏季不同突变热环境下人员热舒适性实验研究[J]. 科学技术与工程, 2020, 20(29): 12097-12103.
XU Chang, LI Nianping, WU Zhibin, et al. Experimental study on thermal comfort in different transient thermal environments in Summer[J]. Science Technology and Engineering, 2020, 20(29): 12097-12103.
[7]兰丽, 连之伟. 改善睡眠热环境可提高睡眠质量[J]. 科学通报, 2020, 65(7): 533-534.
LAN Li, LIAN Zhiwei. Better sleeping thermal environment, better sleep quality[J]. Chinese Science Bulletin, 2020, 65(7): 533-534.
[8]FANG L, WYON D P, CLAUSEN G, et al. Impact of indoor air temperature and humidity in an office on perceived air quality, SBS symptoms and performance[J]. Indoor Air, Supplement, 2004, 14(S): 74-81.
[9]LAN L, WARGOCKI P, WYON D P等. Effects of thermal discomfort in an office on perceived air quality, SBS symptoms, physiological responses, and human performance[J]. Indoor Air, 2011, 21: 376-390.
[10]周翔, 许玲, 谢建彤, 等. 上海地区某高校办公室人员位移及空调器使用行为研究[J]. 建筑科学, 2020, 36(12): 1-7,73.
ZHOU Xiang, XU Ling, XIE Jiantong, et al. Personnel movement and air conditioner usage behavior for a university office in Shanghai[J]. Building Science, 2020, 36(12): 1-7,73.
[11]杨柳, 杨雯, 郑武幸, 等. 风扇对亚热带气候区民居室内热环境影响分析[J]. 西安建筑科技大学学报(自然科学版), 2016, 48(4): 544-550.
YANG Liu, YANG Wen, ZHENG Wuxing, et al. The impact of the fan on rural residential buildings indoor thermal environment in subtropical climate zone[J]. J. Xi'an Univ. of Arch. & Tech.(Natural Science Edition), 2016, 48(4): 544-550.
[12]林宇凡, 杨柳, 闫海燕, 等. 中国气候与人体热舒适气候适应研究[J]. 西安建筑科技大学学报(自然科学版), 2014, 46(2): 251-255,265.
LIN Yufan, YANG Liu, YAN Haiyan, et al. Study on climate adaptation to thermal comfort in China[J]. J. Xi'an Univ. of Arch. & Tech.(Natural Science Edition), 2014, 46(2): 251-255,265.
[13]KIM J, SCHIAVON S, BRAGER G. Personal comfort models-A new paradigm in thermal comfort for occupant-centric environmental control[J]. Building and Environment, 2018, 132: 114-124.
[14]XIE J, LI H, LI C, et al. Review on occupant-centric thermal comfort sensing, predicting, and controlling[J]. Energy and Buildings, 2020, 226: 110392.
[15]李潇婧, 刘一航, 刘朋举, 等. 计算机视觉视频图像处理在暖通空调控制信号采集领域的应用[J]. 暖通空调, 2021, 51(6): 1-12.
LI Xiaojing, LIU Yihang, LIU Pengju, et al. Application of computer vision/video image processing in collecting HVAC control signals[J]. Heating Ventilating & Air Conditioning, 2021, 51(6): 1-12.
[16]YANG B, LI X, HOU Y, et al. Non-invasive(non-contact)measurements of human thermal physiology signals and thermal comfort/discomfort poses: A review[J]. Energy and Buildings, 2020, 224: 110261.
[17]CHENG X, YANG B, OLOFSSON T, et al. A pilot study of online non-invasive measuring technology based on video magnification to determine skin temperature[J]. Building and Environment, 2017, 121: 1-10.
[18]CHENG X, YANG B, HEDMAN A, et al. NIDL: A pilot study of contactless measurement of skin temperature for intelligent building[J]. Energy and Buildings, 2019, 198: 340-352.
[19]YANG B, CHENG X, DAI D, et al. Real-time and contactless measurements of thermal discomfort based on human poses for energy efficient control of buildings[J]. Building and Environment, 2019, 162(March): 106284.
[20]张文利, 郭向, 杨堃等. 面向室内环境控制的人员信息检测系统的设计与实现_张文利[J]. 北京工业大学学报, 2020, 46(5): 456-464.
ZHANG Wenli, GUO Xiang, YANG Kun, et al. Design and Implementation of a Personnel Information Detection System for Indoor Environment Control[J]. Journal of Beijing University of Technology, 2020, 46(5):456-464.
[21]卢知非,刘浩宇,陈文亮等. 红外人体测温精度补偿方法研究[J]. 红外技术, 2021, 43(9):895-901.
LU Zhifei, LIU Haoyu, CHEN Wenliang, et al. Accuracy compensation method for infrared human body temperature measurement accuracy[J]. Infrared Technology, 2021, 43(9):895-901.
[22]RANJAN J, SCOTT J. Thermal sense: determining dynamic thermal comfort preferences using thermographic imaging[C]//Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing.Germany: Heidelberg,2016.
[23]BURZO M, ABOUELENIEN M, ALSTINE D Van, et al. Thermal discomfort detection using thermal imaging[C]//ASME 2017 International Mechanical Engineering Congress and Exposition.USA, Flordia:ASME.2017.
[24]METZMACHER H, WÖLKI D, SCHMIDT C, et al. Real-time human skin temperature analysis using thermal image recognition for thermal comfort assessment[J]. Energy and Buildings, 2018, 158: 1063-1078.
[25]PAVLIN B, PERNIGOTTO G, CAPPELLETTI F, et al. Real-time monitoring of occupants' thermal comfort through infrared imaging: A preliminary study[J]. Buildings, 2017, 7(10): 1-11.
[26]陈庆财, 鹿伟, 张威, 等. 基于人工智能技术预测热感觉的室内热环境控制[J]. 建筑技术, 2019, 50(2): 253-255.
CHEN Qingcai, LU Wei, ZHANG Wei, et al. Indoor thermal environment control base on thermal sensation predicted by artificial intelligence[J]. Architecture Technology, 2019, 50(2): 253-255.
[27]WANG Z, DE DEAR R, LUO M, et al. Individual difference in thermal comfort: A literature review[J]. Building and Environment, 2018, 138: 181-193.
[28]WANG Z, YU H, LUO M, et al. Predicting older people's thermal sensation in building environment through a machine learning approach: Modelling, interpretation, and application[J]. Building and Environment, 2019, 161: 106231.
[29]CHEN C, ROSS A. Evaluation of gender classification methods on thermal and near-infrared face images[C]//2011 International Joint Conference on Biometrics(IJCB).USA, Washington,DC:[s.n.]2011.
[30]WANG S, GAO Z, HE S, et al. Gender recognition from visible and thermal infrared facial images[J]. Multimedia Tools and Applications, 2016, 75: 8419-8442.
[31]NGUYEN D T, KIM K W, HONG H G, et al. Gender recognition from human-body images using visible-light and thermal camera videos based on a convolutional neural network for image feature extraction[J]. Sensors(Switzerland), 2017, 17(3):637-658.
[32]张军挺. 人脸检测及人脸年龄与性别识别方法[D]. 合肥:中国科学技术大学, 2017.
ZHANG Junting. The method of face detection and face age and gender recognization[D]. Heifei:University of Science and Technology of China, 2017.
[33]李超. 基于深度学习的人脸性别识别与年龄段估计的研究与实现[D]. 昆明:云南大学, 2019.
LI Chao. Research and implementation of gender recognition and age estimation by face based on deep learning[D]. Kunming: Yunnan University, 2019.
[34]HUYNH H T, NGUYEN H. Joint age estimation and gender classification of Asian faces using wide resNet[J]. SN Computer Science, 2020, 1: 284.
[35]刘玉妹. 基于人脸图像的性别分类[D]. 石家庄:河北师范大学, 2019.
LIU Yumei. Gender classification based on face images[D]. Shijiazhuang: Hebei Normal University, 2019.
[36]张珂. 基于卷积神经网络的人脸检测和人脸属性识别研究[D]. 济南:山东大学, 2019.
ZHANG Ke. Study on face detection and face attribute recognition based on convolutional neural network[D]. Jinan: Shandong University, 2019.
[37]魏操. 基于卷积神经网络的图像分类算法研究[D]. 成都:成都理工大学, 2019.
WEI Cao. Research of image classification algorithms based on convolutional neural network[D]. Chengdu: Chengdu University of Technology, 2019.
[38]PAN J, YANG Q. A Survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.
[39]MEMIS S, ARSLAN B, BATUR O Z, et al. A comparative study of deep learning methods on food classification problem[C]//2020 Innovations in Intelligent Systems and Applications Conference(ASYU).[s.n.],2020.
[40]HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).USA, Las Vegas:IEEE,2016.
[41]HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017,USA,Hawaii:IEEE,2017.
[42]SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the Inception Architecture for Computer Vision[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.USA,Washington,DC:IEEE,2002.

相似文献/References:

[1]杨 柳,周书兵,闫海燕,等.包头住宅建筑夏季室内热舒适实地调查与分析[J].西安建筑科技大学学报(自然科学版),2012,44(03):369.[doi:10.15986/j.1006-7930.2012.03.011]
　YANG Liu,ZHOU Shu-bing,YAN Hai-yan,et al.Field research of indoor thermal comfort in summer for residential buildings in Baotou[J].J. Xi'an Univ. of Arch. & Tech.(Natural Science Edition),2012,44(03):369.[doi:10.15986/j.1006-7930.2012.03.011]
[2]王登甲,王晗旭,刘艳峰,等.陕西关中乡域中小学教室冬季热舒适调查研究[J].西安建筑科技大学学报(自然科学版),2016,48(02):277.[doi:10.15986/j.1006-7930.2016.02. 023]
　WANG Dengjia,WANG Hanxu,LIU Yanfeng,et al. Field study on thermal comfort of township and villages primary and secondary classrooms in winter in Guanzhong region, Shaanxi province[J].J. Xi'an Univ. of Arch. & Tech.(Natural Science Edition),2016,48(03):277.[doi:10.15986/j.1006-7930.2016.02. 023]
[3]宋 冰,杨 柳,白鲁建.过渡季壁面辐射温度对人体热舒适的影响分析[J].西安建筑科技大学学报(自然科学版),2016,48(03):438.[doi:10.15986/j.1006-7930.2016.03.022]
　SONG Bing,YANG Liu,BAI Lujian.Study of influence of wall radiation temperature on human thermal comfort in transition season[J].J. Xi'an Univ. of Arch. & Tech.(Natural Science Edition),2016,48(03):438.[doi:10.15986/j.1006-7930.2016.03.022]
[4]李雪平,崔羽.关中地区下沉式窑洞冬季室内热环境测试研究[J].西安建筑科技大学学报(自然科学版),2019,51(04):591.[doi:10.15986/j.1006-7930.2019.04.018]
　LI Xueping,CUI Yu.Field study on indoor thermal environment of sinking cave in Guanzhong Area in winter[J].J. Xi'an Univ. of Arch. & Tech.(Natural Science Edition),2019,51(03):591.[doi:10.15986/j.1006-7930.2019.04.018]
[5]孙壬龙,金雨蒙,金虹.基于热舒适与可达性耦合视角的严寒城市街区冬季活力研究[J].西安建筑科技大学学报(自然科学版),2023,55(06):927.[doi:10.15986/j.1006-7930.2023.06.017]
　SUN Renlong,JIN Yumeng,JIN Hong.Research on winter vitality of severe cold city blocks based on the coupling perspective of thermal comfort and accessibility[J].J. Xi'an Univ. of Arch. & Tech.(Natural Science Edition),2023,55(03):927.[doi:10.15986/j.1006-7930.2023.06.017]

备注/Memo

备注/Memo:
收稿日期:2021-07-23修改稿日期:2022-06-02
基金项目:上海市浦江人才计划基金资助项目(2020PJD074); 国家自然科学基金面上项目(52078374); 国家自然科学基金青年科学基金项目(52108090)
第一作者:李佩娴(1992—),女,博士,主要研究方向:建筑环境与热舒适.E-mail:lipx@tongji.edu.cn.通信作者:卢昱杰(1985—),男,博士,教授,主要研究方向:计算机视觉与绿色建筑.E-mail:lu6@tongji.edu.
更新日期/Last Update: 2022-06-28