|本期目录/Table of Contents|

[1]郝腾腾,郑欣,王慧宇,等.基于多模态信息融合的心理负荷评估*[J].中国安全生产科学技术,2022,18(12):12-18.[doi:10.11731/j.issn.1673-193x.2022.12.002]
　HAO Tengteng,ZHENG Xin,WANG Huiyu,et al.Mental load assessment based on multi-modality information fusion[J].JOURNAL OF SAFETY SCIENCE AND TECHNOLOGY,2022,18(12):12-18.[doi:10.11731/j.issn.1673-193x.2022.12.002]
点击复制

基于多模态信息融合的心理负荷评估*
分享到：

《中国安全生产科学技术》[ISSN:1673-193X/CN:11-5335/TB]

卷:
18
期数:
2022年12期
页码:
12-18
栏目:
学术论著
出版日期:
2022-12-31

文章信息/Info

Title:
Mental load assessment based on multi-modality information fusion
文章编号:
1673-193X(2022)-12-0012-07
作者:
郝腾腾; 郑欣; 王慧宇; 许开立; 朱奕嬴
（东北大学 资源与土木工程学院，辽宁 沈阳 110819）
Author(s):
HAO Tengteng;  ZHENG Xin;  WANG Huiyu;  XU Kaili;  ZHU Yiying
(School of Resources and Civil Engineering,Northeastern University,Shenyang Liaoning 110819,China)
关键词:
心理负荷; 眼动; 心率变异性（HRV）; 脑电信号（EEG）; 多模态信息融合; 支持向量机（SVM）; 随机森林（RF）
Keywords:
mental load;  eye movement;  heart rate variability (HRV);  electro encephalo gram(EEG);  multi-modality information fusion;  support vector machine (SVM);  random forest (RF)
分类号:
X910
DOI:
10.11731/j.issn.1673-193x.2022.12.002
文献标志码:
A
摘要:
为了相对稳定地识别人员是否处于心理负荷状态，设计并实施含能材料起爆作业心理负荷诱导试验。首先对心理负荷诱导情况进行判断，然后分别采集27名被试在静息状态和实施含能材料撞击起爆作业状态下的眼动、心率变异性（HRV）和脑电信号（EEG），通过正态性检验和假设检验获得心理负荷表征指标并进行统计功效分析，依据表征指标，采用支持向量机（SVM）和随机森林（RF）算法建立多模态信息融合的心理负荷评估模型，最后采用被试工作特征曲线（ROC曲线）分析各模态组合和分类器的心理负荷识别性能。研究结果表明：双模态（眼动+EEG）下SVM算法和3模态下RF算法评估性能和稳健性较高，多模态信息组合具有优异的识别效果。
Abstract:
In order to relatively stably identify whether a person was in a state of mental load,the blasting operation of energetic materials was designed and implemented as the mental load induction test.Firstly,the induction situation of mental load was judged.Then,the eye movement,heart rate variability (HRV) and electroencephalo-gram (EEG) of 27 subjects were collected respectively in the resting state and the state of implementing the impact detonation operation of energetic materials.Thenormality test and hypothesis test were used to obtain the mental load characterization indexes,and the statistical efficacy analysis was conducted.According to the characterization indexes,the support vector machine (SVM) and random forest (RF) algorithms were used to establish the mental load assessment model with multi-modality information fusion.Finally,the receiver operating characteristic (ROC) curve was used to analyze the recognition performance ofmental load of each modal combination and classifier.The results showed that the SVM algorithm under dual-modality (eye movement + EEG) and the RF algorithm under three-modality had better assessmentperformance and robustness,and the combination of multi-modality information had an excellent recognition effect.

参考文献/References:

［1］王春雪.建筑施工塔式起重机司机作业心理负荷研究［J］.中国安全生产科学技术,2016,12(S1):78-83. WANG Chunxue.Study on mental workload of tower crane operatorin building construction［J］.Journal of Safety Science and Technology,2016,12(S1):78-83.
［2］尚倩.基于心理负荷的生产效率研究［D］.杭州:浙江大学,2013.
［3］CHEN S K,XU K L,YAO X W,et al.Psychophysiological data-driven multi-feature information fusion and recognition of miner fatigue in high-altitude and cold areas ［J］.Computers in Biology and Medicine,2021,133:110-116.
［4］冯佳.高危行业企业员工的心理资本和应对方式的关系研究［D］.南京:南京大学,2013.
［5］吴起,汪丽莉,匡蕾,等.人的不安全行为对高危行业从业人员安全评价的影响研究［J］.中国安全科学学报,2008,18(5):28-35. WU Qi,WANG Lili,KUANG Lei,et al.Research on the effect of people's unsafebehaviors on the safety assessment of employees in high-risk industry ［J］.China Safety Science Journal,2008,18(5):28-35.
［6］李乃梁,陆勇,李开伟.无人机操作主观心理负荷测评与心率数据比较［J］.中国安全生产科学技术,2021,17(4):35-41. LI Nailiang,LU Yong,LI Kaiwei.Comparison of subjective mental load evaluation and heart rate data of UAV operation ［J］.Journal of Safety Science and Technology,2021,17(4):35-41.
［7］HAO T T,ZHENG X,WANG H Y,et al.Linear and nonlinear analyses of heart rate variability signals under mental load ［J］.Biomed Signal Process Contral,2022,77:103758.
［8］靳慧斌,陈健,刘文辉,等.眼动指标在实时测量心理负荷中的应用进展［J］.科学技术与工程,2015,15(30):79-85. JIN Huibin,CHEN Jian,LIU Wenhui,et al.Application progress of eye movement indicators in real-time measurement of mental load ［J］.Science Technology and Engineering,2015,15(30):79-85.
［9］张奇良,杨坤华,曲行达,等.基于多模态生理信号的驾驶人脑力负荷评估［J］.深圳大学学报(理工版),2022,39(3):278-286. ZHANG Qiliang,YANG Kunhua,QU Xingda,et al.Evaluation of drivers’mental workload based on multi-modal physiological signals［J］.Journal of Shenzhen University (Science and Technology Edition),2022,39(3):278-286.
［10］SINGH R R,CONJETI S,BANERJEE R.A comparative evaluation of neural network classifiers for stress level analysis of automotive drivers using physiological signals ［J］.Biomedical Signal Processing and Control,2013,8(6):740-754.
［11］CINAZ B,ARNRICH B,MARCA R L,et al.Monitoring of mental workload levels during an everyday life office-work scenario ［J］.Personal and Ubiquitous Computing,2013,17:229-239.
［12］POURMOHAMMADI S,MALEKI A.Continuous mental stress level assessment using electrocardiogram and electromyogram signals ［J］.Biomedical Signal Processing and Control,2021,68：11456.
［13］DING Y,CAO Y Q,DUFFY V G,et al.Measurement and identification of mental workload during simulated computertasks with multimodal methods and machine learning ［J］.Ergonomics,2020,63(7):896-908.
［14］BLOCK R A,HANCOCK P A，DAN Z.How cognitive load affects duration judgments:a meta-analytic review ［J］.Acta Psychological,2010,134(3):330-343.
［15］FAN X L,ZHAO C Y,ZHANG X,et al.Assessment of mental workload based on multi-physiological signals ［J］.Technology and Health Care,2020,28(S1):67-80.
［16］RICHMAN J S,MOORMAN J R.Physiological time-series analysis using approximate entropy and sample entropy ［J］.AmericanJournal Physiology,2000,278(6):2039-2049.
［17］HUANG S T,LI J,ZHANG P Z,et al.Detection of mental fatigue state with wearable ECG devices ［J］.International Journal of Medical Informatics,2018,119:39-46.

相似文献/References:

[1]晋良海,尹洁,张荣坤,等.塔机驾驶员的持续性视觉注意负荷研究[J].中国安全生产科学技术,2018,14(3):157.[doi:10.11731/j.issn.1673-193x.2018.03.023]
　JIN Lianghai,YIN Jie,ZHANG Rongkun,et al.Study on continuous visual attention load of tower crane driver[J].JOURNAL OF SAFETY SCIENCE AND TECHNOLOGY,2018,14(12):157.[doi:10.11731/j.issn.1673-193x.2018.03.023]
[2]李乃梁,陆勇,李开伟.无人机操作主观心理负荷测评与心率数据比较[J].中国安全生产科学技术,2021,17(4):35.[doi:10.11731/j.issn.1673-193x.2021.04.006]
　LI Nailiang,LU Yong,LI Kaiwei.Comparison of subjective mental workload evaluation and heart rate data of UAV operation[J].JOURNAL OF SAFETY SCIENCE AND TECHNOLOGY,2021,17(12):35.[doi:10.11731/j.issn.1673-193x.2021.04.006]
[3]廖慧敏,吴玉敏,师凤起.建筑物应急疏散标识在干扰情境中的大学生视觉行为研究*[J].中国安全生产科学技术,2022,18(3):11.[doi:10.11731/j.issn.1673-193x.2022.03.002]
　LIAO Huimin,WU Yumin,SHI Fengqi.Research on visual behavior of college students on building emergency evacuation signs in interference situation[J].JOURNAL OF SAFETY SCIENCE AND TECHNOLOGY,2022,18(12):11.[doi:10.11731/j.issn.1673-193x.2022.03.002]
[4]景国勋,孙妍琳,蒋方.安全标志排列顺序的视觉注意特征研究*[J].中国安全生产科学技术,2022,18(11):197.[doi:10.11731/j.issn.1673-193x.2022.11.028]
　JING Guoxun,SUN Yanlin,JIANG Fang.Study on visual attention characteristics of arrangement order of safety signs[J].JOURNAL OF SAFETY SCIENCE AND TECHNOLOGY,2022,18(12):197.[doi:10.11731/j.issn.1673-193x.2022.11.028]
[5]汪磊,张之洋,邵铿睿,等.模拟SPO情境中的飞行员脑力负荷测量与分析*[J].中国安全生产科学技术,2022,18(12):26.[doi:10.11731/j.issn.1673-193x.2022.12.004]
　WANG Lei,ZHANG Zhiyang,SHAO Kengrui,et al.Measurement and analysis on mental workload of pilot in simulated SPO scenario[J].JOURNAL OF SAFETY SCIENCE AND TECHNOLOGY,2022,18(12):26.[doi:10.11731/j.issn.1673-193x.2022.12.004]
[6]师凤起,廖慧敏,郑珂欣.基于眼动的建筑平面疏散图色系优化及疏散效果研究*[J].中国安全生产科学技术,2023,19(7):169.[doi:10.11731/j.issn.1673-193x.2023.07.025]
　SHI Fengqi,LIAO Huimin,ZHENG Kexin.Study on color optimization and evacuation effect of building plane evacuation map based on eye movement[J].JOURNAL OF SAFETY SCIENCE AND TECHNOLOGY,2023,19(12):169.[doi:10.11731/j.issn.1673-193x.2023.07.025]
[7]汪磊,邵铿睿,吴昌旭,等.基于QN-MHP模型的SPO人机功能分配研究*[J].中国安全生产科学技术,2023,19(9):183.[doi:10.11731/j.issn.1673-193x.2023.09.027]
　WANG Lei,SHAO Kengrui,WU Changxu,et al.Research on human-machine function allocation under SPO based on QN-MHP model[J].JOURNAL OF SAFETY SCIENCE AND TECHNOLOGY,2023,19(12):183.[doi:10.11731/j.issn.1673-193x.2023.09.027]

备注/Memo

备注/Memo:
收稿日期： 2022-05-23
* 基金项目： 国家重点研发计划项目（2021YFC3001303）；国家自然科学基金项目（52074066）；中央高校基本科研业务费项目（N180104018）
作者简介： 郝腾腾，硕士研究生，主要研究方向为安全科学理论、系统安全工程与技术。
通信作者： 郑欣，博士，副教授，主要研究方向为安全科学理论、系统安全工程与技术。
更新日期/Last Update: 2023-01-16