Phát triển hệ thống thời gian thực phát hiện và theo dõi lính cứu hỏa bị thương - Luận án tiến sĩ tại Đại học Bách khoa Hà Nội

VIETNAM NATIONAL UNIVERSITY HA NOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY PHAM VAN THANH DEVELOPMENT OF REAL-TIME SYSTEMS TO DETECT AND TRACK ON-DUTY INJURED FIREFIGHTERS USING ADVANCED SIGNAL PROCESSING TECHNIQUES DOCTORAL THESIS IN ELECCTRONICS AND TELECOMMUNICATIONS ENGINEERING Ha Noi - 2022 VIETNAM NATIONAL UNIVERSITY HA NOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY PHAM VAN THANH DEVELOPMENT OF REAL-TIME SYSTEMS TO DETECT AND TRACK ON-DUTY INJURED FIREFIGHTERS USING ADVANCED SIGNAL PROCESSING TECHNIQUES Major: Electronic Engineering Code: 9 52 02 03.01 DOCTORAL THESIS IN ELECCTRONICS AND TELECOMMUNICATIONS ENGINEERING Supervised by: Assoc. Tran Duc Tan Ha Noi - 2022 DECLARATION “I hereby declare that the work contained in this thesis is of my own, and it has been written by myself under the supervison of Professor Tran Duc Tan at Faculty of Electrical and Electronic Engineering, Phenikaa University, Hanoi, Vietnam during the period from September 2017 to August 2021. The thesis has not been previously submitted for a degree or diploma at this or any other higher education institution. I have duly acknowledged all the sources of information which have been used in the thesis.

The thesis content has been partly published in my list of publications as below: 1. Pham Van Thanh, Tuan Khai Nguyen, Duc Anh Nguyen, Nhu Dinh Dang, Huu Tue Huynh, Duc-Tan Tran*, “Adaptive Step Length Estimation Support Indoor Positioning System using Low-Cost Inertial Measurement Units”, 2020 IEEE Eighth International Conference on Communications and Electronics, pp. Pham Van Thanh, Le Quang Bon, Nguyen Duc Anh, Dang Nhu Dinh, Huynh Huu Tue, Tran Duc Tan, "Multi-Sensor Data Fusion in A Real-Time Support System for On-Duty Firefighters", Sensors 2019 (ISSN: 1424-8220 – SCIE). Van Thanh Pham, Duc Anh Nguyen, Nhu Dinh Dang, Hong Hai Pham, Van An Tran, Kumbesan Sandrasegaran and Duc-Tan Tran, “Highly Accurate Step Counting at Various Walking Speeds Using Low-Cost Inertial Measurement Unit Support Indoor Positioning System”, Sensors.

Pham Van Thanh, Duc-Tan Tran, Dinh-Chinh Nguyen, Nguyen Duc Anh, Dang Nhu Dinh, S. El-Rabaie and Kumbesan Sandrasegaran, “Development of a Real-time, Simple and High-Accurate Fall Detection System for Elderly Using 3- 1 DOF Accelerometers”, Arabian Journal for Science and Engineering. Pham Van Thanh, Anh-Dao Nguyen Thi, Quynh Tran Thi Thuy, Dung Chu Thi Phuong, Viet Ho Mau and Duc-Tan Tran, “A Novel Step Counter Supporting For Indoor Positioning Based On Inertial Measurement Unit”, 7th international conference on Integrated Circuit, Design, and Verification (ICDV), IEEE, pp. Nguyen Van Duong, Pham Van Thanh, Tran Van An, Nguyen Tuan Khai, Duong Thi Thuy Hang, Hoang The Hop and Tran Duc Tan, “Elevator Motion States Recognition Using Barometer Support Indoor Positioning System”, The 7th International Conference in Vietnam on the Development of Biomedical Engineering, IFMBE Proceedings, Springer, pp.

The Hop Hoang, Van Thanh Pham, Thuy Quynh Tran Thi, Huu An Nguyen, Tuan Khai Nguyen and Tan Tran-Duc, “Xây dựng hệ thống xác định độ cao bên trong nhà và công trình sử dụng đa cảm biến áp suất”, Hội nghị Quốc gia lần thứ XXI về Điện tử, Truyền thông và Công nghệ Thông tin (The 21st National Conference on Electronics, Communications and Information Technology), 2018, pp. Ha Noi, 16th January, 2022 Author Signature:………………………………………………. 2 ACKNOWLEDGEMENT I would like to express my sincere thanks to my advisor Assoc. Tran Duc Tan, Faculty of Electrical and Electronic Engineering, Phenikaa University for the guidance and support throughout the completion of my thesis.

My thanks go to all lecturers and people in Faculaty of Electronics and Telecommunications, Universisty of Engineering and Technology, Viet Nam National University Hanoi for their teaching and useful help. I give my special thanks to leaders, colleagues and Mr. Tran Van An at University of Fire Prevention and Fighting for their help, guidance and financial support in the entire thesis completion process. I am grateful to all people in MEMS lab as well as my students in Universisty of Engineering and Technology, Vietnam National University Hanoi and University of Fire Prevention and Fighting for their contribution.

I would also like to greatly thank the Vingroup Innovation Foundation (VINIF) - Vingroup Big Data Institute (VinBigdata) for their grant support and encouragement, which help me overcome financial problems and difficulties. Last but the most important, I would like to thank my parents, my brother, my sister-in-law because their comfort and support are the power for me going to success. “This work was supported by the Domestic Master/ PhD Scholarship Programme of Vingroup Innovation Foundation”. Ha Noi, 16th January, 2022 3 CONTENTS CONTENTS.

4 LIST OF ABBREVIATIONS. 7 LIST OF FIGURES. The purpose of thesis. Objectives and Scope of the Thesis.

Scientific significance and Contributions of the Thesis. OVERVIEW OF THE RESEARCH. Related Studies on Injured Detection. Related Studies on Indoor Positioning.

The challenges in study on injury detection and indoor positioning. SYSTEM DESCRIPTION, SENSOR ERRORS ELIMINATION AND MAP PROCESSING. Sensors Errors Elimination. 33 The 3-DOF Accelerometer.

33 The Magnetic Sensor. 36 The MQ7 Sensor. DEVELOPMENT OF A METHOD TO DETECT INJURED FIREFIGHTERS. Fall Detection Method.

46 Fall Detection Module. 46 Post-fall Recognition Module. 48 The Posture Recognition Estimation. 49 The Vertical Velocity Estimation.

Injury Detection for On-Duty Firefighters. 52 The Proposed Fall Detection Algorithm for Firefighter. 53 The Proposed Loss of Physical Performance Detection Algorithm for Firefighter. 57 The CO Detection Algorithm.

Result and Discussion. 61 The Experimental Results. 61 Fall Detection Results. 63 Loss of Physical Performance Detection.

64 The High CO Level Alerting Algorithm. 67 The Comparison on the Experimental Data. 68 The Comparison on Public Datasets. DEVELOPMENT OF A METHOD TO TRACK ON- DUTY INJURED FIREFIGHTERS.

The Step Counting Method. Step Length Estimation. 107 5 The Proposed Method. 107 Results and Discussion.

Turning Time and Direction Estimation. 113 Turning Time Estimation. 113 Turning Direction Estimation. Vertical Position Estimation.

INDOOR FIREFIGHTER POSITIONING AND TRACKING USING MULTI-SENSOR DATA FUSION AND MAP MATCHING ALGORITHM. Combining Data Fusion and Map Matching to Detect Indoor Position. 126 The Scenarios Testing. 135 CONCLUSIONS AND FUTURE WORK.

136 LIST OF PUBLICATIONS. 138 THE RELATED PUBLICATIONS. 141 6 LIST OF ABBREVIATIONS 3-DOF Three Degrees Of Freedom ADLs Activities of Daily Living CO Carbon Monoxide CO2 Carbon Dioxide COHb Blood Carboxyhemoglobin FFT Fast Fourier Transform GPS Global Positioning System iOS iPhone Operating System (Apple) I2C Inter-Integrated Circuit IC Incident Commander IMU Inertial Measurement Unit MEMS Micro-Electro-Mechanical systems NFPA National Fire Protection Association OADs On-Duty Activities PAA Piecewise Aggregate Approximation PASS Personal Alert Safety System ppm Parts Per Millions RMS Root Mean Square ROC receiver operating characteristic curve curve SAX Symbolic Aggregate approximation SCBA Self-Contained Breathing Apparatuses SpO2 Saturation of Peripheral Oxygen SVM Support Vector Machine UFPF University of Fire Prevention and Fighting US United States 7 US U. Occupational Safety and Health Administration Code of Federal OSHA Regulations CFR 𝐴𝑐𝑐 Accuracy 𝐹𝑁 False Negative 𝐹𝑃 False Positive 𝑆𝑒𝑛 Sensitivity 𝑆𝑝𝑒𝑐 Specificity 𝑇𝑁 True Negative 𝑇𝑃 True Positive 8 LIST OF SYMBOLS AND THEIR MEANINGS SYMBOLS MEANINGS Ax, Ay, Az Acceleration Along Ax, Ay, And Az Axes The Acceleration Of Gravity On The Surface Of The G Earth At Sea Level K The Accelerations Measure In Positive (𝑎+) And 𝑎+ and 𝑎− Negative (𝑎−) Of Ax, Ay And Az Axes The Accelerations Measure In Positive (𝑎+) And 𝑎′+ and 𝑎′− Negative (𝑎−) Of Ax, Ay And Az Axes After Multiple With K Factor.

𝐴+ and 𝐴− Acceleration In Each Axes After Calibrated The Root Mean Square (Rms) Of Acceleration Along A𝑐𝑐(𝑡) Ax, Ay, And Az Axes At Time 𝑡 𝐾𝑘 The Kalman Gain 𝑥̂𝑘 The Estimated Signal On The Current State 𝑥̂− The Estimated Signal On The Previous State 𝑘 𝑃𝑘 The Posteriori Error Covariance 𝑃𝑘− The Priori Error Covariance 𝑧𝑘 The Measured Value R The Environment Noise 𝐻𝑘 The Calculated Altitude In Meters 𝑃𝑘 The Measured Pressure P0 The Pressure At The Sea Level (P0 = 1013. 𝐴⊕𝐵 The Dilation Of A By The Structuring Element B 𝐴⊖𝐵 The Erosion Of A By The Structuring Element B The Duration Time From Acc Values Exceed The LFT TFE Threshold Until It Exceeds The UFT Threshold. 9 LFT Lower Fall Threshold UFT Upper Fall Threshold 𝜃 The Angle Between Ay And Gravity 𝑣 Vertical Velocity The Threshold To Distinguish Between The Rest And 𝑣𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 The Active States 𝑈𝑡ℎ The Upper Threshold U𝑝𝑡 The Upper Threshold to check the post-fall condition L𝑝𝑡 The Lower Threshold to check the post-fall condition T The Theta Angle 𝑌, 𝑃, 𝑅 The Yaw, Pitch, And Roll Angles The Upper Threshold To Check Loss Of Physical 𝐿𝑢𝑚𝑜𝑣 Performance Condition The Lower Threshold To Check Loss Of Physical 𝐿𝑙_𝑚𝑜𝑣 Performance Condition 𝛥𝐴𝑙𝑡𝑖𝑡𝑢𝑑𝑒 The Altitude Variations ∆𝐻 The Altitude Change D The Minimal Number Of Samples 𝑤𝑖𝑛_𝑠𝑖𝑧𝑒 The Window Size 𝑇𝑠 The Average Time Period To Perform Each Step AverD The Average Dynamic Thresholding 𝑇ℎ𝐷 Fluctuated Value The Difference Between 𝐴𝑐𝑐(𝑗) And The Gravity 𝑉𝑖𝑏𝐸𝑙𝑖 Acceleration 𝑇𝑖 The Time Range Between Two Neighboring Peak 𝑡𝑃𝑒𝑎𝑘𝑖+1 And 𝑡𝑃𝑒𝑎𝑘 𝑖 The Time Of Peak(I+1) And Peak(I) 𝑆𝑖 The Similarity Between Two Neighboring Peaks Kg Kilogram 10 𝐸 The Number Of The Estimated Steps 𝑇 The Number Of Reference Steps 11 LIST OF FIGURES Figure 0. US firefighter injuries by type of duty during 2015 [45].

The block diagram of the proposed system. The recorded data with and without using the simple Kalman filter. The Magnetic fields along Ax, Ay and Az before and after using the simple Kalman filter. The signal from the magnetic sensor before and after calibration.

The altitude signal comparison between with and without using the simple Kalman filter. The position of CO sensor on the mask. The binary image of a floor used in experimental testing. The map simplification algorithm.

The Erosion and Dilation operations [17]. The result of applying Erosion and Dilation with structuring element size of (7 × 7) for keeping wall. The result of applying Erosion and Dilation with structuring element size of (4 × 4) for keeping windows and stairs. The Map simplification achieved after using Erosion and Dilation operations.

The floor size detected after applying the flood fill algorithm. The unclosed floor structure. The floor size detected after applying the dilation operation and flood fill algorithm. The proposed fall detection algorithm.

An example of a fall event and the UFT, LFT and tFE thresholds. The time cycle of 6 walking steps. The different states of the user along three axes Ax, Ay and Az. The values of acceleration, Velocity and Theta angle of a volunteer when transiting between activities: standing – walking – sitting – walking – lying.

The injury detection algorithm for firefighters. The proposed fall detection algorithm. Firefighters move through the narrow paths or spaces. The proposed loss of physical performance detection algorithm.

The high CO level alerting algorithm. The volunteer is carrying the support device in his trouser pocket in the crawling state viewed from a side (a) and from above (b). (a) The RMS of acceleration of a fall forward from standing, first impact on knees; (b) the theta angle; (c) the pitch and roll angles. The loss of physical performance because of the accident (crawling then falling); (a) the RMS of accelerometer data; (b) the barometric data.

The loss of physical performance because of moving up in an elevator; (a) the RMS of accelerometer data; (b) the barometric data. a) Testing and measuring the CO level in the fire; b) the measured CO values. The RMS of acceleration of a fall forward from standing. The RMS of acceleration of crawling then falling as the scenario of Figure 3.

The RMS of acceleration of crawling then falling as the scenario in Figure 3.