Luận án tiến sĩ: Mixed-signal VLSI robust time-frequency feature extraction

Mixed-Signal VLSI Robust Time-Frequency Feature Extraction Yunbin Deng A dissertation submitted to the Johns Hopkins University in conformity with the requirements for the degree of Doctor of Philosophy. Baltimore, Maryland April, 2006 Copyright © 2006 by Yunbin Deng, All rights reserved. UMI Number: 3240696 Copyright 2006 by Deng, Yunbin All rights reserved. INFORMATION TO USERS The quality of this reproduction is dependent upon the quality of the copy submitted.

Broken or indistinct print, colored or poor quality illustrations and photographs, print bleed-through, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. ® UMI UMI Microform 3240696 Copyright 2007 by ProQuest Information and Learning Company.

All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. ProQuest Information and Learning Company 300 North Zeeb Road P. Box 1346 Ann Arbor, MI 48106-1346 Abstract Robust feature extraction is critical to pattern recognition and intelligent systems.

The problem of extracting reliable features from real-world signals is challenging because these signals are embedded in non-stationary noise; real-time operation requires fast adaptation; and mobile operation requires miniature hardware with low-power consumption. The need for robust real-time, and low-power feature extraction solutions calls for an integrative ap- proach that combines adaptive signal processing, machine learning, and micropower VLSI design. This dissertation presents time-frequency feature extraction algorithms, inspired by bi- ological models of auditory processing, for robust feature extraction. The algorithms map efficiently onto massive parallel VLSI hardware to provide real-time operation.

Analog models of computation are combined with digital programmability and adaptation, leading to significant savings in silicon area and power compared to purely analog or digital ap- proaches. A digitally programmable operational transconductance amplifier (OTA) offers more than three decades transconductance gain and wide linear range. Fully differential continuous-time OT A — C filters provide high dynamic range, high power supply noise re- jection, and programmable filter parameters. Fabrication-induced mismatch in the analog implementation is compensated through calibration using a generalized linear modeling and optimization scheme.

My research has resulted in a reconfigurable biomimetic mixed-signal VLSI feature extraction frontend for sonar and acoustic classification. The reconfigurable system is inte- grated on a single 3mm x 3mm chip in 0.5um CMOS technology. It features a 32-channel filter bank, with reconfigurable filter topology and programmable filter parameters. The chip consumes 9mW power for a total of 96 integrated biquad filters.

My work further led to application of the frontend architecture to robust speech and speaker recognition, sonar signal processing for under water target detection, and statisti- il cal analysis of speckle field for adaptive laser communication and adaptive optics. Exper- iments with robust speech and speaker recognition demonstrated the noise robustness of the extracted features under the auditory perception model, outperforming MFCC feature extraction algorithms. Combining ultrasonic time-frequency features with support ma- chine learning algorithms, we showed detection of buried underwater targets from sonar backscatter of single-ping sonar signals. In adaptive optics experiments, time-frequency analysis of speckle field signals indicated to be an effective laser beam quality metric for adaptive target-in-the-loop control.

Thesis advisor: Gert Cauwenberghs ill Acknowledgements I express my deepest gratitude to my thesis advisor, professor Gert Cauwenberghs, for introducing me into the art and science of analog circuit design, for being an outstanding mentor, for being always patient, extremely supportive, and tremendous inspirative through my Ph. My special thanks goes to professor Mikhail Vorontsov, Andreas Andreou, Sanjeev Khudanpur, and Paul Sotiriadis , whose advice and insights helped me to solve theoretical and practical problems during the course of this dissertation. Sanjeev Khudanpur and William Byrne at CLSP enriched my knowledge in the field of languge and speech processing by outstanding lectures. Mikhail Vorontsov, Ernst Polnau, and Gary Carhart at Army Research Lab and Intelligent Optics Laboratory at University of Maryland.

They provided me the resources that made this dissertation possible. I would like to thank Tim Edwards at Johns Hopkins Applied Physics Lab, Marc Co- hen at University of Maryland, Roman Genov at University of Toronto, and Shantanu Chakrabartty at Michigan State University, Milutin Stanacevic at New York State Uni- versity, for their insightful discussions with different facets of my research. I also would like to thank my colleagues and friends at Adaptive Microsystems Lab, Biomedical instrumentation lab, Computational Sensory-Motor Systems Lab, and Sensory Communication and Microsystems lab: Abdullah Celik, Adeel Abbas, Christian Sauer, Grant Mulliken, Karthikeya Murari, Philippe Pouliquen, and Ralf Philipp, for being sup- portive and providing assistance at different steps of my study. I thank my friends at CLSP, Yonggang Deng and Shankar Kumar for their assistance with speech recognition experi- ments.

Finally, my parents, my brothers and sisters provided a nurturing, loving environment and unconditional support through my rough time. I am grateful to my wife Fang, for her tremendous sacrifice throughout my study and encouragements when I needed the most. iv Contents List of Figures viii List of Tables xi I Introduction 1 Pattern Recognition and Feature Extraction 1.2 Time Frequency Feature Extraction Theory .1 Short-Time Fourier Transform .4 Auditory Feature Extraction .3 Relevance: Robust, Real-Time Feature Extraction. vn vu ng g gi kg và v.v na II Mixed-Signal VLSI Implementation Feature Extraction Circuit Design 2.

Q Q Q Q Q ng kg KV 2.2 Reconfigurable Filter Bank Architecture.3 Three Decades Programmable Fully Differential OTA Design.1 Current Division Circuit 2.2 Regulated Cascode Circuit 2. Common Mode Feedback Ctrcut.4 Full-wave RectiierDesign.5 OTA Noise Analysis. 0002 eee ee ee ees 2.4 Fully Differential Analog Filter Design.1 Biquad Filter Parameter Sensitivity Analysis .2 Biquad Filter Noise Analysis. &« 3 VLSI Implementation and Characterization 3.3 Full Wave Rectfe.4 Filter parameter calibratlon.2 The Generalized LinearModel.5 Programmable Gm-C Filter Calibration Experiments .1 Filter Parameter Modeling.

ee HI Applications 4 Robust Speech and Speaker Recognition 4.2 Auditory Perception Model.3 Kernel Predictive Cepstral Coding .1 Kernel Ridge Regression. ee ee ee 4. eee ee 44 Robust Digit Recognition.5 Robust Speaker RecOgnHHIOI.1 7 GMMand YAHO Database.2 Speaker Identification Experimen. c Q Q Q Q Q ng Q ng vn Q kg v v k va 5 Biomimetic Sonar Signal Processing 5.

c c Q Q Q ng vn nu nà k và v.2 Biosonar Target Detection System ©.22 Dolphin Cochlear Model.3 Biosonar Front-EndProcesor.3 Target Classifier: Support Vector Machines. ee 92 6 Real-Time Speckle Field Statistical Analysis 93 6.2 Speckle Field Statistics as Laser Beam Quality Criterion. SpeckSpect: A Mixed-Signal VLSI Speckle Spectrum Analyzer .4 Speckle Measurement Experiment.42 Sampled Static Speckle Field .3 Speckle Spectrum Statistics 2. ee 100 IV Conclusions 105 7 Conclusions 106 7.

Summary of Contributions. Three Decades Programmable OTA and Filter Design.2 Auditory Perception Model for Robust Speech and Speaker Recog- MitiOn 107 7.3 Real-Time VLSI Systems for Sonar and Speckle Field Signal Pro- COSSING Ce 108 7. eeva 108 Bibliography 110 Vil List of Figures 1.1 Human ear anatomy [16] 2.1 Silicon model of cochlear time-frequency signal processing (a) Parallel model; (b) Cascade model.2 Fully differential programmable OTA.3 Coarse current mirror.4 Fine current division ciITCUI.5 Regulated cascode ciIrCUI.6 Common mode feedback circut.7 Full wave rectifier circuit.8 OTA input referred noise vs, programming bits.9 First-order low-pass filter, .10 Digitally programmable OTA-C biquad filter creult.11 Feasible programming range, in theory, of biquad filter .12 The ideal OTA Gm as a function of programming bits. The four selectable scales depend on the coarse current scaling programming bits S0, 51; fine scaling with 8-bit resolution is provided by current division programming bits BO-— 7 Q v1 vu vn g kg k kg và ki vA 2.13 Biquad filter center frequency senSitVitV.14 Biquad filter quality factor sensitlVIy.15 Biquad filter gain sensitivity.16 Single ended noise analysis model for the fully differential OTA-C biquad filter, each OTA has a input referred noise of V,,, .17 Biquad filter input referred noise for constant Q filters.18 Biquad filter input referred noise for constant center frequency filters.1 Photomicrograph of filter bank fontend.2 Measured OTA dynamic range at different common mode input levels.3 Measured Œ„, as a function of digital programming bits B7_9 and 5}.4 Measured OTA input referred noise as a function of digital programming bits Br_o and Si.

ee ee ee ee te 3.5 Measured differential full wave rectifier response.6 Center frequency GLM data fit on 200 training (+) and test (o) data).7 Quality factor GLM data fit on 200 training (+) and test (o) datas.8 Gain GLM data fit on 200 training (+) and test (o) data.9 Low pass filter response at different CF’, with measured (dotted line), sim- ulated (solid line), and theoretical sensitivity error (vertical bars).10 Band pass filter response at different 7", with measured (dotted line), sim- ulated (solid line), and theoretical sensitivity error (vertical bars).11 Band pass filter response at different Q, with measured (dotted line), simu- lated (solid line), and theoretical sensitivity error (vertical bars).12 Programming error for center Íequency.13 Programming error for quality facto.1 Signal flow of logarithmic (4) and simplified linear (6) auditory perception model. c Q Q Q Q Q HQ ng ng à k k k k kg 63 4.2 Frequency response of the constant-Q bandpass filterbank .3 Speech waveform for digit’? weee 66 4.4 Time-frequency feature, left: generated by chip, right: generated by simu- lation, 1.5 Signal flow in KPCC feature extraction .1 Architecture for sonar object classification. Left: Model of dolphin biosonar echo-location [87, 85]. Right: Real-time hardware implementation, with reconfigurable front-end signal processor and massively parallel VLSI kernel- based classifier [93].2 Sonar backscatter signal, 134 samples, sample rate 150Khz.3 Sonar backscatter time-frequency distribution, left: generated by chip, right: generated by simulation.4 Correlation matrix of sonar training data from simulation.5 Correlation matrix of sonar training data from 64 lattice ispPAC chips.6 Correlation matrix of sonar training data from the designed filter bank chip.7 Correlation matrix of training, cross-validation and test sonar data from Ôi 1Š.8 Correlation matrix of training, cross-validation and test sonar data obtained from 64 lattice ispPAC chips.9 Correlation matrix of training, cross-validation and test sonar data otained from the designed filter bank chip.1 Measured filter bank channel response.2 Speckle field analysis experimental setup.3 Speckle field analysis system diagram.4 Statics Speckle from Camera 1 (a): Focused; (b): Non-focused.5 Statics Speckle from Camera 2 (a): Focused; (b): Non-focused.6 The signal of four channels depending on the position of the focus plane relative to the rough surface.

The center frequencies of the band pass filters in the four channel are programmed as follows: Channel 0 — 5kHz, Channel 1 — 10kHz, Channel 3 — 20kHz, Channel 5- 30kHz.7 Channel ratio as image quality metric (a): ratio of Channel 5 and Channel 1; (b): zoomed in version of ratio of Channel 5 and Channel 1. List of Tables 2.1 Parameters for noise calculation .2 Coarse current division circuit noise contribution.1 Measured OTA Characteristics ©.2 Measured Filter Characteristics, ©. ee ee ee 53 3.3 Filter Bank Chip Characteristics .1 Speech Recognition Rates for MFCC, KPCC, and Auditory Features.2 Speaker Identification Rates for MFCC and Auditory Features .1 SVM sonar target detection error rate, 305 training data.2 SVM sonar target detection error rate, 472 training data. 92 71 Comparative study of feature extraction sysfems.

109 xi Part I Introduction Chapter 1 Pattern Recognition and Feature Extraction 1.1 Introduction There are tremendous research interests in pattern recognition systems due to their wide applications: sequence analysis in bio-informatics, pattern search in data mining, internet information retrieval, biometric identification, image understanding for the blind, speech recognition and many others. The best pattern recognizers in many cases are humans, yet our knowledge of how humans recognize patterns is limited. In spite of almost 50 years of research and develop- ment in this field, the general problem of recognizing complex patterns in real life remains unsolved.