Luận án tiến sĩ của Christopher Nguyen về đào tạo chống phishing tại Đại học Oklahoma

UNIVERSITY OF OKLAHOMA GRADUATE COLLEGE LEARNING NOT TO TAKE THE BAIT: AN EXAMINATION OF TRAINING METHODS AND OVERLEARNING ON PHISHING SUSCEPTIBILITY A DISSERTATION SUBMITTED TO THE GRADUATE FACULTY in partial fulfillment of the requirements for the Degree of DOCTOR OF PHILOSOPHY By CHRISTOPHER NGUYEN Norman, Oklahoma 2018 LEARNING NOT TO TAKE THE BAIT: AN EXAMINATION OF TRAINING METHODS AND OVERLEARNING ON PHISHING SUSCEPTIBILITY A DISSERTATION APPROVED FOR THE DEPARTMENT OF PSYCHOLOGY BY ______________________________ Dr. Eric Day, Chair ______________________________ Dr. Shane Connelly ______________________________ Dr. Michael Kramer ______________________________ Dr.

Matthew Jensen ______________________________ Dr. Jorge Mendoza ______________________________ Dr. Lori Snyder © Copyright by CHRISTOPHER NGUYEN 2018 All Rights Reserved. To my parents Hero Nguyen and Donna Huynh.

Acknowledgements I have so many people that I would like to acknowledge for helping me reach this milestone. First and foremost, I would like to thank my major professor and mentor Eric Day for his guidance throughout my graduate school career. He has pushed me to become a better researcher and person, and I am honored to have learned from someone who reminds me to pursue all things in life with passion. Second, I would like to thank my committee members Shane Connelly, Michael Kramer, Jorge Mendoza, and Lori Snyder for their unwavering support and continuing to believe in my potential even in the most challenging of times.

Third, I would like to thank Alexandra Durcikova, Matthew Jensen, and Ryan Wright for welcoming me with open arms when I joined their grant team and helping me figure out logistics, materials, and everything in between throughout the course of this project. Fourth, I would like to thank Lynnetta Eyachabbe and her team in the OU IT department for partnering with me and making this a project that I am incredibly proud to have put together and excited to share with others. Fifth, I would like to thank Larry Arthur, Murad Moqbel, Shreyasi, and JEB Sheriff for allowing me to incorporate this project in their classes and make this into a meaningful learning opportunity for their students. Sixth, I would like to thank my classmates in the psychology department for their help piloting my materials and providing me with invaluable feedback.

Finally, I would like to thank my dearest friends, mentors, brother Tony, sister-in-law Loan, and parents for their unconditional love and support throughout my time at OU. These past 5 years in graduate school have been some of the most trying times in my life, and I am so incredibly grateful to have these people in my life who have encouraged me every step of the way. iv Table of Contents Acknowledgements. iv List of Tables .viii List of Figures.

4 Combating Phishing Attacks. 7 Anti-Phishing Training. 10 Rule-Based Training. 15 Methodological Issues with Studies of Overlearning.

17 Hypotheses and Research Questions. 28 Email identification tests. 29 Mock phishing attacks. 31 v Disposition to trust.

34 Mindfulness in technology. 34 Perceived Internet risk. 34 Computer self-efficacy. 34 Phishing identification expertise.

35 Pre-training motivation. 35 Big Five personality. 36 Main Effects of Training. 36 Effects of Training on Retention.

37 Effect of Overlearning on Retention. 40 Effect of Training Method and Overlearning on Retention. 42 Signal detection theory. 43 Effect of training and overlearning on d’.

45 Effect of training and overlearning on c. 47 Inclusion of “True” Learners. 49 Anti-Phishing Training on Phishing Identification and Susceptibility. 49 Overlearning on Phishing Identification and Susceptibility.

56 Limitations and Future Research. 65 Appendix A: General Phishing Training Content. 98 Appendix B: Password Management Training Content. 101 Appendix C: Rule-Based Training Content.

105 Appendix D: Mindfulness Training Content. 106 Appendix E: Example Emails with Training Feedback. 107 Appendix F: Practice Test. 109 Appendix G: Overlearning Test.

115 Appendix H: Email Identification Test Version 1. 121 Appendix I: Email Identification Test Version 2. 131 Appendix J: Mock Phishing Test Version 1. 141 Appendix K: Mock Phishing Test Version 2.

146 vii List of Tables Table 1. Summary of Study Hypotheses and Research Questions and Associated Support Found. Means, Standard Deviations, Reliabilities, and Correlations of Study Variables. Results of Mixed Design Analysis of Covariance – Adjusted Means and Standard Errors by Study Conditions for Scores on Email Identification Tests at Time 1 and Time 2.

Results of Mixed Design Analysis of Covariance – Means and Standard Errors by Study Conditions for Scores on Tests of Vulneraibility to Mock Phishing Attacks at Time 1 and Time 2. Results of Mixed Design Analysis of Covariance Predicting Scores for Email Identification Tests at Time 1 and Time 2. Results of Mixed Design Analysis of Covariance Predicting Scores for Tests of Vulnerability to Mock Phishing Attacks at Time 1 and Time 2. Results of Mixed Design Analysis of Covariance – Adjusted Means and Standard Errors by Study Conditions for d’ on Email Identification Tests at Time 1 and Time 2.

Results of Mixed Design Analysis of Covariance – Adjusted Means and Standard Errors by Study Conditions for c on Email Identification Tests at Time 1 and Time 2. Results of Mixed Design Analysis of Covariance Predicting d’ on Email Identification Tests at Time 1 and Time 2. Results of Mixed Design Analysis of Covariance Predicting c on Email Identification Tests at Time 1 and Time 2. 88 viii List of Figures Figure 1.

General study procedures. Interaction between training and test administration on email identification test scores. Effects of training and test administration on mock phishing test scores. Effects of overlearning and test administration on mock phishing test scores.

Signal detection theory response types in a phishing detection context. Signal detection theory distributions in a phishing detection context. Interaction between training and test administration on d’ (discriminability) for email identification tests. Interaction between training and test administration on c (response bias) for email identification tests.

Effects of overlearning and test administration on c (response bias) for email identification tests. 97 ix Abstract As phishing attacks become increasingly common and sophisticated, anti-phishing training must extend beyond teaching individuals about cues and rules associated with phishing. Specifically, training methods that teach individuals effective allocation of time and attentional resources to the nature and context of emails should be examined, as well as strategies for improving skill retention from training. Thus, the present study compared the effectiveness of rule-based and mindfulness training, as well as the influence of overlearning on training, on two tests of skill retention on phishing susceptibility (i., email identification tests and mock phishing attack tests).

Participants were 453 university undergraduates who received training and practice and then were tested immediately following training using an email identification test. Participants were then sent mock phishing emails 1 week and 8 weeks after training, as well as an additional email identification test 10 weeks after training. Results showed that individuals who received mindfulness training were significantly better at discriminating between legitimate and phishing emails, less susceptible to phishing attacks, and more cautious of phishing compared to those who received rule-based training. However, the discriminability effect of mindfulness training was subject to a similar rate of skill decay as rule-based training.

Although training did not differ as a function of overlearning, individuals who received 100% overlearning were significantly less susceptible to phishing attacks and more cautious of phishing compared to those who did not receive overlearning. Results are discussed regarding implications for implementing effective anti-phishing training to protect individuals and their respective organizations and institutions. x Introduction According to a 2016 report by the Federal Bureau of Investigation (McCabe, 2016), successful phishing attacks have increased by 270% since January 2015, which has resulted in an estimated 2.3 billion dollars in annual losses related to fraud, theft, damages to reputation, regulatory violations, and loss of intellectual property. To help combat these attacks, several countermeasures have been suggested to assist organizations and individuals in defending themselves, some of which include preventing messages from reaching users through email filters and website blockings, providing better Web interfaces and tools that warn users of suspicious websites, and training users on how to identify phishing emails and websites (Hong, 2012; Kumaraguru et al.

Although the first two strategies are ideal for protecting users against phishing attacks as a first line of defense, these technological mechanisms are not always foolproof and can be overcome as phishers become more sophisticated in their attack methods. If these attacks manage to bypass automated anti-phishing tools or systems in place and reach individuals’ email inboxes, it is ultimately up to individuals to make the decision as to how they will respond. Thus, it is imperative to educate users on the dangers of phishing attacks and how to avoid the negative consequences. Research has shown that anti-phishing training can increase individuals’ capability to identify phishing emails and can reduce their susceptibility to phishing (Karumbaiah, Wright, Durcikova, & Jensen, 2016; Kumaraguru et al., 2007b, 2010; Sheng, Holbrook, Kumaraguru, Cranor, & Downs, 2010).

However, the majority of previous anti-phishing training methods have focused on educating individuals on specific cues or signals associated with phishing emails through a rule-based approach, 1 which may become obsolete as phishing attacks evolve. Relying upon a fixed set of guidelines may not be effective across all contexts and may still leave individuals susceptible to phishing attacks because emails are not critically evaluated and done so out of habit (Vishwanath, Herath, Chen, Wang, & Rao, 2011). Thus, there is a need to examine alternative anti-phishing training methods that train individuals to allocate adequate attention and time to their emails and the context in which they are received. In addition, few studies have examined the extent to which the skills trained during anti-phishing training are retained over an extended period of time, with most studies examining retention intervals that are less than 1 month.

Although it is fairly established that skill retention decreases as time increases (Arthur, Bennett, Stanush, & McNelly, 1998; Wang, Day, Kowollik, Schuelke, & Hughes, 2013), the phishing literature has not thoroughly investigated the rate in which learning from anti-phishing training decays across time and the point in which these training effects diminish. Because constant refresher courses cost organizations time and money, it is necessary that anti-phishing training programs promote learning that is retained for longer periods of time. In this vein, overlearning has been found to be an effective training strategy in increasing retention by providing practice opportunities for individuals that go beyond the point of initial learning (Driskell, Willis, & Copper, 1992). Even so, studies of overlearning have generally found that these benefits quickly diminish over time and are only beneficial for short-term retention.

However, given that the majority of overlearning research involves laboratory studies with relatively simple tasks (e., verbal recall tasks) and short retention intervals, it is difficult to make clear conclusions 2 about retention rates of overlearning on training material that is more relevant and practical to real-world situations. Thus, there were two goals of the present study. First, I was interested in comparing the effectiveness of two anti-phishing training methods: rule-based training and mindfulness training. In contrast to rule-based training that has constituted the majority of anti-phishing training, anti-phishing training that incorporates components from mindfulness training may be particularly beneficial in helping individuals identify phishing emails (Baer, Smith, & Allen, 2004; Brown, Ryan, & Creswell, 2007; Jensen, Dinger, Wright, & Thatcher, 2017).

In the context of phishing, mindfulness training teaches individuals to devote their attention and effort to the context in which the messages are received. By first forestalling judgment and then reflecting upon the underlying requests or motives associated with email messages, individuals can make more careful and detailed evaluations that could prevent them from falling for a phishing attack.