Simulation based analysis and optimization of the united states a

University of Louisville ThinkIR: The University of Louisville's Institutional Repository Electronic Theses and Dissertations 5-2018 Simulation-based analysis and optimization of the United States Army performance appraisal system. Evans University of Louisville Follow this and additional works at: https://ir.edu/etd Part of the Industrial Engineering Commons Recommended Citation Evans, Lee A., "Simulation-based analysis and optimization of the United States Army performance appraisal system. Electronic Theses and Dissertations.18297/etd/2906 This Doctoral Dissertation is brought to you for free and open access by ThinkIR: The University of Louisville's Institutional Repository. It has been accepted for inclusion in Electronic Theses and Dissertations by an authorized administrator of ThinkIR: The University of Louisville's Institutional Repository.

This title appears here courtesy of the author, who has retained all other copyrights. For more information, please contact thinkir@louisville. SIMULATION-BASED ANALYSIS AND OPTIMIZATION OF THE UNITED STATES ARMY PERFORMANCE APPRAISAL SYSTEM Lee A., United States Military Academy, 2000 M., Georgia Institute of Technology, 2009 A Dissertation Submitted to the Faculty of the J. Speed School of Engineering of the University of Louisville in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Industrial Engineering Department of Industrial Engineering University of Louisville Louisville, Kentucky May 2018 Copyright 2018, Lee A.

Evans All rights reserved SIMULATION-BASED ANALYSIS AND OPTIMIZATION OF THE UNITED STATES ARMY PERFORMANCE APPRAISAL SYSTEM Lee A., United States Military Academy, 2000 M., Georgia Institute of Technology, 2009 Dissertation Approved on April 13, 2018 By the following Dissertation Committee Dr. Bae, Chair Dr. Lihui Bai Dr. Erin Gerber Dr.

Lee Bewley ii ACKNOWLEDGMENTS My sincere gratitude goes to my advisor, Dr. Ki-Hwan Bae, for his mentorship and guidance throughout this process. I would also like to thank my dissertation committee of Dr. Lihui Bai, Dr.

Erin Gerber, and Dr. Lee Bewley for generously sharing their time and ideas. This research would not have been possible without the support from the United States Army Human Resources Command. I would like to thank Mr.

David Martino for his willingness to provide all of the data used to analyze the Army’s performance appraisal system. His intellectual curiosity has forced the Officer Professional Management Directorate to take a critical view of itself, resulting in an organizational culture that demands continuous improve- ment. Martino never lets a subordinate forget that behind every number is a person, a story, and a family; a mantra that has stuck with me throughout this study. Additionally, I would like to thank Mr.

Ike Zeitler, Ms. Teresa Monroe, and MAJ Nick Paul of the Officer Readiness Division for the countless hours spent querying databases in support of this dissertation. I have been extremely fortunate to learn from wonderful public educators; the most influential being my parents, Bill and Linda Evans, who combine for over 50 years experience teaching at the high school level. From flight school to graduate school, their support and encouragement has pushed me to expand my horizons and has made me a better person.

Finally, I would like to thank my wife, Kari, iii and my children, Elin, Brody, and Grant, for their love and continuous support throughout my time at the University of Louisville and during our entire 18-year journey in the Army. iv ABSTRACT SIMULATION-BASED ANALYSIS AND OPTIMIZATION OF THE UNITED STATES ARMY PERFORMANCE APPRAISAL SYSTEM Lee A. Evans April 13, 2018 From 2010 to 2016, the total number of active duty United States Army personnel decreased by over 17%. The Department of Defense uses a variety of instruments to downsize the services, of which the most immediate and impactful is through decreased promotion rates.

The Defense Officer Personnel Management Act of 1980 mandates the termination of officers twice not selected for promotion. As such, the promotion rates to the rank of lieutenant colonel (LTC) for 2015 and 2016 were the lowest over the past two decades. Central to each promotion board is the analysis of officer evaluation reports (OERs), the military version of performance appraisals. The biases associated with evaluating employees are well documented, par- ticularly in management literature.

These biases can often create a disconnect between the actual performance level of an employee and the management’s per- ception of the employee’s performance level. The performance appraisal system in the United States Army is a forced distribution system that restricts the number of above average evaluations raters are allowed to give subordinates. This struc- ture, combined with human behavior and system dynamics, creates an additional bias not currently addressed in literature. v Military personnel systems have long been the subjects for manpower modeling, or workforce planning, due to their size relative to most civilian organizations.

Techniques for manpower modeling include dynamic programming, goal program- ming, Markovian models, and simulation. These techniques assist policy makers with matching the supply of personnel with the available jobs. Rather than analyz- ing the aggregate requirements by occupation and seniority, this study determines the extent to which the current system promotes the best people into the available jobs. While this is often a subjective measurement, the use of discrete event sim- ulations allows us to quantify the effects of the current system and analyze future policy decisions.

In this dissertation, a discrete event simulation framework is considered to replicate the dynamics, structure, and regulatory constraints placed on the offi- cers in the U. Using performance appraisal data provided by the United States Army Human Resources Command, we create a multi-objective response function in order to quantify the human behavior associated with evaluating sub- ordinates. We are able to minimize the squared error of our system output with the multi-objective response function using simulation-optimization techniques. Uti- lizing simulation-optimization techniques for model validation enables estimating unknown input parameters, such as human behavior, based on historical data.

Furthermore, the model allows users to analyze the effects of current constraints on the evaluation system and the effects of proposed personnel policy changes. The effectiveness of the performance appraisal system is based on its ability to vi accurately evaluate the officers’ performance levels. The model output is analyzed by the number of misidentified individuals and the severity of the misidentification. An initial analysis showed that 20.07% of the officers in the system do not receive as many above average evaluations as their performance level warrants.

Additionally, structural changes such as decreasing the average number of a rater’s subordinates from fifteen to five increases the number of misidentified personnel by 59. Ranking and selection methods that include the Kim Nelson (KN) and the Nelson, Swann, Goldsman, Song (NSGS) procedures assists in determining the optimal combination of input parameters such as forced distribution constraints placed on raters, frequency of moves, number of subordinates assigned to each rater, and rater behavior. The simulation will serve as a tool for policy analysis to recommend policies and behavior that maximizes the extent to which the performance appraisal sys- tem accurately identifies the most qualified employees. Consequently, the results demonstrate broad applicability of simulation-optimization in the field of man- power modeling and human resource management.

vii TABLE OF CONTENTS 1 INTRODUCTION 1 1.3 Organization of this Dissertation .1 Military Manpower Modeling and Simulation .2 Performance Appraisal Systems. 49 3 PERSONNEL EVALUATION SIMULATION MODEL 54 3.2 Model Description and Notation .4 Model Verification and Validation .2 Sorting Function Parameter Estimation .1 Preliminary Results and Output Analysis .2 Assessing the Effect of Pool Size .3 Assessing the Effect of Time in Position .6 Response Function Development .2 Parameter Description and Optimization .3 Nelson, Swann, Goldsman, Song (NSGS) Procedure .4 Kim-Nelson (KN) Procedure .5 Applied Simulation Optimization Results .6 Robustness of Responses. 121 6 CONCLUSIONS 124 REFERENCES 127 CURRICULUM VITAE 141 x LIST OF FIGURES 1. Army active duty personnel strength from 1994 to 2016 (Source: Defense Manpower Data Center).

Army active duty lieutenant colonel promotion rate from 1996 to 2016 (Source: U. Army Human Resources Command) .3 Promotion induced attrition pattern prescribed in DOPMA (from Rostker et al.4 Flow chart of the simulated U. Army officer performance appraisal system .5 Excerpt from Department of the Army Form 67-10-2, Field Grade Officer Evaluation Report (Source: Department of the Army Reg- ulation 623-3: Evaluation Reporting System) .6 Promotion rates to the rank of lieutenant colonel by zone of con- sideration (Source: U. Army Human Resources Command) .7 Considered and selected populations to the rank of LTC (Source: U.

Army Human Resources Command) .1 Three sequential functions of performance appraisal systems (from Carroll and Schneier, 1982) .2 Rater motivation to provide accurate or distorted ratings (from Murphy and Cleveland, 1995) .3 Peer-reviewed journal publications on talent management since 1990 (Source: ProQuest) .4 Percent of majors receiving ACOM in first key development (KD) evaluation, fiscal years 2003-2007 (Wardynski et al.5 Seven step process modeling procedure (Hangos and Cameron, 2001) 47 2.6 Basic logic of a simulation-optimization procedure (April et al.7 Simulation-optimization techniques (Carson and Maria, 1997) .8 Fu’s simulation-optimization techniques (Fu, 2001) .1 Distribution of major pool sizes (Source: U. Army Human Re- sources Command) .2 Sample simulation output for 20 entities .3 Distribution of ACOM evaluations by time in grade for U. Army majors in the primary zone of consideration (Source: U. Army Human Resources Command) .4 Distribution of total number of ACOM evaluations for U.

Army majors in PZ zone of consideration (Source: U. Army Human Resources Command) .5 Simulation results for percent of majors receiving top evaluation by years in rank .6 Simulation results for percentages of total top evaluations received by majors .7 Simulation results showing relationship between D, Y , and T for linear sorting function .8 The effect on Y by minimizing weighted multi-objective response function D .9 The effect on T by minimizing weighted multi-objective response function D .10 Box plot showing the distribution of Qi for each number k of ACOM evaluations received .11 Boxplot showing the distribution of Qi for each number k of ACOM evaluations received with varying pool sizes.12 Boxplot showing the distribution of Qi for varying time in position and k number of ACOM evaluations received.13 Percent of officer misidentifications and critical misidentifications when varying the average rating pool size.14 Percent of officer misidentifications and critical misidentifications when varying the average time in position pool size.1 The effect of sorting function Q0ir for an officer with Qi =0.2 A comparison of misidentifications for the current and proposed performance appraisal systems.1 The distribution of top evaluations for the current and proposed performance appraisal systems.2 A comparison of misidentifications for the proposed performance appraisal system with an even and uneven performance distribution. 122 xiv LIST OF TABLES 1.1 LTC promotion rates by number of ACOM evaluations .1 Calculation of expected time in position for optimal p = 0.2 A summary of the minimum Y with sorting function parameters determined by simulation optimization.3 A summary of the minimum T with sorting function parameters determined by simulation-optimization.4 A summary of the minimum D with sorting function parameters determined by simulation optimization.5 The weight of seniority, by year j, in the rater sorting functions.6 Calculations for upper and lower bounds of αj , βj = 1 in Equation (3.7 Calculations for optimized time independent, discrete sorting func- tion parameters αj and βj for use with binary variable Tij and performance percentile Qi .8 A summary of the minimum D with variations of Equation (3.8) sorting function parameters determined by simulation optimization.9 A summary of the percentage of officers receiving k ACOM evalu- ations.10 Classification table of officer misidentification in the current perfor- mance appraisal system.11 The standard deviation and interquartile range of Qi for officers receiving k ACOM evaluations for pool sizes of 15, 10, and 5.12 A summary of the percentage of officers receiving k ACOM evalu- ations for an average pool size of 5.13 The standard deviation and interquartile range of Qi for officers receiving k ACOM evaluations for average time in position (TIP) of 5, 4, 3, 2, and 1 years.14 A summary of misidentified officers deserving k + 1 or k + 2 ACOM evaluations for an average pool size of 15.