Phân tích Tổng quan Tài liệu Luận án Tiến sĩ: Từ tài liệu đào tạo đến kết quả thực tế

Utah State University DigitalCommons@USU All Graduate Theses and Dissertations Graduate Studies 12-2011 An Investigation of the Doctoral Dissertation Literature Review: From the Materials We Use to Prepare Students, to the Materials That Students Prepare Melynda H. Fitt Utah State University Follow this and additional works at: https://digitalcommons.edu/etd Part of the Instructional Media Design Commons Recommended Citation Fitt, Melynda H., "An Investigation of the Doctoral Dissertation Literature Review: From the Materials We Use to Prepare Students, to the Materials That Students Prepare" (2011). All Graduate Theses and Dissertations.edu/etd/1101 This Dissertation is brought to you for free and open access by the Graduate Studies at DigitalCommons@USU. It has been accepted for inclusion in All Graduate Theses and Dissertations by an authorized administrator of DigitalCommons@USU.

For more information, please contact digitalcommons@usu. AN INVESTIGATION OF THE DOCTORAL DISSERTATION LITERATURE REVIEW: FROM THE MATERIALS WE USE TO PREPARE STUDENTS TO THE MATERIALS THAT STUDENTS PREPARE by Melynda Rosalee Harrison Fitt A dissertation submitted in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY in Instructional Technology and Learning Sciences Approved: Dr. Burnham Major Professor Committee Member Dr. Nicholls Eastmond Dr.

Sheri Haderlie Committee Member Committee Member Dr. McLellan Committee Member Vice President for Research and Dean of the School of Graduate Studies UTAH STATE UNIVERSITY Logan, Utah 2011 ii Copyright © Melynda Rosalee Harrison Fitt 2011 All Rights Reserved iii ABSTRACT An Investigation of the Doctoral Dissertation Literature Review: From the Materials We Use to Prepare Students, to the Materials That Students Prepare by Melynda Rosalee Harrison Fitt, Doctor of Philosophy Utah State University, 2011 Major Professor: Dr. Andrew Walker Department: Instructional Technology and Learning Sciences Practically speaking, a well-conducted review of literature is central to a scholar’s ability to pose pertinent and timely questions within their field. As part of the culminating written assessment of a Ph.

candidate, the dissertation literature review provides a unique vantage point to explore future scholars’ preparation. In spite of its central role within the research process, research about how future scholars are taught the doctoral competencies necessary to conduct a review of the literature for the dissertation or how the dissertation literature reviews are assessed is limited. In two separate studies, this research uses the Boote and Beile’s Literature Review Scoring Rubric as a framework to explore the textbooks used in the early stages of doctoral education and the quality of dissertation literature reviews from a field of education research. In the first study, seven of the top selling education research methods textbooks from 2010 were analyzed to determine how well they cover the 12 iv performance criteria on the rubric.

While the results were varied, the majority of textbooks were not adequate in their coverage of the performance criteria identified by Boote and Beile. In short, the materials used to prepare doctoral students may not be equal to conveying critical components of the literature review. Efforts were then devoted to a replication study of exploring the end results of doctoral training and preparation. In the second study, the Literature Review Scoring Rubric was used to assess the quality of 30 randomly selected dissertation literature reviews from Instructional Technology.

The scores of the dissertation literature reviews were also varied. While some dissertation literature reviews in this study were of high quality and scored well, the majority of them were of a lower quality. (137 pages) v PUBLIC ABSTRACT An Investigation of the Doctoral Dissertation Literature Review: From the Materials We Use to Prepare Students, to the Materials That Students Prepare by Melynda Rosalee Harrison Fitt, Doctor of Philosophy Utah State University, 2011 To reduce the risk of repeating prior research efforts or choosing incorrect research methods, a sound literature review should be performed before undertaking a new study. As such, the literature review occupies a well-defined role in the research process.

It is natural to assume much research has been done in how these skills are taught to future scholars. However, this is not the case. Research in this area is limited and varied. This dissertation builds on existing efforts and fills in a portion of the missing research.

This work examines some of the textbooks used to teach doctoral students literature review skills. It also looks at the current state of dissertation literature reviews from a specific field in education, Instructional Technology. The Boote and Beile Literature Review Scoring Rubric is a widely used source of information about important criteria for a dissertation literature review. A scoring rubric is a list of critical features for a piece of work.

Rubrics help students know how their work will be evaluated. In this dissertation, researchers use the Literature Review Scoring Rubric as a framework to examine textbooks used to teach doctoral students literature review skills. They then assess the quality of dissertation literature reviews using the rubric. In the first study, researchers analyzed seven top-selling education research textbooks using content analysis techniques.

They wanted to determine how well the textbooks covered the items on the Literature Review Scoring Rubric. Each textbook received a final letter grade, much like a student in a classroom. Three of the textbooks received a failing grade of F, one received a C-, another received a B, and one received an A-. This study supports the claim that textbooks used to teach doctoral students tend to focus on search strategies and not on the more broad requirements of a dissertation review.

The second study replicates Boote and Beile’s study. Using the Literature Review Scoring Rubric, researchers evaluated 27 randomly chosen dissertations from vi Instructional Technology. They wanted to know if the literature reviews from Instructional Technology scored differently than ones from the general field of education. They also wanted to know if the dissertation study design (i., qualitative, quantitative, or mixed methods) affected the quality of the review.

The researchers also examined the rubric’s ability to consistently measure the quality of the reviews. The study showed the literature reviews from Instructional Technology had a lower average score (19.96 out of 37 possible points) than ones from education as a whole (24.08 out of 37 possible points). The lower average scores may be due to the field itself. It may also be in part because researchers did not select dissertations based on the quality of the program.

Finally, the use of different researchers than the Boote and Beile study may have been a factor in the differences. Study design also had little effect on the overall score of the dissertation literature review. Quantitative dissertations scored better From a practical viewpoint, faculty can use the findings from the first study to guide the selection of teaching materials. They can also examine the curriculum to determine how it can be strengthened or supplemented.

From a scholarly view, these two studies add to the developing discussion about the dissertation literature review. The first study addresses the oft-neglected research surrounding materials used to teach literature review skills. The second study extends Boote and Beile’s research into a specific field of study. vii DEDICATION To God, whose plan is always better than mine.

viii ACKNOWLEDGMENTS Giants have necessarily lined the path to this dissertation. Without the selfless help, sustaining belief, and inspiration of these individuals, I could have not accomplished the task at hand. I would like to thank:  My mentors and committee members who have supported me in my efforts to overcome the challenges I have encountered. Andy, Nick, Byron, Sheri, and Karl—thank you.

 The women who have stood in the gap for me—Kim, Sue, JaNiel, and Marie. Thank you for taking such wonderful care of my babies.  My mother-in-law, Penny, who never lets me take myself too seriously.  My mom, Rose Colleen, for planting the seeds and nourishing my passion for education.

 My new and improved dad, Phil, for restoring hope in my abilities and offering sage advice in the most difficult of moments.  My children, the jewels in my crown. Alee for the audacity to dream of something so big, Matthew for the courage to begin, Luke for the fortitude to continue, and Penelope Rose for the tenacity to finish.  My soul mate, my eternal companion, and my biggest fan, Jeff.

Thank you for never allowing me to surrender, to never give up. Most of all, thank you for not being a Milford.  My Father in Heaven for igniting my imagination those many years ago. Most of all, I thank Him for His many tender mercies that have sustained me through countless trials along this journey.

Much love and belief to you all. Melynda Rosalee Harrison Fitt ix CONTENTS Page ABSTRACT. iii PUBLIC ABSTRACT. viii LIST OF TABLES.

xi LIST OF FIGURES. 1 Why Study the Doctoral Dissertation Literature Review?. 2 Improving Education Research. 4 Teaching Doctoral Students to Do a Dissertation Literature Review.

5 Assessing the Doctoral Dissertation Literature Review: The Literature Review Scoring Rubric. 6 Structure and Objectives. REVIEWING THE DISSERTATION LITERATURE REVIEW. 10 Defining the Dissertation Literature Review.

11 Related Phenomena and Variables. 14 The Study of the Dissertation Literature Review. 19 Themes of Empirical Research. 20 Doctoral Students’ Views and Experiences.

21 Faculty Views and Experiences. 26 Assessment of the Dissertation Literature Review. 28 The Boote and Beile Literature Review Scoring Rubric. 34 Summary and Conclusions.

DISSERTATION LITERATURE REVIEW SKILLS AND EDUCATION RESEARCH METHODS TEXTBOOKS: AN INADEQUATE PEDAGOGY. 38 x Page Review of Literature. 58 Significance of Study. ASSESSING THE QUALITY OF DOCTORAL DISSERTATION LITERATURE REVIEWS IN INSTRUCTIONAL TECHNOLOGY.

66 Review of Literature. 73 Discussion and Conclusions. SUMMARY AND CONCLUSIONS. 82 Practical and Scholarly Implications.

85 Nature, Rationale, and Potential of the Dissertation Literature Review. 108 Appendix A: Textbook Coding Sheet. 109 Appendix B: Literature Review Scoring Rubric. 112 Appendix C: How to Grade a Dissertation.

114 Appendix D: Letters of Permission to Use Material. 122 xi LIST OF TABLES Table Page 3.1 Most Frequently Used Introduction to Education Research Textbooks .2 Descriptive Statistics for First Pass Scores by Criteria .3 Individual Textbook Consensus Coverage Scores by Criteria .4 Total Coverage Score and Overall Grade for Top Textbooks .1 Descriptive Statistics of Consensus Dissertation Scores. 75 xii LIST OF FIGURES Figure Page 3.1 Textbook grades and percentage of doctoral granting institutions in the United States who adopted the textbook for use in coursework .1 Dissertation scores according to coverage and study design. 76 CHAPTER 1 INTRODUCTION Overview The ability to judge the quality of research, extract the meaningful and applicable findings and then synthesize the information into a well crafted review is one indicator of a doctoral candidate’s competency to function within the research community (Golde, 2007; Kamler & Thomson, 2006).

The doctoral dissertation is the culminating written assessment of a PhD candidate’s educational experience and reflects the training received and abilities developed in a doctoral program (Cuetara & LeCapitaine, 1991; Isaac, Quinlan, & Walker, 1992; see also Association of American Universities, 1998; Council of Graduate Schools, 1991, 1997). As part of the doctoral dissertation, the literature review provides a unique vantage point. The dissertation literature review allows others to examine the overall quality of a student’s preparation for future work as an independent researcher who understands the literature related to their topic (Holbrook, Bourke, Fairbairn, & Lovat, 2007; Holbrook, Bourke, Lovat, & Dally, 2004). The centrality of the literature review and its well-defined role within in the research process would lead one to assume much research has been conducted in understanding how the literature review process is taught to doctoral students and how the dissertation literature review is assessed.

However, this is not the case as the research in this area is limited and varied.