Luận án tiến sĩ: Phát triển và phân tích kế hoạch kiểm thử phần mềm

NORTHCENTRAL DNIVERSITY DEVELOPMENT AND ANALYSIS OF COMPREHENSIVE WEBSITE SOFTWARE TEST PLAN INFORMATION PACKAGES A Dissertation submitted to The graduate faculty of the Department of Business & Technology Management In candidacy for the degree of DOCTOR OF PHILOSOPHY By DIANE L. STOTTLEMYER Prescott, Arizona November 2006 UMI Number: 3243181 Copyright 2006 by Stottlemyer, Diane L. All rights reserved. INFORMATION TO USERS The quality of this reproduction is dependent upon the quality of the copy submitted.

Broken or indistinct print, colored or poor quality illustrations and photographs, print bleed-through, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. ® UMI UMI Microform 3243181 Copyright 2007 by ProQuest Information and Learning Company.

All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. ProQuest Information and Learning Company 300 North Zeeb Road P. Box 1346 Ann Arbor, MI 48106-1346 Copyright 2006 Diane L Stottlemyer NORTHCENTRAL UNIVERSITY APPROVAL As Dissertation Chair, I certify that the committee listed below have read this dissertation and approve it as adequate in scope and quality for the degree of Doctor of Philosophy.

Candidate: Diane Stottlemyer Title of Dissertation: DEVELOPMENT AND ANALYSIS OF COMPREHENSIVE WEBSITE SOFTWARE TEST PLAN INFORMATION PACKAGES Dissertation Committee: Naa ` /~⁄2-07 Chair Al Endres, Ph.D Date Members Robert Haussmann, Ph.D Michael Ewald, Ph.D Joey Wallace, Ph.D ABSTRACT Development and Analysis of Comprehensive Website Software Test Plan Information Packages By Diane L. Stottlemyer Northcentral University The focus of this quantitative study was to examine if a relationship existed between the IEEE standards and the addition of maintenance and measurements. This study provided a series of development steps and evaluation rubrics used as a means for test plan generation and review. These steps can be used in addition to the current Institute of Electrical and Electronics Engineers standards.

Developer and evaluator participants were randomly chosen from students and mentors at Northcentral University. Findings from the experimental design indicated that there was evidence of an increase (p=.15) in quality associated with the addition of the proposed maintenance and measurement standards. The additional standards could be used to enhance the website test planning generation and review process. ACKNOWLEDGMENTS I would like to thank Dr.

Al Endres my committee chair for his patience as we went through this process. He is an excellent source of information and undying commitment and understanding. I would like to thank Dr. Ewald and Dr.

Haussmann for taking the time to review my paper. I would also like to thank Donna Bridgham, Ned Campbell and Dr. Joey Wallace for their support and patience as I progressed through this process. TABLE OF CONTENTS CODVTIEh.

ào Họ TH TH ch TH ii APPROVAILL. Ăn nh HT Tu TT TT TH ii. TH HH HT Tà HT Tà ngàngi Vv (V. vi LIST OF VƠpiiiiiiiiŸ.

1 Statement of Problem 0n. 2 Definition of Key TT€TTT. -- HH HT TT gà HT ng 3 Brief Review of Related LIterafUF€. cty TH ng ng nh nh ngờ5 Highlights and Limitations of Methodology.

cong Hy Hy HH ki Ho § Research EXpEeCfalOTNS. cong Hà TT Hà nà g0 01p g0 gà 9 Chapter 2: Review of Related Literature .-- sành ng HH ng ng nen 10 90 ca,. 10 Software Test Strategy and PÏlanning. -- «ngọn ng ng kg 10 Software Mainf€'iafiC€.-- HH HT TH HT TT nàn Hàng 15 RUïA 5-80 J2 2130041, NA.

nh nọ TH HH TT HT TH HH nv 18 Chapter 3: Methodology 0 TT. 19 Restatement of the Problem.- - - sọ ng HT ng ng ng nu ca HH gi re 19 Statement of HypotheSeS,.-- -- HH HH HH TH Tà ngàn nem 20 Description of Research Design. --‹ HH HH HH HH nàng ve20 Application of Power 20. 22 TSU power nh “6e.

22 Operational Definition of Constructs and Key Variables. nen 23 Description of Materials and Instrurnerif§. --- ng ng cư24 Selection of ))1:)1. 27 vi Discussion of Data PfoC€SSINB.

HH HH The 0kg 28 Methodological Assumptions and LimifafÍO'S. nhàn ng ngưng29 Ethical A SSUTäTIC€S. HH ng họ TT HH HT ii 30 Chapter 4: Flindings. ch go gi Tà net 32 OVCTVICN, Q0 nh HH HH Tu TH TT họ Ti 9E 32 FEndinngSs PP.

32 Research Question 1 and Associated Hypotheses.án nen 32 Research Question 2 and Associated Hypotheses.- Hee 33 Data Collection and RÑ€SUÏIS. - cuc HH HH TH Hà nàn Hà 34 Summary of results for research quesfion.- co vn vs key38 Summary of results for research question. «cu snvnvsns 38 Analysis and Evaluation of Findings. HH ng ng ng 40 SUMIMALY ÊPP0Ẽ7ẼẺỀ 8 6.

4I Chapter 5: Summary, Conclusions and RecommendatiOns. se, 43 QUITITđFY. HT HH họ Họ ch T1 001 0. Út HH HH TH in 47 RecomrmmnendafiOT0S.

- -- -- sọ TH Tp 49 R€fET€TC€S. LH ng HH nh HH như Error! Bookmark not defined. Appendix A: IEEE Standards Test Plan Information Package. co 58 Appendix B: IEEE+ Standards Test Plan Information Package.

69 Exhibit C1: Application for Approval for the Use of Animals or Humans. 69 Exhibit C2: Approval for the Use of Animals or Humanss.ssssssecseesseeseneeseneesenee 70 Exhibit C3: Letter of Inquiry sent to NCU students, staff and mentors for Study. 71 Exhibit C4: Online Consent Form for Research Project.ssssssssssseseseesesessseeseseenenee 72 Appendix D: Test Plan Specification Information Package Instructions. 74 Appendix E: Results for TPQ Data Collected for IEEE and IEEE Enhanced TPSIP.

75 Appendix F: Instructions for Evaluating IEEE Standards. son 77 Appendix G:Instructions for Evaluating IEEE+ Standards .csescessssssssscsesessensenees 78 Appendix H: MINITAB TM Results for TPQ and TSU Hypothesis Testing. 79 vii LIST OF TABLES Table 1. Input Parameters/Sample Size Requirements for TSU Power Analysis.

IEEE Enhanced Spreadsheet Data Results. IEEE+ Spreadsheet Data Results for TPQ Test Plan Specification Information J2 e e.Test and CI for Two PrOpOTIOTNS. HH HH HH HH fees 79 Table H2.Test for Equal Variances for IEEE and IEEE+ Test Results. 79 viii LIST OF FIGURES Figure 1.

Flow diagram of research Study. - cu HH HH ng Hang kg26 Figure 2. Boxplot template comparing the IEEE and IEEE+ TPIP quality data sets. Normality probability plot of IEEE totals for 31 samples.

Normality probability plot of IEEE+ totals for 32 samples. F-Test for equality of variances for TSU IEEE and IEEE+ data. Test for equal variances TSU for IEEE and IEEE+. cc eecssecssssesceseeseneeenes 40 LIST OF ABBREVIATIONS ETPDP — Enhanced Test Plan Development PToC€SS.

HH HH Hye 3 IEEE - The Institute of Electrical and Electronics Engin€€rS. «co neeeeesree 3 IEEE+ - The Institute of Electrical and Electronics Engineers’ Standards with Two Additional Proposed Standards. - cà HH HH HT Hà Hà TH BH 4 I9). 4 TPDP - Test Plan Development Process.

4 TPSIP —Test Plan Specification Information Package. - -‹ Hy gu ng,5 TSU - 8 ‹oÀ0 ii. 5 World Wide Web Consortium (W3C). «Hà nàn HH HH HH Hà Hàng 5 LIST OF APPENDICES Appendix A: IEEE Standards Test Plan Information Package.

seo 58 Appendix B: IEEE+ Standards Test Plan Information Package. - 5 HH HH TT HT Hà TH HH TT HT TT Hà Hà nàn Họng 69 Exhibit C1: Application for Approval for the Use of Animals or Humans. 69 Exhibit C2: Approval for the Use of Animals or Humans. Ăn 70 Exhibit C3: Letter of Inquiry sent to NCU students, staff and mentors for Study.

71 Exhibit C4: Online Consent Form for Research PTOJ€C(. 72 Appendix D: Test Plan Specification Information Package Instructions. 74 Appendix E: Results for TPQ Data Collected for IEEE and IEEE Enhanced TPSIP. 75 Appendix F: Instructions for Evaluating IEEE Standards .ccsscscsesecsesereeeenenees 77 Appendix G: Instructions for Evaluating IEEE+ Standards.

--ccccccsseseeses 78 Appendix H: MINITAB TM Results for TPQ and TSU Hypothesis Testing. 79 xi Chapter 1: Introduction This dissertation focused on the development of a new series of web site test plan specification guidelines for use as an extension to existing standards and guidelines outlined in the Institute of Electrical and Electronics Engineers (IEEE) standards number 829 (Schmidt, 2000). The IEEE provided standards which include guidelines for developing and verifying software and web sites. The augmented software standards include guidelines for software developers and testers to use in designing test plans that have consistency in appearance, form, structure and content.

The resultant dissertation has provided a series of development steps and evaluation rubrics that can be used as a means to establish a process for test plan generation and review. The research has identified and analyzed key areas as enhancements to the IEEE standards. The additions have addressed maintenance and measurement test planning. Both of these subject areas were analyzed for their potential to improve the current IEEE test plan guidelines for developing, examining, and testing web sites.

The purpose of the research was to evaluate, analyze, and enhance the existing IEEE guidelines, and to provide an enhanced test plan specification development and assessment process for developers. The additional maintenance and measurement guidelines may also be useful for verifying, validating, and improving preexisting requirements and/or test plans after the initial guidelines from the IEEE have been utilized. A web-site test plan specification information package development process was designed to provide developers with a series of steps for preparing a test plan. The series of steps reflected a reference to both the current IEEE standards as well as the additions for the standards and guidelines.

An assessment rubric was designed for both the current and augmented sets of standards, and used to evaluate and critique the test plan specification information packages. Using the current IEEE and augmented test plan requirements’ guidelines, a comparison was made between the current standard and the augmented test plans’ information packages relative perceived quality, and the relative uniformity of the test plan information packages quality scores. Statement ofProblem Since there are no maintenance or measurement standards in the IEEE test planning, developers and testers do not measure the percentage of errors during development or conduct maintenance after the application is released. So therefore it is necessary to have unified and/or standardized processes/guidelines that can be used to provide standards for consistency throughout the Information Technology industry.

The IEEE is considered a leader in the field of engineering standards and guidelines for development and testing. “The Institute of Electrical and Electronics Engineers (IEEE) Software Engineering Standards provides a comprehensive set of standards for developing and validating software” (Schmidt, 2000, p. Research was undertaken to examine if a relationship exists between the IEEE standards and the additional proposed standards, maintenance and measurements. This study was used to determine whether the inclusion of an additional set of test plan specification information package guidelines and assessment rubrics was significantly correlated with an increase in measured perceived test plan specification information package quality, and increased uniformity of the test plan specification information package evaluation process.

This study therefore addressed the following research questions: 1. To what extent will the use of the augmented testing and maintenance standards be positively correlated with an increase in perceived test plan specification information package (TPSIP) quality? 2. To what extent will the use of a test plan specification information package assessment rubric and the associated test plan guidelines increase the uniformity of the evaluations of test plan quality? The focus of this quantitative study was to examine if a relationship exists between the IEEE standards and the addition of maintenance and measurements. The study was used to address the issue of increasing test planning quality by including measurements and maintenance which provided a more comprehensive and uniform approach to developing test plans.