Great News!

My mentor has accepted my problem statement (I believe the words he used were "well crafted"). Now I need to align the methodology with it. I think the rough draft below is a good start; now I just need to fill it out and back it up with references. It sure feels good to finally be making some progress. Onward!

Rough-out of Methodology

Here's a rough draft of my proposed research methodology:

Design: Within-group (Interrupted Time Series) Experimental (not randomly assigned, one group, one or more interventions, dependent variables measured/observed after each intervention, group becomes its own control). Permits significant control over threats to internal validity such as history, maturation of participants, statistical regression, and selection and treatment.

Independent variable: application of quality management principles/procedures to instructional design/development effort

Dependent variables:

1.       Production metrics (errors made in development, found in internal and external quality control/quality assurance reviews) compared before and after ISO:9000 registration
2.       Instruction metrics (test scores, graduation data) compared before and after ISO:9000 registration

Meta-analysis of Quality Management System (QMS) compared to instructional design theory/process found in ADDIE (not a variable, per se)

Descriptive statistics: average number of errors per number of layers in internal qc/qa reviews; average number of errors per number of layers in external qc/qa reviews; compare before and after registration. Mean only? Are mode and median relevant? Can they be ascertained? What about variance, standard deviation, and range? Do a practice analysis.

Inferential statistics:  ANOVA using t-test statistic for each dependent variable separately and two-way ANOVA using x² statistic comparing both dependent variables together against the independent variable. Software: SPSS

New Problem Statement

Here's my latest problem statement:

The principles and techniques of quality management have been applied to hard product manufacturing for decades (Walton, 1986). However, it is only recently that soft product development, such as instructional design, has begun to explore the need to understand and apply these same principles and techniques to services (Bhalla, 2009). The US government, as a consumer, is keenly aware of the need to use taxpayer money wisely, lest the injudicious procurement of products and services become a viral story on the Internet. This is especially true of the Department of Defense (DoD), whose FY 2010 budget includes 2.3% or $15.2 billion allocated for training (US OMB, 2009).

In order to justify spending taxpayer money and insure the procurement of quality instructional product for military service members, the US government and DoD have turned to the use of quality management principles and procedures as a measure of the quality of the service of government contracts (M. Verrill, personal communication, October 2009). One common measure of implementation of quality management principles is certification by the Organization for International Standards (ISO) under the ISO 9000 platform (ISO, 2010). However, many contracts are earmarked for bidding limited to small businesses only. For many of these companies, an ISO 9000 audit is a major capital investment, involving considerable time and effort (M. Verrill, personal communication, October 2010).

The subject raises several questions pertinent to the issues on which this research is based:

•  Is the requirement for ISO 9000 certification or use of ISO-like procedures a necessary evil, or does it duplicate procedures and practices already embedded in many instructional design models and principles of project management?

• Will focus on quality management principles create an environment like the manufacturing one in which they were designed?

• Will courseware developers tend to rely on them more than on instructional design principles?

• Will ISO 9000 certification, which focuses on the process, not the product, create a false sense in the Government as a customer and in the contractors that the instructional output will meet all the requirements of instructional design and learning theories?

References

Bhalla, K. (2006, February 13). Why a quality management system in service industries? Retrieved October 11, 2009, from http://www.isixsigma.com/library/content/c060213a.asp

 International Organization for Standardization (ISO). (2010). ISO 9000 – Quality management. Retrieved October 17, 2010 from http://www.iso.org/iso/iso_catalogue/management_and_leadership_standards/quality_management.htm

 

United States Office of Management and Budget (US OMB). (2009, May). Budget of the U. S. Government, Fiscal Year 2010. Washington, D.C.: U. S. Government Printing Office.

 Walton, M. (1986). The Deming Management Method. New York: The Putnam Publishing Group.