Wednesday, November 16, 2011

SWTng 10: Assessing Learning with Concept Mapping

     Article ten by Gregoriades, Pampaka, and Michail (2009) is titled "Assessing Students' Learning in MIS using Concept Mapping," published in the Journal of Information Systems Education with 47 references and the author-supplied keywords MIS internationalization, concept mapping, learning assessment, and knowledge gaps. Here is the abstract:
The work described here draws on the emerging need to internationalize the curriculum in higher education. The focus of the study is on the evaluation of a Management Information Systems (MIS) Module, and the specification of appropriate course of action that would support its internationalization. To realize this goal it is essential to identify the possible learning needs of the two dominant cultural groups that compose the university student population in Britain, specifically European and Asian (UUK, 2005). Identification of knowledge pattersn among these cultural groups is achieved through the application of a concept mapping technique. The main research questions addressed are: (1) How to internationalize the MIS module's content and teaching methods to provide for students from different cultural backgrounds? (2) What are the main gaps in knowledge of students in MIS? The paper presents the results of this study and proposes actions needed to streamline the current teaching methods towards improving the quality of the students' learning experience.
     The driving focus of this research is how to internationalize curriculum to match the ethnic diversity of today's student body. However, the use of concept mapping to accomplish this is what I am interested in. A form of concept mapping has been used with domain specialists who develop instructional software but have little or no experience in instructional design. Frequently, their efforts become a tribute to everything they know about the topic, but lack any organization that would benefit long-term learning, real-world application, or even logical branching of the material. Thus, I am interested in the possibilities of application for use of concept mapping techniques in software training. In this research, it is used to evaluate students' level of learning and identify commonalities and gaps in knowledge between the two target groups, British students (representing Western European culture) and Chinese students (representing Asian culture).
     The two groups are identified as having different learning styles based on their approach to memorization. The Chinese style is base primarily on rote learning, perhaps because of the need to memorize the multitude of characters that make up their written language (pictograms). The British style, on the other hand, tends to be more reflective with less passive memorization. Thus, the intent is to identify knowledge gaps or misunderstandings among both groups with regard to the subject matter (MIS).
     The research followed student participation in an MIS module of instruction and used four steps: (1) introduction of concept mapping to students; (2) assessment of student understanding of concept mapping; (3) student preparation of a concept map of their understanding of the MIS module in 30 minutes on paper; and (4) analysis of the models based on students' origin and level of prior Information Systems (IS)/Information Technology (IT) experience. A master concept map key was previously developed based on the same MIS module. Student concept maps were compared to this key. The students maps were scored from three perspectives: (1) a holistic approach was used to assess the students' overall understanding of the module; (2) a relational approach was used to asses the quality and number of propositions specified in each model; and (3) an existential approach was used to assess the existence of concepts in the map compared to the master key. In the holistic method, each map was assigned a score between 1 and 10. In the relational method, some relationships between concepts were coded with low importance, some medium, and some high; each was weighted with a value of 1, 2, and 3, respectively. Color-coding was used to identify them on the key. The relations between concepts, or propositions, "were multiplied by their corresponding weighting factor and subsequently summed before reaching the final relational score of each map" (p. 423). Total score for the key was 282. In the existential method, if a correct concept from the master key was included in the student map, it was assigned a score of 1. If not, it received a 0. Again, a weighting score was applied and a total score of 59 arrived at for the existential method. These three possible scores (10, 282, and 59) were each given a percentage value and then averaged into an overall score with a range of 0-10.
     The main finding of interest to me is the low scores on the relational assessment. "Low performance is attributed to the difficulty in identifying relevant relationships among concepts and specifying them with correct propositions, which is a first indication of surface learning (Biggs, 2003)" (p. 424). Many learners are content simply to learn the facts of a matter and not understand why they are facts. They often lack the logic and critical thinking skills to put two and two together, let alone come up with four as the sum. Thus any software training must embed logic and critical thinking in its content. Questioning strategies also must assess higher levels of cognition rather than simple knowledge recall, e.g. "What must you add to x to make z, and what will happen if you don't?" The rest of the discussion and conclusions pertained to internationalization of the curriculum, which is outside the scope of my study.
REFERENCES
Biggs, J. (2003). Teaching for quality learning at university. Buckingham: Society for Research into Higher Education and Open University Press.
Gregoriades, A., Pampaka, M., and Michail, H. (2009). Assessing students' learning in MIS using concept mapping. Journal of Information Systems Education, 20(4), pp. 419-430.
UUK. (2005). Select committee evidence, Treasury Committee, Impact of China on the world and UK economy.

No comments:

Post a Comment