A Bad Example that Quantitative Research Could Give...

http://www.classroom.umn.edu/projects/ALCOverview.html

I recently read an article talks about the impact of formal learning space on student learning. The author conducted a quasi-experimental research and seek to understand the relationship between formal learning spaces and student learning outcomes.

Here's how the author did his research. He wanted to compare two datasets of students' leaning outcome in two different learning spaces--one is a traditional classroom setting: blackboard in the front, students sitting in rows...etc; another is so-called Active Learning Classroom (ALC), which offers big round tables, switchable laptop technology for presentation, and wall-round marker board...etc. Except for different learning spaces, all other variables were controlled. However, the samples were from students who voluntarily enrolled the course and is impossible being random, this research cannot be categorized as experimental design.

The author found out one of the variable cannot be controlled because of the convenient sample set: students' academic achievement. In order to gain more credible results, the researcher collected students' ACT score, and claimed the score "have been demonstrated to be reliable and valid predictors of grades, especially among first-year college students." The ACT scores showed that students in traditional classroom had higher average of ACT than the students in ALC before they enter the college, statistical method was used by the author to standardized the score, and predicted that under all controlling variables, students in traditional classrooms should will gain higher average score in the end of the semester than students in the ALC classrooms.

The results shows that students in traditional classroom did gain higher score, but not as high as predicted, and the difference is statistic significant. Thus the author concluded that the impact in different learning space did exist and influence student learning.

That's it! Probably the weakest research I've ever read in recent years. The author only collected two sets of data, used t-test and multiple regression to analyze the data, then got the statistic significant result. Regardless the weak connection of the literature review session and the research design, what can this result actually tell us? We are always talking about statistical significant and practical significant in our stats classes. Although there is nothing wrong with the statistics in the study, as a reader, I cannot be persuaded there is a real difference because students' average score in the traditional classroom is only "3 points short" than the research estimate. Ironically, the students in ALC classroom actually earned less average score than the traditional one. The researcher manipulate the simple data and sought to present it in multiple fancy way to show the how much the "difference" the ALC score increases.

What do I expect to see in the topic related to the different learning space? I want to see the how innovate facilitation differ from traditional classroom can impact students' learning. Their learning achievement, can only be shown in their final scores, but different assignments or classroom activities they were engaged in. I want to see the patterns of students interaction in different learning spaces. I want to see how students and teachers percept in a different learning environments. From neither the results nor research design in this study, I am not informed any characteristic that is unique in an innovative learning space. The results are meaningless to me.

It could become more meaningful if this research only a pilot study for other research. Moreover, if the research design was duplicated and conducted at least three time and got the similar results, I will be more convinced.