Investigating Collaboration as a Process with Theory-driven Learning Analytics

Main Article Content

Carmel Kent
Mutlu Cukurova

Abstract

Although collaboration is considered a key 21st-century skill, oftentimes it is only assessed through the outcome measures of individual learners. In this paper, we draw upon collaborative cognitive load theory (CCLT) to explain the process of collaboration in learning from the point of view of the collective, rather than an individual learner. Using CCLT we suggest a new method of measuring the process of collaboration regardless of its outcome measures. Our approach — Collaborative Learning as a Process (CLaP) — uses social network analysis to evaluate the balance between interactivity gains and coordination costs of learner communities. Here, we demonstrate the approach using real-world data derived from the digital tracks of two online discussion communities. We argue that our conceptual approach can enable instructors and learners to unlock the black box of collaboration in learning. It has the potential to support the development of learner skills that go beyond cognition. We conclude the paper with the results of our investigation of the value of the approach to the online module instructor.

Article Details

How to Cite
Kent, C., & Cukurova, M. (2020). Investigating Collaboration as a Process with Theory-driven Learning Analytics. Journal of Learning Analytics, 7(1), 59–71. https://doi.org/10.18608/jla.2020.71.5
Section
Special Section: Beyond Cognitive Ability: Enabling Assessment of 21C Skills

References

Ausubel, D. P. (1968). Educational psychology: A cognitive view. New York: Holt, Rinehart & Winston.

Bernstein, E., Shore, J., & Lazer, D. (2018). How intermittent breaks in interaction improve collective intelligence, PNAS: Proceedings of the National Academy of Sciences of the United States of America, 115(35), 8734–8739. https://dx.doi.org/10.1073/pnas.1802407115

Borgatti, S. P., Mehra, A., Brass, D. J., & Labianca, G. (2009). Network analysis in the social sciences. Science, 323(5916), 892–895. http://dx.doi.org/10.1126/science.1165821

Casner-Lotto, J., & Barrington, L. (2006). Are they really ready to work? Employers’ perspectives on the basic knowledge and applied skills of new entrants to the 21st century US workforce. Washington, DC: Partnership for 21st Century Skills.

Chiocchio, F., & Essiembre, H. (2009). Cohesion and performance: A meta-analytic review of disparities between project teams, production teams, and service teams. Small Group Research, 40(4), 382–420. http://dx.doi.org/10.1177/1046496409335103

Chounta, I. A., Hecking, T., Hoppe, H. U., & Avouris, N. (2014). Two make a network: Using graphs to assess the quality of collaboration of dyads. CYTED-RITOS International Workshop on Groupware, 53–66. Cham, Switzerland: Springer. https://dx.doi.org/10.1007/978-3-319-10166-8_5

Claros, I., Cobos, R., & Collazos, C. A. (2016). An approach based on social network analysis applied to a collaborative learning experience. IEEE Transactions on Learning Technologies, 9(2), 190–195. http://dx.doi.org/10.1109/TLT.2015.2453979

Collazos, C., Padilla, N., Pozzi, F., Guerrero, L., & Gutierrez, F. (2014). Design guidelines to foster cooperation in digital environments. Technology, Pedagogy and Education, 23(3), 375–396. https://dx.doi.org/10.1080/1475939X.2014.943277

Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24, 87–114. http://dx.do.org/10.1017/S0140525X01003922

Cukurova, M. (2018). A syllogism for designing collaborative learning technologies in the age of AI and multimodal data. Proceedings of the 13th European Conference on Technology Enhanced Learning (EC-TEL 2018), 3–5 September 2018, Leeds, UK (pp. 291–296). Lecture Notes in Computer Science, Springer. https://dx.doi.org/10.1007/978-3-319-98572-5_22

Cukurova, M., Bennett, J. M., & Abrahams, I. Z. (2018). Students’ knowledge acquisition and ability to apply knowledge into different science contexts in two different independent learning settings. Research in Science and Technological Education, 36(1), 17–34. http://dx.doi.org/0.1080/02635143.2017.1336709

Cukurova, M., Luckin, R., Millán, E., & Mavrikis, M. (2018). The NISPI framework: Analysing collaborative problem-solving from students’ physical interactions. Computers & Education, 116, 93–109. https://dx.doi.org/10.1016/j.compedu.2017.08.007

Cukurova, M., Luckin, R., Millán, E., Mavrikis, M., & Spikol, D. (2017). Diagnosing collaboration in practice-based learning: Equality and intra-individual variability of physical interactivity. In É. Lavoué, H. Drachsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.), Data Driven Approaches in Digital Education, pp. 30–42. EC-TEL 2017. Lecture Notes in Computer Science, vol. 10474. Springer, Cham. https://dx.doi.org/10.1007/978-3-319-66610-5_3

Damon, W., & Phelps, E. (1989). Critical distinctions among three approaches to peer education. International Journal of Educational Research, 13(1), 9–19. https://dx.doi.org/10.1016/0883-0355(89)90013-X

Dede, C. (2010). Comparing frameworks for 21st century skills. 21st Century Skills: Rethinking How Students Learn, 20, 51–76.

De Laat, M., Lally, V., Lipponen, L., & Simons, R. J. (2007). Investigating patterns of interaction in networked learning and computer-supported collaborative learning: A role for social network analysis. International Journal of Computational Supported Collaborative Learning, 2(1), 87–103. https://dx.doi.org/10.1007/s11412-007-9006-4

Dillenbourg, P. (1999). What do you mean by “collaborative learning”? In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and computational approaches (pp. 1–16). Amsterdam, Netherlands: Elsevier.

Dunbar, R. I. M. (1992). Neocortex size as a constraint on group size in primates. Journal of Human Evolution, 22(6), 469–493. https://dx.doi.org/10.1016/0047-2484(92)90081-J

Easley, D., & Kleinberg, J. (2010). Networks, crowds, and markets: Reasoning about a highly connected world. Cambridge, UK: Cambridge University Press.

Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical literature review. Chicago, IL: University of Chicago Consortium on Chicago School Research.

Freeman, L. C. (1978). Centrality in social networks conceptual clarification. Social Networks, 1(3), 215–239.

Gardner, J. (2011). Assessment and learning. Thousand Oaks, CA: Sage Publications.

Goyal, S., & Vega-Redondo, F. (2005). Network formation and social coordination. Games and Economic Behavior, 50(2), 178–207.

Hutchins, E. (1996). Cognition in the wild. Cambridge, MA: MIT Press.

Ickes, W. (2002). Social cognition: Subjective and intersubjective paradigms. New Review of Social Psychology, 1(June), 112–121.
Ickes, W., & Gonzalez, R. (1996). “Social” cognition and social cognition: From the subjective to the intersubjective. In J. L. Nye & A. M. Brower (Eds.), What’s social about social cognition? Research on socially shared cognition in small groups (pp. 285–308). Thousand Oaks, CA: Sage Publications. https://dx.doi.org/10.4135/9781483327648.n12

Janssen, J., Kirschner, F., Erkens, G., Kirschner, P. A., & Paas, F. (2010). Making the black box of collaborative learning transparent: Combining process-oriented and cognitive load approaches. Educational Psychology Review, 22(2), 139–154. https://dx.doi.org/10.1007/s10648-010-9131-x

Johnson, D. W., & Johnson, R. T. (2002). Learning together and alone: Overview and meta-analysis. Asia Pacific Journal of Education, 22, 95–105. https://dx.doi.org/10.1080/0218879020220110

Johnson, D. W., Johnson, R. T., Roseth, C. J., & Seob Shin, T. (2014). The relationship between motivation and achievement in interdependent situations. Journal of Applied Social Psychology, 44(9), 622–633. https://dx.doi.org/10.1111/jasp.12280

Kalman, R. E. (1963). Mathematical description of linear dynamical systems. Journal of the Society for Industrial and Applied Mathematics, Series A: Control, 1(2), 152–192. https://dx.doi.org/10.1016/S0076-5392(08)63251-8

Kent, C., Laslo, E., & Rafaeli, S. (2016). Interactivity in online discussions and learning outcomes. Computers & Education, 97, 116–128. https://dx.doi.org/10.1016/j.compedu.2016.03.002

Kent, C., & Rechavi, A. (2018). Deconstructing online social learning: Network analysis of the creation, consumption and organization types of interaction. International Journal of Research & Method in Education, 43(1), 1–22. http://dx.doi.org/10.1080/1743727X.2018.1524867

Kent, C., Rechavi, A., & Rafaeli, R. (2019). Networked learning analytics: A theoretically informed methodology for collaborative learning analytics. In Y. Kali, A. Baram-Tsabari, & A., Schejter (Eds.), Learning in a networked society: Spontaneous and designed technology enhanced learning communities (pp. 145–175). Springer. https://dx.doi.org/10.1007/978-3-030-14610-8_9

Kester, L., & Paas, F. (2005). Instructional interventions to enhance collaboration in powerful learning environments. Computers in Human Behavior, 21, 689–696. https://dx.doi.org/10.1016/j.chb.2004.11.008

Kirschner, P. A., Kirschner, F., & Janssen, J. (2014). The collaboration principle in multimedia learning. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning, 2nd ed. (pp. 547–575). New York: Cambridge University Press. https://dx.doi.org/10.1017/CBO9781139547369.027

Kirschner, F., Paas, F., & Kirschner, P. A. (2011). Task complexity as a driver for collaborative learning efficiency: The collective working‐memory effect. Applied Cognitive Psychology, 25(4), 615–624. https://dx.doi.org/10.1002/acp.1730

Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86. https://dx.doi.org/10.1207/s15326985ep4102_1

Kirschner, P. A., Sweller, J., Kirschner, F., & Zambrano, R. J. (2018). From cognitive load theory to collaborative cognitive load theory. International Journal of Computer-Supported Collaborative Learning, 13, 213–233. https://dx.doi.org/10.1007/s11412-018-9277-y

Knight, S., Buckingham Shum, S., & Littleton, K. (2013). Epistemology, pedagogy, assessment and learning analytics, Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), 8–12 April 2013, Leuven, Belgium (pp. 75–84). New York: ACM. https://dx.doi.org/10.1145/2460296.2460312

Kwon, K., Liu, Y. H., & Johnson, L. P. (2014). Group regulation and social-emotional interactions observed in computer supported collaborative learning: Comparison between good vs. poor collaborators. Computers & Education, 78, 185–200. https://dx.doi.org/10.1016/j.compedu.2014.06.004

Liu, Y. Y., Slotine, J. J., & Barabási, A. L. (2011). Controllability of complex networks. Nature, 473, 167–173. https://dx.doi.org/10.1038/nature10011

Martinez, A., Dimitriadis, Y., Rubia, B., Gomez, E., & De La Fuente, P. (2003). Combining qualitative evaluation and social network analysis for the study of classroom social interactions. Computers & Education, 41(4), 353–368. https://dx.doi.org/10.1016/j.compedu.2003.06.001

Murnane, R. J., & Levy, F. (1996). Teaching the new basic skills: Principles for educating children to thrive in a changing economy. New York: Free Press.

Newman, M. E. J. (2001). The structure of scientific collaboration networks. PNAS: Proceedings of the National Academy of Sciences of the United States of America, 98(2), 404–409. https://dx.doi.org/10.1073/pnas.021544898

OECD (Organisation for Economic Co-operation and Development). (2015). PISA 2015 collaborative problem solving framework. Oxford, UK: OECD Publishing.

Paas, F., & Sweller, J. (2012). An evolutionary upgrade of cognitive load theory: Using the human motor system and collaboration to support the learning of complex cognitive tasks. Educational Psychology Review, 24(1), 27–45. https://dx.doi.org/10.1007/s10648-011-9179-2

Palincsar, A. S. (1998). Social constructivist perspectives on teaching and learning. Annual Review of Psychology, 49(1), 345–375. https://dx.doi.org/10.1146/annurev.psych.49.1.345

Peterson, L. R., & Peterson, M. J. (1959). Short-term retention of individual verbal items. Journal of Experimental Psychology, 58, 193–198. http://dx.doi.org/10.1037/h0049234

Rabbany, R., Elatia, S., Takaffoli, M., & Zaïane, O. R. (2013). Collaborative learning of students in online discussion forums: A social network analysis perspective. In A. Peña-Ayala (Ed.), Educational data mining: Applications and trends (pp. 441–466). Springer. https://dx.doi.org/1007/978-3-319-02738-8_16

Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In C. E. O’Malley (Ed.), Computer-supported collaborative learning (pp. 69–197). Berlin: Springer-Verlag. https://dx.doi.org/10.1007/978-3-642-85098-1_5

Siemens, G. (2005). A learning theory for the digital age. Instructional Technology and Distance Education, 2(1), 3–10.

Slavin, R. E. (2014). Cooperative learning and academic achievement: Why does groupwork work? Anales de Psicología/Annals of Psychology, 30, 785–791. https://dx.doi.org/10.6018/analesps.30.3.201201

Soboroff, S. D. (2012). Group size and the trust, cohesion, and commitment of group members [Unpublished doctoral dissertation]. University of Iowa.

Stahl, G. (2010). Group cognition as a foundation for the new science of learning. In M. S. Khine & I. M. Saleh (Eds.), New science of learning: Cognition, computers and collaboration in education (pp. 23–44). https://dx.doi.org/10.1007/978-1-4419-5716-0

Strijbos, J. W. (2011). Assessment of (computer-supported) collaborative learning. IEEE Transactions on Learning Technologies, 4(1), 59–73. https://dx.doi.org/10.1007/s40692-014-0009-7

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://dx.doi.org/10.1207/s15516709cog1202_4

Tsai, W. (2002). Social structure of “coopetition” within a multiunit organization: Coordination, competition, and intraorganizational knowledge sharing. Organization Science, 13(2), 179–190. https://dx.doi.org/10.1287/orsc.13.2.179.536

Tulin, M., Pollet, T. V., & Lehmann-Willenbrock, N. (2018). Perceived group cohesion versus actual social structure: A study using social network analysis of egocentric Facebook networks. Social Science Research, 75, 161–175. https://dx.doi.org/10.1016/j.ssresearch.2018.04.004

Veerman, A., & Veldhuis-Diermanse, E. (2001). Collaborative learning through computer-mediated communication in academic education. In Proceedings of the 1st European Conference on Computer-Supported Collaborative Learning (Euro-CSCL 2001), 22–24 March 2001, Maastricht, Netherlands (pp. 625–632). International Society of the Learning Sciences.

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological process. Cambridge, MA: Harvard University Press.

Wasserman, S., & Faust, K. (1994). Social network analysis: Methods and applications. Cambridge, UK: Cambridge University Press.

West, M. R., Kraft, M. A., Finn, A. S., Martin, R. E., Duckworth, A. L., Gabrieli, C. F., & Gabrieli, J. D. (2016). Promise and paradox: Measuring students’ non-cognitive skills and the impact of schooling. Educational Evaluation and Policy Analysis, 38(1), 148–170. https://dx.doi.org/10.3102/0162373715597298

Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010). Evidence for a collective intelligence factor in the performance of human groups. Science, 330(6004), 686–688. http://dx.doi./org/10.1126/science.1193147