Using machine learning to examine learner's engineering expertise using speech, text, and sketch analysis

Authors:
Worsley, M. & Blikstein, P.
Venue:
Project:
Multimodal Learning Analytics
Project:
Multimodal Learning Analytics
Year:
2011
Type:
Conference Presentation (with paper)
Abstract:
There continues to be a call for learning approaches that promote collaboration, creativity and innovation, as well as culturally-aware, constructivist approaches to STEM learning. Unfortunately, these skills tend to lie in direct opposition to forms of the most commonly used forms of assessment – national standardized tests. Though the education research field has recognized this discontinuity, we do not currently have the technology needed to holistically assess learning which is customized, and well-adapted to the learners’ culture. Accordingly, this study endeavors to fill that gap by presenting results from a multi-modal analysis of naturally derived student data. More specifically, we used student dialogue, and student drawing – two common artifacts in project-based, constructivist learning environments – to develop predictors for student expertise in the area of engineering design. By leveraging the tools of machine learning, natural language processing, speech analysis and sentiment extraction, we were able to identify a number of distinguishing factors of learners at different levels of expertise. As such, this study motivates continued work in this space, and the development of a new paradigm for assessing student knowledge construction.
Citation:

Using machine learning to examine learner's engineering expertise using speech, text, and sketch analysis. Paper presented at the 41st Annual Meeting of the Jean Piaget Society (JPS).

Publications

Year Title Venue Focus
in press Unraveling students’ interaction around a tangible interface using Multimodal Learning Analytics
Schneider, B. & Blikstein, P.
constructionism, data mining, interactive tabletop, machine learning, multimodal learning analytics, tangible user interface
EDM
2014
Programming pluralism: Using learning analytics to detect patterns in novices' learning of computer programming
Blikstein, P., Worsley, M., Piech, C., Gibbons, A., Sahami, M., & Cooper, S.
computer science education, machine learning, process mining
JLS Multimodal Learning Analytics
2013
Learning to paraphrase: using paraphrase detection of spoken utterances to predict learner expertise
Worsley, M. & Blikstein, P.
engineering education, machine learning, natural language
AERA Multimodal Learning Analytics
2013
Programming pathways: A technique for analyzing novice programmers' learning trajectories
Worsley, M. & Blikstein, P.
computational thinking, machine learning, programming
AIED Multimodal Learning Analytics
2013
Student coding styles as predictors of help-seeking behavior
Bumbacher E., Sandes A., Deutsch A., & Blikstein P.
computer science education, machine learning
AIED Multimodal Learning Analytics
2013
Towards the development of multimodal action based assessment
Worsley, M. & Blikstein, P.
computer vision, gesture recognition, machine learning, sequence mining
LAK Multimodal Learning Analytics
2012
An eye for detail: techniques for using eye tracker data to explore learning in computer-mediated environments
Worsley, M. & Blikstein, P.
agent-based modeling, machine learning, process mining, sequence mining
ICLS Multimodal Learning Analytics
2012
Multimodal Learning Analytics: enabling the future of learning through multimodal data analysis and interfaces
Worsley, M.
human dynamics, machine learning, natural language
ICMI Multimodal Learning Analytics
2012 Using automatic logging and machine learning to uncover hidden patterns in learning to program
Blikstein, P. & Worsley, M.
computer science education, machine learning
ICLS Multimodal Learning Analytics
2012 Using dynamic time warping and cluster analysis to analyze the learning of computer programming
Blikstein, P., Safdari. M. & Worsley, M.
computer science education, machine learning
AERA & ICLS Multimodal Learning Analytics
2011 Computing what the eye cannot see: educational data mining, learning analytics and computational techniques for detecting and evaluating learning
Blikstein, P. & Worsley, M.
computer science education, machine learning
AERA
2010 Learning analytics - natural assessments for constructionist learning environments
Worsley, M. & Blikstein, P.
assessment, machine learning, natural language
HSTAR-Cicero Multimodal Learning Analytics
2010
Toward the development of learning analytics: student speech as an automatic and natural form of assessment
Worsley, M. & Blikstein, P.
drawing analysis, machine learning, multimodal, natural language, speech analysis and recognition
AERA Multimodal Learning Analytics