A Western New England University cross-disciplinary research team consisting of Professional Educator of Business David DiSabito, Associate Provost Josephine Rodriguez, Director of Assessment/Mathematics Lisa Hansen, and Associate Professor of Biology Thomas Mennella are boldly venturing into the realm of Generative Artificial Intelligence (GAI) to explore the applications of this powerful, emerging technology to the critical task of academic assessment.
Using a proprietary AI-based assessment tool called "Walter" developed by Professor DiSabito, the team is exploring the use of AI technology to improve the academic assessment process. This novel line of research implements the cutting-edge application of AI technology to address the perpetual challenge of assessment in higher education, and this team is one of few early adopters to begin testing and using AI in this manner.
The team recognizes the opportunities that the use of AI in research and learning brings, emphasizing the ability of AI-based assessment to be used not only as a tool for streamlining the academic assessment process but also as a coaching resource for faculty and students alike to improve teaching pedagogy and student writing performance.
"We'll still be doing the same old human stuff, but we'll be doing it with improved rubrics," DiSabito explained. "If our results are good, next semester we may use Walter as a second reader and use humans to assess higher-level matters, including curriculum. Where we would have spent days grading, we can get right to the discussion about how we can improve."
Findings from their first collaborative project, an application of ChatGPT to a mix of papers submitted for General University Requirements and individual course-level assessments, will be published in a June special edition of the peer-reviewed journal, New Directions in Teaching and Learning.
As with any new technology, there are challenges to using AI that must be investigated and addressed. As part of the findings from their study, the team reported a potential for bias in AI that may affect how the technology delivers feedback and/or generated assessment scores.
Members of the team emphasized that the feedback provided by AI platforms like ChatGPT is only as useful as the pool of information the technology has access to. Specifically, if biases exist in the published information that AI uses to create assessment feedback, these biases may also be reflected in the assessment feedback generated by the platforms. Noting this challenge, the team seeks to focus future research efforts on understanding the potential for bias in AI feedback and identifying strategies to counter this bias for more equitable assessment feedback.
"What we want to do is enhance the assessment methodology at Western New England College of Business. If we are successful here, we'll be able to implement it University-wide," DiSabito said.
Yet Walter has the potential to reach far beyond the WNE campus. Word of the new GAI tool spread quickly, and DiSabito and his team were invited to speak at the New England Educational Assessment Network (NEean) Fall Forum in November 2023. They have more engagements on the horizon.
The development of GAI assessment tools underscores the pivotal role that WNE holds as a transformative leader in higher education. With Governor Healey's recent appointment of President Robert E. Johnson to the Massachusetts Artificial Intelligence Strategic Task Force, the creation of Walter and GAI assessment tools keep WNE at the forefront of educational innovation as we move toward an AI-driven future.
