🤖2020 MOE Professional Forum for Research and Practice AI in Assessment: Review of AI Possibilities by Jean, Lawrence, Suryani and Hai Siang

Breadcrumbs

Presenters: Ms Jean Phua, Mr Lawrence Wee Loo Kang, Dr Suryani Bte Atan (ETD XLABs e-Assessment Research Group), led by Mr Chia Hai Siang.

1. Introduction and Context:

This document summarizes a presentation given at the 2020 MOE Professional Forum for Research and Practice, focusing on the potential of Artificial Intelligence (AI) in transforming educational assessment (e-Assessment). The presentation, delivered by the ETD XLABs e-Assessment Research Group, aimed to provide a "literature scan of how AI technologies can support the different e-assessment stages" and an "evaluation of the readiness of AI technologies to be applied in the e-assessment stages."

The synopsis highlights the disruptive nature of AI and MOE's strategic interest in leveraging it for personalized education. It notes that MOE is already leading three use cases under the National AI Strategy: Adaptive Learning, Automated Assessment, and Learning Companion. The presentation specifically addresses the application of AI within the framework of e-Assessment, which is defined as "the application of computers to assessment processes, including delivery of tests, capture of responses and marking by either computer or human marker (Davies, 2010)." The common stages in e-Assessment systems are identified as: e-Authoring, e-Delivery, e-Scoring, and e-Reporting.

Key Quote: "Artificial Intelligence (AI) is the capability of computational technologies to mimic human action and thought. AI has risen as a disruptive technology in recent years. MOE is leading three use cases (Adaptive Learning, Automated Assessment, Learning Companion) under the National AI Strategy to engage AI to achieve personalised education through adaptive learning and assessment."

2. Key Themes and Proposed Solutions:

The core of the presentation revolves around exploring how AI technologies can enhance various stages of the e-Assessment process. The document mentions several areas of interest for MOE/ETD to consider:

Natural Language Processing (NLP) for Question Generation: The use of NLP to automatically generate comprehension questions is specifically highlighted as an area of interest for research and development efforts. This suggests a focus on automating the creation of assessment content, potentially improving efficiency and allowing for greater variety in question types.
Key Idea: Exploring the potential of AI, specifically NLP, to automate the time-consuming process of creating assessment questions.
Input Method Enhancements: While not directly AI applications within the e-assessment stages themselves, the presentation acknowledges the potential of technologies like "text to speech" and "handwriting to text" to significantly improve students' experience with e-assessment. These technologies can cater to diverse learning needs and provide alternative ways for students to interact with the assessment platform.
Key Idea: Recognizing the importance of user experience and accessibility in e-assessment, and how supportive technologies can enhance student engagement and participation.
Leveraging Existing AI Technologies: The presentation also includes a practical aspect, with a plan to demonstrate "existing AI technologies that can already be easily harnessed by students and teachers, such as speech to text, and handwriting to text." This suggests an immediate focus on empowering educators and learners with readily available AI tools to support learning and assessment.
Key Idea: Emphasizing the immediate applicability of certain AI technologies to enhance teaching and learning practices, even before large-scale integration into e-assessment systems.

3. Objectives and Expected Outcomes:

The presentation aimed to:

Provide a "literature scan" of AI applications in e-assessment.
Evaluate the "readiness of AI technologies" for implementation in these stages.
Summarize how AI can "enhance e-assessment practices."
Present proposed solutions for MOE/ETD consideration, including "projected timeframe" and resource allocation for "R&D efforts."
Inform "current and planned MOE/ETD AI-enhanced technology explorations."
Ultimately, "help teachers better customise students’ learning experiences" and "suggest possibilities to uplift teachers’ productivity."

Key Implication: The MOE/ETD is actively exploring and planning for the integration of AI into e-assessment with the goals of personalizing learning for students and improving efficiency for teachers.

4. Next Steps and Considerations:

The presentation serves as an initial review and evaluation, intended to inform future decisions regarding AI adoption in e-assessment. The mention of "projected timeframe" and "allocate R&D efforts" indicates that MOE/ETD is moving beyond theoretical discussions towards practical implementation. The focus on both enhancing student experiences and improving teacher productivity suggests a holistic approach to leveraging AI in education.

The inclusion of references to the "National AI Strategy" and a JISC publication on "Effective Assessment in a Digital Age" underscores the broader strategic context and foundational research informing these explorations.

In Conclusion:

This presentation from the 2020 MOE Professional Forum provides a valuable overview of the potential of AI in revolutionizing e-assessment. It highlights MOE's proactive stance in exploring these possibilities, driven by the National AI Strategy and a commitment to personalized learning and enhanced teacher productivity. The identified areas of interest, such as NLP for question generation and the integration of supportive AI-powered input methods, signal key directions for future research, development, and implementation within the Singaporean education landscape. The emphasis on demonstrating readily available AI tools further suggests a pragmatic approach to empowering educators and students in the immediate term.

AI in Assessment: A Review of Possibilities - Study Guide

Key Concepts

Artificial Intelligence (AI): The capability of computational technologies to mimic human action and thought.
e-Assessment: The application of computers to assessment processes, including the delivery of tests, capture of responses, and marking by either computer or human.
Adaptive Learning: An educational method that uses technology to tailor learning experiences to an individual's needs.
Automated Assessment: The use of AI or other computer-based systems to automatically score or evaluate student work.
Learning Companion: An AI-powered tool designed to support students in their learning process.
National AI Strategy: A national plan that outlines the use of AI in various sectors, including education. In the context of the source, it includes three education use cases: Adaptive Learning, Automated Assessment, and Learning Companion.
e-Assessment Stages: The typical phases involved in computer-based assessment: e-Authoring (creating assessments), e-Delivery (administering assessments), e-Scoring (evaluating responses), and e-Reporting (generating feedback or results).
Natural Language Processing (NLP): A branch of AI that enables computers to understand, interpret, and generate human language.
Text to Speech: Technology that converts written text into spoken words.
Handwriting to Text: Technology that converts handwritten text into digital text.
MOE/ETD: The Singapore Ministry of Education/Educational Technology Division, the leading educational body exploring AI in assessment discussed in the source.
R&D: Research and Development, efforts dedicated to innovation and improvement in technologies and practices.

Short-Answer Quiz

Define Artificial Intelligence (AI) as presented in the source. What is its significance in recent years according to the text?
Explain the concept of e-Assessment. What are the four common stages involved in e-Assessment systems?
According to the source, what are the three use cases under Singapore's National AI Strategy that MOE is leading in education? Briefly describe one of these use cases.
What is the primary aim of the presentation summarized in the provided excerpts? What two key aspects does it intend to provide regarding AI in e-assessment?
How can Natural Language Processing (NLP) potentially enhance e-assessment, as mentioned in the text? Provide a specific example from the source.
While not directly related to AI applications in e-assessment stages, what potential benefit do input methods like text to speech and handwriting to text offer in the context of e-assessment?
Who are the key individuals and groups mentioned in the text involved in the "AI in Assessment: Review of AI Possibilities" presentation? What are their roles or affiliations?
What is the overall intent behind MOE/ETD's exploration of AI-enhanced technologies in education, as suggested by the synopsis?
The source mentions a literature scan and an evaluation. What were the two main focuses of this review regarding AI technologies and e-assessment?
What is the role of Davies (2010) in the context of the information provided in the excerpt?

Answer Key for Short-Answer Quiz

Artificial Intelligence (AI) is defined as the capability of computational technologies to mimic human action and thought. The source states that AI has risen as a disruptive technology in recent years.
e-Assessment is the application of computers to assessment processes, including delivery of tests, capture of responses, and marking by either computer or human marker. The four common stages are e-Authoring, e-Delivery, e-Scoring, and e-Reporting.
The three use cases are Adaptive Learning, Automated Assessment, and Learning Companion. Adaptive Learning involves tailoring learning experiences using technology to individual student needs.
The primary aim is to provide a review of AI possibilities in e-assessment. It intends to provide a literature scan of how AI can support different e-assessment stages and an evaluation of the readiness of AI technologies for these stages.
NLP can be used to generate comprehension questions, potentially enhancing the e-Authoring stage by automating the creation of assessment items.
Input methods like text to speech and handwriting to text have the potential to improve students' e-assessment experiences by providing more accessible ways to interact with the assessment.
Ms Jean Phua, Mr Lawrence Wee Loo Kang, and Dr Suryani Bte Atan from the ETD XLABs e-Assessment Research Group led by Mr Chia Hai Siang were involved in the presentation. They are Lead Specialists or Senior Specialists in Technologies for Learning (TfL) within ETD.
The intent is to help teachers better customize students’ learning experiences and to suggest possibilities to uplift teachers’ productivity through AI-enhanced technology explorations.
The review focused on how AI technologies can support the different e-assessment stages (literature scan) and an evaluation of the readiness of these technologies for application in those stages.
Davies (2010) is cited as a source for the definition of e-Assessment, indicating their work provides a foundational understanding of this concept in a digital age.

Essay Format Questions

Discuss the potential impact of Artificial Intelligence (AI) on the traditional stages of e-Assessment (e-Authoring, e-Delivery, e-Scoring, and e-Reporting). Provide specific examples of how AI technologies, such as NLP, could be applied in each stage, drawing upon the information provided in the source.
The synopsis highlights three use cases under the National AI Strategy: Adaptive Learning, Automated Assessment, and Learning Companion. Analyze the potential interconnectedness and synergies between these three use cases in achieving personalized education through AI.
Evaluate the significance of the MOE/ETD's initiative to review AI possibilities in assessment. Considering the potential benefits for teachers and students, what factors should MOE/ETD prioritize when considering the implementation of AI technologies in e-assessment?
The source briefly mentions input methods like text to speech and handwriting to text as having potential in improving students' e-assessment experiences, even if not direct AI applications to e-assessment stages. Discuss the broader implications of accessibility and user experience when integrating technology, including AI, into educational assessment.
Based on the information provided, what are some of the key considerations and potential challenges that MOE/ETD might face when translating the "possibilities" of AI in assessment into practical and widespread implementation within the educational system? Consider aspects such as technology readiness, teacher training, and ethical implications.

Glossary of Key Terms

Artificial Intelligence (AI): The simulation of human intelligence in machines that are programmed to think and act like humans.
e-Assessment: The use of electronic technologies to assess learning, encompassing the creation, delivery, marking, and reporting of assessments.
Adaptive Learning: An educational approach that uses computer algorithms to adjust the learning experience in real-time based on an individual's performance and needs.
Automated Assessment: The process of using technology, often AI, to automatically evaluate and score student responses without direct human intervention for grading.
Learning Companion: An AI-powered virtual assistant designed to support students' learning by providing guidance, answering questions, and offering personalized feedback.
National AI Strategy: A government-level plan outlining the strategic development and application of artificial intelligence across various sectors, including education.
e-Authoring: The stage in e-Assessment focused on creating and designing digital assessment tasks and items.
e-Delivery: The administration phase of e-Assessment, where students access and complete assessments electronically.
e-Scoring: The process of evaluating student responses in e-Assessments, which can be done automatically by computers or by human markers using digital tools.
e-Reporting: The generation of feedback, grades, or other information about student performance based on e-Assessment data.
Natural Language Processing (NLP): A field of artificial intelligence concerned with enabling computers to understand and process human language.
Text to Speech: An assistive technology that reads digital text aloud to the user.
Handwriting to Text: Technology that recognizes and converts handwritten input into digital text.
MOE/ETD: Abbreviation for the Ministry of Education/Educational Technology Division in Singapore, the entity leading the exploration of AI in assessment discussed in the source.
R&D: Shorthand for Research and Development, referring to activities undertaken to innovate and create new or improved products, services, or processes.

AI in Assessment: Review of AI Possibilities

By the ETD XLABs e-Assessment Research Group

Ms Jean Phua, Lead Specialist, Technologies for Learning (TfL), ETD

Mr Lawrence Wee Loo Kang, Lead Specialist, TfL, ETD

Dr Suryani Bte Atan, Senior Specialist, TfL, ETD (on secondment to NIE wef July 2020)

led by Area Lead and Chief of ETD XLabs Mr Chia Hai Siang, Lead Specialist, TfL, ETD

2020 MOE Professional Forum for Research and Practice

Symposia

Technology-enabled Assessment: Now and Next

07 Oct 2020 1425 – 1540

Synopsis

Artificial Intelligence (AI) is the capability of computational technologies to mimic human action and thought. AI has risen as a disruptive technology in recent years. MOE is leading three use cases (Adaptive Learning, Automated Assessment, Learning Companion) under the National AI Strategy to engage AI to achieve personalised education through adaptive learning and assessment.

e-Assessment is the application of computers to assessment processes, including delivery of tests, capture of responses and marking by either computer or human marker (Davies, 2010). In e-Assessment systems, the most common stages are e-Authoring, e-Delivery, e-Scoring, and e-Reporting.
This presentation aims to provide

a literature scan of how AI technologies can support the different e-assessment stages;
an evaluation of the readiness of AI technologies to be applied in the e-assessment stages.

The presentation summarises how AI can enhance e-assessment practices. A variety of proposed solutions are evaluated, for MOE/ETD to consider for implementation with projected timeframe, and allocate R&D efforts. Some areas of interest include the use of Natural Language Processing (NLP) to generate comprehension questions. Input methods such as text to speech , and handwriting to text , although not directly related to AI applications to the e-assessment stages, do have potential in improving students’ e-assessment experiences. A demonstration of existing AI technologies that can already be easily harnessed by students and teachers, such as speech to text, and handwriting to text, will be presented. The presentation serves to inform current and planned MOE/ETD AI-enhanced technology explorations, with the intent to help teachers better customise students’ learning experiences, and suggest possibilities to uplift teachers’ productivity.

Davies, S. (2010). Effective Assessment in a Digital Age. JISC publication.

https://go.gov.sg/2020eassess

https://www.smartnation.gov.sg/docs/default-source/default-document-library/national-ai-strategy-summary.pdf?sfvrsn=55179e0f_4

Frequently Asked Questions: AI in Assessment (Based on MOE Professional Forum Presentation)

1. What is the fundamental concept of Artificial Intelligence (AI) as it relates to educational technology?

AI, in the context of educational technology, refers to the capability of computational technologies to mimic human cognitive functions, particularly action and thought processes. This disruptive technology is being explored for its potential to personalize education.

2. What are the primary use cases of AI that the Ministry of Education (MOE) in Singapore is currently focusing on within its National AI Strategy?

MOE is actively pursuing three key use cases under the National AI Strategy:

Adaptive Learning: Utilizing AI to tailor learning experiences to individual student needs and progress.
Automated Assessment: Employing AI to streamline and potentially enhance the efficiency and effectiveness of assessment processes.
Learning Companion: Developing AI-powered tools to support students in their learning journey.

3. How does the concept of "e-Assessment" relate to the application of AI in education?

e-Assessment is the use of computers in various stages of assessment, including test delivery, response capture, and marking (either by computers or human educators). AI technologies can be integrated into different phases of e-assessment (e-Authoring, e-Delivery, e-Scoring, and e-Reporting) to enhance these processes.

4. In what specific ways can AI technologies potentially support and improve the different stages of the e-assessment process?

The presentation outlines that AI can support various e-assessment stages. One area of interest mentioned is the use of Natural Language Processing (NLP) to automatically generate comprehension questions. More broadly, AI could assist with tasks such as creating more varied and adaptive test items, automating the scoring of open-ended responses, and generating insightful reports on student performance.

5. Beyond direct application to the e-assessment stages, what other AI-related technologies are considered valuable for enhancing students' learning experiences?

While not directly part of the e-assessment stages, input methods like text-to-speech and handwriting-to-text are seen as having significant potential to improve students' overall experience with e-assessment. These technologies can make digital assessments more accessible and user-friendly for a wider range of learners.

6. Are there any existing AI technologies that teachers and students can readily utilize to support learning and assessment?

Yes, the presentation mentions that readily available AI technologies such as speech-to-text and handwriting-to-text can already be harnessed by both students and teachers. These tools can aid in various learning activities and provide alternative input methods during assessments.

7. What are the main goals and intentions behind MOE/ETD's exploration of AI-enhanced technologies in education?

The primary intentions behind MOE/ETD's exploration of AI are to:

Help teachers better customize and personalize learning experiences for their students, catering to individual needs and learning paces.
Suggest possibilities for uplifting teachers' productivity by automating certain tasks and providing more efficient tools.

8. What considerations are involved in the potential implementation of AI technologies within MOE's e-assessment practices?

The presentation highlights the need for a literature scan to understand how AI can support e-assessment and an evaluation of the readiness of current AI technologies for application in these stages. This suggests that MOE/ETD is carefully considering the feasibility, effectiveness, and potential timeframe for implementing various AI solutions in e-assessment, including allocating resources for research and development in promising areas.