AI-Powered Educational Assessments for Quality Evaluation

The growing use of Generative Artificial Intelligence in education has created new opportunities for automating the development of examination questions and assessments. Large Language Models can generate multiple choice, short answer, and descriptive questions from course materials within minutes, reducing the workload of instructors. However, generating valid questions individually does not guarantee that a complete assessment is educationally effective. An AI generated question paper may still have an uneven distribution of difficulty levels, limited coverage of higher order cognitive skills, an imbalance in question types, or multiple questions that assess similar concepts. These issues can reduce the overall quality and diagnostic value of an assessment.
This paper proposes a Quality Evaluation Module that evaluates a complete AI generated question set as a whole before publication. Rather than focusing only on individual questions, the module examines four dimensions: difficulty distribution, coverage across Bloom's Taxonomy levels, balance among question types, and semantic redundancy. Each dimension is evaluated and combined to produce an overall quality score, along with recommendations that help instructors improve the assessment.
The module is designed to integrate with AI powered educational assessment platforms where instructors can upload study materials and use Large Language Models to generate and manage questions. It introduces an additional assessment level evaluation to determine whether the selected questions form a balanced and diverse question paper. The approach is based on research in AI assisted question generation, Bloom's Taxonomy alignment, and assessment quality evaluation.
The results obtained from three sample assessments show that the module can help improve the overall structure and quality of an assessment. After applying its recommendations, the question sets showed a better balance in difficulty, improved coverage of higher cognitive levels, and fewer questions with similar meanings. The overall quality scores also improved. The instructor can look at the test change anything that needs changing and choose if it's good enough to use.

Artificial Intelligence, Educational Assessment, Assessment Quality, Large Language Models, Bloom's Taxonomy, Automatic Question Generation

1. Chen, Y., Wu, C., Yan, S., Liu, P., & Xiao, Y. (2024), “Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics”, Vol.1, pp. 3138–3167. https://doi.org/10.18653/v1/2024.acllong.173 [Google Scholar] [Crossref]

2. Deroy, A., Maity, S., & Sarkar, S. (2025), “Proceedings of Large Foundation Models for Educational Assessment”, (Vol. 264, pp. 3–32). [Google Scholar] [Crossref]

3. Elkins, S., Kochmar, E., Cheung, J. C. K., & Serban, I. (2024), “Proceedings of the AAAI Conference on Artificial Intelligence”, Vol. 38(21), pp. 23084–23091. [Google Scholar] [Crossref]

4. Fu, W., Wei, B., Hu, J., Cai, Z., & Liu, J. (2024), “QGEval: Benchmarking multi-dimensional evaluation for question generation”, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 11783–11803). https://doi.org/10.18653/v1/2024.emnlp-main.658 [Google Scholar] [Crossref]

5. Gorgun, G., & Bulut, O. (2024), “Current evaluation methods are a bottleneck in automatic question generation”, Proceedings of the 2024 AAAI Conference on Artificial Intelligence (Vol. 257, pp. 3–8). [Google Scholar] [Crossref]

6. Isley, C., Gilbert, J., Kassos, E., Kocher, M., Nie, A., Brunskill, E., Domingue, B., Hofman, J., Legewie, J., Svoronos, T., Tuminelli, C., & Goel, S. (2026), “Assessing the quality of AI-generated exams: A large-scale field study”. Proceedings of the AAAI Conference on Artificial Intelligence, 40(45), 38626–38634. https://doi.org/10.1609/aaai.v40i45.41205 [Google Scholar] [Crossref]

7. Jung, J. Y., Tyack, L., & von Davier, M. (2025), “Towards the implementation of automated scoring in international large-scale assessments: Scalability and quality control” Computers and Education: Artificial Intelligence, 8, Article 100375. https://doi.org/10.1016/j.caeai.2025.100375 [Google Scholar] [Crossref]

8. Ling, J., & Afzaal, M. (2024). “Automatic question–answer pairs generation using pre-trained large language models in higher education” Computers and Education: Artificial Intelligence, 6, Article 100252. https://doi.org/10.1016/j.caeai.2024.100252 [Google Scholar] [Crossref]

9. Nikolovski, V., Trajanov, D., & Chorbev, I. (2025). “Advancing AI in higher education: A comparative study of large language model-based agents for exam question generation, improvement, and evaluation”. Algorithms, 18(3), Article 144.https://doi.org/10.3390/a18030144 [Google Scholar] [Crossref]

10. Sairaj, R. T., & Balasundaram, S. R. (2025), “Ensemble learning with RAG model to reduce redundant question topics in auto-generated exam questions”, Discover Computing, 28, 161. https://doi.org/10.1007/s10791-025-09683-2 [Google Scholar] [Crossref]

11. Scaria, N., Chenna, S. D., & Subramani, D. (2024), “How good are modern LLMs in generating relevant and high-quality questions at different Bloom’s skill levels”, Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024) (pp. 1–10). [Google Scholar] [Crossref]

12. Villegas-Ch, W., Gutierrez, R., García-Ortiz, J., & Guevara, V. (2025) “Explainable educational assistant integrated in Moodle: Automated semantic assessment and adaptive tutoring based on NLP and XAI”, Discover Artificial Intelligence, 5, Article 191. https://doi.org/10.1007/s44163-025-00438-y [Google Scholar] [Crossref]

13. Witsken, G., Crk, I., & Gultepe, E. (2025). “LLMs in the classroom: Outcomes and perceptions of questions written with the aid of AI”. Proceedings of the AAAI Conference on Artificial Intelligence, 39(26), 27698–27705. https://doi.org/10.1609/aaai.v39i26.34984 [Google Scholar] [Crossref]

14. Xia, Q., Weng, X., Ouyang, F., Lin, T. J., & Chiu, T. K. F. (2024). “A scoping review on how generative artificial intelligence transforms assessment in higher education”, International Journal of Educational Technology in Higher Education, 21, Article 40. https://doi.org/10.1186/s41239-024-00468-z [Google Scholar] [Crossref]

15. Yaacoub, A., Da-Rugna, J., & Assaghir, Z. (2025). “Assessing AI-generated questions’ alignment with cognitive frameworks in educational assessment”, International Journal of Computer Theory and Engineering, 17(3), 14–125 https://doi.org/10.7763/IJCTE.2025.V17.1374 [Google Scholar] [Crossref]

• Impact of AI Tools on Student Learning
• Repogent: An Autonomous Multi-Agent System for End-To-End Repository Maintenance
• Trust-Centric Federated Learning Using Blockchain-Enabled Smart Contracts: An Analysis of Data Privacy, Model Integrity, and Decentralized AI Governance
• Can AI Care? Rethinking Ethics, Clinical Integrity, and Professional Responsibility in the Digital Age of Mental Healthcare
• AI-Based Smart Irrigation System Using IoT and Machine Learning for Precision Agriculture