Teachers’ artificial intelligence (AI) literacy has become a critical determinant of educational digital transformation efficacy amid the deep integration of AI and pedagogy. Nevertheless, systematic, locally grounded, and psychometrically validated assessment instruments remain scarce. Persistent challenges, including cross-level educational misconceptions, the theory-practice gap in ethical evaluation, and disconnection from regional policy contexts – have constrained empirical progress in this domain. Drawing upon the digital transformation practices of an East Chinese province and its officially issued AI literacy frameworks for kindergarten through 12th grade (K-12) and higher education teachers, this study developed two parallel standardized scales. Both models exhibited a hierarchical second-order structure comprising value standards, practical literacy, and cognitive literacy. The K-12 scale yielded five first-order factors: classroom implementation, teaching-research inquiry, class management, cognitive level, and normative level. The higher education scale identified another five first-order factors, namely academic support, academic inquiry, classroom implementation, cognitive level, and normative level. Second-order factor analyses confirmed the theoretically posited hierarchical relationships. Each final scale retained 22 items, with cumulative variance explained rates of 79.30% and 76.90%, respectively. Confirmatory factor analyses demonstrated satisfactory fit comparative fit index (CFI=0.957) and 0.935 and root mean square error of approximation (RMSEA)=0.067 and 0.079. Convergent and discriminant validity met recommended thresholds, with overall Cronbach’s α coefficients of 0.948 and 0.936. Group difference analyses revealed a rigid age-related decline in AI competence, with teaching experience effects largely attributable to generational cohort differences, alongside systematic variations across subject area, professional role, and school location. The resulting scales exhibit robust psychometric properties and serve as scientific tools for regional policy evaluation, tiered professional development, and precision diagnosis of teacher AI literacy. The value-integrated design, which embeds ethical principles within specific application scenarios, offers a novel approach to reconciling the attitude-behavior measurement gap. Findings suggest that policy interventions should transition from generalized training to targeted empowerment, prioritizing subject-specific tool development, remedial programs for mid-career and senior teachers, and rural infrastructure enhancement.
References
[1] Mikeladze, T., Meijer, P. C., Verhoeff, R. P. (2024) A comprehensive exploration of artificial intelligence competence frameworks for educators: a critical review. European Journal of Education, 59(3), e12663.
[2] Ouyang, F., Jiao, P. (2021) Artificial intelligence in education: the three paradigms. Computers and Education: Artificial Intelligence, 2, 100020.
[3] Tsioukas, K., Kostas, A. (2026) Assessing teachers’ AI Literacy for lifelong learning: a systematic review and framework alignment. Artificial Intelligence Advances in Education, 1(1), 1-15.
[4] Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., Qiao, M. S. (2021) Conceptualizing AI literacy: an exploratory review. Computers and Education: Artificial Intelligence, 2, 100041.
[5] Lintner, T. (2024) A systematic review of AI literacy scales. Npj Science of Learning, 9(1), 50.
[6] Celik, I. (2023) Towards intelligent-TPACK: an empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior, 138, 107468.
[7] Chiu, T. K. (2025) Developing intelligent-TPACK (I-TPACK) framework from unpacking AI literacy and competency: implementation strategies and future research direction. Interactive Learning Environments, 33(7), 4189-4192.
[8] Williams, A. (2026) An integrated AI literacy framework for higher-education teachers: a critical review and conceptual synthesis. Cogent Education, 13(1), 2698137.
[9] Delcker, J., Heil, J., Ifenthaler, D. (2025) Evidence-based development of an instrument for the assessment of teachers’ self-perceptions of their artificial intelligence competence. Educational Technology Research and Development, 73(1), 115-133.
[10] Xia, Q., Yang, Y., Weng, X., Cilsalar-Sagnak, H., Cheng, W. K., Chiu, T. K. (2025) Generative artificial intelligence competency self-efficacy scale for teachers and researchers (GAICS-TR) in higher education. Journal of Computers in Education, 1-25.
[11] Zhong, B., Liu, X. (2025) Evaluating AI literacy of secondary students: framework and scale development. Computers & Education, 227, 105230.
[12] Nong, Y., Cui, J., He, Y., Zhang, P., Zhang, T. (2024) Development and validation of an AI literacy scale. Journal of Artificial Intelligence Research, 1(1), 17-26.
[13] Alqarni, A. (2025) Artificial intelligence-critical pedagogic: design and psychologic validation of a teacher-specific scale for enhancing critical thinking in classrooms. Journal of Computer Assisted Learning, 41(3), e70039.
[14] Carolus, A., Koch, M. J., Straka, S., Latoschik, M. E., Wienrich, C. (2023) MAILS-Meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change-and meta-competencies. Computers in Human Behavior: Artificial Humans, 1(2), 100014.
[15] Luo, S., Zou, D. (2025) K‐12 English Teachers’ ChatGPT‐based digital literacy: scale development and validation. European Journal of Education, 60(4), e70273.
[16] Taber, K. S. (2018) The use of Cronbach’s α when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273-1296.
[17] Dos Santos, P. M., Cirillo, M. Â. (2023) Construction of the average variance extracted index for construct validation in structural equation models with adaptive regressions. Communications in Statistics-Simulation and Computation, 52(4), 1639-1650.
[18] Li, Y., Cheng, H., Qin, Q. (2025) Evaluations and improvement methods of deep learning ability in blended learning. International Journal of e-Collaboration (IJeC), 21(1), 1-17.
Share and Cite
Xiang, Y., Zhan, F., Cai, Q. (2026) Creation and Validity and Reliability Testing of an AI Literacy Assessment Scale for Teachers – Based on a Dual-track Empirical Study of Zhejiang Province’s Primary and Secondary Schools and Higher Education Institutions. Global Education Bulletin, 3(3), 17-31. https://doi.org/10.71052/grb2025/NXGJ6555