Validity and reliability of a test used to assess university students' academic language proficiency

This article presents a test model developed to assess university students’ academic language proficiency. The context is that of Italian universities, which are required by law to assess first year students’ initial preparation in all the courses which do not have admission tests. This was made compulsory by law in 2004 and since then there has been a variety of interpretations on how to assess students’ initial preparation and how to follow up with remedial work aimed to fulfill the requirements set by each course of study. A group of researchers in educational linguistics from three Italian universities (Genova, Parma and Urbino) is conducting a study intended to create a cross-curricular assessment programme for language competences and communication skills, in particular reading and listening comprehension addressed to first year students. The framework adopted is based on language education principles and aims to highlight the strategic and cross-curricular role of students’ language competences and communication skills when entering university. The study can rely on the researchers’ decade-long experience in the fields of teaching, learning and assessing of Italian for academic purposes, CLIL methodology, and teaching and learning through ICT. In the academic year 2016-17, a piloting version of a test was created, which first year students from different courses in Parma and Urbino were required to take either online or on paper, after practicing the test format through an online mock test. In the following academic year, a revised version of the test was submitted to students from both universities. The data collected are analyzed in the present article and initial results of the ongoing project are illustrated. The aim is both to verify the validity and reliability of the test construct with the help of statistical tools and to reflect upon language competence issues. The quantitative analysis proposed is accompanied by a reflection on the aspects entailed in an approach focused on academic L1 proficiency of university students studying in different disciplinary fields.