Reflection on Validity

Damian Francis

In my almost 12 years as an English language teacher, I have had the responsibility to elaborate quite a few tests.  When I first had the opportunity to make exams I did not take too much into account the requirements that I was supposed to consider. I had no idea of what validity was. I only made items from the contents I had already given to my students. I did not take into consideration that a language test needs to be elaborated taking into account some specific aspects. For example, at the moment we make a test, we need to be clear what validity is. To me, a test is valid when it gauges what we intend to test.

 When it comes to language testing, validity is an important element. Before making an exam, test crafters have to analyze the following aspects: first, they have to consider what specific competence they want to test. According to my experience, after reflecting on the students’ exams results, most of the time, one of the reasons why test takers fail a test is because it is not elaborated properly. Sometimes the intention is to assess language proficiency; but the way it is done is by applying a written test, mainly, focused on grammatical aspects. In a situation like this, the outcomes are not reliable because a proficiency test must be applied providing the test takers a context in which he or she uses the language, for example an interview. This way the test taker demonstrates how fluent he or she speaks the language, how knowledgeable of the native-like way of speaking the target language he/she is, and how clear this person is able to communicate the ideas, and feelings.

One more aspect that has to be taken into account is the vocabulary used in written tests. Sometimes teachers teach their lessons based on the vocabulary and situations provided by the text books they use, but at the moment test crafters make the test, they just focus on the learning goals, so it is quite common to see frustrated students that have difficulty for understanding the test, when something like this happens, the test does not have validity.

A third component that has to be taken into consideration is the score. Test crafters, and teachers have to be careful with score. This is an aspect that has to be analyzed by teachers because sometimes some themes are assigned more points than others. Test crafters have to ask themselves what criterion of assigning more point to a certain theme is. In standard tests, such as the TKT,  the themes are of equal value. From my point of view, In a language proficiency test, the most important aspects are the listening comprehension and the speaking ones, but in an accuracy written test the most important aspect is different.

This analysis reminds me when I was an English language student. I used to take two different exams; the first one was the midterm and a final test. One aspect that really caught my eye was that the approach used by the teacher and the one the exams had were totally different. I remember that my teachers did not explain grammar deductively.

 On one hand, they focused on developing communicative competence by giving students significant input. On the other hand, the exams were grammar oriented. Some institutions not even took into consideration testing the listening skills, and many tests had irrelevant content, such as verb conjugations, vocabulary translation, and make sentences commands.

During the process of being involved with this subject oriented in learning how to assess in a professional way, I have reflected on the requirements needed to elaborate a test. In language tests, validity is carried out at the extent a written test result turns out similar to what the test taker is able to perform in everyday situations.

From my point of view, being knowledgeable about test validity is as important as having a good teaching methodology because the assessment purpose is to determine what the language competences are. A test crafter has to be able to make a good diagnostic test. In my experience as a teacher, language trainer and supervisor at the Ministry of Education in Dominican Republic, I have noticed that quite a few teachers have difficulty to elaborate a diagnostic test at the beginning of each and every back to school week. If a teacher is not able to identify the students’ needs properly, so they will not be able to really help them during the process. The application of a valid test allows teachers to make a good planning to satisfy students’ necessities.

Test crafters must have the competence to elaborate a placement test. Considering my teaching and supervising experiences, I have noticed that many people are given a high intermediate level of language proficiency, when they, in fact, are in a basic one. Mistakes like this are due to the validity ignorance of the test crafter. I used to work for a language school that allowed students to take a placement test in order to send them to the level they were supposed to be. I usually had complaints because some students did not belong to the level the test assigned them.

To me, a good placement test must be elaborated taking into account the listening, the speaking, the reading and the writing skills. It also has to be based on the specific competence that beginners, basic, intermediate, high intermediate and advanced students are supposed to own.

When it comes to proficiency test, I think that it would be a good idea to reflect on it. Many language institutions misconceive the term proficiency test, with the placement test, in fact in the application; they get confused with these two different tests. Some language schools think that they give a proficiency test when students take a placement test. But from my point of view, a proficiency test could never be conceived as a long written test based mainly on grammar structure knowledge.

A language proficiency test assesses the language competence of the test taker during performing the language. a person demonstrates how competent he or she is at the moment he/she uses the language. The language proficiency is assessed considering how clear a person communicates the ideas, the knowledge of the target language lexis, the pronunciation, and the usage.