0 Notes on Assessing Speaking

Title	:	Notes on Assessing Speaking
Authors	:	Barry O’Sullivan

With all language skill testing, it is necessary to clearly state the specifications before beginning to write the test (just as the previous sections have stated). In his very pragmatic ‘Understanding and Developing Language Tests’ Cyril Weir (1993) presents an approach to skills testing which calls for the identification of a series of language operations which the skill in question will entail. In addition to this list, Weir suggests that we need a set of conditions under which the task will be performed. Together these will allow the test writer to establish the parameters of the task or tasks to be employed in the test. Weir (2005) updated his original ideas and this later framework adds greatly to our conceptualisation of how all four skills can be tested and validated. In the following sections we will be looking at the testing of the four language skills in terms of Weir’s 2005 framework.

Download: Notes on Assessing Speaking [PDF]

Read more

0 An examination of the rating process in the revised IELTS Speaking Test

Title	:	An examination of the rating process in the revised IELTS Speaking Test
Authors	:	Annie Brown
	:	IELTS, Grant awarded Round 9, 2003

Abstract:

In 2001 the IELTS interview format and criteria were revised. A major change was the shift from a single global scale to a set of four analytic scales focusing on different aspects of oral proficiency. This study is concerned with the validity of the analytic rating scales. Through a combination of stimulated verbal report data and questionnaire data, this study seeks to analyse how IELTS examiners interpret the scales and how they apply them to samples of candidate performance.

This study addresses the following questions:

• How do examiners interpret the scales and what performance features are salient to their judgements?
• How easy is it for examiners to differentiate levels of performance in relation to each of the scales?
• What problems do examiners identify when attempting to make rating decisions?

Experienced IELTS examiners were asked to provide verbal reports after listening to, and rating a set of the interviews. Each examiner also completed a detailed questionnaire about their reactions to the approach to assessment. The data were transcribed, coded and analysed according to the research questions guiding the study.

Findings showed that, in contrast with their use of the earlier holistic scale (Brown, 2000), the examiners adhered closely to the descriptors when rating. In general, the examiners found the scales easy to interpret and apply. Problems that they identified related to overlap between the scales, a lack of clear distinction between levels, and the inference-based nature of some criteria. Examiners reported the most difficulty with the Fluency and Coherence scale, and there were concerns that the Pronunciation scale did not adequately differentiate levels of proficiency.

Download: An examination of the rating process in the revised IELTS Speaking Test [PDF]

Read more

0 Assessed Levels of Second Language Speaking Proficiency: How Distinct?

Title	:	Assessed Levels of Second Language Speaking Proficiency: How Distinct?
Authors	:	Noriko Iwashita, Annie Brown, Tim McNamara, Sally O’Hagan
	:	Applied Linguistics 29/1: 24–49

Abstract:

The study reported in this paper is an investigation of the nature of speaking proficiency in English as a second language in the context of a larger project to develop a rating scale for a new international test of English for Academic Purposes, TOEFL iBT (Brown et al. 2005). We report on a large-scale study of the relationship between detailed features of the spoken language produced by test-takers and holistic scores awarded by raters to these performances. Spoken test performances representing five different tasks and five different proficiency levels (200 performances in all) were analyzed using a range of measures of grammatical accuracy and complexity, vocabulary, pronunciation, and fluency. The results showed that features from each category helped distinguish overall levels of performance, with particular features of vocabulary and fluency having the strongest impact. Overall, the study contributes important insights into the nature of spoken proficiency as it develops and can be measured in rating scales for speaking, and has implications for methodological issues of the appropriateness of the use in language testing research contexts of measures developed in research on second language acquisition.

Download: Assessed Levels of Second Language Speaking Proficiency: How Distinct? [PDF]

Read more

0 Constructing rating scales for second language tests

Title	:	Constructing rating scales for second language tests
Authors	:	John A. Upshur and Carolyn E. Turner
	:	ELT Journal Volume 49/1 January 1995

Abstract:

Second language testing increasingly uses rated tasks in place of objectively scored items. Rating language performance is more demanding than scoring discrete-point tests. Lower reliability and validity for ratings are to be expected, especially in instructional settings where differences among students are small, and training for the use of rating scales is infrequent. The need for rating scales has not always led to effective and efficient scales, however.

Standard rating scales require the matching of examinee performance to a verbal description. An alternative type of scale consists of an empirically derived, ordered set of binary questions relating to boundaries between levels on the performance being evaluated. Rating depends on a series of decisions. Two scales of this type were developed to assess the grammatical accuracy and communicative effectiveness of a story-retell task. All scale categories were used, thus demonstrating that category range was appropriate to the learners. Reasons are offered to explain why this type of scale may ameliorate reliability and validity problems associated with standard rating scales.

Download: Constructing rating scales for second language tests [PDF]

Read more

0 TOEFL Practice Test

TOEFL Practice Test

Table of Contents

Section 1: Listening	3
Listening: Part A	3
Listening: Part B	7
Section 2: Structure	15
Section 3: Reading	18
Answer Key	30

 

Download TOEFL Practice Test (31 Halaman | 171 Kb)

Read more