Connecting TOEFL Speaking to Speaking at University

Speaking is critical to success in university studies. Undergraduate and graduate students alike use spoken language to handle a range of tasks on and off campus. Graduate students also frequently teach or tutor undergraduate students or work in collaborative research labs.

The updated TOEFL iBT features two speaking tasks: Listen & Repeat and Take an Interview. Listen & Repeat requires examinees to listen to a sentence and repeat it as accurately as possible, while the Virtual Interview involves open-ended responses to a series of questions related to academic life.

We recently conducted research on these tasks here at the University of Hawaiʻi at Mānoa. As faculty who teach, mentor, and supervise international students, we know how important speaking skills are for student success. As researchers, we are keenly interested in how strongly test tasks, like those featured on TOEFL exams, relate to real-world speaking capabilities.

Language tests are used to make decisions about real people. When using language tests in higher education admissions, the chief concern is making sure a prospective student will be able to handle the linguistic demands of English-medium study. So, when a test score is used in making an admissions decision, we assume the test score tells us something about how well a student will be able to use English at the university.

Test scores are based on test performances. For speaking, student performances can include highly constrained performances, like Read Aloud and Listen & Repeat tasks where what a student should say is presented in written/aural form, as well as more spontaneous, creative performances, like responses to questions in the Virtual Interview task.

To increase confidence in what test scores tell us about real-world capabilities, examining the qualities of test-takers’ performances on test tasks and tasks outside of the test holds value. Essentially, we wanted to know that how fluently and accurately students speak on the test is indicative of how fluently and accurately they can produce language in real-world communication.

We recruited 149 international students, all users of English as a second language. To ensure we would represent the broad range of proficiency, we included students who were engaged in full-time intensive English as a second language programs, students who were concurrently enrolled in university coursework and academic English courses, and fully matriculated undergraduate and graduate students.

Each student completed speaking tasks administered on the operational ETS testing platform. ETS provided us with their speaking scores and audio recordings. We also had each participant complete two academic speaking tasks in a face-to-face lab setting. These tasks involved (1) describing a graph and (2) listening to an 8-minute lecture on bilingualism and then responding to questions from the researcher.

Students were also asked to record and submit a speaking assignment connected to their academic programs. Ninety-nine students did so, with 65 submissions being formal presentations.

We analyzed all speaking samples for a range of complexity, accuracy, and fluency features related to grammar, vocabulary, and pronunciation. We also had experienced teachers of English as a second language rate the communicative effectiveness of the non-test speech samples.

When comparing individuals’ task scores to ratings of speaking from the two lab tasks and the course-based speaking task, we found strong, positive correlations (Table 1).

Table 1.

Correlations between TOEFL speaking scores and non-test speaking ratings.

Note. All correlations were statistically significant at the p < .001 level.

For linguistic features, the results were more complicated. For one, some linguistic characteristics like lexical diversity (diversity of vocabulary use) were not relevant to the Listen & Repeat task which featured highly constrained responses.

Some linguistic features had strong, positive correlations across TOEFL speaking tasks and non-test tasks, such as those related to fluency and accuracy of spoken production. Other linguistic features had small to moderate positive correlations, like complexity features (e.g., the number of words or clauses per spoken sentence). Other features had little association across tasks.

In sum, it appeared that aspects of speech production related to general ability to process English, such as fluency and accuracy of grammar, vocabulary, and pronunciation, were strongly correlated across test and non-test performances. Aspects of speech that were more influenced by task demands, such as grammar and vocabulary choices, were less strongly correlated. 

Our research strongly suggests that students who earn high scores on tasks like Listen & Repeat and Take an Interview are able to communicate effectively outside of a test context. This finding held for both lab-based, simulated academic tasks (Graph Description and Lecture Response) and for authentic presentations related to students’ courses.

While the Take an Interview task, which features more spontaneous and creative language production from test takers, tended to have the strongest correlations with non-test tasks, the Listen & Repeat task had similarly strong correlations in most cases. Notably, it seemed to effectively capture the fluency and accuracy related aspects of test-takers’ speaking ability that are at play in all speaking contexts.

Overall, we found that both item types were good indicators of how well students performed on simulated and actual academic speaking tasks.

Daniel R. Isbell is an Associate Professor in the Department of Second Language Studies at the University of Hawaiʻi at Mānoa, where he conducts research and teaches courses focused on language assessment.

Dustin Crowther is an Associate Professor in the Department of Second Language Studies at University of Hawaiʻi at Mānoa. He conducts research into the intelligibility of second language speech, with a primary focus on Global Englishes.