Psychology

USING THE GENERAL DIAGNOSTIC MODEL TO MEASURE LEARNING AND CHANGE IN A LONGITUDINAL LARGE-SCALE ASSESSMENT

Description
USING THE GENERAL DIAGNOSTIC MODEL TO MEASURE LEARNING AND CHANGE IN A LONGITUDINAL LARGE-SCALE ASSESSMENT
Categories
Published
of 26
5
Categories
Published
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Similar Documents
Share
Transcript
  Listening. Learning. Leading. ®  Using the General Diagnostic Model to Measure Learning and Change in a Longitudinal Large-Scale Assessment Matthias von Davier Xueli Xu Claus H. Carstensen  July 2009  ETS RR-09-28  Research Report   July 2009 Using the General Diagnostic Model to Measure Learning and Change in a Longitudinal Large-Scale Assessment Matthias von Davier and Xueli Xu ETS, Princeton, New Jersey Claus H. Carstensen University of Bamberg, Germany    Copyright © 2009 by Educational Testing Service. All rights reserved. ETS, the ETS logo, and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS).   As part of its nonprofit mission, ETS conducts and disseminates the results of research to advance quality and equity in education and assessment for the benefit of ETS’s constituents and the field. To obtain a PDF or a print copy of a report, please visit: http://www.ets.org/research/contact.html   i Abstract A general diagnostic model was used to specify and compare two multidimensional item-response-theory (MIRT) models for longitudinal data: (a) a model that handles repeated measurements as multiple, correlated variables over time (Andersen, 1985) and (b) a model that assumes one common variable over time and additional orthogonal variables that quantify the change (Embretson, 1991). Using MIRT-model ability distributions that we allowed to vary across subpopulations defined by type of school, we also compared (a) a model with a single two-dimensional ability distribution to (b) extensions of the Andersen and Embretson approaches, assuming multiple populations. In addition, we specified a hierarchical-mixture distribution variant of the (Andersen and Embretson) MIRT models and compared it to all four of the above alternatives. These four types of models are growth-mixture models that allow for variation of the mixing proportions across clusters in a hierarchically organized sample. To illustrate the models presented in this paper, we applied the models to the PISA-I-Plus data for assessing learning and change across multiple subpopulations. The results indicate that (a) the Embretson-type model with multiple-group assumptions fits the data better than the other models investigated, and (b) the higher performing group shows larger improvement at Time Point 2 than the lower performing group. Key words: Item response theory, growth models, multidimensional IRT, longitudinal models, diagnostic models, large scale assessments  1 Introduction Measurement of change in student performance between testing occasions is a central topic in educational research and assessment (Fischer, 1995). Most research on such measurement has been conducted using small-scale data collections in fields such as developmental, educational, clinical, and applied psychology. Change across occasions can be meaningfully measured by focusing on either the group (Andersen, 1985; Andrade & Tavares, 2005; Fischer, 1973, 1976) or the individual (Embretson, 1991; Fischer, 1995).  Measuring Group Differences in Growth Fischer (1973, 1976) proposed a linear logistic test model (LLTM) based on the dichotomous Rasch model (Rasch, 1980). The Rasch model assumes that the probability of a correct response by person v  on item i  can be written as exp()(1)1exp() vivivi Px  θ β θ β  −= =+ − , in which v θ   is the person’s ability and i  β   is the item’s difficulty. The LLTM entails linear constraints across item parameters i  β   for the purpose of representing a structural relationship  between the difficulties of different item sets (here: items given at different points in time). The LLTM can be used to model growth (Fischer, 1995; Glück & Spiel, 1997) by specifying linear constraints that represent time-point effects, group effects, and other item features. For a set of  J   items given at T   time points in G  treatment groups, a group-specific model for growth can be specified in the LLTM using 1  piilll wc  β α  = = + ∑ , (1) in which the effects from α  1  to α   J   are the baseline item difficulties, α   J+1 is the effect of Time Point 2, α   J+T-1  is the effect of time point T  , α   J+T-1+1 is the effect of Group 2, and α   J+T-1+G-1 is the effect   of Group G . This example assumes only main effects for base item difficulties, time  points, and groups; Time Point 1 and Group 1 are the reference groups. A model with group-specific time-point effects is also easily specified within this framework. Note that the LLTM
Search
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks
SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!

x