Author(s): |
Yu, Guoxing |
Source: |
Language Assessment Quarterly, v10 n1 p96-109 2013 |
|
Pub Date: |
2013-00-00 |
Pub Type(s): |
Journal Articles; Reports - Descriptive |
Peer Reviewed: |
Yes |
|
|
|
Descriptors:
Documentation; Writing Tests; Language Usage; Language Tests; English (Second Language); Expertise; Educational Research; Evaluation Criteria
Abstract:
This article reports the lexical diversity of summaries written by experts and test takers in an empirical study and then interrogates the (in)congruity between the conceptualisations of "summary" and "summarize" in the literature of educational research and the operationalization of summarization tasks in three international English language tests in relation to their task directions and assessment criteria. These analyses show that summarization is not a uniform construct; rather, it is multidimensional in terms of its purposes, functions, and practices in real-life contexts, and as a consequence not particularly well operationalized in language tests and so in need of precise definitions for specific assessment contexts. This conclusion has implications for designing summarization tasks and assessing task performance. First, clear and transparent task directions are essential to ensure that test takers and evaluators share a common understanding of a test task; test takers need to be instructed what kind of summary they are expected to produce, particularly what information to include and to exclude. Second, because summarization may well be a unique type of writing process, it is important to employ parameters different from and additional to those for independent composition writing in order to measure the quality of a summary effectively. (Contains 4 tables and 3 footnotes.)
Note:The following two links
are not-applicable for text-based browsers or screen-reading software.
 Show
 Hide
Full Abstract
Related Items: Show Related Items
Full-Text Availability Options:
More Info:
 Help |
 Tutorial
Help Finding Full Text
|
More Info:
Help
Find in a Library
|
Publisher's website
|
Author(s): |
Deane, Paul |
Source: |
Assessing Writing, v18 n1 p7-24 Jan 2013 |
|
Pub Date: |
2013-01-00 |
Pub Type(s): |
Journal Articles; Reports - Evaluative |
Peer Reviewed: |
Yes |
|
|
|
Descriptors:
Scoring; Essays; Text Structure; Writing (Composition); Evaluation Criteria; Persuasive Discourse; Definitions; Evaluation Problems; Criticism; Writing Evaluation; Essay Tests; Computer Assisted Testing; Validity; Measurement; Psychometrics; Writing Skills
Abstract:
This paper examines the construct measured by automated essay scoring (AES) systems. AES systems measure features of the text structure, linguistic structure, and conventional print form of essays; as such, the systems primarily measure text production skills. In the current state-of-the-art, AES provide little direct evidence about such matters as strength of argumentation or rhetorical effectiveness. However, since there is a relationship between ease of text production and ability to mobilize cognitive resources to address rhetorical and conceptual problems, AES systems have strong correlations with overall performance and can effectively distinguish students in a position to apply a broader writing construct from those for whom text production constitutes a significant barrier to achievement. The paper begins by defining writing as a construct and then turns to the e-rater scoring engine as an example of AES state-of-the-art construct measurement. Common criticisms of AES are defined and explicated--fundamental objections to the construct measured, methods used to measure the construct, and technical inadequacies--and a direction for future research is identified through a socio-cognitive approach to AES. (Contains 4 figures.)
Note:The following two links
are not-applicable for text-based browsers or screen-reading software.
Show
Hide
Full Abstract
Related Items: Show Related Items
Full-Text Availability Options:
More Info:
Help |
Tutorial
Help Finding Full Text
|
More Info:
Help
Find in a Library
|
Publisher's website
|
|
|
Pub Date: |
2013-01-00 |
Pub Type(s): |
Journal Articles; Reports - Evaluative |
Peer Reviewed: |
Yes |
|
|
|
Descriptors:
Educational Testing; Guidelines; Scoring; Psychometrics; Evaluation Criteria; Program Descriptions; Vendors; Writing Evaluation; Essay Tests; Computer Assisted Testing; Program Evaluation; Evaluation Methods; Measurement
Abstract:
In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for future research. The description of e-rater includes a summary of characteristics of writing covered by e-rater, variations in modeling techniques available, and the regression-based model building procedure. The evaluation procedures cover multiple criteria, including association with human scores, distributional differences, subgroup differences and association with external variables of interest. Expected levels of performance for each evaluation are provided. We conclude that the "a priori" establishment of performance expectations and the evaluation of performance of e-rater against these expectations help to ensure that automated scoring provides a positive contribution to the large-scale assessment of writing. We call for continuing transparency in the design of automated scoring systems and clear and consistent expectations of performance of automated scoring before using such systems operationally. (Contains 1 figure and 1 table.)
Note:The following two links
are not-applicable for text-based browsers or screen-reading software.
Show
Hide
Full Abstract
Related Items: Show Related Items
Full-Text Availability Options:
More Info:
Help |
Tutorial
Help Finding Full Text
|
More Info:
Help
Find in a Library
|
Publisher's website
|
|
|
Pub Date: |
2013-03-00 |
Pub Type(s): |
Journal Articles; Reports - Research |
Peer Reviewed: |
Yes |
|
|
|
Descriptors:
Cognitive Processes; Information Literacy; Intervention; Teaching Methods; Undergraduate Students; Statistical Significance; Social Networks; Internet; Higher Education; Evaluation; Electronic Learning; Blended Learning; Evaluation Criteria; Control Groups; Experimental Groups
Abstract:
This research sought to determine whether a blended information literacy learning and teaching intervention could statistically significantly enhance undergraduates' information discernment compared to standard face-to-face delivery. A mixture of face-to-face and online activities, including online social media learning, was used. Three interventions were designed to develop the information literacies of first-year undergraduates studying Sport and Exercise at Staffordshire University and focused on one aspect of information literacy: the ability to evaluate source material effectively. An analysis was devised where written evaluations of found information for an assessment were converted into numerical scores and then measured statistically. This helped to evaluate the efficacy of the interventions and provided data for further analysis. An insight into how the information literacy pedagogical intervention and the cognitive processes involved in enabling participants to interact critically with information is provided. The intervention which incorporated social media learning proved to be the most successful learning and teaching approach. The data indicated that undergraduate students' information literacy can be developed. However, additional long-term data is required to establish whether this intervention would have a lasting impact. (Contains 2 figures and 2 tables.)
Note:The following two links
are not-applicable for text-based browsers or screen-reading software.
Show
Hide
Full Abstract
Related Items: Show Related Items
Full-Text Availability Options:
More Info:
Help |
Tutorial
Help Finding Full Text
|
More Info:
Help
Find in a Library
|
Publisher's website
|
Author(s): |
Canales, Genevieve |
Source: |
Journal of Mixed Methods Research, v7 n1 p6-21 Jan 2013 |
|
Pub Date: |
2013-01-00 |
Pub Type(s): |
Journal Articles; Reports - Descriptive |
Peer Reviewed: |
Yes |
|
|
|
Descriptors:
Mixed Methods Research; Check Lists; Psychological Studies; Mexican Americans; Evaluation Criteria; Feminism; Psychology; World Views; Cultural Awareness; Rhetoric; Social Justice; Research Design; Data Collection
Abstract:
This is a description of the creation of a research methods tool, the "Transformative, Mixed Methods Checklist for Psychological Research With Mexican Americans." For conducting literature reviews of and planning mixed methods studies with Mexican Americans, it contains evaluative criteria calling for transformative mixed methods, perspectives from Chicana/o (Mestiza/o) psychology, and Baca-Zinn and Dill's Chicana multiracial feminism. It is useful to researchers, journal editors, teachers, and students in psychology and other social sciences. It may serve as a template in the development of comparable checklists for critiquing mixed methods studies with other cultural groups, including African Americans, American Indians, and Asian Americans. (Contains 1 table.)
Note:The following two links
are not-applicable for text-based browsers or screen-reading software.
Show
Hide
Full Abstract
Related Items: Show Related Items
Full-Text Availability Options:
More Info:
Help |
Tutorial
Help Finding Full Text
|
More Info:
Help
Find in a Library
|
Publisher's website
|
|
|
Pub Date: |
2013-01-00 |
Pub Type(s): |
Journal Articles; Reports - Research |
Peer Reviewed: |
|
|
|
|
Descriptors:
Foreign Countries; Gifted; Student Characteristics; Short Term Memory; Grade 7; Middle School Students; Factor Analysis; Tests; Recognition (Psychology); Recall (Psychology); Evaluation Criteria; Cognitive Processes
Abstract:
The outstandingly able learner has been conceptualised, in terms of test and examination performance, as the learner showing superior academic performance which is markedly better than that of peers and in ways regarded as of value by wider society. In Kuwait, such superior examination performance leads to a classification regarded as being "gifted". This study looks at the inter-correlations between performance in various subjects in examinations and then considers how examination performance correlates with measures of working memory capacity, extent of field dependency, extent of divergency and visual-spatial abilities. A very large sample of grade 7 Kuwaiti students (aged approximately 13) was involved, the sample being selected in such a way that it contained a high proportion of those regarded as "gifted" under the procedures used in Kuwait. While specific learner characteristics have been related to examination performance, this study brings four different characteristics together to gain a picture of the way these characteristics may be seen in those who perform extremely well in examinations. Principal components analysis using varimax rotation, was used to look at the examination data and one factor accounted for 87% of the variance. A consideration of the examination papers led to the conclusion that the national examinations tested only recall-recognition. It was also found that those who performed best in all six subjects tended to be those who are highly divergent and strongly visual-spatial as well as those tending to have higher working memory capacities and being more field independent. The inter-correlations between the various learner characteristics are explained in terms of the way the brain is known to process information. The implications of the findings for assessment and for the way high ability is considered are discussed. (Contains 11 tables and 2 figures.)
Note:The following two links
are not-applicable for text-based browsers or screen-reading software.
Show
Hide
Full Abstract
Related Items: Show Related Items
Full-Text Availability Options:
 ERIC
Full Text (151K)
|
More Info:
Help
Find in a Library
|
|
|
Pub Date: |
2013-01-00 |
Pub Type(s): |
Reports - Evaluative |
Peer Reviewed: |
|
|
|
|
Descriptors:
Evidence; Competition; School Choice; School District Size; Educational Policy; Policy Analysis; Measurement Techniques; Evaluation Criteria; School Districts; Computer Software; Program Evaluation; School Effectiveness; Taxes; Educational Finance; Free Enterprise System; Parent Attitudes; Selection Criteria; Evaluation Problems; Evaluation Methods
Abstract:
In this report the Brown Center on Education Policy at the Brookings Institution presents the results of a self-developed Education Choice and Competition Index (ECCI) along with an interactive application that grades large school districts according to the ECCI. The index is composed of 13 pro choice criteria. The authors present the ECCI as a new approach to education policy, but the ideas are not new. The report repeats many of the same arguments and logic found in other pro-market publications over the past 25 years. The sole unique recommendation is to permit "popularity" to serve as an adequate standard to judge school quality and a sufficient criterion by which to direct taxpayer dollars. Yet, this recommendation also hearkens back to previous calls, common in the 1990's, for unregulated school choice policies. The ECCI assumes benefits of market competition, but these benefits depend on how parents choose schools, and the report includes no research on how parents make school choice decisions. It is also devoid of any evidence that higher scores on the ECCI are related to the outcomes that the authors predict. In fact, the only large district with an "A" rating per the ECCI received a "D" according to its state accountability system. In essence, the report reads like an essay in support of free-market choice policies. Because of the dated assumptions and the subjective nature of the index, the study and its rankings fall short of being an effective policy tool. Notes and references are included.
Note:The following two links
are not-applicable for text-based browsers or screen-reading software.
Show
Hide
Full Abstract
Related Items: Show Related Items
Full-Text Availability Options:
ERIC
Full Text (615K)
|
|
