An Introduction to Evaluation Terminology

An Introduction of Evaluation Terminology provides meaning and scope to the terms used in various steps of evaluation. Evaluation of any learning outcome, be it a programme, a methodology, materials or media, requires a variety of techniques. In language learning, evaluation of Competence and performance uses technologies encompassing both language structure and language use, Choice of tools, validation of items, selecting a suitable format, Providing instruction to candidates, deciding the techniques for administering the tests, the duration, marking scheme/scoring procedure, obtaining information/data for qualitative analysis | and interpretation of results, Making value judgements, etc. are steps for effective evaluation. All these steps utilize concepts which are interdisciplinary in nature. At a time when the country is concerned about examination reform and objective evaluation of abilities and achievements, this book is expected to be a contribution to the growing discipline. I hope, it will be of use to all those desiring to work in this field. Dr. Pon Subbiah deserves congratulations for Preparing this book.

Teaching, learning, and evaluation are interdependent activities. The absence of any one of these will have an adverse effect on the other two. However, in practice, the amount of emphasis given to the first two has not been given to latter even though it is a well known fact that evaluation is a powerful means of determining the quality of a teaching programme. Perhaps, because of this reason, the evaluation system being followed in language education is under severe criticism from all a quarters. Where there is no evaluation, no quality could be assured. More so, evaluation in language education as it exists today suffers from a number of inadequacies and imperfections. Even basic concepts of evaluation have not been properly understood by many of the practitioners. Valuation is confused with evaluation, measurement is often mistakenly interchanged with assessment, and usage of the term tool is mixed up with that of a scale. What is expected of a student evaluation is not distinguished from that of a teacher evaluation as often the same parameters are used. So is the case with differentiating a text book from that of a syllabus or curriculum, or separating out process from that of a product in a programme or scheme. Thus, the question of what to evaluate is not generally found to have been differentiated from that of how and why. The point which is generally missed out is the lack of integration of all these three aspects. More particularly, evaluation in language education is not related to the ultimate objective of education, often identified as the all-round growth of a learner both in terms of scholastic achievement and non-scholastic performance. This is the reason why the certificates issued by the organizations are attributed to have no relevance to real life situations thereby making the whole system of language education a mockery. The CIIL being the apex body for Indian languages has, therefore, made an earnest attempt to overcome all these impediments by initiating R & D work in evaluation during the mid-1980s, after making an informal survey by visiting various national level organizations. It was found that Indian languages were generally given the least priority with hardly any work on evaluation done in these languages. Being the only organisation in the country which has got a full time research faculty of evaluation for Indian languages, the CIIL had to assume greater responsibility for creating not only the basic reference materials in this specialized a area but also trained manpower in each of the Indian languages. However, achieving this to even a reasonable extent was not an easy task considering the number of languages of this country and also those who are involved in teaching them. The process of identifying the areas of training, deciding the level of requirements, prioritising the languages for coverage, etc., is a cumbersome and time-consuming exercise. Despite this, the faculty (a lone member up to 1994) took up this venture as a challenge, provided on-the-job-training to a group of researchers, and practitioners, and produced indigenously more than 50 volumes of materials covering different branches of evaluation under three broad categories, viz., a. Evaluation & Testing: General References, b. MILES Content / Method / Purpose Resources, and special Purpose Assessment Tools.

The materials included under the General Reference Material Series are all prepared in English and the sequence of their production was planned in such a way that the content of the first one is useful to understand the second one better and similarly the second one naturally leads to the third, and so on. It may be seen that the above is reflected in the following sequence. The first one in this series was Introduction to Evaluation Terminology. (The experimental edition of this work was brought out during 1990 followed by 2™ edition during 2002, and it appears now as an enlarged version of the earlier document. It provides the meaning and scope for about one thousand terms with illustrative examples). It was followed by Descriptive Bibliography on Evaluation and Testing which consists of about 600 annotated summaries of books and articles on evaluation. The third one is titled Assessment in Language and Literature: A Conceptual Frame work. Being a comprehensive work in the area of language evaluation, this work highlights various concepts of evaluation and their interrelatedness by citing examples from language and literature. It also consists. of the General Frames of Reference for language, literature, and personality for preparing tests of different types. The practical realization of the theoretical concepts explained here has been made separately in different Indian languages and the some have been brought out under the MILES Base Material series. The fourth one is instruments of Evaluation: Item Types and Techniques. It provides techniques of writing various types of question items under objective and subjective category meant for the assessment of both scholastic and non-scholastic abilities. The last one in this series is Objective Writing and Restructuring the Syllabi of Language and Literature Courses. It provides the mechanism of outlining the course objectives for the already available traditionally prepared syllabi of language and literature courses by getting them appropriately restructured.

Each of the above materials has been used as course materials in various workshops of test preparation in different Indian languages. As a result, all of them have been improved upon for their comprehensiveness, and practical utility on the basis of the feedback obtained. These volumes are the outcome of the R & D work undertaken by Pon Subbiah on various branches of evaluation in the light of his experience as a member of Tamil faculty and subsequently as head of the Centre for Testing & Evaluation in planning and execution of various projects and programmes in different Indian languages meant for different agencies including SSCs, PSCs, and IBPS. Though these materials had been gradually developed and used in different forms over a period of 15 years, they are brought out in full form only now.

I am sure the students, teachers and evaluators in this part of the world will find this series of materials very useful.

Everyone involved in the educational process is expected to be aware of certain key concepts in evaluation and the terminology that represents them. This volume is brought out in partial fulfilment of this requirement. It has the following salient features:

(a) consists of about 1000 entries, each one of them deals with an evaluation concept expressed by a technical term.

(b) the criteria used for identifying the terms for inclusion were decided on the basis of contents, methods, and purposes of evaluation.

(c) content inputs have been enriched with the conceptual explanation, and illustrative examples wherever necessary.

(d) terms of synonymous concepts have been supplemented with cross-references; and conceptual deviations, if any, are cited from authoritative sources.

(e) for certain specified entries, additional figures and tables of computations have been incorporated for further clarity.

(f) worked-out examples in the appendices have been rearranged and. presented under different headings to enhance conceptual clarity.

(g) simplified methods (rules of thumb) of item and test analysis too have been provided for the benefit of those who are not familiar with statistics.

(h) in order to make the material more user-friendly, lists of abbreviations, figures, tables, and terms covered are included in the preliminary pages.

(i) certain titles which were left out earlier have been added to the bibliographical resources consisting books, articles, and journals.

Though the work is intended primarily for the benefit of language teachers, the coverage has been widened in such a way that the teachers of other disciplines can also make use of it. Many of the terms that represent highly technical concepts such as heteroscedasticity, hetero trait-mono method validity, three- parameter (logistic) model, etc., continue to find a place in this version too. Those terms are included to make the readers aware of the availability of such advanced techniques for analysing the item validity. On the other hand, several apparently more simple common terms, such as question, test, etc., have also been included and explained with a view to clarify the ambiguity involved in their meaning and scope. Some of the entries are in phrasal forms like role of evaluation, purposes of test / evaluation, tools of evaluation, types of scale, and so on. Their inclusion in that form is intended to provide an overall idea about the concepts of — evaluation. The reader would otherwise have had to search for theory books.

Another set of forms included here are behavioural objectives, educational objectives, instructional objectives, and so on. Though they are only the extended forms of the common concept of objectives, each one of them is dealt with individually under separate entries in view of the subtle differences noticed among them. This would enable the reader to have a better understanding of the built in relation among them. Certain concepts are often referred to by more than one synonymous forms, such as objective type question, close ended question, fixed response item, and so on. For convenience, each one of these phrasal forms too is dealt with under separate entries with necessary cross-references. There are certain concepts having the same explanations already provided under some other entries. Such terms are presented with a ‘see...’ notation in parenthesis, for example ‘Experience Index [see: Aptitude Index]’. Elaborate explanations with illustrative examples have been provided for the terms like profile, scaling, stained, etc., in view of their frequent practical utility. For the description of certain concepts such as scale, & scaling authoritative sources have been quoted. ‘Wherever the differences in views are noticed about the concepts such as fests, examinations, the lists of the same are provided with the following sequence of information viz., the name(s) of the author(s), the book / work from where the quotations are taken, the year of publication, page number and so on. In case there are more publications by the same author in the same year their chronological order has been indicated as a, b, c, etc., following the year of publication. The inclusion of certain terms which are generally non technical such as ideal, issues, language, literature, theme, etc., 18 intended to highlight their specialised usages in the area of evaluation.

This volume is a bi-product of the encounters I had with terminology during my studies on evaluation methodology and examinations, and also during the course of my work in various branches of evaluation in the last two decades. As the concepts conveyed by many of these terms and also their usage in different contexts are quite confusing and overlapping, the need for explaining them for the benefit of the practitioners, especially the teachers of Indian languages and literature with whom I have been working, was immensely felt. Accordingly, the material has been prepared with special care.

Comprehensive coverage in this type of work is not always possible. As technology grows, everything changes – be it a method, medium, or material. Hence, time and again, it needs updated and improved to meet the growing needs. It is hoped that this volume will serve as a good reference material to all those who are considered to be indispensable with the area of evaluation, like teachers, question setters, examiners, and teacher-learners of staff training colleges, besides those in the academic wings of various boards of examination and agencies of recruitment.

**Contents and Sample Pages**