Assessing Mastery of Literature (Volume 3)

Teaching, learning, and evaluation are interdependent activities. The absence of any. one of these will have an adverse effect on the other two. However, in practice, the amount of emphasis given to the first two has not been given to the latter, even though it is a well known fact that evaluation is a powerful means of determining the quality of a teaching programme. Perhaps, because of this reason, the evaluation system being followed in language education is under severe criticism from all quarters. Where there is no evaluation, no quality could be assured. More so, evaluation in language education as it exists today suffers from a number of inadequacies and imperfections. Even basic concepts of evaluation have not been properly understood by many of the practitioners. Valuation is confused with evaluation; measurement is often mistaken for assessment: and usage of the term fool is mixed up with that of a scale. What is expected of a student evaluation is not distinguished from that of a teacher evaluation as often the same parameters are used. So is the case with differentiating a text book from a syllabus or curriculum; or separating the process from a product in a programme or scheme. Thus, the question of what to evaluate is not generally found to have been differentiated from those of how and why. The point which is generally missed out is the lack of integration of all these three aspects.- More particularly, evaluation in language education is not related to the ultimate objective of education, i.e., the all-round growth of a learner in terms of both scholastic achievement and non-scholastic performance. This is the reason why the certificates issued by organizations are said to have no relevance to real life situations thereby making the whole system of language education a mockery.

The Central Institute of Indian Languages (CIIL) being an apex body for Indian languages has, therefore, made an earnest attempt not only to overcome all these impediments but also to cater to the requirements of the country and abroad with reference to these languages. In the mid-80s, the Union Public Service Commission wanted suggestions for qualitative improvement of their civil service question papers in 13 languages. Staff Selection Commission wanted a list of potential item writers in each of the major Indian languages to develop tests for _ recruiting Group III employees. Karnataka Public Service Commission approached the Institute for ready-made tests in 9 languages for recruiting their clerical staff. Institute of Banking Personnel Management got the CIIL’s help in developing tests as per their specification in the Southern and North-Eastern languages for the recruitment of Probationary Officers and Clerk-cum-Cashiers. An Non Resident Indian agency of USA wanted tests in Punjabi to be administered in L3 contexts at the middle school level. Also an Autonomous Institute of Higher Education in Tamilnadu wanted to have the syllabi of all their courses restructured in the context of evaluation. Further, the National Council of Educational Research and Training wanted to have the inputs of evaluation for their special education primers. All these are only a few samples in addition to our own requirements of L2, tests for the 13 major Indian languages being taught in the Regional Language Centres. Thus, there were different agencies, different languages, different purposes, different levels, different contexts, and also different requirements. Some of such requirements are achievement based, some are intelligence and aptitude based, and some are general proficiency based. Also some of them are syllabus bound and a few are textbook bound. Some are content free and some context free. There were also requirements for different purposes such as, admission, research, certification, recruitment and so on. Thus on many occasions, the Institute was in a fix when faced with a flood of requirements from different quarters. Such multidirectional pressures have resulted in the development of Multipurpose Indian Language Evaluation System (MILES).

As is only well-known, the constraints encountered here are too many; they include the limits imposed by conversational contexts, text-types, or media of operation of tests. What one often fails to understand is that the academic constraints and administrative complexities involved in this kind of work. While setting up academic parameters, or while collating a large number of items for use as instruments of measurement, the amount of care to be taken to achieve academically acceptable authentic data is often impossible to imagine unless one has been involved in large-scale testing operations. Working in Indian languages on a specialized area like this is generally a difficult task due to the lack of well defined vocabulary to represent specialized concepts, non availability of appropriate reference materials and scholars with adequate background of interdisciplinary areas in the languages concerned and also due to the lack of proper softwares. There are innumerable problems at every stage. Notwithstanding these problems, the process of giving practical realization in Indian languages is continued with a hope that all. such problems would be overcome in due course.

Academic resources developed as MILES Base Materials are broadly categorized into three series viz., Content, Method, and Purpose. The 15 volumes, 5 each in Kannada, Malayalam, and Tamil, being broughtout at present, belong to the ‘content’ series. Each of these volumes is equipped with the details of content components and their dimensions along with the item models developed there on. The computer number codes assigned to them and the IFC parameters outlined would help ready access for assessing mastery of Language & Literature and subsequently the corresponding personality. The inter-disciplinarity of the concepts involved in developing the framework, the rationality behind the evolution of this approach, the enormity of the task involved in developing MILES, etc., are explained in the ‘Introductory Part’. The point as to who will derive benefit out of these volumes has been provided in the ‘Preface’. Base materials in respect of other two series ‘method’ and ‘purpose’, are in different stages of their progress. The massive nature of this work may be seen from the number of volumes proposed to be brought out under the scheme of MILES Base Materials, in various Indian Languages i.e., 252 volumes 14 per language for 18 major Indian languages (5 each under ‘Content’ and ‘Method’, and 4 under ‘Purpose’ Series).

I am sure that the materials developed under this massive scheme will have a tremendous effect on all those concerning the language and literature education as the initiatives taken by the institute is aimed at increasing the objectivity of accountability not only for the purpose of testing but also for the teaching of language and literature in the Indian context.

When modern linguistics came to the educational scene in India after Independence, one of the promises was that it would help improve teaching of the Indian languages and thus the linguistic skills of the students for building the new nation. After about half a century of educational planning and innovations, there is a sad realization that the promise has not been fulfilled. It is based on subjective evaluation of the performance of the students after School and on the examination results in the school. Though this evaluation is true if not totally reliable and objective, tools are necessary to scientifically establish this observation.

Acquisition of language skills in formal education depends on the methodology of teaching and the materials used for teaching. Language teaching in India has been content teaching and information dissemination and not skill development. The Central Institute of Indian Languages in the last quarter of a century has tried to change this fundamental misperception of the goals of language teaching through orientation to language teachers and material producers and through preparation of model materials. The change is perceptible in the curriculum and learning materials to some extent and in the teaching methodology to a lesser extent. The examination system , however, still by and large remains a test for information acquisition and recall. This nullifies whatever little gain has been made in methods and materials, because, as it is well known, Indian education is examination oriented.

Language skills are increasingly tested in special examinations for various kinds of employment and in the interviews for this purpose. The recruiting agencies like to have a test which is reliable and valid and is also global in the sense that is not specific to a particular syllabus or text book. The need has increased in the recent times because of variability in the examination marks of the different boards of education and universities and their lack of reliability.

To meet the above purposes, the Central Institute of Indian Languages initiated a massive project to construct multipurpose tests in Indian languages. It was planned to develop them through workshops involving college level language teachers. It was soon realized that the difficulties of the task was underestimated: the teachers were not familiar with the concepts and skills of test construction. They had to be given hands on training. Secondly, the scope of testing in all its dimensions as proposed by educationists, particularly Bloom, et al., for any subject, -was for the first time extended to language and literature as the project wanted to be comprehensive. It was an intimidating task and still problems remain in the identification of right items for the right tests and in the differentiation of different objectives of the test from the language point of view. Thirdly, the problem was deciding on global learning levels irrespective of the school System, nature of the schools in terms of their resources, Socio-economic ; differences in the background of the students etc,

It was therefore decided to write a detailed introduction on the concepts and methods of testing, organisation of language knowledge and the organisation of the test bank. This introduction has become a book itself and will be the first in the series of books of actual tests in Indian languages.

Given the difficulties mentioned above, these tests are a test themselves to be tested in the classroom and improved upon before giving to the country as the standardized tests that could be used without any bias.

The teachers are encouraged to construct tests themselves following the format and the models given in the book and send them to the Institute to be computerized after verification for use by anyone to test any language skill and knowledge for any purpose. _ They are also encouraged to report to the Institute the problems they face in administering the tests, particularly their difficulty level and comprehensibility of instruction.

MILES is the first mile stone to develop the linguistic resources of the students as an important part of human resource development. And we know that we have yet many miles to go.

In partial fulfillment of the objective of promoting Indian Languages and also to meet the requirements of language education in the country and abroad, Research and Development (R & D) in the specialised area of evaluation was initiated sometime during 1980s. As a result, basic reference materials V1Z., Introduction to Evaluation Terminology, and a Descriptive Bibliography of Evaluation and Testing (with reference to language education) were brought out. They were followed by the formulation of a Conceptual Framework for systematically assessing the mastery of language and literature, and also the personality acquired as a result of learning these disciplines as a subject of Study. The propositions made through this framework are centered around the content, method, and purpose of evaluation, the concepts of which have subsequently been used as a basis for giving practical realisation in Indian: languages. Consequent upon this, several such practical realisation volumes of content resources required for the construction of a variety of tests have been developed. The overall purpose of these volumes is to highlight the concept of content for assessment in language and literature. The details of the volumes being brought out in each of the six Indian languages viz., Assamese, Bengali, Kannada, Malayalam, Tamil, and Urdu at the first instance, are as follows :

Volume 1. Assessing Mastery & Personality : General Frame of Reference

Volume 2. Content Resources for Assessing Mastery of Language

Volume 3. Content Resources for Assessing Mastery of Literature

Volume 4. Content Resources for Assessing Personality through Language

Volume 5. Content Resources for Assessing Personality through Literature

The introductory part which is common to all these volumes consists of a brief discussion about the conceptual framework and then the details of the project of developing Multipurpose Indian Language Evaluation System (MILES). The subsequent parts in the respective volume provide the details of content-resources required for the construction of a variety of tests for different purposes and also for different levels. T hus, the objective of Volume 1 is to provide ‘Indian language equivalents’ for the English terms that are encountered while dealing with the concept of contents in language and literature education. The objective of Volume 2 is to exemplify the test contents through scores of illustrative examples. Each of these examples is equipped with a number of academic and statistical resources required for the construction of tests for assessing ‘mastery of language’ at different levels. Similarly, Volume 3 is to exemplify the contents required for the construction of tests for assessing ‘mastery of literature’. Volume 4 exemplifies the test contents that are required for assessing the ‘personality’ developed ‘as a result of mastering a language’.

Likewise, Volume 5 is meant to exemplify the test contents required for assessing the ‘personality’ developed ‘as a result of mastering literature’. As such, the academic resources provided in these five volumes put together, will serve as a perennial source to all those associated with language and literature education for obtaining adequate information. Accordingly, teachers will have a lot of graded resources for appropriate lesson planning with clearly demarcated components of language and literature, and the behavioural traits of personality both at the macro and micro levels. Learners will have a global perspective of what is being learnt as part of their acquiring the whole area of learning. Evaluators will have an all inclusive database of content resources not only for constructing a variety of objective based tests but also for adopting error-free sampling procedures for content coverage and distribution of scores. Researchers will have a systematically arranged comprehensive list of components of language and literature under a common frame of reference to maintain intra-level and inter-language comparability. Textbook writers will have a kind of ready reckoner with the details of content-ability components for their identification, selection, gradation, and distribution before they are presented as inter-related lesson units. Syllabus framers will have an empirical basis for outlining the objectives towards giving a clear direction to all those involved in the process of teaching, learning, and evaluation. Curriculum developers will have a database for determining the extent of emphasis to be given for language and literature education while translating the conceptualized goals of education into concrete action through the aggregate of all the courses as a unifying activity. Examination cells will have an inbuilt mechanism to choose and pick up question items which could be used to construct a variety of ‘test-equivalents’ with academically acceptable content inputs. Selection / Recruitment agencies too will have a wide range of item resources for instant construction of content-free and context-free tests (e.g., aptitude, proficiency tests) for the purposes of determining the suitability of candidates for admissions, jobs, etc., at different levels. Educational planners will have an overall idea about the role of language and literature in developing the learner’s all-round growth in terms of his intellectual, emotional, and physical aspects, as outlined in the taxonomies of education.

All the resources incorporated here were developed gradually, stage by stage, for more than a decade. Initially, General Frame of Reference (GFR) consisting major, minor, and sub-components that constitute the concept of language or literature or personality (objects being measured) were prepared along with their associated dimensions. The draft GFRs so prepared were improved upon for their effectiveness and comprehensiveness in the light of a series of discussions held with scholars representing six Indian languages.

Subsequently, all the components included in the GFRs were assigned computer number codes. Consequent upon this, the preparation of question items on each of the components by accommodating the dimensions associated with them was initiated through a series of workshops. While doing so, the levels of their operation viz., identification / recognition, production, and suitability in usage were taken into account for proportionate coverage. The question items thus prepared were scrutinised for their face validity before initiating action for keying them in. Computer printouts of the draft question items were then _ circulated among those who produced the items, initially for getting the gaps filled in and subsequently for enriching the quality. At this stage, each of the question items included was equipped with a series of additional information as required by the Item File Card (IFC). With these additions, the materials were again circulated and verified for their uniformity and accuracy. The ‘improvement measures based on the feed back obtained were given effect in the materials stored in the computer before they were brought out as the pre- publication version. Subsequently, they were circulated to about 100 senior scholars dealing with this subject across the country, for comments. On receiving a considerable of comments on each of these volumes, a small ~ workshop was organised once again to finalise the materials after discussing the rationality and reasonability of the comments received before they were accepted for inclusion. The materials, thus finalized, are brought out only now as regular publications. Despite having undertaken all these exercises, it is still not claimed that the materials are complete in every respect. Constructive suggestions are therefore welcome for further improvement.

The practical realization work in respect of other two areas viz., test methods and test purposes, is in various stages of progress. On finalisation, they will also be brought out separately. A data bank to be created in due course by putting together all these resources would facilitate the MILES development. The plan is to bring out the practical realization volumes in respect of all the 18 major Indian languages under each of these three areas of evaluation.

However, it has so far been possible only in respect of 3 Indian languages and that too only in one area of evaluation.

**Contents and Sample Pages**