Content Analysis of Education Module

Content Analysis of Education Module CONTENT ANALYSIS ON ENGLISH MODULE BAHASA INGGRIS Dwi Aryani Abstract This research is aimed to evaluate the quality of the English module used by students of SMA N 6 Yogyakarta. There were many problems such as mistype, inappropriate content or material, and so on which were found inside the book. Then, the module used by the students should be evaluated on the content to know whether the book met the criteria of good module or not. The research was about case study using content analysis as the method. It used textbook evaluation criteria from Cunningsworth which contained ten aspects of evaluation and compared with the criteria of evaluation from Hutchinson and Waters. The evaluation concerned on the content such as language content and skill, the organization and appearance of the book. The result from Cunningsworth checklist showed that the book got 74. 25% which considered as good book. While the result from Hutchinson and Waters showed 78. 57%. The aspect which got low percentage was vocabulary and reading material, while the rest were considere d as good. From evaluation using two concepts from Cunningsworth which compared with the evaluation criteria by Hutchinson and Waters showed a little bit difference. The result showed differences in range 4. 32%. Keywords: module, content analysis, evaluation, and evaluation checklist INTRODUCTION Teaching resources are all materials that are used to facilitate in teaching and learning process. They can be divided into three forms which each has different functions and advantages. The first form of material is printed materials such as books, module or workbooks, worksheets, help the beginner teachers to cover and design each lesson carefully in detail. They also give all the plans and lessons that teachers need to present the topic in some details. Moreover, printed material can be used in any location or bring everywhere. Nonprinted materials like audio and video materials have their own advantages such as; inexpensive materials, since nonprinted materials have already prepared by the teachers and students should not buy it. There are materials that comprise both print and nonprint such as materials from internet which is also have some advantages. Internet provides many materials in a form of audio, text, picture, graphic and video. Students only need to access on the inter net pages suggested by the teachers and follow the instruction. It was quite easy for school which provides internet access (Richards-2002). Material has important roles in language teaching such as presentation resources, practice and communication interaction, language competence references, classroom activities sources, syllabus, and help less experience teachers (Cunningsworth-2005). Those are the role of materials in language teaching for both students and teachers. Module in several schools used as the primary sources in conducting teaching and learning activity. Module is categorized as commercial books which contain some potential negative effect as cover inauthentic language, distort content, not reflect students needs, deskill teachers, and expensive (Richards-2002). Related to the negative potential effect, it means that the materials covered by module are needed to be evaluated. In order to judge the quality of the book, the book should cover some criteria of good teaching resources. The criteria of good teaching resources on this context are criteria for printed materials. Teaching resources take a part in the teaching and learning process. The criteria are; 1) arouse the learners interest, 2) remind them earlier learning, 3) tell them what they will be learning next, 4) explain new learning content to them, 5) relate these ideas to learners previous learning, 6) get learners to think about new content, 7) help them get feedback on their learning, 8) encourage them to practice, 9) make sure they know what they are supposed to doing, 10) enable them to check their progress, and 11) help them to do better. Evaluation is measuring of strengths and weakness (Davies-2002). It can be conclude that the materials which are covered by module can be evaluated to know the strength and weakness. The evaluation can be done in three ways; pre-use evaluation, in-use evaluation, and post-use evaluation (Cunningsworth-2005). The process of evaluation depends on the one who desires to evaluate the materials. Moreover, in order to know the strength and weakness of materials (book or module), it can be done through the evaluation checklist from the expert on the field (the expert on this context is expert in teaching). There are many experts who provide the checklist to help in evaluating book. METHOD This research aimed to evaluate the content of module for senior high school. The type evaluation on this research is post-use evaluation since the evaluation conduct after the book was used by the students. The research was focused on the content of the book in order to meet the criteria of good teaching resources by measuring the strength and weakness. Therefore, it is classified as case study research category. Case study is a research method which is done towards a system; it can be program, activity, and event. Case study is aimed to get a data, take a conclusion, and get an understanding from the data itself (Sukmadinata-2012). A case study may be understood as the intensive study of a single case where the purpose of that study is to shed light on a larger class of classes or a population (Gerring-2006). Moreover, it was not aimed to get the conclusion from the population. The conclusion from case study is simply from the case that the researcher taken to heart. There are some technique in collecting the data on case study research such as interview, observation, and documentary study. Those are the technique to collect the data and to get the conclusion. The data source was the materials covered by Bahasa Inggris, a module used by students in SMA N 6 Yogyakarta grade XII on the first semester. The book was written and published by CV Viva Pakarindo which was consisted of eighty pages and divided into five chapters. The book was applying Kurikulum Tingkat Satuan Pendidikan or abbreviated as KTSP as the curriculum. Instruments were needed to collect the data. This research used content analysis as an instrument to collect the data from students module. There were two instruments to support in conducting the research. First instrument used in this research was documentary study. Sukmadinata (2012) explained that documentary study is data collecting technique which is accumulate and analyze the document, whether documentary, picture, or electronic. The document which is accumulated is already chosen with purpose and focus on the problem. Moreover, the other instrument on this research was content analysis where the study is focus on deeply into the content of written information or mass media. Moreover, content analysis is defined as a research method for the subjective interpretation of the content of text data through the systematic classification process of coding and identifying themes or patterns. Wilkinson (2003) mentioned content analysis has been successfully used to analyze text and solve issues of disputed authorship of academic paper. and examples of texts suitable for content analysis includes essays, journal articles, books and chapter in books From the explanation which stated in the Wilkinson give a clear statement that module or workbook which categorized as book can be analyzed. In this case, module included as one of academic paper such as mentioned before. Then, content analysis focused on the characteristics of language as communication with attention to the content or contextual meaning of the te xt. Since the subject on this research was the module, so the researcher prepared some checklists to evaluate the book. The researcher provides two evaluation checklists to evaluate the book. The first evaluation checklist is taken from Cunningsworth which covers ten descriptions such as; design and organization, language content, vocabulary, grammar, language skill, listening, speaking, reading, writing, and practical consideration. The other evaluation checklist is taken from Hutchinson and Waters which cover main aspect such as audience, aims, contents, methodology, and others. Every aspect on evaluation checklist covers some questions or statements to be answered in evaluation process. There were some procedures for conducting this research. The procedures will be explained as follow; a) Preparation, on this step the researcher took a part to choose the book which was needed to be evaluated. Choosing the book which was needed to be evaluated from the external and internal factor; b) Analysis, took a part on analyzing the book using criteria designed by the researcher based on the criteria or concepts from the expert. It means that the evaluation criteria were not pure from the researcher, but adapting from one concept to evaluate the book; c) Report, this step was the last step on this procedure. After the researcher pass the preparation and analysis step, means that the researcher should make the report of the result after conducting the evaluation. After conducting the evaluation towards the book, however, scoring is needed to get the result of the evaluation. Book evaluation using Cunningsworths checklist given score one until five. The indicator is modified using Likert Scale which indicates negative until positive (Sugiyono-2012). Moreover, evaluation checklist suggested by Hutchinson and Waters has the own rate or score by using zero, one, and two. These score and indicators are originally designed by Hutchinson and Waters. The result of the evaluation is calculated using mean deviation. Therefore, the result of calculation will be described in a form of paragraph by using descriptive qualitative. Figure 1. A pattern to calculate the score of each evaluation. Note : Æ ©X = the total of criteria which is met the criteria N  Ãƒâ€šÃ‚   = the total criteria points in each aspect Table 1. The level of Suitability 80% 100% Very Good 60% 79% Good 50% 59% Fair Under 50% Poor FINDINGS AND DISCUSSION The evaluation taken from two sources; Cunningsworth checklist which is used as the evaluation focus, while the result of evaluation from Hutchinson and Waters is used to compare on this findings. Based on the evaluation, Hutchinson and Waters provides twenty one questions and divided into five main aspects including with the additional information. The overall evaluation from Hutchinson and Waters got result 78, 57% in percentage. It will be compare with the result of evaluation from Cunningsworth evaluation checklist. The findings of this research presented ten specific aspects where each of them are analyzed based on Cunningsworths evaluation checklist. Design and Organization This aspect covers five criteria of evaluation with the total score 20 out of 25. The percentage is 80% and it is categorized as very good. The result of design and organization evaluation is presented on the table as follow. Table 2. Design and Organization Checklist No Design and Organization Criteria Score 1 The content is organized 5 2 The content is sequenced 5 3 Provide adequate recycling and revision 0 4 Provide reference sections for grammar and vocabulary 5 5 The materials suitable for individual study 5 Total 20 Percentage 80% The first criteria is about the content which is organized. It means the contents of the workbook such as topic, structure, skills, and function are organized. All units provide complete topic, all four skills in every unit, language functions and language features. The evaluation shows that the first criteria got perfect score. Such as the organization, the second criteria is about the sequential of the content. It discusses whether the workbook has difficult level for the students and also the function of the workbook for individual study. From all units, the materials can be used for the students to study by themselves. The book cannot fulfill the third criteria by presenting recycle and revision section. The fourth criteria can be fulfilled by providing reference sections for grammar and vocabulary. The materials also suitable for individual study. It can be seen from the contents of the workbook which provide material on the first page in every chapter and continued by the exerc ise. There are many exercises on the workbook such as exercises for all skills, language contents (grammar and vocabulary), and also exercise that cover all materials. It is presented in the last part in every chapter. The exercise also in different types such as multiple choice and essay. From all those exercises, the workbook is easy for the students even for individual study. Language Content Language content is the second aspect of evaluation with the achievement 60% from the total score 9 out of 15. The table below presenting the result of language content evaluation. Table 3. Language Content Checklist No Language Content Criteria Score 1 The book include materials for pronunciation work 0 2 The book deal with the structuring and conventions of language use 5 3 Language style is matched to social situation 4 Total 9 Percentage 60% From the table 3 we can conclude that the first criteria cannot fulfill by the book by providing materials for pronunciation.   The second criteria got perfect score related to the structuring and conventions of the language use. It deals with how to take a part in conversation, how to structure a piece of extended writing, and how to identify the main point in reading passage. Language style is the choice of words used by a specific group of people when they speak. For example there are bureaucratese, the words, jargons, and abbreviation. It should be matched with the social situation of the students. There is one part of the book the researcher found that inappropriate language style with the social situation of the students. It was because the students accusing her friend took her money which actually she does not. Grammar The achievement on grammar aspect is on 88% with the total score 22 out of 25. The suitability of grammar materials categorized into very good (see Table 1). The result of the evaluation on grammar aspect is presented on the table below. Table 4. Grammar checklist No Grammar Criteria Score 1 The book covers grammar items for the level 5 2 Grammar presented in small pieces units 2 3 Grammar emphasis on language form 5 4 Grammar emphasis on language use (meaning) 5 5 Language use and form are in balance presentation 5 Total 22 Percentage 88% The book has already provided grammar items based on the level of students. Then, more than half of the book presented the materials for grammar in wide space where it should be presented on the small units to make the students easier to learn the material. Moreover, grammar materials similarly emphasis on language form and use. Both of language form and use should be presented on balance presentation as the criteria mention. Vocabulary Table below is the result of evaluation of vocabulary section where covers five criteria of evaluation. The aspect got low percentage with 44% where there was one criteria cannot fulfill by the book even the worst is contain two score zero. Table 5. Vocabulary checklist No Vocabulary Criteria Score 1 Materials for vocabulary teaching adequate in terms of quantity and range of vocabulary 0 2 Vocabulary materials are strategies for individual learning 0 3 The materials emphasis placed on vocabulary development 5 4 Vocabulary presented in a structured and purposeful way 5 5 Provide distinction between active and passive vocabulary 1 Total 11 Percentage 44% Based on the evaluation, the book minimally covers 1000 new vocabulary for every stage or every grade. The quantity of the vocabulary in every chapter is not enough if we took a look on the material. The availability of vocabulary on the book was too far with the range of the evaluation. The total vocabulary material presented in the book consists of sixty one; chapter one twelve, chapter two is twenty, chapter three with ten, chapter four provides fourteen, and chapter five presents five new vocabulary. Related with the quantity of vocabulary materials, it could not be used for learning individually. However, the vocabulary materials which are presented on the book help the students in semantic relation such as presenting the meaning and synonyms. Moreover, vocabulary which is presented in every chapter also categorized in situational situation. Language Skill Language skill gets 95% on the result of evaluation where it is considered as very good. Most of criteria are fulfilled by the book with achievement 5 in average. Table 6. Language Skill Checklist No Language Skill Criteria Score 1 All four skills were covered 5 2 All four skills are presented balance 4 3 The materials are integrated skills work 5 4 Reading passages and associated activities are suitable with the students levels 4 5 Provide sufficient reading material 5 6 Materials for spoken English are well designed 5 7 Writing activities are suitable in terms of amount of guidance/control 5 8 Writing material is using appropriate style 5 Total 38 Percentage 95% Language skill has already covered all four skills such as listening, speaking, reading, and writing inside the materials in every chapter. Those four skills are presented in the same portion. The book similarly provides materials and exercises for each skill and integration. Most of materials are presented by reading passage where they are supposed to suitable with the level of students. For this reason, the book should provide reading passage in sufficient way in order to present the materials. Moreover, the book also has to provide materials for spoken English where it should be well design. Related with the materials and exercises for each skill, writing section should provide guidance or control in writing activities. The style used for writing skill also should be presented in appropriate style. From all criteria, the book gets high score in these eight criteria of evaluation. Listening Listening aspect covers four criteria where there is one criteria cannot fulfill by the book. The evaluation shows that the book got 75% from the total score 15 out of 20. Table 7. Listening Checklist No Listening Criteria Score 1 Listening form part of dialogue or conversation work 5 2 Provide specific listening passages 5 3 Listening material set in a meaningful context 5 4 Provide pre-listening task 0 Total 15 Percentage 75% The evaluation shows that the book has already provide listening section in a form of dialogue or conversational work in every chapter. There are some listening passages which are provided by the book in order to organize the section. Moreover, all materials and exercises are presented in meaningful context. However, the book does not provide pre-listening tasks before they focus on the listening section. Speaking Speaking got perfect achievement with the percentage 100%. It indicates that the book provide all contents that the criteria mention. [P3] Table 8. Speaking Checklist [P4] No Speaking Criteria Score 1 Provide spoken English materials 5 2 The material contains oral presentation and practice of language items 5 3 The material contains dialogues 5 4 The material contains roleplay 5 Total 20 Percentage 100%   The materials for speaking are presenting inside the book where they contain oral presentation and practice for language items. In order to help the students in practicing their speaking skill, the book provides materials such as dialogue and role-play. Reading From the evaluation, reading got low percentage with the achievement is 32.5%. The total score of fulfillment is 13 where actually the total score is 40. The book provides reading text where it is used for introducing new language items such as grammar and vocabulary. Therefore, the materials are not focus on the development of reading skill and strategies for students. Reading materials, however, provide integrated materials with other skills and work. On the contrary, the book does not provide setting scene and background information on reading passage. Then, the book also does not provide pre-reading activities. Thus, reading materials similarly appears on the middle of units where it is commonly appears on the early unit. Table 9. Reading Checklist No Reading Criteria Score 1 Reading text used for introducing new language items (grammar and vocabulary) 3 2 The material focus on the development of reading skill and strategies 0 3 The material linked to other skills and work 5 4 Reading text appears on the early unit 0 5 The subject matter is appropriate 5 6 The text provide setting scene 0 7 The text provide background information 0 8 Give pre-reading question 0 Total 13 Percentage 32.5% Writing The evaluation shows that writing covers five criteria where there is one criteria cannot fulfill by the book. The score of evaluation is 19 out of 25 and it is presented on the percentage with the achievement 76%. Table 10. Writing Checklist[P5] No Writing Criteria Score 1 The material handle by guided writing 5 2 The material handle by controlled writing 5 3 Provide variety task 5 4 Provide language resources specific to the written form such as punctuation, spelling, and layout 4 5 Learners encourage to review and edit their written work 0 Total 19 Percentage 76% From the evaluation, writing materials are handled by guided writing and control writing which are presented in a form of simple instruction. Moreover, writing also provides various types of task such as writing memo, letter, and so on. Writing materials and tasks, however, do not encourage the learner to review and edit their own work since there is no space or section provided by the book. Practical Consideration The last evaluation is on practical consideration which covers five criteria. The result shows that the book got 92% from the total score 23 out of 25. The evaluation presents that the book represents good value for money. It means that the book neither too expensive nor too cheap. Even though the price is quite cheap, but the quality of the book is strong and long-lasting because the materials which is used to print the book. Related to the price of the book, it is easy to be obtained by the students; the school has already stock the book. Therefore, the students may ask to the school or their teachers. On the contrary, the appearance of the book does not attractive as they should.   The cover of the book is colored by burgundy where it looks like dark and gloomy. The detailed achievement from practical consideration is presented in the following table. Table 11. Practical Consideration Checklist No Practical Consideration Criteria Score 1 The book represent good value for money 5 2 The book is strong and long-lasting