PISA reading literacy assessment and senior high school reading literacy assessments: How do they differ?

The Program for International Student Assessment (PISA) is an international program assessment to evaluate and determine the education system's effectiveness by focusing on reading, mathematics, and science. This study aims to analyze the differences between PISA reading literacy assessment and senior high school reading literacy assessment in West Sumatera. Reading literacy is abilities and skills in reading needed in everyday life to develop students' knowledge and potential. The method of this study was comparative method. Data were taken from PISA reading literacy 2018, which OECD published, and reading tests made by English teachers from ten Senior High Schools in West Sumatera. In addition, this study used document analysis to collect the data. The differences between PISA reading literacy assessment and reading literacy assessment at senior high school were seen from eight indicators: cognitive process, form of text, types of text, form of test, situation, types of question, level of question, text medium, and text environment. It was found that PISA reading literacy assessment was more complex compared to reading literacy assessment at senior high school.


Introduction
Reading literacy assessment has been discussed widely by educators related to the low achievement of Indonesian students in some international literacy assessments, such as PISA and PIRLS (Chamisah, 2016;Harsiati & Priyatni, 2018;Zaim, Zainil, & Fitrawati, 2021). The result of PISA (Program for International Student Assessment) dan PIRLS (Progress in International Reading Literacy Study) indicate that Indonesian students were still in the low-level reading ability compared to other countries in the world (Prasetyo, 2017;Hasanah & Warjana, 2019;Safari, 2020;Zaim, Refnaldi, Zainil, & Irsyad, 2021). Therefore, evaluation should be made on why Indonesian students cannot compete with students with the same level of education from other countries in reading ability.
PISA is an international program assessment to test the academic performance of 15-year-old-students. It is held every three years to measure students' ability in reading, mathematics, and science. It aims to evaluate and determine the education system's effectiveness from an international perspective (Safari, 2020). The PISA study result can indicate whether the school system can effectively prepare students for further study. It measures students' abilities and prepares students for the real world, to acquire life-long thinking skills that enable them to access information (Akbar, Seifoori, & Ahour, 2017). PISA measures the students' knowledge and skills that are crucial for participating in society and understands education systems' relative strengths and weaknesses (Chamisan, 2016). The assessment of PISA aims to find whether students can apply what they learn in school to deal with real-life situations and Breakspear (2014) states that the study of PISA is to provide comparable evidence of student performance internationally on the skills to be essential for adult life. Furthermore, Chamisah (2016) states that the PISA program may evaluate education systems of various countries, assess the extent to which students have to obtain the knowledge and skills, provide analysis, and measures characteristics of students and school. This study focuses on the assessment of reading literacy used by PISA and Senior High School Teachers. Reading literacy is considered the ability to understand, use, reflect, and engage with written texts to achieve the students' goals, develop the knowledge and potential of students, and participate in society (OECD, 2006;Harju-Luukkainen et al. 2016). In addition, Alderson (2003) states that reading literacy includes processes and products. The process involves word recognition, comprehension, fluency, and motivation, and the product is generally associated with static information produced by testing techniques. Moreover, Harsiati & Priyatni (2018) states that reading literacy aims to improve skills live in access and take advantage of the information in the form of discourse for a live life in the educational context, work, and safety.
PISA measures the reading skills required in the modern world (Chamisah, 2016). The reading skills required in PISA reading literacy assessment are (1) to retrieve information, (2) to form a broad understanding, (3) to develop an interpretation, (4) to reflect on and evaluate the content of a text, and (5) to reflect on and evaluate the form of a text (OECD, 2009). In addition, Djiwandono (2011) states that a reader must have eight micro-skills in order to be able to read a text. The eight reading micro-skills are (1) to understand the meanings of words in context, (2) to understand the structure of the text, (3) to identify the main ideas, (4) to identify the explicit meanings, (5) to identify specific facts in different words and expressions, (6) to make inferences, (7) to identify literary expressions, and (8) to identify the writer's intentions and messages. Therefore, the eight micro-skills proposed by Djiwandono (2011) might be simplified into five reading skills in PISA.
There are eight characteristics of PISA reading literacy assessment; they are cognitive process, form of text, types of text, form of test, situation, types of questions, level of questions, text medium, and text environment (OECD, 2009;OECD, 2013;OECD, 2017;OECD, 2018a). Reading cognitive literacy process consists of locating, understanding, evaluating, and reflecting information. The forms of the text are continuous texts, noncontinuous texts, and mixed texts. Types of the text are description, narration, exposition, argumentation, instruction, and transaction. Forms of the test are simple multiple-choice, complex multiple-choice, yes/no questions, short response, close constructed response, and open constructed response. The situations are personal, public, education, and occupation. Types of questions are literal, inferential, and evaluative. Levels of question are level 1a until 5 of taxonomy bloom. Text mediums are print and digital. Text environments are authored environments and message environments. These eight PISA reading literacy assessment characteristics can compare PISA reading literacy assessment with senior high school reading literacy assessment.
Reading literacy assessment in senior high school is a curriculum-based assessment. It required certain content to cover in a certain period, a semester, or an academic year. Reading literacy assessment in senior high school develops students' reading skills from low-level to advanced reading ability levels. The level of questions given to the students also varies, from low-level to high-level thinking skills. However, teachers should provide an assessment with high-level thinking skills because the teachers should prepare the students to have a national examination. There should be a close relationship between PISA reading literacy assessment and senior high school reading literacy assessment since they assess students of the same age (15-16 years old). In addition, the senior high school students have acquired similar reading skills PISA reading literacy assessment. First, however, a study needs to determine the similarities and differences between these two reading literacy assessments.
This study examined the differences between PISA reading literacy assessment and reading literacy assessment at school. The problems to be investigated in this study are the characteristics of PISA reading literacy assessments, the characteristics of and senior high school reading assessment at school, and the differences between PISA reading literacy assessment and senior high school reading literacy assessment.

Method
This study used a comparative research design, gathering information about the tendency from a variable, indication, and condition when the research is done (Arikunto, 2010). It was used to determine the differences between the characteristics of reading literacy assessment of PISA and the characteristics of the reading literacy assessment at senior high school. This research data was PISA reading literacy assessment published by OECD 2018 and reading tests made by grade X English teachers in ten senior high schools at West Sumatera. PISA reading literacy assessments were taken from PISA reading literacy assessment 2018 (OECD, 2018). Reading tests at senior high school were taken from 10 mid-term examinations and semester examinations made by grade X English teachers at ten senior high schools in West Sumatera in academic year of 2020/2021. This study used the document analysis technique to analyze PISA reading literacy assessment and reading literacy assessment at senior high school.

Results and Discussion
To know the differences between PISA literacy reading assessment and reading literacy assessment at senior high school, the characteristics of these two assessments should be determined by using eight indicators as proposed by OECD (2018).
(PISA reading literacy assessment and senior high school reading literacy assessments: How do they differ?)

PISA Reading Literacy Assessment
The comparison between PISA reading literacy assessment and senior high school reading literacy assessment was held based on eight indicators proposed in PISA reading literacy assessment, they are cognitive process, form of text, types of text, form of test, situation, types of questions, level of questions, text medium, and text environment.
There are some cognitive processes in reading literacy assessment of PISA; they are to retrieve information, to form a broad understanding, to develop an interpretation, to reflect on and evaluate the content of a text, to reflect and evaluate the form of a text, and to comprehend literal information (OECD, 2017). Reading literacy processes consists of locating information, understanding information, and evaluating and reflecting information. Locating information involves accessing and retrieving information within a text, searching for, and selecting relevant texts. Understanding information includes literal information, integrating and generating inferences, and integrating and generating inferences across multiple sources. Finally, evaluating and reflecting information assesses quality and credibility, reflecting on content and form, and detecting and handling conflict (OECD, 2018).
There are three forms of text, continuous, non-continuous, and mixed text (OECD, 2018). Continuous texts are constructed by organizing sentences into paragraphs. Continuous text is divided into larger structures, like sections, chapters, and books. The texts such as newspaper reports, essays, novels, short stories, reviews, letters, reviews, blogs, and reports are included in continuous text. Non-continuous texts are organized in matrix format and comprise several lists such as lists, tables, graphs, diagrams, advertisements, schedules, catalogs, and indexes. Mixed texts can be found in the constituents such as an essay and a graph that is mutually supportive through coherence and cohesion links. Mixed text in the print medium can be found in magazines, reference books, and reports. Mixed text in the digital medium, such as authored web pages, is typically mixed with lists, paragraphs, and graphics combinations. Message-based texts like online forms, e-mail messages, and forums combine continuous and non-continuous formats. Multiple texts are those that have been produced independently (Rachel, 2011). The interrelation between the texts may not be apparent; they may be complementary or contradict one another.
The types of texts used in reading are descriptive, narrative, exposition, argumentative, instruction, and transaction (OECD, 2017). Description means texts with information that refers to the properties of objects in space, such as information report in prose, catalog, blog diary. Narration means texts with information that refers to objects in time, such as plays, comic strip stories. Exposition means texts explaining how different elements interrelate in a meaningful way and provide answers to "how" questions, such as book reviews and ratings of online shopping items. Argumentation means texts that present the relationship among concepts or propositions, such as advertisements, blogs in an online forum. Instruction means a text that provides instructions on what to do, such as a recipe, instructions for operating software. Finally, the transaction means a text that aims to achieve a purpose, such as requesting that something is done, organizing a meeting, or making a social engagement with a friend such as a letter, an e-mail, or a text message (OECD, 2018). The forms of test in reading literacy, according to Shiel (2006), are simple multiple-choice, complex multiple-choice, yes/no questions, short response, close constructed response, and open constructed response.
There are four situation variables in reading: personal situation, public situation, educational situation, and occupational situation. The personal situation is intended to satisfy an individual's interests, both practical and intellectual. The public situation relates to the activities of the larger society. The educational situation is usually designed specifically for instruction. Finally, the occupational situation relates to the accomplishment of some immediate tasks. According to Rachel (2011), there are four situations identified in the PISA reading. They are personal such as letters, fiction, diary-style blogs, public such as public notices, news websites, occupational such as job advertisement in a newspaper or online and educational such as textbooks, interactive learning software.
Types of questions in PISA reading literacy are literal, inferential, and evaluative (Muayanah, 2014). Literal is considered the most accessible type, inferential is the medium level, and evaluative is the difficult level. In addition, Daeli et al. (2020) state that types of questions in reading are divided into five types: literal comprehension, reorganization or reinterpretation, inference, evaluation, and personal responses. Anderson revised Bloom's taxonomy and proposes six categories of questions: remember, understand, apply, analyze, evaluate, and create (Anderson, 2001). Thus, revised Bloom's taxonomy means categories of the cognitive processes in learning (Fitria, Syarif, Refnaldi, 2014).
PISA reading literacy classifies the text medium of the test into two classifications; print text and digital text. Examples of print texts are single sheets, brochures, magazines, and books. Digital text is displayed through liquid crystal display (LCD), plasma, thin-film transistor (TFT), and other electronic devices. Digital International Journal of Research in Counseling and Education, Open Access Journal: http://ppsfip.ppj.unp.ac.id technology is very useful to be used in teaching and assessing reading; it can help to improve the literacy outcome of the students (Biancarosa & Griffiths, 2012) Text environments in PISA reading literacy are authored-based text environments and message-based text environments (OECD, 2017). The example of an author text environment is a web page, and the examples of the message-based text environment are e-mail and blog.

Reading Literacy Assessment at Senior High School
The characteristics of reading literacy assessment at senior high school can be explained by using the eight PISA reading literacy assessment indicators. The cognitive process of reading literacy assessment in senior high schools is retrieving information and forming a broad understanding. The formats of the text are continuous and non-continuous. The types of text are description, narration, recount, transaction, and announcement. The forms of the test are multiple-choice, short answer, open-ended question, closed-ended question. The situations of the test are personal, public, and educational. The levels of questions are remember, understand, apply, and analyze. The senior high school types of questions are literal and inferential. The text mediums are printed and digital. The text environments are authored text and messages-based text.
Related to reading questions developed by English teachers at senior high schools, Muayanah (2014) states that the question types that require the readers to provide specific content information in their answer are using wh-question words such as what, when, where, why, who, and how. On the other hand, yes or no questions show students' agreement or disagreement on an issue or statement given.

The Differences between PISA and Reading Literacy Assessment at Senior High School
Based on the analysis of the PISA reading literacy assessment characteristics and Senior High School Reading Literacy Assessment, the differences between them can be concluded in the following table. Text environment Authored, message-based Authored and message-based From the table above, it can be seen that there are eight indicators to differ PISA reading literacy assessment and reading literacy assessment at Senior High School; they are cognitive process, form of text, types of text, form of test, situation, level of question, types of question, text medium, and text environment.
In the cognitive process, PISA reading literacy assessment consists of five components, retrieving information, forming a broad understanding, developing an interpretation, and reflecting on and evaluating the content of a text. Meanwhile, the cognitive process of reading literacy in Senior High School consists of two components only, retrieving information and forming a broad understanding. Three cognitive processes that (PISA reading literacy assessment and senior high school reading literacy assessments: How do they differ?) are not learned in senior high school are forming a broad understanding, developing an interpretation, and reflecting on and evaluating the content of the text. These three cognitive processes are essential to enhance the students' critical thinking and higher order thinking skills.
PISA reading literacy assessment consists of four components in the text format: continuous, noncontinuous, mixed text, and multiple texts. Meanwhile, the text format in reading literacy assessment at senior high school consists of only two components, continuous and non-continuous text. It means that students are not familiar with mixed text and multiple texts widely used in mass media and other printed and digital media sources. Therefore, these two text formats should be used in reading class, limited to continuous and noncontinuous text. Hasanah & Warjana (2019) state that students' ability to understand text deeply can only be formed through the habituation of reading multiple texts.
In terms of types of reading text, PISA reading literacy assessment has six types of reading texts: description, narration, exposition, argumentation, instruction, and transaction. Meanwhile, types of reading texts at Senior High Schools are only five: description, narration, recount, transaction, and announcement. However, two types of reading texts are not used in PISA reading literacy assessment: recount and announcement. These two types of reading texts are essential to learning in senior high school. Nonetheless, there are also three types of reading texts at PISA reading literacy assessment that is not learned at school: exposition, argumentation, and instruction. Therefore, these three types of reading texts should be learned by the students.
There are five forms of test of PISA reading literacy assessment: simple multiple-choice, complex multiplechoice, short answer, closed-ended question, and open-ended questions. Meanwhile, the form of text in reading literacy at Senior High School consists of only four forms: multiple-choice, short answer, closed-ended, and open-ended questions. Simple multiple-choice such as yes/no questions and true/false questions was not used at senior high schools. However, these two simple multiple-choice questions may enhance the argumentative skills of the students to decide whether something is true or false or decide to agree or disagree related to the problems given.
The text situations of the test in PISA reading literacy assessment are personal, public, educational, occupational. Meanwhile, the text situations in reading literacy at Senior High School are personal, public, and educational. No occupational text situation was found at senior high school. Furthermore, the students must know the occupational situation since they will get a particular profession or job when they graduate from school. Therefore, students should read texts related to the occupational situation in the recent era to decide which occupation they will choose after graduating from school.
The level of the question at PISA reading literacy assessment questions are remember, understand, apply, analyze, evaluate, and create. Meanwhile, the levels of questions found in reading literacy at Senior High School are only remember, understand, apply, and analyze. Students should be taught how to evaluate and create something to encourage the students to have the higher order thinking skills. Kemdikbud (2019) mentions that higher order thinking skills need to be possessed by every student to function optimally as individuals and members of society who are critical, independent, and productive.
The types of questions in the PISA reading literacy assessment are literal, inferential, and evaluation. Meanwhile, reading literacy assessments at Senior High Schools include only literal and inferential. This is in line with the level of questions that the students should own that evaluation types were not learned at senior high school. The ability to evaluate is necessary to decide whether something is good or not, agree or disagree, or accept or refuse. The ability to evaluate is essential to make a decision.
The text mediums in PISA reading literacy assessment are print and digital, while the text medium in reading literacy assessment at Senior High School is mostly printed. In the digital era, the teacher should be able to test using digital medium since almost all students now have a gadget that can be used to assess information, including tests given by teachers. Reading literacy assessment through digital media is very effective in facilitating the teacher's task to check the test answers given by student (Rachmawati, Mulyono, & Sitanggang, 2017).
The text environment in PISA reading literacy assessment is authored, and message based. Reading literacy assessments at Senior High School are found similar to those of PISA reading literacy assessments. These two text environments can be found in real life, and we are very familiar with these two texts. However, we have to choose which texts are helpful and suitable to assess students' reading literacy. Harsiati (2018) states that reading is not simply decoding but integrating information to comprehend the content of the text to be used functionally. In addition, Alderson (2000) mentioned that reading is not only knowing literal meaning but reading interactively to comprehend the text critically.
Differences between PISA reading literacy assessment and Senior high school reading literacy assessment will be the gap that must be filled to make the senior high school reading literacy assessment more qualified.
International Journal of Research in Counseling and Education, Open Access Journal: http://ppsfip.ppj.unp.ac.id The reading literacy assessment that has achieved the characteristics of PISA reading literacy assessment in terms of the level of difficulty and types of tests, types of text, and types of questions will be helpful for students to follow international reading literacy assessment later.

Conclusion
PISA model of reading literacy assessment that measures general competencies of 15-year-old students in reading can be implemented in the teaching-learning process of reading at school. Reading literacy assessment at school should mean being technologically literate, critical thinking, sensitivity to the surrounding environment, and applying what is read. The differences between PISA reading literacy and reading literacy at senior high school can be seen in eight indicators: cognitive process, form of text, types of text, form of test, situation, types of question, level of question, and text medium, and text environment. Some indicators provided in PISA reading literacy assessment cannot be found in reading literacy assessment at senior high school. These indicators should be included in reading literacy assessment at senior high school to enable students to compete with other students in other countries when they follow international literacy assessment.