INTRODUCTION

The nations of the world are continually confronted with global competition and the need for mankind to solve problems such as climate change, environmental degradation, resource utilization, and control. On the other hand, companies, industries, and employees are constantly challenged by the demand for innovative products and solutions to emerging problems by their clients. Dealing with these problems and many more requires present and future employees to be able to think critically. Therefore, critical thinking skill is one of the most sought skills by employers of labour, to improve and increase their outputs (P21, 2015; Retnowati, Riyadi and Subanti, 2020). This underscores the importance of education that is relevant to the needs of the 21st century and produces critical thinkers. Therefore, the role of these skills in economic development is seen as a crucial issue in assisting nations to attain higher employment opportunities, economic empowerment, self-reliance as well as coping with increasing competition in the global market and an uncertain future (Kalelioğlu and Gülbahar, 2014; P21, 2015). The emphasis in 21st century learning is not on memorization and rote learning but on the acquisition of competencies and the ability to infer, analyse, deduce, interpret, and draw a conclusion as well as apply these skills to solve problems in real-life. There is an accord in the literature highlighting the significant role of critical thinking skills for meaningful living in the 21st century. However, employers of labour observed that graduates are deficient in critical thinking and problem-solving skills (Kivunja, 2015; Retnowati et al., 2020). This implies that the instructional strategies employed by teachers do not seem to enhance learners’ critical thinking skills.

The educational policy in Nigeria stipulated that “the country’s educational goals shall be set out in terms of their relevance to the needs of the individual and those of the society” in consonance with the world, the 21st century skills such as critical and creative thinking skills and the integration of technology are what is relevant to the present labour market (Papadakis, Vaiopoulou, Sifaki, Stamovlasis, Kalogiannakis and Vassilakis, 2021) among others.

Nevertheless, science teaching and learning do not seem to enhance students’ critical thinking skills in most countries of the world, especially in Nigeria (Pitan and Adedeji, 2012; Retnowati et al., 2020). There seems to be a mismatch between the skills students acquire in the classroom and the skills required in the labour market. They reported a lack of skills such as critical thinking, decision-making, and analytical skills. This, however, calls for a paradigm shift from classroom instructions that are traditionally based and focus on lower skills to a new paradigm that will foster critical thinking skills (Dorouka, Papadakis and Kalogiannakis, 2020). The new paradigm should emphasize students’ active exploration in the instructional process, where instruction is characterized by open-ended problem solving, question prompts, hands-on, minds-on, and authentic learning, among others (Thibaut et al., 2018). These elements may bring about meaningful learning and acquisition of twenty-first-century skills such as critical thinking skills.

However, despite the importance of critical thinking to problem-solving and meaningful living in the 21st century, researchers have reported a deficiency of critical thinking skills among learners across the world, Nigeria inclusive (Grosser and Nel, 2013; Salami, 2013; Wartono, Hudha and Batlolona, 2018). To enhance students’ ability to think and achievement in science, there is a need to adopt appropriate instructional approaches and models among others. Wartono et al. (2018) reported that appropriate instructional models can enhance meaningful learning and the ability to think critically. Therefore, learning models with the potential of enhancing critical thinking skills could be integrated STEM approach because it is characterised by hands-on and mind-on activities as reported by researchers (Duran and Sendag, 2012; Thibaut et al., 2018). Integrated STEM education is an instructional approach in that science and mathematics concepts are learned in the context of technology and engineering. Students engage in scientific inquiry, and technological and engineering design iterative processes. In support of this (Sumarni and Kadarwati, 2020) opined that STEM-based instruction enables students to develop and formulate ideas to solve problems thus engaging their Higher Order Thinking Skills (HOTS). The approach is a learner-centred while the teachers act as a facilitator in the learning process. Thus, it provides the liberty for students to engage in higher cognitive processes such as ideas generation, evaluation, inference, explanation, and drawing conclusions which could enhance learners’ critical thinking skills. In the next subsections, the statement of the problem, research questions, methodology, results, and discussions will be discussed.

Statement of Problem

The impact of globalisation and the evolving economy has influenced the school system to reconsider ways of effective teaching and learning because the current conventional instructional practices in the classroom were designed in the 20th century, which was a period of the industrial-based economy and was not envisioned for the knowledge-based economy of the 21st century (Jacobs, 2010; Wagner, 2008). While the practices in the classroom in Nigeria are keeping the status-quo (conventional instructional practices) the present workplace demand is evolving very fast. There is an agreement in previous literature that the development of 21st century skills is very vital to the economy and technological development (Pickering, 2010; Wagner, 2008). Nevertheless, Nigerian students do not demonstrate the ability to think critically, that is they lack the ability to infer, conclude deduce, and recognise assumptions (Pitan and Adedeji, 2012; Salami, 2013; Stapleton, 2011).

The reason can be attributed to the fact that classroom interaction is dominated by the teacher and knowledge acquisition during instruction is focused on lower thinking skills of Bloom’s taxonomy which is characterised by memorization, recall, and rote learning. Innovative instructional strategies that are student-centred and characterised by active learning which could stimulate thinking skills are not adopted. The evaluation also focuses on facts and lower cognitive skills. Therefore, students exhibit a low level of critical thinking skills and students’ performance in science continues to dwindle in Nigeria (Ezeudu, Ofoegbu and Anyaegbunnam, 2013; Pitan and Adedeji, 2012; Salami, 2013). Hence, the need to explore alternative teaching approaches that may enhance students’ acquisition of critical thinking skills is required. One of such instructional strategies may be integrated STEM education.

Statistics have shown that STEM-based occupations will increase by 17% ability to employ more workers compare to the non-STEM occupation which will be less than 10% (Butcher, 2013). Therefore, STEM education is critical to an individual’s ability to live a meaningful life in the 21st century because it empowers society in several ways. STEM-based knowledge and skills have the potential to enhance innovations and the quality of life. In view of the critical role of STEM education, Nugroho et al. (2021) opined that emphasis should be placed on developing and designing STEM-based instructional activities to enhance meaningful learning and acquisition of learning skills such as critical thinking skills.

The lack of critical thinking skills in the nation’s future workforce will negatively affect the quest to compete effectively in the global market and also impede the nation’s quest for sustainable development. Therefore, this study examined fostering critical thinking skills employing an integrated STEM approach among secondary school students in Minna, Niger State.

Research Question

The research question that is formulated to guide the study:

1. Does the integrated STEM approach enhance secondary school students’ critical thinking skills in the experimental and control group?

Research Hypotheses

The following null hypotheses were formulated and tested at a 0.05 level of significance:

H0₁. There is no significant mean difference in the within-group comparison of secondary school students’ critical thinking skills in the experimental and control group.

H0₂. There is no significant mean difference in secondary school students’ critical thinking skills between the experimental and control group.

LITERATURE REVIEW

Relevant literature to this study was reviewed under the following sub-headings; integrated STEM approach, critical thinking skills, and in each sub-sections, empirical studies were highlighted.

Integrated STEM Approach

STEM education seems to have the potential to foster students’ ability to think critically because it is characterised by question prompts, questioning, real-world open-ended problem, collaboration, and self-directed learning among others. Sumarni and Kadarwati (2020) investigated the effects of Ethno-STEM project-based instruction on critical and creative skills among secondary school students, the findings indicated that students’ critical thinking skills were enhanced. Han, Capraro, and Capraro (2016) reported that STEM-based instruction enhanced students learning outcomes. Similarly, Mater et al. (2020) conducted on the effects of STEM on critical thinking skills, and the results showed that the experimental group who learned with the STEM approach had improved critical thinking skills compared to the traditional group.

Phonchaiya (2014) examined the effect of STEM education on learners’ development of critical and creative skills, the findings showed that STEM education enhanced students’ critical thinking. Similarly, Oonsim and Chanprasert (2017) conducted a study on promoting the critical thinking skills of secondary school students using STEM-based instruction in physics. The findings indicated that STEM education enhanced students’ ability to think critically. Kim, Sharma, Land, and Furlong (2013) reported that instructional approaches characterised by questioning serve as scaffolds that provided students with the opportunity for deeper thought and enhanced the development of critical thinking skills.

Critical Thinking Skills

Critical thinking is a complex phenomenon and has been defined severally and categorised into several dimensions or subskills in the literature (Facione, 1998; Halpern, 2014; Watson and Glaser, 2010). It is defined as a cognitive skill that involves the ability to interpret, infer, analyse, evaluate, and conclude to make decisions (Facione, 2011). Therefore, in this study critical thinking is the process of learning that requires students to generate ideas, analyse, interpret, evaluate, and draw a conclusion to make a decision. The following dimensions have been adopted; inference, interpretation, recognizing assumptions, deduction, and evaluation (Smith, Rama and Helms, 2018; Watson and Glaser, 2010). Critical thinking skills are cognitive processes that results in interpretation, evaluation of arguments, deduction and making inference and recognition of assumption among secondary school students in this study.

As highlighted earlier, critical thinking is an important goal of science education. Therefore, students can be taught critical thinking skills in the science classroom. However, this requires effective teaching and learning strategy as well as a learning environment that will enhance thinking skills development. Kek and Huijser (2011) opined that the learning environment and instructional strategies should help the learner develop the ability to define a problem and generate and analyse information to solve a problem. In view of this, to help students improve their ability to think critically among others. The learning environment should be modified to provide students with the opportunity to take charge of their learning and collaborate to solve a problem (Johns, 2012). Integrated STEM classroom activities that foster experiences that are interdisciplinary and characterized by small group interaction, inquiry, and the open-ended problem can significantly impact students’ ability to think critically (Duran and Sendag, 2012).

The theory that supports critical thinking is rooted in Benjamin Bloom’s work (Bloom, 1956) who classified the cognitive domain into six levels, each of the levels corresponds to the cognitive ability of an individual (Duron, Limback and Waugh, 2006). Critical thinking and scientific thinking are enhanced by activities with higher order thinking abilities (Schulz and FitzPatrick, 2016). Kuhn (2002) conceptualized scientific thinking to mean everyday thinking of an individual and as an example of higher-order thinking. Consequently, the STEM activities are authentic and real-life in nature which could enhance critical thinking which is synonymous with scientific thinking.

Scientific thinking is the process of rational thought that must be adapted to promote the increase of science content knowledge, it is a knowledge-seeking process (Kuhn, 2010). This form of thinking provides the opportunity for exploration and follows the scientific method or procedures of problem-solving where learners find answers to open-ended problems. Scientific procedures such as engage, explore, expand, elaborate, and evaluate (5E), as well as drawing a conclusion (Bybee, 2010). The engineering design process was adopted for this study and seems to align with the 5Es.

The subject-specific learning of critical thinking skills was also adopted; therefore, the instructional content was a genetic concept in biology. The study was supported by the social constructivist theory which emphasis that learning should be student-centred through active engagement in the learning process and learning should take place in a social context.

Integrated STEM Approach and Critical Thinking Skills

The integrated STEM approach is characterized by a driving question that will provide the students to think critically for example what do you understand about the problem? This question will stimulate the ability to think. This approach provides the opportunity for students to generate their ideas, find solutions to an open-ended problem, and engage in hands-on and mind-on activities (English, 2016). The approach is also characterized by collaboration among students where students share and justify their ideas. Han et al. (2016) opined that employing an integrated approach in the instructional process that encourages student engagement and exploration in classroom instruction could promote critical thinking skills and the transfer of these skills from the classroom to a real-life situation.

Literature has advocated that scientific thinking can be seen as part of critical thinking skills (Suciati, Ali, Imaningtyas and Anggraini, 2018). Hence the stages of scientific thinking are investigation, analyse, inference, and argument (Kuhn, 2002; Suciati et al., 2018), which are similar to activities than could foster the individual ability to think critically. It is important to highlight that an integrated STEM education approach could provide the latitudes for students to engage in scientific thinking and in the process enhance their critical thinking skills. Watson and Glaser (2008) critical thinking definition and test were adopted for this study.

RESEARCH METHODOLOGY

The quasi-experimental research design was adopted because of the researcher’s inability to employ true randomization as well as control all external variables. In this case, intact classes were used for both the experimental and control group (Creswell, 2015). The population of the study is all unity schools in Niger State, Nigeria. First, a convenient sampling technique was adopted to select two schools from the population, these schools were selected based on similar characteristics such as they are special science schools created by the Nigerian government with well-equipped science laboratories, the method of teachers’ recruitment is the same and each of the schools has well-equipped computer laboratories. Secondly, the selected schools are randomly assigned to an experimental and control group, two intact classes from two schools will form the experimental and control group. A total of 112 students were involved, where 58 (52%) and 54 (48%) students were assigned to the experimental and control group, respectively.

Science Critical Thinking Instrument

Data was collected using the Science Critical Thinking Test (SCTT), the test was made up of five subsections: inference, recognizing assumption, interpretation, deduction, and evaluation. These subskills were adopted from the Watson and Glaser Critical Thinking Appraisal Test (WGCTA), which test is a multiple-choice test (Watson and Glaser, 1980). The SCTT measures an individual’s ability to define a problem, choose important information for a solution to a problem, recognise whether an assumption is made or not, formulate hypotheses, perform evaluation, and draw conclusions. These skills are related to solving a problem in science (genetics); students are presented with a genetic problem, and the students define the problem, formulate hypotheses, generate ideas, evaluate the ideas, and select the best idea to apply. The learners evaluate the entire process and communicate findings. All the questions in the subskills are based on scientific content. The SCTT items were adopted from previous critical thinking tests.

The SCTT was made up of fifty questions (each section has 10 questions). The instrument was validated by psychometry and science education experts and their observations and comments were used to improve the SCTT. The reliability of the subsections of the instrument was between 0.71, and 0.78 which was considered adequate (Sekaran and Bougie, 2010). The pre-test was administered, followed by the intervention, and after the intervention post-test was administered.

Treatment

An instructional material was developed for the implementation of an integrated STEM approach. The instructional process was based on a 5-phased iterative engineering design process: Identify a problem, gather ideas, design a solution, evaluate the solution, and disseminate the findings. This approach is learner-centred while the teacher acts as a facilitator. The facilitator employed question prompts or questioning and clues. The students learn in a small group of five students through collaboration with each other and with the facilitator. At the beginning of the instruction, the students were presented by an open-ended scenario or problem; A problem or scenario that engages students’ ability to think, the problem must have several ways to resolve the problem. The problem should be relevant to learners’ real-life, for example, the genetic problem or scenario in this study. This problem relates to the social and real-life of the learner.

The first phase is understanding or defining the problem which involves understanding the problem through analysing it and identifying the goal of the problem. The second phase, generation of ideas involves researching for scientific knowledge and ideas in the process students learn the instructional content of biology that can be used to solve the problem. The learners brainstorm to establish the right idea or ideas that will be applied in solving the problem. During this phase, students engage in justifying their ideas. The third phase is the design solution, which includes the application of the ideas generated to solve the open-ended problem. The students evaluated the solution developed to determine whether the goal of solving the problem have been achieved and finally disseminate the findings. These activities involve higher cognitive activities which will lead to the enhancement of critical thinking skills. The learning process is characterized by an open-ended problem, driving question and question prompts among others. An example of an open-ended problem is shown in Figure 1.

Figure 1. An example of an open-ended problem

The approach provided the latitudes for students to define the problem, generate and analyse ideas (the students search for materials online and in textbooks to gather information to understand the problem and to provide the solution. The students work in small groups of five (5) students each while teachers serve as the facilitator of the learning process through question prompts, learning clues and reflective questions, among others. The control group were taught using the conventional (traditional method) where the teacher explains the problem, explain the concepts of genetics that will be required to solve the problem. This method is teacher centred and students are not actively engaging in the classroom instruction, compare to the experimental group that is student-centred and learners are actively engage in the learning process. The pre-test was administered before the intervention began and the post-test was administered after the intervention which lasted for eight (8) weeks.

The data collected was analysed using a dependent t-test and a multivariate analysis of variance (MANOVA). The dependent t-test was used to determine the significant difference between the pre- and post-test groups (within-group comparison). While the MANOVA is to determine the effects of independent variables (integrated STEM approach and lecture method) on multiple continuous variables (critical thinking sub-skills).

RESULTS

To determine the equality of the experimental and control critical thinking skills before treatment, critical thinking pre-test was administered, and the data collected was analysed using MANOVA. The pre-test result is presented in Table 1.

Table 1 reveals the Multivariate Analysis of Variance (MANOVA) result of the experimental and control group critical thinking skills pre-test. The results show that there was no significant difference between the experimental and control group in all critical thinking subskills of inference, recognizing assumption, deduction, interpretation, and evaluation Wilks’ \(\Lambda = .93\), \(F(5,\ 94) = 1.370\), \(p = (.24) > 0.05\). Therefore, the two groups were equivalent in their critical thinking skills before treatment. Hence the post-test data will be analysed using MANOVA.

Within-group Comparison of Critical Thinking Skills

To test the formulated hypothesis one, an independent t-test was used to determine if there was any significant difference between the pre-test and post-test of the experimental and control group. The effect size of treatment for each group was also determined and the analysis is presented in Table 2.

Table 2 reveals a significant mean difference can be observed between the pre-test and the post-test mean scores of the experimental in inference subskills of critical thinking skills \(t\left( 57 \right) = - 7.88\), \(p\left( .00 \right) < .05\); \(d^{2} = 1.56\). The effect size was large. On the other hand, there was no significant mean difference in the inference subskill mean score of the traditional group \(t(53) = - .07\), \(p(.94) > .05\), \(d^{2} = 0.01\), the effect size was small. The result also shows a significant mean difference between the mean of pre-test and post-test of the experimental group in recognising assumption \(t(57) = - 5.87\), \(p(.00) < .05\), \(d^{2} = 1.08\); deduction \(t(57) = - 3.26\), \(p(.00) < .05\), \(d^{2} = 0.65\); interpretation \(t(57) = - 7.31\), \(p(.00) < .05\), \(d^{2} = 1.32\); and evaluation \(t(57) = - 4.46\), \(p(.00) < .05\), \(d^{2} = 0.8\). on the other hand there is no significant mean difference in all the critical thinking subskill of the traditional group: recognising assumption \(t(53) = .97\), \(p(.34) > .05\), \(d^{2} = 0.19\); deduction \(t(53) = - 1.15\), \(p(.25) > .05\), \(d^{2} = 0.21\); interpretation \(t(53) = - 1.13\), \(p(.20) > .05\), \(d^{2} = 0.24\); and evaluation \(t(53) = - 1.31\), \(p(.19)\ > .05\), \(d^{2} = 0.25\).

Given the proceeding, there was a significant mean difference between the pre-test and post-test in the overall critical thinking skills of the experimental group \(t(57) = - 11.26\), \(p = .00 < .05\), the overall effect size of the experimental group (\(d^{2} = 1.56\)), indicating large effect size. On the contrary, there was no significant mean difference between the pre-test and post-test in the overall critical thinking skills of the traditional group \(t(53) = - 1.49\), \(p = .14 > .05\), the overall effect size of the traditional group (\(d^{2} = 0.23\)), indicating small effect size. Therefore, in comparison, the experimental had a large effect on enhancing students’ critical thinking skills while the traditional method had a small effect size in helping students to think critically.

Fostering Critical Thinking Skills Using Integrated STEM Approach

To determine the effects of treatment between the experiment and control group (students treated with an integrated STEM approach and a traditional instructional method respectively), among the selected secondary school students. Post-test data was collected and all the assumptions for use of MANOVA were not violated, thus, Wilks’ lambda was used to interpret the results. The result is presented in Table 3.

Table 3 shows MANOVA results of the post-test comparison between the experimental and control group critical thinking skills. The result shows that there is significant difference between the experimental (group that was instructed with integrated STEM approach) and the control group in critical thinking skills, Wilks’ \(\Lambda = .31\), \(F(5,\ 106) = .68\), \(p = 0.00 < 0.05\), partial \(\eta^{2} = .32\). Therefore, the formulated hypothesis (there is no significant mean difference of secondary school students’ critical thinking skill between the experimental and control group) was rejected. The multivariate partial \(\eta^{2}(.324)\) shows that about 32.4% of total multivariate variances of critical thinking scores (dependent variable) is due to the effect of treatment.

Given the findings in Table 3, the univariate tests for each subskill and the overall critical thinking skills result are presented in Table 4.

Table 4 shows the univariate tests for each subskill the result indicates a significant difference between the experimental and control group in all the subskill of critical thinking skills; inference, recognizing assumption, deduction, interpretation, and evaluation. The p-value is less than 0.05 (\(p < 0.05\)). The overall result of treatment between the experimental and control group in the critical thinking skills were \(F(1,\ 110) = 50.47\), \(p = .00 < \ 0.05\), partial \(\eta^{2} = .32\). Indicating a significant difference between the means of the experimental and control group. The experimental group estimated mean is 54.48 which is higher compared to the estimated mean of the control group 43.79, the significant difference was in favour of the experimental group. The multivariate partial \(\eta^{2}(0.32)\) indicates that about 32% of the total variance on the critical thinking skills is attributed to the intervention. The results also indicated that the intervention group perform better than the control group in all the critical thinking sub-skills, inference, recognition of assumption, deduction, interpretation, and evaluation.

Discussion of Results

This study investigated the fostering of critical thinking skills using an integrated STEM approach among secondary school students. The integrated STEM approach provided the leeway to engage in cognitive processes such as drawing an inference, recognizing assumptions, deduction, interpretation, and evaluation. The findings show that the integrated STEM approach students’ critical thinking, the magnitude of the effect size was large. This finding concurs with Oonsim and Chanprasert (2017) determined the development of critical thinking skills among grade 11 students using the STEM education approach, their findings indicated that the critical thinking of the STEM education group was improved. The findings also agree with Sumarni and Kadarwati (2020) who investigated the effects of Ethno-STEM project-based instruction on critical and creative skills among secondary school students, the findings indicated that students’ critical thinking skills were enhanced.

The outcome of the study also indicated that the students that learn with an integrated STEM approach performed better than the comparison in all the subskills of critical thinking skills of inference, recognition of assumption, deduction, interpretation, and evaluation as well as the overall critical thinking skill score. This result agrees with Mater et al. (2020) who conducted a study on the effects of STEM on critical thinking skills and the Technology Acceptance Model, the findings show that the experimental group who learned the STEM approach have improved critical thinking skills compare to the traditional group. This result also agrees with Han et al. (2016) who concluded that STEM-based instruction enhanced students learning outcomes

The findings of the study could be attributed to the open-ended and real-world problem presented to them, which may have provided a meaningful context that engages the students’ higher cognitive abilities such as the generation of ideas and defining problems leading to the development of critical thinking. For example, during the generation of ideas students present their idea and justify their ideas, group members also prompt one another for clarity and explanation. The entire group assessed each idea or claim and draw an early conclusion, thus engaging in evaluation and inference sub-skills. Question prompt by the teacher could have served as a cognitive scaffold that engages the learners’ critical thinking skills. Question prompts are very vital to the development of critical thinking skills and deepen understanding of the instructional content among the students. This corresponds with Kim et al. (2013) reported that good questioning may serve as scaffolds to provide students with the opportunity for deeper thought and enhance the development of critical thinking skills.

CONCLUSION

Given the findings that emerged from the data, it will be logical to conclude that an integrated STEM approach enhanced the learners’ critical thinking skills of this population. This was probably achieved because the students were actively engaged in the learning process through hands-on and minds-on activities, solving open-ended problem and collaboration, among others. In view of the large effect size, integrated STEM is a viable instructional approach that should be included in the curriculum because if properly implemented in the classroom it will enhance students’ thinking skills. The approach is good for human resource development for the 21st century because critical thinking skills is an important skill for meaningful living in this century.

Limitation of the Study

Like any other study, this study is not without limitation, the data collected for this study could be associated with human and could have impacted the outcome of the study. The limitation of the study is in the use of test items that is objective in nature, which could provide the opportunity for students to guess which could impact the findings.

• Bybee, R. W. (2010). Advancing STEM education: A 2020 vision. Technology and Engineering Teacher, 70(1), 30-35.
• Creswell, J. W. (2015). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (5th Ed.). Los Angeles, CA: SAGE.
• Dorouka, P., Papadakis, S. and Kalogiannakis, M. (2020). Tablets and apps for promoting robotics, mathematics, STEM education, and literacy in early childhood education. International Journal of Mobile Learning and Organisation, 14(2), 255-274. https://doi.org/10.1504/IJMLO.2020.106179
• Duran, M. and Sendag, S. (2012). A preliminary investigation into critical thinking skills of urban high school students: Role of an IT/STEM program. Creative Education, 03(02), 241-250. https://doi.org/10.4236/ce.2012.32038
• Duron, R., Limback, B. and Waugh, W. (2006). Critical thinking framework for any discipline. International Journal of Teaching and Learning in Higher Education, 17(2), 160-166.
• Ezeudu, F. O., Ofoegbu, T. O. and Anyaegbunnam, N. J. (2013). Restructuring STM (Science, Technology, and Mathematics) education for entrepreneurship. US-China Education Review A, 3(1), 27-32.
• Facione, P. A. (1998). Critical thinking: What it is and why it counts. California Academic Press. Available at: http://insightassessment.com/pdf_files/whatandwhy98.pdf (Accessed 10 July 2014).
• Facione, P. A. (2011). Measured reasons and critical thinking. Millbrae, CA: The California Academic Press.
• Grosser, M. M. and Nel, M. (2013). The relationship between the critical thinking skills and the academic language proficiency of prospective teachers. South African Journal of Education, 33(2), 1-17. https://doi.org/10.15700/saje.v33n2a639
• Halpern, D. F. (2014). Thought and knowledge: An introduction to critical thinking. New York, NY: Psychology Press.
• Han, S., Capraro, R. M. and Capraro, M. M. (2016). How science, technology, engineering, and mathematics project-based learning affects high-need students in the U.S. Learning and Individual Differences, 51, 157-166. https://doi.org/10.1016/j.lindif.2016.08.045
• Jacobs, H. H. (2010). Upgrading content: Provocation, invigoration, and replacement, in H. H. Jacobs (ed), Curriculum 21: Essential education for a changing world (pp. 30-59). Alexandria, VA: Association for Supervision and Curriculum Development.
• Johns, R. A. (2012). What were they thinking. The Science Teacher, 79(3), 66-70.
• Kalelioğlu, F. and Gülbahar, Y. (2014). The effect of instructional techniques on critical thinking and critical thinking dispositions in online discussion. Educational Technology & Society, 17(1), 248-258.
• Kek, M. Y. C. A. and Huijser, H. (2011). The power of problem‐based learning in developing critical thinking skills: preparing students for tomorrow’s digital futures in today’s classrooms. Higher Education Research & Development, 30(3), 329-341. https://doi.org/10.1080/07294360.2010.501074
• Kim, K., Sharma, P., Land, S. M. and Furlong, K. P. (2013). Effects of active learning on enhancing student critical thinking in an undergraduate general science course. Innovative Higher Education, 38, 223-235. https://doi.org/10.1007/s10755-012-9236-x
• Kivunja, C. (2015). Exploring the pedagogical meaning and implications of the 4Cs “super skills” for the 21st century through Bruner’s 5E lenses of knowledge construction to improve pedagogies of the new learning paradigm. Creative Education, 6(2), 224-239. https://doi.org/10.4236/ce.2015.62021
• Kuhn, D. (2002). What is scientific thinking and how does it develop? in U. Goswami (ed), Blackwell handbook of childhood cognitive development (pp. 371-393). Hoboken, NJ: Wiley-Blackwell Publishing. https://doi.org/10.1002/9780470996652.ch17
• Kuhn, D. (2010). What is scientific thinking and how does it develop? Blackwell handbook of childhood cognitive development (2^nd ed.). New York; NY: Columbia University. https://doi.org/10.1002/9781444325485.ch19
• Mater, N. R., Hussein, M. J. H., Salha, S. H., Draidi, F. R., Shaqour, A. Z., Qatanani, N. and Affouneh, S. (2020). The effect of the integration of STEM on critical thinking and technology acceptance model. Educational Studies, 48(5), 642-658. https://doi.org/10.1080/03055698.2020.1793736
• Oonsim, W. and Chanprasert, K. (2017). Developing critical thinking skills of grade 11 students by STEM education: A focus on electrostatic in physics. Rangsit Journal of Educational Studies, 4(1), 54-59.
• P21. (2015). Framework for 21st century learning. The partnership for 21st century skills. http://www.p21.org/about-us/p21-framework
• Papadakis, S., Vaiopoulou, J., Sifaki, E., Stamovlasis, D., Kalogiannakis, M. and Vassilakis, K. (2021). Factors that hinder in-service teachers from incorporating educational robotics into their daily or future teaching practice, in B. Csapó and J. Uhomoibhi (eds), Proceedings of the 13^th International Conference on Computer Supported Education (pp. 55-63). ScitePress. https://doi.org/10.5220/0010413900550063
• Phonchaiya, S. (2014). STEM and advanced thinking. IPST Magazine, 42(189), 7-10.
• Pickering, D. (2010). Teaching the thinking skills that higher-order tasks demand, in R. Marzano (ed), On excellence in teaching. Bloomington, IN: Solution Tree Press.
• Pitan, O. S. and Adedeji, S. O. (2012). Skills mismatch among university graduates in the Nigeria labor market. US-China Education Review A, 1(2012) 90-98.
• Retnowati, S., Riyadi and Subanti, S. (2020). The STEM approach: The development of rectangular module to improve critical thinking skill. International Online Journal of Education and Teaching, 7(1), 2-15.
• Salami, C. G. E. (2013). Youth unemployment in Nigeria: A time for creative intervention. International Journal of Business and Marketing Management, 1(2), 18-26.
• Schulz, H. W. and FitzPatrick, B. (2016). Teachers’ understandings of critical and higher order thinking and what this means for their teaching and assessments. Alberta Journal of Educational Research, 62(1), 61-86.
• Smith, T. E., Rama, P. S. and Helms, J. R. (2018). Teaching critical thinking in a GE class: A flipped model. Thinking Skills and Creativity, 28, 73-83. https://doi.org/10.1016/j.tsc.2018.02.010
• Stapleton, P. (2011). A survey of attitudes towards critical thinking among Hong Kong secondary school teachers: Implications for policy change. Thinking Skills and Creativity, 6, 14-23. https://doi.org/10.1016/j.tsc.2010.11.002
• Suciati, M. N., Ali, C. D., Imaningtyas, A. F. and Anggraini, Z. D. (2018). The profile of XI grade students’ scientific thinking abilities on scientific approach implementation. Jurnal Pendidikan IPA Indonesia [Indonesian Science Education Journal], 7(3), 341-346. https://doi.org/10.15294/jpii.v7i3.15382
• Sumarni, W. and Kadarwati, S. (2020). Ethno-STEM project-based learning: Its impact to critical and creative thinking skills. Jurnal Pendidikan IPA Indonesia [Indonesian Science Education Journal], 9(1), 11-21. https://doi.org/10.15294/jpii.v9i1.21754
• Thibaut, L., Ceuppens, S., De Loof, H., De Meester, J., Goovaerts, L., Struyf, A., . . . Depaepe, F. (2018). Integrated STEM education: A systematic review of instructional practices in secondary education. European Journal of STEM Education, 3(1), 1-12. https://doi.org/10.20897/ejsteme/85525
• Wagner, T. (2008). The global achievement gap: Why even our best schools don’t teach the new survival skills our children need and what we can do about it. New York: NY: Basic Books.
• Wartono, W., Hudha, M. N. and Batlolona, J. R. (2018). How are the physics critical thinking skills of the students taught by using inquiry-discovery through empirical and theoretical overview? EURASIA Journal of Mathematics, Science and Technology Education, 14(2), 691-697. https://doi.org/10.12973/ejmste/80632
• Watson, G. and Glaser, E. M. (1980). Watson-Glaser critical thinking appraisal manual. San Antonio, TX: The Psychological Corporation.
• Watson, G. and Glaser, E. M. (2008). Watson-Glaser critical thinking appraisal: Forms A and B manual. Upper Saddle River, NJ: Pearson Education.
• Watson, G. and Glaser, E. M. (2010). Watson-Glaser II critical thinking appraisal®. USA: NCS Pearson.