Integrated STEM Education: A Systematic Review of Instructional Practices in Secondary Education

The shortage of graduates in Science, Technology, Engineering and Mathematics (STEM), has led to numerous attempts to increase students’ interest in STEM. One emerging approach that has the potential to improve students’ motivation for STEM is integrated STEM education. Nonetheless, the implementation of this new instructional strategy is not straightforward due to the lack of consensus about instructional practices in integrated STEM. This paper contributes to this challenge by providing a well-defined framework for instructional practices in integrated STEM in secondary education, based on the results of a systematic review of existing literature. The framework contains five key principles: integration of STEM content, problem-centered learning, inquiry-based learning, design-based learning and cooperative learning. The proposed framework has several benefits, including its applicability in the classroom and the possibility to describe integrated STEM on multiple dimensions. Nonetheless, further research is necessary to investigate the effects of integrated STEM on students’ cognitive and affective learning outcomes.


INTRODUCTION Need for Integrated STEM
In recent years, the importance of providing students with a strong education in Science, Technology, Engineering and Mathematics (STEM) has been stressed. Qualified STEM professionals are needed to remain economically competitive in the global market and to fill contemporary demands such as ensuring sufficient and sustainable energy, efficient healthcare and well-considered technology development (Bøe et al., 2011). Moreover, all citizens, even non-STEM professionals, should have the skills and competences necessary to deal with the challenges of our information-based and highly technological society (National Society of Professional Engineers, 2013). STEM-literacy, i.e. the awareness of the nature of science, technology, engineering, and mathematics and the familiarity with some of the fundamental concepts from each discipline, should be an educational priority for all students (Bybee, 2010;National Academy of Engineering and National Research Council, 2014).
A promising approach in this regard, is the use of an integrated STEM curriculum, which provides opportunities for 'more relevant, less fragmented, and more stimulating experiences for learners' (Furner and Kumar, 2007, p.186). Real-world problems are not fragmented in isolated disciplines as they are taught in schools and to solve these problems people need skills that cut across the disciplines (Beane, 1995;Czerniak et al., 1999). Studies in a broad range of disciplines have shown that students involved in an integrated curriculum perform as well or even better than their peers in traditional instruction with separate disciplines (Czerniak et al., 1999;Hinde, 2005). Moreover, the use of an integrated curriculum has been found to improve students' non-cognitive learning outcomes, such as interest in STEM (Mustafa et al., 2016;Riskowski et al., 2009) and motivation towards STEM learning (Wang et al., 2011), which in turn could lead to increasing numbers of STEM graduates (National Academy of Engineering and National Research Council, 2014).

Current Challenges
Despite the potential benefits and the increased focus on integrated STEM education, implementation of this new instructional strategy faces several challenges. First of all, implementing an integrated STEM approach in an educational system that has a very established segregated and discipline-based structure requires profound restructuring of the curriculum and lessons (Nadelson and Seifert, 2017). Moreover, integrated STEM education often requires numerous materials and resources for students such as construction tools (e.g., saws, measuring devices, and hammers), electronic materials (e.g., computers, design programs, robotics kits, and calculators) and other materials used in design (e.g., wood, styrofoam, glue, cardboard, or construction paper) (Stohlmann et al., 2012). Therefore, creating a school culture and environment that supports an integrated STEM approach to teaching and learning can be costly and time-consuming (Hardy, 2001;Nadelson and Seifert, 2017).
Furthermore, to effectively implement integrated STEM, teachers must have deep knowledge of the science, technology, engineering and mathematics content that they teach (Eckman et al., 2016). Additionally, they must also have specialized knowledge of how to teach STEM content to students-i.e., pedagogical content knowledge (Shulman, 1987). Nonetheless, many teachers report that they feel underprepared to use STEM applications with their students in the classroom (El-Deghaidy and Mansour, 2015). Moreover, a study by El-Deghaidy and Mansour (2015) showed that teachers do not have sufficient understanding of the T in STEM and that they may not have an adequate understanding of the nature of science and technology and the interactions between these two disciplines. Additionally, teachers' beliefs and views about teaching and learning, and their resistance or lack of motivation to change their beliefs and practice, may pose another challenge to the implementation of integrated STEM education (Ashgar et al., 2012).
In addition to the challenges of finding resources and insufficient teacher' knowledge, another major challenge for the implementation of integrated STEM education is the lack of consensus about how integrated STEM learning and teaching should be done. In recent years, many researchers have provided detailed information about the learning units for integrated STEM they designed. However, they often do not explain the instructional principles guiding their design (e.g. Barrett et al., 2014;Gentile et al., 2012). By contrast, other researchers have extensively described their ideas about instructional practices in integrated STEM. However, they often fail to provide a theoretical foundation for these practices chosen (e.g. Moore et al., 2014;Sanders, 2009) and different researchers seem to not always agree about the core instructional practices for teaching integrated STEM. Although several reviews about integrated STEM have been conducted (e.g., Becker and Park, 2011;Gresnigt et al., 2014), they mostly focus on the effects of integration on students' learning outcomes. To our knowledge, no systematic review about instructional practices in integrated STEM education has been done.

Purpose and Research Questions of This Study
The current study aims at addressing the lack of consensus about how integrated STEM learning and teaching should be done. To do so, a systematic review of existing literature about learning theories for and instructional practices in integrated STEM was conducted. Moreover, based on the results of the systematic review, a framework for instructional practices in integrated STEM was constructed. By first conducting a systematic review of existing literature, a theoretical base for the instructional practices in the framework is provided. In the absence of empirical evidence discerning which instructional elements are crucial for the successful implementation of integrated STEM, this is a valuable alternative to provide some form of evidence-based foundation. The specific research questions addressed in the systematic review are: 1. Which learning theories (behaviorism, cognitivism, social constructivism) form the basis of integrated STEM education? 2. Which instructional practices are used in integrated STEM in secondary education?

Methodology
A systematic review was conducted to identify, critically evaluate and summarize the findings of all relevant studies describing learning and teaching in integrated STEM (Gopalakrishnan and Ganeshkumar, 2013). Systematic reviews differ from narrative reviews, which tend to be mainly descriptive and usually focus on a subset of studies that were chosen based on availability or author selection and therefore often include an element of selection bias. By contrast, systematic reviews typically involve a detailed and comprehensive search strategy derived a priori, with the goal of reducing bias by identifying, appraising, and synthesizing all relevant studies on a particular topic (Uman, 2011). For the current study, a systematic review was done using two databases: ERIC and Web of Science. The databases were browsed by using four different combinations of search terms: "Integrated STEM + secondary education" (n = 184), "Interdisciplinary STEM + secondary education" (n = 86), "Multidisciplinary STEM + secondary education" (n = 14), and "STEM integration + secondary education" (n = 121), leading to a total of 405 search hits. Overlapping search results were excluded and the dataset was further reduced using four criteria.
Firstly, all selected articles had to be peer-reviewed journal articles or book chapters written in English, resulting in the exclusion of, for example, conference papers and dissertations. Secondly, the articles had to focus on the integration of at least three of the STEM disciplines. Articles studying the combination of only two disciplines, for example, interdisciplinary mathematics and science or technology integration in mathematics, were excluded. Thirdly, the articles had to provide a clear description of teaching practices for integrated STEM. Finally, all articles had to describe instructional practices for integrated STEM in secondary education. One rater checked whether the articles met the criteria and in case of doubt, papers were discussed with two other raters until consensus was reached. After applying the criteria only 15 articles remained in the sample. Given the small number of articles, the "snowball approach" was used to retrieve additional publications (Doust et al., 2005). Reference lists of all selected articles were inspected and eight additional relevant articles that met the inclusion criteria were added to the dataset, resulting in a total of 23 articles.
To analyse the articles in the dataset, first a within-case analysis was conducted (Miles and Huberman, 1994). Each article was analysed separately and summarized in a table, consisting of two categories: (1) theoretical grounding and (2) instructional practices. Secondly, a cross-case analysis was done (Miles and Huberman, 1994). The instructional practices extracted from all articles were rearranged and similar elements were grouped, leading to nine different categories. Next, the theoretical framework was composed by focusing on the learning theory and instructional categories named most frequently in the articles of the systematic review.

Theoretical Grounding
Learning theories are typically divided in three categories: behaviorism, cognitivism and social constructivism (Ertmer and Newby, 2013;Greeno et al., 1996). Behaviorism is centered around the idea that one behavior leads to another and does not take into consideration the functions of the mind (Dilshad, 2017;Stavredes, 2011). According to behaviorism, knowledge is an organized accumulation of associations and skills that exists outside a person and can be gained through behavior modification (Greeno et al., 1996;Stavredes, 2011). Therefore, learning is seen as a change in behavior that can be conditioned using positive and negative reinforcements such as reward and punishment (Stavredes, 2011).
By contrast, cognitivism focuses on students' internal mental structures and addresses the issues of how information is received, organized, stored, and retrieved by the mind (Ertmer and Newby, 2013). In accordance with behaviorism, knowledge is considered to exist outside of the person. However, unlike behaviorism, cognitivism focuses on understanding how human memory works to acquire knowledge and promote learning, instead of focusing on behavior (Stavredes, 2011). Therefore, learning is seen as the discrete changes between states of knowledge, rather than changes in behavior. Moreover, learning is described as a mental activity that entails internal coding and structuring by the student and the student is viewed as an active participant in the learning process (Ertmer and Newby, 2013).
Both behaviorism and cognitivism start from the underlying assumption that learning is an individual process (Ertmer and Newby, 2013). By contrast, social constructivism states that learning is socially situated and knowledge is constructed through interaction with others (McKinley, 2015). According to social constructivism, students learn by building personal interpretations of the world based on their experiences and interactions with the environment (Ertmer and Newby, 2013;Stavredes, 2011). Therefore, in accordance with cognitivism, the student is seen as an active participant in the learning process (Ertmer and Newby, 2013).
To answer the question: "Which learning theories form the basis of integrated STEM education?", the articles in the dataset were examined for references to learning theories. Of the 23 papers, only seven mentioned an underlying learning theory for instructional practices in integrated STEM: Clark and Ernst (2007) Wang et al. (2011). Moreover, all seven articles referred to learning theories from the social constructivist category. As mentioned above, this category of learning theories states that knowledge cannot be transmitted, but rather is actively constructed by students based on their existing ideas and their experiences (Driver et al., 1994;Eastwell, 2002). Moreover, the social aspect of social constructivism dictates that learning is a shared, rather than an individual experience (Prawat and Floden, 1994).

Integration of STEM content
-Multidisciplinary approach -Interdisciplinary approach -Content integration -Context integration -Integrated curriculum with equal attention to two or more disciplines -Curriculum integration with focus on content knowledge -Explicit assimilation/ integration of concepts from more than one discipline.
-Integration of technology -Translation of representations from different STEM disciplines -Connections among learning goals, principles, concepts and skills across discipline specific domains -Infusing/merging of two or more STEM content areas

Focus on problems
-Problem-based learning -Problem-centered learning -Project-based learning -Defining, formulating, evaluating and solving problems -Meaningful/motivating/engaging context -Focusing on big ideas, concepts, themes -Open-ended, real-world, authentic problems

Inquiry
-Posing questions -Planning and carrying out investigations -Collecting, analysing and interpreting data/information -Discovery learning -Inquiry-based instruction -Scientific inquiry -Authentic scientific practices/processes

Instructional Practices
Instructional practices were extracted from all papers and similar elements were grouped, as shown in Table 1. This led to nine categories of instructional practices: integration of STEM content, focus on problems, inquiry, design, teamwork, student-centered, hands-on, assessment and 21 st century skills.
A summary of the different instructional categories found in each article is shown in Table 2 and discussed more in detail below.
Integration of STEM content. The first category contains instructional practices that advocate making connections between the different STEM disciplines. Several approaches for achieving these connections exist and the terminology used for these approaches differs among the reviewed articles. For example, Wang et al. (2011) distinguish between multidisciplinary and interdisciplinary approaches. According to them, in a multidisciplinary approach, subject-specific concepts and skills are learned separately in each discipline and students are expected to connect the content, taught in different classrooms, on their own. An interdisciplinary approach, on the other hand, starts with a real-world problem or issue and focuses on interdisciplinary content and skills (e.g., critical thinking and problem solving), rather than subject-specific content and skills. By contrast, Satchwell and Loepp (2002), provide a different definition for interdisciplinarity. Moreover, they discriminate interdisciplinary approaches from integrated approaches, rather than multidisciplinary approaches. According to them, interdisciplinary curricula focus on instruction within one domain, while supporting the content with implicit connections to the other disciplines. Integrated curricula, on the other hand, explicitly assimilate concepts from more than one discipline and apply equal attention to two or more disciplines. A similar distinction is made by Roehrig et al. (2012) who differentiate between content and context integration. According to them, content integration focuses on the merging of the disciplines into a single curricular activity or unit to highlight "big ideas" from multiple content areas, while context integration focuses on the content of one discipline and uses contexts from others to make the content more relevant.
Although there is no consensus about the terminology, most of the reviewed articles do agree that making strong connections between the different STEM-disciplines is necessary for integrated STEM to work. Many papers (e.g., Satchwell and Loepp, 2002;Shahali et al., 2016;Stump et al., 2016) emphasize the importance of applying equal attention to two or more STEM disciplines and/or explicitly assimilating concepts from different STEM-disciplines. Making integration explicit is critical, because students do not spontaneously integrate concepts across different representations and materials on their own (Pearson, 2017). Therefore, intentional and explicit support to help students build knowledge and skills across disciplines should be provided (Pearson, 2017). Nonetheless, although the importance of explicitly integrating large amounts of STEM content has been stressed, several papers (e.g., Pearson, 2017) warn that integration should remain meaningful and purposeful and that more integration is not necessarily better. Moreover, students' knowledge in individual disciplines should be supported. Students require sufficient understanding of the relevant concepts in the individual subjects to connect ideas across disciplines (Pearson, 2017). Therefore, integrated STEM education should also focus on learning goals and standards in the individual STEM subjects, so as not to inadvertently undermine student learning in those subjects (Pearson, 2017). Focus on problems. The second category entails the use of real-world problems tied to an engaging and motivating context. In the reviewed papers, different terms are used to designate this: problem-centered learning, problem-based learning and project-based learning. Although all of these approaches are student-centered, promote active learning and advocate the use of authentic real-world problems, specific differences between these instructional approaches exist (Ashgar et al., 2012). In project-based learning, students are provided with the desired specifications for the end product and teachers serve as expert coaches that provide guidelines and suggestions for more effective ways to achieve this predetermined final product (Ashgar et al., 2012). By contrast, in problem-based learning, there is no predetermined end product and students are required to identify and define the problem on their own. The goal of problem-based learning is for students to develop problem-solving skills by going through a realistic self-directed problem-solving process. Therefore, the teacher does not provide specific learning guidance, but rather (s)he serves as one possible resource that students can use to achieve their goal (Ashgar et al., 2012). Finally, problem-centered learning holds the middle between project-based and problembased learning regarding teacher guidance. Although the problem is more open-ended compared to project-based learning, problem-centered learning focuses mainly on applying and transferring knowledge to realistic contexts, while problem-solving skills are seen as an additional outcome (Merrill, 2007;van Merriënboer and Kirschner, 2007). Therefore, in contrast to problem-based learning, a carefully sequenced progression of problems and specific types of guidance are provided (Merrill, 2007;van Merriënboer and Kirschner, 2007).
Nonetheless, despite the different terminology, the same important aspects are emphasized in the different articles. First of all, instruction should start by presenting students with a problematic situation that serves as the organizing center and context for learning (Ashgar et al., 2012;Bybee, 2010). By introducing the problem at the start of the learning unit, students can activate existing mental models early in the learning sequence and connect new information and experiences to their prior knowledge in a meaningful way (Ashgar et al., 2012). Moreover, instruction should take place in a motivating and engaging context involving current events and/or contemporary issues. That way, students can link the knowledge and skills to be learned to their personal experiences and meaningful learning is encouraged . Finally, the problems presented to the students should be authentic, open-ended, ill-structured, real-world problems (Burrows et al., 2014;Satchwell and Loepp, 2002;Shahali et al., 2017). These are 'messy' problems, often with an abundance of information, that resemble challenges encountered by engineers and scientists in the workplace and allow for multiple solution paths and answers (Ashgar et al., 2012). Inquiry. A third category entails instructional practices referring to the use of inquiry. In inquiry-based learning, students engage in hands-on activities that allow them to discover new concepts and develop new understandings (Satchwell and Loepp, 2002). Thus, experiental learning is intentionally used to promote knowledge construction (Wells, 2016) and students are encouraged to test their existing ideas by taking things apart, making predictions, observing and recording their explanations (Satchwell and Loepp, 2002). Although inquiry-based learning originated in science education, where it usually entails that students have to engage in authentic science practices (e.g., planning and designing experiments and collecting data), it is not restricted to this domain, but also occurs in mathematical or technological contexts (Satchwell and Loepp, 2002).
In the reviewed papers, several important aspects of inquiry-based learning are mentioned. First of all, questioning is an important part of inquiry-based learning, because it initiates all knowledge building (Wells, 2016). Students are stimulated to question their current knowledge about a given topic and to identify which additional knowledge they require to move forward (Stump et al., 2016;Wells, 2016). Secondly, students should use this prior knowledge to generate new ideas, design and conduct investigations and discover new concepts. Moreover, they not only need to carry out the experiments, students also need to demonstrate their understanding of the concepts explored (Satchwell and Loepp, 2002). Finally, an appropriate amount of guidance should be provided to help students achieve the desired conceptual change (James et al., 2000;Satchwell and Loepp, 2002). Truly authentic inquiry experiences might be challenging for high school students, due to their lack of experience and knowledge, or because they have not reached the level of cognitive development required for abstract thought (Lawson, 1980;Purser and Renner, 1983). Moreover, pure discovery learning without guidance might be ineffective because students may not come into contact with the to-be-learned content (Mayer, 2004). Therefore, teachers need to provide guidance by questioning students to help them discover flaws in their reasoning and/or research design, ultimately helping them to arrive at a solution (Buck et al., 2008).
Design. The fourth category refers to the use of technological or engineering design. By actively engaging students in engineering design challenges, they not only learn about engineering design processes and engineering practices, but also deepen their understanding of disciplinary core ideas Hernandez et al., 2013;Shahali et al., 2016). Engineering design activities can strengthen students' knowledge of science, technology and mathematics, because they fill the gap between factual content knowledge, abstract knowledge and application (Riskowski et al., 2009).
In the reviewed papers, several aspects of qualitative engineering design are mentioned. First of all, effective design challenges should be open-ended, authentic, hands-on, and multidisciplinary (Shahali et al., 2016). These are challenges that represent industry problems, allow students to explore or develop technologies and require them to work with incomplete information and to consider constraints, safety, risks, and alternative solutions . Moreover, the engineering design process should entail different iterative phases, such a defining the engineering problem, designing engineering solutions, implementing a solution, testing the solution and evaluating and optimizing the solution (Bryan et al., 2015;Wells, 2016). Furthermore, throughout the design process, students must manage risk and uncertainty, consider prior experience and learn from failure (Bryan et al., 2015;. In addition, a final requirement for qualitative engineering design is the incorporation of design justification (Bryan et al., 2015;Wells, 2016). Students should be able to demonstrate what they have learned in the design process, justify their design decisions by means of this newly acquired knowledge and make recommendations about the design, based on the results of their tests (Bryan et al., 2015;Wells, 2016). This design justification serves as an effective mechanism for revealing student development and therefore facilitates conceptual change (Wells, 2016).
Cooperative learning. The fifth category entails the promotion of teamwork and collaboration with others. In the reviewed papers, two approaches to small-group learning are distinguished: collaborative and cooperative learning. Although the papers in the review not explicitly define both approaches, other researches have appointed their difference in teacher guidance as main discriminating factor. In collaborative learning, students structure their own group work, without receiving formal training in small-group social skills. The teacher does not actively monitor the groups and refers all questions back to them, because he wants students to resolve group conflicts on their own (Matthews, 1995). In cooperative learning on the other hand, the teacher moves from team to team, observes the interactions and intervenes when he feels it is appropriate. Moreover, training in small-group social skills is provided and the teacher encourages students to assess the functioning of the group in order to improve the levels of participation and performance (Matthews, 1995).
Although almost all papers in the systematic review refer to the use of teamwork in integrated STEM, the description of how this teamwork should be organized or what the crucial aspects are, is much less elaborated compared to the previous categories. According to , it is important for students to have sufficient time and multiple opportunities to be involved in teamwork, such that they can improve their teamwork skills. Moreover, several authors (e.g., Bryan et al., 2015;Roehrig et al., 2012;Stohlmann et al., 2011) stress the importance of not only stimulating teamwork skills, but fostering communication skills as well. To do so, students need to be encouraged to communicate science concepts, mathematical and engineering thinking through reading, writing, listening and speaking (Stohlmann et al., 2011). Finally, positive interdependence between the group members should also be stimulated (Ashgar et al., 2012). This means that students should work on tasks that they believe are only soluble if all group members contribute to the effort. Positive interdependence can be achieved through: (1) providing rewards for successful interdependence; (2) having activities in which resources are shared; or (3) providing a task that is too difficult for students to do individually (Johnson and Johnson, 1999).
Student-centered. The next category refers to the use of student-centered pedagogies and was only present in four of the reviewed articles.  indicate that lessons and activities in an integrated STEM unit should be student-centered, because students develop better understanding and skills through active participation in learning activities. Nonetheless, specific guidelines for implementing student-centered learning are not discussed in the reviewed articles.
Hands-on. The instructional practices in the seventh category refer to the use of hands-on learning, hands-on activities and manipulatives. Through hands-on activities, students are less restricted and can actively experience learning. Moreover, relevance is conveyed through hands-on learning, because it allows students to observe the role of innovation in everyday life (Clark and Ernst, 2007). As with student-centered learning, no specific recommendations for implementing hands-on instruction are given in the reviewed papers.
Assessment. The eighth category deals with assessment and was only mentioned in three of the reviewed articles. According to Stohlmann et al. (2012), assessment should be used as a part of instruction. Moreover, Satchwell and Loepp (2002) dictate that assessments should present students with authentic tasks that require them to connect all key concepts studied in mathematics, science, and technology and should include a scoring rubric. 21 st century skills. The final category comprises '21 st -century skills', referring to the knowledge, skills, and character traits that are deemed necessary to effectively function as citizens, workers, and leaders in the 21 st -century workplace (Bryan et al., 2015). They were only explicitly mentioned in one of the reviewed articles, i.e. Bryan et al. (2015). Although several lists of 21 st century skills exist, the ones named by Brian et al. (2015) include creativity and innovation, critical thinking, problem solving, communication and collaboration.

DISCUSSION AND CONCLUSION A Framework for Teaching Integrated STEM
Based on the results of the systematic review, a theoretical framework for instructional practices in integrated STEM for secondary education was created. To do so, the five categories of instructional elements most commonly present in the papers of the systematic review were selected. These categories were deemed most essential for teaching integrated STEM. Moreover, when different instructional practices were named within the same category (e.g., cooperative learning versus collaborative learning), the strategy with the most emphasis on teachers' guidance was chosen. This approach was used, because the theoretical framework targets secondary students. Since their knowledge and experience is limited, they might need sufficient teacher assistance to achieve conceptual change (De Groof et al., 2012). The final framework consists of five distinctive but related key principles: integration of STEM content, problem-centered learning, inquiry-based learning, design-based learning and cooperative learning. All these principles are rooted in a social constructivist view on learning. An overview of the framework is given in Figure 1.
The first principle, integration of STEM content, refers to the explicit assimilation of learning goals, content and practices from different STEM disciplines. Due to the lack of consensus about terminology, no specific strategy (e.g., multidisciplinary, interdisciplinary or integrated learning) is proposed. Problem-centered learning, the second principle, indicates that learning environments should involve students in authentic, open-ended, illstructured, real-world problems to increase the meaningfulness of the content to be learned. The third principle, inquiry-based learning, refers to learning environments that engage students in questioning, experiental learning and hands-on activities that allow them to discover new concepts and develop new understandings. Design-based learning, the fourth principle, entails the use of open-ended, hands-on design challenges that provide students with the opportunity to not only learn about engineering design processes and engineering practices, but also deepen their understanding of disciplinary core ideas. Finally, the principle of cooperative learning indicates that students should get the opportunity to communicate and collaborate with each other to deepen their knowledge. All key principles are supported by a social constructivist view on learning, which dictates that knowledge is actively constructed by the students and that learning is a shared, rather than an individual experience.
Although the other instructional categories distinguished in the systematic review were not explicitly included in the framework, this does not mean they are considered to be unimportant. Rather, we believe that many of these categories are overarching aspects that are inextricably linked with one or more of the discerned key principles. For example, both problem-centered learning and inquiry-based learning are student-centered pedagogies and both inquiry-based and design-based learning advocate the use of hands-on learning. Moreover, the 21 st century skills named by Bryan et al. (2015) include creativity and innovation, critical thinking, problem solving, communication and collaboration. Hence, although the importance of developing these skills is not denied, they are already largely present in the other key principles, defying the need for a separate category. In addition, assessment is not explicitly incorporated in the framework, because it is an indispensable aspect of any instructional approach (Black and William, 1998) and not specifically linked to integrated STEM education.

Benefits
The framework proposed in this paper is a valuable contribution to the challenge of implementing integrated STEM education and has several benefits. First of all, in contrast to many studies that reduce the instructional strategy for integrated STEM to either inquiry-based learning, problem-based learning or design-based learning (Moore et al., 2014;Mustafa et al., 2016;Sanders, 2009), this framework allows for a more specific and detailed description of instructional practices through the formulation of five key principles. As can be seen by the description of the principles, they are complementary, but also partially overlapping. For example, teacher guidance is an important aspect of both problem-centered and inquiry-based learning and the use of open-ended, authentic problems is advocated in both problem-centered and design-based learning. Nonetheless, all key principles have their own specific characteristics, making it worthwhile to include them all in the framework. Secondly, as opposed to other researchers who have described instructional practices for integrated STEM and failed to explain how the instructional elements were derived (e.g. Bryan et al., 2015), the current framework is the result of a systematic review of existing literature. In the absence of empirical research about which elements are crucial for the successful implementation of integrated STEM, this is a valuable alternative to provide some form of evidence-based foundation. Finally, the framework presented in this paper is widely applicable and can be used both in teaching practice and in educational research. While teachers and educators can use the five principles to develop curriculum materials for integrated STEM in a fine-grained manner, educational researchers can employ them to analyse instructional practices in more detailed and specific ways.

Limitations
Despite the benefits, the current study and resulting framework also have some limitations. Although a systematic review was done to reduce selection bias, it is still possible that other relevant articles exist that were not included in the dataset. For example, we chose to focus solely on integrated STEM in secondary education. Therefore, other interesting articles about related subjects, such as integrated STEM in primary education (e.g., Gresnigt et al., 2014) or design-based learning in higher education (e.g, Gomez Puente et al., 2013) were not taken into account. Moreover, the conclusions of the review are based on a small dataset and many of the articles have one or several authors in common, which could partially explain why the same instructional categories were named in the different papers. Finally, although the framework can be used as a guideline for designing and implementing curriculum materials for integrated STEM, implementation of the five key principles is not straightforward. For example, creating meaningful connections between STEM disciplines is a challenging task. It requires an iterative act of matching and reorganizing learning goals between the different disciplines, searching an appropriate sequence for these goals and incorporating new learning goals. Therefore, teachers need to be willing to invest time and effort in the implementation of integration. Moreover, letting students conduct their own experiments or build a prototype requires extra time. Hence, teachers might struggle to incorporate new instructional practices, such as inquiry-based and design-based learning, since they usually need all their lessons to finish the traditional curriculum in time. Additionally, creating an interdisciplinary learning approach requires cooperation between teachers of the different STEM disciplines, which demands sufficient human, financial, material and infrastructural resources.

Perspectives for Future Research
Although this paper provides an important step towards improving the implementation of integrated STEM education, the need for further research exists. Empirical research is needed to confirm the validity of a framework consisting of five key principles. Next, systematic examination of the effects of integrated STEM, implemented according to this framework, on students' cognitive and affective learning outcomes is required. Moreover, looking at each key principle's individual impact on these learning outcomes could provide information about the necessity of each principle and lead to a refinement of the framework. Finally, the influence of different factors (e.g., teachers' attitudes, school context) on the implementation of integrated STEM could be examined. Insight into these factors could help to improve the implementation of integrated STEM education, therefore ultimately contributing to students' increased motivation for STEM.