The Process of Designing Integrated STEM Learning Materials: Case Study towards an Evidence-based Model

The Process of Designing Integrated STEM Materials: Study Model. ABSTRACT This study models the process of designing learning materials for integrated STEM (iSTEM) in secondary education, as gone through by four multidisciplinary teams of STEM teachers in Flanders (Belgium). In order to identify the crucial, counterproductive, and missing activities in the observed design processes, the learning materials developed by each team were assessed with respect to the key principles of iSTEM education. Crucial activities in the observed design processes appeared to be the formation of a multidisciplinary design team, a brainstorm on themes, the identification and linking of STEM contents, and the exploration of a feasible, engaging student challenge with interrelated subproblems. The observed processes lacked a specification of the targeted learning goals and the inclusion of research-based instructional strategies aiming at these goals. The evidence-based model of the iSTEM design process resulting from this study can fuel teacher training programs and empower pre- and in-service teachers to create high-quality integrated STEM education. on our study. our


INTRODUCTION
An integrated, problem-centered approach to secondary STEM education is believed to be the key to piquing students' interest and at the same time fostering their robust understanding of STEM subjects (Kennedy et al., 2014). STEM teachers have a key role in establishing such integrated STEM (iSTEM) education, but are not trained nor supported to do so (Shernoff et al., 2017;Dare et al., 2018). The research literature on educational innovation (Voogt et al., 2011;Parke et al., 1997) suggests that teachers will be better prepared for implementing iSTEM education when they are at the driver's seat not only of its implementation, but of its design as well.
With these aspirations for secondary STEM education, the 4-year research project STEM@school was launched in Flanders (Belgium) in 2014 (De Meester et al., 2020). STEM@school engaged 30 teams of STEM teachers to implement iSTEM learning materials in their classroom, 10 of which were also involved as teacher design teams (TDTs) in the design of these learning materials. At the start of STEM@school, no concept for effective iSTEM education or learning materials tailored to the Flemish education system were available. In the scope of STEM@school, the study presented in this article investigated the activities teachers undertake to integrate learning contents from separate STEM subjects into the curricula of grades 9 to 12.

THEORETICAL FRAMEWORK iSTEM Key Principles and Requirements
From a comprehensive review of literature on integrated STEM, Thibaut et al. (2018) extracted four key principles that constitute effective iSTEM education: (1) problem-centered learning, (2) inquiry-and design-based learning, (3) integration of STEM learning contents, and (4) cooperative learning. To endorse the value of longlasting (educational) research in each of the different STEM disciplines (NRC, 2012), we add the extra principle of research-based learning. As a result, we advocate the five key principles for establishing high-quality iSTEM education in secondary school classrooms shown at the left of Figure 1 (De Meester et al., 2020). Based on the research literature on instructional design for integrated STEM education (NAE et al., 2014;Felder et al., 2016), we translated these five key principles into 10 requirements for high-quality iSTEM education, shown at the right of Figure 1.

iSTEM Curriculum Design
An effective way to prepare teachers for implementing a new instructional approach is to immerse them in the key principles underpinning this new approach (Loucks-Horsley et al., 2009). This particular strategy of active teacher involvement fosters their self-efficacy (Ingvarson et al., 2005), their beliefs (Girvan et al., 2016), and their classroom practices (Jeanpierre et al., 2005) in favor of the targeted principles. Involving teachers in curriculum design results in findings that are in line with these effects (Voogt et al., 2011;Mooney Simmie, 2007). Design can be considered a form of complex problem solving, involving interdisciplinary thinking, handling of multiple decisions and unanticipated problems, inquiry, collaboration in multidisciplinary teams, and a scientific basis to start from (de Vries, 2020;Jonassen et al., 2006;Stanovich et al., 2003). Teachers will thus experience the key principles of iSTEM education (Figure 1) firsthand when they participate in cooperative, interdisciplinary design of iSTEM learning materials, which, in turn, will prepare them to implement these principles in their classrooms.
The role of TDTs in curricular innovation has gained much interest in educational research, often in the context of professional development (Handelzalts, 2019;Coenders et al., 2015;Huizinga et al., 2014). However, there is little in-depth research on teacher endeavors to design iSTEM learning materials. Some studies report on teachers' experiences and struggles while designing integrated or context-based STEM materials (James et al., 2000;Stolk et al., 2016). Guzey et al. (2016), who examined 20 teacher-designed iSTEM learning units, concluded from their study: [I]ntegrated or interdisciplinary science curriculum is not a new concept; however, designing instructional materials for integrated STEM education is new for many teachers. Put simply, there are few resources available for teachers to help them develop integrated STEM curriculum materials and designing curriculum materials is a complex process. (Guzey et al., 2016: 14)

Figure 1. Key principles and requirements for high-quality integrated STEM education
Little research has focused on the teachers' process itself when they are designing iSTEM learning materials. McFadden et al. (2017) observed two TDTs' design processes and visualized these processes via flowcharts. These flowcharts contain dilemmas and design decisions that reflect the particular group dynamics in the TDTs, but do not seem to be meant as general guideline for iSTEM curriculum design. Better generalizable are the design activities mentioned in the study of Satchwell et al. (2002). In their study, a team of nine competent middle-school teachers and researchers from different STEM disciplines engaged in (1) choosing a shared theme based on the national math and science standards and key STEM concepts, (2) developing a series of discipline-oriented objectives aligned with the national standards, (3) creating an introductory challenge to raise students' interest, (4) designing learning materials and authentic assessments to have students explore, study, and apply the new key concepts. The designed learning units were reviewed on content by STEM experts, and feedback from field-tests and classroom visits was incorporated afterwards. Satchwell et al. state that "[t]he development of a standardsbased, integrated mathematics, science, and technology curriculum is a challenging task" (p. 16) and indicate that the development of a framework, field testing, collection of feedback, reflection, and revision are important parts of the design process.
These studies suggest that secondary STEM education would considerably benefit from more in-depth research on the specific activities constituting TDTs' actual iSTEM design process and on how these activities affect the designed learning materials. In this study, we translated the research literature's calls for guidance regarding the iSTEM design process into two research questions: RQ1: How can we model the process teacher teams go through when they are designing integrated STEM learning materials from scratch? Which activities can we identify in their design process?
RQ2: Based on the learning materials designed by the teacher teams and the requirements for high-quality integrated STEM education, which crucial, profitable, detrimental, and missing activities can we distinguish in the design process we found in RQ1?
An evidence-based model and assessment of such activities could support school organization, teacher education, and professional development programs to better prepare and support STEM teachers to establish high-quality integrated STEM education (Guzey et al., 2016;Shernoff et al., 2017). It could facilitate teachers' design process, empower them to work in a systematic and organized way, and to communicate with a common language (Woods, 2000).

Outset of This Study
Regarding this study, the following initial conditions should be taken into account: (1) the participating teacher teams were not guided by the iSTEM key principles shown in Figure 1, since these were established during the successive research cycles of STEM@school; (2) the teacher teams were not guided by any design or problem solving framework. They were only guided by the aspiration of deepening students' understanding and stimulating their interest in STEM subjects by meaningfully integrating STEM learning contents. Given these conditions, we looked open-mindedly at how the teacher teams approached the design of iSTEM learning materials, and the design activities they engaged in. The iSTEM requirements in Figure 1 are the criteria against which we evaluated the iSTEM learning materials designed by the TDTs, to assess the effect of each of the TDTs' design activities on the designed learning materials.

METHODS
An inductive, data-driven approach was applied to build a model describing teachers' process of designing iSTEM learning materials. We chose case study as research method because of the real-world and incontrollable character of the context in which this study would take place (Kitchenham et al., 1995;Yin, 2003). Each project year of STEM@school, a TDT was selected for the case study by means of purposive sampling (Mortelmans, 2007) until we obtained theoretical saturation (Eisenhardt, 1989). In total, four cases (C1, C2, C3, and C4) were studied, each involving a TDT engaged in the process of designing iSTEM learning materials from scratch. In order to construct a valid model of this process, we gathered and triangulated data from multiple sources of evidence (Yin, 2003), as shown in Figure 2 and elaborated below.

Construction of the Case-based Model of the TDTs' iSTEM Design Process (RQ1)
Model construction via within-and cross-case process analysis (step 1.1) -Each case was first analyzed separately, in order to be able to identify unique patterns before generalizing them across cases (Eisenhardt, 1989). Therefore, the main researcher (first author) kept field notes and reports on the TDTs' design meetings, and memos on team dynamics. These data were supplemented by secondary data, such as e-mails sent among TDT members and coaches, annual planning schemes, materials, prototypes and presentations produced by the TDTs, as shown in Figure 2. First, open coding was used to label excerpts of meeting notes as particular actions, by means of direct interpretation (Stake, 1995). The main researcher then engaged in axial coding (Charmaz, 2006;Stake, 1995): by means of a codebook, she interrelated codes that seemed to lead to one and the same outcome. Such outcomes were, e.g., a target group, an inventory of prescribed STEM curriculum standards, and a list of constraints for the central student task. Actions leading up to one outcome were classified under one category, i.e., a design activity. These categories were refined based on the secondary data. From C2 on, the main researcher performed a cross-case analysis (Eisenhardt, 1989) after the design process of a case had come to an end (see Figure 2): via constant comparison (Charmaz, 2006), she checked the data collected and codes generated in the different cases to see whether and which actions could be classified under design activities identified in the previous cases. On a weekly basis, the researcher discussed the data and her interpretations with other STEM@school researchers. They shared their perspectives until a consensus on the categorization was reached. Appendix 3 shows the identified categories with their major actions (clustered codes) and outcomes.
Model validation (step 2.2) -To verify the veracity (or representativity) of the constructed model (Van Driel et al., 1999), we asked a teacher who had participated in three of the cases to draft a model based on his own impression of the iSTEM design process. He was asked to send it per e-mail. We compared his digital model to our case-based model to look for discrepancies.

Evaluation of the Design Activities Constituting the Case-based Model (RQ2)
To determine the crucial, detrimental, and missing activities in the observed design processes (RQ2), we triangulated data from multiple sources: the learning materials designed by the TDTs of each case, the iSTEM requirements shown at the right of Figure 1, the design activities constituting the case-based model resulting from RQ1. Per-case product evaluation (step 2.1) -Three STEM@school researchers (among whom the first and last author of this paper) collectively evaluated the products of the studied design processes (i.e., the learning materials designed by the TDTs in cases C1, C2, C3, and C4) against the iSTEM requirements (see Figure 1).
Cross-case linking of the design activities to the iSTEM requirements (step 2.2) -Based on a concise list of their constituting actions, each of the design activities of the process modeled in step 1.1 was linked to one or multiple iSTEM requirements. Therefore, the main researcher and an external researcher (not associated with STEM@school and therefore unbiased by the events) individually assessed per design activity whether or not its actions (as shown in Appendix 3) could affect the fulfillment or failing of each iSTEM requirement. Afterwards, they compared and discussed the resulting associations until they reached a consensus.
Cross-case process evaluation (step 2.3) -We interconnected the result of step 2.1 (i.e., the evaluation of the materials designed in each case against the iSTEM requirements), the result of step 2.2 (i.e., the links between the iSTEM requirements and the design activities of the case-based model), and the result of step 1.1 (i.e., the case-based Figure 2. Data collection and analysis of this study model with the occurrence or absence of the design activities in each case). Through this interconnection, the quality of the product of a design process was traced back to the occurrence or absence of certain design activities in the modeled process, using determination Table 1.

Context and Selected Cases
In autumn 2013, the STEM@school project team launched an open call among Flemish schools to sign up for trying out a new, integrated approach to STEM in their secondary education. The schools had to indicate whether they also wanted to participate in the design of this approach and associated learning materials. Thirty schools signed up to pilot the new iSTEM approach. The STEM@school team initially selected 10 schools to delegate a teacher team for the iSTEM curriculum design, based on whether their motivation favored interest and enthusiasm over political agenda (such as using the new approach merely as a pretext for student recruitment). The selection held a balanced mix between schools providing education with a science focus (S), and schools providing education with a technical focus (T). Each of these schools delegated one team of two to five teachers for the iSTEM curriculum design. From this 'big group' of teacher teams, each project year, the STEM@school team composed several multidisciplinary, cross-school TDTs based on considerations prevalent at that time (see Table 2). Out of these TDTs, we selected one team per project year for our case study by means of purposive sampling. The school team of C1 had presented itself to the STEM@school staff before the project was actually launched and was therefore included as an exploratory case. The context of the selected cases is shown in Table 2 and Table 3.  Figure 1)  = this requirement was fulfilled by the learning materials designed in this case  = this requirement was failed by the learning materials designed in this case Determination of the different types of activities A design activity C is assumed to be crucial for reqA if activity C is linked with reqA AND activity C occurred in all of the cases of which the learning materials fulfilled reqA AND activity C did not occur in any of the cases in which the learning materials failed reqA A non-observed design activity M is assumed to be missing for requirement B if reqB was failed by the learning materials of all of the cases A design activity P is assumed to be profitable if most of the requirements linked with this activity are fulfilled by the learning materials of the cases in which this activity occurred AND most of the requirements linked with this activity are failed by the learning materials of the cases in which this activity did not occur.
A design activity D is assumed to be detrimental if most of the requirements linked with this activity are failed by the learning materials of the cases in which this activity occurred AND most of the requirements linked with this activity are fulfilled by the learning materials of the cases in which this activity did not occur.
For our purposive sampling, we implemented the following criteria (Mortelmans, 2007): informational contribution and convenience − TDTs who did not manage to get started properly in the iSTEM design were excluded from the case study. C1, C2, and C3 were selected because their TDTs were coached by the main researcher, which gave her plenty of opportunities to gather data on the iSTEM design process. homogeneity -To identify the influence of commonalities and to explore evolution in time on the identified pattern of design activities, we selected C3 based on its correspondence with C2 in terms of team composition, learning contents to cover, and team coach. heterogeneity -In order to identify commonalities in the design processes of TDTs despite variation, C4 was selected because its TDT composition differed completely from the TDT composition of C3 (different TDT members and coach). Furthermore, the stage of the TDT of each case in the project varied from zero years of design experience (C1) to three years of design experience (C4). Apart from schools BT and CS (see Table 3), all participating schools agreed to the following terms proposed by the STEM@school team: • The learning materials designed in year i would be implemented in the pilot classrooms in school year i+1.
• A new subject 'iSTEM' would be incorporated into the students' weekly timetable (see Table 4), especially designated for activities concerning STEM integration, inquiry, and design. • Truly integrated learning materials cover contents from different STEM subjects. Therefore, not only the new 'iSTEM' subject but all of the involved STEM classes and their teachers would have to be engaged with the implementation of the materials. C4 a 'S' represents the number of teacher teams from schools providing science-focused secondary education, 'T' represents the number of teacher teams from schools providing technology-focused secondary education b school codes (see Table 3) withing brackets C4_Phys, C4_Bio-Chem1, C4_Bio-Chem2, C4_Math1, C4_Math2; C4_Phys-Chem, C4_Bio, C4_Elec; C4_Coach a school code focus of education, in which 'S' means science focus, and 'T' means technology focus b pseudonym of participating teacher: case(s) in which s/he participated _ subject(s) s/he taught c for background information on the participating teachers: see Appendix 1 • To consistently and profoundly expose students to integrated STEM activities, two to four large learning units were to be implemented per school year, each unit covering several weeks to up to a whole semester. Each TDT would design iSTEM learning materials for one unit. • In the timetables of the teachers participating in the iSTEM design, Tuesday afternoons would be reserved for weekly within-school design meetings and monthly cross-school TDT meetings. • The design process would be limited to the period from September until April each project year.
• Each TDT would be coached by a researcher from the STEM@school project team. These coaches would take the lead in the TDT composition, planning of TDT meetings, writing of a script for the learning materials, and finally bundling the materials developed by the different school teams in the TDT. Figure 3. Model of the phases and activities (indicated by acronyms) identified in the design processes studied in cases C1, C2, C3, and C4 Figure 3 shows the model of the design processes of the four studied cases. In these processes, we distinguished five phases, each consisting of several design activities, which we briefly discuss below. Appendix 2 gives a detailed overview and illustrates the design activities we identified in each case. In the Context analysis phase, the teachers defined the target group of students for whom they would design the iSTEM learning materials (TGI) and analyzed the context, i.e., scanned the national curricula of the different STEM subjects of this target group to look for learning contents that could be integrated (SCA).

Construction of the Case-based Model of the TDTs' iSTEM Design Process (RQ1)
In the Theme selection phase, based on the learning contents extracted from the STEM subject curricula, the TDTs discussed possible themes for the learning units and a sequence in which these units should be implemented in the target group's school year (BCT). Based on the conditions shown in Table 2, the STEM@school team then composed the TDTs and divided the themes among them (TFD).
In the Content/challenge brainstorm phase, the TDT members discussed the learning contents relevant for the theme that was assigned to them (CIL), as exemplified by the following excerpt of a meeting in C4 (theme algae): We made the exercise: which learning contents can be addressed from (…) the curriculum of [grades 11 and 12]? In biology, it's indeed more about microscopy, mitosis: what is that? Eh, the connection with the growth charts, but that is the math. Photosynthesis came up (…) And then, the biochemistry of what plants and algae consist of. The link with lipids, proteins, and which nutrients do we need to [grow] them. And then we get to the link with growth medium. (…) Then, within chemistry, we [distinguished] determination of concentration and dilution, possibly, in preparing the media. (…) And also, if we are working with acidity, if we are measuring that in our medium, eh... we could develop some learning materials on that as well. And eh... then, of course, also the separation and extraction techniques, if we want to go into... how to isolate the lipids. Based on the listed contents, they formulated a student challenge that would raise the need to learn and link these contents (CF). In C2 and C4, the TDTs iterated these activities to better match the challenge with the learning contents. These TDTs also studied the feasibility and open-endedness of the challenge (RFS) and broke it down into smaller subproblems (CBS). The TDTs then conceived learning activities that would lead students of the target group to learn and link the targeted contents in order to solve the challenge (DLA). Figure 5 illustrates this phase for C2. The TDTs also carried out experiments and built example prototypes of their students' future products (EPD). The result of this activity in C3 (theme ball launcher and car) is shown in Figure 4. The TDT of C3 started this phase by forming monodisciplinary subteams who further developed discipline-specific materials regarding the targeted learning contents (SFD).
The Reporting phase took place in one plenary meeting within the big group of TDTs, organized by the STEM@school team in order for the TDTs to present their preliminary designed materials and exchange feedback (REF). In C2 and C3, the TDTs also discussed the rescheduling of the learning unit implementation within the students' school year (TPR). In the Development phase, the TDT coaches wrote a script (or learning-unit scenario) to interconnect the student challenge and the learning activities that had been conceived by the different school teams constituting the TDT (SW). Meanwhile, the TDT members refined their syllabi (SF), elaborating the learning activities and instructions for the target group. These materials were bundled by the TDT coach (MB) and revised and fine-tuned a last time by the TDT members (MRF). The revision of the learning materials, based on the feedback from the teachers who implemented them in the pilot classrooms, fell out of the scope of this study, as it was merely conducted by the STEM@school staff. The final learning materials can be found on the project webpage (STEM@school, 2017).
The left of Figure 6 shows the digital model made by one of the case-study participants (translated to English) upon our request in order to validate our model (step 1.2 of our study design). On the right, we depict the activities of our case-based model, of which the labels and descriptions strongly match the descriptions in the participant's model. Comparison suggests that our case analysis produced a representative and more fine-grained abstraction of the design process, and that we did not overlook any activity. Figure 7. Results of the per-case product evaluation (step 2.1) and the cross-case linking of the design activities to the iSTEM requirements (step 2.2 of the study design)

Evaluation of the Design Activities Constituting the Case-based Model (RQ2)
To identify the crucial, detrimental, and missing activities in our case-based model (Figure 3), we utilized the list of iSTEM requirements shown at the right of Figure 1. The left side of Figure 7 shows how the learning materials designed in each case were evaluated with respect to each of the iSTEM requirements (step 2.1 of our study design). The right side of Figure 7 shows the links between the design activities identified in our case study and each of the iSTEM requirements (step 2.2 of our study design).
The combination of the results shown in Figure 7 and the case-based model in Figure 3, which shows the occurrence or absence of the design activities in each case, is shown in Table 5. This table formed the basis to further analyze the effect of each design activity on the learning materials designed in each case (step 2.3 of our study design). We note that, in these analyses, we did not take into account other, unidentified activities or factors that might have played a role in the design processes.
In order to determine the crucial and missing design activities in the modeled design processes, we performed a cross-case analysis per iSTEM requirement based on Table 5. For each requirement, we determined (a) the minimal combination of design activities linked to this requirement that sufficed to fulfill () the requirement in at least one case, and (b) the maximal combination of design activities linked to the requirement that had not sufficed to fulfill the requirement and had thus resulted in its failing () (see Table 6).
By subtracting the insufficient combination of activities from the sufficient combination (i.e., (a) -(b) in Table 6) and taking into account the causalities between activities (see at the right of Figure 7), we can conclude for certain that the following activities are crucial to fulfill some of the requirements for high-quality iSTEM education: -CIL (learning-content identification and linking), since CIL is also an important precondition for DLA (design of STEM linking learning activities), which is in turn an important precondition for SF (syllabus finalization), -CF (challenge formulation), as important preconditions for CIL and CF: • BCT (brainstorm on learning contents and themes), and therefore: o SCA (specific-context analysis), and therefore:  TGI (target-group identification),  req1  req2  req3  req4  req5  req6  req7  req8  req9  req10 where, e.g., means: design activity SCA, which was linked to reqX, occurred in case Cy and means: design activity SCA, which was linked to reqX, did not occur in case Cy Table 6. Cross-case determination of the activities crucial for each iSTEM requirement to be fulfilled, derived from and/or -CBS (challenge breakdown into subproblems), and -RFS (requirements and feasibility study). Furthermore, the learning materials of none of the cases met requirements 8 and 9, and Table 6 is not conclusive about the activities necessary to fulfill requirement 6. These findings suggest that one or a few design activities are missing in order for requirements 6, 8, and 9 to be fulfilled. More precisely, actions should have been taken in the design processes to ensure: student cooperation (~req6); goal-oriented strategies (~req8); and integration of know-how from discipline-specific educational research (~req9).
In order to determine the profitable and detrimental design activities in the modeled design processes, we performed a cross-case analysis per design activity based on Table 5. For each design activity, we counted the total number of iSTEM requirements linked to this activity: (a) those that were fulfilled () across all cases in which this activity had occurred, and (b) those that were failed () across all cases in which this activity had occurred. We did the same for all cases in which this design activity had not occurred. Taking into account the causalities between activities (see at the right of Figure 7), this resulted in Table 7.
Using determination Table 1, Table 7 suggests that most activities were (rather) profitable, having a moderate to very positive effect on the fulfillment of the requirements for high-quality iSTEM education. Design activities SFD (subteam formation and content division) and SF (syllabus finalization) appear to be rather detrimental.

Interpretation of the Findings
In this section we relate the most significant results on the assessment of each design activity to our observations during the case study. The identification and cross-disciplinary linking of STEM learning contents (CIL) and the formulation of a student challenge (CF) are shown to be crucial activities for the designed materials to fulfill the iSTEM requirements. Particularly in C2 and C4, we saw how iteration of these two activities played a pivotal role in the TDTs' design processes. This finding is remarkable, given the fact that the TDTs of these two cases had no team member or coach in common. This iteration manifested itself in lively cross-disciplinary discussions during TDT meetings, which reflected the iSTEM key principles problem-centered learning, integration and cooperative learning among the participating teachers: The teachers were trying to find a common ground, for which they had to understand each other, and therefore, they had to learn from each other across STEM disciplines. This resulted in a clever alignment of an engaging student challenge and the targeted learning contents from different STEM subjects. In C1, no challenge was formulated. As a result, the learning activities in the designed materials of C1 contained an incoherent set of cross-disciplinary but unrelated experiments and exercises (De Meester et al., 2015). In the other cases, we found the formulated challenge to be the main focus and motivator for the TDTs. In C3, no actual learning activities were designed (no activity DLA), probably because the specific learning contents to be addressed, had not been articulated or discussed (i.e., activity CIL was not found in C3).
The requirements and feasibility study (RFS) appeared to be a crucial design activity as well. This activity had manifested itself in two different ways. In C2, this activity was carried out to ensure that the challenge would be open-ended (i.e., allow for different possible solutions). In C4, the TDT undertook company visits to assess the feasibility of different algae applications, such as waste water treatment, biofuel production, and nutritional supplements. In this design activity, as well as in the investigation of learning contents, we saw teachers engaging in the iSTEM key principle of inquiry-based learning.
The challenge breakdown into subproblems (CBS) seemed to help the TDTs of C2 and C4 to identify the sequence in which to address the targeted learning contents throughout the learning unit, which, in turn, guided the design of the learning activities (DLA). The outcome of this activity formed the basis of the script writing Table 7. Cross-case number of fulfilled versus failed iSTEM requirements linked to each design activity, derived from Table 5 and the causalities in Figure 7 (a) where, e.g., means: across all cases in which design activity SCA occurred and means: across all cases in which design activity SCA did not occur (SW), in which the scenario of the learning activities was outlined (except from C3, in which the TDT coach had to build the script from scratch). We attribute the absence of activities CIL, RFS, CBS, and DLA in C3 to the fact that the TDT split up into smaller, monodisciplinary subteams, who independently developed materials for the separate STEM classes (SFD). These materials contained no links across STEM disciplines and did not contribute to solving the challenge. This could explain why activity SFD came out of the analysis as a 'rather detrimental' activity. Also the syllabus finalization (SF) came out of our analysis as rather detrimental, but no observations in our case study could corroborate this finding.
In C2, C3, and C4, the TDTs devoted a lot of design time to a tryout of the experiments or the design of a prototype (activity EPD). In C2 and C4, this activity fed into the design of the learning activities (DLA), as the pedagogical approach of the learning activities relied on the process and results of the experimentation and prototyping. In C3, the majority of the design process was absorbed by this activity. The intermediary reports of the subteams of C3 and the discussions during the TDT meetings were dominated by the teachers' efforts to produce working prototypes (see Figure 4) and test their accuracy. This resulted in many technical documents which lacked a pedagogical approach. The final designs were so complex that the students, in order to get working prototypes, would have to follow a cookbook plan.
We believe that the composition of multidisciplinary TDTs (activity TFD) has strongly facilitated the crossdisciplinary brainstorming that took place in activities CIL, CF, and the design of cross-disciplinary learning activities (DLA) in C2 and C4. Activity REF (the review and exchange of feedback) was much appreciated by several teachers, as evidenced by their reflections: Actually, I find it quite important: explaining what we want to do, and that [other teams] can say: "yeah, that is maybe a bit odd," or "that can be done better like this," or "I would put more emphasis on that." So that we can take this into account before we [proceed], 'cause... yeah, if we would already have developed everything, and then they [would come up with] a lot of comments, that [would be] quite annoying. (C4_Phys) In none of the observed cases, and nowhere in the design processes did the TDTs take the time to construct a clear and comprehensive list of learning goals they wanted their students to achieve. We believe that this missing though important design activity, explains the fact that certain requirements for high-quality iSTEM education were failed by all designed materials. Although the teachers' main drive was to address the learning contents mentioned in the national curriculum standards, they never discussed what they considered to be important competences with regard to these contents. Furthermore, none of the TDTs consulted scientific articles on students' difficulties with regard to these learning contents. Consequently, the iSTEM key principle of researchbased learning was missing in the TDTs' design practices.

DISCUSSION
We conducted a case study to model the design processes of four TDTs involved in the development of integrated STEM learning materials. Assessing the learning materials designed by each of these TDTs against the requirements for high-quality iSTEM education and linking the modeled design activities to these requirements, allowed us to distinguish the crucial, detrimental, and missing activities in the design processes. In sum, our study shows evidence that a successful iSTEM design process is constituted by (1) the multidisciplinary TDT formation (activity TFD), and (2) the alignment of (a) the identification and linking of STEM learning contents and (b) the formulation of a real-world student challenge (activities CIL and CF). We found that, via these activities, STEM teachers engage in constructive, cross-disciplinary conversations. In turn, these conversations lead to engaging student challenges and activities that do not only address learning contents from different STEM disciplines but also add meaning to students' learning in the light of solving these challenges. On the other hand, our study also revealed that, in order to fulfill all requirements for high-quality iSTEM education, TDTs should explicitly be encouraged to discuss and create learning opportunities that ensure (1) students' achievement of important learning goals associated with (higher-order) STEM competences, (2) student cooperation, and (3) the use of researchbased strategies. In this discussion we first situate our findings in the broader research field on interdisciplinary curriculum design and in its national educational context. We then elaborate the limitations of our study. We conclude with suggestions for future research and opportunities entailed by our findings.
This study presents a first explicit model of the particular activities constituting the design of integrated STEM learning materials as carried out spontaneously by teams of in-service high-school STEM teachers. Table 8 shows the design activities we identified via our inductive approach and compares them to actions and activities we could detect via deeper analysis of other research on interdisciplinary curriculum design, using our case-based model as a lens. A testing stage did not fall within the scope of this study. Table 8   development of student learning objectives and assessments. This activity was lacking in McFadden et al.'s (2017) as well as in our study. Teachers enthusiastically dive into the design of a fun challenge (Guzey et al., 2016) and the inherent hands-on design (see activity EPD in our case study), without setting their objectives first. Furthermore, since teachers do not easily find their way to research literature (See et al., 2016), they do not tend to use evidence-based instructional strategies in their design. Much like in McFadden et al.'s (2017) study, the design discussions in our cases often "remained geared around what types of experiences students should be engaged in and not how students learn best during integrated learning experiences" (p. 13). Unlike the teams of McFadden et al., however, the TDTs in our case study did not seem reluctant to explore the multitude of learning opportunities encompassed by open-ended challenges (see activity RFS). We believe this courage stems from the fact that the TDTs in our study were composed by teachers from different STEM disciplines, from different grade levels and from different schools (see activity TFD). Each TDT thus held a lot of expertise in diverse areas for the TDT members to rely upon.
As the findings we present result from a case study, they should be situated in the specific context in which the study was carried out. Before STEM@school, the subject of 'how to teach integrated STEM' did not make part of STEM teacher education programs in Flanders (Belgium). The high-school teachers participating in this study have been schooled in teaching one or two STEM subjects (e.g., physics, biology, mathematics) in a monodisciplinary way. They were, therefore, used to adhering to subject-specific national curricula and handbooks, and to consulting with colleagues teaching the same STEM subject(s). This traditional way of teaching STEM subjects did, however, not require them (and, therefore, they were not used) to consider viewpoints and curricula from colleagues teaching other STEM subjects. The consequential tendency to resort to monodisciplinary teaching is reflected in activity SFD (monodisciplinary subteam formation) which occurred in C3 and was found to be harmful to the design of high-quality integrated STEM materials. Furthermore, before the start of STEM@school, the concept of 'integrated STEM' at school had been promoted in Flanders via short-term projects or events, such as a STEM day, workshop, or company visit. Such activities did address learning contents from different STEM subjects, but often in a 'fun' and not necessarily integrated or instructive (i.e., standards-based or competence-oriented) way. This idea may have played a role in the fact that the high-school teachers in this study (1) have not considered articulating meaningful learning objectives and assessments around the contents they extracted from their subjectspecific curricula, and (2) sometimes prioritized the development of fun, hands-on activities addressing these contents over the design of minds-on learning activities aiming at meaningful STEM competences related to these contents. We believe that guidelines based on our evidence-based model can favor teachers, teacher educators, and schools in any educational context that shows characteristics similar to those of the Flemish context described here.
This study has some constraints. As in other case study research, certain features, such as team dynamics, the participants' knowledge and beliefs, and the role of the team coach could not be clearly distinguished from the process under study. As a result, we cannot indicate which variables might have influenced the occurrence and quality of the design activities and how. In order to construct a valid model, we implemented several literaturebased methods such as purposeful sampling and theoretical saturation, and recursive cycling through and triangulation of different data sources (Creswell et al., 2000;Eisenhardt, 1989). However, the research was mainly conducted from an insider's view by the STEM@school project staff, and some bias can thus not be ruled out. On the other hand, complete immersion in the process enabled the researchers to experience the endeavors of iSTEM design first-hand, and to interpret the observed activities in their specific context, which they got to know from inside out (Mortelmans, 2007). Another limitation of the study is that we cannot claim that certain design activities re-emerged in the different cases by coincidence, since some learning effect might have set in among the TDTs during the course of STEM@school. However, taking into account these limitations, the combination of observation-based modeling, criterium-based product assessment, linking, and logical deduction, provides a useful new approach to evaluating curriculum design processes.
Future research could advance the case-based model presented in this work. Firstly, the classroom implementation and corresponding revision of the designed materials could be examined and included (e.g., as extra phases) in the model (Satchwell et al., 2002;Berland, 2013). Additionally, further research among pre-and in-service STEM teachers could examine the extent to which the case-based model and findings of this study support other TDTs' endeavors to design integrated STEM education. Our observations showed indications that participants in our TDTs were being immersed in some of the key principles of high-quality iSTEM education. In order to further inform teacher training programs, future research could focus on (1) teachers' immersion in these principles when being involved in iSTEM curriculum design that is guided by our findings, and (2) the effect on their knowledge and self-efficacy and those of their students (van Keulen et al., 2015).
With the notion of its crucial, detrimental, and missing activities, the evidence-based model constructed from our case study can inform (student) teachers and teacher educators who want to transform high-school STEM education into high-quality integrated STEM education. Since the model reflects a path towards iSTEM learning materials that was fully conceptualized by in-service teacher teams during this study, in-and pre-service teachers will naturally relate to these findings. Currently, this particular model together with the good practices and pitfalls found in the case study already form the basis of pre-and in-service teacher training for integrated STEM in several higher education and professional development programs in Flanders. The model and implications of this study provide (student) teachers with a common language and encourage them to engage in cross-disciplinary discourse, to search for ways to create competence-oriented, meaningful, and thus motivating STEM learning opportunities for their students.

LIST OF ABBREVIATIONS
S T E M a for information on the cases: see Table 3 b pseudonym of participating teacher: case(s) in which s/he participated _ school focus of education _ subject(s) s/he taught c no indication means not declared

Description (actions)
Occurrence Example(s) from the cases CONTEXT ANALYSIS Target-group  identification  1 week TGI The big group decides on the target group of students (i.e., grade and study program) of the learning materials C1 C2 C3 C4 The teachers in C1 chose students of grade 7 as target group, to inspire them for STEM already from the beginning of their secondary school career. Specific context analysis 2 weeks SCA The big group analyzes the curricula of the separate STEM subjects of the target group in order to determine the standards that must be achieved by the target group. For each STEM subject, the teachers extract the STEM contents that must be learned by the target group.
C1 C2 C3 C4 The teachers in C2 explored the national curricula for students of grade 9 in the study options 'Sciences' and 'Industrial sciences'. They also looked into the curricula of the prior grades (7 and 8) to familiarize themselves with the expected prior knowledge. And they looked forward to what these students were supposed to learn later-on (grade 11). The teachers examined the curriculum standards defined for the subjects 'natural sciences ', 'physics', 'technology', 'applied sciences', 'engineering', and 'mathematics'.  ', 'batteries', 'composting', 'geothermal energy', 'bioplastics', 'algae', and 'water treatment', 'with a discernable preference for the latter two, to cover 1 semester.
TDT formation and content division 1 week TFD The big group of teachers is split up by the coaches into smaller, multidisciplinary teams (TDTs) of teachers from different schools, with knowledge in different STEM disciplines, with differing classroom or practical experience, with differing degrees, and with teaching assignments in different grades.
C1 C2 C3 C4 In C1, the group split up into several TDTs to develop the different themes (e.g., 'electric circuits', 'biotopes', 'robots', and '(bio)plastics'). The TDT we followed was assigned the theme 'biking', which it broadened to 'Motion 1D'. The teachers who formed the TDT that was studied in C3 were very determined to do something with quadratic equations, trebuchets and cars. They were therefore assembled by the STEM@school coaches before activity BCT took place.

BRAINSTORM
Learning-content identification and linking 2 weeks CIL The TDT lists the learning contents relevant to the assigned theme (TFD) and categorizes them per discipline (S, T, E, M). The TDT collectively establishes cross-disciplinary links between these contents. In cases in which this activity occurred a second time, the TDT added extra learning contents and crossdisciplinary links that appeared necessary to solve the formulated challenge (CF).

C1 C2 2x
C4 2x The TDT members in C4 mentioned the learning contents (extracted from the curricula for grade 11) they wanted to integrate in the scope of the themes 'water pollution ' and 'algae': w.r.t. biology: photosynthesis, mitosis, the connection with growth charts (math), lipids, proteins, nutrients; w.r.t. chemistry: concentration and dilution, acidity, separation and extraction techniques; w.r.t. engineering and electricity: spectrophotometry, the link between the concentration of water pollution particles, the light spectrum, LDRs, phototransistors, and electric current. Subteam formation and content division 1 week SFD The TDT divides the learning contents relevant to the assigned theme (BCT) among its team members per discipline (S, T, E, and M). The TDT splits up into smaller, monodisciplinary subteams to continue designing learning materials per discipline.

C3
The TDT of C3 split up as follows: (1)

APPENDIX 3 -ACTIONS PER DESIGN ACTIVITY
(Process starts with a big, motivated group of STEM teachers) TGI deciding for which target group of students (i.e., grade and study program) to develop Outcome: the target group of the learning materials SCA analysis of the curricula of the STEM subjects (science (or more specifically physics, chemistry, biology, geography), technology (or applied sciences, computer science), engineering, math) of the target group to determine: o the prior knowledge of the target group o the standards (curriculum goals) that must be achieved by the target group extracting from these curricula the contents that must be learned by the target group discussing these learning contents Outcome: prior knowledge and STEM curriculum standards prescribed for the target group BCT grouping/categorization of learning contents to identify overarching themes based on the separate STEM curriculum standards identifying/choosing overarching themes choosing a sequence in which the themes will be scheduled in the students' school year based upon logical succession of learning contents Outcome: themes for the learning materials, and a sequence in which to address them TFD splitting up the big group of enthusiasts into "smaller", multidisciplinary teams (TDTs) of teachers with differing discipline-specific knowledge, and differing classroom or practical experience in one or more of the STEM disciplines, with differing degrees and/or teaching in different grades, of different schools allocating the chosen themes to the formed TDTs Outcome: small(er) multidisciplinary TDTs (Big group of STEM teachers is now split up into teams of about eight teachers from different STEM disciplines) CIL collaboratively listing the learning contents relevant to the theme per discipline (S, T, E, M), collaboratively establishing cross-disciplinary links between these contents, -(when iterated) collaboratively adding learning contents to the list and cross-disciplinary links, necessary to solve the formulated challenge, Outcome: list of learning contents from different STEM subjects with cross-disciplinary links SFD dividing the relevant discipline-specific learning contents splitting up the team of teachers into small, monodisciplinary subteams Outcome: small monodisciplinary subteams CF collaboratively searching for a challenge o that is fun / meaningful to the target group o that includes purposeful design or inquiry o that is more or less authentic / situated in a real-world context -(when iterated) collaboratively fine-tuning the challenge so that the targeted STEM contents will be addressed in a meaningful way and linked when solving -(when iterated) collaboratively adjusting the challenge, taking into account the studied the requirements and feasibility (e.g., open endedness) Outcome: a central student challenge RFS consulting experts in the field of the chosen theme collaboratively identifying the requirements and constraints of the chosen challenge, collaboratively checking these constraints for allowing enough student creativity, and for open-endedness collaboratively checking the practical feasibility of the formulated challenge: making an inventory of the accommodation and equipment needed to solve the challenge and listing extra constraints based on what is (not) possible Outcome: requirements and feasibility criteria for the student challenge

CBS
collaboratively identifying the different subproblems encompassed by the challenge Outcome: list of successive subproblems to be encountered when solving the challenge DLA collaboratively determining what the target group will be doing: brainstorming cross-disciplinary learning activities per subproblem o that form the bridge between the challenge and the targeted learning contents; o that address and link learning contents from different STEM disciplines, e.g., via modeling o that rely on previously constructed learning contents o in which students are actively engaged: involving  theory processing and/or information gathering,  making exercises  programming  experimenting, performing an inquiry Outcome: collection of learning activities EPD designing a prototype, or trying out experiments related to the challenge ordering sample components / equipment keeping documentation: o codes, drawings, experiment results o required equipment Outcome: experiment guidelines, prototypes, drawings, building plans, order lists, practical tips REF presenting intermediary ideas and drafted learning activities to other TDTs (of the big group) and to other stakeholders listening/being updated about the other TDTs' ideas and activities collaboratively giving feedback on the other TDTs' ideas and activities collaboratively documenting feedback from the other TDTs on own ideas and activities Outcome: feedback (suggestions for improvement/adjustment) of the materials TPR collectively revising the initial scheduling of the themes (i.e. verifying if the learning contents are addressed in a logical sequence in the school-year scheduling of the themes) in case of doubts about the sequence: balancing pros and contras for switching the themes in the schoolyear schedule collectively deciding upon the final scheduling of the themes within the target group's school year Outcome: an approved theme planning SW writing the scenario (script) for solving the challenge, based on the identified subproblems including scaffolding questions that successively introduce students to a new subproblem, and scaffolding questions that recapitulate what students (should) have learned when solving that subproblem and reconnect with the challenge identifying gaps among the outlined learning activities and communicating them to the TDT members Outcome: script (scenario) for the learning activities, starting from the student challenge SF elaborating the drafted learning activities, taking into account the feedback from the other TDTs gathering information based on the notes and information gathered during the experiment tryout and prototype design Outcome: elaborated syllabi MB bundling of the script and the syllabi Outcome: a coherent collection of syllabi SRF individually reviewing the syllabi fine-tuning the syllabi based on the feedback of the TDT members Outcome: an approved set of iSTEM syllabi (the designed learning materials)