INTRODUCTION

The growing emphasis on cultivating students’ science, technology, engineering, and mathematics (STEM) skills reflects both societal and workplace evolution in the United States. In addition to needing proficiency in the skills required to carry out their jobs, workers now also need to be technologically competent and able to adapt to changes in work processes and the workplace environment. These changes may require workers to possess skills needed to complete more cognitively-demanding tasks, apply the skills and knowledge to solve complex problems, and work successfully in interdisciplinary and multidisciplinary teams (Heerwagen, Kelly, and Kampschroer, 2016; National Academies of Sciences, Engineering, and Medicine [NASEM], 2017). As such, traditional methods of education, which often rely on rote memorization and recitation of information, are not sufficient to prepare today’s students for work (Barron and Darling-Hammond, 2008). Instead, approaches that build 21^st century skills, such as communication, collaboration, critical thinking, and creativity, are critical to students’ long-term academic success and career pursuits. This is particularly true for preparing students to fill jobs in STEM fields, which are predicted to increase more rapidly over the next decade than jobs in any other sector (Klobuchar, 2014; Committee on STEM Education National Science and Technology Council, 2013; Economics and Statistics Administration, 2017). Changes in education that both provide well-rounded academic experiences and bolster interest in STEM subjects are necessary to create equal opportunities for students of all backgrounds to develop the skills that will help them be qualified to pursue careers in STEM fields.

Policymakers, industry professionals, educators, and researchers have all recognized this need, and inclusive STEM high schools, which aim to increase and expand participation in STEM careers to diverse populations of students, have emerged as one response. Such schools also strive to prepare students for 21^st century workplaces more broadly, and are becoming increasingly popular across the U.S. Leaders, teachers, and staff in these schools employ a variety of strategies to enhance students’ educational experiences and to prepare students for real-world careers, including PBL, which STEM schools use to refer to either problem- or project-based learning. PBL approaches are an essential component of inclusive STEM school models; however, PBL is a multidimensional and multifaceted construct with no singular definition or method of implementation existing across STEM schools (LaForce et al., 2016). As such, before we can understand whether PBL approaches used in these schools are associated with improved STEM attitudes, academic achievement, participation, or workforce preparedness, it is essential to first describe what PBL looks like in inclusive STEM schools. To that end, the current study uses data from classroom observations and teacher interviews to understand how inclusive STEM schools describe PBL, how PBL is used, and how it differs from traditional instructional approaches in classrooms.

Inclusive STEM High Schools

STEM-focused high schools have existed in the U.S. for over 70 years (Almarode et al., 2014). These include selective enrollment math- and science-focused schools, which admit students based on prior academic achievement, as well as more recent inclusive STEM high schools, which do not impose academic or achievement-related admissions criteria for students. Inclusive STEM schools typically serve a more diverse population of students (Means et al., 2008; National Research Council, 2011), and often focus specifically on increasing underrepresented students’ exposure to STEM and preparedness for future STEM careers (Peters-Burton, Lynch, Behrend, and Means, 2014). Recent research also indicates that inclusive STEM high schools strive to prepare students more broadly for life after high school (e.g., careers, or two- or four-year colleges) by focusing on the development of workplace and 21^st century skills and engaging students in real-world applications of the content they are learning (LaForce et al., 2016; Means, Confrey, House, and Bhanot, 2008; Young, House, Wang, Singleton, and Klopfenstein, 2011). Providing students with learning opportunities through PBL represents one approach to holistic learning and application of learned skills, and has become so integral that PBL is oftentimes described as synonymous with STEM education (Gorman, 2013; Laboy-Rush, 2011; Miller, 2017) and a trademark of inclusive STEM schools (LaForce et al., 2016; Peters-Burton et al., 2014).

What is PBL?

Researchers and educators define PBL as either problem- or project-based learning, however, there is no agreed upon definition or operationalization; this is, in part, the motivation for the current study. At a high level, these approaches share underlying principles including learner-centered experiences, inquiry, authentic scenarios, group work, and application of skills. Many of these are echoed in other strategies associated with STEM practices and learning, including, for example, discovery- and design-based approaches, and the Next Generation Science Standards science and engineering practices (Alfieri, Brooks, Aldrich, and Tenenbaum, 2011; English and King, 2015; NASEM, 2018). Teachers in inclusive STEM high schools may employ a range of these in their classrooms, sometimes combining or overlaying strategies in order to best fit the needs of their students as well as their own beliefs about effective teaching—this again points to the need for clear descriptions of what is happening in schools before attributing positive (or negative) outcomes to a singular approach.

Here we highlight foundational project- and problem-based learning literature, although in practice, we find that inclusive STEM high schools often use these terms interchangeably, or that some schools indicate they are doing “project-based learning” that closely aligns more to some definitions of “problem-based learning,” and vice versa. . Given the shared emphasis on student autonomy, connections to real life, and teamwork, we operationally define “PBL” as having components rooted in both well-known project- and problem-based learning definitions.

Project-based learning is often considered to be derived from Kilpatrick’s Project Method (1921), which has been a cornerstone of institutions such as the Buck Institute for over 25 years (Buck Institute for Education, n.d.). Scholars agree that project-based learning is student-centered, connected to the real world, and framed around meaningful and authentic questions or problems, which involves students engaging in inquiry processes over an extended period of time (Brassler and Dettmers, 2017; Buck Institute for Education, n.d.; Holm, 2011). However, researchers and educators are not in complete agreement regarding the implementation of project based learning. For instance, some scholars believe that projects culminate in a final product (e.g. Holm, 2011; Savery, 2006), and in some cases, a public product or presentation [Buck Institute for Education, n.d.]), whereas others specify that projects should not end in a predetermined outcome (Thomas, 2000). Moreover, the Buck Institute (n.d.) specifies that project-based learning experiences should include goals related to learning content and specific skills, such as critical thinking, problem solving, communication, collaboration, and self-management, and that students and teachers should engage in reflection on their learning experience throughout the project; these criteria are not necessarily specified as mandatory by other researchers and educators engaging in project-based learning.

The concept of problem-based learning is first found in medical education models originating in the mid-20^th century. The approach was developed to provide students with student-centered and multi-disciplinary curriculum that would facilitate application of knowledge in clinical settings (Boud and Feletti, 1997; Savery, 2006). Despite arguments about its effectiveness, problem-based learning approaches grew in popularity in medical schools in the US, Canada, and Europe during 1980’s and 90’s, followed by their adoption in universities and expansion into the K-12 educational space more recently (Savery, 2006). Problem-based learning posits that learning experiences should be learner-centered, integrate theory and practice, and require students to conduct research and apply knowledge to solve ill-structured problems (e.g., Brassler and Dettmers, 2017; Savery, 2006). Implementations of problem-based learning typically require students to solve an existing or plausible real-world problem (Torp and Sage, 2002). Some PBLs use problems that have a single finite solution, whereas others indicate that the problem at hand should be complex and without a single, correct solution (e.g., Hmelo-Silver, 2004). In addition, some scholars list the integration of disciplines or subjects as a necessary characteristic of problem-based learning (Barrows, n.d., in Savery, 2006), whereas for others, and in practice, this criterion is not always present. Problem-based learning experiences vary in time-frame (one day, three weeks, or five to six problems per semester), but are often shorter than project-based learning experiences (Brassler and Dettmers, 2017; Mergendoller, Maxwell, and Bellisimo, 2006).

Research examining the effectiveness of PBL approaches for improving student outcomes indicates similar positive outcomes, whether defined as project- or problem-based. Specifically, both have been found to result in improved skills related to communication and collaboration among students (Allen, Duch, and Groh, 1996; Bell, 2010; Baumgartner and Zabin, 2008; Cheng, Lam, and Chan, 2008; Lou, Shih, Diez, and Tseng, 2010; Mergendoller et al., 2006; Mioduser and Betzer, 2007), critical thinking and problem solving skills (Albanese and Mitchell, 1993; Ertmer, Schlosser, Clase, and Adedokun, 2014), students’ ability to self-direct (Albanese and Mitchell, 1993; Bell, 2010; Norman and Schmidt, 1992), and student engagement and enjoyment (Albanese and Mitchell, 1993; Baran and Maskan, 2011; Baumgartner and Zabin, 2008; Faris, 2008; Hmelo-Silver, Duncan, and Chinn, 2007; Vernon and Blake, 1993). Furthermore, project and problem-based literature has identified benefits for low- and middle-achieving students, and for students from ethnically- and racially-diverse backgrounds (LaForce et al., 2019; Han, Capraro, and Capraro, 2014, Mergendoller et al., 2006; Tal, Krajcik, and Blumenfeld, 2006).

Despite the benefits that PBL seems to provide for diverse populations of students, it is difficult to draw conclusions from the extant research investigating “PBL,” as implementations of vary widely. Savery (2006) noted that “misapplications and misconceptions” (p. 11) of problem- and project-based learning have occurred as their use has expanded across disciplines and levels of education. Because this is the case, it is not only important but necessary to investigate implementations of PBL in practice, and within specific contexts, to better understand if, when, and how PBL contributes to desired student outcomes.

Current Study

In the current study, we sought to examine the specific practices involved in PBL experiences in inclusive STEM high schools to gain a deeper, more-contextualized understanding of how teachers engage their students in PBL in the classroom. We were interested in describing PBL implementation in practice, and the specific instructional practices that teachers and students engage in during PBL experiences. As such, the current study investigated the following research questions:

1. How do teachers describe PBL? Are there common instructional practices described across teachers?

   1. How do teachers describe the goals, benefits, and challenges of these instructional practices in the context of PBL?

2. Are there differences in the frequency of these instructional practices in PBL and non-PBL class sessions?

To answer these research questions, we investigated instructional practices identified across the literature and by interviewed teachers in the current study as central to PBL experiences; these included teacher facilitation of cognitively demanding work, student autonomy and risk-taking, student cooperation and teamwork, and making connections across disciplines and to real-world content. As teachers are the primary facilitators of PBL-based curricula and learning opportunities for students, their first-hand perspectives provide rich descriptions of these instructional practices. These were explored through semi-structured teacher interviews which elucidate interindividual differences and variations in teachers’ descriptions of PBL instructional practices in inclusive STEM high schools. Additionally, differences in the presence of such practices between observed PBL and non-PBL classroom experiences (classified by researchers) were quantitatively examined. In these ways, this study can help understand and differentiate PBL implementation in a way that may be underestimated in large-scale, group-level analyses (Molenaar and Nesselroade, 2015).

METHOD

Participants

Interview and observation samples were drawn from 12 inclusive STEM schools located across the U.S. enrolled in a larger National Science Foundation-funded study (see LaForce et al., 2016 for a summary). Schools were selected as a representative sample of schools to participate in on-site data collection, which consisted of a four day visit by two researchers to each school.

Interview sample. The interview sample of 64 teachers was created by randomly selecting three to five teachers from each school. Forty-four of the teachers that participated in an interview were also observed (27.7% of the total number of classrooms observed). Teachers were not asked to provide demographic data during interviews; as a result, demographic information is not known and not reported.

Classroom observation sample. The sample was composed of 129 classroom observations conducted at the same 12 inclusive STEM schools. Approximately half of the observations were of classrooms composed of a relatively equal number of males and females (54.8%) and mostly white students (48.8%). All grades (n = 35; 28.9% only-10^th grade student classes) and achievement (n = 105; 81.4% “regular-level” classes) levels were observed. Observations ranged in length from 25 to 180 minutes (M = 72.67, SD = 28.79 minutes) and included classrooms of 6 to 70 students (M = 20.90, SD = 9.21 students). The majority of observed classes were STEM discipline courses (n = 85, 72.6%). Researchers were instructed to classify an observation as a PBL experience if a) the teacher or students vocally articulated it as such (i.e., the teacher categorized the lesson as PBL), b) the lesson/activity included a tangible problem (as solving problems is a key feature in definitions of PBL in literature, as cited above), or c) the lesson/activity was comprised of several tasks that built upon each other over time (again a feature of PBL cited in literature, this also served to distinguish PBL lessons from those than asked students to solve, for example, a single applied or real-life math word problem); researchers coded 37 (28.7%) observations as PBL projects/experiences.

Procedure

Teacher interviews. During the on-site data collection period, teachers were invited to participate in interviews investigating experiences with STEM school instructional practices, including, but not limited to, PBL. Participating teachers were selected by schools points-of-contact (a school leader, teacher leader, or teacher identified during school recruitment to assist with data collection), who were asked to recruit at least one teacher from each grade level, with three to four teachers having their primary teaching duties in STEM subjects. Points-of-contact were instructed to select teachers randomly within these parameters. Interviews were, on average, 60 minutes in length, with all interviews audio-recorded and transcribed by the research team. Teachers were incentivized $50 (as Amazon.com cards) for their participation.

Classroom observations. Seven coders completed classroom observations at the 12 schools. Classrooms were selected by the schools’ points-of-contact to represent a cross-section of grade-levels and disciplines; each point-of-contact was asked to schedule 10 classroom observations, with at least 2 per grade level (9-12), spanning all disciplines but with a focus on STEM, and including relevant non-academic but core classes (such as advisory) as appropriate for each school. All classroom observations in the current sample were academic classes (core classes and academic electives). Seven classroom observations were double-coded (5.3%) and adequate inter-rater reliability was established across coders (.75-.84; ICC 95% CI: .07-.97, p = .02). Teachers were not incentivized for classroom observations, as these were part of the typical teacher work day.

Measures

Teacher interviews. A semi-structured interview protocol was used to collect data on instructional practices and teacher experiences at inclusive STEM schools. The protocol largely focused on PBL and how teachers think about and incorporate PBL experiences into their instruction. For instance, teachers were asked to describe what PBL looks like (e.g., Could you describe PBL in your experience at your school?; What makes PBL “good” or successful?), how PBL projects are carried out in the classroom (e.g., How do you plan PBL projects? Do you work with other teachers or teachers of different subjects?), and how they impact students’ learning (e.g., How can you tell whether PBL is successful?; How are students graded or scored on PBL?).

Classroom observations. All items were scored using a three-point Likert-type scale with values ranging from 1 (behavior not present) to 3 (behavior is present); these same items were used for both PBL and non-PBL lessons. Each construct represents a category of classroom behaviors identified as central to inclusive STEM high school models (LaForce et al., 2016) and focused on by teachers as elements of their PBL approaches. Average scale scores were computed across items measuring each construct to represent the extent to which each behavior was present during the classroom observation.

Cognitive demand. Eleven items were used to assess the level of cognitive demand and challenge present (e.g., Teacher asks students to reason and support their conclusions with evidence; α = .79).

Student autonomy. Five items were used to assess the degree of student autonomy and independent learning present (e.g., Teacher gives students opportunities to make choices that significantly shape their learning experiences; α = .76).

Student risk-taking. Four items were used to assess the degree of student risk taking present (e.g., Teacher encourages students to answer questions when they are unsure; α = .70).

Integration of concepts. Four items were used to assess interdisciplinary connections that were made (e.g., Teacher points out connections between the content of the lesson and other disciplines; α = .82).

Real world connections. Four items were used to assess the extent which connections to real-world scenarios in the students’ lives were made by both teachers and students (e.g., The teacher connects activities to the students’ lives; α = .79).

Student cooperation and teamwork. Six items were used to assess the presence of student cooperation and teamwork (e.g., Teacher encourages students to collaborate; α = .86).

Analytic Strategy

Teacher interviews. Teacher interviews were subject to inductive and deductive qualitative analysis in HyperResearch software to identify salient PBL instructional practices described by teachers. Qualitative coding began with a directed (deductive) content coding analysis. The first and second authors developed a list of content codes relevant to PBL instructional practices (e.g., cognitively demanding lessons, student autonomy, interdisciplinary content, connections to real-world scenarios) and a codebook outlining definitions for each. Thirteen interviews (20% of the overall sample) were double-coded, after which the first and second authors met to discuss the application of the codebook and areas of disagreement. Adjustments to the codebook were made, and all discrepancies were resolved so that 100% agreement was achieved. The first and second authors divided remaining interviews and coded them using the updated version of the codebook. During this coding, conventional content coding (inductive) was also used to identify emergent sub-themes. Since teacher interviews included discussions of both PBL and instruction more generally, researchers coded only where it was clear that the teacher was referencing PBL. Throughout the coding process, the first and second authors wrote detailed memos of their reflection on how codes were related and should be synthesized into broader categories (DeCuir-Ginby, Marshall, and McCulloch, 2011).

Classroom observations. A series of independent samples t-tests were used to examine mean differences in the presence of classroom behaviors central to the success of STEM schools across classroom observations classified as PBL versus non-PBL projects/experiences.¹

RESULTS

Teacher Interviews

In their descriptions of PBL, a wide variety of approaches and implementations were discussed by teachers. However, a number of instructional practices emerged as most prominent across schools and academic subjects: student autonomy, student cooperation and teamwork, integration of subjects, real-world connections, and cognitive demand. Each of these and associated themes and findings are discussed below.

Student Autonomy. Of the 64 teachers interviewed, 43 (67.2%) discussed student autonomy in the context of PBL, with 114 references in total. Their responses can be organized into three sub-themes, discussed below.

Student independence. Teachers valued students’ development of independence, and they reported that PBL experiences served as a mechanism by which students could become ‘more independent.’ Teachers also noted that over time, through using PBL, students became more autonomous in their learning and teachers were able to shift from a role of information provider to facilitator. As one teacher described it,

“that’s basically how it looks [in PBL], a lot of independent learning, at the same time with the guidance and the support [of the teacher] in the classroom.”

Student-led learning. As students’ autonomy increases over the course of their high school years, teachers described giving students more responsibilities for leading. This was reported to happen in a number of ways. One teacher discussed how she ‘…get[s] the students to where they’re running the class’ whereas others references involving students in the selection and design of projects themselves. Another teacher echoed these comments,

“So it’s kind of shifting who’s developing our objectives for the day. That’s a major shift from like, ‘I’m telling you the objectives, you’re going to learn it’, to, you have questions, you tell me what you want to learn, and we’ll see if we can come together to provide an experience for you.”

Student-led learning allows students to ‘figure it out,’ which was noted as critical in PBL. Teachers reported that they strive to have

“every single [project] always [be] student-centered, student-driven, [where] they get to brainstorm, they get to plan. And they have to solve the problem.”

Students make choices. Both processes (i.e., student independence and student-led learning) require students to make choices about what they want to learn and how they will learn. Having the ability to make choices allows students to drive their own instruction, which teachers noted as central to successful PBL. One teacher mentioned that he has

“a list of projects and [I] let the students choose so that I don’t have everybody doing the same project [because] that’s boring. Boring for me, boring for them.”

Though teachers reported that stepping back to take on the ‘facilitator’ role can be difficult at first, they also said that they

“never having a problem with letting them [the students] choose. The moment I started letting the students choose what they like and what’s personal to them, the projects got way better.”

Overall, findings reflect teachers’ belief in the importance of developing students’ confidence to direct and manage their own learning, and that PBL serves as an avenue in which they can foster these behaviors.

Student Cooperation and Teamwork. Teachers also often discussed the importance of group work and having students work together collaboratively (140 responses by 48 teachers; 75% of the sample). Two major emergent sub-themes are discussed.

Helping students learn to collaborate. Teachers were largely in favor of students working together in groups or pairs, often noting that this was a major feature of their PBL projects. However, teachers also described a type of ‘true’ collaboration—not just being in groups where the work gets completed separately and then pieced back together into a final product—that was seen as a skill in and of itself, and one that must be built over time:

“That’s part of the maturity going from a 9^th grader to working collaboratively. What they see as collaboration is not really what we want it to be (…) So we’re teaching and trying to show them a skill. It’s a process that’s really ongoing.”

Another teacher echoed this point:

“[our principal] loves to say the freshmen love to be together, but not work together. So we’re helping them learn to work together.”

As one example of this, many teachers reported choosing groups for students, rather than letting them select their own groups, at least initially. Teachers did this often based on students’ interests, but also stressed the importance of students learning to work with others who were not their friends or students who they did not necessarily choose to work with. This was discussed as a way to encourage collaboration skills like ‘the ability to collaborate with people that you don’t necessarily choose to work with.’ Over time, when students had more practice with PBL and peer collaboration, more flexibility was built in. Teachers similarly described scaffolding students’ learning in this area by having groups write contracts, again particularly with students early on in their high school careers:

“They articulate the leadership style they’re going to use, the decision-making model they’re going to use, the norms on the team, how they’re going to resolve conflict on the team. Everybody signs the contract in the group, they get it checked by me, and then they start the design process.”

Part of the need for this type of scaffolding came from teachers’ conception that collaboration not only encompasses a set of skills necessary for successfully completing PBL projects from start to finish, but also for students to have generally. One teacher noted that

“collaboration has many subskills like how to make decisions, clear communication, specific, precise, conflict mediation and negotiation, document[ation]. So there’s all these subskills that go with project management and collaboration.”

These skills, teachers noted, are ‘almost equally important’ to the content being learned. As a whole, these findings highlight that group work and collaboration with classmates is not only an existing skillset that can be applied in PBL projects, but one that can be learned through PBL.

Students teaching each other through group work. One particular skill related to collaboration and group work that teachers often discussed was students’ ability to mentor, provide guidance, and/or teach fellow group members. These were noted as components central to successful PBL. As one teacher stated,

“With the PBL projects you can, for some of the content workshops, you can break the students up, they don’t always have to be in their teams. But then they’re in their teams so the stronger students can help the more struggling students.”

Several teachers referenced Vygotsky’s Zone of Proximal Development (Vygotsky, 1978), saying that they try to group students so that ‘everybody raises up’ in some way. “I… set it up in teams,” one teacher said, so that

“all my kids who need the extra help are sitting with someone who can provide it, and that really helps a lot. Then the person providing the help is going to an even higher level.”

In these scenarios, support of student collaboration helps students take shared responsibility for the task or PBL project, and their overall learning as well, rather than relying on their teachers for help or as the one authority. Another teacher underscored this point, saying

“There’s a saying that there’s no better way to learn than to teach it. That’s definitely the case here, and that’s one of the beauties of the way we do PBL. With the kids working in groups, they do teach each other all the time and they’re a lot more willing to listen to each other as opposed to the teacher standing up in front of the class and lecturing all the time.”

Integration of Subjects. Many teachers (46 teachers; 71.9% of the sample; 110 responses total) noted that PBL provides a good way to incorporate interdisciplinary instruction and the integration of content across subject areas. When disciplinary content is integrated, students are able to make connections to understand how such subject areas reinforce and complement each other. Teachers described integrating disciplinary content in three ways, described below.

PBL across multiple classes. Teachers at a number of schools described PBL projects which included content from multiple classes. A number of teachers at different schools described doing this by looking at the standards for commonalities across disciplines, and building out a central question or project from there. In some cases, these types of projects happened at the same time in all subject areas, while in others, teachers described projects that “moved” from class to class over the course of a semester:

“The manuscript project was between myself and the chemistry teacher and the math teacher. In chemistry, they [the students] came up with the study they were going to do - they did it in their class - and they analyzed it in math, and then they brought all of that information into my class and put it into manuscript form to try to submit to a journal.”

Certain structural elements of the school contribute largely to the feasibility of a multi-class approach to integrating disciplinary content in PBL. For instance, structuring PBL so that students are able to work on the same project in multiple classes throughout the academic day is possible only when teachers share the same group of students. As one teacher noted,

“When we started STEM, we were in a full-year long class. Everyone had me for biology and [another teacher] for social studies. There was an entire team that shared the same kids… Now we are double-block. We don’t share the same kids. Something we are struggling with here is to keep the STEM identity, what STEM is, cross-disciplinary units, modules, whatever… [because] we don’t share the same kids at the same time.”

Co-teaching for subject integration. Teacher collaboration, co-teaching, or team teaching allows teachers to collaborate in order to deliver interdisciplinary content to students. This was sometimes described as being done through hybrid classes (e.g., biotechnology or environmental science classes combining social studies and science). One teacher highlighted how subject integration contributes to students’ learning of real-world content in PBL, stating,

“Both of my classes are co-taught across disciplines, which is important because in the real world disciplines aren’t silo-ed. So with a collaborating teacher we look at where our standards overlap and come up with a curriculum map of usually eight to ten projects per year.”

A second teacher supported this idea:

“In bio and tech, the technology is really skills and a means to communicate or means to solve a problem. So the content is almost all biology, but then the students are maybe designing or building a microscope or building a gel electrophoresis or they’re creating a Mixonium [presentation]. So there’s a way that technology kind of serves the content.”

Integrating content within classes/disciplines. PBL project experiences may also be structured to encourage students to incorporate knowledge and skills from multiple disciplines within a single class. Teachers felt that ‘it’s better to integrate than to not,’ and that it’s important to ‘just [bring] that emphasis out for them [students].’ When describing PBL projects that focused on a single discipline or occurred within a single class, teachers often noted trying to be explicit about and draw students’ attention to the connections to other content areas, and working to help students make these connections themselves.

Challenges with integrating content. While even teachers who reported not integrating content into their PBL projects considered integrated projects to be ideal, a number of challenges to doing so were noted. Teachers discussed how time constraints and limited opportunities to enact collaborative teacher planning adversely impacts their ability to incorporate interdisciplinary instruction during PBL. For example, one teacher said,

“[W]ith the time available to us to schedule that kind of stuff [PBL] and the way that kids are scheduled with the blocks, there’s no other teacher that I share all the same kids with now. So it [subject integration in PBL] just doesn’t work out well.”

Teachers also noted how subject integration is ultimately tied to successful cross-department collaboration amongst one’s colleagues, which can be a challenge generally. Another teacher highlighted this point when discussing how, at his school, the teachers ‘usually can do [a] really good connection with four out of the five [subject areas];’ however, without collaboration from all teachers, they do not feel ‘…like they have a really good piece of that project.’

Thus, even when there is opportunity and time for collaborative teacher planning, creating interdisciplinary PBL projects can be challenging. These teacher responses indicate that subject integration in the context of PBL projects is largely dependent on having time and structures in place to support cross-discipline collaboration.

Real-world Connections. Making connections between the real world and the work being done in PBL projects was the instructional practice most frequently referenced by teachers (62 teachers; 96.9% of the sample; 278 responses total). Teachers’ responses can be categorized into two broad themes: 1) strategies by which real-world connections can be made, and 2) differentiating between making content connections in projects and making connections to the real world via skills/processes that students will use in the real world.

Strategies to make real-world connections. Teachers primarily discussed three important strategies that can help students make connections to the real world during PBL projects.

Involvement of external partners. Cultivating relationships with external partners is a central feature of inclusive STEM high schools (LaForce et al., 2016). Schools work with partners to create a presence in their communities, and to provide opportunities for students to apply skills learned in the classroom to real-world settings. Teachers talked about a number of ways in which local community members support their implementation of PBL; for example, one teacher said that

“community members judge my mock congressional hearings. We also have community members come in and judge the debates.”

This role of partners as an external audience for students’ final PBL products or presentations was described by a number of teachers across STEM schools. Teachers also referenced how outside partners can support students’ ability to make real world connections by visiting classrooms and ‘speaking about their particular field or just how to be successful.’

Other teachers described more in-depth involvement in PBL projects by partners, from providing the context for a project to co-design and high levels of collaboration with teachers throughout. Partnerships were not limited to interactions with local businesses, but also extended to partnerships with individuals and groups in higher educational settings, including community colleges and state universities located in the greater community. One teacher discussed a successful partnership with a biology professor from a local university, and how the partnership is continuing to the benefit of his students:

“He [contact at the university] said, ‘I think we should talk about micelles because one of my projects is, I work on drug delivery systems using these things called micelles. There’s a polar and non-polar piece and you can talk about hydrogen bonding and covalent bonding, everything you wanted to talk about, in a very authentic context’. And he [contact at the university] said, ‘I can supply you with all the polymers and some grad students, and they can come out, and they can work with your kids”. It was just beautiful so we’ve been doing it like every year.”

Another teacher referenced the importance of such partnerships for his students’ college readiness and exposure to STEM-related careers prior to selecting a college major:

“[W]e have huge partnerships with places in and out of Columbus, [Ohio]. They helped us create the curriculum, so we want to keep them involved. You [the partnering organization] have problems, can these kids help work on these problems? Is there an internship opportunity? Students can see things they like and don’t like prior to heading off to college.”

A major benefit of such partnerships noted by teachers was that they can ‘help kids get internships, [which is] a big thing.’ Connections to the real world in PBL through external partnerships have tangible benefits to students not only during the course of a given project or even their high school careers, but also as they move forward through their lives and academic careers.

Real-world needs in the community or school. A number of teachers indicated that PBL experiences which engage students in projects or problems that are tied to actual events or problems happening around them—in their schools, their communities, or in the world—while often the most difficult to plan and execute, are the pinnacle of PBL projects. One teacher talked about the importance of

“making it relevant to a real-world problem that is occurring today. It is not just a textbook problem. A lot of the things we have done previously in the 10 years I have been teaching, they are just synthetic. You make up, here’s a scenario, but it doesn’t have a real-world connection. Here’s a scenario I make up, I know it’s fake; the kids know it’s fake. But they will work through it, but it doesn’t have the same sense to them.”

Relationships with external partners often helped to facilitate these types of projects, particularly when they centered on problems or needs outside of the schools themselves. For example, one teacher described a project in which

“We’re collaborating with farmers in Léogâne, Haiti to try to do sustainability projects. They’re [the 9^th grade students] designing solar fruit dryers that we go and implement and will actually happen… I take students to Haiti every year and we go do these projects with our sister school in [the] little community [there].”

While many schools may not have the resources to send students to another country, other teachers described similar real-world PBL projects taking place within the school. One described a project that arose out of a simple classroom need:

“I have a 3D printer in my classroom. It’s just sitting on a table, a very unsteady table. There’s a spot in the back corner of my classroom where it would fit, but there’s no table there. My students are designing a table. So basic ideas like, ‘Hey, I need that designed.’ The kids can do it.”

Teachers also discussed engaging students in outside academic competitions as part of their PBL experiences as an authentic and motivating way to connect to the real world. One teacher described how his students participated in

“a national competition, called CyberPatriot …out of the Air Force, and what they [students] do is they can go in and take out malicious code. So we’re on the good side of hacking.”

In all of these examples, there exists common theme of students not only connecting content they are learning to the real world, but producing meaningful work products that are actually being used or applied, whether in their classroom, their community, or across the world.

Theoretical real-world problems or projects. Realistically, not all PBL projects can be directly connected to a problem faced in the school or community; some projects are better represented as hypothetical scenarios to be investigated in greater depth. Teachers referenced the importance of hypothetical scenarios in that they help shift students’ mindset regarding their academic purpose and potential. For instance, one teacher believed that posing hypothetical problems to students in PBL

“gets them to step out of the ‘I’m a STEM student’ [and, instead gets] them to think ‘I’m a researcher called in by the CDC’ or ‘I’m an inventor’ or ‘I’m a game designer’ or building robotics. We don’t want them to think they’re just a student, and ‘my presentation is just for my teachers.’”

Another teacher similarly discussed a scenario he presented to his students in which he told them,

“You’re a member of an engineering firm who’s been tasked with designing x, y, z for this customer. Here are the requirements. Here are the materials that you have available.”

In this type of PBL, students have to apply the skills they have learned when presented with a hypothetical situation that they may one day encounter in actuality.

Skill/process connections versus content connections. When discussing how PBL experiences connect to the real-world, teachers highlighted two different ways that those links are made: first, through content that is rooted in real world scenarios, and second, through connecting skills that students will need in the real world. The first category includes projects like those mentioned above, where students are solving problems that are actually occurring. Many teachers described making these types of connections, and the importance of ‘authenticity’ of the content they include in PBL projects. However, teachers also noted that

“so much of [what they learn from PBL] is not even content based, it’s just the actual real-life experience. Preparing them to actually be productive outside of school and in the future.”

This preparation included skills that teachers felt would be important for students in the workplace in particular, such as communication and collaboration (as discussed above), work processes, relationship building, and working to meet expectations. For example, one teacher described changing the structure of his rubrics to better align with how performance expectations are communicated in jobs:

“For the overall project I have done away with the category of ‘emerging’. I don’t think that’s useful at all (…) My argument for that is that we strive for authenticity. I’ve never had a job [where my boss said], ‘Here’s your rubric. Look, it’s your emerging column. If you do this you’re only going to get three quarters of your pay.’ I want them to know, here’s my expectations. If you don’t meet them, you’re going to fail. If you go above and beyond, you’re going to get a better grade.”

These skills may be taught in many ways, but many teachers noted that they could be done especially well through PBL projects. As one said,

“one of the big parts of a PBL that’s good is that you combine a lot of different things to kind of solve one problem, [and] that’s what you do in the real world.”

Cognitive demand. Rigorous learning is central to inclusive STEM school curricula (LaForce et al., 2016), and teachers reported that this approach is no different in PBL (124 responses by 45 teachers; 70.3% of the sample).

Rigorous and challenging content rooted in academic standards. Rigorous and challenging content engages students in higher-level, critical thinking. When asked “What makes good PBL?,” one teacher stated that “I would always start with rigorous content, because that to me is very important.” Such content, teachers described, is often derived from and aligned with academic standards set forth by the district, state, or nation. Indeed, one teacher stated, “If you go way back, it always starts with standards.” Other teachers supported this idea, with one noting that

“to just do a project to do a project isn’t that meaningful, but to do a project to build on a learning goal or to build on a standard is what we’re trying to look for.”

Thus, PBL projects must achieve what one teacher described as ‘evidence of learning,’ and also maintain academic rigor and challenge.

Higher-level/critical thinking. Achieving this balance in PBL is possible through an emphasis on the development of critical thinking skills. Teachers were in agreement on this point: “I like to focus on, in my class, the higher-level thinking.” As such, ‘expanding their [students’] cognitive processes’ is central to completion of complex PBL projects as well as students’ overall academic achievement. One teacher elaborated that critical thinking in PBL helps students ‘to think about all the ways content or the project impacts themselves, the world, politics, whatever.’ Through PBL, teachers reported ‘[p]ushing them [students] to that higher level [of thinking]’ that ‘accelerates their learning.’

The scientific method. Underscoring the development of critical thinking skills and focusing on more-advanced, higher-level problem-solving in PBL directly connects to engagement in the scientific method. Carrying out the scientific method requires students to ‘do research in [their] area,’ as well as to ‘make a claim and back it up with evidence.’ This process allows students to build skills related to data collection, research design, statistical analysis, and citing sources, particularly important in STEM disciplines. For instance, one teacher highlighted how PBL projects provide a context for the students to practice and enhance their research skills through the scientific method, and that this process evolves over the course of high school students’ academic careers in STEM schools;

“[In PBL] we’ll research skills and documentation of evidence, so that they’re (...) looking at a community problem and looking at solutions for that problem and advocating for that, that they have the data correct and they know how to cite that data and use the correct sources. That’s still a problem by 12th grade. Just that basic research on that. So those skills would include the writing skills in a research paper, the development of a thesis, and using reliable data, not just WebMD. That still happens. The other big skill, besides the research, is then the cohesiveness or synthesizing the data. Then also looking for solutions on that.”

Differences in Instructional Practices in PBL and Non-PBL Classrooms

Understanding how teachers described their PBL practice, we also explored whether these instructional practices were more prevalent in observed PBL classes than non-PBL ones. Results of a series of independent samples t-tests indicated that class sessions classified as PBL experiences had a higher presence of student autonomy (t(123) = -4.43, p < .001), integration of concepts (t(39.29) = -2.19, p = .04), and student cooperation and teamwork (t(87.37) = -3.29, p = .001) compared to non-PBL class sessions. No significant differences emerged between class sessions classified as PBL versus non-PBL experiences for cognitive demand, student risk taking, or real world connections. Descriptive statistics and correlations for key study variables are presented in Table 1.

DISCUSSION

Overall, these findings contribute to understandings of what “PBL” means in the context of inclusive STEM schools—what instructional practices teachers are using, as well as, from the interviews, why and how. As noted earlier, specifications in the definitions of PBL in the literature and in implementation in schools vary (e.g., Brassler and Dettmers, 2017; Savery, 2006). Therefore, in order to ultimately determine if PBL approaches are working in these schools, there is a need to be clear on what exactly PBL in inclusive STEM schools means. This is not to imply that PBL will or should look the same at every school or in every classroom—one of its strengths, in fact, may lie in its flexibility and adaptability to different disciplines, school structures, and groups of students—but our findings show that there are common features of PBL most prevalent or important to the teachers implementing it in the inclusive STEM school context, and that these do differ in some important ways from what is seen in classrooms not employing PBL.

In our interviews with teachers, student autonomy, student cooperation and teamwork, integration of subjects, real world connections, and cognitively demanding work emerged as critical to teachers’ conceptions of PBL. These findings were echoed in what we saw in practice, as in classrooms using PBL, compared to those that were not, we saw significantly higher levels of student autonomy, student cooperation and teamwork, and integration of subjects. These findings generally support the definitions and common elements of PBL found in the literature, which characterize it as student-centered and driven, often including group work, interdisciplinary, and closely connected to the real world (Buck Institute for Education, n.d.; Brassler and Dettmers, 2017; Savery, 2006; Torp and Sage, 2002). They go further, however, in describing in detail what these practices look like for teachers and students in inclusive STEM high schools, and why PBL serves as a valuable method for implementing them.

The ideas of student autonomy and students’ ability to work in teams may seem like opposing ones, however, both came through as important practices in PBL, and in teachers’ descriptions, it is clear that they can be considered two sides of the same coin. Both involve students developing understandings of their learning styles, needs, and strengths, and the self-confidence to make decisions and follow through on them. For autonomy, this manifests in managing their time and their work, making choices about the learning processes that work best for them, and working without direct oversight to accomplish their tasks. In working with groups, this facilitates understanding where they can contribute and where they can benefit from the support or skills of their peers, and moving towards the true collaboration (rather than a “divide and conquer” approach to group work) described by teachers as the goal. Through PBL, teachers incorporate more practices that help students learn to both depend on themselves and navigate the challenges of having to depend on others. These skills of self-regulation, self-direction, and teamwork are also considered central to both problem- and project-based approaches in the literature (e.g., Savery, 2006), showing that there is consistency in the understanding of the ‘essence’ of PBL.

Our findings also stress PBL as a vehicle for contextualizing learning and knowledge—in the “real world” and as situated across disciplines, rather than siloed within them. Teachers described various strategies for accomplishing both, emphasizing that grounding the content and the learning processes in these ways makes the experiences more meaningful for students. Additionally, having students apply what they are learning in real-world scenarios, in the actual world, and across their classes helps combat the stereotypical student lament of ‘when am I ever going to need to know this?’ as it demonstrates for students how what they are learning can be used. More meaningful experiences, teachers felt, promote student engagement and deeper learning.

None of these types of learning or skills (i.e., autonomy, collaboration, or contextualized learning) are unique to PBL, or can only be accomplished through PBL. However, the quantitative data collected in our classroom observations suggests that teachers more often use instructional practices to facilitate and build student competencies in these behaviors when teaching using PBL than with traditional methods of instruction. Given the variety of ways in which teachers described their approaches to them, it may be that PBL provides a framework that at once emphasizes the importance of these components and allows enough flexibility for teachers to incorporate them in ways that are feasible in their contexts or in which they feel most comfortable. One critique of PBL has been that while it has been shown promote the development of these types of 21^st century skills, as well as student interest and engagement, there is less evidence of its effectiveness in delivering content (Strobel and van Bareveld, 2009). Our findings challenge this, as teachers consistently described a specific focus on incorporating rigorous content into their PBL experiences for students, particularly by planning their projects around discipline standards to ensure that the appropriate content learning is occurring. Teachers also noted repeatedly that they felt PBL supported students’ engaging in inquiry processes, the scientific method specifically, and greater amounts of higher level thinking. As such, our findings suggest that in inclusive STEM schools, high cognitive demand is indeed an essential piece of PBL.

It is important to note that implementing PBL can be a challenge for teachers,especially in schools where there may be less flexibility in structures (e.g. students’ schedules) that support the use of PBL and for teachers in their teaching styles, as it represents a large shift from traditional styles of instruction. Many teachers noted that support from their principals in being able to try and fail was important in their eventual success with PBL—this sort of leadership is characteristic of inclusive STEM schools (Peters-Burton et al., 2014), but may unfortunately not be present for all teachers, regardless of their school type.

LIMITATIONS AND FUTURE DIRECTIONS

There are several considerations for interpreting the results of this study. First, this study is cross-sectional and correlational, with classroom observations and teacher interview data collected at a single time point. As a result, conclusions regarding changes or growth in teachers’ instructional practices and students’ engagement in their PBL experiences inside the classroom cannot be drawn. Future research should observe teachers and students engaging in PBL experiences at multiple time points and interview teachers throughout the academic year. Employing longitudinal research methods will help provide a more-thorough understanding of PBL implementation at inclusive STEM high schools, which has the potential to contribute to a greater understanding of how PBL experiences impact students’ learning outcomes and future participation in STEM careers.

In addition, the multifaceted and multidimensional nature of PBL poses measurement challenges. PBL is not a “one-size, fits all” educational strategy, and PBL experiences often differ across teachers, grade levels, and/or schools, despite sharing the same PBL label. A single definition of PBL, or implementation of PBL, does not exist across all schools. While these inconsistencies in definition and implementation of PBL may somewhat limit the generalizability of these findings to students and teachers in other grade levels and school types, the findings do indicate themes common across all operationalizations of PBL. This suggests that there is some shared understanding of the core elements of PBL approaches across teachers and STEM schools, which are largely consistent with previous research literature. Given that this work focused on high schools, and specifically on inclusive STEM high schools, there is also a limit to the conclusions that can be drawn about the applicability of PBL experiences in other settings. However, one strength of this work, and one that should continue to be investigated in future research, involves pairing quantitative classroom observational data with qualitative teacher interview responses (i.e., mixed-methods) to gain a more in-depth understanding of teachers’ instructional practices and students’ behaviors during PBL, and how classroom experiences are structured to facilitate engagement in PBL.

Future research should also explore students’ perspectives on PBL, particularly given students’ active, self-directed involvement in these learning experiences. A wider age range of students should also be investigated to assess elementary and middle school students’ participation in PBL experiences, and whether they are developmentally-appropriate learning experiences for younger students. Moreover, such perspectives should not be limited to students (and teachers) from inclusive STEM schools as PBL instructional practices are also implemented in traditional, non-STEM schools. Examination from multiple perspectives can provide a more well-rounded understanding of how PBL is carried out in classrooms with high school students, and highlight similarities and differences in PBL implementation across types of school settings (i.e., STEM versus non-STEM schools), grade levels (i.e., elementary and middle school), and diverse populations of students.

1. This approach was chosen instead of conducting a series of one-way analysis of covariance (ANCOVA) tests or a multivariate analysis of variance (MANOVA) due to the overall sample size of classroom observations (N = 129) and because of the relatively small sample size of classroom observations labeled as PBL project/experiences (n = 37).↩

• Albanese, M. A. and Mitchell, S. (1993). Problem-based learning: a review of literature on its outcomes and implementation issues. Academic medicine, 68(1), 52-81. https://doi.org/10.1097/00001888-199301000-00012
• Alfieri, L., Brooks, P. J., Aldrich, N. J. and Tenenbaum, H. R. (2011). Does Discovery-Based Instruction Enhance Learning? Journal of Educational Psychology, 103(1), 1-18. https://doi.org/10.1037/a0021017
• Allen, D. E., Duch, B. J. and Groh, S. E. (1996). The power of problem-based learning in teaching introductory science courses. New directions for teaching and learning, 1996(68), 43-52. https://doi.org/10.1002/tl.37219966808
• Almarode, J. T., Subotnik, R. F., Crowe, E., Tai, R. H., Lee, G. M. and Nowlin, F. (2014). Specialized high schools and talent search programs: Incubators for adolescents with high ability in STEM disciplines. Journal of Advanced Academics, 25(3), 307-331. https://doi.org/10.1177/1932202X14536566
• Baran, M. and Maskan, A. (2011). The effect of project-based learning on pre-service physics teachers electrostatic achievements. Cypriot Journal of Educational Sciences, 5(4), 243-257.
• Barron, B. and Darling-Hammond, L. (2008). Teaching for Meaningful Learning: A Review of Research on Inquiry-Based and Cooperative Learning. Book Excerpt. George Lucas Educational Foundation.
• Baumgartner, E. and Zabin, C. J. (2008). A case study of project‐based instruction in the ninth grade: A semester‐long study of intertidal biodiversity. Environmental Education Research, 14(2), 97-114. https://doi.org/10.1080/13504620801951640
• Bell, S. (2010). Project-based learning for the 21st century: Skills for the future. The Clearing House, 83(2), 39-43. https://doi.org/10.1080/00098650903505415
• Boud, D. and Feletti, G. (1997). The challenge of problem-based learning (2nd ed.). London: Kogan Page.
• Brassler, M. and Dettmers, J. (2017). How to Enhance Interdisciplinary Competence—Interdisciplinary Problem-Based Learning versus Interdisciplinary Project-Based Learning. Interdisciplinary Journal of Problem-Based Learning, 11(2), 12. https://doi.org/10.7771/1541-5015.1686
• Buck Institute for Education (n.d.). What is project-based learning (PBL)? Retrieved from: http://www.bie.org/about/what_pbl
• Cheng, W. Y., Lam, S. F. and Chan, C. Y. (2008). When high achievers and low achievers work in the same group: The roles of group heterogeneity and processes in project‐based learning. British Journal of Educational Psychology, 78(2), 205-221. https://doi.org/10.1348/000709907X218160
• Committee on STEM Education National Science and Technology Council (2013). Federal science, technology, engineering, and mathematics (STEM) education: 5-year strategic plan. Retrieved from http://www.whitehouse.gov/sites/default/files/microsites/ostp/stem_stratplan_2013.pdf
• DeCuir-Gunby, J. T., Marshall, P. L. and McCulloch, A. W. (2011). Developing and using a codebook for the analysis of interview data: An example from a professional development research project. Field Methods, 23, 136-155. https://doi.org/10.1177/1525822X10388468
• Economics and Statistics Administration (2017, March 30). STEM Jobs: 2017 Update. Retrieved from: http://www.esa.doc.gov/reports/stem-jobs-2017-update
• English, L. D. and King, D. T. (2015). STEM learning through engineering design: fourth-grade students’ investigations in aerospace. International Journal of STEM Education, 2(14), 1-18. https://doi.org/10.1186/s40594-015-0027-7
• Ertmer, P. A., Schlosser, S., Clase, K. and Adedokun, O. (2014). The Grand Challenge: Helping Teachers Learn/Teach Cutting-Edge Science via a PBL Approach. Interdisciplinary Journal of Problem-Based Learning, 8(1), 4-20. https://doi.org/10.7771/1541-5015.1407
• Faris, A. (2008). The Impact of PBL on the Students’ Attitudes towards Science among Nine Graders in Hamza Independent School. Online Submission.
• Gorman, M. (2013). STEM and PBL… A natural and essential connection. Retrieved from: https://www.bie.org/blog/stem_and_pbl_a_natural_and_essential_connection
• Han, S., Capraro, R. and Capraro, M. M. (2014). How science, technology, engineering, and mathematics (STEM) project-based learning (PBL) affects high, middle, and low achievers differently: The impact of student factors on achievement. International Journal of Science and Mathematics Education, 1-25. https://doi.org/10.1007/s10763-014-9526-0
• Heerwagen, J., Kelly, K. and Kampschroer, K. (2016). The changing nature of organizations, work, and workplace. Retrieved from: https://www.wbdg.org/resources/changing-nature-organizations-work-and-workplace
• Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235-266. https://doi.org/10.1023/B:EDPR.0000034022.16470.f3
• Hmelo-Silver, C. E., Duncan, R. G. and Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational psychologist, 42(2), 99-107. https://doi.org/10.1080/00461520701263368
• Holm, M. (2011). Project-based instruction: A review of the literature on effectiveness in prekindergarten. River academic journal, 7(2), 1-13.
• Kilpatrick, W. H. (1921). Dangers and difficulties of the project method and how to overcome them: Introductory statement: Definition of terms. Teachers College Record, 22 (4), p. 283- 287 (ID Number: 3982) Retrieved from: http://www.tcrecord.org
• Klobuchar, A. (2014). STEM education for the innovation economy. Washington, DC: Joint Economic Committee. Retrieved from: http://www.jec.senate.gov/public/?a=Files.Serve&File_id=9bfced75-07a0-466b-a94b-8ab399582995
• Laboy-Rush, D. (2011). Integrated STEM education through project-based learning. Learning.com. Retrieved from: https://www.rondout.k12.ny.us/common/pages/DisplayFile.aspx?itemId=16466975
• LaForce, M., Noble, E., King, H., Century, J., Blackwell, C., Holt, S., ... and Loo, S. (2016). The eight essential elements of inclusive STEM high schools. International Journal of STEM Education, 3(1), 21. https://doi.org/10.1186/s40594-016-0054-z
• LaForce, M., Zuo, H., Ferris, K. and Noble, E. (2019). Revisiting Race and Gender Differences in STEM: Can Inclusive STEM High Schools Reduce Gaps?. European Journal of STEM Education, 4(1), 8. https://doi.org/10.20897/ejsteme/5840
• Lou, S. J., Shih, R. C., Diez, C. R. and Tseng, K. H. (2011). The impact of problem-based learning strategies on STEM knowledge integration and attitudes: an exploratory study among female Taiwanese senior high school students. International Journal of Technology and Design Education, 21(2), 195-215. https://doi.org/10.1007/s10798-010-9114-8
• Means, B., Confrey, J., House, A. and Bhanot, R. (2008). STEM high schools: Specialized science technology engineering and mathematics secondary schools in the US. SRI International, 25.
• Mergendoller, J. R., Maxwell, N. L. and Bellisimo, Y. (2006). The effectiveness of problem-based instruction: A comparative study of instructional methods and student characteristics. Interdisciplinary Journal of Problem-based Learning, 1(2), 5. https://doi.org/10.7771/1541-5015.1026
• Miller, A. (2017). PBL and STEAM education: A natural fit. Retrieved from: https://www.edutopia.org/blog/pbl-and-steam-natural-fit-andrew-miller
• Mioduser, D. and Betzer, N. (2007). The contribution of Project-based-learning to high-achievers’ acquisition of technological knowledge and skills. International Journal of Technology and Design Education, 18(1), 59-77. https://doi.org/10.1007/s10798-006-9010-4
• Molenaar, P. C. M. and Nesselroade, J. R. (2015). Systems methods for developmental research. In W. F. Overton and P.C.M. Molenaar (Eds.), Handbook of Child Psychology and Developmental Science, Volume 1: Theory and Method (7th ed., pp. 652-682). Editor-in-chief: R.M. Lerner. Hoboken, NJ: Wiley. https://doi.org/10.1002/9781118963418.childpsy117
• National Academies of Sciences, Engineering, and Medicine (2017). Information Technology and the US Workforce: Where Are We and Where Do We Go from Here?. National Academies Press.
• National Academies of Sciences, Engineering, and Medicine (2018). Science and Engineering for Grades 6-12: Investigation and Design at the Center. Washington, DC: The National Academies Press.
• National Research Council. (2011). Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics. National Academies Press.
• Norman, G. R. and Schmidt, H. G. (1992). The psychological basis of problem-based learning: A review of the evidence. Academic medicine, 67(9), 557-565. https://doi.org/10.1097/00001888-199209000-00002
• Peters-Burton, E. E., Lynch, S. J., Behrend, T. S. and Means, B. B. (2014). Inclusive STEM high school design: 10 critical components. Theory Into Practice, 53(1), 64-71. https://doi.org/10.1080/00405841.2014.862125
• Savery, J. R. (2006). Overview of problem-based learning: Definitions and distinctions. Interdisciplinary Journal of Problem-based Learning, 1(1), 3. https://doi.org/10.7771/1541-5015.1002
• Strobel, J. and van Barneveld, A. (2009). When is PBL more effective? A meta-synthesis of meta-analyses comparing PBL to conventional classrooms. Interdisciplinary Journal of Problem-based Learning, 3(1), 44-58. https://doi.org/10.7771/1541-5015.1046
• Tal, T., Krajcik, J. S. and Blumenfeld, P. C. (2006). Urban schools’ teachers enacting project‐based science. Journal of Research in Science Teaching, 43(7), 722-745. https://doi.org/10.1002/tea.20102
• Thomas, J. W. (2000). A review of research on project-based learning. Retrieved from http://www.bobpearlman.org/BestPractices/PBL_Research.pdf.
• Torp, L. and Sage, S. (2002). Problems as possibilities: Problem-based learning for K-16 education (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
• Vernon, D. T. and Blake, R. L. (1993). Does problem-based learning work? A meta-analysis of evaluative research. Academic medicine, 68(7), 550-563. https://doi.org/10.1097/00001888-199307000-00015
• Vygotsky, L.S. (1978). Mind in Society. Cambridge, MA: Harvard University Press.
• Young, V. M., House, A., Wang, H., Singleton, C. and Klopfenstein, K. (2011, May). Inclusive STEM schools: Early promise in Texas and unanswered questions. In Highly Successful Schools or Programs for K-12 STEM Education: A Workshop. Washington, DC: National Academies. Retrieved May (Vol. 1, p. 2014).