An Iterative Participatory Approach to Developing an Early Childhood Problem-based STEM Curriculum

This paper describes an iterative participatory curriculum design approach to developing a problem-based STEM curriculum for preschool children. The curriculum aims to teach young children problem-solving using an adapted version of the engineering design process (EDP). Despite evidence showing that a rigorous, integrated STEM curriculum promotes cognitive development and curiosity, very little STEM or engineering instruction occurs in classrooms for threeto five-year-old children, and few studies include teachers in the curriculum design process. Research has shown that, when children experience an engineering curriculum, they show an increase in engagement, in the number of engineering behaviors displayed, and in persistence in completing activities. As well, when teachers are involved in designing curriculum, they are more likely to feel empowered and sustain implementation. Qualitative analysis of semistructured interviews with 13 preschool teachers after the development process showed that teachers who participated in the process perceived increased knowledge and self-efficacy in teaching STEM in their classrooms. These reflections support using a participatory curriculum design approach for empowering teachers and enhancing self-efficacy in teaching STEM to young children. High teacher self-efficacy has been associated with positive classroom outcomes and teacher retention in the profession.


INTRODUCTION
This paper describes a collaboration between Science, Technology, Engineering, and Mathematics (STEM) educators, engineering experts, social and cognitive developmental psychologists, and a public preschool program in the northeastern United States to develop a problem-based STEM curriculum for preschool classrooms using a participatory design approach (Auerbach, 1992).The Seeds of STEM eight-unit curriculum guides preschool teachers and their students in the process of problem-solving by following the steps involved in the engineering design process (EDP): 1) identifying problems and defining criteria for successful solutions, 2) brainstorming solutions, 3) selecting testable solutions, 4) creating the solutions, 5) testing and improving the solutions, and 6) sharing the solutions with others.During each 3-week unit of the curriculum, the first half focuses on exploration of specific science concepts (e.g., habitats, buoyancy, forces), while the second half focuses on solving a problem that requires application of the science concepts (e.g., design a container to float cookies across a river).Preschoolers learn to solve environmental problems sequentially with guidance from the teacher.In this paper, we

Background
Stakeholders and experts in education and industry concur that it is essential to support the development of students' STEM literacy, or the ability to apply STEM concepts and skills (Bornfreund, 2011;Zollman, 2012) to address real-world problems.STEM literacy is considered crucial to students' future success (Fayer et al., 2017).However, a large portion of teachers and adolescent children in the United States are underprepared and become uninterested in STEM over time (Archer et al., 2010;NRC, 2011NRC, , 2014)).National investment in developing a pipeline of individuals equipped with STEM competencies has been a priority for building knowledgeable, skilled, and innovative members of the workforce (Carnegie Corporation of New York, 2009).
Research on academic achievement shows a large disparity between low income and ethnic minority students in the United States (U.S.) and their White, middle-class counterparts.Children are first exposed to academic topics and skill-building activities during their pre-kindergarten years (Claessens et al., 2009;Federal Interagency Forum on Child and Family Statistics, 2013;Morgan et al., 2016).Since studies show that poor and ethnic minority children in the U.S. have already fallen behind before they enter public schools (Brooks-Gunn et al., 2007;Mickelson et al., 2013), a steady increase in early childhood education programs and enrollment has shown promising evidence in narrowing the performance gap (Haskins and Rouse, 2005;Howes et al., 2008).
Young children are naturally curious about their world (Engel, 2011;Piaget and Inhelder, 2000).Children actively participate in their environment, reason, define problems, manipulate, build and test prototypes, apply mathematical and scientific concepts, and share solutions with friends and family (Buchter et al., 2017).This results in the natural development of scientific inquiry and observation, measurement, prediction, inference, and communication skills (Platz, 2004) that increase with sophistication as children develop (Piaget and Inhelder, 2000).
The National Research Council (2012) defines engineering as an engagement in a systematic practice of design to achieve solutions to particular human problems (p.11).Young children are natural engineers in the sense that they manipulate their worlds in an attempt to solve problems (Brophy et al., 2008;Cunningham, 2009).Yet, engineering is rarely included in the development of early childhood curricula (Bagiati and Evangelou, 2015).Debates about how early to introduce STEM concepts and the extent to which young children can understand the subject matter and develop the skills are commonplace (Tippett and Milford, 2016;Wood and Hedges, 2016).As a result, early childhood programs vary widely in their resources, quality, effectiveness, and time spent on instruction in the disciplines related to STEM, especially engineering (Cannon et al., 2017;Whitebook and Ryan, 2011).
Despite evidence showing that a rigorous, integrated STEM curriculum promotes cognitive development and curiosity (Fogarty, 2009;Klein and Knitzer, 2006;Zan and Geiken, 2010), very little STEM or engineering instruction occurs in classrooms for three to five year old children (Diamond, et al., 2013), and few studies actively include teachers in the curriculum design process (McFadden, 2015).

Characteristics of a High-Quality Early Education Curriculum
High quality early childhood programs are associated with long term positive outcomes (Reynolds et al., 2004).Such programs include rich materials that incite exploration (Moomaw and Davis, 2010), are developmentally appropriate (National Research Council, 2012), and culturally responsive (Gay, 2000).They encourage frequent and responsive interactions between child and teacher (Vygotsky, 1962) and increase child engagement (Howes et al., 2008).An effective way to foster young children's learning is by meeting their cognitive, emotional, and developmental needs through both direct instruction and play (Moomaw and Davis, 2010).Allowing children to take the lead in exploring phenomena and asking open-ended questions that provide opportunities for children to reflect, form theories, and explore also makes for higher quality programs (Aldemir and Kernami, 2017).
The National Association for the Education of Young Children (NAEYC, 2009) identified key criteria for a set of materials to be considered an effective curriculum: (1) Children must be active and engaged; (2) Goals must be clear and shared by all; (3) Curriculum must be evidence-based; (4) Content must be learned through investigation, exploration, and direct teaching; (5) The curriculum must build on prior learning and experience; (6) Professional standards validate the curriculum's content; and (7) The curriculum, if implemented as intended, must have beneficial effects.

Approaches to Curriculum Development
This paper describes the Seeds of STEM curriculum development process, based on expert defined learning outcomes and models (NRC, 2012).Clements' (2007) seminal work toward creating a curriculum research 3 / 12 framework defined the elements needed to develop a 'research-based' curriculum.Recognizing the iterative nature of curriculum development, Clements argued for multiple testing before, during, and after the curriculum is developed.Accordingly, the Seeds of STEM curriculum was developed based on research-based outcomes, similar to Wiggins and McTighe's (2005) 'backwards design' approach, designed according to established learning models, tested in authentic settings, revised based on feedback from students and teachers, and its activities were evaluated multiple times to verify that the learning outcomes were met.
An iterative design approach helps to ensure that curriculum materials are effective and engaging (Couso, 2016;Diamond and Powell, 2011).Kinzie et al. (2009) developed a math curriculum for pre-Kindergarten which began with creative brainstorming, followed by development of early prototypes that were repeatedly revised, evaluated by teachers, and tried out in classrooms before they were deemed complete.For example, the developers conducted classroom observations in three pilot classrooms to evaluate the connections among activities throughout the year.They found that teachers tended to focus on completing activities rather than on encouraging students to think about them.As a result, they added language for teachers to use to prompt children to think and share their thoughts.
Similarly, a participatory curriculum design approach is one which cyclically involves stakeholders and end users in the design and decision making about the curriculum.Taylor (2003) grounds this approach on developing a partnership between different stakeholders in the curriculum and, in doing so, creating a sense of ownership of the product.By increasing stakeholder participation in the process, positive curricular impacts on learning and sustainability of implementation are thought to increase.This approach has been used in international and community education contexts to create curricular interventions to be maintained on a local level (Peters and Matarasso, 2005;Auerbach, 1992).Likewise, the participatory approach was used in the current project to help maintain preschool teachers motivation to remain involved and empower them to serve as co-developers.
Additional STEM curricula that have included teachers and their classrooms in the development process have shown to have promising outcomes.For example, Building Blocks Foundations for Mathematical Thinking is a mathematics computer-based curriculum for PK-2.It draws on the National Council of Teachers of Mathematics standards and National Association for the Education of Young children recommendations (Clements and Sarama, 2013b).Tools of the Mind is a research based, early childhood model that combines teacher professional development with a play-based curriculum that builds on quantitative and verbal skills for pre-kindergarten to kindergarten (Diamond et al., 2007).It aims to develop children's cognitive, social-emotional, and academic skills using Vygotskian theories and neuroscience research.Engineering is Elementary (EiE), a curriculum developed by the Museum of Science in Boston, was expanded to include preschool children.The preschool version of the curriculum uses a three-step EDP: explore, create, and improve, adapted from the original five-step process of ask, imagine, plan, create, and improve.To introduce children to these terms, EiE teaches children new vocabulary and uses playful materials, songs, and activities to learn engineering.Using qualitative assessments, they observe how children respond to activities and ask teachers to provide feedback (Lachapelle et al., 2011).These examples show us that young children may not be able to spell engineering or read on their own at this age, but they can follow step processes, use technology, and work collaboratively to develop skills needed to think like an engineer (Davis et al., 2017).

THE CURRENT PROJECT Project Goals
The research team set to develop an early childhood problem-based STEM curriculum for preschool children and teachers from diverse neighborhoods with low-income households.For these two populations, the following outcomes were established: First, by experiencing the curriculum children will improve their ability to appropriately use STEM vocabulary, to conduct each step of the engineering design process, as well as experience increase in overall STEM readiness.Second, teachers who teach the curriculum will improve their self-efficacy in teaching STEM to pre-K children and increase their knowledge of STEM and the engineering design process.To meet these goals, the study included the following phases: establishing a guiding framework for the curriculum, developing and testing the curriculum using an iterative participatory design, and pilot-testing the developed curriculum with a new population of children and teachers.The current paper focuses on the first two phases of the project.

Guiding Framework
To guide curriculum development, our team adapted the Dayton Regional STEM Center's Quality STEM framework (Pinnell et al., 2013) and defined eight principles for high-quality early childhood STEM experiences: developmental appropriateness, cultural responsiveness, applications of the EDP, integrity of academic content, quality of technology integration, connections to non-STEM disciplines, real world connections, and curriculumembedded, performance-based assessment (Dubosarsky et al., 2018).
In regard to developmental appropriateness, we relied on the National Head Start Child Development and Early Learning Framework (2010), which specifies standards under the domains of social-emotional development, approaches to learning, logic and reasoning, mathematics knowledge and skills, and science knowledge and skills.
The integrity of academic content criterion in the guiding framework ensured that STEM became an integral part of the learning day, rather than an add-on to an existing curriculum.For that the team reviewed several sets of State and National US academic standards to define the learning outcomes of the curriculum.The reviewed standards included the Massachusetts Framework for Science, Technology, and Engineering for Pre-K (2014), the Common Core State Standards for kindergarten mathematics (2010), and the United States' Next Generation Science Standards (NGSS) for scientific and engineering literacy (NGSS Lead States, 2013).
The practices emphasized by the NGSS were adapted for preschool children and embedded into the curriculum's performance expectations.These practices included asking questions and defining problems, developing and using models, planning and carrying out investigations, analyzing and interpreting data, using mathematics and computational thinking, constructing explanations, designing solutions, using evidence-based argumentation, and communicating information.

Establishing the Process of Iterative Development
The eight-unit Seeds of STEM curriculum was developed over an 18-month period through an iterative process of design-testing-revision-testing and followed the participatory curriculum design (PCD) approach (Taylor, 2003), where the end users of the curriculum (preschool teachers) were involved, and played a key role, in the development and testing of the curriculum.Since participatory curriculum design (PCD) method includes conducting stakeholder needs assessments, teachers completed an online survey at the start of the process gathering information about their background, perceived areas of needed professional development, their attitudes and knowledge about STEM education and engineering, as well as their perspective on multicultural awareness.Consistent with the PCD approach, the project team's role was to present design problems to the teachers, asking them to work together and discuss with one other to develop activities during face-to-face meetings.Teachers were provided with professional development to prepare for the process and continuously involved in the testing and evaluation of the different versions of the curriculum.Unlike some published versions of the PCD approach, the content to be taught was pre-determined by educational standards and frameworks in combination with expressed teacher needs, rather than by the community needs alone.

Teacher Recruitment
The team defined separate groups of teachers to participate in the curriculum creation and revision process: Developer teachers (DTG) and tester teachers (TTG2 and TTG3).To recruit teachers for all groups, the team partnered with a large urban Head Start program which managed 4 centers.A total of 621 children and their families were served by the Head Start program during testing cycles.Fifty-eight percent of families served were Hispanic; 21% were African-American; 20% were White/Caucasian, and 1% were Asian (Worcester Child Development Head Start Program Worcester Public Schools, 2017, p. 12).
All interested teachers completed a questionnaire during the recruitment session to measure their current STEM content knowledge, teaching practice, self-efficacy, and multicultural awareness.Developer teachers, referred to as the Developer Teacher Group (DTG) were charged with collaborating with the project's curriculum development team (CDT) to create the initial drafts of curriculum plans.To be considered for the DTG, the teacher had to meet these criteria: 1) having a Master's degree; 2) working in the urban Head Start center for more than one year; 3) mid to high scores on all Classroom Assessment Scoring System (CLASS) dimensions; 4) good reviews from the teacher's supervisor; and 5) teachers' interest in joining the development team.Recommendations for teachers who fit the criteria were solicited from the collaborating Head Start Center education director.Teachers for the Tester Teacher Groups (TTG2 and TTG3), were recruited from the same Head Start sites.Tester teachers had to be willing to implement activities and provide feedback on implementation days using an online feedback form.All teachers received stipends for time spent outside of the classroom to complete all development activities assigned to their group.In all, 6 preschool teachers from 6 classrooms formed the Developer Teacher Group (DTG), 20 teachers from 10 classrooms formed the second tester teacher group (TTG2), and 8 preschool teachers from 4 classrooms formed the third tester teacher group (TTG 3).On average, there were about 17 children in each classroom.All teachers were female, with the average age being 41.7 years old (standard deviation = 10.3 years old).Most (32 teachers; 91%) were of White/Caucasian descent, and three (3 teachers or 9%) belonged to minority groups.Five teachers held Master's degrees; two had Associate's degrees; 28 had bachelor's degrees.The following sections describe the elements in more detail that led to the development of the curriculum.

Iterative Testing and Revision Process
The team of researchers and developer teachers went through several models of creating the first draft of each unit (Version 1), and through trial and error, found a collaboration model that yielded high-quality units.Prior to a face-to-face meeting, the project team shared with the DTG the expected learning outcomes for the unit to be developed, and the team worked collaboratively at the meeting to create the first unit draft based on the defined outcomes.At each meeting, lasting about four hours, the team researched books, videos, songs, and ideas for center activities, as well as small and large group activities.Then, the teachers implemented each one of the curriculum units in their classrooms on three separate occasions: by the DTG (Version 1), TTG2 (Version 2), and TTG3 (Version 3).Between each test, the unit underwent revisions based on comments provided by the teachers.

Feedback Survey
To obtain teacher feedback on unit activities and keep track of teacher implementation, the curriculum development team developed an online survey using Qualtrics.Teachers were asked to complete a survey following each day they implemented any curriculum activities in their classroom.Survey feedback informed curriculum revisions to be made and gauged the usability and feasibility of activity implementation.The survey consisted of closed-and open-ended items that elicited teacher feedback about the extent to which teachers believed lessons were successful overall and about modifications that teachers made to each activity outlined in the curriculum.These included changes in pedagogical strategies or in materials used, and providing a rationale for curriculum modifications, and general feasibility of use with classroom sub-populations (e.g., Dual Language Learners, children with special needs, 3-year-old children).
The feedback responses from each group of testing teachers were collected and summarized to create one final consolidated report for each activity of the unit.Additionally, a separate report was created to summarize teachers' feedback on the entire unit.The curriculum development team revised each unit following the feedback from each testing session.

Expert Review
Once a unit was developed and tested by the first group of teachers, it was also given to the project's experts that included engineers, early childhood teacher educators, early childhood researchers, a cognitive developmental psychologist, and a social psychologist with expertise in diversity and inclusion.Experts' feedback was incorporated into the second revision of each unit.For instance, during the development of a unit based on plants, the curriculum development team initially considered including the full life cycle of the plants (seed-plant-flower-fruitseed).However, the advisory board suggested that since the full life cycle will not be observed during the timespan of the unit, the concept of a life cycle may be too abstract for some children.They recommended that the unit would only focus on what the children are readily able to see and touch.As a result, the unit was changed to focus on the plant parts (seeds, roots, stem, and leaves) and needs (sunlight, air, and water).

Professional Development
During the iterative curriculum development process, the project leaders conducted three professional development (PD) trainings with the participating developer and tester teachers to help them become familiar with the characteristics of high-quality STEM activities.These training sessions covered the following topics: 1) Engineering Design Process, 2) High Quality STEM education, and 3) Diversity and inclusion in STEM, specifically engineering.The topics of the training sessions were chosen based on teacher-reported needs assessed through a survey at the beginning of the study, and input from the curriculum development team as to the topics that will help teachers teach the curriculum with fidelity.During the first training, the teachers learned about the engineering design process (EDP) and created a childfriendly visual of the "EDP wheel" to use in their classrooms (Dubosarsky et al., 2018).During the second training, teachers were introduced to the project-created framework for high-quality STEM experiences in early childhood and engaged in such activity.The third training covered issues related to diversity and inclusion in STEM education.The goal was to increase teachers' awareness and knowledge in pedagogy and reduce stereotyping based on race, nationality, gender, ability, socioeconomic status, language, and perceptions of engineering and engineers.

ITERATIVE DEVELOPMENT PROCESS IN DETAIL Version 1: Developer Teacher Test
The six curriculum developer teachers were the first set of teachers to test the units in their classrooms.On average, there were about 17 children in each classroom.The teachers were provided with the unit plan (which they all participated in crafting), implementation instructions, materials (such as books and laminated images), and resource lists.The teachers were given full flexibility to add and/or modify activities and materials if deemed needed.
The most critical changes to each unit were made following Version 1 testing.For instance, initially the units were designed to be one-week long experiences that focus mostly on problem solving activities.However, during implementation of Unit 2, in which a character asked the children for help in getting a ring out of a block of ice, one of the developer teachers realized the need for background science experiences on properties of ice and water, and the processes of melting and freezing, prior to implementing the problem-solving activities in the classroom.The teacher then planned and taught some science activities, and in her feedback form she included additional preunit activities that covered the science concepts required for that unit.The teacher commented: I spanned this curriculum over two weeks so we could focus on the properties of water and how it can change specifically from water to ice and back again.The children were allowed to play with colored ice cubes and watch as they melted.We had daily conversations about what would make it melt faster and why.We also discussed how they would work as the children were drawing out their solutions.I think it worked out well that the children were engaged in this conversation throughout the unit so that when they came up with a solution that was based on prior knowledge and they were able to come up with great solutions.
As a result of this successful experience, the curriculum development team revised all units to become two part experiences, with the first half of each unit covering the science concepts related to the second half's problemsolving activities.Another suggestion was made to add images to the provided science vocabulary signs (such as water and the ice) for a more engaging and inclusive learning experience.
Initial units in this round called for using a character puppet named "Mr.Problemo" (modeled after a popular character on a U.S. children's television show) to introduce the problems to the children.However, during implementation of the unit the teachers suggested using a character that the children will be more excited about.Many of them had replaced Mr. Problemo with other story-book and television characters familiar to the children.One of the teachers commented: Instead of using a puppet and naming it Mr. Problemo we used a stuffed Pete the Cat since the children were already familiar with him and we had done a week of Pete the Cat books a few weeks prior.They seemed to respond very well and loved that he was back in our classroom.
In the later revision, the curriculum development team introduced a character called Problem Panda, who was introduced to the children in a laminated picture of real panda bear.The character presents a problem and asks for the help of the children in solving it in each unit.

Version 2: Tester Teacher Group 2
Twenty teachers (in 10 classrooms) participated in the testing of the revised units.As in Round 1 with the developer teachers (DTG), these tester teachers (TTG2) were also given full flexibility to add and/or modify the unit activities and materials.
The units were further revised based on the feedback obtained from the tester teachers.During this round, the teachers were asked to use the laminated Problem Panda as the main character in their activities.However, the teachers felt that using a more realistic looking object such as a stuffed animal or figurine would be more engaging for the students: Children were not interested [in the] laminated Problem Panda.We were able to get a reaction when another teacher pretended to be Problem Panda with another voice.The children enjoyed when Problem Panda introduced his four friends.
In the later revisions, the curriculum development team had introduced a stuffed toy to represent Problem Panda.Only moderate changes were made following the second testing, most of them focusing on adding books and other resources suggested by the teachers.

Version 3: Tester Teacher Group 3
The third group of 8 teachers (in 4 classrooms; TTG 3) was asked to test all 8 units in sequence.A new version of the feedback survey was created to be used by these teachers, asking teachers about the clarity of instructions and observed student outcomes.The teachers were instructed to follow the unit plans exactly as written and provide feedback.The reasoning for this request was that the team wanted to test the usability and feasibility of the curriculum as it was, as well as the clarity of instructions.New questions were added to the survey to obtain feedback on: 1) how well the unit and its activities met the learning objectives mentioned in the unit plan and, 2) ways in which the team can help teachers improve the teaching of this unit (e.g., teacher training, example videos).
The feedback from version 3 tester teachers was shared with the developer teachers and the advisory board to decide on further revisions to the curriculum.For instance, in a unit about the five senses, an activity calls for reading the book to introduce the concept of blindness to children.One teacher commented: My children listened to the story well and focused, but I do not think they understood the picture and blind connection.We decided to play a trust fall game where the children had to use their other senses while their eyes where closed to attempt to feel what it is like for a blind person.
In the revision, the curriculum development team included instructions to invite the children to try on blindfolds and conduct different tasks to experience what it feels like to lose their sense of sight for better understanding of the concepts.

Version 4: Research Team, Developer Teachers, and Advisory Board
One final curricular revision was done following the testing of the entire curriculum and based on the feedback regarding the clarity of instructions.Based on the Round 3 feedback, the developer teachers and the advisory board suggested changes including: adding icons and images to the unit plans to make them more engaging and accessible; determining a consistent structure for each engineering half of the unit; adding a script for teachers that would include specific open-ended questions; adding a unit overview; and adding a list of criteria to evaluate children's solutions.

IMPACT OF CURRICULUM DEVELOPMENT ON PARTICIPATING TEACHERS
At the conclusion of the development process, interviews were conducted with a sample of 13 Head Start preschool teachers who participated.All participating teachers were invited to be interviewed.Interviews were conducted with only those teachers who volunteered to participate.These interviews served in part as the project's formative evaluation from the perspective of the teachers, and were semi-structured in nature to better understand how teachers experienced the process and to allow issues and topics to arise that the team may not have considered previously.The interviews asked teachers to reflect on: 1) the changes to their teaching as a result of their participation; 2) what they perceived to be the essential or potential student outcomes of the curriculum; 3) any gaps they saw in the curriculum; and 4) any advice they had for other teachers who use the curriculum.
The interviews were conducted by the project evaluator.Four of the 9 interviews consisted of 2 teachers from a classroom, while the other five interviews had individual teachers participate.Two of the interviews were with developer teachers, while the other seven interviews were with tester teachers.All interviews took place at the teachers' respective schools and were video-recorded, with each interview lasting 15-20 minutes.Teachers received a copy of the interview questions before appearing on camera.
The interview data were qualitatively analyzed using an emergent thematic coding approach (Braun and Clarke, 2006).This approach allows for themes and patterns to arise based on the narratives shared by teachers in response to the interview questions.Each recorded interview was transcribed in full and de-identified.Transcripts were then reviewed to isolate teacher responses to each of the four main interview questions (changes in teaching; essential student outcomes; gaps in the curriculum; and advice to other teachers), which were then coded into categories based on the responses, until all types of responses were coded.The next section describes in more detail some of the major themes that arose from teachers' reflections.
In terms of how the curriculum development experience impacted their skills, knowledge, and teaching about STEM and engineering, teacher responses were coded into three main themes: increased knowledge of general teaching practices (5 instances), increased knowledge about the problem solving process (5 instances), and gains in general science or engineering knowledge (4 instances).Many teachers commented that their main takeaway was learning to teach problem-solving using the engineering design process (EDP).For instance, one teacher stated: "I've never been taught the engineering design process… I have a lot of experience with science, with my Master's courses… but I've actually never taught the engineering design process or had my children retest and test things and improve…" Another commented: "...[Y]ou start to think of some of the topics you… [taught] before… [and] go further into it.How can we bring the engineering process into it?How can we get the children more involved?" Other teachers stated the process helped serve as a framework to discuss the science concepts (which some already were familiar with), to think about science in an action-oriented way, rather than about 'something that sits on a table'.For example: "…[I]t gave us a procedure to follow to teach the curriculum instead of just putting everything out and say[ing], 'This week we are studying plants and seeds.'It actually gave us steps to follow and teach them." Teachers also commented that they learned to incorporate more hands-on activities, which engaged the children: "It wasn't just pictures or the books, but it was also having that kind of hands-on activities and the children really enjoyed that."One also stated she learned the power of open-ended questioning to encourage further student thinking: "It has given me a great insight on how to ask questions to children, how to keep them thinking; it has really helped [me] grow over this last year using it." More generally, teachers also stated that, in the process of preparing to teach the topics to the children, they increased their own knowledge and vocabulary about the science topics covered as well as about the steps in the engineering problem-solving process.
"[I learned to use]… a lot of the vocabulary, you know, just relating back to the [problem-solving] process.""I definitely learned some things too.I definitely learned things about shadows and different topics of study." Similarly, analysis of researcher notes from informal discussions with the developer and tester teachers during routine classroom visits found changes in teachers' expectations related to children's abilities.Several teachers reported being 'astonished' at the creative ideas children proposed to the problem.Teachers reported being skeptical at first, however adjusted their expectations once they saw children's ideas and solutions.
In terms of essential student outcomes, teacher responses were categorized into the following six themes: student knowledge of the problem solving process (7 instances), student engagement (4 instances), student use of vocabulary (4 instances), student attitude changes (3 instances), student career awareness (2 instances), and student collaboration (2 instances).In terms of gaps in the curriculum, teacher responses were categorized into four main issues: accessibility of materials to three-year-olds or dual language children (6 instances), lack of materials (3 instances), sequence of curriculum units (1 instance), and finding time to do the activities (1 instance).Finally, in terms of advice to other teachers, teacher responses were coded into two main categories: remember about the potential to reuse and adapt the curriculum after participation in the project (5 instances) and suggestions for teacher disposition/attitudes when starting to teach the curriculum (e.g., "have fun with it", "be patient", "try to hold your opinion"; 5 instances).

DISCUSSION
This paper detailed the development process of an early childhood problem-based STEM curriculum.The curriculum was developed to address the lack of high quality STEM curriculum for preschool classrooms.The partnership created between the curriculum development team and preschool teachers from the Worcester Head Start program reflects one of the tenets of using a participatory curriculum design approach.The diversity of expertise, a shared goal, and mutual respect that each member had for others on the team enabled the creation of innovative units.The iterative process of testing-feedback-revision-testing resulted in a high-quality curriculum that has been recently piloted in additional preschool classrooms to assess the potential impacts of the curriculum on teacher and student outcomes (Sibuma et al., 2018).
Our results suggest that the teachers who participated in the iterative development process increased their perceived self-efficacy in STEM pedagogy at the end of their involvement.Specifically, interview data suggested that the experience helped teachers expand their knowledge about science concepts, understand the engineering design process, and apply this process in guiding children to solve problems during the school day.According to Bandura's social cognitive theory (1997), self-efficacy is the belief about one's own capability to perform a task.Research has shown that high teacher self-efficacy has been associated with more willingness to try new techniques, student motivation, and greater enthusiasm and commitment for the teaching profession (Henson, 2001;Rodríguez et al., 2014;Tschannen-Moran and Hoy, 2002).Teachers also reported a change in their teaching practice, such as how to ask children questions that promote critical thinking.A contributing factor to this change was the teachers' observation of their students' growth in their classroom.Teachers who reported being skeptical about teaching STEM and problem solving to young children realized that the children enjoyed solving problems, learned the vocabulary, and followed the process to solve problems in the class.Teacher attitudes towards STEM have been positively correlated with their STEM classroom practices (Thibaut, et al., 2018).Specifically, research has shown that participation in teacher professional development was positively associated with teacher efficacy (Ross and Bruce, 2007) and attitudes towards STEM teaching principles (Thibaut, et al., 2018).

Limitations
As the interviews asked teachers to share their own perceived changes to their STEM knowledge and instructional behavior, observational analyses of classroom activity videos are being considered to verify changes to practice as expressed by the teachers.As is, we acknowledge that the data reported here are not empirical, but are based on individual reflections on the experience.In addition, although the team observed and heard similar feedback during the course of development, the interview data relied solely on teachers who self-selected to be interviewed and was not necessarily representative of the full range of experiences.
The positive outcomes of the curriculum development process, which includes the professional development of the teachers, went beyond the walls of the classroom.Several developer teachers have joined the curriculum development team in presenting the project at professional conferences and conducted professional development to a new group of teachers who implemented the curriculum.Teachers also expressed interest in sharing the importance of STEM with other preschool teachers.
Findings from the current project suggest that STEM curricula that are developed using an iterative participatory design approach may positively impact preschool teachers' self-efficacy and knowledge in teaching STEM and engineering topics.More research is needed in order to determine what and how specific parts of the collaboration in the participatory curriculum design process contributes to increased teacher self-efficacy and knowledge.

Table 1 .
Final iterative participatory development process for each unit in the curriculum.Key: CE = Content Experts; CDT = Curriculum Development Team; DTG = Developer Teacher Group; CDE = Child Development Expert; DE = Diversity Expert; AB = Advisory Board; TTG2 = Tester Teacher Group 2; TTG3 = Tester Teacher Group 3