European Journal of STEM Education
Research Article
2024, 9(1), Article No: 14

Development and Validation of the TSTSIS Instrument to Measure Teachers’ Self-Efficacy to Teach Science Through Integrated STEM Approach

Published in Volume 9 Issue 1: 16 Oct 2024
Download: 156
View: 527

Abstract

This study aimed to develop an instrument with acceptable validity and reliability for assessing two key aspects among science teachers: their self-efficacy in teaching science through an integrated STEM approach (STSIS) and their outcome expectancy when employing this approach (OETSIS), which we refer to as TSTSIS. We administered this tool to 109 science teachers in Phnom Penh city and various provinces in Cambodia. The results revealed that STSIS can be broken down into six essential latent factors. Among these factors, five are grouped under the theme of “Integrated STEM Instructional Self-Efficacy,” covering problem-based, robots-based, inquiry-based, engineering-based, and technology-based instructional self-efficacy. The sixth factor relates to teachers’ self-efficacy in terms of accessing materials. Author formulated the subscale “Expectations of High Students’ Achievement in Science” as the factor representing the OETSIS. Scores on the TSTSIS subscales offered evidence of its criterion validity, with significant differences observed across various teacher-related variables. These variables include teachers’ teaching qualifications, school types, teachers from schools where STEM education and an integrated STEM approach are part of the school’s objectives, and correlation with instructors’ teaching experience. The TSTSIS subscale lays the groundwork for designing STEM training programs tailored to the needs of science instructors.

INTRODUCTION

The Royal Government of Cambodia (RGC), with a particular focus on the Ministry of Education, Youth, and Sport (MoEYS), has made STEM education a top priority within Cambodia and has embarked on significant initiatives to promote it. For instance, in 2016, MoEYS(a) introduced the STEM education policy, along with a new curriculum framework that integrated information and communication technology (ICT). Additionally, in 2022, MoEYS(b) sanctioned the implementation of the STEM manual for upper secondary teachers, with the aim of providing STEM instructors with a structured framework for in-service training. In 2020, MoEYS(c) embraced and implemented the STEM approach within science classes for primary and lower secondary education. Through these comprehensive efforts, it becomes evident that the Royal Government of Cambodia (RGC) places primary emphasis on advancing STEM education.

STEM education equips students with 21st-century skills, knowledge, and competencies. It enhances their social and communication abilities, nurtures scientific thinking, fosters self-discipline, and cultivates innovation and creativity (National Research Council, 2010; Wahono et al., 2021). The integrated STEM approach amalgamates knowledge, skills, values, and information from STEM fields to tackle real-world issues (Cunningham et al., 2020; Ng and Adnan, 2018; Committee on STEM Education of the National Science & Technology Council, 2018; Wendell et al., 2017). It empowers students with crucial skills such as critical thinking, problem-solving, collaboration, teamwork, and especially 21st-century skills (Morrison et al., 2015; Mustafa et al., 2016; Polydoros, 2021; Selcen Guzey et al., 2017; Toma and Greca, 2018). The objectives of integrated STEM education align with those of science education, as reflected in the Next Generation Science Standards (NGSS) (Achieve, Inc., 2023). However, this innovative STEM approach places significant cognitive and emotional demands on teachers. Many educators are unfamiliar with this teaching style and may lack the expertise required to meet the interdisciplinary demands of STEM integration.

Self-efficacy theory bridges the gap between motivational and constructivist thinking and pertains to an individual’s perceived ability to handle a given situation or task (Bandura, 1977, 1982, 2006). Numerous studies have demonstrated that teacher self-efficacy significantly correlates with their commitment to tackling challenging tasks, acceptance of new and innovative classroom activities, persistence in their current job, and overall job satisfaction, all of which positively impact student achievement (Berman and McLaughlin, 1977; Farah, 2011; Granziera and Perera, 2019; Hoy, n.d; Mok and Moore, 2019; Tschannen-Moran and Hoy, 2001). Measuring teachers’ self-efficacy is crucial for comprehending and predicting their behaviours (Dellinger et al., 2008). Additionally, within the realm of STEM education, teachers’ self-efficacy serves as a potent indicator of a teacher’s confidence and ability to proficiently implement an integrated STEM approach (Geng et al., 2019; Gunning and Mensah, 2011; Jaipal-Jamani and Angeli, 2017; Zakariya, 2020). Recognizing the importance of teachers’ self-efficacy in STEM education, numerous studies have been conducted. For instance, Fenton and Essler-Petty (2019) and DeCoito and Myszkal (2018) explored teachers’ personal competence in teaching science, while Ramli et al. (2020) and Yang et al. (2021) investigated teachers’ self-efficacy in implementing STEM education. Additionally, Lee et al. (2019) measured teachers’ self-efficacy in STEM knowledge, and Johnson et al. (2021) and Menon et al. (2023) assessed teachers’ abilities in teaching science within the STEM framework. However, these studies tend to examine teachers’ self-efficacy in STEM education either by focusing on a single discipline, general STEM knowledge, or the STEM framework. None of these studies specifically address teachers’ self-efficacy in using an integrated STEM approach for implementing STEM education, particularly in teaching science through this integrated method in Cambodia educational context.

LITERATURE REVIEW

Integrated STEM Approach

The integrated STEM approach stands out as a prominent teaching and learning method within the STEM educational framework. Wei and Chen (2020) have reported that practical classrooms in integrated STEM education often employ constructivist and transformative approaches, including problem-based learning, robotic activities, projects, science exhibitions, and game competitions. Similarly, Apedoe et al. (2008) and Hmelo-Silver (2004) have indicated that integrated STEM education can be realized by adopting specific instructional approaches, such as project-based learning, problem-based learning, inquiry-based learning, or theme-based methods in the teaching and learning process. Moreover, Ayieko et al. (2017) have emphasized that the integrated STEM approach encompasses a transition from basic technology usage, such as computers, smart boards, smart pens, and calculators, toward more interactive and engaging technological applications in the learning process. In summary, the integrated STEM approach encompasses a wide array of teaching and learning methods, including project-based learning, problem-based learning, inquiry-based learning, robotics, technology-based learning, engineering-based learning (such as science exhibitions and game competitions), and collaborative teamwork. Teaching science through ISTEMA involves students adopting the role of engineers to apply math, technology, and engineering concepts to solve scientific problems (Anwar et al., 2022; Khalil and Osman, 2017; Guzey and Li, 2023; Yaki et al., 2019). Furthermore, employing an integrated STEM approach in science education has a significant influence on students’ achievement and performance. For instance, Cunningham et al. (2020) discovered that students who engaged in classes where science and engineering were integrated exhibited higher levels of performance compared to their peers in the control group. Similarly, Wendell et al. (2017) demonstrated that integrating engineering into science instruction enhances students’ academic achievement. Despite the recognized importance and effectiveness of the integrated STEM approach, to implement this teaching approach it required teachers who have content and pedagogical content knowledge in STEM disciplines, adequacy of learning and teaching materials, clear integrated STEM curricula and lessons that encompass all STEM disciplines and especially teachers’ high self-efficacy and well-structured support (Nadelson and Seifert, 2017; Toma and Greca, 2018; Tawbush et al., 2020)

As explained in the previous section, the operational definition of the integrated STEM approach or integrated STEM education was provided. However, another key term, “STEM Education,” is also used in this study. To avoid confusion, the following operational definition is synthesized from empirical evidence. Despite significant efforts by many countries to enhance STEM education, there is no universal consensus on its definition, which varies among researchers. For example, Sanders (2009) emphasized that STEM education involves teaching and learning the four disciplines either separately or integrated with other subjects. On the other hand, Moore et al. (2014) defined STEM education as the integration of the four disciplines into a single class, unit, or lesson. Furthermore, Kelley and Knowles (2016) described STEM education as an instructional approach that combines two or more STEM domains to promote student learning through inquiry, engineering design, mathematical reasoning, and technological literacy. Drawing from this empirical evidence, the current study defines STEM education as an educational system where science, technology, engineering, and mathematics are taught in an integrated manner (Akgunduz, 2016; Bybee, 2013). This approach equips students with 21st-century skills, knowledge, and competencies while fostering their social and communication skills, scientific thinking, self-discipline, and ability to innovate and create (National Research Council, 2010; Wahono et al., 2021).

Self-Efficacy Theory

Self-efficacy is one of the five components of Social Cognitive Theory, developed by Albert Bandura in 1986. The concept was initially introduced by Bandura in 1977, though at that time, he referred to it as ‘expectation of personal efficacy’ rather than self-efficacy. This idea was used to describe how individuals initiate coping behaviors, the level of effort they exert, and their persistence in overcoming challenges and adverse experiences (Bandura, 1977). By 1982, Bandura began using the term ‘self-efficacy,’ defining it as ‘judgments of how well one can execute courses of action required to deal with prospective situations’ (p. 122). Later, in 1997, and together with Maddux in 1995, Bandura identified self-efficacy as beliefs in one’s ability to take action and learn within specific contexts, tasks, and situations. They emphasized that self-efficacy is not an inherent trait. In 2001, Bandura further elaborated on self-efficacy as a judgment of one’s ability to achieve desired outcomes and avoid undesirable ones. Based on this theoretical foundation, self-efficacy refers to an individual’s confidence in their ability to successfully perform a behaviour. Bandura (1977) identified four main sources from which self-efficacy is developed: mastery experiences (performance outcomes), vicarious experiences (observing others), verbal and social persuasion, and emotional and physiological states.

Albert Bandura (1977) introduced the concept of self-efficacy as a comprehensive framework for understanding human behaviour. Building on this, researchers from the RAND Corporation, including Armor et al. (1976) and Berman and McLaughlin (1977), developed a theoretical framework for teachers’ self-efficacy, consisting of two key components: personal teaching self-efficacy and teaching outcome expectancy. In this framework, personal teaching self-efficacy refers to teachers’ belief in their ability to effectively implement specific instructional strategies; while teaching outcome expectancy relates to their belief in the positive impact these strategies will have on student learning.

In addition to the four primary sources of teachers’ self-efficacy, two key categories of variables—individual-related factors (gender, educational background, teaching experience, and teaching qualification) and school-related factors—were identified as influential on teachers’ self-efficacy. For example, Tschannen-Moran and Johnson (2011) found significant gender differences in self-efficacy for literacy instruction, with female teachers demonstrating higher levels of Teacher Self-Efficacy for Literacy Instruction (TSELI). Similarly, Wolters and Daugherty (2007) discovered that teachers’ self-efficacy was predicted by their teaching experience and academic qualifications. Chen et al. (2021) investigated the relationship between early childhood preservice teachers’ self-efficacy, pedagogical beliefs, and the need for professional development in STEM education. Their findings indicated that teachers with prior STEM experience, a strong interest in STEM, and active participation in STEM-related activities had higher levels of self-efficacy in teaching STEM. Lastly, Knoblauch and Chase (2015) found that urban student teachers had significantly lower self-efficacy compared to their suburban counterparts.

Teachers’ Self-Efficacy Measurement

Teachers’ self-efficacy is a vital factor influencing their motivation, commitment, and perseverance in implementing teaching strategies (Menon et al., 2023). Several studies have investigated teachers’ self-efficacy in the context of integrated STEM education. To measure teachers’ self-efficacy in implementing integrated STEM education, various tools have been adapted and modified for use. For example, Fenton and Essler-Petty (2019) adapted the Science Teaching Efficacy Belief Instrument (STEBI), originally developed by Riggs and Enochs (1990), to assess preservice elementary school teachers’ self-efficacy in STEM-integrated teaching. Another tool, the Ohio State Teachers Efficacy Scale (OSTES), initially created by Tschannen-Moran et al. (1998) and later modified by Geng et al. (2019), has been used to explore the self-efficacy and concerns of primary and secondary school teachers in Hong Kong regarding STEM education. DeCoito and Myszkal (2018) and Kelley and Knowles (2016) employed the Teachers’ Efficacy and Attitude Toward STEM Survey (T-STEM) which originally develop by Friday Institute for Educational Innovation (2012) to measure the confidence and self-efficacy of Canadian high school teachers in teaching STEM subjects. Moreover, Johnson et al. (2021) and Menon et al. (2023) adopted the Teachers’ Self-Efficacy to Teach Science in an Integrated STEM Framework Tool (SETIS) to assess the self-efficacy of preservice elementary teachers in teaching science and mathematics through an integrated STEM framework. Chen et al. (2021) adapted the Survey of Perceived STEM Knowledge (PSTEMK), which developed by (Lee et al., 2019) to investigate the self-efficacy beliefs of Taiwanese preschool teachers in teaching STEM. Another tool, the STEM Teacher Instruction Preparedness Instrument (STEMTIP), was developed by Ramli et al. (2020) to measure the pedagogical readiness of Malaysian high school science teachers in implementing STEM based on the STEM teaching method. Lastly, Yang et al. (2021) created the STEM Teaching Self-Efficacy Scale (STSS), comprising two sub-components—Pedagogical Self-efficacy and Content Self-efficacy—to gauge the self-efficacy of early childhood teachers in STEM education.

In summary, there are seven basic types of tools that researchers have previously applied and adapted to measure teachers’ self-efficacy beliefs related to the implementation of STEM education. These tools vary in their original concepts and philosophies. STEBI and the T-STEM Survey primarily assess teachers’ personal competence in teaching science, while STEMTIP and STSS focus on measuring teachers’ self-efficacy in implementing STEM education. PSTEMK is centred around self-efficacy in STEM knowledge, and OSTES primarily focuses on teachers’ self-efficacy within the general discipline. SETIS specifically targets teachers’ self-efficacy in teaching science within an integrated STEM framework while he defines integrated STEM framework based on two dimensions: curriculum-based integration and context-based integration. Curriculum-based integration connects disciplinary knowledge with personal and real-world experiences, while context-based integration considers the influence of the learning environment and social factors on students’ learning. On the other hand, as explained in detail in the literature review, the integrated STEM approach consists of five components: inquiry-based, technology-based, engineering-based, problem-based, and robotics-based instruction. However, the seven tools reviewed earlier were developed to measure only certain aspects of the integrated STEM approach, as detailed in Table 1. Additionally, the STEBI and OSTES instruments specifically focus on measuring science teaching self-efficacy and teachers’ self-efficacy in teaching general subjects, respectively. Furthermore, one instrument assessed teachers’ instructional preparedness for STEM teaching, while the other four tools focused on various aspects of STEM education.

Table 1. The primary focus of existing tools for measuring teachers’ self-efficacy in STEM education

Instrument

Integrated STEM approach teaching self-efficacy

Overall instructional

Teaching science

Outcome expectancy

Inquiry

Technology

Engineering

Problem

Robotic

Material

SETIS (Mobley, 2015)

 

 

 

 

 

 

STEBI (Riggs, 1990)

 

 

 

 

 

 

 

T-STEM Survey (Friday Institute for Educational Innovation, 2012)

 

 

 

 

 

 

OSTES (Tschannen-Moran et al., 1998)

 

 

 

 

 

 

 

 

PSTEMK (Lee et al., 2019)

 

 

 

 

 

 

STSS (Yang et al., 2021)

 

 

 

 

 

 

 

 

STEMTIP (Ramli et al., 2020)

 

 

 

 

 

 

Neither of these studies specifically investigates teachers’ self-efficacy in the context of teaching science through an integrated STEM approach. In this study, the author defines the integrated STEM approach as the actual instructional practice of teaching science using strategies that integrate inquiry-based, engineering-based, problem-based, teamwork-based, technology-based, and robotics-based methods. This differs from Mobley’s (2015) focus on a purely intended curriculum. Additionally, no previous research has specifically assessed the self-efficacy of teachers teaching science through an integrated STEM approach, particularly in the Cambodian context. To fill this gap, this exploratory study aims to develop and validate a scale for assessing science teachers’ self-efficacy in teaching science through an integrated STEM approach in Cambodia. This scale consolidates the common aspects from previous tools and incorporates additional components of the integrated STEM approach. Therefore, this study covers all components of integrated STEM teaching self-efficacy and outcome expectancy, while the general constructs of instructional and science teaching self-efficacy were excluded, as they are already inherent in the construct of integrated STEM teaching self-efficacy.

Purpose and Research Question

To address the literature gap and advance our understanding of the integrated STEM approach, this study aims to develop a valid and reliable instrument for measuring the latent factor that describes teachers’ self-efficacy in teaching science through an integrated STEM approach (ISTEMA) and their outcome expectancy when using ISTEMA. To achieve these objectives, this study is guided by two main research questions:

  • What is the underlying structure of an instrument with acceptable validity and reliability for measuring the latent factor that describes teaching self-efficacy and outcome expectancy in teaching science through ISTEMA?

  • Are there statistically significant differences in the mean values of teaching self-efficacy and outcome expectancy to teach science through ISTEMA among teachers’ demographic variables, including gender, education background, teaching experience, teaching qualification, teaching level, subject major, school type, and school location?

METHODOLOGY

Instruments

The original Teachers’ Self-Efficacy to Teach Science through an Integrated STEM Approach (TSTSIS) instrument was developed based on a thorough literature review related to science teachers’ self-efficacy and their expectations for teaching science through an integrated STEM approach. For example, the six constructs from the framework for teaching science through an integrated STEM approach including inquiry, problem-solving, engineering, teamwork, technology, and robotic – based instruction, developed by Sokha and Kinya (2023), were adapted. The concept of teachers’ self-efficacy was grounded in Albert Bandura’s theoretical framework on individual self-efficacy (1977, 1986, 1997, 2001) and particularly the concept of teachers’ self-efficacy developed by RAND Corporation were adapted (refer to the literature review section for details).

To develop items for measuring teaching efficacy in science education through an integrated STEM approach as the first aspect of teachers’ self-efficacy, the author incorporated the six components of the integrated STEM approach in science education—namely, inquiry, problem-solving, engineering, teamwork, technology, and robotics-based learning—adapted from Sokha and Kinya (2023) with the theoretical concept of teachers’ self-efficacy, for example item “I am confident in my ability to approach students to create key questions for each lesson which fosters students to stimulate their scientific knowledge and connects the content to engineering designed”. Additionally, two constructs, personal and material accessing self-efficacy, were drawn from Mobley (2015). These two constructs were added because they assess teaching materials and teachers’ personal beliefs regarding professional development for implementing an integrated STEM approach, areas not covered by the six constructs developed by Sokha and Kinya (2023). Example item, “I am confident in my ability to develop new knowledge and skills necessary to teach science from within an integrated STEM approach”. The six constructs representing the integrated STEM approach in science education consist of 32 items designed to assess instructional practices in teaching science through an integrated STEM approach. The personal construct comprises five items related to pedagogical knowledge and skills, as well as pedagogical content knowledge. The material accessing construct includes four items that evaluate a teacher’s ability to learn, use, access technology, and adapt to new teaching methods.

To construct items measuring teachers’ outcome expectancy when using the integrated STEM approach as the second aspect of teachers’ self-efficacy, the author adapted and modified the Science Teaching Outcome Expectancy Scale from Riggs and Enochs (1990). The author adapted these items to incorporate the context of the integrated STEM approach by altering specific terms. For example, item for an original item from this scale, “When a student does better than usual in science, it is often because the teacher exerted a little extra effort,” was adapted to “Students will do better than usual in science when I exert a little extra effort in using an integrated STEM approach.”

The TSTSIS initially consisted of 68 items, categorized into four subcategories: Demographic Information (14 items), Instructional (42 items measuring science teaching self-efficacy), Personal, Materials Accessing, and Outcome Expectation (12 items). Items assessing science teachers’ self-efficacy were rated on a 4-point Likert scale ranging from 1 (Cannot do at all) to 4 (Very confident that I can do this). Similarly, items evaluating science teachers’ outcome expectations were rated on a 4-point Likert scale ranging from 1 (Not confident at all) to 4 (Very confident). The Four-Point Likert Scale has been selected as the preferred forced-choice format for this study. The objective of this study is to explore the teachers’ beliefs which necessitate opinions that are either positive or negative rather than neutral. This inclination is evident in the declarative statement of their responses (Roberts et al., 1999). Furthermore, self-efficacy is to measure what respondents believe they can do at a given moment. The inclusion of neutral or “don’t know” responses hinders the assessment of their abilities at that specific moment. As asserted by Bandura (1977), self-efficacy identifies whether a person is confident or lacks confidence in their abilities in specific circumstances. Therefore, a neutral or “don’t know” response introduces uncertainty into the measurement of the extent to which respondents perceive their ability to handle a given task or action.

To ensure clarity in translation, content validity, practical relevance, and contextual appropriateness, the questionnaire was reviewed by seven experts. This group included three experts in science education and one expert in mathematics education who assessed the content and practical aspects of each statement particularly in science, two experts in English literature who ensured the clarity of the translation, and one expert in social education who checked the grammar of the Cambodian language.

Statistical Analysis

Data cleaning and descriptive analyses were performed in SPSS Version 26 (George and Mallery 2020). To identify the underlying factor, criterion validity, and internal reliability of the instrument, the author conducted exploratory factor analysis, compared mean and correlation analysis, and calculated Cronbach’s Alpha respectively as follows:

Exploratory factor analysis (EFA)

The TSTSIS instrument was initially developed based on a literature review, and an exploratory factor analysis (EFA) was utilized to uncover the underlying factor structure of the questionnaire. In the initial stages, the author performed an EFA on the 42-item TSTSIS, employing the Principal Axis Factoring (PAF) method to unveil the latent factor structure of the questionnaire. In PAF, estimated latent factors are derived from a mathematical model, as outlined by Dunteman in 1989. These underlying factors are assumed to be interrelated or correlated, which is why an oblique rotation was employed. Within this context, the regression coefficients for each variable on each factor (pattern matrix) were scrutinized to identify the factors. Furthermore, the study used the Kaiser–Meyer–Olkin (KMO) and Bartlett’s test of sphericity to assess sampling adequacy and multivariate normality, as described by Kaiser in 1970. KMO values in the range of 0.7 to 0.8 are considered good (Hutcheson and Sofroniou, 1999), and a significant value of less than 0.05 in Bartlett’s test of sphericity indicates that the correlation matrix is not an identity matrix, thereby confirming the suitability of factor analysis. Communality measures the proportion of variance explained by the extracted factors. MacCallum et al. in 1999 noted that samples ranging from 100 to 200 can be sufficient with communalities in the 0.5 range, especially when there are relatively few factors with only a few indicator variables each. Various methods were employed, including the Kaiser Criterion (eigenvalues and scree plot), parallel analysis, Pattern Matrix, and theoretical expectations, to determine the number of factors that should be retained. A factor loading of 0.30 was established as the minimum threshold for saliency, with loadings of 0.40 or higher being preferred, in accordance with the recommendations by Hair et al. in 2009. Cross-loading occurred when an item loaded significantly on one factor and had a loading greater than 0.30 on another factor. Items failing to meet these criteria were sequentially eliminated based on their low loading on the primary factor or cross-loading. The EFA was conducted using IBM Statistical Package for the Social Sciences (SPSS) version 26.0, as detailed by George and Mallery in 2020.

Compare sample mean and correlation analysis

To establish the criterion validity of the TSTSIS, the author conducted an analysis involving the comparison of sample means and correlation analyses to explore the relationship between TSTSIS scores and various demographic variables of teachers, such as gender, teaching experience, and educational background. Prior to conducting the comparison of sample means, a normality test was administered to the dependent variables as detailed by Pallant in 2020. The results of the Shapiro-Wilk test revealed that the significant value for all dependent variables was less than 0.001, indicating that the data did not conform to a normal distribution. Consequently, independent sample Mann-Whitney U Tests and independent sample Kruskal-Wallis Tests were conducted to compare sample means range and determine whether there was sufficient evidence to conclude that the medians of the dependent variables differed among various demographic variables. Spearman’s rho correlation was employed to examine the association between TSTSIS scores and variables related to teachers’ teaching experience. These analyses were conducted using SPSS version 26 (IBM Corp, 2017).

Reliability analysis

To assess the internal consistency of the TSTSIS, where all items are designed to measure the same construct, the author calculated Cronbach’s Alpha (α) for each of the retained constructs and the interpretation of Cronbach’s Alpha values is based on George and Mallery (2020)​ which revealed that \(\alpha < .5\) = unacceptable, \(\alpha > .5\) = poor, \(\alpha > .6\) = questionable, \(\alpha > .7\) = acceptable, \(\alpha > .8\) = good, and \(\alpha > .9\) = excellent.

Participants

A survey was conducted involving 109 science teachers in both Phnom Penh city and several provinces throughout Cambodia. The study focused on teachers currently teaching science across a range of educational levels, from primary to upper secondary and unfortunately, there was only one respondent from the primary level, so the results of this study are more applicable to secondary education than to primary education. Out of the total participants, 103 successfully completed the survey, comprising 60 males and 43 females. Six respondents were excluded from the analysis due to incomplete responses. Table 2 presents an overview of the demographic characteristics of the surveyed participants.

Table 2. Demographic characteristics of participants

Characteristics

Number

Percentage

Gender (N = 103)

Male

60

58.3%

Female

43

41.7%

Education background (N = 103)

Baccalaureate

18

17.5%

Associate

2

1.9%

Bachelor

66

64.1%

Master

17

16.5%

Teaching qualification (N = 103)

PTTC

3

2.9%

RTTC

33

32.0%

NIE

67

61.2%

Major at higher education

Mathematics

7

6.8%

Science (Bio, Chem, Phy, Earth)

80

77.7%

ICT

7

6.8%

Engineering

0

0.0%

Haven’t gone through high education

2

2.9%

Others

7

6.8%

Teaching experiences

0-5

28

27.2%

6-10

19

18.4%

11-20

39

37.9%

20 <

17

16.5%

Teaching grade (N = 103)

Primary 4-6

1

1.0%

Lower secondary 7-9

40

38.8%

Upper secondary 10-12

54

52.2%

Primary + lower secondary 4-9

1

1.0%

Lower + upper secondary 7-12

7

6.8%

School type (N = 103)

Normal

58

56.3%

SBM/GIEP project

20

19.4%

SRS

17

16.5%

SRS network

4

3.9%

New general school

4

3.9%

School location (N = 103)

Town/city

54

52.4%

Nearby town/outskirt

25

24.3%

Rural

24

23.3%

STEM education is one of the school’s vision (N = 103)

Yes

59

57.3%

No

44

42.7%

Integrated STEM approach is one of the school’s vision (N = 103)

Yes

54

52.4%

No

49

47.6%

RESULTS

Exploratory Factor Analysis

Self-efficacy in teaching science through an integrated STEM approach (STSIS)

The Kaiser-Meyer-Olkin coefficient for the original 42-item data set was 0.878, suggesting high sampling adequacy. Bartlett’s Test of Sphericity was statistically significant (\(\chi^{2} = 4769.029\), \(df = 861\), \(p < .001\)), indicating that the correlations between the items were sufficiently large for factor analysis with oblique rotation.

Communality: The extraction number for all 42 items was 41 items with Communality higher than 0.50, except item 4 < 0.50 therefore this item was deleted. On re-analysis, the extraction yielded 41 items with a communality of more than 0.50, which means that more than 50% of the variance of each variable can be explained by the retained factors. From this point of view, these variables were well represented in the common factor space.

Parallel Analysis: Parallel analysis indicated there were five sub-factors in total for the 41-item scale.

Pattern Matrix: The inspection of the pattern matrix of the original 41 items revealed that item 15 had a low factor loading < 0.40 and items 10, 11, 12, 22, 23, 24, and 34 were all cross-loaded and their secondary factor loadings were above 0.30. Therefore, these eight items were considered problematic in the first review and were removed one by one until they all met the criteria. Item 15 was deleted first because it had a low factor value. Item 33 was then deleted because it had the highest cross-loading value in another factor. Based on this rule, items continued to be removed if there were cross-loadings or low-factor loadings for both the primary and secondary factors. Therefore, items 22, 25, 14, 34, 23, 24, and 35 were removed sequentially based on their cross-loading or low-factor loading.

We have done the analysis again. The Kaiser-Meyer-Olkin coefficient for this data set was 0.876, and Bartlett’s Test of Sphericity was statistically significant (\(\chi^{2} = 3548.409\), \(df = 496\), \(p < .01\)). Six latent factors were obtained by oblique factor rotation using the direct oblimin method. The six latent factors had eigenvalues above the Kaiser’s criterion of 1 and in combination explained 79.582% of the variance. The scree plot showed inflections that would justify keeping six factors consistent with the Kaiser’s criterion of 1. The communalities were in general higher than 0.50.

Finally, we double-checked the theoretical meaning of each item. We found no repetitions in the meaning of the 32 items and therefore decided to keep these items in the final scale. As a result, six factors were identified, comprising 32 of the original items. The first factor, called Problem-Based Instruction Self-Efficacy, describes teachers’ self-efficacy to teach science using a problem-based learning approach. This factor accounted for 45.137% of the total variance and had an eigenvalue of 14.444. The second factor was named Robotics-Based Instruction Self-Efficacy and describes teachers’ self-efficacy to teach science using the robotics method. This factor accounted for 60.054% of the total variance and had an eigenvalue of 4.773. The third factor called teachers’ self-efficacy in relation to personnel and materials describes teachers’ self-efficacy in relation to their personal belief in adapting, acquiring, and developing new knowledge, skills, and teaching materials required for teaching science using the integrated STEM. This factor accounted for 67.379% of the total variance with an eigenvalue of 2.344. The fourth factor called Inquiry-Based Instruction Self-Efficacy describes teachers’ belief in teaching science with an inquiry-based approach. This factor accounted for 72.492% of the total variance with an eigenvalue of 1.636. The fifth factor, Engineering-Based Instruction Self-Efficacy, describes teachers’ confidence in their ability to teach science using an engineering-based approach. This factor accounted for 76.258% of the total variance and had an eigenvalue of 1.205. The last factor, self-efficacy of Technology-Based Instruction, describes teachers’ self-efficacy to teach science using a technology-based approach. This factor accounted for 79.587% of the total variance with an eigenvalue of 1.064. In line with teaching and learning theory and the nature of factors 1, 2, 4, 5, and 6, which illustrate actual teaching practice, these five factors have been combined into one component called Instructional Self-Efficacy (see Table 3).

Table 3. Factor loading and Cronbach’s alpha (α) for the items of the specific component of teaching self-efficacy

Item

Name of item

Factor Loading

 

Component I: Integrated STEM instructional self-efficacy: α = .945

 

Factor 1: Problem-based instruction self-efficacy: α = .924

 

18_PFL3

Demonstrate test, and redesign the solution by simulation, visualization, and modelling the results of their design solution

.953

 

20_PFL5

Develop the project as the predetermined products to illustrate the problem solving

.845

 

19_PFL4

Summarize or conclude what was done or highlight the outcome of the solution

.795

 

17_PFL2

Define a solution or solution construct in which students express a range of creative ideas and engaged in a problem analysis by using scientific knowledge and reasoning

.668

 

21_PFL6

Produce the final products as the outputs of the series activities in the lesson such as problem-scoping, identifying, and constructing solutions, testing, and redesigning solutions, and presenting final products

.591

 

16_PFL1

Create the problem setting by presenting a problem of each lesson

.482

 

Factor 2: Robotic instruction self-efficacy: α = .976

 

28_RBL3

Process of robotic design by asking questions about robotics. To answer questions students, engage in some inquiry activities, scientific, robotics, programming, and conducting research on different designs of robots

-.940

 

30_RBL5

Create the code to run robots such as programs to move forwards and backward, navigate, and block robots

-.927

 

29_RBL4

Discuss what a robot was and how to control robots, then let them model robots in the 3D design software and learned to program in a particular software

-.907

 

31_RBL6

Test and validate their design by troubleshooting, debugging, performing robot, and competing challenges

-.899

27_RBL2

Interact with the robot activities by organizing them to complete several activities worksheets and practice programming the robots

-.888

32_RBL7

Write a report about their robotic design

-.886

26_RBL1

Introduce the concept of robot design to students by letting them observe robot activities through the presentation or watching the tutorial program

-.885

Factor 4: Inquiry-based instruction self-efficacy: α = .897

7_IBL7

Approach and facilitate students to present results and conclusion

.789

5_IBL5

Conduct experiments or hands-on activities

.686

6_IBL6

Conduct data analysis by using mathematics concepts and tools such as simple calculations, data tables, graphs, bar charts, and box plots to analyse, measure, and display results

.668

1_IBL1

Introduce the fundamental concepts of each designed lesson

.666

3_IBL3

Conduct observation through videos or actual practices

.654

2_IBL2

Create key questions for each lesson which fosters students to stimulate their scientific knowledge and connects the content to engineering designed

.653

Factor 5: Engineering-based instruction self-efficacy: α = .906

9_EGN2

Develop the engineering design plan in which students have to consider some important things such as design method, requirement materials which were available at schools, time-consuming, costing, and testing method

-.886

10_EGN3

Use scientific knowledge to address engineering design challenges by designing, testing, evaluating, and redesigning their proposed solution

-.677

8_EGN1

Develop the design challenge or engineering design challenges which aim to identify, formulate, and scope the engineering problem that relates to the demand of the current industrial revolution and modern society through the introduction of the big design challenge in each science lesson

-.636

11_EGN4

Writing a report of their design that addresses their design by using evidence-based reasoning and presenting it to the whole class

-.565

Factor 6: Technology-based instruction self-efficacy: α = .822

13_TBL2

Use software for data collection, designing, solving the design challenges, and controlling hardware

-.762

12_TBL1

Use technology devices such as computers, projectors, and internet capabilities in my teaching and learning by leading students to watch program tutorials, instructional videos, and practices of programming the robots, and so on

-.682

Component II: Personal and material self-efficacy: α = .958

Factor 3: Personal and material self-efficacy: α = .958

40_MAS2

Adapt to new teaching situations such as those necessary to teach science from within a framework of the integrated STEM approach

-.865

38_PSE6

Develop new knowledge and skills necessary to teach science from within an integrated STEM approach

-.861

41_MAS3

Use currently available resources to provide my students with technology to engage in learning within an integrated STEM framework

-.834

39_MAS1

Learn new technologies that will enable me to teach science through a framework of an integrated STEM approach

-.833

37_PSE5

Use my understanding of the integrated STEM approach in a way that allows me to teach science effectively

-.788

36_PSE4

Use current knowledge and skills to teach science within an integrated STEM approach

-.783

42_MAS4

Access technology to teach science from within and integrated STEM framework

-.762

Internal Reliability: The Author then calculated Cronbach’s Alpha (\(\alpha\)) to evaluate the internal reliability of the subscales. Internal consistencies were \(\alpha = .924\), \(\alpha = .976\), \(\alpha = .897\), \(\alpha = .906\), \(\alpha = .822\), and \(\alpha = .958\) for the Problem-Based Instruction, Robotic Instruction, Inquiry-Based Instruction, Engineering-Based Instruction, Technology-Based Instruction, and Personal and Material Self-Efficacy subscale respectively which explained excellent internal consistency.

Outcome expectancy to teach science through integrated STEM approach (OETSIS)

The Kaiser-Meyer-Olkin coefficient for the original 12-item data set was 0.870, suggesting high sample adequacy. Bartlett’s Test of Sphericity was statistically significant (\(\chi^{2} = 671.273\), \(df = 66\), \(p < .001\)), indicating that the correlations between the items were sufficiently large for factor analysis with oblique rotation.

Communality: The extraction number for all 12 items, there were 7 items (items 3, 4, 5, 8, 9, 10, and 12) with a communality of less than 0.50, which means that less than 50% of the variance of each variable can be explained by the retained factors, therefore these items were considered to be deleted. Item 3 was the first to be dropped as it had a very low communality of 0.386. Based on this rule, further items were deleted if the communality was less than 0.50. Therefore, items 8, 12, 10, and 9 were deleted one after the other as the communality was less than 0.50. When re-analysed, the extraction resulted in 7 items with a communality of more than 0.50, which means that more than 50% of the variance of each variable can be explained by the retained factors. From this point of view, these variables were well represented in the common factor space.

Parallel Analysis: Parallel analysis indicated there were two sub-factors in total for the 7-item scale.

Pattern Matrix: The inspection of the pattern matrix of the remaining seven items revealed that there was no item that had a low factor loading of < 0.40 or a cross-loading and whose secondary factor loadings were above 0.30. Therefore, these seven items were considered to meet the criteria for factor analysis. The Kaiser-Meyer-Olkin coefficient for this data set was 0.861, and Bartlett’s Test of Sphericity was statistically significant (\(\chi^{2} = 373.225\), \(df = 21\), \(p < .01\)). Two latent factors were obtained by oblique factor rotation using the direct oblimin method. The two latent factors had eigenvalues above the Kaiser’s criterion of 1 and explained 74.222% of the variance together. The scree plot showed inflections that would justify retaining two factors consistent with Kaiser’s criterion of 1 and consistent with the parallel analysis.

The author reviewed the theoretical significance of the individual items. It turned out that items 5 and 6 have the same meaning, believing that inadequate student performance is generally due to a lack of teacher attention and complaints about teacher performance therefore, the author decided to exclude item 6 in the final scale. Re-analysis with 6 items, The Kaiser-Meyer-Olkin coefficient for this data set was 0.883, and Bartlett’s Test of Sphericity was statistically significant (\(\chi^{2} = 321.983\), \(df = 15\), \(p < .01\)). The communalities of item 5 were 0.269 which was lower than 0.50; therefor this item was removed.

Final analysis, The Kaiser-Meyer-Olkin coefficient for this data set was 0.881, and Bartlett’s Test of Sphericity was statistically significant (\(\chi^{2} = 287.524\), \(df = 10\), \(p < .01\)). As a result, a factor comprising of 5 original items were identified. This factor, called Expectations of High Students’ Achievement in Science, describes teachers’ expectations of teaching science with an integrated STEM approach to promote student achievement. This factor accounted for 70.656% of the total variance and had an eigenvalue of 3.533. The communalities were generally greater than 0.50.

Internal Reliability: Author then calculated Cronbach’s Alpha (\(\alpha\)) to evaluate the internal reliability of this factor. Internal consistencies were \(\alpha = .892\) for Expectations of High Students’ Achievement in Science. The detail factor loading, and internal reliability (\(\alpha\)) of each item please see Table 4.

Table 4. Factor loading and Cronbach’s alpha (α) for the items of the specific factor of outcome expectancy

Item

Name of item

Factor Loading

Factor 1: Expectations of high students’ achievement in science: α = .892

7_OE

Increased effort to use an integrated STEM approach in science teaching produces a big change in students’ science achievement

.860

2_OE

The science grades of students improve, it is most often because I have found that the integrated STEM approach is an effective teaching approach in science

.850

1_OE

Students will do better than usual in science when I exerted a little extra effort in using an integrated STEM approach

.828

11_OE

The effectiveness of using an integrated STEM approach in science teaching has a big influence on the achievement of students with low motivation

.727

4_OE

The inadequacy of a student’s science background can be overcome by using an integrated STEM approach

.710

Criterion Validity: Table 4 shows the significant value of comparing the mean results of the seven TSTSIS subscales across teachers’ demographic variables as well as the correlation between TSTSIS and teachers’ teaching experience. The subscale of the Robotic-Based Instruction Self-Efficacy (RBI) score was significantly different across the type of school. A Kruskal-Wallis test revealed a statistically significant difference in RBI score across the five types of school, \(\chi^{2}(4,\ N = 103) = 9.809\), \(p < .044\). The RBI score was lower in the Normal Secondary School, School under SBM/GIEP project, and Secondary Network School (\(Md = 2.00\))in comparison to Secondary Resource School (\(Md = 2.2857\)) and New Generation School (\(Md = 3.00\)). This result can be inferred that teachers from New Generation School and Secondary Resource School had high confidence in their ability to teach science through robotic-based instruction. However, the RBI score was insignificant different across other demographic independent variables (see Table 4).

Also, the findings show some criterion validity for the subscales of Inquiry-Based Instruction (IBI), Engineering-Based Instruction (EBI), and Personal Self-Efficacy (PSA) to access science teaching resources. In contrast to schools that did not include STEM education and integrated STEM approach as one of the school’s vision (\(Md = 2.71\), \(n = 44\)), (\(Md = 2.57\), \(n = 44\)), the Mann-Whitney U test showed that PSA scores were significantly higher in the school that put STEM education and integrated STEM approach as one of the school’s vision (\(Md = 3.00\), \(n = 59\)), (\(Md = 3.00\), \(n = 59\)), \(U = 925.50\), \(U = 723.00\), \(z = - 2.25\), \(z = - 4.080\), \(p = .011\), \(p = .001\) with small effect size \(r = .25\), \(r = .40\) respectively. In addition, the result of the Mann-Whitney U test denoted that the subscale of IBI and EBI scores were significantly high for the school’s integrated STEM approach as one of the school’s visions (\(Md = 3.00\), \(n = 59\)), (\(Md = 2.75\), \(n = 59\)) compared to school without (\(Md = 2.66\), \(n = 44\)), (\(Md = 2.50\), \(n = 44\)) \(U = 920.00\), \(U = 987.00\), \(z = - 2.716\), \(z = - 2.255\), \(p = .007\), \(p = .024\) with small effect size \(r = .26\), \(r = .22\) respectively. Based on these results, it can be concluded that the institution that made sufficient efforts to subsidize teaching resources in support of science instruction has influenced teachers to possess high self-efficacy in implementing inquiry- and engineering-based learning in their classrooms. However, the IBI, EBI, and PSA scores indicated insignificant differences across other important factors, such as gender, educational background, or others (see Table 4).

Another finding with regard to the subscale of the Technology-Based Instruction Self-Efficacy (TBI) score was essentially distinctive over the sort of teaching license that instructors have gotten. A Kruskal-Wallis test showed a statistically significantly difference in TBI score over the three sorts of teaching licenses in which teachers had, \(\chi^{2}(2,\ N = 103) = 6.376\), \(p < .041\). The TBI score was lower for the teachers who have a primary teaching license (\(Md = 2.00\)) in comparison to teachers who have a lower secondary teaching license (\(Md = 2.50\)) and upper secondary teaching license (\(Md = 3.00\)). Hence, teachers who had the lower and an upper-secondary teaching license had high confidence in their ability to teach science through technology-based instruction. However, the TBI was not significantly different from other demographic variables such as gender, educational background and so on (see Table 5). It’s interesting to note two TSTSIS subscales, Problem-Based Instruction (PBI) and Expected High Students’ Success in Science (ESA) were not significant across demographic variables. This suggests that science instructors had similar beliefs in using PBI with clear expectations for student outcomes.

Table 5. Mann-Whitney U test, Kruskal-Wallis test and Spearman rho correlation of TSTSIS score and teachers related demographic variables

Independent variables

PBI (Sig)

RBI (Sig)

IBI (Sig)

EBI (Sig)

TBI (Sig)

PSA

(Sig)

ESA

(Sig)

TSTSIS (Sig)

Results of Mann-Whitney U test

Gender

.977

.671

.345

.194

272

874

.589

.250

STEM education is one of the school’s vision

.331

.186

.088

.062

1.00

.011

.754

.327

The integrated STEM approach is one of the school’s vision

.062

.097

.007

.024

.284

.000

.111

.013

Results of the Kruskal-Wallis test

Major at higher education

.930

.140

.490

.726

.541

.831

.055

.319

Education background

.341

.417

.371

.538

.375

.408

.377

.312

Teaching qualification

.380

.744

.584

.288

.041

.162

.501

.240

Teaching grade

.125

.214

.688

.345

.502

.213

.404

.336

School type

.193

.044

.086

.101

.853

.166

.531

.448

School location

.699

.875

.199

.554

.768

.845

.857

.723

Results of Spearman rho correlation

Teaching experiences

-.092

-.238*

-.153

-.049

-.356**

-.130

-.228*

-.262**

*p < 0.05; **p < 0.01; ***p < 0.001

The last finding, the RBI, TBI and ESA scores correlated negatively with teaching experiences. Hence young generation teachers had higher self-efficacy to teach science through RBI, TBI and ESA compared to teachers who had long-term experience. However, the PBI, IBI, EBI, and PSA scores did not correlate with years of teaching.

DISCUSSION

The Teacher’s Self-Efficacy to Teach Science Through Integrated STEM approach instrument (TSTSIS) development and testing was conducted in this study. To cover the range of teachers’ self-efficacy to teach science using an integrated STEM approach, items were constructed based on literature, and items were retained based on the evaluation of their contribution to psychologically important factors through exploratory factor analysis, criterion validity and internal consistency analysis which was done to ensure the content and construct validity of the scale. Through this statistical analysis, it was confirmed the reliability and validity of the two main components of TSTSIS: Self-Efficacy in Teaching Science Through an Integrated STEM Approach (STSIS), which consists of six factors (Problem-based, robots-based, inquiries-based, engineering-based, technology-based instruction self-efficacy and self-efficacy in relation to personnel and materials accessing), and Outcome Expectation to Teach Science Through Integrated STEM Approach (OETSIS) with one factor (Expectations of High Students’ Achievement in Science).

The TSTSIS was created with the intention of assessing teachers’ self-efficacy and confidence in their abilities to carry out the actual teaching practice of integrated STEM approach and their expectation of doing so in their science classes. The TSTSIS, on the other hand, is a tool designed specifically for science teachers who work in secondary schools, and it focuses on both teaching self-efficacy and teaching outcome expectancy to teach science through an integrated STEM approach. The original purpose of this tool was to measure teachers’ self-efficacy in teaching science through an integrated STEM approach by assessing two components of self-efficacy: teaching self-efficacy and outcome expectancy. Unlike the SETIS, which focuses solely on teaching efficacy, outcome expectancy was not included. Additionally, the TSTSIS’s seven subscales, particularly the six constructs in the component of self-efficacy, corresponded to the core comprehensive concept of a teaching approach that was used in the integrated STEM education. Apedoe et al. (2008); Ayieko et al. (2017); Hmelo-Silver (2004) and Wei and Chen (2020) revealed that constructivism and transformational methodologies, such as problem-based learning, robotic activities, projects, science inquiry, game competitions, and the use of technology, were found to be beneficial in the real practical classroom of integrated STEM education. Moreover, creating an integrated STEM approach self-efficacy evaluation instrument is consistent with global efforts to increase diversity in, and interdisciplinarity in, STEM education, which has significant impacts on students’ critical thinking, problem solving, collaboration, teamwork, other associated and 21st century skills (Morrison et al., 2015; Mustafa et al., 2016; Polydoros, 2021).

For Criterion validity, the study conducted a comparison of sample means and correlation analysis. The results indicated a negative correlation between teachers’ teaching experiences and TSTSIS subscale scores. Moreover, there were significantly different in TSTSIS subscale score across several teacher-related variables. The score on the TSTSIS subscales were significant difference across teachers from the school which STEM education and integrated STEM approach is one of the school vision, teachers teaching qualification, and school type. These results show that the TSTSIS instrument is reliable and valid for its criterion. Moreover, it would be great if other teacher related variables such as gender, major at higher education, educational background, and school location can predict the teachers’ self-efficacy score of the TSTSIS subscales because several studies have identified these factors as significant predictors of teachers’ self-efficacy. For instance, Lee et al. (2019) found that male teachers in Taiwan perceived higher levels of self-efficacy across the five subscales of STEM knowledge compared to female teachers. Cantrell et al. (2003) reported that personal science teaching self-efficacy scores are influenced by the number of college years completed and participation in extracurricular activities. Furthermore, McCarthy et al. (2009) highlighted that rural teachers often face school isolation and limited resources, which negatively impact their self-efficacy in teaching STEM subjects, although those who overcome these challenges tend to report higher self-efficacy. In term of the teachers’ demographic variable which influenced on science teachers’ self-efficacy in using integrated STEM approach, the study found that in comparison to teachers with primary school teaching licenses, those who got lower and upper secondary school teaching licenses scored higher on a test measuring self-efficacy in implementing technology-based instruction in their science classes. The author fear to translate this result since there were only three respondents who have primary school teaching licenses​. The distinction between upper-secondary school teachers and primary and lower-secondary school teachers is that the upper-secondary teacher holds bachelor’s degrees in their fields of study, while the primary and lower-secondary school teachers do not. Through this context, upper secondary school teachers had more opportunities to learn about and practice using technology and their specialty major, which improved their self-efficacy and technological proficiency. This concept was supported by Kelley and Knowles (2016) who revealed that teachers who have experience in the practice of technology during their professional development indicated higher levels of self-efficacy for teaching STEM.

Another significant factor that influenced science teachers’ self-efficacy to use robotic-based, inquiry-based, and engineering-based instruction as well as self-efficacy in assessing teaching resources was the school factor. In comparison to their peers from normal schools, schools participating in the SBM project, Network School (NS), and schools that did not include STEM education as one of the school versions, teachers from Secondary Resources School (SRS), New Generation School (NGS) and school that STEM education is one of the school vision, scored higher on tests measuring their self-efficacy to employ robotic-based, inquiry-based, and engineering-based instruction as well as self-efficacy in accessing the teaching facilities. This significant influence was since those schools, for instance, SRS and NGS had special characteristics and uniqueness which received special support from MoEYS in promoting STEM education (MoEYS(d), 2016). Through these contexts, it could be noticed that these two schools have provided impressive administrative, financial, peers and technological support that were outstanding and had a substantial impact on the self-efficacy of the teachers and their belief of using robotic, inquiry, and engineering instruction in their science classes. This is confirmed by certain research, like those by Dong et al. (2019), who found that administration support had a beneficial impact on teachers’ self-efficacy, and Ramey-Gassert et al. (1996), who found that supportive coworkers and administrators had an impact on individuals’ self-efficacy in teaching science.

Another interesting research finding showed that young instructors had greater self-efficacy scores and high expectations for employing robotic and technology instruction methods compared to their more experienced peers. This result makes sense in the context of Cambodia because these teaching instruction methodologies were only recently implemented in the country’s educational system. For instance, the topic of information and communication technology (ICT) was included to the national curriculum of Cambodia in 2016 (MoEYS(e), 2016). In addition, MoEYS, 2020 has just released the guideline to implement the STEM approach in science classes. As a result, young generation instructors had more opportunities to experience, practice, and get a greater understanding of the usefulness of these teaching instruction approaches which has a positive impact on their degree of self-efficacy and belief in their ability to apply these teaching instructions. This was supported by Wang et al. (2004), who revealed that teachers who had vicarious experiences had high self-efficacy for technology instruction.

The last findings of this study indicated that the TSTSIS subscale scores were not significantly different by gender, school location, or teaching level, which could be interpreted as meaning that these factors did not affect teachers’ self-efficacy to teach science using an integrated STEM approach. These results were in line with those of Lin and Williams (2016) study, which found that gender had no bearing on teachers’ intention to implement STEM teaching approaches. However, evidence from the current study, also showed that the TSTSIS subscale scores did not correspond with majors at higher education and education level. This should be interpreted carefully since the distribution of participants’ majors in higher education were not equally represented, with just 6.8% of participants majoring in ICT and mathematics respectively. This was also true for the educational level, only 17.5% and 16.7% of participants having baccalaureate and master’s degrees respectively, making it challenging to interpret the related significant differences in the data.

CONCLUSION

The self-efficacy of science teachers has an impact on both professional development views and actual teaching practice (Jamil et al., 2018). This study created and tested a tool to gauge science instructors’ confidence in their ability to teach science using an integrated STEM method. Theoretically, Bandura (1977) and Armor et al. (1976) found that teaching efficacy belief and teaching outcome expectancy were the two key components of teachers’ self-efficacy. The study’s findings showed that teaching self-efficacy to teach science through an integrated STEM method (STSIS) derives six key variables, and five of those six factors have philosophical significance for actual teaching practice. As a result, those five subscales are grouped under the theme “Integrated STEM Instructional Self-Efficacy - Problem-based, robots-based, inquiries based​​, engineering-based, and technology-based instruction self-efficacy and another one factor name Teachers’ self-efficacy in relation to personnel and materials accessing. The Expectations of High Students’ Achievement in Science subscale was formulated as the factor for Integrated STEM Approach Outcome Expectation component (OETSIS)”. The TSTSIS instrument consist of seven subscale which offer a psychologically and professionally valuable tool to assess science teachers’ self-efficacy to employ integrated STEM approaches, which would be used as the basis for designing STEM training programs for science instructors. The seven TSTSIS factors also direct policy makers, teacher educators, and teachers’ trainers to concentrate on improving science teachers’ knowledge and expertise regarding the instructional practice of an integrated STEM approach, which would equip them to handle upcoming challenges on the path to providing high-quality integrated STEM education.

Limitations and Further Research

This study has some limitations, despite developing and validating the TSTSIS for assessing teachers’ self-efficacy to teach science using an integrated STEM approach. First, due to the small sample size, the author only used exploratory factor analysis (EFA), internal consistency, significant difference testing, and correlational evidence to support the validity and reliability of the TSTSIS. However, other techniques for validating the instrument, such as confirmatory factor analysis (CFA), concurrent validity, and other methods, may be used in future research. Second, given the discrepancy between teachers’ self-perceptions of teaching science through an integrated STEM approach and actual teaching practice, this study exclusively collects data only from teacher-report surveys. Measuring teachers’ pedagogical content knowledge and real instructional practice will improve the quality of science teaching and learning through integrated STEM approach. Therefore, in-depth qualitative investigations involving teacher interviews and classroom observations should be carried out to ascertain the reasons for the varying degrees of self-efficacy that science instructors may play out when implementing an integrated STEM education. Lastly, because the TSTSIS was only tested in the setting of Cambodian culture, its applicability to other nations may be limited. To further support cross-cultural comparability, it is crucial that future research examines the reliability of this instrument in other cultural situations.

  • Achieve, Inc. (2023). Next Generation Science Standards: For States, By States, Achieve. Available at: https://www.achieve.org/next-generation-science-standards.
  • Akgunduz, D. (2016). A research about the placement of the top thousand students in STEM fields in Turkey between 2000 and 2014. Eurasia Journal of Mathematics, Science and Technology Education, 12(5), 1365–1377. https://doi.org/10.12973/eurasia.2016.1518a
  • Anwar, S., Menekse, M., Guzey, S. S. and Bryan, L. (2022). The effectiveness of an integrated STEM curriculum unit on middle school students’ life science learning. Journal of Research in Science Teaching, 59(7), 1204–1234. https://doi.org/10.1002/tea.21756
  • Apedoe, X. S., Reynolds, B. and Ellefson, M. R. (2008). Bringing engineering design into high school science classrooms: The heating/cooling unit. Journal of Science Education and Technology, 17, 454–465. https://doi.org/10.1007/s10956-008-9114-6
  • Armor, D., Conroy-Osequera, P., Cox, M., King, N., McDonnel, L., Pascal, A., Pauley, E. and Zellman, G. (1976). Analysis of the School Preferred Reading Programs in Selected Los Angeles Minority Schools, (R-2007-LAUSD). Santa Monica (CA): Rand Corporation.
  • Ayieko, R. A., Gokbel, E. N. and Nelson, B. (2017). Vol. 4, Issue 5, Lehigh University. Available at: http://preserve.lehigh.edu/firehttp://preserve.lehigh.edu/fire/vol4/iss1/5.
  • Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–205. https://doi.org/10.1037/0033-295X.84.2.191
  • Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37(2), 122–147. https://doi.org/10.1037/0003-066X.37.2.122
  • Bandura, A. (1986). Fearful expectations and avoidant actions as coeffects of perceived self-inefficacy. American Psychologist, 41(12), 1389–1391. https://doi.org/10.1037/0003-066X.41.12.1389
  • Bandura, A. (1997). Self-Efficacy: The Exercise of Control. New York City (NY): Freeman.
  • Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52, 1–26. https://doi.org/10.1146/annurev.psych.52.1.1
  • Bandura, A. (2006). Adolescent development from an agentic perspective, in F. Pajares and T. Urdan (eds), Self-Efficacy Beliefs of Adolescents (pp.1–43). Greenwich (CT): Information Age Publishing.
  • Berman, P. and McLaughlin, M. (1977). Federal Programs Supporting Educational Change: Volume VII. Factors Affecting Implementation and Continuation (Research Rep. No. R- 1589D-HEW). Santa Monica (CA): Rand Corporation.
  • Bybee, R. W. (2013). The Case for STEM Education: Challenges and Opportunities. Arlington (VA): NSTA Press.
  • Cantrell, P., Young, S. and Moore, A. (2003). Factors affecting science teaching efficacy of preservice elementary teachers. Journal of Science Teacher Education,14(3), 177–192. https://doi.org/10.1023/A:1025974417256
  • Chen, Y. L., Huang, L. F. and Wu, P. C. (2021). Preservice preschool teachers’ self-efficacy in and need for STEM education professional development: STEM pedagogical belief as a mediator. Early Childhood Education Journal, 49(2), 137–147. https://doi.org/10.1007/s10643-020-01055-3
  • Committee on STEM Education of the National Science & Technology Council. (2018). Charting a Course for Success: America’s Strategy for STEM Education, Executive Office of the President. Available at: https://eric.ed.gov/?id=ED590474.
  • Cunningham, C. M., Lachapelle, C. P., Brennan, R. T., Kelly, G. J., Tunis, C. S. A. and Gentry, C. A. (2020). The impact of engineering curriculum design principles on elementary students’ engineering and science learning. Journal of Research in Science Teaching, 57(3), 423–453. https://doi.org/10.1002/tea.21601
  • DeCoito, I. and Myszkal, P. (2018). Connecting science instruction and teachers’ self-efficacy and beliefs in STEM education. Journal of Science Teacher Education, 29(6), 485–503. https://doi.org/10.1080/1046560X.2018.1473748
  • Dellinger, A. B., Bobbett, J. J., Olivier, D. F. and Ellett, C. D. (2008). Measuring teachers’ self-efficacy beliefs: Development and use of the TEBS-self. Teaching and Teacher Education, 24(3), 751–766. https://doi.org/10.1016/j.tate.2007.02.010
  • Dong, Y., Xu, C., Song, X., Fu, Q., Chai, C. S. and Huang, Y. (2019). Exploring the effects of contextual factors on in-service teachers’ engagement in STEM teaching. Asia-Pacific Education Researcher, 28(1), 25–34. https://doi.org/10.1007/s40299-018-0407-0
  • Dunteman, G. E. (1989). Principal components analysis, in Quantitative Applications in the Social Sciences. Newbury Park (CA): SAGE. https://doi.org/10.4135/9781412985475
  • Farah, A. C. (2011). Factors influencing teachers’ technology self-efficacy: A case study [PhD dissertation, Liberty University].
  • Fenton, D. and Essler-Petty, S. (2019). Self-efficacy and STEM: An integrated pedagogical approach for pre-service elementary teachers. International Journal for Cross-Disciplinary Subjects in Education, 10(4), 4160–4168. https://doi.org/10.20533/ijcdse.2042.6364.2019.0508
  • Friday Institute for Educational Innovation. (2012). Teacher Efficacy and Attitudes Toward STEM Survey-Elementary Teachers. Raleigh (NC): Friday Institute for Educational Innovation.
  • Geng, J., Jong, M. S. Y. and Chai, C. S. (2019). Hong Kong teachers’ self-efficacy and concerns about STEM education. Asia-Pacific Education Researcher, 28(1), 35–45. https://doi.org/10.1007/s40299-018-0414-1
  • George, D. and Mallery, P. (2020). IBM SPSS Statistics 26 Step by Step. A Simple Guide and Reference. London (UK): Routledge. https://doi.org/10.4324/9780429056765
  • Granziera, H. and Perera, H. N. (2019). Relations among teachers’ self-efficacy beliefs, engagement, and work satisfaction: A social cognitive view. Contemporary Educational Psychology, 58, 75–84. https://doi.org/10.1016/j.cedpsych.2019.02.003
  • Gunning, A. M. and Mensah, F. M. (2011). Preservice elementary teachers’ development of self-efficacy and confidence to teach science: A case study. Journal of Science Teacher Education, 22(2), 171–185. https://doi.org/10.1007/s10972-010-9198-8
  • Guzey, S. S. and Li, W. (2023). Engagement and science achievement in the context of integrated STEM Education: A longitudinal study. Journal of Science Education and Technology, 32(2), 168–180. https://doi.org/10.1007/s10956-022-10023-y
  • Hair, F. J., Black, C. W., Babin, J. B. and Anderson, E. R. (2009). Multivariate Data Analysis. Hoboken (NJ): Prentice Hall. https://doi.org/10.1016/j.csda.2008.11.030
  • Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16, 235–266. https://doi.org/10.1023/B:EDPR.0000034022.16470.f3
  • Hutcheson, G. and Sofroniou, N. (1999). The Multivariate Social Scientist. London (UK): SAGE. https://doi.org/10.4135/9780857028075
  • IBM Corp. (2017). IBM SPSS Statistics for Windows, Version 26.0. Armonk (NY): IBM Corp.
  • Jaipal-Jamani, K. and Angeli, C. (2017). Effect of robotics on elementary preservice teachers’ self-efficacy, science learning, and computational thinking. Journal of Science Education and Technology, 26(2), 175–192. https://doi.org/10.1007/s10956-016-9663-z
  • Jamil, F. M., Linder, S. M. and Stegelin, D. A. (2018). Early childhood teacher beliefs about STEAM education after a professional development conference. Early Childhood Education Journal, 46(4), 409–417. https://doi.org/10.1007/s10643-017-0875-5
  • Johnson, T. M., Byrd, K. O. and Allison, E. R. (2021). The impact of integrated STEM modeling on elementary preservice teachers’ self-efficacy for integrated STEM instruction: A co-teaching approach. School Science and Mathematics, 121(1), 25–35. https://doi.org/10.1111/ssm.12443
  • Kaiser, H. F. (1970). A second-generation little jiffy. Psychometrika, 35, 401–415. https://doi.org/10.1007/BF02291817
  • Kelley, T. R., & Knowles, J. G. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education, 3, 11. https://doi.org/10.1186/s40594-016-0046-z
  • Khalil, N. M. and Osman, K. (2017). STEM-21CS module: Fostering 21st century skills through integrated STEM. K-12 STEM Education, 3(3), 225–233.
  • Knoblauch, D. and Chase, M. A. (2015). Rural, suburban, and urban schools: The impact of school setting on the efficacy beliefs and attributions of student teachers. Teaching and Teacher Education, 45, 104–114. https://doi.org/10.1016/j.tate.2014.10.001
  • Lee, M. H., Hsu, C. Y. and Chang, C. Y. (2019). Identifying Taiwanese teachers’ perceived self-efficacy for science, technology, engineering, and mathematics (STEM) knowledge. Asia-Pacific Education Researcher, 28(1), 15–23. https://doi.org/10.1007/s40299-018-0401-6
  • Lin, K. Y. and Williams, P. J. (2016). Taiwanese preservice teachers’ science, technology, engineering, and mathematics teaching intention. International Journal of Science and Mathematics Education, 14(6), 1021–1036. https://doi.org/10.1007/s10763-015-9645-2
  • MacCallum, R. C., Widaman, K. F., Zhang, S. and Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84–99. https://doi.org/10.1037/1082-989X.4.1.84
  • Maddux, J. E. (1995). Self-efficacy theory: An introduction, in J. E. Maddux (ed), Self-Efficacy, Adaptation, and Adjustment: Theory, Research, and Application (pp. 3–33). New York City (NY): Plenum Press. https://doi.org/10.1007/978-1-4419-6868-5_1
  • McCarthy, C. J., Lambert, R. G., O’Donnell, M. and Melendres, L. T. (2009). The relation of elementary teachers’ experience, stress, and coping resources to burnout symptoms. Elementary School Journal, 109(3), 282–300. https://doi.org/10.1086/592308
  • Menon, D., Shorman, D. A. A., Cox, D. and Thomas, A. (2023). Preservice elementary teachers conceptions and self-efficacy for integrated STEM. Education Sciences, 13(5), Article 529. https://doi.org/10.3390/educsci13050529
  • Mobley, M. C. (2015). Development of the SETIS instrument to measure teachers’ self-efficacy to teach science in an integrated STEM framework [PhD dissertation, University of Tennessee].
  • MoEYS(a). (2016). Policy on Science Technology Engineering and Mathematics (STEM) Education.
  • MoEYS(b). (2022). STEM Manual for Upper Secondary School Teachers.
  • MoEYS(c). (2020). STEM Teaching and Learning Science According to STEM Method for Lower Secondary School, 7th, 8th and 9th Grades.
  • MoEYS(d). (2016). Policy Guidelines for New Generation School. For Basic Education in Cambodia.
  • MoEYS(e). (2016). Curriculum Framework of General Education and Technical Education.
  • Mok, M. M. C. and Moore, P. J. (2019). Teachers & self-efficacy. Educational Psychology, 39(1), 1–3. https://doi.org/10.1080/01443410.2019.1567070
  • Moore, T. J., Glancy, A. W., Tank, K. M., Kersten, J. A., Smith, K. A. and Stohlmann, M. S. (2014). A framework for quality K-12 engineering education: Research and development. Journal of Pre-College Engineering Education Research, 4(1), Article 2. https://doi.org/10.7771/2157-9288.1069
  • Morrison, J., Mcduffie, A. R. and French, B. (2015). Identifying key components of teaching and learning in a STEM school. School Science and Mathematics, 115(5), 244–255. https://doi.org/10.1111/ssm.12126
  • Mustafa, N., Ismail, Z., Tasir, Z. and Mohamad Said, M. N. H. (2016). A meta-analysis on effective strategies for integrated STEM education. Advanced Science Letters, 22(12), 4225–4288. https://doi.org/10.1166/asl.2016.8111
  • Nadelson, L. S. and Seifert, A. L. (2017). Integrated STEM defined: Contexts, challenges, and the future. Journal of Educational Research, 110(3), 221–223. https://doi.org/10.1080/00220671.2017.1289775
  • National Research Council. (2010). Inquiry and the National Science Education Standards: A Guide for Teaching and Learning. Washington, D.C.: National Academy Press.
  • Ng, C. H. and Adnan, M. (2018). Integrating STEM education through project-based inquiry learning (PIL) in topic space among year one pupils. IOP Conference Series: Materials Science and Engineering, 296, Article 012020. https://doi.org/10.1088/1757-899X/296/1/012020
  • Pallant, J. (2002). SPSS Survival Manual. A Step-by-Step Guide to Data Analysis Using IBM SPSS. London (UK): Routledge.
  • Polydoros, G. (2021). Engaging STEM methodology to teach science in primary education. Journal of Research and Opinion, 8(7), 2991–2994.
  • Ramey-Gassert, L., Shroyer, M. G. and Staver, J. R. (1996). A qualitative study of factors influencing science teaching self-efficacy of elementary level teachers. Science Education, 80(3), 283–315. https://doi.org/10.1002/(SICI)1098-237X(199606)80:3<283::AID-SCE2>3.0.CO;2-A
  • Ramli, N. F., Talib, O., Hassan, S. A. and Manaf, U. K. A. (2020). Development and validation of an instrument to measure STEM teachers’ instructional preparedness. Asian Journal of University Education, 16(3), 193–206. https://doi.org/10.24191/ajue.v16i3.11084
  • Riggs, I. M. and Enochs, L. G. (1990). Toward the development of an elementary teacher’s science teaching efficacy belief instrument. Science Education, 74(6), 625–637. https://doi.org/10.1002/sce.3730740605
  • Roberts, J. E., Shapiro, A. M. and Gamble, S. A. (1999). Level and perceived stability of self-esteem prospectively predict depressive symptoms during psychoeducational group treatment. British Journal of Clinical Psychology, 38(4), 425–429. https://doi.org/10.1348/014466599162917
  • Sanders, M. (2009). STEM, STEM education, STEMmania. The Technology Teacher, 68(4), 20–26.
  • Selcen Guzey, S., Harwell, M., Moreno, M., Peralta, Y. and Moore, T. J. (2017). The impact of design-based stem integration curricula on student achievement in engineering, science, and mathematics. Journal of Science Education and Technology, 26(2), 207–222. https://doi.org/10.1007/s10956-016-9673-x
  • Sokha, K. and Kinya, S. (2023). Integrating STEM approach in K-12 science education teaching practice: A systematic literature review. International Journal of Research in STEM Education, 5, 1–18. https://doi.org/10.33830/ijrse.v5i2.1598
  • Tawbush, R. L., Stanley, S. D., Campbell, T. G. and Webb, M. A. (2020). International comparison of K-12 STEM teaching practices. Journal of Research in Innovative Teaching & Learning, 13(1), 115–128. https://doi.org/10.1108/jrit-01-2020-0004
  • Toma, R. B. and Greca, I. M. (2018). The effect of integrative STEM instruction on elementary students’ attitudes toward science. Eurasia Journal of Mathematics, Science and Technology Education, 14(4), 1383–1395. https://doi.org/10.29333/ejmste/83676
  • Tschannen-Moran, M. and Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17(7), 783–805. https://doi.org/10.1016/S0742-051X(01)00036-1
  • Tschannen-Moran, M. and Johnson, D. (2011). Exploring literacy teachers’ self-efficacy beliefs: Potential sources at play. Teaching and Teacher Education, 27(4), 751–761. https://doi.org/10.1016/j.tate.2010.12.005
  • Tschannen-Moran, M., Hoy, A. W. and Hoy, W. K. (1998). Teacher efficacy: Its meaning and measure. Review of Educational Research, 68(2), 202–248. https://doi.org/10.3102/00346543068002202
  • Wahono, B., Chang, C.-Y. and Thi To Khuyen, N. (2021). Teaching socio-scientific issues through integrated STEM education: An effective practical averment from Indonesian science lessons. International Journal of Science Education, 43(16), 2663–2683. https://doi.org/10.1080/09500693.2021.1983226
  • Wang, L., Ertmer, P. A. and Newby, T. J. (2004). Increasing preservice teachers’ self-efficacy beliefs for technology integration. Journal of Research on Technology in Education, 36(3), 231–250. https://doi.org/10.1080/15391523.2004.10782414
  • Wei, B. and Chen, Y. (2020). Integrated STEM education in K-12: Theory development, status, and prospects, in K. G. Fomunyam (ed), Theorizing STEM Education in the 21st Century. London (UK): IntechOpen. https://doi.org/10.5772/intechopen.88141
  • Wendell, K. B., Wright, C. G. and Paugh, P. (2017). Reflective decision-making in elementary students’ engineering design. Journal of Engineering Education, 106(3), 356–397. https://doi.org/10.1002/jee.20173
  • Wolters, C. A. and Daugherty, S. G. (2007). Goal structures and teachers’ sense of efficacy: Their relation and association to teaching experience and academic level. Journal of Educational Psychology, 99(1), 181–193. https://doi.org/10.1037/0022-0663.99.1.181
  • Yaki, A. A., Saat, R. M., Sathasivam, R. V. and Zulnaidi, H. (2019). Enhancing science achievement utilising an integrated STEM approach. Malaysian Journal of Learning and Instruction, 16(1), 181–205. https://doi.org/10.32890/mjli2019.16.1.8
  • Yang, W., Wu, R. and Li, J. (2021). Development and validation of the STEM teaching self-efficacy scale (STSS) for early childhood teachers. Current Psychology, 42, 7275–7283. https://doi.org/10.1007/s12144-021-02074-y
  • Zakariya, Y. F. (2020). Effects of school climate and teacher self-efficacy on job satisfaction of mostly STEM teachers: A structural multigroup invariance approach. International Journal of STEM Education, 7, Article 10. https://doi.org/10.1186/s40594-020-00209-4
AMA 10th edition
In-text citation: (1), (2), (3), etc.
Reference: Khut S. Development and Validation of the TSTSIS Instrument to Measure Teachers’ Self-Efficacy to Teach Science Through Integrated STEM Approach. European Journal of STEM Education. 2024;9(1), 14. https://doi.org/10.20897/ejsteme/15480
APA 6th edition
In-text citation: (Khut, 2024)
Reference: Khut, S. (2024). Development and Validation of the TSTSIS Instrument to Measure Teachers’ Self-Efficacy to Teach Science Through Integrated STEM Approach. European Journal of STEM Education, 9(1), 14. https://doi.org/10.20897/ejsteme/15480
Chicago
In-text citation: (Khut, 2024)
Reference: Khut, Sokha. "Development and Validation of the TSTSIS Instrument to Measure Teachers’ Self-Efficacy to Teach Science Through Integrated STEM Approach". European Journal of STEM Education 2024 9 no. 1 (2024): 14. https://doi.org/10.20897/ejsteme/15480
Harvard
In-text citation: (Khut, 2024)
Reference: Khut, S. (2024). Development and Validation of the TSTSIS Instrument to Measure Teachers’ Self-Efficacy to Teach Science Through Integrated STEM Approach. European Journal of STEM Education, 9(1), 14. https://doi.org/10.20897/ejsteme/15480
MLA
In-text citation: (Khut, 2024)
Reference: Khut, Sokha "Development and Validation of the TSTSIS Instrument to Measure Teachers’ Self-Efficacy to Teach Science Through Integrated STEM Approach". European Journal of STEM Education, vol. 9, no. 1, 2024, 14. https://doi.org/10.20897/ejsteme/15480
Vancouver
In-text citation: (1), (2), (3), etc.
Reference: Khut S. Development and Validation of the TSTSIS Instrument to Measure Teachers’ Self-Efficacy to Teach Science Through Integrated STEM Approach. European Journal of STEM Education. 2024;9(1):14. https://doi.org/10.20897/ejsteme/15480
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Submit My Manuscript



Phone: +31 (0)70 2190600 | E-Mail: info@lectitojournals.com

Address: Cultura Building (3rd Floor) Wassenaarseweg 20 2596CH The Hague THE NETHERLANDS

Disclaimer

This site is protected by copyright law. This site is destined for the personal or internal use of our clients and business associates, whereby it is not permitted to copy the site in any other way than by downloading it and looking at it on a single computer, and/or by printing a single hard-copy. Without previous written permission from Lectito BV, this site may not be copied, passed on, or made available on a network in any other manner.

Content Alert

Copyright © 2015-2024 LEUKOS BV All rights reserved.