ILETC methodology and methods
Overview
Illustrated in Figure 1, the project comprised of three areas of enquiry: (1) Students’ deep learning outcomes; (2) measurement of teaching practices; and (3) design of learning environments. The guiding research was “Can altering teacher mind frames unlock the potential of innovative learning environments? Three subsidiary research questions provided the scope of the study:
- What is the current situation in Australian and New Zealand schools in terms of the number of, types of, and teacher/student activities within ILEs?
- What characterises ‘good transition’ by teachers into ILEs?
- What correlations can be established between ILEs and high levels of student deep learning, and high incidence of favourable teacher mind frames?
A mixed methods design was utilised, drawing from theoretical frameworks and data collection methods drawn from multiple disciplinary foci, some of which are described here.
The plan
phase 1: exploratory
- The first stage involved defining key concepts and collecting baseline data. This involved a preliminary, whole-population, 15-20 item, web-based survey as well as systematic reviews and meta-analyses, to establish base line data relevant to the broad research question related to ‘ILE affordances’, ‘teacher mind frames’ and ‘deep learning characteristics’. This baseline data was used to develop the sampling frame for Phase 1 of the study.
- The second stage, the telephone census filled in missing ILE data and recruit school sites for case studies.
- The third stage, a detailed participant web-based survey was developed and administered to teachers in schools identified by stage one and two of the study. The detailed survey identified the dependent and background variables to be investigated at potential case study research sites.
- The fourth stage was the case studies. By selecting a sample of great diversity of information-rich cases, the researchers ensured high-quality, in-depth insights into issues of central importance to this research. These case studies were not of specific schools, rather ‘clusters of sites, participants and activities’ that exhibit the required ILE characteristics relevant to each PhD project.
phase 2: development
- The first stage of Phase two was the development of a multi-faceted resource, containing; (1) knowledge tools; (2) strategy tools; (3) evaluation tools; and (4) professional development tools. This resource increased teacher’s spacial competence of teaching and learning activities.
- The second stage was the development of a matrix describing teacher mind frame activities across a two-dimensional axis comprising ILE proficient/non ILE proficient, and Graduate/Lead teacher measures. This matrix was used to test the resource strategies across a limited but representative sample of schools.
- The third stage piloted the resource and matrix in a number of trial sites. This stage involved the refinement of the resources and matrix, which were designed to clarify and exemplify opportunities for sustainable adoption. Positioned and delivered with deep reference to evidence-based research, the resource and matrix provided practical strategies for teachers and schools to maximize the use of ILEs to improve teaching practices.
phase 3: intervention
- The first stage involved implementing the resource in a series of intervention case study sites across the widest possible array of educational settings, including geographical, (rural/urban/remote), socio-economic and differing school systems. Employing a mixed-method quasi-experimental design, evaluation approaches included inferential analyses and qualitative approaches to address the main research question.
- The second (and final) stage involved analysis and reporting. Changes in teachers’ behaviors, lessons, interactions, activities, and assessments were measured over time to chart progress in changing mind frames, and effecting deeper learning with students. The final report consolidated the results of each phase of the project.
Data collection tools
Space Design and Use Survey
The Space Design and Use survey was created to provide baseline data for the ILETC project. The survey consisted of items inquiring about the types of learning spaces, teaching approaches, and learning approaches prevalent in schools. The survey is divided into five sections based on conceptual and/or analytical frameworks derived from literature (see Table 1 for questions and response formats): (a) types of physical learning spaces; (b) types of teaching in these spaces; (c) learning potential; (d) teacher mind frames; and (e) student deep learning (Imms et al., 2017; Mahat & Imms, 2020). For more information, see report
Table 1: Survey questions and response format
| Variable | Survey question | Number of items | Response format |
|---|---|---|---|
| Type of physical learning spaces | Of the five types of learning spaces illustrated below, please indicate the percentage of each type that is prevalent in your school. Please ensure that your answers total 100%. | 5 items | Percentage of type |
| Type of teaching in these spaces | Of the six teaching approaches illustrated below, please indicate the percentage of time devoted to each approach in your school. Please ensure that your answers total 100%. | 6 items | Percentage of time |
| Teacher mind frames | Please indicate the most appropriate response for each statement, reflecting your personal opinion. | 8 items | 4-point Likert scale: Strongly disagree, Disagree, Agree, and Strongly agree. |
| Student deep learning | Please indicate the most appropriate response for each statement, reflecting your personal opinion. | 10 items | 4-point Likert scale: Strongly disagree, Disagree, Agree, and Strongly agree. |
Teacher Transition Survey
The Teacher Transition Survey centres around the five typology of spaces. The aim is to understand teachers’ perspective on their transition from a traditional learning space into an innovative learning environment. The survey is divided into six sections. A survey logic was incorporated in the first section of the survey to ensure respondents only provided responses that were applicable to their experiences in those spaces. Consequently, respondents who do not have prior experiences in ILEs (Types C, D and E) did not respond to Section F. An additional section asked respondents for their background information such as school type and years of teaching. Two open-ended questions asked respondents for their perspectives on effectively transitioning into ILEs. For more information about the teacher ‘s spatial transition framework, see chapter [https://link.springer.com/chapter/10.1007/978-981-15-7497-9_25].
Table 2: Survey questions and response format
| Variable | Survey question | Response format |
|---|---|---|
| Section A: Types of physical learning spaces | Throughout your teaching career, how many of these types of spaces have you taught in? Choose all that are applicable. | Multi-option format: Type A, Type B, Type C, Type D & Type E |
| Section B: Teaching in these spaces | Of these spaces, what do you feel is the percentage of time you have spent in each, across your career. | Percentage of time |
| Section C: Learning space impact on teaching | For each type of learning space, please indicate how it affects your teaching: | 6-point Likert scale: Strongly impedes, Impedes, Neutral, Supports & Strongly supports (for each type of space selected in section A) |
| Section D: Adapting to your teaching spaces | Approximately how long did it take until you felt comfortable teaching in these spaces? | Multi-option format: Not yet comfortable, 0-6 months, 7-12 months, 1-3 years, 4-5 years, & More than 5 years (for each type of space selected in section A) |
| Section E: Preferred teaching space | Please select your preferred space. Choose one only. | Multi-option format: Type A, Type B, Type C, Type D & Type E |
| Section F: Teacher transition into type C/D/E spaces | (a) Would you consider this [factor] useful? | Binary response format: Yes or No (for each factor, total 14) |
| (b) How much help have you received in addressing this? | 3-point Likert scale: No help, Some help, Significant help (for each factor, total 14) |
Learning Approach Questionnaire
Based on Biggs’ (1987) Learning Approach Questionnaire (LAQ), this survey consisted of 16 items within two sub-scales of deep and surface learning approaches (see Table 3). The aim of the survey is to gauge the degree to which students hold deep and surface approaches to studying. (Murphy, forthcoming2020). Both factor scales comprise items gauging motives and strategies indicative of the respective approaches. Responses were given on a five-point Likert scale comprising of:
- This item is never or only rarely true of me
- This item is sometimes true of me
- This item is true of me about half the time
- This item is frequently true of me
- This item is always or almost always true of me.
Table 3: Learning Approach Questionnaire
| Deep approach | Surface Approach |
|---|---|
| Q1. I come to most classes with questions in mind that I want answering. | Q2. As long as I feel I am doing enough to pass, I devote as little time to studying as I can. There are many more interesting things to do. |
| Q6. I work hard at my studies because I find the material interesting. | Q3. I like to be told precisely what to do in assignments. |
| Q7. When I read a textbook, I try to understand what the author means. | Q4. I prefer subjects in which I have to learn just facts to ones which require a lot of reading and understanding material. |
| Q8. I like to do enough work on a topic so that I can form my own conclusions before I am satisfied. | Q.5 I see no point in learning material which is not likely to be in the examination. |
| Q10. I try to relate what I have learned in one subject to what I learn in other subjects. | Q9. I find it is not helpful to study topics in depth. You don’t really need to know much in order to get by in most topics. |
| Q12. I like constructing theories to fit odd things together. | Q11. I think that teachers shouldn’t expect secondary students to work on topics that are outside the set course. |
| Q13. I try to relate new material, as I am reading it, to what I already know on that topic. | Q15. I find I can get by in most assessment by memorising key sections rather than trying to understand them. |
| Q14. I feel that nearly any topic can be highly interesting once I get into it. | Q16. I generally restrict my study to what is specifically set as I think it is unnecessary to do anything extra. |
Teacher Mind Frame Survey
The Teacher Mind Frame Survey is a 34-item survey gauging the degree to which teachers hold eight mind frames based on Hattie’s (2012) identification of eight ways of thinking more likely to have positive impact on student learning (Murphy, forthcoming). The 34 items are organised in 8 sub-scales broadly defined as, Learning, Change Agent, Assessment, Trust, Evaluator, Collaborate, Dialogue and Feeback. Responses were given on a 6-point Likert scale with the options of Never, Rarely, Sometimes, Often, Very often, and Always
Table 4: Teacher Mind Frame Survey
| Item | Scale |
|---|---|
| I use many strategies so I can understand student learning | Learning |
| I teach my students to use different learning strategies | Learning |
| When I plan student learning I employ multiple ways to work out if they have made progress. | Learning |
| I use multiple ways to find out what students know. | Learning |
| You can learn new things but cannot really change your basic intelligence. | Change agent |
| Intelligence is something people are born with and cannot change very much. | Change agent |
| Each student has a certain amount of ability and you can’t really do much to change this natural ability. | Change agent |
| You can greatly change how intelligent you are. | Change agent |
| I reflect on student data to inform my practice. | Assessment |
| I evaluate my practice using student data. | Assessment |
| I use data to drive my professional development. | Assessment |
| I see student data as informing my impact and next steps. | Assessment |
| I use student data to discuss learning steps with my students. | Assessment |
| All students in my class trust each other. | Trust |
| All students in my class respect other students who are expert in their learning. | Trust |
| All students in my class accept help from any other student. | Trust |
| All students in my class feel respected by other students. | Trust |
| I believe student assessment is feedback about me. | Evaluator |
| Test and assignment data inform me about my performance as well as of students. | Evaluator |
| When I gather student data, I see it as a result of my teaching. | Evaluator |
| When looking over students' results I ask myself if I am having the desired impact. | Evaluator |
| I meet with colleagues to discuss my impact & next steps. | Collaborate |
| I actively share learning with my colleagues. | Collaborate |
| I work with others to plan strategic directions as well as student learning. | Collaborate |
| My peers are influential components of my learning. | Collaborate |
| I allow time for discussion among students. | Dialogue |
| I plan lessons to allow for small group discussion & learning. | Dialogue |
| I encourage classroom discussion to complement my instruction. | Dialogue |
| Cooperation with peers is an important part of students’ learning. | Dialogue |
| Feedback is an important part of learning. | Feedback |
| I help students understand the feedback I give them. | Feedback |
| I integrate feedback into my lessons. | Feedback |
| I give feedback to help students improve their learning. | Feedback |
| I facilitate feedback between myself and students. | Feedback |
ILETC glossary of Terms
| Term | Definition |
|---|---|
| Innovative Learning Environment | An innovative learning environment can be defined “as the product of innovative design of space and innovative teaching and learning practices. Innovative learning spaces are physical educational facilities designed and built to facilitate the widest array of flexibility in teaching, learning, and social educational activity while innovative teaching and learning practices are the sum of teaching and learning activities that in combination assist in the best possible learning outcomes and learning skills of students” (Mahat et al., 2018, p. 20). A learning space becomes an innovative learning environment when innovative design is combined with innovative teaching and learning practices. |
| Teacher Mind Frames | Teacher mind frames can be defined as the ways in which teachers consciously think about their teaching roles, the content and pedagogical knowledge, which in turn has an impact on their attitudes, actions and decisions that are likely to have significant impacts on student learning (Mahat et al., 2018, p. 33). Teachers who exhibit these teacher mind frames are “more likely to have major impacts on student learning” (Hattie, 2017, p. 182). |
| Student Deep Learning | Student deep learning is characterised by “critically applying new facts to existing knowledge, searching for (as opposed to accepting) meaning, being actively curious about new knowledge, and accepting that learning is a part of their personal development” (Mahat et al., 2018, p. 40). This definition accepts that deep learning is rarely fully achieved, with students operating across a surface-to-deep learning continuum. |
| Learning Environment Affordances | Learning environment affordances are qualities of the environment (space, objects and people) which enable perceived teaching and learning activities and behaviours (Young, Cleveland, and Imms, 2019, p. 697). |
| Pedagogical Affordances | Pedagogical affordances can defined as the “possible uses of an element of the learning environment to facilitate the learning of another individual” (Villafranca, 2019, p. 192). |
| Curated Learning | Curated learning is a “pedagogical approach that involves purposeful selection, manipulation, and use of elements within a learning environment to set-up conditions for students’ deep learning” (Villafranca, 2019, p. 248). |
| Teacher collaboration | In the context of innovative learning environments, teacher collaboration can be defined as the spatialised intersection of teaching together, working together, and being together, characterised by a set of practices that jointly leverage opportunities engendered by new spaces, contemporary pedagogies, and teams of teachers, to best meet the needs of all learners (Bradbeer, 2020). |
| Teacher Spatial Competency | Teacher Spatial Competency is a teaching skill embedded within a teacher’s professional practice. It manifests as behaviours that are determined by a transaction that occurs between: an individual’s situated presence in a teaching space (which also places the teacher in a specific school context); the sensory materiality of the space (which helps the teacher decide how to engage with it); the individual’s perceived assessment of the spaces’ usefulness to their goal (to teach) based on their experiences; and the teacher’s determination of the environment and its affordances potential contribution (in real time) in order to activate it for teaching (Leighton, 2020). |