Research Design for On-the-fly Feedback – University of Copenhagen

Forward this page to a friend Resize Print Bookmark and Share

ASSIST-ME > Researchers > Research Design for On...

Research Design for On-the-fly Feedback


Focus of the specific research design: In-depth analysis of the enactment of the assessment method “interactions on the fly” in certain situations as well as documentation of relevant challenges.

Specific Research Question Associated with the research design

What are the challenges for teachers’ use of “interactions on the fly” as a means of formative assessment for promoting a selected inquiry competence?

Corresponding project research goals

What systemic support measures and what tools do teachers need in order to integrate formative assessment of student learning in their classroom practice? (1.2)

Illustration of the Research Design

Rationale: The activity sequence should be designed so as to incorporate whole-class discussions and/or discussions between smaller groups of students and the teacher. The work and or discussion will be video recorded. The analysis of the data will focus on the content/quality of the interactions in the different situations and the productivity of the teacher’s feedback. One aspect of the analysis of the content/quality of the interactions is to identify missed opportunities. (Missed opportunities are defined as instances of the teacher-students conversations in which even though the teacher had a chance to build on students’ contributions so as to introduce/elaborate aspects that were among his/her priorities, s/he failed to do so (or refrained from doing so).) Further, teachers are interviewed in a semi-structured fashion on the challenges and opportunities of using this assessment method. The goal of the analysis will be to identify challenges that teachers are confronted with in their attempt to take advantage of “interactions on the fly” as a formative assessment method.

The analysis will focus on the following analytical questions:

1. What is the content of the interactions between the teacher and the students?

2. What factors seem to facilitate or impede teachers’ attempt to guide students towards the inquiry learning goals using interactions on the fly?

a. What are the emergent factors that seem to afford productive teacher feedback?

b. What are the various types of missed opportunities (as identified in the rationale) encountered in the interactions on the fly; and what are the possible interpretations for why these opportunities were missed by the teacher?

3. What challenges and opportunity do teachers report about their use of “interactions on the fly” as an assessment method?

Scope of the research design/Constraints to be satisfied:

This research design assumes that the teaching to be enacted includes multiple possibilities for teachers to interact with students on the fly defined in the following way (cf. D4.7, p. 18)

• “Interactions on-the-fly [are] informal formative assessment[s] of individual students or small groups of students.”

• “On-the-fly assessment cannot be planned beforehand but takes place spontaneously when the teacher recognises good opportunities.”

The application of this research design must meet the following minimal criteria:

A) The teaching/learning sequence must afford multiple possibilities for teachers to interact spontaneously with students. It must be clear from the implementation scheme where interactions on the fly are expected to occur. For example, will the interactions on the fly occur primarily in whole-class discussions, small group discussions, lab work exercises etc.?

B) The LWG should formulate the explicit learning goals and corresponding levels of attainment. This must not only be used to shape the enacted teaching/learning sequence, but also be used during the video analysis that is performed by two or more researchers from the partner.

C) Partners will be asked to provide an English translation of the specific tool they will be using for collecting and coding data from the teacher for this purpose. In particular, the partner must specify which parts of the teaching/learning sequence will be video-recorded and analysed.

D) Partners will be asked to describe how the definition of “missed opportunities” applies to their case: how would “missed opportunities” look like in the video data?

E) Teachers will participate in follow-up semi-structured interviews intended to shed light onto aspects of using interactions on the fly as an assessment method.

Note: You will be asked to describe how you will fulfil these criteria in the implementation scheme, and you will be asked to provide a translated version of the assessment tool that was ultimately implemented.

WP5 will provide:

1. A generic coding scheme for the analysis of the video data with respect to

a. The content of the interactions between the teacher and the student

b. Reporting and categorizing factors that facilitate or impede teachers’ attempt to guide students towards the inquiry learning goals using interactions on the fly

c. Reporting and categorizing missed opportunities encountered in the interactions on the fly, and the associate interpretations for why these opportunities were missed by the teacher?

2. A generic interview protocol for the semi-structured interviews. Note: the individual partner can add to this protocol in order to capture aspects of the teacher’s experience that are relevant to the particular context.

WP5 has provided examples meeting the minimal criteria for the competence of investigation.

Additionally this research design assumes that the following criteria are met:

1. The LWG is responsible for ensuring that the teacher’s implementation is consistent with the plans of the LWG.

2. The LWG is responsible for supporting the process of collecting the required research data during and after the implementation.

3. The responsibility for the research data collection resides with the researchers of the LWG who are also anticipated to safeguard the inter-rater reliability of the data analysis.

Anticipated output of this research design

At the local level this research design will lead to case studies that will focus on describing specific instances of teachers’ attempt to employ “interactions on the fly” as an assessment method. This may include the identification and documentation of challenges associated with the implementation of this assessment method in the context of inquiry-based Teaching/Learning Sequences (TLS) for addressing one of the competences emphasized by the project. This study could bear implications for (a) the practice of formative assessment – e.g., in terms of requirements imposed on students, (b) formative assessment tools (e.g., what tools could support teachers in managing interactions on the fly?) and (c) teachers Continuing Professional Development.

Provided that this research design is implemented by more than one partners it will be possible to also address questions associated with the comparison of the challenges across different situations.