Research Design for Written Feedback – University of Copenhagen

Forward this page to a friend Resize Print Bookmark and Share

ASSIST-ME > Researchers > Research Design for Wr...

Research Design for Written Feedback


Focus of the specific research design: Investigation of the potential of certain tools to facilitate the effective implementation of the assessment method “Written feedback” by supporting teachers’ attempt to interpret students’ data, diagnose difficulties/needs and provide them with productive feedback.

Specific Research Questions Associated with the research design

• To what extent can a specially designed assessment tool for Written feedback provide teachers with productive information so as to diagnose students’ needs and level of attainment of a selected competence and provide feedback to the students on that basis?

• What are the various ways in which students respond to the feedback they receive? Is there a connection between different “responses” and type of feedback?

• What are the challenges and opportunities for using Written Feedback along with a specially designed assessment tool, as a formative assessment method for promoting students’ attainment of a selected competence?

Corresponding project research goals

What systemic support measures and what tools do teachers need in order to integrate formative assessment of student learning in their classroom practice? (1.2)

Illustration of the Research Design

Rationale: At a specific point of the activity sequence, students submit to the teacher certain artefacts they have produced during the enacted teaching, associated with the competence/sub-competence under emphasis (see indicative list of possible types of artefacts at the end of this document). The teacher provides written feedback to each student. For this s/he uses a specially designed tool intended to facilitate his/her attempt to diagnose students’ needs or difficulties, with respect to the competence/sub-competence under emphasis, but also their achievements. The feedback provided by the teacher will be coded. Students will then respond in writing to the comments they received and they will also undertake to revise the initial version of their artefact, taking into account the comment they received. The researchers use the data collected (i.e., initial version of student artefact, feedback comments, revised artefact and response to the comments) to evaluate the extent to which this assessment method was implemented in a productive manner and identify and document possible challenges or intricacies.

The analysis will focus on the following analytical questions:

1. To what extent did the tool intended to support teacher’s attempts to provide students with feedback productively served this purpose?

a. Did the teacher produce relevant, productive feedback that was well targeted at students’ needs? What is the correspondence between the feedback provided by the teacher and the feedback that would be provided by a knowledgeable peer with expertise about the competence/sub-competence under emphasis?

2. How did students respond to the feedback?

a. What are the various ways in which students responded to the feedback they received?

b. To what extent did they actually draw on the feedback comments for revising the initial version of their artefacts?

c. What are the possible interpretations for noteworthy behaviours exhibited by students in terms of using (or acting on) the feedback they received? One instance of such noteworthy behaviour refers to the students who were provided with seemingly productive feedback, though failed to use while revising the initial version of the artefact they had produced.

3. What are the challenges associated with the implementation of the assessment method Written Feedback? What obstacles seem to impede its productive enactment and what are possible ways of addressing them?

Scope of the research design/Constraints to be satisfied:

This research design assumes the implementation of the assessment method Written feedback (cf. D4.7, p. 21-22). This means that the method that is implemented meets four minimal criteria:

1. As part of the activity sequence students should be producing certain artefacts associated with the competence promoted through the teaching intervention (e.g., an argument in the case of the argumentation competence, a model of a physical phenomenon in the case of the modelling competence or a design for an experimental design – which variable to alter, which variable to keep constant - in the case of the investigation competence). Each partner will be asked to describe the artefact that will be constructed and to demonstrate how that is linked to the relevant competence.

2. Teachers’ feedback will be focused on these artefacts. For instance in the case in which the artefact is a student constructed argument, the feedback could be focusing on the extent to which it contains what are considered to be the essential components such as the claim and the data supporting the claim along with a relevant justification. Feedback should be given to individual students.

3. Teacher’s attempt to interpret data from students’ artefacts and produce feedback will be supported through specially designed templates. These will be developed by the LWGs.

4. Upon receiving comments from the teacher, each student revises the relevant artefact, taking into account the feedback s/he received. In addition to just revising the artefact, students are also explicitly asked to briefly respond, in writing, to the feedback comments they received (e.g., what did the feedback mean to you? What did you learn from the feedback and how did you use it in your work?).

This research design also assumes the development and use of certain assessment tools, as follows.

  1. The template that will be used by the students to present the initial version of their artefact. This will be what the teachers’ will be focusing his/her feedback on.
  2. A template that will be used by the students to respond to the comments they received from their teacher.
  3. A template that will be used by the students to present/describe the revised version of their artefact, after receiving feedback comments by the teacher.

Notes:

1. These assessment tools will have to be developed by the individual LWGs (where applicable, partners are encouraged to adapt from the examples provided by WP5 – see last section of this document - and collaborate with each other). WP5 has provided examples meeting the minimal criteria for the following competences: investigation, and design in technology.

2. You will be asked to provide a translated version of the assessments tool that you will use.

Additionally this research design assumes that the following criteria are met:

1. The researchers in the LWG will conduct semi-structured follow-up interviews with students who exhibited a noteworthy behaviour (as described in the second analytical question) in terms of acting on the feedback they received.

2. The LWG is responsible to ensure that the teacher’s implementation is consistent with the plans of the LWG.

3. The LWG is responsible for supporting the process of collecting the required research data during and after the implementation.

4. The responsibility for the research data collection resides with the researchers of the LWG who are also anticipated to safeguard the inter-rater reliability of the data analysis.

Anticipated output of this research design

At the local level this research design will lead to case studies that will focus on the implementation of Written Feedback (as an assessment method) in a specific situation. This could be focused on documenting intricacies/patterns identified in that situation (e.g., how students responded in the cases in which they were provided with productive feedback? Also, it could focus on possible challenges encountered by the teachers (to what extent did the use of a particular tool supported teacher’s attempt to provide productive feedback to the students?)

Provided that this research design will be implemented by more than one partners it will be possible to also address questions associated with the comparison of the challenges across different situations.

Indicative types of artefacts associated with various competences:

  • Argumentation: the artefacts could be student constructed arguments and the feedback comments could be referring to the extent to which the arguments contain certain structural elements (e.g. based on Toulmin’s model).

  • Investigation: the artefacts could be the experimental designs proposed by the students for addressing a specific investigable question (e.g. identification of the variables to change or keep constant). The feedback comments could be focusing on the extent to which the design secures appropriate control of variables and is likely to address the relevant investigable question in a credible manner.

  • Modelling: the artefacts could be student-constructed models for specific physical phenomena and the focus of the feedback could be placed on the extent to which these models are characterized by representational, interpretive and predictive capability with respect to the phenomenon of interest.

  • Design in Technology: The artefact could be a design product/solution developed through the successful completion of the design process as a response to a specific technological problem. The feedback comments could be focusing on the ‘realization stage’ of testing the prototype by collecting, analysing, interpreting and representing data.

  • Problem solving in Mathematics: the artifact could be a students’ solution to a given mathematical problem solving task. The feedback comments in this case could be focusing on the mathematical correctness of the solution given.