Research Design for the Structured Assessment Dialogue – University of Copenhagen

Forward this page to a friend Resize Print Bookmark and Share

ASSIST-ME > Researchers > Research Design for th...

Research Design for the Structured Assessment Dialogue

Focus of the specific research design: Exploration of the facility of structured assessment dialogue to serve as an appropriate context for formative assessment.

Specific Research Questions Associated with the research design

• To what extent can structured assessment dialogue, along with a specially designed assessment tool provide teachers with productive information so as to diagnose students’ needs and level of attainment of a selected competence and provide feedback to the students on that basis?

• What are the challenges and opportunities for using structured assessment dialogue, along with a specially designed assessment tool, as a formative assessment method for promoting students’ attainment of a selected competence?

Corresponding project research goal

What systemic support measures and what tools do teachers need in order to integrate formative assessment of student learning in their classroom practice?

Illustration of the Research Design

Rationale: The teaching plan includes a planned classroom event involving structured assessment dialogue. The teacher uses that as a context for collecting information on student attainment of the targeted competence, for interpreting that information and for offering feedback to the students. The structured assessment dialogue will be recorded for future analysis. In addition, a specially designed assessment tool will be used during the structured assessment dialogue. This assessment tool is intended to focus the teacher’s attention on the crucial aspects of the dialogue, to capture the core ideas associated with the targeted competence and to facilitate reflective feedback to the students. After the enactment, the teacher writes a report on the enactment along with his/her immediate reflections on the viability of the enactment (a template for these reports will be provided by WP5).

As an optional source of data, a partner can interview the teacher by using the stimulated recall interview approach with a focus on exploring selected excerpts of the classroom recording.

WP5 will provide a data-coding tool that will be used at the partner level in order to analyze the collected data.

The analysis will focus on the following analytical questions:

1. To what extent do dialogue episodes reveal information on students’ level of attainment of the various dimensions of the targeted competence?

a. To what extent was this information used to provide students with support and formative feedback during and after the structured dialogue?

2. What is the correspondence between the filled-in assessment tools and what actually occurred during the dialogue?

a. Was there information made available during the dialogue that was not represented in the filled in assessment tool?

b. What are the differences and similarities between the dynamic feedback during the structured assessment dialogue and the feedback that emerges from the use of the assessment tool and the associate teacher reflection?

3. What challenges and opportunities does the teacher identify for using structured assessment dialogue along with a specially designed assessment tool as a formative assessment method for promoting students’ attainment of the learning objective?

Scope of the research design/Constraints to be satisfied:

This research design assumes the implementation of the structured assessment dialogue method. This means that the method that is implemented meets two minimal criteria (cf. D4.7, p. 26-7):

1. Before the implementation, some specific rules for the dialogue should be described in detail. These rules formulate how the dialogue process should be structured. Students must be aware of these rules.

2. During the dialogue, students undertake and alternate between different roles – e.g. active presenters, active feedback givers, and active listeners. These roles should be described explicitly.

Note: You will be asked to describe how you intend to reach these criteria in the teaching scheme, and you will be asked to provide a translated version of the rules and the structure of dialogue.

This research design also assumes the existence of a specially designed assessment tool that helps the teacher collect information about the students’ level of attainment of the given competence in question and also offer feedback. The recording of information could be done during the structured assessment dialogue or immediately afterwards. These assessment tools will have to be developed by the individual LWGs ((where applicable, partners are encouraged to adapt from the examples provided by WP5 – see last section of this document - and collaborate with each other)). There are three minimal criteria for these assessment tools:

1. There needs to be a specific learning objective (and an associated progression of levels of attainment) that clearly corresponds to the competence in question.

2. The assessment tool must allow the teacher to gather information that is demonstrably relevant to the assessment of students’ attainment of the learning goal. For example, the tool could be a richly described rubric.

3. The assessment tool must facilitate formative feedback to the students (after reflection) by including a section where the teacher interprets the recorded information and returns written comments to the students (individual or in groups, or for whole class).

Note: You will be asked to describe how you intend to satisfy these criteria in the teaching scheme, and you will be asked to provide a translated version of the assessment tool that was ultimately implemented.

WP5 has provided examples meeting the minimal criteria for the competence of argumentation.

Additionally this research design assumes that the following criteria are met:

1. The LWG is responsible for ensuring that the teacher’ implementation is consistent with the plans of the LWG.

2. The LWG is responsible for supporting the process of collecting the required research data during and after the implementation.

3. The responsibility for the research data collection resides with the researchers of the LWG who are also anticipated to safeguard the inter-rater reliability of the data analysis.

Anticipated output of this research design

At the local level this research design will lead to case studies that will identify and document the merits associated with “structured assessment dialogue” as a formative assessment method. It will also serve to document challenges associated with the implementation of this assessment method in inquiry-based Teaching/Learning Sequences for certain competences.

The research output that is produced on the partner-level will be pooled together with the research outputs from other partners using this research design. At the project-level there will be a meta-analysis of this pooled research output.

Given that it is predicted that more than one partner implements this research design, it will be possible in the meta-analysis to address questions associated with the challenges/intricacies associated with this assessment method in different contexts (or with different competences). For example:

• What is the effect of specific types of structures of assessment dialogue on the quality of information about students’ attainment of given learning objectives?