An introduction to Design-based Research in TEL
Lorena Sousa, University of Aveiro, Portugal
Luís Pedro, University of Aveiro, Portugal
Carlos Santos, University of Aveiro, Portugal
1. Introduction
The objective of this chapter is to:
- … introduce Design-Based Research (DBR) as a research method, including some definitions, other terminologies, and the structure of the research question;
- … present different models for conducting DBR, their phases (analysis, development and evaluation) and characteristics;
- … suggest a repertoire of activities that may be developed in each phase.
This chapter is organized in three main parts. In the first part, some definitions and other DBR terminologies that might be used as synonyms are presented. Also, there will be an introduction to the structure of the research question in order to help you elaborate yours. In the second part, some models for conducting DBR are introduced, as well as their phases - which are analysis, development and evaluation -, and characteristics. In the third part, a repertoire of activities that may be developed in each phase is suggested. During the chapter, you will have access to other references and be informed about further reading recommendations.
2. DBR Definitions
Two possible purposes of DBR can be identified, and depending on the purpose of the research they may be distinguished between development studies and validation studies (Plomp & Nieveen, 2013):
- Development studies: to develop a research-based intervention as a solution for complex problems in educational practice.
- Validation studies: to develop or validate theories about processes of learning and teaching.
However, regardless of the purpose, (Plomp & Nieveen, 2013) affirms that DBR “encompasses the systematic study of designing, developing and evaluating educational interventions”. These interventions can be programs, learning processes, learning environments, teaching-learning materials, products or systems.
The Design-Based Research Collective (Baumgartner et al., 2003) states that “educational research is often divorced from the problems and issues of everyday practice – a split that resulted in a credibility gap and creates a need for new research approaches that speak directly to problems of practice and that lead to the development of ‘usable knowledge’.”
Van den Akker (Van den Akker et al., 2006) writes “that ‘traditional’ research approaches such as experiments, surveys, correlational analyses, with their emphasis on description hardly provide prescriptions that are useful for design and development problems in education”.
Bakker (Bakker, 2018) says that “most educational research describes or evaluates education as it currently is. Some educational research analyzes education as it was. Design research, however, is about education as it could be or even as it should be.” In other words, traditional research approaches in the education field usually provide a description of the context, and rarely emphasize a plan or suggestion that is useful to tackle real problems in education.
Reeves (Reeves, 2006) argues that “there is an urgent need for a better approach to educational technology research”. Instead of undertaking comparison studies, analyzing if method A is better than method B, educational technologists should conduct research aimed at developing optimal solutions to practical problems in learning environments.
2.1 Terminology
There are a lot of terms related to DBR. Some of them are synonyms, others may vary according to their goals and characteristics. Examples are Design Research in Education or Educational Design Research. Education and Educational were incorporated in the term to denote the area in question and avoid confusion with DBR from other fields.
In addition to these examples, other members of this family are (Plomp & Nieveen, 2013), (Reeves, 2006):
- design experiments
- design studies
- development or developmental research
- engineering research
- formative research
- participatory action research
- design-based implementation research
Reading recommendations:
If you want to have a general and introductory idea about DBR, check the links below:
- 7 things you should know about Educational Design Research, by EDUCAUSE (2012) link
- Design-based research, by Dr. Michele Jacobsen (2014) link
3. Research functions
Plomp (Plomp & Nieveen, 2013) affirms that, in general, various research functions can be identified and distinguished from each other, with each reflecting certain types of research questions:
- to describe: what is the achievement of Chinese grade 8 pupils in mathematics?; what barriers to students’ experience in the learning of mathematical modelling?
- to compare: what are the differences and similarities between the Chinese and the Netherlands curriculum for primary education?; what is the achievement in mathematics of Chinese grade 8 pupils as compared to that in certain other countries?
- to evaluate: how well does a program function in terms of competences of graduates?; what are the strengths and weaknesses of a certain approach?
- to explain or to predict: what are the causes of poor performance in mathematics (i.e. in search of a ‘theory’ predicting a phenomenon when certain conditions or characteristics are met)?
- to design and develop: what are the characteristics of an effective teaching and learning strategy aimed at acquiring certain learning outcomes?
If you are writing your research project, you must begin with a research problem or a research question. There is a logical sequence of development, let’s take a look at it:
problem → research question → (primary) research function → choice of research design
We already saw examples of research questions and their research function. So now you are going to see examples of research designs.
4. Research designs
Examples of research designs and their possible research functions are:
- Survey: to describe, to compare, to evaluate
- Case studies: to describe, to compare, to explain
- Experiments: to explain, to compare
- Action research: to design/develop a solution to a practical problem
- Ethnography: to describe, to explain
- Correlational research: to describe, to compare
- Evaluation research: to determine the effectiveness of a program
Textbooks on research methodology usually do not present and discuss design research, probably due to its recently emerging status and the fact that relatively small groups across several disciplines have been responsible for its development.
- Design research: to design and develop an intervention (such as programs, teaching-learning strategies and materials, products and systems) as a solution to a complex educational problem as well as to advance our knowledge about the characteristics of these interventions and the processes to design and develop them, or alternatively to design and develop educational interventions (about for example, learning processes, learning environments and the like) with the purpose to develop or validate theories.
Reflective activity: Think of a general problem faced by practitioners in the Education and Technology-enhanced Learning field. Try to formulate a research question to this problem. Write your research question and contextualize the problem briefly. What is its research function? What is its research design?
5. The overall research question in DBR
Here we have some examples of research question structures that might help you when elaborating yours, in case you are going to conduct a research using DBR as the research method.
- What are the characteristics of an intervention X for the purpose/outcome Y in context Z? (Plomp & Nieveen, 2013)
- What are the characteristics of an effective teaching and learning strategy aimed at acquiring certain learning outcomes in a certain context?
Examples of research questions using the structure above are:
- What are the characteristics of an intervention for promoting academic research writing which will best support graduates in education in the proposal stage of their research? (Dowse & Howie, 2013)
- What are the characteristics of microscale chemistry curriculum materials so that they contribute to the implementation of effective practical work in chemistry teaching in Tanzania schools? (Mafumiko et al., 2013)
- What are the characteristics of a digital resource for teaching and learning mathematics in secondary schools?
- What are the characteristics of an online course aimed at enhancing second language acquisition in higher education?
Reflective activity: Now, if you are thinking of conducting a Design-Based Research, try to formulate your research question using the structure presented above.
6. Cycles and phases
Whatever the purpose of DBR - development studies (to develop research-based solutions for complex problems in educational practice) or validation studies (to develop or validate theories about processes of learning and teaching) -, the research process always incorporates systematic educational design processes, as illustrated below (Plomp & Nieveen, 2013):
Like all systematic educational and instructional design processes - it is cyclical or iterative. It incorporates cycles of analysis, design, evaluation and revision activities until a balance between the intended ideas and realization has been achieved. This process can be illustrated in several ways, depending on how authors have visualized it.
Let’s see another example.
Reeves’ model
Reeves (Reeves, 2006) presented a model that highlights four main phases of DBR:
The arrows represent the iterative characteristic of the process.
McKenney’s model
This model below was the research design used by Susan McKenney in her thesis (Mckenney, 2001).
It has three main phases: needs & context analysis; design, development & formative evaluation; and semi-summative evaluation.
Each phase consisted of multiple cycles of activities:
In the first micro cycle, for example, literature review and concept validation were conducted; in the second micro cycle, site visits were carried out; in the second phase, there were four micro cycles of prototype development; and then a final evaluation and a query were conducted in the third phase, representing one micro cycle each.
In total, eight micro cycles were conducted, two in the first phase, four in the second, and two in the third one.
The number of participants varied in each cycle. In the first cycle there were about 20 participants; in the second, about 50; in the third, there were almost 60, and so on.
According to the scale, each cycle lasted about six months.
McKenney and Reeves’ model
Based on their survey and analysis of existing models and frameworks for DBR, McKenney and Reeves (McKenney & Reeves, 2012) created the generic model for conducting design research in education:
This model shows the core elements of a flexible process that features the three main stages: analysis/exploration, design/construction, and evaluation/reflection.
The three stages take place in interaction with implementation and spread, and this model provides the dual outputs of intervention and theoretical understanding.
Summary
Authors may vary in the details of how they portray DBR, but they all agree that DBR consists of three main phases: preliminary research, development or prototyping phase, and assessment phase (Plomp & Nieveen, 2013).
- Preliminary research consists of needs and context analysis, review of the literature and of projects addressing questions similar to the ones in your study, and development of a conceptual framework.
- The development or prototyping phase consists of an iterative process where prototypes are going to be tried out and revised on the basis of formative evaluations. It aims at improving and refining the intervention.
- In the assessment phase, a semi-summative evaluation is carried out to conclude whether the solution meets the predetermined specifications and whether it is effective. It often results in recommendations for improving the intervention, that is why this evaluation is called semi-summative.
- Throughout all these activities, the researcher conducts a systematic reflection and documentation to produce the theories as the scientific result of the research, besides the intervention that is more practical.
Reading recommendations:
For a deeper discussion of the DBR cycles and phases, check the references.
We also recommend the book Conducting Educational Design Research, by Susan McKenney and Thomas C. Reeves (2012), chapter 3: Toward a generic model for educational design research. (McKenney & Reeves, 2012) Another recommendation is the book Educational Design Research, chapter 4. (Reeves, 2006)
7. Main activities in each phase
There are no one-size-fits-all steps for tackling different challenges in the context of DBR. There are, however, processes and activities which are often useful. We are going to briefly present to you a repertoire so that you can select and use the most fitting approaches for your study.
| Analysis/exploration | Design/construction | Evaluation/reflection |
|---|---|---|
| preliminary field-based investigation; context analysis and needs assessment; literature review; understanding the problem; professional meetings and networking; site visits | exploring solutions; generating, considering and checking ideas; description of the skeleton design to detailed specifications; creating prototypes; revising prototypes | select strategies and determine methods; prepare the instruments to collect data; collect and analyse data; report the study; organic or structured reflection |
Reading recommendation:
To deepen your knowledge on the main activities and outputs of each phase, please check Part II Core processes from the book Conducting Educational Design Research, by McKenney and Reeves (McKenney & Reeves, 2012). This part of the book is divided into four chapters:
- Chapter 4: Analysis and exploration
- Chapter 5: Design and construction
- Chapter 6: Evaluation and reflection
- Chapter 7: Implementation and spread
8. DBR characteristics
DBR authors also agree on a number of characteristics represented in this type of research (van den Akker, 1999), (Van den Akker et al., 2006), (Kelly, 2012), (Nieveen, 1999), (Plomp & Nieveen, 2013):
- iterative: the research process incorporates cycles of analysis, design, development, evaluation, and revision, as demonstrated in the models presented before.
- interventionist: the research aims at designing an intervention in a real world setting in order to positively impact practice. These interventions include educational products, processes, programs or policies, for example. Its intention is to make a real change on the ground.
- collaborative: the research involves active participation and collaboration among researchers and practitioners in the various stages and activities of the research. The intervention needs to be relevant and practical for the educational context, and this collaboration increases the probability for a successful implementation.
- process oriented: the focus is on understanding and improving interventions. The products of DBR are shaped by the participants, literature, and especially field-testing. The interventions are adjusted based on the empirical data, which are collected in real world settings during the process.
- utility oriented: the quality of a design is measured by its practical use in real contexts to check if the intervention is really useful.
- theory oriented: scientific understanding is used to frame not only the research, but also to shape the design of the intervention / the solution to the real problem. The results also contribute to theoretical understanding and theory building.
Generic Design Research Model
In his Generic Design Research Model, Wademan (Wademan, 2005) pictures the features and characteristics of DBR. His model illustrates that the successive approximation of practical products or results (referred to as ‘interventions’) is working side by side with the successive approximation of theory (referred to as ‘design principles’).
Reading recommendations:
For a deeper discussion of the DBR characteristics, check the references.
We also recommend the book Conducting Educational Design Research, by Susan McKenney and Thomas C. Reeves (2012), chapter 1: About educational design research. (McKenney & Reeves, 2012)
9. Further readings
9.1 Introductory resources
- Design-based research (EduTech Wiki) link
- Design-Based Research (University of Calgary, by Michele Jacobsen) link
- Design-Based Research: An Emerging Paradigm for Educational Inquiry (The Design-Based Research Collective) link
9.2 Research articles
- Design-Based Research: An Emerging Paradigm for Educational Inquiry (Baumgartner, E., Bell, P., Brophy, S., (…), Sandoval, W., Tabak, I., 2003) link
- Design-Based Research: Putting a Stake in the Ground (Barab, S., Squire, K., 2004) link
- Design-based research and technology-enhanced learning environments (Wang, F., Hannafin, M.J., 2005) link
- Design-based research: A decade of progress in education research? (Anderson, T., Shattuck, J., 2012) link
- Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning (Dunleavy, M., Dede, C., Mitchell, R., 2009) link
9.3 Scholarly reading
- Design-Based Research: Putting a Stake in the Ground link, from Sasha Barab of Indiana University and Kurt Squire of the University of Wisconsin
- Design-Based Research and Technology-Enhanced Learning Environments link, from Feng Wang and Michael Hannafin of the University of Georgia
- Design-Based Research Methods for Studying Learning in Context: Introduction link, from William Sandoval of the University of California and Philip Bell of the University of Washington
- On the theoretical breadth of design-based research in education link, from Philip Bell of the University of Washington
- Using a Design-Based Research Study to Identify Principles for Training Instructors to Teach Online link, from Julie Shattuck of Frederick Community College and Terry Anderson of Athabasca University
- Publishing and perishing: The critical importance of educational design research link, from Thomas Reeves of the University of Georgia, Susan McKenney of the University of Twente, and Jan Herrington of Murdoch University
- Design-Based Research and Educational Technology: Rethinking Technology and the Research Agenda link, from Tel Amiel of the State University of Campinas and Thomas Reeves of the University of Georgia
- The Practice of Design-Based Research: Uncovering the Interplay Between Design, Research, and the Real-World Context link, from Diana Joseph of the University of Chicago
- Research Approaches for Innovation and Change link, from Methodological Choice and Design (Part II)
- Design research: A socially responsible approach to instructional technology research in higher education link, from Thomas Reeves of the University of Georgia, Jan Herrington of the University of Willongong, and Ron Oliver of Edith Cowan University
9.4 Practical examples
- Design-based Research Strategies for Studying Situated Learning in a Multi-user Virtual Environment link, from Chris Dede, Brian Nelson, Diane Ketelhut, Jody Clarke, and Cassie Bowman of Harvard University
- Design-based Research Strategies for Developing a Scientific Inquiry Curriculum in a Multi-User Virtual Environment link, from Brian Nelson, Diane Ketelhut, Jody Clarke, Cassie Bowman, and Chris Dede of Harvard University
- Combining design-based research and action research to test management solutions link, from Daniel Andriessen of Jogeschool Utrecht
- Creating context: Design-based research in creating and understanding CSCL link, from Christopher Hoadley of Stanford Research Institute
- Design-based research: case of a teaching sequence on mechanics link, from Andree Tiberghien, Jacques Vince, and Pierre Gaidioz
- An introduction to design-based research with an example from statistics education link, from Arthur Bakker and Dolly van Eerde of Utrecht University
- A journey through a Design-based Research Project link, from Wayne Cotton of the University of Sydney, and Lori Lockyer and Gwyn Brickell of the University of Wollongong
- Design-Based Research in Science Education: One Step Towards Methodology link, from Kalle Juuti and Jari Lavonen of the University of Helsinki
- Design based research to create instructional design templates for learning objects link, from Sameer Sahsasrabudhe, Sahana Murthy, and Sridhar Iyer of Indian Institute of Technology-Bombay