Systematic Literature Review
Md Saifuddin Khalid, Technical University of Denmark
Analia Cicchinelli, Graz University of Technology, Austria
Antigoni Parmaxi, Cyprus University of Technology
Yagmur Cisem Yilmaz, Tallin University, Estonia
Introduction
This chapter present a summary of the methodology to conduct a systematic literature review. First, the difference between literature review and systematic literature review is presented. This is followed by a definition of systematic literature review and a glossary of relevant terms. Finally, the phases for conducting a systematic literature review are presented as a guide.
Below is an interactive widget with the learning goals of this chapter. This tool allows the reader to visualize not only the learning goals, but also his or her own estimation (in percent) of the achievement.
Literature review
A literature review is a common practice and a central part of academic research. It provides an overview of what is known about a particular topic and a critical evaluation of the existing published work in a selected research area. A good literature review is a critical discussion, displaying the writer’s knowledge of relevant theories and approaches and awareness of contrasting arguments. Through the literature review the researcher is able to identify gaps in the literature, the need to fill them with new research and to locate their original work whiting the existing literature.
Literature review vs Systematic Literature Review
Before focusing on the systematic review of the literature, it is considered relevant to present the differences between a literature review and a systematic literature review.
| Literature review | Systematic literature review | |
|---|---|---|
| Definition | Critical evaluation of the existing published work in a selected research area. | Type of literature review that is focused on a particular research question. |
| Goal | To review the existing literature, identify the research gap, place the research study in relation to other studies, to evaluate promising research methods, and to suggest further research. | To identify, review, and summarize the best available research on a specific research question. |
| Research Questions | Formed after writing the literature review and identifying the research gap. | Formed at the beginning of the systematic review. |
| Research Studies | Essential component of a research study and is done at the beginning of the study. | Not followed by a separate research study. |
Systematic Literature Review-Definition
Systematic literature review (also referred to as a systematic review) is a form of secondary study that uses a well-defined methodology to identify, analyse and interpret all available evidence related to a specific research question (Kitchenham & Charters, 2007).
Systematic reviews can be defined as “a review of existing research using explicit, accountable rigorous research methods” (Newman & Gough, 2020).
Glossary
- Primary study: an empirical study investigating a specific research question.
- Secondary study: a study that reviews all the primary studies with the aim of integrating/synthesizing evidence related to a specific research question.
- Systematic review protocol: a plan that describes the conduct of a proposed systematic literature review.
Systematic Literature Review-Process
This section first provides an overview of the phases (see also Fig 1: Xiao & Watson diagram (Xiao & Watson, 2017)) involved in conducting a systematic literature review and then focuses on each phase with a brief explanation and examples.
- Planning the SLR:
- Identify the need for a systematic review
- Formulate of a focused review question
- Consider scope and inclusion and exclusion criteria
- Conducting the SLR:
- Conduct a comprehensive, exhaustive search for primary studies
- Assess and screen the studies
- Extract data from the studies
- Reporting the SLR:
- Summarize and synthesize the study results
- Interpret the results to determine their applicability
- Report-writing
Planning the Systematic Literature Review
The role of the research questions
Specifying the research questions is the most important part of any systematic review. A clear, specific and answerable question is essential to a successful systematic review. A well formulated question will help to determine the protocol and search strategy and help to find relevant and valid information. In this way, the research question guide the whole process:
- The search process must identify primary studies that address the research questions.
- The data extraction process must extract the data items needed to answer the questions.
- The data analysis process must synthesize the data in such a way that the questions can be answered.
Inclusion-Exclusion Criteria
Study selection criteria are used to determine which studies are included in, or excluded from, a systematic review. Identify the criteria that will be used to determine which research studies will be included must be decided before starting the review. Characteristics which make a study eligible or ineligible to be included in your review:
- Date of publication
- Type of study design
- Language of publication
Having explicit exclusion criteria from the beginning allows conducting the screening process and optimize the scope of your research, providing the necessary merit to the research question. For the final product, there should be a section in the review dedicated to the characteristics of excluded studies.
Search process
The search process include the search strategy and a structured approach to conduct the search.
Search strategy
The first step is to identify the search terms that will be used to search the bibliographic databases. Keywords for the search should be derived from the research question(s), breaking down the research question into concept domains and extending them by synonyms, abbreviations, alternative spellings, and related terms in an iterative process.
Simple words (AND, OR, NOT or AND NOT) used as conjunctions to combine or exclude keywords in a search, leads to more focused and productive results. Using OR broaden the research by combining synonyms to appropriately cover a concept. This search will retrieve articles containing each term separately, as well as both terms together. Using AND narrow the search and is used to combine concepts. This search will retrieve articles containing both terms only.
Conducting the search
A structured approach to determine the source material for the review is recommended and include:
- Databases: A systematic search for literature should draw from multiple databases. The major contributions are likely to be in the leading journals. Identification of relevant articles, scanning a journal’s table of contents and also examine selected conference proceedings, especially those with a reputation for quality:
- Scopus.
- Web of Science.
- PubMed.
- ERIC.
- IEEE Xplore.
- ScienceDirect.
- Directory of Open Access Journals (DOAJ)
- JSTOR.
- Go backward: by reviewing the citations for the articles identified in step 1 to determine prior articles you should consider.
- Go forward: by using the Web of Science 3 (the electronic version of the Social Sciences Citation Index) to identify articles citing the key articles identified in the previous steps.
Screening procedure
Many experts suggest at least two reviewers work independently to appraise the studies, matching the established review inclusion and exclusion criteria. The two reviewers should resolve possible disagreement through discussion or by a third party. The list of excluded papers should be maintained for record keeping, reproducibility, and crosschecking.
Articles are screened against pre-defined criteria independently by at least two authors:
- Initially, they should screen the titles and abstracts.
- Then, the full text is screened for any articles which could not be judged as fulfilling (or not fulfilling) all inclusion criteria on the basis of the information in their titles and abstracts.
Assessing the quality of studies
It is a common practice for quality assessment to rank studies based on a checklist. Two or more researchers perform a parallel, independent quality assessment. All disagreements should be resolved through discussion or consultation with an independent arbitrator. The assessment should include an in-depth analysis of the logic from the data collection method, to the data analysis, results, and conclusions. One should then rely on high-quality studies to construct major arguments and research synthesis before moving on to the medium-quality studies.
Data extraction and synthesis
The process of data extraction will often involve coding. It is important to establish whether coding will be inductive or deductive (i.e., whether the coding will be based on the data or preexisting concepts) The way in which studies are coded will have a direct impact on the conclusions of the review. (Xiao & Watson, 2017).
Once the data extraction process is complete, the reviewer will organize the data. Synthesis usually involves some combination of charts, tables, and a textual description. The researcher organize the data according to the review type: a meta-analysis will present regression tables; a meta-summary will report effect and intensity sizes, and a framework synthesis will include a conceptual model.
Reporting finding
Clear reporting in systematic reviews is essential. It is important to provide readers with all information in order to understand the process from the formulation of the research question until the screening of manuscripts and the write-up of findings. With this in mind, an international group of experts to created PRISMA to provide in order for authors to ensure that their research is as useful for researchers and practitioners.
PRISMA statement and guidelines
The PRISMA statement comprises a checklist and a flow diagram. The checklist includes 27 items addressing the introduction, methods, results and discussion sections of a systematic review report. The items can be checked off the list as they are completed.
The flow diagram demonstrates the flow of information through the different phases of a systematic review. It maps out the number of records identified, included and excluded, and the reasons for exclusions. Different templates are available depending on the type of review (new or updated) and sources used to identify studies (PRISMA, 2021). There are simple tools that can be employed for creating a prisma diagram (see for example here: https://hollyhartman.shinyapps.io/PRISMAFlowDiagram/).
How to report a review?
When reporting outcomes, the authors need to present a concise summary of data for each research question. At this stage, different types of visualizations can be employed (e.g. graphs, bar graphs, charts maps, etc.). Authors need to make sure that their visual representation accurately depicts the essence of their findings.
Reflection question: These two questions will help you reflect on what you have learned and think about how to apply it to your PhD project:
- What did you learn today that you didn’t know before?
- On what topic/aspect of your PhD project do you think it would be interesting to do a systematic literature review?