Systematic reviews: Other sources

Reference lists of included studies are commonly searched at the end of the screening process. You may also include reference lists of other publications such as reviews, but you need to describe what you do and include it in the PRISMA flow chart.

PRISMA recording requirements (Page, 2021, Expanded Checklist):

• "If reference lists were examined, specify the types of references examined (e.g. references cited in study reports included in the systematic review, or references cited in systematic review reports on the same or similar topic)." 

Searching Google Scholar does not replace database searching for a systematic review. 

If you do decide to search Google Scholar as an additional source, treat it in a similar way to a database:

• Use the advanced search feature as demonstrated in the following video from the UTS Library
• Record your search strategy including any limits, the date you carried out your search and the number of results retrieved
• If you retrieve a lot of results choose a manageable number to screen eg the first 100 results, sorted by date (this forms part of your methods)
• Export your results for screening to EndNote (which has to be done individually) and then into Covidence, just as for other database results.
• Be particularly aware of quality, as the definition of "scholarly content" retrieved is unclear and constantly changing.

Background material - reference only

Depending on your topic, it may be appropriate to browse websites for grey literature such as association guidelines, government reports, statistics or narratives. If you are just using these sources for background material (rather than including them in the full review process) then you can just reference them.

Study reports to be reviewed - add to flow chart and write up search methodology

You may discover study results or lists of references on websites. If you include these with the studies you review, then they should be added to your search methodology and the PRISMA flow chart.

Consider quality

Be wary of the quality of reports and other information discovered through internet searches, as they are much more likely to include fake research and predatory publishers. 

PRISMA recording requirements (Page, 2021, Expanded Checklist):

• "Specify the date when each source (e.g. database, register, website, organisation) was last searched or consulted."
• "If websites, search engines or other online sources were browsed or searched, specify the name and URL of each source."

Most journals and conference proceedings will be indexed in the key bibliographic databases (Medline or Embase), so this only refers to additional searches, usually through publisher websites. 

Be aware of journal quality, as there are many predatory publishers.

PRISMA recording requirements (Page, 2021, Expanded Checklist):

• "Specify the date when each source (e.g. database, register, website, organisation) was last searched or consulted."
• "If journals or conference proceedings were consulted, specify of the names of each source, the dates covered and how they were searched (e.g. handsearching or browsing online)." 

PRISMA recording requirements (Page, 2021, Expanded Checklist):

• "If individuals were contacted to identify studies, specify the types of individuals contacted (e.g. authors of studies included in the review or researchers with expertise in the area)." 

PRISMA recording requirements (Page, 2021, Expanded Checklist):

• "If organisations or manufacturers were contacted to identify studies, specify the name of each source." 

The following video from University of Manitoba Libraries provides an interesting overview of backwards and forwards citation tracking, including both benefits and limits to its usefulness in the context of systematic reviews.

It is not an alternative to database searching for a systematic review, but may be helpful when initially looking for a set of relevant papers in the process of developing a search.

If you do want to use backwards and forwards citation tracking for a systematic review then we suggest limiting to a specific set of documents, such as the final included papers after screening. Without limits this process can quickly become unmanageable.

SVHM does not have subscriptions to Web of Science or Scopus, but you may have access through a university library. Google Scholar does not currently provide backwards citation tracking, and forwards citation tracking is not reliable.

PRISMA recording requirements (Page, 2021, Expanded Checklist):

• "If cited or citing reference searches (also called backward and forward citation searching) were conducted, specify the bibliographic details of the reports to which citation searching was applied, the citation index or platform used (e.g. Web of Science), and the date the citation searching was done."