HTAi 2019: Best Practices for Leveraging Artificial Intelligence in the Systematic Review Process
August 13, 2019
Insights

During the 2019 annual HTAi meeting in Cologne, Germany, Medlior’s Kelly Larkin-Kaiser along with Peter O’Blenis and Jennifer Tetzlaff from Evidence Partners hosted a workshop titled: “Best Practices for Leveraging Artificial Intelligence in the Systematic Review Process”

The workshop focused on developing practical knowledge and skills for Artificial Intelligence (AI) and Natural Language Processing (NLP) in the context of systematic literature reviews (SLRs), specifically related to citation screening and data extraction activities.

The workshop kicked-off by first highlighting common challenges that researchers experience when conducting SLRs, followed by a discussion of the opportunities for AI to mitigate such difficulties. The workshop attendees identified some of the key challenges from their perspectives:

[table id=1 /]

Participants were keen to understand how AI could be leveraged to ease their perceived difficulties with the following components of the SLR process:

  • Literature search construction and study identification
  • Citation Screening
  • Data extraction (data collection)
  • Evaluating the risk of bias of relevant studies

Participants were then provided with a theoretical baseline for AI, NLP and classifiers in the literature review context. Theory was followed by a summary of the available research on using AI for literature reviews, focusing on the practical implications of the technology.

Participants walked away from the workshop with an understanding of where AI fits in the systematic review process by understanding the current limitations of AI and NLP in literature reviews. Furthermore, participants had the opportunity to practice using AI in different aspects of the review process and explored the various parameters for its use.

Using a sample project on the DistillerSR platform, participants implemented different applications of AI technology, for the literature review process, such as:

  • Using different parameters to train an AI to perform screening
  • Using an AI to rank references in order of likeliness for inclusion
  • Using an AI to assist with screening
  • Using an AI review screened references for inaccurate exclusions

Thank you again to all the attendees who participated in this workshop – we enjoyed the opportunity!

Interested in learning more? Feel free to email Kelly Larkin-Kaiser or contact Medlior to see how we could help your next project.