Process Map
The following seven steps will be carried out (1) setting the research question, (2) searching the literature (3) sampling of quantitative and qualitative data (4) determination of potential impact (5) thematic synthesis, (6) developing a consensus and (7) producing the final paper via expert peer review including solutions. (figure 1)
Defining the Research Question
Using JLA principles, subjects, stakeholders and researchers will work together to agree which, among the uncertainties, matter most and deserve priority attention. Research on the effects of interventions often overlooks the shared interests of people most affected by them. As a result, questions that they all consider important are not addressed and many areas of potentially important research are therefore neglected. Even when researchers address questions of importance to subjects and scientists, they often fail to provide answers that are useful in practice. Hence using the same principles, we will ensure that the research questions provide the evidence needed to develop impactful solutions which will address the persistent mismatch which exists between the performance of medical professionals irrespective of their academic potential, motivation or effort.
Literature Review
Preferred reporting items for systematic review and meta-analysis protocol (PRISMA P) guidelines (12) will inform the protocol for the current review with divergences from the guidelines implemented to meet the specific needs of the review. The main focus of the review will be qualitative, however, quantitative data may emerge during data extraction and will be incorporated. Such data may include demographics and sample sizes from the studies and a tabulation of the frequency of products from the studies, which will be included in the synthesis. Our approach will synthesise both qualitative and quantitative data, as a critical interpretive synthesis (13), utilising purposive sampling rather than the traditional inclusion criteria to determine higher-order messages rather than produce an integrative review (14) of research in DA.
A purposive structured search will be carried out in an exhaustive manner to ensure that all relevant materials are collected. In the context of the current review, materials include conference presentations, study protocols, published peer-reviewed papers/abstracts, unpublished manuscripts, internal symposia, tweets/LinkedIn/ ResearchGate/Facebook information, workshops/ masterclasses, documents related to archived datasets, newsletters and feedback to participants.
An infographic would be designed by the research team as a means to launch each theme of the project to all the relevant stakeholders, international expert community and invited theme leaders from each section of the project to send the authors their dissemination materials.
The most recent annual reports for organisations will be screened as a means to gather citations of all reported dissemination activities including published and in preparation peer-reviewed papers and abstracts. Any posters or oral presentations presented at national and international conferences, workshops or seminars that are referred to in the annual report will be recorded and sought from the researchers.
Traditional literature search methodology using differential attainment, outcomes and names of all researchers will be used as the search terms using ScienceDirect, PsycInfo, CINAHL and PubMed. The researchers in the Alliance will also be contacted to provide unpublished manuscripts, abstracts, and theses (where applicable) that arose from aligned projects. The Twitter pages, ResearchGate and LinkedIn profiles of all the researchers will be collected and recorded and their accounts searched for any discussion or comments related to the DA projects. Where necessary, freedom of information (FOI) requests will be made to all educational, regulatory or academic organisations responsible for designing, implementing careers, delivering assessments, managing curricula, and employers relevant to the journey of medical professionals.
A database of all dissemination materials and products will be created once all items have been gathered. Materials collected during sampling will be recorded using a Microsoft Excel database.
Sampling
No eligibility criteria for the studies will be applied since all data production is considered relevant. Given the exclusive nature of the search to DA projects, quality appraisal techniques (15) will be applied to the materials to ensure rigour in the selection of materials and products collected in the search stage.
Determination of Impact
All documents and data generated through the search will be categorised on two levels (e.g., high or low) with greater weight assigned to those materials identified as high. The features of this categorisation will include for instance the level of detail of content contained in the product in relation to DA or the type of content included in the materials (e.g., results of a study). Decisions regarding categorisation will be dependent on the content of the materials collected and will be made as the appraisal process is on-going and will be discussed by the authors with the thematic leads and the relevant members of the expert panel throughout the review process. Materials may be eliminated from the review during this stage if they are deemed to not be relevant to the review.
Thematic synthesis
The data that emerge from data extraction will be subjected to thematic synthesis. (16) Similar to analysis of primary qualitative datasets, thematic synthesis involves the systematic coding of data and generating of descriptive and analytical themes. It is an inductive approach which is critical given the aim to generate higher-order themes and key messages from the projects. It is a three stage process (17) which begins with line-by-line coding of text where findings from the materials collected will be entered word-for-word into the database and each line of text will be coded according to its meaning and content. Following this step is the development of descriptive themes which involves translating the concepts from one study to another and a hierarchical structure will be created by grouping the codes based on similarities and differences between the codes. Finally, the generation of analytical themes that go beyond the content of the original articles is a critical stage where descriptive themes are used to determine the key messages. Themes and messages will be reviewed independently to consider implications and then discussed as a group to allow for the emergence of more abstract messages and themes that go beyond the content in the original materials.
Scoping Document
The result of the literature review, categorisation and determination of potential impact will result in a scoping document which will provide both a broad overview of the current state of art, sampling of relevant evidence (quantitative and qualitative) and extent of impact to stakeholders. The authors working with section leaders will then generate a draft list of potential solutions, recommendations or a gap analysis. This scoping document will then be presented to all stakeholders (grassroots to organisations) for deliberation.
Developing a Consensus
The next stage will be a process of deliberation through a series of invited roundtables, thematic workshops and focus groups where the evidence presented in the scoping document and the draft recommendations/ solutions will be subjected to a defined process of rigour by independent facilitation and a consensus document produced.
Before each roundtable or workshop discussion, the section leads will present a defined set of issues to discuss based on thematic synthesis to the participants prior to the session. At the session, there will be a predefined amount of discussion time allocated for each of the agreed themes.
The discussions will be solution focussed and the outcome will be expected in the format of (a) well defined action, (b) recommendation, (c) policy or (d) area for further research/ evidence gathering/ pilot project.
All discussion and deliberations for each of the roundtable, workshops or focus groups will be transcribed, thematically analysed, synthesised and incorporated into the pre-final document.
Peer review
The final stage of the process will involve a peer review of the pre-final document by our invited expert panel, and comments integrated to produce the final consensus paper.