Enriching knowledge and enhancing care through health data

i~HD Data Quality Benchmarking Programme

 

 

 

Benchmark your dataset for a specific reuse purpose

 

We verify that your data is fit for purpose

We can perform a formal benchmarking of your dataset against quality criteria that are pre-defined by an (external) stakeholder. For example, we can evaluate whether your EHR data is fit for reuse in clinical trials, in which case the quality thresholds would be defined in partnership with the pharma industry. 

Gain confidence in health data for secondary use

 

 

    Workflow

     

    Preparation & planning: As a first step, a meeting takes place where we discuss your needs and questions regarding a data quality assessment, and how we may be of service to your organisation. Together, we review the dataset that has to be assessed, and we introduce our data quality assessment methodology and tools. We then discuss the pre-defined quality criteria to benchmark your dataset against, or we can offer guidance in constructing such quality criteria prior to starting the assessment. Based on this information, we draft a project proposal. Once we agree on a collaboration, practical arrangements are made and contracts are signed.

    Dataset generation: Once a partnership has been decided on, we schedule an onsite visit where all partners involved in the project meet. IT staff of your organisation is also consulted to start extraction of the required data from your database. If applicable, (clinical) decision rules (see below for more information), tailored to the data under investigation, are drafted by medical and statistical domain experts to evaluate data quality. We assess the first data set extract, based on which database extraction queries can be fine-tuned and a final data set can be extracted.

    Assessment: We validate the final data set and perform a data quality assessment against the pre-defined quality criteria. We provide graphical illustrations of the results, together with interpretations by medical and statistical experts. A follow-up meeting then takes place, where we discuss our preliminary findings and we can exchange insights on causative factors together with partners of your organisation.

    Outcome: We deliver a comprehensive report, detailing the methodology and results of the data quality assessment, which is also presented to your organisation. Based on the strengths and weaknesses that we could identify, we can propose recommendations and improvement strategies. After data quality assessment, successful datasets can be awarded a “Fit for purpose” certificate.

     

     

    A data quality assessment report

     

    Once an assessment has been completed, we deliver a comprehensive report of which a few example diagrams and tables are shown below. The radius diagram, for example, gives an overview of the dataset’s respective strengths and weaknesses across the dimensions that were selected as the most important for its data use purposes.

    Figure 1: Extracts from a sample i~HD data quality assessment: ICHOM heart failure outcomes data set

    To interpret results of the data quality assessment, some important interactions with hospital informatics and clinical experts are required, to discuss and try to explain unexpected patterns that may appear in the data. In addition, it is important to look at the EHR system and the data entry screens that are used to capture the data items being assessed. From this holistic assessment, we can identify systematic errors and start to work out the best strategies for an improvement programme.

     

    Improvement strategies

     

    Examples of improvement strategies

    • Workshops for clinical and administrative staff about the impact of poor data quality across the organization, with examples identified from an assessment performed earlier;
    • Education about how to better use the EHR system to improve the quality of the data they enter;
    • Modification of EHR system data entry screens (templates) to avoid data entry errors, such as making sure the right fields are mandatory, that pick lists really meet clinical and managerial needs, avoiding alternative ways of entering the same information, and introducing more data validation checks;
    • Data quality monitoring dashboards that give real time feedback to departmental managers about the quality of new data;
    • Employing a part time or full time data quality champion who educates, troubleshoots, coaches;
    • Regular feedback throughout the organization (e.g. in newsletters) about how high-quality data has enabled improved care or more successful research.

     

    Data quality dimensions

    Based on iterative discussions during multi-stakeholder consultation workshops, the Data Quality Expert Team has consolidated eight data quality dimensions that are most important for health data if this data is to be useful for patient care, for organisations’ internal quality management purposes, and for research. These eight dimensions are summarised below. Depending on the specific needs and questions of your organisation, a subset of these data quality dimensions can be selected for a data quality assessment.

     

    data quality dimensions

     

    Decision rules

    Together with statistical and clinical domain experts, decision rules are composed to evaluate data quality according to each selected dimension. Below, we provide some example decision rules.

    Basic decision rules:

    • All variables of date format have to comply to the specified format; e.g., check for errors of DD/MM/YYYY vs. MM/DD/YYYY;
    • Variables only have values within a specified range; e.g., height values have to be between 30cm and 270cm;
    • Date of admission should be before or equal to the date of discharge.

     

    Decision rules including clinical expertise:

    • Consistency of medication list: evaluate whether drug combinations are not dangerous and compare drug list to allergy list;
    • Temporal relationships: check whether there is a logical sequence of events within a care pathway;
    • Documentation of data sets that are supposed to be completed: examine whether assessment scales are complete and scores match their component parts;
    • Regularity of repeated events: evaluate missing events.

     

     

    Interested? Get in touch!

    Are you interested to learn more about data quality? Would you like more information regarding an assessment of your institution's health data? Contact our Data Quality Programme Manager: Hannelore.Aerts@i-hd.eu