Harmonisation of variables names prior to conducting statistical analyses with multiple datasets : an automated approach

Bosch-Capblanch X., . (2011) Harmonisation of variables names prior to conducting statistical analyses with multiple datasets : an automated approach. BMC medical informatics and decision making, Vol. 11, H. 1. p. 33.

Full text not available from this repository.

Official URL: http://edoc.unibas.ch/dok/A6002221

Downloads: Statistics Overview


ABSTRACT: BACKGROUND: Data requirements by governments, donors and the international community to measure health and development achievements have increased in the last decade. Datasets produced in surveys conducted in several countries and years are often combined to analyse time trends and geographical patterns of demographic and health related indicators. However, since not all datasets have the same structure, variables definitions and codes, they have to be harmonised prior to submitting them to the statistical analyses. Manually searching, renaming and recoding variables are extremely tedious and prone to errors tasks, overall when the number of datasets and variables are large. This article presents an automated approach to harmonise variables names across several datasets, which optimises the search of variables, minimises manual inputs and reduces the risk of error. RESULTS: Three consecutive algorithms are applied iteratively to search for each variable of interest for the analyses in all datasets. The first search (A) captures particular cases that could not be solved in an automated way in the search iterations; the second search (B) is run if search A produced no hits and identifies variables the labels of which contain certain key terms defined by the user. If this search produces no hits, a third one (C) is run to retrieve variables which have been identified in other surveys, as an illustration. For each variable of interest, the outputs of these engines can be (O1) a single best matching variable is found, (O2) more than one matching variable is found or (O3) not matching variables are found. Output O2 is solved by user judgement. Examples using four variables are presented showing that the searches have a 100% sensitivity and specificity after a second iteration. CONCLUSION: Efficient and tested automated algorithms should be used to support the harmonisation process needed to analyse multiple datasets. This is especially relevant when the numbers of datasets or variables to be included are large
Faculties and Departments:09 Associated Institutions > Swiss Tropical and Public Health Institute (Swiss TPH) > Department of Swiss Centre for International Health (SCIH) > Systems Strengthening and Health Promotion (Prytherch)
UniBasel Contributors:Bosch-Capblanch, Xavier
Item Type:Article, refereed
Article Subtype:Research Article
Publisher:BioMed Central
Note:Publication type according to Uni Basel Research Database: Journal article
Related URLs:
Identification Number:
Last Modified:11 Oct 2012 15:32
Deposited On:11 Oct 2012 15:30

Repository Staff Only: item control page