Preserving Transactional Data

TitlePreserving Transactional Data
Publication TypeReport
Year of Publication2016
AuthorsThompson, S. Day
Date Published2016/05//

Gap Area Study Type:

High-level Gap Areas:

Investigated "the requirements for preserving transactional data and the accompanying challenges facing companies and institutions that aim to re-use these data for analysis or research." The report was commissioned to support the long-term preservation issues faced by UK ESRC-funded centres (Big Data Network Support (BDNS), which includes the Administrative Data Research Network (ADRN) and research centres that form the Business and Local Government Data initiative).
Examined three use cases: Energy Demand Research Project: Early Smart Meter Trials at the UK Data Service (UKDS); Output Area Classification Data at the Consumer Data Research Centre (CDRC); Higher Education Data at the Administrative Data Research Network (ADRN)
"The term ‘transactional’ will be used to refer to data that result from single, logical interactions with a database and the ACID properties (Atomicity, Consistency, Isolation, Durability) that support reliable records of interactions. In some contexts, these data could be fiscal in nature, deriving from business ‘transactions’ such as at an ATM or through a web service such as Amazon. This report, however, considers transactional data more broadly, addressing any data generated through interactions with a database system. Administrative data, for instance, are one important form of transactional data collected primarily for operational purposes, not for research. Examples of administrative data include information collected by government departments and other organizations when delivering a service (e.g. tax, health, or welfare) and can entail significant ethical and legal challenges. Transactional data, whether created by interactions between government database systems and citizens or by automatic sensors or machines, hold potential for future developments in consumer analytics and in academic research. Ultimately, however, these data will only lead to new discoveries and insights if they are effectively curated and preserved to ensure appropriate reproducibility."