What is data management? Sensible data handling

Data management is a relatively new field within information technology, emerging in response to growing connectivity and the ever-increasing flood of data in modern businesses. The topic has also become a focus of research. Data science, in particular, is a young discipline dedicated to the sensible storage and integration of data, as well as efficient methods for searching through large datasets.

Definition of data management

Data management defines clear requirements for handling digital information and refers to a process rather than isolated actions. Data should be systematically organised right from the point of collection and entry. Data minimisation and quality are key aspects to keep in mind. In addition to safeguarding the content, the data must remain practical and usable for its intended purpose. Another important consideration is determining which data should be archived and for how long. Any data that is no longer needed must be easy to locate and securely deleted without delay.

Types of data

When developing a data management plan, begin by identifying the different types of data your business produces. Categorising this information allows you to work in a structured way and ensures that no important area is overlooked.

• Personal data: Information relating to identifiable individuals, such as names, phone numbers, and addresses. It can also include behavioural data like shopping habits. This may apply to customer data, employee records, or third-party data — all of which require special protection under the UK GDPR.
• Protected company data: Internal information such as financial accounts, tax records, and trade secrets. Define precisely what falls into this category to ensure consistent handling.
• Secondary data: Data created for a purpose other than its original intent — for example, CCTV footage installed for security might also capture customer vehicle registrations, or server log files storing visitors’ IP addresses.
• Public data: Data intentionally published, such as website content, brochures, and marketing materials. Copyright law and intellectual property protections apply — in the UK, logos and designs can be protected under the Registered Designs Act 1949.

Tasks and implementation of data management

The role of data management is to integrate all processes from collection to storage or deletion, focusing on efficiency. This considers the entire Data Life Management (DLM) cycle.

Data collection

Data processing begins with collection. Two principles are crucial:

1. Data minimisation – Only gather what is necessary. This principle is embedded in the UK GDPR and Data Protection Act 2018. You must have a lawful basis for processing data, such as consent, contractual necessity, or legal obligation.
2. Data quality – Capture data accurately to avoid rework or errors. Store it in the correct format for later use to prevent loss or corruption during conversions.

Data storage

Choosing the storage location and format is vital. Combining local storage with secure cloud storageis often the most resilient option for critical data.

For large datasets, databases are generally preferred. If industry-specific software is used (e.g. accounting or inventory management), ensure compatibility with export formats for HMRC audits.

Data security

Data must be protected from loss, unauthorised changes, and unauthorised access. In the UK, the National Cyber Security Centre (NCSC) offers guidance on protecting against cyber threats. Certifications like ISO 27001 are widely recognised and can help meet compliance requirements.

Potential threats include:

• Hardware damage (fire, flooding, power surges)
• Accidental deletion or misconfiguration
• Malware or ransomware attacks
• Software bugs or vulnerabilities
• Physical theft of devices

To address the various risks, solutions include not only software-based protection mechanisms but also organisational measures such as fire and burglar alarm systems.

Keep these principles in mind:

• Regular updates: Consider the pros and cons of automated versus manual updates. Automated updates have the advantage of running consistently without being forgotten, while manual updates reduce the risk of faulty installations.
• Secure passwords: There are various strategies to consider here. One effective approach is to set guidelines requiring employees to use complex passwords. Regular password changes are also recommended. However, placing too much emphasis on complexity and frequent changes can backfire, leading employees to write passwords down and leave them at their workspace.
• Antivirus/Firewall: Current antivirus protection is essential for any IT system. Depending on the complexity of the network, it may be advisable to use a firewall and, if needed, an Intrusion Detection System (IDS).
• Backup strategy: One of the most important aspects is undoubtedly the proper backup strategy. Relevant data should be thoroughly and regularly backed up on media located in different places. A particular challenge is backing up databases. It may not be possible to simply copy open files during operation. Instead, backups must be performed from within the application used or by using specialised software like MySQL Dump.

Data protection

Data protection is to be distinguished from data security, although there are overlaps. The goal here is simply to ensure that unauthorised persons have no access to confidential data. Data protection includes the following areas:

1. Prevent external access: This requires data security measures

2. Restrict internal access to personal data: This requires a permissions management system in the software used. This way, individual employees are denied access or data sets are only partially displayed. Additional protection is provided by encrypted transmission and storage of data. This ensures that sensitive data is protected from hardware access, such as during a break-in or by unauthorised employees.

Archiving

Archiving non-current data — such as invoices and tax documents — is a legal requirement under UK law, with retention periods defined by HMRC and other regulators.

• Separate storage – Reduces backup size and improves data protection.
• Suitable media – Tape drives are highly durable but costly to set up. Hard drives must be checked regularly. CDs/DVDs are prone to degradation.

Deleting

When data is no longer required, securely delete it to remove compliance obligations. The UK GDPR includes the “right to erasure” for personal data. Standard OS deletion may only mark data as available for overwriting; use secure erasure tools for full compliance.

Other legal requirements

Key UK legislation and frameworks include:

• UK GDPR and Data Protection Act 2018
• Privacy and Electronic Communications Regulations (PECR)
• Freedom of Information Act 2000 (for public bodies)
• Sector-specific requirements such as NHS Data Security and Protection Toolkit for healthcare

Some organisations must appoint a Data Protection Officer (DPO) — for example, if they carry out large-scale monitoring or process sensitive categories of data.

Types of data management

How data management is organised depends on the size of each company. Various approaches to integrated solutions are available on the market. Possible forms include:

• Enterprise Resource Planning Systems (ERP): These systems offer the most comprehensive approach. All resources of the company are recorded and considered. This includes personnel, equipment, and materials. Well-known commercial vendors are SAP, Sage, Oracle, and Microsoft. However, there are also free software solutions like Odoo and OpenZ.
• Master Data Management (Central Master Data Administration): Centralisation and revision of a company’s core data. This includes employee data, customer data, and information about equipment. The goal is a uniform data quality that leads to improved usability. This approach is usually considered in ERP systems.
• Content Management Systems (CMS): Predominantly information management systems, for example, in the form of a central intranet for the company. Due to their great flexibility, other aspects like form management and the integration of databases are also possible.
• Document Management Systems (DMS): A subset of data management. They provide forms and offer functions such as filing and archiving.

Challenges of data management

Data management is a dynamic process and must be continually adjusted to current requirements. This results in new challenges each time.

Big data

Data volumes are constantly increasing. Consequently, there are high demands on data management, especially in the following areas:

• Scalability of storage and backup capacities
• Organisation and findability of required data
• Data minimisation and filtering of important information

Security

Network administrators constantly face new dangers. Information theft through social engineering and sabotage via ransomware are just some scenarios. The more a company digitizes its data resources, the more dependent it becomes on the functionality of the system used. Therefore, it’s essential to stay informed about new risks and make preparations for hardware failure or lack of access to their own systems.

Legal requirements

The introduction of the UK GDPR and the Data Protection Act 2018 has brought significant changes for businesses operating in the UK. Many organisations faced considerable effort to align processes with the new rules, particularly following the UK’s departure from the EU. It remains likely that further regulations or amendments — for example, updates to data transfer rules or sector-specific requirements — will be introduced in future. Such changes may require companies to review and adjust their data management strategies.

Changes in the company environment

Shifts in a company’s structure or operations — such as mergers, expansions, or new service offerings — can have a direct impact on data management. To prepare for this, businesses should use systems that are scalable and easy to migrate. Regular employee training on internal data governance and compliance requirements is essential, even though it adds to the ongoing workload.