2019 saw the rise of the Data Engineer, the end of PPI remediation, and the resurgence of data warehousing in competition with big data in the cloud. We also witnessed the continued growth of data preparation and its complementary toolkits, as well as businesses putting governance frameworks into practice, such as Open-banking and GDPR. So, what will 2020 hold…?
Our Dufrain Data Management experts share the trends they expect to be the main talking points of next year.
Rise of the Technology Generalist
The most successful Data Engineers, Data Scientists, and Data Analysts are technology generalists rather than specialists. As the breadth and depth of the technology toolkit is ever-expanding; being adaptable and open to diversifying your skillset to stay at the forefront of this growth will be more important in 2020 than ever before. Modern data professionals should be aware of storage solutions: HDFS, NoSQL, and Oracle for example, but also competent in multiple languages such as SAS, SQL, Python, Java, and R, and comfortable navigating cloud, hybrid and on-premise environments. These skills will enable the effective exploration of data, the generation of valuable insights, and presentation of findings to wide audiences in 2020.
Death of the DBA?
Systems are becoming more self-tuning and cloud data platforms, which when provided as-a-service, need much less administration than traditional on-premise databases, are becoming widely adopted. Public cloud platforms can use predictive analytics to project future resource demands and respond by optimising environments automatically and dynamically. In 2020, Dufrain predicts there will be less and less need for dedicated database technical specialists as modern data platforms automate most of the tasks normally associated with Database Administrators (DBAs).
Automation of Data Management Using Software Agents
Software agents, also known as bots or robotic process automation, are goal-oriented computer programs that run without supervision to perform a function. In 2020 Dufrain predict that they will be effectively utilised within data management to improve connecting data to sources, working with AI & Machine-Learning techniques to implement data quality at scale, and to achieve accelerated processing through inferring business rules.
Rise of Microservices
With the data world embracing cloud computing, IoT, agile development, and DataOps, Dufrain predicts there will be wider adoption of microservices based architectures in 2020. Microservices as a concept was previously followed in application design & architecture but can be successfully applied in data architecture. In a microservices architecture, large and complex data tasks are decomposed in multiple loosely coupled self-contained lightweight tasks which are connected to each other. With this approach development, testing, and deployment can be rapid and adaptable to business change.
Bigger Big Data 2020?
All aspects of data are growing at an astronomical pace, and this isn’t going to change anytime soon. Dufrain expects that there will be developments in advanced techniques to hold, process, and visualise enormous amounts of information to help effective decision making. However, not only is the amount of data expanding, but so is the diversity of data: metadata, e-mail messages, audio, video, analog data, images, and files are all captured by organisations and can hold significant value. Dufrain predicts that businesses who can effectively mine their vast and varied data lakes using algorithmic approaches will be able to generate great insights in 2020.
Master Data and Data Quality Management
There is a saying ‘quality is better than quantity’, and this is certainly true when it comes to data. There’s little value in having vast amounts of low-quality & non-harmonised data, and Dufrain doesn’t imagine this to change in 2020. Naturally, through advanced technology, we will see a reduction in manual effort and an increase in processing speed in the application of data quality checks and rules. However, it will be the overall data governance – at point of entry; modelling; definition; setting up of enterprise master data; all leading to acceptable data quality, which will take centre stage in 2020. Without this focus, more organisations will fall foul to the age-old problem of garbage-in-garbage-out, just faster than ever before.
Conclusion
Ultimately, the key to succeeding as a data-driven organisation in 2020 will be through embracing technological advancements, bolstering controls and operations, and ensuring your business has the right framework to fully leverage the potential of its data.
[km-cta-block padding=20 block-classes=”has-dark-teal-background-colour has-white-colour” label=”Contact us to discuss your requirements”]
Want to know more?
Our data experts would love to hear from you
[km_button link=”https://www.dufrain.co.uk/contact/” classes=”cta-2″]Contact us[/km_button] or [km_button link=”tel:08001303656″ classes=”cta-2″]Call us on 0800 130 3656[/km_button][/km-cta-block]