#209 How Successful Transformation Is Driven By Data Engineering Excellence With Richard Glew, Chief Technology Officer, and Natalia Dronova, Senior Data Analyst from Aginic

This week on the Data Futurology podcast, we talk transformation and the importance of having data engineers to guide the strategy and agenda. To provide expert insights into this topic, we have the pleasure of hosting Richard Glew, Chief Technology Officer, and Natalia Dronova, Senior Data Analyst from Aginic.

Aginic is a consultancy that assists organisations with their transformation goals, providing expertise in analytics, agile, and the digital experience.

Transformation remains a challenging goal, with research showing that most projects fail. Glew and Dronova discuss some of the reasons for this, which are many and varied, but according to Dronova, one of the big ones is that organisations make mistakes in their haste to transform quickly.

“One of the challenges with transformation are the people that want everything done within six or eight months,” she said. “They want it now, and they’re finding shortcuts to try and make it happen that are hurting them in the long run. Then, a few years later, when you look at their stack, it’s all over the place.”

Dronova and Glew then go in-depth in discussing the structural problems that can affect transformation efforts, as well as the cultural problems across organisations – the impact that a focus on data governance can have on projects, for example, and why organisations need to move to a position of data enablement.

Finally, the two also discuss the role of the data engineer. As Glew said, traditionally the role has lagged behind that of the software engineer, but with more focus being placed on their role in transformation, the rapidity with which the role is evolving, and the relative scarcity of engineers resulting in higher salaries, now is a great time to consider a career in data engineering. “With the state of data engineering today, it’s the best time to get into it, because it’s still evolving and innovating really quickly.” Glew said.

Tune in to this deep and insightful discussion to learn more about the dynamics behind transformation and the role of the data engineer.

Enjoy the show!

Thank you to our sponsor, Talent Insights Group!

Connect with Richard                 https://www.linkedin.com/in/rlglew/

Connect with Natalia                  https://www.linkedin.com/in/nataliadronova/

Learn more about Aginic           https://aginic.com/

Join us in Melbourne for Scaling AI with MLOPS:  https://www.datafuturology.com/mlops

Join our Slack Community:https://join.slack.com/t/datafuturologycircle/shared_invite/zt-z19cq4eq-ET6O49o2uySgvQWjM6a5ng

“If you’re not in control of your data, you’re not necessarily in control of the narrative anymore about what your area is doing. And if you don’t have those universal metrics about this is how we define this metric globally, then that becomes then next round of challenge.”

WHAT WE DISCUSSED

0:00 Introduction
06:39 Do you recommend people to move throughout the stack in their careers? And if so, do you have an order of preference?
08:06 If you can give us a bit of an overview of that engineering, where do you see it and where's it at the moment?
11:48 Tell me a bit more about the distinction between, a check-in business or delivery and the stakeholders?
13:43 Are there any key differences that you see between the organizations that have the distinction of the business and the delivery teams? And the ones that don't? Are there? Are there, for example, structural differences?
15:08 How have you seen the role change evolve? And where do you see it going?
16:53 As the role evolves, and I guess the discipline matures, do you think that there'll be parts of their engineering that will become a solved problem? Or will we get to a point where we have enough or too many data engineers where they're a hot commodity? And the problems that data engineers are solving, are our pain points for almost every business, will we get to a point where we feel like we have enough data engineers, and that there are parts of the engineering problems that are problems today, but that will be solved in the future?
27:12 How do you see the business transformation or digital transformation as a broader umbrella? What are some of the issues that you see companies having in either embarking on that journey or successfully completing that journey?
38:48 We spoke before about data management is one of those areas. Could you tell us a little bit more about how those two intersect, and what that that that world looks like at the moment?

EPISODE HIGHLIGHTS

• While that sort of dysfunction exists between those parts of organisations, which inevitably does in every large organisation, then I think it's inevitable that you end up with a pretty dysfunctional environment, and people aren't working together in the right ways. We don't have the right tools, we have the right mindset. And so that's what I'm passionate about trying to change.

• The best news about the state of data engineering today is that it's the best time to get into it, because it's still evolving and innovating really fast. And I don't think that that's going to stop.

• I often talk to people and say, we can build these amazing answer machines, and you get it. But if you don't know what questions to ask it, and you can't understand what it tells you, then what's the point?

• Equally, if you pick something too gnarly, then you can be discouraged early on that it will never work. So, finding some use cases that are in the middle is important.

• I think if you can try to structure that change in a way that you are biting off bite-sized chunks that you can actually achieve and accomplish but still have an overarching strategy and a vision of where you're trying to get to.

• Somebody says we need a Data Governance team to sort this out. And then usually larger organisations might have somewhere between two and twenty people that they assign, and it's not even trying to put out a bushfire with a garden hose, it's trying to put it out with a drinking straw in a glass.

• I think we need to step away completely from this idea that we can somehow retrospectively fix our data, and actually focus more on keeping it clean and good in the first place and building in those systems and processes that do it.

---

At Data Futurology, we are always working to bring you use cases, new approaches and everything related to the most relevant topics in data science to help you get the most value out of these technologies! Check out our upcoming events for more amazing content.