#159 Part 2- How do we get data scientists going in 8 hours with Nitish Mathew Global Head of Data Engineering at Afterpay
In part 2 of this episode, Nitish shares his thoughts on how to ensure that the data scientists expertise are well utilized. He says their key metric last year for the entire team was to hit exactly one number - 100 active users for their data platform with only 50 data scientists! He also says organizations should focus on enabling self-service across the business.
Nitish talks about the data support channel at Afterpay which allows for anyone to ask questions about the data and the data platform. He says find the people naturally drawn to the data and support them with assistance, training and coaching.
Also discussed in this episode:
What percentage of data science projects fail
How do we empower other teams to do data engineering themselves
Types of data engineers: data platform engineer, data pipeline engineer
The philosophy behind Data Quality
Data Science Platforms
Enjoy the episode!
Quotes
“I think designing really good tables, giving some coaching on how to get good performance out of analytical systems, and then providing starters, SQL queries that give an example of some basic stuff is good enough.”
“I think data engineering should be divided into two things at big organizations. One, you have a concept of a data platform engineer, and then we have a concept of a data engineer. The data platform engineer is the empowerment platform road.”
“Why many projects fail is because there's a huge lag between the time when a new application or data is created when it's available for analysis.”
“Our philosophy behind Data Quality is that data quality is something that is best owned by the data producers. We don't make any sort of data quality fixes in the data pipeline, if there's something bad then we let it go through. Our job is to make sure that we actually have really good systems to find out the deviations, to find out issues.”
“What is the scariest question that a data engineer can get in a day? The data scientist says This doesn't look right, I see an exception. I tell this to my colleagues all the time, you are guilty until proven innocent.”
Thanks to our sponsor:
And as always, we appreciate your Reviews, Follows, Likes, Shares and Ratings. It really helps new data scientists find us.