#106 Leveraging Machine Learning and NLP for Conversational AI
Today Felipe has a few guests at the Data Science Melbourne Meet-Up Group. Former guest Prashant Natarajan and Romina Sharifpour share about their work with conversational AI. Later, Nic Ryan shares his story and tips for getting started in data science.
Prashant starts by giving some background on the development and basics of NLP. We have made progress with deep learning and bringing machines closer to understanding human language, but it isn’t easy. Humans have a hard enough time understanding each other. What we are asking machines to do is analyze tone, find nouns, and recognize different languages. We already have things like voice to text and smart speakers, but NLP's ultimate goal is for a bot to have a conversation. True NLP is a combination of natural language understanding and language generation.
If we can do this right, we can make a human’s job easier. NLP looks to improve the user experience, not eliminate human interaction. A particularly important application of NLP is in customer service. Here Prashant suggests that the focus should be on improving conversation accuracy; people will get frustrated talking to a bot that doesn’t understand them. NLP can also be used for sentiment analysis. Understanding the sentiments of the person providing the input improves the customer experience, but also has more meaningful implications. For example, Prashant shares a story of a project that was able to predict with high accuracy if a person was depressed and then ensure they get the right resources for help.
Romina shares the most tedious, but crucial step in leveraging machine learning- the data clean up! To improve the accuracy of the NLP, you have to feed it testing data. The data processing and labeling stage are what most directly impacts the quality of your results when it comes to good NLP. There are many complicated things to manage, like question and answer sections and profanity. Romina strongly encourages listeners not to reinvent the wheel, start with programs that are already out there, and invest time in cleaning and labeling your data for more accuracy.
Romina and Prashant wrap up by sharing some of the key takeaways they had from their experience thus far. Building good business sense and relationships with stakeholders is crucial in this field. You have to be able to sell what you are doing because most companies you work with will have a fluent understanding of the work. Prashant encourages us to utilize open source programs and continuously go back to the data to see how you can make it better. They both wrap up with a Q&A.
Nic Ryan also joins our conversation with some tips on getting started and some lessons he’s learned in the business. Nic began by taking courses online to learn about the many different aspects of data analytics during his commute and recommends a similar path for those looking to get started. Just spending a few hours a day building up whatever competency you need to grow- programming, report writing, stats, etc., can help you get started. Nic also stresses the importance of finding a niche. It isn’t just about what you want to do; its what you can get a contract to do and make money doing. Having a specialty will help you stand out.
Nic also talks about his challenges and what he’s learned. For many, the business side, getting clients, and figuring out what to charge, can be the most challenging part of consulting. Nic recommends starting in short-term consulting. The reality is, most companies are at a very basic place with computer science; they are just looked for automated systems or incorporating AI at a small scale. Although these jobs are straightforward, they can be very profitable! As far as getting clients, Nic compares it to dating. There will be a lot of rejection, but you have to put yourself out there. Nic wraps up by sharing his thoughts on deploying machine learning in production, which he thinks is the next step in machine learning. We end with a Q&A.
Enjoy the show!
We speak about:
[00:10] Introduction & welcome guests
[06:45] Basics of NLP & customer service
[17:45] Cleaning data and data processing
[27:30] Key takeaways from an NLP conventional AI project
[42:20] Q&A with Romina and Prashant
[51:50] Meet Nic Ryan
[53:50] Nic’s start in data science
[75:00] Nic’s tips and lessons learned
[80:00] The future of machine learning
Resources:
Data Science Melbourne Meet-Up Group
Prashant Natarajan on LinkedIn
Quotes:
“When you ask computers to do NLP you are asking them to do something that is the holy grail of machine learning.”
“If we don’t make the human's job easier by using NLP then we have to ask ourselves, what the heck are we doing?”
“You have to think about what your audience wants, not just what you are interested in”
“Consulting is like dating, you meet a lot of people, and there’s a lot of rejection.”
Thank you to our sponsors:
Fyrebox - Make Your Own Quiz!
RMIT Online Master of Data Science Strategy and Leadership
Gain the advanced strategic, leadership and data science capabilities required to influence executive leadership teams and deliver organisation-wide solutions.
Visit online.rmit.edu.au for more information.
We are RUBIX. - one of Australia’s leading pure data consulting companies delivering project outcomes for some of the world’s leading brands.
And as always, we appreciate your Reviews, Follows, Likes, Shares and Ratings. Thank you so much for listening. Enjoy the show!