Asif: Tell us about your journey in AI/Data Science leading up to this point.
Vijay: I have been working in AI and Data Science for 17 years in academia and industry now. I did academic research and published some highly cited papers in Machine Learning, Natural Language Processing and Web Search from my work at IIT Bombay and Stanford University. As a Scientist at Yahoo, I invented a new algorithm to learn compact machine learning models for multi-class categorization that was 400 times more compact and over 20 times faster to train and test, compared to the start of art methods. I hold a patent for this. As Founder and CTO of my previous company, Rover, I drove a lot of our content personalization and data science efforts and was also SVP of Data Science at Revcontent that was the company that acquired Rover.
At Turing, I run our data science and machine learning efforts to automatically source and screen software engineers and build detailed skill profiles. We use these skill profiles to make high-quality customer matches and these skill profiles continue to be updated from data about a developer’s collaboration with our customer.
Asif: The automation industry is seeing a rising importance of Big Data and AI. How do you see these emerging technologies impacting the recruiting sector?
Vijay: I expect these technologies to substantially change the recruiting sector. The availability of large amounts of data, coupled with modern AI methods will help effectively automate large chunks of the recruiting industry that has traditionally been very manual in its approach. We see several exciting companies focused on recruiting tech for local hiring as well as companies like Turing that focus entirely on remote software engineer hiring.
Asif: Tell us about your venture ‘Turing’. How is it going to affect recruitment?
Vijay: Turing helps companies hire pre-vetted remote software engineers. As software is eating industry after industry and hiring quality engineers in one’s zip code is becoming challenging, we see more and more companies becoming open to hiring remote engineers. We expect Turing to help change the way companies are fundamentally built and make it easy for companies to hire exceptionally talented software engineers all over the world. For talented software engineers, Turing offers them the opportunity to live anywhere in the world and still be plugged into the best job opportunities in Silicon Valley and the rest of the US.
Asif: How is ‘Turing’ using AI to find the world’s finest talent at the touch of a button?
Vijay: Turing, uses AI to automatically source and screen software engineers and build detailed skill profiles. We use these skill profiles to make high-quality customer matches and these skill profiles continue to be updated from data about a developer’s collaboration with our customer. We use AI methods to automatically improve our tests based on how well they predict which developer will get hired and perform well. We also use AI methods to estimate the health of an ongoing customer collaboration, which in turn helps create more fruitful long term collaborations between our developers and partner companies.
Asif: Can you shed some light on the latest employment trends related to AI and Data Science?
Vijay: At Turing, we make a number of successful matches of our Machine Learning engineers to companies looking to hire Machine Learning engineers and scientists, Data Scientists, Machine Learning for image categorization as well as Machine Learning for NLP. ML engineers with expertise in NLP appear to be in short supply relative to industry demands.
Asif: What would be your advice to budding machine learning and Data Science candidates?
Vijay: I’ll be writing a longer blog post on Turing’s blog when I get more time. But in a nutshell, I would strongly encourage ML and Data Science candidates to spend enough time on mastering the fundamentals including the underlying mathematics. Many software engineers take the view that they can download TensorFlow or sk-learn and play with a few models to gain the necessary competence, but this view is deeply mistaken. If you wish to achieve strong results as a machine learning engineer and have a strong framework of predicting what methods would work well in your case, mastery of the fundamentals is vital. Otherwise, you risk being the kind of ML engineer who generates activity and no results.
Asif: What advice would you give to Startup CEO’s who want to use AI to improve their business?
Vijay: For starters, make sure your business fundamentals are sound. Make sure you are playing in a market that really wants what you have on offer. AI won’t rescue an unsound business that is handling the wrong problem.
When using AI for business, exercise a great degree of pragmatism and look for quick wins first. Make sure you use AI only in areas where simpler Data Science or Data Analytics methods won’t do the trick for you. Make sure any complexity you take on is adequately justified in terms of the actual business upside. You never want to encourage doing AI for AI’s sake. AI can be very valuable to your business but only if applied to the right problems to the right extent and in the right way. If you are working on certain problems like Image Categorization which are relatively commoditized AI problems, you may want to directly start with the methods that are known to work well. If like many entrepreneurs you are looking to apply AI to a brand new problem and type of data, make sure to have enough data analysts and data engineers carefully clean and understand the data. It is better to staff such a team with specialists. Since actual modeling is often a minority of the work, you want to make sure that your team is staffed with enough backend and data engineers and data analysts in addition to machine learning engineers and scientists.
Asif: Can you name some AI/ Data Science books that have influenced your thoughts the most?
Vijay: The field has evolved so far that the best resources today are definitely far superior to the resources I had access to when I started work in AI and Data Science. Here are the resources I would recommend today:
Mathematical Pre-requisites:
a) Probability, statistics, etc.
https://www.amazon.com/Introduction-Probability-Theory-Paul-Hoel/dp/039504636X
https://www.amazon.com/Introduction-Statistical-Theory-Houghton-Mifflin-Statistics/dp/0395046378
https://www.amazon.com/Introduction-Stochastic-Processes-Paul-Gerhard/dp/0881332674
b) Books on linear algebra and multivariate calculus
Machine Learning resources:
https://www.coursera.org/learn/machine-learning.
I strongly recommend not just watching lecture videos, but also doing homework, programming assignments.
ML advanced:
https://www.coursera.org/specializations/deep-learning#courses (complete all 5 courses in the specialization).
https://web.stanford.edu/class/cs224n/
https://www.youtube.com/watch?v=8rXD5-xhemo&list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z
https://www.deeplearningbook.org/
Keeping tabs on the latest and greatest in ML.
Exploiting tools/resources/algorithms that got developed very recently and are a long way away from entering course curricula. This is important since ML advances are happening very rapidly in the last few years and for many applications, the technology available to us this year could be far superior to the technology available even a mere 2 years back! All things equal, you want to use the latest and greatest rather than outdated technology on certain problems. Of course for many other problems, old and simple may be the way to go.
Find great pre-trained models to leverage for transfer learning, great datasets to train on, etc.
Practicing ML, getting swift feedback regarding the effectiveness of your methods:
Do challenges on https://www.kaggle.com/ of different types. Some challenges reward you for some accuracy measure alone, others also put constraints on systems resources that can be consumed by your inference models, which would also force you to consider tradeoffs typical in industry settings.
Work on appearing in the top 1% in the leaderboard ideally or top 5-10% at the very least.
Optional, but still helpful:
Getting a research paper accepted into top ML/NLP/Vision conferences like COLT, NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR with the first authorship ideally, or at the very minimum get a paper something accepted in conferences one tier below these.
Asif: What are your views about MarkTechPost.com?
Vijay: It looks like a very exciting initiative! I am unaware of any other tech blog with such an exclusive focus on AI and Data Science! Best wishes to you guys!
