SF Text: Adithya Rao, Q&A with Alexy Khrabrov @Lithium

hello everybody I'm Alexi kov the hello everybody I'm Alexi kov the organizer of SF text a new metup devoted organizer of SF text a new metup devoted to text mining AI search NLP and to text mining AI search NLP and basically discovering human anten behind basically discovering human anten behind you know the strings of text and this is you know the strings of text and this is our second meet up we're here at lithium our second meet up we're here at lithium and have ad uh lead research engineer at and have ad uh lead research engineer at lithium with us and uh we formerly were lithium with us and uh we formerly were colleagues at Cloud so I know quite a colleagues at Cloud so I know quite a bit about cloud and uh the topic today bit about cloud and uh the topic today is topics U which is something that is topics U which is something that cloud does really well and Adia works on cloud does really well and Adia works on is going to talk about tonight so we'll is going to talk about tonight so we'll have a the talk separately so I'm just have a the talk separately so I'm just going to ask a few questions to kind of going to ask a few questions to kind of get more perspective on on this whole get more perspective on on this whole Space yeah so it's great to have your Space yeah so it's great to have your DET with us thanks foring us uh so uh uh DET with us thanks foring us uh so uh uh how did you get interested in this whole how did you get interested in this whole space I know that you know what you were space I know that you know what you were doing at Cloud you know and you're doing at Cloud you know and you're really interested in the science of really interested in the science of these things how they work so can you these things how they work so can you tell me a little bit about your tell me a little bit about your background you know and what brought you background you know and what brought you to to these problems definitely um I to to these problems definitely um I think it started mainly when I started think it started mainly when I started looking at uh I I come from originally looking at uh I I come from originally from an electrical engineering from an electrical engineering background and I was looking at wireless background and I was looking at wireless networks as my initial field of study networks as my initial field of study okay uh from wireless network Works okay uh from wireless network Works communication was a uh kind of an easy communication was a uh kind of an easy walk through and there uh information walk through and there uh information Theory became like a very interesting Theory became like a very interesting subject as I started going deeper into subject as I started going deeper into that whole life I realized that it's that whole life I realized that it's probably uh easier to work with when you probably uh easier to work with when you look at it from the higher abstraction look at it from the higher abstraction uh and that's when I kind of moved into uh and that's when I kind of moved into the domain of like machine learning in the domain of like machine learning in particular so uh at I did my masters particular so uh at I did my masters from Stanford uh and there I mainly from Stanford uh and there I mainly specialized in machine learning and data specialized in machine learning and data mining and that really got me interested mining and that really got me interested into the various domains where you can into the various domains where you can apply machine learning and uh text was apply machine learning and uh text was one of the big ones and uh that was one of the big ones and uh that was around maybe four years ago when uh you around maybe four years ago when uh you know the social media boom was just like know the social media boom was just like taking off there A lot of lot of people taking off there A lot of lot of people everybody uh getting overwhelmed with so everybody uh getting overwhelmed with so much data around and I thought it was a much data around and I thought it was a per perfect opportunity to kind of look per perfect opportunity to kind of look at uh uh information in social media and at uh uh information in social media and uh how machine learning can be applied uh how machine learning can be applied to that aspect and uh yeah so I joined to that aspect and uh yeah so I joined Cloud at that point and with basically Cloud at that point and with basically there was this huge data set to work on there was this huge data set to work on and uh it's been pretty exciting yes and uh it's been pretty exciting yes yeah I know you know some of the story yeah I know you know some of the story and that's you know it always amazes me and that's you know it always amazes me that you know with a relatively small that you know with a relatively small team Cloud was able to achieve so much team Cloud was able to achieve so much right because so now as I understand it right because so now as I understand it it ingests essentially most of the it ingests essentially most of the Social Web yeah a lot of it yes I mean Social Web yeah a lot of it yes I mean uh I mean I remember you know it was uh I mean I remember you know it was quite a few itations to get there but quite a few itations to get there but how I mean how do you find balance how I mean how do you find balance basically with you know kind of a basically with you know kind of a startup setting you know be able to to startup setting you know be able to to regularly daily you know ingest all of regularly daily you know ingest all of this data and do some science on this this data and do some science on this like how how do you balance the like how how do you balance the engineering work versus the research engineering work versus the research work right right so I think uh I think work right right so I think uh I think the important part is the engineering the important part is the engineering work at the end of the day because work at the end of the day because unless you're able to consume uh this unless you're able to consume uh this kind of data and build up Frameworks kind of data and build up Frameworks that allow you to kind of iterate that allow you to kind of iterate quickly or try to do things which where quickly or try to do things which where you are able to push things out fast to you are able to push things out fast to production and so on no matter how production and so on no matter how complex your algorithms are it's not complex your algorithms are it's not going to amount to anything at the end going to amount to anything at the end of the day so I think uh that is where of the day so I think uh that is where like a lot of effort went in in the like a lot of effort went in in the beginning years of cloud at least and uh beginning years of cloud at least and uh finally when we did have a framework finally when we did have a framework where we could ingest this data and run where we could ingest this data and run like pipelines every day then it became like pipelines every day then it became easy to kind of let's say let's try try easy to kind of let's say let's try try out this algorithm let's try out this out this algorithm let's try out this model let's try out these features let's model let's try out these features let's maybe vary it up a bit and then maybe maybe vary it up a bit and then maybe the Cycles became faster and iteration the Cycles became faster and iteration became faster so I guess uh engine ing became faster so I guess uh engine ing played plays a very big role when it played plays a very big role when it comes to data science although uh when comes to data science although uh when we talk about data science in particular we talk about data science in particular we often ignore that aspect of it yes uh we often ignore that aspect of it yes uh so I remember uh you used a lot of so I remember uh you used a lot of different tools you know we at Cloud can different tools you know we at Cloud can you talk a little bit about what kind of you talk a little bit about what kind of uh tools are using uh in your daily work uh tools are using uh in your daily work sure yeah so uh I think most of the sure yeah so uh I think most of the stand for machine learning at least we stand for machine learning at least we use most of the standard ones like uh use most of the standard ones like uh python libraries and uh WCA and so on uh python libraries and uh WCA and so on uh but in terms of the data pipeline we use but in terms of the data pipeline we use uh main the map produ stack with the uh main the map produ stack with the with our hdfs And Hive on top of that with our hdfs And Hive on top of that for our quering language Hive is for our quering language Hive is actually a pretty powerful abstraction actually a pretty powerful abstraction because it allows us to write like because it allows us to write like complex queries which trigger like complex queries which trigger like multiple jobs without actually worrying multiple jobs without actually worrying about how how you would go about about how how you would go about optimizing those things and the really optimizing those things and the really interesting part is also that it allows interesting part is also that it allows us to have udfs which uh actually us to have udfs which uh actually encapsulate a lot of the complex logic encapsulate a lot of the complex logic that you may not be able to write in that you may not be able to write in Hive but you end up uh like let's say Hive but you end up uh like let's say you wanted to apply a predictive model you wanted to apply a predictive model uh to a a row in Hive you would want to uh to a a row in Hive you would want to have that as a UDF that can maybe look have that as a UDF that can maybe look up some data from different sources do up some data from different sources do some extraction do some NLP and then some extraction do some NLP and then apply a model and all of that can exist apply a model and all of that can exist in uh a UDF so I think that combination in uh a UDF so I think that combination of the Hadoop stack along with the of the Hadoop stack along with the ability to write udfs and then uh ability to write udfs and then uh training offline models using uh machine training offline models using uh machine learning tools has really kind of helped learning tools has really kind of helped us in uh Skilling up cool so you know so us in uh Skilling up cool so you know so when you're doing it's very interesting when you're doing it's very interesting because you're doing you know using both because you're doing you know using both Python and weekend uh you know one of my Python and weekend uh you know one of my huge drives in the scholar Community is huge drives in the scholar Community is to kind of bring scientific comping to to kind of bring scientific comping to the jvm right right and so there is a the jvm right right and so there is a lot of libraries which exists already lot of libraries which exists already but there are some gaps compared to but there are some gaps compared to Python and there is just a habit of a Python and there is just a habit of a lot of data scientists to use Python lot of data scientists to use Python because that's usually the first because that's usually the first language uh they pick up in in school language uh they pick up in in school it's also probably the most developed in it's also probably the most developed in terms of machine learning in uh because terms of machine learning in uh because there there are so many tools that are there there are so many tools that are present over there but yeah right so and present over there but yeah right so and so that you know that's my question to so that you know that's my question to you is kind of practicing with both uh you is kind of practicing with both uh to tool sets what does the jvm community to tool sets what does the jvm community need to do to kind of bring the python need to do to kind of bring the python functionality right where do you see the functionality right where do you see the most gaps and where the community can most gaps and where the community can contribute the most and you know what contribute the most and you know what should happen so for you to stay within should happen so for you to stay within the jvm when you you know do your the jvm when you you know do your experiments right uh I think the main experiments right uh I think the main thing is the comprehensive nature of the thing is the comprehensive nature of the tools that are present in Python I think tools that are present in Python I think a lot of the algorithms that are present a lot of the algorithms that are present in like the python toolkits are not in like the python toolkits are not always in the jvm toolkits and maybe if always in the jvm toolkits and maybe if they are present they're not implemented they are present they're not implemented in a fashion that is very uh efficient in a fashion that is very uh efficient when you want to scale up so uh I think when you want to scale up so uh I think a lot of uh uh there are a lot of small a lot of uh uh there are a lot of small things which the smaller pieces once things which the smaller pieces once they are there and they kind of build up they are there and they kind of build up like Lego bricks that's when maybe the like Lego bricks that's when maybe the jvm uh Community would be at that same jvm uh Community would be at that same scale uh I'm looking forward to using scale uh I'm looking forward to using spark MLB I haven't used it personally spark MLB I haven't used it personally yet but that's seems to be a promising yet but that's seems to be a promising area which could combine like the area which could combine like the advantages of streaming data along with advantages of streaming data along with uh some of the machine learning uh some of the machine learning libraries so uh I'm sure like there are libraries so uh I'm sure like there are a lot of avenues for uh implementing a a a lot of avenues for uh implementing a a lot of the things that are existing in lot of the things that are existing in the python toolkits in into the jvm the python toolkits in into the jvm space yeah and Spark also will run Hive space yeah and Spark also will run Hive right because you know are committed to right because you know are committed to run everything on spark so you know run everything on spark so you know there is spark cql but also Hive itself there is spark cql but also Hive itself will will have spark back end right so will will have spark back end right so which is which is really exciting yes which is which is really exciting yes yeah um interesting interesting so uh so yeah um interesting interesting so uh so you mentioned NLP can you talk a little you mentioned NLP can you talk a little bit about what kind of NLP uh are you bit about what kind of NLP uh are you guys doing what kind of tools are you guys doing what kind of tools are you using and what works in practice Yeah so using and what works in practice Yeah so uh I think one of the biggest uh things uh I think one of the biggest uh things that we found in our uh experiments is that we found in our uh experiments is that if you try to use like s very that if you try to use like s very sophisticated NLP techniques like let's sophisticated NLP techniques like let's say you wanted to use Stanford named say you wanted to use Stanford named entity recognition for passing out uh entity recognition for passing out uh entities from your text uh the what entities from your text uh the what happens is that sure you could run that happens is that sure you could run that on a small scale but you when you want on a small scale but you when you want to run something on 700 million messages to run something on 700 million messages per day it's not going to scale very per day it's not going to scale very well uh so uh one of the approaches that well uh so uh one of the approaches that we took was that we actually used uh we took was that we actually used uh open data sets such as uh Freebase open data sets such as uh Freebase Wikipedia and all of these knowledge Wikipedia and all of these knowledge grafts and using those knowledge crafts grafts and using those knowledge crafts we try to create dictionaries that are we try to create dictionaries that are uh and that are pre-trained and pre uh and that are pre-trained and pre pre-fabricated so that they are they pre-fabricated so that they are they already encapsulate all the entities already encapsulate all the entities that are relevant and the advantage of that are relevant and the advantage of doing that is also that when new doing that is also that when new entities start coming up let's say entities start coming up let's say there's a completely new phenomenon on there's a completely new phenomenon on social media today that we want to social media today that we want to incorporate in our dictionaries we are incorporate in our dictionaries we are able to update our dictionaries very able to update our dictionaries very easily so rather than trying to extract easily so rather than trying to extract uh semantic meaning in uh of course uh semantic meaning in uh of course sometimes that is very useful but in sometimes that is very useful but in some cases what happens is just using a some cases what happens is just using a simple dictionary and trying to have simple dictionary and trying to have those dictionary terms mapping to those dictionary terms mapping to certain entities in your topical certain entities in your topical ontology are uh is pretty effective in ontology are uh is pretty effective in uh going in going maybe 90% of the way uh going in going maybe 90% of the way and sure at the end of the day if you and sure at the end of the day if you really want to be like very highly really want to be like very highly accurate you have to uh get more accurate you have to uh get more sophisticated in terms of the NLP sophisticated in terms of the NLP techniques but even doing Simple techniques but even doing Simple techniques at scale uh can sometimes get techniques at scale uh can sometimes get you a long way if you're using the you a long way if you're using the correct kind of data sets and the data correct kind of data sets and the data sources interesting so so basically you sources interesting so so basically you do your most of yourp offline and and do your most of yourp offline and and you when you do training so uh when you you when you do training so uh when you do that can you describe what kind of do that can you describe what kind of Stack are you using sure uh so basically Stack are you using sure uh so basically we use uh Freebase as one of our primary we use uh Freebase as one of our primary uh data sources so again we use Hive to uh data sources so again we use Hive to kind of extract all these uh entities uh kind of extract all these uh entities uh we have a bunch of tools that we have we have a bunch of tools that we have that in the udfs that do things like that in the udfs that do things like tokenization stemming and maybe a lot of tokenization stemming and maybe a lot of other Transformations that so it's on other Transformations that so it's on jvm as as udfs in in yeah it's all jvm jvm as as udfs in in yeah it's all jvm yes it's all in jvm and uh some of those yes it's all in jvm and uh some of those udfs are also open- sourced in uh the udfs are also open- sourced in uh the project called Brick House okay so uh project called Brick House okay so uh it's actually free to use for a lot of it's actually free to use for a lot of people so uh using so they have UDF such people so uh using so they have UDF such as uh where you can pass in a dictionary as uh where you can pass in a dictionary and you can pass in a piece of text and and you can pass in a piece of text and then the text would efficiently extract then the text would efficiently extract the certain entities out of the the certain entities out of the dictionary and so on so uh uh we have a dictionary and so on so uh uh we have a bunch of these udfs which we then use to bunch of these udfs which we then use to kind of uh uh extract entities from kind of uh uh extract entities from Freebase we use certain ranking Freebase we use certain ranking techniques and relevance techniques to techniques and relevance techniques to say that uh these are the most important say that uh these are the most important entities because there's a lot of of entities because there's a lot of of noise in free Bay as well so you would noise in free Bay as well so you would probably pick the cream do some sort of probably pick the cream do some sort of ranking pick the cream maybe top half a ranking pick the cream maybe top half a million keywords and then map those half million keywords and then map those half a million keywords into 10,000 topics a million keywords into 10,000 topics because multiple keywords would map to because multiple keywords would map to the same topic and so on so uh mainly the same topic and so on so uh mainly the stack that we use is still hi in the stack that we use is still hi in udfs for building these models uh with a udfs for building these models uh with a little bit of uh python to just kind of little bit of uh python to just kind of do the uh little bit whatever a little do the uh little bit whatever a little bit of machine learning training is bit of machine learning training is required interesting so I mean that required interesting so I mean that sounds really kind of a scalable sounds really kind of a scalable production infrastructure and I must say production infrastructure and I must say that you know B's I think lead is Jerome that you know B's I think lead is Jerome Banks who is our common former colleague Banks who is our common former colleague now at F we and actually we are planning now at F we and actually we are planning to present some of the open sourcing uh to present some of the open sourcing uh stages of that project and some of scups stages of that project and some of scups uh coming coming forward yeah uh uh coming coming forward yeah uh interesting so so given that you you you interesting so so given that you you you kind of set up this infrastructure you kind of set up this infrastructure you toate it and you have it in kind of nice toate it and you have it in kind of nice you know ready working production order you know ready working production order and you can iterate an experiment what and you can iterate an experiment what are the next steps where do you see are the next steps where do you see challenges for for for this project in challenges for for for this project in the future yeah I think uh one of the the future yeah I think uh one of the most exciting things for me is uh most exciting things for me is uh actually in the domain of deep learning actually in the domain of deep learning MH uh so deep learning is just I mean MH uh so deep learning is just I mean everybody's talking about it it's it's everybody's talking about it it's it's kind of the hot area of machine learning kind of the hot area of machine learning right now uh the challenge there is that right now uh the challenge there is that if you wanted to run any algorithm on if you wanted to run any algorithm on deep learning you it's I mean it's deep learning you it's I mean it's debatable whether you can do it on the debatable whether you can do it on the cloud and on the jvm uh right now the cloud and on the jvm uh right now the best uh approaches to that have been to best uh approaches to that have been to run it on uh gpus uh now if you wanted run it on uh gpus uh now if you wanted to scale gpus but now you're using to scale gpus but now you're using something like the Hado stack then how something like the Hado stack then how you going to really go about it so I you going to really go about it so I think uh there are certain challenges think uh there are certain challenges there that uh we'll probably be I mean there that uh we'll probably be I mean I'll be keeping my eye out on uh in the I'll be keeping my eye out on uh in the next couple of years to see how that next couple of years to see how that plans out but uh uh that is a very plans out but uh uh that is a very exciting area of research because uh if exciting area of research because uh if you are able to do unsupervised learning you are able to do unsupervised learning using uh deep learning on like large using uh deep learning on like large corpuses then uh you pretty much have corpuses then uh you pretty much have like uh a huge step ahead than what like uh a huge step ahead than what everybody has is that currently on uh in everybody has is that currently on uh in in terms of production systems you know in terms of production systems you know it's interesting you mention this it's interesting you mention this because you know we have this uh uh because you know we have this uh uh conference coming up text by the bay and conference coming up text by the bay and one of the talks they actually Dearing one of the talks they actually Dearing with deing for J on spark and that tool with deing for J on spark and that tool kit actually uses gpus right so because kit actually uses gpus right so because you you can there is a linear algebra you you can there is a linear algebra implementation nd4j which can use Cuda implementation nd4j which can use Cuda and and gpus so hopefully uh we make and and gpus so hopefully uh we make some advances there right and I've some advances there right and I've looked into de learning for it looks looked into de learning for it looks very interesting very interesting Park too right so hopefully all this Park too right so hopefully all this stuff will come together and I think stuff will come together and I think actually it a great application and actually it a great application and hopefully you know uh we'll see what's hopefully you know uh we'll see what's next and I mean I think the committee next and I mean I think the committee will be happy to to work on this stuff will be happy to to work on this stuff yeah well thanks for sharing us we're yeah well thanks for sharing us we're looking forward to your talk thank you looking forward to your talk thank you thanks so much yeah all right thanks