Adam Pingel, Q&A with Alexy Khrabrov of SF Text @VigLink 20150210

uh welcome everybody this is sf text the uh welcome everybody this is sf text the first inaugural meetup uh i'm the first inaugural meetup uh i'm the organizer alexis kravrov and we're on organizer alexis kravrov and we're on location at wiggling location at wiggling the company which works with a lot of the company which works with a lot of text and uses text and uses very interesting technology very interesting technology uh we have uh two talks today from uh we have uh two talks today from wiggling and from grant and gersol wiggling and from grant and gersol and we are going to talk with adam adam and we are going to talk with adam adam pingel who is our pingel who is our gracious host uh gracious host uh he will first tell us about uh wiggling he will first tell us about uh wiggling itself so adam what is weak link what itself so adam what is weak link what are you guys doing are you guys doing well um we help publishers monetize well um we help publishers monetize their content in a sentence and that their content in a sentence and that comes in a couple forms we help comes in a couple forms we help help them affiliate links that they help them affiliate links that they might have already in their content to might have already in their content to product pages and then we also can product pages and then we also can identify product references and text and identify product references and text and turn those into affiliated links turn those into affiliated links and and as we're technology people and we like as we're technology people and we like search and nlp and tax money what are search and nlp and tax money what are the underlying technologies they use the underlying technologies they use what are the challenges in in doing this what are the challenges in in doing this well um well um just to speak to the scale of it first just to speak to the scale of it first we we work with tens of thousands of we we work with tens of thousands of merchants merchants we do many billions of page views we do many billions of page views every month every month so just scaling so just scaling scaling the load is a challenge we run scaling the load is a challenge we run in three different amazon regions databases we use our cassandra mysql uh databases we use our cassandra mysql uh we use elasticsearch we use elasticsearch um um so so there's that side of it and then so so there's that side of it and then on the algorithm side we have a whole on the algorithm side we have a whole data science team and a couple of nlp data science team and a couple of nlp experts who uh who tune our name experts who uh who tune our name identity recognition and our matching identity recognition and our matching technology um to make sure that we are technology um to make sure that we are linking to high quality products that linking to high quality products that are going to yield uh are going to yield uh the highest possible expected uh revenue the highest possible expected uh revenue for the publishers for the publishers uh and i know adam from sf scholar meet uh and i know adam from sf scholar meet up where we've been colleagues in a long up where we've been colleagues in a long time so i wonder what is the rationale time so i wonder what is the rationale behind the choice of scholar and uh its behind the choice of scholar and uh its uh associated technologies yes um we are uh associated technologies yes um we are historically a java shop um and so i historically a java shop um and so i think there have been several attempts think there have been several attempts over the years to use scala i think the over the years to use scala i think the gears really started to mesh a couple of gears really started to mesh a couple of years ago we started using it in the years ago we started using it in the feed systems just to kind of get our our feed systems just to kind of get our our you know our feet wet um and you know our feet wet um and it spread from there so now i think it spread from there so now i think close to about a half of our ad hoc close to about a half of our ad hoc analytics uh queries are written in analytics uh queries are written in scala we do spark scala we do spark we're using akka to do a lot of our feed we're using akka to do a lot of our feed processing now processing now we're slowly introducing akka or we're slowly introducing akka or play-based uh web services into our play-based uh web services into our infrastructure over time um infrastructure over time um so it the the fact that we're coming so it the the fact that we're coming from a java background makes to some from a java background makes to some extent makes that a lot easier because extent makes that a lot easier because it the the migration path is very can be it the the migration path is very can be very gradual and we can go as at a pace very gradual and we can go as at a pace that feels comfortable to us um and then that feels comfortable to us um and then we also have a bunch of folks who are a we also have a bunch of folks who are a little more well-versed in ruby or little more well-versed in ruby or python but i think i think they also python but i think i think they also find that the functional paradigm is find that the functional paradigm is pretty pretty natural for them it pretty pretty natural for them it doesn't require too much ramp up time to doesn't require too much ramp up time to get them productive get them productive cool uh can you talk a lot a little bit cool uh can you talk a lot a little bit about the nlp technologies you're using about the nlp technologies you're using uh i know you're related to some of them uh i know you're related to some of them i wonder what your experience with them i wonder what your experience with them you know uh can you recommend some of you know uh can you recommend some of them them so uh yeah we we use mallet um and i'll so uh yeah we we use mallet um and i'll have to refer you to our nlp experts for have to refer you to our nlp experts for for more on that um for more on that um mallet mallet we've we've looked at epic we've done we've we've looked at epic we've done some benchmarking with that i think some benchmarking with that i think katrina is going to talk about that katrina is going to talk about that tonight um just for the name that any tonight um just for the name that any recognition we got we did get some recognition we got we did get some higher f1 scores but katrina's going to higher f1 scores but katrina's going to go into some depth in that later tonight go into some depth in that later tonight um um mallet's natural successor is factory mallet's natural successor is factory we've we're aware of it we've looked at we've we're aware of it we've looked at it we haven't yet benchmarked that but it we haven't yet benchmarked that but the epic and factory are the two um the epic and factory are the two um if we were to migrate beyond mallet uh if we were to migrate beyond mallet uh those are the two most likely libraries those are the two most likely libraries we'll start using well and so what are we'll start using well and so what are the main challenges in in working with the main challenges in in working with actual data um and what has been actual data um and what has been challenging seeing kind of organizing challenging seeing kind of organizing this information connecting it this information connecting it uh and a little bit about that wow that uh and a little bit about that wow that is a good question is a good question um gabor is going to speak a lot about gabor is going to speak a lot about creating uh training sets i think creating uh training sets i think uh knowing when you're doing the right uh knowing when you're doing the right thing is challenging so getting all of thing is challenging so getting all of the infrastructure in place to the infrastructure in place to um to set up a b tests and make sure um to set up a b tests and make sure that that's all backed up with with that that's all backed up with with solid statistics uh it's very solid statistics uh it's very challenging um challenging um the the traffic we see on any given day the the traffic we see on any given day it varies it varies it varies a lot the products that people it varies a lot the products that people are searching for that the seasonality are searching for that the seasonality of the business um it can be it can be of the business um it can be it can be difficult to know you know when a change difficult to know you know when a change you're making is really having desired you're making is really having desired effect so effect so um i think um i think that's that's one of the challenges i've that's that's one of the challenges i've observed um but again i think gabor is observed um but again i think gabor is is the one who's really leading that is the one who's really leading that charge here um charge here um uh let's see what else would i say this uh let's see what else would i say this scaling search we do we do use search scaling search we do we do use search we've we've gone from using a single box running gone from using a single box running lucine embedded in a tomcat container to lucine embedded in a tomcat container to running a 30 node elastic search cluster running a 30 node elastic search cluster over the last five years there have been over the last five years there have been a couple of points along the way um a couple of points along the way um before before before there was before there was before there was shard sharding before there was shard sharding on solar we had to kind of do our own on solar we had to kind of do our own that was fairly brittle um so about a that was fairly brittle um so about a year ago we moved to elasticsearch year ago we moved to elasticsearch um um yeah i would say between the data yeah i would say between the data science science and the and the scaling science science and the and the scaling issues we definitely have our hands full issues we definitely have our hands full dealing with this this data dealing with this this data and also products products themselves as and also products products themselves as a rich a rich um um understanding understanding a product is more than just its a product is more than just its description there's a rich description there's a rich vocabulary of attributes that products vocabulary of attributes that products might have a rich might have a rich category category a taxonomy that you might build uh to to a taxonomy that you might build uh to to classify um products so so that work is classify um products so so that work is all ongoing um and still very much we're all ongoing um and still very much we're learning new things about that all the learning new things about that all the time great yeah uh so so it's time great yeah uh so so it's interesting because you have essentially interesting because you have essentially a team where you have engineers who have a team where you have engineers who have data scientists who have nlp experts how data scientists who have nlp experts how do you guys balance all of this how do do you guys balance all of this how do you kind of you kind of make sure you have a production code in make sure you have a production code in the end which runs a scale uh can you the end which runs a scale uh can you talk a little bit about uh talk a little bit about uh inter play of these teams and uh how do inter play of these teams and uh how do you get to the actual working yes so we you get to the actual working yes so we have we have a little over 20 engineers have we have a little over 20 engineers here um most of which are located in san here um most of which are located in san francisco although we do have some francisco although we do have some remote offices remote offices we've actually just recently because we've actually just recently because we've been growing so fast split the we've been growing so fast split the engineering team into two halves and engineering team into two halves and theoretically each team is capable of theoretically each team is capable of doing any kind of work in practice each doing any kind of work in practice each team has has their affinities has their team has has their affinities has their their specializations um their specializations um so so we we found it's beneficial to to keep we we found it's beneficial to to keep everyone everyone you know well appraised of what what's you know well appraised of what what's going on across all of engineering so we going on across all of engineering so we do have data science specific events we do have data science specific events we have something called the data science have something called the data science jam every friday at 11 o'clock but jam every friday at 11 o'clock but that's that's open not only to that's that's open not only to engineering but to anyone else in in the engineering but to anyone else in in the company who wants to to check it out company who wants to to check it out it's also a good way for it's also a good way for for the data scientists to kind of uh for the data scientists to kind of uh you know have a you know have a produce a deliverable and have a do a produce a deliverable and have a do a talk for a half hour so um we find talk for a half hour so um we find that's really what are you guys doing that's really what are you guys doing these data science jams uh we'll talk these data science jams uh we'll talk about um about um we basically do two or three we basically do two or three presentations over the course of an hour presentations over the course of an hour and uh the topics can be anything from and uh the topics can be anything from restructuring the way we're doing click restructuring the way we're doing click ids ids um and the math behind that um to um and the math behind that um to to some of gabor's uh to some of gabor's uh research on shallow parsing research on shallow parsing it it really runs a pretty wide range of it it really runs a pretty wide range of topics topics statistics behind the kinds of a b statistics behind the kinds of a b testing we're doing a lot of stuff testing we're doing a lot of stuff awesome do you see uh people learning awesome do you see uh people learning and like engineers not yet familiar with and like engineers not yet familiar with these concepts wanting to actually pick these concepts wanting to actually pick it up and do something about it it up and do something about it absolutely um yeah we actually we just absolutely um yeah we actually we just had a hack day um which was a good had a hack day um which was a good chance for some of the engineers who chance for some of the engineers who don't get don't touch that stuff day to don't get don't touch that stuff day to day to interact not only with the data day to interact not only with the data scientists but but folks in marketing or scientists but but folks in marketing or operations we had a pan company hack day operations we had a pan company hack day so that was a really great opportunity so that was a really great opportunity um um to to to to give folks enough give folks a chance to give folks enough give folks a chance to work with stuff that they don't normally work with stuff that they don't normally uh get to work with day to day excellent uh get to work with day to day excellent excellent uh so grant just join us and excellent uh so grant just join us and you will have to explain to him why you you will have to explain to him why you picked classic search over i know i'm picked classic search over i know i'm gonna have to give him a heads up gonna have to give him a heads up yeah so this is great so but since this yeah so this is great so but since this is the first sf text method and uh and is the first sf text method and uh and we are set in the direction and we are set in the direction and if we link has taxes at score i wonder if we link has taxes at score i wonder what would you as the host and engineer what would you as the host and engineer uh running a team here what would you uh running a team here what would you like to kind of like to kind of get from these meetups and going to the get from these meetups and going to the future how do you see the most future how do you see the most beneficial beneficial uh composition of these meet ups yeah uh composition of these meet ups yeah well i think uh well i think uh a lot of the meetups here in san a lot of the meetups here in san francisco tend to be technology oriented francisco tend to be technology oriented this is something that katrine has this is something that katrine has comments to me about from comments to me about from from joining us from joining us almost a year ago she noticed this so i almost a year ago she noticed this so i think having having a community built think having having a community built around a problem domain more than a around a problem domain more than a technology would be really really useful technology would be really really useful to everyone here at big link to everyone here at big link and of course just having people over in and of course just having people over in person meeting them face to face a lot person meeting them face to face a lot of us see these people in in get commits of us see these people in in get commits or in papers that they might write so i or in papers that they might write so i think just to establish uh more of a think just to establish uh more of a personal connection will be really personal connection will be really helpful to everybody here um helpful to everybody here um i think i think there's so much new there's so much new there's so many problems uh so many there's so many problems uh so many there's so many unknowns in this in this there's so many unknowns in this in this business with this technology um that i business with this technology um that i think the upsides are clearly outweigh think the upsides are clearly outweigh the any uh the any uh uh any uh any i don't yeah maybe i'll allow that the i don't yeah maybe i'll allow that the the part the part but yeah i think i think but yeah i think i think the the the upside of sharing the the the upside of sharing information uh is as far outweighs any information uh is as far outweighs any uh uh any concern about uh any concern about uh you know ip or anything like that i you know ip or anything like that i think i think we can we can you know think i think we can we can you know balance that and balance that and teach each other how to teach each other how to use the tools learn the algorithms true use the tools learn the algorithms true develop open source projects together develop open source projects together potentially yeah i think open source is potentially yeah i think open source is the key work here because i think the the key work here because i think the this group is coming together around this group is coming together around shared open source and we have academics shared open source and we have academics producing it we have practitioners producing it we have practitioners taking it and hopefully by taking it and hopefully by uh providing feedback yeah uh we'll uh providing feedback yeah uh we'll improve uh our common open source and improve uh our common open source and share as much as we can absolutely yeah share as much as we can absolutely yeah well thanks adam it's great to be here well thanks adam it's great to be here and i'm looking forward to to your and i'm looking forward to to your presentation all right thank you yeah presentation all right thank you yeah we're happy to host thanks you