scale.bythebay.io: Vlad Patryshev Interview

I'm Vlad patrushev I worked at I'm Vlad patrushev I worked at Salesforce as a lead data engineer but Salesforce as a lead data engineer but my talk was prepared before I joined my talk was prepared before I joined Salesforce so it was like separate my Salesforce so it was like separate my private vent adventure well see so this private vent adventure well see so this is a pretty advanced company and pretty is a pretty advanced company and pretty high level of programming skills like high level of programming skills like among basically every engineer so the among basically every engineer so the problems I'm trying to solve is they problems I'm trying to solve is they involve like pretty high level scour involve like pretty high level scour issues with the code written by other issues with the code written by other people and designing and developing new people and designing and developing new features and adopting our product to the features and adopting our product to the new things that are around so they're new things that are around so they're like high level and pretty pretty nice like high level and pretty pretty nice and beautiful kind of programming well I and beautiful kind of programming well I would say like in our area we had any would say like in our area we had any issues with scale see a Salesforce uses issues with scale see a Salesforce uses AWS so we have like as main as much AWS so we have like as main as much space as we need and because we're space as we need and because we're developers we don't deal with like developers we don't deal with like terabytes of data we are just preparing terabytes of data we are just preparing the software that will run on those the software that will run on those terabytes and it seemed like that in terabytes and it seemed like that in that relation we have just an open road that relation we have just an open road so we can get as much as we need [Music] [Music] well yeah this I can tell see we are well yeah this I can tell see we are supposed to switch to working with supposed to switch to working with streams of data and what is the stream streams of data and what is the stream like new data coming in and we have to like new data coming in and we have to aggregate them like a maybe in several aggregate them like a maybe in several layers like aggregation per minute per layers like aggregation per minute per hour per week per year so one of the hour per week per year so one of the solutions is to keep all the data and if solutions is to keep all the data and if you want to run like okay so what's an you want to run like okay so what's an average price of something on like Black average price of something on like Black Friday for the last five years so you Friday for the last five years so you have to what either like keep have to what either like keep specifically information on Black Friday specifically information on Black Friday but how did can we guess right like five but how did can we guess right like five years ahead that we need to keep the years ahead that we need to keep the specific data or keep all the data and specific data or keep all the data and then go ahead and scan meaning that will then go ahead and scan meaning that will have like a huge amount of space for have like a huge amount of space for which I am not sure if we are prepared which I am not sure if we are prepared maybe we do it maybe not so that could maybe we do it maybe not so that could be a problem in the future [Music] well she took me some time to get into well she took me some time to get into functional programming but the biggest functional programming but the biggest benefit actually I had not and in this benefit actually I had not and in this company where we are like we are there company where we are like we are there so we don't have like much of a an so we don't have like much of a an opportunity on one hand on the other opportunity on one hand on the other hand it's like working progress there hand it's like working progress there are some things that we should fix but are some things that we should fix but two companies ago I worked I spent three two companies ago I worked I spent three and a half years in a company called and a half years in a company called health expense and there we had the five health expense and there we had the five hundred five hundred service running and hundred five hundred service running and scraping information from insurance scraping information from insurance companies websites on behalf of our companies websites on behalf of our customers and why could we manage to do customers and why could we manage to do it like just several of us well because it like just several of us well because our code was functional we had like no our code was functional we had like no null pointer exceptions per four years null pointer exceptions per four years we took care of all the problem that the we took care of all the problem that the code encounters not the way like by code encounters not the way like by analyzing logs and post factum fix in analyzing logs and post factum fix in our code but making sure that our code our code but making sure that our code is prepared for anything so that's what is prepared for anything so that's what financial programming allows us to do so financial programming allows us to do so we had that opportunity and here in this we had that opportunity and here in this company it seemed like we're already company it seemed like we're already there just just is good you know like there just just is good you know like venue this nice flawed Twitter it's a venue this nice flawed Twitter it's a it's a pretty nice company now so I just it's a pretty nice company now so I just love the like the community of the love the like the community of the people coming and the way it's being people coming and the way it's being done you