sfspark.org: Alexy Khrabrov interviews Dmitriy Setrakyan (GridGain)

[Music] hello everybody I'm Alexi kov uh the hello everybody I'm Alexi kov uh the organizer of SF spark here on location organizer of SF spark here on location at splice machine and the main talk at splice machine and the main talk tonight will be uh about uh aach ignite tonight will be uh about uh aach ignite and G gain and we have with us uhan who and G gain and we have with us uhan who is a chief prot officer uh of gri gain is a chief prot officer uh of gri gain also PMC of Papik Knight and he's also also PMC of Papik Knight and he's also co-founder of G gain great it's great to co-founder of G gain great it's great to have you with us thank you thank you have you with us thank you thank you great tell us a little bit uh how did great tell us a little bit uh how did you come up with this idea of gri gain you come up with this idea of gri gain it's uh it's pretty interesting it's uh it's pretty interesting technology what it is you know how it technology what it is you know how it came to be where it is now well uh came to be where it is now well uh Nikita Nikita my co-founder like the guy Nikita Nikita my co-founder like the guy confounded with and I we had the idea we confounded with and I we had the idea we always uh were part always uh were part of distributed computing and we always of distributed computing and we always were working in all those projects that were working in all those projects that required big scale and required big scale and lots lots of fance and lots lots of fance and scalability so essentially when we just scalability so essentially when we just started I mean the market was not as started I mean the market was not as populated and saturated if you will as populated and saturated if you will as it it is today at that time we had it it is today at that time we had Globus we had Sun engine Sun GD engine Globus we had Sun engine Sun GD engine and probably a couple more and um we and probably a couple more and um we just uh and it was hard it was hard to just uh and it was hard it was hard to use grid Computing actually to spin off use grid Computing actually to spin off a project on lobus probably would take a project on lobus probably would take you maybe several days M so Nikita and I you maybe several days M so Nikita and I we had this idea of grid Computing Made we had this idea of grid Computing Made Simple and that's when we started grid Simple and that's when we started grid gain as an open source project and the gain as an open source project and the whole idea was that you you should be whole idea was that you you should be able to start Computing on a clustered able to start Computing on a clustered environment in a full hor fashion within environment in a full hor fashion within 5 minutes what year was 5 minutes what year was that I'm not going to give you it's that I'm not going to give you it's probably 7even eight years ago okay so probably 7even eight years ago okay so that's before spark yeah but uh yeah that's before spark yeah but uh yeah that's before spark and we started out that's before spark and we started out in a garage kind of mode and uh we're in a garage kind of mode and uh we're working uh after after working hours and working uh after after working hours and then at certain point we land at a then at certain point we land at a Consulting project which we used for Consulting project which we used for sponsoring a team in Russia that helped sponsoring a team in Russia that helped us develop grid game so that's how grid us develop grid game so that's how grid game came out to to be it started out as game came out to to be it started out as a computer grid mhm and uh it actually a computer grid mhm and uh it actually uh became pretty popular from the GetGo uh became pretty popular from the GetGo and as we progress we figured that jazz and as we progress we figured that jazz computer is not going to be enough so we computer is not going to be enough so we added the data site MH and then we had a added the data site MH and then we had a few changes in the company and then we few changes in the company and then we donated our code to Apache ignite and donated our code to Apache ignite and here where Co so so you take GRE here where Co so so you take GRE Computing and so you mentioned sub Computing and so you mentioned sub systems I think this is not kind of the systems I think this is not kind of the traditional distributed systems kind of traditional distributed systems kind of in the uh area right so can you talk a in the uh area right so can you talk a little bit about grid Computing area as little bit about grid Computing area as kind of preced in uh Hadoop and what kind of preced in uh Hadoop and what what is specific what are the customers what is specific what are the customers of this uh you know what is mostly used of this uh you know what is mostly used well U from the uh generally speaking I well U from the uh generally speaking I mean Hadoop and grid gain are designed mean Hadoop and grid gain are designed to solve different types of problems to solve different types of problems Hadoop initially uh was created to take Hadoop initially uh was created to take uh it's a batch oriented system so uh it's a batch oriented system so there's it's very batchy and uh there's it's very batchy and uh essentially if you have a problem that essentially if you have a problem that takes an hour maybe a couple hours and takes an hour maybe a couple hours and you use map Ru you may be able to you use map Ru you may be able to complete it in maybe 30 30 minutes or so complete it in maybe 30 30 minutes or so so it actually provides good so it actually provides good acceleration for those batch oriented acceleration for those batch oriented offline systems that need to Crunch on a offline systems that need to Crunch on a lot of data our Focus was to take a job lot of data our Focus was to take a job that maybe takes a few seconds and that maybe takes a few seconds and execute it in milliseconds MH so it's a execute it in milliseconds MH so it's a real time from the GetGo we designed it real time from the GetGo we designed it as a real time very responsive system M as a real time very responsive system M uh that uh that uh uh was meant to perform and was meant uh uh was meant to perform and was meant to scale so that's where would be the to scale so that's where would be the difference so and all the use cases that difference so and all the use cases that I'm seeing today in Apache ignite there I'm seeing today in Apache ignite there is a lot of use cases that are that are is a lot of use cases that are that are transactional and that require those transactional and that require those hundreds of thousands of uh transactions hundreds of thousands of uh transactions per second MH not to say that we're not per second MH not to say that we're not doing the analytical part but there's a doing the analytical part but there's a lot of a lot of that idea I'm seeing lot of a lot of that idea I'm seeing today when I'm seeing you our users R today when I'm seeing you our users R transactions under huge transactions under huge loads so I I I I think I've read uh agre loads so I I I I think I've read uh agre used on Wall Street and by for trading used on Wall Street and by for trading uh so this is not typical application of uh so this is not typical application of kind of traditional open source we see kind of traditional open source we see around directly right so how you know around directly right so how you know how does that work you know how these how does that work you know how these customers you know use it if you can customers you know use it if you can talk a little bit about it so yeah if talk a little bit about it so yeah if you mention Wall Street then let's talk you mention Wall Street then let's talk about Wall Street a little bit so what about Wall Street a little bit so what is important in financials right I mean is important in financials right I mean Financial uh if you think about Financial uh if you think about financial companies they move money yes financial companies they move money yes uh they don't like mistakes they don't uh they don't like mistakes they don't like down times uh so for them uh to like down times uh so for them uh to move to a new system would be a huge move to a new system would be a huge decision and they and that's why they decision and they and that's why they make moves very rarely and and that's make moves very rarely and and that's why I mean they a lot of times when you why I mean they a lot of times when you walk into a large Wall Street firm walk into a large Wall Street firm you'll see a traditional two three tier you'll see a traditional two three tier architecture where there's one database architecture where there's one database maybe with some replication that gets uh maybe with some replication that gets uh loaded but it's transactional and it loaded but it's transactional and it performs under high Lo and generally the performs under high Lo and generally the main goal the main um goal there to main goal the main um goal there to achieve when they start migrating to an achieve when they start migrating to an in memory architecture is performance in memory architecture is performance and scale so at certain point buying and scale so at certain point buying more Hardware stops working for them and more Hardware stops working for them and uh from skillability standpoint and uh from skillability standpoint and probably from uh economics uh from um probably from uh economics uh from um simply it just gets too pricey and uh simply it just gets too pricey and uh the there comes two features in aide the there comes two features in aide that actually differentiated from uh the that actually differentiated from uh the rest of the products on the market and rest of the products on the market and one is full asset compliance we're not one is full asset compliance we're not eventually consistent we're fully eventually consistent we're fully consistent so you can take an oral consistent so you can take an oral transaction or my transaction and uh run transaction or my transaction and uh run it an ignite or G gain and not only we it an ignite or G gain and not only we will execute it uh in with full asset will execute it uh in with full asset guarantees uh regardless of whatever guarantees uh regardless of whatever failures happen on the cluster we will failures happen on the cluster we will not we will actually make your database not we will actually make your database part of the transaction as well so if part of the transaction as well so if you still want to write to database uh you still want to write to database uh then database transaction will become then database transaction will become part of transaction and another side is part of transaction and another side is SQL so because all these companies are SQL so because all these companies are so database Centric uh they the main so database Centric uh they the main language they talk used to talk to their language they talk used to talk to their data is SQL so our support for SQL with data is SQL so our support for SQL with strong indexing so we care about strong indexing so we care about performance wi index a lot makes it a performance wi index a lot makes it a lucrative solution stre interesting this lucrative solution stre interesting this actually very interesting because uh SP actually very interesting because uh SP machine itself is actually doing rgbm machine itself is actually doing rgbm which works on app and LP so uh I just which works on app and LP so uh I just talk about M as been the C so this is talk about M as been the C so this is very I think like what I realized when very I think like what I realized when we're hosting you know uh g game up at we're hosting you know uh g game up at splice machine actually we fin a lot of splice machine actually we fin a lot of synergies so I think you know what would synergies so I think you know what would be interesting tonight uh is kind of uh be interesting tonight uh is kind of uh I think we'll kind of these topics will I think we'll kind of these topics will be raised right and maybe kind of be raised right and maybe kind of questions kind of can come up which will questions kind of can come up which will not come up by itself but basically have not come up by itself but basically have two different companies probably two different companies probably addressing two different markets but addressing two different markets but kind of talking about you know SQL asset kind of talking about you know SQL asset right and you know Essen going after right and you know Essen going after Oracle from different angles from Oracle from different angles from different from different performance different from different performance angles different use case angles yes angles different use case angles yes yeah yeah yeah yeah this exciting so yeah yeah yeah yeah this exciting so tell me how so I think Apachi U ignite tell me how so I think Apachi U ignite started a year ago so not or two years started a year ago so not or two years ago kind of putting to a test with this ago kind of putting to a test with this dates not too not too long I think uh dates not too not too long I think uh year over a year so over year ago so uh year over a year so over year ago so uh there are two parts right there's there are two parts right there's incubation and then there is graduation incubation and then there is graduation yes so if you start from when we donated yes so if you start from when we donated uh the code to Apache it was 2 years ago uh the code to Apache it was 2 years ago yep and and we graduated within 9 months yep and and we graduated within 9 months so we graduated pretty fast and uh the so we graduated pretty fast and uh the reason for that was the very rapidly reason for that was the very rapidly growing Community oh so it's uh we're I growing Community oh so it's uh we're I I mean if you put spark outside I think I mean if you put spark outside I think we're one of the fastest uh growing we're one of the fastest uh growing projects in projects in Apache and uh uh the community is very Apache and uh uh the community is very responsive the community is very willing responsive the community is very willing to uh engage and uh the stuff we're to uh engage and uh the stuff we're working on is very interesting I me it's working on is very interesting I me it's all a lot of distributed stuff all a lot of distributed stuff distributed locks transactions distributed locks transactions distributed data consistency so all distributed data consistency so all these problems uh create like this these problems uh create like this unique environment where you can get one unique environment where you can get one of experience that you wouldn't get of experience that you wouldn't get anywhere else and that attracts a lot of anywhere else and that attracts a lot of commuters and a lot of contributors to commuters and a lot of contributors to the project so we've been growing ever the project so we've been growing ever since cool yeah that was actually my since cool yeah that was actually my question and how how it affected you question and how how it affected you guys but how did it affect the company guys but how did it affect the company right so basically you know you have right so basically you know you have dual roles uh does it mean you spend dual roles uh does it mean you spend more time in the open source or does it more time in the open source or does it kind of like do you have a lot of kind of like do you have a lot of synergies so it kind of feeds the synergies so it kind of feeds the business how do you kind of connect this business how do you kind of connect this to the two two things well I mean one to the two two things well I mean one first and foremost I mean it's very first and foremost I mean it's very important that as a grid gain as a the important that as a grid gain as a the all the features we provide on top of a all the features we provide on top of a Patrick Knight so we first of all the Patrick Knight so we first of all the difference between grid G and Patrick difference between grid G and Patrick Knight is g game provides Enterprise Knight is g game provides Enterprise certain Enterprise level features like certain Enterprise level features like data center application production data center application production upgrade or rolling upgrades uh security upgrade or rolling upgrades uh security so things that you would need in serious so things that you would need in serious Enterprise deployment but you probably Enterprise deployment but you probably could be get by in could be get by in simpler open source type of project simpler open source type of project deployments MH and the uh the main uh deployments MH and the uh the main uh goal that we we're trying to pursue here goal that we we're trying to pursue here is that you should not be worried when is that you should not be worried when you write a code with when you code to a you write a code with when you code to a pach you should not be wor which Edition pach you should not be wor which Edition whether it's a patch ignite or GD G whether it's a patch ignite or GD G Enterprise Edition use so GD gain Enterprise Edition use so GD gain actually does not introduce any new API actually does not introduce any new API most of the uh uh most of the most of the uh uh most of the functionality that grid game adds is a functionality that grid game adds is a configuration okay so you want security configuration okay so you want security it's a configuration you want uh rolling it's a configuration you want uh rolling production upgrades it's a flack you production upgrades it's a flack you enable on a configuration you want uh enable on a configuration you want uh data center application again it's data center application again it's configuration if you want to get some configuration if you want to get some metrics specific to Data Center metrics specific to Data Center application then yeah there's some API application then yeah there's some API there but generally if you use just there but generally if you use just plain vanilla patch that you can plain vanilla patch that you can continue using it in G cool cool cool so continue using it in G cool cool cool so this is this is really exciting uh we're this is this is really exciting uh we're looking forward to your technical talk looking forward to your technical talk and thanks you know thanks very much for and thanks you know thanks very much for coming all right thank you good to be coming all right thank you good to be here [Music]