WEBVTT Kind: captions Language: en 00:00:17.849 --> 00:00:20.970 align:start position:0% hello<00:00:18.849> everybody<00:00:19.230> I'm<00:00:20.230> electric<00:00:20.619> robber<00:00:20.860> of 00:00:20.970 --> 00:00:20.980 align:start position:0% hello everybody I'm electric robber of 00:00:20.980 --> 00:00:23.490 align:start position:0% hello everybody I'm electric robber of the<00:00:21.190> organizer<00:00:21.730> of<00:00:21.850> SF<00:00:22.240> Sparkle<00:00:22.630> friends<00:00:22.990> give 00:00:23.490 --> 00:00:23.500 align:start position:0% the organizer of SF Sparkle friends give 00:00:23.500 --> 00:00:26.670 align:start position:0% the organizer of SF Sparkle friends give it<00:00:23.619> up<00:00:23.800> about<00:00:24.550> spark<00:00:25.029> in<00:00:25.180> its<00:00:25.300> ecosystem<00:00:25.960> such 00:00:26.670 --> 00:00:26.680 align:start position:0% it up about spark in its ecosystem such 00:00:26.680 --> 00:00:29.370 align:start position:0% it up about spark in its ecosystem such as<00:00:26.830> Sakha<00:00:27.039> Kafka<00:00:27.550> Cassandra<00:00:28.180> Pasha<00:00:28.689> blink<00:00:29.020> and 00:00:29.370 --> 00:00:29.380 align:start position:0% as Sakha Kafka Cassandra Pasha blink and 00:00:29.380 --> 00:00:31.200 align:start position:0% as Sakha Kafka Cassandra Pasha blink and you<00:00:29.800> see<00:00:29.920> simple<00:00:30.189> men<00:00:30.340> date<00:00:30.550> that<00:00:30.820> will<00:00:30.939> hear 00:00:31.200 --> 00:00:31.210 align:start position:0% you see simple men date that will hear 00:00:31.210 --> 00:00:33.660 align:start position:0% you see simple men date that will hear about<00:00:31.300> later<00:00:31.930> today<00:00:32.230> and<00:00:32.860> we're<00:00:33.340> very<00:00:33.370> happy 00:00:33.660 --> 00:00:33.670 align:start position:0% about later today and we're very happy 00:00:33.670 --> 00:00:36.330 align:start position:0% about later today and we're very happy to<00:00:33.820> have<00:00:33.940> here<00:00:34.420> without<00:00:34.540> send<00:00:35.080> ariza<00:00:35.379> who<00:00:36.190> is 00:00:36.330 --> 00:00:36.340 align:start position:0% to have here without send ariza who is 00:00:36.340 --> 00:00:38.490 align:start position:0% to have here without send ariza who is the<00:00:36.489> senior<00:00:36.879> data<00:00:37.330> scientist<00:00:37.720> at<00:00:37.839> caldera<00:00:38.290> and 00:00:38.490 --> 00:00:38.500 align:start position:0% the senior data scientist at caldera and 00:00:38.500 --> 00:00:40.530 align:start position:0% the senior data scientist at caldera and the<00:00:38.860> author<00:00:39.040> of<00:00:39.280> the<00:00:39.670> new<00:00:39.790> book<00:00:39.940> from<00:00:40.150> o'reilly 00:00:40.530 --> 00:00:40.540 align:start position:0% the author of the new book from o'reilly 00:00:40.540 --> 00:00:43.890 align:start position:0% the author of the new book from o'reilly called<00:00:40.780> advanced<00:00:41.110> analytical<00:00:41.650> spark<00:00:42.900> which 00:00:43.890 --> 00:00:43.900 align:start position:0% called advanced analytical spark which 00:00:43.900 --> 00:00:45.149 align:start position:0% called advanced analytical spark which we<00:00:44.049> have<00:00:44.199> multiple<00:00:44.530> copies<00:00:44.830> here<00:00:44.949> in<00:00:45.100> the 00:00:45.149 --> 00:00:45.159 align:start position:0% we have multiple copies here in the 00:00:45.159 --> 00:00:46.740 align:start position:0% we have multiple copies here in the office<00:00:45.309> and<00:00:45.489> they're<00:00:46.030> very<00:00:46.059> much<00:00:46.510> enjoyment 00:00:46.740 --> 00:00:46.750 align:start position:0% office and they're very much enjoyment 00:00:46.750 --> 00:00:50.700 align:start position:0% office and they're very much enjoyment great<00:00:47.199> to<00:00:47.290> hear<00:00:47.729> and<00:00:48.729> the<00:00:49.150> topic<00:00:49.870> of<00:00:50.199> Senna's 00:00:50.700 --> 00:00:50.710 align:start position:0% great to hear and the topic of Senna's 00:00:50.710 --> 00:00:55.049 align:start position:0% great to hear and the topic of Senna's talk<00:00:51.479> about<00:00:52.479> semantic<00:00:53.470> analysis<00:00:53.949> of<00:00:54.220> formula 00:00:55.049 --> 00:00:55.059 align:start position:0% talk about semantic analysis of formula 00:00:55.059 --> 00:00:57.299 align:start position:0% talk about semantic analysis of formula documents<00:00:55.479> the<00:00:55.839> patches<00:00:56.080> park<00:00:56.320> is<00:00:56.530> highly 00:00:57.299 --> 00:00:57.309 align:start position:0% documents the patches park is highly 00:00:57.309 --> 00:00:59.250 align:start position:0% documents the patches park is highly relevant<00:00:57.549> nitra<00:00:58.269> which<00:00:58.600> is<00:00:58.780> a<00:00:58.809> document 00:00:59.250 --> 00:00:59.260 align:start position:0% relevant nitra which is a document 00:00:59.260 --> 00:01:02.100 align:start position:0% relevant nitra which is a document productivity<00:00:59.860> company<00:01:00.420> we<00:01:01.420> work<00:01:01.720> with 00:01:02.100 --> 00:01:02.110 align:start position:0% productivity company we work with 00:01:02.110 --> 00:01:04.109 align:start position:0% productivity company we work with hundreds<00:01:02.830> of<00:01:02.860> millions<00:01:03.190> of<00:01:03.250> pdfs<00:01:03.610> want<00:01:04.030> to 00:01:04.109 --> 00:01:04.119 align:start position:0% hundreds of millions of pdfs want to 00:01:04.119 --> 00:01:06.090 align:start position:0% hundreds of millions of pdfs want to understand<00:01:04.600> what<00:01:04.780> they<00:01:04.900> are<00:01:04.930> evolved<00:01:05.229> so<00:01:05.680> this 00:01:06.090 --> 00:01:06.100 align:start position:0% understand what they are evolved so this 00:01:06.100 --> 00:01:08.429 align:start position:0% understand what they are evolved so this is<00:01:06.130> the<00:01:06.909> technology<00:01:07.150> we<00:01:07.450> chose<00:01:07.930> to<00:01:07.960> use<00:01:08.200> as 00:01:08.429 --> 00:01:08.439 align:start position:0% is the technology we chose to use as 00:01:08.439 --> 00:01:10.260 align:start position:0% is the technology we chose to use as well<00:01:08.619> so<00:01:09.310> welcome<00:01:09.670> to<00:01:09.700> india<00:01:09.939> is<00:01:10.000> great<00:01:10.180> to 00:01:10.260 --> 00:01:10.270 align:start position:0% well so welcome to india is great to 00:01:10.270 --> 00:01:12.929 align:start position:0% well so welcome to india is great to have<00:01:10.390> you<00:01:10.450> here<00:01:10.479> thanks<00:01:11.409> Evan<00:01:11.590> here<00:01:11.829> so<00:01:12.549> I<00:01:12.579> did 00:01:12.929 --> 00:01:12.939 align:start position:0% have you here thanks Evan here so I did 00:01:12.939 --> 00:01:16.440 align:start position:0% have you here thanks Evan here so I did some<00:01:13.659> research<00:01:14.109> on<00:01:14.590> you<00:01:14.740> and<00:01:15.640> the<00:01:16.000> cold<00:01:16.299> air 00:01:16.440 --> 00:01:16.450 align:start position:0% some research on you and the cold air 00:01:16.450 --> 00:01:19.740 align:start position:0% some research on you and the cold air has<00:01:16.689> a<00:01:16.719> nice<00:01:17.460> blog<00:01:18.460> post<00:01:18.759> type<00:01:19.030> of<00:01:19.180> meeting<00:01:19.450> the 00:01:19.740 --> 00:01:19.750 align:start position:0% has a nice blog post type of meeting the 00:01:19.750 --> 00:01:22.649 align:start position:0% has a nice blog post type of meeting the data<00:01:19.899> scientists<00:01:20.350> and<00:01:21.270> and<00:01:22.270> we<00:01:22.479> both<00:01:22.600> know 00:01:22.649 --> 00:01:22.659 align:start position:0% data scientists and and we both know 00:01:22.659 --> 00:01:25.550 align:start position:0% data scientists and and we both know Josh<00:01:23.020> real<00:01:23.229> pretty<00:01:23.439> well<00:01:23.619> so<00:01:23.799> it's<00:01:24.100> kind<00:01:24.340> of 00:01:25.550 --> 00:01:25.560 align:start position:0% Josh real pretty well so it's kind of 00:01:25.560 --> 00:01:27.929 align:start position:0% Josh real pretty well so it's kind of expanding<00:01:26.560> on<00:01:26.649> Josh<00:01:26.799> reals<00:01:27.009> theory<00:01:27.280> of<00:01:27.490> what<00:01:27.880> a 00:01:27.929 --> 00:01:27.939 align:start position:0% expanding on Josh reals theory of what a 00:01:27.939 --> 00:01:30.510 align:start position:0% expanding on Josh reals theory of what a data<00:01:28.149> scientist<00:01:28.570> is<00:01:28.719> a<00:01:29.409> little<00:01:29.950> bit<00:01:30.100> and<00:01:30.310> that 00:01:30.510 --> 00:01:30.520 align:start position:0% data scientist is a little bit and that 00:01:30.520 --> 00:01:32.760 align:start position:0% data scientist is a little bit and that but<00:01:30.729> that<00:01:30.850> was<00:01:31.030> a<00:01:31.060> thing<00:01:31.329> a<00:01:31.420> year<00:01:31.600> ago<00:01:31.719> so<00:01:32.560> I'm 00:01:32.760 --> 00:01:32.770 align:start position:0% but that was a thing a year ago so I'm 00:01:32.770 --> 00:01:34.859 align:start position:0% but that was a thing a year ago so I'm wondering<00:01:32.920> so<00:01:33.579> it<00:01:33.820> basically<00:01:33.939> can<00:01:34.600> appreciate 00:01:34.859 --> 00:01:34.869 align:start position:0% wondering so it basically can appreciate 00:01:34.869 --> 00:01:37.889 align:start position:0% wondering so it basically can appreciate that<00:01:35.710> here<00:01:35.950> the<00:01:36.249> data<00:01:36.429> scientist<00:01:36.880> is<00:01:37.090> a<00:01:37.119> little 00:01:37.889 --> 00:01:37.899 align:start position:0% that here the data scientist is a little 00:01:37.899 --> 00:01:41.669 align:start position:0% that here the data scientist is a little janitor<00:01:38.259> and<00:01:38.829> and<00:01:40.079> has<00:01:41.079> also<00:01:41.350> some 00:01:41.669 --> 00:01:41.679 align:start position:0% janitor and and has also some 00:01:41.679 --> 00:01:44.550 align:start position:0% janitor and and has also some interesting<00:01:42.070> points<00:01:42.600> but<00:01:43.600> I<00:01:43.719> wondering<00:01:44.170> you 00:01:44.550 --> 00:01:44.560 align:start position:0% interesting points but I wondering you 00:01:44.560 --> 00:01:46.230 align:start position:0% interesting points but I wondering you know<00:01:44.590> after<00:01:44.829> that<00:01:45.070> year<00:01:45.520> hope<00:01:45.820> they're<00:01:46.030> wrong 00:01:46.230 --> 00:01:46.240 align:start position:0% know after that year hope they're wrong 00:01:46.240 --> 00:01:48.590 align:start position:0% know after that year hope they're wrong and<00:01:46.450> I'm<00:01:47.079> special<00:01:47.439> after<00:01:47.679> writing<00:01:47.829> this<00:01:48.100> book 00:01:48.590 --> 00:01:48.600 align:start position:0% and I'm special after writing this book 00:01:48.600 --> 00:01:54.929 align:start position:0% and I'm special after writing this book your<00:01:49.600> initial<00:01:50.310> vision<00:01:51.310> change<00:01:51.939> or<00:01:53.670> was<00:01:54.670> there 00:01:54.929 --> 00:01:54.939 align:start position:0% your initial vision change or was there 00:01:54.939 --> 00:01:57.120 align:start position:0% your initial vision change or was there some<00:01:55.210> evolution<00:01:55.719> of<00:01:55.869> this<00:01:56.049> and<00:01:56.409> what<00:01:56.619> what<00:01:57.009> it 00:01:57.120 --> 00:01:57.130 align:start position:0% some evolution of this and what what it 00:01:57.130 --> 00:01:59.609 align:start position:0% some evolution of this and what what it is<00:01:57.600> that's<00:01:58.600> a<00:01:58.689> great<00:01:58.810> question<00:01:58.990> yeah<00:01:59.319> I<00:01:59.499> would 00:01:59.609 --> 00:01:59.619 align:start position:0% is that's a great question yeah I would 00:01:59.619 --> 00:02:01.480 align:start position:0% is that's a great question yeah I would say<00:01:59.770> that<00:01:59.829> probably<00:02:00.219> if<00:02:00.700> anything<00:02:01.060> my<00:02:01.149> vision 00:02:01.480 --> 00:02:01.490 align:start position:0% say that probably if anything my vision 00:02:01.490 --> 00:02:05.260 align:start position:0% say that probably if anything my vision then<00:02:01.729> reinforced<00:02:02.509> in<00:02:02.750> that<00:02:03.110> a<00:02:03.979> lot<00:02:04.909> of<00:02:05.000> my<00:02:05.060> job 00:02:05.260 --> 00:02:05.270 align:start position:0% then reinforced in that a lot of my job 00:02:05.270 --> 00:02:07.030 align:start position:0% then reinforced in that a lot of my job is<00:02:05.299> doing<00:02:05.659> engagements<00:02:06.259> at<00:02:06.409> different 00:02:07.030 --> 00:02:07.040 align:start position:0% is doing engagements at different 00:02:07.040 --> 00:02:09.309 align:start position:0% is doing engagements at different customers<00:02:08.000> oh<00:02:08.239> yeah<00:02:08.569> help<00:02:08.840> you<00:02:08.899> fellow<00:02:09.110> come 00:02:09.309 --> 00:02:09.319 align:start position:0% customers oh yeah help you fellow come 00:02:09.319 --> 00:02:11.770 align:start position:0% customers oh yeah help you fellow come something<00:02:09.619> you<00:02:09.709> get<00:02:09.860> it<00:02:10.009> set<00:02:10.220> up<00:02:10.250> for<00:02:10.780> whatever 00:02:11.770 --> 00:02:11.780 align:start position:0% something you get it set up for whatever 00:02:11.780 --> 00:02:14.470 align:start position:0% something you get it set up for whatever days<00:02:11.989> whatever<00:02:12.349> people<00:02:12.650> want<00:02:12.800> um<00:02:13.640> and<00:02:14.090> very 00:02:14.470 --> 00:02:14.480 align:start position:0% days whatever people want um and very 00:02:14.480 --> 00:02:16.000 align:start position:0% days whatever people want um and very rarely<00:02:14.840> does<00:02:15.080> it<00:02:15.200> come<00:02:15.380> down<00:02:15.530> to<00:02:15.560> actually<00:02:15.980> i 00:02:16.000 --> 00:02:16.010 align:start position:0% rarely does it come down to actually i 00:02:16.010 --> 00:02:17.199 align:start position:0% rarely does it come down to actually i would<00:02:16.160> say<00:02:16.250> applying<00:02:16.580> machine<00:02:16.970> learning 00:02:17.199 --> 00:02:17.209 align:start position:0% would say applying machine learning 00:02:17.209 --> 00:02:21.640 align:start position:0% would say applying machine learning algorithm<00:02:17.420> um<00:02:17.989> often<00:02:18.830> people<00:02:19.300> turn<00:02:20.300> to<00:02:20.650> people 00:02:21.640 --> 00:02:21.650 align:start position:0% algorithm um often people turn to people 00:02:21.650 --> 00:02:23.020 align:start position:0% algorithm um often people turn to people were<00:02:21.830> called<00:02:22.130> data<00:02:22.280> scientists<00:02:22.819> to<00:02:22.880> have 00:02:23.020 --> 00:02:23.030 align:start position:0% were called data scientists to have 00:02:23.030 --> 00:02:25.270 align:start position:0% were called data scientists to have someone<00:02:23.300> to<00:02:23.360> think<00:02:24.110> about<00:02:24.290> the<00:02:24.410> data<00:02:24.620> reason 00:02:25.270 --> 00:02:25.280 align:start position:0% someone to think about the data reason 00:02:25.280 --> 00:02:28.509 align:start position:0% someone to think about the data reason about<00:02:25.519> it<00:02:25.930> and<00:02:26.930> try<00:02:27.560> that<00:02:27.739> sort<00:02:27.950> through<00:02:28.099> do 00:02:28.509 --> 00:02:28.519 align:start position:0% about it and try that sort through do 00:02:28.519 --> 00:02:30.580 align:start position:0% about it and try that sort through do the<00:02:28.610> same<00:02:28.790> common<00:02:29.120> sense<00:02:29.390> stuff<00:02:29.989> that<00:02:30.349> you 00:02:30.580 --> 00:02:30.590 align:start position:0% the same common sense stuff that you 00:02:30.590 --> 00:02:33.250 align:start position:0% the same common sense stuff that you know<00:02:30.680> you<00:02:31.099> might<00:02:31.280> use<00:02:31.640> another<00:02:31.849> spheres<00:02:32.330> but<00:02:32.540> i 00:02:33.250 --> 00:02:33.260 align:start position:0% know you might use another spheres but i 00:02:33.260 --> 00:02:34.900 align:start position:0% know you might use another spheres but i don't<00:02:33.410> know<00:02:33.500> some<00:02:33.860> people<00:02:34.040> i'm<00:02:34.340> confident<00:02:34.760> in 00:02:34.900 --> 00:02:34.910 align:start position:0% don't know some people i'm confident in 00:02:34.910 --> 00:02:37.059 align:start position:0% don't know some people i'm confident in doing<00:02:35.060> that<00:02:35.330> in<00:02:36.049> relation<00:02:36.170> to<00:02:36.470> Lord<00:02:36.500> data<00:02:36.799> sets 00:02:37.059 --> 00:02:37.069 align:start position:0% doing that in relation to Lord data sets 00:02:37.069 --> 00:02:40.360 align:start position:0% doing that in relation to Lord data sets um<00:02:37.630> so<00:02:38.630> yeah<00:02:39.200> if<00:02:39.290> anything<00:02:39.440> I<00:02:39.739> would<00:02:39.799> say<00:02:40.100> that 00:02:40.360 --> 00:02:40.370 align:start position:0% um so yeah if anything I would say that 00:02:40.370 --> 00:02:45.400 align:start position:0% um so yeah if anything I would say that uh<00:02:40.850> that<00:02:41.720> a<00:02:42.530> Chandler<00:02:43.220> Bing<00:02:43.700> is<00:02:44.090> still<00:02:44.690> some 00:02:45.400 --> 00:02:45.410 align:start position:0% uh that a Chandler Bing is still some 00:02:45.410 --> 00:02:46.780 align:start position:0% uh that a Chandler Bing is still some huge<00:02:45.650> percent<00:02:45.950> of<00:02:46.010> it<00:02:46.130> and<00:02:46.310> then<00:02:46.430> another 00:02:46.780 --> 00:02:46.790 align:start position:0% huge percent of it and then another 00:02:46.790 --> 00:02:48.009 align:start position:0% huge percent of it and then another percent<00:02:47.180> of<00:02:47.239> it<00:02:47.330> before<00:02:47.600> you<00:02:47.660> even<00:02:47.750> get<00:02:47.959> to 00:02:48.009 --> 00:02:48.019 align:start position:0% percent of it before you even get to 00:02:48.019 --> 00:02:49.660 align:start position:0% percent of it before you even get to doing<00:02:48.290> math<00:02:48.470> or<00:02:48.650> statistics<00:02:48.769> it's<00:02:49.310> just<00:02:49.489> sort 00:02:49.660 --> 00:02:49.670 align:start position:0% doing math or statistics it's just sort 00:02:49.670 --> 00:02:52.690 align:start position:0% doing math or statistics it's just sort of<00:02:49.780> what<00:02:50.780> makes<00:02:51.140> sense<00:02:51.410> here<00:02:51.590> gorilla<00:02:52.130> right 00:02:52.690 --> 00:02:52.700 align:start position:0% of what makes sense here gorilla right 00:02:52.700 --> 00:02:54.759 align:start position:0% of what makes sense here gorilla right over<00:02:52.910> there<00:02:53.120> I<00:02:53.330> think<00:02:53.600> there<00:02:54.170> is<00:02:54.200> a<00:02:54.350> number 00:02:54.759 --> 00:02:54.769 align:start position:0% over there I think there is a number 00:02:54.769 --> 00:02:56.500 align:start position:0% over there I think there is a number falls<00:02:54.950> in<00:02:55.130> around<00:02:55.340> that<00:02:55.580> eighty<00:02:55.760> percent<00:02:55.850> of 00:02:56.500 --> 00:02:56.510 align:start position:0% falls in around that eighty percent of 00:02:56.510 --> 00:02:59.890 align:start position:0% falls in around that eighty percent of identify<00:02:57.350> his<00:02:57.530> job<00:02:57.799> spam<00:02:58.549> Monching<00:02:59.540> the<00:02:59.690> data 00:02:59.890 --> 00:02:59.900 align:start position:0% identify his job spam Monching the data 00:02:59.900 --> 00:03:01.509 align:start position:0% identify his job spam Monching the data do<00:03:00.410> you<00:03:00.470> think<00:03:00.680> that's<00:03:00.860> roughly<00:03:01.130> the<00:03:01.489> right 00:03:01.509 --> 00:03:01.519 align:start position:0% do you think that's roughly the right 00:03:01.519 --> 00:03:03.610 align:start position:0% do you think that's roughly the right ratio<00:03:02.209> I<00:03:02.599> think<00:03:02.959> that's<00:03:03.080> perf<00:03:03.230> little<00:03:03.500> better 00:03:03.610 --> 00:03:03.620 align:start position:0% ratio I think that's perf little better 00:03:03.620 --> 00:03:08.349 align:start position:0% ratio I think that's perf little better ratio<00:03:04.330> and<00:03:05.330> so<00:03:05.510> I<00:03:05.540> mean<00:03:05.810> in<00:03:05.989> my<00:03:06.200> theory<00:03:07.160> I<00:03:07.430> I 00:03:08.349 --> 00:03:08.359 align:start position:0% ratio and so I mean in my theory I I 00:03:08.359 --> 00:03:11.590 align:start position:0% ratio and so I mean in my theory I I kind<00:03:08.660> of<00:03:08.720> also<00:03:08.959> have<00:03:09.230> some<00:03:09.440> kind<00:03:09.680> of<00:03:10.600> simple 00:03:11.590 --> 00:03:11.600 align:start position:0% kind of also have some kind of simple 00:03:11.600 --> 00:03:15.039 align:start position:0% kind of also have some kind of simple way<00:03:11.720> sore<00:03:12.019> eyes<00:03:12.140> were<00:03:12.290> the<00:03:12.350> scientists<00:03:13.480> are<00:03:14.480> so 00:03:15.039 --> 00:03:15.049 align:start position:0% way sore eyes were the scientists are so 00:03:15.049 --> 00:03:16.090 align:start position:0% way sore eyes were the scientists are so I<00:03:15.079> wonder<00:03:15.350> I<00:03:15.470> know<00:03:15.650> or<00:03:15.769> they<00:03:15.799> kind<00:03:16.040> of 00:03:16.090 --> 00:03:16.100 align:start position:0% I wonder I know or they kind of 00:03:16.100 --> 00:03:18.009 align:start position:0% I wonder I know or they kind of exaggerated<00:03:16.850> but<00:03:16.970> I<00:03:17.030> will<00:03:17.269> run<00:03:17.450> the<00:03:17.570> Bayou<00:03:17.750> so 00:03:18.009 --> 00:03:18.019 align:start position:0% exaggerated but I will run the Bayou so 00:03:18.019 --> 00:03:22.660 align:start position:0% exaggerated but I will run the Bayou so one<00:03:18.680> theory<00:03:18.890> runs<00:03:19.190> that<00:03:19.370> no<00:03:19.810> it's<00:03:20.810> also<00:03:21.140> a<00:03:21.670> data 00:03:22.660 --> 00:03:22.670 align:start position:0% one theory runs that no it's also a data 00:03:22.670 --> 00:03:25.110 align:start position:0% one theory runs that no it's also a data scientist<00:03:23.150> a<00:03:23.510> small<00:03:23.930> insight<00:03:24.350> is<00:03:24.560> a<00:03:24.590> Python 00:03:25.110 --> 00:03:25.120 align:start position:0% scientist a small insight is a Python 00:03:25.120 --> 00:03:27.580 align:start position:0% scientist a small insight is a Python programmer<00:03:26.120> cook<00:03:26.329> and<00:03:26.540> you<00:03:26.780> know<00:03:26.930> sort<00:03:27.350> and 00:03:27.580 --> 00:03:27.590 align:start position:0% programmer cook and you know sort and 00:03:27.590 --> 00:03:31.000 align:start position:0% programmer cook and you know sort and ready<00:03:27.910> right<00:03:28.910> and<00:03:29.329> and<00:03:29.570> and<00:03:30.410> because 00:03:31.000 --> 00:03:31.010 align:start position:0% ready right and and and because 00:03:31.010 --> 00:03:33.430 align:start position:0% ready right and and and because typically<00:03:31.880> i<00:03:32.299> wonder<00:03:32.780> if<00:03:32.840> you<00:03:32.900> agree<00:03:33.079> that 00:03:33.430 --> 00:03:33.440 align:start position:0% typically i wonder if you agree that 00:03:33.440 --> 00:03:35.830 align:start position:0% typically i wonder if you agree that most<00:03:33.680> algorithms<00:03:34.100> most<00:03:34.970> results<00:03:35.569> can<00:03:35.750> be 00:03:35.830 --> 00:03:35.840 align:start position:0% most algorithms most results can be 00:03:35.840 --> 00:03:37.840 align:start position:0% most algorithms most results can be obtained<00:03:36.079> with<00:03:36.260> the<00:03:36.350> elder<00:03:36.650> a<00:03:36.680> delicious<00:03:37.069> sort 00:03:37.840 --> 00:03:37.850 align:start position:0% obtained with the elder a delicious sort 00:03:37.850 --> 00:03:40.930 align:start position:0% obtained with the elder a delicious sort uniq<00:03:38.239> take<00:03:39.140> top<00:03:39.380> tab<00:03:39.620> right<00:03:40.370> which<00:03:40.640> is<00:03:40.670> super 00:03:40.930 --> 00:03:40.940 align:start position:0% uniq take top tab right which is super 00:03:40.940 --> 00:03:42.580 align:start position:0% uniq take top tab right which is super easy<00:03:41.060> to<00:03:41.299> do<00:03:41.540> a<00:03:41.569> new<00:03:41.750> leak<00:03:41.930> stories<00:03:42.380> because 00:03:42.580 --> 00:03:42.590 align:start position:0% easy to do a new leak stories because 00:03:42.590 --> 00:03:44.289 align:start position:0% easy to do a new leak stories because you<00:03:42.709> can<00:03:42.859> fight<00:03:43.040> these<00:03:43.220> things<00:03:43.280> right<00:03:43.850> and<00:03:44.060> you 00:03:44.289 --> 00:03:44.299 align:start position:0% you can fight these things right and you 00:03:44.299 --> 00:03:46.210 align:start position:0% you can fight these things right and you can<00:03:44.480> but<00:03:44.900> you<00:03:44.930> do<00:03:45.109> explore<00:03:45.380> it<00:03:45.650> at<00:03:45.769> once<00:03:46.010> you 00:03:46.210 --> 00:03:46.220 align:start position:0% can but you do explore it at once you 00:03:46.220 --> 00:03:48.729 align:start position:0% can but you do explore it at once you can<00:03:46.370> very<00:03:46.519> easily<00:03:46.819> as<00:03:47.530> education<00:03:48.530> whether 00:03:48.729 --> 00:03:48.739 align:start position:0% can very easily as education whether 00:03:48.739 --> 00:03:50.319 align:start position:0% can very easily as education whether you're<00:03:48.920> sorting<00:03:49.340> right<00:03:49.850> so<00:03:50.030> once<00:03:50.209> you<00:03:50.299> have 00:03:50.319 --> 00:03:50.329 align:start position:0% you're sorting right so once you have 00:03:50.329 --> 00:03:52.479 align:start position:0% you're sorting right so once you have something<00:03:50.780> to<00:03:50.810> soar<00:03:51.109> it<00:03:51.319> then<00:03:51.950> it's<00:03:52.220> probably 00:03:52.479 --> 00:03:52.489 align:start position:0% something to soar it then it's probably 00:03:52.489 --> 00:03:53.949 align:start position:0% something to soar it then it's probably was<00:03:52.790> going<00:03:52.970> to<00:03:53.030> be<00:03:53.120> here<00:03:53.329> just<00:03:53.480> trying<00:03:53.720> to<00:03:53.780> get 00:03:53.949 --> 00:03:53.959 align:start position:0% was going to be here just trying to get 00:03:53.959 --> 00:03:54.740 align:start position:0% was going to be here just trying to get things<00:03:54.139> you<00:03:54.410> know 00:03:54.740 --> 00:03:54.750 align:start position:0% things you know 00:03:54.750 --> 00:03:56.240 align:start position:0% things you know in<00:03:54.810> the<00:03:54.990> state<00:03:55.590> that<00:03:55.620> they<00:03:55.800> can<00:03:55.980> be<00:03:56.040> ordered 00:03:56.240 --> 00:03:56.250 align:start position:0% in the state that they can be ordered 00:03:56.250 --> 00:03:57.770 align:start position:0% in the state that they can be ordered that's<00:03:56.760> right<00:03:56.850> you<00:03:57.030> some<00:03:57.300> shorts<00:03:57.630> and 00:03:57.770 --> 00:03:57.780 align:start position:0% that's right you some shorts and 00:03:57.780 --> 00:03:59.540 align:start position:0% that's right you some shorts and transformed<00:03:58.170> into<00:03:58.590> pulse<00:03:58.950> is<00:03:59.130> sorting<00:03:59.400> some 00:03:59.540 --> 00:03:59.550 align:start position:0% transformed into pulse is sorting some 00:03:59.550 --> 00:04:01.520 align:start position:0% transformed into pulse is sorting some field<00:03:59.850> and<00:04:00.060> then<00:04:00.480> you<00:04:00.600> get<00:04:00.750> the<00:04:00.870> red<00:04:00.990> thread<00:04:01.290> so 00:04:01.520 --> 00:04:01.530 align:start position:0% field and then you get the red thread so 00:04:01.530 --> 00:04:05.720 align:start position:0% field and then you get the red thread so so<00:04:02.040> this<00:04:02.220> code<00:04:02.610> usually<00:04:03.240> starts<00:04:04.340> in<00:04:05.340> spalding 00:04:05.720 --> 00:04:05.730 align:start position:0% so this code usually starts in spalding 00:04:05.730 --> 00:04:08.510 align:start position:0% so this code usually starts in spalding so<00:04:06.000> the<00:04:06.210> big<00:04:06.570> date<00:04:06.840> as<00:04:07.140> I<00:04:07.290> this<00:04:07.350> is<00:04:07.740> somebody 00:04:08.510 --> 00:04:08.520 align:start position:0% so the big date as I this is somebody 00:04:08.520 --> 00:04:11.390 align:start position:0% so the big date as I this is somebody who<00:04:08.580> can<00:04:08.820> sort<00:04:09.060> the<00:04:09.210> array<00:04:09.240> loop<00:04:09.780> or<00:04:10.070> spark<00:04:11.070> or 00:04:11.390 --> 00:04:11.400 align:start position:0% who can sort the array loop or spark or 00:04:11.400 --> 00:04:13.760 align:start position:0% who can sort the array loop or spark or something<00:04:12.150> which<00:04:12.420> spans<00:04:12.750> more<00:04:13.500> than<00:04:13.560> one 00:04:13.760 --> 00:04:13.770 align:start position:0% something which spans more than one 00:04:13.770 --> 00:04:15.050 align:start position:0% something which spans more than one machine<00:04:14.070> and<00:04:14.280> then<00:04:14.459> of<00:04:14.640> course<00:04:14.670> it<00:04:14.910> becomes 00:04:15.050 --> 00:04:15.060 align:start position:0% machine and then of course it becomes 00:04:15.060 --> 00:04:17.690 align:start position:0% machine and then of course it becomes much<00:04:15.240> much<00:04:15.420> harder<00:04:15.630> because<00:04:16.400> even<00:04:17.400> getting 00:04:17.690 --> 00:04:17.700 align:start position:0% much much harder because even getting 00:04:17.700 --> 00:04:20.030 align:start position:0% much much harder because even getting air<00:04:17.820> in<00:04:17.970> the<00:04:18.090> air<00:04:18.120> and<00:04:18.660> certain<00:04:19.019> it<00:04:19.109> and<00:04:19.440> it's 00:04:20.030 --> 00:04:20.040 align:start position:0% air in the air and certain it and it's 00:04:20.040 --> 00:04:22.760 align:start position:0% air in the air and certain it and it's become<00:04:20.430> so<00:04:20.670> so<00:04:21.209> I<00:04:21.239> wonder<00:04:21.630> you<00:04:21.989> know<00:04:22.079> if<00:04:22.200> you<00:04:22.320> do 00:04:22.760 --> 00:04:22.770 align:start position:0% become so so I wonder you know if you do 00:04:22.770 --> 00:04:26.720 align:start position:0% become so so I wonder you know if you do this<00:04:23.010> and<00:04:23.790> kind<00:04:24.210> of<00:04:24.360> in<00:04:25.220> their<00:04:26.220> kind<00:04:26.550> of<00:04:26.640> the 00:04:26.720 --> 00:04:26.730 align:start position:0% this and kind of in their kind of the 00:04:26.730 --> 00:04:28.520 align:start position:0% this and kind of in their kind of the question<00:04:27.060> is<00:04:27.180> do<00:04:27.480> we<00:04:27.510> have<00:04:27.570> two<00:04:27.900> kinds<00:04:28.110> of<00:04:28.260> data 00:04:28.520 --> 00:04:28.530 align:start position:0% question is do we have two kinds of data 00:04:28.530 --> 00:04:30.710 align:start position:0% question is do we have two kinds of data scientists<00:04:29.010> because<00:04:29.190> machine<00:04:29.490> learning<00:04:29.790> is 00:04:30.710 --> 00:04:30.720 align:start position:0% scientists because machine learning is 00:04:30.720 --> 00:04:32.630 align:start position:0% scientists because machine learning is harder<00:04:31.560> and<00:04:31.770> very<00:04:32.040> different<00:04:32.340> thing<00:04:32.520> and 00:04:32.630 --> 00:04:32.640 align:start position:0% harder and very different thing and 00:04:32.640 --> 00:04:34.010 align:start position:0% harder and very different thing and usually<00:04:33.000> it's<00:04:33.180> explained<00:04:33.540> through<00:04:33.750> our 00:04:34.010 --> 00:04:34.020 align:start position:0% usually it's explained through our 00:04:34.020 --> 00:04:38.090 align:start position:0% usually it's explained through our presence<00:04:34.980> and<00:04:35.310> statistics<00:04:35.940> right<00:04:36.090> so<00:04:36.360> so<00:04:37.100> in 00:04:38.090 --> 00:04:38.100 align:start position:0% presence and statistics right so so in 00:04:38.100 --> 00:04:39.560 align:start position:0% presence and statistics right so so in reality<00:04:38.340> I'm<00:04:38.730> not<00:04:38.880> just<00:04:39.030> that<00:04:39.150> a<00:04:39.330> lot<00:04:39.540> of 00:04:39.560 --> 00:04:39.570 align:start position:0% reality I'm not just that a lot of 00:04:39.570 --> 00:04:41.930 align:start position:0% reality I'm not just that a lot of people<00:04:39.690> who<00:04:40.020> can<00:04:40.230> really<00:04:40.890> properly<00:04:41.640> sort 00:04:41.930 --> 00:04:41.940 align:start position:0% people who can really properly sort 00:04:41.940 --> 00:04:44.720 align:start position:0% people who can really properly sort distributed<00:04:42.930> array<00:04:43.169> are<00:04:44.100> different<00:04:44.460> from<00:04:44.580> the 00:04:44.720 --> 00:04:44.730 align:start position:0% distributed array are different from the 00:04:44.730 --> 00:04:47.230 align:start position:0% distributed array are different from the people<00:04:44.850> who<00:04:45.030> can<00:04:45.390> reason<00:04:46.140> about<00:04:46.230> you<00:04:46.770> know 00:04:47.230 --> 00:04:47.240 align:start position:0% people who can reason about you know 00:04:47.240 --> 00:04:50.300 align:start position:0% people who can reason about you know gradient<00:04:48.240> boosting<00:04:48.390> and<00:04:48.870> various<00:04:49.380> optimizers 00:04:50.300 --> 00:04:50.310 align:start position:0% gradient boosting and various optimizers 00:04:50.310 --> 00:04:54.890 align:start position:0% gradient boosting and various optimizers and<00:04:50.990> very<00:04:51.990> special<00:04:52.440> algorithms<00:04:53.210> and<00:04:54.210> the<00:04:54.540> the 00:04:54.890 --> 00:04:54.900 align:start position:0% and very special algorithms and the the 00:04:54.900 --> 00:04:56.360 align:start position:0% and very special algorithms and the the letter<00:04:55.590> kind<00:04:55.860> of<00:04:55.950> people<00:04:56.130> use<00:04:56.250> the 00:04:56.360 --> 00:04:56.370 align:start position:0% letter kind of people use the 00:04:56.370 --> 00:04:59.000 align:start position:0% letter kind of people use the development<00:04:56.820> matlab<00:04:57.210> and<00:04:57.510> the<00:04:58.440> top<00:04:58.650> for 00:04:59.000 --> 00:04:59.010 align:start position:0% development matlab and the top for 00:04:59.010 --> 00:05:00.469 align:start position:0% development matlab and the top for instance<00:04:59.220> vision<00:04:59.580> sciences<00:04:59.940> program<00:05:00.210> in 00:05:00.469 --> 00:05:00.479 align:start position:0% instance vision sciences program in 00:05:00.479 --> 00:05:03.580 align:start position:0% instance vision sciences program in matlab<00:05:00.600> and<00:05:01.110> and<00:05:01.950> so<00:05:02.340> they<00:05:02.520> still<00:05:03.000> have<00:05:03.150> to 00:05:03.580 --> 00:05:03.590 align:start position:0% matlab and and so they still have to 00:05:03.590 --> 00:05:06.469 align:start position:0% matlab and and so they still have to integrate<00:05:04.590> in<00:05:04.919> some<00:05:05.070> way<00:05:05.220> so<00:05:05.820> I<00:05:05.850> wonder<00:05:06.210> what 00:05:06.469 --> 00:05:06.479 align:start position:0% integrate in some way so I wonder what 00:05:06.479 --> 00:05:07.490 align:start position:0% integrate in some way so I wonder what you're<00:05:06.630> taking<00:05:06.840> this<00:05:06.930> what<00:05:07.080> do<00:05:07.169> you<00:05:07.260> see<00:05:07.440> it 00:05:07.490 --> 00:05:07.500 align:start position:0% you're taking this what do you see it 00:05:07.500 --> 00:05:09.140 align:start position:0% you're taking this what do you see it caldera<00:05:07.890> what<00:05:08.190> you<00:05:08.310> seen<00:05:08.520> these<00:05:08.610> are<00:05:08.790> distinct 00:05:09.140 --> 00:05:09.150 align:start position:0% caldera what you seen these are distinct 00:05:09.150 --> 00:05:11.570 align:start position:0% caldera what you seen these are distinct merging<00:05:09.540> they<00:05:10.169> still<00:05:10.590> kind<00:05:10.830> of<00:05:10.950> develop<00:05:11.400> in<00:05:11.490> a 00:05:11.570 --> 00:05:11.580 align:start position:0% merging they still kind of develop in a 00:05:11.580 --> 00:05:15.530 align:start position:0% merging they still kind of develop in a parallel<00:05:12.440> what's<00:05:13.440> your<00:05:13.680> views<00:05:14.340> um<00:05:14.729> yeah 00:05:15.530 --> 00:05:15.540 align:start position:0% parallel what's your views um yeah 00:05:15.540 --> 00:05:17.360 align:start position:0% parallel what's your views um yeah certainly<00:05:16.050> I<00:05:16.080> mean<00:05:16.290> I<00:05:16.350> think<00:05:16.380> in<00:05:17.190> general 00:05:17.360 --> 00:05:17.370 align:start position:0% certainly I mean I think in general 00:05:17.370 --> 00:05:19.430 align:start position:0% certainly I mean I think in general people<00:05:17.760> columns<00:05:18.210> of<00:05:18.360> data<00:05:18.510> scientists<00:05:19.260> or<00:05:19.410> a 00:05:19.430 --> 00:05:19.440 align:start position:0% people columns of data scientists or a 00:05:19.440 --> 00:05:22.219 align:start position:0% people columns of data scientists or a great<00:05:19.680> extent<00:05:19.890> people<00:05:20.400> that<00:05:20.640> come<00:05:21.060> from<00:05:21.330> or 00:05:22.219 --> 00:05:22.229 align:start position:0% great extent people that come from or 00:05:22.229 --> 00:05:24.080 align:start position:0% great extent people that come from or math<00:05:22.560> or<00:05:22.590> physics<00:05:22.860> background<00:05:23.430> and<00:05:23.880> people 00:05:24.080 --> 00:05:24.090 align:start position:0% math or physics background and people 00:05:24.090 --> 00:05:26.810 align:start position:0% math or physics background and people who<00:05:24.180> come<00:05:24.419> from<00:05:24.990> more<00:05:25.820> programming 00:05:26.810 --> 00:05:26.820 align:start position:0% who come from more programming 00:05:26.820 --> 00:05:29.510 align:start position:0% who come from more programming background<00:05:26.970> um<00:05:27.450> I<00:05:28.260> think<00:05:29.070> the<00:05:29.130> whole<00:05:29.250> pieces 00:05:29.510 --> 00:05:29.520 align:start position:0% background um I think the whole pieces 00:05:29.520 --> 00:05:32.300 align:start position:0% background um I think the whole pieces of<00:05:29.880> having<00:05:30.780> like<00:05:31.080> data<00:05:31.410> science<00:05:31.770> as<00:05:31.860> a<00:05:31.890> term<00:05:32.160> at 00:05:32.300 --> 00:05:32.310 align:start position:0% of having like data science as a term at 00:05:32.310 --> 00:05:34.760 align:start position:0% of having like data science as a term at all<00:05:32.460> is<00:05:32.700> like<00:05:32.970> the<00:05:33.120> idea<00:05:33.390> that<00:05:33.479> is<00:05:33.930> useful<00:05:34.500> for 00:05:34.760 --> 00:05:34.770 align:start position:0% all is like the idea that is useful for 00:05:34.770 --> 00:05:36.500 align:start position:0% all is like the idea that is useful for these<00:05:35.130> people<00:05:35.580> to<00:05:35.669> get<00:05:35.850> together<00:05:35.970> and<00:05:36.210> for 00:05:36.500 --> 00:05:36.510 align:start position:0% these people to get together and for 00:05:36.510 --> 00:05:38.719 align:start position:0% these people to get together and for single<00:05:36.780> people<00:05:37.020> to<00:05:37.140> try<00:05:37.290> to<00:05:37.550> gain<00:05:38.550> a<00:05:38.580> little 00:05:38.719 --> 00:05:38.729 align:start position:0% single people to try to gain a little 00:05:38.729 --> 00:05:41.060 align:start position:0% single people to try to gain a little bit<00:05:38.850> of<00:05:39.000> both<00:05:39.090> of<00:05:39.150> these<00:05:39.330> but<00:05:40.050> these<00:05:40.169> skills<00:05:40.560> um 00:05:41.060 --> 00:05:41.070 align:start position:0% bit of both of these but these skills um 00:05:41.070 --> 00:05:43.159 align:start position:0% bit of both of these but these skills um you<00:05:41.580> know<00:05:41.640> in<00:05:41.700> many<00:05:41.940> situations<00:05:42.390> you<00:05:42.840> call<00:05:43.050> the 00:05:43.159 --> 00:05:43.169 align:start position:0% you know in many situations you call the 00:05:43.169 --> 00:05:44.630 align:start position:0% you know in many situations you call the latter<00:05:43.350> group<00:05:43.410> of<00:05:43.770> computer<00:05:44.070> science<00:05:44.400> group 00:05:44.630 --> 00:05:44.640 align:start position:0% latter group of computer science group 00:05:44.640 --> 00:05:48.080 align:start position:0% latter group of computer science group data<00:05:45.390> engineers<00:05:45.870> mm-hmm<00:05:46.880> that's<00:05:47.880> the 00:05:48.080 --> 00:05:48.090 align:start position:0% data engineers mm-hmm that's the 00:05:48.090 --> 00:05:49.460 align:start position:0% data engineers mm-hmm that's the breakdown<00:05:48.419> that<00:05:48.630> happens<00:05:49.050> to<00:05:49.140> be<00:05:49.229> a<00:05:49.260> lot<00:05:49.440> of 00:05:49.460 --> 00:05:49.470 align:start position:0% breakdown that happens to be a lot of 00:05:49.470 --> 00:05:50.630 align:start position:0% breakdown that happens to be a lot of companies<00:05:49.860> but<00:05:50.040> I<00:05:50.100> don't<00:05:50.250> really<00:05:50.370> see<00:05:50.610> that 00:05:50.630 --> 00:05:50.640 align:start position:0% companies but I don't really see that 00:05:50.640 --> 00:05:54.740 align:start position:0% companies but I don't really see that division<00:05:51.600> going<00:05:51.900> away<00:05:52.020> and<00:05:52.940> so<00:05:53.940> but<00:05:54.240> grab<00:05:54.570> this 00:05:54.740 --> 00:05:54.750 align:start position:0% division going away and so but grab this 00:05:54.750 --> 00:05:56.810 align:start position:0% division going away and so but grab this kind<00:05:55.020> of<00:05:55.080> unicorns<00:05:55.680> who<00:05:55.979> do<00:05:56.280> both<00:05:56.430> right<00:05:56.790> i 00:05:56.810 --> 00:05:56.820 align:start position:0% kind of unicorns who do both right i 00:05:56.820 --> 00:05:58.129 align:start position:0% kind of unicorns who do both right i think<00:05:57.000> you're<00:05:57.270> closer<00:05:57.570> to 00:05:58.129 --> 00:05:58.139 align:start position:0% think you're closer to 00:05:58.139 --> 00:06:00.559 align:start position:0% think you're closer to uniform<00:05:58.650> right<00:05:59.009> i<00:05:59.069> mean<00:05:59.189> the<00:05:59.550> dream<00:06:00.090> is<00:06:00.389> that 00:06:00.559 --> 00:06:00.569 align:start position:0% uniform right i mean the dream is that 00:06:00.569 --> 00:06:02.119 align:start position:0% uniform right i mean the dream is that you<00:06:00.749> are<00:06:00.840> able<00:06:00.930> to<00:06:01.169> build<00:06:01.379> frameworks<00:06:01.889> like 00:06:02.119 --> 00:06:02.129 align:start position:0% you are able to build frameworks like 00:06:02.129 --> 00:06:04.420 align:start position:0% you are able to build frameworks like spark<00:06:02.580> for<00:06:02.789> example<00:06:03.210> that<00:06:03.360> makes<00:06:03.539> it<00:06:03.629> easy<00:06:03.870> to 00:06:04.420 --> 00:06:04.430 align:start position:0% spark for example that makes it easy to 00:06:04.430 --> 00:06:08.300 align:start position:0% spark for example that makes it easy to dupe<00:06:05.430> up<00:06:05.580> right<00:06:05.879> to<00:06:06.409> work<00:06:07.409> with<00:06:07.590> you<00:06:07.710> data<00:06:07.979> sets 00:06:08.300 --> 00:06:08.310 align:start position:0% dupe up right to work with you data sets 00:06:08.310 --> 00:06:11.439 align:start position:0% dupe up right to work with you data sets to<00:06:08.849> put<00:06:09.029> things<00:06:09.210> in<00:06:09.330> production<00:06:09.810> but<00:06:10.710> also<00:06:10.770> to 00:06:11.439 --> 00:06:11.449 align:start position:0% to put things in production but also to 00:06:11.449 --> 00:06:13.489 align:start position:0% to put things in production but also to easily<00:06:12.449> let<00:06:12.629> River<00:06:13.050> knowledge<00:06:13.349> you<00:06:13.469> have 00:06:13.489 --> 00:06:13.499 align:start position:0% easily let River knowledge you have 00:06:13.499 --> 00:06:15.890 align:start position:0% easily let River knowledge you have about<00:06:13.710> an<00:06:14.129> asset<00:06:14.580> istics<00:06:14.939> hmm<00:06:15.419> I'm<00:06:15.539> in<00:06:15.719> a<00:06:15.779> way 00:06:15.890 --> 00:06:15.900 align:start position:0% about an asset istics hmm I'm in a way 00:06:15.900 --> 00:06:17.659 align:start position:0% about an asset istics hmm I'm in a way it<00:06:15.960> doesn't<00:06:16.169> require<00:06:16.379> putting<00:06:16.860> up<00:06:17.099> in<00:06:17.279> Java<00:06:17.520> or 00:06:17.659 --> 00:06:17.669 align:start position:0% it doesn't require putting up in Java or 00:06:17.669 --> 00:06:20.570 align:start position:0% it doesn't require putting up in Java or a<00:06:17.729> MapReduce<00:06:18.150> yes<00:06:18.590> so<00:06:19.590> you're<00:06:20.249> also<00:06:20.430> a 00:06:20.570 --> 00:06:20.580 align:start position:0% a MapReduce yes so you're also a 00:06:20.580 --> 00:06:22.489 align:start position:0% a MapReduce yes so you're also a contributor<00:06:21.090> to<00:06:21.629> purchase<00:06:21.870> park<00:06:22.080> so<00:06:22.229> a<00:06:22.259> year 00:06:22.489 --> 00:06:22.499 align:start position:0% contributor to purchase park so a year 00:06:22.499 --> 00:06:26.209 align:start position:0% contributor to purchase park so a year ago<00:06:22.620> you<00:06:23.599> described<00:06:24.599> the<00:06:25.020> job<00:06:25.289> is<00:06:25.529> spending 00:06:26.209 --> 00:06:26.219 align:start position:0% ago you described the job is spending 00:06:26.219 --> 00:06:28.129 align:start position:0% ago you described the job is spending half<00:06:26.370> the<00:06:26.520> time<00:06:26.699> with<00:06:26.849> customers<00:06:27.330> and<00:06:27.539> how<00:06:28.080> the 00:06:28.129 --> 00:06:28.139 align:start position:0% half the time with customers and how the 00:06:28.139 --> 00:06:30.409 align:start position:0% half the time with customers and how the time<00:06:28.379> contribute<00:06:28.830> into<00:06:28.949> tools<00:06:29.310> such<00:06:29.550> as<00:06:29.580> much 00:06:30.409 --> 00:06:30.419 align:start position:0% time contribute into tools such as much 00:06:30.419 --> 00:06:34.219 align:start position:0% time contribute into tools such as much respond<00:06:30.719> uses<00:06:31.620> steal<00:06:32.159> the<00:06:32.310> same<00:06:32.490> ratio<00:06:33.169> it's<00:06:34.169> a 00:06:34.219 --> 00:06:34.229 align:start position:0% respond uses steal the same ratio it's a 00:06:34.229 --> 00:06:36.260 align:start position:0% respond uses steal the same ratio it's a good<00:06:34.349> question<00:06:34.500> yeah<00:06:34.919> I<00:06:35.129> think<00:06:35.189> that<00:06:35.580> in<00:06:35.879> the 00:06:36.260 --> 00:06:36.270 align:start position:0% good question yeah I think that in the 00:06:36.270 --> 00:06:38.149 align:start position:0% good question yeah I think that in the very<00:06:36.360> recent<00:06:36.659> past<00:06:36.930> I've<00:06:37.620> moved<00:06:37.889> a<00:06:37.949> little<00:06:38.039> bit 00:06:38.149 --> 00:06:38.159 align:start position:0% very recent past I've moved a little bit 00:06:38.159 --> 00:06:39.559 align:start position:0% very recent past I've moved a little bit away<00:06:38.310> from<00:06:38.430> sparking<00:06:39.029> a<00:06:39.060> little<00:06:39.240> bit<00:06:39.330> more 00:06:39.559 --> 00:06:39.569 align:start position:0% away from sparking a little bit more 00:06:39.569 --> 00:06:41.959 align:start position:0% away from sparking a little bit more towards<00:06:40.289> the<00:06:40.590> data<00:06:40.770> science<00:06:41.159> side<00:06:41.639> of<00:06:41.729> the<00:06:41.789> job 00:06:41.959 --> 00:06:41.969 align:start position:0% towards the data science side of the job 00:06:41.969 --> 00:06:45.110 align:start position:0% towards the data science side of the job um<00:06:43.159> yeah<00:06:44.159> I've<00:06:44.250> been<00:06:44.370> on<00:06:44.460> a<00:06:44.490> lot<00:06:44.550> of<00:06:44.669> customer 00:06:45.110 --> 00:06:45.120 align:start position:0% um yeah I've been on a lot of customer 00:06:45.120 --> 00:06:48.050 align:start position:0% um yeah I've been on a lot of customer engagements<00:06:45.960> not<00:06:46.229> doing<00:06:46.560> a<00:06:46.650> lot<00:06:46.770> of<00:06:47.060> Education 00:06:48.050 --> 00:06:48.060 align:start position:0% engagements not doing a lot of Education 00:06:48.060 --> 00:06:51.079 align:start position:0% engagements not doing a lot of Education for<00:06:48.509> Clara<00:06:49.789> talking<00:06:50.789> and<00:06:50.969> speaking 00:06:51.079 --> 00:06:51.089 align:start position:0% for Clara talking and speaking 00:06:51.089 --> 00:06:53.269 align:start position:0% for Clara talking and speaking engagements<00:06:51.240> this<00:06:51.689> kind<00:06:51.900> of<00:06:51.930> stuff<00:06:52.279> careful 00:06:53.269 --> 00:06:53.279 align:start position:0% engagements this kind of stuff careful 00:06:53.279 --> 00:06:55.640 align:start position:0% engagements this kind of stuff careful with<00:06:53.460> yeah<00:06:54.449> not<00:06:54.689> enough<00:06:54.810> to<00:06:54.930> do<00:06:55.050> it<00:06:55.139> and<00:06:55.319> I 00:06:55.640 --> 00:06:55.650 align:start position:0% with yeah not enough to do it and I 00:06:55.650 --> 00:06:57.829 align:start position:0% with yeah not enough to do it and I think<00:06:55.889> I<00:06:56.129> might<00:06:56.639> be<00:06:56.669> coming<00:06:56.969> back<00:06:57.389> a<00:06:57.449> bit<00:06:57.659> more 00:06:57.829 --> 00:06:57.839 align:start position:0% think I might be coming back a bit more 00:06:57.839 --> 00:07:01.100 align:start position:0% think I might be coming back a bit more towards<00:06:58.110> Barca<00:06:58.710> and<00:06:58.919> near<00:06:59.729> future<00:06:59.939> like<00:07:00.810> get 00:07:01.100 --> 00:07:01.110 align:start position:0% towards Barca and near future like get 00:07:01.110 --> 00:07:04.369 align:start position:0% towards Barca and near future like get tired<00:07:01.349> of<00:07:01.469> flying<00:07:02.069> around<00:07:02.310> them<00:07:02.900> enough<00:07:03.900> and 00:07:04.369 --> 00:07:04.379 align:start position:0% tired of flying around them enough and 00:07:04.379 --> 00:07:06.559 align:start position:0% tired of flying around them enough and when<00:07:04.710> you<00:07:05.039> are<00:07:05.460> can<00:07:05.669> you<00:07:05.699> just<00:07:06.000> park<00:07:06.150> what<00:07:06.539> a 00:07:06.559 --> 00:07:06.569 align:start position:0% when you are can you just park what a 00:07:06.569 --> 00:07:09.439 align:start position:0% when you are can you just park what a race<00:07:06.870> are<00:07:07.020> going<00:07:07.169> to<00:07:07.229> get<00:07:07.349> into<00:07:07.680> um<00:07:08.120> so<00:07:09.120> the<00:07:09.240> two 00:07:09.439 --> 00:07:09.449 align:start position:0% race are going to get into um so the two 00:07:09.449 --> 00:07:13.459 align:start position:0% race are going to get into um so the two main<00:07:10.020> areas<00:07:10.289> are<00:07:11.120> darn<00:07:12.120> and<00:07:12.509> then<00:07:12.599> statistics 00:07:13.459 --> 00:07:13.469 align:start position:0% main areas are darn and then statistics 00:07:13.469 --> 00:07:15.619 align:start position:0% main areas are darn and then statistics functionality<00:07:14.069> and<00:07:14.250> ml<00:07:14.639> is<00:07:14.789> um<00:07:15.000> we're 00:07:15.619 --> 00:07:15.629 align:start position:0% functionality and ml is um we're 00:07:15.629 --> 00:07:17.179 align:start position:0% functionality and ml is um we're probably<00:07:15.659> a<00:07:16.169> much<00:07:16.349> more<00:07:16.529> of<00:07:16.620> an<00:07:16.710> emphasis<00:07:17.039> on 00:07:17.179 --> 00:07:17.189 align:start position:0% probably a much more of an emphasis on 00:07:17.189 --> 00:07:19.219 align:start position:0% probably a much more of an emphasis on yarn<00:07:17.490> so<00:07:17.819> I<00:07:17.849> come<00:07:18.300> from<00:07:18.389> a<00:07:18.539> Hadoop<00:07:18.839> background 00:07:19.219 --> 00:07:19.229 align:start position:0% yarn so I come from a Hadoop background 00:07:19.229 --> 00:07:22.040 align:start position:0% yarn so I come from a Hadoop background huh<00:07:19.589> committed<00:07:20.339> on<00:07:20.399> yarn<00:07:20.699> mm-hmm<00:07:21.389> so<00:07:21.960> I've 00:07:22.040 --> 00:07:22.050 align:start position:0% huh committed on yarn mm-hmm so I've 00:07:22.050 --> 00:07:24.290 align:start position:0% huh committed on yarn mm-hmm so I've been<00:07:22.110> doing<00:07:22.319> a<00:07:22.379> lot<00:07:22.469> of<00:07:22.789> yarn<00:07:23.789> scheduling 00:07:24.290 --> 00:07:24.300 align:start position:0% been doing a lot of yarn scheduling 00:07:24.300 --> 00:07:26.629 align:start position:0% been doing a lot of yarn scheduling stuff<00:07:24.569> right<00:07:25.259> now<00:07:25.289> one<00:07:26.189> of<00:07:26.219> the<00:07:26.370> big<00:07:26.610> things 00:07:26.629 --> 00:07:26.639 align:start position:0% stuff right now one of the big things 00:07:26.639 --> 00:07:29.689 align:start position:0% stuff right now one of the big things I'm<00:07:26.939> working<00:07:26.969> on<00:07:27.330> is<00:07:28.039> reviewing<00:07:29.039> a<00:07:29.339> patch<00:07:29.669> that 00:07:29.689 --> 00:07:29.699 align:start position:0% I'm working on is reviewing a patch that 00:07:29.699 --> 00:07:32.329 align:start position:0% I'm working on is reviewing a patch that will<00:07:30.089> bring<00:07:30.360> in<00:07:30.539> the<00:07:30.779> ability<00:07:31.050> to<00:07:31.199> do<00:07:31.439> executor 00:07:32.329 --> 00:07:32.339 align:start position:0% will bring in the ability to do executor 00:07:32.339 --> 00:07:35.240 align:start position:0% will bring in the ability to do executor replacement<00:07:32.819> based<00:07:33.060> on<00:07:33.830> data<00:07:34.830> locality 00:07:35.240 --> 00:07:35.250 align:start position:0% replacement based on data locality 00:07:35.250 --> 00:07:37.189 align:start position:0% replacement based on data locality preferences<00:07:35.879> mm-hmm<00:07:36.389> and<00:07:36.569> this<00:07:36.930> has<00:07:37.080> been 00:07:37.189 --> 00:07:37.199 align:start position:0% preferences mm-hmm and this has been 00:07:37.199 --> 00:07:38.360 align:start position:0% preferences mm-hmm and this has been something<00:07:37.409> that<00:07:37.500> sort<00:07:37.680> of<00:07:37.770> it<00:07:37.860> missing<00:07:38.189> its 00:07:38.360 --> 00:07:38.370 align:start position:0% something that sort of it missing its 00:07:38.370 --> 00:07:41.629 align:start position:0% something that sort of it missing its park<00:07:38.610> in<00:07:38.729> a<00:07:38.759> while<00:07:38.969> maybe<00:07:39.330> a<00:07:40.460> place<00:07:41.460> where 00:07:41.629 --> 00:07:41.639 align:start position:0% park in a while maybe a place where 00:07:41.639 --> 00:07:43.629 align:start position:0% park in a while maybe a place where MapReduce<00:07:42.029> they'll<00:07:42.330> have<00:07:42.509> differentiation 00:07:43.629 --> 00:07:43.639 align:start position:0% MapReduce they'll have differentiation 00:07:43.639 --> 00:07:47.719 align:start position:0% MapReduce they'll have differentiation but<00:07:44.639> this<00:07:45.569> guy<00:07:46.020> sigh<00:07:46.319> sigh<00:07:46.529> Xiao<00:07:46.889> at<00:07:47.250> Intel 00:07:47.719 --> 00:07:47.729 align:start position:0% but this guy sigh sigh Xiao at Intel 00:07:47.729 --> 00:07:50.209 align:start position:0% but this guy sigh sigh Xiao at Intel eyes<00:07:48.379> thankfully<00:07:49.379> taking<00:07:49.680> this<00:07:49.770> up<00:07:49.949> and<00:07:50.099> we're 00:07:50.209 --> 00:07:50.219 align:start position:0% eyes thankfully taking this up and we're 00:07:50.219 --> 00:07:52.389 align:start position:0% eyes thankfully taking this up and we're trying<00:07:50.310> to<00:07:50.610> move<00:07:51.240> that<00:07:51.389> forward<00:07:51.449> oh<00:07:51.930> nice 00:07:52.389 --> 00:07:52.399 align:start position:0% trying to move that forward oh nice 00:07:52.399 --> 00:07:54.139 align:start position:0% trying to move that forward oh nice that's<00:07:53.399> something<00:07:53.669> that's<00:07:53.759> great<00:07:54.029> to<00:07:54.120> hear 00:07:54.139 --> 00:07:54.149 align:start position:0% that's something that's great to hear 00:07:54.149 --> 00:07:55.909 align:start position:0% that's something that's great to hear because<00:07:54.449> I<00:07:54.689> think<00:07:54.750> little<00:07:55.469> college<00:07:55.770> is<00:07:55.889> really 00:07:55.909 --> 00:07:55.919 align:start position:0% because I think little college is really 00:07:55.919 --> 00:07:59.029 align:start position:0% because I think little college is really important<00:07:56.610> for<00:07:56.639> performance<00:07:57.319> so<00:07:58.319> yeah<00:07:59.009> I 00:07:59.029 --> 00:07:59.039 align:start position:0% important for performance so yeah I 00:07:59.039 --> 00:08:00.469 align:start position:0% important for performance so yeah I think<00:07:59.550> it's<00:07:59.819> very<00:07:59.849> interesting<00:08:00.029> that<00:08:00.330> the 00:08:00.469 --> 00:08:00.479 align:start position:0% think it's very interesting that the 00:08:00.479 --> 00:08:02.749 align:start position:0% think it's very interesting that the other<00:08:00.629> yarn<00:08:00.930> so<00:08:01.649> it's<00:08:01.949> I<00:08:02.159> think<00:08:02.219> it's<00:08:02.370> really 00:08:02.749 --> 00:08:02.759 align:start position:0% other yarn so it's I think it's really 00:08:02.759 --> 00:08:04.679 align:start position:0% other yarn so it's I think it's really kind<00:08:03.149> of<00:08:03.210> shocked<00:08:03.870> eating<00:08:04.289> the 00:08:04.679 --> 00:08:04.689 align:start position:0% kind of shocked eating the 00:08:04.689 --> 00:08:08.100 align:start position:0% kind of shocked eating the can<00:08:05.369> kind<00:08:06.369> of<00:08:06.459> folks<00:08:06.699> who<00:08:06.879> can<00:08:07.059> actually<00:08:07.419> make 00:08:08.100 --> 00:08:08.110 align:start position:0% can kind of folks who can actually make 00:08:08.110 --> 00:08:09.359 align:start position:0% can kind of folks who can actually make real<00:08:08.319> progress<00:08:08.439> as<00:08:08.769> far<00:08:08.949> because<00:08:09.099> you<00:08:09.279> know 00:08:09.359 --> 00:08:09.369 align:start position:0% real progress as far because you know 00:08:09.369 --> 00:08:11.100 align:start position:0% real progress as far because you know yarn<00:08:09.580> as<00:08:09.729> a<00:08:09.759> divorcee<00:08:10.239> things<00:08:10.569> right<00:08:10.809> like<00:08:10.899> how 00:08:11.100 --> 00:08:11.110 align:start position:0% yarn as a divorcee things right like how 00:08:11.110 --> 00:08:13.469 align:start position:0% yarn as a divorcee things right like how the<00:08:11.169> actual<00:08:11.559> raleigh's<00:08:11.999> doesn't<00:08:12.999> it's<00:08:13.239> not 00:08:13.469 --> 00:08:13.479 align:start position:0% the actual raleigh's doesn't it's not 00:08:13.479 --> 00:08:15.209 align:start position:0% the actual raleigh's doesn't it's not obvious<00:08:13.899> thing<00:08:14.169> in<00:08:14.319> you<00:08:14.439> cover<00:08:14.619> because<00:08:14.829> usual 00:08:15.209 --> 00:08:15.219 align:start position:0% obvious thing in you cover because usual 00:08:15.219 --> 00:08:17.249 align:start position:0% obvious thing in you cover because usual star<00:08:15.429> on<00:08:15.759> amazon<00:08:16.179> but<00:08:16.839> then<00:08:17.019> he<00:08:17.110> got<00:08:17.199> a 00:08:17.249 --> 00:08:17.259 align:start position:0% star on amazon but then he got a 00:08:17.259 --> 00:08:18.839 align:start position:0% star on amazon but then he got a production<00:08:17.739> environment<00:08:17.799> like<00:08:18.129> natural<00:08:18.639> once 00:08:18.839 --> 00:08:18.849 align:start position:0% production environment like natural once 00:08:18.849 --> 00:08:20.279 align:start position:0% production environment like natural once the<00:08:19.029> question<00:08:19.299> is<00:08:19.389> how<00:08:19.719> the<00:08:19.779> actual<00:08:20.139> wrong 00:08:20.279 --> 00:08:20.289 align:start position:0% the question is how the actual wrong 00:08:20.289 --> 00:08:21.839 align:start position:0% the question is how the actual wrong with<00:08:20.469> the<00:08:20.529> 47<00:08:20.979> really<00:08:21.189> lightly<00:08:21.369> have<00:08:21.819> a 00:08:21.839 --> 00:08:21.849 align:start position:0% with the 47 really lightly have a 00:08:21.849 --> 00:08:23.729 align:start position:0% with the 47 really lightly have a liability<00:08:22.119> of<00:08:22.389> them<00:08:22.539> is<00:08:22.689> on<00:08:22.869> what<00:08:22.989> yarn<00:08:23.229> or<00:08:23.469> or 00:08:23.729 --> 00:08:23.739 align:start position:0% liability of them is on what yarn or or 00:08:23.739 --> 00:08:26.879 align:start position:0% liability of them is on what yarn or or mrs.<00:08:24.069> right<00:08:24.789> then<00:08:24.999> the<00:08:25.299> historically<00:08:26.229> spark 00:08:26.879 --> 00:08:26.889 align:start position:0% mrs. right then the historically spark 00:08:26.889 --> 00:08:28.889 align:start position:0% mrs. right then the historically spark was<00:08:27.129> developed<00:08:27.459> on<00:08:27.489> VESA<00:08:27.699> sexually<00:08:28.149> messes 00:08:28.889 --> 00:08:28.899 align:start position:0% was developed on VESA sexually messes 00:08:28.899 --> 00:08:30.329 align:start position:0% was developed on VESA sexually messes was<00:08:29.139> the<00:08:29.349> project<00:08:29.499> but<00:08:29.739> they<00:08:29.860> was<00:08:30.009> working<00:08:30.279> on 00:08:30.329 --> 00:08:30.339 align:start position:0% was the project but they was working on 00:08:30.339 --> 00:08:33.119 align:start position:0% was the project but they was working on that<00:08:30.459> his<00:08:30.579> bond<00:08:30.789> was<00:08:30.879> a<00:08:31.179> cute<00:08:31.809> ab<00:08:32.019> demo<00:08:32.469> right 00:08:33.119 --> 00:08:33.129 align:start position:0% that his bond was a cute ab demo right 00:08:33.129 --> 00:08:36.569 align:start position:0% that his bond was a cute ab demo right then<00:08:33.339> suddenly<00:08:33.759> kind<00:08:34.000> of<00:08:34.149> it<00:08:34.889> reversed<00:08:35.889> and<00:08:36.159> so 00:08:36.569 --> 00:08:36.579 align:start position:0% then suddenly kind of it reversed and so 00:08:36.579 --> 00:08:38.699 align:start position:0% then suddenly kind of it reversed and so messes<00:08:37.360> with<00:08:37.479> motive<00:08:37.810> is<00:08:37.899> the<00:08:38.079> twitter<00:08:38.319> but 00:08:38.699 --> 00:08:38.709 align:start position:0% messes with motive is the twitter but 00:08:38.709 --> 00:08:40.829 align:start position:0% messes with motive is the twitter but now<00:08:38.860> there<00:08:39.099> is<00:08:39.129> Mississippi<00:08:39.699> here<00:08:39.879> and<00:08:40.089> a<00:08:40.719> lot 00:08:40.829 --> 00:08:40.839 align:start position:0% now there is Mississippi here and a lot 00:08:40.839 --> 00:08:43.319 align:start position:0% now there is Mississippi here and a lot of<00:08:40.899> folks<00:08:41.169> actually<00:08:41.560> are<00:08:42.189> running<00:08:42.969> sparking 00:08:43.319 --> 00:08:43.329 align:start position:0% of folks actually are running sparking 00:08:43.329 --> 00:08:46.170 align:start position:0% of folks actually are running sparking messes<00:08:43.689> so<00:08:43.980> and<00:08:44.980> I'm<00:08:45.579> probably<00:08:45.879> more<00:08:46.060> familiar 00:08:46.170 --> 00:08:46.180 align:start position:0% messes so and I'm probably more familiar 00:08:46.180 --> 00:08:48.150 align:start position:0% messes so and I'm probably more familiar with<00:08:46.480> companies<00:08:47.079> running<00:08:47.259> messages<00:08:47.529> bitter 00:08:48.150 --> 00:08:48.160 align:start position:0% with companies running messages bitter 00:08:48.160 --> 00:08:50.249 align:start position:0% with companies running messages bitter and<00:08:48.490> sigmoid<00:08:49.209> only<00:08:49.569> takes<00:08:49.779> guys<00:08:49.990> their 00:08:50.249 --> 00:08:50.259 align:start position:0% and sigmoid only takes guys their 00:08:50.259 --> 00:08:52.379 align:start position:0% and sigmoid only takes guys their bicycles<00:08:50.680> very<00:08:50.860> easy<00:08:51.160> to<00:08:51.670> come<00:08:51.850> into<00:08:52.120> a<00:08:52.149> shop 00:08:52.379 --> 00:08:52.389 align:start position:0% bicycles very easy to come into a shop 00:08:52.389 --> 00:08:54.840 align:start position:0% bicycles very easy to come into a shop and<00:08:52.600> install<00:08:53.139> messes<00:08:53.529> alright<00:08:54.370> if<00:08:54.550> you<00:08:54.579> don't 00:08:54.840 --> 00:08:54.850 align:start position:0% and install messes alright if you don't 00:08:54.850 --> 00:08:58.290 align:start position:0% and install messes alright if you don't have<00:08:55.060> the<00:08:55.240> full<00:08:55.449> cdh<00:08:56.079> there<00:08:56.379> already<00:08:56.740> so<00:08:57.639> I'm 00:08:58.290 --> 00:08:58.300 align:start position:0% have the full cdh there already so I'm 00:08:58.300 --> 00:09:01.259 align:start position:0% have the full cdh there already so I'm wondering<00:08:59.009> kind<00:09:00.009> of<00:09:00.040> kind<00:09:00.160> of<00:09:00.279> emphasize<00:09:00.939> or 00:09:01.259 --> 00:09:01.269 align:start position:0% wondering kind of kind of emphasize or 00:09:01.269 --> 00:09:04.769 align:start position:0% wondering kind of kind of emphasize or highlight<00:09:02.069> the<00:09:03.069> differences<00:09:03.490> are<00:09:03.670> in<00:09:04.269> between 00:09:04.769 --> 00:09:04.779 align:start position:0% highlight the differences are in between 00:09:04.779 --> 00:09:08.189 align:start position:0% highlight the differences are in between erm<00:09:05.290> SSN<00:09:05.829> you<00:09:06.100> know<00:09:06.250> why<00:09:06.939> they<00:09:07.149> obviously<00:09:07.899> by 00:09:08.189 --> 00:09:08.199 align:start position:0% erm SSN you know why they obviously by 00:09:08.199 --> 00:09:10.139 align:start position:0% erm SSN you know why they obviously by contribute<00:09:08.620> to<00:09:08.740> yarn<00:09:08.949> then<00:09:09.250> you<00:09:09.430> you<00:09:09.939> bet<00:09:10.120> on 00:09:10.139 --> 00:09:10.149 align:start position:0% contribute to yarn then you you bet on 00:09:10.149 --> 00:09:12.869 align:start position:0% contribute to yarn then you you bet on the<00:09:10.329> yarn<00:09:10.509> or<00:09:10.899> you're<00:09:11.050> like<00:09:11.199> you're<00:09:11.410> so<00:09:12.100> how 00:09:12.869 --> 00:09:12.879 align:start position:0% the yarn or you're like you're so how 00:09:12.879 --> 00:09:15.809 align:start position:0% the yarn or you're like you're so how would<00:09:12.939> somebody<00:09:13.509> new<00:09:14.319> peak<00:09:15.040> between<00:09:15.370> yarn 00:09:15.809 --> 00:09:15.819 align:start position:0% would somebody new peak between yarn 00:09:15.819 --> 00:09:18.869 align:start position:0% would somebody new peak between yarn messes<00:09:16.439> yeah<00:09:17.439> it's<00:09:17.620> a<00:09:17.709> great<00:09:17.829> question<00:09:18.009> um<00:09:18.430> I 00:09:18.869 --> 00:09:18.879 align:start position:0% messes yeah it's a great question um I 00:09:18.879 --> 00:09:20.639 align:start position:0% messes yeah it's a great question um I mean<00:09:19.269> in<00:09:19.420> theory<00:09:19.629> they<00:09:19.839> did<00:09:19.990> the<00:09:20.050> same<00:09:20.230> thing<00:09:20.259> I 00:09:20.639 --> 00:09:20.649 align:start position:0% mean in theory they did the same thing I 00:09:20.649 --> 00:09:22.319 align:start position:0% mean in theory they did the same thing I personally<00:09:21.160> wish<00:09:21.670> that<00:09:21.939> we<00:09:22.059> could<00:09:22.300> just 00:09:22.319 --> 00:09:22.329 align:start position:0% personally wish that we could just 00:09:22.329 --> 00:09:24.509 align:start position:0% personally wish that we could just standardized<00:09:22.870> on<00:09:23.050> one<00:09:23.230> of<00:09:23.319> them<00:09:23.470> um<00:09:24.009> may 00:09:24.509 --> 00:09:24.519 align:start position:0% standardized on one of them um may 00:09:24.519 --> 00:09:29.160 align:start position:0% standardized on one of them um may supplement<00:09:24.759> fine<00:09:25.259> fine<00:09:26.259> um<00:09:27.269> but<00:09:28.269> as<00:09:28.509> it<00:09:28.750> ended 00:09:29.160 --> 00:09:29.170 align:start position:0% supplement fine fine um but as it ended 00:09:29.170 --> 00:09:30.780 align:start position:0% supplement fine fine um but as it ended up<00:09:29.290> handing<00:09:29.439> out<00:09:29.680> we<00:09:29.860> have<00:09:30.100> two<00:09:30.250> of<00:09:30.370> these<00:09:30.519> I 00:09:30.780 --> 00:09:30.790 align:start position:0% up handing out we have two of these I 00:09:30.790 --> 00:09:32.759 align:start position:0% up handing out we have two of these I think<00:09:31.660> what<00:09:31.870> is<00:09:31.990> sort<00:09:32.199> of<00:09:32.230> ended<00:09:32.470> up<00:09:32.589> happening 00:09:32.759 --> 00:09:32.769 align:start position:0% think what is sort of ended up happening 00:09:32.769 --> 00:09:35.369 align:start position:0% think what is sort of ended up happening is<00:09:33.220> that<00:09:33.279> people<00:09:34.120> who<00:09:34.269> are<00:09:34.329> trying<00:09:34.569> to<00:09:34.750> use 00:09:35.369 --> 00:09:35.379 align:start position:0% is that people who are trying to use 00:09:35.379 --> 00:09:37.679 align:start position:0% is that people who are trying to use clusters<00:09:36.069> for<00:09:36.370> scheduling<00:09:36.939> much<00:09:37.120> of<00:09:37.240> services 00:09:37.679 --> 00:09:37.689 align:start position:0% clusters for scheduling much of services 00:09:37.689 --> 00:09:40.230 align:start position:0% clusters for scheduling much of services I've<00:09:37.930> got<00:09:38.079> 10<00:09:38.290> more<00:09:38.470> tomatoes<00:09:39.069> and<00:09:39.759> yarns 00:09:40.230 --> 00:09:40.240 align:start position:0% I've got 10 more tomatoes and yarns 00:09:40.240 --> 00:09:43.379 align:start position:0% I've got 10 more tomatoes and yarns become<00:09:40.600> more<00:09:40.750> of<00:09:40.779> a<00:09:40.870> framework<00:09:41.139> or<00:09:41.970> that<00:09:42.970> high 00:09:43.379 --> 00:09:43.389 align:start position:0% become more of a framework or that high 00:09:43.389 --> 00:09:46.799 align:start position:0% become more of a framework or that high throughput<00:09:43.750> data<00:09:44.079> processing<00:09:44.559> mm-hmm<00:09:45.040> um<00:09:45.809> so 00:09:46.799 --> 00:09:46.809 align:start position:0% throughput data processing mm-hmm um so 00:09:46.809 --> 00:09:48.569 align:start position:0% throughput data processing mm-hmm um so most<00:09:47.079> people<00:09:47.230> i<00:09:47.439> talked<00:09:47.589> to<00:09:47.709> run<00:09:47.920> spark<00:09:48.430> on 00:09:48.569 --> 00:09:48.579 align:start position:0% most people i talked to run spark on 00:09:48.579 --> 00:09:50.069 align:start position:0% most people i talked to run spark on yarn<00:09:48.790> it<00:09:49.029> could<00:09:49.180> be<00:09:49.300> a<00:09:49.329> fluid<00:09:49.600> in<00:09:49.689> some<00:09:49.870> sort<00:09:50.019> of 00:09:50.069 --> 00:09:50.079 align:start position:0% yarn it could be a fluid in some sort of 00:09:50.079 --> 00:09:52.499 align:start position:0% yarn it could be a fluid in some sort of insulated<00:09:50.410> Cloudera<00:09:51.040> situation<00:09:52.000> but<00:09:52.209> my 00:09:52.499 --> 00:09:52.509 align:start position:0% insulated Cloudera situation but my 00:09:52.509 --> 00:09:55.079 align:start position:0% insulated Cloudera situation but my hunch<00:09:53.019> is<00:09:53.290> that<00:09:53.490> sport<00:09:54.490> gets<00:09:54.670> run<00:09:54.850> a<00:09:54.879> little 00:09:55.079 --> 00:09:55.089 align:start position:0% hunch is that sport gets run a little 00:09:55.089 --> 00:09:56.020 align:start position:0% hunch is that sport gets run a little bit<00:09:55.240> more<00:09:55.480> on<00:09:55.660> your 00:09:56.020 --> 00:09:56.030 align:start position:0% bit more on your 00:09:56.030 --> 00:09:58.750 align:start position:0% bit more on your it<00:09:56.300> does<00:09:56.480> on<00:09:56.750> missus<00:09:57.110> I<00:09:57.320> can<00:09:58.250> talk<00:09:58.340> about<00:09:58.400> sort 00:09:58.750 --> 00:09:58.760 align:start position:0% it does on missus I can talk about sort 00:09:58.760 --> 00:10:01.500 align:start position:0% it does on missus I can talk about sort of<00:09:58.910> specific<00:09:59.690> you<00:09:59.930> know<00:10:00.080> sells<00:10:00.980> for<00:10:01.280> yarn 00:10:01.500 --> 00:10:01.510 align:start position:0% of specific you know sells for yarn 00:10:01.510 --> 00:10:03.760 align:start position:0% of specific you know sells for yarn ourselves<00:10:02.510> for<00:10:02.660> missus<00:10:02.960> but<00:10:03.200> in<00:10:03.350> theory<00:10:03.590> is 00:10:03.760 --> 00:10:03.770 align:start position:0% ourselves for missus but in theory is 00:10:03.770 --> 00:10:05.380 align:start position:0% ourselves for missus but in theory is you<00:10:03.860> know<00:10:03.980> their<00:10:04.370> systems<00:10:04.760> that<00:10:04.940> do<00:10:05.090> the<00:10:05.210> same 00:10:05.380 --> 00:10:05.390 align:start position:0% you know their systems that do the same 00:10:05.390 --> 00:10:09.310 align:start position:0% you know their systems that do the same thing<00:10:05.660> okay<00:10:06.170> I<00:10:07.150> hope<00:10:08.150> you<00:10:08.390> know<00:10:08.420> I<00:10:08.750> would<00:10:08.810> like 00:10:09.310 --> 00:10:09.320 align:start position:0% thing okay I hope you know I would like 00:10:09.320 --> 00:10:11.470 align:start position:0% thing okay I hope you know I would like for<00:10:09.530> us<00:10:09.650> I'll<00:10:09.830> be<00:10:09.950> able<00:10:09.980> to<00:10:10.160> pick<00:10:10.310> one<00:10:10.550> and<00:10:10.760> move 00:10:11.470 --> 00:10:11.480 align:start position:0% for us I'll be able to pick one and move 00:10:11.480 --> 00:10:13.390 align:start position:0% for us I'll be able to pick one and move forward<00:10:11.750> with<00:10:11.930> it<00:10:12.080> but<00:10:12.320> maybe<00:10:12.890> to<00:10:13.070> stuff<00:10:13.280> with 00:10:13.390 --> 00:10:13.400 align:start position:0% forward with it but maybe to stuff with 00:10:13.400 --> 00:10:16.090 align:start position:0% forward with it but maybe to stuff with a<00:10:13.430> flipper<00:10:14.270> interpretation<00:10:14.870> Oh<00:10:14.980> couldn't<00:10:15.980> do 00:10:16.090 --> 00:10:16.100 align:start position:0% a flipper interpretation Oh couldn't do 00:10:16.100 --> 00:10:18.100 align:start position:0% a flipper interpretation Oh couldn't do a<00:10:16.130> bit<00:10:16.310> of<00:10:16.340> altaf<00:10:16.670> achieve<00:10:17.030> myriad<00:10:17.450> which<00:10:17.870> I 00:10:18.100 --> 00:10:18.110 align:start position:0% a bit of altaf achieve myriad which I 00:10:18.110 --> 00:10:20.590 align:start position:0% a bit of altaf achieve myriad which I think<00:10:18.320> that's<00:10:18.560> some<00:10:18.710> kind<00:10:18.830> of<00:10:19.150> preparation<00:10:20.150> of 00:10:20.590 --> 00:10:20.600 align:start position:0% think that's some kind of preparation of 00:10:20.600 --> 00:10:22.810 align:start position:0% think that's some kind of preparation of this<00:10:20.750> ah<00:10:21.020> I<00:10:21.500> actually<00:10:21.860> not<00:10:22.190> her<00:10:22.310> that<00:10:22.550> is<00:10:22.700> that 00:10:22.810 --> 00:10:22.820 align:start position:0% this ah I actually not her that is that 00:10:22.820 --> 00:10:24.250 align:start position:0% this ah I actually not her that is that something<00:10:23.060> that<00:10:23.090> everyone<00:10:23.600> you're<00:10:24.110> not 00:10:24.250 --> 00:10:24.260 align:start position:0% something that everyone you're not 00:10:24.260 --> 00:10:27.070 align:start position:0% something that everyone you're not Messrs<00:10:24.640> that's<00:10:25.640> what<00:10:26.060> I<00:10:26.090> you<00:10:26.540> know<00:10:26.660> what<00:10:26.930> it 00:10:27.070 --> 00:10:27.080 align:start position:0% Messrs that's what I you know what it 00:10:27.080 --> 00:10:28.960 align:start position:0% Messrs that's what I you know what it does<00:10:27.290> mommy<00:10:27.680> know<00:10:27.950> we<00:10:28.310> don't<00:10:28.670> have<00:10:28.850> talked 00:10:28.960 --> 00:10:28.970 align:start position:0% does mommy know we don't have talked 00:10:28.970 --> 00:10:31.660 align:start position:0% does mommy know we don't have talked about<00:10:29.060> it<00:10:29.180> um<00:10:29.480> yeah<00:10:30.140> I<00:10:30.170> mean<00:10:30.410> there's<00:10:30.670> been 00:10:31.660 --> 00:10:31.670 align:start position:0% about it um yeah I mean there's been 00:10:31.670 --> 00:10:33.970 align:start position:0% about it um yeah I mean there's been quite<00:10:31.820> a<00:10:31.910> few<00:10:32.000> of<00:10:32.360> my<00:10:32.750> proposal<00:10:33.260> to<00:10:33.410> run<00:10:33.590> yarns 00:10:33.970 --> 00:10:33.980 align:start position:0% quite a few of my proposal to run yarns 00:10:33.980 --> 00:10:36.670 align:start position:0% quite a few of my proposal to run yarns and<00:10:34.070> mrs.<00:10:34.490> and<00:10:34.730> mrs.<00:10:34.850> inside<00:10:35.150> of<00:10:35.240> yarn<00:10:35.480> um<00:10:36.290> you 00:10:36.670 --> 00:10:36.680 align:start position:0% and mrs. and mrs. inside of yarn um you 00:10:36.680 --> 00:10:38.140 align:start position:0% and mrs. and mrs. inside of yarn um you know<00:10:36.800> it<00:10:37.040> seems<00:10:37.190> a<00:10:37.310> little<00:10:37.340> weird<00:10:37.550> to<00:10:37.700> me<00:10:37.880> like 00:10:38.140 --> 00:10:38.150 align:start position:0% know it seems a little weird to me like 00:10:38.150 --> 00:10:40.120 align:start position:0% know it seems a little weird to me like within<00:10:38.810> yarn<00:10:39.230> you're<00:10:39.440> already<00:10:39.620> doing<00:10:39.800> to 00:10:40.120 --> 00:10:40.130 align:start position:0% within yarn you're already doing to 00:10:40.130 --> 00:10:41.910 align:start position:0% within yarn you're already doing to level<00:10:40.370> scheduling<00:10:40.700> or<00:10:40.970> are<00:10:41.120> you<00:10:41.270> already<00:10:41.420> have 00:10:41.910 --> 00:10:41.920 align:start position:0% level scheduling or are you already have 00:10:41.920 --> 00:10:44.190 align:start position:0% level scheduling or are you already have you<00:10:42.920> are<00:10:43.010> doing<00:10:43.160> scheduling<00:10:43.460> and<00:10:43.820> then 00:10:44.190 --> 00:10:44.200 align:start position:0% you are doing scheduling and then 00:10:44.200 --> 00:10:46.570 align:start position:0% you are doing scheduling and then sparking<00:10:45.200> scheduling<00:10:45.620> that's<00:10:45.860> right<00:10:45.980> we<00:10:46.430> can 00:10:46.570 --> 00:10:46.580 align:start position:0% sparking scheduling that's right we can 00:10:46.580 --> 00:10:48.550 align:start position:0% sparking scheduling that's right we can hear<00:10:46.760> on<00:10:46.880> scheduling<00:10:47.240> so<00:10:47.420> why<00:10:47.570> another<00:10:48.320> layer 00:10:48.550 --> 00:10:48.560 align:start position:0% hear on scheduling so why another layer 00:10:48.560 --> 00:10:52.240 align:start position:0% hear on scheduling so why another layer is<00:10:48.830> needed<00:10:49.430> a<00:10:49.490> nut<00:10:49.640> sure<00:10:49.970> um<00:10:50.920> you<00:10:51.920> know<00:10:52.010> at 00:10:52.240 --> 00:10:52.250 align:start position:0% is needed a nut sure um you know at 00:10:52.250 --> 00:10:53.560 align:start position:0% is needed a nut sure um you know at least<00:10:52.310> in<00:10:52.550> Malaga<00:10:52.790> maybe<00:10:53.120> is<00:10:53.210> a<00:10:53.240> temporary 00:10:53.560 --> 00:10:53.570 align:start position:0% least in Malaga maybe is a temporary 00:10:53.570 --> 00:10:56.320 align:start position:0% least in Malaga maybe is a temporary solution<00:10:53.810> yeah<00:10:54.290> make<00:10:54.830> sense<00:10:54.980> yes<00:10:55.400> yeah<00:10:56.120> that 00:10:56.320 --> 00:10:56.330 align:start position:0% solution yeah make sense yes yeah that 00:10:56.330 --> 00:10:57.550 align:start position:0% solution yeah make sense yes yeah that was<00:10:56.450> my<00:10:56.600> question<00:10:56.690> as<00:10:56.960> well<00:10:57.110> right<00:10:57.290> because 00:10:57.550 --> 00:10:57.560 align:start position:0% was my question as well right because 00:10:57.560 --> 00:10:59.920 align:start position:0% was my question as well right because obviously<00:10:57.950> it's<00:10:58.370> already<00:10:58.660> the<00:10:59.660> way<00:10:59.750> to<00:10:59.810> do 00:10:59.920 --> 00:10:59.930 align:start position:0% obviously it's already the way to do 00:10:59.930 --> 00:11:01.680 align:start position:0% obviously it's already the way to do performances<00:11:00.650> to<00:11:00.770> basically<00:11:01.100> make<00:11:01.490> the 00:11:01.680 --> 00:11:01.690 align:start position:0% performances to basically make the 00:11:01.690 --> 00:11:03.760 align:start position:0% performances to basically make the schedule<00:11:02.690> Kevin<00:11:02.930> transparent<00:11:03.410> like<00:11:03.530> over 00:11:03.760 --> 00:11:03.770 align:start position:0% schedule Kevin transparent like over 00:11:03.770 --> 00:11:05.200 align:start position:0% schedule Kevin transparent like over assembly<00:11:04.130> right<00:11:04.580> between<00:11:04.820> different 00:11:05.200 --> 00:11:05.210 align:start position:0% assembly right between different 00:11:05.210 --> 00:11:07.330 align:start position:0% assembly right between different frameworks<00:11:05.540> so<00:11:05.870> that<00:11:06.410> kind<00:11:06.590> of<00:11:06.710> what<00:11:07.040> my 00:11:07.330 --> 00:11:07.340 align:start position:0% frameworks so that kind of what my 00:11:07.340 --> 00:11:11.140 align:start position:0% frameworks so that kind of what my personal<00:11:07.730> was<00:11:07.850> ah<00:11:08.330> interesting<00:11:09.020> so<00:11:09.830> so<00:11:10.820> months 00:11:11.140 --> 00:11:11.150 align:start position:0% personal was ah interesting so so months 00:11:11.150 --> 00:11:14.280 align:start position:0% personal was ah interesting so so months definitely<00:11:11.990> the<00:11:12.200> kind<00:11:12.770> of<00:11:12.920> the<00:11:13.640> Golden<00:11:14.120> Age<00:11:14.210> of 00:11:14.280 --> 00:11:14.290 align:start position:0% definitely the kind of the Golden Age of 00:11:14.290 --> 00:11:16.750 align:start position:0% definitely the kind of the Golden Age of spark<00:11:15.290> right<00:11:15.560> there<00:11:15.920> is<00:11:16.040> tremendous<00:11:16.280> interest 00:11:16.750 --> 00:11:16.760 align:start position:0% spark right there is tremendous interest 00:11:16.760 --> 00:11:19.960 align:start position:0% spark right there is tremendous interest we<00:11:17.090> see<00:11:17.300> IBM<00:11:18.080> announcing<00:11:18.860> spark<00:11:19.190> technology 00:11:19.960 --> 00:11:19.970 align:start position:0% we see IBM announcing spark technology 00:11:19.970 --> 00:11:22.840 align:start position:0% we see IBM announcing spark technology center<00:11:20.330> right<00:11:21.260> here<00:11:21.290> on<00:11:21.920> market<00:11:22.280> street<00:11:22.310> and 00:11:22.840 --> 00:11:22.850 align:start position:0% center right here on market street and 00:11:22.850 --> 00:11:27.460 align:start position:0% center right here on market street and get<00:11:23.840> behind<00:11:23.960> spark<00:11:24.710> before<00:11:24.830> us<00:11:25.130> so<00:11:26.320> this<00:11:27.320> is 00:11:27.460 --> 00:11:27.470 align:start position:0% get behind spark before us so this is 00:11:27.470 --> 00:11:28.420 align:start position:0% get behind spark before us so this is something<00:11:27.650> this<00:11:27.860> is<00:11:27.920> this<00:11:28.160> is<00:11:28.250> a<00:11:28.280> fantastic 00:11:28.420 --> 00:11:28.430 align:start position:0% something this is this is a fantastic 00:11:28.430 --> 00:11:31.900 align:start position:0% something this is this is a fantastic time<00:11:28.760> sourcing<00:11:29.360> way<00:11:30.460> some<00:11:31.460> folks<00:11:31.610> are<00:11:31.700> saying 00:11:31.900 --> 00:11:31.910 align:start position:0% time sourcing way some folks are saying 00:11:31.910 --> 00:11:33.850 align:start position:0% time sourcing way some folks are saying maybe<00:11:32.600> couple<00:11:33.020> years<00:11:33.140> ago<00:11:33.200> that<00:11:33.500> Hadoop<00:11:33.740> is 00:11:33.850 --> 00:11:33.860 align:start position:0% maybe couple years ago that Hadoop is 00:11:33.860 --> 00:11:35.110 align:start position:0% maybe couple years ago that Hadoop is going<00:11:34.040> to<00:11:34.100> go<00:11:34.190> with<00:11:34.340> it<00:11:34.400> is<00:11:34.670> actually<00:11:34.820> going<00:11:35.060> to 00:11:35.110 --> 00:11:35.120 align:start position:0% going to go with it is actually going to 00:11:35.120 --> 00:11:37.300 align:start position:0% going to go with it is actually going to be<00:11:35.180> sparked<00:11:35.600> a<00:11:35.810> dupe<00:11:35.960> name<00:11:36.290> is<00:11:36.590> going<00:11:37.070> to<00:11:37.160> be 00:11:37.300 --> 00:11:37.310 align:start position:0% be sparked a dupe name is going to be 00:11:37.310 --> 00:11:39.880 align:start position:0% be sparked a dupe name is going to be there<00:11:37.640> as<00:11:37.880> a<00:11:38.120> brand<00:11:38.510> but<00:11:38.870> in<00:11:39.080> fact<00:11:39.410> is<00:11:39.650> going<00:11:39.830> to 00:11:39.880 --> 00:11:39.890 align:start position:0% there as a brand but in fact is going to 00:11:39.890 --> 00:11:43.060 align:start position:0% there as a brand but in fact is going to be<00:11:40.010> spark<00:11:40.670> is<00:11:40.850> going<00:11:41.030> to<00:11:41.090> be<00:11:41.240> big<00:11:41.660> data<00:11:41.930> now<00:11:42.680> do 00:11:43.060 --> 00:11:43.070 align:start position:0% be spark is going to be big data now do 00:11:43.070 --> 00:11:45.940 align:start position:0% be spark is going to be big data now do you<00:11:43.190> agree<00:11:43.310> with<00:11:43.490> this<00:11:43.790> um<00:11:44.260> so<00:11:45.260> I<00:11:45.290> think<00:11:45.830> it 00:11:45.940 --> 00:11:45.950 align:start position:0% you agree with this um so I think it 00:11:45.950 --> 00:11:47.980 align:start position:0% you agree with this um so I think it partly<00:11:46.370> comes<00:11:46.670> from<00:11:46.970> the<00:11:47.180> original 00:11:47.980 --> 00:11:47.990 align:start position:0% partly comes from the original 00:11:47.990 --> 00:11:50.050 align:start position:0% partly comes from the original completion<00:11:48.590> with<00:11:48.800> Hadoop<00:11:49.280> and<00:11:49.460> hadoop 00:11:50.050 --> 00:11:50.060 align:start position:0% completion with Hadoop and hadoop 00:11:50.060 --> 00:11:52.060 align:start position:0% completion with Hadoop and hadoop mapreduce<00:11:50.360> hmm<00:11:50.960> so<00:11:51.380> yeah<00:11:51.500> I<00:11:51.560> wonder<00:11:51.890> doesn't 00:11:52.060 --> 00:11:52.070 align:start position:0% mapreduce hmm so yeah I wonder doesn't 00:11:52.070 --> 00:11:52.660 align:start position:0% mapreduce hmm so yeah I wonder doesn't believe 00:11:52.660 --> 00:11:52.670 align:start position:0% believe 00:11:52.670 --> 00:11:54.970 align:start position:0% believe that<00:11:52.790> produce<00:11:52.940> is<00:11:53.240> going<00:11:53.420> to<00:11:53.480> go<00:11:54.080> away<00:11:54.200> nobody 00:11:54.970 --> 00:11:54.980 align:start position:0% that produce is going to go away nobody 00:11:54.980 --> 00:11:57.760 align:start position:0% that produce is going to go away nobody knows<00:11:55.100> how<00:11:55.190> soon<00:11:55.720> Burke<00:11:56.720> has<00:11:56.870> a<00:11:57.290> lot<00:11:57.500> of<00:11:57.530> you<00:11:57.650> to 00:11:57.760 --> 00:11:57.770 align:start position:0% knows how soon Burke has a lot of you to 00:11:57.770 --> 00:11:59.800 align:start position:0% knows how soon Burke has a lot of you to catch<00:11:57.920> up<00:11:58.130> but<00:11:58.400> ultimately<00:11:59.150> matters<00:11:59.450> can<00:11:59.780> be 00:11:59.800 --> 00:11:59.810 align:start position:0% catch up but ultimately matters can be 00:11:59.810 --> 00:12:02.320 align:start position:0% catch up but ultimately matters can be totally<00:12:00.050> gone<00:12:00.380> I<00:12:00.590> think<00:12:00.800> spark<00:12:01.130> and<00:12:01.700> satisfy 00:12:02.320 --> 00:12:02.330 align:start position:0% totally gone I think spark and satisfy 00:12:02.330 --> 00:12:04.990 align:start position:0% totally gone I think spark and satisfy all<00:12:02.480> the<00:12:02.600> use<00:12:02.720> cases<00:12:03.310> I'm<00:12:04.310> not<00:12:04.460> convinced<00:12:04.730> with 00:12:04.990 --> 00:12:05.000 align:start position:0% all the use cases I'm not convinced with 00:12:05.000 --> 00:12:06.820 align:start position:0% all the use cases I'm not convinced with the<00:12:05.090> other<00:12:05.240> parts<00:12:05.990> of<00:12:06.080> Hadoop<00:12:06.380> are<00:12:06.560> going<00:12:06.770> to 00:12:06.820 --> 00:12:06.830 align:start position:0% the other parts of Hadoop are going to 00:12:06.830 --> 00:12:09.070 align:start position:0% the other parts of Hadoop are going to go<00:12:06.950> away<00:12:07.040> not<00:12:07.550> convinced<00:12:07.970> that<00:12:08.030> at<00:12:08.270> HDFS<00:12:08.930> is 00:12:09.070 --> 00:12:09.080 align:start position:0% go away not convinced that at HDFS is 00:12:09.080 --> 00:12:11.290 align:start position:0% go away not convinced that at HDFS is going<00:12:09.230> to<00:12:09.290> go<00:12:09.410> away<00:12:09.530> or<00:12:10.190> or<00:12:10.670> you<00:12:10.730> know<00:12:10.910> even<00:12:10.970> if 00:12:11.290 --> 00:12:11.300 align:start position:0% going to go away or or you know even if 00:12:11.300 --> 00:12:12.490 align:start position:0% going to go away or or you know even if it<00:12:11.420> gets<00:12:11.540> replaced<00:12:11.750> by<00:12:11.960> some<00:12:12.140> other<00:12:12.260> file 00:12:12.490 --> 00:12:12.500 align:start position:0% it gets replaced by some other file 00:12:12.500 --> 00:12:15.220 align:start position:0% it gets replaced by some other file system<00:12:12.950> those<00:12:13.220> Hadoop<00:12:13.790> api's<00:12:14.240> I've<00:12:14.870> sold<00:12:15.110> away 00:12:15.220 --> 00:12:15.230 align:start position:0% system those Hadoop api's I've sold away 00:12:15.230 --> 00:12:19.630 align:start position:0% system those Hadoop api's I've sold away mm-hmm<00:12:16.070> I'm<00:12:16.970> interacting<00:12:17.480> you<00:12:17.990> know<00:12:18.080> of<00:12:18.640> the 00:12:19.630 --> 00:12:19.640 align:start position:0% mm-hmm I'm interacting you know of the 00:12:19.640 --> 00:12:21.100 align:start position:0% mm-hmm I'm interacting you know of the processing<00:12:20.210> layer<00:12:20.390> to<00:12:20.570> interact<00:12:20.900> with<00:12:21.050> the 00:12:21.100 --> 00:12:21.110 align:start position:0% processing layer to interact with the 00:12:21.110 --> 00:12:23.320 align:start position:0% processing layer to interact with the storage<00:12:21.440> layer<00:12:21.620> and<00:12:22.460> then<00:12:22.640> the<00:12:23.030> resource 00:12:23.320 --> 00:12:23.330 align:start position:0% storage layer and then the resource 00:12:23.330 --> 00:12:25.840 align:start position:0% storage layer and then the resource management<00:12:23.600> aspect<00:12:23.990> yarn<00:12:24.620> on<00:12:24.710> top<00:12:24.890> of<00:12:25.040> that<00:12:25.160> um 00:12:25.840 --> 00:12:25.850 align:start position:0% management aspect yarn on top of that um 00:12:25.850 --> 00:12:27.430 align:start position:0% management aspect yarn on top of that um and<00:12:26.420> then<00:12:26.570> of<00:12:26.690> course<00:12:26.900> there's<00:12:27.140> the<00:12:27.170> whole 00:12:27.430 --> 00:12:27.440 align:start position:0% and then of course there's the whole 00:12:27.440 --> 00:12:31.920 align:start position:0% and then of course there's the whole broader<00:12:28.370> ecosystem<00:12:28.760> right<00:12:30.260> boom 00:12:31.920 --> 00:12:31.930 align:start position:0% broader ecosystem right boom 00:12:31.930 --> 00:12:34.870 align:start position:0% broader ecosystem right boom century<00:12:32.930> every<00:12:33.080> sort<00:12:33.380> of<00:12:33.440> sort<00:12:33.710> of<00:12:33.740> project<00:12:33.950> so 00:12:34.870 --> 00:12:34.880 align:start position:0% century every sort of sort of project so 00:12:34.880 --> 00:12:36.700 align:start position:0% century every sort of sort of project so of<00:12:35.120> course<00:12:35.150> those<00:12:35.480> aren't<00:12:35.750> technically<00:12:36.500> had 00:12:36.700 --> 00:12:36.710 align:start position:0% of course those aren't technically had 00:12:36.710 --> 00:12:40.510 align:start position:0% of course those aren't technically had you<00:12:36.830> but<00:12:37.340> I<00:12:37.640> wouldn't<00:12:38.480> say<00:12:38.720> that<00:12:38.920> spark<00:12:39.920> spark 00:12:40.510 --> 00:12:40.520 align:start position:0% you but I wouldn't say that spark spark 00:12:40.520 --> 00:12:41.830 align:start position:0% you but I wouldn't say that spark spark has<00:12:40.610> extremely<00:12:40.910> important<00:12:41.510> role<00:12:41.630> but<00:12:41.810> I 00:12:41.830 --> 00:12:41.840 align:start position:0% has extremely important role but I 00:12:41.840 --> 00:12:43.180 align:start position:0% has extremely important role but I wouldn't<00:12:42.140> say<00:12:42.290> that<00:12:42.320> everything<00:12:42.650> revolves 00:12:43.180 --> 00:12:43.190 align:start position:0% wouldn't say that everything revolves 00:12:43.190 --> 00:12:45.940 align:start position:0% wouldn't say that everything revolves around<00:12:43.400> it<00:12:43.700> yes<00:12:44.000> and<00:12:44.270> so<00:12:44.780> we<00:12:44.930> see<00:12:45.110> I<00:12:45.410> think<00:12:45.740> you 00:12:45.940 --> 00:12:45.950 align:start position:0% around it yes and so we see I think you 00:12:45.950 --> 00:12:48.430 align:start position:0% around it yes and so we see I think you see<00:12:46.250> definitely<00:12:46.910> everybody<00:12:47.390> with<00:12:47.990> also<00:12:48.290> led 00:12:48.430 --> 00:12:48.440 align:start position:0% see definitely everybody with also led 00:12:48.440 --> 00:12:50.050 align:start position:0% see definitely everybody with also led to<00:12:48.470> use<00:12:48.620> in<00:12:48.710> HDFS<00:12:49.130> so<00:12:49.310> I<00:12:49.340> think<00:12:49.640> there<00:12:49.790> is<00:12:49.820> not 00:12:50.050 --> 00:12:50.060 align:start position:0% to use in HDFS so I think there is not 00:12:50.060 --> 00:12:52.990 align:start position:0% to use in HDFS so I think there is not really<00:12:50.300> any<00:12:50.990> other<00:12:51.490> simple<00:12:52.490> alternative<00:12:52.670> to 00:12:52.990 --> 00:12:53.000 align:start position:0% really any other simple alternative to 00:12:53.000 --> 00:12:54.460 align:start position:0% really any other simple alternative to just<00:12:53.180> break<00:12:53.360> into<00:12:53.510> in<00:12:53.630> stores<00:12:53.840> and<00:12:54.020> um<00:12:54.170> txt 00:12:54.460 --> 00:12:54.470 align:start position:0% just break into in stores and um txt 00:12:54.470 --> 00:12:56.860 align:start position:0% just break into in stores and um txt files<00:12:54.650> into<00:12:54.830> it<00:12:55.040> forever<00:12:55.730> right<00:12:55.970> or<00:12:56.240> up<00:12:56.270> arcade 00:12:56.860 --> 00:12:56.870 align:start position:0% files into it forever right or up arcade 00:12:56.870 --> 00:12:59.490 align:start position:0% files into it forever right or up arcade or<00:12:57.050> doing<00:12:57.500> that<00:12:57.650> so<00:12:57.830> what<00:12:58.550> we<00:12:58.760> see<00:12:58.910> a<00:12:58.940> lot<00:12:59.120> of 00:12:59.490 --> 00:12:59.500 align:start position:0% or doing that so what we see a lot of 00:12:59.500 --> 00:13:03.190 align:start position:0% or doing that so what we see a lot of places<00:13:00.500> running<00:13:01.070> down<00:13:01.760> some<00:13:02.150> somebody<00:13:02.840> came 00:13:03.190 --> 00:13:03.200 align:start position:0% places running down some somebody came 00:13:03.200 --> 00:13:05.440 align:start position:0% places running down some somebody came up<00:13:03.260> with<00:13:03.410> the<00:13:03.530> term<00:13:03.950> smack<00:13:04.340> which<00:13:04.580> is<00:13:04.730> spark 00:13:05.440 --> 00:13:05.450 align:start position:0% up with the term smack which is spark 00:13:05.450 --> 00:13:10.660 align:start position:0% up with the term smack which is spark mrs.<00:13:06.460> acha<00:13:07.600> Cassandra<00:13:08.600> Kafka<00:13:09.080> so<00:13:09.710> so<00:13:10.520> it's 00:13:10.660 --> 00:13:10.670 align:start position:0% mrs. acha Cassandra Kafka so so it's 00:13:10.670 --> 00:13:12.130 align:start position:0% mrs. acha Cassandra Kafka so so it's kind<00:13:10.910> of<00:13:10.940> a<00:13:11.060> new<00:13:11.210> stack<00:13:11.510> yes<00:13:11.750> I<00:13:11.870> think<00:13:11.930> that's 00:13:12.130 --> 00:13:12.140 align:start position:0% kind of a new stack yes I think that's 00:13:12.140 --> 00:13:14.800 align:start position:0% kind of a new stack yes I think that's it<00:13:12.380> folks<00:13:12.920> came<00:13:13.280> up<00:13:13.400> with<00:13:13.520> this<00:13:13.640> and<00:13:14.120> I<00:13:14.540> told 00:13:14.800 --> 00:13:14.810 align:start position:0% it folks came up with this and I told 00:13:14.810 --> 00:13:16.000 align:start position:0% it folks came up with this and I told you<00:13:14.900> know<00:13:15.020> this<00:13:15.290> should<00:13:15.440> be<00:13:15.530> an<00:13:15.590> extensional 00:13:16.000 --> 00:13:16.010 align:start position:0% you know this should be an extensional 00:13:16.010 --> 00:13:18.300 align:start position:0% you know this should be an extensional it's<00:13:16.160> my<00:13:16.280> card<00:13:16.520> which<00:13:16.820> is<00:13:16.850> how<00:13:17.330> available 00:13:18.300 --> 00:13:18.310 align:start position:0% it's my card which is how available 00:13:18.310 --> 00:13:21.250 align:start position:0% it's my card which is how available resilient<00:13:19.540> distributed<00:13:20.540> is<00:13:20.630> not<00:13:20.780> in<00:13:20.900> it<00:13:21.020> all 00:13:21.250 --> 00:13:21.260 align:start position:0% resilient distributed is not in it all 00:13:21.260 --> 00:13:23.890 align:start position:0% resilient distributed is not in it all right<00:13:21.380> you<00:13:21.530> just<00:13:21.680> by<00:13:22.460> default<00:13:22.730> it<00:13:22.910> is<00:13:23.030> so<00:13:23.540> I'm 00:13:23.890 --> 00:13:23.900 align:start position:0% right you just by default it is so I'm 00:13:23.900 --> 00:13:25.690 align:start position:0% right you just by default it is so I'm not<00:13:24.230> like<00:13:24.470> a<00:13:24.500> pirate<00:13:24.770> that's<00:13:25.190> right<00:13:25.400> that's 00:13:25.690 --> 00:13:25.700 align:start position:0% not like a pirate that's right that's 00:13:25.700 --> 00:13:28.180 align:start position:0% not like a pirate that's right that's right<00:13:25.760> that's<00:13:25.910> right<00:13:26.120> about<00:13:26.270> so<00:13:26.630> um<00:13:26.990> so<00:13:27.560> do<00:13:28.130> you 00:13:28.180 --> 00:13:28.190 align:start position:0% right that's right about so um so do you 00:13:28.190 --> 00:13:31.030 align:start position:0% right that's right about so um so do you think<00:13:28.220> that<00:13:28.930> running<00:13:29.930> spark<00:13:30.650> on<00:13:30.800> top<00:13:30.920> of 00:13:31.030 --> 00:13:31.040 align:start position:0% think that running spark on top of 00:13:31.040 --> 00:13:32.650 align:start position:0% think that running spark on top of Cassandra<00:13:31.730> and<00:13:32.120> the<00:13:32.210> rest<00:13:32.360> of<00:13:32.450> small 00:13:32.650 --> 00:13:32.660 align:start position:0% Cassandra and the rest of small 00:13:32.660 --> 00:13:36.160 align:start position:0% Cassandra and the rest of small suitcases<00:13:33.410> ah<00:13:33.940> you<00:13:34.940> know<00:13:35.090> people<00:13:35.870> doing 00:13:36.160 --> 00:13:36.170 align:start position:0% suitcases ah you know people doing 00:13:36.170 --> 00:13:40.360 align:start position:0% suitcases ah you know people doing industry<00:13:36.590> or<00:13:36.860> it's<00:13:37.820> still<00:13:38.210> necessary<00:13:39.190> in<00:13:40.190> most 00:13:40.360 --> 00:13:40.370 align:start position:0% industry or it's still necessary in most 00:13:40.370 --> 00:13:44.920 align:start position:0% industry or it's still necessary in most cases<00:13:40.490> to<00:13:41.000> set<00:13:41.720> up<00:13:41.900> a<00:13:42.700> HDFS<00:13:43.700> and<00:13:44.090> remember<00:13:44.810> the 00:13:44.920 --> 00:13:44.930 align:start position:0% cases to set up a HDFS and remember the 00:13:44.930 --> 00:13:48.100 align:start position:0% cases to set up a HDFS and remember the full<00:13:45.170> system<00:13:45.770> I<00:13:46.160> think<00:13:47.150> a<00:13:47.180> question<00:13:47.540> I<00:13:47.720> mean<00:13:47.870> it 00:13:48.100 --> 00:13:48.110 align:start position:0% full system I think a question I mean it 00:13:48.110 --> 00:13:49.120 align:start position:0% full system I think a question I mean it depends<00:13:48.500> what<00:13:48.620> you're<00:13:48.740> trying<00:13:48.920> to<00:13:49.010> optimize 00:13:49.120 --> 00:13:49.130 align:start position:0% depends what you're trying to optimize 00:13:49.130 --> 00:13:53.260 align:start position:0% depends what you're trying to optimize for<00:13:49.450> um<00:13:50.590> what<00:13:51.590> cassandra<00:13:52.190> is<00:13:52.220> great<00:13:52.460> for<00:13:52.670> is<00:13:52.940> if 00:13:53.260 --> 00:13:53.270 align:start position:0% for um what cassandra is great for is if 00:13:53.270 --> 00:13:56.020 align:start position:0% for um what cassandra is great for is if you<00:13:53.540> really<00:13:53.840> want<00:13:54.050> to<00:13:54.140> have<00:13:54.980> like<00:13:55.700> random 00:13:56.020 --> 00:13:56.030 align:start position:0% you really want to have like random 00:13:56.030 --> 00:13:58.150 align:start position:0% you really want to have like random writes<00:13:56.210> and<00:13:56.450> reads<00:13:56.690> book<00:13:57.380> both<00:13:57.500> Cassandra<00:13:57.980> in 00:13:58.150 --> 00:13:58.160 align:start position:0% writes and reads book both Cassandra in 00:13:58.160 --> 00:13:58.720 align:start position:0% writes and reads book both Cassandra in space 00:13:58.720 --> 00:13:58.730 align:start position:0% space 00:13:58.730 --> 00:14:00.850 align:start position:0% space no<00:13:58.940> sequel<00:13:59.240> store<00:13:59.449> optimized<00:14:00.079> for<00:14:00.350> a<00:14:00.680> huge 00:14:00.850 --> 00:14:00.860 align:start position:0% no sequel store optimized for a huge 00:14:00.860 --> 00:14:03.519 align:start position:0% no sequel store optimized for a huge data<00:14:01.100> and<00:14:01.430> then<00:14:01.579> getting<00:14:02.510> doing<00:14:03.290> point 00:14:03.519 --> 00:14:03.529 align:start position:0% data and then getting doing point 00:14:03.529 --> 00:14:05.530 align:start position:0% data and then getting doing point lookups<00:14:03.889> that<00:14:04.010> kind<00:14:04.190> of<00:14:04.220> stuff<00:14:04.490> if<00:14:05.360> what<00:14:05.510> you 00:14:05.530 --> 00:14:05.540 align:start position:0% lookups that kind of stuff if what you 00:14:05.540 --> 00:14:08.290 align:start position:0% lookups that kind of stuff if what you really<00:14:05.779> care<00:14:06.019> about<00:14:06.079> is<00:14:06.610> top<00:14:07.610> performance<00:14:08.120> for 00:14:08.290 --> 00:14:08.300 align:start position:0% really care about is top performance for 00:14:08.300 --> 00:14:11.230 align:start position:0% really care about is top performance for scans<00:14:08.540> and<00:14:08.810> doing<00:14:09.050> batch<00:14:09.260> batch<00:14:10.070> data<00:14:10.339> flip<00:14:11.120> of 00:14:11.230 --> 00:14:11.240 align:start position:0% scans and doing batch batch data flip of 00:14:11.240 --> 00:14:13.389 align:start position:0% scans and doing batch batch data flip of stuff<00:14:11.449> then<00:14:12.260> I<00:14:12.560> think<00:14:12.740> issue<00:14:12.860> if<00:14:13.040> that's<00:14:13.190> going 00:14:13.389 --> 00:14:13.399 align:start position:0% stuff then I think issue if that's going 00:14:13.399 --> 00:14:15.639 align:start position:0% stuff then I think issue if that's going to<00:14:13.459> be<00:14:13.730> what<00:14:14.300> you<00:14:14.420> want<00:14:14.630> for<00:14:14.870> a<00:14:14.899> while<00:14:15.079> mm-hmm 00:14:15.639 --> 00:14:15.649 align:start position:0% to be what you want for a while mm-hmm 00:14:15.649 --> 00:14:18.400 align:start position:0% to be what you want for a while mm-hmm oh<00:14:16.389> yeah<00:14:17.389> I<00:14:17.420> mean<00:14:17.630> it's<00:14:17.690> also<00:14:17.959> a<00:14:18.110> little<00:14:18.290> bit 00:14:18.400 --> 00:14:18.410 align:start position:0% oh yeah I mean it's also a little bit 00:14:18.410 --> 00:14:21.160 align:start position:0% oh yeah I mean it's also a little bit less<00:14:18.589> operationally<00:14:19.250> complex<00:14:19.880> to<00:14:20.810> deal<00:14:20.990> with 00:14:21.160 --> 00:14:21.170 align:start position:0% less operationally complex to deal with 00:14:21.170 --> 00:14:22.870 align:start position:0% less operationally complex to deal with a<00:14:21.199> no<00:14:21.649> sequel<00:14:21.920> database<00:14:22.279> for<00:14:22.459> just<00:14:22.610> until<00:14:22.790> with 00:14:22.870 --> 00:14:22.880 align:start position:0% a no sequel database for just until with 00:14:22.880 --> 00:14:26.259 align:start position:0% a no sequel database for just until with the<00:14:22.940> file<00:14:23.089> system<00:14:23.240> right<00:14:24.339> so<00:14:25.339> see<00:14:25.699> i<00:14:25.730> don't<00:14:26.029> i 00:14:26.259 --> 00:14:26.269 align:start position:0% the file system right so see i don't i 00:14:26.269 --> 00:14:28.449 align:start position:0% the file system right so see i don't i don't<00:14:26.300> see<00:14:26.540> i<00:14:26.990> cain't<00:14:27.620> are<00:14:27.709> taking<00:14:27.980> over<00:14:28.070> but 00:14:28.449 --> 00:14:28.459 align:start position:0% don't see i cain't are taking over but 00:14:28.459 --> 00:14:31.480 align:start position:0% don't see i cain't are taking over but they're<00:14:28.850> certainly<00:14:29.029> valid<00:14:30.190> right<00:14:31.190> and<00:14:31.399> the 00:14:31.480 --> 00:14:31.490 align:start position:0% they're certainly valid right and the 00:14:31.490 --> 00:14:33.009 align:start position:0% they're certainly valid right and the kind<00:14:31.699> of<00:14:31.790> thing<00:14:31.940> that<00:14:31.970> relates<00:14:32.480> to<00:14:32.750> another 00:14:33.009 --> 00:14:33.019 align:start position:0% kind of thing that relates to another 00:14:33.019 --> 00:14:35.650 align:start position:0% kind of thing that relates to another development<00:14:33.980> so<00:14:34.310> we<00:14:34.550> recently<00:14:34.820> CA<00:14:35.180> a<00:14:35.360> passive 00:14:35.650 --> 00:14:35.660 align:start position:0% development so we recently CA a passive 00:14:35.660 --> 00:14:37.840 align:start position:0% development so we recently CA a passive link<00:14:35.899> and<00:14:36.170> I<00:14:36.560> am<00:14:36.740> trying<00:14:37.010> the<00:14:37.339> space<00:14:37.639> of 00:14:37.840 --> 00:14:37.850 align:start position:0% link and I am trying the space of 00:14:37.850 --> 00:14:39.460 align:start position:0% link and I am trying the space of streaming<00:14:38.120> systems<00:14:38.510> and<00:14:38.690> today<00:14:38.990> the<00:14:39.170> second 00:14:39.460 --> 00:14:39.470 align:start position:0% streaming systems and today the second 00:14:39.470 --> 00:14:41.170 align:start position:0% streaming systems and today the second talk<00:14:39.620> which<00:14:39.740> would<00:14:39.889> have<00:14:40.040> is<00:14:40.279> on<00:14:40.790> min<00:14:40.970> data 00:14:41.170 --> 00:14:41.180 align:start position:0% talk which would have is on min data 00:14:41.180 --> 00:14:43.689 align:start position:0% talk which would have is on min data which<00:14:41.480> is<00:14:41.510> a<00:14:41.660> storm<00:14:41.959> like<00:14:42.279> very<00:14:43.279> efficient 00:14:43.689 --> 00:14:43.699 align:start position:0% which is a storm like very efficient 00:14:43.699 --> 00:14:46.150 align:start position:0% which is a storm like very efficient streaming<00:14:44.540> system<00:14:44.779> so<00:14:45.079> these<00:14:45.290> systems<00:14:45.680> put 00:14:46.150 --> 00:14:46.160 align:start position:0% streaming system so these systems put 00:14:46.160 --> 00:14:48.220 align:start position:0% streaming system so these systems put streaming<00:14:46.610> first<00:14:46.880> on<00:14:47.180> patch<00:14:47.360> second<00:14:47.839> I<00:14:47.899> flink 00:14:48.220 --> 00:14:48.230 align:start position:0% streaming first on patch second I flink 00:14:48.230 --> 00:14:50.410 align:start position:0% streaming first on patch second I flink make<00:14:48.440> this<00:14:48.560> point<00:14:48.829> that<00:14:49.040> there's<00:14:49.790> a<00:14:49.880> continuum 00:14:50.410 --> 00:14:50.420 align:start position:0% make this point that there's a continuum 00:14:50.420 --> 00:14:53.590 align:start position:0% make this point that there's a continuum and<00:14:51.079> batch<00:14:51.709> is<00:14:51.980> just<00:14:52.250> a<00:14:52.399> limit<00:14:52.880> version<00:14:53.360> of<00:14:53.510> a 00:14:53.590 --> 00:14:53.600 align:start position:0% and batch is just a limit version of a 00:14:53.600 --> 00:14:55.930 align:start position:0% and batch is just a limit version of a stream<00:14:53.990> right<00:14:54.500> so<00:14:54.860> so<00:14:55.310> where<00:14:55.490> are<00:14:55.670> you<00:14:55.790> know<00:14:55.910> a 00:14:55.930 --> 00:14:55.940 align:start position:0% stream right so so where are you know a 00:14:55.940 --> 00:14:57.970 align:start position:0% stream right so so where are you know a third<00:14:56.240> of<00:14:56.269> storm<00:14:56.750> and<00:14:57.079> then<00:14:57.230> spark<00:14:57.589> was<00:14:57.769> any 00:14:57.970 --> 00:14:57.980 align:start position:0% third of storm and then spark was any 00:14:57.980 --> 00:14:59.860 align:start position:0% third of storm and then spark was any initial<00:14:58.399> page<00:14:58.730> but<00:14:58.970> responded<00:14:59.540> with<00:14:59.690> mini 00:14:59.860 --> 00:14:59.870 align:start position:0% initial page but responded with mini 00:14:59.870 --> 00:15:02.860 align:start position:0% initial page but responded with mini patching<00:15:00.320> now<00:15:00.500> we<00:15:01.459> have<00:15:01.880> language<00:15:02.360> basically 00:15:02.860 --> 00:15:02.870 align:start position:0% patching now we have language basically 00:15:02.870 --> 00:15:05.230 align:start position:0% patching now we have language basically do<00:15:03.380> instrument<00:15:03.860> first<00:15:04.069> and<00:15:04.370> beige<00:15:04.880> being 00:15:05.230 --> 00:15:05.240 align:start position:0% do instrument first and beige being 00:15:05.240 --> 00:15:08.079 align:start position:0% do instrument first and beige being eliminated<00:15:05.630> right<00:15:06.350> so<00:15:06.680> I<00:15:06.709> wonder<00:15:07.339> right<00:15:07.850> you 00:15:08.079 --> 00:15:08.089 align:start position:0% eliminated right so I wonder right you 00:15:08.089 --> 00:15:09.850 align:start position:0% eliminated right so I wonder right you guys<00:15:08.240> because<00:15:08.720> they're<00:15:08.899> essentially<00:15:09.620> serve 00:15:09.850 --> 00:15:09.860 align:start position:0% guys because they're essentially serve 00:15:09.860 --> 00:15:12.759 align:start position:0% guys because they're essentially serve your<00:15:10.069> customers<00:15:10.360> right<00:15:11.360> how<00:15:12.139> do<00:15:12.199> you<00:15:12.380> see<00:15:12.620> the 00:15:12.759 --> 00:15:12.769 align:start position:0% your customers right how do you see the 00:15:12.769 --> 00:15:17.079 align:start position:0% your customers right how do you see the space<00:15:13.130> of<00:15:13.339> all<00:15:13.459> being<00:15:13.779> do<00:15:14.779> you<00:15:14.870> see<00:15:15.199> spark<00:15:16.089> and 00:15:17.079 --> 00:15:17.089 align:start position:0% space of all being do you see spark and 00:15:17.089 --> 00:15:18.819 align:start position:0% space of all being do you see spark and flame<00:15:17.420> kind<00:15:17.899> of<00:15:18.019> and<00:15:18.170> similar<00:15:18.440> systems 00:15:18.819 --> 00:15:18.829 align:start position:0% flame kind of and similar systems 00:15:18.829 --> 00:15:21.400 align:start position:0% flame kind of and similar systems co-evolving<00:15:19.630> woofers<00:15:20.630> is<00:15:20.810> this<00:15:21.050> kind<00:15:21.350> of 00:15:21.400 --> 00:15:21.410 align:start position:0% co-evolving woofers is this kind of 00:15:21.410 --> 00:15:24.069 align:start position:0% co-evolving woofers is this kind of steering<00:15:22.100> first<00:15:22.459> base<00:15:22.670> second<00:15:23.089> base<00:15:23.240> first<00:15:23.750> in 00:15:24.069 --> 00:15:24.079 align:start position:0% steering first base second base first in 00:15:24.079 --> 00:15:27.040 align:start position:0% steering first base second base first in a<00:15:24.199> second<00:15:24.589> dichotomy<00:15:25.550> is<00:15:25.610> it<00:15:26.120> you<00:15:26.720> know<00:15:26.870> kind 00:15:27.040 --> 00:15:27.050 align:start position:0% a second dichotomy is it you know kind 00:15:27.050 --> 00:15:28.990 align:start position:0% a second dichotomy is it you know kind of<00:15:27.079> reduced<00:15:27.829> oh<00:15:27.949> this<00:15:28.220> it's<00:15:28.370> hard<00:15:28.459> to<00:15:28.730> change 00:15:28.990 --> 00:15:29.000 align:start position:0% of reduced oh this it's hard to change 00:15:29.000 --> 00:15:31.960 align:start position:0% of reduced oh this it's hard to change for<00:15:29.360> smaller<00:15:29.600> essence<00:15:30.139> or<00:15:30.920> fall<00:15:31.459> fling<00:15:31.730> to 00:15:31.960 --> 00:15:31.970 align:start position:0% for smaller essence or fall fling to 00:15:31.970 --> 00:15:33.579 align:start position:0% for smaller essence or fall fling to achieve<00:15:32.360> the<00:15:32.510> same<00:15:32.660> best<00:15:32.959> performance<00:15:33.440> right 00:15:33.579 --> 00:15:33.589 align:start position:0% achieve the same best performance right 00:15:33.589 --> 00:15:36.460 align:start position:0% achieve the same best performance right i<00:15:33.620> don't<00:15:33.740> think<00:15:33.920> it<00:15:34.220> kind<00:15:34.550> of<00:15:34.839> relates<00:15:35.839> to<00:15:36.139> this 00:15:36.460 --> 00:15:36.470 align:start position:0% i don't think it kind of relates to this 00:15:36.470 --> 00:15:38.170 align:start position:0% i don't think it kind of relates to this edge<00:15:36.680> defensive<00:15:37.190> that<00:15:37.310> because<00:15:37.550> the<00:15:37.790> normal 00:15:38.170 --> 00:15:38.180 align:start position:0% edge defensive that because the normal 00:15:38.180 --> 00:15:39.879 align:start position:0% edge defensive that because the normal setup<00:15:38.480> right<00:15:38.660> now<00:15:38.810> she's<00:15:39.050> done<00:15:39.470> gigantic 00:15:39.879 --> 00:15:39.889 align:start position:0% setup right now she's done gigantic 00:15:39.889 --> 00:15:42.009 align:start position:0% setup right now she's done gigantic amount<00:15:40.100> of<00:15:40.190> stuff<00:15:40.399> into<00:15:40.730> the<00:15:40.760> APIs<00:15:41.120> then<00:15:41.750> read 00:15:42.009 --> 00:15:42.019 align:start position:0% amount of stuff into the APIs then read 00:15:42.019 --> 00:15:43.870 align:start position:0% amount of stuff into the APIs then read some<00:15:42.199> subset<00:15:42.529> of<00:15:42.560> it<00:15:42.740> the<00:15:42.860> membrane<00:15:43.250> spark<00:15:43.699> to 00:15:43.870 --> 00:15:43.880 align:start position:0% some subset of it the membrane spark to 00:15:43.880 --> 00:15:45.850 align:start position:0% some subset of it the membrane spark to beijing<00:15:44.269> ready<00:15:44.449> and<00:15:44.779> ghetto<00:15:45.110> streaming<00:15:45.620> comes 00:15:45.850 --> 00:15:45.860 align:start position:0% beijing ready and ghetto streaming comes 00:15:45.860 --> 00:15:48.129 align:start position:0% beijing ready and ghetto streaming comes out<00:15:46.040> of<00:15:46.220> that<00:15:46.250> one<00:15:46.550> in<00:15:46.699> production<00:15:47.120> nice<00:15:47.300> you 00:15:48.129 --> 00:15:48.139 align:start position:0% out of that one in production nice you 00:15:48.139 --> 00:15:49.660 align:start position:0% out of that one in production nice you put<00:15:48.440> it<00:15:48.560> into<00:15:48.829> a<00:15:48.860> stream<00:15:49.100> in<00:15:49.220> context<00:15:49.550> but 00:15:49.660 --> 00:15:49.670 align:start position:0% put it into a stream in context but 00:15:49.670 --> 00:15:51.340 align:start position:0% put it into a stream in context but essentially<00:15:50.180> assume<00:15:50.389> this<00:15:50.510> batch<00:15:50.720> model<00:15:51.170> so<00:15:51.290> i 00:15:51.340 --> 00:15:51.350 align:start position:0% essentially assume this batch model so i 00:15:51.350 --> 00:15:54.009 align:start position:0% essentially assume this batch model so i think<00:15:51.410> that's<00:15:51.680> kind<00:15:51.920> of<00:15:52.089> why<00:15:53.089> is<00:15:53.149> it<00:15:53.360> rather 00:15:54.009 --> 00:15:54.019 align:start position:0% think that's kind of why is it rather 00:15:54.019 --> 00:15:55.629 align:start position:0% think that's kind of why is it rather than<00:15:54.170> that<00:15:54.410> right<00:15:54.980> yeah<00:15:55.190> i<00:15:55.220> mean<00:15:55.370> so<00:15:55.519> i<00:15:55.550> think 00:15:55.629 --> 00:15:55.639 align:start position:0% than that right yeah i mean so i think 00:15:55.639 --> 00:15:57.759 align:start position:0% than that right yeah i mean so i think first<00:15:56.060> of<00:15:56.240> all<00:15:56.420> you<00:15:56.630> know<00:15:56.750> giving<00:15:57.529> this<00:15:57.589> to<00:15:57.709> the 00:15:57.759 --> 00:15:57.769 align:start position:0% first of all you know giving this to the 00:15:57.769 --> 00:15:59.410 align:start position:0% first of all you know giving this to the continuum<00:15:58.160> is<00:15:58.279> great<00:15:58.519> primarily<00:15:59.120> because<00:15:59.209> it 00:15:59.410 --> 00:15:59.420 align:start position:0% continuum is great primarily because it 00:15:59.420 --> 00:16:00.260 align:start position:0% continuum is great primarily because it allows<00:15:59.569> you<00:15:59.690> to<00:15:59.779> have<00:15:59.899> you<00:16:00.019> know 00:16:00.260 --> 00:16:00.270 align:start position:0% allows you to have you know 00:16:00.270 --> 00:16:02.600 align:start position:0% allows you to have you know ap<00:16:00.600> is<00:16:00.720> for<00:16:01.050> these<00:16:01.140> two<00:16:01.410> things<00:16:01.620> yeah<00:16:01.980> what 00:16:02.600 --> 00:16:02.610 align:start position:0% ap is for these two things yeah what 00:16:02.610 --> 00:16:04.880 align:start position:0% ap is for these two things yeah what realize<00:16:03.030> maybe<00:16:03.360> even<00:16:03.570> a<00:16:03.690> while<00:16:03.840> ago<00:16:04.110> if<00:16:04.530> you 00:16:04.880 --> 00:16:04.890 align:start position:0% realize maybe even a while ago if you 00:16:04.890 --> 00:16:07.130 align:start position:0% realize maybe even a while ago if you can<00:16:05.220> create<00:16:06.210> platforms<00:16:06.420> that<00:16:06.750> make<00:16:06.960> you<00:16:07.050> able 00:16:07.130 --> 00:16:07.140 align:start position:0% can create platforms that make you able 00:16:07.140 --> 00:16:08.780 align:start position:0% can create platforms that make you able to<00:16:07.290> use<00:16:07.470> the<00:16:07.590> same<00:16:07.710> code<00:16:07.980> or<00:16:08.370> fashion 00:16:08.780 --> 00:16:08.790 align:start position:0% to use the same code or fashion 00:16:08.790 --> 00:16:12.170 align:start position:0% to use the same code or fashion pre-processing<00:16:09.510> um<00:16:10.380> in<00:16:10.860> terms<00:16:11.100> of<00:16:11.550> support 00:16:12.170 --> 00:16:12.180 align:start position:0% pre-processing um in terms of support 00:16:12.180 --> 00:16:13.880 align:start position:0% pre-processing um in terms of support for<00:16:12.300> this<00:16:12.420> fling<00:16:12.690> so<00:16:13.020> I<00:16:13.050> would<00:16:13.680> you<00:16:13.860> know 00:16:13.880 --> 00:16:13.890 align:start position:0% for this fling so I would you know 00:16:13.890 --> 00:16:16.070 align:start position:0% for this fling so I would you know always<00:16:14.280> say<00:16:14.490> if<00:16:14.970> you<00:16:15.090> can<00:16:15.330> one<00:16:15.690> system<00:16:15.960> is 00:16:16.070 --> 00:16:16.080 align:start position:0% always say if you can one system is 00:16:16.080 --> 00:16:17.540 align:start position:0% always say if you can one system is better<00:16:16.200> than<00:16:16.380> two<00:16:16.560> so<00:16:16.920> if<00:16:17.070> it<00:16:17.160> turns<00:16:17.370> out<00:16:17.490> that 00:16:17.540 --> 00:16:17.550 align:start position:0% better than two so if it turns out that 00:16:17.550 --> 00:16:19.550 align:start position:0% better than two so if it turns out that spark<00:16:17.970> into<00:16:18.180> everything<00:16:18.360> definitely<00:16:19.260> can<00:16:19.380> do 00:16:19.550 --> 00:16:19.560 align:start position:0% spark into everything definitely can do 00:16:19.560 --> 00:16:23.320 align:start position:0% spark into everything definitely can do um<00:16:19.860> having<00:16:20.850> that<00:16:21.240> in<00:16:21.570> the<00:16:21.690> mix<00:16:21.900> you<00:16:22.170> know 00:16:23.320 --> 00:16:23.330 align:start position:0% um having that in the mix you know 00:16:23.330 --> 00:16:26.240 align:start position:0% um having that in the mix you know additional<00:16:24.330> overhead<00:16:24.570> on<00:16:25.530> the<00:16:25.650> other<00:16:25.800> hand<00:16:25.980> if 00:16:26.240 --> 00:16:26.250 align:start position:0% additional overhead on the other hand if 00:16:26.250 --> 00:16:29.210 align:start position:0% additional overhead on the other hand if it<00:16:26.370> turns<00:16:26.580> out<00:16:26.730> that<00:16:26.790> flink<00:16:27.380> sort<00:16:28.380> of<00:16:28.470> you<00:16:29.190> know 00:16:29.210 --> 00:16:29.220 align:start position:0% it turns out that flink sort of you know 00:16:29.220 --> 00:16:30.800 align:start position:0% it turns out that flink sort of you know this<00:16:29.430> business<00:16:29.640> a<00:16:29.940> long<00:16:29.970> way<00:16:30.270> to<00:16:30.330> go<00:16:30.480> i<00:16:30.510> believe 00:16:30.800 --> 00:16:30.810 align:start position:0% this business a long way to go i believe 00:16:30.810 --> 00:16:35.180 align:start position:0% this business a long way to go i believe have<00:16:30.930> both<00:16:31.110> spark<00:16:31.470> and<00:16:31.680> flank<00:16:31.950> to<00:16:32.190> be<00:16:33.120> a<00:16:34.190> full 00:16:35.180 --> 00:16:35.190 align:start position:0% have both spark and flank to be a full 00:16:35.190 --> 00:16:37.640 align:start position:0% have both spark and flank to be a full training<00:16:35.580> system<00:16:35.970> whatever<00:16:36.270> that<00:16:36.480> means<00:16:36.650> so 00:16:37.640 --> 00:16:37.650 align:start position:0% training system whatever that means so 00:16:37.650 --> 00:16:39.770 align:start position:0% training system whatever that means so if<00:16:37.980> link<00:16:38.310> is<00:16:38.460> somehow<00:16:38.670> way<00:16:39.060> faster<00:16:39.450> at<00:16:39.480> getting 00:16:39.770 --> 00:16:39.780 align:start position:0% if link is somehow way faster at getting 00:16:39.780 --> 00:16:42.020 align:start position:0% if link is somehow way faster at getting there<00:16:39.930> then<00:16:40.440> the<00:16:40.950> spark<00:16:41.310> but<00:16:41.490> i<00:16:41.580> don't<00:16:41.760> see<00:16:41.790> any 00:16:42.020 --> 00:16:42.030 align:start position:0% there then the spark but i don't see any 00:16:42.030 --> 00:16:44.030 align:start position:0% there then the spark but i don't see any reason<00:16:42.090> why<00:16:42.390> we<00:16:42.630> wouldn't<00:16:42.660> you<00:16:43.110> know<00:16:43.140> in<00:16:43.620> cdh 00:16:44.030 --> 00:16:44.040 align:start position:0% reason why we wouldn't you know in cdh 00:16:44.040 --> 00:16:45.590 align:start position:0% reason why we wouldn't you know in cdh let's<00:16:44.310> say<00:16:44.430> include<00:16:44.820> that<00:16:44.970> in<00:16:45.090> our<00:16:45.150> platform 00:16:45.590 --> 00:16:45.600 align:start position:0% let's say include that in our platform 00:16:45.600 --> 00:16:49.340 align:start position:0% let's say include that in our platform mm-hmm<00:16:46.050> okay<00:16:46.980> and<00:16:47.250> I<00:16:47.790> guess<00:16:48.030> my<00:16:48.360> wife<00:16:48.780> ambition 00:16:49.340 --> 00:16:49.350 align:start position:0% mm-hmm okay and I guess my wife ambition 00:16:49.350 --> 00:16:51.560 align:start position:0% mm-hmm okay and I guess my wife ambition will<00:16:49.470> be<00:16:49.620> force<00:16:50.010> on<00:16:50.310> Scola<00:16:50.580> because<00:16:51.030> in<00:16:51.360> a<00:16:51.540> way 00:16:51.560 --> 00:16:51.570 align:start position:0% will be force on Scola because in a way 00:16:51.570 --> 00:16:55.130 align:start position:0% will be force on Scola because in a way of<00:16:51.840> weeks<00:16:52.230> Kyle<00:16:52.560> shop<00:16:52.860> we<00:16:53.630> there's<00:16:54.630> a<00:16:54.690> real 00:16:55.130 --> 00:16:55.140 align:start position:0% of weeks Kyle shop we there's a real 00:16:55.140 --> 00:16:59.810 align:start position:0% of weeks Kyle shop we there's a real difference<00:16:55.290> of<00:16:55.590> Scala<00:16:55.890> and<00:16:56.330> and<00:16:58.550> sabari<00:16:59.550> can 00:16:59.810 --> 00:16:59.820 align:start position:0% difference of Scala and and sabari can 00:16:59.820 --> 00:17:02.750 align:start position:0% difference of Scala and and sabari can and<00:17:00.330> so<00:17:01.020> natural<00:17:01.740> use<00:17:02.070> a<00:17:02.100> scholar<00:17:02.460> of<00:17:02.700> the 00:17:02.750 --> 00:17:02.760 align:start position:0% and so natural use a scholar of the 00:17:02.760 --> 00:17:05.180 align:start position:0% and so natural use a scholar of the spark<00:17:03.210> and<00:17:03.560> almost<00:17:04.560> everything<00:17:04.950> I<00:17:05.010> will 00:17:05.180 --> 00:17:05.190 align:start position:0% spark and almost everything I will 00:17:05.190 --> 00:17:09.530 align:start position:0% spark and almost everything I will simply<00:17:05.550> play<00:17:05.910> app<00:17:06.270> and<00:17:06.870> akka<00:17:07.440> so<00:17:08.100> ah<00:17:08.550> sure 00:17:09.530 --> 00:17:09.540 align:start position:0% simply play app and akka so ah sure 00:17:09.540 --> 00:17:13.280 align:start position:0% simply play app and akka so ah sure that's<00:17:10.290> correct<00:17:10.589> so<00:17:11.400> and<00:17:12.390> we<00:17:12.570> have<00:17:12.720> this<00:17:12.839> big 00:17:13.280 --> 00:17:13.290 align:start position:0% that's correct so and we have this big 00:17:13.290 --> 00:17:15.350 align:start position:0% that's correct so and we have this big dance<00:17:13.440> call<00:17:13.650> conference<00:17:14.040> nagas<00:17:14.400> where<00:17:14.700> we're 00:17:15.350 --> 00:17:15.360 align:start position:0% dance call conference nagas where we're 00:17:15.360 --> 00:17:16.760 align:start position:0% dance call conference nagas where we're emphasizing<00:17:15.540> end-to-end<00:17:16.290> data<00:17:16.470> pipeline 00:17:16.760 --> 00:17:16.770 align:start position:0% emphasizing end-to-end data pipeline 00:17:16.770 --> 00:17:19.520 align:start position:0% emphasizing end-to-end data pipeline scholar<00:17:17.339> right<00:17:17.550> so<00:17:17.700> basically<00:17:17.820> we<00:17:18.750> say<00:17:19.170> noisy 00:17:19.520 --> 00:17:19.530 align:start position:0% scholar right so basically we say noisy 00:17:19.530 --> 00:17:21.620 align:start position:0% scholar right so basically we say noisy yeah<00:17:19.650> we<00:17:19.800> don't<00:17:19.920> want<00:17:20.160> export<00:17:20.580> and<00:17:20.760> report<00:17:21.360> and 00:17:21.620 --> 00:17:21.630 align:start position:0% yeah we don't want export and report and 00:17:21.630 --> 00:17:23.210 align:start position:0% yeah we don't want export and report and low<00:17:21.750> the<00:17:21.810> rule<00:17:22.140> to<00:17:22.290> keep<00:17:22.470> it<00:17:22.560> everything<00:17:23.010> in 00:17:23.210 --> 00:17:23.220 align:start position:0% low the rule to keep it everything in 00:17:23.220 --> 00:17:25.970 align:start position:0% low the rule to keep it everything in very<00:17:24.089> type<00:17:24.330> of<00:17:24.360> manner<00:17:24.660> and<00:17:25.080> move<00:17:25.320> around<00:17:25.440> so<00:17:25.950> I 00:17:25.970 --> 00:17:25.980 align:start position:0% very type of manner and move around so I 00:17:25.980 --> 00:17:27.740 align:start position:0% very type of manner and move around so I wonder<00:17:26.250> if<00:17:26.640> you<00:17:26.910> see<00:17:27.089> this<00:17:27.270> is<00:17:27.480> a<00:17:27.510> feasible 00:17:27.740 --> 00:17:27.750 align:start position:0% wonder if you see this is a feasible 00:17:27.750 --> 00:17:31.610 align:start position:0% wonder if you see this is a feasible vision<00:17:29.210> in<00:17:30.210> the<00:17:30.270> industry<00:17:30.930> which<00:17:31.440> is 00:17:31.610 --> 00:17:31.620 align:start position:0% vision in the industry which is 00:17:31.620 --> 00:17:34.100 align:start position:0% vision in the industry which is experiences<00:17:32.400> with<00:17:32.580> Scala<00:17:32.910> you<00:17:33.630> know<00:17:33.780> can<00:17:33.990> we 00:17:34.100 --> 00:17:34.110 align:start position:0% experiences with Scala you know can we 00:17:34.110 --> 00:17:35.840 align:start position:0% experiences with Scala you know can we move<00:17:34.260> enough<00:17:34.530> data<00:17:34.710> science<00:17:35.100> from<00:17:35.250> Python<00:17:35.700> to 00:17:35.840 --> 00:17:35.850 align:start position:0% move enough data science from Python to 00:17:35.850 --> 00:17:39.200 align:start position:0% move enough data science from Python to scala<00:17:36.210> what<00:17:36.900> would<00:17:37.050> it<00:17:37.170> take<00:17:37.790> where<00:17:38.790> should<00:17:39.000> we 00:17:39.200 --> 00:17:39.210 align:start position:0% scala what would it take where should we 00:17:39.210 --> 00:17:41.330 align:start position:0% scala what would it take where should we you<00:17:39.600> know<00:17:39.720> ask<00:17:40.140> our<00:17:40.290> community<00:17:40.830> focus<00:17:41.220> on 00:17:41.330 --> 00:17:41.340 align:start position:0% you know ask our community focus on 00:17:41.340 --> 00:17:43.700 align:start position:0% you know ask our community focus on making<00:17:41.520> data<00:17:41.790> science<00:17:42.330> in<00:17:42.750> Scala<00:17:43.170> when<00:17:43.410> I<00:17:43.440> kind 00:17:43.700 --> 00:17:43.710 align:start position:0% making data science in Scala when I kind 00:17:43.710 --> 00:17:46.490 align:start position:0% making data science in Scala when I kind of<00:17:43.830> the<00:17:43.980> cubicle<00:17:44.550> um<00:17:45.240> yeah<00:17:45.570> great<00:17:45.870> question<00:17:46.020> so 00:17:46.490 --> 00:17:46.500 align:start position:0% of the cubicle um yeah great question so 00:17:46.500 --> 00:17:49.730 align:start position:0% of the cubicle um yeah great question so at<00:17:46.950> first<00:17:47.010> used<00:17:47.760> a<00:17:48.060> la<00:17:48.270> maybe<00:17:48.690> a<00:17:48.870> year<00:17:49.320> year<00:17:49.530> and 00:17:49.730 --> 00:17:49.740 align:start position:0% at first used a la maybe a year year and 00:17:49.740 --> 00:17:50.990 align:start position:0% at first used a la maybe a year year and a<00:17:49.830> half<00:17:49.950> ago<00:17:49.980> when<00:17:50.310> i<00:17:50.400> started<00:17:50.460> working<00:17:50.850> on 00:17:50.990 --> 00:17:51.000 align:start position:0% a half ago when i started working on 00:17:51.000 --> 00:17:53.260 align:start position:0% a half ago when i started working on this<00:17:51.090> part<00:17:51.330> i<00:17:51.510> hadn't<00:17:52.290> touched<00:17:52.380> it<00:17:52.590> before<00:17:52.680> um 00:17:53.260 --> 00:17:53.270 align:start position:0% this part i hadn't touched it before um 00:17:53.270 --> 00:17:55.910 align:start position:0% this part i hadn't touched it before um since<00:17:54.270> then<00:17:54.450> I'm<00:17:54.630> a<00:17:54.780> total<00:17:55.110> convert<00:17:55.440> think 00:17:55.910 --> 00:17:55.920 align:start position:0% since then I'm a total convert think 00:17:55.920 --> 00:17:57.980 align:start position:0% since then I'm a total convert think it's<00:17:56.010> you<00:17:56.220> know<00:17:56.340> the<00:17:56.640> bee's<00:17:56.790> knees<00:17:56.870> best<00:17:57.870> thing 00:17:57.980 --> 00:17:57.990 align:start position:0% it's you know the bee's knees best thing 00:17:57.990 --> 00:18:00.110 align:start position:0% it's you know the bee's knees best thing ever<00:17:58.140> I<00:17:58.610> you<00:17:59.610> know<00:17:59.670> I<00:17:59.730> would<00:17:59.940> love<00:18:00.060> for 00:18:00.110 --> 00:18:00.120 align:start position:0% ever I you know I would love for 00:18:00.120 --> 00:18:01.600 align:start position:0% ever I you know I would love for everyone<00:18:00.450> to<00:18:00.660> use<00:18:00.690> skull<00:18:00.990> for<00:18:01.230> everything 00:18:01.600 --> 00:18:01.610 align:start position:0% everyone to use skull for everything 00:18:01.610 --> 00:18:03.710 align:start position:0% everyone to use skull for everything whether<00:18:02.610> that's<00:18:02.850> a<00:18:02.910> realistic<00:18:03.150> goal<00:18:03.360> of 00:18:03.710 --> 00:18:03.720 align:start position:0% whether that's a realistic goal of 00:18:03.720 --> 00:18:05.360 align:start position:0% whether that's a realistic goal of course<00:18:03.900> is<00:18:04.020> a<00:18:04.200> whole<00:18:04.380> a<00:18:04.770> whole<00:18:04.830> other<00:18:05.160> question 00:18:05.360 --> 00:18:05.370 align:start position:0% course is a whole a whole other question 00:18:05.370 --> 00:18:09.020 align:start position:0% course is a whole a whole other question um<00:18:05.790> so<00:18:06.690> I've<00:18:07.320> started<00:18:07.620> developing<00:18:08.220> on<00:18:08.700> top<00:18:08.880> of 00:18:09.020 --> 00:18:09.030 align:start position:0% um so I've started developing on top of 00:18:09.030 --> 00:18:10.000 align:start position:0% um so I've started developing on top of spark<00:18:09.270> this<00:18:09.390> time<00:18:09.600> series 00:18:10.000 --> 00:18:10.010 align:start position:0% spark this time series 00:18:10.010 --> 00:18:12.450 align:start position:0% spark this time series library<00:18:10.310> a<00:18:11.020> lot<00:18:12.020> of<00:18:12.230> statistics 00:18:12.450 --> 00:18:12.460 align:start position:0% library a lot of statistics 00:18:12.460 --> 00:18:14.350 align:start position:0% library a lot of statistics functionality<00:18:13.460> and<00:18:13.550> stuff<00:18:13.820> to<00:18:14.000> make<00:18:14.120> it<00:18:14.240> easy 00:18:14.350 --> 00:18:14.360 align:start position:0% functionality and stuff to make it easy 00:18:14.360 --> 00:18:17.050 align:start position:0% functionality and stuff to make it easy to<00:18:14.480> do<00:18:14.630> concerts<00:18:15.020> analysis<00:18:15.590> on<00:18:16.220> top<00:18:16.670> of<00:18:16.820> spark 00:18:17.050 --> 00:18:17.060 align:start position:0% to do concerts analysis on top of spark 00:18:17.060 --> 00:18:19.270 align:start position:0% to do concerts analysis on top of spark and<00:18:17.270> so<00:18:17.510> that<00:18:18.200> I<00:18:18.230> start<00:18:18.650> building<00:18:18.920> in<00:18:19.040> Scala 00:18:19.270 --> 00:18:19.280 align:start position:0% and so that I start building in Scala 00:18:19.280 --> 00:18:21.160 align:start position:0% and so that I start building in Scala and<00:18:19.550> you<00:18:20.300> know<00:18:20.390> the<00:18:20.510> hope<00:18:20.660> there<00:18:20.870> is<00:18:21.020> that<00:18:21.110> we 00:18:21.160 --> 00:18:21.170 align:start position:0% and you know the hope there is that we 00:18:21.170 --> 00:18:22.870 align:start position:0% and you know the hope there is that we can<00:18:21.290> add<00:18:21.470> a<00:18:21.500> lot<00:18:21.680> of<00:18:21.830> functionality<00:18:22.190> the 00:18:22.870 --> 00:18:22.880 align:start position:0% can add a lot of functionality the 00:18:22.880 --> 00:18:24.880 align:start position:0% can add a lot of functionality the currently<00:18:23.240> exists<00:18:23.720> and<00:18:23.840> let's<00:18:23.960> say<00:18:24.080> MATLAB<00:18:24.530> or 00:18:24.880 --> 00:18:24.890 align:start position:0% currently exists and let's say MATLAB or 00:18:24.890 --> 00:18:27.660 align:start position:0% currently exists and let's say MATLAB or pine<00:18:25.160> one<00:18:25.370> package<00:18:25.790> it<00:18:25.880> looks<00:18:26.060> that's<00:18:26.270> models 00:18:27.660 --> 00:18:27.670 align:start position:0% pine one package it looks that's models 00:18:27.670 --> 00:18:30.910 align:start position:0% pine one package it looks that's models on<00:18:28.670> top<00:18:28.850> of<00:18:28.970> Scala<00:18:29.210> so<00:18:29.780> yeah<00:18:30.170> I<00:18:30.200> would<00:18:30.560> say<00:18:30.890> that 00:18:30.910 --> 00:18:30.920 align:start position:0% on top of Scala so yeah I would say that 00:18:30.920 --> 00:18:34.510 align:start position:0% on top of Scala so yeah I would say that I'm<00:18:31.280> a<00:18:31.510> low-grade<00:18:32.510> Scala<00:18:32.900> evangelist<00:18:33.520> so 00:18:34.510 --> 00:18:34.520 align:start position:0% I'm a low-grade Scala evangelist so 00:18:34.520 --> 00:18:37.120 align:start position:0% I'm a low-grade Scala evangelist so obviously<00:18:35.090> Python<00:18:35.570> an<00:18:36.320> R<00:18:36.530> have<00:18:36.800> a<00:18:36.830> huge 00:18:37.120 --> 00:18:37.130 align:start position:0% obviously Python an R have a huge 00:18:37.130 --> 00:18:38.830 align:start position:0% obviously Python an R have a huge amounts<00:18:37.610> of<00:18:37.670> momentum<00:18:38.030> for<00:18:38.180> data<00:18:38.300> science<00:18:38.510> I 00:18:38.830 --> 00:18:38.840 align:start position:0% amounts of momentum for data science I 00:18:38.840 --> 00:18:41.830 align:start position:0% amounts of momentum for data science I mean<00:18:39.530> Java<00:18:40.100> has<00:18:40.550> a<00:18:41.030> huge<00:18:41.270> amount<00:18:41.480> of<00:18:41.600> momentum 00:18:41.830 --> 00:18:41.840 align:start position:0% mean Java has a huge amount of momentum 00:18:41.840 --> 00:18:44.010 align:start position:0% mean Java has a huge amount of momentum just<00:18:42.290> for<00:18:42.410> Kate<00:18:42.620> engineering<00:18:43.070> stuff<00:18:43.280> yes<00:18:43.610> um 00:18:44.010 --> 00:18:44.020 align:start position:0% just for Kate engineering stuff yes um 00:18:44.020 --> 00:18:47.170 align:start position:0% just for Kate engineering stuff yes um so<00:18:45.020> we'll<00:18:45.530> see<00:18:45.710> how<00:18:46.010> things<00:18:46.490> pan<00:18:46.730> out<00:18:46.790> in<00:18:47.090> the 00:18:47.170 --> 00:18:47.180 align:start position:0% so we'll see how things pan out in the 00:18:47.180 --> 00:18:50.680 align:start position:0% so we'll see how things pan out in the next<00:18:47.650> five<00:18:48.650> to<00:18:48.770> ten<00:18:48.920> years<00:18:48.950> all<00:18:49.790> right<00:18:49.970> well 00:18:50.680 --> 00:18:50.690 align:start position:0% next five to ten years all right well 00:18:50.690 --> 00:18:51.730 align:start position:0% next five to ten years all right well thank<00:18:50.990> you<00:18:51.050> much<00:18:51.230> we're<00:18:51.410> looking<00:18:51.590> forward 00:18:51.730 --> 00:18:51.740 align:start position:0% thank you much we're looking forward 00:18:51.740 --> 00:18:53.980 align:start position:0% thank you much we're looking forward you're<00:18:51.890> talkin<00:18:52.310> the<00:18:52.550> is<00:18:52.790> great<00:18:53.480> to<00:18:53.600> go<00:18:53.720> again 00:18:53.980 --> 00:18:53.990 align:start position:0% you're talkin the is great to go again 00:18:53.990 --> 00:18:57.130 align:start position:0% you're talkin the is great to go again thanks<00:18:54.560> very<00:18:54.710> much<00:18:54.740> less<00:18:55.010> a