Zac Farrell, Hotdata — Interview with Alexy

All right. Hello everybody. I'm Alexa All right. Hello everybody. I'm Alexa Kroboth. Kroboth. Uh and here we are on location at Uh and here we are on location at LinkedIn office at the LinkedIn office at the San Francisco Apache DataFusion meetup San Francisco Apache DataFusion meetup and we have a lot of awesome speakers and we have a lot of awesome speakers including Zach Farrell who is the including Zach Farrell who is the co-founder and co-founder and >> principal engineer >> principal engineer >> principal engineer at Hot Data. Uh >> principal engineer at Hot Data. Uh welcome Zach. Tell us a little bit what welcome Zach. Tell us a little bit what you do and like let's hope this thing is you do and like let's hope this thing is striking you better than it's doing striking you better than it's doing right now like this. Okay. right now like this. Okay. >> Hey. I'm Zach Farrell and uh Hot Data >> Hey. I'm Zach Farrell and uh Hot Data we're on a mission to give everyone and we're on a mission to give everyone and everything a database. everything a database. Small ephemeral Small ephemeral big large and everything in between. big large and everything in between. >> So >> So uh uh like why does everybody need a database like why does everybody need a database and why Hot Data is positioned to give and why Hot Data is positioned to give them one if they want to? them one if they want to? >> Um so >> Um so with the with the how prolific agents have become, how prolific agents have become, everything is becoming outsourced to everything is becoming outsourced to that. There's more and more agents that. There's more and more agents running, pipelines running all the time running, pipelines running all the time and we and we see the need for more uh more databases, see the need for more uh more databases, more more uh isolation uh isolation um um and really giving the space for an agent and really giving the space for an agent to run wild. So to run wild. So Hot Data is we're obsessed with Hot Data is we're obsessed with performance, making things fast, um performance, making things fast, um reducing latency reducing latency and uh we're going to show tonight some and uh we're going to show tonight some some work we've been doing to make it some work we've been doing to make it really fast and quick to create really fast and quick to create databases. databases. >> And you told us before we started like >> And you told us before we started like really awesome story how you guys really awesome story how you guys discovered Rust and you apparently discovered Rust and you apparently discovered it from necessity from discovered it from necessity from databases from DataFusion. Can you databases from DataFusion. Can you briefly kind of recount how that briefly kind of recount how that happened? happened? >> Yeah, we were when we started Hot Data >> Yeah, we were when we started Hot Data we were kind of reevaluating what what we were kind of reevaluating what what tools and frameworks were out there. tools and frameworks were out there. Um we have a a deep history of using Um we have a a deep history of using DuckDB in the past which we love love DuckDB in the past which we love love the the DuckDB folks. the the DuckDB folks. Um but we kind of stumbled across Um but we kind of stumbled across DataFusion and found that it was like DataFusion and found that it was like the perfect building blocks for us to the perfect building blocks for us to build our product around. Um and really build our product around. Um and really just was able to run and quickly build just was able to run and quickly build around that using the Datafusion around that using the Datafusion primitives. primitives. >> So, my gimbal is doing like some >> So, my gimbal is doing like some gymnastics here. Look at the cursor gymnastics here. Look at the cursor sitting there. sitting there. Obedience again. Uh so, it's Obedience again. Uh so, it's interesting, right? So, when you look at interesting, right? So, when you look at Java, obviously Java is old. Java, obviously Java is old. Uh we don't want it because Python and Uh we don't want it because Python and native and half. native and half. Uh but DuckDB is in C++ and you Uh but DuckDB is in C++ and you basically say like Rust is easier to basically say like Rust is easier to handle than C++. handle than C++. >> Yeah. >> Yeah. >> Awesome. And so, are these databases >> Awesome. And so, are these databases cloud hosted? Do you give people cloud cloud hosted? Do you give people cloud hosted databases? Like where do they get hosted databases? Like where do they get them basically? If I want to a database them basically? If I want to a database from Hot Data, from Hot Data, where do I go to get one? where do I go to get one? >> Yeah, so Hot Data is a cloud hosted >> Yeah, so Hot Data is a cloud hosted service. And so, with your account, you service. And so, with your account, you provision as many databases as you need. provision as many databases as you need. Um effectively, you have an isolated Um effectively, you have an isolated workspace, which we isolated the compute workspace, which we isolated the compute um as well as storage layers. um as well as storage layers. And then from there, you can um create And then from there, you can um create infinite databases. infinite databases. >> Are they their own databases? Your own >> Are they their own databases? Your own databases or are they custom like uh databases or are they custom like uh like can you put DuckDB there or like can you put DuckDB there or whatever? Is it your own database? whatever? Is it your own database? >> It's it's our own database built on top >> It's it's our own database built on top of Datafusion. of Datafusion. >> Okay. And is it SQL? >> Okay. And is it SQL? >> Mhm. >> Mhm. >> Okay. So, SQL will never die. >> Okay. So, SQL will never die. >> Never. >> Never. >> always be there. >> always be there. Well, so this is a really competitive Well, so this is a really competitive space. So, are you going in against Ursa space. So, are you going in against Ursa and DuckDB? Are they competitors? and DuckDB? Are they competitors? >> I Yeah, I think all of them are doing >> I Yeah, I think all of them are doing interesting things in slightly different interesting things in slightly different ways. Um ways. Um and so, yeah, definitely look to both of and so, yeah, definitely look to both of them as an inspiration. them as an inspiration. >> Okay. >> Okay. So, this is very exciting. Like So, this is very exciting. Like everybody loves databases. Everybody everybody loves databases. Everybody wants more of them. wants more of them. >> Definitely. >> Definitely. >> Especially done in Rust. So, looking >> Especially done in Rust. So, looking forward to your main talk and thanks for forward to your main talk and thanks for coming and sharing with us. coming and sharing with us. >> Thank you so much. >> Thank you so much. >> Thank you, Zach.