DevReal: Charles Frye Interview

hello everybody I'm Alexi ker the hello everybody I'm Alexi ker the founder and organizer of Bay Area AI founder and organizer of Bay Area AI which is the longest running deepest which is the longest running deepest technical biggest a up in the world technical biggest a up in the world running in Bay Area since running in Bay Area since 2015 and tonight we're on location at WS 2015 and tonight we're on location at WS J Loft and we have an amazing event full J Loft and we have an amazing event full stack open source AI from chips to apps stack open source AI from chips to apps and with us we have Charles fry who is and with us we have Charles fry who is uh the guy model yep and I must say that uh the guy model yep and I must say that full step comes in no small part from full step comes in no small part from Full step D learning which Charles Full step D learning which Charles founded and ran successfully also uh we founded and ran successfully also uh we did that as a part of uh scale by the did that as a part of uh scale by the bay conference which run here yearly and bay conference which run here yearly and uh I must just jump in because you spoke uh I must just jump in because you spoke at um uh scale by last year and you came at um uh scale by last year and you came up with this notion of lmos yeah which up with this notion of lmos yeah which seemed extremely prophetic then and like seemed extremely prophetic then and like people are coming around with this idea people are coming around with this idea now so can you tell us you know how do now so can you tell us you know how do you see lmos evolve and yeah um yeah I you see lmos evolve and yeah um yeah I think yeah the original idea I think the think yeah the original idea I think the terminology is maybe Andre Kathy's terminology is maybe Andre Kathy's originally but the like core idea is originally but the like core idea is that the operating system is like uh that the operating system is like uh wraps the hardware and makes it more wraps the hardware and makes it more approachable for humans approachable for humans uh and like like Hardware is like uh and like like Hardware is like completely asynchronous and you know completely asynchronous and you know like programming drivers is so much more like programming drivers is so much more difficult even than like low-level difficult even than like low-level operating system other like lowlevel operating system other like lowlevel systems programming and so the operating systems programming and so the operating system just like takes all that and system just like takes all that and makes it like makes it approachable to makes it like makes it approachable to humans and and encapsulates it and I see humans and and encapsulates it and I see like there's like a similar opportunity like there's like a similar opportunity with like more intelligent machines with like more intelligent machines machines that have more semantic machines that have more semantic understanding to put like another rapper understanding to put like another rapper around uh now now around the like around uh now now around the like machine language that we invented that machine language that we invented that was like our original bridge between was like our original bridge between humans and machines um and I think like humans and machines um and I think like there's a couple of like cluster related there's a couple of like cluster related ideas like natural user interfaces which ideas like natural user interfaces which has been around for a long time but got has been around for a long time but got picked up again by people like Sam picked up again by people like Sam Whitmore and Jason Yen um and I think Whitmore and Jason Yen um and I think like to to an extent part of that the like to to an extent part of that the excitement around that was with this big excitement around that was with this big release of super capable Foundation release of super capable Foundation models and like a bunch of like kind of models and like a bunch of like kind of en visioning what the future of en visioning what the future of computing looks like I think you know computing looks like I think you know that it's the devil's in the details of that it's the devil's in the details of making something like that work has making something like that work has turned out to be you know a little bit turned out to be you know a little bit harder than just like combining a couple harder than just like combining a couple of Lang chain programs together um and of Lang chain programs together um and so like while I still have that as like so like while I still have that as like my big long-term vision of where we're my big long-term vision of where we're going as a field of making more human uh going as a field of making more human uh machines it's uh it's sort of like machines it's uh it's sort of like receded into the Horizon is like here's receded into the Horizon is like here's where we're going um here's where we where we're going um here's where we hope to be in 10 years yeah interesting hope to be in 10 years yeah interesting and so you know we talked about lmos and and so you know we talked about lmos and it having memory right it having State it having memory right it having State and so Eric Meer started this Lang and so Eric Meer started this Lang universales and he talks about AI native universales and he talks about AI native programming languages where LM is the programming languages where LM is the runtime of these languages right and so runtime of these languages right and so recently I came across uh start up uh I recently I came across uh start up uh I think they autogen people and they talk think they autogen people and they talk about agent OS right so what do you about agent OS right so what do you think are going to be the elements of think are going to be the elements of this like for instance are agents the this like for instance are agents the elements of this like what do you see is elements of this like what do you see is like units like units uh uh and and obviously like where are uh uh and and obviously like where are the gpus like are they so deep in the the gpus like are they so deep in the bottom I don't even see them yeah I bottom I don't even see them yeah I think I think a lot of that is going to think I think a lot of that is going to have to recede uh I think there's kind have to recede uh I think there's kind of two parts one is that some stuff is of two parts one is that some stuff is going to be going to be super it's like less time sensitive it's super it's like less time sensitive it's maybe more beneficial to have a lot more maybe more beneficial to have a lot more int it's like adding more intelligence int it's like adding more intelligence makes it better uh like without bound makes it better uh like without bound and those kinds of workloads will move and those kinds of workloads will move will like remain in cloud data centers will like remain in cloud data centers which is where the majority of like which is where the majority of like neural network executions currently live neural network executions currently live um but despite being at a cloud provider um but despite being at a cloud provider and like uh and and really like thinking and like uh and and really like thinking about and focusing on that kind of about and focusing on that kind of deployment like there's another side deployment like there's another side that's actually probably where a lot of that's actually probably where a lot of this like humanizing uh machine this like humanizing uh machine interfaces is going to happen which will interfaces is going to happen which will just be like purely local and I think just be like purely local and I think like apple is already Dem already like like apple is already Dem already like accidentally stumbled upon the right accidentally stumbled upon the right formula like high bandwidth memory um formula like high bandwidth memory um and and like a GPU with unified memory and and like a GPU with unified memory between this like with the uh with the between this like with the uh with the main processor to get like very like main processor to get like very like buttery smooth experience Nvidia seems buttery smooth experience Nvidia seems to be going in a similar Direction with to be going in a similar Direction with digits and their like other sort of like digits and their like other sort of like Jetson platform uh and like Orin uh Jetson platform uh and like Orin uh platform stuff uh and so I would see platform stuff uh and so I would see like a lot of that actually ending up like a lot of that actually ending up ending up local like it's it's just a ending up local like it's it's just a thing that computers do and like we keep thing that computers do and like we keep getting the that the like power of those getting the that the like power of those things keeps getting greater the our things keeps getting greater the our ability to squeeze intelligence into ability to squeeze intelligence into smaller and smaller modules so neural smaller and smaller modules so neural networks with fewer parameters or with networks with fewer parameters or with faster inference uh gets better and faster inference uh gets better and that's how we get like more intelligence that's how we get like more intelligence at at the edge um or like at the at the at at the edge um or like at the at the users compute so I mean that sounds users compute so I mean that sounds great and like it already kind of gives great and like it already kind of gives us a lot of details you know from you us a lot of details you know from you know sub nuts like you talk about gpus know sub nuts like you talk about gpus and local compute right and then we go and local compute right and then we go all the way to OS so I'm really curious all the way to OS so I'm really curious about your personal uh path in this about your personal uh path in this because you know with like you you did because you know with like you you did the PD berlay and you did the Deep the PD berlay and you did the Deep learning which was a high level concept learning which was a high level concept and now basically and you talk about and now basically and you talk about this ground vision of LMS and then you this ground vision of LMS and then you end up with model which is really close end up with model which is really close to the metal yeah right and and so but to the metal yeah right and and so but then you obstruct uh the metal for the then you obstruct uh the metal for the developer so can you tell me like about developer so can you tell me like about your personal Journey like why is this a your personal Journey like why is this a sweet spot for you what makes it fun sweet spot for you what makes it fun yeah yeah there's a there's like a lot yeah yeah there's a there's like a lot of things that make it fun I think the of things that make it fun I think the you mentioned full stack deep learning a you mentioned full stack deep learning a couple of times like kind of what couple of times like kind of what happened is I was teaching that class happened is I was teaching that class that was like mlops and like deploying that was like mlops and like deploying an operationalizing machine learning um an operationalizing machine learning um after you know working on that at after you know working on that at weights and biases for a couple years uh weights and biases for a couple years uh and teaching people that class and like and teaching people that class and like I had like a pretty sad story to tell I had like a pretty sad story to tell them when it came to deployment it was them when it came to deployment it was like well with computer vision models like well with computer vision models that do classification you can squeeze that do classification you can squeeze them down small enough to fit to like them down small enough to fit to like run fast on a CPU so then just put them run fast on a CPU so then just put them in a Lambda an aw like on AWS like run in a Lambda an aw like on AWS like run them serously on a CPU like them serously on a CPU like unfortunately there are no options with unfortunately there are no options with serverless gpus and that's what that was serverless gpus and that's what that was like 2022 and I was like man this is a like 2022 and I was like man this is a startup shaped problem there has to be startup shaped problem there has to be somebody working on this and so I found somebody working on this and so I found banana replicate modal um tried out any banana replicate modal um tried out any scale lightning AI a bunch of these sort scale lightning AI a bunch of these sort of like newer like Cloud providers or or of like newer like Cloud providers or or like Cloud abstractions um with a focus like Cloud abstractions um with a focus on GPU and gpus and data intensive on GPU and gpus and data intensive workloads and modal was the one that I workloads and modal was the one that I like enjoyed using the most and that I like enjoyed using the most and that I kept coming back to um and that most kept coming back to um and that most importantly like extended my like vision importantly like extended my like vision of what was possible with computers or of what was possible with computers or sort of like extended my Powers um it sort of like extended my Powers um it was like oh wow I can turn this I don't was like oh wow I can turn this I don't just have to like run this into jupyter just have to like run this into jupyter notebook or run this locally like it's notebook or run this locally like it's actually so easy to turn this into a actually so easy to turn this into a service I can just do it right away like service I can just do it right away like oh I should be doing this all the time oh I should be doing this all the time like I should be making tiny little apps like I should be making tiny little apps to solve all kinds of problems not even to solve all kinds of problems not even just one that have big neural networks just one that have big neural networks in them but just like yeah I have like a in them but just like yeah I have like a little yoga tracking app that I made in little yoga tracking app that I made in about 10 minutes using Claude and then about 10 minutes using Claude and then deployed on motal and now I like track deployed on motal and now I like track whether I'm keeping up with my like whether I'm keeping up with my like personal yoga goals and that was like personal yoga goals and that was like you know that the like Cloud magic of you know that the like Cloud magic of like easy deployment plus the like magic like easy deployment plus the like magic of foundational models has like is what of foundational models has like is what drew me into like using modal over and drew me into like using modal over and over again telling people they should over again telling people they should use it and then eventually joining um use it and then eventually joining um and then it's been an opportunity to and then it's been an opportunity to just like really dive deep work with just like really dive deep work with some of the like you know best um you some of the like you know best um you know some of the like frankly best know some of the like frankly best engineers in the world on really hard engineers in the world on really hard like you know really hard problems and like you know really hard problems and on you know learn more about like the on you know learn more about like the nature of computers and then get to nature of computers and then get to share that with people that's exciting share that with people that's exciting and I mean you've been out there like and I mean you've been out there like really teaching people a lot having a really teaching people a lot having a great following people flew in SFO great following people flew in SFO specifically for full step dear from specifically for full step dear from around the world which was amazing to me around the world which was amazing to me right like it was a local workshop with right like it was a local workshop with mostly people from around the world so mostly people from around the world so uh so I'm curious what do you think of uh so I'm curious what do you think of the role of devels in the revolution how the role of devels in the revolution how do you see kind of us collectively do you see kind of us collectively teaching the developers at large all teaching the developers at large all these things what works what doesn't these things what works what doesn't where should the developers focus when where should the developers focus when learning about this yeah it's a lot of learning about this yeah it's a lot of lot of great questions I think like lot of great questions I think like internally from like the you know internally from like the you know developer relations as like an industry developer relations as like an industry or as a as a community of people solving or as a as a community of people solving similar problems it's like similar problems it's like there's evangelism and advocacy uh and there's evangelism and advocacy uh and in between them is like relations so in between them is like relations so like an evangelist is going out there like an evangelist is going out there and like sharing how to use the and like sharing how to use the technology with people and like teaching technology with people and like teaching people this like um BTO malsky is like a people this like um BTO malsky is like a hasal evangelist I think is his title um hasal evangelist I think is his title um and he's like out there like getting and he's like out there like getting excited about functional got me excited excited about functional got me excited about functional programming um also about functional programming um also Shar a of great Parisian food oh yeah Shar a of great Parisian food oh yeah that's uh yeah he's great Twitter um that's uh yeah he's great Twitter um great social media presence in general great social media presence in general um and then there's also the like yeah um and then there's also the like yeah on the other side is the advocate who is on the other side is the advocate who is like representing the needs of like representing the needs of developers internally and I think yeah developers internally and I think yeah Kelsey high tower is maybe the best Kelsey high tower is maybe the best example of this um it's like your example of this um it's like your primary goal is actually to like hold primary goal is actually to like hold the organization to account a bit it's the organization to account a bit it's like you are the developers like you are the developers representative inside the company that's representative inside the company that's the kind of that's like that's the title the kind of that's like that's the title that I chose like and that's the that I chose like and that's the direction I would definitely want my direction I would definitely want my career to go because I think it's like career to go because I think it's like that is what delivers the right balance that is what delivers the right balance I think for me between like value for I think for me between like value for other people as in like expressing their other people as in like expressing their needs and giving them like you know needs and giving them like you know helping them have a seat at the table um helping them have a seat at the table um and then also D driving value for the and then also D driving value for the organization which like they actually do organization which like they actually do want those opinions they do want that want those opinions they do want that information and they struggle to get it information and they struggle to get it through different means through support through different means through support or through Engineers or through like you or through Engineers or through like you know other mechanisms of like Word of know other mechanisms of like Word of Mouth um so like in this particular AI Mouth um so like in this particular AI world I don't know that there's world I don't know that there's like a specific thing I think maybe one like a specific thing I think maybe one of the most important things is just of the most important things is just like cutting through the like cutting through the honestly because there's a lot of like honestly because there's a lot of like linked influencing there's a lot of like linked influencing there's a lot of like um yeah Naro well crypto refugees um yeah Naro well crypto refugees entering into the latest hot new thing entering into the latest hot new thing and people who have like a prominent and people who have like a prominent following and who are sufficiently following and who are sufficiently technical to be developer Advocates technical to be developer Advocates evangelists or relations people uh like evangelists or relations people uh like have the ability to like you know cut have the ability to like you know cut like silence those voices or amplify like silence those voices or amplify other voices and like be TR more trusted other voices and like be TR more trusted Source more High signal um so I think Source more High signal um so I think like that's particularly important in like that's particularly important in this in this moment in this field where this in this moment in this field where there's so very much noise I totally there's so very much noise I totally agree and thank you by the way for agree and thank you by the way for helping us put this together because I helping us put this together because I think personally open source and it's think personally open source and it's like self-evident right we hold this Tru like self-evident right we hold this Tru self-evident and we show people code and self-evident and we show people code and in this case we actually put together a in this case we actually put together a stack right and we show people how to do stack right and we show people how to do this basically as a as a stacks and I this basically as a as a stacks and I think to me this like Integrations is think to me this like Integrations is the most we create network effects MH the most we create network effects MH and you guys are running a lot of this and you guys are running a lot of this stuff so thank you for that and looking stuff so thank you for that and looking forward to your talk and thank you for forward to your talk and thank you for being such a great part of the community being such a great part of the community yeah yeah thanks Lexi always a pleasure yeah yeah thanks Lexi always a pleasure thanks Charles for