Bay.Area.AI: Interview with Michael Ryan, Stanford

hello everybody my name is Alexi kov I'm hello everybody my name is Alexi kov I'm the founder and organizer of B AI which the founder and organizer of B AI which is the most established longest running is the most established longest running deep technical metop on AI in the Bay deep technical metop on AI in the Bay Area and the world run for 10 years Area and the world run for 10 years we've been NLP KD machine learning now we've been NLP KD machine learning now ai and we do it from location GitHub ai and we do it from location GitHub headquarters we have like top uh AI headquarters we have like top uh AI developers coming to learn about developers coming to learn about interesting projects and I call it DP interesting projects and I call it DP you can tell me what you call it is been you can tell me what you call it is been top of mind uh I uh top of mind uh I uh found found discovered Tomar kab who was found found discovered Tomar kab who was who created thisp student of M Zakaria who created thisp student of M Zakaria we we launched the very first Spark me we we launched the very first Spark me up when I found spark in 2012 we up when I found spark in 2012 we launched mate at you know at SCA as launched mate at you know at SCA as spark so then I've seen Omar and he spark so then I've seen Omar and he spoke at our conference Kill by the way spoke at our conference Kill by the way and then now Omar told me he wants more and then now Omar told me he wants more folks working for thisp to talk about it folks working for thisp to talk about it and here come Michael as one of the top and here come Michael as one of the top developers and Michael is a master developers and Michael is a master student in Stanford that's correct so student in Stanford that's correct so Michael tell us how did you get to do Michael tell us how did you get to do disp and like what's what made you you disp and like what's what made you you know like what's fun about it and how know like what's fun about it and how come like you made one of the key come like you made one of the key components with yeah for sure so I got components with yeah for sure so I got involved with so I call it DSP I think involved with so I call it DSP I think any pronunciation kind kind of worked any pronunciation kind kind of worked but um I got involved with DSP when I but um I got involved with DSP when I got started at Stanford um so about a got started at Stanford um so about a year ago I was taking a class on meta year ago I was taking a class on meta learning and one of the projects that we learning and one of the projects that we decided we wanted to work on was like decided we wanted to work on was like prompt optimization and I had just seen prompt optimization and I had just seen a talk from Omar at the NLP group about a talk from Omar at the NLP group about dspi and how he was working on dspi and how he was working on especially F shot bootstrapping especially F shot bootstrapping optimization for language model programs optimization for language model programs in dsy so I was working with somebody in dsy so I was working with somebody named Christa on this project uh we were named Christa on this project uh we were just both students in this class and I just both students in this class and I reached out to Omar and said like we reached out to Omar and said like we want to work on prompt optimization and want to work on prompt optimization and you said this is maybe like a next step you said this is maybe like a next step for DSP so maybe we can collaborate on for DSP so maybe we can collaborate on something here and immediately we got to something here and immediately we got to work on brainstorming and that's how we work on brainstorming and that's how we started making something called the started making something called the co-pro optimizer uh which has now been co-pro optimizer uh which has now been kind of deprecated in favor of the MEO kind of deprecated in favor of the MEO Optimizer and that's what the talk will Optimizer and that's what the talk will be on today is our Meo Optimizer which be on today is our Meo Optimizer which is like the state-of-the-art optimizer is like the state-of-the-art optimizer for language model programs in in DSP for language model programs in in DSP and so some of the stuff I'm really and so some of the stuff I'm really excited about with DSP I think it's such excited about with DSP I think it's such an amazing framework for just an amazing framework for just considering what you should uh sort of considering what you should uh sort of conceptualize for a language model conceptualize for a language model program so the idea that you can break program so the idea that you can break it into your your uh structure of your it into your your uh structure of your program into inputs um a metric and the program into inputs um a metric and the program itself and then optimize the program itself and then optimize the entire system end to end I think having entire system end to end I think having that abstraction removes all of the that abstraction removes all of the confusion about what should I optimize confusion about what should I optimize is it the weights is it the prompts um is it the weights is it the prompts um and what uh different data do I need to and what uh different data do I need to collect and how do I actually optimize collect and how do I actually optimize this do I need a gradient signal DSP this do I need a gradient signal DSP actually can kind of take all of that actually can kind of take all of that confusion and allow you to just write a confusion and allow you to just write a program that solves the task that you program that solves the task that you want to solve and optimize it for you in want to solve and optimize it for you in the best way possible so you have our the best way possible so you have our prompt optimizers you have weight prompt optimizers you have weight optimizers and we're working on like RL optimizers and we're working on like RL based optimizers too so I think that based optimizers too so I think that just the abstraction itself is really just the abstraction itself is really cool and it's empowering people to do cool and it's empowering people to do really amazing things with language really amazing things with language models that you can't do with just a models that you can't do with just a single language model like you have to single language model like you have to build a system a compound system awesome build a system a compound system awesome you know it's interesting right like I you know it's interesting right like I was spending this mic and Industry world was spending this mic and Industry world like I did a few moves back and forth like I did a few moves back and forth you know did my PhD in computer science you know did my PhD in computer science at un of Pennsylvania kind of you know at un of Pennsylvania kind of you know went back and forth to Industry and like went back and forth to Industry and like finally went back you know to Silicon finally went back you know to Silicon Valley so like I kind of understand how Valley so like I kind of understand how it works right academics live in their it works right academics live in their own world and they really like isolate own world and they really like isolate themselves with very smart people and themselves with very smart people and they like Ste in their own kind of they like Ste in their own kind of environment and so you're kind of doing environment and so you're kind of doing this but you bring this open source to this but you bring this open source to the world right the people start doing the world right the people start doing stuff with it and they not at Stanford stuff with it and they not at Stanford right and like some of them don't know right and like some of them don't know basics of this so I wonder when you like basics of this so I wonder when you like do you uh interact with the usual do you uh interact with the usual developers or the regular folk and like developers or the regular folk and like does it inform how you build uh gpy yeah does it inform how you build uh gpy yeah definitely so one thing I'll say that's definitely so one thing I'll say that's amazing about dspi is the community amazing about dspi is the community that's formed around it and I have to that's formed around it and I have to attribute that to Omar uh he does an attribute that to Omar uh he does an amazing job building the community uh amazing job building the community uh actively involved in the Discord actively involved in the Discord obviously on X he's very um well obviously on X he's very um well presented late interaction yes yes at presented late interaction yes yes at late interaction um so I think I get a late interaction um so I think I get a lot of experience interacting with lot of experience interacting with people outside of Academia because of people outside of Academia because of that and actually I've been very that and actually I've been very fortunate this summer I'm interning at fortunate this summer I'm interning at snowflake so I'm working with Folks at snowflake so I'm working with Folks at snowflake who are also using dspi so I'm snowflake who are also using dspi so I'm getting that perspective um a bunch of getting that perspective um a bunch of startups reach out and uh ask U we're startups reach out and uh ask U we're implementing this in dsy how can we do implementing this in dsy how can we do that and I think it actually gives me a that and I think it actually gives me a great perspective to uh stepping out of great perspective to uh stepping out of Academia a bit and seeing how people are Academia a bit and seeing how people are actually building language model systems actually building language model systems in industry and how how can we optimize in industry and how how can we optimize those and one thing that we're working those and one thing that we're working on right now is actually a sort of on right now is actually a sort of Benchmark of language model programs and Benchmark of language model programs and how people are actually using them in how people are actually using them in Industry because we want to show that Industry because we want to show that our optimizers work in real world our optimizers work in real world settings so I'd love for more people to settings so I'd love for more people to reach out if you're using dspi let me reach out if you're using dspi let me know how you're using it because we know how you're using it because we might add your task to our Benchmark and might add your task to our Benchmark and show like which Optimizer which language show like which Optimizer which language model is best for this particular task model is best for this particular task fantastic it makes me very happy because fantastic it makes me very happy because you know first of all no bunch of folks you know first of all no bunch of folks at snowflake a lot of my friends went to at snowflake a lot of my friends went to work there right it's a big company like work there right it's a big company like a lot of developers in all areas so now a lot of developers in all areas so now you know uh I'm a j which is the you know uh I'm a j which is the category defining graph database and I category defining graph database and I don't know anything yet about apply don't know anything yet about apply disput to graph databases but I wonder disput to graph databases but I wonder if you think that you know we can add if you think that you know we can add that to your stable of tasks right that to your stable of tasks right because you see relationally to data because you see relationally to data Lakes uh do you think uh the graph Lakes uh do you think uh the graph databases make a good use case for disp databases make a good use case for disp I think that would be a great use case I think that would be a great use case actually I think um dpy started off with actually I think um dpy started off with um you know Omar has this background in um you know Omar has this background in retrieval augmented generation and retrieval augmented generation and retrieval as a whole um and so a lot of retrieval as a whole um and so a lot of the initial implementations in dspi were the initial implementations in dspi were for optimizing rag systems end to end um for optimizing rag systems end to end um and I think plugging in a a graph and I think plugging in a a graph database instead of a relational database instead of a relational database or instead of a CO bear index database or instead of a CO bear index or you know I think that is a really or you know I think that is a really cool way to see um potentially now we cool way to see um potentially now we typically optimize the language model typically optimize the language model part parts of the the rag pipeline so part parts of the the rag pipeline so like rewriting your query rewriting the like rewriting your query rewriting the generation step I think plugging that in generation step I think plugging that in with uh with graphs uh graph databases with uh with graphs uh graph databases is like a really interesting use case is like a really interesting use case yeah so the interesting thing I'm going yeah so the interesting thing I'm going to you know show a little bit in in my to you know show a little bit in in my talk on graph rag today that we you know talk on graph rag today that we you know in addition to to textual prompt in addition to to textual prompt obviously we have we build a knowledge obviously we have we build a knowledge graph on right on on the fly or like graph on right on on the fly or like from document collection and so we can from document collection and so we can actually add uh Cipher query so Cipher actually add uh Cipher query so Cipher is a graph query language which new for is a graph query language which new for uses so very similar to SQL right so can uses so very similar to SQL right so can you optimize SQL query in addition to you optimize SQL query in addition to the texal prom yes yeah we can optimize the texal prom yes yeah we can optimize that fantastic so then then we probably that fantastic so then then we probably can figure out how to plug in CER of EX can figure out how to plug in CER of EX which is now also a standard called gql which is now also a standard called gql right almost replace one letter you have right almost replace one letter you have a different query language fantastic so a different query language fantastic so and uh one more question about D and and uh one more question about D and then I just ask you a fun question uh so then I just ask you a fun question uh so uh like we really have a lot of uh like we really have a lot of developers here and so we really need to developers here and so we really need to figure out figure out right how do we kind of help them learn right how do we kind of help them learn all of this like this overwhelming mod all of this like this overwhelming mod information Academia is a machine information Academia is a machine optimized for knowledge consumption you optimized for knowledge consumption you drink from the high fire hose you read drink from the high fire hose you read papers you have people who do this papers you have people who do this together like this you have special together like this you have special skills superpowers you know how to kind skills superpowers you know how to kind of very quickly triage information right of very quickly triage information right a lot of developers do not do this it a lot of developers do not do this it may be difficult to them like what may be difficult to them like what advice do you have for developers who advice do you have for developers who want to come into DP do they need to want to come into DP do they need to read papers can they just take code and read papers can they just take code and play with this like and they take your play with this like and they take your kind of examples what's a good entry kind of examples what's a good entry point how they can start learning and point how they can start learning and incrementally learn more and feel happy incrementally learn more and feel happy yeah that's a really great question and yeah that's a really great question and we're working on even more documentation we're working on even more documentation for dspi I think the best way to get for dspi I think the best way to get started right now is go to the started right now is go to the repository and we have an examples repository and we have an examples folder um where we have uh collab folder um where we have uh collab notebooks about here's how you use the notebooks about here's how you use the MEO Optimizer here's how you use MEO Optimizer here's how you use assertions um and you know here's how assertions um and you know here's how you do fine-tuning in DSP and just go you do fine-tuning in DSP and just go through those notebooks they're already through those notebooks they're already set up to work out of the box in Google set up to work out of the box in Google uh so that's a great way and I think uh so that's a great way and I think watching talks like the one I'm going to watching talks like the one I'm going to give today will give the insights into give today will give the insights into how Meo Works behind the scenes and what how Meo Works behind the scenes and what dsy is actually doing to optimize your dsy is actually doing to optimize your language model program so I think just language model program so I think just watching this video doing the collabs watching this video doing the collabs that's the best way to get started that's the best way to get started fantastic well and last question tell us fantastic well and last question tell us some fun fact about yourself what do you some fun fact about yourself what do you like to do for fun yeah yeah so a fun like to do for fun yeah yeah so a fun fact about me is I have I guess I would fact about me is I have I guess I would say three different licenses so I have say three different licenses so I have my driver's license that's that's my driver's license that's that's obvious um but I also have a boating obvious um but I also have a boating license in Florida because I wanted to license in Florida because I wanted to drive a jet ski so I took the course uh drive a jet ski so I took the course uh at the age I wanted to I had to take the at the age I wanted to I had to take the course and I also have my Scuba diving course and I also have my Scuba diving license which is something I like to do license which is something I like to do for fun um although the water's very for fun um although the water's very cold out here in the bay so I don't get cold out here in the bay so I don't get to do it too often well fantastic so to do it too often well fantastic so you're like a very licensed guy and so you're like a very licensed guy and so you know you're doing your master so you know you're doing your master so you're going in the industry so we wish you're going in the industry so we wish you you know uh luck and uh really uh you you know uh luck and uh really uh looking forward to learn from you looking forward to learn from you looking forward to your J thank you so looking forward to your J thank you so much thank you Michael cheers