Jaikumar Ganesh, Anyscale, on Reliable AI — Interview with Alexy

Hi, my name is Jaikumar Ganesh. I'm the Hi, my name is Jaikumar Ganesh. I'm the head of engineering at Any Scale. Any head of engineering at Any Scale. Any scale is the company behind Ray, the scale is the company behind Ray, the popular software for scaling AI and popular software for scaling AI and machine learning systems. I think AI has machine learning systems. I think AI has enabled so many use cases. You know enabled so many use cases. You know there is a small company called Halter there is a small company called Halter in New Zealand which in New Zealand which makes farmers a lot more efficient with makes farmers a lot more efficient with their cow and they use AI to figure out their cow and they use AI to figure out where the cow should be going um in that where the cow should be going um in that particular farm on that particular day. particular farm on that particular day. So there is biioarma companies using AI So there is biioarma companies using AI for drug discovery. Right? So there is a for drug discovery. Right? So there is a lot of such use cases in different lot of such use cases in different verticals which have impacted humanity verticals which have impacted humanity in a positive way, improved efficiency in a positive way, improved efficiency and improved productivity. So that's and improved productivity. So that's what excites me about AI. I think what excites me about AI. I think reliable AI is a really deep topic. So reliable AI is a really deep topic. So reliable AI can mean different things reliable AI can mean different things for different people, right? So if you for different people, right? So if you pull up chat GPD and it just works, that pull up chat GPD and it just works, that may be reliable for someone. And if you may be reliable for someone. And if you ask the same question, if you keep ask the same question, if you keep getting the same answer, that may be getting the same answer, that may be reliable for someone. That doesn't reliable for someone. That doesn't happen by default because of the way happen by default because of the way LLM's uh work. But reliability for me is LLM's uh work. But reliability for me is more at a higher level. If a user wants more at a higher level. If a user wants to do X and whatever the AI system that to do X and whatever the AI system that they're using allows them to do X in a they're using allows them to do X in a very efficient way without coming in the very efficient way without coming in the way and enables to achieve their goal. way and enables to achieve their goal. Um that's what a reliable AI is. It's Um that's what a reliable AI is. It's like you know the magic moments. If AI like you know the magic moments. If AI can create those magic moments it can be can create those magic moments it can be a magic moment in a mundane task. It can a magic moment in a mundane task. It can be a magic moment in an exploratory be a magic moment in an exploratory task. when the user gets satisfied that task. when the user gets satisfied that for me is reliability. Yeah. So I think for me is reliability. Yeah. So I think we at least people living in the Bay we at least people living in the Bay Area here uh are in a bubble and you Area here uh are in a bubble and you know we talk about the latest AI know we talk about the latest AI advancements we talk about like GPU advancements we talk about like GPU availability all those are real problems availability all those are real problems but there is a huge huge world outside but there is a huge huge world outside of the Bay Area there are so many of the Bay Area there are so many different industries which can benefit different industries which can benefit uh with AI. So coming back to my uh with AI. So coming back to my definition of reliability using AI definition of reliability using AI techniques, AI techniques, AI agents, using AI applications, agents, using AI applications, working with all these industries that working with all these industries that Bay Area does not think on a day-to-day Bay Area does not think on a day-to-day basis is what I believe Bay Area should basis is what I believe Bay Area should focus more on. Some people call it quote focus more on. Some people call it quote unquote the boring AI. Uh but it's very unquote the boring AI. Uh but it's very very impactful. The AI stack has very impactful. The AI stack has fundamentally changed as the technology fundamentally changed as the technology has changed. Um, previously it used to has changed. Um, previously it used to be the LAMP stack, Linux, Apache, MySQL, be the LAMP stack, Linux, Apache, MySQL, PHP. Now we have a new stack. We're PHP. Now we have a new stack. We're calling it the park stack. Pyarch, calling it the park stack. Pyarch, AI foundation models, agents, AI foundation models, agents, frameworks, ray, and kubernetes. P A R K frameworks, ray, and kubernetes. P A R K park stack. And it's kind of funny park stack. And it's kind of funny because it has got AI within the AI because it has got AI within the AI stack in a recursive way. So this stack stack in a recursive way. So this stack we are seeing adoption across lots of we are seeing adoption across lots of companies Pinterest, Uber, Roblox, companies Pinterest, Uber, Roblox, Netflix, Apple, Shopify, cursor, bunch Netflix, Apple, Shopify, cursor, bunch of these companies are adopting some of these companies are adopting some form of this stack. So this is the AI form of this stack. So this is the AI infrastruct 3 to 5 years.