SF Scala, Yaron Minsky Interview

Acco everybody I'm Alexa karo the Acco everybody I'm Alexa karo the organizer of SF Scala and today we have organizer of SF Scala and today we have the first join the top of SF skull and the first join the top of SF skull and Saphan Taksin Saphan Taksin SF in tech is the new Meetup the full SF in tech is the new Meetup the full name was San Francisco from tech and name was San Francisco from tech and blockchain developers and developers is blockchain developers and developers is the keyword so we are basically an the keyword so we are basically an overlap of developers using functional overlap of developers using functional programming in novel technologies for programming in novel technologies for FinTech and the first inaugural event FinTech and the first inaugural event today we're very lucky to have your own today we're very lucky to have your own Minsky from Jane Street Capital which is Minsky from Jane Street Capital which is in New York trading firm and Iran is a in New York trading firm and Iran is a well-known person in the world of a well-known person in the world of a camel Jane Street is based on a camel camel Jane Street is based on a camel right and so I'll let you run introduce right and so I'll let you run introduce himself and tell us what he does just himself and tell us what he does just read welcome Iran thank you so much read welcome Iran thank you so much so I've been the change speed for a long so I've been the change speed for a long time and did a bunch of different time time and did a bunch of different time things since I was there I was the things since I was there I was the person who is roughly speaking the one person who is roughly speaking the one who's responsible for the decision to who's responsible for the decision to bring go camel into the company and bring go camel into the company and helped kind of create that as the kind helped kind of create that as the kind of core development platform we use for of core development platform we use for building all of our software and in that building all of our software and in that time I helped found the quantitative time I helped found the quantitative research group at Chan Street and then I research group at Chan Street and then I moved on to focus on software various moved on to focus on software various kinds doing a lot of work in trading kinds doing a lot of work in trading systems and these days I spend a lot of systems and these days I spend a lot of time in the tools of compilers part of time in the tools of compilers part of the world we have now our own kind of the world we have now our own kind of serious development tools of compilers serious development tools of compilers group and also continue to do work in group and also continue to do work in the trading system space mm-hmm so for the trading system space mm-hmm so for those who don't you know know much about those who don't you know know much about the camel all right it's a amel type the camel all right it's a amel type language and skull is very related so language and skull is very related so you know it's a skull is the biggest FP you know it's a skull is the biggest FP meetup in the world meetup in the world I feel very responsible for other I feel very responsible for other function foreign languages so we function foreign languages so we regularly host airline folks and Haskell regularly host airline folks and Haskell people so we kind of you know think that people so we kind of you know think that skull is the language really made a big skull is the language really made a big time time in industry and so but I think Jane suit in industry and so but I think Jane suit is the most successful company you know is the most successful company you know curve also can us here tell us a little curve also can us here tell us a little bit why IO criminal works so well for bit why IO criminal works so well for jested so there's a number of things jested so there's a number of things that that mako-chan will work well I think one of mako-chan will work well I think one of the most important things is it's a I the most important things is it's a I mean in general the ml family of mean in general the ml family of languages sit in a kind of sweet spot in languages sit in a kind of sweet spot in the space of programming language design the space of programming language design in that they the type system of a in that they the type system of a language like oh camel is relatively language like oh camel is relatively simple to understand simple to understand mm-hm it's relatively straightforward to mm-hm it's relatively straightforward to use and at the same time does a really use and at the same time does a really good job of catching bugs and preventing good job of catching bugs and preventing errors and allowing you to express errors and allowing you to express invariance of the program in terms of invariance of the program in terms of the type system and though this whole the type system and though this whole business of writing software that trades business of writing software that trades on the markets is thought about on the markets is thought about correctly terrifying you are writing correctly terrifying you are writing salt you know anyone who's a programmer salt you know anyone who's a programmer understands how easy it is to make understands how easy it is to make mistakes and the idea that you could mistakes and the idea that you could write something which both can connect write something which both can connect to the markets and make big monetary to the markets and make big monetary decisions and has access to your wallet decisions and has access to your wallet like you should be afraid of that and so like you should be afraid of that and so we think Oh camel does a lot towards we think Oh camel does a lot towards helping us build really reliable helping us build really reliable believable software and to do so believable software and to do so relatively cheaply ready to kind of it's relatively cheaply ready to kind of it's a it's a real force multiplier in that a it's a real force multiplier in that it lets you automate away a lot of the it lets you automate away a lot of the kind of mechanical checking that you kind of mechanical checking that you would have to do by hand and do it in a would have to do by hand and do it in a much more scalable way so I think that's much more scalable way so I think that's kind of the core of why it's so valuable kind of the core of why it's so valuable to us another thing that I think makes a to us another thing that I think makes a camel really good is a very wide camel really good is a very wide spectrum language it's usable for a spectrum language it's usable for a little tiny script and it's usable for little tiny script and it's usable for large scale programs so we run a tight large scale programs so we run a tight ship we have a fairly small number of ship we have a fairly small number of developers gene treat has 600 employees developers gene treat has 600 employees maybe 150 developers full-time in that maybe 150 developers full-time in that maybe another 100 people who program is maybe another 100 people who program is a significant part of your job mm-hm a significant part of your job mm-hm by the substantial we build a huge by the substantial we build a huge number of systems and we trade in number of systems and we trade in countless different exchanges countless countless different exchanges countless regulatory regimes huge amounts of regulatory regimes huge amounts of complexity in the systems that we build complexity in the systems that we build and in order for that to scale up we and in order for that to scale up we need a language that really maximizes need a language that really maximizes the reach of each individual programmer the reach of each individual programmer and by having one language that crosses and by having one language that crosses a variety of domains it makes it a variety of domains it makes it possible people to reach into all possible people to reach into all different parts of the system and one different parts of the system and one last thing is you can make it go really last thing is you can make it go really fast we we write Oh camel programs that fast we we write Oh camel programs that can take down five million messages a can take down five million messages a second second and consume and emit a packet in five and consume and emit a packet in five microseconds or less you can write microseconds or less you can write really extremely performative camel coat really extremely performative camel coat the it's in part has to do with really the it's in part has to do with really understanding how to use the language understanding how to use the language well and in part just has to do with the well and in part just has to do with the fact that the language is understandable fact that the language is understandable the runtime is relatively simple mm-hmm the runtime is relatively simple mm-hmm the language itself is is relatively the language itself is is relatively easy to understand its performance easy to understand its performance characteristics and so that's all very characteristics and so that's all very valuable so thank you you mentioned well valuable so thank you you mentioned well that you introduced the camel togethers that you introduced the camel togethers in 2002 yeah that's I mean that's longer in 2002 yeah that's I mean that's longer than persons like Scala started in 2006 than persons like Scala started in 2006 right so like this is really a lifetime right so like this is really a lifetime right like a lot of companies did not right like a lot of companies did not exist Facebook did not exist in 2002 exist Facebook did not exist in 2002 right so Google was kind of right so Google was kind of up-and-coming so I'm very curious you up-and-coming so I'm very curious you it's we observed the evolution of a it's we observed the evolution of a chemical how do you did you become chemical how do you did you become interested in a camel in the first place interested in a camel in the first place so I learned about our camel as a so I learned about our camel as a graduate student I was doing my my graduate student I was doing my my background is not programming languages background is not programming languages distributed systems and I was building distributed systems and I was building prototypes at first in Java prototypes at first in Java because that would seem like a because that would seem like a reasonable I wish to do things in and it reasonable I wish to do things in and it just it was just too painful like I just it was just too painful like I remember at some point I built some remember at some point I built some little clock synchronization algorithm little clock synchronization algorithm and I wanted to spin up 200 copies of and I wanted to spin up 200 copies of the program to kind of communicate with the program to kind of communicate with each other on my box and like the Box each other on my box and like the Box basically just fell over because a huge basically just fell over because a huge amount of memory required for each amount of memory required for each instance and there were other people you instance and there were other people you know my some of my some of my office know my some of my some of my office mates who had been using oh camel and I mates who had been using oh camel and I learned from them and tried it out and I learned from them and tried it out and I was kind of hooked from that point it's was kind of hooked from that point it's so lightweight mm-hmm you know and and so lightweight mm-hmm you know and and and much more concise much simpler than and much more concise much simpler than writing Java in many ways and so that I writing Java in many ways and so that I kind of started using it from there and kind of started using it from there and I brought it in to Jane Street roughly I brought it in to Jane Street roughly by mistake like I didn't mean to I was by mistake like I didn't mean to I was when I started doing some stuff at Jane when I started doing some stuff at Jane Street it was very early in the history Street it was very early in the history of the company just a couple of years in of the company just a couple of years in and I didn't think I was staying I and I didn't think I was staying I thought I was going to go off to be a thought I was going to go off to be a professor mm-hmm and so I approached the professor mm-hmm and so I approached the work I was doing his research codes I'll work I was doing his research codes I'll throw away I'll get some results I was throw away I'll get some results I was doing statistical research on trading doing statistical research on trading strategies at the time mm-hm and eighty strategies at the time mm-hm and eighty thousand lines of code later it turns thousand lines of code later it turns out I wasn't going back to be a out I wasn't going back to be a professor and we had this whole camel professor and we had this whole camel codebase and we started using it from codebase and we started using it from there there and it was a few years in in 2005 where and it was a few years in in 2005 where we really made the decision to start we really made the decision to start experimenting with building our experimenting with building our production infrastructure in Ocala production infrastructure in Ocala and you know that worked out well and and you know that worked out well and and we built our first trading systems and we built our first trading systems in no camel and that initial project was in no camel and that initial project was very successful and kind of the rest is very successful and kind of the rest is history we kept on using it to build out history we kept on using it to build out new things and replace old new things and replace old infrastructure and and from there it infrastructure and and from there it became the foundation of how we approach became the foundation of how we approach the process of building software do have the process of building software do have an engineer's in New York or do also an engineer's in New York or do also have them in in London and Tokyo and have them in in London and Tokyo and other places we have and so we have other places we have and so we have three offices one in New York one in three offices one in New York one in London one in Hong Kong mm-hm and we London one in Hong Kong mm-hm and we have software developers in all of them have software developers in all of them mm-hmm I think one of the reasons I mm-hmm I think one of the reasons I think software is fundamental to the think software is fundamental to the kind of work that we do mm-hmm kind of work that we do mm-hmm people sometimes ask are you a trading people sometimes ask are you a trading company or more of a technology company company or more of a technology company and my answer is we're a trading firm and my answer is we're a trading firm when trading is a fundamentally when trading is a fundamentally technological business and everywhere we technological business and everywhere we have people actively thinking about have people actively thinking about monitoring trading we need to have the monitoring trading we need to have the people who understand those systems people who understand those systems deeply there as well so we it's really deeply there as well so we it's really important to us to have serious important to us to have serious development groups in all of the offices development groups in all of the offices and also it's a place to find talent yes and also it's a place to find talent yes so uh can you characterize Jesse as a so uh can you characterize Jesse as a high-frequency trader no so nothing's high-frequency trader no so nothing's anything wrong with high frequency anything wrong with high frequency trading and some of the strategies we do trading and some of the strategies we do are depend on on low latency and we are depend on on low latency and we certainly build infrastructure of that certainly build infrastructure of that kind but it's not the heart of the kind but it's not the heart of the business that we do we are maybe more business that we do we are maybe more than anything else a risk-taking than anything else a risk-taking organization like we are liquidity organization like we are liquidity providers mm-hmm and providing liquidity providers mm-hmm and providing liquidity means taking risk means that you you means taking risk means that you you have to hold positions if in some sense have to hold positions if in some sense you think like someone who sits in the you think like someone who sits in the middle a middleman if you're if you run middle a middleman if you're if you run a grocery store you're a middleman and a grocery store you're a middleman and the thing you're doing is you are the thing you're doing is you are connecting buyers and sellers across connecting buyers and sellers across time and space mm-hmm right not every time and space mm-hmm right not every wants to go straight to the farmer to wants to go straight to the farmer to buy their tomatoes it's more convenient buy their tomatoes it's more convenient to go to your local grocery store who's to go to your local grocery store who's like kept the inventory there available like kept the inventory there available for you mm-hmm and for that for that for you mm-hmm and for that for that work they earn their in some some money work they earn their in some some money from the gap between the price you're from the gap between the price you're able to buy in the price able to sell able to buy in the price able to sell mm-hmm we do very much the same thing in mm-hmm we do very much the same thing in markets mm-hmm but more than moving markets mm-hmm but more than moving things through space which we also do things through space which we also do were often moving liquidity were often moving liquidity across time mm-hmm and that involves across time mm-hmm and that involves taking risk right at the close of the taking risk right at the close of the markets people demand a lot of liquidity markets people demand a lot of liquidity and we want to be on the other side of and we want to be on the other side of that we want to be able to supply that we want to be able to supply liquidity people who are demanding it liquidity people who are demanding it and that means we hold lots of risk and that means we hold lots of risk overnight so the kind of liquidity overnight so the kind of liquidity provision and risk-taking is a provision and risk-taking is a fundamental part of what we do and fundamental part of what we do and that's often less of like the kind of that's often less of like the kind of the often the true high-frequency firms the often the true high-frequency firms are doing very small trading back and are doing very small trading back and forth and try and go home flat at the forth and try and go home flat at the end of the day and that's very far from end of the day and that's very far from the kind of trading that we do mm-hmm so the kind of trading that we do mm-hmm so is it true that again I'm kind of is it true that again I'm kind of further than you in the space is it true further than you in the space is it true so then the market maker should be as so then the market maker should be as agile as a hyper concentrator because agile as a hyper concentrator because eventually they're all throwing away a eventually they're all throwing away a platform so you should be a little bit platform so you should be a little bit ahead of most of the traders you should ahead of most of the traders you should be really fast these are the fundamental be really fast these are the fundamental requirement for the market maker so requirement for the market maker so speed has always been a part of trading speed has always been a part of trading like when people were trading down by like when people were trading down by the Buttonwood tree where even the old the Buttonwood tree where even the old new york stock exchange getting new york stock exchange getting information more quickly or someone else information more quickly or someone else mattered and there's always some mattered and there's always some temporal component to trading but people temporal component to trading but people can provide liquidity at many different can provide liquidity at many different time horizons so I think it's not the time horizons so I think it's not the case that the people sometimes imagine case that the people sometimes imagine the only kind of market making is the only kind of market making is high-frequency trading that's just not high-frequency trading that's just not true mm-hmm you say so you know I did my true mm-hmm you say so you know I did my PhD essentially in multiple functional PhD essentially in multiple functional programming languages I kind of use this programming languages I kind of use this opportunity to learn Haskell and Scala opportunity to learn Haskell and Scala but I actually ended up with a camel but I actually ended up with a camel wrestle I ended up a thing in the sweet wrestle I ended up a thing in the sweet spot because you know and I was doing spot because you know and I was doing this from about 2008 to 2011 and when this from about 2008 to 2011 and when the crisis happened right and I the crisis happened right and I interacted with a lot of folks at Gene's interacted with a lot of folks at Gene's 3 - contributors to open source and I 3 - contributors to open source and I it's somewhat I was worried that you it's somewhat I was worried that you know the financial crisis like is gonna know the financial crisis like is gonna bring down James Street where all these bring down James Street where all these people who are providing this beautiful people who are providing this beautiful open source you know like what's going open source you know like what's going to happen so I asked them you know our to happen so I asked them you know our guys doing alright and they told me but guys doing alright and they told me but in beautifully like this whole crisis in beautifully like this whole crisis for us is a good thing and essentially I for us is a good thing and essentially I got the feeling that because of this got the feeling that because of this risk-taking because I've been smart so I risk-taking because I've been smart so I wonder if you can talk about that how wonder if you can talk about that how okay like how the fact that okay like how the fact that you know you have all this code bases you know you have all this code bases and a camel helped you if it did to and a camel helped you if it did to weather crisis like 2008 are you kind of weather crisis like 2008 are you kind of is there any kind of way kind of it is there any kind of way kind of it helps you be prepared and change helps you be prepared and change strategies is there any connection strategies is there any connection between the technology use and your between the technology use and your ability to weather crisis so I think so ability to weather crisis so I think so without saying anything about how we did without saying anything about how we did in any particular circumstance I think in any particular circumstance I think we're liquidity providers and in we're liquidity providers and in situations where there's lots of risk in situations where there's lots of risk in the market mm-hmm it's harder to be a the market mm-hmm it's harder to be a liquidity provider you're paid more for liquidity provider you're paid more for the service and I think that is a kind the service and I think that is a kind of dynamic of the of the business that of dynamic of the of the business that we're in I think we're in I think the stability and flexibility of the the stability and flexibility of the software that you write is essential to software that you write is essential to doing the business well and it's doing the business well and it's especially important when things get especially important when things get very busy yes this is simple matter very busy yes this is simple matter having software that doesn't fall over having software that doesn't fall over what things get busy is important but what things get busy is important but also having software that can adapt also having software that can adapt quickly that's reliable that you quickly that's reliable that you understand its behavior your confidence understand its behavior your confidence doing the thing you expect to do helps doing the thing you expect to do helps you do a good job of providing liquidity you do a good job of providing liquidity even in turbulent do you deploy often even in turbulent do you deploy often more often like you know the like some more often like you know the like some case happens the market do you find like case happens the market do you find like you need to deploy software more often you need to deploy software more often and those are I'm allowing you to kind and those are I'm allowing you to kind of deploy it faster so I'd say as a of deploy it faster so I'd say as a general matter we are very leery of general matter we are very leery of deploying software in a hurry mm-hmm deploying software in a hurry mm-hmm again I think one of the core issues again I think one of the core issues when you think about software and when you think about software and trading is it's a dangerous business you trading is it's a dangerous business you want to be very careful and deliberate want to be very careful and deliberate and how you will out software and when and how you will out software and when we have especially busy and important we have especially busy and important times we're normally just like more times we're normally just like more careful and more conservative about what careful and more conservative about what we do I absolutely think a language like we do I absolutely think a language like Oh camel helps you move faster in some Oh camel helps you move faster in some sense anything that gives you confidence sense anything that gives you confidence in the correctness of your code makes it in the correctness of your code makes it easier for you to make changes more easier for you to make changes more quickly because you can have more quickly because you can have more confidence that the change you made is confidence that the change you made is actually right and the stakes are very actually right and the stakes are very high and so you really need to be highly high and so you really need to be highly confident about the software that you confident about the software that you build build mm-hmm so so you basically were thinking mm-hmm so so you basically were thinking you are gonna go to academia be a you are gonna go to academia be a professor about the other dog is crying professor about the other dog is crying on Wall Street right so I'm very curious on Wall Street right so I'm very curious how do you find kind of reconcile this how do you find kind of reconcile this to these two cultures are thinner fokin to these two cultures are thinner fokin all in terms of secrecy right academics all in terms of secrecy right academics like to share they like to kind of talk like to share they like to kind of talk about knowledge and and what is about knowledge and and what is extremely secretive and siloed and and extremely secretive and siloed and and so when I you know when I bring together so when I you know when I bring together community saying you know like I want to community saying you know like I want to build a conference I find it's very hard build a conference I find it's very hard to make people talk about stuff and and to make people talk about stuff and and and share in general right so do you and share in general right so do you find that it's it's an obstacle to find that it's it's an obstacle to progress if you know we have this kind progress if you know we have this kind of highly evolved silos we should kind of highly evolved silos we should kind of so well finance I can hire like of so well finance I can hire like basically think they can hire all the basically think they can hire all the people and do inside what they want to people and do inside what they want to do but wouldn't you know like is there do but wouldn't you know like is there any way essentially to talk about this any way essentially to talk about this stuff right to talk about research or is stuff right to talk about research or is it just bound to be like this so we take it just bound to be like this so we take the sort of kind of trade secret aspect the sort of kind of trade secret aspect of the work we do seriously and stuff of the work we do seriously and stuff that we think of as important that we think of as important competitively that we're very careful competitively that we're very careful about how we talk about it but we also about how we talk about it but we also try and be enlightened about the way in try and be enlightened about the way in which sharing information can make for which sharing information can make for faster progress so internally a chain faster progress so internally a chain Street we try and do a good job of being Street we try and do a good job of being very open so that if if someone needs to very open so that if if someone needs to understand something in order to do understand something in order to do their job better they'll generally have their job better they'll generally have access and be able to understand access and be able to understand understand that thing and when understand that thing and when interacting with the outside world were interacting with the outside world were actually in many ways quite open we have actually in many ways quite open we have published an enormous amount of open published an enormous amount of open source software roughly ten percent of source software roughly ten percent of our code base is available on github our code base is available on github mm-hmm which i think is a very is a very mm-hmm which i think is a very is a very high percentage for almost any company high percentage for almost any company it's a lot to have any coming it's not it's a lot to have any coming it's not like purely doing open source stuff that like purely doing open source stuff that has some out of probably work that's a has some out of probably work that's a really big chunk and we've done a lot in really big chunk and we've done a lot in terms of talking about the software we terms of talking about the software we build and the ways that we approach and build and the ways that we approach and the libraries that we've constructed and the libraries that we've constructed and so I think we've actually had a lot of so I think we've actually had a lot of very productive relationships with very productive relationships with people in the outside people in the Oh people in the outside people in the Oh camel community camel community the academic community other people the the academic community other people the functional programming world so I don't functional programming world so I don't I think the in some sense the conflict I think the in some sense the conflict is less extreme than it seems like I is less extreme than it seems like I think if you take an enlightened view think if you take an enlightened view and think and don't just like and think and don't just like reflexively close off everything because reflexively close off everything because it's information and therefore any it's information and therefore any leakage of information it's bad but you leakage of information it's bad but you really think what is what are your kind really think what is what are your kind of core competencies what are things of core competencies what are things that are really important to you that that are really important to you that you really want to keep private and for you really want to keep private and for things that aren't like that you can things that aren't like that you can engage in more sharing and I think it's engage in more sharing and I think it's both satisfying for the people at James both satisfying for the people at James Street I think a lot of people who are Street I think a lot of people who are catching on the software side really catching on the software side really like being able to contribute to open like being able to contribute to open source in a serious way mm-hmm source in a serious way mm-hmm and the same time I think you get a lot and the same time I think you get a lot out of it you get better interaction and out of it you get better interaction and feedback on what you do from the outside feedback on what you do from the outside world and you get to learn things from world and you get to learn things from things that other people have done I things that other people have done I think I mean like from my experience think I mean like from my experience surveying the space your guys are one of surveying the space your guys are one of the most advanced right because you both the most advanced right because you both have open source and you talk about it have open source and you talk about it some companies put out open source in some companies put out open source in contact but they don't really talk much contact but they don't really talk much about it's very hard to get speakers so about it's very hard to get speakers so I mean I commend you guys for this I mean I commend you guys for this that's that's really great and I mean that's that's really great and I mean you do a lot of talks which are very you do a lot of talks which are very highly rated right like I think like you highly rated right like I think like you do a lot of advocacy for function do a lot of advocacy for function programming which is really available programming which is really available and apply it applies across the space and apply it applies across the space right so that's why we're very happy to right so that's why we're very happy to have you have you you know the subscribe because I think you know the subscribe because I think the principles are the same essentially the principles are the same essentially so so there's a question about the so so there's a question about the camera locker system right so I kind of camera locker system right so I kind of found this very interesting I have this found this very interesting I have this very interesting feeling when I you know very interesting feeling when I you know was doing a Carol being a graduate was doing a Carol being a graduate student because I didn't really have the student because I didn't really have the time to like learn all the newest hype time to like learn all the newest hype and kind of I in want 10 Jason libraries and kind of I in want 10 Jason libraries because then I would have to figure out because then I would have to figure out which one is the best it seems like oh which one is the best it seems like oh come on very sweet spot there's one come on very sweet spot there's one library for everything library for everything almost and it's like it may be done by almost and it's like it may be done by french guy in tokyo who's been like french guy in tokyo who's been like doing it for 10 years doing it for 10 years but you know for sure they don't ask the but you know for sure they don't ask the questions he's gonna reply it he's still questions he's gonna reply it he's still maintaining this so so it seems like a maintaining this so so it seems like a very nicely slow-moving space where it's very nicely slow-moving space where it's like most bases are covered right right like most bases are covered right right and so so there is like the scarcity of and so so there is like the scarcity of choice which even with distractions so choice which even with distractions so the pace of change is is relatively slow the pace of change is is relatively slow and I don't know is it because it starts and I don't know is it because it starts as a kind of French academic project as a kind of French academic project that you still is that you still is right but I wonder what like what's your right but I wonder what like what's your feeling in terms of Oscar Mallik feeling in terms of Oscar Mallik assistant so one day I'll say I think assistant so one day I'll say I think that is in some ways an out of data and that is in some ways an out of data and out of date view of the ecosystem and I out of date view of the ecosystem and I agree there are some there are some good agree there are some there are some good good aspects to the kind of slow-moving good aspects to the kind of slow-moving piece but actually the pace of Oh camel piece but actually the pace of Oh camel development certainly of the core development certainly of the core compiler itself has massively increased compiler itself has massively increased in the last year's there was a period in the last year's there was a period maybe a decade ago where things almost maybe a decade ago where things almost stopped moving mm-hmm but there's been a stopped moving mm-hmm but there's been a series of changes some driven by us a series of changes some driven by us a lot driven by not by other people mm-hm lot driven by not by other people mm-hm and it's no longer the case where and it's no longer the case where there's just one JSON library now there's just one JSON library now there's now there's a bunch of them I there's now there's a bunch of them I mean I think that as much as a bunch of mean I think that as much as a bunch of energy has gone into the kind of energy has gone into the kind of packaging and distribution story so it's packaging and distribution story so it's much easier people to publish packages much easier people to publish packages so there are more of them now mm-hmm so there are more of them now mm-hmm it's I mean I think it's still smaller it's I mean I think it's still smaller than say the set of packages that you than say the set of packages that you find it's a the Haskell community which find it's a the Haskell community which has lots and lots and lots of little TT has lots and lots and lots of little TT packages and I do think you know the packages and I do think you know the dynamic could describe there being fewer dynamic could describe there being fewer packages about an average of packages about an average of higher-quality I think is I think is higher-quality I think is I think is right but I think the language itself I right but I think the language itself I think it has been moving much faster of think it has been moving much faster of late mm-hmm but it definitely picks up late mm-hmm but it definitely picks up advances later which i think is both advances later which i think is both good and bad mm-hmm so the the if you good and bad mm-hmm so the the if you one nice example is type classes mm-hmm one nice example is type classes mm-hmm which are a great feature Haskell has which are a great feature Haskell has had it for 20 years had it for 20 years mm-hmm and there are some fundamental mm-hmm and there are some fundamental issues with the way Haskell of those issues with the way Haskell of those type clauses like you talk to the people type clauses like you talk to the people who kind of are at the heart of the who kind of are at the heart of the Haskell world they like have some Haskell world they like have some complaints about how type classes are complaints about how type classes are built and camels in the process of built and camels in the process of getting what are effectively typed getting what are effectively typed classes modular implements and it's 20 classes modular implements and it's 20 years late mm-hmm but it's a more years late mm-hmm but it's a more principled than cleaner design than you principled than cleaner design than you find in in other languages that got the find in in other languages that got the future earlier and you see this in a future earlier and you see this in a number of different spots interesting so number of different spots interesting so I think oh camel has like the language I think oh camel has like the language features have moved forward at a kind of features have moved forward at a kind of deliberate pace but with making really deliberate pace but with making really excellent high quality choices excellent high quality choices I think this wasn't always straighting I think this wasn't always straighting some there are some features that got some there are some features that got into early on in the language which I'm into early on in the language which I'm less excited about less excited about I think the object system would be I think the object system would be mostly very happy to live without mm-hmm mostly very happy to live without mm-hmm and basically in James to you more or and basically in James to you more or less live as if that didn't exist we less live as if that didn't exist we basically don't use it there's almost basically don't use it there's almost everybody else is there the Oh is there everybody else is there the Oh is there it was I think in some sense the Oh was it was I think in some sense the Oh was there are no camel because in the there are no camel because in the mid-90s to be cool you had to have an mid-90s to be cool you had to have an auditorium auditorium all right look and in many ways it's all right look and in many ways it's like a beautifully designed system I like a beautifully designed system I don't it's just not inside it's not so don't it's just not inside it's not so much that it's a bad thing to have it in much that it's a bad thing to have it in the language this is kind of an the language this is kind of an attractive nuisance which is to say in attractive nuisance which is to say in ml objects are almost never the simplest ml objects are almost never the simplest way to do things way to do things mmm-hmm there are kind of high-level mmm-hmm there are kind of high-level advanced power tool they use advanced power tool they use occasionally in fairly rare occasionally in fairly rare circumstances mm-hmm but which you come circumstances mm-hmm but which you come to a camel from in object-oriented to a camel from in object-oriented language you see objects can they go language you see objects can they go that's the natural thing to use and that's the natural thing to use and that's just a mistake right you should that's just a mistake right you should go to them last not first right I mean I go to them last not first right I mean I you know Scala is an object function you know Scala is an object function language but I'm not just kind of I language but I'm not just kind of I think the menus at least for me for you think the menus at least for me for you know object systems Carl it's kind of know object systems Carl it's kind of its packaging things it's I think it its packaging things it's I think it improve the syntax that's why I like improve the syntax that's why I like Scala or expression because it lets you Scala or expression because it lets you basically use the subject system to glue basically use the subject system to glue things together right because you have things together right because you have you know object of methods which you can you know object of methods which you can omit the dot and you have you know omit the dot and you have you know syntax which looks to some people more syntax which looks to some people more like math than than Haskell rock mo like math than than Haskell rock mo although to other people it looks although to other people it looks cleaner you know if you did a lot of cleaner you know if you did a lot of husk a little caramel right so but I husk a little caramel right so but I think it's not it's not a primary think it's not it's not a primary feature which function programmers using feature which function programmers using Scala so so your version fast enough Scala so so your version fast enough obviously that's another language which obviously that's another language which is very much up and coming in the is very much up and coming in the FinTech world right so the result of FinTech world right so the result of banks using Haskell there is a lot of banks using Haskell there is a lot of interest in a you know so look at this interest in a you know so look at this conference called scale by the bay which conference called scale by the bay which is to be called Scala by the bay we kind is to be called Scala by the bay we kind of called Scala a by the bay last year of called Scala a by the bay last year and now it's scale by the bay because we and now it's scale by the bay because we always covered more more languages right always covered more more languages right which like functional programming is the which like functional programming is the core of this but mostly as a synonym for core of this but mostly as a synonym for thoughtful programming so functional thoughtful programming so functional programming used to basically I think programming used to basically I think it's still this kind of the main it's still this kind of the main foundation of thoughtful software foundation of thoughtful software engineers and they can express engineers and they can express themselves in Haskell themselves in Haskell Oscar okay I'm a laugh sharp and Oscar okay I'm a laugh sharp and we have folks who did Scala and now they we have folks who did Scala and now they do Haskell in well funded startups which do Haskell in well funded startups which I didn't see before so I see multiple I didn't see before so I see multiple Haskell is startups coming online and so Haskell is startups coming online and so how do you see kind of the relationship how do you see kind of the relationship between husky macam well infant tech and between husky macam well infant tech and wider and I'm just curious about your wider and I'm just curious about your thoughts in that seems in question I thoughts in that seems in question I when I see when you look at the kind of when I see when you look at the kind of big industrial adoption of a camel most big industrial adoption of a camel most of the ones I see are outside of FinTech of the ones I see are outside of FinTech so Facebook has a very serious Oh camel so Facebook has a very serious Oh camel effort significantly bigger as far as I effort significantly bigger as far as I understand than the Haskell work it has understand than the Haskell work it has although they have Haskell stuff as well although they have Haskell stuff as well there's this very cool hacks a language there's this very cool hacks a language that sounded Marlo and versus all the that sounded Marlo and versus all the people who were ready right but there's people who were ready right but there's a bunch of major oh camel projects a bunch of major oh camel projects mm-hmm they're things like hack which is mm-hmm they're things like hack which is a programming language is basically a a programming language is basically a extension of PHP with a gradual type extension of PHP with a gradual type system flow which is a similar effort system flow which is a similar effort for a JavaScript infer which is a set of for a JavaScript infer which is a set of static analysis tools reason which is oh static analysis tools reason which is oh yes reason which is essentially a kind yes reason which is essentially a kind of alternative syntax and tool chain Pro of alternative syntax and tool chain Pro camel and in fact interestingly camel and in fact interestingly something like 50 percent of JavaScript something like 50 percent of JavaScript code in facebook Messenger is now code in facebook Messenger is now derived from oh camel I'm very excited derived from oh camel I'm very excited about this because you know like I about this because you know like I cannot bring myself to learn JavaScript cannot bring myself to learn JavaScript so I'm looking I'm waiting for it kind so I'm looking I'm waiting for it kind of to to kind of talk laws something to of to to kind of talk laws something to basically lost it like an objective-c is basically lost it like an objective-c is off last by Swift so reason seems like a off last by Swift so reason seems like a good way to do full stack programming good way to do full stack programming yeah yeah I mean and because you can use yeah yeah I mean and because you can use reason but you can also use no camel is reason but you can also use no camel is just a difference in price all right the just a difference in price all right the same core language but that's all same core language but that's all integrator free market and everything integrator free market and everything else I think that's using we think the else I think that's using we think the interact interact from the react in some interact interact from the react in some sense I think should be orthogonal right sense I think should be orthogonal right again reason is primarily like some again reason is primarily like some alternate tools mm-hmm and some nice alternate tools mm-hmm and some nice kind of integration and all that but kind of integration and all that but mostly an alternative syntax but the mostly an alternative syntax but the same group which is Nasri also use them same group which is Nasri also use them together and like you know this is gonna together and like you know this is gonna work together because it's the same work together because it's the same people people although anything although just to be although anything although just to be clear anything that anything that you clear anything that anything that you compile in reason always work with other compile in reason always work with other Oh camel because it just compiles down Oh camel because it just compiles down to this just an alternate alternative to this just an alternate alternative syntax so you have guaranteed full syntax so you have guaranteed full interoperability between no camel on interoperability between no camel on reason are you guys using anything like reason are you guys using anything like this for your UI components we don't use this for your UI components we don't use react we have react we have similar stuff that some stuff that's similar stuff that some stuff that's kind of similar in principle but instead kind of similar in principle but instead of using the react model we use a of using the react model we use a different incremental computing library different incremental computing library that we have called appropriately enough that we have called appropriately enough incremental for doing the optimization incremental for doing the optimization in some sense every time you have a UI in some sense every time you have a UI toolkit mm-hmm part of getting a good UI toolkit mm-hmm part of getting a good UI toolkit is coming up with a clean way of toolkit is coming up with a clean way of expressing the logic of the UI mm-hmm expressing the logic of the UI mm-hmm but a large part of it is just but a large part of it is just optimization making it actually perform optimization making it actually perform sufficiently well mm-hmm and you know sufficiently well mm-hmm and you know when you have changes to the data the when you have changes to the data the changes to the state not having to changes to the state not having to recompute everything yeah and there are recompute everything yeah and there are different approaches to that react has different approaches to that react has one particular incrementalization one particular incrementalization approach Elm is another language that approach Elm is another language that has a very similar story for how it the has a very similar story for how it the incremental computing stuff that's kind incremental computing stuff that's kind of built into it an incremental is a of built into it an incremental is a fairly different one it's based on a fairly different one it's based on a thing called self adjusting computations thing called self adjusting computations which is some research that came out of which is some research that came out of CMU I got a new motorcar which is a some CMU I got a new motorcar which is a some ways richer and more powerful ways richer and more powerful optimization system which you know optimization system which you know there's some trade-offs you have to live there's some trade-offs you have to live in a monad which not a tree likes to do in a monad which not a tree likes to do mm-hmm mm-hmm but it gives you more optimization power but it gives you more optimization power so you can build richer and more so you can build richer and more complicated dynamic views which is complicated dynamic views which is important in a financial context because important in a financial context because you're the UI's you build are all about you're the UI's you build are all about fast streaming data it's very large fast streaming data it's very large always changing data sets leave in this always changing data sets leave in this UI so they've actually use them to UI so they've actually use them to execute I mean there's and I think they execute I mean there's and I think they probably the the UI they live in more probably the the UI they live in more than anything else is Excel mm-hmm and than anything else is Excel mm-hmm and we have lots of nice integration into we have lots of nice integration into Excel as well but yeah there are a lot Excel as well but yeah there are a lot of these UIs that are increasingly of these UIs that are increasingly central to the traders workflows in fact central to the traders workflows in fact the system I'm going to talk about the system I'm going to talk about tonight I'm not going to talk about the tonight I'm not going to talk about the UI piece but there is a web UI for it UI piece but there is a web UI for it that is built using exactly this that is built using exactly this incremental ization framework and that incremental ization framework and that is absolutely fundamental to the way is absolutely fundamental to the way number the different trading desks number the different trading desks operate interesting so I think we're operate interesting so I think we're kind of kind of coming to the end of kind of kind of coming to the end of this interview so I've only got two this interview so I've only got two questions here so one is we have a lot questions here so one is we have a lot of folks who start to do machine of folks who start to do machine learning in factual programming so and learning in factual programming so and skull is increasingly useful data right skull is increasingly useful data right and and and so although we're kind of in the and so although we're kind of in the time when pythons dominating data time when pythons dominating data science right a lot of people learn science right a lot of people learn Python first so I hope feeling that Python first so I hope feeling that languages like Scala vascular camel languages like Scala vascular camel they're going to be very successful in they're going to be very successful in machine learning they're going forward machine learning they're going forward and I think what you do falls under this and I think what you do falls under this umbrella because essentially I'm making umbrella because essentially I'm making a using machine learning and data mining a using machine learning and data mining to make decisions right which actually to make decisions right which actually affects millions of dollars so so the affects millions of dollars so so the you're thinking you're one of the you're thinking you're one of the mission-critical applications I wonder mission-critical applications I wonder if you have any intuitions as to why if you have any intuitions as to why people who do much learning should look people who do much learning should look at faster program even if they don't at faster program even if they don't know about it right because most most of know about it right because most most of them start in Python right now coming them start in Python right now coming into data science what is it you know into data science what is it you know about protein which can make kind of a about protein which can make kind of a machine learning easier better more machine learning easier better more reliable so a couple of H one is its reliable so a couple of H one is its birthing like we've been doing kind of birthing like we've been doing kind of statistically driven machine learning statistically driven machine learning style trading strategies for a long time style trading strategies for a long time but a key that's not like the kind of but a key that's not like the kind of core of what we do I think the I think core of what we do I think the I think the thing that James she's probably best the thing that James she's probably best at is a kind of rich integration between at is a kind of rich integration between human intuition and interaction and and human intuition and interaction and and the kind of quantitative understanding the kind of quantitative understanding that you get from having a good that you get from having a good mathematical understanding of and the mathematical understanding of and the good game playing understanding of how good game playing understanding of how trading works mm-hmm so I think people trading works mm-hmm so I think people often imagine where it's like the whole often imagine where it's like the whole place is just about building models and place is just about building models and watching things trade and not not having watching things trade and not not having this kind of deep human interaction and this kind of deep human interaction and that's just not true where the the human that's just not true where the the human part is incredibly important to what we part is incredibly important to what we do that's it we do do a lot of do that's it we do do a lot of statistical analysis and machine statistical analysis and machine learning style techniques indeed that's learning style techniques indeed that's the first thing that I did when I the first thing that I did when I started was working on that kind of started was working on that kind of stuff stuff I think functional programming and I think functional programming and sometimes I mostly don't think of sometimes I mostly don't think of functional programming as a functional programming as a special-purpose tool I think ml is an special-purpose tool I think ml is an incredibly good general-purpose language incredibly good general-purpose language and it's good for a wide variety of and it's good for a wide variety of different techniques and machine different techniques and machine learning in some fits in in a very learning in some fits in in a very natural way natural way here's like one simple story about the here's like one simple story about the ways in which these kind of since it's ways in which these kind of since it's very ordinary benefits come up with very ordinary benefits come up with machine learning context so we also use machine learning context so we also use Python and R and other things for doing Python and R and other things for doing data analysis but we ended up with a Oh data analysis but we ended up with a Oh camel wrapper for tensorflow camel wrapper for tensorflow mm-hmm in addition to using a Python mm-hmm in addition to using a Python wrapper for one of these we discovered wrapper for one of these we discovered was it was really nice to use the Ocala was it was really nice to use the Ocala version because sometimes we use the version because sometimes we use the Python version you'd set up your your Python version you'd set up your your model and you'd go off and do the model and you'd go off and do the compute intensive part of it and it compute intensive part of it and it could take a long time it can take hours could take a long time it can take hours mm-hmm to run the analysis and then mm-hmm to run the analysis and then you'd run it and then it would blow up you'd run it and then it would blow up completely stupid reason that you could completely stupid reason that you could have caught at the type system well yes have caught at the type system well yes and when you have these kind of compute and when you have these kind of compute intensive jobs it goes to the old sort intensive jobs it goes to the old sort of like you want to catch errors as it of like you want to catch errors as it really as possible mhm and that is true really as possible mhm and that is true in all sorts of in all sorts of software in all sorts of in all sorts of software circumstances but it's very much true in circumstances but it's very much true in machine learning machine learning just from a kind of pure efficiency just from a kind of pure efficiency point of view of getting stuff done well point of view of getting stuff done well just having more confidence that the just having more confidence that the thing that you set up is actually right thing that you set up is actually right before you spend 3,000 compute hours on before you spend 3,000 compute hours on it mm-hmm is super valuable I think it mm-hmm is super valuable I think there's lots of benefits of that kind of there's lots of benefits of that kind of good type system is valuable and the good type system is valuable and the kind of freedom of flexibility that you kind of freedom of flexibility that you get from the kind of higher or easy get from the kind of higher or easy lightweight higher-order programming you lightweight higher-order programming you could do in a language like ml mm-hmm could do in a language like ml mm-hmm it's just all very valuable is that okay it's just all very valuable is that okay well wrapper open-source yes nice well wrapper open-source yes nice thanks for mentioning this I think it's thanks for mentioning this I think it's you know kind of I think it'll be very you know kind of I think it'll be very interesting to deploy that so what we're interesting to deploy that so what we're going to have is you guys are going to have is you guys are contributing a lot of stuff to the contributing a lot of stuff to the community right what do you want to see community right what do you want to see in the from the community right you so in the from the community right you so you give a lot of open source like do you give a lot of open source like do you want to hire smart people we want you want to hire smart people we want people to help you develop a camera people to help you develop a camera ecosystem what is kind of what are you ecosystem what is kind of what are you looking for from the community to help looking for from the community to help you with in the ochem Malachor system so you with in the ochem Malachor system so all of the above so we definitely do a all of the above so we definitely do a lot of hiring and I think these days lot of hiring and I think these days it's no longer there was a time where it's no longer there was a time where the majority of our hiring early on the the majority of our hiring early on the delivery of my personal hiring came out delivery of my personal hiring came out of the Oh camel community mm-hmm and now of the Oh camel community mm-hmm and now it's much more diverse than that but I it's much more diverse than that but I think we still get a lot of benefits think we still get a lot of benefits hiring people who have experience hiring people who have experience directly there may be higher loss people directly there may be higher loss people with no functional program experience at with no functional program experience at all because it turn it turns out it's all because it turn it turns out it's not hard to learn okay I'm all program not hard to learn okay I'm all program first force March for smart developers first force March for smart developers learning a new programming language is learning a new programming language is just not that big of a deal mm-hmm just not that big of a deal mm-hmm but still hiring is an important aspect but still hiring is an important aspect of it and I mean we have a lot of of it and I mean we have a lot of collaborations with people on the collaborations with people on the outside to work on you know there's outside to work on you know there's library external libraries that we library external libraries that we depend on and worries very grateful for depend on and worries very grateful for that kind of work there's tools so one that kind of work there's tools so one tool that we're really excited about and tool that we're really excited about and have you out is a tool called Merlin have you out is a tool called Merlin which gives essentially IDE like which gives essentially IDE like functionality for oh camel and we've functionality for oh camel and we've been using it for some years now and been using it for some years now and that was not something that we thought that was not something that we thought to develop mmm-hmm to develop mmm-hmm it was saying that other people thought it was saying that other people thought was important and they and they created was important and they and they created and then we realized was valuable and we and then we realized was valuable and we started using and in fact we've hired started using and in fact we've hired since then one of the developers to since then one of the developers to actually work at Jane Street but it was actually work at Jane Street but it was created from the outside I think there created from the outside I think there are lots of good ideas both at the level are lots of good ideas both at the level of what libraries to build and also at of what libraries to build and also at the level of developing the language the level of developing the language itself that come from the outside and itself that come from the outside and that's an enormous ly rich in productive that's an enormous ly rich in productive interaction there so wewe I this is in interaction there so wewe I this is in no way a one-way street no way a one-way street I think we I'd like to think that we I think we I'd like to think that we provide a lot of value mm-hmm but we provide a lot of value mm-hmm but we also get an enormous amount of value also get an enormous amount of value from from the oh camel community and to from from the oh camel community and to see a straightforward thing from the see a straightforward thing from the team that develops with camel we have team that develops with camel we have started recently putting more of our own started recently putting more of our own people working on the compiler mm-hmm people working on the compiler mm-hmm but the compiler is by and large the but the compiler is by and large the work of people who do not work at Jane work of people who do not work at Jane Street and we are enormous ly grateful Street and we are enormous ly grateful for the extremely high quality of for the extremely high quality of engineering and the good design and the engineering and the good design and the good taste with which it continues to be good taste with which it continues to be evolved and give the most important evolved and give the most important thing that the the core compiler team in thing that the the core compiler team in in RIA does is say no you said no to in RIA does is say no you said no to lots of bad ideas and kept oh camel a lots of bad ideas and kept oh camel a tight beautiful well-designed easy to tight beautiful well-designed easy to use easy to learn language and I almost use easy to learn language and I almost never look at a new release of the never look at a new release of the compiler and the gall that seems like compiler and the gall that seems like some piece of garbage I'm sad about some piece of garbage I'm sad about right the the things that are added are right the the things that are added are almost always well thought out sensible almost always well thought out sensible things and sometimes when I think things and sometimes when I think they're not interesting I'm just wrong they're not interesting I'm just wrong so for example when geo DTS were added so for example when geo DTS were added to a camel I had no idea what I wanted to a camel I had no idea what I wanted them for it sounded like the them for it sounded like the kind of nonsense you get when you allow kind of nonsense you get when you allow people who write compilers you let them people who write compilers you let them design a programming language and they design a programming language and they build things that are nice for their build things that are nice for their problems but I'm a systems programmer problems but I'm a systems programmer right I you know but it turns out gee right I you know but it turns out gee these are great for systems programming these are great for systems programming and it's been very important for a lot and it's been very important for a lot of the systems that we've built in a way of the systems that we've built in a way that we totally did not expect just a so that we totally did not expect just a so I mean you know III think I must say I mean you know III think I must say that this is really exciting to hear that this is really exciting to hear that basically you know we've got this that basically you know we've got this global company trading in like you go global company trading in like you go all the way back to the compiler which all the way back to the compiler which plays a key role in in that that's plays a key role in in that that's really exciting you know for like it's a really exciting you know for like it's a guilty geek to hear that so we're very guilty geek to hear that so we're very happy to have you and we're looking happy to have you and we're looking forward to a talk awesome thank you forward to a talk awesome thank you thanks