In this episode, Audrow Nash interviews Adrian Macneil, Co-founder and CEO of Foxglove. Foxglove makes Foxglove Studio, an open source visualization and debugging tool for robotics. Adrian speaks about the origin of Foxglove, Foxglove’s business model, web and robotics, and gives advice to those interested in getting more involved in robotics.

Start

Introducing Adrian and Foxglove

Origin of Foxglove

Developer productivity and the future of robotics

Building the Foxglove team

Why Adrian’s optimistic about opportunities in robotics

On putting 1000 engineers on a problem

Role of open source

Foxglove’s business model

Why Foxglove “Studio”?

How name Foxglove?

Explaining Foxglove Studio

Web and robotics

Web technologies on the backend

Reflecting on his work at Coinbase and Cruise

Timeline for autonomous cars + levels of autonomy

Advice for those interested in robotics

Info on getting in touch and contributing

Outro

Hi, everyone. Welcome to the Sensing Think Act Podcast. I'm the host Audrow Nash. And in this episode I interview Adrian Macneil, co founder and CEO of Foxglove, Foxglove makes Foxglove Studio, which is an open source visualization debugging tool for robotics. In this interview, Adrian and I talk about the origin of Foxglove, Foxglove's business model, web technologies and robotics, the timeline for autonomous cars, and Adrian gives advice for those interested in getting more involved in robotics. I really enjoyed talking to Adrian, he has an interesting perspective on technology and seems to be involved in interesting technologies, right as they're coming up. For example, He was an early employee at Coinbase, and Cruise. As always, a big thank you to our founding sponsor, Open Robotics, and I hope you enjoy. Hi, Adrian, would you introduce yourself? Hi, I'm

Adrian Macneil, founder and CEO of Foxglove.

Now, tell me a bit about foxglove what it does and what problem it's solving. And then I'd love to get into the backstory of that after.

Yeah, definitely. So foxglove is a developer tools company for robotics. We have our main product right now is an open source visualization platform for robotics. And it's browser based cross platform. We can allow you to really dive into your robotics data understanding of your sense of editing and planning, visualize things like point clouds, images, overlays, markers, bounding boxes, plots, and drill into kind of raw messages

and you guys you interface tightly with the robot operating system or ROS ecosystem, correct?

Right? Yeah, correct. So we we work very well with ROS. We don't we're not strictly limited to Russell though. That's definitely our best supported platform right now. We do allow you to connect other types of data sources like for example, we can connect just to a velodyne LIDAR and and talk directly to that sensor and visualize data coming off the sensor. But when you're in a ROS environment, we can obviously make a connection to ROS and then we have a lot of custom visualizations that help you understand ROS messages like point clouds and images and other things like

how does it compare to existing ROS tools like ARV is our ROS visualizer for visualization?

Yeah, so yeah, so we have I would say the biggest difference is that it's cross platform and using a web based stack that was something you know, we can get into the origin a little bit that that was something that was kind of a key underpinning of building this technology is that we wanted to work with a browser based stack which means that they can run cross platform we actually have desktop apps that you can install on your computer that run on Mac Windows and Linux equally well and then you can also access it in the browser. So that you know that makes it like very easy to just jump into an environment and start debugging it also kind of unlocks that something that I think is as important to me as breaking robotics development out of being stuck with like Linux desktops and you know Linux is great but a lot of robotics development now is very the software the developing is like very tied to the platform that you're developing on and you know if you're using ROS Vision X you know ROS galactic you need to be using Ubuntu 2004 and you know that's the only choice you have you pretty much stuck with that for your development handle and get developed well

it does work so galactic and many of the other ROS 2 distributions also work on Windows not very well I think tier three support for Mac this kind of thing Yeah, but

yeah, it's getting there I did say yeah, you're right.

But this was this was definitely which was kind of the origin of like foxglove was a rough one. Or Weber's Yeah, we

support both ROS 1 roster Yeah. But yeah, for sure our windows support is out there. Although you know, most of the people I've seen doing ROS development on windows are using the windows subsystem for Linux so they're actually still running you know, the actual Linux version. Oh, that's fine. And then yeah, and then Mac actually you know, Mac came out I think ROS 2 initially came out and said we're gonna suppose a lot of work. Yeah. Like no one kind of stepped up to volunteer. Yeah, so the and the actual you know, code that you're writing big Linux only is fine, right? Oh, we want to break out of as the developer tools so there's no reason that you can't be on a Mac developing inside a Docker container. But But your actual tools to like visualizing things, it's, you know, those graphical tools are going to be easier if you can just connect to them from a browser or connect or, you know, fire up a native emulator.

what's the what's the big win? Or what are some of the big wins doing it browser based as opposed to like qT? or something, which is what ARV is uses?

Yeah. Yeah, yeah. And I think I should jump into some of the other things to talk to the browser based thing. And then also some of the other things that kind of differentiate us with with Office and Matthew two things. browser based, I think, you know, there's a few points there. One is that it's it's inherently cross platform, you know, the browser is a really great environment where, you know, as long as you fire up your browser and go to a website, you can pretty much expect the same experience, regardless of what platform you're on. That makes it really good as you're scaling a robotics team. You know, if you start out with just one or two people, you're working on a project, that's, that's, you know, it's fine if you've got a pretty custom environment set up on your computer. But as you scale up that team, you want people to just be able to, you know, on board start working on the code base, you know, being able to like visit a URL and look at things directly is really great. And then another point there is that it makes it really easy for you to access remote data as well, right? So we can, you know, get into talking about data storage a little bit as well. But a lot of the times, you know, the state of the art and robotics today, a lot of people are just storing, you know, huge 100 gig bag files on a computer, and then I've been trying to debug it. And that's ROS bag. You want to share this, my friend? Yeah.

And ROS, too. It's an SQL lite database. Right. And I forget what it is in Roscoe, and

yeah, it's just a custom format, bag. Bag file format. That's just basically your ROS message. But having

a big monolithic data storage, basically. Massive.

Yeah, yeah, yeah. And so those can get really huge if you're dealing with multiple cameras, especially if you start getting into LIDAR and point clouds and radars and sensors, bossier. We're also recording all of the other messages that ROS is sending back and forth. And so, you know, a lot of people are working with these files locally. But sharing that among team members is really challenging. It's, you know, a case of like, either just get someone to wait here over and have a look at your desk. Or you know, you can't, you can't just like share a link in slack and have someone else click on that and immediately start looking at it. So that's, that's where we kind of see the browser's really unlocking these workflows where you can just click on data.

And is that talking about having data storage happening in the browser?

Yeah, yeah. So not always, I mean, the data storage happening on a server? Yeah. For sure, yeah. Yeah. But yeah, once you get to having a data stored in a centralized location, and the browser can just easily be streaming that, you know, it just makes the whole internet experience really, really compelling.

Let's see. So I feel like this is a good time to get into the origin story of foxglove. So can you tell me where it started?

Yeah, sure. So I spent the last five years prior to this cruise, building self driving cars. And I led basically all of the developer tools and infrastructure there. So when I joined, it was, you know, a very small company, maybe 40 4050 people or so over the over that period, we grew into the sort of 1000 plus range, I think it was around 1700, when I left out, and you know, over that time, or the engineering team grew right from, from 20 people to well over 1000. And so, you know, they they started, as you know, I'm sure a lot of our products company, so it was a ROS stack originally, on ROS 1, and you have a bunch of people just, you know, contributing code, building features, testing things out. But as you scale that and 100 and 1000 plus engineers, do you really need to start getting a lot more disciplined around the developer productivity? What tools are you making available to people? What are the workflows that you encourage people to do? You know, the less you standardize things, I guess the more pain you're in for as you start getting into 1000 plus engineers contributing to a project.

Yeah. And so crews had an internal effort to make something for visualization, rather than using the standard ROS 1 arvis. Or our Qt graph. Yeah. Anything?

Yeah, for sure. It started out as as visit Accutane. As I mentioned, they started out on a ROS 1 platform, positive ROS definitely gotten kind of switched out over time and replaced with in house things. But in the early days, everyone was just using, you know, the standard tools like ours. And the project that we that we kicked off with visualization there. I think we started back in about 2017. It came out of a hackathon actually it was not instead of a top down approach of you know, this is something we think we need to invest in, it was just something that some engineers really felt needed to exist and really felt would would kind of streamline the workflow. So yeah, that was kind of the origin of width is which, you know, some people may be familiar with, they presented with us at ROS con in 2019, I think it was eventually open sourced in 2019. But back in 2017, it really just started as, you know, some engineers on the team feeling that that we could, you know, we could do better on a couple of aspects. One is, you know, bringing these tools to the web where it's a lot easier for, for, you know, people to contribute to, like easier to find engineers.

Easier to contribute to it or easier to, for people, why is web easier for people to contribute to?

Yeah, I mean, I guess, you know, that could be a bit of a loaded statement, because obviously, if you're familiar with c++ development and development, then it's going to be a lot harder for you personally, to contribute to but you know, on a scale of, of larger commercial engineering teams, it's, on average, a lot easier to find front end and JavaScript engineers who can build web applications. There's just, you know, a much larger pool of people that you can draw

from, would you guess the order? Is there like, there's 10 times more people that are familiar with JavaScript than c++ or I mean, just ballpark for these kinds of jobs? Yeah, I think it's 10. Or, like this order of magnitude,

probably in the Bay Area. That wouldn't be far off. Yeah. Yeah. Yeah. Yeah. I haven't seen any numbers specifically on it. But yeah, anecdotally going out to hire a front end engineer versus a c++ engineer who has experience experience. Yeah, Qt experience and things like that. Like for sure there's a lot of really talented c++ engineers out there, but not so much. On the visualization side, there's this argument that I've, you know, done game development and things like that. But yeah, just just much easier for, especially once your company, you start building a lot of internal tools, usually a lot of internal tools are web based. And I don't want to push on, on being easy to contribute to is like, the main thing, but it's that combined with the fact that anyone at the company can just hit a URL and access that you don't need to worry about like deploying this app across your team. And also, you know, there was something that was important to us was being able to like unify these various tools into one interface. So

yeah, yeah, exactly. So are you able to bring

speak to what each does just a little bit?

Yeah, yeah. So avars is primarily a 3d based visualization tool. So you can go in and you can, you can open up some data, and you can look at where your model is sitting in the world. And you can see sensor data. And you can see, you know, three name markers, 3d bounding boxes, and things like that. You can see things like your transforms like the shape of your robot, and you know, where your robot is relative to something that it's trying to act on.

And if I understand correctly, it's not a 3d simulator. It's more just 3d visualization. There's no physics engine associated with it.

Right? Right. Right, exactly. So it's purely taking necessarily information from a robot, and then just showing you, you know, how your robot is thinking, and the world is gonna have an insight into your robot. And then you have, you know, acuity is sort of a collection of tools that you can open a standard one on Windows, or you can kind of open them in a sort of attached panel configuration. And it comes with a handful of other tools, which can give you things like reading diagnostics, from a robot reading direct messages from a robot, reading plots from a robot or reading plots, but displaying plots based on data from your robot. And then you start seeing that there's other tools in law recently that have come out like plot juggler and things, you know, a separate again. And so yeah, that was kind of one of our insights, I

And one of my early in one of my early days there I have a vivid memory of attending we so we had Friday demos, basically, every weekend, people could jump up and kind of just show something called what they've built. And someone from the QA team jumped up and demoed the script that they've written that would basically take a bag file and then open it, start ROS in the background, and then launch about like 12 different windows on your screen and rearrange them into position so that you'd be ready to start debugging. And I was like, you know, and they got a standing ovation for that right because it was so much. Such a massive time saver compared to Compared to like, manually going through that process and you know, taking a backfile and wanting ROS and then opening these various windows, you know, I just remember that so vividly because it's, it's, it's, you know, like seeing the pain of how difficult it is just to get to the point where you're actually getting your work done. It's like, this is not, this is just like admin before you actually get

20 minutes of work before you can start work.

Right. Right. And, you know, and that's actually sort of another difference that another point of differentiation we took is that, you know, the tools like others are, they're very coupled with ROS. So you have to have ROS, you run ROS, in the background, you stone, your ROS stack, and then you start others, and others will talk to ROS, and it'll show you kind of exactly what's happening. And if you want to play back a bag file, playback recording that you have, then you need to play that back into ROS. And then kind of you'll see it as like a byproduct of

Right, right. Everything's kind of going over the ROS network. And, again, when you start thinking about different workflows across the company, you have QA engineers, and they have a look at right.

So they want to talk up to good quality, and we're doing things well.

Okay, right, right, exactly. And if they're taking let's say, there's an incident where, you know, something that happened on the road, right? We pulled out this or not, we got too close to a cyclist, or we, you know, whatever we did, and you've been assigned to triage, an engineer has been assigned to that kind of triage this and figure out, maybe not like, what is the root cause just yet, but where do we think the likely root causes? You want to be able to just like, open that kind of incident directly and start looking at it? You

know, you're saying that the incident? Yeah, yes. rosbank file or okay.

Right, right. But you want to, like, what you actually want to do is get into visualizing that incident as quickly as possible. And you don't, you know, the fact that there steps like downloading a bag file,

which is huge. codebase, which is huge, probably. So it probably takes

uploads are often slow. You get it from a thumb drive.

Okay, yeah, tell you all about No. So it's just a process, right, kind of take that back file, and you bring it down to your computer, and then you open up, you know, check out the code base, and you go back to the version of the code base that you had last week that was actually running on and then you launch ROS launch hours, and like, now you're looking at the data. And, you know, we just wanted to simplify the whole process. So it's like, you click on the link, and then you're looking at it immediately.

Yeah, that is way easier, where you can just open it. And so you, and then this gets into the cross platform and everything because they could be on any computer. And they don't need to spend their money for OS installed. And they don't need to have ROS bag installed.

Just run it on a Chromebook or something and just, you know, immediately jump into things.

Gotcha. Okay, so then I see the motivation for this quite clearly. So you were at cruise, and you were working on web servers. And you had this experience where someone automated some of the setup that they might go through when trying to do some debugging. And that was like an aha moment for you. And then is that how web is started?

Yeah, and, you know, to clarify, I guess I, I was the director of engineering, so I didn't like contribute much, or anything to the webfest card itself that the team was under me, but all the talented engineers did the work not, not me. But you know, I very much agreed with the vision of of what they were trying to achieve there. And I think, you know, we, the team that was working on that there was some really talented engineers, they kind of made a proof of concept. Like I said, at a hackathon. They went around, you know, a handful of different teams, and kind of went team by him and said, Hey, you know, what problems you're having with your tools? How can we solve them with your visualization, and just incorporated all of the various workflows that were happening across the company into this one tool. And so, you know, over that period of a year or so, it just became kind of an indispensable that everyone was using at the company. And that really just opened my eyes up to you know, how much of a productivity boost a good quality visualization tool could be. And, you know, that's what ultimately led us to consulting.

Gotcha. And so you open sourced web is and then now you are with foxglove. Can you tell me a bit about that transition

Yeah, yeah, for sure. So we open sourced my visit cruise in around 2019. The goal there was to, you know, give back to the community. And, you know, also just to put some of that code out there as as, you know, engineers working on projects that companies quite often you want to just put, you know, anything that's not sort of like, tied critical IP to the company. Yeah, yeah, critical company secrets, it's always nice to be able to publish things. And we published a lot of open source code occurs around, you know, even just unrelated things, like how we're managing our Kubernetes infrastructure and things like that. But, you know, we put those out there, and it was available for the community to use. And, you know, they presented a Roscommon 2019, I think, and, you know, picked up a bit of steam. Certainly people were out there using it. But I guess, you know, here's where I would draw differentiation between non open source coders create an equal, I guess there's a difference between kind of code that is public versus actual communities that are being built. And, you know, I don't vote Cruz for this in any way. I think Chris is absolutely making the right decisions for their own prioritization and things. But, I mean, I was like part of these decision making but but Chris doesn't have time to build features and weathers that are not useful to crews, for example, you know, they have the project out there, but like, no one is building ROS 2 support and the weapons are known as

stack now. I mean, the stacks kind of, you know, hitting off in a different direction entirely. So, you know, it's the code was out there, and it was available for people to use, but there was never a kind of a pretense that crews wanted to sort of invest heavily in building a community and adding features that were like, not useful to them at all. Like I say, this is, you know, absolutely the right decision. If you're, if you're, the purpose of your company, is get a self driving car going in San Francisco, you know, the purpose of the company is not build unrelated kind of open source communities and invest heavily in that sometimes that happens, you know, if you're a company the size of Facebook, and then you build something like react exactly like, shot, you have hundreds of 1000s of employees, and you can spare a small team of 30 to work on XYZ thing, but

it's crazy. It's really that small. Yeah, that makes sense. But yeah, I didn't know. It could Yeah, or something. Was that just guess yeah.

But yeah, it's Yeah, it's, it's when you're a sufficiently large company, you can kind of afford to make those bets. But for the majority of companies out there, you're, you know, when they're open sourcing code, it's usually sort of isn't code that we wrote, use it if you like it, but we're not supporting it. still focused on my internal needs. Yeah. Makes sense. Yeah. So that's, you know, that's what ultimately led to foxglove. We had a lot of chances, the windows team with courage and leadership and several other weapons team are working Foxcroft now. But we wanted to make a clean break, I guess we, you know, another bit of context, as to the weapons, as it exists today is pretty tightly coupled with a lot of internal crew systems. So the open source version of weapons is only you know, only about half of the code and weapons is actually open source, right. And the rest of it is special custom source that exists inside our crews. And those two are quite tightly intertwined. It's not sort of super easy. For us, as founding foxglove, it's not super easy for us to go and kind of contribute back to that project without having access to all of the closed source things. Yep. Makes sense. So we made the decision, when we founded Fox five at the start of this year that we wanted to kind of carry on weathers in spirit. And you know, we use the we forked the code base originally, which means we took that code base, and we're basically saying, you know, where versus here, it's, it's the open source version of web, this is not really undergoing a lot of significant changes. Right now, most of the team that still works on that occurs is primarily focused on career specific additions that are not open source yet. But you know, we said we're gonna, we're gonna just, you know, we want to continue this in the spirit, but we're going to, it's going to become a different product over time. And we don't want to sort of create kind of confusion there. We just want to say, Hey, we're building Foxtrot. We're using some of the stuff that was done in the past. we're incorporating a whole bunch of Elon ideas, but we're not kind of being held back by what's of company technical. Yeah, exactly, like company interests, or just like, you know, technical limitations around how how projects are kind of tied to internal code.

Yep. And how did you decide to form a company around this as opposed to any of these Other things that you could have done like you could have just been a side project or like, decide to jump on and run? Yeah,

I guess? Yeah, that's a really good question. I think, I believe very strongly in the future of robotics. I think, you know, there's an enormous industry out there for our enormous opportunity out there for robotics to just completely change. What our economy looks like, what society looks like, I think, you know, I think people are massively under estimating how much robotics is going to impact the next 10 or 20 years of kind of human life. And I also really believe that high quality, you know, I love engineering, productivity, development, productivity, I believe that high quality tools can help people focus on doing what they do best. And I've seen, you know, I've seen that kurzen and other companies I've seen, really talented engineers are, like deep, deep domain experts struggle and spend 80% of their time fighting with something that they don't properly understand. Because it's not, it's not their expertise. And, you know, likewise, I guess that's where I always felt that I and others can kind of contribute, because, you know, I'm not, I'm not kind of, you know, controls PhD or machine learning PhD or anything like I don't claim to be a super. Right, but I can tell you how, like build systems work and data works and and systems brought together and visualization and how these kind of things work. And I really believe that kind of, you know, good quality tooling means that domain experts can all focus on kind of their, their area of expertise.

Gotcha. So you're basically making it easier for people to work on what they are good at.

Right, exactly. And yeah, and then so that, I guess, tying it back to your question was, you know, how did that leave the founding box live was that we were foxtel. If we wanted to, you know, visualization, that's where we're starting. But the ultimate goal of our slogan is to make robotics development easier, right? We want to be able to help roboticists focus on their domain problems, focus on on shipping products, and not worry about, you know, how they're thinking about their tools and their data and their development workflow and all of these things. So the visualization pace was our starting point, because I believe that there's just, you know, such big, low hanging fruit there. It's something that we selfie, so impactful at cruise, but you know, we have kind of a roadmap of other areas that we also really think we can help them.

Awesome. I'm interested to talk about that. But first, you also had a lot of several people go from cruise to working at Box club with you. Would you tell me a bit about that?

Yeah, so we, my co founder, and I both. So my co founder, Roman, he's our CTO, we have worked together for a long time, we actually worked together before careers, we worked together for years before careers, we work together at Coinbase. And that was where we met. So I was an early engineering hire, and the first head of engineering and Coinbase, back in back in the early days. And yeah, we met there. We worked together for a long time, he was actually the one that introduced me to careers and convince me to come there. And then so you know, we really wanted to do something together around robotics, developer tools, we got chatting to a few folks. Many, you know, several of our team members have come from careers. Mostly not directly, I guess, you know, like I said, it was sort of five years since I or more now since I started at Chris or Robin, and, you know, several of the other folks on our team had come from, had been at cruise for, you know, two or three years and then had gone to other companies, products like this. And then, you know, we kind of just reconnected with a bunch of people on our network and to see who would be interested.

Sorry, Siri popped up. Why would you say, for the last?

Yeah, just just that we, you know, all of those people had been that had been that curse. And then had gone on to work at other companies. And we just reconnected with, with some folks in our network to see her. You know, I was interested in doing something around this. Yeah.

And now, you said you strongly believe in the future of robotics. Would you just tell me a bit more about that, like, what is making you so optimistic about the future of robotics?

Yeah, I mean, it's, it's, it's inevitable, right? You see. I mean, and partly, maybe this comes from having worked their careers and seeing what is possible. I think, you know, a lot of people haven't really had that glimpse into what is possible, but when you do Sit in a car that's driving itself around a busy intersection with, you know, potentially hundreds of people walking around it and cyclists and cars and, you know, you see a car that can, I mean, I was, you know, six, seven years ago very skeptical about self driving cars. But you know, having having gone on scene and setting those cars and seeing the progress that's made and seeing things that, you know, you hear people while you're talking to people on the street that have not worked in the industry, and they say, Well, I don't think it's after because of course it won't because sometimes you just need to be able to understand you know, one of the other humans doing and then you see the team was just plugging away at problems like you know, a four way stop sign is a very common in America, we have we, you know, cars have to basically take turns and giving way to each other. And especially in San Francisco, you have to be quite assertive in a four way stop sign up to a stop sign. And you know, technically it's your right away if you got there first and someone else got there half a second later. But if you don't start pulling out into that intersection, you know, the other car is gonna is gonna go very quickly, right? So you look at these problems and you know, you start thinking about people signaling and things like that, but but he seems like chip away at these problems and get to a point where they've tuned the car where you know, as as the cars turns coming up, it has like a very human like behavior where it pulls forward slightly to kind of indicate its intention that it's going to go and that actually causes the other cars not to pull out because they they can kind of see your intention that you're planning on going and you know, these these things, they sound like very human behaviors, but you can actually just really get down into into kind of encoding and machine learning models or, or even you know, hard coded logic in some cases where the car is like exhibiting very human like Vega so yeah, it's, it's really interesting, you kind of see, see that evolve, and then you look around the world, and you just see how many things done, how many sort of, you know, tasks, or jobs are done by humans today that are both like a, like, very interesting, very repetitive, you know, couldn't be done better, more reliably cheaper by machines? And the kicker is that they're way easier than driving around San Francisco. Yeah, so like, you know, this is great course, you know, the future is here, it's just not on definitely, it's not evenly distributed, right? It's, it's like it's just inevitable to me that the launch of burgers is going to if you took all the technology that was currently being developed and self driving cars and apply that to, you know, warehouses, logistics, manufacturing, agriculture, shipping like this, you know, food prep for production, there's, there's so many areas where all of these environments are much more constrained than then, you know, driving around a city. And if you took 1000 engineers and apply that to that, there's no doubt that those loads will be solved. You know, very quickly, the thing is, right now, there's not that, I guess, you know, economic investment is not there to put 1000 engineers on a problem about self driving forklift or something. We're certainly getting there. But it's not at that scale yet. But you get this kind of flywheel effect where as more people start trying to solve those problems, better tools come along, better frameworks come along, better libraries come along, there's more off the shelf that people can use, you know, the flywheel start spinning, and then teams of 10 engineers can all of a sudden have the output of October 1000, you know, a decade earlier. So yeah, just, I mean, that's what makes me so I guess, so interested in optimistic about the future of robotics is just that, you know, it's, it's, it's going to have a massive impact. And I think that people are, I think it's really kind of reached the public consciousness yet, you know, everyone's talking about blockchains. And AI. You know, I think like, robotics is kind of the defining application of AI maybe, and people haven't really sort of grasp that yet, I think.

Do you? So you mentioned several, like, I don't know, domains, agriculture, manufacturing, etc. Are there any specific areas within those are any specific problems that you think are particularly interesting and ripe to be solved by robotics?

Yeah, yeah, I've seen some very interesting stuff happening. You know, and warehouses, like, Fetch robotics are doing really interesting stuff. They're obviously being on your podcast earlier. And saying, you know, I think all of these things where you have self driving forklifts, for example, a lot of this stuff, I mean, I've and you know, coming into the agriculture industry, for example, I grew up in New Zealand, and kind of very in a rural region that does a lot of fruit growing and picking, and I had, you know, jobs in high school where I was packing Kiwis and boxes, and I've got all sorts of things that don't pruning. And I've done a number of these tasks that kind of happened on the agriculture industry. And you know, I mean, some of the tasks that you do, I like, sit here and watch these fruit roll by and grab the ones that have blemishes and put them on, you know, grate them into like a BSE and grab the ones that have blemishes and put them onto this other conveyor belt. And it's just not a sensible task for a human to be doing in this day and age. Right. And I mean, maybe it was, you know, 15 years ago or something when I was doing it, but like, you know, computers with computer vision and like manipulation on some things can just do a much better job of this now, they're gonna be far more reliable, they can do it at a higher pace. And like, guess what, that wasn't a fun job for anyone, I certainly didn't enjoy it. Like, there's much like more productive things that as humans can be kind of putting our time and energy towards them, watching fruit roll by and picking out the ones that like, look strange.

Gotcha. So, and one other thing that kind of caught my ear, was you saying, putting 1000 engineers towards the problem? Can you just talk a bit about that, that it's an interesting idea to me. And it's very different than, say, like a startup thing, where you have the 10 engineers that you were mentioning, and building a product? Or just a couple?

Yeah, yeah, I guess. I would look at it from a perspective of what you know, in 1999, right. So you wanted to start a startup in 1999. And you're gonna build some web, you're gonna build Amazon or something, I guess they were a little bit earlier. Okay, you're gonna go to Amazon 97 or something, okay. You're going out and you're like, you know, you need some people to write code. But guess what, you also need some people to go and buy some servers and racks and servers, you need, you know, just like multiplies. You need people doing all that, especially as website starts getting traction, you're building everything from scratch, right? You know, when you say there's 1000 engineers working on a problem, it's not that they're all kind of in they're super productive writing the key kind of code day to day, it's that you have to build kind of all of the infrastructure of the pyramid underneath you as well, because there's nothing

because it doesn't exist off the shelf. Whereas if you're building PayPal or something, or Amazon, you can use react or some other framework that does a lot of the lifting for you.

Right, if you're doing this 20 years ago, you're building Google or Facebook or Amazon or Ebay or any of these things you're building everything from scratch, you're not you know, you're not just grabbing like a web framework like react or Ruby on Rails even know something you're kind of you're writing all of the all of the you know, went from scratch, you're writing all of the deployment from scratch, you're buying hardware, you're racking hardware, you're worrying about like network connections, you're making deals with ISP so that you get enough bandwidth to service like, it's crazy, man, it sounds crazy, right? But like, that's how one of these companies got started. And that's where we are in the robotics industry today, right? We're at this place where if you want to build a, you know, build a self driving car company, I mean, 90% of the engineering effort is not going to the core kind of, you know, the core kind of like algorithms and models that are being developed on the cards. That's all the stuff around that everything that's supporting and available off the shelf today. Yeah. So you know, if you if you talk that, you know, one approaches kind of brute force, which is like you take those 1000 engineers and say, Hey, you know, we're pivoting to like self driving forklifts now. Sure, like, maybe it's an easier problem, and all that talent could could get it done. But I think, you know, the, the more kind of scalable approach over time is that better and better quality tooling comes out. And you know, in the in the web industry, that was AWS coming out, that was Heroku coming out, that was data, dog, and Splunk, and New Relic and Sumo logic, and, you know, all of these kind of startups were founded in the latter half of 2010, latter half of the 2000s. where, you know, by 2015, you want to go and set a website up you just kind of like slap a few things together, and you know, two people or one person and you have a company. And, you know, I just I think robotics needs to get there. Obviously, it's, it's never going to be as easy as building a website and you have like hardware and the real world to deal with. But it could come a long way from where it is today in terms of like, having this reusable components that you can grab off the shelf glue together and have a company

where so in all of this building infrastructure, in your mind, where does open source?

Yeah, that's a really good question. Um, open source is has you know, it's both like, I guess very important. It's contributed a lot and, you know, we deeply believe in open source. I think the open source industry has matured a lot over the last kind of five or 10 years, especially even the last five years, and what business models work and what business models don't. You know, you've had some of these things recently, like Elasticsearch, MongoDB, a lot of these companies have kind of gone through a little bit of a crisis where they said, Hey, you know, we give everything away for free. And then someone like Amazon comes along and takes that thing that you're giving away for free and starts reselling it, and they've kind of like suck the air out of the fact that you actually storing it to pay employees. So I think, you know, we've, we're an advantage that, you know, sitting here in 2021, is a lot, a lot of evidence of what models work in open source and more models don't, I think, the fact remains that open source is a really good way to get products out there and make them easy and free for people to play with, it's a really good way to build a community around projects. It's also frankly, a lot easier to work with, you know, as a consumer of open source. Even if you're paying for support or services around something, if you if you get some you know, you're working with a tool, as a developer, you're working with a tool, and then it's not quite working for you, or you find a bug or you need a feature, it's so much easier if you can just like look at the code and fix the bug and build the feature. Yeah, rather than stuck in this like, yeah, you know, it's not gonna it's like crazy, where you've got to, like, go to their support, and, you know, like, turn into a ticket and goes to a pm somewhere and they farm it out to a team and you wait, like two months when you could have just changed a bit of code yourself? Yeah. So you know, yeah, open source is really important. I think we strive to, and we can talk a little bit about, you know, well, how we're thinking and where we're drawing our mind. But I think it's important to, you know, try and open source as much as you can and make that available to the community and build a community around it. But also, you know, you need to be realistic about the fact that you need to be able to pay your employees and you need to kind of fund the development and the other things that come around. And so it's important to have kind of clear delineation there, I guess.

Yeah. And so now, natural segue. What's your business model at cruise? How are you kind of balancing open source and paying your employees and yeah,

Yeah, yeah. Yeah, exactly. So we, yeah, I mean, in keeping with that, I think the the model, one open source model? And actually, you know, I think I was just listening to your podcast with Nick. And I think, I mean, I probably share some similar views on this, I think I don't like the open source model, where you kind of have an open source version, and then you have kind of a better version that you have to pay for, which is just, you know, more features around it. I think that that frustrates people want to, you know, it kind of feels like you're just kind of keeping the crown jewels for yourself, I think, you know, much better models that we're seeing around open source. There's, there's kind of two related models that work quite well. One is just that you have an open source product that people can run themselves, and then you just sell basically hosting of that, of that thing. So you know, that's where you see a lot of companies kind of doing this today. That's, you know, what, like Elasticsearch and MongoDB. And some of these people do, you also see it with, like, next js, and wsl, they kind of offer like, a paid hosting. So that's a front end framework, web development framework and a web hosting company, and they kind of they sponsor the thing. So that's one option, as you know, kind of you build a thing, which is completely free, and then you have the hosting of it. The other option, and you know, where we're going as you build a thing that's completely free, you get all the features button, this kind of add on services that you know, you need if and I guess more specifically, in our case, we give you Foxconn studio, it's free, it does everything you want, you can run it on your computer, you can run it on the web browser, there's no limitations. If you want our services for collaboration, and for kind of like team communication, which we're building around that, then those are things that you need to log in for, and those are automatically going to be paid features. So we actually, I guess, you know, sort of pre announcing this a little bit, we'll be coming out with more than a couple of weeks. But we have some features in foxglove, where today, you can kind of have these layouts that you have personally that live on your computer, you can set up different layouts that you might use for different tasks when you're debugging a particular type of event. And we are coming out very soon with a couple of things. One is the ability to kind of share these layouts among team members. So being able to actually like login, starting to get into this world of where we're sharing views among among your team and making it easier for people to collaborate over robots data, robotics data. And then the other thing that we're building which will be kind of, you know, hosted service that you can work with is actually that data storage elements or, you know, how can we be the central repository if we get data so that you know, as I said earlier on so that you can just click on a link can immediately start looking at it without having to go through that whole rigmarole of downloading 100 gig bag for your computer and sharing it to your friend on a hard drive. And, you know, running any of these things locally, you can just have a kind of central repository for your data, click on a link and immediately like jump into it.

Now I'm struck by so you were saying you and Dave are sharing Dave Coleman from move it or picnic are kind of sharing a mind for these things. I'm struck by you guys both calling your service with something studio. So your Fox Studios movie studio, their new release? Why did you pick the name studio? What's the What's there?

Yeah, yeah. And I mean, since creating foxglove Studio, I think maybe it's just, I've started seeing it everywhere. I think, you know, we're not the only two companies. So far five other companies with like, you know, XYZ company name and then XYZ studio and

Yeah, I mean, I think it's it's, it's, it's intended to be just, you know, a product that you can jump into, you know, you can you can see what's happening, I guess we wanted to separate that product slightly from the company name, because, you know, as I kind of alluded to it, we're planning to bring on some of these other services and things. So we needed something to differentiate differentiate. Yeah, exactly. This is our, our kind of visualization tool that you can see and open and probably picnic, maybe a movie felt the same way. Yeah. But yeah, I guess, you know, and it starts becoming a little bit of a mouthful, if you have another studio releases, like an AC word that makes sense. But if you come up with maybe, you know, you already have a company name, and then you forget something else, rather long winded on the end of the year, your product name, you know, you get these things like hashey, Corp terraform. And you're like,

what is that? Yeah, yeah. And speaking of What's in a name, and although it's a bit of a digression, I'd like to get back to more of foxglove Studio, but how did you come up with a name? foxglove?

Yeah, yeah, this is a question I get a few times. So I have, you know, my philosophy on naming is that I think that if I say the name to you, you should be able to write it down. And if I see it written down, or if you see it written down, you should know how to say it. And I get kind of frustrated, but some of these company names where, you know, they're either just like, insane, not even company just project names. And then you have like, Kubernetes, for example, everyone goes around saying Kubernetes like, it's an easy thing to say. Yeah.

It's like some Starcraft thing or something. All right.

That's a Greek god reference design. Yes. I

mean, it just sounds like some futuristic device. Yeah,

yeah. Yeah, that's sorry. And I have this predisposition towards names that are easy to say, and easy to easy to spell and easy to read. But of course, you know, most of us are taken. So it's been a long time with this kind of short list of names that I was going through. And I was last year, just traveling around New Zealand. And I was doing a bit of a road trip with my wife and child and this kind of foxglove flowers are all over the side of the road. Did the song flower I have in the background? So it was cool. Yeah, just just a word. And I thought that's a cool word. And you know, it wasn't already taken by a startup. So they have no robotics reference. I'm sorry.

I didn't know. I just had no idea where it came from. So it's a flower in the hills of New Zealand somewhere. Yeah, that sounds lovely. Yeah. Okay. So if I understand correctly, so foxglove Studio, the. So you have all of foxglove, where you can visualize the data you see, and it's for live playing of ROS 2 data and also playing from a bag file

playback. Exactly. So you can connect to a live system, or you can just drag it back file and immediately stop looking at that bag file. You don't need ROS installed or anything. If you have a bag fine, you can just open up.

And then the studio. What are the kind of killer features that you're adding with this? So it's layout support. So you can save a layout, eventually, you can share a layout. And it's data storage coming soon.

Yeah, these are the these are the paid features that we're building around it, basically. So anything that involves kind of collaborating on your robotics data, where you have multiple team members, again, maybe you have a team of five, or maybe you have a team of 20, maybe you have a team of 100. But the larger your team grows, the more important it is that that you can collaborate over that data. This is I think people naturally understand this because, you know, normally if you have data stored in data dog or Splunk, or Google Analytics, the thing is, it's kind of sad. In nature that you can just look around a website and then copy a link, if you if you want to send something to your friend, you just copy a link and paste in slack and they can start looking at it. And so, you know, this kind of collaboration around robotics data, I think is something that that just doesn't exist in the industry today, if the kind of, you know, it's very much a single focus on just, you know, like, hair is my personal workflow, I have some data on my computer, or I have a bunch of tools locally on my computer. And if my friend wants to look at it, then he's gonna, you know, they're kind of out of luck,

or you have to write or read me up, and then they can try to reproduce what you do, right? Because the Yeah, exactly 10 steps are more

or less file, you know, these topics kind of that, you know, it's just, it's a very, sort of add surface process.

Gotcha. Now, I want to segue into talking about the web and connecting it to robotics, in some capacity. Like, I'm very excited about this, but would you speak on it? Just playing Yeah. What do you think the potential is there? Why the web will be good. You mentioned it a bit before, and that there's a lot of people that are knowledgeable of how to do web programming. Yeah. But anything else? Yeah,

yeah, I think I mean, this is something that a lot of companies independently coming to this conclusion, the web is kind of the main platform that anything that was built on these days, and you know, that can mean developer tools like elsewhere, you're kind of diving into specific instances. And since of your robot, it can mean kind of Fleet operations tooling, we have companies like forment.io, and freedom robotics, and all that where you know, you have a web based interface where you can see you know, your whole fleet of robots and see what's going on with any of them. It can also mean things like tele operations and remote assistance, where you need your robot to kind of call out and get human support and a human eyes on things. It can even mean you know, and web based labeling tools, for example, where where people are annotating data to use for machine learning training, those tools are again, primarily web based. And it's just because you have this platform where you can build, you know, visual interfaces, and they're inherently plugged into remote data stores, it's easy to save, collaborate, it's kind of you know, a similar transition from having Microsoft Word installed on your computer, to Google Docs, right? And it's just, you know, now you're moving to something that is inherently collaborative. And so, you know, when we talk about like the web and robots, I think, we're not really advocating that your robot is is you know, written in JavaScript or something like that, although it's perfectly possible if you want to go that route. But um, but you know, the robot is the robot, usually it's, it's, you know, it's running ROS or something similar. It's written in Python or c++ or something similar, but it needs to kind of communicate with that outside world. And so that means often use web based technologies for connecting from the robot to services, for example, like WebSockets are a really good way if a robot communicating with a server or browser communicating with a server. You also have technologies like web RTC, which is a way for men, that's probably how our video is getting to each other right now, it's kind of a way of streaming videos and things like that across the internet. So you have a lot of these kind of web based technologies, and they play really well with robots, they're really good for, you know, communicating with fleets, they're really good with communicating, you know, even to local devices. And it's really good for building interfaces where, you know, you don't have to worry about which browser or platform on things you're running, you can just kind of, you know, you can just open it up, and it works.

And do you think, can you talk a little bit more about the benefit of making it so that a lot of robotics related things are made in a way that like getting us a bigger pool of programmers to work on robotics problems through using web technologies?

Yeah, yeah, I think, I mean, that is something that we that always I found interesting and Kurtis actually was as we were hiring people, candidates vastly underestimated the need for robotics experienced and work or self driving experience to work on a self driving car company. And kind of, you know, am I talking about the thing with the 1000 engineers and how, you know, 80% of them are working on you know, the the permit that kind of supports the actual key algorithm development. We needed to hire a lot of people at careers that were just experts in infrastructure and data processing and data management and and, you know, web development, front end development JavaScript engineers, and we didn't need any of those people to have, you know, a PhD in machine learning or robotics or anything like that. But there's kind of this perception out there that like, I'm not a robotics developer, I can go and work at a self driving car company. But, you know, like I said, I think there's this, there's, you know, there's a large number of candidates out, there are just, you know, people AI engineers out there that work for companies doing all sorts of stuff in the Silicon Valley today. And around the world, and a lot of that stuff is, you know, much less meaningful, there's a lot of people working in AD tech and things like that, that, you know, could be working on our bonds, but maybe things that you think that they can't, so I would implore those people to, you know, have a look at some other really awesome stuff that is happening out there. And realize that one of the most exciting things to me about robotics is just how diverse of a space it is, how many different skill sets you need, and you know, you can come into it, and maybe you're experienced in front end development, or maybe you're experienced in DevOps. And you'll sit down with people who, you know, like I said, machine learning PhDs or something, but you sit down with people who are physicists, and people who are working on, you know, hardware, and you sit down with electrical engineers, and all of these people need to kind of come together and, you know, it's, it's, it's really important to bring all of those skill sets together, and robotics and I think, yeah, I don't know, I think there's like, a gigantic, huge opportunity there as more of the maybe for a look for for a long time, robotics is kind of being a little bit off in a corner. You know, a lot of the industry has been building, you know, ad tech, and, you know, consumer startups and even enterprise startups and a lot of stuff for processing enterprise data and a lot of things for for, you know, social media and social networking and things. But robotics is kind of spent a long time and a slightly academic corner of the world, I guess, you know, people have been working on robots, for, you know, decades to this point. And certainly the last 15 years, there's been a lot of really interesting stuff happening. But you know, it hasn't had as much kind of attention, I guess, as like the as the wider startup industry. And I think we're kind of reaching a tipping point now and more and more engineers coming over who have experienced and maybe more traditional software development, maybe they don't understand the robotics spaces, but they can, you know, they can sit down on paper with the people who really understand, you know, control theory or, or machine learning or planning or things like this, and you put that together, you get some really high quality software.

1 post with the visualization tag