Summation with Auren Hoffman
Non-obvious ideas that move the world. Auren Hoffman hosts leaders across tech, business, markets, and government.
Summation is the permanent home for the relentlessly curious.
Auren is CEO of NQB8, GP at Flex Capital, Chairman of Dialog; former CEO of SafeGraph and LiveRamp (NYSE: RAMP).
Summation with Auren Hoffman
CodeRabbit CEO Harjot Gill on software's new bottleneck and zero to $50m ARR in 2 years
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Harjot Gill is the co-founder and CEO of CodeRabbit, the AI code review company that became the most installed AI app on GitHub and GitLab. Over about two years, CodeRabbit went from zero to over $50M in ARR and a $1B+ valuation. It is his third startup.
In this episode of Summation, Harjot and Auren discuss:
- Why the bottleneck in software just moved from writing code to reviewing it
- Why mature companies now adopt AI first
- Why you should never let a coding agent review its own work
- The playbook that took CodeRabbit from zero to $50M+ in two years
You can find Auren Hoffman on X at @auren and Harjot Gill on X at @harjotsgill
What A is doing is like taking the best practices it has learned to an average person. For example, I can do marketing. I may not be the best marketer, but it does level the playing field for me. So we're seeing that imbalance where sometimes you'll see that people are being held back because they're too afraid to let go.
SPEAKER_00Right now, I would say is the best time I've seen in the history to do startups and create value.
SPEAKER_02Hello, fellow data nerds. My guest today is Harja Gill. Harjett is the co-founder and CEO of Code Rabbit, the AI code review company that became the most installed AI app on GitHub and GitLab. Over the last two years, they've gone from zero to over 50 million ARR and over a billion valuation. Harja, welcome to Sumation.
SPEAKER_01Yeah, thanks for having me here on. Really excited to be on the show.
SPEAKER_02Really excited. Now just a full disclosure: Flex is an investor in CodeRabbit. We invested in the seed. We invested in the A, we invested in the B. We're about to invest in the C. We love CodeRabbit. So that's the full, you know, disclosure before we get going on the show. Now, AI is writing more code every month. And I think the bet that you were making is that the bottleneck is going to move from like writing to reviewing. How has that played out?
SPEAKER_01And it played out even in more interesting ways than we imagined it would play out like two years back, especially with the capabilities of these agents improving and leapfrogging every few months. Now we're seeing like these agents are becoming more capable, able to run for longer periods of time. And recently also we are now seeing like the adoption of background agents take off. As you can imagine, like the code is getting generated. Not just from CLI and your code editors. It's being also being generated by linear. Yesterday Notion announced that they will be writing code. Datadog wants to write code, post hoc wants to write code. Almost like the floodgates opened and now you have this deluge of pull requests that are getting created, right? So the human attention is now all about how do I validate and how do I decide and build trust in whether the changes that we're seeing from these coding agents can be shipped safely or not. So yeah, I mean it's pretty interesting. Like, I mean, we when we started that time was just AI co-pilots. You were like doing tab completion, the GitHub co-pilot cursor came in a little bit around the same time as Code Rabbit started. So the velocity was very different, and the code review was still like very much human in the loop. Like humans were reading Code Rabbit comments and going and addressing those comments. But increasingly, what we are seeing is now we are part of the loop, right? So it's just agents reading the comments from this adversarial review that we provide and get to a really good implementation before some things get shipped.
SPEAKER_02And it does seem like even most organizations that are of any size now, and once once you get past maybe 10, 20 engineers, they're using many different systems to write their code, right? They've got codecs, they've got Claude Code, they've got uh cursor, they've got many, many other systems that are, and as you mentioned, you know, Notion, now all these other folks are helping them write code. And so there isn't like one system that is like the main input to their code. Is that correct?
SPEAKER_01That's right. I mean, this has become more of like a taste thing, just like every other tool in the developers that have in the past, as you know, like it was about Vim versus Emacs, a lot of the editor wars. Now we are seeing the harness wars or the model wars, like you have cults forming, and it's really about the the choice of the model. Like people have built fondness to opus versus pragmatic personality of GPD 5.6 or 5.5. And then it's about the surface and the harness where you want to consume these models. So it can be in the IDE, that's still like where cursor is still very popular as an IDE solution. Then in the terminal, you have cloud code CLI. And now very recently we have seen uh a lot of developers, including myself, use desktop applications where Codex Desktop app is kind of ahead of the Mac. So yeah, so it's kind of like what the organizations are now doing is like giving developers freedom on which model, harness, surface they want to consume these agents. Um, and then when it comes to code reviews, that's where you want to centralize as the guardrail. Um, while you have proliferation of these coding agents, everything has to go through a single choke point, which is now code rabbit and your GitHub and and so on. Yeah.
SPEAKER_02I personally like when I even when I'm in the same repo with the, you know, let on my GitHub, I will sometimes use Codex and sometimes use Claude Code for different reasons. And I don't know, sometimes I get excited about one versus the other, or it could be like so one week I'm using one, the next week I'm using the other. And so even like one person may have like, in some ways, weird personalities and move around different systems.
SPEAKER_01No, that's interesting. I we have seen that with our team as well. Like, for example, like when it comes to chick chat and like planning, like you will find cloud code models to be more uh curious and guide you in the right direction. Whereas uh the open air models are more like you just tell them what to do and then they'll no questions asked. They will just go and do it, right? What you ask. So yeah, I think people are also like seeing mixing the coding agents for various reasons because the capabilities do differ across different model providers, the personalities of these tools.
SPEAKER_02Or you just run out of like credits in one and you switch to the other for the rest of the week, and then you jump to the other, and it's like you know, and it's like and they you're kind of like in there or doing it.
SPEAKER_01That's right. I mean, right now we are also living in the time when OpenAI entropic are kind of testing the market on these hundred, two hundred dollar plans, which they feel that would be the most stable pricing point for these coding agents. And currently they're subsidizing, so of course there are like some rate limits, but as you know, open AI keeps resetting and and so on. And at the same time, the once you're hooked to these coding agents, like there's a lot of anxiety, like a lot of the developers are feeling so empowered that the first thing they're doing waking up is like looking at their coding agent, and that's the last thing they do when they're going to sleep. They're just keeping these systems busy. Um and uh and and that's where like a lot of people are like switching harnesses and buying multiple of these like $100, $200 plans.
SPEAKER_02Yeah, yeah. Yeah, I personally have two $200 plans going. I mean, probably maybe it may add a third one at some point. Like, how does like organization think of it? Because like I imagine with all these people committing, all these people doing, you know, committing different types of code, they're they want to have some sort of better, some more standard review process, right? They would then they and each organization may want to have some sort of centralized decision of how they should do reviews, what's important to do reviews, how much should they think about security versus speed versus like how does how does one think about like the review process as an organization?
SPEAKER_01Yeah, I think people are looking at like SDLC as a whole. Like, if you look at the software development lifecycle, like pre-genai, there was a very distinct planning phase because downstream your engineering resources were very expensive, so you needed to be sure that what they were building or working on at the same at any time was the right thing to work on. That's where you had linear, Jira, Notion, bunch of these planning tools come in, right? And downstream, then you had code reviews because you had the these are the outer loops, right? So they were always collaborative. So your planning was collaborative, then the developers will work on their machine, and then the reviews were always collaborative, right? So as soon as the middle inner loop became fully automated, now we can go from a prompt to an implementation which is thousands of lines within like a few minutes or hours, right? The collaboration has broken now, right? So agents are good at writing code, but um the collaboration is what suffers, right? So you have now processes broken upstream and downstream. Planning is being bypassed increasingly. So there is no reason to track something when you can just go from an idea to implementation without going through linear, and that's why linear is also saying issue systems are dead. They wrote a blog about it and they are like going downstream into code reviews themselves. And code reviews become the only choke point because really the floodgates have opened. There's no process or choke point upstream. You can't really tell these agents to get an approval before opening a PR. I mean, that work stream, that system of plans or approvals does not exist today, right? Git is the only approval-based system that we have, the workflow around that, the pull request model, right? So naturally, like everything's now hitting GitHub, and no wonder GitHub is failing to scale as well, just the PR volume has increased so much. So it's very interesting. Like the review process, when companies look at it, they clearly see that becoming a pain point. Even pre-Gen AI, it was a pain point. Around 20 to 25% time, every software engineer, like especially the senior talent, would spend on code reviews. Now it's even more than that, much more. And in fact, to a point, I'll tell you like when we enabled background agents in our code base, like a few months back, we launched a Slack agent where which could respond to Datadog alerts or support tickets. We have over 1,000 pull requests open in GitHub right now, even though we are like automating the reviews, but even deciding what to ship, what to spend time on. Like processes are broken, our CI CD was broken. So it's not just the review, even like downstream, how do you build, validate? So a lot of things suddenly became um uh hit the breaking point as the middle loop in the loop got automated.
SPEAKER_02I built this like super silly app for myself. And even when I built this app, I had I I had to use, I use Claude Code, then I use Codex, I had GitHub, I had Superbase, I had Vercell, I had Modal, I had Resend, I ended up using Code Rabbit as well, I had a bunch of other tools just to build this like kind of a silly app there. Like, how do you think about going across all of these different things? Because it's similar, it's like, yeah, there's the code, which might sit on GitHub, but it's interacting with Vercel. And you have to understand what Vercel is, and then it's interacting with Superbase, and you have to understand, okay, well, what is the the good, you have to understand, like, okay, how does this work with Superbase? Because it might not work with something else. And then it's interacting with modal, which has this like. So, how do you think about like these like tool ecosystem that that kind of interacts with the code?
SPEAKER_01No, that's a great question. In fact, now with the agents, you can pretty much automate a lot of the DevOps and operational PCs as well. And it's really now a matter of bringing the right tools to the agents. It's about the surface area they can access, right? And right now, if you look at the model capabilities, they're right there, but the products that capture the capabilities are still lagging behind, right? To the point that it's really the model labs pushing some of those capabilities by adding plugins in cloud code and so on.
SPEAKER_02Yeah, sometimes it's like, okay, we're stopped here. You have to go into Superbase yourself and then you know put this little API key in there, go into Vercell, and it's like, oh great, like I didn't know how to go log into UI to go do this.
SPEAKER_01Yeah, providing access to these agents is the challenge right now that everyone's trying to solve in many, many ways. Right now, most of the solutions, they are the reason they're successful locally is because developers are able to do a local OAuth, run an MCP or a plug-in, and provide their own credentials, right? And which means that if you have like a company of 100 engineers, roughly like, I don't know, 90% are probably not even connecting the right tools and not seeing the the fullest extent the agents are how successful they are in accessing Versell or GCP and other tools, right? And that's where the enterprises are still like lagging behind in these capabilities because you don't have the infrastructure. Um, for example, you need sandboxes, and that's not enough. You don't want the agents to see those API keys. So you're gonna add some sort of like a networking proxy layer, like credentials proxy, right? So people are trying to solve that, like what the agent stack look like, the agent cloud for the background agents. And then the other question is how do you convince IT to bring differential access? Like who, when the when, like depending on which environment the agent is running, or for instance, which Slack channel, like what kind of credentials you want to bring to the agent, right? Um, so all those problems are still unsolved. Like we're very early, as I said, the models are way there, but the products that are really using these capabilities are still lagging behind. So a lot of opportunity right now.
SPEAKER_02Yeah, and it's one thing if like you're like some people listening to this, they're like individual creators. Um, and then you can kind of move fast. But once you start to become a part of a larger organization, like the complexity goes up exponentially right now.
SPEAKER_01That's right. The individuals are where like we're seeing the biggest, like the AI-pilled audience, like top commuters and all. And the enterprises feel like constrained on a couple of things. One is like the token budgets, like because most of these tools are successful locally. One of the things they don't have is auditability. Are these tokens being spent on their own projects or are people doing their own side projects? You don't have any idea, right? Oh, interesting.
SPEAKER_02I didn't realize that. Okay.
SPEAKER_01Like, yeah, yeah. I mean, for a startup like us, it's okay. Like, I mean, but for larger companies, they have to think about if some developers are spending like 5,000 bucks a month.
SPEAKER_02Right, what are they spending it on, right?
SPEAKER_01Are we even seeing the impact, or are they just doing some side projects and because it's so easy to do to do build anything now, right? You could be building your home home automation system for God's sake, like anything.
SPEAKER_02Totally, which which we've all done, right?
SPEAKER_01So, yeah, so so that's that's the biggest thing right now. Like a lot of the success has been like with the prosumer audience, like a lot of individuals and running ahead with like this hundred, two hundred order plans. Um, but the enterprise is still like the problem has not been fully solved in terms of the cost visibility and how do I operationalize the agents in terms of security. So a lot of gaps are still there. That's why the background agents have been a hit and a miss. People have been trying those ideas for the last uh more than a year now, but we haven't seen like big success in moving the token spent uh to cloud and the background agents.
SPEAKER_02Okay, there's like code review and then there's like security review, security vulnerabilities. How related are they?
SPEAKER_01I would say that two sides of the same coin. Like during code review, you're looking at um your best practices, your whatever the rails you have, and not and and of course security as well. You want to understand whether you are going to have PII leakage because these agents do make um a lot of mistakes that do compromise the security posture, right? When we look at security review, like you have had these like tools of the past um around um SCA and SBOM tools, a lot of like supply chain analysis, a lot of like dynamic tools as well, static analyzer like SemGrep and all. So those are getting supplanted with now a lot of AI-based analysis. So you could really reduce the false positives because that's been one of the biggest complaints with the SNCC and SAMGREPs of the world is like a lot of false positives. So people have, I don't know, thousands of findings in a mature repository, but they don't know what to prioritize or which ones are even real threats. Now with AI, some of that has been accelerated in doing a security review, and it's also like all almost like an arms race, which is very interesting. Now you're seeing really capable models are able to also find vulnerabilities, right? That's why the government's trying to regulate some of these like Mythos class and the bigger models because they're so good in finding zero-day problems. At the same time, your review has to catch up. It's like an arms race. I mean, you you will have to fight fire with fire, you have to now do security review with another AI. Um, so it's pretty exciting time. I think reviews and security reviews are really pretty much both sides of the same coin. It's really about the latency at the at this point. Like, yes, you want like a fast review on a PR that can finish in 10 minutes, but if it's an important feature, maybe you want to run a thorough security review that can take half an hour. It's just about spending time and more tokens in making sure that they're like uh no threats uh that can get in.
SPEAKER_02And there's many different types of reviews because like you could even have like an agent like use, like if they're using the product, right? And trying to do prompt injection or trying to do other types of things to really try to jailbreak the product as well.
SPEAKER_01Yeah, that's right. I mean, the if you're building an AA product, um securing them is also like challenging given how non-deterministic those systems are. So you could casual an agent, talk to it nicely, and it will give you spill all its secrets, including I don't know, it's system prompt or whatever credentials you put in. That's why it's not safe to add your credentials. And people have had so much problem with open claw, for example, right? I mean, it's great as in hobbyist, but if you're like really a serious enterprise, you have to take those concerns very seriously around guardrails. And um, I think we're like in very early. I mean, a lot of the products of the future are going to focus on how do we even like pick up the agents' reasoning traces and do analysis on them. Like, did these agents did they did these sessions got compromised in some way? So there's going to be like a lot of new startups um uh which we'll be tackling um say especially the agentic security. It's not about like a single prompt anymore, it's about the trajectory now, like even the evals and security looks very different now in the new world.
SPEAKER_02What can a human do in code review that we cannot do with an agent yet?
SPEAKER_01Right. So it's very interesting, right? I mean, if you look at the code reviews, um, they have certainly evolved than how they were two years back. So the type of mistakes an AI agent makes is not the same type of mistake a human makes. Right? So you're not looking at issues which are simple try-catch misses or some error handling is not there. Like the code is better than what most humans will write. Right? If you look at that code in isolation, now if you zoom out and look at the bigger picture, I mean, is this pull request something that is should be merged or not merged? Which is like, is it adding to the entropy of the code base? Is it reusing the right patterns? Is it um because in isolation it looks perfect, but half the time, I mean, you will see that it's like leaving some shadow paths behind. It's like changing the core flows of your product and so on.
SPEAKER_02This chain that you're adding could make things slower or could add, you know, your memory leaks and all these other issues that could happen, right?
SPEAKER_01Right. Because when agents are generating code, they have a very similar focus, taking the user's prompt and going all the way. And depending on how assertively you prompt the agent, they will just go out and do that thing, even whether it's the right thing or not. Now, as an enterprise, as a company, like I mean, you have a team of developers, some of them are good with prompting, so they don't like push the agents too much, but sometimes you will like force the agents to do something so unnatural that you have to detect that in the adversarial review against whatever your hard invariants are and so on. Are they getting violated, right? So the nature of review is definitely of human review, is moving away from line by line of code reviews because that's where AI is already doing a better job than humans, to higher order reviews. So you're looking at higher abstractions in understanding the architecture, understanding the other sort of blast radius impact of this change and so on. Yeah, one thing is sure the review is not gonna go away. Like, I mean, your guard deals never go away. For example, like Datadog is one example. Your cloud platforms are becoming better and better with time. Your Kubernetes makes sure you don't crash, but doesn't mean you don't you switch off your monitoring, right? So the so the review is gonna evolve, but it's never gonna go away.
SPEAKER_02Now, Cursor bought a code review company, but for whatever reason that hasn't like really taken off. Like, why why why has that not taken off?
SPEAKER_01That's a great question. So, so yeah, I mean they exited because also like because Code Rabbit was like still like a better product in many ways. And also that company started pre-gen AI and with a different philosophy that if you look at the time before pre-gen AI, humans were still reviewing, but the review was still a bottleneck. It's not like it wasn't a bottleneck.
SPEAKER_02Yeah, it's been a bottleneck my whole life. Exactly. Sometimes you're waiting and you're like, I wanna I wanna push this, and you're like, you know, you've been you're waiting for your colleague to take a look at it, but your colleague's on vacation or something, or you know, whatever it might be.
SPEAKER_01Right. So for that company, like the focus has been around graphite, right? So the focus had been around stacked pull requests. So the idea is like if the humans were in the review path, they can only review if the changes are small and byte-sized. So the idea is like you checkpoint your big feature into smaller pull requests or stack of commits that you can review one at a time and then stream all of them together, right? And the then the onus is on the author of a pull request to actually checkpoint the commits that way, to create a stack, which is meaningful enough for the reviewer to understand. For instance, the first commit of a stack is you made a schema change. The second commit could be you made um some business logic around it, then you add a API route. So you can actually stack your changes as you're building software. But that was like the idea, like pre gen AI, right? Now, with AI, you can't really you can ask the AI to stack, but usually it makes no sense. Like you can just ask it to go build a feature and let the reviewer figure out the right way of reviewing these changes. So that's where the disruption happened, like the techniques and the tools that we had designed around managing code changes. Before Gen AI looked very different with Gen AI.
SPEAKER_02You think like maybe there's a way of like integrating the code review into your wherever you're storing your repo, GitHub or whatever it might be. Like you would think like somehow like the closer the integration there would be better, but for whatever reason, like GitHub doesn't really have a good solution either. Like, how do how do you see like the world evolving when it comes to code review?
SPEAKER_01Right. I mean, this is a space which is a big addressable market. So platforms like GitHub will always compete with every other player in the market as well. I mean, it's almost like how Amazon was AWS, right? They had Cloudwash, but then still Datadog became very successful. So the way we say this market is there's going to be best of breed products like Code Rabbit, which will because a lot of companies care about best of breed. Like if there's some CFOs and say, okay, we anyways have a big contract with Microsoft, let's just buy everything from one place. But there are a lot of companies that go and buy best of breed solutions, like they'll buy Zoom, they'll buy Slack, right? And they're buying Code Rabbit, right? Yep. Um, so so that that's what we're seeing. I think this is one of those times where uh a lot of the VCs are also a little bit uh anxious about model labs going and winning everything, where we see kind of the world in a different way. We see there's a huge opportunity to solve problems with custom harnesses. Like code review is one such use case where Code Rabbit has built a custom harness which is like gives a best price to performance right now than anyone else in the industry. And that resonates very well. And that's why our like growth has been uh pretty much unobstructed despite GitHub competing with us over the last two years and so on.
SPEAKER_02You have a very interesting lens because you're seeing code across lots of organizations and helping fix code across lots of organizations. Is there some kind of like in some ways like a co-op that you can that could happen where okay, if you can fix one in one organization, that can help you fix another organization? Like, does that somehow make your model better? Or is it just so proprietary that you can't actually use that data to the next one?
SPEAKER_01Yeah, some of the learnings across orgs do carry forward. For instance, well, the reason why we have been further ahead of everyone is because we have a large install base in open source. And we every few seconds there's a pull request getting reviewed in open source. In fact, yesterday I was looking at like Kubernetes community sent out a newsletter saying that they love Code Rabbit, right? So we are in like very popular mission critical repositories right now, and that's like actually have uh help um acting as like a feedback loop.
SPEAKER_02Yeah, that's great because you can use that, you're allowed to use it, and you don't have to, it's not proprietary. Yeah, okay.
SPEAKER_01That's right. We are in a very unique position compared to a lot of the AA products out there which work in a more private setting, so you can't really train on their data in our case. We have been taking all the learnings we have from those runs, which are all in the sandbox, there is agent trajectories involved, and taking it back into improving the hardness, improving the context assembly, and even like choosing the right models that we want to use. Like we have so much data coming in that we are able to pick the right cocktail. Um, so we're not a product where we let the user pick a model from a drop-down. We we come up with the right cocktail of agent and subagents and different other models, right? And that's proved very valuable. So we're taking the learnings across these projects and helping everyone else out there. At the same time, the product also builds proprietary learnings within an organization. So there's still like a lot of tribal knowledge that does not translate well across organizations because sometimes people do things which are counterintuitive. I mean, the AI agent will say, okay, this is the best practice, but then someone will go and correct Code Rabbit. No, this is not how we do things. We are not your average data set, we are different, right? And Code Rabbit will remember that fact and then fine-tune the reviews based on those learnings for that organization.
SPEAKER_02Okay, so when when we met you roughly two years ago and when you did your seed, okay, it was clear like you had a cool product, you guys were like super technical, you're like really into the you're into the product. What we didn't know then is also you guys are like masters of go to market and marketing um as well. Like, walk me through like the marketing playbook that you guys have used over the last couple of years.
SPEAKER_01This is my third startup, first of all. So and go to market across all my startups has looked very different. Like my first startup was mostly in the cloud infrastructure space, so right when Kubernetes was taking off. So the GTM there was very content focused. Like getting into hacker news led to and led to a lot of like signups and meetings. We were mostly SDR focused in my first startup. This one has been all uh word-of-mouth, product-led growth company, right? And we kind of ground up, built that intentionally because this is one of those markets where we had seen previous successes where Cynic, for instance, is one company we took a leaf out of their playbook a little bit by making the product free for open source. And there was another reason when we started back in 2023, those were very early days of AI. And a lot of people didn't even have this new habit of using AI. Right? So we wanted to make sure that uh light as many fires as we can, like get the product out there so that people build this new habit. Right? That was a biggest battleground more than pricing and other things. It was like, would people accept or reject AI in in pull request workflow, which is very serious, right? Um, so one of the strategies we did there was to open up the product for open source for a variety of reasons. One is wanted to show people that how effective this product is, like seeing is believing. So those kinds of products, you can't just put a message on website and make promises. You have to really show people, right? And open source was the best showcase we could find. Second is open source maintainers actually had a pain point. I mean, there were a lot of like contributions from unknown people, and this was like a thankless job. So they were already struggling with code reviews. And thirdly, we knew that the open source adoption is going to set the flywheel for constant product improvement. So we kind of like hit two or three birds with one stone.
SPEAKER_02Right. You're making your marketing and you're making your product better on one big thing.
SPEAKER_01That's right. And these open source projects were like almost like influencers. These maintainers are influencers, they have a lot of weight in on Twitter and other places. And so one door led to another, and we got like viral adoption in open source. We also had a lot of viral adoption in the early days in Japan, very interestingly. So we had no boots on ground, but we had I don't know 30 blog posts written by end users about how good the solution is.
SPEAKER_02Yeah, we noticed that too. Like, why why did the Japan thing take off the way it did?
SPEAKER_01That we will never find out. Like, I mean, we don't it just got viral independently. We had no hand in that, it just went viral on its own, while it also went viral in the US, thanks to open source. And it was also like I think the time in the market, like we were early movers. Uh, people, it was AI was a novelty, so a lot of people were curious. At the same time, the product lived up to its promise. It wasn't just slop, it was actually adding real value. So a lot of things came together, and even on the marketing, we were like very influencer focused. So we would work with top content creators to talk about Code Rabbit. We had like, for example, fire ship video um placements and all. So that always led to like signups and awareness. Uh we have billboards, like you have seen SF, like we have like we did the station takeovers. Um, so all those things helped a lot.
SPEAKER_02I was in uh the Amtrak station in New York, taking a train to DC, and the whole station was was wrapped in code rabbit. And I'm like, wait, why, why it like are there even that many developers? Like, what was your reasoning to do something like that?
SPEAKER_01Yeah, it's about the mind share. I think we just wanted to be in the series where we think there's like developer count um population. We did one in Austin, I believe. We did in Bangalore, we are running ads, we're running in in the Bay Area, of course. Um, South Bay SF. Um, and we know that the pain point is so prevalent. Like, this is a prosumer product. So if you look at our customer base, it's millions of developers that we want to reach out to.
SPEAKER_02Yeah, you even have a free price. It starts, you know, you can have a freemium version, and then people like upgrade to uh a little bit more, and then you can upgrade to a little bit more, and then all of a sudden, like the whole organization starts using it. Is that that's kind of typically the way it works? Like, like one developer starts using it, they tell another one inside, they tell a few more, then you've got a pod of 10 and then kind of spreads, and then eventually you do like an enterprise deal.
SPEAKER_01That's right. It's it's always about getting people hooked into this new habit. So a lot of times we get pulled into these larger deals because some developers in that company have tried Code Rabbit in their own personal repositories or some open source project that they care about or they contribute to. And that's usually like how we get into like we don't have a very strong outbound operation. It's all inbound for us. And that's that's a primary driver. If when we do a survey on where did you hear about us, majority would be through the developers in that company who tried out in open source or something. And so there's a very effective GTM motion. I don't know how much it translates to other domains, but for DevTooling, this has worked really, really well.
SPEAKER_02You have like very, very big companies that are already adopting um, like some of the more iconic companies are already adopting Code Rabbit. Uh, it's interesting to go from like this prosumer to a very enterprisey thing where I'm sure you need lots of systems and you need to review things and admin privileges. And uh, like how does one decide how to build the products for this like wider enterprise?
SPEAKER_01It's a very interesting challenge. And you know, the product does get complicated as you add more knobs to it, which at the same time um become too overwhelming for the prosumer audience. Yeah, they like simplicity and striking that right balance is always been like a big debate. I mean, of course, you have to build the go-to market motion first, right? I mean, so for the longest time our revenue had been mostly PLG self-serve revenue. And since then, we have started building a very successful sales engineering organization, um, a lot of like enterprise AAEs and further segmenting them, like mid-mark, mid-enterprise, strategic, like we've done all the segmentation, right? Um, then the other thing is a product, right? The product has to be enterprise ready. And we have customers who want self-hosted and they want to uh be uh want us to be in Europe, like whatever the GDPR reasons, right? And so we you have to do all that. Like now with the agents, it's also so easy to build all those configurability that people need. The real trick is in the product layer. Like, how do you keep the product simple, but also like still programmable enough that the enterprises, even on the legacy code basis with different um velocities of shipping, can still tailor it, right? So, yeah, I mean, everything has to go hand in hand.
SPEAKER_02I mean, you still going like essentially from zero to 50 plus million in two years is it's kind of insane, though, still, right? Like, and like even just an organization, you're hiring all these people. Like, how do you yeah, maybe you're not hiring as many people as like a company would have done 10 years ago if they're going that fast, but you still have a lot of people you have to absorb and train. Like, how do you think about just just generally growing a company in that kind of hypergrowth mode?
SPEAKER_01No, that's a good like we have 250 people, so it's not like the AI has eliminated.
SPEAKER_02That's a lot of people, and you gotta train them, you gotta motivate them, you have to get them excited, you have to get them all on the same page, right? Uh just even like interviewing that many people, a lot of people.
SPEAKER_01Yeah, that's why you have to scale with like uh professional leadership, like and the needs change as you went from let's say zero to one million, one to ten was a different need, and then 10 to 50 has been a different need. Like, so some of those needers uh either they up-level themselves or we have to bring in um other leaders on top, right? And tap into their network to hire more A players and so on. And that's what we have done with the sales team and product teams and it's run. Like, but on the engineering side, I would say we have not grown that much. Now it's very interesting that we are living in a time where our shipping velocity has exceeded our ability to market those products. This is the first time I'm seeing that. Usually you will have marketing run ahead, promise the features which are like a quarter or two away, and it's flipped. Now we have already shipped the changes, and marketing is strike trying to figure out what happened, right?
SPEAKER_02It's funny, yeah. You you're usually selling like three quarters ahead, and now you're selling like one or two quarters behind.
SPEAKER_01That's the reality, very interesting. And not just us, even the model labs, they're shipping, building in public, broadcasting live. So the marketing nature of marketing has changed, right? And you don't need like a big engineering organization to keep your go-to market team busy. You need a lot of sales because you still need to have those meetings. You can't automate with AI, but engineering and marketing to some extent, content, we are seeing rapid um shipping velocity thanks to AI. Um, but on the ground, you still need sellers. You do you need that DNA. You have to still sell the value, the ROI, the why to the end user and customers, especially the leadership in the enterprise teams, where the more stakeholders involved and get those deals to the finish line, right? The POC and the bottom swell is never enough. Like you have to find the economic buyer and convince them as well, right?
SPEAKER_02Now there's there's this consensus, or some people believe AI is gonna wipe out or or significantly reduce the junior engineers. And I've heard you say it's kind of closer to the opposite that new grads can do now in two hours, but used to take two weeks. Like, what does the entry-level job look like in a few years?
SPEAKER_01We you still need engineers of all kinds of calibers, like the senior engineers definitely bring in experience and system level knowledge. And it's about like the pod that you want to create when you're delivering software. So we have started like organizing our teams into smaller um these units called pods, where you have someone in the driving seat with an agent, but you have the UX team in the same room, you have the product team, they're all touching the code, by the way, but their own PCs, right? So the interns, we also have a lot of interns and we are able to make them successful. In fact, what I love about interns is they are just limited by imagination at this time, like whereas some of the engineers will still try to go a little bit deeper into the code, not willing to let go. Try to like do more tactical driving when you could now take a step back and let the agent also make a few decisions and run faster. So we're seeing that imbalance where sometimes even the senior people are being held back because they're too afraid to let go. Yeah. Yeah. Right. And at the same time, you also need their experience to watch the backs on what to review and where to how to validate, how to scale, because it's easy to prototype. But some of these features, once they go out, you still have to solve the scaling problems, right? Which you only discover in production. Yeah. So you need like people who are great with imagination and curiosity. You need engineers who understand distributed system scalability, you need product UX, everyone to come together and touch code at this point. So everyone is coding in a way, just their own PCs.
SPEAKER_02Now, when I started my career writing in Java, then maybe in Scala or something, um, I I always read code, but I never in my life have ever read assembly, or certainly never read machine level code. Are we going to move to a world where nobody reads code just like when I started a career, no one read assembly?
SPEAKER_01I think we are already almost there. Like, even I am mostly using my agent or code rabbit to ask the right questions about the code. Because as an architect level, like I usually just with a few questions, I'll have an idea that this is going in the right direction or not. Right? It's just about talking to the agent now more than writing the code. Um, and that quality is already there. And a lot of this is going to be codified in tools like Code Rabbit. So you don't need an expert person to do all this in every company. So the tools can do it for you. But yeah, I mean, I think the that time is just around the corner when you are really not reading the code. It's like taking the steering wheel off the car. We are almost at that point.
SPEAKER_00Yeah.
SPEAKER_01And we have already seen that IDEs are not future-proof, right? I mean, I have the first time I have moved out of my terminal. I used to be a NeoVim person for the last 30 years. And I've not been in my terminal in the last six months. I'm in Codex Desktop app.
SPEAKER_02Yeah, yeah. Yeah, which is also beautiful and easy to use and kind of fun, right?
SPEAKER_01Yeah, and just talking to the agent, right? And you need tools like Code Rabbit, like to do an adversarial review. So I have a loop set up. So until Code Rabbit is happy, we just keep committing, right? And and and then once everything is great, and if there's a feature is not high risk, you don't need to involve another human. You can just ship it, right? So we started getting your confidence that even an agentic approval is good enough for 80% of your features.
SPEAKER_02There's one that okay, there's a review, there's a code review. Um, there are a lot of other people are using whether it's codecs or callers to create content. Um, so I've got like I've got long-running codecs tasks, which are essentially creating content for me, um, doing lots of in some ways that's kind of like code. And I could use a review system to review my content, just like people review their code in a way. Like, could you imagine there's like a like analogy to review content?
SPEAKER_01Yeah, it's a very interesting thing. Like any problem today that can be translated to a coding problem, you're gonna move 10x faster. So if you can run your marketing as code, yeah. I mean, the idea is like can you run your company as code, right? Yeah, exactly. And if you can eval against something, uh it can be your design taste or something, the agent can loop around and get to really a picture perfect one pager, slide deck, whatever it is, right? And we are also we've been moving in that direction. In fact, I have codec sessions running, doing sales deck, one pagers, right? In in Markdown and MARP. So I use this MARP as one of the tools where it creates slides, right? Um, so the idea there is like the humans still need to understand these changes. So if these agents are gonna run longer and longer periods of time, there's gonna be a new concept of explainability that's going to emerge. Just like how Datog did observability, there was a lot of chaos in fast-moving environments, just like that. There is going to be a lot of chaos. Agents are gonna create massive amounts of work, and you're gonna be bottlenecked on reviewing and understanding those outputs. Yeah. Like whether it's content or even information retrieval. You want to like build trust in the output, is like, did this agent even go and look at the right data? If you are asking it to get your I don't know, weekly website visitors, did it even access the right tool? Right, you want to build trust, right? And I think there's going to be, like, as I said, a lot of products that are gonna take agents, reasoning traces, or reverse engineers from the output, just like how we do in review, we kind of reverse engineer what could have happened to build sort of an explainability walkthrough. And that's what the humans are gonna review, whether you're doing content generation, because everything is code. Code is now the execution layer for these agents, right? And so you need reviews not just for your software, but also for content now, because everything is code end of the day.
SPEAKER_02But that's exactly what I'm trying to. I I did a long-running codex task. It took about five days, it created all this content for me. But honestly, I in the end, like I didn't really understand what happened. I'm like, wait, what did you do? Like, and then I'm trying to like it was very, very hard for me to really get a handle on what it did and what was the level I could like go in and inspect some things, but it was it was like almost like a very it was almost like a code review, like a human code review that I had to go in and do. When will something like Code Rabbit be able to do that for me, or can it already do it? And I just don't understand it.
SPEAKER_01I mean, with the agents, you could do it. Like the thing is that for us, our audience primarily has been the developer audience, right? And we have been like bringing some of the products into other surfaces given how much how many installs we have in GitHub. So it's kind of natural for us to go from GitHub into Slack and other systems like Jira Linear. Now, Jira Linear are losing importance, interestingly, but Slack is becoming even more important, is what we're saying. Yes, for sure. Yeah, right. And those are the surfaces where we are trying to bring general purpose agents, which is building on the code context, but then you can use them to other work. And one of the things we are doing is instrumenting those agents. So one of the cool things we have done is like because we control the agents, it's a background agent, we are picking up I know tens of megabytes of reasoning traces out of these runs and running layers of AI on top to review those, right? And we're building visualizations to understand not just your um coding outputs but knowledge work as well. So building a more generalized explainability layer that goes beyond code reviews into any kind of agentic run visualization. And we are very bullish about the future. I think the explainability is going to be even a bigger category than what observability was in the past. And it's gonna start with code review because that's been the first biggest PMF area software agents. But very soon people are gonna realize that they have to run marketing as code now.
SPEAKER_02Accounting as code, legal as code, when you and we need to explain all these changes all the way through.
SPEAKER_01Right. Just like how you did infrastructure as code, you're gonna start committing your marketing assets in GitHub. So it's all gonna end up in GitHub eventually, right?
SPEAKER_02Yeah, yeah. And by the way, it'd be very helpful. I mean, like if you if you have a legal agreement, if Very helpful to have uh you know be able to go back in time and understand like your commits to your legal agreement and how those changed over time and who committed what. And in a legal agreement, you may be writing contract with your customer and you both might be committing to it, and you have to resolve these kind of commits in some sort of way as well.
SPEAKER_01Yeah, that's right. The way we are going is like, I don't like a lot of people on Twitter, some thought leaders will say, okay, the PR is dead, like you don't need pull requests. But we disagree. I think it's going to become the center of gravity in this entire movement in many, many ways, because there's no choke point. You can't just ship product without taking a look first, and especially if you're in a team setting, right? So this central quality gate is going to try to go beyond even software reviews that we are doing right now to other things. And yeah, I mean, I think the review step is going to be the most important step. That's what humans are going to end up doing, just doing the work, right? And maybe upstream giving some taste and collaborating on the taste as well. So it's either like you're collaborating on what to build and reviewing the prompts that the agents will run, or you're going to end up reviewing the end output.
SPEAKER_02It's interesting because so many people I know who have never coded before, just a you know, even a few months ago, they're now like they're now using things like they didn't even have a GitHub account. Now they have a GitHub account. They have all these kind of things that are so you can start to see these like other tools that are out there really explode. It seems like the Microsoft acquisition of GitHub, which was, I don't know, seven years ago-ish, seems to be a very good thing. Like there seems to be a lot of these like tools that are in that will be enduring going forward.
SPEAKER_01Yeah, that's right. I mean, I think GitHub was a great pick by Microsoft a few years back. No one saw it coming that how important this is going to become to the agent economy, right? Going forward. And of course, people are trying to build serverless Git. I mean, Cursor is launching a competitor, so are like a few players. Um, but it's almost going to be like very hard. Like it's uh it's kind of the blue sky moment. It's once you have the ecosystem, the tooling around it, the open source is all in GitHub. So it's gonna be very hard for another platform to emerge.
SPEAKER_02And you want your like your repo to be independent of your you like because if you move from cursor to something else, you don't want to all of a sudden like have to like you you want it to almost be an independent system.
SPEAKER_01That's right, right. And you want your reviewers to be independent. Um lot of people buy Code Rabbit because of that reason. Is if you were to use your coding agents for review, their context is already polluted, they're like too agreeable. Like once you have a session, it wouldn't question its own judgment and decisions, right? So you need a very adversarial clean session, which is programmed in a different way with different guidelines. And also it can be very thorough. You don't want to do that inline on the developer laptop, like you want to do that in a pull request model in a background agent that can run for 15-20 minutes and be very thorough before some things go out.
SPEAKER_02Now, a couple personal questions. Um, okay, the flex capital, we're we're very happy investors in uh Code Rabbit. We're also investors in Tracer, which is run by your brother Tanveer. Okay, what did your parents do well? Because obviously, like they did something well. So, what did they do well? How are you guys raised?
SPEAKER_01No, we've just been like very entrepreneurial. Like we started the first company together that was back in 2014 called Nexil. Um, and then we did second company together called Flux Ninja, which by the way didn't do very well. We ended up um Code Rabbit kind of came out of that startup as well.
SPEAKER_02Yep.
SPEAKER_01Um, so yeah, so me and my brother have been like very close.
SPEAKER_02Was there something your parents did that got you entrepreneurial?
SPEAKER_01It's about risk taking, right? So the one of the things the parents can teach you is to either be like risk averse and maybe get a job somewhere. Like that's what most people say, go get an MBA and getting hired by maybe Google is seen like at like 10 years, 15 years back, everything. That was the thing. Like you were going into a great brand name company, starting your career, whether it's consulting or something, right? Um, but my parents were more like just they were like they always like very much um risk focused. Like, I mean, the higher the risk, the higher the reward. And that's the game we learned to play growing up. Like, whether it was the schooling that we got, even going to the schools that were more focused on leadership and those qualities, right? Uh, to even the culture at home, it was about what's the worst gonna happen? Like, you always have something to fall back to. Yeah. So we could be like pretty much unshackled and take risk early days in our career. And even now we take risk, it's just like different appetite. Um, so I think that would be the biggest thing that I would say we did growing up.
SPEAKER_02Now, if you're gonna give advice to an 18-year-old for their career, you know, it's uncertain time. A lot of 18-year-olds are not sure what they should be doing. Like, what advice would you be giving?
SPEAKER_01Right now, I would say is the best time I've seen in the history to do startups and create value, right? These inflection points are very, very rare. Like it's uh once in a decade or two decades kind of an opportunity. And there is just so much, like I would say, land grab opportunity in terms of the use cases. We are like in the very early innings of AI. Like what we are building, it's not even 1% of what these models even today are capable of, right? So you're limited by imagination. Like, right now is the time if you are like not financial, there's no financial reason to join like or pay off the loans. Uh go out and take a risk, right? And the lot of VCs in the Bay Area, like, I mean think this space, this place is very special in terms of even how much uh people are willing to bet on the young talent. Like if you look at the cursor founders, cognition, they're all like young talent, like on the younger side. Yeah, um, because even VCs have realized that they are the ones which are moving fast with AI compared to a lot of the experienced folks who are now still like hesitant hesitant on adopting and questioning the priors, right? So I think it's a great time for like 18-year-olds, like first of all, learn how to use these agents really effectively. Even a single person can now be effective as I don't know, 20, 30 people, right? Very easily, even if you're doing it in the right way.
SPEAKER_02How are you personally using AI like in your personal life?
SPEAKER_01So, in my personal life, I mean, beyond coding, I mean, I am trying to use codecs more than chat GPT, I would say, given that code is now the foundation of everything. So, so that's become my go-to tool. Like early, like nine months back, I used to go to chat GPT, use sometimes a pro model as well, like especially when I'm doing some deeper research into planning something uh in my house or something. Uh, but now it's becoming more and more code-oriented, right?
SPEAKER_02Yeah, I found like even planning a vacation using codecs is better than planning a vacation using chat GPT.
SPEAKER_01That's right. That's right. We're seeing like the coding agents are going to be maybe the main substrate now, the underlying platform powering uh all kind of work or non-work, um, coding or non-coding kind of use cases. Um, that I'm already seeing in my personal life. I mean, unfortunately, I don't get much time for personal stuff. Of course, yeah. My wife's been meaning to like having my own, like I've two daughters like to teach them coding at some point, especially with codecs, not even finding time. Uh, but overall, yeah. I mean, in terms of AI, I use Grok a lot in my Tesla. So I've got Grok set up, so it's so convenient, like on the wavy, um, just to keep the kids busy, we'll just talk to Grok.
SPEAKER_02And and you just just for fun, ask questions or keep it going.
SPEAKER_01Like, it's really good with grams, like you want to like do mad libs or something. It's fun. Like, try try those things. It's it's cool in the car, yeah.
SPEAKER_02Yeah. All right, this is amazing. Last question we ask all of our guests what conventional wisdom or advice do you think is generally bad advice?
SPEAKER_01So, I mean, the I mean all kinds of advice can be good or bad advice. It all depends on the situation. Do you always see like um the main thing is like in terms of um a lot of the good advice that used to be good, like few years back, is now bad advice, is I would say like a lot of the things have changed under everyone's feet on how to build businesses, on how to think about funding, on what makes someone successful, not successful. Everything has changed. So I would say a lot of the advice that even comes from YC. Like one of the examples is like a few years back, you will start in AI any kind of company, you will go after some of the marquee logos. You will go say, Okay, I want to get Uber as my customer. Because if they are buying your solution, because everyone looks up to them.
SPEAKER_02It's a social proof or something.
SPEAKER_01Yeah, it's almost like the crossing the chasm curve. You have seen that. So you have the early adopters and and so on. Now, YC still does that. They'll go and tell their entire bash to sell to these marquee logos, they're gonna fool around, spend time. Interestingly, that curve has now inverted. It's invert crossing the chasm. In fact, the companies that are benefiting the most from AI are actually more mature companies. In fact, they are one of the first ones to adopt because they benefit a lot more. They have a lot of average talent. Yeah, the models are making them move faster. Even like a lot of people at companies which are like Bay Area um startups, um, they already had good engineers. So the initially they didn't see enough value because they were anyway smarter, agent, smarter engineers than the A models were, right? Yeah, yeah. That's right. So now maybe that has inverted. Now they are adopting agents as well. But at least 12 months back that was not the case.
SPEAKER_02Right, like the average engineer at, I don't know, Nike. Maybe Nike was never able to hire like the most talented engineers. And so they really needed these tools really quickly.
SPEAKER_01Right. Because what AA is doing is like taking the best practices it has learned to an average person. So it may not be like, I mean, for example, I can do marketing. I may not be the best marketer, but it does level the playing field for me more than someone who's already like a top marketer, they may not see enough value with these agents, right? So so the the curve has inverted, and that's a bad advice everyone's still like living by that when you go to the VCs and you want to like look at the quality of the logos, but none of that holds true in building a real business. Code Rabbit is one example. Now we have great logos, but to be honest, in a year back when you guys came in, you will remember like half our customers were unknown, I don't know, Japanese companies. Yeah, but the love was real, right?
SPEAKER_02I mean, yeah, absolutely.
SPEAKER_01So this is the first time I've seen it's an invert curve. Like companies you will usually reach out five years back at CD C or D stage. We were closing them in series A or CED.
SPEAKER_02Yeah, this is super interesting. Thank you, Hajj Dil uh CEO of Code Rabbit for joining us on Summation. By the way, I follow you at Harja S Gill on X. I definitely encourage our listeners to engage with you there. This has been a ton of fun. Really appreciate you being on. Thank you. Thanks for having me on. One more thing before we go. I read a blog called Summation. It's the same as this podcast. The blog is about non-obvious idea sharing on business, talent, data, longevity, and random contrarian takes. If you like the conversations on this show, you'll probably like the blog. It's free. New content comes out twice a month. You could subscribe at oron.substack.com. That's oron.substack.com.