Temporal Founders Samar Abbas and Maxim Fateev

Listen on Spotify, Apple, Amazon, and Podcast Addict | Watch on YouTube.

Madrona Managing Director Soma Somasegar hosts Temporal Co-Founders Samar Abbas and Maxim Fateev in studio. Temporal was founded in 2019 based on an open-source microservices orchestration engine project but is the result of a more than decade-long partnership between the two co-founders that spanned their time working on different iterations of the same thing at AWS, Microsoft, and Uber.

In this episode, Soma, Samar, and Maxim dive into the challenges of building an open-source ecosystem while also working on a commercial offering and scaling a successful company, how to navigate and adjust a product roadmap in an ever-changing world of AI and large language models, and how to successfully build an early startup team, navigate through a CEO transition, and bring on a startup’s first independent board member. It’s a very compelling conversation and a must-listen for every builder out there.

This transcript was automatically generated and edited for clarity.

Soma: To kick it off for the audience’s sake, why don’t you start with what Temporal does and, more importantly, how does it help developers?

Samar: Temporal is an open-source application platform that ensures the durable execution of your code. So let’s say if you have a function, and during the execution of that function, the machine crashes or another kind of hardware failure happens. What Temporal provides is it’ll resurrect your application in exactly the same state where it left off and continues on. So, as a software developer, for you, it’s like magic, like failure did not happen at all. So what does that mean? Let’s imagine you have an application that is about money transfer, and during the execution of such an application, one of the transactions fails because the application crashes in the middle of it. Where it debited the money from the source account but was not able to credit it into the destination account. If such an application is built on Temporal as a platform, it will reselect the state of your application on a different host, exactly where it left off with all local variables and call tags, and then continues executing from that point onward.

So what that means for you as a developer is now you can never have transactions that are left in an inconsistent state. So now you won’t have money transactions lost in the ether. And if it was debited from one account, then we as a platform provide a guarantee that that entire code executes to its completion in the presence of all sorts of failures. That completely eliminates 90% of the code people are writing to deal with those infrastructure or hardware failures essentially. So this is a very powerful primitive powering a lot of applications. Every Snapchat story is a Temporal Workflow. Every Coinbase transaction is a Temporal Workflow. Every Airbnb booking is a Temporal Workflow. Underneath the cover, they have been backed by Temporal as a platform, which is powering those use cases.

Soma: You guys know a little bit about my background, but I was for a while there involved with .Net Framework and Windows Workflow Foundation. So I really appreciate what it means to say, “Hey, let’s build an orchestration engine for workflows.” But I don’t want to sort of age myself here a little too much, but let’s fast-forward to today. Temporal is a great solution for people who care about orchestrating workflows and managing that. I’m sure there are other solutions out there. Can you talk a little bit about why Temporal is best in class and better than all the other solutions out there?

Samar: I think there are two areas where these kinds of systems typically fall short. The first one is the developer experience. Just by the nature of imposing that, “Oh, you have now… You can build stateful applications through this orchestrator, which inherently gives you durability.” It forces a certain programming model or certain restrictions that come along with that programming model. So that’s typically one area where they kind of fall short. The other one is if you look at these orchestrators, they have very severe limitations on scalability and performance characteristics. So this is where Temporal really shines compared to other systems out there. First of all, from a developer experience perspective, today, we have SDKs in six languages. So, if you know how to build applications in those six languages, you can build applications on top of Temporal.

So, there’s a little bit of a learning curve to understand the SDK, the API, and all of that. But after that, if you are a Typescript developer, you can write Temporal Workflows. So there is no extra thing that you have to learn. And then on top of that, the scalability and reliability characteristics is — I mentioned earlier, every Snapchat story is a Temporal Workflow. You can imagine New Year’s Eve, how many of those Snapchat stories are getting posted. So, this is a system that scales horizontally with billions of workflow executions running concurrently.

Max: I think the main point is people can build distributed systems and large-scale systems. But the main point of this abstraction, what we call durable execution, is that with Windows Workflow Foundation, for example, you cannot use code, but you actually create some sort of abstract syntax tree. And in Temporal, you just run code directly. So you actually can use a breakpoint, you can use a debugger, and if you say this code call ABC, it’ll be just ABC calls. You don’t need to create a sequence of whatever tasks and so on. So, it’s a very different experience. And I think this experience allows you to hide a lot of complexity.

So the whole idea of Temporal is that this high level abstraction allows to hide all the complexity of distributed systems underneath. So, it’s highly scalable system, fully asynchronous. So it’s event-driven architecture underneath without all the negatives of an event-driven architecture.

Soma: I’m actually glad that you mentioned Typescript. It did bring back fond memories of DevDiv days. But more importantly, it is one of those things where we said right from day one like, “Hey, we thought about open source because we were building it on top of JavaScript.” And usually, in my opinion, at least, there is a huge chasm to cross when you go from a cool open-source project to building a commercial entity or a company around that open-source project. For you both, what was that “Aha” moment when you said, “Hey, I can see why a commercial entity around this successful open-source project could make sense for the world.”

Max: Okay, first, you need to have a successful open-source project that solves a real problem. And we proved that multiple times because the first time this idea appeared at Amazon when I was tech-lead for the Simple Workflow service, which was the first service that introduced those ideas. We didn’t nail the developer experience. That’s why this service is still not very popular, but I think at least we learned a lot. Then Samar built a similar thing at Microsoft, so he pioneered the Durable Task Framework, which later was adopted by Azure Durable Functions. Then we built it at Uber, so we had multiple iterations. Even Simple Workflow, the public version, is the third version, so we had three iterations inside of Amazon. Then we had iterations within Microsoft and Uber, so Temporal is practically the fifth or six iteration of that. So that’s why we kind of learned that.

And when we built it at Uber, I think three things happened. First is that because we iterated a lot, we’ve finally nailed the developer experience. Second, because we built it at Uber, we had the ability to prove this platform on real applications. So we’ve got a lot of internal adoption first. For the first two years, we didn’t get any external adoption — nobody cared about our approach, but we were building it in the open. We were building from scratch on GitHub in the open. We grew in three years over a hundred use cases within Uber. And the third part was that people knew us from Amazon. So, when the first open-source users came, for example, with HashiCorp. The developer experience, high-scale production use, and practically knowing us personally, helped us to propel that and start to get adoption within pretty significant companies pretty fast.

Also, one thing I didn’t mention is every company, like at Amazon, at Microsoft, and at Uber, we’ve got all this bottoms up adoption very organically. None of this project was ever sanctioned from the top. Even at Amazon, it was a bottoms-up project. Samar did it also kind of bottoms up. And at Uber, it was the same thing. It never was like management came and said, “Let’s build this.” We just knew that it was needed, and we built it. And we got adoption every time. So, we absolutely were sure that the world needed it. And then from the monetization point of view, because from the beginning, it was built as a backend service, and then SDKs, and SDKs needed backend service to run, it was pretty clear how to monetize that. Because you cannot monetize a library. Okay, you can, but it’s certainly super complicated. If you look at every successful open-source company, it’s either big data or it has a persistence component, right? We have Confluent, we have Mongo, and we have all the other similar companies — and we fit into that category very well.

It’s very easy to monetize it as a service, and this is what we planned. If you look at our seed-level presentations, we didn’t need to pivot at all. We are practically just executing on that path. Certainly, our scope is much larger right now, and we added a lot of ideas, but our original idea about monetization as a cloud service, our original idea of building an open-source community — all these ideas are still the same.

Soma: When I think about open source, the other thing that I always wonder is particularly if you’re running a commercial company. You had to think about the balance between how much energy and effort you’re putting into keeping the open-source ecosystem thriving and staying engaged with what you’re doing in the open-source world. While at the same time thinking about how you’re moving the ball on the commercial front. Do you feel like you’re able to maintain that balance in a good way or do you feel like you’re often struggling to get to the right balance?

Max: It’s hard. It depends on your business model. Our business model is that we have fully featured open source under a very permissive MIT license, and we don’t do open core. So, it means that there is no crippled open-source version and then an enterprise version. Our open source is fully featured. And the only way we monetize it is by hosting the backend component of that on the cloud. And we also promise our users that it’s fully compatible. So, if you are running our SDKs against the open-source version, you can switch a connection string, and within a few minutes, you can run it against our cloud. But we also promise if you run against our cloud, you can switch connection string and connect to the open source without changing a line of your code. So I think that keeps us honest because it means we cannot have features that will break compatibility. It’s pretty straightforward. We are innovating on scale. We are innovating on control plane. We are innovating on cloud features. But the core abstractions and how code runs and SDKs, they’re open source, and they’re compatible.

Samar: The way I would think about it is — there are three models out there that you typically see with open-source companies. The first one is essentially that everything is open source. But the company’s coming to other organizations, “Oh, probably you don’t want to run it yourself.” Let us create a blessed build and then sell that with some kind of support contracts, which allows you to run them, essentially. Then, I think probably a decade ago, you started seeing a lot of open core models coming up where open source, in all honesty, was more like a marketing tool. But for any meaningful application, there are features that are proprietary that those companies were monetizing by packaging it with an enterprise offering. We, as Max explained, fall in the third category, which is that we have a full commitment — open source is a big part of our strategy just because of the critical nature of applications people are building on top of us.

Where we have 100% commitment that the core product experience, your application, you can port that application with zero lines of code change from open source to cloud, and from cloud back to open source. And we keep ourselves honest on the quality of the service that we are providing behind cloud. Where yes, you’d love durable executions, you’d love Temporal as a platform, which enables durable execution workloads or Temporal Cloud is a differentiated way to run those workloads. So, I think if you look at the next generation of open-source companies like Databricks, or even MongoDB transitioning with Atlas into that model, and Confluent building Confluent Cloud — we believe we were lucky enough to have started the company in an era where we started in this world.

Soma: We are talking about developers, we are talking about open source, and I know that two years ago you guys started this annual Replay as a way to engage with developers in the developer community. And I know that you guys have already announced that this year, it’s going to happen in September. Can you talk a little bit about what you envision happening there and why it is exciting for you to have this annual conference?

Samar: The way we think about that conference is a few things. First of all, we believe Temporal is… Or durable execution, this is a whole paradigm shift where it’s a new way of building applications. So this is a conference where we just want to be broader Temporal as a technology. We want it to be a real backend conference for developers who are excited about durable execution as a category play. This conference is kind of becoming where, once a year, you can renew your vows about this movement that we started around this backend development. It doesn’t have to be so hard, why it’s so hard?

So this is where these users come together and then talk about their experiences on building on top of Temporal, first of all. This actually helps us build this connection with our developer community to drive feedback into the product. And this actually, even a lot of our product roadmap are things that come out of our interactions on these conferences. This conference is also showing how people are using and leveraging Temporal for all sorts of use cases out there. For instance, last year we had 46 talks at our conference, out of that, only seven where internal Temporal employees were presenters. The rest of the talks were actually Temporal users or our customers coming in and showcasing how they are leveraging the technology.

Soma: Let’s switch focus and talk a little bit about AI. When you guys started in 2019, it almost feels like it was a pre-AI world, right? The world hadn’t heard about LLMs to the extent they know about it today. It was mostly microservices, cloud-based services, and distributed services. Those were the things that people were talking about in 2019. Fast-forward to today, you can’t have a conversation without talking about AI, generative AI, LLMs, and a whole bunch of other things. The rate of innovation is pretty fast here. How has that changed your view in terms of how your roadmap is evolving, what your product is going to be capable of doing, and what are you frankly doing with AI?

Max: We do use AI on surrounding area. For example, in our education, we have AI bot in our Slack channel and in our website, but actually our core product didn’t change. Because our core product is about making execution of code reliable and guaranteeing execution. And you wouldn’t be surprised that a lot of AI companies are our clients and users. I think we have over a hundred companies with a .ai domain, for example, in our customer base. Obviously not all of them are doing LLMs, but I think a pretty significant chunk of them are. They use us for a few things. One is just the whole infrastructure. Because Temporal is the best tool to create practically any control plane. If you do deployments or even creating your own cloud service, like for example, HashiCorp Cloud is built around that. And then if you’re doing any data pipelines, any machine learning, any big data — so control plane for that is a very, very common use case. And machine learning and AI training, all of those very, very popular.

And then when you use those models, if you are not just doing chat interaction, but you’re actually using for something more serious, like for example, you are generating text from video and then you’re analyzing that. There are multiple steps and some of these steps would be parallelized. Some of those steps should execute on specific machines because Temporal allows routing through specific machines and so on. So those are very, very common use cases.

Another exciting one that comes up more and more is agents. First agents were pretty primitive. If you just ask something, it use a couple of tools synchronously and returns result. But the new generation of agents are actually performing actions in the real world. And these actions can fail, and these actions can be long-running and sometimes these actions need to wait for humans to come back. And when you start making this kind of thing, it’s practically transaction orchestration, and it’s a very common use case. When you need to make those models run reliably at scale, Temporal is used a lot for those types of use cases.

Soma: So in a world of what people call agentic workflows, Temporal is a must have?

Max: Absolutely. Yeah. I’ve seen some very interesting examples of whole platforms built around that.

Samar: The analogy I would give is back at Uber when both me and Max kind of joined the company, Uber was going through this interesting phase there. Of course, they had a monolith, and then they started teasing it apart into microservices, but somehow, they ended up in a place where they had more microservices than engineers in the company. So what we saw there is this durable execution systems became the glue to tie these microservices together in a fault-tolerant, resilient, and scalable fashion. Sometimes, people used to describe us as asynchronous service mesh.

Then if you think about… Look at what’s happening with this whole AI ecosystem out there is at least what I see is a lot of these big foundational models are kind of hitting their limits. And now there is a lot more focus on these much more targeted specialized models which are really good at doing things, and then, this is why this whole agent ecosystem is starting to emerge around. Which means that building real-world applications that eventually will be tying in these agents and having a mechanism where these agents can have communication with each other in a fault-tolerant, reliable way. So you need that same service mesh to kind of enable that. So we believe this is where Temporal is very differentiated, even in the world of AI when a lot of those applications will be these agents talking to each other.

Soma: So, should I think about Temporal as an all-in AI company today?

Samar: Every company is all-in AI in today’s world.

Max: But I think it’s more like a tool for all AI companies.

Soma: Particularly if we believe in a world of agents and agentic workflows, and that’s what is going to be how the world evolves. You guys are right at the center of that.

Samar: Here’s how I see the AI ecosystem kind of three categories. One is a company building these large foundational models, which are the OpenAIs of the world. Then you have a bunch of companies building tooling, enabling those who want to train your models and all of that. It is kind of providing the basics for people who are building those things. And the last category is people who are building these verticals — really SaaS solutions powered by AI to do some really meaningful thing and add value to the user. I would categorize us in the second category, which we call the picks and shovels-

Soma: Yeah, what we call picks and shovels. I was just thinking that. That’s great.I want to switch a little bit to talk about you two guys now. You’ve known each other for 10, 15 years now and have worked together in multiple environments, whether it’s Amazon or Uber or you deciding to start Temporal. I’m sure along the way you guys thought about like, “Hey, do I want to start this with Maxim” or “Do I want to start with Samar?” And you decided to do that and have been at it now for 5+ years. What advice would you have for a new entrepreneur who’s thinking, “How do I decide who my co-founder should be?” Any words of wisdom?

Samar: I don’t know what advice I would have for other people, but I can share a little. One of the things is that we’ve been really privileged. We have known each other for almost 15 years now. I think over this long period of time, we have built a lot of respect and appreciation for each other’s skill set. Both of us bring very different things to the table. At the same time, both of us have very strong opinions that are, luckily, loosely held. So, I think those are the kinds of things that have helped us develop that relationship over the last 15 years.

The reason I was a little bit hesitant, like “What advice you would provide?” Because a lot of times, people don’t have that luxury where when they’re starting a new journey when they are looking for a co-founder. Like, yes, you don’t have 15 years to first build that relationship. But I think for us at least, I felt that helped us a lot. That helped us to build appreciation and trust for each other and kind of know each other’s boundaries. And in all honesty, for me, I think that trust that we have accumulated over lots of years of experience working together, when we were starting Temporal, I didn’t even think twice. I knew, if I wanted to start a company, Max was the person I really wanted to start a company with.

Max: I think one thing worth mentioning is that we never were people thinking about starting a company. We actually didn’t discuss starting the company until we got the first VCs talking to us. We were just working on the project. We were trying to solve real problems. We were doing it as an open-source project. There was kind of an assumption that maybe it would become successful one day and something would come out of that. But the reality is that it was never like, “Okay, we need to build the company, let’s build something…” It was just solving real problems and trying to make the project successful. At some point, it became successful, and we started getting interest. Then we started to discuss that. And then initially we actually were hesitant because we had good jobs and both of us never managed a single person before starting the company. So we were just ICs, just engineers, and we had a good life at Uber building our project, but we just realized that the idea we had and the product we had was much larger than the company. So the full potential couldn’t be realized within any company just because obviously priorities are different. And then we decided that taking money was the option. And yes, I don’t think we had any doubts that the two of us should do it together. Let’s say even the open-source project wouldn’t be successful, I think, if both of us weren’t there.

Soma: But in the last few months you also went through a transition. You literally swapped roles, right? Maxim, you started off as a CEO and somehow you had the CTO. A few months ago you decided to sort of switch roles. Not every company goes through this transition as seamless as you guys have made it look for the rest of the world. So tell us a little bit about the transition and what drove that and how did you manage that?

Samar:

When we started the company, as Max said, we had absolute clarity on the product we wanted to build. Both of us are first-time founders. We have very little idea what we are jumping into because you can imagine, building a successful company is a lot more than just a successful product.

Initially, the things that really mattered: do we have the real right strategy in place, like what’s the product, how it transforms? And we made certain strategic choices literally the day we were starting the company, a large percentage of those choices still hold. That strategy of the company is roughly exactly the same as the day when we started the company, when it was just the two of us. But at the same time, what we are finding out is it’s no longer just two of us. It’s like over 200 people in the company.

So, we are semblings of basically a real enterprise SaaS organization. We have all of those functions right now, the organizational challenges, how everything fits together. So there’s a lot more operational challenges that have started to show up, requiring a very different rigor and a different skill set. So then both me and Max took a step back and thought about our core strengths that we bring on the table. And one of the things that Max is amazing at and that I have a lot of trust in, and that I’ve built a lot of respect for Max is that Max can take any problem lifted at a high level of abstraction and solve it there — like setting the long-term direction where the company needs to be in the next five years. I’ve always been the one who is more kind of operational — what steps we need to take to deliver on that mission.

And what we are finding out is a lot of those challenges and the kind of growth phase the company is going through, the kind of operational rigor that we want to bring in is in this second category now. So it made a lot of sense for me to step into the CEO role, to be an operational CEO, to bring that rhythm to navigate through this next phase of the company growth. And although on the surface it looks like it’s a role swap but Max, for instance, Max is the CTO, he is kind of driving product strategy but not managing people essentially. So it was a very conscious decision that I feel plays to natural strengths for us as leaders in this next stage of the company.

Max: As you said, in a lot of cases this transition happened because they have to happen. In our case, we didn’t have any forcing function technically because as you know, you sit on our board — it was not like, “Okay, something is going wrong, we need to fix something.” Actually, I think everything is going pretty well. Even in this pretty tough environment. There wasn’t a forced decision in any case, it was more like me finally realizing what the work of CEO is, and clearly understanding that the further we go, the much less technical the job becomes. It becomes very much more management and execution. And I know that my strength is technology, my strength is product, my strength is architecture. And yes, I tried and I found that managing people is not something I really enjoy doing. I clearly saw that Samar, besides we started company both with zero management experience, he clearly has knack for that. He clearly can manage, and he can drive the execution I think very, very efficiently. And I’m lucky because as Samar said, we had such a big trust. So for me, it was like a no-brainer. I 100% trust him. So, for me, giving him the CEO role was something that I didn’t hesitate on at all. And I know if I didn’t have a co-founder like him, it would be much, much harder decision because losing control is something that most people have good reasons to fear. And I think having somebody you can trust a hundred percent, that’s why it was seamless transition. And other part I think funny one was that nobody is agreed. Usually if something happens which is not natural, people usually complain or at least talk about that. I think everybody looked at us and said, “Yeah, it’s right,” and we just keep doing what we’re doing.

Soma: I should tell you, hearing from both of you about this, hats-off to you both for orchestrating and doing this change. And doing it at a time when there’s — like Max just said, there’s no forcing function. Making such decisions and doing such transitions when there is no forcing function is far better than when there is a forcing function. Because you really are looking at like, “Hey, what is the right thing for Temporal given where the company is at this stage and what is required for the next 2, 3, 4, 5 years?” So hats off to both of you for doing that. I also want to take this opportunity and congratulate both of you on one recent announcement you guys made. You just announced that Sahir Azam, the chief product officer at MongoDB, is now on board as an independent board director at Temporal?

I’ve known Sahir for many years. In fact, he and I are sitting on the board of another company. So I’ve got a chance to work with him over the years and he’s a terrific fellow, so congratulations for getting him on board.

There’s an interesting decision that every entrepreneur has to go through. Where at some stage in their company’s life cycle, they have to start thinking about, “Hey, how do I bring on some independent board directors?” So, can you share a little bit about your thinking in terms of, “Hey, why do I need an independent board director now and what attributes we are looking for,” as you made the decision to bring Sahir on board?

Samar:

It was pretty big decision because this is the first independent board member. So, so far everyone who sits on our board is either the two of us, who are co-founders or VCs who have invested money essentially. Few months ago when we started looking back is especially when I’m stepping into this new role, I’m first time CEO. And what kind of structure I want to build around me to help me have the right support for me to navigate the company through the next level of challenges. Clearly, one of the things that I really cared about is people who’ve been there, done that. And because for us, every new challenge coming in, every new problem is an unknown. Both of us are first principles thinker. Yes, we can apply our first principle thinking to solve problems, but it takes time. It takes a lot of trial and error to finally get to the right answer.

People who have been there and done that, this is one of the criteria that we really cared about a lot. But not being there and done that in a similar kind of a setup in a similar kind of a business that we are trying to build. So I feel Sahir is really differentiated on many things. First of all, we ourselves as a company look up at Mongo as a business just from the open source nature and how they grew that community to a successful business. There is a lot of learnings that we even look up to and even more than that, one of the things that we really like about Mongo is their user base, how they operate as a company. Those are the companies, they are obsessed about developer experience, and that’s what has driven really… Help them build this engine that makes their business thrive. So, based on those similarities, we just couldn’t ask for a better fit for the first independent board member that we were looking for. So it just was an amazing match for us.

Soma: Max, do you want to add anything to this?

Max: The only thing is that it’s one thing to interview somebody and decide to bring him on board. But I think after he actually joined and we had conversations, we clearly saw that this confirmed our choice. Because just in a very short conversation, we were able to extract so much value, and clearly, we see a lot of alignment there.

Soma: Having known Sahir for a little while, I’m a huge supporter of him and I’m so glad that he comes to the Temporal board meetings now. With that, I think it’s time to wrap up again. I want to thank both of you for this conversation. I think we covered a lot of different topics, and it was great to hear your candid perspectives and views. On the product, on the company, on the open-source ecosystem, on some of the organization decisions that you’ve had to wrestle with in the recent past. A lot of interesting nuggets of wisdom, at least for me in terms of how you guys are thinking and how you guys are executing. So, thank you again. It was great to have you on this podcast.

Samar: Thanks a lot, so much. It was a fun conversation. Really enjoyed the talk.

Max: Yeah, thanks for having us.

‍