Why would a highly technical and backend engineering–focused company even need a designer?

When I first started my design career, there was no YouTube, Twitter, or responsive design. Frankly, designers and developers didn’t talk. Designers and developers were on different floors, in different buildings, and in different cities. This was quite problematic, and co-locating teams became the norm to force and foster collaboration.

In my earlier days I used to be insecure about being the only woman at the table and believed that I had to be territorial about my UX title. When engineers shared their opinions on what the user experience should be, I regrettably didn’t take their input very generously. I thought it was my job to be the expert and the sole person with an opinion on the UX.  During the past 20 years I’ve had the privilege of working with some truly amazing engineers, solution architects, scrum masters, engineering managers, you name it. From those working experiences, I’ve really come to depend on my engineering counterparts to help shape and realize the UX vision.

This personal transformation took time. There were bumps, and I learned some really valuable lessons the hard way. The most important lesson I learned was how to build meaningful relationships across the aisle. I can say with a high degree of confidence that I owe a great part of my success to collaborating, pairing, and partnering with engineers.  

What I love most about my job is that product designers cut across entire organizations. The success of the role is largely dependent on building trust and establishing true partnerships with key stakeholders. Design interfaces with growth marketing, dev rel, product, engineering, and customer success. This gives us a unique vantage point inside a business because we synthesize pain points, needs, and business challenges across multiple business units. Contrast this with the visibility of prod-eng teams, who largely operate within their own scope, mandate, and service area ownership. Although these prod-eng teams have a very deep and specific understanding of their services and use cases, they have a limited knowledge of how other services or parts of the product work.

As companies grow, the service topology, dependencies, and complexities grow hand-in-hand. Along with this growth comes increased debt (both UX and tech), high on-call volumes, constant interruptions, unplanned work, and weak or unreliable institutional knowledge. Not only do these issues surface in the UX, but the product design teams are often assigned to fix them, without fixing the underlying problems.

For example, I worked with a prod-eng team to improve the usability of a core feature. We found, to our surprise, that when the user finished a three-step flow creation, the object wasn’t immediately created. The user was redirected back to the landing page without receiving a success message or any reference of the newly created item, because it in fact had not yet been created due to our adoption of eventual consistency. Unbeknown to the user, the request went into a queue and would eventually be created, and the user would retry creating the item. 

What happens next depends on how long it takes for the queue to process the request. In most cases, by the time the user tried to re-enter the name input, the inline validation would throw an error that the name already exists. We observed that users would slightly change the name to satisfy the error message and continue, now adding a second item to the queue and re-entering this loop. In other scenarios, users would move to the final step in the process, hit the create button, and get some gnarly errors because the name already existed and wasn’t caught in the first step because it was still in the queue.

We pulled 90 days of historical data to understand what was actually causing delays for processing the request. We found that the majority of requests took 3 to 8 seconds but could potentially last up to 5 minutes, and in rare circumstances even longer. With this data in hand, we started to ideate on different ways to handle this delay in the UI. We could definitely hold the button with a spinner to solve the 3-second delay, but that wouldn’t work for a 5-minute request. Anything longer than a few seconds and the user would assume the app had frozen. Because we didn’t know how long the delay would be, we couldn’t actually design for various time ranges.

The only remedy was to go to the pipeline team and have them extract all the creation requests from the pipeline. This was well beyond the scope of this prod-eng team and considered part of the legacy machinery. Simply put, it’s just how things worked. The potential cost of this investment solely for presenting near real-time user experiences and eliminating confusing user errors and duplicative work was unjustifiable. We accepted that the new UX was a big improvement as it was and moved on.

This is one simple and rather isolated example of issues that creep up in complex systems. That being said, it does illustrate how these siloed back-end services can quickly become UX issues. These issues also show up as errors when the UX, PM, and engineering manager weren’t aware of all the points of failure and hadn’t invested enough in graceful degradation. This is especially true in distributed systems where financial ledgers and payments are dropped or are charged twice, in growth funnel conversions when onboarding numbers plummet when applications crash and are not recoverable, and in customer success volumes when user interactions don’t perform as expected.

Organizations that have complex and distributed services become increasingly and informally dependent on the holistic operation of the product design team to have a deep knowledge of where these issues live and how to solve them. I started by saying that this was one of the best parts of the job, and it is, but it also has a dark side.

When any new product designer joins a company, builds relationships with product and engineering and gains a deep product understanding, they ultimately come to the realization that design alone cannot fix failures in distributed systems. Putting bandages, workarounds, and half measures in place becomes overwhelming, uninspiring, and a grind.

These constraints also have hidden implications for a designer’s career. At the onset of the create-workflow example mentioned earlier, the prod-eng team established KPIs and built the instrumentation to measure adoption and success of this effort. We were held accountable for improving this core feature and modernizing the user experience, even though we didn’t have access to the other services that interfered with our desired path. This dichotomy starts to seed into all initiatives, where there are no lack of high goals and aspirations but little actual ability to effect real change.

Hitting those KPIs is paramount for the designer, as a large part of their performance reviews and promotion cycles depend on it. This is after all how we quantify designers' direct impact on the business. However, so many factors are managed directly by the PM and EM that the designer has limited decision-making abilities. Our role in these situations is often to help provide the insights and data to enable the PM and EM to make the best decision. We may want the outcome to be different, but if the plumbing can’t do it or the business won’t fund it, we can’t do it. It’s that simple.

As a design leader, I am often making the case for my tremendously talented team and arguing that any deficiencies in their work product reflects the limitations of the systems and not those of  the designer. In the end, it ultimately comes down to whether leadership believes you, and hopefully they do. As a direct result, designers staffed on product areas supported by microservices and more modern tech stacks will be perceived as being more successful than their peers assigned to monolithic and legacy systems. It doesn’t take too many promotion cycles for designers to recognize this and ask for different assignments. Designers need to ship high-quality work they can put in their portfolio, or they leave.

I have worked in media, government, financial institutions, agencies, and client side. I’ve worked at startups and for the largest employers in the country. I know there are always challenges, and I know that the combination of leadership and culture can make or break teams. There’s no escaping the complexities of distributed complex systems, whether it be systems of people or servers. Or is there?

A friend recently reached out to me and told me that this really exciting startup, Temporal, was looking to hire their first designer. I took one look at their website, and the promise of eliminating the failures of complex distributed systems sounded like absolute magic. I remember thinking to myself, if this actually works, it would be unstoppable. I jumped at the opportunity and can say with absolute certainty that it does work as advertised.

At Temporal, we are embarking on a mission to change the way developers build applications. This presents itself initially as a paradigm shift in the way engineers think about building their applications, for the better. As a designer, building out these learning and onboarding flows, educational materials, an enterprise-grade cloud platform, and a whole new suite of developer tools is a dream come true.

As I write this, I am so wildly energized by the fact that app crashes can now go unnoticed to users and developers, eventual consistency is replaced with strong consistency, and that I might never have to design an error message ever again. 


an-error-occurred

Why would a highly technical and backend engineering–focused company even need a designer?

Designing for the Journey

First Designer

Candace van Oostrum

candace van oostrum

Like many engineers (I think), I have wondered if the heyday of Internet engineering was over. I’m not suggesting that there isn't plenty of work left to be done... really, we haven't scratched the surface of what software can do for society (unfortunately both harm and good). But while systems architecture has evolved over the past 10 years, I believe most of the change has been "solidification" of existing techniques, along with a vast migration to Cloud and Open Source that has made things more democratic. In other words, advances in the past 10 years have been incremental rather than fundamental.

As for me: I’ve helped build some of the larger pieces of the internet. I was the lead engineer on Amazon's V2 website architecture (Gurupa), and was also the TL on a significant piece of Google's computational infrastructure called Millwheel, along with quite a long list of smaller projects. For perspective, at one point I had checked in more code to Amazon's codebase than any other single engineer. I would say I did a fair job of choosing my first two internet gigs :)

That’s all to say that while I truly love software engineering, I didn't really need another gig, and I certainly wasn't interested in anything less than “world-changing”. Mostly I was busy studying and exploring ML (which I'm certain *will* change the world) and otherwise enjoying life. Then I got wind of Temporal.

Having worked directly with Max (Maxim Fateev, one of the two founders of Temporal) at both Amazon and Google, I was well aware of his remarkable technical capabilities. Long ago at Amazon, I had also taken a stab at building a Workflow processing framework; I bought fully into the potential. What I had missed was Max’s unique and critical insight: that by making the logical processing deterministic, replay can recreate internal state in such a way that business logic can be written as a simple linear flow that can be made durable across any time period.

This key insight is a thing of absolute beauty. It enables a developer using Temporal to turn a complicated mess of business logic, timers, retry, error recovery, and persistence calls into a simple and easy-to-understand sequence of instructions that look exactly like the business logic they implement. Along with the reduced complexity comes the added bonus of a simple scheme for *repairing* damage done by any bugs that do creep in (not to mention a hugely reduced likelihood of bugs due to the simpler structure of the application).  Businesses and developers both value this experience very highly. It's not just cost savings... in many cases, it's the difference between feasible and infeasible.

Max and Samar's creation is, for me, similar in beauty and power to Euler's equation, or (to draw on more recent advances) the elegance of the "reverse computation" model of a spreadsheet, or the incredible power of the SQL model over the hard-coded data records we used historically. It didn't take me long to realize that this conceptual advance would change the world... and I believe its potential effect on how we solve business problems will be as large as those two examples.

This motivates me like nothing else does: a chance to help shape a solution to the problems faced by mankind. At its core, practically everything goal-oriented that needs doing, by man or machine, can be described as a Workflow (which I define as a sequence of steps needed to achieve a goal). I joined Temporal to be part of building this solution, and I’m writing this blog in pursuit of that goal: in the hope that others with the relevant skills and passion will come across my words and see an opportunity to work on something incredibly meaningful, challenging, rewarding, and valuable, backed by a superb culture.


paul-why-i-joined

Max and Samar's creation is, for me, similar in beauty and power to Euler's equation.

Why I Joined Temporal -  Paul Nordstrom

Software Engineer

Paul Nordstrom

paul nordstrom

Paul Nordstrom: Why I Joined Temporal

![Furthur 02](//images.ctfassets.net/0uuz8ydxyd9p/5bTlU2zPUBQ6dfcaq3DvaT/1ba04e67e402f63e8eb5b8d5398dc6b4/Furthur_02.jpg)

In the summer of last year I had the good fortune to be offered the chance to join Temporal as the company's Chief Product Officer. This was a very easy decision for me but not as obvious to friends and colleagues. I find this is common with Temporal; at first glance others might find the idea a bit abstruse, but once it clicks in their head, they realize the potential, see how far and wide it will go, and then won't want to think about anything else. This post is my attempt to share what clicked for me and why I had no choice but to join Temporal's co-founders, Maxim and Samar, on this special, quirky bus.

For nearly my entire career, I've worked in B2B software; I can't imagine doing anything else. Software is an innately creative medium. If you learn a craft and find the right place to apply it, you can manifest change with your own two hands and play a positive and meaningful role in the march of progress.

I've been especially lucky to get to play a role in two category-defining companies that have each left an impact larger than their revenues or market caps. For 5 years I worked at SAP, relatively late in its life, and then for another 10 years at Cloudera, starting in the very early days before the first shipped product. Temporal is a marriage of the domains of both companies: transactional applications and distributed open source platforms.

Transactional and business process applications have been the backbone of the global economy for as long as there's been software. All of the world's materials, assets, locations, people, customers, money, and obligations arrive, move, evolve, and depart at the behest of these kinds of applications. Many of these are packaged applications written by SAP and its successors, and even more of them are bespoke applications written by engineering teams working in companies of all shapes and sizes.

In the past 15 years, the infrastructure for how we run these applications has changed dramatically, and this has transformed how applications are built and architected. Cloud infrastructure—like containers, kubernetes, and PaaS—have made it easy and inexpensive to build applications from an ever greater number of discrete components. These cloud architectures have produced all sorts of benefits.  Ever-larger engineering teams can now develop & enhance applications with greater agility. Scaling has become much simpler and less expensive. Reuse is increasingly common, with developers borrowing from neighboring applications or from commercial APIs that allow teams to efficiently outsource things like messaging, identity, or payment processing.

Of course there's a catch. **Cloud architectures have unwittingly turned backend engineers into distributed systems engineers without anyone asking them.** Distributed applications present a different set of engineering concerns compared to those of a decade ago, and the familiar tools and platforms of the past have few answers. Thanks to cloud architectures, engineering teams can almost take application deployment, scale, and infrastructure automation for granted. But for these same applications, failure has become inevitable, and as we add more components and services, the mean time between failures will only decrease.

The more cloud applications built, refactored, or run, the more the hidden costs of being distributed compound. Engineering teams have been working through familiar stages of grief:

**Denial**: "No problem to see here, we just have to move fast and break things."

![denial gif](https://c.tenor.com/_OCVuSjrggcAAAAC/chaos-fire.gif)

**Anger**: "This would all go great if the (infrastructure/service/neighboring team) would stop letting us down!"

![anger gif](https://media.giphy.com/media/N5PsztQSjkYMw/giphy.gif)

**Bargaining**: "OK, we should try to rig something up to handle this. I mean, how hard can it be?"

![bargaining gif](https://media.giphy.com/media/mYqaRkXyoGbcY/giphy.gif)

These explanations have not sufficed and the costs are growing. Intermittent failures cost us in hard dollars with every order that gets dropped or payment that fails to settle. Engineers pay an equally large cost in productivity as they try to diagnose failures within a constellation of services written in various languages, on disparate infrastructure, each operating and logging differently from the other.

Temporal upends all of this. Temporal is a new kind of platform. It's not a database, cache, queue, or a means to run your code. It's a runtime for distributed applications that transparently manages their state at any scope and scale. By externalizing state to a generalized platform like Temporal, application teams get to outsource the majority of cross-cutting concerns presented by cloud applications. In doing so, applications become:

- **More reliable to run:** Temporal applications fail to execute less often, and when parts of the application do fail, they always recover to a consistent state - **More productive to enhance:** It's not uncommon to see examples of engineers implementing a feature in Temporal with 40-60% fewer lines of code than before, as they're outsourcing most of the distributed application concerns to Temporal. - **Easier to operate:** Temporal helps users see how their production application executes its logic, consolidates errors, and provides simple but powerful mechanisms to make fixes without interrupting the operation of the running application.

During development, the Temporal SDK provides engineers a simple set of primitives native to their preferred language, the most central of which are Workflows. These primitives can be incorporated selectively or extensively into any kind of stateful application. The more extensively they're used, the greater the benefit, because **these primitives enable developers to write durable code**.

At run time, the parts of an application written with Temporal are still just code that now talk periodically to the Temporal Server. This crucial design point, **"what you write is what you run" means runtimes like Temporal meet engineering teams where they're at**, whereas most other platforms attempting to solve this problem require engineers to change their process, deployment, tools, or other architectural decisions.

**Commoditize and democratize**

Anticipating and managing failure in applications has traditionally been a sophisticated subject handled by specialized technologists that work with complex niche technologies. Commonly this involves purchasing two of everything, while opting for every imaginable premium platform feature and splashing out on expensive "bulletproof" infrastructure. Most companies I've met try to organize all of their applications into different criticality buckets in order to parcel out reliability to the most needy.

Prior to 2009, scale was another kind of sophisticated topic handled by specialized practitioners, but Cloudera changed all of that. Cloudera used open source and no small amount of inspiration from Google to make scale a ubiquitous commodity. When I joined Cloudera in 2010, "petabyte scale" was synonymous with "exotic gear." So exotic that the leading database players referred to petabyte scale organizations as a "club" whose membership list was small enough to fit on a single sheet of paper. Cloudera lowered the cost of scale by 2 orders of magnitude, and today there are many thousands of companies who run at petabyte scale. This went on to have a large & positive impact on the world. Markets and businesses were created, new revenue and cost savings were unearthed, customer careers were made, and sometimes even lives were saved.

In a similar way, **Temporal is using open source and distributed systems to make reliability a ubiquitous commodity at any scale**, and this will have an even more substantial impact than commoditizing scale. Lowering the human and technical cost of reliability and reparability by orders of magnitude means we will come to expect these characteristics more often and in more places. Reliability as a commodity will give engineers freedom to take risks, because they can now move fast and not break things. Engineers building more reliable and repairable software faster will lead to a more automated and productive world.

The software industry has grown by orders of magnitude since when I first graduated college, giving rise to many new software platforms, commonly segmented and specialized for one niche or another. This made Temporal even more special to me. A new kind of platform that can offer a significant upgrade to such a universal need as writing and running software applications reliably. As much as software has eaten the world, it still has the potential to surprise itself with a seminal and broad-based renewal.

Software is still the right industry to develop my craft, and Temporal is the best place to apply it. Maxim and Samar have manifested a singular vision, and it's a privilege to help them build a singular company to match. We aspire to make Temporal responsive to the open source community, informative to prospects, and accountable to customers—and figuring out how to do that is half the fun. If you'd like to help us do this, I'd love to hear from you.


Furthur 02

In the summer of last year I had the good fortune to be offered the chance to join Temporal as the company's Chief Product Officer. This was a very easy decision for me but not as obvious to friends and colleagues.

Why I Joined Temporal - Charles Zedlewski

Charles Zedlewski

charles zedlewski

Charles Zedlewski: Why I Joined Temporal

My 12-year journey with workflows has finally led me to [Temporal](https://temporal.io/), and I do not believe it was by chance.
For a long time I was trying to fit in with different workflow ecosystems, without much success. 
I have been involved in writing workflow runtimes such as jBPM, Web and Editor tools, libraries, as well as 
[defining workflow language specifications](https://serverlessworkflow.io/), but often felt as something was missing.

### Looking back

I have been involved with open-source workflow technologies since 2009, right when they started taking off. Back then we were enthusiastic about it but more importantly, the work was done with the right mindset while focusing on the community.
Fast-forward 12 years and I find myself in a situation where things seem to be heading in a significantly different direction. 
The ecosystem I was involved with had stagnated, mostly due to its inability to adapt and keep up with the increasing demand for workflow technologies in modern architectures.
Innovation took a back seat to pulling wool over people's eyes, luring them into vendor locks and outdated technologies all while using buzz words like “standards” and “best practices” as bait.

The situation made me want to try my luck in a different tech field, but I never gave up hope that there are open source companies still doing it right. My search ended at Temporal, and it could not have been a better fit.

On the technical side of things, Temporal is no joke. When they say they enable developers to build resilient apps at a massive scale, they really mean it. There are no fake promises, no switch and bait tactics, no hidden agendas. 
Temporal has a strong focus on the community. If you have any questions, just reach out on our [Community Website](https://community.temporal.io/) or [Slack](https://join.slack.com/t/temporalio/shared_invite/zt-onhti57l-J0bl~Tr7MqSUnIc1upjRkw).

### Control

Temporal is very un-intrusive which really stood out to me, given my past experiences. Typically, when you want to adopt a new workflow technology, you’re forced to change the way you code and architect applications. You sacrifice a lot of control and have to conform to the rules of the workflow engines you adopt.

With Temporal, the story is completely different. It doesn’t matter if you have an existing application or are developing a new one; you can easily integrate it with a lot of different architectures and programming models. You pick and choose which parts of your code (typically your core business logic) you want to turn into resilient, transactional, and stateful Workflows.

### Programming model

Another thing I want to mention is how workflows are defined in Temporal. My past experience revolved around representing workflows via some domain-specific language defined with XML, JSON, or YAML. DSLs can be great and often very useful, assuming they target a specific business domain and do that well. 
There are currently very few workflow DSLs that are open source and vendor neutral that focus on this important fact.

With Temporal, users write workflow definitions with standard programming languages. This is a much more powerful approach to writing workflows. Not only are programming languages better at representing control flow logic (compared to DSLs for example), but along with the Temporal SDKs it’s easy to implement very complex things like:

* Sync/async execution
* Compensation (SAGA)
* Retries
* Periodic execution (cron)
* Event handling
* True parallel execution
* Cancellations
* Complex error handling

These are things that are very hard to define right or at all using any DSL. Temporal also solves a very profound problem with DSLs and workflows in general - testing. Temporal SDKs provide a very sophisticated testing API which can easily be integrated into existing testing/mocking frameworks and CI/CD pipelines.

If you are interested in using any DSL, you can write any DSL interpreter on top of Temporal.
This instantly brings all the Temporal benefits to your DSL of choice.

There are many more technical benefits I’d love to talk about, but will leave that for future posts. After a month and a half I am still learning cool new things every day.
Check out the [Temporal intro video](https://www.youtube.com/watch?v=23rX78xqYUg) I made which shows off some of the benefits of Temporal we touched upon in this blog.

### Joining Temporal

Being able to work on amazing technology is great, but what about the people inTemporal? From the beginning of my interview process until now, I still cannot believe how great the people are that work here. There hasn't been a single person I have interacted with that I didn't feel comfortable with. From the start, my co-workers went above and beyond to try to help me; from setting up health benefits, to signing up into the payment systems, and getting my laptop shipped. 
Shortly after I started, I received a box full of Temporal swag in the mail. Everyone has been patient with me while trying to learn as much as possible to start being productive. And if it’s not enough to be surrounded by nice people, this team is also super smart and are leaders in the open source community. You can learn so much from them!
The team atmosphere at Temporal is real, and I think it’s something that a lot of us long for. I sure have.

If you have any questions or would like to share your thoughts, feel free to reach out to me at [tihomir@temporal.io](mailto:tihomir@temporal.io).
Also don't forget, [Temporal is hiring](https://temporal.io/careers).


bbyrstkmia07llagoz14

I have been involved with open-source workflow technologies since 2009, right when they started taking off. Back then we were enthusiastic about it but more importantly, the work was done with the right mindset.

My journey to Temporal

Developer Advocate

Tihomir Surdilovic

tihomir surdilovic

I wrote about how it started in Why I Joined Temporal. Somebody suggested I should post an update about how things are six months later. How am I feeling after jumping from a corporate cliff into the whitewater of startup life?

![how-its-going-comic](//images.ctfassets.net/0uuz8ydxyd9p/2PITeBjDaL8Kss0mOybUOb/503d6d3b456c893d02d4116ca0c8a223/how-its-going-comic.png)

## Technology

I used Windows as my primary platform, and haven’t touched a Mac since the 90s. I was fairly confident that switching to a whole new platform would be a speedbump. Surprisingly, it was much smoother than I anticipated. Obviously, MacOS has its own idiosyncrasies, and I had to rebuild muscle memory for some keyboard shortcuts, but overall it’s close enough to Windows with WSL2. I prefer the Surface Pro for personal use because of its form factor, touch screen and how comfortable I am using it.

I cheated by continuing to use the Microsoft Ergonomic Keyboard and a two-button mouse, so I didn't have to use long button clicks for context menus. The one thing that continues to drive me crazy is the inconsistency of how Home and End buttons are interpreted in different apps.

About half of Temporal engineers develop on Linux machines. I realized why over time; just like on Windows, Docker does not work natively on MacOS. I recently wasted several hours trying to figure out why sed and even date weren't working as expected, only to find that it was because of the difference between GNU and BSD versions of these tools!

From the coding point of view, JetBrains is simply awesome; it is lightweight, cross-platform, and cross-language. In the hindsight, I should have started using Rider back at Microsoft for its lightweightness and cross-platform support. At Temporal, I mostly use GoLand, IntelliJ (occasionally) and Rider(I try to stay up to speed with the fast development in the Orleans world).

## Golang

Go is a bit strange, with some tradeoffs I understand but others, not so much.

On the plus side:

- Simple, limited syntax
- Easy to read and understand unfamiliar code
- Inline access to code from dependency packages
- Goroutines and channels are good tools for concurrency
- Very fast compilation

What's lacking:

- No generics and even overloads
- Simplistic visibility (public/private) model
- Unusable plugin model, no IoC/DI
- Debugging tooling is weak

Overall, Go is a reasonable tool for the job. I think C# would be more powerful as a language. What’s less clear to me is how much we would lose not having access to Go modules.

## Development

I joined a couple of weeks before Temporal v1.0 was released. It was a huge milestone after almost a year of development since co-founders left Uber - we switched to gRPC, added support for mTLS, and made a myriad of other changes.

As I ramped up, I primarily contributed to Temporal server. My major focus was developing Temporal's security features: authentication and authorization. The authorization model was my first design effort. Through the process of formulating the initial proposal, having internal discussions, implementing, and releasing the code, I got to know the team better. I also learned a number of "how can this be done in Go" things and established relations with some key customers.

The security work naturally morphed into contributing to the nascent Temporal Cloud effort. As Ryland mentioned in his latest [Transparency Report](https://temporal.io/blog/temporal-transparency-10), by late December we were able to start onboarding a small number of select paying design partners to our cloud service. It's crazy that we have customers paying to use our service on our Cloud, long before we GA it. This speaks to their level of confidence in our product. The best part is that we even have a waitlist to join the program.

Currently, development of Temporal focuses on three major areas:

- Open Source Temporal Server
- Open Source Client Runtimes that we, unjustly IMO, call SDKs
- Temporal Cloud

Nowadays, I spend most of my time on the Cloud and occasionally the Server. The Client piece is what I’m the least familiar with for now.

## Team

The company is still fairly small. The setup is very simple and requires little overhead. Everyone is focused on developing the product, whether that means adding new functionality or improving what we already have. I can sense strong motivation beaming through the Zoom screen.

The current company size allows for a very flat structure. On the engineering side, we don't have a single manager yet. Everybody except our co-founders(the CEO and CTO) are simply engineers. Decisions are made within the three dev teams if possible, where engineers would happily work with other teams when necessary. Company-wide decisions are primarily product-level and about improving our engineering processes. Task tracking and planning is lightweight and done using Notion, an interesting product that deserves its own deep dive.

I found the switch to an IC role refreshing. I’ve always liked the balance of Microsoft's dev lead role that combines technical work with managing a small team. But it forces you, for a good reason, to prioritize your IC contributions behind the team management priorities. I'm enjoying the plunge into building stuff, learning new languages and tools, figuring things out by collaborating with colleagues and partners.

Remoteness is a nuisance, of course. It’s definitely a challenge to connect and stay connected with colleagues that I don't work closely with. Fortunately, we had an in-person celebration for the v1 release back in October. That was the only chance I’ve had to meet a bunch of co-workers and their families. Aside from that, all communication is done through Zoom and Discord.

It's interesting that the team uses a suite of communication tools that broadly overlap in functionality. Zoom is for scheduled and sometimes ad-hoc meetings - it has the richer feature set and best screen sharing performance out of the box. Discord is used for spontaneous voice conversations, with an adjacent goal that such interactions are overheard by other people that hang out in the same channel. Discord is also used for quick screen sharing because it is more natural to supplement an already going voice conversation with a video stream.

Slack (not surprisingly) is the main method of written communications, within the company and with partners. Discord has chat features as well, but Slack is much stronger in that and has more business features and integrations. I was surprised to learn that Slack also has screen sharing. But I only saw it used once, as an experiment. What's important is that the team is open to trying new ideas. That's how Discord and Notion got added to the toolset.

## Culture

Temporal defines its culture along the three no-nonsense pillars.

- Developers, developers, developers
- Reliable like running water
- Seek the truth

### Developers, developers, developers

Our customers are engineers like us. If they are happy with our product, they will find ways to apply it and build great systems with it. Some of them will extend, give us valuable feedback and even contribute back. In the end, they will be the ones who have to convince management that they need Temporal. On the flip side, if developers don't like our tech, no manager will successfully force them to use it.

I'm very used to a world of developers as customers, considering it’s what I've been doing for the last ~15 years. I recently heard a story about a team of engineers at some company who tried to convince their management to use Temporal. When a higher-up opted for the low-code solution instead, the engineer who led the convincing effort simply quit and started looking for a new job where he could apply Temporal.

### Reliable like running water

This one was borrowed from Uber. The key value proposition of Temporal is that it makes it possible for application code to execute reliably without needing the usual complexity. Hence, reliability is our product. If Temporal is not reliable, behaves non-deterministically, loses workflows, there is no point in using Temporal. Needless to say, the bar is set pretty high.

My background in building services for game developers is helpful here because gamers are probably the most demanding end users. However, no matter how little patience gamers have, those are still games in make-believe worlds. On the other hand, Temporal powers many workloads with very real monetary value and the highest consistency and availability expectations.

### Seek the truth

This is common sense for people with engineering experience - that it's not the most vocal, articulate or persuasive person in a technical discussion who is necessarily right. It can be somebody quiet or early in their career. Of course, experience does help us recognize typical mistakes, design flaws, and corner cases. At the same time, mistakes that an experienced engineer overlooks tend to be of higher impact.

Intuition, gut feels, and instincts are all useful. That's our internal "AI" at work. But in the end, we have to operate on facts. Intuition should only help us find facts faster. It is also important to actively manage your ego and to depersonalize opinions, ideas, facts, designs, code, etc. That way we can suppress our human desire to win an argument or to feel proud with that useless "I told you so!" remark.

## Product

I gained a much better understanding of Temporal, it's capabilities, and architecture. Obviously, being neck deep in the code while adding and improving features helped. So did design discussions, reviewing PRs, watching what other people work on, and chatting with them.

In addition, we hold bi-weekly brain dump sessions where our co-founders interactively whiteboard to explain how certain core features work and why. Being able to ask questions like "why this way and not that" and "is this essentially X" helped me to build a conceptual mental model of those features. It also helped me separate core functionality from optimizations in the "whys". These sessions are recorded and some are even transcribed for our future employees.

The product does appear to define a new software category. I'm going on a limb here because somebody will always come up with a "but how is it different from X?" I think Temporal strikes a very delicate balance between on the one hand working "like magic" (that's what some customers literally say) and being very down-to-Earth practical, with pragmatic tradeoffs that experienced systems engineers understand and are generally comfortable with.

The Inversion of Execution model (which deserves its own post), seems to give developers confidence that they are in control of how their code runs. They can easily debug and fix their application code while Temporal ensures reliable execution of workflows and tasks, with a clear boundary between then.

The other day, a candidate I was interviewing shared an interesting analogy: Temporal has a chance of becoming TCP for developers. How so? By providing an illusion of smooth execution by automatically handling a range of failures and retrying behind the scenes. Just in the way TCP retransmits lost packets without exposing that to the higher levels of the stack.

There’s one typical pattern we see. One team of developers in a company will start building something with Temporal, and then in a few months we hear that a dozen other teams have already adopted Temporal. It spreads like a virus despite social distancing. I hear about 20 of Fortune 100 companies already using Temporal. From behemoths to tiny startups, we see people building software systems across many different domains: infrastructure provisioning, financial transactions, traditional business workflows, so on, and so forth.

We feel a special kind of pride when some of the most sophisticated technology companies choose to use Temporal. I still can't name all of them! But several went public about their dependency on Temporal: Coinbase, Hashicorp, Box, Checkr, Netflix, Snap, Datadog and Stripe. Stripe and Datadog even have job postings specifically calling out working with Temporal.

## Reflections

Looking back, I see that the decision to leave Microsoft was much harder for me to make than the actual transition into a new life. Big tech companies like Microsoft definitely have an advantage over smaller ones in that they provide opportunities for employees to change jobs, sometimes dramatically, across the wide variety of businesses they run. In my tenure at Microsoft I moved from Servers to Embedded to Bing to Research to Gaming, in several cases continuing to work with many of the same colleagues. This is definitely a benefit but it’s also a "trap". A barrier making leaving the walled garden harder. One day I should tell a story of my public ... let's say disagreement with Microsoft's head of HR at the time about the policy of internal transfers. 🙂

From my previous job changes I learned that what I'd be doing at a new place is almost always different from what I had in mind going into it. Not better or worse, just different. This transition was another example of that rule. Before I joined, we discussed that I would likely focus on the programming model of Temporal, leveraging my experience of building frameworks and tools for developers. In reality, I am working on the server and cloud side, leveraging my other experiences in building systems and services. No surprise for me there any more.

It was interesting to move from the Redmond bubble to the Startup bubble. In the Redmond Bubble there is still a lot less trust in using open source software and a bias toward in-house stuff. There are obviously good and bad reasons for that.

I knew beforehand it was the case, but I was still shocked to discover that Microsoft products are pretty much non-existent within the Startup Bubble. Strangely, I rarely hear Azure being mentioned, much less than GCP, and more like a necessary future integration tax to pay. Recently, I found myself arguing with coworkers who claimed that MacOS is more popular among developers than Windows!

I have been reminiscing about what I miss from my previous life. I definitely miss some of my colleagues that I enjoyed working with. I miss the great Orleans community that has formed and grown over the last 6-7 years. I love how that community started a number of great initiatives, such as the Gitter chat room, a community driven OrleansContrib GitHub org, and the virtual meetups. As the core team, we only supported these initiatives and contributed to them.

The irony here is that I am not cut off from the community. That's the beauty of open source. I just have so little time at the moment with all the things I'm busy with. I try to stay in touch with what's going on in the Orleans world and hope I'll continue to help one way or another.

I miss the availability of Microsoft Research. I wish I could still walk into the hallways of building 99 to ask Phil Bernstein or Sebastian Burckhardt advice on how best to ensure the atomicity/consistency guarantees that we need in Temporal. MSR is a unique institution that can provide state of the art help on nearly any Computer Science topic. You just need to know how to work with it.

I realized I mostly miss people, individuals. At the same time, I’m getting exposed to a great number of people from the new bubble. The Sequoia/Amplify/Madrona “mafia” can connect you to seemingly any top expert in the field because they are so well connected and broadly invested in the industry. Like with MSR, one just needs to know how and when to tap this resource.

I’m trying to make this as much of an honest assessment as I can, but it seems to sound very upbeat. That must be a reflection of how I feel now. I don’t know if it’s some kind of an extended honeymoon period. Time will tell. I think that regardless, this was a good and necessary change for me, no matter what happens next. I believe in our profession it is important to periodically shake yourself up, circumstances permitting. In a big scheme of things, we have the luxury of being able to do that when most other people can’t. Why not use it?

“Steh auf!”, as the famous German modern artist sings.


nkululeko-jonas-fSeKcdzeZTE-unsplash

I wrote about how it started in Why I Joined Temporal. Somebody suggested I should post an update about how things are six months later. How am I feeling after jumping from a corporate cliff into the whitewater of startup life?

How it's going

Engineering

Sergey Bykov

sergey bykov

Back in 2010 I was contemplating my next move after working on Microsoft's Project Oslo. Oslo was an effort to deliver 10X productivity to developers which inspired me to set the following criteria for my next job:

1. __Iterate with Developers__: I want to work in a place where we can start small, get something in the hands of developers and then iterate over the product to solve real problems.
2. __Cloud__: So far I had been focussed on building platforms using bare-metal products. But I was starting to see the kind of requirements modern applications have around scale and resiliency. I knew Cloud had to be huge part of the experience to deliver on those requirements.

## Start of Journey

![a-journey-1](//images.ctfassets.net/0uuz8ydxyd9p/6Oo8j0I5GK8XFgNClnxS5W/7009026a1bd94c52dc9f2549926b7a92/a-journey-1.png)

The Oslo framework led me to join the [AWS Simple Workflow (SWF)](https://aws.amazon.com/swf/) team. The team was led by Maxim Fateev, who came from a strong messaging background. Maxim was one of the most brilliant software engineers I had ever worked with, especially when it came to designing large scale distributed systems. Little did I know that I was about to embark on a long journey which is still being written to this day.

When I joined, the team was wrapping up a second version of the service which was already seeing decent usage within AWS. Even at the time, we could clearly see a pattern emerging. Developers were spending significant amounts of time building resiliency into applications, using low level primitives like queues, databases, retry mechanisms, durable timers, etc. Those same developers were able to produce higher quality systems with far less effort when using SWF instead of implementing resiliency themselves. Considering how useful the service was within AWS, the next natural step was to offer SWF publicly. I was part of the core team which worked on the public version of SWF which was launched in early 2012.

## Durable Task Framework (DTFx)

![a-journey-2](//images.ctfassets.net/0uuz8ydxyd9p/2G4gLHqv4JbovlsE61HKky/aa8aa5881aeb6053690b1449b9cdfd83/a-journey-2.png)

After shipping the public SWF service, I took an opportunity at Microsoft Azure and ended up joining the [Azure Service Bus](https://azure.microsoft.com/en-us/services/service-bus/) team that owns the messaging stack for Azure. Cloud was steadily gaining momentum and as more and more workloads started to get migrated, teams like Azure Service Bus became a focal point. As application developers increasingly started adopting microservices architecture to keep up with scale and availability requirements for modern applications, services like Azure Service Bus became the backbone to orchestrate calls across microservices. To keep up with the explosive growth, I worked as part of the team focused on large scale ingestion through [Azure EventHubs](https://azure.microsoft.com/en-us/services/event-hubs/). This solved the scalability and reliability issues at a messaging layer but developers still had to work with very low level primitives whenever they need to reliably orchestrate calls across microservices. The result was complex architectures which were expensive to build, hard to operate, and still came with reliability challenges. Reliability challenges stemming from all sorts of failure cases which needed to be handled due to the distributed nature of the application.

I could clearly see that the developers building applications on Microsoft Azure were facing eerily similar challenges to what I had seen back at AWS. The same challenges we had tried to address with SWF. So I used one of the internal team hackathons as an opportunity to pair up with Affan Dar and take another stab at solving the problem. Affan had a very deep understanding of Azure ServiceBus so he was the perfect person to build the backend for the stateful C# experience I had in mind. Microsoft had recently added async/await capabilities into C# and it turned out to be an amazing fit for writing stateful applications which need to orchestrate calls among microservices. Since Java lacked an async/await like primitive, we had to rely on Promise-based async approach when building SWF. But with C#, we were able to deliver a much cleaner and synchronous programming model using async/await. This hackathon project resulted in [Azure Durable Task Framework](https://github.com/Azure/durabletask) as an OSS client SDK which uses Azure ServiceBus as the backend to provide a stateful workflow-as-code experience for applications. I'm so glad to see Microsoft has continued investing in the experience with [Azure Durable Functions](https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview) as the latest reincarnation of the original effort. An effort which started with that hackathon project.

## Transportation as Reliable as Running Water

![a-journey-3](//images.ctfassets.net/0uuz8ydxyd9p/7nG8dxN7l3B3ObtYvLih72/50591ca43882b43d536f3ebe994fecad/a-journey-3.jpeg)

In 2015, Uber opened a development center in Seattle and I decided to take the leap and join the engineering team. Coincidently enough, Maxim Fateev ended up joining the Uber team in Seattle only a month after I did. At the time, Uber was running on Kafka 7 as the messaging backbone. Based on the scale they were running, they were encountering some serious operational issues. Considering Maxim and I had more than a decade worth of experience building messaging systems similar to Kafka, we decided to create the OSS project [Cherami](https://github.com/uber-archive/cherami-server) to address this Uber sized problem. After a year of working on the project, we were observing a very similar pattern to the one that we tried to solve with SWF and Durable Task Framework. When engineers needed to build with raw infrastructure primitives like queues and databases they were spending 80% of their time building resiliency into the application. This was clearly not sustainable for Uber, which was growing at an amazing pace and building a brand of "Transportation as Reliable as Running Water". This need to increase developer productivity without compromising on reliability of the system was the motivation for Maxim and I to create the OSS project [Cadence](https://github.com/uber/cadence). Within a very short period of time, we built a multi-tenant service hosted by our team. Cadence provided a great developer experience by enabling developers to use Golang to build and run stateful applications with very little operational overhead. Cadence grew organically within Uber and quickly became popular among developers. It slowly but surely began to emerge as the standard way to build stateful applications when reliability cannot be compromised.

## Magic of Open Source

![a-journey-4](//images.ctfassets.net/0uuz8ydxyd9p/25stnVr8qULEek8w0uu3fc/382ee5aea46a5456d471b3a80316fd11/a-journey-4.png)

Today, more businesses are turning to software for running mission critical parts of the system and software is becoming key part of the end-user experience. The problems faced by engineers at places like AWS, Microsoft Azure and Uber have become more and more common across the industry. This belief was validated in early 2019. Developers from companies like Hashicorp, Box, Doordash, Checkr and dozens of other places organically discovered the Temporal technology and immediately started using it for their mission critical workloads.

We have a strong belief that an infrastructure technology of this magnitude needs to built as an Open-source project. This led both Maxim and I to quit our jobs at Uber and launch [Temporal Technologies](https://temporal.io/) in October 2019. Over the last year we made huge advances with our developer experience and released [Temporal](https://github.com/temporalio/temporal/) as an Open Source Software under [MIT license](https://github.com/temporalio/temporal/blob/master/LICENSE). We recently launched our first production release of Temporal v1.0.0 which is already being used by numerous companies for critical workloads.

[https://temporal.io](https://temporal.io)

[https://github.com/temporalio/temporal](https://github.com/temporalio/temporal)

clem-onojeghuo-T7gi cyrkdg-unsplash

Back in 2010 I was contemplating my next move after working on Microsoft's Project Oslo. Oslo was an effort to deliver 10X productivity to developers which inspired me to set the following criteria for my next job:

A Journey

CTO and Co-Founder

Samar Abbas

samar abbas

I left Microsoft last month after a long and successful career. It's an understatement to say that most of my colleagues were surprised. What could possibly motivate me to leave the environment where I learned how the system works, built many relationships and knew how to make things happen? Not to mention the sizable amount of unvested stock.

To answer this question, we have to travel back more than a decade. The year was 2009, and I had recently moved to Microsoft Research from the Online Services Division. I joined a newly established lab with a charter to explore the future of Cloud Computing (still relatively novel at the time). More than joining the lab, I joined the project Orleans. A project that had a name, a 60,000-ft vision, and not a single line of written code. The vision was bold - to reimagine how cloud-scale application should be coded. At the time highly available high-performance scalable systems were only achievable by experts, and Orleans wanted to make it something possible for every developer. The question was how to reduce the amount of complexity inherent in this space. We believed the answer was a new programming model, that would change how developers conceptualize applications, how they structure code, and ultimately how they think about the problems they face.

As we were churning through early prototypes of Orleans, one crazy idea after another, I met a guy who was sitting around the corner on the same floor - Maxim Fateev. Maxim was on a different team working on some unrelated projects, but he also had some ideas of his own that he was prototyping. As developers do, we immediately started discussing our views on “what should be done and how”, pros and cons of the different approaches, the usual stuff. It became clear from those debates that I was interested in the domain of interactive workloads - with RPC-style request-response, low latency, high throughput, client-side retries when deemed necessary, and eventual consistency in case of node failures. That was the viral thing at the time, started by the Dynamo paper from Amazon. Maxim was interested in building a reliable workflow system that persisted variables and virtual call stacks, so that application code could "resume" its execution after a delay or failure. Almost as if it stayed on that line of code the whole time. At the time it sounded too slow to me considering the emerging workloads of interactive entertainment, Internet of Things, etc. I was younger, less experienced, and more opinionated then.

Maxim left Microsoft soon after, continuing to pursue his passion to build a workflow-based programming model. In the meantime, we shaped Orleans into a promising prototype, and eventually put it into production with the Halo team. This instantly boosted the credibility of our project and led to a number of collaborations/co-designs with internal and external partners.

Sometime in 2013 or 2014 we organized an Orleans hackathon for a select group of big partner companies. They coded while isolated from each other in separate rooms. To my surprise, out of seven companies three ended up building workflows as a major part of their application. A pattern started to emerge that I only recognized much later. Orleans users loved the simplicity and power of the “get an eternal object and invoke its method” model. At the same time, in some scenarios they needed a workflow solution for longer-running operations.

In Orleans, we had a suggested solution for building workflows - use reminders (persistent timers) to periodically activate and invoke workflow objects. That way they can check if it's time for them to perform an action or to transition to a new state. This minimalistic approach generally worked, although it left most of the complexity to the application developer. The community also built a number of workflow-like solutions, such as Orleans.Activities and Orleans.Sagas to fill the gap.

As we moved out of Research and into the product group, I got even more intimately involved in building and operating backend services for games, but the shadow of workflows continued following me. I kept encountering more and more workflow use cases. Either an operation inherently takes longer than an RPC call can afford to wait, for example, starting a virtual machine. Or a scenario where an operation succeeds most of the time promptly, but fails regularly enough that it could potentially time out and require a follow up. In such cases, an application typically needs a delayed action that would check if the operation succeeded, retry it if necessary or explicitly give up on the operation and to clean up any associated resources.

It is possible to implement such actions with timers and reminders and would involve ~following steps.

1. Register a reminder for the requested operation, so that it can recover from failures.
2. Execute an external API call to initiate a long running operation, such as allocation of a cloud resource.
3. Schedule a timer to periodically check if the operation succeeded until we reach a timeout we allotted to it.
4. If the operation successfully completed or failed, record or report the result, unregister timer, unregister reminder, clean up any intermediate state we recorded.
5. If the operation hasn't completed or failed within the timeout window, try to cancel it, record the fact that we've given up, perform the same cleanup as in 4.
6. Follow up on all cancellations to make sure we didn't leak any resources. This requires another piece of logic and a timer/reminder combination to guard its execution from potential failures.

Steps may vary depending on the types of resources being managed and their likelihood to fail and leak. But the main problem still looms - to perform a single line reliably, we were forced to put a lot of scaffolding code around that single line which expresses what we actually needed to do. Such complexity slows down developers at all stages:

- Initial development
- Failure testing and stabilization of the codebase
- Investigation of production incidents

We added a long list of features to Orleans over the years that helped developers build robust scalable systems. We even dared to integrate distributed ACID transactions, with help from our great partners in Research. But when somebody would inevitably ask, “how do I implement a long-running operation”, we would describe the same 6-step process with timers and reminders.

It eventually dawned on me that RPC and workflows are sort of yin and yang of interactive applications, that the debates from 10 years ago about RPC vs workflows should have been about merging the two and not choosing one over the other. Our team met with people at Microsoft Research about a year ago to discuss where they could help. Workflows were our number one ask. Our rationale was that if we could add an abstraction for workflows, that would greatly complement the RPC style that Orleans already excelled at. I'm pretty sure a number of people will think, "Doh, that's obvious. Why did it take you so long to realize it?" I'm okay with this embarrassment, as long as it serves as a useful lesson for others.

In the meantime, I heard through some mutual contacts that Maxim was now at Uber and was leading some ambitious project. Later I learned that it was called Cadence, yet another reliable workflow system that Maxim and Samar were working on, and it was open source. This put Cadence on my radar but in that “when have time to take a closer look” bucket. Somebody told me a few months ago that Maxim and Samar had started Temporal Technologies last fall. But I never considered startups to be my thing. However, as I began talking to them, my attitude rapidly started to change. As I learned more about the programming model of Temporal and descended down the rabbit hole of its implementation, I started seeing “workflows, workflows everywhere”, big and small, quick and long running, just like in that popular meme.

![sergey-why-i-joined-temporal-1](//images.ctfassets.net/0uuz8ydxyd9p/JDJ55cTn4TlhG4cxSnQHQ/737bafcc2bd537904d3e1e4bc3cdfc5d/sergey-why-i-joined-temporal-1.png)

If we could turn the six steps in the above list back into a few lines of trivial code, we would free developers from the burden of scaffolding, and enable them to focus on interesting problems. Workflows are somewhat like grains in Orleans - units of distribution, execution, and fault isolation. They are "just" executed differently - by the application supplied workers. I call this model Inversion of Execution, not sure if it will stick. There are, of course, other major architectural differences, but the high-level goals are the same - qualitatively reduce complexity of application code; make applications resilient (invincible) to inevitable failures; empower developers to solve more interesting problems than those we can take care of.

Besides the workflows everywhere, I also saw the two co-founders, roughly my age and level of experience, sharing the same passion I've had for many years. Both with track records of delivering innovative solutions following that passion. The startup environment provides a nearly complete freedom to pursue what we agree is right. There's no management hierarchy above to approve a decision or to allocate a budget. You just need to deliver the product and delight your customers. I kept thinking that if I were ever to join a startup, Temporal had to be it.

It is very difficult to get up and leave a comfortable well paid place where I've learned how to be successful, built many professional relationships and connections, and can navigate with my eyes closed. Yet I felt that if I play it safe and not jump into the unknown now, I might regret it for the rest of my life. So I decided to take the plunge, break the proverbial golden handcuffs, and go all in on Temporal.

praewthida-k-5Ui5glVjUAQ-unsplash

I left Microsoft last month after a long and successful career. It's an understatement to say that most of my colleagues were surprised. 

Why I joined Temporal

We know that the recent delay caused some concern in the community. We empathize entirely and are seriously dedicated to correcting this going forward.

First I want to outline the two main guiding principles which drives most of the release related decisions at Temporal:

1. __Simplify__: Temporal presented us with an opportunity to make product enhancements which significantly simplify the overall experience without needing worrying about backwards compatibility. We front loaded these enhancements as it sets us up for long term success.
2. __Quality__: We know that users rely on Temporal for mission critical parts of their business applications. Since our main value proposition is improved application resiliency, we could not afford to compromise on the quality of our release. If we did, it would completely diminish the core value of the product.

Many factors directly or indirectly contributed to the delay of production ready release for Temporal. I'm going to outline few of the top reasons and how do we plan to address them from now on.

## Uncertainty

When Maxim and I left Uber to start Temporal we had a lot of uncertainty in front of us. There were big open questions about the roadmap, potential collaboration with Uber on Cadence, the business model for the Company, etc. Figuring out a plan during these uncertain first few months took a lot of time and energy. As we talked with our users during this initial phase it became evident that there is one thing which is going in our favor despite all these challenges - an amazing community of Cadence developers who are super excited at the potential of our technology. Once we had this realization everything started to fall in place. Every decision we have made since then is quantified by the value it brings to the community. We chose our first three focus areas explicitly based on the value they would bring to community:

1. Simplify experience of writing applications using Go and Java SDKs
2. gRPC migration for better security and ability to easily provide SDKs in other languages
3. Persistence improvements so we can easily provide bindings to other databases like PostgresSQL in the future

Although we did relatively well to navigate through this period of uncertainty, we made the key mistake of committing to dates without having a clear engineering plan in place. We would frequently communicate release dates to users which were based on the assumption of ideal days. This meant all of our estimates were based on the happy path leaving no room for unknowns.

## Bootstrapping the Company

As first time founders, bootstrapping the company was a huge learning experience for Maxim and I. We knew starting a company would be time consuming, but did not expect the time sink it turned out to be. There is a never ending stream of operational tasks such as:

- Legal processes - setting up a business entity, contracts etc
- Online accounts and services - payroll, benefits etc
- Hiring and Recruiting - hours worth of screenings alone

These tasks ended up eating a lot of our time meaning that we were left with only a fraction of the ideal day for actual release-relevant tasks. After this initial experience, we feel much better calibrated to provide realistic estimates which accounts for the operational overhead of running a startup.

9 months after starting Temporal I now have lot more appreciation for founders who leave their day jobs to start this crazy journey. I feel so blessed to have access to some amazing folks in the industry who provided great support and advice to help us through these growing pains.

## Recruiting

If I have to pick one thing we grossly underestimated the most, it has to be the time and energy spent on recruiting. Recruiting top tier talent for early stage startup itself is a daunting task which COVID-19 only worsened. Revamping our full-loop process so we could hire engineers virtually was quite challenging. The fun doesn't stop once we hire someone either. Ramping up new hires remotely on a product like Temporal was interesting... I really miss the days when we had whiteboard discussions.

I'm so glad that we spent the necessary time on recruiting and were able to grow Temporal into a now 10 person company. This is definitely the most satisfying part of the journey so far - to be able to build a team of such amazing folks who share the same passion in bringing the power of Temporal to rest of the World.

We now have most of the essential founding team in place, which is not only key to having a great quality release of the product but also the ability to improve and support post release.

## Looking Forward

A key learning from this process is to not treat releases as a binary milestone. Going forward we will be providing much more context on key milestones for releases:

1. Clearly define the scope for a milestone
2. Provide visibility into all the work which goes into achieving goals for the milestone
3. Report progress to community


lubo-minar-qRGuuVOzsTc-unsplash

We know that the recent delay caused some concern in the community. We empathize entirely and are seriously dedicated to correcting this going forward.

Life at Temporal