For the engineers on your team to feel a deep sense of ownership about your product and to continue to raise the quality bar & engineering practices, you must empower them to drive the technical strategy
Through this article, I hope to convince you to set up a technical group of ICs in your org (the size of the Benefits Engineering team at Gusto is ~20 engineers) by demonstrating some fantastic accomplishments and lessons learned in the process.
First, let’s get familiar with a few acronyms that will show up in this article a few times:
- IC → Individual Contributor
- PE → People Empowerer (That’s right. At Gusto, we refer to Managers as People Empowerers since we don’t believe in adults needing to be managed)
- TCs → Technical Consultants (These are the individuals who would be part of the technical group)
Setting the stage
It was time for Fiscal Year planning at Gusto, and we needed to identify some of the big rocks that we wanted to tackle from a technical point of view. In addition, it needed to serve a core product need, staying true to Martin Fowler's words - we are not gold plating something that isn’t valuable. Typically, an Engineering PE or a passionate IC would volunteer to identify and drive these efforts. This technique had been successful, too (e.g., building the 2nd version of our enrollment system, migrating the complex logic of calculating payroll contributions & deductions from the benefits system to the payroll system).
As you can imagine, such an effort was hard to scale since it's highly dependent on specific individuals who are passionate about work that serves a core product need. Also, this did not allow us to have a cohesive strategy that involved all ICs being equal participants.
This time, we didn't want to repeat the same pattern. We decided to set up a group of ICs known as Technical Consultants to help us identify some of the big rocks to tackle and build a technical strategy for the Benefits team.
We set up this group with ~5 engineers (senior or tenured engineers across three engineering teams) with the following goals in mind:
- TCs will play a key role in shaping how the Benefits product continues to evolve. They will play the role of mentors/coaches to engineers and essentially act as culture carriers.
- TCs will continue to raise the bar on Benefits Engineering by making decisions on technical standards through close partnerships with engineers outside of Benefits Engineering.
- TCs will help drive the execution of technical strategy by working with ICs and PEs on the teams that they are part of.
Often, short-term goals cloud the necessity to take a step back and re-evaluate our needs as a whole. These include an increasing backlog of errors, on-call processes, and specific teams needing support during times of transition, to name a few.
Additionally, Engineering PEs often don't have the same lens to look at things from and aren't in a position to fully empathize with the challenges that ICs go through.
Expecting every team to do the right thing results in inconsistent outcomes. As an example, when we started the process of modularizing parts of our system, each team approached it in similar yet different ways. The result was that the ruby gems created were inconsistent, which would make it harder for someone to onboard and navigate across these systems.
Therefore, it is imperative to enable ICs on your team to have a sense of complete ownership by taking control of your technical strategy. The Engineering PEs have a critical role in bringing the business perspective to the technical strategy.
Our first initiative was to get our list of Sidekiq Dead Jobs under control. For those not familiar, Sidekiq enables background processing for your Ruby application. In the past, we've had some of the background processing jobs die due to different reasons (e.g., bugs, network timeouts, state change b/w the time the job was queued, etc.), and looking at these would be the responsibility of an on-call person. However, since clearing up the Sidekiq morgue wasn't top of mind for everyone, it would happen inconsistently depending on the bandwidth of the on-call individuals who had to deal with other issues.
TCs strongly believed that a dead sidekiq job meant we had failed a customer (e.g., failing to generate necessary documents to submit to carriers to ensure the customer has active health coverage when they visit a doctor). Additionally, this only added to the noise and reduced our ability to identify signals to prioritize these dead jobs to resolve quickly. With that in mind, TCs came up with Sidekiq ZERO, a program whose goal was for teams to increase the signal-to-noise ratio by re-evaluating jobs that would often die, analyzing the root causes, rethinking them, and making them more reliable.
Having determined Sidekiq ZERO as the initiative, the TCs group drove the strategy across Benefits like any Product Manager would do across their teams. It included setting success metrics for the teams, closely working with them to ensure that we are on track, leveraging their expertise to help unblock teams, and escalating when support is needed.
Each TC drove progress towards our overall goals by working with their team and PE to develop a plan for Sidekiq ZERO that could be part of their weekly sprint work. Some teams decided to make an exception for a couple of the jobs (since the ROI to address the root cause was too low), just like any product-focused teams would. Other teams went above and beyond and set up Datadog alerts for a few of their key jobs to help them address them proactively.
The Sidekiq ZERO program changed how we treated our background processing work and enabled us to do so in a sustainable way without having it be a function of an on-call individual.
We took this a step further and treated it as any product initiative by including it in our monthly cycle check-ins. Individual teams shared their progress and accomplishments and any new information gained. These check-ins aim to provide an avenue for teams, including cross-functional partners like Operations and Finance, to share their progress with a broader audience. It also creates a way for folks to get on the same page and better understand the business.
The TCs group also dedicated time to mentoring and coaching other engineers, which allowed individuals to get access to mentors who may not be on their immediate team.
Individuals have used office hours for jamming (aka. mob programming) on something together (e.g., extracting certain parts of the codebase into its pack).
These office hours set an excellent stage for engineers to learn from each other and enable TCs to evaluate how teams have incorporated the technical strategy into their day-to-day work, identify what's working and what's not, and continue refining it.
Most importantly, this enables the entire group to feel like one unit.
It is critical to continue raising the quality bar for our product offerings. Our customers deserve it, but our moral obligation is to continue delivering more value for our customers. It has amazing second-order benefits too. Raising the quality bar makes it easier for the teams to operate the product more consistently, making onboarding and ramping for newer engineers easier and accelerating product development.
Since teams and projects are constantly changing, we must ensure that we continue to grow such efforts. Initially, the TCs had a slightly larger scope (being the owner for work that doesn’t fall on any teams, etc.), resulting in the TCs feeling burnt out. However, as we learned from this process, we were able to refine and have more clarity around expectations.
The Technical Consultants continue defining and driving ambitious technical strategies as part of Benefits Engineering. These initiatives have only increased in scope and impact over time.
Hopefully, I’ve shared a glimpse of the possibilities and some of the learnings you can incorporate.
Serving small businesses has been an enriching journey for many of us at Gusto. If you are interested in helping employers take care of their employees' health & financial wellness and enabling employees with access to affordable and equitable healthcare options, please do reach out.