flamegraph: Anything... and that's it!

Have you ever found yourself wanting to understand why a block of Ruby code, a particular web request, or a certain controller action was running so slowly? Maybe you’ve used some profiling tools like speedscope in the past, but found it cumbersome, or you’ve tried but failed to figure out how to use them.

Introducing Singed, a new profiling frontend from Gusto written by Josh Nichols (aka @technicalpickles). Singed was built to be a swiss-army-knife frontend to several tools (stackprof, rbspy, and speedscope), helping you to easily capture and view flamegraphs, where before you had to write a bunch of boilerplate or temporary code just to get started.

Why Another Profiling Tool?

Singed adds a new method on Kernel, available everywhere:

Run that code and Singed will capture the flamegraph and open a new browser window with the output in speedscope, the same visualization tool used by rack-mini-profiler.

Primarily, most Ruby developers who want to look at flamegraphs use rack-mini-profiler, which is an excellent and easy to use frontend for stackprof. Just add ?pp=flamegraph to the end of any URL and whiz-bang, you’ve got a flamegraph open in your browser.

However, we often want to profile things which are not just the current URL in our browser’s navigation bar.

For example, we may want to profile a GraphQL request, which is made through a gateway like Apollo. We may not be able to easily find the URL that Apollo Gateway is using to make it’s request, so it’s difficult to replicate and use rack-mini-profiler to capture a flamegraph.

Or, we might want to just profile a block of Ruby code or a background job, or even a test. You could write your own code to do this and call stackprof yourself, but it’s a good few dozen lines of boilerplate to redirect the output to a file, and then you still probably have to open it in your browser manually.

Why Use Speedscope, Stackprof and rbspy?

Singed is basically a wrapper around three different tools:

  • Speedscope, a flamegraph visualizer.
  • stackprof, a sampling profiler for Ruby.
  • rbspy, another sampling profiler for Ruby, written in Rust.

Speedscope is an excellent flamegraph visualizer. It’s extremely fast and scales well to even the most complex flamegraph. Its three different visualization modes - Time Order, Left-Heavy and Sandwich - provide many different views onto your profiling data and help you to understand it from all perspectives. But, perhaps most useful at a large company like Gusto, speedscope makes it very easy to share flamegraphs around the team: simply use the export and import functions to portably share simple JSON files around with your team members so everyone can get a look at the same profile on their own machines.

Speedscope's visualizer is beautiful and quite fast, even on large profiles.

stackprof is a sampling profiler for Ruby. It’s a C-extension for the Ruby VM, which means it has to be “started” with your Ruby process at the same time. rbspy works differently - it actually uses sudo privileges to read the memory of the Ruby process from the outside. As such, it can be attached at any time to a Ruby process, which is why Singed uses it to profile Ruby processes from the CLI.

Sampling profilers like stackprof and rbspy are a great choice because they scale quite well to larger codebases. Tracing profilers, like ruby-prof, tend to fail at Gusto-scale due to the huge amounts of code involved in running our application.

Finally, stackprof and rbspy have a great amount of support in the community. We don’t have to do our own upgrading of the profiler to support new Ruby versions, for example.

A Quick Feature Tour

Let’s take a quick look at what Singed can actually do. One of my favorite ways to use Singed is to just profile a simple block of code in our Rails app.

For example, let’s say I wanted to profile a background job, MyWorker. I would create a file called myprofile.rb, put it in our Rails application root, and then add:

Now, all I have to do is run:

… and the rails runner utility will boot my app, run the script, and then Singed will automatically record a profile and open it in my browser. This saves me about 5 minutes of writing boilerplate code to take a Stackprof profile and turn it into JSON, and saves me from having to drag and drop a file into Speedscope every time I run a profile. It’s brilliant.

Another common thing I run into with profiles is that it’s sometimes quite complicated to set up the state required to profile a piece of code. Maybe you just need a very specific database state or lots of code needs to be run first. Often, this setup already exists in our tests.

Singed makes it really easy to profile a test. This means you can, of course, profile and optimize your test suite, but really I find it more useful for profiling code that needs a lot of particular setup:

Running this example will do the same thing as the rails runner example: it opens up a flamegraph in my browser. Wonderful! Now profiling the most complex code is easy if I already have a test case for it.

Another use case for Singed is profiling requests. At Gusto, we have a GraphQL gateway (Apollo) that makes requests to our Rails backend. Sometimes, I want to profile those requests, but it’s quite difficult to do so with rack-mini-profiler because I don’t know the URL and request body that the Apollo backend is sending to Rails. Instead, with Singed, what I do is:

  • Trigger the GraphQL backend request from the frontend.
  • In Chrome DevTools, I copy that request “as fetch”. This allows me to paste it into my Javascript console and trigger this one GraphQL request on-demand.
  • I stop and restart my backend server, this time adding the SINGED_MIDDLEWARE_ALWAYS_CAPTURE=1 environment variable.
  • I trigger the request again in my Javascript console.

Voila! Now my browser opens a flamegraph for that GraphQL request. You can also add an X-Singed: true header to requests, which might be easier depending on your situation.

You can also easily profile a Rails application boot:

... and voila, a new profile opens in my browser window, powered by rbspy.

A Profiler Frontend With Many Uses

Hopefully I’ve convinced you to give Singed a try. I use it almost daily here at Gusto.

If you’ve found this interesting, check it out on GitHub.