Embedding Rack apps into Sidekiq

Adding Sidekiq Web UI into a Rails app is pretty straightforward:

# config/routes.rb
require "sidekiq/web"
Rails.application.routes.draw do
  mount Sidekiq::Web => "/sidekiq"
  # dont forget to protect sidekiq web ui with authentication/authorization
end

Sidekiq::Web is essentially a Rack app that mounts into Rails routing and serves any requests matching /sidekiq/* path. To access the UI we open up http://myapp.com/sidekiq, and that's pretty much end of the story, it just works in the same server (Puma/Unicorn/Thin etc) process as your application.

For applications that don't run web services, ie. don't expose any client-facing pages, we might still want to have access to Sidekiq UI or any additional Rack apps. There are few ways to go about it:

Run a standalone process via rackup and only serve Sidekiq::Web app.
Embed the Sidekiq::Web into Sidekiq server process.

Embedded HTTP server

Sidekiq workers are usually started with bundle exec sidekiq ... command, so one easy way to embed another blocking process inside is to spin up a separate ruby thread. We're gonna run a Webrick HTTP server, and plug in necessary Rack applications as part of startup server configuration hook.

# in sidekiq initialization file
require "sidekiq"

Sidekiq.configure_server do |config|
  config.redis = { url: ENV.fetch("REDIS_URL", "redis://localhost:6379/0") }

  config.on(:startup) do
    if ENV["SIDEKIQ_WEB_ENABLED"] == "1"
      require_relative "sidekiq_embedded_server"
    end
  end
end

HTTP server auto-start is gated by a SIDEKIQ_WEB_ENABLED environment variable in case if we need to disable it without code changes. Now, the sidekiq_embedded_server.rb file:

require "webrick"
require "rack"
require "sidekiq/web"
require "sidekiq/prometheus/exporter" # <-- additional rack app

# Sidekiq web app requires the cookie for CSRF protection
Sidekiq::Web.use(Rack::Session::Cookie, secret: "<REPLACEME>")

# Our http server will run on 0.0.0.0:9292 by default
listen_port = ENV.fetch("SIDEKIQ_WEB_PORT", 9292).to_i

server = WEBrick::HTTPServer.new(Port: listen_port).tap do |s|
  # Serve standard sidekiq UI
  s.mount("/", Rack::Handler::WEBrick, Sidekiq::Web)

  # Expose additional endpoint with sidekiq metrics
  s.mount("/metrics", Rack::Handler::WEBrick, Sidekiq::Prometheus::Exporter)
end

Thread.start { server.start }

Once the sidekiq process is up and running we should be able to hit the UI:

curl http://localhost:9292/        # <- sidekiq web
curl http://localhost:9292/metrics # <- prometheus metrics

A separate thread is not ideal in some scenarios. For example, if sidekiq process runs out of memory (or just crashes), the HTTP server will go down along with it. Also, running additional sidekiq processes on the same machine wont be possible due to HTTP port already being in use. Though, for simple use cases we can still get away by embedding the server into sidekiq and keep the complexity at bay.

Use this wisely!