Battle of the Frameworks: Benchmarking High-Performance HTTP Libraries

Photo by Riccardo Pierri / Unsplash

Have you ever wondered about how high performance HTTP servers will do across different languages? In this post we will try to compare many http frameworks from different languages: Nodejs, Java, C#, and Go.

The world of web development is filled with endless options for HTTP frameworks across programming languages. But with so many choices, how do developers know which ones deliver truly high-performance results? In this blog post, we'll cut through the noise and directly compare some of the top contenders for speed. Looking at popular options in Javascript/Bun, Java, C#, Go, and rust, we will benchmark and evaluate their throughput and response times when put to the test. The frameworks we've selected have a reputation for performance, but we'll see how they stack up across languages.

• Java 21 + vertex 4.4.6

• JS/Bun 1.0.6 + elysiajs 0.7

• C# 12 + dotnet/ASP.NET 8.0 RC2

• Go 1.21.3 + fiber 2.49.2

• Rust 1.73.0 + actix-web 4

By stress testing them, we'll get hard data on their capabilities. This head-to-head comparison focuses on raw speed and scalability with zero tuning, so you can pick the right framework for your next high-traffic web project. Whether you're looking to turbo-charge an API, build lower-latency systems, or squeeze the most out of your servers, this evaluation aims to help you choose a high-performant HTTP framework tailored to your tech stack.

Environment & Test Set up:

The test will be done by spinning the minimal version of the HTTP server that return a hello world response when pining / .

We will run server on a Hetzner Machine:

• OS: Ubuntu 22.04.3 LTS

• Kernel: 5.15.0-86-generic

• Arch: ARM aarch64

• Resources: 4 vcpu & 8 GB Ram

The client load generator will be on a separate machine that has similar properties except resources will be higher 8 vcpu & 16 GB Ram.

Java & Vertex:

We will generate the vertex starter project using the starter site from vertex. Also, we will use java 21-oracle that we just installed using sdkman.

package io.codereliant.performance;

import io.vertx.core.AbstractVerticle;
import io.vertx.core.Promise;

public class MainVerticle extends AbstractVerticle {

  @Override
  public void start(Promise<Void> startPromise) throws Exception {
    vertx.createHttpServer().requestHandler(req -> {
      req.response()
        .putHeader("content-type", "text/plain")
        .end("Hello World!");
    }).listen(80, http -> {
      if (http.succeeded()) {
        startPromise.complete();
        System.out.println("HTTP server started on port 80");
      } else {
        startPromise.fail(http.cause());
      }
    });
  }
}

After just updating the port to 80 instead of 8888 and modify the string return to be Hello World instead of the default text; we build our server using mvn package and run it using java -jar target/performance-1.0.0-SNAPSHOT-fat.jar.

java -jar target/performance-1.0.0-SNAPSHOT-fat.jar
HTTP server started on port 80
Oct 15, 2023 10:56:15 PM io.vertx.core.impl.launcher.commands.VertxIsolatedDeployer
INFO: Succeeded in deploying verticle

Bun & Elysia:

Elysia makes it easier to create a project by just invoking bun create elysia perf-app, then invoke bun run index.ts.

For reference we are using Bun 1.0.6 and Elysia 0.7.0.

import { Elysia } from "elysia";

const app = new Elysia().get("/", () => "Hello World").listen(80);

console.log(
  `🦊 Elysia is running at ${app.server?.hostname}:${app.server?.port}`
);

C# & Dotnet:

For C# 12 and dotnet 8.0 RC, we will use ASP.NET Core Minimal APIs.

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.MapGet("/", () => "Hello World!");

app.Run();

dotnet new web -o perf-app
cd perf-app
dotnet build -c Release
ASPNETCORE_URLS="http://*:80/" ./bin/Release/net8.0/dotnet-app

Golang & Fiber:

Fiber is a Go web framework built on top of Fasthttp, which was designed for high performance.

mkdir go-app
cd go-app
go mod init github.com/codereliant/go-app
go get -u github.com/gofiber/fiber/v2
touch main.go

Then in main.go, we will use the hello world example from teh landing page of https://gofiber.io/ .

for building and running the example, you can use these 2 lines:

go build main.go
./main

Rust & Actix-web:

From the main site of actix-web:

Actix Web is a powerful, pragmatic, and extremely fast web framework for Rust

We can create a project using:

cargo new actix-hello
cd actix-hello

Then replacing `src/main.rs` content with the content below, which is taking from actix-web getting started page:

use actix_web::{web, App, HttpResponse, HttpServer, Responder};

async fn manual_hello() -> impl Responder {
    HttpResponse::Ok().body("Hey there!")
}

#[actix_web::main]
async fn main() -> std::io::Result<()> {
    HttpServer::new(|| {
        App::new()
            .route("/", web::get().to(manual_hello))
    })
    .bind(("0.0.0.0", 80))?
    .run()
    .await
}

Then we just build the binary and run it:

cargo build --release
# .......
# Finished release [optimized] target(s) in 1m 58s
./target/release/actix-hello

Client:

For the client we will use oha an HTTP benchmarking tool written in rust and inspired by Hey.

For how to install oha you can check its installation steps on the github page.

We will run the bombardier with 500 connections for a total of 3 millions requests, we will repeat this experiment 3 times.

oha -c 500 -n 3000000 http://perf-experiment-host/

The response should look something like this:

Results:

For each of the language + framework option above we benchmarked the application 3 times, then we grabbed the best data to compare.

Throughput:

1. Rust/Actix-web leads the pack with the highest throughput, closely followed by Go/fiber.

2. C#/ASP.NET, despite its popularity, lags behind the top performers in this benchmark.

3. Java/Vertex and Bun/Elysia demonstrate comparable and mid-range throughput values.

Latency:

1. 99.9% Latency: Represents the latency at which 99.9% of the requests are processed.

2. 99% Latency: Represents the latency at which 99% of the requests are processed.

3. Mean Latency: Represents the average latency for all requests.

From the graph, a few observations can be made:

• Rust/Actix-web and Go/fiber not only display impressive throughput but also exhibit lower latencies across all three metrics.

• C#/ASP.NET demonstrates relatively lower mean latency compared to some other frameworks, despite its lower throughput.

• There's a noticeable difference between the 99% and 99.9% latencies, highlighting the variability of request processing times within the frameworks.

Conclusion:

In our exploration of the performance of various HTTP frameworks, Rust/Actix-web is obviously a clear winner, getting not only the highest throughput but also super low latencies across all metrics. Following closely was Go/fiber, which impressed with its blend of high request processing speeds and efficient response times. While C#/ASP.NET might not have matched the top 2 in throughput, its mean latency was notably competitive, suggesting that it remains a viable choice for many applications. On the other hand, frameworks like Java/Vertex and Bun/Elysia presented okay performance and they might probably need more tuning to compete with other frameworks.

We are sure that these numbers and comparison might have looked different if we spent some time tuning the language/frameworks with all of their bells and whistles; however, we wanted this comparison to be done with zero tuning.

---