The Goblin Core chicken, a black-and-white Silkie

Redis semantics, specialized properly

Goblin
Core

A compact Redis®-compatible server for strings, sets, sorted sets, hashes, lists, sparse arrays, and Pub/Sub over TCP with kernel sockets or XLIO Ultra, UDS, ringbuffers, one-sided RDMA, and libfabric/EFA. Spend less to run the hot state you already have, or take Redis-shaped operations down to triple-digit nanoseconds.

New in v0.10.4 DPU-side Pub/Sub processing: the BlueField matches subscriptions and performs local-first fanout on-card; the host-side server remains authoritative for everything else. Measured on the DPU path: 12.5 µs p50 Pub/Sub with native XLIO Ultra over direct 100 GbE. Benchmark and raw data.
New in v0.10.2 Libfabric RDM: RESP2 and SBE over EFA, tcp, or verbs;ofi_rxm. 100 Gb personal-lab benchmark. Native AWS fast follow. Measured: 23.06 µs average median across eight operations on native EFA.
Build from source Read the numbers Feed the goblinsChickens eat software bugs.
Apache 2.0C++23RESP + opt-in SBETCP/TLS + XLIO Ultra + UDS + Ringbuffer + RDMA + libfabric/EFA
28.65 BSET RSS/member vs 44.90 B for the leanest incumbent
48.67 BHSET RSS/field vs 60.88 B on Redis 8.8
220 nssingle-element SBE ring round trip
3.39Mpipelined SBE ring ZADD/sec on one serving core

Two roads into the same engine

Choose the overhead you are done paying.

Goblin Core keeps familiar Redis command semantics while specializing the storage and transport underneath them. RESP and opt-in trusted-fabric SBE work over TLS-capable TCP, native XLIO Ultra TCP, Unix-domain sockets, shared-memory rings, polled one-sided RDMA, and libfabric reliable datagrams including AWS EFA. Run it like a lean server, or put it beside your process and leave the kernel off the hot path.

01 / GROWING PRODUCTS

Your leaderboard should not become a fundraising event.

When sorted sets and hashes are scaling faster than the budget, packed layouts turn infrastructure growth back into a product problem instead of a RAM bill.

Interface
RESP or opt-in SBE over TCP, UDS, rings, RDMA, or libfabric/EFA
Best at
Leaderboards, rankings, hot hashes
Measured
763.8 vs 1,486.7 MB: 48.6% less RSS than Redis 8.8 on Lichess
See 14.3 billion updates →
02 / LATENCY-SENSITIVE SYSTEMS

Assemble real infrastructure before you invent a protocol.

Use the same Redis-shaped operations over any transport, then enable SBE over a trusted shared-memory ring for the lowest latency. You get hashes, sorted sets, TTL semantics, and synchronous calls fast enough to keep the obvious code obvious.

Interface
RESP or opt-in SBE over TCP, UDS, rings, RDMA, or libfabric/EFA
Best at
Market data and execution-adjacent state
Measured
~220 ns HSET/ZADD round trip
Inspect the latency path →
Field test / Lichess

One leaderboard. 21.6 million players.

Every rated game in Lichess history became a ZADD: 14.3 billion updates across 12 years and 9 months. The final ordering matched Redis member for member and score for score. Goblin held it in 763.8 MB.

Read the full replay and methodology →
FINAL RESIDENT MEMORY / LOWER IS BETTER
Goblin Core763.8 MB
Dragonfly1,187.8 MB
Redis 8.81,486.7 MB
Valkey 9.11,450.7 MB
Redis 7.2.42,058.6 MB

The low-latency wire

One cache-line handoff. No syscall. No ASCII.

RESP and opt-in SBE are both available over TCP, Unix-domain sockets, and rings. The path below is the fastest combination: on Linux, pin client and server to separate cores and write a typed SBE request into shared memory; Goblin Core busy-polls, executes it, and returns the reply through the companion ring. Between hosts, choose polled one-sided RDMA rings or keep ordinary TCP on the wire while accelerating both endpoints with native XLIO Ultra.

01 / CLIENT

Native call

A one-header C++ client encodes doubles and integers directly.

02 / SHARED MEMORY

SQ / CQ rings

Cache-line-isolated indexes move requests and replies without the kernel.

03 / GOBLIN CORE

Jump-table dispatch

Known message types reach packed hashes and sorted sets with no tokenization.

What the number means

About 220 ns is a full, synchronous, unpipelined request and reply for a one-field HSET or one-member ZADD on a pinned Threadripper PRO 5995WX.

What it costs

The server busy-polls and owns a core. Linux core isolation provides the reliable low-latency deployment; macOS is useful for development but cannot offer the same scheduler control.

Same host. Same timer. Every incumbent.

Read the full latency shootout →
33–40×faster median with Goblin's SBE shared-memory ring than the fastest RESP Unix-domain-socket result across SET/GET, HSET/HGET, and ZADD/ZSCORE
0.24–0.28 µsring p99 across all six operations; the fastest UDS p99s are 9.24–12.10 µs

Published measurements / reproducible artifacts

Storage, transport, and latency.

Current-tree runs cover structure build cost, resident memory, repeated application operations, and the network paths that carry them. Every report names its transport, pipeline depth, buffer size, unsupported commands, and measurement source.

01 / Sorted sets

Ranked state without the usual RAM bill.

52.32 B/member

One million members after packing. RESP/TCP ZADD reaches 1.06M/s; the equally pipelined 2 MiB SBE ring reaches 3.39M/s.

Open ZSET report →Raw JSON
02 / Hashes

Efficient by default, bounded-tail when selected.

48.67 B/field

The efficient million-field hash uses 20.1% less RSS than Redis 8.8. RT builds at 2.54M fields/s and holds mixed p99.9 to 16.14 µs.

Open HSET report →Raw JSON
03 / Lists

Segmented for space. PMA for the middle.

1.99× LINDEX

The default segmented list is 1.99× the fastest incumbent at the middle of 100k items. Its RSS delta is 25.35 B/item versus 21.42 B for the leanest incumbent.

Open LIST report →Raw JSON
04 / Sets

Membership that stays cache-sized.

28.65 B/member

One million members use 36.2% less RSS than the leanest incumbent. RESP/TCP SISMEMBER reaches 2.44M/s; the 2 MiB SBE ring reaches 6.73M/s.

Open SET report →Raw JSON
05 / Sparse arrays

Memory-first Classic. Bounded-tail RT.

36.11 µs P99.99

Across 500k individual RESP/TCP appends, resident-locked RT beats Redis 8.8's 54.48 µs tail. Dense RT builds at 7.09M values/s and 29.71 B/value; Classic keeps sparse arrays lean.

Open ARRAY report →Raw JSON
06 / BlueField DPU-side Pub/Sub

Process local Pub/Sub on the DPU.

12.5 µs p50

A DPU-resident BlueField-2 edge—not the host-side server—does subscription matching and local-first fanout. Native XLIO Ultra measures 12.5 µs p50 over direct 100 GbE; a kernel TCP client measures 107.8 µs.

Open BlueField report →Pub/Sub CSVPING CSV

Comparison discipline: HSET is RESP2/UDS throughout. LIST, SET, and ZSET compare Goblin RESP/TCP with RESP/TCP incumbents, then show Goblin's equally pipelined SBE ring separately. ARRAY is RESP/TCP with native AR* commands throughout; unsupported engines receive an X, never a hash fallback. Runs are medians on the same dedicated AMD 5995WX host, with one serving core per engine and process RSS read from Linux rather than trusted server counters.

More transport evidence: BlueField DPU-side Pub/Sub · native AWS EFA latency · local libfabric provider matrix · native XLIO Ultra over 100 Gb Ethernet · Pub/Sub latency, fanout, and memory · unpipelined ring latency shootout · same-node versus remote-ring NUMA cost

Field notes

The workloads behind the numbers.

Long-form notes on what was measured, why the workload matters, and where the design still has honest limits.

Leaderboard / Memory

14.3 billion Lichess rating updates in one leaderboard

Nearly thirteen years of real player ratings, replayed chronologically into one sorted set and checked against Redis member for member.

Read the leaderboard story →
Shared memory / Latency

Sub-microsecond round trips on the shared-memory ring

How SBE, busy polling, and a cache-line handoff bring synchronous Redis-shaped operations into triple-digit nanoseconds.

Read the ring latency story →
InfiniBand / RESP

Redis commands between 16-year-old machines in 2.7 microseconds

Another physical server over an $80 InfiniBand card is only 553 nanoseconds slower than talking to a CPU two NUMA hops away.

Read the RDMA story →

Direction, stated plainly

Redis semantics, without the old overhead.

Goblin Core is growing toward the full Redis command surface, implemented with tighter memory layouts and faster execution paths. Today it already covers the core data and scripting workflows. Adaptive-PMA and segmented-listpack lists shipped July 13, one week ahead of schedule; in the current 100k benchmark, both list engines lead all eight fixed-command probes, while the default segmented implementation reaches 1.99× the fastest incumbent at middle-list indexing and accounts for 17.16 bytes per item. Pub/Sub over RESP2, RESP3, and SBE shipped July 15, five days ahead of schedule.

The deliberate boundary is infrastructure policy: durable write logs belong in Kafka, where logging is the product. Firehose replication now streams canonical writes over TCP, UDS, shared-memory rings, or RDMA, including replica-to-replica chains. Pair it with native Kafka journaling and replay for a snapshot -> durable log -> live-stream recovery path that completes before client listeners open.

Keys and encoded values use compact 16-bit lengths. Raw values hold up to 65,534 bytes; optional LZ4 admits larger compressible values while the encoded form fits. Everything larger belongs in purpose-built blob storage; Goblin Store provides a fast, cost-effective path for those objects.

Available now

Deliberate boundaries

  • No append-only log
  • No cluster mode
  • No automatic failover
  • Kafka owns durable history
  • Save/load owns restart points

Source-only release

Build it. Measure it.

Goblin Core is Apache 2.0 licensed C++23. Choose authenticated RESP or trusted-fabric SBE over TCP or a Unix-domain socket, then opt into shared-memory rings wherever owning the core makes sense. The protocol choice stays independent of the transport.

~/goblin-core
$ git clone https://github.com/adamdeprince/goblin-core
$ cd goblin-core
$ cmake -S . -B build-release -DCMAKE_BUILD_TYPE=Release
$ cmake --build build-release
$ ctest --test-dir build-release --output-on-failure
$ ./build-release/goblin-core --port 6379
Install from source → Documentation → Architecture → Release history → Source releases →