celestia-indexer

Celestia Indexer | Celenium

Go-based blockchain indexer and REST API for the Celestia Data Availability (DA) network. It reads data from a Celestia full node, decodes blocks, transactions, messages, events, and blobs, and stores them in a PostgreSQL (TimescaleDB) database. A public Echo HTTP/WebSocket API and a private admin API serve the indexed data.

Architecture

cmd/
  indexer/          # Core indexer daemon
  api/              # Public REST API (port 9876)
  private_api/      # Admin REST API (port 9877)
  celestials/       # Off-chain Celestials metadata indexer
pkg/
  indexer/
    receiver/       # Fetches blocks from CometBFT RPC / REST / WebSocket
    parser/         # Decodes raw blocks, txs, messages, events, blobs
    storage/        # Persists parsed data inside a single DB transaction
    rollback/       # Handles chain reorganizations
    genesis/        # Handles the genesis block
internal/
  storage/          # Domain model structs and storage interfaces
    postgres/       # Bun ORM implementations, migrations, hypertables
node/
  rpc/              # CometBFT RPC client
  api/              # Node REST API client
  dal/              # DAL API client (blob retrieval)

Indexer pipeline: CometBFT RPC/WS → Receiver → Parser → Storage → PostgreSQL

Prerequisites

• Docker and Docker Compose
• Go 1.26+ (for local development and testing)

Quick Start

Clone the repository:

git clone https://github.com/celenium-io/celestia-indexer.git
cd celestia-indexer

Copy the example env file and fill in the required values:

cp .env.example .env
$EDITOR .env

Required environment variables:

Build and start all services:

docker compose up -d

This starts the indexer, public API, private API, TimescaleDB, and Valkey cache.

Services

Indexed Entities

Development

First-time setup:

make init

Run individual services locally:

make indexer      # go run ./cmd/indexer -c ./configs/dipdup.yml
make api          # go run ./cmd/api -c ./configs/dipdup.yml
make private_api  # go run ./cmd/private_api -c ./configs/dipdup.yml
make celestials   # go run ./cmd/celestials -c ./configs/dipdup.yml

Other useful commands:

make build        # build all binaries to /bin
make test         # run all tests (requires Docker for DB integration tests)
make generate     # regenerate mocks and enums
make api-docs     # regenerate Swagger docs (swag init)
make lint         # run golangci-lint
make cover        # generate HTML coverage report
make compose      # docker compose up --build

Run a specific test:

go test ./internal/storage/postgres/... -run TestBlockByHeight -v
go test ./cmd/api/handler/... -timeout 30s

Configuration

The YAML config at configs/dipdup.yml uses ${ENV_VAR:-default} substitution. All settings can be overridden via environment variables or the .env file. Key options beyond the required variables above:

Features

• Full block, transaction, and message indexing
• Blob and namespace tracking
• Validator, staking, and governance indexing
• IBC transfer and channel indexing
• Hyperlane cross-chain message indexing
• Chain rollback handling
• TimescaleDB hypertables for time-series performance
• WebSocket real-time notifications
• Public REST + WebSocket API with Swagger docs
• Private admin API
• Valkey/Redis response cache
• Deterministic IDs (no autoincrement sequences for tx/messages)

License