Multi-Registry Setup

Configuring multiple container registries

Multi-Registry Setup Guide

This guide covers setting up and using Cornucopia with multiple package registries simultaneously on a single server.

Overview

Cornucopia supports five package ecosystems:

Registry	Protocol	Use Case
PyPI	PEP 503/691	Python packages
Docker	OCI/Docker API	Container images
Go Modules	Module Proxy Protocol	Go packages
NPM	NPM Registry API	JavaScript/Node packages
Maven	Maven Central compatible	Java/JVM packages

All registries are available simultaneously on a single Cornucopia instance.

Configuration

Create config.yml to enable all registries:

server:
  addr: ":8080"
  base_url: "http://cornucopia.example.com"
  max_upload_size: 104857600  # 100MB

cache:
  dir: "./cache"
  metadata_ttl: 10m
  enable_memory: true

# PyPI Configuration
pypi:
  json_api_url: "https://pypi.org/pypi"
  timeout: 30s
  max_retries: 3

# Docker Configuration
docker:
  registry_url: "https://registry-1.docker.io"
  timeout: 60s
  max_retries: 3

# Go Modules Configuration
golang:
  proxy_url: "https://proxy.golang.org"
  timeout: 30s
  max_retries: 3

# NPM Configuration
npm:
  registry_url: "https://registry.npmjs.org"
  timeout: 30s
  max_retries: 3

# Maven Configuration
maven:
  central_url: "https://repo.maven.apache.org/maven2"
  timeout: 30s
  max_retries: 3

# Authentication for uploads
auth:
  disabled: false
  tokens:
    "pypi-token-123": "pypi-user"
    "docker-token-456": "docker-user"
    "npm-token-789": "npm-user"
    "maven-token-abc": "maven-user"

Run with the configuration:

CORNUCOPIA_CONFIG=./config.yml go run ./cmd/cornucopia

Or use environment variables:

CORNUCOPIA_ADDR=:8080 \
CORNUCOPIA_CACHE_DIR=/data/cache \
CORNUCOPIA_PYPI_JSON_API_URL=https://pypi.org/pypi \
go run ./cmd/cornucopia

Docker Compose

Use docker-compose.yml for quick multi-registry setup:

version: '3.8'

services:
  cornucopia:
    build: .
    ports:
      - "8080:8080"
    volumes:
      - cache:/app/cache
      - ./config.yml:/etc/cornucopia/config.yml
    environment:
      CORNUCOPIA_CONFIG: /etc/cornucopia/config.yml
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3

volumes:
  cache:

Start:

docker-compose up -d

Per-Registry Setup

PyPI (Python)

Client Configuration:

# Via environment
export PIP_INDEX_URL=http://localhost:8080/simple/

# Via ~/.pip/pip.conf
[global]
index-url = http://localhost:8080/simple/

# Via requirements.txt
--index-url http://localhost:8080/simple/
requests==2.31.0

Upload Internal Packages:

# Configure ~/.pypirc
[cornucopia]
repository: http://localhost:8080/legacy/
username: __token__
password: pypi-token-123

# Upload
twine upload -r cornucopia dist/*

Docker (Container Images)

Client Configuration (Linux):

Edit /etc/docker/daemon.json:

{
  "registry-mirrors": [
    "http://localhost:8080"
  ],
  "insecure-registries": [
    "localhost:8080"
  ]
}

Restart Docker and test:

systemctl restart docker
docker pull localhost:8080/ubuntu:22.04

macOS/Windows: Use Docker Desktop settings:

Preferences → Docker Engine
Add registry mirror: http://host.docker.internal:8080

Push Internal Images:

# Tag image
docker tag my-app:1.0 localhost:8080/my-app:1.0

# Push (requires auth)
echo "docker-token-456" | docker login -u docker-user --password-stdin http://localhost:8080
docker push localhost:8080/my-app:1.0

Go Modules

Client Configuration:

# Via environment (one-time)
GOPROXY=http://localhost:8080/go go get github.com/some/module@v1.2.3

# Via ~/.config/go/env (persistent)
echo "GOPROXY=http://localhost:8080/go" >> ~/.config/go/env

# Via go.mod
go env -w GOPROXY=http://localhost:8080/go

Module Proxy Behavior:

# Works seamlessly with existing workflows
go get github.com/some/module
go build ./...
go test ./...

Cornucopia implements the Module Proxy Protocol, compatible with:

GET /module/@v/list                      # List all versions
GET /module/@v/version.mod               # Go module file
GET /module/@v/version.zip               # Module download
GET /module/@v/version.info              # Version metadata

NPM (JavaScript)

Client Configuration:

# Set registry
npm config set registry http://localhost:8080/npm

# Via .npmrc
registry=http://localhost:8080/npm

# Via environment
npm install --registry http://localhost:8080/npm

Install Packages:

npm install react react-dom typescript

# Works with version specifiers
npm install react@^18.0.0

Publish Internal Packages:

# Configure auth in ~/.npmrc
registry=http://localhost:8080/npm
//localhost:8080/npm/:_authToken=npm-token-789

# Publish
npm publish

Maven (Java)

Client Configuration in pom.xml:

<repositories>
  <repository>
    <id>cornucopia</id>
    <name>Cornucopia Maven Mirror</name>
    <url>http://localhost:8080/maven/</url>
  </repository>
</repositories>

Mirror Central in ~/.m2/settings.xml:

<mirrors>
  <mirror>
    <id>cornucopia</id>
    <name>Cornucopia Maven Mirror</name>
    <url>http://localhost:8080/maven/</url>
    <mirrorOf>*</mirrorOf>
  </mirror>
</mirrors>

Authentication in ~/.m2/settings.xml:

<servers>
  <server>
    <id>cornucopia</id>
    <username>maven-user</username>
    <password>maven-token-abc</password>
  </server>
</servers>

Build with Maven:

mvn clean install
mvn test
mvn package

Health Monitoring

Check all registries at once:

curl http://localhost:8080/health

Response:

{
  "status": "ok",
  "registries": {
    "pypi": "healthy",
    "docker": "healthy",
    "golang": "healthy",
    "npm": "healthy",
    "maven": "healthy"
  },
  "cache": {
    "size_bytes": 1024000,
    "items": 42
  }
}

Caching Strategy

Metadata Caching

Cornucopia caches:

Package metadata (versions, dependencies)
File hashes (SHA256, MD5, etc.)
Image manifests
Module checksums

TTL controlled by cache.metadata_ttl (default: 10m).

File Caching

Downloaded packages/images are cached permanently in cache.dir. Implement manual cleanup or integrate with your storage system.

In-Memory Cache

Optional fast cache layer with cache.enable_memory: true. Holds frequently accessed metadata in RAM for sub-millisecond lookups.

Performance Characteristics

With caching enabled:

Metric	Value
Metadata latency	<1ms (in-memory)
File lookup	<10ms (disk)
Cold start	~100ms (upstream)
Concurrent dedup	Unlimited (singleflight)
Memory baseline	~50MB
Cache memory per 1K packages	~1MB

Troubleshooting

Registry Unavailable

Cornucopia falls back to stale cache if upstream is down:

# Upstream fails
docker pull localhost:8080/ubuntu:22.04
# ✓ Returns cached version if available

# Check health
curl http://localhost:8080/health

Checksum Validation Failures

Corrupted cached files are automatically deleted:

# If download fails checksum
# Cornucopia deletes bad cached file and re-fetches
go get github.com/module@v1.2.3  # Retries automatically

Auth/Token Issues

PyPI:

twine upload -r cornucopia dist/*
# Error: 401 Unauthorized
# Fix: Check ~/.pypirc token and CORNUCOPIA_TOKENS config

Docker:

docker push localhost:8080/image:tag
# Error: unauthorized
# Fix: docker login and verify credentials

NPM:

npm publish
# Error: 401 Unauthorized
# Fix: Check ~/.npmrc _authToken and registry config

Migration from Public Registries

From PyPI

# Old: public PyPI
pip install requests

# New: use Cornucopia
pip install --index-url http://localhost:8080/simple/ requests

From Docker Hub

# Old: docker.io
docker pull ubuntu:22.04

# New: via Cornucopia mirror
docker pull localhost:8080/ubuntu:22.04

From Go Proxy

# Old: Default Go proxy
GOPROXY=https://proxy.golang.org go get github.com/module@v1.0

# New: use Cornucopia
GOPROXY=http://localhost:8080/go go get github.com/module@v1.0

From npm

# Old: public npm
npm install react

# New: use Cornucopia
npm config set registry http://localhost:8080/npm
npm install react

From Maven Central

# Old: central.maven.org
mvn clean install

# New: configure mirror in settings.xml (see above)
mvn clean install

Advanced Configuration

Rate Limiting

Configure upstream rate limits to avoid overloading public registries:

pypi:
  max_retries: 3
  timeout: 30s
  # Add per-registry rate limiting in code if needed

Custom Headers

For private upstreams, add auth headers:

docker:
  registry_url: "https://private-registry.example.com"
  auth_header: "Authorization: Bearer token-xyz"

Proxy Networks

Route through a corporate proxy:

export HTTP_PROXY=http://proxy.company.com:8080
export HTTPS_PROXY=http://proxy.company.com:8080
CORNUCOPIA_CONFIG=config.yml go run ./cmd/cornucopia

Multi-Registry Setup

Multi-Registry Setup Guide

Overview

Configuration

Docker Compose

Per-Registry Setup

PyPI (Python)

Docker (Container Images)

Go Modules

NPM (JavaScript)

Maven (Java)

Health Monitoring

Caching Strategy

Metadata Caching

File Caching

In-Memory Cache

Performance Characteristics

Troubleshooting

Registry Unavailable

Checksum Validation Failures

Auth/Token Issues

Migration from Public Registries

From PyPI

From Docker Hub

From Go Proxy

From npm

From Maven Central

Advanced Configuration

Rate Limiting

Custom Headers

Proxy Networks

See Also