MinIO as a Self-Hosted S3 Backend in Docker
S3 is the de facto standard for object storage, but the AWS dependency and egress costs add up fast. MinIO is an S3-compatible object store you run yourself. Your app code changes nothing: same SDK, same API calls, different endpoint URL. This post covers running MinIO in Docker with a persistent volume, health checks, and a Console UI, then connecting a Python app to it using boto3. Assumed knowledge: Docker Compose basics, basic familiarity with S3 concepts (buckets, objects, access keys).
Prerequisites¶
- Docker Engine 24+
- Docker Compose v2
- A shared Docker network your app containers can join (this post uses
infra_net) - Traefik v3 running as your reverse proxy
Versions used in this post
MinIO latest (RELEASE.2024+) · Docker Compose v2 · boto3 1.34+
Related posts
This post uses infra_net from Self-Hosted Dev Infrastructure with Docker Compose. If you're running MinIO standalone, create the network first: docker network create infra_net.
Background¶
MinIO implements the Amazon S3 REST API specification. Any client that speaks S3: boto3, the AWS CLI, @aws-sdk/client-s3, the Laravel S3 storage driver, Terraform's S3 backend. All of them work against MinIO by changing one config value, the endpoint URL.
MinIO runs in two modes. Single-node mode (used here) runs one process writing to one volume. It is simple, low-overhead, and sufficient for dev stacks and small production workloads. Distributed mode spans multiple nodes for high availability and erasure coding. That matters when you cannot afford data loss on hardware failure. It does not matter for a team sharing a dev server.
Binding the S3 API to 0.0.0.0 exposes your access keys
MinIO's S3 API authenticates with access key and secret key only. There is no IP allowlist, no MFA, no session tokens at the API level. Binding to 0.0.0.0 means the keys are the only thing between your objects and anyone on the internet. Bind to 127.0.0.1 and let app containers reach MinIO through the Docker network.
Implementation¶
1. MinIO service definition¶
minio:
image: minio/minio:latest
container_name: minio
restart: unless-stopped
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USER}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
volumes:
- minio_data:/data
networks:
- infra_net
ports:
- "127.0.0.1:9000:9000"
- "127.0.0.1:9001:9001"
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 15s
timeout: 5s
retries: 1
start_period: 15s
--console-address ":9001" pins the Console to a fixed port. Without it, MinIO binds the Console to a random port on every restart, breaking SSH tunnel aliases and any tooling that expects a consistent address.
start_period: 15s matters here. MinIO takes several seconds to initialize on first boot while writing the data directory. Without it, Docker marks the container unhealthy before initialization completes.
2. Environment variables¶
Root credentials cannot be rotated without data loss
MinIO writes the root credentials into the data directory on first initialization. If you change MINIO_ROOT_USER or MINIO_ROOT_PASSWORD and restart, MinIO ignores the new values. The old credentials are locked in. To change them, you must delete the data volume and reinitialize from scratch. Set strong credentials before first boot and use service accounts for all application access.
3. Creating a bucket and service account¶
Access the Console at http://localhost:9001 via SSH tunnel:
Log in with the root credentials.
Create a bucket: navigate to Buckets, then Create Bucket. Set the access policy to Private. Public buckets expose all objects to unauthenticated HTTP requests.
Create a service account: navigate to Identity, then Service Accounts, then Create Service Account. Copy the Access Key and Secret Key immediately. MinIO only shows the secret once. These go in your app's .env, not the root credentials.
One service account per application
Separate service accounts limit blast radius. If one app's credentials are compromised, revoke that account only. Root credentials in application code is a hard no.
4. Connecting a Python app¶
boto3 works against MinIO without any changes to how you call it. The only required difference from standard AWS usage is endpoint_url:
import boto3
from botocore.config import Config
s3 = boto3.client(
"s3",
endpoint_url="http://minio:9000",
aws_access_key_id=settings.MINIO_ACCESS_KEY,
aws_secret_access_key=settings.MINIO_SECRET_KEY,
region_name="us-east-1",
config=Config(signature_version="s3v4"),
)
region_name is required by boto3 but ignored by MinIO. "us-east-1" is a safe placeholder.
For apps running outside Docker, swap the endpoint:
Presigned URLs embed the endpoint hostname at generation time
A presigned URL generated with endpoint_url="http://minio:9000" contains minio as the hostname. That DNS name only resolves inside Docker. Any presigned URL returned to a browser or mobile client will 404. Set MINIO_SERVER_URL to your public hostname before generating presigned URLs for external use:
5. Uploading and retrieving objects¶
def upload_file(local_path: str, object_key: str, bucket: str = "app-uploads") -> str:
s3.upload_file(local_path, bucket, object_key)
return object_key
def get_presigned_url(
object_key: str, bucket: str = "app-uploads", expires: int = 3600
) -> str:
return s3.generate_presigned_url(
"get_object",
Params={"Bucket": bucket, "Key": object_key},
ExpiresIn=expires,
)
6. Using the AWS CLI against MinIO¶
aws configure --profile minio
# AWS Access Key ID: your-service-account-key
# AWS Secret Access Key: your-service-account-secret
# Default region name: us-east-1
# Default output format: json
# List buckets
aws --profile minio --endpoint-url http://localhost:9000 s3 ls
# Upload a file
aws --profile minio --endpoint-url http://localhost:9000 s3 cp file.txt s3://app-uploads/
# List objects in a bucket
aws --profile minio --endpoint-url http://localhost:9000 s3 ls s3://app-uploads/
Testing and Verification¶
Confirm the container is healthy:
Run a full upload-download round-trip:
echo "hello minio" > /tmp/test.txt
aws --profile minio --endpoint-url http://localhost:9000 \
s3 cp /tmp/test.txt s3://app-uploads/test.txt
aws --profile minio --endpoint-url http://localhost:9000 \
s3 cp s3://app-uploads/test.txt /tmp/test-retrieved.txt
diff /tmp/test.txt /tmp/test-retrieved.txt
# no output means files are identical
Verify Console access:
Pitfalls¶
No named volume means no data on restart
Without minio_data:/data, MinIO writes to the container's ephemeral filesystem. Every docker compose down deletes all stored objects. Named volumes survive container restarts and recreation. If you use a bind mount instead, ensure the host directory exists with correct permissions before MinIO starts. It will not create it.
MinIO healthcheck requires mc, which is bundled in the official image
["CMD", "mc", "ready", "local"] works because the MinIO image ships with the mc client. If you're using a custom or stripped image, fall back to curl -f http://localhost:9000/minio/health/live.
Presigned URL hostname must be externally resolvable
This is the most common MinIO gotcha in containerized setups. A presigned URL generated with endpoint_url="http://minio:9000" contains minio as the hostname. That name only resolves inside the Docker network. Set MINIO_SERVER_URL to your real hostname before generating any URL intended for external clients.
Production Considerations¶
Enable TLS by mounting a certificate and key at /root/.minio/certs/public.crt and /root/.minio/certs/private.key. MinIO picks them up automatically on startup. The only required app-side change is swapping http:// to https:// in the endpoint URL.
Enable Prometheus metrics by setting MINIO_PROMETHEUS_AUTH_TYPE=public and scraping http://minio:9000/minio/v2/metrics/cluster. The endpoint exposes object counts, storage utilization, request rates, and error rates. Enough to know when you are approaching capacity before you hit it.
Enable bucket versioning for any bucket storing user uploads. Versioning turns every overwrite into a new object version and gives you soft-delete and recovery without a separate backup strategy. Pair it with a lifecycle policy that expires non-current versions after 30 days to prevent unbounded storage growth.
Lifecycle policies also handle incomplete uploads. Presigned PUT URLs for client-side uploads will sometimes never complete: the user abandoned the form, the network dropped, the client crashed. A lifecycle rule that expires objects tagged status=pending after 24 hours keeps the bucket clean automatically.
Wrapping Up¶
MinIO gives you S3-compatible object storage with no AWS dependency and no per-request egress cost. Set endpoint_url to http://minio:9000, create service accounts through the Console, and your application code never knows it is not talking to AWS.
The next piece of the stack is email testing in development:
