# ngx_s3_module

**Repository Path**: H-kernel/ngx_s3_module

## Basic Information

- **Project Name**: ngx_s3_module
- **Description**: linux本地存储空间通过nginx扩增模块提供S3服务
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-04-28
- **Last Updated**: 2026-04-30

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# ngx_s3_module

A native nginx C module that implements a complete S3-compatible object storage server. It exposes local filesystem storage via the S3 REST API, running directly inside nginx's event loop with no separate process or thread pool.

## Features

- Full S3 REST API — ~107 data plane operations routed and dispatched
- Service: ListBuckets
- Bucket: CreateBucket, DeleteBucket, HeadBucket, ListObjects (v1/v2), ListObjectVersions, sub-resources (ACL, CORS, lifecycle, policy, tagging, versioning, encryption, …)
- Object: GetObject (with Range), PutObject, DeleteObject, HeadObject, CopyObject, DeleteObjects
- Multipart upload: CreateMultipartUpload, UploadPart, UploadPartCopy, CompleteMultipartUpload, AbortMultipartUpload, ListParts, ListMultipartUploads
- **Bucket lifecycle management**: per-bucket expiry rules via standard S3 XML API; objects are automatically deleted when their age exceeds the configured `Days` value
- Event notifications: webhook (HTTP/HTTPS) and Kafka (`librdkafka`, optional)
- Webhook rollback: if the webhook endpoint returns non-2xx on an object-created event, the uploaded object is automatically deleted
- AWS SigV2/SigV4 authentication (`s3_auth v2|v4|both|off`)
- Zero-copy file serving via nginx sendfile
- Atomic write via temp-file + rename (crash-safe)
- JSON metadata sidecars (`.meta`) and bucket config files

## Dependencies

| Dependency | Notes |
|---|---|
| nginx ≥ 1.24 | Source tree required for building the dynamic module |
| OpenSSL | For SHA-256/HMAC/MD5/Base64; linked via nginx |
| cJSON | Bundled in `third_party/cJSON/` |
| librdkafka | Optional; only needed when `NGX_S3_HAVE_KAFKA=1` |

## Building

### 1. Build as a dynamic module (recommended)

```bash
# Download and extract nginx source (example: 1.28.3)
wget https://nginx.org/download/nginx-1.28.3.tar.gz
tar xf nginx-1.28.3.tar.gz
cd nginx-1.28.3

# Configure nginx with this module
./configure \
    --add-dynamic-module=/path/to/ngx_s3_module \
    --with-http_ssl_module \
    --with-cc-opt='-O2'

# Build only the module (fast)
make modules

# The loadable module is now at:
#   objs/ngx_s3_module.so
```

Copy `ngx_s3_module.so` to your nginx modules directory (e.g. `/etc/nginx/modules/`).

### 2. Build as a static module

```bash
cd nginx-1.28.3
./configure \
    --add-module=/path/to/ngx_s3_module \
    --with-http_ssl_module
make
make install
```

### 3. Development build (symbol check only)

```bash
cd /path/to/ngx_s3_module
make                          # builds bare ngx_s3_module.so for quick iteration
make NGINX_SRC=/path/to/nginx # override nginx source path
```

> **Note:** The dev `make` target produces a bare `.so` useful for compiler feedback, but it lacks the `ngx_modules` symbol required by nginx. For actual deployment, always use `make modules` in the nginx source tree.

### 4. Build with Kafka support

```bash
# Install librdkafka development package first
# Ubuntu/Debian:
apt-get install librdkafka-dev
# RHEL/CentOS:
yum install librdkafka-devel

# Then build with Kafka enabled
cd nginx-1.28.3
./configure \
    --add-dynamic-module=/path/to/ngx_s3_module \
    --with-http_ssl_module \
    --with-cc-opt='-O2 -DNGX_S3_HAVE_KAFKA' \
    --with-ld-opt='-lrdkafka'
make modules
```

## Configuration

### Load the module

Add to the top of `nginx.conf` (dynamic module only):

```nginx
load_module /path/to/ngx_s3_module.so;
```

### Directives

| Directive | Context | Default | Description |
|---|---|---|---|
| `s3` | location | — | Enable the S3 handler for this location |
| `s3_root <path>` | location | — | Filesystem root for object storage (required) |
| `s3_region <name>` | location | `us-east-1` | AWS region name returned in responses |
| `s3_access_key <key>` | location | — | AWS access key ID for authentication |
| `s3_secret_key <key>` | location | — | AWS secret access key for authentication |
| `s3_auth <mode>` | location | `off` | Authentication mode: `off`, `v2`, `v4`, `both` |
| `s3_versioning <on\|off>` | location | `off` | Enable S3 bucket versioning |
| `s3_buffer_size <size>` | location | `64k` | I/O buffer size for streaming |
| `s3_max_object_size <size>` | location | `0` (unlimited) | Maximum allowed object size |
| `s3_notification <type> <endpoint> [topic]` | location | — | Event notification (see below) |

### Minimal example (no authentication)

```nginx
load_module /etc/nginx/modules/ngx_s3_module.so;

events { worker_connections 1024; }

http {
    server {
        listen 9000;

        location / {
            s3;
            s3_root /data/s3storage;
            s3_region us-east-1;
            s3_auth off;
            s3_buffer_size 64k;
            s3_max_object_size 5g;
            client_max_body_size 0;   # required to allow large uploads
        }
    }
}
```

### With SigV4 authentication

```nginx
location / {
    s3;
    s3_root       /data/s3storage;
    s3_region     us-east-1;
    s3_access_key AKIAIOSFODNN7EXAMPLE;
    s3_secret_key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY;
    s3_auth       both;          # accept SigV2 or SigV4
}
```

### With HTTPS (SSL handled by nginx core)

```nginx
server {
    listen 443 ssl;

    ssl_certificate     /etc/ssl/certs/s3.crt;
    ssl_certificate_key /etc/ssl/private/s3.key;
    ssl_protocols       TLSv1.2 TLSv1.3;

    location / {
        s3;
        s3_root       /data/s3storage;
        s3_access_key AKIAIOSFODNN7EXAMPLE;
        s3_secret_key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY;
        s3_auth       both;
    }
}
```

### Event notifications

**Webhook (HTTP or HTTPS):**

```nginx
location / {
    s3;
    s3_root /data/s3storage;
    s3_auth off;
    s3_notification webhook http://127.0.0.1:8080/s3events;
    # s3_notification webhook https://hooks.example.com/s3events;
}
```

The module POSTs an AWS S3-compatible event JSON payload to the webhook URL on every write operation. If the endpoint returns a non-2xx status code for an `s3:ObjectCreated:*` event, the uploaded object is **automatically deleted** (rollback). The request timeout is 5 seconds.

**Kafka (requires `NGX_S3_HAVE_KAFKA=1` build):**

```nginx
location / {
    s3;
    s3_root /data/s3storage;
    s3_auth off;
    s3_notification kafka broker1:9092,broker2:9092 s3-events;
}
```

The third argument is the Kafka topic name. Messages follow the same AWS S3 event JSON format.

**Event payload example (`s3:ObjectCreated:Put`):**

```json
{
  "Records": [{
    "eventVersion": "2.1",
    "eventSource": "aws:s3",
    "awsRegion": "us-east-1",
    "eventTime": "2026-04-29T12:00:00.000Z",
    "eventName": "s3:ObjectCreated:Put",
    "requestParameters": { "sourceIPAddress": "127.0.0.1" },
    "s3": {
      "bucket": { "name": "my-bucket" },
      "object": {
        "key": "path/to/object.txt",
        "eTag": "d8e8fca2dc0f896fd7cb4cb0031ba249",
        "versionId": ""
      }
    },
    "requestId": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
  }]
}
```

### Bucket lifecycle management

Lifecycle rules control automatic object expiry on a per-bucket basis. Rules are stored using the standard S3 `PUT /{bucket}?lifecycle` API and executed by an in-process hourly timer inside each nginx worker.

**Set lifecycle rules:**

```bash
curl -X PUT "http://localhost:9000/my-bucket?lifecycle" \
     -H "Content-Type: application/xml" \
     --data-binary @- <<'EOF'
<?xml version="1.0" encoding="UTF-8"?>
<LifecycleConfiguration>
  <Rule>
    <ID>expire-logs-after-7-days</ID>
    <Status>Enabled</Status>
    <Filter><Prefix>logs/</Prefix></Filter>
    <Expiration><Days>7</Days></Expiration>
  </Rule>
  <Rule>
    <ID>expire-all-after-30-days</ID>
    <Status>Enabled</Status>
    <Filter><Prefix></Prefix></Filter>
    <Expiration><Days>30</Days></Expiration>
  </Rule>
</LifecycleConfiguration>
EOF
```

**Get lifecycle rules:**

```bash
curl "http://localhost:9000/my-bucket?lifecycle"
```

**Delete lifecycle rules:**

```bash
curl -X DELETE "http://localhost:9000/my-bucket?lifecycle"
```

**Behavior:**

- Each nginx worker scans all buckets **once per hour** via an internal `ngx_add_timer` event
- An object is deleted when: `now − last_modified > Days × 86400 seconds`
- `last_modified` is read from the object's `.meta` sidecar; falls back to filesystem `mtime`
- Rules with `<Status>Disabled</Status>` or `<Days>0</Days>` (or omitted `<Expiration>`) are ignored — no objects are deleted
- Buckets with no lifecycle configuration are never scanned
- Deletion is logged at `INFO` level in the nginx error log
- Only `Expiration/Days` is supported; `Transition`, `NoncurrentVersionExpiration`, and `AbortIncompleteMultipartUpload` are ignored

## Storage layout

```
{s3_root}/
├── {bucket}/                       # One directory per bucket
│   ├── {object-key}                # Object data
│   ├── {object-key}.meta           # JSON metadata sidecar (ETag, Content-Type, …)
│   └── .versions/{key}/{version}   # Versioned objects
├── .s3bucket_{bucket}.json         # Bucket configuration (ACL, CORS, policy, …)
└── .s3_multipart/{upload-id}/      # Multipart upload staging area
    ├── 1.data
    ├── 2.data
    └── manifest.json
```

Object keys ending in `.meta` are rejected to prevent collisions with sidecar files.

Bucket names must be DNS-compatible: 3–63 characters, lowercase letters, digits, and hyphens only.

## Client usage examples

### AWS CLI (path-style addressing)

```bash
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
ENDPOINT=http://localhost:9000

# Create bucket
aws s3 mb s3://my-bucket --endpoint-url $ENDPOINT

# Upload object
aws s3 cp file.txt s3://my-bucket/file.txt --endpoint-url $ENDPOINT

# Download object
aws s3 cp s3://my-bucket/file.txt ./file.txt --endpoint-url $ENDPOINT

# List objects
aws s3 ls s3://my-bucket/ --endpoint-url $ENDPOINT

# Delete object
aws s3 rm s3://my-bucket/file.txt --endpoint-url $ENDPOINT

# Multipart upload (automatic for files > 8 MB)
aws s3 cp large-file.bin s3://my-bucket/large-file.bin --endpoint-url $ENDPOINT
```

> When `s3_auth off` is set, any credentials (or none) are accepted.

### curl

```bash
# Create bucket
curl -X PUT http://localhost:9000/my-bucket

# Upload object
curl -X PUT http://localhost:9000/my-bucket/hello.txt \
     -H "Content-Type: text/plain" \
     --data "Hello, World!"

# Download object
curl http://localhost:9000/my-bucket/hello.txt

# Head object
curl -I http://localhost:9000/my-bucket/hello.txt

# Range request
curl -H "Range: bytes=0-99" http://localhost:9000/my-bucket/large.bin

# Delete object
curl -X DELETE http://localhost:9000/my-bucket/hello.txt

# List objects
curl http://localhost:9000/my-bucket/

# Copy object
curl -X PUT http://localhost:9000/my-bucket/copy.txt \
     -H "x-amz-copy-source: /my-bucket/hello.txt"
```

## Running tests

Integration tests require a running nginx instance with the module loaded.

```bash
# Start nginx (using the bundled example config)
nginx -p /path/to/ngx_s3_module -c conf/s3svr.conf

# Run all tests
cd tests
python3 test_integration.py --endpoint http://localhost:9000

# Run a specific test class
python3 -m pytest test_integration.py::TestBucketOperations -v
```

## Supported S3 operations

| Category | Operations |
|---|---|
| Service | ListBuckets |
| Bucket | CreateBucket, DeleteBucket, HeadBucket, ListObjects (v1/v2), ListObjectVersions |
| Bucket sub-resources | GetBucketLocation, GetBucketVersioning, GetBucketACL, GetBucketCORS, GetBucketLifecycle, GetBucketPolicy, GetBucketTagging, GetBucketEncryption, GetBucketWebsite, GetBucketLogging, GetBucketObjectLock, GetBucketNotification, GetBucketReplication, GetBucketInventory, GetBucketAccelerateConfiguration, GetBucketPublicAccessBlock, GetBucketOwnershipControls, PutBucketVersioning, PutBucketACL, PutBucketCORS, PutBucketLifecycle, PutBucketPolicy, PutBucketTagging, PutBucketEncryption, PutBucketWebsite, PutBucketLogging, PutBucketObjectLock, PutBucketNotification, PutBucketReplication, PutBucketPublicAccessBlock, PutBucketOwnershipControls, DeleteBucketCORS, DeleteBucketLifecycle, DeleteBucketPolicy, DeleteBucketTagging, DeleteBucketWebsite, DeleteBucketReplication, DeleteBucketEncryption, DeleteBucketPublicAccessBlock, DeleteBucketOwnershipControls |
| Object | GetObject (Range), PutObject, DeleteObject, HeadObject, CopyObject, DeleteObjects |
| Object sub-resources | GetObjectACL, PutObjectACL, GetObjectTagging, PutObjectTagging, DeleteObjectTagging, GetObjectLegalHold, PutObjectLegalHold, GetObjectRetention, PutObjectRetention, RestoreObject |
| Multipart upload | CreateMultipartUpload, UploadPart, UploadPartCopy, CompleteMultipartUpload, AbortMultipartUpload, ListParts, ListMultipartUploads |

## License

See [LICENSE](LICENSE) for details.