File Processing Pipeline

Overview

Cloudillo processes uploaded files through an asynchronous pipeline that generates multiple variants optimized for different use cases. The system uses FFmpeg for multimedia processing and supports images, videos, audio, and PDFs.

Processing Architecture

File Upload
    ↓
Store original blob
    ↓
Create FileIdGeneratorTask
    ↓
Detect file type (MIME)
    ↓
Generate variants (async)
    ├─ Image: thumbnails, SD, MD, HD
    ├─ Video: transcoded variants, thumbnails
    ├─ Audio: normalized, compressed
    └─ PDF: text extraction, thumbnails
    ↓
Create file descriptor
    ↓
Content-address all variants
    ↓
Return file ID (f1~...)

Supported File Types

Images

Video

Audio

Documents

Variant System

Cloudillo uses a two-level variant system with format <class>.<quality>:

vis.sd   →  class: visual (image), quality: standard definition
vid.hd   →  class: video, quality: high definition
aud.md   →  class: audio, quality: medium

Variant Classes

Quality Levels

Visual Variants (Images)

Video Variants

Video processing also extracts a vis.tn thumbnail from the first few seconds.

Audio Variants

Document Variants

Variant Fallback

When a requested variant isn’t available, the system falls back to lower quality:

Request: vis.hd
Fallback chain: vis.md → vis.sd → vis.tn

File Descriptor Format

File descriptors encode all variant information:

d2,vis.tn:b1~abc123:f=avif:s=4096:r=128x128;vis.sd:b1~def456:f=webp:s=32768:r=720x540;vid.hd:b1~xyz789:f=mp4:s=5242880:r=1920x1080:dur=120.5:br=5000

Processing Presets

Presets define which variants to generate for different use cases:

All presets preserve the original file as orig unless configured otherwise.

FFmpeg Integration

Video and audio processing uses FFmpeg:

Video Transcoding

ffmpeg -i input.mov \
  -c:v libx264 -preset medium -crf 23 \
  -c:a aac -b:a 128k \
  -vf "scale=1280:720:force_original_aspect_ratio=decrease" \
  output.mp4

Audio Transcoding

ffmpeg -i input.mp3 \
  -c:a libopus -b:a 96k \
  output.opus

Audio Extraction

From video files:

ffmpeg -i input.mp4 \
  -vn -c:a libopus -b:a 96k \
  output.opus

Thumbnail Generation

ffmpeg -i input.mp4 \
  -ss 00:00:01 -vframes 1 \
  -vf "scale=150:150:force_original_aspect_ratio=decrease" \
  thumbnail.jpg

Content-Addressing

All variants are content-addressed:

1. Blob level: Raw bytes → b1~{SHA256(bytes)}
2. Descriptor level: Descriptor string → f1~{SHA256(descriptor)}

This enables:

• Deduplication: Identical files share blobs
• Verification: Hashes prove integrity
• Caching: Immutable content can be cached forever

Task Scheduling

File processing uses the task scheduler:

FileUploadTask
    ↓
Creates FileIdGeneratorTask (depends on upload)
    ↓
FileIdGeneratorTask generates variants
    ↓
Action can reference file (depends on FileIdGeneratorTask)

Dependencies ensure actions only reference fully processed files.

Federation Sync

When syncing files across instances:

Metadata-Only Sync

For efficiency, only file descriptors are synced initially:

1. Receive action with file attachment
2. Fetch file descriptor from origin
3. Store descriptor locally
4. Fetch variants on demand

Variant Fetching

Client requests file
    ↓
Check if variant exists locally
    ↓
Yes → Serve from local storage
    ↓
No → Fetch from origin server
    Store locally
    Serve to client

Error Handling

See Also

• Files API - File upload endpoints
• Blob Storage - Storage layer
• Content-Addressing - Hash computation