File Processing Pipeline

Overview

Cloudillo processes uploaded files through an asynchronous pipeline that generates multiple variants optimized for different use cases. The system uses FFmpeg for multimedia processing, resvg for SVG rasterization, and poppler-utils for PDF handling. Supported media types include images, SVG, videos, audio, PDFs, and raw files.

Processing Architecture

Upload (POST /api/files/{preset}/{file_name})
    ↓
Detect MIME type → Map to VariantClass
    ↓
Validate against preset's allowed_media_classes
    ↓
Route to type-specific handler:
    ├─ Image: Read into memory → thumbnail sync → schedule ImageResizerTask per variant
    ├─ SVG: Sanitize → store as vis.sd → rasterize thumbnail sync
    ├─ Video: Stream to temp → FFprobe → extract frame → thumbnail sync
    │         → schedule VideoTranscoderTask + optional AudioExtractorTask
    ├─ Audio: Stream to temp → FFprobe → schedule AudioExtractorTask per tier
    ├─ PDF: Read into memory → store original → schedule PdfProcessorTask
    └─ Raw: Stream to temp → store as-is (orig variant only)
    ↓
Schedule FileIdGeneratorTask (depends on all variant tasks)
    ↓
Create file descriptor → Content-address all variants
    ↓
Return file ID (f1~...)

The upload handler runs directly (not as a scheduled task). Thumbnails are generated synchronously so clients receive an immediate preview. Additional variants are generated asynchronously via the task scheduler.

Supported File Types

Images

Video

Audio

Documents

Raw Files

Any file type not listed above can be uploaded using presets that allow the Raw variant class (e.g., archive, orig-only). Raw files are stored as-is with no processing beyond content-addressing.

Variant System

Cloudillo uses a two-level variant system with format <class>.<quality>:

Variant Classes

Quality Levels

Variant Fallback

When a requested variant isn’t available, the system falls back to lower quality:

Request: vis.hd
Fallback chain: vis.md → vis.sd → vis.tn

File Descriptor Format

File descriptors encode all variant information:

d2,vis.tn:b1~abc123:f=webp:s=4096:r=256x192;vis.sd:b1~def456:f=webp:s=32768:r=720x540;vid.hd:b1~xyz789:f=mp4:s=5242880:r=1920x1080:dur=120.5:br=5000

Processing Presets

Presets define which variants to generate for different use cases:

Presets that set store_original: true (default, high_quality, archive, podcast, video, orig-only, apkg) preserve the original file as orig. Profile-picture, cover, mobile, and thumbnail-only do not store the original.

The archive and orig-only presets also accept raw (unrecognized) file types. Other presets reject uploads with unsupported MIME types.

FFmpeg Integration

Video Transcoding

ffmpeg -i input.mov \
  -c:v libx264 -preset medium -crf 23 \
  -vf "scale=1280:720:force_original_aspect_ratio=decrease" \
  output.mp4

Audio Transcoding

ffmpeg -i input.mp3 \
  -c:a libopus -b:a 128k \
  output.opus

Thumbnail extraction seeks to 10% of video duration (min 3s) and extracts a single frame, which is then resized through the image processing pipeline.

Content-Addressing

All variants are content-addressed:

1. Blob level: Raw bytes → b1~{SHA256(bytes)}
2. Descriptor level: Descriptor string → f1~{SHA256(descriptor)}

This enables deduplication (identical files share blobs), integrity verification, and permanent caching of immutable content.

Task Scheduling

File processing uses the task scheduler for asynchronous variant generation:

Dependencies ensure actions only reference fully processed files.

Federation Sync

When syncing files across instances, only file descriptors are synced initially. Variants are fetched on demand from the origin server and cached locally.

Error Handling

See Also

• Files API - File upload endpoints
• Blob Storage - Storage layer
• Content-Addressing - Hash computation