Cloudillo Documentation

Welcome to Cloudillo

Cloudillo is an open-source, decentralized collaboration platform that empowers users to maintain complete control over their data while collaborating globally. Whether you’re building collaborative applications, hosting your own instance, or exploring decentralized architectures, this documentation guides you through every aspect of Cloudillo.

Choose your path below:

Open collaborative document formats

There are no open standards for how real-time collaborative documents should be structured. We are designing open, documented CRDT-based formats for presentations, spreadsheets, whiteboards, and more — and we are looking for people who care about document freedom to help.

Get involved — Learn about the initiative and how you can contribute.


👥 For Everyone: Getting Started

New to Cloudillo? Start here to understand what it is and how to get up and running.

  • What is Cloudillo? — Understand Cloudillo’s vision: global accessibility with local control, revolutionary authentication, and seamless interconnectivity
  • Prerequisites — System requirements and what you need to know before getting started
  • Getting Started — Quickstart guide to begin using Cloudillo
  • Install Yourself — Self-hosting guide for those who want to run their own instance
  • First Steps — Your first interactions and setup on Cloudillo
  • Status Page — Service status and availability

💻 For Developers: API & Integration

Build applications on top of Cloudillo using our comprehensive API and client libraries.

Quick Start for Developers

Client Libraries

REST API Reference

Real-Time APIs

Developer Guides


🏗️ Technical Deep Dives: Architecture

Understand Cloudillo’s technical architecture, data models, and how the system works internally.

Core Concepts & Fundamentals

Data Storage & Access

Actions & Federation

Runtime Systems


📄 Document Formats

Detailed specifications of the CRDT document formats used by each Cloudillo application.


🔧 Implementation & Operations


🚀 Next Steps

Subsections of Cloudillo Documentation

Get Involved

You can open a collaborative presentation in one app and continue editing it in another — as long as both apps speak the same format. But today, no open standard defines how a real-time collaborative document should be structured. Every platform invents its own proprietary model, and your documents are trapped inside whichever tool you started with.

We are designing open, CRDT-based document format specifications so that collaborative documents can move freely between applications. We need people who care about document freedom to help us get it right.

The problem: no open standards for collaborative documents

Traditional document formats like ODF and OOXML define how a finished document looks — paragraphs, cells, slides. But they have no concept of the structures that make real-time collaboration work: CRDT state, concurrent edit resolution, operational history, or presence information.

As a result, every collaboration platform builds its own proprietary data model:

  • Google Docs, Sheets, and Slides use internal formats that are never exposed
  • Microsoft 365 layers proprietary real-time structures on top of OOXML
  • Notion, CryptPad, ONLYOFFICE, and others each have their own approach
  • None of these formats are documented, interoperable, or standardized

The consequence is vendor lock-in at the collaboration layer. Even when the static export format is open, the collaboration data model is not. You cannot take a collaboratively edited document from one platform and continue collaborating on it in another without losing structure, history, and concurrent editing capability.

Document Freedom Day 2026

Document Freedom Day is March 25, 2026 — an annual celebration of open document standards and the right to communicate freely. The need for open collaborative document formats is exactly the kind of challenge DFD exists to highlight.

What we are building

We are creating open, documented format specifications for collaborative document types:

  • Prezillo — Collaborative presentations (14 object types, style system, templates)
  • Calcillo — Collaborative spreadsheets (cells, formulas, multi-sheet)
  • Ideallo — Collaborative whiteboards (9 object types, linked copies)
  • Notillo — Collaborative notes (planned)
  • Quillo — Collaborative rich text editor (planned)

These formats are built on Yjs CRDTs (Conflict-free Replicated Data Types), which allow multiple users to edit the same document simultaneously without conflicts and without a central server deciding the outcome.

Each specification documents the exact data structures, field names, types, and relationships — enough for any developer to build a compatible reader, writer, or editor. See the Document Formats overview for the full specifications and the CRDT Design Guide for the patterns and principles behind the format design.

Early stage

These specifications are working documents, not ratified standards. The formats are implemented in Cloudillo apps and are actively used, but they have not gone through a formal standardization process. Community review and feedback at this stage has the most impact — the designs are mature enough to be meaningful but flexible enough to incorporate improvements.

How you can help

Review and critique the format specifications

Read through the Prezillo, Calcillo, and Ideallo format specs. Look for ambiguities, missing edge cases, unnecessary complexity, or things that would make implementation difficult. File issues or start discussions on GitHub.

Design formats for new document types

Notillo (notes) and Quillo (rich text) are planned but not yet specified. If you have experience with collaborative text editing, rich text data models, or CRDT-based editors, your input on these formats would be valuable. The CRDT Design Guide describes the patterns we follow.

Test concurrent editing scenarios

Collaborative formats need to handle multi-user editing, conflict resolution, offline edits, and large documents gracefully. Testing these scenarios and reporting issues helps ensure the formats work in practice, not just on paper.

Contribute to standardization efforts

If you have experience with standards bodies (OASIS, W3C, IETF) or with the ODF/OOXML standardization process, your perspective on how to move these specifications toward formal standardization is welcome.

Spread the word

Share this page with people who care about document freedom, open standards, and interoperability. The more eyes on these formats, the better they will be.

Where to start

About Cloudillo

Cloudillo is an open-source, decentralized collaboration platform where each user or organization hosts their own data while collaborating globally through federation. The collaborative document formats described here are part of that platform, but the formats themselves are designed to be useful to anyone building collaborative tools — you do not need to use Cloudillo to benefit from or contribute to these specifications. Learn more at What is Cloudillo?.

See also

Subsections of The Basics

What Is Cloudillo

Early Development

Cloudillo is under active development. The Rust-based server is 90% complete, with core functionality operational but some API endpoints still in progress. The platform is suitable for developers and early adopters but not yet recommended for production use.

Cloudillo is a versatile Decentralized, Self-Hosted, Open-Source Application Platform designed to empower your creativity, facilitate seamless collaboration, and enable easy sharing.

Cloudillo can be:

  • Your Document Store: Store all your documents securely, accessible only to you or shared with selected individuals or groups.
  • Your Personal or Group Knowledge Base: Compile and organize your knowledge effortlessly, whether it’s for personal use or sharing within your team or community.
  • Your Collaboration Platform: Work together with colleagues or friends on projects, documents, or tasks in real-time, enhancing productivity and efficiency.
  • Your Community Network: Build a vibrant community network where members can connect, share ideas, and collaborate on common interests or goals.

What sets Cloudillo apart?

Global Accessibility, Local Control

Cloudillo takes a unique approach by storing all your data locally while making it globally accessible through its unparalleled API. This ensures you have full control over your data while enjoying the benefits of a worldwide reach.

Revolutionary Global Authentication and Authorization

Cloudillo introduces a groundbreaking method for Global Authentication, Authorization, and Verification. Share your data securely without requiring users to register on your instance, mirroring the convenience of Cloud-based platforms.

Seamless Interconnectivity

Thanks to this fresh perspective, Cloudillo-based applications can establish connections that might seem impossible on other platforms. This newfound flexibility opens doors to innovative and seamless collaborations between applications.

Ready to experience collaboration on a whole new level? Explore Cloudillo and witness the difference.

Prerequisites

Before installing or using Cloudillo, ensure your system meets the requirements below.


For Self-Hosting (Rust Version)

System Requirements

Minimum:

  • CPU: 1 core (2+ recommended)
  • RAM: 512MB (1GB+ recommended)
  • Disk: 1GB + storage for user data
  • OS: Linux (x86_64 or ARM64), macOS, Windows with WSL2

Recommended for 10+ users:

  • CPU: 2+ cores
  • RAM: 2GB+
  • Disk: 10GB+ SSD
  • OS: Linux (Ubuntu 22.04+ or Debian 12+)

Recommended for 100+ users:

  • CPU: 4+ cores
  • RAM: 8GB+
  • Disk: 50GB+ SSD
  • OS: Linux on dedicated hardware/VPS

Software Requirements

For Docker Installation (recommended):

  • Docker 20.10+
  • Docker Compose 2.0+ (optional but recommended)

For Building from Source:

  • Rust 1.70+ with cargo
  • Git

For Domain-Based Identity:

  • Domain name with DNS control
  • SSL/TLS certificate (Let’s Encrypt supported via ACME)

Network Requirements

Required Ports:

  • 443 (HTTPS) - Required for application access
  • 80 (HTTP) - Required for ACME/Let’s Encrypt certificate validation
  • OR custom ports if using reverse proxy

DNS Records Required:

  • A record: yourdomain.com → server IP address
  • A record: cl-o.yourdomain.com → server IP address

Example:

yourdomain.com          A    203.0.113.42
cl-o.yourdomain.com     A    203.0.113.42

Firewall:

  • Allow inbound TCP on ports 80 and 443
  • Allow outbound HTTPS (443) for federation

Browser Compatibility (for end users)

Supported:

  • Chrome/Edge 90+
  • Firefox 88+
  • Safari 14+

Recommended:

  • Latest stable versions of above browsers
  • JavaScript enabled
  • Cookies enabled

For App Development

Required Software

  • Node.js: 18+ (20+ recommended)
  • pnpm: 8+
  • TypeScript: 5+
  • Git: Latest stable version
  • VS Code with extensions:
    • TypeScript
    • ESLint
    • Prettier
    • React (if building UI apps)
  • Browser DevTools knowledge
  • Postman or similar API testing tool (optional)

Knowledge Prerequisites

  • JavaScript/TypeScript fundamentals
  • Async/await patterns
  • REST API concepts
  • WebSocket basics (for real-time features)
  • React basics (for UI apps using @cloudillo/react)

For Contributing to Cloudillo

Required

  • Rust: 1.70+ with cargo
  • Git: Latest stable version
  • GitHub account
  • SQLite: 3.x (for testing adapters)
  • Docker: For testing deployments
  • Rust experience:
    • Understanding of Tokio/async Rust
    • Familiarity with Axum web framework
    • Knowledge of Rust error handling
  • Testing tools:
    • cargo test
    • cargo clippy
    • cargo fmt

Useful Background

  • Understanding of web server architecture
  • Database experience (SQLite, key-value stores)
  • WebSocket protocols
  • Authentication systems (JWT, OAuth)
  • CRDT concepts (for collaborative editing features)

Deployment Scenarios

Personal/Family Use

Typical setup:

  • 1-20 users
  • Home server or small VPS
  • 1 vCPU, 512MB-1GB RAM, 50GB disk

Recommended:

  • Raspberry Pi 4 (4GB+)
  • Small VPS ($5-10/month)
  • Docker installation

Small Organization (10-50 users)

Typical setup:

  • 10-50 users
  • VPS or cloud instance
  • 2 vCPU, 1-2GB RAM, 500GB disk

Recommended:

  • DigitalOcean Droplet / AWS t3.small
  • Docker with monitoring
  • Regular backups

Community Server (50-500 users)

Typical setup:

  • 50-500 users
  • Dedicated server or powerful VPS
  • 2-4 vCPU, 4-8GB RAM, 1TB+ SSD

Recommended:

  • Dedicated server or large VPS
  • Load balancing (if high traffic)
  • Database optimization
  • CDN for static assets
  • Professional monitoring

Development/Testing

Typical setup:

  • Local development machine
  • 2+ cores, 4GB+ RAM, 20GB disk
  • Docker or build from source

Recommended:

  • Modern laptop/desktop
  • Fast SSD for rebuilds
  • Multiple browser for testing
  • Local domain setup (hosts file)

Quick Checklist

Before proceeding with installation, verify:

Self-Hosting Checklist

  • Server meets minimum requirements (1 core, 512MB RAM, 1GB disk)
  • Docker 20.10+ installed OR Rust 1.70+ installed
  • Domain name registered and DNS configured
  • Ports 80 and 443 accessible (firewall configured)
  • SSH access to server (if remote)
  • Backup plan in place

Development Checklist

  • Node.js 18+ installed
  • pnpm 8+ installed
  • TypeScript 5+ installed
  • Code editor ready (VS Code recommended)
  • Git configured
  • Access to Cloudillo instance (for testing)

Contributing Checklist

  • Rust 1.70+ installed
  • GitHub account created
  • Repository forked/cloned
  • Tests running (cargo test)
  • Code style configured (cargo fmt, cargo clippy)
  • Architecture documentation reviewed

Verification Commands

Check Docker Installation

docker --version
# Should show: Docker version 20.10+ or higher

docker compose version
# Should show: Docker Compose version 2.0+ or higher

Check Rust Installation

rustc --version
# Should show: rustc 1.70.0 or higher

cargo --version
# Should show: cargo 1.70.0 or higher

Check Node.js Installation

node --version
# Should show: v18.0.0 or higher

pnpm --version
# Should show: 8.0.0 or higher

Check DNS Configuration

# Check A records
dig yourdomain.com A
dig cl-o.yourdomain.com A

# Or use nslookup
nslookup yourdomain.com
nslookup cl-o.yourdomain.com

Check Port Accessibility

# Check if ports are open (from another machine)
nc -zv yourserver.com 80
nc -zv yourserver.com 443

# Or use telnet
telnet yourserver.com 80
telnet yourserver.com 443

Common Issues

“Cannot connect to Docker daemon”

Solution: Start Docker service

sudo systemctl start docker
sudo systemctl enable docker

“Port 443 already in use”

Solution:

  • Check what’s using port 443: sudo lsof -i :443
  • Stop conflicting service or use reverse proxy
  • Configure Cloudillo to use alternate port

“DNS records not propagating”

Solution:

  • Wait 24-48 hours for DNS propagation
  • Use TTL of 300 seconds for faster updates
  • Verify with multiple DNS checkers online

“Rust compiler version too old”

Solution: Update Rust

rustup update stable
rustup default stable

Next Steps

Once your system meets the prerequisites:

  1. Self-Hosting: Install Cloudillo →
  2. Development: Getting Started Guide →
  3. Contributing: Architecture Documentation →

Need Help?

Feature Status

Version: v0.1.0-alpha (Rust) | Progress: 90% | Last Updated: 2025-10-29

Info

Cloudillo is under active development. Features marked 🟢 are production-ready. 🟡 partial. 🔴 not started.


Implementation Status

Component Status Progress Notes
Backend Infrastructure
Core Server & Adapters 🟢 Complete 100% All 5 storage adapters fully implemented
Response Envelope System 🟢 Complete 100% Structured responses with error codes
Security & Auth 🟢 Complete 100% JWT, bcrypt, ABAC, TLS
API Endpoints
Authentication 🟡 Partial 50% Register/login working, token endpoints legacy
Profile Management 🟢 Complete 100% Get, update, media upload all working
Actions & Social 🟡 Partial 60% List/create working, others in progress
File Storage 🟡 Partial 40% Upload/download working, management incomplete
Real-Time Systems
RTDB Backend 🟢 Complete 100% Fully implemented with redb
CRDT Backend 🟢 Complete 100% Yjs-compatible collaborative editing
Message Bus 🟢 Complete 100% WebSocket pub/sub working
Client Libraries (TypeScript)
@cloudillo/types 🟢 Stable 100% Type definitions with validation
@cloudillo/core 🟡 Partial 95% API client works with Rust backend
@cloudillo/react 🟡 Partial 95% React hooks & components
@cloudillo/rtdb 🟢 Complete 95% Real-time database client, needs testing
Frontend & Apps
Shell/UI 🟢 Production 90% Dashboard, profile, files, messaging
Quillo (Editor) 🟢 Active 100% Collaborative rich text editor
Prello (Presentations) 🟢 Active 100% Presentation tool
Sheello (Spreadsheets) 🟢 Active 100% Spreadsheet application
Todollo (RTDB Demo) 🟢 Active 100% Todo list with real-time sync
Formillo (Forms) 🟢 Active 100% Form builder & survey tool
Federation 🟡 Partial 60% Basic federation working, envelope updates needed

Legend: 🟢 Production-ready | 🟡 Partial/In-progress | 🔴 Not started


Known Issues

  • Some API endpoints still use legacy response format (Phase 1 migration in progress)
  • Some action operations incomplete (reactions, stats, etc.)
  • File management incomplete (update, delete, tags)
  • Client library integration with RTDB/CRDT needs testing

For detailed API endpoint status, feature breakdowns, and roadmap timelines, check the repository status.


Support

Getting Started

Development Status

Cloudillo is under active development (90% complete). This guide explains core concepts that apply to all versions. For installation, see Installing Cloudillo for the Rust version, and check the Status Page for available features.

When you are ready to join the Cloudillo Network, you’ll need to make two decisions:

1. Identity

You identity is like your unique username on the network. Cloudillo uses the Domain Name System — the backbone of the internet — to create profile identities. This approach eliminates the need for a central provider, although it might seem overwhelming for newcomers. To simplify the process, you have two options:

  • Option 1: If you already have a domain name, you can use it to create your identity. This option may require technical know-how — but we’ve got you covered.

  • Option 2: If you don’t have a domain name or prefer to avoid the hassle, you can choose a Cloudillo Identity Provider. These services make it super easy to create your identity.

    Note

    cloudillo.net identity provider is planned for launch in Q1 2026 (Phase 4). Until then, you’ll need to use your own domain or wait for community providers to emerge.

2. Storage Provider

This is where your data will be stored.

  • Option 1: If you have your own server or a home NAS (Network Attached Storage), you can use that as your storage provider. You can even invite your family and friends to use it.

  • Option 2: Don’t want to manage your own storage? No problem. You can opt for a Cloudillo Storage Provider. They handle all the technical stuff for you.

Why Two Services?

Cloudillo is all about giving you control and keeping your data secure. Separating your identity from your storage provider makes it easy to switch storage options whenever you need to.

Using Your Own Domain

Many people and organizations already have domain names. You can use yours as your identity and even create sub-identities for different purposes. This helps organizations give their members identities tied to their domain.

Changing Your Storage Provider

If you control your identity, switching storage providers is simple. Whether you move to self-hosting or another provider, your identity remains intact, making the transition seamless.

Can You Use the Same Provider for Both?

Technically, yes. However, for security reasons, it’s better to choose different providers. Using separate providers avoids potential conflicts of interest if you ever need to switch storage providers.

Trust Your Identity Provider

Your identity provider is crucial. Ensure you trust them, as they could theoretically take control of your identity. If this concerns you, registering your own domain name can offer maximum security.

Warning

Keep in mind, on Cloudillo you own not only your data, but also your network, followers, and likes. No one can take these away from you, unless they gain control of your identity.

Installing Cloudillo Yourself

Rust Version

This guide is for the Rust implementation of Cloudillo (v0.1.0-alpha). The Node.js version is deprecated and no longer maintained.

Alpha Software

The Rust version is 90% complete. Core functionality works, but some API endpoints are still in development. See Status Page for details.

Before You Begin

  • Review Prerequisites to ensure your system meets requirements
  • Have a domain name ready with DNS control
  • Decide on deployment mode (standalone vs proxy)

Installation Options

You have several options for installing Cloudillo:

  1. Docker (recommended) - Simplest and most versatile
  2. Build from source - For development or custom builds

You also need to decide whether to run in:

  • Standalone mode - Cloudillo handles HTTPS with Let’s Encrypt
  • Proxy mode - Run behind a reverse proxy (nginx, Caddy, etc.)

Running Cloudillo using Docker in Standalone Mode

Warning

Docker images for the Rust version may not be published yet. Check the cloudillo-rs repository for latest status. If not available, use the build from source method below.

Here is the Docker command (adjust image name when available):

docker run -d --name cloudillo \
  -v /var/vol/cloudillo:/data \
  -p 443:443 -p 80:80 \
  -e LOCAL_IPS=1.2.3.4 \
  -e BASE_ID_TAG=agatha.example.com \
  -e BASE_APP_DOMAIN=agatha.example.com \
  -e BASE_PASSWORD=SomeSecret \
  -e ACME_EMAIL=user@example.com \
  -e ACME_TERMS_AGREED=true \
  cloudillo/cloudillo-rs:latest

You basically need a local directory on your server and mount it to /data inside the container and publish port 443 (HTTPS) and 80 (HTTP). You can read about the configuration environment variables below.

Cloudillo in standalone mode uses TLS certificates managed by Let’s Encrypt. If you want to use different certificates then you have to use Proxy Mode.

DNS Required

For Let’s Encrypt certificate issuance to work you have to ensure that $BASE_APP_DOMAIN and cl-o.$BASE_ID_TAG both have DNS A records pointing to one of $LOCAL_IPS before first starting the container.

Here is a docker-compose for the same:

version: "3"
services:
  cloudillo:
    image: cloudillo/cloudillo-rs:latest  # Adjust when image is published
    container_name: cloudillo
    volumes:
      - /var/vol/cloudillo:/data
    ports:
      - 443:443
      - 80:80
    environment:
      - LOCAL_IPS: 1.2.3.4
      - BASE_ID_TAG: agatha.example.com
      - BASE_APP_DOMAIN: agatha.example.com
      - BASE_PASSWORD: SomeSecret
      - ACME_EMAIL: user@example.com
      - ACME_TERMS_AGREED: true
Security

You should change the default password as soon as possible!

Configuration Environment Variables

Variable Value Default
MODE “standalone” or “proxy” “standalone”
LOCAL_IPS Comma separated list of IP addresses this node serves *
BASE_ID_TAG ID tag for the admin user to create *
BASE_APP_DOMAIN App domain for the admin user to create *
BASE_PASSWORD Password for the admin user to create *
ACME_EMAIL Email address for ACME registration -
ACME_TERMS_AGREED Whether to agree to ACME terms -
DATA_DIR Path to the data directory /data
PRIVATE_DATA_DIR Path to the private data directory $DATA/priv
PUBLIC_DATA_DIR Path to the public data directory $DATA/pub

In case you are not familiar with the Cloudillo Identity System we provide some information about the notions used above:

The ID tag is the unique identifier of a Cloudillo profile. It can be any domain name associated with a user.

The App Domain is the domain address used by the user to access their Cloudillo shell. Currently the App Domain is also unique for every user, but it might change in the future. It is preferably the same as the ID tag of the user, but it can be different when the domain is used by an other site.

For a domain to work as a Cloudillo Identity it must serve a Cloudillo API endpoint accessable at the cl-o subdomain. In the above example you have to create the following DNS records:

Name Type Value Purpose
agatha.example.com A 1.2.3.4 App domain (APP_DOMAIN)
cl-o.agatha.example.com A 1.2.3.4 API domain (cl-o.ID_TAG)

Running Cloudillo using Docker in Proxy Mode

An other one-liner:

docker run -d --name cloudillo \
  -v /var/vol/cloudillo:/data \
  -p 1443:1443 \
  -e MODE=proxy \
  -e LOCAL_IPS=1.2.3.4 \
  -e BASE_ID_TAG=agatha.example.com \
  -e BASE_APP_DOMAIN=agatha.example.com \
  -e BASE_PASSWORD=SomeSecret

In proxy mode you have to provide your own solution for TLS certificates. In proxy mode Cloudillo serves its API port on 1443 but using HTTP and it doesn’t serve the HTTP port by default. You should provide your own redirections. However, you can turn it on with the LISTEN_HTTP=1080 environment variable.

We provide an example nginx configuration:

server {
	listen   80;
	server_name  agatha.example.com cl-o.agatha.example.com;

	location /.well-known/ {
		root    /var/www/certbot;
		autoindex off;
	}
	location / {
		return 301 https://$host$request_uri;
	}
}

server {
	listen 443 ssl;
	server_name agatha.example.com cl-o.agatha.example.com;
	ssl_certificate /etc/letsencrypt/live/agatha.example.com/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/agatha.example.com/privkey.pem;

	location /.well-known/cloudillo/id-tag {
		add_header 'Access-Control-Allow-Origin' '*';
		return 200 '{"idTag":"agatha.example.com"}\n';
	}
	location /api {
		rewrite /api/(.*) /api/$1 break;
		proxy_pass http://localhost:1443/;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Proto https;
		proxy_set_header X-Forwarded-Host $host;
		client_max_body_size 100M;
	}
	location /ws {
		proxy_pass http://localhost:1443;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Proto https;
		proxy_set_header X-Forwarded-Host $host;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection "Upgrade";
	}
	location / {
		# You can serve the cloudillo shell locally, or proxy it.
		root /home/agatha/cloudillo/shell;
		try_files $uri /index.html;
		autoindex off;
		expires 0;
	}
}

Building from Source

For the latest features or if Docker images aren’t available yet, build from source:

Prerequisites

# Install Rust (if not already installed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

# Verify installation
rustc --version  # Should be 1.70+
cargo --version

Clone and Build

# Clone the Rust repository
git clone https://github.com/cloudillo/cloudillo-rs.git
cd cloudillo-rs

# Build release version
cargo build --release

# Binary will be at:
./target/release/cloudillo-server

Run the Server

# Set environment variables
export DATA_DIR=/var/vol/cloudillo
export BASE_ID_TAG=agatha.example.com
export BASE_APP_DOMAIN=agatha.example.com
export BASE_PASSWORD=SomeSecret
export ACME_EMAIL=user@example.com
export ACME_TERMS_AGREED=true

# Create data directory
mkdir -p $DATA_DIR

# Run the server
./target/release/cloudillo-server

Next Steps

After installation:

  1. Verify Installation: Access https://your-domain.com
  2. Configure First User: Login with credentials you set
  3. Review Features: Check Status Page for available features
  4. Follow First Steps: First Steps Guide →

More Information

First Steps

Once you’ve set up your Cloudillo Identity and Storage, it’s time to get creative and start sharing content.

Accessing your Cloudillo Space

To access your Cloudillo Space, use your own URL, which usually matches your Identity. First, authenticate yourself using the method you set up during installation or registration.

Important

Remember, always access Cloudillo using your own URL, and never give your credentials on any other site!

Inside your Cloudillo Space you’ll run various applications from other spaces. Cloudillo will always let you know when this happens and shows the trust level of the application.

The Cloudillo Shell

When you enter your Cloudillo Space, you’ll see the Cloudillo Shell. This is the main application and it typically displays a header on the top of the window, unless you are running a full-screen application.

Connecting with Others

Cloudillo doesn’t have a central database, so you’ll need to know the identities of other users, organizations, or groups to connect with them. Once you’ve joined some groups, you can connect with others through those groups.

You can start by following the cloudillo.net identity, which is the official identity of the Cloudillo Community. From there you can find and join other groups.

Core Concept & Architecture

Cloudillo is an open-source collaboration platform that allows users to store their own data wherever they choose. Users can:

  • Self-host their data
  • Store data with community-hosted servers
  • Use third-party storage providers offering Cloudillo as a service

This ensures privacy-conscious users maintain full control while enabling seamless onboarding for less technical users. The design also ensures full user autonomy, preventing vendor lock-in by allowing seamless migration between storage providers or self-hosting at any time.

Architecture Diagram

Cloudillo Architecture Cloudillo Architecture

Identity System & User Profiles

Cloudillo decouples identity from storage through a Domain Name System (DNS)-based identity system. Each user and community has a stable, trusted identifier independent of storage location.

Users can create identities using their own domain name, allowing them to maintain full control and leverage their trusted brand within the platform.

Cloudillo Identity Providers (CIP) help users create and manage identities without manual DNS setup. Any domain owner can implement a CIP, with the first provider being cloudillo.net.

Each Cloudillo Identity has a publicly accessible profile containing:

  • Identity Name
  • Identity Public Key
  • Optional additional metadata

The Identity Public Key verifies user authenticity within the Cloudillo network.

Content-Addressing & Merkle Trees

Cloudillo uses content-addressing throughout its architecture, where every action, file, and data blob is identified by the cryptographic hash of its content. This creates a merkle tree structure that provides:

  • Cryptographic proof of authenticity: Anyone can verify content integrity
  • Immutability: Content cannot be modified without changing its identifier
  • Tamper-evidence: Any modification is immediately detectable
  • Deduplication: Identical content produces identical identifiers
  • Trustless verification: No need to trust storage providers

Every resource in Cloudillo—from action tokens to image blobs—is part of this merkle tree, creating a verifiable chain of trust from user actions down to individual bytes of data.

Learn more: Content-Addressing & Merkle Trees

Actions

Cloudillo supports event-driven communication between nodes. Actions are event-driven interactions that users perform, such as:

  • Connecting with another user
  • Following a user
  • Posting, commenting, or reacting
  • Sharing a document

When an action occurs, Cloudillo generates a cryptographically signed Action Token, distributing it to involved parties. This mechanism prevents spam and unauthorized actions.

Action tokens are content-addressed, meaning each action has a unique identifier derived from the hash of its content. This ensures actions are immutable and verifiable.

Access Control & Resource Sharing

When a user wants to access a resource stored on another node (e.g., editing a document hosted by another user), the following process occurs:

  • The user’s node requests access using their Identity Key.
  • The remote node validates the request and grants an Access Token.
  • The application uses this token to interact with the resource on behalf of the user.

This process ensures secure, decentralized access control without requiring direct trust between storage providers. This process is designed to be seamless, requiring no additional user interaction—offering the ease of centralized cloud platforms while maintaining full decentralization.

Authentication & Authorization Tokens

Cloudillo utilizes cryptographic tokens for authentication and authorization:

  • Action Tokens: Signed by users, representing their activities (e.g., posting, following).
  • Access Tokens: Used for resource access, granted by the data owner’s node.
  • Verification: All tokens are cryptographically signed and validated by the recipient nodes before processing.

Resource Types & Storage Model

Cloudillo supports multiple resource types:

  • Extensible Actions: Developers can define new action types that integrate seamlessly with Cloudillo’s decentralized architecture, allowing for future expansion of platform capabilities.
  • Immutable Resources: Content such as images, videos, and published documents. These are content-addressed using cryptographic hashes to ensure integrity and enable deduplication.
  • Editable Documents: Built-in collaborative structures allow real-time multi-user editing of documents.
  • Databases: Cloudillo provides a real-time database API, enabling developers to create custom applications on top of the platform.

This design ensures Cloudillo remains flexible and adaptable to evolving user and developer needs.

Architectural Documentation

Dive deeper into Cloudillo’s architecture:

Fundamentals

Data Storage & Access

Actions & Federation

Runtime Systems

Subsections of Core Concept & Architecture

Fundamentals

The foundational concepts underlying Cloudillo’s architecture. These documents explain the core principles, system design patterns, identity mechanisms, and security considerations that enable Cloudillo’s federated, decentralized design.

Core Topics

These fundamentals form the basis for understanding Cloudillo’s data storage, action systems, and federation mechanisms.

Subsections of Fundamentals

System Architecture Overview

This document provides a technical overview of the Cloudillo system architecture, explaining the core patterns and components that enable its federated, privacy-focused design.

Workspace Structure

Cloudillo is organized as a Rust workspace with feature-specific crates:

cloudillo-rs/
├── server/                      # Binary: cloudillo-server
├── crates/
│   ├── cloudillo/               # Main integrator (routes, websocket, bootstrap)
│   ├── cloudillo-types/         # Shared types, adapter traits, error types
│   ├── cloudillo-core/          # Infrastructure (scheduler, worker, middleware, ACME)
│   ├── cloudillo-auth/          # Authentication (login, WebAuthn, QR, API keys)
│   ├── cloudillo-action/        # Federation actions (posts, delivery, verification)
│   ├── cloudillo-file/          # File processing (images, video, audio, PDF, SVG)
│   ├── cloudillo-crdt/          # CRDT collaborative editing (Yjs WebSocket)
│   ├── cloudillo-rtdb/          # Real-time database
│   ├── cloudillo-push/          # Web Push notifications
│   ├── cloudillo-email/         # Email notifications (SMTP, templates)
│   ├── cloudillo-idp/           # Identity provider, tenant management
│   ├── cloudillo-profile/       # Profile management
│   ├── cloudillo-admin/         # System administration
│   ├── cloudillo-ref/           # Reference/lookup API
│   └── cloudillo-proxy/         # Reverse proxy with TLS
└── adapters/
    ├── auth-adapter-sqlite/     # Authentication & cryptography (SQLite)
    ├── meta-adapter-sqlite/     # Metadata storage (SQLite)
    ├── blob-adapter-fs/         # Binary blob storage (Filesystem)
    ├── rtdb-adapter-redb/       # Real-time database (Redb)
    └── crdt-adapter-redb/       # Collaborative editing CRDT (Redb)

Crate Architecture

The workspace follows a three-layer architecture:

  • cloudillo-types: Foundation layer — adapter trait definitions (AuthAdapter, MetaAdapter, BlobAdapter, RtdbAdapter, CrdtAdapter), shared domain types, and error types
  • cloudillo-core: Infrastructure layer — task scheduler, worker pool, authentication middleware, ACME certificate management, custom extractors, WebSocket infrastructure, and HTTP client
  • cloudillo-*: Feature crates — each domain (auth, action, file, etc.) is an independent crate with its own handlers, tasks, and settings
  • cloudillo: Integrator — assembles all feature crates into HTTP routes, WebSocket handling, and application bootstrap
  • server: Binary crate (cloudillo-server) — integrates all feature crates and adapters into a runnable server
  • adapters: Five pluggable storage backends implementing the traits defined in cloudillo-types

Adapter Types

The five adapters separate concerns and enable flexible deployments:

  1. AuthAdapter - Authentication, JWT tokens, certificate management, cryptographic operations
  2. MetaAdapter - Tenant/profile data, action tokens, file metadata, tasks
  3. BlobAdapter - Content-addressed immutable binary data (files, images, snapshots)
  4. RtdbAdapter - Real-time hierarchical JSON database with queries and subscriptions
  5. CrdtAdapter - Collaborative document editing with conflict-free merges

Core Architecture Patterns

The Five-Adapter Architecture

Cloudillo’s architecture is built on five fundamental adapters that separate concerns and enable flexible deployments:

1. AuthAdapter

Purpose: Authentication, authorization, and cryptographic operations

Responsibilities:

  • JWT/token validation and creation
  • TLS certificate management (ACME integration)
  • Profile signing key storage and rotation
  • VAPID keys for push notifications
  • Password hashing and WebAuthn credentials
  • Tenant management

Key Operations: Validate tokens, create access tokens, manage TLS certificates, handle profile keys, store passwords/credentials

Why separate? Authentication and cryptography require special security considerations and may need different storage backends (HSM, vault services, etc.).

2. MetaAdapter

Purpose: Structured metadata storage

Responsibilities:

  • Tenant and profile information
  • Action tokens (posts, comments, reactions, etc.)
  • File metadata with variants
  • Task persistence for the scheduler
  • Database metadata (for RTDB)

Key Operations: Create/query tenants, store/retrieve actions, manage file metadata, persist tasks, handle profiles

Why separate? Metadata can be stored in different databases (SQLite, PostgreSQL, MongoDB) based on scale and requirements.

3. BlobAdapter

Purpose: Immutable binary data storage

Responsibilities:

  • Content-addressed blob storage
  • File data (images, videos, documents)
  • Database snapshots (for RTDB)
  • Both buffered and streaming I/O

Key Operations: Write/read blobs (buffered or streaming), check blob existence and size

Why separate? Blob storage can use filesystem, S3, CDN, or specialized object storage based on scale and cost considerations.

4. RtdbAdapter

Purpose: Real-time structured database

Responsibilities:

  • Path-based hierarchical storage (Firebase-like)
  • Query with filtering, sorting, pagination
  • Real-time subscriptions via WebSocket
  • Transaction support for atomic writes
  • Secondary indexes for performance
  • Multi-tenant database isolation

Key Operations: Query/create/update/delete documents, transactions, real-time subscriptions

Why separate? Real-time database functionality can use different backends (redb, PostgreSQL, MongoDB) based on query requirements and scale.

Learn more: RTDB with redb

5. CrdtAdapter

Purpose: Collaborative document storage with CRDTs

Responsibilities:

  • Binary CRDT update storage (Yjs protocol)
  • Document metadata management
  • Real-time change subscriptions
  • Conflict-free merge guarantees
  • Awareness state (presence, cursors)
  • Multi-tenant document isolation

Key Operations: Create/read documents, append updates, retrieve update history, subscribe to changes

Why separate? CRDT storage can use different backends (redb, dedicated CRDT stores) and has different performance characteristics than traditional databases.

Learn more: CRDT Collaborative Editing

Benefits of the Adapter Pattern

  • Flexible Deployment: Switch storage backends without changing core logic
  • Separation of Concerns: Security, metadata, and binary data have different requirements
  • Testing: Easy to create in-memory adapters for testing
  • Scalability: Can distribute adapters across different services
  • Cost Optimization: Use appropriate storage for each data type

Content-Addressed Architecture

Cloudillo uses content-addressing throughout its architecture, where resource identifiers are cryptographic hashes of their content. This creates a merkle tree structure that provides cryptographic proof of authenticity and immutability.

Hash-Based Identifiers

All resource IDs are SHA-256 hashes with versioned prefixes:

Prefix Resource Type Hash Input Example
a1~ Action Entire JWT token (header + payload + signature) a1~8kR3mN9pQ2vL6xW...
f1~ File File descriptor string f1~Qo2E3G8TJZ2HTGh...
b1~ Blob Blob bytes (actual image/video data) b1~abc123def456ghi...
d2, Descriptor (not a hash, the encoded string itself) d2,vis.tn:b1~abc:f=avif:...

Version Scheme

Format: {prefix}{version}~{base64_encoded_hash}

  • Version 1: SHA-256 with base64url encoding (no padding)
  • Future versions: Can upgrade to SHA-3, BLAKE3, etc. without breaking old content
  • Backward compatibility: Old content remains valid forever
  • Algorithm agility: Migrate to new algorithms without breaking existing references

Example upgrade path:

a1~...  (SHA-256)
a2~...  (SHA-3)
a3~...  (BLAKE3)

Six-Level Merkle Tree

Content-addressing creates a hierarchical merkle tree:

Level 1: Blob Data (raw bytes)
   ↓ SHA-256 hash
Level 2: Blob ID (b1~hash)
   ↓ collected in descriptor
Level 3: File Descriptor (d2,class.variant:b1~hash:format:size:resolution;...)
   ↓ SHA-256 hash of descriptor
Level 4: File ID (f1~hash)
   ↓ referenced in action
Level 5: Action Token (JWT with content, parent, attachments)
   ↓ SHA-256 hash of entire JWT
Level 6: Action ID (a1~hash)

Properties

  • Immutable: Content cannot change without changing the ID
  • Tamper-Evident: Any modification is immediately detectable
  • Deduplicatable: Identical content produces identical IDs
  • Verifiable: Anyone can recompute and verify hashes
  • Cacheable: Content-addressed data can be cached forever
  • Trustless: No need to trust storage providers—verify the hash

Integration with Adapters

Content-addressing is implemented across multiple adapters:

BlobAdapter:

  • Stores blobs indexed by blob_id (b1~...)
  • Blob IDs are SHA-256 hashes of the blob bytes
  • Enables deduplication and integrity verification

MetaAdapter:

  • Stores action tokens indexed by action_id (a1~...)
  • Action IDs are SHA-256 hashes of the JWT token
  • Stores file metadata with file_id (f1~...)
  • File IDs are SHA-256 hashes of the descriptor

AuthAdapter:

  • Signs action tokens with profile keys (ES384)
  • Signature + content hash = complete authenticity proof

Learn more: Content-Addressing & Merkle Trees

Task-Based Asynchronous Processing

Complex operations in Cloudillo are modeled as persistent tasks that can execute asynchronously, survive restarts, and depend on other tasks.

Task System Components

Tasks

Tasks implement the Task<S> trait:

pub trait Task<S>: Debug + Send + Sync {
    fn kind() -> &'static str;
    async fn run(&self, state: &S) -> Result<()>;
    fn priority(&self) -> Priority { Priority::Medium }
    fn dependencies(&self) -> Vec<TaskId> { vec![] }
}

Built-in Task Types:

  • ActionCreatorTask: Creates and signs action tokens for federation
  • ActionVerifierTask: Validates incoming federated actions
  • ActionDeliveryTask: Delivers actions to remote instances with retry
  • FileIdGeneratorTask: Generates content-addressed file IDs
  • ImageResizerTask: Creates image variants (thumbnails, etc.)
  • VideoTranscoderTask: Transcodes video to web-optimized formats
  • AudioExtractorTask: Extracts audio metadata
  • PdfProcessorTask: Extracts text and metadata from PDFs
  • EmailSenderTask: Sends emails asynchronously
  • CertRenewalTask: Handles ACME certificate renewal
  • ProfileRefreshBatchTask: Batch-refreshes remote profiles
  • TenantImageUpdaterTask: Updates tenant avatar images

Scheduler

The scheduler manages task lifecycle with dependency resolution:

Features:

  • Task registry with dynamic builders
  • Dependency resolution (DAG-based)
  • Scheduled execution (cron-like)
  • Persistence via MetaAdapter (survives restarts)
  • Notification system for task completion

Example Flow:

ActionCreatorTask (depends on FileIdGeneratorTask)
    ↓
    waits for file processing to complete
    ↓
FileIdGeneratorTask completes
    ↓
ActionCreatorTask auto-starts
    ↓
Creates signed JWT, stores in MetaAdapter

Worker Pool

A priority-based thread pool for CPU-intensive and blocking operations:

Architecture:

  • Three priority tiers: High > Medium > Low
  • Each tier has configurable worker thread count
  • Uses flume MPMC channels for work distribution
  • Returns futures for async integration

Default Configuration (cloudillo-server):

  • 1 high-priority worker
  • 2 medium-priority workers
  • 1 low-priority worker

Use Cases:

  • Image processing (CPU-intensive)
  • Cryptographic operations
  • File compression
  • Blocking I/O

Application State Management

AppState Structure

The core application state contains: scheduler, worker pool, HTTP client, TLS certificates, and all five adapters (auth, meta, blob, rtdb, crdt).

AppBuilder Pattern

Configuration uses a fluent builder API with mode, identity, domain, data directory, and adapter selections.

Configuration Options:

  • Server mode (Standalone, Proxy, StreamProxy)
  • Network binding (HTTPS/HTTP ports)
  • Domain configuration
  • Directory paths (dist, tmp, data)
  • Adapter injection
  • Worker pool sizing

Crate Organization

The workspace is organized into feature-specific crates under crates/:

cloudillo-types - Foundation Layer

Shared types and trait definitions used across all crates:

  • Adapter trait definitions (AuthAdapter, MetaAdapter, BlobAdapter, RtdbAdapter, CrdtAdapter)
  • Core domain types and type aliases
  • Error types with unified error handling
  • HTTP extractors and utility types

cloudillo-core - Infrastructure Layer

Core system components providing foundational services:

  • scheduler.rs: Task scheduling with dependencies
  • app.rs: Application state and builder
  • acme.rs: Let’s Encrypt/ACME certificate management
  • middleware.rs: Authentication middleware
  • extract.rs: Custom Axum extractors (TnId, IdTag, Auth)
  • ws_bus.rs: WebSocket message bus
  • ws_broadcast.rs: WebSocket broadcast manager
  • rate_limit/: Rate limiting (governor-based)
  • request.rs: HTTP client for federation
  • roles.rs: Role definitions
  • settings/: Global settings system

cloudillo-auth - Authentication

  • Login/logout endpoints, token generation, password management
  • WebAuthn passwordless authentication
  • QR code-based login flow
  • API key generation and validation

cloudillo-action - Federation Actions

  • Action creation, verification, and delivery tasks
  • JWT verification and token processing
  • Action CRUD endpoints and federation inbox
  • Federated delivery with retry and audience computation

cloudillo-file - File Storage & Processing

  • File upload/download endpoints
  • Image processing (resize, format conversion, SVG rasterization)
  • Video transcoding and audio extraction (via FFmpeg)
  • PDF processing and file descriptor management
  • Variant generation and preset system

cloudillo-profile - Profile Management

  • Tenant profile endpoints
  • Profile federation and sync
  • Avatar and banner image processing

cloudillo-rtdb - Real-Time Database

  • Database CRUD endpoints
  • WebSocket subscription-based data sync

cloudillo-crdt - Collaborative Editing

  • Yjs WebSocket protocol handler for CRDT sync

cloudillo-push - Push Notifications

  • Web Push notification delivery (RFC 8291 + VAPID)
  • Subscription management and per-user preferences

cloudillo-email - Email Notifications

  • SMTP delivery via lettre
  • Handlebars template rendering
  • Async email sender task

cloudillo-idp - Identity Provider

  • Tenant registration and lifecycle management

cloudillo-admin - Administration

  • System admin endpoints for instance management

cloudillo-ref - Reference API

  • Namespace lookups and reference resolution

cloudillo-proxy - Reverse Proxy

  • HTTP/WebSocket proxying with TLS termination

cloudillo - Integrator

Assembles all feature crates into a runnable application:

  • routes.rs: HTTP endpoint definitions (public and protected route groups)
  • websocket.rs: WebSocket protocol handler
  • bootstrap.rs: First-run tenant setup
  • webserver.rs: Server configuration (TLS, CORS, compression)

Concurrency Model

Multi-threaded Tokio Runtime

Cloudillo uses Tokio’s multi-threaded async runtime for handling concurrent requests.

Concurrency Layers

  1. Async Layer (Tokio): HTTP request handling, WebSocket connections, I/O
  2. Worker Pool: CPU-intensive tasks, blocking operations
  3. Scheduler: Background task execution with dependencies

Interaction Example:

HTTP Request → Tokio async handler
    ↓
Spawn ImageResizerTask on scheduler
    ↓
Scheduler dispatches to worker pool (CPU-intensive)
    ↓
Worker completes, updates MetaAdapter
    ↓
Scheduler notifies waiting tasks
    ↓
Response returned to client

Request Handling Flow

Middleware Pipeline

HTTP requests flow through these layers:

  1. HTTPS/SNI Resolution: CertResolver selects TLS certificate by domain
  2. Tracing/Logging: Request tracing with structured logging
  3. Authentication: require_auth or optional_auth middleware
  4. Custom Extractors: TnId, IdTag, Auth extract from request context
  5. Handler: Business logic execution
  6. Response: JSON serialization, error handling

Custom Extractors

Axum extractors provide typed access to request context:

  • TnId: Tenant ID (database primary key, u32)
  • IdTag: Tenant identifier string (e.g., “alice.example.com”)
  • Auth: Full authentication context (tn_id, id_tag, scope, etc.)

Error Handling

Custom error enum with automatic HTTP response conversion: NotFound (404), PermissionDenied (403), DbError/Unknown/Parse/Io (500).

Server Modes

Cloudillo supports different deployment modes:

Standalone (Default)

Self-contained single instance:

  • HTTPS on configured port
  • Optional HTTP for ACME challenges
  • All adapters run locally

Use case: Personal servers, small communities

Proxy

Used if Cloudillo is behind a reverse proxy:

  • Listens on HTTP port
  • Certificate handled is the responsibility of the proxy

Use case: Managed hosting providers, self-hosting with multiple services on one IP address

Security Architecture

Implemented in Rust

Maximal memory and concurrency safety. Minimal attack surface.

No Unsafe Code

Cloudillo enforces memory safety:

#![forbid(unsafe_code)]

ABAC Permission System

Cloudillo uses Attribute-Based Access Control (ABAC) for fine-grained permissions across all resources. ABAC provides flexible permission rules based on:

  • User attributes (identity, roles, relationships)
  • Resource attributes (owner, visibility, type)
  • Contextual factors (time, environment)

Key Features:

  • Six visibility levels: Public (P), Verified (V), SecondDegree (2), Follower (F), Connected (C), Direct (NULL)
  • Policy-based access control (TOP/BOTTOM policies)
  • Relationship-aware (following, connections)
  • Time-based permissions
  • Custom policy support

Learn more: ABAC Permission System

Cryptographic Algorithms

  • P384: Elliptic curve for action signing
  • ES384: JWT signature algorithm
  • SHA256: Content addressing and hashing
  • bcrypt: Password hashing

Security Layers

  1. TLS/HTTPS: All connections encrypted (Rustls)
  2. ACME: Automatic certificate management (Let’s Encrypt)
  3. JWT: Cryptographically signed tokens
  4. Content Addressing: Tamper detection via SHA256
  5. Permission Checks: Authorization at every access point

Bootstrap Process

Initial setup when starting a new instance:

  1. Create tenant with base_id_tag
  2. Set password (if provided)
  3. Generate profile signing key (P384)
  4. Initiate ACME for TLS certificates (if email provided)
  5. Start scheduler and worker pool
  6. Start HTTP/HTTPS servers

Example configuration:

BASE_ID_TAG=alice.example.com        # Required
BASE_PASSWORD=secret                 # Initial password
ACME_EMAIL=alice@example.com         # Let's Encrypt email
MODE=standalone                      # Server mode
LISTEN=0.0.0.0:8443                  # HTTPS binding
DATA_DIR=./data                      # storage path

Key Dependencies

Web Framework

  • axum (0.8): Async web framework
  • tower: Service abstractions
  • tower-http: CORS, static files

TLS & Crypto

  • rustls (0.23): Pure Rust TLS
  • instant-acme (0.8): ACME client
  • jsonwebtoken (9.3): JWT handling
  • p384: Elliptic curve operations

Async Runtime

  • tokio (1.48): Multi-threaded async

Serialization

  • serde (1.0): Serialization framework
  • serde_json: JSON support

Database

  • sqlx (0.8): Async SQL (SQLite support)

Utilities

  • image (0.25): Image processing
  • sha2: SHA256 hashing
  • croner (3.0): Cron expressions
  • flume: MPMC channels

Architectural Strengths

  • Pluggable Adapters: Easy to swap storage backends
  • Self-Contained: No external dependencies required
  • Federated: Communicates with other instances
  • Task-Based: Persistent, resumable async execution
  • Type-Safe: Leverages Rust’s type system
  • Memory-Safe: Complete #![forbid(unsafe_code)]
  • Observable: Built-in tracing integration

Next Steps

Identity System & User Profiles

Cloudillo Profiles are located using a DNS-based identity system. Each Cloudillo Identity is associated with a specific API endpoint, which can be accessed via the “cl-o” subdomain of the identity. For example, the API domain of the cloudillo.net identity is available at https://cl-o.cloudillo.net/.

Retrieving a Cloudillo Profile

You can retrieve a Cloudillo profile by making an API request to the /api/me endpoint of the identity’s API domain:

curl https://cl-o.cloudillo.net/api/me

Example response:

{
  "idTag": "cloudillo.net",
  "name": "Cloudillo",
  "type": "community",
  "profilePic": "QoEYeG8TJZ2HTGhVlrtTDBpvBGOp6gfGhq4QmD6Z46w",
  "keys": [
    {
      "keyId": "20250205",
      "publicKey": "MHYwEAYHKoZIzj0CAQYFK4EEACIDYgAENSq6EZZ+xCypWNkm+0+MHIZfLX7I01wTT+SOw7DoOUOuDAWKMkhBVG+SUb9AzCxEOlmkefuEW5zmXNwmH2MphEQW/r18RDjd+Nt5nbemBsoQzsm2Wg/mUyWBsKYs1oe5"
    }
  ]
}

Profile Fields Explained

  • idTag: The Identity Tag associated with the profile.
  • name: The display name of the profile.
  • type: Either empty for a personal profile or contains “community” for community profiles.
  • profilePic: The identifier for the profile picture, which can be retrieved via the storage API.
  • keys: A list of cryptographic keys associated with the profile.

Profile Picture Retrieval

The profilePic field contains an identifier that allows retrieval of the profile’s profile picture using the Cloudillo Storage API. A request to retrieve the profile picture may look like this:

curl https://cl-o.nikita.cloudillo.net/api/store/NMzE4gNI29aEkiV6Q7I1UNWh4x2gFZ7753Pl74veYtU

Key Management

A profile supports multiple cryptographic keys, enabling periodic key rotation for enhanced security. Older keys can be deprecated while maintaining account integrity.

Key Structure

Each profile key contains:

pub struct ProfileKey {
    key_id: String,        // Identifier (e.g., "20250205")
    public_key: String,    // Base64-encoded P384 public key
    created_at: i64,       // Unix timestamp
    expires_at: Option<i64>,  // Optional expiration
    key_type: KeyType,     // Purpose of the key
}

pub enum KeyType {
    Signing,    // For action token signatures
    VAPID,      // For push notifications
    // Future: Encryption, Authentication, etc.
}

Cryptographic Algorithms

Cloudillo uses elliptic curve cryptography for signing:

  • Curve: P384 (NIST P-384, also known as secp384r1)
  • Algorithm: ES384 (ECDSA using P-384 and SHA-384)
  • Key Size: 384 bits
  • Signature Size: ~96 bytes

Key Rotation

Keys are rotated by generating a new key (keyId = current date), adding it to the profile alongside existing keys, and marking old keys as expired after a 30-90 day grace period.

VAPID Keys

VAPID (Voluntary Application Server Identification) keys are used for web push notifications:

pub struct VapidKey {
    key_id: String,
    public_key: String,   // Base64-encoded
    private_key: String,  // Stored in AuthAdapter, never exposed
    contact: String,      // mailto: or https: URI
}

These are separate from signing keys to isolate concerns and enable different rotation policies.

Identity Resolution

DNS-Based Discovery

Cloudillo uses DNS to discover the API endpoint for an identity:

  1. Identity: alice.example.com
  2. API Domain: cl-o.alice.example.com
  3. API Endpoint: https://cl-o.alice.example.com/api/me

Cloudillo Identity Providers (CIP)

Users without their own domain can use a Cloudillo Identity Provider:

CIP Responsibilities:

  • Domain management (subdomains or custom domains)
  • DNS configuration
  • Dynamic DNS support
  • CIPs never store or access any user data, nor have any rescponsibilities about it.

Example CIP: cloudillo.net

  • Provides identities like alice.cloudillo.net
  • API available at https://cl-o.alice.cloudillo.net
  • Users can migrate to their own domain later

Custom Domain Setup

To use your own domain with Cloudillo:

  1. DNS Configuration:
cl-o.alice.example.com.  A      <your-server-ip>
cl-o.alice.example.com.  AAAA   <your-server-ipv6>
app.example.com.  A             <your-server-ip>
app.example.com.  AAAA          <your-server-ipv6>
  1. TLS Certificate: Automatic via ACME (Let’s Encrypt)
ACME_EMAIL=admin@example.com
  1. Bootstrap: Configure tenant
BASE_ID_TAG=alice.example.com
BASE_APP_DOMAIN=app.example.com
BASE_PASSWORD=<secure-password>

Profile Metadata

Storage

Profile metadata is stored in the MetaAdapter:

pub struct ProfileMetadata {
    tn_id: TnId,           // Internal tenant ID
    id_tag: String,        // Public identifier
    name: String,          // Display name
    profile_type: ProfileType,
    profile_pic: Option<String>,  // File ID
    bio: Option<String>,
    created_at: i64,
    updated_at: i64,
}

pub enum ProfileType {
    Personal,
    Community,
}

Synchronization

Profiles can be synchronized across instances for caching:

GET https://cl-o.{remote_id_tag}/api/me
Parse JSON response
Cache locally in MetaAdapter

Security Considerations

Public Key Infrastructure

  • Public keys are publicly accessible via /api/me
  • Private keys are stored securely in AuthAdapter
  • No private key export - keys never leave the server
  • Key verification happens on every action token

Identity Trust Model

Trust is established through:

  1. DNS ownership: Control of domain proves identity ownership
  2. Key signatures: Private key proves control of identity
  3. HTTPS: TLS proves server authenticity
  4. Action signatures: ES384 signatures prove action authenticity

Attack Mitigation

DNS Hijacking:

  • Old signatures remain valid (past actions can’t be forged)
  • Users can verify key changes

Key Compromise:

  • Rotate immediately to new key
  • Mark compromised key as expired
  • Past actions become invalid

Server Compromise:

  • Private keys in AuthAdapter may need HSM/vault for high-security deployments
  • Separate AuthAdapter storage from other adapters
  • Regular security audits

API Reference

GET /api/me

Retrieve public profile information.

Request:

curl https://cl-o.example.com/api/me

Response (200 OK):

{
  "idTag": "example.com",
  "name": "Alice",
  "type": "",
  "profilePic": "f1~Qo...46w",
  "keys": [
    {
      "keyId": "20250205",
      "publicKey": "MHYwEAYHKoZIzj0CAQ..."
    }
  ]
}

GET /api/me/keys

Returns all public keys with createdAt, expiresAt, and keyType fields (for action token verification).

See Also

Content-Addressing & Merkle Trees

Cloudillo implements a merkle tree structure using content-addressed identifiers throughout its architecture. Every action, file, and data blob is identified by the cryptographic hash of its content, creating an immutable, verifiable chain of trust.

What is Content-Addressing?

Content-addressing means identifying data by what it is (its content) rather than where it is (its location). Instead of using arbitrary IDs or URLs, Cloudillo computes a cryptographic hash of the content itself and uses that hash as the identifier.

Benefits

  • Immutable: Content cannot change without changing its identifier
  • Tamper-Evident: Any modification is immediately detectable
  • Deduplicatable: Identical content produces identical identifiers
  • Verifiable: Anyone can recompute and verify hashes independently
  • Cacheable: Content-addressed data can be cached forever
  • Trustless: No need to trust storage providers—verify the hash

Hash Function

Cloudillo uses SHA-256 for all content-addressing:

  • Algorithm: SHA-256 (256-bit Secure Hash Algorithm)
  • Encoding: Base64url without padding (URL-safe)
  • Output: 43-character base64-encoded string
  • Collision Resistance: Cryptographically secure
compute_hash(prefix, data):
    hash = SHA256(data)
    encoded = base64url_encode(hash)  // URL-safe, no padding
    return "{prefix}1~{encoded}"

// Example:
compute_hash("b", blob_bytes) → "b1~abc123def456..." (43 chars)
compute_hash("f", descriptor)  → "f1~Qo2E3G8TJZ..." (43 chars)
compute_hash("a", jwt_token)   → "a1~8kR3mN9pQ2vL..." (43 chars)

Merkle Tree Structure

Cloudillo’s content-addressing creates a variable-depth merkle tree where actions can reference other actions recursively. The example below shows a six-level hierarchy for a POST action with image attachments:

┌─────────────────────────────────────────────────────────┐
│ Level 6: Action ID (a1~8kR3mN9pQ2vL6xW...)              │
│   ↑ SHA-256 hash of ↓                                   │
├─────────────────────────────────────────────────────────┤
│ Level 5: Action Token (JWT)                             │
│   Header: {"alg":"ES384","typ":"JWT"}                   │
│   Payload: {                                            │
│     "iss": "alice.example.com",                         │
│     "t": "POST:IMG",                                    │
│     "c": "Amazing photo!",                              │
│     "a": ["f1~Qo2E3G8TJZ..."],                          │
│     "iat": 1738483100                                   │
│   }                                                     │
│   Signature: <ES384 signature>                          │
│   ↓ references ↓                                        │
├─────────────────────────────────────────────────────────┤
│ Level 4: File ID (f1~Qo2E3G8TJZ2HTGhVlrtTDBp...)        │
│   ↑ SHA-256 hash of ↓                                   │
├─────────────────────────────────────────────────────────┤
│ Level 3: File Descriptor (d2,...)                       │
│   "d2,vis.tn:b1~abc:f=avif:s=4096:r=150x150;            │
│        vis.sd:b1~def:f=avif:s=32768:r=640x480;          │
│        vis.md:b1~ghi:f=avif:s=262144:r=1920x1080"       │
│   ↓ references ↓                                        │
├─────────────────────────────────────────────────────────┤
│ Level 2: Variant IDs                                    │
│   b1~abc123... (vis.tn)                                 │
│   b1~def456... (vis.sd)                                 │
│   b1~ghi789... (vis.md)                                 │
│   ↑ SHA-256 hash of ↓                                   │
├─────────────────────────────────────────────────────────┤
│ Level 1: Blob Data (raw bytes)                          │
│   <AVIF encoded image data - tn variant>                │
│   <AVIF encoded image data - sd variant>                │
│   <AVIF encoded image data - md variant>                │
└─────────────────────────────────────────────────────────┘

Level 1: Blob Data

Raw bytes of actual content (images, videos, documents).

Level 2: Variant IDs (b1~…)

Content-addressed blob identifiers computed as SHA256(blob_bytes).

Level 3: File Descriptor (d2,…)

Encoded string listing all available variants of a file.

Format:

d2,{class}.{variant}:{variant_id}:f={format}:s={size}:r={width}x{height};...

Example:

d2,vis.tn:b1~abc123:f=avif:s=4096:r=150x150;vis.sd:b1~def456:f=avif:s=32768:r=640x480

Level 4: File ID (f1~…)

Content-addressed file identifier computed as SHA256(descriptor_string).

Level 5: Action Token (JWT)

Cryptographically signed JSON Web Token representing a user action.

Structure:

{
  "header": {
    "alg": "ES384",
    "typ": "JWT"
  },
  "payload": {
    "iss": "alice.example.com",
    "k": "20250101",
    "t": "POST:IMG",
    "c": "Check out this amazing photo!",
    "a": ["f1~Qo2E3G8TJZ..."],
    "iat": 1738483100
  },
  "signature": "<ES384 signature with alice's private key>"
}

Level 6: Action ID (a1~…)

Content-addressed action identifier computed as SHA256(complete_jwt_token).

Hash Versioning Scheme

All identifiers use a versioned prefix format for future-proofing:

{prefix}{version}~{base64_encoded_hash}

Current Prefixes

Prefix Resource Type Hash Input Example
a1~ Action Entire JWT token a1~8kR3mN9pQ2vL6xW...
f1~ File File descriptor string f1~Qo2E3G8TJZ2HTGh...
d2, Descriptor (not a hash, the encoded format itself) d2,vis.tn:b1~abc:f=avif:...
b1~ Blob Blob bytes (raw data) b1~abc123def456ghi...

Version Scheme

  • Version 1: SHA-256 with base64url encoding (no padding)
  • Future versions: Can upgrade to SHA-3, BLAKE3, etc.
  • Backward compatibility: Old content remains valid forever
  • Algorithm agility: Migrate to new algorithms without breaking existing references

Example upgrade path:

a1~...  (SHA-256)
a2~...  (SHA-3)
a3~...  (BLAKE3)

Merkle Tree Properties

Cloudillo’s content-addressing creates a merkle tree with these properties:

Immutability

Once created, content cannot be modified without changing its identifier.

Example:

Original Post:
  content: "Hello World"
  action_id: a1~abc123...

Modified Post:
  content: "Hello Universe"
  action_id: a1~xyz789...  ← DIFFERENT ID!

Any attempt to modify content results in a completely new action with a different ID. The original remains unchanged.

Tamper-Evidence

Any modification anywhere in the tree is immediately detectable.

Example:

Post (a1~abc...)
  └─ Attachment: f1~Qo2...
      └─ Variants: b1~tn..., b1~sd..., b1~md...

If someone modifies the thumbnail image:
  ✗ New variant_id (b1~xyz...)
  ✗ Descriptor changes
  ✗ New file_id (f1~uvw...)
  ✗ Post attachment no longer matches
  ✗ Verification fails!

Deduplication

Identical content produces identical identifiers, enabling automatic deduplication.

Example:

Alice posts photo.jpg → file_id: f1~abc123...
Bob posts photo.jpg (same file) → file_id: f1~abc123... (same!)

Storage: Only one copy of the blob bytes needed
Bandwidth: Can skip downloading if already cached

Verifiability

Anyone can independently verify the entire chain.

Process:

  1. Download action token
  2. Verify JWT signature (proves author identity)
  3. Recompute action_id = SHA256(JWT)
  4. Compare with claimed action_id
  5. For each attachment:
    • Download file descriptor
    • Download all variant blobs
    • Verify each variant_id = SHA256(blob)
    • Recompute file_id = SHA256(descriptor)
    • Compare with attachment reference
  6. ✓ Complete verification

No trust required: Pure mathematics ensures integrity.

Chain of Trust

Parent references create an immutable chain.

Example:

Post (a1~abc...)
  ↑ parent_id
Comment (a1~def...)
  ↑ parent_id
Reply (a1~ghi...)
  • Reply references Comment by content hash
  • Comment references Post by content hash
  • Cannot modify Post without breaking Comment reference
  • Cannot modify Comment without breaking Reply reference
  • Entire thread is cryptographically bound together

Proof of Authenticity

Cloudillo provides two complementary layers of proof:

Layer 1: Author Identity (Cryptographic Signatures)

Action tokens are signed with ES384 (ECDSA with P-384 curve and SHA-384 hash):

Action Token = Header.Payload.Signature

Signature = ECDSA_sign(SHA384(Header.Payload), alice_private_key)

Verification:

  1. Fetch alice’s public key from https://cl-o.alice.example.com/api/me/keys
  2. Verify signature using public key
  3. ✓ Proves Alice created this action

Layer 2: Content Integrity (Content Hashes)

All identifiers are SHA-256 hashes:

action_id = SHA256(entire JWT token)
file_id = SHA256(descriptor string)
variant_id = SHA256(blob bytes)

Verification:

  1. Download content
  2. Recompute hash
  3. Compare with claimed identifier
  4. ✓ Proves content hasn’t been tampered with

Verification Example

Scenario: Verify a LIKE action on a POST with image attachment.

verify_like_action(like_id):
    // 1. Verify LIKE action
    like_token = fetch_action_token(like_id)
    verify_signature(like_token, bob_public_key)
    verify like_id == SHA256(like_token)
    ✓ Bob authored this LIKE, content is intact

    // 2. Verify parent POST action
    post_id = like_token.parent_id
    post_token = fetch_action_token(post_id)
    verify_signature(post_token, alice_public_key)
    verify post_id == SHA256(post_token)
    ✓ Alice authored this POST, content is intact

    // 3. Verify file attachment
    file_id = post_token.attachments[0]
    descriptor = fetch_descriptor(file_id)
    verify file_id == SHA256(descriptor)
    ✓ File descriptor is intact

    // 4. Verify each variant blob
    variants = parse_descriptor(descriptor)
    for each variant:
        blob_data = fetch_blob(variant.blob_id)
        verify variant.blob_id == SHA256(blob_data)
        verify blob_data.size == variant.size
    ✓ All image variants are intact

    RESULT: Complete chain verified!

DAG Structure

Cloudillo’s action system forms a Directed Acyclic Graph (DAG) with these properties:

Multiple Roots

Unlike a traditional tree with one root, Cloudillo has multiple independent threads:

Post 1 (a1~abc...)                Post 2 (a1~xyz...)
│                                  │
├─ Comment 1.1 (a1~def...)        ├─ Comment 2.1 (a1~uvw...)
│  │                               │
│  ├─ Reply 1.1.1 (a1~ghi...)     └─ Like 2.1 (a1~rst...)
│  │                                  (parent: a1~uvw...)
│  └─ Reply 1.1.2 (a1~jkl...)
│
├─ Comment 1.2 (a1~mno...)
│
└─ Like 1 (a1~pqr...)
   (parent: a1~abc...)

Each top-level post is a root node. Comments and reactions form child nodes.

Performance Implications

Caching Strategy

Content-addressed data is immutable, enabling aggressive caching:

GET /api/files/f1~Qo2E3G8TJZ...

Response Headers:
  Cache-Control: public, max-age=31536000, immutable
  Content-Type: image/avif

Benefits:

  • Browsers cache forever (max-age = 1 year)
  • CDN cache forever
  • No cache invalidation needed
  • Reduces bandwidth for federated instances

Security Considerations

Trust Model

Cloudillo’s merkle tree creates a trustless verification model:

What Verification Method Trust Required
Author identity JWT signature (ES384) DNS + Public key infrastructure
Content integrity SHA-256 hash None (pure mathematics)
Parent references Content hashes None (pure mathematics)
Attachment integrity SHA-256 hash chain None (pure mathematics)

Storage providers don’t need to be trusted: Even if a storage provider is malicious, they cannot:

  • Modify content without breaking hashes ✗
  • Forge signatures without private keys ✗
  • Create false parent references ✗
  • Tamper with attachments without detection ✗

Users verify everything cryptographically—no trust required.

Attack Resistance

Known attacks and mitigations:

Attack Mitigation
Modify action content Hash mismatch detected
Forge author signature Signature verification fails
Swap file attachment Hash mismatch detected
Modify parent reference Breaks cryptographic chain
Replay old actions Timestamp validation, deduplication
Storage provider tampering Hash verification fails

See Also

Network & Security

Cloudillo’s network layer uses Rustls with the AWS-LC-RS cryptographic provider for TLS, automatic certificate management via ACME, and runs a dual-server architecture.

TLS architecture

Cloudillo uses Rustls for TLS termination, configured with HTTP/2 (preferred) and HTTP/1.1 via ALPN negotiation.

SNI-based certificate resolution

A custom CertResolver serves the correct certificate for each domain using SNI (Server Name Indication). This is essential for multi-tenant hosting where many users share a single server.

The resolver maintains an in-memory RwLock<HashMap<domain, CertifiedKey>> cache, prepopulated on startup from the database. On a TLS handshake:

  1. Check in-memory cache (fast path, read lock)
  2. On cache miss, load from database on a blocking worker thread
  3. Parse PEM certificate and private key, insert into cache
  4. Return certificate for TLS handshake

Each tenant gets entries for both its canonical domain (cl-o.{id_tag}) and any custom domain.

Certificate management

Cloudillo uses instant-acme to automatically provision and renew TLS certificates from Let’s Encrypt using the HTTP-01 challenge method.

Dual-server setup

  • HTTPS server (primary): Handles all application traffic with TLS via Rustls and the SNI-based certificate resolver
  • HTTP server (optional): Serves ACME HTTP-01 challenge responses at /.well-known/acme-challenge/{token} and redirects everything else to HTTPS

Certificate renewal

A scheduled CertRenewalTask checks all certificates periodically. Certificates expiring within 30 days are automatically renewed. The renewal uses exponential backoff (1s initial, 1.5x factor, 90s timeout) for resilience.

Certificates are stored via the AuthAdapter trait, which provides create_cert, read_cert, delete_cert, and list_domains methods.

Cryptography

Algorithms

Purpose Algorithm Details
Action token signing ES384 (P-384) Federated action tokens between instances
Access tokens HS256 (HMAC-SHA256) Session JWTs, symmetric secret per instance
Web Push (VAPID) ES256 (P-256) Push notification subscription keys
Password hashing bcrypt (cost 10) Per-password random salt
Content hashing SHA256 File IDs, content addressing, deduplication
TLS TLS 1.2/1.3 Rustls defaults, modern cipher suites

Key management

Profile signing keys use the P-384 elliptic curve (ES384). Keys are identified by date-based key IDs (format: YYMMDD) and stored in PKCS#8 PEM format. Each tenant has its own signing key pair:

  • Public key: Published for other instances to verify action tokens
  • Private key: Used to sign outgoing action tokens

A key failure cache (default size: 100 entries) prevents repeated fetch attempts for unreachable remote keys during federation.

Security policies

Memory safety

Cloudillo enforces unsafe_code = "forbid" as a workspace-wide lint, along with strict clippy rules (unwrap_used = "deny", expect_used = "deny", panic = "deny").

CORS

API endpoints use a permissive CORS policy (CorsLayer::very_permissive()). This is intentional: Cloudillo apps run in sandboxed iframes served from the /apps/ directory and need cross-origin access to the API.

Request size limits

File upload size is configurable per tenant via the file.max_file_size_mb setting (default: 50 MiB).

See also

Data Storage & Access

Cloudillo’s data storage systems and access control mechanisms. These documents explain how data is stored, organized, queried, and protected across the decentralized network.

Storage Types

Cloudillo provides three storage systems for different use cases:

Feature Blob RTDB CRDT
Primary Use Static files Structured data with queries Collaborative editing
Data Model Immutable binary Collections of JSON documents Shared types (text, maps, arrays)
Mutability Immutable (new version = new blob) Mutable with real-time sync Mutable with automatic merge
Conflict Resolution N/A (content-addressed) Last-write-wins Automatic merge (no conflicts)
Offline Support Cache only Reconnection sync Full offline with local persistence
Query Capabilities By ID, metadata, tags Rich queries (where, orderBy, limit) Read entire document
Best For Images, videos, PDFs, attachments Todos, settings, lists, forms Text editors, whiteboards, real-time docs

Blob Storage

Blob Storage - Content-addressed immutable binary data. Every file is identified by its SHA-256 hash, enabling deduplication and integrity verification. Supports automatic variant generation (thumbnails, transcoded video). Choose Blob for static files that don’t change frequently.

RTDB (Real-Time Database)

RTDB - Firebase-like API for structured data. Choose RTDB when you need to query and filter data, or when your application works with structured records (users, posts, settings). Changes sync in real-time, but concurrent edits use last-write-wins semantics.

CRDT (Collaborative Editing)

CRDT - Conflict-free replicated data types using Yjs. Choose CRDT when multiple users edit the same content simultaneously (documents, spreadsheets, presentations). All changes merge automatically without conflicts, even when users are offline.

Choosing the Right System

Use Blob when:

  • Storing static files (images, videos, PDFs)
  • Content is immutable or versioned
  • You need content-addressing and deduplication
  • Generating variants (thumbnails, transcodes)

Use RTDB when:

  • You need to query/filter data (e.g., “show incomplete todos”)
  • Data is structured as records/documents
  • Users typically edit different records
  • You need server-side validation

Use CRDT when:

  • Multiple users edit the same content simultaneously
  • You’re building a collaborative editor
  • Offline-first is critical
  • Character-level or element-level merging is needed

Many applications use all three: Blob for attachments, CRDT for document content, RTDB for metadata and settings.

Access Control

Access Control - How resources are protected and shared while maintaining user privacy and sovereignty through token-based authentication and attribute-based permissions.

Subsections of Data Storage & Access

Blob Storage

Cloudillo’s blob storage uses content-addressed storage for immutable binary data (files, images, videos). Intelligent variant generation for images ensures data integrity, deduplication, and efficient delivery across different use cases.

Content-Addressed Storage

File Identifier Format

Cloudillo uses multiple identifier types in its content-addressing system:

{prefix}{version}~{base64url_hash}

Components:

  • {prefix}: Resource type indicator (a, f, b, d)
  • {version}: Hash algorithm version (currently 1 = SHA-256)
  • ~: Separator
  • {base64url_hash}: Base64url-encoded hash (43 characters, no padding)

Identifier Types

Prefix Resource Type Hash Input Example
b1~ Blob Blob bytes (raw image/video data) b1~abc123def456...
f1~ File File descriptor string f1~QoEYeG8TJZ2HTGh...
d2, Descriptor (not a hash, the encoded format itself) d2,vis.tn:b1~abc:f=avif:...
a1~ Action Complete JWT token a1~8kR3mN9pQ2vL...

Important: d2, is not a content-addressed identifier—it’s the actual encoded descriptor string. The file ID (f1~) is the hash of this descriptor.

Examples

Blob ID:       b1~QoEYeG8TJZ2HTGhVlrtTDBpvBGOp6gfGhq4QmD6Z46w
File ID:       f1~m8Z35EIa3prvb3bhjsVjdg9SG98xd0bkoWomOHQAwCM
Descriptor:    d2,vis.tn:b1~xRAVuQtgBx_kLqZnoOSd5XqCK_aQolhq1XeXk73Zn8U:f=avif:s=1960:r=90x128
Action ID:     a1~8kR3mN9pQ2vL6xWpYzT4BjN5FqGxCmK9RsH2VwLnD8P

All file and blob IDs use SHA-256 content-addressing. See Content-Addressing & Merkle Trees for hash computation details.

File Types

Cloudillo supports four file types, each handled by different adapters based on mutability and use case:

Type Adapter Mutability Description
BLOB BlobAdapter Immutable Binary content (images, videos, documents)
CRDT CrdtAdapter Mutable Collaborative documents (Yjs-based real-time editing)
RTDB RtdbAdapter Mutable Real-time database files for app state
FLDR MetaAdapter Mutable Folder/directory metadata

File Type Selection

Files are created via different API endpoints based on their type:

Endpoint File Type Use Case
POST /api/files/{preset}/{file_name} BLOB File uploads with preset-based variant generation
POST /api/files CRDT/RTDB/FLDR Metadata-only creation for mutable file types

Available presets for BLOB uploads: default, profile-picture, cover, high_quality, mobile, archive, podcast, video, orig-only, thumbnail-only, apkg

Per-File Access Control

Each file has independent access control:

Field Values Description
Visibility P/V/F/C/null Who can discover this file
Access Level R/W Read-only vs read-write access

See Access Control for detailed permission handling.

File Variants

Concept

A single uploaded image automatically generates multiple variants optimized for different use cases:

  • pf (profile): Profile picture icon (~80px)
  • tn (thumbnail): Small preview (~256px)
  • sd (standard definition): Mobile/low bandwidth (~720px)
  • md (medium definition): Desktop viewing (~1280px)
  • hd (high definition): High quality display (~1920px)
  • xd (extra definition): 4K/maximum quality (~3840px)

File Descriptor Encoding

A file descriptor encodes all available variants in a compact format.

File Descriptor Format Specification

Format

d2,{class}.{variant}:{blob_id}:f={format}:s={size}:r={width}x{height}[:{optional}];...

Components

  • d2, - Descriptor prefix (version 2)
  • {class} - Media class:
    • vis - Visual (images: jpeg, png, webp, avif)
    • vid - Video (mp4/h264)
    • aud - Audio (opus, mp3)
    • doc - Documents (pdf)
    • raw - Original unprocessed file
  • {variant} - Quality tier: pf, tn, sd, md, hd, xd, or orig
  • {blob_id} - Content-addressed ID of the blob (b1~...)
  • f={format} - Format: avif, webp, jpeg, png, mp4, opus, pdf
  • s={size} - File size in bytes (integer, no separators)
  • r={width}x{height} - Resolution in pixels (width × height)
  • ; - Semicolon separator between variants (no spaces)

Optional Fields

For video, audio, and document files:

  • dur={seconds} - Duration in seconds (floating point, video/audio only)
  • br={kbps} - Bitrate in kbps (integer, video/audio only)
  • pg={count} - Page count (integer, documents only)

Example

d2,vis.tn:b1~abc123:f=avif:s=4096:r=150x150;vis.sd:b1~def456:f=avif:s=32768:r=640x480

This descriptor encodes two variants:

  • Thumbnail: AVIF format, 4096 bytes, 150×150 pixels, blob ID b1~abc123
  • Standard: AVIF format, 32768 bytes, 640×480 pixels, blob ID b1~def456

Video Example

d2,vis.tn:b1~abc:f=avif:s=4096:r=150x84;vid.sd:b1~def:f=mp4:s=5242880:r=720x404:dur=120.5:br=350;vid.hd:b1~ghi:f=mp4:s=20971520:r=1920x1080:dur=120.5:br=1400

This descriptor includes:

  • Thumbnail: AVIF image preview
  • SD Video: 720p MP4, 120.5 seconds, 350 kbps
  • HD Video: 1080p MP4, 120.5 seconds, 1400 kbps

Parsing Rules

  1. Check prefix: Verify descriptor starts with d2,
  2. Split by semicolon (;): Get individual variant entries
  3. For each variant, split by colon (:) to get components:
    • Component [0] = class.variant (vis.tn, vis.sd, vid.hd)
    • Component [1] = blob_id (b1~...)
    • Components [2..] = key=value pairs
  4. Parse key=value pairs:
    • f={format} → Format string
    • s={size} → Parse as u64 (bytes)
    • r={width}x{height} → Split by x, parse as u32 × u32
    • dur={seconds} → Parse as f64 (optional)
    • br={kbps} → Parse as u32 (optional)
    • pg={count} → Parse as u32 (optional)

Parsing logic: split by semicolons for variants, then by colons for fields, then parse key=value pairs.

Variant Size Classes - Exact Specifications

Cloudillo generates image variants at specific size targets to optimize bandwidth and storage:

Quality Code Max Dimension Use Case
Profile pf 80px Profile picture icons
Thumbnail tn 256px List views, previews, avatars
Standard sd 720px Mobile devices, low bandwidth
Medium md 1280px Desktop viewing
High hd 1920px High quality display
Extra xd 3840px 4K displays, maximum quality
Original orig - Unprocessed source file

Generation Rules

Which variants are generated depends on the preset configuration. The default preset generates: tn, sd, md, hd. The high_quality preset adds xd. Variants larger than the original image are automatically skipped (smaller originals are never upscaled).

Properties:

  • Each variant maintains the original aspect ratio
  • Uses Lanczos3 filter for high-quality downscaling
  • Maximum dimension constraint prevents oversizing
  • Smaller originals don’t get upscaled

Variant Selection

Clients request a specific variant:

GET /api/files/f1~Qo2E3G8TJZ...?variant=hd

Response: Returns HD variant if available, otherwise falls back to smaller variants.

Automatic Fallback

If the requested variant doesn’t exist, the server returns the best available:

  1. Try requested variant (e.g., hd)
  2. Fall back to next smaller (e.g., md)
  3. Continue until variant found
  4. Return smallest if none larger

Fallback order: xdhdmdsdtn

Content-Addressing Flow

File storage uses a three-level content-addressing hierarchy:

Level 1: Blob Storage

Upload image → Save as blob → Compute SHA256 of blob bytes → Store blob with ID: b1~{hash}

blob_data = read_file("thumbnail.avif")
blob_id = compute_hash("b", blob_data)
// Result: "b1~abc123..." (thumbnail blob ID)

Example: b1~abc123... identifies the thumbnail AVIF blob

See Content-Addressing & Merkle Trees for hash computation details.

Level 2: Variant Collection

Generate all variants (tn, sd, md, hd) → Each variant gets its own blob ID (b1~...) → Collect all variant metadata → Create descriptor string encoding all variants

variants = [
    { class: "vis.tn", blob_id: "b1~abc123", format: "avif", size: 4096, width: 150, height: 150 },
    { class: "vis.sd", blob_id: "b1~def456", format: "avif", size: 32768, width: 640, height: 480 },
    { class: "vis.md", blob_id: "b1~ghi789", format: "avif", size: 262144, width: 1920, height: 1080 },
]

descriptor = build_descriptor(variants)
// Result: "d2,vis.tn:b1~abc123:f=avif:s=4096:r=150x150;vis.sd:b1~def456:f=avif:s=32768:r=640x480;vis.md:b1~ghi789:f=avif:s=262144:r=1920x1080"

Level 3: File Descriptor

Build descriptor → Compute SHA256 of descriptor string → Final file ID: f1~{hash} → This file ID goes into action attachments

descriptor = "d2,vis.tn:b1~abc:f=avif:s=4096:r=150x150;vis.sd:b1~def:f=avif:s=32768:r=640x480"
file_id = compute_hash("f", descriptor.as_bytes())
// Result: "f1~Qo2E3G8TJZ..." (file ID)

Example Complete Flow

1. User uploads photo.jpg (3MB, 3024x4032px)

2. System generates variants:
   vis.tn:  150x200px → 4KB   → b1~abc123
   vis.sd:  600x800px → 32KB  → b1~def456
   vis.md:  1440x1920px → 256KB → b1~ghi789
   vis.hd:  2880x3840px → 1MB → b1~jkl012

3. System builds descriptor:
   "d2,vis.tn:b1~abc123:f=avif:s=4096:r=150x200;
       vis.sd:b1~def456:f=avif:s=32768:r=600x800;
       vis.md:b1~ghi789:f=avif:s=262144:r=1440x1920;
       vis.hd:b1~jkl012:f=avif:s=1048576:r=2880x3840"

4. System hashes descriptor:
   file_id = f1~Qo2E3G8TJZ2... = SHA256(descriptor)

5. Action references file:
   POST action attachments = ["f1~Qo2E3G8TJZ2..."]

6. Anyone can verify:
   - Download all variants
   - Verify each blob_id = SHA256(blob)
   - Rebuild descriptor
   - Verify file_id = SHA256(descriptor)
   - Cryptographic proof established ✓

File attachments integrate into Cloudillo’s merkle tree structure. See Content-Addressing & Merkle Trees for how files fit into the verification chain.

Image Processing Pipeline

Upload Flow

When a client uploads an image:

  1. Client Request
POST /api/files/default/profile-picture.jpg
Authorization: Bearer <access_token>
Content-Type: image/jpeg
Content-Length: 2458624

<binary image data>
  1. Dimension Extraction

Extract image dimensions and determine which variants to generate based on the preset:

img = load_image_from_memory(data)
(width, height) = img.dimensions()
max_dim = max(width, height)

# Variants come from the preset configuration
# Default preset: ["vis.tn", "vis.sd", "vis.md", "vis.hd"]
# Variants larger than the original are automatically skipped
variants = preset.image_variants.filter(|v| v.max_dim <= max_dim * 1.10)

The intermediate steps (task scheduling, hash computation, blob storage, variant generation, and metadata storage) are shown in the Complete Upload Flow Diagram below.

  1. Response

Return descriptor ID to client:

{
  "file_id": "f1~QoE...46w",
  "descriptor": "d2,vis.tn:b1~QoE...46w:f=avif:s=4096:r=128x96;vis.sd:b1~xyz...789:f=avif:s=8192:r=640x480",
  "variants": [
    {"name": "vis.tn", "format": "avif", "size": 4096, "dimensions": "128x96"},
    {"name": "vis.sd", "format": "avif", "size": 8192, "dimensions": "640x480"}
  ],
  "processing": true
}

Complete Upload Flow Diagram

Client uploads image
  ↓
POST /api/files/{preset}/filename.jpg
  ↓
Save to temp file
  ↓
Extract dimensions
  ↓
Determine variants to generate
  ↓
Create FileIdGeneratorTask
  ├─ Compute SHA256 hash
  ├─ Move to permanent storage (BlobAdapter)
  └─ Generate file_id
  ↓
Create ImageResizerTask (for each variant)
  ├─ Depends on FileIdGeneratorTask
  ├─ Load source image
  ├─ Resize with Lanczos3
  ├─ Encode to AVIF/WebP/JPEG
  ├─ Compute variant ID (SHA256)
  └─ Store in BlobAdapter
  ↓
Create file descriptor
  ├─ Collect all variant IDs
  ├─ Encode as descriptor
  └─ Store metadata in MetaAdapter
  ↓
Return descriptor ID to client

Download Flow

Client Request

GET /api/files/f1~...?variant=hd
Authorization: Bearer <access_token>

Server Processing

  1. Parse Descriptor
variants = parse_file_descriptor(file_id)
# Returns list of VariantInfo
  1. Select Best Variant
selected = select_best_variant(
    variants,
    requested_variant,   # "hd"
)

# Falls back if exact match not available:
# hd/avif → hd/webp → md/avif → md/webp → sd/avif → ...
  1. Stream from BlobAdapter
stream = blob_adapter.read_blob_stream(tn_id, selected.file_id)

# Set response headers
response.headers["Content-Type"] = f"image/{selected.format}"
response.headers["X-Cloudillo-Variant"] = selected.blob_id
response.headers["X-Cloudillo-Descriptor"] = descriptor
response.headers["Content-Length"] = selected.size

# Stream response
return stream_response(stream)

Response

HTTP/1.1 200 OK
Content-Type: image/avif
Content-Length: 16384
X-Cloudillo-Variant: b1~m8Z35EIa3prvb3bhjsVjdg9SG98xd0bkoWomOHQAwCM
X-Cloudillo-Descriptor: d2,vis.tn:b1~xRAVuQtgBx_kLqZnoOSd5XqCK_aQolhq1XeXk73Zn8U:f=avif:s=1960:r=90x128;vis.sd:b1~m8Z35EIa3prvb3bhjsVjdg9SG98xd0bkoWomOHQAwCM:f=avif:s=8137:r=256x364;vis.orig:b1~5gU72rRGiaogZuYhJy853pBd6PsqjPOjS__Kim9-qE0:f=avif:s=15012:r=256x364
Cache-Control: public, max-age=31536000, immutable

<binary image data>

Note: Content-addressed files are immutable, so can be cached forever.

Metadata Structure

FileMetadata

Stored in MetaAdapter:

FileMetadata {
    tn_id: TnId
    file_id: String           # Descriptor ID
    original_filename: String
    mime_type: String
    size: u64                 # Original size
    width: Optional[u32]
    height: Optional[u32]
    variants: List[VariantInfo]
    created_at: i64
    owner: String             # Identity tag
    permissions: FilePermissions
}

VariantInfo {
    name: String              # "tn", "sd", "md", "hd", "xd"
    file_id: String           # Content-addressed ID
    format: String            # "avif", "webp", "jpeg", "png"
    size: u64                 # Bytes
    width: u32
    height: u32
}

FilePermissions {
    public_read: bool
    shared_with: List[String]  # Identity tags
}

File Presets

Concept

Presets define how files should be processed:

FilePreset:
    Image      # Auto-generate variants
    Video      # Future: transcode, thumbnails
    Document   # Future: preview generation
    Database   # RTDB database files
    Raw        # No processing, store as-is

Upload with Preset

POST /api/files/{preset}/{filename}

Examples:
POST /api/files/default/avatar.jpg       // Generate default image variants
POST /api/files/archive/document.pdf     // Store with minimal processing

Storage Organization

BlobAdapter Layout

{data_dir}/
├── blobs/
│   ├── {tn_id}/
│   │   ├── f1~QoE...46w           // Original file
│   │   ├── f1~xyz...789           // Variant 1
│   │   ├── f1~abc...123           // Variant 2
│   │   └── ...
│   └── {other_tn_id}/
│       └── ...

MetaAdapter (SQLite)

CREATE TABLE files (
    id INTEGER PRIMARY KEY,
    tn_id INTEGER NOT NULL,
    file_id TEXT NOT NULL,
    original_filename TEXT,
    mime_type TEXT,
    size INTEGER,
    width INTEGER,
    height INTEGER,
    variants TEXT,  -- JSON array
    created_at INTEGER,
    owner TEXT,
    permissions TEXT,  -- JSON object
    UNIQUE(tn_id, file_id)
);

CREATE INDEX idx_files_owner ON files(owner);
CREATE INDEX idx_files_created ON files(created_at);

See Also

RTDB (Real-Time Database)

Query-oriented database system providing Firebase-like functionality for structured JSON data with real-time subscriptions.

Overview

The RTDB system enables:

  • JSON document storage and retrieval
  • Query filters (equals, greater than, less than)
  • Sorting and pagination
  • Computed values (increment, aggregate, functions)
  • Atomic transactions
  • Real-time subscriptions via WebSocket

Documents

  • Overview - Introduction to RTDB architecture and features
  • redb Implementation - How RTDB is implemented using the lightweight redb embedded database

Use Cases

  • User profiles and settings
  • Task lists and project management
  • E-commerce catalogs
  • Analytics and reporting
  • Structured forms and surveys

Subsections of RTDB (Real-Time Database)

RTDB Overview

Cloudillo’s RTDB (Real-Time Database) provides Firebase-like functionality for structured JSON data with queries, subscriptions, and real-time synchronization. It integrates seamlessly with Cloudillo’s federated architecture while maintaining privacy and user control.

CRDT Collaborative Editing (Separate System)

Cloudillo also provides a separate CRDT API for collaborative editing:

Technology: Yrs - Rust implementation of Yjs CRDT

Features:

  • Conflict-free replicated data types (CRDTs)
  • Rich data structures (Text, Map, Array, XML)
  • Automatic conflict resolution
  • Time-travel and versioning
  • Awareness (presence, cursors)
  • Yjs ecosystem compatibility

Use Cases:

  • Collaborative text editors (Google Docs-like)
  • Shared whiteboards
  • Real-time collaborative forms
  • Collaborative spreadsheets
  • Multiplayer game state

Learn more: CRDT Collaborative Editing

Comparison: RTDB vs CRDT

Feature RTDB (redb) CRDT (Yrs)
Purpose Structured data storage Concurrent editing
Queries Rich (filter, sort, paginate, aggregate) Limited (document-based)
Conflict Resolution Last-write-wins + document locking Automatic merge (CRDT)
Locking Soft (advisory) and hard (enforced) Not applicable
Aggregations Server-side (sum, avg, min, max, groupBy) Not applicable
Best For Traditional database needs Collaborative editing
API Style Firebase-like Yjs-compatible

Note: These are separate, complementary systems. Use RTDB for structured data with queries, and CRDT for collaborative editing scenarios.

Core Concept: Database-as-File

Both systems use the same foundational concept: databases/documents are special files in the Cloudillo file system.

How It Works

  1. File Metadata (MetaAdapter) stores:

    • Database ID, name, owner
    • Creation timestamp, last accessed
    • Permission rules
    • Configuration (max size, retention policy)
  2. Database Content (RtdbAdapter or CrdtAdapter) stores:

    • Actual data (documents, CRDT state)
    • Indexes (for query performance)
    • Snapshots (for fast loading)
  3. File ID serves as database identifier:

    /ws/rtdb/:fileId  // WebSocket connection endpoint

Benefits

  • Natural Integration: Databases managed like files
  • Permission Reuse: File permissions apply to databases
  • Federation Ready: Databases can be shared across instances
  • Content Addressing: Database snapshots are tamper-proof
  • Discoverable: Find databases through file APIs

Example

// Create database file
const response = await fetch('/api/db', {
  method: 'POST',
  headers: { 'Authorization': `Bearer ${token}` },
  body: JSON.stringify({
    name: 'My Tasks',
    type: 'redb',  // or 'yrs' for CRDT
    permissions: {
      public_read: false,
      readers: ['bob.example.com'],
      writers: ['bob.example.com']
    }
  })
});

const { fileId } = await response.json();
// fileId: "f1~abc123..."

// Connect to database via WebSocket
const ws = new WebSocket(`wss://cl-o.alice.example.com/ws/rtdb/${fileId}`);

Architecture Overview

Components

┌─────────────────────────────────────────────────────┐
│ Client Application                                  │
│  - JavaScript/TypeScript                            │
│  - React hooks / Vue composables                    │
│  - WebSocket connection                             │
└─────────────────────────────────────────────────────┘
                          ↓ WebSocket
┌─────────────────────────────────────────────────────┐
│ Cloudillo Server                                    │
│  ┌─────────────────────────────────────────────┐    │
│  │ WebSocket Handler                           │    │
│  │  - Authentication                           │    │
│  │  - Message routing                          │    │
│  │  - Subscription management                  │    │
│  └─────────────────────────────────────────────┘    │
│                        ↓                            │
│  ┌─────────────────────────────────────────────┐    │
│  │ Database Manager                            │    │
│  │  - Instance lifecycle (load/evict)          │    │
│  │  - Snapshot management                      │    │
│  │  - Memory limits                            │    │
│  └─────────────────────────────────────────────┘    │
│         ↓                           ↓               │
│  ┌─────────────┐           ┌──────────────┐         │
│  │ RtdbAdapter │           │ CrdtAdapter  │         │
│  │  (redb)     │           │  (Yrs)       │         │
│  └─────────────┘           └──────────────┘         │
│         ↓                           ↓               │
│  ┌──────────────────────────────────────────────┐   │
│  │ Storage Layer                                │   │
│  │  - MetaAdapter (metadata)                    │   │
│  │  - BlobAdapter (snapshots, data)             │   │
│  └──────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘

Permission Model

Connection-Time Access Check

Permissions are checked once at WebSocket connection time using file_access::check_file_access_with_scope(). This function evaluates multiple access sources:

  1. Scoped tokens: Share links with restricted access
  2. Ownership: File owner has full access
  3. Tenant roles: Role-based access within the tenant
  4. FSHR action tokens: Federation-based file sharing permissions

The result determines whether the connection operates in read_only or read_write mode. Clients can also request a specific access level via the ?access=read or ?access=write query parameter.

Info

There is no per-operation permission check — access level is determined at connection time and applies for the duration of the WebSocket session.

Future: Fine-Grained Permissions

Planned for future releases:

  • Per-collection permissions: Different access per table
  • Per-document permissions: Filter queries by ownership
  • Runtime rules: JavaScript-like expressions evaluated at runtime
  • Attribute-based: Permissions based on user attributes

WebSocket Protocol

Both systems (RTDB and CRDT) use WebSocket for real-time communication, though with different protocols:

Connection

const ws = new WebSocket(
  `wss://cl-o.example.com/ws/rtdb/${fileId}`,
  {
    headers: {
      'Authorization': `Bearer ${accessToken}`
    }
  }
);

Message Format

JSON messages with type field:

// Client → Server
{
  "type": "query",      // or "subscribe", "create", "update", "delete"
  "id": 123,            // Request ID for correlation
  // ... type-specific fields
}

// Server → Client
{
  "type": "queryResult", // or "change", "error"
  "id": 123,             // Matches request ID
  // ... response data
}

Storage Strategy

RTDB data is stored directly in per-tenant redb database files. The RtdbAdapter handles persistence through ACID transactions – each write operation is committed atomically. Snapshots use zstd compression, and inactive databases are evicted via LRU.

For details on the storage layout, see RTDB with redb.

Federation Support

Databases can be shared across Cloudillo instances through the file sharing mechanism (FSHR action tokens). Access from remote users is granted via the same check_file_access_with_scope() system used for local access control.

Note

Full database replication (read-only replicas, bidirectional sync) is planned for a future release. Currently, remote users connect directly to the origin instance via WebSocket.

Security Considerations

  • WebSocket connections require valid access tokens, validated on establishment
  • Permissions checked on connection; every read/write operation validated
  • Optional schema validation, size limits, and rate limiting per user
  • TLS/WSS for all connections; content-addressed snapshots prevent tampering

Choosing Between RTDB and CRDT

Use RTDB (redb) for structured data with schemas, complex queries (filters, sorts, aggregates), computed values, document locking, and atomic transactions.

Use CRDT (Yrs) for concurrent multi-user editing, conflict-free merging, rich text editing, offline-first design, and Yjs ecosystem compatibility.

Both can be used together – for example, Yrs for collaborative document editing and redb for structured metadata.

API Overview

Database Management

POST /api/db                  # Create database
GET /api/db                   # List databases
GET /api/db/:fileId          # Get metadata
PATCH /api/db/:fileId        # Update metadata
DELETE /api/db/:fileId       # Delete database

WebSocket Connection

GET /ws/rtdb/:fileId
Upgrade: websocket
Authorization: Bearer <token>

Export/Import

GET /api/db/:fileId/export?format=json
POST /api/db/:fileId/import

Next Steps

redb Implementation

The query-based RTDB uses redb, a lightweight embedded database, to provide Firebase-like functionality with minimal overhead. This approach is ideal for structured data, complex queries, and traditional database operations.

Cloudillo uses redb, a lightweight pure-Rust embedded database with ACID transactions and zero-copy reads.

Architecture

Layered Design

┌──────────────────────────────────────┐
│ Client Application (JavaScript)      │
│  - Query API                         │
│  - Subscriptions                     │
│  - Transactions                      │
└──────────────────────────────────────┘
                ↓ WebSocket
┌──────────────────────────────────────┐
│ WebSocket Handler                    │
│  - Message parsing                   │
│  - Authentication                    │
│  - Subscription tracking             │
└──────────────────────────────────────┘
                ↓
┌──────────────────────────────────────┐
│ RtdbAdapter Trait                    │
│  - transaction() → Transaction       │
│  - query(), get()                    │
│  - subscribe()                       │
│  - acquire_lock(), release_lock()    │
│  - create_index(), stats()           │
└──────────────────────────────────────┘
                ↓
┌──────────────────────────────────────┐
│ redb Implementation                  │
│  - Key-value storage                 │
│  - Secondary indexes                 │
│  - Query execution                   │
└──────────────────────────────────────┘
                ↓
┌──────────────────────────────────────┐
│ Real-Time Layer                      │
│  - tokio::broadcast channels         │
│  - Change event propagation          │
│  - Subscription filtering            │
└──────────────────────────────────────┘

RtdbAdapter Trait

The core interface for database operations. All methods are tenant-aware (tn_id parameter). Write operations go through the separate Transaction trait.

#[async_trait]
pub trait RtdbAdapter: Debug + Send + Sync {
    /// Begin a new transaction for write operations
    async fn transaction(&self, tn_id: TnId, db_id: &str) -> ClResult<Box<dyn Transaction>>;

    /// Close a database instance, flushing pending changes
    async fn close_db(&self, tn_id: TnId, db_id: &str) -> ClResult<()>;

    /// Query documents with optional filtering, sorting, and pagination
    async fn query(&self, tn_id: TnId, db_id: &str, path: &str, opts: QueryOptions)
        -> ClResult<Vec<Value>>;

    /// Get a single document at a specific path
    async fn get(&self, tn_id: TnId, db_id: &str, path: &str) -> ClResult<Option<Value>>;

    /// Subscribe to real-time changes (returns a stream of ChangeEvents)
    async fn subscribe(&self, tn_id: TnId, db_id: &str, opts: SubscriptionOptions)
        -> ClResult<Pin<Box<dyn Stream<Item = ChangeEvent> + Send>>>;

    /// Create an index on a field for query performance
    async fn create_index(&self, tn_id: TnId, db_id: &str, path: &str, field: &str)
        -> ClResult<()>;

    /// Get database statistics (size, record count, table count)
    async fn stats(&self, tn_id: TnId, db_id: &str) -> ClResult<DbStats>;

    /// Export all documents from a database
    async fn export_all(&self, tn_id: TnId, db_id: &str) -> ClResult<Vec<(Box<str>, Value)>>;

    /// Acquire a lock on a document path
    async fn acquire_lock(&self, tn_id: TnId, db_id: &str, path: &str,
        user_id: &str, mode: LockMode, conn_id: &str) -> ClResult<Option<LockInfo>>;

    /// Release a lock on a document path
    async fn release_lock(&self, tn_id: TnId, db_id: &str, path: &str,
        user_id: &str, conn_id: &str) -> ClResult<()>;

    /// Check if a path has an active lock
    async fn check_lock(&self, tn_id: TnId, db_id: &str, path: &str)
        -> ClResult<Option<LockInfo>>;

    /// Release all locks held by a specific user (on disconnect)
    async fn release_all_locks(&self, tn_id: TnId, db_id: &str,
        user_id: &str, conn_id: &str) -> ClResult<()>;
}

Transaction Trait

All write operations (create, update, delete) are performed within a transaction:

#[async_trait]
pub trait Transaction: Send + Sync {
    /// Create a new document with auto-generated ID
    async fn create(&mut self, path: &str, data: Value) -> ClResult<Box<str>>;

    /// Update an existing document (full replacement)
    async fn update(&mut self, path: &str, data: Value) -> ClResult<()>;

    /// Delete a document at a path
    async fn delete(&mut self, path: &str) -> ClResult<()>;

    /// Read a document (with read-your-own-writes semantics)
    async fn get(&self, path: &str) -> ClResult<Option<Value>>;

    /// Commit all changes atomically
    async fn commit(&mut self) -> ClResult<()>;

    /// Rollback all changes
    async fn rollback(&mut self) -> ClResult<()>;
}

Data Model

Collections and Documents

Data is organized into collections containing JSON documents:

database/
├── users/
│   ├── user_001
│   ├── user_002
│   └── ...
├── posts/
│   ├── post_abc
│   ├── post_def
│   └── ...
└── comments/
    └── ...

Document Structure

Documents are JSON objects with auto-generated IDs:

{
  "_id": "user_001",
  "_createdAt": 1738483200,
  "_updatedAt": 1738486800,
  "name": "Alice",
  "email": "alice@example.com",
  "age": 30,
  "active": true
}

System Fields (auto-managed):

  • _id: Unique document identifier
  • _createdAt: Creation timestamp (Unix)
  • _updatedAt: Last modification timestamp

Path Syntax

Paths use slash-separated segments:

users                    // Collection
users/user_001          // Specific document
posts/post_abc/comments // Sub-collection

Query System

QueryOptions

pub struct QueryOptions {
    pub filter: Option<QueryFilter>,
    pub sort: Option<Vec<SortField>>,  // Multiple sort fields supported
    pub limit: Option<u32>,
    pub offset: Option<u32>,
    pub aggregate: Option<AggregateOptions>,
}

pub struct SortField {
    pub field: String,
    pub ascending: bool,  // true for ascending, false for descending
}

QueryFilter

QueryFilter is a flat struct (not an enum) where each field is a HashMap<String, Value>. Multiple conditions within the struct are ANDed implicitly — a document must satisfy all specified constraints. All field names use camelCase serialization.

#[serde(rename_all = "camelCase")]
pub struct QueryFilter {
    pub equals: HashMap<String, Value>,
    pub not_equals: HashMap<String, Value>,
    pub greater_than: HashMap<String, Value>,
    pub greater_than_or_equal: HashMap<String, Value>,
    pub less_than: HashMap<String, Value>,
    pub less_than_or_equal: HashMap<String, Value>,
    pub in_array: HashMap<String, Vec<Value>>,
    pub array_contains: HashMap<String, Value>,
    pub not_in_array: HashMap<String, Vec<Value>>,
    pub array_contains_any: HashMap<String, Vec<Value>>,
    pub array_contains_all: HashMap<String, Vec<Value>>,
}
Info

There are no And/Or combinators — multiple conditions are ANDed implicitly. Each HashMap maps field names to expected values.

Query Examples

Simple query:

{
  "type": "query",
  "id": 1,
  "path": "users",
  "filter": {
    "equals": { "active": true }
  },
  "sort": [{ "field": "name", "ascending": true }],
  "limit": 50
}

Complex query (multiple conditions are ANDed):

{
  "type": "query",
  "id": 2,
  "path": "posts",
  "filter": {
    "equals": { "published": true },
    "greaterThan": { "views": 100 }
  },
  "sort": [{ "field": "createdAt", "ascending": false }],
  "limit": 20
}

WebSocket Protocol

Message Types

Client → Server

1. Query

{
  "type": "query",
  "id": 1,
  "path": "users",
  "filter": { "equals": { "active": true } },
  "limit": 50
}

2. Subscribe

{
  "type": "subscribe",
  "id": 2,
  "path": "posts",
  "filter": { "equals": { "published": true } }
}

3. Get (single document)

{
  "type": "get",
  "id": 3,
  "path": "users/user_001"
}

4. Transaction (wraps create/update/delete operations)

{
  "type": "transaction",
  "id": 4,
  "operations": [
    {
      "type": "create",
      "path": "posts",
      "ref": "$post",
      "data": { "title": "Hello", "author": "alice" }
    },
    {
      "type": "update",
      "path": "users/alice",
      "data": { "postCount": { "$op": "increment", "by": 1 } }
    },
    {
      "type": "replace",
      "path": "users/alice",
      "data": { "name": "Alice", "role": "admin" }
    },
    {
      "type": "delete",
      "path": "posts/post_old"
    }
  ]
}
Info

All write operations (create, update, replace, delete) must be wrapped in a transaction message. There are no standalone write message types. The update operation merges fields into the existing document, while replace does a full document replacement.

5. Lock

{
  "type": "lock",
  "id": 5,
  "path": "todos/task_123",
  "mode": "hard"
}

6. Unlock

{
  "type": "unlock",
  "id": 6,
  "path": "todos/task_123"
}

7. Create Index

{
  "type": "createIndex",
  "id": 7,
  "path": "users",
  "field": "email"
}

8. Ping

{
  "type": "ping",
  "id": 8
}

Server → Client

1. Query Result

{
  "type": "queryResult",
  "id": 1,
  "data": [
    { "_id": "user_001", "name": "Alice", "active": true },
    { "_id": "user_002", "name": "Bob", "active": true }
  ],
  "total": 2
}

2. Get Result

{
  "type": "getResult",
  "id": 3,
  "data": { "_id": "user_001", "name": "Alice", "active": true }
}

3. Subscribe Result

{
  "type": "subscribeResult",
  "id": 2,
  "subscriptionId": "sub_abc123",
  "data": [
    { "_id": "post_001", "title": "Hello", "published": true }
  ]
}

4. Change Event

Change events use a single event object with an action field, not a changes array:

{
  "type": "change",
  "subscriptionId": "sub_abc123",
  "event": {
    "action": "create",
    "path": "posts",
    "data": { "_id": "post_002", "title": "New Post", "published": true }
  }
}

Possible action values: create, update, delete, lock, unlock, ready

The ready action is sent after the initial subscription data has been delivered.

5. Transaction Result

{
  "type": "transactionResult",
  "id": 4,
  "results": [
    { "ref": "$post", "id": "post_new_001" },
    { "id": "users/alice" },
    { "id": "posts/post_old" }
  ]
}

6. Lock Result

{
  "type": "lockResult",
  "id": 5,
  "locked": true
}

If the lock is denied:

{
  "type": "lockResult",
  "id": 5,
  "locked": false,
  "holder": "bob@example.com",
  "mode": "hard"
}

7. Error

{
  "type": "error",
  "id": 4,
  "code": "permission_denied",
  "message": "Insufficient permissions to update this document"
}

8. Pong

{
  "type": "pong",
  "id": 8
}

Real-Time Subscriptions

Subscription Flow

Client sends subscribe message
  ↓
Server validates permissions
  ↓
Server creates broadcast channel
  ↓
Server executes initial query
  ↓
Server sends subscribeResult with data
  ↓
Server watches for changes matching filter
  ↓
On change: Server sends change event
  ↓
Client updates local state

Implementation

Subscription Structure:

  • id: Unique subscription identifier
  • path: Collection path being subscribed to
  • filter: Optional query filter to match changes
  • sender: Broadcast channel for sending change events

Change Event Types

ChangeEvent is a tagged enum with #[serde(tag = "action")] serialization:

#[serde(tag = "action", rename_all = "camelCase")]
pub enum ChangeEvent {
    Create { path: Box<str>, data: Value },
    Update { path: Box<str>, data: Value, old_data: Option<Value> },
    Delete { path: Box<str>, old_data: Option<Value> },
    Lock   { path: Box<str>, data: Value },
    Unlock { path: Box<str>, data: Value },
    Ready  { path: Box<str>, data: Option<Value> },
}

This serializes as {"action": "create", "path": "...", "data": {...}} — the action field determines the variant.

Transactions

Atomic Operations

Transactions ensure multiple operations execute atomically:

{
  "type": "transaction",
  "id": 10,
  "operations": [
    {
      "type": "update",
      "path": "accounts/alice",
      "data": { "balance": { "$op": "increment", "by": -100 } }
    },
    {
      "type": "update",
      "path": "accounts/bob",
      "data": { "balance": { "$op": "increment", "by": 100 } }
    }
  ]
}

Guarantees:

  • All operations succeed or all fail
  • Intermediate states never visible
  • Sequential consistency

Temporary References

Reference documents created within the same transaction:

{
  "type": "transaction",
  "id": 11,
  "operations": [
    {
      "type": "create",
      "path": "posts",
      "ref": "$post",
      "data": { "title": "My Post", "author": "alice" }
    },
    {
      "type": "create",
      "path": "comments",
      "data": {
        "postId": { "$ref": "$post" },
        "text": "First comment!",
        "author": "alice"
      }
    }
  ]
}

How it works:

  1. First operation creates post, saves ID as $post
  2. Second operation references $post, replaced with actual ID
  3. Comment gets correct post ID even though it wasn’t known initially

Document Locking

The RTDB supports document-level locking for exclusive or advisory editing access.

Lock Modes

  • Soft lock (advisory): Other clients can still write but receive a notification that the document is locked. Useful for signaling editing intent.
  • Hard lock (enforced): The server rejects writes from other clients while the lock is held. Only the lock holder (identified by conn_id) can modify the document.

Lock/Unlock Messages

Client → Server:

{ "type": "lock", "id": 1, "path": "todos/task_123", "mode": "soft" }
{ "type": "unlock", "id": 2, "path": "todos/task_123" }

Server → Client:

{ "type": "lockResult", "id": 1, "locked": true }
{ "type": "lockResult", "id": 1, "locked": false, "holder": "bob@example.com", "mode": "hard" }

TTL-Based Expiration

Locks expire automatically after a TTL (time-to-live) period. This prevents permanently locked documents when clients disconnect unexpectedly or crash without releasing their locks. The server cleans up expired locks during its periodic maintenance cycle.

Connection-Based Echo Suppression

The server tracks lock ownership by conn_id. When a lock change event is broadcast to subscribers, the originating connection is excluded from the notification (echo suppression), similar to how write operations suppress echoes. This prevents the client that acquired the lock from receiving its own lock notification.

Lock Status in Change Events

Active subscriptions receive lock/unlock events as part of the change stream:

{
  "type": "change",
  "subscriptionId": "sub_abc123",
  "event": {
    "action": "lock",
    "path": "todos/task_123",
    "data": { "holder": "alice@example.com", "mode": "hard" }
  }
}

Aggregate Queries

The RTDB supports server-side aggregate computations on query results.

Aggregate Request

Add the aggregate option to a query or subscribe message:

{
  "type": "query",
  "id": 1,
  "path": "tasks",
  "filter": { "equals": { "completed": false } },
  "aggregate": {
    "groupBy": "status",
    "ops": [
      { "op": "sum", "field": "hours" },
      { "op": "avg", "field": "hours" }
    ]
  }
}

Aggregate Operations

Operation Description
sum Sum of a numeric field
avg Average of a numeric field
min Minimum value of a field
max Maximum value of a field

Each group always includes a count of matching documents.

Aggregate Response

{
  "type": "queryResult",
  "id": 1,
  "aggregate": {
    "groups": [
      { "group": "todo", "count": 12, "sum_hours": 36, "avg_hours": 3.0 },
      { "group": "in_progress", "count": 5, "sum_hours": 20, "avg_hours": 4.0 }
    ]
  }
}

Incremental Aggregate Subscriptions

When aggregate is used with a subscribe message, the server computes aggregates incrementally. On each change event that affects the subscribed path and filter, the server recalculates the affected groups and sends an updated aggregate snapshot rather than the full document set. This keeps aggregate subscriptions efficient even for large collections.

Computed Values

Field Operations

Modify field values with special operations:

Increment:

{
  "views": { "$op": "increment", "by": 1 }
}

Append (to array):

{
  "tags": { "$op": "append", "value": "javascript" }
}

Remove (from array):

{
  "tags": { "$op": "remove", "value": "draft" }
}

Other field operations: decrement, multiply, concat, min, max, setIfNotExists — all use the {"$op": "operation_name", ...} format.

Query Operations

Aggregate data within queries using $query:

Count:

{
  "type": "query",
  "id": 12,
  "path": "posts",
  "query": { "$query": "count" },
  "filter": { "equals": { "published": true } }
}

The sum and avg operations follow the same pattern with an additional "field" parameter (e.g., { "$query": "sum", "field": "total" }).

Function Operations

Server-side functions for computed values:

Now (current timestamp):

{
  "createdAt": { "$fn": "now" }
}

Slugify (URL-safe string):

{
  "slug": { "$fn": "slugify", "input": "Hello World!" }
  // Results in: "hello-world"
}

Other function operations: lowercase, hash (SHA256) — all use the {"$fn": "function_name", "input": "..."} format.

Indexing

Secondary Indexes

Improve query performance:

Index Definition:

  • name: Unique name for the index (e.g., “idx_email”)
  • fields: Vector of field names to index (single or compound)
  • unique: Boolean flag ensuring no duplicate values (for unique constraints)

Index Usage

Queries automatically use indexes when available:

{
  "type": "query",
  "path": "users",
  "filter": { "equals": { "email": "alice@example.com" } }
}
// Uses idx_email if available, otherwise full scan

Index Strategies

Single-field indexes:

fields: vec!["email"]           // For email lookups
fields: vec!["createdAt"]       // For sorting by date

Compound indexes:

fields: vec!["category", "price"]  // For category + price queries

Unique indexes:

unique: true  // Ensures no duplicates (e.g., email, username)

Client SDK

See the @cloudillo/rtdb Client SDK documentation for JavaScript and React integration examples.

See Also

CRDT (Collaborative Editing)

Conflict-free replicated data types enabling true collaborative editing with automatic conflict resolution.

Overview

The CRDT system provides:

  • Conflict-free replicated data types (CRDTs)
  • Rich data structures (Text, Map, Array, XML)
  • Automatic conflict resolution
  • Real-time synchronization
  • Offline editing support
  • Yrs/Yjs ecosystem compatibility
For App Developers

If you’re building collaborative applications, see the CRDT Design Guide for schema design patterns, best practices, and common pitfalls when working with Yjs/CRDTs.

Documents

Subsections of CRDT (Collaborative Editing)

CRDT Overview

Cloudillo’s CRDT system uses Yrs, a Rust implementation of the Yjs CRDT (Conflict-free Replicated Data Type), to enable true collaborative editing with automatic conflict resolution. This is a separate API from RTDB, optimized specifically for concurrent editing scenarios where multiple users modify the same data simultaneously.

What are CRDTs?

Conflict-free Replicated Data Types (CRDTs) are data structures that can be replicated across multiple nodes and modified independently, then merged automatically without conflicts.

Key Properties

  1. Eventual Consistency: All replicas converge to the same state
  2. No Central Authority: No server needed to resolve conflicts
  3. Deterministic Merging: Same operations always produce same result
  4. Commutative: Order of operations doesn’t matter
  5. Idempotent: Applying same operation twice has no extra effect

Example: Concurrent Editing

Alice's editor:
  "Hello"
  ↓ (adds " World")
  "Hello World"

Bob's editor (at same time):
  "Hello"
  ↓ (adds "!")
  "Hello!"

After sync (both converge to):
  "Hello World!"
  ↑ CRDT algorithm automatically merges

Why Yrs?

Cloudillo uses Yrs, the Rust port of the battle-tested Yjs CRDT library:

  • Production-Ready: Used in Google Docs-like applications
  • Pure Rust: Memory-safe, high performance
  • Rich Data Types: Text, Map, Array, XML
  • Ecosystem Compatible: Works with Yjs clients (JavaScript)
  • Battle-Tested Protocol: Proven sync algorithm
  • Awareness API: Presence, cursors, selections

Architecture

Components

┌──────────────────────────────────────────────┐
│ Client Application                           │
│  - Yjs Document (Y.Doc)                      │
│  - Shared types (Y.Text, Y.Map, Y.Array)     │
│  - Awareness (presence, cursors)             │
└──────────────────────────────────────────────┘
               ↓ WebSocket (binary protocol)
┌──────────────────────────────────────────────┐
│ Cloudillo Server (stateless — no Y.Doc)      │
│  ┌────────────────────────────────────────┐  │
│  │ WebSocket Handler                      │  │
│  │  - Yrs message decoding                │  │
│  │  - Binary message parsing              │  │
│  │  - Authentication (at connection)      │  │
│  └────────────────────────────────────────┘  │
│                    ↓                         │
│  ┌────────────────────────────────────────┐  │
│  │ Document Registry (CRDT_DOCS)          │  │
│  │  - Broadcast channels per document     │  │
│  │  - Connection tracking                 │  │
│  │  - Cleanup on last disconnect          │  │
│  └────────────────────────────────────────┘  │
│                    ↓                         │
│  ┌─────────────────────────────────────────┐ │
│  │ Storage Layer                           │ │
│  │  - CrdtAdapter (binary updates)         │ │
│  │  - Optimization on last disconnect      │ │
│  └─────────────────────────────────────────┘ │
└──────────────────────────────────────────────┘

Connection Tracking

Each connected client is tracked with a CrdtConnection:

struct CrdtConnection {
    conn_id: String,       // Unique connection ID (distinguishes multiple tabs)
    user_id: String,
    doc_id: String,
    tn_id: TnId,
    awareness_tx: Arc<broadcast::Sender<(String, Vec<u8>)>>,  // Awareness channel
    sync_tx: Arc<broadcast::Sender<(String, Vec<u8>)>>,       // Sync channel
    last_access_update: Mutex<Option<Instant>>,   // Throttled access tracking
    last_modify_update: Mutex<Option<Instant>>,   // Throttled modification tracking
    has_modified: AtomicBool,                      // Whether session made changes
}
Info

The server is stateless — it does not hold a Y.Doc in memory. Instead, it stores raw binary updates via the CrdtAdapter and broadcasts them to other clients via channels. Document state only exists in the connected clients.

Document Registry

Active documents are tracked in a global registry:

type DocChannels = (
    Arc<broadcast::Sender<(String, Vec<u8>)>>,  // Awareness: (conn_id, data)
    Arc<broadcast::Sender<(String, Vec<u8>)>>,  // Sync: (conn_id, data)
);

static CRDT_DOCS: LazyLock<RwLock<HashMap<String, DocChannels>>> =
    LazyLock::new(|| RwLock::new(HashMap::new()));

When a client connects, the server gets or creates broadcast channels for the document. When a client disconnects, if both channels have zero receivers, the entry is removed from the registry.

Data Types

Cloudillo supports all Yjs shared types: Y.Text (collaborative text), Y.Map (key-value), Y.Array (ordered lists), and Y.XmlFragment (structured documents). See the Yjs documentation for usage details.

WebSocket Sync Protocol

Connection Flow

Client                          Server
  |                               |
  |--- GET /ws/crdt/:docId ------>|
  |    (Authorization: Bearer...) |
  |                               |--- Validate token
  |                               |--- Load database instance
  |                               |--- Create session
  |<-- 101 Switching Protocols ---|
  |                               |
  |<====== WebSocket Open =======>|
  |                               |
  |<-- Update (stored update 1) --|  Server sends all stored
  |<-- Update (stored update 2) --|  updates from CrdtAdapter
  |<-- Update (stored update N) --|  as SyncMessage::Update
  |                               |
  |<====== Synchronized =========>|
  |                               |
  |--- Update (user edits) ------>|--- Store via CrdtAdapter
  |<-- Update (echo back) --------|--- Echo to sender
  |                               |--- Broadcast to other clients
  |<-- Update (remote edits) -----|
  |                               |
  |--- Awareness Update --------->|--- Echo to sender
  |<-- Awareness Update ----------|--- Broadcast to others

Message Types

All messages use the Yjs sync protocol binary format (lib0 encoding, not JSON):

  • MSG_SYNC (0): Sync protocol messages (SyncStep1, SyncStep2, Update)
  • MSG_AWARENESS (1): User presence/cursor updates

Messages are encoded/decoded using yrs::sync::Message.

Update (Primary message type)

Document updates (changes), used for both initial sync and live editing:

YMessage::Sync(SyncMessage::Update(update_data))

The server sends stored updates as SyncMessage::Update messages during initial sync. It does not use state vector exchange — since the server has no Y.Doc in memory, it cannot compute a state vector.

Awareness Update

Presence information (cursors, selections):

YMessage::Awareness(awareness_update)

WebSocket Connection Handler

Algorithm: Handle CRDT WebSocket Connection

Input: WebSocket, user_id, doc_id, app, tn_id, read_only
Output: ()

1. Connection Setup:
   - Generate unique conn_id
   - Get or create broadcast channels for this document (CRDT_DOCS registry)
   - Create CrdtConnection struct
   - Record initial file access (throttled)

2. Send Initial Sync:
   - Load all stored updates via CrdtAdapter.get_updates()
   - If no updates exist: create initial Y.Doc with meta map, store it
   - Send each stored update as SyncMessage::Update to client

3. Spawn Concurrent Tasks:
   - Heartbeat task: sends ping frames every 15 seconds
   - Receive task: processes incoming WebSocket messages
   - Sync broadcast task: forwards CRDT updates from other clients
   - Awareness broadcast task: forwards awareness updates from other clients

4. Message Loop (receive task):
   For each binary WebSocket message:

   a. SYNC Update:
      - If read_only: silently reject, return
      - Validate update with Update::decode_v1() (catches corruption)
      - Store via CrdtAdapter.store_update()
      - If store fails: skip broadcasting (prevents data loss)
      - Broadcast to other clients via sync_tx channel
      - Echo back to sender

   b. AWARENESS Update:
      - Broadcast to other clients via awareness_tx channel
      - Echo back to sender

   c. Broadcast tasks (per client):
      - Receive from channel, skip messages from own conn_id
      - Forward to client via WebSocket

5. Connection Close:
   - Record final file access/modification
   - Abort heartbeat, sync, and awareness tasks
   - Check if last connection: if so, wait 2s grace period
   - If still no connections after grace: optimize document

This pattern ensures:
- Stateless server (no Y.Doc in memory)
- Echo + broadcast (sender gets echo, others get broadcast)
- Persistence before broadcasting (no data loss on crash)
- Automatic optimization when all clients disconnect

Awareness (Presence)

What is Awareness?

Awareness tracks ephemeral state like:

  • User presence (online/offline)
  • Cursor positions
  • Text selections
  • Custom user state (name, color, etc.)

See the Yjs Awareness documentation for client-side integration.

Storage Strategy

Update Storage

All CRDT changes are stored as individual binary updates via the CrdtAdapter. When a new client connects, all stored updates are sent one by one. There is no snapshot system — updates accumulate and are merged during optimization.

Optimization (Update Merging)

When the last client disconnects from a document, the server merges all stored updates into a single compacted update:

Algorithm: Optimize Document

Input: app, tn_id, doc_id
Output: ()

1. Wait Grace Period:
   - After last disconnect, wait 2 seconds
   - Re-check that no new connections were established
   - If new client connected: skip optimization

2. Load All Updates:
   - Get all stored updates via CrdtAdapter.get_updates()
   - If 0 or 1 updates: skip (nothing to optimize)

3. Merge Updates (CPU-bound, runs in spawn_blocking):
   - Decode all updates with Update::decode_v1()
   - Skip corrupted updates (log warnings)
   - Create a temporary Y.Doc
   - Apply all decoded updates to the Doc
   - Encode full state as a single update via encode_state_as_update_v1()

4. Size Check:
   - Compare merged size vs total original size
   - If no size reduction: skip (optimization would not help)

5. Replace Stored Updates:
   - Delete all old updates via CrdtAdapter.delete_doc()
   - Store the single merged update via CrdtAdapter.store_update()
   - Mark as system-generated (client_id = "system")

Benefits:
- Reduces initial sync time for subsequent connections
- Reduces storage usage
- Happens automatically when document is idle

Memory Management

Document channel cleanup is based on receiver count. When a client disconnects, the server checks if both broadcast channels (awareness and sync) have zero receivers. If so, the document entry is removed from the CRDT_DOCS registry.

Cleanup Algorithm:
1. Client disconnects
2. Check awareness_tx.receiver_count() and sync_tx.receiver_count()
3. If both are 0:
   - Remove entry from CRDT_DOCS registry
   - Wait 2s grace period
   - Re-check for new connections
   - If still no connections: run optimization
4. If receivers remain: no cleanup needed

Since the server holds no Y.Doc in memory, there is no document eviction needed — only the lightweight broadcast channels are held per active document.

Client Integration

Connect to Cloudillo’s CRDT endpoint using the standard y-websocket provider with the WebSocket URL wss://cl-o.{domain}/ws/crdt/{fileId} and an auth token as a query parameter.

Security Considerations

Authentication

WebSocket connections use axum’s OptionalAuth extractor for authentication:

  1. Extract auth context from the WebSocket upgrade request
  2. If no auth context: reject with close code 4401 (“Unauthorized”)
  3. Auth context provides: id_tag, tn_id, roles, and optional scope

Permission Enforcement

Permissions are checked once at connection time using file_access::check_file_access_with_scope(). This function evaluates:

  1. Scoped tokens: Share links with restricted access (read-only or read-write)
  2. Ownership: File owner has full access
  3. Tenant roles: Role-based access within the tenant
  4. FSHR action tokens: Federation-based file sharing permissions

The result determines whether the connection is read_only or read_write. Clients can also request a specific access level via the ?access=read or ?access=write query parameter.

Warning

Access level is checked once at connection time but not re-validated during the session. If a user’s access is revoked (e.g., an FSHR action is deleted), they retain their original access level until they reconnect.

Read-Only Enforcement

Read-only connections (determined at connection time) are enforced at the message handler level. When a read-only client sends an Update message, the server silently rejects it — the update is not stored and not broadcast. The client will see its changes rejected on the next sync cycle.

See Also

redb Implementation

The CRDT adapter stores collaborative document updates persistently using redb, enabling conflict-free replicated data types to survive server restarts while maintaining real-time synchronization capabilities.

Architecture Overview

The CRDT adapter bridges between the Yrs CRDT engine (in-memory) and persistent storage, storing binary update streams that can be replayed to reconstruct document state.

Client 1 ─┐
          ├─► Yrs CRDT ─► Binary Updates ─► redb Storage
Client 2 ─┘      │                              │
                 │◄──────── Load on startup ────┘
                 └─► Broadcast ─► Subscribed Clients

Key Insight

CRDT systems work by accumulating operation updates rather than storing full document state. Each update is a binary-encoded operation (insert, delete, format, etc.) that can be:

  • Applied to reconstruct current document state
  • Sent to new subscribers for synchronization
  • Replayed in any order (commutative property)

Storage Layout

The adapter uses a single redb table per database:

Updates Table (crdt_updates_v2)

Stores binary CRDT update blobs using structured binary keys for efficient range scanning.

Schema: binary_key → update_bytes

Key Format (34 bytes total):
  [version: 1 byte]       Protocol version (currently 1)
  [doc_id: 24 bytes]      Fixed-length document ID (zero-padded)
  [type: 1 byte]          Record type (0=update, 1=state_vector, 2=metadata)
  [seq: 8 bytes BE]       Sequence number in big-endian (for proper sorting)

Value: Binary CRDT update blob (from Yrs)

Properties:

  • Binary keys enable efficient range scans per document
  • Big-endian sequence numbers ensure correct sort order in B-tree
  • Fixed-length doc_id allows prefix-based document scanning
  • Updates are append-only (immutable)
  • Record type field reserves space for future state vector/metadata records

The adapter uses only this single table. Document metadata (ownership, permissions) is managed by the MetaAdapter, not the CRDT storage layer. Statistics (update count, byte size) are computed dynamically from the stored updates via the CrdtAdapter::stats() default implementation.

Multi-Tenancy Storage Modes

The adapter supports two storage strategies configured at initialization:

Per-Tenant Files Mode (per_tenant_files=true)

Each tenant gets a dedicated redb file:

storage/
├── tn_{tn_id}.db      (e.g., tn_42.db for tenant 42)
├── tn_{tn_id}.db      (one file per tenant)
└── ...

Advantages:

  • ✅ Complete isolation between tenants
  • ✅ Independent backups per tenant
  • ✅ Easier to delete/archive specific tenants
  • ✅ Better fault isolation

Trade-offs:

  • ⚠️ More file handles required
  • ⚠️ Slightly higher disk overhead

Use case: Multi-tenant SaaS deployments where tenant isolation is critical

Single File Mode (per_tenant_files=false)

All tenants share one database:

storage/
└── crdt.db      (All tenants)

Advantages:

  • ✅ Fewer file handles
  • ✅ Simpler operational management
  • ✅ Easier bulk operations

Trade-offs:

  • ⚠️ No physical isolation between tenants
  • ⚠️ Tenant deletion requires filtering

Use case: Single-user deployments or trusted environments

In-Memory Document Instances

The adapter caches document instances in memory to optimize performance and enable real-time subscriptions.

DocumentInstance Structure

struct DocumentInstance {
    broadcaster: broadcast::Sender<CrdtChangeEvent>,
    last_accessed: AtomicU64,   // Timestamp for LRU eviction
    update_count: AtomicU64,    // Sequence counter (initialized from DB max seq)
}

Each instance provides:

  • Broadcast channel: Real-time notifications to subscribed clients
  • Sequence counter: Monotonically increasing update numbers
  • LRU tracking: Timestamp for idle document eviction

Caching Strategy

Cache population:

  1. Document accessed → Check cache (DashMap)
  2. If missing → Create instance with broadcast channel
  3. Store in cache with initial timestamp

LRU eviction (configurable):

  • max_instances: Maximum cached documents (default: 100)
  • idle_timeout_secs: Evict after N seconds idle (default: 300s)
  • auto_evict: Enable/disable automatic eviction (default: true)

Benefits:

  • Avoid reopening redb transactions repeatedly
  • Enable efficient pub/sub without polling
  • Reduce memory usage for inactive documents

Core Operations

Storing Updates

When a client modifies a CRDT document:

1. Client sends binary update → Server
2. Adapter fetches/creates DocumentInstance
3. Sequence number assigned (atomic increment)
4. Update stored with binary key: updates[encode_update_key(doc_id, seq)] = update_bytes
5. Broadcast update to all subscribers
6. Return success

Atomicity: Each update is stored in a redb write transaction, ensuring crash consistency.

Key generation:

fn encode_update_key(doc_id: &str, seq: u64) -> [u8; 34] {
    // [version:1][doc_id:24][type:1][seq:8_BE]
    let mut key = [0u8; 34];
    key[0] = 1; // version
    let doc_bytes = doc_id.as_bytes();
    key[1..1 + doc_bytes.len().min(24)].copy_from_slice(&doc_bytes[..doc_bytes.len().min(24)]);
    key[25] = 0; // record_type::UPDATE
    key[26..].copy_from_slice(&seq.to_be_bytes());
    key
}

Loading Updates

When a client opens a document:

1. Range scan: updates.range(make_doc_range(doc_id))
   (scans from [version, doc_id, UPDATE, 0] to [version, doc_id, UPDATE, u64::MAX])
2. Read binary blobs in sequence order (big-endian keys ensure correct order)
3. Return Vec<CrdtUpdate>
4. Client applies updates to Yrs document → reconstructs state

Performance: Prefix scans are efficient in redb’s B-tree structure.

Subscriptions

Clients can subscribe to document changes:

Without snapshot (new updates only):

let mut rx = instance.broadcaster.subscribe();
while let Ok(event) = rx.recv().await {
    // Send update to client
}

With snapshot (full sync):

// 1. Send all existing updates (from redb)
for update in get_updates(doc_id).await? {
    yield update;
}

// 2. Then stream new updates (from broadcaster)
let mut rx = instance.broadcaster.subscribe();
while let Ok(event) = rx.recv().await {
    yield event;
}

Snapshot mode enables new clients to:

  1. Receive complete document history
  2. Reconstruct current state
  3. Continue receiving live updates

Deleting Documents

1. Begin write transaction
2. Range scan to find all updates: make_doc_range(doc_id)
3. Collect keys, then delete each one
4. Commit transaction
5. Remove from instance cache

Note: Compaction (merging updates) is performed automatically by the WebSocket layer when the last client disconnects from a document (see CRDT Overview).

Configuration

struct AdapterConfig {
    max_instances: usize,        // Default: 100
    idle_timeout_secs: u64,      // Default: 300 (5 minutes)
    broadcast_capacity: usize,   // Default: 1000 messages
    auto_evict: bool,            // Default: true
}

Tuning guidance:

  • High traffic: Increase max_instances and broadcast_capacity
  • Memory constrained: Reduce max_instances, lower idle_timeout_secs
  • Long-running docs: Disable auto_evict or increase timeout

Update Compaction

The WebSocket layer automatically merges updates when the last client disconnects from a document:

  1. Wait 2-second grace period (in case a new client connects)
  2. Load all updates and apply to a temporary Y.Doc
  3. Encode current state as a single update
  4. If the merged update is smaller: replace all updates with the single merged one
  5. Sequence counter resets for the optimized document

Benefits:

  • Faster document loading for subsequent connections
  • Reduced storage usage
  • Shorter sync times for new clients

Trade-off: Loses granular edit history

Comparison with RTDB Storage

Aspect CRDT (crdt-adapter-redb) RTDB (rtdb-adapter-redb)
Data model Binary operation stream Structured JSON documents
Storage Sequential update blobs Hierarchical key-value
Queries None (replays all updates) Filters, sorting, pagination
Concurrency Automatic conflict resolution Last-write-wins
Use case Collaborative text editing Structured data with queries
Sync protocol Full update stream Partial updates via queries

Performance Characteristics

Write performance:

  • Insert update: O(log N) (B-tree insert)
  • Broadcast: O(M) where M = subscriber count
  • Typical: <1ms for small updates

Read performance:

  • Load document: O(K log N) where K = update count
  • Subscription: O(1) after initial load
  • Typical: 50-200ms for 1000 updates

Memory usage:

  • Per instance: ~1KB overhead
  • Broadcast buffer: broadcast_capacity × avg_update_size
  • Total: ~100KB for 100 cached docs

See Also

Access Control

Cloudillo’s access control and permission systems for protecting resources while maintaining user privacy and enabling secure resource sharing across the decentralized network.

Core Subsystems

  • Access Control & Resource Sharing — Token-based authentication and cross-instance authorization
  • Permission System — Two-pillar system: ABAC policies (community-level constraints/guarantees) and discretionary access control (visibility, shares, audience)

Key Concepts

When a user wants to access a resource stored on another node, Cloudillo uses cryptographic tokens to grant access without requiring direct trust between storage providers. This process is designed to be seamless, requiring no additional user interaction while maintaining full decentralization.

Subsections of Access Control

Access Control & Resource Sharing

Access tokens are used to authenticate and authorize requests to the API. They are usually bound to a resource, which can reside on any node within the Cloudillo network.

Token Types

Cloudillo uses different token types for different purposes:

AccessToken

Session tokens for authenticated API requests.

Purpose: Grant a client access to specific resources Format: JWT (JSON Web Token) Lifetime: 1-24 hours (configurable)

JWT Claims:

{
  "iss": "alice.example.com",   // Issuer (identity of the node)
  "sub": "alice.example.com",   // Subject (user identity, for cross-instance)
  "exp": 1738483200,            // Expiration timestamp
  "scope": "resource_id",       // Scope (resource identifier)
  "r": "USR"                    // Roles
}

ActionToken

Cryptographically signed tokens representing user actions (see Actions.

Purpose: Federated activity distribution Format: JWT signed with profile key (ES384) Lifetime: Permanent (immutable, content-addressed)

ProxyToken

Inter-instance authentication tokens.

Purpose: Get access tokens from a remote instance (on behalf of a user) Format: JWT Lifetime: Short-lived (1-5 minutes)

JWT Claims:

{
  "sub": "alice.example.com",  // Requesting identity
  "aud": "bob.example.com",    // Target identity
  "exp": 1738400000,
  "iat": 1738396800,
  "scope": "resource_id"       // Scope (resource identifier)
}

Access Token Lifecycle

1. Token Request

Client requests an access token from their node.

2. Token Generation

The AuthAdapter creates a JWT with appropriate claims.

3. Token Validation

Every API request validates the token before processing.

4. Token Expiration

Tokens expire and must be refreshed.

Requesting an Access Token

When a user wants to access a resource, they follow this process:

  1. The user’s node requests an access token.
  2. If the resource is local, the node issues the token directly.
  3. If the resource is remote, the node authenticates with the remote node and requests a token on behalf of the user.
  4. The access token is returned to the user, allowing them to interact with the resource directly on its home node.

Security & Trust Model

  • Access tokens are cryptographically signed to prevent tampering.
  • Tokens have expiration times and scopes to limit misuse.
  • Nodes validate access tokens before granting access to a resource.

Example 1: Request access to own resource

sequenceDiagram
    box Alice frontend
        participant Alice shell
        participant Alice app
    end
    participant Alice node
    Alice shell ->>+Alice node: Initiate access token request
    Note right of Alice node: Create access token
    Alice node ->>+Alice shell: Access token granted
    deactivate Alice node
    Alice shell ->>+Alice app: Open resource with this token
    deactivate Alice shell
    Alice app ->+Alice node: Use access token
    loop Edit resource
        Alice app --> Alice node: Edit resource
    end
    deactivate Alice app
  • Alice opens a resource using her Cloudillo Shell
  • Her shell initiates an access token request at her node
  • Her node creates an access token and sends it to her shell
  • Her shell gives the access token to the App Alice uses to open the resource
  • The App uses the access token to edit the resource

Example 2: Request access to resource of an other identity

sequenceDiagram
    box Alice frontend
        participant Alice shell
        participant Alice app
    end
    participant Alice node
    participant Bob node
    Alice shell ->>+Alice node: Initiate access token request
    Note right of Alice node: Create signed request
    Alice node ->>+Bob node: Request access token
    Note right of Bob node: Verify signed request
    Note right of Bob node: Create access token
    deactivate Alice node
    Bob node ->>+Alice node: Grant access token
    deactivate Bob node
    Alice node ->>+Alice shell: Access token granted
    deactivate Alice node
    Alice shell ->>+Alice app: Open resource with this token
    deactivate Alice shell
    Alice app ->+Bob node: Use access token
    loop Edit resource
        Alice app --> Bob node: Edit resource
    end
    deactivate Alice app
    deactivate Bob node
  • Alice opens a resource using her Cloudillo Shell
  • Her shell initiates an access token request through her node
  • Her node creates a signed request and sends it to Bob’s node
  • Bob’s node creates an access token and sends it back to Alice’s node
  • Alice’s node sends the access token to her shell
  • Her shell gives the access token to the App Alice uses to open the resource
  • The App uses the access token to edit the resource

Token Validation Process

Authentication Middleware

Cloudillo uses Axum middleware to validate tokens on protected routes:

Handler Patterns:

Pattern 1: Required Authentication
async fn protected_handler(auth: Auth) -> Result<Response> {
    // auth.tn_id, auth.id_tag, auth.scope available
    // Access granted only if middleware validated token
}

Pattern 2: Optional Authentication
async fn public_handler(auth: Option<Auth>) -> Result<Response> {
    if let Some(auth) = auth {
        // Authenticated user - access Auth context
    } else {
        // Anonymous access - no Auth context
    }
}

The Axum extractor validates token before passing to handler.
If validation fails on required routes, request is rejected.

Validation Steps

When a request includes an Authorization: Bearer <token> header:

  1. Extract Token: Parse JWT from Authorization header
  2. Decode JWT: Parse header and claims (no verification yet)
  3. Verify Signature: Validate using AuthAdapter-stored secret
  4. Check Expiration: Ensure exp > current time
  5. Validate Claims: Check aud, scope, tid
  6. Create Auth Context: Build Auth struct for handler
pub struct AuthCtx {
    pub tn_id: TnId,                   // Tenant ID (database key)
    pub id_tag: Box<str>,              // Identity tag (e.g., "alice.example.com")
    pub roles: Box<[Box<str>]>,        // Roles (e.g., ["SADM", "USR"])
    pub scope: Option<Box<str>>,       // Optional scope (e.g., "apkg:publish")
}

Custom Extractors

Axum extractors provide typed access to authentication context:

TnId Extractor:

  • struct TnId(pub u32) - Wraps internal tenant ID
  • Usage: handler(TnId(tn_id): TnId) extracts from Auth context

IdTag Extractor:

  • struct IdTag(pub String) - Wraps user identity domain
  • Usage: handler(IdTag(id_tag): IdTag) extracts from Auth context

Auth Extractor (Full Context):

  • tn_id: Internal tenant identifier (TnId(u32))
  • id_tag: User identity (e.g., “alice.example.com”)
  • roles: Assigned roles (e.g., [“SADM”, “USR”])
  • scope: Optional scope string (e.g., “apkg:publish”)

Usage: Check auth.roles for role-based access, auth.scope for scoped API key permissions

Permission System

Cloudillo uses ABAC (Attribute-Based Access Control) for comprehensive permission management. Access tokens work in conjunction with ABAC policies to determine what actions users can perform.

Learn more: ABAC Permission System

Scope-Based Permissions

Access tokens include a scope claim that specifies permissions.

Resource-Level Permissions

Permissions are checked at multiple levels:

  1. File-Level: Who can access a file
  2. Database-Level: Who can access a database (RTDB)
  3. Action-Level: Who can see an action token
  4. API-Level: Rate limiting, quota enforcement

Permission checks combine token scope, resource ownership, and sharing permissions. See ABAC Permission System for the full evaluation flow.

Cross-Instance Authentication

ProxyToken Flow

When Alice (on instance A) wants to access Bob’s resource (on instance B):

  1. Alice’s client requests access from instance A
  2. Instance A creates a ProxyToken signed with its profile key
  3. Instance A sends ProxyToken to instance B: POST /api/auth/proxy
  4. Instance B validates ProxyToken:
    • Fetches instance A’s public key
    • Verifies signature
    • Checks expiration
  5. Instance B creates AccessToken for Alice
  6. Instance B returns AccessToken to instance A
  7. Instance A returns AccessToken to Alice’s client
  8. Alice’s client uses AccessToken to access Bob’s resource directly on instance B

ProxyToken Verification

Algorithm: Verify ProxyToken

Input: JWT token string, requester_id_tag
Output: Result<ProxyTokenClaims>

1. Decode JWT without verification (read claims)

2. Fetch requester's profile:
   - GET /api/me from requester's instance
   - Extract public keys from profile

3. Find signing key:
   - Look up key by key_id (kid) in claims
   - If not found: Return KeyNotFound error

4. Verify signature:
   - Use requester's public key to verify JWT signature

5. Check expiration:
   - If exp < current_time: Return TokenExpired

6. Return verified claims

Token Lifecycle

Access tokens are JWTs signed with the instance’s private key. They can be refreshed before expiration via POST /api/auth/refresh.

API Reference

GET /api/auth/access-token

Request an access token.

Request:

GET /api/auth/access-token
Content-Type: application/json

{
  "resource_id": "f1~abc123...",
  "scope": "read write",
  "duration": 3600
}

Response (200 OK):

{
  "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "token_type": "Bearer",
  "expires_in": 3600,
  "scope": "read write"
}

POST /api/auth/refresh

Refresh an expiring access token.

Request:

POST /api/auth/refresh
Authorization: Bearer <current_token>

Response (200 OK):

{
  "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "expires_in": 3600
}

POST /api/auth/proxy-token

Request an access token on behalf of a user (cross-instance).

Request:

POST /api/auth/proxy-token
Content-Type: application/json
Authorization: Bearer <proxy_token>

{
  "user_id_tag": "alice.example.com",
  "resource_id": "f1~xyz789...",
  "scope": "read"
}

Response (200 OK):

{
  "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "expires_in": 3600
}

See Also

Permission System

Cloudillo’s permission system has two pillars that work together to control access to all resources:

  1. ABAC Policies (profile/community-level) — Configurable TOP and BOTTOM policy rules that define hard constraints and guarantees
  2. Discretionary Access Control — The content creator’s own choices: visibility levels, explicit audience, file shares, and access grants

These two pillars combine in a layered evaluation:

1. TOP POLICY (ABAC)   → Hard constraints — what is NEVER allowed
       ↓
2. BOTTOM POLICY (ABAC) → Hard guarantees — what is ALWAYS allowed
       ↓
3. DISCRETIONARY ACCESS  → Creator's choices: visibility, shares, ownership
       ↓
4. DEFAULT DENY          → If nothing matched, deny access

Pillar 1: ABAC Policies

Attribute-Based Access Control evaluates rules based on attributes of users, resources, and context. In Cloudillo, ABAC is used for profile-level (community/company) policies that set boundaries around the discretionary access decisions.

The Four-Object Model

ABAC decisions involve four types of objects:

1. Subject (Who)

The user or entity requesting access.

pub struct AuthCtx {
    pub tn_id: TnId,                   // Tenant ID (database key)
    pub id_tag: Box<str>,              // Identity (e.g., "alice.example.com")
    pub roles: Box<[Box<str>]>,        // Roles (e.g., ["USR", "moderator"])
    pub scope: Option<Box<str>>,       // Optional scope (e.g., "apkg:publish")
}

2. Action (What)

The operation being attempted, in resource:operation format:

file:read        → Read a file
file:write       → Modify a file (CRDT/RTDB files only)
file:delete      → Delete a file
action:read      → View an action token
action:create    → Create an action token
profile:update   → Update a profile
profile:admin    → Administrative access to profile
Action tokens are immutable

Action tokens are cryptographically signed JWTs. Once created, they cannot be modified — only viewed, accepted/rejected (status change), or revoked. There is no action:write operation.

3. Object (Resource)

The resource being accessed. Must implement the AttrSet trait:

pub trait AttrSet: Send + Sync {
    fn get(&self, key: &str) -> Option<&str>;
    fn get_list(&self, key: &str) -> Option<Vec<&str>>;
    fn contains(&self, key: &str, value: &str) -> bool;
}

4. Environment (Context)

Contextual factors for the request:

pub struct Environment {
    pub time: Timestamp,    // Current Unix timestamp
}

TOP Policy (Constraints)

Defines maximum permissions — what is never allowed, regardless of discretionary settings. Evaluated first; if a rule matches with Deny effect, access is immediately denied.

Use case: Community or company-wide restrictions.

Examples:

TopPolicy:
    Rule 1:
        Condition: visibility == "public" AND size > 100MB
        Effect: DENY
        # Community rule: files larger than 100MB cannot be shared publicly

    Rule 2:
        Condition: subject.banned == true
        Effect: DENY
        # Banned users cannot access any resource

BOTTOM Policy (Guarantees)

Defines minimum permissions — what is always allowed, regardless of other rules. Evaluated second; if a rule matches with Allow effect, access is immediately granted.

Use case: Platform guarantees and special role privileges.

Examples:

BottomPolicy:
    Rule 1:
        Condition: subject.id_tag == resource.owner
        Effect: ALLOW
        # Owner can always access their own resources

    Rule 2:
        Condition: subject.HasRole("leader")
        Effect: ALLOW
        # Community leaders have full access

Policy Operators

ABAC supports these operators for building policy rules:

Comparison: Equals, NotEquals, GreaterThan, LessThan

Set: Contains, NotContains, In

Role: HasRole — checks if subject has a specific role

Logical: And, Or — combine conditions


Pillar 2: Discretionary Access Control

Between the TOP and BOTTOM policies, discretionary access control determines access based on the content creator’s own choices. This is the primary day-to-day access mechanism.

Ownership

The simplest check: owners always have full access to their own resources.

For communities, the tenant profile is treated as equivalent to the owner — community administrators (with the “leader” role) have the same access as the resource owner.

if resource.owner == subject.id_tag → ALLOW
if subject.id_tag == tenant_id_tag → ALLOW (tenant = owner equivalent)

Visibility Levels

Content creators set visibility when creating resources. This is a discretionary choice stored as a single character in the database (or NULL for direct).

Hierarchy (most to least permissive):

Code Level Who can access
P Public Anyone, including unauthenticated users
V Verified Any authenticated user from any federated instance
2 SecondDegree Friend of friend (reserved for voucher token system)
F Follower Authenticated users who follow the owner
C Connected Authenticated users with mutual connection
NULL Direct Only owner + explicit audience

The system computes the subject’s access level based on their relationship with the resource owner, then checks if it meets the visibility requirement:

Subject Access Level Description
Owner Is the resource owner (highest)
Connected Has mutual CONN with owner
Follower Has FLLW to owner
SecondDegree Friend of friend (future)
Verified Authenticated user
Public Unauthenticated (lowest)

These levels form an ordered enum (using Rust’s PartialOrd derive), where higher levels grant access to all visibility settings that lower levels can access.

Access check: subject_access_level.can_access(resource_visibility)

For Direct visibility, the system also checks explicit audience membership — if the subject’s identity is listed in the resource’s audience field, access is granted.

Explicit Access Grants

Beyond visibility, access can be granted explicitly through several mechanisms:

File Shares (FSHR Actions)

When a user shares a file with another user, the system creates:

  1. A share entry in the database (linking file to recipient with R/W permission)
  2. An FSHR action token for federation (so the recipient’s instance knows about the share)

The recipient gets Confirmation status and must accept the share. Once accepted, the file appears in their file list with the granted access level.

Share Flow:
    Alice shares file with Bob (Write access)
        ↓
    Create share_entry: file_id → bob, permission='W'
        ↓
    Create FSHR action: audience=bob, subject=file_id, subType="WRITE"
        ↓
    Bob accepts → file appears in Bob's file listing
        ↓
    Bob can now read AND write the file

Scoped Access Tokens

Access tokens can include a scope field that limits access to specific resources:

  • Format: file:{file_id}:{R|W}
  • Grants access only to the specified file with the specified permission level
  • Child files within a document tree inherit the parent’s scope

Role-Based File Access (Communities)

For files owned by a community (tenant), user roles determine access:

  • leader, moderator, contributor → Write access
  • Any community role → Read access

This only applies to files owned by the community profile, not files owned by individual users.

Discretionary Evaluation Order

When neither TOP nor BOTTOM policy matches, the discretionary layer evaluates in this order:

1. Leader role override → ALLOW (leaders can do everything)

2. For write/update/delete operations:
   a. Check ownership → ALLOW
   b. Check explicit access_level = "write" → ALLOW
   c. Otherwise → DENY

3. For read operations:
   a. Check explicit access grants (shares, scoped tokens) → ALLOW
   b. Check visibility against subject's access level → ALLOW/DENY
   c. For Direct visibility, also check audience membership → ALLOW

4. For create operations:
   a. Check collection policies (quota, tier) → ALLOW/DENY

5. Default → DENY

Complete Evaluation Flow

When a permission check is requested, the full flow is:

1. Load Subject, Object, Environment
   ↓
2. TOP POLICY check
   ├─ If Deny → return DENY (hard constraint)
   └─ No match → continue
   ↓
3. BOTTOM POLICY check
   ├─ If Allow → return ALLOW (hard guarantee)
   └─ No match → continue
   ↓
4. DISCRETIONARY ACCESS check
   ├─ Ownership → ALLOW
   ├─ Explicit grants (shares, scoped tokens) → ALLOW
   ├─ Visibility check → ALLOW/DENY
   └─ Audience membership (for Direct) → ALLOW
   ↓
5. DEFAULT DENY

Evaluation Example

Request: Bob wants to read Alice’s connected-only file

Subject:
    id_tag: "bob.example.com"
    roles: ["USR"]

Action: "file:read"

Object:
    owner: "alice.example.com"
    visibility: 'C' (Connected)
    file_id: "f1~abc123"

Evaluation:
1. TOP Policy: No blocking rules → continue
2. BOTTOM Policy: Not owner → continue
3. Discretionary:
   a. Is owner? No (alice ≠ bob)
   b. Explicit grants? No shares found
   c. Visibility = Connected
   d. Check connection:
      - Alice has CONN to Bob? Yes
      - Bob has CONN to Alice? Yes
      - Subject access level = Connected
      - Connected.can_access(Connected) = true
   → ALLOW

Integration with Routes

Cloudillo uses permission middleware to enforce access control on HTTP routes:

Protected Routes:
    # Actions (immutable — read and create only)
    GET  /api/actions           + check_perm_action("read")
    POST /api/actions           + check_perm_create("action", "create")

    # Files
    GET  /api/files/:id         + check_perm_file("read")
    PATCH /api/files/:id        + check_perm_file("write")
    DEL  /api/files/:id         + check_perm_file("delete")

    # Profiles
    PATCH /api/me               + check_perm_profile("update")
    PATCH /api/admin/profiles/:id + require_admin

Each middleware loads the resource, computes the subject’s relationship to the owner, and runs the full evaluation flow before the handler executes.


Examples

Example 1: Public File Access

Alice uploads a public file:

POST /api/files/default/logo.png
Authorization: Bearer <alice_token>
Body: <image data>

Response:
    fileId: "f1~abc123"
    visibility: "P"

Bob reads the file (no authentication needed):

GET /api/files/f1~abc123

Permission Check:
    Subject: unauthenticated (Public access level)
    Action: file:read
    Object: { owner: alice, visibility: P }
    Visibility check: Public.can_access(Public) = true
    Decision: ALLOW

Example 2: Connected-Only File

Alice uploads a connected-only file:

POST /api/files/default/private-notes.pdf
Authorization: Bearer <alice_token>

Response:
    fileId: "f1~xyz789"
    visibility: "C"

Bob tries to read (not connected):

Permission Check:
    Subject: bob (Verified access level — no connection)
    Object: { owner: alice, visibility: C }
    Visibility check: Verified.can_access(Connected) = false
    Decision: DENY

Charlie reads (connected to Alice):

Permission Check:
    Subject: charlie (Connected access level)
    Object: { owner: alice, visibility: C }
    Visibility check: Connected.can_access(Connected) = true
    Decision: ALLOW

Example 3: File Share Override

Alice has a Connected-only file, but wants to share it with Dave (who is not connected):

POST /api/files/f1~xyz789/shares
Authorization: Bearer <alice_token>
Body: { "idTag": "dave.example.com", "permission": "R" }

Dave can now read the file despite not being connected:

Permission Check:
    Subject: dave
    Action: file:read
    Object: { owner: alice, visibility: C }
    1. TOP Policy: No blocking rules → continue
    2. BOTTOM Policy: No match → continue
    3. Discretionary:
       a. Ownership? No
       b. Explicit grants? YES — share entry found (permission=R)
       → ALLOW (share overrides visibility restriction)

Attribute Set Implementations

Cloudillo implements the AttrSet trait for different resource types, providing consistent attribute access for permission evaluation.

File Attributes

file_id          → Content-addressed ID
owner_id_tag     → File owner identity
visibility       → P/V/2/F/C or NULL
access_level     → Pre-computed from shares/scopes ("read" or "write")
following        → Subject follows owner (bool)
connected        → Subject connected to owner (bool)

Action Attributes

issuer_id_tag    → Action creator identity
tenant_id_tag    → Storage location (may differ from issuer)
audience_tag     → Target recipient(s)
visibility       → P/V/2/F/C or NULL
following        → Subject follows issuer (bool)
connected        → Subject connected to issuer (bool)
status           → A/C/N/D/R/S

Action status codes:

Code Status Description
A Active Published and in good standing
C Confirmation Awaits user decision (e.g., CONN request, FSHR share)
N Notification Auto-processed, informational (e.g., mutual CONN, REACT)
D Deleted Rejected or revoked
R Draft Not yet published, editable
S Scheduled Draft with scheduled publish time

Profile Attributes

id_tag           → Profile identity
profile_type     → "community" or empty
tenant_tag       → Owner/issuer
roles            → Subject's roles on this profile
status           → Profile status
following        → Subject follows profile (bool)
connected        → Subject connected to profile (bool)

Subject Attributes (for CREATE operations)

id_tag           → Requesting user identity
roles            → User roles
tier             → "free", "standard", "premium"
quota_remaining  → Remaining storage quota
banned           → Whether user is banned
email_verified   → Whether email is verified

Security Best Practices

Default Deny

The system defaults to denying access unless explicitly allowed. Unknown visibility values are parsed as Direct (most restrictive). Unknown access levels default to None.

Validate Server-Side

Client-side visibility checks are for UX only (show/hide UI elements). The server always validates permissions before serving resources, regardless of client-side checks.

Audit Permission Denials

All permission denials are logged with debug-level tracing, including subject identity, action attempted, visibility level, access level, and relationship status.

Immutable Actions

Action tokens are cryptographically signed and content-addressed. They cannot be modified after creation. Visibility and audience are set at creation time and cannot be changed.


See Also

File Processing Pipeline

Overview

Cloudillo processes uploaded files through an asynchronous pipeline that generates multiple variants optimized for different use cases. The system uses FFmpeg for multimedia processing, resvg for SVG rasterization, and poppler-utils for PDF handling. Supported media types include images, SVG, videos, audio, PDFs, and raw files.

Processing Architecture

Upload (POST /api/files/{preset}/{file_name})
    ↓
Detect MIME type → Map to VariantClass
    ↓
Validate against preset's allowed_media_classes
    ↓
Route to type-specific handler:
    ├─ Image: Read into memory → thumbnail sync → schedule ImageResizerTask per variant
    ├─ SVG: Sanitize → store as vis.sd → rasterize thumbnail sync
    ├─ Video: Stream to temp → FFprobe → extract frame → thumbnail sync
    │         → schedule VideoTranscoderTask + optional AudioExtractorTask
    ├─ Audio: Stream to temp → FFprobe → schedule AudioExtractorTask per tier
    ├─ PDF: Read into memory → store original → schedule PdfProcessorTask
    └─ Raw: Stream to temp → store as-is (orig variant only)
    ↓
Schedule FileIdGeneratorTask (depends on all variant tasks)
    ↓
Create file descriptor → Content-address all variants
    ↓
Return file ID (f1~...)

The upload handler runs directly (not as a scheduled task). Thumbnails are generated synchronously so clients receive an immediate preview. Additional variants are generated asynchronously via the task scheduler.

Supported File Types

Images

Format Extensions Processing
JPEG .jpg, .jpeg Resize, format conversion
PNG .png Resize, format conversion
GIF .gif First frame extraction, resize
WebP .webp Resize
AVIF .avif Resize
SVG .svg Sanitization, rasterized thumbnail
Image Format Configuration

The vis.pf (profile) variant always uses AVIF. For all other variants, the format is configurable: file.thumbnail_format (default: WebP) controls vis.tn, and file.image_format (default: WebP) controls vis.sd through vis.xd.

SVG Security

SVG files are sanitized before storage: <script>, <foreignObject>, and animation elements are removed, on* event handlers are stripped, and javascript:/data:text/html/vbscript: URLs are blocked. The sanitized SVG is stored as vis.sd (vector format scales infinitely) and rasterized via resvg for the thumbnail variant.

Video

Format Extensions Processing
MP4 .mp4 H.264 transcode, thumbnails
WebM .webm H.264 transcode, thumbnails
MOV .mov H.264 transcode, thumbnails
MKV .mkv H.264 transcode, thumbnails
AVI .avi H.264 transcode, thumbnails

Audio

Format Extensions Processing
MP3 .mp3 OPUS conversion
WAV .wav OPUS conversion
OGG .ogg OPUS conversion
FLAC .flac OPUS conversion
AAC .aac OPUS conversion
WebM Audio .weba OPUS conversion

Documents

Format Extensions Processing
PDF .pdf Page count extraction, first-page thumbnail

Raw Files

Any file type not listed above can be uploaded using presets that allow the Raw variant class (e.g., archive, orig-only). Raw files are stored as-is with no processing beyond content-addressing.

Variant System

Cloudillo uses a two-level variant system with format <class>.<quality>:

Variant Classes

Class Code Description Source Types
Visual vis Static images JPEG, PNG, WebP, AVIF, GIF, SVG
Video vid Video content MP4, WebM, MKV, AVI, MOV
Audio aud Audio tracks MP3, WAV, OGG, FLAC, AAC, OPUS
Document doc Documents PDF
Raw raw Original file Any (unprocessed)

Quality Levels

Quality Code Max Size / Bitrate Use Case
Profile pf 80px (always AVIF) Profile pictures
Thumbnail tn 256px Small previews
Standard sd 720px / 1.5 Mbps / 64 kbps Mobile/low bandwidth
Medium md 1280px / 3 Mbps / 128 kbps Desktop viewing
High hd 1920px / 5 Mbps / 256 kbps High quality
Extra xd 3840px / 15 Mbps 4K/maximum quality
Original orig Unprocessed Source file

Variant Fallback

When a requested variant isn’t available, the system falls back to lower quality:

Request: vis.hd
Fallback chain: vis.md → vis.sd → vis.tn

File Descriptor Format

File descriptors encode all variant information:

d2,vis.tn:b1~abc123:f=webp:s=4096:r=256x192;vis.sd:b1~def456:f=webp:s=32768:r=720x540;vid.hd:b1~xyz789:f=mp4:s=5242880:r=1920x1080:dur=120.5:br=5000
Component Description
d2, Descriptor version prefix
; Variant separator
vis.tn, vid.hd Two-level variant code
b1~... Blob ID (SHA-256 hash)
f= Format (avif, webp, mp4, opus)
s= Size in bytes
r= Resolution (WxH)
dur= Duration in seconds (video/audio)
br= Bitrate in kbps (video/audio)
pg= Page count (PDFs)

Processing Presets

Presets define which variants to generate for different use cases:

Preset Visual Video Audio Use Case
default vis.tn, vis.sd, vis.md, vis.hd vid.sd, vid.md, vid.hd aud.md General uploads
profile-picture vis.pf, vis.tn, vis.sd, vis.md, vis.hd - - Profile images
cover vis.tn, vis.sd, vis.md, vis.hd - - Cover/banner images
high_quality vis.tn, vis.sd, vis.md, vis.hd, vis.xd vid.sd, vid.md, vid.hd, vid.xd aud.md, aud.hd Maximum quality
mobile vis.tn, vis.sd, vis.md vid.sd, vid.md aud.sd Optimized for mobile
archive vis.tn only - - Minimal (keeps original)
podcast vis.tn vid.sd aud.sd, aud.md, aud.hd Audio-focused
video vis.tn, vis.sd, vis.md, vis.hd vid.sd, vid.md, vid.hd - Video-focused
orig-only - - - Store original only, no processing
thumbnail-only - - - Generate thumbnail only, discard original
apkg vis.pf (icon extraction) - - App packages (zip)

Presets that set store_original: true (default, high_quality, archive, podcast, video, orig-only, apkg) preserve the original file as orig. Profile-picture, cover, mobile, and thumbnail-only do not store the original.

The archive and orig-only presets also accept raw (unrecognized) file types. Other presets reject uploads with unsupported MIME types.

FFmpeg Integration

Video Transcoding

ffmpeg -i input.mov \
  -c:v libx264 -preset medium -crf 23 \
  -vf "scale=1280:720:force_original_aspect_ratio=decrease" \
  output.mp4

Audio Transcoding

ffmpeg -i input.mp3 \
  -c:a libopus -b:a 128k \
  output.opus

Thumbnail extraction seeks to 10% of video duration (min 3s) and extracts a single frame, which is then resized through the image processing pipeline.

Content-Addressing

All variants are content-addressed:

  1. Blob level: Raw bytes → b1~{SHA256(bytes)}
  2. Descriptor level: Descriptor string → f1~{SHA256(descriptor)}

This enables deduplication (identical files share blobs), integrity verification, and permanent caching of immutable content.

Task Scheduling

File processing uses the task scheduler for asynchronous variant generation:

Task Type Description
image.resize Resize image to target variant dimensions and format
video.transcode Transcode video to target resolution and bitrate
audio.extract Extract/transcode audio to OPUS at target bitrate
pdf.process Extract page count (pdfinfo) and render first-page thumbnail (pdftoppm)
file.id_gen Generate file descriptor after all variant tasks complete

Dependencies ensure actions only reference fully processed files.

Federation Sync

When syncing files across instances, only file descriptors are synced initially. Variants are fetched on demand from the origin server and cached locally.

Error Handling

Error Action
Unknown format, preset allows Raw Store as-is via raw handler
Unknown format, preset disallows Raw Reject with “unsupported media type” error
FFmpeg failure Log, mark task as failed, allow retry
Storage full Queue for retry, alert admin
Timeout Retry with extended timeout

See Also

Actions & Federation

Cloudillo’s event-driven action system and federation architecture. These documents explain how users perform actions (posting, following, connecting), how actions are distributed across the network, and how independent instances communicate.

Core Subsystems

Action Tokens

Cryptographically signed events representing user activities and interactions. Actions enable event-driven communication between nodes in a federated network.

Federation

Cloudillo’s federated architecture enables independent instances to communicate, share content, and enable collaboration while maintaining user sovereignty and privacy.

Subsections of Actions & Federation

Actions & Action Tokens

An Action Token represents a user action within Cloudillo. Examples of actions include creating a post, adding a comment, leaving a like, or performing other interactions.

Why Action Tokens?

Traditional social platforms store your posts, likes, and comments in their private databases. If the platform disappears, so does your content. Cloudillo takes a different approach: your actions are portable, verifiable, and truly yours.

Think of action tokens like signed letters:

  • Anyone can verify who wrote them (cryptographic signature)
  • They can be delivered to any server (federation)
  • They can’t be tampered with without detection (content-addressing)
  • They belong to you, not to any platform (decentralization)

Real-world example: When Alice posts a photo, her server creates a signed action token. This token can be delivered to Bob’s server (federation), verified as authentic (no trust required), and displayed in Bob’s feed. If Alice’s server goes offline, Bob still has a cryptographic proof that Alice created that post.

Key benefits:

  • Portable identity: Your actions follow your identity, not a server
  • Trustless verification: Anyone can verify authenticity without trusting intermediaries
  • Censorship-resistant: No single entity controls your content
  • Offline-capable: Actions can be verified without network access

Each Action Token is:

  • Cryptographically signed by it’s creator.
  • Time-stamped with an issue time.
  • Structured with relevant metadata about the action.

Action tokens are implemented as JSON web tokens (JWTs).

Action Token Fields

Standard JWT Claims

Field Type Required Description
iss identity * The identity of the creator of the Action Token.
aud identity The audience of the Action Token (recipient for directed actions).
sub identity The subject of the Action Token (references content/user WITHOUT creating hierarchy).
iat timestamp * The time when the Action Token was issued.
exp timestamp The time when the Action Token will expire.

Cloudillo-Specific Claims

Field Type Required Description
k string * The ID of the key the identity used to sign the Token.
t string * The type of the Action Token (e.g., POST, CMNT, FLLW).
c string / object The content of the Action Token (specific to the token type).
p string The ID of the parent token (creates hierarchical threading - CMNT only).
a string[] The IDs of the attachments (file references).
vis char Visibility level: P=Public, V=Verified, F=Follower, C=Connected, null=Direct.
f string Capability flags: R/r (reactions enabled/disabled), C/c (comments), O/o (open).
_ string Nonce for proof-of-work (used in CONN actions for rate limiting).

Field Semantics

Parent (p) vs Subject (sub):

  • p (parent): Creates TRUE hierarchy (threading). Only used for CMNT tokens to form comment chains.
  • sub (subject): References content WITHOUT creating hierarchy. Used for reactions, follows, connections.

Visibility (vis):

  • P - Public: Anyone can view
  • V - Verified: Only authenticated users
  • F - Follower: Only user’s followers
  • C - Connected: Only mutual connections
  • null - Direct: Only owner + explicit audience

Flags (f):

  • Uppercase = enabled, lowercase = disabled
  • R/r - Reactions allowed
  • C/c - Comments allowed
  • O/o - Open (anyone can interact)

Action Status Codes

Each action has a lifecycle status that determines how it appears in the UI and whether it requires user interaction:

Code Name Description
A Active Action is active, accepted, or approved. Normal operational state.
C Confirmation Awaiting user confirmation (e.g., connection requests, invitations). Shows in notifications.
N Notification Informational only, auto-processed. No user action required.
D Deleted Action has been deleted or rejected. Excluded from most queries.
P Pending (Files only) Awaiting processing before becoming active.

Status Transitions

         ┌──────────────────────────────────────────────┐
         │                                              │
         ▼                                              │
     ┌───────┐     user accepts      ┌───────┐          │
 ───►│   C   │ ─────────────────────►│   A   │          │
     │ Conf  │                       │Active │          │
     └───┬───┘                       └───────┘          │
         │                               │              │
         │ user rejects                  │ user deletes │
         │                               │              │
         ▼                               ▼              │
     ┌───────┐                       ┌───────┐          │
     │   D   │◄──────────────────────│   D   │──────────┘
     │Delete │                       │Delete │
     └───────┘                       └───────┘

Examples:

  • CONN request: Arrives as C → user accepts → becomes A (mutual connection)
  • INVT to conversation: Arrives as C → user accepts → creates SUBS with status A
  • Mutual CONN: When both users have sent CONN → auto-accepted → status N
  • DELETE subtype: Changes target action status to D

Merkle Tree Structure

Cloudillo’s action system implements a merkle tree structure where every action, file, and attachment is content-addressed using SHA-256 hashing. This creates cryptographic proof of authenticity and immutability through a six-level hierarchy:

  1. Blob Data → hashed to create Variant IDs (b1~...)
  2. File Descriptor → hashed to create File IDs (f1~...)
  3. Action Token JWT → hashed to create Action IDs (a1~...)
  4. Parent References → create immutable chains between actions
  5. Attachment References → bind files to actions cryptographically
  6. Complete DAG → forms a verifiable directed acyclic graph

Each level is tamper-evident: modifying any content changes all parent hashes, making tampering immediately detectable.

See Content-Addressing & Merkle Trees for complete details on how this creates proof of authenticity for all resources.

Attachment and Token IDs

All identifiers use SHA-256 content-addressing:

  • Action IDs (a1~): Hash of entire JWT token → a1~8kR3mN9pQ2vL6xW...
  • File IDs (f1~): Hash of descriptor string → f1~Qo2E3G8TJZ2HTGh...
  • Variant IDs (b1~): Hash of raw blob bytes → b1~abc123def456ghi...

Verification Chain

This creates a verifiable chain of hashes:

Action (a1~8kR...)
  ├─ Signed by user (ES384)
  ├─ Content-addressed (SHA256 of JWT)
  └─ Attachments: [f1~Qo2...]
       └─ File (f1~Qo2...)
            ├─ Content-addressed (SHA256 of descriptor)
            └─ Descriptor: "d2,vis.tn:b1~abc...;vis.sd:b1~def..."
                 ├─ Variant vis.tn (b1~abc...)
                 │   └─ Content-addressed (SHA256 of blob)
                 ├─ Variant vis.sd (b1~def...)
                 │   └─ Content-addressed (SHA256 of blob)
                 └─ Variant vis.md (b1~ghi...)
                     └─ Content-addressed (SHA256 of blob)

Properties:

  • Immutable: Content cannot change without changing all IDs
  • Verifiable: Anyone can recompute hashes to verify integrity
  • Deduplicate: Identical content produces identical IDs
  • Tamper-evident: Any modification breaks the hash chain

Overriding Action Tokens

  • Each token type is linked to a database key, allowing previous tokens to be overridden where applicable.
  • The database key always contains the “iss” (issuer) field and may include other relevant fields.
  • Example: A REACT token (representing a reaction to a post) uses a key composed of “iss” and “p” (parent post ID). If a user reacts to the same post multiple times, the latest reaction replaces the previous one.

Root ID Handling

Important: The root_id field is NOT included in the action token JWT.

  • root_id is stored in the database for query optimization
  • It is a computed field, derived by traversing the parent chain to find the root action
  • It is NOT cryptographically signed (not in the JWT payload)
  • Recipients must compute root_id by following parent references

Why Root ID is Computed

  • Smaller JWT payload: Keeps tokens compact and efficient
  • Avoids redundancy: Root ID can be derived from the parent chain
  • Maintains flexibility: Thread structure can be recomputed if needed

Action Creation Pipeline

Client
  ↓ POST /api/actions
Server creates ActionCreatorTask
  ↓
Scheduler checks dependencies
  ↓
Wait for FileIdGeneratorTask (if attachments)
  ↓
ActionCreatorTask runs
  ├─ Build JWT claims
  ├─ Fetch private key from AuthAdapter
  ├─ Sign JWT (ES384)
  ├─ Compute action ID (SHA256)
  └─ Store in MetaAdapter
  ↓
Response to client

Action Verification Pipeline

Remote Instance
  ↓ POST /api/inbox
Create ActionVerifierTask
  ↓
Decode JWT (unverified)
  ↓
Fetch issuer's public keys
  ↓ GET https://cl-o.{issuer}/api/me
Verify JWT signature (ES384)
  ↓
Check expiration
  ↓
Verify permissions
  ├─ Following/Connected status
  ├─ Audience matches
  └─ Parent ownership (for replies)
  ↓
Sync attachments (if any)
  ↓ GET https://cl-o.{issuer}/api/files/{id}
Store in MetaAdapter
  ↓
Trigger hooks (notifications, etc.)

Detailed Processing Pipelines

The following sections describe the complete processing pipelines implemented in the codebase.

Inbound Pipeline (12 Steps)

When receiving a federated action at /api/inbox:

Step Operation Description
1 Decode Parse JWT token without verification
2 PoW Check Verify proof-of-work nonce (for CONN actions)
3 Signature Verify Fetch public key, verify ES384 signature
4 DSL Validation Validate against action type definition schema
5 Permission Check Verify sender has permission (following/connected)
6 Subscription Check For subscribable actions, verify subscription exists
7 Store Action Persist to MetaAdapter with computed action ID
8 Execute Hooks Run on_receive hooks from DSL definition
9 WebSocket Forward Broadcast to connected tenant clients
10 Fan-out Create delivery tasks for related actions
11 Related Actions Process APRV fan-out to followers
12 ACK Response Generate acknowledgment token if required

Outbound Pipeline (8 Steps)

When a user creates an action via /api/actions:

Step Operation Description
1 Validate Check request parameters against DSL schema
2 Serialize Build JWT claims (iss, iat, t, c, p, a, etc.)
3 Generate Sign JWT with user’s private key (ES384)
4 Compute ID Calculate action ID as SHA256(token)
5 Store Persist to MetaAdapter
6 Execute Hooks Run on_create hooks from DSL definition
7 WebSocket Forward Notify local connected clients
8 Delivery Tasks Schedule ActionDeliveryTask for each recipient

Action Retrieval

GET /api/actions

Retrieve actions owned by or visible to the authenticated user:

Request:

GET /api/actions?type=POST&limit=50&offset=0
Authorization: Bearer <access_token>

Response (200 OK):

{
  "actions": [
    {
      "id": "a1~xyz789...",
      "type": "POST",
      "issuer": "alice.example.com",
      "content": "Hello, Cloudillo!",
      "attachments": ["f1~abc123..."],
      "created_at": 1738483200,
      "token": "eyJhbGc..."
    }
  ],
  "total": 150,
  "limit": 50,
  "offset": 0
}

GET /api/actions/:id

Retrieve a specific action by ID:

Request:

GET /api/actions/a1~xyz789...
Authorization: Bearer <access_token>

Response (200 OK):

{
  "id": "a1~xyz789...",
  "type": "POST",
  "issuer": "alice.example.com",
  "content": "Hello, Cloudillo!",
  "attachments": ["f1~abc123..."],
  "parent": null,
  "created_at": 1738483200,
  "token": "eyJhbGc..."
}

Federation & Distribution

Outbound Distribution

Determine recipients based on action type:
  - POST: send to all followers
  - CMNT/REACT: send to parent action owner
  - CONN/FLLW: send to audience

For each recipient:
  POST https://cl-o.{recipient}/api/inbox
  Body: {"token": "eyJhbGc..."}

The /api/inbox endpoint is public (no authentication required) because the action token itself contains the cryptographic proof of authenticity.

Security Considerations

Action Token Immutability

Action tokens are content-addressed using SHA-256: action_id = SHA256(entire_jwt_token).

This includes:

  • Header (algorithm, type)
  • Payload (issuer, content, attachments, timestamps, etc.)
  • Signature (cryptographic proof of authorship)

Immutability properties:

  • Tokens cannot be modified without changing the ID
  • Duplicate actions are automatically deduplicated
  • References to actions are tamper-proof
  • Parent references create immutable chains
  • Attachment references are cryptographically bound

Merkle Tree Verification

The content-addressing system creates a merkle tree that can be verified at multiple levels:

  1. Signature Verification: Verify the action was created by the claimed author using their public key
  2. Action ID Verification: Recompute action_id and verify it matches (proves no tampering)
  3. Parent Chain Verification: Recursively verify parent actions exist and are valid
  4. Attachment Verification: Verify file descriptors and blob variants match their hashes

See Content-Addressing & Merkle Trees for complete verification examples.

Signature Verification

Every federated action undergoes cryptographic verification:

  1. Signature: Proves the issuer created the action
  2. Key Ownership: Public key fetched from issuer’s /api/me
  3. Expiration: Optional exp claim prevents token replay
  4. Audience: Optional aud claim ensures intended recipient

Spam Prevention

Multiple mechanisms prevent spam:

  1. Relationship Requirements: Only receive actions from connected/followed users
  2. Rate Limiting: Limit actions per user per time period
  3. Proof of Work: Required for CONN (connection request) actions via the _ nonce field, preventing spam connection requests
  4. Reputation: (Future) Trust scores based on user behavior

Action Token Types

User Relationships

CONN - Connect
Represents one side of a connection between two profiles. A connection is established when both parties issue a connection token to each other.
FLLW - Follow
Represents a follow relationship (a profile follows another profile).

Content

POST - Post
Represents a post created by a profile. Can include text, images, videos, or other attachments.
REPOST - Repost/Share
Represents the reposting/sharing of another user’s content to your profile.
CMNT - Comment
Represents a comment attached to another token (post, comment, etc.).
REACT - Reaction
Represents a reaction (like, emoji, etc.) to another token.

Communication

MSG - Message
Represents a direct message sent from one profile to another, or a message within a conversation.
CONV - Conversation
Represents a group conversation (group chat) between multiple participants.
SUBS - Subscription
Represents a subscription to a subscribable action (e.g., joining a conversation).
INVT - Invitation
Represents an invitation for a user to join a subscribable action.

Metadata

STAT - Statistics
Represents statistics about another token (number of reactions, comments, views, etc.).
APRV - Approval
Represents an approval of another user’s content, enabling federated fan-out to followers.
PRES - Presence
Represents ephemeral real-time presence (typing indicators, online status). NOT persisted to database.

File Sharing

FSHR - File Share
Represents sharing a file with another user. Requires acceptance. Supports subtypes: WRITE (grant write permission), DEL (revoke share).

Identity Provider

IDP:REG - Identity Registration
Represents an identity registration request to an Identity Provider instance. Enables federated identity creation for community-owned identities.

Native Hook Registry

Action types have lifecycle hooks that execute during specific events. These hooks are implemented in native Rust code for performance and security:

Action Type on_create on_receive on_accept on_reject
CONN
FLLW - -
REACT - -
CMNT - -
FSHR - -
SUBS - -
CONV - -
INVT -
APRV - - -
IDP:REG - - -

Hook Descriptions:

  • on_create: Executes when the local user creates this action type
  • on_receive: Executes when a federated action of this type arrives at /api/inbox
  • on_accept: Executes when the user accepts a confirmation action (CONN, FSHR, INVT)
  • on_reject: Executes when the user rejects a confirmation action (CONN only)
Info

POST, MSG, STAT, and PRES action types do not have native hooks - they use the default action processing pipeline.

Complete Example: LIKE → POST → Attachments → Variants

This example demonstrates the complete merkle tree structure from a LIKE action down to the individual image blob bytes.

Example Data

LIKE Action (Bob reacts to Alice's post)
├─ Action ID: a1~m9K7nP2qR8vL3xWpYzT4BjN...
├─ Type: REACT:LIKE
├─ Issuer: bob.example.com
├─ Subject: a1~8kR3mN9pQ2vL6xW... (Alice's POST)
└─ Created: 2025-01-02T10:30:00Z

POST Action (Alice's post with 3 images)
├─ Action ID: a1~8kR3mN9pQ2vL6xWpYzT4BjN...
├─ Type: POST:IMG
├─ Issuer: alice.example.com
├─ Content: "Check out these amazing photos from our trip!"
├─ Attachments:
│   ├─ f1~Qo2E3G8TJZ2HTGhVlrtTDBpvBGOp6gfGhq4QmD6Z46w (Image 1)
│   ├─ f1~7xW4Y9K5LM8Np2Qr3St6Uv8Xz9Ab1Cd2Ef3Gh4Ij5 (Image 2)
│   └─ f1~9mN1P6Q8RS2Tu3Vw4Xy5Za6Bc7De8Fg9Hi0Jk1Lm2 (Image 3)
└─ Created: 2025-01-02T09:15:00Z

Image 1 File Descriptor
├─ File ID: f1~Qo2E3G8TJZ2HTGhVlrtTDBpvBGOp6gfGhq4QmD6Z46w
├─ Descriptor: d2,vis.tn:b1~abc123...:f=avif:s=4096:r=150x150;
│              vis.sd:b1~def456...:f=avif:s=32768:r=640x480;
│              vis.md:b1~ghi789...:f=avif:s=262144:r=1920x1080
└─ Variants:
    ├─ vis.tn: b1~abc123def456ghi789... (4KB, 150×150px)
    ├─ vis.sd: b1~def456ghi789jkl012... (32KB, 640×480px)
    └─ vis.md: b1~ghi789jkl012mno345... (256KB, 1920×1080px)

Merkle Tree Visualization

flowchart TB
    subgraph "Action Layer"
        LIKE[LIKE Action<br/>a1~m9K7nP2qR8vL3xW...<br/>Type: REACT:LIKE<br/>Issuer: bob.example.com]
        POST[POST Action<br/>a1~8kR3mN9pQ2vL6xW...<br/>Type: POST:IMG<br/>Issuer: alice.example.com<br/>Content: Check out these photos!]
    end

    subgraph "Subject Reference"
        LIKE -->|subject| POST
    end

    subgraph "Attachment References"
        POST -->|attachments[0]| FILE1
        POST -->|attachments[1]| FILE2
        POST -->|attachments[2]| FILE3
    end

    subgraph "File Descriptor Layer"
        FILE1[File 1<br/>f1~Qo2E3G8TJZ2...]
        FILE2[File 2<br/>f1~7xW4Y9K5LM8...]
        FILE3[File 3<br/>f1~9mN1P6Q8RS2...]
    end

    subgraph "File 1 Variants"
        FILE1 --> V1TN[tn variant<br/>b1~abc123def456...<br/>AVIF, 4KB<br/>150×150px]
        FILE1 --> V1SD[sd variant<br/>b1~def456ghi789...<br/>AVIF, 32KB<br/>640×480px]
        FILE1 --> V1MD[md variant<br/>b1~ghi789jkl012...<br/>AVIF, 256KB<br/>1920×1080px]
    end

    subgraph "File 2 Variants"
        FILE2 --> V2TN[tn variant<br/>b1~jkl012mno345...<br/>AVIF, 4KB<br/>150×150px]
        FILE2 --> V2SD[sd variant<br/>b1~mno345pqr678...<br/>AVIF, 28KB<br/>640×480px]
        FILE2 --> V2MD[md variant<br/>b1~pqr678stu901...<br/>AVIF, 248KB<br/>1920×1080px]
    end

    subgraph "File 3 Variants"
        FILE3 --> V3TN[tn variant<br/>b1~stu901vwx234...<br/>AVIF, 4KB<br/>150×150px]
        FILE3 --> V3SD[sd variant<br/>b1~vwx234yza567...<br/>AVIF, 35KB<br/>640×480px]
        FILE3 --> V3MD[md variant<br/>b1~yza567bcd890...<br/>AVIF, 280KB<br/>1920×1080px]
        FILE3 --> V3HD[hd variant<br/>b1~bcd890efg123...<br/>AVIF, 1.2MB<br/>3840×2160px]
    end

    subgraph "Hash Computation"
        COMP1[Action ID = SHA256 of JWT token]
        COMP2[File ID = SHA256 of descriptor string]
        COMP3[Blob ID = SHA256 of blob bytes]
    end

    style LIKE fill:#ffcccc
    style POST fill:#ccffcc
    style FILE1 fill:#ccccff
    style FILE2 fill:#ccccff
    style FILE3 fill:#ccccff
    style V1TN fill:#ffffcc
    style V2TN fill:#ffffcc
    style V3TN fill:#ffffcc
    style V1SD fill:#ffeecc
    style V2SD fill:#ffeecc
    style V3SD fill:#ffeecc
    style V1MD fill:#ffddcc
    style V2MD fill:#ffddcc
    style V3MD fill:#ffddcc
    style V3HD fill:#ffcccc

Verification Steps

To verify this complete chain:

  1. Verify LIKE action signature and action_id
  2. Verify parent POST action (signature + action_id)
  3. Verify each file attachment (file_id = SHA256(descriptor))
  4. Verify all variants for each file (blob_id = SHA256(blob_data))

Complete verification example: see Content-Addressing & Merkle Trees.

TODO

Permissions

  • Public
  • Followers
  • Connections
  • Tags

Flags

  • Can react
  • Can comment
  • With permissions?

Tokens

Review
Represents a review post with a rating, attached to something
Patch
Represents a patch of a token (modify the content, or flags for example)
Resource
Represents a resource that can be booked (e.g. a conference room)
Book
Represents a booking of a resource.

See Also

Subsections of Actions & Action Tokens

User Relationships

Action tokens representing connections and relationships between users on the Cloudillo network.

Contains:

Subsections of User Relationships

Connect Token

This token represents a connection intent created by a user.

The connect token can contain a content (c) field in markdown format. The token must contain an audience (aud) field which points to the identity the user is connecting with. For other constraints see the Action Tokens.

Content-Addressing

This token is content-addressed using SHA-256:

Database Key

The database key for a connect token is [iss, t, aud]

Purpose: This key ensures that a user can only have one active connection intent to a specific identity. The key components are:

  • iss: Issuer identity (who is connecting)
  • t: Token type (“CONN”)
  • aud: Audience (who they’re connecting with)

Example:

  • Alice sends connection to Bob → Stored with key [alice.example.com, "CONN", bob.example.com]
  • Alice updates the connection message → New token with same key, previous one is marked deleted
  • Only ONE connection intent from Alice to Bob at a time

Example

User @alice.cloudillo.net wants to connect with @bob.cloudillo.net:

Field Value
iss alice.cloudillo.net
aud bob.cloudillo.net
iat 2024-04-13T00:01:10.000Z
k 20240101
t CONN
c Hi! Remember me? We met last week on the bus.

Follow Token

This token represents a follow intent created by a user.

The token must contain an audience (aud) field which points to the identity the user is following. For other constraints see the Action Tokens.

Content-Addressing

This token is content-addressed using SHA-256:

Database Key

The database key for a follow token is [iss, t, aud]

Purpose: This key ensures that a user can only have one active follow relationship to a specific identity. The key components are:

  • iss: Issuer identity (who is following)
  • t: Token type (“FLLW”)
  • aud: Audience (who they’re following)

Example:

  • Alice follows Bob → Stored with key [alice.example.com, "FLLW", bob.example.com]
  • If Alice unfollows and re-follows → New token with same key, previous one is marked deleted
  • Only ONE follow relationship from Alice to Bob at a time

Example

User @alice.cloudillo.net follows @bob.cloudillo.net:

Field Value
iss alice.cloudillo.net
aud bob.cloudillo.net
iat 2024-04-13T00:01:10.000Z
k 20240101
t FLLW

Content Actions

Action tokens representing content creation, sharing, and interaction on the Cloudillo network. These tokens enable users to post, share, comment on, and react to content.

Contains:

Subsections of Content Actions

Post Token

This token represents a post created by a user.

The post token must contain a content (c) field. For other constraints see the Action Tokens.

There can be multiple subtypes of this token, which can represent the content of the post in different ways.

POST

A simple text only post. The content must be in markdown format.

POST:IMG

A post with an image. The content must be in markdown format, the attachment (a) field must contain exactly one item which must be an image.

POST:VID

A post with a video. The content must be in markdown format, the attachment (a) field must contain exactly one item which must be a video.

Content-Addressing

This token is content-addressed using SHA-256:

  • The entire JWT token (header + payload + signature) is hashed
  • Action ID format: a1~{base64_hash}
  • Changing any field invalidates the action_id
  • See Content-Addressing & Merkle Trees for details

Immutability: Once created, a POST token cannot be modified without changing its action ID.

Attachments

The a (attachments) field can contain file references:

  • Each entry is a file_id (f1~...)
  • File IDs are content-addressed (SHA256 of file descriptor)
  • Files contain multiple variants (different resolutions)
  • See File Storage for details

Properties:

  • Attachments are cryptographically bound to the post
  • Cannot swap images without breaking the action signature
  • Deduplication: same image in multiple posts = same file_id
  • Federation: remote instances can verify attachment integrity

Database Key

The database key for a post token is [iss, "POST", id]

Purpose: The database key is used to identify and potentially invalidate previous versions of this action type. For POST tokens, the key includes the action ID itself, meaning each post is unique and not overridden by subsequent posts.

Example

User @someuser.cloudillo.net writes a post on the wall of @somegroup.cloudillo.net, attaching an image:

Field Value
iss someuser.cloudillo.net
aud somegroup.cloudillo.net
iat 2024-04-13T00:00:00.000Z
k 20240109
t POST:IMG
c “Love U All <3”
a [“ohst:51mp8Oe5gekbAualO6jydbOriq0OfuZ5zpBY-I30U00,CFN6hm21Z73m12CK2igjFy8bVDhSV8oFZS4xOrzHE98,rk9n8iz–t0ov4sJAnBzEktmyZVsLmcKkPEVhkK4688,nfpr7eTtApLNTRS5qDokBsodo4UQ_zj7kzNWwvj7oEs”]

React Token

This token represents a reaction created by a user.

The react token must not contain a content (c) field. The token must contain a subject (sub) field which points to the action being reacted to. For other constraints see the Action Tokens.

Content-Addressing

This token is content-addressed using SHA-256:

  • The entire JWT token (header + payload + signature) is hashed
  • Action ID format: a1~{base64_hash}
  • Changing any field invalidates the action_id
  • See Content-Addressing & Merkle Trees for details

Immutability: Once created, a REACT token cannot be modified without changing its action ID.

Subject Reference

The sub (subject) field references the action being reacted to:

  • Contains the target action’s action_id (a1~...)
  • Target action must exist and be verified
  • Creates a non-hierarchical reference (reactions don’t create visible threading)
  • Cannot modify subject without breaking reference

Why Subject Instead of Parent:

  • parent (p) is used for hierarchical threading (comments create visible child hierarchy)
  • subject (sub) is used for non-hierarchical references (reactions reference without creating hierarchy)
  • Reactions don’t create visible child actions in the timeline
  • This semantic distinction keeps threading clean

Properties:

  • Subject references are immutable
  • Cannot change which post you’re reacting to
  • Merkle tree ensures subject hasn’t been tampered with
  • Federation: remote instances can verify the complete reference

Database Key

The database key for a react token is {type}:{sub}:{iss}

Purpose: This key ensures that a user can only have one active reaction of each type to a specific action. The key components are:

  • {type}: Full type including subtype (e.g., “REACT:LIKE”)
  • {sub}: Subject ID (what they’re reacting to)
  • {iss}: Issuer identity (who is reacting)

Example:

  • Alice LIKEs a post → Stored with key REACT:LIKE:a1~post123:alice.example.com
  • Alice changes to LOVE → New LOVE token, previous LIKE is marked deleted
  • Only ONE reaction of each type per user per post
  • Changing reaction type creates a new action but invalidates the previous one

Audience Resolution

The aud (audience) field is automatically resolved from the subject action:

  • Set to the issuer of the subject action (the post owner)
  • Ensures the reaction notification reaches the content creator
  • Does not need to be explicitly provided when creating the reaction

Example

User @someotheruser.cloudillo.net likes a post:

Field Value
iss someotheruser.cloudillo.net
aud somegroup.cloudillo.net
iat 2024-04-13T00:01:10.000Z
k 20240301
t REACT:LIKE
sub NAado5PS4j5+abYtRpBELU0e5OQ+zGf/tuuWvUwQ6PA=

Subtypes

REACT tokens use subtypes to indicate the reaction type:

Subtype Description
REACT:LIKE Standard like reaction
REACT:LOVE Love/heart reaction
REACT:LAUGH Laughing reaction
REACT:WOW Surprised/amazed reaction
REACT:SAD Sad reaction
REACT:ANGRY Angry reaction
REACT:DEL Delete/remove the reaction

See Also

Comment Token

This token represents a comment created by a user.

The comment token must contain a content (c) field which contains the text of the comment in markdown format. The token must also contain a parent (p) field which points to the parent object the comment is referring to.

For other constraints see the Action Tokens.

Content-Addressing

This token is content-addressed using SHA-256:

Parent Reference

The p (parent) field references the parent action:

  • Contains the parent’s action_id (a1~...)
  • Creates immutable parent-child relationship
  • Parent can be a POST, another CMNT, or any commentable action
  • Federation: remote instances can verify the complete chain

Database Key

The database key for a comment token is [iss, t, id]

Purpose: Each comment is unique and not overridden by subsequent comments from the same user. The key includes the action ID itself, allowing multiple comments from the same user on the same parent.

Example

User @someotheruser.cloudillo.net writes a comment on a post:

Field Value
iss someotheruser.cloudillo.net
aud somegroup.cloudillo.net
iat 2024-04-13T00:01:00.000Z
k 20240301
t CMNT
c “I love U too!”
p NAado5PS4j5+abYtRpBELU0e5OQ+zGf/tuuWvUwQ6PA=

Repost Token

This token represents a repost (share) of another user’s content.

A repost allows a user to share someone else’s post with their own followers, optionally adding their own commentary. This is similar to “retweeting” in Twitter or “sharing” in other social platforms.

The token must contain a parent (p) field pointing to the original post being reposted. For other constraints see the Action Tokens.

Content-Addressing

This token is content-addressed using SHA-256:

Parent Reference

The p (parent) field references the original post being reposted:

  • Contains the original post’s action_id (a1~...)
  • Parent must exist and be verified
  • Creates immutable link between repost and original content
  • Federation: remote instances can verify the complete chain

Database Key

The database key for a repost token is [iss, t, p]

Purpose: This key ensures that a user can only repost the same content once. The key components are:

  • iss: Issuer identity (who is reposting)
  • t: Token type (“REPOST”)
  • p: Parent ID (what’s being reposted)

Example:

  • Alice reposts Bob’s post → Stored with key [alice.example.com, "REPOST", a1~post123]
  • Alice reposts the same post again → New token with same key, previous one is marked deleted
  • Only ONE repost of the same content per user

Types of Reposts

A repost can be a simple repost (no c field), a repost with commentary, or a quote repost (longer commentary). The token structure is identical in all cases – only the presence and length of the optional c field differs.

{
  "iss": "alice.example.com",
  "iat": 1738483200,
  "k": "20240101",
  "t": "REPOST",
  "p": "a1~xyz789...",
  "c": "This is an excellent analysis!"
}

Omit the c field for a simple repost without commentary.

Fields

Field Required Description
iss The identity reposting the content
iat Timestamp when repost was created
k Key ID used to sign the token
t Token type (always “REPOST”)
p Parent token ID (the original post being reposted)
c Optional commentary on the repost (markdown)
aud Optional audience restriction

Example

User @alice.example.com reposts @bob.example.com’s post with commentary:

Field Value
iss alice.example.com
iat 2024-04-13T00:01:10.000Z
k 20240101
t REPOST
p a1~xyz789abc…
c Great insights on distributed systems!

Visibility and Federation

Repost tokens are broadcast actions, meaning they are:

  • Sent to all followers of the reposter
  • Displayed in the reposter’s timeline/feed
  • Credit the original author
  • Link back to the original post

Federation Flow

When Alice reposts Bob’s post:

  1. Alice’s instance creates a REPOST token referencing Bob’s original POST token
  2. The REPOST is broadcast to Alice’s followers
  3. The original POST token is fetched/synchronized if not already available locally
  4. Followers see the repost in Alice’s timeline with proper attribution to Bob

Permission Checks

When creating a repost:

  1. Original post exists: Verify the parent post ID is valid
  2. Permission to view: Ensure the reposter can access the original post
  3. Repost allowed: Check if original author allows reposts (future feature)
  4. Audience restrictions: Honor any audience limitations on original post

Statistics Impact

Reposts affect the statistics of the original post:

  • Original post’s STAT token includes repost count
  • Reposts increase content visibility and reach
  • Original author can see who reposted their content

Undo/Delete Repost

To remove a repost:

  • Create a new action that overwrites the REPOST (using same database key)
  • Or delete the REPOST token entirely
  • This removes the repost from the reposter’s timeline
  • Original post’s repost count is decremented

Comparison with Other Actions

Action Purpose Visibility
REPOST Share content with followers Broadcast to your followers
REACT Express opinion privately Only visible to post author/viewers
CMNT Add threaded discussion Attached to original post

See Also

Communication

Action tokens for direct communication and group collaboration between users on the Cloudillo network, including direct messages, conversations, subscriptions, and invitations.

Contains:

Subsections of Communication

Message Token

This token represents a direct message sent from one profile to another.

A message token enables private, one-on-one communication between users. Unlike posts (which are broadcast to followers), messages are sent directly to a specific recipient and are not federated to other instances.

The token must contain an audience (aud) field pointing to the recipient of the message. For other constraints see the Action Tokens.

Content-Addressing

This token is content-addressed using SHA-256:

Attachments

The a (attachments) field can contain file references:

  • Each entry is a file_id (f1~...)
  • File IDs are content-addressed (SHA256 of file descriptor)
  • Files contain multiple variants (different resolutions)
  • See File Storage for details

Parent Reference

The optional p (parent) field enables message threading:

  • Contains the parent message’s action_id (a1~...)
  • Creates conversation threads and reply chains
  • Parent must exist and be verified

Database Key

The database key for a message token is [iss, t, id]

Purpose: Each message has a unique ID, allowing multiple messages in a conversation thread. The key includes the action ID itself, so messages are never overridden.

Message Example

{
  "iss": "alice.example.com",
  "aud": "bob.example.com",
  "iat": 1738483200,
  "k": "20240101",
  "t": "MSG",
  "c": "Hey Bob, want to grab coffee tomorrow?",
  "a": ["f1~abc123..."],
  "p": "a1~xyz789..."
}

All fields shown; in practice a (attachments) and p (parent/reply) are optional. The structure is the same regardless of whether the message is a simple text, has attachments, or is a reply – only the presence of optional fields differs.

Fields

Field Required Description
iss The identity sending the message
aud The identity receiving the message
iat Timestamp when message was sent
k Key ID used to sign the token
t Token type (always “MSG”)
c Message content (markdown)
p Parent message ID (for threading/replies)
a Attachments (files, images, etc.)
exp Optional expiration (for disappearing messages)

Visibility and Federation

Message tokens are direct actions, meaning they are:

  • Sent only to the specified audience (recipient)
  • NOT broadcast to followers
  • Private between sender and recipient
  • Delivered to POST https://cl-o.{recipient}/api/inbox

Federation Flow

When Alice sends a message to Bob:

  1. Alice’s instance creates a MSG token with aud: bob.example.com
  2. The token is signed by Alice’s private key
  3. The message is sent to https://cl-o.bob.example.com/api/inbox
  4. Bob’s instance verifies the signature
  5. Bob’s instance stores the message for Bob to read
  6. Bob receives a notification (via WebSocket bus or push notification)

Permission Checks

When receiving a message token:

  1. Verify signature: Ensure the token is signed by the claimed issuer
  2. Check audience: Verify aud field matches the local user
  3. Check relationship: Verify sender is connected or followed (configurable)
  4. Spam prevention: Check sender is not blocked
  5. Rate limiting: Enforce message rate limits per sender

Message Delivery

Messages use a different delivery mechanism than broadcast actions:

Aspect Broadcast Actions (POST) Direct Messages (MSG)
Delivery To all followers To specific recipient only
Federation Sent to multiple instances Sent to one instance
Storage Stored publicly (with permissions) Stored privately
Visibility Timeline/feed Message inbox
Retry Best-effort Reliable delivery with retry

Message Threading

Messages can be threaded using the parent (p) field:

Message 1 (no parent)
  ├─ Reply 1 (p: Message 1)
  ├─ Reply 2 (p: Message 1)
  │   └─ Reply 3 (p: Reply 2)
  └─ Reply 4 (p: Message 1)

Message Storage

Messages are stored in MetaAdapter with:

  • Sender and recipient identity tags
  • Read/unread status
  • Delivery status
  • Optional expiration timestamp

Privacy considerations:

  • Messages stored locally at sender and recipient instances
  • Not replicated to other instances
  • Can be deleted by either party
  • Retention policies configurable per user

See Also

Conversation Token

This token represents a group conversation (group chat) between multiple participants.

The conversation token must contain a content (c) field with conversation metadata. For other constraints see the Action Tokens.

Note: For 1-to-1 direct messaging, see Message Token (MSG). CONV is for group conversations with multiple participants.

Content-Addressing

This token is content-addressed using SHA-256:

Immutability: Once created, a CONV token cannot be modified without changing its action ID.

Content Structure

The content (c) field must be a JSON object with the following structure:

{
  "name": "Project Discussion",
  "description": "Weekly sync about the project"
}
Property Type Required Description
name string Yes Display name for the conversation
description string No Optional description of the conversation purpose

Flags

CONV tokens use the flags (f) field to control conversation behavior:

Flag Default Description
R/r r (disabled) Reactions allowed on the conversation itself
C/c c (disabled) Comments allowed on the conversation itself
O/o o (closed) Open (anyone can join) vs Closed (invite-only)

Default flags: rco (no reactions, no comments, closed/invite-only)

To create an open conversation that anyone can join, use flags O.

Subtypes

Subtype Description
CONV Create a new conversation
CONV:UPD Update conversation metadata (name, description, flags)
CONV:DEL Delete/archive the conversation

Subscribable Behavior

CONV is a subscribable action type, meaning:

  • Users can subscribe to receive messages in the conversation
  • Subscription is managed via SUBS tokens
  • Users can be invited via INVT tokens
  • Messages sent to the conversation (MSG with parent=CONV) are delivered to all subscribers

Auto-Subscription

When a CONV is created, the creator is automatically subscribed as an admin:

  1. CONV token is created and signed
  2. System automatically creates a SUBS token for the creator
  3. Creator’s SUBS has role = “admin”

Participant Roles

Conversation participants have roles (observer, member, moderator, admin) managed via SUBS tokens. See the Subscription Token roles table for detailed permissions.

Federation

Creating a Conversation

Creator creates CONV token
         │
         ▼
┌─────────────────────────────┐
│ ActionCreatorTask runs      │
│ - Signs CONV token          │
│ - Creates admin SUBS        │
└───────────┬─────────────────┘
            │
            ▼
    CONV ready for messages

Adding Participants

Participants join via invitation or (for open conversations) self-subscription:

Admin creates INVT for new participant
         │
         ▼
INVT delivered to invitee (includes CONV token)
         │
         ▼
Invitee creates SUBS token to accept
         │
         ▼
SUBS auto-accepted (INVT exists)
         │
         ▼
Participant now receives MSG in conversation

Message Fan-Out

When a message is sent to a conversation:

  1. Local CONV: If the CONV owner sends/receives a message, they fan it out to all subscribers
  2. Remote CONV: If a subscriber sends a message, it goes to the CONV owner who fans it out

See Subscriber Fan-Out for detailed federation flow.

Example

Field Value
iss alice.cloudillo.net
aud
iat 2024-04-13T00:00:00.000Z
k 20240109
t CONV
c {“name”: “Project Team”, “description”: “Discussion for project X”}
f rco

See Also

Subscription Token

This token represents a subscription to a subscribable action, such as a conversation (CONV). Subscriptions control who receives messages and updates from group activities.

The subscription token must contain both a subject (sub) field referencing the subscribable action and an audience (aud) field specifying the owner of that action. For other constraints see the Action Tokens.

Content-Addressing

This token is content-addressed using SHA-256:

  • The entire JWT token (header + payload + signature) is hashed
  • Action ID format: a1~{base64_hash}
  • Changing any field invalidates the action_id
  • See Content-Addressing & Merkle Trees for details

Immutability: Once created, a SUBS token cannot be modified without changing its action ID.

Purpose

SUBS tokens serve several important purposes:

  1. Membership Management: Track who is part of a conversation or group
  2. Message Delivery: Determine who receives messages sent to the group
  3. Role Assignment: Assign permissions (observer, member, moderator, admin)
  4. Participation Control: Enable joining, leaving, and role changes

Required Fields

Field Required Description
iss Yes The subscriber’s identity
aud Yes The owner of the subject action (e.g., CONV creator)
sub Yes The action_id being subscribed to (a1~...)
t Yes “SUBS”, “SUBS:UPD”, or “SUBS:DEL”
c Optional Subscription metadata (role, invitedBy, message)

Subtypes

Subtype Description
SUBS Create a new subscription (join)
SUBS:UPD Update subscription (change role)
SUBS:DEL Delete subscription (leave)

Content Structure

The optional content (c) field contains subscription metadata:

{
  "role": "member",
  "invitedBy": "alice.cloudillo.net",
  "message": "Joining the team discussion"
}
Property Type Required Description
role string No Participant role: “observer”, “member”, “moderator”, “admin”
invitedBy string No Identity of the user who invited this subscriber
message string No Optional message when joining

Database Key

The database key for a subscription token is {type}:{sub}:{iss}

This ensures only one active subscription per user per subject. Example: SUBS:a1~conv123:alice.example.com.

Auto-Accept Logic

When a SUBS token is received, the system determines whether to accept it automatically:

Subscription received
         │
         ▼
┌─────────────────────────────┐
│ Check auto-accept conditions│
└───────────┬─────────────────┘
            │
    ┌───────┴───────┬───────────────┬───────────────┐
    ▼               ▼               ▼               ▼
Subject has    INVT exists     Issuer is      Otherwise
'O' flag?      for user?       subject creator?
    │               │               │               │
    ▼               ▼               ▼               ▼
 Accept          Accept          Accept          Reject
 (Open)         (Invited)       (Creator)      (status='R')

If none of the conditions shown in the diagram are met, the subscription is rejected (status=‘R’).

Participant Roles

Each role is a superset of the one below it. Default role is member.

Role Additional permissions
observer Read only
member + write (send messages)
moderator + invite/remove members
admin + manage conversation settings

Federation Flow

Subscribing to a Conversation

User creates SUBS token
         │
         ▼
┌─────────────────────────────┐
│ Local processing            │
│ - Validate subject exists   │
│ - Check for INVT or 'O'     │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ schedule_delivery()         │
│ - Send to subject owner     │
└───────────┬─────────────────┘
            │
            ▼
    Owner receives SUBS
         │
         ▼
┌─────────────────────────────┐
│ on_receive hook             │
│ - Check auto-accept logic   │
│ - Accept or reject          │
│ - Update subscription list  │
└─────────────────────────────┘

Message Fan-Out and Leaving

Once subscribed, messages are delivered via schedule_subscriber_fanout() which walks the parent chain to the CONV and delivers to all subscribers. See Subscriber Fan-Out for details.

To leave, the user creates a SUBS:DEL token. The subject owner marks the subscription as deleted and removes the user from the subscriber list.

Example

User @bob.cloudillo.net subscribes to a conversation owned by @alice.cloudillo.net:

Field Value
iss bob.cloudillo.net
aud alice.cloudillo.net
iat 2024-04-13T00:01:10.000Z
k 20240301
t SUBS
sub a1~NAado5PS4j5+abYtRpBELU0e5OQ+zGf/tuuWvUwQ6PA=
c {“role”: “member”, “invitedBy”: “alice.cloudillo.net”}

Status Values

Status Description
A (Active) Subscription is active, user receives messages
R (Rejected) Subscription was rejected (no invitation, not open)
D (Deleted) User left or was removed from the group

See Also

Invitation Token

This token represents an invitation for a user to join a subscribable action, such as a conversation (CONV). Invitations enable controlled access to closed groups.

The invitation token must contain a subject (sub) field referencing the subscribable action and an audience (aud) field specifying the invited user. For other constraints see the Action Tokens.

Content-Addressing

This token is content-addressed using SHA-256:

  • The entire JWT token (header + payload + signature) is hashed
  • Action ID format: a1~{base64_hash}
  • Changing any field invalidates the action_id
  • See Content-Addressing & Merkle Trees for details

Immutability: Once created, an INVT token cannot be modified without changing its action ID.

Purpose

INVT tokens serve several important purposes:

  1. Access Control: Enable users to join closed/private groups
  2. Role Assignment: Pre-assign roles for invited users
  3. Discoverability: Notify users about groups they can join
  4. Subject Delivery: Include the subject action (CONV) with the invitation

Required Fields

Field Required Description
iss Yes The inviter’s identity (must have moderator+ role)
aud Yes The invited user’s identity
sub Yes The action_id being invited to (a1~...)
t Yes “INVT” or “INVT:DEL”
c Optional Invitation metadata (role, message)

Subtypes

Subtype Description
INVT Create a new invitation
INVT:DEL Revoke an invitation

Content Structure

The optional content (c) field contains invitation metadata:

{
  "role": "member",
  "message": "Welcome to our project discussion!"
}
Property Type Required Description
role string No Role to assign when accepting: “observer”, “member”, “moderator”, “admin”
message string No Optional invitation message

Database Key

The database key for an invitation token is {type}:{sub}:{aud}

Purpose: This key ensures that only one active invitation exists per user per subject. The key components are:

  • {type}: “INVT” (base type)
  • {sub}: Subject ID (what they’re invited to)
  • {aud}: Audience identity (who is invited)

Example:

  • Alice invites Bob to a CONV → Stored with key INVT:a1~conv123:bob.example.com
  • A new invitation to the same user replaces the previous one

Permission Requirements

The inviter must have at least moderator role on the subject action. See Subscription Token roles for the full role hierarchy.

Subject Delivery

INVT has deliver_subject=true behavior, meaning:

  • When an invitation is delivered, the subject action (e.g., CONV) is included
  • The invitee receives both the INVT and the CONV token
  • This allows the invitee to see conversation details before accepting
{
  "token": "eyJhbGciOi...INVT_TOKEN",
  "related": ["eyJhbGciOi...CONV_TOKEN"]
}

Federation Flow

Sending an Invitation

Moderator creates INVT token
         │
         ▼
┌─────────────────────────────┐
│ Validate permissions        │
│ - Check inviter has mod+    │
│ - Validate subject exists   │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ schedule_delivery()         │
│ - Deliver to invitee (aud)  │
│ - Include subject action    │
└───────────┬─────────────────┘
            │
            ▼
    Invitee receives INVT + CONV

Accepting an Invitation

The invitee accepts by creating a SUBS token referencing the CONV. Because an INVT exists, the subscription is auto-accepted. See SUBS auto-accept logic for details.

Revoking an Invitation

A moderator creates an INVT:DEL token, which is delivered to the invitee. The original INVT is marked deleted and can no longer be used to subscribe.

Invitation Lifecycle

┌──────────┐     Create      ┌──────────┐
│  None    │────────────────▶│  Active  │
└──────────┘                 └──────────┘
                                   │
                    ┌──────────────┼──────────────┐
                    ▼              ▼              ▼
              SUBS created    INVT:DEL       Expires
                    │              │              │
                    ▼              ▼              ▼
              ┌──────────┐  ┌──────────┐  ┌──────────┐
              │  Used    │  │ Revoked  │  │ Expired  │
              └──────────┘  └──────────┘  └──────────┘

Example

User @alice.cloudillo.net invites @bob.cloudillo.net to a conversation:

Field Value
iss alice.cloudillo.net
aud bob.cloudillo.net
iat 2024-04-13T00:01:10.000Z
k 20240301
t INVT
sub a1~NAado5PS4j5+abYtRpBELU0e5OQ+zGf/tuuWvUwQ6PA=
c {“role”: “member”, “message”: “Join our project discussion!”}

Delivery includes both the INVT token and the referenced CONV token (see Subject Delivery above).

See Also

Action Type DSL

The Action Type DSL (Domain-Specific Language) defines action types declaratively, configuring validation rules, processing behavior, and lifecycle hooks without modifying core code.

ActionDefinition Structure

Each action type is defined by an ActionDefinition:

struct ActionDefinition {
    action_type: String,      // e.g., "POST", "CMNT", "FLLW"
    version: u32,             // Schema version
    metadata: Metadata,       // Display name, description, etc.
    subtypes: Vec<String>,    // e.g., POST:IMG, POST:VID
    fields: Vec<FieldDef>,    // Field definitions
    schema: Schema,           // JSON Schema for validation
    behavior: Behavior,       // Processing behavior flags
    hooks: Hooks,             // Lifecycle hooks
    permissions: Permissions, // Permission requirements
    key_pattern: String,      // Database key pattern for overriding
}

Fields

Each field in an action is defined with validation rules:

struct FieldDef {
    name: String,             // Field name in JWT (e.g., "c", "p", "vis")
    field_type: FieldType,    // string, object, array, number, boolean
    required: bool,           // Is this field required?
    validate: Option<Schema>, // Additional validation rules
}

Behavior Flags

Behavior flags control how actions are processed:

Flag Description Example Types
broadcast Send to multiple recipients (followers) POST, REPOST
allow_unknown Accept actions from unknown senders CONN
ephemeral Don’t persist to database PRES (presence)
approvable Can be approved/rejected by recipient CONN, FLLW
requires_subscription Sender must be subscribed MSG, CONV
deliver_subject Deliver to subject owner CMNT, REACT
deliver_to_subject_owner Alternative delivery to subject owner CMNT

Behavior Examples

POST Action (broadcast content):

Behavior {
    broadcast: true,              // Send to all followers
    allow_unknown: false,         // Must be from followed/connected user
    ephemeral: false,             // Persist to database
    approvable: false,            // No approval needed
    requires_subscription: false, // No subscription required
    deliver_subject: false,       // Not a reply
}

CONN Action (connection request):

Behavior {
    broadcast: false,             // Send only to audience
    allow_unknown: true,          // Accept from anyone (with PoW)
    ephemeral: false,             // Persist to database
    approvable: true,             // Recipient can accept/reject
    requires_subscription: false, // No subscription required
    deliver_subject: false,       // Direct to audience
}

Hook System

Hooks allow custom logic at different points in the action lifecycle:

struct Hooks {
    on_create: Option<HookFn>,   // Called when action is created locally
    on_receive: Option<HookFn>,  // Called when action is received from federation
    on_accept: Option<HookFn>,   // Called when approvable action is accepted
    on_reject: Option<HookFn>,   // Called when approvable action is rejected
}

Hooks handle tasks such as: validating business logic and generating notifications on create/receive, establishing connections or granting permissions on accept, and cleaning up pending state on reject.

Permissions

Define who can perform which operations:

struct Permissions {
    create: PermissionRule,    // Who can create this action type
    read: PermissionRule,      // Who can read actions of this type
    delete: PermissionRule,    // Who can delete (owner only typically)
}

Permission Rules

Rule Description
Owner Only the action issuer
Audience The action’s audience field
Connected Mutually connected users
Follower Users who follow the issuer
Verified Any authenticated user
Public Anyone, including unauthenticated

Key Pattern

The key_pattern determines how actions are stored and whether they can be overridden:

Action Type Key Pattern Override Behavior
POST {iss}:{id} Each post is unique
CMNT {iss}:{p}:{id} Each comment is unique (p = parent)
REACT {iss}:{sub} One reaction per user per subject
FLLW {iss}:{aud} One follow per user per target
CONN {iss}:{aud} One connection request per pair

Override example:

User reacts to post with LIKE
  → Stored at key: "alice.example.com:a1~post123"

User changes reaction to LOVE
  → Same key: "alice.example.com:a1~post123"
  → Previous LIKE is REPLACED by LOVE

Example Definition

POST Definition

ActionDefinition {
    action_type: "POST",
    version: 1,
    metadata: Metadata {
        display_name: "Post",
        description: "A public post or status update",
        category: Category::Content,
    },
    subtypes: vec!["IMG", "VID", "LNK", "TXT"],
    fields: vec![
        FieldDef { name: "c", field_type: String, required: true },
        FieldDef { name: "a", field_type: Array, required: false },
        FieldDef { name: "vis", field_type: String, required: false },
        FieldDef { name: "f", field_type: String, required: false },
    ],
    behavior: Behavior {
        broadcast: true,
        allow_unknown: false,
        ephemeral: false,
        approvable: false,
        requires_subscription: false,
        deliver_subject: false,
    },
    hooks: Hooks {
        on_create: Some(post_on_create),
        on_receive: Some(post_on_receive),
        on_accept: None,
        on_reject: None,
    },
    permissions: Permissions {
        create: PermissionRule::Owner,
        read: PermissionRule::ByVisibility,
        delete: PermissionRule::Owner,
    },
    key_pattern: "{iss}:{id}",
}

Other action types (CMNT, CONV, etc.) follow the same pattern with type-specific fields, behavior flags, and hooks.

DSL Validation

When an action is created or received, it’s validated against its DSL definition:

validate_action(action, definition):
    1. Check required fields exist
    2. Validate field types
    3. Run schema validation
    4. Check behavior constraints:
       - If requires_subscription: verify sender is subscribed
       - If !allow_unknown: verify sender is known (following/connected)
       - If approvable: set initial status to pending
    5. Return validation result

See Also

Metadata Actions

Action tokens for metadata, system operations, and auxiliary information about other tokens and actions on the Cloudillo network.

Contains:

Subsections of Metadata Actions

Statistics Token

This token represents the statistics of reactions on an object (post, comment, etc.)

The issuer (iss) of the token must be the audience (aud) of the parent token (p). A statistics token must not contain an audience (aud) field. The token must contain a parent (p) field which points to the parent object the statistics are referring to. The statistics token must contain a content (c) field which is a JSON object in the following format:

Field Value Description
c number The number of comments (optional)
r number The number of reactions (optional)

For other constraints see the Action Tokens.

Content-Addressing

This token is content-addressed using SHA-256:

Parent Reference

The p (parent) field references the action being counted:

  • Contains the parent action’s action_id (a1~...)
  • Statistics token issued by the audience of the parent token
  • Creates aggregated view of reactions and comments

Database Key

The database key for a statistics token is [iss, t, p]

Purpose: This key ensures that only one statistics token exists per parent action. The key components are:

  • iss: Issuer identity (audience of the parent token)
  • t: Token type (“STAT”)
  • p: Parent ID (what’s being counted)

Example:

  • Group posts statistics for a post → Stored with key [somegroup.cloudillo.net, "STAT", a1~post123]
  • Statistics update → New token with same key, previous one is marked deleted
  • Only ONE statistics token per parent action

Example

User @someotheruser.cloudillo.net also likes the post:

Field Value
iss somegroup.cloudillo.net
iat 2024-04-13T00:01:10.000Z
k 20240301
t STAT
p NAado5PS4j5+abYtRpBELU0e5OQ+zGf/tuuWvUwQ6PA=
c { “c”: 1, “r”: 2 }

Fileshare Token

Overview

The FSHR (Fileshare) token represents sharing a file with another user. It grants the recipient access to view or edit the specified file.

Token Structure

Field Type Required Description
iss identity Yes The identity sharing the file
aud identity Yes The recipient of the share
iat timestamp Yes Issue time
k string Yes Signing key ID
t string Yes FSHR or FSHR:WRITE or FSHR:DEL
sub string Yes File ID being shared (e.g., f1~abc123)
c object Yes Share content (see below)

Subtypes

Subtype Type String Description
Read (default) FSHR View-only access to the file
Write FSHR:WRITE Edit access to the file
Delete FSHR:DEL Revokes a previous share

Content Schema

Field Type Required Description
contentType string Yes MIME type (e.g., application/pdf)
fileName string Yes Original filename
fileTp string Yes File type: BLOB, CRDT, or RTDB

File Types

Type Description
BLOB Static binary file (images, PDFs, etc.)
CRDT Collaborative document (real-time editing)
RTDB Real-time database document

Content-Addressing

This token is content-addressed using SHA-256:

Database Key

The database key for a fileshare token is [iss, t, sub]

Purpose: This key ensures that a user can only have one active fileshare for a given file. The key components are:

  • iss: Issuer identity (who is sharing)
  • t: Token type (FSHR)
  • sub: Subject (the file being shared)

Example:

  • Alice shares file with Bob → Stored with key [alice.example.com, "FSHR", f1~file123]
  • Alice updates the share → New token with same key, previous one is marked deleted
  • Only ONE fileshare of the same file from the same user to the same recipient

Examples

Read-Only Share

User @alice.cloudillo.net shares a PDF with @bob.cloudillo.net (read-only):

Field Value
iss alice.cloudillo.net
aud bob.cloudillo.net
iat 2024-04-13T00:01:10.000Z
k 20240101
t FSHR
sub f1~7NtuTab_K4FwYmARMNuk4
c.contentType application/pdf
c.fileName report.pdf
c.fileTp BLOB

Write Access Share

User @alice.cloudillo.net grants edit access to a collaborative document:

Field Value
iss alice.cloudillo.net
aud bob.cloudillo.net
iat 2024-04-13T00:01:10.000Z
k 20240101
t FSHR:WRITE
sub f1~collaborative_doc_id
c.contentType application/vnd.cloudillo.doc
c.fileName Team Notes.quillo
c.fileTp CRDT

Revoke Share

User @alice.cloudillo.net revokes Bob’s access:

Field Value
iss alice.cloudillo.net
aud bob.cloudillo.net
iat 2024-04-13T00:05:00.000Z
k 20240101
t FSHR:DEL
sub f1~7NtuTab_K4FwYmARMNuk4

Integration with References

File shares can be combined with references for public sharing:

  1. Create FSHR action to share with specific user
  2. Or create a reference with type: share.file for public/guest access
  3. Reference tokens support accessLevel: read or accessLevel: write

Access Control Flow

Owner creates FSHR action
  ↓
Action sent to recipient via federation
  ↓
Recipient's server verifies:
  - JWT signature
  - Issuer owns the file
  - Recipient matches audience
  ↓
Access granted based on subtype:
  - FSHR → Read access
  - FSHR:WRITE → Write access
  ↓
Access revoked on FSHR:DEL receipt

See Also

IDP:REG Token

Overview

The IDP:REG (Identity Provider Registration) token enables federated identity registration. It allows a registrar to request identity creation on a remote Identity Provider (IDP) instance, enabling community-owned identities and cross-instance identity management.

Use Cases

  • Community-Owned Identities: A community owner creates identities under their domain on a remote IDP
  • Federated Registration: Registrar on instance A creates identities on IDP instance B
  • Bulk Identity Provisioning: Automated identity creation for organizations

Token Structure

Field Type Required Description
iss identity Yes The registrar’s identity
aud identity Yes The IDP instance to register on
iat timestamp Yes Issue time
k string Yes Signing key ID
t string Yes IDP:REG
c object Yes Registration content (see below)

Content Schema

Field Type Required Description
id_tag string Yes Identity to register (e.g., alice.cloudillo.net)
email string No Email address (required if no owner_id_tag)
owner_id_tag string No Owner identity for community-owned identities
issuer string No Issuer role: registrar (default) or owner
expires_at number No Identity expiration timestamp

Content-Addressing

This token is content-addressed using SHA-256:

Database Key

The database key for an IDP:REG token is [type, issuer, audience, content.id_tag]

Purpose: This key ensures that only one registration request exists for a given identity from a given registrar. The key components are:

  • type: Token type (IDP:REG)
  • issuer: Registrar identity
  • audience: Target IDP instance
  • content.id_tag: The identity being registered

Behavior Flags

Flag Value Description
broadcast false Not broadcast to followers
allow_unknown true Allows additional content fields
requires_acceptance false Processed immediately on receipt

Authorization

Role Can Create Can Receive
Registrar Yes (authenticated) -
IDP Instance - Yes (any)

Issuer Roles

The issuer field in content determines the relationship:

Role Description
registrar Default. Registrar manages identity until activation
owner Issuer is the owner of the identity

After activation, registrar control is revoked - only the owner retains access.

Example

A community owner (community@cloudillo.net) registers a new member identity:

{
  "iss": "community@cloudillo.net",
  "aud": "cl-o.cloudillo.net",
  "iat": 1735000000,
  "k": "20250101",
  "t": "IDP:REG",
  "c": {
    "id_tag": "alice.cloudillo.net",
    "email": "alice@example.com",
    "owner_id_tag": "community@cloudillo.net",
    "issuer": "owner",
    "expires_at": 1766536000
  }
}
Field Value
iss community@cloudillo.net
aud cl-o.cloudillo.net
iat 2025-01-02T00:00:00Z
k 20250101
t IDP:REG
c.id_tag alice.cloudillo.net
c.owner_id_tag community@cloudillo.net

Processing Flow

1. Registrar creates IDP:REG action
   ↓
2. Action sent to IDP instance (/api/inbox)
   ↓
3. IDP verifies:
   - JWT signature
   - Issuer has permission to create identities
   - Domain matches IDP's domain
   ↓
4. Identity created with status "pending"
   ↓
5. IDP sends activation link directly to identity owner (via email)
   ↓
6. Identity owner activates via /api/idp/activate
   ↓
7. Identity status becomes "active"

Federation Diagram

sequenceDiagram
    participant R as Registrar
    participant IDP as IDP Instance
    participant O as Identity Owner

    R->>R: Create IDP:REG JWT
    R->>IDP: POST /api/inbox {token}
    IDP->>IDP: Verify JWT signature
    IDP->>IDP: Create identity (pending)
    IDP-->>R: 200 OK

    IDP->>O: Send activation email/link
    O->>IDP: POST /api/idp/activate {refId}
    IDP->>IDP: Activate identity
    IDP-->>O: 200 OK {identity}

Community Ownership

When owner_id_tag is provided:

  1. The identity is owned by the specified community/user
  2. The registrar only has control while status is pending
  3. After activation, only the owner can manage the identity
  4. Email is optional (owner relationship is the primary identifier)

Security Considerations

Signature Verification

The receiving IDP verifies:

  1. JWT signature matches issuer’s public key
  2. Issuer is allowed to create identities on this IDP
  3. Requested id_tag domain matches IDP’s domain

Domain Validation

The id_tag in content must match the IDP’s domain:

  • Request: alice.cloudillo.net → IDP: cloudillo.net
  • Request: alice.other.net → IDP: cloudillo.net ❌ (rejected)

Registrar Control Limits

Registrar control is time-limited:

  • Full control while identity status is pending
  • No control after identity is activated
  • Prevents registrar from managing user’s active identity

See Also

Approval Token

This token represents an approval of another user’s content. When you approve someone’s action (e.g., a POST), it signals trust and enables federated fan-out to your followers.

The approval token must NOT contain a content (c) or attachments (a) field. The token must contain a subject (sub) field referencing the action being approved. For other constraints see the Action Tokens.

Content-Addressing

This token is content-addressed using SHA-256:

  • The entire JWT token (header + payload + signature) is hashed
  • Action ID format: a1~{base64_hash}
  • Changing any field invalidates the action_id
  • See Content-Addressing & Merkle Trees for details

Immutability: Once created, an APRV token cannot be modified without changing its action ID.

Purpose

APRV tokens serve several important purposes in the federated network:

  1. Trust Signal: Indicates that you endorse the referenced content
  2. Federated Fan-Out: When you approve a POST, it gets broadcast to your followers along with the approved content
  3. Content Discovery: Helps content spread across the network through trusted connections
  4. Status Update: Updates the original action’s status to ‘Active’ (A) on the original author’s instance

Required Fields

Field Required Description
iss Yes Your identity (the approver)
aud Yes The issuer of the approved action (content creator)
sub Yes The action_id being approved (a1~...)
t Yes “APRV”
c Forbidden Content field is not allowed
a Forbidden Attachments field is not allowed

Subject Reference

The sub (subject) field references the action being approved:

  • Contains the target action’s action_id (a1~...)
  • Target action must exist and be verifiable
  • Creates a non-hierarchical reference

Why Subject Instead of Parent:

  • APRV doesn’t create a visible hierarchy (unlike comments)
  • It references the action without threading
  • The semantic is “this action is about that action”

Broadcast Behavior

APRV tokens have broadcast=true behavior, meaning:

  • When you create an APRV, it’s sent to all your followers
  • The approved action (e.g., the POST) is bundled with the APRV delivery
  • Recipients receive both the APRV and the related action in a single delivery

This is how content spreads across the federated network through trust relationships.

Federation Flow

Creating an Approval

User approves remote content
         │
         ▼
┌─────────────────────────────┐
│ Create APRV token           │
│ - aud = content creator     │
│ - sub = content action_id   │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ schedule_delivery()         │
│ - Check subject broadcast   │
│ - Schedule broadcast fanout │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ schedule_broadcast_delivery │
│ - Get all followers         │
│ - Include original author   │
│ - Bundle approved action    │
└───────────┬─────────────────┘
            │
            ▼
    Delivered to all followers
    with approved POST bundled

Receiving an Approval

When you receive an APRV for your content:

APRV arrives at /inbox
         │
         ▼
┌─────────────────────────────┐
│ ActionVerifierTask          │
│ - Verify JWT signature      │
│ - Check permissions         │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ on_receive hook             │
│ - Find subject action       │
│ - Update status to 'A'      │
└───────────┬─────────────────┘
            │
            ▼
    Your content is now approved

Auto-Approval

The system can automatically create APRV tokens for content from trusted connections. See Auto-Approval for details.

Auto-approval conditions:

  1. Action type must be approvable (POST, MSG, REPOST)
  2. Action is addressed to you (audience = your id_tag)
  3. Sender is different from you
  4. federation.auto_approve setting is enabled
  5. Sender is connected (bidirectional connection established)

When an APRV is delivered, the approved action is included:

{
  "token": "eyJhbGciOi...APRV_TOKEN",
  "related": ["eyJhbGciOi...APPROVED_POST_TOKEN"]
}

Recipients receive both tokens in a single request. The related action is processed after the APRV, with permission checks skipped (pre-approved by the APRV issuer’s trust).

Example

User @alice.cloudillo.net approves a post from @bob.cloudillo.net:

Field Value
iss alice.cloudillo.net
aud bob.cloudillo.net
iat 2024-04-13T00:01:10.000Z
k 20240301
t APRV
sub a1~NAado5PS4j5+abYtRpBELU0e5OQ+zGf/tuuWvUwQ6PA=

Flow:

  1. Alice approves Bob’s post
  2. APRV is sent to Bob (notification)
  3. APRV + Bob’s POST are broadcast to Alice’s followers
  4. Bob’s post status is updated to ‘A’ (Active/Approved)

See Also

Presence Token

The PRES token represents an ephemeral presence indication, such as typing status or online presence. Unlike other action types, presence tokens are not persisted – they are forwarded via WebSocket in real-time only, with no action_id generated and no delivery retry logic. They have a default TTL of 30 seconds.

The presence token must contain a subject (sub) field referencing the context (e.g., a conversation). For other constraints see the Action Tokens.

Fields

Field Required Description
iss Yes The user’s identity
sub Yes Context (e.g., CONV action_id)
aud Optional Specific target user (if omitted, broadcast to context)
t Yes “PRES:TYPING”, “PRES:ONLINE”, etc.
c Optional Additional metadata
a Forbidden Attachments are not allowed

Subtypes

Subtype Description
PRES:TYPING User is currently typing
PRES:ONLINE User is online/active
PRES:AWAY User is away/idle
PRES:OFFLINE User has gone offline

Additional subtypes can be defined for application-specific presence states.

Subject Reference

The sub (subject) field specifies the context for the presence update (e.g., a CONV action_id for typing indicators, or a profile context for online status). Unlike the parent field used by comments, sub expresses a non-hierarchical contextual reference.

Processing Flow

When a PRES token is received, the server verifies the signature, skips database storage, and immediately forwards it via WebSocket to the audience (single user if aud is set, otherwise all context subscribers).

For cross-instance presence, the token is forwarded via HTTP POST to the context owner’s instance, which then broadcasts to its local subscribers.

Time-To-Live (TTL)

PRES tokens have an implicit 30-second TTL. Clients should send periodic updates to maintain presence (e.g., PRES:TYPING every 5 seconds while typing). After the TTL expires without a new update, recipients consider the presence stale.

Example

User @alice.cloudillo.net is typing in a conversation:

Field Value
iss alice.cloudillo.net
sub a1~NAado5PS4j5+abYtRpBELU0e5OQ+zGf/tuuWvUwQ6PA=
iat 2024-04-13T00:01:10.000Z
k 20240301
t PRES:TYPING

Security Considerations

Since PRES tokens are ephemeral, they still require valid signature verification, rate limiting applies to prevent spam, and the user must have access to the referenced context.

Comparison with Persistent Actions

Aspect PRES (Ephemeral) Other Actions
Database storage No Yes
Action ID None SHA256 hash
Delivery tasks None Scheduled with retry
TTL 30 seconds Permanent (until deleted)
Use case Real-time status Permanent records

See Also

Federation Architecture

Cloudillo is designed as a federated system where independent instances communicate to share content, enable collaboration, and maintain user sovereignty. Like email, any Cloudillo server can communicate with any other – users don’t need to be on the same instance.

Core Principles

  • No central authority: Each instance operates autonomously
  • User sovereignty: Users choose where their data lives
  • Explicit consent: Relationship-based sharing with cryptographic verification
  • Standard protocols: HTTP/HTTPS, WebSocket, JWT, DNS-based identity
  • Content addressing: SHA256 ensures integrity across instances

Inter-Instance Communication

Request Module

The request module provides HTTP client functionality for federation:

Request Methods:

  • get(url) - Generic HTTP GET
  • post(url, body) - Generic HTTP POST with JSON body
  • fetch_profile(id_tag) - GET /api/me from remote instance
  • fetch_action(id_tag, action_id) - GET /api/actions/:id from remote
  • post_inbox(id_tag, token) - POST /api/inbox with action token
  • fetch_file(id_tag, file_id) - GET /api/files/:id from remote

Federation Flow

When Alice (on instance A) follows Bob (on instance B):

Instance A                          Instance B
  |                                   |
  |--- GET /api/me ------------------>| (Fetch Bob's profile)
  |<-- 200 OK {profile} --------------|
  |                                   |
  |--- Create FLLW action token ---   |
  |                                   |
  |--- POST /api/inbox -------------->| (Send follow action)
  |    {token: "eyJhbGc..."}         |
  |                                   |--- Verify signature
  |                                   |--- Check permissions
  |                                   |--- Store action
  |<-- 202 Accepted ------------------|

See Also

Subsections of Federation Architecture

Action Delivery

How actions are distributed and received across the federated Cloudillo network.

Outbound Actions

When a user creates an action, it’s distributed to relevant recipients:

Algorithm: Distribute Action

Input: action_token, action_type
Output: Result<()>

1. Determine recipients based on action_type:
   a. "POST" → All followers of the user
   b. "CMNT" / "REACT" → Owner of parent post
   c. "CONN" / "FLLW" → Target from action audience (aud claim)
   d. "MSG" → All conversation participants
   e. Other → No recipients (skip)

2. For each recipient instance:
   a. Construct URL: https://cl-o.{recipient_id_tag}/api/inbox
   b. POST action token as JSON: {"token": "..."}
   c. Continue on error (best-effort delivery)

3. Return success

Inbound Actions

Cloudillo provides two endpoints for receiving federated actions, optimized for different use cases.

Async Inbox (/api/inbox)

The standard endpoint for most federated actions:

HTTP Endpoint: POST /api/inbox

Handler Algorithm:
1. Extract action token from JSON payload
2. No authentication required (token is self-authenticating)
3. Create ActionVerifierTask with token
4. Schedule task in task scheduler (for async processing)
5. Return HTTP 202 (Accepted) immediately
   - Verification and processing happen asynchronously
   - Prevents network timeouts on slow verification

Sync Inbox (/api/inbox/sync)

A synchronous endpoint for actions requiring immediate confirmation:

HTTP Endpoint: POST /api/inbox/sync

Handler Algorithm:
1. Extract action token from JSON payload
2. No authentication required (token is self-authenticating)
3. Process action SYNCHRONOUSLY (not queued)
4. Verify signature, check permissions, store action
5. Return HTTP 200 (OK) with result, or error response

Use Cases:
- Identity registration tokens (IDP:REG)
- Actions where sender needs immediate confirmation
- Critical federation handshakes

When to use each endpoint:

  • /api/inbox: Default for most actions (POST, CMNT, REACT, MSG, etc.)
  • /api/inbox/sync: Only for critical operations requiring immediate feedback

Delivery Guarantees

Persistent Delivery with Retry

  • Delivery tasks are persisted to survive server restarts
  • Automatic retry with exponential backoff
  • Comprehensive failure handling

Retry Policy

Cloudillo uses exponential backoff for delivery retries:

RetryPolicy Configuration:
  initial_wait:  10 seconds    # First retry delay
  max_wait:      12 hours      # Maximum delay between retries
  backoff_step:  50 seconds    # Backoff increment per retry

Retry Schedule Example:

Attempt 1: Immediate
  ↓ fails
Attempt 2: 10s later (initial_wait)
  ↓ fails
Attempt 3: 60s later (10s + 50s backoff)
  ↓ fails
Attempt 4: 110s later (60s + 50s backoff)
  ↓ fails
...continues with 50s backoff steps...
  ↓
Maximum wait capped at 12 hours between attempts
  • Task deduplication: Key pattern delivery:{action_id}:{recipient} prevents duplicate deliveries
  • Persistent delivery: Tasks survive server restarts via MetaAdapter storage

Failure Handling

On POST /api/inbox result:

1. Success (2xx):
   - Remove from delivery queue
   - Log successful delivery

2. Temporary Error (network timeout, 5xx, connection refused):
   - Keep in queue
   - Schedule retry with backoff
   - Continue retrying until max attempts

3. Permanent Error (4xx status, validation error):
   - Log error with context
   - Mark as undeliverable
   - Remove from queue (don't waste resources)

When certain actions are approved, they trigger delivery of related actions to additional recipients:

APRV (Approval) Fan-out

When a user approves another user’s content (e.g., repost), the system distributes the original content to the approver’s followers:

APRV Fan-out Algorithm:

1. User A creates POST action
2. User B reposts (creates APRV referencing POST)
3. On APRV creation:
   a. Fetch User B's followers
   b. For each follower:
      - Create delivery task for original POST
      - Include APRV context (who approved)
   c. Followers receive both POST and APRV

Use Cases:

  • Repost/Share: Original post reaches approver’s audience
  • Endorsement: Endorsed content gets wider distribution
  • Curated feeds: Content discovery through trusted connections
Trigger Action Related Actions Delivered Recipients
APRV (Approval) Original referenced action Approver’s followers
CONV (Conversation) Participant subscriptions All participants
INVT (Invitation) Conversation context Invited user

See Also

Key Verification & Caching

When receiving federated actions, the server must verify the JWT signature using the issuer’s public key. This involves a 3-tier caching strategy to balance security with performance.

3-Tier Caching Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     Verification Request                        │
└───────────────────────────────┬─────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│  Tier 1: In-Memory Failure Cache                                │
│  ├─ Purpose: Prevent repeated requests to unreachable instances │
│  ├─ TTL (network errors): 5 minutes                             │
│  ├─ TTL (persistent errors): 1 hour                             │
│  └─ LRU eviction when capacity exceeded                         │
└───────────────────────────────┬─────────────────────────────────┘
                                │ (cache miss or expired)
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│  Tier 2: SQLite Key Cache                                       │
│  ├─ Purpose: Persistent cache of successful key fetches         │
│  ├─ Key: (issuer, key_id)                                       │
│  ├─ Stores: public key + expiration timestamp                   │
│  └─ If valid & not expired: verify signature immediately        │
└───────────────────────────────┬─────────────────────────────────┘
                                │ (cache miss or expired)
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│  Tier 3: HTTP Fetch from Remote                                 │
│  ├─ Endpoint: GET https://cl-o.{issuer}/api/me                  │
│  ├─ Find matching key by key_id                                 │
│  ├─ On success: cache in SQLite, clear failure cache            │
│  └─ On failure: record in failure cache                         │
└─────────────────────────────────────────────────────────────────┘

Failure Types and TTLs

Error Type TTL Reason
Network timeout 5 minutes May recover quickly
Connection refused 5 minutes Server may restart
404 Not Found 1 hour Key doesn’t exist
403 Forbidden 1 hour Permission denied
Parse error 1 hour Invalid response format

Verification Flow

verify_action_token(token):
    1. Decode JWT without verifying (extract issuer, key_id)

    2. Check failure cache:
       if failure_cache.has(issuer, key_id) and not expired:
           return Error(CachedFailure)

    3. Check SQLite cache:
       if key_cache.has(issuer, key_id):
           key = key_cache.get(issuer, key_id)
           if key.expires_at > now:
               return verify_signature(token, key.public_key)

    4. Fetch from remote:
       response = HTTP GET https://cl-o.{issuer}/api/me
       if error:
           failure_cache.record(issuer, key_id, error_type)
           return Error(KeyFetchFailed)

       public_key = find_key_by_id(response.keys, key_id)
       if not found:
           failure_cache.record(issuer, key_id, NotFound)
           return Error(KeyNotFound)

       key_cache.store(issuer, key_id, public_key)
       failure_cache.clear(issuer, key_id)

       return verify_signature(token, public_key)

See Also

Trust & Content Distribution

How trust relationships enable automatic content approval and efficient message distribution across the federated network.

Auto-Approval for Trusted Connections

When receiving actions from connected users, the system can automatically create approval (APRV) tokens, enabling content to spread through the network via trusted relationships.

Auto-Approval Conditions

For an action to be auto-approved, ALL of the following must be true:

  1. Approvable action type: Action type has approvable=true behavior
    • Applies to: POST, MSG, REPOST
  2. Addressed to us: action.audience == our_id_tag
  3. From different user: action.issuer != our_id_tag
  4. Setting enabled: federation.auto_approve = true in settings
  5. Connection established: Issuer has bidirectional connection with us

Auto-Approval Flow

Inbound action received
         │
         ▼
┌─────────────────────────────┐
│ Check auto-approval         │
│ conditions (all 5)          │
└───────────┬─────────────────┘
            │
    ┌───────┴───────┐
    │               │
    ▼               ▼
Conditions      Conditions
met             not met
    │               │
    ▼               ▼
┌─────────────┐  No action
│ Update      │  (standard
│ action      │  processing)
│ status='A'  │
└──────┬──────┘
       │
       ▼
┌─────────────────────────────┐
│ Create APRV token           │
│ - aud = original issuer     │
│ - sub = action_id           │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ Deliver APRV to issuer      │
│ (notification)              │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ Broadcast APRV + action     │
│ to our followers            │
└─────────────────────────────┘

Trust Indicator

Trust = Bidirectional Connection

The system checks issuer_profile.connected.is_connected() to determine trust. A connection is established when both parties have sent CONN tokens to each other.

Subscriber Fan-Out

For subscribable actions (like CONV), messages need to be delivered to all subscribers. The fan-out mechanism ensures efficient message distribution across the federated network.

Subscribable Actions

Actions with subscribable=true behavior can have subscribers:

  • CONV (Conversation) - subscribers receive messages in the group

Fan-Out Algorithm

schedule_subscriber_fanout(action_id, parent_id, issuer):
    1. Walk parent chain to find subscribable root:
       current = parent_id
       while current:
           parent_action = fetch_action(current)
           if is_subscribable(parent_action.type):
               subscribable_root = parent_action
               break
           current = parent_action.parent_id

    2. Check if we own the subscribable root:
       is_local = (subscribable_root.audience == null
                   && subscribable_root.issuer == our_id_tag)
                  || subscribable_root.audience == our_id_tag

       if not is_local:
           return  // Remote owner handles fan-out

    3. Get subscribers:
       subscribers = query_subscriptions(subscribable_root.action_id)
                     .filter(status = 'A')  // Active only

    4. Create delivery tasks:
       for subscriber in subscribers:
           if subscriber != our_id_tag
              && subscriber != issuer:  // Exclude self and sender
               schedule_delivery_task(
                   action_id,
                   subscriber,
                   key = "fanout:{action_id}:{subscriber}"
               )

Local vs Remote Fan-Out

Scenario 1: Local CONV Owner Sends Message

Alice (CONV owner) sends MSG
         │
         ▼
┌─────────────────────────────┐
│ ActionCreatorTask           │
│ - Creates MSG               │
│ - Calls schedule_delivery() │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ schedule_subscriber_fanout()│
│ - Finds CONV (subscribable) │
│ - CONV is local (we own it) │
│ - Gets all SUBS             │
│ - Schedules delivery to each│
└───────────┬─────────────────┘
            │
            ▼
    Bob, Charlie receive MSG
    (all other subscribers)

Scenario 2: Remote Subscriber Sends Message

Bob (subscriber, not owner) sends MSG
on his instance
         │
         ▼
┌─────────────────────────────┐
│ schedule_subscriber_fanout()│
│ - Finds CONV (subscribable) │
│ - CONV is NOT local         │
│ - No local fan-out          │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ schedule_delivery()         │
│ - Deliver to CONV owner     │
│   (Alice)                   │
└───────────┬─────────────────┘
            │
            ▼ (federation)
┌─────────────────────────────┐
│ Alice's instance receives   │
│ MSG at /inbox               │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ schedule_subscriber_fanout()│
│ - CONV is local to Alice    │
│ - Fan out to all except Bob │
└───────────┬─────────────────┘
            │
            ▼
    Charlie, others receive MSG

When delivering certain actions (like APRV), related actions are bundled together in a single delivery to provide context.

Bundling Use Cases

  1. APRV + Approved POST: When broadcasting an approval, include the approved content
  2. INVT + Subject CONV: When inviting, include the conversation details

Delivery Payload Structure

{
  "token": "eyJhbGciOi...MAIN_ACTION_TOKEN",
  "related": [
    "eyJhbGciOi...RELATED_ACTION_TOKEN_1",
    "eyJhbGciOi...RELATED_ACTION_TOKEN_2"
  ]
}
Inbox receives action with related tokens
         │
         ▼
┌─────────────────────────────┐
│ Store related tokens        │
│ - Status = 'W' (waiting)    │
│ - ack_token = main action   │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ Process main action         │
│ - Verify signature          │
│ - Check permissions         │
│ - Store action              │
│ - Execute on_receive hook   │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ process_related_actions()   │
│ - Get tokens with           │
│   ack_token = main action   │
│ - For each:                 │
│   - Skip permission check   │
│   - Verify signature        │
│   - Store action            │
└─────────────────────────────┘

Key point: Related actions skip permission checks because they are pre-approved by the main action issuer. The trust flows from the APRV issuer to the related content.

APRV Broadcast Example

Alice approves Bob's POST
         │
         ▼
┌─────────────────────────────┐
│ Create APRV                 │
│ - sub = Bob's post action_id│
│ - Check: subject.broadcast  │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ schedule_broadcast_delivery │
│ - Get Alice's followers     │
│ - Include Bob's POST token  │
│   as related action         │
└───────────┬─────────────────┘
            │
            ▼
┌─────────────────────────────┐
│ For each follower:          │
│ POST /inbox                 │
│ {                           │
│   token: APRV_token,        │
│   related: [POST_token]     │
│ }                           │
└─────────────────────────────┘

See Also

ProxyToken Authentication

For cross-instance requests, ProxyTokens authenticate the requesting instance.

ProxyToken Structure

{
  "iss": "alice.example.com",      // Requesting instance
  "aud": "bob.example.com",         // Target instance
  "sub": "alice.example.com",       // User identity
  "exp": 1738400000,                // Short-lived (5-60 min)
  "iat": 1738396800,
  "action": "read_file",            // Requested operation
  "resource": "f1~abc123...",       // Resource identifier
  "k": "20250205"                   // Signing key ID
}

Creation

Algorithm: Create ProxyToken

Input: requester id_tag, target_instance, action, resource
Output: JWT token string

1. Retrieve latest signing key:
   - Query latest key_id for tenant
   - Load private key from AuthAdapter

2. Build JWT claims:
   - iss: Requester's id_tag
   - aud: Target instance domain
   - sub: Requester's id_tag
   - exp: Current time + 30 minutes
   - iat: Current time
   - action: Requested operation
   - resource: Resource identifier
   - k: Key ID used for signing

3. Sign JWT using ES384 algorithm:
   - Use private key
   - Standard JWT encoding

4. Return base64-encoded JWT

Validation

Algorithm: Validate ProxyToken

Input: JWT token string
Output: Result<ProxyTokenClaims>

1. Decode JWT without signature verification (read claims)
2. Extract issuer and key_id from unverified claims
3. Fetch issuer's profile from remote instance
4. Look up public key by key_id in profile
5. Verify JWT signature using issuer's public key (ES384)
6. Check expiration timestamp:
   - If exp < current_time: Return TokenExpired error
7. Check audience claim:
   - If aud != this_instance.base_id_tag: Return InvalidAudience error
8. Return verified claims

Validation ensures:
- Token signed by claimed issuer
- Token not expired
- Token intended for this instance

Use Cases

ProxyTokens are used for:

  • File fetching: Downloading attachments from remote instances
  • Profile queries: Accessing extended profile information
  • Database sync: Read access to federated databases

See Also

Data Synchronization

How files, profiles, and databases are synchronized across federated instances.

File Synchronization

Attachment Fetching

When receiving an action with file attachments:

Algorithm: Sync Attachments

Input: attachment_ids from remote action
Output: Result<()>

For each attachment_id:
1. Check if already exists locally
   - If exists: Skip to next (already synced)

2. Construct remote URL:
   https://cl-o.{issuer_id_tag}/api/files/{attachment_id}

3. Download file from remote instance

4. Verify content integrity:
   - Compute SHA256 hash of downloaded data
   - Compare hash with attachment_id
   - If mismatch: Return FileIntegrityCheckFailed error

5. Store file data in blob adapter

6. Extract and store metadata:
   - Read X-File-Metadata header (if present)
   - Parse as JSON
   - Store in metadata adapter

7. Continue to next attachment

This ensures:
- Content-addressed files (hash = ID)
- No duplicate downloads
- Cryptographic integrity verification

Lazy Loading

Files are fetched on-demand rather than proactively:

User views post with image attachment
  ↓
Check if image exists locally
  ↓
If not, fetch from remote instance
  ↓
Verify content hash
  ↓
Store locally
  ↓
Serve to user

Profile Synchronization

Remote Profile Caching

Cache remote profiles locally for performance:

Algorithm: Sync Profile with Caching

Input: id_tag (remote user identifier)
Output: Result<Profile>

1. Check local cache for profile:
   - If cached AND cache_age < 24 hours: Return cached profile
   - If cache_age >= 24 hours: Continue to step 2

2. Fetch profile from remote instance:
   - GET https://cl-o.{id_tag}/api/me

3. Update local cache:
   - Store profile with current timestamp

4. Return profile

Benefits:
- Reduces network requests (24h TTL)
- Improves performance for repeated access
- Staleness acceptable for user profiles

Profile Updates

Profiles don’t push updates; instances pull when needed:

Need to display Alice's profile
  ↓
Check cache (last updated < 24h?)
  ↓
If fresh: use cache
If stale: fetch from Alice's instance
  ↓
Update cache
  ↓
Display profile

Database Federation

Read-Only Replication

Subscribe to remote database updates:

FederatedDatabase Structure:

  • origin_instance: Source instance domain (e.g., alice.example.com)
  • local_replica: Whether to maintain local copy for fast access
  • sync_mode: Synchronization mode (see below)

SyncMode Enum:

  • ReadOnly: Subscribe to updates from remote, no local edits
  • ReadWrite: Bidirectional synchronization
  • Periodic(Duration): Full sync every N seconds (fallback for network issues)

Sync Protocol

Using action/inbox mechanism:

DatabaseSyncAction Structure:

  • db_file_id: SHA256 identifier of database file
  • updates: Binary update payload (Yrs CRDT or redb operations)
  • state_vector: Current state hash for conflict detection
  • timestamp: Unix timestamp of update creation

Database Update Distribution Algorithm:

For each subscriber instance:

  1. Create DatabaseSyncAction with:

    • Database file ID
    • Binary updates (from CRDT or redb)
    • Computed state vector
    • Current timestamp
  2. POST to subscriber’s inbox:

    • Endpoint: https://cl-o.{subscriber_id_tag}/api/inbox
    • Send DatabaseSyncAction as JSON
  3. Subscriber’s ActionVerifierTask processes:

    • Extracts binary updates
    • Applies to local replica
    • Merges with any local changes

This pattern allows:

  • Real-time database synchronization
  • Conflict resolution via CRDTs
  • Federation of collaborative databases

See Also

Relationship Management

How following and connection relationships are established and managed across federated instances.

Following

When Alice follows Bob:

Algorithm: Follow User

Input: follower_id_tag, target_id_tag
Output: Result<()>

1. Create FLLW action token:
   - issuer: follower's id_tag
   - subject: target's id_tag
   - action type: "FLLW"
   - Sign with follower's private key (ES384)

2. Store action locally:
   - Record in metadata adapter
   - Marks that follower follows target

3. Send to target instance:
   - POST https://cl-o.{target_id_tag}/api/inbox
   - Include signed action token

4. Return success

Connection Establishment

Connections require mutual agreement:

Alice sends CONN to Bob
  ↓
Bob receives, stores CONN
  ↓
Bob sends CONN to Alice
  ↓
Alice receives, detects mutual CONN
  ↓
Connection established (both sides)

Connection vs Following

Aspect Following Connection
Direction One-way Bidirectional
Consent None required Mutual agreement
Trust level Low High (auto-approval)
Use case Content subscription Direct messaging, trusted sharing

Unfollowing

Unfollowing creates a new action that supersedes the previous follow:

Algorithm: Unfollow User

Input: follower_id_tag, target_id_tag
Output: Result<()>

1. Create FLLW action token with removed=true:
   - issuer: follower's id_tag
   - subject: target's id_tag
   - action type: "FLLW"
   - removed: true (indicates unfollow)
   - Sign with follower's private key (ES384)

2. Store action locally:
   - New FLLW action with removed=true
   - Supersedes previous FLLW action
   - Marks unfollow event

3. Send to target instance:
   - POST https://cl-o.{target_id_tag}/api/inbox
   - Include signed action token

4. Return success

The removed=true flag indicates this action cancels the previous follow.

Disconnecting

Similar to unfollowing, disconnection uses the removed=true flag:

Algorithm: Disconnect

Input: user_id_tag, target_id_tag
Output: Result<()>

1. Create CONN action token with removed=true
2. Store locally (marks disconnection)
3. Send to target instance
4. Target removes connection status

See Also

Security Considerations

Security model, spam prevention, and protection mechanisms for federated communication.

Trust Model

DNS-Based Trust

  • Domain ownership proves identity
  • TLS certificates prove server authenticity
  • Action signatures prove content authenticity

Progressive Trust

  • Initial federation is cautious
  • Trust builds through successful interactions
  • Users can block instances/users

Spam Prevention

Relationship-Based Action Acceptance

Algorithm: Should Accept Action

Input: action_type, issuer, local_user
Output: bool (accept or reject)

Decision logic by action_type:

1. "POST" → Only from followed or connected users
   - Check relationship between issuer and local_user
   - Accept if: relationship.following OR relationship.connected
   - Reject if: stranger with no relationship

2. "CMNT" / "REACT" → Always accept
   - Verification phase checks ownership of target content
   - Accept all, let verification handle validation

3. "CONN" / "FLLW" → Always accept
   - Relationship requests always received
   - User can block after receiving

4. Other action types → Reject

Federation Rate Limits

Default rate limits (hardcoded):

Limit Value Description
max_actions_per_instance_per_hour 1000 Per federated instance
max_actions_per_user_per_hour 100 Per remote user
max_concurrent_connections 100 Simultaneous federation connections
max_file_requests_per_hour 500 File sync requests

Blocklisting

Users can block instances or specific users:

Algorithm: Block Instance

Input: blocked_instance domain
Output: Result<()>

1. Add instance domain to user's blocklist
2. Store in metadata adapter
3. All future actions from this instance are rejected
4. Return success

User can later unblock by removing from blocklist.

Blocking Users

Individual users can be blocked without blocking the entire instance:

Algorithm: Block User

Input: blocked_user id_tag
Output: Result<()>

1. Add user id_tag to user's blocklist
2. Store in metadata adapter
3. All future actions from this user are rejected
4. Return success

Signature Verification

All federated actions must pass signature verification:

  1. JWT signature - Proves token signed by claimed issuer
  2. Key fetch - Public key retrieved from issuer’s instance
  3. Expiration check - Token not expired
  4. Audience check - Token intended for this instance

See Key Verification for details.

See Also

Operations Guide

Monitoring, best practices, and troubleshooting for federation.

Monitoring & Observability

Metrics

FederationMetrics Structure:

  • outbound_actions_sent (AtomicU64) - Successfully sent federation actions
  • outbound_actions_failed (AtomicU64) - Failed outbound deliveries
  • inbound_actions_received (AtomicU64) - Received federated actions
  • inbound_actions_rejected (AtomicU64) - Rejected actions (spam, invalid, blocked)
  • profiles_synced (AtomicU64) - Remote profiles cached/updated
  • files_synced (AtomicU64) - Attachment files downloaded
  • active_federation_connections (AtomicUsize) - Open federation connections

Logging

Sent Action Log:

  • instance: Target instance domain
  • action_type: Type of action (POST, FLLW, etc.)
  • action_id: Unique action identifier

Rejected Action Log:

  • instance: Source instance domain
  • action_type: Type of action
  • reason: Why rejected (spam, invalid sig, blocked, etc.)

Best Practices

For Instance Operators

  • Enable HTTPS: Always use TLS for federation
  • Monitor logs: Watch for spam or abuse
  • Set rate limits: Protect against DoS
  • Backup regularly: Federation doesn’t replace backups
  • Update promptly: Security patches are critical

For Developers

  • Verify signatures: Never trust unverified content
  • Check relationships: Enforce connection requirements
  • Handle failures: Network is unreliable
  • Cache wisely: Balance freshness vs performance
  • Test federation: Use multiple instances for testing

Troubleshooting

Actions Not Federating

Symptoms: Actions created locally but not received on remote instances

Checks:

  1. DNS resolution of target instance
  2. TLS certificate validity
  3. Firewall rules (port 443)
  4. Federation logs for errors

Common causes:

  • Target instance unreachable
  • Certificate expired
  • Rate limited

Signature Verification Failures

Symptoms: Inbound actions rejected with signature errors

Checks:

  1. Issuer’s public key is fetchable (GET /api/me)
  2. Key expiration
  3. Algorithm matches (ES384)

Common causes:

  • Key rotated without propagation
  • Clock skew between instances
  • Corrupted token in transit

File Sync Failures

Symptoms: Attachments not displaying, download errors

Checks:

  1. Content hash computation
  2. Blob adapter permissions
  3. Sufficient storage space

Common causes:

  • Hash mismatch (corrupted file)
  • Disk full
  • Permission denied

Profile Sync Issues

Symptoms: Stale profile data, missing avatars

Checks:

  1. Cache TTL (24h default)
  2. Remote instance reachable
  3. Profile endpoint returning valid data

Common causes:

  • Cache not invalidating
  • Remote instance down
  • Profile changed but not refreshed

See Also

Runtime Systems

Cloudillo’s internal runtime systems that coordinate task execution, manage concurrent processing, and enable real-time communication.

Core Systems

Subsections of Runtime Systems

Task Scheduler System

Cloudillo’s Task Scheduler is a sophisticated persistent task execution system that enables reliable, async background processing with dependency management, automatic retries, and cron-style scheduling. This system is critical for federation, file processing, and any operations that need to survive server restarts.

Why Task-Based Processing?

Simple async handlers cannot provide the guarantees needed for federation and file processing:

Problems with simple async:

  • ✗ Lost on server restart (no persistence)
  • ✗ No dependency ordering (file must finish before action)
  • ✗ No automatic retry on transient failures
  • ✗ No progress tracking or observability
  • ✗ No priority management for resource allocation

Task scheduler solutions:

  • ✅ Tasks survive server restarts (persisted via MetaAdapter)
  • ✅ Dependency resolution (DAG-based ordering)
  • ✅ Automatic retry with exponential backoff
  • ✅ Task lifecycle tracking (pending → running → complete/failed)
  • ✅ Priority-based execution via worker pool

Architecture Overview

Components

┌────────────────────────────────────────────┐
│ Task Scheduler                             │
│  ┌──────────────────────────────────────┐  │
│  │ Task Registry                        │  │
│  │  - TaskBuilder functions             │  │
│  │  - Task type mapping                 │  │
│  └──────────────────────────────────────┘  │
│  ┌──────────────────────────────────────┐  │
│  │ Task Queue                           │  │
│  │  - Pending tasks                     │  │
│  │  - Running tasks                     │  │
│  └──────────────────────────────────────┘  │
│  ┌──────────────────────────────────────┐  │
│  │ Dependency Tracker                   │  │
│  │  - DAG (Directed Acyclic Graph)      │  │
│  │  - Waiting tasks                     │  │
│  │  - Completion notifications          │  │
│  └──────────────────────────────────────┘  │
│  ┌──────────────────────────────────────┐  │
│  │ Retry Manager                        │  │
│  │  - Exponential backoff               │  │
│  │  - Retry counters                    │  │
│  │  - Backoff timers                    │  │
│  └──────────────────────────────────────┘  │
│  ┌──────────────────────────────────────┐  │
│  │ Cron Scheduler                       │  │
│  │  - Cron expressions                  │  │
│  │  - Next execution time               │  │
│  │  - Recurring task tracking           │  │
│  └──────────────────────────────────────┘  │
└────────────────────────────────────────────┘
                    ↓
┌────────────────────────────────────────────┐
│ MetaAdapter (Persistence)                  │
│  - Task metadata storage                   │
│  - Task status tracking                    │
│  - Dependency relationships                │
└────────────────────────────────────────────┘
                    ↓
┌────────────────────────────────────────────┐
│ Worker Pool (Execution)                    │
│  - High priority workers                   │
│  - Medium priority workers                 │
│  - Low priority workers                    │
└────────────────────────────────────────────┘

Task Trait System

All tasks implement the Task<S> trait:

#[async_trait]
pub trait Task<S: Clone>: Send + Sync + Debug {
    /// Task type identifier (e.g., "ActionCreator", "ImageResizer")
    fn kind() -> &'static str where Self: Sized;

    /// Build task from serialized context
    fn build(id: TaskId, context: &str) -> ClResult<Arc<dyn Task<S>>>
    where Self: Sized;

    /// Serialize task for persistence
    fn serialize(&self) -> String;

    /// Execute the task
    async fn run(&self, state: &S) -> ClResult<()>;

    /// Get task type of this instance
    fn kind_of(&self) -> &'static str;
}

Generic State Parameter

The S parameter is the application state (typically Arc<AppState>):

// Task receives app state for accessing adapters
async fn run(&self, state: &Arc<AppState>) -> ClResult<()> {
    // Access adapters through state
    state.meta_adapter.create_action(...).await?;
    state.blob_adapter.create_blob(...).await?;
    Ok(())
}

Task Serialization

Tasks must serialize to strings for persistence:

MyTask.serialize():
    return JSON.stringify(self)

MyTask.build(id, context):
    task = JSON.parse(context)
    return Arc(task)

Key Features

1. Dependency Tracking

Tasks can depend on other tasks forming a Directed Acyclic Graph (DAG):

# Create file processing task
file_task_id = scheduler
    .task(FileIdGeneratorTask(temp_path))
    .schedule()

# Create image resizing task that depends on file task
resize_task_id = scheduler
    .task(ImageResizerTask(file_id, "hd"))
    .depend_on([file_task_id])    # Wait for file task
    .schedule()

# Create action that depends on both
action_task_id = scheduler
    .task(ActionCreatorTask(action_data))
    .depend_on([file_task_id, resize_task_id])  # Wait for both
    .schedule()

Dependency Resolution:

FileIdGeneratorTask (no dependencies)
  ↓ completes
ImageResizerTask (depends on file task)
  ↓ completes
ActionCreatorTask (depends on both)
  ↓ executes

2. Exponential Backoff Retry

Tasks can automatically retry on failure with increasing delays:

retry_policy = RetryPolicy(
    min_wait: 10s
    max_wait: 3600s (1 hour)
    max_attempts: 5
)

scheduler
    .task(ActionDeliveryTask(action_token, recipient))
    .with_retry(retry_policy)
    .schedule()

Retry Schedule:

Attempt 1: Immediate
  ↓ fails
Attempt 2: 10s later (min wait)
  ↓ fails
Attempt 3: 20s later (2x backoff)
  ↓ fails
Attempt 4: 40s later (2x backoff)
  ↓ fails
Attempt 5: 80s later (capped at max)
  ↓ fails
Attempt 6: Not attempted (max retries reached)
  ↓
Task marked as failed

3. Cron Scheduling

Tasks can run on a schedule using cron expressions:

# Every day at 2:30 AM
scheduler
    .task(CleanupTask(temp_dir))
    .daily_at(2, 30)
    .schedule()

# Every hour at :00
scheduler
    .task(BackupTask(db_path))
    .hourly_at(0)
    .schedule()

# Every Monday at 9:00 AM
scheduler
    .task(WeeklyReportTask())
    .weekly_on(Weekday::Monday, 9, 0)
    .schedule()

# 1st of every month at midnight
scheduler
    .task(MonthlyTask())
    .monthly_on(1, 0, 0)
    .schedule()

4. Persistence

Tasks are persisted to MetaAdapter and survive server restarts:

start_scheduler(app):
    scheduler = Scheduler(app)

    # Load unfinished tasks from database
    pending_tasks = app.meta_adapter.list_pending_tasks()

    for task_meta in pending_tasks:
        # Rebuild task from serialized context
        task = TaskRegistry.build(
            task_meta.kind,
            task_meta.id,
            task_meta.context
        )

        # Re-queue task
        scheduler.enqueue(task_meta.id, task)

    # Start processing
    scheduler.start()

Built-in Task Types

Task Purpose Location
ActionCreatorTask Creates and signs action tokens (JWT/ES384) for federation cloudillo-action/src/task.rs
ActionVerifierTask Validates incoming federated action tokens (signature, permissions, attachments) cloudillo-action/src/task.rs
ActionDeliveryTask Delivers actions to remote instances with retry (POST to /api/inbox) cloudillo-action/src/task.rs
FileIdGeneratorTask Computes SHA256 content-addressed file IDs, moves files to BlobAdapter cloudillo-file/src/descriptor.rs
ImageResizerTask Generates image variants (tn/sd/md/hd/xd) using Lanczos3 filter cloudillo-file/src/image.rs
VideoTranscoderTask Transcodes video to web formats via FFmpeg cloudillo-file/src/video.rs
PdfProcessorTask Extracts text, metadata, and page thumbnails from PDFs cloudillo-file/src/pdf.rs
AudioExtractorTask Extracts ID3 tags, duration, and waveform previews cloudillo-file/src/audio.rs
EmailSenderTask Sends emails asynchronously via SMTP with retry cloudillo-email/src/task.rs
CertRenewalTask Automatic TLS certificate renewal via ACME (daily check) cloudillo-core/src/acme.rs
ProfileRefreshBatchTask Batch-refreshes cached remote profile data cloudillo-profile/src/sync.rs
TenantImageUpdaterTask Processes and stores tenant avatar/banner images cloudillo-profile/src/media.rs

Builder Pattern API

The scheduler uses a fluent builder API for task configuration:

retry = RetryPolicy(min: 10s, max: 3600s, max_attempts: 5)

scheduler
    .task(ActionDeliveryTask(token, recipient))
    .key(format("deliver-{}-{}", action_id, recipient))
    .depend_on([action_creator_task])
    .with_retry(retry)
    .priority(Priority::Medium)
    .schedule()

Integration with Worker Pool

CPU-intensive tasks should use the worker pool:

impl Task<App> for ImageResizerTask:
    run(state):
        # Load image (async I/O)
        image_data = state.blob_adapter.read_blob_buf(
            self.tn_id, self.source_file_id
        )

        # Resize image (CPU-intensive - use worker pool)
        resized = state.worker.run(closure:
            img = image.load_from_memory(image_data)
            resized = img.resize(
                self.target_width,
                self.target_height,
                FilterType::Lanczos3
            )
            # Encode to format
            buffer = resized.encode_to(self.format)
            return buffer
        )

        # Store variant (async I/O)
        variant_id = compute_file_id(resized)
        state.blob_adapter.create_blob_buf(
            self.tn_id, variant_id, resized
        )

See Also

Worker Pool Architecture

Cloudillo’s Worker Pool provides a three-tier priority thread pool for executing CPU-intensive and blocking operations. This system complements the async runtime by handling work that shouldn’t block async tasks, ensuring responsive performance even under heavy computational load.

Architecture Overview

Three-Tier Priority System

┌───────────────────────────────────────────┐
│ High Priority Queue (1 dedicated thread)  │
│  - User-facing operations                 │
│  - Time-sensitive tasks                   │
│  - Cryptographic signing                  │
└───────────────────────────────────────────┘
                ↓
┌───────────────────────────────────────────┐
│ Medium Priority Queue (2 threads)         │
│  - Processes: High + Medium               │
│  - Image resizing for posts               │
│  - File compression                       │
│  - Normal background work                 │
└───────────────────────────────────────────┘
                ↓
┌───────────────────────────────────────────┐
│ Low Priority Queue (1 thread)             │
│  - Processes: High + Medium + Low         │
│  - Batch operations                       │
│  - Cleanup tasks                          │
│  - Non-urgent processing                  │
└───────────────────────────────────────────┘

Default Configuration:

  • High priority: 1 dedicated thread
  • Medium priority: 2 threads (also process High)
  • Low priority: 1 thread (also processes High + Medium)
  • Total threads: 4

Priority Cascading

Higher priority workers process lower priority work when idle:

Thread 1 (High):     High tasks only
Threads 2-3 (Med):   High → Medium (if no High)
Thread 4 (Low):      High → Medium → Low (if none above)

Benefits:

  • High-priority work always has dedicated capacity
  • Lower-priority work still gets done when system is idle
  • Automatic load balancing

Core Components

WorkerPool Structure

pub struct WorkerPool {
    high_tx: flume::Sender<Job>,
    medium_tx: flume::Sender<Job>,
    low_tx: flume::Sender<Job>,
    handles: Vec<JoinHandle<()>>,
}

type Job = Box<dyn FnOnce() -> ClResult<Box<dyn Any + Send>> + Send>;

Initialization

Algorithm: Initialize Worker Pool

1. Create three unbounded message channels (high, medium, low)
2. Create thread handles vector
3. For each high-priority worker:
   - Clone high-priority receiver
   - Spawn worker thread (processes high priority only)
   - Store thread handle
4. For each medium-priority worker:
   - Clone high and medium receivers
   - Spawn cascading worker (processes high first, then medium)
   - Store thread handle
5. For each low-priority worker:
   - Clone all three receivers
   - Spawn cascading worker (processes high → medium → low)
   - Store thread handle
6. Return WorkerPool struct with senders and handles

Worker Thread

Basic Worker Spawning:

Algorithm: Spawn Single-Priority Worker

1. Spawn OS thread with receiver
2. Loop:
   a. Receive job from channel (blocking)
   b. Execute job
   c. Log success or error
   d. If channel disconnected, exit thread

Cascading Worker Spawning:

Algorithm: Spawn Multi-Priority Cascading Worker

1. Spawn OS thread with multiple receivers (high → medium → low)
2. Continuous loop:
   a. For each receiver in priority order:
      - Try to receive job (non-blocking)
      - If job available: execute, log result, restart loop
      - If empty: try next priority
      - If disconnected: exit thread
   b. If all queues empty: sleep 10ms and retry

This ensures high priority work is processed first, then medium, then low.

API Usage

Execute Method

Algorithm: Execute Work on Worker Pool

Input: priority level, CPU-bound function F
Output: Result<T>

1. Create async oneshot channel (tx, rx)
2. Wrap function F:
   - Execute F on worker thread
   - Send result through tx
   - Return success boxed as Any
3. Send wrapped job to appropriate priority queue:
   - High priority → high_tx
   - Medium priority → medium_tx
   - Low priority → low_tx
4. Await result through rx (async, doesn't block executor)
5. Return result to caller

This pattern allows async code to offload CPU work without blocking the async runtime.

Priority Enum

pub enum Priority {
    High,    // User-facing, time-sensitive
    Medium,  // Normal operations
    Low,     // Background, non-urgent
}

Metrics

The WorkerPoolMetrics struct tracks: queue depths per priority, total jobs completed/failed, and cumulative execution time.


Priority Guidelines

Priority Use For Characteristics
High Crypto during login, profile picture processing, real-time compression User actively waiting, <100ms typical, <100 jobs/sec
Medium Image variants for posts, file compression, data transforms Background but needed soon, 100ms-10s, default choice
Low Batch processing, cleanup, pre-generation, optimization No user waiting, can be delayed indefinitely

Integration Patterns

With Task Scheduler

Tasks often use the worker pool for CPU-intensive work following this pattern:

Algorithm: Task with Worker Pool Integration

1. Async I/O: Load data from blob storage
2. CPU work: Execute on worker pool:
   - Load image from memory
   - Resize using high-quality filter (Lanczos3)
   - Encode to desired format
   - Return resized buffer
3. Async I/O: Store result to blob storage

This separates I/O (async) from CPU (worker thread), allowing:
- The async runtime to continue processing other tasks during image resizing
- Background tasks (Low priority) not to starve user-facing operations
- Efficient resource utilization

Configuration

Sizing the Worker Pool

Default thread counts (balanced, recommended):

high = 1
medium = 2
low = 1
→ Suitable for mixed I/O and CPU workloads

Factors to consider:

  • CPU cores: More cores → more workers
  • Workload type: Heavy CPU → more workers
  • Memory: Each thread has stack (usually 2-8 MB)
  • Async runtime: Leave cores for Tokio

Environment Configuration

Configuration loading pattern:

high_workers = parse_env("WORKER_POOL_HIGH", default: 1)
medium_workers = parse_env("WORKER_POOL_MEDIUM", default: 2)
low_workers = parse_env("WORKER_POOL_LOW", default: 1)
worker_pool = WorkerPool::new(high_workers, medium_workers, low_workers)

Environment variables:

WORKER_POOL_HIGH=2    # 2 high-priority threads
WORKER_POOL_MEDIUM=4  # 4 medium-priority threads
WORKER_POOL_LOW=2     # 2 low-priority threads

See Also

WebSocket Bus

Cloudillo’s WebSocket Bus provides real-time notifications for connected clients. It is a general-purpose broadcast channel — separate from the RTDB and CRDT WebSocket protocols.

WebSocket Endpoints Overview

Cloudillo provides three WebSocket endpoints for different real-time use cases:

Endpoint Purpose Protocol Documentation
/ws/bus Notifications and action forwarding JSON messages This page
/ws/rtdb/{file_id} Real-time database subscriptions Binary protocol RTDB Protocol
/ws/crdt/{doc_id} Collaborative document editing (Yjs) Binary sync protocol CRDT Protocol

Connection

URL: wss://cl-o.{domain}/ws/bus

Authentication: Required — the client is authenticated at connection time via the Axum auth middleware. Once connected, the client is registered in the BroadcastManager which tracks active connections per tenant.

Message Protocol

All messages use JSON format. The bus uses a generic, flexible message structure rather than strict typed enums.

BusMessage (Client ↔ Server)

{
  "id": "msg_001",
  "cmd": "ping",
  "data": {}
}
Field Type Description
id string Unique message ID (for matching responses)
cmd string Command type
data object JSON payload (command-specific)

Client → Server Messages

ping

Keep connection alive:

{ "id": "1", "cmd": "ping", "data": {} }

Server responds with:

{ "id": "1", "cmd": "ack", "data": { "status": "pong" } }

Other Commands

Any other cmd value is accepted and acknowledged:

{ "id": "2", "cmd": "ack", "data": { "status": "ok" } }

The bus is designed as a generic transport — application-specific command handling can be added without changing the protocol.

Server → Client Messages (Broadcasts)

The server pushes broadcast messages to connected clients via the BroadcastManager. Messages are delivered to all connections for a tenant (send_to_tenant) or to a specific user (send_to_user).

ACTION

Sent when an action is created locally or received via federation:

{ "id": "evt_001", "cmd": "ACTION", "data": { "actionId": "a1~xyz789...", "type": "POST", "issuer": "alice.example.com" } }

This is the primary real-time event — it notifies connected clients about new posts, comments, reactions, messages, connection requests, and all other action types.

notification

Sent by the action DSL hook system for targeted notifications:

{ "id": "evt_002", "cmd": "notification", "data": { ... } }

FILE_ID_GENERATED

Sent when a file upload completes and the content-addressed file ID is computed:

{ "id": "evt_003", "cmd": "FILE_ID_GENERATED", "data": { "fileId": "f1~Qo2E3G8TJZ..." } }

Architecture

BroadcastManager

The BroadcastManager maintains a registry of connected users per tenant, using tokio::broadcast channels for message distribution:

  • Multi-tenant isolation — each tenant’s broadcasts are separate
  • Per-user targeting — messages can be sent to all tenant users or a specific user
  • Configurable buffer — default 128 messages per connection
  • Automatic cleanup — connections are removed when the WebSocket closes

Action Forwarding

The bus integrates with the action processing pipeline:

Outbound (local action creation):

  1. User creates action via POST /api/actions
  2. ActionCreatorTask completes
  3. Action broadcast to all connected sessions for the user’s tenant

Inbound (federated action reception):

  1. Remote action arrives at POST /api/inbox
  2. ActionVerifierTask verifies and stores the action
  3. Action broadcast to all connected sessions for the target tenant

See Also

Rate Limiting

Overview

Cloudillo implements rate limiting using GCRA (Generic Cell Rate Algorithm) with hierarchical address grouping and dual-tier limits to protect against abuse, DDoS attacks, and credential stuffing.

Hierarchical Address Levels

Requests are rate-limited at multiple network levels simultaneously. All levels must pass for a request to succeed.

IPv4 Levels

Level Mask Description Example
Individual /32 Single IP address 192.168.1.100
Network /24 Class C network (256 IPs) 192.168.1.0/24

IPv6 Levels

Level Mask Description Example
Subnet /64 Single subnet 2001:db8:1234:5678::/64
Provider /48 ISP allocation 2001:db8:1234::/48
Multi-Level Protection

A single abusive IP can’t overwhelm the system, but neither can a botnet spread across a /24 network. Both individual and network-level limits apply.

Dual-Tier Limits

Each address level has two tiers of limits:

Tier Period Purpose
Short-term Per-second Burst protection
Long-term Per-hour Sustained abuse protection

Both tiers must pass.

Default Rate Limits

Category IPv4 Individual IPv4 Network (/24) IPv6 Subnet (/64) IPv6 Provider (/48)
Auth 5/s (burst 10), 60/h 15/s (burst 30), 200/h 5/s (burst 10), 60/h 15/s (burst 30), 200/h
Federation 100/s (burst 200), 1000/h 500/s (burst 750), 5000/h 100/s (burst 200), 1000/h 500/s (burst 750), 5000/h
General 300/s (burst 500), 5000/h 600/s (burst 1000), 50000/h 300/s (burst 500), 5000/h 600/s (burst 1000), 50000/h
WebSocket 100/s (burst 200), 1000/h 100/s (burst 200), 1000/h 100/s (burst 200), 1000/h 100/s (burst 200), 1000/h
  • Auth: Login, registration, password reset
  • Federation: Inbox, sync, token exchange
  • General: Profile viewing, action listing, file access
  • WebSocket: Notification bus, CRDT, RTDB connections

Proof-of-Work Protection

CONN (connection request) actions require proof-of-work when violations are detected from an IP address. Violations (failed signature, duplicate pending, rejected CONN) increment a counter tracked at both individual IP and network range levels. The counter decays by 1 every hour.

When PoW is required, the action token must end with N A characters (where N = counter value). For example, counter=2 requires the token to end with AA. The server responds with HTTP 428 (Precondition Required) when PoW is insufficient.

Parameter Default Description
max_counter 10 Maximum requirement (10 ‘A’ characters)
decay_interval_secs 3600 Counter decay interval (1 hour)
max_individual_entries 50,000 LRU cache size for individual IPs
max_network_entries 10,000 LRU cache size for network ranges

HTTP Response Headers

Rate-limited responses include informative headers:

HTTP/1.1 429 Too Many Requests
Retry-After: 60
X-RateLimit-Limit: 30
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1735003600
X-RateLimit-Level: ipv4_individual
Header Description
Retry-After Seconds until request can be retried
X-RateLimit-Limit Maximum requests per period
X-RateLimit-Remaining Requests remaining in period
X-RateLimit-Reset Unix timestamp when limit resets
X-RateLimit-Level Which address level triggered the limit

Configuration

Rate limits can be configured per-tenant via settings:

{
  "rateLimit.auth.ipv4Individual.shortRps": 2,
  "rateLimit.auth.ipv4Individual.shortBurst": 5,
  "rateLimit.auth.ipv4Individual.longRph": 30,
  "rateLimit.auth.ipv4Individual.longBurst": 30
}

See Also

Push Notifications

Overview

Cloudillo implements Web Push notifications using the VAPID (Voluntary Application Server Identification) protocol. Push notifications are sent when users receive actions while offline or not connected via WebSocket.

Web Push Standards

The implementation follows these RFCs:

RFC Title Purpose
RFC 8292 VAPID for Web Push Server identification
RFC 8188 Encrypted Content-Encoding for HTTP Payload encryption
RFC 8291 Message Encryption for Web Push End-to-end encryption

VAPID Keys

Each tenant has a VAPID key pair for authenticating with push services. The private key is stored in the database; the public key is shared with clients. Keys are automatically generated on first request if they don’t exist.

Subscription Flow

sequenceDiagram
    participant C as Client
    participant SW as Service Worker
    participant S as Cloudillo Server
    participant PS as Push Service

    C->>S: GET /api/auth/vapid
    S-->>C: {vapidPublicKey: "BM5..."}

    C->>SW: pushManager.subscribe({userVisibleOnly: true, applicationServerKey})
    SW->>PS: Subscribe request
    PS-->>SW: PushSubscription
    SW-->>C: PushSubscription

    C->>S: POST /api/notifications/subscription {subscription}
    S-->>C: {id: 12345}

Push Delivery

When an action is received for an offline user:

sequenceDiagram
    participant A as Action Sender
    participant S as Cloudillo Server
    participant PS as Push Service
    participant B as User's Browser

    A->>S: POST /api/inbox {action}
    S->>S: Process action
    S->>S: Check if recipient is online

    alt User is online (WebSocket connected)
        S->>B: WebSocket message
    else User is offline
        S->>S: Load push subscriptions
        S->>S: Encrypt payload with user's public key
        S->>PS: POST {encrypted payload}
        PS->>B: Push notification
        B->>B: Display notification
    end

Notification Types

Users can configure which notification types they receive:

Setting Default Description
notify.push.message true New direct messages
notify.push.mention true Mentioned in content
notify.push.reaction false Reactions to your content
notify.push.connection true Connection requests
notify.push.follow true New followers

Settings are stored per-user and checked before sending notifications.

Encryption

Push notification payloads are encrypted end-to-end using the client’s P-256 key pair and a shared secret. The push service cannot read the encrypted data.

Parameter Description
p256dh Client’s P-256 public key
auth 16-byte authentication secret
content-encoding aes128gcm

Payload Structure

{
  "type": "action",
  "actionType": "MSG",
  "issuer": "alice@example.com",
  "preview": "New message from Alice",
  "actionId": "a1~xyz789..."
}
Field Description
type Notification type (action, system)
actionType Action token type (MSG, CONN, FLLW)
issuer Action issuer identity
preview Short preview text
actionId Action ID for deep linking

Subscription Management

Users can have multiple push subscriptions (one per device/browser). Each subscription has a unique ID, and all active subscriptions receive notifications.

Push subscriptions can expire when the browser reports expiration, the push service responds with HTTP 410 Gone, or the user manually unsubscribes. Expired subscriptions are removed automatically.

Error Handling

Push Service Response Action
201 Created Success
400 Bad Request Log error, don’t retry
410 Gone Remove subscription
429 Too Many Requests Retry with backoff
5xx Server Error Retry with backoff

See Also

API Documentation

Welcome to the Cloudillo API documentation for application developers. This guide will help you build applications on top of the Cloudillo decentralized collaboration platform.

What is Cloudillo?

Cloudillo is an open-source, decentralized collaboration platform that enables users to maintain control over their data while seamlessly collaborating with others. Built on DNS-based identity and cryptographically signed action tokens, Cloudillo allows users to self-host, use community servers, or choose third-party providers without vendor lock-in.

For Application Developers

Cloudillo provides a comprehensive set of APIs and client libraries for building:

  • Collaborative applications with real-time synchronization
  • Social features using action tokens (posts, comments, reactions)
  • Rich content editors with CRDT-based conflict-free editing
  • File management with automatic image variants
  • Real-time databases with Firebase-like APIs
  • Microfrontend applications that integrate with the Cloudillo shell

API Overview

Client Libraries (TypeScript/JavaScript)

REST API

The Cloudillo server provides a comprehensive REST API for:

  • Authentication - Login, registration, token management
  • Profiles - User and community profiles
  • Actions - Social interactions (posts, comments, reactions)
  • Files - File upload, download, and management
  • Settings - User preferences and configuration
  • References - Bookmarks and shortcuts

WebSocket API

Real-time features are provided via WebSocket connections:

  • Message Bus - Pub/sub notifications and presence
  • RTDB - Real-time database synchronization
  • CRDT - Collaborative document editing

Getting Started

New to Cloudillo development? Start here:

  1. Getting Started Guide - Create your first Cloudillo app
  2. Authentication Guide - Understand token-based auth
  3. Microfrontend Integration - Build apps for the Cloudillo shell

Developer Guides

Practical guides to help you build Cloudillo applications:

Key Concepts

DNS-Based Identity

Every user in Cloudillo has an identity tag (idTag) based on a domain name (e.g., alice@example.com). This decouples identity from storage location, allowing users to migrate their data between providers while maintaining their identity.

Action Tokens

Actions are cryptographically signed events that represent user activities:

  • POST - Creating posts and content
  • CMNT - Adding comments
  • REACT - Reactions (e.g., LOVE)
  • FLLW - Following users
  • CONN - User connections
  • FSHR - File sharing

See Actions API for details.

Multi-Tenancy

Cloudillo is designed for multi-tenant deployments. Every request includes a tenant ID (tnId) that isolates data between tenants. Application developers typically don’t need to manage this directly - it’s handled by the client libraries.

Real-Time Collaboration

Cloudillo provides three levels of real-time collaboration:

  1. CRDT - Conflict-free collaborative editing using Yjs
  2. RTDB - Real-time database with structured queries
  3. Message Bus - Pub/sub for notifications and presence

Architecture

┌─────────────────────────────────────────┐
│         Your Application                │
│  (React, Vue, vanilla JS, etc.)         │
├─────────────────────────────────────────┤
│     Cloudillo Client Libraries          │
│  @cloudillo/core, @cloudillo/react      │
├─────────────────────────────────────────┤
│         REST + WebSocket APIs            │
├─────────────────────────────────────────┤
│       Cloudillo Server (Rust)            │
├─────────────────────────────────────────┤
│    Pluggable Storage Adapters            │
│  (Database, Blob, Auth, CRDT, etc.)     │
└─────────────────────────────────────────┘

Example: Your First App

import { getAppBus, createApiClient, openYDoc } from '@cloudillo/core'
import * as Y from 'yjs'

// Get message bus singleton
const bus = getAppBus()

// Initialize your app
await bus.init('my-app')

// Access state via bus properties
console.log('User:', bus.idTag)
console.log('Tenant:', bus.tnId)
console.log('Roles:', bus.roles)

// Create an API client
const api = createApiClient({
  idTag: bus.idTag!,
  authToken: bus.accessToken
})

// Fetch the user's profile
const profile = await api.profiles.getOwn()

// Open a collaborative document
const yDoc = new Y.Doc()
const { provider } = await openYDoc(yDoc, 'owner:my-document-id')

// Use the CRDT
const yText = yDoc.getText('content')
yText.insert(0, 'Hello, Cloudillo!')

Client Libraries

REST API

Developer Guides

Advanced Topics

Support

License

Cloudillo is open source software licensed under the MIT License.

Subsections of API Documentation

Quick Start

Cloudillo applications are microfrontends that run inside the Cloudillo shell. The shell handles authentication, theming and navigation — your app just needs to initialize and start building.

import { getAppBus, createApiClient } from '@cloudillo/core'

const bus = getAppBus()
await bus.init('my-app')

// You now have: bus.idTag, bus.accessToken, bus.tnId, bus.roles, bus.darkMode

You never implement registration, login or token management — the shell provides all of that through the message bus.

Choose your path

Pick the type of app you’re building and follow the links to the most relevant guides:

Building a… Start with Then explore
Social app (posts, comments, reactions, follows) Getting Started Actions API, Common Patterns
Collaborative editor (real-time document editing) Getting Started CRDT Guide, WebSocket API
Data-driven app (structured collections, queries) Getting Started RTDB Guide, WebSocket API
React UI app (hooks, components) Getting Started React Library, React Components
File management (upload, share, variants) Common Patterns Files API, Shares API

Key concepts at a glance

  • idTag — DNS-based user identity (e.g. alice.cloudillo.net). Decouples identity from where data is stored.
  • Action tokens — Cryptographically signed events for social interactions: POST, CMNT, REACT, FLLW, CONN, FSHR.
  • Message bus — Communication channel between your app and the Cloudillo shell. Accessed via getAppBus().
  • Tenant — Data isolation boundary. Handled automatically by the client libraries — you rarely interact with it directly.

For more detail, see Key Concepts in the API overview.

Next steps

Getting Started

This guide will walk you through creating your first Cloudillo application.

Prerequisites

  • Node.js 18+ and pnpm installed
  • Basic knowledge of TypeScript/JavaScript
  • Familiarity with React (optional, for UI apps)

Installation

For Standalone Apps

pnpm add @cloudillo/core

For React Apps

pnpm add @cloudillo/core @cloudillo/react

For Real-Time Database

pnpm add @cloudillo/rtdb

For Collaborative Editing

pnpm add @cloudillo/core yjs y-websocket

Your First App: Hello Cloudillo

Let’s create a simple app that displays the current user’s profile.

Step 1: Initialize the App

Create src/index.ts:

import { getAppBus } from '@cloudillo/core'

async function main() {
  // Get the singleton message bus
  const bus = getAppBus()

  // Initialize with your app name
  await bus.init('hello-cloudillo')

  console.log('Initialized successfully!')
  console.log('Access Token:', bus.accessToken)
  console.log('User ID:', bus.idTag)
  console.log('Tenant ID:', bus.tnId)
  console.log('Roles:', bus.roles)
}

main().catch(console.error)

Step 2: Fetch User Profile

import { getAppBus, createApiClient } from '@cloudillo/core'

async function main() {
  const bus = getAppBus()
  await bus.init('hello-cloudillo')

  // Create an API client
  const api = createApiClient({
    idTag: bus.idTag!,
    authToken: bus.accessToken
  })

  // Fetch the current user's profile
  const profile = await api.profiles.getOwn()

  console.log('Profile:', profile)
  console.log('Name:', profile.name)
  console.log('ID Tag:', profile.idTag)
  console.log('Profile Picture:', profile.profilePic)
}

main().catch(console.error)

Step 3: Create a Post

import { getAppBus, createApiClient } from '@cloudillo/core'

async function main() {
  const bus = getAppBus()
  await bus.init('hello-cloudillo')

  const api = createApiClient({
    idTag: bus.idTag!,
    authToken: bus.accessToken
  })

  // Create a new post
  const newPost = await api.actions.create({
    type: 'POST',
    content: {
      text: 'Hello from my first Cloudillo app!',
      title: 'My First Post'
    }
  })

  console.log('Post created:', newPost)
}

main().catch(console.error)

React Example

For React applications, use the provided hooks:

import React from 'react'
import { useCloudillo, useAuth, useApi } from '@cloudillo/react'

function App() {
  // useCloudillo handles initialization
  const { token } = useCloudillo('hello-cloudillo')

  if (!token) return <div>Loading...</div>

  return <Profile />
}

function Profile() {
  const [auth] = useAuth()  // Returns tuple [auth, setAuth]
  const { api } = useApi()  // Returns { api, authenticated, setIdTag }
  const [profile, setProfile] = React.useState(null)

  React.useEffect(() => {
    if (!api) return
    api.profiles.getOwn().then(setProfile)
  }, [api])

  if (!api) return <div>No API client</div>
  if (!profile) return <div>Loading...</div>

  return (
    <div>
      <h1>Welcome, {profile.name}!</h1>
      <p>ID: {auth?.idTag}</p>
      {profile.profilePic && (
        <img src={profile.profilePic} alt="Profile" />
      )}
    </div>
  )
}

export default App

Real-Time Database Example

Here’s how to use the real-time database:

import { getAppBus, getRtdbUrl } from '@cloudillo/core'
import { RtdbClient } from '@cloudillo/rtdb'

async function main() {
  const bus = getAppBus()
  await bus.init('rtdb-example')

  // Create RTDB client
  const rtdb = new RtdbClient({
    dbId: 'my-database-file-id',
    auth: { getToken: () => bus.accessToken },
    serverUrl: getRtdbUrl(bus.idTag!, 'my-database-file-id', bus.accessToken!)
  })

  // Connect to the database
  await rtdb.connect()

  // Get a collection reference
  const todos = rtdb.collection('todos')

  // Subscribe to real-time updates
  todos.onSnapshot((snapshot) => {
    console.log('Todos updated:', snapshot.docs.map(doc => doc.data()))
  })

  // Create a document using batch
  const batch = rtdb.batch()
  batch.create(todos, {
    title: 'Learn Cloudillo',
    completed: false,
    createdAt: Date.now()
  })
  await batch.commit()

  // Query documents
  const incompleteTodos = await todos.query({
    filter: { equals: { completed: false } },
    sort: [{ field: 'createdAt', ascending: false }]
  })

  console.log('Incomplete todos:', incompleteTodos)
}

main().catch(console.error)

Collaborative Editing Example

Create a collaborative text editor:

import { getAppBus, openYDoc } from '@cloudillo/core'
import * as Y from 'yjs'

async function main() {
  const bus = getAppBus()
  await bus.init('collab-editor')

  // Create a Yjs document
  const yDoc = new Y.Doc()

  // Open collaborative document (format: ownerTag:documentId)
  const { provider } = await openYDoc(yDoc, 'owner.cloudillo.net:my-doc-id')

  // Get shared text
  const yText = yDoc.getText('content')

  // Listen for changes
  yText.observe(() => {
    console.log('Text changed:', yText.toString())
  })

  // Insert text
  yText.insert(0, 'Hello, collaborative world!')

  // See awareness (other users' cursors/selections)
  provider.awareness.on('change', () => {
    const states = provider.awareness.getStates()
    console.log('Connected users:', states.size)
  })
}

main().catch(console.error)

Microfrontend Integration

If you’re building an app to run inside the Cloudillo shell:

import { getAppBus, createApiClient } from '@cloudillo/core'

async function main() {
  // Get the singleton message bus
  const bus = getAppBus()

  // init() automatically handles the shell protocol
  await bus.init('my-microfrontend')

  // Create an API client
  const api = createApiClient({
    idTag: bus.idTag!,
    authToken: bus.accessToken
  })

  // The shell provides via bus properties:
  // - bus.idTag (user's identity)
  // - bus.tnId (tenant ID)
  // - bus.roles (user roles)
  // - bus.darkMode (theme preference)
  // - bus.access ('read' or 'write')

  // Your app logic here...
}

main().catch(console.error)

Error Handling

All API calls can throw errors. Handle them appropriately:

import { getAppBus, createApiClient } from '@cloudillo/core'

const bus = getAppBus()
await bus.init('my-app')

try {
  const api = createApiClient({
    idTag: bus.idTag!,
    authToken: bus.accessToken
  })
  const profile = await api.profiles.getOwn()
} catch (error) {
  if (error instanceof Error) {
    console.error('Error:', error.message)
  }
}

Next Steps

Now that you’ve created your first Cloudillo app, explore more features:

Common Patterns

Handling Dark Mode

import { getAppBus } from '@cloudillo/core'

const bus = getAppBus()
await bus.init('my-app')

// Check dark mode preference
if (bus.darkMode) {
  document.body.classList.add('dark-theme')
}

Using Query Parameters

import { getAppBus, createApiClient } from '@cloudillo/core'

const bus = getAppBus()
await bus.init('my-app')

const api = createApiClient({
  idTag: bus.idTag!,
  authToken: bus.accessToken
})

// List actions with filters
const posts = await api.actions.list({
  type: 'POST',
  status: 'A', // Active
  limit: 20
})

Uploading Files

import { getAppBus, createApiClient } from '@cloudillo/core'

const bus = getAppBus()
await bus.init('my-app')

const api = createApiClient({
  idTag: bus.idTag!,
  authToken: bus.accessToken
})

// Upload a file using the uploadBlob helper
const result = await api.files.uploadBlob(
  'gallery',      // preset
  'image.png',    // fileName
  imageBlob,      // file data
  'image/png'     // contentType
)

console.log('Uploaded file:', result.fileId)

Build and Deploy

Development

Most apps run as microfrontends inside the Cloudillo shell. Use your preferred build tool (Rollup, Webpack, Vite):

# Using Rollup (like the example apps)
pnpm build

# Using Vite
vite build

Production

Deploy your built app to any static hosting:

# The built output goes to the shell's apps directory
cp -r dist /path/to/cloudillo/shell/public/apps/my-app

Troubleshooting

“Failed to initialize”

Make sure you’re either:

  1. Running inside the Cloudillo shell (as a microfrontend), or
  2. Providing authentication manually for standalone apps

“CORS errors”

Ensure your Cloudillo server is configured to allow requests from your app’s origin.

“WebSocket connection failed”

Check that:

  1. The WebSocket URL is correct (wss:// for production)
  2. The server is running and accessible
  3. Your authentication token is valid

Example Apps

Check out the example apps in the Cloudillo repository:

  • Quillo - Rich text editor with Quill
  • Prello - Presentation tool
  • Sheello - Spreadsheet application
  • Formillo - Form builder
  • Todollo - Task management

All use the same patterns described in this guide.

Authentication

Cloudillo uses JWT-based authentication with three types of tokens for different use cases. This guide explains how authentication works and how to use it in your applications.

Token Types

1. Access Token (Session Token)

The access token is your primary authentication credential for API requests.

Characteristics:

  • JWT signed with ES384 elliptic curve algorithm
  • Contains: tnId (tenant ID), idTag (user identity), roles, iat (issued at), exp (expiration)
  • Used for all authenticated API requests
  • Typically valid for the duration of a session

Usage:

// The access token is automatically managed by @cloudillo/core
import * as cloudillo from '@cloudillo/core'

const token = await cloudillo.init('my-app')
// Token is now stored and used automatically for all API calls

// Or access it directly
console.log(cloudillo.accessToken)

2. Action Token (Federation Token)

Action tokens are cryptographically signed events used for federation between Cloudillo instances.

Characteristics:

  • Represents a specific action (POST, CMNT, REACT, etc.)
  • Signed by the issuer’s private key
  • Can be verified by anyone with the issuer’s public key
  • Enables trust-free federation

Usage:

// Action tokens are created automatically when you post actions
const api = cloudillo.createApiClient()

const action = await api.actions.create({
  type: 'POST',
  content: { text: 'Hello, world!' }
})

// The server automatically signs the action with your key
// Other instances can verify it without trusting your server

3. Proxy Token (Cross-Instance Token)

Proxy tokens enable accessing resources on remote Cloudillo instances.

Characteristics:

  • Short-lived (typically 5 minutes)
  • Grants read access to specific resources
  • Used for federation scenarios

Usage:

const api = cloudillo.createApiClient()

// Get a proxy token for accessing a remote instance
const proxyToken = await api.auth.proxyToken.get()

// Use it to fetch resources from another instance
// (typically handled automatically by the client)

Authentication Flow

For Microfrontend Apps

When running inside the Cloudillo shell, authentication is handled automatically:

import * as cloudillo from '@cloudillo/core'

// The init() function receives the token from the shell via postMessage
const token = await cloudillo.init('my-app')

// All API calls now use this token automatically
const api = cloudillo.createApiClient()
const profile = await api.profiles.getOwn() // Authenticated request

For Standalone Apps

For standalone applications, you need to handle authentication manually:

import * as cloudillo from '@cloudillo/core'

// Option 1: Manual token management
cloudillo.accessToken = 'your-jwt-token-here'
const api = cloudillo.createApiClient()

// Option 2: Login flow
const response = await fetch('https://your-server.com/auth/login', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    idTag: 'alice@example.com',
    password: 'secret123'
  })
})

const { token, tnId, idTag, name, roles } = await response.json()

cloudillo.accessToken = token
cloudillo.tnId = tnId
cloudillo.idTag = idTag
cloudillo.roles = roles

Authentication Endpoints

POST /auth/register

Register a new user account.

Request:

{
  "idTag": "alice@example.com",
  "password": "secure-password",
  "name": "Alice Johnson",
  "profilePic": "https://example.com/alice.jpg"
}

Response:

{
  "data": {
    "tnId": 12345,
    "idTag": "alice@example.com",
    "name": "Alice Johnson",
    "token": "eyJhbGc..."
  }
}

POST /auth/login

Authenticate and receive an access token.

Request:

{
  "idTag": "alice@example.com",
  "password": "secure-password"
}

Response:

{
  "data": {
    "tnId": 12345,
    "idTag": "alice@example.com",
    "name": "Alice Johnson",
    "profilePic": "/file/b1~abcd1234",
    "roles": ["user", "admin"],
    "token": "eyJhbGc...",
    "settings": [
      ["theme", "dark"],
      ["language", "en"]
    ]
  }
}

POST /auth/logout

Invalidate the current session.

Request:

POST /auth/logout
Authorization: Bearer eyJhbGc...

Response:

{
  "data": "ok"
}

GET /auth/access-token

Exchange credentials for a scoped access token.

Query Parameters:

  • idTag - User identity
  • password - User password
  • roles - Optional: Requested roles (comma-separated)
  • ttl - Optional: Token lifetime in seconds

Response:

{
  "data": {
    "token": "eyJhbGc...",
    "expiresAt": 1735555555
  }
}

GET /auth/proxy-token

Get a proxy token for accessing remote resources.

Request:

GET /auth/proxy-token?target=bob@remote.com
Authorization: Bearer eyJhbGc...

Response:

{
  "data": {
    "token": "eyJhbGc...",
    "expiresAt": 1735555555
  }
}

Role-Based Access Control

Cloudillo supports role-based access control (RBAC) for fine-grained permissions.

Default Roles

  • user - Standard user permissions (read/write own data)
  • admin - Administrative permissions (manage server, users)
  • read - Read-only access
  • write - Write access to resources

Checking Roles

import * as cloudillo from '@cloudillo/core'

await cloudillo.init('my-app')

// Check if user has a specific role
if (cloudillo.roles?.includes('admin')) {
  console.log('User is an admin')
}

// Enable/disable features based on roles
const canModerate = cloudillo.roles?.includes('admin') ||
                    cloudillo.roles?.includes('moderator')

Requesting Specific Roles

// Request an access token with specific roles
const response = await fetch(
  '/auth/access-token?idTag=alice@example.com&password=secret&roles=user,admin'
)

Token Validation

All tokens are validated on the server for:

  1. Signature verification - Using ES384 algorithm
  2. Expiration check - Tokens expire after a set period
  3. Tenant isolation - Tokens are tied to specific tenants
  4. Role validation - Roles must be granted by the server

Security Best Practices

1. Token Storage

For web apps:

// Don't store tokens in localStorage (XSS vulnerable)
// ❌ localStorage.setItem('token', token)

// Use memory storage (managed by @cloudillo/core)
// ✅ cloudillo.accessToken = token

// Or use httpOnly cookies (server-side)

2. Token Renewal

// Implement token renewal before expiration
async function renewToken() {
  const api = cloudillo.createApiClient()

  try {
    const newToken = await api.auth.loginToken.get()
    cloudillo.accessToken = newToken.token
  } catch (error) {
    // Token expired, redirect to login
    window.location.href = '/login'
  }
}

// Renew every 50 minutes (if token lasts 60 minutes)
setInterval(renewToken, 50 * 60 * 1000)

3. Error Handling

import { FetchError } from '@cloudillo/core'

try {
  const api = cloudillo.createApiClient()
  const data = await api.profiles.getOwn()
} catch (error) {
  if (error instanceof FetchError) {
    if (error.code === 'E-AUTH-UNAUTH') {
      // Unauthorized - token expired or invalid
      window.location.href = '/login'
    } else if (error.code === 'E-AUTH-FORBID') {
      // Forbidden - insufficient permissions
      alert('You do not have permission to access this resource')
    }
  }
}

4. HTTPS Only

Always use HTTPS in production:

// ✅ Good
const api = cloudillo.createApiClient({
  baseUrl: 'https://api.cloudillo.com'
})

// ❌ Bad (only for local development)
const api = cloudillo.createApiClient({
  baseUrl: 'http://localhost:3000'
})

WebAuthn Support

Cloudillo supports WebAuthn for passwordless authentication.

Registration Flow

// 1. Get registration options from server
const optionsResponse = await fetch('/auth/webauthn/register/options', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ idTag: 'alice@example.com' })
})

const options = await optionsResponse.json()

// 2. Create credential with browser WebAuthn API
const credential = await navigator.credentials.create({
  publicKey: options
})

// 3. Verify credential with server
const verifyResponse = await fetch('/auth/webauthn/register/verify', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    idTag: 'alice@example.com',
    credential: {
      id: credential.id,
      rawId: Array.from(new Uint8Array(credential.rawId)),
      response: {
        clientDataJSON: Array.from(new Uint8Array(credential.response.clientDataJSON)),
        attestationObject: Array.from(new Uint8Array(credential.response.attestationObject))
      },
      type: credential.type
    }
  })
})

Authentication Flow

// 1. Get authentication options
const optionsResponse = await fetch('/auth/webauthn/login/options', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ idTag: 'alice@example.com' })
})

const options = await optionsResponse.json()

// 2. Get assertion with browser WebAuthn API
const assertion = await navigator.credentials.get({
  publicKey: options
})

// 3. Verify assertion with server
const verifyResponse = await fetch('/auth/webauthn/login/verify', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    idTag: 'alice@example.com',
    credential: {
      id: assertion.id,
      rawId: Array.from(new Uint8Array(assertion.rawId)),
      response: {
        clientDataJSON: Array.from(new Uint8Array(assertion.response.clientDataJSON)),
        authenticatorData: Array.from(new Uint8Array(assertion.response.authenticatorData)),
        signature: Array.from(new Uint8Array(assertion.response.signature)),
        userHandle: assertion.response.userHandle ?
          Array.from(new Uint8Array(assertion.response.userHandle)) : null
      },
      type: assertion.type
    }
  })
})

const { token } = await verifyResponse.json()
cloudillo.accessToken = token

Multi-Tenant Considerations

Every request in Cloudillo is scoped to a tenant:

// The tnId is automatically included in all requests
console.log('Tenant ID:', cloudillo.tnId)

// Tokens are tenant-specific and cannot access other tenants' data
// This is enforced at the database level for security

Common Authentication Scenarios

Scenario 1: User Login

async function login(idTag: string, password: string) {
  const response = await fetch('/auth/login', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ idTag, password })
  })

  if (!response.ok) {
    throw new Error('Login failed')
  }

  const data = await response.json()

  cloudillo.accessToken = data.data.token
  cloudillo.tnId = data.data.tnId
  cloudillo.idTag = data.data.idTag
  cloudillo.roles = data.data.roles

  return data.data
}

Scenario 2: Automatic Token Management

// @cloudillo/react handles this automatically
import { CloudilloProvider } from '@cloudillo/react'

function App() {
  return (
    <CloudilloProvider appName="my-app">
      {/* Token is managed automatically */}
      <YourApp />
    </CloudilloProvider>
  )
}

Scenario 3: Token Refresh

// Check token expiration and refresh
async function ensureAuthenticated() {
  const api = cloudillo.createApiClient()

  try {
    // Try to use the current token
    await api.profiles.getOwn()
  } catch (error) {
    if (error.code === 'E-AUTH-UNAUTH') {
      // Token expired, get a new one
      const { token } = await api.auth.loginToken.get()
      cloudillo.accessToken = token
    }
  }
}

Next Steps

Client Libraries

Overview

Cloudillo provides a comprehensive set of TypeScript/JavaScript client libraries for building applications. These libraries handle authentication, API communication, real-time synchronization, and React integration.

Available Libraries

@cloudillo/core

Core SDK for initialization and API access. This is the foundation for all Cloudillo applications.

Key Features:

  • App initialization via message bus (getAppBus())
  • Type-safe REST API client
  • URL helper functions
  • Storage, settings, and media picker APIs
  • Camera, document embedding, and CRDT cache management

Install:

pnpm add @cloudillo/core

@cloudillo/react

React hooks for Cloudillo integration.

Key Features:

  • useAuth() hook for authentication state (returns tuple)
  • useApi() hook for API client access (returns object)
  • useCloudillo() hook for microfrontend initialization
  • useCloudilloEditor() hook for CRDT editors
  • useInfiniteScroll() hook for pagination

Install:

pnpm add @cloudillo/react

@cloudillo/types

Shared TypeScript types with runtime validation using @symbion/runtype.

Key Features:

  • All data types (Profile, Action, File, etc.)
  • Runtime type validation
  • Compile-time type safety
  • Action type enums
  • Type guards and validators

Install:

pnpm add @cloudillo/types

@cloudillo/rtdb

Real-time database client with Firebase-like API.

Key Features:

  • Firebase-like API (familiar to developers)
  • Real-time subscriptions
  • Type-safe queries
  • Batch operations

Install:

pnpm add @cloudillo/rtdb

@cloudillo/crdt

CRDT document synchronization using Yjs with WebSocket transport.

Key Features:

  • openYDoc() for collaborative document editing
  • WebSocket-based Yjs synchronization
  • Offline caching and persistence support

Install:

pnpm add @cloudillo/crdt

@cloudillo/canvas-tools

React components and hooks for interactive object manipulation in SVG canvas applications.

Key Features:

  • Transform gizmo with rotation, scaling, and positioning
  • Rotation and pivot handle components
  • Gradient picker with presets
  • Coordinate and geometry utilities

Install:

pnpm add @cloudillo/canvas-tools

@cloudillo/fonts

Font metadata and pairing suggestions for typography systems.

Key Features:

  • Curated metadata for 22 Google Fonts
  • Pre-defined font pairings (heading + body combinations)
  • Helper functions for filtering by category and role
  • Full TypeScript support

Install:

pnpm add @cloudillo/fonts

Quick Comparison

Library Purpose Use When
@cloudillo/core Core functionality Every app
@cloudillo/react React integration Building React apps
@cloudillo/types Type definitions TypeScript projects
@cloudillo/crdt CRDT document sync Collaborative editing
@cloudillo/rtdb Real-time database Need structured real-time data
@cloudillo/canvas-tools SVG canvas manipulation Building drawing/design apps
@cloudillo/fonts Font metadata and pairings Typography selection UI

Installation

Minimal Setup (vanilla JS)

pnpm add @cloudillo/core

React Setup

pnpm add @cloudillo/core @cloudillo/react

Full Setup (with real-time features)

pnpm add @cloudillo/core @cloudillo/react @cloudillo/crdt @cloudillo/rtdb yjs y-websocket

Basic Usage

With @cloudillo/core

import { getAppBus, createApiClient } from '@cloudillo/core'

// Get the singleton message bus
const bus = getAppBus()

// Initialize (communicates with shell)
await bus.init('my-app')

// Access state
console.log(bus.idTag)       // User's identity
console.log(bus.accessToken) // JWT token

// Create API client
const api = createApiClient({
  idTag: bus.idTag!,
  authToken: bus.accessToken
})

// Make requests
const profile = await api.profiles.getOwn()

With @cloudillo/react

import { useCloudillo, useAuth, useApi } from '@cloudillo/react'

function App() {
  // useCloudillo handles initialization
  const { token, idTag, ownerTag, fileId } = useCloudillo('my-app')

  if (!token) return <div>Loading...</div>

  return <MyComponent />
}

function MyComponent() {
  const [auth] = useAuth()  // Returns tuple [auth, setAuth]
  const { api } = useApi()  // Returns { api, authenticated, setIdTag }

  if (!api) return <div>Loading...</div>

  // Use auth and api...
}

With @cloudillo/rtdb

import { RtdbClient } from '@cloudillo/rtdb'
import { getRtdbUrl } from '@cloudillo/core'

const rtdb = new RtdbClient({
  dbId: 'my-db-file-id',
  auth: { getToken: () => bus.accessToken },
  serverUrl: getRtdbUrl(bus.idTag!, 'my-db-file-id', bus.accessToken!)
})

await rtdb.connect()

const todos = rtdb.collection('todos')
todos.onSnapshot(snapshot => {
  console.log(snapshot.docs.map(doc => doc.data()))
})

Common Patterns

Pattern 1: Authentication Flow

import { getAppBus } from '@cloudillo/core'

// Get message bus singleton
const bus = getAppBus()

// Initialize (gets token from shell)
const state = await bus.init('my-app')

// Access auth state via bus properties
console.log(bus.idTag)       // User's identity
console.log(bus.tnId)        // Tenant ID
console.log(bus.roles)       // User roles
console.log(bus.accessToken) // JWT token
console.log(bus.access)      // 'read' or 'write'

Pattern 2: API Requests

import { getAppBus, createApiClient } from '@cloudillo/core'

const bus = getAppBus()
await bus.init('my-app')

const api = createApiClient({
  idTag: bus.idTag!,
  authToken: bus.accessToken
})

// GET requests
const profile = await api.profiles.getOwn()

// POST requests
const action = await api.actions.create({
  type: 'POST',
  content: { text: 'Hello!' }
})

// Query with parameters
const actions = await api.actions.list({
  type: 'POST',
  limit: 20
})

Pattern 3: Real-Time CRDT

import { getAppBus } from '@cloudillo/core'
import { openYDoc } from '@cloudillo/crdt'
import * as Y from 'yjs'

const bus = getAppBus()
await bus.init('my-app')

// Open CRDT document
const yDoc = new Y.Doc()
const { provider } = await openYDoc(yDoc, 'alice.cloudillo.net:doc-id')

// Use shared types
const yText = yDoc.getText('content')
yText.insert(0, 'Hello!')

// Listen for changes
yText.observe(() => {
  console.log('Text updated:', yText.toString())
})

Pattern 4: React Integration

import { useCloudillo, useApi, useAuth } from '@cloudillo/react'
import { useEffect, useState } from 'react'

function App() {
  const { token } = useCloudillo('my-app')

  if (!token) return <div>Initializing...</div>

  return <PostsList />
}

function PostsList() {
  const { api } = useApi()
  const [posts, setPosts] = useState([])

  useEffect(() => {
    if (!api) return

    api.actions.list({ type: 'POST', limit: 20 })
      .then(setPosts)
  }, [api])

  if (!api) return <div>Loading...</div>

  return (
    <div>
      {posts.map(post => (
        <div key={post.actionId}>{post.content?.text}</div>
      ))}
    </div>
  )
}

TypeScript Support

All libraries are written in TypeScript and provide full type definitions.

import type { Profile, Action, NewAction } from '@cloudillo/types'
import { createApiClient } from '@cloudillo/core'

// Types are automatically inferred
const api = createApiClient({ idTag: 'alice.cloudillo.net' })
const profile: Profile = await api.profiles.getOwn()

// Type-safe action creation
const newAction: NewAction = {
  type: 'POST',
  content: { text: 'Hello!' }
}

const created: Action = await api.actions.create(newAction)

Error Handling

API errors are thrown as standard errors with status codes:

import { createApiClient } from '@cloudillo/core'

try {
  const api = createApiClient({ idTag: 'alice.cloudillo.net' })
  const data = await api.profiles.getOwn()
} catch (error) {
  if (error instanceof Error) {
    console.error('Error:', error.message)
  }
}

Library Details

Explore each library in detail:

Next Steps

Subsections of Client Libraries

@cloudillo/core

Overview

The @cloudillo/core library is the core SDK for Cloudillo applications. It provides initialization via the message bus, API client creation, CRDT document support, and URL helpers.

Installation

pnpm add @cloudillo/core

Core Pattern: AppMessageBus

The main API is accessed through the getAppBus() singleton. This provides authentication state, storage, and shell communication for apps running in the Cloudillo shell.

import { getAppBus } from '@cloudillo/core'

// Get the singleton message bus
const bus = getAppBus()

// Initialize your app (communicates with shell)
const state = await bus.init('my-app')

// Access state via bus properties
console.log('Token:', bus.accessToken)
console.log('User ID:', bus.idTag)
console.log('Tenant ID:', bus.tnId)
console.log('Roles:', bus.roles)
console.log('Access level:', bus.access)
console.log('Dark mode:', bus.darkMode)

AppState Properties

After calling bus.init(), these properties are available on the bus:

Property Type Description
accessToken string | undefined Current JWT access token
idTag string | undefined User’s identity tag (e.g., “alice.cloudillo.net”)
tnId number | undefined Tenant ID
roles string[] | undefined User’s roles
access 'read' | 'write' Access level to current resource
darkMode boolean Dark mode preference
tokenLifetime number | undefined Token lifetime in seconds
displayName string | undefined Display name (for anonymous guests)
embedded boolean Whether the app is running as an embedded document
theme string | undefined Theme name
navState string | undefined Initial navigation state (from parent embed or shell)
ancestors string[] | undefined Ancestor file IDs in the embed chain

Storage API

The bus provides namespaced key-value storage:

const bus = getAppBus()
await bus.init('my-app')

// Store data
await bus.storage.set('my-app', 'settings', { theme: 'dark' })

// Retrieve data
const settings = await bus.storage.get<{ theme: string }>('my-app', 'settings')

// List keys
const keys = await bus.storage.list('my-app', 'user-')

// Delete data
await bus.storage.delete('my-app', 'settings')

// Clear namespace
await bus.storage.clear('my-app')

// Check quota
const quota = await bus.storage.quota('my-app')
console.log(`Used ${quota.used} of ${quota.limit} bytes`)

App Lifecycle Notifications

Notify the shell about your app’s loading progress:

const bus = getAppBus()
await bus.init('my-app')

// After auth init (called automatically by init())
bus.notifyReady('auth')

// After CRDT sync complete
bus.notifyReady('synced')

// When fully interactive
bus.notifyReady('ready')

Token Refresh

Request a fresh token when needed:

const bus = getAppBus()

// Manually refresh token
const newToken = await bus.refreshToken()

// Listen for token updates pushed from shell
bus.on('auth:token.push', (msg) => {
  console.log('Token updated:', bus.accessToken)
})

Error Notification

Notify the shell about errors in your app:

bus.notifyError(404, 'Document not found')

Event Handling

Register and unregister message handlers:

bus.on('auth:token.push', handler)
bus.off('auth:token.push', handler)

Media Picker

Open the shell’s media picker dialog:

const result = await bus.pickMedia({
  mediaType: 'image/*',       // MIME filter
  enableCrop: true,            // Enable crop UI
  cropAspects: ['16:9', '1:1'], // Allowed aspect ratios
  documentFileId: 'abc123',   // Context document
  title: 'Choose image'
})
// Returns: { fileId, fileName, contentType, dim?, visibility?, croppedVariantId? }

Document Picker

Open the shell’s document picker:

const result = await bus.pickDocument({
  fileTp: 'CRDT',                      // File type filter
  contentType: 'cloudillo/quillo',     // Content type filter
  sourceFileId: 'abc123',             // Source context
  title: 'Choose document'
})
// Returns: { fileId, fileName, contentType, fileTp?, appId? }

Camera Capture

Capture an image from the device camera:

const result = await bus.captureImage({
  facing: 'environment',  // 'user' or 'environment'
  maxResolution: 1920
})
// Returns: { imageData (base64), width, height }

Camera Preview

Open a camera preview with overlay support:

const session = await bus.openCamera({ facing: 'environment' })
// Preview frames
bus.previewFrames(session.sessionId, (frameData) => { /* ... */ })
// Add overlay shapes
bus.overlayShapes(session.sessionId, shapes)
// Wait for capture
const result = await session.result

Document Embedding

Request an embedded document view:

const { embedUrl, nonce, resId } = await bus.requestEmbed({
  targetFileId: 'abc123',
  targetContentType: 'cloudillo/quillo',
  sourceFileId: 'current-doc',
  access: 'read',
  navState: 'page=3'
})

Settings API

Access user settings through the bus:

await bus.settings.get('key')
await bus.settings.set('key', value)
const items = await bus.settings.list('prefix')

CRDT Cache Management

Manage offline CRDT caching:

const clientId = await bus.requestClientId(docId)
await bus.crdtCacheAppend(docId, update, clientId, clock)
const updates = await bus.crdtCacheRead(docId)
await bus.crdtCacheCompact(docId, state)

resetAppBus()

Reset the singleton message bus instance (useful for testing):

import { resetAppBus } from '@cloudillo/core'

resetAppBus()

API Client

createApiClient(opts: ApiClientOpts): ApiClient

Create a type-safe REST API client.

import { createApiClient } from '@cloudillo/core'

const api = createApiClient({
  idTag: 'alice.cloudillo.net',  // Required: target tenant
  authToken: 'jwt-token'         // Optional: authentication token
})

// Use the API
const profile = await api.profiles.getOwn()
const posts = await api.actions.list({ type: 'POST', limit: 20 })

Options:

interface ApiClientOpts {
  idTag: string       // Required: identity tag of the tenant
  authToken?: string  // Optional: JWT token for authentication
}
idTag is Required

Unlike the old documentation, idTag is required to create an API client. This specifies which Cloudillo instance to connect to.

Using with AppMessageBus

import { getAppBus, createApiClient } from '@cloudillo/core'

const bus = getAppBus()
const state = await bus.init('my-app')

const api = createApiClient({
  idTag: bus.idTag!,
  authToken: bus.accessToken
})

const files = await api.files.list({ limit: 20 })

CRDT Document Functions

openYDoc(yDoc: Y.Doc, docId: string): Promise<DocConnection>

Open a Yjs document for collaborative editing with WebSocket synchronization.

Info

openYDoc is exported from @cloudillo/crdt, not from @cloudillo/core.

import { getAppBus } from '@cloudillo/core'
import { openYDoc } from '@cloudillo/crdt'
import * as Y from 'yjs'

const bus = getAppBus()
await bus.init('my-app')

const yDoc = new Y.Doc()
const { yDoc: doc, provider } = await openYDoc(yDoc, 'alice.cloudillo.net:document-id')

// Use shared types
const yText = yDoc.getText('content')
yText.insert(0, 'Hello, collaborative world!')

// Listen for changes
yText.observe(() => {
  console.log('Text changed:', yText.toString())
})

// Access awareness (other users' cursors, selections)
provider.awareness.on('change', () => {
  const states = provider.awareness.getStates()
  console.log('Connected users:', states.size)
})

Parameters:

  • yDoc - The Yjs document to synchronize
  • docId - Document ID in format "targetTag:resourceId"

Returns:

interface DocConnection {
  yDoc: Y.Doc
  provider: WebsocketProvider
}

Error Handling:

  • Throws if no access token (must call init() first)
  • Throws if docId format is invalid
  • WebSocket close codes 4401/4403/4404 stop reconnection (auth/permission/not found errors)

URL Helper Functions

Build URLs for Cloudillo services:

getInstanceUrl(idTag: string): string

Build the base URL for a Cloudillo instance.

import { getInstanceUrl } from '@cloudillo/core'

const url = getInstanceUrl('alice.cloudillo.net')
// Returns: "https://cl-o.alice.cloudillo.net"

getApiUrl(idTag: string): string

Build the API base URL.

import { getApiUrl } from '@cloudillo/core'

const url = getApiUrl('alice.cloudillo.net')
// Returns: "https://cl-o.alice.cloudillo.net/api"

getFileUrl(idTag: string, fileId: string, variant?: string): string

Build URL for file access with optional variant.

import { getFileUrl } from '@cloudillo/core'

// Basic file URL
const url = getFileUrl('alice.cloudillo.net', 'file-123')
// Returns: "https://cl-o.alice.cloudillo.net/api/files/file-123"

// With variant
const thumbnailUrl = getFileUrl('alice.cloudillo.net', 'file-123', 'vis.tn')
// Returns: "https://cl-o.alice.cloudillo.net/api/files/file-123?variant=vis.tn"

getCrdtUrl(idTag: string): string

Build the CRDT WebSocket URL.

import { getCrdtUrl } from '@cloudillo/core'

const url = getCrdtUrl('alice.cloudillo.net')
// Returns: "wss://cl-o.alice.cloudillo.net/ws/crdt"

getRtdbUrl(idTag: string, fileId: string, token: string): string

Build the RTDB WebSocket URL with authentication.

import { getRtdbUrl } from '@cloudillo/core'

const url = getRtdbUrl('alice.cloudillo.net', 'db-file-id', 'jwt-token')
// Returns: "wss://cl-o.alice.cloudillo.net/ws/rtdb/db-file-id?token=jwt-token"

getMessageBusUrl(idTag: string, token: string): string

Build the Message Bus WebSocket URL.

import { getMessageBusUrl } from '@cloudillo/core'

const url = getMessageBusUrl('alice.cloudillo.net', 'jwt-token')
// Returns: "wss://cl-o.alice.cloudillo.net/ws/bus?token=jwt-token"

Image/Video Variant Helpers

getOptimalImageVariant(context, localVariants?): string

Get the optimal image variant for a display context.

import { getOptimalImageVariant } from '@cloudillo/core'

// For thumbnail display
const variant = getOptimalImageVariant('thumbnail')  // 'vis.tn'

// For preview
const variant = getOptimalImageVariant('preview')    // 'vis.sd'

// For fullscreen/lightbox
const variant = getOptimalImageVariant('fullscreen') // 'vis.hd'

// With available variants
const variant = getOptimalImageVariant('fullscreen', ['vis.sd', 'vis.hd', 'vis.xd'])
// Returns highest quality available: 'vis.xd'

Image variants: vis.tn (150px), vis.sd (640px), vis.md (1280px), vis.hd (1920px), vis.xd (original)

getOptimalVideoVariant(context, localVariants?): string

Get the optimal video variant for a display context.

import { getOptimalVideoVariant } from '@cloudillo/core'

const variant = getOptimalVideoVariant('preview')    // 'vid.sd'
const variant = getOptimalVideoVariant('fullscreen') // 'vid.hd'

Video variants: vid.sd, vid.md, vid.hd, vid.xd

getImageVariantForDisplaySize(width, height): string

Get optimal variant for a specific display size in pixels.

import { getImageVariantForDisplaySize } from '@cloudillo/core'

// For a 300x200 canvas at 2x zoom (600x400 screen pixels)
const variant = getImageVariantForDisplaySize(600, 400)
// Returns: 'vis.sd'

API Client Structure

The API client provides access to all REST endpoints organized by namespace:

const api = createApiClient({ idTag: 'alice.cloudillo.net', authToken: token })

// ============================================================================
// AUTH - Authentication & Authorization
// ============================================================================
await api.auth.login({ idTag, password })
await api.auth.logout()
await api.auth.loginInit()                          // Combined init (auth or QR+WebAuthn)
await api.auth.getLoginToken()
await api.auth.getAccessToken({ scope, lifetime })
await api.auth.getAccessTokenByRef(refId)           // Guest access via reference
await api.auth.getAccessTokenVia(via, scope)        // Scoped token via cross-document link
await api.auth.getAccessTokenByApiKey(apiKey)       // API key authentication
await api.auth.getProxyToken(targetIdTag)           // Federation proxy token
await api.auth.getVapidPublicKey()                  // Push notification key
await api.auth.changePassword({ oldPassword, newPassword })
await api.auth.setPassword({ refId, password })     // Set password via reset link
await api.auth.forgotPassword({ email })

// WebAuthn
await api.auth.listWebAuthnCredentials()
await api.auth.getWebAuthnRegChallenge()
await api.auth.registerWebAuthnCredential({ token, credential })
await api.auth.deleteWebAuthnCredential(credentialId)
await api.auth.getWebAuthnLoginChallenge()
await api.auth.webAuthnLogin({ token, credential })

// API Keys
await api.auth.listApiKeys()
await api.auth.createApiKey({ name, scopes })
await api.auth.deleteApiKey(keyId)

// QR Login
await api.auth.initQrLogin()                       // Start QR login session
await api.auth.getQrLoginStatus(sessionId, secret) // Poll for login status
await api.auth.getQrLoginDetails(sessionId)        // Get session details (mobile)
await api.auth.respondQrLogin(sessionId, data)     // Approve/deny from mobile

// ============================================================================
// PROFILE - Registration & Profile Creation
// ============================================================================
await api.profile.verify({ type, idTag, email })    // Check availability
await api.profile.register({ type, idTag, name, email, password })

// ============================================================================
// PROFILES - Profile Management
// ============================================================================
await api.profiles.getOwn()                         // GET /me
await api.profiles.getOwnFull()                     // GET /me/full
await api.profiles.getRemoteFull(idTag)             // GET full profile from remote server
await api.profiles.updateOwn({ name, bio })         // PATCH /me
await api.profiles.list({ q, type, role })          // List/search profiles
await api.profiles.get(idTag)                       // Get profile by idTag
await api.profiles.updateConnection(idTag, data)    // Update relationship
await api.profiles.adminUpdate(idTag, { status, roles }) // Admin operations

// ============================================================================
// ACTIONS - Social Interactions
// ============================================================================
await api.actions.list({ type, status, limit })     // List actions
await api.actions.listPaginated({ cursor, limit })  // With cursor pagination
await api.actions.create({ type, content, ... })    // Create action
await api.actions.get(actionId)                     // Get single action
await api.actions.update(actionId, patch)           // Update draft
await api.actions.delete(actionId)                  // Delete action
await api.actions.accept(actionId)                  // Accept (e.g., connection)
await api.actions.reject(actionId)                  // Reject
await api.actions.dismiss(actionId)                  // Dismiss notification
await api.actions.updateStat(actionId, stat)        // Update statistics
await api.actions.addReaction(actionId, { type })   // Add reaction
await api.actions.publish(actionId, { publishAt })  // Publish draft (optionally scheduled)
await api.actions.cancel(actionId)                  // Cancel scheduled (revert to draft)

// ============================================================================
// FILES - File Management
// ============================================================================
await api.files.list({ parentId, tag, limit })      // List files
await api.files.listPaginated({ cursor, limit })    // With cursor pagination
await api.files.create({ fileName, fileTp })        // Create metadata-only file
await api.files.uploadBlob(preset, name, data, contentType) // Upload file
await api.files.get(fileId)                         // Get file content
await api.files.getVariant(variantId)               // Get specific variant
await api.files.getDescriptor(fileId)               // Get file metadata
await api.files.update(fileId, { fileName })        // Update metadata
await api.files.delete(fileId)                      // Soft delete (to trash)
await api.files.permanentDelete(fileId)             // Permanent delete
await api.files.restore(fileId, parentId)           // Restore from trash
await api.files.updateUserData(fileId, { starred }) // Update user-specific data
await api.files.setStarred(fileId, true)            // Toggle starred
await api.files.setPinned(fileId, true)             // Toggle pinned
await api.files.addTag(fileId, 'tag')               // Add tag
await api.files.removeTag(fileId, 'tag')            // Remove tag
await api.files.duplicate(fileId, { fileName })     // Duplicate a CRDT/RTDB file
await api.files.listShares(fileId)                  // List share entries for file
await api.files.createShare(fileId, data)           // Create share entry
await api.files.deleteShare(fileId, shareId)        // Delete share entry

// ============================================================================
// SHARES - Share Entry Queries
// ============================================================================
await api.shares.listBySubject(subjectId, subjectType) // List shares by subject

// ============================================================================
// TRASH - Trash Management
// ============================================================================
await api.trash.list({ limit })                     // List trashed files
await api.trash.empty()                             // Empty trash permanently

// ============================================================================
// TAGS - Tag Management
// ============================================================================
await api.tags.list({ prefix, withCounts })         // List tags

// ============================================================================
// SETTINGS - User Settings
// ============================================================================
await api.settings.list({ prefix })                 // List settings
await api.settings.get(name)                        // Get setting value
await api.settings.update(name, { value })          // Update setting

// ============================================================================
// NOTIFICATIONS - Push Notifications
// ============================================================================
await api.notifications.subscribe({ subscription }) // Subscribe to push

// ============================================================================
// REFS - Share Links & References
// ============================================================================
await api.refs.list({ type, resourceId })           // List references
await api.refs.get(refId)                           // Get reference
await api.refs.create({ type, resourceId, access }) // Create share link
await api.refs.delete(refId)                        // Delete reference

// ============================================================================
// IDP - Identity Provider (End User)
// ============================================================================
await api.idp.getInfo(providerDomain)               // Get provider info
await api.idp.activate({ refId })                   // Activate identity

// ============================================================================
// IDP MANAGEMENT - Identity Provider Admin
// ============================================================================
await api.idpManagement.listIdentities({ q, status })
await api.idpManagement.createIdentity({ idTag, email, createApiKey })
await api.idpManagement.getIdentity(idTag)
await api.idpManagement.updateIdentity(idTag, { dyndns })
await api.idpManagement.deleteIdentity(idTag)
await api.idpManagement.listApiKeys(idTag)
await api.idpManagement.createApiKey({ idTag, name })
await api.idpManagement.deleteApiKey(keyId, idTag)

// ============================================================================
// COMMUNITIES - Community Management
// ============================================================================
await api.communities.create(idTag, { type, name, ownerIdTag })
await api.communities.verify({ type, idTag })       // Deprecated: use profile.verify

// ============================================================================
// ADMIN - System Administration
// ============================================================================
await api.admin.listTenants({ q, status, limit })   // List all tenants
await api.admin.sendPasswordReset(idTag)            // Send password reset
await api.admin.sendTestEmail(to)                   // Test SMTP config
await api.admin.listProxySites()                    // List proxy site configs
await api.admin.createProxySite(data)               // Create proxy site
await api.admin.getProxySite(siteId)                // Get proxy site
await api.admin.updateProxySite(siteId, data)       // Update proxy site
await api.admin.deleteProxySite(siteId)             // Delete proxy site
await api.admin.renewProxySiteCert(siteId)          // Renew proxy site TLS cert

Helper Functions

delay(ms: number): Promise<void>

Delay execution for a specified time.

import { delay } from '@cloudillo/core'

await delay(1000) // Wait 1 second

See Also

@cloudillo/react

Overview

React hooks and components for integrating Cloudillo into React applications.

Installation

pnpm add @cloudillo/react @cloudillo/core

Hooks

useAuth()

Access authentication state using Jotai atoms. Returns a tuple [auth, setAuth] for reading and updating auth state.

import { useAuth } from '@cloudillo/react'

function UserInfo() {
  const [auth, setAuth] = useAuth()

  if (!auth?.idTag) {
    return <div>Not authenticated</div>
  }

  return (
    <div>
      <p>User: {auth.idTag}</p>
      <p>Name: {auth.name}</p>
      <p>Tenant: {auth.tnId}</p>
      <p>Roles: {auth.roles?.join(', ')}</p>
      {auth.profilePic && <img src={auth.profilePic} alt="Profile" />}
    </div>
  )
}

Returns: [AuthState | undefined, SetAtom<AuthState>]

interface AuthState {
  tnId: number          // Tenant ID
  idTag?: string        // Identity tag (e.g., "alice.cloudillo.net")
  name?: string         // Display name
  profilePic?: string   // Profile picture ID
  roles?: string[]      // Community roles
  token?: string        // JWT access token
}

Usage patterns:

// Destructure the tuple
const [auth, setAuth] = useAuth()

// Check if user is authenticated
if (!auth?.idTag) {
  return <LoginPrompt />
}

// Check for specific role
if (auth?.roles?.includes('admin')) {
  return <AdminPanel />
}

// Update auth state (typically done by useCloudillo)
setAuth({
  tnId: 123,
  idTag: 'alice.cloudillo.net',
  token: 'jwt-token'
})

useApi()

Get a type-safe API client with automatic caching per idTag/token combination.

import { useApi } from '@cloudillo/react'
import { useEffect, useState } from 'react'

function PostsList() {
  const { api, authenticated, setIdTag } = useApi()
  const [posts, setPosts] = useState([])
  const [loading, setLoading] = useState(true)

  useEffect(() => {
    if (!api) return

    api.actions.list({ type: 'POST', limit: 20 })
      .then(setPosts)
      .finally(() => setLoading(false))
  }, [api])

  if (!api) return <div>No API client (no idTag)</div>
  if (!authenticated) return <div>Please log in</div>
  if (loading) return <div>Loading...</div>

  return (
    <div>
      {posts.map(post => (
        <div key={post.actionId}>{post.content}</div>
      ))}
    </div>
  )
}

Returns:

interface ApiHook {
  api: ApiClient | null    // Type-safe API client (null if no idTag)
  authenticated: boolean   // Whether user has a token
  setIdTag: (idTag: string) => void // Set idTag for login flow
}

The API client is the same as returned by createApiClient() from @cloudillo/core.

api can be null

api is null until an idTag is available (either from auth state or set via setIdTag). Always check for null before using.

useCloudillo()

Unified hook for microfrontend app initialization. Handles shell communication, authentication, and document context.

import { useCloudillo } from '@cloudillo/react'

function MyApp() {
  const { token, ownerTag, fileId, idTag, roles, access, displayName } = useCloudillo('my-app')

  if (!token) return <div>Loading...</div>

  return (
    <div>
      <p>Document: {fileId}</p>
      <p>Owner: {ownerTag}</p>
      <p>Access: {access}</p>
      <p>User: {idTag}</p>
    </div>
  )
}

This hook:

  1. Calls getAppBus().init(appName) on mount
  2. Parses ownerTag and fileId from location.hash (format: #ownerTag:fileId)
  3. Updates auth state via useAuth()
  4. Returns combined state from bus and URL

Returns:

interface UseCloudillo {
  token?: string           // Access token
  ownerTag: string         // Owner of the current document (from URL hash)
  fileId?: string          // Current file/document ID (from URL hash)
  idTag?: string           // Current user's identity tag
  tnId?: number            // Tenant ID
  roles?: string[]         // User's roles
  access?: 'read' | 'write' // Access level to current resource
  displayName?: string     // User's display name (for anonymous guests)
}

useCloudilloEditor()

Extended hook for CRDT-based collaborative document editing. Returns everything from useCloudillo() plus Yjs document and sync state.

import { useCloudilloEditor } from '@cloudillo/react'

function CollaborativeEditor() {
  const { token, yDoc, provider, synced, ownerTag, fileId, access } = useCloudilloEditor('quillo')

  if (!synced) return <LoadingSpinner />

  // Use yDoc with your editor binding (Quill, TipTap, Monaco, etc.)
  const yText = yDoc.getText('content')

  return <QuillEditor yText={yText} provider={provider} />
}

This hook:

  1. Calls useCloudillo(appName) internally
  2. Opens CRDT connection via openYDoc() from @cloudillo/crdt when token and docId are available
  3. Listens for sync events and notifies shell via bus.notifyReady('synced')
  4. Cleans up WebSocket provider on unmount

Returns:

interface UseCloudilloEditor extends UseCloudillo {
  yDoc: Y.Doc              // Yjs document instance
  provider?: WebsocketProvider // WebSocket sync provider
  synced: boolean          // Whether initial sync is complete
  error: { code: number; reason?: string } | null // Connection error state
}

useInfiniteScroll()

Cursor-based infinite scroll pagination with IntersectionObserver.

import { useInfiniteScroll } from '@cloudillo/react'

function FileList() {
  const { api } = useApi()

  const { items, isLoading, hasMore, sentinelRef, prepend, reset } = useInfiniteScroll({
    fetchPage: async (cursor, limit) => {
      const result = await api.files.listPaginated({ cursor, limit })
      return {
        items: result.data,
        nextCursor: result.cursorPagination?.nextCursor ?? null,
        hasMore: result.cursorPagination?.hasMore ?? false
      }
    },
    pageSize: 30,
    deps: [folderId, sortField] // Reset when these change
  })

  return (
    <div>
      {items.map(file => <FileCard key={file.fileId} file={file} />)}
      {/* Sentinel triggers loadMore when visible */}
      <div ref={sentinelRef} />
      {isLoading && <LoadingSpinner />}
    </div>
  )
}

Options:

interface UseInfiniteScrollOptions<T> {
  fetchPage: (cursor: string | null, limit: number) => Promise<{
    items: T[]
    nextCursor: string | null
    hasMore: boolean
  }>
  pageSize?: number           // Default: 20
  deps?: React.DependencyList // Reset when these change
  enabled?: boolean           // Default: true
}

Returns:

interface UseInfiniteScrollReturn<T> {
  items: T[]                    // All loaded items
  isLoading: boolean            // Initial load in progress
  isLoadingMore: boolean        // Loading more pages
  error: Error | null           // Last fetch error
  hasMore: boolean              // More items available
  loadMore: () => void          // Manually load next page
  reset: () => void             // Reset and reload
  prepend: (items: T[]) => void // Add items to start (real-time)
  sentinelRef: RefObject<HTMLDivElement> // Attach to trigger element
}

useDebouncedValue()

Debounce a rapidly changing value:

import { useDebouncedValue } from '@cloudillo/react'

function SearchField() {
  const [query, setQuery] = useState('')
  const debouncedQuery = useDebouncedValue(query, 300) // 300ms delay

  useEffect(() => {
    if (debouncedQuery) {
      api.profiles.list({ q: debouncedQuery })
    }
  }, [debouncedQuery])

  return <Input value={query} onChange={(e) => setQuery(e.target.value)} />
}

useDocumentEmbed()

Manage embedded document state for cross-document embedding:

import { useDocumentEmbed } from '@cloudillo/react'

function EmbeddedDoc({ fileId, contentType }) {
  const { embedUrl, loading, error } = useDocumentEmbed({
    targetFileId: fileId,
    targetContentType: contentType,
    sourceFileId: currentFileId
  })

  if (loading) return <LoadingSpinner />
  if (error) return <div>Embed error: {error.message}</div>

  return <iframe src={embedUrl} />
}

Common Patterns

Pattern 1: Microfrontend App

The typical pattern for a Cloudillo microfrontend app:

import { useCloudillo, useApi } from '@cloudillo/react'
import { useEffect, useState } from 'react'

function App() {
  const { token, idTag, ownerTag, fileId, access } = useCloudillo('my-app')
  const { api } = useApi()
  const [data, setData] = useState(null)

  useEffect(() => {
    if (!api || !fileId) return

    api.files.getDescriptor(fileId)
      .then(setData)
      .catch(console.error)
  }, [api, fileId])

  if (!token) return <div>Initializing...</div>
  if (!data) return <div>Loading...</div>

  return (
    <div>
      <h1>{data.fileName}</h1>
      <p>Owner: {ownerTag}</p>
      <p>Access: {access}</p>
    </div>
  )
}

Pattern 2: Collaborative Editor

import { useCloudilloEditor } from '@cloudillo/react'
import { useEffect } from 'react'

function Editor() {
  const { yDoc, provider, synced, access } = useCloudilloEditor('my-editor')

  useEffect(() => {
    if (!synced) return

    const yText = yDoc.getText('content')

    // Set up your editor binding here
    // e.g., QuillBinding, TipTapExtension, etc.

    return () => {
      // Clean up binding
    }
  }, [yDoc, synced])

  if (!synced) return <div>Syncing document...</div>

  return (
    <div>
      {access === 'read' && <div className="read-only-banner">Read only</div>}
      <div id="editor-container" />
    </div>
  )
}

Pattern 3: Fetching Data with useApi

import { useApi } from '@cloudillo/react'
import { useEffect, useState } from 'react'

function Profile({ idTag }) {
  const { api } = useApi()
  const [profile, setProfile] = useState(null)
  const [error, setError] = useState(null)

  useEffect(() => {
    if (!api) return

    api.profiles.get(idTag)
      .then(setProfile)
      .catch(setError)
  }, [api, idTag])

  if (!api) return <div>No API client</div>
  if (error) return <div>Error: {error.message}</div>
  if (!profile) return <div>Loading...</div>

  return (
    <div>
      <h1>{profile.name}</h1>
      <p>{profile.idTag}</p>
    </div>
  )
}

Pattern 4: Creating Actions

import { useApi } from '@cloudillo/react'
import { useState } from 'react'

function CreatePost() {
  const { api, authenticated } = useApi()
  const [text, setText] = useState('')
  const [posting, setPosting] = useState(false)

  if (!authenticated) return <div>Please log in to post</div>

  const handleSubmit = async (e) => {
    e.preventDefault()
    if (!api || !text.trim()) return

    setPosting(true)

    try {
      await api.actions.create({
        type: 'POST',
        content: { text }
      })
      setText('')
    } catch (error) {
      console.error('Failed to create post:', error)
    } finally {
      setPosting(false)
    }
  }

  return (
    <form onSubmit={handleSubmit}>
      <textarea
        value={text}
        onChange={(e) => setText(e.target.value)}
        placeholder="What's on your mind?"
      />
      <button type="submit" disabled={posting || !text.trim()}>
        {posting ? 'Posting...' : 'Post'}
      </button>
    </form>
  )
}

Pattern 5: Role-Based Rendering

import { useAuth } from '@cloudillo/react'

function AdminPanel() {
  const [auth] = useAuth()

  if (!auth?.roles?.includes('admin')) {
    return <div>Access denied. Admin role required.</div>
  }

  return (
    <div>
      <h1>Admin Panel</h1>
      {/* Admin-only features */}
    </div>
  )
}

Pattern 6: File Upload

import { useApi } from '@cloudillo/react'
import { useState } from 'react'

function ImageUpload() {
  const { api } = useApi()
  const [uploading, setUploading] = useState(false)
  const [result, setResult] = useState(null)

  const handleFileChange = async (e) => {
    const file = e.target.files?.[0]
    if (!file || !api) return

    setUploading(true)

    try {
      // Upload the file
      const uploaded = await api.files.uploadBlob(
        'gallery',      // preset
        file.name,      // fileName
        file,           // file data
        file.type       // contentType
      )

      setResult(uploaded)
    } catch (error) {
      console.error('Upload failed:', error)
    } finally {
      setUploading(false)
    }
  }

  return (
    <div>
      <input type="file" accept="image/*" onChange={handleFileChange} />
      {uploading && <div>Uploading...</div>}
      {result && <div>Uploaded: {result.fileId}</div>}
    </div>
  )
}

Pattern 7: Real-Time Updates with RTDB

import { useApi, useAuth } from '@cloudillo/react'
import { useEffect, useState } from 'react'
import { RtdbClient } from '@cloudillo/rtdb'
import { getRtdbUrl } from '@cloudillo/core'

function TodoList({ dbFileId }) {
  const [auth] = useAuth()
  const [todos, setTodos] = useState([])

  useEffect(() => {
    if (!auth?.token || !auth?.idTag) return

    const rtdb = new RtdbClient({
      dbId: dbFileId,
      auth: { getToken: () => auth.token },
      serverUrl: getRtdbUrl(auth.idTag, dbFileId, auth.token)
    })

    const todosRef = rtdb.collection('todos')

    // Subscribe to real-time updates
    const unsubscribe = todosRef.onSnapshot((snapshot) => {
      setTodos(snapshot.docs.map(doc => ({ id: doc.id, ...doc.data() })))
    })

    // Cleanup on unmount
    return () => {
      unsubscribe()
      rtdb.disconnect()
    }
  }, [auth?.token, auth?.idTag, dbFileId])

  return (
    <ul>
      {todos.map(todo => (
        <li key={todo.id}>{todo.title}</li>
      ))}
    </ul>
  )
}

TypeScript Support

All hooks are fully typed:

import type { AuthState, ApiHook } from '@cloudillo/react'
import { useAuth, useApi } from '@cloudillo/react'

function MyComponent() {
  const [auth, setAuth] = useAuth()
  const { api, authenticated, setIdTag }: ApiHook = useApi()

  // TypeScript knows the types
  auth?.idTag    // string | undefined
  auth?.tnId     // number | undefined
  auth?.roles    // string[] | undefined

  api?.profiles.getOwn()  // Returns Promise<ProfileKeys>
}

See Also

React Components Reference

Complete reference for all 85+ components, 12+ hooks, and utility functions exported by @cloudillo/react.

Layout Components

Container

Centered max-width container for page content.

import { Container } from '@cloudillo/react'

<Container>
  <h1>Page Content</h1>
</Container>

HBox, VBox, Group

Flexbox layout primitives.

import { HBox, VBox, Group } from '@cloudillo/react'

// Horizontal layout
<HBox gap="md">
  <Button>Left</Button>
  <Button>Right</Button>
</HBox>

// Vertical layout
<VBox gap="sm">
  <Input label="Name" />
  <Input label="Email" />
</VBox>

// Grouped items with spacing
<Group>
  <Tag>React</Tag>
  <Tag>TypeScript</Tag>
</Group>

Props (shared):

  • gap?: 'xs' | 'sm' | 'md' | 'lg' | 'xl' - Spacing between children
  • align?: 'start' | 'center' | 'end' | 'stretch' - Alignment
  • justify?: 'start' | 'center' | 'end' | 'between' | 'around' - Justification

Panel

Bordered container with optional header.

import { Panel } from '@cloudillo/react'

<Panel title="Settings">
  <p>Panel content here</p>
</Panel>

Card

Elevated card container.

import { Card } from '@cloudillo/react'

<Card>
  <h3>Card Title</h3>
  <p>Card content</p>
</Card>

Fcd (Filter-Content-Details)

Three-column responsive layout pattern for list views.

import { Fcd, FcdContainer, FcdFilter, FcdContent, FcdDetails } from '@cloudillo/react'

<FcdContainer>
  <FcdFilter>
    <FilterBar />
  </FcdFilter>
  <FcdContent>
    <ItemList />
  </FcdContent>
  <FcdDetails>
    <ItemDetails />
  </FcdDetails>
</FcdContainer>

Collapsible sidebar with mobile support.

import {
  useSidebar,
  Sidebar,
  SidebarContent,
  SidebarHeader,
  SidebarFooter,
  SidebarNav,
  SidebarSection,
  SidebarToggle,
  SidebarBackdrop,
  SidebarResizeHandle
} from '@cloudillo/react'

function Layout() {
  const sidebar = useSidebar({ defaultOpen: true })

  return (
    <div>
      <SidebarBackdrop {...sidebar} />
      <Sidebar {...sidebar}>
        <SidebarHeader>
          <Logo />
        </SidebarHeader>
        <SidebarContent>
          <SidebarNav>
            <SidebarSection title="Main">
              <NavItem to="/home">Home</NavItem>
              <NavItem to="/files">Files</NavItem>
            </SidebarSection>
          </SidebarNav>
        </SidebarContent>
        <SidebarFooter>
          <ProfileCard />
        </SidebarFooter>
        <SidebarResizeHandle />
      </Sidebar>
      <SidebarToggle {...sidebar} />
    </div>
  )
}

useSidebar options:

interface UseSidebarOptions {
  defaultOpen?: boolean
  defaultWidth?: number
  minWidth?: number
  maxWidth?: number
  breakpoint?: number // Mobile breakpoint
}

Additional components:

  • SidebarBackdrop - Mobile overlay backdrop that closes sidebar when tapped
  • SidebarResizeHandle - Draggable handle for resizing sidebar width
  • SidebarContext / useSidebarContext - Context provider and hook for nested components

Navigation menu components.

import { Nav, NavGroup, NavItem, NavLink } from '@cloudillo/react'

<Nav>
  <NavGroup title="Main">
    <NavItem icon={<HomeIcon />}>
      <NavLink to="/home">Home</NavLink>
    </NavItem>
    <NavItem icon={<FilesIcon />}>
      <NavLink to="/files">Files</NavLink>
    </NavItem>
  </NavGroup>
</Nav>

Tabs, Tab

Tabbed interface.

import { Tabs, Tab } from '@cloudillo/react'

<Tabs defaultValue="tab1">
  <Tab value="tab1" label="General">
    <GeneralSettings />
  </Tab>
  <Tab value="tab2" label="Advanced">
    <AdvancedSettings />
  </Tab>
</Tabs>

Context: TabsContext is available for building custom tab implementations.


Form Components

Input

Text input with label and validation.

import { Input } from '@cloudillo/react'

<Input
  label="Email"
  type="email"
  placeholder="you@example.com"
  error="Invalid email address"
/>

Props:

  • label?: string - Input label
  • error?: string - Error message
  • hint?: string - Helper text
  • size?: 'sm' | 'md' | 'lg' - Input size
  • All standard <input> props

TextArea

Multi-line text input.

import { TextArea } from '@cloudillo/react'

<TextArea
  label="Description"
  rows={4}
  placeholder="Enter description..."
/>

Select

Custom dropdown select.

import { Select } from '@cloudillo/react'

<Select
  label="Country"
  options={[
    { value: 'us', label: 'United States' },
    { value: 'uk', label: 'United Kingdom' },
    { value: 'de', label: 'Germany' }
  ]}
  onChange={(value) => console.log(value)}
/>

NativeSelect

Native browser select element.

import { NativeSelect } from '@cloudillo/react'

<NativeSelect label="Priority">
  <option value="low">Low</option>
  <option value="medium">Medium</option>
  <option value="high">High</option>
</NativeSelect>

NumberInput

Numeric input with increment/decrement buttons.

import { NumberInput } from '@cloudillo/react'

<NumberInput
  label="Quantity"
  min={0}
  max={100}
  step={1}
  value={5}
  onChange={(value) => console.log(value)}
/>

ColorInput

Color picker input.

import { ColorInput } from '@cloudillo/react'

<ColorInput
  label="Theme Color"
  value="#3b82f6"
  onChange={(color) => console.log(color)}
/>

Toggle

On/off toggle switch.

import { Toggle } from '@cloudillo/react'

<Toggle
  label="Enable notifications"
  checked={enabled}
  onChange={setEnabled}
/>

Fieldset

Group related form fields.

import { Fieldset } from '@cloudillo/react'

<Fieldset legend="Contact Information">
  <Input label="Phone" />
  <Input label="Address" />
</Fieldset>

InputGroup

Group input with addons (prefix/suffix).

import { InputGroup } from '@cloudillo/react'

<InputGroup>
  <InputGroup.Addon>https://</InputGroup.Addon>
  <Input placeholder="example.com" />
</InputGroup>

TagInput

Multi-value tag input with autocomplete.

import { TagInput } from '@cloudillo/react'

<TagInput
  label="Tags"
  value={['react', 'typescript']}
  onChange={setTags}
  suggestions={['react', 'vue', 'angular', 'typescript']}
  placeholder="Add tags..."
/>

InlineEditForm

Inline editable text field.

import { InlineEditForm } from '@cloudillo/react'

<InlineEditForm
  value={title}
  onSave={(newValue) => updateTitle(newValue)}
  placeholder="Click to edit..."
/>

Button Components

Button

Primary button component.

import { Button } from '@cloudillo/react'

<Button variant="primary" onClick={handleClick}>
  Save Changes
</Button>

<Button variant="secondary" size="sm">
  Cancel
</Button>

<Button variant="danger" loading>
  Deleting...
</Button>

Props:

  • variant?: 'primary' | 'secondary' | 'danger' | 'ghost'
  • size?: 'xs' | 'sm' | 'md' | 'lg'
  • loading?: boolean - Shows spinner
  • disabled?: boolean
  • icon?: ReactNode - Left icon
  • iconRight?: ReactNode - Right icon

LinkButton

Button styled as a link.

import { LinkButton } from '@cloudillo/react'

<LinkButton to="/settings">Go to Settings</LinkButton>

Dialog & Overlay Components

Dialog

Modal dialog with backdrop.

import { Dialog, useDialog } from '@cloudillo/react'

function MyComponent() {
  const dialog = useDialog()

  return (
    <>
      <Button onClick={dialog.open}>Open Dialog</Button>

      <Dialog {...dialog} title="Confirm Action">
        <p>Are you sure you want to proceed?</p>
        <HBox gap="sm">
          <Button variant="secondary" onClick={dialog.close}>Cancel</Button>
          <Button variant="primary" onClick={handleConfirm}>Confirm</Button>
        </HBox>
      </Dialog>
    </>
  )
}

useDialog returns:

interface UseDialogReturn {
  isOpen: boolean
  open: () => void
  close: () => void
  toggle: () => void
}

DialogContainer

Wrapper component for rendering dialogs at the root level.

import { DialogContainer } from '@cloudillo/react'

function App() {
  return (
    <>
      <MainContent />
      <DialogContainer />
    </>
  )
}

Low-level modal wrapper.

import { Modal } from '@cloudillo/react'

<Modal isOpen={isOpen} onClose={handleClose}>
  <div className="modal-content">
    Custom modal content
  </div>
</Modal>

BottomSheet

Mobile-friendly bottom sheet.

import { BottomSheet } from '@cloudillo/react'

<BottomSheet
  isOpen={isOpen}
  onClose={handleClose}
  snapPoints={['50%', '90%']}
>
  <div>Sheet content</div>
</BottomSheet>

Dropdown menu container.

import { Dropdown } from '@cloudillo/react'

<Dropdown
  trigger={<Button>Options</Button>}
  align="end"
>
  <MenuItem onClick={handleEdit}>Edit</MenuItem>
  <MenuItem onClick={handleDelete}>Delete</MenuItem>
</Dropdown>

Popper

Floating positioned element (tooltips, popovers).

import { Popper } from '@cloudillo/react'

<Popper
  trigger={<Button>Hover me</Button>}
  placement="top"
>
  <div>Tooltip content</div>
</Popper>

QRCodeDialog

Dialog showing a QR code.

import { QRCodeDialog } from '@cloudillo/react'

<QRCodeDialog
  isOpen={isOpen}
  onClose={handleClose}
  value="https://cloudillo.net/share/abc123"
  title="Share Link"
/>

Context menu and dropdown menu items.

import { Menu, MenuItem, MenuDivider, MenuHeader } from '@cloudillo/react'

<Menu>
  <MenuHeader>Actions</MenuHeader>
  <MenuItem icon={<EditIcon />} onClick={handleEdit}>
    Edit
  </MenuItem>
  <MenuItem icon={<CopyIcon />} onClick={handleCopy}>
    Copy
  </MenuItem>
  <MenuDivider />
  <MenuItem icon={<TrashIcon />} variant="danger" onClick={handleDelete}>
    Delete
  </MenuItem>
</Menu>

ActionSheet

Mobile action sheet (bottom menu).

import { ActionSheet, ActionSheetItem, ActionSheetDivider } from '@cloudillo/react'

<ActionSheet isOpen={isOpen} onClose={handleClose}>
  <ActionSheetItem onClick={handleShare}>Share</ActionSheetItem>
  <ActionSheetItem onClick={handleCopy}>Copy Link</ActionSheetItem>
  <ActionSheetDivider />
  <ActionSheetItem variant="danger" onClick={handleDelete}>
    Delete
  </ActionSheetItem>
</ActionSheet>

Feedback Components

Toast

Toast notification system.

import { useToast, ToastContainer } from '@cloudillo/react'

// In your app root
function App() {
  return (
    <>
      <ToastContainer />
      <MainContent />
    </>
  )
}

// In any component
function MyComponent() {
  const toast = useToast()

  const handleSave = async () => {
    try {
      await saveData()
      toast.success('Saved successfully!')
    } catch (err) {
      toast.error('Failed to save')
    }
  }

  return <Button onClick={handleSave}>Save</Button>
}

useToast methods:

  • toast.success(message, options?) - Success toast
  • toast.error(message, options?) - Error toast
  • toast.warning(message, options?) - Warning toast
  • toast.info(message, options?) - Info toast
  • toast.custom(content, options?) - Custom toast

Toast sub-components:

For building custom toast layouts:

import {
  Toast,
  ToastIcon,
  ToastContent,
  ToastTitle,
  ToastMessage,
  ToastActions,
  ToastClose,
  ToastProgress
} from '@cloudillo/react'

// Custom toast layout
<Toast variant="success">
  <ToastIcon />
  <ToastContent>
    <ToastTitle>Success!</ToastTitle>
    <ToastMessage>Your changes have been saved.</ToastMessage>
  </ToastContent>
  <ToastActions>
    <Button size="sm">Undo</Button>
  </ToastActions>
  <ToastClose />
  <ToastProgress />
</Toast>

Context: ToastContext and useToastContext are available for building custom toast providers.

LoadingSpinner

Spinning loader indicator.

import { LoadingSpinner } from '@cloudillo/react'

<LoadingSpinner size="md" />

{isLoading && <LoadingSpinner />}

Skeleton, SkeletonText, SkeletonCard, SkeletonList

Loading skeleton placeholders.

import { Skeleton, SkeletonText, SkeletonCard, SkeletonList } from '@cloudillo/react'

// Basic skeleton
<Skeleton width={200} height={20} />

// Text skeleton
<SkeletonText lines={3} />

// Card skeleton
<SkeletonCard />

// List skeleton
<SkeletonList count={5} />

Progress

Progress bar.

import { Progress } from '@cloudillo/react'

<Progress value={75} max={100} />

<Progress value={uploadProgress} showLabel />

EmptyState

Empty state placeholder.

import { EmptyState } from '@cloudillo/react'

<EmptyState
  icon={<FilesIcon />}
  title="No files yet"
  description="Upload your first file to get started"
  action={<Button>Upload File</Button>}
/>

Profile Components

Avatar, AvatarStatus, AvatarBadge, AvatarGroup

User avatar components.

import { Avatar, AvatarStatus, AvatarBadge, AvatarGroup } from '@cloudillo/react'

// Basic avatar
<Avatar src={profilePic} name="Alice" size="md" />

// With status indicator
<Avatar src={profilePic}>
  <AvatarStatus status="online" />
</Avatar>

// With badge
<Avatar src={profilePic}>
  <AvatarBadge>3</AvatarBadge>
</Avatar>

// Group of avatars
<AvatarGroup max={3}>
  <Avatar src={user1.pic} name={user1.name} />
  <Avatar src={user2.pic} name={user2.name} />
  <Avatar src={user3.pic} name={user3.name} />
  <Avatar src={user4.pic} name={user4.name} />
</AvatarGroup>

ProfilePicture, UnknownProfilePicture

Cloudillo profile picture components.

import { ProfilePicture, UnknownProfilePicture } from '@cloudillo/react'

<ProfilePicture idTag="alice.cloudillo.net" size="lg" />

<UnknownProfilePicture size="md" />

IdentityTag

Display identity tag with icon.

import { IdentityTag } from '@cloudillo/react'

<IdentityTag idTag="alice.cloudillo.net" />

ProfileCard

Profile card with picture, name, and actions.

import { ProfileCard } from '@cloudillo/react'

<ProfileCard
  profile={profile}
  onConnect={handleConnect}
  onFollow={handleFollow}
/>

ProfileAudienceCard

Profile card optimized for audience selection.

import { ProfileAudienceCard } from '@cloudillo/react'

<ProfileAudienceCard
  profile={profile}
  selected={isSelected}
  onSelect={handleSelect}
/>

EditProfileList

Editable list of profiles (for managing connections, etc.).

import { EditProfileList } from '@cloudillo/react'

<EditProfileList
  profiles={connections}
  onRemove={handleRemove}
  emptyMessage="No connections yet"
/>

ProfileSelect

Profile selection input with search.

import { ProfileSelect } from '@cloudillo/react'

<ProfileSelect
  value={selectedProfile}
  onChange={setSelectedProfile}
  placeholder="Search profiles..."
/>

Data Display Components

TreeView, TreeItem

Hierarchical tree view.

import { TreeView, TreeItem } from '@cloudillo/react'

<TreeView>
  <TreeItem label="Documents" icon={<FolderIcon />}>
    <TreeItem label="Report.pdf" icon={<FileIcon />} />
    <TreeItem label="Notes.txt" icon={<FileIcon />} />
  </TreeItem>
  <TreeItem label="Images" icon={<FolderIcon />}>
    <TreeItem label="Photo.jpg" icon={<ImageIcon />} />
  </TreeItem>
</TreeView>

Accordion, AccordionItem

Collapsible accordion sections.

import { Accordion, AccordionItem } from '@cloudillo/react'

<Accordion>
  <AccordionItem title="General Settings">
    <GeneralSettings />
  </AccordionItem>
  <AccordionItem title="Advanced Settings">
    <AdvancedSettings />
  </AccordionItem>
</Accordion>

PropertyPanel, PropertySection, PropertyField

Property inspector panel (like IDE properties).

import { PropertyPanel, PropertySection, PropertyField } from '@cloudillo/react'

<PropertyPanel>
  <PropertySection title="Appearance">
    <PropertyField label="Width">
      <NumberInput value={width} onChange={setWidth} />
    </PropertyField>
    <PropertyField label="Color">
      <ColorInput value={color} onChange={setColor} />
    </PropertyField>
  </PropertySection>
</PropertyPanel>

TimeFormat

Formatted time display (relative or absolute).

import { TimeFormat } from '@cloudillo/react'

<TimeFormat value={createdAt} />
// Renders: "2 hours ago" or "Jan 15, 2025"

Badge

Status badge.

import { Badge } from '@cloudillo/react'

<Badge variant="success">Active</Badge>
<Badge variant="warning">Pending</Badge>
<Badge variant="danger">Error</Badge>

Tag, TagList

Tags and tag lists.

import { Tag, TagList } from '@cloudillo/react'

<Tag>React</Tag>

<TagList
  tags={['react', 'typescript', 'cloudillo']}
  onRemove={handleRemoveTag}
/>

Filter & Toolbar Components

FilterBar

Filter bar with search and filters.

import {
  FilterBar,
  FilterBarSearch,
  FilterBarItem,
  FilterBarSection,
  FilterBarDivider
} from '@cloudillo/react'

<FilterBar>
  <FilterBarSearch
    value={search}
    onChange={setSearch}
    placeholder="Search files..."
  />
  <FilterBarDivider />
  <FilterBarSection>
    <FilterBarItem
      label="Type"
      value={typeFilter}
      onChange={setTypeFilter}
      options={typeOptions}
    />
    <FilterBarItem
      label="Date"
      value={dateFilter}
      onChange={setDateFilter}
      options={dateOptions}
    />
  </FilterBarSection>
</FilterBar>

Alternative export: FilterBarComponent is also exported for cases where you need to avoid naming conflicts.

Toolbar

Action toolbar.

import { Toolbar, ToolbarGroup, ToolbarDivider, ToolbarSpacer } from '@cloudillo/react'

<Toolbar>
  <ToolbarGroup>
    <Button icon={<BoldIcon />} />
    <Button icon={<ItalicIcon />} />
    <Button icon={<UnderlineIcon />} />
  </ToolbarGroup>
  <ToolbarDivider />
  <ToolbarGroup>
    <Button icon={<AlignLeftIcon />} />
    <Button icon={<AlignCenterIcon />} />
  </ToolbarGroup>
  <ToolbarSpacer />
  <Button>Save</Button>
</Toolbar>

Infinite Scroll Components

LoadMoreTrigger

Intersection observer trigger for infinite scroll.

import { LoadMoreTrigger } from '@cloudillo/react'
import { useInfiniteScroll } from '@cloudillo/react'

function FileList() {
  const { items, isLoading, sentinelRef } = useInfiniteScroll({
    fetchPage: async (cursor, limit) => {
      const result = await api.files.list({ cursor, limit })
      return {
        items: result.data,
        nextCursor: result.cursorPagination?.nextCursor ?? null,
        hasMore: result.cursorPagination?.hasMore ?? false
      }
    }
  })

  return (
    <div>
      {items.map(file => <FileCard key={file.fileId} file={file} />)}
      <LoadMoreTrigger ref={sentinelRef} isLoading={isLoading} />
    </div>
  )
}

Utility Components

FormattedText

Render formatted text with markdown support.

import { FormattedText } from '@cloudillo/react'

<FormattedText content="**Bold** and *italic* text" />

FontPicker

Font selection component. Works with the @cloudillo/fonts library for font metadata and pairing suggestions.

import { FontPicker } from '@cloudillo/react'
import { getSuggestedBodyFonts } from '@cloudillo/fonts'

<FontPicker
  value={fontFamily}
  onChange={setFontFamily}
/>

// Use with font pairings
const suggestedBodies = getSuggestedBodyFonts(headingFont)

See @cloudillo/fonts for available fonts and pairing APIs.

ZoomableImage

Image component with pinch-to-zoom and pan support.

import { ZoomableImage } from '@cloudillo/react'

<ZoomableImage src={imageUrl} alt="Zoomable photo" />

DocumentEmbed, DocumentEmbedIframe, SvgDocumentEmbed

Components for embedding documents within other documents.

import { DocumentEmbedIframe, SvgDocumentEmbed, useDocumentEmbed } from '@cloudillo/react'

// Iframe-based embed (for HTML/interactive documents)
<DocumentEmbedIframe
  fileId={targetFileId}
  contentType="cloudillo/quillo"
  sourceFileId={currentFileId}
/>

// SVG-based embed (for embedding within SVG canvases)
<SvgDocumentEmbed
  fileId={targetFileId}
  contentType="cloudillo/quillo"
  sourceFileId={currentFileId}
  width={400}
  height={300}
/>

The useDocumentEmbed hook manages the embed lifecycle (requesting embed URLs from the shell, tracking loading state).


Utility Hooks

useMergedRefs

Combine multiple refs into one.

import { useMergedRefs } from '@cloudillo/react'

function MyComponent({ forwardedRef }) {
  const localRef = useRef()
  const mergedRef = useMergedRefs(localRef, forwardedRef)

  return <div ref={mergedRef} />
}

useBodyScrollLock

Lock body scroll (for modals).

import { useBodyScrollLock } from '@cloudillo/react'

function Modal({ isOpen }) {
  useBodyScrollLock(isOpen)

  return isOpen ? <div className="modal">...</div> : null
}

useEscapeKey

Handle Escape key press.

import { useEscapeKey } from '@cloudillo/react'

function Modal({ onClose }) {
  useEscapeKey(onClose)

  return <div className="modal">...</div>
}

useOutsideClick

Detect clicks outside an element.

import { useOutsideClick } from '@cloudillo/react'

function Dropdown({ onClose }) {
  const ref = useRef()
  useOutsideClick(ref, onClose)

  return <div ref={ref} className="dropdown">...</div>
}

useMediaQuery

Responsive media query hook.

import { useMediaQuery } from '@cloudillo/react'

function MyComponent() {
  const isMobile = useMediaQuery('(max-width: 768px)')

  return isMobile ? <MobileView /> : <DesktopView />
}

useIsMobile

Shorthand for mobile detection.

import { useIsMobile } from '@cloudillo/react'

function MyComponent() {
  const isMobile = useIsMobile()

  return isMobile ? <MobileNav /> : <DesktopNav />
}

usePrefersReducedMotion

Accessibility: detect reduced motion preference.

import { usePrefersReducedMotion } from '@cloudillo/react'

function AnimatedComponent() {
  const prefersReducedMotion = usePrefersReducedMotion()

  return (
    <div className={prefersReducedMotion ? 'no-animation' : 'animated'}>
      ...
    </div>
  )
}

useDebouncedValue

Debounce a value that changes rapidly (e.g., search input).

import { useDebouncedValue } from '@cloudillo/react'

function SearchField() {
  const [query, setQuery] = useState('')
  const debouncedQuery = useDebouncedValue(query, 300)

  // debouncedQuery updates 300ms after the last query change
}

useDocumentEmbed

Manage state for embedding documents (requesting embed URLs from the shell).

import { useDocumentEmbed } from '@cloudillo/react'

function EmbedManager({ fileId, contentType }) {
  const { embedUrl, loading, error } = useDocumentEmbed({
    targetFileId: fileId,
    targetContentType: contentType,
    sourceFileId: currentFileId
  })
}

See Also

@cloudillo/types

Overview

TypeScript type definitions for Cloudillo with runtime validation using @symbion/runtype.

Installation

pnpm add @cloudillo/types

Core Types

Profile

User or community profile information.

import type { Profile } from '@cloudillo/types'

const profile: Profile = {
  idTag: 'alice@example.com',
  name: 'Alice Johnson',
  profilePic: '/file/b1~abc123'
}

Fields:

  • idTag: string - Unique identity (DNS-based)
  • name?: string - Display name
  • type?: 'person' | 'community' - Profile type
  • profilePic?: string - Profile picture URL
  • status?: ProfileStatus - Profile status
  • connected?: ProfileConnectionStatus - Connection status
  • following?: boolean - Whether the current user follows this profile
  • roles?: string[] - Community roles (e.g., ['leader'], ['moderator'])

Action

Represents a social action or activity.

import type { Action } from '@cloudillo/types'

const action: Action = {
  actionId: 'act_123',
  type: 'POST',
  issuerTag: 'alice@example.com',
  content: {
    text: 'Hello, world!',
    title: 'My First Post'
  },
  createdAt: 1735000000,
  status: 'A'
}

Fields:

  • actionId: string - Unique action identifier
  • type: ActionType - Type of action (POST, CMNT, REACT, etc.)
  • issuerTag: string - Who created the action
  • content?: unknown - Action-specific content
  • createdAt: number - Unix timestamp (seconds)
  • status?: ActionStatus - P/A/D/C/N/R/S
  • subType?: string - Action subtype/category
  • parentId?: string - Parent action (for threads)
  • rootId?: string - Root action (for deep threads)
  • audienceTag?: string - Target audience
  • subject?: string - Subject/target (e.g., who to follow)
  • attachments?: string[] - File IDs
  • expiresAt?: number - Expiration timestamp

ActionType

Literal union of all action types.

import type { ActionType } from '@cloudillo/types'

const type: ActionType =
  | 'CONN'   // Connection request
  | 'FLLW'   // Follow user
  | 'POST'   // Create post
  | 'REPOST' // Repost/share
  | 'REACT'  // Reaction
  | 'CMNT'   // Comment
  | 'SHRE'   // Share resource
  | 'MSG'    // Message
  | 'FSHR'   // File share
  | 'PRINVT' // Profile invite
Additional Action Types

ACK (Acknowledgment) and RSTAT (Reaction Statistics) exist as action variants in the tagged union (tBaseAction) but are not part of the ActionType literal type. They are used internally for specific action handling.

ActionStatus

Action status enumeration.

import type { ActionStatus } from '@cloudillo/types'

const status: ActionStatus =
  | 'P' // Pending (draft/unpublished)
  | 'A' // Active (default - published/finalized)
  | 'D' // Deleted (soft delete)
  | 'C' // Created (pending approval, e.g., connection requests)
  | 'N' // New (notification awaiting acknowledgment)
  | 'R' // Draft (saved but not yet published)
  | 'S' // Scheduled (draft with confirmed publish time)

ProfileStatus

Profile status codes.

import type { ProfileStatus } from '@cloudillo/types'

const status: ProfileStatus =
  | 'A' // Active
  | 'T' // Trusted
  | 'B' // Blocked
  | 'M' // Muted
  | 'S' // Suspended

ProfileConnectionStatus

Connection status between profiles.

import type { ProfileConnectionStatus } from '@cloudillo/types'

// true = connected
// 'R' = request pending
// undefined = not connected

CommunityRole

Community role hierarchy.

import type { CommunityRole } from '@cloudillo/types'
import { ROLE_LEVELS } from '@cloudillo/types'

type CommunityRole =
  | 'public'      // Level 0 - Anyone
  | 'follower'    // Level 1 - Following the community
  | 'supporter'   // Level 2 - Supporter/subscriber
  | 'contributor' // Level 3 - Can create content
  | 'moderator'   // Level 4 - Can moderate
  | 'leader'      // Level 5 - Full admin access

// Use ROLE_LEVELS for permission checks
console.log(ROLE_LEVELS)
// { public: 0, follower: 1, supporter: 2, contributor: 3, moderator: 4, leader: 5 }

// Check if user has sufficient role
function hasPermission(userRole: CommunityRole, required: CommunityRole): boolean {
  return ROLE_LEVELS[userRole] >= ROLE_LEVELS[required]
}

NewAction

Data for creating a new action.

import type { NewAction } from '@cloudillo/types'

const newAction: NewAction = {
  type: 'POST',
  content: {
    text: 'Hello!',
    title: 'Greeting'
  },
  attachments: ['file_123']
}

Fields:

  • type: string - Action type
  • subType?: string - Action subtype
  • parentId?: string - Parent action ID
  • rootId?: string - Root action ID
  • audienceTag?: string - Target audience
  • content?: unknown - Action content
  • attachments?: string[] - Attached file IDs
  • subject?: string - Subject/target
  • expiresAt?: number - Expiration time
  • visibility?: string - Visibility level ('P' = Public, 'C' = Connected, 'F' = Followers)
  • draft?: boolean - If true, save as draft (status 'R')
  • publishAt?: number - Unix timestamp for scheduled publishing

ActionView

Extended action with resolved references and statistics.

import type { ActionView } from '@cloudillo/types'

const actionView: ActionView = {
  actionId: 'act_123',
  type: 'POST',
  issuer: {
    idTag: 'alice@example.com',
    name: 'Alice Johnson',
    profilePic: '/file/b1~abc'
  },
  content: { text: 'Hello!' },
  attachments: [
    { fileId: 'f1~abc', dim: [1920, 1080], localVariants: ['vis.tn', 'vis.sd'] }
  ],
  createdAt: 1735000000,
  stat: {
    ownReaction: 'LOVE',
    reactions: '10,3,1',
    comments: 3,
    commentsRead: 2
  }
}

Additional Fields (beyond Action):

  • issuer: ProfileInfo - Resolved issuer profile
  • audience?: ProfileInfo - Resolved audience profile
  • attachments?: Array<{ fileId: string, dim?: [number, number] | null, localVariants?: string[] }> - Attachments with dimensions and available variants
  • status?: ActionStatus - Action status
  • stat?: object - Statistics:
    • ownReaction?: string - Current user’s reaction
    • reactions?: string - Encoded reaction counts
    • comments?: number - Number of comments
    • commentsRead?: number - Number of comments read by current user
  • visibility?: string - Visibility level
  • x?: unknown - Extensible metadata (e.g., x.role for subscriptions)

FileView

File metadata with owner information.

import type { FileView } from '@cloudillo/types'

const file: FileView = {
  fileId: 'f1~abc123',
  status: 'M',
  contentType: 'image/png',
  fileName: 'photo.png',
  fileTp: 'BLOB',
  createdAt: new Date('2025-01-01'),
  tags: ['vacation', 'beach'],
  owner: {
    idTag: 'alice@example.com',
    name: 'Alice'
  }
}

Fields:

  • fileId: string - File identifier
  • status: 'P' | 'M' - Pending or Metadata-ready
  • contentType: string - MIME type
  • fileName: string - Original filename
  • fileTp?: string - File type (CRDT/RTDB/BLOB)
  • createdAt: string | Date - Creation time
  • tags?: string[] - Tags
  • owner?: Profile - Owner profile
  • preset?: string - Image preset configuration

Runtime Validation

All types include runtime validators using @symbion/runtype:

import { ProfileValidator, ActionValidator } from '@cloudillo/types'

// Validate data at runtime
const data = await api.profiles.getOwn()

if (ProfileValidator.validate(data)) {
  // TypeScript knows data is a valid Profile
  console.log(data.idTag)
} else {
  console.error('Invalid profile data')
}

Type Guards

Use type guards to check types at runtime:

import { isAction, isProfile } from '@cloudillo/types'

function processData(data: unknown) {
  if (isAction(data)) {
    console.log('Action:', data.type)
  } else if (isProfile(data)) {
    console.log('Profile:', data.idTag)
  }
}

Enum Constants

Action Types

import { ACTION_TYPES } from '@cloudillo/types'

console.log(ACTION_TYPES)
// ['CONN', 'FLLW', 'POST', 'REPOST', 'REACT', 'CMNT', 'SHRE', 'MSG', 'FSHR', 'PRINVT']

// Use in UI
{ACTION_TYPES.map(type => (
  <option key={type} value={type}>{type}</option>
))}

Action Statuses

import { ACTION_STATUSES } from '@cloudillo/types'

console.log(ACTION_STATUSES)
// ['P', 'A', 'D', 'C', 'N', 'R', 'S']

Action Type Variants

Typed action structures for type-safe action creation.

User Relationships

import type { ConnectAction, FollowAction } from '@cloudillo/types'

// Connection request
const connect: ConnectAction = {
  type: 'CONN',
  subject: 'bob.cloudillo.net',      // Who to connect with
  content: 'Would love to connect!'  // Optional message
}

// Follow relationship
const follow: FollowAction = {
  type: 'FLLW',
  subject: 'news.cloudillo.net'      // Who to follow
}

Content Actions

import type { PostAction, CommentAction, ReactAction } from '@cloudillo/types'

// Create a post
const post: PostAction = {
  type: 'POST',
  subType: 'IMG',                    // Optional: IMG, VID, etc.
  content: 'Check out this photo!',
  attachments: ['fileId123'],        // Optional attachments
  audience: 'friends.cloudillo.net'  // Optional target audience
}

// Add a comment
const comment: CommentAction = {
  type: 'CMNT',
  parentId: 'actionId123',           // Parent action to comment on
  content: 'Great post!',
  attachments: []                    // Optional
}

// Add a reaction
const reaction: ReactAction = {
  type: 'REACT',
  parentId: 'actionId123',           // Action to react to
  content: 'LOVE'                    // Reaction type
}

Content Spreading

import type { AckAction, RepostAction, ShareAction } from '@cloudillo/types'

// Acknowledge content (accept to feed)
const ack: AckAction = {
  type: 'ACK',
  parentId: 'actionId123'            // Action to acknowledge
}

// Repost content
const repost: RepostAction = {
  type: 'REPOST',
  parentId: 'actionId123',           // Original action
  content: 'Adding my thoughts...'   // Optional comment
}

// Share directly to someone
const share: ShareAction = {
  type: 'SHRE',
  subject: 'actionId123',            // What to share
  audience: 'bob.cloudillo.net',     // Who to share with
  content: 'You might like this!'    // Optional message
}

Messages

import type { MessageAction } from '@cloudillo/types'

// Direct message
const dm: MessageAction = {
  type: 'MSG',
  subType: 'TEXT',
  content: 'Hello!',
  audience: 'bob.cloudillo.net'      // Recipient
}

// Group message (reply to conversation)
const groupMsg: MessageAction = {
  type: 'MSG',
  subType: 'TEXT',
  content: 'Thanks everyone!',
  parentId: 'conversationId123'      // Conversation ID
}

File Sharing

import type { FileShareAction } from '@cloudillo/types'

// Share a file with read access
const fileShare: FileShareAction = {
  type: 'FSHR',
  subType: 'READ',                   // 'READ' or 'WRITE'
  subject: 'fileId123',              // File to share
  audience: 'bob.cloudillo.net',     // Who to share with
  content: {
    fileName: 'document.pdf',
    contentType: 'application/pdf',
    fileTp: 'BLOB'
  }
}

Reaction Statistics

import type { ReactionStatAction } from '@cloudillo/types'

// Aggregated reaction stats (usually system-generated)
const stats: ReactionStatAction = {
  type: 'RSTAT',
  parentId: 'actionId123',
  content: {
    comment: 5,                      // Number of comments
    reactions: [10, 3, 1]            // Reaction counts by type
  }
}

ProfileInfo

Embedded profile information in actions.

import type { ProfileInfo } from '@cloudillo/types'

const info: ProfileInfo = {
  idTag: 'alice.cloudillo.net',
  name: 'Alice',                     // Optional
  profilePic: 'picId123',            // Optional
  type: 'person'                     // 'person' or 'community'
}

See Also

@cloudillo/canvas-tools

Overview

React components and hooks for interactive object manipulation in SVG canvas applications. Provides transform gizmos, rotation handles, pivot controls, and gradient pickers for building drawing and design tools.

Installation

pnpm add @cloudillo/canvas-tools

Peer Dependencies:

  • react >= 18
  • react-svg-canvas

Components

TransformGizmo

Complete transform control for SVG objects with rotation, scaling, and positioning.

import { TransformGizmo } from '@cloudillo/canvas-tools'

function CanvasEditor() {
  const [bounds, setBounds] = useState({ x: 100, y: 100, width: 200, height: 150 })
  const [rotation, setRotation] = useState(0)

  return (
    <svg width={800} height={600}>
      <TransformGizmo
        bounds={bounds}
        rotation={rotation}
        onBoundsChange={setBounds}
        onRotationChange={setRotation}
        showRotationHandle
        showPivotHandle
      />
    </svg>
  )
}

Props (TransformGizmoProps):

  • bounds: Bounds - Object position and size { x, y, width, height }
  • rotation?: number - Rotation angle in degrees
  • pivot?: Point - Pivot point { x, y }
  • onBoundsChange?: (bounds: Bounds) => void - Bounds change callback
  • onRotationChange?: (angle: number) => void - Rotation change callback
  • onPivotChange?: (pivot: Point) => void - Pivot change callback
  • showRotationHandle?: boolean - Show rotation arc handle
  • showPivotHandle?: boolean - Show pivot point control

RotationHandle

Circular arc handle for rotating objects.

import { RotationHandle } from '@cloudillo/canvas-tools'

<RotationHandle
  center={{ x: 200, y: 200 }}
  radius={80}
  currentAngle={rotation}
  onRotate={(angle) => setRotation(angle)}
  snapAngles={[0, 45, 90, 135, 180, 225, 270, 315]}
/>

Props (RotationHandleProps):

  • center: Point - Center point of rotation
  • radius: number - Arc radius
  • currentAngle: number - Current rotation angle
  • onRotate: (angle: number) => void - Rotation callback
  • snapAngles?: number[] - Angles to snap to (default: 45-degree increments)
  • snapZoneRatio?: number - Snap zone size ratio

PivotHandle

Draggable handle for setting the pivot/rotation center point.

import { PivotHandle } from '@cloudillo/canvas-tools'

<PivotHandle
  position={{ x: 200, y: 200 }}
  bounds={{ x: 100, y: 100, width: 200, height: 200 }}
  onPivotChange={(point) => setPivot(point)}
  snapToCenter
  snapThreshold={10}
/>

Props (PivotHandleProps):

  • position: Point - Current pivot position
  • bounds: Bounds - Object bounds for snapping
  • onPivotChange: (point: Point) => void - Position change callback
  • snapToCenter?: boolean - Snap to center when close
  • snapThreshold?: number - Snap distance threshold

GradientPicker

Complete gradient editor with color stops, angle control, and presets.

import { GradientPicker } from '@cloudillo/canvas-tools'
import type { Gradient } from '@cloudillo/canvas-tools'

function GradientEditor() {
  const [gradient, setGradient] = useState<Gradient>({
    type: 'linear',
    angle: 90,
    stops: [
      { offset: 0, color: '#ff0000' },
      { offset: 1, color: '#0000ff' }
    ]
  })

  return (
    <GradientPicker
      value={gradient}
      onChange={setGradient}
      showPresets
      showAngleControl
    />
  )
}

Props (GradientPickerProps):

  • value: Gradient - Current gradient value
  • onChange: (gradient: Gradient) => void - Change callback
  • showPresets?: boolean - Show gradient preset grid
  • showAngleControl?: boolean - Show angle rotation control
  • showPositionControl?: boolean - Show radial gradient position control

GradientBar

Horizontal bar for editing gradient color stops.

import { GradientBar } from '@cloudillo/canvas-tools'

<GradientBar
  stops={gradient.stops}
  onStopsChange={(stops) => setGradient({ ...gradient, stops })}
  selectedStop={selectedIndex}
  onSelectStop={setSelectedIndex}
/>

GradientPresetGrid

Grid of predefined gradient presets.

import { GradientPresetGrid, GRADIENT_PRESETS } from '@cloudillo/canvas-tools'

<GradientPresetGrid
  presets={GRADIENT_PRESETS}
  onSelect={(preset) => setGradient(expandGradient(preset.gradient))}
  category="warm"
/>

AngleControl

Circular control for setting gradient angle.

import { AngleControl, DEFAULT_ANGLE_PRESETS } from '@cloudillo/canvas-tools'

<AngleControl
  value={angle}
  onChange={setAngle}
  presets={DEFAULT_ANGLE_PRESETS}
/>

PositionControl

XY position control for radial gradient centers.

import { PositionControl } from '@cloudillo/canvas-tools'

<PositionControl
  value={{ x: 0.5, y: 0.5 }}
  onChange={setPosition}
/>

Hooks

useTransformGizmo

Hook for managing transform gizmo state and interactions.

import { useTransformGizmo } from '@cloudillo/canvas-tools'

function CanvasObject({ object, onUpdate }) {
  const {
    state,
    handlers,
    isDragging,
    isRotating,
    isResizing
  } = useTransformGizmo({
    bounds: object.bounds,
    rotation: object.rotation,
    pivot: object.pivot,
    onBoundsChange: (bounds) => onUpdate({ ...object, bounds }),
    onRotationChange: (rotation) => onUpdate({ ...object, rotation }),
    onPivotChange: (pivot) => onUpdate({ ...object, pivot }),
    snapAngles: [0, 45, 90, 135, 180, 225, 270, 315],
    maintainAspectRatio: true
  })

  return (
    <g {...handlers}>
      {/* Object rendering */}
    </g>
  )
}

Options (TransformGizmoOptions):

  • bounds: Bounds - Initial bounds
  • rotation?: number - Initial rotation
  • pivot?: Point - Initial pivot
  • onBoundsChange?: (bounds: Bounds) => void
  • onRotationChange?: (angle: number) => void
  • onPivotChange?: (pivot: Point) => void
  • snapAngles?: number[] - Rotation snap angles
  • snapZoneRatio?: number - Snap sensitivity
  • maintainAspectRatio?: boolean - Lock aspect during resize
  • minWidth?: number - Minimum width constraint
  • minHeight?: number - Minimum height constraint

Returns (UseTransformGizmoReturn):

  • state: TransformGizmoState - Current transform state
  • handlers: TransformGizmoHandlers - Event handlers
  • isDragging: boolean - Move operation active
  • isRotating: boolean - Rotation operation active
  • isResizing: boolean - Resize operation active

Gradient Utilities

Creating Gradients

import {
  DEFAULT_LINEAR_GRADIENT,
  DEFAULT_RADIAL_GRADIENT,
  expandGradient,
  compactGradient
} from '@cloudillo/canvas-tools'

// Start with defaults
const linear = { ...DEFAULT_LINEAR_GRADIENT }
const radial = { ...DEFAULT_RADIAL_GRADIENT }

// Expand compact notation to full gradient
const full = expandGradient({ t: 'l', a: 90, s: [[0, '#f00'], [1, '#00f']] })

// Compact for storage
const compact = compactGradient(full)

Manipulating Stops

import {
  addStop,
  removeStop,
  updateStop,
  sortStops,
  reverseStops
} from '@cloudillo/canvas-tools'

// Add a stop at 50%
const newStops = addStop(gradient.stops, 0.5, '#00ff00')

// Remove stop at index 1
const filtered = removeStop(gradient.stops, 1)

// Update stop color
const updated = updateStop(gradient.stops, 1, { color: '#ff00ff' })

// Sort by offset
const sorted = sortStops(stops)

// Reverse direction
const reversed = reverseStops(stops)

Converting to CSS/SVG

import {
  gradientToCSS,
  createLinearGradientDef,
  createRadialGradientDef
} from '@cloudillo/canvas-tools'

// CSS background value
const css = gradientToCSS(gradient)
// "linear-gradient(90deg, #ff0000 0%, #0000ff 100%)"

// SVG gradient definition
const svgLinear = createLinearGradientDef('myGradient', gradient)
const svgRadial = createRadialGradientDef('myGradient', gradient)

Color Utilities

import {
  interpolateColor,
  getColorAtPosition
} from '@cloudillo/canvas-tools'

// Blend two colors
const mixed = interpolateColor('#ff0000', '#0000ff', 0.5)
// "#800080"

// Get color at position in gradient
const colorAt25 = getColorAtPosition(gradient.stops, 0.25)

Geometry Utilities

Coordinate Transformation

import {
  getCanvasCoordinates,
  getCanvasCoordinatesWithElement,
  getSvgElement
} from '@cloudillo/canvas-tools'

function handleClick(event: MouseEvent) {
  const svg = getSvgElement(event.target as Element)
  const point = getCanvasCoordinates(event, svg)
  console.log('Canvas position:', point.x, point.y)
}

Rotation Matrix

Pre-calculated trigonometry for performance-critical operations.

import {
  createRotationMatrix,
  rotatePointWithMatrix,
  unrotatePointWithMatrix,
  rotateDeltaWithMatrix,
  unrotateDeltaWithMatrix
} from '@cloudillo/canvas-tools'

// Create matrix once
const matrix = createRotationMatrix(45) // 45 degrees

// Rotate points efficiently
const rotated = rotatePointWithMatrix({ x: 100, y: 0 }, matrix, center)
const original = unrotatePointWithMatrix(rotated, matrix, center)

// Rotate deltas (movement vectors)
const rotatedDelta = rotateDeltaWithMatrix({ x: 10, y: 0 }, matrix)

View Coordinates

import {
  canvasToView,
  viewToCanvas,
  isPointInView,
  boundsIntersectsView
} from '@cloudillo/canvas-tools'

// Convert between canvas and view coordinates
const viewPoint = canvasToView(canvasPoint, viewTransform)
const canvasPoint = viewToCanvas(viewPoint, viewTransform)

// Visibility checks
const visible = isPointInView(point, viewport)
const intersects = boundsIntersectsView(objectBounds, viewport)

Resize Calculations

import {
  initResizeState,
  calculateResizeBounds,
  getAnchorForHandle,
  getRotatedAnchorPosition
} from '@cloudillo/canvas-tools'

// Initialize resize operation
const resizeState = initResizeState(
  bounds,
  rotation,
  'se', // corner handle
  startMousePosition
)

// Calculate new bounds during drag
const newBounds = calculateResizeBounds(
  resizeState,
  currentMousePosition,
  { maintainAspectRatio: true }
)

Types

Core Types

import type {
  Point,
  Bounds,
  ResizeHandle,
  RotationState,
  PivotState,
  RotatedObjectBounds,
  TransformedObject,
  RotationMatrix,
  ResizeState
} from '@cloudillo/canvas-tools'

type Point = { x: number; y: number }

type Bounds = {
  x: number
  y: number
  width: number
  height: number
}

type ResizeHandle = 'n' | 's' | 'e' | 'w' | 'ne' | 'nw' | 'se' | 'sw'

Gradient Types

import type {
  GradientType,
  GradientStop,
  Gradient,
  CompactGradient,
  GradientPreset,
  GradientPresetCategory
} from '@cloudillo/canvas-tools'

type GradientType = 'linear' | 'radial'

type GradientStop = {
  offset: number  // 0-1
  color: string   // hex color
}

type Gradient = {
  type: GradientType
  angle?: number        // linear: degrees
  cx?: number           // radial: center x (0-1)
  cy?: number           // radial: center y (0-1)
  stops: GradientStop[]
}

type CompactGradient = {
  t: 'l' | 'r'          // type
  a?: number            // angle
  cx?: number
  cy?: number
  s: [number, string][] // stops as tuples
}

Constants

Rotation Defaults

import {
  DEFAULT_SNAP_ANGLES,
  DEFAULT_SNAP_ZONE_RATIO,
  DEFAULT_PIVOT_SNAP_POINTS,
  DEFAULT_PIVOT_SNAP_THRESHOLD
} from '@cloudillo/canvas-tools'

// [0, 45, 90, 135, 180, 225, 270, 315]
console.log(DEFAULT_SNAP_ANGLES)

// 0.1 (10% of arc)
console.log(DEFAULT_SNAP_ZONE_RATIO)

Arc Sizing

import {
  ARC_RADIUS_MIN_VIEWPORT_RATIO,
  ARC_RADIUS_MAX_VIEWPORT_RATIO,
  DEFAULT_ARC_PADDING,
  calculateArcRadius
} from '@cloudillo/canvas-tools'

const radius = calculateArcRadius({
  objectBounds: bounds,
  viewportSize: { width: 800, height: 600 },
  padding: DEFAULT_ARC_PADDING
})

Angle Presets

import { DEFAULT_ANGLE_PRESETS } from '@cloudillo/canvas-tools'

// [0, 45, 90, 135, 180, 225, 270, 315]
console.log(DEFAULT_ANGLE_PRESETS)

Gradient Presets

import {
  GRADIENT_PRESETS,
  getPresetsByCategory,
  getPresetById,
  getCategories
} from '@cloudillo/canvas-tools'

// Get all categories
const categories = getCategories()
// ['warm', 'cool', 'vibrant', 'subtle', 'monochrome']

// Get presets in a category
const warmGradients = getPresetsByCategory('warm')

// Get specific preset
const sunset = getPresetById('sunset')

See Also

@cloudillo/fonts

Overview

The @cloudillo/fonts library provides a curated collection of Google Fonts metadata and pairing suggestions for Cloudillo applications. It enables font selection UIs, typography systems, and design tools.

Key Features:

  • Curated font metadata for 22 Google Fonts
  • Pre-defined font pairings (heading + body combinations)
  • Helper functions for filtering and lookup
  • Full TypeScript support

Installation

pnpm add @cloudillo/fonts

Font Metadata API

FONTS Constant

The FONTS array contains metadata for all available fonts.

import { FONTS } from '@cloudillo/fonts'

// List all fonts
FONTS.forEach(font => {
  console.log(font.displayName, font.category, font.roles)
})

Each font entry includes:

  • family - CSS font-family value (e.g., 'Roboto')
  • displayName - Human-readable name
  • category - 'sans-serif' | 'serif' | 'display' | 'monospace'
  • roles - Suitable uses: 'heading' | 'body' | 'display' | 'mono'
  • weights - Available font weights
  • hasItalic - Whether italic variants exist
  • license - 'OFL' or 'Apache-2.0'
  • directory - Font directory name

getFontByFamily

Look up a font by its family name.

import { getFontByFamily } from '@cloudillo/fonts'

const roboto = getFontByFamily('Roboto')
// { family: 'Roboto', category: 'sans-serif', roles: ['body', 'heading'], ... }

const unknown = getFontByFamily('Unknown Font')
// undefined

getFontsByCategory

Filter fonts by category.

import { getFontsByCategory } from '@cloudillo/fonts'

const serifFonts = getFontsByCategory('serif')
// Returns: Playfair Display, Merriweather, Lora, Crimson Pro, Source Serif 4, DM Serif Display

const displayFonts = getFontsByCategory('display')
// Returns: Oswald, Bebas Neue, Abril Fatface, Permanent Marker

Available categories:

  • sans-serif - Clean, modern fonts (Roboto, Open Sans, Montserrat, etc.)
  • serif - Traditional fonts with serifs (Playfair Display, Merriweather, etc.)
  • display - Decorative fonts for headlines (Oswald, Bebas Neue, etc.)
  • monospace - Fixed-width fonts (JetBrains Mono)

getFontsByRole

Filter fonts by intended use.

import { getFontsByRole } from '@cloudillo/fonts'

const headingFonts = getFontsByRole('heading')
// Fonts suitable for headings: Roboto, Montserrat, Poppins, Playfair Display, etc.

const bodyFonts = getFontsByRole('body')
// Fonts suitable for body text: Roboto, Open Sans, Lato, Inter, etc.

Available roles:

  • heading - Suitable for titles and headings
  • body - Suitable for body text
  • display - Decorative, for large display text
  • mono - Monospace, for code

Font Pairings API

The library includes curated heading + body font combinations that work well together.

FONT_PAIRINGS Constant

import { FONT_PAIRINGS } from '@cloudillo/fonts'

FONT_PAIRINGS.forEach(pairing => {
  console.log(`${pairing.name}: ${pairing.heading} + ${pairing.body}`)
})

Available pairings:

ID Name Heading Body Description
modern-professional Modern Professional Oswald Roboto Business presentations
elegant-editorial Elegant Editorial Playfair Display Source Sans 3 Articles and long-form
clean-modern Clean Modern Montserrat Open Sans Tech and startups
readable-classic Readable Classic Merriweather Lato Blogs and docs
contemporary-tech Contemporary Tech Poppins Inter Digital products
literary-warm Literary Warm Lora Nunito Sans Classic with friendly body
light-minimalist Light Minimalist Raleway Work Sans Minimal designs
academic-formal Academic Formal Crimson Pro DM Sans Scholarly content
bold-impact Bold Impact Bebas Neue Source Serif 4 Impactful headlines
geometric-harmony Geometric Harmony DM Serif Display DM Sans Cohesive DM family

getPairingById

Look up a specific pairing.

import { getPairingById } from '@cloudillo/fonts'

const pairing = getPairingById('modern-professional')
// { id: 'modern-professional', name: 'Modern Professional', heading: 'Oswald', body: 'Roboto', ... }

getPairingsForFont

Find pairings that use a specific font.

import { getPairingsForFont } from '@cloudillo/fonts'

const robotoPairings = getPairingsForFont('Roboto')
// Returns pairings where Roboto is used as heading or body

getSuggestedBodyFonts

Get body font suggestions for a heading font.

import { getSuggestedBodyFonts } from '@cloudillo/fonts'

const bodyOptions = getSuggestedBodyFonts('Oswald')
// ['Roboto']

const playfairBodies = getSuggestedBodyFonts('Playfair Display')
// ['Source Sans 3']

getSuggestedHeadingFonts

Get heading font suggestions for a body font.

import { getSuggestedHeadingFonts } from '@cloudillo/fonts'

const headingOptions = getSuggestedHeadingFonts('Inter')
// ['Poppins']

const latoHeadings = getSuggestedHeadingFonts('Lato')
// ['Merriweather']

TypeScript Types

import type {
  FontCategory,
  FontRole,
  FontWeight,
  FontMetadata,
  FontPairing
} from '@cloudillo/fonts'

FontCategory

type FontCategory = 'sans-serif' | 'serif' | 'display' | 'monospace'

FontRole

type FontRole = 'heading' | 'body' | 'display' | 'mono'

FontWeight

interface FontWeight {
  value: number    // CSS font-weight value (400, 700, etc.)
  label: string    // Display name ('Regular', 'Bold', etc.)
  italic?: boolean // Whether this is an italic variant
}

FontMetadata

interface FontMetadata {
  family: string           // CSS font-family value
  displayName: string      // Human-readable name
  category: FontCategory   // Font category
  roles: FontRole[]        // Suitable roles
  weights: FontWeight[]    // Available weights
  hasItalic: boolean       // Has italic variants
  license: 'OFL' | 'Apache-2.0'
  directory: string        // Font directory name
}

FontPairing

interface FontPairing {
  id: string          // Unique identifier
  name: string        // Human-readable name
  heading: string     // Heading font family
  body: string        // Body font family
  description: string // Pairing description
}

Integration with FontPicker

The @cloudillo/fonts library works with the FontPicker component from @cloudillo/react.

import { FontPicker } from '@cloudillo/react'
import { FONTS, FONT_PAIRINGS, getSuggestedBodyFonts } from '@cloudillo/fonts'

function TypographySettings() {
  const [headingFont, setHeadingFont] = useState('Montserrat')
  const [bodyFont, setBodyFont] = useState('Open Sans')

  // Get suggested body fonts when heading changes
  const suggestedBodies = getSuggestedBodyFonts(headingFont)

  return (
    <div>
      <FontPicker
        label="Heading Font"
        value={headingFont}
        onChange={setHeadingFont}
      />
      <FontPicker
        label="Body Font"
        value={bodyFont}
        onChange={setBodyFont}
      />
      {suggestedBodies.length > 0 && (
        <p>Suggested body fonts: {suggestedBodies.join(', ')}</p>
      )}
    </div>
  )
}

Available Fonts

Sans-Serif

Font Roles Weights
Roboto body, heading 400, 700
Open Sans body 400, 700
Montserrat heading, body 400, 700
Lato body 400, 700
Poppins heading, body 400, 700
Inter body 400, 700
Nunito Sans body 400, 700
Work Sans body 400, 700
Raleway heading 400, 700
DM Sans body 400, 700
Source Sans 3 body 400, 700

Serif

Font Roles Weights
Playfair Display heading, display 400, 700
Merriweather heading, body 400, 700
Lora heading, body 400, 700
Crimson Pro heading, body 400, 700
Source Serif 4 body 400, 700
DM Serif Display heading, display 400

Display

Font Roles Weights
Oswald heading, display 400, 700
Bebas Neue heading, display 400
Abril Fatface display 400
Permanent Marker display 400

Monospace

Font Roles Weights
JetBrains Mono mono 400, 700

See Also

Common Patterns

Idiomatic ways to combine Cloudillo APIs when building apps. Each pattern shows the minimal Cloudillo-specific code — wrap in your own components as needed. For full API details, see the linked reference pages.

Data loading

Paginated lists (infinite scroll)

Cloudillo list APIs use cursor-based pagination. The useInfiniteScroll hook from @cloudillo/react manages cursor tracking, accumulates items across pages, and triggers loading via IntersectionObserver when a sentinel element becomes visible.

import { useInfiniteScroll } from '@cloudillo/react'

const { items, isLoading, hasMore, sentinelRef, prepend } = useInfiniteScroll({
  fetchPage: async (cursor, limit) => {
    const result = await api.actions.listPaginated({ type: 'POST', cursor, limit })
    return {
      items: result.data,
      nextCursor: result.cursorPagination?.nextCursor ?? null,
      hasMore: result.cursorPagination?.hasMore ?? false
    }
  },
  pageSize: 20,
  deps: [filterType]  // resets when dependencies change
})

Attach sentinelRef to a <div> at the bottom of your list — it automatically triggers loadMore when scrolled into view. Use prepend() to insert real-time updates at the top without resetting the list.

See: @cloudillo/react | Actions API

The profiles API supports text search. Combine with your own debounce logic to reduce API calls:

const results = await api.profiles.list({ q: searchQuery, limit: 10 })

See: Profiles API

Real-time collaboration

Collaborative editing (CRDT)

The useCloudilloEditor hook sets up a Yjs document with WebSocket sync. It returns a Y.Doc and a WebsocketProvider — bind these to any Yjs-compatible editor (Quill, ProseMirror, CodeMirror, BlockNote). Wait for synced before initializing the editor binding.

import { useCloudilloEditor } from '@cloudillo/react'

const { yDoc, provider, synced, error, ownerTag, fileId } = useCloudilloEditor('my-app')

// Once synced, bind to your editor:
const yText = yDoc.getText('content')
const binding = new QuillBinding(yText, quill, provider.awareness)

The error field captures connection errors (e.g., 440x codes from the CRDT server) so you can show appropriate UI.

See: CRDT (Collaborative Editing) | @cloudillo/react

Presence awareness

The provider from useCloudilloEditor includes a Yjs awareness instance. Subscribe to it to show who is currently viewing or editing:

provider.awareness.on('change', () => {
  const states = provider.awareness.getStates()
  // Each state contains { user: { name, profilePic, ... } }
  // Filter out own clientID: provider.awareness.clientID
})

See: CRDT (Collaborative Editing)

Social features

Connection requests

Cloudillo connections are bidirectional. Send a request by creating a CONN action, and accept incoming requests with accept:

// Send a connection request
await api.actions.create({
  type: 'CONN',
  subject: profile.idTag,
  content: 'Would love to connect!'
})

// Accept an incoming request
await api.actions.accept(actionId)

Check profile.connected for the current connection state:

  • true — connected
  • 'R' — request pending
  • absent — not connected

See: Actions API

Role-based access

Community roles follow a numeric hierarchy defined in ROLE_LEVELS. Use useAuth() to get the current user’s roles and compare:

import { ROLE_LEVELS, type CommunityRole } from '@cloudillo/types'

function hasRole(userRoles: string[] | undefined, required: CommunityRole): boolean {
  if (!userRoles?.length) return false
  const level = Math.max(...userRoles.map(r => ROLE_LEVELS[r as CommunityRole] ?? 0))
  return level >= ROLE_LEVELS[required]
}

// Usage: hasRole(auth?.roles, 'moderator')

Role hierarchy: public < follower < supporter < contributor < moderator < leader.

See: @cloudillo/react | @cloudillo/types

File management

Uploading files

Use api.files.uploadBlob with a preset name. The preset determines what variants (thumbnail, standard definition) are automatically generated server-side:

const result = await api.files.uploadBlob(
  'gallery',    // preset: generates thumbnail + SD variants
  file.name,
  file,
  file.type
)
// result: { fileId, variantId }

See: Files API

General tips

  • Use useToast() from @cloudillo/react for user feedback after API calls — see components reference
  • Wrap route-level components in an error boundary for graceful failure handling — see error handling
  • For reactions and toggles, use optimistic UI updates: save previous state, update immediately, rollback in catch
  • All list APIs support cursor pagination — prefer useInfiniteScroll over manual fetch loops
  • Check api is not null (user is authenticated) before making API calls
  • CRDT and RTDB serve different use cases: CRDT for rich document editing, RTDB for structured data collections — see RTDB and CRDT

See also

REST API

REST API Reference

Cloudillo provides a comprehensive REST API for building applications. All endpoints return JSON and use standard HTTP methods.

Base URL

https://your-cloudillo-server.com

For local development:

http://localhost:3000

Authentication

Most endpoints require authentication via JWT tokens in the Authorization header:

Authorization: Bearer eyJhbGciOiJFUzM4NCIsInR5cCI6IkpXVCJ9...

See Authentication for details on obtaining and managing tokens.

Response Format

All successful responses follow this format:

{
  "data": <payload>,
  "time": "2025-01-01T12:00:00Z",
  "reqId": "req_abc123"
}

For list endpoints, cursor-based pagination is recommended:

{
  "data": [...],
  "cursorPagination": {
    "nextCursor": "eyJzIjoiY3JlYXRlZCIsInYiOjE3MzUwMDAwMDAsImlkIjoiYTF-YWJjMTIzIn0",
    "hasMore": true
  },
  "time": "2025-01-01T12:00:00Z"
}

Legacy offset-based pagination (deprecated):

{
  "data": [...],
  "pagination": {
    "total": 150,
    "offset": 0,
    "limit": 20
  },
  "time": "2025-01-01T12:00:00Z"
}

Error Format

Errors return this structure:

{
  "error": {
    "code": "E-AUTH-UNAUTH",
    "message": "Unauthorized access",
    "details": {...}
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

See Error Handling for all error codes.

Common Query Parameters

Many list endpoints support these parameters:

  • limit - Maximum number of results (default: 20)
  • cursor - Opaque cursor for pagination (from previous response)
  • sort - Sort field (e.g., created, modified, name)
  • sortDir - Sort direction (asc or desc)

Endpoint Categories

Authentication

User authentication and token management.

  • POST /api/auth/login - Login and get token
  • POST /api/auth/login-init - Combined login init (token + QR + WebAuthn)
  • POST /api/auth/logout - Logout
  • POST /api/auth/password - Change password
  • GET /api/auth/login-token - Refresh token
  • GET /api/auth/access-token - Get scoped token
  • GET /api/auth/proxy-token - Get federation token
  • POST /api/auth/qr-login/init - QR login init
  • GET /api/auth/qr-login/{session_id}/status - QR login status
  • GET /api/me - Get tenant profile (public)
  • GET /.well-known/cloudillo/id-tag - Resolve identity

Profiles

User and community profiles.

  • POST /api/profiles/register - Register new user
  • POST /api/profiles/verify - Verify identity availability
  • GET /api/me - Get own profile
  • PATCH /api/me - Update own profile
  • PUT /api/me/image - Upload profile image
  • PUT /api/me/cover - Upload cover image
  • GET /api/profiles - List profiles
  • GET /api/profiles/:idTag - Get specific profile
  • PATCH /api/profiles/:idTag - Update relationship
  • PATCH /api/admin/profiles/:idTag - Admin update profile

Actions

Social features: posts, comments, reactions, connections.

  • GET /api/actions - List actions
  • POST /api/actions - Create action
  • GET /api/actions/{actionId} - Get action
  • PATCH /api/actions/{actionId} - Update draft action
  • DELETE /api/actions/{actionId} - Delete action
  • POST /api/actions/{actionId}/accept - Accept action
  • POST /api/actions/{actionId}/reject - Reject action
  • POST /api/actions/{actionId}/dismiss - Dismiss notification
  • POST /api/actions/{actionId}/publish - Publish draft
  • POST /api/actions/{actionId}/cancel - Cancel scheduled
  • POST /api/actions/{actionId}/stat - Update statistics
  • POST /api/actions/{actionId}/reaction - Add reaction
  • POST /api/inbox - Federation inbox (async)
  • POST /api/inbox/sync - Federation inbox (sync)

Files

File upload, download, and management.

  • GET /api/files - List files
  • POST /api/files - Create file metadata (CRDT/RTDB)
  • POST /api/files/{preset}/{file_name} - Upload file (BLOB)
  • GET /api/files/{fileId} - Download file
  • GET /api/files/{fileId}/descriptor - Get file info
  • GET /api/files/{fileId}/metadata - Get file metadata
  • PATCH /api/files/{fileId} - Update file
  • DELETE /api/files/{fileId} - Delete file
  • POST /api/files/{fileId}/duplicate - Duplicate CRDT/RTDB file
  • POST /api/files/{fileId}/restore - Restore from trash
  • PUT /api/files/{fileId}/tag/{tag} - Add tag
  • DELETE /api/files/{fileId}/tag/{tag} - Remove tag
  • GET /api/files/variant/{variantId} - Get variant

Apps

App package management.

  • GET /api/apps - List available apps
  • POST /api/apps/install - Install app
  • GET /api/apps/installed - List installed apps
  • DELETE /api/apps/@{publisher}/{name} - Uninstall app

Shares

File sharing and access grants.

  • GET /api/shares - List shares by subject
  • GET /api/files/{fileId}/shares - List file shares
  • POST /api/files/{fileId}/shares - Create share
  • DELETE /api/files/{fileId}/shares/{shareId} - Delete share

Settings

User preferences and configuration.

  • GET /api/settings - List all settings
  • GET /api/settings/:name - Get setting
  • PUT /api/settings/:name - Update setting

References

Bookmarks and shortcuts.

  • GET /api/refs - List references
  • POST /api/refs - Create reference
  • GET /api/refs/:refId - Get reference
  • DELETE /api/refs/:refId - Delete reference

Tags

File and content tagging.

  • GET /api/tags - List tags
  • PUT /api/files/:fileId/tag/:tag - Add tag
  • DELETE /api/files/:fileId/tag/:tag - Remove tag

Trash

Trash management.

  • GET /api/files?parentId=__trash__ - List trashed files
  • POST /api/files/:fileId/restore - Restore from trash
  • DELETE /api/files/:fileId?permanent=true - Permanently delete
  • DELETE /api/trash - Empty trash

Communities

Community creation and management.

  • PUT /api/profiles/:idTag - Create community
  • POST /api/profiles/verify - Verify availability

Admin

System administration (requires admin role).

  • GET /api/admin/tenants - List tenants
  • POST /api/admin/tenants/{idTag}/password-reset - Send password reset
  • POST /api/admin/email/test - Test SMTP
  • PATCH /api/admin/profiles/{idTag} - Admin profile update
  • GET /api/admin/proxy-sites - List proxy sites
  • POST /api/admin/proxy-sites - Create proxy site
  • PATCH /api/admin/proxy-sites/{siteId} - Update proxy site
  • DELETE /api/admin/proxy-sites/{siteId} - Delete proxy site
  • POST /api/admin/proxy-sites/{siteId}/renew-cert - Renew certificate
  • POST /api/admin/invite-community - Invite community

IDP Management

Identity provider administration.

  • GET /api/idp/identities - List managed identities
  • POST /api/idp/identities - Create identity
  • GET /api/idp/identities/:idTag - Get identity
  • PATCH /api/idp/identities/:idTag - Update identity
  • PUT /api/idp/identities/:idTag/address - Update identity address (DNS)
  • DELETE /api/idp/identities/:idTag - Delete identity
  • GET /api/idp/api-keys - List API keys
  • POST /api/idp/api-keys - Create API key
  • DELETE /api/idp/api-keys/:keyId - Delete API key

Rate Limiting

API requests are rate-limited per tenant:

  • Default: 1000 requests per minute
  • Authenticated: 5000 requests per minute
  • Admin: Unlimited

Rate limit headers:

X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4950
X-RateLimit-Reset: 1735000000

CORS

CORS is enabled for all origins in development mode. In production, configure allowed origins in the server settings.

Timestamps

All response timestamps are in ISO 8601 format:

{
  "createdAt": "2025-01-01T12:00:00Z"
}

Query parameter timestamps accept both ISO 8601 strings and Unix seconds:

GET /api/actions?createdAfter=2025-01-01T00:00:00Z
GET /api/actions?createdAfter=1735689600

Content Types

Request Content-Type

Most endpoints accept:

Content-Type: application/json

File uploads use:

Content-Type: multipart/form-data

Response Content-Type

All responses return:

Content-Type: application/json; charset=utf-8

Except file downloads which return the appropriate MIME type.

HTTP Methods

  • GET - Retrieve resources
  • POST - Create resources
  • PATCH - Partially update resources
  • PUT - Replace resources
  • DELETE - Delete resources

Idempotency

PUT, PATCH, and DELETE operations are idempotent. POST operations are not idempotent unless you provide an idempotencyKey:

{
  "idempotencyKey": "unique-key-123",
  "type": "POST",
  "content": {...}
}

Pagination

List endpoints use cursor-based pagination for stable results:

GET /api/actions?limit=20

Response includes pagination info:

{
  "data": [...],
  "cursorPagination": {
    "nextCursor": "eyJzIjoiY3JlYXRlZCIsInYiOjE3MzUwMDAwMDAsImlkIjoiYTF-YWJjMTIzIn0",
    "hasMore": true
  },
  "time": "2025-01-01T12:00:00Z"
}

To fetch the next page, use the cursor:

GET /api/actions?limit=20&cursor=eyJzIjoiY3JlYXRlZCIsInYiOjE3MzUwMDAwMDAsImlkIjoiYTF-YWJjMTIzIn0

Filtering

Many endpoints support filtering via query parameters. Each endpoint documents its available filters.

GET /api/actions?type=POST&status=A&createdAfter=2025-01-01T00:00:00Z

WebSocket Endpoints

For real-time features, use WebSocket connections:

  • WSS /ws/crdt/{doc_id} - Collaborative documents
  • WSS /ws/rtdb/{file_id} - Real-time database
  • WSS /ws/bus - Message bus

See WebSocket API for details.

Quick Start

Using @cloudillo/core

import * as cloudillo from '@cloudillo/core'

await cloudillo.init('my-app')
const api = cloudillo.createApiClient()

// Make requests
const profile = await api.profiles.getOwn()
const posts = await api.actions.list({ type: 'POST' })

Using fetch directly

const response = await fetch('/api/me', {
  headers: {
    'Authorization': `Bearer ${token}`,
    'Content-Type': 'application/json'
  }
})

const data = await response.json()
console.log(data.data) // Profile object
console.log(data.time) // Timestamp
console.log(data.reqId) // Request ID

Cursor Pagination Details

Cursors are opaque base64-encoded strings containing:

  • Sort field (s)
  • Sort value (v)
  • Last item ID (id)
{
  "s": "created",
  "v": 1735000000,
  "id": "a1~abc123"
}

Benefits:

  • Stable: Results don’t shift when new items are added
  • Efficient: No offset scanning in database
  • Reliable: Works with large datasets

Use the SDK for easier pagination:

let cursor = undefined
while (true) {
  const result = await api.actions.list({ type: 'POST', limit: 20, cursor })
  // Process result.data
  if (!result.cursorPagination?.hasMore) break
  cursor = result.cursorPagination.nextCursor
}

Next Steps

Explore specific endpoint categories:

Subsections of REST API

Authentication API

Overview

User authentication and token management endpoints. For user registration, see Profiles API.

Endpoints

Login

POST /api/auth/login

Authenticate with email/password and receive an access token.

Authentication: Not required

Request:

{
  "idTag": "alice@example.com",
  "password": "secure-password"
}

Response:

{
  "data": {
    "tnId": 12345,
    "idTag": "alice@example.com",
    "name": "Alice Johnson",
    "token": "eyJhbGc...",
    "roles": ["user"],
    "profilePic": "b1~abc123",
    "settings": [["theme", "dark"], ["lang", "en"]],
    "swEncryptionKey": "base64url-encoded-key"
  },
  "time": "2025-01-01T12:00:00Z"
}
Field Type Description
tnId number Tenant ID
idTag string User identity tag
name string Display name
token string JWT access token
roles string[] User roles (e.g., user, SADM)
profilePic string Profile picture file ID
settings [string, string][] User settings as key-value pairs
swEncryptionKey string Service worker encryption key for push notifications (optional)

Combined Login Init

POST /api/auth/login-init

Combined login initialization that returns all available authentication methods in a single request. If the user already has a valid token, returns the authenticated session directly.

Authentication: Optional

Response (unauthenticated):

{
  "data": {
    "status": "unauthenticated",
    "qrLogin": {
      "sessionId": "session_abc123",
      "secret": "base64-secret"
    },
    "webAuthn": {
      "challenge": "base64-encoded-challenge",
      "rpId": "example.com",
      "allowCredentials": [...]
    }
  },
  "time": "2025-01-01T12:00:00Z"
}

Response (already authenticated):

{
  "data": {
    "status": "authenticated",
    "login": {
      "tnId": 12345,
      "idTag": "alice@example.com",
      "token": "eyJhbGc..."
    }
  },
  "time": "2025-01-01T12:00:00Z"
}

Logout

POST /api/auth/logout

Invalidate the current session.

Authentication: Required

Change Password

POST /api/auth/password

Change the authenticated user’s password.

Authentication: Required

Request:

{
  "currentPassword": "current-password",
  "newPassword": "new-secure-password"
}

Response:

{
  "data": {
    "success": true
  },
  "time": "2025-01-01T12:00:00Z"
}

Set Password

POST /api/auth/set-password

Set a new password using a reset token. Used during password recovery flow.

Authentication: Not required

Request:

{
  "token": "reset-token-from-email",
  "password": "new-secure-password"
}

Response:

{
  "data": {
    "success": true
  },
  "time": "2025-01-01T12:00:00Z"
}

Forgot Password

POST /api/auth/forgot-password

Request a password reset email.

Authentication: Not required

Request:

{
  "idTag": "alice@example.com"
}

Response:

{
  "data": {
    "sent": true
  },
  "time": "2025-01-01T12:00:00Z"
}

Refresh Login Token

GET /api/auth/login-token

Refresh the authentication token before it expires.

Authentication: Not required (uses existing valid token)

Response:

{
  "data": {
    "token": "eyJhbGc...",
    "expiresAt": 1735086400
  },
  "time": "2025-01-01T12:00:00Z"
}

Get Access Token

GET /api/auth/access-token

Exchange credentials or tokens for a scoped access token. Supports multiple authentication methods.

Authentication: Not required

Query Parameters:

  • token - Existing token to exchange
  • refId - Reference ID for share links
  • apiKey - API key for programmatic access
  • scope - Requested scope (optional)
  • refresh - Set to true to refresh an existing token

Response:

{
  "data": {
    "token": "eyJhbGc...",
    "expiresAt": 1735086400,
    "scope": "read:files"
  },
  "time": "2025-01-01T12:00:00Z"
}

Get Proxy Token

GET /api/auth/proxy-token

Get a proxy token for accessing remote resources via federation.

Authentication: Required

Query Parameters:

  • target - Target identity for federation

Response:

{
  "data": {
    "token": "eyJhbGc...",
    "expiresAt": 1735555555
  },
  "time": "2025-01-01T12:00:00Z"
}

QR Code Login

QR code login allows users to authenticate on a desktop browser by scanning a QR code with their mobile device.

Initialize QR Login

POST /api/auth/qr-login/init

Start a QR login session. The returned sessionId is encoded in the QR code, while the secret is kept by the initiating client for verification.

Authentication: Not required

Response:

{
  "data": {
    "sessionId": "session_abc123",
    "secret": "base64-secret-key"
  },
  "time": "2025-01-01T12:00:00Z"
}

Poll QR Login Status

GET /api/auth/qr-login/{session_id}/status

Long-poll the QR login session for status changes. The request blocks until the status changes or the timeout expires.

Authentication: Not required

Path Parameters:

  • session_id - The session ID from the init response

Query Parameters:

  • timeout - Long-poll timeout in seconds (default: 15, max: 30)

Headers:

  • x-qr-secret - The secret from the init response (required for verification)

Response:

{
  "data": {
    "status": "approved",
    "login": {
      "tnId": 12345,
      "idTag": "alice@example.com",
      "token": "eyJhbGc...",
      "name": "Alice Johnson"
    }
  },
  "time": "2025-01-01T12:00:00Z"
}

Status Values:

Status Description
pending Waiting for mobile device to scan
approved User approved on mobile - login field included
denied User denied on mobile
expired Session timed out

Get QR Login Details

GET /api/auth/qr-login/{session_id}/details

Get the details of a QR login session. Called by the mobile device after scanning the QR code to display the login request to the user.

Authentication: Required

Path Parameters:

  • session_id - The session ID encoded in the QR code

Response:

{
  "data": {
    "sessionId": "session_abc123",
    "idTag": "alice@example.com",
    "name": "Alice Johnson"
  },
  "time": "2025-01-01T12:00:00Z"
}

Respond to QR Login

POST /api/auth/qr-login/{session_id}/respond

Approve or deny a QR login request from the mobile device.

Authentication: Required

Path Parameters:

  • session_id - The session ID from the QR code

Request:

{
  "approved": true
}
Field Type Required Description
approved boolean Yes Whether to approve or deny the login

Response:

{
  "data": {
    "status": "approved"
  },
  "time": "2025-01-01T12:00:00Z"
}

WebAuthn (Passkey) Authentication

WebAuthn enables passwordless authentication using passkeys (biometrics, security keys, etc.).

List Passkey Registrations

GET /api/auth/wa/reg

List all registered passkeys for the authenticated user.

Authentication: Required

Response:

{
  "data": [
    {
      "keyId": "abc123",
      "name": "MacBook Touch ID",
      "createdAt": "2025-01-01T12:00:00Z",
      "lastUsedAt": "2025-01-15T09:30:00Z"
    }
  ],
  "time": "2025-01-01T12:00:00Z"
}

Get Registration Challenge

GET /api/auth/wa/reg/challenge

Get a challenge for registering a new passkey.

Authentication: Required

Response:

{
  "data": {
    "challenge": "base64-encoded-challenge",
    "rpId": "example.com",
    "rpName": "Cloudillo",
    "userId": "base64-user-id",
    "userName": "alice@example.com"
  },
  "time": "2025-01-01T12:00:00Z"
}

Register Passkey

POST /api/auth/wa/reg

Complete passkey registration with the WebAuthn response.

Authentication: Required

Request:

{
  "name": "MacBook Touch ID",
  "credential": {
    "id": "credential-id",
    "rawId": "base64-raw-id",
    "response": {
      "clientDataJSON": "base64-client-data",
      "attestationObject": "base64-attestation"
    },
    "type": "public-key"
  }
}

Response:

{
  "data": {
    "keyId": "abc123",
    "name": "MacBook Touch ID",
    "createdAt": "2025-01-01T12:00:00Z"
  },
  "time": "2025-01-01T12:00:00Z"
}

Delete Passkey

DELETE /api/auth/wa/reg/{key_id}

Remove a registered passkey.

Authentication: Required

Path Parameters:

  • key_id - The passkey identifier to delete

Response:

{
  "data": "ok",
  "time": "2025-01-01T12:00:00Z"
}

Get Login Challenge

GET /api/auth/wa/login/challenge

Get a challenge for passkey authentication.

Authentication: Not required

Query Parameters:

  • idTag - User identity (optional, for usernameless flow)

Response:

{
  "data": {
    "challenge": "base64-encoded-challenge",
    "rpId": "example.com",
    "allowCredentials": [
      {
        "type": "public-key",
        "id": "credential-id"
      }
    ]
  },
  "time": "2025-01-01T12:00:00Z"
}

Login with Passkey

POST /api/auth/wa/login

Authenticate using a passkey.

Authentication: Not required

Request:

{
  "credential": {
    "id": "credential-id",
    "rawId": "base64-raw-id",
    "response": {
      "clientDataJSON": "base64-client-data",
      "authenticatorData": "base64-auth-data",
      "signature": "base64-signature"
    },
    "type": "public-key"
  }
}

Response:

{
  "data": {
    "tnId": 12345,
    "idTag": "alice@example.com",
    "name": "Alice Johnson",
    "token": "eyJhbGc...",
    "roles": ["user"]
  },
  "time": "2025-01-01T12:00:00Z"
}

API Key Management

API keys enable programmatic access without interactive login.

List API Keys

GET /api/auth/api-keys

List all API keys for the authenticated user.

Authentication: Required

Response:

{
  "data": [
    {
      "keyId": "key_abc123",
      "name": "CI/CD Pipeline",
      "scope": "read:files,write:files",
      "createdAt": "2025-01-01T12:00:00Z",
      "lastUsedAt": "2025-01-15T09:30:00Z",
      "expiresAt": "2026-01-01T12:00:00Z"
    }
  ],
  "time": "2025-01-01T12:00:00Z"
}

Create API Key

POST /api/auth/api-keys

Create a new API key.

Authentication: Required

Request:

{
  "name": "CI/CD Pipeline",
  "scope": "read:files,write:files",
  "expiresAt": "2026-01-01T12:00:00Z"
}

Response:

{
  "data": {
    "keyId": "key_abc123",
    "name": "CI/CD Pipeline",
    "key": "ck_live_abc123xyz...",
    "scope": "read:files,write:files",
    "createdAt": "2025-01-01T12:00:00Z",
    "expiresAt": "2026-01-01T12:00:00Z"
  },
  "time": "2025-01-01T12:00:00Z"
}
Warning

The key field is only returned once at creation. Store it securely.

Get API Key

GET /api/auth/api-keys/{key_id}

Get details of a specific API key.

Authentication: Required

Path Parameters:

  • key_id - The API key identifier

Response:

{
  "data": {
    "keyId": "key_abc123",
    "name": "CI/CD Pipeline",
    "scope": "read:files,write:files",
    "createdAt": "2025-01-01T12:00:00Z",
    "lastUsedAt": "2025-01-15T09:30:00Z",
    "expiresAt": "2026-01-01T12:00:00Z"
  },
  "time": "2025-01-01T12:00:00Z"
}

Update API Key

PATCH /api/auth/api-keys/{key_id}

Update an API key’s metadata.

Authentication: Required

Path Parameters:

  • key_id - The API key identifier

Request:

{
  "name": "Production Pipeline",
  "scope": "read:files"
}

Response:

{
  "data": {
    "keyId": "key_abc123",
    "name": "Production Pipeline",
    "scope": "read:files",
    "createdAt": "2025-01-01T12:00:00Z",
    "expiresAt": "2026-01-01T12:00:00Z"
  },
  "time": "2025-01-01T12:00:00Z"
}

Delete API Key

DELETE /api/auth/api-keys/{key_id}

Revoke and delete an API key.

Authentication: Required

Path Parameters:

  • key_id - The API key identifier

Response:

{
  "data": "ok",
  "time": "2025-01-01T12:00:00Z"
}

Public Endpoints

Get Tenant Profile (Public)

GET /api/me
GET /api/me/full

Get the tenant (server) profile with public keys. This is a public endpoint that returns the server’s identity information.

Note: Both paths return the same data; /full is an alias for compatibility.

Authentication: Not required

Response:

{
  "data": {
    "idTag": "server@example.com",
    "name": "Example Server",
    "publicKey": "-----BEGIN PUBLIC KEY-----...",
    "serverInfo": {
      "version": "1.0.0",
      "features": ["federation", "crdt", "rtdb"]
    }
  },
  "time": "2025-01-01T12:00:00Z"
}

Resolve Identity Tag

GET /.well-known/cloudillo/id-tag

Resolve a domain-based identity to a Cloudillo server. This is part of the DNS-based identity system.

Authentication: Not required

Query Parameters:

  • idTag - The identity to resolve (e.g., alice@example.com)

Response:

{
  "data": {
    "idTag": "alice@example.com",
    "serverUrl": "https://cloudillo.example.com",
    "publicKey": "-----BEGIN PUBLIC KEY-----..."
  },
  "time": "2025-01-01T12:00:00Z"
}

Get VAPID Public Key

GET /api/auth/vapid

Get the VAPID public key for push notification subscriptions.

Authentication: Required

Response:

{
  "data": {
    "publicKey": "BNxwfD..."
  },
  "time": "2025-01-01T12:00:00Z"
}

See Also

Profiles API

Overview

Manage user and community profiles, including registration and community creation.

Registration

Verify Profile

POST /api/profiles/verify

Verify identity availability before registration or community creation. This endpoint checks if an identity tag is available and can be used.

Authentication: Not required

Request:

{
  "idTag": "alice@example.com",
  "type": "idp"
}
Field Type Required Description
idTag string Yes The identity tag to verify
type string Yes Identity type: idp (hosted) or domain (self-hosted)
appDomain string No Application domain (for domain type)
token string No Verification token (for unauthenticated requests)

Response:

{
  "data": {
    "available": true,
    "idTag": "alice@example.com"
  },
  "time": "2025-01-01T12:00:00Z"
}

Example:

const result = await api.profile.verify({
  idTag: 'alice@example.com',
  type: 'idp'
})
if (result.data.available) {
  // Proceed with registration
}

Register User

POST /api/profiles/register

Register a new user account. This initiates the registration process and sends a verification email.

Authentication: Not required

Request:

{
  "type": "idp",
  "idTag": "alice@example.com",
  "email": "alice@gmail.com",
  "token": "verification-token",
  "lang": "en"
}
Field Type Required Description
type string Yes Identity type: idp or domain
idTag string Yes Desired identity tag
email string Yes Email address for verification
token string Yes Registration token
appDomain string No Application domain (for domain type)
lang string No Preferred language code

Response:

{
  "data": {
    "tnId": 12345,
    "idTag": "alice@example.com",
    "name": "Alice",
    "token": "eyJhbGc..."
  },
  "time": "2025-01-01T12:00:00Z"
}

Example:

const result = await api.profile.register({
  type: 'idp',
  idTag: 'alice@example.com',
  email: 'alice@gmail.com',
  token: 'registration-token'
})
// User is now registered and logged in
console.log('Registered:', result.data.idTag)

Create Community

PUT /api/profiles/{id_tag}

Create a new community profile. The authenticated user becomes the community owner.

Authentication: Required

Path Parameters:

  • id_tag - The identity tag for the new community

Request:

{
  "type": "idp",
  "name": "Developer Community",
  "profilePic": "b1~abc123",
  "appDomain": "devs.example.com"
}
Field Type Required Description
type string Yes Identity type: idp or domain
name string No Community display name
profilePic string No Profile picture file ID
appDomain string No Application domain (for domain type)

Response:

{
  "data": {
    "idTag": "devs@example.com",
    "name": "Developer Community",
    "type": "community",
    "createdAt": "2025-01-01T12:00:00Z"
  },
  "time": "2025-01-01T12:00:00Z"
}

Example:

// First verify the community name is available
const check = await api.profile.verify({
  idTag: 'devs@example.com',
  type: 'idp'
})

if (check.data.available) {
  const response = await fetch('/api/profiles/devs@example.com', {
    method: 'PUT',
    headers: {
      'Authorization': `Bearer ${token}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      type: 'idp',
      name: 'Developer Community'
    })
  })
  const community = await response.json()
  console.log('Created community:', community.data.idTag)
}
Community Ownership

The authenticated user who creates the community becomes the owner. Only the owner can manage community settings and membership.

Profile Management

Get Own Profile

GET /api/me

Get the authenticated user’s profile (requires authentication). For the public server identity endpoint, see GET /api/me in the Authentication API.

Authentication: Required

Example:

const api = cloudillo.createApiClient()
const profile = await api.profiles.getOwn()

Response:

{
  "data": {
    "tnId": 12345,
    "idTag": "alice@example.com",
    "name": "Alice Johnson",
    "profilePic": "/file/b1~abc123",
    "x": {
      "bio": "Software developer",
      "location": "San Francisco"
    }
  }
}

Update Own Profile

PATCH /api/me

Update the authenticated user’s profile.

Authentication: Required

Request:

await api.profiles.updateOwn({
  name: 'Alice Smith',
  x: {
    bio: 'Senior developer',
    website: 'https://alice.example.com'
  }
})

Upload Profile Image

PUT /api/me/image

Upload or update your profile picture.

Authentication: Required

Content-Type: Image type (e.g., image/jpeg, image/png)

Request Body: Raw image binary data

Example:

const imageFile = document.querySelector('input[type="file"]').files[0]

const response = await fetch('/api/me/image', {
  method: 'PUT',
  headers: {
    'Authorization': `Bearer ${token}`,
    'Content-Type': imageFile.type
  },
  body: imageFile
})

const result = await response.json()
console.log('Profile image updated:', result.data.profilePic)

Response:

{
  "data": {
    "profilePic": "/api/file/b1~abc123",
    "fileId": "b1~abc123"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Upload Cover Image

PUT /api/me/cover

Upload or update your cover photo.

Authentication: Required

Content-Type: Image type (e.g., image/jpeg, image/png)

Request Body: Raw image binary data

Example:

const coverFile = document.querySelector('input[type="file"]').files[0]

const response = await fetch('/api/me/cover', {
  method: 'PUT',
  headers: {
    'Authorization': `Bearer ${token}`,
    'Content-Type': coverFile.type
  },
  body: coverFile
})

const result = await response.json()
console.log('Cover image updated:', result.data.coverPic)

Response:

{
  "data": {
    "coverPic": "/api/file/b1~xyz789",
    "fileId": "b1~xyz789"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Get Profile

GET /api/profiles/:idTag

Get a specific user or community profile.

Example:

const profile = await api.profiles.get('bob@example.com')

List Profiles

GET /api/profiles

List all accessible profiles.

Authentication: Required

Query Parameters:

  • type - Filter by type (person, community)
  • status - Filter by status (comma-separated)
  • connected - Filter by connection status (disconnected, pending, connected)
  • following - Filter by follow status (true/false)
  • q - Search query for name
  • idTag - Filter by specific identity tag

Example:

// List all communities
const communities = await api.profiles.list({
  type: 'community'
})

// List connected profiles
const friends = await api.profiles.list({
  connected: 'connected'
})

// Search for profiles
const results = await api.profiles.list({
  q: 'alice'
})

Update Relationship

PATCH /api/profiles/:idTag

Update your relationship with another profile (follow, block, etc.).

Authentication: Required

Request:

{
  "relationship": "follow"
}

Response:

{
  "data": {
    "idTag": "bob@example.com",
    "relationship": "follow",
    "updatedAt": 1735000000
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Admin: Update Profile

PATCH /api/admin/profiles/:idTag

Admin endpoint to update any user’s profile.

Authentication: Required (admin role)

Request:

{
  "name": "Updated Name",
  "status": "active",
  "roles": ["user", "moderator"]
}

Response:

{
  "data": {
    "tnId": 12345,
    "idTag": "bob@example.com",
    "name": "Updated Name",
    "status": "active",
    "roles": ["user", "moderator"]
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

See Also

Actions API

Overview

Actions represent social interactions and activities in Cloudillo. This includes posts, comments, reactions, connections, and more.

The Actions API allows you to:

  • Create posts, comments, and reactions
  • Manage user connections and follows
  • Share files and resources
  • Send messages
  • Track action statistics

Action Types

Content Actions

Type Description Audience Examples
POST Create a post or content Followers, Public, Custom Blog posts, status updates
CMNT Comment on an action Parent action audience Thread replies
REPOST Share existing content Followers Retweets
SHRE Share resource/link Followers, Custom Link sharing

User Actions

Type Description Audience Examples
FLLW Follow a user/community Target user Subscribe to updates
CONN Connection request Target user Friend requests

Communication Actions

Type Description Audience Examples
MSG Private message Specific user Direct messages
FSHR File sharing Specific user(s) Share documents

Metadata Actions

Type Description Audience Examples
REACT React to content None (broadcast) Likes, loves
ACK Acknowledgment Parent action issuer Read receipts

Endpoints

List Actions

GET /api/actions

Query all actions with optional filters. Uses cursor-based pagination for stable results.

Authentication: Optional (visibility-based access control)

Query Parameters:

Filtering:

  • type - Filter by action type(s), comma-separated (e.g., POST,CMNT)
  • status - Filter by status(es), comma-separated (P=Pending, A=Active, D=Deleted, C=Created, N=New)
  • issuer - Filter by issuer identity (e.g., alice@example.com)
  • audience - Filter by audience identity
  • parentId - Filter by parent action (for comments/reactions)
  • rootId - Filter by root/thread ID (for nested comments)
  • subject - Filter by subject identity (for CONN, FLLW actions)
  • involved - Filter by actions involving a specific identity (issuer, audience, or subject)
  • actionId - Filter by specific action ID
  • tag - Filter by content tag

Time-based:

  • createdAfter - ISO 8601 timestamp or Unix seconds

Pagination:

  • limit - Max results (default: 20)
  • cursor - Opaque cursor for next page (from previous response)

Sorting:

  • sort - Sort field: created (default)
  • sortDir - Sort direction: asc or desc (default: desc)

Examples:

Get recent posts:

const api = cloudillo.createApiClient()

const posts = await api.actions.list({
  type: 'POST',
  status: 'A',
  limit: 20,
  sort: 'created',
  sortDir: 'desc'
})

Get comments on a specific post:

const comments = await api.actions.list({
  type: 'CMNT',
  parentId: 'act_post123',
  sort: 'created',
  sortDir: 'asc'
})

Get all actions involving a specific user:

const userActivity = await api.actions.list({
  involved: 'alice@example.com',
  limit: 50
})

Get posts from a time range:

const lastWeek = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString()

const recentPosts = await api.actions.list({
  type: 'POST',
  status: 'A',
  createdAfter: lastWeek,
  limit: 100
})

Get pending connection requests:

const connectionRequests = await api.actions.list({
  type: 'CONN',
  status: 'P',
  subject: cloudillo.idTag // Requests to me
})

Get thread with all nested comments:

const thread = await api.actions.list({
  rootId: 'act_original_post',
  type: 'CMNT',
  sort: 'created',
  sortDir: 'asc',
  limit: 200
})

Cursor-based pagination:

// First page
const page1 = await api.actions.list({ type: 'POST', limit: 20 })

// Next page using cursor
if (page1.cursorPagination?.hasMore) {
  const page2 = await api.actions.list({
    type: 'POST',
    limit: 20,
    cursor: page1.cursorPagination.nextCursor
  })
}

Response:

{
  "data": [
    {
      "actionId": "act_abc123",
      "type": "POST",
      "issuer": {
        "idTag": "alice@example.com",
        "name": "Alice Johnson",
        "profilePic": "/file/b1~abc"
      },
      "content": {
        "text": "Hello, Cloudillo!",
        "title": "My First Post"
      },
      "createdAt": "2025-01-01T12:00:00Z",
      "visibility": "P",
      "stat": {
        "reactions": 5,
        "comments": 3,
        "ownReaction": "LOVE"
      }
    }
  ],
  "cursorPagination": {
    "nextCursor": "eyJzIjoiY3JlYXRlZCIsInYiOjE3MzUwMDAwMDAsImlkIjoiYWN0X2FiYzEyMyJ9",
    "hasMore": true
  },
  "time": "2025-01-01T12:00:00Z"
}

Create Action

POST /api/actions

Create a new action (post, comment, reaction, etc.).

Authentication: Required

Request Body:

interface NewAction {
  type: string // Action type (POST, CMNT, etc.)
  subType?: string // Optional subtype/category
  parentId?: string // For comments, reactions
  rootId?: string // For deep threads
  audienceTag?: string // Target audience
  content?: unknown // Action-specific content
  attachments?: string[] // File IDs
  subject?: string // Target (e.g., who to follow)
  expiresAt?: number // Expiration timestamp
}

Examples:

Create a Post:

const post = await api.actions.create({
  type: 'POST',
  content: {
    text: 'Hello, Cloudillo!',
    title: 'My First Post'
  },
  attachments: ['file_123', 'file_456']
})

Create a Comment:

const comment = await api.actions.create({
  type: 'CMNT',
  parentId: 'act_post123',
  content: {
    text: 'Great post!'
  }
})

Create a Reaction:

const reaction = await api.actions.create({
  type: 'REACT',
  subType: 'LOVE',
  parentId: 'act_post123'
})

Follow a User:

const follow = await api.actions.create({
  type: 'FLLW',
  subject: 'bob@example.com'
})

Connect with a User:

const connection = await api.actions.create({
  type: 'CONN',
  subject: 'bob@example.com',
  content: {
    message: 'Would like to connect!'
  }
})

Share a File:

const share = await api.actions.create({
  type: 'FSHR',
  subject: 'bob@example.com',
  attachments: ['file_789'],
  content: {
    permission: 'READ', // or 'WRITE'
    message: 'Check out this document'
  }
})

Response:

{
  "data": {
    "actionId": "act_newpost123",
    "type": "POST",
    "issuerTag": "alice@example.com",
    "content": {
      "text": "Hello, Cloudillo!",
      "title": "My First Post"
    },
    "createdAt": 1735000000,
    "status": "A"
  }
}

Get Action

GET /api/actions/:actionId

Retrieve a specific action by ID.

Example:

const action = await api.actions.get('act_123')

Response:

{
  "data": {
    "actionId": "act_123",
    "type": "POST",
    "issuerTag": "alice@example.com",
    "issuer": {
      "idTag": "alice@example.com",
      "name": "Alice Johnson",
      "profilePic": "/file/b1~abc"
    },
    "content": {
      "text": "Hello!",
      "title": "Greeting"
    },
    "createdAt": 1735000000,
    "stat": {
      "reactions": 10,
      "comments": 5,
      "ownReaction": null
    }
  }
}

Update Draft Action

PATCH /api/actions/{action_id}

Update a draft action before publishing. Only actions in draft status (R) can be updated.

Authentication: Required (must be issuer)

Request Body:

{
  "content": {
    "text": "Updated post content",
    "title": "Updated Title"
  },
  "attachments": ["b1~file123"],
  "visibility": "P"
}
Field Type Description
content object Updated content (action-type specific)
attachments string[] Updated file attachment IDs
visibility string Visibility level (P, V, F, C)
flags string Action flags
x object Extension data

Response:

{
  "data": {
    "actionId": "act_123",
    "type": "POST",
    "status": "R",
    "content": {
      "text": "Updated post content",
      "title": "Updated Title"
    }
  },
  "time": "2025-01-01T12:00:00Z"
}
Info

Only draft actions (status R) can be updated. Published actions are signed JWTs and cannot be modified.

Delete Action

DELETE /api/actions/:actionId

Delete an action. Only the issuer can delete their actions.

Authentication: Required (must be issuer)

Example:

await api.actions.delete('act_123')

Response:

{
  "data": "ok"
}

Accept Action

POST /api/actions/:actionId/accept

Accept a pending action (e.g., connection request, follow request).

Authentication: Required (must be the subject/target)

DSL Hooks: When an action is accepted, the server triggers the on_accept hook defined in the action type’s DSL configuration. This can execute custom logic such as:

  • Creating reciprocal connections (CONN actions)
  • Adding the user to groups (INVT actions)
  • Granting permissions (FSHR actions)

Example:

// Accept a connection request
await api.actions.accept('act_connreq123')

// Accept a follow request (for private accounts)
await api.actions.accept('act_follow456')

Response:

{
  "data": {
    "actionId": "act_connreq123",
    "status": "A"
  }
}

Reject Action

POST /api/actions/:actionId/reject

Reject a pending action.

Authentication: Required (must be the subject/target)

DSL Hooks: When an action is rejected, the server triggers the on_reject hook defined in the action type’s DSL configuration. This can execute cleanup logic such as:

  • Removing pending permissions
  • Notifying the issuer of the rejection
  • Cleaning up temporary resources

Example:

await api.actions.reject('act_connreq123')

Response:

{
  "data": {
    "actionId": "act_connreq123",
    "status": "D"
  }
}

Update Statistics

POST /api/actions/:actionId/stat

Update action statistics (typically called by the system).

Authentication: Required (admin)

Request Body:

{
  reactions?: number
  comments?: number
  views?: number
}

Dismiss Notification

POST /api/actions/{action_id}/dismiss

Dismiss a notification action. This acknowledges a notification without accepting or rejecting it.

Authentication: Required

Request Body: Empty

Response:

{
  "data": null,
  "time": "2025-01-01T12:00:00Z"
}

Publish Draft

POST /api/actions/{action_id}/publish

Publish a draft action, making it visible to the intended audience. Optionally schedule for future publication.

Authentication: Required (must be issuer)

Request Body:

{
  "publishAt": "2025-02-01T12:00:00Z"
}
Field Type Description
publishAt string Optional future publication timestamp (ISO 8601). If omitted, publishes immediately.

Response:

{
  "data": {
    "actionId": "act_123",
    "type": "POST",
    "status": "A"
  },
  "time": "2025-01-01T12:00:00Z"
}
Info

If publishAt is provided, the action moves to scheduled status (S) and will be published automatically at the specified time.

Cancel Scheduled Action

POST /api/actions/{action_id}/cancel

Cancel a scheduled action, reverting it back to draft status.

Authentication: Required (must be issuer)

Request Body: Empty

Response:

{
  "data": {
    "actionId": "act_123",
    "type": "POST",
    "status": "R"
  },
  "time": "2025-01-01T12:00:00Z"
}

Federation Inbox

POST /api/inbox

Receive federated actions from other Cloudillo instances. This is the primary endpoint for cross-instance action delivery. Processing is asynchronous.

Authentication: Not required (actions are verified via signatures)

Request Body: Action token (JWT)

Example:

// This is typically called by other Cloudillo servers
const actionToken = 'eyJhbGc...' // Signed action token

const response = await fetch('/api/inbox', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/jwt'
  },
  body: actionToken
})

Response:

{
  "data": {
    "actionId": "act_federated123",
    "status": "received",
    "verified": true
  },
  "time": "2025-01-01T12:00:00Z"
}

Federation Inbox (Synchronous)

POST /api/inbox/sync

Receive federated actions with synchronous processing. Unlike the standard inbox, this endpoint processes the action immediately and returns the result.

Authentication: Not required (actions are verified via signatures)

Request Body: Action token (JWT)

Response:

{
  "data": {
    "actionId": "act_federated123",
    "status": "processed",
    "verified": true
  },
  "time": "2025-01-01T12:00:00Z"
}
Info

Both inbox endpoints verify the action signature against the issuer’s public key before accepting it.

Action Status Flow

Actions have a lifecycle represented by status:

R (Draft)     → S (Scheduled) → A (Active/Published)
              → A (Active)      ↘ R (Cancelled)
              ↘ D (Deleted)

C (Created)   → A (Active/Accepted)
              ↘ D (Deleted/Rejected)

N (New)       → (Dismissed)

P (Pending)   → A (Active)
              ↘ D (Deleted)
  • R (Draft): Unpublished draft, can be edited
  • S (Scheduled): Scheduled for future publication
  • A (Active): Published, visible, and finalized
  • D (Deleted): Soft-deleted or rejected
  • C (Created): Awaiting acceptance (e.g., connection requests)
  • N (New): Notification awaiting acknowledgment
  • P (Pending): Legacy pending status

Status transitions:

  • Draft actions can be updated, published, scheduled, or deleted
  • Scheduled actions can be cancelled (reverts to Draft) or auto-publish at scheduled time
  • Created actions can be accepted (→ Active) or rejected (→ Deleted)
  • New notifications can be dismissed
  • Any action can be deleted by issuer (changes status to Deleted)

Content Schemas

Different action types have different content structures:

POST Content

{
  title?: string
  text?: string
  summary?: string
  category?: string
  tags?: string[]
}

CMNT Content

{
  text: string
}

MSG Content

{
  text: string
  subject?: string
}

FSHR Content

{
  permission: 'READ' | 'WRITE'
  message?: string
  expiresAt?: number
}

CONN Content

{
  message?: string
}

Action Statistics

Actions can have aggregated statistics:

interface ActionStat {
  reactions?: number // Total reactions
  comments?: number // Total comments
  commentsRead?: number // Comments user has read
  ownReaction?: string // User's own reaction type
  views?: number // View count
  shares?: number // Share count
}

Accessing statistics:

const action = await api.actions.get('act_123')

console.log('Reactions:', action.stat?.reactions)
console.log('Comments:', action.stat?.comments)
console.log('My reaction:', action.stat?.ownReaction)

Threading

Actions support threading via parentId and rootId:

POST (root action)
└── CMNT (comment)
    └── CMNT (reply to comment)
        └── CMNT (nested reply)

Creating a thread:

// Original post
const post = await api.actions.create({
  type: 'POST',
  content: { text: 'Main post' }
})

// First level comment
const comment1 = await api.actions.create({
  type: 'CMNT',
  parentId: post.actionId,
  rootId: post.actionId,
  content: { text: 'A comment' }
})

// Reply to comment
const reply = await api.actions.create({
  type: 'CMNT',
  parentId: comment1.actionId,
  rootId: post.actionId, // Still points to original post
  content: { text: 'A reply' }
})

Fetching a thread:

// Get all comments in a thread
const thread = await api.actions.list({
  rootId: 'act_post123',
  type: 'CMNT',
  sort: 'created',
  sortDir: 'asc'
})

Federation

Actions are the core of Cloudillo’s federation model. Each action is cryptographically signed and can be verified across instances.

Action Token Structure:

Header: { alg: "ES384", typ: "JWT" }
Payload: {
  actionId: "act_123",
  type: "POST",
  issuerTag: "alice@example.com",
  content: {...},
  createdAt: 1735000000,
  iat: 1735000000,
  exp: 1735086400
}
Signature: <ES384 signature>

This enables:

  • Trust-free verification
  • Cross-instance action delivery
  • Tamper-proof audit trails

Best Practices

1. Use Cursor-Based Pagination

// ✅ Use cursor pagination for stable results
async function fetchAllPosts() {
  const allPosts = []
  let cursor = undefined

  while (true) {
    const result = await api.actions.list({
      type: 'POST',
      limit: 50,
      cursor
    })

    allPosts.push(...result.data)

    if (!result.cursorPagination?.hasMore) break
    cursor = result.cursorPagination.nextCursor
  }

  return allPosts
}

2. Optimistic UI Updates

// Update UI immediately, rollback on error
const optimisticAction = {
  actionId: 'temp_' + Date.now(),
  type: 'POST',
  content: { text: 'New post' },
  issuer: { idTag: cloudillo.idTag },
  createdAt: new Date().toISOString()
}

setPosts([optimisticAction, ...posts])

try {
  const created = await api.actions.create(optimisticAction)
  setPosts(posts => posts.map(p =>
    p.actionId === optimisticAction.actionId ? created : p
  ))
} catch (error) {
  setPosts(posts => posts.filter(p =>
    p.actionId !== optimisticAction.actionId
  ))
}

3. Handle Visibility

Actions have visibility levels that control who can see them:

Code Level Description
P Public Anyone can view
V Verified Authenticated users only
F Follower User’s followers only
C Connected Mutual connections only
null Direct Owner + explicit audience
// Create a public post
const publicPost = await api.actions.create({
  type: 'POST',
  visibility: 'P',
  content: { text: 'Hello everyone!' }
})

// Create a followers-only post
const followersPost = await api.actions.create({
  type: 'POST',
  visibility: 'F',
  content: { text: 'Just for my followers' }
})

See Also

Files API

Overview

The Files API handles file upload, download, and management in Cloudillo. It supports four file types: BLOB (binary files), CRDT (collaborative documents), RTDB (real-time databases), and FLDR (folders).

File Types

Type Description Use Cases
BLOB Binary files (images, PDFs, etc.) Photos, documents, attachments
CRDT Collaborative documents Rich text, spreadsheets, diagrams
RTDB Real-time databases Structured data, forms, todos
FLDR Folders Organize files hierarchically

File Status

Code Status Description
A Active File is available
P Pending File is being processed
D Deleted File is in trash

Image Variants

For BLOB images, Cloudillo automatically generates 5 variants:

Variant Code Max Dimension Quality Format
Thumbnail tn 150px Medium JPEG/WebP
Icon ic 64px Medium JPEG/WebP
SD sd 640px Medium JPEG/WebP
HD hd 1920px High JPEG/WebP
Original orig Original Original Original

Automatic format selection:

  • Modern browsers: WebP or AVIF
  • Fallback: JPEG or PNG

Endpoints

List Files

GET /api/files

List all files accessible to the user. Uses cursor-based pagination.

Authentication: Optional (visibility-based access control)

Query Parameters:

Filtering:

  • fileTp - Filter by file type (BLOB, CRDT, RTDB, FLDR)
  • contentType - Filter by MIME type
  • tags - Filter by tags (comma-separated)
  • parentId - Filter by parent folder (for hierarchical listing)
  • status - Filter by status (A, P, D)

Pagination:

  • limit - Max results (default: 20)
  • cursor - Opaque cursor for next page

Sorting:

  • sort - Sort field: created, modified, name, recent
  • sortDir - Sort direction: asc or desc

Example:

const api = cloudillo.createApiClient()

// List all images
const images = await api.files.list({
  fileTp: 'BLOB',
  contentType: 'image/*',
  limit: 20
})

// List files in a folder
const folderContents = await api.files.list({
  parentId: 'f1~folder123',
  sort: 'name',
  sortDir: 'asc'
})

// List tagged files
const projectFiles = await api.files.list({
  tags: 'project-alpha,important'
})

// Cursor-based pagination
if (images.cursorPagination?.hasMore) {
  const page2 = await api.files.list({
    fileTp: 'BLOB',
    cursor: images.cursorPagination.nextCursor
  })
}

Response:

{
  "data": [
    {
      "fileId": "b1~abc123",
      "parentId": null,
      "status": "A",
      "contentType": "image/png",
      "fileName": "photo.png",
      "fileTp": "BLOB",
      "createdAt": "2025-01-01T12:00:00Z",
      "tags": ["vacation", "beach"],
      "visibility": "P",
      "owner": {
        "idTag": "alice@example.com",
        "name": "Alice"
      },
      "userData": {
        "pinned": false,
        "starred": true,
        "accessedAt": "2025-01-15T09:30:00Z"
      }
    }
  ],
  "cursorPagination": {
    "nextCursor": "eyJzIjoiY3JlYXRlZCIsInYiOjE3MzUwMDAwMDAsImlkIjoiYjF-YWJjMTIzIn0",
    "hasMore": true
  },
  "time": "2025-01-01T12:00:00Z"
}

Create File Metadata (CRDT/RTDB)

POST /api/files

Create file metadata for CRDT or RTDB files. For BLOB files, use the one-step upload endpoint instead (see “Upload File (BLOB)” below).

Authentication: Required

Request Body:

{
  fileTp: string // CRDT or RTDB
  fileName?: string // Optional filename
  tags?: string // Comma-separated tags
}

Example:

// Create CRDT document
const file = await api.files.create({
  fileTp: 'CRDT',
  fileName: 'team-doc.crdt',
  tags: 'collaborative,document'
})

console.log('File ID:', file.fileId) // e.g., "f1~abc123"

// Now connect via WebSocket to edit the document
import * as Y from 'yjs'
const yDoc = new Y.Doc()
const { provider } = await cloudillo.openYDoc(yDoc, file.fileId)

Response:

{
  "data": {
    "fileId": "f1~abc123",
    "status": "P",
    "fileTp": "CRDT",
    "fileName": "team-doc.crdt",
    "createdAt": 1735000000
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Note: For BLOB files (images, PDFs, etc.), use POST /api/files/{preset}/{file_name} instead, which creates metadata and uploads the file in a single step.

Upload File (BLOB)

POST /api/files/{preset}/{file_name}

Upload binary file data directly. This creates the file metadata and uploads the binary in a single operation.

Authentication: Required

Path Parameters:

  • preset (string, required) - Image processing preset (e.g., default, profile-picture, cover-photo)
  • file_name (string, required) - Original filename with extension

Query Parameters:

  • created_at (number, optional) - Unix timestamp (seconds) for when the file was created
  • tags (string, optional) - Comma-separated tags (e.g., vacation,beach,2025)

Content-Type: Binary content type (e.g., image/png, image/jpeg, application/pdf)

Request Body: Raw binary file data

Example:

const api = cloudillo.createApiClient()

// Upload image file
const imageFile = document.querySelector('input[type="file"]').files[0]
const blob = await imageFile.arrayBuffer()

const response = await fetch('/api/files/default/vacation-photo.jpg?tags=vacation,beach', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${token}`,
    'Content-Type': 'image/jpeg'
  },
  body: blob
})

const result = await response.json()
console.log('File ID:', result.data.fileId) // e.g., "b1~abc123"

Complete upload with File object:

async function uploadFile(file: File, preset = 'default', tags?: string) {
  const queryParams = new URLSearchParams()
  if (tags) queryParams.set('tags', tags)

  const url = `/api/files/${preset}/${encodeURIComponent(file.name)}${
    queryParams.toString() ? '?' + queryParams.toString() : ''
  }`

  const response = await fetch(url, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${token}`,
      'Content-Type': file.type
    },
    body: file
  })

  const result = await response.json()
  return result.data.fileId
}

// Usage
const fileInput = document.querySelector('input[type="file"]')
fileInput.addEventListener('change', async (e) => {
  const file = e.target.files[0]
  const fileId = await uploadFile(file, 'default', 'vacation,beach')
  console.log('Uploaded:', fileId)

  // Display uploaded image
  const img = document.createElement('img')
  img.src = `/api/files/${fileId}?variant=sd`
  document.body.appendChild(img)
})

Response:

{
  "data": {
    "fileId": "b1~abc123",
    "status": "M",
    "fileTp": "BLOB",
    "contentType": "image/jpeg",
    "fileName": "vacation-photo.jpg",
    "createdAt": 1735000000,
    "tags": ["vacation", "beach"]
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Download File

GET /api/files/:fileId

Download a file. Returns binary data with appropriate Content-Type.

Query Parameters:

  • variant - Image variant (tn, ic, sd, hd, orig)

Example:

// Direct URL usage
<img src="/api/files/b1~abc123" />

// Get specific variant
<img src="/api/files/b1~abc123?variant=sd" />

// Fetch with API
const response = await fetch(`/api/files/${fileId}`)
const blob = await response.blob()
const url = URL.createObjectURL(blob)

Responsive images:

<picture>
  <source srcset="/api/files/b1~abc123?variant=hd" media="(min-width: 1200px)">
  <source srcset="/api/files/b1~abc123?variant=sd" media="(min-width: 600px)">
  <img src="/api/files/b1~abc123?variant=tn" alt="Photo">
</picture>

Get File Descriptor

GET /api/files/:fileId/descriptor

Get file metadata and available variants.

Example:

const descriptor = await api.files.getDescriptor('b1~abc123')

console.log('File type:', descriptor.contentType)
console.log('Size:', descriptor.size)
console.log('Variants:', descriptor.variants)

Response:

{
  "data": {
    "fileId": "b1~abc123",
    "contentType": "image/png",
    "fileName": "photo.png",
    "size": 1048576,
    "createdAt": "2025-01-01T12:00:00Z",
    "variants": [
      {
        "id": "tn",
        "width": 150,
        "height": 100,
        "size": 5120,
        "format": "webp"
      },
      {
        "id": "sd",
        "width": 640,
        "height": 426,
        "size": 51200,
        "format": "webp"
      },
      {
        "id": "hd",
        "width": 1920,
        "height": 1280,
        "size": 204800,
        "format": "webp"
      },
      {
        "id": "orig",
        "width": 3840,
        "height": 2560,
        "size": 1048576,
        "format": "png"
      }
    ]
  }
}

Get File Metadata

GET /api/files/{file_id}/metadata

Get file metadata only (without variant information).

Authentication: Optional (visibility-based access control)

Path Parameters:

  • file_id - The file ID

Response:

{
  "data": {
    "fileId": "b1~abc123",
    "status": "A",
    "contentType": "image/png",
    "fileName": "photo.png",
    "fileTp": "BLOB",
    "createdAt": "2025-01-01T12:00:00Z",
    "tags": ["vacation", "beach"],
    "visibility": "P",
    "owner": {
      "idTag": "alice@example.com",
      "name": "Alice"
    }
  },
  "time": "2025-01-01T12:00:00Z"
}

Duplicate File

POST /api/files/{file_id}/duplicate

Create a copy of a CRDT or RTDB file. The new file gets a new ID and timestamps but copies the content.

Authentication: Required

Path Parameters:

  • file_id - The file ID to duplicate

Request Body:

{
  "fileName": "Copy of team-doc.crdt",
  "parentId": "f1~folder456"
}
Field Type Required Description
fileName string No Name for the new file (defaults to original name)
parentId string No Parent folder for the new file

Response:

{
  "data": {
    "fileId": "f1~newfile789"
  },
  "time": "2025-01-01T12:00:00Z"
}
Info

File duplication is only available for CRDT and RTDB file types. BLOB files cannot be duplicated through this endpoint.

Update File Metadata

PATCH /api/files/:fileId

Update file metadata (tags, filename, etc.).

Authentication: Required (must be owner)

Request Body:

{
  fileName?: string
  tags?: string // Comma-separated
}

Example:

await api.files.update('b1~abc123', {
  fileName: 'renamed-photo.png',
  tags: 'vacation,beach,2025'
})

Delete File

DELETE /api/files/{file_id}

Move a file to trash (soft delete).

Authentication: Required (must have write access)

Example:

await api.files.delete('b1~abc123')

Response:

{
  "data": "ok",
  "time": "2025-01-01T12:00:00Z"
}

Restore File

POST /api/files/{file_id}/restore

Restore a file from trash.

Authentication: Required (must have write access)

Example:

await fetch('/api/files/b1~abc123/restore', {
  method: 'POST',
  headers: { 'Authorization': `Bearer ${token}` }
})

Response:

{
  "data": "ok",
  "time": "2025-01-01T12:00:00Z"
}

Empty Trash

DELETE /api/trash

Permanently delete all files in trash.

Authentication: Required

Example:

await fetch('/api/trash', {
  method: 'DELETE',
  headers: { 'Authorization': `Bearer ${token}` }
})

Response:

{
  "data": {
    "deleted": 15
  },
  "time": "2025-01-01T12:00:00Z"
}

Update User File Data

PATCH /api/files/{file_id}/user

Update user-specific file metadata (pinned, starred). This updates only the authenticated user’s relationship with the file, not the file itself. Users can pin/star any file they have read access to.

Authentication: Required

Request:

{
  "pinned": true,
  "starred": false
}

Response:

{
  "data": {
    "pinned": true,
    "starred": false,
    "accessedAt": "2025-01-01T12:00:00Z"
  },
  "time": "2025-01-01T12:00:00Z"
}

Example:

// Star a file
await fetch('/api/files/b1~abc123/user', {
  method: 'PATCH',
  headers: {
    'Authorization': `Bearer ${token}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({ starred: true })
})

List Tags

GET /api/tags

List all tags used in files owned by the authenticated user.

Authentication: Required

Response:

{
  "data": [
    {
      "tag": "vacation",
      "count": 15
    },
    {
      "tag": "project-alpha",
      "count": 8
    },
    {
      "tag": "important",
      "count": 3
    }
  ]
}

Example:

const response = await fetch('/api/tags', {
  headers: {
    'Authorization': `Bearer ${token}`
  }
})
const { data: tags } = await response.json()

// Display tags with counts
tags.forEach(({ tag, count }) => {
  console.log(`${tag}: ${count} files`)
})

Add Tag

PUT /api/files/:fileId/tag/:tag

Add a tag to a file.

Authentication: Required (must be owner)

Example:

await fetch(`/api/files/b1~abc123/tag/important`, {
  method: 'PUT',
  headers: {
    'Authorization': `Bearer ${token}`
  }
})

Remove Tag

DELETE /api/files/:fileId/tag/:tag

Remove a tag from a file.

Authentication: Required (must be owner)

Example:

await fetch(`/api/files/b1~abc123/tag/important`, {
  method: 'DELETE',
  headers: {
    'Authorization': `Bearer ${token}`
  }
})

Get File Variant

GET /api/files/variant/:variantId

Get a specific image variant directly.

Example:

// Variant IDs are in the format: {fileId}~{variant}
<img src="/api/files/variant/b1~abc123~sd" />

File Identifiers

Cloudillo uses content-addressable identifiers:

Format: {prefix}{version}~{hash}

Prefixes:

  • b - BLOB files
  • f - CRDT files
  • r - RTDB files

Examples:

  • b1~abc123def - BLOB file
  • f1~xyz789ghi - CRDT file
  • r1~mno456pqr - RTDB file

Variants:

  • b1~abc123def~sd - SD variant of BLOB
  • b1~abc123def~tn - Thumbnail variant

CRDT Files

CRDT files store collaborative documents using Yjs.

Creating a CRDT file:

// 1. Create file metadata
const file = await api.files.create({
  fileTp: 'CRDT',
  tags: 'document,collaborative'
})

// 2. Open for editing
import * as Y from 'yjs'

const yDoc = new Y.Doc()
const { provider } = await cloudillo.openYDoc(yDoc, file.fileId)

// 3. Use shared types
const yText = yDoc.getText('content')
yText.insert(0, 'Hello, CRDT!')

See CRDT documentation for details.

RTDB Files

RTDB files store structured real-time databases.

Creating an RTDB file:

// 1. Create file metadata
const file = await api.files.create({
  fileTp: 'RTDB',
  tags: 'database,todos'
})

// 2. Connect to database
import { RtdbClient } from '@cloudillo/rtdb'

const rtdb = new RtdbClient({
  fileId: file.fileId,
  token: cloudillo.accessToken,
  url: 'wss://server.com/ws/rtdb'
})

// 3. Use collections
const todos = rtdb.collection('todos')
await todos.create({ title: 'Learn Cloudillo', done: false })

See RTDB documentation for details.

Image Presets

Configure automatic image processing with presets:

const file = await api.files.create({
  fileTp: 'BLOB',
  contentType: 'image/jpeg',
  preset: 'profile-picture' // Custom preset
})

Default presets:

  • default - Standard 5-variant generation
  • profile-picture - Square crop, 400x400 max
  • cover-photo - 16:9 crop, 1920x1080 max
  • thumbnail-only - Only generate thumbnails

Tagging

Tags help organize and filter files.

Best practices:

  • Use lowercase tags
  • Use hyphens for multi-word tags (e.g., project-alpha)
  • Limit to 3-5 tags per file
  • Use namespaced tags for projects (e.g., proj:alpha, proj:beta)

Tag filtering:

// Files with ANY of these tags
const files = await api.files.list({
  tags: 'vacation,travel'
})

// Files with ALL of these tags (use multiple requests)
const vacationFiles = await api.files.list({ tags: 'vacation' })
const summerFiles = vacationFiles.data.filter(f =>
  f.tags?.includes('summer')
)

Permissions

File access is controlled by:

  1. Ownership - Owner has full access
  2. FSHR actions - Files shared via FSHR actions grant temporary access
  3. Public files - Files attached to public actions are publicly accessible
  4. Audience - Files attached to actions inherit action audience permissions

Sharing a file:

// Share file with read access
await api.actions.create({
  type: 'FSHR',
  subject: 'bob@example.com',
  attachments: ['b1~abc123'],
  content: {
    permission: 'READ', // or 'WRITE'
    message: 'Check out this photo!'
  }
})

Storage Considerations

File size limits:

  • Free tier: 100 MB per file
  • Pro tier: 1 GB per file
  • Enterprise: Configurable

Total storage:

  • Free tier: 10 GB
  • Pro tier: 100 GB
  • Enterprise: Unlimited

Variant generation:

  • Only for image files (JPEG, PNG, WebP, AVIF, GIF)
  • Automatic async processing
  • Lanczos3 filtering for high quality
  • Progressive JPEG for faster loading

Best Practices

1. Always Create Metadata First

// ✅ Correct order
const metadata = await api.files.create({ fileTp: 'BLOB', contentType: 'image/png' })
await uploadBinary(metadata.fileId, imageBlob)

// ❌ Wrong - upload will fail without metadata
await uploadBinary('unknown-id', imageBlob)

2. Use Appropriate Variants

// ✅ Use thumbnails in lists
<img src={`/api/files/${fileId}?variant=tn`} />

// ✅ Use HD for detail views
<img src={`/api/files/${fileId}?variant=hd`} />

// ❌ Don't use original for thumbnails (wastes bandwidth)
<img src={`/api/files/${fileId}`} width="100" />

3. Handle Upload Errors

async function uploadWithRetry(file: File, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      const result = await api.files.uploadBlob('default', file.name, file, file.type)
      return result.fileId
    } catch (error) {
      if (i === maxRetries - 1) throw error
      await new Promise(r => setTimeout(r, 1000 * (i + 1)))
    }
  }
}

4. Progressive Enhancement

// Show preview immediately, upload in background
function previewAndUpload(file: File) {
  // Show local preview
  const reader = new FileReader()
  reader.onload = (e) => {
    setPreview(e.target.result)
  }
  reader.readAsDataURL(file)

  // Upload in background
  uploadImage(file).then(fileId => {
    setUploadedId(fileId)
  })
}

5. Clean Up Unused Files

// Delete old temporary files
const oldFiles = await api.files.list({
  tags: 'temp',
  createdBefore: Date.now() / 1000 - 86400 // 24 hours ago
})

for (const file of oldFiles.data) {
  await api.files.delete(file.fileId)
}

See Also

Settings API

Settings API

User preferences and configuration key-value store.

Settings Scopes

Settings operate at different scopes with cascading resolution:

Scope Description Permission
system Server-wide defaults (read-only) Requires restart to change
global Shared across all tenants (tn_id=0) Admin only
tenant User-specific settings Owner

Permission Levels

Level Access
system Non-changeable (compile-time)
admin Requires SADM role
user Any authenticated user

Resolution Order

When retrieving a setting value, the system resolves in this order:

  1. Tenant-specific value (if exists)
  2. Global value (if exists)
  3. System default

Endpoints

List Settings

GET /api/settings

Get all settings for the authenticated user.

Authentication: Required

Response:

{
  "data": [
    ["theme", "dark"],
    ["language", "en"],
    ["notifications", "true"]
  ]
}

Get Setting

GET /api/settings/:name

Get a specific setting value.

Example:

const theme = await api.settings.name('theme').get()

Update Setting

PUT /api/settings/:name

Set or update a setting value.

Request:

await api.settings.name('theme').put('dark')

Usage

// Get all settings
const settings = await api.settings.get()

// Get specific setting
const theme = await api.settings.name('theme').get()

// Update setting
await api.settings.name('theme').put('light')
await api.settings.name('fontSize').put(16)
await api.settings.name('notifications').put(true)

Common Settings

  • theme - UI theme preference
  • language - User language
  • fontSize - Text size
  • notifications - Enable/disable notifications
  • darkMode - Dark mode preference

Apps API

Overview

The Apps API manages app packages (APKGs) in Cloudillo. Apps are microfrontend plugins that extend the platform with new functionality.

Endpoints

List Available Apps

GET /api/apps

List all available apps published on the platform.

Authentication: Optional

Query Parameters:

  • search - Search term (matches app name, description, tags)

Response:

{
  "data": [
    {
      "name": "docillo",
      "publisherTag": "apps.cloudillo.org",
      "version": "1.0.0",
      "actionId": "a1~abc123",
      "fileId": "b1~def456",
      "capabilities": ["crdt", "rtdb"]
    }
  ]
}

Example:

curl "https://cl-o.alice.cloudillo.net/api/apps?search=document"

Install App

POST /api/apps/install

Install an app from an APKG action.

Authentication: Required

Permission: Requires app management permission (leader-level)

Request Body:

{
  "actionId": "a1~abc123"
}

Response (201 Created):

{
  "data": {
    "appName": "docillo",
    "publisherTag": "apps.cloudillo.org",
    "version": "1.0.0",
    "actionId": "a1~abc123",
    "fileId": "b1~def456",
    "status": "A",
    "capabilities": ["crdt", "rtdb"],
    "autoUpdate": true
  }
}

Example:

curl -X POST -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"actionId":"a1~abc123"}' \
  "https://cl-o.alice.cloudillo.net/api/apps/install"

List Installed Apps

GET /api/apps/installed

List all apps installed on the current tenant.

Authentication: Required

Permission: Requires app management permission (leader-level)

Response:

{
  "data": [
    {
      "appName": "docillo",
      "publisherTag": "apps.cloudillo.org",
      "version": "1.0.0",
      "actionId": "a1~abc123",
      "fileId": "b1~def456",
      "status": "A",
      "capabilities": ["crdt", "rtdb"],
      "autoUpdate": true
    }
  ]
}

Uninstall App

DELETE /api/apps/@{publisher}/{name}

Uninstall an app.

Authentication: Required

Permission: Requires app management permission (leader-level)

Path Parameters:

  • publisher - Publisher identity tag
  • name - App name

Response: 204 No Content

Example:

curl -X DELETE -H "Authorization: Bearer $TOKEN" \
  "https://cl-o.alice.cloudillo.net/api/apps/@apps.cloudillo.org/docillo"

Get App Container Content

GET /api/files/{file_id}/content/{path}

Serve static assets from an installed app package. Used by the shell to load app resources.

Authentication: Optional

Path Parameters:

  • file_id - The app package file ID
  • path - Path to the asset within the package (e.g., index.html, assets/main.js)

Response: The file content with appropriate MIME type headers.

See Also

References API

Overview

References (refs) are shareable tokens for various workflows including file sharing, email verification, password reset, and invitations. They support configurable expiration, usage limits, and access levels.

Endpoints

List References

GET /api/refs

List references for the current tenant.

Authentication: Required

Query Parameters:

Parameter Type Required Description
type string No Filter by type (e.g., share.file, email-verify)
filter string No Status filter: active, used, expired, all (default: active)
resourceId string No Filter by resource ID (e.g., file ID)

Response:

{
  "data": [
    {
      "refId": "abc123def456",
      "type": "share.file",
      "description": "Project document",
      "createdAt": 1735000000,
      "expiresAt": 1735604800,
      "count": null,
      "resourceId": "f1~xyz789",
      "accessLevel": "read"
    }
  ],
  "pagination": {
    "offset": 0,
    "limit": 10,
    "total": 1
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Example:

// List all active file shares
const shares = await api.refs.list({
  type: 'share.file',
  filter: 'active'
})

// List shares for a specific file
const fileShares = await api.refs.list({
  resourceId: 'f1~xyz789'
})

Create Reference

POST /api/refs

Create a new reference token.

Authentication: Required

Request:

{
  "type": "share.file",
  "description": "Project document",
  "expiresAt": 1735604800,
  "count": null,
  "resourceId": "f1~xyz789",
  "accessLevel": "read"
}
Field Type Required Description
type string Yes Reference type
description string No Human-readable description
expiresAt number No Expiration timestamp (Unix seconds)
count number/null No Usage count: omit for 1, null for unlimited, or number
resourceId string No Resource ID (required for share.file)
accessLevel string No Access level: read or write (default: read)

Response:

{
  "data": {
    "refId": "abc123def456",
    "type": "share.file",
    "description": "Project document",
    "createdAt": 1735000000,
    "expiresAt": 1735604800,
    "count": null,
    "resourceId": "f1~xyz789",
    "accessLevel": "read"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Example:

// Create a read-only share link with unlimited uses
const shareLink = await api.refs.create({
  type: 'share.file',
  description: 'Team document',
  resourceId: 'f1~xyz789',
  accessLevel: 'read',
  count: null  // Unlimited uses
})

// Create a write-access share link (single use)
const editLink = await api.refs.create({
  type: 'share.file',
  resourceId: 'f1~xyz789',
  accessLevel: 'write'
  // count omitted = single use
})

Get Reference

GET /api/refs/{refId}

Get details of a specific reference. Returns full details if authenticated, minimal details (only refId and type) if not.

Authentication: Optional

Path Parameters:

Parameter Type Description
refId string Reference ID

Response (authenticated):

{
  "data": {
    "refId": "abc123def456",
    "type": "share.file",
    "description": "Project document",
    "createdAt": 1735000000,
    "expiresAt": 1735604800,
    "count": 5,
    "resourceId": "f1~xyz789",
    "accessLevel": "read"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Response (unauthenticated):

{
  "data": {
    "refId": "abc123def456",
    "type": "share.file"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Delete Reference

DELETE /api/refs/{refId}

Delete/revoke a reference. The reference will immediately become invalid.

Authentication: Required

Path Parameters:

Parameter Type Description
refId string Reference ID to delete

Response:

{
  "data": null,
  "time": 1735000000,
  "reqId": "req_abc123"
}

Guest Access with References

References can be used to grant guest access to resources without requiring authentication.

Exchange Reference for Access Token

GET /api/auth/access-token

Exchange a reference for a scoped access token. This enables unauthenticated users to access shared resources.

Authentication: None

Query Parameters:

Parameter Type Required Description
ref string Yes Reference ID

Response:

{
  "data": {
    "token": "eyJhbGc...",
    "expiresAt": 1735086400,
    "accessLevel": "read",
    "resourceId": "f1~xyz789"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Example:

// Guest accessing a shared file
const { token, accessLevel } = await api.auth.getAccessToken({
  ref: 'abc123def456'
})

// Use token for subsequent API calls
const file = await api.files.get('f1~xyz789', {
  headers: { 'Authorization': `Bearer ${token}` }
})

Reference Types

Type Description Requires resourceId
share.file File sharing link Yes
email-verify Email verification No
password-reset Password reset link No
invite User invitation No
welcome Welcome/onboarding link No

Usage Count Behavior

The count field controls how many times a reference can be used:

Value Behavior
Omitted Single use (default = 1)
null Unlimited uses
Number That many uses remaining

Each use decrements the count. When count reaches 0, the reference becomes invalid.

Access Levels

For share.file references, the accessLevel controls permissions:

Level Description
read View-only access (default)
write Edit access to the resource

Complete File Sharing Example

// 1. Create a shareable link
const share = await api.refs.create({
  type: 'share.file',
  description: 'Quarterly report',
  resourceId: 'f1~xyz789',
  accessLevel: 'read',
  count: null,  // Unlimited
  expiresAt: Math.floor(Date.now() / 1000) + 7 * 24 * 60 * 60  // 1 week
})

// 2. Generate share URL
const shareUrl = `https://example.cloudillo.net/share/${share.data.refId}`

// 3. Guest accesses the share (on their side)
const { token } = await fetch('/api/auth/access-token?ref=' + refId)
  .then(r => r.json())
  .then(r => r.data)

// 4. Guest uses token to access file
const file = await fetch('/api/files/f1~xyz789', {
  headers: { 'Authorization': `Bearer ${token}` }
})

See Also

Shares API

Overview

The Shares API manages persistent file access grants. Shares allow file owners to grant read or write access to specific users or via links.

Shares vs References vs FSHR Actions
  • Shares are persistent access grants stored on the file (managed here)
  • References are shareable tokens/links (see References API)
  • FSHR actions are federation-level file sharing events (see Actions API)

When you create a user share, the system automatically creates an FSHR action for federation.

Share Entry Fields

Field Type Description
id number Share entry ID
resourceType string Resource type (F for file)
resourceId string ID of the shared resource
subjectType string Subject type: U (user), L (link), F (file)
subjectId string ID of the recipient/subject
permission string Permission level: R (read), W (write), A (admin)
expiresAt string Expiration timestamp (ISO 8601) or null
createdBy string Identity that created the share
createdAt string Creation timestamp (ISO 8601)

Endpoints

List Shares by Subject

GET /api/shares

List all shares where a given identity is the subject (recipient).

Authentication: Required

Query Parameters:

  • subjectId (required) - The subject identity to search for
  • subjectType (optional) - Subject type filter: U (user), L (link), F (file)

Response:

{
  "data": [
    {
      "id": 1,
      "resourceType": "F",
      "resourceId": "b1~abc123",
      "subjectType": "U",
      "subjectId": "bob.cloudillo.net",
      "permission": "R",
      "expiresAt": null,
      "createdBy": "alice.cloudillo.net",
      "createdAt": "2025-01-15T10:30:00Z"
    }
  ],
  "time": "2025-01-15T10:30:00Z"
}

List File Shares

GET /api/files/{file_id}/shares

List all shares for a specific file.

Authentication: Required (must have write access to file)

Path Parameters:

  • file_id - The file ID

Response:

{
  "data": [
    {
      "id": 1,
      "resourceType": "F",
      "resourceId": "b1~abc123",
      "subjectType": "U",
      "subjectId": "bob.cloudillo.net",
      "permission": "W",
      "expiresAt": "2025-06-01T00:00:00Z",
      "createdBy": "alice.cloudillo.net",
      "createdAt": "2025-01-15T10:30:00Z"
    }
  ],
  "time": "2025-01-15T10:30:00Z"
}

Create Share

POST /api/files/{file_id}/shares

Grant access to a file by creating a share entry.

Authentication: Required (must have write access to file)

Path Parameters:

  • file_id - The file ID to share

Request Body:

{
  "subjectType": "U",
  "subjectId": "bob.cloudillo.net",
  "permission": "R",
  "expiresAt": "2025-06-01T00:00:00Z"
}
Field Type Required Description
subjectType string Yes U (user), L (link), or F (file)
subjectId string Yes Non-empty ID of the recipient
permission string Yes R (read), W (write), or A (admin)
expiresAt string No Expiration timestamp (ISO 8601)

Response (201 Created):

{
  "data": {
    "id": 2,
    "resourceType": "F",
    "resourceId": "b1~abc123",
    "subjectType": "U",
    "subjectId": "bob.cloudillo.net",
    "permission": "R",
    "expiresAt": "2025-06-01T00:00:00Z",
    "createdBy": "alice.cloudillo.net",
    "createdAt": "2025-01-15T10:30:00Z"
  },
  "time": "2025-01-15T10:30:00Z"
}
Info

When creating a user share (subjectType: "U"), the system automatically creates an FSHR action for federation delivery.

Example:

curl -X POST -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"subjectType":"U","subjectId":"bob.cloudillo.net","permission":"R"}' \
  "https://cl-o.alice.cloudillo.net/api/files/b1~abc123/shares"

Delete Share

DELETE /api/files/{file_id}/shares/{share_id}

Revoke a file share.

Authentication: Required (must have write access to file)

Path Parameters:

  • file_id - The file ID
  • share_id - The share entry ID to delete

Response:

{
  "data": null,
  "time": "2025-01-15T10:30:00Z"
}
Info

When deleting a user share, the system automatically creates an FSHR action with subType: "DEL" for federation.

See Also

Identity Provider API

Overview

The Identity Provider (IDP) API enables identity management for Cloudillo’s DNS-based identity system. It allows identity providers to register, manage, and activate identities within their domain.

IDP Availability

IDP functionality must be enabled for the tenant. When disabled, all IDP endpoints return 404 Not Found.

Public Endpoints

These endpoints are available without authentication.

Get IDP Info

GET /api/idp/info

Get public information about this Identity Provider. Used by registration UIs to help users choose a provider.

Authentication: None

Response:

{
  "data": {
    "domain": "cloudillo.net",
    "name": "Cloudillo",
    "info": "Free identity hosting for the Cloudillo network",
    "url": "https://cloudillo.net/identity"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}
Field Type Description
domain string Provider domain (e.g., cloudillo.net)
name string Display name of the provider
info string Short description (pricing, terms)
url string Optional URL for more information

Check Availability

GET /api/idp/check-availability

Check if an identity tag is available for registration.

Authentication: Optional

Query Parameters:

Parameter Type Required Description
idTag string Yes Identity to check (e.g., alice.cloudillo.net)

Response:

{
  "data": {
    "available": true,
    "idTag": "alice.cloudillo.net"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Example:

const result = await api.idp.checkAvailability({
  idTag: 'alice.cloudillo.net'
})
if (result.data.available) {
  // Identity is available for registration
}

Activate Identity

POST /api/idp/activate

Activate an identity using a reference token. This is used when the identity owner activates their identity after the registrar has created it.

Authentication: None (uses reference token)

Request:

{
  "refId": "ref_abc123def456"
}
Field Type Required Description
refId string Yes The activation reference ID

Response:

{
  "data": {
    "idTag": "alice.cloudillo.net",
    "status": "active",
    "address": "cloudillo.example.com"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Identity Management

List Identities

GET /api/idp/identities

List identities managed by the authenticated user.

Authentication: Required

Query Parameters:

Parameter Type Required Description
email string No Filter by email (partial match)
registrarIdTag string No Filter by registrar
ownerIdTag string No Filter by owner
status string No Filter by status: pending, active, suspended
limit number No Maximum results to return
offset number No Pagination offset

Response:

{
  "data": [
    {
      "idTag": "alice.cloudillo.net",
      "email": "alice@example.com",
      "registrarIdTag": "admin.cloudillo.net",
      "ownerIdTag": null,
      "address": "cloudillo.example.com",
      "addressUpdatedAt": 1735000000,
      "status": "active",
      "createdAt": 1734900000,
      "updatedAt": 1735000000,
      "expiresAt": 1766436000
    }
  ],
  "time": 1735000000,
  "reqId": "req_abc123"
}

Example:

const identities = await api.idp.identities.list({
  status: 'active',
  limit: 20
})

Create Identity

POST /api/idp/identities

Create a new identity. The authenticated user becomes the registrar.

Authentication: Required

Request:

{
  "idTag": "alice.cloudillo.net",
  "email": "alice@example.com",
  "ownerIdTag": null,
  "address": "cloudillo.example.com"
}
Field Type Required Description
idTag string Yes Identity tag (must be in registrar’s domain)
email string No Email address (required if no ownerIdTag)
ownerIdTag string No Owner identity for community-owned identities
address string No Initial server address

Response:

{
  "data": {
    "idTag": "alice.cloudillo.net",
    "email": "alice@example.com",
    "registrarIdTag": "admin.cloudillo.net",
    "status": "pending",
    "createdAt": 1735000000,
    "updatedAt": 1735000000,
    "expiresAt": 1766436000,
    "apiKey": "clid_abc123..."
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}
API Key

The apiKey field is only returned during creation. Store it securely - it cannot be retrieved later.

Example:

const identity = await api.idp.identities.create({
  idTag: 'alice.cloudillo.net',
  email: 'alice@example.com'
})
console.log('Created identity:', identity.data.idTag)
console.log('API key:', identity.data.apiKey) // Store this!

Get Identity

GET /api/idp/identities/{id}

Get details of a specific identity.

Authentication: Required (must be owner or registrar)

Path Parameters:

Parameter Type Description
id string Identity tag (e.g., alice.cloudillo.net)

Response:

{
  "data": {
    "idTag": "alice.cloudillo.net",
    "email": "alice@example.com",
    "registrarIdTag": "admin.cloudillo.net",
    "ownerIdTag": null,
    "address": "cloudillo.example.com",
    "addressUpdatedAt": 1735000000,
    "status": "active",
    "createdAt": 1734900000,
    "updatedAt": 1735000000,
    "expiresAt": 1766436000
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Update Identity Address

PUT /api/idp/identities/{id}/address

Update the server address for an identity.

Authentication: Required (must be owner or registrar while pending)

Path Parameters:

Parameter Type Description
id string Identity tag

Request:

{
  "address": "new-server.example.com",
  "autoAddress": false
}
Field Type Required Description
address string No New server address
autoAddress boolean No If true and no address provided, use requester’s IP

Response:

{
  "data": {
    "idTag": "alice.cloudillo.net",
    "address": "new-server.example.com",
    "addressUpdatedAt": 1735000000,
    "status": "active"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Delete Identity

DELETE /api/idp/identities/{id}

Delete an identity. This action cannot be undone.

Authentication: Required (must be owner or registrar while pending)

Path Parameters:

Parameter Type Description
id string Identity tag to delete

Response:

{
  "data": {
    "deleted": true,
    "idTag": "alice.cloudillo.net"
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}
Permanent Deletion

Deleting an identity is permanent. The identity tag may be reused after deletion.

API Key Management

API keys provide programmatic access to identity operations.

Create API Key

POST /api/idp/api-keys

Create a new API key for the authenticated identity.

Authentication: Required

Request:

{
  "name": "Server automation",
  "expiresAt": 1766436000
}
Field Type Required Description
name string No Human-readable name
expiresAt number No Expiration timestamp (Unix seconds)

Response:

{
  "data": {
    "apiKey": {
      "id": 123,
      "idTag": "alice.cloudillo.net",
      "keyPrefix": "clid_abc",
      "name": "Server automation",
      "createdAt": 1735000000,
      "lastUsedAt": null,
      "expiresAt": 1766436000
    },
    "plaintextKey": "clid_abc123def456ghi789..."
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}
Store Key Securely

The plaintextKey is only shown once during creation. Store it securely - it cannot be retrieved later.

List API Keys

GET /api/idp/api-keys

List API keys for the authenticated identity.

Authentication: Required

Query Parameters:

Parameter Type Required Description
limit number No Maximum results
offset number No Pagination offset

Response:

{
  "data": [
    {
      "id": 123,
      "idTag": "alice.cloudillo.net",
      "keyPrefix": "clid_abc",
      "name": "Server automation",
      "createdAt": 1735000000,
      "lastUsedAt": 1735050000,
      "expiresAt": 1766436000
    }
  ],
  "time": 1735000000,
  "reqId": "req_abc123"
}

Get API Key

GET /api/idp/api-keys/{id}

Get details of a specific API key.

Authentication: Required

Path Parameters:

Parameter Type Description
id number API key ID

Response:

{
  "data": {
    "id": 123,
    "idTag": "alice.cloudillo.net",
    "keyPrefix": "clid_abc",
    "name": "Server automation",
    "createdAt": 1735000000,
    "lastUsedAt": 1735050000,
    "expiresAt": 1766436000
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Delete API Key

DELETE /api/idp/api-keys/{id}

Delete an API key. The key will immediately become invalid.

Authentication: Required

Path Parameters:

Parameter Type Description
id number API key ID to delete

Response:

{
  "data": {
    "deleted": true,
    "id": 123
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Authorization Model

IDP operations use a two-tier authorization model:

Role Access Duration
Owner Full control Permanent
Registrar Create, view, update, delete Only while status is pending

After an identity is activated (status changes from pending to active), the registrar loses control. Only the owner retains access.

Identity Status

Status Description
pending Created but not yet activated by owner
active Activated and operational
suspended Temporarily disabled by administrator

See Also

Tags API

The Tags API provides tag management and listing functionality for organizing files and content.

Endpoints

List Tags

GET /api/tags

List all tags used by the current user.

Query Parameters:

  • prefix (optional) - Filter tags by prefix
  • withCounts (optional) - Include usage counts
  • limit (optional) - Maximum tags to return

Response:

{
  "data": [
    {
      "tag": "project-alpha",
      "count": 15
    },
    {
      "tag": "important",
      "count": 8
    }
  ],
  "time": 1705315800,
  "req_id": "req_xyz"
}

Example:

curl -H "Authorization: Bearer $TOKEN" \
  "https://cl-o.alice.cloudillo.net/api/tags?prefix=proj-&withCounts=true"

File Tag Endpoints

Tags are primarily managed through the Files API.

Add Tag to File

PUT /api/files/{fileId}/tag/{tag}

Add a tag to a file.

Path Parameters:

  • fileId - File ID
  • tag - Tag name (URL-encoded)

Response:

{
  "data": {
    "tags": ["project-alpha", "important", "new-tag"]
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Example:

curl -X PUT -H "Authorization: Bearer $TOKEN" \
  "https://cl-o.alice.cloudillo.net/api/files/file_abc123/tag/important"

Remove Tag from File

DELETE /api/files/{fileId}/tag/{tag}

Remove a tag from a file.

Path Parameters:

  • fileId - File ID
  • tag - Tag name (URL-encoded)

Response:

{
  "data": {
    "tags": ["project-alpha", "important"]
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Example:

curl -X DELETE -H "Authorization: Bearer $TOKEN" \
  "https://cl-o.alice.cloudillo.net/api/files/file_abc123/tag/old-tag"

Client SDK Usage

import { createApiClient } from '@cloudillo/core'

const api = createApiClient({ idTag: 'alice.cloudillo.net', authToken: token })

// List all tags
const tags = await api.tags.list()

// List tags with prefix
const projectTags = await api.tags.list({ prefix: 'proj-' })

// Add tag to file
await api.files.addTag(fileId, 'important')

// Remove tag from file
await api.files.removeTag(fileId, 'old-tag')

Tag Naming Conventions

  • Tags are case-sensitive
  • Use lowercase with hyphens for consistency: project-alpha, q1-2025
  • Avoid special characters except hyphens and underscores
  • Keep tags concise but descriptive

Filtering Files by Tag

// List files with a specific tag
const files = await api.files.list({ tag: 'important' })

// Multiple tags (all must match)
const files = await api.files.list({ tags: ['project-alpha', 'active'] })

See Also

Push Notifications API

Overview

The Push Notifications API enables Web Push notifications for Cloudillo. It uses the VAPID (Voluntary Application Server Identification) protocol to securely deliver notifications to users’ browsers when they’re offline.

Push notifications are sent when actions are received for the user while they are not connected via WebSocket.

Endpoints

Get VAPID Public Key

GET /api/auth/vapid

Get the VAPID public key for subscribing to push notifications. The key is automatically generated on first request if it doesn’t exist.

Authentication: Required

Response:

{
  "vapidPublicKey": "BM5..."
}
Field Type Description
vapidPublicKey string Base64-encoded VAPID public key

Example:

const response = await api.notifications.getVapidPublicKey()
const vapidPublicKey = response.vapidPublicKey

// Use with browser Push API
const subscription = await registration.pushManager.subscribe({
  userVisibleOnly: true,
  applicationServerKey: vapidPublicKey
})

Register Subscription

POST /api/notifications/subscription

Register a push notification subscription. The subscription is stored and used to send notifications when the user is offline.

Authentication: Required

Request:

{
  "subscription": {
    "endpoint": "https://fcm.googleapis.com/fcm/send/...",
    "expirationTime": null,
    "keys": {
      "p256dh": "BKgS...",
      "auth": "Qs..."
    }
  }
}
Field Type Required Description
subscription.endpoint string Yes Push service endpoint URL
subscription.expirationTime number No Expiration timestamp (Unix milliseconds)
subscription.keys.p256dh string Yes P-256 public key (base64url)
subscription.keys.auth string Yes Auth secret (base64url)

Response:

{
  "id": 12345
}
Field Type Description
id number Subscription ID for later deletion

Example:

// After getting VAPID public key and subscribing via Push API
const browserSubscription = await registration.pushManager.subscribe({
  userVisibleOnly: true,
  applicationServerKey: vapidPublicKey
})

// Send subscription to server
const result = await api.notifications.subscribe({
  subscription: browserSubscription.toJSON()
})

// Store subscription ID for later unsubscription
localStorage.setItem('pushSubscriptionId', result.id)

Unregister Subscription

DELETE /api/notifications/subscription/{id}

Remove a push notification subscription. The subscription will no longer receive notifications.

Authentication: Required

Path Parameters:

Parameter Type Description
id number Subscription ID to delete

Response: 204 No Content

Example:

const subscriptionId = localStorage.getItem('pushSubscriptionId')
if (subscriptionId) {
  await api.notifications.unsubscribe(subscriptionId)
  localStorage.removeItem('pushSubscriptionId')
}

Complete Integration Example

// 1. Check if push is supported
if (!('PushManager' in window)) {
  console.log('Push notifications not supported')
  return
}

// 2. Get service worker registration
const registration = await navigator.serviceWorker.ready

// 3. Get VAPID public key from server
const { vapidPublicKey } = await api.notifications.getVapidPublicKey()

// 4. Subscribe via browser Push API
const subscription = await registration.pushManager.subscribe({
  userVisibleOnly: true,
  applicationServerKey: urlBase64ToUint8Array(vapidPublicKey)
})

// 5. Send subscription to Cloudillo server
const result = await api.notifications.subscribe({
  subscription: subscription.toJSON()
})

console.log('Push subscription registered with ID:', result.id)

// Helper function to convert base64 key
function urlBase64ToUint8Array(base64String: string): Uint8Array {
  const padding = '='.repeat((4 - base64String.length % 4) % 4)
  const base64 = (base64String + padding)
    .replace(/-/g, '+')
    .replace(/_/g, '/')
  const rawData = atob(base64)
  return Uint8Array.from([...rawData].map(char => char.charCodeAt(0)))
}

Notification Settings

Users can control which notification types they receive through the Settings API:

Setting Type Description
notify.push.message boolean Receive notifications for new messages
notify.push.mention boolean Receive notifications when mentioned
notify.push.reaction boolean Receive notifications for reactions
notify.push.connection boolean Receive notifications for connection requests
notify.push.follow boolean Receive notifications for new followers

Example:

// Disable reaction notifications
await api.settings.set('notify.push.reaction', false)

// Get current notification settings
const settings = await api.settings.list()
const pushSettings = Object.entries(settings)
  .filter(([key]) => key.startsWith('notify.push.'))

Web Push Standards

The implementation follows these RFCs:

RFC Title
RFC 8292 VAPID for Web Push
RFC 8188 Encrypted Content-Encoding for HTTP
RFC 8291 Message Encryption for Web Push

See Also

Trash API

The Trash API manages soft-deleted files, allowing users to restore or permanently delete items.

Concepts

When a file is deleted via DELETE /api/files/{fileId}, it is moved to the trash rather than being permanently deleted. This allows users to:

  • Recover accidentally deleted files
  • Review deleted items before permanent removal
  • Empty the trash to free up storage

Files remain in trash until explicitly restored or permanently deleted.

Endpoints

List Trashed Files

GET /api/files?parentId=__trash__

List all files currently in the trash.

Query Parameters:

  • parentId=__trash__ - Required to list trash
  • limit (optional) - Maximum files to return

Response:

{
  "data": [
    {
      "fileId": "file_abc123",
      "fileName": "document.pdf",
      "contentType": "application/pdf",
      "deletedAt": "2025-01-15T10:30:00Z",
      "originalParentId": "folder_xyz"
    }
  ],
  "time": 1705315800,
  "req_id": "req_xyz"
}

Example:

curl -H "Authorization: Bearer $TOKEN" \
  "https://cl-o.alice.cloudillo.net/api/files?parentId=__trash__&limit=50"

Restore File from Trash

POST /api/files/{fileId}/restore

Restore a file from the trash.

Path Parameters:

  • fileId - File ID to restore

Request Body:

{
  "parentId": "folder_xyz"
}

If parentId is omitted or null, the file is restored to the root folder.

Response:

{
  "data": {
    "fileId": "file_abc123",
    "parentId": "folder_xyz",
    "restoredAt": "2025-01-15T11:00:00Z"
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Example:

curl -X POST -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"parentId": "folder_xyz"}' \
  "https://cl-o.alice.cloudillo.net/api/files/file_abc123/restore"

Permanently Delete File

DELETE /api/files/{fileId}?permanent=true

Permanently delete a file from trash. The file must already be in trash.

Path Parameters:

  • fileId - File ID to permanently delete

Query Parameters:

  • permanent=true - Required for permanent deletion

Response:

{
  "data": {
    "fileId": "file_abc123",
    "permanent": true
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Example:

curl -X DELETE -H "Authorization: Bearer $TOKEN" \
  "https://cl-o.alice.cloudillo.net/api/files/file_abc123?permanent=true"

Empty Trash

DELETE /api/trash

Permanently delete all files in the trash.

Response:

{
  "data": {
    "deletedCount": 15
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Example:

curl -X DELETE -H "Authorization: Bearer $TOKEN" \
  "https://cl-o.alice.cloudillo.net/api/trash"

Client SDK Usage

import { createApiClient } from '@cloudillo/core'

const api = createApiClient({ idTag: 'alice.cloudillo.net', authToken: token })

// Move file to trash (soft delete)
await api.files.delete(fileId)

// List trashed files
const trashed = await api.trash.list()

// Restore a file
await api.files.restore(fileId, 'folder_xyz')

// Restore to root
await api.files.restore(fileId)

// Permanently delete a single file
await api.files.permanentDelete(fileId)

// Empty entire trash
const result = await api.trash.empty()
console.log(`Deleted ${result.deletedCount} files`)

File Lifecycle

┌──────────────┐     DELETE      ┌──────────────┐
│    Active    │ ───────────────>│    Trash     │
│    File      │                 │    (soft)    │
└──────────────┘                 └──────────────┘
                                        │
                                        │ restore
                                        │
                ┌───────────────────────┘
                │
                v
        ┌──────────────┐
        │    Active    │
        │    File      │
        └──────────────┘


┌──────────────┐  DELETE?permanent  ┌──────────────┐
│    Trash     │ ──────────────────>│   Deleted    │
│    (soft)    │                    │  (permanent) │
└──────────────┘                    └──────────────┘

Automatic Trash Cleanup

Info

Automatic trash cleanup (retention period) is planned for future releases. Currently, files remain in trash indefinitely until manually restored or deleted.

See Also

Communities API

The Communities API enables creation and management of community profiles.

Concepts

Communities are special profile types that can have multiple members with different roles. They provide:

  • Shared content feeds
  • Member management with role hierarchy
  • Community-specific settings

Community Roles

Roles form a hierarchy from least to most privileged:

Role Level Description
public 0 Anyone (non-member)
follower 1 Following the community
supporter 2 Supporter/subscriber
contributor 3 Can create content
moderator 4 Can moderate content
leader 5 Full administrative access

Endpoints

Create Community

PUT /api/profiles/{idTag}

Create a new community profile.

Path Parameters:

  • idTag - Identity tag for the new community (e.g., mygroup.cloudillo.net)

Request Body:

{
  "type": "community",
  "name": "My Community",
  "bio": "A community for enthusiasts",
  "ownerIdTag": "alice.cloudillo.net"
}

Response:

{
  "data": {
    "idTag": "mygroup.cloudillo.net",
    "name": "My Community",
    "type": "community",
    "bio": "A community for enthusiasts",
    "createdAt": "2025-01-15T10:30:00Z"
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Example:

curl -X PUT -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"type":"community","name":"My Community","ownerIdTag":"alice.cloudillo.net"}' \
  "https://cl-o.alice.cloudillo.net/api/profiles/mygroup.cloudillo.net"

Verify Community Availability

POST /api/profiles/verify

Check if a community identity is available.

Request Body:

{
  "type": "community",
  "idTag": "mygroup.cloudillo.net"
}

Response:

{
  "data": {
    "available": true,
    "errors": [],
    "serverAddresses": ["192.168.1.100"]
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

If not available:

{
  "data": {
    "available": false,
    "errors": ["E-PROFILE-EXISTS"],
    "serverAddresses": []
  }
}

Client SDK Usage

import { createApiClient } from '@cloudillo/core'

const api = createApiClient({ idTag: 'alice.cloudillo.net', authToken: token })

// Verify community name is available
const verification = await api.profile.verify({
  type: 'community',
  idTag: 'mygroup.cloudillo.net'
})

if (verification.available) {
  // Create the community
  const community = await api.communities.create('mygroup.cloudillo.net', {
    type: 'community',
    name: 'My Community',
    ownerIdTag: 'alice.cloudillo.net'
  })
}

Member Management

Member roles are managed through the Profiles API.

Update Member Role

// As community leader/moderator
await api.profiles.adminUpdate('member.cloudillo.net', {
  roles: ['contributor']
})

List Community Members

const members = await api.profiles.list({
  community: 'mygroup.cloudillo.net',
  role: 'contributor'
})

Role-Based Permissions

Use the ROLE_LEVELS constant from @cloudillo/types for permission checks:

import { ROLE_LEVELS, CommunityRole } from '@cloudillo/types'

function hasPermission(userRole: CommunityRole, requiredRole: CommunityRole): boolean {
  return ROLE_LEVELS[userRole] >= ROLE_LEVELS[requiredRole]
}

// Check if user can moderate
if (hasPermission(userRole, 'moderator')) {
  // Allow moderation actions
}

See Also

Admin API

Overview

The Admin API provides administrative operations for system administrators.

Warning

Admin endpoints require elevated privileges. These operations are restricted to users with the SADM (system admin) role.

Endpoints

List Tenants

GET /api/admin/tenants

List all tenants (identities) managed by this server.

Authentication: Required (admin role)

Query Parameters:

  • limit - Maximum results (default: 20)
  • offset - Skip N results for pagination

Response:

{
  "data": [
    {
      "tnId": 12345,
      "idTag": "alice.cloudillo.net",
      "name": "Alice",
      "type": "person",
      "profilePic": "b1~abc123",
      "createdAt": "2025-01-01T00:00:00Z"
    },
    {
      "tnId": 12346,
      "idTag": "bob.cloudillo.net",
      "name": "Bob",
      "type": "person",
      "createdAt": "2025-01-02T00:00:00Z"
    }
  ],
  "time": "2025-01-15T10:30:00Z"
}

Example:

curl -H "Authorization: Bearer $ADMIN_TOKEN" \
  "https://cl-o.admin.cloudillo.net/api/admin/tenants?limit=50"

Send Password Reset

POST /api/admin/tenants/{id_tag}/password-reset

Send a password reset email to a tenant.

Authentication: Required (admin role)

Path Parameters:

  • id_tag - Identity tag of the tenant

Response:

{
  "data": {
    "sent": true
  },
  "time": "2025-01-15T10:30:00Z"
}

Example:

curl -X POST -H "Authorization: Bearer $ADMIN_TOKEN" \
  "https://cl-o.admin.cloudillo.net/api/admin/tenants/alice.cloudillo.net/password-reset"

Send Test Email

POST /api/admin/email/test

Send a test email to verify SMTP configuration.

Authentication: Required (admin role)

Request Body:

{
  "to": "admin@example.com"
}

Response:

{
  "data": {
    "sent": true
  },
  "time": "2025-01-15T10:30:00Z"
}

Example:

curl -X POST -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"to":"admin@example.com"}' \
  "https://cl-o.admin.cloudillo.net/api/admin/email/test"

Update Profile (Admin)

PATCH /api/admin/profiles/{id_tag}

Admin update of a profile (roles, status, ban metadata).

Authentication: Required (admin role)

Path Parameters:

  • id_tag - Identity tag of the profile to update

Request Body:

{
  "name": "Updated Name",
  "status": "Suspended",
  "roles": ["user", "moderator"],
  "banExpiresAt": "2025-02-01T00:00:00Z",
  "banReason": "Terms of service violation"
}

Profile Status Values:

Status Description
Active Normal active account
Trusted Verified/trusted account
Blocked Blocked from interactions
Muted Content hidden from feeds
Suspended Account suspended (temporary)
Banned Account banned (permanent)

Response:

{
  "data": {
    "idTag": "user.cloudillo.net",
    "name": "Updated Name",
    "status": "Suspended",
    "roles": ["user", "moderator"]
  },
  "time": "2025-01-15T10:30:00Z"
}

Proxy Site Management

Manage reverse proxy sites for hosting custom domains.

List Proxy Sites

GET /api/admin/proxy-sites

List all configured proxy sites.

Authentication: Required (admin role)

Response:

{
  "data": [
    {
      "siteId": 1,
      "domain": "docs.example.com",
      "backendUrl": "http://localhost:8080",
      "status": "A",
      "type": "basic",
      "certExpiresAt": "2025-06-01T00:00:00Z",
      "config": {},
      "createdAt": "2025-01-01T00:00:00Z",
      "updatedAt": "2025-01-01T00:00:00Z"
    }
  ],
  "time": "2025-01-15T10:30:00Z"
}

Create Proxy Site

POST /api/admin/proxy-sites

Create a new proxy site configuration.

Authentication: Required (admin role)

Request Body:

{
  "domain": "docs.example.com",
  "backendUrl": "http://localhost:8080",
  "type": "basic",
  "config": {}
}
Field Type Required Description
domain string Yes Domain name for the proxy site
backendUrl string Yes Backend URL to proxy requests to
type string No Proxy type: basic (default) or advanced
config object No Additional proxy configuration

Response (201 Created): Returns the created ProxySite object.

Get Proxy Site

GET /api/admin/proxy-sites/{site_id}

Get details of a specific proxy site.

Authentication: Required (admin role)

Path Parameters:

  • site_id - The proxy site ID

Update Proxy Site

PATCH /api/admin/proxy-sites/{site_id}

Update a proxy site configuration.

Authentication: Required (admin role)

Path Parameters:

  • site_id - The proxy site ID

Request Body:

{
  "backendUrl": "http://localhost:9090",
  "status": "A",
  "config": {}
}
Field Type Description
backendUrl string Updated backend URL
status string A (active) or D (disabled)
type string Proxy type
config object Updated configuration

Delete Proxy Site

DELETE /api/admin/proxy-sites/{site_id}

Delete a proxy site configuration.

Authentication: Required (admin role)

Path Parameters:

  • site_id - The proxy site ID

Response: 204 No Content

Renew Proxy Site Certificate

POST /api/admin/proxy-sites/{site_id}/renew-cert

Trigger TLS certificate renewal for a proxy site.

Authentication: Required (admin role)

Path Parameters:

  • site_id - The proxy site ID

Community Invite

Invite Community

POST /api/admin/invite-community

Send an invitation to a community to join the server.

Authentication: Required (admin role)

Request Body:

{
  "targetIdTag": "community.example.com",
  "expiresInDays": 30,
  "message": "Join our platform!"
}
Field Type Required Description
targetIdTag string Yes Identity tag of the community to invite
expiresInDays number No Invitation expiry in days (default: 30)
message string No Optional invitation message

Response:

{
  "data": {
    "refId": "ref_abc123",
    "inviteUrl": "https://example.com/invite/ref_abc123",
    "targetIdTag": "community.example.com",
    "expiresAt": 1740000000
  },
  "time": "2025-01-15T10:30:00Z"
}

Client SDK Usage

import { createApiClient } from '@cloudillo/core'

const api = createApiClient({ idTag: 'admin.cloudillo.net', authToken: adminToken })

// List all tenants
const tenants = await api.admin.listTenants({ limit: 100 })

// Search tenants
const results = await api.admin.listTenants({ q: 'alice' })

// Send password reset
await api.admin.sendPasswordReset('alice.cloudillo.net')

// Send test email
await api.admin.sendTestEmail('admin@example.com')

// Suspend a user
await api.profiles.adminUpdate('baduser.cloudillo.net', {
  status: 'S',
  ban_reason: 'Spam'
})

Security Considerations

  • Admin endpoints require admin role authentication
  • All admin actions are logged for audit purposes
  • Password reset emails are rate-limited
  • Suspension requires a reason for accountability

See Also

IDP Management API

The IDP (Identity Provider) Management API enables identity provider administrators to manage identities and API keys for their hosted identities.

Info

This API is for identity provider administrators who host identities for other users (e.g., cloudillo.net service). For end-user identity operations, see IDP API.

Endpoints

List Managed Identities

GET /api/idp/identities

List all identities managed by this identity provider.

Query Parameters:

  • q (optional) - Search query
  • status (optional) - Filter by status
  • cursor (optional) - Pagination cursor
  • limit (optional) - Maximum results

Response:

{
  "data": [
    {
      "idTag": "alice.cloudillo.net",
      "email": "alice@example.com",
      "status": "active",
      "createdAt": "2025-01-01T00:00:00Z",
      "ownerIdTag": null,
      "dyndns": true
    },
    {
      "idTag": "bob.cloudillo.net",
      "email": "bob@example.com",
      "status": "active",
      "createdAt": "2025-01-02T00:00:00Z",
      "ownerIdTag": "alice.cloudillo.net",
      "dyndns": false
    }
  ],
  "time": 1705315800,
  "req_id": "req_xyz"
}

Example:

curl -H "Authorization: Bearer $IDP_TOKEN" \
  "https://cl-o.cloudillo.net/api/idp/identities?limit=50"

Create Identity

POST /api/idp/identities

Create a new identity under this provider.

Request Body:

{
  "idTag": "newuser.cloudillo.net",
  "email": "newuser@example.com",
  "ownerIdTag": "alice.cloudillo.net",
  "createApiKey": true,
  "apiKeyName": "Initial Key"
}

Response:

{
  "data": {
    "idTag": "newuser.cloudillo.net",
    "email": "newuser@example.com",
    "status": "pending",
    "createdAt": "2025-01-15T10:30:00Z",
    "apiKey": "clak_abc123xyz..."
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}
Warning

The apiKey is only returned once when createApiKey: true. Store it securely.

Example:

curl -X POST -H "Authorization: Bearer $IDP_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"idTag":"newuser.cloudillo.net","email":"newuser@example.com","createApiKey":true}' \
  "https://cl-o.cloudillo.net/api/idp/identities"

Get Identity Details

GET /api/idp/identities/{idTag}

Get details for a specific managed identity.

Path Parameters:

  • idTag - Identity tag (URL-encoded)

Response:

{
  "data": {
    "idTag": "alice.cloudillo.net",
    "email": "alice@example.com",
    "status": "active",
    "createdAt": "2025-01-01T00:00:00Z",
    "lastLoginAt": "2025-01-15T10:30:00Z",
    "ownerIdTag": null,
    "dyndns": true
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Update Identity

PATCH /api/idp/identities/{idTag}

Update identity settings.

Path Parameters:

  • idTag - Identity tag (URL-encoded)

Request Body:

{
  "dyndns": true
}

Response:

{
  "data": {
    "idTag": "alice.cloudillo.net",
    "dyndns": true
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Update Identity Address

PUT /api/idp/identities/{idTag}/address

Update the DNS address mapping for a managed identity. This is used for dynamic DNS updates when self-hosting.

Path Parameters:

  • idTag - Identity tag (URL-encoded)

Request Body:

{
  "address": "192.168.1.100"
}

Response:

{
  "data": {
    "idTag": "alice.cloudillo.net",
    "address": "192.168.1.100"
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Delete Identity

DELETE /api/idp/identities/{idTag}

Delete a managed identity.

Path Parameters:

  • idTag - Identity tag (URL-encoded)

Response:

{
  "data": null,
  "time": 1705315800,
  "req_id": "req_xyz"
}

List API Keys for Identity

GET /api/idp/api-keys?idTag={idTag}

List API keys for a specific managed identity.

Query Parameters:

  • idTag - Identity tag to list keys for

Response:

{
  "data": [
    {
      "keyId": 1,
      "name": "Production Key",
      "createdAt": "2025-01-01T00:00:00Z",
      "lastUsedAt": "2025-01-15T10:30:00Z"
    }
  ],
  "time": 1705315800,
  "req_id": "req_xyz"
}

Create API Key for Identity

POST /api/idp/api-keys

Create a new API key for a managed identity.

Request Body:

{
  "idTag": "alice.cloudillo.net",
  "name": "New API Key"
}

Response:

{
  "data": {
    "keyId": 2,
    "name": "New API Key",
    "apiKey": "clak_newkey123...",
    "createdAt": "2025-01-15T10:30:00Z"
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Get API Key Details

GET /api/idp/api-keys/{keyId}

Get details of a specific API key.

Path Parameters:

  • keyId - API key ID

Response:

{
  "data": {
    "keyId": 1,
    "name": "Production Key",
    "createdAt": "2025-01-01T00:00:00Z",
    "lastUsedAt": "2025-01-15T10:30:00Z"
  },
  "time": 1705315800,
  "req_id": "req_xyz"
}

Revoke API Key

DELETE /api/idp/api-keys/{keyId}?idTag={idTag}

Revoke an API key for a managed identity.

Path Parameters:

  • keyId - Key ID to revoke

Query Parameters:

  • idTag - Identity tag the key belongs to

Response:

{
  "data": null,
  "time": 1705315800,
  "req_id": "req_xyz"
}

Client SDK Usage

import { createApiClient } from '@cloudillo/core'

const api = createApiClient({ idTag: 'cloudillo.net', authToken: idpToken })

// List managed identities
const identities = await api.idpManagement.listIdentities({ limit: 100 })

// Create new identity with API key
const newIdentity = await api.idpManagement.createIdentity({
  idTag: 'newuser.cloudillo.net',
  email: 'newuser@example.com',
  createApiKey: true,
  apiKeyName: 'Initial Key'
})
console.log('API Key:', newIdentity.apiKey) // Store securely!

// Get identity details
const identity = await api.idpManagement.getIdentity('alice.cloudillo.net')

// Update identity settings
await api.idpManagement.updateIdentity('alice.cloudillo.net', { dyndns: true })

// Delete identity
await api.idpManagement.deleteIdentity('olduser.cloudillo.net')

// Manage API keys
const keys = await api.idpManagement.listApiKeys('alice.cloudillo.net')
const newKey = await api.idpManagement.createApiKey({
  idTag: 'alice.cloudillo.net',
  name: 'Backup Key'
})
await api.idpManagement.deleteApiKey(keyId, 'alice.cloudillo.net')

Use Cases

Provisioning New Users

async function provisionUser(email: string, subdomain: string) {
  const idTag = `${subdomain}.cloudillo.net`

  // Create identity with initial API key
  const result = await api.idpManagement.createIdentity({
    idTag,
    email,
    createApiKey: true,
    apiKeyName: 'Setup Key'
  })

  // Send setup instructions with API key
  await sendSetupEmail(email, {
    idTag,
    apiKey: result.apiKey
  })

  return result
}

Automating Identity Management

// Deactivate inactive users
const identities = await api.idpManagement.listIdentities()

for (const identity of identities.data) {
  const daysSinceLogin = daysSince(identity.lastLoginAt)
  if (daysSinceLogin > 365) {
    await api.idpManagement.deleteIdentity(identity.idTag)
  }
}

See Also

WebSocket API

Cloudillo provides three WebSocket endpoints for real-time features.

Endpoint Purpose Protocol
/ws/bus Notifications and direct messaging JSON ({ id, cmd, data })
/ws/rtdb/{file_id} Real-time database sync JSON ({ id, type, ... })
/ws/crdt/{doc_id} Collaborative document editing Binary (Yjs sync protocol)

Authentication

Tokens are passed as a query parameter:

wss://server.com/ws/bus?token=eyJhbGc...
wss://server.com/ws/rtdb/file_123?token=eyJhbGc...
wss://server.com/ws/crdt/doc_123?token=eyJhbGc...&access=write

Additional query parameters for RTDB and CRDT:

Parameter Values Description
access read, write Force access level (default: determined by permissions)
via file ID Container file ID for embedded access (caps access by share entry)
Info

The /ws/bus endpoint requires authentication – unauthenticated connections are rejected with close code 4401. The RTDB and CRDT endpoints support guest (unauthenticated) access with read-only permissions for public files.

WebSocket close codes

Code Meaning
4400 Invalid store ID format
4401 Unauthorized (authentication required)
4403 Access denied or write access denied
4404 File/document not found
4409 Store type mismatch (e.g. RTDB endpoint for a CRDT file)
4500 Internal server error

Message bus (/ws/bus)

The bus provides direct user-to-user messaging and notifications. All messages use the format { id, cmd, data }.

Client → Server:

cmd Description
ping Keepalive; server responds with ack "pong"
ACTION Send an action; server responds with ack "ok"
Any other Custom command (presence, typing, etc.); server acks with "ok"

Server → Client:

Messages from other users are forwarded with the same { id, cmd, data } format. The bus does not use channels or subscriptions – it’s a direct messaging system where the server forwards relevant messages to registered users.

Client-side connection

The openMessageBus() function from @cloudillo/core returns a raw WebSocket:

import { openMessageBus } from '@cloudillo/core'

const ws = openMessageBus({ idTag: 'alice', authToken: token })

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data)
  console.log('Command:', msg.cmd, 'Data:', msg.data)
}

// Send a command
ws.send(JSON.stringify({ id: '1', cmd: 'ping', data: {} }))

Real-time database (/ws/rtdb/{file_id})

Real-time synchronization of structured data. All messages use the format { id, type, ...payload }. See RTDB for the client library documentation.

Client → server messages

Type Key fields Description
transaction operations: [{type, path, data}] Atomic batch of create/update/replace/delete operations
query path, filter?, sort?, limit?, offset?, aggregate? Query documents with optional filtering and aggregation
get path Get a single document
subscribe path, filter?, aggregate? Start real-time change notifications for a path
unsubscribe subscriptionId Stop receiving change notifications
lock path, mode Lock a document ("soft" or "hard")
unlock path Release a document lock
ping Keepalive

Server → client messages

Type Key fields Description
ack status, timestamp Acknowledges a transaction/command
transactionResult results: [{ref?, id}] Per-operation results from a transaction
queryResult data: [...] Query results
getResult data Single document result
subscribeResult subscriptionId, data Initial subscription data
change subscriptionId, event: {action, path, data?} Real-time change notification
lockResult locked, holder?, mode? Lock operation result
pong Keepalive response
error code, message Error response

Transaction operations

Each operation in a transaction has a type and a path:

Operation Description
create Create a document; returns generated ID. Supports ref for cross-referencing within the transaction
update Shallow merge (Firebase-style) with existing document
replace Full document replacement (no merge)
delete Delete a document

All operations in a transaction are atomic – if any fails, the entire transaction rolls back. Operations support computed values ($op, $fn, $query) and reference substitution (${$ref} patterns) for creating related documents in a single transaction.

Change event actions

The event.action field in change notifications can be: create, update, delete, lock, unlock, or ready.

Client-side connection

The openRTDB() function from @cloudillo/core returns a raw WebSocket. For higher-level usage, use the @cloudillo/rtdb client library:

import { createRtdbClient } from '@cloudillo/rtdb'
import { getRtdbUrl } from '@cloudillo/core'

const db = createRtdbClient({
  dbId: fileId,
  auth: { getToken: () => bus.accessToken },
  serverUrl: getRtdbUrl(bus.idTag!, fileId, bus.accessToken!)
})

Store files

RTDB supports auto-created store files using the pattern s~{app_id} (e.g. s~taskillo). These are created automatically on first WebSocket connection, providing persistent app-specific data storage without manual file creation.

Collaborative documents (/ws/crdt/{doc_id})

CRDT synchronization using the Yjs binary protocol. See CRDT for full documentation.

Protocol

The endpoint uses the y-websocket binary protocol:

Message type Code Description
MSG_SYNC 0 Sync protocol (SyncStep1, SyncStep2, Update)
MSG_AWARENESS 1 User presence and cursor updates

The sync flow:

  1. Client sends SyncStep1 (state vector)
  2. Server responds with SyncStep2 (missing updates)
  3. Both sides exchange Updates incrementally as edits happen
  4. Awareness messages broadcast cursor positions and user presence

Read-only connections can receive sync and awareness data but cannot send Update messages.

Client-side connection

Use openYDoc() from @cloudillo/crdt, which handles authentication, client ID reuse, offline caching, and token refresh automatically:

import { openYDoc } from '@cloudillo/crdt'
import * as Y from 'yjs'

const yDoc = new Y.Doc()
const { provider, persistence, offlineCached } = await openYDoc(yDoc, 'ownerTag:docId')

// Access shared types
const yText = yDoc.getText('content')

// Awareness is available via provider.awareness
Note

openYDoc() automatically handles WebSocket close codes: on 4401 (unauthorized) it requests a fresh token from the shell and reconnects. On other 44xx errors it stops reconnection and notifies the shell via bus.notifyError().

Connection lifecycle

All three endpoints share common behaviors:

  • Heartbeat: Server sends WebSocket ping frames every 30 seconds
  • Multi-tab support: Each connection gets a unique conn_id; multiple connections per user are supported
  • Cleanup: On disconnect, locks are released (RTDB), subscriptions cancelled, and user unregistered (bus)
  • Activity tracking: File access and modification times are recorded (throttled to 60-second intervals)

See also

  • RTDB – real-time database client library
  • CRDT – collaborative editing with Yjs
  • Authentication – token management

RTDB (Real-Time Database)

The Cloudillo RTDB provides a Firebase-like real-time database with TypeScript support.

RTDB vs CRDT

RTDB is best for structured data with queries (todos, settings, lists). For collaborative document editing where multiple users edit simultaneously, see CRDT. Compare both in Data Storage & Access.

Installation

pnpm add @cloudillo/rtdb

Quick Start

import { getAppBus } from '@cloudillo/core'
import { RtdbClient } from '@cloudillo/rtdb'
import { getRtdbUrl } from '@cloudillo/core'

// Initialize Cloudillo
const bus = getAppBus()
await bus.init('my-app')

// Create RTDB client
const rtdb = new RtdbClient({
  dbId: 'my-database-id',
  auth: {
    getToken: () => bus.accessToken
  },
  serverUrl: getRtdbUrl(bus.idTag!, 'my-database-id', bus.accessToken!)
})

// Connect to the database
await rtdb.connect()

// Get collection reference
const todos = rtdb.collection('todos')

// Create document
const batch = rtdb.batch()
batch.create(todos, {
  title: 'Learn Cloudillo RTDB',
  completed: false,
  createdAt: Date.now()
})
await batch.commit()

// Subscribe to changes
todos.onSnapshot((snapshot) => {
  console.log('Todos:', snapshot.docs.map(doc => doc.data()))
})

RtdbClient Constructor

interface RtdbClientOptions {
  dbId: string                    // Database/file ID
  auth: {
    getToken: () => string | undefined | Promise<string | undefined>
  }
  serverUrl: string               // WebSocket URL
  options?: {
    enableCache?: boolean         // Default: false
    reconnect?: boolean           // Default: true
    reconnectDelay?: number       // Default: 1000ms
    maxReconnectDelay?: number    // Default: 30000ms
    debug?: boolean               // Default: false
  }
}

Example with all options:

import { RtdbClient } from '@cloudillo/rtdb'
import { getRtdbUrl } from '@cloudillo/core'

const rtdb = new RtdbClient({
  dbId: 'my-database-id',
  auth: {
    // Token provider function - called when connection needs auth
    getToken: async () => {
      // Can be sync or async
      return bus.accessToken
    }
  },
  serverUrl: getRtdbUrl(bus.idTag!, 'my-database-id', bus.accessToken!),
  options: {
    enableCache: true,        // Enable local caching
    reconnect: true,          // Auto-reconnect on disconnect
    reconnectDelay: 1000,     // Initial reconnect delay
    maxReconnectDelay: 30000, // Max reconnect delay (exponential backoff)
    debug: false              // Enable debug logging
  }
})

Connection Management

// Connect to database
await rtdb.connect()

// Disconnect
await rtdb.disconnect()

// Check connection status
if (rtdb.isConnected()) {
  console.log('Connected')
}

// Diagnostics
console.log('Pending requests:', rtdb.getPendingRequests())
console.log('Active subscriptions:', rtdb.getActiveSubscriptions())

Core Concepts

Collections

Collections are groups of documents, similar to tables in SQL.

const users = rtdb.collection('users')
const posts = rtdb.collection('posts')
const comments = rtdb.collection('comments')

// Typed collections
interface Todo {
  title: string
  completed: boolean
  createdAt: number
}
const todos = rtdb.collection<Todo>('todos')

Documents

Documents are individual records accessed by path.

// Reference a document by path
const userDoc = rtdb.ref('users/alice')

// Get document data
const snapshot = await userDoc.get()
if (snapshot.exists) {
  console.log(snapshot.data())
}

CRUD Operations

Create

Use batch operations to create documents:

const todos = rtdb.collection('todos')
const batch = rtdb.batch()

// Create with auto-generated ID
batch.create(todos, {
  title: 'New task',
  completed: false
})

// Create with ref for tracking
batch.create(todos, {
  title: 'Another task',
  completed: false
}, { ref: 'task-ref' })

// Commit returns results with IDs
const results = await batch.commit()
console.log('Created IDs:', results.map(r => r.id))

Read

// Get single document
const doc = rtdb.ref('todos/task_123')
const snapshot = await doc.get()
if (snapshot.exists) {
  console.log(snapshot.data())
}

// Query collection
const todos = rtdb.collection('todos')
const results = await todos.get()
results.docs.forEach(doc => {
  console.log(doc.id, doc.data())
})

Update

const batch = rtdb.batch()
const todoRef = rtdb.ref('todos/task_123')

// Partial update
batch.update(todoRef, {
  completed: true
})

await batch.commit()

Delete

const batch = rtdb.batch()
const todoRef = rtdb.ref('todos/task_123')

batch.delete(todoRef)

await batch.commit()

Queries

Query Options

interface QueryOptions {
  filter?: {
    equals?: Record<string, any>
    notEquals?: Record<string, any>
    greaterThan?: Record<string, any>
    lessThan?: Record<string, any>
    greaterThanOrEqual?: Record<string, any>
    lessThanOrEqual?: Record<string, any>
    in?: Record<string, any[]>
    notIn?: Record<string, any[]>
    arrayContains?: Record<string, any>
    arrayContainsAny?: Record<string, any[]>
    arrayContainsAll?: Record<string, any[]>
  }
  sort?: Array<{ field: string; ascending: boolean }>
  limit?: number
  offset?: number
}

You can also build queries using the chainable .where() builder:

type WhereFilterOp =
  | '=='  | '!='
  | '<'   | '>'
  | '<='  | '>='
  | 'in'  | 'not-in'
  | 'array-contains'
  | 'array-contains-any'
  | 'array-contains-all'

collection.where(field: string, op: WhereFilterOp, value: any)

Filtering

const todos = rtdb.collection('todos')

// Filter with options object
const incomplete = await todos.query({
  filter: {
    equals: { completed: false }
  }
})

// Filter with chainable .where() builder
const incomplete2 = await todos
  .where('completed', '==', false)
  .get()

Filter Operators

// Equality
const active = await todos.where('status', '==', 'active').get()
const notDone = await todos.where('status', '!=', 'done').get()

// Comparison
const highPriority = await todos.where('priority', '>', 3).get()
const recent = await todos.where('createdAt', '>=', lastWeek).get()

// Set membership
const selected = await todos
  .where('status', 'in', ['active', 'pending'])
  .get()
const excluded = await todos
  .where('status', 'not-in', ['archived', 'deleted'])
  .get()

// Array filters
const tagged = await todos
  .where('tags', 'array-contains', 'urgent')
  .get()
const anyTag = await todos
  .where('tags', 'array-contains-any', ['urgent', 'important'])
  .get()
const allTags = await todos
  .where('tags', 'array-contains-all', ['frontend', 'bug'])
  .get()

// Chain multiple filters
const filtered = await todos
  .where('completed', '==', false)
  .where('priority', '>', 2)
  .get()

Sorting

// Sort ascending
const sorted = await todos.query({
  sort: [{ field: 'createdAt', ascending: true }]
})

// Sort descending
const newest = await todos.query({
  sort: [{ field: 'createdAt', ascending: false }]
})

// Multiple sorts
const prioritized = await todos.query({
  sort: [
    { field: 'priority', ascending: false },
    { field: 'createdAt', ascending: true }
  ]
})

Pagination

// Limit results
const first10 = await todos.query({
  limit: 10
})

// Pagination with offset
const page2 = await todos.query({
  limit: 20,
  offset: 20
})

Combined Queries

const results = await todos.query({
  filter: {
    equals: { completed: false }
  },
  sort: [{ field: 'createdAt', ascending: false }],
  limit: 10
})

Real-Time Subscriptions

Collection Subscriptions

const todos = rtdb.collection('todos')

// Subscribe to all documents
const unsubscribe = todos.onSnapshot((snapshot) => {
  console.log('Total todos:', snapshot.size)

  snapshot.docs.forEach(doc => {
    console.log(doc.id, doc.data())
  })

  // Track changes
  const changes = snapshot.docChanges()
  changes.forEach(change => {
    switch (change.type) {
      case 'added':
        console.log('New:', change.doc.id)
        break
      case 'modified':
        console.log('Updated:', change.doc.id)
        break
      case 'removed':
        console.log('Deleted:', change.doc.id)
        break
    }
  })
})

// Unsubscribe later
unsubscribe()

Document Subscriptions

const todoRef = rtdb.ref('todos/task_123')

const unsubscribe = todoRef.onSnapshot((snapshot) => {
  if (snapshot.exists) {
    console.log('Todo updated:', snapshot.data())
  } else {
    console.log('Todo deleted')
  }
})

Filtered Subscriptions

// Using options object
const unsubscribe = todos.subscribe({
  filter: {
    equals: { completed: false }
  }
}, (snapshot) => {
  console.log('Incomplete todos:', snapshot.size)
})

// Using .where() builder
const unsubscribe2 = todos
  .where('completed', '==', false)
  .onSnapshot((snapshot) => {
    console.log('Incomplete todos:', snapshot.size)
  })

Document Locking

Lock documents for exclusive or advisory editing access.

Lock Modes

  • soft — Advisory lock. Other clients can still write, but are notified that the document is locked.
  • hard — Enforced lock. The server rejects writes from other clients while the lock is held.

Locking and Unlocking

const docRef = rtdb.ref('todos/task_123')

// Acquire a soft (advisory) lock
const result = await docRef.lock('soft')
// result: { locked: true }

// Acquire a hard (exclusive) lock
const result2 = await docRef.lock('hard')
// result: { locked: true }

// If already locked by another client
const result3 = await docRef.lock('hard')
// result: { locked: false, holder: 'bob@example.com', mode: 'hard' }

// Release the lock
await docRef.unlock()

Lock Result

interface LockResult {
  locked: boolean         // Whether the lock was acquired
  holder?: string         // Identity of current lock holder (if denied)
  mode?: 'soft' | 'hard' // Lock mode of existing lock (if denied)
}

Lock Events

Listen for lock changes on a document using the onLock callback in snapshot options:

const docRef = rtdb.ref('todos/task_123')

const unsubscribe = docRef.onSnapshot({
  onLock: (event) => {
    if (event.action === 'lock') {
      console.log(`Locked by ${event.holder} (${event.mode})`)
    } else if (event.action === 'unlock') {
      console.log('Document unlocked')
    }
  }
}, (snapshot) => {
  console.log('Data:', snapshot.data())
})

Example: Exclusive Editing

async function startEditing(docRef: DocumentRef) {
  const result = await docRef.lock('hard')

  if (!result.locked) {
    alert(`Document is locked by ${result.holder}`)
    return false
  }

  // Edit the document...
  return true
}

async function stopEditing(docRef: DocumentRef) {
  await docRef.unlock()
}
Info

Locks have a TTL (time-to-live) and expire automatically if the client disconnects or fails to renew them. This prevents permanently locked documents from abandoned sessions.

Aggregate Queries

Perform server-side aggregations on collections.

Aggregate API

interface AggregateOptions {
  groupBy?: string             // Field to group results by
  ops: AggregateOp[]           // Aggregation operations
}

type AggregateOp = 'sum' | 'avg' | 'min' | 'max'

interface AggregateGroupEntry {
  group: any                   // Value of the groupBy field
  count: number                // Number of documents in the group
  [key: string]: any           // Aggregate results (e.g., sum_hours, avg_hours)
}

interface AggregateSnapshot {
  groups: AggregateGroupEntry[]
}

Basic Aggregation

const todos = rtdb.collection('todos')

// Aggregate with filters
const result = await todos
  .where('completed', '==', false)
  .aggregate({
    groupBy: 'status',
    ops: ['sum', 'avg']
  })
  .get()

result.groups.forEach(group => {
  console.log(`${group.group}: ${group.count} items`)
})

Real-Time Aggregate Subscriptions

Aggregate queries support real-time updates via onSnapshot:

const unsubscribe = todos
  .where('completed', '==', false)
  .aggregate({
    groupBy: 'status',
    ops: ['sum']
  })
  .onSnapshot((snapshot: AggregateSnapshot) => {
    snapshot.groups.forEach(group => {
      console.log(`${group.group}: ${group.count} items`)
    })
  })

Example: Task Dashboard

const tasks = rtdb.collection('tasks')

// Group tasks by status with count and total estimated hours
const unsubscribe = tasks
  .aggregate({
    groupBy: 'status',
    ops: ['sum', 'avg']
  })
  .onSnapshot((snapshot) => {
    snapshot.groups.forEach(({ group, count, sum_hours, avg_hours }) => {
      console.log(`${group}: ${count} tasks, ${sum_hours}h total, ${avg_hours}h avg`)
    })
    // Output:
    // todo: 12 tasks, 36h total, 3h avg
    // in_progress: 5 tasks, 20h total, 4h avg
    // done: 28 tasks, 84h total, 3h avg
  })

Batch Operations

Perform multiple operations atomically:

const todos = rtdb.collection('todos')
const batch = rtdb.batch()

// Create multiple
batch.create(todos, { title: 'Task 1', completed: false })
batch.create(todos, { title: 'Task 2', completed: false })

// Update existing
batch.update(rtdb.ref('todos/task_123'), { completed: true })

// Delete
batch.delete(rtdb.ref('todos/task_456'))

// Commit all operations atomically
const results = await batch.commit()
console.log('Batch results:', results)

BatchResult:

interface BatchResult {
  ref?: string   // Reference ID if provided
  id?: string    // Generated document ID
}

Indexes

Create indexes for efficient queries:

// Create index on a field
await rtdb.createIndex('todos', 'createdAt')
await rtdb.createIndex('todos', 'completed')

TypeScript Support

Full type safety with generics:

interface Todo {
  title: string
  completed: boolean
  createdAt: number
  tags?: string[]
}

const todos = rtdb.collection<Todo>('todos')

// TypeScript knows the shape
const snapshot = await todos.get()
snapshot.docs.forEach(doc => {
  const data = doc.data()
  console.log(data.title)     // string
  console.log(data.completed) // boolean
  // @ts-error: Property 'invalid' does not exist
  // console.log(data.invalid)
})

React Integration

import { useEffect, useState } from 'react'
import { useAuth } from '@cloudillo/react'
import { RtdbClient } from '@cloudillo/rtdb'
import { getRtdbUrl } from '@cloudillo/core'

interface Todo {
  title: string
  completed: boolean
}

function TodoList({ dbId }: { dbId: string }) {
  const [auth] = useAuth()
  const [todos, setTodos] = useState<Todo[]>([])
  const [rtdb, setRtdb] = useState<RtdbClient | null>(null)

  // Initialize RTDB client
  useEffect(() => {
    if (!auth?.token || !auth?.idTag) return

    const client = new RtdbClient({
      dbId,
      auth: { getToken: () => auth.token },
      serverUrl: getRtdbUrl(auth.idTag, dbId, auth.token!)
    })

    setRtdb(client)

    return () => {
      client.disconnect()
    }
  }, [auth?.token, auth?.idTag, dbId])

  // Subscribe to todos
  useEffect(() => {
    if (!rtdb) return

    const todos = rtdb.collection<Todo>('todos')
    const unsubscribe = todos.onSnapshot((snapshot) => {
      setTodos(snapshot.docs.map(doc => ({
        id: doc.id,
        ...doc.data()
      })))
    })

    return () => unsubscribe()
  }, [rtdb])

  return (
    <ul>
      {todos.map(todo => (
        <li key={todo.id}>{todo.title}</li>
      ))}
    </ul>
  )
}

Error Handling

The RTDB client provides typed errors:

import {
  RtdbError,
  ConnectionError,
  AuthError,
  PermissionError,
  NotFoundError,
  ValidationError,
  TimeoutError
} from '@cloudillo/rtdb'

try {
  await rtdb.connect()
} catch (error) {
  if (error instanceof ConnectionError) {
    console.log('Connection failed:', error.message)
  } else if (error instanceof AuthError) {
    console.log('Authentication failed:', error.message)
  } else if (error instanceof PermissionError) {
    console.log('Permission denied:', error.message)
  } else if (error instanceof NotFoundError) {
    console.log('Not found:', error.message)
  } else if (error instanceof TimeoutError) {
    console.log('Request timed out:', error.message)
  }
}

Best Practices

1. Use Connection Management

// Initialize once, reuse
const rtdb = new RtdbClient({ ... })
await rtdb.connect()

// Use throughout your app
const todos = rtdb.collection('todos')

2. Clean Up Subscriptions

// Always unsubscribe to prevent memory leaks
useEffect(() => {
  const unsubscribe = todos.onSnapshot(callback)
  return () => unsubscribe()
}, [])

3. Use Batch for Multiple Operations

// Good: Atomic batch operation
const batch = rtdb.batch()
batch.create(todos, { title: 'Task 1' })
batch.create(todos, { title: 'Task 2' })
await batch.commit()

// Avoid: Multiple separate requests
// await create({ title: 'Task 1' })
// await create({ title: 'Task 2' })

4. Use Typed Collections

// Define your types
interface Todo {
  title: string
  completed: boolean
}

// Get type safety
const todos = rtdb.collection<Todo>('todos')

5. Handle Connection State

// Check connection before operations
if (!rtdb.isConnected()) {
  await rtdb.connect()
}

// Handle reconnection in UI
const [connected, setConnected] = useState(false)

See Also

CRDT (Collaborative Editing)

Cloudillo uses Yjs for real-time collaborative editing with CRDT (Conflict-Free Replicated Data Types).

CRDT vs RTDB

CRDT is best for collaborative editing where multiple users edit simultaneously. For structured data with queries (todos, settings, lists), see RTDB. Compare all storage types in Data Storage & Access.

Installation

pnpm add yjs y-websocket

Quick Start

import * as cloudillo from '@cloudillo/core'
import * as Y from 'yjs'

// Initialize
await cloudillo.init('my-app')

// Create Yjs document
const yDoc = new Y.Doc()

// Open collaborative document
const { provider } = await cloudillo.openYDoc(yDoc, 'my-document-id')

// Use shared text
const yText = yDoc.getText('content')
yText.insert(0, 'Hello, collaborative world!')

// Listen for changes
yText.observe(() => {
  console.log('Text updated:', yText.toString())
})

Shared Types

Yjs provides several shared data types:

YText - Shared Text

Best for plain text or rich text content.

const yText = yDoc.getText('content')

// Insert text
yText.insert(0, 'Hello ')
yText.insert(6, 'world!')

// Delete text
yText.delete(0, 5) // Delete 5 characters from position 0

// Format text (for rich text)
yText.format(0, 5, { bold: true })

// Get text content
console.log(yText.toString()) // "world!"

// Observe changes
yText.observe((event) => {
  event.changes.delta.forEach(change => {
    if (change.insert) {
      console.log('Inserted:', change.insert)
    }
    if (change.delete) {
      console.log('Deleted', change.delete, 'characters')
    }
  })
})

YMap - Shared Object

Best for key-value data like form fields or settings.

const yMap = yDoc.getMap('settings')

// Set values
yMap.set('theme', 'dark')
yMap.set('fontSize', 14)
yMap.set('notifications', true)

// Get values
console.log(yMap.get('theme')) // "dark"

// Delete keys
yMap.delete('fontSize')

// Check existence
console.log(yMap.has('theme')) // true

// Iterate
yMap.forEach((value, key) => {
  console.log(`${key}: ${value}`)
})

// Observe changes
yMap.observe((event) => {
  event.changes.keys.forEach((change, key) => {
    if (change.action === 'add') {
      console.log(`Added ${key}:`, yMap.get(key))
    } else if (change.action === 'update') {
      console.log(`Updated ${key}:`, yMap.get(key))
    } else if (change.action === 'delete') {
      console.log(`Deleted ${key}`)
    }
  })
})

YArray - Shared Array

Best for lists like todos, comments, or items.

const yArray = yDoc.getArray('todos')

// Push items
yArray.push([
  { title: 'Task 1', done: false },
  { title: 'Task 2', done: false }
])

// Insert at position
yArray.insert(0, [{ title: 'Urgent task', done: false }])

// Delete
yArray.delete(0, 1) // Delete 1 item at position 0

// Get items
console.log(yArray.get(0)) // First item
console.log(yArray.toArray()) // All items as array

// Iterate
yArray.forEach((item, index) => {
  console.log(index, item)
})

// Observe changes
yArray.observe((event) => {
  console.log('Array changed:', event.changes)
})

YXmlFragment - Shared XML/HTML

Best for rich text editors with complex formatting.

const yXml = yDoc.getXmlFragment('document')

// Create elements
const paragraph = new Y.XmlElement('p')
paragraph.setAttribute('class', 'text')
paragraph.insert(0, [new Y.XmlText('Hello world')])

yXml.insert(0, [paragraph])

Awareness

Awareness tracks user presence, cursors, and selections in real-time.

const { provider } = await cloudillo.openYDoc(yDoc, 'doc_123')

// Set local awareness state
provider.awareness.setLocalState({
  user: {
    name: cloudillo.name,
    idTag: cloudillo.idTag,
    color: '#ff6b6b'
  },
  cursor: {
    line: 10,
    column: 5
  },
  selection: {
    from: 100,
    to: 150
  }
})

// Listen for awareness changes
provider.awareness.on('change', () => {
  const states = provider.awareness.getStates()

  states.forEach((state, clientId) => {
    if (state.user) {
      console.log(`User ${state.user.name} at cursor ${state.cursor}`)
    }
  })
})

// Get specific client state
const clientId = provider.awareness.clientID
const state = provider.awareness.getStates().get(clientId)

Editor Bindings

Yjs provides bindings for popular editors:

Quill (Rich Text)

pnpm add quill y-quill
import Quill from 'quill'
import { QuillBinding } from 'y-quill'

// Create Yjs document
const yDoc = new Y.Doc()
const { provider } = await cloudillo.openYDoc(yDoc, 'doc_123')

const yText = yDoc.getText('content')

// Create Quill editor
const editor = new Quill('#editor', {
  theme: 'snow'
})

// Bind Yjs to Quill
const binding = new QuillBinding(yText, editor, provider.awareness)

// Quill now syncs with Yjs automatically

CodeMirror (Code Editor)

pnpm add codemirror y-codemirror
import { EditorView, basicSetup } from 'codemirror'
import { yCollab } from 'y-codemirror.next'

const yText = yDoc.getText('content')

const editor = new EditorView({
  extensions: [
    basicSetup,
    yCollab(yText, provider.awareness)
  ],
  parent: document.querySelector('#editor')
})

Monaco (VS Code Editor)

pnpm add monaco-editor y-monaco
import * as monaco from 'monaco-editor'
import { MonacoBinding } from 'y-monaco'

const yText = yDoc.getText('content')

const editor = monaco.editor.create(document.getElementById('editor'), {
  value: '',
  language: 'javascript'
})

const binding = new MonacoBinding(
  yText,
  editor.getModel(),
  new Set([editor]),
  provider.awareness
)

ProseMirror (Rich Text)

pnpm add prosemirror-view prosemirror-state y-prosemirror
import { EditorState } from 'prosemirror-state'
import { EditorView } from 'prosemirror-view'
import { ySyncPlugin, yCursorPlugin, yUndoPlugin } from 'y-prosemirror'

const yXml = yDoc.getXmlFragment('prosemirror')

const state = EditorState.create({
  schema,
  plugins: [
    ySyncPlugin(yXml),
    yCursorPlugin(provider.awareness),
    yUndoPlugin()
  ]
})

const view = new EditorView(document.querySelector('#editor'), {
  state
})

Offline Support

Yjs documents work offline and sync when reconnected.

import { IndexeddbPersistence } from 'y-indexeddb'

const yDoc = new Y.Doc()

// Persist to IndexedDB
const indexeddbProvider = new IndexeddbPersistence('my-doc-id', yDoc)

indexeddbProvider.on('synced', () => {
  console.log('Loaded from IndexedDB')
})

// Also connect to server
const { provider } = await cloudillo.openYDoc(yDoc, 'my-doc-id')

// Now works offline with local persistence
// Syncs to server when connection available

Transactions

Group multiple changes into a single transaction:

yDoc.transact(() => {
  const yText = yDoc.getText('content')
  yText.insert(0, 'Hello ')
  yText.insert(6, 'world!')
  yText.format(0, 11, { bold: true })
})

// All changes sync as one update
// Only one observer event fired

Undo/Redo

import { UndoManager } from 'yjs'

const yText = yDoc.getText('content')
const undoManager = new UndoManager(yText)

// Make changes
yText.insert(0, 'Hello')

// Undo
undoManager.undo()

// Redo
undoManager.redo()

// Track who made changes
undoManager.on('stack-item-added', (event) => {
  console.log('Change by:', event.origin)
})

Document Lifecycle

// Create document
const yDoc = new Y.Doc()

// Open collaborative connection
const { provider } = await cloudillo.openYDoc(yDoc, 'doc_123')

// Use document...

// Close connection
provider.destroy()

// Destroy document
yDoc.destroy()

Best Practices

1. Use Subdocs for Large Documents

const yDoc = new Y.Doc()
const yMap = yDoc.getMap('pages')

// Create subdocument for each page
const page1 = new Y.Doc()
yMap.set('page1', page1)

const page1Text = page1.getText('content')
page1Text.insert(0, 'Page 1 content')

2. Batch Operations in Transactions

// ✅ Single transaction
yDoc.transact(() => {
  for (let i = 0; i < 100; i++) {
    yText.insert(i, 'x')
  }
})

// ❌ Many transactions
for (let i = 0; i < 100; i++) {
  yText.insert(i, 'x') // Sends 100 updates!
}

3. Clean Up Observers

// Add observer
const observer = (event) => {
  console.log('Changed:', event)
}
yText.observe(observer)

// Remove observer when done
yText.unobserve(observer)

4. Handle Connection State

provider.on('status', ({ status }) => {
  if (status === 'connected') {
    setConnectionStatus('online')
  } else {
    setConnectionStatus('offline')
  }
})

React Example

Complete collaborative editor in React:

import { useEffect, useState } from 'react'
import { useApi } from '@cloudillo/react'
import * as Y from 'yjs'
import * as cloudillo from '@cloudillo/core'

function CollaborativeEditor({ docId }) {
  const [yDoc, setYDoc] = useState(null)
  const [provider, setProvider] = useState(null)
  const [connected, setConnected] = useState(false)

  useEffect(() => {
    const doc = new Y.Doc()
    setYDoc(doc)

    cloudillo.openYDoc(doc, docId).then(({ provider: p }) => {
      setProvider(p)

      p.on('status', ({ status }) => {
        setConnected(status === 'connected')
      })
    })

    return () => {
      provider?.destroy()
      doc.destroy()
    }
  }, [docId])

  if (!yDoc) return <div>Loading...</div>

  return (
    <div>
      <div className="status">
        {connected ? '🟢 Connected' : '🔴 Disconnected'}
      </div>
      <Editor yDoc={yDoc} provider={provider} />
    </div>
  )
}

See Also

Error Handling

Cloudillo uses structured error codes and standardized error responses for consistent error handling across the platform.

Error Response Format

All API errors return this structure:

{
  "error": {
    "code": "E-AUTH-UNAUTH",
    "message": "Unauthorized access",
    "details": {
      "reason": "Token expired",
      "expiredAt": 1735000000
    }
  },
  "time": 1735000000,
  "reqId": "req_abc123"
}

Fields:

  • error.code - Structured error code (see below)
  • error.message - Human-readable error message
  • error.details - Optional additional context
  • time - Unix timestamp (seconds)
  • reqId - Request ID for tracing

Error Code Format

Error codes follow the pattern: E-MODULE-ERRTYPE

  • E- - Prefix (all error codes start with this)
  • MODULE - Module identifier (AUTH, CORE, SYS, etc.)
  • ERRTYPE - Error type (UNAUTH, NOTFOUND, etc.)

Error Codes

Authentication Errors (AUTH)

Code HTTP Status Description
E-AUTH-UNAUTH 401 Unauthorized - Invalid or missing token
E-AUTH-FORBID 403 Forbidden - Insufficient permissions
E-AUTH-EXPIRED 401 Token expired
E-AUTH-INVALID 400 Invalid credentials
E-AUTH-MISMATCH 400 Password mismatch
E-AUTH-EXISTS 409 User already exists

Core Errors (CORE)

Code HTTP Status Description
E-CORE-NOTFOUND 404 Resource not found
E-CORE-CONFLICT 409 Resource conflict (duplicate)
E-CORE-INVALID 400 Invalid request data
E-CORE-BADREQ 400 Malformed request
E-CORE-LIMIT 429 Rate limit exceeded

System Errors (SYS)

Code HTTP Status Description
E-SYS-UNAVAIL 503 Service unavailable
E-SYS-INTERNAL 500 Internal server error
E-SYS-TIMEOUT 504 Request timeout
E-SYS-STORAGE 507 Storage full

File Errors (FILE)

Code HTTP Status Description
E-FILE-NOTFOUND 404 File not found
E-FILE-TOOLARGE 413 File too large
E-FILE-BADTYPE 415 Unsupported file type
E-FILE-CORRUPT 422 Corrupted file

Action Errors (ACTION)

Code HTTP Status Description
E-ACTION-NOTFOUND 404 Action not found
E-ACTION-INVALID 400 Invalid action data
E-ACTION-DENIED 403 Action not allowed
E-ACTION-EXPIRED 410 Action expired

Federation Errors (FED)

Code HTTP Status Description
E-FED-SIGFAIL 400 Action signature verification failed
E-FED-KEYNOTFOUND 400 Issuer public key not found
E-FED-EXPIRED 400 Action token expired
E-FED-NOTRUST 403 No trust relationship with issuer
E-POW-REQUIRED 428 Proof-of-work required (CONN actions)

Validation Errors (VAL)

Code HTTP Status Description
E-VAL-INVALID 400 Validation error
E-AUTH-NOPERM 403 Permission denied

Network Errors (NET)

Code HTTP Status Description
E-NET-TIMEOUT 408 Network timeout

Database Errors (CORE)

Code HTTP Status Description
E-CORE-DBERR 500 Database error
E-CORE-PARSE 500 Parse error

Identity Provider Errors (IDP)

Code HTTP Status Description
E-IDP-NOTFOUND 404 Identity not found
E-IDP-EXISTS 409 Identity already exists
E-IDP-INVALID 400 Invalid identity format

RTDB/CRDT Errors

Code HTTP Status Description
E-RTDB-NOTFOUND 404 RTDB document not found
E-CRDT-NOTFOUND 404 CRDT document not found
E-CRDT-CONFLICT 409 CRDT merge conflict

Handling Errors with FetchError

The @cloudillo/core library provides FetchError for consistent error handling:

import { FetchError } from '@cloudillo/core'

try {
  const api = cloudillo.createApiClient()
  const profile = await api.profiles.getOwn()
} catch (error) {
  if (error instanceof FetchError) {
    console.error('Error code:', error.code)
    console.error('Message:', error.message)
    console.error('HTTP status:', error.status)
    console.error('Details:', error.details)

    // Handle specific errors
    switch (error.code) {
      case 'E-AUTH-UNAUTH':
        // Redirect to login
        window.location.href = '/login'
        break

      case 'E-AUTH-FORBID':
        // Show permission error
        alert('You do not have permission to access this resource')
        break

      case 'E-CORE-NOTFOUND':
        // Show not found message
        console.log('Resource not found')
        break

      case 'E-CORE-LIMIT':
        // Rate limited
        console.log('Too many requests, please slow down')
        break

      default:
        // Generic error
        console.error('An error occurred:', error.message)
    }
  } else {
    // Non-API error (network, parsing, etc.)
    console.error('Unexpected error:', error)
  }
}

Error Handling Patterns

Pattern 1: Global Error Handler

function createApiWithErrorHandling() {
  const api = cloudillo.createApiClient()

  // Wrap API methods with error handling
  return new Proxy(api, {
    get(target, prop) {
      const original = target[prop]

      if (typeof original === 'function') {
        return async (...args) => {
          try {
            return await original.apply(target, args)
          } catch (error) {
            handleApiError(error)
            throw error
          }
        }
      }

      return original
    }
  })
}

function handleApiError(error) {
  if (error instanceof FetchError) {
    switch (error.code) {
      case 'E-AUTH-UNAUTH':
      case 'E-AUTH-EXPIRED':
        // Redirect to login
        window.location.href = '/login'
        break

      case 'E-CORE-LIMIT':
        // Show rate limit toast
        showToast('Too many requests. Please slow down.')
        break

      default:
        // Log to error tracking service
        logErrorToSentry(error)
    }
  }
}

Pattern 2: React Error Boundary

import { Component } from 'react'
import { FetchError } from '@cloudillo/core'

class ApiErrorBoundary extends Component {
  state = { error: null }

  static getDerivedStateFromError(error) {
    return { error }
  }

  componentDidCatch(error, errorInfo) {
    if (error instanceof FetchError) {
      console.error('API Error:', error.code, error.message)

      // Handle specific errors
      if (error.code === 'E-AUTH-UNAUTH') {
        window.location.href = '/login'
      }
    }
  }

  render() {
    if (this.state.error) {
      return (
        <div className="error">
          <h1>Something went wrong</h1>
          <p>{this.state.error.message}</p>
          <button onClick={() => this.setState({ error: null })}>
            Try again
          </button>
        </div>
      )
    }

    return this.props.children
  }
}

Pattern 3: Retry Logic

async function fetchWithRetry(fn, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await fn()
    } catch (error) {
      if (error instanceof FetchError) {
        // Retry on transient errors
        if (error.code === 'E-SYS-UNAVAIL' ||
            error.code === 'E-SYS-TIMEOUT') {
          if (i < maxRetries - 1) {
            // Exponential backoff
            await new Promise(r => setTimeout(r, 1000 * Math.pow(2, i)))
            continue
          }
        }

        // Don't retry on auth or client errors
        if (error.code.startsWith('E-AUTH-') ||
            error.code.startsWith('E-CORE-')) {
          throw error
        }
      }

      throw error
    }
  }
}

// Usage
const data = await fetchWithRetry(() => api.profiles.getOwn())

Pattern 4: User-Friendly Messages

function getUserFriendlyMessage(error: FetchError): string {
  const messages: Record<string, string> = {
    'E-AUTH-UNAUTH': 'Please log in to continue',
    'E-AUTH-FORBID': 'You don\'t have permission to do that',
    'E-AUTH-EXPIRED': 'Your session has expired. Please log in again',
    'E-CORE-NOTFOUND': 'The item you\'re looking for doesn\'t exist',
    'E-CORE-LIMIT': 'You\'re making too many requests. Please slow down',
    'E-FILE-TOOLARGE': 'This file is too large. Maximum size is 100MB',
    'E-FILE-BADTYPE': 'This file type is not supported',
    'E-SYS-UNAVAIL': 'The service is temporarily unavailable. Please try again later',
  }

  return messages[error.code] || 'An unexpected error occurred. Please try again'
}

// Usage
try {
  await api.files.uploadBlob('default', file.name, file)
} catch (error) {
  if (error instanceof FetchError) {
    alert(getUserFriendlyMessage(error))
  }
}

Pattern 5: Typed Error Handling

type ApiError = {
  code: string
  message: string
  status: number
}

function isAuthError(error: ApiError): boolean {
  return error.code.startsWith('E-AUTH-')
}

function isClientError(error: ApiError): boolean {
  return error.status >= 400 && error.status < 500
}

function isServerError(error: ApiError): boolean {
  return error.status >= 500
}

// Usage
try {
  const data = await api.actions.create(newAction)
} catch (error) {
  if (error instanceof FetchError) {
    if (isAuthError(error)) {
      handleAuthError(error)
    } else if (isServerError(error)) {
      showRetryPrompt()
    }
  }
}

Validation Errors

Client-side validation can prevent many errors:

import type { NewAction } from '@cloudillo/types'

function validateAction(action: NewAction): string[] {
  const errors: string[] = []

  if (!action.type) {
    errors.push('Action type is required')
  }

  if (action.type === 'POST' && !action.content) {
    errors.push('Post content is required')
  }

  if (action.attachments && action.attachments.length > 10) {
    errors.push('Maximum 10 attachments allowed')
  }

  return errors
}

// Usage
const newAction = {
  type: 'POST',
  content: { text: 'Hello!' }
}

const validationErrors = validateAction(newAction)
if (validationErrors.length > 0) {
  alert('Validation errors:\n' + validationErrors.join('\n'))
  return
}

// Proceed with API call
await api.actions.create(newAction)

Logging and Monitoring

function logApiError(error: FetchError, context?: any) {
  const logData = {
    code: error.code,
    message: error.message,
    status: error.status,
    url: error.response?.url,
    context,
    timestamp: new Date().toISOString(),
    userAgent: navigator.userAgent,
    user: cloudillo.idTag
  }

  // Send to logging service
  fetch('/api/logs/error', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(logData)
  })

  // Also log to console in development
  if (process.env.NODE_ENV === 'development') {
    console.error('API Error:', logData)
  }
}

Best Practices

1. Always Handle Errors

// ✅ Good - handle errors
try {
  await api.actions.create(newAction)
} catch (error) {
  handleError(error)
}

// ❌ Bad - unhandled errors crash the app
await api.actions.create(newAction)

2. Provide User Feedback

// ✅ Good - user knows what happened
try {
  await api.files.uploadBlob('default', file.name, file)
  showToast('File uploaded successfully!')
} catch (error) {
  showToast('Upload failed. Please try again.')
}

// ❌ Bad - silent failure
try {
  await api.files.uploadBlob('default', file.name, file)
} catch (error) {
  console.error(error)
}

3. Differentiate Error Types

// ✅ Good - handle different errors appropriately
if (error.code === 'E-AUTH-UNAUTH') {
  redirectToLogin()
} else if (error.code === 'E-CORE-NOTFOUND') {
  show404Page()
} else {
  showGenericError()
}

// ❌ Bad - same handling for all errors
alert('Error!')

4. Log Errors for Debugging

// ✅ Good - logs help debug issues
catch (error) {
  console.error('Failed to create post:', {
    error,
    action: newAction,
    user: cloudillo.idTag
  })
  showError('Failed to create post')
}

// ❌ Bad - no debugging info
catch (error) {
  showError('Failed')
}

See Also

Microfrontend Integration

Cloudillo uses a microfrontend architecture where apps run as sandboxed iframes and communicate with the shell via a typed postMessage protocol.

Architecture overview

Apps are loaded into the Cloudillo shell as iframes with opaque origins (no allow-same-origin), ensuring strong isolation between apps and the shell. All communication flows through a message bus layer provided by @cloudillo/core.

  • Isolation – apps are sandboxed, preventing access to the shell’s DOM, cookies, or service worker keys
  • Technology agnostic – use any framework (React, Vue, vanilla JS)
  • Independent deployment – update apps without redeploying the shell
  • Shared authentication – tokens are managed by the shell and pushed to apps via the message bus

Getting started

Initialization with @cloudillo/core

The getAppBus() singleton provides the main API for apps to communicate with the shell:

import { getAppBus } from '@cloudillo/core'

async function main() {
  const bus = getAppBus()
  const state = await bus.init('my-app')

  // state contains: idTag, tnId, roles, accessToken, access, darkMode, theme, ...
  console.log('User:', state.idTag)
  console.log('Access:', state.access) // 'read' | 'write'

  // Token is also available as bus.accessToken
  // bus.init() automatically calls notifyReady('auth')
}

main().catch(console.error)

AppState fields

The init() call returns an AppState object:

Field Type Description
idTag string? User’s identity tag
tnId number? Tenant ID
roles string[]? User roles
accessToken string? JWT access token for API calls
access 'read' | 'write' Access level for the current resource
darkMode boolean Dark mode preference
theme string UI theme variant (e.g. 'glass')
tokenLifetime number? Token lifetime in seconds
displayName string? Display name (for guest users via share links)
navState string? Navigation state passed from the shell

Lifecycle notifications

Apps signal their readiness to the shell in stages using bus.notifyReady(stage):

Stage When to call Notes
'auth' After authentication completes Called automatically by bus.init()
'synced' After CRDT/data sync completes Call manually when your data is loaded
'ready' When the app is fully interactive Call when UI is ready for user interaction

The shell shows a loading indicator until the app signals 'ready'. You can also report errors with bus.notifyError(code, message).

React integration

useCloudillo hook

The primary hook for React apps. It calls bus.init() internally and provides the app state:

import { useCloudillo, useAuth, useApi } from '@cloudillo/react'

function MyApp() {
  const { token, ownerTag, fileId, idTag, roles, access, displayName } = useCloudillo('my-app')
  const [auth] = useAuth()       // [AuthState | null, setter] tuple
  const { api } = useApi()       // { api: ApiClient | null, authenticated, setIdTag }

  if (!auth) return <div>Loading...</div>

  return <div>Hello, {auth.idTag}</div>
}
Info

useCloudillo extracts ownerTag and fileId from the URL hash (#ownerTag:fileId). The hash is how the shell passes resource context to apps.

useCloudilloEditor hook

For collaborative document apps using CRDT:

import { useCloudilloEditor } from '@cloudillo/react'

function Editor() {
  const { yDoc, provider, synced, error } = useCloudilloEditor('quillo')

  if (error) return <div>Error: {error.code}</div>
  if (!synced) return <div>Syncing...</div>

  // yDoc is a Y.Doc connected to the collaborative backend
  return <MyEditor yDoc={yDoc} />
}

This hook handles the full lifecycle: initialization, WebSocket connection, CRDT persistence, and cleanup on unmount. It automatically calls notifyReady('synced') when the document is synchronized.

Communication protocol

All messages follow the envelope format:

{ cloudillo: true, v: 1, type: 'category:action.verb', ... }

Message categories

Category Direction Purpose
auth:init.req app → shell Request initialization
auth:init.res shell → app Respond with state and token
auth:init.push shell → app Proactively push init (theme/state changes)
auth:token.push shell → app Push refreshed token
auth:token.refresh.req/res both Manual token refresh
app:ready.notify app → shell Signal readiness stage
app:error.notify app → shell Report error to shell
storage:op.req/res both Key-value storage operations
settings:get.req/res both App settings access
media:pick.req/result both Media picker dialog
doc:pick.req/result both Document picker dialog
crdt:* both CRDT cache operations
sensor:compass.* both Device sensor subscriptions
Note

Apps don’t need to handle the protocol directly. The AppMessageBus class (via getAppBus()) provides typed methods for all operations. The protocol details are mainly useful for debugging or building non-JS integrations.

Storage and settings

Storage API

Apps have access to namespaced key-value storage via the message bus:

const bus = getAppBus()

// Store and retrieve data
await bus.storage.set('my-app', 'preferences', { theme: 'dark' })
const prefs = await bus.storage.get<{ theme: string }>('my-app', 'preferences')

// List keys and check quota
const keys = await bus.storage.list('my-app', 'cache:')
const { limit, used } = await bus.storage.quota('my-app')
Method Signature Description
get get<T>(ns, key): Promise<T?> Get a value by key
set set(ns, key, value): Promise<void> Set a value
delete delete(ns, key): Promise<void> Delete a key
list list(ns, prefix?): Promise<string[]> List keys with optional prefix
clear clear(ns): Promise<void> Clear all data in namespace
quota quota(ns): Promise<{limit, used}> Get storage quota info

Settings API

Apps can read and write server-side settings scoped to app.<appName>.*:

Method Signature Description
get get<T>(key): Promise<T?> Get a setting value
set set(key, value): Promise<void> Set a setting value
list list(prefix?): Promise<Array<{key, value}>> List settings

Security

Warning

The shell loads app iframes with sandbox="allow-scripts allow-forms allow-downloads". The allow-same-origin attribute is deliberately excluded to create opaque origins, which prevents apps from accessing the shell’s service worker registration keys or cookies. This is a critical security boundary.

Token handling: Access tokens are held in memory only (inside the AppMessageBus instance). Never store tokens in localStorage or sessionStorage – even with opaque origins, this would create unnecessary persistence of credentials.

Message validation: The SDK validates all incoming messages automatically (protocol version, envelope structure, message type). Apps using getAppBus() do not need to implement manual postMessage validation.

Token refresh: The shell proactively pushes refreshed tokens via auth:token.push messages. Apps can also request a refresh manually with bus.refreshToken().

Debugging

Enable debug logging by passing a config to getAppBus():

const bus = getAppBus({ debug: true })
await bus.init('my-app')
// All message bus traffic will be logged to the console

To inspect an app’s iframe context in DevTools, use the console’s context selector dropdown to switch to the iframe’s execution context.

Example apps

Cloudillo includes several built-in apps that use these patterns:

App Tech Features
Quillo Quill + Yjs Collaborative rich text editor
Prezillo Custom + Yjs Presentation slides and animations
Calcillo Fortune Sheet + Yjs Excel-like spreadsheets
Formillo React + RTDB Form builder and response collection
Taskillo React + RTDB Task management
Notillo React + Yjs Note-taking
Scanillo React Document scanning
Mapillo React Map visualization
Ideallo React + Yjs Collaborative ideation board

See also

CRDT Design Guide

A comprehensive guide to designing collaborative data structures using Yjs and CRDTs for real-time applications.

Overview

Building collaborative applications requires careful consideration of how data structures behave when multiple users edit simultaneously. This guide covers the design patterns, best practices, and common pitfalls when working with Conflict-free Replicated Data Types (CRDTs) in Cloudillo applications.

Early Stage Documentation

The patterns in this guide are based on internal Cloudillo applications (Calcillo, Ideallo, Prezillo, Quillo) which have shown promising results in our testing. However, Cloudillo has not yet achieved wide adoption, so these recommendations should be considered with appropriate caution. We’re sharing our experience to help the community, but real-world usage at scale may reveal patterns that need adjustment.

Who This Guide Is For

This guide is for developers who:

  • Are building collaborative features in Cloudillo applications
  • Need to understand how to structure data for real-time synchronization
  • Want to learn from real-world patterns used in Cloudillo apps (Calcillo, Ideallo, Prezillo, Quillo)

Prerequisites

Before diving in, you should be familiar with:

  • Basic JavaScript/TypeScript
  • The CRDT API for Cloudillo integration
  • General concepts of collaborative editing

Topics

Fundamentals

Core Yjs concepts including shared types, document structure, and how CRDTs work internally.

Design Patterns

Proven patterns for structuring collaborative data: ID-based storage, nested maps, content separation, and more.

Application Types

Specific guidance for different document types: text editors, spreadsheets, canvas/whiteboard apps, and presentations.

Collaboration Features

Implementing multi-user features: transactions, undo/redo, presence awareness, and conflict resolution.

Pitfalls

Common mistakes that cause data corruption, sync issues, or unexpected behavior—and how to avoid them.

Quick Start

If you’re new to CRDT design, start with:

  1. Shared Types - Understand Y.Map, Y.Array, Y.Text
  2. ID-Based Storage - The most important pattern for collaborative apps
  3. Transactions - Properly batching changes

See Also

Subsections of CRDT Design Guide

Fundamentals

Core Yjs concepts you need to understand before designing collaborative data structures.

Overview

Yjs provides a set of shared data types that automatically synchronize across clients and resolve conflicts. Understanding these primitives is essential for building robust collaborative applications.

Topics

Key Concepts

Shared Types Are Not Regular Objects

Yjs shared types look similar to JavaScript objects and arrays, but they behave differently:

// Regular JavaScript - changes are local only
const obj = { name: 'Alice' }
obj.name = 'Bob'  // Only visible locally

// Yjs shared type - changes synchronize
const yMap = yDoc.getMap('user')
yMap.set('name', 'Alice')
yMap.set('name', 'Bob')  // Syncs to all connected clients

Documents Contain All Shared Data

A Y.Doc is the root container for all collaborative data. Different parts of your application access different top-level keys:

const yDoc = new Y.Doc()

// Each getMap/getArray/getText creates a named root-level shared type
const cells = yDoc.getMap('cells')      // Spreadsheet data
const order = yDoc.getArray('rowOrder') // Row ordering
const meta = yDoc.getMap('metadata')    // Document metadata

Changes Must Go Through Yjs APIs

Yjs only tracks changes made through its APIs. Direct mutation of extracted values does not synchronize:

// WRONG - changes not tracked
const obj = yMap.get('config')
obj.theme = 'dark'  // This change is lost!

// CORRECT - use Yjs API
yMap.set('config', { ...yMap.get('config'), theme: 'dark' })

Subsections of Fundamentals

Shared Types

Understanding Yjs shared types: Y.Map, Y.Array, Y.Text, and Y.XmlFragment.

Overview

Yjs provides four primary shared types. Each has specific characteristics for different use cases.

Y.Map

A key-value store similar to JavaScript’s Map. Supports nested shared types for hierarchical structures.

Key behavior: Setting a plain object snapshots it—later mutations don’t sync. For granular updates, nest Y.Map instances.

Best For:

  • Keyed data where order doesn’t matter
  • Configuration objects
  • Entity storage (keyed by ID)
  • Hierarchical structures (nested maps)

Y.Array

An ordered list similar to JavaScript’s Array. Handles concurrent insertions gracefully with position-aware merging.

Key behavior: Reordering items (delete + insert) creates copies, not moves. For reorderable collections, store only IDs in the array.

Best For:

  • Ordered lists where sequence matters
  • ID arrays for ordering (with content in a separate Y.Map)
  • Text editor paragraphs
  • Timeline events
Avoid Complex Objects in Arrays

Don’t store complex objects in Y.Array if you need to reorder them. Store IDs in the array and content in a Y.Map instead. See ID-Based Storage.

Y.Text

A shared string optimized for collaborative text editing. Supports character-level insertions, deletions, and formatting attributes.

Key behavior: Concurrent edits merge at character positions. Formatting uses Quill-style delta operations.

// Rich text formatting
yText.insert(0, 'Bold text', { bold: true })
yText.format(0, 4, { italic: true })  // Apply to range

Editor Bindings: Integrates with Quill, ProseMirror, Monaco, CodeMirror, and TipTap through official bindings.

Best For:

  • Rich text documents
  • Code editors
  • Chat messages
  • Any content needing character-level merging

Y.XmlFragment

An XML-like structure for representing DOM or document trees. Used primarily with ProseMirror for complex rich text with nested elements.

Best For:

  • ProseMirror integration
  • DOM-like document structures
  • Complex rich text with nested elements

Type Selection Guide

Data Type Use Case Conflict Resolution
Y.Map Key-value data, entities Last-writer-wins per key
Y.Array Ordered lists, sequences Position-aware insertion
Y.Text Text content, rich text Character-level merging
Y.XmlFragment DOM structures Element-level operations

See Also

Document Structure

Organizing your Y.Doc for maintainable and efficient collaborative applications.

Overview

A Y.Doc is the root container for all collaborative data. Structure affects performance, maintainability, and how concurrent edits merge.

Structure by Purpose

Organize data into distinct categories:

Document
├── Content Data       (cells, objects, text)
├── Ordering Data      (arrays of IDs)
├── Metadata           (title, createdAt, version)
└── Configuration      (user preferences, settings)

Example: Spreadsheet

Document
├── cells: Map<rowId, Map<colId, CellData>>
├── rowOrder: Array<rowId>
├── colOrder: Array<colId>
├── rowProps: Map<rowId, {height}>
├── colProps: Map<colId, {width}>
├── namedRanges: Map<name, Range>
└── styles: Map<styleId, Style>

Example: Canvas App

Document
├── objects: Map<objectId, DrawingObject>
├── layers: Map<layerId, LayerData>
├── layerOrder: Map<layerId, Array<objectId>>
├── paths: Map<pathId, PathData>
├── textBoxes: Map<textId, Y.Text>
└── metadata: Map<key, value>

Example: Presentation

Document
├── slides: Map<slideId, SlideData>
├── slideOrder: Array<slideId>
├── containers: Map<containerId, Container>
├── masters: Map<masterId, MasterTemplate>
├── styles: Map<styleId, Style>
└── notes: Map<slideId, string>

Design Guidelines

Separate Content from Order

  • Store content in Y.Map by ID
  • Store ordering in Y.Array (IDs only)
  • Never put complex objects in reorderable arrays

Flatten When Possible

  • Deep nesting has overhead
  • Prefer 2-3 levels max
  • Use flat maps with composite keys when appropriate

Use TypeScript Interfaces

  • Define types for your document structure
  • Create helper functions for typed access

Initialize Required Structure

  • Access top-level types on document creation
  • Set defaults in a transaction

Performance Notes

  • Many top-level types is fine—enables granular sync
  • Deeply nested types (4+ levels) add overhead
  • Large arrays (10,000+ items): consider pagination or chunking

See Also

Internals

How Yjs works under the hood—understanding the mechanics behind CRDT synchronization.

The CRDT Model

Simplified Overview

This page provides a practical understanding of Yjs internals. For definitive details, consult the Yjs documentation.

Yjs is primarily operation-based: it stores and transmits operations (inserts, deletes). However, it also supports state encoding for snapshots and initial sync. This combination provides:

  • Efficient sync - Only missing operations are transmitted based on vector clock comparison
  • Full state snapshots - New clients can receive complete state without operation replay
  • Compact updates - Ongoing changes are small binary operation deltas

Items and the Item List

Internally, all data is a linked list of “items”:

[item1] <-> [item2] <-> [item3] <-> [item4]

Each item contains:

  • ID - Unique (clientId, clock) pair
  • Content - The actual data
  • Origin - Item this was inserted after
  • Right Origin - Item this was inserted before

Client IDs and Clocks

Every client has a unique ID and logical clock. Item IDs are (clientId, clock) pairs, ensuring globally unique identifiers without coordination.

Vector Clocks

State vectors track what each client has seen:

{
  clientA: 15,  // Has seen A's operations up to clock 15
  clientB: 8,   // Has seen B's operations up to clock 8
}

When syncing, only missing operations are sent based on vector clock comparison.

Conflict Resolution

Y.Map: Last-writer-wins by logical timestamp. Higher clock wins.

Y.Array: Concurrent insertions at same position are both preserved. Order determined by client ID.

Y.Text: Character-level merging. Concurrent insertions both appear; order by position and client ID.

The Update Format

Changes are encoded as compact binary:

yDoc.on('update', (update: Uint8Array) => {
  // Send over network or store for persistence
})

Y.applyUpdate(yDoc, update)  // Apply received update
Y.mergeUpdates([u1, u2, u3]) // Compact multiple updates

Garbage Collection

Deleted items become tombstones (needed for concurrent operation resolution). Tombstones are eventually garbage collected when all clients have moved past them.

GC Implications

Heavy editing accumulates tombstones until GC. Very long-lived, heavily-edited documents may grow larger than expected.

Subdocuments

For large documents, split into subdocuments for lazy loading:

const mainDoc = new Y.Doc()
const subDoc = new Y.Doc({ guid: 'chapter-1' })
mainDoc.getMap('subdocs').set('chapter1', subDoc)

Performance Characteristics

The following are approximate complexities for typical use cases. Actual performance varies by implementation details, document structure, and operation history:

Operation Approximate Complexity
Map get/set O(1) average
Array push O(1)
Array insert at index O(n)
Text insert O(log n) typical*
Sync (diff) O(changes)

* Text insertion complexity depends on the document’s internal structure and edit history.

Space overhead: For typical documents, expect 2-10x the raw data size due to CRDT metadata and tombstones. Very long-lived, heavily-edited documents may accumulate more overhead. Documents with minimal edits will be closer to the lower bound.

See Also

Design Patterns

Proven patterns for structuring collaborative data that scales and handles concurrent edits gracefully.

Overview

These patterns emerge from real-world collaborative applications. They solve common problems like maintaining order while allowing concurrent edits, organizing complex hierarchies, and keeping data structures efficient.

Topics

Pattern Selection Guide

Your Need Recommended Pattern
Items that can be reordered ID-Based Storage + Ordering Arrays
Many small entities vs complex objects JSON vs Y.Map Tradeoffs
Hierarchical data (trees) ID-Based Storage with parent references
Large documents with many object types Separate Content Maps
Theming or default values Style Inheritance
Simple key-value storage Direct Y.Map (no special pattern needed)

The Most Important Rule

Never store complex objects in arrays that will be reordered.

This single rule prevents the most common and destructive CRDT bugs. When you need to reorder items:

  1. Store the actual content in a Y.Map keyed by ID
  2. Store only the IDs in a Y.Array for ordering
  3. Reorder by moving IDs in the array, not the content

See ID-Based Storage for complete examples.

Subsections of Design Patterns

ID-Based Storage

The most important pattern for collaborative applications: storing content by ID and referencing by ID.

Overview

ID-based storage separates what content exists from where it appears. Store content in a map keyed by unique IDs, and reference those IDs from elsewhere.

This prevents data loss during concurrent reordering and enables reliable cross-references.

The Problem

Storing content directly in a reorderable array:

// WRONG - content in array
const slides = yDoc.getArray('slides')
slides.push([{ id: 'k8d2fn3m', title: 'Intro', content: [...] }])

When two users reorder the same slide concurrently, delete + insert creates copies—resulting in duplicated or lost content.

The Solution

Separate content storage from ordering:

const slideContent = yDoc.getMap('slides')
const slideOrder = yDoc.getArray('slideOrder')

// Store content by ID
slideContent.set('k8d2fn3m', { title: 'Intro', content: [...] })

// Store only ID in ordering array
slideOrder.push(['k8d2fn3m'])

Now reordering only moves IDs—lightweight operations that merge cleanly.

Core Operations

All operations should be wrapped in yDoc.transact() for atomicity:

Operation Steps
Create content.set(id, data) + order.push([id])
Reorder Delete ID from old index, insert at new index
Delete order.delete(index, 1) + content.delete(id)
Read order.toArray().map(id => content.get(id))

Always delete from both order and content to avoid orphaned data.

Benefits

  • Safe Reordering: Moving items only moves IDs, which merge cleanly
  • Stable References: Other parts can reference by ID without breaking
  • Efficient Updates: Content changes don’t affect order, and vice versa
  • Easy Deletion: References become stale IDs that can be filtered
  • Undo Granularity: Content and order changes are separate undo steps

Hierarchical Data (Trees)

ID-based storage extends naturally to trees. Store all nodes flat with parent references:

nodes/
├── k8d2fn3m → { name: 'Documents', parentId: null }
├── m4x9pt2q → { name: 'Work', parentId: 'k8d2fn3m' }
├── j7n3ks8w → { name: 'Report.md', parentId: 'm4x9pt2q' }
└── p2r6vm4c → { name: 'Notes.md', parentId: 'm4x9pt2q' }

Why flat beats nested maps for trees:

  • Moving nodes is a single parentId update, not delete + insert
  • Cross-references (shortcuts, symlinks) work naturally
  • Depth changes don’t require restructuring

For sibling ordering, add a childOrder array per parent or a separate ordering map.

Common Mistakes

Mistake Problem Solution
Using array indices as references Indices shift when items are inserted/deleted Use stable IDs for references
Deleting from order only Orphaned content accumulates in the map Delete from both order and content
Not using transactions Sync may occur between operations Wrap related changes in yDoc.transact()

See Also

JSON vs Y.Map Tradeoffs

Choosing between plain JSON objects and nested Y.Maps is a critical design decision.

The Core Tradeoff

Aspect JSON Objects Nested Y.Maps
Update granularity Whole object replaced Per-field
Concurrent field edits Last-write-wins (data loss!) Merge correctly
Memory overhead Lower Higher (CRDT metadata)
Code complexity Simpler More complex

The Concurrent Edit Problem

This is the critical consideration. With JSON objects, concurrent field edits cause data loss.

// Two users editing same entity
entities.set('player', { name: 'Alice', score: 100 })

// User A updates name:
entities.set('player', { ...entities.get('player'), name: 'Bob' })

// User B updates score (concurrently):
entities.set('player', { ...entities.get('player'), score: 200 })

// Result: ONE UPDATE IS LOST!
// Either { name: 'Bob', score: 100 } or { name: 'Alice', score: 200 }

With Y.Map, both changes merge correctly → { name: 'Bob', score: 200 }

Decision Framework

Use JSON when:

  • Thousands of small entities (memory matters)
  • Entities rarely updated after creation
  • Users won’t edit same entity simultaneously
  • Values are primitives

Use Y.Map when:

  • Concurrent editing of same entity is likely
  • Entities have many fields
  • Frequent partial updates
  • Field-level merging matters

Hybrid: Separate by Volatility

Different parts of entities may have different update patterns:

// Cell values: frequent, small, different users → different cells
const values = yDoc.getMap('values')     // JSON primitives
values.set('A1', 42)

// Styles: rare updates, can replace whole
const styles = yDoc.getMap('styles')     // JSON objects
styles.set('A1', { bold: true, color: '#000' })

// Canvas objects: concurrent x/y/size edits on same object
const objects = yDoc.getMap('objects')   // Nested Y.Maps

Memory Impact

Structure ~Overhead per item
JSON in Y.Map ~50 bytes
Nested Y.Map (5 fields) ~300 bytes

For 10,000 items: ~500KB (JSON) vs ~3MB (Y.Map)

Guidelines Summary

Scenario Recommendation
Large collection of small values JSON
Complex objects, unlikely concurrent edits JSON
Complex objects, likely concurrent edits Y.Map required
Mixed update patterns Separate maps by volatility

See Also

Separate Content Maps

Splitting data by type into separate Y.Maps for better organization and performance.

The Pattern

Instead of one nested structure, use separate maps by type:

// Instead of one big data.objects with mixed types...
const shapes = yDoc.getMap('shapes')
const images = yDoc.getMap('images')
const textBoxes = yDoc.getMap('textBoxes')
const paths = yDoc.getMap('paths')

Benefits

  • Targeted observers: Subscribe only to relevant types (shapes.observe() won’t fire for image changes)
  • Faster lookups: images.toJSON() is faster than filtering a mixed collection by type
  • Type safety: Each map can have its own TypeScript interface (Y.Map<ShapeData>, Y.Map<ImageData>)

Shared Ordering

Objects from different maps can share ordering via layer arrays:

const layerOrder = yDoc.getMap('layerOrder')  // Map<layerId, Array<objectId>>

// Object IDs in layers, regardless of type
layerOrder.get('default').push([shapeId])
layerOrder.get('default').push([imageId])

Getting All Objects

When you need all objects regardless of type, spread the values from each map or check each map when looking up by ID. With consistent ID formats (e.g., type-prefixed IDs like shape-xxx, image-xxx), lookups can go directly to the right map.

When to Separate

Separate when:

  • Different types have different properties
  • You query by type frequently
  • Different UI areas handle different types

Keep together when:

  • Objects are always processed together
  • You have very few objects total

See Also

Ordering with Arrays

Using Y.Array to maintain element order in collaborative applications.

Overview

Y.Array provides ordered sequences that handle concurrent insertions gracefully. Combined with ID-based storage, Y.Array becomes the standard way to represent ordered collections.

The Pattern

Store only IDs in Y.Array; store content in Y.Map. See ID-Based Storage for the complete pattern and core operations.

Concurrent Insert Behavior

When two users insert at the same position simultaneously:

Initial: [A, B, C]

User 1 inserts X after A: [A, X, B, C]
User 2 inserts Y after A: [A, Y, B, C]

Merged: [A, X, Y, B, C] or [A, Y, X, B, C]

The order of X and Y is determined by client IDs (arbitrary but consistent). Both items appear—neither is lost.

Grouped Ordering

For items grouped into categories, use a Map of Arrays:

items/           → Map<itemId, ItemData>
groupOrders/     → Map<groupId, Y.Array<itemId>>
  ├── 'todo'     → ['k8d2fn3m', 'j7n3ks8w']
  ├── 'doing'    → ['m4x9pt2q']
  └── 'done'     → ['p2r6vm4c', 'q5t8wn2x']

Moving an item between groups: delete ID from source array, insert into target array, update item’s groupId field—all in one transaction.

Performance Tips

Large arrays (10,000+ items):

  • Cache order.toArray() instead of calling repeatedly
  • Use UI pagination/virtualization

Batch operations:

// WRONG: multiple syncs
for (const id of idsToRemove) {
  order.delete(order.toArray().indexOf(id), 1)
}

// CORRECT: single transaction, delete from end first
yDoc.transact(() => {
  const indices = idsToRemove
    .map(id => order.toArray().indexOf(id))
    .filter(i => i !== -1)
    .sort((a, b) => b - a)  // Descending
  for (const i of indices) order.delete(i, 1)
})

See Also

Style Inheritance

Implementing prototype chains and style inheritance for themes, defaults, and templates.

Application Pattern

Style inheritance is an application-level pattern built on top of CRDTs, not a CRDT feature itself. It leverages CRDT properties (automatic sync, conflict resolution) while keeping the inheritance logic in application code.

The Pattern

Instead of storing all properties on each element:

  1. Store a reference to a parent style/template
  2. Store only the overridden properties
const styles = yDoc.getMap('styles')
styles.set('heading', {
  fontSize: 24,
  fontWeight: 'bold',
  color: '#333333'
})

const elements = yDoc.getMap('elements')
elements.set('title', {
  text: 'Welcome',
  styleId: 'heading',
  overrides: { color: '#0066cc' }  // Override just this
})

Resolving Styles

function resolveStyle(element: ElementData): ResolvedStyle {
  const baseStyle = styles.get(element.styleId) || {}
  return { ...baseStyle, ...element.overrides }
}

// { fontSize: 24, fontWeight: 'bold', color: '#0066cc' }

Multi-Level Inheritance

Chain multiple levels for complex theming:

theme/colors     → { primary: '#0066cc', text: '#333' }
      ↓
templates/card   → { padding: 16, titleColor: 'theme.colors.primary' }
      ↓
elements/card-1  → { templateId: 'card', overrides: { padding: 24 } }

Resolution cascades: element overrides → template defaults → theme values.

Named Style Classes

Like CSS classes, elements can reference multiple styles by name. Store class definitions in a map (styleClasses), and let elements specify an array of class names. Resolution merges classes in order, then applies overrides.

Cascading Updates

When a base style changes, all dependent elements automatically get new values:

theme.observeDeep(() => {
  renderAll()  // Elements resolve to new values
})

See Also

Application Types

Specific design guidance for different types of collaborative documents.

Overview

Different application types have different data structure needs. A text editor has very different requirements from a spreadsheet or drawing canvas. This section provides tailored guidance for each category.

Application Categories

  • Text Editors - Rich text with Y.Text and editor bindings
  • Spreadsheets - 2D grids with cells, rows, and columns (Calcillo patterns)
  • Canvas Apps - Drawing and whiteboard applications (Ideallo patterns)
  • Presentations - Slide-based documents with containers and views (Prezillo patterns)

Choosing the Right Approach

Application Type Primary Data Structure Key Pattern
Text editor Y.Text Editor binding (Quill, ProseMirror)
Spreadsheet Y.Map of cells ID-based cells, ordered rows/columns
Canvas/Whiteboard Y.Map of objects Separate content maps by type
Presentations Y.Map of containers Style inheritance, templates

Common Themes

Despite their differences, all collaborative applications share these needs:

  1. Stable references - Use IDs, not indices, to reference other elements
  2. Separate ordering from content - Store order in arrays, content in maps
  3. Batch changes - Use transactions to group related modifications
  4. Local state separation - Keep UI state (selection, scroll) out of the CRDT

Each application type page shows how to apply these principles to that specific domain.

Subsections of Application Types

Text Editors

Designing collaborative rich text editors with Y.Text.

Document Structure

Document
├── content: Y.Text             (main document)
├── metadata: Map<key, value>   (title, created, etc.)
└── comments: Map<commentId, Comment>

Use Y.Text for the main content. Editor bindings (y-quill, y-prosemirror, y-tiptap) handle synchronization automatically.

Comments with Relative Positions

Comments reference text ranges that must survive edits. Use Y.RelativePosition:

interface Comment {
  id: string
  text: string
  startPosition: Y.RelativePosition  // Survives text changes
  endPosition: Y.RelativePosition
}

Convert between absolute and relative positions when creating/reading comments. Relative positions stay valid even when text is inserted or deleted around them.

Common Mistakes

Storing HTML as plain text:

// WRONG - loses semantic structure
yText.insert(0, '<p><strong>Hello</strong></p>')

// CORRECT - use formatting attributes
yText.insert(0, 'Hello', { bold: true })

Absolute positions for persistent references:

// WRONG: position 42 becomes invalid when text changes
const savedPosition = 42

// CORRECT: relative position adapts to edits
const relPos = Y.createRelativePositionFromTypeIndex(yText, 42)

See Also

Spreadsheets

Designing collaborative spreadsheets with 2D grids, cell addressing, and formula support.

Document Structure

There are two common approaches for cell storage, each with tradeoffs:

Approach 1: Nested Maps

Document
├── cells: Map<rowId, Map<colId, CellData>>
├── rowOrder: Array<rowId>
├── colOrder: Array<colId>
...

Pros: Efficient row operations (iterate all cells in a row), natural grouping. Cons: More complex access pattern, nested map setup for each row.

Approach 2: Flat Map with Composite Keys

Document
├── cells: Map<"rowId:colId", CellData>
├── rowOrder: Array<rowId>
├── colOrder: Array<colId>
...

Pros: Simpler access (cells.get(`${rowId}:${colId}`)), flat structure. Cons: Row iteration requires filtering all keys, slightly more parsing overhead.

Common Structure (Both Approaches)

Document
├── cells: (see above)
├── rowOrder: Array<rowId>
├── colOrder: Array<colId>
├── rowProps: Map<rowId, {height}>
├── colProps: Map<colId, {width}>
├── namedRanges: Map<name, Range>
└── styles: Map<styleId, Style>

Choose based on your access patterns: nested if you frequently operate on entire rows, flat if you primarily access individual cells.

Cell Addressing

Cells are addressed via row/column IDs, not traditional A1 notation. Convert for display:

// Internal: ID-based
const cellRef = { rowId: 'r8k2mf9n', colId: 'c3nd8k2m' }

// Display: convert to A1 when showing to user
function idsToA1(rowId: string, colId: string): string {
  const colIndex = colOrder.toArray().indexOf(colId)
  const rowIndex = rowOrder.toArray().indexOf(rowId)
  return indexToLetter(colIndex) + (rowIndex + 1)  // "B3"
}

Formula References

Use IDs, not indices. When rows/columns are inserted or deleted, A1-style references break. Store references as cell IDs:

interface ParsedFormula {
  expression: string
  references: Array<{
    type: 'cell' | 'range'
    rowId: string
    colId: string
    endRowId?: string  // For ranges
    endColId?: string
  }>
}

// Example: =SUM(B1:B10) stored as
{
  expression: 'SUM(range1)',
  references: [{
    type: 'range',
    rowId: 'r8k2mf9n', colId: 'c3nd8k2m',
    endRowId: 'r9m3kf2n', endColId: 'c3nd8k2m'
  }]
}

Dependency Tracking

Track which cells depend on which for efficient recalculation:

const dependencies = new Map<string, Set<string>>()

// On cell change, only recalculate affected cells
function onCellChange(cellKey: string) {
  const dependents = dependencies.get(cellKey)
  if (dependents) {
    for (const dep of dependents) {
      recalculateCell(dep)
    }
  }
}
Circular References

Implement cycle detection when evaluating formulas. Circular references should show an error rather than causing infinite loops.

Collaborative Features

Selection awareness: Share current cell selection via awareness (not CRDT):

awareness.setLocalStateField('selection', {
  user: { name: 'Alice', color: '#f783ac' },
  cell: { rowId: 'r8k2mf9n', colId: 'c3nd8k2m' }
})

Virtual rendering: For large sheets, only render visible cells based on viewport.

Common Mistakes

Using indices in formulas:

// WRONG: indices shift
formula: '=A1 * B2'

// CORRECT: use stable IDs
formula: { refs: ['r3k9mf8n:c8m2pt3q', 'r3k9mf8n:c2n7ks4w'], expr: '*' }

Recalculating everything:

// WRONG: O(n) on every change
cells.observe(() => recalculateAllCells())

// CORRECT: only affected cells
cells.observe(event => {
  event.changes.keys.forEach((_, key) => {
    recalculateDependents(key)
  })
})

See Also

Canvas Apps

Designing collaborative drawing and whiteboard applications.

Document Structure

Document
├── objects: Map<objectId, DrawingObject>
├── layers: Map<layerId, LayerData>
├── layerOrder: Map<layerId, Array<objectId>>
├── paths: Map<pathId, PathData>        (large path data)
├── textContent: Map<textId, Y.Text>    (rich text)
└── metadata: Map<key, value>

Store large data (path points, rich text) separately from object metadata—reference by ID.

Object Model

interface BaseObject {
  id: string
  type: 'rect' | 'ellipse' | 'path' | 'text' | 'image' | 'group'
  layerId: string
  x: number; y: number
  width: number; height: number
  rotation: number
  opacity: number
  visible: boolean
  locked: boolean
}

// Large data stored separately
interface PathObject extends BaseObject {
  type: 'path'
  pathDataId: string  // Reference to paths map
}

interface TextObject extends BaseObject {
  type: 'text'
  textContentId: string  // Reference to Y.Text
}

Z-Order (Layering)

Each layer has an ordered array of object IDs:

function bringToFront(objectId: string) {
  const obj = objects.get(objectId)
  const order = layerOrder.get(obj.layerId) as Y.Array<string>
  const index = order.toArray().indexOf(objectId)

  if (index !== -1 && index < order.length - 1) {
    yDoc.transact(() => {
      order.delete(index, 1)
      order.push([objectId])
    })
  }
}

Freehand Drawing

For performance, collect points locally, then commit on stroke end:

let currentPoints: Point[] = []

function onPointerMove(x: number, y: number) {
  currentPoints.push({ x, y })
  renderLocalPreview()  // Don't sync yet
}

function onPointerUp() {
  const simplified = simplifyPath(currentPoints)  // Reduce points
  const pathId = nanoid(8)
  const objectId = nanoid(8)

  yDoc.transact(() => {
    paths.set(pathId, { points: simplified })
    objects.set(objectId, {
      type: 'path',
      pathDataId: pathId,
      // ... other properties
    })
    layerOrder.get('default').push([objectId])
  })

  currentPoints = []
}

Collaborative Cursors

Throttle cursor updates to avoid flooding:

const updateCursor = throttle((x: number, y: number) => {
  awareness.setLocalStateField('cursor', {
    x, y,
    user: { name: 'Alice', color: '#f783ac' }
  })
}, 50)  // Max 20 updates/second

Common Mistakes

Storing render state in CRDT:

// WRONG: UI state in CRDT
objects.set(id, { ...obj, isSelected: true, isDragging: true })

// CORRECT: keep render state local
const localState = new Map<string, { isSelected: boolean }>()

Large path data in object:

// WRONG: thousands of points in object
objects.set(id, { ...obj, points: hugePointArray })

// CORRECT: store separately
paths.set(pathId, { points: hugePointArray })
objects.set(id, { ...obj, pathDataId: pathId })

See Also

Presentations

Designing collaborative presentation software with slides, templates, and views.

Looking for Prezillo’s exact format?

This page covers generic CRDT design patterns for presentation apps. For Prezillo’s complete type definitions, field tables, and data model, see the Prezillo Format Specification.

Document Structure

Document
├── slides: Map<slideId, SlideData>
├── slideOrder: Array<slideId>
├── containers: Map<containerId, Container>
├── masters: Map<masterId, MasterTemplate>
├── styles: Map<styleId, Style>
└── notes: Map<slideId, string | Y.Text>

Slides contain references to containers (text boxes, shapes, images). Masters define reusable layouts and styles.

Slide/Container Relationship

interface Slide {
  id: string
  masterId: string
  containerIds: string[]
  background?: Background
}

interface Container {
  id: string
  slideId: string
  type: 'text' | 'image' | 'shape'
  x: number; y: number
  width: number; height: number
  contentId?: string  // For text: Y.Text ID; for image: blob ID
  styleId?: string
  zIndex: number
}

When duplicating a slide, duplicate its containers too:

function duplicateSlide(slideId: string): string {
  const slide = slides.get(slideId)
  const newId = nanoid(8)
  const newContainerIds: string[] = []

  yDoc.transact(() => {
    for (const cid of slide.containerIds) {
      const container = containers.get(cid)
      const newCid = nanoid(8)
      containers.set(newCid, { ...container, id: newCid, slideId: newId })
      newContainerIds.push(newCid)
    }

    slides.set(newId, {
      ...slide, id: newId,
      containerIds: newContainerIds
    })

    const index = slideOrder.toArray().indexOf(slideId)
    slideOrder.insert(index + 1, [newId])
  })

  return newId
}

Style Inheritance

Resolve styles by cascading: master → slide → container:

function resolveStyle(container: Container): ResolvedStyle {
  const slide = slides.get(container.slideId)
  const master = masters.get(slide.masterId)

  let style = {}

  // 1. Master base style
  if (master?.styles?.[container.styleId]) {
    style = { ...master.styles[container.styleId] }
  }

  // 2. Slide overrides
  if (slide?.styleOverrides?.[container.styleId]) {
    style = { ...style, ...slide.styleOverrides[container.styleId] }
  }

  // 3. Container overrides
  if (container.styleOverrides) {
    style = { ...style, ...container.styleOverrides }
  }

  return style
}

Presentation Mode

Presentation state is local (not in CRDT), but can be shared via awareness:

// Broadcast current slide for "follow presenter" mode
awareness.setLocalStateField('presenting', {
  slideIndex: currentIndex,
  user: getCurrentUser()
})

// Follow mode: sync to presenter's slide
awareness.on('change', () => {
  if (followingClientId) {
    const state = awareness.getStates().get(followingClientId)
    if (state?.presenting) {
      goToSlide(state.presenting.slideIndex)
    }
  }
})

Common Mistakes

Slide content directly in slideOrder:

// WRONG: content in array
slideOrder.push([{ title: 'Intro', containers: [...] }])

// CORRECT: content separate
slides.set(id, { title: 'Intro', containerIds: [...] })
slideOrder.push([id])

Not cleaning up containers on slide delete:

function deleteSlide(slideId: string) {
  const slide = slides.get(slideId)
  yDoc.transact(() => {
    // Delete containers first
    for (const cid of slide.containerIds) {
      containers.delete(cid)
    }
    // Then slide
    const index = slideOrder.toArray().indexOf(slideId)
    if (index !== -1) slideOrder.delete(index, 1)
    slides.delete(slideId)
  })
}

See Also

Collaboration Features

Implementing multi-user collaboration features: transactions, undo/redo, presence, and conflict handling.

Overview

Beyond basic data synchronization, collaborative applications need features that make the multi-user experience smooth and intuitive. This section covers the key collaboration patterns.

Topics

Why These Features Matter

Transactions

Without transactions, each change triggers a separate sync and observer event. This causes:

  • Performance issues from excessive network traffic
  • Visual flickering as partial changes render
  • Undo capturing individual operations instead of logical units

Undo/Redo

Users expect undo to reverse their own changes, not their collaborators’. Proper undo implementation requires:

  • Per-user tracking with trackedOrigins
  • Integration with transactions for atomic operations
  • Cursor restoration for text editing

Awareness

Seeing where other users are working prevents conflicts and enables coordination:

  • Cursor positions and selections
  • User names and colors
  • Typing indicators and active states

Conflict Resolution

Understanding how CRDTs resolve conflicts helps you design structures that merge intuitively:

  • Last-writer-wins for simple values
  • Character-level merging for text
  • Concurrent array insertions

Subsections of Collaboration Features

Transactions

Batching changes with yDoc.transact() for atomic operations and better performance.

Why Transactions

Transactions ensure that:

  • All changes sync together (not partially)
  • Observers fire once (not for each operation)
  • Undo captures the entire transaction
// Without transaction: 2 syncs, 2 observer events
items.set('k8d2fn3m', { name: 'Task 1' })
order.push(['k8d2fn3m'])

// With transaction: 1 sync, 1 observer event
yDoc.transact(() => {
  items.set('k8d2fn3m', { name: 'Task 1' })
  order.push(['k8d2fn3m'])
})

Atomic Sync

Without transactions, remote clients might see partial state:

// WRONG: order might sync before content
order.push(['k8d2fn3m'])
items.set('k8d2fn3m', data)

// CORRECT: atomic
yDoc.transact(() => {
  items.set('k8d2fn3m', data)
  order.push(['k8d2fn3m'])
})

Transaction Origins

The second argument identifies the change source:

yDoc.transact(() => {
  content.set('title', 'New Title')
}, 'user-action')

With UndoManager

Filter which changes to track:

const undoManager = new UndoManager([content], {
  trackedOrigins: new Set(['user-action'])
})

// Tracked for undo
yDoc.transact(() => content.set('title', 'New'), 'user-action')

// NOT tracked (remote sync, system updates)
yDoc.transact(() => metadata.set('modified', Date.now()), 'system')

In Observers

items.observe((event, transaction) => {
  if (transaction.origin === 'import') return  // Skip re-render
  renderItems()
})

Nested Transactions

Nested transactions merge into the outermost:

function addItem(data) {
  yDoc.transact(() => {
    items.set(data.id, data)
    order.push([data.id])
  })
}

function addMultiple(dataArray) {
  yDoc.transact(() => {
    for (const data of dataArray) {
      addItem(data)  // Inner transact merges into outer
    }
  })
  // Single sync, single observer event
}

Anti-Patterns

Async inside transactions:

// WRONG: async breaks transaction
yDoc.transact(async () => {
  items.set('a', { ... })
  await saveToServer()  // Transaction already ended!
  items.set('b', { ... })  // NOT in same transaction
})

// CORRECT
yDoc.transact(() => {
  items.set('a', { ... })
  items.set('b', { ... })
})
await saveToServer()

Over-large transactions:

// For huge imports, chunk to avoid blocking UI
async function importLarge(data: any[]) {
  const CHUNK = 1000
  for (let i = 0; i < data.length; i += CHUNK) {
    yDoc.transact(() => {
      data.slice(i, i + CHUNK).forEach(item => items.set(item.id, item))
    }, 'import')
    await new Promise(r => setTimeout(r, 0))  // Yield to UI
  }
}

See Also

  • Undo/Redo - How transactions integrate with UndoManager
  • API Usage - Transaction-related mistakes

Undo/Redo

Implementing per-user undo stacks with Yjs UndoManager.

Overview

In collaborative apps, users expect undo to reverse their own changes, not collaborators’. Yjs UndoManager provides this through “tracked origins.”

Basic Setup

import { UndoManager } from 'yjs'

const undoManager = new UndoManager([content])

undoManager.undo()
undoManager.redo()
undoManager.undoStack.length > 0  // canUndo
undoManager.redoStack.length > 0  // canRedo

Tracked Origins

Without tracked origins, undo captures remote changes too:

// WRONG: captures everything
const undoManager = new UndoManager([content])

// CORRECT: only capture local changes
const undoManager = new UndoManager([content], {
  trackedOrigins: new Set(['user-action'])
})

// Mark local changes
yDoc.transact(() => {
  content.set('title', 'New Title')
}, 'user-action')  // Tracked

// Remote changes have no origin — not tracked

With Editor Bindings

const binding = new QuillBinding(yText, quill, awareness)
const undoManager = new UndoManager(yText, {
  trackedOrigins: new Set([binding])
})

Scoping to Shared Types

// Only track changes to cells, rowOrder, colOrder
const undoManager = new UndoManager([cells, rowOrder, colOrder], {
  trackedOrigins: new Set(['user-action'])
})

// Changes to other types (metadata) not tracked

Capturing Metadata

Restore cursor position after undo:

const undoManager = new UndoManager([content], {
  trackedOrigins: new Set(['user-action']),
  captureTransaction: (transaction) => ({
    cursorPosition: getCursorPosition()
  })
})

undoManager.on('stack-item-popped', (event) => {
  if (event.stackItem.meta.cursorPosition) {
    setCursorPosition(event.stackItem.meta.cursorPosition)
  }
})

Transaction Grouping

Each transaction becomes one undo item:

yDoc.transact(() => {
  items.set('a', data1)
  items.set('b', data2)
  items.set('c', data3)
}, 'user-action')

undoManager.undo()  // Reverts all three at once

For typing, use captureTimeout to group rapid changes:

const undoManager = new UndoManager([content], {
  trackedOrigins: new Set([binding]),
  captureTimeout: 500  // Group changes within 500ms
})

Clear History

undoManager.clear()  // Clear undo/redo stacks
undoManager.stopCapturing()  // End current group, start new one

Stack Events

undoManager.on('stack-item-added', (event) => {
  updateUndoButtons()
})

undoManager.on('stack-item-popped', (event) => {
  // Restore metadata
  updateUndoButtons()
})

Common Mistakes

Forgetting trackedOrigins:

// WRONG
const undoManager = new UndoManager([content])

// CORRECT
const undoManager = new UndoManager([content], {
  trackedOrigins: new Set(['user-action'])
})

Inconsistent origins:

// WRONG: one tracked, one not
content.set('title', 'New')  // Not tracked
yDoc.transact(() => metadata.set('time', now()), 'user-action')  // Tracked

// CORRECT: consistent
yDoc.transact(() => {
  content.set('title', 'New')
  metadata.set('time', now())
}, 'user-action')

See Also

Awareness

Implementing presence, cursors, and real-time user status with the Yjs awareness protocol.

Overview

Awareness provides ephemeral state synchronization—data that syncs in real-time but doesn’t persist: cursor positions, user presence, selections, typing indicators.

Basic Setup

const awareness = provider.awareness

awareness.setLocalStateField('user', {
  name: 'Alice',
  color: '#f783ac'
})

Setting Local State

// Set single field (preserves others)
awareness.setLocalStateField('cursor', { x: 100, y: 200 })

// Replace entire state
awareness.setLocalState({
  user: { name: 'Alice', color: '#f783ac' },
  cursor: { x: 100, y: 200 }
})

// Clear state (on disconnect)
awareness.setLocalState(null)

Observing Other Users

awareness.on('change', ({ added, updated, removed }) => {
  console.log('Joined:', added)
  console.log('Updated:', updated)
  console.log('Left:', removed)
  renderPresence()
})

function renderPresence() {
  awareness.getStates().forEach((state, clientId) => {
    if (clientId === yDoc.clientID) return  // Skip self
    renderCursor(state.cursor, state.user, clientId)
  })
}

Cursor Synchronization

// Throttle updates to avoid flooding
const updateCursor = throttle((x: number, y: number) => {
  awareness.setLocalStateField('cursor', { x, y })
}, 50)

function onMouseMove(e: MouseEvent) {
  updateCursor(e.clientX, e.clientY)
}

function onMouseLeave() {
  awareness.setLocalStateField('cursor', null)
}

Text Editor Cursors

Use relative positions so cursors survive text edits:

function updateTextCursor(selection) {
  awareness.setLocalStateField('cursor', {
    anchor: Y.createRelativePositionFromTypeIndex(yText, selection.anchor),
    head: Y.createRelativePositionFromTypeIndex(yText, selection.head),
    user: awareness.getLocalState()?.user
  })
}

Activity Status

let idleTimeout: NodeJS.Timeout

function setActive() {
  awareness.setLocalStateField('status', 'active')
  clearTimeout(idleTimeout)
  idleTimeout = setTimeout(() => {
    awareness.setLocalStateField('status', 'idle')
  }, 60000)
}

document.addEventListener('mousemove', setActive)
document.addEventListener('visibilitychange', () => {
  awareness.setLocalStateField('status', document.hidden ? 'away' : 'active')
})

Typing Indicator

let typingTimeout: NodeJS.Timeout

function onTextInput() {
  awareness.setLocalStateField('activity', { type: 'typing' })
  clearTimeout(typingTimeout)
  typingTimeout = setTimeout(() => {
    awareness.setLocalStateField('activity', null)
  }, 2000)
}

Cleanup

window.addEventListener('beforeunload', () => {
  awareness.setLocalState(null)
})

Performance Tips

  • Throttle updates: Max 10-20 cursor updates/second
  • Minimal state: Don’t put large objects in awareness
  • Conditional updates: Only send when position changes significantly:
if (Math.abs(e.clientX - lastX) > 5 || Math.abs(e.clientY - lastY) > 5) {
  awareness.setLocalStateField('cursor', { x: e.clientX, y: e.clientY })
}

Common Mistakes

Forgetting to filter self:

// WRONG: renders own cursor
awareness.getStates().forEach(state => renderCursor(state))

// CORRECT
awareness.getStates().forEach((state, clientId) => {
  if (clientId !== yDoc.clientID) renderCursor(state)
})

Not cleaning up removed users:

awareness.on('change', ({ removed }) => {
  for (const clientId of removed) {
    removeCursor(clientId)
  }
})

See Also

Conflict Resolution

Understanding how Yjs CRDTs automatically merge concurrent changes.

Core Principle

CRDTs guarantee eventual consistency: all clients receiving the same operations converge to identical state, regardless of operation order.

Y.Map: Last-Writer-Wins

When multiple clients set the same key, the highest logical timestamp wins. When timestamps are equal, the client ID breaks ties (arbitrary but consistent ordering):

Alice: map.set('color', 'blue')   // timestamp: 100
Bob:   map.set('color', 'green')  // timestamp: 101

Result: 'green' (Bob's timestamp higher)

// When timestamps match:
Alice: map.set('color', 'blue')   // timestamp: 100, clientId: 'abc'
Bob:   map.set('color', 'green')  // timestamp: 100, clientId: 'xyz'

Result: determined by client ID comparison (consistent across all peers)

Design tip: To preserve both values, use unique keys:

map.set(`comment-${odieId}`, 'X')  // Both preserved
map.set(`comment-${bobId}`, 'Y')

Y.Array: Position-Aware Merge

Concurrent insertions at the same position are both preserved:

Initial: [A, B, C]
Alice inserts X after A
Bob inserts Y after A

Result: [A, X, Y, B, C] or [A, Y, X, B, C]

Order of X/Y is deterministic (by client ID) but arbitrary. Design UIs that tolerate this.

Y.Text: Character-Level Merge

Initial: "Hello World"
Alice: "Hello Beautiful World"
Bob: "Hello Amazing World"

Result: "Hello Beautiful Amazing World" (or reversed)

Different formatting attributes merge (bold + italic). Same attribute uses LWW (both set color → one wins).

Designing for Good Merges

Use ID-based references - Indices shift; IDs are stable.

Separate order from content - Content edits and reordering merge independently.

Avoid computed data - Don’t store totals; compute when needed.

Use granular keys - user.name, user.email instead of one user object.

Accept non-determinism - Concurrent inserts at same position have arbitrary order.

When Automatic Merge Isn’t Enough

For critical data, detect conflicts and show resolution UI:

yMap.observe((event, transaction) => {
  if (!transaction.local && hasConflict(event)) {
    showConflictDialog(localValue, remoteValue)
  }
})

Example: Counters and votes. Rather than storing a single number (where concurrent increments get lost to LWW), track each user’s contribution separately:

// WRONG: concurrent increments overwrite each other
counter.set('votes', counter.get('votes') + 1)

// CORRECT: track per-user contributions, sum at read time
const userVotes = yDoc.getMap('votes')
userVotes.set(userId, (userVotes.get(userId) || 0) + 1)

function getTotalVotes(): number {
  let total = 0
  userVotes.forEach(count => total += count)
  return total
}

This pattern preserves all concurrent operations by giving each user their own key.

See Also

Pitfalls

Common mistakes that cause data corruption, sync issues, or unexpected behavior—and how to avoid them.

Overview

CRDT-based applications can fail in subtle ways that are hard to debug. Many issues only appear when multiple users edit simultaneously, making them difficult to reproduce in development. This section documents the most common pitfalls organized by what you’re doing when you encounter them.

Topics

The Cost of CRDT Bugs

CRDT bugs are particularly problematic because:

  1. Data corruption spreads - Once corrupted data syncs to other clients, it’s everywhere
  2. Hard to reproduce - Issues may only occur with specific timing of concurrent edits
  3. Silent failures - Changes may be lost without any error messages
  4. Difficult to fix - Corrupted documents may need manual repair or data loss

Quick Reference

What You’re Doing Common Issues
Designing schema Complex objects in arrays, index references, binary data in CRDT
Calling Yjs methods Mutating after set, wrong getMap/new confusion, missing transactions
Handling events Not checking local flag, flooding awareness, assuming sync order

Quick Checklist

Before deploying a collaborative feature, verify:

  • Complex content in Y.Map with IDs, order in Y.Array
  • All references use IDs, not indices
  • Never mutating objects after set()
  • Using yDoc.getMap() not new Y.Map()
  • Changes batched in transactions
  • Undo uses trackedOrigins to isolate users
  • Observers check transaction.local when needed
  • Awareness updates are debounced
  • Large binary data uses blob storage, not CRDT

Most Common Issues

Symptom Likely Cause See
Data disappears after reorder Objects in array Data Modeling
Changes don’t sync Mutating after insertion API Usage
Undo affects other users Missing trackedOrigins API Usage
Slow sync, high memory Large data in CRDT Data Modeling
Duplicate events Not checking local flag Events & Sync

Subsections of Pitfalls

Data Modeling

Schema decisions that cause data corruption or sync issues.

Complex Objects in Reorderable Arrays

// WRONG
const slides = yDoc.getArray('slides')
slides.push([{ id: 'k8d2fn3m', title: 'Intro', content: [...] }])

Y.Array delete + insert creates copies, not moves. Concurrent reordering causes duplicates or data loss.

Fix: Store content in Y.Map, only IDs in Y.Array. See ID-Based Storage.

Index-Based References

// WRONG
cells.set('A1', { formula: '=B1 * C1' })  // References shift when columns inserted

Array indices change on insert/delete. References break silently.

Fix: Use stable IDs:

cells.set('r8k2mf9n:c3nd8k2m', { formula: { refs: ['r8k2mf9n:c9m2pt3q', 'r8k2mf9n:c2n7ks4w'], expr: '*' } })

Large Binary Data in CRDT

// WRONG
images.set('hero', { data: base64EncodedImage })  // 500KB+ in CRDT

CRDT metadata adds 2-10x overhead. History accumulates. Sync becomes slow.

Fix: Use blob storage, store only reference:

images.set('hero', { blobId: 'b8k2mf9n', width: 1920, height: 1080 })

Checklist

  • Complex content in Y.Map with IDs, order in Y.Array
  • All references use stable IDs, not indices
  • Binary data external, only references in CRDT

See Also

API Usage

Mistakes when calling Yjs methods.

Mutating Objects After Insertion

// WRONG
const config = { theme: 'light' }
yMap.set('config', config)
config.theme = 'dark'  // NOT synced!

Yjs snapshots on set(). Later mutations don’t propagate.

Fix: Create new objects, or use nested Y.Map for granular updates:

yMap.set('config', { ...yMap.get('config'), theme: 'dark' })

Moving Shared Types Between Parents

// WRONG
const items = folder1.get('items')  // Y.Array
folder2.set('items', items)  // Error!

Shared types can only have one parent.

Fix: Copy data to new shared type, delete old.

Creating New Instances Instead of Accessing

// WRONG
function addItem(item) {
  const items = new Y.Map()  // Disconnected!
  items.set(item.id, item)
}

new Y.Map() is unattached. Use yDoc.getMap().

Not Using Transactions

// WRONG: 2N syncs, 2N observer events
for (const item of items) {
  content.set(item.id, item)
  order.push([item.id])
}

Fix: Wrap in transact():

yDoc.transact(() => {
  for (const item of items) {
    content.set(item.id, item)
    order.push([item.id])
  }
})

Forgetting trackedOrigins

// WRONG: captures remote changes too
const undoManager = new UndoManager([content])

Without trackedOrigins, undo captures collaborators’ changes.

Fix:

const undoManager = new UndoManager([content], {
  trackedOrigins: new Set(['user-action'])
})
yDoc.transact(() => { ... }, 'user-action')

Checklist

  • Never mutate after set()
  • Don’t move shared types between parents
  • Use yDoc.getMap(), not new Y.Map()
  • Wrap related changes in transact()
  • Configure UndoManager with trackedOrigins

See Also

Events & Sync

Mistakes when handling events and the distributed nature of CRDTs.

Ignoring Local vs Remote

// WRONG: runs twice (local + sync confirmation)
content.observe(event => {
  saveToLocalStorage(content.toJSON())
  sendAnalytics('changed')
})

Observers fire for both local and remote changes.

Fix: Check transaction.local:

content.observe((event, transaction) => {
  updateUI()  // Always
  if (transaction.local) {
    saveToLocalStorage(content.toJSON())  // Only local
  }
})

Flooding Awareness Updates

// WRONG: 60+ messages/second
document.addEventListener('mousemove', e => {
  awareness.setLocalStateField('cursor', { x: e.clientX, y: e.clientY })
})

Floods network, overwhelms clients.

Fix: Throttle to 10-20 updates/second:

const updateCursor = throttle((x, y) => {
  awareness.setLocalStateField('cursor', { x, y })
}, 50)

Expecting Sync Order

// WRONG: assumes remote sees operations in order
yDoc.transact(() => items.set('config', { init: false }))
yDoc.transact(() => items.set('config', { init: true }))

CRDT sync order isn’t guaranteed across clients.

Fix: Design for eventual consistency. Use timestamps if order matters.

Checklist

  • Observers check transaction.local for side effects
  • Awareness updates throttled (10-20/sec max)
  • No assumptions about sync order

See Also

Document Formats

Detailed specifications of the document formats used by each Cloudillo application.

Overview

Each Cloudillo app stores collaborative documents using application-specific data models. Most apps use Yjs CRDTs for conflict-free concurrent editing, while some use the Real-Time Database (RTDB) for query-oriented structured data. This section provides the complete format specification for each app’s document structure, enabling third-party developers, tool builders, and contributors to understand, extend, and interoperate with Cloudillo documents.

Relationship to CRDT Design Guide

The CRDT Design Guide teaches generic patterns for building collaborative apps (ID-based storage, style inheritance, separate content maps). This section documents the concrete format specifications — the exact field names, types, and structures used by each app.

Application Formats

Application Content Type Storage Complexity Status
Prezillo application/vnd.cloudillo.prezillo+json CRDT (Yjs) High (14 object types, palette, templates) Documented
Calcillo application/vnd.cloudillo.calcillo+json CRDT (Yjs) Medium (cells, formulas, sheets) Documented
Ideallo application/vnd.cloudillo.ideallo+json CRDT (Yjs) Medium (9 object types, linked copies) Documented
Notillo application/vnd.cloudillo.notillo+json RTDB Low (rich text document) Planned
Quillo application/vnd.cloudillo.quillo+json CRDT (Yjs) Low (rich text editor) Planned

Common Conventions

Compact Field Names

CRDT-based apps use short field names (typically 1-3 characters) to minimize wire overhead during real-time synchronization. For example, t for type, xy for position, wh for dimensions. Each app’s format spec documents the mapping from compact names to their meanings.

ID Format

All entity IDs (objects, containers, views, styles, templates) use 72-bit entropy encoded as 12 base64url characters. IDs are generated client-side using crypto.getRandomValues() and are typed using branded types (ObjectId, ContainerId, ViewId, StyleId, RichTextId, TemplateId) for compile-time safety.

ChildRef Tuples

Several apps use a ChildRef tuple to reference children that can be either objects or containers:

type ChildRef = [0 | 1, string]  // [0, objectId] or [1, containerId]

The discriminant (0 or 1) allows a single ordered array to contain references to both types without ambiguity.

v3 Generic Export Format

Starting with format version 3.0.0, all CRDT-based apps use a unified export function (exportYDoc()) from @cloudillo/crdt. This function walks yDoc.share and serializes all shared types with inline @T type markers that preserve the original Yjs type information.

Export Envelope

Every exported document shares this envelope structure:

{
  "contentType": "application/vnd.cloudillo.<app>+json",
  "appVersion": "<semver>",
  "formatVersion": "3.0.0",
  "exportedAt": "<ISO 8601 timestamp>",
  "data": { ... }
}

The data object contains one entry per top-level Yjs shared type, using the raw CRDT key names (e.g., o, r, m instead of objects, rootChildren, meta).

@T Type Markers

Each Yjs type is serialized with an inline marker so importers can reconstruct the correct Yjs type:

Yjs Type Marker Serialized Form
Y.Map "@T": "M" { "@T": "M", "key1": ..., "key2": ... }
Y.Array "@T:A" [ "@T:A", element1, element2, ... ]
Y.Text "@T": "T" { "@T": "T", "text": "...", "delta": [...] }
Y.XmlText "@T": "XT" { "@T": "XT", "text": "...", "delta": [...] }
Y.XmlElement "@T": "XE" { "@T": "XE", "n": "...", "a": {...}, "c": [...] }
Y.XmlFragment "@T": "XF" { "@T": "XF", "c": [...] }

Nested types are serialized recursively. For example, a Y.Map<Y.Text> becomes a map with "@T": "M" containing entries that each have "@T": "T" with text and delta fields.

Primitive values pass through

Plain values (strings, numbers, booleans, null) inside Yjs types are serialized as-is without markers. The @T marker only appears on Yjs shared types.

Changes from Previous Versions

Aspect Pre-v3 v3
Data keys Long descriptive names (objects, rootChildren, richTexts) Raw CRDT keys (o, r, rt)
Type information Implicit (consumer must know the schema) Explicit @T markers on every Yjs type
Text content Plain text string or { plainText, delta } { "@T": "T", "text": "...", "delta": [...] }
Export function App-specific custom serialization Generic exportYDoc() from @cloudillo/crdt
Format version App-specific (1.0.0 or 2.0.0) Unified 3.0.0 across all apps

See Also

Subsections of Document Formats

Calcillo Format

Complete format specification for Calcillo, Cloudillo’s collaborative spreadsheet application.

Overview

Calcillo is a real-time collaborative spreadsheet that supports multiple sheets, formulas, merged cells, borders, hyperlinks, data validation, conditional formatting, and frozen panes. Documents are stored as Yjs CRDT structures for conflict-free concurrent editing. The cell data model is derived from FortuneSheet with inline styling.

  • Content type: application/vnd.cloudillo.calcillo+json
  • Format version: 3.0.0
  • File extension: .calcillo

Design Philosophy

  • Per-sheet encapsulation: Each sheet is a self-contained unit with its own rows, columns, merges, borders, and other features. This prevents cross-sheet conflicts and enables efficient partial sync – only active sheets need to be observed.
  • Nested row maps: Cell data uses rows[rowId][colId] = Cell, allowing efficient row-level operations (insert, delete, move) without touching unrelated cells.
  • ID-based addressing: Rows, columns, and sheets use random IDs (not sequential indices) for CRDT stability. IDs survive concurrent insertions and deletions without rebasing.
  • Inline styles: All cell formatting (font, color, alignment) is stored directly on each cell. There is no named style system – this trades storage efficiency for simplicity and avoids the style cascade complexity of apps like Prezillo.

Document Architecture

graph LR
    Doc[Y.Doc]

    subgraph Root
        so["sheetOrder (Y.Array&lt;SheetId&gt;)<br/>Sheet tab order"]
        sheets["sheets (Y.Map&lt;YSheetStructure&gt;)<br/>All sheets"]
        meta["meta (Y.Map)<br/>Document metadata"]
    end

    subgraph "Per-Sheet (YSheetStructure)"
        name["name (Y.Text)"]
        rows["rows (Y.Map&lt;Y.Map&lt;Cell&gt;&gt;)"]
        ro["rowOrder (Y.Array&lt;RowId&gt;)"]
        co["colOrder (Y.Array&lt;ColId&gt;)"]
        merges["merges (Y.Map&lt;MergeInfo&gt;)"]
        borders["borders (Y.Map&lt;BorderInfo&gt;)"]
        more["... 7 more sub-types"]
    end

    Doc --> Root
    sheets --> name
    sheets --> rows
    sheets --> ro
    sheets --> co
    sheets --> merges
    sheets --> borders
    sheets --> more

Quick Reference

Root-Level Shared Types

Key Yjs Type Purpose
sheetOrder Y.Array<SheetId> Sheet tab order
sheets Y.Map<YSheetStructure> All sheets keyed by SheetId
meta Y.Map Document metadata

Per-Sheet Sub-Types

Sub-type Yjs Type Purpose
name Y.Text Sheet name (collaborative editing)
rowOrder Y.Array<RowId> Stable row ordering
colOrder Y.Array<ColId> Stable column ordering
rows Y.Map<Y.Map<Cell>> Nested: rows[rowId][colId] = Cell (sparse)
merges Y.Map<MergeInfo> Key: "${startRow}_${startCol}"
borders Y.Map<BorderInfo> Key: "${rowId}_${colId}"
hyperlinks Y.Map<HyperlinkInfo> Key: "${rowId}_${colId}"
validations Y.Map<ValidationRule> Key: unique validation ID
conditionalFormats Y.Array<ConditionalFormat> Ordered rules
hiddenRows Y.Map<boolean> rowId → true if hidden
hiddenCols Y.Map<boolean> colId → true if hidden
rowHeights Y.Map<number> rowId → height in pixels
colWidths Y.Map<number> colId → width in pixels
frozen Y.Map<string|number> Freeze pane settings

Detailed Documentation

  • Document Structure – Root Y.Doc, per-sheet structure, ID system, metadata, and initialization
  • Cells – Cell data model, values, formulas, inline styles, and cell types
  • Grid Structure – Row/column ordering, sizing, hidden rows/cols, and merged cells
  • Sheet Features – Borders, hyperlinks, data validation, conditional formatting, and frozen panes
  • Sheets – Multi-sheet support, sheet ordering, and sheet operations
  • Export Format – JSON export envelope with complete example

See Also

  • CRDT Design Guide – Generic collaboration patterns and best practices
  • Prezillo Format – Prezillo uses a more complex architecture with hierarchy, views, and style inheritance
  • Ideallo Format – Ideallo uses a flat object model for infinite-canvas whiteboards

Subsections of Calcillo Format

Document Structure

The root CRDT document structure, per-sheet structure, ID system, metadata, and initialization defaults.

Root-Level Shared Types

A Calcillo document is a Y.Doc with only 3 named shared types:

Key Yjs Type Description
sheetOrder Y.Array<SheetId> Sheet tab order (presentation sequence)
sheets Y.Map<YSheetStructure> All sheets keyed by SheetId
meta Y.Map Document metadata
Minimal root structure

Unlike Prezillo’s 12 or Ideallo’s 5 top-level shared types, Calcillo uses only 3. The complexity lives inside each sheet’s self-contained structure, which keeps the root document clean and enables efficient per-sheet synchronization.

Accessing the Document

import * as Y from 'yjs'

const yDoc = new Y.Doc()

// Access root shared types
const sheetOrder = yDoc.getArray('sheetOrder')  // SheetId ordering
const sheets     = yDoc.getMap('sheets')         // YSheetStructure entries
const meta       = yDoc.getMap('meta')           // Metadata

Per-Sheet Structure (YSheetStructure)

Each sheet is a self-contained Y.Map with 14 named sub-types:

Sub-type Yjs Type Description
name Y.Text Sheet name (collaborative text editing)
rowOrder Y.Array<RowId> Stable row ordering
colOrder Y.Array<ColId> Stable column ordering
rows Y.Map<Y.Map<Cell>> Cell data: rows[rowId][colId] = Cell
merges Y.Map<MergeInfo> Merged cell ranges, keyed by "${startRow}_${startCol}"
borders Y.Map<BorderInfo> Per-cell border info, keyed by "${rowId}_${colId}"
hyperlinks Y.Map<HyperlinkInfo> Per-cell hyperlinks, keyed by "${rowId}_${colId}"
validations Y.Map<ValidationRule> Data validation rules, keyed by unique validation ID
conditionalFormats Y.Array<ConditionalFormat> Ordered conditional formatting rules
hiddenRows Y.Map<boolean> rowId → true for hidden rows
hiddenCols Y.Map<boolean> colId → true for hidden columns
rowHeights Y.Map<number> Custom row heights in pixels (absent = default)
colWidths Y.Map<number> Custom column widths in pixels (absent = default)
frozen Y.Map<string|number> Freeze pane configuration

Accessing Sheet Sub-Types

// Get a sheet by ID
const sheet = sheets.get(sheetId) as Y.Map<unknown>

// Access sub-types
const name     = sheet.get('name') as Y.Text
const rowOrder = sheet.get('rowOrder') as Y.Array<string>
const colOrder = sheet.get('colOrder') as Y.Array<string>
const rows     = sheet.get('rows') as Y.Map<Y.Map<Cell>>
const merges   = sheet.get('merges') as Y.Map<MergeInfo>
const borders  = sheet.get('borders') as Y.Map<BorderInfo>

// Access a specific cell
const rowMap = rows.get(rowId) as Y.Map<Cell>
const cell = rowMap?.get(colId) as Cell | undefined
Per-sheet encapsulation

Each sheet manages its own rows, columns, merges, borders, and all other features independently. There are no cross-sheet references within the CRDT – formulas referencing other sheets are stored as plain strings and resolved by the calculation engine at runtime.

ID System

Calcillo uses 3 branded types with variable-length IDs, all encoded as base64url characters:

Branded Type Length Entropy Used In
SheetId 12 chars 72 bits sheets map keys, sheetOrder entries
RowId 9 chars 54 bits rowOrder entries, rows map keys, composite keys
ColId 5 chars 30 bits colOrder entries, inner rows map keys, composite keys

All IDs are generated client-side using crypto.getRandomValues() and encoded as base64url (A-Z, a-z, 0-9, -, _).

Why variable-length IDs?

Sheets use standard 72-bit IDs like other Cloudillo apps. Row and column IDs use shorter lengths because spreadsheets contain many more rows and columns than typical document entities, and the shorter IDs reduce CRDT storage overhead. The entropy levels are chosen to keep collision probability negligible for practical spreadsheet sizes.

Composite Key Convention

Several per-sheet maps use composite keys formed by joining two IDs with an underscore:

Map Key Format Example
merges "${startRowId}_${startColId}" "aB3x_Qm7k_Xk2nR"
borders "${rowId}_${colId}" "aB3x_Qm7k_Xk2nR"
hyperlinks "${rowId}_${colId}" "aB3x_Qm7k_Xk2nR"

This convention provides a deterministic, unique key for cell-position-based features without requiring additional ID generation.

Metadata

The meta map stores document-level settings as individual key-value entries:

Key Type Default Description
initialized boolean true Set during initialization, prevents re-initialization
name string "Untitled Spreadsheet" Document name

The meta map is compatible with Cloudillo’s cloudillo.init() initialization system – the initialized flag is checked before setting up default content.

Document Initialization

When a new empty document is created, the following defaults are established in a single Yjs transaction:

yDoc.transact(() => {
    // 1. Mark as initialized
    meta.set('initialized', true)
    meta.set('name', 'Untitled Spreadsheet')

    // 2. Create default sheet
    const sheetId = generateSheetId()  // 12 base64url chars
    sheetOrder.push([sheetId])

    // 3. Set up sheet structure
    const sheet = new Y.Map()
    sheets.set(sheetId, sheet)

    const name = new Y.Text()
    name.insert(0, 'Sheet 1')
    sheet.set('name', name)

    // 4. Generate 26 columns (A-Z)
    const colOrder = new Y.Array()
    for (let i = 0; i < 26; i++) {
        colOrder.push([generateColId()])  // 5 base64url chars each
    }
    sheet.set('colOrder', colOrder)

    // 5. Generate 100 rows
    const rowOrder = new Y.Array()
    for (let i = 0; i < 100; i++) {
        rowOrder.push([generateRowId()])  // 9 base64url chars each
    }
    sheet.set('rowOrder', rowOrder)

    // 6. Initialize empty sub-type maps
    sheet.set('rows', new Y.Map())
    sheet.set('merges', new Y.Map())
    sheet.set('borders', new Y.Map())
    sheet.set('hyperlinks', new Y.Map())
    sheet.set('validations', new Y.Map())
    sheet.set('conditionalFormats', new Y.Array())
    sheet.set('hiddenRows', new Y.Map())
    sheet.set('hiddenCols', new Y.Map())
    sheet.set('rowHeights', new Y.Map())
    sheet.set('colWidths', new Y.Map())
    sheet.set('frozen', new Y.Map())
})

The default sheet starts with 26 columns (corresponding to A-Z) and 100 rows. All cell data maps start empty – cells are created on first edit (sparse storage).

Cells

The cell data model, storage pattern, values, formulas, inline styles, and cell types.

Cell Storage

Cells are stored in a nested Y.Map structure within each sheet:

rows: Y.Map<Y.Map<Cell>>
       ^         ^
       |         └── Inner map: colId → Cell
       └── Outer map: rowId → inner map

This two-level nesting provides efficient row-level operations:

  • Insert/delete row: Add or remove a single entry in the outer map
  • Move row: Reorder in rowOrder array; the row’s cell data stays intact
  • Access cell: Two map lookups: rows.get(rowId)?.get(colId)
// Read a cell
const rowMap = rows.get(rowId) as Y.Map<Cell> | undefined
const cell = rowMap?.get(colId) as Cell | undefined

// Write a cell
let rowMap = rows.get(rowId) as Y.Map<Cell>
if (!rowMap) {
    rowMap = new Y.Map()
    rows.set(rowId, rowMap)
}
rowMap.set(colId, cellData)
Sparse storage

Only cells with actual content or formatting are stored. Empty cells have no entry in the map. When all cells in a row are cleared, the empty inner Y.Map can be garbage collected by deleting the outer map entry.

Cell Data Model

The Cell interface defines all possible fields on a cell. All fields are optional – an empty object {} represents a blank cell with all defaults.

Value Fields

Field Type Description
v string | number | boolean Display value (the computed/entered value)
f string Formula string (starts with =, e.g. "=SUM(A1:A10)")

Cell Type and Format

Field Type Description
ct CellType Cell type descriptor (see below)

The ct object describes how the cell value should be interpreted and formatted:

interface CellType {
    t?: 't' | 'n' | 's' | 'b' | 'g'  // Type code
    fa?: string                         // Format code
    s?: Array<{ v: string }>           // Rich text segments
}

Type Codes (ct.t)

Code Meaning Example Values
g General Auto-detected type
n Number 42, 3.14, -100
s String "Hello", "ABC"
t Time/Date "2026-01-15", "14:30:00"
b Boolean true, false

Format Codes (ct.fa)

The fa field contains a format pattern string compatible with spreadsheet number formatting:

Pattern Description Example Output
"General" Auto-format (default) 1234.5
"0" Integer 1235
"0.00" Two decimal places 1234.50
"#,##0" Thousands separator 1,235
"#,##0.00" Thousands + decimals 1,234.50
"0%" Percentage 12%
"0.00%" Percentage with decimals 12.35%
"$#,##0.00" Currency $1,234.50
"yyyy-mm-dd" Date 2026-01-15
"h:mm:ss" Time 14:30:00

Rich Text Segments (ct.s)

When a cell contains rich text (mixed formatting within a single cell), the ct.s array holds text segments:

{
    "ct": {
        "s": [
            { "v": "Bold text" },
            { "v": " and normal text" }
        ]
    }
}

Each segment contains a v field with the text content. Additional formatting fields may appear on individual segments.

Font Styling

All styling is inline on the cell – there is no named style or style inheritance system.

Field Type Default Description
bl number 0 Bold: 0 = off, 1 = on
it number 0 Italic: 0 = off, 1 = on
un number 0 Underline: 0 = off, 1 = on
cl number 0 Strikethrough: 0 = off, 1 = on
ff number 0 Font family code (0 = default)
fs number 10 Font size in points
fc string Font color (hex, e.g. "#333333")

Alignment

Field Type Default Description
ht number Horizontal: 1 = left, 2 = center, 3 = right, 4 = justify
vt number Vertical: 0 = top, 1 = middle, 2 = bottom

Cell Appearance

Field Type Default Description
bg string Background color (hex, e.g. "#ffeb3b")
tb number 0 Text wrap: 0 = off, 1 = on
tr number 0 Text rotation in degrees

Formula Storage

Formulas are stored as plain strings in the f field:

{
    "v": 150,
    "f": "=SUM(A1:A10)"
}
  • The f field contains the formula string as entered by the user (always starts with =)
  • The v field contains the last computed result
  • Formula references use A1-style notation (e.g. A1, B2:D10, Sheet2!A1)
  • The CRDT stores only the formula string – no ID-based cell references
  • Formula evaluation is delegated to the FortuneSheet calculation engine at runtime
A1 references vs ID-based addressing

Internally, Calcillo uses random IDs for rows and columns in the CRDT. However, formulas use traditional A1-style references. The conversion between column IDs (from colOrder) and letter codes (A, B, C…) happens at the application layer. This means formulas may need adjustment when rows or columns are inserted or deleted – this is handled by the formula engine, not the CRDT.

Default Stripping

To minimize CRDT storage overhead, fields that match their default values are stripped when saving a cell. The following fields are omitted when they match these defaults:

Field Default Value
bl 0
it 0
un 0
cl 0
ff 0
fs 10
tb 0
tr 0

Fields with no default (like v, f, fc, bg, ht, vt) are stored whenever present.

Cell Addressing

Cells are addressed internally by their (rowId, colId) pair. For display, the column position in colOrder is converted to a letter code:

Position Letter Code
0 A
1 B
25 Z
26 AA
27 AB

Row position in rowOrder is converted to a 1-based number. For example, the cell at position (row index 2, col index 0) displays as A3.

Grid Structure

Row and column ordering, sizing, visibility, and merged cells.

Row and Column Ordering

Each sheet maintains two Y.Array structures for row and column ordering:

Sub-type Yjs Type Description
rowOrder Y.Array<RowId> Ordered list of row IDs (9-char base64url)
colOrder Y.Array<ColId> Ordered list of column IDs (5-char base64url)

The position of an ID in its array determines the display position. Row rowOrder[0] displays as row 1, rowOrder[1] as row 2, etc. Column colOrder[0] displays as column A, colOrder[1] as column B, etc.

Row and Column Operations

// Insert a new row at position 5
const newRowId = generateRowId()
rowOrder.insert(5, [newRowId])

// Delete row at position 3
rowOrder.delete(3, 1)
// Also clean up cell data:
rows.delete(rowOrder.get(3))

// Move a column from position 2 to position 5
const colId = colOrder.get(2)
colOrder.delete(2, 1)
colOrder.insert(5, [colId])
ID-based stability

Because rows and columns are identified by random IDs rather than sequential indices, concurrent insertions and deletions merge cleanly. Two users inserting rows at different positions will both succeed without conflict, and the resulting order is deterministic.

Row and Column Sizing

Custom dimensions override the application defaults:

Sub-type Yjs Type Description
rowHeights Y.Map<number> rowId → height in pixels
colWidths Y.Map<number> colId → width in pixels

Rows and columns not present in these maps use application default sizes. Only non-default sizes are stored.

// Set column width to 200px
colWidths.set(colId, 200)

// Get row height (with default fallback)
const height = rowHeights.get(rowId) ?? DEFAULT_ROW_HEIGHT

Hidden Rows and Columns

Sub-type Yjs Type Description
hiddenRows Y.Map<boolean> rowId → true for hidden rows
hiddenCols Y.Map<boolean> colId → true for hidden columns

Hidden rows and columns remain in the rowOrder/colOrder arrays (preserving their position) but are not rendered. Only rows/columns that are actually hidden have entries in these maps.

// Hide a row
hiddenRows.set(rowId, true)

// Unhide a row
hiddenRows.delete(rowId)

// Check if a column is hidden
const isHidden = hiddenCols.get(colId) === true

Merged Cells

Merged cells are stored in the per-sheet merges map:

Sub-type Yjs Type Key Format
merges Y.Map<MergeInfo> "${startRowId}_${startColId}"

MergeInfo Structure

interface MergeInfo {
    startRow: RowId    // Top-left row ID
    endRow: RowId      // Bottom-right row ID
    startCol: ColId    // Top-left column ID
    endCol: ColId      // Bottom-right column ID
}

The merge key uses the top-left cell’s composite key. The value stores all four corner IDs, which define the rectangular merge region.

Merge Operations

// Create a merge spanning rows 2-4, columns B-D
merges.set(`${rowId2}_${colIdB}`, {
    startRow: rowId2,
    endRow: rowId4,
    startCol: colIdB,
    endCol: colIdD
})

// Remove a merge
merges.delete(`${rowId2}_${colIdB}`)
Concurrent deletion and merges

When a row or column that participates in a merge is deleted concurrently, the merge must be validated. If the start row/column of a merge no longer exists in rowOrder/colOrder, the merge entry becomes orphaned and should be cleaned up. The application layer handles this validation when processing CRDT updates.

Cell Content in Merged Regions

Only the top-left cell of a merged region holds content. Other cells in the region should be empty. When a merge is created, content from non-top-left cells is discarded. When a merge is removed, only the top-left cell retains its content.

Cell Access Patterns

Common patterns for working with the grid:

// Iterate over all cells in a row
const rowMap = rows.get(rowId) as Y.Map<Cell>
if (rowMap) {
    for (const [colId, cell] of rowMap.entries()) {
        // Process cell
    }
}

// Get the display position of a cell
const rowIndex = rowOrder.toArray().indexOf(rowId)  // 0-based
const colIndex = colOrder.toArray().indexOf(colId)  // 0-based
const cellRef = `${indexToColumnLetter(colIndex)}${rowIndex + 1}`  // e.g. "B3"

// Find the ID for a display reference like "C5"
const colId = colOrder.get(2)   // C = index 2
const rowId = rowOrder.get(4)   // 5 = index 4 (0-based)

Sheet Features

Per-sheet features including borders, hyperlinks, data validation, conditional formatting, and frozen panes.

Borders

Borders are stored per-cell in the borders map:

Sub-type Yjs Type Key Format
borders Y.Map<BorderInfo> "${rowId}_${colId}"

BorderInfo Structure

interface BorderInfo {
    top?: BorderEdge
    right?: BorderEdge
    bottom?: BorderEdge
    left?: BorderEdge
}

interface BorderEdge {
    style: number    // Border style code
    color: string    // Border color (hex)
}

Each cell can have independent borders on all four edges. Only edges with explicit borders are stored.

Border Style Codes

Code Style
1 Thin
2 Medium
3 Thick
4 Dashed
5 Dotted
6 Double

Example

// Set a thick red bottom border on a cell
borders.set(`${rowId}_${colId}`, {
    bottom: { style: 3, color: '#ff0000' }
})
Adjacent cell borders

Border rendering between adjacent cells follows the convention that each cell owns its own border edges. When two adjacent cells define conflicting borders on the shared edge (e.g. cell A’s right border vs cell B’s left border), the application layer resolves which to display.

Hyperlinks are stored per-cell in the hyperlinks map:

Sub-type Yjs Type Key Format
hyperlinks Y.Map<HyperlinkInfo> "${rowId}_${colId}"

HyperlinkInfo Structure

interface HyperlinkInfo {
    type: 'external' | 'internal' | 'email'
    address: string    // URL, cell reference, or email address
    tooltip?: string   // Hover tooltip text
}
Type Address Format Example
external Full URL "https://example.com"
internal Cell or sheet reference "Sheet2!A1"
email Email address "user@example.com"

Example

hyperlinks.set(`${rowId}_${colId}`, {
    type: 'external',
    address: 'https://docs.cloudillo.org',
    tooltip: 'Cloudillo Documentation'
})

Data Validation

Validation rules are stored in the validations map, keyed by unique validation IDs:

Sub-type Yjs Type Key Format
validations Y.Map<ValidationRule> Unique validation ID

ValidationRule Structure

interface ValidationRule {
    ranges: Array<{
        startRow: RowId
        endRow: RowId
        startCol: ColId
        endCol: ColId
    }>
    type: 'dropdown' | 'checkbox' | 'number' | 'date' | 'text'
    options?: ValidationOptions
}

Validation Types

Type Description Options
dropdown Dropdown list selection values: string[] – allowed values
checkbox Boolean checkbox checkedValue?: string, uncheckedValue?: string
number Numeric constraint operator, value1, value2 (for between)
date Date constraint operator, value1, value2 (for between)
text Text constraint operator, value1

Operators

Number and date validations support:

Operator Description
between Value between value1 and value2
notBetween Value not between value1 and value2
equal Value equals value1
notEqual Value not equal to value1
greaterThan Value greater than value1
lessThan Value less than value1
greaterThanOrEqual Value >= value1
lessThanOrEqual Value <= value1

Example

validations.set(validationId, {
    ranges: [{
        startRow: rowId1,
        endRow: rowId10,
        startCol: colIdB,
        endCol: colIdB
    }],
    type: 'dropdown',
    options: {
        values: ['Active', 'Inactive', 'Pending']
    }
})
Range-based rules

Unlike borders and hyperlinks which are per-cell, validation rules apply to rectangular ranges. A single rule can cover multiple cells, avoiding duplication when the same constraint applies to an entire column or region.

Conditional Formatting

Conditional formatting rules are stored as an ordered array:

Sub-type Yjs Type Description
conditionalFormats Y.Array<ConditionalFormat> Rules evaluated in order (first match wins)

ConditionalFormat Structure

interface ConditionalFormat {
    ranges: Array<{
        startRow: RowId
        endRow: RowId
        startCol: ColId
        endCol: ColId
    }>
    type: 'cellValue' | 'colorScale' | 'dataBar' | 'iconSet'
    rule: ConditionalRule
    format?: CellFormat
}

Rule Types

Cell Value Rules (type: 'cellValue')

Apply formatting when cells meet a condition:

{
    "type": "cellValue",
    "rule": {
        "operator": "greaterThan",
        "value1": 100
    },
    "format": {
        "bg": "#c8e6c9",
        "fc": "#2e7d32"
    }
}

Color Scale Rules (type: 'colorScale')

Apply a gradient color scale across cell values:

{
    "type": "colorScale",
    "rule": {
        "minColor": "#f44336",
        "midColor": "#ffeb3b",
        "maxColor": "#4caf50"
    }
}

Data Bar Rules (type: 'dataBar')

Show proportional bars within cells:

{
    "type": "dataBar",
    "rule": {
        "color": "#2196f3"
    }
}

Icon Set Rules (type: 'iconSet')

Display icons based on value thresholds:

{
    "type": "iconSet",
    "rule": {
        "iconType": "arrows3",
        "thresholds": [33, 67]
    }
}

Frozen Panes

Freeze pane settings are stored in the per-sheet frozen map:

Sub-type Yjs Type Description
frozen Y.Map<string|number> Freeze pane configuration

Frozen Map Keys

Key Type Description
type string Freeze type: "row", "column", "both", or "range"
rowIndex number Number of frozen rows (from top)
colIndex number Number of frozen columns (from left)

Freeze Types

Type Description
row Freeze rows above the focus row
column Freeze columns to the left of the focus column
both Freeze both rows and columns
range Freeze a specific range (uses both rowIndex and colIndex)

Example

// Freeze the first 2 rows and first column
frozen.set('type', 'both')
frozen.set('rowIndex', 2)
frozen.set('colIndex', 1)

// Remove freeze
frozen.delete('type')
frozen.delete('rowIndex')
frozen.delete('colIndex')

Sheets

Multi-sheet support, sheet ordering, and sheet operations.

Sheet Definition

Each sheet is a self-contained Y.Map entry in the root sheets map, keyed by a SheetId (12-char base64url). The sheet’s name is stored as a Y.Text instance, enabling collaborative editing of the sheet name itself.

// Get sheet name
const sheet = sheets.get(sheetId) as Y.Map<unknown>
const name = sheet.get('name') as Y.Text
console.log(name.toString())  // "Sheet 1"

// Rename a sheet (collaborative)
name.delete(0, name.length)
name.insert(0, 'Revenue Data')
Why Y.Text for sheet names?

Using Y.Text instead of a plain string allows two users to concurrently edit a sheet name (e.g. both typing in the rename field). While rare, this prevents the last-writer-wins conflict that a plain string would cause.

Sheet Ordering

Sheet tab order is maintained by the root sheetOrder array:

const sheetOrder = yDoc.getArray('sheetOrder')  // Y.Array<SheetId>

The position of a SheetId in this array determines its tab position. The first entry is the leftmost tab, etc.

// Get the first sheet
const firstSheetId = sheetOrder.get(0) as string
const firstSheet = sheets.get(firstSheetId) as Y.Map<unknown>

// Iterate over all sheets in tab order
for (let i = 0; i < sheetOrder.length; i++) {
    const sheetId = sheetOrder.get(i) as string
    const sheet = sheets.get(sheetId) as Y.Map<unknown>
    const name = (sheet.get('name') as Y.Text).toString()
    console.log(`Tab ${i + 1}: ${name}`)
}

Sheet Independence

Each sheet is fully self-contained. All of the following are scoped to individual sheets and do not reference other sheets:

  • Row and column ordering (rowOrder, colOrder)
  • Cell data (rows)
  • Merged cells (merges)
  • Borders, hyperlinks, validations, conditional formats
  • Hidden rows/columns, sizing, frozen panes

This design means:

  • Deleting a sheet cleanly removes all associated data
  • Sheets can be synced independently (only observe active sheets)
  • No orphaned references when sheets are removed

Sheet Operations

Adding a Sheet

yDoc.transact(() => {
    const sheetId = generateSheetId()

    // Create sheet structure
    const sheet = new Y.Map()
    sheets.set(sheetId, sheet)

    // Set name
    const name = new Y.Text()
    name.insert(0, 'Sheet 2')
    sheet.set('name', name)

    // Initialize columns and rows
    const colOrder = new Y.Array()
    for (let i = 0; i < 26; i++) {
        colOrder.push([generateColId()])
    }
    sheet.set('colOrder', colOrder)

    const rowOrder = new Y.Array()
    for (let i = 0; i < 100; i++) {
        rowOrder.push([generateRowId()])
    }
    sheet.set('rowOrder', rowOrder)

    // Initialize empty sub-type maps
    sheet.set('rows', new Y.Map())
    sheet.set('merges', new Y.Map())
    sheet.set('borders', new Y.Map())
    sheet.set('hyperlinks', new Y.Map())
    sheet.set('validations', new Y.Map())
    sheet.set('conditionalFormats', new Y.Array())
    sheet.set('hiddenRows', new Y.Map())
    sheet.set('hiddenCols', new Y.Map())
    sheet.set('rowHeights', new Y.Map())
    sheet.set('colWidths', new Y.Map())
    sheet.set('frozen', new Y.Map())

    // Add to tab order (at the end)
    sheetOrder.push([sheetId])
})

Removing a Sheet

yDoc.transact(() => {
    // Remove from tab order
    const index = sheetOrder.toArray().indexOf(sheetId)
    if (index !== -1) {
        sheetOrder.delete(index, 1)
    }

    // Remove sheet data
    sheets.delete(sheetId)
})
Last sheet protection

The application layer should prevent deleting the last sheet. A Calcillo document must always have at least one sheet.

Reordering Sheets

yDoc.transact(() => {
    // Move sheet from position 2 to position 0 (make it first tab)
    const sheetId = sheetOrder.get(2) as string
    sheetOrder.delete(2, 1)
    sheetOrder.insert(0, [sheetId])
})

Duplicating a Sheet

Sheet duplication involves deep-copying all sub-type data into a new sheet with fresh IDs. The new sheet gets a new SheetId, but internal row and column IDs must also be regenerated to avoid ID collisions. Cell content and formatting are copied, but formulas referencing the original sheet’s cells are not automatically updated.

Active Sheet

The currently active (visible) sheet is not stored in the CRDT. Each user can view a different sheet independently. The active sheet is communicated through Yjs awareness:

awareness.setLocalStateField('activeSheet', sheetId)

This ensures that one user switching tabs does not affect other users’ views.

Export Format

Calcillo documents can be exported as self-contained JSON files for backup, sharing, and interoperability.

File Format

  • File extension: .calcillo
  • Content type: application/vnd.cloudillo.calcillo+json
  • Format version: 3.0.0
  • Encoding: UTF-8 JSON
v3 generic export format

Since format version 3.0.0, Calcillo uses the generic exportYDoc() serializer from @cloudillo/crdt. All Yjs types carry inline @T type markers and data keys match the raw CRDT shared type names. See v3 Generic Export Format for the full specification.

Envelope Structure

{
  "contentType": "application/vnd.cloudillo.calcillo+json",
  "appVersion": "0.1.0",
  "formatVersion": "3.0.0",
  "exportedAt": "2026-02-20T10:00:00.000Z",
  "data": {
    "meta": { "@T": "M", ... },
    "sheetOrder": [ "@T:A", ... ],
    "sheets": { "@T": "M", ... }
  }
}

Envelope Fields

Field Type Description
contentType string Always "application/vnd.cloudillo.calcillo+json"
appVersion string Calcillo version that created this export
formatVersion string Export format version (currently "3.0.0")
exportedAt string ISO 8601 timestamp of export

Data Fields

Key @T Yjs Type Description
meta M Y.Map Document metadata (name, initialized flag)
sheetOrder A Y.Array<SheetId> Sheet IDs in tab order
sheets M Y.Map<YSheetStructure> All sheets keyed by SheetId

Sheet Structure

Each sheet in the sheets map is a Y.Map (@T: "M") containing nested shared types:

Key @T Yjs Type Description
name T Y.Text Sheet name (with text and delta fields)
rowOrder A Y.Array<RowId> Row IDs in display order
colOrder A Y.Array<ColId> Column IDs in display order
rows M Y.Map<Y.Map<Cell>> Nested: rowId → colId → Cell
merges M Y.Map<MergeInfo> Merge definitions keyed by composite key
borders M Y.Map<BorderInfo> Border definitions keyed by composite key
hyperlinks M Y.Map<HyperlinkInfo> Hyperlink definitions keyed by composite key
validations M Y.Map<ValidationRule> Validation rules keyed by validation ID
conditionalFormats A Y.Array<ConditionalFormat> Ordered conditional formatting rules
hiddenRows M Y.Map<boolean> Hidden row flags
hiddenCols M Y.Map<boolean> Hidden column flags
rowHeights M Y.Map<number> Custom row heights in pixels
colWidths M Y.Map<number> Custom column widths in pixels
frozen M Y.Map<string | number> Freeze pane settings
Sheet names are Y.Text in v3

The name field is serialized as a Y.Text with @T: "T", containing both text and delta fields. This preserves any concurrent edit state. In pre-v3 exports, sheet names were plain text strings from Y.Text.toString().

Default cell values are stripped

Calcillo applies a transformSheets post-processing step that removes default cell property values to reduce export size. Empty rows (containing only the @T marker) are also omitted.

Numeric Precision

All numeric values are rounded to 3 decimal places in the export to produce cleaner output. Cell values retain their original precision.

Complete Example

A spreadsheet with 2 sheets. The first sheet has values, a formula, styling, a merge, and a border. The second sheet is a simple data table.

{
  "contentType": "application/vnd.cloudillo.calcillo+json",
  "appVersion": "0.1.0",
  "formatVersion": "3.0.0",
  "exportedAt": "2026-02-20T10:00:00.000Z",
  "data": {
    "meta": {
      "@T": "M",
      "initialized": true,
      "name": "Q1 Sales Report"
    },
    "sheetOrder": ["@T:A", "aB3x_Qm7kL9p", "Xk2nR8vH_wYq"],
    "sheets": {
      "@T": "M",
      "aB3x_Qm7kL9p": {
        "@T": "M",
        "name": {
          "@T": "T",
          "text": "Summary",
          "delta": [{ "insert": "Summary" }]
        },
        "rowOrder": ["@T:A", "r_Hw5qT2m", "r_m4JfL1p", "r_Nz9cKvW", "r_Qp4rW9x"],
        "colOrder": ["@T:A", "cAb3x", "cXk2n", "cM4Jf", "cNz9c"],
        "rows": {
          "@T": "M",
          "r_Hw5qT2m": {
            "@T": "M",
            "cAb3x": {
              "@T": "M",
              "v": "Product",
              "bl": 1,
              "bg": "#e3f2fd",
              "ht": 2
            },
            "cXk2n": {
              "@T": "M",
              "v": "Q1",
              "bl": 1,
              "bg": "#e3f2fd",
              "ht": 2
            },
            "cM4Jf": {
              "@T": "M",
              "v": "Q2",
              "bl": 1,
              "bg": "#e3f2fd",
              "ht": 2
            },
            "cNz9c": {
              "@T": "M",
              "v": "Total",
              "bl": 1,
              "bg": "#e3f2fd",
              "ht": 2
            }
          },
          "r_m4JfL1p": {
            "@T": "M",
            "cAb3x": { "@T": "M", "v": "Widget A" },
            "cXk2n": {
              "@T": "M",
              "v": 15000,
              "ct": { "@T": "M", "t": "n", "fa": "$#,##0.00" }
            },
            "cM4Jf": {
              "@T": "M",
              "v": 18500,
              "ct": { "@T": "M", "t": "n", "fa": "$#,##0.00" }
            },
            "cNz9c": {
              "@T": "M",
              "v": 33500,
              "f": "=B2+C2",
              "ct": { "@T": "M", "t": "n", "fa": "$#,##0.00" },
              "bl": 1
            }
          },
          "r_Nz9cKvW": {
            "@T": "M",
            "cAb3x": { "@T": "M", "v": "Widget B" },
            "cXk2n": {
              "@T": "M",
              "v": 22000,
              "ct": { "@T": "M", "t": "n", "fa": "$#,##0.00" }
            },
            "cM4Jf": {
              "@T": "M",
              "v": 19800,
              "ct": { "@T": "M", "t": "n", "fa": "$#,##0.00" }
            },
            "cNz9c": {
              "@T": "M",
              "v": 41800,
              "f": "=B3+C3",
              "ct": { "@T": "M", "t": "n", "fa": "$#,##0.00" },
              "bl": 1
            }
          },
          "r_Qp4rW9x": {
            "@T": "M",
            "cAb3x": {
              "@T": "M",
              "v": "Grand Total",
              "bl": 1,
              "it": 1
            },
            "cNz9c": {
              "@T": "M",
              "v": 75300,
              "f": "=D2+D3",
              "ct": { "@T": "M", "t": "n", "fa": "$#,##0.00" },
              "bl": 1,
              "bg": "#fff9c4"
            }
          }
        },
        "merges": {
          "@T": "M",
          "r_Qp4rW9x_cAb3x": {
            "@T": "M",
            "startRow": "r_Qp4rW9x",
            "endRow": "r_Qp4rW9x",
            "startCol": "cAb3x",
            "endCol": "cM4Jf"
          }
        },
        "borders": {
          "@T": "M",
          "r_Hw5qT2m_cAb3x": {
            "@T": "M",
            "bottom": { "@T": "M", "style": 2, "color": "#1565c0" }
          },
          "r_Hw5qT2m_cXk2n": {
            "@T": "M",
            "bottom": { "@T": "M", "style": 2, "color": "#1565c0" }
          },
          "r_Hw5qT2m_cM4Jf": {
            "@T": "M",
            "bottom": { "@T": "M", "style": 2, "color": "#1565c0" }
          },
          "r_Hw5qT2m_cNz9c": {
            "@T": "M",
            "bottom": { "@T": "M", "style": 2, "color": "#1565c0" }
          }
        },
        "hyperlinks": { "@T": "M" },
        "validations": { "@T": "M" },
        "conditionalFormats": ["@T:A"],
        "hiddenRows": { "@T": "M" },
        "hiddenCols": { "@T": "M" },
        "rowHeights": { "@T": "M" },
        "colWidths": {
          "@T": "M",
          "cAb3x": 150,
          "cXk2n": 120,
          "cM4Jf": 120,
          "cNz9c": 120
        },
        "frozen": {
          "@T": "M",
          "type": "row",
          "rowIndex": 1
        }
      },
      "Xk2nR8vH_wYq": {
        "@T": "M",
        "name": {
          "@T": "T",
          "text": "Raw Data",
          "delta": [{ "insert": "Raw Data" }]
        },
        "rowOrder": ["@T:A", "r_Jx8mP3q", "r_Vn4wK7r", "r_Bt6yH2s"],
        "colOrder": ["@T:A", "cPq7r", "cWm3s"],
        "rows": {
          "@T": "M",
          "r_Jx8mP3q": {
            "@T": "M",
            "cPq7r": { "@T": "M", "v": "Date", "bl": 1 },
            "cWm3s": { "@T": "M", "v": "Amount", "bl": 1 }
          },
          "r_Vn4wK7r": {
            "@T": "M",
            "cPq7r": {
              "@T": "M",
              "v": "2026-01-15",
              "ct": { "@T": "M", "t": "t", "fa": "yyyy-mm-dd" }
            },
            "cWm3s": {
              "@T": "M",
              "v": 5200,
              "ct": { "@T": "M", "t": "n", "fa": "#,##0" }
            }
          },
          "r_Bt6yH2s": {
            "@T": "M",
            "cPq7r": {
              "@T": "M",
              "v": "2026-02-01",
              "ct": { "@T": "M", "t": "t", "fa": "yyyy-mm-dd" }
            },
            "cWm3s": {
              "@T": "M",
              "v": 7800,
              "ct": { "@T": "M", "t": "n", "fa": "#,##0" }
            }
          }
        },
        "merges": { "@T": "M" },
        "borders": { "@T": "M" },
        "hyperlinks": { "@T": "M" },
        "validations": { "@T": "M" },
        "conditionalFormats": ["@T:A"],
        "hiddenRows": { "@T": "M" },
        "hiddenCols": { "@T": "M" },
        "rowHeights": { "@T": "M" },
        "colWidths": { "@T": "M" },
        "frozen": { "@T": "M" }
      }
    }
  }
}

In this example:

  • Summary sheet (aB3x_Qm7kL9p):
    • Header row with bold text, blue background, and centered alignment
    • Product data with currency formatting ($#,##0.00)
    • Formula cells computing totals (=B2+C2, =B3+C3, =D2+D3)
    • “Grand Total” row with a merge spanning columns A-C and italic styling
    • Medium blue bottom border on all header cells
    • Custom column widths (150px for product name, 120px for data columns)
    • Frozen first row (header stays visible while scrolling)
  • Raw Data sheet (Xk2nR8vH_wYq):
    • Simple 2-column table with date and amount data
    • Date cells with yyyy-mm-dd format, number cells with thousands separator
    • No merges, borders, or custom sizing (all defaults)
  • All Y.Map entries carry "@T": "M", arrays carry "@T:A", and sheet names are Y.Text with "@T": "T"

Ideallo Format

Complete format specification for Ideallo, Cloudillo’s collaborative infinite-canvas whiteboard and diagramming app.

Overview

Ideallo is a real-time collaborative whiteboard that supports freehand drawing, shapes, text labels, sticky notes, polygons, connectors, and images on an infinite canvas. Documents are stored as Yjs CRDT structures for conflict-free concurrent editing.

  • Content type: application/vnd.cloudillo.ideallo+json
  • Format version: 3.0.0
  • File extension: .ideallo

Design Philosophy

  • Compact field names: All stored types use short keys (t, xy, wh, sc) to minimize CRDT sync overhead. This section documents the compact names directly – they map 1:1 to what you see in the code and on the wire.
  • Separate maps by content type: Objects, text content, polygon geometry, freehand paths, and z-order each get their own top-level Yjs shared type. This enables targeted observers and prevents the CRDT shared-type nesting pitfall.
  • Flat object model: Unlike Prezillo’s hierarchy of layers, groups, views, and style inheritance, Ideallo uses a flat structure – all objects live in a single map with no parent-child relationships, no layers, and no global style system. Every style property is inline on each object.

Document Architecture

graph LR
    Doc[Y.Doc]

    subgraph Content
        o["o (Y.Map&lt;StoredObject&gt;)<br/>Objects"]
        r["r (Y.Array&lt;string&gt;)<br/>Z-Order"]
        txt["txt (Y.Map&lt;Y.Text&gt;)<br/>Text Content"]
        geo["geo (Y.Map&lt;Y.Array&lt;number&gt;&gt;)<br/>Polygon Geometry"]
        paths["paths (Y.Map&lt;string&gt;)<br/>Freehand Paths"]
    end

    subgraph Settings
        m["m (Y.Map)<br/>Metadata"]
    end

    Doc --> Content
    Doc --> Settings

Quick Reference

Map Key Yjs Type Purpose
o Y.Map<StoredObject> All canvas objects (rects, text, images, etc.) keyed by ObjectId
r Y.Array<string> Z-order array of ObjectId values (index 0 = backmost)
txt Y.Map<Y.Text> Text content for Text and Sticky objects (Quill Delta format)
geo Y.Map<Y.Array<number>> Flat vertex arrays for Polygon objects
paths Y.Map<string> SVG path strings for Freehand objects
m Y.Map Document metadata (name, background color, grid settings)

Detailed Documentation

  • Document Structure – Top-level CRDT maps, initialization, ID system, and metadata
  • Object Types – All 9 object types with their fields and supporting types
  • Linked Copies – Content sharing between objects via tid/gid/pid
  • Export Format – JSON export envelope and serialization details

See Also

Subsections of Ideallo Format

Document Structure

The top-level CRDT document structure, initialization defaults, ID system, and metadata.

Top-Level Shared Types

An Ideallo document is a Y.Doc with 6 named shared types:

Map Key Yjs Type Description
o Y.Map<StoredObject> All canvas objects keyed by ObjectId
r Y.Array<string> Z-order array of ObjectId values (index 0 = backmost)
m Y.Map Document metadata
txt Y.Map<Y.Text> Text content keyed by ObjectId (Quill Delta format)
geo Y.Map<Y.Array<number>> Polygon vertex arrays keyed by ObjectId (flat [x, y, x, y, ...])
paths Y.Map<string> SVG path strings keyed by ObjectId
Why compact map keys?

Map keys like o, m, txt are used instead of objects, metadata, texts because these keys appear in every Yjs sync message. Shorter keys reduce wire overhead during real-time collaboration without affecting readability – this documentation provides the complete mapping.

Accessing the Document

import * as Y from 'yjs'

const yDoc = new Y.Doc()

// Access each shared type by its map key
const objects  = yDoc.getMap('o')      // StoredObject entries
const zOrder   = yDoc.getArray('r')    // Z-order (ObjectId list)
const meta     = yDoc.getMap('m')      // Metadata
const texts    = yDoc.getMap('txt')    // Y.Text entries
const geometry = yDoc.getMap('geo')    // Y.Array<number> entries
const paths    = yDoc.getMap('paths')  // SVG path strings

ID System

All entity IDs use 72-bit entropy encoded as 12 base64url characters, generated client-side via crypto.getRandomValues().

Ideallo uses a single branded type for compile-time safety:

Branded Type Used In Example
ObjectId o map keys, txt/geo/paths map keys, tid/gid/pid field values "aB3x_Qm7kL9p"

Unlike Prezillo’s 6 branded types, Ideallo only needs ObjectId because there are no containers, views, styles, or templates. The same ID type is used for both object identifiers and content map keys.

Linked copy IDs

Objects with text, geometry, or path content normally use their own ObjectId as the key in the corresponding content map. Linked copies override this with explicit tid, gid, or pid fields pointing to a different object’s content entry. See Linked Copies for details.

Metadata

The m map stores document-level settings as individual key-value entries:

Key Type Default Description
initialized boolean true Set during initialization, prevents re-initialization
name string "Untitled Board" Document name
backgroundColor string "#f8f9fa" Canvas background color
gridSize number Grid spacing in pixels (if grid enabled)
snapToGrid boolean Whether objects snap to grid

Document Initialization

When a new empty document is created, the following defaults are established in a single Yjs transaction:

yDoc.transact(() => {
    doc.m.set('initialized', true)
    doc.m.set('name', 'Untitled Board')
    doc.m.set('backgroundColor', '#f8f9fa')
})

The initialized flag prevents re-initialization when the document is reopened. Unlike Prezillo, there are no default layers, views, styles, or palette entries – just the metadata.

Z-Order

The r shared type is a Y.Array<string> that defines the front-to-back rendering order of all objects on the canvas. Each entry is an ObjectId. Index 0 is the backmost object; the last index is the frontmost.

Why a Separate Array?

Z-order is stored as a standalone Y.Array rather than as a numeric property on each object because:

  • Reordering is a list operation: Moving an object forward or backward changes the position of one entry relative to others. A CRDT array handles concurrent reorder operations (insert/delete at positions) naturally, while numeric z-index values on objects would require renumbering and create conflicts.
  • Rendering order is global: The canvas needs a single authoritative ordering of all objects. A separate array makes this explicit and avoids scanning every object to reconstruct the order.

Operations

Operation Implementation
Add object r.push([objectId]) – new objects appear on top
Delete object Find index of objectId in r, then r.delete(index, 1)
Bring to front Delete from current position, r.push([objectId])
Send to back Delete from current position, r.insert(0, [objectId])
Move forward/backward Delete from current position, r.insert(newIndex, [objectId])

Separate Content Maps

Ideallo uses three separate content maps to store data that cannot be embedded directly in object entries:

Text Content (txt)

Stores Y.Text instances for Text (T) and Sticky (S) objects. Uses Quill Delta format for rich text operations (bold, italic, links, etc.). The map key is the object’s own ObjectId (or the tid value for linked copies).

Polygon Geometry (geo)

Stores Y.Array<number> instances for Polygon (P) objects. Vertices are stored as flat interleaved pairs: [x1, y1, x2, y2, ...]. Using Y.Array allows efficient incremental updates – new points can be appended without replacing the entire vertex set.

Freehand Paths (paths)

Stores SVG path strings (e.g., "M 0 0 C 10 5 20 10 30 15") for Freehand (F) objects. Unlike geometry, paths are stored as plain strings rather than Y.Array because freehand strokes are committed as a complete unit after the stroke ends (not incrementally during drawing).

Why separate maps?

Y.Text and Y.Array are Yjs shared types that need their own identity for collaborative editing – they cannot be stored as plain JSON values inside a Y.Map entry. Even paths uses a separate map for consistency and to support the linked copy pattern where multiple objects share the same content entry.

Object Types

Ideallo supports 9 object types, all sharing a common set of base fields and inline style fields with type-specific extensions.

Base Fields

Every stored object has the following fields defined by StoredObjectBase:

Field Type Default Description
t ObjectTypeCode (required) Object type discriminant
xy [number, number] (required) Position [x, y] on the infinite canvas
r number 0 Rotation in degrees (omitted if 0)
pv [number, number] [0.5, 0.5] Pivot point normalized 0–1 (omitted if center)
lk true false Locked – only stored when true
sn true false Snapped – Smart Ink auto-detected as shape (only stored when true)

Inline Style Fields

Every stored object also includes inline style fields from StoredStyle:

Field Type Default Description
sc string 'n0' Stroke color (palette key or CSS color)
fc string 'transparent' Fill color (palette key or CSS color)
sw number 2 Stroke width in pixels
ss StrokeStyleCode 'S' (solid) Stroke style
op number 1 Opacity (0–1)

All style fields are optional – when omitted, the default value is used. Only non-default values are stored.

No global style system

Unlike Prezillo, Ideallo has no global style definitions or style inheritance. All style properties are stored inline on each object. This keeps the format simple and avoids the complexity of style cascading for a whiteboard use case.

Object Type Codes

Code Name Has Content Map Description
F Freehand paths Bezier path stored as SVG path string
R Rectangle Rectangle with optional corner radius
E Ellipse Ellipse/circle
L Line Two-point line
A Arrow Two-point arrow with configurable arrowheads
T Text txt Text label with rich text content
P Polygon geo Multi-vertex polygon (triangle, pentagon, etc.)
S Sticky txt Sticky note with rich text content
I Image Uploaded image

Per-Type Fields

Freehand (F)

Field Type Default Description
wh [number, number] (required) Bounding box [width, height] of the path
pid string Path content ID. If omitted, the object’s own ID is used as the key in the paths map. Set explicitly for linked copies
cl true false Closed path – only stored when true

The SVG path data (e.g., "M 0 0 C 10 5 20 10 30 15") is stored in the paths map, not on the object itself.

Rectangle (R)

Field Type Default Description
wh [number, number] (required) Dimensions [width, height]
cr number Corner radius (omitted if sharp corners)

Ellipse (E)

Field Type Default Description
wh [number, number] (required) Dimensions [width, height]

No additional fields beyond wh.

Line (L)

Field Type Default Description
pts [[number, number], [number, number]] (required) Start and end points as absolute canvas coordinates [[startX, startY], [endX, endY]]
Absolute coordinates

Unlike Prezillo where line points are relative to the bounding box, Ideallo stores line and arrow endpoints as absolute canvas coordinates.

Arrow (A)

Field Type Default Description
pts [[number, number], [number, number]] (required) Start and end points as absolute canvas coordinates [[startX, startY], [endX, endY]]
ah ArrowheadPosition 'E' (end) Where to draw arrowheads (omitted if end-only, the default)

Text (T)

Field Type Default Description
wh [number, number] (required) Dimensions [width, height]
tid string Text content ID. If omitted, the object’s own ID is used as the key in the txt map. Set explicitly for linked copies
ff string Font family override
fz number Font size override

Rich text content is stored as a Y.Text entry in the txt map (keyed by the object’s own ID or the tid value), using Quill Delta format. See Document Structure > Text Content.

Polygon (P)

Field Type Default Description
gid string Geometry content ID. If omitted, the object’s own ID is used as the key in the geo map. Set explicitly for linked copies

Vertices are stored in the geo map as a flat Y.Array<number> of interleaved coordinate pairs: [x1, y1, x2, y2, ...].

Sticky (S)

Field Type Default Description
wh [number, number] (required) Dimensions [width, height]
tid string Text content ID. If omitted, the object’s own ID is used as the key in the txt map. Set explicitly for linked copies

Sticky notes use the standard fc (fill color) field for their background color and store rich text content in the txt map, identical to Text objects.

Image (I)

Field Type Default Description
wh [number, number] (required) Dimensions [width, height]
fid string (required) File ID from the Cloudillo media/file system

Supporting Types

StrokeStyleCode

Code Name
S Solid
D Dashed
T Dotted

ArrowheadPosition

Code Name Description
S Start Arrowhead at the start point only
E End Arrowhead at the end point only (default)
B Both Arrowheads at both start and end

Linked Copies

Ideallo supports linked copies – objects that share the same underlying content while maintaining independent position, style, and other metadata.

Overview

When an object is duplicated as a linked copy, the new object shares the same text content, polygon geometry, or freehand path data as the original. Editing the shared content in one object immediately updates all linked copies. However, each copy has its own position (xy), rotation (r), style fields (sc, fc, etc.), and dimensions (wh).

This is useful for repeating elements like labels, shapes, or icons that should stay synchronized across the board.

Content ID Fields

Three optional fields control which content map entry an object references:

Field Object Types Content Map Description
tid Text (T), Sticky (S) txt Text content ID – keys into the Y.Text map
gid Polygon (P) geo Geometry ID – keys into the Y.Array<number> map
pid Freehand (F) paths Path ID – keys into the SVG path string map

Rule: When the content ID field is omitted, the object’s own ObjectId is used as the key in the corresponding content map. When present, it points to a different object’s content entry.

How It Works

graph LR
    subgraph "Objects Map (o)"
        A["Object A<br/>(original)<br/>tid: --"]
        B["Object B<br/>(linked copy)<br/>tid: 'A'"]
    end

    subgraph "Text Map (txt)"
        T["'A' → Y.Text<br/>'Hello, world!'"]
    end

    A -- "uses own ID as key" --> T
    B -- "tid points to A" --> T

Object A uses its own ID (A) as the key in the txt map. Object B has tid: 'A', so it references the same Y.Text entry. Editing the text through either object updates the shared content.

True Copy vs Linked Copy

Ideallo provides two duplication operations with different content semantics:

True Copy (duplicateObject)

Creates a fully independent copy with its own content entry:

// Original object "aB3x_Qm7kL9p"
{ "t": "T", "xy": [100, 100], "wh": [200, 50] }
// txt map: "aB3x_Qm7kL9p" → "Hello"

// True copy "Xk2nR8vH_wYq" — independent content
{ "t": "T", "xy": [120, 120], "wh": [200, 50] }
// txt map: "Xk2nR8vH_wYq"  "Hello"  (separate Y.Text copy)

The true copy gets a new Y.Text entry with the same initial content but is fully independent – editing one does not affect the other. No tid field is stored.

Linked Copy (duplicateAsLinkedCopy)

Creates a copy that shares the original’s content entry:

// Original object "aB3x_Qm7kL9p"
{ "t": "T", "xy": [100, 100], "wh": [200, 50] }
// txt map: "aB3x_Qm7kL9p" → "Hello"

// Linked copy "Hw5_qT2mLkJx" — shared content
{ "t": "T", "xy": [120, 120], "wh": [200, 50], "tid": "aB3x_Qm7kL9p" }
// No new txt map entry  uses "aB3x_Qm7kL9p" from original

The linked copy has an explicit tid field pointing to the original’s content. Both objects render the same text, and editing through either one updates both.

Chained Linking

When duplicating a linked copy as another linked copy, the new copy points to the original source, not to the intermediate copy. This prevents chains of indirection:

// Object B is a linked copy of A: B.tid = 'A'
// Creating a linked copy of B:
duplicated.tid = existing.tid ?? objectId
// Result: C.tid = 'A' (not 'B')

All linked copies in a group always point directly to the same source content entry, keeping lookups to a single indirection.

Deletion and Orphan Handling

When an object is deleted, its associated content map entries are not deleted. This is for two reasons:

  1. CRDT tombstones: Deleting a Y.Map entry creates a tombstone that takes space anyway – no storage is reclaimed by also deleting content entries.
  2. Linked copies may still reference the content: Other objects might have tid/gid/pid pointing to the deleted object’s content.

Orphaned content entries (those with no remaining objects referencing them) are cleaned up during export or compaction operations.

Content Access Helpers

The CRDT module provides helper functions that automatically resolve content ID indirection:

Function Returns Description
getObjectYText(doc, objectId) Y.Text | undefined Resolves tid or falls back to objectId for Text/Sticky objects
getObjectYArray(doc, objectId) Y.Array<number> | undefined Resolves gid or falls back to objectId for Polygon objects
getObjectPathData(doc, objectId) string | undefined Resolves pid or falls back to objectId for Freehand objects

These helpers look up the stored object, check for a content ID field, and return the corresponding content map entry. Application code should always use these helpers rather than accessing content maps directly.

Export Format

Ideallo documents can be exported as self-contained JSON files for backup, sharing, and interoperability.

File Format

  • File extension: .ideallo
  • Content type: application/vnd.cloudillo.ideallo+json
  • Format version: 3.0.0
  • Encoding: UTF-8 JSON
v3 generic export format

Since format version 3.0.0, Ideallo uses the generic exportYDoc() serializer from @cloudillo/crdt. All Yjs types carry inline @T type markers and data keys match the raw CRDT shared type names. See v3 Generic Export Format for the full specification.

Envelope Structure

{
  "contentType": "application/vnd.cloudillo.ideallo+json",
  "appVersion": "0.5.0",
  "formatVersion": "3.0.0",
  "exportedAt": "2026-01-15T14:30:00.000Z",
  "data": {
    "m": { "@T": "M", ... },
    "o": { "@T": "M", ... },
    "r": [ "@T:A", ... ],
    "txt": { "@T": "M", ... },
    "geo": { "@T": "M", ... },
    "paths": { "@T": "M", ... }
  }
}

Envelope Fields

Field Type Description
contentType string Always "application/vnd.cloudillo.ideallo+json"
appVersion string Ideallo version that created this export
formatVersion string Export format version (currently "3.0.0")
exportedAt string ISO 8601 timestamp of export

Data Fields

Key @T Yjs Type Description
m M Y.Map Document metadata (name, background color, grid settings)
o M Y.Map<StoredObject> All objects keyed by ObjectId, using compact field names
r A Y.Array<string> Z-order array of ObjectId values (index 0 = backmost)
txt M Y.Map<Y.Text> Text content keyed by ObjectIdY.Text with text and delta fields
geo M Y.Map<Y.Array<number>> Polygon vertices keyed by ObjectId – flat [x, y, x, y, ...] arrays
paths M Y.Map<string> SVG path strings keyed by ObjectId
Text content preserves formatting in v3

The txt entries are Y.Text instances serialized with @T: "T", containing both text (plain text) and delta (Quill Delta operations). This preserves rich text formatting. In pre-v3 exports, text was a plain string from Y.Text.toJSON().

Numeric Precision

All numeric values are rounded to 3 decimal places in the export to produce cleaner output. For example, a position of [100.123456, 200.789012] becomes [100.123, 200.789].

Complete Example

A minimal whiteboard with a rectangle, text label, sticky note, and freehand path:

{
  "contentType": "application/vnd.cloudillo.ideallo+json",
  "appVersion": "0.5.0",
  "formatVersion": "3.0.0",
  "exportedAt": "2026-02-20T10:00:00.000Z",
  "data": {
    "m": {
      "@T": "M",
      "initialized": true,
      "name": "Project Planning",
      "backgroundColor": "#f8f9fa"
    },
    "o": {
      "@T": "M",
      "aB3x_Qm7kL9p": {
        "@T": "M",
        "t": "R",
        "xy": [200, 150],
        "wh": [300, 200],
        "fc": "#4a90d9",
        "sc": "#2d5a87",
        "cr": 8
      },
      "Xk2nR8vH_wYq": {
        "@T": "M",
        "t": "T",
        "xy": [250, 180],
        "wh": [200, 40],
        "sc": "n0"
      },
      "m4Jf_L1pZq8w": {
        "@T": "M",
        "t": "S",
        "xy": [550, 150],
        "wh": [200, 200],
        "fc": "#fff3cd"
      },
      "Hw5_qT2mLkJx": {
        "@T": "M",
        "t": "F",
        "xy": [100, 400],
        "wh": [180, 60],
        "sw": 3
      }
    },
    "r": [
      "@T:A",
      "aB3x_Qm7kL9p",
      "Xk2nR8vH_wYq",
      "m4Jf_L1pZq8w",
      "Hw5_qT2mLkJx"
    ],
    "txt": {
      "@T": "M",
      "Xk2nR8vH_wYq": {
        "@T": "T",
        "text": "Project Title",
        "delta": [{ "insert": "Project Title" }]
      },
      "m4Jf_L1pZq8w": {
        "@T": "T",
        "text": "Remember to update the timeline",
        "delta": [{ "insert": "Remember to update the timeline" }]
      }
    },
    "geo": { "@T": "M" },
    "paths": {
      "@T": "M",
      "Hw5_qT2mLkJx": "M 0 30 C 20 0 40 60 60 30 C 80 0 100 60 120 30 C 140 0 160 60 180 30"
    }
  }
}

In this example:

  • Rectangle (aB3x_Qm7kL9p): Blue filled rectangle with rounded corners, custom stroke color
  • Text (Xk2nR8vH_wYq): Text label positioned inside the rectangle, using default stroke color
  • Sticky (m4Jf_L1pZq8w): Yellow sticky note with reminder text
  • Freehand (Hw5_qT2mLkJx): Wavy freehand path with custom stroke width, SVG path data stored in paths map
  • The r array lists objects from back to front – the rectangle is behind everything, and the freehand path is on top
  • All Y.Map entries carry "@T": "M", the z-order array carries "@T:A", and text content uses "@T": "T" with text and delta fields

Prezillo Format

Complete format specification for Prezillo, Cloudillo’s collaborative presentation editor.

Overview

Prezillo is a real-time collaborative presentation editor that supports slides, layers, groups, rich text, shapes, images, connectors, templates, a palette system, and style inheritance. Documents are stored as Yjs CRDT structures for conflict-free concurrent editing.

  • Content type: application/vnd.cloudillo.prezillo+json
  • Format version: 3.0.0
  • File extension: .prezillo

Design Philosophy

  • Compact field names: All stored types use short keys (t, xy, wh, si) to minimize CRDT sync overhead. This section documents the compact names directly — they map 1:1 to what you see in the code and on the wire.
  • Separate maps by purpose: Objects, containers, views, styles, and templates each get their own top-level Yjs shared type. This enables targeted observers, type-safe access, and efficient partial sync.
  • ID-based storage: All entities are stored in Y.Map keyed by random IDs. Ordering is maintained separately in Y.Array structures. This prevents the CRDT data-loss pitfall of storing complex objects directly in arrays.

Document Architecture

graph LR
    Doc[Y.Doc]

    subgraph Content
        o["o (Y.Map&lt;StoredObject&gt;)<br/>Objects"]
        c["c (Y.Map&lt;StoredContainer&gt;)<br/>Containers"]
        rt["rt (Y.Map&lt;Y.Text&gt;)<br/>Rich Texts"]
    end

    subgraph Hierarchy
        r["r (Y.Array&lt;ChildRef&gt;)<br/>Root Children"]
        ch["ch (Y.Map&lt;Y.Array&lt;ChildRef&gt;&gt;)<br/>Container Children"]
    end

    subgraph Presentation
        v["v (Y.Map&lt;StoredView&gt;)<br/>Views"]
        vo["vo (Y.Array&lt;string&gt;)<br/>View Order"]
        m["m (Y.Map)<br/>Metadata"]
    end

    subgraph Styling
        st["st (Y.Map&lt;StoredStyle&gt;)<br/>Styles"]
        pl["pl (Y.Map&lt;StoredPalette&gt;)<br/>Palette"]
    end

    subgraph Templates
        tpl["tpl (Y.Map&lt;StoredTemplate&gt;)<br/>Templates"]
        tpo["tpo (Y.Map&lt;Y.Array&lt;string&gt;&gt;)<br/>Template Prototypes"]
    end

    Doc --> Content
    Doc --> Hierarchy
    Doc --> Presentation
    Doc --> Styling
    Doc --> Templates

Quick Reference

Map Key Yjs Type Purpose
o Y.Map<StoredObject> All drawable objects (rects, text, images, connectors, etc.)
c Y.Map<StoredContainer> Layers and groups
r Y.Array<ChildRef> Root-level children (top-level layers/objects)
ch Y.Map<Y.Array<ChildRef>> Children arrays per container
v Y.Map<StoredView> Views (slides/pages) with dimensions and backgrounds
vo Y.Array<string> View ordering (presentation sequence)
m Y.Map Document metadata (name, default dimensions, grid settings)
rt Y.Map<Y.Text> Rich text content for text objects (Quill Delta format)
st Y.Map<StoredStyle> Global style definitions (shape and text styles)
pl Y.Map<StoredPalette> Color palette (single entry keyed by 'default')
tpl Y.Map<StoredTemplate> Templates (background, dimensions, snap guides)
tpo Y.Map<Y.Array<string>> Template prototype objects: templateId → objectId array

Detailed Documentation

See Also

Subsections of Prezillo Format

Document Structure

The top-level CRDT document structure, initialization defaults, ID system, and metadata.

Top-Level Shared Types

A Prezillo document is a Y.Doc with 12 named shared types:

Map Key Yjs Type Description
o Y.Map<StoredObject> All drawable objects keyed by ObjectId
c Y.Map<StoredContainer> Layers and groups keyed by ContainerId
r Y.Array<ChildRef> Root-level children (ordered)
ch Y.Map<Y.Array<ChildRef>> Children arrays keyed by ContainerId
v Y.Map<StoredView> Views (slides/pages) keyed by ViewId
vo Y.Array<string> View order (presentation sequence of ViewId strings)
m Y.Map Document metadata
rt Y.Map<Y.Text> Rich text content keyed by ObjectId (Quill Delta)
st Y.Map<StoredStyle> Global style definitions keyed by StyleId
pl Y.Map<StoredPalette> Palette (single entry keyed by 'default')
tpl Y.Map<StoredTemplate> Templates keyed by TemplateId
tpo Y.Map<Y.Array<string>> Template prototype objects: TemplateIdObjectId[]
Why compact map keys?

Map keys like o, c, r are used instead of objects, containers, rootChildren because these keys appear in every Yjs sync message. Shorter keys reduce wire overhead during real-time collaboration without affecting readability — this documentation provides the complete mapping.

Accessing the Document

import * as Y from 'yjs'

const yDoc = new Y.Doc()

// Access each shared type by its map key
const objects    = yDoc.getMap('o')     // StoredObject entries
const containers = yDoc.getMap('c')     // StoredContainer entries
const root       = yDoc.getArray('r')   // Root ChildRef array
const children   = yDoc.getMap('ch')    // Per-container children
const views      = yDoc.getMap('v')     // StoredView entries
const viewOrder  = yDoc.getArray('vo')  // ViewId ordering
const meta       = yDoc.getMap('m')     // Metadata
const richTexts  = yDoc.getMap('rt')    // Y.Text entries
const styles     = yDoc.getMap('st')    // StoredStyle entries
const palette    = yDoc.getMap('pl')    // StoredPalette
const templates  = yDoc.getMap('tpl')   // StoredTemplate entries
const tplProtos  = yDoc.getMap('tpo')   // Template prototype arrays

ID System

All entity IDs use 72-bit entropy encoded as 12 base64url characters, generated client-side via crypto.getRandomValues().

Six branded types prevent accidental ID mixing at compile time:

Branded Type Used In Example
ObjectId o map keys, ChildRef[1] when [0] is 0 "aB3x_Qm7kL9p"
ContainerId c map keys, ch map keys, ChildRef[1] when [0] is 1 "Xk2nR8vH_wYq"
ViewId v map keys, vo array entries, object vi field "m4Jf_L1pZq8w"
StyleId st map keys, object si/ti fields, style p field "Nz9cK_vW3xRb"
RichTextId rt map keys (same value as ObjectId for text objects) "aB3x_Qm7kL9p"
TemplateId tpl map keys, tpo map keys, view tpl field "Hw5_qT2mLkJx"

ChildRef Convention

The ChildRef tuple encodes a reference that can point to either an object or a container:

type ChildRef = [0 | 1, string]
//               ^       ^
//               |       └── ObjectId or ContainerId
//               └── 0 = object, 1 = container

Examples:

  • [0, "aB3x_Qm7kL9p"] — references an object
  • [1, "Xk2nR8vH_wYq"] — references a container (layer or group)

ChildRef arrays are used in the root children (r) and per-container children (ch) to maintain draw order.

Metadata

The m map stores document-level settings as individual key-value entries:

Key Type Default Description
name string "Untitled Presentation" Document name
defaultViewWidth number 1920 Default width for new views
defaultViewHeight number 1080 Default height for new views
gridSize number Grid spacing in pixels (if grid enabled)
snapToGrid boolean Whether objects snap to grid
snapToObjects boolean Whether objects snap to other objects

Document Initialization

When a new empty document is created, the following defaults are established in a single Yjs transaction:

  1. Default layer: A container of type 'L' named "Layer 1" with expanded state (x: true), added to root children as [1, layerId]
  2. Default view: A view named "Page 1" at position (0, 0) with dimensions 1920×1080, white background, border visible
  3. Metadata: defaultViewWidth: 1920, defaultViewHeight: 1080, name: "Untitled Presentation"
  4. Default palette: The built-in palette with background, text, 6 accent colors, and 4 gradients (see Styling and Palette)
  5. Default styles: 10 built-in styles — 6 shape styles (Default Shape, Primary, Secondary, Accent, Outline, Connector) and 4 text styles (Default Text, Heading, Body, Caption) with parent-child inheritance chains

Rich Text Storage

Text objects (type 'T') store their formatted content as Y.Text entries in the rt map, keyed by the same ObjectId used in the o map. The Y.Text uses Quill Delta format for rich text operations (bold, italic, links, etc.).

A legacy tx field on the stored object held plain text in older documents. On load, migration code automatically converts tx strings to Y.Text entries in rt and removes the tx field.

Separate map for rich text

Rich text content is stored in the rt map, not embedded in the object data in o. This is because Y.Text is a Yjs shared type that needs its own identity for collaborative editing — it cannot be stored as a plain JSON value inside a Y.Map entry.

Object Types

Prezillo supports 14 object types, all sharing a common set of base fields with type-specific extensions.

Base Fields

Every stored object has the following fields defined by StoredObjectBase:

Field Type Default Description
t ObjectTypeCode (required) Object type discriminant
p string Parent ContainerId (omitted if root-level)
vi string ViewId — if set, xy is relative to this view’s origin
proto string Prototype ObjectId — inherit properties from this object
xy [number, number] (required) Position [x, y] — global canvas coords or view-relative if vi is set
wh [number, number] (required) Dimensions [width, height]
r number 0 Rotation in degrees (omitted if 0)
pv [number, number] [0.5, 0.5] Pivot point relative (0–1), center is default
o number 1 Opacity (0–1, omitted if 1)
v false true Visible — only stored when false
k true false Locked — only stored when true
hid true false Hidden — visible in editor at 50% opacity, invisible in presentation mode
n string User-assigned name
si string Shape StyleId reference
ti string Text StyleId reference
s ShapeStyle Inline shape style overrides (applied on top of si chain)
ts TextStyle Inline text style overrides (applied on top of ti chain)

Object Type Codes

Code Name Description
R Rect Rectangle with optional corner radius
E Ellipse Ellipse/circle
L Line Two-point line with optional arrows
P Path Freeform SVG path
G Polygon Multi-point polygon or polyline
T Text Rich text box
I Image Uploaded image
M Embed Embedded media (iframe, video, audio)
C Connector Object-to-object connector with routing
Q QR Code QR code generator
F Poll Frame Interactive voting element
Tg Table Grid Visual grid layout with snap points
S Symbol Reference to symbol library
V State Variable Displays dynamic runtime values

Per-Type Fields

Rect (R)

Field Type Default Description
cr number | [number, number, number, number] Corner radius — single value for uniform, or [topLeft, topRight, bottomRight, bottomLeft]

Ellipse (E)

No additional fields beyond the base.

Line (L)

Field Type Default Description
pts [[number, number], [number, number]] (required) Start and end points relative to bounding box
sa ArrowDef Start arrow
ea ArrowDef End arrow

Path (P)

Field Type Default Description
d string (required) SVG path data (M, L, C, Z commands, etc.)

Polygon (G)

Field Type Default Description
pts [number, number][] (required) Array of points
cl boolean Closed polygon (if true, last point connects to first)

Text (T)

Field Type Default Description
tx string Legacy plain text (migrated to rt map on load)
mh number Minimum height — original height at creation for auto-sizing

Rich text content is stored as a Y.Text entry in the rt map (keyed by the same ObjectId), using Quill Delta format. See Document Structure > Rich Text Storage.

Image (I)

Field Type Default Description
fid string (required) File ID from the Cloudillo media/file system

Embed (M)

Field Type Default Description
mt 'iframe' | 'video' | 'audio' (required) Media type
src string (required) Source URL

Connector (C)

Field Type Default Description
so_ string Start ObjectId (uses so_ to avoid conflict with style override s.o)
sa AnchorPoint Start anchor point
eo string End ObjectId
ea AnchorPoint End anchor point
wp [number, number][] Waypoints for manual routing
rt RoutingCode Routing algorithm
sar ArrowDef Start arrow
ear ArrowDef End arrow

QR Code (Q)

Field Type Default Description
url string (required) URL to encode
ecl QrErrorCorrectionLevel 'M' Error correction level
fg string '#000000' Foreground color (hex)
bg string '#ffffff' Background color (hex)

Poll Frame (F)

Field Type Default Description
sh 'R' | 'E' 'R' Frame shape: R=rectangle, E=ellipse
lb string Label text to display on the frame

Table Grid (Tg)

Field Type Default Description
c number (required) Column count
rw number (required) Row count
cw number[] Column widths as proportions (0–1, equal if omitted)
rh number[] Row heights as proportions (0–1, equal if omitted)

Border styling uses the inherited ShapeStyle (s field) for line color and thickness.

Symbol (S)

Field Type Default Description
sid string (required) Symbol ID — reference to the symbol library

State Variable (V)

Field Type Default Description
var StateVarTypeCode (required) Which variable to display

Currently defined state variable type codes:

Code Name Description
u Users Displays connected user information

Supporting Types

BlendModeCode

Code CSS Equivalent Description
N normal Normal (default)
M multiply Multiply
S screen Screen
O overlay Overlay
D darken Darken
L lighten Lighten
CD color-dodge Color dodge
CB color-burn Color burn
HL hard-light Hard light
SL soft-light Soft light
DF difference Difference
EX exclusion Exclusion

ArrowTypeCode

Code Name
N None
A Arrow (open)
T Triangle (filled)
C Circle
D Diamond
B Bar

ArrowDef

A tuple defining an arrow marker:

type ArrowDef = [ArrowTypeCode, number?, boolean?]
//               ^              ^        ^
//               |              |        └── filled (default: depends on type)
//               |              └── size multiplier (default: 1)
//               └── arrow type code

AnchorPoint

An anchor point can be a named code or a relative position:

type AnchorPoint = AnchorPointCode | [number, number]
Code Position
c Center
t Top center
b Bottom center
l Left center
r Right center
tl Top left
tr Top right
bl Bottom left
br Bottom right
a Auto (closest point)

When specified as [number, number], values are relative positions (0–1) within the object’s bounding box.

RoutingCode

Code Name Description
S Straight Direct line between anchors
O Orthogonal Axis-aligned segments with right-angle turns
C Curved Smooth Bezier curve

QrErrorCorrectionLevel

Code Recovery Capacity
L ~7%
M ~15% (default)
Q ~25%
H ~30%

Containers and Hierarchy

Containers organize objects into layers and groups, forming a tree hierarchy with ordered children.

Container Types

Code Name Purpose
L Layer Top-level organizational container (like Photoshop layers). Typically added to root children.
G Group User-created group of objects. Can nest inside layers or other groups.

StoredContainer Fields

Field Type Default Description
t 'L' | 'G' (required) Container type
p string Parent ContainerId (omitted if root-level)
n string User-assigned name (e.g., "Layer 1", "Header Group")
xy [number, number] (required) Position [x, y]
r number 0 Rotation in degrees
sc [number, number] Scale [scaleX, scaleY]
o number 1 Opacity (0–1)
bm BlendModeCode 'N' Blend mode
v false true Visible — only stored when false
k true false Locked — only stored when true
x boolean Expanded in the UI layer panel

Hierarchy Model

The document hierarchy is a tree rooted at the r (root children) array:

graph TB
    Root["r (Root Children)<br/>Y.Array&lt;ChildRef&gt;"]

    Root -->|"[1, id1]"| L1["Layer 1<br/>(Container, t='L')"]
    Root -->|"[1, id2]"| L2["Layer 2<br/>(Container, t='L')"]

    L1 -->|"[0, id3]"| R1["Rectangle<br/>(Object, t='R')"]
    L1 -->|"[1, id4]"| G1["Group<br/>(Container, t='G')"]
    L1 -->|"[0, id5]"| T1["Text Box<br/>(Object, t='T')"]

    G1 -->|"[0, id6]"| E1["Ellipse<br/>(Object, t='E')"]
    G1 -->|"[0, id7]"| I1["Image<br/>(Object, t='I')"]

    L2 -->|"[0, id8]"| C1["Connector<br/>(Object, t='C')"]

ChildRef System

Children are referenced using ChildRef tuples that discriminate between objects and containers:

type ChildRef = [0 | 1, string]
Tuple Meaning Look Up In
[0, objectId] References an object o map
[1, containerId] References a container c map

This allows a single ordered array to contain a mix of direct objects and nested containers.

Children Arrays

Container children are stored in the ch map, not embedded in the container data:

// ch: Y.Map<Y.Array<ChildRef>>
// Key: ContainerId
// Value: Y.Array of ChildRef tuples

// Example: get children of a layer
const layerChildren = doc.ch.get(layerId)  // Y.Array<ChildRef>

// Iterate children
for (const [type, id] of layerChildren) {
  if (type === 0) {
    const obj = doc.o.get(id)      // StoredObject
  } else {
    const container = doc.c.get(id) // StoredContainer
  }
}
Children are separate from container data

The ch map stores Y.Array<ChildRef> entries — these are Yjs shared types that enable collaborative reordering. They are not embedded in the StoredContainer JSON in the c map. Always look up children via doc.ch.get(containerId), never by reading a children property from the container object.

Z-Ordering

The position of a ChildRef within its parent’s children array determines draw order:

  • Index 0 is drawn first (bottom/back)
  • Last index is drawn last (top/front)

Reordering is done by moving entries within the Y.Array, which Yjs handles correctly for concurrent edits.

Parent References

Objects and containers store a p (parent) field pointing to their containing ContainerId. This enables:

  • Quick lookup of an item’s parent without scanning children arrays
  • Efficient tree traversal in both directions

When an item is at root level (directly in r), the p field is omitted.

Views

Views represent slides, pages, or artboards — named rectangular regions on the infinite canvas.

What Views Represent

A view defines a visible area on the canvas that corresponds to a slide in presentation mode or a page in print/export. Objects can be associated with a view via their vi field, making their coordinates relative to that view’s origin.

StoredView Fields

Field Type Default Description
name string (required) Display name (e.g., "Page 1", "Title Slide")
x number (required) X position on the infinite canvas
y number (required) Y position on the infinite canvas
width number 1920 View width in pixels
height number 1080 View height in pixels
backgroundColor string Background color (hex)
backgroundGradient StoredBackgroundGradient Background gradient (takes precedence over backgroundColor)
backgroundImage string Background image file ID
backgroundFit 'contain' | 'cover' | 'fill' | 'tile' How background image is sized
showBorder boolean Whether to show the view border in the editor
transition object Slide transition for presentation mode
notes string Speaker notes (plain text)
hidden boolean If true, visible in editor at 50% opacity but invisible in presentation mode
duration number Auto-advance duration in seconds (presentation mode)
tpl string TemplateId reference — view inherits template background when own background fields are absent
View fields use full names

Unlike objects and containers, view fields use human-readable names (backgroundColor, not bc). This is because views are low-volume data (typically dozens, not thousands) — readability was prioritized over wire efficiency.

View Ordering

The vo array holds ViewId strings in presentation order:

// vo: Y.Array<string>
const viewOrder = doc.vo.toArray()  // ["m4Jf_L1pZq8w", "Xk2n_R8vHwYq", ...]

// First view in presentation
const firstViewId = doc.vo.get(0)
const firstView = doc.v.get(firstViewId)

// Reorder: move view from index 2 to index 0
doc.vo.delete(2, 1)
doc.vo.insert(0, [viewId])

Page-Relative Coordinates

When an object’s vi field is set, its xy position is interpreted relative to the view’s origin (x, y), not the global canvas:

Global position = (view.x + object.xy[0], view.y + object.xy[1])

This allows slides to be repositioned on the canvas without affecting the relative positions of their objects.

Background System

View backgrounds resolve in priority order:

  1. backgroundGradient — if present, renders a gradient background
  2. backgroundImage — if present (and no gradient), renders an image
  3. backgroundColor — if present (and no gradient/image), renders a solid color
  4. Template background — if tpl is set and the view’s own background fields are absent, the template’s background is used

StoredBackgroundGradient

Field Type Default Description
gt 'l' | 'r' Gradient type: 'l'=linear, 'r'=radial
ga number Angle in degrees (linear gradients only)
gx number Center X position 0–1 (radial gradients only)
gy number Center Y position 0–1 (radial gradients only)
gs [string, number][] Color stops: [color, position] pairs where position is 0–1

Transitions

The transition object defines how a slide enters during presentation mode:

Field Type Default Description
type 'none' | 'fade' | 'slide' | 'zoom' | 'push' 'none' Transition effect
duration number Duration in milliseconds
direction 'left' | 'right' | 'up' | 'down' Direction (for slide and push types)

Default Dimensions

New documents use 1920×1080 (Full HD landscape) as the default view size. This can be changed via the defaultViewWidth and defaultViewHeight metadata fields.

Hidden Views

When a view has hidden: true:

  • In the editor: The view is displayed at 50% opacity to distinguish it from active views
  • In presentation mode: The view is skipped entirely
  • In view order: The view remains in the vo array at its current position

Styling and Palette

Prezillo uses a cascading style system with global style definitions, inline overrides, and a document-wide color palette.

Style Resolution Cascade

Styles resolve in layers, with later layers overriding earlier ones:

graph LR
    D["Built-in Defaults"] --> S["Style Chain<br/>(si/ti → parent p)"]
    S --> O["Inline Override<br/>(s/ts on object)"]

    style D fill:#e8e8e8,stroke:#999
    style S fill:#d4e6f1,stroke:#2980b9
    style O fill:#d5f5e3,stroke:#27ae60
  1. Built-in defaults: Hard-coded fallback values (e.g., fill #cccccc, strokeWidth 1)
  2. Style chain: The object’s si (shape style) or ti (text style) references a global StoredStyle in the st map. That style may have a parent (p field), forming a chain. Properties are inherited up the chain until a value is found.
  3. Inline overrides: The object’s s (shape style) or ts (text style) fields contain direct property overrides that take highest precedence.

StoredStyle Fields

Global styles are stored in the st map, keyed by StyleId.

Field Type Description
n string Style name (user-visible, e.g., "Primary", "Heading")
t 'S' | 'T' Type: S=shape style, T=text style
p string Parent StyleId (for inheritance chain)

Shape style properties (when t='S'):

Field Type Description
f ColorValue Fill color (hex string or palette reference)
fo number Fill opacity (0–1)
s ColorValue Stroke color (hex string or palette reference)
sw number Stroke width
so number Stroke opacity (0–1)
sd string Stroke dasharray (SVG format, e.g., "5,5")
sc 'butt' | 'round' | 'square' Stroke linecap
sj 'miter' | 'round' | 'bevel' Stroke linejoin
sh [number, number, number, ColorValue] Shadow: [offsetX, offsetY, blur, color]
cr number Corner radius (for rects using this style)

Text style properties (when t='T'):

Field Type Description
ff string Font family (e.g., "Inter, system-ui")
fs number Font size in pixels
fw 'normal' | 'bold' | number Font weight
fi boolean Font italic
td 'u' | 's' Text decoration: u=underline, s=strikethrough
fc ColorValue Text fill color (hex string or palette reference)
ta 'l' | 'c' | 'r' | 'j' Text align: left/center/right/justify
va 't' | 'm' | 'b' Vertical align: top/middle/bottom
lh number Line height multiplier
ls number Letter spacing in pixels
lb string List bullet character (UTF-8, e.g., "•")

ShapeStyle and TextStyle (Inline)

The s and ts fields on objects use the same property names as StoredStyle, but without the n, t, and p fields. They contain only the properties being overridden.

// Example: object with style reference + inline override
{
  t: 'R',
  xy: [100, 200],
  wh: [300, 150],
  si: 'Nz9cK_vW3xRb',           // references "Primary" style
  s: { f: '#ff0000', sw: 3 }     // overrides fill and stroke width
}

Palette System

The palette provides a set of named color and gradient slots that can be referenced by any style or object. This allows changing the document’s color theme by updating a single palette entry.

StoredPalette Structure

The palette is stored as a single entry in the pl map under the key 'default':

Field Type Description
n string Palette name (e.g., "Default", "Office Blue")
bg StoredPaletteColor Background color slot
tx StoredPaletteColor Text color slot
a1a6 StoredPaletteColor Accent color slots 1–6
g1g4 StoredBackgroundGradient Gradient slots 1–4

Where StoredPaletteColor is simply { c: string } (a hex color).

Palette Slot Codes

Color slots:

Code Name
bg Background
tx Text
a1 Accent 1
a2 Accent 2
a3 Accent 3
a4 Accent 4
a5 Accent 5
a6 Accent 6

Gradient slots:

Code Name
g1 Gradient 1
g2 Gradient 2
g3 Gradient 3
g4 Gradient 4

StoredPaletteRef

A palette reference replaces a literal color value in any ColorValue field:

Field Type Description
pi PaletteSlot Palette slot code (e.g., 'a1', 'tx', 'g2')
o number Opacity override (0–1, default 1)
t number Tint/shade adjustment (-1 to 1, 0 = original). Positive values tint toward white, negative values shade toward black.

ColorValue Union

Any color field (f, s, fc, shadow color, etc.) accepts either a raw hex string or a palette reference:

type ColorValue = string | StoredPaletteRef

Examples:

// Raw hex color
{ f: '#4a90d9' }

// Palette reference: accent 1, full opacity
{ f: { pi: 'a1' } }

// Palette reference: text color at 50% opacity, slightly lighter
{ fc: { pi: 'tx', o: 0.5, t: 0.2 } }

When a palette slot is updated, all objects and styles referencing that slot automatically reflect the new color — resolution happens at render time.

Default Styles

A new document is initialized with these styles:

Shape styles:

Name Fill Stroke Stroke Width Notes
Default Shape #e0e0e0 #999999 1
Primary #4a90d9 #2d5a87 2 Corner radius 8
Secondary #5cb85c #3d8b3d (inherited) Parent: Primary
Accent #f0ad4e #c87f0a (inherited) Parent: Primary
Outline none #333333 2 Dashed (5,5)
Connector none #666666 2

Text styles:

Name Font Family Size Weight Color Notes
Default Text system-ui 16 normal #333333
Heading Inter, system-ui 48 bold #1a1a2e
Body Inter, system-ui 18 normal #333333 Line height 1.5
Caption (inherited) 14 (inherited) #666666 Parent: Body, italic

Default Palette

Slot Color Description
bg #ffffff Background (white)
tx #333333 Text (dark gray)
a1 #4a90d9 Accent 1 (blue)
a2 #5cb85c Accent 2 (green)
a3 #f0ad4e Accent 3 (orange)
a4 #d9534f Accent 4 (red)
a5 #9b59b6 Accent 5 (purple)
a6 #1abc9c Accent 6 (teal)
g1 Blue gradient (180°) #4a90d9#2d5a87
g2 Green gradient (135°) #5cb85c#3d8b3d
g3 Orange radial gradient #f0ad4e#c87f0a
g4 Purple gradient (90°) #667eea#764ba2

See Also

Templates and Prototypes

Templates provide reusable slide backgrounds, dimensions, snap guides, and prototype objects for consistent presentation design.

What Templates Provide

A template defines:

  • Default dimensions for views using this template
  • Background (color, gradient, and/or image) that views inherit when their own background fields are absent
  • Snap guides for aligning objects to a consistent layout grid
  • Prototype objects that views can instantiate with single-level property inheritance

StoredTemplate Fields

Templates are stored in the tpl map, keyed by TemplateId.

Field Type Default Description
n string (required) Template name (e.g., "Title Slide", "Two Column")
w number (required) Default page width
h number (required) Default page height
bc string Background color (hex)
bg StoredBackgroundGradient Background gradient
bi string Background image file ID
bf 'contain' | 'cover' | 'fill' | 'tile' Background image fit mode
sg StoredSnapGuide[] Snap guides for object alignment

Snap Guides

Snap guides are visual and functional alignment lines defined by templates.

StoredSnapGuide

Field Type Description
d 'h' | 'v' Direction: h=horizontal, v=vertical
p number Position: percentage (0–1) or absolute pixels
a boolean If true, p is in absolute pixels; otherwise p is a percentage of the view dimension

Example: A centered two-column layout with margin guides:

{
  "sg": [
    { "d": "v", "p": 0.1 },
    { "d": "v", "p": 0.5 },
    { "d": "v", "p": 0.9 },
    { "d": "h", "p": 0.1 },
    { "d": "h", "p": 0.9 }
  ]
}

Template-to-View Relationship

A view references a template via its tpl field:

// View with template
{
  name: 'Title Slide',
  x: 0, y: 0,
  width: 1920, height: 1080,
  tpl: 'Hw5_qT2mLkJx'  // TemplateId
  // No backgroundColor → inherits from template
}

Background resolution: The view’s own background fields (backgroundColor, backgroundGradient, backgroundImage) take precedence. When absent, the template’s background (bc, bg, bi) is used as a fallback.

Prototype System

Templates can define prototype objects — objects that serve as editable defaults for views using the template.

How It Works

graph LR
    T["Template<br/>(tpl map)"] -->|"defines"| P["Prototype Object<br/>(o map, referenced via tpo)"]
    P -->|"proto field"| I["Instance Object<br/>(o map, in a view)"]

    style T fill:#d4e6f1,stroke:#2980b9
    style P fill:#d5f5e3,stroke:#27ae60
    style I fill:#fdebd0,stroke:#e67e22
  1. Template prototype objects are stored in the tpo map: TemplateId → Y.Array<ObjectId>. These are regular objects stored in the o map, but logically belonging to a template.
  2. Instance objects reference a prototype via their proto field (an ObjectId pointing to the prototype).
  3. Property inheritance: Instance objects inherit all properties from their prototype. Only overridden properties need to be stored on the instance.

Storage

// tpo: Y.Map<Y.Array<string>>
// Key: TemplateId
// Value: array of ObjectIds that are prototypes for this template

const protoIds = doc.tpo.get(templateId)  // Y.Array<string>
// e.g., ["aB3x_Qm7kL9p", "Xk2nR8vH_wYq"]

Property Override Tracking

When a user modifies an instance:

  • The changed properties are stored directly on the instance object
  • Unchanged properties continue to resolve from the prototype
  • At render time, instance properties override prototype properties (similar to the style cascade)
Single-level inheritance

Prototype inheritance is deliberately single-level — an instance references one prototype, and prototypes do not chain to other prototypes. This keeps conflict resolution predictable: when two users concurrently modify a prototype and an instance, the CRDT can resolve the conflict without ambiguity across multiple inheritance levels.

Lock/Unlock

Prototype instances can be locked or unlocked:

  • Locked (k: true): The instance cannot be individually edited; it reflects prototype changes automatically
  • Unlocked: The instance can be edited independently, with changes stored as overrides

Export Format

Prezillo documents can be exported as self-contained JSON files for backup, sharing, and interoperability.

File Format

  • File extension: .prezillo
  • Content type: application/vnd.cloudillo.prezillo+json
  • Format version: 3.0.0
  • Encoding: UTF-8 JSON
v3 generic export format

Since format version 3.0.0, Prezillo uses the generic exportYDoc() serializer from @cloudillo/crdt. All Yjs types carry inline @T type markers and data keys match the raw CRDT shared type names. See v3 Generic Export Format for the full specification.

Envelope Structure

{
  "contentType": "application/vnd.cloudillo.prezillo+json",
  "appVersion": "0.5.0",
  "formatVersion": "3.0.0",
  "exportedAt": "2026-01-15T14:30:00.000Z",
  "data": {
    "m": { "@T": "M", ... },
    "o": { "@T": "M", ... },
    "c": { "@T": "M", ... },
    "r": [ "@T:A", ... ],
    "ch": { "@T": "M", ... },
    "v": { "@T": "M", ... },
    "vo": [ "@T:A", ... ],
    "rt": { "@T": "M", ... },
    "st": { "@T": "M", ... },
    "tpl": { "@T": "M", ... },
    "tpo": { "@T": "M", ... },
    "pl": { "@T": "M", ... }
  }
}

Envelope Fields

Field Type Description
contentType string Always "application/vnd.cloudillo.prezillo+json"
appVersion string Prezillo version that created this export
formatVersion string Export format version (currently "3.0.0")
exportedAt string ISO 8601 timestamp of export

Data Fields

Key @T Yjs Type Description
m M Y.Map Document metadata (name, default dimensions, grid settings)
o M Y.Map<StoredObject> All objects keyed by ObjectId
c M Y.Map<StoredContainer> All containers keyed by ContainerId
r A Y.Array<ChildRef> Root-level children in order
ch M Y.Map<Y.Array<ChildRef>> Children per container, keyed by ContainerId
v M Y.Map<StoredView> All views keyed by ViewId
vo A Y.Array<string> ViewId strings in presentation order
rt M Y.Map<Y.Text> Rich text content keyed by ObjectId
st M Y.Map<StoredStyle> All styles keyed by StyleId
tpl M Y.Map<StoredTemplate> All templates keyed by TemplateId
tpo M Y.Map<Y.Array<string>> Prototype object arrays keyed by TemplateId
pl M Y.Map<StoredPalette> The document palette (single entry keyed by 'default')

Rich Text Serialization

Rich text entries in the rt map are Y.Text instances, serialized with the @T: "T" marker containing both the plain text and the full Quill Delta operations:

{
  "@T": "M",
  "aB3x_Qm7kL9p": {
    "@T": "T",
    "text": "Hello, world!",
    "delta": [
      { "insert": "Hello, " },
      { "insert": "world!", "attributes": { "bold": true } }
    ]
  }
}
Field Type Description
@T "T" Type marker indicating a Y.Text instance
text string Plain text content (for search/indexing)
delta object[] Quill Delta operations (for restoring formatting)

Numeric Precision

All numeric values are rounded to 3 decimal places in the export to produce cleaner output. For example, a position of [100.123456, 200.789012] becomes [100.123, 200.789].

Complete Example

A minimal presentation with 2 slides, one text box, and one rectangle:

{
  "contentType": "application/vnd.cloudillo.prezillo+json",
  "appVersion": "0.5.0",
  "formatVersion": "3.0.0",
  "exportedAt": "2026-01-15T14:30:00.000Z",
  "data": {
    "m": {
      "@T": "M",
      "name": "My Presentation",
      "defaultViewWidth": 1920,
      "defaultViewHeight": 1080
    },
    "o": {
      "@T": "M",
      "aB3x_Qm7kL9p": {
        "@T": "M",
        "t": "T",
        "vi": "m4Jf_L1pZq8w",
        "p": "Xk2nR8vH_wYq",
        "xy": [560, 440],
        "wh": [800, 200],
        "si": "Tz8_kLmNqR2v",
        "mh": 200
      },
      "Hw5_qT2mLkJx": {
        "@T": "M",
        "t": "R",
        "vi": "nP7r_S3wKxUe",
        "p": "Xk2nR8vH_wYq",
        "xy": [460, 290],
        "wh": [1000, 500],
        "si": "Qp4_rW9xJdLm",
        "s": { "@T": "M", "f": "#4a90d9", "sw": 2 },
        "cr": 12
      }
    },
    "c": {
      "@T": "M",
      "Xk2nR8vH_wYq": {
        "@T": "M",
        "t": "L",
        "n": "Layer 1",
        "xy": [0, 0],
        "x": true
      }
    },
    "r": [
      "@T:A",
      ["@T:A", 1, "Xk2nR8vH_wYq"]
    ],
    "ch": {
      "@T": "M",
      "Xk2nR8vH_wYq": [
        "@T:A",
        ["@T:A", 0, "aB3x_Qm7kL9p"],
        ["@T:A", 0, "Hw5_qT2mLkJx"]
      ]
    },
    "v": {
      "@T": "M",
      "m4Jf_L1pZq8w": {
        "@T": "M",
        "name": "Title Slide",
        "x": 0,
        "y": 0,
        "width": 1920,
        "height": 1080,
        "backgroundColor": "#ffffff",
        "showBorder": true
      },
      "nP7r_S3wKxUe": {
        "@T": "M",
        "name": "Content",
        "x": 2020,
        "y": 0,
        "width": 1920,
        "height": 1080,
        "backgroundColor": "#f5f5f5",
        "showBorder": true
      }
    },
    "vo": ["@T:A", "m4Jf_L1pZq8w", "nP7r_S3wKxUe"],
    "rt": {
      "@T": "M",
      "aB3x_Qm7kL9p": {
        "@T": "T",
        "text": "Welcome to My Presentation",
        "delta": [
          { "insert": "Welcome to " },
          { "insert": "My Presentation", "attributes": { "bold": true } },
          { "insert": "\n" }
        ]
      }
    },
    "st": {
      "@T": "M",
      "Tz8_kLmNqR2v": {
        "@T": "M",
        "n": "Heading",
        "t": "T",
        "ff": "Inter, system-ui",
        "fs": 48,
        "fw": "bold",
        "fc": "#1a1a2e"
      },
      "Qp4_rW9xJdLm": {
        "@T": "M",
        "n": "Primary",
        "t": "S",
        "f": "#4a90d9",
        "s": "#2d5a87",
        "sw": 2,
        "cr": 8
      }
    },
    "tpl": { "@T": "M" },
    "tpo": { "@T": "M" },
    "pl": {
      "@T": "M",
      "default": {
        "@T": "M",
        "n": "Default",
        "bg": { "@T": "M", "c": "#ffffff" },
        "tx": { "@T": "M", "c": "#333333" },
        "a1": { "@T": "M", "c": "#4a90d9" },
        "a2": { "@T": "M", "c": "#5cb85c" },
        "a3": { "@T": "M", "c": "#f0ad4e" },
        "a4": { "@T": "M", "c": "#d9534f" },
        "a5": { "@T": "M", "c": "#9b59b6" },
        "a6": { "@T": "M", "c": "#1abc9c" }
      }
    }
  }
}