RTDB Overview

Cloudillo’s RTDB (Real-Time Database) provides Firebase-like functionality for structured JSON data with queries, subscriptions, and real-time synchronization. It integrates seamlessly with Cloudillo’s federated architecture while maintaining privacy and user control.

CRDT Collaborative Editing (Separate System)

Cloudillo also provides a separate CRDT API for collaborative editing:

Technology: Yrs - Rust implementation of Yjs CRDT

Features:

• Conflict-free replicated data types (CRDTs)
• Rich data structures (Text, Map, Array, XML)
• Automatic conflict resolution
• Time-travel and versioning
• Awareness (presence, cursors)
• Yjs ecosystem compatibility

Use Cases:

• Collaborative text editors (Google Docs-like)
• Shared whiteboards
• Real-time collaborative forms
• Collaborative spreadsheets
• Multiplayer game state

Learn more: CRDT Collaborative Editing

Comparison: RTDB vs CRDT

Note: These are separate, complementary systems. Use RTDB for structured data with queries, and CRDT for collaborative editing scenarios.

Core Concept: Database-as-File

Both systems use the same foundational concept: databases/documents are special files in the Cloudillo file system.

How It Works

1. File Metadata (MetaAdapter) stores:

   • Database ID, name, owner
   • Creation timestamp, last accessed
   • Permission rules
   • Configuration (max size, retention policy)

2. Database Content (RtdbAdapter or CrdtAdapter) stores:

   • Actual data (documents, CRDT state)
   • Indexes (for query performance)
   • Snapshots (for fast loading)

3. File ID serves as database identifier:

   /ws/rtdb/:fileId  // WebSocket connection endpoint

Benefits

• Natural Integration: Databases managed like files
• Permission Reuse: File permissions apply to databases
• Federation Ready: Databases can be shared across instances
• Discoverable: Find databases through file APIs

Creating and Opening a Database

There is no dedicated “create database” call. A database is opened by connecting to its WebSocket endpoint with a file ID. For authenticated users the backing store file is created lazily on first connect:

• Store databases (s~<app-id>): an app’s persistent store, auto-created with file type RTDB.
• Meta databases (<fileId>~meta): per-file comment/metadata stores, auto-created against the parent file’s permissions.

// Connecting opens (and, for store/meta IDs, lazily creates) the database
const ws = new WebSocket(`wss://cl-o.alice.example.com/ws/rtdb/${fileId}`);

Access is granted through the file’s existing permissions — see the permission model below.

Architecture Overview

Components

┌─────────────────────────────────────────────────────┐
│ Client Application                                  │
│  - JavaScript/TypeScript                            │
│  - React hooks / Vue composables                    │
│  - WebSocket connection                             │
└─────────────────────────────────────────────────────┘
                          ↓ WebSocket
┌─────────────────────────────────────────────────────┐
│ Cloudillo Server                                    │
│  ┌─────────────────────────────────────────────┐    │
│  │ WebSocket Handler                           │    │
│  │  - Authentication                           │    │
│  │  - Message routing                          │    │
│  │  - Subscription management                  │    │
│  └─────────────────────────────────────────────┘    │
│                        ↓                            │
│  ┌─────────────────────────────────────────────┐    │
│  │ Database Manager                            │    │
│  │  - Instance lifecycle (load/evict)          │    │
│  │  - LRU eviction of idle databases           │    │
│  │  - Memory limits                            │    │
│  └─────────────────────────────────────────────┘    │
│         ↓                           ↓               │
│  ┌─────────────┐           ┌──────────────┐         │
│  │ RtdbAdapter │           │ CrdtAdapter  │         │
│  │  (redb)     │           │  (Yrs)       │         │
│  └─────────────┘           └──────────────┘         │
│         ↓                           ↓               │
│  ┌──────────────────────────────────────────────┐   │
│  │ Storage Layer                                │   │
│  │  - redb files (documents, indexes, meta)     │   │
│  │  - MetaAdapter (file metadata, permissions)  │   │
│  └──────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘

Permission Model

Connection-Time Access Check

Permissions are checked once at WebSocket connection time using file_access::check_file_access_with_scope(). This function evaluates multiple access sources:

1. Scoped tokens: Share links with restricted access
2. Ownership: File owner has full access
3. Tenant roles: Role-based access within the tenant
4. FSHR action tokens: Federation-based file sharing permissions

The result determines whether the connection operates in read_only or read_write mode. Clients can also request a specific access level via the ?access=read or ?access=write query parameter.

Future: Fine-Grained Permissions

Planned for future releases:

• Per-collection permissions: Different access per table
• Per-document permissions: Filter queries by ownership
• Runtime rules: JavaScript-like expressions evaluated at runtime
• Attribute-based: Permissions based on user attributes

WebSocket Protocol

Both systems (RTDB and CRDT) use WebSocket for real-time communication, though with different protocols:

Connection

The endpoint is authenticated like any other Cloudillo request (session cookie or token); the browser WebSocket constructor takes only the URL. An optional ?access=read|write query parameter requests a specific access level.

const ws = new WebSocket(`wss://cl-o.example.com/ws/rtdb/${fileId}`);

Message Format

JSON messages with type field:

// Client → Server
{
  "type": "query",      // or "subscribe", "create", "update", "delete"
  "id": 123,            // Request ID for correlation
  // ... type-specific fields
}

// Server → Client
{
  "type": "queryResult", // or "change", "error"
  "id": 123,             // Matches request ID
  // ... response data
}

Storage Strategy

Each database is backed by a redb file. The RtdbAdapter persists every write through an ACID transaction committed atomically — redb is the durable store, so there is no separate snapshot format. Opened databases are cached in memory; idle instances are evicted by a background LRU task, releasing the file handle until the next connection reopens it.

For details on the storage layout, see RTDB with redb.

Federation Support

Databases can be shared across Cloudillo instances through the file sharing mechanism (FSHR action tokens). Access from remote users is granted via the same check_file_access_with_scope() system used for local access control.

Security Considerations

• WebSocket connections require valid access tokens, validated when the connection is established
• Permissions are evaluated once at connection time, fixing the session to read-only or read-write (there is no per-operation re-check)
• TLS/WSS is used for all connections

Choosing Between RTDB and CRDT

Use RTDB (redb) for structured data with schemas, complex queries (filters, sorts, aggregates), computed values, document locking, and atomic transactions.

Use CRDT (Yrs) for concurrent multi-user editing, conflict-free merging, rich text editing, offline-first design, and Yjs ecosystem compatibility.

Both can be used together – for example, Yrs for collaborative document editing and redb for structured metadata.

API Overview

RTDB has a single endpoint — the WebSocket. All reads, writes, subscriptions, locks, and index management happen over it (see RTDB with redb for the message protocol).

GET /ws/rtdb/:fileId
Upgrade: websocket
?access=read | write     # optional requested access level

Because a database is a file, lifecycle and metadata are handled through the standard file APIs (/api/files/...): listing, deletion, sharing, and permission changes all apply to the underlying store file. There are no RTDB-specific create, metadata, export, or import endpoints.

Next Steps

• RTDB with redb - Query-based database with WebSocket protocol
• CRDT Collaborative Editing - Yrs-based conflict-free editing
• System Architecture - Overall architecture context
• File Storage - Database-as-file implementation