RTDB (Real-Time Database)

The Cloudillo RTDB provides a Firebase-like real-time database with TypeScript support.

RTDB vs CRDT

RTDB is best for structured data with queries (todos, settings, lists). For collaborative document editing where multiple users edit simultaneously, see CRDT. Compare both in Data Storage & Access.

Installation

pnpm add @cloudillo/rtdb

Quick Start

import { getAppBus } from '@cloudillo/core'
import { RtdbClient } from '@cloudillo/rtdb'
import { getRtdbUrl } from '@cloudillo/core'

// Initialize Cloudillo
const bus = getAppBus()
await bus.init('my-app')

// Create RTDB client
const rtdb = new RtdbClient({
  dbId: 'my-database-id',
  auth: {
    getToken: () => bus.accessToken
  },
  serverUrl: getRtdbUrl(bus.idTag!, 'my-database-id', bus.accessToken!)
})

// Connect to the database
await rtdb.connect()

// Get collection reference
const todos = rtdb.collection('todos')

// Create document
const batch = rtdb.batch()
batch.create(todos, {
  title: 'Learn Cloudillo RTDB',
  completed: false,
  createdAt: Date.now()
})
await batch.commit()

// Subscribe to changes
todos.onSnapshot((snapshot) => {
  console.log('Todos:', snapshot.docs.map(doc => doc.data()))
})

RtdbClient Constructor

interface RtdbClientOptions {
  dbId: string                    // Database/file ID
  auth: {
    getToken: () => string | undefined | Promise<string | undefined>
  }
  serverUrl: string               // WebSocket URL
  options?: {
    enableCache?: boolean         // Default: false
    reconnect?: boolean           // Default: true
    reconnectDelay?: number       // Default: 1000ms
    maxReconnectDelay?: number    // Default: 30000ms
    debug?: boolean               // Default: false
  }
}

Example with all options:

import { RtdbClient } from '@cloudillo/rtdb'
import { getRtdbUrl } from '@cloudillo/core'

const rtdb = new RtdbClient({
  dbId: 'my-database-id',
  auth: {
    // Token provider function - called when connection needs auth
    getToken: async () => {
      // Can be sync or async
      return bus.accessToken
    }
  },
  serverUrl: getRtdbUrl(bus.idTag!, 'my-database-id', bus.accessToken!),
  options: {
    enableCache: true,        // Enable local caching
    reconnect: true,          // Auto-reconnect on disconnect
    reconnectDelay: 1000,     // Initial reconnect delay
    maxReconnectDelay: 30000, // Max reconnect delay (exponential backoff)
    debug: false              // Enable debug logging
  }
})

Connection Management

// Connect to database
await rtdb.connect()

// Disconnect
await rtdb.disconnect()

// Check connection status
if (rtdb.isConnected()) {
  console.log('Connected')
}

// Diagnostics
console.log('Pending requests:', rtdb.getPendingRequests())
console.log('Active subscriptions:', rtdb.getActiveSubscriptions())

Core Concepts

Collections

Collections are groups of documents, similar to tables in SQL.

const users = rtdb.collection('users')
const posts = rtdb.collection('posts')
const comments = rtdb.collection('comments')

// Typed collections
interface Todo {
  title: string
  completed: boolean
  createdAt: number
}
const todos = rtdb.collection<Todo>('todos')

Documents

Documents are individual records accessed by path.

// Reference a document by path
const userDoc = rtdb.ref('users/alice')

// Get document data
const snapshot = await userDoc.get()
if (snapshot.exists) {
  console.log(snapshot.data())
}

CRUD Operations

Create

Use batch operations to create documents:

const todos = rtdb.collection('todos')
const batch = rtdb.batch()

// Create with auto-generated ID
batch.create(todos, {
  title: 'New task',
  completed: false
})

// Create with ref for tracking
batch.create(todos, {
  title: 'Another task',
  completed: false
}, { ref: 'task-ref' })

// Commit returns results with IDs
const results = await batch.commit()
console.log('Created IDs:', results.map(r => r.id))

Read

// Get single document
const doc = rtdb.ref('todos/task_123')
const snapshot = await doc.get()
if (snapshot.exists) {
  console.log(snapshot.data())
}

// Query collection
const todos = rtdb.collection('todos')
const results = await todos.get()
results.docs.forEach(doc => {
  console.log(doc.id, doc.data())
})

Update

const batch = rtdb.batch()
const todoRef = rtdb.ref('todos/task_123')

// Partial update
batch.update(todoRef, {
  completed: true
})

await batch.commit()

Delete

const batch = rtdb.batch()
const todoRef = rtdb.ref('todos/task_123')

batch.delete(todoRef)

await batch.commit()

Queries

Query Options

interface QueryOptions {
  filter?: {
    equals?: Record<string, any>
    notEquals?: Record<string, any>
    greaterThan?: Record<string, any>
    lessThan?: Record<string, any>
    greaterThanOrEqual?: Record<string, any>
    lessThanOrEqual?: Record<string, any>
    in?: Record<string, any[]>
    notIn?: Record<string, any[]>
    arrayContains?: Record<string, any>
    arrayContainsAny?: Record<string, any[]>
    arrayContainsAll?: Record<string, any[]>
  }
  sort?: Array<{ field: string; ascending: boolean }>
  limit?: number
  offset?: number
}

You can also build queries using the chainable .where() builder:

type WhereFilterOp =
  | '=='  | '!='
  | '<'   | '>'
  | '<='  | '>='
  | 'in'  | 'not-in'
  | 'array-contains'
  | 'array-contains-any'
  | 'array-contains-all'

collection.where(field: string, op: WhereFilterOp, value: any)

Filtering

const todos = rtdb.collection('todos')

// Filter with options object
const incomplete = await todos.query({
  filter: {
    equals: { completed: false }
  }
})

// Filter with chainable .where() builder
const incomplete2 = await todos
  .where('completed', '==', false)
  .get()

Filter Operators

// Equality
const active = await todos.where('status', '==', 'active').get()
const notDone = await todos.where('status', '!=', 'done').get()

// Comparison
const highPriority = await todos.where('priority', '>', 3).get()
const recent = await todos.where('createdAt', '>=', lastWeek).get()

// Set membership
const selected = await todos
  .where('status', 'in', ['active', 'pending'])
  .get()
const excluded = await todos
  .where('status', 'not-in', ['archived', 'deleted'])
  .get()

// Array filters
const tagged = await todos
  .where('tags', 'array-contains', 'urgent')
  .get()
const anyTag = await todos
  .where('tags', 'array-contains-any', ['urgent', 'important'])
  .get()
const allTags = await todos
  .where('tags', 'array-contains-all', ['frontend', 'bug'])
  .get()

// Chain multiple filters
const filtered = await todos
  .where('completed', '==', false)
  .where('priority', '>', 2)
  .get()

Sorting

// Sort ascending
const sorted = await todos.query({
  sort: [{ field: 'createdAt', ascending: true }]
})

// Sort descending
const newest = await todos.query({
  sort: [{ field: 'createdAt', ascending: false }]
})

// Multiple sorts
const prioritized = await todos.query({
  sort: [
    { field: 'priority', ascending: false },
    { field: 'createdAt', ascending: true }
  ]
})

Pagination

// Limit results
const first10 = await todos.query({
  limit: 10
})

// Pagination with offset
const page2 = await todos.query({
  limit: 20,
  offset: 20
})

Combined Queries

const results = await todos.query({
  filter: {
    equals: { completed: false }
  },
  sort: [{ field: 'createdAt', ascending: false }],
  limit: 10
})

Real-Time Subscriptions

Collection Subscriptions

const todos = rtdb.collection('todos')

// Subscribe to all documents
const unsubscribe = todos.onSnapshot((snapshot) => {
  console.log('Total todos:', snapshot.size)

  snapshot.docs.forEach(doc => {
    console.log(doc.id, doc.data())
  })

  // Track changes
  const changes = snapshot.docChanges()
  changes.forEach(change => {
    switch (change.type) {
      case 'added':
        console.log('New:', change.doc.id)
        break
      case 'modified':
        console.log('Updated:', change.doc.id)
        break
      case 'removed':
        console.log('Deleted:', change.doc.id)
        break
    }
  })
})

// Unsubscribe later
unsubscribe()

Document Subscriptions

const todoRef = rtdb.ref('todos/task_123')

const unsubscribe = todoRef.onSnapshot((snapshot) => {
  if (snapshot.exists) {
    console.log('Todo updated:', snapshot.data())
  } else {
    console.log('Todo deleted')
  }
})

Filtered Subscriptions

// Using options object
const unsubscribe = todos.subscribe({
  filter: {
    equals: { completed: false }
  }
}, (snapshot) => {
  console.log('Incomplete todos:', snapshot.size)
})

// Using .where() builder
const unsubscribe2 = todos
  .where('completed', '==', false)
  .onSnapshot((snapshot) => {
    console.log('Incomplete todos:', snapshot.size)
  })

Document Locking

Lock documents for exclusive or advisory editing access.

Lock Modes

  • soft — Advisory lock. Other clients can still write, but are notified that the document is locked.
  • hard — Enforced lock. The server rejects writes from other clients while the lock is held.

Locking and Unlocking

const docRef = rtdb.ref('todos/task_123')

// Acquire a soft (advisory) lock
const result = await docRef.lock('soft')
// result: { locked: true }

// Acquire a hard (exclusive) lock
const result2 = await docRef.lock('hard')
// result: { locked: true }

// If already locked by another client
const result3 = await docRef.lock('hard')
// result: { locked: false, holder: 'bob@example.com', mode: 'hard' }

// Release the lock
await docRef.unlock()

Lock Result

interface LockResult {
  locked: boolean         // Whether the lock was acquired
  holder?: string         // Identity of current lock holder (if denied)
  mode?: 'soft' | 'hard' // Lock mode of existing lock (if denied)
}

Lock Events

Listen for lock changes on a document using the onLock callback in snapshot options:

const docRef = rtdb.ref('todos/task_123')

const unsubscribe = docRef.onSnapshot({
  onLock: (event) => {
    if (event.action === 'lock') {
      console.log(`Locked by ${event.holder} (${event.mode})`)
    } else if (event.action === 'unlock') {
      console.log('Document unlocked')
    }
  }
}, (snapshot) => {
  console.log('Data:', snapshot.data())
})

Example: Exclusive Editing

async function startEditing(docRef: DocumentRef) {
  const result = await docRef.lock('hard')

  if (!result.locked) {
    alert(`Document is locked by ${result.holder}`)
    return false
  }

  // Edit the document...
  return true
}

async function stopEditing(docRef: DocumentRef) {
  await docRef.unlock()
}
Info

Locks have a TTL (time-to-live) and expire automatically if the client disconnects or fails to renew them. This prevents permanently locked documents from abandoned sessions.

Aggregate Queries

Perform server-side aggregations on collections.

Aggregate API

interface AggregateOptions {
  groupBy?: string             // Field to group results by
  ops: AggregateOp[]           // Aggregation operations
}

type AggregateOp = 'sum' | 'avg' | 'min' | 'max'

interface AggregateGroupEntry {
  group: any                   // Value of the groupBy field
  count: number                // Number of documents in the group
  [key: string]: any           // Aggregate results (e.g., sum_hours, avg_hours)
}

interface AggregateSnapshot {
  groups: AggregateGroupEntry[]
}

Basic Aggregation

const todos = rtdb.collection('todos')

// Aggregate with filters
const result = await todos
  .where('completed', '==', false)
  .aggregate({
    groupBy: 'status',
    ops: ['sum', 'avg']
  })
  .get()

result.groups.forEach(group => {
  console.log(`${group.group}: ${group.count} items`)
})

Real-Time Aggregate Subscriptions

Aggregate queries support real-time updates via onSnapshot:

const unsubscribe = todos
  .where('completed', '==', false)
  .aggregate({
    groupBy: 'status',
    ops: ['sum']
  })
  .onSnapshot((snapshot: AggregateSnapshot) => {
    snapshot.groups.forEach(group => {
      console.log(`${group.group}: ${group.count} items`)
    })
  })

Example: Task Dashboard

const tasks = rtdb.collection('tasks')

// Group tasks by status with count and total estimated hours
const unsubscribe = tasks
  .aggregate({
    groupBy: 'status',
    ops: ['sum', 'avg']
  })
  .onSnapshot((snapshot) => {
    snapshot.groups.forEach(({ group, count, sum_hours, avg_hours }) => {
      console.log(`${group}: ${count} tasks, ${sum_hours}h total, ${avg_hours}h avg`)
    })
    // Output:
    // todo: 12 tasks, 36h total, 3h avg
    // in_progress: 5 tasks, 20h total, 4h avg
    // done: 28 tasks, 84h total, 3h avg
  })

Batch Operations

Perform multiple operations atomically:

const todos = rtdb.collection('todos')
const batch = rtdb.batch()

// Create multiple
batch.create(todos, { title: 'Task 1', completed: false })
batch.create(todos, { title: 'Task 2', completed: false })

// Update existing
batch.update(rtdb.ref('todos/task_123'), { completed: true })

// Delete
batch.delete(rtdb.ref('todos/task_456'))

// Commit all operations atomically
const results = await batch.commit()
console.log('Batch results:', results)

BatchResult:

interface BatchResult {
  ref?: string   // Reference ID if provided
  id?: string    // Generated document ID
}

Indexes

Create indexes for efficient queries:

// Create index on a field
await rtdb.createIndex('todos', 'createdAt')
await rtdb.createIndex('todos', 'completed')

TypeScript Support

Full type safety with generics:

interface Todo {
  title: string
  completed: boolean
  createdAt: number
  tags?: string[]
}

const todos = rtdb.collection<Todo>('todos')

// TypeScript knows the shape
const snapshot = await todos.get()
snapshot.docs.forEach(doc => {
  const data = doc.data()
  console.log(data.title)     // string
  console.log(data.completed) // boolean
  // @ts-error: Property 'invalid' does not exist
  // console.log(data.invalid)
})

React Integration

import { useEffect, useState } from 'react'
import { useAuth } from '@cloudillo/react'
import { RtdbClient } from '@cloudillo/rtdb'
import { getRtdbUrl } from '@cloudillo/core'

interface Todo {
  title: string
  completed: boolean
}

function TodoList({ dbId }: { dbId: string }) {
  const [auth] = useAuth()
  const [todos, setTodos] = useState<Todo[]>([])
  const [rtdb, setRtdb] = useState<RtdbClient | null>(null)

  // Initialize RTDB client
  useEffect(() => {
    if (!auth?.token || !auth?.idTag) return

    const client = new RtdbClient({
      dbId,
      auth: { getToken: () => auth.token },
      serverUrl: getRtdbUrl(auth.idTag, dbId, auth.token!)
    })

    setRtdb(client)

    return () => {
      client.disconnect()
    }
  }, [auth?.token, auth?.idTag, dbId])

  // Subscribe to todos
  useEffect(() => {
    if (!rtdb) return

    const todos = rtdb.collection<Todo>('todos')
    const unsubscribe = todos.onSnapshot((snapshot) => {
      setTodos(snapshot.docs.map(doc => ({
        id: doc.id,
        ...doc.data()
      })))
    })

    return () => unsubscribe()
  }, [rtdb])

  return (
    <ul>
      {todos.map(todo => (
        <li key={todo.id}>{todo.title}</li>
      ))}
    </ul>
  )
}

Error Handling

The RTDB client provides typed errors:

import {
  RtdbError,
  ConnectionError,
  AuthError,
  PermissionError,
  NotFoundError,
  ValidationError,
  TimeoutError
} from '@cloudillo/rtdb'

try {
  await rtdb.connect()
} catch (error) {
  if (error instanceof ConnectionError) {
    console.log('Connection failed:', error.message)
  } else if (error instanceof AuthError) {
    console.log('Authentication failed:', error.message)
  } else if (error instanceof PermissionError) {
    console.log('Permission denied:', error.message)
  } else if (error instanceof NotFoundError) {
    console.log('Not found:', error.message)
  } else if (error instanceof TimeoutError) {
    console.log('Request timed out:', error.message)
  }
}

Best Practices

1. Use Connection Management

// Initialize once, reuse
const rtdb = new RtdbClient({ ... })
await rtdb.connect()

// Use throughout your app
const todos = rtdb.collection('todos')

2. Clean Up Subscriptions

// Always unsubscribe to prevent memory leaks
useEffect(() => {
  const unsubscribe = todos.onSnapshot(callback)
  return () => unsubscribe()
}, [])

3. Use Batch for Multiple Operations

// Good: Atomic batch operation
const batch = rtdb.batch()
batch.create(todos, { title: 'Task 1' })
batch.create(todos, { title: 'Task 2' })
await batch.commit()

// Avoid: Multiple separate requests
// await create({ title: 'Task 1' })
// await create({ title: 'Task 2' })

4. Use Typed Collections

// Define your types
interface Todo {
  title: string
  completed: boolean
}

// Get type safety
const todos = rtdb.collection<Todo>('todos')

5. Handle Connection State

// Check connection before operations
if (!rtdb.isConnected()) {
  await rtdb.connect()
}

// Handle reconnection in UI
const [connected, setConnected] = useState(false)

See Also