Federation Architecture

Cloudillo is designed as a federated system where independent instances communicate to share content, enable collaboration, and maintain user sovereignty. This architecture ensures users can self-host while remaining connected to the broader network.

Core Principles

Decentralization

  • No central authority: No single server controls the network
  • User sovereignty: Users choose where their data lives
  • Instance independence: Each instance operates autonomously
  • Opt-in federation: Instances choose federation partners

Privacy-First

  • Explicit consent: Users control what they share
  • Relationship-based: Only share with connected/following users
  • End-to-end verification: Cryptographic signatures prove authenticity
  • No data harvesting: Federation for communication, not surveillance

Interoperability

  • Standard protocols: HTTP/HTTPS, WebSocket, JWT
  • Content addressing: SHA256 ensures integrity across instances
  • Action tokens: Portable, verifiable activity records
  • DNS-based identity: Discoverable profiles via domain names

Inter-Instance Communication

Request Module

The request module provides HTTP client functionality for federation:

Request Structure:

  • client: HTTP client (reqwest) for making requests
  • timeout: Request timeout duration
  • retry_policy: Configuration for automatic retries

Request Methods:

  • get(url) - Generic HTTP GET
  • post(url, body) - Generic HTTP POST with JSON body
  • fetch_profile(id_tag) - GET /api/me from remote instance
  • fetch_action(id_tag, action_id) - GET /api/action/:id from remote
  • post_inbox(id_tag, token) - POST /api/inbox with action token
  • fetch_file(id_tag, file_id) - GET /api/file/:id from remote

Federation Flow

When Alice (on instance A) follows Bob (on instance B):

Instance A                          Instance B
  |                                   |
  |--- GET /api/me ------------------>| (Fetch Bob's profile)
  |<-- 200 OK {profile} --------------|
  |                                   |
  |--- Create FLLW action token ---   |
  |                                   |
  |--- POST /api/inbox -------------->| (Send follow action)
  |    {token: "eyJhbGc..."}         |
  |                                   |--- Verify signature
  |                                   |--- Check permissions
  |                                   |--- Store action
  |<-- 202 Accepted ------------------|

Action Distribution

Outbound Actions

When a user creates an action, it’s distributed to relevant recipients:

Algorithm: Distribute Action

Input: action_token, action_type
Output: Result<()>

1. Determine recipients based on action_type:
   a. "POST" → All followers of the user
   b. "CMNT" / "REACT" → Owner of parent post
   c. "CONN" / "FLLW" → Target from action audience (aud claim)
   d. "MSG" → All conversation participants
   e. Other → No recipients (skip)

2. For each recipient instance:
   a. Construct URL: https://cl-o.{recipient_id_tag}/api/inbox
   b. POST action token as JSON: {"token": "..."}
   c. Continue on error (best-effort delivery)

3. Return success

Inbound Actions

The /api/inbox endpoint receives federated actions:

HTTP Endpoint: POST /api/inbox

Handler Algorithm:
1. Extract action token from JSON payload
2. No authentication required (token is self-authenticating)
3. Create ActionVerifierTask with token
4. Schedule task in task scheduler (for async processing)
5. Return HTTP 202 (Accepted) immediately
   - Verification and processing happen asynchronously
   - Prevents network timeouts on slow verification

Delivery Guarantees

Best Effort Delivery:

  • Actions sent once to each recipient
  • No delivery confirmation
  • No automatic retry (recipient may request later)

Failure Handling Algorithm:

On POST /api/inbox result:

1. Success (Ok):
   - Mark action as delivered in metadata
   - Update delivery status in history

2. Temporary Error (network timeout, 5xx, connection refused):
   - Schedule retry with exponential backoff
   - Maximum 3-5 retry attempts

3. Permanent Error (4xx status, validation error):
   - Log error with context
   - Mark as undeliverable
   - Continue (don't block other deliveries)

ProxyToken Authentication

For cross-instance requests, ProxyTokens authenticate the requesting instance:

ProxyToken Structure

{
  "iss": "alice.example.com",      // Requesting instance
  "aud": "bob.example.com",         // Target instance
  "sub": "alice.example.com",       // User identity
  "exp": 1738400000,                // Short-lived (5-60 min)
  "iat": 1738396800,
  "action": "read_file",            // Requested operation
  "resource": "f1~abc123...",       // Resource identifier
  "k": "20250205"                   // Signing key ID
}

Creation

Algorithm: Create ProxyToken

Input: requester id_tag, target_instance, action, resource
Output: JWT token string

1. Retrieve latest signing key:
   - Query latest key_id for tenant
   - Load private key from AuthAdapter

2. Build JWT claims:
   - iss: Requester's id_tag
   - aud: Target instance domain
   - sub: Requester's id_tag
   - exp: Current time + 30 minutes
   - iat: Current time
   - action: Requested operation
   - resource: Resource identifier
   - k: Key ID used for signing

3. Sign JWT using ES384 algorithm:
   - Use private key
   - Standard JWT encoding

4. Return base64-encoded JWT

Validation

Algorithm: Validate ProxyToken

Input: JWT token string
Output: Result<ProxyTokenClaims>

1. Decode JWT without signature verification (read claims)
2. Extract issuer and key_id from unverified claims
3. Fetch issuer's profile from remote instance
4. Look up public key by key_id in profile
5. Verify JWT signature using issuer's public key (ES384)
6. Check expiration timestamp:
   - If exp < current_time: Return TokenExpired error
7. Check audience claim:
   - If aud != this_instance.base_id_tag: Return InvalidAudience error
8. Return verified claims

Validation ensures:
- Token signed by claimed issuer
- Token not expired
- Token intended for this instance

File Synchronization

Attachment Fetching

When receiving an action with file attachments:

Algorithm: Sync Attachments

Input: attachment_ids from remote action
Output: Result<()>

For each attachment_id:
1. Check if already exists locally
   - If exists: Skip to next (already synced)

2. Construct remote URL:
   https://cl-o.{issuer_id_tag}/api/file/{attachment_id}

3. Download file from remote instance

4. Verify content integrity:
   - Compute SHA256 hash of downloaded data
   - Compare hash with attachment_id
   - If mismatch: Return FileIntegrityCheckFailed error

5. Store file data in blob adapter

6. Extract and store metadata:
   - Read X-File-Metadata header (if present)
   - Parse as JSON
   - Store in metadata adapter

7. Continue to next attachment

This ensures:
- Content-addressed files (hash = ID)
- No duplicate downloads
- Cryptographic integrity verification

Lazy Loading

Files are fetched on-demand rather than proactively:

User views post with image attachment
  ↓
Check if image exists locally
  ↓
If not, fetch from remote instance
  ↓
Verify content hash
  ↓
Store locally
  ↓
Serve to user

Profile Synchronization

Remote Profile Caching

Cache remote profiles locally for performance:

Algorithm: Sync Profile with Caching

Input: id_tag (remote user identifier)
Output: Result<Profile>

1. Check local cache for profile:
   - If cached AND cache_age < 24 hours: Return cached profile
   - If cache_age >= 24 hours: Continue to step 2

2. Fetch profile from remote instance:
   - GET https://cl-o.{id_tag}/api/me

3. Update local cache:
   - Store profile with current timestamp

4. Return profile

Benefits:
- Reduces network requests (24h TTL)
- Improves performance for repeated access
- Staleness acceptable for user profiles

Profile Updates

Profiles don’t push updates; instances pull when needed:

Need to display Alice's profile
  ↓
Check cache (last updated < 24h?)
  ↓
If fresh: use cache
If stale: fetch from Alice's instance
  ↓
Update cache
  ↓
Display profile

Relationship Management

Following

When Alice follows Bob:

Algorithm: Follow User

Input: follower_id_tag, target_id_tag
Output: Result<()>

1. Create FLLW action token:
   - issuer: follower's id_tag
   - subject: target's id_tag
   - action type: "FLLW"
   - Sign with follower's private key (ES384)

2. Store action locally:
   - Record in metadata adapter
   - Marks that follower follows target

3. Send to target instance:
   - POST https://cl-o.{target_id_tag}/api/inbox
   - Include signed action token

4. Return success

Connection Establishment

Connections require mutual agreement:

Alice sends CONN to Bob
  ↓
Bob receives, stores CONN
  ↓
Bob sends CONN to Alice
  ↓
Alice receives, detects mutual CONN
  ↓
Connection established (both sides)

Unfollowing

Unfollowing creates a new action that supersedes the previous follow:

Algorithm: Unfollow User

Input: follower_id_tag, target_id_tag
Output: Result<()>

1. Create FLLW action token with removed=true:
   - issuer: follower's id_tag
   - subject: target's id_tag
   - action type: "FLLW"
   - removed: true (indicates unfollow)
   - Sign with follower's private key (ES384)

2. Store action locally:
   - New FLLW action with removed=true
   - Supersedes previous FLLW action
   - Marks unfollow event

3. Send to target instance:
   - POST https://cl-o.{target_id_tag}/api/inbox
   - Include signed action token

4. Return success

The removed=true flag indicates this action cancels the previous follow.

Database Federation

Read-Only Replication

Subscribe to remote database updates:

FederatedDatabase Structure:

  • origin_instance: Source instance domain (e.g., alice.example.com)
  • local_replica: Whether to maintain local copy for fast access
  • sync_mode: Synchronization mode (see below)

SyncMode Enum:

  • ReadOnly: Subscribe to updates from remote, no local edits
  • ReadWrite: Bidirectional synchronization
  • Periodic(Duration): Full sync every N seconds (fallback for network issues)

Sync Protocol

Using action/inbox mechanism:

DatabaseSyncAction Structure:

  • db_file_id: SHA256 identifier of database file
  • updates: Binary update payload (Yrs CRDT or redb operations)
  • state_vector: Current state hash for conflict detection
  • timestamp: Unix timestamp of update creation

Database Update Distribution Algorithm:

For each subscriber instance:

  1. Create DatabaseSyncAction with:

    • Database file ID
    • Binary updates (from CRDT or redb)
    • Computed state vector
    • Current timestamp
  2. POST to subscriber’s inbox:

    • Endpoint: https://cl-o.{subscriber_id_tag}/api/inbox
    • Send DatabaseSyncAction as JSON
  3. Subscriber’s ActionVerifierTask processes:

    • Extracts binary updates
    • Applies to local replica
    • Merges with any local changes

This pattern allows:

  • Real-time database synchronization
  • Conflict resolution via CRDTs
  • Federation of collaborative databases

Security Considerations

Trust Model

DNS-Based Trust:

  • Domain ownership proves identity
  • TLS certificates prove server authenticity
  • Action signatures prove content authenticity

Progressive Trust:

  • Initial federation is cautious
  • Trust builds through successful interactions
  • Users can block instances/users

Spam Prevention

Relationship-Based Action Acceptance:

Algorithm: Should Accept Action

Input: action_type, issuer, local_user
Output: bool (accept or reject)

Decision logic by action_type:

1. "POST" → Only from followed or connected users
   - Check relationship between issuer and local_user
   - Accept if: relationship.following OR relationship.connected
   - Reject if: stranger with no relationship

2. "CMNT" / "REACT" → Always accept
   - Verification phase checks ownership of target content
   - Accept all, let verification handle validation

3. "CONN" / "FLLW" → Always accept
   - Relationship requests always received
   - User can block after receiving

4. Other action types → Reject

Federation Rate Limits (hardcoded):

  • max_actions_per_instance_per_hour: 1000 (per federated instance)
  • max_actions_per_user_per_hour: 100 (per remote user)
  • max_concurrent_connections: 100 (simultaneous federation connections)
  • max_file_requests_per_hour: 500 (file sync requests)

Blocklisting

Users can block instances or specific users:

Algorithm: Block Instance

Input: blocked_instance domain
Output: Result<()>

1. Add instance domain to user's blocklist
2. Store in metadata adapter
3. All future actions from this instance are rejected
4. Return success

User can later unblock by removing from blocklist.

Monitoring & Observability

Metrics

FederationMetrics Structure:

  • outbound_actions_sent (AtomicU64) - Successfully sent federation actions
  • outbound_actions_failed (AtomicU64) - Failed outbound deliveries
  • inbound_actions_received (AtomicU64) - Received federated actions
  • inbound_actions_rejected (AtomicU64) - Rejected actions (spam, invalid, blocked)
  • profiles_synced (AtomicU64) - Remote profiles cached/updated
  • files_synced (AtomicU64) - Attachment files downloaded
  • active_federation_connections (AtomicUsize) - Open federation connections

Logging

Sent Action Log:

  • instance: Target instance domain
  • action_type: Type of action (POST, FLLW, etc.)
  • action_id: Unique action identifier

Rejected Action Log:

  • instance: Source instance domain
  • action_type: Type of action
  • reason: Why rejected (spam, invalid sig, blocked, etc.)

Best Practices

For Instance Operators

Enable HTTPS: Always use TLS for federation ✅ Monitor logs: Watch for spam or abuse ✅ Set rate limits: Protect against DoS ✅ Backup regularly: Federation doesn’t replace backups ✅ Update promptly: Security patches are critical

For Developers

Verify signatures: Never trust unverified content ✅ Check relationships: Enforce connection requirements ✅ Handle failures: Network is unreliable ✅ Cache wisely: Balance freshness vs performance ✅ Test federation: Use multiple instances for testing

Troubleshooting

Common Issues

Actions not federating:

  • Check DNS resolution of target instance
  • Verify TLS certificate validity
  • Check firewall rules
  • Review federation logs

Signature verification failures:

  • Ensure issuer’s public key is fetchable
  • Check key expiration
  • Verify algorithm matches (ES384)

File sync failures:

  • Verify content hash computation
  • Check blob adapter permissions
  • Ensure sufficient storage space

See Also

  • Identity System - DNS-based identity and keys
  • [Actions](/architecture/actions-federation/actions - Action token distribution
  • [Access Control](/architecture/data-layer/access-control/access - ProxyToken authentication
  • Network & Security - TLS and ACME