Federation Architecture
Cloudillo is designed as a federated system where independent instances communicate to share content, enable collaboration, and maintain user sovereignty. This architecture ensures users can self-host while remaining connected to the broader network.
Core Principles
Decentralization
- No central authority: No single server controls the network
- User sovereignty: Users choose where their data lives
- Instance independence: Each instance operates autonomously
- Opt-in federation: Instances choose federation partners
Privacy-First
- Explicit consent: Users control what they share
- Relationship-based: Only share with connected/following users
- End-to-end verification: Cryptographic signatures prove authenticity
- No data harvesting: Federation for communication, not surveillance
Interoperability
- Standard protocols: HTTP/HTTPS, WebSocket, JWT
- Content addressing: SHA256 ensures integrity across instances
- Action tokens: Portable, verifiable activity records
- DNS-based identity: Discoverable profiles via domain names
Inter-Instance Communication
Request Module
The request module provides HTTP client functionality for federation:
Request Structure:
client: HTTP client (reqwest) for making requeststimeout: Request timeout durationretry_policy: Configuration for automatic retries
Request Methods:
get(url)- Generic HTTP GETpost(url, body)- Generic HTTP POST with JSON bodyfetch_profile(id_tag)- GET /api/me from remote instancefetch_action(id_tag, action_id)- GET /api/action/:id from remotepost_inbox(id_tag, token)- POST /api/inbox with action tokenfetch_file(id_tag, file_id)- GET /api/file/:id from remote
Federation Flow
When Alice (on instance A) follows Bob (on instance B):
Instance A Instance B
| |
|--- GET /api/me ------------------>| (Fetch Bob's profile)
|<-- 200 OK {profile} --------------|
| |
|--- Create FLLW action token --- |
| |
|--- POST /api/inbox -------------->| (Send follow action)
| {token: "eyJhbGc..."} |
| |--- Verify signature
| |--- Check permissions
| |--- Store action
|<-- 202 Accepted ------------------|Action Distribution
Outbound Actions
When a user creates an action, it’s distributed to relevant recipients:
Algorithm: Distribute Action
Input: action_token, action_type
Output: Result<()>
1. Determine recipients based on action_type:
a. "POST" → All followers of the user
b. "CMNT" / "REACT" → Owner of parent post
c. "CONN" / "FLLW" → Target from action audience (aud claim)
d. "MSG" → All conversation participants
e. Other → No recipients (skip)
2. For each recipient instance:
a. Construct URL: https://cl-o.{recipient_id_tag}/api/inbox
b. POST action token as JSON: {"token": "..."}
c. Continue on error (best-effort delivery)
3. Return successInbound Actions
The /api/inbox endpoint receives federated actions:
HTTP Endpoint: POST /api/inbox
Handler Algorithm:
1. Extract action token from JSON payload
2. No authentication required (token is self-authenticating)
3. Create ActionVerifierTask with token
4. Schedule task in task scheduler (for async processing)
5. Return HTTP 202 (Accepted) immediately
- Verification and processing happen asynchronously
- Prevents network timeouts on slow verificationDelivery Guarantees
Best Effort Delivery:
- Actions sent once to each recipient
- No delivery confirmation
- No automatic retry (recipient may request later)
Failure Handling Algorithm:
On POST /api/inbox result:
1. Success (Ok):
- Mark action as delivered in metadata
- Update delivery status in history
2. Temporary Error (network timeout, 5xx, connection refused):
- Schedule retry with exponential backoff
- Maximum 3-5 retry attempts
3. Permanent Error (4xx status, validation error):
- Log error with context
- Mark as undeliverable
- Continue (don't block other deliveries)ProxyToken Authentication
For cross-instance requests, ProxyTokens authenticate the requesting instance:
ProxyToken Structure
{
"iss": "alice.example.com", // Requesting instance
"aud": "bob.example.com", // Target instance
"sub": "alice.example.com", // User identity
"exp": 1738400000, // Short-lived (5-60 min)
"iat": 1738396800,
"action": "read_file", // Requested operation
"resource": "f1~abc123...", // Resource identifier
"k": "20250205" // Signing key ID
}Creation
Algorithm: Create ProxyToken
Input: requester id_tag, target_instance, action, resource
Output: JWT token string
1. Retrieve latest signing key:
- Query latest key_id for tenant
- Load private key from AuthAdapter
2. Build JWT claims:
- iss: Requester's id_tag
- aud: Target instance domain
- sub: Requester's id_tag
- exp: Current time + 30 minutes
- iat: Current time
- action: Requested operation
- resource: Resource identifier
- k: Key ID used for signing
3. Sign JWT using ES384 algorithm:
- Use private key
- Standard JWT encoding
4. Return base64-encoded JWTValidation
Algorithm: Validate ProxyToken
Input: JWT token string
Output: Result<ProxyTokenClaims>
1. Decode JWT without signature verification (read claims)
2. Extract issuer and key_id from unverified claims
3. Fetch issuer's profile from remote instance
4. Look up public key by key_id in profile
5. Verify JWT signature using issuer's public key (ES384)
6. Check expiration timestamp:
- If exp < current_time: Return TokenExpired error
7. Check audience claim:
- If aud != this_instance.base_id_tag: Return InvalidAudience error
8. Return verified claims
Validation ensures:
- Token signed by claimed issuer
- Token not expired
- Token intended for this instanceFile Synchronization
Attachment Fetching
When receiving an action with file attachments:
Algorithm: Sync Attachments
Input: attachment_ids from remote action
Output: Result<()>
For each attachment_id:
1. Check if already exists locally
- If exists: Skip to next (already synced)
2. Construct remote URL:
https://cl-o.{issuer_id_tag}/api/file/{attachment_id}
3. Download file from remote instance
4. Verify content integrity:
- Compute SHA256 hash of downloaded data
- Compare hash with attachment_id
- If mismatch: Return FileIntegrityCheckFailed error
5. Store file data in blob adapter
6. Extract and store metadata:
- Read X-File-Metadata header (if present)
- Parse as JSON
- Store in metadata adapter
7. Continue to next attachment
This ensures:
- Content-addressed files (hash = ID)
- No duplicate downloads
- Cryptographic integrity verificationLazy Loading
Files are fetched on-demand rather than proactively:
User views post with image attachment
↓
Check if image exists locally
↓
If not, fetch from remote instance
↓
Verify content hash
↓
Store locally
↓
Serve to userProfile Synchronization
Remote Profile Caching
Cache remote profiles locally for performance:
Algorithm: Sync Profile with Caching
Input: id_tag (remote user identifier)
Output: Result<Profile>
1. Check local cache for profile:
- If cached AND cache_age < 24 hours: Return cached profile
- If cache_age >= 24 hours: Continue to step 2
2. Fetch profile from remote instance:
- GET https://cl-o.{id_tag}/api/me
3. Update local cache:
- Store profile with current timestamp
4. Return profile
Benefits:
- Reduces network requests (24h TTL)
- Improves performance for repeated access
- Staleness acceptable for user profilesProfile Updates
Profiles don’t push updates; instances pull when needed:
Need to display Alice's profile
↓
Check cache (last updated < 24h?)
↓
If fresh: use cache
If stale: fetch from Alice's instance
↓
Update cache
↓
Display profileRelationship Management
Following
When Alice follows Bob:
Algorithm: Follow User
Input: follower_id_tag, target_id_tag
Output: Result<()>
1. Create FLLW action token:
- issuer: follower's id_tag
- subject: target's id_tag
- action type: "FLLW"
- Sign with follower's private key (ES384)
2. Store action locally:
- Record in metadata adapter
- Marks that follower follows target
3. Send to target instance:
- POST https://cl-o.{target_id_tag}/api/inbox
- Include signed action token
4. Return successConnection Establishment
Connections require mutual agreement:
Alice sends CONN to Bob
↓
Bob receives, stores CONN
↓
Bob sends CONN to Alice
↓
Alice receives, detects mutual CONN
↓
Connection established (both sides)Unfollowing
Unfollowing creates a new action that supersedes the previous follow:
Algorithm: Unfollow User
Input: follower_id_tag, target_id_tag
Output: Result<()>
1. Create FLLW action token with removed=true:
- issuer: follower's id_tag
- subject: target's id_tag
- action type: "FLLW"
- removed: true (indicates unfollow)
- Sign with follower's private key (ES384)
2. Store action locally:
- New FLLW action with removed=true
- Supersedes previous FLLW action
- Marks unfollow event
3. Send to target instance:
- POST https://cl-o.{target_id_tag}/api/inbox
- Include signed action token
4. Return success
The removed=true flag indicates this action cancels the previous follow.Database Federation
Read-Only Replication
Subscribe to remote database updates:
FederatedDatabase Structure:
origin_instance: Source instance domain (e.g., alice.example.com)local_replica: Whether to maintain local copy for fast accesssync_mode: Synchronization mode (see below)
SyncMode Enum:
ReadOnly: Subscribe to updates from remote, no local editsReadWrite: Bidirectional synchronizationPeriodic(Duration): Full sync every N seconds (fallback for network issues)
Sync Protocol
Using action/inbox mechanism:
DatabaseSyncAction Structure:
db_file_id: SHA256 identifier of database fileupdates: Binary update payload (Yrs CRDT or redb operations)state_vector: Current state hash for conflict detectiontimestamp: Unix timestamp of update creation
Database Update Distribution Algorithm:
For each subscriber instance:
-
Create DatabaseSyncAction with:
- Database file ID
- Binary updates (from CRDT or redb)
- Computed state vector
- Current timestamp
-
POST to subscriber’s inbox:
- Endpoint: https://cl-o.{subscriber_id_tag}/api/inbox
- Send DatabaseSyncAction as JSON
-
Subscriber’s ActionVerifierTask processes:
- Extracts binary updates
- Applies to local replica
- Merges with any local changes
This pattern allows:
- Real-time database synchronization
- Conflict resolution via CRDTs
- Federation of collaborative databases
Security Considerations
Trust Model
DNS-Based Trust:
- Domain ownership proves identity
- TLS certificates prove server authenticity
- Action signatures prove content authenticity
Progressive Trust:
- Initial federation is cautious
- Trust builds through successful interactions
- Users can block instances/users
Spam Prevention
Relationship-Based Action Acceptance:
Algorithm: Should Accept Action
Input: action_type, issuer, local_user
Output: bool (accept or reject)
Decision logic by action_type:
1. "POST" → Only from followed or connected users
- Check relationship between issuer and local_user
- Accept if: relationship.following OR relationship.connected
- Reject if: stranger with no relationship
2. "CMNT" / "REACT" → Always accept
- Verification phase checks ownership of target content
- Accept all, let verification handle validation
3. "CONN" / "FLLW" → Always accept
- Relationship requests always received
- User can block after receiving
4. Other action types → RejectFederation Rate Limits (hardcoded):
max_actions_per_instance_per_hour: 1000 (per federated instance)max_actions_per_user_per_hour: 100 (per remote user)max_concurrent_connections: 100 (simultaneous federation connections)max_file_requests_per_hour: 500 (file sync requests)
Blocklisting
Users can block instances or specific users:
Algorithm: Block Instance
Input: blocked_instance domain
Output: Result<()>
1. Add instance domain to user's blocklist
2. Store in metadata adapter
3. All future actions from this instance are rejected
4. Return success
User can later unblock by removing from blocklist.Monitoring & Observability
Metrics
FederationMetrics Structure:
outbound_actions_sent(AtomicU64) - Successfully sent federation actionsoutbound_actions_failed(AtomicU64) - Failed outbound deliveriesinbound_actions_received(AtomicU64) - Received federated actionsinbound_actions_rejected(AtomicU64) - Rejected actions (spam, invalid, blocked)profiles_synced(AtomicU64) - Remote profiles cached/updatedfiles_synced(AtomicU64) - Attachment files downloadedactive_federation_connections(AtomicUsize) - Open federation connections
Logging
Sent Action Log:
- instance: Target instance domain
- action_type: Type of action (POST, FLLW, etc.)
- action_id: Unique action identifier
Rejected Action Log:
- instance: Source instance domain
- action_type: Type of action
- reason: Why rejected (spam, invalid sig, blocked, etc.)
Best Practices
For Instance Operators
✅ Enable HTTPS: Always use TLS for federation ✅ Monitor logs: Watch for spam or abuse ✅ Set rate limits: Protect against DoS ✅ Backup regularly: Federation doesn’t replace backups ✅ Update promptly: Security patches are critical
For Developers
✅ Verify signatures: Never trust unverified content ✅ Check relationships: Enforce connection requirements ✅ Handle failures: Network is unreliable ✅ Cache wisely: Balance freshness vs performance ✅ Test federation: Use multiple instances for testing
Troubleshooting
Common Issues
Actions not federating:
- Check DNS resolution of target instance
- Verify TLS certificate validity
- Check firewall rules
- Review federation logs
Signature verification failures:
- Ensure issuer’s public key is fetchable
- Check key expiration
- Verify algorithm matches (ES384)
File sync failures:
- Verify content hash computation
- Check blob adapter permissions
- Ensure sufficient storage space
See Also
- Identity System - DNS-based identity and keys
- [Actions](/architecture/actions-federation/actions - Action token distribution
- [Access Control](/architecture/data-layer/access-control/access - ProxyToken authentication
- Network & Security - TLS and ACME