Content-Addressing & Merkle Trees

Cloudillo implements a merkle tree structure using content-addressed identifiers throughout its architecture. Every action, file, and data blob is identified by the cryptographic hash of its content, creating an immutable, verifiable chain of trust.

What is Content-Addressing?

Content-addressing means identifying data by what it is (its content) rather than where it is (its location). Instead of using arbitrary IDs or URLs, Cloudillo computes a cryptographic hash of the content itself and uses that hash as the identifier.

Benefits

  • ✅ Immutable: Content cannot change without changing its identifier
  • ✅ Tamper-Evident: Any modification is immediately detectable
  • ✅ Deduplicatable: Identical content produces identical identifiers
  • ✅ Verifiable: Anyone can recompute and verify hashes independently
  • ✅ Cacheable: Content-addressed data can be cached forever
  • ✅ Trustless: No need to trust storage providers—verify the hash

Hash Function

Cloudillo uses SHA-256 for all content-addressing:

  • Algorithm: SHA-256 (256-bit Secure Hash Algorithm)
  • Encoding: Base64url without padding (URL-safe)
  • Output: 43-character base64-encoded string
  • Collision Resistance: Cryptographically secure
compute_hash(prefix, data):
    hash = SHA256(data)
    encoded = base64url_encode(hash)  // URL-safe, no padding
    return "{prefix}1~{encoded}"

// Example:
compute_hash("b", blob_bytes) → "b1~abc123def456..." (43 chars)
compute_hash("f", descriptor)  → "f1~Qo2E3G8TJZ..." (43 chars)
compute_hash("a", jwt_token)   → "a1~8kR3mN9pQ2vL..." (43 chars)

Merkle Tree Structure

Cloudillo’s content-addressing creates a variable-depth merkle tree where actions can reference other actions recursively. The example below shows a six-level hierarchy for a POST action with image attachments:

┌─────────────────────────────────────────────────────────┐
│ Level 6: Action ID (a1~8kR3mN9pQ2vL6xW...)              │
│   ↑ SHA-256 hash of ↓                                   │
├─────────────────────────────────────────────────────────┤
│ Level 5: Action Token (JWT)                             │
│   Header: {"alg":"ES384","typ":"JWT"}                   │
│   Payload: {                                            │
│     "iss": "alice.example.com",                         │
│     "t": "POST:IMG",                                    │
│     "c": "Amazing photo!",                              │
│     "a": ["f1~Qo2E3G8TJZ..."],                          │
│     "iat": 1738483100                                   │
│   }                                                     │
│   Signature: <ES384 signature>                          │
│   ↓ references ↓                                        │
├─────────────────────────────────────────────────────────┤
│ Level 4: File ID (f1~Qo2E3G8TJZ2HTGhVlrtTDBp...)        │
│   ↑ SHA-256 hash of ↓                                   │
├─────────────────────────────────────────────────────────┤
│ Level 3: File Descriptor (d2,...)                       │
│   "d2,vis.tn:b1,abc:f=avif:s=4096:r=150x150;            │
│        vis.sd:b1,def:f=avif:s=32768:r=640x480;          │
│        vis.md:b1,ghi:f=avif:s=262144:r=1920x1080"       │
│   ↓ references ↓                                        │
├─────────────────────────────────────────────────────────┤
│ Level 2: Variant IDs                                    │
│   b1~abc123... (vis.tn)                                 │
│   b1~def456... (vis.sd)                                 │
│   b1~ghi789... (vis.md)                                 │
│   ↑ SHA-256 hash of ↓                                   │
├─────────────────────────────────────────────────────────┤
│ Level 1: Blob Data (raw bytes)                          │
│   <AVIF encoded image data - tn variant>                │
│   <AVIF encoded image data - sd variant>                │
│   <AVIF encoded image data - md variant>                │
└─────────────────────────────────────────────────────────┘

Level 1: Blob Data

Raw bytes of actual content (images, videos, documents).

Level 2: Variant IDs (b1~…)

Content-addressed blob identifiers computed as SHA256(blob_bytes).

Level 3: File Descriptor (d2,…)

Encoded string listing all available variants of a file.

Format (variants separated by ;, fields by :):

d2,{class}.{variant}:{variant_id}:f={format}:s={size}:r={width}x{height};...

Example:

d2,vis.tn:b1,abc123:f=avif:s=4096:r=150x150;vis.sd:b1,def456:f=avif:s=32768:r=640x480

The d2 format uses , as the ID separator inside variant IDs (b1,abc123). An optional leading R={root} field carries the document-tree root.

Level 4: File ID (f1~…)

Content-addressed file identifier computed as SHA256(descriptor_string).

Level 5: Action Token (JWT)

Cryptographically signed JSON Web Token representing a user action.

Structure:

{
  "header": {
    "alg": "ES384",
    "typ": "JWT"
  },
  "payload": {
    "iss": "alice.example.com",
    "k": "250101",
    "t": "POST:IMG",
    "c": "Check out this amazing photo!",
    "a": ["f1~Qo2E3G8TJZ..."],
    "iat": 1738483100
  },
  "signature": "<ES384 signature with alice's private key>"
}

Level 6: Action ID (a1~…)

Content-addressed action identifier computed as SHA256(complete_jwt_token).

Hash Versioning Scheme

All identifiers use a versioned prefix format for future-proofing:

{prefix}{version}~{base64_encoded_hash}

Current Prefixes

Prefix Resource Type Hash Input Example
a1~ Action Entire JWT token a1~8kR3mN9pQ2vL6xW...
f1~ File File descriptor string f1~Qo2E3G8TJZ2HTGh...
d2, Descriptor (not a hash, the encoded format itself) d2,vis.tn:b1,abc:f=avif:...
b1~ Blob Blob bytes (raw data) b1~abc123def456ghi...

Version Scheme

  • Version 1: SHA-256 with base64url encoding (no padding)
  • Future versions: Can upgrade to SHA-3, BLAKE3, etc.
  • Backward compatibility: Old content remains valid forever
  • Algorithm agility: Migrate to new algorithms without breaking existing references

Example upgrade path:

a1~...  (SHA-256)
a2~...  (SHA-3)
a3~...  (BLAKE3)

Merkle Tree Properties

Content-addressing gives the tree several properties for free:

  • Immutability: Changing any content changes its ID, so the original is never overwritten — edits produce a new action with a new a1~... ID.
  • Tamper-evidence: A modification anywhere propagates upward. Altering a thumbnail changes its b1~... ID, which changes the descriptor, the f1~... file ID, and breaks the post’s attachment reference — verification fails.
  • Deduplication: Identical content yields identical IDs, so a file shared by many users is stored once.

Chain of Trust

Each reply references its parent by content hash (the p claim), binding a thread together cryptographically:

Post (a1~abc...)
  ↑ p (parent)
Comment (a1~def...)
  ↑ p (parent)
Reply (a1~ghi...)

Modifying the Post would change its ID and break the Comment’s reference, which would break the Reply’s — the whole thread is bound together.

Proof of Authenticity

Cloudillo provides two complementary layers of proof:

Layer 1: Author Identity (Cryptographic Signatures)

Action tokens are signed with ES384 (ECDSA with P-384 curve and SHA-384 hash):

Action Token = Header.Payload.Signature

Signature = ECDSA_sign(SHA384(Header.Payload), alice_private_key)

Verification:

  1. Fetch alice’s public keys from https://cl-o.alice.example.com/api/me (the keys array)
  2. Verify signature using public key
  3. ✓ Proves Alice created this action

Layer 2: Content Integrity (Content Hashes)

All identifiers are SHA-256 hashes:

action_id = SHA256(entire JWT token)
file_id = SHA256(descriptor string)
variant_id = SHA256(blob bytes)

Verification:

  1. Download content
  2. Recompute hash
  3. Compare with claimed identifier
  4. ✓ Proves content hasn’t been tampered with

Verification Example

Scenario: Verify a LIKE action on a POST with image attachment.

verify_like_action(like_id):
    // 1. Verify LIKE action
    like_token = fetch_action_token(like_id)
    verify_signature(like_token, bob_public_key)
    verify like_id == SHA256(like_token)
    ✓ Bob authored this LIKE, content is intact

    // 2. Verify parent POST action (p = parent claim)
    post_id = like_token.p
    post_token = fetch_action_token(post_id)
    verify_signature(post_token, alice_public_key)
    verify post_id == SHA256(post_token)
    ✓ Alice authored this POST, content is intact

    // 3. Verify file attachment (a = attachments claim)
    file_id = post_token.a[0]
    descriptor = fetch_descriptor(file_id)
    verify file_id == SHA256(descriptor)
    ✓ File descriptor is intact

    // 4. Verify each variant blob
    variants = parse_descriptor(descriptor)
    for each variant:
        blob_data = fetch_blob(variant.blob_id)
        verify variant.blob_id == SHA256(blob_data)
        verify blob_data.size == variant.size
    ✓ All image variants are intact

    RESULT: Complete chain verified!

DAG Structure

Cloudillo’s action system forms a Directed Acyclic Graph (DAG) with these properties:

Multiple Roots

Unlike a traditional tree with one root, Cloudillo has multiple independent threads:

Post 1 (a1~abc...)                Post 2 (a1~xyz...)
│                                  │
├─ Comment 1.1 (a1~def...)        ├─ Comment 2.1 (a1~uvw...)
│  │                               │
│  ├─ Reply 1.1.1 (a1~ghi...)     └─ Like 2.1 (a1~rst...)
│  │                                  (parent: a1~uvw...)
│  └─ Reply 1.1.2 (a1~jkl...)
│
├─ Comment 1.2 (a1~mno...)
│
└─ Like 1 (a1~pqr...)
   (parent: a1~abc...)

Each top-level post is a root node. Comments and reactions form child nodes.

Performance Implications

Caching Strategy

Content-addressed data is immutable, enabling aggressive caching:

GET /api/files/f1~Qo2E3G8TJZ...

Response Headers:
  Cache-Control: public, max-age=31536000, immutable
  Content-Type: image/avif

Benefits:

  • Browsers cache forever (max-age = 1 year)
  • CDN cache forever
  • No cache invalidation needed
  • Reduces bandwidth for federated instances

Security Considerations

Trust Model

Cloudillo’s merkle tree creates a trustless verification model:

What Verification Method Trust Required
Author identity JWT signature (ES384) DNS + Public key infrastructure
Content integrity SHA-256 hash None (pure mathematics)
Parent references Content hashes None (pure mathematics)
Attachment integrity SHA-256 hash chain None (pure mathematics)

Storage providers don’t need to be trusted: Even if a storage provider is malicious, they cannot:

  • Modify content without breaking hashes ✗
  • Forge signatures without private keys ✗
  • Create false parent references ✗
  • Tamper with attachments without detection ✗

Users verify everything cryptographically—no trust required.

Attack Resistance

Known attacks and mitigations:

Attack Mitigation
Modify action content Hash mismatch detected
Forge author signature Signature verification fails
Swap file attachment Hash mismatch detected
Modify parent reference Breaks cryptographic chain
Replay old actions Timestamp validation, deduplication
Storage provider tampering Hash verification fails

See Also