How to Create Your Own Decentralized Messenger Protocol

Let's say you want to create a system where each server can "join" a network and start exchanging something with other members within that network. How would you approach such a task? You could create a small centralized orchestrator that will accept join requests and handle that for the recipients, but that's not decentralized. What if we want a truly "plug and play" solution without any centralized servers? One such solution is federation . What is federation A good example would be SMTP (the protocol for sending emails). It's mostly decentralized. Anyone can run their own mail server. There is no single central authority that says "yes! this server can have email, but that one? NOOO!". Each server communicates using a well-known protocol and exchanges information server-to-server. You can even call it an "API with known payload schemas", although that is a bit of a simplification. Let's go with the email thing and start with the obvious. Have you ever considered why email addresses contain an @ part? Why can't it just be my_nick ? It's necessary to know which domain is responsible for that identity! duh! That domain plays a very important role in the whole federation exchange. That's because if you write from your email, let's say [email protected] , to [email protected] , the sending mail server will take that y.com part, do DNS discovery for MX records, connect to the mail server responsible for that domain, and start exchanging messages. This is the core idea of federation: servers talk to each other directly after some "discovery process". In our small email example, the discovery part would be a DNS lookup for MX records. Federated Messenger Protocol Ok, but we came here to create our own messenger protocol, so let's start. First, we need to gather some requirements like good system (and in this case, protocol) designers. Let's assume that we want to have end-to-end encryption - so the server should only see encrypted message data, and end clients will handle encryption and decryption. We also don't want to blindly trust anyone who sends an HTTP request and says "hello, I am x.com".So we need a way to verify that a federation request was signed by a server that controls the source domain. Also, we don't want to make it super complicated, so let's drop any fancy things like attachments, multi-device solutions, groups, read receipts, etc. For now, we want to have a decentralized, simple text exchange with only one device. Welcome to the 90s! :D That should be the basic minimum for our needs. So how do we even start doing it? Let's start with the easiest, and then move to more complex stuff. User Identity Let's also assume that every server already has some users with registration, login, etc. So we don't spend a million years doing everything from scratch. We need to enforce that user identity will be like an email. So @ , otherwise our federation will not know who to call. The important thing here is that [email protected] is not an email address. It only uses the same shape because the shape is useful. The domain tells other servers where to discover federation metadata. We could go with, for example, x.com > adam , x.com#adam , or adam on x.com . It's your call. Discovery Now the first real work. Discovery is the part similar to the MX record lookup process mentioned earlier. We have full freedom on how we want to handle this, so let's do it with a simple file on the server. For this, we can use the typical .well-known directory. In our case, we can call our protocol lnchat (from LearN chat). This will be our first requirement for the protocol. Server MUST provide a file under: GET /.well-known/lnchat.json Inside we can put the version and other important information that we will craft later in the article. For now, we can go with only the version: { "protocol": "lnchat.federation.v1" } By the way, it's good practice to always version protocols. It enforces the contract and saves you from a lot of headaches. With this, servers can check if the servers we want to reach even support our protocol. If yes, they can pull all the necessary information from this one .well-known file. End to End Since we want to have end-to-end encryption, we need client-side encryption keys. Usually that means one private key that stays with the client, and one public key that the user's server can share with other servers. The public key is used by senders as input to encryption for that user. The private key is used by the receiver to decrypt it. The private key MUST NOT be sent to the server as plaintext. If the server can read the private key, then the server can read the messages, and we don't really have end-to-end encryption anymore. Important: if a server is responsible for publishing users' public keys, a malicious or compromised server could lie about a key. Real end-to-end encryption needs some way to notice that, for example contact key warnings, fingerprints, TOFU, or key transparency. So we need an endpoint for sharing users' public keys. Every server should implement it as part of our protocol. So let's craft it! First, let's add information to our discovery file about where to look for the endpoint. This gives every server the ability to choose its own URLs. It's better not to enforce strict paths here as it may collide with already existing routes on the server. { "protocol": "lnchat.federation.v1", "endpoints": { "resolve_users": "https://example.com/federation/v1/users/resolve" } } But the resolve_users URL could be a different URL on every server. Resolving users The resolve_users endpoint will be responsible for returning current public encryption keys for users from that server. Other servers will call it before sending a message, so they know which key should be used for which recipient. We can craft a schema for this endpoint like this: { "protocol": "lnchat.federation.v1", "users": [" [email protected] ", " [email protected] "] } Why an array of users and not just one? Let's imagine that it will be used in high-volume servers. We should offer a way to ease the performance. If the server can resolve several users in one request, it will be much more performant - fewer HTTP calls. And for the response, let's do this: { "protocol": "lnchat.federation.v1", "status": "ok", "users": [ { "user": " [email protected] ", "status": "ok", "key": { "key_id": "adam-enc-2026-07-a", "public_key": "base64url-raw-32-byte-x25519-public-key" } }, { "user": " [email protected] ", "status": "not_deliverable" } ] } So the user can expose a key with a key_id and public_key . Why do we need a key_id ? Well, imagine how the flow would look like. The client gets the user's public key and probably stores it in some cache, then sends messages using that key. What if that key changes? The receiver will never decrypt it. We need to have a way to distinguish keys from each other and even show a warning that the key changed! Maybe it was compromised! I also added two statuses: ok and not_deliverable . ok is self-explanatory, but I added not_deliverable for cases where we don't have a public key for that user yet. Remember that the user needs to generate those keys. We cannot safely do it on the backend if we want real end-to-end encryption. So there is a chance that we already have users who never generated them. We should clearly state that this user is not a valid user for receiving encrypted messages. Publishing keys For now, we only handled the endpoint for server-to-server key exchange. But how will that public key appear in the first place? The user can use the Web Crypto API to generate keys, store the private key locally, and send the public key to the backend to keep it in the DB. const keyPair = await window.crypto.subtle.generateKey( { name: "X25519" }, false, ["deriveBits"] ); con