Network

Pub/Sub

As we already explained above, the network of NextGraph is composed of Brokers that talk to each other in the Core Network. Then Clients that are attached to one specific broker, only talk to that broker in a client-server manner. This is the 2-tier topology.

The brokers are always online and deal with events received from some client, that they forward to other brokers in the core network, that in turn dispatch them to the clients that are connected to them.

The events are end-to-end-encrypted and only the clients can decrypt them.

Events belong to a Pub-Sub topic. The broker is in charge of dealing with the maintenance of each Pub-Sub topic. Clients tell their broker that they want to subscribe to a specific topic in the pub-sub (read access), and can also tell their broker that they are going to publish in a specific topic (read-write access).

In order to subscribe to a Topic, the only thing the broker needs to know from the client, is the TopicID, which is a public key.

But when a client wants to publish in a topic, it has to prove that it is in possession of the private key of the topic, by signing the event with this private key.

The broker does not have this private key, but it will verify that the signature is valid, in regards to the public key (the topic ID).

In addition to that, the broker that forwards the events in the core network (in an overlay, to be more precise), needs to show to the other brokers that he is allowed to do so. The broker will obtain a signature by the topic privkey, over their own PeerID. Only the clients can sign this proof, and they will do it and give this proof to the broker, so it can enter the overlay. This is called the PublisherAdvert.

There is a whole Core protocol that deals with overlay joining and leaving, with topic subscriptions and publisher adverts, based on the LoCaPs paper, and we will not enter into much details here about that. This core network is modelled as a general (cyclic) undirected graph.

What is important to understand is that the brokers only see encrypted events that belong to a topic, that they forward those events to clients that have subscribed to the topic, and that’s pretty much all that the broker can do.

An event is sent for each commit, so it contains all the blocks of the commit itself, plus some additional blocks if needed.

The Broker stores those blocks locally if they are told to do so (by pinning), or if they are the first broker to publish this event (when the publishing client is connected to this broker), this is called “home pinning”.

Also, the brokers manage a list of clients that have created an account with them. This is mandatory because there is a legal contract agreement between clients and brokers (ToS) in the case of a public Broker Service Provider. In the case of a private self-hosted broker, the list of accounts is used to know who is allowed to use the broker, but there is no ToS, as registration works by invitation only.

Finally, the broker enforces the rule that a specific user, and all their devices, have to be connected to the same broker at any given time (and for a given overlay).

Apart from this rule, clients can connect to any broker that they want (if they have an account with them) and can even connect to multiple broker at a time (if on different overlays).

Usually, a client will only talk to one broker at a time, but in case redundancy is needed, the wallet can be configured to try to connect to several brokers.

The User can also decide to migrate to another broker, and transfer all their encrypted data there.

The events/commits are encrypted and fully portable, they can move to another broker easily.

To summarise: The brokers only deal with encrypted events, topicIDs, pub/sub subscription and publishers, overlays, other brokers in the core network, and the users/clients that have registered with them.

Brokers know nothing about concepts like Documents, Commits, Branches, Stores, or Wallets.

Broker

We distinguish 3 types of brokers:

• core broker : They form the core of the overlay, they must have a public IP interface and run 24/7. Core brokers should be shared resources. They are trust-less and enable anonymity of the users participating in an overlay, as their devices (phone, computer) are not connected directly to other users’ devices. Instead the core brokers are intermediaries that offer constant connectivity, NAT traversal solutions, mutualization of resources, and routing/caching/backup of the encrypted data. Core brokers can be self-hosted by individuals and companies, shared community hosting, or cloud SaaS services offered by businesses to customers. Core broker communicate between each other with the Core Protocol.

• edge broker : They are brokers ran as self-hosted by individuals or offices at the edge of the IP network (behind ADSL/VDSL/fiber subscriber lines) and are used by the local users on the LAN interface where they expose a server (they also expose a server on the public IP, for apps who would connect from the public IP network, and for other core/edge broker to contact them). They behave like the core brokers (they participate in the core network, on their public IP interface). An edge broker will most probably run a verifier too, which means that this broker is trusted by a user that stores his private keys on such machine.

• local broker: A broker that is running as a client and connects to another server broker (edge or core) with the Client Protocol. A client broker only connects to one server broker at a time, and runs in a local daemon or a native app (web app is also planned). A local broker will always be running a Verifier, and serves as a cache of blocks for the Verifier. The app connects to Verifier with the App Protocol.

The verifier can decrypt the commits and verify their validity. They need the encryption keys in order to do so. For this reason they are placed at the edge of the overlay, on machines trusted by the user.

Communication between peers is implemented with WebSockets on port 1440 (HTTP) without TLS (except in the webapp), as the transport channel encryption is implemented inside the WebSocket with Noise Protocol. This architecture is important as TLS requires centralized certificate mechanisms (and NextGraph is a totally decentralized ecosystem), and self-signed certificates are not easy to handle for end-users. We are planning to add also QUIC connections(without TLS) between brokers and for native apps, in another iteration (web apps will always stay with websocket and maybe will add webrtc support later on).

We at NextGraph, will provide a set of public brokers available in different locations. But anybody can run their own broker, self-hosted (coming soon), and without the need to buy a domain name. Brokers will always find each other and sync between each other, based on what we call an overlay.

Overlay

An overlay is like a network subspace where some peers (replicas and brokers) can interact. If those peers have joined the same overlay, they can talk to each other. If they are in disjoint overlays, they don’t even see each other.

A peer can participate in several overlay at the same time, specially the brokers that will be doing that all the time.

There is one Overlay for each Store, but we distinguish between the Outer Overlay and the Inner Overlay for each Store.

The Inner Overlay, where all the editing and signing happens, protected by a secret key that only editors can have. And the Outer Overlay which is used only for read purposes for external users. The Outer Overlay ID never changes, but the Inner Overlay ID can change when we renew its epoch (after kicking out a member, by example).

The Outer Overlay lets anonymous or non-member users join the overlay and reach the brokers in order to submit Post messages, ExtRequests or ServiceRequests to them, and subscribe to the Pub/Sub topics. They can only perform read operations on the repositories. For privacy reasons, the non-member peers that connect to this overlay do not flood their PeerAdvert to the Overlay. Their IP address remains only known to the specific set of Brokers the non-member peer decided to connect to. The non-member peers can nevertheless ask to receive a full list of Brokers available in this overlay, with the options they offer for a specific topic, so they can decide to which broker they should send their request.

The Inner Overlay is used by all the peers and brokers of the members of the repository, meaning, the users who have a write access to the repository. Peers in this overlay know each other mutually (but their IDs are hidden).

Each store has one Inner Overlay, that regroups all the repositories of such store.

The brokers of each member of a repo, join the Inner overlay, and optionally the Outer overlay. Those who join both overlays serve as relays between the 2 overlays, exposing the data modified in the Inner overlay, into the read-only Outer overlay, the topic ids being identical across the 2 overlays.

An option in the configuration of each broker for that repo/store and user, tells if to join the Outer overlay or not.

NextGraph will offer in the future a unique Global Overlay (of Read type) used for indexing and broadcasting public Sites with a global audience.

Each store has a pair of overlays (Outer and Inner), except the private store that only has one Inner overlay and no Outer overlay.

We also have a pair of overlay for each Group, and only one Inner overlay for each Dialog.

The Outer overlay ID is just the Blake3 hash of the Store ID.

While the Inner overlay ID is a Blake3 keyed hash of the Store ID. The key used for the hash is derived from the ReadCapSecret of the Overlay branch of the Store.

We will now dive more into details about repositories, branches, blocks and commits, that compose the NextGraph Protocol

Here is an illustration of our Network Architecture.

A higher resolution version of this diagram can be found here in PNG format or in SVG format.