Client Protocol

All our protocols and formats use the binary codec called BARE.

The Client Protocol is used by the Verifier in order to contact the Broker of the User.

It maintains this connection throughout the session that was opened by the User (by opening the wallet in the app, by example).

From this connection, the Verifier gets all the pushed updates (called Events), after it subscribed to some topics.

The verifier also sends the updates that it wants to publish, in the form of an Event to the Pub/Sub Topic, and the Broker deals with forwarding this Event to all the other devices and users that have subscribed to this topic.

The communication on the Client Protocol is using a WebSocket, encrypted from within with the Noise Protocol.

In addition, all the Events, that are sent and received with this protocol, are also encrypted end-to-end.

For now, the Verifier only connects to one Broker, but for redundancy and failsafe purposes, it will be possible in the future that it tries to connect to several Brokers.

But one rule should always be followed: for any given Overlay, a User can only participate in this Overlay from one and only one Broker at the same time.

Let’s dive into the format of the messages and actions/commands that can be exchanged on the Client Protocol.

The initiation of the connection is common to all protocols, and involves some Noise handshake. It isn’t detailed here, please refer to the code for now. We will provide more documentation on that part later on.

For a reference of the common types, please refer to the Repo format documentation

ClientMessage

All the subsequent messages sent and receive on this protocol, are encapsulated inside a ClientMessage.

The ClientRequestV0.id is set by the requester in an incremental way. Request IDs must be unique by session. They should start from 1 after every start of a new session. This ID is present in the response, in order to match requests and responses.

enum ClientMessage {
    V0(ClientMessageV0),
}

struct ClientMessageV0 {
    overlay: OverlayId,
    content: ClientMessageContentV0,
    /// Optional padding
    padding: Vec<u8>,
}

enum ClientMessageContentV0 {
    ClientRequest(ClientRequest),
    ClientResponse(ClientResponse),
    ForwardedEvent(Event),
    ForwardedBlock(Block),
}

enum ClientRequest {
    V0(ClientRequestV0),
}

struct ClientRequestV0 {
    /// Request ID
    id: i64,

    /// Request content
    content: ClientRequestContentV0,
}

enum ClientRequestContentV0 {
    OpenRepo(OpenRepo),
    PinRepo(PinRepo),
    UnpinRepo(UnpinRepo),
    RepoPinStatusReq(RepoPinStatusReq),

    // once repo is opened or pinned:
    TopicSub(TopicSub),
    TopicUnsub(TopicUnsub),

    BlocksExist(BlocksExist),
    BlocksGet(BlocksGet),
    CommitGet(CommitGet),
    TopicSyncReq(TopicSyncReq),

    // For Pinned Repos only :
    ObjectPin(ObjectPin),
    ObjectUnpin(ObjectUnpin),
    ObjectDel(ObjectDel),

    // For InnerOverlay's only :
    BlocksPut(BlocksPut),
    PublishEvent(PublishEvent),

    WalletPutExport(WalletPutExport),
}

enum ClientResponse {
    V0(ClientResponseV0),
}

struct ClientResponseV0 {
    /// Request ID
    id: i64,

    /// Result (including but not limited to ServerError)
    result: u16,

    /// Response content
    content: ClientResponseContentV0,
}

enum ClientResponseContentV0 {
    EmptyResponse,
    Block(Block),
    RepoOpened(RepoOpened),
    TopicSubRes(TopicSubRes),
    TopicSyncRes(TopicSyncRes),
    BlocksFound(BlocksFound),
    RepoPinStatus(RepoPinStatus),
}

• ClientResponseV0.result :
  • 0 means success
  • 1 means PartialContent (the response is a stream. each element in the stream will have this result code)
  • 2 means EndOfStream (that’s the last response in the stream)
  • 3 means False
  • 4 and above are errors. for the list, see ng-repo/src/errors.rs starting at line 265.

When an error occurs (result >= 3), the content is of the type ClientResponseContentV0::EmptyResponse

• ClientMessageContentV0::ForwardedEvent(Event) is used by the Broker when it wants to push the events it received from other brokers or devices, down to the Client.

BlocksGet

Request a Block by ID.

Can be used to retrieve the content of a ReadCap, or any Commit that didn’t arrive from the Pub/Sub already. Images and Binary files also use BlocksGet when opened, read and downloaded. ReadCaps are preferably retrieved with CommitGet though.

commit_header_key is always set to None in the reply when request is made on OuterOverlay of Protected or Group overlays.

Request

enum BlocksGet {
    V0(BlocksGetV0),
}

struct BlocksGetV0 {
    /// Block IDs to request
    ids: Vec<BlockId>,

    /// Whether or not to include all children recursively
    include_children: bool,

    topic: Option<TopicId>,
}

• topic : (optional) Topic the object is referenced from, if it is known by the requester. Can be used by the Broker to do a BlockSearchTopic in the core overlay, in case the block is not present locally in the Broker.

Response

The response is a stream of Blocks.

CommitGet

Request a Commit by ID

Replied with a stream of Blocks.

commit_header_key of the replied blocks is always set to None when request is made on OuterOverlay of Protected or Group overlays.

The difference with BlocksGet is that the Broker will try to return all the commit blocks as they were sent in the Pub/Sub Event, if it has the event. This will help in receiving all the blocks (including the header and body blocks) in one ClientProtocol message, while a BlocksGet would inevitably return only the blocks of the ObjectContent, and not the header nor the body. And the load() would fail with CommitLoadError::MissingBlocks. That’s what happens when the Commit is not present in the pubsub, and then we need to default to using BlocksGet instead.

Request

enum CommitGet {
    V0(CommitGetV0),
}

struct CommitGetV0 {
    /// Commit ID to request
    id: ObjectId,

    /// Topic the commit is referenced from, if it is known by the requester.
    /// can be used to do a BlockSearchTopic in the core overlay.
    topic: Option<TopicId>,

}

RepoPinStatus

Request the pinning status for a repo on the broker (for the current user’s session).

Returns an error code if not pinned, otherwise returns a RepoPinStatusV0. The overlay entered in ClientMessage is important. if it is the outer, only outer pinning will be checked. If it is the inner overlay, only the inner pinning will be checked.

Request

struct RepoPinStatusReqV0 {
    /// Repo Hash
    hash: RepoHash,

}

Response

enum RepoPinStatus {
    V0(RepoPinStatusV0),
}

struct RepoPinStatusV0 {
    /// Repo Hash
    hash: RepoHash,

    /// only possible for Inner overlays
    expose_outer: bool,

    /// list of topics that are subscribed to
    topics: Vec<TopicSubRes>,
}

enum TopicSubRes {
    V0(TopicSubResV0),
}

struct TopicSubResV0 {
    /// Topic subscribed
    topic: TopicId,
    /// current HEADS at the broker
    known_heads: Vec<ObjectId>,
    /// was the topics subscribed as publisher
    publisher: bool,

    commits_nbr: u64,
}

• commits_nbr : used for a subsequent TopicSyncReq in order to properly size the bloomfilter

PinRepo

Request to pin a repo on the broker.

When client will disconnect, the subscriptions and publisherAdvert of the topics will remain active on the broker. Replied with a RepoOpened

Request

enum PinRepo {
    V0(PinRepoV0),
}

struct PinRepoV0 {
    /// Repo Hash
    hash: RepoHash,

    overlay: OverlayAccess,

    overlay_root_topic: Option<TopicId>,

    expose_outer: bool,

    peers: Vec<PeerAdvert>,

    max_peer_count: u16,

    ro_topics: Vec<TopicId>,

    rw_topics: Vec<PublisherAdvert>,
}

enum PublisherAdvert {
    V0(PublisherAdvertV0),
}

struct PublisherAdvertV0 {
    content: PublisherAdvertContentV0,

    /// Signature over content by topic key
    sig: Sig,
}

struct PublisherAdvertContentV0 {
    /// Topic public key
    topic: TopicId,

    /// Peer public key
    peer: DirectPeerId,
}

enum OverlayAccess {

    ReadOnly(OverlayId),

    ReadWrite((OverlayId, OverlayId)),

    WriteOnly(OverlayId),
}

• OverlayAccess::ReadOnly : The repo will be accessed on the Outer Overlay in Read Only mode. This can be used for Public, Protected or Group overlays. Value should be an OverlayId::Outer

• OverlayAccess::ReadWrite : The repo will be accessed on the Inner Overlay in Write mode, and the associated Outer overlay is also given. This is used for Public, Protected and Group overlays. First value in tuple should be the OverlayId::Inner, second the OverlayId::Outer. The overlay that should be used in the ClientMessageV0 is the InnerOverlay

• OverlayAccess::WriteOnly : The repo will be accessed on the Inner Overlay in Write mode, and it doesn’t have an Outer overlay. This is used for Private and Dialog overlays. Value should be an OverlayId::Inner

• PinRepoV0.overlay_root_topic : Root topic of the overlay, used to listen to overlay refreshes. Only set for inner overlays (RW or WO) overlays. Not implemented yet

• PinRepoV0.expose_outer : only possible for RW overlays. not allowed for private or dialog overlay. not implemented yet

• PinRepoV0.peers : Broker peers to connect to in order to join the overlay. If the repo has previously been opened (during the same session) or if it is a private overlay, then peers info can be omitted. If there are no known peers in the overlay yet, vector is left empty (creation of a store, or repo in a store that is owned by user).

• PinRepoV0.max_peer_count : Maximum number of peers to connect to for this overlay (only valid for an inner (RW/WO) overlay). not implemented yet

• PinRepoV0.ro_topics : list of topics that should be subscribed to. If the repo has previously been opened (during the same session) and the list of RO topics does not need to be modified, then ro_topics info can be omitted

• PinRepoV0.rw_topics : list of topics for which the client will be a publisher. Only possible with inner (RW or WO) overlays. If the repo has previously been opened (during the same session) then rw_topics info can be omitted

Response

type RepoOpened = Vec<TopicSubRes>;

for more details about TopicSubRes see above in RepoPinStatus.

TopicSub

Request subscription to a Topic of an already opened or pinned Repo

replied with a TopicSubRes containing the current heads that should be used to do a TopicSync

Request

enum TopicSub {
    V0(TopicSubV0),
}

struct TopicSubV0 {
    /// Topic to subscribe to
    topic: TopicId,

    /// Hash of the repo that was previously opened or pinned
    repo_hash: RepoHash,

    /// Publisher needs to provide a signed `PublisherAdvert`
    // for the PeerId of the broker
    publisher: Option<PublisherAdvert>,
}

Response

A TopicSubRes. see above in RepoPinStatus for more details.

TopicSyncReq

Topic synchronization request

The broker will send all the missing events (or commits if it cannot find events) that are absent from the DAG of the Client.

The events/commits are ordered respecting their causal relation one with another.

Replied with a stream of TopicSyncRes

Request

enum TopicSyncReq {
    V0(TopicSyncReqV0),
}

struct TopicSyncReqV0 {
    /// Topic public key
    topic: TopicId,

    /// Fully synchronized until these commits
    known_heads: Vec<ObjectId>,

    /// Stop synchronizing when these commits are met.
    /// if empty, the local HEAD at the responder is used instead
    target_heads: Vec<ObjectId>,

    /// optional Bloom filter of all the commit IDs present locally
    known_commits: Option<BloomFilter>,

}

Response

enum TopicSyncRes {
    V0(TopicSyncResV0),
}

enum TopicSyncResV0 {
    Event(Event),
    Block(Block),
}

BlocksExist

Request to know if some blocks are present locally on the responder

used by a Client before publishing an event with FILES, to know what to push, and save bandwidth if the blocks are already present on the Broker (content deduplication). commits without FILES cannot be deduplicated because they are unique, due to their unique position in the DAG, and the embedded BranchId.

Replied with BlocksFound

Request

enum BlocksExist {
    V0(BlocksExistV0),
}

struct BlocksExistV0 {
    /// Ids of Blocks to check
    blocks: Vec<BlockId>,
}

Response

enum BlocksFound {
    V0(BlocksFoundV0),
}

struct BlocksFoundV0 {
    /// Ids of Blocks that were found locally
    /// this might be deprecated soon (as it is useless)
    found: Vec<BlockId>,

    /// Ids of Blocks that were missing locally
    missing: Vec<BlockId>,
}

BlocksPut

Request to store one or more blocks on the broker

Replied with an EmptyResponse

Request

enum BlocksPut {
    V0(BlocksPutV0),
}

struct BlocksPutV0 {
    /// Blocks to store
    blocks: Vec<Block>,

}

PublishEvent

Request to publish an event in pub/sub

Replied with an EmptyResponse

Request

struct PublishEvent(Event);