DID & Nuri

Before you read more about Nuri and DID, we suggest you to have a look at the Verifier documentation, in order to understand key concepts that are used here.

A document has a URI, that we call Nuri (NextGraph URI) and that uses the DID scheme.

The DID method did:ng is pending registration with W3C DID Working Group, with a specification about how a DID resolver can apply the CRUD operations to create, read, update, and deactivate a DID document did:ng. Not to confuse a NextGraph Document with a DID document. A DID document means the cryptographic materials.

Here we will detail the DID method and its format, and how it relates to RDF.

Most of the identifiers used in this scheme are of the PubKey, SymKey or Digest types, which are all 32 bytes arrays. They are all preceded by a byte detailing the version number of the format, which is zero at the moment.

The byte array is reversed, and for this reason all the IDs end with the A letter. The arrays are encoded into base64_url strings and always have 44 characters. Here is an example of an ID : JQ5gCLoX_jalC9diTDCvx-Wu5ZQUcYWEE821nhVRMcEA

We will use here the placeholder [XXX] to denote one of those IDs in the format, and explain which part it is for.

Those IDs and Keys will be combined in different ways detailed below, in order to form different types of capabilities and links.

Capabilities

Some special format that need explanation :

  • peers : a serialisation of a vector of BrokerServer that each contain the IP and port and the PeerID of the broker (and a version number). this is the “locator” part of the URI which tells which brokers should be contacted to join the overlay.

  • ReadCap : an ObjectRef (id+key) towards the current Branch definition commit

  • OverlayLink: In case of an Inner Overlay, a ReadCap to the current Branch definition commit of the overlay branch of the store.

  • ReadToken : is a hash of a ReadCap, it is mostly used in ExtRequests and SPARQL requests from untrusted apps or federated SPARQL requests. it gives the same level of permission than the hashed ReadCap, without revealing its content.

  • PermaCap: a durable capability ( for read, write or query)

We omit here the prefix did:ng as it is common to all the schemes.

did:ng capabilities
Profile:a/b/g:[profileid] (a storeid)
A profile’s Inbox:a/b/g:[profileid]:p:[inboxInfo]:l:[peers]
A Document’s inbox:o:[repoid]:p:[inboxInfo]:l:[peers]
fully qualified read-only Document Nuri:o:[repoid]:r:[readcap]:v:[overlayID]:l:[peers]
fully qualified read-write Document Nuri:o:[repoid]:r:[readcap]:w:[overlayLink]:l:[peers]
document accessed by Token:o:[repoid]:n:[readtoken]:v:[overlayID]:l:[peers]
PermaCap:o:[repoid]:s:[permacap]:v:[overlayID]:l:[peers]
specific commit:o:[repoid]:c:[commitid]:k:[commitkey]:h:[topicid]:v:[overlayID]:l:[peers]
head with 2 commits:o:[repoid]:c:[commitid]:k:[commitkey]:c:[commitid]:k:[commitkey]:h:[topicid]:v:[overlayID]:l:[peers]
named commit or branch:o:[repoid]:a:[name]:r:[repo_readcap]:v:[overlayID]:l:[peers]
branchID:o:[repoid]:b:[branchId]:r:[branch_readcap]:v:[overlayID]:l:[peers]
binary file:j:[objectid]:k:[key]:v:[overlayID]:l:[peers]

overlay and peers can be omitted if the capability is about a repo/object that belongs to the same store.

Nuri stored in triplestore

When those capabilities are stored inside an RDF document, they are decomposed into several parts, and each part is inserted as a separate triple. Here more details :

partcorresponding triple(s)
:a/b/g/o:[repoid]:p:[inboxInfo]<did:ng:a/b/g/o:[repoid]> <ng:inbox> <did:ng:p:[inboxInfo]>
:p:[inboxInfo]:l:[peers]<did:ng:v:[overlayid]> <ng:locator> <did:ng:l:[peers]>
:o:[repoid]:r:[readcap]<did:ng:o:[repoid]> <ng:access> <did:ng:r:[readcap]>
(if readcap is a branch)<did:ng:o:[repoid]> <ng:revision> <did:ng:b:[branchid]>
(if readcap is a branch)<did:ng:b:[branchid]> <ng:access> <did:ng:r:[branch_readcap]>
:o:[repoid]:v:[overlayid]<did:ng:o:[repoid]> <ng:overlay> <did:ng:v:[overlayid]>
:o:[repoid]:w:[overlaylink]<did:ng:v:[overlayid]> <ng:inner> <did:ng:w:[overlaylink]>
<did:ng:v:[overlayid]> <ng:locator> <did:ng:l:[peers]>
:o:[repoid]:n:[readtoken]<did:ng:o:[repoid]> <ng:access> <did:ng:n:[readtoken]>
:o:[repoid]:s:[permacap]<did:ng:o:[repoid]> <ng:access> <did:ng:s:[permacap]>
:o:[repoid]:c:[commitid]…<did:ng:o:[repoid]> <ng:revision> <did:ng:c:[]:c:[]>
<did:ng:c:[commitid]> <ng:access> <k:[key]:h:[topicid]>
:o:[repoid]:a:[name]<did:ng:o:[repoid]> <ng:revision> <did:ng:a:[name]>
:o:[repoid]:b:[branchid]<did:ng:o:[repoid]> <ng:revision> <did:ng:b:[branchid]>
<did:ng:b:[branchid]> <ng:access> <did:ng:r:[branch_readcap]>
:j:[objectid]:k:[key]<did:ng:j:[objectid]> <ng:access> <did:ng:k:[key]:h:[topic]>
<did:ng:j:[objectid]> <ng:overlay> <did:ng:v:[overlayid]>
if it is an attachment<did:ng:o:[repoid]> <ng:attachment> <did:ng:j:[objectid]>

When a capability is added to an RDF document, 2 additional triples can indicate a special purpose for this capability:

  • <did:ng:o:[repoid]> <ng:includes> <did:ng:[o/a/b/c]> means that the <object> should be included, if needed it should be fetched and kept in “cache” in the User Storage, and when a query is made on the Document, that <object> should be included in the target graphs of the query.

  • <did:ng:o:[repoid]> <ng:subscribe> <did:ng:[o/a/b]> if included, and the ng:subscribe predicate is present, the topic will be subscribed to and the branch will be kept in sync.

Otherwise, without those triples, the capability is here because the repo, branch, commit or object is referenced somewhere in the RDF (as subject or object) or in the discrete doc (a link or embedded media).

The decomposition into several triples enables to deduplicate some information (if there are several links to the same overlay, the ng:access and ng:locator triple will be the same, so it will deduplicate) and also enables that the URI used to reference the foreign resource, will only contain the RepoId or ObjectId, which is important, specially for RepoId, as with RDF we want to establish facts between resources, that can be traversed with SPARQL queries. The unique identifier of a resource is <did:ng:o:[repoid]> and not the full DID capability. No matter where the resource is actually stored (the locator and overlay parts of the DID capability), which access we possess for that resource (read or write) or which revision we want to use at the moment, the unique identifier of the resource should remain the same. This is why we never use the full DID capability as URI for subject or object in RDF. Instead we use the minimal form <did:ng:o:[repoid]> .

We should probably mention here also that NextGraph is fully compatible with <http(s)://…> URIs and that they can coexist in any Document. <ng:includes> might work for http resources that can be dereferenced online with a GET, but we are not going to implement that right now (not a priority) and <ng:subscribe> will never work for those URLs, but we thought of using <ng:refresh> "3600“ and <ng:cache> “3600" instead, which would refresh periodically the dereferenced resource in the former case, every hour, or would cache the resource after the first dereferencing for a period of 1h in the later case, but then would delete the cache to save space and would only dereference again if need be (before processing any query on the resource) and cache again for another hour. those 2 addition predicate are not planned for now as they are not a priority.

Identifiers

Here are the identifiers that can be used to refer to any resource anywhere in the graph. they are unique global identifiers.

Identifiersthat can be used in subject or object of any triple
<did:ng:a/b/g:[storeid]>profile of a public, protected or group store
<did:ng:o:[repoid]>a document
<did:ng:o:[repoid]#fragment>fragment of a Document
<did:ng:o:[repoid]:u:[randomid]>for skolemized blank nodes
<did:ng:o:[repoid]:t:[commitid]>for commit nodes (not implemented yet)
<did:ng:j:[objectid]>a binary file

did:ng:o:[repoid] can be omitted when the identifier is understood as being within a specified named graph, or when using a BASE in queries.

Nuri in SPARQL

The graph part of any quad in NextGraph is reserved. The GSP (Graph Store Protocol) is not accessible by any client. Still, it is possible to use the graph part in SPARQL Query and SPARQL Update, with some limitations :

  • for SPARQL Query: the named graph can only have the forms :

    • <did:ng:v:[overlayid]> for the whole store

    • <did:ng:o:[repoid]:v:[overlayid]> for the main branch of a repo,

    • <did:ng:o:[repoid]:v:[overlayid]:b:[branchid]> for a specific branch of a repo,

    • <did:ng:o:[repoid]:v:[overlayid]:a:[name]> for a named branch or named commit,

    • <did:ng:o:[repoid]:v:[overlayid]:c:[commitid]> for a specific HEAD (the :c: part can repeated if we want a multi-head).

    • <did:ng:o:[repoid]:v:[overlayid]:n:[readtoken]> for a query with readtoken (to a repo, branch or store)

  • for SPARQL Update, the named graph can only have the forms :

    • <did:ng:o:[repoid]:v:[overlayid]> for the main branch of a repo, or

    • <did:ng:o:[repoid]:v:[overlayid]:b:[branchid]> for a specific branch of a repo,

    • <did:ng:o:[repoid]:v:[overlayid]:a:[name]> for a named branch.

  • it is not possible to create new documents with the SPARQL API, the App API should be used instead.

  • apps have access to a <> graph name (BASE) which represents the doc attached to the App instance

when no default graph is included in the Query (with USING, GRAPH, FROM), the target passed in the App API call is used. Target URIs given in the following table can also be used in the graph part of a SPARQL query. Target can have the form :

Targets

TargetURIexplanation
UserSitedid:ng:iwhole dataset of the user (User Storage) personal identity
PublicStoredid:ng:aall the documents of the public store of the user personal identity
ProtectedStoredid:ng:bidem for protected store
PrivateStoredid:ng:cidem for private store
AllDialogsdid:ng:dunion of all the Dialogs of personal identity (all the DMs)
Dialog(String)did:ng:d:[name]shortname given locally to a Dialog (i.e. “bob”, “alice”)
did:ng:d:a/b/g:[ProfileID]a Dialog with a specific ProfilerId
AllGroupsdid:ng:gunion of all group Stores and all their documents
Group(String)did:ng:g:[name]shortname given locally to a Group (i.e. “birthday_party”, “trip_to_NL”)
did:ng:g:g/o:[storeid]union of all the documents of a specific group store
Identity(UserId)did:ng:i:[userid]search inside the User Storage of another Identity of the user, that is present in their wallet and has been opened. all the URI described here (except :i) can be used as suffixes behind :i:[xxx]
did:ng:i:n:[name]same as above but with the shortname for the identity (i.e “work“ )
a documentdid:ng:o

for all the above URIs that are stores, adding a trailing :r will restrict the graph to the root document of the store.

Special branches

Branch targetURI suffixexplanation
Chat:hinternal chat accessible to members
Stream:sfeed of activity, following, reactions sent, new content advert, repost, boost
Comments:mall the comments on the document
BackLinks:vmentions, followers, inverse relationships, reactions received and backlinks
Context:xcontains the JSON-LD context (prefixes → link to ontologies)
Follower:hlist of followers
Following:ylist of following

Context and ontologies

As shown above, there is a Context branch in each Document.

What is it for ?

the Context branch is an RDF mini file that contains some triples describing the JSON-LD Context. it uses the JSON-LD Vocabulary, and can be retrieved in its usual JSON-LD format.

It expresses the mapping of prefixes to the URI of the ontologies used in the document. It is common to all branches of the document. In every Header of a Document, the predicate <ng:x> contains as object the full link (DID cap) needed in order to open the branch and read the context. If the same context is shared across documents, then this predicate can point to a common context branch that sits outside of the document. In this case the context branch can be empty.

In order to see and modify the triples of the context, the suffix :x should be added at the end of the URI of the document, as this is the shortcut to access the context branch.

Binary files

Binary files can be uploaded to an overlay, and then linked to a specific branch of a repo.

Their content is deduplicated at the level of the overlay.

If 2 branches/docs use the same binary, they should add it both in their respective branch (with AddFile). The content will only be uploaded and downloaded once.

Once this is done, there are 4 cases on how to use those files within the branch :

  • inside a rich text document, the file can be embedded (image, video, svg)

  • inside a JSON/XML data document, the file can be “linked” by just entering its URI in a string value.

  • inside the RDF document, the file can be linked as subject or object in a triple, by example to add a profile picture with the predicate <ng:j> which is equivalent to <http://www.w3.org/2006/vcard/ns#photo> or <http://xmlns.com/foaf/0.1/img>

  • if the file has to appear as attachment (downloadable) then it must be added with an <ng:attachment> predicate.

All the files of a branch (added with the AddFile commit) will be listed by the meta predicate <ng:f>

System ontology

predicatetypeformatdescription
ng:mainNuridid:ng:r:[readcap]main branch (from the header branch)
ng:headerNuridid:ng:r:[readcap]header branch (from any branch)
ng:accessNuridid:ng:r:[readcap]readcap of a branch, gives option to
subscribe to topic and retrieve
all the content (domain did:ng:o:b)
did:ng:n:[readtoken]same but without subscription possible
did:ng:s:[permacap]a permacap : a traditional capability (like z-caps)
did:ng:k:[key]:h:[topicid]key and optional topicid of a commit
ng:revisionNuridid:ng:c:[]:c:[]tells we are interested in a specific HEADS
did:ng:a:[name]idem with a specific named branch or commit
did:ng:b:[branchid]idem with a specific branch ID
ng:overlayNuridid:ng:v:[overlayid]on which overlay ID (always outer)
is the doc or object
ng:innerNuridid:ng:w:[overlaylink]used by members to join the
inner overlay
ng:locatorNuridid:ng:l:[locator]list of peers to contact in order
to join the overlay
ng:inboxNuridid:ng:p:[inboxInfo]the details of inbox
ng:includesNuri
URL
did:ng:[o/a/b/c]:[…] or http…the doc/branch/commit/rdfs:Resource
should be included, fetched if needed,
and its named graph included
ng:subscribeNuridid:ng:[o/a/b]:[…]the referenced doc/branch should be
subscribed to, using ng:access
ng:refreshintegerfor includes that are http(s)
resources, the refresh interval
ng:cacheintegerfor includes that are http(s) resources,
the timeout before clearing cache.
(conflicts with ng:refresh)
ng:federateNurithe remote Nuri will be included
in the graph traversals
ng:attachmentNuridid:ng:j:[]:v:[]a link to a file that is an attachment
to the document
ng:dialogNuridid:ng:d:i:[dialogid]the dialog of a contact
ng:contentstringcontent of a comment or message
(domain is did:ng:o:t)
ng:correctionstringcorrection of a comment or message
(domain is did:ng:o:t)
ng:commentNuridid:ng:o:b:[branch]
did:ng:o:t:[token]
branch containing a comment
on doc, or commit node
ng:messageNuridid:ng:o:b:[branch]
did:ng:o:t:[token]
branch containing a message in chat,
or commit node
ng:repliestoNuridid:ng:o:b:[branch]
did:ng:o:t:[token]
comment or message is in reply to
another comment or message
ng:storesNuridid:ng:oa store stores a document
(auto-generated)
ng:siteNuridid:ng:oto link the public store from
the protected store (optional)
ng:protectedNuridid:ng:o:v:r:llink to protected store main branch,
by example from a follow request
ng:followsNuridid:ng:oprofile that is followed. needs
:access, :overlay, :locator
ng:indexNuridid:ng:o:rentrypoint of a service or app