DID & Nuri
Before you read more about Nuri and DID, we suggest you to have a look at the Verifier documentation, in order to understand key concepts that are used here.
A document has a URI, that we call Nuri (NextGraph URI) and that uses the DID scheme.
The DID method did:ng is pending registration with W3C DID Working Group, with a specification about how a DID resolver can apply the CRUD operations to create, read, update, and deactivate a DID document did:ng. Not to confuse a NextGraph Document with a DID document. A DID document means the cryptographic materials.
Here we will detail the DID method and its format, and how it relates to RDF.
Most of the identifiers used in this scheme are of the PubKey, SymKey or Digest types, which are all 32 bytes arrays. They are all preceded by a byte detailing the version number of the format, which is zero at the moment.
The byte array is reversed, and for this reason all the IDs end with the A letter. The arrays are encoded into base64_url strings and always have 44 characters. Here is an example of an ID : JQ5gCLoX_jalC9diTDCvx-Wu5ZQUcYWEE821nhVRMcEA
We will use here the placeholder [XXX] to denote one of those IDs in the format, and explain which part it is for.
Those IDs and Keys will be combined in different ways detailed below, in order to form different types of capabilities and links.
Capabilities
Some special format that need explanation :
-
peers : a serialisation of a vector of BrokerServer that each contain the IP and port and the PeerID of the broker (and a version number). this is the “locator” part of the URI which tells which brokers should be contacted to join the overlay.
-
ReadCap : an ObjectRef (id+key) towards the current Branch definition commit
-
OverlayLink: In case of an Inner Overlay, a ReadCap to the current Branch definition commit of the overlay branch of the store.
-
ReadToken : is a hash of a ReadCap, it is mostly used in ExtRequests and SPARQL requests from untrusted apps or federated SPARQL requests. it gives the same level of permission than the hashed ReadCap, without revealing its content.
-
PermaCap: a durable capability ( for read, write or query)
We omit here the prefix did:ng as it is common to all the schemes.
| did:ng capabilities | |
|---|---|
| Profile | :a/b/g:[profileid] (a storeid) |
| A profile’s Inbox | :a/b/g:[profileid]:p:[inboxInfo]:l:[peers] |
| A Document’s inbox | :o:[repoid]:p:[inboxInfo]:l:[peers] |
| fully qualified read-only Document Nuri | :o:[repoid]:r:[readcap]:v:[overlayID]:l:[peers] |
| fully qualified read-write Document Nuri | :o:[repoid]:r:[readcap]:w:[overlayLink]:l:[peers] |
| document accessed by Token | :o:[repoid]:n:[readtoken]:v:[overlayID]:l:[peers] |
| PermaCap | :o:[repoid]:s:[permacap]:v:[overlayID]:l:[peers] |
| specific commit | :o:[repoid]:c:[commitid]:k:[commitkey]:h:[topicid]:v:[overlayID]:l:[peers] |
| head with 2 commits | :o:[repoid]:c:[commitid]:k:[commitkey]:c:[commitid]:k:[commitkey]:h:[topicid]:v:[overlayID]:l:[peers] |
| named commit or branch | :o:[repoid]:a:[name]:r:[repo_readcap]:v:[overlayID]:l:[peers] |
| branchID | :o:[repoid]:b:[branchId]:r:[branch_readcap]:v:[overlayID]:l:[peers] |
| binary file | :j:[objectid]:k:[key]:v:[overlayID]:l:[peers] |
overlay and peers can be omitted if the capability is about a repo/object that belongs to the same store.
Nuri stored in triplestore
When those capabilities are stored inside an RDF document, they are decomposed into several parts, and each part is inserted as a separate triple. Here more details :
| part | corresponding triple(s) |
|---|---|
:a/b/g/o:[repoid]:p:[inboxInfo] | <did:ng:a/b/g/o:[repoid]> <ng:inbox> <did:ng:p:[inboxInfo]> |
:p:[inboxInfo]:l:[peers] | <did:ng:v:[overlayid]> <ng:locator> <did:ng:l:[peers]> |
:o:[repoid]:r:[readcap] | <did:ng:o:[repoid]> <ng:access> <did:ng:r:[readcap]> |
| (if readcap is a branch) | <did:ng:o:[repoid]> <ng:revision> <did:ng:b:[branchid]> |
| (if readcap is a branch) | <did:ng:b:[branchid]> <ng:access> <did:ng:r:[branch_readcap]> |
:o:[repoid]:v:[overlayid] | <did:ng:o:[repoid]> <ng:overlay> <did:ng:v:[overlayid]> |
:o:[repoid]:w:[overlaylink] | <did:ng:v:[overlayid]> <ng:inner> <did:ng:w:[overlaylink]> |
<did:ng:v:[overlayid]> <ng:locator> <did:ng:l:[peers]> | |
:o:[repoid]:n:[readtoken] | <did:ng:o:[repoid]> <ng:access> <did:ng:n:[readtoken]> |
:o:[repoid]:s:[permacap] | <did:ng:o:[repoid]> <ng:access> <did:ng:s:[permacap]> |
:o:[repoid]:c:[commitid]… | <did:ng:o:[repoid]> <ng:revision> <did:ng:c:[]:c:[]> |
<did:ng:c:[commitid]> <ng:access> <k:[key]:h:[topicid]> | |
:o:[repoid]:a:[name] | <did:ng:o:[repoid]> <ng:revision> <did:ng:a:[name]> |
:o:[repoid]:b:[branchid] | <did:ng:o:[repoid]> <ng:revision> <did:ng:b:[branchid]> |
<did:ng:b:[branchid]> <ng:access> <did:ng:r:[branch_readcap]> | |
:j:[objectid]:k:[key] | <did:ng:j:[objectid]> <ng:access> <did:ng:k:[key]:h:[topic]> |
<did:ng:j:[objectid]> <ng:overlay> <did:ng:v:[overlayid]> | |
| if it is an attachment | <did:ng:o:[repoid]> <ng:attachment> <did:ng:j:[objectid]> |
When a capability is added to an RDF document, 2 additional triples can indicate a special purpose for this capability:
-
<did:ng:o:[repoid]> <ng:includes> <did:ng:[o/a/b/c]>means that the<object>should be included, if needed it should be fetched and kept in “cache” in the User Storage, and when a query is made on the Document, that<object>should be included in the target graphs of the query. -
<did:ng:o:[repoid]> <ng:subscribe> <did:ng:[o/a/b]>if included, and the ng:subscribe predicate is present, the topic will be subscribed to and the branch will be kept in sync.
Otherwise, without those triples, the capability is here because the repo, branch, commit or object is referenced somewhere in the RDF (as subject or object) or in the discrete doc (a link or embedded media).
The decomposition into several triples enables to deduplicate some information (if there are several links to the same overlay, the ng:access and ng:locator triple will be the same, so it will deduplicate) and also enables that the URI used to reference the foreign resource, will only contain the RepoId or ObjectId, which is important, specially for RepoId, as with RDF we want to establish facts between resources, that can be traversed with SPARQL queries. The unique identifier of a resource is <did:ng:o:[repoid]> and not the full DID capability. No matter where the resource is actually stored (the locator and overlay parts of the DID capability), which access we possess for that resource (read or write) or which revision we want to use at the moment, the unique identifier of the resource should remain the same. This is why we never use the full DID capability as URI for subject or object in RDF. Instead we use the minimal form <did:ng:o:[repoid]> .
We should probably mention here also that NextGraph is fully compatible with <http(s)://…> URIs and that they can coexist in any Document. <ng:includes> might work for http resources that can be dereferenced online with a GET, but we are not going to implement that right now (not a priority) and <ng:subscribe> will never work for those URLs, but we thought of using <ng:refresh> "3600“ and <ng:cache> “3600" instead, which would refresh periodically the dereferenced resource in the former case, every hour, or would cache the resource after the first dereferencing for a period of 1h in the later case, but then would delete the cache to save space and would only dereference again if need be (before processing any query on the resource) and cache again for another hour. those 2 addition predicate are not planned for now as they are not a priority.
Identifiers
Here are the identifiers that can be used to refer to any resource anywhere in the graph. they are unique global identifiers.
| Identifiers | that can be used in subject or object of any triple |
|---|---|
<did:ng:a/b/g:[storeid]> | profile of a public, protected or group store |
<did:ng:o:[repoid]> | a document |
<did:ng:o:[repoid]#fragment> | fragment of a Document |
<did:ng:o:[repoid]:u:[randomid]> | for skolemized blank nodes |
<did:ng:o:[repoid]:t:[commitid]> | for commit nodes (not implemented yet) |
<did:ng:j:[objectid]> | a binary file |
did:ng:o:[repoid] can be omitted when the identifier is understood as being within a specified named graph, or when using a BASE in queries.
Nuri in SPARQL
The graph part of any quad in NextGraph is reserved. The GSP (Graph Store Protocol) is not accessible by any client. Still, it is possible to use the graph part in SPARQL Query and SPARQL Update, with some limitations :
-
for SPARQL Query: the named graph can only have the forms :
-
<did:ng:v:[overlayid]>for the whole store -
<did:ng:o:[repoid]:v:[overlayid]>for the main branch of a repo, -
<did:ng:o:[repoid]:v:[overlayid]:b:[branchid]>for a specific branch of a repo, -
<did:ng:o:[repoid]:v:[overlayid]:a:[name]>for a named branch or named commit, -
<did:ng:o:[repoid]:v:[overlayid]:c:[commitid]>for a specific HEAD (the :c: part can repeated if we want a multi-head). -
<did:ng:o:[repoid]:v:[overlayid]:n:[readtoken]>for a query with readtoken (to a repo, branch or store)
-
-
for SPARQL Update, the named graph can only have the forms :
-
<did:ng:o:[repoid]:v:[overlayid]>for the main branch of a repo, or -
<did:ng:o:[repoid]:v:[overlayid]:b:[branchid]>for a specific branch of a repo, -
<did:ng:o:[repoid]:v:[overlayid]:a:[name]>for a named branch.
-
-
it is not possible to create new documents with the SPARQL API, the App API should be used instead.
-
apps have access to a
<>graph name (BASE) which represents the doc attached to the App instance
when no default graph is included in the Query (with USING, GRAPH, FROM), the target passed in the App API call is used. Target URIs given in the following table can also be used in the graph part of a SPARQL query. Target can have the form :
Targets
| Target | URI | explanation |
|---|---|---|
| UserSite | did:ng:i | whole dataset of the user (User Storage) personal identity |
| PublicStore | did:ng:a | all the documents of the public store of the user personal identity |
| ProtectedStore | did:ng:b | idem for protected store |
| PrivateStore | did:ng:c | idem for private store |
| AllDialogs | did:ng:d | union of all the Dialogs of personal identity (all the DMs) |
| Dialog(String) | did:ng:d:[name] | shortname given locally to a Dialog (i.e. “bob”, “alice”) |
| did:ng:d:a/b/g:[ProfileID] | a Dialog with a specific ProfilerId | |
| AllGroups | did:ng:g | union of all group Stores and all their documents |
| Group(String) | did:ng:g:[name] | shortname given locally to a Group (i.e. “birthday_party”, “trip_to_NL”) |
| did:ng:g:g/o:[storeid] | union of all the documents of a specific group store | |
| Identity(UserId) | did:ng:i:[userid] | search inside the User Storage of another Identity of the user, that is present in their wallet and has been opened. all the URI described here (except :i) can be used as suffixes behind :i:[xxx] |
| did:ng:i:n:[name] | same as above but with the shortname for the identity (i.e “work“ ) | |
| a document | did:ng:o |
for all the above URIs that are stores, adding a trailing :r will restrict the graph to the root document of the store.
Special branches
| Branch target | URI suffix | explanation |
|---|---|---|
| Chat | :h | internal chat accessible to members |
| Stream | :s | feed of activity, following, reactions sent, new content advert, repost, boost |
| Comments | :m | all the comments on the document |
| BackLinks | :v | mentions, followers, inverse relationships, reactions received and backlinks |
| Context | :x | contains the JSON-LD context (prefixes → link to ontologies) |
| Follower | :h | list of followers |
| Following | :y | list of following |
Context and ontologies
As shown above, there is a Context branch in each Document.
What is it for ?
the Context branch is an RDF mini file that contains some triples describing the JSON-LD Context. it uses the JSON-LD Vocabulary, and can be retrieved in its usual JSON-LD format.
It expresses the mapping of prefixes to the URI of the ontologies used in the document. It is common to all branches of the document. In every Header of a Document, the predicate <ng:x> contains as object the full link (DID cap) needed in order to open the branch and read the context. If the same context is shared across documents, then this predicate can point to a common context branch that sits outside of the document. In this case the context branch can be empty.
In order to see and modify the triples of the context, the suffix :x should be added at the end of the URI of the document, as this is the shortcut to access the context branch.
Binary files
Binary files can be uploaded to an overlay, and then linked to a specific branch of a repo.
Their content is deduplicated at the level of the overlay.
If 2 branches/docs use the same binary, they should add it both in their respective branch (with AddFile). The content will only be uploaded and downloaded once.
Once this is done, there are 4 cases on how to use those files within the branch :
-
inside a rich text document, the file can be embedded (image, video, svg)
-
inside a JSON/XML data document, the file can be “linked” by just entering its URI in a string value.
-
inside the RDF document, the file can be linked as subject or object in a triple, by example to add a profile picture with the predicate
<ng:j>which is equivalent to<http://www.w3.org/2006/vcard/ns#photo>or<http://xmlns.com/foaf/0.1/img> -
if the file has to appear as attachment (downloadable) then it must be added with an
<ng:attachment>predicate.
All the files of a branch (added with the AddFile commit) will be listed by the meta predicate <ng:f>
System ontology
| predicate | type | format | description |
|---|---|---|---|
| ng:main | Nuri | did:ng:r:[readcap] | main branch (from the header branch) |
| ng:header | Nuri | did:ng:r:[readcap] | header branch (from any branch) |
| ng:access | Nuri | did:ng:r:[readcap] | readcap of a branch, gives option to subscribe to topic and retrieve all the content (domain did:ng:o:b) |
| did:ng:n:[readtoken] | same but without subscription possible | ||
| did:ng:s:[permacap] | a permacap : a traditional capability (like z-caps) | ||
| did:ng:k:[key]:h:[topicid] | key and optional topicid of a commit | ||
| ng:revision | Nuri | did:ng:c:[]:c:[] | tells we are interested in a specific HEADS |
| did:ng:a:[name] | idem with a specific named branch or commit | ||
| did:ng:b:[branchid] | idem with a specific branch ID | ||
| ng:overlay | Nuri | did:ng:v:[overlayid] | on which overlay ID (always outer) is the doc or object |
| ng:inner | Nuri | did:ng:w:[overlaylink] | used by members to join the inner overlay |
| ng:locator | Nuri | did:ng:l:[locator] | list of peers to contact in order to join the overlay |
| ng:inbox | Nuri | did:ng:p:[inboxInfo] | the details of inbox |
| ng:includes | Nuri URL | did:ng:[o/a/b/c]:[…] or http… | the doc/branch/commit/rdfs:Resource should be included, fetched if needed, and its named graph included |
| ng:subscribe | Nuri | did:ng:[o/a/b]:[…] | the referenced doc/branch should be subscribed to, using ng:access |
| ng:refresh | integer | for includes that are http(s) resources, the refresh interval | |
| ng:cache | integer | for includes that are http(s) resources, the timeout before clearing cache. (conflicts with ng:refresh) | |
| ng:federate | Nuri | the remote Nuri will be included in the graph traversals | |
| ng:attachment | Nuri | did:ng:j:[]:v:[] | a link to a file that is an attachment to the document |
| ng:dialog | Nuri | did:ng:d:i:[dialogid] | the dialog of a contact |
| ng:content | string | content of a comment or message (domain is did:ng:o:t) | |
| ng:correction | string | correction of a comment or message (domain is did:ng:o:t) | |
| ng:comment | Nuri | did:ng:o:b:[branch] did:ng:o:t:[token] | branch containing a comment on doc, or commit node |
| ng:message | Nuri | did:ng:o:b:[branch] did:ng:o:t:[token] | branch containing a message in chat, or commit node |
| ng:repliesto | Nuri | did:ng:o:b:[branch] did:ng:o:t:[token] | comment or message is in reply to another comment or message |
| ng:stores | Nuri | did:ng:o | a store stores a document (auto-generated) |
| ng:site | Nuri | did:ng:o | to link the public store from the protected store (optional) |
| ng:protected | Nuri | did:ng:o:v:r:l | link to protected store main branch, by example from a follow request |
| ng:follows | Nuri | did:ng:o | profile that is followed. needs :access, :overlay, :locator |
| ng:index | Nuri | did:ng:o:r | entrypoint of a service or app |