Datashards is a generalized and universal system for storing encrypted immutable data through [[!IDSC]] and mutable data through [[!MDSC]]. This document provides an overview of the system.
The web has been a great tool for the wide distribution of content. Unfortunately, reliance on HTTP based URIs has meant that content is liable to disappear or change unexpectedly. Additionally, access control for private content is difficult to make secure on the contemporary web.
Content Addressed Storage allows for us to refer to objects by their hash without binding that object to a specific location, and may be referred to with such addresses as URNs or IPFS addresses. The advantage of content addressing is that by naming documents by their contents, they can live anywhere: whether stored on a usb key, a peer to peer network, or at a specific website, a content-addressed identifier has the same name everywhere. If Bob takes a photo of Alice's cat, Alice may wish to be able to refer to this cat photo regardless of whether or not Bob's server goes down.
Unfortunately, content addressing alone is not a sufficient solution. Perhaps the photo of Alice's cat contains visual details of Alice's house which Alice would prefer to be only seen by her friends and family. Alice and Bob would like to maintain this level of privacy, but how to do so while upholding the advantages provided by a content addressed system?
Datashards provides a solution by combining encryption with uniform
chunking.
By encrypting the document, Alice and Bob can choose to store and
distribute the file in less trusted locations.
By chunking the file, an adversary has less of an opportunity to guess
if it is a file of specific interest simply by comparing its file size.
With Datashards' [[!IDSC]] layer, Alice and Bob can pass around a single
idsc: style URI which composes the necessary information
and authority to retrieve, reassemble, and decrypt the file through
their IDSC clients.
Meanwhile the IDSC clients ask whatever stores they are configured
with for the relevant encrypted chunks without revealing to the stores
what the decryption keys are.
Immutable files may be sufficient for a timeless cat photo, but many other documents are subject to change. Alice and Bob are both playing a role playing game with Alice's character sheet described as a document. Alice would like to give Bob access to read her character sheet as easily as copying and pasting a URL, but retain the ability to write for herself. Meanwhile she would like to work with a datashards provider that is able to verify valid updates to her document without knowing anything about its contents (other than knowing that changes have occured). Datashards provides such tooling through [[!MDSC]].
TODO: explain this
.------------------------------------------------.
| MDSC |
|================================================|
| mdsc: uris |
| [write-rv cap] [read-v cap] [verify cap] |
|------------------------------------------------|
| client |
|------------------------------------------------|
| client registry "object" | server registry? |
'------------------------------------------------'
. .
|------------------------------'
|
V
_________ _________ _________
( (_) ( (_) ( (_)
\ \ \ \ \ \
) cert3 ) ~> ) cert2 ) ~> ) cert1 )
/ / / / / /
(_[enc]_(_) (_[enc]_(_) (_[enc]_(_)
. . .
| | |
V V V
.---------------------------------------.
| IDSC |
|=======================================|
| idsc:uri |
|---------------------------------------|
| client |
|---------------------------------------|
| client store "object" | server store? |
'---------------------------------------'
. .
|------------------------------'
V
.---. .---. .---.
;.---.; ;.---.; ;.---.;
||___|| ||___|| ||___||
.' '. .' '. .' '.
| entry | -> | data | -> | data |
| chunk | | chunk | | chunk |
'-------' '-------' '-------'