What is Decentralized Storage, and Why Do We Need It

This article was written by Oort’s Director of Systems Engineering, Ventseslav Sapundzhiev.

What is Decentralized Storage?

In decentralized storage systems, storage responsibilities are shared among many independent operators, forming a single storage network.

The modern Internet is highly centralized. Over the last decade, several cloud-based service providers have made huge fortunes by providing distinct cloud-based offerings. However, due to their centralized nature, these networks are prone to the consequences of disasters that could quickly stem from the outage of a singular provider.

Source: Wikipedia, Content Delivery Network

To counter consolidation, developers need new foundational building blocks that are just as easy to compose. Decentralized data storage is one cornerstone that is a prerequisite for a more distributed network.

Key features of decentralized storage

Resiliency

The modern Internet is surprisingly and terrifyingly fragile because of its highly centralized structure. Web content today resides behind URLs, each belonging to a single definitive server at any time. If for some reason, that server goes offline caused of single points of failure, the content will instantly become inaccessible.

The centralized web also makes de-platforming and censorship easy, meaning what you create and publish online, or what you can see or access on the Internet, can be easily manipulated. For example, link rotation (which is precisely what it sounds like: you will visit another link when a link is broken or permanently unavailable) is widespread on today's Internet, making state-level censorship easier and enlarging the effects of distributed denial-of-service attacks that can now disrupt access to almost any file.

In an ideal decentralized system, the loss of an operator should not prevent access to previously stored and served content. By distributing responsibilities across many network nodes, decentralized systems also have a natural resilience to censorship and other denial-of-service attempts since there is no centralized target against which attackers can direct resources.

Efficiency

All computing architectures come with some strengths and some weaknesses: no single solution fits all possible use cases. Unfortunately, the modern network's emphasis on centralization is no different.

Today, a few centralized data centers in a small number of cities store most of our digital content. For example, if two users on the same network want to send messages to each other, those messages usually go to one of those data centers first. Suppose a hundred users are in the same room and watching the same video on their devices. In that case, they will use each log into a central server and download a hundred copies in parallel instead of downloading one copy and sharing it on the local network.

Simply put, decentralized storage makes it easier to share files without sending requests bouncing over the Internet to a few designated data centers. Instead, nodes establish connections with each other directly instead of using intermediaries. For example, connecting to nodes in other countries will still require multiple jumps, but nodes on the same network could share files directly. The ultimate goal of decentralized data storage systems would be to have so many nodes that anyone can find relatively local peer nodes for the information they seek.

Decentralized storage solutions introduce fundamentally different outlooks and much higher efficiency to the cloud service landscape. By bypassing sparse data centers, a distributed system can locate nodes much closer to end users than even modern content delivery networks, resulting in significantly faster file retrieval. Peer-to-peer file sharing over local area networks can also save valuable bandwidth, especially in areas with limited access to the Internet.

The Ideal Image of Decentralized Storage

While resiliency and efficiency are hallmarks of decentralized storage, there are several additional features that an ideal storage system can offer:

Accessibility

The ideal distributed system should be affordable and geographically accessible. Network participation should be easy, allowing as many nodes as possible to store and distribute files on behalf of the network around the globe.

Programmable

One of the critical aspects for decentralized storage service providers to succeed is the ability to provision and manage storage through code via APIs. Any competing system should be able to offer the same level of convenience.

Content Addressing

As discussed, URLs contain some inherent design tradeoffs. They describe the location of the data, not it's content.

Trustless

The trustless system allows for collaboration between two parties without needing to know each other or turn to a third party. Instead, the system's incentives drive the participants to behave in ways necessary for the network to function.

Verifiable

An ideal storage system should provide continuous proof that nodes are storing the data they promised. This type of verifiability is key to achieving trustlessness. Users would know their information is stored securely, privately, and permanently without any "trust" for the developers.

Open-sourced

Finally, the ideal distributed storage system is open: its code is open source and ready to be audited by anyone, encouraging developers to implement, build and improve.