We all know how important it is to protect our personal data, especially with so many crimes happening because of leaked information. This is why choosing where to store your data is so crucial, especially in today’s digital world.
We usually store our data on centralized platforms. The fact that you might not know is that you simply upload your data and you don’t know where it’s stored or exactly how it’s being used. I have already discussed this issue in this article.
It is time for us to move forward and try Web3, a concept of decentralization that is very transparent and open to everyone. Web3 is decentralized, meaning ownership is not controlled by a central authority, and it focuses on the user using blockchain technology.
Walrus Decentralized Storage Service on Sui
We can develop programs, apps, or platforms in Web3. One platform you must try is Walrus. Walrus is a decentralized data storage platform built on the Sui blockchain. Walrus allows us to store, read, and manage large data. You can upload your documents, images, videos, and more.
What makes Walrus special is not just its function but the innovative system they created.
In Web3, there are two common ways to store data. The first one is Full Replication Method. This method forces every node to store the exact same data as every other node. When one file is uploaded, all nodes must keep a copy of that file. This method wastes a lot of storage. If we imagine there are 10 nodes, and we upload a 5MB file, those 10 nodes must replicate the file, using a total of 50MB of space. This replication process also creates a huge surge in network traffic. But, this method has high security because if one node or one file is lost, we can simply replicate it from another node that definitely has a copy.
Second is RS Encoding (Reed Solomon) Method This method breaks a file into several pieces and creates new files called parity fragments from complicated math calculations. Parity are new files created for recovery if one piece is lost. These pieces and parities are then spread to different nodes. This method is much more efficient because nodes only store one piece of the file or its parity. But, it’s slightly weaker in security because not all nodes hold a piece of the file. For example, even if we have 100 nodes, but the file pieces are only on nodes 1–10, if those specific nodes crash, the file cannot be recovered
Both methods have their pros and cons. Full Replication has high security but poor storage efficiency, resulting in up to 25 times the storage overhead to achieve ideal security. Reed Solomon is much more storage-efficient but is less secure.
Red Stuff
Walrus adopts an innovative version of the Reed Solomon method called Red Stuff. The Red Stuff method solves all the problems that the traditional Reed Solomon method has. Specifically, Red Stuff creates a 2D (Two-Dimensional) Erasure Coding method. This aims to fix the main weakness of common 1D erasure coding (like RS encoding) in decentralized storage.
In the traditional Reed Solomon method, file pieces are arranged in a single row. If one part is lost, all nodes must reconstruct it, and the recovery cost is very big and centralized. But Red Stuff method arranges the file pieces and parity fragments into a matrix. If one part is lost, only the column connected to the missing part works to reconstruct the piece, and the costs are also shared among every node involved. This method allows for self healing, making it fast and efficient.
Okay, it’s really technical, but basically, Red Stuff successfully makes a relation between slivers more efficient by creating two complementary redundancy ways using primary slivers (rows) and secondary slivers (colomns). With this 2D encoding, the errored node can just request data from other node that are still in the same colomn and reconstruct the data by itself and automatically at a very low bandwidth or cost, unlike 1D encoding.
Async Network Solution
This is one of the most important innovations. Walrus specifically defines and solves the problem of Asynchronous Complete Data-Sharing. With an innovative asynchronous challenge protocol and proof-of-storage, Walrus guarantees data security and integrity even when the network is not in good condition (has lag). This feature can also handle sybil attacks and Latency Exploitation Attacks. This is a feature many other systems do not have.
Sui and Seal Integration
Walrus uses the unique architecture of the Sui blockchain. The concept of Sharded by id, means that every blob (data file) is treated as an independent object. Because Sui can process transactions related to different objects at the same time in parallel, this greatly boosts Walrus to handle requests.
If you look at the Walrus official website, it shows that Walrus works together with Seal. Seal is a separate protocol built to handle the confidentiality layer. Its main function is to encrypt data and manage access control.
With the integration of Seal, Walrus adds the following functions:
- Decentralized Encryption: Seal allows the data stored on Walrus to be encrypted. This way, even though the data is stored publicly on the network, its content remains private. This lets users store sensitive data like medical records or financial documents.
- Programmable Access Control: This is a key feature. Seal allows developers to create very specific rules that can be checked by the Sui blockchain, such as: (1) Only holders of a certain token can access this file. (2) Access to this data will expire after 48 hours. (3) Only a verified wallet can view this content.
The cooperation between Walrus and Seal creates a high level of security for decentralized data.
With Walrus introducing some innovations like Red Stuff and the async network solution, one has to wonder, “Is this the moment decentralized storage officially becomes the most reliable, secure, and efficient choice for everyone?“