March 1, 2022

What is Dynamic State Sharding in Shardeum?

Dynamic state sharding is a blockchain data scaling method that breaks it into smaller shards for efficiency. Learn more about dynamic state...

Written By

Shardeum Community

Home » Blog » Shardeum Innovations » What is Dynamic State Sharding in Shardeum?

According to a recent report by crypto.com, the total number of global cryptocurrency owners may grow by 3x in 2022. We are really looking at billion+ users sooner rather than later. When we look at existing networks and their capabilities though, we can see they are all falling agonizingly short of mass adoption. The current layer one (L1) blockchains are simply not designed to serve at such a large scale yet. That said, there have been no dearth of L1 blockchains attempting to scale up the network in the recent past. Even those who are successful at it today, end up compromising decentralization and/or the security tenets of open-source technology. Other measures like increasing the block sizes or having layer two (L2) solutions come with their own problems.

We are also realizing that vertical scaling with more hardware resources is not sufficient to take the underlying technologies toward mass adoption. It is safe to say that permissionless blockchain has rendered Moore’s law more or less unpredictable in the world of distributed computing networks. This lays the ground for our next question. How can sharding be the panacea to a challenging “scalability trilemma” problem?

What is Sharding in blockchain? — Investopedia.com

Table of Contents

What is Dynamic State Sharding?

Dynamic state sharding or dynamic sharding is one of the sharding types that is built to be flexible, and easily adaptable to changes in the overall blockchain ecosystem. Dynamic state sharding is the most advanced and complex way to shard the state of a network because it shards the State, Network, and Transactions and that too dynamically, as opposed to, in a pre-determined/fixed way. Sharding is further done both on the current and historical state of the network that includes past transaction history and trends. True to its name, dynamic state sharding allows for easy integration of upgrades without hindering the workflow of the ecosystem.

Dynamic sharding makes room for the addition or removal of blockchain shards as new updates demand, and may also allow changes in the method of data storage within blockchain shards. The primary benefit of dynamic state sharding, therefore, is the efficient scalability of a blockchain network without halting the overall flow of operations or performance.

What is Sharding?

Before we get into the details of dynamic state sharding by Shardeum, let’s have an answer to the basic question: what is sharding?

Sharding means partitioning to distribute a network’s computational and storage workload. That way each node operator can relieve themselves from shouldering the total transactional load on the network. Further, sharding breaks the job of validating and confirming transactions into small, manageable bits. Each node maintains information related to the transactions it handles. Transactions are then processed within the partition or shard it is on by reaching a consensus.

Sharding by Zilliqa & Polkadot

Shardeum is not the first blockchain to attempt Sharding. One of the prominent examples of sharded chains is Polkadot, which acts as a layer zero (L0) solution. Here, multiple heterogeneous chains (parachains) attach themselves to the Polkadot base layer and execute various transactions. Polkadot claims to provide security and communication across parachains. Another example is Zilliqa, which claims to be the world’s first L1 blockchain to use the sharding method that employs network-level sharding. This then raises the question: what makes Shardeum ground-breaking?

Sharded blockchains like Zilliqa group transactions into blocks and seek consensus at the block level. They can theoretically achieve higher efficiency with this, but it is not the most efficient way to employ sharding. Since consensus and commitment takes place at a block level, the finality of transactions takes more time and resources. On the other hand, Shardeum achieves consensus on each transaction. A transaction in this case does not have to wait till a consensus is reached across multiple shards for processing. This allows the network to introduce the below mentioned features which when combined together allows Shardeum to be truly scalable, and that too, in a linear and horizontal way.

💡Want to know more about sharding? Check our detailed guide on Types of Sharding in Blockchain.

Dynamic State Sharding by Shardeum

Note, the following contents are an excerpt from a recently published blog that comprehensively explains how Shardeum will eventually solve blockchain trilemma. To begin with, the network won’t have a static group of nodes as fixed shards. Nodes on Shardeum are free to move around and accommodate more data as dynamic shards. Dynamic state sharding will work hand in hand with Shardeum’s auto-scaling feature allowing the network to automatically adjust the number and size of shards based on the current workload. This allows the system to optimize performance and maintain high levels of scalability as it grows and evolves.

Static state sharding only enables a blockchain network to have a static/pre-defined group of shards and transactions can only be processed sequentially after a minimum number of nodes join the network to create a new shard. There are two issues at play here. For one, this does not allow for growing dynamically in proportion to the demand in the network. Two, sequential processing slows down the network as a result of high latency arising from the time taken for new nodes added to sync to the latest state of shards they are involved in.

Dynamic state sharding allows validator nodes on Shardeum to be assigned dynamic address ranges across multiple shards. Unlike static state sharding where all the nodes in a shard cover the same address range of an account, dynamic state sharding requires each node to hold a different address range, but there is significant overlap between the addresses covered by nodes in respective shards. Since consensus is done at the transaction level, a transaction that affects multiple shards will be processed simultaneously by these shards rather than sequentially as with block-level consensus. This not only reduces the time to process the transaction even if it affects multiple shards but also ensures atomic processing. Shardeum will further ensure cross shard composability.

Dynamic State Shardings vs Adaptive State Sharding

Adaptive state sharding is used by networks like MultiversX (formerly Elrond). It is a more advanced type of sharding compared to static state sharding where the shards are adapted to the current state of the network. This is done by changing the number of shards or the size of the shards based on factors such as the number of nodes in the network and the amount of data being processed. For example, if there is a sudden increase in the number of transactions, the network deploying adaptive state sharding can automatically create more shards to handle the increased load. This helps to ensure that transactions are processed quickly and efficiently, even during periods of high traffic with parallel transaction execution. Essentially, this type of sharding aims to combine the benefits of state, network, and transactions sharding. That being said, adaptive state sharding renders load balancing on a reactive basis (and not on a proactive basis like dynamic state sharding) because the distribution of load happens based on the current state of the network and not on the historical state of the network.

Dynamic state sharding takes adaptive state sharding to another level by helping a network to distribute the load more efficiently and on a pro-active basis. It shards the state, network and transactions evenly based on not just the current state of the network but also based on the historical states of the blockchain (which includes latest state and transaction history). For example, a dynamic state sharding algorithm might track the average number of transactions processed by each shard over time. This information could then be used to predict which shards are likely to become overloaded in the future. The algorithm could then take steps to pre-emptively balance the load across the shards. Another example is, the algorithm might notice that certain types of transactions are typically processed by specific shards. This information could then be used to proactively route new transactions to the appropriate shards.

Atomic Processing & Cross Shard Composability

Cross-shard communication allows for transactions to access and utilize data and state from different shards, enabling complex transactions and smart contracts to be executed in a sharded environment. Atomic composability is also important because it ensures that transactions are executed atomically, meaning that either all parts of the transaction are executed successfully or none of them are. Without atomic composability, transactions could potentially fail or leave the blockchain in an inconsistent state, leading to security risks and reduced reliability. Shardeum will ensure complex transactions and smart contracts are executed effectively in a sharded environment while maintaining the integrity and consistency of the blockchain.

Shardeum Scales Linearly

Through dynamic state sharding, every node added to the network will increase the transaction throughput instantly. So basically, by simply adding more nodes from the network’s ‘standby’ validator pool during peak demand, the TPS will increase proportionally making Shardeum the first Web3 network to scale linearly. And this is the main X factor that impacts every other outcome on a blockchain network favorably including throughput, decentralization, security, and constant transaction fees irrespective of the demand in the network.

That’s how Shardeum’s underlying protocol, Shardus, was able to demonstrate 500 TPS with 100 nodes over the last 3 years. Shardeum is aiming to enable up to 1 TPS or more per node which will be a huge breakthrough for the Web3 ecosystem. Even with 2k active nodes on existing blockchain networks, we can only see them process an average of 350 TPS at best when Web2 peers like PayPal and Visa process an average of 5000 TPS every day. Shardeum’s idea is to mobilize millions of nodes eventually which can process over 1 million TPS to realistically host a ton of dApps providing products and services to billions of users thereby eliminating middlemen taking advantage of our data/privacy.

Shardeum Maintains Low Gas Fees Forever

And why does linear scaling matter? In a nutshell, one key advantage of Shardeum is that the consensus and processing are done at the transaction level and not at the block level. And, through dynamic state sharding, the network will shard its state by evenly and dynamically distributing compute workload, storage, and bandwidth among all the nodes. This not only allows for parallel processing of transactions but also very low overhead for validator nodes as they will store only the state data of transactions they are involved in.

This is how Shardeum will get to maintain low transaction fees for developers and end users perpetually. Just to be clear, dynamic state sharding is the most advanced version of static, transaction, network, and adaptive sharding employed by more recent sharded chains which runs into both inter-related and standalone problems such as high latency, vertical scaling (as opposed to linear scaling), Sybil attack, weak finality and lack of cross shard composability. That said, dynamic state sharding is also the most complex way to shard the state of a network.

Shardeum: A Unique Blockchain for the Future

Shardeum pioneers a new approach with sharding. With dynamic state sharding on top of its EVM compatibility, Shardeum will be truly capable of handling a multitude of dapps and layer 2 solutions across industries and consumers with optimum performance. The network will indeed maneuver in unchartered territories in the blockchain world as it aims to better even Web2 networks in terms of throughput capacity while retaining high security and decentralization that Web3 networks are known for. These are really exciting times ahead for the Web3 world. So let’s stay tuned 🙂

Conclusion

We hope this post has helped you get an understanding of ‘what is sharding?’, along with the concepts of dynamic state sharding and how Shardeum is using the concept to solve scalability issues that contemporary blockchains are facing. As dynamic state sharding combines with its auto-scaling feature, Shardeum is aiming to finally solve the blockchain trilemma once and for all and take Web3 mainstream!

Frequently Asked Questions (FAQs)

1. What is Adaptive State Sharding?

Adaptive state sharding is similar to (and comeptes with) dynamic state sharding. They both have same objective. Adaptive & dynamic state sharding are both better than state/static state sharding. But dynamic state sharding is considered superior because it does a better work (more pro-active) in load balancing by monitoring the historical state of the blockchain (that includes current state, most recent state, and the complete transaction history). Adaptive state sharding is more reactive in the way that it distributes load based ‘only’ on the current state of the network.

2. How many Types of Sharding are there?

There are six main types of sharding in blockchain technology: Network Sharding, Transaction Sharding, State Sharding, Static State Sharding, Adaptive State Sharding, and Dynamic State Sharding. Each type aims to enhance the scalability and efficiency of blockchain networks by dividing and processing data in different ways, addressing various challenges and trade-offs in distributed ledger systems.

3. How Dynamic is Better than Adaptive?

Dynamic state sharding enhances adaptive state sharding by distributing network load proactively and efficiently. It balances state, network, and transactions using both current and historical blockchain data. By analyzing transaction trends over time, it can anticipate and manage shard overloads and route transactions to optimal shards.