May 4, 2023

Why Linear Scalability is the Holy Grail of Web3 Today and How Shardeum Achieves it?

Linear scalability ensures consistent performance as system resources increase, maintaining efficiency and speed seamlessly, as in the case with...

Written By

Subbu

Home » Blog » Shardeum Innovations » Why Linear Scalability is the Holy Grail of Web3 Today and How Shardeum Achieves it?

In today’s world, linear scalability is a critical concept that organizations must understand to meet the needs of a universe of data and users that are growing and changing. Linear scalability is the ability of a system or application to add resources in a way that makes its capacity grow linearly. By knowing how linear scalability works and what affects it, organizations can use systems that can handle more work reliably and predictably.

What is Linear Scalability?

Linear scalability is a property of a system or application that lets it handle more work by adding more machines/server/nodes in a way proportional to the increase in work. A linearly scalable system typically can handle twice as much work when its resources are doubled, and thrice as much work when its resources are tripled, and so on.

This differs from vertical scaling where a network can scale by adding more power (CPU, RAM) to existing machines. In this case, the resources added don’t always mean the performance increases in the same proportion. For example, a system that isn’t linearly scalable might only see a slight improvement in performance after a certain point, or it might show diminishing returns, where adding more resources results in improvements initially but slows down with time and increased load.

Linear scalability in blockchain is essential for systems that handle large and growing workloads. It lets networks add resource to meet increasing demand while keeping performance steady and efficient.

How Does Linear Scalability Work?

Linear scalability lets a system handle more work according to how many resources are added. This means that if the system is built to be linearly scalable, its performance will be optimal and constant no matter how many resources are used. As more and better resources are added to the system, it can handle more users and process more data without slowing down.

To achieve linear scalability in blockchain, the system needs to be set up so that resources can be added or taken away quickly without causing downtime or other problems. This can be done by using distributed systems, load balancing, sharding, and other methods that make it possible to add resources on the fly.

Examples of Linear Scalability

1. Distributed Computing

When a set of tasks is distributed to multiple machines, and when each of them collaborates to accomplish the common goal, this method of computing is known as “distributed computing.” Scalability is achieved in a distributed computing system by adding resources on demand. Essentially, even if you double the number of computers or servers working on a task, the amount of time required to complete the task will remain constant.

2. Cloud Computing

It is a style of computing that uses a third-party service provider’s pool of shared computing resources, such as data storage, processing power, and memory. It allows businesses to quickly and easily adjust to fluctuating workloads by adding or removing resources as needed.

3. Virtualization

Whether it’s an OS, a server, or an entire network, virtualization makes a digital copy of the original. The virtualized environment can scale up to handle greater workloads by adding additional resources on demand.

Factors Affecting Linear Scalability

Despite its adaptability and scalability, several factors can limit linear scalability. Key ones include:

1. Technical Limitations

Even linearly scalable systems have hardware limits. Resources do not increase productivity linearly to infinity at all times.

2. Network Latency

Network latency can hinder distributed system’s scalability. Due to increased communication and coordination among multiple nodes, the risk of bottlenecks and slowdowns often increases.

3. Application Design

Adding hardware may not improve performance if the application was not written to use it.

By considering these factors, businesses can design and implement linearly-scalable systems that make it easier to handle increased workloads.

Achieving True Linear Scalability

To get true linear scalability, you must plan, design, and implement the plan carefully. For systems to be able to grow linearly, they must be prepared to be distributed, able to handle problems, and highly available. Also, the workload must be spread evenly across the system using load balancing, sharding, caching, and other methods. Further, adding more resources must be done to let the system use the extra resources without creating more bottlenecks or points of failure.

Getting true linear scalability in the blockchain is a difficult task that requires knowledge of distributed systems, performance optimization, and software engineering. But organizations can achieve linear scalability and efficiently handle any amount of work if they carefully consider the aforementioned factors and use the right strategies.

Why is Linear Scalability a Holy Grail for Blockchains?

Blockchain Industry Works 24x7x365

Unlike banks or the stock market, the blockchain networks and the industry works 24x7x365 to facilitate users:

i) pay transaction fees in the form of crypto or network coin while creating smart contracts and decentralized applications on a network

ii) to contribute to a network in terms of security, hardware and software

iii) to use crypto as a medium of exchange in return for goods and services without an intermediary

iv) make cross border payments with a fraction of time and fees compared to Web2 networks

v) to invest and trade in a network coin/crypto for future gains

These are just the few among increasing use cases of blockchain technology today. You might now wonder why the decentralized Web3 ecosystem still hasn’t replaced Web2 yet? Features like low fees, zero partiality, high speed, immutability, decentralized applications (or dapps), and high security should be more than enough to convince us to adopt Web3, right? This tweet talks about the key bottleneck that prevents Web3 from going mainstream yet.

Linear scalability a holy grail for blockchains

Self-imposed Scaling Limits & Consensus Algorithms

Blockchain was invented in the aftermath of the 2008 financial crisis. Hence early layer 1 networks like Bitcoin and Ethereum used the distributed digital ledger primarily to make commercial transactions secure and decentralized. Now if you see a Web2 network like Citigroup or Meta don’t face issues with scalability. But, in order to execute transactions in a trustless way to increase decentralization and security, blockchain networks distribute transactions across multiple unrelated nodes around the world. After nodes validate transactions assigned to them, networks proceed to perform another resource intensive task i.e. consensus, on multiple transactions grouped together in a block based on their validity.

Further, most of the current distributed ledger protocols have self-imposed scaling limits as they stipulate maximum size for their blocks and the rate at which blocks are produced thereby setting a ceiling on the rate at which their network can process transactions. If they were to increase the block size limit, they will need to increase the already elevated processing powers which will lead to more centralized entities running nodes on the network. Remember, consensus and self imposed block rates are crucial to the security and trustless nature of public decentralized networks. Banks or corporate companies, in contrast, process transactions through trusted resources like authenticated personnel and augmented hardware devices without having to worry about scalability trilemma.

Although public blockchains have the potential to get the best of both the Web2 and Web3 world, it is clear that they weren’t built to handle an industry that gradually started working 24x7x365 even during market downturns. When the volume of transactions goes up in a public blockchain, shortage of adequate resources leads to stop gap measures like front-running and vertical scaling which shoots up gas/transaction fees. Further, the network congestion slows down the speed at which transactions are processed significantly, thus taking away two of the most important factors that are critical for mass adoption – speed and transaction costs. Layer 2 blockchain networks and rollups are the other stop gap measures adopted by layer 1 networks to mitigate their scalability and high transaction fee issues to a certain extent. But that comes at the cost of lower levels of decentralization and security. So solving the scalability trilemma at the layer 1/protocol level is the only solution here.

Scalability Trilemma

There is no better way to solve the scalability trilemma other than by scaling the network linearly which will allow it to maintain high decentralization, security and scalability at all times irrespective of the demand. But so far no Web3 network has demonstrated its capability to scale linearly. Ethereum co-founder Vitalik Buterin, suggested sharding as a potential solution for blockchain’s scalability issues. Through sharding, the network will shard its state by evenly (and dynamically in the case of Shardeum) distributing compute workload, storage, and bandwidth among all the nodes. While sharding is ultimately the best way to tackle the scalability issue, applying it to blockchain-based networks is not nearly as easy as applying it to centralized databases.

There are more recent blockchain networks that have employed sharding with limited success. One major challenge developers are increasingly running into is the inability of the sharded network they are building their dapps on, to retain atomic and cross-shard composability. Secondly, block level consensus and sequential processing of transactions slows down the network as a result of high latency arising from the time taken for new nodes to sync up to the latest state of shards they will be involved in. They, further, do not allow for dynamically addressing varying loads on the network at a given point in time.

Shardeum Solves Scalability Trilemma by Scaling Linearly

Shardeum introduces the most advanced and complex type of sharding from the ground up called “dynamic state sharding” where each node will be assigned dynamic address spaces across multiple shards with significant overlaps. The network won’t have a static group of nodes as fixed shards. Nodes on Shardeum are free to move around and accommodate more data as dynamic shards. Dynamic state sharding will work hand in hand with Shardeum’s auto-scaling feature allowing the network to automatically adjust the number and size of shards based on the current workload. This makes the network very efficient in conserving resources and using them when needed.

The validator nodes (or validators) on the network, who are responsible to process transactions would need to maintain only the current state of the shards they handle. The historical data will be offloaded to archive nodes allowing average users with low hardware to operate a validator on the network. Since consensus on Shardeum will be done at the transaction level, a transaction that affects multiple shards will be processed simultaneously by these shards rather than sequentially as with block level consensus while retaining both atomic and cross-shard composability. The network will ensure complex transactions and smart contracts are executed effectively in a sharded environment while maintaining the integrity and consistency of the blockchain for a smooth developer experience.

With this in place, every node added to the network will increase the transaction throughput instantly. So basically, by simply adding more nodes from the network’s ‘standby’ validator pool during peak demand, the TPS will increase proportionally making Shardeum the first Web3 network to scale linearly. And this is the main X factor that impacts every other outcome on a blockchain network favorably including throughput, decentralization, security, and constant transaction fees irrespective of the demand in the network.

Conclusion

Linear or horizontal scalability is critical for modern technology systems, allowing them to handle increased workloads proportionally and consistently. Achieving true linear scalability requires careful planning, design, and implementation, including distributed computing, load balancing, sharding, auto-scaling, and fault tolerance. With linear scalability, organizations can confidently take on any challenge and grow their systems to meet the needs of a growing and evolving universe of data and users. All said, we now know how difficult it is for public distributed blockchain platforms to scale linearly.

Shardeum’s underlying technology is a game changer but innovation is just part of the game. Shardeum has started open sourcing its codebase in a phased manner which will be completed by the time it launches its mainnet in Q2 of 2023. More than competition, the project is aiming to inspire others to innovate and onboard the world to Web3.

Frequently Asked Questions (FAQs)

1. What is Linear Scaling, in simple words?

Linear scaling refers to the ability of a system or application to increase its capacity proportionally to the number of resources added. If the workload doubles, adding twice as many resources will enable the system to handle the increased workload simultaneously.

2. How do you Scale Linearly?

To scale linearly, systems must be designed to take advantage of additional resources as they are added. This involves creating distributed systems, auto-scaling, load balancing, sharding, fault tolerance, and other techniques that enable systems to handle increased workloads consistently and proportionally.

3. What is Horizontal vs Vertical Scaling?

Linear scaling is a property of a system or application that lets it handle more work by adding more machines/server/nodes in a way proportional to the increase in work. Vertical scaling is when a network scales by adding more power (CPU, RAM) to existing machines. In this case, resources added doesn’t always mean the performance increases in the same proportion. For example, a system that isn’t linearly scalable might only see a slight improvement in performance after a certain point, or it might show diminishing returns, where adding more resources results in improvements initially but slows down with time and increased load.

4. How Does Shardeum Achieve Linear Scalability?

Shardeum introduces the most advanced and complex type of sharding from the ground up called “dynamic state sharding” where each node will be assigned dynamic address spaces across multiple shards. The network won’t have a static group of nodes as fixed shards. Nodes on Shardeum are free to move around and accommodate more data as dynamic shards. Dynamic state sharding will work hand in hand with Shardeum’s auto-scaling feature allowing the network to automatically adjust the number and size of shards based on the current workload. Since archive nodes on the network stores the historical transactions, validator nodes are light weight and proceeds to consensus on transactions individually and a transaction that affects multiple shards will be processed simultaneously by these shards rather than sequentially as with block level consensus.

With this in place, every node added to the network will increase the transaction throughput instantly. So basically, by simply adding more nodes from the network’s ‘standby’ validator pool during peak demand, the TPS will increase proportionally. And this is the main X factor that impacts every other outcome on a blockchain network favorably including throughput, decentralization, security, and constant transaction fees irrespective of the demand in the network.