Can Blockchain Bloat Ever Be Solved?

Blockchain is an important technology that has wide-ranging potential to change many of the existing solutions that currently power our world economy.

As a result, many entrepreneurs are trying to using it to overcome the problems that arise from the shortfalls of existing technology solutions.

However, in order that blockchain my become a far more viable solution, software engineers and entrepreneurs must overcome the dreaded blockchain bloat.

Can blockchain bloat ever be solved and what are the current solutions that developers have proposed to resolve this huge problem and allow blockchain to reach its full potential?

Lets take a closer look.

Contents

How did blockchain emerge?
“Distributed Ledger Technology” (DLT): Another aspect of blockchain
Blockchain bloat: The downside of built-in redundancy
Bloat impedes blockchain adoption
How to solve blockchain bloat?
Exploring the right solution for addressing blockchain bloat?

How did blockchain emerge?

An understanding of blockchain bloat requires some understanding of how the technology emerged. This brings us to Bitcoin:

  • Bitcoin, the famous digital currency emerged a decade ago.
  • It invented blockchain technology to solve the double spending problem.
  • It’s a decentralized “Peer-to-Peer” (P2P) network, where every computer can communicate with other computers without intervention. We call these computers “nodes”. Read more about this in “How do Bitcoin transactions work?”.
  • Bitcoin users send and receive Bitcoins over this network.
  • They don’t send any real-world currency backed by a central bank. Instead, Bitcoin is mathematical money. A Bitcoin transaction is just an exchange of information between two users.
  • One user initiates a transaction that involves sending a certain quantity of Bitcoin to another user.
  • The transaction sits in a pool called “mempool”. Read “Mempool explained” for more insights.
  • “Bitcoin miners”, i.e. a community of node operators on this network with special hardware and software pick up these transactions from mempool.
  • The transaction then goes through a process called “Crypto mining”, which involves a computing-intensive consensus algorithm called “Proof of Work” (POW). You can read more about POW in “Proof of work vs proof of stake comparison”.
  • “Miners” validate the transaction and finally it resides in a “block” on the Bitcoin blockchain. A block has several transactions.
  • Subsequent transactions go through the same process and new blocks are added to the network.
  • Each block has a reference to the earlier block, in the form of a cryptographic hash of that block. Read more about cryptographic hash in “Cryptographic hash functions explained: A beginner’s guide”.
  • When you visualize it, this looks like a chain of blocks. The name “Blockchain” derives from this.

“Distributed Ledger Technology” (DLT): Another aspect of blockchain

The distributed nature of blockchain is another key to understanding the blockchain bloat problem. Satoshi Nakamoto, the mysterious person or group behind Bitcoin had a specific motive behind creating this digital currency. Nakamoto wanted to conduct financial transactions without the intervention of banks or governments.

Download Our Project Specification Template

This required the following:

  • If a government or a malicious agent shut down one server, the network must still operate.
  • This means that the network needed to be a decentralized one.
  • Such a network also needs built-in redundancy, so that shutting down one server doesn’t cripple it.
  • Effectively, each node in the network should have all the transaction records.
  • This essentially means that each node has a complete ledger of all transactions in the blockchain.
  • As the number of blocks increases in the blockchain, all nodes sync up and store all blocks.

You can now see why we call blockchain a “Distributed Ledger Technology” (DLT). Read “Distributed ledgers definition” for more insights.

The obvious upside of this distributed nature of blockchain is that hackers have fewer incentives to hack. This built-in redundancy makes blockchain permanent.

Blockchain bloat: The downside of built-in redundancy

As a blockchain network grows, there are ever more transactions. Consequently, there are more blocks. The built-in redundancy of the decentralized network requires that every full node stores every one of these blocks.

The volume of data duplicated in every node increases over time, and we now have “Blockchain bloat”. This Cryptomaniaks definition explains it well. Bloat impacts the transaction speed, consequently, transaction fees tend to increase.

Running a new Node becomes harder. A user needs to download the entire blockchain first, to run a full node. This process is called “Initial Block Download” (IBD). Bitcoin, the highly popular cryptocurrency has such a large network now that IBD can take weeks! Read this Bitcointalk thread to get an idea of this challenge.

Bloat impedes blockchain adoption

Blockchain has significant potential, and the following indicators attest to that:

However, slow transaction processing, which is a direct result of blockchain bloat, can adversely impact the adoption of the technology. Experts cite this as a key challenge, as you can read in “Five things that must happen for blockchain to see widespread adoption, according to Deloitte”.

How to solve blockchain bloat?

Blockchain experts are working on quite a few solutions to solve blockchain bloat. I will now explain a few such solutions, as follows:

Read How We Helped a Marketing Company to Build a Back-Office Custom Ads Dashboard

1. Apollo’s “Sharding protocol”

Apollo, a privacy-focused cryptocurrency has come up with a solution using sharding, to solve blockchain bloat. Before reviewing this solution, let’s first understand sharding, as follows:

  • Sharding is originally a concept from the world of database management. Database experts use this technique to improve the efficiency of databases.
  • A shard is essentially a horizontal portion of a database. Database experts store each shard in separate server instances. They programmatically manage the querying process so that appropriate shards are used depending on the database query. This makes the database faster.
  • When we implement sharding in a blockchain, it’s a bit different. Software architects identify nodes and segregate parts of the blockchain database to create shards. Nodes in a particular shard only maintain that part of the database, and not the entire blockchain.
  • Since each node no longer loads the entire blockchain, the transaction speed improves.
  • The transaction validation process no longer involves all nodes. Naturally, POW consensus algorithm can’t work with sharding, and “Proof of Stake” (PoS) needs to be used. Each shard has its own set of transaction validators.

Read “Blockchain FAQ #3: What is sharding in the blockchain?” for more insights.

Apollo (APL) has implemented its “Sharding protocol”. The key characteristics of this protocol are as follows:

  • This protocol uses the sharding technique.
  • Apollo’s sharding protocol also uses “forging”. Forging is the transaction validation process in PoS algorithm. Here, unlike POW, only a few nodes take care of the transaction validation. This algorithm chooses transaction validators based on the number of crypto tokens they have “staked”, i.e., kept in deposit for transaction validation. Validators with longer duration and higher amount of stake have a higher probability of getting transactions to validate. Read more about forging in “What is proof of stake?”.
  • Sharding reduces the size of the database that every node needs to maintain.
  • Apollo claims to have successfully implemented sharding at a protocol level, which they claim to be unprecedented.
  • Their solution is based on a key design premise. Apollo finds that most operations in a blockchain use only a relatively smaller part of the database, and not the entire database. This is why they believe that sharding at a protocol level is the right solution.
  • Apollo claims that they have achieved two-second block speed, which is significant considering how blockchain bloat slows down networks.

You can read about Apollo’s solution in “What is blockchain bloat?”.

2. Ephemeral blockchains

Ephemeral blockchains could offer a way out of the bloating challenge. It would work as follows:

  • Ephemeral blockchains have a basic difference with blockchain networks we commonly hear about. Commonly known blockchain networks are permanent, whereas ephemeral blockchains are not!
  • Consider a blockchain you are building from scratch.
  • You create the genesis block, i.e., the first block in the blockchain.
  • Users of the blockchain network execute transactions, and new blocks are added to the blockchain.
  • Over time, it builds up a certain volume of data.
  • You define a cut-off point, based on how much transaction data the blockchain should record.
  • Once the chain reaches this cut-off point, it has fulfilled its objective.
  • Blockchain developers can then take a snapshot of the entire blockchain at that point.
  • The project team can save this state on a public blockchain like Ethereum. This End-of-state snapshot is now permanently saved, thanks to the decentralized security of a public blockchain.
  • Subsequently, you destroy this blockchain and create an entirely new ephemeral blockchain. The genesis block of the new ephemeral blockchain reflects the end-of-state snapshot of the earlier blockchain.
  • This process continues, with subsequent sets of snapshots capturing the incremental changes to the blockchain.

This approach limits the amount of data every node will need to load. Read more about this solution in “Ephemeral µBlockchains: the mayflies tackling data storage limitations on the blockchain”.

3. Using zk-SNARKs to solve blockchain bloat

Yet another method to solve the bloating challenge is to use “Zero-Knowledge Succinct Not-Interactive Argument of Knowledge” or “zk-SNARK”, which is a relatively new technology. This technology works as follows:

  • Zk-SNARKs are built on the “Zero knowledge proof” (ZKP) concept.
  • “Zero knowledge proof” enables one entity to prove the possession of a piece of information, without disclosing it. Read “Explain like I’m 5: Zero knowledge proof (Halloween edition)” to learn more about zero-knowledge proof.
  • Think of a hypothetical scenario where you log into a website by providing proof that you have the password, without actually entering the password. The user authentication program on the website doesn’t ask for the password. It allows you to login since you have given the proof that you have the password. At no point did you divulge the password!
  • Blockchain experts have built zk-SNARKs based on this foundation, and several blockchain projects use it.
  • Since the users don’t need to reveal the underlying information, some privacy-focused blockchain-crypto projects use zk-SNARKs to build privacy-oriented cryptocurrencies. Zcash is one example of a cryptocurrency that uses zk-SNARKs. Read more about zk-SNARKs in “What is zk-SNARKs? An introduction to this privacy protocol”.

Now that we have a brief understanding of zk-SNARKs, let’s see how this can help in addressing the bloating challenge. Coda cryptocurrency protocol has such a solution, which works as follows:

  • This project uses zk-SNARKs to compress the blockchain.
  • The solution creates a ZKP of the blockchain, which is a small set of data, compared to several Terabytes (TB) of data that well-known cryptocurrency blockchains have.
  • The entire state of the Coda blockchain is compressed and represented as a 1 KB zk-SNARK proof.
  • This is proof of the authenticity of the blockchain. Nodes have no need to store the entire ledger of the blockchain, rather they use this zk-SNARK proof to validate the ledger.
  • Along with the proof, the nodes need to store a small quantity of data. This data represents the path from the ledger to a users’ account.
  • This drastically reduces the amount of data each node needs to store. Users of Coda require merely 22 KB space to run a Coda node!
  • One can understand the impact of this in a different way. Running a full node of famous cryptocurrency blockchains requires a laptop computer at the minimum. You can’t operate a full node using a smartphone. However, Coda users can use their smartphones to run a Coda node!
  • The team at Coda uses the term “Succinct blockchain” to refer to their blockchain.
  • Irrespective of the number of transactions on the network, the size of the zk-SNARK proof remains constant. Synchronizing a node takes very less time.

Read “Reducing blockchain bloat: Coda protocol & instant-sync Bitcoin nodes” to gain more insights into Coda.

Exploring the right solution for addressing blockchain bloat?

Blockchain is a relatively new technology. While blockchain bloat has emerged as a challenge over the last few years, solutions to this problem are only recently emerging.

These solutions are also complex. Blockchain sharding, ephemeral blockchain, and zk-SNARK are all new innovations, and solutions based on them are evolving.

Given this complexity, finding the right solution to address this challenge can be tricky. You need the right expertise to evaluate available solutions, to choose the right approach, plan your blockchain project, and finally develop your blockchain app.

Planning is a major exercise here since it involves infrastructure, platform, development approach, as well as in many more aspects. Consider engaging a reputed development partner for your project. Before you do so, consult out guide “What to plan for when undertaking blockchain software development?”.

Download Our Project Specification Template

Aran Davies

Blockchain Expert | Developer | Writer | Photographer
Aran Davies

Latest posts by Aran Davies (see all)