As Data Becomes More Important, We Desperately Need Better Ways to Store It

It’s impossible to calculate exactly how much data exists in the world today, but it’s safe to say that it’s a colossal amount. Google alone processes 40,000 search queries every second, adding up to billions over the course of a single day.

By 2020, it’s thought that we will be creating about 44 trillion gigabytes of data every year, compared to 4.4 trillion in 2013. And as the Internet of Things and AI explode, it’s impossible to say how quickly this number will go up.

This is all happening extremely quickly, and it’s no surprise that we’re struggling to keep up with it. Our data infrastructure has a lot of adapting to do, and one of the areas which will feel the most strain is our storage capabilities.

Imagine any physical asset increasing in quantity at the rate data is. We’d very quickly run out of places to put it, and the same is happening in the digital world. We need to store our data in a way that is secure and accessible and allows for permanent storage.

That’s proving to be a challenge, and time is running out. We urgently need to tackle the problem of data storage — but how? Let’s take a look at our current responses to the issue, and how well they’re holding up.

How we store data right now

Cloud storage is currently the go-to for much of data storage. There are lots of good reasons for its popularity, too — it’s quick, simple, and scales pretty well.

Storing data off-site in the cloud reduces the hardware load on businesses and can be a big money saver. But the cloud isn’t perfect. Far from it. One of the main concerns surrounding cloud storage is the issue of security. Storing data online like this leaves it vulnerable to hackers and data breaches, and there have been quite a few.

For example, there was the time Dropbox was hacked and 5 gigabytes of data was stolen. In 2016, the National Electoral Institute of Mexico suffered a similar breach, where attackers gained access to 93 million voter registration records.

But this was in the past, ancient history by tech standards — surely things are better today? Well, not as much as you might hope. In the past year 51% of companies publicly exposed at least one cloud storage device, and 27% experienced potential compromises thanks to cloud security failures.

Ouch. These are hardly reassuring figures and are downright frightening in a world where data security becomes more important every day.

One of the new technologies which looks to challenge the cloud in terms of security is Blockchain Blockchain Blockchain comprises a digital network of blocks with a comprehensive ledger of transactions made in a cryptocurrency such as Bitcoin or other altcoins.One of the signature features of blockchain is that it is maintained across more than one computer. The ledger can be public or private (permissioned). In this sense, blockchain is immune to the manipulation of data, making it not only open but verifiable. Because a blockchain is stored across a network of computers, it is very difficult to tamp Blockchain comprises a digital network of blocks with a comprehensive ledger of transactions made in a cryptocurrency such as Bitcoin or other altcoins.One of the signature features of blockchain is that it is maintained across more than one computer. The ledger can be public or private (permissioned). In this sense, blockchain is immune to the manipulation of data, making it not only open but verifiable. Because a blockchain is stored across a network of computers, it is very difficult to tamp Read this Term. Since it began life as the building blocks of Bitcoin Bitcoin While some may still be wondering what is Bitcoin, who created Bitcoin, or how does Bitcoin work, one thing is certain: Bitcoin has changed the world.No one can remain indifferent to this revolutionary, decentralized, digital asset nor to its blockchain technology.In fact, we’ve gone a long way ever since a Florida resident Laszlo Hanyecz made BTC’s first official commercial transaction with a real company by trading 10,000 Bitcoins for 2 pizzas at his local Papa John’s.One could now argue that While some may still be wondering what is Bitcoin, who created Bitcoin, or how does Bitcoin work, one thing is certain: Bitcoin has changed the world.No one can remain indifferent to this revolutionary, decentralized, digital asset nor to its blockchain technology.In fact, we’ve gone a long way ever since a Florida resident Laszlo Hanyecz made BTC’s first official commercial transaction with a real company by trading 10,000 Bitcoins for 2 pizzas at his local Papa John’s.One could now argue that Read this Term almost a decade ago, blockchain has gradually seized the tech community’s imagination and is now widely considered a promising solution to our data storage woes.

Blockchain’s big advantage is its security. It’s immutable, making it impossible to hack and tamper with the data stored in a blockchain. It’s also transparent and famously resistant to corruption by third parties, as it functions without a central point.

But blockchain isn’t perfect, and it has a few kinks to work through before it can rise to the challenge of global data storage.

The main issues are scalability and cost. Right now, as blockchains become larger and store more data, they tend to become unwieldy and slow. It’s hard to access the relevant data and wait times — not to mention costs — for transactions can be crippling.

The clearest example of this was seen at the end of 2017, when Bitcoin underwent a massive surge in price to almost $20,000. This rapid growth was a lot for the blockchain to handle, and transaction fees skyrocketed from a few dollars to more than $50 at points.

If blockchain is to come to the rescue and meet our data storage needs, it must be able to handle large amounts of data without major issues. The good news is, that could soon be possible.

How to upgrade blockchain storage

A blockchain startups called Arweave want to address the data storage crisis by improving the way blockchains function.

Right now, it’s necessary to build third-party protocols on top of existing blockchains to meet storage needs. Projects like Filecoin act as a decentralized hard disk, whereas Arweave offers permanent data storage. With Arweave’s ‘blockweave’ technology, things are done a little differently, and much more efficiently.

"The blockweave ties the necessities of two groups together: those that want to store data for long periods and those that want to earn scarce tokens. By building a system that satisfies the needs of one group with the needs of the other, the Arweave presents an efficient, cheap, and reliable data storage system. This stands in stark contrast to existing blockchain systems. Instead, data can be stored easily on the blockchain itself, in a way that’s scalable and accessible" says Sam Williams, CEO of Arweave. "Storage will be permanent, and it only requires a one-time fee from users. Cloud services and similar software tend to demand a monthly fee, which can really add up over a lifetime of storage" he adds.

With Arweave’s approach, the amount of hashing required decreases as the amount of data stored in the system increases. This way, the blockchain responds and adapts as the data load gets bigger.

Working through blockchain’s faults is something we’ll need to address if we want to access a secure and reliable storage method. In a world where the amount of data is increasing at breakneck speed, this kind of innovation can’t come quickly enough.

Disclaimer: This is a contributed article and should not be taken as investment advice