Blockchain Consensus And Fault Tolerance In A Nutshell

Photo by rawpixel on Unsplash

The blockchain is a distributed and decentralised system, which means that it needs to have a way of tracking the official current state of the system. Since the blockchain can include financial transactions and business agreements, it is important that all parties involved are in sync regarding the terms of the agreement.

So, one of the most important components to blockchain is this idea of group consensus.

Even though it might appear odd, blockchain is a very inefficient system.
We’re asking multiple nodes, sometimes tens of thousands of computer nodes, to all repeat the same work. They’re all keeping a copy of the same data.

And the reason we agree to this tremendous inefficiency is that if we can get all or most of those nodes to agree on what the truth is we can have a lot of trust that that’s actually the truth, that that record hasn’t been tampered with or altered or changed in any way.

So, the consensus is kind of at the base of the blockchain.

There are several different methods we use right now to have all these nodes reach consensus.

Essentially, when we talk about consensus, you can think of every block in a blockchain as being like a sheet of paper. It has a fixed amount of space. We write a transaction on every line, and when that sheet of paper is full, it’s important that we all, as a group, come together and compare our different sheets and select the sheet or the version of the paper that the majority agree with.

The consensus is a way to ensure the nodes on the network verify the transactions and agree with their order and existence on the ledger. In the case of applications like a cryptocurrency, this process is critical to prevent double spending or other invalid data being written to the underlying ledger, which is a database of all the transactions.

Consensus mechanisms are essential in a decentralised world where there are no middlemen and where trust has truly become decentralised with the trustless movement of value.

And so, there are several different methods that we use to come to a consensus on a block. Different solutions that fit different situations.
The main difference between consensus mechanisms is the way in which they delegate and reward the verification of transactions. It’s important to mention that most blockchain ecosystems have a hybrid of different consensus mechanisms. There is no need to choose one over the other.

The oldest and most widespread and known method is what’s called Proof of Work.

Proof of Work or simply PoW has served us well for the past almost ten years, it started in Bitcoin, and it’s used in every major public and most private blockchain.

But we’re also starting to see some of the limitations of Proof of Work.
One of the big limitations behind Proof of Work right now is how big and how fast it can scale.

Currently, on Proof of Work blockchains, we’re able to process somewhere between 50 and 20 transactions worldwide per second, which may sound like a lot until you realise that modern payment processing networks like Mastercard and Visa can scale up to over 70,000 transactions a second.

As you can see, in order to compete with conventional technology, blockchain really needs to speed up that transaction rate.

There are many proposed alternative consensus methods for how we might be able to reach that kind of scale. There are things in production right now like Tangle which use a block-less solution, and there are also new and emerging consensus methods like Proof of Stake or Proof of Activity that we’re currently examining to take the work out of Proof of Work.

The takeaway point to understand is that, it’s this consensus, it is this idea of asking all of these nodes, potentially tens of thousands of nodes, to all repeat the same work and then periodically come together and agree on whatever the majority select the right version of the truth to be, that gives blockchain that high level of trust and makes it such a secure record store.

The blockchain is designed to be a shared, synchronised historical ledger, meaning that there needs to be a final decision at some point on what should and shouldn’t be included in the official record. Since blockchain is decentralised, there is no “higher authority” that can rubber-stamp and finalise the contents of a blockchain block.

The method that Satoshi Nakamoto, the creator of Bitcoin blockchain, invented to achieve consensus is based on scarcity. In one way or another, blockchain consensus algorithms boil down to some kind of vote where the number of votes that a user has is tied to the amount of a limited resource that is under the user’s control. Based on the economic Laws of Supply and Demand, collecting enough of an asset to have a controlling share will drive up the price of the asset enough to make achieving that level of control unfeasibly expensive.

Satoshi Nakamoto invented a consensus algorithm called Proof of Workfor the use of Bitcoin. Since then, several other consensus algorithms have been invented to fit different use cases. These include Proof of StakeDelegated Proof of StakePractical Byzantine Fault ToleranceDirected Acyclic GraphsProof of Elapsed Time, etc.
The most commonly used consensus algorithms are Proof of Work and Proof of Stake.


In Proof of Work or PoW, users in the blockchain network who want to create the next block (and win the reward) are called miners. To win the right to mine a block, miners race to find an acceptable solution to a “hard” cryptographic problem. As we’ve discussed previously, “hard” mathematical problems can only be solved by random guessing. When a miner finds an acceptable solution, they create a block and broadcast it to the network, finalising that block and all the transactions in it.

Proof of Work exploits the scarcity of computational resources by choosing a problem that can only be solved by guessing. There is no limit on the number of guesses that a miner can make at once.

So, Proof of Work incentives miners to run as many mining machines as possible to maximise the probability that they are the first to find a solution to the problem. Since mining computers take money to purchase and money to run, the amount of control that a user can exert over the blockchain network is limited by the amount of money they have available to invest in mining equipment.

The security of the Proof of Work consensus is based on the assumption that no-one controls more than half of the computational resources of a blockchain’s mining network.

If this was the case, the miner has a high probability of finding an acceptable solution to the mining puzzle before anyone else for every block in the blockchain. This gives the miner complete control of the blockchain network and breaks the decentralisation of the blockchain.

Basically, miners are solving hard math problems to verify transactions and secure the overall network.

Miners are GPUs or ASICs chips running computational cycles to solve a math problem with the goal of reaching a set number previously provided to them. This set number is called a “target”, which is an SHA-256 hash with a long list of leading zeros and the “difficulty” (another term in the Bitcoin world) of this “target” adjusts every 2016 blocks (roughly 2 weeks), to ensure it takes roughly ten minutes for the miners to mine a new block.

There are three major ingredients needed to find this “target”:

  • A nonce (which is a number only used once)
  • The transactional data
  • The previous blocks hash.

This is all then hashed (combined) over and over again with the nonce changing each time until the hash created from these three ingredients is lower than the “target” provided.

Once the Miner has reached this “target”, they’re gifted with a transaction fee and mining reward (at the time of this article was is 12.5 bitcoins). The reward gets cut in half every 210,000 blocks (roughly 4 years).

The next step is for the miner to broadcast to all the other miners that they have achieved the set “target” and have confirmed the block. Once that has been completed, they’ll move onto the next block.


Users in a Proof of Stake or PoS blockchain network can “stake” or promise not to use the tokens they own. This gives them the opportunity to be selected as the next user to create or “forge” a new block and earn the reward. A block forger is pseudo-randomly selected from all of the users who have staked some of their assets, and the selection process is biased based on the size of the stake.

For example, imagine that a wheel is divided into sections where the size of a section is proportional to the size of a user’s stake. The next block forger would be chosen by spinning the wheel and seeing whose section comes out on top. In Proof of Stake, each user has a copy of the wheel and they are all synchronised so that each person can independently determine the selection and get the same result. This is why Proof of Stake uses a pseudo-random instead of a random selection process.

In Proof of Stake, an attacker needs to control enough of the staked currency to guarantee they will be selected to create every block. Since cryptocurrency is a limited asset, buying up enough of it to do this is expensive, making attacks on Proof of Stake systems economically infeasible.

With Proof of Stake we have “Validators” — “Forging”, instead of “Miners” — “Mining”.

There are no computational cycles running through massive amounts of math problems trying to solve a problem like PoW. With PoS, we have validators sending a special type of transaction across the network, which gets locked into a deposit (also known as validator pool) and that’s called “staking”.

Once this validator has staked the amount of token wanted, then an algorithm pseudo-randomly selects a validator during each time slot (for example, every period of 10 seconds might be a time slot), and assigns that validator the right to create a single block. This block must point to some previous block at the end of the previously longest chain, and over time, most blocks converge into a single constantly growing chain.

Once the validator has been selected, the next step for the validator in order to create a block is to validate a grouping of transactions.
Once that’s completed, they receive their staked funds back, plus the transaction fees (sometimes rewards when coin supply is being inflated from time-to-time) for that block.

If the validator decides to act in a bad way, like a bad actor, and validate fraudulent transactions, they lose their stake that’s being held at the moment and are booted from the validator pool going forward (losing rights to forge). This is a built-in incentive mechanism to ensure they are forging valid transactions and not fraudulent ones.


Specific Consensus Implementations

Ethereum:

Ethereum currently uses Proof of Work for consensus. And Casper is the planned migration of Ethereum from Proof of Work to Proof of Stake.

Ethereum was designed from the beginning to use Proof of Work for consensus, until a forced hard fork to the Proof of Stake implementation (codenamed Casper). This forced hard fork is baked into the Ethereum protocol and will be accomplished by slowly increasing the difficulty of the Proof of Work problem until the time taken to solve it increases to the point where Proof of Work becomes unusable. Proof of Stake does not require the same energy consumption as Proof of Work and is a more sustainable and scalable consensus mechanism.

Bitcoin:

Bitcoin uses Proof Of Work invented by Satoshi Nakamoto.

Hyperledger Fabric:

Hyperledger Fabric breaks out consensus into components, allowing users to pick a consensus algorithm for their particular use.

Hyperledger Fabric deliberately avoided hard-coding a consensus mechanism into the protocol by defining an “orderer component” that performs all of the consensus-related operations. This allows users of Hyperledger Fabric to select a consensus algorithm that fits their use case without being forced to make large-scale code edits.

Corda:

Each Corda network has a notary service made up of independent parties that approve blocks using any applicable consensus algorithms.

Corda does not follow the standard blockchain model of transactions being bundled into blocks and then being finalised by the network as a whole. Instead, a Corda network contains one or more notaries consisting of several independent parties. Transactions in Corda are finalised by a notary with a multiparty digital signature using an algorithm like Raft.


Fault Tolerance in the Blockchain

The blockchain is a distributed, decentralised system that maintains a shared state. While consensus algorithms are designed to make it possible for the network to agree on the state, there is the possibility that agreement does not occur. Fault tolerance is an important aspect of blockchain technology.

The blockchain is inefficient and redundant, and that is by design. That’s what gives us immutability. And another thing it gives us is an extreme level of fault tolerance.

At its heart, blockchain runs on a peer-to-peer network architecture in which every node is considered equal to every other node.
And unlike traditional client-server models, every node acts as both a client and a server.

And so, we continue this redundancy down at the network level, where we’re asking all these nodes to perform the same work as all these other nodes.
Like any peer-to-peer system, we have an extremely high degree of fault tolerance. In fact, if we have two or more nodes online in a blockchain system, we still have a working blockchain.

And when you think about that amazing fact given the scale of major public blockchains, you can see the inbuilt fault tolerance.

Let’s think about Bitcoin for an example.
That’s a blockchain that consists of over 30 thousand nodes coming to a consensus on every block. As long as we have two or more of those nodes online and able to communicate, we still have a working solution.
That gives us a tremendous margin for error, for nodes coming and going offline, for network transport issues, and it makes blockchain really a great platform to use in environments with less than ideal networks and power infrastructure. Because we can have nodes come offline, go back online and when a node comes back online after being offline for a while, all it has to do is sync up, and get all the data that it missed while it’s been offline from all of its peers, and then it’s right back online participating like all the rest.

This is very different from the centralised systems that blockchain aims to replace.

In a traditional client-server model, if that server is offline, those clients have no way of getting the data that they requested or performing the operations they’d like to perform.

This is not the case in blockchains.

And if we look back historically at other peer-to-peer solutions, solutions like BitTorrent or Napster, we’ve seen the tremendous difficulty that authorities have had taking some of these networks offline.
That is due to the fault tolerance you get from a peer-to-peer architecture.
In fact, we saw this recently during the Arab Spring, when the Egyptian government decided one night to shut down Internet access for the entire country.

Well, within 24 hours Egypt was back online and connected to the Internet through a network-sharing mechanism known as mesh networking, which at its heart is just a peer-to-peer method for sharing Internet connectivity.
So, we know that peer-to-peer has a long history of providing extremely high fault tolerance and reliability, and that’s why it’s been chosen to build a platform like a blockchain on top of it.

So, if you’re looking for a solution platform that offers you that kind of incredible fault tolerance, if you’re looking to deploy a solution into areas with less than ideal infrastructure or under conditions where nodes may come online and go offline frequently, then blockchain may be a really good platform to look at.


The Byzantine Generals’ Problem:

(you can read my dedicated article about it here: Byzantine Fault Tolerance In a Nutshell).

  • Several generals needing to agree on a coordinated plan of attack.
  • One or more generals may be traitors.
  • All generals will abide by the majority decision but may try to influence it.

Blockchains are designed to have Byzantine Fault Tolerance:

  • All nodes are untrusted.
  • Nodes must come to a consensus on the official state of the blockchain.

The Byzantine Generals’ Problem is a scenario designed to demonstrate the difficulty of multiple parties coming to an agreement when communication can only be accomplished on a one-to-one basis and is untrusted. In the story, several Byzantine Generals are surrounding a city with their separate armies. If they all attack together or all retreat together, they will be ok, but if some attack while others retreat, they will be destroyed.

The generals can only communicate by messengers, who could be intercepted and forced to carry fake messages, and one or more generals may be a traitor. The goal is to find a way to achieve a consensus on strategy despite the possibility of traitors and false messages. Presumably, all generals will abide by what they believe is the majority consensus. The Byzantine Generals’ Problem is solvable as long as two-thirds of the generals are honest.

Blockchains are designed to be Byzantine Fault Tolerant, meaning that the network will come to a consensus on the official state of the blockchain, despite the fact that some members may misbehave. The solution to the Byzantine Generals’ Problem is inefficient, so blockchains need some way of being confident of consensus without going through a full solution.


Proof of Work provides a game-theoretical distributed consensus algorithm:

  • Proof of Work incentivises mining nodes on the network to reach for the thermodynamic limit of computational cycles. This incentivises decentralisation because heat from mining nodes dissipates better in two separate places rather than one centralised location. Note, this decentralisation is solely physical and network distribution.
  • Proof of Work has empirically proven that game-theory can be weaved into a protocol because it successfully applies incentives at every possible action within the network.
  • Proof of Work only works because it is optimisation-free and approximation-free.

Optimisation-free means there is no possible way to circumvent the hashing of the mining protocol necessary to secure a block.

Approximation-free means there is no possible way to almost have a block. The process is binary; there are blocks and not blocks.

Proof of Stake provides an experimental internally game-theoretical consensus algorithm:

  • It relies on nodes already having cryptocurrency to stake. It rewards nodes with the most money staked, and not the most computational power.
  • It requires that each validating node be identifiable. This is because the staked coins must be held accountable for any malicious acts. Proof of Work does not require identification.
  • In Proof of Stake, you are competing with a much larger group of nodes. There is no transactional friction involved in staking coins, unlike in Proof of Work, which requires buying mining hardware, hooking up internet, providing cooling systems, etc.

Proof of Work versus Proof of Stake (summary)

Proof of Work is the oldest and the original consensus protocol. It’s coming up on its tenth anniversary; the first one into production with Bitcoin back in 2009. And as a consensus protocol, it served us very well.

There have been a number of hacks and exploits that have been committed against various smart contracts and solutions written on top of the blockchain. But for almost a 10-year history with over half a trillion dollar market cap, no one’s been able to successfully exploit Proof of Work itself,
which really shows the security and the reliability of the protocol.

However, there are some shortcomings and criticisms to Proof of Work that are now leading us to look at alternative consensus mechanisms, like Proof of Stake.

One of those is our transaction processing capability.
On a good day, Proof of Work is capable of processing anywhere between 10 and 20 transactions per second worldwide.

This may sound like a lot, but it still leaves us a wide gap to conventional processing powers, that something like Visa’s payment processing network which can scale up to over 70,000 transactions per second.

So, in order for blockchain to continue to be a successful solutions platform, we know that we’re going to need to find other consensus mechanisms, which allow us to scale up that transaction processing speed into a range where we start to compete with conventional technology.

There are also some other criticisms behind Proof of Work that are leading us to alternative methods, like Proof of Stake.

One of those is the idea of centralisation.

As you know from this post, one of the keys of blockchain is the idea of decentralisation, that no-one central authority, intermediary or participant, should ever have too much power or control in a blockchain network.

What we’re seeing right now with Proof of Work is an arm’s race, where folks are competing with very various specialised pieces of equipment, specialised hardware, specialised mining rigs, in order to mine most efficiently.
And this can be done most efficiently in large data centres where electricity is cheap.

Right now, almost 80% of the processing power behind the Bitcoin network resides in six major data centers in China.

A lot of advocates and blockchain purists think that this is far too much centralisation in one geopolitical region of the world. (I kinda agree with that).

One way we can aim to change that is through Proof of Stake, where we remove the work component of group consensus and we replace it with a specialised form of gambling.

The idea is that if we no longer require specialised hardware in order to come to a consensus, we can allow anybody with any kind of device to participate in consensus.

That may be you at home with an old laptop or a friend with a smartphone, or a tablet, or an iPad, that sits on your desktop most days and doesn’t get used.

This allows for a much wider and more decentralised range of devices and potentially a much larger network size to participate in consensus.

Speed and the idea of decentralisation are big drivers behind the move to Proof of Stake. And we’re going to see how well that works out.
We’ll see Casper finally go live in Ethereum later this year, certainly with more blockchains to follow if it becomes a successful consensus mechanism.

When you hear the debate these days about Proof of Work versus Proof of Stake, and you’re trying to understand what do they mean and why are we looking at transitioning from one to the other, the point to understand is that we’re just trying to overcome some of those big limitations behind Proof of Work, the consensus mechanism that has served us so well until now.

We’re trying to find consensus mechanisms that most importantly allow us to scale up, to get our transaction processing power on par with conventional technology, and it is also trying to remove some of the centralised aspects that we’ve seen to form around Proof of Work.


Attribution – This article is originally published on our Coinmonks publication by Demiro Massessi

Public Vs Private Blockchain In A Nutshell

When we try to understand the main difference between a public and private blockchain, it is important to notice that the terminology in the media gets often improperly stated.

A Public Blockchain is a permissionless blockchain. Anyone can join the blockchain network, meaning that they can read, write, or participate with a public blockchain. Public blockchains are decentralised, no one has control over the network, and they are secure in that the data can’t be changed once validated on the blockchain.

On the other hand, a Private Blockchain is a permissioned blockchain.
Permissioned networks place restrictions on who is allowed to participate in the network and in what transactions.

When people get into blockchain, there’s a natural discussion about what type of blockchain, because blockchain comes in many different types.

Normally, we hear about public versus private blockchains.
One factor that often gets left out is also the idea of an open versus closed blockchain. And it’s important to consider both parameters, so you know which possible solutions are good for your idea.

When we talk about public and private, what we’re really talking about is who is able to write data onto that blockchain or onto that ledger.
The open versus closed brings in to consideration who’s able to read that data.
And so, we can talk about solutions which are public and open, public and closed, private and open, private and closed.

When we talk about public blockchains, what most people are really talking about is a public open blockchain. It’s a blockchain where anybody can write data to the blockchain, and anybody else can come and read that data.
Public blockchain platforms like BitcoinEthereumLitecoin tend to get talked a lot right now, these are what we also refer to as permissionless blockchainplatforms, meaning that they really strive to, by design, increase and protect the user’s anonymity.

There’s a perception that public blockchain platforms like Ethereum can’t be used to build permission scenarios or to control access to data.
The truth is that they can, they just don’t give you all the built-in tools that you can find on a private or permission blockchain platform. But basically, you can always use these open public platforms to build a permission solution, you just need to be aware that it’s upon you, your architects and your developers to create that permissioning model, and that all starts with some kind of identity management system.

When we think of public blockchains, we think about blockchain platforms like Ethereum and Bitcoin, which by default, by the very nature, they’re designed to protect anonymity.

If we don’t know who a user is, we really have no way of creating permissions, role-based access and controlling what data they can read or write.
In a lot of situations, this is desirable, this is why we see cryptocurrencies based on public blockchain platforms, because having that anonymity is important, it is one of the major benefits of using cryptocurrencies.

Why?

Well because, if a user has a currency, something of value, they should be able to exchange it and spend it, and do what they want with it, just like anybody else. We don’t want to treat any class of users differently than any others in those scenarios. So, for that, we have public blockchain, and that’s a very different creature from the private permission blockchain.

Now, the counterpart to a public blockchain is naturally a private blockchain.
These are platforms like HyperledgerHashgraphCorda, etc.

Private blockchains are more specifically known as permissioned blockchains.
When we talk about private blockchain or when we hear people talking about private blockchain solutions, they tend to be talking about things on the private and closed end of the spectrum. We want to control who can write data to this blockchain, and we want to control who can read data from this blockchain. And in order to do that, the first step is identity. We need to know who is part of the blockchain network. If we don’t know who a user is, it becomes difficult, if not impossible, to define rules about what data they can commit to the ledger and what data they can consume from the ledger.

When we talk about private blockchain, think about a permissioned blockchain, which from the beginning has an idea of who you are.
And this is very different from a public platform like Ethereum, in which the platform tries to protect and maximise anonymity.
Which as said before, by design, don’t know who a user is.

Most of the time, private blockchains tend to come with identity management tools or a modular architecture, where you can plug in your own identity management solution. This can be anything from a Membership Service Provider to an OAuth solution using Google, Facebook, LinkedIn, etc.

This is the idea behind private blockchains.
All starts with understanding who a user is, because once we understand that,
we can determine what role they’re in, and we can use that role to determine what information should they and should they not have access to.

In Public Blockchain, we need to change a bit the incentive for good behaviour, because we don’t know who a user is.

We rely on economics and game theory incentives to ensure that everybody in the system behaves honestly and according to the rules.
We set up situations through group consensus, through which honest participants are economically rewarded, where dishonest ones only incur work or cost, with no possibility of ever recouping that cost.

In a Private Permissioned Blockchain, we rely on the fact that we know who a user is.

So, in a corporate case scenario, blockchain for the business, blockchain for supply value chains, because we know who an individual is, what organisation they’re associated with and what their role is, we also assume that they’re going to behave fairly, because if not, we know exactly who’s misbehaving and they know that they’re gonna suffer the consequences for that.

So, public and private blockchains have two very different offerings.
A lot of people get the impression that they compete with one another, but they really don’t. They just serve to provide different types of solutions.

Lets discuss what private blockchains are and why they are utilised by enterprises:

Private Blockchains are also referred to as Permissioned or EnterpriseBlockchains. Enterprises need to ensure some level of security, privacy, compliance, performance, and many of the properties that a private blockchain can provide.

Can be open sourced, consortium, or privately developed.
There are many options for a private blockchain, and the most common ones are Hyperledger, R3 Corda and Quorum.

Transactions are processed by selected nodes in the blockchain.
From a performance perspective, this is where having only a few nodes process transactions vs 12,000 nodes in Ethereum’s case can really create a performance gain around latency and transaction speed.

Transactions are not publicly viewable (transparent) in the blockchain, and only selected nodes can access the ledger.
Examples include: R3 Corda, which can transact between nodes, and the rest of the blockchain does not participate.

Some of the benefits of public blockchains are:

  • Open Read and Write
    Anyone can participate by submitting transactions to the blockchain, such as Ethereum or Bitcoin; transactions can be viewed on the blockchain explorer.
  • Ledger Is Distributed 
    The database is not centralised like in a client-server approach, and all nodes in the blockchain participate in the transaction validation.
  • Immutable 
    When something is written to the blockchain, it can not be changed.
  • Secure Due to Mining (51% rule)
    For example, with Bitcoin, obtaining a majority of network power could potentially enable massive double spending, and the ability to prevent transaction confirmations, among other potentially malicious acts.

Some of the benefits of private blockchains are:

  • Enterprise Permissioned 
    The enterprise controls the resources and access to the blockchain, hence private and/or permissioned.
  • Faster Transactions
    When you distribute the nodes locally, but also have much less nodes to participate in the ledger, the performance is faster.
  • Better Scalability
    Being able to add nodes and services on demand can provide a great advantage to the enterprise.
  • Compliance Support
    As an enterprise, you likely would have compliance requirements to adhere to, and having control of your infrastructure would enable this requirement more seamlessly.
  • Consensus More Efficient (less nodes)
    Enterprise or private blockchains have less nodes and usually have a different consensus algorithm, such as BFT vs POW.

Now that we have talked about public and private blockchains, we can talk about some of the differences and what each different type of platform aims to provide.

So, Public Blockchains are really good for scenarios where protecting the anonymity of users is important or adds value to the solution, and they’re really great platforms for solutions where all users should be treated equally.

And we can see this, because most public blockchains right now are used to support cryptocurrencies, and this is a real great use case.

With a cryptocurrency like Bitcoin, Litecoin, Ethereum, etc., we don’t want to have any permissioning or role-based access.
Anyone should be able to own some Bitcoin.
Anyone should be able to trade it with anyone else.
And so, that leads to an open permissionless model with full transparency.

This is very different in the corporate world, where we see private blockchains being adopted at scale, because the concerns are the opposite.

In a corporate scenario, in a business scenario, anonymity is a bad thing.
I want to know exactly who all the participants are, and I need to know who they are, because I don’t want full transparency.

I don’t want to share all my business data with all the participants in my business network or the public at large scale.

I want to control who sees what type of information under what circumstances, and I also want to control who is able to write that information onto the blockchain.

So, I might use a private blockchain solution to manage supplier-vendor relationships, where only myself and my suppliers can see the price that I’m paying for a certain item. Only a particular supplier gets to see the details of the contract that I have with them, and not the details of a contract I have with any other suppliers.

And I may wish to share some of this high-level macro data with consumers, so that they can see the origins of the products they’re buying, but of course I’d want to hide the financials aspect behind all that.

So, when we look at how these two solution types differ, we see the public blockchains tend to focus more on B2C or Business to Consumer scenarios,
whereas private blockchain like Hyperledger, really are well structured to B2B or Business to Business scenarios, supply chain, value chain relationships
or creating any kind of shared infrastructure between enterprises.

As we said before, it is important to remember that there’s a big misconception that public and private blockchains are competitors.
But, they’re not and many real-world use cases use components of both.

It’s important to understand the difference between them and it really all starts with identity management.

Basically, when you’re thinking in your head about public vs private blockchain think that it all begins with identity management, and in a private blockchain you know who all the participants are right from the beginning.
In the public blockchain, again, you don’t know who those participants are, and that’s not to say that you can’t build a permission solution on a public platform, but yourself, your architects, and your developers are going to have to develop the logic and the mechanisms behind identity management.

And that’s Public and Private Blockchain in a Nutshell. They are two very different things.

They serve different purposes and you will realise that a lot of real-world use cases you’re going to find make use of both types of blockchain integrated seamlessly.

Attribution – This article was originally published on our Coinmonks publication by Demiro Massessi

An introduction to the Block-Lattice

Next-generation payment cryptocurrencies like Nano and Nollar make some bold promises that sound almost too good to be true. They offer fast and feeless transactions without the need for environmentally-harmful mining. Today, we take a look at the system architecture underlying these next-gen payment cryptocurrencies, the so-called block-lattice, and compare it to its predecessor, the blockchain.

We will try to keep the explanations as simple as possible, without getting too deep into the technical details, so that readers that are new to the world of cryptocurrencies can get the most value out of this article. We will thus abstract away some of the technical aspects, which will be covered in future articles.


The block-lattice architecture was introduced by software developer and Nano founder Colin LeMahieu in his Nano white paper in 2014. LeMahieu took the basic principles that had been introduced in the Bitcoin white paper six years earlier and improved on them. His goal was to design a cryptocurrency which would meet the requirements of a global payment system and that whichcould thus compete with corporations like MasterCard, VISA or PayPal. Since the NOS protocol is derivedfrom Nano, it inherited all of its features and abilities.

To understand how the block-lattice architecture works and why it is needed, we should first take a look at traditional blockchains and their shortcomings.


blockchain is a data structure consisting of chronologically ordered blocks which contain pieces of information, where each of the blocks refers to the precedingblock, all the way to the first one, which is called the genesis block. This creates the eponymous chain-like structure:

The blockchain data structure offers some interesting characteristics that make it a useful tool to create a distributed ledger of information.

One of the most common use-cases of blockchain technology is the transfer of monetary value, because the chaining mechanism seamlessly tracks the transactional history of every participant on the network and makes sure that the possession of a single coin is not assigned to multiple accounts at the same time.Among others, each piece of information includes the sender and the receiver of a certain transaction and the number of coins to be sent. By analysing all the transactions from the genesis block to the current block, one can derive a ledger of all the accounts’ current balances.

How are new transactions created? The process of making a new transaction is defined in the protocol of the specific cryptocurrency. In the case of Bitcoin, this process includes the following (simplified) steps:

  1. The sender of a transaction authorizes and broadcasts his desired transaction to the rest of the network.
  2. Special network participantscalled miners pick up the broadcasted transaction and add it to a pool of unprocessed transactions.
  3. A competition starts, in which the winning miner chooses and verifies some of the unprocessed transactions (e.g. make sure that the sender has enough balance to make the desired transaction and that he cannot send the same coins to two different accounts). The successful miner adds the transactions to a new block and appends the block to the chain. He is rewarded for this effort with newly minted Bitcoins.
  4. Once the successful miner has added his new block to the chain and everyone else agreed to the result, a new competition between the miners starts.

In a system that uses Proof-of-Work, the miners try to guess a random number, called the nonce. The first miner to correctly guess the nonce wins. Because coming up with random numbers is a computational task, the probability of winning is proportional to the share of computational power that a miner adds to the total computational power of the entire network.


Blockchains have many use cases, like Distributed Storage, Digital Identities, Decentralized Apps or Smart Contracts. However, they have some inherent drawbacks that make them unsuitable as a tool for monetary transactions:

  • Firstly, the PoW system underlying traditional payment cryptocurrencies creates an economic incentive for the miners to use ever more computational power and thus consume more electricity.
  • Secondly, each block can only contain a limited number of transactions, creating a bottleneck that limits the transaction throughput of blockchain-based cryptocurrencies. For instance, the Bitcoin protocol can only handle a maximum of seven transactions per second. If the demand for transactions is high, the successful miner must choose which transactions to add to his block. To make sure that a specific transaction gets added, the sender includes a fee in the transaction. As a result, Bitcoin transactions become more costly when the demand for transactions is high. On December 21, 2017, the daily average transaction fee on the Bitcoin network reached an all-time high of $37.50.
  • Thirdly, blockchains are slow. In the case of Bitcoin, a new block is created every 10 minutes on average. To make sure that a transaction is settled and thus irreversible, the sender must wait at least 30 minutes. In a world where transactions at retail stores usually take just a few seconds, this amount of time is clearly not reasonable and will prevent the adoption of traditional cryptocurrencies for everyday transactions.

Now that we have a basic understanding of the functionality of blockchains and of their shortcomings, let us take a look at block-lattices and how they overcome these shortcomings.

To understand the architecture of a block-lattice, just imagine a row of parallel blockchains. Each of the chains belongs to one account on the network and is thus called an account chain. Only the owner of the account himself can update his account chain and append new blocks to it.

Icons made by Freepik from Flaticon

An account chain is created by broadcasting an open block. All subsequent blocks refer to the previous block on the same account chain. This design allows the network participants to update their account chains asynchronously of the other chains.

In contrast to a traditional blockchain system, the blocks in a block-lattice system do not contain thousands of bundled transactions. Instead, every transaction consists of a send block on the sender’s account chain and a corresponding receive block on the receiver’s account chain. This separation of transactions into two parts, combined with the fact that each network participant can alter his own chain allow the sender to make transactions even when the receiver is offline.

To make a transaction, the sender creates a send block which contains the address of the receiving account, the number of coins to be sent and a reference to the last block on his account chain. He then adds the block to his chain and broadcasts it to the network. Once the network agrees on the transaction’s validity, the coins are deducted from the sender’s balance and the transaction is settled. This entire process usually takes less than 3 seconds.

To receive the funds, the owner of the receiving account now simply creates a corresponding receive block and appends it to his own account chain. Once he broadcasts the block to the network, the funds are added to his balance.

The Nano protocol uses a consensus technique called delegated Proof-of-Stake (dPoS) to prevent users from double-spending their coins. According to dPoS, every account owner appoints a representative to verify new blocks on his behalf. If one of the network participants tries to double-spend his coins by broadcasting two blocks that refer to the same preceding block, the representatives vote on which block to keep and which one to discard. Their voting weight is proportional to the number of coins that other accounts have delegated to them. A block is considered settled once it has been verified by representatives with a combined voting weight of more than 50%. This way, only one of the conflicting blocks is kept and double-spending is prevented. Since the account chains are independent from another, the rest of the accounts can continue to make transactions during the voting process.

The dPoS-architecture allows for feeless and near-instantaneous transactions and a much higher transaction throughput, as it removes the costs and time associated with the block mining. On August 17, 2018, the Nano beta network reached a maximum transaction throughput of 756 transactions per second, compared to Bitcoin’s 7 transactions per second.

We at NOS are confident that next-gen payment cryptocurrencies like Nano or Nollar have the potential to replace traditional payment cryptocurrencies and meet the technical requirements to compete with the traditional financial institutions involved in the process of moving money around the world.

Attribution – This article was originally published on our Coinmonks publication by Nos.Cash

Advanced EOS Series — Part 1 — Cryptographic Hashes

Public Key Cryptography, Office of The Privacy Commissioner of Canada

Welcome to the Advanced EOS development series where I’ll be touching on techniques and functionality which is rarely covered by tutorials and courses. The purpose of this series is to bring together the missing pieces you’ll need to complete your skills as an application developer on the EOS network. Each post is ordered by difficulty, so if you’d like a general overview I’d recommend starting right here and working your way up. The full code for these examples can be found here on GitHub.MitchPierias/Advanced-EOS-Examples
A series of EOS contract development examples. Contribute to MitchPierias/Advanced-EOS-Examples development by creating…github.com

As these are advanced or extended topics, I’m dangerously assuming you already know the basics and are looking to further your knowledge. For that reason, the code shared in these articles will be concise to the topic being discussed.


The Cryptographic Hash Function

Let’s begin by looking at hashes, all resources for this example can be found here on GitHub. In cryptography, a hashing algorithm takes an input and generates a fixed sized jumble known as the output digest. The digest is deterministic and irreversible, meaning the same input will always generate the same output and the output can’t be reversed to obtain the original input. A good hashing algorithm minimizes output collision by using a ‘high degree of difficulty’ in the shortest amount of computation time. Collision or clashing being the likelihood of two different inputs producing the same output.

Application of Hashes

Typically we will use a hash function for authentication and indexing data, I like to think of it as a method of compressing uniqueness. For example, comparing two arbitrarily sized files or users emails would be time intensive and expose vulnerable user data. We could instead generate a fingerprint by hashing the files contents or users email and more efficiently index and compare the data using the hash result.

This is as about as much as we need to know at this point, but if you’d like to itch your curiosity then I’d encourage some proactive Googling or take a look at Blockgeeks article on The In’s and Outs of Cryptographic Hash Functions.


Coding Our Hash Function

Now that we have a very basic understanding of hashes, we can look into how to generate our very own within our EOS smart contract. We’re going to create an action which takes an input string and outputs a checksum256hash digest, then for the purpose of this example, print the result.

Let’s start by including the crypto.h library and the EOS print.h wrapper from the EOS framework.

include <eosiolib/crypto.h>

Notice the .h extension on our crypto.h filename? This means we’re using a Clibrary and not C++, so our methods will be expecting C type inputs.

Now we can define our action and allocate a reference for our checksum256output like so.

checksum256 sum{};

The EOS framework provides a set of methods for various hashing algorithms. We’ll use the sha256 for this example because it’s fast, secure and typical used in most cases.

sha256(const_cast<char*>(str.c_str()), str.size(), &sum);

Our sha256 method expects C type inputs, so we first need to convert our string to a char set, then pass the size of the string and a checksum256reference which will be updated with the output digest. Now all that remains is printing the hashed output result.

printhex(&sum, sizeof(sum));

We could also return the checksum256 here and use it as a private internal action.


Taking it Further — Checksum to String

Credit for this implementation goes to Miguel Mota for developing a solution to convert checksum256 too a hexadecimal string.

So what happens if we want to convert our digest back into a string? For this we will need a new method. To begin with, let’s define a template for our input argument and define the method itself. This template is defined outside of our method definition, typically at the top of our file. Templates simply allow us to define the variable type at the time we call the function, then the compiler will take care of substitution and making it all work.

template <typename CharT>
string to_hex(const CharT* data, uint32_t length) {
// Method Body
}

Now we can start working on the body of our method. To start with we will create a string for our result, a char set to draw characters from later, and force our data to a uint8_t.

string result;
const char* hex_chars = "0123456789abcdef";
uint8_t* c = (uint8_t*)data;

Finally we will iterate over each char in our input data, match the corresponding hexadecimal character, and append it to our resulting string.

for (uint32_t i = 0; i < length; ++i) {
(result += hex_chars[(c[i] >> 4)]) += hex_chars[(c[i] & 0x0f)];
}
return result;

And that’s it! We now have a method to create a checksum256 from a charset, and a method to convert that checksum256 into a string. In the next article we will be looking at singletons and how we can utilize them to store application state and configuration.

What’s next?

Make sure you click the follow button if you would like to be notified when I share more examples in this series. Part 2 will be covering how we can utilize Singletons in EOS to store contract state and configuration.

Attribution – This article was originally published on our Coinmonks publication by Mitch Pierias.

What we learned building our first Ethereum Dapp

Introduction

A couple of months ago, myself and my colleague Samm decided it was past time we learned Ethereum and Solidity development. After passively following Ethereum since its launch, we decided to build an Ethereum Dapp as a side project to help us better understand the underlying technology.

We had a few goals in mind for the Dapp we wanted to build:

  1. The Dapp had to be small in scope, but cover all the meaningful features of Ethereum and Solidity.
  2. The Dapp had to run without us needing to manage any cloud services ourselves — meaning we would only write client code and Ethereum smart contracts.
  3. We would open source whatever code we wrote and share what we learned.

What we ended up building was a small game called Eth Plot! Eth Plot was inspired by Reddit’s famous r/place April Fool’s joke and by the million dollar homepage. It allows you to buy “plots” of digital space on a grid. In the plot you purchase, you can place an image and a link to a website of your choosing. You can also resell your plots at any given time.

App screenshots showing a user purchasing a new plot

Feel free to check out the Dapp here! It is running on the mainnet as well as Ropsten, Rinkeby and Kovan. We hope some folks use the application, but our primary goal with this project was our own education. We learned a lot, and in this article, we will share with you what we learned during this endeavor.

Warning: this is a bit of a long read! Want to skip directly to the code? Here’s the main smart contract, and here is the React/Redux application that forms the UI. We hope the sample code will be helpful to those embarking on the journey of building their first Dapp.

Our intended audience for this article is developers who are familiar with Ethereum, but perhaps haven’t written their first Dapp yet and would like an overview of what it entails, accompanied by some tips and tricks we picked up along the way. We do not claim to be experts, and aren’t aiming to write another in-depth tutorial on Dapp development. Here’s what we will be covering in this article:

  • A high level overview of our tech stack — all the technologies we used to build Eth Plot.
  • The details of the Eth Plot smart contract — implementing the functionality needed for this app in an efficient way was surprisingly complicated.
  • Tips and tricks — a catch-all collection of helpful tidbits we learned while developing Eth Plot.

Our Tech Stack

For this project, we used the following stack:

  • Solidity Smart Contracts — a primary contract and one helper
  • IPFS for storing image data via Infura
  • Truffle and Ganache for our development and testing framework
  • React / Redux / Material UI / TypeChain for our front-end development
  • MetaMask for our web3 provider in production

Smart Contracts

The core functionality of Eth Plot (the ability to buy and sell plots in a decentralized manner) is only made possible by Ethereum’s smart contract functionality. We also make frequent use of events, which provide a cheaper form of storage and allow for responsiveness in our web app. Because the main smart contract that powers Eth Plot is the most interesting aspect of the project, we’ve dedicated an entire section of this article to discussing it.

IPFS

Trying to store image data on-chain is prohibitively expensive. As is now fairly standard, we instead upload the image to IPFS, receiving a hash of that image file in the process. This hash is what we store on-chain. This allows Eth Plot to remain purely decentralized despite not storing all the data on Ethereum.

To avoid the hassles associated with hosting and managing an IPFS node ourselves, we are using Infura’s IPFS service.

Truffle/Ganache

Truffle is a helpful set of tools that makes testing Ethereum contracts much easier. We used Truffle to compile and deploy our smart contracts to a local Ethereum node. Ganache is the local Ethereum node that we used, which is a lightweight JavaScript based Ethereum client. We also used Truffle as our testing framework to test our smart contracts.

React/Redux/Material UI/TypeChain

For our front end, we chose to use React, along with Redux for state management. This is a pretty standard web dev stack so we won’t spend any time here, but feel free to checkout the source code for details.

We also utilized components from the Material UI project. Material UI is set of React components that implements Google’s Material Design spec. This is a very well written library, with good documentation, that provides a bunch of nice looking components.

We enjoy writing our web based projects with TypeScript, so we utilized a helpful project called TypeChain which provides TypeScript bindings for Solidity contracts.

MetaMask

As is pretty standard, we require that users install MetaMask to interact with Eth Plot. Working with MetaMask was pretty straightforward, with a few exceptions:

  1. MetaMask is not ideal for local testing due to the unpredictable amount of caching. Even with the “Reset Account” feature of MetaMask, there are times where transaction state can be cached, making development difficult. Instead, we directly used the web3 provider from ganache while developing locally.
  2. Dealing with changes in user accounts for MetaMask state is not ideal. The recommended approach is to run an interval that repeatedly checks the current account.

That being said, MetaMask is a fantastic project, and the Dapp space would be nowhere near what it is today without MetaMask.


Smart Contract Deep Dive

The main smart contract which backs Eth Plot was one of the most challenging and unique aspects of the project. This was primarily due to how expensive it is to store data and perform lots of computations in a contract.

To start, we listed out the requirements of our contract:

  1. Define a grid system with constrained dimensions of at least 250×250.
  2. Each coordinate in the grid represents a 1×1 plot which has an owner, a buyout price, a website, and visual data associated with it.
  3. Plots can be larger than 1×1 to create a continuous area with an image
  4. To allow users to buy any plot they want — rather than being forced to buy an existing plot in its entirety — partial sections of plots can be sold and one plot can overlap another.

Naïve Approach

A naïve approach to the problem would be to create a contract with a large 2D array. Each entry in that array would represent a coordinate of the full grid and contain the corresponding information for that coordinate. When purchasing a plot, the buyer would send in a transaction which indicates all the coordinates they’d like to purchase and the data to associate.

Unfortunately, looking at the gas price of storing data (the SSTORE operation) it costs 20,000 gas to store a single 32 byte word. For our array we’d need to store at least 62,500 words (250×250) which would cost 1,250,000,000 gas. At a gas price of 5 gwei, that would cost ~6.25 ETH (~3,750 USD at the time of writing!) in transaction costs. This is also ~300 times higher than the maximum gas allowed in a single block (8,000,000 at time of writing). Even if we could get this kind of contract deployed, the transaction costs for interacting with it would be too high to ever see any adoption.

Our Approach — Contract Storage

Ultimately, we came up with an approach that makes efficient use of the limited resources available and allows Eth Plot to function as intended.

To start, let’s look at how we store the state of our contract. First, instead of a 2D array representing individual coordinates, we store summaries of plots: their starting coordinate (x & y) and dimensions (height & width). This has the benefit of larger plots not costing any more to store than smaller ones. The contract stores an array of these plots in the order they were purchased, with later plots appearing after earlier ones in the array. This is called the ownership array in the contract. This ownership array contains an entry for each plot which includes its geometry (x, y, w, h) and the owning address. Because the geometry has a range of 0–250, we can store each of the four values in a three byte uint24 variable. The owner’s address is 20 bytes, which means each entry in the array takes 20 + (4 * 3) = 32 bytes. This is the exact word size in the EVM, making the storage more efficient.

With this data representation, you might wonder how we support selling subsections of plots. We accomplish this with the holes mapping, which tracks information about which plots overlap each other, meaning that the later plot bought a subsection of the earlier plot. Here is an example representation of the relationship between the ownership and holes arrays.

Adding this overlap logic was crucial as it allowed us to efficiently verify that the plot someone is trying to purchase is valid. By keeping the holesmapping up-to-date, we can keep much less state in memory when validating new purchases — preventing us from exceeding gas limits as seen in the naïve approach.

Next, instead of storing the image data directly in the blockchain, we upload the image for a particular plot to IPFS, then store the IPFS hash of that image in the contract. Along with the image hash, we store the website associated with the plot. This information is stored in its own mapping separate from the ownership object so we don’t need to read it into memory while computing new purchases, making purchase transactions cheaper. This information is stored in the data mapping.

Finally, we store the current buyout price for a particular plot in a separate mapping called plotIdToPrice. The buyout price for a plot can be updated by the owning user at any point. Just like the data mapping, this information is stored separately from the ownership data because we will access it much less often (only when computing payouts).

Our Approach — Purchasing Plots

Now that we’ve looked at the data storage, let’s walk through what happens when purchasing a plot. The function you call to purchase a new plot is called purchaseAreaWithData. The data is specified in a somewhat unique format to make the execution of the contract as efficient as possible.

In our initial versions, the caller simply passed in the rectangle they wanted to purchase, and the contract attempted to compute all of the sub-plots which needed to be purchased. This worked in small cases, but we quickly pushed the limits of the EVM and encountered inflated transaction costs and stack-too-deep errors caused by the amount of data which was loaded into memory. Instead, the caller of the method (the web app in our case) does all of the computation beforehand and the contract merely validates this data and transfers funds. To do this, the caller sends in an array of sub-plots which form a complete tiling of the purchased area. These sub-plots represent sections of the existing plots which are about to be purchased. In addition to the sub-plots, the indices where these sub-plots exist are also passed into the function.

The contract itself is heavily commented and worth taking a look at, but here is a high level overview of what the purchase function does:

  • Validates & bounds checks input parameters.
  • Check to see that the sub-plots passed in form a complete tiling of the plot being purchased.
  • Checks to make sure that all of the sub-plots being purchased are for sale and are still owned by the plot which the caller said they are purchasing from (this is where the holes structure is used).
  • Pays out the owners of all the sub-plots with the funds sent in with the transaction and emits events for the transfers.
  • Stores the new plot, the new data, the buyout price, updates all of the holes arrays with all the new purchases, and emits a purchased event.

Other Functionality

The remainder of the contract contains a few trivial functions which are less interesting. This includes owner accessible functions for changing the buyout price of a plot and changing the plot’s data, as well as admin functions for marking content as illegal and withdrawing the funds which have collected in the contract. Finally, there are some view functions which make reading the contract’s data more efficient by aggregating information from the various separate data structures.

Why Not Use ERC-721?

When we were working on our design, we came across the ERC-721 specification which looked promising, but ultimately didn’t fit with the requirements we had. The biggest issue we weren’t able to solve with ERC-721 was how to subdivide plots to allow for selling just a section of a plot. The other issue we saw with ERC-721 was there was no way to recombine the tokens into larger plots once they had been subdivided.


Tips and Tricks

We had some other interesting takeaways from this project which we wanted to share.

It’s tough to determine transaction costs, so write some tests then guess and check — One of the most useful optimization techniques for us was towrite some unit tests which ran through what we thought was a representative set of transactions for our contract. By calculating the gas cost of this test, we could experiment with refactoring the contract and seeing what impact that had on gas cost. We were able to make the contract~40% more efficient using this technique.

Compute off-chain, validate on-chain — Any computation you can move out of the contract is worth it. If you find yourself writing complex contract code, see if you can instead compute more things off-chain and just have the contract validate that data. For example, rather than doing all the purchase computations in the contract itself, we pushed much of that work to the client — code sample here.

Pay attention to how storage is structured, mappings may be better than arrays — One interesting observation which came out of our optimization exercises was how much of a difference it made to split up our related data into multiple objects. Since the EVM has to load full words into memory at a time, only storing the plot geometry and owner in the ownership array prevented loading unneeded data about images, buyout prices, etc. Additionally, using mappings instead of parallel arrays for those other data structures was much cheaper because we don’t need to update the length value of the arrays (which involves an expensive SSTORE operation).

The truffle debugger is massively helpful, when it works— Using the truffle framework was very helpful as it allowed us to deploy local testnets, write unit tests, and debug contracts. The truffle debugger (truffle debugcommand) was really helpful for debugging transactions which failed during manual testing. Unfortunately, due to how truffle unit tests are set up, the debugger doesn’t allow you to debug transactions originating from unit tests.

truffle doesn’t enable optimization by default — while writing this article, we realized that truffle compile doesn’t enable contract optimization by default. Turning it on saved another ~20% gas in our tests. You can enable it via the truffle-config file.


Conclusion

Thank you for reading! We learned a great deal during this project and plan to do much more Ethereum development in the future.

Again, checkout the Eth Plot application in action! We aren’t looking to earn whatever fake internet points are available here, but we would love to engage with you, so please leave any comments or questions here or on the GitHub repo. Everything is open sourced, so take a look if you are searching for some starter code for the development of your first (or next) Dapp.

Update: August 2018

We learned something else after having this on the mainnet! If you’ve tried out the app recently on the mainnet, you’ll notice it’s painfully slow to load in the beginning. For those curious, the reason for this is that we are storing our app data in event logs for the sake of price. The downside here is that it is extremely time consuming to search through logs, especially on the mainnet. We didn’t notice this flaw when testing locally and on test nets due to size.

In order to solve this, we would need to add a dedicated caching layer. We likely won’t spend the time adding it to EthPlot since we are working on some other projects currently, but it’s helpful to keep in mind. I can foresee a future Infura (or Infura competitor!) making a feature to help with this issue as well.

Update: November 2018

EthPlot was one of the motivating factors that resulted in us starting Nodesmith.io. We wanted to build a service that gives you everything you need to build a decentralized application that doesn’t feel like it’s usability is hindered by the underling blockchain infrastructure. With Nodesmith, we’re giving you APIs that not only give you access to underlying blockchain infrastructure (like sending transactions and reading network state), but also gives you services that supercharge your application to make it more user friendly. For example, we’re adding smart contract event caches to make reading event logs really fast. We’re also adding a webhooks API so you don’t have to constantly poll the network.

Check the project out on Twitter or Medium if you want more information!

Attribution — This article was originally published on Coinmonks publication by Brendan Lee

How to Generate a Bitcoin Address — Step by Step

TL;DR: Here is a bash script that does what is outlined below: https://bit.ly/2MIgeOD

Introduction

This is a hands on, technical guide about the generation of Bitcoin addresses including private and public keys, and the cryptography involved.

This guide will walk you through all the steps to generate a Bitcoin address using the command line on a Mac. Similar steps should be possible on other operating systems using similar cryptographic tools. Lines starting with $denote terminal commands, which you can type and run (without the $ of course).

Mac Terminal (iTerm 2)

Dependencies

  • brew — Installation: https://brew.sh/
  • pip — Installation: sudo easyinstall pip
  • libressl — Installation: brew install libressl
  • base58 — Installation: pip install base58

Note: To do the contained opensslcli commands, I installed libressl in order for some of the elliptic curve commands to work as the current version of openssl cli on mac has a bug.

Cryptography Primer

Public Key Cryptography

Or asymmetric cryptography, is a type of cryptography that uses key pairs, each of which is unique. The pair of keys includes a public key and a private key. This is the type of cryptography that Bitcoin uses to control funds. A public key can be generated from a private key, but not vice-versa (computationally too difficult). Also, something encrypted with a private key can be decrypted with the public key, and vice-versa, hence they are asymmetric.

Assymetric Encryption

Use Cases

  • Encryption: When a user has a public key, a message can be encrypted using a public key, which can only be read by the person with the private key. This also works in reverse.
  • Digital Signatures: A user can, with their private key and a hash of some data, use a digital signature algorithm such as ECDSA, to calculate a digital signature. Then, another user can use the algorithm to verify that signature using the public key and the hash of the same data. If it passes, this proves a user did in fact submit a specific message, which has not been tampered with.
  • Digital Fingerprint: Is a way to represent an arbitrarily large data set by computing the hash of it to generate a fingerprint of a standard size. This fingerprint would be so difficult to replicate without the same exact data, which can be assumed to have not been tampered with.

Private keys are what prove you can send Bitcoin that has been sent to you. It is like the password to your bank account. If you lose it or someone else gets a hold of it, you’re toast.

Public keys help people know how to send you Bitcoin.

Creating a Bitcoin Address

Private Key Generation

Private keys can be any 256 bit (32 byte) value from 0x1 to 0xFFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF BAAE DCE6 AF48 A03B BFD2 5E8C D036 4140 .¹

The total possible number of private keys is therefore 2²⁵⁶ or 1.16 x 10⁷⁷. Imagine the total number of atoms in your body, then imagine that each of those atoms is an earth. The total number of atoms on all of those earths is about 7 x 10⁷⁷.² There is virtually no chance that your random private key will ever be generated randomly or found by someone else.

A common way of creating a private key is to start with a seed, such as a group of words or passphrases picked at random. This seed is then passed through the SHA256 algorithm, which will always conveniently generate a 256 bit value. This is possible because every computer character is represented by an integer value (see ASCII and Unicode).

Note: SHA256 is a one-way, deterministic function meaning that it is easy to compute in one direction, but you cannot reverse it. In order to find a specific output, you have to try all the possible inputs until you get the desired output (brute forcing) and it will always produce the same output given the same input, respectively.

The seed can be used to generate the same private key if the same hashing algorithm is used in the future, so it is only necessary to save the seed.

$ echo "this is a group of words that should not be considered random anymore so never use this to generate a private key" | openssl sha256

a966eb6058f8ec9f47074a2faadd3dab42e2c60ed05bc34d39d6c0e1d32b8bdf

This private key is in hexadecimal or base 16. Every 2 digits represents 8 bits or 1 byte. So, with 64 characters, there are 256 bits total.

Public Key Generation

Public keys are generated from the private keys in Bitcoin using elliptic curve (secp256k1) multiplication using the formula K = k * G, where K is the public key, k is the private key, and G is a constant called the Generator Point⁴, which for secp256k1 is equal to:

04 79BE667E F9DCBBAC 55A06295 CE870B07 029BFCDB 2DCE28D9 59F2815B 16F81798 483ADA77 26A3C465 5DA4FBFC 0E1108A8 FD17B448 A6855419 9C47D08F FB10D4B8

It doesn’t seem to be known how this point was chosen by they designers of the curve. Also, this algorithm is a one-way algorithm, or a “trap door” function so that a private key cannot be derived from the public key. It is important to note that elliptic curve multiplication is not the same as scalar multiplication, though it does share similar properties.

Elliptic Curve Example

To do this in the terminal from our private key earlier,

$ openssl ec -inform DER -text -noout -in <(cat <(echo -n “302e0201010420”) <(echo -n “a966eb6058f8ec9f47074a2faadd3dab42e2c60ed05bc34d39d6c0e1d32b8bdf”) <(echo -n “a00706052b8104000a”) | xxd -r -p) 2>/dev/null | tail -6 | head -5 | sed ‘s/[ :]//g’ | tr -d ‘\n’ && echo

043cba1f4d12d1ce0bced725373769b2262c6daa97be6a0588cfec8ce1a5f0bd092f56b5492adbfc570b15644c74cc8a4874ed20dfe47e5dce2e08601d6f11f5a4

This public key contains a prefix 0x04 and the x and y coordinates on the elliptic curve secp256k1, respectively.

Compressed Public Key

Most wallets and nodes implement compressed public key as a default format because it is half as big as an uncompressed key, saving blockchain space. To convert from an uncompressed public key to a compressed public key, you can omit the x value because the y value can be solved for using the equation of the elliptic curve: y² = x³ + 7. Since the equation solves for y², the right side of the equation could be either positive or negative. So, 0x02is prepended for positive y values, and 0x03is prepended for negative ones. If the last binary digit of the y coordinate is 0, then the number is even, which corresponds to positive. If it is 1, then it is negative. The compressed version of the public key becomes:

023cba1f4d12d1ce0bced725373769b2262c6daa97be6a0588cfec8ce1a5f0bd09

The prefix is 0x02because the y coordinate ends in 0xa4, which is even, therefore positive.

Address Generation

There are multiple Bitcoin address types, currently P2SH or pay-to-script hash is the default for most wallets. P2PKH was the predecessor and stands for Pay to Public Key Hash. Scripts give you more functionality, which is one reason why they are more popular. We’ll first generate a P2PKH original format address, followed by the now standardP2SH .

Hash

The public key from the previous output is hashed first using sha256 and then hashed using ripemd160 . This shortens the number of output bytes and ensures that in case there is some unforeseen relationship between elliptic curve and sha256, another unrelated hash function would significantly increase the difficulty of reversing the operation:

$ echo 023cba1f4d12d1ce0bced725373769b2262c6daa97be6a0588cfec8ce1a5f0bd09 | xxd -r -p | openssl sha256
(stdin)= 8eb001a42122826648e66005a549fc4b4511a7ad3fc378221aa1c73c5efe77ef

$ echo 8eb001a42122826648e66005a549fc4b4511a7ad3fc378221aa1c73c5efe77ef | xxd -r -p | openssl ripemd160
(stdin)= 3a38d44d6a0c8d0bb84e0232cc632b7e48c72e0e

Note that since the input is a string, the xxd -r -p will convert the hex string into binary and then output it in hexdump style (ascii), which is what the openssl hashing functions expect as input.

Encoding

Now that we have hashed the public key, we now perform base58checkencoding. Base58check allows the hash to be displayed in a more compact way (using more letters of the alphabet) while avoiding characters that could be confused with each other such as 0 and O where a typo could result in your losing your funds. A checksum is applied to make sure the address was transmitted correctly without any data corruption such as mistyping the address.

Base58 Encoding Table

Address format

Bitcoin P2PKH addresses begin with the version byte value 0x00 denoting the address type and end with a 4 byte checksum. First we prepend the version byte (prefix) to our public key hash and calculate and append the checksum before we encode it using base58 :

$ echo 003a38d44d6a0c8d0bb84e0232cc632b7e48c72e0e | xxd -p -r | base58 -c && echo
16JrGhLx5bcBSA34kew9V6Mufa4aXhFe9X

Note: -c denotes a checksum is to be applied. The checksum is calculated as checksum = SHA256(SHA256(prefix+data)) and only the first 4 bytes of the hash are appended to the end of the data.

The resulting value is a P2PKH address that can be used to receive Bitcoin: 16JrGhLx5bcBSA34kew9V6Mufa4aXhFe9X

Pay-to-Script Hash

The new default address type is a pay-to-script-hash, where instead of paying to a pubKey hash, it is a script hash. Bitcoin has a scripting language, you can read more about it here. Basically it allows for things like multiple signature requirements to send Bitcoin or a time delay before you are allowed to send funds, etc. A commonly used script is a P2WPKH (Pay to Witness Public Key Hash): OP_0 0x14 <PubKey Hash> where the PubKey Hash is the RIPEMD160 of the SHA256 of the public key, as before, and 0x14 is the number of bytes in the PubKey Hash. So, to turn this script into an address, you simply apply BASE58CHECK to the RIPEMD160 of the SHA256 of the script OP_0 0x14 <PubKey Hash> except you prepend 0x05 to the script hash instead of 0x00 to denote the address type is a P2SH address.

$ echo 00143a38d44d6a0c8d0bb84e0232cc632b7e48c72e0e | xxd -r -p | openssl sha256
(stdin)= 1ae968057eaef06c3e13439695edd7a54982fc99f36c3aa41d8cc41340f30195

$ echo 1ae968057eaef06c3e13439695edd7a54982fc99f36c3aa41d8cc41340f30195 | xxd -r -p | openssl ripemd160
(stdin)= 1d521dcf4983772b3c1e6ef937103ebdfaa1ad77

$ echo 051d521dcf4983772b3c1e6ef937103ebdfaa1ad77 | xxd -p -r | base58 -c && echo
34N3tf5m5rdNhW5zpTXNEJucHviFEa8KEq

Attribution– This article was originally published on our Coinmonks publication by Jordan Baczuk


Step by Step Towards Hyperledger Fabric — Part 1

Blockchain is the new buzz word in the industry now a days. But finding good resources to learn this fascinating technology is not a cake walk. To make it easier for others I am starting a series in the field of Blockchain & Distributed Ledger Technology (DLT) platforms. I will try to cover each and every step require to get a hands on these technologies. Firstly, I am starting with Hyperledger Fabric.

Hyperledger Fabric is one of the projects under Hyperledger umbrella.

Hyperledger is a platform for distributed ledger solutions, underpinned by a modular architecture delivering high degrees of confidentiality, resiliency, flexibility, and scalability.

Lets start over first Step by Step hands on:

Firstly we have to learn how to install the prerequisite for the Hyperledger Fabric:

  • Install cURL.
  • Install Go Language.
  • Install Docker and Docker Compose.
  • Install Git.
  • Install Pip
  • Install Hyperledger Fabric on Ubuntu LTS 16.0.4

Once we are done with installing prerequisite, we will be going to hands on with:
1. Build & Deploy your first-network on Hyperledger Fabric
2. Build & Deploy a demo example Fabcar on Hyperledger Fabric

We are using Ubuntu LTS 16.04 for this hands on tutorial:

Step 1:
Press Ctrl + Alt + T to open a terminal
$ sudo su
Enter your password
$ cd

Step 2: Install Google golang
$ cd $HOME/ && wget https://storage.googleapis.com/golang/go1.8.1.linux-amd64.tar.gz

$ tar -xvf go1.8.1.linux-amd64.tar.gz

Set the go path
$ mkdir $HOME/gopath
$ export GOPATH=$HOME/gopath
$ export GOROOT=$HOME/go
$ export PATH=$PATH:$GOROOT/bin
$ go version

Step 3: Install libltdl-dev
$ apt-get install libltdl-dev


Step 4: Install docker-­ce 
$wget https://download.docker.com/linux/ubuntu/dists/xenial/pool/stable/amd64/docker­-ce_17.06.0~ce­-0~ubuntu_amd64.deb

$ dpkg ­i docker-­ce_17.06.0~ce-­0~ubuntu_amd64.deb

$ docker -­ -­version

Note: The above approach simply leverages the Docker images that the Hyperledger Fabric project publishes to Docker Hub

$ docker run hello­-world

Step 5: Installl python-­pip
$ apt-­get install python­-pip

$ pip ­- -­version

Step 6: Install docker­-compose
$ pip install docker­-compose
$ docker­-compose ­- -­version

Step 7: Install git
$ apt­-get install git
$ git ­-­ -version

Step 8: Install curl
$ apt-­get install curl
$ curl ­- -version

Step 9: Install node.js and npm
$ curl ­sL https://deb.nodesource.com/setup_8.x | sudo ­-E bash ­-

$ node -­ -­version
$ npm ­- -version

Step 10: Clone the fabric-­samples from the github
$ git clone https://github.com/hyperledger/fabric-­samples.git

Step 11: Enter the fabric-­samples directory and install the platform specific binaries
$ cd fabric-­samples
$ curl -­sSL https://goo.gl/byy2Qj | bash -­s 1.0.5

$ docker images

If everything goes well you will see the above output on your screen.

Step 12: To have a look on the download binaries execute the following from your terminal:
$ cd bin
$ ls

Step 13: Enter into the first-network directory
$ cd ../
$ cd first-network
$ ls

Step 14: Generate the required certificates and articates for your first network
$ ./byfn.sh -m generate

Step 15: To see the generate certificates use the following command:
$ ls
$ cd crypto-config
$ ls

Step 16: Create your first network using the following command
$ cd ../
$ ./byfn.sh -m up

You will see the above message once your first-network is created using Hyperledger Fabric.
Step 17: Check the generates images and running containers using the following command:
$ docker images

$ docker ps

Step 18: To bring down the created network executed the following command
$ ./byfn.sh -m down

Step 19: You can check the created images have been removed using the following:
$ docker images

We have successfully created our first-network using Hyperledger Fabric.

Now lets try another example
Step 20: Move to the fabcar directory
$ cd ../
$ ls
$ cd fabcar

Step 21: Install the node modules using the following command
$ sudo npm install

Step 22: Install grpc module for communication with Hyperledger Fabric using the following command:
$ sudo npm install grpc

Step 23: Start the Hyperledger Fabric network for fabcar by executing the following command:
$ ./startFabric.sh

Step 24: To enroll the users firstly you have to enroll an Admin that will help to enroll other users with
Hyperledger Fabric network of Fabcar.
$ node enrollAdmin.js

You can find the private and public key for admin using the following
$ ls
$ cd hfc-key-store/
$ ls

Step 25: Enroll the user to query and invoke fabcar network (As Hyperledger Fabric is a permissioned
blockchain that is why firstly we have to register the user using its certificate)
$ cd ../
$ node registerUser.js

Step 26: Query the Fabcar network using the following command (To access the ledger state)
$ node query.js

Step 27: Now lets try to submit a new record for our Fabcar application on top of Hyperledger Fabric
$ gedit invoke.js

Make the below changes in the file above to create a new record:

Step 28: Now invoke the transaction using the below command. (To make any changes in the ledger state we have to invoke function)
$ Ctrl + c
$ node invoke.js

Step 29: Query the ledger to find the changes made by you using the following
$ node query.js

Congrats to all for successfully running Fabcar demo. In the next part i will going to cover technical details on these two examples. If you have any specific topic that you want to be covered from these two examples make a comment to include it.

Attribution– This article was originally published on Coinmonks Jobs by Ashish

How to run a Lightning Network node on Windows

This article will go through the steps needed to setup a Lightning Network (LN) node on Bitcoin mainnet using a Windows computer. We will be using ACINQ’S eclair implementation.

Requirements:

– A computer capable of running a Bitcoin full node

– A small amount of bitcoin that could potentially be lost (ideally at least $5 worth)

I will also assume you have a general idea of what the LN is (article explaining Lightning Network here: https://medium.com/@johnny_pham/lightning-network-how-does-it-work-ceeeda8ad21e). This guide is based on the one found on eclair’s github (https://github.com/ACINQ/eclair and https://github.com/ACINQ/eclair/wiki/Configure) and has been modified to be more beginner-friendly.

Why should you run a LN node?

1. Harness the full power of the LN

2. Support the development of both the Bitcoin network and LN

3. Learn about a new technology

4. Be part of a digital network that may change the world

WARNING: eclair and other LN implementations are still in beta and any funds committed to your LN node could be permanently lost. Use only amounts you are comfortable with losing.

Ok, let’s get started!

What is eclair?

Eclair is simply one implementation of the LN; in other words, it is one of many available programs that allows users to interact with the LN, other popular examples being c-lightning and lnd. This is similar to how Bitcoin Core is only one way of running a Bitcoin full node. Bitcoin Core itself is not Bitcoin, just like eclair itself is not the LN.

Step 1. Download eclair at https://github.com/ACINQ/eclair/releases. The first link contains the executable installer. Install eclair on your Windows computer.

Step 2. Ensure your Bitcoin full node meets the following specifications.

a. It must be version 0.16 or higher. The current version is 0.16.1. Download the latest version if needed.

b. It must be non-pruning; that is, it must contain a complete copy of the Bitcoin blockchain. Download the blockchain if needed.

c. It must be a transaction indexing node. This is done by modifying your bitcoin.conf file, which you will have to create if you don’t have one yet:

Go to C:\ > Users > your_username > AppData > Roaming > Bitcoin

If you cannot find the AppData folder, do the following:

Search your computer for “File Explorer Options”. Go to the “View” tab. Select “Show hidden files, folders, and drives”.

In your Bitcoin folder, create a new text document called “bitcoin.txt”. Edit the properties of the file and rename the “.txt” extension to “.conf” (or if your file explorer is set up to view extensions, simply change the extension by renaming the file). Now open up bitcoin.conf with a text editor and add the following:

testnet=0
server=1
rpcuser=your-rpc-user-here
rpcpassword=your-rpc-password-here
txindex=1
zmqpubrawblock=tcp://127.0.0.1:29000
zmqpubrawtx=tcp://127.0.0.1:29000
addresstype=p2sh-segwit

If your version of Bitcoin Core is 0.17 or higher, add the following to bitcoin.conf:

deprecatedrpc=signrawtransaction

Your rpcuser and rpcpassword can be set to anything. Leave everything else as it is. Your bitcoin.conf should look something like this:

Contents of bitcoin.conf

Save your updated bitcoin.conf file.

You will now have to restart Bitcoin Core for the changes to take effect. When it starts up again, it will begin indexing the entire blockchain. This can take up to a few days depending on how fast your computer is. Be patient; after this, there’s not much more to go before your LN node is up and running!

The estimated time is not accurate

You can continue to step 3 while waiting for your full node to index.

Step 3. We will now edit eclair’s configuration in a similar manner. Create an “eclair.conf” file in C:\Users\your_username\.eclair:

Add the following to eclair.conf:

eclair.chain=mainnet
eclair.bitcoind.rpcport=8332
eclair.bitcoind.rpcuser=yourusername
eclair.bitcoind.rpcpassword=yourpassword
eclair
eclair

Set “yourusername” and “yourpassword” to the username and password you specified in bitcoin.conf.

You can give your LN node any name by setting eclair.node-alias.

Eclair.node-color is the color your node will have on LN explorers such as https://lnmainnet.gaben.win/. The color is specified in hexadecimal format.

Your eclair.conf should look something like this:

Contents of eclair.conf

Save your changes to eclair.conf.

Step 4. Once your Bitcoin full node is indexed and synced, start up eclair with your Bitcoin node running in the background. If you get an error message from eclair saying that a Bitcoin full node could not be found, your full node is not properly configured. Go through step 2 again.

Step 5. To establish your first LN payment channel, you will need to fund your Bitcoin Core wallet. Eclair will use any funds it finds in your Bitcoin Core wallet. Any returned bitcoin from closing a channel is also sent back to your wallet.

Send some bitcoin to your Bitcoin Core wallet. Again, since eclair is still in beta, any funds used to open a LN payment channel can be permanently lost. Use amounts you don’t mind losing. Some nodes will require payment channels to be above a certain value to connect to them, so I recommend at least 60,000 satoshis (0.0006 BTC, about $5 worth at the time of writing).

Step 6. Once your wallet is funded, open a channel in eclair:

You will need to find a target node URI. If you don’t already have one in mind, you can find one at https://lnmainnet.gaben.win/ or https://1ml.com/. Click on a node and look for the URI of the node.

Choose how much bitcoin you want to fund your channel with. Do not send all of your bitcoin or you may get an error. The fee rate is the fee with which your bitcoin transaction will be sent. The push amount is optional. Double check everything and then click connect. Your channel’s status will be updated:

Once your transaction confirms on the Bitcoin network, your LN payment channel has been established:

That’s it! You now have an operational Bitcoin full node and Lightning Network node.

You can check for your node at https://lnmainnet.gaben.win/. It usually takes no more than 30 mins for a node to appear.

Attribution–This article was originally published on Coinmonks publication by Johnny Pham

How does blockchain work in 7 steps — A clear and simple explanation

In 2017 the spirit of Bitcoin finally got out of the bottle. After years of silence, Bitcoin suddenly rose to almost $20,000 and everybody suddenly turned into a ‘professional crypto trader’.

Yes, there was a lot of money to be made for a short period of time. But let’s face it; that party is over now.

Fact is though that even though it was mostly the money that got most of the adrenaline pumping, it is the technology that got the ball rolling in the first place. And this technology, blockchain, still holds huge potential. Now is actually the time for business developers, entrepreneurs and curious individuals to jump on the blockchain train and to be inspired. But such inspiration will require a better understanding of how the technology works first. Unfortunately, most of the current explanations out there are either covered in complex technical jargon or are way too shallow and lack in-depth details, neither of them which leads to a clear understanding. So where to start? Allow me to suggest you to start here. This ten minute read will explain why blockchain is supposedly so revolutionary. It will be well worth your time. Enjoy reading.


Introduction

A blockchain is a digital concept to store data. This data comes in blocks, so imagine blocks of digital data. These blocks are chained together, and this makes the data immutable. When a block of data is chained to the other blocks, its data can never be changed again. It will be publicly available to anyone who wants to see it ever again, in exactly the way it was once added to the blockchain. That is quite revolutionary, because it allows us to keep track records of pretty much anything we can think of (to name some: property rights, identities, money balances, medical records), without being at risk of someone tampering with those records. If I buy a house right now and add a photo of the property rights to a blockchain, I will always and forever be able to prove that I owned those rights at that point. Nobody can change that information if it is put on the blockchain. So, it is a way to save data and make it immutable. That sounds great, but the big question of course is: How does that work?

Step 1 — Transaction data

Alright, let’s start off with an example: the Bitcoin blockchain. The Bitcoin blockchain is the oldest blockchain in existence. The blocks on the Bitcoin blockchain are 1 MB of data each. At the time of writing it counts about 525,000 blocks, meaning roughly a total of 525,000 MB has been stored on this blockchain. The data on the Bitcoin blockchain however, only exists out of transaction data in regard to Bitcoin transactions. It is a giant track record of all the Bitcoin transactions that have ever occurred, all the way back to the very first Bitcoin transaction. This article refers to a blockchain that stores transaction data.

Step 2 — Chaining the blocks (with a hash)

Imagine a bunch of blocks of transaction data (image 1).

Not really special yet, you can compare it to some stand-alone word documents. Document 1 would then chronologically describe the first transactions that have occurred up to 1 MB, where after the next transactions would be described in document 2 up to another MB, and so on. These documents are the blocks of data. These blocks are now being linked (aka chained) together. To do this, every block gets a unique (digital) signature that corresponds to exactly the string of data in that block. If anything inside a block changes, even just a single digit change, the block will get a new signature. This happens through hashing and will be thoroughly explained in step 3.

Let’s say block 1 registers two transactions, transaction 1 and transaction 2. Imagine that these transactions make up a total of 1 MB (in reality this would be much more transactions). This block of data now gets a signature for this specific string of data. Let’s say the signature is ‘X32’. Here is what this looks like:

Remember, a single digit change to the data in block 1 would now cause it to get a different signature! The data in block 1 is now linked to block 2 by adding the signature of block 1 to the data of block 2. The signature of block 2 is now partially based on the signature of block 1, because it is included in the string of data in block 2. Here is what this looks like:

The signatures link the blocks together, making them a chain of blocks. Let’s picture adding another block to this chain of blocks; block 3. Here is what this looks like:

Now imagine if the data in block 1 is altered. Let’s say that the transaction between Damian and George is altered and Damian now supposedly sent 500 Bitcoin to George instead of 100 Bitcoin. The string of data in block 1 is now different, meaning the block also gets a new signature. The signature that corresponds with this new set of data is no longer X32. Let’s say it is now ‘W10’ instead. Here is what happens now:

The signature W10 does not match the signature that was previously added to block 2 anymore. Block 1 and 2 are now no longer chained to each other. This indicates to other users of this blockchain that some data in block 1 has been altered, and because the blockchain should be immutable, they reject this change by shifting back to a previous record of the blockchain where all the blocks are still chained together. The only way that an alteration can stay undetected, is if all the blocks stay chained together. This means for the alteration to go undetected, the new signature of block 1 must replace the old one in the data of block 2. But if the data of block 2 changes, this will cause block 2 to have a different signature as well. Let’s say the signature of block 2 is now ‘PP4’ instead of 9BZ. Now block 2 and 3 are no longer chained together!

The blocks on a blockchain are publicly available to anyone. So, if an alteration is supposed to stay undetected on a blockchain, all the blocks need to stay properly chained together (otherwise people can tell that certain blocks don’t properly link to each other). This means that altering a single block requires a new signature for every other block that comes after it all the way to the end of the chain. This is considered to be impossible. In order to understand why, you will need to understand how the signatures are created.

Step 3 — How the signature (hash) is created

So, let’s picture a block again; block 1. Block 1 is a record of only one transaction. Thomas sends 100 Bitcoin to David. This specific string of data now requires a signature. In blockchain, this signature is created by a cryptographic hash function. A cryptographic hash function is a very complicated formula that takes any string of input and turns it into a unique 64-digit string of output. You could for example insert the word ‘Jinglebells’ into this hash function, and you will see that the output for this specific string of data is:

761A7DD9CAFE34C7CDE6C1270E17F773025A61E511A56F700D415F0D3E199868

If a single digit of the input changes, including a space, changing a capital letter or adding a period for example, the output will be totally different. If you add a period to this word and make it ‘Jinglebells.’ instead, you will see that the output for this specific string of data is:

B9B324E2F987CDE8819C051327966DD4071ED72D998E0019981040958FEC291B

If we now remove the period again, we will get the same output as before:

761A7DD9CAFE34C7CDE6C1270E17F773025A61E511A56F700D415F0D3E199868

A cryptographic hash function always gives the same output for the same input, but always a different output for different input. This cryptographic hash function is used by the Bitcoin blockchain to give the blocks their signatures. The input of the cryptographic hash function in this case is the data in the block, and the output is the signature that relates to that. Let’s have a look at block 1 again. Thomas sends 100 Bitcoin to David.

Now imagine that the string of data from this block looks like this:

Block 1 Thomas -100 David +100

If this string of data is inserted in the hashing algorithm, the output (signature) will be this:

BAB5924FC47BBA57F4615230DDBC5675A81AB29E2E0FF85D0C0AD1C1ACA05BFF

This signature is now added to the data of block 2. Let’s say that David now transfers 100 Bitcoin to Jimi. The blockchain now looks like this:

The string of data of block 2 now looks like:

Block 2 David -100 Jimi +100 BAB5924FC47BBA57F4615230DDBC5675A81AB29E2E0FF85D0C0AD1C1ACA05BFF

If this string of data is inserted in the hashing algorithm, the output (signature) will be this:

25D8BE2650D7BC095D3712B14136608E096F060E32CEC7322D22E82EA526A3E5

And so, this is the signature of block 2. The cryptographic hash function is used to create the digital signature for each unique block. There is a large variety of hash functions, but the hashing function that is used by the Bitcoin blockchain is the SHA-256 hashing algorithm. But how do the signatures stop someone from simply inserting a new signature for each block after altering one (a change goes undetected if all blocks are properly linked, people won’t be able to tell there was a change)? The answer is that only hashes (signatures) that meet certain requirements are accepted on the blockchain. This is the mining process and is explained in step 4.

Step 4 — When does the signature qualify, and who signs a block?

A signature doesn’t always qualify. A block will only be accepted on the blockchain if its digital signature starts with a consecutive number of zeroes. For example; only blocks with a signature starting with at least ten consecutive zeroes qualify to be added to the blockchain. However, as explained in chapter 3, every string of data has only one unique hash bound to it. What if the signature (hash) of a block doesn’t start with ten zeroes? Well, in order to give the block a signature that meets the requirements, the string of data of a block needs to be changed repeatedly until a specific string of data is found that leads to a signature starting with ten zeroes. Because the transaction data and metadata (block number, timestamp, et cetera) need to stay the way they are, a small specific piece of data is added to every block that has no purpose except for being changed repeatedly in order to find an eligible signature. This piece of data is called the nonce of a block. The nonce is completely random and could literally form any set of digits, ranging from spaces to question marks to numbers, periods, capital letters and other digits. To summarize, a block now contains; 1) transaction data, 2) the signature of the previous block, and 3) a nonce. The process of repeatedly changing the nonce to find an eligible signature is called mining and is what miners do. Miners spend electricity in the form of computational power in order to constantly try different nonces. The more computational power they have, the faster they can insert random nonces and the more likely they are to find an eligible signature faster. It is a form of trial and error. You can picture it like this:

Any user on a blockchain network can participate in this process by downloading and starting the according mining software for that specific blockchain. When a user does this, they will simply put their computational power to work in order to try to solve the nonce for a block. Here is an example of a block of transactions that was recently added to the Bitcoin blockchain, block 521,477:

As you can see, the hash (signature) of this block and the hash of the previous block both start with a number of zeroes. Finding a hash like that is not easy, it requires a lot of computational power and time, or a lot of luck. Yes, it sometimes occurs that a miner gets incredibly lucky and finds a matching signature with very little computational power in a matter of minutes. An extremely rare example recently occurred on block 523,034. A very small miner with only very little computational power found an eligible signature real fast, while all other miners combined had 7 trillion times as much computational power. In comparison, the chances of winning the powerball lottery jackpot are one in 292 million, meaning it is 24,000 times easier to win the powerball lottery jackpot than it was for this miner to win the competition versus the rest of the network. Talk about a number of zeroes. Anyway, important to understand from this chapter is that finding an eligible signature is hard.

Step 5 — How does this make the blockchain immutable?

As discussed previously in step 3, altering a block will unchain it from the subsequent blocks. In order for an altered block to be accepted by the rest of the network, it needs to be chained to the subsequent blocks again. See where this is going? It was previously explained that this requires every block that comes after it to get a new signature. Giving all of these blocks a new signature will be very costly and time-consuming but doesn’t seem impossible. It is considered impossible though, and here is the reason why:

Let’s say a corrupt miner has altered a block of transactions and is now trying to calculate new signatures for the subsequent blocks in order to have the rest of the network accept his change. The problem for him is, the rest of the network is also calculating new signatures for new blocks. The corrupt miner will have to calculate new signatures for these blocks too as they are being added to the end of the chain. After all, he needs to keep all of the blockslinked, including the new ones constantly being added. Unless the miner has more computational power than the rest of the network combined, he will never catch up with the rest of the network finding signatures.

Millions of users are mining on the Bitcoin blockchain, and therefore it can be assumed that a single bad actor or entity on the network will never have more computational power than the rest of the network combined, meaning the network will never accept any changes on the blockchain, making the blockchain immutable. Once data has been added to the blockchain, it can never be changed again. There is an exception though. What if a bad actor hasmore computational power than the rest of the network combined? Theoretically yes, this is possible. It is called a 51% attack (I wrote an article about it here) and has occurred on various blockchains in the past. In reality though, a 51% attack on the Bitcoin blockchain would be far more costly to execute than it would yield in return. It would not just require an immense amount of hardware, cooling equipment and storage space for the computational power, but also involves the risk of prosecution and, more importantly, would dramatically harm the ecosystem of the according blockchain itself, rendering the potential returns in Bitcoin to drop significantly in value. Attempting a 51% attack is practically trying to fight all the other users on a blockchain just by yourself. This is also the reason that the more users participate in the mining process, the more secure a blockchain becomes. Congratulations if you are still here, you now (hopefully) understand why a large blockchain is considered immutable. An important question arises now though. How are miners stopped from adding corrupt data to the blockchain (like falsified transaction data)? That is simply impossible, more details about transactions on the blockchain in this article here.

Step 6 — How is the blockchain governed? Who determines the rules?

The Bitcoin blockchain follows a governance model of democracy, and therefore updates its’ record of transactions (and thus the Bitcoin balances) according to what the majority of its’ users say is the truth. The blockchain protocol does this automatically by always following the record of the longest blockchainthat it has, because it assumes that this chain is represented by the majority. After all, it requires the majority of the computational power to create the longest version of the blockchain. This is also how an altered block is automatically rejected by the majority of the network. The majority of the network rejects an altered block automatically because it is no longer chained to the longest chain.

On the Bitcoin blockchain, all transaction history and wallet balances are public (blockchain.info). Anyone can look up any wallet or transaction that has ever occurred all the way back to the first transaction that was ever made (on January 3rd, 2009). Although wallet balances can be checked by anyone publicly, the owners of those wallets remain largely unknown. Here is an example of a wallet still containing 69,000 Bitcoin, at the time of writing worth about roughly 500 million dollars. It was last used in April 2015, only to show no activity ever since.

Final step, step 7 — Where does this leave cryptocurrencies?

Cryptocurrencies are basically an altered form of Bitcoin. Most cryptocurrencies are built upon their own blockchain protocol that may have different rules from the Bitcoin blockchain. Bitcoin is supposed to be a currency, meaning it is explicitly supposed to function as money. Monero is a cryptocurrency with the same function, but its blockchain protocol has implemented some extra rules that make it a more private currency (transactions are much harder to trace). Cryptocurrencies can however be given any kind of value, depending on their issuer. They could be referred to as ‘tokens’. These tokens can give the owners the right to ‘something’, varying from a gaming license or access to social media to downright electricity or water, you name it. Any sort of value can be attached to a ‘cryptocurrency’ token. All these cryptocurrency transactions are registered on various blockchains and can be exchanged online through cryptocurrency exchanges such as Binance. It is the new money of the internet. A good example of an industry that might be disrupted soon is the stock market industry. There is a good chance that company shares and other property rights will be registered as tokens on a blockchain in the near future. But blockchains are not just limited to registering materialistic value in the form of tokens. Blockchains also have the potential to safely register data in the form of medical records, identities, history records, tax records and much, much more. This is why the technology is so huge, and I haven’t even mentioned decentralization yet (another huge aspect of blockchain)!

Was this article helpful? Help others find it by applauding or sharing. You can read any of my other short blockchain articles of:

Beginner 1: How blockchain works in 7 steps

Beginner 2: How mining works and how transactions are processed

Beginner 3: How a hacker performs a 51% attack

Beginner 4: Nodes and masternodes

Beginner 5: Mining difficulty and block time

Attribution– This article was originally published on our Coinmonks publication by Jimi S.


Building A Simple Blockchain Data Structure With Python

Photo by Mateusz Dach from Pexels

Here, I am going to build a simple blockchain data structure which is the foundation of Bitcoin. This data structure only is not enough to build even a simple cryptocurrency. But we have to start somewhere.

Before building a blockchain data structure, I have to explain about hashing. Bitcoin uses SHA-256. Here how you can do it in Python:

>>> import hashlib
>>> hashlib.sha256(b”hello world”).hexdigest()
‘b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9’

That is how you do hashing (SHA-256) in Python. But what is actually hash? What does 256 in SHA-256 means actually?

Hashing is a process which you turn anything (as long as you can represent it as a string) into a fixed 256 bit string. In the previous example, the string “hello world” has a length 11. Actually the length of “hello world” is depended on how you count it. But for simplicity, we just count how many characters. That “hello world” string is turned into a fixed-size string, which is ‘b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9’. Say I hash another string with different length.

>>> import hashlib
>>> hashlib.sha256(b”I am the best president. Ever.”).hexdigest()
‘fba0e4cc233f9df81278130ed748a31a842bce8f911766b661f8a4f7ddff5341’

The string “I am the best president. Ever.” has a different length than “hello world”. But the output has the same length, which is about 64 characters. So any input will be turned into 64 random characters string. Even a string which has a 23 kilometers length will be turned into a 64 random characters string.

This is a hexadecimal string. That’s why it has 64 characters. If you turn it into a bit string, it will have a 256 characters length.

>>> bin(0xb94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9)
‘0b1011100101001101001001111011100110010011010011010011111000001000101001010010111001010010110101111101101001111101101010111111101011000100100001001110111111100011011110100101001110000000111011101001000010001000111101111010110011100010111011111100110111101001’

That is why it is called SHA-256.

Now, there are some properties from hashing which is very important for Bitcoin. The first is called collision free. I mean if you are going to turn anything into a fixed 256 bit string, surely there will be more than 1 input that has the same output. The size of possible inputs is bigger than the size of possible output. Yes, that’s correct. But finding x and y where x is different than y and hash(x) is equal hash(y) is extremely hard (in SHA-256 case; some smart people have found collision in SHA-1).

So there is no apparent relation between the input and the output. Even you change the tiny bit of the input, the output will be totally different. This is the second property.

>>> hashlib.sha256(b”1").hexdigest()
‘6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b’
>>> hashlib.sha256(b”2").hexdigest()
‘d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35’
>>> hashlib.sha256(b”3").hexdigest()
‘4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce’
>>> hashlib.sha256(b”11").hexdigest()
‘4fc82b26aecb47d2868c4efbe3581732a3e7cbcc6c2efb32062c08170a05eeb8’

So the only way to to find different inputs which have the same output, you need to test all combination of characters with different length. “abc”, “maybe a loooooong string”, “17”, etc. It’s totally impractical.

But is there any possibility that when you hash something, the output is already same as hashing of “hello world”? Or just any two different strings but have the same hash? Of course, there is. The question is how minuscule the probability is. There are around 2²⁵⁶ possibilities of the output of SHA-256. How big is 2²⁵⁶? 115792089237316195423570985008687907853269984665640564039457584007913129639936. Or 1.15 e+77. Or roughly 10⁷⁷. If you think that number is big but not very big, I have a bad news for you. The total atom in observable universe (that is the universe that you can see up to 46 billions light years in any direction from your chair) is 10⁷⁸ to 10⁸². https://www.universetoday.com/36302/atoms-in-the-universe/

My computer has Nvidia Geforce 1080 Ti. It has 11.3 teraflops (tera = 10¹²). Flop is floating operation. Hashing is integer operation. So it’s apple to orange. But for simplicity, say hashing is an floating operation as well and requires 3000 operations per hash. So my graphic card can compute 3766666666 hash per second. To find a collision, described in birthday attack, we need only to compute 2¹²⁸ hashes. Say every human on this planet has my graphic card and together we compute the collision attack. It takes:

>>> 2**128 / (7000000000 * 3766666666.6666665)
1.2905778770705631e+19

That number is longer than the age of universe (around 10¹⁷ seconds). https://www.space.com/24054-how-old-is-the-universe.html

To describe the hashing algorithm, it is quite a work. But someday I’ll explain the code behind SHA-256. Now that you can comprehend the vastness of hashing, let’s move on. Blockchain is like a linked list, a data structure known by many computer science students. Let’s create a block. The first block in Bitcoin is called genesis block.

import hashlib, json
block_genesis = {
‘prev_hash’: None,
‘transactions’: [1, 3, 4, 2]
}

The transactions represents the… well, transactions. In Bitcoin, it will be like “Jason pays 2 btc to Mary Sue”, “Kylo Rein pays 10 btc to Yoda”. For simplicity, we just put normal integers.

We serialized the block so it can be hashed.

block_genesis_serialized = json.dumps(block_genesis, sort_keys=True).encode(‘utf-8’)
block_genesis_hash = hashlib.sha256(block_genesis_serialized).hexdigest()

Now we have another block.

block_2 = {
‘prev_hash’: block_genesis_hash,
‘transactions’: [3, 3, 3, 8, 7, 12]
}

We hash the block 2.

block_2_serialized = json.dumps(block_2, sort_keys=True).encode(‘utf-8’)
block_2_hash = hashlib.sha256(block_2_serialized).hexdigest()

We build another block.

block_3 = {
‘prev_hash’: block_2_hash,
‘transactions’: [3, 4, 4, 8, 34]
}

We hash the block 3. This will be the last block, I promise.

block_3_serialized = json.dumps(block_3, sort_keys=True).encode(‘utf-8’)
block_3_hash = hashlib.sha256(block_3_serialized).hexdigest()

To make sure that data has not been tampered, I only need to check the last block’s hash, instead of checking all the data from genesis block to the last block. If it is different, than someone tried to tamper the data.

import hashlib, json
block_genesis = {
‘prev_hash’: None,
‘transactions’: [1, 3, 4, 2]
}
block_2 = {
‘prev_hash’: None,
‘transactions’: [3, 3, 3, 8, 7, 12]
}
block_3 = {
‘prev_hash’: None,
‘transactions’: [3, 4, 4, 8, 34]
}
def hash_blocks(blocks):
prev_hash = None
for block in blocks:
block[‘prev_hash’] = prev_hash
block_serialized = json.dumps(block, sort_keys=True).encode(‘utf-8’)
block_hash = hashlib.sha256(block_serialized).hexdigest()
prev_hash = block_hash
 return prev_hash
print(“Original hash”)
print(hash_blocks([block_genesis, block_2, block_3]))
print(“Tampering the data”)
block_genesis[‘transactions’][0] = 3
print(“After being tampered”)
print(hash_blocks([block_genesis, block_2, block_3]))

The result:

Original hash
45eda4f7a76bf0f92a0acda2ce4752dfbe167473376f766f22d7ec68501cac40
Tampering the data
After being tampered
27d68dae05428be6aa244869196a481f431fca6645dd33c3df7a740afa03b7d9

This is the basic of the blockchain. But this is not enough. How do you decide the next block to be added? You need consensus and proof of work. Then the block structure in Blockchain is much more complicated. We’ll cover that in next articles.

If you like this article, you can fund my campaign on creating Cryptocurrencies and Blockchain Course For Hackers.

Attribution– This article was original published on Coinmonks publication written by arjuna sky kok