Artificial neural networks (ANNs) have proven to be extremely useful for solving problems such as classification, regression, function estimation and dimensionality reduction. However, it turns out that different neural network architectures are able to achieve higher performances for certain problems. This article will provide an overview of the most common neural network architectures — including recurrent neural networks and convolutional neural — and how they can be implemented to aid blockchain technology.
Convolutional Neural Networks
Convolutional neural networks (CNNs) are a type of neural network that is designed to capture increasingly more complex features within its input data. To do this, CNNs are constructed from a sequence of layers, each of which consists of a series of cube-shaped filters. The most common layers that are used within CNNs are the convolutional layer, the max-pooling layer and the fully connected layer.
• Convolutional Layer: The convolutional layer consists of a set of cube-shaped filters that are convolved with the input data by computing the dot product between both, resulting in a so-called convolved feature map. The objective of a convolutional layer is to extract features from the input data, and usually, multiple convolutional layers are used within the same network to allow it to learn increasingly more complex features as the data is propagated.
• Pooling Layers: Pooling layers are periodically inserted in convolutional networks and are responsible for reducing the spatial size of the convolved features. The main reason for doing this is to decrease the computational power required to process the data by means of dimensionality reduction. In general, two types of pooling layers are commonly used: max pooling layers and average pooling layers.
• Fully Connected Layers: Fully connected layers are added to the end of the network and take as input the flattened vector representation of the convolutional and pooling layers. In essence, the fully connected layers represent a regular fully connected neural network that is trained with the purpose of classifying the input data. The output of the fully connected layer is a one-dimensional vector that represents the probabilities of the input data belonging to a certain class.
Recurrent Neural Networks
Many machine learning problems require the analysis of data in which a relation between individual training instances can be observed. This is the case when dealing with the so-called sequencing problem: a problem where the data that is fed into the network represents a sequence — commonly referred to as a “time-series” — of data points.
Many practical examples can be found that represent such data sequence, including day-to-day air temperatures, the daily closing price of an individual stock or the sequence of words that is represented by a sentence. From these examples, it is clear to see that valuable information may be hidden in the order that instances are represented to the network. To capture this information, more complex networks are needed, which encouraged the invention of the type of neural network that is commonly used today for the purpose of dealing with sequential data: recurrent neural networks (RNNs).
Conceptionally, RNNs do this by introducing feedback loops into the network’s architecture, enabling them to use information from previous calculations in order to determine a new output. This property gives recurrent neural networks a memory-like capability which allows them to look back a few steps when dealing with sequential data.
Long Short-Term Memory Networks
Instead of using regular neurons like RNNs do, long short-term memory networks (LSTMs) make use of memory units. These units are composed out of three gates: a forget gate, an output gate and an input gate. By regulating these gates, the network is capable of remembering specific values for a certain amount of time. In addition, since long short-term memory networks are able to reduce the vanishing gradient problem, they can be used in a deep recurrent network architecture. This makes them especially well suited to being used for complex sequencing tasks such as time-series prediction, speech recognition and semantic parsing.
Artificial Intelligence For The Blockchain Industry
Distributed storage systems like the blockchain technology require consensus protocols to decide which node will add newly issued blocks to the blockchain and thereby provide the network with the most recent transactional information. Currently, different protocols exist for reaching consensus in a distributed system, with the most prominent ones being the proof-of-work (PoW) and proof-of-stake (PoS) protocols. However, these protocols are flawed in that they consume a lot of energy (PoW) or tend to favor nodes with large coins stakes, potentially monopolizing its blockchain (PoS).
Artificial intelligence — and especially neural networks — poses a solution to these problems by using them in a new, energy-saving protocol called proof of artificial intelligence (PoAI). Proposed by J. Chen et al. (2018), the proof of artificial intelligence protocol uses a pretrained CNN to determine the average transaction number (AVN) of every node and, using the outcome of this calculation and node characteristics, to determine the mining node from the node pool. By using this protocol, the consensus mechanism ensures a fair node selection, keeps the blockchain decentralized, and reduces the problems with energy waste and mining conflicts.
The Future Of Blockchain Application
The introduction of artificial intelligence, RNNs and especially LSTMs has enabled complex time-series forecasting, which is the sector of machine learning focused on predicting parameters in the future by referencing parameters from the past.
Using data on bitcoin’s (or any cryptocurrency, for that matter) previous price points, RNNs can be trained in order to estimate its future price. This enables players in the retail industry to account for future price increases/decreases, possibly facilitating the transition to the implementation of digital currencies.
It’s important for technology professionals to learn as much as they can about the future of AI and neural networks in order to stay ahead of the curve. There are many great resources that can help you with this, including blogs such as Learn Neural Networks and videos from GoogleTechTalks and Geoffrey E. Hinton. Take a look around the web, and get invested in the future — it will behoove you in more ways than you know.