The neural network model consists of two layers – an LSTM layer and an output Dense layer. The reason for choosing an LSTM layer is the need to process sequences of time-related data and to find data correlations. To perform these operations is needed a layer with memory, such as the LSTM layer that is capable of detecting long-term dependencies. The Dense layer limits the number of output parameters to one (corresponding to the closing price), applying an activating function to the outputs of the previous layer. A linear activation function is chosen for the Dense layer so that the neural network can predict higher values than those it was trained with. This can not be achieved with a hyperbolic tangent activation function or a logical sigmoidal activation function.

The input layer of the neural network has 4 neurons – one for each of the input parameters, the hidden LSTM layer – 128 and the output Dense layer – 1, which is the number of possible outputs (remember that we are having a regression task). The number of neurons in the LSTM layer can be changed in order to achieve better results and it is part of the hyperparameters’ tuning (trial and error process).

Choosing the right hyperparameters is essential for successful network training. Poorly selected hyperparameters may result in slow or no learning at all.

Batch size is the number of examples propagated through the network. With a selected value of 100 for this hyperparameter, the algorithm divides the training data into groups of 100 records and trains the network with each of the groups. A good starting point when tuning the batch size is 32. Other common choices are 64 and 128.

Each epoch represents a complete passage of all training data through the neural network. The small number of epochs is not enough for the network to learn good values of its weight matrixes. Too many epochs lead to a problem known as overfitting, in which the neural network predicts the training data very accurately, but makes bad predictions on data it has never seen. The reason for this is that the network remembers the training data instead of learning the correlations between its items. The “Early stopping” technique can be used for selecting a suitable number of epochs in which the neural network stops learning at the point where the error function stops decreasing. Using this technique solves the overfitting problem.

The number of iterations is equal to the number of all training examples divided by the batch size value. If the data has 1000 examples and the batch size is 250, then the number of iterations is 4, ie. with 4 iterations, 1 epoch can be completed.

It is one of the most important hyperparamers. Too small or too large values for the learning rate may lead to very poor, very slow or no training. Values typically range from 0.1 to 1.10-6. 1.10-3 is a good starting point to experiment with.

The LSTM layer has a sigmoidal activating function used to control the LSTM gates because the sigmoidal function has output values ranging from 0 to 1. A linear activation function is selected for the Dense layer activation function.

The mean squared error function is used for solving this regression problem. The produced error by the neural network is measured as the arithmetic mean of the sum of the differences between the predictions and the actual observations on the degree of two. The following formula gives a better explanation of the equation.

After creating and configuring the neural network model, the training process takes place, where the network is trained on the training data. The following graph shows the process of training the neural network. The blue line represents the correct outputs for each example, and the orange one – the predictions made by the network.

The testing determines how satisfactory the predictions made by the neural network are. The network predicts data that it has not seen, and the predicted values are compared to the correct outputs. The smaller the deviations between the two values, the better predictions it makes. The following graph shows the process of testing the neural network. The blue line represents the correct outputs for each example, and the orange one – the predictions made by the network.

After comparing the two graphs it can be easily seen that the errors on the test data are greater than those on the training. This is the expected result of the comparison. By looking at the test graph, we come to the conclusion that the results of the predictions are satisfactory. The neural network has managed to predict the trend of the cryptocurrency’s market closing price. The next thing we can do is to try other network configurations or fine tune the hyperparameters of the network.

Here is a series of different combinations of hyperparametric values in order to achieve more accurate predictions. The results of the experiments are presented in the following figures. Each of the combinations is trained using the “Early stopping” method.

After the hyperparameter setting has been made, it is found that the most optimal configuration for the neural network is:

- learning rate: 1.10
^{-3} - batch size: 128
- number of neurons in the hidden layer: 512

Machine learning models are often trained in a Python environment and are used in production in another environment. The neural network for predicting cryptocurrency prices is modelled in Python and then loaded in Java environment.

To save the trained Python model, use the following expression:

The model is exported to a file named “model” with extension .h5. Apart from the architecture of the neural network, the weighting coefficients of the network as a result of its training are also recorded. In this way, it can be used directly for forecasting.

Loading the model in Java environment is done using the method importKerasSequentialModelAndWeights()of the KerasModelImport.class which is part of the DL4J machine learning library.

Digital Solutions Consulting GmbH

Ober der Kirch 3

DE-56412 Girod

Tel.: +49 (0) 179 / 4479005

management@digital-solutions.consulting

Du bist Softwareentwickler oder Consultant mit Berufserfahrung? Dann schau doch mal bei Karriere vorbei! Nichts passendes dabei? Wir freuen uns auch auf Initiativbewerbungen!

We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.

Manage consent

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.