Machine learning concepts. Data preparation
Each machine learning process related to the use of neural networks consists of at least two parts. The first part is related to data loading and preparation for training. The action is known as ETL (extract, transform, load). The second part concerns the actual training of the network. The overall process can be divided into the following parts and steps.
1. Extract, Transform, Load
1. Selection of input variables according to the problem being solved;
2. Structuring the data in a format suitable for loading it in the software environment;
3. Loading data into the environment;
4. Conversion of the data into an appropriate form;
5. Data separation – training and test data;
6. Data normalization.
2. Network training and evaluation
1. Building a network model according to the problem being solved;
2. Setting up network hyperparameters;
3. Network training;
4. Network testing;
5. Comparing the results;
6. Tuning the hyperparameters in order to achieve more satisfactory results;
7. Persisting the trained model. Exporting the model for further use and loading in other environments.
2.1. Extract, Transform, Load
Selection of input variables according to the problem being solved.
- price_close
- news_sentiment
- twitter_sentiment
- reddit_sentiment
2.2. Structuring the data in a format suitable for loading it in the software environment
The input data is structured in a four-column text file, each of which contains values for the corresponding parameter. The text file has the following structure.
3. Loading data into the environment
4. Conversion of the data into an appropriate form
5. Data separation - training and test data
6. Data normalization
Letzte Beiträge
Share :
Share :
Weitere Beiträge
Designing REST API architecture
The web layer is the top layer in the architecture. It is responsible for processing user queries and returning back responses to the user. These operations happen in the controllers.
6 Fragen an unseren Geschäftsführer Dimitar
Heute möchten wir euch Dimitar vorstellen, einen unserer zwei Geschäftsführer bei der DiSC. Er ist Spezialist für Mobile Netzwerke, Java Backend, Java Enterprise Edition sowie für Java Spring Boot
6 Fragen an unseren Senior Consultant Dominique
Heute möchten wir euch unseren IT-Berater und Entwickler Dominique vorstellen. Er ist seit November 2020 Teil unseres Teams und Spezialist in Java, Spring, Microsoft und Enterprise.