Machine learning is a field of science that studies how to extract knowledge from observed data. Machine learning (ML) is a relatively new discipline, and its terminology is not yet established.

Machine Learning is a field which emerged as an intersection of three other fields: statistics, computer science and optimization.

Machine Learning is a tool to make the past data useful to solve today and future tasks.

The field of machine learning is very broad. There are different models for a single task and for the different tasks.

Models can be classified in several different ways.
One way to classify machine learning models is inherited from statistics.
- There is a frequentist and Bayesian approach to machine learning.
- Models can be divided into probabilistic and deterministic.
There are pros and cons of different models choosing for the practical situations.

The task is to find and develop the most suitable model.

Useful:
- Randomly-weighted random networks which are also called Extreme Learning Machines.
- Temporal Gaussian processes and state-space models.
The time dimensionality of the data gives benefits for tasks that could be solved in other ways.

Models are used as blocks for complex tasks solving.
These tasks are :
- Spatio-temporal models.
- System identification.
- Reinforcement learning.
The improvement of models takes time and needs data.

Two different ways to approach the task solving:
1. Regression.
2. Imputation of missing values.
Building of optimal ensemble of approaches is the way to:
1. Develop approaches.
2. Improve the correlations between parameters.

Model selection is one of the central topics in machine learning.
The marginal likelihood (or evidence) optimization can be used for probabilistic models, while cross-validation or bootstrap procedure are usually exploited for non-probabilistic models.

The data modeling may be started from the probabilistic model. This model describes the relationships between variables in the data and may include unobserved latent variables. The probabilistic model is a compulsory component in the Bayesian approach to statistics, however, it is NOT MUST in frequentist approach.

There are supervised and unsupervised learning.

Supervised learning.
It consist of input variables and output variables. The task is to learn the dependency between inputs and outputs which predicts the corresponding output for any new input observation.

Two most typical cases in supervised learning are classification and regression.
The measurements in the dataset can be noisy. The methods which learn the dependency may take this into account. Probabilistic model is one way to incorporate this noisiness into the mapping between inputs and outputs.

Unsupervised learning.
In the unsupervised learning the dataset consist only of one set of variables. The purpose of this task is to find some hidden structure or pattern in the data.
In clustering the aim is to map each data point to some cluster. The clustering problem is similar to the classification, except that clustering is completely unsupervised.
In the probabilistic clustering the values of interest are probabilities. In a different approach the inputs are mapped to a latent space. The most known latent variable model is Principal Component Analysis (PCA). The computationally efficient and stable way to compute PCA is via Singular Value Decomposition (SVD).

Missing values problem.
The missing value problem is like some intermediate task between supervised and unsupervised tasks.
1. We estimate missing values which is more similar to the supervised task.
2. We devise an algorithm which is able to perform latent variable model with missing data.
Then the missing data can be filled by projecting back from the latent space.

Time series prediction.
Time series (TS) prediction and modeling. Time series is a sequential data where the order of samples plays an important role. A general type of time series models consists of two type of variables. The first component models the latent/non-observed process. The second component models the observations of the time series and is linked with the first component. Noise terms allow taking into account the uncertainty. Uncertainty is important for time series prediction because it improves the decision making for the future observations.

It is possible to select several different models for predicting the output for given data.

It is possible to create new models for predicting the output for given data.

What we need to do is to check the results of predicting. Getting closer and closer to obtaining the best result of the model calculation is the task of developing a model for a selected data stream.

Processing big data in multi-stage model calculations is a resource-intensive operation. It requires powerful computers, significant time investment, and convenient storage.

Therefore, resource and performance costs must be taken into account. Performance requirements influence the choice of model and may also motivate the choice of variables.

However, the predictive power of a combination of models is better than the predictive power of individual models. If the focus is limited only to forecasting accuracy, then it is preferable to consider a combination of models.

Model selection is directly related to the complexity of the model. The best performing model has the optimal complexity. A model that is too complex leads to overfitting. Conversely, a model that is too small leads to underfitting.

The choice of the class of methods to be used is determined by many criteria, including ease of optimization, speed of training and prediction, and ease of uncertainty assessment.