Log analysis with deep/machine learning

https://hackernoon.com/log-analytics-with-deep-learning-and-machine-learning-20a1891ff70e

Need of Deep Learning Neural Network

There are various methods that are introduced for the analysis of log file such as pattern recognition methods like K-N Algorithm, Support Vector Machine, Naive Bayes Algorithm etc. due to the presence of a large amount of log data, these traditional methods are not feasible to produce efficient results.

Deep Learning Neural Network shows excellent performance in analyzing the log data. It consists of excellent computational power and automatically extracts the features required for the solution of the problem. Deep learning is a subpart of Artificial Intelligence. It is a deeply layered learning process of the sensor areas in the brain.

Why is Deep Learning called Deep?

The traditional neural network consists of at most 2 layers and this type of structure of the neural network is not suitable for the computation of larger networks. Therefore, a neural network having more than 10 or even 100 layers are introduced.

This type of structure is meant for Deep Learning. In this, a stack of the layer of neurons is developed. The lowest layer in the stack is responsible for the collection of raw data such as images, videos, text, etc.

Each neuron of the lowest layer will store the information and pass the information further to the next layer of neurons and so on. As the information flows within the neurons of layers hidden information of the data is extracted.

So, we can conclude that as the data moves from lowest layer to highest layer (moving deep inside the neural network) more abstracted information is collected.