BLEK : beats, logstash, elastic search, kibana

• The usual log management stack : more inclined towards a reactive analysis.
• for a more real time (and predictive) analysis, checkout the Monitoring node
• checkout Opensearch (& Dashboard), BLEK isn't open source anymore
  • has the premium features of BLEK and is open source (Apache 2.0 OSL)

• Beats: Lightweight data shippers on edge devices forward data to Logstash
• Logstash: ingests, transforms, and ships data from various sources into Elasticsearch
• Elasticsearch: A distributed, RESTful search and analytics engine.
• Kibana: A visualization and exploration tool

    [Web Server]:::B(Apache Logs)===O
                                     \
 [Load Balancer]:::::B(NGINX Logs)====O
                                       \
 [Application Server]::B(App Logs)======O-->[Logstash]-->[Elasticsearch]-->[Kibana]
                                       /
 [Database]::B(MySQL Logs)============O

B = Beats

• Various sources (web server, app server, etc.) generate logs.
• Beats installed on edge devices push these logs to logstash
• Logstash collects, processes, and normalizes these diverse logs.
• Processed logs are sent to Elasticsearch for indexing and storage.
• Kibana queries and visualizes the indexed data from Elasticsearch.

    [Web Server]:::B(Apache Logs)===O
                                     \
 [Load Balancer]:::::B(NGINX Logs)====O
                                       \   |  RMQ   |
 [Application Server]::B(App Logs)======O--| Buffer |-->[Logstash]-->[Elasticsearch]-->[Kibana]
                                       /   | AKafka |
 [Database]::B(MySQL Logs)============O

B = Beats

• check out https://logz.io/learn/complete-guide-elk-stack/#installing-elk
• docker + config managment systems

• https://www.elastic.co/elastic-stack
• https://logz.io/learn/complete-guide-elk-stack/
• https://github.com/deviantony/docker-elk
• https://github.com/opensearch-project/OpenSearch