Tag Archives: metrics

Apache or nginx real-time log analysis

Real-time log analysis through a Dashboard

GoAccess‘ dashboard gives you an overview of server metrics by displaying summaries of different reports as panels on a single scrollable view.

 

Logstalgia is a website traffic visualization that replays or streams web-server access logs as a pong-like battle between the web server and an never ending torrent of requests.

Requests appear as colored balls (the same color as the host) which travel across the screen to arrive at the requested location. Successful requests are hit by the paddle while unsuccessful ones (eg 404 – File Not Found) are missed and pass through.

The paths of requests are summarized within the available space by identifying common path prefixes. Related paths are grouped together under headings. For instance, by default paths ending in png, gif or jpg are grouped under the heading Images. Paths that don’t match any of the specified groups are lumped together under a Miscellaneous section.

 

 

 

Logstash+ElasticSearch+Kibana

Some references for Logstash+ElasticSearch+Kibana

These log files are still hard to use on their own. They are made by a computer for a computer and I am human trying to understand other humans. Logstash, ElasticSearch and Kibana to the rescue! Logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (statement humbly borrowed fromhttp://logstash.net). ElasticSearch is a distributed restful search and analytics engine (yet again borrowed, but fromhttp://www.elasticsearch.org/overview/). Kibana is a GUI tool to visualize logs and time-stamped data in realtime (yet again borrowed, but from http://www.elasticsearch.org/overview/kibana/). These three tools make up a pretty good toolkit for creating some graphs and dashboards.

Read more at  http://nhhagen.wordpress.com/2013/11/28/query-log-analysis-using-logstash-elasticsearch-and-kibana/

 

Continue reading Logstash+ElasticSearch+Kibana