Logstash… When splunk is too expensive/complex.

What is it?

Logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs. It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.

It’s really simple… Want to monitor your syslog and send it to ElasticSearch? Check the tutorial here.

You can build your log monitoring setup by using an “INPUT”…

Then adding a FILTER to the data…

And transfering it to an OUTPUT

Practical use-case…? Use logstash to monitor certain logfiles (syslog, Oracle alert file, catalina.out, etc), add a filter to limit only see the messages that are worthwhile, and send them to a message queue (RabbitMQ?). Then use logstash on your Nagios server to read that messaging queue and add it to Nagios & Elastic Search. That way you get the needed alerting in your Nagios and are able to search via Elastic Search.

2 thoughts on “Logstash… When splunk is too expensive/complex.

  1. When you collect a big amount of data, Logstash does the job but ElasticSearch is more limited. Splunk is expensive but it scales better theoretically.
    Add terabytes every day into ElasticSearch and run big search queries through Kibana and you won’t like the response times.

    1. By now, there are even other options ; Azure Search or Azure Log Analytics, that offer solutions in the same space.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.