Posted on by Douglas Muth

Using Eventgen in Splunk Lab

What Is Eventgen?

Because a Docker container is quicker than spinning up a VM.

According to the docs, Eventgen is a Splunk App that lets users built real-time event “generators” so that one-off event generators don’t need to be built.

What does this mean? Let’s say you run a Splunk platform, and you want to create some new dashboards for a data source in production, but want to do this on dev. Without Eventgen, you would need to write a script to generate fake events and write them to a file which is read in by Splunk. That is a lot of work.

And we all have better things to do than write one-off code.

Why Use Eventgen?

With Eventgen, you can create a sample file with say, 1,000 events from that data source, and configure Eventgen to write a random event from that file straight to Splunk via its API, with current timestamps. The end result is that you’ll have a steady stream of realistic events flowing into Splunk with current timestamps, without the need to read from (and rotate) logfiles in the filesystem.

How Eventgen Is Used in Splunk Lab

I took approximately 1,400 lines of logs from my blog’s webserver and included them into Splunk Lab. When Eventgen is used, a random event from that file will be written into Splunk at the rate of approximately once per second. Because the events that make their way into Splunk are random, there will be a short-term fluctuation in the frequency of specific URLs, HTTP verbs, HTTP statuses, etc. This is perfect for creating dashboards that mimic what you might see in a production environment.

How To Spin Up Splunk Lab with Eventgen?

Starting up Splunk Lab with Eventgen enabled is super easier! Just run this shell command on any system with Docker installed:

SPLUNK_EVENTGEN=1 bash <(curl -Ls https://bit.ly/splunklab)

You’ll see the startup screen for Splunk Lab, with confirmation that Eventgen will be run when Splunk starts:

ASCII Art. Just like logging into your favorite BBS.

Hit Enter to confirm that you want to start Splunk Lab, and wait 15-20 seconds for Splunk to spin up. If it is your first time running Splunk Lab, go grab some coffee while the Docker images are downloaded.

Head on over to https://localhost:8000/ to log in, and run your first search for index=main sourcetype=nginx. Events should already be flowing at this point, at the rate of about 10 per second.

“Now with CIM Compliance!”

Charts and graphs can also be made against these events. Note the variation in both total event volume HTTP status codes due to the randomness introduced by Eventgen:

Still a better love story than Twilight.

How To Configure Eventgen?

Let’s say you have your own data source and want to pull its data into Splunk Lab for dashboard development. Here’s how to do that:

  • Clone the Splunk Lab repo: git clone git@github.com:dmuth/splunk-lab.git
  • Put your sample events into a file in the directory splunk-lab-app/samples/.
    • Optionally replace all IP addresses with the string %IP% if you like.
  • Edit the file splunk-lab-app/default/eventgen.conf and add a new stanza with the filename of the sample events file.
  • Copy the settings from the [nginx.sample] stanza. This will add randomness for the amount of events flowing in for a specific timestam. Tweak token.0 to a regular expression for datestamps and their replacements.
  • Keeping token.1 in is optional, if you are interested in randomizing IP addresses.

Once configuration is done, you’ll want to start Splunk Lab with a raw Docker command to pull in your changes:

docker run -d -p 8000:8000 \
    -v $(pwd)/splunk-lab-app:/opt/splunk/etc/apps/splunk-lab \
   -e SPLUNK_PASSWORD=password1 \
   -e SPLUNK_EVENTGEN=1 \
   --name splunk-lab \
   dmuth1/splunk-lab

If you want to spawn a shell on the Docker container in order to check on the status of Splunk or restart Splunk, that can be done with:

docker exec -it splunk-lab bash

Credits

I couldn’t have done this without Eventgen, which is now one of my favorite Splunk Labs. The app can be viewed on Splunkbase at https://splunkbase.splunk.com/app/1924, and there is loads of documentation available at http://splunk.github.io/eventgen/, which can be used to configure Eventgen to your specific use case.

Seriously, give Eventgen a try in your Splunk environment. It’s one of those apps which turned out to be so useful you’ll be surprised you didn’t start using it sooner.

Mirror on Medium.