Philly DevOps Meetup Notes, September 2013


"I LOVE Boxen!"

I attended tonight's Philly DevOps Meetup at the Comcast Center in Philadelphia, PA. As a service to the tech community, I decided to share the notes I took from the presentations.

Boxen (presented by Adam Ochonicki):

  • configuration management for Macs
  • Does things like installs Skype, VLC, etc.
  • is a wrapper for Puppet
  • Warnings:
    • Installs its own Homebrew and rbenv
    • Installs services on non-standard ports
    • YMMV on "older" hardware
  • http://boxen.github.com/

Docker (presented by Dave Konopka):

  • Docker is to Linux containers what Vagrant is to VMs
  • Linux containers (lxc): isolate proceses, files, and networking running on a host system
  • AUFS: Layered file system
  • Docker is NOT:
    • package manager
    • configuration management
    • VM
  • Requirements:
    • 3.8 kernel
    • LXC and AUFS support
  • Layered file system:
    • Base image can be shared across multiple docker containers
  • Unlike VMs, the guest OSes and kernels are not duplicated
  • Host kernel + smaller resource footprint + layered file system = insanely fast spinup
  • Can run on top of other forms of virtualization (EC2)
  • Starting a container:
    • Spawn a container from an image
    • Fire one process
  • Repositories
  • Ideal use cases:
    • Experiment with distributed applications
    • Portable stand alone applications
    • App deployment
    • Replicate test environments
    • High number of concurrent isolated "systems"
  • "production ready" due in October, 2014
  • https://www.docker.io/

Puppet (presented by Jon Mosco):

  • Why use configuration management?
    • Consistency
    • Less manual intervention
    • Smaller chance of error (fat-fingered files!)
    • Automation
    • Easily reproducible systems
  • Automation
    • Gives us the ability to repeat processes
  • Consistency!
    • Inconsistent server configuration from manually editing files:
      • /etc/resolv.conf
  • Abstraction
    • I don't want to learn 5 different ways to install 1 application
  • Supports multiple operating systems
    • Various LInux, BSD, Solaris, Windows
  • Multiple ways to start services
    • Ubuntu has upstart, Fedora has systemd, etc.
  • Multiple applications
    • Different versions of the same applications
    • Multiple packages for applications
  • So, what is Puppet?
    • Configuration tool, written in Ruby, uses its own DSL to describe system state
  • Why Puppet?
    • High Level DSL
      • Sysadmins not required to learn programming skills
    • Multiple OSes supported
    • Very active development and community
  • Resources
    • Puppet manages system configuration as a collection of "resources"
    • Resources are declared and they describe a state that is being managed
  • Declarative
    • Resource describe what PUppet should manage and abstract away the rest
    • You design the way you want your systems to look
  • Idempotent
    • Apply the same resource multiple ties wth the end result always being the same
    • "If nothing needs to be changed, don't do anything!"
  • Resource Evaluation
    • Puppet takes resources and creates a graph of dependencies so that resources are applied in the correct order
  • Types and Providers:
    • Types are the resource that determines the system component puppet manages (users, etc.)
    • Providers: The subsystem that does the work: yum, apt-get, etc.
  • Manifests
    • Use the .pp file extension
    • Describes the desired final state of the system
  • Classes
    • Named chunks of Puppet code
    • Reusable and singleton
    • Grouped together resources
  • Modules
    • Self contained collections of manifests and data
    • Reusable, shared units of Puppet code
    • Pre-built modules are available on the Puppet Forge: http://forge.puppetlabs.com/
  • Puppet agent
    • Runs as a service or out of cron every 30 minutes
    • Uses a tool called facter to provide system information to the Puppet Master
      • In Chef, the equivalent tool is "ohai"

Node.js/Grunt Infrastructure management (presented by Hunter Hutchinson):

  • This won't replace Chef or Puppet
  • It "just works". Can be used for some quick hacks.
  • Node.js is surprisingly efficient, and can be deployed in constrained environments
  • Grunt is a modern build tool for node.js:
    • Deployment
    • Unit Testing
    • Quality Control
    • Documentation Building
    • Project Templating
  • Why do DevOps care?
    - They have single page apps (management, deployment, quality control)
  • Basics: grunt.initConfig(options)
  • CSS can be built from SASS in Grunt
  • Modules are installed with npm
    • npm install jshint --save-dev
    • --save-dev adds the project to the package.json
  • Unit testing
    • Pre-built Grunt tasks exist for Qunit, Jasmine, Nodeunit, and Selenium
    • PhantomJs (a headless webkit) comes with Grunt
  • "connect" can be used to run a webserver on demand for testing purposes
  • npm install -g grunt-init
  • You can deploy to S3 or via SSH
  • Learn from my mistakes:
    • Grunt tasks are often too basic
    • Most tasks do not expose API level interfaces
    • If it looks like you'll need to roll your own, just roll your own
    • If you have comments in your JSON files, use an external JSON minifiers
    • Don't use built in option parser, use commander
      • npm install commander

Systemd (presented by Michael Dur):

  • What is systemd? A drop-in replacement for sysvinit.
  • Used by default on Fedora, OpenSUSE, Arch, and many others
  • Unit files
    • Encode information about a service, socket, device, path, target, etc.
    • They are named "name.type". Example: sshd.service
  • SysV init scripts
    • Overridden by systemd unit files
    • Uses both priority number AND LSB header for dependency information
  • /etc/fstab
    • May or may not be overridden by systemd. Depends on the distro.
  • systemctl
    • systemctl (start|stop|reload|restart) service
    • systemctl (enable|disable) service
    • systemctl status service
      • Example: systemctl sshd.service
  • Socket activation
    • Parallelization (D-Bus, syslog)
    • On-demand: singleton (CUPS)
      • You don't want to start this at bootup, but once running, keep it running - On-demand: per connection (inetd-like)
  • File locations:
    • Your units should go in /etc/systemd/system
  • Runlevel equivalents:
    • 0: powerof.target
    • 1: rescue.target
    • 2,3,4: multi-user.target
    • 5: graphical.target
    • 6: reboot.target
  • Also obsoletes: ssylog, watchdog, cron, atd, inetd
    • But you can still use each of those if you want, systemd's facilities are optional
  • The Journal:
    • systemd's replacement for syslog
    • forwards to syslog/kmsg/console
    • structured, indexed
    • can be cryptographically sealed to prevent tampering
  • Troubleshooting
    • systemctl --failed
    • systemctl status service
  • Fun stuff:
    • systemd-analyze
    • systemd-bootchart
    • 1-2 second boot with system
  • http://www.freedesktop.org/wiki/Software/systemd/

Logstash (presented by Chris Nehren):

  • "Like UNIX for your logs"
    • It's a small thing that does one thing really well
  • Takes input in various different formats, filters it, and puts it somewhere else
  • There a number of different kinds of inputs, outputs, and filters- You can take things from:
    • logfiles on your system
    • RabbitMQ
    • Twitter
  • Filtering:
    • Regular expressions
    • Substitutions
  • Output:
    • Nagios or another monitoring system
    • Postgres
    • Twitter
  • Logstash trifecta:
    • Combine with Elastic Search for indexing and Kibana
  • Examples:
    • "Who logged into this machine on this date via SSH?"
    • "Make sure an employee who was terminated is not logging in anymore"
      • A filter could be created to search for terminated employee activity, and send alerts to Nagios
  • http://logstash.net/
4.5
Average: 4.5 (2 votes)
Your rating: None