This weekend I attended the devopsdays. The conference topic was “agility in system administration”, and breaking down the wall between development and operation. I like the idea and I think this is the way forward. This is also import for me from a research perspective, because an important aspect is the use of system configuration tools. By using this tools operations also becomes development using the API provided by the tool.

Although the conference was nice and I especially liked Teyo from reductive labs talk, I was a bit disappointed. Most of the attendees where part of small companies or startups. As Teyo said in his talk, this is where a lot of these new ideas are formed and they trickle up to enterprises. The second part of the conference, after the presentations, was an openspace meeting. It was there that I got a bit disappointed. Most discussions were at a low level and did not touch the real problems that the devops concept faces.

Instead of talking about the next generation patch tools, how to migrate partially puppet manged debian boxes, how evil mailing lists are or the “next yet another” cloud management tool. These are all really interesting discussions if I was wearing my sysadmin hat, but they do not have much to do with devops (and I was wearing my researcher hat).

Devops is much easier to do when you talk about a small team, where the developer is setting next to the sysadmin. This is, like Teyo said, much the case in small companies or startups. According to me the discussions should have been about:

  1. How do you move from the current model in enterprises to the devops model?
  2. How do you make devops compatible with stuff like ITIL or COBIT? This is important for enterprises, you can not ignore this.
  3. What about security? Are you going to give everyone access to the API? How do you control access? ACHEL is a step in the right direction according to me.
  4. How far do you want to go? Will everyone be dev and ops? Or is there still some specialization? Because even in development, nobody does everything.

I know this is the whole point of open space, everyone can propose a session. I tried one about the security aspect, without talking about ACHEL. But nobody was interested. I also did not want to push this, because forcing would have been contra productive. Maybe a last question: How do you bring these kinds of questions on as a sysadmin, without looking like a researcher in his ivory tower? I have no idea …

Configuration management with puppet

I know it has been very quit here and on eAccelerator because I’ve been working really hard on my master thesis. It’s about managing large scale webserver infrastructure. This means it’s a thesis on configuration management applied to the problem of managing webservers and everything related. A lot of the work I’m doing could just as well be applied to any other computer infrastructure.

I’m a heavy open source supporter and I really hate duplicate work, so I always try to reuse existing software and extend or incorporate it. For my thesis I basically had the choice between building a layer on top or extend an existing system. I’ve studied four major open source systems that are referenced a lot in the literature: cfengine, bcfg2, lcfg and Puppet. For reasons I’m going to explain in my thesis (it’s going to be in Dutch), I chose to use puppet and extend it.

Webservers are more or less just a gateway to an application that runs in some sort of environment like WordPress in a Linux-Mysql-Php environment or a Ruby-Rails-Linux-Postgresql environment. This requires installing software, creation configuration files, … Almost all configuration management systems already do this, on Fedora I can even do this by installing the wordpress or drupal package.

However some more powerful features are required that aren’t provided or not sufficiently developed. The virtual/exported resource system in Puppet is a good start but it could be improved a lot. An other features are creation rules like proposed in the PoDIM system by my thesis counselor and my supervisor. Today I finished a first version that introduces these rules in Puppet, available in my features/collection branch of my puppet git repo (listed on the puppet wiki).

These rules allow to say “make sure at least 2 dns server configured on this group of hosts”. Puppet will make sure that if two hosts are available that match the criteria in the rule, they will be configured as a dns server. The servers can then export their ip addresses so all machines in the network can add them in their /etc/resolv.conf This is just one of the great things you could do and I’m very excited about them!