ShipIt Day 4: SaltStack, Front-end Exploration, and Django Core

Last week everyone at Caktus stepped away from client work for a day and a half to focus on learning and experimenting. This was our fourth ShipIt day at Caktus, our first being almost exactly a year ago. Each time we all learn a ton, not only by diving head first into something new, but also by hearing the experiences of everyone else on the team.

DevOps: Provisioning with SaltStack & LXC+Vagrant

We have found SaltStack to be useful in provisioning servers. It is a Python based tool for spinning up and configuring servers with all of the services that are needed to run your web application. This work is a natural progression from the previous work that we have done at Caktus in deploying code in a consistent way to servers. SaltStack really shines with larger more complicated systems designed for high availability (HA) and scaling where each service runs on its own server. Salt will make sure that the setup is reproducible.

This is often important while testing ideas for HA and scaling. The typical cycle looks like:

• Work on states in Salt

• Run Salt through Fabric building the whole system, locally through vagrant, on a cloud provider, or on physical hardware.

• Pound on the system using benchmarking tools narrowing in on bottlenecks and single points of failure.

• Start fixing the problems you uncovered in your states starting the cycle over again.

Victor, Vinod, David, and Dan all worked on learning more about SaltStack through scratching different itches they’ve felt while working on client projects. Some of the particular issues folks looked at included understanding the differences between running Salt with and without a master, how to keep passwords safe on the server while sharing them internally on the development team, and setting up Halite, a new web interface for Salt.

In order to test these complicated server configurations, we often rely on Vagrant to run these full system configurations on developer’s laptops. This is the quickest way to start building out systems. The problem with this though is that our laptops are not as fast as the hardware that the services will eventually be provisioned on. In order to reduce the time waiting in the code-rebuild-test cycle, Scott our friendly system administrator delved into running Vagrant with LXC containers on the backend with our development laptops. LXC containers are more lightweight than VirtualBox virtual machines and can be created more quickly on the fly. This involved learning about how to upgrade the development laptop’s kernels on Ubuntu long term support image. Scott was successful and there was a response of “Oooh” and “Ahhh” from the developers when he demoed the speed of creating a new VM with LXC through vagrant.

Front-end Web + Visualizations: Realtime Web, Cellular Automata, Javascript

MVC, & SVG Splines

Caktus has a growing team of front-end developers and folks interested in user interaction. There were a number of projects this ShipIt day that explored different tools for designing visualizations and user experiences.

Mark dove into WebRTC building a new project, rtc- demo with the actual demo hosted on Github static pages (note: it only works on recent Firefox builds so far). It’s neat that this is hosted on a static web assets host since the project does allow users to interact with one another. Red & Blue users go to the static site and create links that they share with one another and connect directly through their browsers without any server in between. Both users see a tic-tac-toe board and can alternate taking turns. The moves are relayed directly to their opponent. This exploration allowed Mark to evaluate some new technologies and their support in different browsers. He was able to play around with new modes of interaction on the web where the traditional server-client model can be challenged and hybrid peer- to-peer apps can be built.

Caleb has been taking a class on programming in C outside of work and wanted to continue to work on one of his personal projects during the ShipIt day. There’s something about pointer arithmetic and syntax that Caleb finds fascinating Caleb extended some of the features of his cellular automata simulation engine, gameoflife. This included some of his rule files, initial states, and rule file engine. The result was experiments with the Wireworld simulation including a working XOR gate using simple cellular automata rules and Brian’s Brain which produces a visually interesting chaotic simulation. Caleb’s demo was a crowd pleaser with everyone enthralled by the puffers, spaceships, and gliders moving around and interacting on the screen.

Rebecca made some amazing usability improvements to our own django- timepiece. Timepiece is part of the suite of tools that we use internally for managing contracts, hours, timesheets, and schedules. Rebecca focused on some of the timesheet related workflows including verifying timesheets. She made these processes more fluid by building an API and making calls to it in using Backbone. This allowed users to do things like delete or edit entries on their timesheet before verifying it without leaving the page.

Wray was also decided to work on a front-end project. His project focused on building SVGs on the fly in raw Javascript without any supporting libraries. Wray started by showing us his passion for how to mathematically define curves and how useful the different definitions are from a user experience point of view as designer working in a vector drawing program. The easier it is for the designer to get from what they meant to draw to what is represented on the screen the better the resulting design will be. He showed a particular interest in the Spiro curve definition. Wray dug into this further by looking at the definition of curves supported by the SVG standard and built an interactive tool for drawing curves by editing the string defining the SVG element on the fly. The resulting project is still experimental at this point, but is an interesting exploration into what can be done with SVGs in Javascript without any supporting libraries.

Django Core

Karen and Tobias waded into some internals of bleeding edge of Django. In particular, Tobias offered up comments on a pull request related to ticket #21271. Tobias gave feedback to the original pull request author and Django Core committer Tim Graham on the difference between instance and class variables and when it’s appropriate to use class variables in Python.

Karen gave an illuminating talk on transitioning a particular client project to Django 1.6 from Django 1.5. Her slides are available to view online. She generalized the testing, debugging, and eventually fixing strategies she used while discussing the particular problems she encountered on the project. The strategy Karen used was running manage.py check to check for new settings that may have changed. This is a new management comment added in 1.6 and can be used from now on to ease upgrading versions.

She found the following issues when upgrading:

• UNUSABLE_PASSWORD was part of an internal API that changed. We used this internal in the project as sometimes you must, but special care must be taken when upgrading code that relies on unstable APIs.

• MAXIMUM_PASSWORD_LENGTH was used on this project after the recent Django security fix. Based on subsequent security updates, this is no longer needed and can be removed in 1.6.

• BooleanFields now do not automatically default to False. This was changed to encourage coder to be explicit and to increase the level of standards within the code.

• Django debug toolbar broke in 1.6! This is a shame, but will hopefully be updated by the time 1.6 comes out or soon after.

After this, Karen ran the project’s test suite and discovered the following remaining changes:

• The default session serializer used to be picked. Now it’s JSON.
• BoundField.label_tag now includes the form’s label_suffix.

Karen urged us to check out the Django 1.6 release notes and remember where you’re using unsupported internals and to check and write tests for that code particularly carefully. Also, she encouraged everyone to run python -Wall manage.py test to help expose more deprecation warnings and bugs.

Estimation Process Review

Ben and Daryl, our Project Managers at Caktus, worked up a full proposal and project schedule for an internal process of reworking our estimation process. We don’t shy away from large fixed bid projects and pride ourselves on meticulous estimates informed by careful client communication and requirements gathering. Our PMs wanted to help us look at this process and make it more formal as we grow to a larger group with lots of people leading the same process for different incoming projects.

Wrap Up

We had a great time with our latest ShipIt day. Each time we learn a ton from each other and by digging into some tough technical problems. We also learn a little bit more about what makes a good ShipIt day project. There were a number of crowd pleasers in this batch of projects and while some folks decided to try a completely new library or technology others decided to stay a bit closer to home by extending learning they did not have time look into more during a client project. We had a lot of fun and are looking forward to our next ShipIt day.