Description
freckles is a Python framework that helps getting computational environments (physical & virtual machines, as well as containers, filesystems, even whole infrastructures ...) into a defined state, with minimal amount of hassle.
"What problem does it solve?" Well, to be perfectly honest, I always thought that was a pretty boring question, and if it has a straight-up answer then the product in question is usually boring too. Probably successfully so, but at what price!!!
If you really must know: The problem is that there is no straight-forward, easy and non-opinionated way to formalize and automate everything, from the most mundane piece of work to the most challenging and complex, in the same, single process. Even if a tool claims to do that, there is often a 'before you begin, download and install X, Y and Z' part to it. freckles solves that by providing a unified interface for every task and sub-task.
I think the solution to that problem is not all that interesting by itself. But, the things that can be built on top of it once it exists, those most definitely can be...
Main characteristics:
- freckles comes with a small (15-20mb) standalone binary which can be used in scripts or directly via its command-line interface
- every invocation of freckles creates a certain, specified state on a chosen target
- you can control that resulting state by selecting one of the list of included or BYO frecklets, and, optionally, a set of parameters
- frecklets are composable, they can incorporate other frecklets
This all might not sound like much, but that simplicity provides great flexibility. There is nothing that freckles can do that could not also be done with other tools. But because freckles does not need any setup or external requirements -- only one binary, one instruction, and optionally one set of parameters -- it is very easy to incorporate it into any type of existing or new workflow (for example in combination with Docker, Terraform, cloud-init, Kubernetes, ...).
freckles can of course also be used in shell-scripts, and from any programming language. If you ever wanted to be able to create, deploy and provision a VM (with one function call) directly from, say, Python, or JavaScript, freckles is your friend.
freckles does not depend on any service- or cloud-provider, but you should be able to use it with all of them. And, of course, locally, on its own.
The ultimate goal for freckles is to eventually become a tool that allows you to:
- describe your whole infrastructure (application and image-building, service provisioning, as well as orchestration) with a single associative array ('dictionary', 'map', ...)
- lets you set-up (or set the state of) a single machine as well as a whole infrastructure with a single command (optionally also including the bootstrapping of freckles itself)
- provide an API and client libraries so that one can easily execute any of those tasks, or parts of them, from within your own applications, services, CI-pipelines, etc.
Well. That's the very big picture anyway. I don't imagine that's where it'll end up once it's exposed to the real world, but what else is new, right?
So, does it solve my problems?
Dunno. freckles is (on purpose) a very generic tool. Some will say to a point where one could consider it plenty over-engineered.
So I guess, it really depends what sort of problems you are facing in your daily (working) life, and whether you are happy enough with how you are currently solving them. And whether you like to try new approaches (and possibly waste a bit of time while doing so).
Check out the freckles use-cases list to get an idea what freckles can be used for.
Overall, for now, I'd say: freckles can help you, if...
- ...you never quite seem to find the time or resources to apply provisioning (and other) best practices. You'll "clean everything up later"...
- ...you find yourself trawling Medium way too often, for blog-posts about how to set up some service or software that does not really fall into your core-competency as a developer. And once you set up that thing, you are never quite sure what exactly you did, and why it works. And for how long it will continue to do so...
- ...you want to be able to re-use every setup-related task you already implemented once, without having to spend too much time formalizing the task in question
- ...you want to create (or adapt) re-usable, easily maintainable and share-able deployment (and other) workflows, quickly. For public consumption, or just internally. Without sacrificing quality or flexibility
- ...you want to provide an easy to use installer script for your (non-trivial to set-up) software or service
- ...you work in a small team, or are the single sys-admin, and you want to transform your currently un-managed infrastructure into a managed environment, bit by bit
- ...you are at the beginning of transitioning to a more automated or dev-opsy workflow, but don't yet know which of the upstream technologies will be the best fit. With freckles, work you do on formalizing your recurring tasks won't be lost, as you'll be able to use it from most of those candidates. And if not, it'll be easier to do a transition, as you can use the documentation freckles auto-generates as a guide.
- ...you want to experiment with 'Infrastructure as code'
- ...you want to contribute to a comprehensive stdlib for devops
It's probably not that interesting for you, if...
- ...you already operate in a highly complex environment, with full blown container/Kubernetes/config-management deployments (freckles can still make your life a bit easier if you give it a chance)
- ...you are happy enough with Docker and you don't plan to ever switch to a different technology. And Dockerfiles are all you've ever needed in terms of creating immutable (or not) infrastructure. The problems you have are in line and a good fit with how Docker does things (one process per container, etc.).
- ...you don't really see a need for a framework for re-usable deployment scripts and similar, 'bash is fine'...
History
A little while ago I checked my savings account, figured it's enough for a year or two, and took a sabbatical of sorts. To work on a few of the ideas I had over the last few years. Without any of the sort of the restrictions and annoyances you usually encounter when working for somebody who gives you money. Just see where the code takes you, sorta.
Which is the reason I re-wrote freckles about 3 or 4 times. Which, in turn, is the reason there are not many unit tests just yet, and not all that much documentation in the code itself. In case you were wondering. All of this is on the map. Just takes some time, on account of me being the only one working on all this currently.
Alternatives
freckles has its fingers in a few different pots, so none of the below are strict 'alternatives'. They just inherit the same space. Some of them can be used from, with, or within freckles, some of them are very polished, specific implementation of something that could probably be built on top of freckles.
- Ansible
- Ansible is a special case, as it is used as a freckles backend with the freckles nsbl adapter. Which means that, obviously, instead of using the freckles nsbl adapter, you could use Ansible directly. Might take you a bit longer to set up, and to share your pipeline with others, but nothing impossible. Ansible is great, honestly. The main reason I picked Ansible as the first backend to implement is because of the myriads of modules and roles that are available for it.
- Salt, Puppet, other configuration tools
- Same as with Ansible, those can do the same kind of things: automated, deterministic state management of compute environments. If I had discovered Salt before I eventually did, I might have never written freckles...
- shell scripting
- You can do everything with a shell script. They are just notoriously hard to maintain once they reach a certain size and complexity, esp. if you want them to be flexible. Again, there is nothing freckles can do that you could not also do with a shell script. Might be a few more lines of code though.
- Docker
-
freckles shares some characteristics with Docker (you could consider a frecklet a sort of 'universal' Dockerfile), but in other ways Docker is a totally different technology. There are opportunities of interaction between the two, though:
- freckles can be used from within a Dockerfile
- freckles can be used to manage the Docker service as well as builds, images, and the running of containers
- freckles can be used as a 'zero-dependencies' installer for you Docker application (installing Docker as well as your container or comtainer assembly) -- 'orchestration light', kinda.
- Kubernetes
-
Kubernetes does orchestration of containerized applications, and is very complete in that regard.
Similar to Docker, freckles can be used from within or in combination with Kubernetes. It can also be used to provision the initial Kubernetes cluster, and manage it's nodes. There will be a 'freckles-kubernetes-adapter' at some point in the future, I'm pretty sure.
- Terraform, Packer
-
Terraform and Packer are products developed by Hashicorp -- a company that does not seem to be able to create bad products (honestly, everything they do makes sense!). Terraform deals with creating infrastructure, and Packer lets you create images (Docker, VMs, etc.). They work well together and are both fairly mature, and especially Terraform is well established in this space. As with Docker, freckles can be used within those products (as a provisioner) or to control them, as a sort of meta-provisioning wrapper.
If you already use Terraform, there is probably not a good reason for you to use freckles, except maybe if your project is not really all that involved, and you consider the complexity that comes with Terraform overkill in that instance. I intend to implement a freckles-terraform-adapter for the next version of freckles, but that is still a way off.
I'd say, the main difference could be described as: Terraform is 'Infrastructure as code', and freckles is 'Infrastructure as a script'. Make of that what you will :-) freckles has probably a gentler learning curve (but of course I'd say that!).
- cloud-init
-
From their web-site: "Cloud-init is the defacto multi-distribution package that handles early initialization of a cloud instance." cloud-init lets you do pretty much the same thing that freckles can do: setup and configure a vanilla box of some sort. cloud-init is usually integrated with VPS providers (Proxmox supports it, EC2 has it, if I'm not mistaken, also Hetzner Cloud and lots of others), to let you run scripts or apply configuration when running a machine for the first time.
The one slight disadvantage of cloud-init is that it needs to be available for you to use it, and in some cases you are in no position to install it. If you use freckles, you can just bring it wherever you go. You can however, and I've done that quite a few times, kick off a freckles run from cloud-init. That way you get to keep your machine configuration independent of the need to have cloud-init available.
- Pulumi
-
This is interesting too, I haven't heard of it up until a few months ago. Pulumi also does 'Infrastructure as code', similar (but not too similar) to how freckles does it.
One difference is that Pulumi directly supports 3 languages (Node, Python, Go) and handles everything within the context of a 'Pulumi project' (if I'm not mistaken), whereas freckles is designed to be used from whatever you have laying around (similar to the way unix-y pipelines work): if you want to get something into a certain state, you feed freckles some input (a list- or dict-type datastructure) describing that state, and you get back some output (a dict-type data structure). How you create and ingest those data structures is none of freckles concern. You can use whatever language or setup you want with it.
I guess it is a bit like the difference between having a full-scale object-oriented Java application versus a few small Python scripts. There are advantages and disadvantages to both approaches, and which one you choose depends heavily on the requirements. Pulumi is most definitely worth checking out though, if you are in the market for a tool like that.
- CloudFormation
- Amazons 'Infrastructure as Code' tool. Can only be used with AWS itself. If you are the sort of person who is ok with that, freckles is probably not for you. You can use freckles from and with CloudFormation though, if you want to.
- Sandstorm
- I have never used that one myself, but it looks very polished. It basically installs and manages self-hosted services for you. freckles can do that too, but in a much more bare-bones way, and it does not have a nice looking graphical interface (yet). freckles is less opinionated, and I would think it is easier to add new application templates to freckles than to Sandstorm (I might be wrong though). freckles would probably be a good backend to create a tool like Sandstorm.