bacillμs (Build Automation/Continuous Integration Low-Linecount μ(micro)-Service) listens for HTTP GET or POST events, executing specified actions on receipt of matching endpoint requests. Use it to respond to webhooks from SCM managers such as github, gitlab, gogs.io, etc. or from wget or curl requests made from plain git commit hooks, or anything else one can think of.
The goal of this project is to offer an extremely minimal Build Automation (BA), Continuous Integration (CI) and Continuous Deployment (CD) system with zero dependencies on large frameworks, VMs or containers (though you can use those from bacillμs jobs, if you must). It basically should run on a potato, if that potato can run binaries compiled with Go, without breaking a sweat.
Bacillμs is language-agnostic. Any script or binary that can be launched from a shell can also be launched by bacillμs. Bacillμs doesn't force you to learn any flavour-of-the-week DSL (Domain-Specific Language).
Core features reflect those the author found essential while using, administering and customizing a more traditional ('butler-based') build automation system for a large dev team over multiple years. Experience showed that most of the 'extra stuff' was unnecessary and better achieved by utilizing common external tools.
Bacillμs is a single static binary written in Go, with nearly zero external configuration.
If you want a point-and-click build server that lets you make jobs without knowing what a shell or cron scheduler is, this probably isn't for you. But if you want a build server that serves as a launch point, has a minimal but useful web interface, and otherwise stays out of your way, read on.
Login as user account that will run bacillμs
$ git clone https://gogs.blitter.com/Russtopia/bacillus
$ cd bacillus
$ go build .
$ go install .
## .. finally, if you don't usually have $GOPATH/bin in your $PATH:
$ cp ./bacillus $PREFIX/bin # .. where $PREFIX = $HOME, /usr/local, ... your choice
## .. Try it out!
$ ./bacillus_launch.sh
Visit http://localhost:9990/
bacillμs, being a simple tool, has little configuration. Almost all is encapsulated in the tool's invocation command-line and in the worker scripts themselves. Individual job configuration can be controlled by defining environment variables, either statically as passed to each job within the tool invocation via endpoint arguments, or dynamically, via job parameters encoded within each job script, which are parsed at job launch to present a form that the user can fill in prior to each run (parameterized jobs).
Sample installation tree
/$HOME/
bacillus/ (project tree)
bacillus (main binary)
example_workdir/ (home of job scripts and running job workspaces)
jobA.{sh,py,...} (job entry script for 'jobA')
artifacts/ (where jobs place their 'artifacts' during/after run)
images/ (image assets used by main binary)
bacillus_launch.sh (example launch script with a few demo job endpoints)
bacillμs launches jobs as child processes, waiting on their exit and tagging their main stdout/stderr output, named 'console.out' within each worker's workspace (eg., workdir/bacillus<JOBID>). No external state or other meta-data is maintained, so there is no way to get out of sync with spawned jobs. If you kill the bacillμs daemon, all currently-running jobs die too in standard UNIX fashion, unless jobs themselves detach via nohup.
The repository contains sample scripts and git hooks:
In summary, to perform build/CI tasks with bacillμs, one should
Visual matching of job trigger and completion entries in the runlog can be indicated in various ways, controlled by the -i switch.
Valid values are [ none | indent | colour | both ]
.
Jobs launched by bacillμs get some default environment variables, which should be sufficient to bootstrap typical tasks:
git checkout
command to switch to that branchA single run of a job will have workdir/ and artifacts/ dirs named bacillus_<jobOpts>_${BACILLUS_JOBTAG}_${BACILLUS_JOBID}
.
The design of bacillμs follows the Unix tool philosophy: do one thing and do it well. As such, scheduling of repeated jobs and reaping of old job workspaces/artifacts to save disk space, archiving etc. are left to external tools (consider using cron, anacron, rsync, etc.). An example cron job to reap old workspaces and artifacts is given within the 'bacillus_launch.sh' script's comments.
To keep different categories of jobs logically separated and more manageable, consider grouping similar jobs together into the same launch script, and define a separate one for each such group (ie., daily builds vs. weekly builds vs. test jobs vs. git-triggered commit checks ...). Just change the endpoints specified in each copy of the launch script and the server port so each instance has its own web pages and bacillμs daemon. They can all run within the same install tree if one wants, or separately; the tool does not enforce any specific policy.
If launched with the --auth
option, bacillμs gates access to all served content via HTTP basic auth. Over plain HTTP this is not secure, so one should run behind a reverse proxy (eg., define a subdomain 'bacillus.yourdomain.net' mapping to 'localhost:9990'). This protects the HTML UI for manually triggering jobs, viewing status and artifacts, and controlling soft and hard shutdowns, as well as git/SCM triggered endpoint actions to run jobs. See the AUTH options in the example bacillus_launch.sh.
For scripts and SCM hooks, to trigger an endpoint (job) via eg. wget
or curl
, an initial login request must first be made to the bacillμs server, which will reply with an HTTP basic auth challenge and the text Not logged in.
. Subsequent requests made with the proper username:password then can proceed. See bacillus_launch.sh for more information.
For jobs which may need user-settable parameters at each job invocation, parameters may be placed within comments in the main job file; these are picked up by the tool to generate a web form the user can fill out before launching the job. The basic syntax is
-?T?VAR?DEFVALUE?COMMENT
.. where T = [ s (string) | c (choice) | b (bool) ] choices are separated by a pipe (|) character.
Example
#-?s?DELAY?5?Delay in seconds
#-?c?SUITE?small|big|huge?Size of something
#-?b?DEBUG?1?
Param lines such as the above should start at column 0 alone on a line, after the comment character on a new line in the script (acceptable prefixes currently are '#', '/*' and '//')
A job containing the above would present a form with a text box, a dropdown list and a checkbox for each of the job parameters. Each variable is added to the job's environment variables.
NOTE the ?DEFVALUE? above does not ensure a script sets the required variable to a default; it just specifies the HTML form's default value. The job script must itself check for undefined parameters and give them defaults.
SECURITY String parameters (?s?...) named with a NOPATH_ prefix or a _URI suffix are exempt from path sanitization. Use caution naming job parameters in this manner, being sure not to interpret such variables as filesystem paths or URIs within job scripts, to prevent path-traversal security violations (ie., running arbitrary binaries or scripts from outside the workdir/${BACILLUS_JOBID}
dir).
Parameterized builds should check if their parameters are set or not, and substitute defaults if required.
For example in bash use PARAM=${PARAM:-"defaultval"}
to ensure $PARAM gets a sensible default.
With the above in mind, use curl or wget to trigger a job with one or more non-default parameters:
$ curl -i "https://<bacillus-server>/<jobName>?usingParams&DELAY=2&SUITE=big&DEBUG=true"
There is support for a simple 'pipeline view' of the stages of running jobs. See
the 'stage' function in examples workdir/xs_pushbuild.sh
and workdir/bacillus-pushbuild.sh
. Stages up to and including the running stage will be displayed at the end of the running job's entry in the runlog view.
Some projects may wish to show the status of the most recent build on their own pages, or fetch a simple status update about a build job. As of v0.2.9.4, some endpoints may be referenced in order to see the build status of a job:
.../jobName/lastStatus # returns Content-type: text/plain, "n" (where n=0 for success, otherwise error)
.../jobName/lastStatusIcon # returns Content-type: image/jpeg (by default, a 32x32 JPEG) indicating success (0) or failure
.../jobName/lastRunDuration # returns Content-type: text/plain, "str" (where str=duration as a string. eg. 1.4s)
The lastStatusIcon
endpoint sends an HTTP Header Cache-Control: no-cache
with the sent image, in order to aid clients querying the endpoint in displaying the most up-to-date CI status icon for the job; however initial testing indicates one might still need to force a refresh of one's webpage view to see the updated icon. Try an auto-refresh web extension which is capable of doing automated 'hard' refreshes (ignoring the browser cache) such as Auto Refresh Plus for Chrome. If you have control over your monitoring client, add a random URL parameter to the endpoint, eg. .../jobName/lastStatusIcon?234034508
to defeat browser cacheing.
Prerequisites: golang (for example xs_pushbuild.sh build script as well as bacillμs itself)
[terminal A - CI server]
$ cd <installation_dir>
$ ./bacillus_launch.sh
Visit tool main page on localhost:9990/ (see bacillus_launch.sh to configure port)
[terminal B - client event test]
$ curl -s http://localhost:9990/onPush-hkexsh-build
If --auth
is used, the curl request will require credentials to activate the job endpoint, eg:
$ curl -s --netrc-file hooks/auth.txt http://localhost:9990/onPush-bacillus-build
Likewise, a web browser will present a user/pass authorization popup before allowing access to the web interface.
$ sloc *.go
Languages Files Code Comment Blank Total CodeLns
Total 3 806 744 124 1674 100.0%
Go 3 806 744 124 1674 100.0%
Nope. But I might add a REST API someday, as use cases come up.