(λ (x) (create x) '(knowledge))

Auskultanto

Prototyping things in Golang · August, 15th, 2022

Over the past two years I've absolutely fallen in love with Golang. It's a great (not so little) language that Google has been shepherding, but really we have the brilliant minds behind Plan9 to thank for it. All of that might just upset some people, not everyone's a big G fan after all, but I don't mind when the tool fits the job. For me, that job is typically prototyping. In my line of work being able to move fast and get a function example or MVP out is a huge step, and the faster I can do it the better. Obviously I still want to be able to use that prototype, kick the tires and such what, but so long as I'm not worried about the size and to some degree the performance I will readily and happily reach for Golang.

You might think to yourself, what does Golang have going that couldn't be accomplished in Python, or pretty much any language you could cherry pick out the list. And my answer to you would be nothing I suppose. But it has some bells and whistles which make me reach for it before the likes of Python. (And for those of you who know me, while I do actively avoid reaching for Python at all, it has its time and place too. Just not on my machines..). Anyways, those features. Golang's a lot like C, you can compile it down to a tiny static binary, a few megabytes in size. You can natively cross compile it to multiple different operating systems. The community and core libraries are absolutely bananas, nearly a library for anything you could need or want. And the compiler is so obtusely opinionated that you have to try to write buggy code!

All of this makes it a joy to work with. I know that something I write in Golang can compile and run on Alpine Linux, or Ubuntu, or Arch, but also Windows or MacOS, and funnily enough even Plan9! And while that last one might seem silly, I frequently find myself utilizing the same tooling on my Plan9 terminals that I reach for when I have my droid. At $work I use it to quickly build backend services, little glue bits that extract and ex-filtrate data, or keep an eye on things that can't be tied into the larger monitoring picture. And I've even written a couple of silly HTTP monitoring/utility services like ATSRV, PNTP, fServ, and auskultanto. In order that's a /proc info server, a poorman's NTP server, a file download/upload server, and a configurable status system. You get the picture I hope, you can kind of throw together these neat little micro services that can compile for anything, run anywhere, and due to the availability of the language's libraries be built in a somewhat trivial effort/timeframe.

Once again, yes I could just do this with Fennel/Lua or Common Lisp, but it just isn't as fuss free. And as much as I love Common Lisp, the binaries end up being 30mb+, so I can't complain in the slightest about a 6mb Golang binary. Also, have you tried using Fennel on Plan9? It's stuck on an ancient version and needs some love before that's going to be a viable option.

Latest Prototype

My latest prototype utility is another HTTP micro server, it's a little configurable custom check system, meant to be thrown behind a load balancer like the AWS ALB so you can define a custom health check. Normally not a necessary step, but sometimes just checking to see if Apache is serving something at a path isn't enough, so auskultanto (listener in Esperanto) listens in for those little health checks and returns up to date information about, well, whatever you decide to configure really! Here let me show you.


~|>> curl http://127.0.0.1:8092/command?name=uptime
{"Stdout":" 21:10:43 up  3:39,  0 users,  load average: 1.17, 0.95, 1.17\n","Stderr":"","Timestamp":"2022-08-15 21:10:43.007781982 -0400 EDT m=+23.979705812"}

~|>> curl http://127.0.0.1:8092/service?name=sshd
{"Stdout":" * status: started\n","Stderr":"","Timestamp":"2022-08-15 21:11:44.101013183 -0400 EDT m=+85.072937013"}

~|>> curl http://127.0.0.1:8092/script?name=test.sh
{"Stdout":"neuromancer\n","Stderr":"","Timestamp":"2022-08-15 21:11:59.156463623 -0400 EDT m=+100.128387453"}
  

Auskultanto exposes three endpoints, command, service, and script, and for each one it returns a little JSON blob with the stdout/stderr of the command chosen and a timestamp from the execution. Really simple, and pretty nifty! And you're probably thinking, this is absolutely horrible, it's remote code execution over HTTP! Well.. yes and no. Lets look at the config file.


Log: auskultanto.log
ScriptDir: /var/auskultanto/scripts/
Init: openrc
# /service?name=sshd
Services:
  - sshd
  - iptables
# /command?name="netstat%20-tlpn"
# /command?name=uptime
Commands:
  - uptime
  - hostname
  - netstat -tlpn
# /script?name="test.sh"
Scripts:
  - test.sh
  

Nothing crazy, but you'll note that under each endpoint we list out our valid checks, each as a single line under the endpoint. And while it might not stick out, it's perfectly fine to include multi argument commands such as netstat -tlpn, auskultanto will be happy to consume that. The only caveat is that you should escape your spaces with %20, it is a URL after all.

Not too shabby for a prototype right? We can define a couple of simple commands, write a quick script, or look for a service status and then write a little match using the JSON output. But what happens if we run a command that isn't configured? Obviously with something like this the very first thing we should try is a /command?name=whoami, or maybe a /command?name=sudo%20whoami. And if those work, we definitely need to try a /command?name=sudo%20rm%20-rf$20/%2A right?

Whenever an endpoint is queried, auskultanto records the endpoint, what the name of the request was, and then any error messages related to that event. And obviously a timestamp, because it wouldn't be much of a log otherwise would it? Here's the log from the example above:


2022/08/15 21:10:43 Queried: /command
2022/08/15 21:10:43 Command key: uptime
2022/08/15 21:11:43 Queried: /service
2022/08/15 21:11:43 Service key: sshd
2022/08/15 21:11:59 Queried: /script
2022/08/15 21:11:59 Script key: test.sh
  

And this is what happens when we start trying to run things that aren't configured. Auskultanto isn't particularly fond of it. Lets try a whole bunch of unconfigured things!

  • /service?name=wildfly
  • /command?name=sudo%20whoami
  • /command?name=uptime%26%26whoami
  • /command?name=rm%20-rf%20/%2A
  • /script?name=test.sh%3B%20rm%20-rf%20/%2A

2022/08/15 21:31:03 Queried: /service
2022/08/15 21:31:03 Service key: wildfly
2022/08/15 21:31:03 wildfly is not a configured service.
2022/08/15 21:31:24 Queried: /command
2022/08/15 21:31:24 Command key: sudo whoami
2022/08/15 21:31:24 sudo whoami is not a configured command
2022/08/15 21:42:28 Queried: /command
2022/08/15 21:42:28 Command key: uptime&&whoami
2022/08/15 21:42:28 uptime&&whoami is not a configured command.
2022/08/16 01:53:00 Queried: /command
2022/08/16 01:53:00 Command key: rm -rf /*
2022/08/16 01:53:00 rm -rf /* is not a configured command.
2022/08/16 01:52:01 Queried: /script
2022/08/16 01:52:01 Script key: test.sh; rm -rf /*
2022/08/16 01:52:01 test.sh; rm -rf /* is not a configured script.
  

From the client side, when auskultanto doesn't recognize a command it silently logs the information like above, and doesn't return anything to the client. I may change this in the future, but my logic is that if there's no return people are less likely to poke at it. Adding more information, such as the requesting IP address is a solid next step for this little tool, so that iptables rules could be automated based on bad requests. If it ends up getting abused.

Once again, this tool is just a prototype, something thrown together in a couple of hours to see if it can even be done. I'm proud of how resilient and flexible it seems right out the box. I'll continue to work at the idea and expand on the functionality in the future, but for now enjoy a toy monitoring system.

Bio

(defparameter *Will_Sinatra* '((Age . 29) (Occupation . DevOps Engineer) (FOSS-Dev . true) (Locale . Maine) (Languages . ("Lisp" "Scheme" "Fennel" "Lua" "Go")) (Certs . ("LFCS"))))

"Very little indeed is needed to live a happy life." - Aurelius