((λ (x) (create x)) '(knowledge))

Embedding Lua in Nim

The humble beginnings of yet another project ยท October 6th, 2025

This one is inspired by a conversation I had recently with acdw, and several I've had over the years with mio. For those that don't know my day to day work involves a ton of DevOps tooling, pretty much all of which requires some sort of templated yaml DSL. You know, the ridiculously large Kubernetes yaml manifests, your Ansible yaml+jinja playbooks, the different but still yaml+jinja salt state files, oh wait cloud-init too with its custom yaml configuration. Ah and a series of personal and internal tools I've written that also use a yaml based specific syntax. You get the picture, the commonality amongst all of these great tools is yaml. The industry has all sort of settled on yaml being the syntax for configuration, and we've taken configuration to mean everything from simple config file definitions to quite literally the definition of entire infrastructure deployments spanning multiple hyperscalers and domains. It's wild!

And for a lot of people we don't really think any further about it. I need to learn the quirks of this yaml thing that yaml thing. It just is what it is, and little documents like the yaml document from hell exist for us all to laugh about and accept that, yeah that's a thing, but what would you replace it with if not yaml? And at this point I could simply provide a list of lots of different not yaml options that could replace yaml, some of them very yaml like, some of the exact opposite. I have no intention of doing that, if you want to create yet another configuration language syntax then please by all means do so. But that's not the point of this post!

The point of this post is to explore the viability of using Lua as a configuration system. Yes I did link to that intentionally, for if you read the about section it explicitly says this is the intent of Lua. I always sort of saw it as its own minimalist scripting language, so choosing to hold it in this somewhat foreign way is a change of pace for me. Let's dive in!

What is the problem precisely?

Right, so I can't just do a thing, I need a project to do the thing around. To that extent we'll be building our own Infrastructure as Code tool. I have a ton of experience doing this professionally, and actively contribute to and help maintain SaltStack for Alpine, so we're just going to go straight for SaltStack but without Python. To track my thought process a little bit, this is in the same vein initially as my verkos project which uses a yaml configuration to generate templated shell scripts. That has been the tool of choice to configure systems in my homelab for a couple of years now, but it isn't idempotent, can't produce dry runs, it's truly just generate a script and run it. Infrastructure as Code, even in a simple sense like that implode by a Salt master-less minion or Ansible playbook needs to be able to detect, report, and validate state to be of use.

We'll start with a really simple example, I need a way to install packages for Ubuntu and alpine systems. In Salt we'd do something like

Ensure Nebula is installed:
  pkg.installed:
    - name: nebula

Well sort of, pkg.installed doesn't work super well on Alpine, and maybe I'd prefer to use the snap package for nebula on my Ubuntu system.

Ensure Nebula is installed:
{% if grains['os'] == "Alpine" %}
  cmd.run:
    - name: apk -U add nebula
{% elseif grains['os'] == "Ubuntu" %}
  cmd.run:
    - snap install nebula
{% endif %}

That got out of hand quickly. We immediately had to introduce jinja into the mix to handle flow control on top of our yaml state definition. Yaml is not a programming language, adding jinja on top of it just allows us to change the resulting generated yaml depending on which OS we're targetting. This isn't a terribly design choice, and I feel confident saying that. But it could be better I think.

If we used lua for this configuration maybe we could express this configuration a little more clearly. Assuming we use a lua table as the data structure that is passed between our IaC agent and the configuration we write, then we'd be able to just us Lua code to express our intent.

state = {
  type = "package",
  name = "nebula",
  manager = ""
}

if grains.os == "Ubuntu" then
   state.manager = "snap"
elseif grains.os == "Alpine" then
   state.manager = "apk"
end

Now you might say, this isn't really all that different than yaml+jinja, and I'd argue yes and no. At this very basic level you're correct it isn't. Just bear with me. If you take away nothing else from this I'd argue that the Lua code is easier to test and debug than the yaml+jinja templating. At a more complex level you can be more expressive using Lua than yaml+jinja.

Right anyways, for the sake of the argument lets just assume we like the second syntax more. How do we consume it? By sticking a programming language into our programming language of course.

Embedding Lua in Nim

Fortunately for the curious there are a couple of different lua libraries available for Nim, so we can skip over creating a shim layer over the C library and just get to hacking by running nimble install lua. That will net us a lua5.1 runtime which is good enough.

Calling Lua from inside Nim is incredibly simplistic, but it requires thinking about stack machines, sort of like Forth. This is done for a few reasons, mostly to facilitate memory management nuances.

import lua

proc main() =
  # Load the Lua VM into memory
  var L = newstate()
  openlibs(L)

  # Read in and load our Lua script
  if L.dofile("state.lua") != 0:
    let error = tostring(L, -1)
	echo "Error: ", $error
	pop(L, 1)

  # Push the value of the global variable state onto the stack
  L.getglobal("state")
  # From the global state table, push the value of the type key onto the stack
  L.getfield(-1, "type")
  # Use the value of the type variable that's at the head of the stack as a string value
  echo "State of type: " $L.tostring(-1)
  # Pop the value of type off the stack
  L.pop(1)

  # This denotes the first key in the table
  L.pushnil()
  # Until there are no more values in the state table iterate
  while L.next(-2) != 0:
    # Push the value of the second element on the stack into a lexical variable named key
    let key = $L.tostring(-2)
    # Push the value of the first elemenet on the stack into a lexical variable named value
    let value = $L.tostring(-1)
    echo key, " = ", value
    # Pop the values off the stack
    L.pop(1)

  # And close the VM
  close(L)

main()

Assuming we have a lua script called script.lua in the path of of nim program when it's run it will do the following:

  • Create an instance of the Lua VM in memory
  • Read in the script, and then execute it against that VM
  • If there's an error, return that as a string by pushing the value of the error onto the stack, then popping it off once it's done.
  • Then we iterate over the state table and echo out the key, value tuples from it.
  • And finally we close the VM and clean up.

This doesn't really do much, but it's all you'd need to use Lua as a configuration system! The hardest part is wrapping your brain around how the stack machine works. Once you internalize that iterating over a table results in a reverse ordered list, precisely because you are first reading in the value of the key and then its value; well after that the entire thing becomes extremely easy to reason about.

Our Lua stack machine looks a bit like this in the nim code above:

[stack]                    [  ]
[stack] push "type"     -> [-1] "type"
[stack] push "package"  -> [-1] "package" [-2] "type"
[stack] pop             -> [  ]
[stack] push "name"     -> [-1] "name"
[stack] push "nebula"   -> [-1] "nebula" [-2] "name"

Now lets expand the state configuration and actually do something with it, and while we're at it, lets change name to packages and turn it into a list of packages to install.

state = {
  type = "package",
  packages = {"nebula", "htop", "iftop", "mg"},
  manager = ""
}

if grains.os == "Ubuntu" then
   state.manager = "snap"
elseif grains.os == "Alpine" then
   state.manager = "apk"
end

We then need to do two things in our nim code, map the global lua variable state to an object, and then parse & action off of it. In a real IaC system only specific values are valid, so instead of iterating over everything, we'll be explicit about what we want to read in.

import lua, os, osproc

# The nim object is pretty straight forward, we have three values, we only really need to use two. Two are strings (cstrings technicaly) and the other is an integer indexed table, so a sequence of strings in nim.
type State = object
  `type`: string
  packages: seq[string]
  manager: string

proc main()=
  # Load the Lua VM
  var L = newstate()
  openlibs(L)

  # Read in our state configuration
  if L.dofile("state.lua") != 0:
    let error = tostring(L, -1)
    echo "Error: ", $error
    pop(L, 1)

  # Instantiate an object to hold over configuration
  var state: State
  state.packages = @[] # and an empty seq for our package list

  # Get the global state table onto the stack
  L.getglobal("state")

  # Then the package table
  L.getfield(-1, "packages")  # Stack: [state_table, packages_array]
  L.pushnil()  # First key for iteration
  while L.next(-2) != 0:  # -2 is the packages array
    # Stack: [state_table, packages_array, key, value]
    state.packages.add($L.tostring(-1))  # Read in the package, append it to the seq
    L.pop(1)  # Pop value, keep key for next iteration
  L.pop(1)  # Pop the packages array
  # note these two pops are different, one is to clean up the values in the packages array
  # the other is to clean up the reference to the array itself

  # It's a lot easier to fetch the value of manager, since it's just a simple string.
  L.getfield(-1, "manager")
  state.manager = $L.tostring(-1)
  L.pop(1)

  var pkgcheck = ""
  var pkgman = ""

  # from this point forward we just use the values as expected in nim
  case $state.manager:
    of "apk":
      pkgcheck = "apk info --installed "
      pkgman = "apk add "
    of "snap":
      pkgcheck = "snap list | grep "
      pkgman = "snap install "

  for package in state.packages:
    let (check, code) = execCmdEx(pkgcheck & $package)
    if code == 1:
      let install = execCmdEx(pkgman & $package)
      echo "Installed " & $package
    else:
      echo $package & " is already installed"

  close(L)

main()

Now you're probably rightly saying that the example is obviously broken. Where is the grains value coming from? It isn't defined anywhere at all. Worry not, we can easily fix that by writing a little bit of lua code, that will nets us an ad hoc grain system.

# A new global table, we could reference this in our nim code
grains = {
  os = ""
}

# And to populate it we just need a way to figure out what OS we're on, parsing /etc/os-release is a tried and true method
function get_linux_version()
    local file = io.open("/etc/os-release", "r")
    if not file then
        return nil, "Could not open /etc/os-release"
    end

    local content = file:read("*all")
    file:close()

    local version_id = nil
    for line in content:gmatch("([^\n]+)") do
        if line:find("VERSION_ID=") then
            version_id = line:match("VERSION_ID=\"([^\"]+)\"")
            break
        end
    end

    return version_id
end

grains.os = get_linux_version()

state = {
  type = "package",
  packages = {"nebula", "htop", "iftop", "mg"},
  manager = ""
}

# This check will now actually work instead of throwing errors
if grains.os == "Ubuntu" then
   state.manager = "snap"
elseif grains.os == "Alpine" then
   state.manager = "apk"
end

Executing our little micro IaC nets us a stateful result! It detects what is in compliance, reports it, and corrects what's out of compliance. Pretty cool for around 100 lines of nim and lua combined.

~/Development/lambdacreate/views/posts|>> ./main
nebula is already installed
htop is already installed
Installed iftop
Installed mg

In essence, this means the system described above can fulfill the role of static configuration data much the same as yaml, but also the role of imperative configuration state, something that modifies itself depending on the scenario in which it's deployed. And that can be extended by simply adding more Lua instead of modifying the Nim runtime. This is a really neat trick, but it's not an idempotent declarative IaC system. It's nowhere even close to it in fact! But it has frankly been a great learning opportunity, and I'm excited to keep tinkering with it and see what I come up with. Worst case this will become a replacement for Verkos, my shell templating tool, but I'm kind of hoping I can replace Salt itself with my own home brewed Lua based system. Time will tell!

If you're curious and want to tinker a bit with these the state.lua file can be found here and the nim program is here.

Credit

My friends, acdw and mio , both deserve a massive shout out for so many reasons. Putting up with me pasting massive code snippets into IRC buffers for years and bickering with them over yaml being the only real way to do any of this stuff. Thanks for putting up with me, and nerd sniping me into making a thing. Here's to hoping it becomes something really cool! At very least, enjoy the learning!

Knowledge doesn't exist in a vacuum, Lucas Klassmann who wrote this absolutely phenomenal blog post about embedding Lua in C, deserves a shout out as well. It is an absolutely delightful read and an eye opening explanation as to how truly simple it is to embed Lua into something.