Credit: Dreamstime
In the last dozen years or so, we’ve witnessed a dramatic shift from general purpose databases (Oracle, SQL Server, etc.) to purpose-built databases (360 of them and counting). Now programming languages seem to be heading in the same direction.
As developers move to API-driven, highly elastic infrastructure (where resources may live for days instead of years), they’re building infrastructure as code (IaC). But how to build IaC remains an open question.
For a variety of reasons, the obvious place to start was with imperative languages like C or JavaScript, telling a program how to achieve a desired end state.
But a new model of declarative languages — like HCL from HashiCorp or Polar from Oso — has emerged, where the developer tells the program what the desired end state is, not overly worrying about how it gets to that state. Of the two approaches, declarative programming might well be the better option, resulting in leaner, safer code. Let’s look at why this is.
The why of declarative programming
When I asked Sam Scott, co-founder and CTO of Oso, whether we really needed another programming language, his answer was “yes.” The longer version: “With imperative languages there is often a mismatch between the language and its purpose; that is, these languages were designed for people to build apps and scripts from the ground up, as opposed to defining configurations, policies, etc.”
Introducing new, declarative languages like Oso’s Polar may actually save us from language proliferation, Scott went on: “It’s not just about introducing another language for solving a specific set of problems. Rather, it’s creating something to avoid people from having to invent their own language time and time again to do some form of embedded logic.”
Take, for example, the JSON code written for AppSync authorisation:
{
“version” : “2017-02-28”,
“operation” : “PutItem”,
“key” : {
“postId” : $util.dynamodb.toDynamoDBJson($context.arguments.id)
},
“attributeValues” : {
“Author” : $util.dynamodb.toDynamoDBJson($context.identity.username)
#foreach( $entry in $context.arguments.entrySet() )
#if( $entry.key !="id" )
,”${entry.key}” : $util.dynamodb.toDynamoDBJson($entry.value)
#end
#end
},
“condition” : {
“expression” : “attribute_not_exists(postId)”
}
}To get to balance between data and logic, the developer has written inline, comment-style permissions in Apache Velocity Template Language. This is just one example of the gymnastics developers go through to try to express authorisation (“authZ”) logic using a combination of static configuration and templating.
But, really, this isn’t about whether we should use imperative programming, VMware’s Jared Rosoff stressed in an interview.
Instead, “It’s a question of ‘We don’t currently use a programming language to express authorisation rules, but maybe we should…’.” After all, we use data, not a programming language, to express authZ rules. This is fine, Rosoff notes, when your authZ rules are pretty simple. You basically have a lookup table that you consult when a request comes in to decide if it’s authorised or not.
Easy, peasy.
Imperative may not fit IaC
But the AppSync example above shows how the data-oriented, imperative approach gets complicated quickly.
“What started as a relatively simple JSON document evolved into something with branching and conditionals and variables,” Rosoff continued. In taking this approach, the developer is trying to recreate language-like semantics within the syntax of static data, resulting in a “crappy programming language that has terrible syntax, no libraries, no debuggers, etc.” It’s not ideal.
It’s also not concise, says Scott. “It’s hard to encode your authorisation logic in traditional languages,” he notes. “Doing it in a special-purpose, declarative language like Polar is more expressive and more concise.”
As Scott further explained, there are a lot of problem domains where if a developer were to try to encode a process in Java or Python it would require thousands of lines of code, which is a problem in and of itself, but becomes more problematic in that it obscures the actual business logic. By contrast, a declarative language can accomplish the same job but in tens of lines of code instead.
So instead of forklifting configuration syntax onto imperative languages, says Rosoff, we should be using little programs to handle authZ and related functions. There are many programming problems today where we use data to configure a system but it might make more sense to use a program, according to Rosoff. To name a few:
- Authorisation
- Deployment templates (e.g. Terraform, Ansible, etc.). “Each of these tries to create language-like constructs within the confines of a data language with terrible results.” Pulumi offers an interesting approach to this.
- Workflows. Temporal is an interesting example of taking what would have previously been done in a diagrammatic workflow language and bringing it into a more general purpose programming language format.
Just enough freedom, with simplicity and safety
The trick, says Rosoff, is to give the programmer enough of a language to express the authorisation rule, but not so much freedom that they can break the entire application if they have a bug. How does one determine which language to use? Rosoff offers three decision criteria:
- Does the language allow me to express the complete breadth of programs I need to write? (In the case of authorisation, does it let me express all of my authZ rules?)
- Is the language concise? (Is it fewer lines of code and easier to read and understand than the YAML equivalent?)
- Is the language safe? (Does it stop the programmer from introducing defects, even intentionally?)
We still have a ways to go to make declarative languages the easy and obvious answer to infrastructure-as-code programming.
One reason developers turn to imperative languages is that they have huge ecosystems built up around them with documentation, tooling, and more. Thus it’s easier to start with imperative languages, even if they’re not ideal for expressing authorisation configurations in IaC.
We also still have work to do to make the declarative languages themselves approachable for newbies. This is one reason Polar, for example, tries to borrow imperative syntax.
Given the benefits of a declarative approach, however, it may simply be a matter of time until they become standard for IaC.
