Azure Resource Manager (ARM) templates are the foundation of Infrastructure as Code in Azure. An ARM template lets you describe the resources you want and Azure will make it happen. ARM templates are nothing more than JSON files. When deploying the templates to Azure, Azure checks for syntax validity and then tries to deploy the template. When something goes wrong you get an error back.
While working with different customers all building complex ARM templates with multiple teams, I started thinking about how we could do things better.
Shift left
When the fuel light comes on while driving your car, what do you do? You’ll probably look for the first gas station you can find and fill up your car. You know that if you wait and run out of fuel it will take much more time to walk to a gas station, fill your jerrycan and walk back. Both options solve the problem but no one would choose the second option over the first.
The sooner you catch an issue, the less money and time it costs. The same is true in software development. If you imagine a timeline ranging from idea through code, test and deploy, the farther right you catch an issue the more it will cost you.
Now apply this to ARM templates. Writing a template, deploying it to Azure, going into the portal and checking if everything is ok is as far right as it can be. Shifting left to find issues earlier saves time and money and will improve your overall performance.
SonarQube
SonarQube is a very cool product that helps you analyze the quality of your code. To see what it can do you can browse the analysis of open source projects at https://sonarcloud.io/. Take for example the Roslyn compiler: https://sonarcloud.io/dashboard?id=roslyn. Figure 1 shows you a screenshot of SonarCloud showing some of the issues in the Roslyn repository.
A list of issues detected in the Roslyn project on SonarCloud
SonarQube scans your code without actually having to run or deploy it. This means you can run it as a part of your continuous integration build. It’s fast and accurate.
SonarQube has a plugin model to add scanning capabilities for new languages. Rules are added to verify specific issues and raise warnings or errors. The rules are written in Java and deployed as JAR files in a plugin folder in the SonarQube install directory (there are other options such as importing results into SonarQube from an external tool but that doesn’t give the best user experience).
If you don’t have a SonarQube server ready, you can find an ARM template that deploys a server on GitHub: https://github.com/Azure/azure-quickstart-templates/tree/master/sonarqube-azuresql
Getting started with a SonarQube plugin
My goal was to build a proof of concept to show that you can use SonarQube to validate ARM templates. Nothing fancy yet, just making sure it’s possible in a reasonable amount of time and worth pursuing.
Since SonarQube plugins are build in Java, I first had get a Java dev environment up and running. I choose to use VS Code with some Java extensions. After getting my Hello World in Java working I started looking at custom SonarQube rules and plugins.
SonarQube plugins target a specific language. A plugin parses the language and offers functionality that rules can then use to analyze a language for specific issues. The language in this case is JSON and fortunately there is a really good plugin available on GitHub for analyzing JSON: https://github.com/racodond/sonar-json-plugin. The plugin describes the JSON language and offers a parser that you can use as the basis for your rules. The plugin has a couple generic JSON rules and some specific rules for Puppet available that serve as sample code.
If you examine the plugin you’ll see the following:
- sonar-json-plugin is the bootstrapper for the plugin.
- json-frontend is where the magic happens. Here the JSON language is defined and a parser and visitor are defined. This is the code you build on when building custom JSON rules
- json-checks-testkit defines JSONCheckverifier, a class that you can use in your unit tests to verify your custom rules.
- json-checks defines the custom rules that are part of the plugin. This is excellent sample code to create your own rules.
All parts of the plugin have unit tests and are build and packaged with Maven.
Creating a custom rule for ARM templates
Using the JSON plugin as a foundation I created a simple rule that checks if a key $schema is present in the JSON file and if it has the correct value:
@Rule(key = "schema-key", priority = Priority.MAJOR, name = "The ARM template needs to define a schema key with the correct value", tags = {
"bug" })
@SqaleConstantRemediation("5min")
public class SchemaKeyCheck extends DoubleDispatchVisitorCheck {
private static final String SCHEMA_KEY = "$schema";
private static final String SCHEMA_VALUE = "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#";
private final List definedKeys = new ArrayList<>();
@Override
public void visitJson(JsonTree tree) {
definedKeys.clear();
super.visitJson(tree);
if (!definedKeys.contains(SCHEMA_KEY)) {
addFileIssue("The key " + SCHEMA_KEY + " is required.");
}
}
@Override
public void visitPair(PairTree pair) {
definedKeys.add(pair.key().actualText());
if (SCHEMA_KEY.equals(pair.key().actualText())) {
if (!pair.value().value().is(Tree.Kind.STRING)) {
createIssue(pair.value());
} else {
if (!SCHEMA_VALUE.equals(((StringTree) pair.value().value()).actualText())) {
createIssue(pair.value());
}
}
}
super.visitPair(pair);
}
private void createIssue(ValueTree tree) {
addPreciseIssue(tree, "The key $schema needs to have the value " + SCHEMA_VALUE);
}
}
You can find the full code on GitHub.
The attributes on top of the class offer metadata to SonarQube for your rule name, description, issue type, and a technical debt ratio (the Sqale Constant). After that comes the class definition. The method visitPair records a list of all keys in the JSON file and then checks each key value pair for a key named $schema and the correct value. The visitJson method is called after all pairs in the JSON file are visited. It then checks if a key named $schema is present at all. If not, issues are added. The key/value issues are added on the exact line where the issue is discovered. The missing $schema key is added on a file level.
In addition to the actual check, there is also a corresponding HTML file that’s used to render a description on the SonarQube web application:
The key $schema needs to be defined with the value ”http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#“.
And finally, a set of unit tests that validate my rule:
public class SchemaKeyCheckTest {
@Test
public void testValidSchemaKey() {
JSONCheckVerifier.verify(new SchemaKeyCheck(), new File("src/test/resources/checks/armwithschemakey.json"))
.noMore();
}
@Test
public void testInvalidSchemaKey() {
JSONCheckVerifier.verify(new SchemaKeyCheck(), new File("src/test/resources/checks/armwithinvalidschemakey.json"))
.next().withMessage("The key $schema needs to have the value http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#")
.noMore();
}
@Test
public void testNoSchemaKey() {
JSONCheckVerifier.verify(new SchemaKeyCheck(), new File("src/test/resources/checks/emptyjson.json"))
.next().withMessage("The key $schema is required.")
.noMore();
}
}
Using the plugin
To create the JAR file, run maven clean package from a command line. This creates a JAR file in the target folder. You then copy the JAR file to {YourSonarQubeInstallFolder}\extensions. After that you need to restart SonarQube to load the plugin. If something goes wrong, look into the log folder to see any errors. I’ve created a simple PowerShell script deploy.ps1 to stop the service, copy the JAR file and start the service.
The new rule is now available in SonarQube. You need to add the rule to a quality profile to apply the rule to JSON files.
To test the plugin, I created a couple of ARM templates and used sonar-scanner to run a SonarQube analysis on a folder. The end result is that the new issues nicely show in SonarQube:
And that’s it!
Setting up a CI/CD pipeline for your new rules with VSTS is a couple of clicks but that’s for another day. For now, I’ve written my first rule and it actually works!
Conclusion
After some getting started issues, writing custom SonarQube rules is pretty easy. I think there is real value in adding more rules for ARM templates and sharing these as open source.
What rules would you like to see? Some that I’m thinking of:
- No unused parameters
- All secrets from key vault
- No hardcoded URLs
- Configurable required tags/parameters
- …
If you think this is useful please leave a comment. Or even better, build a rule and leave a pull request!