Apache Freemarker to build code from graphical models
In this post, we are going to describe how we use Freemarker in the Onesait Platform Center to build executable code from the graphical models built with our diagram editor.
In many applications, it is usual to have to generate texts and files from templates, that is to say, files with a predetermined structure that is organized based on a label language, so that, when they are processed and when certain variables in the template are replaced or certain blocks added, its result becomes something concrete and personalized. Examples of this type of application can range from something as simple as generating a generic email, to sending it to hundreds of users where we want to customize the name of the recipient. Another example is automatic source code generation tools, where we generate executable code from a graphic model.
Apache Freemarker stands out among the main open source template engines, both for its simplicity and for its template processing speed. It is a Java library, which, when added to our projects, allows us to define templates through files in its own language, called FTL, and process them in Java through its API, passing the data to generate text or files in any format, (including files in Office Word format).
Freemarker’s FTL language allows us to build templates where, among other operations, you can:
- Assign values to simple variables.
- Include conditional blocks.
- Iterate over a list and apply it to a block.
- Carry out treatment and formatting of text strings.
- Execute arithmetic operations.
Within the Onesait Platform Center, Apache Freemarker is a widely used library, and it is interesting to note that it is used to generate emails personalized to the project and the user, automatic documentation, but its main power within the Center is found in code generation:
- Generation of Dockerfile files: from containerization diagrams.
- Generation of Helm Charts: from Helm diagrams.
- Terraform code generation: for infrastructure deployment from Terraform Diagrams.
In all cases, the Onesait Platform Center generates executable code from a visual model such as a diagram modeled in the Center graphic tool and which is stored internally in JSON format in an ontology, for example: generating a Dockerfile with which to invoke the Docker daemon to build and publish a container image.
We will illustrate the use of Freemarker by reviewing how we convert a Helm Chart into a Kubernetes-installable Chart.
- Graphically, a Helm Diagram consists of a set of graphic elements related to each other, each one with a set of properties and that model the deployment of our application in Kubernetes:
- When saving the diagram, it is converted internally into a JSON document stored in an entity/ontology of Onesait Platform:
- In order to convert said JSON into a Chart that can be installed in Kubernetes, the first step is to include the Freemarker dependency in all the modules where it is going to be used — in this case, in the onesaitplatform-center-helm-generator module:
<dependency>
<groupId>org.freemarker</groupId>
<artifactId>freemarker</artifactId>
</dependency>- The next step is to define the Freemarker FTL templates. A Chart Helm consists of a set of YAML files within a directory structure. Each directory stores one type of descriptor:
In each directory, a YAML descriptor is stored for each element of that type in the Chart. For example: in the deployments directory, all the descriptors of the objects of type Deployment are stored:
In this way, we can guess that, in our onesaitplatform-center-helm-generator project, we will have a Freemarker template for each type of element:
Each of these templates represents the descriptor of a type of Helm object, where, using the Freemarker language, everything that has to be completed for each specific case, has been externalized with the information extracted from the JSON of the diagram. For example, the Deployment template that we can see below:
apiVersion: apps/v1
kind: Deployment
metadata:
name: ${name}
namespace: ${namespace}
spec:
selector:
matchLabels:
workload.user.cattle.io/workloadselector: ${workloadselector}
progressDeadlineSeconds: 600
replicas: 1
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
labels:
workload.user.cattle.io/workloadselector: ${workloadselector}
spec:
{{- if .Values.global.setNodesOnDeployments }}
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: {{ .Values.global.nodeKey }}
operator: In
values:
{{- range .Values.global.workerNodes }}
- {{ .node }}
{{- end }}
{{ end }}
containers:
- name: ${containerName}
image: ${containerImage}
resources:
limits:
cpu: ${containerCpuLimit}
memory: ${containerMemoryLimit}
requests:
cpu: 1m
memory: 1Mi
<#if envVars?has_content || envVarsFromConfigMap?has_content || envVarsFromSecret?has_content >
env:
<#list envVars as envVar>
- name: ${envVar.name}
value: ${envVar.value}
</#list>
<#list envVarsFromConfigMap as envVarFromConfigMap>
- name: ${envVarFromConfigMap.envVarName}
valueFrom:
configMapKeyRef:
key: ${envVarFromConfigMap.configMapKey}
name: ${envVarFromConfigMap.configMapName}
</#list>
<#list envVarsFromSecret as envVarFromSecret>
- name: ${envVarFromSecret.envVarName}
valueFrom:
secretKeyRef:
key: ${envVarFromSecret.secretKey}
name: ${envVarFromSecret.secretName}
</#list>
</#if>
imagePullPolicy: ${imagePulPolicy}
<#if containerPorts?has_content>
ports:
<#list containerPorts as port>
- containerPort: ${port.containerPort}
name: ${port.containerPortName}
protocol: ${port.containerPortProtocol}
</#list>
</#if>
<#if volumeMounts?has_content || volumeMountsConfigMap?has_content || volumeMountsSecret?has_content>
volumeMounts:
<#list volumeMounts as volumeMount>
- name: ${volumeMount.mountName}
mountPath: ${volumeMount.mountPath}
</#list>
<#list volumeMountsConfigMap as volumeMountConfigMap>
- name: ${volumeMountConfigMap.volumeName}
mountPath: ${volumeMountConfigMap.mountPath}
subPath: ${volumeMountConfigMap.subPath}
</#list>
<#list volumeMountsSecret as volumeMountSecret>
- name: ${volumeMountSecret.volumeName}
mountPath: ${volumeMountSecret.mountPath}
subPath: ${volumeMountSecret.subPath}
</#list>
</#if>
<#if imagePullSecrets?has_content>
imagePullSecrets:
- name: ${imagePullSecrets}
</#if>
restartPolicy: Always
terminationGracePeriodSeconds: 30
<#if volumeMounts?has_content || volumeMountsConfigMap?has_content || volumeMountsSecret?has_content>
volumes:
<#list volumeMounts as volumeMount>
- name: ${volumeMount.mountName}
persistentVolumeClaim:
claimName: ${volumeMount.pvcName}
</#list>
<#list volumeMountsConfigMap as volumeMountConfigMap>
- name: ${volumeMountConfigMap.volumeName}
configMap:
defaultMode: 256
items:
- key: ${volumeMountConfigMap.configMapKey}
mode: 420
path: ${volumeMountConfigMap.configMapPath}
name: ${volumeMountConfigMap.configMapName}
optional: false
</#list>
<#list volumeMountsSecret as volumeMountSecret>
- name: ${volumeMountSecret.volumeName}
secret:
defaultMode: 256
items:
- key: ${volumeMountSecret.secretKey}
mode: 420
path: ${volumeMountSecret.secretPath}
secretName: ${volumeMountSecret.secretName}
optional: false
</#list>
</#if>
externalizes:
- Simple values, such as the name of the Deployment (name: ${name}).
- Conditional blocks, such as the possibility of having to add environment variables (<#if envVars?has_content || envVarsFromConfigMap?has_content || envVarsFromSecret?has_content >).
- Lists, such as the list of environment variables to add (<#list envVars as envVar>).
Thus, each of the YAML files (Helm descriptors of a Deployment object), which we saw in the image:
corresponds to the fact of applying, to the template deployment.ftl that we have described, the properties extracted from the diagram’s JSON, for each element of type Deployment of the previous diagram.
- Once the templates are defined, we use the Freemarker API to load them in our code and be able to process them. To do this, we load each ftl template in a Java object of type Template:
private Template templateChartDescriptorFile;
private Template templateHelmIgnoreFile;
private Template templateNotesFile;
private Template templateNamespace;
private Template templateSecret;
private Template templateRegistryCredentials;
private Template templateCertificate;
private Template templateConfigMap;
private Template templateStorageClass;
private Template templateDeployment;
private Template templateIngress;
private Template templatePersistentVolume;
private Template templatePersistentVolumeClaim;
private Template templateSevice;
private Template templateHelper;
@PostConstruct
public void init() throws IOException {
Configuration cfg = new Configuration(Configuration.VERSION_2_3_27);
cfg.setClassForTemplateLoading(this.getClass(), "/");
cfg.setDefaultEncoding("UTF-8");
cfg.setTemplateExceptionHandler(TemplateExceptionHandler.RETHROW_HANDLER);
cfg.setLogTemplateExceptions(false);
cfg.setWrapUncheckedExceptions(true);
try {
// Load Templates
this.templateChartDescriptorFile = cfg.getTemplate("templates/chartDescriptorFile.ftl");
this.templateHelmIgnoreFile = cfg.getTemplate("templates/helmignore.ftl");
this.templateNotesFile = cfg.getTemplate("templates/notes.ftl");
this.templateNamespace = cfg.getTemplate("templates/namespace.ftl");
this.templateSecret = cfg.getTemplate("templates/secret.ftl");
this.templateRegistryCredentials = cfg.getTemplate("templates/registryCredentials.ftl");
this.templateCertificate = cfg.getTemplate("templates/certificate.ftl");
this.templateConfigMap = cfg.getTemplate("templates/configMap.ftl");
this.templateStorageClass = cfg.getTemplate("templates/storageClass.ftl");
this.templateDeployment = cfg.getTemplate("templates/deployment.ftl");
this.templateIngress = cfg.getTemplate("templates/ingress.ftl");
this.templatePersistentVolume = cfg.getTemplate("templates/pv.ftl");
this.templatePersistentVolumeClaim = cfg.getTemplate("templates/pvc.ftl");
this.templateSevice = cfg.getTemplate("templates/service.ftl");
this.templateHelper = cfg.getTemplate("templates/helpers.ftl");
} catch (IOException e) {
log.error("Error loading templates", e);
throw e;
}
}
- Next, we analyze the JSON of the diagram and, for each element that we find, we extract its properties, we put them in the properties map that we will pass to the Template of, forgive the repetition, the template corresponding to the element, and we invoke Freemarker to generate the resulting file in the desired path:
Map<String, Object> model = new HashMap<>();
model.put("name", json.getString("certName"));
model.put("namespace", json.getString("namespace"));
model.put("certificate", json.getString("certificate"));
model.put("certificateKey", json.getString("certificateKey"));
Writer reg = new FileWriter(
new File(chartBaseDirectory.concat("templates").concat(File.separator).concat("secrets"),
json.getString("certName").toLowerCase().concat("-certificate.yaml")));
templateCertificate.process(model, reg);
This way, by extracting all the JSON components from the Helm Diagram, extracting their properties and passing them to the corresponding Freemarker template, we are generating all the YAML descriptors that make up the Helm Chart that can be installed in Kubernetes.
Conclusion
One of the main tasks of the Onesait Platform Center as a development support accelerator tool, is to free the user from performing tedious and complicated tasks that add little value to the business of the product under development. In this context, having Open Source libraries such as Freemarker, which allow the generation of code from a graphic model in a simple way and with hardly any latency, is essential.
Header image: de Lawrence Hookham at Unsplash.
✍🏻 Author(s)
