13 KiB

Raw Blame History

Lazy Coding Day - Code/Text Generation

17 Feb 2022

Goals
Proof of Concept Walkthrough
Implementation Walkthrough
Seeing everything in action
Tea Break

Goals

Generating text is a common coding task, and one popular approach is to use a template engine. While code and text might seem like two different things, they're pretty much the same when it comes to creating them.

I'm playing around with a template-based solution that meets two goals:

Keep it simple for Java developers who only need a quick tutorial to understand it.
To generate multiple files from a single template file - this is a pet peeve of mine.
1. https://stackoverflow.com/questions/55025283/how-to-create-multiple-files-using-one-template-in-maven-custom-archetype-apach

Warning!

Code generation is a well trodden field. There are better alternatives than doing DIY code generation from scratch. The whole exercise I present here is really about exploring ”mentally unburdened” code generation within the confines of a single afternoon.

Serious Code Generation

I've used a few different code generation stacks in the past, and they're all good for different purposes. These are all better alternatives to what I demonstrate here:

Full blown DSL + Code generation (Antlr, JCup + JFlex)
Template engine (Apache Velocity)
Visual programming language driven code generation/VM. E.g., Scratch and Marama https://tinyurl.com/y9no2u5p
Macro (supported by any LISP like language)

Proof of Concept Walkthrough

Step 1

Intending to create something that is both lightweight and easy to use, I've decided on mixing Java and non-Java bits. By doing this, I will be able to get the full power of Java within my reach and delegate commonly occurring text output tasks to a few non-Java bits - from this point on, the non-Java bits are called generative functions.

Let’s mock-up template structures:

Template Mock-up 1 Template Mock-up 2

The first part is to create models that drive text output; the second half involves file and text related things such as file creation and text output.

Consider the following template that I created for POC:

	package lorem.ipsum.javatester.file;
	
	import java.util.UUID;
	import java.io.Serializable;
	import lorem.ipsum.javatester.domain.CodeModel;	
	import org.chulk.codegen.file.GeneratedFile;
	import org.chulk.codegen.file.FileCreator;
	
	/*
		Comment
	*/

	<<:begin>>
        <<:method generate CodeModel model>>
        <<:file x ./test {UUID.randomUUID().toString()}.txt>>	
        if (model.getName().equals("Lorem Ipsum")) {
            <<:out x Hello, Lorem Ipsum!\n>>
            <<:out x ---{UUID.randomUUID().toString()}---\n>>
        } else {
            <<:file y ./ something-else-{UUID.randomUUID().toString()}.txt>>
            <<:out y Hello, {model.getName().toUpperCase()}!\n>>

        }
        <<:end>>
	<<:end>>

<<:function-symbol arg0 arg1 … argN>> expressions that appear in the above template are generative functions. Each generative function is mapped into Java code when a template is loaded for code generation.

What the template does here is self-explanatory:

Creates a file at ./text whose name is a concatenation of a freshly generated UUID and ".txt".
If the name of the model object is "Lorem Ipsum" then writes the literal string and evaluated values as specified in Line X - Y.
If the name of the model object is not "Lorem Ipsum" then
1. Creates a file at ./ whose name is a concatenation of "something-else", a freshly generated UUID and ".txt".
2. Writes "Hello," and an uppercased name of the model object.

Step 2

The next step, which happens during run-time, is to load a template file. The loaded template file is turned into Java code by mapping generative functions into Java code.

Template loading

The following Java code has been generated based on the template from the previous step.

package lorem.ipsum.javatester.file;

import java.util.UUID;
import java.io.Serializable;
import lorem.ipsum.javatester.domain.CodeModel;
import org.chulk.codegen.file.GeneratedFile;
import org.chulk.codegen.file.FileCreator;

/* 
	Comment 
*/

public class Generator {
	public void generate(CodeModel model) {

		final GeneratedFile x = FileCreator.create("./test", "" + UUID.randomUUID().toString() + ".txt");
		if (model.getName().equals("Lorem Ipsum")) {
			x.out("Hello, Lorem Ipsum!\n");
			x.out("---" + UUID.randomUUID().toString() + "---\n");
		} else {
			final GeneratedFile y = FileCreator.create("./", "something-else-" + UUID.randomUUID().toString() + ".txt");
			y.out("Hello, " + model.getName().toUpperCase() + "!\n");

		}
	}
}

Step 3

The user feeds a domain model to the template-driven Java code and generates text output.

final CodeModel model = new CodeModel();
model.setName("Lorem Ipsum");

final CodeGenerator codeGen = CodeGenerator.create(Charsets.UTF_8);
codeGen.generateCode(codeGen.writeGeneratorCode(Resources.getResource("test-template.txt").getPath()), model);

Text Output (./test/6c6d0e84-7c1a-4ad4-8d3d-f28c713a6dee.txt)

Hello, Lorem Ipsum!
---e9eb85c7-547e-417b-a34b-dbe8964d9cec---

final CodeModel model = new CodeModel();
model.setName("Not Lorem Ipsum");

final CodeGenerator codeGen = CodeGenerator.create(Charsets.UTF_8);
codeGen.generateCode(codeGen.writeGeneratorCode(Resources.getResource("test-template.txt").getPath()), model);

Text Output (./something-else-8ed3c1fa-43b8-4cd3-8072-ea36c7327939.txt)

Hello, NOT LOREM IPSUM!

Implementation Walkthrough

Processing Template

Each line of a template file can only be either Java code or generative function.

Template Line := Java Code | Generative-Function

The template loader can easily detect a generative function line by looking for generative function start and end markers; “<<:” and “>>”. When the template loader gets a generative function line then the loader invokes a corresponding Java code mapper for the function symbol of the generative function.

Generative Functions:

begin
end
file
- creates a new file.
method
- a unary generative method that takes a model as an argument; a template needs to have one generative method.
out (a workhorse function that directs the text to file)

When we're done with this simple process, there is Java code that does what the generative functions intend to do.

A Little Enhancement

The initial design was limited so that a single generative function could not take over more than one line. After a few tries, it became apparent the limitation had to go - so changes were made!

out - The workhorse function has been modified so that it can spread over multiple lines as shown below:

		<<:out x 
			Line 1 \n
			Line 2 \n
			Line 3 \n
		>>

Not

		<<:out x Line 1 \n>>
		<<:out x Line 2 \n>>
		<<:out x Line 3 \n>>

So the line detection - primitive lexer - of the template loader had to change to deal with the multi-line out function. A simple solution was when a “<<:out” line does not end with “>>” then “delayed evaluation” kicks in until we have the whole out file function.

Code Generation - Text Output

At the end of the line detection and code mapping, we get a String value which is Java code. We hand over a collection of String values to a reflection facility, turning the text input into live Java code at runtime - the whole process has been completed!

FYI: I am using jOOR to build Java code during runtime: https://www.jooq.org/products/jOOR/javadoc/latest/org.jooq.joor/org/joor/Reflect.html

Seeing everything in action

Let's create a template so that you can personalise your greeting message for each person.

Goal - Timezone aware greeting text

Given a list of people, for each one of them we want to generate a timezone appropriate greeting text output.

Assuming it is 3:30 pm CEST and we want to generate a greeting for someone in NY, we would print out “Good morning, Mary.” If the person lives in London, we would print out “Good afternoon, Mary.” And if the person lives in Perth, we would print out “Good evening, Mary.” This can be done by using a simple function that takes in the current time and the person's location and outputs the appropriate greeting.

By doing this, we can make sure that everyone gets the appropriate greetings no matter what time it is where they are.

Domain Class - Person

Name
ZoneID

https://docs.oracle.com/middleware/12212/wcs/tag-ref/MISC/TimeZones.html

public class Person {

	private final String name;

	private final ZoneId zoneId;
	
	public Person(String name, ZoneId zoneId) {
		this.name = name;
		this.zoneId = zoneId;
	}

	public String getName() {
		return name;
	}

	public ZoneId getZoneId() {
		return zoneId;
	}

}

Now that I have a domain class to feed to a template. Let’s sketch out a template.

Firstly, we need to create a list of "Person" objects. The list is fed to CodeGenerator, and the ZoneId of the Person object is used for each Person to find out what greeting to use.

<<:begin>>
	<<:method generate List<Person> model>>
	for (Person person : model) {	
		ZonedDateTime zonedTime = Instant.now().atZone(person.getZoneId());	    
		int hour = zonedTime.getHour();
		String greeting = "";
		
		if (hour >=0 && hour < 5) {
			greeting = "Good night";
		} else if (hour >= 5 && hour <= 12 ) {
			greeting = "Good morning";
		} else if (hour > 12 && hour <= 17) {
			greeting = "Good afternoon";
		} else if (hour > 17 && hour <= 21) {
			greeting = "Good evening";
		} else if (hour > 21) {
			greeting = "Good night";
		}
	}
       <<:end>>
<<:end>>

Secondly, a new file with the text content is created for each Person.

<<:begin>>
	<<:method generate List<Person> model>>
	for (Person person : model) {
		<<:file x ./test greeting-{UUID.randomUUID()}-{person.getName()}.txt>>
		
		ZonedDateTime zonedTime = Instant.now().atZone(person.getZoneId());	    
		int hour = zonedTime.getHour();
		String greeting = "";
		
		if (hour >=0 && hour < 5) {
			greeting = "Good night";
		} else if (hour >= 5 && hour <= 12 ) {
			greeting = "Good morning";
		} else if (hour > 12 && hour <= 17) {
			greeting = "Good afternoon";
		} else if (hour > 17 && hour <= 21) {
			greeting = "Good evening";
		} else if (hour > 21) {
			greeting = "Good night";
		}		
	}
       <<:end>>
<<:end>>

Lastly, we append the appropriate greeting for each Person to the Person's name.

<<:out x {greeting}, {person.getName()}.\n>>

After filling out import statements and few extra generative functions, We now have a complete template to achieve the goal.

package lorem.ipsum.javatester.test;
	
import java.lang.reflect.InvocationTargetException;
import java.time.Instant;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
import org.chulk.codegen.CodeGenerator;
import org.chulk.codegen.file.GeneratedFile;
import org.chulk.codegen.file.FileCreator;
import lorem.ipsum.javatester.domain.Person;
	
<<:begin>>
<<:method generate List<Person> model>>
for (Person person : model) {
	<<:file x ./test greeting-{UUID.randomUUID()}-{person.getName()}.txt>>
	
	ZonedDateTime zonedTime = Instant.now().atZone(person.getZoneId());		    
	int hour = zonedTime.getHour();
	String greeting = "";
	
	if (hour >=0 && hour < 5) {
		greeting = "Good night";
	} else if (hour >= 5 && hour <= 12 ) {
		greeting = "Good morning";
	} else if (hour > 12 && hour <= 17) {
		greeting = "Good afternoon";
	} else if (hour > 17 && hour <= 21) {
		greeting = "Good evening";
	} else if (hour > 21) {
		greeting = "Good night";
	}
	
	<<:out x {greeting}, {person.getName()}.\n>>	
}
<<:end>>
<<:end>>

Let’s try out the template as shown below:

Person mary = new Person("Mary", ZoneId.of("America/New_York"));
Person bob = new Person("Bob", ZoneId.of("Pacific/Auckland"));

List<Person> ppl = new ArrayList<>();
ppl.add(mary);
ppl.add(bob);		
		
final CodeGenerator codeGen = CodeGenerator.create(Charsets.UTF_8);				
codeGen.generateCode("lorem.ipsum.javatester.test.Generator", codeGen.writeGeneratorCode(Resources.getResource("greeting-template.txt").getPath()), ppl);

We got new files!

The test code was run at 11:26 on Sunday, 3 April 2022 (CEST). And indeed the text outputs did look good.

Mary

Good afternoon, Mary.

Bob

Good morning, Bob.

Tea Break

If you'll excuse me, I'm off for a cuppa. Cheers!

13 KiB Raw Blame History