Creating an Extension

An extension is a reusable class that can define custom tags, filters, functions, providers, etc.

We create extensions when we want to reuse our Latte customizations in different projects or share them with others. It is also useful to create an extension for each web project that will contain all the specific tags and filters you want to use in the project templates.

Extension Class

Extension is a class inheriting from Latte\Extension. It is registered with Latte using addExtension() (or via configuration file):

$latte = new Latte\Engine;
$latte->addExtension(new MyLatteExtension);

If you register multiple extensions and they define identically named tags, filters, or functions, the last added extension wins. This also implies that your extensions can override native tags/filters/functions.

Whenever you make a change to a class and auto-refresh is not turned off, Latte will automatically recompile your templates.

A class can implement any of the following methods:

abstract class Extension
{
	/**
	 * Initializes before template is compiler.
	 */
	public function beforeCompile(Engine $engine): void;

	/**
	 * Returns a list of parsers for Latte tags.
	 * @return array<string, callable>
	 */
	public function getTags(): array;

	/**
	 * Returns a list of compiler passes.
	 * @return array<string, callable>
	 */
	public function getPasses(): array;

	/**
	 * Returns a list of |filters.
	 * @return array<string, callable>
	 */
	public function getFilters(): array;

	/**
	 * Returns a list of functions used in templates.
	 * @return array<string, callable>
	 */
	public function getFunctions(): array;

	/**
	 * Returns a list of providers.
	 * @return array<mixed>
	 */
	public function getProviders(): array;

	/**
	 * Returns a value to distinguish multiple versions of the template.
	 */
	public function getCacheKey(Engine $engine): mixed;

	/**
	 * Initializes before template is rendered.
	 */
	public function beforeRender(Template $template): void;
}

For an idea of what the extension looks like, take a look at the built-in CoreExtension.

beforeCompile(Latte\Engine $engine): void

Called before the template is compiled. The method can be used for compilation-related initializations, for example.

getTags(): array

Called when the template is compiled. Returns an associative array tag name ⇒ callable, which are tag parsing functions.

public function getTags(): array
{
	return [
		'foo' => [FooNode::class, 'create'],
		'bar' => [BarNode::class, 'create'],
		'n:baz' => [NBazNode::class, 'create'],
		// ...
	];
}

The n:baz tag represents a pure n:attribute, i.e. it is a tag that can only be written as an attribute.

In the case of the foo and bar tags, Latte will automatically recognize whether they are pairs, and if so, they can be written automatically using n:attributes, including variants with the n:inner-foo and n:tag-foo prefixes.

The order of execution of such n:attributes is determined by their order in the array returned by getTags(). Thus, n:foo is always executed before n:bar, even if the attributes are listed in reverse order in the HTML tag as <div n:bar="..." n:foo="...">.

If you need to determine the order of n:attributes across multiple extensions, use the order() helper method, where the before xor after parameter determines which tags are ordered before or after the tag.

public function getTags(): array
{
	return [
		'foo' => self::order([FooNode::class, 'create'], before: 'bar')]
		'bar' => self::order([BarNode::class, 'create'], after: ['block', 'snippet'])]
	];
}

getPasses(): array

It is called when the template is compiled. Returns an associative array name pass ⇒ callable, which are functions representing so-called compiler passes that traverse and modify the AST.

Again, the order() helper method can be used. The value of the before or after parameters can be * with the meaning before/after all.

public function getPasses(): array
{
	return [
		'optimize' => [Passes::class, 'optimizePass'],
		'sandbox' => self::order([$this, 'sandboxPass'], before: '*'),
		// ...
	];
}

beforeRender(Latte\Engine $engine): void

It is called before each template rendering. The method can be used, for example, to initialize variables used during rendering.

getFilters(): array

It is called before the template is rendered. Returns filters as an associative array filter name ⇒ callable.

public function getFilters(): array
{
	return [
		'batch' => [$this, 'batchFilter'],
		'trim' => [$this, 'trimFilter'],
		// ...
	];
}

getFunctions(): array

It is called before the template is rendered. Returns functions as an associative array function name ⇒ callable.

public function getFunctions(): array
{
	return [
		'clamp' => [$this, 'clampFunction'],
		'divisibleBy' => [$this, 'divisibleByFunction'],
		// ...
	];
}

getProviders(): array

It is called before the template is rendered. Returns an array of providers, which are usually objects that use tags at runtime. They are accessed via $this->global->....

public function getProviders(): array
{
	return [
		'myFoo' => $this->foo,
		'myBar' => $this->bar,
		// ...
	];
}

getCacheKey(Latte\Engine $engine): mixed

It is called before the template is rendered. The return value becomes part of the key whose hash is contained in the name of the compiled template file. Thus, for different return values, Latte will generate different cache files.

How Does Latte Work?

To understand how to define custom tags or compiler passes, it is essential to understand how Latte works under the hood.

Template compilation in Latte simplistically works like this:

• First, the lexer tokenizes the template source code into small pieces (tokens) for easier processing
• Then, the parser converts the stream of tokens into a meaningful tree of nodes (the Abstract Syntax Tree, AST)
• Finally, the compiler generates a PHP class from the AST that renders the template and caches it.

Actually, the compilation is a bit more complicated. Latte has two lexers and parsers: one for the HTML template and one for the PHP-like code inside the tags. Also, the parsing doesn't run after tokenization, but the lexer and parser run in parallel in two “threads” and coordinate. It's rocket science :-)

Furthermore, all tags have their own parsing routines. When the parser encounters a tag, it calls its parsing function (it returns Extension::getTags()). Their job is to parse the tag arguments and, in the case of paired tags, the inner content. It returns a node that becomes part of the AST. See Tag parsing function for details.

When the parser finishes its work, we have a complete AST representing the template. The root node is Latte\Compiler\Nodes\TemplateNode. The individual nodes inside the tree then represent not only the tags, but also the HTML elements, their attributes, any expressions used inside the tags, etc.

After this, the so-called Compiler passes come into play, which are functions (returned by Extension::getPasses()) that modify the AST.

The whole process, from loading the template content, through parsing, to generating the resulting file, can be sequenced with this code, which you can experiment with and dump the intermediate results:

$latte = new Latte\Engine;
$source = $latte->getLoader()->getContent($file);
$ast = $latte->parse($source);
$latte->applyPasses($ast);
$code = $latte->generate($ast, $file);

Example of AST

To get a better idea of the AST, we add a sample. This is the source template:

{foreach $category->getItems() as $item}
	<li>{$item->name|upper}</li>
	{else}
	no items found
{/foreach}

And this is its representation in the form of AST:

Latte\Compiler\Nodes\TemplateNode(
   Latte\Compiler\Nodes\FragmentNode(
      - Latte\Essential\Nodes\ForeachNode(
           expression: Latte\Compiler\Nodes\Php\Expression\MethodCallNode(
              object: Latte\Compiler\Nodes\Php\Expression\VariableNode('$category')
              name: Latte\Compiler\Nodes\Php\IdentifierNode('getItems')
           )
           value: Latte\Compiler\Nodes\Php\Expression\VariableNode('$item')
           content: Latte\Compiler\Nodes\FragmentNode(
              - Latte\Compiler\Nodes\TextNode('  ')
              - Latte\Compiler\Nodes\Html\ElementNode('li')(
                   content: Latte\Essential\Nodes\PrintNode(
                      expression: Latte\Compiler\Nodes\Php\Expression\PropertyFetchNode(
                         object: Latte\Compiler\Nodes\Php\Expression\VariableNode('$item')
                         name: Latte\Compiler\Nodes\Php\IdentifierNode('name')
                      )
                      modifier: Latte\Compiler\Nodes\Php\ModifierNode(
                         filters:
                            - Latte\Compiler\Nodes\Php\FilterNode('upper')
                      )
                   )
                )
            )
            else: Latte\Compiler\Nodes\FragmentNode(
               - Latte\Compiler\Nodes\TextNode('no items found')
            )
        )
   )
)

Custom Tags

Three steps are needed to define a new tag:

• defining tag parsing function (responsible for parsing the tag into a node)
• creating a node class (responsible for generating PHP code and AST traversing)
• registering the tag using Extension::getTags()

Tag Parsing Function

Parsing of tags is handled by its parsing function (the one returned by Extension::getTags()). Its job is to parse and check any arguments inside the tag (it uses TagParser to do this). Furthermore, if the tag is a pair, it will ask TemplateParser to parse and return the inner content. The function creates and returns a node, which is usually a child of Latte\Compiler\Nodes\StatementNode, and this becomes part of the AST.

We create a class for each node, which we'll do now, and elegantly place the parsing function into it as a static factory. As an example, let's try creating the familiar {foreach} tag:

use Latte\Compiler\Nodes\StatementNode;

class ForeachNode extends StatementNode
{
	// a parsing function that just creates a node for now
	public static function create(Latte\Compiler\Tag $tag): self
	{
		$node = $tag->node = new self;
		return $node;
	}

	public function print(Latte\Compiler\PrintContext $context): string
	{
		// code will be added later
	}

	public function &getIterator(): \Generator
	{
		// code will be added later
	}
}

The parsing function create() is passed an object Latte\Compiler\Tag, which carries basic information about the tag (whether it is a classic tag or n:attribute, what line it is on, etc.) and mainly accesses the Latte\Compiler\TagParser in $tag->parser.

If the tag must have arguments, check for their existence by calling $tag->expectArguments(). The methods of the $tag->parser object are available for parsing them:

• parseExpression(): ExpressionNode for a PHP-like expression (e.g. 10 + 3)
• parseUnquotedStringOrExpression(): ExpressionNode for an expression or unquoted-string
• parseArguments(): ArrayNode content of the array (e.g. 10, true, foo => bar)
• parseModifier(): ModifierNode for a modifier (e.g. |upper|truncate:10)
• parseType(): expressionNode for typehint (e.g. int|string or Foo\Bar[])

and a low-level Latte\Compiler\TokenStream operating directly with tokens:

• $tag->parser->stream->consume(...): Token
• $tag->parser->stream->tryConsume(...): ?Token

Latte extends the PHP syntax in small ways, for example by adding modifiers, shortened ternary operators, or allowing simple alphanumeric strings to be written without quotes. This is why we use the term PHP-like instead of PHP. Thus, the parseExpression() method parses foo as 'foo', for example. In addition, unquoted-string is a special case of a string that also does not need to be quoted, but at the same time does not need to be alphanumeric. For example, it is the path to a file in the {include ../file.latte} tag. The parseUnquotedStringOrExpression() method is used to parse it.

Studying the node classes that are part of Latte is the best way to learn all the nitty-gritty details of the parsing process.

Let's go back to the {foreach} tag. In it, we expect arguments of the form expression + 'as' + second expression, which we parse as follows:

use Latte\Compiler\Nodes\StatementNode;
use Latte\Compiler\Nodes\Php\ExpressionNode;
use Latte\Compiler\Nodes\AreaNode;

class ForeachNode extends StatementNode
{
	public ExpressionNode $expression;
	public ExpressionNode $value;

	public static function create(Latte\Compiler\Tag $tag): self
	{
		$tag->expectArguments();
		$node = $tag->node = new self;
		$node->expression = $tag->parser->parseExpression();
		$tag->parser->stream->consume('as');
		$node->value = $parser->parseExpression();
		return $node;
	}
}

The expressions we have written into the variables $expression and $value represent subnodes.

Define variables with subnodes as public so that they can be modified in further processing steps if necessary. It is also necessary to make them available for traversing.

For paired tags, like ours, the method must also let TemplateParser parse the inner contents of the tag. This is handled by yield, which returns a pair [inner content, end tag]. We store the inner content in the $node->content variable.

public AreaNode $content;

public static function create(Latte\Compiler\Tag $tag): \Generator
{
	// ...
	[$node->content, $endTag] = yield;
	return $node;
}

The yield keyword causes the create() method to terminate, returning control back to the TemplateParser, which continues parsing the content until it hits the end tag. It then passes control back to create(), which continues from where it left off. Using the yield, method automatically returns Generator.

You can also pass an array of tag names to yield for which you want to stop parsing if they occur before the end tag. This helps us implement the {foreach}...{else}...{/foreach} construct. If {else} occurs, we parse the content after it into $node->elseContent:

public AreaNode $content;
public ?AreaNode $elseContent = null;

public static function create(Latte\Compiler\Tag $tag): \Generator
{
	// ...
	[$node->content, $nextTag] = yield ['else'];
	if ($nextTag?->name === 'else') {
		[$node->elseContent] = yield;
	}

	return $node;
}

Returning node completes the tag parsing.

Generating PHP Code

Each node must implement the print() method. Returns PHP code that renders the given part of the template (runtime code). It is passed an object Latte\Compiler\PrintContext as a parameter, which has a useful format() method that simplifies the assembly of the resulting code.

The format(string $mask, ...$args) method accepts the following placeholders in the mask:

• %node prints Node
• %dump exports the value to PHP
• %raw inserts the text directly without any transformation
• %args prints ArrayNode as arguments to the function call
• %line prints a comment with a line number
• %escape(...) escapes the content
• %modify(...) applies a modifier
• %modifyContent(...) applies a modifier to blocks

Our print() function might look like this (we neglect the else branch for simplicity):

public function print(Latte\Compiler\PrintContext $context): string
{
	return $context->format(
		<<<'XX'
			foreach (%node as %node) %line {
				%node
			}

			XX,
		$this->expression,
		$this->value,
		$this->position,
		$this->content,
	);
}

The $this->position variable is already defined by the Latte\Compiler\Node class and is set by the parser. It contains an Latte\Compiler\Position object with the position of the tag in the source code in the form of a row and column number.

Runtime code may use auxiliary variables. To avoid collision with variables used by the template itself, it is convention to prefix them with $ʟ__ characters.

It can also use arbitrary values at runtime, which are passed to the template in the form of providers using the Extension::getProviders() method. It accesses them using $this->global->....

AST Traversing

In order to traverse the AST tree in depth, it is necessary to implement the getIterator() method. This will provide access to subnodes:

public function &getIterator(): \Generator
{
	yield $this->expression;
	yield $this->value;
	yield $this->content;
	if ($this->elseContent) {
		yield $this->elseContent;
	}
}

Note that getIterator() returns a reference. This is what allows node visitors to replace individual nodes with other nodes.

If a node has subnodes, it is necessary to implement this method and make all subnodes available. Otherwise, a security hole could be created. For example, sandbox mode would not be able to control subnodes and ensure that unallowed constructs are not called in them.

Since the yield keyword must be present in the method body even if it has no child nodes, write it as follows:

public function &getIterator(): \Generator
{
	if (false) {
		yield;
	}
}

AuxiliaryNode

If you are creating a new tag for Latte, it is advisable to create a dedicated node class for it, which will represent it in the AST tree (see the ForeachNode class in the example above). In some cases, you might find the trivial helper node class AuxiliaryNode useful, which allows you to pass the body of the print() method and the list of nodes made accessible by the getIterator() method as constructor parameters:

// Latte\Compiler\Nodes\Php\Expression\AuxiliaryNode
// or Latte\Compiler\Nodes\AuxiliaryNode

$node = new AuxiliaryNode(
	// body of the print() method:
	fn(PrintContext $context, $argNode) => $context->format('myFunc(%node)', $argNode),
	// nodes accessed via getIterator() and also passed into the print() method:
	[$argNode],
);

Compiler Passes

Compiler Passes are functions that modify ASTs or collect information in them. They are returned by the Extension::getPasses() method.

Node Traverser

The most common way to work with the AST is by using a Latte\Compiler\NodeTraverser:

use Latte\Compiler\Node;
use Latte\Compiler\NodeTraverser;

$ast = (new NodeTraverser)->traverse(
	$ast,
	enter: fn(Node $node) => ...,
	leave: fn(Node $node) => ...,
);

The enter function (ie. visitor) is called when a node is first encountered, before its subnodes are processed. The leave function is called after all subnodes have been visited. A common pattern is that enter is used to collect some information and then leave performs modifications based on that. At the time when leave is called, all the code inside the node will have already been visited and necessary information collected.

How to modify AST? The easiest way is to simply change the properties of the nodes. The second way is to replace the node entirely by returning a new node. Example: the following code will change all integers in the AST to strings (e.g. 42 will be changed to '42').

use Latte\Compiler\Nodes\Php;

$ast = (new NodeTraverser)->traverse(
	$ast,
	leave: function (Node $node) {
		if ($node instanceof Php\Scalar\IntegerNode) {
            return new Php\Scalar\StringNode((string) $node->value);
        }
	},
);

An AST can easily contain thousands of nodes, and traversing over all of them may be slow. In some cases, it is possible to avoid a full traversal.

If you are looking for all Html\ElementNode in a tree, you know that once you've seen Php\ExpressionNode, there is no point in also checking all it's child nodes, because HTML cannot be inside in expressions. In this case, you can instruct the traverser to not recurse into the class node:

$ast = (new NodeTraverser)->traverse(
	$ast,
	enter: function (Node $node) {
		if ($node instanceof Php\ExpressionNode) {
			return NodeTraverser::DontTraverseChildren;
        }
        // ...
	},
);

If you are only looking for one specific node, it is also possible to abort the traversal entirely after finding it.

$ast = (new NodeTraverser)->traverse(
	$ast,
	enter: function (Node $node) {
		if ($node instanceof Nodes\ParametersNode) {
			return NodeTraverser::StopTraversal;
        }
        // ...
	},
);

Node Helpers

Class Latte\Compiler\NodeHelpers provides some methods which can find AST nodes that either satisfy a certain callback etc. A couple of examples are shown:

use Latte\Compiler\NodeHelpers;

// finds all HTML element nodes
$elements = NodeHelpers::find($ast, fn(Node $node) => $node instanceof Nodes\Html\ElementNode);

// finds first text node
$text = NodeHelpers::findFirst($ast, fn(Node $node) => $node instanceof Nodes\TextNode);

// converts PHP value node to real value
$value = NodeHelpers::toValue($node);

// converts static textual node to string
$text = NodeHelpers::toText($node);