Writing a Language Extension

lrc can be extended to support new languages other than plain Lua.

An example for a language extension is lrcs integrated Teal support.

Setup

To add support for a new language, create a new folder containing a file lrocket/lang/<name>.lua.

Packaging

Note If you are defining a new language in the frame of creating a custom otype, you may skip the following steps and just add a file at lrocket/lang/<name.lua> to your otype directory.

If your language extension is standalone (not part of a custom otype), you may wish to generate a rockspec via luarocks, in order to install, test and distribute the language support:

> mkdir -p <lrocket-toolchain-my-toolchain>/lrocket/lang
> cd <lrocket-lang-my-language>
> touch lrocket/toolchain/<my-language>.lua
> luarocks write_rockspec

Your directory should then look like this:

> tree
.
├── lrocket
│   └── lang
│       └── my-language.lua
└── lrocket-toolchain-my-language-dev-1.rockspec

2 directories, 2 files

Implementation

The main file of your language extension is lrocket/lang/<my-language>.lua.

It should export a Lua table which implements the following functions:

Basic Structure

--lr:abi 1.2.0

local <my-language> = {}

-- implementation
...

return <my-language>

`lrocket.ModInfo`

The lrocket.ModInfo structure and represents a resolved module.

It is the data type returned (yielded) by the searchpath function.

Since new languages will usually require preprocessing or generating Lua-compatible code in some way, the module contents are *not* stored as static string but as a function:

-- minimal embed function example

---@type lrocket.ModInfo
local modinfo = {
	embed = function(modinfo, targetarchive, flags)
		routines.writelua(
			targetarchive,  -- output archive
			io.open(modinfo.path):read '*a',  -- module code as string
			'lua_modules/' .. modinfo.modname:gsub('%.', '/'),  -- target path inside output archive
			modinfo.chunkname,  -- display name for stacktraces
			flags:find 'g' and true  -- whether to include debug info
		)
	end,
	...
}

lrc calls the modinfo.embed function to read - or if needed preprocess or generate - the contents of the module (text or binary).

If a language needs specific per-module compiler flags, you can implement the modinfo.configure (modinfo, cfg) function, where cfg will be a read/writable lrocket.CompilerConf.

See the full list of lrocket.ModInfo fields below:

Fields

Field	Description
`modname`	Module name as used in `require`-statements
`path`	Filename of the module source code / binary on disk
`lang`	Module language (The name of your `lrocket.lang.<name>` extension)
`chunkname`	A display name for the module for printing stacktraces (should respect the debuginfo path setting `spec.debug` = `relative`\|`full`\|`none`)
`embed`	optional: Function writing file contents of this module to the given lrocket archive
`configure`	optional: Function for configuring this module before starting the compilation
`entrypoint`	optional: `luaopen_` C symbol (for native modules)
`resources`	optional: List of additional resource files that should be embedded with this module (can be empty, as list of triples: `{scope, filename, content}`)
`requires`	optional: List of `lrocket.RequireInfo` (can be empty, modules required from inside this module)

function `searchpath (glob, path)`

Parameters

glob – the module name that is being looked up (may contain * wildcards for multiple modules)
path – the expanded search path (haystack) for this language

This function is required and will be invoked as a coroutine.

It should loop over the search path (provided via the path parameter) and yield module(s) matching the given pattern on the way (glob parameter).

Modules should be yielded in the shape of lrocket.ModInfo: (2 return values) modname, modinfo.

To see how exisiting language extensions implement this function, take a look at the following examples:

An example searchpath function could look like this:

Example

-- example language extension 'minimalang'
local routines = require 'lrocket.routines'
local sfs = require 'sfs'
local utils = require 'lrocket.utils'

--lr:abi 1.2.0
local minimalang = {}

function minimalang.searchpath(glob, path)
	-- iterate over search path
	for subpath in path:gmatch '[^;]+' do
		local globpath = subpath:gsub('%?', (glob:gsub('%.', '/')))

		for file in sfs.glob(globpath) do
			local modname = glob  -- let's ignore possible wildcards in require statements for this minimal example

			local requires  = ...  -- optional: see 'Scanning' below
			local resources = ...  -- optional: see 'Scanning' below

			---@type lrocket.ModInfo
			local modinfo = {
				modname = modname,
				path = file,
				lang = 'minimalang',
				chunkname = modname,
				requires = requires,
				resources = resources,
				embed = function(modinfo, targetarchive, flags)
					routines.writelua(
						targetarchive,  -- output archive
						io.open(modinfo.path):read '*a',  -- module code as string
						'lua_modules/' .. modinfo.modname:gsub('%.', '/'),  -- target path inside output archive
						modinfo.chunkname,  -- display name for stacktraces
						flags:find 'g' and true  -- whether to include debug info
					)
				end
			}

			coroutine.yield(modname, modinfo)
		end
	end
end

-- see the docs for `expandpath` below
function minimalang.expandpath(path) ... end

-- see the docs for `defaultpath` below
function minimalang.defaultpath(trees) ... end

return minimalang

Scanning

In order to inform lrc which dependency modules are required from inside a module and where additional resource files are opened, it is possible to construct the two lists requires and resources.

Both lists are optional, they can be ignored when a language never requires dependency modules or embeds resource files.

Optional: Telling lrc to scan for dependency modules:

local Lua = require 'lrocket.lang.lua'

---@type lrocket.RequireInfo[]
local requires = {}
for modname, line, isopt in Lua.scanrequires(path, true, sourcemodname) do
	requires[#requires+1] = {
		modname = modname,
		constraint = isopt and 'opt' or 'required',
		source = path..(line ~= -1 and ":"..line or '')
	}
	requires[modname] = requires[#requires]
end

Optional: Finding additional required resource files for embedding them:

local resources = {}
for scope, line, lrpath in Lua.scanresources(path) do
	resources[#resources+1] = {
		lrpath = lrpath,
		scope = scope,
		source = path..(line ~= -1 and ":"..line or '')
	}
	resources[scope] = lrpath
end

function `expandpath (path)`

Parameters

path – the raw search path for this language, that has been configured by the user

This function is required and should expand the ?? wildcard in the raw search path for this language, as it was configured by the user.

Example

function minimalang.expandpath(path)
	-- expand the `??` path wildcard
	return (path:gsub('(([^;]+)%?%?)', '%1/?.exl;%/?/init.exl'))
end

function `defaultpath(trees)`

Parameters

trees – Tree roots to construct the default search paths from (e.g. ~/.luarocks, /usr/local, /usr)

This function is required and should calculate the value for the ;; wildcard (the default user / system path) for this language, based on the tree roots which have been configured by the user.

Example

function minimalang.defaultpath(trees)
	-- return the value that should be substituted for the `;;` path wildcard
	local res = {}
	for _, p in ipairs(trees) do
		res[#res+1] = p .. '/share/minimalang/??'
	end

	return table.concat(res, ';')
end

ABI specification

When you are writing compiler extensions, you will likely be using parts of the compiler API that are not under semantic versioning.

It is therefor recommended to specify the LRocket ABI version by placing a comment in your file:

--lr:abi 1.2.0

This is technically optional, but lrc will throw a warning if the LRocket ABI is not specified.

Testing the Language Extension

For testing your language extension you need:

An input file written in the target language (e.g. example.exl)
--xpath (at least once, e.g. lrc --xpath "minimalang:?.expl;;"

After installing your language extension locally you can go ahead and test:

Note If you added your language extension as part of a custom otype or toolchain, install your otype or toolchain locally instead.

> luarocks make  # install the language extension
> lrc <test.exl> --xpath "minimalang:?.expl;;" -o <ouptut.xxx>

or in a test .rockspec file:

...
build = {
	type = 'lrocket',
	entrypoint = 'test.expl',
	output = '<output.xxx>',
	modpath = 'minimalang:?.expl;;'
}

Publishing

Note If you added your language extension as part of a custom otype or toolchain, continue reading at Publishing (Writing an otype) or Publishing (Writing a toolchain) respectively instead.

Once you have tested your language extension, move on to choosing a license and follow Publishing your code online at the LuaRocks documentation.

Writing a Language Extension

Setup

Packaging

Implementation

Required Functions

Basic Structure

lrocket.ModInfo

function searchpath (glob, path)

function expandpath (path)

function defaultpath(trees)

ABI specification

Testing the Language Extension

Publishing

`lrocket.ModInfo`

function `searchpath (glob, path)`

function `expandpath (path)`

function `defaultpath(trees)`