# Writing a Language Extension

`lrc` can be extended to support new languages other than plain Lua.

An example for a language extension is `lrc`s integrated [Teal support](https://codeberg.org/lrocket/lrc/src/branch/main/lrocket/lang/teal.lua).

## Setup

To add support for a new language, create a new folder containing a file `lrocket/lang/<name>.lua`.

### Packaging

> **Note** If you are defining a new language in the frame of creating a custom `otype`, you may skip the following steps and just add a file at `lrocket/lang/<name.lua>` to your `otype` directory.

If your language extension is standalone (not part of a custom `otype`), you may wish to generate a [rockspec](https://github.com/luarocks/luarocks/blob/main/docs/luarocks_write_rockspec.md) via luarocks, in order to install, test and distribute the language support:

```bash
> mkdir -p <lrocket-toolchain-my-toolchain>/lrocket/lang
> cd <lrocket-lang-my-language>
> touch lrocket/toolchain/<my-language>.lua
> luarocks write_rockspec
```

Your directory should then look like this:

```
> tree
.
├── lrocket
│   └── lang
│       └── my-language.lua
└── lrocket-toolchain-my-language-dev-1.rockspec

2 directories, 2 files
```

## Implementation

The main file of your language extension is `lrocket/lang/<my-language>.lua`.

It should export a Lua table which implements the following functions:

### Required Functions

* [`searchpath (glob, path)`](#function-searchpath-glob-path)
* [`expandpath (path)`](#function-expandpath-path)
* [`defaultpath (trees)`](#function-defaultpath-trees)

### Basic Structure

```lua
--lr:abi 1.2.0

local <my-language> = {}

-- implementation
...

return <my-language>
```

### `lrocket.ModInfo`

The `lrocket.ModInfo` structure and represents a resolved module.

It is the data type returned (yielded) by the [`searchpath`](#function-searchpath-glob-path) function.

Since new languages will usually require preprocessing or generating Lua-compatible code in some way, the module contents are \*not\* stored as static `string` but as a `function`:

```lua
-- minimal embed function example

---@type lrocket.ModInfo
local modinfo = {
	embed = function(modinfo, targetarchive, flags)
		routines.writelua(
			targetarchive,  -- output archive
			io.open(modinfo.path):read '*a',  -- module code as string
			'lua_modules/' .. modinfo.modname:gsub('%.', '/'),  -- target path inside output archive
			modinfo.chunkname,  -- display name for stacktraces
			flags:find 'g' and true  -- whether to include debug info
		)
	end,
	...
}
```

`lrc` calls the `modinfo.embed` function to read - or if needed preprocess or generate - the contents of the module (text or binary).

If a language needs specific per-module compiler flags, you can implement the `modinfo.configure (modinfo, cfg)` function, where `cfg` will be a read/writable `lrocket.CompilerConf`.

See the full list of `lrocket.ModInfo` fields below:

**Fields**

| Field | Description |
| ----- | ----------- |
| `modname` | Module name as used in `require`-statements |
| `path` | Filename of the module source code / binary on disk |
| `lang` | Module language<br/><br/>(The name of your `lrocket.lang.<name>` extension) |
| `chunkname` | A display name for the module for printing stacktraces<br/><br/>(should respect the debuginfo path setting `spec.debug` = `relative`\|`full`\|`none`) |
| `embed` | **optional:** Function writing file contents of this module to the given lrocket archive |
| `configure` | **optional:** Function for configuring this module before starting the compilation |
| `entrypoint` | **optional:** `luaopen_` C symbol (for native modules) |
| `resources` | **optional:** List of additional resource files that should be embedded with this module<br/><br/>(can be empty, as list of triples: `{scope, filename, content}`) |
| `requires` | **optional:** List of `lrocket.RequireInfo`<br/><br/>(can be empty, modules required from inside this module) |

### function `searchpath (glob, path)`

**Parameters**

* `glob` – the module name that is being looked up (may contain `*` wildcards for multiple modules)
* `path` – the expanded search path (haystack) for this language

This function is required and will be <u>invoked as a coroutine</u>.

It should loop over the search path (provided via the `path` parameter) and yield module(s) matching the given pattern on the way (`glob` parameter).

Modules should be yielded in the shape of [`lrocket.ModInfo`](#lrocket-modinfo): (2 return values) `modname`, `modinfo`.

To see how exisiting language extensions implement this function, take a look at the following examples:

* [The `lua` Language Extension](https://codeberg.org/lrocket/lrc/src/branch/main/lrocket/lang/lua.lua)
* [The `shared` (library) Language Extension](https://codeberg.org/lrocket/lrc/src/branch/main/lrocket/lang/shared.lua)
* [The `teal` Language Extension](https://codeberg.org/lrocket/lrc/src/branch/main/lrocket/lang/teal.lua)

An example `searchpath` function could look like this:

**Example**

```lua
-- example language extension 'minimalang'
local routines = require 'lrocket.routines'
local sfs = require 'sfs'
local utils = require 'lrocket.utils'

--lr:abi 1.2.0
local minimalang = {}

function minimalang.searchpath(glob, path)
	-- iterate over search path
	for subpath in path:gmatch '[^;]+' do
		local globpath = subpath:gsub('%?', (glob:gsub('%.', '/')))

		for file in sfs.glob(globpath) do
			local modname = glob  -- let's ignore possible wildcards in require statements for this minimal example

			local requires  = ...  -- optional: see 'Scanning' below
			local resources = ...  -- optional: see 'Scanning' below

			---@type lrocket.ModInfo
			local modinfo = {
				modname = modname,
				path = file,
				lang = 'minimalang',
				chunkname = modname,
				requires = requires,
				resources = resources,
				embed = function(modinfo, targetarchive, flags)
					routines.writelua(
						targetarchive,  -- output archive
						io.open(modinfo.path):read '*a',  -- module code as string
						'lua_modules/' .. modinfo.modname:gsub('%.', '/'),  -- target path inside output archive
						modinfo.chunkname,  -- display name for stacktraces
						flags:find 'g' and true  -- whether to include debug info
					)
				end
			}

			coroutine.yield(modname, modinfo)
		end
	end
end

-- see the docs for `expandpath` below
function minimalang.expandpath(path) ... end

-- see the docs for `defaultpath` below
function minimalang.defaultpath(trees) ... end

return minimalang
```

**Scanning**

In order to inform `lrc` which dependency modules are required from inside a module and where additional resource files are opened, it is possible to construct the two lists `requires` and `resources`.

Both lists are optional, they can be ignored when a language never requires dependency modules or embeds resource files.

**Optional:** Telling `lrc` to scan for dependency modules:

```lua
local Lua = require 'lrocket.lang.lua'

---@type lrocket.RequireInfo[]
local requires = {}
for modname, line, isopt in Lua.scanrequires(path, true, sourcemodname) do
	requires[#requires+1] = {
		modname = modname,
		constraint = isopt and 'opt' or 'required',
		source = path..(line ~= -1 and ":"..line or '')
	}
	requires[modname] = requires[#requires]
end
```

**Optional:** Finding additional required resource files for embedding them:

```lua
local resources = {}
for scope, line, lrpath in Lua.scanresources(path) do
	resources[#resources+1] = {
		lrpath = lrpath,
		scope = scope,
		source = path..(line ~= -1 and ":"..line or '')
	}
	resources[scope] = lrpath
end
```

### function `expandpath (path)`

**Parameters**

* `path` – the raw search path for this language, that has been configured by the user

This function is required and should expand the `??` wildcard in the raw search path for this language, as it was configured by the user.

**Example**

```lua
function minimalang.expandpath(path)
	-- expand the `??` path wildcard
	return (path:gsub('(([^;]+)%?%?)', '%1/?.exl;%/?/init.exl'))
end
```

### function `defaultpath(trees)`

**Parameters**

* `trees` – Tree roots to construct the default search paths from (e.g. `~/.luarocks`, `/usr/local`, `/usr`)

This function is required and should calculate the value for the `;;` wildcard (the default user / system path) for this language, based on the tree roots which have been configured by the user.

**Example**

```lua
function minimalang.defaultpath(trees)
	-- return the value that should be substituted for the `;;` path wildcard
	local res = {}
	for _, p in ipairs(trees) do
		res[#res+1] = p .. '/share/minimalang/??'
	end

	return table.concat(res, ';')
end
```

### ABI specification

When you are writing compiler extensions, you will likely be using parts of the compiler API that are not under semantic versioning.

It is therefor recommended to specify the LRocket ABI version by placing a comment in your file:

```lua
--lr:abi 1.2.0
```

This is technically optional, but `lrc` will throw a warning if the LRocket ABI is not specified.

## Testing the Language Extension

For testing your language extension you need:

* An input file written in the target language (e.g. `example.exl`)
* `--xpath` (at least once, e.g. `lrc --xpath "minimalang:?.expl;;"`

After installing your language extension locally you can go ahead and test:

> **Note** If you added your language extension as part of a custom `otype` or `toolchain`, [install your otype](writing-an-otype.md#testing-the-otype) or [toolchain](writing-a-toolchain.md#testing-the-toolchain) locally instead.

```bash
> luarocks make  # install the language extension
> lrc <test.exl> --xpath "minimalang:?.expl;;" -o <ouptut.xxx>
```

or in a test `.rockspec` file:

```lua
...
build = {
	type = 'lrocket',
	entrypoint = 'test.expl',
	output = '<output.xxx>',
	modpath = 'minimalang:?.expl;;'
}
```

## Publishing

> **Note** If you added your language extension as part of a custom `otype` or `toolchain`, continue reading at [Publishing (Writing an `otype`)](writing-an-otype.md#publishing) or [Publishing (Writing a `toolchain`)](writing-a-toolchain.md#publishing) respectively instead.

Once you have [tested your language extension](#testing-the-language-extension), move on to [choosing a license](https://choosealicense.com/) and follow [Publishing your code online](https://github.com/luarocks/luarocks/blob/main/docs/creating_a_rock.md#publishing-your-code-online) at the LuaRocks documentation.