File Format¶

The configuration files are in TOML format. Below, we document the semantics of a well-formed TOML file, and how they are validated.

Paths can include environment references using the syntax {reference}. Undefined references are replaced with an empty string.

Todo

Should variable expansion also apply to non-path values?

Top-level Elements¶

basepath

Any relative path in the configuration is resolved against this path.

locales

List of locale codes to enable for the described project. This is a list of BCP 47 locale identifiers, with the notable exception of ja-JP-mac, which we temporarily need to keep for compatibility reasons.

env

Table of environment aliases, which allows to define short-hands to be used in the configuration file. Example:

[env]
    l = "{locale}/"

build

Table of build configuration entries. This is a good location to store locale lists for special builds. Example:

[build]
    exclude-multi-locale = [
        "bn-BD",
        "zam",
    ]
    test-single-locale = [
        "it",
        "he",
        "zh-TW",
    ]

This could for example mean that from all the locales that are in a project (think Firefox for Android), some are excluded from a multi-locale build and only part of the single-locale builds. In addition, we could have a short list of locales that we always want to generate single-locale builds for to make sure they don’t break? The builds to actually use and test would be the multi-locale build, though.

The entries in build are project specific.

Includes¶

The l10n configuration files can include external configuration files to make the whole configuration system modular. This is done by adding includes tables to the file.

[[includes]]
    path = "some/file/path.toml"

The paths are resolved like this (filedir being the path to the directory of the configuration file): filedir/{basepath}/path.

The global localization scope is defined by a root configuration file, which utilizes includes to define projects. Projects themselves can refer to further configuration files, to reuse the l10n configuration of shared code fragments.

Todo

Define what makes a project. Can projects include projects?

Excludes¶

Top-level configuration files can also exclude an external configuration file. This can be used to define localization projects where one excludes the other. When excluding a configuration with includes, all child configurations of that are also excluded.

It’s an error to use an exclude in a child configuration.

If an included configuration (or the top-level configuration) explicitly includes an excluded sub-configuration file, the file is included, i.e., includes have precedence over excludes.

There’s a detailed example in Handling Excludes.

Paths¶

This is the heart of the actual configuration, describing where to find the reference files, and where to put the localized files. In defining the localization paths, use the {locale} variable to denote the locale code. Use * to denote a single level wildcard, or ** to denote all files in the given directory hierarchy.

Example:

[[paths]]
    reference = "locales/en-US/my-file.ftl"
    l10n = "locales/{locale}/my-file.ftl"

For bilingual file formats, it’s acceptable to just specify the l10n path, though it’s encouraged to define the reference to enable more tests.

To limit a particular path pattern to a subset of locales, you can specify an optional locales entry:

[[paths]]
    reference = "locales/en-US/my-file.ftl"
    l10n = "locales/{locale}/my-file.ftl"
    locales = [
        "de",
        "fr",
    ]

The list of locales for a particular path is assumed to be a subset of the top-level locales entry. Additional locales will be ignored.

The tests entry is optional and allows to specify additional tests to be run on this set of files. One example would be to run platform-specific tests, for example on Android, on a particular set of files.

[[paths]]
    reference = "mobile/android/base/locales/en-US/**"
    l10n = "{l}mobile/android/base/**"
    test = [
        "android-dtd",
    ]

Todo

Do we need a way to disable tests? Something locale specific?

Filters¶

In some cases localizations can differ from the reference locale. For example it can be fine for some files to be missing, for particular entries to be missing in a file, or to have extra entries. Filters allow the configuration to specify what’s acceptable. The filter table entries are defined as follows

path

A full file path, or a list of file paths. Wildcards (*, **) and expansions are allowed.

key

A key name, or list of key names that are ignored. If you want to specify a set of keys, you can use regular expressions by prefixing the key entry with re:. This entry is optional and, if missing, the filter will only apply to missing or additional files in the localization. If you want to ignore all entries in a file, specify a generic key, re:..

action

What to do when the filter matches. The value should be one of

error: Report the missing or obsolete string or file. This is the default behavior, but it can be useful to make decision on particular files or keys before taking more generic actions.
ignore: Don’t report the missing or obsolete string or file.
warning: Report the missing string or file but, for example, don’t take action when doing l10n-merge. This used to be called report in filter.py.

For any particular project, a string or a file of a localization has a status within that particular project, and a status within the global configuration.

As the global l10n scope and the projects scopes are defined by aggregating individual configuration files, the status of a string or file is also determined by how the statuses of the string from each configuration aggregate.

Evaluation¶

In a single configuration file, the filter is evaluated as the action of the first rule with a matching path and key pattern. Both path and key can be arrays, which is just a shorthand for a filter rule for each combination.

[[filters]]
path = [
    "browser",
    "toolkit",
]
action = "ignore"

is a shorthand for

[[filters]]
path = "browser"
action = "ignore"

[[filters]]
path = "toolkit"
action = "ignore"

If none of the path and key combinations match, there is no resulting action.

The status of a file or string in a project is ignore if any filter returns ignore.

The final status of a file or string is ignore if all project filters return ignore. If all project filters return ignore or warning, the status is warning, otherwise error. Thus, a string or file that’s not covered by any filter has an error status.

Here’s pseudo code in Python that demonstrates the algorithm.

def filter(config, path, key=None):
    """Determine the status of a file or string
    in a configuration file.
    """
    for rule in config.filter_rules:
        if not rule.path.matches(path):
            continue
        if key is None and rule.key is None:
            return rule.action
        if key is not None and rule.key is not None:
            if rule.key.matches(key):
                return rule.action

def project_status(project, path, key=None, global=True):
    """Determine the status of a file or string
    in a project with a set of configuration files.
    """
    default_action = 'error' if global else 'ignore'
    return aggregate(
        (filter(config, path, key) for config in project.configs),
        'ignore', default=default_action)

def global_status(global_scope, path, key=None):
    """Determine the status of a file or string
    in the global context.
    """
    return aggregate(
        (project_status(project, path, key) for project in global_scope.projects),
        'error', default='error')

def aggregate(actions_iterable, stop_action, default=None):
    """Helper function to aggregate actions over an
    iterable.
    """
    actions = set()
    for action in actions_iterable:
        if action == stop_action:
            return stop_action
        actions.add(action)
    for aggregated_action in ('error', 'warning', 'ignore'):
        if aggregated_action in actions:
            return aggregated_action
    return default

This evaluation of filters builds a system that allows for strings and files to be ignored for a particular project, but if any project needs the string, it needs to be translated. At the same time, files or strings not used anywhere are reported as obsolete, and can be removed.

Meta Data¶

Meta data for a project is defined in the configuration repository only, and maintained by l10n-drivers. It’s used to describe project management-related data about a project, like priorities, schedules, descriptions. Meta data can also be restricted to particular files or locales.

Example:

[[project]]
    name = "Firefox"
    description = """The mozilla Desktop User Agent"""
    [[project.import]]
        repository = "https://hg.mozilla.org/mozilla-central"
        revision = "default"
        path = "browser/locales/l10n.toml"
        vcs = "mercurial"

    [[project.metadata]]

        priority = 2

        flags = [
        ]

        [[project.metadata.schedule]]
            when = 2017-02-29T00:00:00Z
            title = "Code freeze"

        [[project.metadata.schedule]]
            when = 2017-03-29T00:00:00Z
            title = "Go-Live to production"
            notify = 2017-03-22T00:00:00Z

    [[project.metadata]]

        locales = [
            "de",
        ]
        priority = 1

    [[project.metadata]]

        locales = [
            "fr",
        ]
        path = "{l}browser/some/directory/file.properties"
        priority = 1

File Format¶

Top-level Elements¶

Includes¶

Excludes¶

Paths¶

Filters¶

Evaluation¶

Meta Data¶

L10n Project Configuration

Navigation

Related Topics