Profile File

An environment profile is serialized as a yaml file on disk.

Hint

yaml is an human readable file format that allow to define tree hierarchies as key:value pair, where the value can also be a key:value pair and so on …

Here are the rules defining an environment profile file, files not matching those rules are discarded:

• The file CAN have an abitrary name.

• The file extension MUST be .yml.

• The file MUST NOT be empty.

• The file MUST have a __magic__ root key with a value starting by kloch_profile.

The content of the file is then defined as:

➡parent	:
⬇key
identifier	required	yes
	type	str
	description	unique arbitrary chain of character used to retrieve a profile
version	required	yes
	type	str
	description	arbitrary chain of characters indicating the version of this profile
inherit	required	no
	type	str
	description	optional identifier of the profile that must be inherited
launchers	required	yes
	type	dict
	description	configuration of each launcher

Inheritance

As explained in Usage it’s possible to merge profiles “on top of each other” using the inherit key, or from the CLI.

Important

Only the content of the launchers root key is merged with the other profile.

Assuming profile-B is declaring a base: profile-A then the merge will happen as follow:

• keep all root keys of profile-B that are not the launchers key.

• merge the launchers as profile-A + profile-B

Note

We use the same term as Python to explain inheritance: specifying inherit: profileA in profileB can be described as profileA is the super profile of the sub profile profileB

Tokens

All keys in the launchers root key can make use of tokens.

A token is a specific chain of characters that is removed (resolved) in the final config, but allow to be replaced by a dynamic value or provide contextual information to the key.

Merge Tokens

Those tokens indicate how the value of the key must be merged with a “base” value of similar type.

In the context of environment profile it indicate how to merge the key/value pair of the current profile with the value of the profile specified in inherit.

A merge token:

• MUST only appear in keys

• MUST be prefixed to its key

• CAN be optional

The following merge tokens are available:

token	description
+=	indicate the key’s value should be appended to the base’s value
-=	indicate the base’s key should be removed if it exist
==	indicate the base’s value should be overriden
!=	add the key and value only if the key doesn’t exist in base
unset	no token is similar to += (append)

Here is an example making use of all of the tokens and the python API used to merge:

import json
from kloch import MergeableDict

base = MergeableDict(
    {
        "rezenv": {
            "+=config": {"quiet": True},
            "requires": {"houdini": "20", "devUtils": "2.1"},
            "environ": {"STATUS": "wip"},
            "roots": ["/d/packages"],
        }
    }
)
top = MergeableDict(
    {
        "rezenv": {
            "+=config": {"quiet": False, "debug": True},
            "requires": {"maya": "2023", "-=devUtils": "", "!=houdini": "19"},
            "==environ": {"PROD": "test"},
            "roots": ["/d/prods"],
        }
    }
)
print(json.dumps(base + top, indent=4))

{
    "rezenv": {
        "+=config": {
            "quiet": false,
            "debug": true
        },
        "requires": {
            "houdini": "20",
            "maya": "2023"
        },
        "==environ": {
            "PROD": "test"
        },
        "roots": [
            "/d/packages",
            "/d/prods"
        ]
    }
}

Launchers

Specified as key/value pair where the key is the launcher name and the value its configuration.

➡parent	:launchers
⬇key
{launcher name}	required	no
	type	dict
	description	a registred launcher name with its configuration

List of available built-in launchers (see Launcher Plugins for extending it):

• .base : An abstract launcher that whose purpose is to be merged with other launchers.

• system : A simple launcher executing the given command in the default system console.

• @python : A launcher that execute the given python file with kloch’s own interpreter.

.base

This launcher is never launched and is simply merged with other launchers defined in the profile.

➡parent	:launchers:.base
⬇key
environ	required	no
	type	Dict[str, Union[str, List[str]]]
	description	mapping of environment variable to set before starting the environment. The value can either be a regular string or a list of string. The list of string has each item joined using the system path separator [2]. All values have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$ All values are turned absolute and normalized [4] if they are existing paths.
merge_system_environ	required	no
	type	<class ‘bool’>
	description	True to implicitly merge the system environment (from the machine reading the profile) with the potentially specified environ field. The system environ is merged as “base” so any key specified in the environ will override it.
command	required	no
	type	List[str]
	description	Arbitrary list of command line arguments to call at the end of the launcher execution.
cwd	required	no
	type	<class ‘str’>
	description	Filesystem path to an existing directory to use as “current working directory”. The path will have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$. You can also use variables defined in the environ key. The path is turned absolute and normalized [4].

system

Useless without a command specified. The launcher will just set the given environment variables for the session, execute the command, then exit.

➡parent	:launchers:system
⬇key
environ	required	no
	type	Dict[str, Union[str, List[str]]]
	description	mapping of environment variable to set before starting the environment. The value can either be a regular string or a list of string. The list of string has each item joined using the system path separator [2]. All values have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$ All values are turned absolute and normalized [4] if they are existing paths.
merge_system_environ	required	no
	type	<class ‘bool’>
	description	True to implicitly merge the system environment (from the machine reading the profile) with the potentially specified environ field. The system environ is merged as “base” so any key specified in the environ will override it.
command	required	no
	type	List[str]
	description	Arbitrary list of command line arguments to call at the end of the launcher execution.
cwd	required	no
	type	<class ‘str’>
	description	Filesystem path to an existing directory to use as “current working directory”. The path will have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$. You can also use variables defined in the environ key. The path is turned absolute and normalized [4].

@python

Execute the given python file with kloch internal python interpreter.

All command keys are passed as args to the python script.

➡parent	:launchers:@python
⬇key
environ	required	no
	type	Dict[str, Union[str, List[str]]]
	description	mapping of environment variable to set before starting the environment. The value can either be a regular string or a list of string. The list of string has each item joined using the system path separator [2]. All values have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$ All values are turned absolute and normalized [4] if they are existing paths.
merge_system_environ	required	no
	type	<class ‘bool’>
	description	True to implicitly merge the system environment (from the machine reading the profile) with the potentially specified environ field. The system environ is merged as “base” so any key specified in the environ will override it.
command	required	no
	type	List[str]
	description	Arbitrary list of command line arguments passed to the python file.
cwd	required	no
	type	<class ‘str’>
	description	Filesystem path to an existing directory to use as “current working directory”. The path will have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$. You can also use variables defined in the environ key. The path is turned absolute and normalized [4].
python_file	required	yes
	type	<class ‘str’>
	description	Filesystem path to an existing python file. The path will have environment variables expanded with os.expandvars [1]. The path is turned absolute and normalized. [4]

Examples

Assuming the file structure:

./profiles/beta.yml

__magic__: kloch_profile:3
identifier: knots:echoes:beta
version: 0.3.2
launchers:
  +=rezenv:
    +=config:
      quiet: false
      package_filter:
        - excludes:
            - glob(*.dev)
    +=requires:
      houdini: 20.1
      maya: 2023
      devUtils: 1+
    environ:
      PROD_STATUS: beta
      PROD_NAME: echoes

./profiles/prod.yml

__magic__: kloch_profile:3
identifier: knots:echoes
version: 0.2.0
inherit: knots:echoes:beta
launchers:
  +=rezenv:
    +=config:
      +=package_filter:
        - excludes:
            - after(1714574770)
    +=params:
      - --verbose
    +=requires:
      houdini: 20.2
      -=devUtils: _
    +=environ:
      PROD_STATUS: prod

We execute the following command:

kloch resolve knots:echoes --profile_roots ./profiles/

__magic__: kloch_profile:3
identifier: knots:echoes
version: 0.2.0
launchers:
  +=rezenv:
    +=config:
      quiet: false
      +=package_filter:
      - excludes:
        - glob(*.dev)
      - excludes:
        - after(1714574770)
    +=params:
    - --verbose
    +=requires:
      maya: 2023
      houdini: 20.2
    +=environ:
      PROD_NAME: echoes
      PROD_STATUS: prod

---

References

[1] (1,2,3,4,5,6,7)

https://docs.python.org/3/library/os.path.html#os.path.expandvars

[2] (1,2,3)

: on UNIX, ; on Windows

[4] (1,2,3,4,5,6,7)

with https://docs.python.org/3.9/library/pathlib.html#pathlib.Path.resolve