Profile File

An environment profile is serialized as a yaml file on disk.

Hint

yaml is an human readable file format that allow to define tree hierarchies as key:value pair, where the value can also be a key:value pair and so on …

Here are the rules defining an environment profile file, files not matching those rules are discarded:

• The file CAN have an abitrary name.

• The file extension MUST be .yml.

• The file MUST NOT be empty.

• The file MUST have a __magic__ root key with a value starting by kloch_profile.

The content of the file is then defined as:

➡parent	:
⬇key
identifier	required	yes
	type	str
	description	unique arbitrary chain of character used to retrieve a profile
version	required	yes
	type	str
	description	arbitrary chain of characters indicating the version of this profile
inherit	required	no
	type	str
	description	optional identifier of the profile that must be inherited
launchers	required	yes
	type	dict
	description	configuration of each launcher

Inheritance

As explained in Usage it’s possible to merge profiles “on top of each other” using the inherit key, or from the CLI.

Important

Only the content of the launchers root key is merged with the other profile.

Assuming profile-B is declaring a base: profile-A then the merge will happen as follow:

• keep all root keys of profile-B that are not the launchers key.

• merge the launchers as profile-A + profile-B

Note

We use the same term as Python to explain inheritance: specifying inherit: profileA in profileB can be described as profileA is the super profile of the sub profile profileB

Tokens

All keys in the launchers root key can make use of tokens.

A token is a specific chain of characters that is removed (resolved) in the final config, but allow to be replaced by a dynamic value or provide contextual information to the key.

Merge Tokens

Those tokens indicate how the value of the key must be merged with a “base” value of similar type.

In the context of environment profile it indicate how to merge the key/value pair of the current profile with the value of the profile specified in inherit.

A merge token:

• MUST only appear in keys

• MUST be prefixed to its key

• CAN be optional

The following merge tokens are available:

token	description
+=	indicate the key’s value should be appended to the base’s value
-=	indicate the base’s key should be removed if it exist
==	indicate the base’s value should be overriden
!=	add the key and value only if the key doesn’t exist in base
unset	no token is similar to += (append)

Here is an example making use of all of the tokens and the python API used to merge:

import json
from kloch import MergeableDict

base = MergeableDict(
    {
        "rezenv": {
            "+=config": {"quiet": True},
            "requires": {"houdini": "20", "devUtils": "2.1"},
            "environ": {"STATUS": "wip"},
            "roots": ["/d/packages"],
        }
    }
)
top = MergeableDict(
    {
        "rezenv": {
            "+=config": {"quiet": False, "debug": True},
            "requires": {"maya": "2023", "-=devUtils": "", "!=houdini": "19"},
            "==environ": {"PROD": "test"},
            "roots": ["/d/prods"],
        }
    }
)
print(json.dumps(base + top, indent=4))

{
    "rezenv": {
        "+=config": {
            "quiet": false,
            "debug": true
        },
        "requires": {
            "houdini": "20",
            "maya": "2023"
        },
        "==environ": {
            "PROD": "test"
        },
        "roots": [
            "/d/packages",
            "/d/prods"
        ]
    }
}

Context Tokens

Those tokens are exclusive to the launcher’s name key and allow to specify for which system context the launcher must be used. This allow you for example to have a same launcher with 2 different configuration depending on the operating system.

A context token:

• MUST only appear in the {launcher name} key

• MUST be suffixed to the launcher name

• CAN be optional

A context have several property, to declare which properties the launcher must match you use the following syntax:

@{property name}={property value}
# multiple properties can be chained
@{property name}={property value}@{property name}={property value}@...

Tip

If you need the literal @ character in the launcher name or in the properties values you can escape it by doubling it like @@.

The following context properties are availables:

property name	property value	description
os	one of linux, windows, mac	name of the operating system
user	arbitrary string	name of the user

Context tokens are then used with the following logic:

• when reading a profile, a context is created from the current system.

• this context is used to filter out launcher which doesn’t match it.

  • example: .system@os=linux: ... will be deleted if the current os = windows.

• the remaining launchers which points to the same launcher are then merged from top to bottom using the same Merge Tokens logic, thus leaving only a launcher name with no context token at the end.

Example:

__magic__: kloch_profile:4
identifier: greetings
version: 0.1.0
launchers:
  .base:
    environ:
      BASE_GREETING: "hello"
  .system@os=windows:
    command: powershell -c "echo $Env:BASE_GREETING there 👋"
  .system@os=linux:
    command: sh -c "echo $BASE_GREETING there 👋"

In the above example we use the builtin system launcher to run a different command depending on the operating system. Make note that if the os is mac then this mean the .system launcher will just not exist at all in the profile !

Launchers

Specified as key/value pair where the key is the launcher name and the value its configuration.

➡parent	:launchers
⬇key
{launcher name}	required	no
	type	dict
	description	a registred launcher name with its configuration

List of available built-in launchers (see Launcher Plugins for extending it):

• .base : An abstract launcher that whose purpose is to be merged with other launchers.

• .system : A simple launcher executing the given command in the default system console.

• .python : A launcher that execute the given python file with kloch’s own interpreter.

.base

This launcher is never launched and is simply merged with other launchers defined in the profile.

➡parent	:launchers:.base
⬇key
environ	required	no
	type	Dict[str, Union[str, List[str]]]
	description	mapping of environment variable to set before starting the environment. The value can either be a regular string or a list of string. The list of string has each item joined using the system path separator [2]. All values have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$ All values are turned absolute and normalized [4] if they are existing paths.
merge_system_environ	required	no
	type	<class ‘bool’>
	description	True to implicitly merge the system environment (from the machine reading the profile) with the potentially specified environ field. The system environ is merged as “base” so any key specified in the environ will override it.
command	required	no
	type	List[str]
	description	Arbitrary list of command line arguments to call at the end of the launcher execution.
cwd	required	no
	type	<class ‘str’>
	description	Filesystem path to an existing directory to use as “current working directory”. The path will have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$. You can also use variables defined in the environ key. The path is turned absolute and normalized [4].
priority	required	no
	type	<class ‘int’>
	description	How much you should privilege this launcher to be used over other launchers.Higher number means higher priority.

.system

Useless without a command specified. The launcher will just set the given environment variables for the session, execute the command, then exit.

➡parent	:launchers:.system
⬇key
environ	required	no
	type	Dict[str, Union[str, List[str]]]
	description	mapping of environment variable to set before starting the environment. The value can either be a regular string or a list of string. The list of string has each item joined using the system path separator [2]. All values have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$ All values are turned absolute and normalized [4] if they are existing paths.
merge_system_environ	required	no
	type	<class ‘bool’>
	description	True to implicitly merge the system environment (from the machine reading the profile) with the potentially specified environ field. The system environ is merged as “base” so any key specified in the environ will override it.
command	required	no
	type	List[str]
	description	Arbitrary list of command line arguments to call at the end of the launcher execution.
cwd	required	no
	type	<class ‘str’>
	description	Filesystem path to an existing directory to use as “current working directory”. The path will have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$. You can also use variables defined in the environ key. The path is turned absolute and normalized [4].
priority	required	no
	type	<class ‘int’>
	description	How much you should privilege this launcher to be used over other launchers.Higher number means higher priority.
subprocess_kwargs	required	no
	type	<class ‘dict’>
	description	Mapping of kwargs to pass to the internal subprocess.run call.

.python

Execute the given python file with kloch internal python interpreter.

All command keys are passed as args to the python script.

➡parent	:launchers:.python
⬇key
environ	required	no
	type	Dict[str, Union[str, List[str]]]
	description	mapping of environment variable to set before starting the environment. The value can either be a regular string or a list of string. The list of string has each item joined using the system path separator [2]. All values have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$ All values are turned absolute and normalized [4] if they are existing paths.
merge_system_environ	required	no
	type	<class ‘bool’>
	description	True to implicitly merge the system environment (from the machine reading the profile) with the potentially specified environ field. The system environ is merged as “base” so any key specified in the environ will override it.
command	required	no
	type	List[str]
	description	Arbitrary list of command line arguments passed to the python file.
cwd	required	no
	type	<class ‘str’>
	description	Filesystem path to an existing directory to use as “current working directory”. The path will have environment variables expanded with os.expandvars [1]. You can escape the expansion by doubling the $ like $$. You can also use variables defined in the environ key. The path is turned absolute and normalized [4].
priority	required	no
	type	<class ‘int’>
	description	How much you should privilege this launcher to be used over other launchers.Higher number means higher priority.
python_file	required	yes
	type	<class ‘str’>
	description	Filesystem path to an existing python file. The path will have environment variables expanded with os.expandvars [1]. The path is turned absolute and normalized. [4]

Examples

Assuming the file structure:

./profiles/beta.yml

__magic__: kloch_profile:4
identifier: knots:echoes:beta
version: 0.3.2
launchers:
  .base:
    environ:
      PROD_STATUS: beta
      PROD_NAME: echoes
  rezenv:
    requires:
      houdini: 20.1
      maya: 2023
      devUtils: 1+
  rezenv@os=windows:
    config:
      default_shell: "powershell"

./profiles/prod.yml

__magic__: kloch_profile:4
identifier: knots:echoes
version: 0.2.0
inherit: knots:echoes:beta
launchers:
  .base:
    environ:
      PROD_STATUS: prod
  rezenv:
    requires:
      houdini: 20.2
      -=devUtils: _

We execute the following command:

kloch resolve knots:echoes --profile_roots ./profiles/ --skip-context-filtering

__magic__: kloch_profile:4
identifier: knots:echoes
version: 0.2.0
launchers:
  rezenv@os=windows:
    config:
      default_shell: powershell
  .base:
    environ:
      PROD_NAME: echoes
      PROD_STATUS: prod
  rezenv:
    requires:
      maya: 2023
      houdini: 20.2

---

References

[1] (1,2,3,4,5,6,7)

https://docs.python.org/3/library/os.path.html#os.path.expandvars

[2] (1,2,3)

: on UNIX, ; on Windows

[4] (1,2,3,4,5,6,7)

with https://docs.python.org/3.9/library/pathlib.html#pathlib.Path.resolve