Skip to main content

Edit Helmfiles with Python

· 4 min read

Editing helmfiles with ruamel.yaml in Python​

Managing helmfiles in python can be tricky due to the presence of Go templates in them, which are not compatible with the standard YAML parsers. This guide demonstrates how to ignore the curly braces in helmfiles and edit them using the ruamel.yaml python library.

The issue​

Helmfiles contain Go template syntax like {{ .Value.example }}. And the existing yaml parser cannot correctly interpret this syntax is it is not a YAML syntax.

Let's try dumping this line using the ruamel.yaml library:

 - host: {{ .Values.domain }}

This syntax is being interpreted it as a key with a null value, resulting in:

 - host: {{ .Values.domain: null}: null}

This makes the task of automating the construction of helmfiles difficult. I have chosen 3 approaches for dealing with this issue.

  1. The most easy way is to prerender the helmfile using helm before the processing.

  2. Introduce placeholders in order to keep the YAML syntax valid.

  3. Use ruamel.yaml library's support of jinja2 templates.

Strategy 1: Pre-Rendering the helmfile​

The simplest solution is to pre-render the helmfile before processing. This will replace the Go templates with actual values and make the YAML file valid for the parsers.

If prerendeing is not a solution for you then follow the next strategy.

Strategy 2: Use placeholders​

Temporarily replace the {{. Go.templates }} to make all the editing you need and then bring them back.

Step 1: Write functions to (un)escape the Go syntax​

Here are the methods for the respective replacings:

def escape_go_templates(content: str) -> str:
    return re.sub(r"\s{{(.*?)}}", r" __GO_TEMPLATE__\1__GO_TEMPLATE__", content)

def unescape_go_templates(content: str) -> str:
    return re.sub(r"__GO_TEMPLATE__(.+?)__GO_TEMPLATE__", r"{{\1}}", content)

Once you have a valid YAML document, make the changes you want then revert the Go templates using the unescape_go_templates function.

Step 2: Process the document​

This is an example of helmfile editing following this approach.

def process_helmfile(helmfile_path: str) -> None:
    
    with open(helmfile_path, "r") as helm_file:
        helm_content = helm_file.read()
        helmfile_escaped = escape_go_templates(helm_content)

    yaml_documents = yaml.load(helmfile_escaped)
    for doc in yaml_documents:
        # Example modification:
        if "releases" in doc:
            doc["releases"][0]["name"] = "my-service"

    with open(helmfile_path, "w") as updated_helmfile:
        updated_content = yaml.dump(yaml_documents)
        final_content = unescape_go_templates(updated_content)
        updated_helmfile.write(final_content)

The last way of editing helmfiles and the one that I prefer is to use ruamel.yaml and it's support of jinja2.

Strategy 3: Utilize ruamel.yaml and jinja2 templates​

The {{. Go.templates }} used in helmfiles have a similar purpose and syntax with jinja2, which is a templating engine for Python. Both use the concept of placeholders {{ }}.

Install the ruamel.yaml['jinja2] library. Configure ruamel.yaml instance so it ignores the jinja2. Change the default value of width parameter.

yaml = YAML(typ="jinja2")
yaml.width = 4096

In case you want to prevent ruamel.yaml of adding single quotes to the Go placeholders '{{ }}' you can write a custom representer function for that.

tip

Everything is an object in Python! So aren't functions as well? You can create a function and pass it as an argument to another function. An example of this is shown below. Read more.

def _represent_str(representer: SafeRepresenter, data: str | None) -> ScalarNode:
    if data and data.startswith("{{"):
        return representer.represent_scalar("tag:yaml.org,2002:str", data, style="-")

    return representer.represent_str(data)

This function,_represent_str, customizes how strings are represented in your YAML instance. It specifically addresses how to handle strings that start with {{.

When encountering such strings, the function employs a special YAML tag, tag:yaml.org,2002:str, to ensure these strings are output without additional quotation marks.

Add this function as an argument to your yaml representer and the default behavor of putting quotes to {{. Go.templates }} will be modified.

yaml.representer.add_representer(str, _represent_str)

Now you can edit your helmfile as a normal YAML document. Happy automation!