Elegant Env Vars for Jenkins Python

In my case, I use Jenkins to drive some Python scripts. We can set configuration in Jenkins via System Properties, Node Configuration or Build Parameters.

We can get the configurations through environment variables, and also pass stage outputs to other stages via environment variables.

Visualizing and retaining environment variables is crucial, So I want to manage environment variables in an elegant way.

I implemented a class EnvSettings extend from pydantic.BaseSettings to manage environment variables:

env_settings.py

import inspect
import json
from pathlib import Path
from threading import Lock
from typing import ClassVar, Optional

from pydantic_settings import BaseSettings, SettingsConfigDict


class EnvSettings(BaseSettings):
    """Environment settings management using Pydantic."""

    model_config = SettingsConfigDict(env_file='.env', env_ignore_empty=True, extra='ignore')

    _instance: ClassVar[Optional['EnvSettings']] = None
    _lock: ClassVar[Lock] = Lock()

    def __new__(cls, *args, **kwargs) -> 'EnvSettings':
        """Create a singleton instance of EnvSettings."""
        if cls is EnvSettings:
            raise TypeError('EnvSettings is a singleton base class and cannot be instantiated directly.')

        if not cls._instance:
            with cls._lock:
                if not cls._instance:
                    cls._instance = super().__new__(cls, *args, **kwargs)
        return cls._instance

    def __setattr__(self, name: str, value: object) -> None:
        """In a Python process, if environment variables are modified, append them to the env_file.

        Args:
            name: The attribute name.
            value: The attribute value.
        """
        super().__setattr__(name, value)

        if name in type(self).model_fields:
            with open('.env', 'a') as f:
                f.write(f'\n# Modified by {self.called_by}')
                f.write(
                    f'\n{name}={json.dumps(self.model_dump(include={name}, exclude_none=True, mode="json").get(name))}'
                )

    @property
    def called_by(self) -> Path:
        """Get the absolute path of the file that called the current method."""
        return Path(inspect.stack()[-1].filename)

    def snapshot(self, file: Path) -> Path:
        """Save the current environment variables to a file.

        Args:
            file: The file to save the environment variables to.

        Returns:
            The file path.
        """
        with open(file, 'a') as f:
            f.write(f'\n# Snapshot by {self.called_by}')
            for key, value in self.model_dump(exclude_none=True, mode='json').items():
                f.write(f'\n{key}={json.dumps(value)}')
        return file

This class overrides the __setattr__ method to ensure that when fields defined in BaseSettings are reassigned, they are written to the .env file. This way, when the current process ends and a new one starts, it reads the updated values generated by the previous process. Additionally, called_by is used to track the actual modifier of the environment variables.

It also provides a snapshot method to save the current environment variables to a specified file.

To use this class, we can define our own settings class that inherits from EnvSettings:

my_settings.py

class MySettings(EnvSettings):
    NAME: str = 'default_name'

ms = MySettings()
ms.NAME = 'new_name'

After running the above code, the .env file will contain:

# Modified by my_settings.py
NAME="new_name"

This approach provides an elegant way to manage environment variables in Jenkins-driven Python scripts.

Elegant Env Vars for Jenkins Python

Comments