Logging#

Tango makes heavy use of the logging module from the standard library to convey information to users. When you’re writing your own Step implementations we encourage you to also use standard Python logging as opposed to print() or other functions that write directly to stdout or stderr. This is easy enough since each Step class already comes with its own logger: Step.logger.

When using the Tango CLI you can set the log level in several different ways:

  1. Through a Tango global settings file.

  2. With the environment variable TANGO_LOG_LEVEL.

  3. Or with the --log-level command-line option.

In some cases (like when running on Beaker) you may also want to enable “file friendly logging”.

Configuring logging in your own CLI#

If you’re writing your own CLI that uses tango, you can utilize the initialize_logging() function to easily configure logging properly.

For example,

from tango.common.logging import initialize_logging, teardown_logging

initialize_logging(log_level="info")

logger = logging.getLogger()
logger.info("Running script!")

teardown_logging()

[...] INFO     Running script! ...

If you want to have logs written to a file, you can use the file_handler() context manager.

Logging from worker processes or threads#

If you have steps or other functions that spawn workers, and you want to enable logging within those workers, you can call the initialize_worker_logging() function to configure logging within each worker. This assumes that you’ve called initialize_logging() from the main process (the tango CLI does this for you).

For example,

import logging
import multiprocessing as mp

from tango import Step
from tango.common.logging import initialize_worker_logging

@Step.register("multiprocessing_step_example")
class MultiprocessingStep(Step):
    def run(self, num_proc: int = 2) -> bool:  # type: ignore
        workers = []
        for i in range(num_proc):
            worker = mp.Process(target=_worker_function, args=(i,))
            workers.append(worker)
            worker.start()
        for worker in workers:
            worker.join()
        return True


def _worker_function(worker_id: int):
    initialize_worker_logging(worker_rank=worker_id)
    logger = logging.getLogger(MultiprocessingStep.__name__)
    logger.info("Hello from worker %d!", worker_id)

Reference#

tango.common.logging.TANGO_LOG_LEVEL: Optional[str] = None#

The log level to use globally. The value can be set from the corresponding environment variable (TANGO_LOG_LEVEL) or field in a TangoGlobalSettings file (log_level), or from the command line with the --log-level option. Possible values are “debug”, “info”, “warning”, or “error” (not case sensitive). For example,

$ tango --log-level info run ...

Note

This does not affect the cli_logger or logs from Tqdm progress bars.

tango.common.logging.FILE_FRIENDLY_LOGGING: bool = False#

If this flag is set to True, we remove special styling characters from log messages, add newlines to Tqdm output even on an interactive terminal, and we slow down Tqdm’s output to only once every 10 seconds.

Attention

Unfortunately this won’t affect tqdm output from other libraries that don’t use Tango’s Tqdm wrapper.

By default, it is set to False. It can be changed by setting the corresponding environment variable (FILE_FRIENDLY_LOGGING) or field in a TangoGlobalSettings file (file_friendly_logging) to “true” or “false”, or from the command line with the --file-friendly-logging flag. For example,

$ tango --file-friendly-logging run ...
tango.common.logging.cli_logger = <Logger tango.__main__ (WARNING)>#

A logger that emits messages directly to stdout/stderr using rich’s Console class.

This provides a convenient way for command-line apps to log pretty, styled messages uses the markup style provided by rich.

tango.common.logging.initialize_logging(*, log_level=None, enable_cli_logs=None, file_friendly_logging=None)[source]#

Initialize logging, which includes setting the global log level, format, and configuring handlers.

Tip

This should be called as early on in your script as possible.

Tip

You should also call teardown_logging() as the end of your script.

Tip

For worker threads/processes, use initialize_worker_logging() instead.

Parameters:

  • log_level (Optional[str], default: None) – Can be one of “debug”, “info”, “warning”, “error”. Defaults to the value of TANGO_LOG_LEVEL, if set, or “error”.

  • enable_cli_logs (Optional[bool], default: None) – Set to True to enable messages from the cli_logger.

  • file_friendly_logging (Optional[bool], default: None) – Enable or disable file friendly logging. Defaults to the value of FILE_FRIENDLY_LOGGING.

tango.common.logging.initialize_worker_logging(worker_rank=None)[source]#

Initialize logging in a worker thread/process.

Parameters:

worker_rank (Optional[int], default: None) – The rank/ID of the worker.

tango.common.logging.initialize_prefix_logging(*, log_level=None, prefix=None, main_process=False)[source]#

Initialize logging with a prefix.

Parameters:

  • log_level (Optional[str], default: None) – Can be one of “debug”, “info”, “warning”, “error”. Defaults to the value of TANGO_LOG_LEVEL, if set, or “error”.

  • prefix (Optional[str], default: None) – The string prefix to add to the log message.

  • main_process (bool, default: False) – Whether it is for the main/worker process.

tango.common.logging.teardown_logging()[source]#

Cleanup any logging fixtures created from initialize_logging(). Should be called at the end of your script.

tango.common.logging.file_handler(filepath)[source]#

A context manager that can be used to route logs to a file by adding a logging.FileHandler to the root logger’s handlers.

For example,

from tango.common.logging import initialize_logging, file_handler, teardown_logging

initialize_logging(log_level="info")

logger = logging.getLogger()
logger.info("Hi!")

with file_handler("log.out"):
    logger.info("This message should also go into 'log.out'")

teardown_logging()
Return type:

AbstractContextManager[None]