Pipeline Monitoring API's

Overview

The following set of API's provide a mechanism to monitor the various parameters of the pipeline run.

1. Get logs of the pipeline run

The following API, provides the mechanism for the users to access the logs of the pipeline run both at a pipeline level and block level

Usage - to get the block level logs

from razor import LogLevel, LogType

prun.logs(
    log_type=LogType.BLOCK,
    log_levels=LogLevel.all_levels()
)

Usage - to get the system level logs

from razor import LogLevel

prun.logs(
    log_type=LogType.SYSTEM,
    log_levels=LogLevel.all_levels()
)

Attributes

Argument Description
log_type type of the log, the different type of logs are
1. LogType.BLOCK - for getting the block level logs
2. LogType.SYSTEM - For getting the system level logs
log_levels Standard logging levels from python logger

2. Get all metrics

The following API, shows the metrics for a particular pipeline run

prun.metrics()

3. Get queue status of the pipeline

The following API, shows the Queue status of the pipeline.

queue_list = prun.queue()

4. Get run status of the pipeline

The following API, shows the current run status of the pipeline

prun.run_status()

5. Terminate a pipeline run

The following API, shows how a pipeline run can be terminated

prun.stop()