Pipeline API

class radical.entk.Pipeline[source]

A pipeline represents a collection of objects that have a linear temporal execution order. In this case, a pipeline consists of multiple ‘Stage’ objects. Each `Stage_i` can execute only after all stages up to `Stage_(i-1)` have completed execution.

add_stages(value)[source]

Appends stages to the current Pipeline

Argument: List of Stage objects

as_dict()[source]

Convert current Pipeline (i.e. its attributes) into a dictionary

Returns: python dictionary

property completed

Returns whether the Pipeline has completed

Returns: Boolean

property current_stage

Returns the current stage being executed

Returns: Integer

from_dict(d)[source]

Create a Pipeline from a dictionary. The change is in inplace.

Argument: python dictionary
Returns: None

property lock

Returns the lock over the current Pipeline

Returns: Lock object

property luid

Unique ID of the current pipeline (fully qualified). For the pipeline class, his is an alias to uid.

Example

>>> pipeline.luid
pipe.0001

Getter: Returns the fully qualified uid of the current pipeline
Type: String

property name

Name of the pipeline useful for bookkeeping and to refer to this pipeline while data staging. Do not use a ‘,’ or ‘_’ in an object’s name.

Getter: Returns the name of the pipeline
Setter: Assigns the name of the pipeline
Type: String

resume()[source]

Continue execution of paused stages and tasks.

The resume() method can only be called on a suspended pipeline, an exception will be raised if that condition is not met.

The state of a resumed pipeline will be set to the state the pipeline had before suspension.

property stages

Stages of the list

Getter: Returns the stages in the current Pipeline
Setter: Assigns the stages to the current Pipeline
Type: List

property state

Current state of the pipeline

Getter: Returns the state of the current pipeline
Type: String

property state_history

Returns a list of the states obtained in temporal order

Returns: list

suspend()[source]

Pause execution of the pipeline: stages and tasks that are executing will continue to execute, but no new stages and tasks will be eligible for execution until resume() is called.

The suspend() method can not be called on a suspended or completed pipeline, doing so will result in an exeption.

The state of the pipeline will be set to SUSPENDED.

property uid

Unique ID of the current pipeline

Getter: Returns the unique id of the current pipeline
Type: String