7.1.1. Pipeline API

class radical.entk.Pipeline

A pipeline represents a collection of objects that have a linear temporal execution order. In this case, a pipeline consists of multiple ‘Stage’ objects. Each `Stage_i` can execute only after all stages up to `Stage_(i-1)` have completed execution.

add_stages(value)

Appends stages to the current Pipeline

Argument:List of Stage objects

completed

Returns whether the Pipeline has completed

Returns:Boolean

current_stage

Returns the current stage being executed

Returns:Integer

from_dict(d)

Create a Pipeline from a dictionary. The change is in inplace.

Argument:python dictionary Returns:None

lock

Returns the lock over the current Pipeline

Returns:Lock object

luid

Unique ID of the current pipeline (fully qualified). For the pipeline class, his is an alias to uid.

example:

>>> pipeline.luid
pipe.0001

Getter:Returns the fully qualified uid of the current pipeline Type:String

name

Name of the pipeline useful for bookkeeping and to refer to this pipeline while data staging. Do not use a ‘,’ or ‘_’ in an object’s name.

Getter:Returns the name of the pipeline Setter:Assigns the name of the pipeline Type:String

resume()

Continue execution of paused stages and tasks.

• The resume() method can only be called on a suspended pipeline, an exception will be raised if that condition is not met.
• The state of a resumed pipeline will be set to the state the pipeline had before suspension.

stages

Stages of the list

Getter:Returns the stages in the current Pipeline Setter:Assigns the stages to the current Pipeline Type:List

state

Current state of the pipeline

Getter:Returns the state of the current pipeline Type:String

state_history

Returns a list of the states obtained in temporal order

Returns:list

suspend()

Pause execution of the pipeline: stages and tasks that are executing will continue to execute, but no new stages and tasks will be eligible for execution until resume() is called.

• The suspend() method can not be called on a suspended or completed pipeline, doing so will result in an exeption.
• The state of the pipeline will be set to SUSPENDED.

to_dict()

Convert current Pipeline (i.e. its attributes) into a dictionary

Returns:python dictionary

uid

Unique ID of the current pipeline

Getter:Returns the unique id of the current pipeline Type:String