DataKitchen DataOps Documention

Synchronize Nodes

Empty-function nodes used as placeholders or workflow convergence points.

Synchronize Nodes contain no keys. These nodes typically serve as placeholders while users are building graphs. They also can be convergence steps in complex workflows, ensuring that all previous nodes have executed before the workflow continues.

The default recipe and the template used for all new recipes contain two synchronize nodes, which allow one to run a recipe for demonstration purposes without first configuring data sources and/or data sinks.

Node Type Code

The code value for the synchronize node type is DKNode_NoOp.

Description.json

{
   "type" : "DKNode_NoOp",
   "description": "[YOUR DESCRIPTION HERE]"
}

Notebook.json

The notebook.json file is optional for synchronize nodes.

Examples

Graph Conversion Points

Synchronize nodes are useful as conversion points after the processing up upstream parallel nodes in a recipe variation's graph.

{
   "type" : "DKNode_NoOp",
   "description": "[YOUR DESCRIPTION HERE]"
}

Planned Parallelism

Move nodes down a graph level by inserting a synchronize node upstream.

Test Built-in Variables

{
    "tests": {
      "description": "A test against the value of 1 found in variables.json.",
      "action": "stop-on-error",
      "type:": "test-contents-as-integer",
      "test-variable": "failure_var",
      "keep-history": true,
      "test-logic": {
        "test-compare": "less-than",
        "test-metric": 0
      }
    }
}
{
	  "variable-list": {
  	   "failure_var": 1
    }
}

Insert Pauses in Graph Processing

{
    "sleep": 30
}

Updated 2 months ago


Synchronize Nodes


Empty-function nodes used as placeholders or workflow convergence points.

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.