DataKitchen DataOps Documention

CLI Quickstart3

In this guide we'll cover an overview of DataOps orchestration via DKCloudCommand, DataKitchen's command line tool. Specific topics covered include:

  • Adding & Configuring a Container-based Node for GUI tools, specifically for Pentaho DI
  • Validating Local Changes Applied to a Recipe
  • Creating Recipe Variations with Variable Overrides to Provide Runtime Flexibility
  • Using Locally Configured Tool to View Local vs. Remote File Differences
  • Compiling Entire Recipes to Check the Proper Application of Parameterization
  • Viewing Lists of Recipe Variations
  • Deleting Remote Copies of Recipe Files
  • Deleting Entire Versions of Remote Recipes

Super-Fast CLI Quickstart3

Super-Fast CLI Quickstart3

Advanced users who need less detailed documentation to get up and running with DataKitchen's DataOps platform may jump right to the SUPER-FAST CLI QUICKSTART3 tab of this guide's Synopsis for a menu of abridged commands. This route provides a fully complete Recipe for this guide. Alternatively, you may follow the steps below to build out the Recipe starting from a copy of the Recipe built in CLI Quickstart2.

Check the Menu

What is our current lineup of kitchens?

~/Kitchens $ dk kitchen-list
~/Kitchens $ dk kl
YYYY-MM-DD HH:MM:SS - Getting the list of kitchens
kitchen-list returned 3 kitchens
+---- master
    +---- Production
        +---- Dev_Sprint_1

What recipes exist in the Dev_Sprint_1 Kitchen?

~/Kitchens $ cd Dev_Sprint_1
~/Kitchens/Dev_Sprint_1 $ dk recipe-list
~/Kitchens $ cd Dev_Sprint_1
~/Kitchens/Dev_Sprint_1 $ dk rl
~/Kitchens $ dk rl -k Dev_Sprint_1
YYYY-MM-DD HH:MM:SS - Getting the list of Recipes for Kitchen 'Dev_Sprint_1'
list_recipe - elapsed: 0
DKCloudCommand.list_recipe returned 1 recipes
  Recipe-Template

Use the kitchen-create command to generate a new Child Kitchen named Feature_Sprint_3 that we will used to further build out the Recipe-Template Recipe.

Create a Uniquely-Named Kitchen

Append your name when creating the Kitchen so that you have your own, segregated workspace. For example, if your name is Chef, you would create a Kitchen named Feature_Sprint_3_Chef.

~/Kitchens/Dev_Sprint_1 $ cd ..
~/Kitchens $ dk kitchen-create --parent Dev_Sprint_1 Feature_Sprint_3
~/Kitchens/Dev_Sprint_1 $ cd ..
~/Kitchens $ dk kc -p Dev_Sprint_1 Feature_Sprint_3
YYYY-MM-DD HH:MM:SS - Creating kitchen Feature_Sprint_3 from parent kitchen Dev_Sprint_1
DKCloudCommand.create_kitchen created Feature_Sprint_3

Use the kitchen-get command to get a local copy of the new Kitchen and the Recipe-Template Recipe it has inherited from its Parent Kitchen.

~/Kitchens $ dk kitchen-get Feature_Sprint_3 --recipe Recipe-Template
~/Kitchens $ dk kg Feature_Sprint_3 -r Recipe-Template
YYYY-MM-DD HH:MM:SS - Getting kitchen 'Feature_Sprint_3'
list_recipe - elapsed: 0
Got Kitchen 'Feature_Sprint_3'
DKCloudCommand.get_recipe has 7 sections
  Recipe-Template/placeholder-node1
  Recipe-Template
  Recipe-Template/resources
  Recipe-Template/resources/email-templates
  Recipe-Template/load-profits-node
  Recipe-Template/placeholder-node2
  Recipe-Template/load-profits-node/actions

Add Pentaho DI Node

Introduction

We previously added, to our Recipe-Template Recipe, an Action Node that leveraged raw SQL to copy data from an AWS S3 bucket to an AWS Redshift database. Next, let's create another Action Node that will complete data work similar to that performed by the load-profits-node we built in CLI Quickstart2.

However, rather than leveraging raw SQL, this new node will encapsulate the data integration work that might be completed by a Data Engineer using Pentaho's Data Integration graphical interface tool. Recall that DataKitchen's DataOps platform plays nice with all existing tools in your team's toolchain, meaning that you can implement DataOps at your organization without reinventing the wheel.

As we'll see below, the new node we are creating is designed to create schemas and generate tables in AWS Redshift, populate said tables with data gathered from an AWS S3 bucket, and finally, procure data, via a Key, to be leveraged by a node test.

A view from [Transmit](https://panic.com/transmit/) of the AWS S3 bucket containing source data in .txt files.

A view from Transmit of the AWS S3 bucket containing source data in .txt files.

Creating an AWS Redshift database schema using [Pentaho DI](http://www.pentaho.com/product/data-integration).  Note the use of parameterization, which allows for runtime flexibility.

Creating an AWS Redshift database schema using Pentaho DI. Note the use of parameterization, which allows for runtime flexibility.

Creating AWS Redshift tables and populating them with data copied from an AWS S3 bucket using [Pentaho DI](http://www.pentaho.com/product/data-integration).

Creating AWS Redshift tables and populating them with data copied from an AWS S3 bucket using Pentaho DI.

Using [Pentaho DI](http://www.pentaho.com/product/data-integration) to generate a row count from one of the newly created tables, which will serve as automated test data for the **pentaho-di-node**.  The row count is exported to a .txt file.

Using Pentaho DI to generate a row count from one of the newly created tables, which will serve as automated test data for the pentaho-di-node. The row count is exported to a .txt file.

Update Variables.json

Recall the list of Secrets that exist in the Vault associated with your account.

~/Kitchens $ dk secret-list --recursive
~/Kitchens $ dk sl -rc
YYYY-MM-DD HH:MM:SS - Getting the list of secrets
secret_list - elapsed: 0
secret-list returned 14 secrets
        vault://aws/account_id
        vault://aws/role
        vault://dockerhub/email
        vault://dockerhub/namespace
        vault://dockerhub/password
        vault://dockerhub/username
        vault://redshift/database
        vault://redshift/hostname
        vault://redshift/password
        vault://redshift/port
        vault://redshift/username
        vault://s3_schema/bucket

Update variables.json to include a number of new Docker Hub variables. Since we are building our new node to leverage a GUI tool rather than raw SQL, we will place the data operations completed by the new node with a Docker Container. Replace the contents of the file with the text below. Note the ability to group variables for a common tool within a Python dictionary.

Update Email and Schema Values

Just as we did in CLI Quickstart2, update the email and schema values after copying the text below into the variables.json file. Replace [YOUR EMAIL HERE] with your preferred destination email address and save the file. Replace [YOUR SCHEMA NAME HERE] with your first initial and last name, e.g. jdoe.

{
    "name": "Recipe-Template QS3 Variables",
    "variable-list": {
        "awsaccountid": "#{vault://aws/account_id}",
        "awsrole": "#{vault://aws/role}",
        "dockerhub": {
              "email": "#{vault://dockerhub/email}",
              "namespace": "#{vault://dockerhub/namespace}",
              "password": "#{vault://dockerhub/password}",
              "username": "#{vault://dockerhub/username}"
        },
        "alerts": {
              "start": "[YOUR EMAIL HERE]", 
              "success": "[YOUR EMAIL HERE]", 
              "error": "[YOUR EMAIL HERE]"
        }
        "redshiftdatabase": "#{vault://redshift/database}",
        "redshifthostname": "#{vault://redshift/hostname}",
        "redshiftpassword": "#{vault://redshift/password}",
        "redshiftport": "#{vault://redshift/port}",
        "redshiftschema": "[YOUR SCHEMA NAME HERE]",
        "redshiftusername": "#{vault://redshift/username}",
        "s3bucketname": "#{vault://s3_schema/bucket}",
        "s3bucketpath": "superstore/customer/sales-profitability"
      }
}
Comparison of **variables.json** from **CLI Quickstart2** and **CLI Quickstart3**.

Comparison of variables.json from CLI Quickstart2 and CLI Quickstart3.

~/Kitchens $ cd Feature_Sprint_3/Recipe-Template
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status
~/Kitchens $ cd Feature_Sprint_3/Recipe-Template
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
YYYY-MM-DD HH:MM:SS - Getting the status of Recipe 'Recipe-Template' in Kitchen 'Feature_Sprint_3'
    versus directory '~/Kitchens/Feature_Sprint_3/Recipe-Template'
1 files are modified on local:
    variables.json

12 files are unchanged

For now, withhold updating single-file updates to remote version of the Recipe-Template Recipe. Later, we will batch update both new and modified Recipe files using the recipe-update command.

Add Node Description

Every Recipe graph node requires its own sub-directory and description.json file. Name this new node pentaho-di-node:

~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/description.json

Paste the following text to your command-line prompt:

{
    "type": "DKNode_Action",
    "description": "Encapsulates work done by a Data Engineer using Pentaho DI to setup an AWS Redshift database schema, create tables, populate tables with data from an AWS S3 bucket, and generate a row count to be used for automated testing."
}

Type Enter followed by Ctrl+D to save the new file and return to the prompt.

A view of **Recipe-Template**'s **pentaho-di-node**'s **description.json** file.

A view of Recipe-Template's pentaho-di-node's description.json file.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
YYYY-MM-DD HH:MM:SS - Getting the status of Recipe 'Recipe-Template' in Kitchen 'Feature_Sprint_3'
    versus directory '~/Kitchens/Feature_Sprint_3/Recipe-Template'
1 files are modified on local:
    variables.json

1 files are local only:
    pentaho-di-node/description.json

1 directories are local only:
    pentaho-di-node

12 files are unchanged

Update Variations.json

We will continue at present with a single Variation for our Recipe, Variation1, so there is no need to update the variations.json Recipe file. Later, we'll cover adding and configuring Variations in detail.

Configure Node

Much like node named action-node that we created in CLI Quickstart2, the pentaho-di-node is also an Action Node, and thus contains an /actions subdirectory. However, because pentaho-di-node leverages a container, configuration is somewhat different.

First, create an /actions sub-directory and then a .json configuration file named DKDataSource_Container.json:

~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node/actions
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/actions/DKDataSource_Container.json

Copy the following text to your command-line prompt:

{
    "name": "DKDataSource_Container",
    "type": "DKDataSource_Container",
    "dockerhub-email": "{{dockerhub.email}}",
    "dockerhub-namespace": "{{dockerhub.namespace}}",
    "dockerhub-password": "{{dockerhub.password}}",
    "dockerhub-username": "{{dockerhub.username}}",
    "image-repo": "ac_pentahodi_container",
    "image-tag": "latest",
    "data-container-name": "data_ac_pentahodi",
    "inside-container-file-mount": "/dk/AnalyticContainer/PentahoDIContainer",
    "inside-container-file-directory": "docker-share",
    "container-input-configuration-file-path": "Recipe-Template/pentaho-di-node/actions/docker-share",
    "container-input-configuration-file-name": "config.json",
    "container-output-log-file": "ac_logger.log",
    "container-output-progress-file": "progress.json",
    "hide-dockerfile-location": "/dk/AnalyticContainer/PentahoDIContainer/Dockerfile",
    "keys": {
        "get-test-data": {
            "container-key": "get-test-data"
        }
    },
    "tests": {
        "get-test-data-count": {
            "test-variable": "get-test-data",
            "type": "test-contents-as-integer",
            "applies-to-keys": ["get-test-data"],
            "action": "stop-on-error",
            "keep-history": "True",
            "test-logic": {
                "test-variable": "get-test-data",
                "test-compare": "greater-than",
                "test-metric": "10000"
            }
        }
    }
}

Type Enter followed by Ctrl+D to save the new file and return to the prompt.

A view of **DKDataSource_Container.json**, the configuration file for the container used by **pentaho-di-node**.  This container encapsulates the processing of files generated from Pentaho DI's GUI tool.

A view of DKDataSource_Container.json, the configuration file for the container used by pentaho-di-node. This container encapsulates the processing of files generated from Pentaho DI's GUI tool.

Note that the configuration for the node depicted above includes the definition of a Key and a Test, much like we've seen previously.

The data work completed by this node (creating schemas and tables) is driven by the Pentaho Data Integration .ktr files we will add below, which are located within a container. Still, the code for the automated testing functionality for this node is located outside of the container in the container configuration file above. In short, Pentaho DI generates a row count from one of the tables it populates, generates a temporary file to house this value, and the node's configuration file tests this row count.

In this case, if the test fails, the Order will stop in place as the action field has been set to stop-on-error. Implementing automated tests for each node in your Recipe allows your teams to be confident in the quality the insights generated by your data analytic pipelines.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
YYYY-MM-DD HH:MM:SS - Getting the status of Recipe 'Recipe-Template' in Kitchen 'Feature_Sprint_3'
    versus directory '~/Kitchens/Feature_Sprint_3/Recipe-Template'
1 files are modified on local:
    variables.json

2 files are local only:
    pentaho-di-node/description.json
    pentaho-di-node/actions/DKDataSource_Container.json

2 directories are local only::
    pentaho-di-node
    pentaho-di-node/actions

12 files are unchanged

Next, create an additional node sub-directory under /actions named /docker-share. This directory is common to all Recipe graph nodes that leverage containers to encapsulate toolchain files. This is the Recipe directory where Pentaho's .ktr files will be located.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node/actions/docker-share

Container Node Files

The files that perform data work for a given node are most often located in a Recipe's /resources directory. However, for nodes that leverage containers, these files are instead located within the node itself, within the node's /docker-share directory. This is because container nodes encapsulate everything needed to complete the work for which they were designed.

You also need to create a config.json file to sit within pentaho-di-node/actions/docker-share. This file passes the required infrastructure credentials into the container.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/actions/docker-share/config.json

Copy the following text to your command-line prompt:

{
  "loglevel": "Maximum",
  "DKDOC-logelevel": ["Miminum", "Maximum"],
  "keys": {
    "create-schema": {
      "filename": "create-schema.ktr",
      "file-parameters": {
        "database": "{{redshiftdatabase}}",
        "hostname": "{{redshifthostname}}",
        "password": "{{redshiftpassword}}",
        "port": "{{redshiftport}}",
        "schema": "{{redshiftschema}}",
        "username": "{{redshiftusername}}"
      }
    },
    "load-data": {
      "filename": "create-and-load-superstore-data.ktr",
      "file-parameters": {
        "accountid": "{{awsaccountid}}",
        "role": "{{awsrole}}",
        "database": "{{redshiftdatabase}}",
        "hostname": "{{redshifthostname}}",
        "password": "{{redshiftpassword}}",
        "port": "{{redshiftport}}",
        "bucket": "{{s3bucketname}}",
        "schema": "{{redshiftschema}}",
        "username": "{{redshiftusername}}"
      }
    },
    "get-test-data": {
      "filename": "get-count-of-superstore-data.ktr",
      "file-parameters": {
        "database": "{{redshiftdatabase}}",
        "hostname": "{{redshifthostname}}",
        "password": "{{redshiftpassword}}",
        "port": "{{redshiftport}}",
        "schema": "{{redshiftschema}}",
        "username": "{{redshiftusername}}"
      },
      "integration-output-filename": "get-test-data.txt"
    }
  }
}

Type Enter then Ctrl+D to save the file and return to the prompt.

A view of **pentaho-di-node**'s configuration file, **config.json**.

A view of pentaho-di-node's configuration file, config.json.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
YYYY-MM-DD HH:MM:SS - Getting the status of Recipe 'Recipe-Template' in Kitchen 'Feature_Sprint_3'
    versus directory '~/Kitchens/Feature_Sprint_3/Recipe-Template'
1 files are modified on local:
    variables.json

3 files are local only:
    pentaho-di-node/description.json
    pentaho-di-node/actions/DKDataSource_Container.json
    pentaho-di-node/actions/docker-share/config.json

3 directories are local only::
    pentaho-di-node
    pentaho-di-node/actions
    pentaho-di-node/actions/docker-share

12 files are unchanged

Add Toolchain Files

This example container node reflects the data integration work a Data Engineer might complete leveraging the Pentaho Data Integration GUI, specifically to create database schemas in AWS Redshift, populate tables, and return test data. Navigate to your local /docker-share sub-directory for the pentaho-di-node and execute the following commands to download copies of the Pentaho .ktr files to be used by this Recipe.

  • create-schema.ktr
  • create-and-load-superstore-data.ktr
  • get-count-of-superstore-data.ktr
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd pentaho-di-node/actions/docker-share
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file1)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file2)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file3)
A view of the structure of **Recipe-Template** in **Feature_Kitchen_3**.  Note the new addition of .ktr files generated by *Pentaho DI*.

A view of the structure of Recipe-Template in Feature_Kitchen_3. Note the new addition of .ktr files generated by Pentaho DI.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
YYYY-MM-DD HH:MM:SS - Getting the status of Recipe 'Recipe-Template' in Kitchen 'Feature_Sprint_3'
    versus directory '~/Kitchens/Feature_Sprint_3/Recipe-Template'
1 files are modified on local:
    variables.json

6 files are local only:
    pentaho-di-node/description.json
    pentaho-di-node/actions/DKDataSource_Container.json
    pentaho-di-node/actions/docker-share/config.json
    pentaho-di-node/actions/docker-share/create-and-load-superstore-data.ktr
    pentaho-di-node/actions/docker-share/create-schema.ktr
    pentaho-di-node/actions/docker-share/get-count-of-superstore-data.ktr

3 directories are local only::
    pentaho-di-node
    pentaho-di-node/actions
    pentaho-di-node/actions/docker-share

12 files are unchanged

Update Graph.json

Finally, update graph.json to reflect the addition of the new node. Add the new node to the end of the existing Recipe graph by replacing the contents of graph.json with the text below:

{
  "nodes": [
    ["placeholder-node1", {}],
    ["placeholder-node2", {}],
    ["load-profits-node", {}],
    ["pentaho-di-node", {}]
  ],
  "edges": [
    ["placeholder-node1", "placeholder-node2", {"weight": 0}],
    ["placeholder-node2", "load-profits-node", {"weight": 0}],
    ["load-profits-node", "pentaho-di-node", {"weight": 0}]
  ]
}
The **pentaho-di-node** has been added as the last node in the graph.

The pentaho-di-node has been added as the last node in the graph.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
YYYY-MM-DD HH:MM:SS - Getting the status of Recipe 'Recipe-Template' in Kitchen 'Feature_Sprint_3'
    versus directory '~/Kitchens/Feature_Sprint_3/Recipe-Template'
2 files are modified on local:
    graph.json
    variables.json

6 files are local only:
    pentaho-di-node/description.json
    pentaho-di-node/actions/DKDataSource_Container.json
    pentaho-di-node/actions/docker-share/config.json
    pentaho-di-node/actions/docker-share/create-and-load-superstore-data.ktr
    pentaho-di-node/actions/docker-share/create-schema.ktr
    pentaho-di-node/actions/docker-share/get-count-of-superstore-data.ktr

3 directories are local only::
    pentaho-di-node
    pentaho-di-node/actions
    pentaho-di-node/actions/docker-share

11 files are unchanged

Recipe Validation I

Before updating local Recipes to remote, use the recipe-validate command to check for errors that would prevent a successful Order-Run.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-validate --variation Variation1
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rv -v Variation1
YYYY-MM-DD HH:MM:SS - Validating recipe/variation Recipe-Template.Variation1 in Kitchen Feature_Sprint_3
Validating recipe with local changes applied
DKCloudCommand.recipe_validate succeeded
No recipe issues identified.

Recipe Update I

Since no issues were identified by the Recipe validation, push local changes in batch to Recipe-Template to its remote copy via the recipe-update command.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-update --message "Add Pentaho DI node"
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk ru -m "Add Pentaho DI node"
YYYY-MM-DD HH:MM:SS - Updating all changed files in Recipe (Recipe-Template) in Kitchen(Feature_Sprint_3) with message (Add Pentaho DI node)
Update results:

New files:
    pentaho-di-node/description.json
    pentaho-di-node/actions/DKDataSource_Container.json
    pentaho-di-node/actions/docker-share/config.json
    pentaho-di-node/actions/docker-share/create-and-load-superstore-data.ktr
    pentaho-di-node/actions/docker-share/create-schema.ktr
    pentaho-di-node/actions/docker-share/get-count-of-superstore-data.ktr
Updated files:
    graph.json
    variables.json
Deleted files:
    None

Issues:

No issues found

Finally, confirm that the local and remote versions of the Recipe-Template Recipe are now equivalent using the recipe-status command.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
YYYY-MM-DD HH:MM:SS - Getting the status of Recipe 'Recipe-Template' in Kitchen 'Feature_Sprint_3'
    versus directory '~/Kitchens/Feature_Sprint_3/Recipe-Template'
19 files are unchanged

Test Order I

Next, use the order-run command to run an Order for the updated Recipe-Template Recipe.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk order-run Variation1 --yes
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk or Variation1 -y
~/Kitchens $ dk or -k Feature_Sprint_3 -r Recipe-Template Variation1 -y
YYYY-MM-DD HH:MM:SS - Create an Order:
    Kitchen: Feature_Sprint_3
    Recipe: Recipe-Template
    Variation: Variation1

Order ID is: DKRecipe#hw#Recipe-Template#Variation1#Feature_Sprint_3#1508524645286

Review information on the last Order-Run processed in the Feature_Sprint_3 Kitchen using the orderrun-info command.

You can also check the Order Run Details page in the DataKitchen UI for order run results.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk orderrun-info
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk ori
~/Kitchens $ dk ori -k Feature_Sprint_3
YYYY-MM-DD HH:MM:SS - Display Order-Run details from kitchen Feature_Sprint_3

ORDER RUN SUMMARY

Order ID:       DKRecipe#hw#Recipe-Template#Variation1#Feature_Sprint_3#1508524645286
Order Run ID:   ct:1508524645448:0:DKRecipe#hw#Recipe-Template#Variation1#Feature_Sprint_3#1508524645286:
Status:        COMPLETED_SERVING
Kitchen:    Feature_Sprint_3
Recipe:        Recipe-Template
Variation:    Variation1
Start time:    YYYY-MM-DD HH:MM:SS
Run duration:    0:00:32 (H:M:S)
Email confirmation of successful Order-Run of **Recipe-Template**'s **Variation1**.  Note the feedback that two tests passed.

Email confirmation of successful Order-Run of Recipe-Template's Variation1. Note the feedback that two tests passed.

Create New Variation

At present, the Recipe-Template Recipe has a single Variation named Variation1, which we have referenced previously when cooking Recipe Order-Runs. However, should we wish to executed a differently-shaped Recipe graph, or to apply different variable values to our Recipe, we need to create a new Recipe Variation. Leveraging Variations allows us to parameterize our processing, thereby providing runtime flexibility for our Recipes.

In the following steps we will build a new Recipe Variation named Variation2 that will alter the way the Recipe Graph is processed.

Recall that when creating the Recipe-Template Recipe from scratch we started with two placeholder nodes, placeholder-node1 and placeholder-node2, which do no real data work and are solely used for demonstration purposes. Variation2 will omit these nodes altogether from its defined Recipe graph.

Update Graph.json

As Variation2 is meant to alter Recipe-Template's Recipe Graph, first replace the existing content in graph.json with the text below, which defines an alternative set of Recipe nodes and edges.

{
  "nodes": [
    ["placeholder-node1", {}],
    ["placeholder-node2", {}],
    ["load-profits-node", {}],
    ["pentaho-di-node", {}]
  ],
  "edges": [
    ["placeholder-node1", "placeholder-node2", {"weight": 0}],
    ["placeholder-node2", "load-profits-node", {"weight": 0}],
    ["load-profits-node", "pentaho-di-node", {"weight": 0}]
  ],
  "nodes-Variation2": [
    ["load-profits-node", {}],
    ["pentaho-di-node", {}]
  ],
  "edges-Variation2": [
    ["load-profits-node", "pentaho-di-node", {"weight": 0}]
  ]
}
**Recipe-Template**'s **graph.json** file changes after adding **Variation2**.

Recipe-Template's graph.json file changes after adding Variation2.

Update Description.json

Since we wish to provide runtime flexibility for the Recipe graph, such that either set of nodes and edges detailed above can be leveraged, we need to parameterize the nodes and edges variable values, which are located in a Recipe's description.json file. Replace the existing content in Recipe-Template's description.json file with the text below:

{
  "recipe-name":"Recipe-Template",
  "nodes_to_use": "{{nodestouse}}",
  "edges_to_use": "{{edgestouse}}",
  "recipe-emails": {
        "email-delivery-service": "aws-email",
        "recipe-startup-email-recipients": "{{alerts.start}}",
        "recipe-complete-email-recipients": "{{alerts.success}}",
        "recipe-error-email-recipients": "{{alerts.error}}"
  }
}
**Recipe-Template**'s **description.json** file changes after adding **Variation2**.

Recipe-Template's description.json file changes after adding Variation2.

Update Variables.json

Now that the values for the Recipe graph's nodes and edges has been parameterized we need to assign these variables values within the variables.json file for the Recipe. Replace the existing content in Recipe-Template's variables.json file with the text below:

{
    "name": "Recipe-Template QS3 Variables",
    "variable-list": {
        "awsaccountid": "#{vault://aws/account_id}",
        "awsrole": "#{vault://aws/role}",
        "dockerhub": {
              "email": "#{vault://dockerhub/email}",
              "namespace": "#{vault://dockerhub/namespace}",
              "password": "#{vault://dockerhub/password}",
              "username": "#{vault://dockerhub/username}"
        },
        "edgestouse": "edges",
        "nodestouse": "nodes",
        "alerts": {
              "start": "[YOUR EMAIL HERE]", 
              "success": "[YOUR EMAIL HERE]", 
              "error": "[YOUR EMAIL HERE]"
        }
        "redshiftdatabase": "#{vault://redshift/database}",
        "redshifthostname": "#{vault://redshift/hostname}",
        "redshiftpassword": "#{vault://redshift/password}",
        "redshiftport": "#{vault://redshift/port}",
        "redshiftschema": "[YOUR SCHEMA NAME HERE]",
        "redshiftusername": "#{vault://redshift/username}",
        "s3bucketname": "#{vault://s3_schema/bucket}",
        "s3bucketpath": "superstore/customer/sales-profitability"
      }
}

Update Email and Schema Values

Be sure to edit the email and redshiftschema values in the text above to match the values you used earlier in CLI Quickstart3.

The **variables.json** file before and after the addition of **nodes** and **edges** variables.

The variables.json file before and after the addition of nodes and edges variables.

Update Variations.json

Next, update Recipe-Template's variations.json file. To create Variation2, first replace the existing contents of variations.json with the text below:

{
    "active-variation": "Variation1",
    "environment-setting-list": {
    },
    "mesos-setting-list": {
        "production-mesos": {
            "schedule": "now"
        }
    },
    "override-setting-list": {
        "production-overrides": {
          "Override1": "Override-Value1"
        },
        "Variation2-overrides": {
          "edgestouse": "edges-Variation2",
          "nodestouse": "nodes-Variation2"
        }
    },
    "variation-list": {
        "Variation1": {
            "mesos-setting": "production-mesos"
        },
        "Variation2": {
            "mesos-setting": "production-mesos",
            "override-setting": "Variation2-overrides"
        }
    }
}

Recipe Status

Confirm the local changes applied via the recipe-status command.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
YYYY-MM-DD HH:MM:SS - Getting the status of Recipe 'Recipe-Template' in Kitchen 'Feature_Sprint_3'
    versus directory '~/Kitchens/Feature_Sprint_3/Recipe-Template'
4 files are modified on local:
    description.json
    graph.json
    variables.json
    variations.json

22 files are unchanged

File Differential

We have no taken the steps to build a new Recipe Variation named Variation2. Per the recipe-status response above we know that the changes we have applied to Recipe-Template only exist at present in the local copy of the Recipe. We can view the differences in Recipe files between their local and remote copies using the merge and diff tool we configured to work with DKCloudCommand.

We do this via the file-diff command, which brings up a two-pane window highlighting the differences between the local and remote copies of the file specified (in this case variations.json).

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk file-diff variations.json
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk fdi variations.json
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd ../..
~/Kitchens $ dk fdi -k Feature_Sprint_3 -r Recipe-Template variations.json
YYYY-MM-DD HH:MM:SS - File Diff for file variations.json, in Recipe (Recipe-Template) in Kitchen (Feature_Sprint_3)
Executing command: /usr/local/bin/charm diff ~/Kitchens/Feature_Sprint_3/Recipe-Template/variations.json  {USER_HOME}/.dk/diffs/Feature_Sprint_3/Recipe-Template/variations.json

File diff done.
File differential between *local* and *remote* copies of **variations.json**.

File differential between local and remote copies of variations.json.

Recipe Validation II

Next, validate the local changes you have made to the Recipe-Template Recipe by using the recipe-validate command.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-validate --variation Variation1
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rv -v Variation1
YYYY-MM-DD HH:MM:SS - Validating recipe/variation Recipe-Template.Variation1 in Kitchen Feature_Sprint_3
Validating recipe with local changes applied
DKCloudCommand.recipe_validate succeeded
No recipe issues identified.

Recipe Compile

Next, use the recipe-compile command to compile the totality of Recipe-Template's Recipe files, per the Variation2 Variation, to confirm your syntax is correct and parameterization has been implemented correctly.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-compile --variation Variation2
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rc -v Variation2
~/Kitchens $ dk rc -k Feature_Sprint_3 -r Recipe-Template -v Variation2
YYYY-MM-DD HH:MM:SS - Compiling Recipe:
Kitchen:        Feature_Sprint_2
Recipe:         Recipe-Template
Variation:        Variation2

DKCloudCommand.get_compiled_serving succeeded, compiled recipe stored in folder 'compiled-recipe'

As expected, the compiled Recipe within the new, local recipe-compile directory omits the placeholder nodes as Variation2 no longer includes these in its defined Recipe graph.

The **recipe-compile** command generates a new **recipe-compile** directory within the local copy of the Recipe you are editing.

The recipe-compile command generates a new recipe-compile directory within the local copy of the Recipe you are editing.

Delete Local Copy of Compiled Recipe

Be sure to delete the new, local recipe-compile directory with the command below before proceeding with a subsequent Recipe update. We do not wish to push its contents to the remote version of the Recipe-Template Recipe.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ rm -r compiled-recipe

Recipe Update II

Now that we've confirmed that Recipe-Template compiles properly, push your local changes to the remote version of the Recipe using the recipe-update command.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-update --message "Adding Variation2 to remote copy of Recipe-Template"
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk ru -m "Adding Variation2 to remote copy of Recipe-Template"
YYYY-MM-DD HH:MM:SS - Updating all changed files in Recipe (Recipe-Template) in Kitchen(Feature_Sprint_3) with message (Adding Variation2 to remote copy of Recipe-Template)
Update results:

New files:
    None
Updated files:
    description.json
    graph.json
    variations.json
Deleted files:
    None

Issues:

No issues found

Test Order II

Use the order-run command to run an Order for the updated Recipe-Template Recipe, this time leveraging Variation2.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk order-run Variation2 --yes
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk or Variation2 -y
~/Kitchens $ dk or -k Feature_Sprint_3 -r Recipe-Template Variation2 -y
YYYY-MM-DD HH:MM:SS - Create an Order:
    Kitchen: Feature_Sprint_3
    Recipe: Recipe-Template
    Variation: Variation2

Order ID is: DKRecipe#hw#Recipe-Template#Variation2#Feature_Sprint_3#1508539853555

Use the orderrun-info command to view details regarding the last Order-Run in the Feature_Sprint_3 Kitchen. Alternatively, check the Order Run Details page in the DataKitchen UI for your order run results.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk orderrun-info
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk ori
~/Kitchens $ dk ori -k Feature_Sprint_3
YYYY-MM-DD HH:MM:SS - Display Order-Run details from kitchen Feature_Sprint_3

ORDER RUN SUMMARY

Order ID:       DKRecipe#hw#Recipe-Template#Variation2#Feature_Sprint_3#1508539853555
Order Run ID:   ct:1508539853715:0:DKRecipe#hw#Recipe-Template#Variation2#Feature_Sprint_3#1508539853555:
Status:        COMPLETED_SERVING
Kitchen:    Feature_Sprint_3
Recipe:        Recipe-Template
Variation:    Variation2
Start time:    YYYY-MM-DD HH:MM:SS
Run duration:    0:00:24 (H:M:S)
A view of the notification email generated by the successful completion of the Order-Run.

A view of the notification email generated by the successful completion of the Order-Run.

Merge Parent to Child

Kitchen-Merge-Preview

Start the Parent-to-Child Kitchen merge process by using the kitchen-merge-preview command with the --clean_previous_run option.

~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd ../../Dev_Sprint_1
~/Kitchens/Dev_Sprint_1 $ dk kitchen-merge-preview --target_kitchen Feature_Sprint_3 --clean_previous_run
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd ../../Dev_Sprint_1
~/Kitchens/Dev_Sprint_1 $ dk kmp -tk Feature_Sprint_3 -cpr
~/Kitchens $ dk kmp -sk Dev_Sprint_1 -tk Feature_Sprint_3 -cpr
YYYY-MM-DD HH:MM:SS - Previewing merge Kitchen Dev_Sprint_1 into Kitchen Feature_Sprint_3
Merge Preview Results (only changed files are being displayed):
--------------------------------------------------------------

Nothing to merge.

Kitchen merge preview done.

Kitchen-Merge

Since the Parent-to-Child Kitchen merge preview indicated that there was nothing to merge we need not complete a subsequent Parent-to-Child Kitchen merge.

Child Kitchen Test Order

Since no changes were merged into the Recipe-Template Recipe that sits within the Feature_Sprint_3 Child Kitchen there is no need to execute the order-run command to run an Order to gather test results.

Merge Child to Parent

Kitchen-Merge-Preview

Use the kitchen-merge-preview command with the --clean_previous_run option to preview the Child-to-Parent merge of Feature_Sprint_3 into the Dev_Sprint_1 Kitchen.

~/Kitchens/Dev_Sprint_1 $ cd ../Feature_Sprint_3
~/Kitchens/Feature_Sprint_3 $ dk kitchen-merge-preview --target_kitchen Dev_Sprint_1 --clean_previous_run
~/Kitchens/Dev_Sprint_1 $ cd ../Feature_Sprint_3
~/Kitchens/Feature_Sprint_3 $ dk kitchen-merge-preview -tk Dev_Sprint_1 -cpr
~/Kitchens $ dk kitchen-merge-preview -sk Feature_Sprint_3 -tk Dev_Sprint_1 -cpr
YYYY-MM-DD HH:MM:SS - Previewing merge Kitchen Feature_Sprint_3 into Kitchen Dev_Sprint_1
Merge Preview Results (only changed files are being displayed):
--------------------------------------------------------------

      ok                Recipe-Template/pentaho-di-node/actions/docker-share/create-schema.ktr
      ok                Recipe-Template/pentaho-di-node/actions/docker-share/get-count-of-superstore-data.ktr
      ok                Recipe-Template/variations.json
      ok                Recipe-Template/pentaho-di-node/actions/docker-share/create-and-load-superstore-data.ktr
      ok                Recipe-Template/pentaho-di-node/actions/DKDataSource_Container.json
      ok                Recipe-Template/graph.json
      ok                Recipe-Template/pentaho-di-node/actions/docker-share/config.json
      ok                Recipe-Template/pentaho-di-node/description.json
      ok                Recipe-Template/description.json
      ok                Recipe-Template/variables.json

Kitchen merge preview done.

Kitchen-Merge

Since the Child-to-Parent Kitchen merge preview indicated that all files to be merged are free of conflicts we may proceed with the Kitchen merge. Use the kitchen-merge command to merge the updates you've applied to the Recipe-Template Recipe in the Feature_Sprint_3 Child Kitchen into its Parent Kitchen Dev_Sprint_1.

~/Kitchens/Feature_Sprint_3 $ dk kitchen-merge --target_kitchen Dev_Sprint_1 --yes
~/Kitchens/Feature_Sprint_3 $ dk km -tk Dev_Sprint_1 -y
~/Kitchens $ dk km -sk Feature_Sprint_3 -tk Dev_Sprint_1 -y
YYYY-MM-DD HH:MM:SS - Merging Kitchen Feature_Sprint_3 into Kitchen Dev_Sprint_1
looking for manually merged files in temporary directory {USER_HOME}/.dk/merges/Feature_Sprint_3_to_Dev_Sprint_1 
Calling Merge ...
 description.json                              4         ++--                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
 graph.json                                   13         +++++++++++--                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        
 variables.json                               12         ++++++++++--                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
 variations.json                              13         +++++++++----                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        
 config.json                                  44         ++++++++++++++++++++++++++++++++++++++++++++                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
 create-and-load-superstore-data.ktr         765         +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++        
 create-schema.ktr                           594         ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++                                                                                                                                                                                   
 get-count-of-superstore-data.ktr            628         ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++                                                                                                                                                 
 DKDataSource_Container.json                  37         +++++++++++++++++++++++++++++++++++++                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
 description.json                              4         ++++                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
10 files changed, 2104 insertions(+), 10 deletions(-)

List Recipe Variations

Next, use the recipe-variation-list command to confirm that Variation2 has been merged into the Dev_Sprint_1 Kitchen.

~/Kitchens/Feature_Sprint_3 $ cd ../Dev_Sprint_1/Recipe-Template
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk recipe-variation-list
~/Kitchens/Feature_Sprint_3 $ cd ../Dev_Sprint_1/Recipe-Template
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk rvl
~/Kitchens $ dk rvl -k Dev_Sprint_1 -r Recipe-Template
Getting variations from local ...
YYYY-MM-DD HH:MM:SS - Listing variations for recipe Recipe-Template in Kitchen Dev_Sprint_1
Variations:
        Variation1
        Variation2

Parent Test Order

Finally, run a test order using the order-run command to confirm that the merged Recipe-Template Recipe runs as expected in the Dev_Sprint_1 Kitchen.

~/Kitchens/Dev_Sprint_1 $ cd Recipe-Template
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk order-run Variation2 --yes
~/Kitchens/Dev_Sprint_1 $ cd Recipe-Template
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk or Variation2 -y
~/Kitchens/Dev_Sprint_1
~/Kitchens $ dk or -k Dev_Sprint_1 -r Recipe-Template Variation2 -y
YYYY-MM-DD HH:MM:SS - Create an Order:
    Kitchen: Dev_Sprint_1
    Recipe: Recipe-Template
    Variation: Variation2

Order ID is: DKRecipe#hw#Recipe-Template#Variation2#Dev_Sprint_1#1508768898876

Use the orderrun-info command with the - -summary and - -test options to confirm that the Order-Run in the Dev_Sprint_1 Kitchen processed successfully. Alternatively, check the Order Run Details page in the DataKitchen UI for your order run results.

~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk orderrun-info --summary --test
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk ori -s -q
~/Kitchens $ dk ori -k Dev_Sprint_1 -s -q
YYYY-MM-DD HH:MM:SS - Display Order-Run details from kitchen Dev_Sprint_1

ORDER RUN SUMMARY

Order ID:       DKRecipe#hw#Recipe-Template#Variation2#Dev_Sprint_1#1508768898876
Order Run ID:   ct:1508768899024:0:DKRecipe#hw#Recipe-Template#Variation2#Dev_Sprint_1#1508768898876:
Status:         COMPLETED_SERVING
Kitchen:        Dev_Sprint_1
Recipe:         Recipe-Template
Variation:      Variation2
Start time:     YYYY-MM-DD HH:MM:SS
Run duration:   0:00:22 (H:M:S)

TEST RESULTS

Tests: Failed
        No Tests Failed

Tests: Warning
        No Tests Gave Warning Messages

Tests: Log
        No Tests Gave Log Messages

Tests: Passed
   Step (load-profits-node)
      1. test-load-data-key (1589 greater-than-equal-to 1589)
   Step (pentaho-di-node)
      1. get-test-data-count (51289 greater-than 10000)
A notification email confirming the successful processing of the Order-Run of **Variation2** of the **Recipe-Template** Recipe in the **Dev_Sprint** Kitchen.

A notification email confirming the successful processing of the Order-Run of Variation2 of the Recipe-Template Recipe in the Dev_Sprint Kitchen.

Delete Recipe Files

Now that we have successfully merged our updated Recipe-Template Recipe into the Dev_Sprint_1 Parent Kitchen we may teardown the Feature_Sprint_3 Child Kitchen and its associated Recipe copies. However, before we do so, we can practice deleting the remote and local copies of individual Recipe files using the file-delete command.

First, create a throwaway Recipe in the Feature_Sprint_3 Kitchen.

~/Kitchens/Dev_Sprint_1/Recipe-Template $ cd ../../Feature_Sprint_3
~/Kitchens/Feature_Sprint_3 $ dk recipe-create TempRecipe
~/Kitchens/Dev_Sprint_1/Recipe-Template $ cd ../../Feature_Sprint_3
~/Kitchens/Feature_Sprint_3 $ dk re TempRecipe
YYYY-MM-DD HH:MM:SS - Creating Recipe TempRecipe for Kitchen 'Feature_Sprint_3'
list_recipe - elapsed: 1
DKCloudCommand.recipe_create created recipe TempRecipe

Next, get a local copy of TempRecipe.

~/Kitchens/Feature_Sprint_3 $ dk recipe-get TempRecipe
~/Kitchens/Feature_Sprint_3 $ dk rg TempRecipe
YYYY-MM-DD HH:MM:SS - Getting the latest version of Recipe 'TempRecipe' in Kitchen 'Feature_Sprint_3'
DKCloudCommand.get_recipe has 5 sections
  TestRecipe/resources
  TestRecipe
  TestRecipe/placeholder-node2
  TestRecipe/placeholder-node1
  TestRecipe/resources/email-templates

Finally, delete both the local and remote copies of the graph.json file for TempRecipe.

~/Kitchens/Feature_Sprint_3 $ cd TempRecipe
~/Kitchens/Feature_Sprint_3/TempRecipe $ dk file-delete --message "Practice deleting a Recipe file" graph.json
~/Kitchens/Feature_Sprint_3 $ cd TempRecipe
~/Kitchens/Feature_Sprint_3/TempRecipe $ dk fd -m "Practice deleting a Recipe file" graph.json
~/Kitchens/Feature_Sprint_3 $ cd ..
~/Kitchens $ dk fd -k Feature_Sprint_3 -r TempRecipe -m "Practice deleting a Recipe file" graph.json
YYYY-MM-DD HH:MM:SS - Deleting ((u'graph.json',)) in Recipe (TempRecipe) in kitchen(Feature_Sprint_3) with message (Practice deleting a Recipe file)
DKCloudCommand.delete_file for test.txt succeed on local delete
DKCloudCommand.delete_file for test.txt succeed on remote delete

Delete Entire Recipes

We can also delete only the remote copy of a Recipe using the recipe-delete command.

~/Kitchens/Feature_Sprint_3/TempRecipe $ cd ..
~/Kitchens/Feature_Sprint_3 $ dk recipe-delete TempRecipe --yes
~/Kitchens/Feature_Sprint_3/TempRecipe $ cd ..
~/Kitchens/Feature_Sprint_3 $ dk rd TempRecipe -y
~/Kitchens $ dk rd -k Feature_Sprint_3 TempRecipe -y
This command will delete the remote copy of recipe 'TempRecipe' for kitchen 'Feature_Sprint_3'.
YYYY-MM-DD HH:MM:SS - Deleting Recipe TempRecipe for Kitchen 'Feature_Sprint_3'
recipe_delete - elapsed: 0
DKCloudCommand.recipe_delete deleted recipe TempRecipe

Teardown Child Kitchen

Our last step is to teardown the remote copy of the Feature_Sprint_3 Child Kitchen and its Recipes. Use the kitchen-list command to view a list of Kitchens that exist under your company's account.

~/Kitchens/Feature_Sprint_3 $ cd ..
~/Kitchens $ dk kitchen-list
~/Kitchens/Feature_Sprint_3 $ cd ..
~/Kitchens $ dk kl
YYYY-MM-DD HH:MM:SS - Getting the list of kitchens
kitchen-list returned 3 kitchens
+---- master
    +---- Production
            +---- Dev_Sprint_1
                   +---- Feature_Sprint_3

Since we have already merged its contents to the Dev_Sprint_1 Kitchen, we can use the kitchen-delete command to teardown the Feature_Sprint_3 Kitchen. Note that we are deleting the remote copy of the Feature_Sprint_3 Kitchen; we retain a local copy and its Recipe for continued reference.

~/Kitchens $ dk kitchen-delete Feature_Sprint_3 --yes
~/Kitchens $ dk kd Feature_Sprint_3 -y
No child kitchens found.
YYYY-MM-DD HH:MM:SS - Deleting remote copy of kitchen Feature_Sprint_3. Local files will not change.
><
Kitchen Feature_Sprint_3 has been deleted

Wrapping Up

To recap, this CLI Quickstart3 provided an overview of DataOps orchestration via DKCloudCommand, DataKitchen's command line tool. Specific topics covered include:

  • Adding & Configuring a Container-based Node for GUI tools, specifically for Pentaho DI
  • Validating Local Changes Applied to a Recipe
  • Creating Recipe Variations with Variable Overrides to Provide Runtime Flexibility
  • Using Locally Configured Tool to View Local vs. Remote File Differences
  • Compiling Entire Recipes to Check the Proper Application of Parameterization
  • Viewing Lists of Recipe Variations
  • Deleting Remote Copies of Recipe Files
  • Deleting Entire Versions of Remote Recipes

We've now covered how, using the existing tools in your data analytic pipeline, to implement DataOps in your organization via orchestration by the DataKitchen platform. By mirroring the solid DataOps foundation we've quickly built in just these three CLI Quickstart guides, your organization is well on its way to delivering to your stakeholders insights of substantially greater quality at a drastically increased rate of speed!

Synopsis

CHECK THE MENU
~/Kitchens $ dk kitchen-list
~/Kitchens $ cd Dev_Sprint_1
~/Kitchens/Dev_Sprint_1 $ dk recipe-list
~/Kitchens/Dev_Sprint_1 $ cd ..
~/Kitchens $ dk kitchen-create --parent Dev_Sprint_1 Feature_Sprint_3
~/Kitchens $ dk kitchen-get Feature_Sprint_3 --recipe Recipe-Template

ADD PENTAHO NODE - UPDATE VARIABLES.JSON
~/Kitchens $ dk secret-list --recursive
[REPLACE VARIABLES.JSON CONTENT WITH PROVIDED TEXT]
[UPDATE VARIABLES.JSON EMAIL VALUE; CHECK JSON SYNTAX]
~/Kitchens $ cd Feature_Sprint_3/Recipe-Template
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status

ADD PENTAHO NODE - ADD NODE DIRECTORY & DESCRIPTION.JSON
~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/description.json
[PASTE SUPPLIED TEXT AT COMMAND LINE, <ENTER>, <CTRL+D>]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status

ADD PENTAHO NODE - RETAIN VARIATIONS.JSON
[NO STEPS REQUIRED - WILL WILL ADD AN ADDITIONAL RECIPE VARIATION LATER]

ADD PENTAHO NODE - CONFIGURE NODE
~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node/actions
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/actions/DKDataSource_Container.json
[PASTE SUPPLIED TEXT AT COMMAND LINE, <ENTER>, <CTRL+D>]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status
~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node/actions/docker-share
/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/actions/docker-share/config.json
[PASTE SUPPLIED TEXT AT COMMAND LINE, <ENTER>, <CTRL+D>]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status

ADD PENTAHO NODE - ADD TOOLCHAIN FILES
[ADD FOUR .ktr FILES TO /docker-share DIRECTORY WITH CURL COMMANDS]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd pentaho-di-node/actions/docker-share
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file1)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file2)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file3)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ cd ../../..
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status

ADD PENTAHO NODE - UPDATE GRAPH.JSON
[REPLACE GRAPH.JSON CONTENT WITH PROVIDED TEXT]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status

ADD PENTAHO NODE - RECIPE VALIDATION I
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-validate --variation Variation1

ADD PENTAHO NODE - RECIPE UPDATE I
~Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-update --message "Change message"
~Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status

ADD PENTAHO NODE - TEST ORDER I
~Kitchens/Feature_Sprint_3/Recipe-Template $ dk order-run Variation1 --yes
~Kitchens/Feature_Sprint_3/Recipe-Template $ dk orderrun-info

CREATE NEW RECIPE VARIATION
[REPLACE GRAPH.JSON CONTENT WITH PROVIDED TEXT]
[REPLACE DESCRIPTION.JSON CONTENT WITH PROVIDED TEXT]
[REPLACE VARIATIONS.JSON CONTENT WITH PROVIDED TEXT]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-status

CREATE NEW RECIPE VARIATION - FILE DIFFERENTIAL
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk file-diff variations.json

CREATE NEW RECIPE VARIATION - RECIPE VALIDATION II
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-validate --variation Variation1

CREATE NEW RECIPE VARIATION - RECIPE COMPILE
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-compile --variation Variation2
~/Kitchens/Feature_Sprint_3/Recipe-Template $ rm -r compiled-recipe/

CREATE NEW VARIATION - RECIPE UPDATE II
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk recipe-update --message "Change message"

CREATE NEW VARIATION - TEST ORDER II
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk order-run Variation2 --yes
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk orderrun-info

MERGE PARENT TO CHILD
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd ../../Dev_Sprint_1
~/Kitchens/Dev_Sprint_1 $ dk kitchen-merge-preview --target_kitchen Feature_Sprint_3 --clean_previous_run

MERGE CHILD TO PARENT
~/Kitchens/Dev_Sprint_1 $ cd ../Feature_Sprint_3
~/Kitchens/Feature_Sprint_3 $ dk kitchen-merge-preview --target_kitchen Dev_Sprint_1 --clean_previous_run
~/Kitchens/Feature_Sprint_3 $ dk kitchen-merge --target_kitchen Dev_Sprint_1 --yes

LIST RECIPE VARIATIONS
~/Kitchens/Feature_Sprint_3 $ cd ../Dev_Sprint_1/Recipe-Template
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk recipe-variation-list

PARENT TEST ORDER
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk order-run Variation2 --yes
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk orderrun-info --summary --test

DELETE RECIPE FILES
~/Kitchens/Dev_Sprint_1/Recipe-Template $ cd ../../Feature_Sprint_3
~/Kitchens/Feature_Sprint_3 $ dk recipe-create TempRecipe
~/Kitchens/Feature_Sprint_3 $ dk recipe-get TempRecipe
~/Kitchens/Feature_Sprint_3 $ cd TempRecipe
~/Kitchens/Feature_Sprint_3/TempRecipe $ dk file-delete --message "Change message" graph.json

DELETE ENTIRE RECIPE
~/Kitchens/Feature_Sprint_3/TempRecipe $ cd ..
~/Kitchens/Feature_Sprint_3 $ dk recipe-delete TempRecipe --yes

TEARDOWN CHILD KITCHEN
~/Kitchens/Feature_Sprint_3 $ cd ..
~/Kitchens $ dk kitchen-list
~/Kitchens $ dk kitchen-delete Feature_Sprint_3 --yes
CHECK THE MENU
~/Kitchens $ dk kl
~/Kitchens $ cd Dev_Sprint_1
~/Kitchens/Dev_Sprint_1 $ dk rl
~/Kitchens/Dev_Sprint_1 $ cd ..
~/Kitchens $ dk kc -p Dev_Sprint_1 Feature_Sprint_3
~/Kitchens $ dk kg Feature_Sprint_3 -r Recipe-Template

ADD PENTAHO NODE - UPDATE VARIABLES.JSON
~/Kitchens $ dk sl -rc
[REPLACE VARIABLES.JSON CONTENT WITH PROVIDED TEXT]
[UPDATE VARIABLES.JSON EMAIL VALUE; CHECK JSON SYNTAX]
~/Kitchens $ cd Feature_Sprint_3/Recipe-Template
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - ADD NODE DIRECTORY & DESCRIPTION.JSON
~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/description.json
[PASTE SUPPLIED TEXT AT COMMAND LINE, <ENTER>, <CTRL+D>]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - RETAIN VARIATIONS.JSON
[NO STEPS REQUIRED - WILL WILL ADD AN ADDITIONAL RECIPE VARIATION LATER]

ADD PENTAHO NODE - CONFIGURE NODE
~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node/actions
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/actions/DKDataSource_Container.json
[PASTE SUPPLIED TEXT AT COMMAND LINE, <ENTER>, <CTRL+D>]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node/actions/docker-share
/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/actions/docker-share/config.json
[PASTE SUPPLIED TEXT AT COMMAND LINE, <ENTER>, <CTRL+D>]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - ADD TOOLCHAIN FILES
[ADD FOUR .ktr FILES TO /docker-share DIRECTORY WITH CURL COMMANDS]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd pentaho-di-node/actions/docker-share
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file1)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file2)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file3)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ cd ../../..
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - UPDATE GRAPH.JSON
[REPLACE GRAPH.JSON CONTENT WITH PROVIDED TEXT]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - RECIPE VALIDATION I
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rv -v Variation1

ADD PENTAHO NODE - RECIPE UPDATE I
~Kitchens/Feature_Sprint_3/Recipe-Template $ dk ru -m "Change message"
~Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - TEST ORDER I
~Kitchens/Feature_Sprint_3/Recipe-Template $ dk or Variation1 -y
~Kitchens/Feature_Sprint_3/Recipe-Template $ dk ori

CREATE NEW RECIPE VARIATION
[REPLACE GRAPH.JSON CONTENT WITH PROVIDED TEXT]
[REPLACE DESCRIPTION.JSON CONTENT WITH PROVIDED TEXT]
[REPLACE VARIATIONS.JSON CONTENT WITH PROVIDED TEXT]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

CREATE NEW RECIPE VARIATION - FILE DIFFERENTIAL
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk fdi variations.json

CREATE NEW RECIPE VARIATION - RECIPE VALIDATION II
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rv -v Variation1

CREATE NEW RECIPE VARIATION - RECIPE COMPILE
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rc -v Variation2
~/Kitchens/Feature_Sprint_3/Recipe-Template $ rm -r compiled-recipe/

CREATE NEW VARIATION - RECIPE UPDATE II
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk ru -m "Change message"

CREATE NEW VARIATION - TEST ORDER II
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk or Variation2 -y
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk ori

MERGE PARENT TO CHILD
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd ../../Dev_Sprint_1
~/Kitchens/Dev_Sprint_1 $ dk kmp -tk Feature_Sprint_3 -cpr

MERGE CHILD TO PARENT
~/Kitchens/Dev_Sprint_1 $ cd ../Feature_Sprint_3
~/Kitchens/Feature_Sprint_3 $ dk kmp -tk Dev_Sprint_1 -cpr
~/Kitchens/Feature_Sprint_3 $ dk km -tk Dev_Sprint_1 -y

LIST RECIPE VARIATIONS
~/Kitchens/Feature_Sprint_3 $ cd ../Dev_Sprint_1/Recipe-Template
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk rvl

PARENT TEST ORDER
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk or Variation2 -y
~/Kitchens/Dev_Sprint_1/Recipe-Template $ dk ori -s -q

DELETE RECIPE FILES
~/Kitchens/Dev_Sprint_1/Recipe-Template $ cd ../../Feature_Sprint_3
~/Kitchens/Feature_Sprint_3 $ dk re TempRecipe
~/Kitchens/Feature_Sprint_3 $ dk rg TempRecipe
~/Kitchens/Feature_Sprint_3 $ cd TempRecipe
~/Kitchens/Feature_Sprint_3/TempRecipe $ dk fd -m "Change message" graph.json

DELETE ENTIRE RECIPE
~/Kitchens/Feature_Sprint_3/TempRecipe $ cd ..
~/Kitchens/Feature_Sprint_3 $ dk rd TempRecipe -y

TEARDOWN CHILD KITCHEN
~/Kitchens/Feature_Sprint_3 $ cd ..
~/Kitchens $ dk kl
~/Kitchens $ dk kid Feature_Sprint_3 -y
CHECK THE MENU
~/Kitchens $ dk kl
~/Kitchens $ dk rl -k Dev_Sprint_1
~/Kitchens $ dk kc -p Dev_Sprint_1 Feature_Sprint_3
~/Kitchens $ dk kg Feature_Sprint_3 -r Recipe-Template

ADD PENTAHO NODE - UPDATE VARIABLES.JSON
~/Kitchens $ dk sl -rc
[REPLACE VARIABLES.JSON CONTENT WITH PROVIDED TEXT]
[UPDATE VARIABLES.JSON EMAIL VALUE; CHECK JSON SYNTAX]
~/Kitchens $ cd Feature_Sprint_3/Recipe-Template
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - ADD NODE DIRECTORY & DESCRIPTION.JSON
~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/description.json
[PASTE SUPPLIED TEXT AT COMMAND LINE, <ENTER>, <CTRL+D>]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - UPDATE VARIATIONS.JSON
[NO STEPS REQUIRED - WILL WILL ADD AN ADDITIONAL RECIPE VARIATION LATER]

ADD PENTAHO NODE - CONFIGURE NODE
~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node/actions
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/actions/DKDataSource_Container.json
[PASTE SUPPLIED TEXT AT COMMAND LINE, <ENTER>, <CTRL+D>]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
~/Kitchens/Feature_Sprint_3/Recipe-Template $ mkdir pentaho-di-node/actions/docker-share
/Kitchens/Feature_Sprint_3/Recipe-Template $ cat > pentaho-di-node/actions/docker-share/config.json
[PASTE SUPPLIED TEXT AT COMMAND LINE, <ENTER>, <CTRL+D>]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - ADD PENTAHO DI .ktr FILES
[ADD FOUR .ktr FILES TO /docker-share DIRECTORY WITH CURL COMMANDS]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd pentaho-di-node/actions/docker-share
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file1)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file2)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ curl -L -O (link to raw public repo file3)
~/Kitchens/Feature_Sprint_3/Recipe-Template/pentaho-di-node/actions/docker-share $ cd ../../..
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - UPDATE GRAPH.JSON
[REPLACE GRAPH.JSON CONTENT WITH PROVIDED TEXT]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - RECIPE VALIDATION I
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rv -v Variation1

ADD PENTAHO NODE - RECIPE UPDATE I
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk ru -m "Change message"
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

ADD PENTAHO NODE - TEST ORDER I
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd ../..
~/Kitchens $ dk or -k Feature_Sprint_3 -r Recipe-Template Variation1 -y
~/Kitchens $ dk ori -k Feature_Sprint_3

CREATE NEW RECIPE VARIATION
[REPLACE GRAPH.JSON CONTENT WITH PROVIDED TEXT]
[REPLACE DESCRIPTION.JSON CONTENT WITH PROVIDED TEXT]
[REPLACE VARIATIONS.JSON CONTENT WITH PROVIDED TEXT]
~/Kitchens $ cd Feature_Sprint_3/Recipe-Template
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

CREATE NEW RECIPE VARIATION - FILE DIFFERENTIAL
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd ../..
~/Kitchens $ dk fdi -k Feature_Sprint_3 -r Recipe-Template variations.json

CREATE NEW VARIATION - RECIPE COMPILE
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd ../..
~/Kitchens $ dk rc -k Feature_Sprint_3 -r Recipe-Template -v Variation2
~/Kitchens $ rm -r /Feature_Sprint_3/Recipe-Template/compiled-recipe/

CREATE NEW VARIATION - RECIPE UPDATE II
~/Kitchens $ cd Feature_Sprint_3/Recipe-Template
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk ru -m "Change message"

CREATE NEW VARIATION - TEST ORDER II
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd ../..
~/Kitchens $ dk or -k Feature_Sprint_3 -r Recipe-Template Variation2 -y
~/Kitchens $ dk ori -k Feature_Sprint_3

MERGE PARENT TO CHILD & RUN TEST ORDER
~/Kitchens $ dk kmp -sk Dev_Sprint_1 -tk Feature_Sprint_3 -cpr

MERGE CHILD TO PARENT & RUN TEST ORDER
~/Kitchens $ dk kmp -sk Feature_Sprint_3 -tk Dev_Sprint_1 -cpr
~/Kitchens $ dk km -sk Feature_Sprint_3 -tk Dev_Sprint_1 -y

LIST RECIPE VARIATIONS
~/Kitchens $ dk rvl -k Dev_Sprint_1 -r Recipe-Template

PARENT TEST ORDER
~/Kitchens $ dk or -k Dev_Sprint_1 -r Recipe-Template Variation2 -y
~/Kitchens $ dk ori -k Dev_Sprint_1 -s -q

DELETE RECIPE FILES
~/Kitchens $ dk re -k Feature_Sprint_3 TempRecipe
~/Kitchens $ dk fd -k Feature_Sprint_3 -r TempRecipe -m "Change message" graph.json

DELETE ENTIRE RECIPE
~/Kitchens $ dk rd -k Feature_Sprint_3 TempRecipe -y

TEARDOWN CHILD KITCHEN
~/Kitchens $ dk kl
~/Kitchens $ dk kd Feature_Sprint_3 -y
CHECK THE MENU
~/Kitchens $ dk kl
~/Kitchens $ dk rl -k Dev_Sprint_1
~/Kitchens $ dk kc -p Dev_Sprint_1 Feature_Sprint_3
~/Kitchens $ dk kg Feature_Sprint_3 -r Recipe-Template

DELETE QS2 RECIPE & CREATE QS3 RECIPE FROM TEMPLATE
[SKIPS STEPS TO BUILD FROM SCRATCH QS3 RECIPE-TEMPLATE VIA QS2 RECIPE-TEMPLATE]
~/Kitchens $ cd Feature_Sprint_3
~/Kitchens/Feature_Sprint_3 $ dk rd Recipe-Template -y
~/Kitchens/Feature_Sprint_3 $ rm -r Recipe-Template
~/Kitchens/Feature_Sprint_3 $ dk re -tm qs3 Recipe-Template
~/Kitchens/Feature_Sprint_3 $ dk rg Recipe-Template

MAKE LOCAL RECIPE CHANGES AND UPDATE TO REMOTE
~/Kitchens/Feature_Sprint_2 $ cd Recipe-Template
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs
[UPDATE EMAIL VALUE IN VARIABLES.JSON]
[UPDATE REDSHIFT SCHEMA VALUE IN VARIABLES.JSON - e.g. "jdoe"]
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk fdi variables.json
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rv -v Variation1
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk ru -m "Push local changes to remote"
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rs

VIEW RECIPE VARIATIONS
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rvl
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cat variations.json

RECIPE COMPILE
~/Kitchens/Feature_Sprint_3/Recipe-Template $ dk rc -v Variation2
~/Kitchens/Feature_Sprint_3/Recipe-Template $ rm -r compiled-recipe/

TEST ORDER
~/Kitchens/Feature_Sprint_3/Recipe-Template $ cd ../..
~/Kitchens $ dk or -k Feature_Sprint_3 -r Recipe-Template Variation2 -y
~/Kitchens $ dk ori -k Feature_Sprint_3

MERGE PARENT TO CHILD
~/Kitchens $ dk kmp -sk Dev_Sprint_1 -tk Feature_Sprint_3 -cpr

MERGE CHILD TO PARENT
~/Kitchens $ dk kmp -sk Feature_Sprint_3 -tk Dev_Sprint_1 -cpr
~/Kitchens $ dk km -sk Feature_Sprint_3 -tk Dev_Sprint_1 -y

PARENT TEST ORDER
~/Kitchens $ dk or -k Dev_Sprint_1 -r Recipe-Template Variation2 -y
~/Kitchens $ dk ori -k Dev_Sprint_1 -s -q

DELETE RECIPE FILES
~/Kitchens $ dk re -k Feature_Sprint_3 TempRecipe
~/Kitchens $ dk fd -k Feature_Sprint_3 -r TempRecipe -m "Change message" graph.json

DELETE ENTIRE RECIPE
~/Kitchens $ dk rd -k Feature_Sprint_3 TempRecipe -y

TEARDOWN CHILD KITCHEN
~/Kitchens $ dk kl
~/Kitchens $ dk kd Feature_Sprint_3 -y

Updated 6 days ago

CLI Quickstart3


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.