#!/bin/bash
MODE=$1
if [[ -z $2 ]];then HEADTAIL_LIMIT=0; else HEADTAIL_LIMIT=$2;fi
cd "$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )/../.."
source scripts/funcs

echo "## Running a task locally

In this section, we will learn how to run a task with handoff locally,
using project 01_word_count.
"

Prompt "Ready? (Hit q at prompt to quit anytime. Any other key to proceed.)"

echo "
### project.yml

Each project directory contains:
"

ListDir "01_word_count"

Prompt

echo "
project.yml looks like:
"

Cat "01_word_count/project.yml"

Prompt

echo '
Here,

- `tasks` is a list defining each task
- Under the `tasks`, there is a `pipeline`
- `pipeline` lists commands and their arguments
'

echo "
The example from 01_word_count runs a command line equivalent of:
"

Code "cat files/the_great_dictator_speech.txt | wc -w"

echo '
As indicated by `|`, cat writes out the content of the file to the stdin, and
it is picked up by wc command through Unix pipeline and it counts the number
of words.'

echo "
Now let's run. Try entering this command below:
"

Command "handoff run local --project 01_word_count --workspace workspace"

Prompt

echo "
If you see the output that looks like:
"

Code "
[2020-12-28 22:02:09,182] [ WARNING] - 01_word_count/.secrets/secrets.yml does not exsist - (admin.py:223)\n
[2020-12-28 22:02:09,297] [    INFO] - Found credentials in shared credentials file: ~/.aws/credentials - (credentials.py:1182)\n
[2020-12-28 22:02:09,634] [ WARNING] - Environment variable HO_BUCKET is not set. Remote file read/write will fail. - (admin.py:132)\n
[2020-12-28 22:02:09,639] [    INFO] - Job started at 2020-12-28 22:02:09.639435 - (__init__.py:178)\n
[2020-12-28 22:02:09,639] [    INFO] - Running pipeline word_count - (operators.py:193)\n
[2020-12-28 22:02:09,647] [    INFO] - Checking return code of pid 2900 - (operators.py:262)\n
[2020-12-28 22:02:09,647] [    INFO] - Checking return code of pid 2901 - (operators.py:262)\n
[2020-12-28 22:02:09,648] [    INFO] - Pipeline word_count exited with code 0 - (task.py:32)\n
[2020-12-28 22:02:09,648] [    INFO] - Job ended at 2020-12-28 22:02:09.648888 - (__init__.py:184)\n
[2020-12-28 22:02:09,648] [    INFO] - Processed in 0:00:00.009453 - (__init__.py:186)\n
"

Prompt

echo "
Then great! You just ran handoff task locally.
    
For now, let's not worry about the warnings in the log such as:
"

Code "\n
[WARNING] - 01_word_count/.secrets/secrets.yml does not exsist
"

echo "
This is for using handoff's template feature with secret variables.

"

Code "\n
[WARNING] - Environment variable HO_BUCKET is not set. Remote file read/write will fail.
"

echo "
This is for running the task with remotely stored configurations and files.
I will explain in the later section.
"

Prompt

echo "
### Output in workspace directory

The previous run created a workspace directory that looks like:
"

ListDir workspace

echo "
And the word count is stored at workspace/artifacts/word_count_stdout.log. Here is the content:
"

Cat "workspace/artifacts/word_count_stdout.log"

Prompt

echo "
By the way, the example text is from the awesome speech by Charlie Chaplin's
in the movie the Great Dictator.

Here is a link to the famous speech scene.
Check out on YouTube: https://www.youtube.com/watch?v=J7GY1Xg6X20
"

Prompt

echo "
And here is the first few paragraphs of the text:
"

Run "head -n 4 01_word_count/files/the_great_dictator_speech.txt" $BLUE 0

Prompt

echo "
### Variables

You can define in-memory and environment variables.
The second project 02_commands_and_vars is an example.
This time project.yml looks like:
"

Cat 02_commands_and_vars/project.yml

Prompt

echo '
There are a couple of new sections.
`vars` section contains key-value pairs. At run-time they are defined as
in-memory variables. The key-values in `envs` section will be defined as
envrionment variables.
'

Prompt

echo '
### Commands vs. pipeline

In this file, there are two tasks: word_count and show_content.
The first task is the same as the previous example, but it is deactivated.
You can active and deactivate tasks and commands by setting `active: False`.
If it is not set, the task or command is active.

The second one has commands instead of pipeline. Instead of running the
commands as a pipeline, it runs them as independent commands just like:
'

Code 'echo $TITLE && head -n {{ file_path }}'

Prompt

echo '
As indicated by `&&`, the second command only runs if the first runs
successfully, exiting with code 0.

{{ file_path }} will be replaced by the corresponding variable defined in vars.
'

Prompt

echo "
Now let's run. Try entering this command below. Notice that I'm using
shorthands for options:
"

Command "handoff run local -p 02_commands_and_vars -w workspace" $DIM 0

echo "
Let's check out the contents of the second command:
"

Prompt

Cat "workspace/artifacts/show_content_stdout.log" $DIM 0

Prompt

echo "
### Secrets

We intend the files under project directory to be stored in a code
repository such as git. So we don't want to store sensitive information such
as password.

.secrets/secrets.yml instead is the place to keep the secrets.
03_secrets project has the secrets defined and the file looks like this:
"

Cat "03_secrets/.secrets/secrets.yml"

Prompt

echo "
Let's see how we can use secrets in the project:
"

Cat "03_secrets/project.yml"

echo "
You can see {{ username }} and {{ password }} in the curl command. This project
does not actually run curl command. It just writes out the command in
stdout.log file.

Now let's run.
"

Command "handoff run local -p 03_secrets -w workspace" $DIM 0

echo "
show_curl_command_stdout.log shows the curl command with the actual username
and password:
"

Cat "workspace/artifacts/show_curl_command_stdout.log"

Prompt

echo "
### External secret file

You can refer an external file as the secret value. 03_secrets project's files
folder contains google_client_secret.json. This is a RSA file for Google
Cloud Platform's
[service account](https://cloud.google.com/iam/docs/service-accounts#service_account_keys)
whose values are replaced with fake ones.

The original file under the project directory  does not contain the value:
"

Cat "03_secrets/files/google_client_secret.json"

Prompt

echo "
During the run, google_client_secret.json is copied to workspace/files. When that happens,
{{ google_client_secret }} is replaced with the actual values:
"

Cat "workspace/files/google_client_secret.json"

Prompt

echo "
**Important:**

The content of workspace directory is created at run-time, and you should not
submit them in the code repository because sensitive information may be
inserted.

With a same reason, you should not commit .secrets directory to a code
repository. The content of secrets.yml will be uploaded to a secure remote
storage when the task run in the cloud platform such as AWS Fargate. More
about this later.

We recommand adding \"workspace\" and \".secrets\"to .gitigonore file to avoid
accidentally checking them in.
"

Prompt

Thanks

echo "
In the next section, we will try pulling the currency exchange rate data.
You will also learn how to create Python virtual enviroments for each command
and install programs.
"

Continue scripts/aws_get_started/02_venv_and_install
