Run YesWorkflow Docker Image on Ubuntu 20.04

YesWorkflow

YesWorkflow is a modeling and provenance management tool for scripting languages.

Run Docker image of YesWorkflow

Download YesWorkflow repository from: https://github.com/yesworkflow-org/yw-docker
Open a terminal in yw-cli directory.
Create a folder named codes inside current directory.

Create a file example.py with the following content in the codes directory:

import netCDF4
import numpy as np
from netCDF4 import ma
import matplotlib.pyplot as plt
from matplotlib.backends.backend_pdf import PdfPages

# @BEGIN main
# @PARAM db_pth
# @PARAM fmodel
# @IN input_mask_file  @URI file:{db_pth}/land_water_mask/LandWaterMask_Global_CRUNCEP.nc
# @IN input_data_file  @URI file:{db_pth}/NEE_first_year.nc
# @OUT result_NEE_pdf  @URI file:result_NEE.pdf

def main(db_pth = '.', fmodel = 'clm'):

    # @BEGIN fetch_mask
    # @PARAM db_pth
    # @IN g  @AS input_mask_file  @URI file:{db_pth}/land_water_mask/LandWaterMask_Global_CRUNCEP.nc
    # @OUT mask  @AS land_water_mask
    g = netCDF4.Dataset(db_pth+'/land_water_mask/LandWaterMask_Global_CRUNCEP.nc', 'r')
    mask = g.variables['land_water_mask']
    mask = mask[:].swapaxes(0,1)
    # @END fetch_mask


    # @BEGIN load_data
    # @PARAM db_pth
    # @IN input_data_file  @URI file:{db_pth}/NEE_first_year.nc
    # @OUT data  @AS NEE_data
    f = netCDF4.Dataset(db_pth+'/NEE_first_year.nc', 'r')
    data = f.variables['NEE']
    data = data[:]
    data = data.swapaxes(0,2)
    adj = 60*60*24*(365/12)*1000
    data = data*adj
    # @END load_data


    # @BEGIN standardize_with_mask
    # @IN data @AS NEE_data
    # @IN mask @AS land_water_mask
    # @OUT data @AS standardized_NEE_data
    native = data.mean(2)
    latShape = mask.shape[0]
    logShape = mask.shape[1]
    for x in range(latShape):
        for y in range(logShape):
            if mask[x,y] == 1 and ma.getmask(native[x,y]) == 1:
                for index in range(data.shape[2]):
                    data[x,y,index] = 0
    # @END standardize_with_mask


    # @BEGIN simple_diagnose
    # @PARAM fmodel
    # @IN data @AS standardized_NEE_data
    # @OUT pp  @AS result_NEE_pdf  @URI file:result_NEE.pdf
    plt.imshow(np.mean(data,2))
    plt.xlabel("Mean 1982-2010 NEE [gC/m2/mon]")
    plt.title(fmodel + ":BG1")
    pp = PdfPages('result_NEE.pdf')
    pp.savefig()
    pp.close()    
    # @END simple_diagnose

# @END main

Current directory structure:

.
├── build.sh
├── codes
│   └── example.py
├── Dockerfile
├── run.ps1
└── run.sh

Build Docker image based on the Dockerfile:
```
docker build -t yesworkflow .
```
Run Docker image by mounting the codes directory as volume:
```
docker run --rm -it -v $(pwd)/codes:/codes yesworkflow
```
This will open a session in YesWorkflow environment:
```
yw@be484cd73b89:~$ pwd
/home/yw
```
Change directory to the codes directory in YesWorkflow session:
```
cd /codes
```
Creating a workflow graph for a script: We will use the graph command to produce a graphical representations of the script based on the YW comments it contains. Run the example python program:
```
yw graph example.py
```
It will output the result to the terminal.
Storing output to file: YesWorkflow natively outputs GraphViz’s DOT format (file extension .gv). We can store the output to a file example.gv:
```
yw graph example.py>example.gv
```
Generating Graph PDF: Render DOT output file (example.gv) as PDF file using Graphviz’s dot command:
```
dot -Tpdf example.gv -o example.pdf
```
Check the \codes folder and it will contain the graph pdf file example.pdf.
- Directory structure after creating the DOT file and PDF file:
```
.
├── build.sh
├── codes
│   ├── example.gv
│   ├── example.pdf
│   └── example.py
├── Dockerfile
├── run.ps1
└── run.sh
```
Exit YesWorkflow session: To exit the session press ctrl+d.
The output PDF example.pdf:

Run YesWorkflow Docker Image on Ubuntu 20.04

By Ahmedur Rahman Shovon

YesWorkflow

Run Docker image of YesWorkflow

Reference

Cite This Work

Related Contents in this Website

Previous Post

Next Post

Ahmedur Rahman Shovon