Run YesWorkflow Docker Image on Ubuntu 20.04

Published on , Updated on installation ubuntu yesworkflow docker

YesWorkflow

YesWorkflow is a modeling and provenance management tool for scripting languages.

Run Docker image of YesWorkflow

  • Download YesWorkflow repository from: https://github.com/yesworkflow-org/yw-docker
  • Open a terminal in yw-cli directory.
  • Create a folder named codes inside current directory.
  • Create a file example.py with the following content in the codes directory:

    import netCDF4
    import numpy as np
    from netCDF4 import ma
    import matplotlib.pyplot as plt
    from matplotlib.backends.backend_pdf import PdfPages
    
    # @BEGIN main
    # @PARAM db_pth
    # @PARAM fmodel
    # @IN input_mask_file  @URI file:{db_pth}/land_water_mask/LandWaterMask_Global_CRUNCEP.nc
    # @IN input_data_file  @URI file:{db_pth}/NEE_first_year.nc
    # @OUT result_NEE_pdf  @URI file:result_NEE.pdf
    
    def main(db_pth = '.', fmodel = 'clm'):
    
        # @BEGIN fetch_mask
        # @PARAM db_pth
        # @IN g  @AS input_mask_file  @URI file:{db_pth}/land_water_mask/LandWaterMask_Global_CRUNCEP.nc
        # @OUT mask  @AS land_water_mask
        g = netCDF4.Dataset(db_pth+'/land_water_mask/LandWaterMask_Global_CRUNCEP.nc', 'r')
        mask = g.variables['land_water_mask']
        mask = mask[:].swapaxes(0,1)
        # @END fetch_mask
    
    
        # @BEGIN load_data
        # @PARAM db_pth
        # @IN input_data_file  @URI file:{db_pth}/NEE_first_year.nc
        # @OUT data  @AS NEE_data
        f = netCDF4.Dataset(db_pth+'/NEE_first_year.nc', 'r')
        data = f.variables['NEE']
        data = data[:]
        data = data.swapaxes(0,2)
        adj = 60*60*24*(365/12)*1000
        data = data*adj
        # @END load_data
    
    
        # @BEGIN standardize_with_mask
        # @IN data @AS NEE_data
        # @IN mask @AS land_water_mask
        # @OUT data @AS standardized_NEE_data
        native = data.mean(2)
        latShape = mask.shape[0]
        logShape = mask.shape[1]
        for x in range(latShape):
            for y in range(logShape):
                if mask[x,y] == 1 and ma.getmask(native[x,y]) == 1:
                    for index in range(data.shape[2]):
                        data[x,y,index] = 0
        # @END standardize_with_mask
    
    
        # @BEGIN simple_diagnose
        # @PARAM fmodel
        # @IN data @AS standardized_NEE_data
        # @OUT pp  @AS result_NEE_pdf  @URI file:result_NEE.pdf
        plt.imshow(np.mean(data,2))
        plt.xlabel("Mean 1982-2010 NEE [gC/m2/mon]")
        plt.title(fmodel + ":BG1")
        pp = PdfPages('result_NEE.pdf')
        pp.savefig()
        pp.close()    
        # @END simple_diagnose
    
    # @END main
    
  • Current directory structure:

    .
    ├── build.sh
    ├── codes
    │   └── example.py
    ├── Dockerfile
    ├── run.ps1
    └── run.sh
    
  • Build Docker image based on the Dockerfile:

    docker build -t yesworkflow .
    
  • Run Docker image by mounting the codes directory as volume:

    docker run --rm -it -v $(pwd)/codes:/codes yesworkflow
    

    This will open a session in YesWorkflow environment:

    yw@be484cd73b89:~$ pwd
    /home/yw
    
  • Change directory to the codes directory in YesWorkflow session:

    cd /codes
    
  • Creating a workflow graph for a script: We will use the graph command to produce a graphical representations of the script based on the YW comments it contains. Run the example python program:

    yw graph example.py
    

    It will output the result to the terminal.

  • Storing output to file: YesWorkflow natively outputs GraphViz’s DOT format (file extension .gv). We can store the output to a file example.gv:

    yw graph example.py>example.gv
    
  • Generating Graph PDF: Render DOT output file (example.gv) as PDF file using Graphviz’s dot command:

    dot -Tpdf example.gv -o example.pdf
    

    Check the \codes folder and it will contain the graph pdf file example.pdf.

    • Directory structure after creating the DOT file and PDF file:

      .
      ├── build.sh
      ├── codes
      │   ├── example.gv
      │   ├── example.pdf
      │   └── example.py
      ├── Dockerfile
      ├── run.ps1
      └── run.sh
      
      
  • Exit YesWorkflow session: To exit the session press ctrl+d.

  • The output PDF example.pdf: alt Run YesWorkflow Docker Image on Ubuntu 20.04

Reference

Related Contents