YesWorkflow

YesWorkflow is a modeling and provenance management tool for scripting languages.

Run Docker image of YesWorkflow

  • Download YesWorkflow repository from: https://github.com/yesworkflow-org/yw-docker
  • Open a terminal in yw-cli directory.
  • Create a folder named codes inside current directory.
  • Create a file example.py with the following content in the codes directory:

    import netCDF4
    import numpy as np
    from netCDF4 import ma
    import matplotlib.pyplot as plt
    from matplotlib.backends.backend_pdf import PdfPages
    
    # @BEGIN main
    # @PARAM db_pth
    # @PARAM fmodel
    # @IN input_mask_file  @URI file:{db_pth}/land_water_mask/LandWaterMask_Global_CRUNCEP.nc
    # @IN input_data_file  @URI file:{db_pth}/NEE_first_year.nc
    # @OUT result_NEE_pdf  @URI file:result_NEE.pdf
    
    def main(db_pth = '.', fmodel = 'clm'):
    
        # @BEGIN fetch_mask
        # @PARAM db_pth
        # @IN g  @AS input_mask_file  @URI file:{db_pth}/land_water_mask/LandWaterMask_Global_CRUNCEP.nc
        # @OUT mask  @AS land_water_mask
        g = netCDF4.Dataset(db_pth+'/land_water_mask/LandWaterMask_Global_CRUNCEP.nc', 'r')
        mask = g.variables['land_water_mask']
        mask = mask[:].swapaxes(0,1)
        # @END fetch_mask
    
    
        # @BEGIN load_data
        # @PARAM db_pth
        # @IN input_data_file  @URI file:{db_pth}/NEE_first_year.nc
        # @OUT data  @AS NEE_data
        f = netCDF4.Dataset(db_pth+'/NEE_first_year.nc', 'r')
        data = f.variables['NEE']
        data = data[:]
        data = data.swapaxes(0,2)
        adj = 60*60*24*(365/12)*1000
        data = data*adj
        # @END load_data
    
    
        # @BEGIN standardize_with_mask
        # @IN data @AS NEE_data
        # @IN mask @AS land_water_mask
        # @OUT data @AS standardized_NEE_data
        native = data.mean(2)
        latShape = mask.shape[0]
        logShape = mask.shape[1]
        for x in range(latShape):
            for y in range(logShape):
                if mask[x,y] == 1 and ma.getmask(native[x,y]) == 1:
                    for index in range(data.shape[2]):
                        data[x,y,index] = 0
        # @END standardize_with_mask
    
    
        # @BEGIN simple_diagnose
        # @PARAM fmodel
        # @IN data @AS standardized_NEE_data
        # @OUT pp  @AS result_NEE_pdf  @URI file:result_NEE.pdf
        plt.imshow(np.mean(data,2))
        plt.xlabel("Mean 1982-2010 NEE [gC/m2/mon]")
        plt.title(fmodel + ":BG1")
        pp = PdfPages('result_NEE.pdf')
        pp.savefig()
        pp.close()    
        # @END simple_diagnose
    
    # @END main
    
  • Current directory structure:

    .
    ├── build.sh
    ├── codes
    │   └── example.py
    ├── Dockerfile
    ├── run.ps1
    └── run.sh
    
  • Build Docker image based on the Dockerfile:

    docker build -t yesworkflow .
    
  • Run Docker image by mounting the codes directory as volume:

    docker run --rm -it -v $(pwd)/codes:/codes yesworkflow
    

    This will open a session in YesWorkflow environment:

    yw@be484cd73b89:~$ pwd
    /home/yw
    
  • Change directory to the codes directory in YesWorkflow session:

    cd /codes
    
  • Creating a workflow graph for a script: We will use the graph command to produce a graphical representations of the script based on the YW comments it contains. Run the example python program:

    yw graph example.py
    

    It will output the result to the terminal.

  • Storing output to file: YesWorkflow natively outputs GraphViz’s DOT format (file extension .gv). We can store the output to a file example.gv:

    yw graph example.py>example.gv
    
  • Generating Graph PDF: Render DOT output file (example.gv) as PDF file using Graphviz’s dot command:

    dot -Tpdf example.gv -o example.pdf
    

    Check the \codes folder and it will contain the graph pdf file example.pdf.

    • Directory structure after creating the DOT file and PDF file:

      .
      ├── build.sh
      ├── codes
      │   ├── example.gv
      │   ├── example.pdf
      │   └── example.py
      ├── Dockerfile
      ├── run.ps1
      └── run.sh
      
      
  • Exit YesWorkflow session: To exit the session press ctrl+d.

  • The output PDF example.pdf: alt Run YesWorkflow Docker Image on Ubuntu 20.04

Reference

Advertisement

Citation

APA Style
Shovon, A. R. (2020, December 3). Run YesWorkflow Docker Image on Ubuntu 20.04. Ahmedur Rahman Shovon. Retrieved May 21, 2024, from https://arshovon.com/blog/yesworkflow-docker-run/
MLA Style
Shovon, Ahmedur Rahman. “Run YesWorkflow Docker Image on Ubuntu 20.04.” Ahmedur Rahman Shovon, 3 Dec. 2020. Web. 21 May. 2024. https://arshovon.com/blog/yesworkflow-docker-run/.
BibTeX entry
@misc{ shovon_2020,
    author = "Shovon, Ahmedur Rahman",
    title = "Run YesWorkflow Docker Image on Ubuntu 20.04",
    year = "2020",
    url = "https://arshovon.com/blog/yesworkflow-docker-run/",
    note = "[Online; accessed 21-May-2024; URL: https://arshovon.com/blog/yesworkflow-docker-run/]"
}
Related contents in this website
Run YesWorkflow Docker Image on Ubuntu 20.04
Run YesWorkflow Docker Image on Ubuntu 20.04