{ "cells": [ { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "# Lab Session 6: Automation and Make\n", "\n", "* **Statistics 159/259, Spring 2022**\n", "* Prof. F. Pérez and GSI F. Sapienza, Department of Statistics, UC Berkeley.\n", "* 03/14/2022\n", "\n", "\n", "Useful links: \n", "\n", "- [Automation and Make - The Carpentries](https://swcarpentry.github.io/make-novice/)\n", "- [Jupytext Repository](https://github.com/mwouts/jupytext/blob/main/README.md)\n", "- [JupyterHub](https://stat159.datahub.berkeley.edu/hub/login?next=%2Fhub%2F)\n", "- [Convenient credentials management](https://ucb-stat-159-s22.github.io/site/lectures/intro-git/Git-Tutorial.html#convenient-credentials-management)\n", "\n", "**Acknowledgment:** The contents we are following for this course is based on the amazing tutorial about Automation and Make created by [The Carpentries](https://carpentries.org)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Make is called a **build tool**: it builds files, plots, papers, etc. Today we are going to do everything from the the shell. If you haven't done the setup for the tutorial during last lecture, please follow the [setup page](https://swcarpentry.github.io/make-novice/setup) in order to start working with the contents of the tutorial. \n", "\n", "We will start from the following version of our `Makefile` created inside the folder `make-lesson` provided in the tutorial." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
# Generate summary table.\n",
       "results.txt : testzipf.py isles.dat abyss.dat last.dat\n",
       "\tpython $< *.dat > $@\n",
       "\n",
       "# Count words.\n",
       ".PHONY : dats\n",
       "dats : isles.dat abyss.dat last.dat\n",
       "\n",
       "isles.dat : books/isles.txt countwords.py\n",
       "\tpython countwords.py $< $@\n",
       "\n",
       "abyss.dat : books/abyss.txt countwords.py\n",
       "\tpython countwords.py $< $@\n",
       "\n",
       "last.dat : books/last.txt countwords.py\n",
       "\tpython countwords.py $< $@\n",
       "\n",
       ".PHONY : clean\n",
       "clean :\n",
       "\trm -f *.dat\n",
       "\trm -f results.txt\n",
       "
\n" ], "text/latex": [ "\\begin{Verbatim}[commandchars=\\\\\\{\\}]\n", "\\PY{c}{\\PYZsh{} Generate summary table.}\n", "\\PY{n+nf}{results.txt }\\PY{o}{:} \\PY{n}{testzipf}.\\PY{n}{py} \\PY{n}{isles}.\\PY{n}{dat} \\PY{n}{abyss}.\\PY{n}{dat} \\PY{n}{last}.\\PY{n}{dat}\n", "\tpython \\PYZdl{}\\PYZlt{} *.dat \\PYZgt{} \\PY{n+nv}{\\PYZdl{}@}\n", "\n", "\\PY{c}{\\PYZsh{} Count words.}\n", "\\PY{n+nf}{.PHONY }\\PY{o}{:} \\PY{n}{dats}\n", "\\PY{n+nf}{dats }\\PY{o}{:} \\PY{n}{isles}.\\PY{n}{dat} \\PY{n}{abyss}.\\PY{n}{dat} \\PY{n}{last}.\\PY{n}{dat}\n", "\n", "\\PY{n+nf}{isles.dat }\\PY{o}{:} \\PY{n}{books}/\\PY{n}{isles}.\\PY{n}{txt} \\PY{n}{countwords}.\\PY{n}{py}\n", "\tpython countwords.py \\PYZdl{}\\PYZlt{} \\PY{n+nv}{\\PYZdl{}@}\n", "\n", "\\PY{n+nf}{abyss.dat }\\PY{o}{:} \\PY{n}{books}/\\PY{n}{abyss}.\\PY{n}{txt} \\PY{n}{countwords}.\\PY{n}{py}\n", "\tpython countwords.py \\PYZdl{}\\PYZlt{} \\PY{n+nv}{\\PYZdl{}@}\n", "\n", "\\PY{n+nf}{last.dat }\\PY{o}{:} \\PY{n}{books}/\\PY{n}{last}.\\PY{n}{txt} \\PY{n}{countwords}.\\PY{n}{py}\n", "\tpython countwords.py \\PYZdl{}\\PYZlt{} \\PY{n+nv}{\\PYZdl{}@}\n", "\n", "\\PY{n+nf}{.PHONY }\\PY{o}{:} \\PY{n}{clean}\n", "\\PY{n+nf}{clean }\\PY{o}{:}\n", "\trm \\PYZhy{}f *.dat\n", "\trm \\PYZhy{}f results.txt\n", "\\end{Verbatim}\n" ], "text/plain": [ "# Generate summary table.\n", "results.txt : testzipf.py isles.dat abyss.dat last.dat\n", "\tpython $< *.dat > $@\n", "\n", "# Count words.\n", ".PHONY : dats\n", "dats : isles.dat abyss.dat last.dat\n", "\n", "isles.dat : books/isles.txt countwords.py\n", "\tpython countwords.py $< $@\n", "\n", "abyss.dat : books/abyss.txt countwords.py\n", "\tpython countwords.py $< $@\n", "\n", "last.dat : books/last.txt countwords.py\n", "\tpython countwords.py $< $@\n", "\n", ".PHONY : clean\n", "clean :\n", "\trm -f *.dat\n", "\trm -f results.txt" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from IPython import display\n", "display.Code('Makefile')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before continuing, be sure to know the answer to the following questions.\n", "\n", "1. What is a phony target? When and how to use them?\n", "1. What does `$^`, `$<` and `$@` designate?\n", "1. What happen is we modify the Python script we used to generate dat?\n", "1. What does it means to do a dry run of your make file?\n", "1. How do you execute a make file? Does the name of the file needs to be `Makefile`?\n", "1. Try to make a schematic plot of the tree of dependencies of this makefiles." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Part 1: Recap\n", "\n", "1. Add to your `Makefile` the command to create a plot for the different books.\n", "1. Replace the repeated commands by using a [pattern rule](https://swcarpentry.github.io/make-novice/05-patterns/index.html). \n", " 1. What does `%` designates?\n", " 1. What does `$*` do and how to use it?\n", " 1. When can and cannot use or the other?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Part 2: Variables and functions\n", "\n", "All this exercises can be found in the [Variables](https://swcarpentry.github.io/make-novice/06-variables/index.html)\n", "and [Functions](https://swcarpentry.github.io/make-novice/07-functions/index.html) chapters. \n", "\n", "1. Update `Makefile` so that the `%.dat` rule references the variables `COUNT_SRC` and `COUNT_EXE`. Then do the same for the `testzipf.py` script and the `results.txt` rule, using `ZIPF_SRC` and `ZIPF_EXE` as variable names.\n", "1. Move the varaible declaration to a new file `config.mk` that you import into `Makefiles`. What happens when you touch `config.mk` and then `make` again? Why? Try changing `LANGUAGE=python` to `LANGUAGE=python3` to see if there is any difference. \n", "1. Follow the functions tutorial to simplify all the unnecessary explicit syntax in `Makefile`. This includes the use of both function `wildcard` and `patsubst`. If you feel adventurous, you can add more books to `books` and test Zipf's Law. \n", "1. Explore how to add documentation to the makefiles. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Part 3: Make graduation\n", "\n", "The tutorial we follow has a [final exercises](https://swcarpentry.github.io/make-novice/09-conclusion/index.html) for you to try. Test how much have you learned about makefiles and try to do it! You can use as an starting point the end of the previous tutorial. " ] } ], "metadata": { "celltoolbar": "Create Assignment", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.9" }, "toc-autonumbering": false, "toc-showcode": false, "toc-showmarkdowntxt": false, "toc-showtags": false }, "nbformat": 4, "nbformat_minor": 4 }