{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## CS 200: Decorators\n", "\n", "\n", "

\n", "\n", "

\n", " \n", "Reading: Python Cookbook, Chapter 9, metaprogramming\n", " \n", "Video: Tutorial: Introduction to Decorators Geir Arne Hjelle, PyCon 2018.\n", " \n", "A decorator is a function that takes another function as an argument, and returns a new function that modifies or decorates the input function.\n", " \n", "First, we load some modules we will use below. " ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import time\n", "from timeit import *\n", "from functools import wraps" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Simple Memoization\n", "\n", "We will write a slow procedure (from Hjelle video). Uses f string formatting (new in Python 3.6)." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "def slow_square(number):\n", " ''' Take a nap and then print the square of a number.'''\n", " print(f\"Sleeping for {number} seconds\")\n", " time.sleep(number)\n", " return number**2" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sleeping for 3 seconds\n" ] }, { "data": { "text/plain": [ "9" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_square(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will now import the \"least recently used cache\" decorator from the functools module." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from functools import lru_cache\n", "lru_cache" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "lru_cache is a Python function that takes another Python function as its argument. We can modify our slow_square function." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "slow_square = lru_cache(slow_square)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sleeping for 3 seconds\n" ] }, { "data": { "text/plain": [ "9" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_square(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "OK. It looks like nothing has changed. What's the big deal?" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "9" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_square(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lookie here! It did not take a nap! It remembered that it had already solved this problem and gave us the old solution. It had cached the answer. It had memoized it.\n", "\n", "Let's try again with a different argument." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sleeping for 4 seconds\n" ] }, { "data": { "text/plain": [ "16" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_square(4)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "16" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_square(4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Fool me once, shame on you. Fool me twice, shame on me! The decorated slow_square function is not so slow any more.\n", "\n", "Now, we introduce the syntactic sugar for decoration: the @ syntax." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "@lru_cache\n", "def slow_square(number):\n", " ''' Take a nap and then print the square of a number.'''\n", " print(f\"Now sleeping for {number} seconds\")\n", " time.sleep(number)\n", " return number**2" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Now sleeping for 3 seconds\n" ] }, { "data": { "text/plain": [ "9" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_square(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The new syntax has the same meaning as the old, but it is more succinct." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "9" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_square(3)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Now sleeping for 2 seconds\n" ] }, { "data": { "text/plain": [ "4" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_square(2)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "4" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_square(2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's try it with a function that has 2 arguments." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "@lru_cache\n", "def slow_add(a,b):\n", " ''' Take a nap and then add two numbers.'''\n", " print(f\"Sleeping for {a+b} seconds\")\n", " time.sleep(a+b)\n", " return a+b" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sleeping for 4 seconds\n" ] }, { "data": { "text/plain": [ "4" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_add(2,2)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "4" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_add(2,2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that lru_cache can decorate functions with more than one variable." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['__annotations__',\n", " '__call__',\n", " '__class__',\n", " '__copy__',\n", " '__deepcopy__',\n", " '__delattr__',\n", " '__dict__',\n", " '__dir__',\n", " '__doc__',\n", " '__eq__',\n", " '__format__',\n", " '__ge__',\n", " '__get__',\n", " '__getattribute__',\n", " '__gt__',\n", " '__hash__',\n", " '__init__',\n", " '__init_subclass__',\n", " '__le__',\n", " '__lt__',\n", " '__module__',\n", " '__name__',\n", " '__ne__',\n", " '__new__',\n", " '__qualname__',\n", " '__reduce__',\n", " '__reduce_ex__',\n", " '__repr__',\n", " '__setattr__',\n", " '__sizeof__',\n", " '__str__',\n", " '__subclasshook__',\n", " '__wrapped__',\n", " 'cache_clear',\n", " 'cache_info',\n", " 'cache_parameters']" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dir(slow_add)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "' Take a nap and then add two numbers.'" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_add.__doc__" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'slow_add'" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_add.__name__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We observe that slow_add knows its doc string and name. This is because lru_cache does the right thing when it creates the decorated function. It also remembers the old (undecorated) value of the slow_add function." ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sleeping for 4 seconds\n" ] }, { "data": { "text/plain": [ "4" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_add.__wrapped__(2,2)" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sleeping for 4 seconds\n" ] }, { "data": { "text/plain": [ "4" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_add.__wrapped__(2,2)" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slow_add.cache_info()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Examples: @bold and @italic\n", "\n", "We have seen how to use a decorator. Now will we see how to write one." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "def bold(func):\n", " def g(*args):\n", " print (\"\", end=\"\")\n", " func(*args)\n", " print (\"\", end=\"\")\n", " return g " ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "def italic(func):\n", " def g(*args):\n", " print (\"\", end=\"\")\n", " func(*args)\n", " print (\"\", end=\"\")\n", " return g " ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "@bold\n", "@italic\n", "def test(s):\n", " 'this is test'\n", " print (str(s), end=\"\")" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "this is a string." ] } ], "source": [ "test(\"this is a string.\")" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "12345" ] } ], "source": [ "test(12345)" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "another test" ] } ], "source": [ "test('another test')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The italic decorator surrounds the text string with open and close italic HTML tags: \n", "

\n",
    "<i>...</i> \n",
    "
\n", "The bold decorator surrounds the text with open and close bold HTML tags: \n", "
\n",
    "<b>...</b>\n",
    "
\n", "\n", "### Example: @debug\n", "\n", "The next example is actually useful. You can add debugging output to your functions without having to edit the functions themselves." ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [], "source": [ "def debug(f):\n", " def g(*args):\n", " print (\"Calling: {} with args: {}\".format(f.__name__, args))\n", " val = f(*args)\n", " print (\"Exiting: {} with value: {}\".format(f.__name__, val))\n", " return val\n", " return g " ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "@debug\n", "def add2(a):\n", " return a + 2" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Calling: add2 with args: (9,)\n", "Exiting: add2 with value: 11\n" ] }, { "data": { "text/plain": [ "11" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "add2(9)" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Calling: add2 with args: (-2,)\n", "Exiting: add2 with value: 0\n" ] }, { "data": { "text/plain": [ "0" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "add2(-2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### From Python Cookbook, chapter 9.1\n", "\n", "Another useful decoration. This one tells you how fast (or slow) a function executed. Note that we call the decorator wraps(funct) within the decorator. This does \n", "the bookkeeping to fix the doc string, function name, and old (undecorated) function." ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [], "source": [ "def timethis(func):\n", " '''\n", " Decorator that reports the execution time.\n", " '''\n", " @wraps(func)\n", " def wrapper(*args, **kwargs):\n", " start = time.time()\n", " result = func(*args, **kwargs)\n", " end = time.time()\n", " print(func.__name__, end-start)\n", " return result\n", " return wrapper" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1666630749.5489774" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "time.time()" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1666630749.9888592" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "time.time()" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [], "source": [ "@timethis\n", "def countdown(n):\n", " '''\n", " Counts down\n", " '''\n", " while n > 0:\n", " n -= 1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We did not use wraps in debug or the bold or italic decorators." ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on function g in module __main__:\n", "\n", "g(*args)\n", "\n" ] } ], "source": [ "help(test)" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [], "source": [ "test.__doc__" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on function timethis in module __main__:\n", "\n", "timethis(func)\n", " Decorator that reports the execution time.\n", "\n" ] } ], "source": [ "help(timethis)" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'\\n Decorator that reports the execution time.\\n '" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "timethis.__doc__" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on function countdown in module __main__:\n", "\n", "countdown(n)\n", " Counts down\n", "\n" ] } ], "source": [ "help(countdown)" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "countdown 0.0001289844512939453\n" ] } ], "source": [ "countdown(1000)" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "countdown 0.05829787254333496\n" ] } ], "source": [ "countdown(1000000)" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "countdown 0.3950803279876709\n" ] } ], "source": [ "countdown(10000000)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can unwrap countdown." ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [], "source": [ "oldcountdown = countdown.__wrapped__" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [], "source": [ "oldcountdown(10000)" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on function countdown in module __main__:\n", "\n", "countdown(n)\n", " Counts down\n", "\n" ] } ], "source": [ "help(oldcountdown)" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['__annotations__',\n", " '__builtins__',\n", " '__call__',\n", " '__class__',\n", " '__closure__',\n", " '__code__',\n", " '__defaults__',\n", " '__delattr__',\n", " '__dict__',\n", " '__dir__',\n", " '__doc__',\n", " '__eq__',\n", " '__format__',\n", " '__ge__',\n", " '__get__',\n", " '__getattribute__',\n", " '__globals__',\n", " '__gt__',\n", " '__hash__',\n", " '__init__',\n", " '__init_subclass__',\n", " '__kwdefaults__',\n", " '__le__',\n", " '__lt__',\n", " '__module__',\n", " '__name__',\n", " '__ne__',\n", " '__new__',\n", " '__qualname__',\n", " '__reduce__',\n", " '__reduce_ex__',\n", " '__repr__',\n", " '__setattr__',\n", " '__sizeof__',\n", " '__str__',\n", " '__subclasshook__',\n", " '__wrapped__']" ] }, "execution_count": 49, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dir(countdown)" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'\\n Counts down\\n '" ] }, "execution_count": 50, "metadata": {}, "output_type": "execute_result" } ], "source": [ "countdown.__doc__" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [], "source": [ "test.__doc__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "#### Trace as a decorator\n", "\n", "We can trace the excution of a function - particularly a recursive one." ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [], "source": [ "def trace(f):\n", " f.indent = 0\n", " def g(x):\n", " print('| ' * f.indent + '|--', f.__name__, x)\n", " f.indent += 1\n", " value = f(x)\n", " print('| ' * f.indent + '|--', 'return', repr(value))\n", " f.indent -= 1\n", " return value\n", " return g" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [], "source": [ "@trace\n", "def fib(n):\n", " if n == 0:\n", " return 1\n", " elif n == 1:\n", " return 1\n", " else:\n", " return fib(n - 1) + fib(n - 2)" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "|-- fib 4\n", "| |-- fib 3\n", "| | |-- fib 2\n", "| | | |-- fib 1\n", "| | | | |-- return 1\n", "| | | |-- fib 0\n", "| | | | |-- return 1\n", "| | | |-- return 2\n", "| | |-- fib 1\n", "| | | |-- return 1\n", "| | |-- return 3\n", "| |-- fib 2\n", "| | |-- fib 1\n", "| | | |-- return 1\n", "| | |-- fib 0\n", "| | | |-- return 1\n", "| | |-- return 2\n", "| |-- return 5\n" ] }, { "data": { "text/plain": [ "5" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fib(4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Memoized fib\n", "\n", "Instead of using lru_cache we can write our own memoization decorator.\n", "\n", "First, we will write a memoized fibonacci from scratch." ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [], "source": [ "def memofib(n):\n", " table = [False for x in range(n+1)]\n", " return memofibaux(n,table)\n", "\n", "def memofibaux(n, table):\n", " if table[n]: return table[n]\n", " if n <= 1: return 1\n", " else:\n", " table[n] = memofibaux(n-1, table) + memofibaux(n-2, table)\n", " return table[n]" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "5" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "memofib(4)" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [], "source": [ "memofibaux = trace(memofibaux)" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "ename": "TypeError", "evalue": "trace..g() takes 1 positional argument but 2 were given", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)", "Input \u001b[0;32mIn [58]\u001b[0m, in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0m memofib(\u001b[38;5;241m4\u001b[39m)\n", "Input \u001b[0;32mIn [55]\u001b[0m, in \u001b[0;36mmemofib\u001b[0;34m(n)\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mmemofib\u001b[39m(n):\n\u001b[1;32m 2\u001b[0m table \u001b[38;5;241m=\u001b[39m [\u001b[38;5;28;01mFalse\u001b[39;00m \u001b[38;5;28;01mfor\u001b[39;00m x \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(n\u001b[38;5;241m+\u001b[39m\u001b[38;5;241m1\u001b[39m)]\n\u001b[0;32m----> 3\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mmemofibaux\u001b[49m\u001b[43m(\u001b[49m\u001b[43mn\u001b[49m\u001b[43m,\u001b[49m\u001b[43mtable\u001b[49m\u001b[43m)\u001b[49m\n", "\u001b[0;31mTypeError\u001b[0m: trace..g() takes 1 positional argument but 2 were given" ] } ], "source": [ "memofib(4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have to modify trace to handle multiple arguments using *args (splat args)." ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [], "source": [ "def trace2(f): \n", " f.indent = 0\n", " def g(*x):\n", " print('| ' * f.indent + '|--', f.__name__, x)\n", " f.indent += 1\n", " value = f(*x)\n", " print('| ' * f.indent + '|--', 'return', repr(value))\n", " f.indent -= 1\n", " return value\n", " return g" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [], "source": [ "@trace2\n", "def memofibaux(n, table):\n", " if table[n]: return table[n]\n", " if n <= 1: return 1\n", " else:\n", " table[n] = memofibaux(n-1, table) + memofibaux(n-2, table)\n", " return table[n]" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "|-- memofibaux (4, [False, False, False, False, False])\n", "| |-- memofibaux (3, [False, False, False, False, False])\n", "| | |-- memofibaux (2, [False, False, False, False, False])\n", "| | | |-- memofibaux (1, [False, False, False, False, False])\n", "| | | | |-- return 1\n", "| | | |-- memofibaux (0, [False, False, False, False, False])\n", "| | | | |-- return 1\n", "| | | |-- return 2\n", "| | |-- memofibaux (1, [False, False, 2, False, False])\n", "| | | |-- return 1\n", "| | |-- return 3\n", "| |-- memofibaux (2, [False, False, 2, 3, False])\n", "| | |-- return 2\n", "| |-- return 5\n" ] }, { "data": { "text/plain": [ "5" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" } ], "source": [ "memofib(4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the table is passed as a parameter and that extra recursive calls are avoided." ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "|-- memofibaux (10, [False, False, False, False, False, False, False, False, False, False, False])\n", "| |-- memofibaux (9, [False, False, False, False, False, False, False, False, False, False, False])\n", "| | |-- memofibaux (8, [False, False, False, False, False, False, False, False, False, False, False])\n", "| | | |-- memofibaux (7, [False, False, False, False, False, False, False, False, False, False, False])\n", "| | | | |-- memofibaux (6, [False, False, False, False, False, False, False, False, False, False, False])\n", "| | | | | |-- memofibaux (5, [False, False, False, False, False, False, False, False, False, False, False])\n", "| | | | | | |-- memofibaux (4, [False, False, False, False, False, False, False, False, False, False, False])\n", "| | | | | | | |-- memofibaux (3, [False, False, False, False, False, False, False, False, False, False, False])\n", "| | | | | | | | |-- memofibaux (2, [False, False, False, False, False, False, False, False, False, False, False])\n", "| | | | | | | | | |-- memofibaux (1, [False, False, False, False, False, False, False, False, False, False, False])\n", "| | | | | | | | | | |-- return 1\n", "| | | | | | | | | |-- memofibaux (0, [False, False, False, False, False, False, False, False, False, False, False])\n", "| | | | | | | | | | |-- return 1\n", "| | | | | | | | | |-- return 2\n", "| | | | | | | | |-- memofibaux (1, [False, False, 2, False, False, False, False, False, False, False, False])\n", "| | | | | | | | | |-- return 1\n", "| | | | | | | | |-- return 3\n", "| | | | | | | |-- memofibaux (2, [False, False, 2, 3, False, False, False, False, False, False, False])\n", "| | | | | | | | |-- return 2\n", "| | | | | | | |-- return 5\n", "| | | | | | |-- memofibaux (3, [False, False, 2, 3, 5, False, False, False, False, False, False])\n", "| | | | | | | |-- return 3\n", "| | | | | | |-- return 8\n", "| | | | | |-- memofibaux (4, [False, False, 2, 3, 5, 8, False, False, False, False, False])\n", "| | | | | | |-- return 5\n", "| | | | | |-- return 13\n", "| | | | |-- memofibaux (5, [False, False, 2, 3, 5, 8, 13, False, False, False, False])\n", "| | | | | |-- return 8\n", "| | | | |-- return 21\n", "| | | |-- memofibaux (6, [False, False, 2, 3, 5, 8, 13, 21, False, False, False])\n", "| | | | |-- return 13\n", "| | | |-- return 34\n", "| | |-- memofibaux (7, [False, False, 2, 3, 5, 8, 13, 21, 34, False, False])\n", "| | | |-- return 21\n", "| | |-- return 55\n", "| |-- memofibaux (8, [False, False, 2, 3, 5, 8, 13, 21, 34, 55, False])\n", "| | |-- return 34\n", "| |-- return 89\n" ] }, { "data": { "text/plain": [ "89" ] }, "execution_count": 62, "metadata": {}, "output_type": "execute_result" } ], "source": [ "memofib(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Another memoized fib\n", "\n", "\n", "Now we create a decorator, like lru_cache." ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [], "source": [ "def memoized(f):\n", " \"\"\"Decorator that caches a function's return value each time it is\n", " called. If called later with the same arguments, the cached value\n", " is returned, and not re-evaluated.\n", "\n", " \"\"\"\n", " cache = {}\n", " @wraps(f)\n", " def wrapped(*args):\n", " try:\n", " result = cache[args]\n", " except KeyError:\n", " result = cache[args] = f(*args)\n", " return result\n", " return wrapped" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will now use two decorators to trace and memoize another fib function." ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [], "source": [ "@trace\n", "@memoized\n", "def mfib(n):\n", " if n <= 0: return 1\n", " if n == 1: return 1\n", " else:\n", " return mfib(n-1) + mfib(n-2)" ] }, { "cell_type": "code", "execution_count": 65, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "|-- mfib 10\n", "| |-- mfib 9\n", "| | |-- mfib 8\n", "| | | |-- mfib 7\n", "| | | | |-- mfib 6\n", "| | | | | |-- mfib 5\n", "| | | | | | |-- mfib 4\n", "| | | | | | | |-- mfib 3\n", "| | | | | | | | |-- mfib 2\n", "| | | | | | | | | |-- mfib 1\n", "| | | | | | | | | | |-- return 1\n", "| | | | | | | | | |-- mfib 0\n", "| | | | | | | | | | |-- return 1\n", "| | | | | | | | | |-- return 2\n", "| | | | | | | | |-- mfib 1\n", "| | | | | | | | | |-- return 1\n", "| | | | | | | | |-- return 3\n", "| | | | | | | |-- mfib 2\n", "| | | | | | | | |-- return 2\n", "| | | | | | | |-- return 5\n", "| | | | | | |-- mfib 3\n", "| | | | | | | |-- return 3\n", "| | | | | | |-- return 8\n", "| | | | | |-- mfib 4\n", "| | | | | | |-- return 5\n", "| | | | | |-- return 13\n", "| | | | |-- mfib 5\n", "| | | | | |-- return 8\n", "| | | | |-- return 21\n", "| | | |-- mfib 6\n", "| | | | |-- return 13\n", "| | | |-- return 34\n", "| | |-- mfib 7\n", "| | | |-- return 21\n", "| | |-- return 55\n", "| |-- mfib 8\n", "| | |-- return 34\n", "| |-- return 89\n" ] }, { "data": { "text/plain": [ "89" ] }, "execution_count": 65, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mfib(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we call it again, it remembers the old answer." ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "|-- mfib 10\n", "| |-- return 89\n" ] }, { "data": { "text/plain": [ "89" ] }, "execution_count": 66, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mfib(10)" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "|-- mfib 12\n", "| |-- mfib 11\n", "| | |-- mfib 10\n", "| | | |-- return 89\n", "| | |-- mfib 9\n", "| | | |-- return 55\n", "| | |-- return 144\n", "| |-- mfib 10\n", "| | |-- return 89\n", "| |-- return 233\n" ] }, { "data": { "text/plain": [ "233" ] }, "execution_count": 67, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mfib(12)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here is a better way with constant space. Do not need n-entry table" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [], "source": [ "def fibbetter(n):\n", " return fibbetteraux(n, 1, 1)\n", "\n", "\n", "## Note: this is tail recursive. Reflected in trace output.\n", "def fibbetteraux(n, x, y):\n", " if n <= 0:\n", " return x \n", " else:\n", " return fibbetteraux(n-1,y, x+y)" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "89" ] }, "execution_count": 69, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fibbetter(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Timing the fibs\n", "\n", "We first untrace the functions." ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [], "source": [ "## no trace this time.\n", "def fib(n):\n", " if n == 0:\n", " return 1\n", " elif n == 1:\n", " return 1\n", " else:\n", " return fib(n - 1) + fib(n - 2)\n", " \n", "def memofibaux(n, table):\n", " if table[n]: return table[n]\n", " if n <= 1: return 1\n", " else:\n", " table[n] = memofibaux(n-1, table) + memofibaux(n-2, table)\n", " return table[n]\n", " \n", "@memoized\n", "def mfib(n):\n", " if n <= 0: return 1\n", " if n == 1: return 1\n", " else:\n", " return mfib(n-1) + mfib(n-2)" ] }, { "cell_type": "code", "execution_count": 71, "metadata": {}, "outputs": [], "source": [ "def doit():\n", " f1 = timeit('fib(10)', setup = 'from __main__ import fib', number=1)\n", " f2 = timeit('memofib(10)', setup = 'from __main__ import memofib', number=1)\n", " f3 = timeit('mfib(10)', setup = 'from __main__ import mfib', number=1)\n", " f4 = timeit('fibbetter(10)', setup = 'from __main__ import fibbetter', number=1)\n", " return f1, f2, f3, f4" ] }, { "cell_type": "code", "execution_count": 72, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(6.396399112418294e-05,\n", " 2.163101453334093e-05,\n", " 3.597204340621829e-05,\n", " 2.214504638686776e-05)" ] }, "execution_count": 72, "metadata": {}, "output_type": "execute_result" } ], "source": [ "doit()" ] }, { "cell_type": "code", "execution_count": 73, "metadata": {}, "outputs": [], "source": [ "def doit2(n):\n", " f1 = timeit('fib(' + str(n) + ')', setup = 'from __main__ import fib', number=1)\n", " f2 = timeit('memofib(' + str(n) + ')', setup = 'from __main__ import memofib', number=1)\n", " f3 = timeit('mfib(' + str(n) + ')', setup = 'from __main__ import mfib', number=1)\n", " f4 = timeit('fibbetter(' + str(n) + ')', setup = 'from __main__ import fibbetter', number=1)\n", " return f1, f2, f3, f4" ] }, { "cell_type": "code", "execution_count": 74, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(6.357498932629824e-05,\n", " 2.0317966118454933e-05,\n", " 3.1539821065962315e-06,\n", " 8.261995390057564e-06)" ] }, "execution_count": 74, "metadata": {}, "output_type": "execute_result" } ], "source": [ "doit2(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Wraps and Wrappers\n", "\n", "See Python Cookbook, chapter 9.2" ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [], "source": [ "from functools import wraps\n", "\n", "def without_wraps(func):\n", " def __wrapper(*args, **kwargs):\n", " return func(*args, **kwargs)\n", " return __wrapper\n", " \n", "def with_wraps(func):\n", " @wraps(func)\n", " def __wrapper(*args, **kwargs):\n", " return func(*args, **kwargs)\n", " return __wrapper" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have created two decorators - without and with wraps. We will decorate two dummy functions." ] }, { "cell_type": "code", "execution_count": 76, "metadata": {}, "outputs": [], "source": [ "@without_wraps\n", "def my_func_a():\n", " \"\"\"Here is my_func_a doc string text.\"\"\"\n", " pass\n", " \n", "@with_wraps\n", "def my_func_b():\n", " \"\"\"Here is my_func_b doc string text.\"\"\"\n", " pass" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will now look at the properties of these functions." ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['__annotations__',\n", " '__builtins__',\n", " '__call__',\n", " '__class__',\n", " '__closure__',\n", " '__code__',\n", " '__defaults__',\n", " '__delattr__',\n", " '__dict__',\n", " '__dir__',\n", " '__doc__',\n", " '__eq__',\n", " '__format__',\n", " '__ge__',\n", " '__get__',\n", " '__getattribute__',\n", " '__globals__',\n", " '__gt__',\n", " '__hash__',\n", " '__init__',\n", " '__init_subclass__',\n", " '__kwdefaults__',\n", " '__le__',\n", " '__lt__',\n", " '__module__',\n", " '__name__',\n", " '__ne__',\n", " '__new__',\n", " '__qualname__',\n", " '__reduce__',\n", " '__reduce_ex__',\n", " '__repr__',\n", " '__setattr__',\n", " '__sizeof__',\n", " '__str__',\n", " '__subclasshook__']" ] }, "execution_count": 77, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dir(my_func_a)" ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['__annotations__',\n", " '__builtins__',\n", " '__call__',\n", " '__class__',\n", " '__closure__',\n", " '__code__',\n", " '__defaults__',\n", " '__delattr__',\n", " '__dict__',\n", " '__dir__',\n", " '__doc__',\n", " '__eq__',\n", " '__format__',\n", " '__ge__',\n", " '__get__',\n", " '__getattribute__',\n", " '__globals__',\n", " '__gt__',\n", " '__hash__',\n", " '__init__',\n", " '__init_subclass__',\n", " '__kwdefaults__',\n", " '__le__',\n", " '__lt__',\n", " '__module__',\n", " '__name__',\n", " '__ne__',\n", " '__new__',\n", " '__qualname__',\n", " '__reduce__',\n", " '__reduce_ex__',\n", " '__repr__',\n", " '__setattr__',\n", " '__sizeof__',\n", " '__str__',\n", " '__subclasshook__',\n", " '__wrapped__']" ] }, "execution_count": 78, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dir(my_func_b)" ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "None\n", "__wrapper\n", "__main__\n" ] } ], "source": [ "print (my_func_a.__doc__)\n", "print (my_func_a.__name__)\n", "print (my_func_a.__module__)" ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [ { "ename": "AttributeError", "evalue": "'function' object has no attribute '__wrapped__'", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)", "Input \u001b[0;32mIn [80]\u001b[0m, in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28mprint\u001b[39m (my_func_a\u001b[38;5;241m.\u001b[39m__wrapped__)\n", "\u001b[0;31mAttributeError\u001b[0m: 'function' object has no attribute '__wrapped__'" ] } ], "source": [ "print (my_func_a.__wrapped__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We get an exception because we did not wrap the function, saving original definition." ] }, { "cell_type": "code", "execution_count": 81, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Here is my_func_b doc string text.\n", "my_func_b\n", "__main__\n", "\n" ] } ], "source": [ "print (my_func_b.__doc__)\n", "print (my_func_b.__name__)\n", "print (my_func_b.__module__)\n", "print (my_func_b.__wrapped__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### More decorators\n", "\n", "Here is a decorator that executes a function twice and returns both results as a tuple." ] }, { "cell_type": "code", "execution_count": 82, "metadata": {}, "outputs": [], "source": [ "def do_twice(func):\n", " ''' Decorator that executes a function twice and returns the results as a 2-tuple'''\n", " @wraps(func)\n", " def g(*args):\n", " val1 = func(*args)\n", " val2 = func(*args)\n", " return (val1, val2)\n", " return g" ] }, { "cell_type": "code", "execution_count": 83, "metadata": {}, "outputs": [], "source": [ "@do_twice\n", "def add2(n):\n", " ''' add 2 to the input'''\n", " return n+2" ] }, { "cell_type": "code", "execution_count": 84, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(7, 7)" ] }, "execution_count": 84, "metadata": {}, "output_type": "execute_result" } ], "source": [ "add2(5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is really not very interesting. If a function always returns the same value for a given input, why execute multiple times? That was why we were memoizing functions!\n", "\n", "Here is a better example." ] }, { "cell_type": "code", "execution_count": 85, "metadata": {}, "outputs": [], "source": [ "import random\n", "def roll_dice():\n", " ''' roll a fair die - a random value between 1 and 6 inclusive.'''\n", " return random.randint(1,6)" ] }, { "cell_type": "code", "execution_count": 86, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2" ] }, "execution_count": 86, "metadata": {}, "output_type": "execute_result" } ], "source": [ "roll_dice()" ] }, { "cell_type": "code", "execution_count": 87, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1" ] }, "execution_count": 87, "metadata": {}, "output_type": "execute_result" } ], "source": [ "roll_dice()" ] }, { "cell_type": "code", "execution_count": 88, "metadata": {}, "outputs": [], "source": [ "@do_twice\n", "def roll_dice():\n", " ''' roll a fair die - a random value between 1 and 6 inclusive.'''\n", " return random.randint(1,6)" ] }, { "cell_type": "code", "execution_count": 89, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(6, 3)" ] }, "execution_count": 89, "metadata": {}, "output_type": "execute_result" } ], "source": [ "roll_dice()" ] }, { "cell_type": "code", "execution_count": 90, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(6, 5)" ] }, "execution_count": 90, "metadata": {}, "output_type": "execute_result" } ], "source": [ "roll_dice()" ] }, { "cell_type": "code", "execution_count": 91, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on function roll_dice in module __main__:\n", "\n", "roll_dice()\n", " roll a fair die - a random value between 1 and 6 inclusive.\n", "\n" ] } ], "source": [ "help(roll_dice)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "End of Decorator notebook." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.7" } }, "nbformat": 4, "nbformat_minor": 4 }