{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## CS 200: Decorators\n", "\n", "\n", "
\n", "\n", "
\n",
" \n",
"Reading: Python Cookbook, Chapter 9, metaprogramming\n",
" \n",
"Video: Tutorial: Introduction to Decorators Geir Arne Hjelle, PyCon 2018.\n",
" \n",
"A decorator is a function that takes another function as an argument, and returns a new function that modifies or decorates the input function.\n",
" \n",
"First, we load some modules we will use below. "
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import time\n",
"from timeit import *\n",
"from functools import wraps"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Simple Memoization\n",
"\n",
"We will write a slow procedure (from Hjelle video). Uses f string formatting (new in Python 3.6)."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"def slow_square(number):\n",
" ''' Take a nap and then print the square of a number.'''\n",
" print(f\"Sleeping for {number} seconds\")\n",
" time.sleep(number)\n",
" return number**2"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sleeping for 3 seconds\n"
]
},
{
"data": {
"text/plain": [
"9"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_square(3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will now import the \"least recently used cache\" decorator from the functools
module."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"lru_cache
is a Python function that takes another Python function as its argument. We can modify our slow_square
function."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"slow_square = lru_cache(slow_square)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sleeping for 3 seconds\n"
]
},
{
"data": {
"text/plain": [
"9"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_square(3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"OK. It looks like nothing has changed. What's the big deal?"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"9"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_square(3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Lookie here! It did not take a nap! It remembered that it had already solved this problem and gave us the old solution. It had cached the answer. It had memoized it.\n",
"\n",
"Let's try again with a different argument."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sleeping for 4 seconds\n"
]
},
{
"data": {
"text/plain": [
"16"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_square(4)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"16"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_square(4)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Fool me once, shame on you. Fool me twice, shame on me! The decorated slow_square function is not so slow any more.\n",
"\n",
"Now, we introduce the syntactic sugar for decoration: the @ syntax."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"@lru_cache\n",
"def slow_square(number):\n",
" ''' Take a nap and then print the square of a number.'''\n",
" print(f\"Now sleeping for {number} seconds\")\n",
" time.sleep(number)\n",
" return number**2"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Now sleeping for 3 seconds\n"
]
},
{
"data": {
"text/plain": [
"9"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_square(3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The new syntax has the same meaning as the old, but it is more succinct."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"9"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_square(3)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Now sleeping for 2 seconds\n"
]
},
{
"data": {
"text/plain": [
"4"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_square(2)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"4"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_square(2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's try it with a function that has 2 arguments."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"@lru_cache\n",
"def slow_add(a,b):\n",
" ''' Take a nap and then add two numbers.'''\n",
" print(f\"Sleeping for {a+b} seconds\")\n",
" time.sleep(a+b)\n",
" return a+b"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sleeping for 4 seconds\n"
]
},
{
"data": {
"text/plain": [
"4"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_add(2,2)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"4"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_add(2,2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that lru_cache
can decorate functions with more than one variable."
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['__annotations__',\n",
" '__call__',\n",
" '__class__',\n",
" '__copy__',\n",
" '__deepcopy__',\n",
" '__delattr__',\n",
" '__dict__',\n",
" '__dir__',\n",
" '__doc__',\n",
" '__eq__',\n",
" '__format__',\n",
" '__ge__',\n",
" '__get__',\n",
" '__getattribute__',\n",
" '__gt__',\n",
" '__hash__',\n",
" '__init__',\n",
" '__init_subclass__',\n",
" '__le__',\n",
" '__lt__',\n",
" '__module__',\n",
" '__name__',\n",
" '__ne__',\n",
" '__new__',\n",
" '__qualname__',\n",
" '__reduce__',\n",
" '__reduce_ex__',\n",
" '__repr__',\n",
" '__setattr__',\n",
" '__sizeof__',\n",
" '__str__',\n",
" '__subclasshook__',\n",
" '__wrapped__',\n",
" 'cache_clear',\n",
" 'cache_info',\n",
" 'cache_parameters']"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dir(slow_add)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"' Take a nap and then add two numbers.'"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_add.__doc__"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'slow_add'"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_add.__name__"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We observe that slow_add knows its doc string and name. This is because lru_cache does the right thing when it creates the decorated function. It also remembers the old (undecorated) value of the slow_add
function."
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sleeping for 4 seconds\n"
]
},
{
"data": {
"text/plain": [
"4"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_add.__wrapped__(2,2)"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sleeping for 4 seconds\n"
]
},
{
"data": {
"text/plain": [
"4"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_add.__wrapped__(2,2)"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"slow_add.cache_info()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Examples: @bold and @italic\n",
"\n",
"We have seen how to use a decorator. Now will we see how to write one."
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"def bold(func):\n",
" def g(*args):\n",
" print (\"\", end=\"\")\n",
" func(*args)\n",
" print (\"\", end=\"\")\n",
" return g "
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [],
"source": [
"def italic(func):\n",
" def g(*args):\n",
" print (\"\", end=\"\")\n",
" func(*args)\n",
" print (\"\", end=\"\")\n",
" return g "
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"@bold\n",
"@italic\n",
"def test(s):\n",
" 'this is test'\n",
" print (str(s), end=\"\")"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"this is a string."
]
}
],
"source": [
"test(\"this is a string.\")"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"12345"
]
}
],
"source": [
"test(12345)"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"another test"
]
}
],
"source": [
"test('another test')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The italic decorator surrounds the text string with open and close italic HTML tags: \n",
"\n",
"<i>...</i> \n",
"
\n",
"The bold decorator surrounds the text with open and close bold HTML tags: \n",
"\n",
"<b>...</b>\n",
"
\n",
"\n",
"### Example: @debug\n",
"\n",
"The next example is actually useful. You can add debugging output to your functions without having to edit the functions themselves."
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [],
"source": [
"def debug(f):\n",
" def g(*args):\n",
" print (\"Calling: {} with args: {}\".format(f.__name__, args))\n",
" val = f(*args)\n",
" print (\"Exiting: {} with value: {}\".format(f.__name__, val))\n",
" return val\n",
" return g "
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"@debug\n",
"def add2(a):\n",
" return a + 2"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Calling: add2 with args: (9,)\n",
"Exiting: add2 with value: 11\n"
]
},
{
"data": {
"text/plain": [
"11"
]
},
"execution_count": 32,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"add2(9)"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Calling: add2 with args: (-2,)\n",
"Exiting: add2 with value: 0\n"
]
},
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"add2(-2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### From Python Cookbook, chapter 9.1\n",
"\n",
"Another useful decoration. This one tells you how fast (or slow) a function executed. Note that we call the decorator wraps(funct)
within the decorator. This does \n",
"the bookkeeping to fix the doc string, function name, and old (undecorated) function."
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [],
"source": [
"def timethis(func):\n",
" '''\n",
" Decorator that reports the execution time.\n",
" '''\n",
" @wraps(func)\n",
" def wrapper(*args, **kwargs):\n",
" start = time.time()\n",
" result = func(*args, **kwargs)\n",
" end = time.time()\n",
" print(func.__name__, end-start)\n",
" return result\n",
" return wrapper"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1666630749.5489774"
]
},
"execution_count": 35,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"time.time()"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1666630749.9888592"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"time.time()"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [],
"source": [
"@timethis\n",
"def countdown(n):\n",
" '''\n",
" Counts down\n",
" '''\n",
" while n > 0:\n",
" n -= 1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We did not use wraps in debug or the bold or italic decorators."
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Help on function g in module __main__:\n",
"\n",
"g(*args)\n",
"\n"
]
}
],
"source": [
"help(test)"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {},
"outputs": [],
"source": [
"test.__doc__"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Help on function timethis in module __main__:\n",
"\n",
"timethis(func)\n",
" Decorator that reports the execution time.\n",
"\n"
]
}
],
"source": [
"help(timethis)"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'\\n Decorator that reports the execution time.\\n '"
]
},
"execution_count": 41,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"timethis.__doc__"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Help on function countdown in module __main__:\n",
"\n",
"countdown(n)\n",
" Counts down\n",
"\n"
]
}
],
"source": [
"help(countdown)"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"countdown 0.0001289844512939453\n"
]
}
],
"source": [
"countdown(1000)"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"countdown 0.05829787254333496\n"
]
}
],
"source": [
"countdown(1000000)"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"countdown 0.3950803279876709\n"
]
}
],
"source": [
"countdown(10000000)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can unwrap countdown."
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {},
"outputs": [],
"source": [
"oldcountdown = countdown.__wrapped__"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [],
"source": [
"oldcountdown(10000)"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Help on function countdown in module __main__:\n",
"\n",
"countdown(n)\n",
" Counts down\n",
"\n"
]
}
],
"source": [
"help(oldcountdown)"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['__annotations__',\n",
" '__builtins__',\n",
" '__call__',\n",
" '__class__',\n",
" '__closure__',\n",
" '__code__',\n",
" '__defaults__',\n",
" '__delattr__',\n",
" '__dict__',\n",
" '__dir__',\n",
" '__doc__',\n",
" '__eq__',\n",
" '__format__',\n",
" '__ge__',\n",
" '__get__',\n",
" '__getattribute__',\n",
" '__globals__',\n",
" '__gt__',\n",
" '__hash__',\n",
" '__init__',\n",
" '__init_subclass__',\n",
" '__kwdefaults__',\n",
" '__le__',\n",
" '__lt__',\n",
" '__module__',\n",
" '__name__',\n",
" '__ne__',\n",
" '__new__',\n",
" '__qualname__',\n",
" '__reduce__',\n",
" '__reduce_ex__',\n",
" '__repr__',\n",
" '__setattr__',\n",
" '__sizeof__',\n",
" '__str__',\n",
" '__subclasshook__',\n",
" '__wrapped__']"
]
},
"execution_count": 49,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dir(countdown)"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'\\n Counts down\\n '"
]
},
"execution_count": 50,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"countdown.__doc__"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {},
"outputs": [],
"source": [
"test.__doc__"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"#### Trace as a decorator\n",
"\n",
"We can trace the excution of a function - particularly a recursive one."
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {},
"outputs": [],
"source": [
"def trace(f):\n",
" f.indent = 0\n",
" def g(x):\n",
" print('| ' * f.indent + '|--', f.__name__, x)\n",
" f.indent += 1\n",
" value = f(x)\n",
" print('| ' * f.indent + '|--', 'return', repr(value))\n",
" f.indent -= 1\n",
" return value\n",
" return g"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {},
"outputs": [],
"source": [
"@trace\n",
"def fib(n):\n",
" if n == 0:\n",
" return 1\n",
" elif n == 1:\n",
" return 1\n",
" else:\n",
" return fib(n - 1) + fib(n - 2)"
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"|-- fib 4\n",
"| |-- fib 3\n",
"| | |-- fib 2\n",
"| | | |-- fib 1\n",
"| | | | |-- return 1\n",
"| | | |-- fib 0\n",
"| | | | |-- return 1\n",
"| | | |-- return 2\n",
"| | |-- fib 1\n",
"| | | |-- return 1\n",
"| | |-- return 3\n",
"| |-- fib 2\n",
"| | |-- fib 1\n",
"| | | |-- return 1\n",
"| | |-- fib 0\n",
"| | | |-- return 1\n",
"| | |-- return 2\n",
"| |-- return 5\n"
]
},
{
"data": {
"text/plain": [
"5"
]
},
"execution_count": 54,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"fib(4)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Memoized fib\n",
"\n",
"Instead of using lru_cache
we can write our own memoization decorator.\n",
"\n",
"First, we will write a memoized fibonacci from scratch."
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {},
"outputs": [],
"source": [
"def memofib(n):\n",
" table = [False for x in range(n+1)]\n",
" return memofibaux(n,table)\n",
"\n",
"def memofibaux(n, table):\n",
" if table[n]: return table[n]\n",
" if n <= 1: return 1\n",
" else:\n",
" table[n] = memofibaux(n-1, table) + memofibaux(n-2, table)\n",
" return table[n]"
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"5"
]
},
"execution_count": 56,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"memofib(4)"
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {},
"outputs": [],
"source": [
"memofibaux = trace(memofibaux)"
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {},
"outputs": [
{
"ename": "TypeError",
"evalue": "trace.*args
(splat args)."
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {},
"outputs": [],
"source": [
"def trace2(f): \n",
" f.indent = 0\n",
" def g(*x):\n",
" print('| ' * f.indent + '|--', f.__name__, x)\n",
" f.indent += 1\n",
" value = f(*x)\n",
" print('| ' * f.indent + '|--', 'return', repr(value))\n",
" f.indent -= 1\n",
" return value\n",
" return g"
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {},
"outputs": [],
"source": [
"@trace2\n",
"def memofibaux(n, table):\n",
" if table[n]: return table[n]\n",
" if n <= 1: return 1\n",
" else:\n",
" table[n] = memofibaux(n-1, table) + memofibaux(n-2, table)\n",
" return table[n]"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"|-- memofibaux (4, [False, False, False, False, False])\n",
"| |-- memofibaux (3, [False, False, False, False, False])\n",
"| | |-- memofibaux (2, [False, False, False, False, False])\n",
"| | | |-- memofibaux (1, [False, False, False, False, False])\n",
"| | | | |-- return 1\n",
"| | | |-- memofibaux (0, [False, False, False, False, False])\n",
"| | | | |-- return 1\n",
"| | | |-- return 2\n",
"| | |-- memofibaux (1, [False, False, 2, False, False])\n",
"| | | |-- return 1\n",
"| | |-- return 3\n",
"| |-- memofibaux (2, [False, False, 2, 3, False])\n",
"| | |-- return 2\n",
"| |-- return 5\n"
]
},
{
"data": {
"text/plain": [
"5"
]
},
"execution_count": 61,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"memofib(4)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the table is passed as a parameter and that extra recursive calls are avoided."
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"|-- memofibaux (10, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| |-- memofibaux (9, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| | |-- memofibaux (8, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| | | |-- memofibaux (7, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| | | | |-- memofibaux (6, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| | | | | |-- memofibaux (5, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| | | | | | |-- memofibaux (4, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| | | | | | | |-- memofibaux (3, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| | | | | | | | |-- memofibaux (2, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| | | | | | | | | |-- memofibaux (1, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| | | | | | | | | | |-- return 1\n",
"| | | | | | | | | |-- memofibaux (0, [False, False, False, False, False, False, False, False, False, False, False])\n",
"| | | | | | | | | | |-- return 1\n",
"| | | | | | | | | |-- return 2\n",
"| | | | | | | | |-- memofibaux (1, [False, False, 2, False, False, False, False, False, False, False, False])\n",
"| | | | | | | | | |-- return 1\n",
"| | | | | | | | |-- return 3\n",
"| | | | | | | |-- memofibaux (2, [False, False, 2, 3, False, False, False, False, False, False, False])\n",
"| | | | | | | | |-- return 2\n",
"| | | | | | | |-- return 5\n",
"| | | | | | |-- memofibaux (3, [False, False, 2, 3, 5, False, False, False, False, False, False])\n",
"| | | | | | | |-- return 3\n",
"| | | | | | |-- return 8\n",
"| | | | | |-- memofibaux (4, [False, False, 2, 3, 5, 8, False, False, False, False, False])\n",
"| | | | | | |-- return 5\n",
"| | | | | |-- return 13\n",
"| | | | |-- memofibaux (5, [False, False, 2, 3, 5, 8, 13, False, False, False, False])\n",
"| | | | | |-- return 8\n",
"| | | | |-- return 21\n",
"| | | |-- memofibaux (6, [False, False, 2, 3, 5, 8, 13, 21, False, False, False])\n",
"| | | | |-- return 13\n",
"| | | |-- return 34\n",
"| | |-- memofibaux (7, [False, False, 2, 3, 5, 8, 13, 21, 34, False, False])\n",
"| | | |-- return 21\n",
"| | |-- return 55\n",
"| |-- memofibaux (8, [False, False, 2, 3, 5, 8, 13, 21, 34, 55, False])\n",
"| | |-- return 34\n",
"| |-- return 89\n"
]
},
{
"data": {
"text/plain": [
"89"
]
},
"execution_count": 62,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"memofib(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### Another memoized fib\n",
"\n",
"\n",
"Now we create a decorator, like lru_cache
."
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {},
"outputs": [],
"source": [
"def memoized(f):\n",
" \"\"\"Decorator that caches a function's return value each time it is\n",
" called. If called later with the same arguments, the cached value\n",
" is returned, and not re-evaluated.\n",
"\n",
" \"\"\"\n",
" cache = {}\n",
" @wraps(f)\n",
" def wrapped(*args):\n",
" try:\n",
" result = cache[args]\n",
" except KeyError:\n",
" result = cache[args] = f(*args)\n",
" return result\n",
" return wrapped"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will now use two decorators to trace and memoize another fib function."
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {},
"outputs": [],
"source": [
"@trace\n",
"@memoized\n",
"def mfib(n):\n",
" if n <= 0: return 1\n",
" if n == 1: return 1\n",
" else:\n",
" return mfib(n-1) + mfib(n-2)"
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"|-- mfib 10\n",
"| |-- mfib 9\n",
"| | |-- mfib 8\n",
"| | | |-- mfib 7\n",
"| | | | |-- mfib 6\n",
"| | | | | |-- mfib 5\n",
"| | | | | | |-- mfib 4\n",
"| | | | | | | |-- mfib 3\n",
"| | | | | | | | |-- mfib 2\n",
"| | | | | | | | | |-- mfib 1\n",
"| | | | | | | | | | |-- return 1\n",
"| | | | | | | | | |-- mfib 0\n",
"| | | | | | | | | | |-- return 1\n",
"| | | | | | | | | |-- return 2\n",
"| | | | | | | | |-- mfib 1\n",
"| | | | | | | | | |-- return 1\n",
"| | | | | | | | |-- return 3\n",
"| | | | | | | |-- mfib 2\n",
"| | | | | | | | |-- return 2\n",
"| | | | | | | |-- return 5\n",
"| | | | | | |-- mfib 3\n",
"| | | | | | | |-- return 3\n",
"| | | | | | |-- return 8\n",
"| | | | | |-- mfib 4\n",
"| | | | | | |-- return 5\n",
"| | | | | |-- return 13\n",
"| | | | |-- mfib 5\n",
"| | | | | |-- return 8\n",
"| | | | |-- return 21\n",
"| | | |-- mfib 6\n",
"| | | | |-- return 13\n",
"| | | |-- return 34\n",
"| | |-- mfib 7\n",
"| | | |-- return 21\n",
"| | |-- return 55\n",
"| |-- mfib 8\n",
"| | |-- return 34\n",
"| |-- return 89\n"
]
},
{
"data": {
"text/plain": [
"89"
]
},
"execution_count": 65,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mfib(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If we call it again, it remembers the old answer."
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"|-- mfib 10\n",
"| |-- return 89\n"
]
},
{
"data": {
"text/plain": [
"89"
]
},
"execution_count": 66,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mfib(10)"
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"|-- mfib 12\n",
"| |-- mfib 11\n",
"| | |-- mfib 10\n",
"| | | |-- return 89\n",
"| | |-- mfib 9\n",
"| | | |-- return 55\n",
"| | |-- return 144\n",
"| |-- mfib 10\n",
"| | |-- return 89\n",
"| |-- return 233\n"
]
},
{
"data": {
"text/plain": [
"233"
]
},
"execution_count": 67,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mfib(12)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here is a better way with constant space. Do not need n-entry table"
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {},
"outputs": [],
"source": [
"def fibbetter(n):\n",
" return fibbetteraux(n, 1, 1)\n",
"\n",
"\n",
"## Note: this is tail recursive. Reflected in trace output.\n",
"def fibbetteraux(n, x, y):\n",
" if n <= 0:\n",
" return x \n",
" else:\n",
" return fibbetteraux(n-1,y, x+y)"
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"89"
]
},
"execution_count": 69,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"fibbetter(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Timing the fibs\n",
"\n",
"We first untrace the functions."
]
},
{
"cell_type": "code",
"execution_count": 70,
"metadata": {},
"outputs": [],
"source": [
"## no trace this time.\n",
"def fib(n):\n",
" if n == 0:\n",
" return 1\n",
" elif n == 1:\n",
" return 1\n",
" else:\n",
" return fib(n - 1) + fib(n - 2)\n",
" \n",
"def memofibaux(n, table):\n",
" if table[n]: return table[n]\n",
" if n <= 1: return 1\n",
" else:\n",
" table[n] = memofibaux(n-1, table) + memofibaux(n-2, table)\n",
" return table[n]\n",
" \n",
"@memoized\n",
"def mfib(n):\n",
" if n <= 0: return 1\n",
" if n == 1: return 1\n",
" else:\n",
" return mfib(n-1) + mfib(n-2)"
]
},
{
"cell_type": "code",
"execution_count": 71,
"metadata": {},
"outputs": [],
"source": [
"def doit():\n",
" f1 = timeit('fib(10)', setup = 'from __main__ import fib', number=1)\n",
" f2 = timeit('memofib(10)', setup = 'from __main__ import memofib', number=1)\n",
" f3 = timeit('mfib(10)', setup = 'from __main__ import mfib', number=1)\n",
" f4 = timeit('fibbetter(10)', setup = 'from __main__ import fibbetter', number=1)\n",
" return f1, f2, f3, f4"
]
},
{
"cell_type": "code",
"execution_count": 72,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(6.396399112418294e-05,\n",
" 2.163101453334093e-05,\n",
" 3.597204340621829e-05,\n",
" 2.214504638686776e-05)"
]
},
"execution_count": 72,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"doit()"
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {},
"outputs": [],
"source": [
"def doit2(n):\n",
" f1 = timeit('fib(' + str(n) + ')', setup = 'from __main__ import fib', number=1)\n",
" f2 = timeit('memofib(' + str(n) + ')', setup = 'from __main__ import memofib', number=1)\n",
" f3 = timeit('mfib(' + str(n) + ')', setup = 'from __main__ import mfib', number=1)\n",
" f4 = timeit('fibbetter(' + str(n) + ')', setup = 'from __main__ import fibbetter', number=1)\n",
" return f1, f2, f3, f4"
]
},
{
"cell_type": "code",
"execution_count": 74,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(6.357498932629824e-05,\n",
" 2.0317966118454933e-05,\n",
" 3.1539821065962315e-06,\n",
" 8.261995390057564e-06)"
]
},
"execution_count": 74,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"doit2(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Wraps and Wrappers\n",
"\n",
"See Python Cookbook, chapter 9.2"
]
},
{
"cell_type": "code",
"execution_count": 75,
"metadata": {},
"outputs": [],
"source": [
"from functools import wraps\n",
"\n",
"def without_wraps(func):\n",
" def __wrapper(*args, **kwargs):\n",
" return func(*args, **kwargs)\n",
" return __wrapper\n",
" \n",
"def with_wraps(func):\n",
" @wraps(func)\n",
" def __wrapper(*args, **kwargs):\n",
" return func(*args, **kwargs)\n",
" return __wrapper"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We have created two decorators - without and with wraps. We will decorate two dummy functions."
]
},
{
"cell_type": "code",
"execution_count": 76,
"metadata": {},
"outputs": [],
"source": [
"@without_wraps\n",
"def my_func_a():\n",
" \"\"\"Here is my_func_a doc string text.\"\"\"\n",
" pass\n",
" \n",
"@with_wraps\n",
"def my_func_b():\n",
" \"\"\"Here is my_func_b doc string text.\"\"\"\n",
" pass"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will now look at the properties of these functions."
]
},
{
"cell_type": "code",
"execution_count": 77,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['__annotations__',\n",
" '__builtins__',\n",
" '__call__',\n",
" '__class__',\n",
" '__closure__',\n",
" '__code__',\n",
" '__defaults__',\n",
" '__delattr__',\n",
" '__dict__',\n",
" '__dir__',\n",
" '__doc__',\n",
" '__eq__',\n",
" '__format__',\n",
" '__ge__',\n",
" '__get__',\n",
" '__getattribute__',\n",
" '__globals__',\n",
" '__gt__',\n",
" '__hash__',\n",
" '__init__',\n",
" '__init_subclass__',\n",
" '__kwdefaults__',\n",
" '__le__',\n",
" '__lt__',\n",
" '__module__',\n",
" '__name__',\n",
" '__ne__',\n",
" '__new__',\n",
" '__qualname__',\n",
" '__reduce__',\n",
" '__reduce_ex__',\n",
" '__repr__',\n",
" '__setattr__',\n",
" '__sizeof__',\n",
" '__str__',\n",
" '__subclasshook__']"
]
},
"execution_count": 77,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dir(my_func_a)"
]
},
{
"cell_type": "code",
"execution_count": 78,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['__annotations__',\n",
" '__builtins__',\n",
" '__call__',\n",
" '__class__',\n",
" '__closure__',\n",
" '__code__',\n",
" '__defaults__',\n",
" '__delattr__',\n",
" '__dict__',\n",
" '__dir__',\n",
" '__doc__',\n",
" '__eq__',\n",
" '__format__',\n",
" '__ge__',\n",
" '__get__',\n",
" '__getattribute__',\n",
" '__globals__',\n",
" '__gt__',\n",
" '__hash__',\n",
" '__init__',\n",
" '__init_subclass__',\n",
" '__kwdefaults__',\n",
" '__le__',\n",
" '__lt__',\n",
" '__module__',\n",
" '__name__',\n",
" '__ne__',\n",
" '__new__',\n",
" '__qualname__',\n",
" '__reduce__',\n",
" '__reduce_ex__',\n",
" '__repr__',\n",
" '__setattr__',\n",
" '__sizeof__',\n",
" '__str__',\n",
" '__subclasshook__',\n",
" '__wrapped__']"
]
},
"execution_count": 78,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dir(my_func_b)"
]
},
{
"cell_type": "code",
"execution_count": 79,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"None\n",
"__wrapper\n",
"__main__\n"
]
}
],
"source": [
"print (my_func_a.__doc__)\n",
"print (my_func_a.__name__)\n",
"print (my_func_a.__module__)"
]
},
{
"cell_type": "code",
"execution_count": 80,
"metadata": {},
"outputs": [
{
"ename": "AttributeError",
"evalue": "'function' object has no attribute '__wrapped__'",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)",
"Input \u001b[0;32mIn [80]\u001b[0m, in \u001b[0;36m