{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CS 201: Memoization\n",
    "<p>\n",
    "<script language=\"JavaScript\">\n",
    "    document.write(\"Last modified: \" + document.lastModified)\n",
    "</script>\n",
    "    \n",
    "Memoization is a technique to improve the efficiency of your code by avoiding repetitive calculations.  The idea is to remember past calculations so as not to have to repeat them.\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "(require racket)\n",
    "(require racket/base)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Factorial\n",
    "\n",
    "When calculating a factorial, it can be helpful to remember past calculations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "(define (factorial n)\n",
    "  (if (= n 1)\n",
    "      1\n",
    "      (* n (factorial (- n 1)))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<code>24</code>"
      ],
      "text/plain": [
       "24"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(factorial 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<code>120</code>"
      ],
      "text/plain": [
       "120"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(factorial 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "(require racket/trace)\n",
    "(trace factorial)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">(factorial 6)\n",
      "> (factorial 5)\n",
      "> >(factorial 4)\n",
      "> > (factorial 3)\n",
      "> > >(factorial 2)\n",
      "> > > (factorial 1)\n",
      "< < < 1\n",
      "< < <2\n",
      "< < 6\n",
      "< <24\n",
      "< 120\n",
      "<720\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>720</code>"
      ],
      "text/plain": [
       "720"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(factorial 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we evaluate <code>(factorial 6)</code>, we need to calculate <code>(factorial 5)</code>.\n",
    "\n",
    "When we evaluate <code>(factorial 5)</code>, we need to calculate <code>(factorial 4)</code>.\n",
    "\n",
    "It would save us time if we just remembered the results of the past calculations.  We can use a <b>hash table</b> to store those values. For those of you who know python, a hash table is a <b>dict</b> or dictionary.  See <a target=wer href=\"https://docs.racket-lang.org/guide/hash-tables.html\">racket hash tables</a>\n",
    "\n",
    "Below we define a function <code>memoize</code> which adds a hash table memory to a given function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "(define ht (make-hash))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<code>'#hash()</code>"
      ],
      "text/plain": [
       "'#hash()"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ht"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "(hash-set! ht 'john 23)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<code>#t</code>"
      ],
      "text/plain": [
       "#t"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(hash-has-key? ht 'john)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<code>23</code>"
      ],
      "text/plain": [
       "23"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(hash-ref ht 'john)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<code>#f</code>"
      ],
      "text/plain": [
       "#f"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(hash-has-key? ht 'mary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "hash-ref: no value found for key\n",
      "  key: 'mary\n",
      "  context...:\n",
      "   eval-one-top12\n",
      "   /usr/share/racket/pkgs/sandbox-lib/racket/sandbox.rkt:510:0: call-with-custodian-shutdown\n",
      "   /usr/share/racket/collects/racket/private/more-scheme.rkt:148:2: call-with-break-parameterization\n",
      "   .../more-scheme.rkt:261:28\n",
      "   /usr/share/racket/pkgs/sandbox-lib/racket/sandbox.rkt:878:5: loop\n"
     ]
    }
   ],
   "source": [
    "(hash-ref ht 'mary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will use a hash table as the memory for <code>memoize</code>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "(define (memoize func [table (make-hash)])\n",
    "  (lambda (arg)\n",
    "    (cond ((hash-has-key? table arg) \n",
    "           (begin \n",
    "            (display (list 'used-hash-for arg)) \n",
    "            (newline)\n",
    "            (hash-ref table arg)))\n",
    "          (else \n",
    "           (hash-set! table arg  (func arg)) \n",
    "           (hash-ref table arg)))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "(define factorial (memoize factorial))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">(factorial 4)\n",
      "> (factorial 3)\n",
      "> >(factorial 2)\n",
      "> > (factorial 1)\n",
      "< < 1\n",
      "< <2\n",
      "< 6\n",
      "<24\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>24</code>"
      ],
      "text/plain": [
       "24"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(factorial 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">(factorial 5)\n",
      "(used-hash-for 4)\n",
      "<120\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>120</code>"
      ],
      "text/plain": [
       "120"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(factorial 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">(factorial 6)\n",
      "(used-hash-for 5)\n",
      "<720\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>720</code>"
      ],
      "text/plain": [
       "720"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(factorial 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have avoided the repeated calculations!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fibonacci\n",
    "\n",
    "The <a target=rt href=\"https://en.wikipedia.org/wiki/Fibonacci_number\">Fibonacci sequence</a> is found in math and in nature. Here is a simple recursive function to calculate a Fibonacci number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "(define (fib n)\n",
    "  (if (< n 2)\n",
    "      1\n",
    "      (+ (fib (- n 1)) (fib (- n 2)))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<code>5</code>"
      ],
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(fib 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<code>8</code>"
      ],
      "text/plain": [
       "8"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(fib 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<code>13</code>"
      ],
      "text/plain": [
       "13"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(fib 6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "(trace fib)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">(fib 6)\n",
      "> (fib 5)\n",
      "> >(fib 4)\n",
      "> > (fib 3)\n",
      "> > >(fib 2)\n",
      "> > > (fib 1)\n",
      "< < < 1\n",
      "> > > (fib 0)\n",
      "< < < 1\n",
      "< < <2\n",
      "> > >(fib 1)\n",
      "< < <1\n",
      "< < 3\n",
      "> > (fib 2)\n",
      "> > >(fib 1)\n",
      "< < <1\n",
      "> > >(fib 0)\n",
      "< < <1\n",
      "< < 2\n",
      "< <5\n",
      "> >(fib 3)\n",
      "> > (fib 2)\n",
      "> > >(fib 1)\n",
      "< < <1\n",
      "> > >(fib 0)\n",
      "< < <1\n",
      "< < 2\n",
      "> > (fib 1)\n",
      "< < 1\n",
      "< <3\n",
      "< 8\n",
      "> (fib 4)\n",
      "> >(fib 3)\n",
      "> > (fib 2)\n",
      "> > >(fib 1)\n",
      "< < <1\n",
      "> > >(fib 0)\n",
      "< < <1\n",
      "< < 2\n",
      "> > (fib 1)\n",
      "< < 1\n",
      "< <3\n",
      "> >(fib 2)\n",
      "> > (fib 1)\n",
      "< < 1\n",
      "> > (fib 0)\n",
      "< < 1\n",
      "< <2\n",
      "< 5\n",
      "<13\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>13</code>"
      ],
      "text/plain": [
       "13"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(fib 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Oof!  That's a lot of redundant calculations!  Let's memoize that puppy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "(define fib (memoize fib))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(used-hash-for 1)\n",
      "(used-hash-for 2)\n",
      "(used-hash-for 3)\n",
      "(used-hash-for 4)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>13</code>"
      ],
      "text/plain": [
       "13"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(fib 6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(used-hash-for 6)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>13</code>"
      ],
      "text/plain": [
       "13"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(fib 6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(used-hash-for 6)\n",
      "(used-hash-for 5)\n",
      "(used-hash-for 6)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>34</code>"
      ],
      "text/plain": [
       "34"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(fib 8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's a big improvement."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ackermann\n",
    "\n",
    "The <a target=wer href=\"https://en.wikipedia.org/wiki/Ackermann_function\">Ackermann function</a> is a famous (or infamous) recursive function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "(define (ack m n)\n",
    "  (cond ((= m 0) (+ n 1))\n",
    "        ((= n 0) (ack (- m 1) 1))  ;; m > 0\n",
    "        (else \n",
    "         (ack (- m 1) (ack m (- n 1))))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<code>5</code>"
      ],
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(ack 2 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "(trace ack)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">(ack 2 1)\n",
      "> (ack 2 0)\n",
      "> (ack 1 1)\n",
      "> >(ack 1 0)\n",
      "> >(ack 0 1)\n",
      "< <2\n",
      "> (ack 0 2)\n",
      "< 3\n",
      ">(ack 1 3)\n",
      "> (ack 1 2)\n",
      "> >(ack 1 1)\n",
      "> > (ack 1 0)\n",
      "> > (ack 0 1)\n",
      "< < 2\n",
      "> >(ack 0 2)\n",
      "< <3\n",
      "> (ack 0 3)\n",
      "< 4\n",
      ">(ack 0 4)\n",
      "<5\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>5</code>"
      ],
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(ack 2 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While <code>ack</code> would benefit from being memoized, we cannot use our prevous <code>memoize</code> function because <code>ack</code>, unlike <code>factorial</code> and <code>fib</code>, takes two arguments, not one.  We need to modify <code>memoize</code> accordingly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "(define (memoize2 func [table (make-hash)])\n",
    "  (lambda (arg1 arg2)\n",
    "    (cond ((hash-has-key? table (list arg1 arg2)) \n",
    "           (begin \n",
    "            (display (list 'used-hash-for arg1 arg2))\n",
    "            (newline)\n",
    "            (hash-ref table (list arg1 arg2)))) \n",
    "          (else \n",
    "           (hash-set! table (list arg1 arg2) (func arg1 arg2)) \n",
    "           (hash-ref table (list arg1 arg2))))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "(define ack (memoize2 ack))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">(ack 2 1)\n",
      "> (ack 2 0)\n",
      "> >(ack 1 1)\n",
      "> > (ack 1 0)\n",
      "> > >(ack 0 1)\n",
      "< < <2\n",
      "< < 2\n",
      "> > (ack 0 2)\n",
      "< < 3\n",
      "< <3\n",
      "< 3\n",
      "> (ack 1 3)\n",
      "> >(ack 1 2)\n",
      "(used-hash-for 1 1)\n",
      "> > (ack 0 3)\n",
      "< < 4\n",
      "< <4\n",
      "> >(ack 0 4)\n",
      "< <5\n",
      "< 5\n",
      "<5\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>5</code>"
      ],
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(ack 2 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">(ack 2 2)\n",
      "(used-hash-for 2 1)\n",
      "> (ack 1 5)\n",
      "> >(ack 1 4)\n",
      "(used-hash-for 1 3)\n",
      "> > (ack 0 5)\n",
      "< < 6\n",
      "< <6\n",
      "> >(ack 0 6)\n",
      "< <7\n",
      "< 7\n",
      "<7\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>7</code>"
      ],
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(ack 2 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">(ack 3 1)\n",
      "> (ack 3 0)\n",
      "(used-hash-for 2 1)\n",
      "< 5\n",
      "> (ack 2 5)\n",
      "> >(ack 2 4)\n",
      "> > (ack 2 3)\n",
      "(used-hash-for 2 2)\n",
      "> > >(ack 1 7)\n",
      "> > > (ack 1 6)\n",
      "(used-hash-for 1 5)\n",
      "> > > >(ack 0 7)\n",
      "< < < <8\n",
      "< < < 8\n",
      "> > > (ack 0 8)\n",
      "< < < 9\n",
      "< < <9\n",
      "< < 9\n",
      "> > (ack 1 9)\n",
      "> > >(ack 1 8)\n",
      "(used-hash-for 1 7)\n",
      "> > > (ack 0 9)\n",
      "< < < 10\n",
      "< < <10\n",
      "> > >(ack 0 10)\n",
      "< < <11\n",
      "< < 11\n",
      "< <11\n",
      "> >(ack 1 11)\n",
      "> > (ack 1 10)\n",
      "(used-hash-for 1 9)\n",
      "> > >(ack 0 11)\n",
      "< < <12\n",
      "< < 12\n",
      "> > (ack 0 12)\n",
      "< < 13\n",
      "< <13\n",
      "< 13\n",
      "<13\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<code>13</code>"
      ],
      "text/plain": [
       "13"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(ack 3 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Without memoization, <code>ack</code> burns through a boatload of cycles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Racket",
   "language": "racket",
   "name": "racket"
  },
  "language_info": {
   "codemirror_mode": "scheme",
   "file_extension": ".rkt",
   "mimetype": "text/x-racket",
   "name": "Racket",
   "pygments_lexer": "racket",
   "version": "7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}