CS 201: Memoization

Memoization is a technique to improve the efficiency of your code by avoiding repetitive calculations. The idea is to remember past calculations so as not to have to repeat them.

In [1]:
(require racket)
(require racket/base)

Factorial

When calculating a factorial, it can be helpful to remember past calculations.

In [2]:
(define (factorial n)
  (if (= n 1)
      1
      (* n (factorial (- n 1)))))
In [3]:
(factorial 4)
Out[3]:
24
In [4]:
(factorial 5)
Out[4]:
120
In [5]:
(require racket/trace)
(trace factorial)
In [6]:
(factorial 6)

>(factorial 6)
> (factorial 5)
> >(factorial 4)
> > (factorial 3)
> > >(factorial 2)
> > > (factorial 1)
< < < 1
< < <2
< < 6
< <24
< 120
<720
Out[6]:
720

When we evaluate (factorial 6), we need to calculate (factorial 5).

When we evaluate (factorial 5), we need to calculate (factorial 4).

It would save us time if we just remembered the results of the past calculations. We can use a hash table to store those values. For those of you who know python, a hash table is a dict or dictionary. See racket hash tables

Below we define a function memoize which adds a hash table memory to a given function.

In [7]:
(define ht (make-hash))
In [8]:
ht
Out[8]:
'#hash()
In [9]:
(hash-set! ht 'john 23)
In [10]:
(hash-has-key? ht 'john)
Out[10]:
#t
In [11]:
(hash-ref ht 'john)
Out[11]:
23
In [12]:
(hash-has-key? ht 'mary)
Out[12]:
#f
In [13]:
(hash-ref ht 'mary)

hash-ref: no value found for key
  key: 'mary
  context...:
   eval-one-top12
   /usr/share/racket/pkgs/sandbox-lib/racket/sandbox.rkt:510:0: call-with-custodian-shutdown
   /usr/share/racket/collects/racket/private/more-scheme.rkt:148:2: call-with-break-parameterization
   .../more-scheme.rkt:261:28
   /usr/share/racket/pkgs/sandbox-lib/racket/sandbox.rkt:878:5: loop

Now we will use a hash table as the memory for memoize

In [45]:
(define (memoize func [table (make-hash)])
  (lambda (arg)
    (cond ((hash-has-key? table arg) 
           (begin 
            (display (list 'used-hash-for arg)) 
            (newline)
            (hash-ref table arg)))
          (else 
           (hash-set! table arg  (func arg)) 
           (hash-ref table arg)))))
In [46]:
(define factorial (memoize factorial))
In [47]:
(factorial 4)

>(factorial 4)
> (factorial 3)
> >(factorial 2)
> > (factorial 1)
< < 1
< <2
< 6
<24
Out[47]:
24
In [48]:
(factorial 5)

>(factorial 5)
(used-hash-for 4)
<120
Out[48]:
120
In [49]:
(factorial 6)

>(factorial 6)
(used-hash-for 5)
<720
Out[49]:
720

We have avoided the repeated calculations!

Fibonacci

The Fibonacci sequence is found in math and in nature. Here is a simple recursive function to calculate a Fibonacci number.

In [50]:
(define (fib n)
  (if (< n 2)
      1
      (+ (fib (- n 1)) (fib (- n 2)))))
In [51]:
(fib 4)
Out[51]:
5
In [16]:
(fib 5)
Out[16]:
8
In [52]:
(fib 6)
Out[52]:
13
In [18]:
(trace fib)
In [19]:
(fib 6)

>(fib 6)
> (fib 5)
> >(fib 4)
> > (fib 3)
> > >(fib 2)
> > > (fib 1)
< < < 1
> > > (fib 0)
< < < 1
< < <2
> > >(fib 1)
< < <1
< < 3
> > (fib 2)
> > >(fib 1)
< < <1
> > >(fib 0)
< < <1
< < 2
< <5
> >(fib 3)
> > (fib 2)
> > >(fib 1)
< < <1
> > >(fib 0)
< < <1
< < 2
> > (fib 1)
< < 1
< <3
< 8
> (fib 4)
> >(fib 3)
> > (fib 2)
> > >(fib 1)
< < <1
> > >(fib 0)
< < <1
< < 2
> > (fib 1)
< < 1
< <3
> >(fib 2)
> > (fib 1)
< < 1
> > (fib 0)
< < 1
< <2
< 5
<13
Out[19]:
13

Oof! That's a lot of redundant calculations! Let's memoize that puppy.

In [53]:
(define fib (memoize fib))
In [54]:
(fib 6)

(used-hash-for 1)
(used-hash-for 2)
(used-hash-for 3)
(used-hash-for 4)
Out[54]:
13
In [55]:
(fib 6)

(used-hash-for 6)
Out[55]:
13
In [56]:
(fib 8)

(used-hash-for 6)
(used-hash-for 5)
(used-hash-for 6)
Out[56]:
34

That's a big improvement.

Ackermann

The Ackermann function is a famous (or infamous) recursive function.

In [22]:
(define (ack m n)
  (cond ((= m 0) (+ n 1))
        ((= n 0) (ack (- m 1) 1))  ;; m > 0
        (else 
         (ack (- m 1) (ack m (- n 1))))))
In [26]:
(ack 2 1)
Out[26]:
5
In [27]:
(trace ack)
In [28]:
(ack 2 1)

>(ack 2 1)
> (ack 2 0)
> (ack 1 1)
> >(ack 1 0)
> >(ack 0 1)
< <2
> (ack 0 2)
< 3
>(ack 1 3)
> (ack 1 2)
> >(ack 1 1)
> > (ack 1 0)
> > (ack 0 1)
< < 2
> >(ack 0 2)
< <3
> (ack 0 3)
< 4
>(ack 0 4)
<5
Out[28]:
5

While ack would benefit from being memoized, we cannot use our prevous memoize function because ack, unlike factorial and fib, takes two arguments, not one. We need to modify memoize accordingly.

In [29]:
(define (memoize2 func [table (make-hash)])
  (lambda (arg1 arg2)
    (cond ((hash-has-key? table (list arg1 arg2)) 
           (begin 
            (display (list 'used-hash-for arg1 arg2))
            (newline)
            (hash-ref table (list arg1 arg2)))) 
          (else 
           (hash-set! table (list arg1 arg2) (func arg1 arg2)) 
           (hash-ref table (list arg1 arg2))))))
In [30]:
(define ack (memoize2 ack))
In [31]:
(ack 2 1)

>(ack 2 1)
> (ack 2 0)
> >(ack 1 1)
> > (ack 1 0)
> > >(ack 0 1)
< < <2
< < 2
> > (ack 0 2)
< < 3
< <3
< 3
> (ack 1 3)
> >(ack 1 2)
(used-hash-for 1 1)
> > (ack 0 3)
< < 4
< <4
> >(ack 0 4)
< <5
< 5
<5
Out[31]:
5
In [32]:
(ack 2 2)

>(ack 2 2)
(used-hash-for 2 1)
> (ack 1 5)
> >(ack 1 4)
(used-hash-for 1 3)
> > (ack 0 5)
< < 6
< <6
> >(ack 0 6)
< <7
< 7
<7
Out[32]:
7
In [58]:
(ack 3 1)

>(ack 3 1)
> (ack 3 0)
(used-hash-for 2 1)
< 5
> (ack 2 5)
> >(ack 2 4)
> > (ack 2 3)
(used-hash-for 2 2)
> > >(ack 1 7)
> > > (ack 1 6)
(used-hash-for 1 5)
> > > >(ack 0 7)
< < < <8
< < < 8
> > > (ack 0 8)
< < < 9
< < <9
< < 9
> > (ack 1 9)
> > >(ack 1 8)
(used-hash-for 1 7)
> > > (ack 0 9)
< < < 10
< < <10
> > >(ack 0 10)
< < <11
< < 11
< <11
> >(ack 1 11)
> > (ack 1 10)
(used-hash-for 1 9)
> > >(ack 0 11)
< < <12
< < 12
> > (ack 0 12)
< < 13
< <13
< 13
<13
Out[58]:
13

Without memoization, ack burns through a boatload of cycles.

In [ ]: