Spring 2014: Computer Science 201 Lecture Summaries

Lecture 1: Introductory Lecture. (1/13/14)
Lecture 2: Racket I. (1/15/14)
Lecture 3: Racket II. (1/17/14)
Lecture 4: Racket III. (1/22/14)
Lecture 5: Racket IV. (1/24/14)
Lecture 6: Racket V. (1/27/14)
Lecture 7: Racket VI. (1/29/14)
Lecture 8: Racket VII, Computability I. (1/31/14)
Lecture 9: Computability II. (2/3/14)
Lecture 10: Computability III. (2/5/14)
Lecture 11: Computability IV. (2/7/14)
Lecture 12: Boolean functions and expressions I. (2/10/14)
Lecture 13: Boolean functions and expressions II. (2/12/14)
Lecture 14: Boolean functions and expressions III. (2/14/14)
Lecture 15: Review for Midterm 1. (2/17/14)
Lecture 16: Gates and circuits I. (2/19/14)
Lecture 17: Gates and circuits II. (2/21/14)
Lecture 18: Gates and circuits III. (2/24/14)
Lecture 19: Gates and circuits IV. (2/26/14)
Lecture 20: Computer Architecture I. (2/28/14)
Lecture 21: Computer Architecture II. (3/3/14)
Lecture 22: Computer Architecture III. (3/5/14)
Lecture 23: Computer Architecture IV. (3/7/14)
Lecture 24: Strings and languages I. (3/24/14)
Lecture 25: Strings and languages II. (3/26/14)
Lecture 26: Strings and languages III. (3/28/14)
Lecture 27: Strings and languages IV. (3/31/14)
Lecture 28: Strings and languages V. (4/2/14)
Lecture 29: Strings and languages VI. (4/4/14)
Lecture 30: Review for second exam. (4/7/14)
Lecture 31: Compilers. (4/9/14)
Lecture 32: Running time of programs I. (4/11/14)
Lecture 33: Running time of programs II. (4/14/14)
Lecture 34: Running time of programs III. (4/16/14)
Lecture 35: Running time of programs IV. (4/18/14)
Lecture 36: Mutators (and a little bit about objects). (4/21/14)
Lecture 37: Recursion and memoization. (4/23/14)
Lecture 38: Recursion, memoization and dynamic programming; P vs. NP. (4/25/14)

4/25/14	Lecture 38: Recursion, memoization and dynamic programming; P vs. NP. Please see the notes: Recursion and memoization and Making change: memoization and dynamic programming. Note that P vs. NP will NOT be on the final exam.
4/23/14	Lecture 37: Recursion and memoization. Please see the notes: Recursion and memoization and Making change: memoization and dynamic programming.
4/21/14	Lecture 36: Mutators (and a little bit about objects). Summary. Representing an "object" as a procedure with local data. The Racket mutator set! and how to use it to implement a counter object. Please see the notes: Environments in Racket and the Mutator set!. In a setting of pure functional programming, calling a procedure with the same arguments will always produce the same result. This is not the case with Racket's library procedure random. Calling (random 10) might return 6 one time and 0 the next. The random procedure has some local state that changes with each call and allows it to produce different responses to the same argument. An "object" in the sense of object oriented programming can be considered to be a bundle of data and procedures (or "methods") to operate on the data. We can represent an object in Racket as a procedure with local state. Before we do that, we look at one of Racket's mutators, namely set! The exclamation point is part of the name of the procedure, and is a convention to indicate that the procedure is a mutator, that is, changes the value of a variable or other data. The form of set! is (set! variable expression) The expression is evaluated to get a value, and then the variable is looked up in the relevant environment (just as we would look it up to find its value) -- in that environment is binding is changed to the value of the expression. Note that the variable must already be defined in order to use it in a set! expression. Attempting to set! a variable that is not defined is an error. This is analogous to having to declare a variable before it can have a value assigned to it. Example of the use of set! in the top-level environment. > (define count 0) > count 0 > (set! count (+ 1 count)) > count 1 > Evaluating the expression (define count 0) adds count to the top-level environment with its value bound to 0. Then evaluating the expression (set! count (+ 1 count)) evaluates the expression (+ 1 count) in the usual way to get 1, and looks up count, finding it in the top-level environment, where its value is currently 0. The value of count is changed to 1 in the top-level environment; now when its value is looked up, it will be 1. This behavior is enough to define a simple counter procedure which will return different values for the same arguments, depending on the value of count. We write the following procedure. > (define (counter cmd) (case cmd [(increment!) (set! count (+ 1 count))] [(zero!) (set! count 0)]) count) > (counter 'increment!) 2 > (counter 'increment!) 3 > (counter 'zero!) 0 > (counter 'increment!) 1 > count 1 > Note that when this procedure refers to count, the search to find the relevant environment finds it in the top-level environment, (where we assume we previously defined it and incremented it to 1.) Thus in this case the variable count functions as a "global" variable, accessible everywhere in the file. (We could also have written counter without the case statement, using a cond expression and equal? tests.) One property we might want for an object is that its data is not global, but local and private to the object, so that it cannot be read or changed without invoking the "methods" (or procedures) associated with the object. We can achieve this goal with the following definition. Assume that we have re-entered the Racket interpreter afresh, so that the preceding definition of count and counter are gone. (This is not what we did in lecture, but should be less confusing for the notes.) > (define counter1 (let ((count 0)) (lambda (cmd) (case cmd [(increment!) (set! count (+ 1 count))] [(zero!) (set! count 0)]) count))) > (counter1 'increment!) 1 > (counter1 'increment!) 2 > This creates a procedure named counter1 with a private local variable count, whose value can only be inspected and changed by calls to the procedure. This protects against other procedures accidentally or deliberately changing the value of count other than through the interface provided by this procedure. For an analysis of (a variant of) this procedure, and a higher-level counter-creation procedure, in terms of Racket environments, please see the notes: Environments in Racket and the Mutator set!. If we arrange the lets and lambdas in a different fashion, we get a counter procedure that UTTERLY FAILS at its task. (define not-a-counter (lambda (cmd) (let ((count 0)) (case cmd [(increment!) (set! count (+ 1 count)) count] [(zero!) (set! count 0) count])))) As examples of the behavior of this procedure, we have the following. > (not-a-counter 'increment!) 1 > (not-a-counter 'increment!) 1 > (not-a-counter 'zero!) 0 > (not-a-counter 'increment!) 1 > Think about what is happening in terms of environments to keep this from behaving like a counter.
4/18/14	Lecture 35: Running time of programs IV. Summary. Running times of insertion sort and merge sort, continued. Please see the notes: Sorting.
4/16/14	Lecture 34: Running time of programs III. Summary. Running times of insertion sort and merge sort. Please see the notes: Sorting.
4/14/14	Lecture 33: Running time of programs II. Summary. Definitions and examples for asymptotic notation: big-Oh, Big Omega, Big Theta. An (insert item lst) procedure in Racket. Analyzing running time of Racket programs: what are the constant time operations? A representation of lists in TC-201 memory, and analysis to show that car (first), cdr (rest), and cons are constant time operations. Box and pointer representation of lists. Please see the notes: Running time of a program and List representation. For the insert procedure, see Sorting.
4/11/14	Lecture 32: Running time of programs I. Summary. Introduction to the idea of running time and what we can count as constant time operations. We analyzed the insertion sort pseudocode from the Wikipedia article Insertion sort. The pseudocode for insertion sort from the Wikipedia article: for i ← 1 to length(A) x ← A[i] j ← i while j > 0 and A[j-1] > x A[j] ← A[j-1] j ← j - 1 A[j] ← x This is (slightly) incorrect: the array A has 0-based indexing, so when i = length(A), the array reference A[i] will be out of bounds; imagine the first line corrected to have length(A)-1 instead of length(A). We analyzed this program by thinking of it translated into TC-201 assembly language and estimating the number of TC-201 assembly language instructions that would be executed to sort an array of n numbers. The new element here is how to account for the time to access the array A. In this pseudocode, the array A is a finite sequence of mutable elements (like a vector in Racket) each of which is accessed by its index, a number from 0 to the length of the array minus 1. If we consider a TC-201 implementation of an array A of numbers we could allocate a number of contiguous memory registers equal to the length of the array, and store the numbers in order in those memory locations. In addition, we would store the address of the start of the array in another memory location, call it astart. Then the TC-201 instructions to find the value of A[i] would load astart into the accumulator, add the value of i to it, and store it in a temporary location, say temp, and then execute "loadi temp" to get the value of A[i] into the accumulator. This is a constant number of TC-201 instructions to access any of the elements of the array. To change the value of A[i], we could use a similar address calculation to get the memory address of the i-th element of A into temp, and then execute "load value" and "storei temp" to change the value of A[i] to the number in the memory register value. This is similarly a constant number of instructions. Thus it makes sense to count array references a constant time operations. With that understanding of array references, a straightforward translation of a for-loop into TC-201 instructions, and the observation that assignment, addition, subtraction and comparison are constant time operations, our conclusions about the running time of this implementation of insertion sort were: best case time of Theta(n) and worst case time of Theta(n^2). Please see the notes: Running time of a program.
4/9/14	Lecture 31: Compilers. Summary. Overview of what a compiler does. For a description of Backus-Naur Form (BNF) notation for context free grammars, please see the notes: Context free languages, context free grammars, and BNF. Example of a BNF grammar: BNF for MiniJava. Example of a Statement in MiniJava: MiniJava Statement. Character-level view of the file: characters comprising MiniJava Statement. (This was produced by the Racket program: show-chars.rkt.) Description of how a simple compiler might generate assembly-language code from the parse tree for the MiniJava Statement. A compiler takes as input a program in a higher-level language like Java or C, and produces as output an equivalent program in assembly language (or machine language) for a particular kind of computer, which can then be assembled and loaded into the memory of that kind of computer, and run. A program in a higher-level language is typically represented by a (longish) sequence of characters, that is, a string. (See characters comprising MiniJava Statement. for a character-level view of the MiniJava Statement.) The work of a compiler can be understood in terms of phases. First, during the lexical analysis phase, the input string is divided into separate "tokens" including identifiers, keywords, numbers, strings, operator symbols (eg, +, &&), delimiters (eg, parentheses, braces, commas, semicolons) and the like; this process is often done with the aid of a DFA. In the example given, the lexical analysis would ignore the first 4 spaces, and produce a token "{", then ignore the newline character and the next 5 spaces, then collect the characters "s", "u", "m", followed by space, to produce the IDENTIFIER "sum", ignore the trailing space, produce the token "=", ignore the next space, collect the character "0" followed by ";" to produce the INTEGER_LITERAL "0", then produce the token ";", and so on through the file. This process transmutes the string of characters into a string of tokens -- the BNF grammar is written in terms of a terminal alphabet of tokens rather than characters. Next is the parsing phase; the sequence of tokens is parsed using a context free grammar of the higher-level language, producing a parse tree for the program. If the program does not parse correctly according to the grammar (ie, is not in the language of all correct programs in that higher-level language), the compiler attempts to return an informative error message allowing the programmer to isolate the error. There is a phase of semantic checking, which includes matching the types of arguments with the types of functions; an error message is returned if the types do not match properly. There is a phase of code generation; assembly language instructions (or machine instructions) are generated from the parse tree to construct an equivalent program in assembly language (or machine language.) There is a phase of "optimization" in which the compiler attempts various improvements of the resulting program, eliminating redundant or unreachable instructions, moving unnecessarily repeated instructions out of loops, and many others. A production compiler ususally offers the programmer options for more or less aggressive optimization of his or her program. Unfortunately, even the most aggressive optimization cannot compensate for a poor choice of algorithm, so the results may be far from "optimal". In modern compilers these and other phases may be repeated and intermingled. We have not described how to parse a string according to a context-free grammar to get a parse tree. Instead, we will assume we already have a parse tree for a MiniJava Statement, and sketch the process of generating an equivalent TC-201 assembly language program. This sketch is intended to give some insight into the code-generation phase of a compiler. We consider the MiniJava Statement: { sum = 0; n = 1; while (n < 10) {sum = sum + n; n = n + 1;} System.out.println(sum); } This is a correct MiniJava Statement, which has a parse tree as follows (with abbreviations St, Exp, ID, and IL for Statement, Expression, IDENTIFIER, and INTEGER_LITERAL.) Note that since Identifier can only be replaced by IDENTIFIER, they have been identified below. St --------------------------------------------- / \| \ \ \ \ { \| \ \ \ } St St St St ------ ------ ------------- ------------------------ / \| \ \ / \| \ \ \| \ \ \ \ \| \ \ \ \ ID = Exp ; ID = Exp ; while ( Exp ) \ System.out.println ( Exp ) ; \| --- \| --- --- \ --- sum \| n \| / \| \ \ \| IL IL Exp < Exp \ ID \| \| --- --- \ \| 0 1 \| \| \ sum ID IL \ \| \| St n 10 --------------------- / / \ \ { / \ } St St ------ ------- / \| \ \ / \| \ \ ID = Exp ; ID = Exp ; \| --- \| --- sum / \| \ n / \| \ / + \ / + \ Exp Exp Exp Exp --- --- --- --- \| \| \| \| ID ID ID IL \| \| \| \| sum n n 1 Note that we have interpreted productions of the form Statement ::= "{" (Statement)* "}" as allowing a parse tree in which a node labeled by the nonterminal symbol Statement may have a leftmost child labeled by {, a rightmost child labeled by }, and zero or more children (in between) labeled by Statement. We also assume that the lexical items ID and IL are annotated by the actual identifier or integer literal they correspond to. (For another example of a MiniJava Statement parse tree, see Another MiniJava parse tree.) Now the goal is to sketch the process of generating code for the TC-201 computer for this parse tree to suggest how it could be automated. We need to reserve one memory location in the TC-201 for each identifier and integer literal in the program, so we can first gather them up and construct data statements for them: sum: data 0 n: data 0 c0: data 0 c1: data 1 c10: data 10 We've assigned simple symbolic names to the integer literals 0, 1, 10. According to our usual convention, these data statements will be placed after the instructions for the program. Then we may proceed recursively, processing the parse tree from the root (top node) down. At the root we have a Statement that consists of {, four Statements, and }. Clearly the { and }, though necessary for the syntax of the program, do not further contribute to the meaning of the program. A Statement that consists of a sequence of Statements can be compiled by (recursively) generating code for each of constituent Statements, and then placing that code in sequence in the memory, because the meaning of a sequence of Statements in MiniJava is: execute the first Statement, then execute the second Statement, then execute the third Statement, and so on, for each of the Statements in the sequence. The first of the four Statements is an assignment, which is parsed as ID = Exp ; (Recall that we've replaced Identifier by IDENTIFIER, abbreviated by ID.) The instructions for an assignment are (recursively) generate code to calculate the value of the Expression on the right hand side and get that value into the accumulator, and follow that by a store instruction into the memory location for the Identifier, in this case, sum. Schematically, we have (code to get value of Expression in accumulator) store sum In this case, the Expression is just an INTEGER_LITERAL, so the instructions to get its value into the accumulator consist of just a load instruction from the memory location that holds the integer literal, in this case, c0. Thus, the code for the first of the four Statements is: load c0 store sum This correctly represents the intended meaning of the MiniJava Statement "sum = 0;". The generation of instructions for the next MiniJava Statements, "n = 1;" is similar, and results in the instructions: load c1 store n The third Statement of the four is more complex, it is a while Statement that is parsed as "while" "(" Expression ")" Statement The Expression gives the condition for the while body to be executed, and the Statement is the while body. The instructions generated form a loop that tests the while condition -- if it is false, execution jumps to the next Statement in sequence, and if it is true, the instructions for the while body are executed, and at the end there is a jump back to the instructions to test the while condition again. Schematically, we have: loop: (instructions to put value of Expression in accumulator) skippos jump next (instructions for the Statement (while body)) jump loop next: ... Here we have assumed that the while body will be executed as long as the value of the Expression is positive. If we make the convention that positive numbers represent true and non-positive numbers represent false, then an Expression with a positive value will cause the while body to be executed again, and an Expression with a zero or negative value will cause the while Statement to terminate. The Expressions we have in this program are just IDENTIFIERS, INTEGER_LITERALS, sum Expressions, of the form Exp + Exp and less than Expressions, of the form Exp < Exp Sum expressions can be evaluated by recursively evaluating the first Expression and saving its value in a temporary location, recursively evaluating the second Expression, and using an add instruction to add the value in the temporary location to it. Schematically, we have (instructions to put value of first Expression in accumulator) store temp1 (instructions to put value of second Expression in accumulator) add temp1 Of course, we have to be careful with the allocation of temporary locations -- we don't want the instructions calculating the value of the second Expression (which itself could involve numerous recursive calculations) to overwrite the value of temp1. For this program, the sum Expressions are simple. For Expression "sum + n" we get load sum store temp1 load n add temp1 It is clear that strictly following our scheme, we will produce redundant instructions -- above, there is no need to use temp1, since we could just use "load sum" and "add n". This is one kind of local optimization that a compiler might make. For Expression "n + 1" we get load n store temp1 load c1 add temp1 which is similarly redundant. For a less than Expression, we can (recursively) calculate the value of the first Expression and save it in a temporary location, (recursively) calculate the value of the second Expression, and subtract the value in the temporary location from it. This will have the effect of putting the value of the second Expression minus the value of the first Expression in the accumulator. Barring arithmetic overflow (which we here ignore) this will be positive if and only if the first Expression is less than the second Expression. Schematically: (instructions to calculate the value of the first Expression) store temp1 (instructions to calculate the value of the second Expression) sub temp1 For the Expression "n < 10" we get the instructions: load n store temp1 load c10 sub temp1 Once again, rather redundant -- we could just use "load c10" followed by "sub n" in this case. This will leave a positive number in the accumulator if n < 10, and a zero or negative number if n >= 10. Because the body of the while Statement is itself a sequence of two Statements that are assignments, and we have considered all the types of Expressions that occur, we see that the instructions generated for the while Statement (without optimization) are: loop: load n while (n < 10) store temp1 load c10 sub temp1 skippos jump next load sum {sum = sum + n; store temp1 load n add temp1 store sum load n n = n + 1;} store temp1 load c1 add temp1 store n jump loop next: ... where next: is the label of the following statement. The fourth Statement at the top level is a print statement, parsed as: "System.out.println" "(" Expression ")" ";" For this we have the schematic: (code to put value of Expression in accumulator) output This will print out the integer value of the Expression. Putting it all together (with an added halt instruction to stop the program after it has executed), we get the following TC-201 assembly-language program: load c0 {sum = 0; store sum load c1 store n n = 1; loop: load n while (n < 10) store temp1 load c10 sub temp1 skippos 0 jump next load sum {sum = sum + n; store temp1 load n add temp1 store sum load n n = n + 1; store temp1 load c1 add temp1 store n jump loop } next: load sum System.out.println(sum); output 0 halt 0 } halt added to stop program sum: data 0 n: data 0 c0: data 0 c1: data 1 c10: data 10 temp1: data 0 additional temporary location needed
4/7/14	Lecture 30: Review for second exam. Summary. Solutions for practice second exam were explained. Second exam took place 7-8:30 pm Tuesday, April 8.
4/4/14	Lecture 29: Strings and languages VI. Summary. The Chomsky hierarchy of regular, context free, context sensitive, and type 0 languages. An example context free grammar: G1. Definitions of context free grammar, derivation, parse tree, the language of a context free grammar. A DFA to recognize the language L(G1). Examples of context free languages that are NOT regular: the language of n a's followed by n b's, the language of balanced parentheses. What about the language of all strings of a's whose length is a Fibonacci number? It is not regular, but is it context free? The Chomsky hierarchy (named after the linguist Noam Chomsky) consists of four families of languages, each family a proper subset of the next. The smallest family is the regular languages, next is the context free languages, third is the context sensitive languages, and the largest is the type-0 or "recursively enumerable" languages. This lecture will consider the second family, the context free languages. Please see the notes: Context free languages, context free grammars, and BNF. We'll start with an example of a context free grammar, and then explain the general definition. S -> NP VP NP -> Det N \| PN Det -> the \| a N -> cat \| dog \| mouse PN -> it VP -> VI \| VT NP \| V3 that S VI -> slept \| swam VT -> chased \| evaded V3 -> believed \| dreamed A context free grammar has two disjoint finite alphabets, the nonterminal symbols (above these are: S, NP, Det, N, PN, VP, VI, VT, V3) and the terminal symbols (above these are: the, a, cat, dog, mouse, it, that, slept, swam, chased, evaded, believed, dreamed). One of the nonterminal symbols is distinguished as the start nonterminal (above: S, traditionally abbreviating "sentence" in linguistic applications of context free grammars.) These symbols appear in rules, each of which has a left hand side that is a nonterminal, and a right hand side that is a concatenation of terminals and nonterminals. The first rule above is "S -> NP VP" which has lefthand side S and right hand side "NP VP", which is a concatenation of the nonterminals NP and VP. The rule "PN -> it" has left hand side PN, and righthand side just the terminal symbol "it". Several rules with the same left hand side may be abbreviated by listing the left hand side once, the arrow ->, then the several right hand sides separated by the symbol \| (for "or", as in regular expressions.) Thus, the line "NP -> Det N \| PN" represents two rules, "NP -> Det N" and "NP -> PN". Similarly, the line "VP -> VI \| VT NP \| V3 that S" represents three rules: "VP -> VI", "VP -> VT NP", and "VP -> V3 that S". What can we do with a context free grammar? We can "derive" strings of terminal symbols from the start symbol. Starting with the start symbol, we can repeatedly apply the following procedure: find a nonterminal symbol in the current string, and a rule with that nonterminal symbol as its left hand side. Replace one occurrence of the nonterminal symbol with the right hand side of the rule to yield a new string. This stops when the string consists entirely of terminal symbols -- this is the string derived by the sequence of steps we took. So, for the example grammar, starting with the start symbol S, we have just the string: S We choose a nonterminal symbol (not much choice: just S) and a rule with S on the left hand side (again, not much choice, just S -> NP VP) and substitute the right hand side for one occurrence of the nonterminal to get: NP VP Because this string has at least one nonterminal symbol, we can keep going. If we choose nonterminal symbol NP to rewrite, we have a choice between the rules "NP -> Det N" and "NP -> PN". If we choose the former and replace NP with the right hand side, we get: Det N VP If we now choose the nonterminal VP, we have a choice of three possible rules. If we choose the rule "VP -> V3 that S", we get: Det N V3 that S Note that the terminal symbol "that" will stay put -- the rules do not allow us to rewrite terminal symbols. We could choose the nonterminal Det and the rule "Det -> the" to get: the N V3 that S Now we show several steps of this process: the N dreamed that S (using V3 -> dreamed) the mouse dreamed that S (using N -> mouse) the mouse dreamed that NP VP (using S -> NP VP) the mouse dreamed that PN VP (using NP -> PN) the mouse dreamed that it VP (using PN -> it) the mouse dreamed that it VI (using VP -> VI) the mouse dreamed that it slept (using VI -> slept) At this point, the process stops because the string consists entirely of terminal symbols. We have derived the string "the mouse dreamed that it slept" from the start symbol S. The language of a context free grammar is the set of all strings of terminal symbols that can be derived in this way from the start nonterminal. So "the mouse dreamed that it slept" is a string in the language of this grammar. There are infinitely many others, for example, "the cat believed that a mouse dreamed that the cat evaded a dog" and "it swam". A language (ie, set of strings) is context free if and only if there exists a context free grammar of which it is the language. It's not too hard to show (though beyond the scope of this course) that every regular language is context free. Shortly we'll see a context free grammar for the language {a^nb^n : n > 0}, that is, the set of strings of n a's followed by n b's, for all positive integers n. This language is NOT regular, and is therefore evidence that not all context free languages are regular. An alternative way to indicate that a terminal string can be derived from the start nonterminal is to give a parse tree for the string with respect to the grammar. For example, a parse tree for the string "the mouse dreamed that it slept" with respect to the example grammar is the following. S ---- / \ NP VP ---- -------- / \ / \ \ Det N V3 that S --- -- -- ---- \| \| \| / \ the mouse dreamed NP VP -- -- \| \| PN VI -- -- \| \| it slept The root node is labeled by the start nonterminal, S. Each node is either an internal node (and has children) or a leaf node (and has no children). Each leaf node is labeled by a terminal symbol. Each internal node is labeled by a nonterminal symbol. Taking the nonterminal symbol on an internal node as the left hand side, and the sequence of symbols (in order) of its (direct) children as the right hand side, the result is a rule in the grammar. Thus, at the root node, the label is S, the children are labeled NP and VP, and there is a rule "S -> NP VP" and so on for all the internal nodes of the parse tree. Note that this does demonstrate that there is a derivation of the string "the mouse dreamed that it slept" from S, though it does not completely specify the order in which the grammar rules were used to rewrite the nonterminals. So, in a derivation, we could expand NP before VP or vice versa, but in the parse tree it does not matter which comes first. In fact, the language of this particular example grammar happens to be regular. To verify this, we can give a deterministic finite acceptor (DFA) to recognize the language: Diagram of DFA for mouse, cat, dog context free language. To see that there are context free languages that are not regular, we can consider the example of n a's followed by n b's, for all positive integers n, or the language of balanced parentheses, neither of which is regular. A context free grammar for {a^nb^n : n > 0} is as follows, where there is one nonterminal symbol S, which is also the start symbol, and two terminal symbols {a, b}, and two rules given as follows. S -> ab \| aSb For example, a parse tree for aaabbb with respect to this grammar is the following. S --- / \| \ a S b --- / \| \ a S b --- / \ a b A grammar of balanced parentheses can be given as follows. S -> () \| (S) \| SS Here there is one nonterminal, S, which is also the start symbol, and two terminal symbols "(" and ")", and three rules. To see that the string (())() is in the language of this grammar, we consider the following parse tree. S --- / \ / \ S S --- --- / \| \ / \ ( S ) ( ) --- / \ ( ) A slightly more complex grammar could generate strings of balanced parentheses and square brackets, whose language would include strings like [()](([])). Question: What about the non-regular language of {a^n : n is a Fibonacci number} -- is this language context free or not?
4/2/14	Lecture 28: Strings and languages V. Summary. Algorithms relating to regular languages. Are there non-regular languages? YES. Examples of non-regular languages. There are some useful algorithms related to regular languages, which we will not be covering in this course. There are (not necessarily efficient) algorithms to convert a DFA to a regular expression and to convert a regular expression to a DFA. There are efficient algorithms for the following two problems. Given a DFA and a string, determine whether the DFA accepts the string. This is called the membership problem (is the string a member of the language of the DFA?) and is part of the homework for this topic. Given two DFAs, say M1 and M2, determine whether they accept the same language. This is called the equivalence problem (do M1 and M2 have the same behavior, even though as machines they may look quite different?) It is not part of the homework for this topic. Are there non-regular languages? YES. Reason (1): there are uncountably many languages, but only countably many regular languages. Reason (2): there are languages whose membership problem is uncomputable, but the membership problem is computable (efficiently!) for every regular language. Reason (3): there are specific languages we can show to be non-regular, for example, the language of all strings of a's whose length is a Fibonacci number, the language of all strings of n a's followed by n b's, for any nonnegative integer n, the language of all strings of balanced parentheses. For reason (1): if we fix a (finite) alphabet A of symbols, then the set of all regular expressions over the alphabet A is countable. To see this, note that we can list all the strings over the alphabet A together with the (finite set) of symbols used to express the empty string, concatenation, union, Kleene start, and parentheses in an infinite sequence by first listing the strings of length 1, then the strings of length 2, then the strings of length 3, and so on. Many of these strings are not correct regular expressions, but we can imagine just crossing all of them off, and the resulting subsequence of correct regular expressions will still be numberable by the positive integers, and will include EVERY regular expression over the alphabet A. Thus, the set of all regular languages over the alphabet A is also countable. However, the set of ALL languages over the alphabet A is uncountable. To see this, we first list all of the strings over A (in order of increasing length, as above), say s1, s2, s3, s4, ... . Then we can represent any language L over the alphabet A by an infinite sequence of 0's and 1's, say b1, b2, b3, b4, ..., where bi = 1 if si is in L, and bi = 0 if si is not in L. Then a diagonalization argument similar to the one we saw before shows that the set of ALL languages is not countable. Hence there must be (uncountably many, but at least one) language over the alphabet A that is not regular. Reason (2) (not covered in lecture, not material for exams) is that for every regular set L there exists a procedure P_L that takes as input a string s and decides whether s is in L and outputs #t if so and #f if not. P_L is the membership procedure for L. This follows from the fact that there is a procedure to decide for a DFA M and a string s, whether or not M accepts s. However, we can define a language L_H (that is, set of strings) for which there is no membership procedure. L_H consists of the set of all strings s encoding Turing machines such that s on input s halts. Because a membership procedure for L_H would allow us to solve the Halting Problem for Turing machines, we know that no membership procedure can exist for L_H. Thus L_H is a non-regular language. This is a specific non-regular language (as opposed to the non-specific existence proof based on countability and uncountability), but still pretty abstract! Reason (3): there are specific, concrete languages we can show to be non-regular. One example, covered in lecture, is the language of all strings of a's that have a length that is a Fibonacci number. In symbols: L_F = {a^n : n is a Fibonacci number} Here we've used the "exponential" notation for repeated concatenation, so that a^n represents a string of n a's. The Fibonacci numbers are 1, 1, 2, 3, 5, 8, 13, 21, ..., defined by the recurrence: F(n) = F(n-1) + F(n-2) for n > 1 F(1) = F(0) = 1 Each Fibonacci number in the sequence (except for the first two, which are stipulated to be 1) is the sum of its two predecessors. To see why the language L_F is non-regular, we further explore the properties of regular languages. If L is a regular language over the finite alphabet A = {a}, then there is a DFA M that recognizes L. M has a start state, and a transition on symbol "a" to another (possibly the same) state, and a transition on symbol "a" to another (possibly the same) state, and so on. Because M has only finitely many states, there must eventually be a transition back to a previously visited state, at which point the rest of the behavior of M is completely determined. In pictures, assuming q1 is the start state of M: a a a a a a a q1 --> q2 --> q3 --> .... --> qr --> qr+1 --> ... --> qs ^ \| \| a \| -------------------------- Here we've renumbered the states so that qi is the state reached after (i-1) a's. From the start state, a string of a's visits new states up through (s-1) a's, and then on the next a, goes back to a previously-visited state, namely qr. Then more a's just goes around the loop of states from qr back to qr. If there is no accepting state in the loop, then once a string of a's gets longer than (r-1), it will not be accepted, so in this case, only finitely many strings could be in the language. However, if there is an accepting state in the loop, say qm, then every string of a's of length (m + k(s-r+1)) for k = 0, 1, 2, 3, ... will also be accepted, because the length of the loop is (s-r+1). Neither of these cases is compatible with recognizing L_F, the Fibonacci language, because L_F is infinite (so a DFA that recognizes a finite language is not suitable) and because the distances between successive Fibonacci numbers increase without bound, so no matter how large (s-r+1) is, for sufficiently large Fibonacci numbers it will be smaller than the distance between consecutive Fibonacci numbers. Hence, F_L cannot be regular. Similar types of proofs can be given for other specific concrete languages, for example L = {a^nb^n : n >= 0} that is, the language of all strings of n a's followed by n b's, for every nonnegative integer n. Another example of a nonregular language is the set of all strings of a's and b's that are palindromes, that is, such that the string is equal to its reverse. Please see the notes: Deterministic finite acceptors for a sketch of a proof that the language of palindromes is not regular.
3/31/14	Lecture 27: Strings and languages IV. Summary. Deterministic finite state acceptors (DFAs), alphabet, states, start state, accepting state, transitions, acceptance or rejection of a string by a DFA. Examples of DFAs to recognize the language L(c(a\|d)+r) and the language L((a\|b)b(a\|b)). Theorem (not proved): every language that can be denoted by a regular expression can be recognized by a DFA, and every language that can be recognized by a DFA can be denoted by a regular expression. That is, the two methods of specifying languages specify exactly the same class of languages, the regular languages. Please see the notes: Deterministic finite acceptors. Last lecture we saw the DFA M1, which recognizes the set of all strings of a's and b's with an even number of a's and any number of b's, corresponding to the CORRECTED regular expression b(abab)b What about the regular expression c(a\|d)+r (or, equivalently, the regular expression c(a\|d)(a\|d)r) -- is there a DFA to recognize this language? We constructed one with five states, whose description is as follows. alphabet = {a, c, d, r} states = {q1, q2, q3, q4, q5} start state = q1 accepting states = {q4} transition function symbols states \| a c d r ------------------------ q1 \| q5 q2 q5 q5 q2 \| q3 q5 q3 q5 q3 \| q3 q5 q3 q4 q4 \| q5 q5 q5 q5 q5 \| q5 q5 q5 q5 Note that this DFA, which we'll call M2, accepts the string cadr because starting in state q1, we visit the following states: c a d r q1 q2 q3 q3 q4 and, since we end in q4, an accepting state, the string is accepted. However, for the string card, we get the following states: c a r d q1 q2 q3 q5 q5 and, since we end in q5, a non-accepting state, the string is rejected. A diagram of M2 is available: DFA, c(a\|d)(a\|d)r. In fact, this is a general phenomenon. There is a theorem (which we won't prove) to the effect that (1) every language that can be denoted by a regular expression can also be recognized by a DFA, and vice versa, that is, (2) every language that can be recognized by a DFA can also be denoted by a regular expression. The proof of this theorem consists of two algorithms: (1) an algorithm that takes a regular expression as input and produces as output a DFA to recognize the same language, and (2) an algorithm that takes a DFA as input and produces as output a regular expression that denotes the same language. Thus, these two different methods of specifying sets of strings have the same expressive power: they can specify all and only the regular languages. However, this doesn't necessarily mean that the specifications (as a regular expression or as a DFA) are equally concise. As an example of this phenomenon, we consider the following regular expression. (a\|b)b(a\|b) This regular expression denotes the set of all strings of a's and b's in which the next-to-last symbol is a b. For example, ba, bb, aba, abb, bba, bbb, aaba, aabb, and so on, are strings in L((a\|b)b(a\|b)), while the empty string, a, b, aa, ab, aaa, aab, baa, bab, and so on, are strings that are not in the language of this expression. Given the theorem described above, we know that there is a DFA that recognizes this language, so we set out to find one. As a first attempt, we considered a three-state acceptor defined as follows. alphabet = {a,b} states = {q1, q2, q3} start state = q1 final states = {q3} transitions (q1,a) -> q1 (q1,b) -> q1 (q1,b) -> q2 (q2,a) -> q3 (q2,b) -> q3 However, this is an NFA and NOT a DFA. NFA stands for Nondeterministic Finite State Acceptor, as opposed to DFA, which stands for Deterministic Finite State Acceptor. In the acceptor above, there is a choice of what transition to follow from state q1 on symbol b -- we can either go to state q1 OR go to state q2. In a DFA, we have no choice -- given a state and a symbol there is always exactly one transition defined. (Or at most one transition, if we consider incomplete DFAs.) For an NFA, the definition of whether a string is accepted is whether there exists a way to choose the transitions so as to end in an accepting state. (The notion of nondeterminism is related to the famous P = NP? question, which we'll see later in the course.) Indeed the language of this NFA is the same as the language of the expression (a\|b)b(a\|b), but we still need to keep looking for a DFA* that recognizes this language. The next observation was that if only we could read in the input string backwards we would have an easier time of it. That is, we can consider the language consisting of the reverses of all the strings in L((a\|b)b(a\|b)), which consists of all strings of a's or b's that have b as the second symbol of the string, that is, L((a\|b)b(a\|b)). This reverse language has a DFA with four states, as follows. alphabet = {a,b} states = {q1, q2, q3, q4} start state = q1 accepting states = {q3} transitions (q1, a) -> q2 (q1, b) -> q2 (q2, a) -> q4 (q2, b) -> q3 (q3, a) -> q3 (q3, b) -> q3 (q4, a) -> q4 (q4, b) -> q4 Note that q4 is a dead state, so we could omit q4 and all transitions involving it and get an incomplete DFA for the same language. But, since we might not have the freedom to read the strings in reverse, we must still look for a DFA to recognize the unreversed language, L((a\|b)b(a\|b)). We finally came to the following four state DFA for this language alphabet = {a,b} states = {q1, q2, q3, q4} start state = q1 accepting states = {q3, q4} transitions (q1, a) -> q1 (q1, b) -> q2 (q2, a) -> q3 (q2, b) -> q4 (q3, a) -> q1 (q3, b) -> q2 (q4, a) -> q3 (q4, b) -> q4 Note that this DFA does not accept the empty string (because q1 is not an accepting state), a, aa, b, ab, aaa, aab, baa, bab, but does accept ba, bb, aba, abb, bba, bbb, so this machine seems quite promising. Once the machine has read at least 2 symbols of the input, then the function of each of the states are as follows: q1 -- previous two input symbols were aa q2 -- previous two input symbols were ab q3 -- previous two input symbols were ba q4 -- previous two input symbols were bb This is enough information to decide whether the state should be accepting or not. A more verbose, but also correct, DFA for this language is as follows. alphabet = {a,b} states = {q1, q2, q3, q4, q5, q6, q7} start state = q1 accepting states = {q6, q7} transitions: (q1, a) -> q2 (q1, b) -> q3 (q2, a) -> q4 (q2, b) -> q5 (q3, a) -> q6 (q3, b) -> q7 (q4, a) -> q4 (q4, b) -> q5 (q5, a) -> q6 (q5, b) -> q7 (q6, a) -> q4 (q6, b) -> q5 (q7, a) -> q6 (q7, b) -> q7 In this DFA, if we are in the state q1, then no symbols have been read from the input. If we are in state q2, then just a single symbol a has been read from the input; similarly, in state q3, just a single symbol b has been read from the input. We are in state q4 if at least two symbols have been read, and the last two of them were aa. Similarly, state q5 "remembers" that the last two symbols were ab, q6 that the last two symbols were ba, and q7 that the last two symbols were bb. This DFA also accepts the language L((a\|b)b(a\|b)). Suppose we generalize the language to test a position further from the end of the string, as follows. L2 = L((a\|b)b(a\|b)(a\|b)) L3 = L((a\|b)b(a\|b)(a\|b)(a\|b)) L4 = L((a\|b)*b(a\|b)(a\|b)(a\|b)(a\|b)) ... The regular expression just gets longer by one more (a\|b) at each step, but the corresponding DFA would have to have states to remember the last 3 symbols, or the last 4 symbols, or the last 5 symbols, and so on. Because there are 2^n combinations of a and b for the possibilities for the last n symbols, these DFAs will have an exponentially increasing number of states, 2^3, 2^4, 2^5, and so on. This shows that regular expressions can be exponentially more concise than DFAs for some families of regular languages. Other examples (which we won't be covering) show that DFAs can be exponentially more concise than regular expressions for some other families of regular languages.
3/28/14	Lecture 26: Strings and languages III. Summary. Some egrep (or grep -E) extensions of basic regular expressions: dot, plus, question mark, sets, ranges, and how they can be expressed using basic regular expressions. A language is regular if it can be denoted by a regular expression. Are there non-regular languages? Discussion of whether (a suitable formalization of) English might be a non-regular language, whether the lexicon (set of words) is infinite, whether there are infinitely many different sentences. Digression on the fact that there are different cardinalities of infinity, and sketch of the proof that the real numbers are uncountable. This proof uses the technique of diagonalization, which is also useful in theoretical areas of computer science. Introduction to deterministic finite state acceptors (DFAs), and a DFA to recognize the language of all strings of a's and b's with an even number of a's and any number of b's. Some egrep extensions. In linux, a synonym for egrep is grep -E. Basic regular expressions can be rather verbose -- to make regular expressions more useful, practical implementations introduce some extensions. We'll discuss a few of them, and show that they can be expressed in basic regular expressions, albeit less conveniently. One handy feature is that . can match any single symbol. If we explicitly list all the alphabet symbols (quite a few in ASCII), separated by \|, then we get an expression we could use instead. Another useful feature is Kleene +, which is like Kleene , but requires at least 1 string from the enclosed expression. Thus, the expression c(a\|d)+r denotes the set of all strings that start with c, have one or more* a's or d's and then end in r. Thus, L(c(a\|d)+r) = L(c(a\|d)(a\|d)r), and we can use this same trick to eliminate the Kleene plus from (E)+ by using E(E). Another useful feature is the ? operation, which indicates that we can take 0 or 1 string from the enclosed expression. Thus, L(yog(h)?urt) = {yogurt, yoghurt}. To eliminate the ? from (E)? we can just take ([emptystring]\|E), where [emptystring] stands for the variant epsilon that we chose to name the empty string. Another useful feature is a set, for example [aeiou], which matches any single symbol between the square brackets. To eliminate this, we could write (a\|e\|i\|o\|u). The final feature we describe are ranges, like a-z, A-Z, or 0-9, which can be used in sets, and match any single symbol in the range. Thus, [a-z] matches any single lower case letter. Using these features, we could write an expression [A-Za-z]([A-Za-z0-9])* which matches any string that starts with an upper or lower case letter and is followed by 0 or more upper case letters, lower case letters, or decimal digits. This might be the specification of an identifier in some computer language, and its language contains such strings as Start, x32 and t3mp. Are there languages (ie, sets of strings) that are not regular? One candidate might be a natural language like English. Of course, we have to agree on a "formal" version of English, so that it is a well-defined set of sentences. One approach would be to consider the alphabet to be upper and lower case letters, digits, and punctuation characters, including, say, blanks, periods, commas, colons, semicolons, question marks, exclamation points, parentheses and quotation marks. We imagine there is some "ideal" speaker of English who could examine any finite sequence of such symbols and declare it to be a "correct" English sentence or not. A variant of that approach would be to agree on a lexicon (word list), and take the alphabet to be the set of all the words in the lexicon, so that a sentence would be interpreted as a sequence of words rather than a sequence of characters; we'd still need the "ideal" speaker to determine whether a sequence of words is a "correct" English sentence. There is an issue of whether the lexicon (word list) is a finite set. Compared to some other languages, English has few "automatic" rules to produce new words, so it might be reasonable to think of the lexicon as a fixed finite set. With these stipulations, is "formal" English a non-regular language? If we believe that there are only finitely many "correct" sentences in this language, we could simply list them all, separated by the regular expression or symbol (\|), and have a regular expression for the language. In general, any finite set of strings is a regular language. However, it seems that English contains infinitely many different "correct" sentences, for example the following. I had a bad day. I had a very bad day. I had a very very bad day. I had a very very very bad day. ... This, by itself, doesn't mean that the language is non-regular. In fact, a regular expression for this set of sentences is the following. I had a (very)* bad day. In fact, linguists generally agree that a reasonable definition of "formal" English is not a regular language. A digression on infinities of different sizes. There is a notion of "size" for sets, even infinite ones, that results in different sizes of infinities. The notion of size, or cardinality, is that two sets have the same cardinality if there exists a one-to-one correspondence between them. A one-to-one correspondence is a function that assigns to every element of the first set an element of the second set in such a way that (1) no two elements of the first set are assigned the same element of the second set, and (2) every element of the second set is assigned to some element of the second set. That is, the elements of the first set are paired up with the elements of the second set such that there are no overlaps and no elements left out. This makes sense for finite sets -- if I have two different sets of three elements, I can form three separate pairs consisting of an element of the first set and an element of the second set, using all the elements of both sets. This concept allows an infinite set, for example, the positive integers, to have the same size (cardinality) as a proper subset of itself, for example, the even positive integers. In this case, a one-to-one correspondence between the positive integers and the even positive integers can be the function that takes n to 2n. This matches up the two sets as follows. 1 --> 2 2 --> 4 3 --> 6 4 --> 8 ... It's clear that two different positive integers are matched with two different even positive integers, and that every even positive integer is matched to some positive integer. A set that is either finite or has the same cardinality as the positive integers is said to be countable. However, countable infinities are not the only kind of infinity. To see this, we looked at the sketch of the proof (by diagonalization) of the theorem that the set of real numbers between 0 and 1 is uncountable. That is, this set of real numbers is infinite, but of strictly larger cardinality than the set of positive integers. This proof starts by associating each real number between 0 and 1 with an infinite decimal. For example, the number (pi minus 3) starts out 0.1415962..., and the number 1/3 is 0.3333333.. where the 3's continue indefinitely. This system has a "kink" in it -- some numbers have two different representations. For example, we have the following, where the sequence of 9's on the left is inifinite, and the sequence of 0's on the right is also infinite. 0.499999999.... = 0.50000000... Note that this says these two numbers are equal, not just very close to each other. To be sure of this, we need to think about the definition of what real number is specified by an infinite decimal like this. The actual definition takes the number to be the limit of the sequence of finite decimals, where we extend the fraction by one digit at a time. So the number 0.499999... with an infinite number of 9's is just the limit of the sequence of finite decimals, 0.4, 0.49, 0.499, 0.4999, and so on. It is clear that this sequence of finite decimals gets arbitrarily close to 0.5 without every exceeding it, so the limit must be 0.5. Now that we understand this representation somewhat better, we can proceed to the proof that this set of real numbers is not countable. The proof proceeds by contradiction -- we assume that the set is countable, and then proceed by diagonalization to produce an element of the set that we haven't properly accounted for. Assuming the set is countable, we make a list of all its elements, numbered 1, 2, 3, and so on. Imagine we put these infinite decimals in a table: index real number with that index ----- --------------------------- 1 0.1415962..... 2 0.3333333..... 3 0.2340000..... 4 0.5454545..... . . . . . . Our assumption is that every real number between 0 and 1 eventually gets listed as a row in this table. Now we use "diagonalization" to define a real number z between 0 and 1 that is different from every number in this table. To do this, we specify the decimal digits of z one by one using the following rule: if the i-th digit of the number with index i is not 4, then the i-th digit of z is 4, and if the i-th digit of the number with index i is 4, then the i-th digit of z is 5. Thus, the number z will have only 4's and 5's in its decimal expansion. For the beginning of the table above, we would define z's first decimal digit to be 4 (because the first digit of the first row is 1, not 4), z's second digit to be 4 (because the second digit of the second row is 3, not 4), z's third digit to be 5 (because the third digit of the third row is 4), z's fourth digit to be 5 (because the fourth digit of the fourth row is 4), and so on. Thus, our number z would start out as follows. z = 0.4455..... Note that z is not one of those numbers that has two names, and it cannot appear anywhere in the table, because it has a different digit in the i-th position from the number in the i-th row, for every i = 1,2,3,... Thus, we have a contradiction -- we have defined a real number z between 0 and 1 that is not on the list that we assumed contained all such numbers. The assumption, that the real numbers between 0 and 1 are countable, is therefore false. This concludes the proof that the set of real numbers between 0 and 1 is uncountable. (Note that "uncountable" is not a specific size or cardinality -- it just means a set that is not finite or countable.) End of digression on different infinite cardinalities. Deterministic finite state acceptors. We consider another method of specifying a language, namely, a deterministic finite state acceptor (or DFA). This is a kind of computing machine with similarities to a Turing machine with no tape. A DFA has an alphabet (ie, a finite set of symbols), a finite set of states, one of which is the initial state, a set of accepting states, and a transition function. The transition function takes two inputs, a state and a symbol, and produces one output, a state. As an example, we can consider a DFA M1 with two states, defined as follows. alphabet = {a, b} states = {q1, q2} start state = q1 accepting states = {q1} transition function: symbol state\| a b ------------------ q1 \| q2 q1 q2 \| q1 q2 A DFA can also be represented by a diagram, in which the states are represented by circles (labeled with the state name), a transition is represented by an arrow from one state to another (or to itself) labeled by a symbol, the start state is represented by a "sourceless" arrow into one state, and the set of accepting states is represented by putting an extra circle around each accepting state. A diagram of M1 is available: DFA, even a's. To determine whether a string is accepted or rejected by a finite state machine, we start out in the start state of the machine and follow the transitions indicated by the successive symbols of the string (processed from left to right) until we have processed all the symbols of the string. If the state we end in is accepting, then the string is accepted; otherwise, it is rejected. As examples, M1 accepts the empty string (because we start in q1, follow no transitions, and end in q1, which is an accepting state), and the strings b, aa, bb, aab, aba, baa, bbb, aaaa, aabb, and so on, and rejects the strings a, ab, ba, aaa, abb, bab, bba, aaa, and so on. The language recognized by a DFA M is the set of all strings accepted by M; we denote the language recognized by M by L(M). For M1, L(M1) is the set of all strings of a's and b's that contain an even number of a's (note that 0 is an even number) and any number of b's. This is the same language that is denoted by the CORRECTED regular expression from the previous lecture: b(abab)b*
3/26/14	Lecture 25: Strings and languages II. Summary. Symbol, alphabet, strings, empty string, concatenation of strings, language (a set of strings), concatenation of languages, union of languages, Kleene closure of a language, the empty language, an inductive definition of the syntax and semantics of basic regular expressions. See also: Regular expressions. Strings and languages. Recall that an alphabet is a finite set of symbols and a string is a finite sequence of symbols from an alphabet. The length of a string is the number of occurrences of symbols in it. The empty string, denoted by variant epsilon, is the unique string of length 0. The concatenation of two strings x and y is the string obtained by taking the symbols of x followed by the symbols of y. Concatenation is associative, but not commutative. The concatenation of the empty string and s (in either order) is just s itself. A language is a just a (finite, infinite, or empty) set of strings. So the empty language, containing no strings, is a language. Recall from last lecture the following language over the alphabet {a, c, d, r}. {car, cdr, caar, cadr, cdar, cddr, caaar, caadr, cadar, caddr, ...} It contains an infinite number of strings, namely, every string that starts with a "c", ends with an "r" and has a string of 1 or more "a"'s or "d"'s in between. We can write the following regular expression for this language. c(a\|d)(a\|d)r This expression contains all three of the operations that we use to define "basic regular expressions", namely concatenation (indicated by juxtaposition here), alternation (indicated by the vertical bar), and Kleene star (indicated by the asterisk.) Please see Regular expressions. We proceeded to construct regular expressions to denote two different languages over the alphabet {a, b}. The first language is the set of all strings of even length. Because 0 is an even number, the language contains the empty string and such strings as aa, ab, ba, bb, aaaa, aaab, aaba, aabb, abaa, abab, abba, abbb, baaa, and so on. A basic regular expression denoting this language is the following. ((a\|b)(a\|b)) If we iterate the -loop zero times, we get the empty string. If we iterate it once, then we can choose a or b from the first (a\|b) expression, and (independently) a or b from the second (a\|b) expression, so we can get aa, ab, ba, or bb. If we iterate it twice, we can get all strings of length 4, and so on. The second language over {a,b} that we considered is the set of all strings with an even number of a's and any number of b's. Because 0 is even, some examples of this language are the empty string, b, aa, bb, aab, aba, baa, bbb, aaaa, aabb, and so on. We considered several candidates before settling on one that works. The first candidate was (aa\|b). Examples of strings in its language are: the empty string, b, aa, bb, aab, baa, bbb, and so on. All of the strings in its language have an even number of a's, but there are some strings that have an even number of a's that are not in its language, for example, aba. We then considered (aa\|b\|aba). The language of this expression contains only strings that have an even number of a's, and it contains strings like aba and abaaa, but it does not contain the string abba. The next candidate was (aba), whose language also contains only strings with an even number of a's, including strings like aba, abba, ababbba. However, every non-empty string in its language must start and end with a, so a string like aabb is not in its language. Our final candidate DOESN'T QUITE work: b(aba)b. This expression allows 0 or more b's to start, zero or more repetitions of a pair of a's separated by an arbitrary number of b's, and 0 or more b's to end. UNFORTUNATELY, this fails to be able to generate a string like abababa, although we can generate abaaba. The occurrence of b between the 2nd and 3rd is not accounted for by this regular expression. To CORRECT THIS, we need to allow for b's after the second a in this expression: b(abab)b. This could then be simplified by noting that b's after the last a can be generated by the last b* inside the parentheses, so an equivalent expression would be b(abab).
3/24/14	Lecture 24: Strings and languages I. Summary. Transition to strings and languages, compilers, example of translating a higher-level language program to an assembly-language program, examples of Linux utility egrep. Basic definitions: alphabet, symbols, strings, empty string, concatenation, languages (sets of strings). The machine instructions executed by a computer give us a basis for understanding (1) what a program in a higher-level language (like Java) is translated to in order to be executed by a computer and (2) where our measures of the "time" and "memory" used by a program come from. For example, the wall-clock time used by a program runnning on a particular input is related to the number of machine language instructions that are executed in the course of running that program on that input. We now turn to item (1) -- how does a compiler translate a higher-level language program into an assembly-language (or machine-language) program? As an example, we consider the following portion of a "mini-Java" program. sum = 0; n = 1; while (n < 10) { sum = sum + n; n = n + 1; } System.out.println(sum); This is not a terribly interesting program: it is computing the sum of the numbers from 1 to 9 and printing out the sum. We can translate this program to the following TC-201 assembly-language program: TC-201 assembly language mini-Java program load constant-0 (sum = 0;) store sum load constant-1 (n = 1;) store n while-test load constant-10 (while (n < 10)) sub n skippos jump done body load sum ({sum = sum + n;) add n store sum load n (n = n + 1;}) add constant-1 store n jump while-test done load sum (System.out.println(sum);) output 0 halt 0 (we assume the program is finished.) sum data 0 for the variable sum n data 0 for the variable n constant-0 data 0 for the constant 0 constant-1 data 1 for the constant 1 constant-10 data 10 for the constant 10 Note that an assignment like "n = 1;" is translated to a load and a store. An expression like "sum + n" is translated to a load and an add. The while statement includes code to test the while condition (n < 10) and code corresponding to the while body to execute when the condition is true. (Note that I've replaced the skipzero test that we used in lecture with the slightly more robust skippos test above, which also better reflects the meaning of the test (n < 10).) The task of a compiler is to perform the translation of programs from a higher-level language to corresponding assembly-language programs (or all the way to machine-language programs.) The most common representation of a program is as a (longish) string of characters in a file; the next few lectures will talk about notations and other technologies for dealing with strings and sets of strings. Searching: another use of strings. Suppose I have a score file (such as I email to you giving the scores for your homework) named grade3.txt and I want to see the lines with an occurrence of the string WRONG. I can use the Linux utility egrep as follows. > egrep "WRONG" grade3.txt Your output is WRONG: (conf 'q4 '() 'b '(x y)) > The search found one line of the file containing the string WRONG as a substring, and printed it out. Now suppose I want to see the line(s) with the string Total, to see the total score. > egrep "Total" grade3.txt Total test cases score: 90 / 91 + 9/9 for TM descriptions: Total: 99/100 > This search found two lines in the file containing the string Total and printed them out. If I want to see the scores for each problem, I can search for the lines containing the string Problem as follows. > egrep "Problem" grade3.txt ========= Problem 0 ========= Problem 0 test cases score: 1 / 1 ========= Problem 1 ========= Problem 1 test cases score: 12 / 12 ========= Problem 2 ========= Problem 2 test cases score: 10 / 10 ========= Problem 3 ========= Problem 3 test cases score: 9 / 9 ========= Problem 4 ========= Problem 4 test cases score: 10 / 10 ========= Problem 5 ========= Problem 5 test cases score: 9 / 10 ========= Problem 6 ========= Problem 6 test cases score: 15 / 15 ========= Problem 7 ========= Problem 7 test cases score: 12 / 12 ========= Problem 8 ========= Problem 8 test cases score: 12 / 12 > This is all the lines containing the string Problem, but the ones with all the ===='s do not convey any information. I can specify a pattern that will match just the lines above with scores. A sufficient pattern would be the word Problem followed by a blank followed by any single character followed by a blank followed by the string test. I can specify that as follows. > egrep "Problem . test" grade3.txt Problem 0 test cases score: 1 / 1 Problem 1 test cases score: 12 / 12 Problem 2 test cases score: 10 / 10 Problem 3 test cases score: 9 / 9 Problem 4 test cases score: 10 / 10 Problem 5 test cases score: 9 / 10 Problem 6 test cases score: 15 / 15 Problem 7 test cases score: 12 / 12 Problem 8 test cases score: 12 / 12 > The "." in the pattern does not literally match a period, but will match any single character. This pattern matched all the lines with scores, and did not match the lines with the ===='s. Note that if there were ten or more problems, this pattern would fail to match those with numbers 10 or above. If I want a list of the problem scores and also the total scores, I can specify a somewhat more complex pattern as follows. > egrep "(Problem . test)\|Total" grade3.txt Problem 0 test cases score: 1 / 1 Problem 1 test cases score: 12 / 12 Problem 2 test cases score: 10 / 10 Problem 3 test cases score: 9 / 9 Problem 4 test cases score: 10 / 10 Problem 5 test cases score: 9 / 10 Problem 6 test cases score: 15 / 15 Problem 7 test cases score: 12 / 12 Problem 8 test cases score: 12 / 12 Total test cases score: 90 / 91 + 9/9 for TM descriptions: Total 99/100 This pattern matches any line that contains either the string Total OR a string consisting of Problem, blank, any single character, blank, and then test. The Linux utility egrep is just one tool that uses regular expressions -- they feature prominently in scripting languages like Perl, because one important use of scripting languages is to transform data in one format into another format. Regular expressions are used in Perl to match and extract portions of an input file, so that they may be rearranged and reformatted for an output file. Unfortunately many tools use slightly different conventions for regular expressions, so it pays to be alert for possible differences. For our practical examples, we'll refer to egrep. Many other fields make heavy use of strings, languages and operations on strings, for example, linguistics, in which strings of letters or phonemes are a common representation for words and sentences, and biology, in which strings of nucleotides (represented by the four letters A, C, G, and T) are used to encode DNA sequences, for example, GATTACA, and strings of amino acids (which in nature form an alphabet of 20 elements) to encode proteins. The machinery of the cell translates DNA sequences to RNA sequences (which have a U instead of T), and RNA sequences to amino acid sequences. The latter process is carried out by ribosomes, which can be thought of as finite-state "transducers." (In effect, every cell in a human body teems with finite-state machines.) In biology, one important operation is to find "approximate matches" of DNA strings in huge databases very long DNA strings encoding the genetic information about various organisms. Basics of strings and languages. An alphabet is a finite set of symbols, for example, {a, c, d, r}. A string is a finite sequence of symbols, for example, daccr. The length of a string is the number of occurrences of symbols in it, for example, length(daccr) = 5. There is a unique string of length 0, the empty string, which we will denote by a "variant epsilon" (which looks a bit like a backwards 3.) We can concatenate two strings, an operation indicated by a centered dot. This constructs the string that consists of the symbols of the first string followed by the symbols of the second string. For example, the concatenation of dacc and rda is daccrda. (Note the resemblance to appending two lists.) The concatenation operation is not commutative, for example, the concatenation of rda and dacc is rdadacc, which is not equal to daccrda. The operation of concatenation is associative, which allows us to leave out parentheses, since ((x concatenated with y) concatenated with z) is equal to (x concatenated with (y concatenated with z)). The empty string is an identity for concatenation, because the empty string concatenated with any string is that string itself, and similarly for that string concatenated with the empty string. This means that the set of all strings over an alphabet with the operation of concatenation is a "monoid". (Increment cool word score here.) A language is a set (empty, finite, or infinite) of strings. For example, the set {car, cdr} is a language containing just two strings. The empty set (of no strings) is a (rather trivial) language. The set containing just the empty string is also pretty trivial, but because it has one element, it is not equal to the empty set. The language {car, cdr, caar, cadr, cdar, cddr, caaar, caadr, cadar, caddr, ...} contains an infinite number of strings, namely, every string that starts with a "c", ends with an "r" and has a string of 1 or more "a"'s or "d"'s in between. Next time: regular expressions give a concise notation for this language.
3/7/14	Lecture 23: Computer Architecture IV. Summary: Reading in and storing a zero-terminated sequence of numbers, new TC-201 instructions loadi (load indirect) and storei (store indirect). In the preceding lecture, we considered the problem of reading in a zero-terminated sequence of numbers, printing out the reverse of the sequence, and halting. Here we temporarily simplify the task to reading in a zero-terminated sequence of numbers, storing them in consecutive memory locations, and then halting. (The task of printing out the reverse of the sequence will be deferred to the homework.) In the last lecture, we considered the following, NOT a solution. read-num: input skipzero jump store-num halt store-num: store num jump read-num num: data 0 This is not a solution because each number read in overwrites the previous one, so that at the end only the most recently read number will be available. What we need is a way of storing into num the first time through the loop, storing into the next memory location the next time through the loop, and so on. We considered two possible solutions to this problem. The first, highly deprecated, solution is to write self-modifying code. That is, we may treat the store instruction as data, load it into the accumulator, add 1 to it, and store it back into its memory register. This would have the effect of changing the instruction from "store 6" (in the original program, because num corresponds to address 6) to "store 7", so that the next number read in from the user would be stored in memory location 7. The code for this kind of solution would be as follows. read-num: input skipzero jump store-num halt store-num: store num load store-num add constant-one store store-num jump read-num num: data 0 Once your TC-201 simulator is running, you might want to try this running this program to see what it does. Self-modifying programs are very difficult for human beings to understand, debug, and maintain, because of the large amount of "state" the human has to keep track of -- not only the current data values, but also the current version of the program that is running. So instead we introduce the last two instructions for the TC-201 computer, which allow us to implement "pointers" in our programs. A pointer indicates where in memory an operation is to be performed. The two instructions are as follows. name opcode operation ---- ------ --------- loadi 1011 acc := Mem[extract-address(Mem[addr])] storei 1100 Mem[extract-address(Mem[addr])] := acc The names stand for "load indirect" and "store indirect". The extract-address() operation indicated above means that we extract the rightmost 12 bits of a 16-bit quantity and treat it as the address of a register in RAM. Thus, the instruction "loadi addr" is executed as follows: read from memory the contents of the register with address addr, and take the rightmost 12 bits of that as another address, addr'. Copy the contents of the memory register with address addr' to the accumulator. Finally, increment the program counter, as usual. Similarly, the instruction "storei addr" is executed as follows: read from memory the contents of the register with address addr, and take the rightmost 12 bits of that as another address, addr'. Copy the contents of the accumulator to the memory register with address addr'. The following example may help illuminate the function of loadi and storei. We will simulate the TC-201 for three instructions, starting with the following configuration. acc: 0000 0000 0000 0000 pc: 0000 0000 0000 rf: 1 aeb: 0 address contents of memory register addressed ------- ------------------------------------- 0 1011 0000 0000 0101 1 1100 0000 0000 0110 2 0000 0000 0000 0000 . . 5 0000 0000 0010 1100 6 0000 0000 0010 1111 . . 44 0000 1111 0000 1111 45 1010 1010 1010 1010 46 1111 0000 1111 0000 47 0101 0101 0101 0101 Because the pc contains address 0, we read the instruction at address 0: 1011 0000 0000 0101, decode the opcode 1011 to find that it is a loadi instruction. We take the address field of the instruction, 0000 0000 0101, which is address 5, and read the contents of memory register 5, which is 0000 0000 0010 1100. We take the rightmost 12 bits of that, 0000 0010 1100, and interpret it as address 44. Then the contents of memory register 44 are copied to the accumulator, and the program counter is incremented by 1, to produce the following configuration after the loadi instruction completes. acc: 0000 1111 0000 1111 pc: 0000 0000 0001 rf: 1 aeb: 0 address contents of memory register addressed ------- ------------------------------------- 0 1011 0000 0000 0101 1 1100 0000 0000 0110 2 0000 0000 0000 0000 . . 5 0000 0000 0010 1100 6 0000 0000 0010 1111 . . 44 0000 1111 0000 1111 45 1010 1010 1010 1010 46 1111 0000 1111 0000 47 0101 0101 0101 0101 Now the program counter holds address 1, so we read the instruction at address 1, which is 1100 0000 0000 0110. We decode the opcode, 1100, and find that it is a storei instruction. We take the address field, 0000 0000 0110, which is address 6, and read from address 6, getting the contents 0000 0000 0010 1111. We take the rightmost 12 bits of that as an address, address 47, and copy the contents of the accumulator to memory register 47. The program counter is then incremented, resulting in the following configuration after the storei instruction has been executed. acc: 0000 1111 0000 1111 pc: 0000 0000 0010 rf: 1 aeb: 0 address contents of memory register addressed ------- ------------------------------------- 0 1011 0000 0000 0101 1 1100 0000 0000 0110 2 0000 0000 0000 0000 . . 5 0000 0000 0010 1100 6 0000 0000 0010 1111 . . 44 0000 1111 0000 1111 45 1010 1010 1010 1010 46 1111 0000 1111 0000 47 0000 1111 0000 1111 Note that the contents of registers 5 and 6 are unchanged, but we have copied the contents of register 44 (pointed to by register 5) to register 47 (pointed to by register 6). Next, the instruction at address 2 is executed. Because it is a halt, all that happens is that the run flag (rf) is set to 0, and execution halts. We can make use of the idea of a pointer (which we repeatedly increment) and the storei instruction to solve the problem of reading in a zero-terminated sequence of numbers and storing them in consecutive locations in memory, and then halting. read-num: input skipzero jump store-num halt store-num: storei pointer load pointer add constant-one store pointer jump read-num pointer: data table constant-one: data 1 table: data 0 The memory location pointer initially contains the address corresponding to table. Thus, the first number read in from the user will be stored in the memory register corresponding to table. Then the sequence of instructions adds one to the value of the pointer, so that it now holds the address of the memory register after table. Thus, the next number read in will be stored in the next memory location, and so on, until a zero is input by the user. To see why this solution works in a little more detail, we first consider how the assembler that you will write for the homework will translate the above program into a sequence of 16-bit patterns to store in the RAM starting at address 0. Conceptually, the assembler first needs to determine how to translate all the symbolic addresses in the program into numbers. To do this, it merely numbers the instructions and data statements starting with 0, to determine the addresses that the corresponding instructions or data elements will have. Numbering the instructions and data statements of the above program, we have the following. 0 read-num: input 1 skipzero 2 jump store-num 3 halt 4 store-num: storei pointer 5 load pointer 6 add constant-one 7 store pointer 8 jump read-num 9 pointer: data table 10 constant-one: data 1 11 table: data 0 From this we can extract a "symbol table" allowing us to translate the symbolic labels to numbers: label corresponding address ----- --------------------- read-num 0 store-num 4 pointer 9 constant-one 10 table 11 With this table in hand, we can now translate the sequence of instructions and data statements one by one into the corresponding 16-bit patterns. For an instruction, we look up the 4 bit opcode corresponding to the name of the instruction, and, if the instruction has an address field, we convert the numeric address into a 12-bit pattern to combine with the opcode. If there is no address field, we fill the remaining 12 bits with 0's. For a data statement, we take the number after the "data" field and convert it, using 16-bit sign/magnitude representation, into a 16-bit pattern. For example, the first instruction is "input" (which takes no address field), so the resulting bit pattern is 0101 0000 0000 0000. The second instruction, "skipzero" is translated into 1000 0000 0000 0000, because the skipzero opcode is 1000. The third instruction is "jump store-num". The opcode is 0111, for jump, and looking up store-num in the symbol table, we find it is address 4, which we put in the address field in binary, to get the pattern 0111 0000 0000 0100. For the data statement "data 1" we convert 1 to a 16-bit quantity in sign/magnitude representation to get the pattern 0000 0000 0000 0001. For the data statement "data table", we look up table in the symbol table and find that it is address 11, which we convert to 16 bit sign/magnitude representation, giving the pattern 0000 0000 0000 1011. Putting all this together, the complete translation of the above program would be as follows. addr assembly language machine language ---- ------------------------- -------------------- 0 read-num: input 0101 0000 0000 0000 1 skipzero 1000 0000 0000 0000 2 jump store-num 0111 0000 0000 0100 3 halt 0000 0000 0000 0000 4 store-num: storei pointer 1100 0000 0000 1001 5 load pointer 0001 0000 0000 1001 6 add constant-one 0011 0000 0000 1010 7 store pointer 0010 0000 0000 1001 8 jump read-num 0111 0000 0000 0000 9 pointer: data table 0000 0000 0000 1011 10 constant-one: data 1 0000 0000 0000 0001 11 table: data 0 0000 0000 0000 0000 Note that we have arranged for the table to be the last piece of data in the program, because numbers read from the user will be stored in consecutive memory locations starting with table. We simulated this program for one loop to understand a bit better how it works. We spent a little while discussing how to extend this program to print out in reverse the sequence of numbers read in before halting. The basic idea is to use "loadi pointer" to get the number into the accumulator and "output" to print it. Then the pointer is decreased by 1 and the loop is repeated. To know when to stop, there were two proposals -- one using a sentinel of 0 (which the user cannot cause to be stored in the table) to mark the beginning of the table, and the other to save the original start of the table in a separate variable, so that the pointer could be compared to it. (Note that in the above program, pointer points to the next available location in memory, not to the last stored number.)
3/5/14	Lecture 22: Computer Architecture III. Summary: Computer representations of integers: unsigned binary, sign/magnitude, one's complement, two's complement. Choice of a representation for this term's TC-201: sign/magnitude. Start of task: read in a zero-terminated sequence of numbers, print them out in reverse, and then halt. In the TC-201 we have 16 bits to represent an integer. There are 2^(16) = 65536 possible patterns of 16 bits, so we could represent that many different integers. If we needed only non-negative integers, a natural choice would be "unsigned binary", which would just be the 16-bit binary representations of the numbers 0 through 65,535 as the patterns 0000000000000000 through 1111111111111111, respectively. However, we'd like to represent a range of positive and negative integers, but the number of integers from -n to n is 2n+1, an odd number. As a result, the number representations we'll consider each have an "anomaly" because the number of patterns is even. The three systems we consider are: sign/magnitude, one's complement, and two's complement. For illustration, we'll consider the systems using just 4 bits (instead of 16) -- the principles apply to any number of bits, for example, the more common 32 or 64 bits of modern computers. With 4 bits we have 2^4 = 16 possible bit patterns. Here is what they represent, in unsigned binary and in each of the three systems we consider. bit pattern unsigned binary sign/magnitude one's complement two's complement ----------- --------------- -------------- ---------------- ---------------- 0000 0 0 0 0 0001 1 1 1 1 0010 2 2 2 2 0011 3 3 3 3 0100 4 4 4 4 0101 5 5 5 5 0110 6 6 6 6 0111 7 7 7 7 1000 8 -0* -7 -8* 1001 9 -1 -6 -7 1010 10 -2 -5 -6 1011 11 -3 -4 -5 1100 12 -4 -3 -4 1101 13 -5 -2 -3 1110 14 -6 -1 -2 1111 15 -7 -0* -1 (The asterisk (*) indicates the "anomaly" in each system we consider: a second representation of 0 in sign/magnitude and one's complement, and a number, -8, with no corresponding positive number in the representation, in two's complement.) Note that all three systems represent the numbers 0 through 7 in the same way (except for the extra representations of 0) -- as 0 followed by the 3-bit unsigned binary representation of the integer; this is also the unsigned binary representation of these numbers. (In general, for b bits, the numbers 0 through 2^(b-1)-1 are represented as 0 followed by the (b-1)-bit unsigned binary representation of the number.) The three systems differ in how they represent negative numbers. For sign/magnitude, the representation of a negative number is 1 followed by the 3-bit unsigned binary representation of the absolute value of the number. Thus, the high-order bit is a sign bit (0 for + and 1 for -), and the remaining bits give the magnitude (or absolute value) of the number. To convert a positive to a negative number, it suffices to change the high-order bit from 0 to 1, and to convert a negative to a positive number, it suffices to change the high-order bit from 1 to 0. Note that we have two representations of 0, namely 0000 and 1000, that latter being referred to as -0. For one's complement, the negative of a number is found by complementing each bit individually (changing 0 to 1 and 1 to 0.) Thus, we represent -1 in 4-bit one's complement by complementing each bit of 0001 to get 1110. Of course, since complementation is self-inverse (the complement of the complement is the original argument), the negative of a negative number is also found by complementing each bit. In one's complement (also) the number zero has two representations: 0000 and 1111 -- the latter being referred to as -0. The rationale for two's complement is that it implements arithmetic modulo 2^b, where b is the number of bits. If we consider the integers 0 through 15 modulo 16, then we have 1+15 = 0 (modulo 16) and 5+11 = 0 (modulo 16). Thus, modulo 16, 15 "behaves like" -1, in the sense that 1 and 15 add to 0 (modulo 16), and 11 "behaves like" -5. If we consider the 4-bit unsigned representation of 15, we have 1111, which is the representation of -1 in 4-bit two's complement arithmetic, and the unsigned 4-bit representation of 11 is 1011, which is the representation of -5 in 4-bit two's complement. The number 8 is a bit anomalous modulo 16, because 8+8 = 0 (modulo 16). The representation 1000 is taken to be -8, which means that the high-order bit signifies whether the number is negative (if the bit is 1) or non-negative (if the bit is 0.) Why would two's complement be desirable? If you recall the circuit we designed to add two 4-bit numbers to get a 5-bit sum, if we just ignore the high-order bit of the result, the circuit computes the sum of its two inputs, modulo 16. Two's complement is in fact the most common choice for the representation of integers in modern computers. The rule some have learned for negating a two's complement number is to complement each bit individually and then add 1. For example, to complement 5, we start with 0101 and complement each bit: 1010, and then add 1: 1011, which is the correct representation of -5 in two's complement. Going the other direction, from -5, we start with 1011, complement each bit: 0100, and add 1, to get 0101. Why does this work? Hint: complementing each bit is equivalent to subtracting the unsigned value of the number from 15. In a class vote, it was decided that this term's TC-201 computer would have sign/magnitude representation of integers. We also decided that skipzero will skip on both positive zero (0000 0000 0000 0000) and "negative zero" (1000 0000 0000 000). Next, we turn to the following task: read in a zero-terminated sequence of numbers, print them out in reverse, and then halt. A sample interaction with the user might look like the following. input = 17 input = -13 input = 22 input = 0 output = 22 output = -13 output = 17 We start with the following idea, which is NOT a solution. read-num: input skipzero jump store-num jump produce-output store-num: store num jump read-num num: data 0 The reason this doesn't work is that the first number read in will be stored in num, and then the second number read in will be stored in num (overwriting the first number) and then the third number will be stored in num (overwriting the second number), and so on. When this jumps to the code for produce-output, only the latest number read in will be available in num, and we won't have the information we need to print out the reverse of the sequence of numbers read in. (See the next lecture for how we solve this problem.)
3/3/14	Lecture 21: Computer Architecture II. Summary: Continued specification of the TC-201 computer, instructions: halt, load, store, add, sub, input, output, jump, skipzero, skippos, skiperr. Illustration of the fetch/execute cycle executing a three-instruction program in the TC-201. Machine language, assembly language, and the function of an assembler. Assembly language program (with numeric addresses) to read in a zero-terminated sequence of numbers, print out their sum, and halt. The function of the "data" statement in assembly language. In the TC-201 computer, a configuration is specified by the contents of the RAM (4096 memory registers of 16 bits each, with addresses 0 through 4095), the contents of the accumulator (a 16-bit register), the contents of the program counter (a 12-bit register), the contents of the run flag (a 1-bit register) and the contents of the arithmetic error bit (a 1-bit register). We'll use Mem[addr] to specify the contents of the memory register whose address is addr, acc to specify the accumulator, pc to specify the program counter, rf to specify the contents of the run-flag, and aeb to specify the contents of the arithmetic error bit. The TC-201 has the following instructions. We use x := y to indicate assignment, that is, copying the contents of y to x. The opcode is the leftmost 4 bits of an instruction, and addr is the address contained in the remaining 12 bits of the instruction. name opcode operation -------------------------------------------------------- halt 0000 rf := 0, stops the execution of instructions load 0001 acc := Mem[addr], copies from RAM into acc store 0010 Mem[addr] := acc, copies from acc into RAM add 0011 acc := acc + Mem[addr], adds to acc using RAM sub 0100 acc := acc - Mem[addr], subtracts from acc using RAM input 0101 acc := number read from user output 0110 print out number in acc for user jump 0111 pc := addr, copies address field to program counter skipzero 1000 if (acc == 0) then pc := pc + 2, else pc := pc + 1 skippos 1001 if (acc > 0) then pc := pc + 2, else pc := pc + 1 skiperr 1010 if (aeb == 1) then pc := pc + 2, else pc := pc + 1 Note that for instructions that do not mention the pc, the default is to set pc to pc+1 after the instruction is executed, except for halt, which does not change the value of the pc. The aeb is set by the add and sub instructions when the result of an add or sub operation cannot be represented in the number system of the machine. It is tested by the skiperr instruction, which also sets aeb to 0 after it is tested. All other instructions do not change the value of the aeb. The fetch/execute cycle. As an illustration of how the fetch/execute cycle works, imagine that the contents of RAM and the central registers are as follows. The longer bit patterns are grouped into groups of 4 for human readability. acc: 0000 0000 0000 0000 pc: 0000 0000 0000 rf: 1 aeb: 0 address memory register 0 0001 0000 0001 0110 1 0010 0000 0001 0111 2 0000 0000 0000 0000 . . . 22 1111 0000 1111 0000 23 1010 1010 1010 1010 . . . The fetch/execute cycle repeatedly performs the following operations. If the run flag (rf) is 1, then interpret the contents of the program counter as a memory address, and read the 16 bit contents of that memory register. Interpret the leftmost 4 bits as an opcode, and the rightmost 12 bits as a memory address. Perform the operation indicated by the opcode, updating the contents of the RAM and the central registers as appropriate (and possibly performing input or output to the user), and then repeat. If at the start of this cycle, the run flag (rf) is 0, then no fetching, decoding, or executing takes place. For the particular example above, the first fetch/execute cycle gets the contents of the pc, 0000 0000 0000, and reads from address 0 in the RAM to get the instruction 0001 0000 0001 0110. The leftmost four bits are 0001, which is the opcode for load. The remaining twelve bits are 0000 0001 0110, which are the number 22 in binary. To execute the load instruction, the contents of register 22 are read from the RAM and copied into the accumulator. In addition, the default update adds 1 to the program counter, so the configuration of the machine after the instruction is executed is the following. acc: 1111 0000 1111 0000 pc: 0000 0000 0001 rf: 1 aeb: 0 address memory register 0 0001 0000 0001 0110 1 0010 0000 0001 0111 2 0000 0000 0000 0000 . . . 22 1111 0000 1111 0000 23 1010 1010 1010 1010 . . . Notice that the contents of the acc and the pc have changed, but nothing else has. The fetch/execute cycle now repeats. The run flag (rf) is still 1, so the contents of the pc, 0000 0000 0001, are interpreted as an address, and the next instruction is read from memory register 1. The instruction is 0010 0000 0001 0111. The opcode is 0010, which is store. The rightmost twelve bits are 0000 0001 0111, which is 23 in binary. To execute the store instruction, the contents of the accumulator are copied to memory register 23. The default update to the program counter adds 1 to the pc. The configuration of the machine after the store instruction has been executed is as follows. acc: 1111 0000 1111 0000 pc: 0000 0000 0010 rf: 1 aeb: 0 address memory register 0 0001 0000 0001 0110 1 0010 0000 0001 0111 2 0000 0000 0000 0000 . . . 22 1111 0000 1111 0000 23 1111 0000 1111 0000 . . . Note that the contents of memory register 23 and the pc have changed, but nothing else has changed. In the next fetch/execute cycle, the run flag (rf) is found to be 1, so the next instruction is read from memory, this time from memory register 2 because the program counter contains 0000 0000 0010. The instruction is 0000 0000 0000 0000. Its opcode is 0000, which is halt. The effect of halt is to set the value of the run flag to 0, and nothing else is changed. Thus, once the halt instruction has been executed, the configuration is as follows. acc: 1111 0000 1111 0000 pc: 0000 0000 0010 rf: 0 aeb: 0 address memory register 0 0001 0000 0001 0110 1 0010 0000 0001 0111 2 0000 0000 0000 0000 . . . 22 1111 0000 1111 0000 23 1111 0000 1111 0000 . . . Note that the run flag has changed, but everything else remains the same. The fetch/execute cycle now finds the run flag is 0, and does no further fetching or executing of instructions. This low-level process of fetching and executing instructions is what your TC-201 simulator will do. We refer to this as the machine language level. It is difficult for humans to program in terms of bits, so we consider a somewhat higher level of representation of programs: assembly language. In assembly language, we will use symbolic names for opcodes (that is, "load" instead of "0001") and also symbolic addresses. A program called an assembler (which you'll also write for the TC-201) translates programs written in assembly language into the bits of the machine language level of representation. To get a taste of assembly language programming, we'll write a program to read in a zero-terminated sequence of numbers from the user, print out their sum, and halt. The input and output instructions will interact with the user to request and print numbers, so once our program is done, a sample interaction with the user might look as follows. input = 13 input = 4 input = -2 input = 0 output = 15 Here the program read in numbers 13,4,-2,0 from the user, and, once it read the 0, printed the sum of the numbers, 15. The program we devised in class for this task was as follows. Each instruction is shown with the address of the memory register that contains it, using a symbolic opcode and a decimal number for the address field. If the instruction (eg, halt) does not take an address, no address field is shown. address assembly language instruction comment ------- ----------------------------- ------- 0 load 11 initialize sum to zero 1 store 105 2 input input new number 3 skipzero test whether it is zero 4 jump 8 go add it if nonzero 5 load 105 otherwise: 6 output print the sum 7 halt and halt 8 add 105 add sum to new 9 store 105 store back in sum 10 jump 2 repeat loop 11 data 0 a guaranteed 0 data value Here we chose memory register 105 to hold the running sum of the numbers as we input them from the user. The first two instructions copy 0 into 105 to initialize the running sum to 0. The "data" statement associated with memory register 11 is not an opcode, but instead indicates that one memory register should be reserved and the indicated number (in this case, 0) should be placed into it. This means that memory register 11 will have a 0 in it when the program starts executing. The instructions from address 2 through 10 are a loop which repeatedly prompts the user for a number and reads it into the accumulator. If the number is zero (indicating the end of the sequence of numbers from the user), the instructions at addresses 5, 6, and 7 are executed, which loads the running sum into the accumulator, prints it out for the user, and halts. If the number is nonzero, the instructions at addresses 8, 9, 10 are executed, which adds the running sum to the number in the accumulator, stores the new running sum back into the register at address 105, and jumps back to the instruction at memory register 2 to read the next number from the user.
2/28/14	Lecture 20: Computer Architecture I. Summary: The organization of a simple RAM (Random Access Memory) as 2^A memory registers of B bits each, with an A-bit MAR (Memory Address Register) and a B-bit MDR (Memory Data Register), how memory reads and writes work. Alternate views of the organization of a simple Von Neumann computer in terms of main memory (RAM), CPU (Central Processing Unit) consisting of central registers, the ALU (Arithmetic Logical Unit) and the CU (Control Unit). The fetch-decode-execute cycle. Start of the specification of the TC-201 computer. Perlis epigram #44: Sometimes I think the only universal in the computing field is the fetch-execute cycle. The term "random access memory" conveys that the time to access any particular value in the memory is (essentially) the same, as opposed to a "sequential access memory" like disk or magnetic tape, where accessing "the next" value can be considerably less time-consuming than accessing an arbitrary value (because of the travel time of the physical media, eg, radial movement of a disk head, the spinning of the disk, or the winding of the tape.) We abstract the interface for a random access memory (RAM) as follows. There are two special registers, the memory address register (MAR) and the memory data register (MDR), and a signal indicating whether to read from the memory or write to the memory. The memory itself consists of a number n = 2^A of memory registers, each of which holds B bits. Then the MDR is also B bits (it can hold the value of one memory register) and the MAR is A bits (it can hold the address -- a number between 0 and 2^A-1 -- of any one of the memory registers.) When the operation is a read, the contents of the memory register addressed by the MAR is copied to the MDR. When the operation is a write, the contents of the MDR is copied to the memory register addressed by the MAR, replacing its previous contents. (In the "tiny RAM" example of the previous lecture, there was no MDR, and we had A = 2 address bits, n = 2^A = 4 memory registers, and B = 4 bits in each memory register.) To use this interface to read the contents of a register from RAM, we put the address of the register in the MAR and send the "read" signal. This causes the contents of the addressed memory register to be copied to the MDR, where we may access it. To place a particular value in a particular memory register of the RAM, we put the address of the register in the MAR, and put the value we wish to store in the MDR and send the "write" signal. This causes the contents of the MDR to be copied into the addressed memory register. (The previous contents of that memory register are replaced by the new contents.) These operations are at the "micro-instruction" level, not visible to a person programming the computer at the assembly language level. (There was an aside on how "core memory" works, and an example of 1024 bits of it from the Computer Museum in Boston. The 4 Gigabytes of RAM on a modern laptop were calculated to be equivalent to more than 32 million such arrays of tiny ferrite cores.) We looked at two "block diagrams" of the organization of a Von Neumann computer. Both of them showed main memory, the arithmetic/logical unit (ALU), the control unit (CU), and input/output functions. The ALU and CU are usually grouped together into the "central processing unit" or CPU. The main memory is the RAM that we discussed above. Input/output covers communication of the central computer with devices such as keyboards, mice, displays, microphones, speakers, DVD players, network interfaces, and secondary storage like disks, and so on. The ALU can be thought of as a collection of combinational circuits to perform operations on data like addition, subtraction, comparison, logical operations, and the like. The CU is a collection of registers and combinational logic that collectively are responsible for fetching and executing instructions, (in some sense the conductor of the orchestra that is the computer.) Both diagrams indicated one or more "central registers", that is, registers local to the CPU that are integral to the execution of programs by the computer. The "Von Neumann architecture", in which the instructions of a program and the data for the program are both located in the same (main) memory, is generally contrasted with the "Harvard architecture", in which there are separate memories for the instructions of a program and its data. In the Von Neumann architecture, a program is run as follows. There is a CPU register called the "instruction counter" or "program counter", which holds the address of the next instruction to execute. The instruction is "fetched" (read from main memory into a CPU register called the "instruction register") and "decoded" (which determines what operation is specified by the instruction) and "executed." The execution of one instruction may involve copying data between central registers, copying data from central registers to memory (using the RAM write capability), copying data from memory to central registers (using the RAM read capability), sending data to output devices, receiving data from input devices, or routing data to ALU circuits to perform operations and routing the results to central registers or memory registers. Usually after an instruction is executed, the program counter is increased by 1, and the whole "fetch-execute" cycle is repeated to execute the next instruction in memory. This process continues until a halt instruction is executed. Some instructions (eg, jumps and skips) cause a different update to the program counter, altering the pattern of executing consecutive instructions. We describe the design of the TC-201 computer ("TC" for "tiny computer" or "toy computer"), to make the ideas of computer architecture, machine language and assembly language concrete. It consists of 3 parts: main memory (or RAM), a central processing unit (or CPU), and an input/output system to communicate with the outside world (that is, you, the user.) The main memory, or RAM, of the TC-201 consists of 4096 memory registers, also called "words", of 16 bits each, addressed by integers from 0 to 4095. Because 4096 = 2^(12), 12 bits are sufficient to represent any memory address. There is a central register: the accumulator (or ACC), which has 16 bits. The other CPU state consists of the program counter (or PC), which has 12 bits and holds the address of the next instruction to be executed, the run-flag, which is one bit and indicates whether the computer is running (1) or halted (0), and the arithmetic error bit (or AEB), which indicates whether there has been an arithmetic overflow error. TC-201 instructions. Each TC-201 instruction occupies 16 bits in the RAM. The bits are numbered 0 to 15 from left to right. The top four bits, 0 through 3, are the operation code (or "opcode") and determine which operation will be executed. Thus, there could be 2^4 = 16 possible different opcodes. Most of the instructions involve the contents of the accumulator, and some of them also involve the contents of a word in memory. In that case, the remaining 12 bits, 4 through 15, are the address of the memory register that will be involved in the operation. The TC-201 instructions mentioned in this lecture: opcode operation ------ --------- 0000 halt -- stops execution (sets the run flag to 0) 0001 load -- copies contents of addressed memory register to accumulator 0010 store -- copies contents of accumulator to addressed memory register If we denote the contents of the accumulator by acc, the address field of the instruction by addr, and the contents of the memory register with address addr by Mem[addr], we can abbreviate the function of load as acc := Mem[addr] and the function of store as Mem[addr] := acc, where := is an assignment operator.
2/26/14	Lecture 19: Gates and circuits IV. Summary: review of the NAND latch and its stable states, using 4 of the stable states to store one bit. The design of a D flipflop using a NAND latch. Using four D flipflops and a common set line to implement a 4-bit register. Organizing four 4-bit registers and a 2-bit address register into a tiny Random Access Memory. Please see the notes: NAND latch and D flip flop and Memory. The NAND latch (previous lecture) is the heart of a D flipflop, which we now describe. (There are numerous other types of flipflops for various purposes, but the D flip flop will illustrate the basic ideas.) Consider the diagram of a D flipflop at the bottom of the page: NAND latch and D flipflop. The inputs of the circuit are d (for data) and s (for set), and the output is q. The subcircuit in the box is just the NAND latch, with the second output, u, used only internally. (We could just as well have made it an output of the circuit, but we chose not to.) To understand the behavior of this circuit, first consider the case when s = 0. Because s is an input to the two NAND gates on the left, we know that the (internal) wires x and y will both be 1 (regardless of the value of d.) By our previous analysis of the NAND latch, we know that the wire q will retain its previous value, regardless of the value of d. In effect, when s = 0, the D-flip flop just "remembers" the previous value of q. When s = 1, then what happens depends on the value of d. We know that if one of the inputs to a NAND gate is 1, then the output is the complement of the input (because NAND(1,0) = 1 and NAND(1,1) = 0). Looking at the values of the internal wires x and y, we see that when s = 1, x = d' and y = d'' = d. Thus, when d = 1, we have x = 0 and y = 1, so q = 1 by our previous analysis of the NAND latch. And when d = 0, we have x = 1 and y = 0, so q = 0 by our previous analysis of the NAND latch. In effect, when s = 1, q is set to the value of d. This behavior is used to store one bit of information. Normally we keep s = 0, and the circuit stores the previous value of q (either 0 or 1), which is continuously available on the output wire q. At the time in the computation when the wire d has the value that we want to store, we set s = 1 for a short time. This causes the NAND latch to set q to the value on the wire d, and then s is set back to 0, which causes this value to be the new stored value. We can arrange several one-bit memories in parallel to form a "register" -- a circuit that can store several bits of data, with a common set line. For example, see the 4-bit register constructed out of 4 D-flip flops in the notes: Memory. The idea of the register is that when its set line = 1, each one-bit memory copies the value on its individual input line to be its new stored value, and when its set line = 0, the values on each output line stably remain what they were, regardless of the values on the input lines in the meantime. We consider a set of four 4-bit registers and a single 2-bit register, organized into a tiny "random access memory" (or RAM). We consider each of the four registers to have an address consisting of two bits, so that the addresses are 00, 01, 10, 11. The 2-bit register (which is the "memory address register" (or MAR)), can hold one of these addresses; we denote its two output wires as a1 (for the high-order bit) and a0 (for the low-order bit). "Reading" from the RAM consists of arranging for the contents of the memory register whose address is in the MAR to appear on a selected set of output wires. We can use three 4-bit selectors to accomplish this, as follows. One selector has the 4 outputs of register 00 as its left inputs and the 4 outputs of register 01 as its right inputs, and a0 (the low-order bit of the address) as its selector input. Thus, its output wires have the contents of register 00 if a0 = 0 and the contents of register 01 if a0 = 1. The second selector has the 4 outputs of register 10 as its left inputs and the 4 outputs of register 11 as its right inputs, and a0 (again, the low-order bit of the address) as its selector input. Thus, its output wires have the contents of register 10 if a0 = 0 and the contents of register 11 if a0 =1. The third selector has the outputs of the first selector as its left inputs, and the outputs of the second selector as its right inputs, and a1 (the high-order bit of the address) as its selector input. The output wires of the third selector will have the desired values, namely, the contents of the memory register addressed by the two bits of the MAR. A diagram: Memory read with selectors. Consider now what happens if the MAR contains the address 10. Because a0 = 0, the output wires of the first selector will have the value of register 00, and the output wires of the second selector will have the value of register 10. Because a1 = 1, the output wires of the third selector will have the value of register 10, as desired. If we generalize this construction to a RAM with 2^a memory registers and an a-bit MAR, the selectors using each bit of the address operate in parallel, so the time for the contents of the addressed register to appear on the output wires of the final selector is linearly proportional to the quantity a (the number of address bits). An alternative view of the memory read circuit, without the abstraction of the 4-bit selectors: Memory read. While we didn't discuss how to implement writing to memory, a picture of the Fall 2012 class design for a write circuit may be found in Memory write.
2/24/14	Lecture 18: Gates and circuits III. Summary: Using a selector to choose one of two inputs to be an output. Sequential circuits: a NOT gate whose input is its output wire can function as a clock, an OR gate one of whose inputs is its output wire and its analysis, the NAND latch and the analysis of its stable states. Please also see the diagrams: NAND latch and D flipflop. Previously we saw the design of a circuit to add two 4-bit numbers to produce a 5-bit sum. We could also design a circuit to subtract two 4-bit numbers to produce a 5-bit difference. (Exactly what that means will become clearer when we talk about number systems for computers.) Suppose we want to choose between the sum and difference of two 4-bit numbers based on a Boolean value s. That is, we want a circuit with inputs x3, x2, x1, x0 (representing a binary number) and y3, y2, y1, y0 (representing a binary number) and s (to choose between sum and difference), and outputs z4, z3, z2, z1, z0, where if s = 1, then z4, z3, z2, z1, z0 is the sum of the two input numbers, and if s = 0, then z4, z3, z2, z1, z0 is the difference of the two input numbers. In a programming language this is just an if statement, for example, in Racket we would have something like (if (s = 1) (add x y) (subtract x y)). In hardware, we have a different solution: compute both the sum and difference, in separate circuits, in parallel, and then use s to select which values will appear on the output wires z4, z3, z2, z1, z0. We can think about this one bit at a time. A 1-bit selector circuit has inputs s, x, and y, and one output z, where z is equal to x if s = 0 and z is equal to y if s = 1. If we construct a truth table for this Boolean function, we get the following. s x y \| sel(s,x,y) ---------------------- 0 0 0 \| 0 0 0 1 \| 0 0 1 0 \| 1 0 1 1 \| 1 1 0 0 \| 0 1 0 1 \| 1 1 1 0 \| 0 1 1 1 \| 1 Note that in the top half of the table (where s = 0), the output just copies x, and in the bottom half of the table (where s = 1), the output just copies y. We could write out an expression with four terms using the sum of products algorithm, but there is a simpler equivalent expression, namely z = (s' * x) + (s * y) This can be verified by construting the truth table for this expression, or by noting that when s = 0, s' = 1, so we have z = (1 * x) + (0 * y), which simplifies to z = x, and when s = 1, s' = 0, so we have z = (0 * x) + (1 * y), which simplifies to z = y. A circuit based on this expression takes time 3 gate delays. This one-bit selector can be generalized to any number of bits. That is, if we have a selector input s and two n-bit inputs a1, a2, ..., an, and b1, b2, ..., bn, and an n-bit output c1, c2, ..., cn, we can arrange that if s = 0 then c1 = a1, c2 = a2, ..., cn = an and if s = 1 then c1 = b1, c2 = b2, ..., cn = an, by using n one-bit selectors with the same s input. The first one-bit selector has a1 and b1 as its other inputs, and c1 as its output. The second one-bit selector has a2 and b2 as its other inputs, and c2 as its output, and so on. Each of these one-bit selectors operates in parallel with the others, so after just 3 gate delays the n-bit output c1, c2, ..., cn is available. Thus, with a 5-bit selector we can solve our original problem of selecting the sum or difference of two 4-bit inputs. Sequential circuits. Up to now we have considered only combinational circuits, which have no loops of wires and gates. In a combinational circuit, the eventual final outputs of the circuit are completely determined by the values of the circuit inputs. However, the outputs of a sequential circuit may depend on both the inputs and the past values of the wires of the circuit. We'll consider several examples of sequential circuits. The first is a single NOT gate whose input is its own output, indicated in the following diagram. ---\|NOT>-------- z \| \| ------------ Here the wire z is both the output of the NOT gate and its input. We examine the behavior of this circuit assuming that z is initally 0. This is the input to the NOT gate, which one gate delay later outputs z = 1. After one further gate delay later, the output of the NOT gate is z = 0. Thus, the value of z alternates between 0 and 1 at intervals of one gate delay, indefinitely. Graphing the value of z as a function of time (measured in gate delays): 1 * 0 ** 0 1 2 3 4 5 6 7 The output value of this circuit never stabilizes, but nonetheless, it could be useful as a "clock" marking off time in units of one gate delay. Another sequential circuit ("Garden of Eden"). We now consider a sequential circuit consisting of an OR gate whose output is fed back as one of the inputs to the OR gate. A picture of this situation follows. x ----\| \|OR>--------- z ---\| \| \| \| ------------ This is an OR gate with inputs x and z and output z. Some assignments of values to the wires x and z are "stable" in the sense that they are not different after one gate delay. To find these values, we consider the equation: z = x OR z This has three solutions: x = 0 and z = 0 x = 1 and z = 1 x = 0 and z = 1 The combination x = 1 and z = 0 is not stable, because after one gate delay we have x = 1 and z = 1. If we consider the three solutions as states of the circuit, and see how changes in the input x can transition from one state to another, we have the following diagram. (x = 0, z = 0) -------> (x = 1, z = 1) -------> (x = 0, z = 1) x = 1 ^ x = 0 \| \| \| ------------------------- x = 1 We see that the state x = 0 and z = 0 is a so-called "Garden of Eden" state, because once we leave it (by setting the input x to 1), we cannot re-enter it. From the state x = 1 and z = 1 we can reach the state x = 0 and z = 1 by setting the input x to 0, and from the state x = 0 and z = 1 we can reach the state x = 1 and z = 1 by setting x to 1, but the fact that z = 1 in both states means that z will continue to be 1. This circuit exhibits a kind of "memory", recording whether the input x has ever been set to 1. The NAND latch. Please also see the diagrams: NAND latch and D flip flop. The third kind of sequential circuit that we consider provides a more useful kind of memory. It consists of two NAND gates with the output of each one fed back to be an input of the other. A diagram follows. x ----\| \|NAND>--------- q ---\| / \| / ----------/--- / \| --------/ \| \| \| ---\| \| \|NAND>----*----- u y ----\| The inputs to the circuit are x and y, and the outputs are q and u. The output q is an input with y to a NAND gate whose output is u, and the output u is an input with x to a NAND gate whose output is q. The stable states of this circuit are those assignments of 0 and 1 to the wires x, y, q, and u that satisfy the following equations. q = x NAND u u = y NAND q This set of equations has 5 solutions, as follows. (1) x = 0, y = 0, q = 1, u = 1 (2) x = 1, y = 0, q = 0, u = 1 (3) x = 0, y = 1, q = 1, u = 0 (4) x = 1, y = 1, q = 1, u = 0 (5) x = 1, y = 1, q = 0, u = 1 Note that the values of the output wires q and u are not determined by the values of the input wires (see (4) and (5)). If we avoid state (1), we can use the other four states to "remember" one bit of information. Consider how we can move from one of these four states to another by changing the value on one input wire, either x or y. (y=0) <-------- (x=1,y=0,q=0,u=1) ---------> (x=1,y=1,q=0,u=1) ^ (y=1) / \ / \ / \ / \ / \ / X / \ (x=0) / \ (y=0) / \ / \ / \ / \ V (x=0) \ <-------- (x=0,y=1,q=1,u=0) ---------> (x=1,y=1,q=1,u=0) (x=1) In the states on the left, the inputs (x=1,y=0) or (x=0,y=1) determine the outputs. Then when y (or x) is set to 1, there is a transition to the corresponding state on the top, where the inputs are x=1 and y=1, but the state on top indicates that the preceding left state was (x=1,y=0) while the state on the bottom indicates that the preceding left state was (x=0,y=1). The NAND latch is the heart of a D-flip flop, which we'll see next lecture.
2/21/14	Lecture 17: Gates and circuits II. Summary: A combinational circuit to compare two 4-bit numbers -- output 1 if they are equal and 0 if they are unequal. Comparing the time taken by different organizations of the tree of AND gates combining the results for individual bits. A combinational circuit to add two 4-bit binary numbers, getting a 5-bit result: a half-adder, a full-adder, and combining them into a ripple-carry adder. The time taken by the ripple-carry adder, and the distinction between linear time (BAD) and logarithmic time (GOOD) time for circuits. Please see the notes: Equality circuit and Addition circuit. In this lecture we consider two combinational circuits, one to compare two 4-bit numbers for equality, and one to add two 4-bit numbers to get a 5-bit sum. In a computer, combinational circuits perform basic operations like adding, subtracting, multiplying or dividing numbers, comparing numbers and other quantities, performing shifts and logical operations, and the like. The other important type of circuit, sequential circuits, are used to implement memory, storing values before and after they are operated on. The next lecture will consider sequential circuits. To construct a circuit to compare two 4-bit quantities, x1, x2, x3, x4 and y1, y2, y3, y4 for equality, we first consider a circuit to compare two single bits for equality. The input is two bits, x and y, and the output is one bit z, where z = 1 if and only if x = y. A truth table for this Boolean function is as follows. x y \| z ------------ 0 0 \| 1 0 1 \| 0 1 0 \| 0 1 1 \| 1 We could use the sum-of-products algorithm to write an expression for z, namely, z = x'y' + xy. We could also observe that the function in this table is just the negation of the XOR of x and y, because x y \| (x XOR y) (x XOR y)' ------------------------------- 0 0 \| 0 1 0 1 \| 1 0 1 0 \| 1 0 1 1 \| 0 1 Thus, we can also write z = (x XOR y)'. These two expressions lead to two different circuits for the given Boolean function. Using the expression z = x'y' + xy, we get a circuit with two NOT gates, two AND gates and an OR gate, with a gate delay of 3 in the longest path. Using the expression z = (x XOR y)', we get a circuit with one XOR gate and one NOT gate, and a gate delay of 2 in the longest path. Of course, the second circuit assumes that gates computing the XOR operation are available. Once we have a circuit available for computing whether two bits are equal, we draw a rectangle around it, and give it a label, say EQ, and use it as a "subcircuit" in our further circuit design. This kind of abstraction, like writing auxiliary procedures in Racket, helps to keep us from being overwhelmed in details as our designs get more complex. Given the design of EQ, we use four copies of it and three AND gates to compare two 4-bit numbers, a number x represented by inputs x1, x2, x3, and x4, and a number y represented by inputs y1, y2, y3, y4, and produces an output z, where z = 1 if and only if x1 = y1 and x2 = y2 and x3 = y3 and x4 = y4, that is, the two 4-bit numbers x and y are equal. Please see the notes: Equality circuit. A circuit for 4-bit binary addition. Please see the notes: Addition circuit.
2/19/14	Lecture 16: Gates and circuits I. Summary: Introducing logic gates (for AND, OR, NOT, XOR, NAND, NOR), wires, and circuits. Feedforward or combinational versus sequential ("loopy") circuits. Computing the time (gate delays) of a combinational circuit; the role of parallelism in circuits. Another useful gate: XOR. Boolean bases: gates of types {AND, OR, NOT} are sufficient to compute any Boolean function, by the sum of products representation. However, each of the following sets is also a complete basis for the Boolean functions: {AND, NOT}, {OR, NOT}, {NAND}, {NOR}. We left open the question of whether {AND, OR} or {XOR, NOT} is a complete Boolean basis. Please also see the notes: Gates and circuits and Boolean bases. We looked at a circuit for the key/door/seatbelt alarm signal, derived from the Boolean expression a = k * (d' + b'). Recall that this computes an alarm signal (a) as a function of sensor values for the key in the ignition (k), the door closed (d), and the seatbelt fastened (b). Drawings of the symbols for six types of gates and for the alarm circuit are in: Gates and circuits. In the alarm circuit, there are 4 gates (two NOT gates, one AND gate, and one OR gate.) We model the time for this circuit to compute its output as follows. Each gate takes some small but non-zero time to get the correct answer on its output wire after its inputs are changed, its delay. In reality, different gates will have different delays, but for our purposes we assume that each gate takes the same time, a unit we call 1 gate delay. If at a given time t the inputs k, d, b are correct and stable, then after 1 gate delay, the outputs of the two NOT gates will be correct and stable, (namely, d' and b'.) Note that the two NOT gates operate in parallel -- neither has to wait for the other's output. After another 1 gate delay, the output of the OR gate is correct and stable (namely, (d' + b')). Note that we cannot assume that the inputs to the OR gate are correct and stable until the NOT gates have produced correct outputs. Finally, after 1 more gate delay, the output of the AND gate is correct and stable (namely, k(d' + b').) Thus, the delay for the whole circuit is 1+1+1 = 3 gate delays. This is also the number of gates in the longest path of wires and gates from an input wire to an output wire, the depth* of the circuit. Because any Boolean function can be expressed using AND, OR and NOT, we know we can build a circuit to compute any Boolean function using just AND, OR and NOT gates. However, we note that (x + y) = (x' * y')' that is, OR can be expressed using just AND and NOT. (This equivalence can be checked using a truth table, or by applying one of DeMorgan's laws and the law of double negation: (x' * y')' = (x')' + (y')' = x + y Thus, every where we have an OR gate, we can substitute a small circuit of 3 NOT gates and an AND gate. That is, we can build a circuit for any Boolean function using just AND and NOT gates; the set of gate types {AND, NOT} is a complete Boolean basis. By duality, we could replace every AND gate using 3 NOT gates and an OR gate using the dual equivalence: (x * y) = (x' + y')' Thus, the set of gate types {OR, NOT} is a complete Boolean basis. In fact, if we consider the NAND gate, which computes the NOT-AND function (x NAND y) = (x * y)' then we can build a circuit for any Boolean function just using NAND gates. The truth table for NAND is as follows. x y \| (x NAND y) -------------------- 0 0 \| 1 0 1 \| 1 1 0 \| 1 1 1 \| 0 Because (x NAND x) = x', we can use a single NAND gate both of whose inputs are x as a replacement for a NOT gate. And because (x NAND y)' = ((x * y)')' = (x * y), we can use a pair of NAND gates to replace each AND gate in a circuit. Because we previously showed that {AND, NOT} is a complete Boolean basis, we conclude that {NAND} is a complete Boolean basis. Dually, we consider the NOR function: (x NOR y) = (x + y)' and can similarly show that {NOR} is a complete Boolean basis. (As a footnote: The Wikipedia article on logical NOR claims that "The computer used in the spacecraft that first carried humans to the moon, the Apollo Guidance Computer, was constructed entirely using NOR gates with three inputs.") We left open the questions of whether {AND, OR} or {XOR, NOT} might be a complete Boolean basis.
2/17/14	Lecture 15: Review for Midterm 1. Summary: solutions of practice exam 1, questions related to the midterm.
2/14/14	Lecture 14: Boolean functions and expressions III. Summary: A Boolean expression for alarm signal as a function of sensors for key in ignition, door closed, seat belt fastened. Expressing XOR, implication, and equivalence in terms of AND, OR, NOT. The equivalence of Boolean expressions; a sound and complete set of axioms for the equivalence of Boolean expressions. See also the list: Boolean axioms. See the notes Sum-of-products algorithm for the truth table for the alarm signal (a) as a function of sensors for key in ignition (k), door closed (d) and seat belt fastened (b), and two Boolean expressions for a in terms of k, d, and b, one derived informally and one derived using the sum-of-products algorithm. Three useful Boolean functions of 2 variables are XOR, implication (->), and equivalence (<->), whose truth tables are as follows. x y \| (x XOR y) x y \| (x -> y) x y \| (x <-> y) ------------------ ----------------- ------------------ 0 0 \| 0 0 0 \| 1 0 0 \| 1 0 1 \| 1 0 1 \| 1 0 1 \| 0 1 0 \| 1 1 0 \| 0 1 0 \| 0 1 1 \| 0 1 1 \| 1 1 1 \| 1 Each of these can be expressed in terms of AND, OR and NOT, using the sum of products algorithm as follows. (x XOR y) = x'y + xy' (x -> y) = x'y' + x'y + xy (x <-> y) = x'y' + xy Note that equivalence is true when x and y are equal, and XOR is true when x and y are unequal, so that each is the negation of the other. Another expression for (x -> y) is (x' + y), as verified by the following truth table. x y \| (x' + y) ----------------- 0 0 \| 1 0 1 \| 1 1 0 \| 0 1 1 \| 1 Equivalence of Boolean expressions. We say that two Boolean expressions E1 and E2 are equivalent, which we'll denote by E1 = E2 in these notes (because we don't have a three-bar "equivalent" sign available), if for every environment env that assigns values to all the variables in either E1 or E2, we have the value of E1 in env is the same as the value of E2 in env. In terms of truth tables, if we make a combined truth table with all the variables in either expression, the column of values for E1 is the same as the column of values for E2. A set of axioms for the equivalence of Boolean expressions is available here: Boolean axioms. Each axiom expresses an equivalence between two Boolean expressions. For example, axiom (A10) is the following. a + (b c) = (a + b) * (a + c) This is the property that in Boolean algebra, OR distributes over AND. To verify the equivalence of the two sides, we consider the following truth table. a b c \| a + (b * c) (a + b) * (a + c) -------------------------------------------- 0 0 0 \| 0 0 0 0 1 \| 0 0 0 1 0 \| 0 0 0 1 1 \| 1 1 1 0 0 \| 1 1 1 0 1 \| 1 1 1 1 0 \| 1 1 1 1 1 \| 1 1 Because the columns giving the values of the two expressions are equal, the expressions are equivalent. These axioms have the property that consistently substituting an expression for each of the variables on both sides of the axiom gives another equivalent pair of expressions. So, if we take axiom (A7), which is a * (a + b) = a and substitute (x' + y) for a and z' for b on both sides, we get the equivalence (x' + y) * ((x' + y) + z') = (x' * y) Moreover, if we have an equivalence E1 = E2 and another expression E3 that contains one or more sub-expressions equal to E1, then we can replace one or more of the sub-expressions E1 in E3 by E2 and get a new expression equivalent to E3. Thus, from axiom (A13) we have the equivalence 1 * a = a and as a substitution instance of (A16) we have x + x' = 1 and we may substitute (x + x') for 1 in the previous equivalence to get (x + x') * a = a These axioms and rules of inference give a system that is "sound" (every equivalence derivable using the rules is in fact true) and "complete" (every true equivalence between Boolean expressions is in fact derivable.) The axioms are in fact redundant; as an example, we'll derive (A8) from the other axioms using the rules of inference. The axiom (A8) is one of the "absorption laws", namely, a + (a * b) = a One way to derive it is to start with a + (a * b) and use (A10) to get a + (a * b) = (a + a) * (a + b) Using (A2), (a + a) = a, so we have a + (a * b) = a * (a + b) Then using (A7) (the other "absorption law"), a * (a + b) = a, so a + (a * b) = a Here are three very useful equivalences that are not among the axioms. (a + b)' = a' * b' (a * b)' = a' + b' (a')' = a The first two are called DeMorgan's laws, after the logician Augustus DeMorgan (who tutored Ada Lovelace in mathematics, after whom the Ada programming language was named.) The third is the law of double negation. These equivalences are easily verified using truth tables, but are somewhat challenging to derive using the given axioms and inference rules. Note that the axioms on the list come in "dual" pairs. That is, if we replace + by , replace by +, replace 0 by 1 and replace 1 by 0, we will find a "dual" for every axiom on the list. For example, the dual of (A11), a + 0 = 0 is (A14), a * 1 = 1. This means that if we take any equivalence E1 = E2 and apply the dual operation to the two sides to get D1 and D2, then the equivalence D1 = D2 is also true. We considered the question: is there an algorithm that takes as input a Boolean expression E1 and outputs a Boolean expression E2 equivalent to E1 such that E2 has the smallest "size" among all Boolean expressions equivalent to E1? (Where we define "size" as the number of symbols to write down the expression, for example.)
2/12/14	Lecture 13: Boolean functions and expressions II. Summary: Recursive procedure to find the value of a Boolean expression in an environment and its relation to the truth table for the expression, expressiveness of Boolean expressions, the sum-of-products algorithm, the fact that AND, OR, and NOT are a complete basis for the Boolean functions. See also: Sum-of-products algorithm. Inductive/recursive definition of the value of a Boolean expression in an environment. Recall that an environment is an assignment of Boolean values (0 or 1) to the variables. For example, for the expression (x' + y)' we might consider the environment x = 0 and y = 1. In the previous lecture we saw a construction of the truth table for the expression (x' + y)' as follows. x y \| x' (x' + y) (x' + y)' --------------------------------- 0 0 \| 1 1 0 0 1 \| 1 1 0 1 0 \| 0 0 1 1 1 \| 0 1 0 The environment x = 0 and y = 1 corresponds to the second line of this truth table, where x and y have these values. The other columns of the second line show the values taken on by the expressions x', (x' + y), and (x' + y)' in this environment. A recursive definition of the value of a Boolean expression in an environment follows the form of the recursive definition of the syntax of a Boolean expression, as follows. If E is a Boolean expression and env is an environment, then the value of E in env is defined as follows: Base cases If E = 0, then the value of E is 0 in any environment If E = 1, then the value of E is 1 in any environment If E = x (a variable), then the value of E is the value that env assigns to x Recursive cases If E = (E1)', then the value of E in env is the NOT of the value of E1 in env If E = (E1 * E2), then the value of E in env is the AND of the value of E1 in env and the value of E2 in env If E = (E1 + E2), then the value of E in env is the OR of the value of E1 in env and the value of E2 in env We can apply this definition to the expression E = (x' + y)' and the environment x = 0 and y = 1, and we get a tree of recursive applications of the definition as follows. value of (x' + y)' (=> 0) / \| NOT value of (x' + y) (=> 1) / \| \ OR value of x' value of y (=> 1) (=> 1) / \| NOT value of x ( => 0) Note that the leaves are the base cases x and y, whose values are determined by the environment (x = 0 and y = 1), and as the values are returned back up the tree, the values for the sub-expressions x' and (x' + y) are also found, corresponding to the "auxiliary" columns in the truth table. You'll be asked to implement this definition as a recursive procedure in the next homework. How expressive are Boolean expressions? What class of Boolean functions can be expressed by Boolean expression? The answer is: EVERY Boolean function can be expressed by a Boolean expression. We'll show this by means of the "sum of products" algorithm that can take an arbitrary truth table and produce a Boolean expression for that truth table. The sum-of-products algorithm. We start with a truth table for an arbitrary Boolean function, for example the following one, where we've given the 3 arguments the names x, y and z. (This is not so arbitrary -- we saw that it is the "majority vote" function, but the same procedure works on an arbitrary function.) x y z \| f(x,y,z) ----------------- 0 0 0 \| 0 0 0 1 \| 0 0 1 0 \| 0 0 1 1 \| 1 1 0 0 \| 0 1 0 1 \| 1 1 1 0 \| 1 1 1 1 \| 1 Can we write down a Boolean expression with this truth table? The answer is yes, and one method is the sum-of-products algorithm. If we look at the first 1 in the table above, we see that it corresponds to the environment where x = 0, y = 1, and z = 1. The expression (x' * y * z) has the property that it is 1 in this environment, and 0 in all the other environments in the table. That is, it has the truth table: x y z \| (x' * y * z) --------------------------- 0 0 0 \| 0 0 0 1 \| 0 0 1 0 \| 0 0 1 1 \| 1 1 0 0 \| 0 1 0 1 \| 0 1 1 0 \| 0 1 1 1 \| 0 Note, for example that when x = 1, y = 0, and z = 1, x' = 0, so the AND of x', y, and z is 0. For each of the other 1's in the truth table of f we can find an expression that is 1 on that line of the truth table and 0 elsewhere. For example, for the environment x = 1, y = 0, z = 1, we get such an expression by conjoining x, y', and z. (Note that we take the variable if its value is 1, and the NOT of the variable if its value is 0.) x y z \| (x * y' * z) --------------------------- 0 0 0 \| 0 0 0 1 \| 0 0 1 0 \| 0 0 1 1 \| 0 1 0 0 \| 0 1 0 1 \| 1 1 1 0 \| 0 1 1 1 \| 0 Similarly, for the other two 1's in the table, we get the expressions (x * y * z') and (x * y * z). Putting all four of these expressions in a truth table, together with the expression gotten by OR'ing all of them together, we get: x y z \| (x'yz) (xy'z) (xyz') (xyz) (x'yz)+(xy'z)+(xyz')+(xyz) ------------------------------------------------------------------------------------------ 0 0 0 \| 0 0 0 0 0 0 0 1 \| 0 0 0 0 0 0 1 0 \| 0 0 0 0 0 0 1 1 \| 1 0 0 0 1 1 0 0 \| 0 0 0 0 0 1 0 1 \| 0 1 0 0 1 1 1 0 \| 0 0 1 0 1 1 1 1 \| 0 0 0 1 1 Note that the final expression, (x'yz) + (xy'z) + (xyz') + (xyz), has the truth table of f as specified above. The sum of products algorithm takes any truth table for a Boolean function and returns a Boolean expression with that truth table. We may re-state the algorithm in general terms as follows. The Sum of Products Algorithm ------------------------------ Input: a truth table Output: a Boolean expression with that truth table If all of the rows of the truth table have value 0, output the Boolean expression: 0. Otherwise, for each row with the value 1, construct a term (product) by and'ing together all the variables that are 1 in that row and the negations of all the variables that are 0 in that row. Output the Boolean expression gotten by or'ing together all the terms (products) constructed for rows with value 1. The term "sum of products" refers to the fact that the main connectors are ORs ("sums") and the minor connectors are ANDs ("products"). (There is a dual "product-of-sums" algorithm -- it might be interesting to derive it.) Thus, all Boolean functions may be expressed by Boolean expressions. This also shows that the operations of NOT, AND, and OR are a "complete basis" for the Boolean functions -- every Boolean function can be represented by an expression using only NOT, AND, and OR. The sum of products algorithm in general does not return the "most concise" possible expression for a given truth table. Consider the truth table for the expression (xy + xz + yz), as follows. x y z \| (xy + xz + yz) ------------------------------ 0 0 0 \| 0 0 0 1 \| 0 0 1 0 \| 0 0 1 1 \| 1 1 0 0 \| 0 1 0 1 \| 1 1 1 0 \| 1 1 1 1 \| 1 This expression also has the desired truth table for f(x,y,z), and is much more concise than the expression that the sum of products algorithm found. In general, we may define a notion of (semantic) equivalence of two Boolean expressions as follows. Two Boolean expressions E1 and E2 are equivalent if for every environment env that assigns values to all of the variables that occur in either E1 or E2, the value of E1 in env is equal to the value of E2 in env. Note that we need to consider environments that give values to all of the variables that occur in either expression. (The notation for the equivalence of two Boolean expressions is an "equal sign" with 3 horizontal bars, which we'll somewhat inaccurately render as "=" in these notes.) Thus, we've seen the equivalence of the following two Boolean expressions. (xyz' + xy'z + x'yz + xyz) = (xy + xz + y+z) Question: is there an algorithm that inputs a Boolean expression E1 and outputs a "most concise" Boolean expression E2 equivalent to E1? (We can define the size of a Boolean expression as the number of occurrences of symbols needed to write it down; then a "most concise" expression is one that has the smallest possible size.)
2/10/14	Lecture 12: Boolean functions and expressions I. Summary: Boolean values 0 and 1, Boolean functions, Boolean expressions (syntax and semantics), truth tables. We start our investigation of computer hardware with Boolean functions. For this part of the course we will use the hardware convention for Boolean values, 0 = false and 1 = true (instead of the Racket convention of #f and #t). A Boolean function maps n-tuples of Boolean values to Boolean values, where n is a non-negative integer. (An n-tuple of values is an ordered sequence of n values, generally represented with parentheses and commas: (13,7,44) is a 3-tuple (or triple) of values, with first element 13, second element 7, and third element 44.) Note that the domain of a Boolean function is finite, so we can write down a table containing every possible input and the corresponding value of the function for that input. Four examples of Boolean functions when n = 1. input \| value input \| value input \| value input \| value ------------- ------------- ------------- ------------- 0 \| 0 0 \| 1 0 \| 0 0 \| 1 1 \| 1 1 \| 0 1 \| 0 1 \| 1 The first function shown above is the identity: it takes input x and returns value x, for x = 0,1. The second function is negation, or NOT -- if the input is 0 (false), the value is 1 (true); if the input is 1 (true), the output is 0 (false). The last two functions "ignore" their inputs: the third one is the constant function 0, and the fourth one is the constant function 1. These are the only possible Boolean functions when n = 1. To see this, note that the table for any such function has 2 lines, one for input 0 and one for input 1. input \| value ------------- 0 \| ? 1 \| ? Each of the two question marks can (independently) be replaced by 0 or 1 to get a Boolean function. Two choices for the first question mark times two choices for the second question mark gives four choices overall for Boolean functions when n = 1. Some examples of Boolean functions when n = 2. input \| value input \| value input \| value input \| value ------------- ------------- ------------- ------------- 0 0 \| 0 0 0 \| 0 0 0 \| 0 0 0 \| 1 0 1 \| 0 0 1 \| 1 0 1 \| 1 0 1 \| 1 1 0 \| 0 1 0 \| 1 1 0 \| 1 1 0 \| 0 1 1 \| 1 1 1 \| 1 1 1 \| 0 1 1 \| 1 (Note that I have written "0 1" instead of the 2-tuple (0,1).) The first function above is AND -- it is only 1 (true) when both of its inputs are 1 (true), and 0 (false) otherwise. The second function above is (inclusive) OR -- it is only 0 (false) when both of its inputs are 0, and 1 (true) otherwise. The third function is something called Exclusive OR, whose abbreviation is XOR -- it is 1 (true) if and only if exactly one of its arguments is 1 (true). Note that it differs from (inclusive) OR on the last line, when both arguments are 1 (true). In this case, inclusive OR is 1 but Exclusive OR is 0. We will use OR to mean (inclusive) OR, and use XOR when we mean Exclusive OR. The fourth function is one that logicians like; it is IMPLICATION, where the intuitive meaning is "if the first argument is true then the second argument is true." After long debate mathematical logicians agreed on this Boolean function as a good meaning for implication, though the first two lines of the table are a bit counterintuitive. This table represents the meaning intended by a mathematician who says "If p then q." Another useful Boolean function of two arguments is equivalence, given by the following table. input \| value ------------- 0 0 \| 1 0 1 \| 0 1 0 \| 0 1 1 \| 1 This function is 1 if and only if its two arguments are equal (both 0 or both 1.) How many total Boolean functions for n = 2? There are clearly four lines in the table, and on each line we can make a choice of 0 or 1 for the value of the Boolean function, so there are 2x2x2x2 = 2^4 = 16 possible Boolean functions when n = 2. We've shown 5 of them above; there are 11 others, including the constant function 0 and the constant function 1, whose tables are: input \| value input \| value ------------- ------------- 0 0 \| 0 0 0 \| 1 0 1 \| 0 0 1 \| 1 1 0 \| 0 1 0 \| 1 1 1 \| 0 1 1 \| 1 Note that these (technically) are different functions from the constant functions 0 and 1 for n = 1, because their domains are different (one has 4 elements, the other has 2 elements.) An example of a Boolean function for n = 3. input \| value --------------- 0 0 0 \| 0 0 0 1 \| 0 0 1 0 \| 0 0 1 1 \| 1 1 0 0 \| 0 1 0 1 \| 1 1 1 0 \| 1 1 1 1 \| 1 Note that this function is 1 if and only if at least two of its arguments are 1. Thus, it computes a "majority vote" of its inputs. How many Boolean functions are there for n = 3? Clearly, there are 2x2x2 = 2^3 = 8 lines in the table, and for each of these 8 lines we may independently choose a 0 or a 1 to enter for the value. Thus, there are 2x2x2x2x2x2x2x2 = 2^8 = 256 Boolean functions when n = 3. For a general non-negative integer n, how many Boolean functions are there? There are n arguments, each of which can be 0 or 1, so there are 2^n lines in the table for such a function. On each of those lines, we may (independently) enter a 0 or a 1 as the value, for a total of (2)^(2^n) possible Boolean functions of n arguments. Check for n = 3: we get 2^n = 2^3 = 8 lines, and 2^(2^n) = 2^8 = 256 functions. Here is a table of some values of this function n \| 2^(2^n) ---------------- 0 \| 2 1 \| 4 2 \| 16 3 \| 256 4 \| 65,536 (= 2^(16)) Note that each line is the square of its predecessor, so that this sequence can be described by the recurrence: A_0 = 2 A_(n+1) = (A_n)^2 for n >= 0 You may want to convince yourself of that algebraically. Note that by convention exponentiation associates to the right, and in general (a^b)^c is not equal to (a)^(b^c). Boolean expressions. We first introduce the syntax of a notation for representing Boolean functions. The definition we give is inductive (or recursive). We assume a countable supply of variable symbols, for example, x, y, z, x_1, y_1, z_1, x_2, y_2, z_2, ... . The definition: (Base cases) 0 is a Boolean expression 1 is a Boolean expression x is a Boolean expression, for any variable x (Inductive cases) If E1 and E2 are Boolean expressions, then so are: (E1)' (E1 * E2) (E1 + E2) (Note that in the above I've used the asterisk () instead of the centered dot that I've used on the blackboard.) As is customary with these inductive definitions, we agree that the only Boolean expressions are those that can be obtained in a finite number of steps using the rules above. As examples of Boolean expressions, we have: 0 is a Boolean expression (base case) 1 is a Boolean expression (base case) x is a Boolean expression (base case) y is a Boolean expression (base case) (0)' is a Boolean expression (from 0 and inductive case) (x 1) is a Boolean expression (from x and 1 and inductive case) ((0)' + (x * 1)) is a Boolean expression (from (0)' and (x * 1) ... There are a variety of other notations for the operations ', , +, but these will be our conventions in the part of the course. The syntactic definition just given requires a lot of parentheses, which are not so congenial for humans. We will use an order of operations convention for these operations: parentheses, ', , +. This agrees with the PEMDAS (or BEDMAS or BODMAS) mnemonic, if we note that ' looks like exponentiation, * looks like multiplication, and + looks like addition. Hence we can interpret the expression (x' + y)' as (fully parenthesized): (((x)' + y))'. The meaning (semantics) of Boolean expressions. We have a syntax, recursively defined, but we'd like to know what these expressions mean or denote. We start from the truth tables that many of you have previously encountered. To construct a truth table for the expression (x' + y)' we proceed as follows. There are two variables, x and y, so we have two columns labeled by x and y. We have a line in the table for each possible way to assign 0 or 1 to the variables x and y, thus: x y \| --------------- 0 0 \| 0 1 \| 1 0 \| 1 1 \| Now we look at the most deeply-nested subexpressions of our expression (the ones we'd have to construct first using our inductive definition of the syntax.) In this case, we need x' So we label a column of the table with x' and proceed to fill in values for it by applying NOT to the value of x on that line: x y \| x' --------------- 0 0 \| 1 0 1 \| 1 1 0 \| 0 1 1 \| 0 Now we take the next expression: (x' + y), and label a column with this expression, and enter values for it on each line by combining the value of x' for that line with the value of y for that line: x y \| x' (x' + y) ---------------------- 0 0 \| 1 1 0 1 \| 1 1 1 0 \| 0 0 1 1 \| 0 1 Note that this shows that the value of (x' + y) is 0 if and only if x = 1 and y = 0. Finally, to get the value of the top-level expression (x' + y)', we label another column of the table with this expression and enter values for it on each line by applying NOT to the value of (x' + y) on that line: x y \| x' (x' + y) (x' + y)' --------------------------------- 0 0 \| 1 1 0 0 1 \| 1 1 0 1 0 \| 0 0 1 1 1 \| 0 1 0 Note that the value of (x' + y)' is 1 if and only if x = 1 and y = 0. Also, this expression is not symmetric in x and y. If our expression had involved the * symbol, then we would have used the AND function to combine the values of the two expressions on each line. To relate this truth-table procedure to a slightly different formal treatment, we can view a Boolean expression as representing a map from environments to Boolean values. What is an environment? It is a map from variables to Boolean values, assigning a 0 or 1 to each variable. (In fact, we only need values for the variables that appear in the expression.) If we are only concerned with the values of the variables x and y, then there are four possibilities for environments: x = 0 and y = 0, or x = 0 and y = 1, or x = 1 and y = 0, or x = 1 and y = 1. Note that these correspond to the values in the columns labeled x and y in the four lines of the truth table above. What is shown in the other three columns, respectively, are the values taken by the expressions x', (x' + y) and (x' + y)' in those four environments. We'll see a recursive view of the process of determining the value of a Boolean expression in an environment in the next lecture.
2/7/14	Lecture 11: Computability IV. Summary: the definition of computability, the Church-Turing thesis, statement and proof of the unsolvability of the Halting Problem for Racket programs. Perlis epigram #83: What is the difference between a Turing machine and the modern computer? It's the same as that between Hillary's ascent of Everest and the establishment of a Hilton hotel on its peak. A formal definition of computability. A function from strings of 0's and 1's to strings of 0's and 1's is computable if there exists a Turing machine to compute it. We have seen some examples of computable functions, for example, the function whose input is a binary number and whose output is that binary number plus 1. There are of course many others, including addition, subtraction, multiplication, division, prime testing, reversing the input string, changing every 0 to a 1 and every 1 to a 0, and so on. In fact, any function from strings of 0's and 1's to strings of 0's and 1's that you could compute with a Racket program could also be computed with a Turing machine. Because we know that a large number of different computational systems could be substituted for Turing machines in the above definition (lambda expressions, general recursive functions, idealized Java programs, idealized Racket programs, ...) without changing the set of computable functions, we have a fair amount of confidence in the "naturalness" of the definition. (The point of "idealized" Java or Racket programs is that we consider programs with no memory limitations, unlike actual Java or Racket programs, that run on actual computers with some finite (though large) amount of memory.) The Church/Turing thesis claims that the class of functions formally defined as computable captures what we mean by our intuitive notion of "computability." It is a thesis rather than a theorem, because it concerns the relationship between a mathematically defined concept and an intuitive, not formal, notion of ours. It is possible to imagine that we might decide to discard the formal definition of computable functions if we discovered that there were better ways to think about computability, for example, because of new discoveries in physics. However, in the meantime, the formal definition is what we mean by "computable functions." Are there uncomputable functions? The answer to this question is YES. One example is a function associated with Hilbert's 10th problem, namely, the function that takes in a binary string representing a Diophantine equation and outputs 1 if the Diophantine equation has a solution in integers, and 0 if it doesn't. Another approach to seeing that there are uncomputable functions is to consider the sizes, or cardinalities, of the sets involved. The set of all Turing machines is a countable set, that is, it can be put into one-to-one correspondence with the positive integers. Thus, the set of all computable functions is also a countable set. However, the set of all functions from strings of 0's and 1's to all strings of 0's and 1's is an uncountable set, that is, cannot be put into one-to-one correspondence with the positive integers. Thus, there must be uncomputable functions from strings of 0's and 1's to strings of 0's and 1's. (In fact, "almost all" of them are uncomputable.) This is interesting, but not terribly satisfying, because we don't actually get our hands on one of the uncomputable functions. We'll see one important uncomputable function below, the Halting Problem, and a proof that it is uncomputable. The Halting Problem for Racket programs. It would be convenient to have a procedure (halts? proc expr) that takes a Racket procedure proc of one argument and an arbitrary Racket expression expr, and returns #t if (proc expr) halts and returns #f if (proc expr) doesn't halt. As an example, suppose we define the following. (define (nope n) (nope (- n 1))) Then clearly (nope 10) => (nope 9) => (nope 8) => .. (nope -2014) => .. never halts. Then we should have (halts? sqr 4) => #t, because the built in Racket procedure sqr halts on the input 4, but we should have (halts? nope 10) => #f, because the procedure nope does not halt on the input 10. The procedure halts? would facilitate the grading of your procedures -- if we could test to see that some procedure of yours was not going to halt on a test case, we could skip trying to run your procedure on that input, and simply assign 0 points for that test case. (As it is, we use a fixed "time out" to cut off the evaluation of your procedures on test cases if they run too long.) However convenient or useful, no such halts? procedure can exist. To see this, we argue by contradiction, as follows. If we had the prcedure (halts? proc expr), we could define the following other procedures. (define (s proc expr) (if (halts? proc expr) (nope 10) "hi!")) What does the procedure s do? Clearly, using the definition of halts?, we have: (s proc expr) returns the string "hi!" if (proc expr) doesn't halt doesn't halt if (proc expr) halts We may also define a procedure (q proc) as follows: (define (q proc) (s proc proc)) That is, q takes a procedure proc of one argument and calls (s proc proc). What does the procedure q do? (q proc) calls (s proc proc) which returns the string "hi!" if (proc proc) doesn't halt and doesn't halt if (proc proc) halts Now we ask the question of whether (q q) halts or not? It is somewhat complicated to follow the logic of the procedure definitions, so we'll consider cases: 1) if (q q) halts, then when (q q) calls (s q q) we see that (s q q) doesn't halt, because (q q) halts. Thus, if (q q) halts, then (q q) doesn't halt. 2) if (q q) doesn't halt, then when (q q) calls (s q q) we see that (s q q) returns the string "hi!", because (q q) doesn't halt. Thus, if (q q) doesn't halt, it halts (and returns the string "hi!") Together these cases imply that (q q) halts if and only if it doesn't halt, a contradiction. Thus (assuming, as usual, the consistency of mathematics), there must have been some "flaw" in our reasoning. The "flaw" in this case is the assumption that there exists a procedure (halts? proc expr) at all -- what the contradiction proves is that assumption is false, that is, the function described in the specification of halts? is uncomputable. This is the famous Halting Problem, and it is a problem for every sufficiently powerful programming language, eg, (idealized) Racket, C, C++, Java, Python, and so on. The theorem statement for Racket is as follows. There can be no Racket procedure (halts? proc expr) that returns #t if (proc expr) halts, and #f if (proc expr) does not halt. It might strike you as a bit odd to write (proc proc), that is, to call a procedure on itself. You might find it more reasonable to imagine writing a C program that strips the comments out of a C program, and then running it on itself. In fact, in the life of a new programming language it is sometimes a major milestone when a compiler for the language can be re-written in the language and succeeds in compiling itself.
2/5/14	Lecture 10: Computability III. Summary: design of a Turing machine to make a second copy of its input, design of another Turing machine to add 1 in binary, pictorial representation of Turing machines. There is a complete solution of the problem of designing a Turing machine to make a copy of its input in the notes: Turing machines. The design in the notes is a little different from the one we did in class -- an x or y is used to mark the symbol currently being copied, instead of a blank, which means that there can be a common state scanning left to the marked symbol and replacing it with its original value. A diagram showing the pictorial representation of the Turing machine in the notes here: Diagram: TM that copies its input. We also designed a Turing machine to add 1 to a binary number. For example, if the input is 10111, the output should be 11000. We review the addition algorithm specialized to binary numbers: 1 0 1 1 1 + 1 ----------- 1 1 0 0 0 We start at the rightmost digit and add 1 to the digit. If the result is 1, then we can just copy the rest of the digits to the left. If the result is 2 (in binary 10) then we write a 0 and move to the left with a carry of 1, which means we repeat the same operation on the next digit to the left. Thus, the procedure is: start at the rightmost digit and change 1's to 0's until a 0 is encountered -- change the 0 to a 1 and then just copy the remaining digits to the left. A special case arises if we encounter a blank before a 0, for example: 1 1 1 + 1 ------- 1 0 0 0 In that case, we replace the blank by a 1 and the result is complete. We then just have to re-position the head over the leftmost symbol of the output and halt. A pictorial representation of a Turing machine to accomplish this is here: Diagram: TM that adds 1 in binary.
2/3/14	Lecture 9: Computability II. Summary: the Collatz ("3x+1") Problem and conjecture, Hilbert's 10th problem, mathematical definitions of computation, Turing machines. See also the notes: Turing machines. We ended last lecture looking at a simple Racket procedure, as follows. (define (f n) (cond [(= n 1) 1] [(even? n) (f (/ n 2))] [else (f (+ 1 (* 3 n)))])) As an example, we have (f 10) => (f 5) => (f 16) => (f 8) => (f 4) => (f 2) => (f 1) => 1 It is clear that if we manage to get to a power of 2, then the second cond clause will repeatedly apply until n is reduced to 1, and the result will be 1. It is also clear that if the procedure terminates with any value, that value will be 1. However, it is unknown whether (f n) will terminate for every positive integer n. This question is known as the Collatz Problem (or Collatz conjecture, for the conjecture that it does terminate for every positive integer n) or 3x+1 problem. Many many hours have been spent on this problem without resolving it. Termination is known for values of n up through some quite large numbers, but we have no proof or counterexample to establish the truth or falsity of the conjecture. This is not exactly an earth-shattering problem, but it does illustrate how even a very simple computational procedure can elude our complete understanding. Hilbert's 10th problem. In 1900 the mathematician David Hilbert published a list of problems meant to stimulate and focus mathematical research for the 20th century. The 10th problem on his list can be formulated as follows. Given a Diophantine equation with any number of integer coefficients: to devise a process according to which it can be determined in a finite number of operations whether the equation is solvable in integers. A Diophantine equation is an equation containing a finite number of variables that may be raised to positive integer powers and multiplied together, with positive or negative integer coefficients. The question to be answered is whether or not there are positive or negative integer values of the variables that satisfy the equation. As examples of Diophantine equations, we have 3xy - 5y^2 - 1 = 0 x^2 + y^2 - 13 = 0 x^2 + y^2 - 6 = 0 For the first two equations, the answer should be "yes". For the first equation, x = 2 and y = 1 gives a solution (321 - 511 - 1 = 0), so the answer is "yes." For the second equation, x = 2 and y = 3 (or x = 3 and y = 2) gives a solution (22 + 3+3 - 13 = 0), so the answer is "yes." For the third equation, the answer is "no". To see this, we can reason as follows. If there is a solution in integers x and y, then there is a solution in non-negative integers, because (-x)^2 = x^2. If x is greater than or equal to 3, then x^2 is greater than or equal to 9, and there can be no solution of x^2 + y^2 - 6 = 0, because y^2 is greater than or equal to 0 for every real number y. Thus, to verify that there is no solution to the third equation, it suffices to check each possibility with x and y being an integer between 0 and 2, inclusive. Checking these nine possibilities shows that there is no solution of the third equation in integers. What Hilbert was asking for in the 10th problem was, in effect, an algorithm* that could take in an arbitrary Diophantine equation as input, and answer "yes" or "no" according to whether the equation has a solution in integers. It took until 1970, but the answer to Hilbert's question turned out to be "there is no such algorithm" because the problem as stated is uncomputable -- no algorithm can exist to solve it. How did this answer come about? In the 1930's, logicians and other mathematicians worked hard to give a formal definition of what it means to be an effective process or algorithm, and thereby establish a criterion for when a function is computable or uncomputable. A variety of different formalisms were proposed, among them the following. Church's lambda calculus Kleene's general recursive functions Markov's algorithms Post's rewriting systems Turing's machines One of the outgrowths of Church's lambda calculus is the LISP-Scheme-Racket family of programming languages. These formalisms were apparently very different, but mathematicians found that they all defined the same set of computable functions. This was established by means of simulations -- for example, for each Turing machine, a lambda expression could be defined that would simulate the computation of the Turing machine. The resulting set of computable functions was then taken to be the definition of what we mean by a computable function. Returning to Hilbert's 10th problem, work by mathematicians Julia Robinson and Martin Davis showed that Diophantine equations are sufficient to express computation if Diophantine equations whose solutions have certain growth properties exist, and in 1970 the young Russian mathematician Yuri Vladimirovich Matiyasevich proved that they did exist, thereby completing the proof that Hilbert's 10th problem is algorithmically unsolvable. We will look in detail at one of the formal definitions of computation: Turing machines. These were defined in a 1936 paper by Alan Turing written when he was 23 years old, titled "On Computable Numbers with an Application to the Entscheidungsproblem." Despite the formidable title of the paper, part of it consists of an appeal to intuition to establish that the abstract machines he proposed (later named "Turing machines") captured the essence of the operations of a human computer following a specific series of steps in a mathematical procedure. He takes as his model of storage a child's exercise book, ruled into squares, meant to keep the columns of an addition or multiplication problem lined up by putting one digit in each square. Instead of a two-dimensional array of squares, Turing assumes that the machine has a tape, ruled into squares, and each square may contain one symbol from a finite alphabet of symbols. A typical alphabet might be blank, 0 and 1. The length of the tape is indefinite -- we imagine that it extends as far as needed in both directions. In general, there will be a finite number of squares containing non-blank symbols, and all the other squares are assumed to contain the blank symbol. To operate on the tape, the machine has a read/write head located at one particular square of the tape, and the symbol on that square of the tape is the "current symbol." Only the current symbol may be read or written -- to operate on other symbols on the tape, the machine may move the head left or right one square at a time. In addition to the storage of symbols on the tape, the machine has some "temporary memory", which consists of a state from a finite set of states, which we will denote q1, q2, q3, and so on. At any time, the machine is in one of the possible states, which is the "current state." As it operates, it may change from one state to another to "remember" some information (for example, what part of the computation is it in.) There is a designated "start state" of the machine, typically q1. The Turing machine has a finite set of instructions, which determine how its computation will proceed. Each instruction consists of five parts: (current state, current symbol, new state, new symbol, head direction) where the current state and the new state are states from the finite set of states for the machine, the current symbol and new symbol are symbols from the finite alphabet of the machine, and the head direction is either L (for left) or R (for right). So a typical instruction might be (q3, 0, q6, 1, R) The computation of a Turing machine may be viewed as a sequence of steps, each step transforming a configuration of the machine to another configuration of the machine according to one of the instructions of the machine. A configuration of the machine specifies: the symbols contained by the squares on the tape, the position of the read/write head on the tape, and the current state of the machine. For example, we can specify a configuration of a machine as follows. ----------------------------------- .. \| \| 1 \| \| 0 \| 0 \| 1 \| \| .. ----------------------------------- ^ q3 The dots (..) are meant to indicate that the tape continues indefinitely to the left and right, and all the squares not pictured contain the blank symbol. Of the squares pictured, we have the sequence of symbols: blank, 1, blank, 0, 0, 1, blank The caret (^) is meant to indicate what square the read/write head of the machine is scanning. In this case, that square contains a 0, so the current symbol is 0 in this configuration. The current state of the machine is indicated below the caret -- the current state of the machine is q3. In this configuration of the machine, the instruction (q3, 0, q6, 1, R) applies, because the current state is q3 and the current symbol is 0. The rest of the instruction indicate how to update the configuration to get the new configuration. The new state should be q6, the current symbol should be replaced by 1, and, finally, the position of the read/write head should be moved one symbol to the right. These changes yield the following configuration. ----------------------------------- .. \| \| 1 \| \| 1 \| 0 \| 1 \| \| .. ----------------------------------- ^ q6 This represents one step of a Turing machine computation -- an instruction applies to a configuration to yield a new configuration. As our first example of constructing a Turing machine we'll consider the problem of taking an input consisting of a continguous sequences of 0's and 1's, surrounded by blanks, and modifying it so that every 0 is replaced by a 1 and every 1 is replaced by a 0, and then halting. Our convention for a Turing machine's input is that it consists of a continguous sequence of non-blank symbols surrounded by blanks. The initial configuration of the machine is obtained by positioning the head on the leftmost non-blank symbol of the input, and putting the machine into the start state, q1. Thus, the initial configuration for the input string 110 can be pictured as follows. --------------------- ... \| \| 1 \| 1 \| 0 \| \| ... --------------------- ^ q1 Looking at this configuration, we see we need an instruction for q1 and 1 that changes the 1 to a 0, moves right one step, and stays in state q1. That is, we need the instruction: (q1, 1, q1, 0, R) This instruction applies in the initial configuration to give the following configuration after one step: ----------------------- ... \| \| 0 \| 1 \| 0 \| \| ... ----------------------- ^ q1 The same instruction also applies in this configuration to give the following configuration: ----------------------- ... \| \| 0 \| 0 \| 0 \| \| ... ----------------------- ^ q1 There is no instruction that applies in this configuration yet, so we define one: it should change the 0 to a 1, move one step to the right, and stay in state q1. The instruction is (q1, 0, q1, 1, R) After this instruction is applied to the above configuration, we have the following configuration. ----------------------- ... \| \| 0 \| 0 \| 1 \| \| ... ----------------------- ^ q1 In this configuration no instruction applies (we have not defined an instruction for q1 reading a blank), so the machine halts. Thus, we can accomplish the task we set ourselves with the following program of two instructions. (q1, 1, q1, 0, R) (q1, 0, q1, 1, R) Now suppose we want to elaborate our program to return the read/write head to the first non-blank symbol of the result (namely, the leftmost 0 above.) We can use another state, say q2, to accomplish this task. First, we need a way of transitioning from q1 to q2: we define an instruction for q1 with current symbol = blank. Because it is hard to indicate a blank, we'll use the symbol b to indicate it. So the instruction that takes current state q1 and current symbol blank to new state q2, new symbol blank, and one step left is as follows: (q1, b, q2, b, L) After this instruction is applied to the above configuration, we get the following configuration. ----------------------- ... \| \| 0 \| 0 \| 1 \| \| ... ----------------------- ^ q2 Now in state q2 we want the machine to move left on the non-blank symbols not changing them, until it encounters the first blank to the left of them. The way we "don't change" a symbol is to write the same symbol that we read. Thus, for q2 we can add the two instructions: (q2, 0, q2, 0, L) (q2, 1, q2, 1, L) The second of these two instructions applies to the above configuration, then the first, then the first again, yielding (after 3 more steps) the following configuration. ---------------------- ... \| \| 0 \| 0 \| 1 \| \| ... ---------------------- ^ q2 Now we have almost what we wanted: the head has to be moved one square to the right, and the machine has to halt. We accomplish this with one more instruction, that transitions to a new state, q3. (q2, b, q3, b, R) The final configuration of the machine is then as follows. ----------------------- ... \| \| 0 \| 0 \| 1 \| \| ... ----------------------- ^ q3 The machine is halted in this configuration because there is no instruction for q3 and 0 (in fact, no instructions for q3 at all.)
1/31/14	Lecture 8: Racket VII, Computability I. Summary: demonstration of the game "Shut the Box", the special forms let and let, the start of the new topic of Computability. If you missed the class demonstration of "Shut the Box", you can read about it on the web, for example, Wikipedia's article. Special forms let and let. The special forms let and let* create a new local environment for the evaluation of one or more expressions. As an example: > (let ((x 13) (y 17)) (+ (sqr x) (* 3 x y) (sqr y))) 1121 > The first argument of let is a list of two-element lists consisting of an identifier (in the example, x or y) and an arbitrary expression (in the example, the constants 13 and 17.) This causes a temporary local environment to be set up in which each identifier is bound to the value of its corresponding expression. The "search pointer" for the local environment is the environment in which the let expression is being evaluated. The rest of the let expression is its "body", which is typically one expression (in the example, the expression (+ (sqr x) (* 3 x y) (sqr y)).) This expression is evaluated in the just-created local environment, and its value is returned as the value of the whole let expression. (The body may consist of more than one expression; in this case, they are evaluated in order and the value of the last one is the value of the whole let expression.) Once the value of the let expression has been determined, the local environment it created is typically no longer accessible, and is available to be "recycled." We can picture the environment structure for the example just before the body is evaluated as follows. top-level environment: no search pointer --------------------- \| ... \| ----------------------------------------------------- \| + \| the built-in addition procedure \| ----------------------------------------------------- \| * \| the built-in multiplication procedure \| ----------------------------------------------------- \| sqr \| the built-in squaring procedure \| ----------------------------------------------------- local environment: search pointer to top-level environment ------------------ --------------------- \| x \| 13 \| --------------------- \| y \| 17 \| --------------------- Evaluating (+ (sqr x) (* 3 x y) (sqr y)) in the local environment follows the previous rules for evaluation, except that the word "relevant" in the rule for identifiers (look it up in the relevant environment) is now interpreted as follows: the current environment is the local environment shown. If an identifier is not found in the local environment, we follow the search pointer and look in that environment. If it is not found there, we again follow the search pointer until there is no search pointer to follow. The value for the identifier is the first one found in this process, or results in an undefined identifier error if it is not found in any of the environments searched. Thus, in the example, the values of x and y are found in the local environment, whereas the values of +, , and sqr are found in the top-level environment. A let expression may occur in the body of a let expression. An example of this is the following. > (let ((x 3) (y 4)) (let ((z (+ x y))) ( x y z))) 84 > Here the outer let expression creates a local environment in which x is bound to 3 and y is bound to 4. The inner let expression creates another local environment in which z is bound to 7, and in this second local environment the expression (* x y z) is evaluated to give a value of 84. A picture of the environments just before the body of the inner let expression is evaluated is as follows. top-level environment: no search pointer --------------------- \| ... \| ----------------------------------------------------- \| + \| the built-in addition procedure \| ----------------------------------------------------- \| * \| the built-in multiplication procedure \| ----------------------------------------------------- \| sqr \| the built-in squaring procedure \| ----------------------------------------------------- local environment #1: search pointer to top-level environment -------------------- --------------------- \| x \| 3 \| --------------------- \| y \| 4 \| --------------------- local environment #2: search pointer to local environment #1 -------------------- --------------------- \| z \| 7 \| --------------------- When the body of the inner let expression, (* x y z), is evaluated in local environment #2, because of the search pointers the binding of * is found in the top-level environment, the bindings of x and y are found in local environment #1, and the binding of z is found in local environment #2. Suppose we tried to combine the two let expressions in the following INCORRECT way. > (let ((x 3) (y 4) (z (+ x y))) (* x y z)) x: undefined; cannot reference an identifier before its definition > We get an error message saying that x is undefined because when trying to evaluate the expression for the value of z, x is not yet defined (that is, not defined in the environment in which the let expression is being evaluated.) This problem can be avoided by using let* instead of let in this case, as follows. > (let* ((x 3) (y 4) (z (+ x y))) (* x y z)) 84 > One way to understand let* is that it behaves as though there were a nested let expression for each successive identifier, so that later ones in the sequence can refer to earlier ones. Thus, order is important -- putting the entry for z before the entries for x and y would elicit an error message. Why would we ever use let or let? There are at least two answers: (1) to save typing and reduce errors, and (2) to save computation. If we have a complicated sub-expression that occurs several times in a larger expression, by giving a name to the value of that complicated sub-expression, we can reduce typing (name instead of sub-expression), make the larger expression clearer (we more easily see when a value repeats in the larger expression), and avoid duplicating code (there is only one instance of the complicated sub-expression -- we only have to get it correct once, and there is only one place to change if it requires changes.) In addition, if a sub-expression requires a lot of computation time to evaluate, by evaluating it once and naming the resulting value, we avoid repeating the computation, and instead just look the value up in the environment. This effect may be particularly important in a recursive procedure, where duplicated computation may be re-duplicated and re-duplicated in successively deeper levels of the recursion. As a simple (and somewhat silly) example of this, suppose I have decided on the following recursive procedure to compute 2 raised to the power n for non-negative integers n. (define (h n) (if (= n 0) 1 (+ (h (- n 1)) (h (- n 1))))) This is correct, based on the observation that 2^0 = 1 and for n > 0, 2^n = 2^(n-1) + 2^(n-1). (Let's ignore for the moment that I might have written ( 2 (h (- n 1))).) Running this procedure for n = 1000 will warm up the room and complain about running out of stack space, but will not give an answer. Let's look at the tree of recursive calls for (h 3), leaving out other operations. (h 3) / \ / \ / \ / \ / \ (h 2) (h 2) / \ / \ / \ / \ (h 1) (h 1) (h 1) (h 1) / \ / \ / \ / \ (h 0) (h 0) (h 0) (h 0) (h 0) (h 0) (h 0) (h 0) Each of the leaves (h 0) evaluates to 1 using the base case, and the 1's get summed going up the tree to yield a value of 8 for (h 3). Note that (h 0) is evaluated 8 = 2^3 separate times to get this result. In the case of (h 1000), in order for the computation to complete, there would have to be 2^(1000) evaluations of (h 0), which is way more than any computer is going to complete. However, we can improve the situation using the special form let as follows. (define (new-h n) (if (= n 0) 1 (let ((x (new-h (- n 1)))) (+ x x)))) If you try (new-h 1000), then DrRacket will quickly respond with the answer (using its unlimited precision integers), which is a number with 302 decimal digits, starting 1071508 ... and ending ... 8069376. This new version of the procedure only evaluates (h m) once for each value of m from n down to 0. The tree of recursive calls for (new-h 3) is as follows. (new-h 3) \| (new-h 2) \| (new-h 1) \| (new-h 0) You will find at least one opportunity to make use of this idea in assignment #2 (Spring 2014.) There was an aside on the relationship between the number of binary digits to represent a number n, and the number of decimal digits to represent the same number n. Clearly the decimal representation of n will be shorter than the binary representation in general, but by how much? In binary, 2^(1000) has 1001 binary digits (one 1 followed by 1000 0's.) From the above, we see that 2^(1000) has 302 decimal digits. The number of digits base b of a non-negative integer n is nearly the log to base b of n. (Actually, the ceiling of the log base b of (n+1).) For example, the log base 10 of 1000 is 3, which is (nearly) the number of decimal digits in 1000. Consider the number c = the log base 2 of 10, which is approximately 3.32. We have 2^c = 10, so taking both sides to the power d, we get 2^(cd) = 10^d. So, roughly speaking, a number n with cd binary digits will have d decimal digits. Dividing our 1001 binary digits by c = 3.32, we get about 301.5, nearly the 302 digits in the binary representation of 2^1000. A simpler case is converting a binary number to octal (base 8) -- just group the digits into threes from the right, and write down the corresponding octal digits, so 10110111 in binary becomes (10)(110)(111), or 267 in octal. (Or tl;dr, the relationship between the number of digits in a number in two different bases is essentially a constant factor.) New topic: Computability. Many of you will have solved the problem of finding a unit fraction representation of m/n (from assignment #1) by the following "greedy" method, which we illustrate for m/n = 6/7. Find the smallest positive integer k such that 6/7 is greater than or equal to 1/k. In this case, k = 2, so we have the following. 6/7 = 1/2 + ? To find the value of ?, we subtract 1/2 from 6/7 as follows. 6/7 - 1/2 = 12/14 - 7/14 = 5/14. Now we continue (recursively) with 5/14. We find the smallest positive integer k such that 5/14 is greater than or equal to 1/k. In this case, k = 3, so we have the following. 6/7 = 1/2 + 1/3 + ?? To find the value of ??, we subtract 1/3 from 5/14 as follows. 5/14 - 1/3 = 15/42 - 14/42 = 1/42 Thus we have the desired unit fraction representation of 6/7. 6/7 = 1/2 + 1/3 + 1/42 But can we be sure that this process will always terminate? That is, if we carry out this process, can it happen that we just keep getting smaller and smaller unit fractions, without ever finding that the result of our subtraction is itself a unit fraction? We could run our procedure on all kinds of proper fractions m/n and discover that it terminated for each one of them. But that says nothing about whether it would terminate for every proper fraction m/n. One line of argument might be that a proper fraction couldn't be equal to an infinite sum of unit fractions. However, we note the following. 5/6 = 1/3 + 1/4 + 1/8 + 1/16 + 1/32 + ... where after the initial 1/3, the remaining geometric series sums to 1/2. So we must have to use some more specific information about our algorithm to make sure it terminates for every proper faction m/n. The observation that we need is that with each step (find the smallest positive integer k such that m/n is greater than or equal to 1/k, then subtract 1/k from m/n) the numerator of the resulting fraction will be strictly less than the numerator of the input. In our example for 6/7, the next numerator was 5, then the next numerator was 1. Why is the numerator of the resulting fraction strictly less than m? We have that m/n is strictly less than 1/(k-1) and greater than or equal to 1/k. The difference is computed as follows. m/n - 1/k = km/kn - n/kn = (km - n)/kn And we have the following. m/n < 1/(k-1) (k-1)m < n (after multiplying both side by (k-1)n, a positive number) km - m < n km - n < m (after adding (m-n) to both sides) This shows that (km - n) is strictly less than m. Because each step must strictly decrease the numerator, and the numerator must be a nonnegative integer, there can only be a finite number of steps. Sometimes it is obvious why a procedure will terminate (eg, our recursive procedure for factorial) and sometimes it is quite a bit less obvious (eg, our greedy unit fraction procedure.) As another example, here is the definition of a simple procedure whose inputs are positive integers. (define (f n) (cond [(= n 1) 1] [(even? n) (f (/ n 2))] [else (f (+ 1 (* 3 n)))])) As an example, we'll compute (f 3). (f 3) => (f 10) => (f 5) => (f 16) => (f 8) => (f 4) => (f 2) => (f 1) => 1 It took a few steps, but we found (f 3) => 1. Question: is it the case that for every positive integer n, (f n) evaluates to 1? It is pretty clear that if it evaluates to anything, it evaluates to 1. So the question comes down to: does (f n) terminate for every positive integer n?
1/29/14	Lecture 7: Racket VI. Summary: double-each and square-each procedures, generalization to the map procedure, the append procedure, deep recursion and the flatten procedure. We continue with recursion. So far we've seen recursion on integers and "flat" or "top-level" recursion on lists. recursion on integers base cases recursive cases --------------------- ---------- --------------- (factorial n) (= n 1) (factorial (- n 1)) (first-digit n) (< n 10) (first-digit (quotient n 10)) (countdown n) (= n 0) (countdown (- n 1)) recursion on lists base cases recursive cases ------------------ ---------- --------------- (length lst) (null? lst) (length (rest lst)) (member item lst) (null? lst) (member item (rest lst)) (equal? item (first lst)) For integers the recursive case has involved (- n 1) or (quotient n 10) -- the latter is what mathematicians would call "recursion on notation". For lists so far the recursive case has involved (rest lst), the rest of the top-level elements of the input lst. We consider another pattern of top-level or "flat" recursion on lists. Suppose we want to write a procedure to double each element of a list of numbers, eg, (double-each '(3 4 7)) => '(6 8 14). First we think about the base case: what result do we want when we double each element of an empty list? It seems that the result should be the empty list in this case. If the list is not empty, then it has at least one element, and we may apply car (or first) to it without error to get the first element of the list. If we think in terms of our example, if lst => '(3 4 7), then (first lst) => 3 and (rest lst) => '(4 7). A recursive call to double-each with (rest lst) will (if it is functioning correctly) simply return '(8 14). Then, to get the correct result for (double-each lst), we need to cons 6 to the front of this result. And 6 is just (* 2 (car lst)). Thus, we may write the procedure double-each as follows. (define (double-each lst) (if (null? lst) '() (cons (* 2 (first lst)) (double-each (rest lst))))) Just to be sure we understand this procedure, we can construct the tree of recursive calls and returns for our example, (double-each '(3 4 7)). (double-each '(3 4 7)) => '(6 8 14) / \ \ cons 6 (double-each '(4 7)) => '(8 14) / \ \ cons 8 (double-each '(7)) => '(14) / \ \ cons 14 (double-each '()) => '() This procedure is clearly not tail recursive, since another valued is cons'ed onto the result of the recursive call before it is returned as the value of the top-level call. Note that there are at least two ways of thinking about a recursive procedure of this kind. One is the "executive assistant" view -- a recursive call is like a very efficient executive assistant, who can be trusted to return the correct result for the recursive call, without micromanaging. All "we" (the top-level procedure) have to focus on is how to modify or combine the correct results of the recursive call(s) to get the desired top-level answer. Another view, evident when we trace the tree of recursive calls all the way to the base cases, is to "dive into" the recursive calls to understand their overall structure. Both views can be useful in fully understanding a recursive procedure, but the "executive assistant" view can be a particularly helpful focus when designing recursive procedures. Suppose we now want to write a procedure to square each element of a list of numbers, eg, (square-each '(3 4 7)) => '(9 16 49). Now instead of doubling (first lst), we can instead square it (using the built-in procedure sqr). We could just copy, paste, and modify our double-each procedure, as follows. (define (square-each lst) (if (null? lst) '() (cons (sqr (first lst)) (square-each (rest lst))))) Done. Well, not so fast -- it is often a bad idea to duplicate code, or duplicate and modify code, because when (and it is usually "when", not "if") we decide to modify the code slightly, we have to be sure to track down all the places it has been copied, make the changes in a parallel fashion, and generally, create more opportunities for error. The pattern of "apply a procedure to each element of a list and return the list of results" has been abtracted to a built-in procedure map. To accomplish the effects of double-each or square-each we could use map as follows. > (map (lambda (x) (* 2 x)) '(3 4 7)) '(6 8 14) > (map sqr '(3 4 7)) '(9 16 49) > In each of these cases, the first argument to map is a procedure of one argument and the second argument to map is a list of elements to which the procedure may be applied. The result is the list of values resulting from applying the procedure to each element of the list (in order). Note that the expression (lambda (x) (* 2 x)) evaluates to a procedure to double its argument, and the expression sqr evaluates to the built-in procedure to square numbers. We can easily write a procedure with this behavior by abstracting the procedure to be applied (multiply by 2 or square) into an argument to the procedure, as follows. (define (our-map proc lst) (if (null? lst) '() (cons (proc (first lst)) (our-map proc (rest lst))))) Note the strong similarity of this procedure to the code for double-each and square-each. If the input list lst is empty, then the result should be the empty list. Otherwise, we extract the first element of lst and apply the procedure proc to it using the expression (proc (first lst)), and cons the resulting value to the front of the list produced by a recursive call with proc and the rest of the elements in the list, (our-map proc (rest lst)). In fact, the built-in map procedure can deal with procedures proc of more than one argument; please read its description for details. We've seen two list constructors, the basic one, cons, that takes an item and a list and returns a new list equal to the input list with the item inserted in the first position, and the built-in procedure list, which takes any number of arguments, evaluates them in order and makes a list of their values. Now we consider the built-in procedure append, which takes any number of lists as arguments and returns a new list that contains the elements of all of them, in order. For example we have the following. > (append '(1 2 3) '(4 5)) '(1 2 3 4 5) > We can write our own version of append that combines the elements of two lists into a single list, as follows. (define (our-append lst1 lst2) (if (null? lst1) lst2 (cons (first lst1) (our-append (rest lst1) lst2)))) Although the problem might at first seem complicated by the fact that there are two argument lists, this procedure simply does flat recursion on lst1, with lst2 essentially just coming along for the ride. The base case is when lst1 is empty; in that case, the combined elements of lst1 and lst2 are just lst2. In the recursive case, the recursive call makes a list out of the rest of the elements of lst1 and lst2, and then the first element of lst1 is cons'ed onto the result. The tree of recursive calls and returns for (our-append '(1 2 3) '(4 5)) is as follows. (our-append '(1 2 3) '(4 5)) => '(1 2 3 4 5) / \| \ cons 1 (our-append '(2 3) '(4 5)) => '(2 3 4 5) / \| \ cons 2 (our-append '(3) '(4 5)) => '(3 4 5) / \| \ cons 3 (our-append '() '(4 5)) => '(4 5) This procedure is not tail recursive, because a value is cons'ed onto the result of the recursive call before it is returned as the value of the top-level procedure. We briefly talked about a tail recursive version of this procedure, using the built-in procedure reverse to reverse a list. That version looks as follows. (define (another-our-append lst1 lst2) (another-our-append-aux (reverse lst1) lst2)) (define (another-our-append-aux rlst1 lst2) (if (null? rlst1) lst2 (another-our-append-aux (rest rlst1) (cons (first rlst1) lst2)))) The recursive procedure another-our-append-aux is tail recursive, because the result of the recursive call is returned unmodified at the top level. The wrapper procedure another-our-append reverses the first list before it calls another-our-append-aux, so that the elements from the end of lst1 can be moved one by one to lst2. We can construct the tree of calls for (another-our-append '(1 2 3) '(4 5)). (another-our-append '(1 2 3) '(4 5)) => '(1 2 3 4 5) \| (another-our-append-aux '(3 2 1) '(4 5)) => '(1 2 3 4 5) \| (another-our-append-aux '(2 1) '(3 4 5)) => '(1 2 3 4 5) \| (another-our-append-aux '(1) '(2 3 4 5)) => '(1 2 3 4 5) \| (another-our-append-aux '() '(1 2 3 4 5)) => '(1 2 3 4 5) Note that in this case, the second argument is not just along for the ride, but acts as an accumulator for the final result. "Deep" recursion. In our previous procedures on lists, we have used "flat" recursion, performing some operation on each element of a list (possibly stopping early, as with member), but not recursively processing the elements of the list itself. Now we consider "deep" recursion, in which we recursively process lists within lists within lists, and so on. As an example, we consider the built-in procedure (flatten expr), which, intuitively, takes all but the outermost pair of parentheses out of a list. For example we have the following. > (flatten '((2 (3)) () ((4 5)))) '(2 3 4 5) > The procedure flatten can accept a non-list as an input, for example, (flatten 13) => '(13). In this case, it places the value in a list. We can write our own version of flatten, as follows. (define (our-flatten expr) (cond [(not (list? expr)) (list expr)] [(null? expr) '()] [else (append (our-flatten (first expr)) (our-flatten (rest expr)))])) This procedure first tests whether its argument expr is a list. If not, it simply returns the value of expr in a list. If expr is a list, then it next tests whether it is the empty list. If so, the answer is just the empty list. If not, then the value of expr is a list that is not empty, which means we can apply first and rest to it without error. In this case, there are two recursive calls to our-flatten, one with the first element of the list, and one with the rest of the elements of the list. The two resulting lists are append'ed together to form the final list of elements, containing elements of the flattened version of (first expr) followed by the elements of the flattened version of (rest expr). To get a better sense of how this works, we construct the tree of recursive calls for (our-flatten '(((3)) () 4)). I abbreviate our-flatten to o-f. (o-f '(((3)) () 4)) ------------------------- / \| \ / \| \ / \| \ append (o-f '((3))) (o-f '(() 4)) ----------- ------------- / \| \ / \| \ / \| \ / \| \ append (o-f '(3)) (o-f '()) append (o-f '()) (o-f '(4)) ---------- ---------- / \| \ / \| \ append (o-f 3) (o-f '()) append (o-f 4) (o-f '()) Noting that the base cases (our-flatten 3) => '(3), (our-flatten 4) => '(4), and (our-flatten '()) => '(), and appending the results appropriately, we see that the result of (our-flatten '(((3)) () 4)) is '(3 4).
1/27/14	Lecture 6: Racket V. Perlis epigram #55: A LISP programmer knows the value of everything, but the cost of nothing. Summary: lists continued, making our-length tail recursive, member and our-member, list constructors cons and list, the special form quote, the procedure countdown. Recall that a list is a finite sequence of values. The built in predicate (null? expr) tests whether the value of expr is the empty list. The built in predicate (list? expr) tests whether the value of expr is a list (empty or nonempty.) Last time we defined our own version of the procedure length that returns the length (the number of top-level elements) of a list. The procedure we wrote was as follows. (define (our-length lst) (if (null? lst) 0 (+ 1 (our-length (rest lst))))) This procedure is not tail recursive because the result of the recursive call, (our-length (rest lst)), is not returned as is, but has 1 added to it before it is returned as the value of the top-level call. We can rewrite this procedure to be tail recursive by including an additional argument to act as an "accumulator" of the value we are trying to compute. (define (our-length-aux lst len) (if (null? lst) len (our-length-aux (rest lst) (+ 1 len)))) This procedure is clearly tail recursive because the value of the recursive call, (our-length-aux (rest lst) (+ 1 len)), is returned without further modification as the value of the top-level call. For the application (our-length-aux '(13 14 15) 0) the tree of recursive calls looks as follows. (our-length-aux '(13 14 15) 0) => 3 \| (our-length-aux '(14 15) 1) => 3 \| (our-length-aux '(14) 2) => 3 \| (our-length-aux '() 3) => 3 Note that the second argument, len, starts at 0 and is used to "accumulate" the partial value of the length as elements are discarded from the front of the list. Note also that the evaluations of (rest lst) and (+ 1 len) take place before the recursive call of the procedure our-length-aux, in accordance with the rule for evaluating the application of a procedure. This is all very well, but we'd like a procedure of one argument rather than two. We can simply include a "wrapper" procedure that supplies the correct initial value of the additional argument, as follows. (define (new-our-length lst) (our-length-aux lst 0)) This, as we have seen, will correctly return the value of the length of lst. This strategy can be generalized to convert many procedures that are not tail recursive into ones that are tail recursive -- introduce one or more additional arguments to "accumulate" the partial results rather than waiting until the recursion reaches a base case to begin "accumulating" the final result. A wrapper is then used to supply the necessary initial values for the additional arguments, and present the correct interface for the procedure. There is a built in procedure (member item lst) to determine whether item is equal? to a top-level element of lst. If not, the result is #f, but, if so, then what is returned is the list starting with the first top-level element equal? to item. As examples of member, we have the following. > (member 2 '(1 2 3)) '(2 3) > (member 4 '(1 2 3)) #f > (member 2 '(1 2 3 2 1)) '(2 3 2 1) > We can write our own version of member as follows. (define (our-member item lst) (cond [(null? lst) #f] [(equal? item (first lst)) lst] [else (our-member item (rest lst))])) If the list is empty, item cannot be equal? to any element of it, so the result is #f. If the list is non-empty, we can compare item with the first element of the list. If they are equal?, we return the list, since it will start with the first element equal? to item. If they are not equal?, we continue with item and (rest lst). This procedure is clearly tail recursive, because the value of the recursive call, (our-member item (rest lst)), is returned unmodified as the value of the top-level call. As an example, we can consider the tree of recursive calls for (our-member 14 '(12 13 14 15)). (our-member 14 '(12 13 14 15)) => '(14 15) \| (our-member 14 '(13 14 15)) => '(14 15) \| (our-member 14 '(14 15)) => '(14 15) Note that this procedure does not examine every element of the list if it finds an element equal? to item before the end. Suppose instead we wanted a procedure to return #f if item is not equal? to any top-level element of lst, but #t it is equal? to some top-level element of lst. In this case, the only modification we'd have to make to the procedure our-member would be to return #t instead of lst in the second clause of the cond. The most basic list constructor is (cons item lst), which takes an item and a list and returns a new list containing the elements of lst, with item included at the beginning. Thus, (cons 12 '(13 14 15)) => '(12 13 14 15). The procedure list takes any number of arguments, evaluates them in order, and returns a list of the values. Thus, (list 12 13 14 15) => '(12 13 14 15), and also (list (* 2 6) (+ 3 10) (/ 28 2) (- 18 3)) => '(12 13 14 15). Note that list is different from quote. For example, (list (* 3 5)) => '(15), while '(* 3 5) => '(* 3 5). The special form quote (abbreviated with a single quote to the left) inhibits the usual rules of evaluation, so that '(* 3 5) or (quote (* 3 5)) does NOT evaluate the expression (* 3 5). Next we write a procedure (countdown n) which takes a non-negative integer n as input and returns a list containing the integers n down to 1 followed by the string "blast off!". Thus, (countdown 3) => '(3 2 1 "blast off!") The base case, n = 0, is not so clear, but the recursive case is pretty easy to understand: if we have (countdown 2) => '(2 1 "blast off!"), then to get (countdown 3) => '(3 2 1 "blast off!"), we simply have to cons 3 to the beginning of the result of (countdown 2). Thus we have the partially defined procedure (define (countdown n) (if (= n 0) ???? (cons n (countdown (- n 1))))) Now we can figure out the base case by what we need to get (countdown 1) => '(1 "blast off!") to work. In particular, we'd like to cons 1 to '("blast off!"), so this is the base case we need, and the final procedure is as follows. (define (countdown n) (if (= n 0) '("blast off!") (cons n (countdown (- n 1))))) Note in particular if we had simply returned the string "blast off!" for n = 0, instead of a list containing this string, then for n = 1, the attempt to cons 1 onto a string would give an error message. (In class we stopped at n = 1, with a base case of '(1 "blast off!"), which works correctly for all positive integers n.)
1/24/14	Lecture 5: Racket IV. Perlis epigram #3: Syntactic sugar causes cancer of the semicolon. Recall the conditional expressions if and cond. (if <expre> <expre> <expre>) (cond [<expre> <expr>] [<expre> <expr>] ... [<expre> <expr>]) This is not the most general form of cond, but is a common form. The first expression in the last cond clause is usually else. We have seen the predicate = to test equality of numbers. The predicate equal? tests the equality of general values, not just numbers. Thus, the expression (= 1.0 #t) will give an error message because the second expression is a Boolean value, not a number, but (equal? 1.0 #t) will return #f. Additional Boolean expressions: not, and, or. The built-in procedure (not <expr>) evaluates its argument and returns #t if its argument is #f, and returns #f if its argument is not #f. Thus, it behaves as expected on Boolean values, but has some additional properties, as follows. (not #f) => #t (not #t) => #f (not 13) => #f (not (not 13)) => #t The keyword and signals a special form with the following syntax and evaluation behavior. (and <expr> <expr> ... <expr>) It takes zero or more expressions, and evaluates them in order, left to right. If one of them evaluates to #f, then no further expressions are evaluated, and the value of the and expression is #f. If none of them evaluates to #f, then the value of the last one is returned as the value of the and expression. This produces the expected behavior on Boolean expressions, and some additional behavior on non-Boolean expression. Examples follow. (and #t #t) => #t (and #f #t) => #f (and (> 3 1) (= (* 2 2) 4)) => #t (and 1 2) => 2 Note that not all of the arguments of and will be evaluated if an earlier one evaluates to #f. The keyword or signals a special form with the following syntax and evaluation behavior. (or <expr> <expr> ... <expr>) There are zero or more expressions. They are evaluated in left-to-right order until the first one that evaluates to a non-#f value -- that non-#f value is returned as the value of the whole or expression, and no further expressions in the or are evaluated. If all of the expressions evaluate to #f, then the value of the or expression is #f. Similarly to and, this produces the expected behavior for Boolean values, and some additional behavior for non-Boolean values. (or #t #t) => #t (or #f #t) => #t (or (> 1 3) (= 4 (+ 1 2))) => #f (or 1 2) => 1 Note that there may be no expressions after the keywords and, or. In that case, what is returned is the identity element for the operation, which in the case of or is #f, and in the case of and is #t. (Because (x or false) is equivalent to x, and (x and true) is equivalent to x.) Thus, we have (and) => #t (or) => #f A recursive data type: lists. A list is a finite sequence of values. Note that "sequence" means that order is important, and we may have duplicate values in the list. We'll first introduce some constant lists, which are preceded by a single quote. '() the empty, or null, list, with no elements '(18) a list with one element, the number 18 '(18 22) a list with two elements, the number 18 followed by the number 22 A list may have elements of different types, for example: '(13 #f "hi") a list with three elements, the number 13, followed by the Boolean value #f, followed by the string "hi" Lists can contain other lists as elements. '((18 22) (22 26) (26 30)) This is a list of three elements, the first being a list containing the numbers 18 and 22, the second being a list containing the numbers 22 and 26, and the third being a list containing the numbers 26 and 30. Type predicates. The built-in predicate (null? <expr>) evaluates the expression and returns #t if it evaluates to the empty list, and returns #f otherwise. It is equivalent to (equal? <expr> '()). The predicate (list? <expr>) evaluates the expression and returns #t if it is a list, and #f otherwise. Given a data type, we are interested in its selectors and constructors. The selectors allow us to access parts of an object of the given type, and the constructors allow us to construct an object of the given type out of its parts. The selectors for a list are the following. (car lst) returns the value of the first element of the list lst (cdr lst) returns a list equal to lst with its first element removed Note that the procedure (first lst) is equivalent to (car lst), and the procedure (rest lst) is equivalent to (cdr lst). Both of these will return an error message if lst is the empty list, so make sure that lst is non-empty before you apply car or cdr. Examples. > (define x '(18 22 26)) > x '(18 22 26) > (car x) 18 > (first x) 18 > (cdr x) '(22 26) > (rest x) '(22 26) > (car (cdr x)) 22 > (cdr (cdr x)) '(26) > (cdr (cdr (cdr x))) '() > (car '((18 22) (22 26) (26 30))) '(18 22) > (cdr '((18 22) (22 26) (26 30))) '((22 26) (26 30)) > Note that car and cdr are not "symmetrical" -- car extracts the first element of the list, while cdr returns a list containing the rest of the elements of the list, with the first element excluded. There are built-in abbreviations (up to about 5 levels) for compositions of car and cdr. That is, (car (car x)) can be abbreviated (caar x), and (car (cdr x)) can be abbreviated (cadr x). Thus, (caddr x) is an abbreviation of (car (cdr (cdr x))). Constructing lists. The basic list constructor is (cons item lst). This returns a list equal to lst with item added as the first element. Examples. > (cons 26 '()) '(26) > (cons 22 (cons 26 '())) '(22 26) > (cons 18 (cons 22 (cons 26 '()))) '(18 22 26) > (cons '(12 13) '((13 14) (14 15))) '((12 13) (13 14) (14 15)) > More about constructors will be covered in the next lecture. There is a built-in procedure (length lst) that returns the number of (top-level) elements in a list. Examples. (length '()) => 0 (length '(18 22 26)) => 3 (length '((12 13) (13 14) (14 15))) => 3 We now write our own version of length using flat recursion on lists. The base case is the empty list -- we know that in this case, the answer should be 0. Hence, we have the following partial procedure. (define (our-length lst) (if (null? lst) 0 ????)) Now we need to figure out what to do for the recursive case. The expression (cdr lst) will be the input list with the first element removed, and if we can find the length of that, then to get the length of the input list, we merely have to add 1. Thus, we can complete our procedure as follows. (define (our-length lst) (if (null? lst) 0 (+ 1 (our-length (cdr lst))))) This is a very common pattern for flat (or top-level) recursion on lists: the base case is the empty list, and the recursive call is with the rest of the list, that is, (cdr lst).
1/22/14	Lecture 4: Racket III. Perlis epigram #17: If a listener nods his head when you're explaining your program, wake him up. Recursive procedures. You've probably already used recursion in your programming experiences to date; we'll be using it quite heavily. A classic recursive (or inductive) definition of a function from mathematics is the factorial procedure (n!) defined on positive integers as follows: n! = 1 if n = 1 n * (n-1)! if n > 1 The definition is divided into two cases. In the base case, when n = 1, we can just return the correct value (1) for the function without any further uses of the factorial function. In the recursive case, when n > 1, we first calculate the value of the function for (n-1), that is, (n-1)!, and then multiply the result by n to get the value of the function for n. Using this definition to calculate 4!, we have 4! = 4 * 3! = 4 * 3 * 2! = 4 * 3 * 2 * 1! = 4 * 3 * 2 * 1 = 24 To mirror this logic in a Racket procedure, we'll need a conditional expression -- something that allows us to test a condition and do one thing if it is true, and something else if it is false. Boolean values and predicates. We've already seen the Boolean constants #t (for true) and #f (for false). A predicate is a procedure that returns #t or #f. Some built-in predicates allow us to compare numbers: =, <, >, <=, >=, (equal, greater than, less than, less than or equal to, greater than or equal to, respectively.) Examples: > (= 4 (+ 3 1)) #t > (> 5 7) #f > (< -3 -4) #f > (>= 4 (* 2 2)) #t > (<= 4 (* 2 2)) #t > Two more built-in predicates are even? and odd?, which test whether an integer is even or odd. Note that the names of many predicates end in ?. The special form if. The special form if allows us to test a condition and evaluate one expression if it is not #f, and evaluate a different expression if it is #f. The syntax in the Racket Guide is as follows. (if <expr> <expr> <expr>) The keyword is if, which is followed by three expressions. The first expression is evaluated. If its value is not #f, then the second expression is evaluated and its value is the value of the whole if expression. (In this case, the third expression is not evaluated at all.) If the value of the first expression is #f, then the third expression is evaluated and its value is the value of the whole if expression. (In this case, the second expression is not evaluated at all.) Examples: > (if (> 2 1) "greater" "not greater") "greater" > (if (odd? 4) -1 1) 1 > Now we have the means of writing a procedure to compute the factorial of a positive integer n. (define (factorial n) (if (= n 1) 1 (* n (factorial (- n 1))))) To get a sense of what this procedure is doing, we can draw a tree of the recursive calls of factorial for the call (factorial 4). Computer scientist's trees are upside down, with the root at the top and the leaves at the bottom. We put the top-level call at the root. (factorial 4) / \| \ * 4 (factorial 3) / \| \ * 3 (factorial 2) / \| \ * 2 (factorial 1) This indicates that the call (factorial 4) invoked the call (factorial 3) and is waiting for the result to be returned so that it can be multiplied by 4 to get the value of 4!. In turn, the call (factorial 3) invoked the call (factorial 2) and is waiting for the result to be returned so that it can be multiplied by 3 to get the value of 3! Similarly, the call (factorial 2) invoked the call (factorial 1) and is waiting for the result to be returned so that it can be multiplied by 2 to get the value of 2!. The case n = 1 is the base case -- no recursive call is made; instead, the correct value, 1, is simply returned. We can then propagate the returned values "up" the tree (from leaf to root), multiplying by the appropriate values. This may be pictured as follows. (factorial 4) => 24 / \| \ * 4 (factorial 3) => 6 / \| \ * 3 (factorial 2) => 2 / \| \ * 2 (factorial 1) => 1 One common type of error in writing recursive procedures is to make the recursive call on the same value as the top-level call, instead of on a value "closer" to the base case. In the case of our factorial procedure, this would look as follows. (define (bad-fact1 n) (if (= n 1) 1 (* n (bad-fact1 n)))) Here the recursive call is with n instead of (n-1). If we consider the tree of recursive calls for (bad-fact1 4), we get the following. (bad-fact1 4) / \| \ * 4 (bad-fact1 4) / \| \ * 4 (bad-fact1 4) / \| \ * 4 (bad-fact1 4) / \| \ * 4 .... The argument, 4, is unchanged, so we end up calling (bad-fact1 4) over and over and over ... until we run out of stack space or manually stop it with the stop button. Remember that the recursive call(s) should be moving the argument(s) closer to the base case(s). Another common type of error in writing recursive procedures is to omit a necessary base case. For example, we might write the following. (define (bad-fact2 n) (* n (bad-fact2 (- n 1)))) Here we haven't forgotten to change the argument in the recursive call, but we have forgotten to put in a test to make the recursive calls stop when we get to 1. The tree of recursive calls of (bad-fact2 3) is as follows. (bad-fact2 3) / \| \ * 3 (bad-fact2 2) / \| \ * 2 (bad-fact2 1) / \| \ * 1 (bad-fact2 0) / \| \ * 0 (bad-fact2 -1) / \| \ * -1 .... The arguments are decreasing: 3, 2, 1, 0, -1, -2, ..., but there is no condition to stop this process. This procedure will continue until it uses up the stack space, or we manually stop it. (Note that we would get a similar problem if we called the original procedure on an argument outside its specified domain, for example (factorial 0).) To get a better understanding of these behaviors, you can run the "bad" procedures under the control of the DrRacket debugger. Another recursive procedure: first-digit. Suppose we want to write a procedure (first-digit n) that takes a non-negative integer n and returns a number that is the first decimal digit of n. For example (first-digit 3) => 3 (first-digit 437) => 4 A good base case is numbers less than 10, because they already consist of one digit. So we can write (define (first-digit n) (if (< n 10) n ????)) where the ???? is to be replaced by a recursive call to first-digit. What will bring us closer to a single digit number? We could subtract 1, but the first digits of n and (n-1) don't seem to be usefully related. If we take the quotient of n and 10, we get a number with one fewer digits and the same first digit. For example, (quotient 437 10) => 43, and 43 has only two digits, and the same first digit as 437. Now we can complete the recursive case as follows. (define (first-digit n) (if (< n 10) n (first-digit (quotient n 10)))) To illustrate this procedure, we consider the tree of recursive calls for (first-digit 437). (first-digit 437) => 4 \| (first-digit 43) => 4 \| (first-digit 4) => 4 The special form cond. As an alternative to nested ifs, we can use the special form cond. We write two versions of a procedure (classify n) that takes a number n and returns "positive", "negative", or "zero" according to the value of n. A version with nested ifs is as follows. (define (classify n) (if (< n 0) "negative" (if (= n 0) "zero" "positive"))) Because the if expressions are nested, it is difficult to verify the conditions under which each string will be returned. Instead we can use the special form cond as follows. (define (classify n) (cond [(< n 0) "negative"] [(= n 0) "zero"] [else "positive"])) This makes it easier for a human to understand the conditions under which each string will be returned by classify. The general syntax of cond from the Racket Guide is as follows. (cond {[<expr> <expr>]}) The whole cond expression is enclosed in left parenthesis ( and right parenthesis ). The keyword is cond. Then there are zero or more clauses, each delimited by left square bracket [, and right square bracket ]. Each clause consists of one or more expressions. Note that the curly braces { and } are not literally part of the cond expression, but are used to indicate the scope of the second *. The evaluation of a cond expression proceeds as follows. In the first clause, the first expression (the condition) is evaluated. If it is not #f, then the remaining expressions in that clause are evaluated, in order, and the value of the last one is the value of the cond expression. If the first expression (the condition) in the first clause evaluates to #f, then the rest of the first clause is skipped, and the first expression (the condition) in the second clause is evaluated. If the condition in the second clause is not #f, then the remaining expressions in the second clause are evaluated, in order, and the value of the last one is the value of the cond expression. If the condition in the second clause evaluates to #f, then we proceed to the third clause, and so on. Note that if the condition expression is else, this is treated as a non-#f value, and the remaining expressions in the clause are evaluated, in order, and the value of the last one is the value of the cond expression. If there is no else clause, and evaluation "runs off the end" of the cond expression, then the value returned is the void value, which is not printed by the REPL loop. It is a good idea to include an else clause in your cond expressions.
1/17/14	Lecture 3: Racket II. Perlis epigram #17: To understand a program you must become both the machine and the program. The special form define. In the Racket Guide, the first syntax for define is shown as: (define <id> <expr>) What this indicates is that the keyword is define, the first argument is an identifier, and the second argument is an arbitrary expression. As an example: > (define age (- 2014 1996)) > Rule (4): define evaluates its second argument (the expression) and binds its first argument (the identifier) to that value. In this case, the identifier age is bound to the value of the expression (- 2014 1996), which is 18. What does "binding" mean? It is an association between an identifier and a value in some environment. When you first start DrRacket, your current environment is the "top-level environment", which contains bindings for all the built-in procedures (eg, +, , quotient, remainder). You can think of it as a table listing all the initially defined identifiers and their values. Part of the initial top-level environment can be pictured as follows. top-level environment --------------------- \| ... \| ----------------------------------------------------- \| + \| the built-in addition procedure \| ----------------------------------------------------- \| \| the built-in multiplication procedure \| ----------------------------------------------------- \| quotient \| the built-in quotient procedure \| ----------------------------------------------------- After evaluating (define age (- 2014 1996)), the identifier age is added to the top-level environment, with the value 18. The top-level environment then looks like the following. top-level environment --------------------- \| ... \| ----------------------------------------------------- \| + \| the built-in addition procedure \| ----------------------------------------------------- \| * \| the built-in multiplication procedure \| ----------------------------------------------------- \| quotient \| the built-in quotient procedure \| ----------------------------------------------------- \| age \| the number 18 \| ----------------------------------------------------- Notice that each value has a type, indicating whether it is a number, a string, a boolean, a procedure, or some other type. Types in Racket are "dynamic" in the sense that checking that the types expected by procedures are the ones supplied by actual arguments is done when the procedures are being evaluated. By contrast, "static" types are checked at compile time -- the functional languages ML and Haskell have static types, which means that type errors will be detected at compile time rather than run time. Racket has predicates to test whether an expression is of a given type. A predicate is a procedure or function that returns a Boolean value, that is, either #t or #f. It is a convention in Racket to end the name of a predicate with a question mark; the type predicates for numbers, strings, booleans and procedures are number?, string?, boolean? and procedure? As an example: > (number? "hi!") #f > (number? 19.2) #t > The idea of binding a value to an identifier leads to the next rule of evaluation. Rule (5): the value of an identifier is found by looking it up in the "relevant" environment. For now, the "relevant" environment is the top-level environment; this understanding will be refined as the course progresses. Having added the identifier age to the top-level environment we may now evaluate it. > age 18 > We may also use it in expressions. > (+ age 4) 22 > In fact, when the expression (+ age 4) is evaluated, the rule for identifiers is used to find the value of the identifier + as well as the identifier age. The identifier + is looked up in the top-level environment, where its value is found to be the built-in procedure to add numbers. This is all very well, but we'd like to be able to write our own procedures, not just use the built-in ones. Creating new procedures is done by means of the special form lambda. (Aside on the typographic history of lambda in the lambda calculus.) For example: (lambda (n) (* n n)) ------ --- ------- (1) (2) (3) The keyword (1) is lambda, the formal arguments (2) in this case are just the identifier n, and the body (3) in this case is the expression (* n n). The formal arguments specify names for the inputs to the procedure, and the body gives one or more expressions to evaluate to find the value of the procedure when it is applied. Rule (6): Evaluating a lambda expression creates a new procedure. That procedure is the value of the lambda expression. As an example: > (lambda (n) (* n n)) #<procedure> > We created a procedure, but we neither named nor applied it -- a bit underwhelming. We can apply a procedure without naming it: > ((lambda (n) (* n n)) 7) 49 > What happened here? The outer pair of parentheses (because the left one is not followed by a keyword) indicate a procedure application. In a procedure application, Racket evaluates the expressions between the parentheses in order, first (lambda (n) (* n n)) and then 7. Evaluating (lambda (n) (* n n)) creates a procedure to square a number, and 7 evaluates to the number 7, so the application calls the procedure on the actual argument 7. The procedure returns 49, which is the value of the application. Sometimes we only want to use a procedure once, and don't need to name it. Most of the time, we want to call a procedure repeatedly on different arguments, and we'd like to give it a name. For that, we just combine define and lambda, to create a procedure and give it a name. For example: > (define square (lambda (n) (* n n))) > (square 7) 49 > (square 4) 16 > What happened here? The evaluation of define adds the identifier square to the top-level environment, and binds it to the value of the lambda expression (lambda (n) (* n n)), which is a procedure to square a number. After the define has been evaluated, the initial top-level environment is modified to look like the following. top-level environment --------------------- \| ... \| ----------------------------------------------------- \| + \| the built-in addition procedure \| ----------------------------------------------------- \| * \| the built-in multiplication procedure \| ----------------------------------------------------- \| quotient \| the built-in quotient procedure \| ----------------------------------------------------- \| square \| procedure, formals (n), body (* n n) \| ----------------------------------------------------- Notice that the identifier square has been bound to a procedure with formal arguments (n) and body (* n n). Now when we evaluate (square 7), this is an application (because square is not a keyword), and the identifier square is looked up in the top-level environment, and its value is found to be a procedure with formal arguments (n) and body (* n n). The expression 7 evaluates to the number 7, and then the procedure is applied to the actual argument, the number 7. In more detail, a new "local" environment is set up in which each identifier from the formal arguments is bound to the corresponding value of the actual argument. This local environment will have a "search pointer", which in this case will point to the top-level environment. After the local environment is set up, the environment picture is as follows. top-level environment --------------------- \| ... \| ----------------------------------------------------- \| + \| the built-in addition procedure \| ----------------------------------------------------- \| * \| the built-in multiplication procedure \| ----------------------------------------------------- \| quotient \| the built-in quotient procedure \| ----------------------------------------------------- \| square \| procedure, formals (n), body (* n n) \| ----------------------------------------------------- local environment, search pointer: top-level environment -------------------------------------------------------- ------------------------ \| n \| the number 7 \| ------------------------ Then the body of the procedure, (* n n) is evaluated in the local environment. This expression is an application (because * is not a keyword), so the expressions , n, and n are evaluated. To evaluate the identifier , we look in the current environment (which is the local environment) and don't find it there. We then follow the search pointer to the top-level environment, and find a binding for * there, namely, the built-in multiplication procedure. To evaluate n (twice), we look in the current environment (which is the local environment) and find a binding for n, namely, the number 7. We then call the built-in multiplication procedure on the arguments 7 and 7, and it returns the number 49, which the the value of the expression (* n n) in the local environment, which is also the value of the procedure application (square 7). Once the procedure application finishes evaluation, the local environment that was created is no longer accessible and can be "garbage-collected" (or "recycled"), resulting in an environment picture as follows. top-level environment --------------------- \| ... \| ----------------------------------------------------- \| + \| the built-in addition procedure \| ----------------------------------------------------- \| * \| the built-in multiplication procedure \| ----------------------------------------------------- \| quotient \| the built-in quotient procedure \| ----------------------------------------------------- \| square \| procedure, formals (n), body (* n n) \| ----------------------------------------------------- There is an alternate, more concise, syntax for defining and naming a procedure. Instead of writing the procedure square as above, we could just write the following. > (define (square n) (* n n)) > This saves some typing, but be aware that this abbreviated syntax is translated into the syntax with lambda that we gave above. With these tools available, we may write our own procedures. For example, we can write a procedure (last n) that returns the last decimal digit of a non-negative integer n. We introduce the notation of a "thick" arrow (=>) to stand for "evaluates to". Thus, we would like (last 144) => 4 and (last 37) => 7. Taking n modulo 10, or the remainder of n divided by 10 (they are the same when n is a non-negative integer) seems to accomplish what we want. For example, dividing 37 by 10 gives a quotient of 3 and a remainder of 7. We write (using the lambda syntax) and test our procedure as follows. > (define last (lambda (n) (remainder n 10))) > (last 37) 7 > (last 144) 4 > The last part of the lecture was devoted to a DrRacket demo in which we entered the following definitions in the definitions area (the upper window): (define square (lambda (n) (* n n))) (define (cube num) (* num (square num))) and pressed the "Run" button. This causes all of the expressions in the definitions area to be evaluated, which adds the identifiers square and cube to the top-level environment with the indicated procedures as their bindings. Then in the interactions window we tested both square and cube to see that they were working. We then introduced an error into the square procedure, replacing (* n n) with the incorrect expression (n * n) and pressed "Run" again to update the bindings of square and cube. Note that no error is detected at this point. However, when in the interactions area we attempt to evaluate (square 4) we get an error message: procedure application: expected procedure, given: 4; arguments were: #<procedure:> 4 The expression whose evaluation provoked the error is highlighted in pink. Clicking on the "stack" of red/white x's to the left of the message opens a "Backtrace" window showing how the error traces back through previous applications. This is somewhat more interesting if we attempt to evaluate (cube 3), and get the error message: procedure application: expected procedure, given: 3; arguments were: #<procedure:> 3 The expression that provoked the error is again highlighted, and now has an arrow pointing to the expression whose evaluation caused the erroneous expression to be evaluated. Clicking open the Backtrace window, we see the two expressions, with an indication of where they occur. We also demonstrated a little of the behavior of DrRacket when the "Debug" button is pressed. Among other capabilities, this allows the evaluation of the program to proceed by "single steps." It is worth experimenting with.
1/15/14	Lecture 2: Racket I. Perlis epigram #26: There will always be things we wish to say in our programs that in all known languages can only be said poorly. This term we'll be programming in Racket, which is an offshoot of Scheme, which is an offshoot of LISP, which was one of the original higher-level programming languages, devised by John McCarthy for writing programs for artificial intelligence. The goal of this part of the course is to install a tiny interpreter for the core of Racket in your heads. The aim is a very thorough understanding of the core of the language, rather than a looser grasp of a wider range of features. In DrRacket, there are two main windows, an upper "definitions area" and a lower "interaction area." The interaction area functions as a Read-Eval(uate)-Print-Loop (or REPL), in which you type an expression which is read and evaluated by the Racket interpreter, which then prints out the resulting value. You are then prompted for another expression to evaluate. We'll explain the evaluation process by means of rules illustrated by examples. Rule (1): constants evaluate to themselves. In Racket there are several types of constants; for now we'll be content to think about three types: numbers, strings, and Booleans. As examples, we have: > 18 18 > -1 -1 > "hi there!" "hi there!" > #t #t > #f #f > Numbers are actually a rather complex type -- we'll mostly be interested in integers (positive, negative, and zero) for now. The Boolean true is #t, the Boolean false is #f. Rule (2): A procedure application is enclosed in parentheses, with the procedure first, followed by its arguments; it is evaluated by finding the values of the arguments, calling the procedure on those values, and returning the value that the procedure returns as the value of the whole expression. Other names for a procedure application are procedure call or function call. In Racket, parentheses are used rather differently from how they are used in mathematics; you will need to be alert to the differences. There are a number of built-in procedures for operations on numbers: + for addition, - for subtraction, * for multiplication, / and quotient for division, remainder and modulo for remainder. As one example, we have: > (+ 18 4) 22 > The left parenthesis is not followed by a keyword, so this is a procedure application. The arguments, 18 and 4, are evaluated (to themselves, using rule (1)), and the built-in procedure to add numbers is called on 18 and 4. That procedure returns the value 22, which is the value of the expression (+ 18 4). Note that the procedure and its arguments are separated by whitespace. Additional examples with other procedures: > (- 3 4) -1 > (* 3 4) 12 > (quotient 11 4) 2 > (remainder 11 4) 3 > (modulo 11 4) 3 > Note that in each case, the order is: left parenthesis, procedure to apply, arguments in order, right parenthesis. If (or rather, when) you forget and put the operation in the middle, you will get a colorful red error message, for example, > (3 - 4) application: not a procedure; expected a procedure that can be applied to arguments given: 3 arguments...: #<procedure:-> 4 > The preamble "application:" signifies that this is an error message from the part of the interpreter that is intended to evaluate procedure applications. The error is that the first expression is not a procedure, it is the number 3. Values in Racket have types (though they are not explicitly declared), and a value of type number is not a procedure. The "arguments", that is, the expressions after the initial one, are the built-in subtraction procedure and the number 4. Try to suppress the urge to think of error messages as impenetrable -- with a bit of practice, they will make sense and be helpful. As an aside, we can evaluate the expression consisting of just a minus sign: > - #<procedure:-> > In fact it evaluates to the built-in procedure to subtract numbers, which the interpreter represents as indicated. The division operation / will return an exact rational number (a reduced fraction) if its arguments are integers, and a number with a decimal point if at least one of its arguments is a number with a decimal point. The quotient operation returns the integer quotient of its arguments. To figure out what (quotient 11 4) is, we perform "long division" of 4 into 11, getting a quotient and remainder: 2 ----- 4 ) 11 8 --- 3 The quotient is 2, so (quotient 11 4) evaluates to 2, and the remainder is 3, so (remainder 11 4) evaluates to 3. When the arguments are positive integers, remainder and modulo return the same values. However, when the first argument (the dividend) is negative and the second argument (the divisor) is positive, we get different results: > (quotient -11 4) -2 > (remainder -11 4) -3 > (modulo -11 4) 1 > To understand a bit more about the relationship between the Racket operations remainder and modulo, we consider the theorem called "The Division Algorithm". That theorem says that if a is an integer and b is a positive integer, then there are unique integers q and r such that a = bq + r and 0 <= r < b. When a is non-negative, q is the value returned by (quotient a b) and r is the value returned by (remainder a b) or (modulo a b). However, when a is negative, the numbers returned by (quotient a b) and (remainder a b) are NOT the numbers q and r in this theorem, even though multiplying (quotient a b) by b and adding (remainder a b) does return a -- the second condition is not satisfied. However, (modulo a b) does return the value of r in the theorem. Concretely, we have -11 = 4(-2) + -3 -11 = 4(-3) + 1 The Division Theorem applied to a = -11 and b = 4 gives q = -3 and r = 1. Rule (3): The evaluation rules apply recursively. In particular, the arguments in a procedure application can themselves be procedure applications. Example: > (+ ( 3 6) 4) 22 > Here we have a procedure application with procedure +, first argument (* 3 6), and second argument 4. To find the value of the first argument, we must evaluate the procedure application (* 3 6), which we do by evaluating its arguments, which are the numbers 3 and 6 (by rule (1)), calling the built-in multiplication procedure on these arguments, which returns the number 18, which is the value of the first argument; the second argument evaluates to 4 (by rule (1)), and then we call the built-in addition procedure on the arguments 18 and 4. It returns the value 22, which is the value of the whole expression. Next we'll consider our first "special form": define. A special form is a Racket operation that does not obey the usual rules for the evaluation of a procedure application. The define operation may remind you of assignment in Java or other imperative programming languages. However, it will be used quite differently, primarily just to give names to constants and procedures. An example: > (define age (- 2014 1996)) > age 18 > The special form define expects an identifier as the first argument, and an arbitrary Racket expression as the second argument. The second argument is evaluated, and the identifier is bound to the resulting value. In this case, the expression (- 2014 1996) is evaluated to give the number 18, and the identifier age is bound to the number 18. After that, we can find the value bound to the identifier age by evaluating it. If we try to evaluate an undefined identifier, we get an error message: > x x: undefined; cannot reference an identifier before its definition > which is self-explanatory. We'll continue with the rules for evaluation in the next lecture
1/13/14	Lecture 1: Introductory Lecture. Discussion of the syllabus, other computer science courses to consider and the overall structure of the course. If you're thinking of taking CPSC 201, please do the following. Sign up for a course account for CPSC 201 on the Zoo: Course account signup. Your course account should be created within one hour of signing up. Start the reading assignment from the Assignments portion of the course web page. Familiarize yourself with the Zoo computers by attending a (soon-to-be-scheduled) Zoo help session or by using the Zoo tutorial available from the course web page. (Optional.) Download and install DrRacket on your computer (Racket) and familiarize yourself with it (Choose language: "Use the language declared in the source", #lang racket. Then press the Run button to actually change the language.) If you don't want to install and run DrRacket on your machine, you may run it in person or remotely using your Zoo account. Perlis epigram #19: A language that doesn't affect the way you think about programming is not worth knowing. The language we'll use this term, Racket, is an offshoot of Scheme, which is an offshoot of LISP, which was designed as a higher-level language for writing artificial intelligence programs. We choose Racket because it is high-level, functional (as opposed to imperative), emphasizes recursion, and is unfamiliar to almost all students taking CPSC 201. It is definitely not widely used in industry. One goal is for you to learn new tools and paradigms for programming, to "affect the way you think about programming." It will be frustrating at times -- no assignment statements, no obvious analogs of for and while statements -- but it will expand your view of what programming can be. We concluded by briefly glancing at the world's oldest algorithms textbook, the Rhind or Ahmose papyrus, which was written about 4,000 years ago. The scroll is essentially a textbook of specific solved problems illustrating methods of handling different kinds of calculations. One interesting feature of the Egyptian mathematics of the time was that fractions were represented as sums of distinct "unit fractions", that is, a fraction was represented as 1/n1 + 1/n2 + ... + 1/nk, where n1, n2, ..., nk are pairwise distinct positive integers. For example, 5/8 could be represented as 1/2 + 1/8. What about 2/7? One explanation that has been given for why fractions were represented this way has to do with making the division of goods (eg, loaves of bread) equal in appearance. If we were trying to divide 2 loaves of bread among 7 laborers, so that each laborer gets 2/7 of a loaf, we might first divide the 2 loaves into 8 equal pieces, giving one to each laborer, with one piece left over. That one leftover piece could then be divided into 7 equal portions and distributed among the laborers. Each laborer gets two pieces: 1/4 of a loaf and 1/28 of a loaf. That is, we have written the fraction 2/7 as 1/4 + 1/28, a sum of unit fractions chosen so that each laborer gets 2/7 of a loaf. (Note: in class we incorrectly thought that the sum was 1/8 + 1/56, forgetting that each of the eight original pieces was 1/4 of a loaf, not 1/8 of a loaf.) Some questions arise: can we write every (proper) fraction this way? Can some fractions be written in two or more different ways as sums of distinct unit fractions? Is there an algorithm to write a fraction as a sum of distinct unit fractions?

[Home]

Last modified: January 11, 2017