{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## CS 200: Iterators and Generators\n", "\n", "\n", "\n", "\n", "\n", "Reading: \n", "- Learning Python, Chapter 14\n", "- Python Cookbook, Chapter 4\n", "\n", "The concept of “iterable objects” is relatively recent in Python,\n", "but it has come to permeate the language’s design. It’s essentially a\n", "generalization of the notion of sequences—an object is considered\n", "iterable if it is either a physically stored sequence, or an object\n", "that produces one result at a time in the context of an iteration tool\n", "like a for loop. In a sense, iterable objects include both physical\n", "sequences and virtual sequences computed on demand.\n", "\n", "Below we iterate over a file implicitly using a for loop:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "def it0(file=\"iterators.py\"):\n", " count = 0\n", " for line in open(file):\n", " count += 1\n", " if count < 10:\n", " print (line, end='')" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "#! /usr/bin/python3\n", "\n", "'''\n", "From Learning Python, Chapter 14\n", "\n", "The concept of “iterable objects” is relatively recent in Python,\n", "but it has come to permeate the language’s design. It’s essentially a\n", "generalization of the notion of sequences—an object is considered\n", "iterable if it is either a physically stored sequence, or an object\n" ] } ], "source": [ "it0()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we iterate explicitly using next and readline. Note that print() adds a newline." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def it1(file=\"iterators.py\"):\n", " f = open(file)\n", " print(f.readline())\n", " print(f.readline())\n", " print(f.__next__())\n", " print(f.__next__())\n", " print(next(f))\n", " print(next(f))\n", " f.close()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "#! /usr/bin/python3\n", "\n", "\n", "\n", "'''\n", "\n", "From Learning Python, Chapter 14\n", "\n", "\n", "\n", "The concept of “iterable objects” is relatively recent in Python,\n", "\n" ] } ], "source": [ "it1()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " \\_\\_next\\_\\_() and readline() are equivalent for files.\n", " They each iterate over the file a line at a time.\n", "\n", "\\_\\_next\\_\\_() is an ITERATOR in Python.\n", "The for loop calls \\_\\_next\\_\\_()\n", "\n", "next(f)calls the iterator for f\n", "\n", "Can explicitly get (or make) an iterator using iter()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "def it2(lst = [1,2,3,4]):\n", " it = iter(lst)\n", " print (it.__next__())\n", " print (next(it))\n", " print (next(it))" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1\n", "2\n", "3\n" ] } ], "source": [ "it2()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1\n", "2\n", "3\n" ] } ], "source": [ "it2([1,2,3,4,5])" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1\n", "2\n" ] }, { "ename": "StopIteration", "evalue": "", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mStopIteration\u001b[0m Traceback (most recent call last)", "Input \u001b[0;32mIn [8]\u001b[0m, in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0m it2([\u001b[38;5;241m1\u001b[39m,\u001b[38;5;241m2\u001b[39m])\n", "Input \u001b[0;32mIn [5]\u001b[0m, in \u001b[0;36mit2\u001b[0;34m(lst)\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28mprint\u001b[39m (it\u001b[38;5;241m.\u001b[39m\u001b[38;5;21m__next__\u001b[39m())\n\u001b[1;32m 4\u001b[0m \u001b[38;5;28mprint\u001b[39m (\u001b[38;5;28mnext\u001b[39m(it))\n\u001b[0;32m----> 5\u001b[0m \u001b[38;5;28mprint\u001b[39m (\u001b[38;5;28;43mnext\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mit\u001b[49m\u001b[43m)\u001b[49m)\n", "\u001b[0;31mStopIteration\u001b[0m: " ] } ], "source": [ "it2([1,2])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Below we manually implement iteration." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "def it4(lst = [1,2,3,4]):\n", " it = iter(lst)\n", " while True:\n", " try:\n", " x = next(it)\n", " except StopIteration:\n", " break\n", " print (x, ' ', end='')" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1 2 3 4 " ] } ], "source": [ "it4()" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1 2 3 4 5 6 7 8 " ] } ], "source": [ "it4([1,2,3,4,5,6,7,8])" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "it4([])" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "a b c d e f " ] } ], "source": [ "it4('abcdef')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Other built-in iterators include dictionaries." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "def it5(dict = {'a':1, 'b':2, 'c': 3}):\n", " for key in dict.keys():\n", " print (key, dict[key])" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "a 1\n", "b 2\n", "c 3\n" ] } ], "source": [ "it5()" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "it5({})" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "john 20\n", "mary 30\n", "joe 45\n", "anne 23\n" ] } ], "source": [ "it5({'john':20, 'mary':30, 'joe':45, 'anne':23})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Range objects are implicit lists.\n" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "def it6(n = 6):\n", " R = range(n)\n", " it = iter(R)\n", " print (next(it))\n", " print (next(it))\n", " print (list(it))" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0\n", "1\n", "[2, 3, 4, 5]\n" ] } ], "source": [ "it6()" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0\n" ] }, { "ename": "StopIteration", "evalue": "", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mStopIteration\u001b[0m Traceback (most recent call last)", "Input \u001b[0;32mIn [20]\u001b[0m, in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0m it6(\u001b[38;5;241m1\u001b[39m)\n", "Input \u001b[0;32mIn [18]\u001b[0m, in \u001b[0;36mit6\u001b[0;34m(n)\u001b[0m\n\u001b[1;32m 3\u001b[0m it \u001b[38;5;241m=\u001b[39m \u001b[38;5;28miter\u001b[39m(R)\n\u001b[1;32m 4\u001b[0m \u001b[38;5;28mprint\u001b[39m (\u001b[38;5;28mnext\u001b[39m(it))\n\u001b[0;32m----> 5\u001b[0m \u001b[38;5;28mprint\u001b[39m (\u001b[38;5;28;43mnext\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mit\u001b[49m\u001b[43m)\u001b[49m)\n\u001b[1;32m 6\u001b[0m \u001b[38;5;28mprint\u001b[39m (\u001b[38;5;28mlist\u001b[39m(it))\n", "\u001b[0;31mStopIteration\u001b[0m: " ] } ], "source": [ "it6(1)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0\n", "1\n", "[]\n" ] } ], "source": [ "it6(2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Enumerate objects are iterable. \n" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "def it7(text = \"hello, world!\"):\n", " it = iter(enumerate(text))\n", " print (next(it))\n", " print (next(it))\n", " print (list(it))" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(0, 'h')\n", "(1, 'e')\n", "[(2, 'l'), (3, 'l'), (4, 'o'), (5, ','), (6, ' '), (7, 'w'), (8, 'o'), (9, 'r'), (10, 'l'), (11, 'd'), (12, '!')]\n" ] } ], "source": [ "it7()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can iterate through the output of unix commands." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "import os\n", "def it8(cmd = 'ls -l'):\n", " p = os.popen(cmd)\n", " count = 0\n", " for x in p:\n", " count += 1\n", " if count < 10:\n", " print (x, end='')" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "total 31924\n", "-rw-r--r-- 1 sbs5 cs200ta 5619 Aug 31 12:46 0831.html\n", "-rw-rw-r-- 1 sbs5 sbs5 620327 Aug 1 18:57 0831nb.html\n", "-rw-rw-r-- 1 sbs5 cs200ta 33828 Aug 1 18:57 0831nb.ipynb\n", "-rw-rw-r-- 1 sbs5 cs200ta 1834 Aug 31 15:32 0831.script\n", "-rw-r--r-- 1 sbs5 cs200ta 4919 Oct 7 08:40 0902.html\n", "-rw-rw-r-- 1 sbs5 cs200ta 24567 Sep 2 15:38 0902.script\n", "-rw-r--r-- 1 sbs5 cs200ta 4567 Oct 7 08:40 0907.html\n", "-rw-rw-r-- 1 sbs5 cs200ta 27126 Sep 7 16:08 0907.script\n" ] } ], "source": [ "it8()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "map is an iterable." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "def it9(lst = [1,2,3,4,5,6,7,8]):\n", " x = map(lambda x: x*x, lst)\n", " print (x)\n", " for e in x:\n", " print (e)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "1\n", "4\n", "9\n", "16\n", "25\n", "36\n", "49\n", "64\n" ] } ], "source": [ "it9()" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "def it10(lst = [1,2,3,4,5,6,7,8]):\n", " x = map(lambda x: x*x, lst)\n", " try:\n", " while x:\n", " print (next(x))\n", " except:\n", " print ('done')" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1\n", "4\n", "9\n", "16\n", "25\n", "36\n", "49\n", "64\n", "done\n" ] } ], "source": [ "it10()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "filter is an iterable." ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [], "source": [ "def it11(lst = [1,2,3,4,5,6,7,8]):\n", " x = filter(lambda x: x%2, lst)\n", " print (x)\n", " for e in x:\n", " print (e)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "1\n", "3\n", "5\n", "7\n" ] } ], "source": [ "it11()" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "def it12(lst = [1,2,3,4,5,6,7,8]):\n", " x = filter(lambda x: x%2, lst)\n", " try:\n", " while x:\n", " print (next(x))\n", " except:\n", " print ('done')" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1\n", "3\n", "5\n", "7\n", "done\n" ] } ], "source": [ "it12()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### zip !\n", "\n", "zip is an iterable.\n", "\n", "zip combines two or more lists into a single list of tuples." ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [], "source": [ "z = zip([1,2,3], [4,5,6])" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "z" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[(1, 4), (2, 5), (3, 6)]" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list(z)" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[(1, 4, 7), (2, 5, 8), (3, 6, 9)]" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list(zip([1,2,3], [4,5,6], [7,8,9]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If the list arguments are not the same length, zip truncates the longer list." ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[(1, 5), (2, 6), (3, 7)]" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list(zip([1,2,3,4], [5,6,7]))" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[(1, 7, 1), (2, 6, 1), (3, 5, 1)]" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list(zip([1,2,3,4,5,6,7], [7,6,5,4,3,2,1], [1,1,1]))" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "def it13(lst1 = [1,2,3,4], lst2=[5,6,7,8]):\n", " x = zip(lst1, lst2)\n", " print(x)\n", " for a,b in x:\n", " print (a,b)" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "1 5\n", "2 6\n", "3 7\n", "4 8\n" ] } ], "source": [ "it13()" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [], "source": [ "def it14(lst1 = [1,2,3,4], lst2=[5,6,7,8]):\n", " x = zip(lst1, lst2)\n", " print(x)\n", " try:\n", " while x:\n", " print (next(x))\n", " except:\n", " print ('done')" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "(1, 5)\n", "(2, 6)\n", "(3, 7)\n", "(4, 8)\n", "done\n" ] } ], "source": [ "it14()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Synchronized vs independent iterators\n", "\n", "You may have multiple iterators over the same objects. Some are synchronized, and some are not.\n", "\n", "zip iterators are synchronized." ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [], "source": [ "def it15():\n", " z = zip([1,2,3], [4,5,6])\n", " i1 = iter(z)\n", " i2 = iter(z)\n", " print (next(i1))\n", " print (next(i1))\n", " print (next(i2))" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(1, 4)\n", "(2, 5)\n", "(3, 6)\n" ] } ], "source": [ "it15()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "map iterators are synchronized." ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [], "source": [ "def it16():\n", " z = map(lambda x: x*x, [1,2,3])\n", " i1 = iter(z)\n", " i2 = iter(z)\n", " print (next(i1))\n", " print (next(i1))\n", " print (next(i2))" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1\n", "4\n", "9\n" ] } ], "source": [ "it16()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Range iterators are independent!" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [], "source": [ "def it17():\n", " z = range(10)\n", " i1 = iter(z)\n", " i2 = iter(z)\n", " print (next(i1))\n", " print (next(i1))\n", " print (next(i2))" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0\n", "1\n", "0\n" ] } ], "source": [ "it17()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Class iteration.\n", "\n", "Simple iteration in a class" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [], "source": [ "class MyNumbers:\n", " def __iter__(self):\n", " self.a = 1\n", " return self\n", "\n", " def __next__(self):\n", " x = self.a\n", " self.a += 1\n", " return x" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1\n", "2\n", "3\n", "4\n", "5\n" ] } ], "source": [ "myclass = MyNumbers()\n", "myiter = iter(myclass)\n", "\n", "print(next(myiter))\n", "print(next(myiter))\n", "print(next(myiter))\n", "print(next(myiter))\n", "print(next(myiter))" ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "6" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "next(myiter)" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "7" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "next(myiter)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### iteration using deep recursion - actually deep iteration\n", "\n", "What would it mean to have an iterator for the person class?\n", "\n", "To iterate over a person's friends and family?" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [], "source": [ "class Node:\n", " def __init__(self, value):\n", " self.value = value\n", " self.children = []\n", "\n", " def __repr__(self):\n", " return 'Node({!r})'.format(self.value)\n", "\n", " def add_child(self, node):\n", " self.children.append(node)\n", "\n", " ## a depth first traversal of the tree\n", " def __iter__(self):\n", " print (\"Calling from: \" + str(self.value))\n", " # first, yield everthing every one of the child nodes would yield.\n", " for child in self.children:\n", " for item in child:\n", " # the two for loops is because there're multiple children,\n", " # and we need to iterate over each one.\n", " yield item\n", "\n", " # finally, yield self\n", " yield self" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [], "source": [ "def it18():\n", " root = Node(0)\n", " child1 = Node(1)\n", " child2 = Node(2)\n", " child3 = Node(3)\n", " child4 = Node(4)\n", " child5 = Node(5)\n", " child6 = Node(6)\n", " root.add_child(child1)\n", " root.add_child(child2)\n", " child1.add_child(child3)\n", " child3.add_child(child4)\n", " child4.add_child(child5)\n", " child5.add_child(child6)\n", "\n", " for ch in root:\n", " print (ch)" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Calling from: 0\n", "Calling from: 1\n", "Calling from: 3\n", "Calling from: 4\n", "Calling from: 5\n", "Calling from: 6\n", "Node(6)\n", "Node(5)\n", "Node(4)\n", "Node(3)\n", "Node(1)\n", "Calling from: 2\n", "Node(2)\n", "Node(0)\n" ] } ], "source": [ "it18()" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [], "source": [ "grandpa = Node('Grandpa')\n", "dad = Node('Dad')\n", "uncle = Node('Uncle')\n", "aunt = Node('Aunt')\n", "son = Node('son')\n", "daughter = Node('daughter')\n", "grandpa.add_child(dad)\n", "grandpa.add_child(uncle)\n", "grandpa.add_child(aunt)\n", "dad.add_child(son)\n", "dad.add_child(daughter)" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Calling from: Grandpa\n", "Calling from: Dad\n", "Calling from: son\n", "Node('son')\n", "Calling from: daughter\n", "Node('daughter')\n", "Node('Dad')\n", "Calling from: Uncle\n", "Node('Uncle')\n", "Calling from: Aunt\n", "Node('Aunt')\n", "Node('Grandpa')\n" ] } ], "source": [ "for offspring in grandpa:\n", " print (offspring)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Generators\n", "\n", "Define an iterator using yield:\n" ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [], "source": [ "class yrange:\n", " def __init__(self, n):\n", " self.i = 0\n", " self.n = n\n", "\n", " def __iter__(self):\n", " while self.i < self.n:\n", " i = self.i\n", " self.i += 1\n", " yield i\n", " else:\n", " ## raise StopIteration()\n", " pass" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [], "source": [ "def it19():\n", " print (list(yrange(5)))\n", " print (sum(yrange(5)))\n", " y = iter(yrange(3))\n", " print (next(y))\n", " print (next(y))\n", " print (next(y)) " ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[0, 1, 2, 3, 4]\n", "10\n", "0\n", "1\n", "2\n" ] } ], "source": [ "it19()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can use generators to create iterators." ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [], "source": [ "def zrange(n):\n", " i = 0\n", " while i < n:\n", " yield i\n", " i += 1" ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [], "source": [ "def it20():\n", " print (list(zrange(5)))\n", " print (sum(zrange(5)))\n", " z = zrange(3)\n", " print (next(z))\n", " print (next(z))\n", " print (next(z))" ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[0, 1, 2, 3, 4]\n", "10\n", "0\n", "1\n", "2\n" ] } ], "source": [ "it20()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How yield and next operate together:" ] }, { "cell_type": "code", "execution_count": 65, "metadata": {}, "outputs": [], "source": [ "def foo():\n", " print (\"begin\")\n", " for i in range(3):\n", " print (\"before yield\", i)\n", " yield i\n", " print (\"after yield\", i)\n", " print (\"end\")" ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [], "source": [ "def it23():\n", " f = foo()\n", " print (f)\n", " next(f)\n", " next(f)\n", " next(f)\n", " next(f)" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "begin\n", "before yield 0\n", "after yield 0\n", "before yield 1\n", "after yield 1\n", "before yield 2\n", "after yield 2\n", "end\n" ] }, { "ename": "StopIteration", "evalue": "", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mStopIteration\u001b[0m Traceback (most recent call last)", "Input \u001b[0;32mIn [67]\u001b[0m, in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0m it23()\n", "Input \u001b[0;32mIn [66]\u001b[0m, in \u001b[0;36mit23\u001b[0;34m()\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[38;5;28mnext\u001b[39m(f)\n\u001b[1;32m 6\u001b[0m \u001b[38;5;28mnext\u001b[39m(f)\n\u001b[0;32m----> 7\u001b[0m \u001b[38;5;28;43mnext\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mf\u001b[49m\u001b[43m)\u001b[49m\n", "\u001b[0;31mStopIteration\u001b[0m: " ] } ], "source": [ "it23()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Infinite data objects using generators" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [], "source": [ "def integers():\n", " \"\"\"Infinite sequence of integers.\"\"\"\n", " i = 1\n", " while True:\n", " yield i\n", " i = i + 1" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [], "source": [ "x = integers()" ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1 2 3 4 5 6 7 8 9 10 " ] } ], "source": [ "for i in range(10):\n", " print (next(x), end=' ')" ] }, { "cell_type": "code", "execution_count": 71, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "11 12 13 14 15 16 17 18 19 20 " ] } ], "source": [ "for i in range(10):\n", " print (next(x), end=' ')" ] }, { "cell_type": "code", "execution_count": 72, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "21 22 23 24 25 26 27 28 29 30 " ] } ], "source": [ "for i in range(10):\n", " print (next(x), end=' ')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Infinite squares." ] }, { "cell_type": "code", "execution_count": 73, "metadata": {}, "outputs": [], "source": [ "def squares():\n", " for i in integers():\n", " yield i * i" ] }, { "cell_type": "code", "execution_count": 74, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1 4 9 16 25 36 49 64 81 100 " ] } ], "source": [ "s = squares()\n", "\n", "for i in range(10):\n", " print (next(s), end=' ')" ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "121 144 169 196 225 256 289 324 361 400 " ] } ], "source": [ "for i in range(10):\n", " print (next(s), end=' ') " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "take will return the first n values from the given sequence." ] }, { "cell_type": "code", "execution_count": 76, "metadata": {}, "outputs": [], "source": [ "def take(n, seq):\n", " \"\"\"Returns first n values from the given sequence.\"\"\"\n", " seq = iter(seq)\n", " result = []\n", " try:\n", " for i in range(n):\n", " result.append(next(seq))\n", " except StopIteration:\n", " pass\n", " return result" ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]" ] }, "execution_count": 77, "metadata": {}, "output_type": "execute_result" } ], "source": [ "take(10, integers())" ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]" ] }, "execution_count": 78, "metadata": {}, "output_type": "execute_result" } ], "source": [ "take(10, squares())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Added finally clause to return value even if exception is thrown." ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [], "source": [ "def take2(n, seq):\n", " \"\"\"Returns first n values from the given sequence.\"\"\"\n", " seq = iter(seq)\n", " result = []\n", " try:\n", " for i in range(n):\n", " result.append(next(seq))\n", " except StopIteration:\n", " pass\n", " finally: \n", " return result" ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [], "source": [ "def it24():\n", " print (take(5,squares()))" ] }, { "cell_type": "code", "execution_count": 81, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[1, 4, 9, 16, 25]\n" ] } ], "source": [ "it24()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Can use comprehensions to create generators. Use () instead of []" ] }, { "cell_type": "code", "execution_count": 82, "metadata": {}, "outputs": [], "source": [ "g = (x*x for x in range(100))" ] }, { "cell_type": "code", "execution_count": 83, "metadata": {}, "outputs": [], "source": [ "def it25():\n", " print (take(10,g))" ] }, { "cell_type": "code", "execution_count": 84, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]\n" ] } ], "source": [ "it25()" ] }, { "cell_type": "code", "execution_count": 85, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[100, 121, 144, 169, 196, 225, 256, 289, 324, 361]\n" ] } ], "source": [ "it25()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our old prime number friends." ] }, { "cell_type": "code", "execution_count": 86, "metadata": {}, "outputs": [], "source": [ "noprimes = [j for i in range(2, 8) for j in range(i*2, 100, i)]\n", "yesprimes = (x for x in range(2, 100) if x not in noprimes)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note: does not work if noprimes is a generator." ] }, { "cell_type": "code", "execution_count": 87, "metadata": {}, "outputs": [], "source": [ "def it26():\n", " print (take(10,yesprimes))" ] }, { "cell_type": "code", "execution_count": 88, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]\n" ] } ], "source": [ "it26()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "End of Iterators and Generators notebook." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.7" } }, "nbformat": 4, "nbformat_minor": 4 }