Due 2:00 AM, Wednesday, 3 May 2017 (extension)
We can read the above text, even though it is missing word boundaries.
breaking up is hard to do
Your job in this assignment is to solve the string segmentation problem.
Words [-dict filename] [-debug]that reads standard input and writes to standard output. It first loads a word file (or dictionary) into a hash table, as you did in hw3. In fact, we will give you the hash table code from hw3 (hash.c, hash.h, and Makefile). The word file will have one word per line. When you load the words into the hash table, you will convert them to lower case and ignore any words containing non-alphabetic characters, such as digits or punctuation.
The default word file is simply "words" in the current directory. This should make it simpler to test your code without having to load 400,000 words from the online dictionary every time you run your code. However, your program should be able to process the online dictionary, if given as a command line argument.
All error output (usage and "Fatal Error" messages below) should be printed to standard error. For example,
fprintf(stderr, "usage ...");Note that not all errors are fatal. All other output should be printed to standard output. (Use normal printf.)
There are many ways to approach this problem. For starters, you might try to see if you can break the string into two words. You simply divide the string into a left and right substring, and check to see if both left and right parts are in the dictionary. We will call this the two word solution. This is pretty efficient, but also limited.
Another approach is depth first search. You find the first prefix that is a word, and then recursively try to segment the remainder of the string. This is a general solution, but not very efficient. For long strings, you will likely perform the same calculation repeatedly.
That should ring a bell in your head: dynamic programming. You want your program to avoid repetitive calculations.
The current solution, Wordsx, implements both the two word and the DP approaches, using the following DP algorithm:
$ cat words I a am ace mace ma mama for ever forever car rot carrot breaking break up is hard to do $ ./Words mama Two words: SUCCESS: ma ma. DP: SUCCESS: mama --- mamama Two words: SUCCESS: ma mama. mama ma. DP: SUCCESS: mama ma --- forevercarrot Two words: SUCCESS: forever carrot. DP: SUCCESS: forever carrot --- Iamace Two words: FAILURE DP: SUCCESS: i am ace --- xxxx Two words: FAILURE DP: FAILURE --- notindictionary Two words: FAILURE DP: FAILURE ---Note that at this point, Words prints all possible two word segmentations, but not all possible segmentations.
Here is from the algorithm example, including the debug flag.
$ ./Words -debug Loading dictionary: words Word count: 20 breakingupishardtodo Input: 'breakingupishardtodo' Two words: FAILURE DP: SUCCESS: breaking up is hard to do 8 -1 -1 -1 -1 -1 -1 -1 10 -1 12 -1 16 -1 -1 -1 18 -1 20 -1 ---The debug flag causes the program to print out the array and other information.
We can also feed the test words from a file using redirected input.
$ cat test mama mamama breakingupishardtodo $ ./Words -debug < test Loading dictionary: words Word count: 20 Input: 'mama' Two words: SUCCESS: ma ma. DP: SUCCESS: mama 4 3 4 4 --- Input: 'mamama' Two words: SUCCESS: ma mama. mama ma. DP: SUCCESS: mama ma 4 3 6 5 6 6 --- Input: 'breakingupishardtodo' Two words: FAILURE DP: SUCCESS: breaking up is hard to do 8 -1 -1 -1 -1 -1 -1 -1 10 -1 12 -1 16 -1 -1 -1 18 -1 20 -1 ---Words should:
Use the submit command (see below) to turn in the source file(s) for Words.c and your log file. Do not submit the hash.c, hash.h, and Makefile. Your code should
#include "/c/cs223/hw7/hash.h"YOU MUST SUBMIT YOUR FILES (INCLUDING THE LOG FILE) AT THE END OF ANY SESSION WHERE YOU SPEND AT LEAST ONE-HALF HOUR WRITING OR DEBUGGING CODE, AND AT LEAST ONCE EVERY HOUR DURING LONGER SESSIONS. (All submissions are retained.)
% /c/cs223/hw7/Tests/test.WordsYou may invoke a given test as follows:
% /c/cs223/hw7/Tests/test.Words 01(you may specify more than one test here).
If your output looks the same as what is expected, but your program still fails the test, there are probably some invisible characters in your output.
ESTIMATE of time to complete assignment: 10 hours Time Time Date Started Spent Work completed ---- ------- ---- -------------- 1/13 10:15pm 0:45 Read assignment and relevant material in K&R 1/16 4:45pm 1:15 Sketched solution using a finite-state machine with one-character look-ahead 1/19 9:00am 2:20 Wrote the program and eliminated compile-time errors; code passes eight tests 1/20 7:05pm 2:00 Discovered and corrected two logical errors; code now passes eleven tests 1/23 11:00am 1:35 Finished debugging; program passes all public tests ---- 7:55 TOTAL time spent I discussed my solution with: Peter Salovey, Ben Polak, Tamar Gendler, and Jonathan Holloway (and watched four episodes of The Simpsons). *A brief discussion of the major difficulties encountered*but MUST contain
N.B. To facilitate analysis, the log file MUST be the only file submitted whose name contains the string "log" and the estimate / total MUST be on the only line in that file that contains the string "ESTIMATE" / "TOTAL".
% /c/cs223/bin/submit 7 Makefile Words.c time.logsubmits the named source files as your solution to Homework #7
% /c/cs223/bin/check 7lists the files that you submitted for Homework #7;
% /c/cs223/bin/unsubmit 7 error.submit bogus.solutiondeletes the named files that you submitted previously for Homework #7 (which is useful if you rename a file or accidentally submit the wrong one);
% /c/cs223/bin/makeit 7 Wordsruns "make" on the files that you submitted previously for Homework #7;
% /c/cs223/bin/testit 7 Wordsruns the public test script for Words using the files that you submitted previously for Homework #7; This might not be working. Use the testing instructions given above.
% /c/cs223/bin/protect 7 Words.c time.logprotects the named files that you submitted previously for Homework #7 (so they cannot be deleted accidentally);
% /c/cs223/bin/unprotect 7 util.c time.logunprotects the named files that you submitted previously for Homework #7 (so they can be deleted); and
% /c/cs223/bin/retrieve 8 common.c time.logand
% /c/cs223/bin/retrieve 8 -d"2017/01/21 20:00" util.cretrieve copies of the named files that you submitted previously for Homework #8 (in case you accidentally delete your own copies). The day and hour are optional and request the latest submission prior to that time (see the -d flag under "man co" for how to specify times).
When assignments are style graded, EVERY source file found in the submit directory will be reviewed. Thus prudence suggests using unsubmit to remove a file from the directory when you change its name or it ceases to be part of your solution. See http://zoo.cs.yale.edu/classes/cs223/doc/Style
Prudence (and a 5-point penalty for code that does not make) suggests that you run makeit ("makeit 7 Words") after you have submitted the final version of your source files. Better yet, run testit ("testit 7 Words").