Programming Assignments

> > > > Programming Assignments > Assignment 6 - Neural Networks

Objectives

to gather good training data
to choose a good representation of inputs to a neural network
to train and use a neural network in an agent for a game

Assignment

Create a Python 3 module called nn that creates and trains a neural network for playing solitaire Yahtzee. This module must define

a function called train that reads your training examples from standard inputa and returns a trained neural network; and
a class called NNStrategy that has an initializer that can take the object returned from train as its parameter and has two methods choose_dice and choose_category that meet the requirements of the yahtzee.evaluate_strategy function.

There are four steps to creating the neural network.

1. Gather training data

Training examples will be solitaire Yahtzee positions and the optimal choice (dice to keep for reroll positions, category to score in for end-of-turn positions) for those positions. You may use no more than 100,000 examples for training, so you will have to devise a way to sample the approximately 1.5 billion distinct positions in solitaire Yahtzee. Once you have chosen your training inputs, you can determine the optimal choices with the query_optimal.sh program in /c/cs474/hw6. That program reads positions from standard input where each position is given as a comma-separated list of the state of the scoresheet (as returned by YahtzeeScoresheet.as_state_string), the current roll, and the number of rerolls, and outputs the positions with either the dice chosen to keep or the string representation of the category to score in.

[jrg94@scorpion Yahtzee]$ /c/cs474/hw6/query_optimal.sh
UP0,11223,2
UP0,11223,2,[22]
2 3 4 5 6 3K 4K FH SS LS C UP50,11166,2
2 3 4 5 6 3K 4K FH SS LS C UP50,11166,2,[111]
2 3 4 5 6 3K 4K FH SS LS UP60,11666,2
2 3 4 5 6 3K 4K FH SS LS UP60,11666,2,[11]
2 3 4 5 6 3K 4K FH SS LS UP60,11666,1
2 3 4 5 6 3K 4K FH SS LS UP60,11666,1,[11]
2 3 4 5 6 3K 4K FH SS LS UP60,11666,0
2 3 4 5 6 3K 4K FH SS LS UP60,11666,0,C
2 3 4 5 6 3K 4K FH SS LS C Y UP60,16666,2
2 3 4 5 6 3K 4K FH SS LS C Y UP60,16666,2,[1]
2 3 4 5 6 3K 4K FH SS LS C Y+ UP60,16666,2
2 3 4 5 6 3K 4K FH SS LS C Y+ UP60,16666,2,[6666]

2. Transform and label the training data

You will have to devise a way of representing the positions as inputs to your neural network. You will have to devise a way of interpreting the output of the neural network as a decision in Yahtzee. For the latter, we suggest you implement a multi-class classifier where each input is classified as a position in which you should "try for 1s", "try for 2s", ..., "try for 6s", "try for Yahtzee", "try for full house", "try for a straight", or "try for chance", where what each class means in each position is defined using hand-written rules such as the ones defined in the yahtzee.YahtzeeRoll.select_for_xxx methods. You will have to write code that determines which of these classes best matches the output of the optimal player for each position.

3. Design and train your neural network

You may use at most 300 neurons in your hidden layers (not counting Dropout layers) but the architecture may be whatever you wish. The nn.train function should read your training examples from standard input, train your neural network, and return it.

4. Implement a strategy that uses your neural network

Create a class called NNStrategy that defines

an initializer that can take as its argument the object returned by train;
a method choose_dice that takes a YahtzeeScoresheet, a YahtzeeRoll, and the number of rerolls, encodes that position as input suitable for your neural network, computes the output of your neural network, and returns a subroll of the given roll as indicated by the output of the neural network; and
a method choose_category that takes a YahtzeeScoresheet and a YahtzeeRoll, encodes that position as input suitable for your neural network, computes the output of your neural network, and returns an unused category as indicated by the output of the neural network.

Files

In /c/cs474/hw6 are three Python 3 modules:

yahtzee.py that defines classes that implement the rules of Yahtzee;
multiset.py that defines classes required by the yahtzee module; and
test_yahtzee.py that implements the test driver

Additionally, there is a Java implementation of an optimal solitaire Yahtzee player and a program to determine what choices the optimal strategy makes:

yahtzee.jar that contains the compiled Java implementation
bonusyahtzeestate.dat a data file required by that implementation
query_optimal.sh a shell script that reads solitaire Yahtzee positions and outputs those positions with the optimal choice

Submissions

Submit just your nn.py module along with any other supporting modules, your training data as training.dat, and your log.

Submissions may use no more than 100,000 examples for training and may not use more than 300 neurons in hidden layers in the neural network (not including anything in Dropout layers).