Kaggle Digit Recognizer series

From yesterday’s post:

Octave appears to be failing on the first iteration inside the fmincg() optimization function. The error message is not very helpful. It complains of being out of memory or an index exceeding its range. No line number given.

I could not reproduce this failure. This was on a “fresh” run of Octave so perhaps it was caused by some weird program state left over from ongoing development. Note to self: try restarting Octave when mysterious crashes with unhelpful error messages occur.

So then before the science begins let’s try one more Kaggle submission, this time using all 42,000 samples to train the neural network. Same as last attempt are the 350 and 100 node hidden layers, lambda 1, and 500 training iterations. Run time was about 5000 seconds (1 hour, 13 minutes), final cost function 0.0671, and self-classified accuracy 99.714%.

Slight improvement but probably not meaningful. There is so much random stuff going on here, from the random initialization of Thetas, to whatever random selection is done at Kaggle in scoring the results. The cross validation and analysis discussed at the end of yesterday’s post really are next.