From: Martin Pitt <martin@piware.de>
Date: Sun, 30 Aug 2020 08:19:10 +0000 (+0200)
Subject: Use linearly falling learning rate
X-Git-Url: https://piware.de/gitweb/?p=handwriting-recognition.git;a=commitdiff_plain;h=59f4fd752941f39ddcd36760202a0dc742747106;hp=59f4fd752941f39ddcd36760202a0dc742747106

Use linearly falling learning rate

This makes big strides on the completely untrained network, but treads
more carefully later on. Prevents overshooting with reLU, and even
slightly improves Sigmoid.
---