# Resources Basics: - [Learn numpy](https://numpy.org/learn/) - [MNIST database of handwritten digits](http://yann.lecun.com/exdb/mnist/) - [Neuron](https://en.wikipedia.org/wiki/Artificial_neuron) - [Perceptron](https://en.wikipedia.org/wiki/Perceptron) - [Backpropagation](https://en.wikipedia.org/wiki/Backpropagation) - [3Blue1Brown video series](https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi) Too high-level for first-time learning, but apparently very abstract and powerful for real-life: - [keras](https://keras.io/) - [tutorial how to recognize handwriting with keras/tensorflow](https://data-flair.training/blogs/python-deep-learning-project-handwritten-digit-recognition/) # Dependencies sudo dnf install -y python3-numpy python3-matplotlib # Steps - Do the [NumPy quickstart tutorial](https://numpy.org/devdocs/user/quickstart.html); example: ```py import numpy as np import matplotlib.pyplot as plt grad = np.linspace(0,1,10000).reshape(100,100) plt.imshow(grad, cmap='gray') plt.show() plt.imshow(np.sin(np.linspace(0,10000,10000)).reshape(100,100) ** 2, cmap='gray') # non-blocking does not work with QT_QPA_PLATFORM=wayland plt.show(block=False) plt.close() ``` - Get the handwritten digits training data with `./download-mnist.sh` - Read the MNIST database into numpy arrays with `./read_display_mnist.py`. Plot the first ten images and show their labels, to make sure the data makes sense: ![visualize training data](screenshots/mnist-visualize-training-data.png) - Define the structure of the neural network: two hidden layers with parametrizable sizes. Initialize weights and biases randomly. This gives totally random classifications of course, but at least makes sure that the data structures and computations work: ``` $ ./train.py output vector of first image: [ 0. 52766.88424917 0. 0. 14840.28619491 14164.62850135 0. 7011.882333 0. 46979.62976127] classification of first image: 1 with confidence 52766.88424917019; real label 5 correctly recognized images after initialization: 10.076666666666668% ``` - Add backpropagation algorithm and run a first training round. This is slow, as expected: ``` $ time ./train.py output vector of first image: [ 0. 52766.88424917 0. 0. 14840.28619491 14164.62850135 0. 7011.882333 0. 46979.62976127] classification of first image: 1 with confidence 52766.88424917019; real label 5 correctly recognized images after initialization: 10.076666666666668% round #0 of learning... ./train.py:18: RuntimeWarning: overflow encountered in exp return 1 / (1 + np.exp(-x)) correctly recognized images: 14.211666666666666% real 0m37.927s user 1m19.103s sys 1m10.169s ```