2020-03-06 18:19:03 +00:00
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
2020-08-21 19:36:27 +00:00
"from fastai.vision.all import *\n",
2020-03-06 18:19:03 +00:00
"from utils import *\n",
"\n",
"matplotlib.rc('image', cmap='Greys')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"# Convolutional Neural Networks"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## The Magic of Convolutions"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"top_edge = tensor([[-1,-1,-1],\n",
" [ 0, 0, 0],\n",
" [ 1, 1, 1]]).float()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"path = untar_data(URLs.MNIST_SAMPLE)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"Path.BASE_PATH = path"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAADyElEQVR4nO2aTSg1URjHf1eIS9gQETYWPhOiFLGwkiTJzs7OXpFsWMlKsqEoRT4WFmKhlI+wsWWluCtECIVh3oX3mNd5hzvGneum51d3MzPOee7//p3zPM8Zn2maCBZRPx1ApCGCaIggGiKIhgiiER3k/m/egnx2F8UhGiKIhgiiIYJoiCAaIoiGCKIhgmiIIBrBMlVbHh8fAVhfXwcgPj4egO3tbQCur68BGBkZAaClpQWArKysD8fMzMwEoLm5GYDs7Gw3oX0bcYiGL0jHzPbm0NAQAN3d3SEPKCrq9TeqqKgAoLOzE4DW1lYAUlJSQjWV1DJOcOWQgoICAA4PD23/KC0tDYCamppPJ8/Pzwfg4OCAs7MzADY3N22f3d/fB6C0tPTTMb+AOMQJrnaZra0tAE5OToD/d4TY2FgAEhMTHY/58PAAQGFhIQBHR0fv7s/PzwMhdYgt4hANV2uIF2xsbABQV1f37npcXBzwus4A5OTkhGpKWUMcYZrmZx9PMQzDNAzD7O3tNf1+v+n3+02fz/fuEwgEzEAg4MX0tt9ZHKLhapf5Lip/mZiYAGB4ePjtXkxMDACLi4sApKenhzU2cYhGWB1yfHwMQHFxMQDPz8//PaNqGVUZ+3y2m4FniEM0wuqQ2dlZwN4ZCpWxlpWVAVBfXw9Ae3s7AE1NTQBkZGR4EmNYEzOVjvf39wOwtrYGwOnpqeMx1L/U4OAgAF1dXQAkJCR8NRxJzJzwo6m7ajXe3NxweXkJwMzMDGA1oYLE99aeXFhYAL60CItDnBAxxZ2OKvYGBgYAa735iMnJSQA6OjqcTiEOccKPpO5OqK2tBWB1dRWwmsxLS0u2z6v2wHcRh2hErEMUKu+oqqoCPnZIUVFRaOYLySi/CE8dcnt7C8D09DQAJSUlAFRXVzse4+XlBbCOIXSio1+/QmVlpes4/0UcouGJQ5QzGhoaANjb2wPg/v7e8Rh3d3cAjI2NAVYmqlNeXg5AXl6eu2A1xCEanjhEHYIrZyguLi4A66hTtQsBnp6eABgfHwegp6cHsOodhcqsk5OTAZiamgpp7OIQDU9qmZWVFQAaGxtt76tD8NTU1Ldr5+fnwMeH3YqkpCQAdnZ2AOvA3AVSyzjBE4dcXV0B0NfXB8Do6KibYQArz1Adsra2NgByc3Ndj/kXcYgTPO2HGIYBwO7uLgDLy8uAVXfMzc29PatewlGo9Uc54bMX9lwiDnFCxHbMwoA4xAkiiIYIoiGCaIggGsGq3fC+ixABiEM0RBANEURDBNEQQTREEI0/H3jyQ4wdtXsAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 72x72 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"im3 = Image.open(path/'train'/'3'/'12.png')\n",
"show_image(im3);"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([[-0., -0., -0.],\n",
" [0., 0., 0.],\n",
" [0., 0., 0.]])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"im3_t = tensor(im3)\n",
"im3_t[0:3,0:3] * top_edge"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(0.)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(im3_t[0:3,0:3] * top_edge).sum()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<style type=\"text/css\" >\n",
" #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col0 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col1 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col2 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col3 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col4 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col5 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col6 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col7 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col8 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col9 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col10 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col11 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col12 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col13 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col14 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col15 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col16 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col17 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col18 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col19 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col0 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col1 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col2 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col3 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col4 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col5 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col6 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col7 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col8 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col9 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col10 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col11 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col12 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col13 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col14 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col15 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col16 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col17 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col18 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col19 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col0 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col1 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col2 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col3 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col4 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col5 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col6 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col7 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col8 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col9 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col10 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col11 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col12 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col13 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col14 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col15 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col16 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col17 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col18 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col19 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col0 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col1 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col2 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col3 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col4 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col5 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col6 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col7 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col8 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col9 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col10 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col11 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col12 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col13 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col14 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col15 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col16 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col17 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col18 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col19 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col0 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col1 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col2 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col3 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col4 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col5 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col6 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col7 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col8 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col9 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col10 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col11 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col12 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col13 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col14 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col15 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col16 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col17 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col18 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col19 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col0 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col1 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col2 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col3 {\n",
" font-size: 6pt;\n",
" background-color: #f9f9f9;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col4 {\n",
" font-size: 6pt;\n",
" background-color: #b9b9b9;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col5 {\n",
" font-size: 6pt;\n",
" background-color: #c1c1c1;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col6 {\n",
" font-size: 6pt;\n",
" background-color: #858585;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col7 {\n",
" font-size: 6pt;\n",
" background-color: #777777;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col8 {\n",
" font-size: 6pt;\n",
" background-color: #090909;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col9 {\n",
" font-size: 6pt;\n",
" background-color: #5b5b5b;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col10 {\n",
" font-size: 6pt;\n",
" background-color: #777777;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col11 {\n",
" font-size: 6pt;\n",
" background-color: #777777;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col12 {\n",
" font-size: 6pt;\n",
" background-color: #777777;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col13 {\n",
" font-size: 6pt;\n",
" background-color: #777777;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col14 {\n",
" font-size: 6pt;\n",
" background-color: #919191;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col15 {\n",
" font-size: 6pt;\n",
" background-color: #e1e1e1;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col16 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col17 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col18 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col19 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col0 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col1 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col2 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col3 {\n",
" font-size: 6pt;\n",
" background-color: #727272;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col4 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col5 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col6 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col7 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col8 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col9 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col10 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col11 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col12 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col13 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col14 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col15 {\n",
" font-size: 6pt;\n",
" background-color: #020202;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col16 {\n",
" font-size: 6pt;\n",
" background-color: #363636;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col17 {\n",
" font-size: 6pt;\n",
" background-color: #9d9d9d;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col18 {\n",
" font-size: 6pt;\n",
" background-color: #dfdfdf;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col19 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col0 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col1 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col2 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col3 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col4 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col5 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col6 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col7 {\n",
" font-size: 6pt;\n",
" background-color: #161616;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col8 {\n",
" font-size: 6pt;\n",
" background-color: #535353;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col9 {\n",
" font-size: 6pt;\n",
" background-color: #535353;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col10 {\n",
" font-size: 6pt;\n",
" background-color: #535353;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col11 {\n",
" font-size: 6pt;\n",
" background-color: #535353;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col12 {\n",
" font-size: 6pt;\n",
" background-color: #7c7c7c;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col13 {\n",
" font-size: 6pt;\n",
" background-color: #535353;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col14 {\n",
" font-size: 6pt;\n",
" background-color: #3d3d3d;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col15 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col16 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col17 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col18 {\n",
" font-size: 6pt;\n",
" background-color: #999999;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col19 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col0 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col1 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col2 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col3 {\n",
" font-size: 6pt;\n",
" background-color: #eaeaea;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col4 {\n",
" font-size: 6pt;\n",
" background-color: #d0d0d0;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col5 {\n",
" font-size: 6pt;\n",
" background-color: #eeeeee;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col6 {\n",
" font-size: 6pt;\n",
" background-color: #eeeeee;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col7 {\n",
" font-size: 6pt;\n",
" background-color: #f3f3f3;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col8 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col9 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col10 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col11 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col12 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col13 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col14 {\n",
" font-size: 6pt;\n",
" background-color: #f9f9f9;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col15 {\n",
" font-size: 6pt;\n",
" background-color: #232323;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col16 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col17 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col18 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col19 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col0 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col1 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col2 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col3 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col4 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col5 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col6 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col7 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col8 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col9 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col10 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col11 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col12 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col13 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col14 {\n",
" font-size: 6pt;\n",
" background-color: #c2c2c2;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col15 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col16 {\n",
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col17 {\n",
" font-size: 6pt;\n",
" background-color: #080808;\n",
" color: #f1f1f1;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col18 {\n",
" font-size: 6pt;\n",
" background-color: #c4c4c4;\n",
" color: #000000;\n",
" } #T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col19 {\n",
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
" }</style><table id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35\" ><thead> <tr> <th class=\"blank level0\" ></th> <th class=\"col_heading level0 col0\" >0</th> <th class=\"col_heading level0 col1\" >1</th> <th class=\"col_heading level0 col2\" >2</th> <th class=\"col_heading level0 col3\" >3</th> <th class=\"col_heading level0 col4\" >4</th> <th class=\"col_heading level0 col5\" >5</th> <th class=\"col_heading level0 col6\" >6</th> <th class=\"col_heading level0 col7\" >7</th> <th class=\"col_heading level0 col8\" >8</th> <th class=\"col_heading level0 col9\" >9</th> <th class=\"col_heading level0 col10\" >10</th> <th class=\"col_heading level0 col11\" >11</th> <th class=\"col_heading level0 col12\" >12</th> <th class=\"col_heading level0 col13\" >13</th> <th class=\"col_heading level0 col14\" >14</th> <th class=\"col_heading level0 col15\" >15</th> <th class=\"col_heading level0 col16\" >16</th> <th class=\"col_heading level0 col17\" >17</th> <th class=\"col_heading level0 col18\" >18</th> <th class=\"col_heading level0 col19\" >19</th> </tr></thead><tbody>\n",
" <tr>\n",
" <th id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col0\" class=\"data row0 col0\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col1\" class=\"data row0 col1\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col2\" class=\"data row0 col2\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col3\" class=\"data row0 col3\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col4\" class=\"data row0 col4\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col5\" class=\"data row0 col5\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col6\" class=\"data row0 col6\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col7\" class=\"data row0 col7\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col8\" class=\"data row0 col8\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col9\" class=\"data row0 col9\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col10\" class=\"data row0 col10\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col11\" class=\"data row0 col11\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col12\" class=\"data row0 col12\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col13\" class=\"data row0 col13\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col14\" class=\"data row0 col14\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col15\" class=\"data row0 col15\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col16\" class=\"data row0 col16\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col17\" class=\"data row0 col17\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col18\" class=\"data row0 col18\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row0_col19\" class=\"data row0 col19\" >0</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col0\" class=\"data row1 col0\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col1\" class=\"data row1 col1\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col2\" class=\"data row1 col2\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col3\" class=\"data row1 col3\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col4\" class=\"data row1 col4\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col5\" class=\"data row1 col5\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col6\" class=\"data row1 col6\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col7\" class=\"data row1 col7\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col8\" class=\"data row1 col8\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col9\" class=\"data row1 col9\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col10\" class=\"data row1 col10\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col11\" class=\"data row1 col11\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col12\" class=\"data row1 col12\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col13\" class=\"data row1 col13\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col14\" class=\"data row1 col14\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col15\" class=\"data row1 col15\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col16\" class=\"data row1 col16\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col17\" class=\"data row1 col17\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col18\" class=\"data row1 col18\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row1_col19\" class=\"data row1 col19\" >0</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col0\" class=\"data row2 col0\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col1\" class=\"data row2 col1\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col2\" class=\"data row2 col2\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col3\" class=\"data row2 col3\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col4\" class=\"data row2 col4\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col5\" class=\"data row2 col5\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col6\" class=\"data row2 col6\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col7\" class=\"data row2 col7\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col8\" class=\"data row2 col8\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col9\" class=\"data row2 col9\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col10\" class=\"data row2 col10\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col11\" class=\"data row2 col11\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col12\" class=\"data row2 col12\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col13\" class=\"data row2 col13\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col14\" class=\"data row2 col14\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col15\" class=\"data row2 col15\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col16\" class=\"data row2 col16\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col17\" class=\"data row2 col17\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col18\" class=\"data row2 col18\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row2_col19\" class=\"data row2 col19\" >0</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col0\" class=\"data row3 col0\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col1\" class=\"data row3 col1\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col2\" class=\"data row3 col2\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col3\" class=\"data row3 col3\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col4\" class=\"data row3 col4\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col5\" class=\"data row3 col5\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col6\" class=\"data row3 col6\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col7\" class=\"data row3 col7\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col8\" class=\"data row3 col8\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col9\" class=\"data row3 col9\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col10\" class=\"data row3 col10\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col11\" class=\"data row3 col11\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col12\" class=\"data row3 col12\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col13\" class=\"data row3 col13\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col14\" class=\"data row3 col14\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col15\" class=\"data row3 col15\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col16\" class=\"data row3 col16\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col17\" class=\"data row3 col17\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col18\" class=\"data row3 col18\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row3_col19\" class=\"data row3 col19\" >0</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col0\" class=\"data row4 col0\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col1\" class=\"data row4 col1\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col2\" class=\"data row4 col2\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col3\" class=\"data row4 col3\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col4\" class=\"data row4 col4\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col5\" class=\"data row4 col5\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col6\" class=\"data row4 col6\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col7\" class=\"data row4 col7\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col8\" class=\"data row4 col8\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col9\" class=\"data row4 col9\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col10\" class=\"data row4 col10\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col11\" class=\"data row4 col11\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col12\" class=\"data row4 col12\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col13\" class=\"data row4 col13\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col14\" class=\"data row4 col14\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col15\" class=\"data row4 col15\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col16\" class=\"data row4 col16\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col17\" class=\"data row4 col17\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col18\" class=\"data row4 col18\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row4_col19\" class=\"data row4 col19\" >0</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col0\" class=\"data row5 col0\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col1\" class=\"data row5 col1\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col2\" class=\"data row5 col2\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col3\" class=\"data row5 col3\" >12</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col4\" class=\"data row5 col4\" >99</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col5\" class=\"data row5 col5\" >91</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col6\" class=\"data row5 col6\" >142</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col7\" class=\"data row5 col7\" >155</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col8\" class=\"data row5 col8\" >246</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col9\" class=\"data row5 col9\" >182</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col10\" class=\"data row5 col10\" >155</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col11\" class=\"data row5 col11\" >155</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col12\" class=\"data row5 col12\" >155</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col13\" class=\"data row5 col13\" >155</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col14\" class=\"data row5 col14\" >131</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col15\" class=\"data row5 col15\" >52</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col16\" class=\"data row5 col16\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col17\" class=\"data row5 col17\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col18\" class=\"data row5 col18\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row5_col19\" class=\"data row5 col19\" >0</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col0\" class=\"data row6 col0\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col1\" class=\"data row6 col1\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col2\" class=\"data row6 col2\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col3\" class=\"data row6 col3\" >138</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col4\" class=\"data row6 col4\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col5\" class=\"data row6 col5\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col6\" class=\"data row6 col6\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col7\" class=\"data row6 col7\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col8\" class=\"data row6 col8\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col9\" class=\"data row6 col9\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col10\" class=\"data row6 col10\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col11\" class=\"data row6 col11\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col12\" class=\"data row6 col12\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col13\" class=\"data row6 col13\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col14\" class=\"data row6 col14\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col15\" class=\"data row6 col15\" >252</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col16\" class=\"data row6 col16\" >210</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col17\" class=\"data row6 col17\" >122</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col18\" class=\"data row6 col18\" >33</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row6_col19\" class=\"data row6 col19\" >0</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col0\" class=\"data row7 col0\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col1\" class=\"data row7 col1\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col2\" class=\"data row7 col2\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col3\" class=\"data row7 col3\" >220</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col4\" class=\"data row7 col4\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col5\" class=\"data row7 col5\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col6\" class=\"data row7 col6\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col7\" class=\"data row7 col7\" >235</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col8\" class=\"data row7 col8\" >189</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col9\" class=\"data row7 col9\" >189</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col10\" class=\"data row7 col10\" >189</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col11\" class=\"data row7 col11\" >189</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col12\" class=\"data row7 col12\" >150</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col13\" class=\"data row7 col13\" >189</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col14\" class=\"data row7 col14\" >205</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col15\" class=\"data row7 col15\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col16\" class=\"data row7 col16\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col17\" class=\"data row7 col17\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col18\" class=\"data row7 col18\" >75</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row7_col19\" class=\"data row7 col19\" >0</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col0\" class=\"data row8 col0\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col1\" class=\"data row8 col1\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col2\" class=\"data row8 col2\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col3\" class=\"data row8 col3\" >35</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col4\" class=\"data row8 col4\" >74</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col5\" class=\"data row8 col5\" >35</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col6\" class=\"data row8 col6\" >35</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col7\" class=\"data row8 col7\" >25</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col8\" class=\"data row8 col8\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col9\" class=\"data row8 col9\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col10\" class=\"data row8 col10\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col11\" class=\"data row8 col11\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col12\" class=\"data row8 col12\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col13\" class=\"data row8 col13\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col14\" class=\"data row8 col14\" >13</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col15\" class=\"data row8 col15\" >224</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col16\" class=\"data row8 col16\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col17\" class=\"data row8 col17\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col18\" class=\"data row8 col18\" >153</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row8_col19\" class=\"data row8 col19\" >0</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col0\" class=\"data row9 col0\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col1\" class=\"data row9 col1\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col2\" class=\"data row9 col2\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col3\" class=\"data row9 col3\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col4\" class=\"data row9 col4\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col5\" class=\"data row9 col5\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col6\" class=\"data row9 col6\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col7\" class=\"data row9 col7\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col8\" class=\"data row9 col8\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col9\" class=\"data row9 col9\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col10\" class=\"data row9 col10\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col11\" class=\"data row9 col11\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col12\" class=\"data row9 col12\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col13\" class=\"data row9 col13\" >0</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col14\" class=\"data row9 col14\" >90</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col15\" class=\"data row9 col15\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col16\" class=\"data row9 col16\" >254</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col17\" class=\"data row9 col17\" >247</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col18\" class=\"data row9 col18\" >53</td>\n",
" <td id=\"T_508423a8_5672_11ea_9acc_8f0047ef1a35row9_col19\" class=\"data row9 col19\" >0</td>\n",
" </tr>\n",
" </tbody></table>"
],
"text/plain": [
"<pandas.io.formats.style.Styler at 0x7fb709e80750>"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = pd.DataFrame(im3_t[:10,:20])\n",
"df.style.set_properties(**{'font-size':'6pt'}).background_gradient('Greys')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(762.)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(im3_t[4:7,6:9] * top_edge).sum()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(-29.)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(im3_t[7:10,17:20] * top_edge).sum()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def apply_kernel(row, col, kernel):\n",
" return (im3_t[row-1:row+2,col-1:col+2] * kernel).sum()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(762.)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"apply_kernel(5,7,top_edge)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Mapping a Convolution Kernel"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[[(1, 1), (1, 2), (1, 3), (1, 4)],\n",
" [(2, 1), (2, 2), (2, 3), (2, 4)],\n",
" [(3, 1), (3, 2), (3, 3), (3, 4)],\n",
" [(4, 1), (4, 2), (4, 3), (4, 4)]]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"[[(i,j) for j in range(1,5)] for i in range(1,5)]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAE1UlEQVR4nO2c104cSxRFF9iYYHIGk0EkiSQsXuA3+Ak+iI/hAT8ihEBgRI4i2ORsgkn3wdpT0weu1YN75KurWi893dNd01TvPnXOrhIpz8/PeBypf/sG/mv4DjH4DjH4DjH4DjG8/92Xw8PD/9shaGhoKOW1414hBt8hBt8hBt8hBt8hht+OMuLp6QmAk5MTAM7OzgDY39+PnfPjxw8AVlZWALi9vQ20UVJSAsCHDx8Cx9PS0gAoKyuLHfv06RMA5eXlAGRmZoa5zUjwCjGEUoiUMDs7C8DY2BgABwcHoX9ofX099LkZGRkAVFRUANDb2wtAU1MT4NRk1RYFXiEG3yGGUK/M1dUV4CQqKXd1dcXOKSgoAKClpQWAwsJCALKysgA4PT0FIDU1+AwUfBWMAaanpwFYXV0FYGpqKvD7+fn5gf0o8QoxhFJIf38/ALW1tQBUV1cDkJ2d7Rp6/6upd+/eBbZWERZZmEdHR7FjCwsLAIyMjACwu7sLuGB7c3MDQE5OTpjbTwivEEMohZSWlgJQWVkJuCRLKgCXvN3f3wNwd3cHwMPDQ6AtfS8UQ+IVos9VVVUA1NXVBa6RqmxbUeAVYgilEI0Ah4eHAKSnp784R6n7z58/AacQ7Qs93Z2dHcDFi/jzpIzBwcHAvtjY2ABgc3MzzO0nhFeIIZRC9LTji7lEUS4jZUxOTgbabG5ujp2r/EaxQ/mGYouUmgy8QgyhFPInXF9fAzA3NwfAxMRE4HhfXx8AHR0dsWuklr29PcDFHRWIujYZeIUYkq6Q8/NzwOUjNTU1gDOB6uvrAVf7AIyOjgJuNPn27Rvgyv6GhobAvjLYKPAKMSRdIaqMW1tbAZfdpqT8mie6uLgAgnmIMtDj42MAvn79GjhXmbKMo87OTsCpLv53EsUrxJB0hagylkKkGJnLUoOePrjqViNPd3c34DLmmZkZwI1Yqod6enpibTQ2Nr7pfr1CDL5DDEl/ZWQdyqGXpVhUVBTYjzebNATLbtDQrGFYr4YKw+/fvwPuVQPIzc0FXAAOi1eIIekKUVGnQLi4uAg4ZeTl5QFQXFwcu0blvlT08eNHwD112ZJqQwVjPNZ2CItXiCHpCrHIStBWMSbe7FGs0NCpaQchZUg5GtJfI9F1uF4hhsgVoiJOxo+KO01TqCBT7HgNpexabfBvKP2Pjz9CprfaCotXiCEyhUgZy8vLgLMINSK0t7cD4SaXHh8fAVf2yzpUuq91I7ISXmvzrSaSV4ghMoUoS9ze3gbcdKMKNE2DyszRKCPiDWzZjePj44BTwOfPnwFnEKlNrTBS3IC3G9FeIYbIFKL6Q3WGtlLMly9fAJifnwderhu7vLyMfZZ6tMRC0xIyn2UMKYZIGYo5tr1E8AoxRKaQtrY24GVWKWUo+5QRZCe9NHIADAwMAC7+yCLUGjPFDilJ1e7W1tYf/x1eIQbfIYbIXhnJVyW7jBkVXprJl7xlGOm618p/tWHXlGmlgdpSoH5rII3HK8QQmUIULPXkZeZo9ZG2SuETQWm4htW1tTXAGUMyoaLAK8QQeQxZWloCnFI0L6MkS2ayCjXFhXjLT7FA1oFihewAxZBk4BViiNwgssZMogbN38YrxJDi/xlCEK8Qg+8Qg+8Qg+8Qg+8Qg+8Qwz/aP/Y2oVu6fAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 72x72 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"rng = range(1,27)\n",
"top_edge3 = tensor([[apply_kernel(i,j,top_edge) for j in rng] for i in rng])\n",
"\n",
"show_image(top_edge3);"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAEa0lEQVR4nO2cyUosSxCGv9brPM8TDqgLZ1EXIgouRHwZH8dXEcGFS9GFKCoqqOCEA87zrGdx+Dur8nhaqy37Xi75baqpLLOTyD8jIiOrjby/v+MwJP3bA/iv4Qxi4Qxi4Qxi4Qxi8U+sxrGxsf9tCBodHY18dN8pxMIZxMIZxMIZxMIZxCJmlLF5eXkB4PT0FICKiopo2/n5OQBZWVkAnJycANDS0gLA7e0tAI+Pjx/2nZaWFv2cn58PwP39PQBXV1dBhvktnEIsAilkY2MDMDPmVUiYRCK/U4Tc3FzAqE73pUYpKEycQiycQSwCLZmpqSkA2tra/vpMSkoKANnZ2TH7yszMBOD4+BiApCQzN5eXlwAUFRX5/kYOen19HYDt7e0vj/2rOIVYBFLIw8MDYGY/PT092paXlweY8KnZVfj9jLu7u+jnyclJAEZGRgAoLS319amyp1NIAgikkM7OTgBqampCH8jh4WH08+zsLAADAwMAHB0dAZCTkwPAxcVF6N8vnEIsAimkr68PMGv4I56engC4vr6O2dfNzQ0Ae3t7fzyvREz09vYCcHZ2BjiFJJRAClHuILyRQby9vQF/38RplpV2y3coTQcYGhoCoLu7GzDRRSn72tpakGEHwinEIpBCvoOyz7m5OQCam5t97eXl5dHPlZWVADQ2NgKQkZEBwPT0tK+vn8ApxCJhClHpQOtf+yGpwFtK0DZf7OzsACYzLisrA4wP+yyiBcEpxCJhClHuoKv8gh25wOQ5UoYUoAgmBdXV1QFGKbu7u9E+lA8FxSnEImEK0W54cHAQMDOo+ok3+1UUGR8fB0xEkqpsH6O+S0pKovdmZmbiGqdTiIUziEXClozCq9Lvr6Bt/+vrK2CWhq4Kw4WFhYBJ6AAaGhoA2NzcDDROpxCLhClEp3G6qhyp4nKskoLaFhYWAOjo6PC165yovb09es8pJCQSphAbb4HaRkVlu5QwPz8PGJUVFBQApqQg1QEUFxfHNS6nEIsfU4hmVQdRTU1NgCkUHxwcfNqHSgJSivxBV1eX7zklal7VxVsicAqxCF0hUoYigtZ7f38/ACsrK4H7kppUSlQeIqQ6XQEWFxcDjx2cQv4gdIUsLy8DsL+/DxiF1NbWAmZLr/xD/kFX7zPPz8++Z5VnVFdXA5CcnAyYvMT7vki8x5xOIRahK0SzK18xPDwMmNnWu2ba9stPqB1MQUhRQ1Gkp6cHMNt8RRv5Dn03mNwkKE4hFqErRL5AvkNHllKMVwmfkZqaChifUV9fD5i3IXXkKd8Rb2Tx4hRi4QxiEfqS0TskCpVK3ScmJgBobW0FTLKl4o73bFfVdF3lePWMHKYKR6urq77738EpxCJ0hVRVVQGmqOPdkoPZdGl2hcIw+FPwj5BTXVpa8vUZBk4hFqErRGcnKuF99S3EWCiUb21tAT/7YwCnEIsfKxApyqgcKHQO6z1lA5PAedHPUBKJU4hFxP0zBD9OIRbOIBbOIBbOIBbOIBbOIBa/AEQyr63rTKk/AAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 72x72 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"left_edge = tensor([[-1,1,0],\n",
" [-1,1,0],\n",
" [-1,1,0]]).float()\n",
"\n",
"left_edge3 = tensor([[apply_kernel(i,j,left_edge) for j in rng] for i in rng])\n",
"\n",
"show_image(left_edge3);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Convolutions in PyTorch"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([4, 3, 3])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"diag1_edge = tensor([[ 0,-1, 1],\n",
" [-1, 1, 0],\n",
" [ 1, 0, 0]]).float()\n",
"diag2_edge = tensor([[ 1,-1, 0],\n",
" [ 0, 1,-1],\n",
" [ 0, 0, 1]]).float()\n",
"\n",
"edge_kernels = torch.stack([left_edge, top_edge, diag1_edge, diag2_edge])\n",
"edge_kernels.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([64, 1, 28, 28])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mnist = DataBlock((ImageBlock(cls=PILImageBW), CategoryBlock), \n",
" get_items=get_image_files, \n",
" splitter=GrandparentSplitter(),\n",
" get_y=parent_label)\n",
"\n",
"dls = mnist.dataloaders(path)\n",
"xb,yb = first(dls.valid)\n",
"xb.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"xb,yb = to_cpu(xb),to_cpu(yb)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(torch.Size([4, 3, 3]), torch.Size([4, 1, 3, 3]))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"edge_kernels.shape,edge_kernels.unsqueeze(1).shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"edge_kernels = edge_kernels.unsqueeze(1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([64, 4, 26, 26])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"batch_features = F.conv2d(xb, edge_kernels)\n",
"batch_features.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAADdUlEQVR4nO2cyUorQRRAT5wVFJxHVARxgDgguHbjzh9wq1s/x1/RrStXggqCExFRcZFExQEnHBeP25W+rw0x6XQ/Hvesmkp1p7x9UnXrdmPi6+sLw1ER9wD+NSwgCguIwgKisIAoqvJ9uLq6+t8uQSsrK4mgdjNEYQFRWEAUFhCFBURhAVFYQBQWEIUFRGEBUeRN3eNCilaXl5cAvL+/A9Df3w9AQ0MDACMjIwAcHx8D8Pz8XPJ3myGK2A15e3sD4Obmxmt7eXkB4PPzE4BEInAfxuDgoO/ci4uLksdjhihCM+Tu7g5wv+/q6urAfufn5wD09PQAzgKxohDk2l1dXQA0NTUVMeJgzBBFSYaIFQD7+/sATE9PAz8bUggyZ/T19QVeq66uztcvjNVFMEMUJRlye3vrHW9tbQEwOTmZ9xy5u4Lc/YmJCa9NVp7t7W0ARkdHfee0t7cDLl+5urr69dh/wgxRhLbKPDw8AC67HB4eBqC1tRVwq4nkDpIzfHx8AFBfX+9dS1acjY0NwBkic8bQ0BDgDL2/vw/rzzBDNCUZkrv+z87OAlBZWVnUtWTeANjb2wP8qxjA2NgYADU1NQCcnp4W9V35MEMUJRnS3NzsHY+PjwP+Ox2EzA/6NQzJYAHW19cBmJmZ8fWRuUSucXBwUMyw82KGKEJbZXp7ewPbr6+vCzo/lUp5x5lMBoDl5WUAOjs7AWhpaQFgc3MTcCtbmJghCguIIvYC0dPTEwA7Ozte29TUFOBS9GQyCbhCkCR/5cAMUcRuiEymMpECLC4uAi5FF1N2d3eB3xWTfosZoojNENnsyRZfSooA3d3dvraqqj/DPDo68p1bDswQRWyGnJycAC5xW1hY8D6TzdvAwAAAh4eHQHnNEMwQReSGPD4+Am7FkEJSbuovZUhZTcJ4AFUoZogickMk35Ai0NLSEgAdHR1eH9nmS9/c0kC5MUMUkRuSzWYB94qD5BptbW1eHylMr62tRTw6M+QvIjdEHipJgVpyDnk8AW4HHObjhUIxQxQWEEXkPxl5S0Cq9ILUTaHwOmw5MEMUkRtydnYGwPz8vK+9sbHRO5aNXxyYIYrIDZH3Q15fXwGora0FoKLC3RtZduPADFFEbsjc3BzgEjIhnU57x7nvrEaNGaJI2D9D8GOGKCwgCguIwgKisIAoLCCKb79WEcYbcUyrAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 72x72 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"show_image(batch_features[0,0]);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Strides and Padding"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Understanding the Convolution Equations"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## Our First Convolutional Neural Network"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating the CNN"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"simple_net = nn.Sequential(\n",
" nn.Linear(28*28,30),\n",
" nn.ReLU(),\n",
" nn.Linear(30,1)\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Sequential(\n",
" (0): Linear(in_features=784, out_features=30, bias=True)\n",
" (1): ReLU()\n",
" (2): Linear(in_features=30, out_features=1, bias=True)\n",
")"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"simple_net"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"broken_cnn = sequential(\n",
" nn.Conv2d(1,30, kernel_size=3, padding=1),\n",
" nn.ReLU(),\n",
" nn.Conv2d(30,1, kernel_size=3, padding=1)\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([64, 1, 28, 28])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"broken_cnn(xb).shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def conv(ni, nf, ks=3, act=True):\n",
" res = nn.Conv2d(ni, nf, stride=2, kernel_size=ks, padding=ks//2)\n",
" if act: res = nn.Sequential(res, nn.ReLU())\n",
" return res"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"simple_cnn = sequential(\n",
" conv(1 ,4), #14x14\n",
" conv(4 ,8), #7x7\n",
" conv(8 ,16), #4x4\n",
" conv(16,32), #2x2\n",
" conv(32,2, act=False), #1x1\n",
" Flatten(),\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([64, 2])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"simple_cnn(xb).shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"learn = Learner(dls, simple_cnn, loss_func=F.cross_entropy, metrics=accuracy)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Sequential (Input shape: ['64 x 1 x 28 x 28'])\n",
"================================================================\n",
"Layer (type) Output Shape Param # Trainable \n",
"================================================================\n",
"Conv2d 64 x 4 x 14 x 14 40 True \n",
"________________________________________________________________\n",
"ReLU 64 x 4 x 14 x 14 0 False \n",
"________________________________________________________________\n",
"Conv2d 64 x 8 x 7 x 7 296 True \n",
"________________________________________________________________\n",
"ReLU 64 x 8 x 7 x 7 0 False \n",
"________________________________________________________________\n",
"Conv2d 64 x 16 x 4 x 4 1,168 True \n",
"________________________________________________________________\n",
"ReLU 64 x 16 x 4 x 4 0 False \n",
"________________________________________________________________\n",
"Conv2d 64 x 32 x 2 x 2 4,640 True \n",
"________________________________________________________________\n",
"ReLU 64 x 32 x 2 x 2 0 False \n",
"________________________________________________________________\n",
"Conv2d 64 x 2 x 1 x 1 578 True \n",
"________________________________________________________________\n",
"Flatten 64 x 2 0 False \n",
"________________________________________________________________\n",
"\n",
"Total params: 6,722\n",
"Total trainable params: 6,722\n",
"Total non-trainable params: 0\n",
"\n",
"Optimizer used: <function Adam at 0x7fbc9c258cb0>\n",
"Loss function: <function cross_entropy at 0x7fbca9ba0170>\n",
"\n",
"Callbacks:\n",
" - TrainEvalCallback\n",
" - Recorder\n",
" - ProgressCallback"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"learn.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.072684</td>\n",
" <td>0.045110</td>\n",
" <td>0.990186</td>\n",
" <td>00:05</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
" <td>0.022580</td>\n",
" <td>0.030775</td>\n",
" <td>0.990186</td>\n",
" <td>00:05</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn.fit_one_cycle(2, 0.01)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Understanding Convolution Arithmetic"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Sequential(\n",
" (0): Conv2d(1, 4, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))\n",
" (1): ReLU()\n",
")"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m = learn.model[0]\n",
"m"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([4, 1, 3, 3])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m[0].weight.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([4])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m[0].bias.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Receptive Fields"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-18 21:18:08 +00:00
"### A Note About Twitter"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-18 21:18:08 +00:00
"## Color Images"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([3, 1000, 846])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"im = image2tensor(Image.open('images/grizzly.jpg'))\n",
"im.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAPQAAAEeCAYAAAC9hziuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy7Sa9tW3bn9ZvlKnZ1invOLd+78V6896J2hNO2CKfTZEoIEiGlhJBoIHcovgE9vgA9Wkj0AClp0ExoICGBUjLglLEdcpQv/Kpbl+eealermNWgsW5GjwgpjIQUuqO7G3vvteZ/jvEvhhIR3tW7ele/G6X///4B7+pdvav/7+odoN/Vu/odqneAflfv6neo3gH6Xb2r36F6B+h39a5+h+odoN/Vu/odKvvrPvwv/8f/XCIDpUDjK9p6wUItON+fUZmKno75fMm96mPGuOYsnrHZrck6QrGIRDKR2rW0yiJW89Hp9/naycegEn/14P9iZluCRF5tnhNSRKNwlSelRG0altWKUgrKKESEkoUxBUbZTT8/CoPsUaIQCjH2+GqGVS3OG1b2AKfgatyQC4jJeGVwrqKxDVlnSimICKEfiCUy5B6Jkdn8kHm1wGqFyECWzEW3JpaMUzUFIZWBWjUoXVBYsokI0JoFferoho7aT99llEcVYZuuAA1ZaAykIhTlcRREMpXRtPUxXYhcp2scGW0qco4UHHM9Z1GvWNUn7LmmD1tKgbEMlFIgKo6XpxxXN1nv3/AyPGbMCS+eyjaMJTCzDUfVEu9aLsZn9GOmbZd888b3aJRmH17x8PI1Z/sLgoy01ZzT1X1C6Xl99ZRGGoJEvLEsm2P6uOdquEKAIiMhRXLM1L5i5pcs/Jx96ujKBZVyNPoQEcUwDFyXK5QovK1QSjDOogRSKiileH91g2V1QMlClxLPNi8REUQEJYVKN9P/diAiWGU5bqBtTomiuByuGGNHF/aoqNjna0RnvJmTVcEbD0VRKQPKolHsxh5rIpWrubG4g1GWddjQjR0xdRjjWFQrxnHEarg1axlyw+PdE0K/Y9msqAxYejIrhtjRy0DjFojSJCK6OAoZ0FRUqJIxVpFVYGaPEQWRgZALkjKbfo3Xmv/iP/yv1W8FaOccqkDxAspSu5qSMs45rPVUSmh0i1E9Ua7Z7S8QbZCs0CJIgaaZ4U2N1ZakMjEnkMD1/oycI8kkYhwIKWCKwRiHZI1SCldXRIkMYcAYReNn1JVnSD0GR8gDWRW0WMa8B9EkDXNbM7czKldRSmbIEW0VTdVQUkaKwlrDdthSWTeBi4RYTYgBbzxJazQKpRSpCDFqgmTCmNFWg84YNMbUOFOhNaSUkKKoXANFyDnhjMUrj9Ya7zzX/QUpJZyqwQox96AaRAt9LqSSwTpq2SBKcNZi8KRSiAJGgXiFtRarYBh7igKlFDkLKUUwsLAtmmsuxzNEDFYUxYEU4XR+By8KJWcMYSRnS+UVbbWksoUuCo+u9rx377scxcjV5gqvFR/e/JhXl89IdaYPa1bmiFk9Y2YqrGmY1we8ePMV+xwpklmtDokxYpVmjD2pdJSSMNaDNmRJUIHqFEZNR7FoQWeoaMiqQwv0ObJQgT6NdDmTiRjlEMkooylGYbImkyhFMN6AccSSSXkgpj27uEOJomiF1Q1aKTQGSkEAbTJWFzSWJAWtoSiHaIcuW5xvad2MIQ4IkThes8kdxnhQDZSeWiKtduwlU7SlKAsaXLHslWGMgTFdUldLSkk4ZfC+wmmHShplpmeQMuz0jto0FAWkkRBHSg7kUn77Di2qYEXRjQXrFW2p2eRr5m4GgLEtJhm2cc92SPRDRHRAi8VVBoxFa40Vh1KaVlucsQxj4eXVOX0Y0SjW3TVZErkEDJnKNtS6wmRLySNj6dBobPFoFI1tOB9eU1IhlpHKLpAM2ihqV6O1xXuPyZaYewoF7cFkAyJkX7janOHNDNPWkDWjypQh4VWF0opSQEShCxQU3dhRiGSVkJIRmS6pyjm8qvDOE13A54xRliFv0Vgq6zBGYbXDiGUIPaSeURdKzjjbUClLKgGFwdsaUHTB0JeBhbck3WJVggCSFbZYdISOLUPpMEVT+5ZMJJaRo/qYpat5tX9KnzqCCAZDrRzHs1PmbkVIV+zynJlpaWmoZjNO21soPWN5o+a9uuWk1mBq7swPqeuamg12ucRI4tnZyMc371NZQZG5Wd1iiDsuN+eQLPuwQyVo1AxrPX3YM5YRESGJx+hEZStSn1FKoZTgtEOU0NgWLZoyJowWxpDoXcfVuCZToYrCAMY5UgqMYY9zFZlC5SyKRMoBrTV92RPzCEmRJFObCmM03jpKhEH2iAheQ9QKJVBEYYyZJoASyTK9K8kOVRRaN4w6oVTEKkfOgcvUMyOikqKtG7RoVFGgagoah6YUQauM5IIgZAno3FDpCqxiLD2FTCRgoqMAznqiicTQQxkZfgNJ/rWAzrGQdaLICMaCShRR1KaiLwWXNBjNMHbEBLFkNBrlQLRCa9DaoyVT8kiSBqUUXnl245YkPSINXewZcsCgcH7689Z6hMxYAqkkLJYh92gBFKQSiSlQ6xaRjHUOoww5F6wYQghklSgUMBqHYxxHohpZby/wrsFajSShpExRCasNfRnRxb49VDCEEa3NNP7kiDOemPYoldDKk2KhbaZOnpMQcsAQJ5A5j+SC1pqQRoyqkAxJFA4NoihFKKag0BRJgEHlzKjK9Cz1knlVEXJCtEIZhfeOLAEliZwjqGlErXRNNpmlOwDZsY07jHHkcU8xkVmpkZjYqQtSHlnNTijRsaw8pzfucVC3CCNWBk6rhOnW+NURJu9oZE+6vsI7w8JsOGlqKtngc6R2c0wFvZ7z4a1P6GLP0zePaIwnlYhzjm5ULNtjdv0Go/x0OSeL0x5nPCH2VFVDo+fUVc0wdmQiqgh93LMZISeLs57a54kipRGtLZgyjepiiDmhlUGVQBctuxTYx3E6NMBYehb2ADKIShhrSSmRS4VTmpgFpRRaNFlnoiSKaEppKSVAyVhtidKQy0BWnpn1hCigB4rKOOWo3YyiIiGDFQW2QjQoDNoaVBFyHElmZCyGlAUhEEkoJUQJtMym55QHlCiSCBr32wM6yIhVGqcrWl2hdEAk048bsq3JIqiYIDm2rMlZEJVodYtVnpQLoQy4eoZSHqUUfdiyGS7ZpS2SNFkltNbUukGjptvQKIY4UNmGLl4RYsT41cS5ckSpiScVk0GDkPDWI2KYVzW1big5cZUuEFNoWWBKzVA6hthR9NQFsiTGFEllpI8dtW2myUQL82rOID1DN+KxjHECc9YJsKQSMfTMqzkhBGKJjERSjGQNxhisaEQLIopMJuiR+PaleVOQVBDR5JzxanrhYxkISlO5ChHFqr2BVY5tvKZnx5gKrvRoW6GsR+PIEsmqUFUVzlYc+QNebL8iFYu1cFTNcBhqNWOkJ/SRmVtxsvyA2fwIL5HD+ZxhuCYOa3TsUaEnXW0wQSBnrp89p8SEW1TUruZkMSO9+Jyzp+fc/9on6DsB26y420Zwc+bqW2grxJwIIbA8OmSz33H//reoKsdXDz+lqERJiSwJcYaZW2I1mKzxYvGmpQtbMJmrsaOxc0BhlaZPe4yxZAqFRCyZStdQHEopYq4ZciLEiFOeWCLKGiQKQ9jjbQUKSil463G6QiEok4ipB5mojqSBwgJra1RJGCJSpjbZ6AOsqihaU5uGPliwI754jDEoVaAIWjvI0zM3ygAK0ZpEwCiLwlB0R84FrRQocNqRVMaZSVex3lLLEmPMbw9orS2qGPq44a4/xdslzmqGMDDXNaUU+pzJhOmH++kgl1ImoYlMXVUYYyaOlEFXE/+7Wd9ma9ecHL3HUfiQF+efkuMIZqTr18zrQ1TRaOWxVpEkIKPgXIUpGqc8osFaTyHhlcfbBi8VWRJBBZyriDkxxIFoItpqGj/HRjeNU0qzDWvU2zi7iCHlnsZMD9mKQ5mR7dCRcoexDaZYBDC6QinYxzWohtbPIQnWG5RoGtWS9AgUdt2Ouq7JecT
"text/plain": [
"<Figure size 360x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"show_image(im);"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAp8AAADnCAYAAABYBj2NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9WawkWXrf9ztLbLnftW5VdVX1vrBnelYuw+EiUxIGJEVLliDR9osBG34zDBiQAQN+MJ8N2K9+NwEZhk0DFk1JpAhJpEgNOcOZYc9M9/RS1V1d213qbrlFxnbO8cMXkdktSy0DFkZS+35AofJmRp44cU5Exv/8v///CxVC4Cqu4iqu4iqu4iqu4iqu4scR+t90B67iKq7iKq7iKq7iKq7i/z9xBT6v4iqu4iqu4iqu4iqu4scWV+DzKq7iKq7iKq7iKq7iKn5scQU+r+IqruIqruIqruIqruLHFlfg8yqu4iqu4iqu4iqu4ip+bGE/7cPmv/6bwV/O0OMhTCao8Zjw5AlEEYxGqOdegpNDwt33oa4hSVhvPx5DXaO+9ouoF9/A/72/g4oiQlHAw4fSBoAxsLcnr72X92czKAr5bD6Xz5wjlBVqNIR+H4ZDUAqmU/m815PvGgMhQFlKn5ZLcA6yDLa3UVoTqgqePpXPQdqJY0hT2W9Zwu7uph/DISSJtDudyvbDobQbAjQNDAbSh4sLyHPpi3NgDOrgAK7dhNMjwmol+zw/hyxDPfciRDHhrTelP9aC1tLXvQPC0yMZx9EY9dVfJFQlfO+PCSfHUBSo69fh+jOEN78r/c9z1J1noTcg3H0XFgtCnsthWgu3bsk4n5zAZCJ9Xi6lP97LsSWJ9MN7iGPUT3xe+nj/LpydbbazVsa7rmEwQF2/KX9fnhNOT2G1AufwswV6NCAsc1S/t2m/qmR8owj17LMQJ4TDJzJ+SsFoBGWJevZ5iGLq3/2HmEGC2t+XYw0Bjo+lrSyT+coymY/RSMa/aeS9Xg8WC3nv4gKiCPs//Jb613Yl/VsSv/XB/xJKV2K0YSuZADCvFlhtmCQTYh1TuILT1SmegEZR+ZpB1Mcoi9WGm/2bbCe73JvdBaDxNRflJbGRa1YrQ9/2CAQUCq0MebOk8Y7YRMyrBQA+OJrg6NseiUmITUwIgcpXACQmQaGITUzta3zwLOscpRRFU9CPeqQmxWpL5SpKV+KCxweHUgqrLEYbnHe44EltSulKfHAkJsEqiyewavJ1v0MIGKUpXElmU4yyBDyVq1FKTocQAoOoT2pTfPCUrgSgdCVaacbxGB885+UFGtW2rUltilV2fSz9qM9Wsk3lSo5Wx6yaHKss/WjAdrLF4+VjAoFlnXPQu0ZqU05XpxSuoGj3aZTMWwiB0pUopUhMwqrJKV2FCw6rLLGJsMrigscozXa6jVGGy2rKsu7mQ45dKYVCE2lLP+oT6Yja1yzrJZWvyBv5jcpMum5PK4NSisbX1L5BobjRv44Pnnk9X89LYhKcdwzjIUYZPpo/QCnFIBqgUSil2jkKxCamaAp6NsMjfatchVYGqw2pkfFfuYJFvcAqy994/j/6TF2z/8nf+X44mxds9RN2hjGT1PJoWmKUYrtv+cqNIT88XvLeyZKicgxSy9m8ZDKIiY1wR7/+xnUOhin/+9uHDGLDonLcO1kSmQ23dHsnQyvwAaxSTEvH5bIiiQzTXK7HonLMi5r9ccYoi9jpy/V+kTcAZJEmizVWKTyBog4sSseqaliWDVv9hGujiKoJFI3nbF5RNg6jZcpia0gjQ+08q6rh2jjlfFFRNZ5RL2KYWFa152S6opdYQpDbgNGKonZYrdkfJVyuapZFg9ZKbt1G88Jejxe2Ux7NSi5XDucDTxcVw9TypRt9ah/41oM5Ze1JIk1kNHv9iK2e5WRRs6wcz0wSfub6FoVz/NGDC85XDXnl+IlrPZ6dZPzue+ckkSavHJ876LGVWb71cMFlXpEXDQHQSvHywQAf4N3DGaMs5vZOxrx0nM5LisoRW40xisQa6sYTR5rXDvqME8P9i5KjWUkI4Hwgiw1aKRrvGSaWyCh2+pbLleNwWlI2jrJyXOYVu8OUZdkwSC1aK4yS6y0vG2rnef3miH6seTytuMwrQoBRFhGCjPGd7YTf++EJwywiiw2NC2wPEo6n8nuwPUiY5RXDLKJynmEaUTuP9wGlFDcmCU/nNYHA6azEec//+Z//5L/0ev1U8Im16F4q4CJNBcSMRmtQwmJOuLwgrFaowUCAgtECnkAAWFMT5hdy49/aEoAQAuszy1ppvwUqTCYCFpyD5ZKwKlBxRCgrfO0wHfDsQE8cC5hpGmlzNtsAE5D/td7sD+DyUgBmB1S1lnbiWNocj6W9ppG2OnBzciLHrrWA4o/vI4qkH94LEC0K+ZdlhBBQ3hGqCpVlhIuLDdDc3Sd8/zsCxFrAqp59DuIUlgtUmglgHYxAGzh+CC+8ito7gFUOL7+BSvtwekI4OkS9/BpcfwbqCnXnOcKP3gJAbW3JmDon+2oaGY+mkT5389UB6jgWUKqUjFVZyt/eb4C4c5tFhLVtm5owm8l8t23q0QC0FuD58fEKQebdGEJdo/pDGeeiEADaNPLZ9AJ18zZ2eyjvddG24S9n6G4uOtDanVtd/7Nsfc6EssJPl/+Kk//fzeiAmRZYSOFWAsS0AK9lk1O6kiY4UpNSuAKjDD4EUiMjolDkzZLSlfRsRhFkzEMILegz1L5Zg8vEJBhlUVpAow8OT8AHj1byW2C0tN2EBqMskbY0wRHrmGWdE2kBbalNKZpi3d8O4K5cQdEUawAMAvhiHVOGksxm1L4hhIBWZv152axovMNqg/MNSilckJvEZsw8Udu/whUoFKHtf+1r2YcrqX1NpCOstpyuTnG+IShDbCL6tk9AgLUA2kCsYxlbPAe9Axpfo1BMki0iHXNZXXJWnHFzcIPMZEQ6Zjvd5oPZhwCM4xGNdzKeweNCQ6wEwMc6pvGO0I6xQrdj7jBK44KjdBWNr9dzqhXrsenmRKFofMOqKdbgPm3n0wf3iXHqxlQhALjyFbGOsToiuIrC1QLu28VCYhIiHbVzrgXIoqh9Q+MbrJa+dMCz8Y7M9lqAW9OzfXzw63Nv3sz/dVwi/9ZFGhnKxpFFmmnpGCQC9Pux4e75iqN5tQaeRe2IrFxTWWxxPpA3DdNVzbLypFZzOC2pG4/VGmsUxmiqJjArGhofuDaMSa0iiQyzVc2qci3ACVit2erHTDKL0bAsBaz1Y43zgRDgsmxIraZoPNYofIDEGgLyG5xazUXerIFnCAjYijSDxHCRe8a9mNoFmhZgKRRZrDmZlVijqZ0nLxt6iaXxcg5GVnOR1zgf6KcW7wPTvMakso/aB+aljONF3uC9gONeZPmn96c4H0gjgwuB1w96aAXff5Lz/G5K7QL7/QitYFHX/OKdbTyBeVVzvZ8RGc3nrhe8dZTzk7cHjGJLz1peu5bxB+8VBGBnmAgX1e63cdLP2gWK2tE4T2xlHLVWDFLL6byUfhay/dmy/gTo9u19zflAZNR6vC/yhrwUAJ7FAjbltZHbaukYZnI8AMM0Yl46itozTAyLQvPRyZzTyLAzTMhiy73TFYPUrs+tVdWglOw7L5tPLGaGaUQayd/aKlZVQwhgjWJVeRZFTd34Tz3vP/3+2wGiqkL1eoTpFDUYEJyTzxdTmM1wswK1qjD9VJjJLBOAlqaQpHD6RICktcIcNo20m6YbwDedCjvVAg7KkrBcokZDwmKBylJM1H5PqQ0QiiIBF+fn8r2dHWlzOt0AQrP5wQ15Ltu3r1WSyPZZJmCl3xdQBRs2tKo+waaGukGlifS3GyMQVnAwkH1qvX5ftcfTAS3iWN6/fkMAZFVtWLrdXQGeVcv8bu+ixluo175KKFdw/Vn0tVtgIkI+QyU9Ql3Ca19Ebe/BredRox1QipD1UGkG774tx+Gc9KHrs/cbcL1YyPtbW7Jf79fbhbMT6X8H8rtzAzZj1TSwWhKmFwLMu+26ee6+04H9jgGOItnWOajKjy1sFrLPrS3Ic8L
"text/plain": [
"<Figure size 864x288 with 3 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"_,axs = subplots(1,3)\n",
"for bear,ax,color in zip(im,axs,('Reds','Greens','Blues')):\n",
" show_image(255-bear, ax=ax, cmap=color)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## Improving Training Stability"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"path = untar_data(URLs.MNIST)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"Path.BASE_PATH = path"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(#2) [Path('testing'),Path('training')]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"path.ls()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def get_dls(bs=64):\n",
" return DataBlock(\n",
" blocks=(ImageBlock(cls=PILImageBW), CategoryBlock), \n",
" get_items=get_image_files, \n",
" splitter=GrandparentSplitter('training','testing'),\n",
" get_y=parent_label,\n",
" batch_tfms=Normalize()\n",
" ).dataloaders(path, bs=bs)\n",
"\n",
"dls = get_dls()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAOsAAAD4CAYAAAANSBHgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAdtUlEQVR4nO2deZQV1dHAf5d9QFYJm2ELIhHBJSibfEBYAi5gCKCicFDCEsBEEQ2iAoIQDIugAkERzCdGXEI0iDlGAcNOHAwIKIuon0GBGFQQEFCgvz+G6u7HvHnvzdu671C/czjT9O3XXXNf11TdunXrGsdxUBQl/BQLWgBFURJDlVVRLEGVVVEsQZVVUSxBlVVRLEGVVVEsQZVVUSwhdMpqjOlnjHnXGPO1MeaYMWa7MWakMcYELVuYMcbca4xZf6bfDhpj1hhjugYtlw3Y8s6VCFqAKHwBPAzsBE4A/wPMAU4CjwUoV9jpACwAcoFjwCBgqTGmneM4awOVLPxY8c4ZGzKYjDGvADiO0yNoWWzCGLMVeNNxnJFBy2IbYXznQucG+zF5NAeuBt4OWh6bMMYUA8oDB4KWxSbC/M6F0Q3GGFMR+BwoBRQHxjuO83iwUlnH/UAlYGHQgtiADe9cKJUVOAxcDpQFWgOTjTF7Hcd5Olix7MAYM4w8Ze3uOM5nQctjCaF/52wZs44G7nQcp0bQsoQdY8w9wHjgBsdxlgUtj62E8Z0Lq2U9m2JA6aCFCDvGmAnACOBax3FWBi2P5YTunQudshpjxgOrgY+BkkBbYBTwTJByhR1jzExgCNAH2GmMEYtwzHGcQ8FJFn5seedC5wYbY2YA3YALgOPkdeACYK7jOKeClC3MGGMK+iL/13Gc27Ipi23Y8s6FTlkVRYlOqOdZFUXxUGVVFEtQZVUUS1BlVRRLiDd1c65Hn1JZIqV9lzzad1FQy6oolqDKqiiWoMqqKJagyqoolqDKqiiWoMqqKJagyqoolqDKqiiWELr1rEp6OHUqb2XX9u3b3XOrVq0CYPfu3QBs3rwZgP3797vXLFiwAICWLVtmRU4lcdSyKoolxFvPGkja1+nTpwH4+9//DsDSpUvdtjlz5hT6frm5ue7xT37yEwCKFUvo71SoU+aOHz8OwK5du9xzDz/8MADr1q0DIq2mfNexCs1feOGFAGzZsgWA0qWTrmwSqr6T333u3LkADB8+vMBrhg4dCkCNGoUrvyTvVqdOnQAoU6ZMcsJquqGi2I0qq6JYQuBu8MGDBwGYP3++e+7JJ58EvEBIOnn88by6zXfccUcilwfmyp08eRKAd999N1+buKi///3vAfj444/dNnFxq1WrBsDNN9/sttWqVQuA888/H4DatWsD0LWrt3+VvA9ff/01ABUrVkz2VwiVG/z9998DKbmmCfPQQw8BMGbMmGRvoW6wothMYFM3R44cAaB169YA7NixIyhRQsmaNWsA6NChg3uuoMDQdddd5x6PHz8egEsuuQSIHSCS7yAar7/+OgC33HJLghKHmxIl8l51CSzNmzevwGvFCvv7WzyORAoMPv/880BKljUqalkVxRICs6zFixcHoEqVKkl9PicnB4DvvvsO8JIA4lGhQoWknpctvvjiCwD69++fr03+qv/2t78FvLFm+/btk3rWX//614j7+rnhhhuSumdYESspMQv5GY133nkHgJo1a7rnDh8+DMCUKVMAWLiw4P2+JDaQbtSyKoolqLIqiiUE5gaLGys5qJJxE+2aCy64wD0nLqC4JZKxI1NA0ahUqZJ73Ldv31TEzjhvvfUWAJ99lrdTo3/qZOrUqYAX9ClbtmxanukPpPTo0SOt97aR5s2b5zsnWXAydIhGo0aNAO97SjdqWRXFEgJfdSOWsUmTJu65Rx99FIDp06cD0LlzZ7ft0KG8DdHatGkDxLaowsaNG93jBHOCA6N8+fKA501IfjTAxRdfnNZnffvtt/nOSQAuVv5wUUUCbcuXLwfg17/+tdu2Z88eAI4dOwZ4/TNkyBD3mokTJwJQuXLljMgX7jdXURSXwNMNE+HEiRPucatWrQBvLWY0fvCDHwCwaNEiIHJqo5CWNVQpc+lCUgkbNGgAeN4KeFYl2ekgH6HuuwMHDgCwYcMG99zixYsBePbZZ+N+Po39FA1NN1QUmwl8zCqpXf51l8Lnn38OeCl0ULBFlcR1gEceeQSITNVTPKRihIz369Sp47ZlyFIEwjfffOMeb9q0CYBp06YBsHbtWiDSq4jFNddcA3hJ+s2aNUuXmAmjllVRLEGVVVEsIXA3WNzfunXrJvX56tWrA7B69Wr3nJQmUaKzfv16wJt+6NmzZ5DipJ033ngDgFtvvdU9l8gU39lIggpAu3btAC+nPQjUsiqKJQQ+dSOTzcla1q1btwLe+s00E+rph8IiK3okLU4CMBJsgbSWIM1638nKq1KlSqXwaA9/VQkJcvbq1Qvw3tcMJY/o1I2i2EzgY9Zkkb9wYiWU+MiUllhUmaa57LLLghIprUjCy4ABAwCvYHmySKlXgFGjRkX8lCmgESNGpPSMwqCWVVEsQZVVUSzBugCTuCH33nsvkHxZmASxPsDkz32V4nSCFPbylytNI4H1nbj5L774YoHXSNmacuXKueekPwQpWgfwt7/9DYCvvvoK8AJL/t0errjiilTE9qMBJkWxGessq+S1ynrWDGOdZZUkEwmOdOnSxW378MMPAa+v33//fSBjVSGs67tY7N27F4DGjRsDXqUSSTCB6BUmkkQtq6LYTMqWVdYA+re6kHFlIkgamH/6QKxtNKQm0UcffQScW2NW2V1Pdo2T9brLli1zr5GqGLL9RoRAZ75rGafdfvvtgFfXCuCHP/xhusQNVd+li7Zt2wJeIsmvfvUrt2327NnpeoxaVkWxmZSTIsSXf+CBB/K1JWJht23bBnhRtniIJS1ZsmSiIlqJVHNYsmSJe06im/66TBBZpDuR9DepvSSWwG8R7rrrLsCztrJQwkb83oW8Z5KWWtj3R1IZ/VVLIHodq0yhllVRLEGVVVEsIeUA0xNPPAHAnXfe6Z4777zzAC/o5C+5IowbNw7wyo0m6k7INvODBw9O6PoUyXqQRNZiSonLWMG2WEjutD8/Vr7rWbNmAZ5bLfu9+q+pWrUqAP369XPbxDWuUaNGIiIEFmCS/WqHDRvmnpMCZ7Lf7aWXXlqoe0qChKxrFXTqRlGUfKRsWWWqINpfFdm2ItqkuwRQpGhyNOrVqwdE/nUfOXIkkLXd4LJiHSZNmuQeT5gwAfCCI4kEj+rXr+8ei6fSrVs3IHbpVSlW57cOMhWxc+fOfNfLthB33313gff0kXXLumLFCgD69OkDeOVGAbZv3w7ARRddVODnjx49GnEfWVkDXoDq7IoTalkVRclH2tINZesAgLFjx6YgkpcOJ3toipUIgIxaB0md9G8PcnYyQzTLKltsSFLD5MmT3Wv81Q2SQayRWGiZmgN46qmngNi7qfvIumWVVMBoXsHo0aMBL56yY8cOwEssAa+vY+31Kx7dH//4RwC6d+/utqWxaoRaVkWxGVVWRbGEtLnB/kCRf41govh345Kq5yHInsmoK7dv3z4AGjZs6J6TfrzxxhsBqFmzptsme8tKBX2ZXgkpWXeDpcSPP089XcgeS/PmzQPSv6PfWagbrCg2k7aCabJLOXgJDjNmzACi5w0PHDgQ8Eo8+ifaz5W9QcVqHjlyJGBJigZXX301ENuy9u7dG0isdK0/AaJFixZAwsG1jKCWVVEsIfBKESGnSK7JzBLad8mjY1ZFsRlVVkWxBFVWRbEEVVZFsYTQKWu9evUwxuT7l6Fd4ooUBw4cYOjQodSqVYvSpUtTv359d/2vUjBTp06lVatWVK5cmUqVKtGmTRt3XXGYCN3GVLm5uRGJ1EePHqVp06aZqhpfZDhy5Aht27blggsuYNGiRdStW5d9+/a5y+CUglmxYgUDBgzgqquuIicnh3nz5nH99dezcuVKd+42DMSbugkcY8wgYA5Q13GcvfGuP1cxxowH+gONHMc5Ee96JTbGmK3Am47
"text/plain": [
"<Figure size 288x288 with 9 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"dls.show_batch(max_n=9, figsize=(4,4))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### A Simple Baseline"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def conv(ni, nf, ks=3, act=True):\n",
" res = nn.Conv2d(ni, nf, stride=2, kernel_size=ks, padding=ks//2)\n",
" if act: res = nn.Sequential(res, nn.ReLU())\n",
" return res"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def simple_cnn():\n",
" return sequential(\n",
" conv(1 ,8, ks=5), #14x14\n",
" conv(8 ,16), #7x7\n",
" conv(16,32), #4x4\n",
" conv(32,64), #2x2\n",
" conv(64,10, act=False), #1x1\n",
" Flatten(),\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
2020-08-21 19:36:27 +00:00
"from fastai.callback.hook import *"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def fit(epochs=1):\n",
" learn = Learner(dls, simple_cnn(), loss_func=F.cross_entropy,\n",
" metrics=accuracy, cbs=ActivationStats(with_hist=True))\n",
" learn.fit(epochs, 0.06)\n",
" return learn"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>2.307071</td>\n",
" <td>2.305865</td>\n",
" <td>0.113500</td>\n",
" <td>00:16</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn = fit()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAs8AAADWCAYAAAAuNG/NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd5iTVfbA8e/JVGDoTUBhpAsiKIgiXVEUdUFdy9p2/a2La1l7wY6uda1rWRV77wUVFQsgvQzSOwy9w8AwM0xN7u+PN28mySSZZCZTkjmf58nD5K03wyTvyX3PPVeMMSillFJKKaXK56jpBiillFJKKRUrNHhWSimllFIqTBo8K6WUUkopFSYNnpVSSimllAqTBs9KKaWUUkqFSYNnpZRSSimlwqTBs1JK1XEiMkxEjIgcWdNtUUqp2k6DZ6WUikMicqQ7IB5W021RqjYQkUtEZJOIHBSRt0UkyWtdgojME5GLa7KNKjZo8KyUUkqpuCYizYG3gXuBwcApwFivTW4FdhhjPq2B5oVNRJLj8VyxRoPnOkJEponImyLyiIjscX/zflREHCLygIjsFpG9IvKo1z6JIjJeRDaKSIGIrBCRa/yOe5OILBaRXBHZJSKfiEgbr/X27eDTRWS6iBwWkZUiMrI6X79S8UpEBonILBHJcT+WuN9fW92bTHW/Bzd57fMvEdnmfj9OBtrXRNuVqkYdgWxjzIfGmGXAN0APABHpAtwEXBvOgdzXxfUiMlpEVotInohMFZFOftv1FZGf3dfHvSLylYh08Fp/tHvZDvd7cZmIXOF3DPva/W8R2QlsD9Kmae73uf/jb17b/Mvd3gIRWSci94pIotf6Te4Y4X8ish+Y5V7exn1tPygi+e5z9QvndxWvNHiuW/4MJAGDsL5l3wN8D6RhfRO/HbhHRM5yb/8GcD5wDXAM8DDwpIj83e+4twO9gPOwLsKfBDj308BjQG8gA/hURJpE7ZUpVQeJSALwLTAPOMH9GA8cdv8McAHQBjjRvc9o4DngWaAP8BnwVHW2W6kasB6oLyL9RKQBMBRYJCICvAncY4zZFcHx2mAF25dh9WI3Ad6yV4pID+B3YA7QDzgVcAK/iEiqe7M04DfgTKxr6ATgbREZ7neui4CWwGnu4wRyvrtN9uMRIA9Y4G7PeKxr9d1Y1/ObsK7tD/od50ZgDzAA+Kv79/MN0B04B+gP7Ha/jhYhf0PxzBijjzrwAKYBi/2WrQCW+S1bghXoHg24gO5+6x/wP47f+uMBA7RzPx/mfn6+1zZHuJeNrOnfiz70EcsPoKn7vTQswLojA60DZgIf+i172r3tkTX9mvShj6p6AOe6r3GZWF8gE4B/AZPc16Vv3OveA9JCHGc8UAK09Fp2ifuamep+/g7wid9+KVhfbMeEOPZE4HWv59OAtYAjgtc5EigEznU/r+8+75l+210JHPR6vgn4zW+b09yfDT38XsdO4IGa/j+tqYenu17VCUv8nu9yP/yXtcL6pixAhvXF0yMR69szYKVlYH2T7YH1zdu+m9EB39tLi+0fjDG7RMQJtK7g61BKAcaYAyLyBjBZRKZg9XR9bYxZE2K3HsDHfstmArdVUTOVqhWMMd8B39nPRSQdGAecBLwArMK6Q/sBcD9wV4jD7TDG7PV6vh3rmtkK2IJ1p6eziOT67ZcKdHGfvz5Wh9S5WL3FyViB6VS/fRYaY1zhvEYR6Yl1N+ku9+sF6AnUA74UEeO1eQKQKiItvV7LfL9D9gT2G2NW2guMMYUiMs+9rk7S4LluKfZ7boIsc1AaBJ+C9Y3VfxtEpD3wA/A+VkrHPqzerl+xPgS8FQVoj6YNKVVJxph/iMh/gTOA04F/i8gNWL1pQXerlsYpVbu9Dow3xmwTkRHAQ8aYEhH5AHionH39r2n2e8rh9e/7wBMB9t3v/vcpYDTWF9fVWGkWzwCN/bbPK++FAIhIK6xUzA+MMc97rbLbdCFWL7a/rHLOFejzQoIsrxM0eFbBLHT/294Y832QbU7E+jZ7szEmH6wBEtXROKVUKWPMcmA58KyIvIpVReBr9+oEv81XAgOB/3ktG1jljVSqFhGRfwBijHndvciBNSYIrM6fynbuZADHARuMO9chgCFYKVSfutvkALpi5RRHRERSsN7zq7Hylr2tAAqAjsaYHyI89AqghYj0sHuf3efqj+9nSJ2iwbMKyBizXkTeAl4XkTuxBj00APpi5Xk9CazD+uZ5m4h8iDUY8IGaarNSdY2IdAb+gXUreivQFmvw7x9Yd4JygTNEZAVQaIw5gNWz9bmIzMe6czQIuCLA4ZWKSyLSDmug3CCvxdOBW90Vp67HSoGqjMewUiA+cN8Z2gukA2OA/xpjMoE1wGgR+RLrvXor1ns44uAZeM2971VAS690y2xjTK6IPAY85l7+C1b81ws43hgTKj1livt1fCQi1wPZWCktqcArFWhnXNDb5iqUsViDKu7F6q36Dfgr1oAKjDFLsQZbXONefztwc420VKm6KQ8rf/ITrNuxXwKzgRvcOZLXY43U3wosAjDGfI11m/hOYClWtYBQF0+l4s1rwBPGmE1ey27ECm4zsN5X5aVthGSMWYWV9pgGTMa6Rr6Odbf2oHuzW4DNWDnOv2HlTX9RwVMOw2r/GqzBfPbjYnd7/u0+39VY459mup9vKud1GKyAfzVWKtgCrMGVpxtj9lWwrTFPgt9NUEoppZRSSnnTnmellFJKKaXCpMGzUkoppZRSYdLgWSmllFJKqTBp8KyUUkoppVSYNHhWSimllFIqTDFV57lFixYmPT29ppuhVK2xcOHCfcaYljXdjkD0/aqUr9r8fgV9zyrlL9h7NqaC5/T0dDIyMmq6GUrVGiKyuabbEIy+X5XyVZvfr6DvWaX8BXvPatqGUkoppZRSYdLgWSmllFJKqTBp8KyUUkoppVSYNHhWSimllFIqTHEVPK/bncMn87fUdDOUUlHgdBle/G0dhwqKa7opStV5uYUl3PnFEmas21vTTVGqxsVV8Hz6c9MZ99Wymm6GUioKfl21m2d+Wcsj36+s6aYoVecZY/gsYxurd+bUdFOUqnFxFTwrpaqOiEwTkQIRyXU/1lTl+QpLXADkFTmr8jRKqTCkJiUAUFCs70el4jJ4drlMTTdBqXh1gzEmzf3oVtONUUpVj6QEB4kOoaBEg2el4jJ4dhoNnpVSSqloqpeUQH6Rq6aboVSNi6vg+ZROzQFroJFSqko8LiL7RGSWiAzzXykiY0UkQ0Qy9u6NzsAiicpRlFKVlZKUoD3PShFnwfPwbq0AKNHgWamqcBfQEWgHTAC+E5FO3hsYYyYYY/oZY/q1bNmyUiczegdJqVolJdFBYbH2PCsVV8FzgsPqo3I69aKrVLQZY+YZY3KMMYXGmHeBWcCoqj6viPY9K1UbJDgEl36pVSq+gufEBOsiW+LSb8ZKVQODZlUoVWc4BA2elSLOgmdPz7OmbSgVVSLSRERGikiqiCSKyGXAEGByTbdNKVU9HA7R66tSQGJNNyCaEh12z7O+uZWKsiTgEaA74ARWA2OMMVVa61kpVXs4RNCOZ6XiLHhOcFgd6frNWKnoMsbsBU6swfOzI7uAxvWSSEuJq48tpWJGgmjOs1IQZ8Gz9jwrFX/mbNjP0Xf/AEDX1mn8fMvQGm6RUnWTiHZOKQVxm/OsAwaVihf7cgs9P6/dncumfXm8P2cTxc6Kv8+LnS427M2NQuuUqjusahs13Qqlal5cBc/a86xU/Ah2d3jY09O4f+IK+j/6a8j98wpLWL3rUMB1j05axWnP/M7O7PzKNlOpOsOhaRtKAXEWPNs9zyVa51mpuHfgcHHI9Td/upgzn59BQXHZGdHmZu63jpEX+hhKqVJaqk4pS1wGz5qTpVTsM4T/Ps7OL2bt7hzenrURsILjX1buBuDxH1aRXU6grZQqn5aqU8oSleBZRJqJyNcikicim0Xk0hDbDheRqSKSLSKbonF+W4KmbShVpzz03QryCkvo/dDPnPHcdB76biUul+GSCXM927w7ZzMPf7+yBlupVHzQUnV
"text/plain": [
"<Figure size 864x216 with 3 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.activation_stats.plot_layer_stats(0)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAs8AAADWCAYAAAAuNG/NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5gcVZ3/8fdnLplJMgmQMAQFQwQTlShhddSVGBfvsKsPuKy/VbKAuhgXFsH1yrqiiK6iq+yqD4Kw3OSqKyCweEEFlODvJwY1YBQCAlHuE0JCZshtZr6/P6o6KZqemZ5MzXR39ef1PPVk+tSp6m/3zEl/+9Q5pxQRmJmZmZnZ6FpqHYCZmZmZWaNw8mxmZmZmViUnz2ZmZmZmVXLybGZmZmZWJSfPZmZmZmZVcvJsZmZmZlYlJ89mZk1O0sGSQtLetY7FzKzeOXk2MysgSXunCfHBtY7FrB5IeqekByStl3SBpPbMvlZJv5T097WM0RqDk2czMzMrNEmzgQuAfwOWAAcByzJVPgQ8HBHfrkF4VZM0pYjP1WicPDcJSTdLOk/S5yQ9nn7z/ndJLZI+JekxSb2S/j1zTJukUyXdL2mzpFWS3l923pMk/VZSn6RHJV0h6TmZ/aXLwW+S9HNJT0v6vaS3TObrNysqSa+RdKukjem2Mm1ff06r3JS2wQcyx3xA0oNpe/wRMLcWsZtNon2BDRFxaUTcCXwP2B9A0nzgJOC4ak6Ufi7eK+kwSXdJ6pd0k6T9yuq9XNIN6edjr6SrJO2T2f/8tOzhtC3eKemosnOUPrs/K+kR4KFhYro5befl27szdT6QxrtZ0j2S/k1SW2b/A2mO8A1JTwC3puXPST/b10valD5XTzXvVVE5eW4ufwe0A68h+Zb9CeB/gS6Sb+IfAT4h6dC0/n8Dfwu8H3gxcBrwRUn/WHbejwAvBd5O8iF8RYXn/jLweWARsAL4tqRdc3tlZk1IUitwLfBL4GXpdirwdPozwBHAc4BXpMccBvwncAZwIPAd4D8mM26zGrgXmCapR9J04K+A30gScB7wiYh4dAznew5Jsr2UpBd7V+D80k5J+wM/A/4v0AO8HhgEfiypM63WBfwUOITkM/Qc4AJJryt7rv8DdANvSM9Tyd+mMZW2zwH9wK/SeE4l+az+V5LP85NIPts/XXaeE4HHgVcDx6Tvz/eAFwFvBV4JPJa+jt1HfIeKLCK8NcEG3Az8tqxsFXBnWdlKkkT3+cAQ8KKy/Z8qP0/Z/r8AAtgrfXxw+vhvM3X2TMveUuv3xZu3Rt6A3dK2dHCFfXtX2gcsBy4tK/tyWnfvWr8mb94magPeln7G3UfyBbIV+ABwffq59L1037eArhHOcyowAHRnyt6ZfmZ2po8vBK4oO66D5Ivt4SOc+xrg3Mzjm4HVQMsYXudbgC3A29LH09LnPaSs3tHA+szjB4CfltV5Q/p/w/5lr+MR4FO1/p3WatveXW9NYWXZ40fTrbxsD5JvygJWJF88t2sj+fYMJMMySL7J7k/yzbt0NWMfnnl56belHyLiUUmDwJydfB1mBkTEk5L+G/iRpBtJerqujoi7Rzhsf+DysrLlwIcnKEyzuhAR1wHXlR5LmgecDLwK+BrwB5IrtJcApwAfH+F0D0dEb+bxQySfmXsAfyK50vMCSX1lx3UC89Pnn0bSIfU2kt7iKSSJ6U1lx9weEUPVvEZJC0muJn08fb0AC4GpwJWSIlO9FeiU1J15LbeVnXIh8ERE/L5UEBFbJP0y3deUnDw3l21lj2OYshZ2JMEHkXxjLa+DpLnA94GLSYZ0rCXp7foJyX8CWVsrxONhQ2bjFBHvk/RV4M3Am4DPSjqBpDdt2MMmJTiz+nYucGpEPCjpjcBnImJA0iXAZ0Y5tvwzrdSmWjL/XgycXuHYJ9J//wM4jOSL610kwyy+AuxSVr9/tBcCIGkPkqGYl0TEf2V2lWJ6B0kvdrl1ozxXpf8vNEx5U3DybMO5Pf13bkT87zB1XkHybfaDEbEJkgkSkxGcme0QEb8DfgecIelsklUErk53t5ZV/z2wGPhGpmzxhAdpVkckvQ9QRJybFrWQzAmCpPNnvJ07K4ADgD9GOtahgteSDKH6dhpTC7CAZEzxmEjqIGnzd5GMW85aBWwG9o2I74/x1KuA3SXtX+p9Tp/rlTzz/5Cm4uTZKoqIeyWdD5wr6WMkkx6mAy8nGef1ReAekm+eH5Z0KclkwE/VKmazZiPpBcD7SC5F/xl4Lsnk31+TXAnqA94saRWwJSKeJOnZ+h9Jt5FcOXoNcFSF05sVkqS9SCbKvSZT/HPgQ+mKU/9MMgRqPD5PMgTikvTKUC8wDzgc+GpE3AfcDRwm6UqStvohkjY85uQZ+GZ67HuA7sxwyw0R0Sfp88Dn0/Ifk+R/LwX+IiJGGp5yY/o6LpP0z8AGkiEtncBZOxFnIfiyuY1kGcmkin8j6a36KXAMyYQKIuIOkskW70/3fwT4YE0iNWtO/STjJ68guRx7JfAL4IR0jOQ/k8zU/zPwG4CIuJrkMvHHgDtIVgsY6cPTrGi+CZweEQ9kyk4kSW5XkLSr0YZtjCgi/kAy7LEL+BHJZ+S5JFdr16fV/gVYQzLG+ack46a/u5NPeTBJ/HeTTOYrbX+fxvPZ9PmOJZn/tDx9/MAoryNIEv67SIaC/YpkcuWbImLtTsba8DT81QQzMzMzM8tyz7OZmZmZWZWcPJuZmZmZVcnJs5mZmZlZlZw8m5mZmZlVycmzmZmZmVmVGmqd59133z3mzZtX6zDM6sbtt9++NiK6ax1HJW6vZs9Uz+0V3GbNyg3XZhsqeZ43bx4rVqyodRhmdUPSmlrHMBy3V7Nnquf2Cm6zZuWGa7O5DtuQNEvS1ZL6Ja2RdOQw9STpi5KeSLcvKXM7HDMzMzOzepR3z/OZwFZgDnAgcL2klRGxqqzeMpI71iwiub3zj0nuWnd2zvGYmZmZmeUmt55nSdOBI4BTIqIvIpYD1wJHVah+DPCViHgwIh4CvgK8O69YzMzMikTSCZJWSNoi6cJR6v6LpEclbZB0vqSOSQrTrCnkOWxjATAYEaszZSuBhRXqLkz3jVYPScvS/zBW9Pb25hasmZlZA3kY+Bxw/kiVJL0FOBl4AzAP2Bf4zEQHZ9ZM8hy20QVsKCvbAMyoou4GoEuSIiKyFSPiHOAcgJ6enmfsK3fuz+9jy8AgJ7x+/lhjNyscSSeQXNF5KXB5RLx7mHrHACcC84GngMuAT0TEQLr/ZuAvgYH0kIci4oXjjW9gcIhv3PxHjl3yfKZNaai5y2aTLiKuApDUA+w9QtVjgPNKwyUlfRa4lCShzs3ZP/sjf3y8L89Tmk2qQ1+6J69/0ZydOjbPT6w+YGZZ2UxgYxV1ZwJ95YnzWP38nl76tgw4eTZLlHqq3gJMHaHeNOCDwC+BbpLhVh8BTs/UOSEi/jvP4K76zUOc8ePVPLVpG5986/55ntqsmS0Ersk8XgnMkTQ7Ip4oryxpGck8JObOnVvVE2wbHOL0H9zFjM42ZnT4i681ppfuvctOH5vnX/1qoE3S/Ii4Jy1bBJRPFiQtWwTcNkq9MRtf+m1WHNX2VEXEWZmHD0m6FHjdBIfHloEhADZtG5zopzJrJpWu7EJyFfhZyfNYru6WbE7b7ElvmM+xS/YdT6xmDSm3Mc8R0Q9cBZwmabqkxcBhwMUVqn8L+JCkvSQ9F/gwcOF4Y5CEc2ezcXstz/4y+wVJayXdKung4Q7cmTkKbrNmuap0ZRcqXwXeKZu3JV98O9p8k2JrTnn/5R9Pcnn4ceBy4LiIWCVpiaTs4KhvAtcBdwK/A65Py8ZF4K5ns3GQ9B6gB/hypvjjJJOO9iLpobpO0n6Vjo+IcyKiJyJ6urvr9kZqZkVWurJbsgh4rNKQjZ1V6nnuaG/N65RmDSXXwUoRsY5k/eby8ltILiWVHgfwsXTLjeReLLOdJelwknHOb4yItaXyiPhlptpFkt4F/DXw9VyeN4+TmBWcpDaSz+xWoFVSJzBQmtib8S3gwnT41SPAJ8nhym7JY09
"text/plain": [
"<Figure size 864x216 with 3 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.activation_stats.plot_layer_stats(-2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Increase Batch Size"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dls = get_dls(512)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>2.309385</td>\n",
" <td>2.302744</td>\n",
" <td>0.113500</td>\n",
" <td>00:08</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn = fit()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAs8AAADWCAYAAAAuNG/NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZxjZZn3/8+VpPalt6re9wWFRpuBVpRNRnTEhRco+uiACM4oDojLOOrMqDgt7j6j8/PxQQRHRBERR1kfEMdRUMAFGqWFBuyGhqYbet+rqmtJcv3+OCfVqXSqKtV1qpLU+b5fr/Oq5OTOyVXpujvXuXOd+zZ3R0REREREhpcodwAiIiIiItVCybOIiIiISImUPIuIiIiIlEjJs4iIiIhIiZQ8i4iIiIiUSMmziIiIiEiJlDyLiMScmZ1uZm5mc8sdi4hIpVPyLCIyAZnZ3DAhPr3csYhUAjN7h5k9a2Z7zey7ZlaT91jSzP5gZm8vZ4xSHZQ8i4iIyIRmZtOA7wKfBE4FTgIuzmvyEeAFd7+pDOGVzMxqJ+JrVRslzzFhZvea2XfM7HNmtj088/68mSXM7NNmts3MdpjZ5/OekzKzVWb2jJl1m9laM3tfwXE/ZGaPmFmHmW01sx+Z2ay8x3NfB7/WzH5jZl1m9riZvW48f3+RicrMTjGzB8zsQLitCfvXprDJPWEffDbvOR8ws81hf/w5ML8csYuMo8XAPne/wd0fBW4FjgEws2XAh4BLSjlQ+Ln4lJmdbWZPmlmnmd1jZksK2p1gZv8dfj7uMLObzWxB3uOLwn0vhH3xUTO7oOAYuc/uz5rZFuD5QWK6N+znhdtFeW0+EMbbbWbrzeyTZpbKe/zZMEf4ppntAh4I988KP9v3mtnB8LVWlvJeTVRKnuPlrUANcArBWfYngP8HNBOciX8U+ISZvT5s/5/AW4D3AUcDVwBfNrO/LzjuR4GXAG8m+BD+UZHX/nfgC8AKYDVwk5lNjuw3E4khM0sCtwN/AI4Pt1VAV3gb4FxgFvCy8DlnA/8BfA04Dvgx8L/HM26RMngKaDSzlWbWBLwK+JOZGfAd4BPuvnUEx5tFkGyfTzCKPRm4NvegmR0D/Br4HbASeDWQAX5hZvVhs2bgl8CZBJ+h1wDfNbO/Lnit/wW0A2eExynmLWFMue1zQCfwUBjPKoLP6n8l+Dz/EMFn+78VHOeDwHbglcCF4ftzK/Bi4E3Ay4Ft4e/RNuQ7NJG5u7YYbMC9wCMF+9YCjxbsW0OQ6C4CssCLCx7/dOFxCh7/K8CBOeH908P7b8lrMzPc97pyvy/atFXzBkwJ+9LpRR6bW+wx4H7ghoJ9/x62nVvu30mbtrHagLPCz7gNBCeQSeADwJ3h59Kt4WPfB5qHOM4qIA205+17R/iZWR/evw74UcHz6ghObM8Z4ti3Ad/Ou38vsA5IjOD3fB3QA5wV3m8MX/fMgnbvAvbm3X8W+GVBmzPC/xuOKfg9tgCfLve/abm2/uF6iYU1Bfe3hlvhvukEZ8oGrA5OPPulCM6egaAsg+BM9hiCM+/ctxkLGPj10iO5G+6+1cwywIwj/D1EBHD3PWb2n8DPzexXBCNdt7j7X4Z42jHAjQX77gf+aYzCFKkI7n4HcEfuvpktBP4FOBH4P8ATBN/Q/gC4HPjnIQ73grvvyLv/PMFn5nTgOYJvepaaWUfB8+qBZeHrNxIMSJ1FMFpcS5CY3lPwnIfdPVvK72hmywm+Tfrn8PcFWA40AD81M89rngTqzaw973d5sOCQy4Fd7v54boe795jZH8LHYknJc7z0Fdz3QfYlOJQEn0RwxlrYBjObD9wFXE9Q0rGTYLTrfwj+E8jXWyQelQ2JjJK7v9fMvg78DfBa4LNmdhnBaNqgTxuX4EQq27eBVe6+2cxeA3zG3dNm9gPgM8M8t/AzLdenEnk/rwe+VOS5u8Kf/xs4m+DE9UmCMouvApMK2ncO94sAmNl0glLMH7j7/5f3UC6mtxGMYhfaPcxrFfv/wgbZHwtKnmUwD4c/57v7/xukzcsIzmY/7O4HIbhAYjyCE5FD3P0x4DHga2b2LYJZBG4JH04WNH8cOBn4Zt6+k8c8SJEKYmbvBczdvx3uShBcEwTB4M9oB3dWAy8Fnvaw1qGI0whKqG4KY0oARxHUFI+ImdUR9PknCeqW860FuoHF7n7XCA+9Fmgzs2Nyo8/ha72cgf+HxIqSZynK3Z8ys2uBb5vZxwkuemgCTiCo8/oysJ7gzPOfzOwGgosBP12umEXixsyWAu8l+Cp6EzCb4OLfPxJ8E9QB/I2ZrQV63H0PwcjWf5nZgwTfHJ0CXFDk8CITkpnNIbhQ7pS83b8BPhLOOPV+ghKo0fgCQQnED8JvhnYAC4FzgK+7+wbgL8DZZvZTgr76EYI+POLkGbg6fO67gfa8cst97t5hZl8AvhDu/wVB/vcS4K/cfajylF+Fv8cPzez9wD6CkpZ64KojiHNC0NfmMpSLCS6q+CTBaNUvgQsJLqjA3f9McLHF+8LHPwp8uCyRisRTJ0H95I8Ivo79KfBb4LKwRvL9BFfqbwL+BODutxB8Tfxx4M8EswUM9eEpMtFcDXzJ3Z/N2/dBguR2NUG/Gq5sY0ju/gRB2WMz8HOCz8hvE3xbuzds9o/ARoIa518S1E3/5Ahf8nSC+P9CcDFfbnt7GM9nw9d7D8H1T/eH958d5vdwgoT/SYJSsIcILq58rbvvPMJYq54N/m2CiIiIiIjk08iziIiIiEiJlDyLiIiIiJRIybOIiIiISImUPIuIiIiIlEjJs4iIiIhIiapqnue2tjZfuHBhucMQqRgPP/zwTndvL3ccxai/igxUyf0V1GdFCg3WZ6sqeV64cCGrV68udxgiFcPMNpY7hsGov4oMVMn9FdRnRQoN1mdVtiEiIiIiUiIlzyIiIiIiJYo0eTazqWZ2i5l1mtlGMztvkHZmZl82s13h9hXLW4hdREREDjGzy8xstZn1mNl1w7T9RzPbamb7zOxaM6sbpzBFYiHqkecrgV5gBnA+cJWZLS/S7mKCtdJXAC8F3gS8L+JYREREJooXgM8B1w7VyMxeB/wLcAawEFgMfGasgxOJk8guGDSzJuBc4Fh37wDuN7PbgQsIOnK+C4Gvuvvm8LlfBd4LfGs0MXz7NxvoSWe47NXLRnMYkVgws8uAi4CXADe6+0WDtLsI+A5wMG/3m9z93ijiuOvRLcxoreeEBVOiOJzIhOTuNwOY2Upg7hBNLwS+4+5rw/afBW7g8M/hMffbp3Zy12NbSJjxwTOW0dZ8ZAPg2/Z3c9W9T9PZk444Qomz179kJq9+8Ywjem6Us20cBWTcfV3evjXAq4q0XR4+lt+u2Aj1iDzw9E52d/YqeRYpTW4k63VAwzBtf+fup4xFEF+5+0leOneykmeRaCwHbsu7vwaYYWbT3H1XYWMzu5jg22Dmz58fWRDP7eri77+3moRBZ2+G6S11/Z/NG3d1sn5bB2ccPZ3BKjY7etLc9sjzHOhO85/3bWB/d5q2ptrI4hN5ydxJR/zcKJPnZmBfwb59QEsJbfcBzWZm7u75DUfSsRtrk2zekxlh2CLxNIKRrDGVcSeT9eEbikgpin2+QvBZfFjy7O7XANcArFy5MpKOmM06H/3JGlIJ478/chqX3vBH7l67lctevYy7Ht3Cx3/yZzp60pxz3Gz+18p5UJA/H+zN8Pm7nmDDjk4Alk1v5sb3voJlM4qlEyLjL8rkuQNoLdjXChwooW0r0FGYOMPIOnZDTYqDvUqeRcbAX5nZTmA3cD3wRXeP5DvUbBayh3d9ETkyxT5fofhn8Zj41ZPbefCZ3XzpLS9h1qQGXn/sTL5w15Pc/MfNfOTHazhu3mROXjqNb977NLc+8kLRY7Q113HDe07k2DmTaKlLkUhoTgGpHFEmz+uAlJktc/f14b4VwNoibdeGjz04TLsRaaxN0tWrmiiRiP0GOBbYSPCV8E1AGvhiYcMj+Qo4k9XIs0iEcp+vPw7vrwC2FSvZGCt3PbqFSQ01nHtC8IXWmctn8YW7nuS
"text/plain": [
"<Figure size 864x216 with 3 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.activation_stats.plot_layer_stats(-2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### 1cycle Training"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def fit(epochs=1, lr=0.06):\n",
" learn = Learner(dls, simple_cnn(), loss_func=F.cross_entropy,\n",
" metrics=accuracy, cbs=ActivationStats(with_hist=True))\n",
" learn.fit_one_cycle(epochs, lr)\n",
" return learn"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.210838</td>\n",
" <td>0.084827</td>\n",
" <td>0.974300</td>\n",
" <td>00:08</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn = fit()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAuUAAAD7CAYAAADNeeo8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3yV9fn/8deVvXcIMwsSRoAECFtw7wEuVBSxWrFqqxattbb+3FqtqK1aFfcoqCiKew9AhRBG2DOEQBhZZO/k8/vjHPqNaZAEknOfcT0fj/Oo3PeHc96nmMPlfa77+ogxBqWUUkoppZR1vKwOoJRSSimllKfTolwppZRSSimLaVGulFJKKaWUxbQoV0oppZRSymJalCullFJKKWUxH6sDOEpMTIxJTEy0OoZSSnXaypUri40xsVbncCT9zFZKuaqj/cz2mKI8MTGR7Oxsq2MopVSnicguqzM4mn5mK6Vc1dF+Zmv7ilJKKaWUUhbTolwppZRSSimLaVGulFJKKaWUxbQoV0oppZRSymJalCullFJKKWUxhxXlIhIlIu+LSLWI7BKR6YdZJyLyiIiU2B+Pioi0Ou8tIg+IyF4RqRSR1SIS4aj3oZRSSimlVFdz5EjEZ4AGIA7IAD4RkRxjzIY262YBU4F0wABfAbnAc/bz9wITgPFAPpAG1HV7eqWUUkoppbqJQ4pyEQkGLgSGGmOqgKUi8iEwA7ijzfKZwBxjzB77750DXAs8JyKRwC1AujHm0AzI9Y54D57OGMPm/ZV8u7mQ+sZmAPx9vQkP9CUyyI9eEQH0jQgkNtSfVl9sKKWUQzS3GP7+2SZGxEcyOjGK2FB/qyMppVxMWU0D+aU1FFbUU1rdQGlNA1dPTMLPxzGNJY66Up4KNBtjtrY6lgMc387aNPu51uvS7P88DGgCLhKRPwIVwD+NMc+096IiMgvblXfi4+OP6Q14so/X7uXJr7exvbAKABEwpv21of4+pMSFMKR3GCPjIxmVEEl8VJAW6kqpbrW7tIY3lu3ihSU7ARiXHMU/LkqnX1SQxcmUUs6ooKyWFTtLWbO7jI37Ktiyv5Ly2sb/WTcloze9wgMdkslRRXkIUN7mWDkQ2oG15UCIva+8LxCOrchPAlKAb0RkqzHmq7ZPZIyZC8wFyMzMPEwZqQ6nqbmFR7/YwtzFuaT1DuOBqUM5c2hPokNsV6DqGpspr22ktLqBfeW17DlYy/bCKjbvr+SD1Xt5c1k+AH0jA5mcGsvJg3pwXEoM/j7eVr4tpZQbSowJZu3dp7N+bzk/7yjh2e93cNY/l/DA+UOZktHH6nhKKYvVNzXz4/ZivttcxPdbC9ldWgtAkJ83g3qGcvbwXiTHBNMvKoieYQFEBfsRFexHkJ/jahZHFeVVQFibY2FAZQfWhgFVxhgjIrX2Y/cZY2qBtSLyFnAWtt5z1UUamlq49vVsfthaxJXjE/jb2UP+5+ubAF9vAny9iQsLYHCvX/7xNrcYthVWsmJnKYu3FbNodQHzlucT6u/DqUPiuGhUX8YlR+PlpVfQlVJdw8/Hi5HxkYyMj+S89N7c8vYabn5rDbUNzVw6Rr8tVcrTtLQYftpRwvurC/hy434q65oI8vNmQv8YrpmYxOikKAb1DMPbSWoRRxXlWwEfEUkxxmyzH0sH2t7kif1YOpDVzrq19v/Vq97d7P6PN/LD1iIemDqUK8YldPr3e3sJg3qGMahnGDPGJ9LQ1MKPO4r5bN0+Plu/n4WrC+gbGcj0sfFcNjqeyGC/bngXSilP1S8qiLdnjePq17L52wfriY8KYsKAGKtjKaUcoKSqnvlZ+by1Yjd7DtYSGuDD6Wk9OXt4Lyb0j3bab+zFHK45uKtfyHZF2wC/xTZ95VNgQtvpKyLyO+Bm4BT+b/rKU8aY5+znFwObgJuAZOAH4DJjzDe/9vqZmZkmOzu7S9+Tu3p7RT5/fm8d101O5i9nDe7y569rbOaLDft5K2s3P+eW4O/jxYWj+vK7yf2Jj9b+T6XaEpGVxphMq3M4Uld9ZlfUNXLhv3/iQEUd7984kf6xIV2QTinljHKLqnhhSS4LVxVQ39TChP7RXDomntOGxBHg67hC/Gg/sx05EvEG4GWgECgBrjfGbBCRScBnxphDn5TPYyu219l//aL92CGXAS/Zn6MQuOtIBbnquPUF5dz1wQYmpcRw+xmDuuU1Any9mZLRhykZfdiyv5JXf9rJu9l7eHvFbqZk9Obmk1NIiA7ultdWSnmWsABfXr5qNOc9vZTbFuTw3u8maNucUm5mR1EVT32zjQ9z9uLr7cUFI/tyzXGJDOjR3q2LzsthV8qtplfKj6ylxXDBsz+x52AtX8+eTESQ41pKDlTUMXdxLv9ZvoumZsNlY+K56eQUHWumFHqlvCu8t3IPty7I4aHzhzF9rPaXK+UOCivqeOLrbbyTvRs/by9mjE/g2knJltcOrnClXDm5d1fuYc3uMh6flu7QghwgLiyAu84ZwqzJyfzrm23My8rng9UF3HRyCjMnJDpsRqhSyj1dMLIP72Tv5pHPN3N6Wtx/p0gppVxPfVMzLy7ZydPfbqeppYUZ4xL4/UkDiHHxn2utdBQA5TWN/P3zzWQmRHL+COvGh8WFBfDg+cP46o+TyUyM5MFPN3HGPxezLLfEskxKKdcnIjwwdSjV9U08/Nlmq+MopY7SD1uLOP2Jxfzjiy0cnxrL17OP557z0ly+IActypXdE19vpaymgfumDHWKjX6SY0N45TdjePmqTBqbW7h07jJufzeH8pr/HeyvlFIdkRIXyjXHJfHeqj1sL2xvIq9SylmVVjdwy1urmflyFl5ewutXj+G5GaPc6h40LcoVhRV1zMvKZ1pmP4b0bjtO3lonDYrjy1uO53fH9+e9VQWc9uQPfL+l0OpYSikXdd3x/Qnw8ebf3+2wOopSqoM+X7+fUx7/gU/W7eOmk1P47OZJTE6NtTpWl9OiXPHCklyamlu4/oT+VkdpV6CfN3ecOYgPbphIeKAvV72ygjvfX0dtQ7PV0ZRSLiYq2I8rxsWzKGcvu0qqrY6jlPoVFXWNzH5nDb97cyW9IwL4+A+TmH1qqtPOGT9WWpR7uIPVDfxneT7npfd2+q+AhvUN58PfH8esycnMW57PuU8vZePeCqtjKaVczLWTkvH2Ep77Qa+WK+WscnaXcc6/lrJozV5uOmkA798wkYE9XWvEYWdpUe7hXvlxJzUNzdxw4gCro3RIgK83d541mDevGUt5bSNT//0j87Py8ZTRnkqpY9cjLIBLMvvx7so97C2rtTqOUqoVYwwvLsnloud+ornF8Pasccw+bSC+3u5fsrr/O1SHVV3fxKs/5XHakDhS41zrvz6PS4nh85snMTYpir8sXMdtC9ZqO4tSqsOuOz6Z5hbDvOX5VkdRStlV1Tdx47xVPPDJJk4c2INPbjqOzMQoq2M5jBblHmzRmr1U1DVx3fHJVkc5KtEh/rz6mzHcdHIK763aw8XP/6RXvZRSHdI3MogTB/bg7ezdNDa3WB1HKY+XW1TF1Gd+5PP1+/nLmYN4fsYoh++ZYjUtyj3YvKxdDOoZysj4SKujHDVvL2H2qam8NDOTvOIaznt6Kdl5pVbHUkq5gMvHxVNUWc/XGw9YHUUpj7ZkWxFTn/mRkqp63rxmLNcd398pxjM7mhblHmrtnjLWF1QwfWy8W/yLf/LgOD64cQIh/j5Mf2E5H6wusDqSUi5NRKJE5H0RqRaRXSIy/TDrIkTkNREptD/uOcy640XEiMgD3Rq8E45P7UGfiED+oy0sSlnmjZ/zuOqVFfQKD+TD3x/HhAExVkeyjBblHuo/y/IJ9PVmqoW7d3a1AT1CWXTjcYxMiOCWt9fw5Ndb9QZQpY7eM0ADEAdcDjwrImntrHsCCAISgTHADBH5TesFIuIL/BNY3p2BO8vbS7hkdD+Wbi8mr1jHIyrlSC0thgc/2chdizZwQmos790wgX5RQVbHspQW5R6ooq6RD3P2cl56b8ICfK2O06XCg3x5/eqxXDiyL09+vY0/v7eWJu0XVapTRCQ
"text/plain": [
"<Figure size 864x288 with 2 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.recorder.plot_sched()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAsQAAADWCAYAAADW1JQ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3hcZ5X48e+ZLo16tyXLcu9xXNIdEpNOJ6EEkgC7QIBQf5AldLK0pbP0JQllN4QSIAklJAHSnAQSx3bcu2XZVu8jzWhGmvL+/rgzsnqxJY00Op/nmcfWnTt33rE1M+eee97zijEGpZRSSimlZitbsgeglFJKKaVUMmlArJRSSimlZjUNiJVSSiml1KymAbFSSimllJrVNCBWSimllFKzmgbESimllFJqVtOAWCmlUoyIXC4iRkTKkj0WpZSaCTQgVkqpGUBEyuJB7uXJHotS04WI3CgiVSLSLiI/FxFnn/vsIvKCiLw5mWNUM4MGxEoppZSacUQkH/g58GngUuBi4NY+u3wUqDXG/DYJwxszEXGl4nPNNBoQz2Ai8pSI/FREviQijfEz5C+LiE1EPiciDSLSJCJf7vMYh4jcKSLHRSQkIvtE5D0DjvthEdkpIn4RqReR34jInD73Jy7HXiUiW0SkS0T2i8g1U/n6lUpFIrJJRJ4Tkc74bVf8vXUqvsuT8fdfVZ/HfFBEquPvxceA8mSMXakpthDwGWPuM8bsAR4CVgKIyBLgw8D7xnKg+PfiURF5rYgcFJGAiDwpIosG7LdBRP4W/35sEpEHRGR+n/sXxLfVxt+Pe0TklgHHSHx3f1FE6oCaYcb0VPy9PvD2jj77fDA+3pCIHBGRT4uIo8/9VfEY4Uci0gI8F98+J/7d3i4iwfhzbRzLv1Wq0oB45nsD4AQ2YZ0Nfwr4C5CBdcZ8O/ApEbkuvv89wPXAe4AVwBeAr4nIOwcc93ZgDfB6rC/X3wzx3N8EvgKsBbYBvxWRnAl7ZUrNMiJiB/4EvACsj9/uBLrifwe4AZgDnBd/zGuB7wDfBs4F7ge+MZXjVipJjgLpIrJRRLzAZcBLIiLAT4FPGWPqx3G8OVgB9E1Y2eYc4GeJO0VkJfA08C9gI/ByIAr8XUQ88d0ygMeBa7G+Q+8Cfi4imwc815uAQuCK+HGGcn18TInbl4AA8GJ8PHdifVd/Euv7/MNY3+2fH3CcDwGNwEXA2+P/Pg8By4FXAecDDfHXUTDiv1AqM8bobYbegKeAnQO27QP2DNi2Cyt4XQDEgOUD7v/cwOMMuH8dYIDS+M+Xx3++vs8+JfFt1yT730VvepupNyA3/j66fIj7yoa6D3gWuG/Atm/G9y1L9mvSm94m8wa8Ov4dV4l1YmgHPgg8HP9eeih+3/8BGSMc504gAhT22XZj/DvTE//5F8BvBjzOjXXC+roRjv1H4O4+Pz8FHAZs43id1wDdwKvjP6fHn/faAfu9DWjv83MV8PiAfa6Ifz6sHPA66oDPJfv/NFm33rS6mrF2Dfi5Pn4buK0I64xWgG3WCWIvB9ZZLmCVRGCdca7EOkNOXEmYT/9LOzsTfzHG1ItIFCg+w9eh1KxnjGkTkXuAx0TkCaxs1IPGmEMjPGwl8OsB254FPjZJw1Rq2jDG/Bn4c+JnEakAPgFcAHwPOIB1JfWXwGeBO0Y4XK0xpqnPzzVY35lFwEmsqzKLRcQ/4HEeYEn8+dOxkkyvxsrqurCCzScHPGa7MSY2ltcoIquwrvzcEX+9AKuANOAPImL67G4HPCJS2Oe1bB1wyFVAizFmf2KDMaZbRF6I3zcraUA884UH/GyG2WbjdGB7MdaZ5cB9EJFy4K/AvVjlFM1Ymal/YL2x++oZYjxahqPUWTDGvFtEvgtcDVwFfFFEPoCV8Rr2YVMyOKWmv7uBO40x1SJyJfCfxpiIiPwS+M9RHjvwOy3xvrL1+fNe4KtDPLYl/uc3gNdinZAexCpx+BaQPWD/wGgvBEBEirDKIH9pjPnvPnclxvRGrGzzQK2jPNdQnxkyzPZZQQPi2WV7/M9yY8xfhtnnPKyzzo8YY4JgTSKYisEppSzGmL3AXuDbIvI/WDPnH4zfbR+w+37gEuBHfbZdMumDVGqaEZF3A2KMuTu+yYY1xwashM7ZJmy2AecAx0y8zmAIL8MqYfptfEw2YClWje64iIgb631/EKsOuK99QAhYaIz56zgPvQ8oEJGViSxx/LnOp//nyKyiAfEsYow5KiI/A+4WkY9jTQzwAhuw6qa+BhzBOkP8mIjchzVh7nPJGrNSs4mILAbejXUJ+BQwF2ty7A6sqzV+4GoR2Qd0G2PasLJPvxORrVhXdzYBtwxxeKVSloiUYk0m29Rn8xbgo/FOS+/HKkE6G1/BKj/4ZfwqThNQAbwO+K4xphI4BLxWRP6A9X79KNb7eNwBMfCT+GP/DSjsU+roM8b4ReQrwFfi2/+OFdOtAdYZY0YqDXki/jp+JSLvB3xY5SQe4MdnMM6UoJe3Z59bsSYefBors/Q48HasSQcYY3ZjTUh4T/z+24GPJGWkSs0+AaxaxN9gXQb9A/BP4APxesP3Y81OPwW8BGCMeRDr8uzHgd1YM+RH+jJUKhX9BPiqMaaqz7YPYQWs27DeW6OVTIzIGHMAq+QwA3gM6zvybqyrqu3x3f4fcAKrZvhxrDrk35/hU16ONf5DWBPeErc3x8fzxfjzvQtrPtGz8Z+rRnkdBiuIP4hVivUi1gTEq4wxzWc41hlPhs/6K6WUUkoplfo0Q6yUUkoppWY1DYiVUkoppdSspgGxUkoppZSa1TQgVkoppZRSs5oGxEoppZRSalZLeh/igoICU1FRkexhKDVtbN++vdkYU5jscQxF369K9afvV6VmjpHer0kPiCsqKti2bVuyh6HUtCEiJ5I9huHo+1Wp/vT9qtTMMdL7VUsmlFJKKaXUrKYBsVJKKaWUmtU0IFZKKaWUUrPahAbEIvIBEdkmIt0i8ouJPLZSSimllFKTYaIn1dUCXwKuAdIm+NhjZozhJ1sqee25c5mTnbRhKKXUlPq/f1Wxbl4ua8qykz0UpdQECEdjPHmwke0n23DabCyfk4kgLC7KYFlJZrKHl1ImNCA2xjwAICIbgbKJPPZ4VDYH+OojB4kZw22XL07WMJRSaso0dob43B/3sbYsm4fefwkikuwhKaXOQmugh9vu287zla247DaixhCNGQBE4A3ry3j/5sVUFHiTPNLUkJS2ayJyK3ArQHl5+YQfv7IpAEBde2jCj62UUtPRM4ebAdhV7WPr8VYuWJif5BGp6a47EuWBHTW8eeM8bDY9gZounjjYwE+ermRPjY9IzPD1G87htevmYszp+OahnTX8/Lnj/G57NZctLeSmC8qZm5NGTrqTstz0JL+CmSkpAbEx5i7gLoCNGzeaiT5+ZZMfgNr24EQfWimlpqWnDzdRkOEiZuDuZyo1IFaj+vOuOj75wB4e21fPf7/5XHLSXcke0qzVGujhsX31vFjVygM7alhQ4OWNG8p448Z5rC49XQK1cm5W75/v2rSAX209ya9eOMmt927v3efChXmcvyCfZcWZXLWyGJdD+yeMRdIX5pgMx+IBcY0GxEqpWSAaMzxzpInNy4uYl5vOdx8/Qp0vqHMo1IhuWF9KMBzlC3/ex6ce3MOPbtqQ7CHNSsea/Lz9Z1upbgvidth456YFfPzaZbgd9hEfV5Tl4SNXLuX9mxfzfGULoXCMww2dPLCjmh88cYSYgbnZHt6woYwLFuYTCkex2wSn3cbOU+1UtwUJR2OsnJPF2nnZzMtNJxwzhMJRBMhOc5Kb7sJmE7p6ItT7QqS7HBRmurEPuKIQCkc52dpFa6CHjmCYmAGP08Zj++o51hTg8mWF5KW7CEdjzM/3UucLsremA4BILEZPxJDpcdARDHOsyY+IMCfbw/ryXB58qYaTrV3c+rKF5KQ72Vvj40RLF3leF4uLMmjoCBEzkOd1cfMF8ynPP7MMeUoGxIlLCpohVkrNBntqfLR1hblsaSGLizL47uNHeL6yhdevS9pUDjUDiAi
"text/plain": [
"<Figure size 864x216 with 3 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.activation_stats.plot_layer_stats(-2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAADNCAYAAAC8XqoPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dzY5kSZqWzf8iIzIrc6qqK5lWA6JpDaAZaYbZgAQIiQ0bJJbcAkskuBtug9sYwYLesGAxQgIBhdQz9ZcZ4X8sanok/77Hw9/0yO7OMD3Pzk/asWPHjp0Tlu7PeW1xPB6HiIiIyMwsf9cNEBEREflN44RHREREpscJj4iIiEyPEx4RERGZHic8IiIiMj1OeERERGR61o/9479c/ptn8876//u3/6RtW+xPP3/1X79rZY5/9surj7n+xc9PPv/w99+2Ml//w00/5qrXVdu6+baXufm2X44vf/mXvf7N6QG++cWrVubdV32ue6DRAFPizTen7VjAKFk9wEbYtLtdXDwecoBjbk8/b/tpj9X7vm0BdY1F37TclfPe9zJE3W+MMbYvT0/05vveiPs3vTOO0K7t69ONy4dehsbc+ofeLrqW6/enG6kN1F8raMe7L08LHm76jlg/cKzjFdpO13b9LnyslWLLXS9C2+g+qtuOMM6p7w8r6Ix6y2z7jodN32+xz673YV32pe4Kx8CyPAd2d1Sob1q97wc9Lk/3pWcMjR0a+/Qsvf/8tCGre+jXR/9inme5hY1hv27Kfbp9BYVgnNN5E3UsLuG5RudNfVif56v7vt/+BbQhaCvVdeh/Xsd/+Y//4ewTxG94REREZHqc8IiIiMj0OOERERGR6XHCIyIiItNzpYL16bF72T2lzfenUtVh0+d3oR+JHG9vTj4/vIb6SQqkaWYpd7jpRZYgHW6/uG3b1t+dGnIoUYLslQp5tW0krxH7F723X/3fU0Pu+5+CvUZ9CP1Tz4nkXTrvKgWOAeLmGOO4KMItyHckMqOkXg75/vdAUCaRj6TG7y+LoSS2Nul3jDGg/VVEfPW/u6n7/sve2C3ckxWSio9wjXDf2lY6HI0dEnpBLN/8cPoZJXgQWweMnXqeNL7o/luAFN33ywTlKv2OEQrioQhMInbdGSVvuN57kNnrf8+PIHRXSXoM7p/7N5f7jNpA503HXJeXIw70DKCxD2OgSsoovNOLFwCJv7uXp5/xuQntevgM+r9c390dtAHq3wfjifqQ/qY8ht/wiIiIyPQ44REREZHpccIjIiIi0+OER0RERKZnGmmZqLLl/q6f7lM64HhzujdKbjClTGRUkl+3IKNWQZkgSZOSP1ORuQpyKBiG/PDVaWdgejEl6AZTdRLaViTvksxJYuD+chkS65Z7EiRPP5M4TdIvyflVgOa02UzAHJRK+93pAe4/74UoETgRNY8kW2L6L5RrlfdNmDpNwxXGwO7laUOqxDzGGFsQNxMRmMbhgdx/CiwvEi6mmqPE2svta9L5GGNRGoJlSKYO3l2gZ0yakF3PiZ+t2b1M6ef1OYASNrSVkolrmjCJ05iQjcL7aTkSv+mrC3qWUspxhZKcqb+wHa0RfdOuv2eD160moqMwDn34GH7DIyIiItPjhEdERESmxwmPiIiITM+zdHhWb/uq5PTbcPU87r/4uA7P8pvTH/XX96+j/ej36fp76x4Cm26+6dvuf9J/EK2/+ZJTsIMQNfytm1bOLb+30srGFJKYBJ+Rq7EHd4m8hersrGBVbOr7PYU8wu/TtRyVodA8dFmSEDXymWih5NJn9Ns9ulhQbkXhliWYbHV/2SMag/tnV/pnT7/n03/DaFu9luS7hOGNGMxYyu3DQESqvwcPQhkKfSRvr4xrdi4ebeFfQ8GJ9T5KV/rGIL1gFWz0CeGebOGscD3ovDH8FbzGVgauUQ35PFeuhpQS6EbRPV+gMYEru4PDWP82jNG9oTq+xuC+Jv+ueZOhYkPtam0I7+XH8BseERERmR4nPCIiIjI9TnhERERkepzwiIiIyPQ8S2k5pQpgFI72FOpq6fdvYMVrCqcjYbEIWet3vQwJnqv7brnVlZ8XYNKSyJwKvZsqmJEUTcIcyoOnn0lgJEGZQvnWIE9XqA9xBWcMEKyf+/EoGBBl2iJwo3RIIj6FidVrBOOLAvgICpGsQm8ayJaMMZQ0ScwOAjxRnIbVoTGwDoT6FsAHsigG0QVhaHSvJSLtGDDGqL8wpA3qCs4pEY/H4Odau7dCsZyk5TqG6XwoWA9fEgm2ofxP1ygIjKR2sQANL3aUbenK6ATdI7X/6TrSeePfIxCekzbQedNK660ueAY/ht/wiIiIyPQ44REREZHpccIjIiIi0+OER0RERKbnWUrLx7/Zk5ZJCqtyIsqWT+HrX518XBx+0tsAQt5xSdbWqbRFwmpd/X2MMe6/6AUjyRQgQTlZkXh3RyZc30SSYUugpURPcjlhW0uIRUOvb0qSkMcYY1/uFl7ZuO9H1y0RrCl9GVcND1xXGoco+YJwW88JJW8YOyh9lnYc4XxIVkwSaGmc0HhavYf6Iem1yaKwYjQKpOS11gBakKRx7ONK34/XTWXG4PR2bH+Vlmkl7lAEbi8lULvCBOsmqQdjYowzz79grGACdPAyw48HLYcLJfXkWUf3KKU2E8dAsMYV56H69Q/QjpvLgjW9gEDUl1XohYoPjVr2Gx4RERGZHic8IiIiMj1OeERERGR6nPCIiIjI9DxLaXn5zQ9t22Hzedu2KsmcT0mojMBEXShHInAxSCkV+trZKaaphv4XSmdl1KxRAoX64QS2r04PuvkO0otfkQUKm6qkHiZrU5IzyYmtDImCaZJsbRsdj6qnlO7S1ySfYzgyyJbHBQiLReZEwZckb5IygycOyqJwTq2vqe9hHGJb6R6psjZckAUIpElicvqCAF24QxOgoS7o51T+r+OJX7yAdoF8XIXeNAkZXy6o4zCUtekcMaX55vHPY/S/KWOMsQvS25Pn6Llyta9J3qVrhH0RPJ/onqH7b18H4oAxQH0PfUjtry+h4IsXcB0fw294REREZHqc8IiIiMj0OOERERGR6XmWDs/x2+/7Ngjzq8FwD28+7mrpu7/3s9KGXua4ynyH9tsqNDV1TepvpOlvxbgSN/zmXqfJabAe1V+31d9txzjjENDIrecULqRLAYLU/zffnFa4IzeH3InAzcBVf+k3eOif5Hfz1AeCn+XbNYqcpDHYVWs+Qi+D7Q/cA3QDyDVJwxvrNlh5nXZMVypv+4EDQ+GQbZxTCCf5J9B+XLm6toPagM+nvu1jhgVWB4nCQfFZl5zjGD3gD9q6e9m3Ub/WdlAb8HlLAZ7VqYLzppXX0fWhsNSyK65UT14dOVvlmUXXYwkBnsfEB0od2UfwGx4RERGZHic8IiIiMj1OeERERGR6nPCIiIjI9DxLaXm8/aJtQpGyBlWFwlzK7rPTA6DgC0QrIGPYF4ldIKaVMLT1u257vX8JdZFAStRANhTaQI4DsbyKs2lQHK7EXUczBetR34Nsiauelz5LV2HG8LgqNUKKGl0PlHDrWKEy0KwV9Q+Fe5VyJKmnK68vi5x4fAVtIKE0EGep7Xg9gtWtxxhjWWRUklhRyoT7ocquFApH0i+uoF6D4uiawQWn59PVojGJwMHLESiMh7QxRhIr1Q99nYyV9Lyvhl4QuLJ+et7i85xehKhtgDKpyLwvIYzrnhGMgbA0zmugI8rh9Kx4BL/hERERkelxwiMiIiLT44RHREREpscJj4iIiEzPs5SWH37/s7YNkznLNhKcVm/ftm37r7+Oyj0UiRgTPamHg3JLWtE5lKK3L0933n6WpRcf1mQBQjtuL583niMkfzablqRikqJJwKxSG6ULB6ugjzHGDmTaVVkVPkrMPrctWPEaIbm2ppuG0m+S6krbaEV7SsimpNcqNeLq0GGiaiL649ihciQk15ceKC03XFm6ysFbGF8ErnjdGtE30croUV0DROMgUfdsO6pgHa4IT5Z9krSMidZBajO2LUyYpvF0se7B9x+2v0rkJBVjonFWLiL921auG60
"text/plain": [
"<Figure size 720x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.activation_stats.color_dim(-2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAADNCAYAAAC8XqoPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dzY5kSZqWzf8iIzIrc6qqK5lWA6JpDaAZaYbZgAQIiQ0bJJbcAkskuBtug9sYwYLesGAxQgIBhdQz9ZcZ4X8sanok/77Hw9/0yO7OMD3Pzk/asWPHjp0Tlu7PeW1xPB6HiIiIyMwsf9cNEBEREflN44RHREREpscJj4iIiEyPEx4RERGZHic8IiIiMj1OeERERGR61o/9479c/ptn8876//u3/6RtW+xPP3/1X79rZY5/9surj7n+xc9PPv/w99+2Ml//w00/5qrXVdu6+baXufm2X44vf/mXvf7N6QG++cWrVubdV32ue6DRAFPizTen7VjAKFk9wEbYtLtdXDwecoBjbk8/b/tpj9X7vm0BdY1F37TclfPe9zJE3W+MMbYvT0/05vveiPs3vTOO0K7t69ONy4dehsbc+ofeLrqW6/enG6kN1F8raMe7L08LHm76jlg/cKzjFdpO13b9LnyslWLLXS9C2+g+qtuOMM6p7w8r6Ix6y2z7jodN32+xz673YV32pe4Kx8CyPAd2d1Sob1q97wc9Lk/3pWcMjR0a+/Qsvf/8tCGre+jXR/9inme5hY1hv27Kfbp9BYVgnNN5E3UsLuG5RudNfVif56v7vt/+BbQhaCvVdeh/Xsd/+Y//4ewTxG94REREZHqc8IiIiMj0OOERERGR6XHCIyIiItNzpYL16bF72T2lzfenUtVh0+d3oR+JHG9vTj4/vIb6SQqkaWYpd7jpRZYgHW6/uG3b1t+dGnIoUYLslQp5tW0krxH7F723X/3fU0Pu+5+CvUZ9CP1Tz4nkXTrvKgWOAeLmGOO4KMItyHckMqOkXg75/vdAUCaRj6TG7y+LoSS2Nul3jDGg/VVEfPW/u6n7/sve2C3ckxWSio9wjXDf2lY6HI0dEnpBLN/8cPoZJXgQWweMnXqeNL7o/luAFN33ywTlKv2OEQrioQhMInbdGSVvuN57kNnrf8+PIHRXSXoM7p/7N5f7jNpA503HXJeXIw70DKCxD2OgSsoovNOLFwCJv7uXp5/xuQntevgM+r9c390dtAHq3wfjifqQ/qY8ht/wiIiIyPQ44REREZHpccIjIiIi0+OER0RERKZnGmmZqLLl/q6f7lM64HhzujdKbjClTGRUkl+3IKNWQZkgSZOSP1ORuQpyKBiG/PDVaWdgejEl6AZTdRLaViTvksxJYuD+chkS65Z7EiRPP5M4TdIvyflVgOa02UzAHJRK+93pAe4/74UoETgRNY8kW2L6L5RrlfdNmDpNwxXGwO7laUOqxDzGGFsQNxMRmMbhgdx/CiwvEi6mmqPE2svta9L5GGNRGoJlSKYO3l2gZ0yakF3PiZ+t2b1M6ef1OYASNrSVkolrmjCJ05iQjcL7aTkSv+mrC3qWUspxhZKcqb+wHa0RfdOuv2eD160moqMwDn34GH7DIyIiItPjhEdERESmxwmPiIiITM+zdHhWb/uq5PTbcPU87r/4uA7P8pvTH/XX96+j/ej36fp76x4Cm26+6dvuf9J/EK2/+ZJTsIMQNfytm1bOLb+30srGFJKYBJ+Rq7EHd4m8hersrGBVbOr7PYU8wu/TtRyVodA8dFmSEDXymWih5NJn9Ns9ulhQbkXhliWYbHV/2SMag/tnV/pnT7/n03/DaFu9luS7hOGNGMxYyu3DQESqvwcPQhkKfSRvr4xrdi4ebeFfQ8GJ9T5KV/rGIL1gFWz0CeGebOGscD3ovDH8FbzGVgauUQ35PFeuhpQS6EbRPV+gMYEru4PDWP82jNG9oTq+xuC+Jv+ueZOhYkPtam0I7+XH8BseERERmR4nPCIiIjI9TnhERERkepzwiIiIyPQ8S2k5pQpgFI72FOpq6fdvYMVrCqcjYbEIWet3vQwJnqv7brnVlZ8XYNKSyJwKvZsqmJEUTcIcyoOnn0lgJEGZQvnWIE9XqA9xBWcMEKyf+/EoGBBl2iJwo3RIIj6FidVrBOOLAvgICpGsQm8ayJaMMZQ0ScwOAjxRnIbVoTGwDoT6FsAHsigG0QVhaHSvJSLtGDDGqL8wpA3qCs4pEY/H4Odau7dCsZyk5TqG6XwoWA9fEgm2ofxP1ygIjKR2sQANL3aUbenK6ATdI7X/6TrSeePfIxCekzbQedNK660ueAY/ht/wiIiIyPQ44REREZHpccIjIiIi0+OER0RERKbnWUrLx7/Zk5ZJCqtyIsqWT+HrX518XBx+0tsAQt5xSdbWqbRFwmpd/X2MMe6/6AUjyRQgQTlZkXh3RyZc30SSYUugpURPcjlhW0uIRUOvb0qSkMcYY1/uFl7ZuO9H1y0RrCl9GVcND1xXGoco+YJwW88JJW8YOyh9lnYc4XxIVkwSaGmc0HhavYf6Iem1yaKwYjQKpOS11gBakKRx7ONK34/XTWXG4PR2bH+Vlmkl7lAEbi8lULvCBOsmqQdjYowzz79grGACdPAyw48HLYcLJfXkWUf3KKU2E8dAsMYV56H69Q/QjpvLgjW9gEDUl1XohYoPjVr2Gx4RERGZHic8IiIiMj1OeERERGR6nPCIiIjI9DxLaXn5zQ9t22Hzedu2KsmcT0mojMBEXShHInAxSCkV+trZKaaphv4XSmdl1KxRAoX64QS2r04PuvkO0otfkQUKm6qkHiZrU5IzyYmtDImCaZJsbRsdj6qnlO7S1ySfYzgyyJbHBQiLReZEwZckb5IygycOyqJwTq2vqe9hHGJb6R6psjZckAUIpElicvqCAF24QxOgoS7o51T+r+OJX7yAdoF8XIXeNAkZXy6o4zCUtekcMaX55vHPY/S/KWOMsQvS25Pn6Llyta9J3qVrhH0RPJ/onqH7b18H4oAxQH0PfUjtry+h4IsXcB0fw294REREZHqc8IiIiMj0OOERERGR6XmWDs/x2+/7Ngjzq8FwD28+7mrpu7/3s9KGXua4ynyH9tsqNDV1TepvpOlvxbgSN/zmXqfJabAe1V+31d9txzjjENDIrecULqRLAYLU/zffnFa4IzeH3InAzcBVf+k3eOif5Hfz1AeCn+XbNYqcpDHYVWs+Qi+D7Q/cA3QDyDVJwxvrNlh5nXZMVypv+4EDQ+GQbZxTCCf5J9B+XLm6toPagM+nvu1jhgVWB4nCQfFZl5zjGD3gD9q6e9m3Ub/WdlAb8HlLAZ7VqYLzppXX0fWhsNSyK65UT14dOVvlmUXXYwkBnsfEB0od2UfwGx4RERGZHic8IiIiMj1OeERERGR6nPCIiIjI9DxLaXm8/aJtQpGyBlWFwlzK7rPTA6DgC0QrIGPYF4ldIKaVMLT1u257vX8JdZFAStRANhTaQI4DsbyKs2lQHK7EXUczBetR34Nsiauelz5LV2HG8LgqNUKKGl0PlHDrWKEy0KwV9Q+Fe5VyJKmnK68vi5x4fAVtIKE0EGep7Xg9gtWtxxhjWWRUklhRyoT7ocquFApH0i+uoF6D4uiawQWn59PVojGJwMHLESiMh7QxRhIr1Q99nYyV9Lyvhl4QuLJ+et7i85xehKhtgDKpyLwvIYzrnhGMgbA0zmugI8rh9Kx4BL/hERERkelxwiMiIiLT44RHREREpscJj4iIiEzPs5SWH37/s7YNkznLNhKcVm/ftm37r7+Oyj0UiRgTPamHg3JLWtE5lKK3L0933n6WpRcf1mQBQjtuL583niMkfzablqRikqJJwKxSG6ULB6ugjzHGDmTaVVkVPkrMPrctWPEaIbm2ppuG0m+S6krbaEV7SsimpNcqNeLq0GGiaiL649ihciQk15ceKC03XFm6ysFbGF8ErnjdGtE30croUV0DROMgUfdsO6pgHa4IT5Z9krSMidZBajO2LUyYpvF0se7B9x+2v0rkJBVjonFWLiL921auG60
"text/plain": [
"<Figure size 720x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.activation_stats.color_dim(-2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Batch Normalization"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def conv(ni, nf, ks=3, act=True):\n",
" layers = [nn.Conv2d(ni, nf, stride=2, kernel_size=ks, padding=ks//2)]\n",
" layers.append(nn.BatchNorm2d(nf))\n",
" if act: layers.append(nn.ReLU())\n",
" return nn.Sequential(*layers)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.130036</td>\n",
" <td>0.055021</td>\n",
" <td>0.986400</td>\n",
" <td>00:10</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn = fit()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAADNCAYAAAC8XqoPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dy5IcR5peIy91AwgQIBtg07ppo+nWQhrrkWkxO61legK9ip5MT6B30E47mcmM1JimmySAuuRFC3aPdf7/qcoPUVm8eJ+zq6CHh4eHR8CZceLzxX6/n0RERERGZvlTN0BERETkqXHCIyIiIsPjhEdERESGxwmPiIiIDI8THhERERkeJzwiIiIyPOuH/uN/Xv5Xv1n/kVj9278/3PDdu1Zm87sv+37f3bRt17/55HC/Z6tW5uy7Tdt2+6oPh/X7Xdu2vD3cdvO677dfLnpb7/pwunl5fM692PZtOxi5F98dtmtz2duw7Kc9TTDK99CsfenG1U3fkc6b2rEu+5593/v57nlvxNm7Xm5z1euv7NbZ9Vj06qft+eG+dD6r217X1Td3bVsdKwvo+yW06/pVH8PnpS/urnp/ra/7CW0vYFz0pvbjfdcH4vayH/PiX3pld88Pz7uOpWmapt1ZbxeN/Xo96DoSS7hGd59Q+w9vkt1Z9v/Fu3MY5+8OT4D6i+6Zxe74vUXjkPr17H2vawXjotZP55P2Ybsf4BLROdLYr+3a0nMN9tvA/VCv7Q/1HZaj67F+3wfi9Wf9IbzcHLbj/Nu+H/17RNeNnkUVuo7/47//t3sfiP7CIyIiIsPjhEdERESGxwmPiIiIDI8THhERERmeB6Vl+fHYX14c/L198zLa793ve7kqcpHcNy36pV9dd/Ht9iXYZMUwu3tO0iHsBtPrKrltQdxcgyi4uu113RZ58OqfQbR73c+HxFYS/uo5bS9aEZSD6zlO0zTtSzFq1wLWuXv3617uk/9zKCK+fwu3NUqTUOy8b7t9USTZ616G6r/+/Kxtq4L1Fo538SfoL5Iay+UleXe/ykRg2rfKwdsLGBNbumd6/2+qrAu3JI0TEqyrLE/CKomtJOFS+2/KxwskpPMHASD0vji8cHXcTxOL+Nef0X16WD9ds1u4RrfwfLqA867Pzfdv+vg9h48LaOzXsUP9vF/2ti7P4EOCMl7pQ5LNMxD2P/RyH97AByaLw7ZekGj8HK4HfrRx+Dd90EJ9QWOsnjdJ/SRYP4S/8IiIiMjwOOERERGR4XHCIyIiIsPjhEdERESGR2n5Z8Li+jAxeffmWSuzvO4yGQmYVUy7+CNIgSDJ7j7tdVFKaZVYMaUWXDIST69fH865UZAE05GSllcldPrDr/oBSawkQXn94bhASn2/CB06klFbXWEq9Pe/KenFsB/1fZqYXK/vEqTDD296H56DfFxFb+rDNCH7w+eHx6RxSG2luuiYtW3Uh5SGTeL65vLw7+df93uSxnQiB68gAfrD590Gv/i2HxPHeWlrFXDvY0mp0EU0pfFF4ildo3o9rl/1vr+EZx21/w4k3/ocoGcfSfBESxz+Yz8hEoFvXxwfT6sbeB6C0EvPippOPk3TdPtJkNROyeAwzs+KUE0foUzQh3fP4P4r7af7Y00fUDyAv/CIiIjI8DjhERERkeFxwiMiIiLDo8PzM+HD7z8/WoYcHuLs3eHLT1z1F1wWem++69lb0/n3JSwQ6qeAMVrV++qfD9/53r6k99O9rsTXWL+DUDV4R06+0c2nFNZYjkehdvDefPWhb9tclbrgfHiFc2hWWXUZryPc6VQ/r+Jd/4ZwSHBB6LrtSp+dQXgZXY+7rrRNZ+9q3XA8qIzGJq6yXXyH6ptNE/tSFOhY20r+ycV34PXQOZU+/P63kN4IvAfPitta7iMIJKV7mQLl1mXbBwgUJFcmGa/U9puXwcrl0z3jvLhEFJRaQx+niYNXq1ty92W/RhScSOdUww5pHJ5/ezwEcJru8fZujj/PW3DmxO2/KUG15NBtgjDNaZqag0S+Hz3DHsJfeERERGR4nPCIiIjI8DjhERERkeFxwiMiIiLDo7T8xKy+eNu27d981rat3x8aWbefduPz+m1fnhuF4SLfkThG0h5BoVRVyKNVw0mYIyGvSsq0X5U0/1wSjnlYjuQ4EmIxxBBWY28re5OYfdm3UchZ23eXybsocG+OB0GiQEyLqi+Ot3UHjuzypm+j1d7rNhI+cWVpGAObqxLAB23A8QTtP/u+H7OGwFG/0thf0tgp46KKwdM0TXcQgEkSfA15ROET7jVeob1vq6GY9IEDB0b29tfnDB2Px0AvV+XpLYxp6i8KtcO+qAGbtPp7ENL3w76Hf++XML5IdiYpuowL7HuQyAkMdi370kcJl/9CQZm9XAut3GXnSP8e1X/b6F4j4f0h/IVHREREhscJj4iIiAyPEx4REREZHic8IiIiMjxKyyeEBOXt19+0bYuvoNzF4dyTE2+zZN9NFZkxnbdvI7kvEWCpLhI3SRZtZSg5k5Jxk9Bp8tlQBM5WY9/XbSSLEuQTBiIwkaQop0nCJAKTMNyuCQmx8L9Oe2hIlYhp/G5phXAYTzXlmFK6KYF2olXu4ZyqpE5tpbFDEnG9jyjxtsrCP+xHdR1uo+u9A4l1hYnJfd+aCn0DqdA0BqgP6zhPP2ag8Vr7rN2P0zStaPV3+LCDPkroYn82DumcmpBM5w33Mo2Ldn/DM5LGCUn89Iyn/ql8+ByE9OBaUj/fvsj+nalyNv6bBeL3Q/gLj4iIiAyPEx4REREZHic8IiIiMjxOeERERGR4lJZPCAnKyz/8u7bt5lWPjFzdHBpZ17+CGFEAU1BLmi0mDlNdII4RLU0zlC1Jkq1CXpoSTHJfTRFFyRtEQZQ5Qe6r0hy1gWRLEveq9EltpT6kpNcqau5AQiQZmaRulMZru6APWYqGbaV6EoEpSRal7sVhZZSWTGOHuP7s+PWmsbOABN0k1ZqEUroe3IclgRbSvUnWprpYBK37ZWL2LvjQgtq1eXZ8v2mapkV51uHzhPoLxjSOi1KMkn3pAwQSpet9ih92wLM7Sb6++FM25qj9JCjXduAzAO7JOg6nqaf6032L140Gf7ln6GMGelY8hL/wiIiIyPA44REREZHhccIjIiIiw6PDc0JwZXQod/7H/gL//W/gRXbhDsLEzsFbuHl5fDVleudLPgWFmvXVlGm/46FztC1d1Ts9ZqsLA/7CttZilJFIOXfBitrkh3AwINRf3m2jAwOeFZ0jnhMFytW6KOww+N8pvLbkFsF1q+eJzgWAwYborxUXDvypNfkbwerfHCwKTYDVpiu4Ajm0FR2eIIQRxxw4Nhiw2drVt62uadtx14f6a0VOR+hG1XJ0OuQITfBcruOJPKvqoE0Tj+F6H1FwH0H3La2WXsuhMwlO4/rdvNXk+blwvA/T59VD+AuPiIiIDI8THhERERkeJzwiIiIyPE54REREZHiUlk/Ji+dt0+Z1X4749tNuhW2uDueeFEp19r4f8vp1n7PW8CcS9CgsEOVmCKqq9W1pZfEgdG6aIJQvlNBoledV7YpQpCVxFldVD0ISSVqmclXARGmWvFAKAAvOG1d/DyRyIg0LTGRRPB61FVe5b60IyrDMuQGpdLmtS333MiRAoxBby9D50H7BExpXGw/HOY7hKi2TYB0GbLbzXPYTp+BBfFbUbz3wAwEQxsOPMdbl+UdyMMrmMPZrv5JYjgT3FgrQaV0gwdexQv1F9wfeb8HYT59P7XrT8zw43F/jLzwiIiIyPE54REREZHic8IiIiMjwOOERERGR4VFaDqEU5Sopb968bEW2F31OiUJvKUYrcdM2EhZJ3GvHgyu/JQmNUoKLTLYFgRiTgylFuW4LU1FRkCzlqG8wnRVWrt5SInNpG5031bWBY1ZpEkVXuoxBCjEJmQT1IaYVl2OiYEh9ncjgYaoyybW1rXRtOUEZmhXIoiTKo/RLK15X6TO9RjQ26zXCFbyhMmg/tqNsIxE1lXDrea+hXTROUMSuq2fDWI2k/nvK3Zb0aEy+BlD+L22lexLHHN1
"text/plain": [
"<Figure size 720x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.activation_stats.color_dim(-4)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.191731</td>\n",
" <td>0.121738</td>\n",
" <td>0.960900</td>\n",
" <td>00:11</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
" <td>0.083739</td>\n",
" <td>0.055808</td>\n",
" <td>0.981800</td>\n",
" <td>00:10</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>0.053161</td>\n",
" <td>0.044485</td>\n",
" <td>0.987100</td>\n",
" <td>00:10</td>\n",
" </tr>\n",
" <tr>\n",
" <td>3</td>\n",
" <td>0.034433</td>\n",
" <td>0.030233</td>\n",
" <td>0.990200</td>\n",
" <td>00:10</td>\n",
" </tr>\n",
" <tr>\n",
" <td>4</td>\n",
" <td>0.017646</td>\n",
" <td>0.025407</td>\n",
" <td>0.991200</td>\n",
" <td>00:10</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn = fit(5, lr=0.1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.183244</td>\n",
" <td>0.084025</td>\n",
" <td>0.975800</td>\n",
" <td>00:13</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
" <td>0.080774</td>\n",
" <td>0.067060</td>\n",
" <td>0.978800</td>\n",
" <td>00:12</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>0.050215</td>\n",
" <td>0.062595</td>\n",
" <td>0.981300</td>\n",
" <td>00:12</td>\n",
" </tr>\n",
" <tr>\n",
" <td>3</td>\n",
" <td>0.030020</td>\n",
" <td>0.030315</td>\n",
" <td>0.990700</td>\n",
" <td>00:12</td>\n",
" </tr>\n",
" <tr>\n",
" <td>4</td>\n",
" <td>0.015131</td>\n",
" <td>0.025148</td>\n",
" <td>0.992100</td>\n",
" <td>00:12</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn = fit(5, lr=0.1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Conclusions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Questionnaire"
]
},
2020-03-18 00:34:07 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. What is a \"feature\"?\n",
"1. Write out the convolutional kernel matrix for a top edge detector.\n",
2020-05-19 23:56:41 +00:00
"1. Write out the mathematical operation applied by a 3×
"1. What is the value of a convolutional kernel apply to a 3×
2020-05-18 21:18:08 +00:00
"1. What is \"padding\"?\n",
"1. What is \"stride\"?\n",
2020-03-18 00:34:07 +00:00
"1. Create a nested list comprehension to complete any task that you choose.\n",
2020-05-18 21:18:08 +00:00
"1. What are the shapes of the `input` and `weight` parameters to PyTorch's 2D convolution?\n",
"1. What is a \"channel\"?\n",
2020-03-18 00:34:07 +00:00
"1. What is the relationship between a convolution and a matrix multiplication?\n",
2020-05-18 21:18:08 +00:00
"1. What is a \"convolutional neural network\"?\n",
2020-03-18 00:34:07 +00:00
"1. What is the benefit of refactoring parts of your neural network definition?\n",
"1. What is `Flatten`? Where does it need to be included in the MNIST CNN? Why?\n",
"1. What does \"NCHW\" mean?\n",
"1. Why does the third layer of the MNIST CNN have `7*7*(1168-16)` multiplications?\n",
2020-05-18 21:18:08 +00:00
"1. What is a \"receptive field\"?\n",
2020-03-18 00:34:07 +00:00
"1. What is the size of the receptive field of an activation after two stride 2 convolutions? Why?\n",
2020-05-18 21:18:08 +00:00
"1. Run *conv-example.xlsx* yourself and experiment with *trace precedents*.\n",
2020-03-18 00:34:07 +00:00
"1. Have a look at Jeremy or Sylvain's list of recent Twitter \"like\"s, and see if you find any interesting resources or ideas there.\n",
"1. How is a color image represented as a tensor?\n",
"1. How does a convolution work with a color input?\n",
2020-05-18 21:18:08 +00:00
"1. What method can we use to see that data in `DataLoaders`?\n",
"1. Why do we double the number of filters after each stride-2 conv?\n",
2020-03-18 00:34:07 +00:00
"1. Why do we use a larger kernel in the first conv with MNIST (with `simple_cnn`)?\n",
"1. What information does `ActivationStats` save for each layer?\n",
"1. How can we access a learner's callback after training?\n",
"1. What are the three statistics plotted by `plot_layer_stats`? What does the x-axis represent?\n",
"1. Why are activations near zero problematic?\n",
"1. What are the upsides and downsides of training with a larger batch size?\n",
"1. Why should we avoid using a high learning rate at the start of training?\n",
"1. What is 1cycle training?\n",
"1. What are the benefits of training with a high learning rate?\n",
"1. Why do we want to use a low learning rate at the end of training?\n",
2020-05-18 21:18:08 +00:00
"1. What is \"cyclical momentum\"?\n",
2020-03-18 00:34:07 +00:00
"1. What callback tracks hyperparameter values during training (along with other information)?\n",
"1. What does one column of pixels in the `color_dim` plot represent?\n",
"1. What does \"bad training\" look like in `color_dim`? Why?\n",
"1. What trainable parameters does a batch normalization layer contain?\n",
"1. What statistics are used to normalize in batch normalization during training? How about during validation?\n",
"1. Why do models with batch normalization layers generalize better?"
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Further Research"
2020-03-06 18:19:03 +00:00
]
},
2020-03-18 00:34:07 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. What features other than edge detectors have been used in computer vision (especially before deep learning became popular)?\n",
"1. There are other normalization layers available in PyTorch. Try them out and see what works best. Learn about why other normalization layers have been developed, and how they differ from batch normalization.\n",
2020-05-18 21:18:08 +00:00
"1. Try moving the activation function after the batch normalization layer in `conv`. Does it make a difference? See what you can find out about what order is recommended, and why."
2020-03-18 00:34:07 +00:00
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"jupytext": {
"split_at_heading": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
2020-03-17 19:15:55 +00:00
"nbformat_minor": 4
2020-03-06 18:19:03 +00:00
}
