2020-03-06 18:19:03 +00:00
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
2020-08-21 19:36:27 +00:00
"from fastai.vision.all import *\n",
2020-03-06 18:19:03 +00:00
"from utils import *\n",
"\n",
"matplotlib.rc('image', cmap='Greys')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"# Under the Hood: Training a Digit Classifier"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## Pixels: The Foundations of Computer Vision"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## Sidebar: Tenacity and Deep Learning"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## End sidebar"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"path = untar_data(URLs.MNIST_SAMPLE)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"Path.BASE_PATH = path"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"(#9) [Path('cleaned.csv'),Path('item_list.txt'),Path('trained_model.pkl'),Path('models'),Path('valid'),Path('labels.csv'),Path('export.pkl'),Path('history.csv'),Path('train')]"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"path.ls()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"(#2) [Path('train/7'),Path('train/3')]"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(path/'train').ls()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(#6131) [Path('train/3/10.png'),Path('train/3/10000.png'),Path('train/3/10011.png'),Path('train/3/10031.png'),Path('train/3/10034.png'),Path('train/3/10042.png'),Path('train/3/10052.png'),Path('train/3/1007.png'),Path('train/3/10074.png'),Path('train/3/10091.png')...]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"threes = (path/'train'/'3').ls().sorted()\n",
"sevens = (path/'train'/'7').ls().sorted()\n",
"threes"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAA9ElEQVR4nM3Or0sDcRjH8c/pgrfBVBjCgibThiKIyTWbWF1bORhGwxARxH/AbtW0JoIGwzXRYhJhtuFY2q1ocLgbe3sGReTuuWbwkx6+r+/zQ/pncX6q+YOldSe6nG3dn8U/rTQ70L8FCGJUewvxl7NTmezNb8xIkvKugr1HSeMP6SrWOVkoTEuSyh0Gm2n3hQyObMnXnxkempRrvgD+gokzwxFAr7U7YXHZ8x4A/Dl7rbu6D2yl3etcw/F3nZgfRVI7rXM7hMUUqzzBec427x26rkmlkzEEa4nnRqnSOH2F0UUx0ePzlbuqMXAHgN6GY9if5xP8dmtHFfwjuQAAAABJRU5ErkJggg==\n",
"text/plain": [
2020-04-15 12:21:02 +00:00
"<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28 at 0x7F10D53E7E90>"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"im3_path = threes[1]\n",
"im3 = Image.open(im3_path)\n",
"im3"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 0, 0, 0, 0, 0, 0],\n",
" [ 0, 0, 0, 0, 0, 29],\n",
" [ 0, 0, 0, 48, 166, 224],\n",
" [ 0, 93, 244, 249, 253, 187],\n",
" [ 0, 107, 253, 253, 230, 48],\n",
" [ 0, 3, 20, 20, 15, 0]], dtype=uint8)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"array(im3)[4:10,4:10]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([[ 0, 0, 0, 0, 0, 0],\n",
" [ 0, 0, 0, 0, 0, 29],\n",
" [ 0, 0, 0, 48, 166, 224],\n",
" [ 0, 93, 244, 249, 253, 187],\n",
" [ 0, 107, 253, 253, 230, 48],\n",
" [ 0, 3, 20, 20, 15, 0]], dtype=torch.uint8)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tensor(im3)[4:10,4:10]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<style type=\"text/css\" >\n",
2020-04-15 12:21:02 +00:00
" #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row0_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #efefef;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #7c7c7c;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #4a4a4a;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #606060;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #4d4d4d;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #7c7c7c;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #bbbbbb;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row1_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #e4e4e4;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #6b6b6b;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #171717;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #4b4b4b;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #010101;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #171717;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row2_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #272727;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #0a0a0a;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #050505;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #333333;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #e6e6e6;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #fafafa;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #fbfbfb;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #fdfdfd;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #fafafa;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #4b4b4b;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #171717;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row3_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #1b1b1b;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #e0e0e0;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #4e4e4e;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #767676;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row4_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #fcfcfc;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #f6f6f6;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #f6f6f6;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #f8f8f8;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #e8e8e8;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #222222;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #090909;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #d0d0d0;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row5_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #060606;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #090909;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #979797;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row6_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #f8f8f8;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #b6b6b6;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #252525;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #010101;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #060606;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #999999;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row7_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #f9f9f9;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #6b6b6b;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #101010;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #010101;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #020202;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #010101;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #545454;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #f1f1f1;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row8_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #f7f7f7;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #060606;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #030303;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #010101;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #020202;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #010101;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #181818;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #303030;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #a9a9a9;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #fefefe;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row9_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col0 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col1 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col2 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col3 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col4 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col5 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col6 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col7 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #e8e8e8;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col8 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #bababa;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col9 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #bababa;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col10 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #393939;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col11 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col12 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col13 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col14 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col15 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col16 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #000000;\n",
" color: #f1f1f1;\n",
2020-04-15 12:21:02 +00:00
" } #T_5949cb40_7ea5_11ea_8470_251223b34593row10_col17 {\n",
2020-03-06 18:19:03 +00:00
" font-size: 6pt;\n",
" background-color: #ffffff;\n",
" color: #000000;\n",
2020-04-15 12:21:02 +00:00
" }</style><table id=\"T_5949cb40_7ea5_11ea_8470_251223b34593\" ><thead> <tr> <th class=\"blank level0\" ></th> <th class=\"col_heading level0 col0\" >0</th> <th class=\"col_heading level0 col1\" >1</th> <th class=\"col_heading level0 col2\" >2</th> <th class=\"col_heading level0 col3\" >3</th> <th class=\"col_heading level0 col4\" >4</th> <th class=\"col_heading level0 col5\" >5</th> <th class=\"col_heading level0 col6\" >6</th> <th class=\"col_heading level0 col7\" >7</th> <th class=\"col_heading level0 col8\" >8</th> <th class=\"col_heading level0 col9\" >9</th> <th class=\"col_heading level0 col10\" >10</th> <th class=\"col_heading level0 col11\" >11</th> <th class=\"col_heading level0 col12\" >12</th> <th class=\"col_heading level0 col13\" >13</th> <th class=\"col_heading level0 col14\" >14</th> <th class=\"col_heading level0 col15\" >15</th> <th class=\"col_heading level0 col16\" >16</th> <th class=\"col_heading level0 col17\" >17</th> </tr></thead><tbody>\n",
2020-03-06 18:19:03 +00:00
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col0\" class=\"data row0 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col1\" class=\"data row0 col1\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col2\" class=\"data row0 col2\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col3\" class=\"data row0 col3\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col4\" class=\"data row0 col4\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col5\" class=\"data row0 col5\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col6\" class=\"data row0 col6\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col7\" class=\"data row0 col7\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col8\" class=\"data row0 col8\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col9\" class=\"data row0 col9\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col10\" class=\"data row0 col10\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col11\" class=\"data row0 col11\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col12\" class=\"data row0 col12\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col13\" class=\"data row0 col13\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col14\" class=\"data row0 col14\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col15\" class=\"data row0 col15\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col16\" class=\"data row0 col16\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row0_col17\" class=\"data row0 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col0\" class=\"data row1 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col1\" class=\"data row1 col1\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col2\" class=\"data row1 col2\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col3\" class=\"data row1 col3\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col4\" class=\"data row1 col4\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col5\" class=\"data row1 col5\" >29</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col6\" class=\"data row1 col6\" >150</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col7\" class=\"data row1 col7\" >195</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col8\" class=\"data row1 col8\" >254</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col9\" class=\"data row1 col9\" >255</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col10\" class=\"data row1 col10\" >254</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col11\" class=\"data row1 col11\" >176</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col12\" class=\"data row1 col12\" >193</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col13\" class=\"data row1 col13\" >150</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col14\" class=\"data row1 col14\" >96</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col15\" class=\"data row1 col15\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col16\" class=\"data row1 col16\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row1_col17\" class=\"data row1 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col0\" class=\"data row2 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col1\" class=\"data row2 col1\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col2\" class=\"data row2 col2\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col3\" class=\"data row2 col3\" >48</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col4\" class=\"data row2 col4\" >166</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col5\" class=\"data row2 col5\" >224</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col6\" class=\"data row2 col6\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col7\" class=\"data row2 col7\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col8\" class=\"data row2 col8\" >234</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col9\" class=\"data row2 col9\" >196</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col10\" class=\"data row2 col10\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col11\" class=\"data row2 col11\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col12\" class=\"data row2 col12\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col13\" class=\"data row2 col13\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col14\" class=\"data row2 col14\" >233</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col15\" class=\"data row2 col15\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col16\" class=\"data row2 col16\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row2_col17\" class=\"data row2 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col0\" class=\"data row3 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col1\" class=\"data row3 col1\" >93</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col2\" class=\"data row3 col2\" >244</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col3\" class=\"data row3 col3\" >249</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col4\" class=\"data row3 col4\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col5\" class=\"data row3 col5\" >187</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col6\" class=\"data row3 col6\" >46</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col7\" class=\"data row3 col7\" >10</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col8\" class=\"data row3 col8\" >8</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col9\" class=\"data row3 col9\" >4</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col10\" class=\"data row3 col10\" >10</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col11\" class=\"data row3 col11\" >194</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col12\" class=\"data row3 col12\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col13\" class=\"data row3 col13\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col14\" class=\"data row3 col14\" >233</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col15\" class=\"data row3 col15\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col16\" class=\"data row3 col16\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row3_col17\" class=\"data row3 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col0\" class=\"data row4 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col1\" class=\"data row4 col1\" >107</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col2\" class=\"data row4 col2\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col3\" class=\"data row4 col3\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col4\" class=\"data row4 col4\" >230</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col5\" class=\"data row4 col5\" >48</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col6\" class=\"data row4 col6\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col7\" class=\"data row4 col7\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col8\" class=\"data row4 col8\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col9\" class=\"data row4 col9\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col10\" class=\"data row4 col10\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col11\" class=\"data row4 col11\" >192</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col12\" class=\"data row4 col12\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col13\" class=\"data row4 col13\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col14\" class=\"data row4 col14\" >156</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col15\" class=\"data row4 col15\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col16\" class=\"data row4 col16\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row4_col17\" class=\"data row4 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col0\" class=\"data row5 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col1\" class=\"data row5 col1\" >3</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col2\" class=\"data row5 col2\" >20</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col3\" class=\"data row5 col3\" >20</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col4\" class=\"data row5 col4\" >15</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col5\" class=\"data row5 col5\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col6\" class=\"data row5 col6\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col7\" class=\"data row5 col7\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col8\" class=\"data row5 col8\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col9\" class=\"data row5 col9\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col10\" class=\"data row5 col10\" >43</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col11\" class=\"data row5 col11\" >224</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col12\" class=\"data row5 col12\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col13\" class=\"data row5 col13\" >245</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col14\" class=\"data row5 col14\" >74</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col15\" class=\"data row5 col15\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col16\" class=\"data row5 col16\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row5_col17\" class=\"data row5 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col0\" class=\"data row6 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col1\" class=\"data row6 col1\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col2\" class=\"data row6 col2\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col3\" class=\"data row6 col3\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col4\" class=\"data row6 col4\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col5\" class=\"data row6 col5\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col6\" class=\"data row6 col6\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col7\" class=\"data row6 col7\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col8\" class=\"data row6 col8\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col9\" class=\"data row6 col9\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col10\" class=\"data row6 col10\" >249</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col11\" class=\"data row6 col11\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col12\" class=\"data row6 col12\" >245</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col13\" class=\"data row6 col13\" >126</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col14\" class=\"data row6 col14\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col15\" class=\"data row6 col15\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col16\" class=\"data row6 col16\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row6_col17\" class=\"data row6 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col0\" class=\"data row7 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col1\" class=\"data row7 col1\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col2\" class=\"data row7 col2\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col3\" class=\"data row7 col3\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col4\" class=\"data row7 col4\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col5\" class=\"data row7 col5\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col6\" class=\"data row7 col6\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col7\" class=\"data row7 col7\" >14</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col8\" class=\"data row7 col8\" >101</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col9\" class=\"data row7 col9\" >223</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col10\" class=\"data row7 col10\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col11\" class=\"data row7 col11\" >248</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col12\" class=\"data row7 col12\" >124</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col13\" class=\"data row7 col13\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col14\" class=\"data row7 col14\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col15\" class=\"data row7 col15\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col16\" class=\"data row7 col16\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row7_col17\" class=\"data row7 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col0\" class=\"data row8 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col1\" class=\"data row8 col1\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col2\" class=\"data row8 col2\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col3\" class=\"data row8 col3\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col4\" class=\"data row8 col4\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col5\" class=\"data row8 col5\" >11</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col6\" class=\"data row8 col6\" >166</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col7\" class=\"data row8 col7\" >239</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col8\" class=\"data row8 col8\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col9\" class=\"data row8 col9\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col10\" class=\"data row8 col10\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col11\" class=\"data row8 col11\" >187</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col12\" class=\"data row8 col12\" >30</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col13\" class=\"data row8 col13\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col14\" class=\"data row8 col14\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col15\" class=\"data row8 col15\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col16\" class=\"data row8 col16\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row8_col17\" class=\"data row8 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col0\" class=\"data row9 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col1\" class=\"data row9 col1\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col2\" class=\"data row9 col2\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col3\" class=\"data row9 col3\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col4\" class=\"data row9 col4\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col5\" class=\"data row9 col5\" >16</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col6\" class=\"data row9 col6\" >248</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col7\" class=\"data row9 col7\" >250</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col8\" class=\"data row9 col8\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col9\" class=\"data row9 col9\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col10\" class=\"data row9 col10\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col11\" class=\"data row9 col11\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col12\" class=\"data row9 col12\" >232</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col13\" class=\"data row9 col13\" >213</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col14\" class=\"data row9 col14\" >111</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col15\" class=\"data row9 col15\" >2</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col16\" class=\"data row9 col16\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row9_col17\" class=\"data row9 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" <tr>\n",
2020-04-15 12:21:02 +00:00
" <th id=\"T_5949cb40_7ea5_11ea_8470_251223b34593level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col0\" class=\"data row10 col0\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col1\" class=\"data row10 col1\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col2\" class=\"data row10 col2\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col3\" class=\"data row10 col3\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col4\" class=\"data row10 col4\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col5\" class=\"data row10 col5\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col6\" class=\"data row10 col6\" >0</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col7\" class=\"data row10 col7\" >43</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col8\" class=\"data row10 col8\" >98</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col9\" class=\"data row10 col9\" >98</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col10\" class=\"data row10 col10\" >208</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col11\" class=\"data row10 col11\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col12\" class=\"data row10 col12\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col13\" class=\"data row10 col13\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col14\" class=\"data row10 col14\" >253</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col15\" class=\"data row10 col15\" >187</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col16\" class=\"data row10 col16\" >22</td>\n",
" <td id=\"T_5949cb40_7ea5_11ea_8470_251223b34593row10_col17\" class=\"data row10 col17\" >0</td>\n",
2020-03-06 18:19:03 +00:00
" </tr>\n",
" </tbody></table>"
],
"text/plain": [
2020-04-15 12:21:02 +00:00
"<pandas.io.formats.style.Styler at 0x7f10d4f3b8d0>"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"im3_t = tensor(im3)\n",
"df = pd.DataFrame(im3_t[4:15,4:22])\n",
"df.style.set_properties(**{'font-size':'6pt'}).background_gradient('Greys')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## First Try: Pixel Similarity"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(6131, 6265)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"seven_tensors = [tensor(Image.open(o)) for o in sevens]\n",
"three_tensors = [tensor(Image.open(o)) for o in threes]\n",
"len(three_tensors),len(seven_tensors)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAADjElEQVR4nO2aPyh9YRjHP/f4k38L5X+ysohsUpTBhEVMJGUyGAwWg0kGkcFqlMFIyv+kSGIwKWUiUvKn5P/9DXrvcR+He+695957+vV8llPnvvd9n77n2/s8z3tOIBgMothYqQ7Ab6ggAhVEoIIIVBBBeoTf/+cUFHC6qQ4RqCACFUSggghUEIEKIlBBBCqIQAURRKpUPeHh4QGAyclJAI6PjwFYXl4GIBgMEgh8FY59fX0A3N7eAlBTUwNAU1MTAC0tLQmNVR0iCEQ4MYupl7m4uABgYmICgJWVFQDOz8/DxhUVFQFQX18fGvMbxcXFAFxeXsYSkhPay7jBkz1ke3sbgLa2NgBeX18BeH9/B6CzsxOAnZ0dAAoLCwFC+4ZlWXx8fISNXVpa8iK0qFGHCDxxyN3dHQBPT09h98vLywGYmpoCoKys7Nc5LMsKu0p6enrijtMN6hCBJ1nm8/MTgOfn57D75mlnZWVFnOPq6gqAxsZGwM5I2dnZAOzu7gJQW1vrJiQ3aJZxgyd7iHFCTk5OzHNUVlYCdmYyzjDVrYfO+BN1iCApvYzk5eUFgM3NTQCGhoZCzsjMzARgenoagIGBgaTGpg4RJMUhpnIdHh4GYH5+HrDrl++0t7cD0NXVlYzQfqAOESSk25WY+iQ/Px8g1LeYqxMlJSUAlJaWAjAyMgLYvY7pg+LAcYKkCCIxRdjJyUno3tjYGAD7+/t//tcIMjc3B0Bubm6sYWhh5oaUOMSJt7c3wHaPScn9/f2O4w8PDwGoq6uLdUl1iBtSUpg5kZGRAUBFRQUAvb29AKyurgKwsLAQNn5tbQ2IyyGOqEMEvnGIxKTV39JrdXV1QtZVhwh8k2Uke3t7ADQ3NwP2sYDh5uYGgIKCgliX0CzjBt/tIWdnZwAMDg4CP51h6pK8vLyErK8OEfhmDzF1RUdHB2AfIhnMEePp6Slg1y1xoHuIG1K6h1xfXwMwOzvL+Pg48PVpxHfMS+6trS3AE2f8iTpE4KlDzBPf2NgA7I9bHh8fATg4OADg6OgIsM807u/vQ3OkpaUB9qvLmZkZIHFZRaIOEXiaZbq7uwFYXFyMOpDW1lYARkdHAWhoaIh6jijRLOMGTx1iPnIxtUQkzEHy+vo6VVVVXwHFf3jsFnWIG3xTqaYAdYgbVBCBCiJQQQQqiCBSL5O0osAvqEMEKohABRGoIAIVRKCCCP4B/PMI7HrW9/wAAAAASUVORK5CYII=\n",
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 72x72 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"show_image(three_tensors[1]);"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([6131, 28, 28])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"stacked_sevens = torch.stack(seven_tensors).float()/255\n",
"stacked_threes = torch.stack(three_tensors).float()/255\n",
"stacked_threes.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(stacked_threes.shape)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"stacked_threes.ndim"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAE1klEQVR4nO2byU8jPRTEf2En7AgQO4gDi9hO8P9fOIE4AGIR+xoU1kAgQAKZA6o4eUMU6O7R983IdbE66XYHv3K98rOJ5fN5PByq/usf8H+DHxADPyAGfkAM/IAY1FT4/l9OQbGvPvQMMfADYuAHxMAPiIEfEINKWSYSaL1kW/t9MWKx2JfX5T6PCp4hBpEwxEb+/f0dgFwuB0A2mwXg6emppH1+fgbg9fWVj48PAGprawGIx+MANDc3A9DU1ARAfX19yX3V1dVAeQb9FJ4hBqEYIkZYJmQyGQBub28BuLi4AGBnZweAvb09AM7PzwFIJpOFPsSEvr4+AMbHxwGYnZ0FYHR0FICuri7AMUiMqar6jHFQpniGGARiSDmtkDZcXV0BsL+/D8Da2hoAu7u7ABwcHACOOalUipeXl5J3tLW1AXB6elrS58LCAgBTU1MA9Pf3A44pYkhQeIYYhGKIMoMYoigXZw+AmprP13R2dgJuvo+NjQGf2qNnbm5uSvp4e3sD4O7uDnC6I41Rn8pK+m1eQyJCJFlG0VDkW1tbARgaGgKgo6MDcFEX5CFyuVwhIx0dHQFwcnICOF3SO8RGsbOc+w0KzxCDUAyRoivSmse6lvIrGymqgqKayWRIJBIAXF9fl/Ste6RD8il6V11dXcn93qlGjEAMURQUFUVP19ISO8/FFGWfx8dH4NNjbGxsALC1tQU4DWloaAAc2wYGBgCXXRobGwHHyrDwDDGIREMsY8QMtVrjKMtcXl4CsLm5CcDq6irr6+uAc7fqc35+HoDe3l7AOVNlsqjWMIW/KdTT/yBCaUglVyjNSKVSgIv+0tISACsrKwAsLy8XHKhYJScqBrS0tABOU6JmhuAZYhBKQyxTBF0rm2ilKp3Q6lcMSSQSBWbIX6gyJv2RPxHburu7S+6zlbOgiLTIbBd9tjygHytBnJ6eBmBkZKTQh50KelZlABWZ2tvbATeFoiol+iljEMnizk4Zu9iTiVIZUNcyZvl8vsAmlR+TySRAwdI/PDwAcHh4CMDw8DDgmGItfFCj5hliEKpAZDWjXDnAFnGkGcXPSyskmipEizlKy/peDBJTrFELWijyDDH4EUMsI2xrmaPoKDVqnlsUM0T3SDO03NcC0pYrldpticFrSEQIpCG2uKxWURLEEEVLGcBuFcRisd88ixiirKN32ixiWesXdxEjlIbYTWw7nxUt6YLdqC4uHIsRcqTb29uA8yF6l5yptEV9e6f6hxDKh2i+q/CjzSQ5UGUCRdEu3IR0Os3x8THgmCEfoj61ua1FXU9PD+BKi9apBoVniMGPGGLnp90qSKfTgCsEyV1KY3SfXcmmUqlCiUDPaAtzcHAQcI50cnIScAUkOVT5FJ9lIkYgDZGiKyrSBm0JCPf394DTA2UQfS6NyWazBdboGMTMzAwAc3NzACwuLgIwMTEBOA0pVw8JCs8Qg0AaomhK2VUA1haBXcvY+8/OzgDnQuPxeEEjdERCtZNymmFLh2Gzi+AZYhCrcIzgyy8rHcOUY1V2kQtVK99SfBRTkbetdMkew4xg+8H/e8h3EIghZW8uc4S70tFu+N3jVGojgGfIdxApQ/4yeIZ8B35ADPyAGFRyqtH+d85fAM8QAz8gBn5ADPyAGPgBMfADYvALMumtb+Vr5kIAAAAASUVORK5CYII=\n",
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 72x72 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"mean3 = stacked_threes.mean(0)\n",
"show_image(mean3);"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAElUlEQVR4nO2bSUszWRSGn3JKUGOM84giCA7oQnHj33ejiKCI4MIpxhES5ylx6oW8dTvnMyaWJd1033dzqVRqyLlPnelWgvf3d7yc6v7pG/i3yRvEyBvEyBvEyBvEqKHK/v9yCAo++9ATYuQNYuQNYuQNYuQNYuQNYuQNYuQNYuQNYlQtU42kaj2Wz/YHwaeJY+TvRZUnxOhHhGimNb69vZWNr6+vX25r/Ep1dR9zVl9f/3HDDQ1l29qvUQRFJckTYvQtQiwRmvGXlxcAnp6eALi9vQXg+voagMvLSwAuLi7KxoeHh/A4nUOjrtHU1ARAR0cHAENDQwAMDw8D0NvbC0BraysAjY2NgCPou6R4QowiEaJZfH5+BhwR+XwegIODAwC2t7cB2NvbA+Dw8BCAo6MjAK6urgAoFovhuUSdRhEiMubn5wFYXFwEYGFhoWx/JZ9SqzwhRpEIUXTQrN7f3wNQKBQA2N/fB2B3dxdwpIicm5ubsvMlEgkSiQTgyBB1Oufj4yPgfMXo6GjZtXWc5KNMTKqJkGqZp2ZDnr25uRmA9vZ2AEZGRgDo7u4GnF/Q/nQ6HRKiGRdNq6urgPM3IsFe0+YlUeUJMaqJEJv9aRaUNYqIzs5OAMbGxsr29/f3A44Mbff19QEffkEzrBxlaWkJgLOzM8DlOJlMpmxsa2sDXP4RNbpInhCjb0WZaoTYGkVEaDuVSgGOJM1uQ0NDmNtYKdroXD09PYDzS+l0GnCE/LQa9oQYRap2K1WglhRFDn2/paUF+LPuCIIgPEY5irLb8/NzwNUyqmEGBgbKrql7+akiPTKSNYx+oH64DCKDCXttS6+vr2G43dzcBGB9fR1wj8zU1BTgHhWFbHsuSamCT91/qB81iKyTFSkiwTo6jXo8NIulUolsNgvA8vIyADs7O4CjTKFcox6VuFuKnhCjSIRU8iX2ubUNJdtYUnGYz+dZWVkBYG1tDXCJ2OzsLACTk5OAC7vJZLLs2nGR4gkximUZwhZalZrOkvarpM/lcmxsbAAuzKoQVENoZmYGcOFXfsq2Cn1iFrNiiTJ22/oSjXYZQknY1tYWuVwOcDM/NzcHOB8yODgIuKRO+YdtGVa6t1rlCTGK1YdUyg7tfvmO09NTALLZbLgkoTxjenoagImJCcBlprbMj4sMyRNiFOtityVBks8olUqAawIpGy0UCiEBKt7Gx8cB6OrqAqrnHT4P+SXFSkiljFRkaGnz5OQEcO3BIAjCdqKWF7TwpMrZRpXfei3CE2IUCyGVXouwC1la6jw+PgZcvZJKpcJ2oho/2lZe8tPlhVrlCTH6FR+ihrF8h7peWmy6u7sDXE6RyWTCaKJqVv0O+Y64apVq8oQY/corVYou8hHKTLUtMuQnEolE+OLL3z+D6C++RJUnxCiWarfaYriI0EKVljIVhZLJZLjgpIxVmWmlrvpvyRNiFFSZ3Zr+YmZ9iH3lStFGPqRYLJYdFwRBSJHIkA+xL9HF2CHzfzGrRbEQ8sdBFc5po9JX37cE/EIe4gmpRdUI+d/JE2LkDWLkDWLkDWLkDWLkDWL0F7hnDWZImx+vAAAAAElFTkSuQmCC\n",
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 72x72 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"mean7 = stacked_sevens.mean(0)\n",
"show_image(mean7);"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAADjElEQVR4nO2aPyh9YRjHP/f4k38L5X+ysohsUpTBhEVMJGUyGAwWg0kGkcFqlMFIyv+kSGIwKWUiUvKn5P/9DXrvcR+He+695957+vV8llPnvvd9n77n2/s8z3tOIBgMothYqQ7Ab6ggAhVEoIIIVBBBeoTf/+cUFHC6qQ4RqCACFUSggghUEIEKIlBBBCqIQAURRKpUPeHh4QGAyclJAI6PjwFYXl4GIBgMEgh8FY59fX0A3N7eAlBTUwNAU1MTAC0tLQmNVR0iCEQ4MYupl7m4uABgYmICgJWVFQDOz8/DxhUVFQFQX18fGvMbxcXFAFxeXsYSkhPay7jBkz1ke3sbgLa2NgBeX18BeH9/B6CzsxOAnZ0dAAoLCwFC+4ZlWXx8fISNXVpa8iK0qFGHCDxxyN3dHQBPT09h98vLywGYmpoCoKys7Nc5LMsKu0p6enrijtMN6hCBJ1nm8/MTgOfn57D75mlnZWVFnOPq6gqAxsZGwM5I2dnZAOzu7gJQW1vrJiQ3aJZxgyd7iHFCTk5OzHNUVlYCdmYyzjDVrYfO+BN1iCApvYzk5eUFgM3NTQCGhoZCzsjMzARgenoagIGBgaTGpg4RJMUhpnIdHh4GYH5+HrDrl++0t7cD0NXVlYzQfqAOESSk25WY+iQ/Px8g1LeYqxMlJSUAlJaWAjAyMgLYvY7pg+LAcYKkCCIxRdjJyUno3tjYGAD7+/t//tcIMjc3B0Bubm6sYWhh5oaUOMSJt7c3wHaPScn9/f2O4w8PDwGoq6uLdUl1iBtSUpg5kZGRAUBFRQUAvb29AKyurgKwsLAQNn5tbQ2IyyGOqEMEvnGIxKTV39JrdXV1QtZVhwh8k2Uke3t7ADQ3NwP2sYDh5uYGgIKCgliX0CzjBt/tIWdnZwAMDg4CP51h6pK8vLyErK8OEfhmDzF1RUdHB2AfIhnMEePp6Slg1y1xoHuIG1K6h1xfXwMwOzvL+Pg48PVpxHfMS+6trS3AE2f8iTpE4KlDzBPf2NgA7I9bHh8fATg4OADg6OgIsM807u/vQ3OkpaUB9qvLmZkZIHFZRaIOEXiaZbq7uwFYXFyMOpDW1lYARkdHAWhoaIh6jijRLOMGTx1iPnIxtUQkzEHy+vo6VVVVXwHFf3jsFnWIG3xTqaYAdYgbVBCBCiJQQQQqiCBSL5O0osAvqEMEKohABRGoIAIVRKCCCP4B/PMI7HrW9/wAAAAASUVORK5CYII=\n",
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 72x72 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"a_3 = stacked_threes[1]\n",
"show_image(a_3);"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(tensor(0.1114), tensor(0.2021))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dist_3_abs = (a_3 - mean3).abs().mean()\n",
"dist_3_sqr = ((a_3 - mean3)**2).mean().sqrt()\n",
"dist_3_abs,dist_3_sqr"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(tensor(0.1586), tensor(0.3021))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dist_7_abs = (a_3 - mean7).abs().mean()\n",
"dist_7_sqr = ((a_3 - mean7)**2).mean().sqrt()\n",
"dist_7_abs,dist_7_sqr"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(tensor(0.1586), tensor(0.3021))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"F.l1_loss(a_3.float(),mean7), F.mse_loss(a_3,mean7).sqrt()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### NumPy Arrays and PyTorch Tensors"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"data = [[1,2,3],[4,5,6]]\n",
"arr = array (data)\n",
"tns = tensor(data)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[1, 2, 3],\n",
" [4, 5, 6]])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"arr # numpy"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([[1, 2, 3],\n",
" [4, 5, 6]])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tns # pytorch"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([4, 5, 6])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tns[1]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([2, 5])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tns[:,1]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([5, 6])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tns[1,1:3]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([[2, 3, 4],\n",
" [5, 6, 7]])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tns+1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'torch.LongTensor'"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tns.type()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([[1.5000, 3.0000, 4.5000],\n",
" [6.0000, 7.5000, 9.0000]])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tns*1.5"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## Computing Metrics Using Broadcasting"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(torch.Size([1010, 28, 28]), torch.Size([1028, 28, 28]))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"valid_3_tens = torch.stack([tensor(Image.open(o)) \n",
" for o in (path/'valid'/'3').ls()])\n",
"valid_3_tens = valid_3_tens.float()/255\n",
"valid_7_tens = torch.stack([tensor(Image.open(o)) \n",
" for o in (path/'valid'/'7').ls()])\n",
"valid_7_tens = valid_7_tens.float()/255\n",
"valid_3_tens.shape,valid_7_tens.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(0.1114)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def mnist_distance(a,b): return (a-b).abs().mean((-1,-2))\n",
"mnist_distance(a_3, mean3)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"(tensor([0.1050, 0.1526, 0.1186, ..., 0.1122, 0.1170, 0.1086]),\n",
2020-03-06 18:19:03 +00:00
" torch.Size([1010]))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"valid_3_dist = mnist_distance(valid_3_tens, mean3)\n",
"valid_3_dist, valid_3_dist.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([2, 3, 4])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tensor([1,2,3]) + tensor([1,1,1])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([1010, 28, 28])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(valid_3_tens-mean3).shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def is_3(x): return mnist_distance(x,mean3) < mnist_distance(x,mean7)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(tensor(True), tensor(1.))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"is_3(a_3), is_3(a_3).float()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([True, True, True, ..., True, True, True])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"is_3(valid_3_tens)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(tensor(0.9168), tensor(0.9854), tensor(0.9511))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"accuracy_3s = is_3(valid_3_tens).float() .mean()\n",
"accuracy_7s = (1 - is_3(valid_7_tens).float()).mean()\n",
"\n",
"accuracy_3s,accuracy_7s,(accuracy_3s+accuracy_7s)/2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-03-17 19:15:55 +00:00
"## Stochastic Gradient Descent (SGD)"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
" -->\n",
"<!-- Title: G Pages: 1 -->\n",
2020-04-15 12:21:02 +00:00
"<svg width=\"591pt\" height=\"78pt\"\n",
" viewBox=\"0.00 0.00 591.49 78.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
2020-03-06 18:19:03 +00:00
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 74)\">\n",
"<title>G</title>\n",
2020-04-15 12:21:02 +00:00
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-74 587.4867,-74 587.4867,4 -4,4\"/>\n",
2020-03-06 18:19:03 +00:00
"<!-- init -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>init</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"27\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"27\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">init</text>\n",
"</g>\n",
"<!-- predict -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>predict</title>\n",
2020-04-15 12:21:02 +00:00
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"126.0969\" cy=\"-18\" rx=\"35.194\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"126.0969\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">predict</text>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"<!-- init->predict -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>init->predict</title>\n",
2020-04-15 12:21:02 +00:00
"<path fill=\"none\" stroke=\"#000000\" d=\"M54.0787,-18C62.3227,-18 71.6196,-18 80.7269,-18\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"80.8626,-21.5001 90.8626,-18 80.8625,-14.5001 80.8626,-21.5001\"/>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"<!-- loss -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>loss</title>\n",
2020-04-15 12:21:02 +00:00
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"225.1938\" cy=\"-52\" rx=\"27\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"225.1938\" y=\"-48.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">loss</text>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"<!-- predict->loss -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>predict->loss</title>\n",
2020-04-15 12:21:02 +00:00
"<path fill=\"none\" stroke=\"#000000\" d=\"M155.2932,-28.0172C166.6224,-31.9043 179.6698,-36.3808 191.4018,-40.406\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"190.2859,-43.7234 200.8806,-43.6582 192.5577,-37.1023 190.2859,-43.7234\"/>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"<!-- gradient -->\n",
"<g id=\"node4\" class=\"node\">\n",
"<title>gradient</title>\n",
2020-04-15 12:21:02 +00:00
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"361.8403\" cy=\"-52\" rx=\"39.7935\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"361.8403\" y=\"-48.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">gradient</text>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"<!-- loss->gradient -->\n",
"<g id=\"edge3\" class=\"edge\">\n",
"<title>loss->gradient</title>\n",
2020-04-15 12:21:02 +00:00
"<path fill=\"none\" stroke=\"#000000\" d=\"M252.5178,-52C269.4967,-52 291.836,-52 311.8929,-52\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"312.1329,-55.5001 322.1329,-52 312.1328,-48.5001 312.1329,-55.5001\"/>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"<!-- step -->\n",
"<g id=\"node5\" class=\"node\">\n",
"<title>step</title>\n",
2020-04-15 12:21:02 +00:00
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"465.4867\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"465.4867\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">step</text>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"<!-- gradient->step -->\n",
"<g id=\"edge4\" class=\"edge\">\n",
"<title>gradient->step</title>\n",
2020-04-15 12:21:02 +00:00
"<path fill=\"none\" stroke=\"#000000\" d=\"M394.0665,-41.4286C405.9515,-37.5298 419.4492,-33.1021 431.4862,-29.1535\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"432.7754,-32.4142 441.1862,-25.9715 430.5935,-25.7629 432.7754,-32.4142\"/>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"<!-- step->predict -->\n",
"<g id=\"edge6\" class=\"edge\">\n",
"<title>step->predict</title>\n",
2020-04-15 12:21:02 +00:00
"<path fill=\"none\" stroke=\"#000000\" d=\"M438.4132,-18C380.3272,-18 243.2155,-18 171.5401,-18\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"171.4571,-14.5001 161.4571,-18 171.4571,-21.5001 171.4571,-14.5001\"/>\n",
"<text text-anchor=\"middle\" x=\"287.1938\" y=\"-21.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">repeat</text>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"<!-- stop -->\n",
"<g id=\"node6\" class=\"node\">\n",
"<title>stop</title>\n",
2020-04-15 12:21:02 +00:00
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"556.4867\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"556.4867\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">stop</text>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"<!-- step->stop -->\n",
"<g id=\"edge5\" class=\"edge\">\n",
"<title>step->stop</title>\n",
2020-04-15 12:21:02 +00:00
"<path fill=\"none\" stroke=\"#000000\" d=\"M492.7897,-18C501.068,-18 510.3085,-18 519.1272,-18\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"519.203,-21.5001 529.203,-18 519.203,-14.5001 519.203,-21.5001\"/>\n",
2020-03-06 18:19:03 +00:00
"</g>\n",
"</g>\n",
"</svg>\n"
],
"text/plain": [
2020-04-15 12:21:02 +00:00
"<graphviz.files.Source at 0x7f10d475e550>"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gv('''\n",
"init->predict->loss->gradient->step->stop\n",
"step->predict[label=repeat]\n",
"''')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def f(x): return x**2"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEMCAYAAADeYiHoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3yV5f3/8dcnm0xWElYGYQ9ZBmQ5aVW0ai1UQdwojmptrbVLq1XbX7Xj2yUqiiI46ihK3a1VVBCRsIQgG0ICJCSMkD0/vz/OoY3xBE5Czn2fJJ/n43E/esaVc7+9m5wP133d93WJqmKMMcY0FuJ2AGOMMcHJCoQxxhifrEAYY4zxyQqEMcYYn6xAGGOM8SnM7QCtpXv37pqenu52DGOMaVNWr15dpKqJvt5rNwUiPT2drKwst2MYY0ybIiI5Tb1np5iMMcb4ZAXCGGOMT1YgjDHG+GQFwhhjjE9WIIwxxvjkeIEQkQEiUikizzXxvojIwyJy0Ls9IiLidE5jjOno3LjM9VFg1XHenwN8GxgJKPBvYCfweOCjGWOMOcbRHoSIzACOAP85TrNrgD+oap6q7gX+AFwbqEzrco/w8LubA/XxxhgTMKrKr9/aRPa+4oB8vmMFQkTigQeAH52g6TBgfYPn672v+frMOSKSJSJZhYWFLcq1Ie8Ijy3dwca9gTnAxhgTKJ/tPMSTn+xiS35JQD7fyR7Eg8B8Vc09QbtYoOG3dTEQ62scQlXnqWqmqmYmJvq8U/yELh7Vm8iwEP6+ak+Lft4YY9zy0qo9xEWFMXV4z4B8viMFQkRGAd8A/s+P5qVAfIPn8UCpBmjpu4RO4VxwSk+WrNtHRXVdIHZhjDGtrri8hnc25nPJqF50iggNyD6c6kGcBaQDe0QkH7gLmCYia3y0zcYzQH3MSO9rAXNZZgollbW8s3F/IHdjjDGtZsn6vVTV1jNjbGrA9uFUgZgH9ANGebfHgbeA83y0XQjcKSK9RaQXnjGLBYEMNz6jK+ndonlp1YnOfhljTHB4aVUuQ3vGM7x3QsD24UiBUNVyVc0/tuE5jVSpqoUicrqIlDZo/gTwBrAB2IinkDwRyHwiwnczU1i56xA7C0tP/APGGOOijXuLyd53lBnjUgK6H1fupFbV+1X1Su/jT1Q1tsF7qqp3q2pX73Z3oMYfGpp+ah9CQ4SXs/ICvStjjDkpf1+1h8iwEC4Z2Tug+7GpNryS46M4e1Air67Oo6au3u04xhjjU0V1HUvW7WPq8B4kRIcHdF9WIBq4fGwqRaVVfLD5gNtRjDHGp7c37KekspbLxgb29BJYgfiKswclkhwfyYuf2z0Rxpjg9OLne+jbPYYJGd0Cvi8rEA2EhYZwWWYKH20tZO+RCrfjGGPMV2wrKCEr5zAzxqbgxBymViAauSzT022zS16NMcHmxc9zCQ8Vpp3ax5H9WYFoJKVrNGcMSOSVrFxqbbDaGBMkKmvqWLw2j3OH9aB7bKQj+7QC4cPMcSnsL67ko60tmwDQGGNa23vZ+Rwpr2FmAO+cbswKhA9ThiTTPTaSFz+300zGmODw4ud7SO0azcR+gR+cPsYKhA/hoSFcltmHDzYXkF9c6XYcY0wHt7OwlM92HmLGuBRCQpxbYNMKRBNmjE2lXm2w2hjjvhc/30NYiDDdocHpY6xANCG1WzRnDEzk76v22GC1McY1lTV1vLI6j/OG9SApLsrRfVuBOI5Zp6Wyv7iSD7fYYLUxxh3vbNzPkfIarjjNucHpY6xAHMeUwUkkx0fy/Moct6MYYzqo5z9z7s7pxqxAHEdYaAiXj03lo62F5B4qdzuOMaaD2Zx/lKycw1wxLtXRweljrECcwIyxKQjY/EzGGMe9sHIPEaEhjt053ZhjBUJEnhOR/SJyVES2isgNTbS7VkTqRKS0wXaWUzkb69W5E+cMTublrDyqa22w2hjjjPLqWl5bs5cLTulB15gIVzI42YP4f0C6qsYDFwMPicipTbRdoaqxDbaljqX0YdZ4zzTg72XnuxnDGNOB/HPdPkqqarnitDTXMjhWIFQ1W1Wrjj31bv2c2v/JOGNAIildO/HcZzZYbYwJPFVl4YocBiXHMTa9i2s5HB2DEJG5IlIObAb2A2830XS0iBR5T0XdKyJhTXzeHBHJEpGswsLAXYoaGiLMOi2NlbsOsbWgJGD7McYYgLW5R9i0/yhXTkhzZFrvpjhaIFT1ViAOOB1YDFT5aPYxMBxIAqYBM4EfN/F581Q1U1UzExMTAxPa67LMFCLCQqwXYYwJuOdW5BAbGcalowO75vSJOH4Vk6rWqeoyoA9wi4/3d6rqLlWtV9UNwAPAdKdzNtY1JoJvndKTxWv2UlpV63YcY0w7daismje/2M93xvQmNtLnyRPHuHmZaxj+jUEo4F4fq4ErJ6RRWlXL62v3uh3FGNNOvZyVS3VdPVeOd29w+hhHCoSIJInIDBGJFZFQETkPz6mjD3y0nSoiyd7Hg4F7gSVO5DyR0SmdGdYrnuc+y0FV3Y5jjGln6uqV51fmcFrfrgxMjnM7jmM9CMVzOikPOAz8HviBqi4RkVTvvQ7HJhqZAnwhImV4BrEXA79xKOdxiQhXjU9jc75nXVhjjGlNH28tJPdQBVdNcL/3AJ7TPAGnqoXAmU28tweIbfD8LuAuJ3K1xMWjevGbt7/k2U93Mza9q9txjDHtyLMrdpMYF8m5Q3u4HQWwqTaaLToijMsyU3h3Yz4FR20xIWNM69hdVMbSLYXMOi2ViLDg+GoOjhRtzFUT0qhT5fmVNj+TMaZ1LFyRQ3iouDKtd1OsQLRAWrcYzh6UxAsr99j8TMaYk1ZWVcsrq3OZOryn44sCHY8ViBa6ekIaRaVVvLNxv9tRjDFt3Gtr91JSWcs1E4NjcPoYKxAtdMaARPp2j+HZT3e7HcUY04Z55l3azfDe8YxJdW/eJV+sQLRQSIjnktc1e46wIa/Y7TjGmDZqxc6DbC0o5eoJ6a7Ou+SLFYiTMD2zD9ERoTzz6S63oxhj2qgFy3fTJTqci0f2cjvK11iBOAnxUeFMP7UPb67fT2GJr3kHjTGmabmHynn/ywJmjkslKjzU7ThfYwXiJF0zMZ3qunpesEtejTHNtHDFbs8MDUFy53RjViBOUr/EWM4alMhzK3PskldjjN/Kqmr5+6pcpg7vQc+ETm7H8ckKRCu4dmI6hSVVvLVhn9tRjDFtxOI1eZRU1nLdpHS3ozTJCkQrOGNAIhmJMTyzfLfN8mqMOaH6euWZT3czok9C0F3a2pAViFYQEiJcNzGdL/KKWbPHZnk1xhzfJ9uL2FlYxnWTgu/S1oasQLSS74zpQ3xUGE8v3+12FGNMkHt62S4S4yK58JTgu7S1ISsQrSQmMoyZ41J5Z8N+cg+Vux3HGBOkthWU8NHWQq4enxY0s7Y2xbF0IvKciOwXkaMislVEbjhO2x+KSL6IFIvI0yIS6VTOk3HNRE930abfMMY05enlu4gMC2FWECwpeiJOlq//B6SrajxwMfCQiJzauJF3OdKf4llZLh3IAH7lYM4W69W5Exec0pOXVuVSUlnjdhxjTJA5WFrFP9bs5Ttj+tA1JsLtOCfkWIFQ1WxVPXa7sXq3fj6aXgPM97Y/DDwIXOtMypM3e3JfSqpqeTkrz+0oxpgg87x3iYDZk9PdjuIXR0+AichcESkHNgP78aw53dgwYH2D5+uBZBHp5uPz5ohIlohkFRYWBiRzc41K6UxmWheeWb6Lunq75NUY41FVW8fCFTmcNSiR/klxbsfxi6MFQlVvBeKA04HFgK8JjGKBhtOjHnv8tSOqqvNUNVNVMxMTE1s7bovNntyXvMMV/Cs73+0oxpgg8c91+ygqrWL25L5uR/Gb40PoqlqnqsuAPsAtPpqUAvENnh97XBLobK3l3GE9SOnaiaeW2SyvxhjPmg/zl+1iUHIck/t3dzuO39y8xioM32MQ2cDIBs9HAgWqetCRVK0gNESYPakvq3MOszrnkNtxjDEu+3hbEZvzS7jxjIygvjGuMUcKhIgkicgMEYkVkVDvlUo
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plot_function(f, 'x', 'x**2')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEMCAYAAADeYiHoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3yV5f3/8dcnm0xWEmYSwh6yDMhy0qpo1VqogrhRHNXaWrVDrdb1q3Z8OxQVRREcdRSlbmsVFUQkbIJsCGEkJIyQPT+/P86hjfEETkLOfZ8kn+fjcT96xpVzv72bnA/Xfd33dYmqYowxxtQX4nYAY4wxwckKhDHGGJ+sQBhjjPHJCoQxxhifrEAYY4zxKcztAM2lc+fOmpaW5nYMY4xpUVasWFGgqom+3ms1BSItLY3MzEy3YxhjTIsiItkNvWenmIwxxvhkBcIYY4xPViCMMcb4ZAXCGGOMT1YgjDHG+OR4gRCRviJSLiIvNvC+iMijInLAuz0mIuJ0TmOMaevcuMz1CWD5Md6fCfwQGAYo8G9gO/BU4KMZY4w5ytEehIhMBQ4D/zlGs6uAP6nqblXdA/wJuDpQmVbnHObRDzYG6uONMSZgVJWH391A1t7CgHy+YwVCROKBB4BfHKfpYGBNnedrvK/5+syZIpIpIpn5+flNyrVu92GeXLSN9XsCc4CNMSZQvtp+kGe+2MGm3KKAfL6TPYgHgTmqmnOcdrFA3W/rQiDW1ziEqs5W1QxVzUhM9Hmn+HFdOLw7kWEh/GP5rib9vDHGuOXV5buIiwpj0pCuAfl8RwqEiAwHvgf8nx/Ni4H4Os/jgWIN0NJ3Ce3COe+krixcvZeyyppA7MIYY5pdYWkV76/P5aLh3WgXERqQfTjVgzgDSAN2iUgucAcwWURW+mibhWeA+qhh3tcC5pKMnhSVV/P++n2B3I0xxjSbhWv2UFFdy9RRKQHbh1MFYjbQGxju3Z4C3gXO8dF2HnC7iHQXkW54xizmBjLcmPSOpHWK5tXlxzv7ZYwxweHV5TkM6hrPkO4JAduHIwVCVUtVNffohuc0Urmq5ovIqSJSXKf508DbwDpgPZ5C8nQg84kIP87oybIdB9meX3z8HzDGGBet31NI1t4jTB3dM6D7ceVOalW9X1Uv9z7+QlVj67ynqnqXqnb0bncFavyhrikn9yA0RHgtc3egd2WMMSfkH8t3ERkWwkXDugd0PzbVhldyfBRn9k/kjRW7qaqpdTuOMcb4VFZZw8LVe5k0pAsJ0eEB3ZcViDouHZVCQXEFn2zc73YUY4zx6b11+ygqr+aSUYE9vQRWIL7lzP6JJMdH8srXdk+EMSY4vfL1Lnp1jmFseqeA78sKRB1hoSFcktGTzzbns+dwmdtxjDHmW7bkFZGZfYipo3rixBymViDquSTD022zS16NMcHmla9zCA8VJp/cw5H9WYGop2fHaE7rm8jrmTlU22C1MSZIlFfVsGDVbs4e3IXOsZGO7NMKhA/TRvdkX2E5n21u2gSAxhjT3D7MyuVwaRXTAnjndH1WIHyYODCZzrGRvPK1nWYyxgSHV77eRUrHaMb1Dvzg9FFWIHwIDw3hkowefLIxj9zCcrfjGGPauO35xXy1/SBTR/ckJMS5BTatQDRg6qgUatUGq40x7nvl612EhQhTHBqcPsoKRANSOkVzWr9E/rF8lw1WG2NcU15Vw+srdnPO4C4kxUU5um8rEMcw/ZQU9hWW8+kmG6w2xrjj/fX7OFxaxWWnODc4fZQViGOYOCCJ5PhIXlqW7XYUY0wb9dJXzt05XZ8ViGMICw3h0lEpfLY5n5yDpW7HMca0MRtzj5CZfYjLRqc4Ojh9lBWI45g6qicCNj+TMcZxLy/bRURoiGN3TtfnWIEQkRdFZJ+IHBGRzSJyXQPtrhaRGhEprrOd4VTO+rq1b8dZA5J5LXM3ldU2WG2McUZpZTVvrtzDeSd1oWNMhCsZnOxB/D8gTVXjgQuBh0Tk5AbaLlXV2DrbIsdS+jB9jGca8A+zct2MYYxpQ/61ei9FFdVcdkqqaxkcKxCqmqWqFUeferfeTu3/RJzWN5GeHdvx4lc2WG2MCTxVZd7SbPonxzEqrYNrORwdgxCRWSJSCmwE9gHvNdB0hIgUeE9F3SsiYQ183kwRyRSRzPz8wF2KGhoiTD8llWU7DrI5ryhg+zHGGIBVOYfZsO8Il49NdWRa74Y4WiBU9WYgDjgVWABU+Gj2OTAESAImA9OAOxv4vNmqmqGqGYmJiYEJ7XVJRk8iwkKsF2GMCbgXl2YTGxnGxSMCu+b08Th+FZOq1qjqYqAHcJOP97er6g5VrVXVdcADwBSnc9bXMSaCH5zUlQUr91BcUe12HGNMK3WwpJJ31u7jRyO7Exvp8+SJY9y8zDUM/8YgFHCvj1XH5WNTKa6o5q1Ve9yOYoxppV7LzKGyppbLx7g3OH2UIwVCRJJEZKqIxIpIqIicg+fU0Sc+2k4SkWTv4wHAvcBCJ3Iez4ie7RncLZ4Xv8pGVd2OY4xpZWpqlZeWZXNKr470S45zO45jPQjFczppN3AI+CPwM1VdKCIp3nsdjk40MhFYKyIleAaxFwCPOJTzmESEK8aksjHXsy6sMcY0p88355NzsIwrxrrfewDPaZ6AU9V84PQG3tsFxNZ5fgdwhxO5muLC4d145L1veOHLnYxK6+h2HGNMK/LC0p0kxkVy9qAubkcBbKqNRouOCOOSjJ58sD6XvCO2mJAxpnnsLChh0aZ8pp+SQkRYcHw1B0eKFuaKsanUqPLSMpufyRjTPOYtzSY8VFyZ1rshViCaILVTDGf2T+LlZbtsfiZjzAkrqajm9RU5TBrS1fFFgY7FCkQTXTk2lYLiCt5fv8/tKMaYFu7NVXsoKq/mqnHBMTh9lBWIJjqtbyK9Osfwwpc73Y5ijGnBPPMu7WRI93hGprg375IvViCaKCTEc8nryl2HWbe70O04xpgWaun2A2zOK+bKsWmuzrvkixWIEzAlowfREaE8/+UOt6MYY1qouUt20iE6nAuHdXM7yndYgTgB8VHhTDm5B++s2Ud+ka95B40xpmE5B0v5+Js8po1OISo81O0432EF4gRdNS6NyppaXrZLXo0xjTRv6U7PDA1Bcud0fVYgTlDvxFjO6J/Ii8uy7ZJXY4zfSiqq+cfyHCYN6ULXhHZux/HJCkQzuHpcGvlFFby7bq/bUYwxLcSClbspKq/mmvFpbkdpkBWIZnBa30TSE2N4fslOm+XVGHNctbXK81/uZGiPhKC7tLUuKxDNICREuGZcGmt3F7Jyl83yaow5ti+2FrA9v4Rrxgffpa11WYFoJj8a2YP4qDCeW7LT7SjGmCD33OIdJMZFcv5JwXdpa11WIJpJTGQY00an8P66feQcLHU7jjEmSG3JK+KzzflcOSY1aGZtbYhj6UTkRRHZJyJHRGSziFx3jLY/F5FcESkUkedEJNKpnCfiqnGe7qJNv2GMachzS3YQGRbC9CBYUvR4nCxf/w9IU9V44ELgIRE5uX4j73Kkv8KzslwakA78zsGcTdatfTvOO6krry7Poai8yu04xpggc6C4gn+u3MOPRvagY0yE23GOy7ECoapZqnr0dmP1br19NL0KmONtfwh4ELjamZQnbsaEXhRVVPNa5m63oxhjgsxL3iUCZkxIczuKXxw9ASYis0SkFNgI7MOz5nR9g4E1dZ6vAZJFpJOPz5spIpkikpmfnx+QzI01vGd7MlI78PySHdTU2iWvxhiPiuoa5i3N5oz+ifRJinM7jl8cLRCqejMQB5wKLAB8TWAUC9SdHvXo4+8cUVWdraoZqpqRmJjY3HGbbMaEXuw+VMZHWbluRzHGBIl/rd5LQXEFMyb0cjuK3xwfQlfVGlVdDPQAbvLRpBiIr/P86OOiQGdrLmcP7kLPju14drHN8mqM8az5MGfxDvonxzGhT2e34/jNzWuswvA9BpEFDKvzfBiQp6oHHEnVDEJDhBnje7Ei+xArsg+6HccY47LPtxSwMbeI609LD+ob4+pzpECISJKITBWRWBEJ9V6
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plot_function(f, 'x', 'x**2')\n",
"plt.scatter(-1.5, f(-1.5), color='red');"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Calculating Gradients"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"xt = tensor(3.).requires_grad_()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(9., grad_fn=<PowBackward0>)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"yt = f(xt)\n",
"yt"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"yt.backward()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(6.)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"xt.grad"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([ 3., 4., 10.], requires_grad=True)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"xt = tensor([3.,4.,10.]).requires_grad_()\n",
"xt"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(125., grad_fn=<SumBackward0>)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def f(x): return (x**2).sum()\n",
"\n",
"yt = f(xt)\n",
"yt"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([ 6., 8., 20.])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"yt.backward()\n",
"xt.grad"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Stepping With a Learning Rate"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### An End-to-End SGD Example"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19.])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"time = torch.arange(0,20).float(); time"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAWy0lEQVR4nO3dfYxcV3nH8e8vtpWsbC+u48XFW9luDLGpExw3i4KIAkhJa0FLcWMkTFIISMhAlKqo1IK0OLh5UQBD/yiEF0sp5LUNhrUhpGA1SlJICikbXMdaYVt1UkPWKawhXrz2OjHu0z/mTjKezM7c8cydlzu/jzSS59wzd56czD5z5txzz1FEYGZm3e2sdgdgZmaNczI3M8sBJ3MzsxxwMjczywEnczOzHJjZjjddsGBBLF26tB1vbWbWtZ544onDETFQ6VhbkvnSpUsZGRlpx1ubmXUtSQenO+ZhFjOzHHAyNzPLASdzM7MccDI3M8sBJ3Mzsxxoy2yWM7Vj1xhbdu7j0JEpFs3rY+Oa5axdPdjusMzM2q5rkvmOXWNcP7yHqZOnABg7MsX1w3sAnNDNrOd1zTDLlp37XkzkRVMnT7Fl5742RWRm1jm6JpkfOjJVV7mZWS/pmmS+aF5fXeVmZr2ka5L5xjXL6Zs147Syvlkz2LhmeZsiMjPrHF1zAbR4kdOzWczMXq5rkjkUErqTt5nZy3XNMIuZmU3PydzMLAeczM3McsDJ3MwsB2omc0mTZY9Tkj5fcvxySXslHZf0sKQl2YZsZmblaibziJhTfAALgSlgG4CkBcAwsAmYD4wA92UXrpmZVVLvMMs7gV8CP0ieXwmMRsS2iDgBbAZWSVrRvBDNzKyWepP5NcCdERHJ85XA7uLBiDgGHEjKTyNpg6QRSSPj4+NnGq+ZmVWQOplLWgy8GbijpHgOMFFWdQKYW/76iNgaEUMRMTQwMHAmsZqZ2TTq6Zm/F3g0Ip4uKZsE+svq9QNHGw3MzMzSqzeZ31FWNgqsKj6RNBtYlpSbmVmLpErmkt4IDJLMYimxHbhA0jpJ5wA3AE9GxN7mhmlmZtWkXWjrGmA4Ik4bPomIcUnrgC8AdwOPA+ubG6KZWffLeg/jVMk8Ij5Y5diDgKcimplNoxV7GPt2fjOzjLViD2MnczOzjLViD2MnczOzjLViD2MnczOzjLViD+Ou2jbOzKwbtWIPYydzM7MWyHoPYw+zmJnlgJO5mVkOOJmbmeWAk7mZWQ44mZuZ5YCTuZlZDjiZm5nlgJO5mVkOOJmbmeWAk7mZWQ6kTuaS1kv6qaRjkg5Iuiwpv1zSXknHJT0saUl24ZqZWSWp1maR9EfAp4F3Af8JvCopXwAMAx8A7gduAu4D3pBFsI3KetsmM7N2SbvQ1t8DN0bEj5LnYwCSNgCjEbEteb4ZOCxpRadt6tyKbZvMzNql5jCLpBnAEDAg6b8lPSPpC5L6gJXA7mLdiDgGHEjKy8+zQdKIpJHx8fHm/Rek1Iptm8zM2iXNmPlCYBbwTuAy4CJgNfAJYA4wUVZ/AphbfpKI2BoRQxExNDAw0FDQZ6IV2zaZmbVLmmRezHafj4hnI+Iw8A/A24BJoL+sfj9wtHkhNkcrtm0yM2uXmsk8Ip4DngGiwuFRYFXxiaTZwLKkvKO0YtsmM7N2SXsB9KvAX0r6HnAS+AjwHWA7sEXSOuAB4AbgyU67+Amt2bbJzPKr02fDpU3mNwELgP3ACeDrwC0RcSJJ5F8A7gYeB9ZnEWgzZL1tk5nlUzfMhkuVzCPiJHBt8ig/9iCwoslxmZl1jGqz4Tolmft2fjOzGrphNpyTuZlZDd0wG87J3Myshm6YDZf2AqiZWc/qhtlwTuZmZil0+mw4D7OYmeWAk7mZWQ44mZuZ5YCTuZlZDjiZm5nlgJO5mVkOOJmbmeWAk7mZWQ44mZuZ5YCTuZlZDjiZm5nlQKpkLukRSSckTSaPfSXHrpJ0UNIxSTskzc8uXDMzq6Senvl1ETEneSwHkLQS+ArwHmAhcBz4YvPDNDOzahpdNfFq4P6I+D6ApE3ATyXNjYijDUdnZmap1NMzv1XSYUmPSXpLUrYS2F2sEBEHgBeA88tfLGmDpBFJI+Pj443EbGZmZdIm848B5wGDwFbgfknLgDnARFndCWBu+QkiYmtEDEXE0MDAQAMhm5lZuVTJPCIej4ijEfF8RNwBPAa8DZgE+suq9wMeYjEza6EznZoYgIBRYFWxUNJ5wNnA/sZDMzOztGpeAJU0D7gE+Hfgt8C7gDcBH0le/0NJlwE/AW4Ehn3x08ystdLMZpkF3AysAE4Be4G1EbEPQNKHgHuAc4EHgfdnE6qZmU2nZjKPiHHg9VWO3wvc28ygzMysPr6d38wsBxq9aain7Ng1xpad+zh0ZIpF8/rYuGY5a1cPtjssMzMn87R27Brj+uE9TJ08BcDYkSmuH94D4IRuZm3nYZaUtuzc92IiL5o6eYotO/dN8wozs9ZxMk/p0JGpusrNzFrJyTylRfP66io3M2slJ/OUNq5ZTt+sGaeV9c2awcY1y9sUkZnZS3wBNKXiRU7PZjGzTuRkXoe1qwedvM2sI3mYxcwsB5zMzcxywMMsZtYT8n4Ht5O5meVeL9zB7WEWM8u9XriD28nczHKvF+7gdjI3s9zrhTu4nczNLPd64Q7uupK5pNdIOiHp7pKyqyQdlHRM0g5J85sfppnZmVu7epBbr7yQwXl9CBic18etV16Ym4ufUP9sltuAHxefSFoJfAX4EwobOm8Fvgisb1aAZmbNkPc7uFMnc0nrgSPAfwCvToqvBu6PiO8ndTYBP5U0NyKONjtYMzOrLNUwi6R+4Ebgo2WHVgK7i08i4gDwAnB+hXNskDQiaWR8fPzMIzYzs5dJO2Z+E3B7RPy8rHwOMFFWNgHMLT9BRGyNiKGIGBoYGKg/UjMzm1bNYRZJFwFXAKsrHJ4E+svK+gEPsZiZtVCaMfO3AEuBn0mCQm98hqQ/AL4HrCpWlHQecDawv9mBmpnZ9NIk863Av5Q8/xsKyf3DwCuBH0q6jMJslhuBYV/8NDNrrZrJPCKOA8eLzyVNAiciYhwYl/Qh4B7gXOBB4P0ZxWpmZtOoe9XEiNhc9vxe4N5mBWRmZvXz7fxmZjngZG5mlgNO5mZmOeBkbmaWA07mZmY54GRuZpYDTuZmZjngZG5mlgNO5mZmOeBkbmaWA07mZmY5UPfaLGZm7bBj1xhbdu7j0JEpFs3rY+Oa5bne07NeTuZm1vF27Brj+uE9TJ08BcDYkSmuH94D4ISe8DCLmXW8LTv3vZjIi6ZOnmLLzn1tiqjzOJmbWcc7dGSqrvJe5GRuZh1v0by+usp7UapkLuluSc9K+o2k/ZI+UHLsckl7JR2X9LCkJdmFa2a9aOOa5fTNmnFaWd+sGWxcs7xNEXWetD3zW4GlEdEP/Blws6SLJS0AhoFNwHxgBLgvk0jNrGetXT3IrVdeyOC8PgQMzuvj1isv9MXPEqlms0TEaOnT5LEMuBgYjYhtAJI2A4clrYiIvU2O1cx62NrVg07eVaQeM5f0RUnHgb3As8C/AiuB3cU6EXEMOJCUl79+g6QRSSPj4+MNB25mZi9Jncwj4lpgLnAZhaGV54E5wERZ1YmkXvnrt0bEUEQMDQwMnHnEZmb2MnXNZomIUxHxKPB7wIeBSaC/rFo/cLQ54ZmZWRpnOjVxJoUx81FgVbFQ0uyScjMza5GayVzSKyWtlzRH0gxJa4B3Aw8B24ELJK2TdA5wA/CkL36ambVWmp55UBhSeQZ4Dvgs8JGI+FZEjAPrgFuSY5cA6zOK1czMplFzamKSsN9c5fiDwIpmBpVXXvXNzLLiVRNbxKu+Wa9zZyZbXpulRbzqm/WyYmdm7MgUwUudmR27xtodWm44mbeIV32zXubOTPaczFvEq75ZL3NnJntO5i3iVd+sl7kzkz0n8xbxqm/Wy9yZyZ5ns7SQV32zXlX83Hs2S3aczM2sJdyZyZaHWczMcsDJ3MwsB5zMzcxywMnczCwHfAG0i3htCzObjpN5l/BCXWZ
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"speed = torch.randn(20)*3 + 0.75*(time-9.5)**2 + 1\n",
"plt.scatter(time,speed);"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def f(t, params):\n",
" a,b,c = params\n",
" return a*(t**2) + (b*t) + c"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def mse(preds, targets): return ((preds-targets)**2).mean()"
]
},
2020-05-14 12:18:31 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Step 1: Initialize the parameters"
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"params = torch.randn(3).requires_grad_()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"orig_params = params.clone()"
]
},
2020-05-14 12:18:31 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Step 2: Calculate the predictions"
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"preds = f(time, params)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def show_preds(preds, ax=None):\n",
" if ax is None: ax=plt.subplots()[1]\n",
" ax.scatter(time, speed)\n",
" ax.scatter(time, to_np(preds), color='red')\n",
" ax.set_ylim(-300,100)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEACAYAAABLfPrqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAc1ElEQVR4nO3dfbBU9Z3n8fcHsYAAd8B4hyxMAaurMHV1iOtNOZuYxIxuTDKbLUf2D81NHGY2koxFaqusMjErRCtKaeLU/pHZyQOWBHWZncQUWuM86M5UxESzk9pmLExuBHeJXBKfckkQuYD4wHf/OKe1bbvvvX3POd2nuz+vqi66z+/Xp78cDv3t83s6igjMzMxaMavTAZiZWfdx8jAzs5Y5eZiZWcucPMzMrGVOHmZm1jInDzMza5mTh5mZtSzX5CFpg6SKpBOSttWVXSxpj6Rjkh6WtKKmbI6krZJekvS8pGvzjMvMzPKV95XHs8AtwNbajZJOB3YAm4DTgArwnZoqNwFnASuADwGfl/SRnGMzM7OcqIgZ5pJuAX4nItalr9cD6yLivenr+cBB4LyI2CPpGeBPIuJ/peU3A2dFxBW5B2dmZpnNbtPnDAG7qy8i4qikfcCQpBeApbXl6fPLGu0oTUTrAebPn3/+6tWrCwvazKwX7dq162BEDGbZR7uSxwJgvG7bYWBhWlZ9XV/2NhGxBdgCMDw8HJVKJd9Izcx6nKSxrPto12irCWCgbtsAcCQto668WmZmZiXUruQxCqypvkj7PM4ERiPiEPBcbXn6fLRNsZmZWYvyHqo7W9Jc4BTgFElzJc0G7gPOkbQ2Lf8S8ERE7EnfejewUdJiSauBq4FtecZmZmb5yfvKYyNwHLge+GT6fGNEjANrgc3AIeACoHYk1Y3APmAMeAS4PSIezDk2MzPLSSFDddvFHeZmZq2TtCsihrPsw8uTmJlZy5w8zMysZU4eZmbWMicPMzNrWbtmmJfS/Y8/w+0P7eXZF4+zdNE8rrt0FZedt6zTYZmZlV7fJo/7H3+GL+74CcdffR2AZ148zhd3/ATACcTMbAp922x1+0N730gcVcdffZ3bH9rboYjMzLpH3yaPZ1883tJ2MzN7U982Wy1dNI9nGiSKpYvmdSAaM7PGyto327dXHtdduop5p57ylm3zTj2F6y5d1aGIzMzeqto3+8yLxwne7Ju9//FnOh1a/yaPy85bxq2Xn8uyRfMQsGzRPG69/NxSZHQzMyh332zfNltBkkCcLMysrMrcN9vXySOrsrZFmllvKHPfbN82W2VV5rZIMyuX+x9/hvfd9n3+9fV/x/tu+/60vyfK3Dfr5DFDZW6LNLPyyPJDs8x9s262mqEyt0WaWXlM9kNzOkmgrH2zvvKYoWZtjmVoizSz8ujVH5ptTR6Sdkp6WdJE+thbU/YJSWOSjkq6X9Jp7YytVWVuizSz8ujVH5qduPLYEBEL0scqAElDwLeATwFLgGPA1zsQ27SVuS3SzMqjV39olqXPYwR4ICJ+ACBpE/CkpIURcaSzoTVX1rZIMyuP6ndErw3r70TyuFXSbcBe4IaI2AkMAT+qVoiIfZJeAc4GdnUgxsJ5johZ/+jFH5rtTh5fAH4GvAJcATwg6d3AAuBwXd3DwML6HUhaD6wHWL58eaHBFsX3EjGzbtfWPo+I+HFEHImIExFxF/AY8DFgAhioqz4AvK3JKiK2RMRwRAwPDg4WH3QBPEfErLvMdJJfL+t0n0cAAkaBNdWNks4A5gBPdSiuQvXq0D2zXuSWgsbaduUhaZGkSyXNlTRb0gjwAeAhYDvwcUnvlzQf+DKwo8yd5Vn06tA9s17kloLG2tlsdSpwCzAOHAQ+B1wWEXsjYhT4LEkS+RVJX8c1bYytrXp16J5ZL3JLQWNta7aKiHHgPZOU/xXwV+2Kp5N6deieWS8q88q2ndTpPo++1YtD98zKKsvQ+OsuXfWWPg9wSwE4eZhZj8va4e2WgsacPMysp2Vd1RbcUtCIk0eX8gx16zczPefd4V0MJ48u5HHn1m+ynPPu8C6G7+fRhTzu3PpNlnPeQ+OL4SuPLpTHZbibvaybZDnn3eFdDCePLpT1MjyPZi8nH2unrOe8O7zz52arLpT1Mjxrs1c1+Tzz4nGCN5OPF4uzorjpqXx85dGFsl6GZ232ymPoo/WfLFerbnoqHyePLpXlMjxrE4CHPlqr8mgqddNTubjZqg9lbQLIuiqw743QfzxCsPf4yqMPZW0CyLLWjzvru1eW4+6r1d7j5NGnsjQBZEk+WftLPEGyM7Ied0/U6z1OHjYjM00+7qzvTlmPu1em7T1OHtZW7qzvnE42O3m0VO9x8rC2yvoLNI/mj27uM5lp7GVodvJoqd7S36Ottm+HlSth1qzkz+3bOx1Rz7vsvGXcevm5LFs0DwHLFs3j1svPbamzPstIsW6e4Jgl9qyjnTxJz+qV5spD0mnAncCHSe5x/sX01rTF2L4d1q+HY8eS12NjyWuAkZHp7+OGG+DAAVi+HDZvnv57+1inOuur78vaYZ/lqiXL+7PE7mYny1tpkgfwl8ArwBLg3cDfSdodEaOFfNoNN7yZOKqOHUu2TycBZE0+TjwzliX5ZPkSzdr0k/X9WWJ3s5PlrRTNVpLmA2uBTRExERGPAn8DfKqwDz1woLXt9SZLPlOpJp6xMYh4M/G42axwWSY4Zm36yfr+LLG72cnyVorkAZwNvB4RT9Vs2w0M1VeUtF5SRVJlfHx85p+4fHlr2+tlST5ZEo9lkuVLNGvTT9b3Z4k9a1+TWb2yJI8FwOG6bYeBhfUVI2JLRAxHxPDg4ODMP3HzZnjHO9667R3vSLZPR5bkk/WqB9zZP0NZvkSzLsuS9f1ZE8Bl5y3jsev/gKdv+0Meu/4PnDgsk7L0eUwAA3XbBoAjhX1itX9hpv0Omze/tc8Dpp98li9PmqoabZ+OPDr7+9hM2+6zDjPOY6Kc+x2sLMpy5fEUMFvSWTXb1gDFdJZXjYzA/v1w8mTyZytfvCMjsGULrFgBUvLnli3T20fWqx43e3VEHr/83XRkvUIR0ekYAJD010AAnyYZbfX3wHsnG201PDwclUqlTRHmLMtoq1mzko72elKSCIv+fDPrapJ2RcRwln2UpdkK4BpgK/Ar4NfAnxU2TLcMRkZm/mXtZi8z67CyNFsREb+JiMsiYn5ELC90gmC3c7OXmXVYaZKHtSBLfwvkM9rLzPqak0e3ytLZn3WOC3iosFmfc/LoR1mbvTxD3qzvOXn0o6zNXu4zMet7pRmqOxNdPVS3m+UxVNjMOiaPobq+8rDW5dFnYmZdzcnDWpdHn4k72826mpOHtS5Ln4k72816gvs8rL1Wrmw8O37FimTIsZkVzn0e1n08QdGsJzh5WHu5s92sJzh5WHtl7Ww3s1Jw8rD2yjpBETxay6wEyrQku/WLLMvRezl5s1LwlYd1Fy+NYlYKTh7WXTxay6wUnDysu3i0llkptCV5SNop6WVJE+ljb135JySNSToq6X5Jp7UjLutCHq1lVgrtvPLYEBEL0seq6kZJQ8C3gE8BS4BjwNfbGJd1kzxGa5lZZmVothoBHoiIH0TEBLAJuFzSwg7HZWWV5S6K4KG+ZjloZ/K4VdJBSY9Juqhm+xCwu/oiIvYBrwBnN9qJpPWSKpIq4+PjhQZsPcgLM5rlol3J4wvAGcAyYAvwgKQz07IFwOG6+oeBhlceEbElIoYjYnhwcLCoeK1XeaivWS4yJ4+0MzyaPB4FiIgfR8SRiDgREXcBjwEfS3cxAQzU7XYAOJI1NrO38VBfs1xknmEeERfN5G2A0uejwJpqgaQzgDnAU1ljM3ub5csbLwnvob5mLSm82UrSIkmXSporabakEeADwENple3AxyW9X9J84MvAjojwlYflz0N9zXLRjj6PU4FbgHHgIPA54LKI2AsQEaPAZ0mSyK9I+jquaUNc1o881NcsF76ToFkrtm9POtcPHEiaujZvduKxrpPHnQS9qq7ZdHlFX7M3lGGSoFl38DBfszc4eZhNl4f5mr3BycNsury
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"show_preds(preds)"
]
},
2020-05-14 12:18:31 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Step 3: Calculate the loss"
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(25823.8086, grad_fn=<MeanBackward0>)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loss = mse(preds, speed)\n",
"loss"
]
},
2020-05-14 12:18:31 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Step 4: Calculate the gradients"
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([-53195.8594, -3419.7146, -253.8908])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loss.backward()\n",
"params.grad"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([-0.5320, -0.0342, -0.0025])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"params.grad * 1e-5"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([-0.7658, -0.7506, 1.3525], requires_grad=True)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"params"
]
},
2020-05-14 12:18:31 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Step 5: Step the weights. "
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"lr = 1e-5\n",
"params.data -= lr * params.grad.data\n",
"params.grad = None"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(5435.5366, grad_fn=<MeanBackward0>)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"preds = f(time,params)\n",
"mse(preds, speed)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEACAYAAABLfPrqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAcYElEQVR4nO3df4wc5Z3n8ffHGGFje9YmzJKzVx4vHNgrwzqcJ+IO8sMJuZCwx4nF9wdkEta3G5wscnQSJxJy2AEFkElY3R/Zu/wwgpgfXjYhMmhJduGyG0yCcxvdEGSSCTaSA0P4mXFijMc25tf3/qjq0HS6Z6amqrpruj8vqTTd9TxV/Z1navrbz/NUVSsiMDMzy2JWpwMwM7OZx8nDzMwyc/IwM7PMnDzMzCwzJw8zM8vMycPMzDJz8jAzs8wKTR6SNkgalnRU0taGsnMl7ZZ0WNKDkgbqyo6TdKuklyW9IOmKIuMyM7NiFd3zeA64Hri1fqWkE4HtwCbgBGAY+FZdlWuBU4EB4APAZyV9pODYzMysICrjCnNJ1wN/FBHr0ufrgXURcXb6fB6wDzgzInZLehb4rxHxf9Ly64BTI+LiwoMzM7PcZrfpdVYCu2pPIuKQpL3ASkkvAovry9PHFzbbUZqI1gPMmzdv9YoVK0oL2sysGz3yyCP7IqI/zz7alTzmA2MN6w4AC9Ky2vPGst8TEVuALQCDg4MxPDxcbKRmZl1O0mjefbTrbKtxoK9hXR9wMC2jobxWZmZmFdSu5DECrKo9Sec8TgFGImI/8Hx9efp4pE2xmZlZRkWfqjtb0hzgGOAYSXMkzQbuAU6XtDYt/wLwWETsTje9HdgoaZGkFcBlwNYiYzMzs+IU3fPYCBwBrgI+nj7eGBFjwFrgBmA/cBZQfybVNcBeYBR4CLgpIu4vODYzMytIKafqtosnzM3MspP0SEQM5tmHb09iZmaZOXmYmVlmTh5mZpaZk4eZmWXWrivMK+neR5/lpgf28NxLR1i8cC5XnrecC89c0umwzMwqr2eTx72PPsvnt/+MI6+9AcCzLx3h89t/BuAEYmY2iZ4dtrrpgT2/Sxw1R157g5se2NOhiMzMZo6eTR7PvXQk03ozM3tLzw5bLV44l2ebJIrFC+d2IBozs+aqOjfbsz2PK89bztxjj3nburnHHsOV5y3vUERmZm9Xm5t99qUjBG/Nzd776LOdDq13k8eFZy5h80VnsGThXAQsWTiXzRedUYmMbmYG1Z6b7dlhK0gSiJOFmVVVledmezp55FXVsUgz6w5Vnpvt2WGrvKo8Fmlm1XLvo89yzo0/4I+v+h7n3PiDKb9PVHlu1sljmqo8Fmlm1ZHng2aV52Y9bDVNVR6LNLPqmOiD5lSSQFXnZt3zmKZWY45VGIs0s+ro1g+abU0eknZIekXSeLrsqSv7mKRRSYck3SvphHbGllWVxyLNrDq69YNmJ3oeGyJifrosB5C0EvgG8AngJOAw8NUOxDZlVR6LNLPq6NYPmlWZ8xgC7ouIHwJI2gQ8LmlBRBzsbGitVXUs0syqo/Ye0W2n9XcieWyWdCOwB7g6InYAK4Ef1ypExF5JrwKnAY90IMbS+RoRs97RjR802508Pgf8AngVuBi4T9K7gPnAgYa6B4AFjTuQtB5YD7B06dJSgy2Lv0vEzGa6ts55RMRPIuJgRByNiNuAncD5wDjQ11C9D/i9IauI2BIRgxEx2N/fX37QJfA1ImYzy3Qv8utmnZ7zCEDACLCqtlLSycBxwBMdiqtU3Xrqnlk38khBc23reUhaKOk8SXMkzZY0BLwPeADYBlwg6b2S5gFfBLZXebI8j249dc+sG3mkoLl2DlsdC1wPjAH7gM8AF0bEnogYAT5NkkR+TTLXcXkbY2urbj11z6wbeaSgubYNW0XEGPDuCcr/Dvi7dsXTSd166p5ZN6rynW07qdNzHj2rG0/dM6uqPKfGX3ne8rfNeYBHCsDJw8y6XN4Jb48UNOfkYWZdLe9dbcEjBc04ecxQvkLdes10j3lPeJfDyWMG8nnn1mvyHPOe8C6Hv89jBvJ559Zr8hzzPjW+HO55zEBFdMM97GUzSZ5j3hPe5XDymIHydsOLGPZy8rF2ynvMe8K7eB62moHydsPzDnvVks+zLx0heCv5+GZxVhYPPVWPex4zUN5ueN5hryJOfbTek6e36qGn6nHymKHydMPzDgH41EfLqoihUg89VYuHrXpQ3iGAvHcF9ncj9B6fIdh93PPoQXmHAPLc68eT9TNXnnZ3b7X7OHn0qDxDAHmST975El8g2Rl5290X6nUfJw+blukmH0/Wz0x52913pu0+Th7WVp6s75xODjv5bKnu4+RhbZX3E2gRwx8zec5kurFXYdjJZ0t1l94+22rbNli2DGbNSn5u29be7XvQhWcuYfNFZ7Bk4VwELFk4l80XnZFpsj7PmWIz+QLHPLHnPdvJF+lZo8r0PCSdANwCfJjkO84/n341bTm2bYP16+Hw4eT56GjyHGBoqD3bX301PP00LF0KN9wwte26QKcm62vb5Z2wz9NrybN9ntg97GRFU0R0OgYAJN1F0hP6K+BdwPeAsyNipNU2g4ODMTw8PL0XXLYsecNvNDAATz1V7vaNiQfg+ONhy5apJ5AeTj55/PFV36PZES/gyRv/bMJtG4d+IPn0PdWeU97t88R+zo0/aDrstGThXHZe9cFJX9u6i6RHImIwzz4qMWwlaR6wFtgUEeMR8TDwD8AnSnvRp5/Otr7I7a+++u2JA5LnV189tdeuJZ/RUYh4q9fjYbNJ5bnAMe/QT97t88TuYScrWiWSB3Aa8EZEPFG3bhewsrGipPWShiUNj42NTf8Vly7Ntr7I7fMmrrzJB3p2vibPm2jeoZ+82+eJPe9ck1mjqiSP+cCBhnUHgAWNFSNiS0QMRsRgf3//9F/xhhuSoaJ6xx+frC97+7yJK2/yKaLnMkOTT5430by3Zcm7fd4EcOGZS9h51Qd58sY/Y+dVH3TisFwqMech6UxgZ0QcX7fuvwNrIuKCVtvlmvOA/PMG090+75xHJ+droJg5mxmo03MeZkUpYs6DiOj4AswDXgVOrVt3O3DjRNutXr06Zqw774wYGIiQkp933plt2+OPj0j6Dcly/PFT34f09m1rizS17QcGmm8/MDD1+Kf7u3fYPT99Js7e/C+x7HPfjbM3/0vc89Nn2rq9WRGA4cj5vl2JngeApL8HAvgkydlW/0iZZ1vNdHl6TXl7HrNmJemikQRvvjnxtj3aazGrkq452yp1OTAX+DVwF/DXEyWOnjc0lLzRv/lm8jPLG2/e+Z48czae7DfrCpVJHhHx24i4MCLmRcTSKPMCwV43NJR80h8YSHoLAwPZPvnnST5VmOw3s9wqkzyszfL0XPIkn7xnmrnnYlYJTh42PdNNPnmHzNxzMasEJw9rr7xDZlXouZiZk4d1QCcn+4vouXjIy8zJw2aYTvZcPORl9juVuc5jOnr6Og+bnjzXmeS9PsasIrrtOg+z8uXpueQd8gIPe1nXqMyXQZm1zdDQ9K5mX7q0ec9jqpP1eb9AzKxC3PMwm6q8k/U+08u6iJOH2VTlnaz3sJd1EQ9bmWUx3SEv8LCXdRX3PMzaxcNe1kWcPMzapdPDXh7ysgJ52MqsnTo17OUhLyuYex5mM0WeYS8PeVnBnDzMZopOX+BoVsfJw2wmme5NJfPejRg8Z2Jv05bkIWmHpFckjafLnobyj0kalXRI0r2STmhHXGY9I++ZXr4ppDVoZ89jQ0TMT5fltZWSVgLfAD4BnAQcBr7axrjMul/eM708Z2INqjBsNQTcFxE/jIhxYBNwkaQFHY7LrLvk+R4VXx1vDdqZPDZL2idpp6Q1detXArtqTyJiL/AqcFqznUhaL2lY0vDY2FipAZtZKu+ciYe9uk67ksfngJOBJcAW4D5Jp6Rl84EDDfUPAE17HhGxJSIGI2Kwv7+/rHjNrJ6vjrcGuZNHOhkeLZaHASLiJxFxMCKORsRtwE7g/HQX40Bfw277gIN5YzOzgnT66nirnNzJIyL
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"show_preds(preds)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def apply_step(params, prn=True):\n",
" preds = f(time, params)\n",
" loss = mse(preds, speed)\n",
" loss.backward()\n",
" params.data -= lr * params.grad.data\n",
" params.grad = None\n",
" if prn: print(loss.item())\n",
" return preds"
]
},
2020-05-14 12:18:31 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Step 6: Repeat the process "
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"5435.53662109375\n",
"1577.4495849609375\n",
"847.3780517578125\n",
"709.22265625\n",
"683.0757446289062\n",
"678.12451171875\n",
"677.1839599609375\n",
"677.0025024414062\n",
"676.96435546875\n",
"676.9537353515625\n"
]
}
],
"source": [
"for i in range(10): apply_step(params)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"params = orig_params.detach().requires_grad_()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA1QAAADMCAYAAAB0vOLuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df4zc9Z3f8dfbLIe9BsemGFJMdn1CHD4Z5CA2ihroBUJVuOS4InxSaCbiSARbtUcvVw6DqY0hgIuJo2suIuW0BMQB2wuXE6Y6LgdVjx8VrlplCYHUxERK8RpsI0xrG/Can/vuH98ZdnY83+/MfOczM98fz4c0Ws/3M7P+7tf78nfe38/n+/mYuwsAAAAA0LkFg94BAAAAAMgrCioAAAAASImCCgAAAABSoqACAAAAgJQoqAAAAAAgJQoqAAAAAEiJggoAAAAAUgpaUJnZtWY2ZWbvm9kDDW0XmdlOM5sxs6fNbLSu7Tgzu9/M3jazN8zsupD7BeQReQLCIlNAWGQKiITuodor6Q5J99dvNLOTJD0q6WZJJ0qakvRI3UtulXSGpFFJF0q6wcwuCbxvQN6QJyAsMgWERaYASebu4b+p2R2STnP3q6rPxyVd5e5fqD5fLOktSee4+04z2yPpG+7+X6vtt0s6w92vCL5zQM6QJyAsMgWERaZQdv26h2q1pBdrT9z9sKRfS1ptZssknVrfXv3z6j7tG5A35AkIi0wBYZEplMpQn/6e4yXtb9h2SNIJ1bba88a2o1SveoxL0uLFi89dtWpV2D0Fmnj++effcvflg96PqmB5ksgUBqOomSJPGBQyBYTVSab6VVC9K2lJw7Ylkt6pttWev9fQdhR3n5A0IUljY2M+NTUVfGeBRmY2Peh9qBMsTxKZwmAUNVPkCYNCpoCwOslUv4b87ZC0pvakOpb2dEk73P2ApH317dU/7+jTvgF5Q56AsMgUEBaZQqmEnjZ9yMwWSjpG0jFmttDMhiRtk3SWma2ttm+S9JK776y+9UFJG81smZmtknSNpAdC7huQN+QJCItMAWGRKSASuodqo6QjktZL+nr1zxvdfb+ktZI2Szog6fOS6mdyuUXRzYrTkp6VtNXdnwi8b0DekCcgLDIFhEWmAPVo2vR+YSwt+sXMnnf3sUHvR6+RKfRLGTJFntBPZAoIq5NM9eseKgAAAAAonH7N8jcQj72wR1uffEV7Dx7RqUsXad3FZ+qyc1YMereA3CJTQDjkCQiLTGFQCltQPfbCHt306C905MOPJUl7Dh7RTY/+QpIIF5ACmQLCIU9AWGQKg1TYIX9bn3zlk1DVHPnwY2198pUB7RGQb2QKCIc8AWGRKQxSYXuo9h480tF2lAvDAjpHppCETHWGPKEVMtUZMoUkvc5TYXuoTl26qKPtKI/asIA9B4/INTcs4LEX9gx61zKNTCEOmeoceUISMtU5MoU4/chTYQuqdRefqUXHHjNv26Jjj9G6i88c0B4hKxgWkA6ZQhwy1TnyhCRkqnNkCnH6kafCDvmrdePFde/RlV5eDAtIh0whDpnqXKs8SWSqzMhU5zhHIU4/8lTYgkqKwtUsLMwEUw5x/3meunSR9jQJEcMCWiNT5UamworLk0SmyoJMhcU5qtwGmafCDvlLQld68SWNl2VYQHhkqvjIVH+RqeIjU/1Dnopv0HkqZUFFV3rxJf3nedk5K3Tn5WdrxdJFMkkrli7SnZefzVWqLpCp4iNT/UWmio9M9Q95Kr5B56nQQ/7i0JVefK3+80waaoPOkaniI1P9RaaKj0z1D3kqvkHnqZQ9VHSlFx/Tp/YXmSo+MtVfZKr4yFT/kKfiG3SeSllQter6e+yFPTpvy1P6zfV/p/O2PMW6DznEf579RaaKj0z1F5kqPjLVP+Sp+Aadp1IO+ZOYCabo2pmSGGGRqWIjU/1HpoqNTPUXeSq2QeeptAVVnFY3tSFbktaVYPx5NpCpfCFT2Uem8qPV2kdkavDIU75k9RxFQdWAmWDyg6tK+UCm8oNM5QOZygfylA/kKT+ynKlS3kOVZNA3taF9rCuRD2QqP8hUPpCpfCBP+UCe8iPLmaKgajDom9rQPq4q5QOZyg8ylQ9kKh/IUz6Qp/zIcqYY8tdg0De14Whx42VZVyIfyFT2kKl8I1PZQp7yjTxlTx4zRUHVBDeJZkfSeNl1F585r03iqlJWkansIFPFQKaygTwVA3nKjrxmioKqQ61m7EE6ccc1abzs9vVfksRVpbwjU+ElHVMyVXxkKjzOUeVGpsIrWqYoqDqQ5dlF8izpuLYaL8tVpXwjU+G1OqZkqtjIVHico8qNTIU3kExNTkobNki7d0sjI9LmzVKlku4HaIJJKTrQanYRVtpOJ+m4MvtOsZGp8FodUzJVbGQqPM5R5UamwutZpiYnpZUrpQULoq+Tk3Pbx8el6WnJPfo6Pj7XHgAFVQeSquZatb3n4BG55qrt+mARuuaSjiuz7xRbN5kiT821urpHpoqNTIXHOarcyFR4PclUUtG0YYM0MzP/9TMz0fZAKKg6kFQ1t3MFo1XBVVZJx/Wyc1bozsvP1oqli2SSVixdpDsvP5tu9oJImynyFK/V1T0yVWxkKjzOUeVGpsLrKlNxvVBJRdPu3c13JG57ChRUHUiqmltdFc7yYmT9EnelptXViMvOWaHt67+kV7d8RdvXf4kTVYGkzRR5Sp8niUwVGZlKj3MUmiFT6aXO1MvPaPtffFOvfudSbf+Lb+qyl5+JXpTUC5VUNI2MNG+L254Ck1J0IGmtgq1PvpI4N36rgqvoM8i0c1NnkX9+NJc2U+SJPKE5MpUOmUIcMpVOq0yt+Mk2febu23Xywf16c+lyvXb9zfrcOZfMFU213qZa0SQl90KNjESvbVSbgKL+e0rS8HC0PRBz92DfrN/GxsZ8ampq0Lsh6ehfHCmqtmvdlOdteapp6FZUQ5T03ryELmk/k37+2jSYWWZmz7v72KD3o9fykqm4k1g7eap97zxnKu95ksqRqSzlSepdpvKSJ4lM5R2Zyp6fbr5bn/luQ2G04Vqdt+Upnbv9J7rhvz+oU99+S3uXnKTv/M6Vev68L2v7Z/Y1L3AmJqLiqFlhNDoa9TY1q1vMpIceiv+elUqqWf46yRQ9VIG0uoKVtBhZq27hPEzX2e1UzUCjVplKk6faiSrvmSJPSKMXmWp8X1bzJJEphFeaTMUUIz/dfLfO+vb1WvTh+5KkTx98U5/69vX6qaSx/7lLdz5xt4Y/itpOe3u/tjxxt26SpF/8Tbr7nZJ6oWrFUVzRVKkEnSa9ET1UfRR3xeE31/+dmv0rmKJu46SrZv2+ipH26l7er/6V4cqflK9MpcnTq1u+0vJ3MQ+ZkpTrPEnlyFSe8iQV4xyV9HOQqfwjU11kKqmHpkXbR1dfo6H35vbpo4WLNPTDe/XGtdfp0wffPOqvemPpyZIU2/bpQ/vje5riiqbR0fihe7VeqMBy2UNlZidKuk/SP5f0lqSb3P0/d/VNe7yIV6fiFiOLC0+rMbi9utIeF9Zuru4l9dChN4qeqTR5ktqbAjfrmfqPX/0seRqA4JnKUJ6k/JyjJDJVBEU/R0npM/X7O54+aqjc366+UI+9sEfP3fbneuSpBz5p+97/ukra9K3o74n7+ZPuS5LmF0zT09FzSapUNLPuRg2/N39fh947opl1N+rkg/ub/twnH9wvs+bH5JRD+9Pf79SqF2qAsjTL3w8kfSDpFEkVSfeY2erU363VIl5x0y4OQNJsJ91O1R63/kFcW9I0n90sxMbUsgORnUz1MW+tZg8qQqbI08CEy1Q7C02mzVTgvPXqHCWRqZLLzzkqcBbXXXym/uCVZ/XcPd/Q/7nrUj13zzf0B688q3UXn6k/fHW7tjxxt057e78WyD8ZKveHr27Xz7f8QLc9/v15bbc9/n39fMsPPulJqv/5P7r6mpbrMM2su3Fe75M0VzBJ0sJ9e5se7oX79urNpcubtr25dLksZgY9qxVBw8PzG+qLpomJqEfKLPpa3wNVqUi7dkmzs9HXDBRTUkaG/JnZYkk
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 864x216 with 4 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"_,axs = plt.subplots(1,4,figsize=(12,3))\n",
"for ax in axs: show_preds(apply_step(params, False), ax)\n",
"plt.tight_layout()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"#### Step 7: stop"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Summarizing Gradient Descent"
2020-03-06 18:19:03 +00:00
]
},
2020-04-15 12:21:02 +00:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": false
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
" -->\n",
"<!-- Title: G Pages: 1 -->\n",
"<svg width=\"591pt\" height=\"78pt\"\n",
" viewBox=\"0.00 0.00 591.49 78.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 74)\">\n",
"<title>G</title>\n",
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-74 587.4867,-74 587.4867,4 -4,4\"/>\n",
"<!-- init -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>init</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"27\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"27\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">init</text>\n",
"</g>\n",
"<!-- predict -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>predict</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"126.0969\" cy=\"-18\" rx=\"35.194\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"126.0969\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">predict</text>\n",
"</g>\n",
"<!-- init->predict -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>init->predict</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M54.0787,-18C62.3227,-18 71.6196,-18 80.7269,-18\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"80.8626,-21.5001 90.8626,-18 80.8625,-14.5001 80.8626,-21.5001\"/>\n",
"</g>\n",
"<!-- loss -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>loss</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"225.1938\" cy=\"-52\" rx=\"27\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"225.1938\" y=\"-48.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">loss</text>\n",
"</g>\n",
"<!-- predict->loss -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>predict->loss</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M155.2932,-28.0172C166.6224,-31.9043 179.6698,-36.3808 191.4018,-40.406\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"190.2859,-43.7234 200.8806,-43.6582 192.5577,-37.1023 190.2859,-43.7234\"/>\n",
"</g>\n",
"<!-- gradient -->\n",
"<g id=\"node4\" class=\"node\">\n",
"<title>gradient</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"361.8403\" cy=\"-52\" rx=\"39.7935\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"361.8403\" y=\"-48.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">gradient</text>\n",
"</g>\n",
"<!-- loss->gradient -->\n",
"<g id=\"edge3\" class=\"edge\">\n",
"<title>loss->gradient</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M252.5178,-52C269.4967,-52 291.836,-52 311.8929,-52\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"312.1329,-55.5001 322.1329,-52 312.1328,-48.5001 312.1329,-55.5001\"/>\n",
"</g>\n",
"<!-- step -->\n",
"<g id=\"node5\" class=\"node\">\n",
"<title>step</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"465.4867\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"465.4867\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">step</text>\n",
"</g>\n",
"<!-- gradient->step -->\n",
"<g id=\"edge4\" class=\"edge\">\n",
"<title>gradient->step</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M394.0665,-41.4286C405.9515,-37.5298 419.4492,-33.1021 431.4862,-29.1535\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"432.7754,-32.4142 441.1862,-25.9715 430.5935,-25.7629 432.7754,-32.4142\"/>\n",
"</g>\n",
"<!-- step->predict -->\n",
"<g id=\"edge6\" class=\"edge\">\n",
"<title>step->predict</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M438.4132,-18C380.3272,-18 243.2155,-18 171.5401,-18\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"171.4571,-14.5001 161.4571,-18 171.4571,-21.5001 171.4571,-14.5001\"/>\n",
"<text text-anchor=\"middle\" x=\"287.1938\" y=\"-21.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">repeat</text>\n",
"</g>\n",
"<!-- stop -->\n",
"<g id=\"node6\" class=\"node\">\n",
"<title>stop</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"556.4867\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"556.4867\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">stop</text>\n",
"</g>\n",
"<!-- step->stop -->\n",
"<g id=\"edge5\" class=\"edge\">\n",
"<title>step->stop</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M492.7897,-18C501.068,-18 510.3085,-18 519.1272,-18\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"519.203,-21.5001 529.203,-18 519.203,-14.5001 519.203,-21.5001\"/>\n",
"</g>\n",
"</g>\n",
"</svg>\n"
],
"text/plain": [
"<graphviz.files.Source at 0x7f10d352e250>"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gv('''\n",
"init->predict->loss->gradient->step->stop\n",
"step->predict[label=repeat]\n",
"''')"
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## The MNIST Loss Function"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
2020-04-15 12:21:02 +00:00
"train_x = torch.cat([stacked_threes, stacked_sevens]).view(-1, 28*28)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(torch.Size([12396, 784]), torch.Size([12396, 1]))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_y = tensor([1]*len(threes) + [0]*len(sevens)).unsqueeze(1)\n",
"train_x.shape,train_y.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(torch.Size([784]), tensor([1]))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dset = list(zip(train_x,train_y))\n",
"x,y = dset[0]\n",
"x.shape,y"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
2020-04-15 12:21:02 +00:00
"valid_x = torch.cat([valid_3_tens, valid_7_tens]).view(-1, 28*28)\n",
"valid_y = tensor([1]*len(valid_3_tens) + [0]*len(valid_7_tens)).unsqueeze(1)\n",
"valid_dset = list(zip(valid_x,valid_y))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def init_params(size, std=1.0): return (torch.randn(size)*std).requires_grad_()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"weights = init_params((28*28,1))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"bias = init_params(1)"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"tensor([20.2336], grad_fn=<AddBackward0>)"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2020-04-15 12:21:02 +00:00
"(train_x[0]*weights.T).sum() + bias"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"tensor([[20.2336],\n",
" [17.0644],\n",
" [15.2384],\n",
" ...,\n",
" [18.3804],\n",
" [23.8567],\n",
" [28.6816]], grad_fn=<AddBackward0>)"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2020-04-15 12:21:02 +00:00
"def linear1(xb): return xb@weights + bias\n",
"preds = linear1(train_x)\n",
"preds"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"tensor([[ True],\n",
" [ True],\n",
" [ True],\n",
" ...,\n",
" [False],\n",
" [False],\n",
" [False]])"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2020-04-15 12:21:02 +00:00
"corrects = (preds>0.0).float() == train_y\n",
"corrects"
2020-03-06 18:19:03 +00:00
]
},
{
2020-04-15 12:21:02 +00:00
"cell_type": "code",
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {},
2020-04-15 12:21:02 +00:00
"outputs": [
{
"data": {
"text/plain": [
"0.4912068545818329"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
2020-03-06 18:19:03 +00:00
"source": [
2020-04-15 12:21:02 +00:00
"corrects.float().mean().item()"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
2020-04-15 12:21:02 +00:00
"weights[0] *= 1.0001"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"0.4912068545818329"
2020-03-06 18:19:03 +00:00
]
},
2020-04-15 12:21:02 +00:00
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
2020-03-06 18:19:03 +00:00
}
],
"source": [
2020-04-15 12:21:02 +00:00
"preds = linear1(train_x)\n",
"((preds>0.0).float() == train_y).float().mean().item()"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
2020-04-15 12:21:02 +00:00
"trgts = tensor([1,0,1])\n",
"prds = tensor([0.9, 0.4, 0.2])"
2020-03-06 18:19:03 +00:00
]
},
{
2020-04-15 12:21:02 +00:00
"cell_type": "code",
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {},
2020-04-15 12:21:02 +00:00
"outputs": [],
2020-03-06 18:19:03 +00:00
"source": [
2020-04-15 12:21:02 +00:00
"def mnist_loss(predictions, targets):\n",
" return torch.where(targets==1, 1-predictions, predictions).mean()"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"tensor([0.1000, 0.4000, 0.8000])"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2020-04-15 12:21:02 +00:00
"torch.where(trgts==1, 1-prds, prds)"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"tensor(0.4333)"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2020-04-15 12:21:02 +00:00
"mnist_loss(prds,trgts)"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"tensor(0.2333)"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2020-04-15 12:21:02 +00:00
"mnist_loss(tensor([0.9, 0.4, 0.8]),trgts)"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-04-15 12:21:02 +00:00
"### Sigmoid"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
2020-04-15 12:21:02 +00:00
"def sigmoid(x): return 1/(1+torch.exp(-x))"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEMCAYAAAA/Jfb8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXyU5b3+8c8XCCQkJGwh7DvIpoBEEBStVetyatViq1VxrxW1bqeeWv1prdra2uOx1VqXUxTFXStuVG3VqrhUWQOEfd9JIGTfk+/vjwk9MSZmgCTPzOR6v17zknnmnuEyzFw83M8z92PujoiIxJY2QQcQEZGmp3IXEYlBKncRkRikchcRiUEqdxGRGKRyFxGJQSp3iTlmdqeZrQs6x35mNsvM3mtkzCVmVtlSmST2qdwlqphZgpndbWZrzazEzPaa2Xwzu67WsP8Gjg4qYz2uB34QdAhpXdoFHUDkAD0CnECoMDOAZGA80H//AHcvBAoDSVcPd88LOoO0Ptpzl2hzFvB7d3/N3Te6e4a7z3L3u/YPqG9axsxuMLNtZlZsZu+a2XQzczPrW/P4JWZWaWYnmNmymn8VfGRmvc3sODNbbGZFZvaemfWp89oXm9kKMyur+T3uMbN2tR7/yrSMhdxtZllmVmhmLwBdmusHJq2Tyl2izU7gVDPrGu4TzOz7hKZqfg+MBZ4HflfP0DbAL4ErgGOA3sCLwF3ADOBYoC/wP7Ve+z+AJ4DZwOHAfwLX1LxOQ64DbgJuBo4EFjUyXuTAubtuukXNjVDpbgaqgKXA48CZgNUacyewrtb9T4HZdV7nt4ADfWvuX1Jzf1ytMTfXbJtQa9uNwJ5a9+cBL9V57euBEqB9zf1ZwHu1Ht8G/LrOc14BKoP++eoWOzftuUtUcfdPgSHAVOApIA34K/CGmVkDTxsF/KvOts/re3lgWa37u2r+u7TOtm5m1rbm/mjg4zqv8xEQX5PzK8wsGegDfFbnoU8ayC5yUFTuEnXcvdLdP3P3+939TEJ73d8Fjvump4Xx0tXuXlX3Oe5eUc/rWD3bqPNYfb/nNz0m0mRU7hILVtb8t0cDj68AJtfZ1lSnSmYCx9fZdhyhaZkNdQd76MyZ7YSml2qre1/kkOhUSIkqZvYRoQOiC4BsYCjwGyAX+GcDT7sfeNHMvgTeBqYAF9U8dqh70PcCb5rZLcCrwDhCc/73u3v5N+S528xWEZou+h5w0iHmEPkK7blLtHkbuAD4G7AaeBJYCxzj7nvqe4K7vwr8F3ALoTn1C4Bf1Txceihh3P1vwGXAxcBy4AHgz7Vevz5/BB6sGbuE0L8q7vqG8SIHzNw19Setj5ndAVzv7t2CziLSHDQtIzHPzOIInX/+N6CI0DdcbwYeDjKXSHPSnrvEvJpvi74FTAA6ARuBpwl901WLdUlMUrmLiMQgHVAVEYlBETHn3r17dx84cGDQMUREosrChQv3uHtqfY9FRLkPHDiQBQsWBB1DRCSqmNnmhh7TtIyISAwKq9zN7FozW1CzXvWsRsbeaGa7zCzPzJ4wsw5NklRERMIW7p77DuAeQutWN8jMTiH0LcATgYHAYL75m3oiItIMwip3d3/V3V8D9jYy9GJgprtnuvs+4G5CK/aJiEgLauo599GErmu5XwaQZmb6ireISAtq6nJPAmpfDHj/rzvVHWhmV9bM4y/Izs5u4hgiIq1bU5d7IaGr0e+3/9cFdQe6++Punu7u6amp9Z6mKSIiB6mpz3PPJHQB4pdq7o8Fdrt7Y3P1IiIxzd3JKSpnV34pWfllZBWUsju/jPH9OzN1WNPv4IZV7jULL7UD2gJtzSye0MV86y669DQwy8yeJXSV+v9H6OLAIiIxrbyymu25JWzbV8y2fSVs31fCjtwStueWsDOvlF35pZRXVn/teTO+NSS4cidU0r+sdf9C4Fdm9gShS5iNcvct7v6Omd1H6Io4CYQuXPzLr72aiEgUqqiqZktOMRuyi9i4p5CNe4rZtKeILTnF7MwrobrWOoxt2xg9k+Pp3Tmecf0606tzPD2TQ7ceyfH06NSB1E4diI9r2/BveAgiYlXI9PR01/IDIhIpqqqdjXsKWbWrgDW7C1m7u4C1WYVs3ltERdX/dWaXjnEM7J7IgK4d6d8tkf5dO9KvSwJ9u3YkrVMH2rVt3kUAzGyhu6fX91hErC0jIhKU0ooqVu8qYNn2PDJ35JG5I5/Vuwooq5lCaWMwoFsiQ3skcfKoNIamJjE4NZHB3ZNI6RgXcPqGqdxFpNVwd7bkFLNw8z4Wb8klY1suK3fm/3tvPCUhjtG9k5l+9ABG9kpmRK9ODElNarapk+akcheRmFVd7azclc8XG3L4cmMOCzbvY09hGQCJ7dtyRN/OXDF1MEf0SWFMnxT6dknAzAJO3TRU7iISUzbtKeKTdXv4dN0ePlu/l7ySCgD6dklg6rDuTBjQhfSBXRjWoxNt28RGkddH5S4iUa20oorP1+/lw9VZfLgmm817iwHonRLPd0alMXlINyYN7kafzgkBJ21ZKncRiTq5xeX8Y8Vu3lu5m4/X7KGkoor4uDZMGdKdy48dxNRhqQzs1jFmplgOhspdRKLCvqJy3sncxd+W7eTz9XuprHZ6pcRzzoS+nDiyB0cP7haVBz6bi8pdRCJWSXkVf1+xizeW7OCjNdlUVjsDunXkx8cN5rQxPTm8T0qr3jv/Jip3EYko7s6iLft4ZeE23srYSUFZJT2T47ns2EF8b2xvRvdOVqGHQeUuIhEhr7iCvy7axnNfbmFdViEJcW05/fBeTJvQh6MHdaNNDJ/Z0hxU7iISqBU78pn12UZeX7KDsspqxvbrzH3TjuD0I3qR1EEVdbD0kxORFldd7by3cjczP9nIFxtziI9rw/eP7MuFR/dndO+UoOPFBJW7iLSYssoqXlu8ncc+3sCG7CL6dE7g1tNHcG56/4hepyUaqdxFpNmVVlTxwpdbePSjDezKL2V072Qe/NF4Th/Ts9lXTmytVO4i0mxKK6p49ostPPrRerILypg4qCu//8ERHDu0u854aWYqdxFpcpVV1byycBt/fH8tO/NKmTKkGw/9aDxHD+4WdLRWQ+UuIk3G3XlvZRb3vr2SDdlFjOvXmft/MJYpQ7sHHa3VUbmLSJNYvj2Pu99awRcbcxicmsjj0ydw8qg0Tb8EROUuIockp6ic37+7mhfmb6FLx/bcfeZozpvYnzgdKA2Uyl1EDkp1tfPcl1v4/burKSyr5NIpg7jh5GEkx+uUxkigcheRA7ZqVz6/eHUZi7fkMnlwN3515miGp3UKOpbUonIXkbCVVlTx4PtrefzjDSQnxPHAuWM5a1wfzatHIJW7iIRl8ZZ93PzKUtZlFXLOhL7cdvpIuiS2DzqWNEDlLiLfqKyyigf+sZbHP15PWnI8T102keOHpwYdSxqhcheRBq3ZXcD1Lyxh5c58zk3vx23fHakDplFC5S4iX+PuPPXZJu59exVJHdrxl4vSOWlUWtCx5ACo3EXkK3KLy/nZy0t5b+VuTjgslfvOGUtqpw5Bx5IDpHIXkX9buHkf1z2/mKyCUm7/7iguO2agzoSJUip3EcHdefLTTfzmbyvp1TmeV66awth+nYOOJYdA5S7SyhWXV/KLV5fx+pIdnDQyjft/OJaUBB00jXYqd5FWbMveYq6cvYDVuwu4+ZTDmHH8EF2IOkaEtbKPmXU1szlmVmRmm83s/AbGdTCzR81st5nlmNmbZtanaSOLSFP4fP1eznz4E3bmlTLr0olcc8JQFXsMCXfZtoeBciANuAB4xMxG1zPuemAycATQG8gFHmqCnCLShJ77YgvTZ35Bt6QOvH7NMfpSUgxqtNzNLBGYBtzu7oXu/gnwBjC9nuGDgHfdfbe7lwIvAPX9JSAiAaiqdu5+awW3zlnGscO68+rVUxjYPTHoWNIMwplzHw5UufuaWtsygOPrGTsT+KOZ7d9rvwB4+5BTisghKymv4oYXF/Nu5m4umTKQ2787iraaholZ4ZR7EpBXZ1seUN/6nmuALcB2oApYBlxb34ua2ZX
2020-03-06 18:19:03 +00:00
"text/plain": [
2020-04-15 12:21:02 +00:00
"<Figure size 432x288 with 1 Axes>"
2020-03-06 18:19:03 +00:00
]
},
2020-04-15 12:21:02 +00:00
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
2020-03-06 18:19:03 +00:00
}
],
"source": [
2020-04-15 12:21:02 +00:00
"plot_function(torch.sigmoid, title='Sigmoid', min=-4, max=4)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def mnist_loss(predictions, targets):\n",
" predictions = predictions.sigmoid()\n",
" return torch.where(targets==1, 1-predictions, predictions).mean()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### SGD and Mini-Batches"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"[tensor([ 3, 12, 8, 10, 2]),\n",
" tensor([ 9, 4, 7, 14, 5]),\n",
" tensor([ 1, 13, 0, 6, 11])]"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2020-04-15 12:21:02 +00:00
"coll = range(15)\n",
"dl = DataLoader(coll, batch_size=5, shuffle=True)\n",
"list(dl)"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"(#26) [(0, 'a'),(1, 'b'),(2, 'c'),(3, 'd'),(4, 'e'),(5, 'f'),(6, 'g'),(7, 'h'),(8, 'i'),(9, 'j')...]"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2020-04-15 12:21:02 +00:00
"ds = L(enumerate(string.ascii_lowercase))\n",
"ds"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
2020-04-15 12:21:02 +00:00
"outputs": [
{
"data": {
"text/plain": [
"[(tensor([17, 18, 10, 22, 8, 14]), ('r', 's', 'k', 'w', 'i', 'o')),\n",
" (tensor([20, 15, 9, 13, 21, 12]), ('u', 'p', 'j', 'n', 'v', 'm')),\n",
" (tensor([ 7, 25, 6, 5, 11, 23]), ('h', 'z', 'g', 'f', 'l', 'x')),\n",
" (tensor([ 1, 3, 0, 24, 19, 16]), ('b', 'd', 'a', 'y', 't', 'q')),\n",
" (tensor([2, 4]), ('c', 'e'))]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
2020-03-06 18:19:03 +00:00
"source": [
2020-04-15 12:21:02 +00:00
"dl = DataLoader(ds, batch_size=6, shuffle=True)\n",
"list(dl)"
2020-03-06 18:19:03 +00:00
]
},
{
2020-04-15 12:21:02 +00:00
"cell_type": "markdown",
2020-03-06 18:19:03 +00:00
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## Putting It All Together"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
2020-04-15 12:21:02 +00:00
"weights = init_params((28*28,1))\n",
2020-03-06 18:19:03 +00:00
"bias = init_params(1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"(torch.Size([256, 784]), torch.Size([256, 1]))"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2020-04-15 12:21:02 +00:00
"dl = DataLoader(dset, batch_size=256)\n",
"xb,yb = first(dl)\n",
"xb.shape,yb.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"valid_dl = DataLoader(valid_dset, batch_size=256)"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([4, 784])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"batch = train_x[:4]\n",
"batch.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"tensor([[-11.1002],\n",
" [ 5.9263],\n",
" [ 9.9627],\n",
" [ -8.1484]], grad_fn=<AddBackward0>)"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"preds = linear1(batch)\n",
"preds"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"tensor(0.5006, grad_fn=<MeanBackward0>)"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loss = mnist_loss(preds, train_y[:4])\n",
"loss"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"(torch.Size([784, 1]), tensor(-0.0001), tensor([-0.0008]))"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loss.backward()\n",
"weights.grad.shape,weights.grad.mean(),bias.grad"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def calc_grad(xb, yb, model):\n",
" preds = model(xb)\n",
" loss = mnist_loss(preds, yb)\n",
" loss.backward()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"(tensor(-0.0002), tensor([-0.0015]))"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"calc_grad(batch, train_y[:4], linear1)\n",
"weights.grad.mean(),bias.grad"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"(tensor(-0.0003), tensor([-0.0023]))"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"calc_grad(batch, train_y[:4], linear1)\n",
"weights.grad.mean(),bias.grad"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"weights.grad.zero_()\n",
"bias.grad.zero_();"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def train_epoch(model, lr, params):\n",
" for xb,yb in dl:\n",
" calc_grad(xb, yb, model)\n",
" for p in params:\n",
" p.data -= p.grad*lr\n",
" p.grad.zero_()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"tensor([[False],\n",
" [ True],\n",
" [ True],\n",
" [False]])"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(preds>0.0).float() == train_y[:4]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def batch_accuracy(xb, yb):\n",
" preds = xb.sigmoid()\n",
" correct = (preds>0.5) == yb\n",
" return correct.float().mean()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"tensor(0.5000)"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"batch_accuracy(linear1(batch), train_y[:4])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def validate_epoch(model):\n",
" accs = [batch_accuracy(model(xb), yb) for xb,yb in valid_dl]\n",
" return round(torch.stack(accs).mean().item(), 4)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"0.5219"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"validate_epoch(linear1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"0.6883"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"lr = 1.\n",
"params = weights,bias\n",
"train_epoch(linear1, lr, params)\n",
"validate_epoch(linear1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2020-04-15 12:21:02 +00:00
"0.8314 0.9017 0.9227 0.9349 0.9438 0.9501 0.9535 0.9564 0.9594 0.9618 0.9613 0.9638 0.9643 0.9652 0.9662 0.9677 0.9687 0.9691 0.9691 0.9696 "
2020-03-06 18:19:03 +00:00
]
}
],
"source": [
"for i in range(20):\n",
" train_epoch(linear1, lr, params)\n",
" print(validate_epoch(linear1), end=' ')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Creating an Optimizer"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"linear_model = nn.Linear(28*28,1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(torch.Size([1, 784]), torch.Size([1]))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"w,b = linear_model.parameters()\n",
"w.shape,b.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class BasicOptim:\n",
" def __init__(self,params,lr): self.params,self.lr = list(params),lr\n",
"\n",
" def step(self, *args, **kwargs):\n",
" for p in self.params: p.data -= p.grad.data * self.lr\n",
"\n",
" def zero_grad(self, *args, **kwargs):\n",
" for p in self.params: p.grad = None"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"opt = BasicOptim(linear_model.parameters(), lr)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def train_epoch(model):\n",
" for xb,yb in dl:\n",
" calc_grad(xb, yb, model)\n",
" opt.step()\n",
" opt.zero_grad()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"0.4157"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"validate_epoch(linear_model)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def train_model(model, epochs):\n",
" for i in range(epochs):\n",
" train_epoch(model)\n",
" print(validate_epoch(model), end=' ')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2020-04-15 12:21:02 +00:00
"0.4932 0.8618 0.8203 0.9102 0.9331 0.9468 0.9555 0.9629 0.9658 0.9673 0.9687 0.9707 0.9726 0.9751 0.9761 0.9761 0.9775 0.978 0.9785 0.9785 "
2020-03-06 18:19:03 +00:00
]
}
],
"source": [
"train_model(linear_model, 20)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2020-04-15 12:21:02 +00:00
"0.4932 0.852 0.8335 0.9116 0.9326 0.9473 0.9555 0.9624 0.9648 0.9668 0.9692 0.9712 0.9731 0.9746 0.9761 0.9765 0.9775 0.978 0.9785 0.9785 "
2020-03-06 18:19:03 +00:00
]
}
],
"source": [
"linear_model = nn.Linear(28*28,1)\n",
"opt = SGD(linear_model.parameters(), lr)\n",
"train_model(linear_model, 20)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dls = DataLoaders(dl, valid_dl)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"learn = Learner(dls, nn.Linear(28*28,1), opt_func=SGD,\n",
" loss_func=mnist_loss, metrics=batch_accuracy)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>batch_accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.636857</td>\n",
" <td>0.503549</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.495584</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.545725</td>\n",
" <td>0.170281</td>\n",
" <td>0.866045</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.199223</td>\n",
" <td>0.184893</td>\n",
" <td>0.831207</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>3</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.086580</td>\n",
" <td>0.107836</td>\n",
" <td>0.911187</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>4</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.045185</td>\n",
" <td>0.078481</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.932777</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>5</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.029108</td>\n",
" <td>0.062792</td>\n",
" <td>0.946516</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>6</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.022560</td>\n",
" <td>0.053017</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.955348</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>7</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.019687</td>\n",
" <td>0.046500</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.962218</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>8</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.018252</td>\n",
" <td>0.041929</td>\n",
" <td>0.965162</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>9</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.017402</td>\n",
" <td>0.038573</td>\n",
" <td>0.967615</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn.fit(10, lr=lr)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## Adding a Nonlinearity"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def simple_net(xb): \n",
" res = xb@w1 + b1\n",
" res = res.max(tensor(0.0))\n",
" res = res@w2 + b2\n",
" return res"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"w1 = init_params((28*28,30))\n",
"b1 = init_params(30)\n",
"w2 = init_params((30,1))\n",
"b2 = init_params(1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD7CAYAAABt0P8jAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXhU5fn/8fctW1hlC7iBCAUUkDXivlRtcZcWtbK4tFYURau2Vluh4lK3trZqKZZvtaggLgiudalVvu7aEDaDEJQdBMIWk7AEkvv7x0x+v3FMyAmZJTPzeV3XXMyc85xz7nOYuXPmOWfux9wdERHJHPslOwAREUksJX4RkQyjxC8ikmGU+EVEMowSv4hIhmmY7ACCaN++vXfp0iXZYYiIpJQ5c+Zscvfs6Okpkfi7dOlCbm5ussMQEUkpZrayqunq6hERyTBK/CIiGUaJX0Qkwyjxi4hkGCV+EZEMU2PiN7MmZvaYma00s2Izm2tmZ+6l/Y1mtt7MiszscTNrEjGvi5m9a2bbzWyxmZ0eqx0REZFggpzxNwRWAycD+wPjgefMrEt0QzMbAtwKnAZ0AboCd0Q0mQ7MBdoBtwEzzOw795iKiEj81Jj43b3U3Se4+wp3r3D3V4HlwKAqml8GPObu+e6+FbgLuBzAzHoAA4Hb3X2Hu78ALASGxWhfRETSxleFJfzxzSXsKa+I+bpr3cdvZh2BHkB+FbN7A/MjXs8HOppZu/C8Ze5eHDW/dzXbGW1muWaWW1hYWNswRURS1vayPYyZOoenP1vF5tKymK+/VonfzBoB04An3H1xFU1aAEURryuft6xiXuX8llVty90nu3uOu+dkZ6s3SEQyg7tz26zPWbqxhIcu7k/HVlkx30bgxG9m+wFPAWXA2GqalQCtIl5XPi+uYl7l/GJERASAaZ+uYtbctdx4eg9O7B6fk95Aid/MDHgM6AgMc/fd1TTNB/pFvO4HbHD3zeF5Xc2sZdT8qrqMREQyzoI127jzlUWc0jObsd//Xty2E/SMfxJwBHCuu+/YS7sngSvMrJeZtQHGAVMA3L0AmAfcbmZZZvYjoC/wwr4GLyKSLraWljFmah7ZLZvw54v6s99+FrdtBbmP/1DgKqA/sN7MSsKPkWbWOfy8M4C7vwE8ALwLrAw/bo9Y3cVADrAVuA+4wN115VZEMlpFhXPjc/MoLN7F30YOpE3zxnHdXo1lmd19JbC3Pz0toto/CDxYzbpWAKcED09EJP1NfPdLZi8p5K6hfejXqXXct6eSDSIiSfTB0k08+HYBQ/sfxKijOydkm0r8IiJJ8nXRDq5/Zi7dO7Tgnh8fSeg+mvhT4hcRSYKyPRVcMy2PXbvLmTRqEM0aJ25AxJQYelFEJN3c868vmLtqGxNHDKRbdouaF4ghnfGLiCTYqwvWMeWjFfz0+C6c3ffAhG9fiV9EJIG+3FjCLTMWMLBza35z5hFJiUGJX0QkQUp3hYqvNWnUgIkjB9K4YXJSsPr4RUQSwN357ayFfFlYwlM/O5oD92+atFh0xi8ikgBTP1nJS/PWcdPpPTihe/ukxqLELyISZ/NWb+POVxfx/Z7ZXBvH4mtBKfGLiMTR1tIyrp2WR4eWWfz5J/EtvhaU+vhFROKkosK54dlQ8bUZY46ldbP4Fl8LSmf8IiJx8sg7X/K/BYXcfl4v+h4S/+JrQSnxi4jEwXsFhfzlPwX8eMDBjBicmOJrQSnxi4jE2NptO/jFM3Pp0aElv/9R4oqvBRV06MWxZpZrZrvMbMpe2j0aMVBLSbh9ccT82Wa2M2L+khjsg4hIvVG2p4Jrp+Wxu9yZNGogTRs3SHZI3xH04u464G5gCFDtrw7c/Wrg6srX4T8SFVHNxrr7P2oXpohIavj9a4uYt3obfxs5kK4JLr4WVKDE7+4zAcwsBzgkyDJm1hwYBpyzz9GJiKSQl+ev44mPV/LzEw7jrCMTX3wtqHj28Q8DCoH3oqbfa2abzOxDMzuluoXNbHS4eym3sFDD8opI/bZ0QzG3vrCAnEPbcMuZhyc7nL2KZ+K/DHjS3T1i2i1AV+BgYDLwipl1q2phd5/s7jnunpOdnR3HMEVE6qZ01x7GTMujWeNQ8bVGDer3fTNxic7MOgEnA09GTnf3T9292N13ufsTwIfAWfGIQUQkEdydW2cuZFlhCQ8PH0DHVlnJDqlG8fqzdCnwkbsvq6GdA/XrPicRkVp48uOVvDJ/Hb/8YU+O65bc4mtBBb2ds6GZZQENgAZmlmVme7swfCkwJWodrc1sSOWyZjYSOAl4cx9jFxFJqrxVW7n7tUWcengHxpxcZa91vRT0jH8csAO4FRgVfj7OzDqH78f/fz9LM7NjCd3583zUOhoRuiW0ENgEXAcMdXfdyy8iKWdLaRljp+XRsVUWf76ofhRfCyro7ZwTgAnVzP7Wjaru/jHQvIp1FAJH1S48EZH6p7zC+cUzc9lUWsbMMcexf7NGyQ6pVur3pWcRkXro4f8s5f2lm7jjvN70OXj/ZIdTa0r8IiK1MHvJRh5+ZynDBh7CxUd1SnY4+0SJX0QkoDVbt3PDs/Po2bEldw/tU++KrwWlxC8iEsCuPeVcOy2P8nJn0qhB9bL4WlAagUtEJIC7X/2C+WuKeHTUIA5r/537V1KKzvhFRGrw0ry1PPXJSkaf1JUz+hyQ7HDqTIlfRGQvCjYUc+sLCxncpS03D+mZ7HBiQolfRKQaJbv2cPXUOTRv0pBHRgyo98XXgkqPvRARiTF355YXFrBiUykPD++fEsXXglLiFxGpwpSPVvDagq+5ecjhKVN8LSglfhGRKHNWbuX3r33B6Ud05OqTuyY7nJhT4hcRibC5ZBdjn87joNZN+dNF/VL2R1p7o/v4RUTCQsXX5rG5svha09QqvhaUzvhFRMIeeruAD77cxF3np2bxtaCU+EVEgHcXb+Thd77kwkGH8JOjOte8QAoLOgLXWDPLNbNdZjZlL+0uN7Py8OAslY9TIuZ3MbN3zWy7mS02s9PrvgsiInWzekuo+NoRB7birqF9kh1O3AXt419HaPSsIUDTGtp+7O4nVDNvOvAxoQHWzwJmmFn38CAtIiIJt2tPOdc+nUdFhTNp5ECyGqVu8bWgAp3xu/tMd38R2LyvGzKzHsBA4HZ33+HuLwALgWH7uk4Rkbq685VFLFhTxB8v6keXFC++FlQ8+vgHmNkmMysws/ERg7L3Bpa5e3FE2/nh6d9hZqPD3Uu5hYX6QiAisTczbw3TPl3FVSd1ZUjv1C++FlSsE/97QB+gA6Ez+eHAzeF5LYCiqPZFQMuqVuTuk909x91zsrOzYxymiGS6xeu/4bezFjL4sPQpvhZUTBO/uy9z9+XuXuHuC4E7gQvCs0uAVlGLtAKKERFJoOKduxkzNY+WWY3464gBNEyT4mtBxXtvHaj82Vs+0NXMIs/w+4Wni4gkhLvz6xkLWLVlO38dPoAOLdOn+FpQQW/nbGhmWUADoIGZZUX03Ue2O9PMOoafHw6MB14CcPcCYB5we3j5HwF9gRdisysiIjV77IPlvP75en49pCdHd22X7HCSIugZ/zhgB3ArMCr8fJyZdQ7fq1/5a4fTgAVmVgr8C5gJ3BOxnouBHGArcB9wgW7lFJFEyV2xhfteX8wPe3Vk9EnpV3wtKHP3ZMdQo5ycHM/NzU12GCKSwjaV7OLsh98nq1EDXh57QtrW4YlkZnPcPSd6uoq0iUjaK69wrp8+l23bdzPrmsEZkfT3RolfRNLeg/9ewkdfbeaBC/rS66DomwszT2bdwyQiGeedxRuY+O5X/CSnExfldEp2OPWCEr+IpK3VW7ZzwzPz6HVgK+44v8oiARlJiV9E0tLO3eVcMy0PBx4dNSgjiq8FpT5+EUlLd7yyiIVri/ifS3Po3K5ZssOpV3TGLyJp54U5a5j+2SquPrkbP+jVMdnh1DtK/CKSVhav/4bbXlzIMV3b8qsf9kh2OPWSEr+IpI3K4mutshrxyPCBGVd8LSj18YtIWnB3bn4+VHxt+pXHkN2ySbJDqrf051BE0sJjHyznjfz
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plot_function(F.relu)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"simple_net = nn.Sequential(\n",
" nn.Linear(28*28,30),\n",
" nn.ReLU(),\n",
" nn.Linear(30,1)\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"learn = Learner(dls, simple_net, opt_func=SGD,\n",
" loss_func=mnist_loss, metrics=batch_accuracy)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>batch_accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.305828</td>\n",
" <td>0.399663</td>\n",
" <td>0.508341</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.142960</td>\n",
" <td>0.225702</td>\n",
" <td>0.807655</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.079516</td>\n",
" <td>0.113519</td>\n",
" <td>0.919529</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>3</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.052391</td>\n",
" <td>0.076792</td>\n",
" <td>0.943081</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>4</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.039796</td>\n",
" <td>0.060083</td>\n",
" <td>0.956330</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>5</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.033368</td>\n",
" <td>0.050713</td>\n",
" <td>0.963690</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>6</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.029680</td>\n",
" <td>0.044797</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.965653</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>7</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.027290</td>\n",
" <td>0.040729</td>\n",
" <td>0.968106</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>8</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.025568</td>\n",
" <td>0.037771</td>\n",
" <td>0.968597</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>9</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.024233</td>\n",
" <td>0.035508</td>\n",
" <td>0.970559</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>10</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.023149</td>\n",
" <td>0.033714</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.972031</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>11</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.022242</td>\n",
" <td>0.032243</td>\n",
" <td>0.972522</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>12</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.021468</td>\n",
" <td>0.031006</td>\n",
" <td>0.973503</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>13</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.020796</td>\n",
" <td>0.029944</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.974485</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>14</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.020207</td>\n",
" <td>0.029016</td>\n",
" <td>0.975466</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>15</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.019683</td>\n",
" <td>0.028196</td>\n",
" <td>0.976448</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>16</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.019215</td>\n",
" <td>0.027463</td>\n",
" <td>0.976448</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>17</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.018791</td>\n",
" <td>0.026806</td>\n",
" <td>0.976938</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>18</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.018405</td>\n",
" <td>0.026212</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.977920</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>19</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.018051</td>\n",
" <td>0.025671</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.977920</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>20</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.017725</td>\n",
" <td>0.025179</td>\n",
" <td>0.977920</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>21</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.017422</td>\n",
" <td>0.024728</td>\n",
" <td>0.978410</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>22</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.017141</td>\n",
" <td>0.024313</td>\n",
" <td>0.978901</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>23</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.016878</td>\n",
" <td>0.023932</td>\n",
" <td>0.979392</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>24</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.016632</td>\n",
" <td>0.023580</td>\n",
" <td>0.979882</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>25</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.016400</td>\n",
" <td>0.023254</td>\n",
" <td>0.979882</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>26</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.016181</td>\n",
" <td>0.022952</td>\n",
" <td>0.979882</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>27</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.015975</td>\n",
" <td>0.022672</td>\n",
" <td>0.980864</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>28</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.015779</td>\n",
" <td>0.022411</td>\n",
" <td>0.980864</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>29</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.015593</td>\n",
" <td>0.022168</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.981845</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>30</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.015417</td>\n",
" <td>0.021941</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.981845</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>31</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.015249</td>\n",
" <td>0.021728</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.981845</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>32</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.015088</td>\n",
" <td>0.021529</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.981845</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>33</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.014935</td>\n",
" <td>0.021341</td>\n",
" <td>0.981845</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>34</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.014788</td>\n",
" <td>0.021164</td>\n",
" <td>0.981845</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>35</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.014647</td>\n",
" <td>0.020998</td>\n",
" <td>0.982336</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>36</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.014512</td>\n",
" <td>0.020840</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.982826</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>37</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.014382</td>\n",
" <td>0.020691</td>\n",
2020-03-06 18:19:03 +00:00
" <td>0.982826</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>38</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.014257</td>\n",
" <td>0.020550</td>\n",
" <td>0.982826</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>39</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.014136</td>\n",
" <td>0.020415</td>\n",
" <td>0.982826</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:00</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn.fit(40, 0.1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
2020-04-15 12:21:02 +00:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD+CAYAAADBCEVaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZfElEQVR4nO3df5Dc9X3f8efrfkv3AxA6JLABBZBsLCei9TlxQ3CSxjGxmwwE7GkMxnRSBxfGE6ZOm3gypqW4M9RuO5k2Q7CZIcbGrmwngQSHGjvT4BjsukbEFa5sfOfElnAMdycJ7m5Pd7e3d+/+8d09rVZ7d1+dVtrd7/f1mLnR7Xe/t/fmo9OLz32+n31/FRGYmVm2dDS7ADMzazyHu5lZBjnczcwyyOFuZpZBDnczswxyuJuZZZDD3cwsg1KFu6T3S9onaUHSQ+uc+68lvSRpStIfS+ptSKVmZpaa0ryJSdINwDJwLbApIv7FKuddC3wK+KfAj4FHgW9ExAfXev2tW7fGjh07TqlwM7O8e/bZZw9HxHC957rSvEBEPAIgaQR49Rqn3go8GBEHyud/GPgMsGa479ixg3379qUpxczMyiQdXO25Rq+57wb2Vz3eD2yTdH6Dv4+Zma2h0eE+AExVPa58Plh7oqTbyuv4+yYnJxtchplZvjU63AvAUNXjyucztSdGxAMRMRIRI8PDdZeMzMxsgxod7geAPVWP9wDjEXGkwd/HzMzWkHYrZJekPqAT6JTUJ6nexdhPAf9S0usknQd8CHioYdWamVkqaWfuHwLmSHa9vLv8+YckXSKpIOkSgIh4Avgo8CRwsPzx7xtetZmZrSnVPvczbWRkJLwV0szs1Eh6NiJG6j2Xap+7mVk9EcFCaZnCQonl5eZPFOsJYHFpmWJpmcWloFhapri0RLEUFFeOJ38WS8ssLC2zWFpeea60tHxG6xvZsYU372r8phKHu1mTlZaSIFksBQtLSyshUx1EC+XHp/Ob9tJyUFgoMbuwxOxCiZmFErPlj5mFEsXS6iEWAQul5Osqr1Eof77UoqHeSNKZe+1/9fOXO9zN1lIJybni0gnhUwmkyuel0wij5QgWS0FxaWklfBfKYVysmfEVq2aDi0vLJ8wSF6qONzMb+3s66e/tYqC3i56uDrRGivV2dTDQ28XwYC8Dvd0M9HYy0Ne18vWdHWcwAU9Td0cHPV0ddHcmfyafi97qY1XP9XZ20t0lejo76Opsz/6KDndruNLSclWYLlFYWKRQni0W5k8O3NrHx4pLye/Sq1iOWAnPZoVkZ4fo7lQ5EDrprQqMSlB0d3Yw2Nd1QoBUB0nvSWFTCZaTgyj5PlozfNfTITFQDuL+3k76e7roaOFAttPjcM+ppeWoWmNcOnEZoLTMfGlpJXRnF0rMzJd/nS9WPq8K5/kSs8Xj5y2s8et9tcpMcKCvi/6eJHQuGOxjU08nHWuEmOCE0KsNyb6uDgb6kpllZVaZBFry0XOaM7Gero6WnqWagcM9MyKCV44t8tL0PC9NzzMxPc+R2SIvzxY5OrvIy8eKHJ0trvw5M1/a0PfZ1F0JzM6VUL7wnL4Tfj3v70kCe7AcpgN9XXWDtrtNf901awcO9xZTWCjxwtFjHDp6jJdniyet31Zf1T8yW2RieoGXpucZn56vO2Pu6+7g/P5ezuvv5rzNPVx6/mbO29zDuZu76evuTJYP6iwF9HZ10N/bxeAJod3ZtuuPZnnjcD+LqmfX49PzvDg1z6Gjx3ih8vHyHEdni2u+RmeHyuu54rz+HrYN9XHVxeey/Zw+tg31sW2ol+1DyedbB3rZ1NN5lv7rzKyVONzPgIjgbw+9wpcPvMSPp+YZn5pfdXbd1SFedd4mLtmymWsvOodLtmzm4i3J4/MHeleu4FfWlb3Wa2ZpONwb6OXZIo986x/43DOHGB0v0NPZwYXnJrPoPRefy/ahXrYN9bH9nD62V/3ppQ4zazSH+2laXg6+8fdH2PvMC3zp/71EcWmZqy4+l/90w0/yq3suYqDXQ2xmZ5+TZ4MWSkt84ms/ZO83D3HwyDHO2dTNTT9zCf/8jRdz5YVD67+AmdkZ5HDfgNHxGe787P/luy9O86bLtvCBX97Ftbu309fti5dm1hoc7qcgIvjk13/IvV98noHeLh68dYRfunJbs8syMzuJwz2liZl5/u2fPMffjE7yi68Z5qPv2MPwYG+zyzIzq8vhnsJffWec3/uz55hdKPHh63bz7jddelo9PszMzjSH+xqOFUt8+C+/y95vHuJ1Fw7x3991FVdcMNjssszM1uVwX8XM/CLv/Nj/5nvjM7zvzZfxgbfuorfLF0zNrD043OtYXg5+5/P7GZso8Me3vpFffO0FzS7JzOyU+K2RdfzRV77Pl78zzu+//UoHu5m1JYd7jSe/N8F//atRrr/qIn7z6h3NLsfMbEMc7lV+eHiWO/d+iyu3D3HvDT/lHTFm1rYc7mWzCyXe9/CzdHSIj9/yBrfKNbO25nAneefp7/7Zc4xNzPCH7/pHXLxlc7NLMjM7LQ534IGv/j2PP/civ/srr+WancPNLsfM7LTlPtyfHjvMR554nn/2kxfyvjdf1uxyzMwaItfh/sLRY7x/79+y84JBPvoOX0A1s+zIdbj/x8e/w9Jy8PFb3kC/b6phZhmS63B/7kdTvOXKbezY2t/sUszMGiq34T49v8iLU/NcccFAs0sxM2u43Ib72HgBgF3b3OXRzLInx+E+A8CubZ65m1n2pAp3SVskPSppVtJBSTetct65kj4paaL8cXdDq22g0fECfd0dXHye37BkZtmTdovIfUAR2AZcBTwuaX9EHKg57w+AzcAO4ALgf0k6GBGfaFC9DTM2McMVFwzQ0eHtj2aWPevO3CX1AzcCd0VEISKeBh4Dbqlz+q8BH42IYxHxQ+BB4DcbWG/DjI0X2OW7KplZRqVZltkFLEXEaNWx/cDuVc5Xzeev32BtZ8zU3CIvTc+z0xdTzSyj0oT7ADBVc2wKqJeMTwAflDQo6QqSWXvdRW1Jt0naJ2nf5OTkqdR82r4/kVxM3eltkGaWUWnCvQAM1RwbAmbqnPvbwBwwBvwFsBf4Ub0XjYgHImIkIkaGh89us65Rb4M0s4xLE+6jQJeknVXH9gC1F1OJiKMRcXNEbI+I3eXX/2ZjSm2c0fEZNnV38urzNjW7FDOzM2Ld3TIRMSvpEeAeSe8l2S1zHfCztedKuhx4pfzxVuA24OcbWnEDjI0XvFPGzDIt7ZuY7gA2ARMkSy23R8QBSddIKlSd9wbg2yRLNvcCN9fZLtl0YxMz7PSbl8wsw1Ltc4+Io8D1dY4/RXLBtfL488DnG1bdGTA1t8j49ILX280s03LXfsBtB8wsD3IX7pWdMjv9BiYzy7AchnuyU+ZV53qnjJllV+7CvXIx1TtlzCzL8hfu4wUvyZhZ5uUq3KeOLTIxs+CLqWaWebkK99GJyk4Zz9zNLNvyFe7lbZB+A5OZZV2uwn1svEB/j3fKmFn25SrcR8eTuy9J3iljZtmWs3Av+AYdZpYLuQn3l2eLHC54p4yZ5UNuwn1sotx2wDN3M8uB3IT76Li3QZpZfuQm3MfGZxjo7eKic/qaXYqZ2RmXm3AfLd99yTtlzCwPchPuYxMzvphqZrmRi3A/OlvkcKHohmFmlhu5CPcxtx0ws5zJRbiPlrdBeqeMmeVFLsJ9bHyGwd4uLvROGTPLiVyE++j4DFds804ZM8uPXIT72HiBXb6YamY5kvlwP1JY4Mhs0RdTzSxXMh/u7iljZnmU/XBf6SnjmbuZ5Ufmw310vMBgbxfbh7xTxszyIwfhPsNO75Qxs5zJfLiPTRT85iUzy51Mh/vhwgJHZ4u+mGpmuZPpcB8br7Qd8MVUM8uXbIf7hO++ZGb5lCrcJW2R9KikWUkHJd20ynm9kj4maVzSUUlfkPSqxpac3qEjx+jr7uCCwd5mlWBm1hRpZ+73AUVgG3AzcL+k3XXOuxP4J8BPARcBrwB/2IA6N2R6fpFzN/V4p4yZ5c664S6pH7gRuCsiChHxNPAYcEu
2020-03-06 18:19:03 +00:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.plot(L(learn.recorder.values).itemgot(2));"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2020-04-15 12:21:02 +00:00
"0.982826292514801"
2020-03-06 18:19:03 +00:00
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"learn.recorder.values[-1][2]"
]
},
2020-05-14 12:18:31 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Going Deeper"
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
2020-04-15 12:21:02 +00:00
" <td>0.082089</td>\n",
" <td>0.009578</td>\n",
" <td>0.997056</td>\n",
2020-03-06 18:19:03 +00:00
" <td>00:11</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"dls = ImageDataLoaders.from_folder(path)\n",
"learn = cnn_learner(dls, resnet18, pretrained=False,\n",
" loss_func=F.cross_entropy, metrics=accuracy)\n",
"learn.fit_one_cycle(1, 0.1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"## Jargon Recap"
2020-03-06 18:19:03 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Questionnaire"
]
},
2020-03-18 00:34:07 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"1. How is a grayscale image represented on a computer? How about a color image?\n",
2020-03-18 00:34:07 +00:00
"1. How are the files and folders in the `MNIST_SAMPLE` dataset structured? Why?\n",
"1. Explain how the \"pixel similarity\" approach to classifying digits works.\n",
"1. What is a list comprehension? Create one now that selects odd numbers from a list and doubles them.\n",
2020-05-14 12:18:31 +00:00
"1. What is a \"rank-3 tensor\"?\n",
2020-03-18 00:34:07 +00:00
"1. What is the difference between tensor rank and shape? How do you get the rank from the shape?\n",
"1. What are RMSE and L1 norm?\n",
"1. How can you apply a calculation on thousands of numbers at once, many thousands of times faster than a Python loop?\n",
2020-05-19 23:56:41 +00:00
"1. Create a 3×
2020-03-18 00:34:07 +00:00
"1. What is broadcasting?\n",
"1. Are metrics generally calculated using the training set, or the validation set? Why?\n",
"1. What is SGD?\n",
2020-05-14 12:18:31 +00:00
"1. Why does SGD use mini-batches?\n",
"1. What are the seven steps in SGD for machine learning?\n",
2020-03-18 00:34:07 +00:00
"1. How do we initialize the weights in a model?\n",
"1. What is \"loss\"?\n",
"1. Why can't we always use a high learning rate?\n",
"1. What is a \"gradient\"?\n",
"1. Do you need to know how to calculate gradients yourself?\n",
"1. Why can't we use accuracy as a loss function?\n",
"1. Draw the sigmoid function. What is special about its shape?\n",
2020-05-14 12:18:31 +00:00
"1. What is the difference between a loss function and a metric?\n",
2020-03-18 00:34:07 +00:00
"1. What is the function to calculate new weights using a learning rate?\n",
"1. What does the `DataLoader` class do?\n",
2020-05-14 12:18:31 +00:00
"1. Write pseudocode showing the basic steps taken in each epoch for SGD.\n",
"1. Create a function that, if passed two arguments `[1,2,3,4]` and `'abcd'`, returns `[(1, 'a'), (2, 'b'), (3, 'c'), (4, 'd')]`. What is special about that output data structure?\n",
2020-03-18 00:34:07 +00:00
"1. What does `view` do in PyTorch?\n",
"1. What are the \"bias\" parameters in a neural network? Why do we need them?\n",
2020-05-14 12:18:31 +00:00
"1. What does the `@` operator do in Python?\n",
2020-03-18 00:34:07 +00:00
"1. What does the `backward` method do?\n",
"1. Why do we have to zero the gradients?\n",
"1. What information do we have to pass to `Learner`?\n",
2020-05-14 12:18:31 +00:00
"1. Show Python or pseudocode for the basic steps of a training loop.\n",
2020-03-18 00:34:07 +00:00
"1. What is \"ReLU\"? Draw a plot of it for values from `-2` to `+2`.\n",
"1. What is an \"activation function\"?\n",
"1. What's the difference between `F.relu` and `nn.ReLU`?\n",
"1. The universal approximation theorem shows that any function can be approximated as closely as needed using just one nonlinearity. So why do we normally use more?"
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-05-14 12:18:31 +00:00
"### Further Research"
2020-03-06 18:19:03 +00:00
]
},
2020-03-18 00:34:07 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Create your own implementation of `Learner` from scratch, based on the training loop shown in this chapter.\n",
2020-05-14 12:18:31 +00:00
"1. Complete all the steps in this chapter using the full MNIST datasets (that is, for all digits, not just 3s and 7s). This is a significant project and will take you quite a bit of time to complete! You'll need to do some of your own research to figure out how to overcome some obstacles you'll meet on the way."
2020-03-18 00:34:07 +00:00
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
2020-03-22 20:03:50 +00:00
"jupytext": {
"split_at_heading": true
},
2020-03-06 18:19:03 +00:00
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
2020-03-17 19:15:55 +00:00
"nbformat_minor": 4
2020-03-06 18:19:03 +00:00
}
