2020-03-06 18:19:03 +00:00
{
"cells": [
{
"cell_type": "code",
2020-03-17 19:15:55 +00:00
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from utils import *"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Your deep learning journey"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deep learning is for everyone"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Neural networks: a brief history"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## What you will learn"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Who we are"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## How to learn deep learning"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Your projects and your mindset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The software: PyTorch, fastai, and Jupyter (and why it doesn't matter)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Your first model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Getting a GPU deep learning server"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Running your first notebook"
]
},
{
"cell_type": "code",
2020-03-17 19:15:55 +00:00
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>error_rate</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.169390</td>\n",
" <td>0.021388</td>\n",
" <td>0.005413</td>\n",
" <td>00:14</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>error_rate</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.058748</td>\n",
" <td>0.009240</td>\n",
" <td>0.002706</td>\n",
" <td>00:19</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# CLICK ME\n",
"from fastai2.vision.all import *\n",
"path = untar_data(URLs.PETS)/'images'\n",
"\n",
"def is_cat(x): return x[0].isupper()\n",
"dls = ImageDataLoaders.from_name_func(\n",
" path, get_image_files(path), valid_pct=0.2, seed=42,\n",
" label_func=is_cat, item_tfms=Resize(224))\n",
"\n",
"learn = cnn_learner(dls, resnet34, metrics=error_rate)\n",
"learn.fine_tune(1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Sidebar: This book was written in Jupyter Notebooks"
]
},
{
"cell_type": "code",
2020-03-17 19:15:55 +00:00
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2"
]
},
2020-03-17 19:15:55 +00:00
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"1+1"
]
},
{
"cell_type": "code",
2020-03-17 19:15:55 +00:00
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAJcAAADACAIAAACGdmZhAADHJElEQVR4nEz9Sa9lS5Ymhn2rMdt7n+Z23r4+XnSZEZXVJMVkUYSKpEoJUhxIIn+ANNH/ECBA/0ITQQQ00qAA6R+wWEpUqqTqsovIrIh4/fPu+r33NHubrUaDfV9AZ+Dw547r551j28zW+rpFwP/088+u/4//h//9//p/8/f+2f/t//T8yfkPfnzd20lks5yHX/3tq7/99avX7ylyk0jKIKQwapXzfDgfH5gAuIjUaUwmM4sIZkJkWCfzgWBLX+4P7Xwmd3JKcyYatGSmu7fWklCnqUzDsJl2l5esFEjPsHAIS9Fknd0skklFS5KmhYoWHQgcQURCxEwCIDLczdycwiQjgohqKbUOKurm3l1YVNXNez8qN+YAEJnpgYhSpC9LApSREaWMkVXrVlW7e2bqMC3NujuxSKkiEhkxN84QQu+niKUqVDnD3XqaZyaSSdkprNtQ6lRrb+18OnNRUgnlBW4RHojORaswpxtzFhGEIQIZlClCTMy6tSD37h76/EX5b/+7//Kf/Od//P/8Z//Xpc03T57NzRHj+Yxvv33/29++efPuFLEDBwHIjPRwJ3BGsLAQA8xMSGSCQCpCoIAxKJGn4+xLS/PoZksT4rRgIkQSEBGlFMvwcPROTVprwiUZSQAxiBKcINWqoAADlAlhJWJ3z3QmKVpAnBmZBASQyBRhruzdIjzCzRqFIxlhiYQDEXBf2lyEtCqDkiiIlmVx91JUpAKoZUwamDQTGZEEyhxq1UwQgSSQ3j3cVaUokwzWI8nN3b2HO9wBYmQiSbhQKVpEBLVmpmdCGMwFAGh9lCiDMolJGMKkdUJ4mHt4pgdoXuYkZYBZ9L/97/7kJz/b/o//4v/x9de//uXPPwjH6zen5dTevpu//urw+o1pecrQyKBMgCLTez/PS1hnYVVhUlAQiBwwj3VVMxkcQb1HLEYehYWkhDkIACKCAACsCuuREdbJpFvvRlBmlmAQUUIyAFZiQWQmCa877/G5IgI4GRkIICM804FgZmYipgBlBMyMDI4IT2JyiYhwq0wEeDMQIjMzwYR8XNHMXMIFEbDM9WfJrA9TUeIMBFE3CzcQSEGSBIIQgTyyeUcmAAKBKSKseRH1jO6OTFZNdxAziFgyM4LcWgJEhUGclJmtNWQmgCRAkGjuAWIigPTlB+fT6VcP7+4+eHn5+vXbr3/7fj6eHu7n4xnLrObTyFtiQxqQzAzP7t6XpkJDHaqKrg8x2D2WSA8vzCoV5ktzqvV8mpfjwunMDFEmJoATBGKiiMwEkjM5giKRTFyUtQpJgkU0ApHklhEJEEDrmcJM60cDAgzKRCZlUqYQQIkwRhABBKLkRGQogQmUwchBqLJ26z08AVXVUrho682Rc+u9d6JlKlyL1FrrOGQiQYhwBIDM9ZENVgbRYn1Zzh5tLJyEJAIAAYOJJeHuYKIQDmIGAM4EEpQsQZLJbuRORElEKgnqPYQLERMRyXrUoKi6RTcLD11OX9y+fri+2P7dr+fT3V07zyrVjEW2ZbhML+dukgvnTIIqmuG9NSDHYRzrQJRMTJzW3ZqlOSLXDUERAkqHNVvOZ5iJqAzD+nVHJINS2Fo3JBdCws3n1nU3QgvVUbggRVjP56V3Z0itg4iYWVEpWsAZYREBJDOIyT0T60ojCchkrP+dSBCl6np/rgcBIdiXRgkFs4rUUkoNQlrvZt3TIkVlaR3BLFylMlMEkImIiAATgCIMASg9ovXWbclkFWLRyKBAEjlzJCK8e5ZEUXHPSFMp7s5gZk4PC1QRApQgxB4R7kzy+9vEA5FBBQ5YhIXrfHw7SHn3+jVFIkTrpaB4ZNDQPRymQ812SuuULCyUBOS6LzOi9YUAAqw7EkRUVYgouqV1b30+nclzqoMndfc+LypKiXBnIlGNiABUKSN663Y+lzYWrcRZmAB2p4giMl7uL54/fzZNm+V8fn//vvfZvIf3REoRLQzkeiICCfDjLYPMQEYiLXM9gwjElAEQZSShaCEREBzo1h3ZWvdIYq7TZhq3drZMuLu7qxZhZubM9AgkmEmYHJFIYpBQdFta5FBVJD0DQSDKJNFBJTykDMO0cbM2L8ycICZWlmAIN6EAQYSZGUA3MgsSZmWRwkQBrG8sRTNSOQpBOQbvwRApY2vmHkAHpdRSSnj4fF6oJWdkBiGQcOtEaEubzycRqaXCASYVjdZ7b+F9OZ7aeVnOM3UrpONQ5/REmrv3TkQaAUJz97aQktYyIzb73bivmTIvnk7z8Xx1efP06bOnT55eXFyMwzA+H+4Od99+++XrN9+15gkDgimJAsgfPjwTAZEW3cM5ACZi8Lq8Yd26mxNyKHUcBiqytG6ttXDzsEh//GrVPQm0vta7VoQzU4UACXe3nolgmDszbcbB/eyewhwRS+8ErlVAkgnhwooEnZupSBlGawYpSHhASHfjNsmW3pJYRUEpkkQCLsRKJInMBK/VR2YKlFq1BFlNcwd6ZngyQSRIO3PLAGUIJyUyPSKEWZREJMy9m3sIK0EjPT3cIxIApVNbupmFBzwISRAIiFgEVMBEIkrCHL6YGdITTFS4zoelXuzI2RZ89unPLnbX07Qd62SdT2bWeBwuPvn4J0z61Te/DRARzIwoQUGQxysEBAIQgAeCASJmhvdu1jJChVULE/fo2ax1m1vr3UiKiogosRAzHD+0MRQRbrZ2L0gIEgjPiG7BxEwU6dGFiGV9nNg9LCI8tYiqMnFEmEW3hXL9H8z1N4VYREU1c1GiTCQYBNVCpCAmZoCICBmPHUMkMtUbA5RBlMiMyC4MVbAEUcJbZCCkFiUiAtx8/Yjrv8XEVWstVYQZHBkAsQoTI5JFHh9gUSFKpshgIhAzY/0rd2dVBUWYMNcykOckA/WsVD/4+KNPPvo8UzLJDdaDhYhQatnvds9f+N3D3e3771v2UokZGQk4EYhIuDAzy1oSO+DrFu3u3UyIiImYuru1JZBEwsTMAmImJS7MAiIAJCSkTExrtRKZCGIQggFO97B0lFo9fJnPWphEKFNEpnFqzdxTWFUqLAAKJNYFIayHE8yDUDJShUilaCQygkFj0YgfjgNQEGkSETLT3COgnIK1BxLKTKSJkHIKgxERMHMigTAThTtlqBZChnUGD8NQa2VQAGuBHplEFIgAWDUSHklMIiWJ0vpacEQEZYRnN9MKpzRLcIZnLM4TqfKzpy8++ejz8JJODna3BFUuxLV3tOU8DvsPX3727t2r1vo4bQGPiIi1NoxkZyGPzKRIyiQCRB4fKzC6R4/GTM17EokIqQgJESM5LIJIpYhwrj8jqsq8fpVuCKdMymCEMNKSMzOSIgQqzIu1BKZxo1yXbkLCQdGDCWBKJhYBk7knsJbfQhSggDAIlGBWYVU1cyBz7foBEDI90ynD03W9liOM2Zkjw4XBROTESZzKyWC2tetBEhMTZa5tWaoIiMMMImWYQJnhTGx9OZ98Xtp5Waz35B/2OoiIcv2yI5mIRCJhkZHRzOJ03kyXp7uHD5998tkHn/az94467jkRSeDIRGtOKsvSh2H77Onz7fbicDJhNbe1ngFIWEQKaxEtyEAmKDLN+hKUJMqM8O7m025UIouITETQ2tt4khMxKVhZg0DCvB6RzASEhZtlerohQphIiSPBOdUSGRQJC4uOChVdm3hYSFJEpIchjJFExJSRzCSiomUtws2DM0Ufb1Na67CIBDiTidLN09kDGUocYc19AUgEzCHMlCsexZqiTM7ZvUc4gVaYRkTdo83NugmTWUCcShXWQIR7azafzud5tghWpuTebemNagEg6wVFYKLI9HAQaqkQAnFb/EcvXj67ecnQwmMZh0hlSlWBOHN6JkhKHTPJArvtZevH1jyADCRIRUodiNQy16+dKQmcmcvi5o4IYiVRJOZuzEogj2CQakmHsBYRZmViJIkyrXdIurtTpvXFeg/ryGBmEVbW7inM0zgdz0c3L1wc4d2JEYGIoJQqggCFA5SZiVwL2PWOsYhI0jrM59ncq3BGtOa1CBIRa3WZSaIiCgn
"text/plain": [
"<PIL.Image.Image image mode=RGB size=151x192 at 0x7F67678F7E90>"
]
},
2020-03-17 19:15:55 +00:00
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"img = PILImage.create('images/chapter1_cat_example.jpg')\n",
"img.to_thumb(192)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### End sidebar"
]
},
{
"cell_type": "code",
2020-03-17 19:15:55 +00:00
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "f78619047d7544908daa7fadd3c6f0c4",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"FileUpload(value={}, description='Upload')"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"uploader = widgets.FileUpload()\n",
"uploader"
]
},
{
"cell_type": "code",
2020-03-17 19:15:55 +00:00
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {
"hide_input": true
},
"outputs": [],
"source": [
"#hide\n",
"# For the book, we can't actually click an upload button, so we fake it\n",
"uploader = SimpleNamespace(data = ['images/chapter1_cat_example.jpg'])"
]
},
{
"cell_type": "code",
2020-03-17 19:15:55 +00:00
"execution_count": null,
2020-03-06 18:19:03 +00:00
"metadata": {},
"outputs": [
{
"data": {
"text/html": [],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Is this a cat?: True.\n",
"Probability it's a cat: 0.999986\n"
]
}
],
"source": [
"img = PILImage.create(uploader.data[0])\n",
"is_cat,_,probs = learn.predict(img)\n",
"print(f\"Is this a cat?: {is_cat}.\")\n",
"print(f\"Probability it's a cat: {probs[1].item():.6f}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### What is machine learning?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": false
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
" -->\n",
"<!-- Title: G Pages: 1 -->\n",
"<svg width=\"285pt\" height=\"58pt\"\n",
" viewBox=\"0.00 0.00 284.59 58.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 54)\">\n",
"<title>G</title>\n",
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-54 280.5882,-54 280.5882,4 -4,4\"/>\n",
"<!-- program -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>program</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"172.9942,-50 104.9942,-50 100.9942,-46 100.9942,0 168.9942,0 172.9942,-4 172.9942,-50\"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"168.9942,-46 100.9942,-46 \"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"168.9942,-46 168.9942,0 \"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"168.9942,-46 172.9942,-50 \"/>\n",
"<text text-anchor=\"middle\" x=\"136.9942\" y=\"-21.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">program</text>\n",
"</g>\n",
"<!-- results -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>results</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"242.7912\" cy=\"-25\" rx=\"33.5952\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"242.7912\" y=\"-21.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">results</text>\n",
"</g>\n",
"<!-- program->results -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>program->results</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M173.1077,-25C181.3637,-25 190.2284,-25 198.7746,-25\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"198.9789,-28.5001 208.9789,-25 198.9788,-21.5001 198.9789,-28.5001\"/>\n",
"</g>\n",
"<!-- inputs -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>inputs</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"32.4971\" cy=\"-25\" rx=\"32.4942\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"32.4971\" y=\"-21.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">inputs</text>\n",
"</g>\n",
"<!-- inputs->program -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>inputs->program</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M65.2739,-25C73.2739,-25 81.9845,-25 90.4897,-25\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"90.7006,-28.5001 100.7006,-25 90.7005,-21.5001 90.7006,-28.5001\"/>\n",
"</g>\n",
"</g>\n",
"</svg>\n"
],
"text/plain": [
"<graphviz.files.Source at 0x7fb712ad6310>"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gv('''program[shape=box3d width=1 height=0.7]\n",
"inputs->program->results''')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
" -->\n",
"<!-- Title: G Pages: 1 -->\n",
"<svg width=\"296pt\" height=\"98pt\"\n",
" viewBox=\"0.00 0.00 296.29 98.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 94)\">\n",
"<title>G</title>\n",
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-94 292.2872,-94 292.2872,4 -4,4\"/>\n",
"<!-- model -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>model</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"184.6932,-70 116.6932,-70 112.6932,-66 112.6932,-20 180.6932,-20 184.6932,-24 184.6932,-70\"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"180.6932,-66 112.6932,-66 \"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"180.6932,-66 180.6932,-20 \"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"180.6932,-66 184.6932,-70 \"/>\n",
"<text text-anchor=\"middle\" x=\"148.6932\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">model</text>\n",
"</g>\n",
"<!-- results -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>results</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"254.4902\" cy=\"-45\" rx=\"33.5952\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"254.4902\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">results</text>\n",
"</g>\n",
"<!-- model->results -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>model->results</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M184.8066,-45C193.0627,-45 201.9273,-45 210.4735,-45\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"210.6779,-48.5001 220.6778,-45 210.6778,-41.5001 210.6779,-48.5001\"/>\n",
"</g>\n",
"<!-- inputs -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>inputs</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"38.3466\" cy=\"-72\" rx=\"32.4942\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"38.3466\" y=\"-68.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">inputs</text>\n",
"</g>\n",
"<!-- inputs->model -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>inputs->model</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M68.208,-64.6934C78.7724,-62.1085 90.9176,-59.1367 102.5174,-56.2985\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"103.6359,-59.6281 112.5174,-53.8516 101.9721,-52.8287 103.6359,-59.6281\"/>\n",
"</g>\n",
"<!-- weights -->\n",
"<g id=\"node4\" class=\"node\">\n",
"<title>weights</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"38.3466\" cy=\"-18\" rx=\"38.1938\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"38.3466\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">weights</text>\n",
"</g>\n",
"<!-- weights->model -->\n",
"<g id=\"edge3\" class=\"edge\">\n",
"<title>weights->model</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M72.3546,-26.3212C81.8534,-28.6454 92.3387,-31.211 102.4171,-33.677\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"101.763,-37.1201 112.3084,-36.0972 103.4268,-30.3207 101.763,-37.1201\"/>\n",
"</g>\n",
"</g>\n",
"</svg>\n"
],
"text/plain": [
"<graphviz.files.Source at 0x7fb71350edd0>"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gv('''model[shape=box3d width=1 height=0.7]\n",
"inputs->model->results; weights->model''')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
" -->\n",
"<!-- Title: G Pages: 1 -->\n",
"<svg width=\"483pt\" height=\"98pt\"\n",
" viewBox=\"0.00 0.00 483.08 98.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 94)\">\n",
"<title>G</title>\n",
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-94 479.0773,-94 479.0773,4 -4,4\"/>\n",
"<!-- model -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>model</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"185.6932,-77 117.6932,-77 113.6932,-73 113.6932,-27 181.6932,-27 185.6932,-31 185.6932,-77\"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"181.6932,-73 113.6932,-73 \"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"181.6932,-73 181.6932,-27 \"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"181.6932,-73 185.6932,-77 \"/>\n",
"<text text-anchor=\"middle\" x=\"149.6932\" y=\"-48.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">model</text>\n",
"</g>\n",
"<!-- results -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>results</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"292.4902\" cy=\"-52\" rx=\"33.5952\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"292.4902\" y=\"-48.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">results</text>\n",
"</g>\n",
"<!-- model->results -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>model->results</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M185.7269,-52C204.7235,-52 228.1886,-52 248.278,-52\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"248.4342,-55.5001 258.4342,-52 248.4341,-48.5001 248.4342,-55.5001\"/>\n",
"</g>\n",
"<!-- inputs -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>inputs</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"38.3466\" cy=\"-72\" rx=\"32.4942\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"38.3466\" y=\"-68.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">inputs</text>\n",
"</g>\n",
"<!-- inputs->model -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>inputs->model</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M69.3628,-66.4289C79.914,-64.5337 91.9459,-62.3725 103.4279,-60.3101\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"104.102,-63.7452 113.3257,-58.5323 102.8644,-56.8554 104.102,-63.7452\"/>\n",
"</g>\n",
"<!-- performance -->\n",
"<g id=\"node5\" class=\"node\">\n",
"<title>performance</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"419.1822\" cy=\"-52\" rx=\"55.7903\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"419.1822\" y=\"-48.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">performance</text>\n",
"</g>\n",
"<!-- results->performance -->\n",
"<g id=\"edge4\" class=\"edge\">\n",
"<title>results->performance</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M326.4413,-52C334.7307,-52 343.8829,-52 353.1124,-52\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"353.1206,-55.5001 363.1205,-52 353.1205,-48.5001 353.1206,-55.5001\"/>\n",
"</g>\n",
"<!-- weights -->\n",
"<g id=\"node4\" class=\"node\">\n",
"<title>weights</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"38.3466\" cy=\"-18\" rx=\"38.1938\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"38.3466\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">weights</text>\n",
"</g>\n",
"<!-- weights->model -->\n",
"<g id=\"edge3\" class=\"edge\">\n",
"<title>weights->model</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M70.5526,-27.8342C80.9847,-31.0197 92.7624,-34.616 103.982,-38.042\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"103.064,-41.4211 113.6503,-40.9942 105.1084,-34.7263 103.064,-41.4211\"/>\n",
"</g>\n",
"<!-- performance->weights -->\n",
"<g id=\"edge5\" class=\"edge\">\n",
"<title>performance->weights</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M380.7799,-38.6552C364.2203,-33.4733 344.5068,-28.0582 326.2872,-25 242.9542,-11.0123 144.1752,-12.4408 86.651,-15.0281\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"86.2233,-11.5445 76.4031,-15.5212 86.5598,-18.5364 86.2233,-11.5445\"/>\n",
"<text text-anchor=\"middle\" x=\"222.1932\" y=\"-19.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">update</text>\n",
"</g>\n",
"</g>\n",
"</svg>\n"
],
"text/plain": [
"<graphviz.files.Source at 0x7f3e58071bd0>"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gv('''ordering=in\n",
"model[shape=box3d width=1 height=0.7]\n",
"inputs->model->results; weights->model; results->performance\n",
"performance->weights[constraint=false label=update]''')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
" -->\n",
"<!-- Title: G Pages: 1 -->\n",
"<svg width=\"285pt\" height=\"58pt\"\n",
" viewBox=\"0.00 0.00 284.59 58.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 54)\">\n",
"<title>G</title>\n",
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-54 280.5882,-54 280.5882,4 -4,4\"/>\n",
"<!-- model -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>model</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"172.9942,-50 104.9942,-50 100.9942,-46 100.9942,0 168.9942,0 172.9942,-4 172.9942,-50\"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"168.9942,-46 100.9942,-46 \"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"168.9942,-46 168.9942,0 \"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"168.9942,-46 172.9942,-50 \"/>\n",
"<text text-anchor=\"middle\" x=\"136.9942\" y=\"-21.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">model</text>\n",
"</g>\n",
"<!-- results -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>results</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"242.7912\" cy=\"-25\" rx=\"33.5952\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"242.7912\" y=\"-21.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">results</text>\n",
"</g>\n",
"<!-- model->results -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>model->results</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M173.1077,-25C181.3637,-25 190.2284,-25 198.7746,-25\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"198.9789,-28.5001 208.9789,-25 198.9788,-21.5001 198.9789,-28.5001\"/>\n",
"</g>\n",
"<!-- inputs -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>inputs</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"32.4971\" cy=\"-25\" rx=\"32.4942\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"32.4971\" y=\"-21.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">inputs</text>\n",
"</g>\n",
"<!-- inputs->model -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>inputs->model</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M65.2739,-25C73.2739,-25 81.9845,-25 90.4897,-25\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"90.7006,-28.5001 100.7006,-25 90.7005,-21.5001 90.7006,-28.5001\"/>\n",
"</g>\n",
"</g>\n",
"</svg>\n"
],
"text/plain": [
"<graphviz.files.Source at 0x7fb712ade110>"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gv('''model[shape=box3d width=1 height=0.7]\n",
"inputs->model->results''')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### What is a neural network?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### A bit of deep learning jargon"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
" -->\n",
"<!-- Title: G Pages: 1 -->\n",
"<svg width=\"489pt\" height=\"134pt\"\n",
" viewBox=\"0.00 0.00 489.18 134.20\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 130.2026)\">\n",
"<title>G</title>\n",
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-130.2026 485.1822,-130.2026 485.1822,4 -4,4\"/>\n",
"<!-- model -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>model</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"217.0911,-79.2026 141.0911,-79.2026 137.0911,-75.2026 137.0911,-29.2026 213.0911,-29.2026 217.0911,-33.2026 217.0911,-79.2026\"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"213.0911,-75.2026 137.0911,-75.2026 \"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"213.0911,-75.2026 213.0911,-29.2026 \"/>\n",
"<polyline fill=\"none\" stroke=\"#000000\" points=\"213.0911,-75.2026 217.0911,-79.2026 \"/>\n",
"<text text-anchor=\"middle\" x=\"177.0911\" y=\"-50.5026\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">architecture</text>\n",
"</g>\n",
"<!-- predictions -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>predictions</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"340.1367\" cy=\"-54.2026\" rx=\"50.0912\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"340.1367\" y=\"-50.5026\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">predictions</text>\n",
"</g>\n",
"<!-- model->predictions -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>model->predictions</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M217.3945,-54.2026C236.277,-54.2026 259.1631,-54.2026 280.0334,-54.2026\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"280.0905,-57.7027 290.0905,-54.2026 280.0905,-50.7027 280.0905,-57.7027\"/>\n",
"</g>\n",
"<!-- inputs -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>inputs</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"50.0456\" cy=\"-74.2026\" rx=\"32.4942\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"50.0456\" y=\"-70.5026\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">inputs</text>\n",
"</g>\n",
"<!-- inputs->model -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>inputs->model</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M81.4501,-69.2588C95.1932,-67.0953 111.6581,-64.5034 127.0224,-62.0847\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"127.5863,-65.5391 136.9204,-60.5265 126.4977,-58.6242 127.5863,-65.5391\"/>\n",
"</g>\n",
"<!-- loss -->\n",
"<g id=\"node6\" class=\"node\">\n",
"<title>loss</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"454.1822\" cy=\"-83.2026\" rx=\"27\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"454.1822\" y=\"-79.5026\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">loss</text>\n",
"</g>\n",
"<!-- predictions->loss -->\n",
"<g id=\"edge5\" class=\"edge\">\n",
"<title>predictions->loss</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M381.3097,-64.6723C393.5479,-67.7843 406.8441,-71.1653 418.6879,-74.177\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"418.105,-77.6401 428.6592,-76.7125 419.8302,-70.856 418.105,-77.6401\"/>\n",
"</g>\n",
"<!-- parameters -->\n",
"<g id=\"node4\" class=\"node\">\n",
"<title>parameters</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"50.0456\" cy=\"-20.2026\" rx=\"50.0912\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"50.0456\" y=\"-16.5026\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">parameters</text>\n",
"</g>\n",
"<!-- parameters->model -->\n",
"<g id=\"edge3\" class=\"edge\">\n",
"<title>parameters->model</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M90.2436,-30.9605C101.8295,-34.0611 114.6176,-37.4834 126.7368,-40.7268\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"126.1536,-44.1938 136.7185,-43.3981 127.9633,-37.4318 126.1536,-44.1938\"/>\n",
"</g>\n",
"<!-- labels -->\n",
"<g id=\"node5\" class=\"node\">\n",
"<title>labels</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"340.1367\" cy=\"-108.2026\" rx=\"31.3957\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"340.1367\" y=\"-104.5026\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">labels</text>\n",
"</g>\n",
"<!-- labels->loss -->\n",
"<g id=\"edge4\" class=\"edge\">\n",
"<title>labels->loss</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M369.5062,-101.7645C384.4486,-98.489 402.7335,-94.4808 418.483,-91.0283\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"419.3874,-94.4132 428.4059,-88.8531 417.8884,-87.5756 419.3874,-94.4132\"/>\n",
"</g>\n",
"<!-- loss->parameters -->\n",
"<g id=\"edge6\" class=\"edge\">\n",
"<title>loss->parameters</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M441.3968,-67.1028C429.6802,-53.7017 411.0259,-35.5188 390.1822,-27.2026 295.3447,10.6354 173.4836,.5619 104.3447,-9.9693\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"103.4538,-6.5667 94.1247,-11.5883 104.5492,-13.4805 103.4538,-6.5667\"/>\n",
"<text text-anchor=\"middle\" x=\"253.5911\" y=\"-6.0026\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">update</text>\n",
"</g>\n",
"</g>\n",
"</svg>\n"
],
"text/plain": [
"<graphviz.files.Source at 0x7f3e2dc3ba90>"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gv('''ordering=in\n",
"model[shape=box3d width=1 height=0.7 label=architecture]\n",
"inputs->model->predictions; parameters->model; labels->loss; predictions->loss\n",
"loss->parameters[constraint=false label=update]''')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Limitations inherent to machine learning\n",
"\n",
"From this picture we can now see some fundamental things about training a deep learning model:\n",
"\n",
"- A model cannot be created without data ;\n",
"- A model can only learn to operate on the patterns seen in the input data used to train it ;\n",
"- This learning approach only creates *predictions*, not recommended *actions* ;\n",
"- It's not enough to just have examples of input data; we need *labels* for that data too (e.g. pictures of dogs and cats aren't enough to train a model; we need a label for each one, saying which ones are dogs, and which are cats).\n",
"\n",
"Generally speaking, we've seen that most organizations that think they don't have enough data, actually mean they don't have enough *labeled* data. If any organization is interested in doing something in practice with a model, then presumably they have some inputs they plan to run their model against. And presumably they've been doing that some other way for a while (e.g. manually, or with some heuristic program), so they have data from those processes! For instance, a radiology practice will almost certainly have an archive of medical scans (since they need to be able to check how their patients are progressing over time), but those scans may not have structured labels containing a list of diagnoses or interventions (since radiologists generally create free text natural language reports, not structured data). We'll be discussing labeling approaches a lot in this book, since it's such an important issue in practice.\n",
"\n",
"Since these kinds of machine learning models can only make *predictions* (i.e. attempt to replicate labels), this can result in a significant gap between organizational goals and model capabilities. For instance, in this book you'll learn how to create a *recommendation system* that can predict what products a user might purchase. This is often used in e-commerce, such as to customize products shown on a home page, by showing the highest-ranked items. But such a model is generally created by looking at a user and their buying history (*inputs*) and what they went on to buy or look at (*labels*), which means that the model is likely to tell you about products they already have, or already know about, rather than new products that they are most likely to be interested in hearing about. That's very different to what, say, an expert at your local bookseller might do, where they ask questions to figure out your taste, and then tell you about authors or series that you've never heard of before."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### How our image recognizer works"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### What our image recognizer learned"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Image recognizers can tackle non-image tasks"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Jargon recap"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deep learning is not just for image classification"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>2.906601</td>\n",
" <td>2.347491</td>\n",
" <td>00:02</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>1.988776</td>\n",
" <td>1.765969</td>\n",
" <td>00:02</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
" <td>1.703356</td>\n",
" <td>1.265247</td>\n",
" <td>00:02</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>1.591550</td>\n",
" <td>1.309860</td>\n",
" <td>00:02</td>\n",
" </tr>\n",
" <tr>\n",
" <td>3</td>\n",
" <td>1.459745</td>\n",
" <td>1.102660</td>\n",
" <td>00:02</td>\n",
" </tr>\n",
" <tr>\n",
" <td>4</td>\n",
" <td>1.324229</td>\n",
" <td>0.948472</td>\n",
" <td>00:02</td>\n",
" </tr>\n",
" <tr>\n",
" <td>5</td>\n",
" <td>1.205859</td>\n",
" <td>0.894631</td>\n",
" <td>00:02</td>\n",
" </tr>\n",
" <tr>\n",
" <td>6</td>\n",
" <td>1.102528</td>\n",
" <td>0.809563</td>\n",
" <td>00:02</td>\n",
" </tr>\n",
" <tr>\n",
" <td>7</td>\n",
" <td>1.020853</td>\n",
" <td>0.805135</td>\n",
" <td>00:02</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"path = untar_data(URLs.CAMVID_TINY)\n",
"dls = SegmentationDataLoaders.from_label_func(\n",
" path, bs=8, fnames = get_image_files(path/\"images\"),\n",
" label_func = lambda o: path/'labels'/f'{o.stem}_P{o.suffix}',\n",
" codes = np.loadtxt(path/'codes.txt', dtype=str)\n",
")\n",
"\n",
"learn = unet_learner(dls, resnet34)\n",
"learn.fine_tune(8)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAH6CAYAAAD7gBCoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9ebwlV1nv/V017vnsM089d6e7E4YMEMLwKirghCAkBFB5ATWEMIjc14mL9ypXvb731fu5oswIXiYNMoOgKOirgIwyZU463X1On3k+e6xd8/1jrV1nd6f75HQSupOwvp9Pf/rs2lW1V1WtX631POtZzxJpmqLRaDQazU4wLnYBNBqNRvPIQTcaGo1Go9kxutHQaDQazY7RjYZGo9FodoxuNDQajUazY3SjodFoNJodoxsNjeYhRgjxFSFEKoT4ovpsqc+pEOK/PATnv6HnfLsefIk1mp2jG40fMoQQUz0vnHP9e9PFLmcvQohDPWV7yTn2+S0hRCSEGBRC/NFZrmlVCPFVIcQvXujyAynwDfVvdqcHndn49LDccz7/ISulRrMDrItdAM0F57vAovp7FzCp/v4eWy+gHb/YzkQI4aRpGjzw4j1gngd8OU3TNSFE7/bvAjFwBHgK8BQhxFCapn9xrhMJIew0TcOHqmBpmsbAkx/C830G+MxDdT6N5nzQlsYPGWmaPj9N0yenafpk4D09X/Vu/4gQ4tPKKmkJIXwhxD1CiDcJIezuAb09YSHEG4UQ88Cc+q4qhLhZHb8ghPhdIcSH1P739pzDEEL8mhDiFiFERwixqX77UvX9DcCxnnJ+8MzetxBiBPlS/tRZLvm5aZpeDVzCVqP40u65eyyRFwghvi2ECIGfUt9fI4T4nBBiQ92DW4QQL+s9uRBijxDiH4QQnrpfv3pmAc7lnhJCjAsh3i2EmBFCBEKIZSHEp7r7A09Tuz6j1x11LveUEOJ56pk01b38nhDixnOU44+FEG8XQqyr3/0zIYTuRGruF11JNGejADwXaZHcDQwjX7q/D7jAfz5j/x8Bfkzt66htfwU8X/09C/wOZ++kvA24Sf19BzCkfvtHhBBXIV0x3wOuUPscB1bVvl1+Xp370+d1lafz18AKMA2kQogfAf4ZsIElYAF4HPA+IUQ1TdM/F9Kk+QTwBKQLqg38hfp7W4QQw0j30m616Zj6rWez5c66DCgDdeBOtd9Z3VFCiJcD/1t9XAI6wOXAu4QQu9M0/a9nHPKb6rweMAG8Hril5xwazdlJ01T/+yH9B7wJ+YJKgX0923PA0TP2vVntd7Jn21d6jn+m2mYCh3u2/5naPgHU1LZ71bZDQKK2/UrPb9+ptr2jZ7/u+V5yluv4HPDdns9/1LP/d4BvIl+Q3W2vU/vd0LPtA4DRcw1fUtv/BbDU9t9X2zaRjeOzeo5/jdrnMUCktn1RbbN69vsvatsf9Gx7YU/ZrzrL/f3iGdfbW+5datus+vx1ZMMugI+obT7Qd0Y5jiEbpDyyc5ACH7rYdVL/e/j/0+4pzdmIgJcLIY4pt0wKvFh9N3GW/W9P0/SLkPnvH9vz3c1q+zzwb2ccdzXy5QbwXvU7HnBUbbvfcQAhRAl4Bmd3TQFcqX4nBL6GbHTONp7xF2maJj3XcI3a/uNAqMr2JrWtD7gUaXl0+Yg69nbg9vsrd8/5703T9CPdjWmafmcHx56GEGKCrbGpT6Rp6qdpmgIfVtsc4PFnHPbpNE0baZp6wJTaNnq+v6354UO7pzRn43eR7iSQL5QltgbNzbPsv3iWbV22c9X0jlj3DsR3mdm2lJKfQfasz9Vo7E7TdCcD++e6hjnOHhiw0+u60Ow0bfVmz9+R+v9illvzCEFbGpqz0e3h35Gm6X7kgOxt53H8rT1/vwCy3vDTz9jvWz1/fyhVA/GpHIx/LfC/1Hftnv2KZ5zjecBUmqbfP4/y7YRu2WaBn+gp13ORLrdbOP2eXA8ghLgMORZxf3xD/X9ICHFdd6MQ4oqefbrXfeY1n4ay4ubUx+uEEK4ab+lahwFyvEKjedDoRkNzNrovmMuEECeAU0gXz45I0/QY8En18beFEHchxymss+z3l+rj/1TRR98XQqwjX9rPUN8tstUz/lMhxDeEEK9WkVw/y7mtjAfD7yJ74NcAC0KI7wohZpAD4v9d7fMFZEgvwFuEELerckdnnuwsvIUtS+pjKjrtOKc3pHep/5+s7svntjlfNyrrGuRg/klUQwb8SZqmtR2USaO5X3SjoTkbfwh8EDlwXQE+BLz7PM/xK0ifehvoR1oNX1DfeT373cRW5M4osB/5Yn47qjFQYw2vQEZOFYAnAXuQEVtVfgCNRpqm/wb8KHKQPUZaDwnwWeD31D4pMkLsn5BjJhXgDcB/7OD8K8gX/F8irZn9yIHp3obhT5ARXC3kmMQTtjnf+1RZvqrOMwZ8H7gpvW/klEbzgBGy3ms0Dy1CiD3AcpqmHfV5BDlAPAR8ME3Tlz4Ev/E24EXAqBq81mg0P2D0QLjmB8ULgf8shPg2shf+VKRV0AD+x0P0G7cCX9ENhkZz4dCWhuYHghDiJ5FuriPIgdxl4F+BP0zT9K5tDtVoNA9jdKOh0Wg0mh2jB8I1Go1Gs2N0o6HRaDSaHaMbDY1Go9HsGN1oaDQajWbH6EZDo9FoNDtGNxoajUaj2TG60dBoNBrNjtGNhkaj0Wh2jG40NBqNRrNjdKOh0Wg0mh2jGw2NRqPR7BjdaGg0Go1mx+hGQ6PRaDQ7RjcaGo1Go9kxutHQaDQazY7RjYZGo9FodoxuNDQajUazY3SjodFoNJodoxsNjUaj0ewY3WhoNBqNZsfoRkOj0Wg0O0Y3GhqNRqPZMbrR0Gg0Gs2O0Y2GRqPRaHaMbjQ0Go1Gs2N0o6HRaDSaHaMbDY1Go9HsGN1oaDQajWbH6EZDo9FoNDtGNxoajUaj2TG60dBoNBrNjtGNhkaj0Wh2jG40NBqNRrNjdKOh0Wg0mh2jGw2NRqPR7BjdaGg0Go1mx+hGQ6PRaDQ7RjcaGo1Go9kxutHQaDQazY7RjYZGo9FodoxuNDQajUazY3SjodFoNJodY233Ze0L06kQp2+LowiAdqvNV/w/ptWQOxw9cpB8MYft2ACYtoHj2rSabfnZMrAsi2ZdfhaGCSLlrtuPyeOPHqFUdfnW178DwEB/P7v27sKyDLU/mIZFq90BIElSkjih0lcAYH5hnpmpRSYnJgEoV4o0mi0Or79cltfrUOkr8O6DLgCvOZVg2Sak8rrOvM7zJU5S0jTFNFV5e7YDfNt+B8PHn0v/QBWAud2fQO1KLpfDMgULS4sAbKxuUCkP4bXktXodj3zZod6sAVBf3ySNYXBwWN4LEZHP5ylX5OdOq0U7aDA6Ju/FxkqdFMHmxiYA6/VTuAUHy8jJMsYhUeBRLJTkvWr4TE3dztBwv7yX5TLT07M89UefCUA+N8DCzCnuPX4LAD/yY8/AMosIM5HnC1OiKGKgf1CWp+Pj+x3WVpcAGBkfpN2IcfPy3pyauRWvAY5bBGDfgSMEQYBty2frNT2ElVAZkOWxTJtKOU+jIetSHHV4xtP/x4N8gheGEyf+PB068bzTtp2pqfz3ng/AkSMHyBfzOJmmBLZr02p6AFiWgWmZNOvyszAM1g59iu/f8yMA/ARD+E/4/LaaGjn5AtpteXycpCRxSl9fHoC5hQWpqUmlqbLU1GC/rMNdTaWpvPWOY11QTU2dWsAgyTTVCeJMU/lcns3Dn840ZXzz6YSXf4XCrT8NwOzEzY84TVXvfjbJE78CQPH2n2F+14cviqa2bTSEgDSVhQj8kM31JrWNpix00aEZplz9lCfIiy6YpGlCHMuLTJMYv5NgqIfotTrk8i5pKr+3DZswCamWhwAoVEymT8wwWJ2QN7Uvx+rKOn5HvjiFAdWBKlGQqt/L0Wy3WFxekedveDwp/Q2W3U8AMDH7It57JEep/H4A/rH4Mm46GWLbJgCmpSrhg6zYaff/NCVOUt66R35+xTGftufx/stkpXntqRtp9/k0lOBJE2ZOzQPgFiwsI09
"text/plain": [
"<Figure size 504x576 with 6 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn.show_results(max_n=6, figsize=(7,8))"
]
},
{
"cell_type": "code",
"execution_count": null,
2020-03-17 19:15:55 +00:00
"metadata": {
"scrolled": true
},
2020-03-06 18:19:03 +00:00
"outputs": [
{
"data": {
"text/html": [],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.594912</td>\n",
" <td>0.407416</td>\n",
" <td>0.823640</td>\n",
" <td>01:35</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.268259</td>\n",
" <td>0.316242</td>\n",
" <td>0.876000</td>\n",
" <td>03:03</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
" <td>0.184861</td>\n",
" <td>0.246242</td>\n",
" <td>0.898080</td>\n",
" <td>03:10</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>0.136392</td>\n",
" <td>0.220086</td>\n",
" <td>0.918200</td>\n",
" <td>03:16</td>\n",
" </tr>\n",
" <tr>\n",
" <td>3</td>\n",
" <td>0.106423</td>\n",
" <td>0.191092</td>\n",
" <td>0.931360</td>\n",
" <td>03:15</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from fastai2.text.all import *\n",
"\n",
"dls = TextDataLoaders.from_folder(untar_data(URLs.IMDB), valid='test')\n",
"learn = text_classifier_learner(dls, AWD_LSTM, drop_mult=0.5, metrics=accuracy)\n",
"learn.fine_tune(4, 1e-2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"('pos', tensor(1), tensor([0.0041, 0.9959]))"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"learn.predict(\"I really liked that movie!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Sidebar: The order matter"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### End sidebar"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from fastai2.tabular.all import *\n",
"path = untar_data(URLs.ADULT_SAMPLE)\n",
"\n",
"dls = TabularDataLoaders.from_csv(path/'adult.csv', path, y_names=\"salary\",\n",
" cat_names = ['workclass', 'education', 'marital-status', 'occupation',\n",
" 'relationship', 'race'],\n",
" cont_names = ['age', 'fnlwgt', 'education-num'],\n",
" procs = [Categorify, FillMissing, Normalize])\n",
"\n",
"learn = tabular_learner(dls, metrics=accuracy)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.359960</td>\n",
" <td>0.357917</td>\n",
" <td>0.831388</td>\n",
" <td>00:11</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
" <td>0.353458</td>\n",
" <td>0.349657</td>\n",
" <td>0.837991</td>\n",
" <td>00:10</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>0.338368</td>\n",
" <td>0.346997</td>\n",
" <td>0.843213</td>\n",
" <td>00:10</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn.fit_one_cycle(3)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>1.554056</td>\n",
" <td>1.428071</td>\n",
" <td>00:01</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>1.393103</td>\n",
" <td>1.361342</td>\n",
" <td>00:01</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
" <td>1.297930</td>\n",
" <td>1.159169</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>1.052705</td>\n",
" <td>0.827934</td>\n",
" <td>00:01</td>\n",
" </tr>\n",
" <tr>\n",
" <td>3</td>\n",
" <td>0.810124</td>\n",
" <td>0.668735</td>\n",
" <td>00:01</td>\n",
" </tr>\n",
" <tr>\n",
" <td>4</td>\n",
" <td>0.711552</td>\n",
" <td>0.627836</td>\n",
" <td>00:01</td>\n",
" </tr>\n",
" <tr>\n",
" <td>5</td>\n",
" <td>0.657402</td>\n",
" <td>0.611715</td>\n",
" <td>00:01</td>\n",
" </tr>\n",
" <tr>\n",
" <td>6</td>\n",
" <td>0.633079</td>\n",
" <td>0.605733</td>\n",
" <td>00:01</td>\n",
" </tr>\n",
" <tr>\n",
" <td>7</td>\n",
" <td>0.622399</td>\n",
" <td>0.602674</td>\n",
" <td>00:01</td>\n",
" </tr>\n",
" <tr>\n",
" <td>8</td>\n",
" <td>0.629075</td>\n",
" <td>0.601671</td>\n",
" <td>00:00</td>\n",
" </tr>\n",
" <tr>\n",
" <td>9</td>\n",
" <td>0.619955</td>\n",
" <td>0.601550</td>\n",
" <td>00:01</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from fastai2.collab import *\n",
"path = untar_data(URLs.ML_SAMPLE)\n",
"dls = CollabDataLoaders.from_csv(path/'ratings.csv')\n",
"learn = collab_learner(dls, y_range=(0.5,5.5))\n",
"learn.fine_tune(10)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>userId</th>\n",
" <th>movieId</th>\n",
" <th>rating</th>\n",
" <th>rating_pred</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>157</td>\n",
" <td>1200</td>\n",
" <td>4.0</td>\n",
" <td>3.558502</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>23</td>\n",
" <td>344</td>\n",
" <td>2.0</td>\n",
" <td>2.700709</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>19</td>\n",
" <td>1221</td>\n",
" <td>5.0</td>\n",
" <td>4.390801</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>430</td>\n",
" <td>592</td>\n",
" <td>3.5</td>\n",
" <td>3.944848</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>547</td>\n",
" <td>858</td>\n",
" <td>4.0</td>\n",
" <td>4.076881</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>292</td>\n",
" <td>39</td>\n",
" <td>4.5</td>\n",
" <td>3.753513</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>529</td>\n",
" <td>1265</td>\n",
" <td>4.0</td>\n",
" <td>3.349463</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>19</td>\n",
" <td>231</td>\n",
" <td>3.0</td>\n",
" <td>2.881087</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>475</td>\n",
" <td>4963</td>\n",
" <td>4.0</td>\n",
" <td>4.023387</td>\n",
" </tr>\n",
" <tr>\n",
" <th>9</th>\n",
" <td>130</td>\n",
" <td>260</td>\n",
" <td>4.5</td>\n",
" <td>3.979703</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn.show_results()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Sidebar: Datasets: food for models"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### End sidebar"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Validation sets and test sets"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Use judgment in defining test sets"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## A _Choose Your Own Adventure_ moment"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Questionnaire"
]
},
2020-03-18 00:34:07 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It can be hard to know in pages and pages of prose what are the key things you really need to focus on and remember. So we've prepared a list of questions and suggested steps to complete at the end of each chapter. All the answers are in the text of the chapter, so if you're not sure about anything here, re-read that part of the text and make sure you understand it. Answers to all these questions are also available on the [book website](https://book.fast.ai). You can also visit [the forums](https://forums.fast.ai) if you get stuck to get help from other folks studying this material."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Do you need these for deep learning?\n",
" - Lots of math T / F\n",
" - Lots of data T / F\n",
" - Lots of expensive computers T / F\n",
" - A PhD T / F\n",
"1. Name five areas where deep learning is now the best in the world.\n",
"1. What was the name of the first device that was based on the principle of the artificial neuron?\n",
"1. Based on the book of the same name, what are the requirements for \"Parallel Distributed Processing\"?\n",
"1. What were the two theoretical misunderstandings that held back the field of neural networks?\n",
"1. What is a GPU?\n",
"1. Open a notebook and execute a cell containing: `1+1`. What happens?\n",
"1. Follow through each cell of the stripped version of the notebook for this chapter. Before executing each cell, guess what will happen.\n",
"1. Complete the Jupyter Notebook online appendix.\n",
"1. Why is it hard to use a traditional computer program to recognize images in a photo?\n",
"1. What did Samuel mean by \"Weight Assignment\"?\n",
"1. What term do we normally use in deep learning for what Samuel called \"Weights\"?\n",
"1. Draw a picture that summarizes Arthur Samuel's view of a machine learning model\n",
"1. Why is it hard to understand why a deep learning model makes a particular prediction?\n",
"1. What is the name of the theorem that a neural network can solve any mathematical problem to any level of accuracy?\n",
"1. What do you need in order to train a model?\n",
"1. How could a feedback loop impact the rollout of a predictive policing model?\n",
"1. Do we always have to use 224x224 pixel images with the cat recognition model?\n",
"1. What is the difference between classification and regression?\n",
"1. What is a validation set? What is a test set? Why do we need them?\n",
"1. What will fastai do if you don't provide a validation set?\n",
"1. Can we always use a random sample for a validation set? Why or why not?\n",
"1. What is overfitting? Provide an example.\n",
"1. What is a metric? How does it differ to \"loss\"?\n",
"1. How can pretrained models help?\n",
"1. What is the \"head\" of a model?\n",
"1. What kinds of features do the early layers of a CNN find? How about the later layers?\n",
"1. Are image models only useful for photos?\n",
"1. What is an \"architecture\"?\n",
"1. What is segmentation?\n",
"1. What is `y_range` used for? When do we need it?\n",
"1. What are \"hyperparameters\"?\n",
"1. What's the best way to avoid failures when using AI in an organization?"
]
},
2020-03-06 18:19:03 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Further research"
]
2020-03-18 00:34:07 +00:00
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Each chapter also has a \"further research\" with questions that aren't fully answered in the text, or include more advanced assignments. Answers to these questions aren't on the book website--you'll need to do your own research!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Why is a GPU useful for deep learning? How is a CPU different, and why is it less effective for deep learning?\n",
"1. Try to think of three areas where feedback loops might impact use of machine learning. See if you can find documented examples of that happening in practice."
]
2020-03-06 18:19:03 +00:00
}
],
"metadata": {
"jupytext": {
"split_at_heading": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
