chapter4: compared standard SGD with Adam Optimizer

Browse Source

Made a little graph to compare plain SGD (lr=2e-5) with plain Adam optimizer, on a very simple quaddratic function with 3 parameters. It's like a "guess the 3 numbers I'm thinking of" game.

Adam was a good improvement, but I wonder if there isn't some way to do much better.

...

This commit is contained in:

Jim Slattery 2024-03-19 00:07:49 -04:00

parent 2c5db23a34

commit f54965d266

1 changed files with 1887 additions and 1254 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2805

04_mnist_basics.ipynb

View File

File diff suppressed because one or more lines are too long