(StreetFighterAI) PS C:\Users\unitec\Documents\AIProjects\street-fighter-ai\001_image_stack> python .\train.py Using cuda device Current state: ChampionX.Level4.ChunLiVsDhalsim ------------------------------ | time/ | | | fps | 1489 | | iterations | 1 | | time_elapsed | 28 | | total_timesteps | 43200 | ------------------------------ Current state: ChampionX.Level6.ChunLiVsEHonda ----------------------------------------- | time/ | | | fps | 693 | | iterations | 2 | | time_elapsed | 124 | | total_timesteps | 86400 | | train/ | | | approx_kl | 0.008018286 | | clip_fraction | 0.0528 | | clip_range | 0.2 | | entropy_loss | -8.31 | | explained_variance | -0.000782 | | learning_rate | 0.0003 | | loss | 189 | | n_updates | 10 | | policy_gradient_loss | -0.00354 | | value_loss | 398 | ----------------------------------------- Current state: ChampionX.Level1.ChunLiVsKen ----------------------------------------- | time/ | | | fps | 476 | | iterations | 3 | | time_elapsed | 271 | | total_timesteps | 129600 | | train/ | | | approx_kl | 0.010610209 | | clip_fraction | 0.119 | | clip_range | 0.2 | | entropy_loss | -8.3 | | explained_variance | 0.0323 | | learning_rate | 0.0003 | | loss | 0.228 | | n_updates | 20 | | policy_gradient_loss | -0.00663 | | value_loss | 103 | ----------------------------------------- Current state: ChampionX.Level10.ChunLiVsVega ----------------------------------------- | time/ | | | fps | 467 | | iterations | 4 | | time_elapsed | 369 | | total_timesteps | 172800 | | train/ | | | approx_kl | 0.011115557 | | clip_fraction | 0.122 | | clip_range | 0.2 | | entropy_loss | -8.3 | | explained_variance | 0.125 | | learning_rate | 0.0003 | | loss | 5.37 | | n_updates | 30 | | policy_gradient_loss | -0.00485 | | value_loss | 83.8 | ----------------------------------------- Current state: ChampionX.Level11.ChunLiVsSagat ----------------------------------------- | time/ | | | fps | 462 | | iterations | 5 | | time_elapsed | 466 | | total_timesteps | 216000 | | train/ | | | approx_kl | 0.012769428 | | clip_fraction | 0.133 | | clip_range | 0.2 | | entropy_loss | -8.29 | | explained_variance | 0.409 | | learning_rate | 0.0003 | | loss | 18.4 | | n_updates | 40 | | policy_gradient_loss | -0.00746 | | value_loss | 31.6 | ----------------------------------------- Current state: ChampionX.Level7.ChunLiVsBlanka ----------------------------------------- | time/ | | | fps | 460 | | iterations | 6 | | time_elapsed | 563 | | total_timesteps | 259200 | | train/ | | | approx_kl | 0.014561476 | | clip_fraction | 0.184 | | clip_range | 0.2 | | entropy_loss | -8.29 | | explained_variance | 0.15 | | learning_rate | 0.0003 | | loss | 0.66 | | n_updates | 50 | | policy_gradient_loss | -0.00799 | | value_loss | 23.2 | ----------------------------------------- Current state: ChampionX.Level12.ChunLiVsBison ----------------------------------------- | time/ | | | fps | 447 | | iterations | 7 | | time_elapsed | 675 | | total_timesteps | 302400 | | train/ | | | approx_kl | 0.013581872 | | clip_fraction | 0.147 | | clip_range | 0.2 | | entropy_loss | -8.28 | | explained_variance | 0.447 | | learning_rate | 0.0003 | | loss | 0.794 | | n_updates | 60 | | policy_gradient_loss | -0.00405 | | value_loss | 33.4 | ----------------------------------------- Current state: ChampionX.Level4.ChunLiVsDhalsim ----------------------------------------- | time/ | | | fps | 440 | | iterations | 8 | | time_elapsed | 784 | | total_timesteps | 345600 | | train/ | | | approx_kl | 0.015053411 | | clip_fraction | 0.186 | | clip_range | 0.2 | | entropy_loss | -8.28 | | explained_variance | 0.39 | | learning_rate | 0.0003 | | loss | 0.313 | | n_updates | 70 | | policy_gradient_loss | -0.00594 | | value_loss | 22.3 | ----------------------------------------- Current state: ChampionX.Level5.ChunLiVsRyu ---------------------------------------- | time/ | | | fps | 433 | | iterations | 9 | | time_elapsed | 896 | | total_timesteps | 388800 | | train/ | | | approx_kl | 0.01407744 | | clip_fraction | 0.152 | | clip_range | 0.2 | | entropy_loss | -8.27 | | explained_variance | 0.326 | | learning_rate | 0.0003 | | loss | 0.396 | | n_updates | 80 | | policy_gradient_loss | -0.00862 | | value_loss | 15.7 | ---------------------------------------- Current state: ChampionX.Level9.ChunLiVsBalrog ----------------------------------------- | time/ | | | fps | 428 | | iterations | 10 | | time_elapsed | 1007 | | total_timesteps | 432000 | | train/ | | | approx_kl | 0.013460734 | | clip_fraction | 0.148 | | clip_range | 0.2 | | entropy_loss | -8.27 | | explained_variance | 0.384 | | learning_rate | 0.0003 | | loss | 0.227 | | n_updates | 90 | | policy_gradient_loss | -0.00498 | | value_loss | 16.7 | ----------------------------------------- Current state: ChampionX.Level9.ChunLiVsBalrog ------------------------------------------ | time/ | | | fps | 424 | | iterations | 11 | | time_elapsed | 1118 | | total_timesteps | 475200 | | train/ | | | approx_kl | 0.0145851895 | | clip_fraction | 0.165 | | clip_range | 0.2 | | entropy_loss | -8.26 | | explained_variance | 0.352 | | learning_rate | 0.0003 | | loss | 0.147 | | n_updates | 100 | | policy_gradient_loss | -0.00597 | | value_loss | 19.8 | ------------------------------------------ Current state: ChampionX.Level4.ChunLiVsDhalsim ----------------------------------------- | time/ | | | fps | 424 | | iterations | 12 | | time_elapsed | 1219 | | total_timesteps | 518400 | | train/ | | | approx_kl | 0.015144574 | | clip_fraction | 0.161 | | clip_range | 0.2 | | entropy_loss | -8.25 | | explained_variance | 0.383 | | learning_rate | 0.0003 | | loss | 1.52 | | n_updates | 110 | | policy_gradient_loss | -0.00749 | | value_loss | 24.1 | ----------------------------------------- Current state: ChampionX.Level6.ChunLiVsEHonda ----------------------------------------- | time/ | | | fps | 425 | | iterations | 13 | | time_elapsed | 1318 | | total_timesteps | 561600 | | train/ | | | approx_kl | 0.015790801 | | clip_fraction | 0.158 | | clip_range | 0.2 | | entropy_loss | -8.25 | | explained_variance | 0.555 | | learning_rate | 0.0003 | | loss | 0.665 | | n_updates | 120 | | policy_gradient_loss | -0.00889 | | value_loss | 20.7 | ----------------------------------------- Current state: ChampionX.Level6.ChunLiVsEHonda ----------------------------------------- | time/ | | | fps | 426 | | iterations | 14 | | time_elapsed | 1417 | | total_timesteps | 604800 | | train/ | | | approx_kl | 0.016785465 | | clip_fraction | 0.173 | | clip_range | 0.2 | | entropy_loss | -8.24 | | explained_variance | 0.609 | | learning_rate | 0.0003 | | loss | 0.313 | | n_updates | 130 | | policy_gradient_loss | -0.00758 | | value_loss | 14.9 | ----------------------------------------- Current state: ChampionX.Level10.ChunLiVsVega ----------------------------------------- | time/ | | | fps | 428 | | iterations | 15 | | time_elapsed | 1513 | | total_timesteps | 648000 | | train/ | | | approx_kl | 0.017042443 | | clip_fraction | 0.176 | | clip_range | 0.2 | | entropy_loss | -8.24 | | explained_variance | 0.759 | | learning_rate | 0.0003 | | loss | 0.634 | | n_updates | 140 | | policy_gradient_loss | -0.00617 | | value_loss | 15.4 | ----------------------------------------- Current state: ChampionX.Level3.ChunLiVsZangief ----------------------------------------- | time/ | | | fps | 427 | | iterations | 16 | | time_elapsed | 1618 | | total_timesteps | 691200 | | train/ | | | approx_kl | 0.017714709 | | clip_fraction | 0.176 | | clip_range | 0.2 | | entropy_loss | -8.23 | | explained_variance | 0.79 | | learning_rate | 0.0003 | | loss | 0.941 | | n_updates | 150 | | policy_gradient_loss | -0.00703 | | value_loss | 17.5 | ----------------------------------------- Current state: ChampionX.Level7.ChunLiVsBlanka ----------------------------------------- | time/ | | | fps | 424 | | iterations | 17 | | time_elapsed | 1728 | | total_timesteps | 734400 | | train/ | | | approx_kl | 0.018709755 | | clip_fraction | 0.196 | | clip_range | 0.2 | | entropy_loss | -8.22 | | explained_variance | 0.746 | | learning_rate | 0.0003 | | loss | 0.505 | | n_updates | 160 | | policy_gradient_loss | -0.00795 | | value_loss | 11.9 | ----------------------------------------- Current state: ChampionX.Level11.ChunLiVsSagat ----------------------------------------- | time/ | | | fps | 423 | | iterations | 18 | | time_elapsed | 1837 | | total_timesteps | 777600 | | train/ | | | approx_kl | 0.017850244 | | clip_fraction | 0.182 | | clip_range | 0.2 | | entropy_loss | -8.21 | | explained_variance | 0.687 | | learning_rate | 0.0003 | | loss | 0.379 | | n_updates | 170 | | policy_gradient_loss | -0.00568 | | value_loss | 15.2 | ----------------------------------------- Current state: ChampionX.Level3.ChunLiVsZangief ---------------------------------------- | time/ | | | fps | 420 | | iterations | 19 | | time_elapsed | 1950 | | total_timesteps | 820800 | | train/ | | | approx_kl | 0.02048213 | | clip_fraction | 0.221 | | clip_range | 0.2 | | entropy_loss | -8.21 | | explained_variance | 0.707 | | learning_rate | 0.0003 | | loss | 0.391 | | n_updates | 180 | | policy_gradient_loss | -0.00419 | | value_loss | 13 | ---------------------------------------- Current state: ChampionX.Level11.ChunLiVsSagat ----------------------------------------- | time/ | | | fps | 418 | | iterations | 20 | | time_elapsed | 2062 | | total_timesteps | 864000 | | train/ | | | approx_kl | 0.016617421 | | clip_fraction | 0.172 | | clip_range | 0.2 | | entropy_loss | -8.2 | | explained_variance | 0.744 | | learning_rate | 0.0003 | | loss | 1.66 | | n_updates | 190 | | policy_gradient_loss | -0.00437 | | value_loss | 15.8 | ----------------------------------------- Current state: ChampionX.Level6.ChunLiVsEHonda ----------------------------------------- | time/ | | | fps | 417 | | iterations | 21 | | time_elapsed | 2174 | | total_timesteps | 907200 | | train/ | | | approx_kl | 0.017259926 | | clip_fraction | 0.171 | | clip_range | 0.2 | | entropy_loss | -8.19 | | explained_variance | 0.756 | | learning_rate | 0.0003 | | loss | 0.457 | | n_updates | 200 | | policy_gradient_loss | -0.00897 | | value_loss | 14.9 | ----------------------------------------- Current state: ChampionX.Level10.ChunLiVsVega ----------------------------------------- | time/ | | | fps | 417 | | iterations | 22 | | time_elapsed | 2276 | | total_timesteps | 950400 | | train/ | | | approx_kl | 0.018794816 | | clip_fraction | 0.19 | | clip_range | 0.2 | | entropy_loss | -8.18 | | explained_variance | 0.834 | | learning_rate | 0.0003 | | loss | 3.33 | | n_updates | 210 | | policy_gradient_loss | -0.00535 | | value_loss | 15.5 | ----------------------------------------- Current state: ChampionX.Level5.ChunLiVsRyu ----------------------------------------- | time/ | | | fps | 418 | | iterations | 23 | | time_elapsed | 2374 | | total_timesteps | 993600 | | train/ | | | approx_kl | 0.019361915 | | clip_fraction | 0.188 | | clip_range | 0.2 | | entropy_loss | -8.18 | | explained_variance | 0.825 | | learning_rate | 0.0003 | | loss | 0.235 | | n_updates | 220 | | policy_gradient_loss | -0.00762 | | value_loss | 13.4 | ----------------------------------------- Current state: ChampionX.Level2.ChunLiVsChunLi ----------------------------------------- | time/ | | | fps | 419 | | iterations | 24 | | time_elapsed | 2471 | | total_timesteps | 1036800 | | train/ | | | approx_kl | 0.022115083 | | clip_fraction | 0.233 | | clip_range | 0.2 | | entropy_loss | -8.18 | | explained_variance | 0.8 | | learning_rate | 0.0003 | | loss | 0.211 | | n_updates | 230 | | policy_gradient_loss | -0.00771 | | value_loss | 11.7 | ----------------------------------------- Current state: ChampionX.Level3.ChunLiVsZangief ----------------------------------------- | time/ | | | fps | 419 | | iterations | 25 | | time_elapsed | 2574 | | total_timesteps | 1080000 | | train/ | | | approx_kl | 0.023090197 | | clip_fraction | 0.233 | | clip_range | 0.2 | | entropy_loss | -8.17 | | explained_variance | 0.759 | | learning_rate | 0.0003 | | loss | 0.445 | | n_updates | 240 | | policy_gradient_loss | -0.00523 | | value_loss | 13.7 | ----------------------------------------- Current state: ChampionX.Level11.ChunLiVsSagat ----------------------------------------- | time/ | | | fps | 418 | | iterations | 26 | | time_elapsed | 2683 | | total_timesteps | 1123200 | | train/ | | | approx_kl | 0.024867734 | | clip_fraction | 0.281 | | clip_range | 0.2 | | entropy_loss | -8.16 | | explained_variance | 0.688 | | learning_rate | 0.0003 | | loss | 0.557 | | n_updates | 250 | | policy_gradient_loss | 0.00215 | | value_loss | 13.9 | ----------------------------------------- Current state: ChampionX.Level10.ChunLiVsVega ----------------------------------------- | time/ | | | fps | 417 | | iterations | 27 | | time_elapsed | 2793 | | total_timesteps | 1166400 | | train/ | | | approx_kl | 0.020454599 | | clip_fraction | 0.203 | | clip_range | 0.2 | | entropy_loss | -8.19 | | explained_variance | 0.766 | | learning_rate | 0.0003 | | loss | 0.314 | | n_updates | 260 | | policy_gradient_loss | -0.0058 | | value_loss | 17.1 | ----------------------------------------- Current state: ChampionX.Level10.ChunLiVsVega ----------------------------------------- | time/ | | | fps | 416 | | iterations | 28 | | time_elapsed | 2904 | | total_timesteps | 1209600 | | train/ | | | approx_kl | 0.020690009 | | clip_fraction | 0.208 | | clip_range | 0.2 | | entropy_loss | -8.17 | | explained_variance | 0.827 | | learning_rate | 0.0003 | | loss | 1.38 | | n_updates | 270 | | policy_gradient_loss | 2.12e-05 | | value_loss | 20.4 | ----------------------------------------- Current state: ChampionX.Level10.ChunLiVsVega ----------------------------------------- | time/ | | | fps | 415 | | iterations | 29 | | time_elapsed | 3015 | | total_timesteps | 1252800 | | train/ | | | approx_kl | 0.020646438 | | clip_fraction | 0.208 | | clip_range | 0.2 | | entropy_loss | -8.15 | | explained_variance | 0.81 | | learning_rate | 0.0003 | | loss | 0.235 | | n_updates | 280 | | policy_gradient_loss | -0.00852 | | value_loss | 12.5 | ----------------------------------------- Current state: ChampionX.Level9.ChunLiVsBalrog ----------------------------------------- | time/ | | | fps | 414 | | iterations | 30 | | time_elapsed | 3128 | | total_timesteps | 1296000 | | train/ | | | approx_kl | 0.021910097 | | clip_fraction | 0.212 | | clip_range | 0.2 | | entropy_loss | -8.15 | | explained_variance | 0.79 | | learning_rate | 0.0003 | | loss | 0.0264 | | n_updates | 290 | | policy_gradient_loss | -0.00872 | | value_loss | 12.5 | ----------------------------------------- Current state: ChampionX.Level12.ChunLiVsBison ----------------------------------------- | time/ | | | fps | 412 | | iterations | 31 | | time_elapsed | 3243 | | total_timesteps | 1339200 | | train/ | | | approx_kl | 0.025281599 | | clip_fraction | 0.254 | | clip_range | 0.2 | | entropy_loss | -8.13 | | explained_variance | 0.773 | | learning_rate | 0.0003 | | loss | 1.18 | | n_updates | 300 | | policy_gradient_loss | -0.00679 | | value_loss | 12.6 | ----------------------------------------- Current state: ChampionX.Level2.ChunLiVsChunLi ----------------------------------------- | time/ | | | fps | 412 | | iterations | 32 | | time_elapsed | 3349 | | total_timesteps | 1382400 | | train/ | | | approx_kl | 0.026466375 | | clip_fraction | 0.259 | | clip_range | 0.2 | | entropy_loss | -8.12 | | explained_variance | 0.647 | | learning_rate | 0.0003 | | loss | 0.518 | | n_updates | 310 | | policy_gradient_loss | 0.000522 | | value_loss | 18.8 | -----------------------------------------