fmcquillan99 commented on pull request #518:
URL: https://github.com/apache/madlib/pull/518#issuecomment-699088843
(5)
In this test the 2nd batch is worse than the 1st batch:
```
madlib=# SELECT madlib.madlib_keras_automl('cifar10_train_packed',
madlib(# 'automl_cifar10_output',
madlib(# 'model_arch_library',
madlib(# 'automl_cifar10_mst_table',
madlib(# ARRAY[1,2],
madlib(# $${'loss':
['categorical_crossentropy'],
madlib$# 'optimizer_params_list': [
madlib$# {'optimizer':
['Adam'],'lr': [0.0005, 0.01, 'log']},
madlib$# {'optimizer':
['SGD'],'lr': [0.001, 0.01, 'log'], 'momentum': [0.90, 0.91,'log']}],
madlib$# 'metrics':['accuracy']}$$,
madlib(# $${'batch_size': [64,128],
'epochs': [1]}$$,
madlib(# 'hyperopt',
madlib(# 'num_models=10, num_iters=2,
algorithm=tpe',
madlib(# NULL, --
random state
madlib(# NULL, -- object
table
madlib(# FALSE, -- use GPUs
madlib(# 'cifar10_val_packed', --
validation table
madlib(# 1, -- metrics
compute freq
madlib(# NULL, -- name
madlib(# NULL); -- descr
INFO:
Time for training in iteration 1: 1070.08548617 sec
DETAIL:
Training set after iteration 1:
mst_key=1: metric=0.526019990444, loss=1.5796225071
mst_key=5: metric=0.562780022621, loss=1.37457168102
mst_key=4: metric=0.65314000845, loss=0.969262957573
mst_key=2: metric=0.10000000149, loss=2.30277729034
mst_key=3: metric=0.464760005474, loss=1.50251471996
Validation set after iteration 1:
mst_key=1: metric=0.523299992085, loss=1.60135388374
mst_key=5: metric=0.559000015259, loss=1.40086007118
mst_key=4: metric=0.640600025654, loss=1.00141870975
mst_key=2: metric=0.10000000149, loss=2.30277705193
mst_key=3: metric=0.463400006294, loss=1.50355136395
CONTEXT: PL/Python function "madlib_keras_automl"
INFO:
Time for training in iteration 2: 1064.05229688 sec
DETAIL:
Training set after iteration 2:
mst_key=1: metric=0.703299999237, loss=0.840497553349
mst_key=5: metric=0.664420008659, loss=0.98249232769
mst_key=4: metric=0.754859983921, loss=0.701396882534
mst_key=2: metric=0.10000000149, loss=2.30290770531
mst_key=3: metric=0.535520017147, loss=1.26898431778
Validation set after iteration 2:
mst_key=1: metric=0.68900001049, loss=0.903733193874
mst_key=5: metric=0.653800010681, loss=1.02747094631
mst_key=4: metric=0.735899984837, loss=0.765290260315
mst_key=2: metric=0.10000000149, loss=2.30290770531
mst_key=3: metric=0.530799984932, loss=1.28327465057
CONTEXT: PL/Python function "madlib_keras_automl"
INFO:
Best training loss so far:
DETAIL:
mst_key=4: metric=0.754859983921, loss=0.701396882534
Best validation loss so far:
mst_key=4: metric=0.735899984837, loss=0.765290260315
INFO: ***Evaluating 5 newly suggested model configurations***
CONTEXT: PL/Python function "madlib_keras_automl"
INFO:
Time for training in iteration 1: 1058.9834702 sec
DETAIL:
Training set after iteration 1:
mst_key=6: metric=0.553560018539, loss=1.41088056564
mst_key=10: metric=0.353260010481, loss=1.84440767765
mst_key=9: metric=0.519379973412, loss=1.35283255577
mst_key=7: metric=0.517820000648, loss=1.31637942791
mst_key=8: metric=0.51289999485, loss=1.32810485363
Validation set after iteration 1:
mst_key=6: metric=0.547699987888, loss=1.44565415382
mst_key=10: metric=0.358599990606, loss=1.83988237381
mst_key=9: metric=0.518999993801, loss=1.36131703854
mst_key=7: metric=0.515799999237, loss=1.32058382034
mst_key=8: metric=0.507099986076, loss=1.33919155598
CONTEXT: PL/Python function "madlib_keras_automl"
INFO:
Time for training in iteration 2: 1072.08968186 sec
DETAIL:
Training set after iteration 2:
mst_key=6: metric=0.435039997101, loss=1.98741734028
mst_key=10: metric=0.437579989433, loss=1.54891061783
mst_key=9: metric=0.606719970703, loss=1.1007784605
mst_key=7: metric=0.550220012665, loss=1.23340415955
mst_key=8: metric=0.590960025787, loss=1.13847112656
Validation set after iteration 2:
mst_key=6: metric=0.424100011587, loss=2.0407743454
mst_key=10: metric=0.438899993896, loss=1.54700648785
mst_key=9: metric=0.595899999142, loss=1.13602638245
mst_key=7: metric=0.544099986553, loss=1.24483549595
mst_key=8: metric=0.583800017834, loss=1.15895962715
CONTEXT: PL/Python function "madlib_keras_automl"
INFO:
Best training loss so far:
DETAIL:
mst_key=4: metric=0.754859983921, loss=0.701396882534
Best validation loss so far:
mst_key=4: metric=0.735899984837, loss=0.765290260315
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
