[
https://issues.apache.org/jira/browse/MADLIB-1462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikhil Kak updated MADLIB-1462:
-------------------------------
Fix Version/s: v1.18.0
> DL - Fit multiple does not print timing for validation evaluate
> ---------------------------------------------------------------
>
> Key: MADLIB-1462
> URL: https://issues.apache.org/jira/browse/MADLIB-1462
> Project: Apache MADlib
> Issue Type: Improvement
> Components: Deep Learning
> Reporter: Nikhil Kak
> Priority: Major
> Fix For: v1.18.0
>
>
> Currently when running fit_multiple with validation dataset, we don't print
> the timing for the validation runs
> {code}
> select madlib.madlib_keras_fit_multiple_model('cifar10_train_batched',
> 'cifar10_out', 'cifar10_mst_table', 100, TRUE, 'cifar10_train_batched', 1);
> INFO:
> Time for training in iteration 1: 33.6217501163 sec
> DETAIL:
> Training set after iteration 1:
> mst_key=12: metric=0.260340005159, loss=2.13081121445
> ...
> mst_key=2: metric=0.164859995246, loss=2.25495767593
> Validation set after iteration 1:
> mst_key=12: metric=0.260340005159, loss=2.13081121445
> ...
> mst_key=2: metric=0.164859995246, loss=2.25495767593
> CONTEXT: PL/Python function "madlib_keras_fit_multiple_model"
> INFO:
> Time for training in iteration 2: 24.7699511051 sec
> DETAIL:
> ....
> {code}
> We should print the time it took to run validation evaluate for both training
> and validation dataset
>
> If the user specifies only the training dataset, then we should add the
> following to the existing output
> 1. The cumulative time it took for all the msts to run eval for the training
> dataset for that iteration
> {code}
> select
> madlib.madlib_keras_fit_multiple_model('iris_data_packed','iris_multiple_model','mst_table_4row',2,
> FALSE,NULL,1);
> INFO:
> Time for training in iteration 1: 2.24381709099 sec
> DETAIL:
> Training set after iteration 1:
> mst_key=2: metric=0.333333343267, loss=1.33550834656
> mst_key=1: metric=0.333333343267, loss=1.12043237686
> mst_key=4: metric=0.333333343267, loss=3.90859818459
> mst_key=3: metric=0.333333343267, loss=4.37875080109
> Time for evaluating training dataset in iteration 1: 0.652065515518
> CONTEXT: PL/Python function "madlib_keras_fit_multiple_model"
> INFO:
> Time for training in iteration 2: 2.32056617737 sec
> DETAIL:
> Training set after iteration 2:
> mst_key=2: metric=0.666666686535, loss=1.14192306995
> mst_key=1: metric=0.666666686535, loss=0.917088747025
> mst_key=4: metric=0.340000003576, loss=2.98958563805
> mst_key=3: metric=0.333333343267, loss=3.86314368248
> Time for evaluating training dataset in iteration 2: 0.679529428482
> {code}
> If the user specifies a validation dataset, then we should add the following
> to the existing output
> 1. The cumulative time it took for all the msts to run eval for the training
> dataset for that iteration
> 1. The cumulative time it took for all the msts to run eval for the
> validation dataset for that iteration
> {code}
> select
> madlib.madlib_keras_fit_multiple_model('iris_data_packed','iris_multiple_model','mst_table_4row',2,
> FALSE,'iris_data_packed',1);
> INFO:
> Time for training in iteration 1: 4.27021813393 sec
> DETAIL:
> Training set after iteration 1:
> mst_key=2: metric=0.333333343267, loss=1.39633440971
> mst_key=1: metric=0.333333343267, loss=1.04632723331
> mst_key=4: metric=0.333333343267, loss=3.96611213684
> mst_key=3: metric=0.333333343267, loss=4.38052940369
> Time for evaluating training dataset in iteration 1: 0.649274587631
> Validation set after iteration 1:
> mst_key=2: metric=0.333333343267, loss=1.39633440971
> mst_key=1: metric=0.333333343267, loss=1.04632723331
> mst_key=4: metric=0.333333343267, loss=3.96611213684
> mst_key=3: metric=0.333333343267, loss=4.38052940369
> Time for evaluating validation dataset in iteration 1: 0.75797867775
> CONTEXT: PL/Python function "madlib_keras_fit_multiple_model"
> INFO:
> Time for training in iteration 2: 2.1767308712 sec
> DETAIL:
> Training set after iteration 2:
> mst_key=2: metric=0.666666686535, loss=1.10426521301
> mst_key=1: metric=0.666666686535, loss=1.02108848095
> mst_key=4: metric=0.333333343267, loss=3.10222005844
> mst_key=3: metric=0.333333343267, loss=3.85620188713
> Time for evaluating training dataset in iteration 2: 0.784633874893
> Validation set after iteration 2:
> mst_key=2: metric=0.666666686535, loss=1.10426521301
> mst_key=1: metric=0.666666686535, loss=1.02108848095
> mst_key=4: metric=0.333333343267, loss=3.10222005844
> mst_key=3: metric=0.333333343267, loss=3.85620188713
> Time for evaluating validation dataset in iteration 2: 0.639858007431
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
