[
https://issues.apache.org/jira/browse/MADLIB-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Frank McQuillan updated MADLIB-1335:
------------------------------------
Description:
JIRA: https://issues.apache.org/jira/browse/MADLIB-1335
Context
Getting per iteration loss and other metrics for training and validation data
sets can be expensive. This parameter is intended to give control to user on
how often to do this computation.
Story
As a data scientist, I want to specify how often to calculate loss and other
metrics for training and validation data (if provided), i.e, every n iterations
Interface
{code}
madlib_keras_fit(
source_table VARCHAR,
model VARCHAR,
dependent_varname VARCHAR,
independent_varname VARCHAR,
model_arch_table VARCHAR,
model_arch_id INTEGER,
compile_params VARCHAR,
fit_params VARCHAR,
num_iterations INTEGER,
use_gpu BOOLEAN,
validation_table VARCHAR,
metrics_compute_frequency INTEGER <-------- NEW OPTIONAL PARAM
name VARCHAR,
description VARCHAR
{code}
where `metrics_compute_frequency` is an optional parameter that means:
{code}
NULL - calculate loss and metrics only on final model after last iteration
(default)
n - calculate loss metrics every n-th iteration and on final model after last
iteration
{code}
and `metrics_compute_frequency must be >=1 and <=num_iterations`
Acceptance
1) Set `num_iterations = 12` and leave ` metrics_compute_frequency` as default
and get 1 loss and metrics value at end after 12 iterations
2) Set `num_iterations = 12` and set `metrics_compute_frequency = 5` and get
loss and metrics after 5th and 10th iterations and at end after 12 iterations
3) Set `metrics_compute_frequency = 0` and get an error
4) Set `metrics_compute_frequency = num_iterations+1` and get an error
5) Test with different metrics: from https://keras.io/metrics/ try 'mae' ,
'acc' etc.
Reference
[1] https://keras.io/metrics/
was:
As a data scientist, I want to specify how often to calculate loss/accuracy for
training and validation data (if provided), i.e, every n iterations
Interface
{code}
madlib_keras_fit(
source_table VARCHAR,
model VARCHAR,
dependent_varname VARCHAR,
independent_varname VARCHAR,
model_arch_table VARCHAR,
model_arch_id INTEGER,
compile_params VARCHAR,
fit_params VARCHAR,
num_iterations INTEGER,
use_gpu BOOLEAN,
validation_table VARCHAR,
loss_compute_frequency INTEGER <-------- NEW OPTIONAL PARAM
name VARCHAR,
description VARCHAR
{code}
where {{loss_compute_frequency}} is an optional parameter that means:
{code}
NULL - calculate loss/frequency only on final model after last iteration
(default)
n - calculate loss/frequency every n-th iteration and on final model after last
iteration
{code}
and {{loss_compute_frequency must be >=1 and <=num_iterations}}
Test cases to consider:
1) Set {{num_iterations = 12}} and leave {{loss_compute_frequency}} as default
and get 1 loss/accuracy value at end after 12 iterations
2) Set {{num_iterations = 12}} and set {{loss_compute_frequency = 5}} and get
loss/accuracy after 5th and 10th iterations and at end after 12 iterations
3) Set {{num_iterations = 12}} and set {{loss_compute_frequency = 99}} and get
1 loss/accuracy value at end after 12 iterations
4) Set {{loss_compute_frequency = 0}} and get an error
5) Set {{loss_compute_frequency = num_iterations+1}} and get an error
> Add new param loss_compute_frequency to madlib_keras_fit()
> ----------------------------------------------------------
>
> Key: MADLIB-1335
> URL: https://issues.apache.org/jira/browse/MADLIB-1335
> Project: Apache MADlib
> Issue Type: New Feature
> Components: Deep Learning
> Reporter: Ekta Khanna
> Priority: Major
> Fix For: v1.16
>
>
> JIRA: https://issues.apache.org/jira/browse/MADLIB-1335
> Context
> Getting per iteration loss and other metrics for training and validation data
> sets can be expensive. This parameter is intended to give control to user on
> how often to do this computation.
> Story
> As a data scientist, I want to specify how often to calculate loss and other
> metrics for training and validation data (if provided), i.e, every n
> iterations
> Interface
> {code}
> madlib_keras_fit(
> source_table VARCHAR,
> model VARCHAR,
> dependent_varname VARCHAR,
> independent_varname VARCHAR,
> model_arch_table VARCHAR,
> model_arch_id INTEGER,
> compile_params VARCHAR,
> fit_params VARCHAR,
> num_iterations INTEGER,
> use_gpu BOOLEAN,
> validation_table VARCHAR,
> metrics_compute_frequency INTEGER <-------- NEW OPTIONAL PARAM
> name VARCHAR,
> description VARCHAR
> {code}
> where `metrics_compute_frequency` is an optional parameter that means:
> {code}
> NULL - calculate loss and metrics only on final model after last iteration
> (default)
> n - calculate loss metrics every n-th iteration and on final model after last
> iteration
> {code}
> and `metrics_compute_frequency must be >=1 and <=num_iterations`
> Acceptance
> 1) Set `num_iterations = 12` and leave ` metrics_compute_frequency` as
> default and get 1 loss and metrics value at end after 12 iterations
> 2) Set `num_iterations = 12` and set `metrics_compute_frequency = 5` and get
> loss and metrics after 5th and 10th iterations and at end after 12 iterations
> 3) Set `metrics_compute_frequency = 0` and get an error
> 4) Set `metrics_compute_frequency = num_iterations+1` and get an error
> 5) Test with different metrics: from https://keras.io/metrics/ try 'mae' ,
> 'acc' etc.
> Reference
> [1] https://keras.io/metrics/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
