[
https://issues.apache.org/jira/browse/MADLIB-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16380832#comment-16380832
]
Nandish Jayaram commented on MADLIB-1206:
-----------------------------------------
Proposed workflow, please comment for improving/changing it:
The input table must be pre-processed using
https://issues.apache.org/jira/browse/MADLIB-1200. The output of the
pre-processing step is the input to MLP for mini-batch.
{code}
mlp_classification(
source_table, -- for minibatch use output_table from preprocessor
output_table,
independent_varname, -- can be NULL for minibatch
dependent_varname, -- can be NULL for minibatch
hidden_layer_sizes,
optimizer_params,
activation,
weights,
warm_start,
verbose,
grouping_col -- can be NULL for minibatch
)
{code}
Params we don’t need to specify above since we have info from pre-processing
step:
* Independent vars
* Dependent vars
* Grouping cols
Optimizer parameters:
{code}
'learning_rate_init = <value>,
learning_rate_policy = <value>,
gamma = <value>,
power = <value>,
iterations_per_step = <value>,
n_iterations = <value>,
n_tries = <value>,
lambda = <value>,
tolerance = <value>,
batch_size = <value>, -- for minibatch
n_epochs = <value>’ -- for minibatch
{code}
Pros:
* Single MLP interface
* Explicit signal that user wants to minibatch (no unintended consequences of
forget to drop table)
Cons:
* User has to do separate preprocessor step
* MLP has 3 redundant parameters for MLP that have to be explained
Rules:
* Independent_varname, dependent_varname, grouping_col can be NULL if MLP
using mini batching. However, if they are specified using mini-batching, they
must match that params from the preprocessor, or else throw an error. We don’t
want the user to change one of these parameters in MLP and expect a different
outcome.
Note that the above interface applies accordingly for mlp_regression as well.
> Add mini batch based gradient descent support to MLP
> ----------------------------------------------------
>
> Key: MADLIB-1206
> URL: https://issues.apache.org/jira/browse/MADLIB-1206
> Project: Apache MADlib
> Issue Type: New Feature
> Components: Module: Neural Networks
> Reporter: Nandish Jayaram
> Assignee: Nandish Jayaram
> Priority: Major
> Fix For: v1.14
>
>
> Mini-batch gradient descent is typically the algorithm of choice when
> training a neural network.
> MADlib currently supports IGD, we may have to add extensions to include
> mini-batch as a solver for MLP. Other modules will continue to use the
> existing IGD that does not support mini-batching. Later JIRAs will move other
> modules over one at a time to use the new mini-batch GD.
> Related JIRA that will pre-process the input data to be consumed by
> mini-batch is https://issues.apache.org/jira/browse/MADLIB-1200
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
