[GitHub] incubator-madlib pull request #77: Cross validation in Elastic Net

[njayaram2]

Github user njayaram2 commented on a diff in the pull request:

    https://github.com/apache/incubator-madlib/pull/77#discussion_r91387261
  
    --- Diff: src/ports/postgres/modules/elastic_net/elastic_net.sql_in ---
    @@ -212,6 +203,87 @@ computation on only one lambda value.</DD>
     <DT>warmup_tolerance</DT>
     <DD>The value of tolerance used during warmup. The default is the same as 
the
     \e tolerance argument.</DD>
    +</DL>
    +
    +
    +<em><b>Cross validation parameters</b></em>
    +@note Cross validation is not supported if grouping is used.
    +<pre class="syntax">
    +  $$
    +    n_folds = &lt;value>,
    +    validation_result = &lt;value>,
    +    lambda_value = &lt;value>,
    +    n_lambdas = &lt;value>,
    +    alpha = &lt;value>
    +  $$
    +</pre>
    +
    +Hyperparameter optimization can be carried out using the built-in cross
    +validation mechanism, which is activated by assigning a value greater than 
1 to
    +the parameter \e n_folds in \e params.  Presently, misclassification error 
is used
    +for classification and mean squared error is used for regression.
    +
    +The values of a parameter to cross validate should be provided in a list. 
For
    +example, to regularize with the L1 norm and use a lambda value
    +from the set {0.3, 0.4, 0.5}, include 'lambda_value={0.3, 0.4, 0.5}' in
    +\e other_params. Note that the use of '{}' and '[]' are both valid
    +here.
    +
    +
    +<DL class="arglist">
    +
    +<DT>n_folds</dt>
    +<DD>Default: 0.
    +Number of folds (k). Must be at least 2 to activate cross validation.
    +If a value of k > 2 is specified, each fold is then used as a validation 
set once,
    +while the other k - 1 folds form the training set.
    +</DD>
    +
    +
    +<DT>validation_result</dt>
    +<DD>Default: NULL.
    +Name of the table to store the cross validation results including the 
values of
    +parameters and their averaged error values. The table is only created if 
the name is not NULL.
    +</DD>
    +
    +<DT>lambda_value</DT>
    +<DD>Regularization value. If a list is provided for cross validation, then 
warmup is
    +disabled on each lambda for performance reasons. </DD>
    +
    +<DT>n_lambdas</DT>
    +<DD>Number of lambdas to cross validate over. If a list of lambda values 
is not
    +provided, this parameter can be used to autogenerate a list of lambdas 
(using the
    +warmup procedure)
    +disabled on each lambda for performance reasons. </DD>
    +
    --- End diff --
    
    `validation_result` and `n_lambdas` are not described in the online docs, 
for both IGD and fista. These should be mentioned there too.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---