Frank McQuillan created MADLIB-1226:

             Summary: Add option for 1-hot encoding to minibatch preprocessor
                 Key: MADLIB-1226
             Project: Apache MADlib
          Issue Type: Improvement
          Components: Module: Utilities
            Reporter: Frank McQuillan
             Fix For: v1.14

I was testing MNIST dataset with minibatch preprocessor + MLP and could not get 
it to converge.   It turned out to be user error (me) and not a problem with 
convergence at all, because I forgot to 1-hot encode the dependent variable.

But I am wondering if other people might do the same thing that I did and get 

Here's what I did.  For this input data:

madlib=# \d+ public.mnist_train

                                              Table "public.mnist_train"

 Column |   Type    |                        Modifiers                         
| Storage  | Stats target | Description 


 y      | integer   |                                                          
| plain    |              | 

 x      | integer[] |                                                          
| extended |              | 

 id     | integer   | not null default nextval('mnist_train_id_seq'::regclass) 
| plain    |              | 

I called minibatch preprocessor:

SELECT madlib.minibatch_preprocessor('mnist_train',         -- Source table
                                     'mnist_train_packed',  -- Output table
                                     'y',                   -- Dependent 
                                     'x'                    -- Independent 

then mlp:

SELECT madlib.mlp_classification(
    'mnist_train_packed',        -- Source table from preprocessor output
    'mnist_result',              -- Destination table
    'independent_varname',       --  Independent
    'dependent_varname',        -- Dependent
    ARRAY[5],                    -- Hidden layer sizes
    learning_rate_policy=exp, n_epochs=20,
    lambda=0.0001,                 -- Regularization
    'tanh',                      -- Activation function
    '',                          -- No weights
    FALSE,                       -- No warmstart
    TRUE);                       -- Verbose

with the result:

INFO:  Iteration: 2, Loss: <-79.5295531257>
INFO:  Iteration: 3, Loss: <-79.529408892>
INFO:  Iteration: 4, Loss: <-79.5291940436>
INFO:  Iteration: 5, Loss: <-79.5288964944>
INFO:  Iteration: 6, Loss: <-79.5285051451>
INFO:  Iteration: 7, Loss: <-79.5280094708>
INFO:  Iteration: 8, Loss: <-79.5273995189>
INFO:  Iteration: 9, Loss: <-79.5266665607>

So it did not error out but clearly is not working on data in the right format.

I suggest 2 changes:

1) Add an explicit param in the mini-batch preprocessor for 1-hot encoding of 
dependent variable (this JIRA)

2) add a check to the MLP classification code to check that the dependent var 
has been 1-hot encodeded, and error out if that is not the case. (JIRA xxx)



This message was sent by Atlassian JIRA

Reply via email to