Pavan Kumar Nagula created MADLIB-1467: ------------------------------------------
Summary: madlib_keras_fit_multiple_model goes down with an IndexError: tuple index out of range Key: MADLIB-1467 URL: https://issues.apache.org/jira/browse/MADLIB-1467 Project: Apache MADlib Issue Type: Bug Components: Deep Learning Reporter: Pavan Kumar Nagula SELECT madlib.madlib_keras_fit_multiple_model('ml_ntwrk_training_final_dl_packed', -- source_table 'ml_ntwrk_multi_model', -- model_output_table 'mst_table', -- model_selection_table 10, -- num_iterations TRUE, -- use gpus 'ml_ntwrk_validation_final_dl_packed', 3, FALSE ); ERROR: spiexceptions.InternalError: IndexError: tuple index out of range (plpy_elog.c:121) (seg3 10.138.15.194:40003 pid=15416) (plpy_elog.c:121) CONTEXT: Traceback (most recent call last): PL/Python function "madlib_keras_fit_multiple_model", line 25, in <module> fit_obj.fit_multiple_model() PL/Python function "madlib_keras_fit_multiple_model", line 280, in fit_multiple_model PL/Python function "madlib_keras_fit_multiple_model", line 319, in train_multiple_model PL/Python function "madlib_keras_fit_multiple_model", line 996, in run_training configauration in mst_table config in mst_table SELECT * FROM mst_table ORDER BY mst_key; [4:39|https://pivotal.slack.com/archives/D01MS1EBQ4T/p1613032760009700] mst_key model_id compile_params fit_params 1 2 loss='categorical_crossentropy',optimizer='Adam(lr=0.001)',metrics=['accuracy'] batch_size=128,epochs=3, class_weight={0: 96.67510706743768, 1: 7099.175362318841, 2: 26621.907608695652, 3: 4018.4011484823627, 4: 17747.9384057971, 5: 17.063958085855717, 6: 10141.679089026915, 7: 23663.917874396135, 8: 30425.037267080745, 9: 0.19866780178818547, 10: 91.95823008185026, 11: 70991.7536231884, 12: 53243.815217391304, 13: 806.7244729907774, 14: 20.452824437680324, 15: 21297.52608695652, 16: 13.401413344422679, 17: 0.07584896996158862, 18: 106487.63043478261, 19: 217.54367810987253, 20: 208.79927536231884, 21: 10648.76304347826, 22: 0.2189344373189497} PL/Python function "madlib_keras_fit_multiple_model -- This message was sent by Atlassian Jira (v8.3.4#803005)