[
https://issues.apache.org/jira/browse/SYSTEMML-995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15538214#comment-15538214
]
Matthias Boehm edited comment on SYSTEMML-995 at 10/1/16 9:09 AM:
------------------------------------------------------------------
ok, I now have patches for (1) the frame meta data handling (support for {{DF}}
and {{DF_WITH_INDEX}} in {{FrameFormat}} and correct matrix characteristics
updates), as well as (2) reblock handling (frames just inherited a bug from the
old {{MLContext}} which was solved via a HACK at runtime level for matrices -
anyway this is now fixed for both matrices and frames). Note that we don't need
to explicitly include vectors into the {{FrameFormat}} as we anyway allow mixed
schemas where one among many columns can be a vector.
cc [~niketanpansare] please correct me if I'm wrong regarding the unnecessary
reblock handling of equal blocksizes for matrices.
was (Author: mboehm7):
ok, I now have patches for (1) the frame meta data handling (support for {{DF}}
and {{DF_WITH_INDEX}} in {{FrameFormat}} and correct matrix characteristics
updates), as well as (2) reblock handling (frames just inherited a bug from the
old {{MLContext}} which was solved via a HACK at runtime level for matrices -
anyway this is now fixed for both matrices and frames).
cc [~niketanpansare] please correct me if I'm wrong regarding the unnecessary
reblock handling of equal blocksizes for matrices.
> MLContext dataframe-frame conversion with index column & vector column
> ----------------------------------------------------------------------
>
> Key: SYSTEMML-995
> URL: https://issues.apache.org/jira/browse/SYSTEMML-995
> Project: SystemML
> Issue Type: Bug
> Components: APIs
> Affects Versions: SystemML 0.11
> Reporter: Matthias Boehm
> Priority: Blocker
>
> MLContext currently always assumes data frame to frame conversion without
> existing index column. Since the user cannot communicate the existence of
> this column, the data conversion leads to incorrect results as an additional
> column is included in the output frame. We need make the MLContext handling
> of frames consistent with the handling of matrices.
> Additionally, the conversion code in
> {{MLContextConversionUtil.dataFrameToFrameObject()}} does not yet take into
> account frames with vectors, although the recent addition adds this support
> in the underlying {{FrameRDDConverterUtils.java}} class. Therefore, the
> number of columns set when {{mc == null}} is incorrect.
> Thanks [[email protected]] for catching this issue. cc [~acs_s] [~deron]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
