[
https://issues.apache.org/jira/browse/MADLIB-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320905#comment-15320905
]
Orhan Kislal commented on MADLIB-908:
-------------------------------------
Here is how I am planning to handle null values in this phase.
NULL Index: Assume that row doesn't exist
NULL Pivot: Assume that row doesn't exist
NULL Value: Leave it the the aggregate function to handle
For the next phase: Handling NULL Pivot can be changed. We can take NULL just
like any other pivot value and create a new column (i.e. piv_10, piv_20,
piv_30, piv_NULL) Handling NULL Index values is more challenging. If we assume
it is a valid record then do we group all NULL index records together? AFAIK
Pandas implementation just ignores rows with NULL indices for the pivoting
purposes.
> Pivoting - Phase 1 (basic pivot)
> --------------------------------
>
> Key: MADLIB-908
> URL: https://issues.apache.org/jira/browse/MADLIB-908
> Project: Apache MADlib
> Issue Type: New Feature
> Components: Module: Utilities
> Reporter: Frank McQuillan
> Fix For: v1.9.1
>
>
> Story
> As a data scientist, I want to perform *basic* pivot operation on my data, so
> that I can prepare it for input to predictive analytics algorithms.
> Details
> Basic pivot for this story means:
> * single index column
> * single pivot column
> * single value column
> * single aggregate function = SUM
> * no fill value
> PDL Tools [5] supports some of this functionality but its interface is
> confusing. Pandas [2], PostgreSQL [4] and Aster [6] also have some version
> of pivoting that we can learn from.
> In general, we are following Pandas ideas.
> References
> [1] Pivot table general information, like what is pivoting?
> https://en.wikipedia.org/wiki/Pivot_table
> [2] Pandas pivot tables and cross-tabulations
> http://pandas.pydata.org/pandas-docs/stable/reshaping.html#pivot-tables-and-cross-tabulations
> http://pandas.pydata.org/pandas-docs/stable/cookbook.html#cookbook-pivot
> http://pbpython.com/pandas-pivot-table-explained.html
> [3] GPDB pivot_sum function
> http://gpdb.docs.pivotal.io/4320/admin_guide/query.html#topic30
> [4] PostgreSQL tablefunc
> http://www.postgresql.org/docs/9.4/static/tablefunc.html
> [5] PDL tools pivoting routines
> http://pdl-tools.pa.pivotal.io/group__grp__pivot.html
> http://pdl-tools.pa.pivotal.io/group__grp__pivot01.html
> [6] Aster Pivot and Unpivot functions
> User Guide
> http://www.info.teradata.com/eDownload.cfm?itemid=122580002
> [7] PostgreSQL aggregates
> http://www.postgresql.org/docs/8.2/static/functions-aggregate.html
> [8] PostgreSQL basic statements/assignment operator,
> http://www.postgresql.org/docs/8.2/static/plpgsql-statements.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
