[jira] [Commented] (ARROW-3246) [Python][Parquet] direct reading/writing of pandas categoricals in parquet

2019-08-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905735#comment-16905735 ] Wes McKinney commented on ARROW-3246: - This has been quite the saga, but I should be able to get a

[jira] [Commented] (ARROW-3246) [Python][Parquet] direct reading/writing of pandas categoricals in parquet

2019-08-09 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904020#comment-16904020 ] Wes McKinney commented on ARROW-3246: - Making some progress on this. It's a can of worms because of

[jira] [Commented] (ARROW-3246) [Python][Parquet] direct reading/writing of pandas categoricals in parquet

2019-08-08 Thread Hatem Helal (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902896#comment-16902896 ] Hatem Helal commented on ARROW-3246: >  If the dictionary is written all at once then this property

[jira] [Commented] (ARROW-3246) [Python][Parquet] direct reading/writing of pandas categoricals in parquet

2019-08-07 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902652#comment-16902652 ] Wes McKinney commented on ARROW-3246: - OK, I was able to get the initial refactor done today. Now we

[jira] [Commented] (ARROW-3246) [Python][Parquet] direct reading/writing of pandas categoricals in parquet

2019-08-07 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902341#comment-16902341 ] Wes McKinney commented on ARROW-3246: - Writing BYTE_ARRAY can also definitely be made more efficient.

[jira] [Commented] (ARROW-3246) [Python][Parquet] direct reading/writing of pandas categoricals in parquet

2019-08-07 Thread Hatem Helal (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901988#comment-16901988 ] Hatem Helal commented on ARROW-3246: Adding {{TypedColumnWriter::WriteArrow(const ::arrow::Array&)}}

[jira] [Commented] (ARROW-3246) [Python][Parquet] direct reading/writing of pandas categoricals in parquet

2019-08-06 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901508#comment-16901508 ] Wes McKinney commented on ARROW-3246: - I created ARROW-6152 to cover the initial feature-preserving

[jira] [Commented] (ARROW-3246) [Python][Parquet] direct reading/writing of pandas categoricals in parquet

2019-08-06 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901506#comment-16901506 ] Wes McKinney commented on ARROW-3246: - I've been looking at what's required to write