[jira] [Commented] (ARROW-4359) [Python] Column metadata is not saved or loaded in parquet

2019-08-22 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913270#comment-16913270 ] Joris Van den Bossche commented on ARROW-4359: -- I converted the example to be compatible

[jira] [Commented] (ARROW-2572) [Python] Add factory function to create a Table from Columns and Schema.

2019-08-22 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913308#comment-16913308 ] Joris Van den Bossche commented on ARROW-2572: -- Given that Columns no longer exist, I think

[jira] [Closed] (ARROW-2572) [Python] Add factory function to create a Table from Columns and Schema.

2019-08-22 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche closed ARROW-2572. Fix Version/s: (was: 0.15.0) Resolution: Not A Problem > [Python] Add

[jira] [Updated] (ARROW-6395) [Python] Bug when using bool arrays with stride greater than 1

2019-09-01 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6395: - Component/s: Python > [Python] Bug when using bool arrays with stride greater

[jira] [Updated] (ARROW-6395) [pyarrow] Bug when using bool arrays with stride greater than 1

2019-09-01 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6395: - Fix Version/s: 0.15.0 > [pyarrow] Bug when using bool arrays with stride greater

[jira] [Resolved] (ARROW-6395) [pyarrow] Bug when using bool arrays with stride greater than 1

2019-09-01 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche resolved ARROW-6395. -- Resolution: Duplicate > [pyarrow] Bug when using bool arrays with stride

[jira] [Updated] (ARROW-6395) [Python] Bug when using bool arrays with stride greater than 1

2019-09-01 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6395: - Summary: [Python] Bug when using bool arrays with stride greater than 1 (was:

[jira] [Commented] (ARROW-6395) [pyarrow] Bug when using bool arrays with stride greater than 1

2019-09-01 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920367#comment-16920367 ] Joris Van den Bossche commented on ARROW-6395: -- Yes, this is already fixed on master. So

[jira] [Assigned] (ARROW-6431) [CI][Crossbow] Nightly nopandas jobs fail

2019-09-03 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6431: Assignee: Joris Van den Bossche > [CI][Crossbow] Nightly nopandas jobs

[jira] [Created] (ARROW-6305) [Python] scalar pd.NaT incorrectly parsed in conversion from Python

2019-08-21 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6305: Summary: [Python] scalar pd.NaT incorrectly parsed in conversion from Python Key: ARROW-6305 URL: https://issues.apache.org/jira/browse/ARROW-6305

[jira] [Created] (ARROW-6325) [Python] wrong conversion of DataFrame with boolean values

2019-08-22 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6325: Summary: [Python] wrong conversion of DataFrame with boolean values Key: ARROW-6325 URL: https://issues.apache.org/jira/browse/ARROW-6325 Project:

[jira] [Commented] (ARROW-6325) [Python] wrong conversion of DataFrame with boolean values

2019-08-22 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913575#comment-16913575 ] Joris Van den Bossche commented on ARROW-6325: -- A numpy only reproducer. Starting from a 2D

[jira] [Updated] (ARROW-6325) [Python] wrong conversion of DataFrame with boolean values

2019-08-22 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6325: - Description: >From https://github.com/pandas-dev/pandas/issues/28090 {code} In

[jira] [Commented] (ARROW-6325) [Python] wrong conversion of DataFrame with boolean values

2019-08-22 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913572#comment-16913572 ] Joris Van den Bossche commented on ARROW-6325: -- So when converting an array of the column

[jira] [Commented] (ARROW-842) [Python] Handle more kinds of null sentinel objects from pandas 0.x

2019-08-21 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16912398#comment-16912398 ] Joris Van den Bossche commented on ARROW-842: - Updated example from ARROW-6305 that shows the

[jira] [Closed] (ARROW-6305) [Python] scalar pd.NaT incorrectly parsed in conversion from Python

2019-08-21 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche closed ARROW-6305. Resolution: Duplicate > [Python] scalar pd.NaT incorrectly parsed in conversion

[jira] [Commented] (ARROW-6301) [Python] atexit: pyarrow.lib.ArrowKeyError: 'No type extension with name arrow.py_extension_type found'

2019-08-21 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16912401#comment-16912401 ] Joris Van den Bossche commented on ARROW-6301: -- Should we put the call to unregister the

[jira] [Commented] (ARROW-5630) [Python] Table of nested arrays doesn't round trip

2019-08-23 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913988#comment-16913988 ] Joris Van den Bossche commented on ARROW-5630: -- Yes, get the same error on latest master. >

[jira] [Commented] (ARROW-5337) [C++] Add RecordBatch::field method, possibly deprecate "column"

2019-08-23 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913991#comment-16913991 ] Joris Van den Bossche commented on ARROW-5337: -- Since there is also a {{arrow::Field}} which

[jira] [Assigned] (ARROW-6325) [Python] wrong conversion of DataFrame with boolean values

2019-08-23 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6325: Assignee: Joris Van den Bossche > [Python] wrong conversion of DataFrame

[jira] [Commented] (ARROW-5494) [Python] Create FileSystem bindings

2019-08-23 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914027#comment-16914027 ] Joris Van den Bossche commented on ARROW-5494: -- I would happy to help here, although I will

[jira] [Comment Edited] (ARROW-5494) [Python] Create FileSystem bindings

2019-08-23 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914027#comment-16914027 ] Joris Van den Bossche edited comment on ARROW-5494 at 8/23/19 7:48 AM:

[jira] [Commented] (ARROW-5220) [Python] index / unknown columns in specified schema in Table.from_pandas

2019-08-23 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914040#comment-16914040 ] Joris Van den Bossche commented on ARROW-5220: -- What I suggested here (about

[jira] [Updated] (ARROW-5220) [Python] index / unknown columns in specified schema in Table.from_pandas

2019-08-23 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-5220: - Fix Version/s: 0.15.0 > [Python] index / unknown columns in specified schema in

[jira] [Commented] (ARROW-2428) [Python] Add API to map Arrow types (including extension types) to pandas ExtensionArray instances for to_pandas conversions

2019-08-23 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914231#comment-16914231 ] Joris Van den Bossche commented on ARROW-2428: -- I am working on the actual ability to create

[jira] [Created] (ARROW-6548) [Python] consistently handle conversion of all-NaN arrays across types

2019-09-12 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6548: Summary: [Python] consistently handle conversion of all-NaN arrays across types Key: ARROW-6548 URL: https://issues.apache.org/jira/browse/ARROW-6548

[jira] [Closed] (ARROW-5104) [Python/C++] Schema for empty tables include index column as integer

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche closed ARROW-5104. Fix Version/s: (was: 0.15.0) Resolution: Won't Fix > [Python/C++]

[jira] [Assigned] (ARROW-5220) [Python] index / unknown columns in specified schema in Table.from_pandas

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-5220: Assignee: Joris Van den Bossche > [Python] index / unknown columns in

[jira] [Comment Edited] (ARROW-5220) [Python] index / unknown columns in specified schema in Table.from_pandas

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929175#comment-16929175 ] Joris Van den Bossche edited comment on ARROW-5220 at 9/13/19 1:03 PM:

[jira] [Assigned] (ARROW-6556) [Python] Prepare for pandas release without SparseDataFrame

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6556: Assignee: Joris Van den Bossche > [Python] Prepare for pandas release

[jira] [Assigned] (ARROW-6520) [Python] Segmentation fault on writing tables with fixed size binary fields

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6520: Assignee: Joris Van den Bossche (was: Wes McKinney) > [Python]

[jira] [Commented] (ARROW-5104) [Python/C++] Schema for empty tables include index column as integer

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929121#comment-16929121 ] Joris Van den Bossche commented on ARROW-5104: -- Yeah, I don't think there is anything we can

[jira] [Comment Edited] (ARROW-5220) [Python] index / unknown columns in specified schema in Table.from_pandas

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929175#comment-16929175 ] Joris Van den Bossche edited comment on ARROW-5220 at 9/13/19 1:07 PM:

[jira] [Commented] (ARROW-5220) [Python] index / unknown columns in specified schema in Table.from_pandas

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929175#comment-16929175 ] Joris Van den Bossche commented on ARROW-5220: -- I started looking into this (as it was

[jira] [Comment Edited] (ARROW-5220) [Python] index / unknown columns in specified schema in Table.from_pandas

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929175#comment-16929175 ] Joris Van den Bossche edited comment on ARROW-5220 at 9/13/19 1:15 PM:

[jira] [Comment Edited] (ARROW-5220) [Python] index / unknown columns in specified schema in Table.from_pandas

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929175#comment-16929175 ] Joris Van den Bossche edited comment on ARROW-5220 at 9/13/19 1:14 PM:

[jira] [Commented] (ARROW-6486) [Python] Allow subclassing & monkey-patching of Table

2019-09-08 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925371#comment-16925371 ] Joris Van den Bossche commented on ARROW-6486: -- Can you give an example use case where you

[jira] [Assigned] (ARROW-6506) [C++] Validation of ExtensionType with nested type fails

2019-09-10 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6506: Assignee: Joris Van den Bossche > [C++] Validation of ExtensionType with

[jira] [Commented] (ARROW-6522) [Python] Test suite fails with pandas 0.23.4, pytest 3.8.1

2019-09-11 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927336#comment-16927336 ] Joris Van den Bossche commented on ARROW-6522: -- Apparently I mixed the keyword of the

[jira] [Assigned] (ARROW-6522) [Python] Test suite fails with pandas 0.23.4, pytest 3.8.1

2019-09-11 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6522: Assignee: Joris Van den Bossche > [Python] Test suite fails with pandas

[jira] [Created] (ARROW-6492) [Python] file written with latest fastparquet cannot be read with latest pyarrow

2019-09-09 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6492: Summary: [Python] file written with latest fastparquet cannot be read with latest pyarrow Key: ARROW-6492 URL: https://issues.apache.org/jira/browse/ARROW-6492

[jira] [Assigned] (ARROW-6488) [Python] pyarrow.NULL equals to itself

2019-09-09 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6488: Assignee: Joris Van den Bossche > [Python] pyarrow.NULL equals to itself

[jira] [Commented] (ARROW-6488) [Python] pyarrow.NULL equals to itself

2019-09-09 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925636#comment-16925636 ] Joris Van den Bossche commented on ARROW-6488: -- On C++ this gives NULL, will do a PR to

[jira] [Commented] (ARROW-6492) [Python] file written with latest fastparquet cannot be read with latest pyarrow

2019-09-09 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925656#comment-16925656 ] Joris Van den Bossche commented on ARROW-6492: -- This is related to a difference in the

[jira] [Updated] (ARROW-6529) [C++] Feather: slow writing of NullArray

2019-09-11 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6529: - Description: >From

[jira] [Commented] (ARROW-6520) [Python] Segmentation fault on writing tables with fixed size binary fields

2019-09-11 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927504#comment-16927504 ] Joris Van den Bossche commented on ARROW-6520: -- So this is due to an invalid creation of the

[jira] [Commented] (ARROW-6520) [Python] Segmentation fault on writing tables with fixed size binary fields

2019-09-11 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927505#comment-16927505 ] Joris Van den Bossche commented on ARROW-6520: -- [~kszucs] yes, will add a test case for the

[jira] [Created] (ARROW-6529) [C++] Feather: slow writing of NullArray

2019-09-11 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6529: Summary: [C++] Feather: slow writing of NullArray Key: ARROW-6529 URL: https://issues.apache.org/jira/browse/ARROW-6529 Project: Apache Arrow

[jira] [Commented] (ARROW-6520) [Python] Segmentation fault on writing tables with fixed size binary fields

2019-09-11 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927496#comment-16927496 ] Joris Van den Bossche commented on ARROW-6520: -- I can reproduce this on 0.14.1, but not any

[jira] [Commented] (ARROW-6520) [Python] Segmentation fault on writing tables with fixed size binary fields

2019-09-11 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927530#comment-16927530 ] Joris Van den Bossche commented on ARROW-6520: -- So the reason it was passing on master, is

[jira] [Assigned] (ARROW-5853) [Python] Expose boolean filter kernel on Array

2019-09-10 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-5853: Assignee: Joris Van den Bossche > [Python] Expose boolean filter kernel

[jira] [Commented] (ARROW-6561) [Python] pandas-master integration test failure

2019-09-15 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929933#comment-16929933 ] Joris Van den Bossche commented on ARROW-6561: -- Sorry, this is because I merged

[jira] [Assigned] (ARROW-6561) [Python] pandas-master integration test failure

2019-09-15 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6561: Assignee: Joris Van den Bossche > [Python] pandas-master integration test

[jira] [Assigned] (ARROW-6560) [Python] Failures in *-nopandas integration tests

2019-09-15 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6560: Assignee: Joris Van den Bossche > [Python] Failures in *-nopandas

[jira] [Assigned] (ARROW-6564) [Python] Do not require pandas for invoking Array.__array__

2019-09-17 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6564: Assignee: Joris Van den Bossche > [Python] Do not require pandas for

[jira] [Commented] (ARROW-5139) [Python/C++] Empty column selection no longer restores index

2019-09-17 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931417#comment-16931417 ] Joris Van den Bossche commented on ARROW-5139: -- Now we add the {{preserve_index=None /

[jira] [Updated] (ARROW-5139) [Python/C++] Empty column selection no longer restores index

2019-09-17 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-5139: - Fix Version/s: (was: 0.15.0) 1.0.0 > [Python/C++] Empty

[jira] [Comment Edited] (ARROW-2428) [Python] Add API to map Arrow types (including extension types) to pandas ExtensionArray instances for to_pandas conversions

2019-09-07 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924959#comment-16924959 ] Joris Van den Bossche edited comment on ARROW-2428 at 9/7/19 5:55 PM:

[jira] [Commented] (ARROW-2428) [Python] Add API to map Arrow types (including extension types) to pandas ExtensionArray instances for to_pandas conversions

2019-09-07 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924959#comment-16924959 ] Joris Van den Bossche commented on ARROW-2428: -- > It seems like the pandas glue can be part

[jira] [Commented] (ARROW-5853) [Python] Expose boolean filter kernel on Array

2019-09-09 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925717#comment-16925717 ] Joris Van den Bossche commented on ARROW-5853: -- [~bhaugen] did you do more work on this? Or

[jira] [Assigned] (ARROW-5682) [Python] from_pandas conversion casts values to string inconsistently

2019-09-09 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-5682: Assignee: Joris Van den Bossche > [Python] from_pandas conversion casts

[jira] [Commented] (ARROW-2051) [Python] Support serializing UUID objects to tables

2019-09-18 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932447#comment-16932447 ] Joris Van den Bossche commented on ARROW-2051: -- What is the exact idea here? To provide a

[jira] [Commented] (ARROW-4830) [Python] Remove backward compatibility hacks from pyarrow.pandas_compat

2019-09-18 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932708#comment-16932708 ] Joris Van den Bossche commented on ARROW-4830: -- For this, I think we should ideally decide

[jira] [Commented] (ARROW-6157) [Python][C++] UnionArray with invalid data passes validation / leads to segfaults

2019-09-18 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932703#comment-16932703 ] Joris Van den Bossche commented on ARROW-6157: -- The ListArray validation actually does

[jira] [Commented] (ARROW-1664) [Python] Support for xarray.DataArray and xarray.Dataset

2019-09-18 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932827#comment-16932827 ] Joris Van den Bossche commented on ARROW-1664: -- In general, xarray datasets/dataarrays do

[jira] [Created] (ARROW-6488) [Python] pyarrow.NULL equals to itself

2019-09-08 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6488: Summary: [Python] pyarrow.NULL equals to itself Key: ARROW-6488 URL: https://issues.apache.org/jira/browse/ARROW-6488 Project: Apache Arrow

[jira] [Commented] (ARROW-6488) [Python] pyarrow.NULL equals to itself

2019-09-08 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925238#comment-16925238 ] Joris Van den Bossche commented on ARROW-6488: -- Thanks [~jbrockmendel] for noticing! >

[jira] [Created] (ARROW-6506) [C++] Validation of ExtensionType with nested type fails

2019-09-10 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6506: Summary: [C++] Validation of ExtensionType with nested type fails Key: ARROW-6506 URL: https://issues.apache.org/jira/browse/ARROW-6506 Project:

[jira] [Created] (ARROW-6507) [C++] Add ExtensionArray::ExtensionValidate for custom validation?

2019-09-10 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6507: Summary: [C++] Add ExtensionArray::ExtensionValidate for custom validation? Key: ARROW-6507 URL: https://issues.apache.org/jira/browse/ARROW-6507

[jira] [Created] (ARROW-6556) [Python] prepare on pandas release without SparseDataFrame

2019-09-13 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6556: Summary: [Python] prepare on pandas release without SparseDataFrame Key: ARROW-6556 URL: https://issues.apache.org/jira/browse/ARROW-6556 Project:

[jira] [Updated] (ARROW-6556) [Python] Prepare for pandas release without SparseDataFrame

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6556: - Summary: [Python] Prepare for pandas release without SparseDataFrame (was:

[jira] [Updated] (ARROW-6556) [Python] prepare on pandas release without SparseDataFrame

2019-09-13 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6556: - Issue Type: Improvement (was: Test) > [Python] prepare on pandas release

[jira] [Assigned] (ARROW-6187) [C++] fallback to storage type when writing ExtensionType to Parquet

2019-09-19 Thread Joris Van den Bossche (Jira)
[ https://issues.apache.org/jira/browse/ARROW-6187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6187: Assignee: Joris Van den Bossche > [C++] fallback to storage type when

[jira] [Created] (ARROW-6618) [Python] Reading a zero-size buffer can segfault

2019-09-19 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6618: Summary: [Python] Reading a zero-size buffer can segfault Key: ARROW-6618 URL: https://issues.apache.org/jira/browse/ARROW-6618 Project: Apache Arrow

[jira] [Closed] (ARROW-4032) [Python] New pyarrow.Table functions: from_pydict(), from_pylist() and to_pylist()

2019-07-31 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche closed ARROW-4032. Resolution: Duplicate > [Python] New pyarrow.Table functions: from_pydict(),

[jira] [Commented] (ARROW-4032) [Python] New pyarrow.Table functions: from_pydict(), from_pylist() and to_pylist()

2019-07-31 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896867#comment-16896867 ] Joris Van den Bossche commented on ARROW-4032: -- Closing this issue in favor of your new

[jira] [Updated] (ARROW-6001) [Python] Add from_pylist() and to_pylist() to pyarrow.Table to convert list of records

2019-07-31 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6001: - Component/s: Python Summary: [Python] Add from_pylist() and to_pylist()

[jira] [Commented] (ARROW-6001) Add from_pydict(), from_pylist() and to_pylist() to pyarrow.Table + improve pandas.to_dict()

2019-07-31 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896870#comment-16896870 ] Joris Van den Bossche commented on ARROW-6001: -- See also ARROW-4032 for similar discussion

[jira] [Commented] (ARROW-6001) [Python] Add from_pylist() and to_pylist() to pyarrow.Table to convert list of records

2019-07-31 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896888#comment-16896888 ] Joris Van den Bossche commented on ARROW-6001: -- I think the functionality to convert to /

[jira] [Commented] (ARROW-5952) [Python] Segfault when reading empty table with category as pandas dataframe

2019-07-31 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-5952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896980#comment-16896980 ] Joris Van den Bossche commented on ARROW-5952: -- [~nugend] Thanks for the report! Looking at

[jira] [Assigned] (ARROW-6132) [Python] ListArray.from_arrays does not check validity of input arrays

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche reassigned ARROW-6132: Assignee: Joris Van den Bossche > [Python] ListArray.from_arrays does not

[jira] [Created] (ARROW-6159) [C++] PrettyPrint of arrow::Schema missing identation for first line

2019-08-07 Thread Joris Van den Bossche (JIRA)
Joris Van den Bossche created ARROW-6159: Summary: [C++] PrettyPrint of arrow::Schema missing identation for first line Key: ARROW-6159 URL: https://issues.apache.org/jira/browse/ARROW-6159

[jira] [Updated] (ARROW-6159) [C++] PrettyPrint of arrow::Schema missing identation for first line

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6159: - Labels: beginner (was: ) > [C++] PrettyPrint of arrow::Schema missing

[jira] [Created] (ARROW-6157) [Python][C++] UnionArray with invalid data passes validation / leads to segfaults

2019-08-07 Thread Joris Van den Bossche (JIRA)
Joris Van den Bossche created ARROW-6157: Summary: [Python][C++] UnionArray with invalid data passes validation / leads to segfaults Key: ARROW-6157 URL: https://issues.apache.org/jira/browse/ARROW-6157

[jira] [Commented] (ARROW-5151) [C++] Support take from UnionArray, ListArray, StructArray

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-5151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901953#comment-16901953 ] Joris Van den Bossche commented on ARROW-5151: -- [~bkietz] Based on your comment in

[jira] [Commented] (ARROW-6158) [Python] possible to create StructArray with type that conflicts with child array's types

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901978#comment-16901978 ] Joris Van den Bossche commented on ARROW-6158: -- Found an example where it starts to give

[jira] [Created] (ARROW-6158) [Python] possible to create StructArray with type that conflicts with child array's types

2019-08-07 Thread Joris Van den Bossche (JIRA)
Joris Van den Bossche created ARROW-6158: Summary: [Python] possible to create StructArray with type that conflicts with child array's types Key: ARROW-6158 URL:

[jira] [Updated] (ARROW-6157) [Python][C++] UnionArray with invalid data passes validation / leads to segfaults

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6157: - Description: >From the Python side, you can create an "invalid" UnionArray:

[jira] [Commented] (ARROW-6132) [Python] ListArray.from_arrays does not check validity of input arrays

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901945#comment-16901945 ] Joris Van den Bossche commented on ARROW-6132: -- {{DictionaryArray.from_arrays}} has a

[jira] [Commented] (ARROW-6132) [Python] ListArray.from_arrays does not check validity of input arrays

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901966#comment-16901966 ] Joris Van den Bossche commented on ARROW-6132: -- I was actually just planning to open an

[jira] [Updated] (ARROW-6159) [C++] PrettyPrint of arrow::Schema missing identation for first line

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6159: - Labels: (was: first) > [C++] PrettyPrint of arrow::Schema missing identation

[jira] [Updated] (ARROW-6159) [C++] PrettyPrint of arrow::Schema missing identation for first line

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6159: - Labels: first (was: ) > [C++] PrettyPrint of arrow::Schema missing identation

[jira] [Commented] (ARROW-5610) [Python] Define extension type API in Python to "receive" or "send" a foreign extension type

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902129#comment-16902129 ] Joris Van den Bossche commented on ARROW-5610: -- [~lidavidm] no apologies needed, it was just

[jira] [Commented] (ARROW-5610) [Python] Define extension type API in Python to "receive" or "send" a foreign extension type

2019-08-07 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902176#comment-16902176 ] Joris Van den Bossche commented on ARROW-5610: -- > But which also means that you loose all

[jira] [Commented] (ARROW-5682) [Python] from_pandas conversion casts values to string inconsistently

2019-08-02 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898787#comment-16898787 ] Joris Van den Bossche commented on ARROW-5682: -- This seems to be specific to the code paths

[jira] [Updated] (ARROW-5682) [Python] from_pandas conversion casts values to string inconsistently

2019-08-02 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-5682: - Issue Type: Bug (was: Improvement) > [Python] from_pandas conversion casts

[jira] [Created] (ARROW-6115) [Python] support LargeList, LargeString, LargeBinary in conversion to pandas

2019-08-02 Thread Joris Van den Bossche (JIRA)
Joris Van den Bossche created ARROW-6115: Summary: [Python] support LargeList, LargeString, LargeBinary in conversion to pandas Key: ARROW-6115 URL: https://issues.apache.org/jira/browse/ARROW-6115

[jira] [Updated] (ARROW-6114) Datatypes are not preserved when a pandas dataframe partitioned and saved as parquet file using pyarrow

2019-08-02 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6114: - Labels: parquet (was: ) > Datatypes are not preserved when a pandas dataframe

[jira] [Commented] (ARROW-6114) Datatypes are not preserved when a pandas dataframe partitioned and saved as parquet file using pyarrow

2019-08-02 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898737#comment-16898737 ] Joris Van den Bossche commented on ARROW-6114: -- [~bnriiitb] thanks for opening the issue.

[jira] [Updated] (ARROW-6114) Datatypes are not preserved when a pandas dataframe partitioned and saved as parquet file using pyarrow

2019-08-02 Thread Joris Van den Bossche (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche updated ARROW-6114: - Labels: dataset parquet (was: parquet) > Datatypes are not preserved when a

<    1   2   3   4   5   6   7   8   9   10   >