[jira] [Created] (ARROW-3776) [Rust] Mark methods that do not perform bounds checking as unsafe

2018-11-12 Thread Paddy Horan (JIRA)
Paddy Horan created ARROW-3776: -- Summary: [Rust] Mark methods that do not perform bounds checking as unsafe Key: ARROW-3776 URL: https://issues.apache.org/jira/browse/ARROW-3776 Project: Apache Arrow

[jira] [Resolved] (ARROW-3238) [Python] Can't read pyarrow string columns in fastparquet

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-3238. - Resolution: Not A Problem I don't believe there is anything we can fix here > [Python] Can't rea

[jira] [Updated] (ARROW-3774) [C++] Change parquet::arrow::FileReader::ReadRowGroups to read into contiguous arrays

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3774: Summary: [C++] Change parquet::arrow::FileReader::ReadRowGroups to read into contiguous arrays (wa

[jira] [Updated] (ARROW-3766) [Python] pa.Table.from_pandas doesn't use schema ordering

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3766: Summary: [Python] pa.Table.from_pandas doesn't use schema ordering (was: pa.Table.from_pandas does

[jira] [Moved] (ARROW-3775) [C++] Handling Arrow reads that overflow a BinaryArray capacity

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney moved PARQUET-1186 to ARROW-3775: -- Fix Version/s: (was: cpp-1.5.0) 0.12.0 Com

[jira] [Updated] (ARROW-3775) [C++] Handling Arrow reads that overflow a BinaryArray capacity

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3775: Labels: parquet (was: ) > [C++] Handling Arrow reads that overflow a BinaryArray capacity > --

[jira] [Updated] (ARROW-3771) [C++] GetRecordBatchReader in parquet/arrow/reader.h should be able to specify chunksize

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3771: Labels: parquet (was: ) > [C++] GetRecordBatchReader in parquet/arrow/reader.h should be able to

[jira] [Commented] (ARROW-3775) [C++] Handling Arrow reads that overflow a BinaryArray capacity

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684428#comment-16684428 ] Wes McKinney commented on ARROW-3775: - Moved to Arrow. I think this might be a duplic

[jira] [Moved] (ARROW-3771) [C++] GetRecordBatchReader in parquet/arrow/reader.h should be able to specify chunksize

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney moved PARQUET-1257 to ARROW-3771: -- Component/s: (was: parquet-cpp) C++

[jira] [Updated] (ARROW-3774) [C++] Change parquet::arrow::FileReader::ReadRowGroups to read into contigous arrays

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3774: Summary: [C++] Change parquet::arrow::FileReader::ReadRowGroups to read into contigous arrays (was

[jira] [Updated] (ARROW-3774) [C++] Change parquet::arrow::FileReader::ReadRowGroups to read into continuous arrays

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3774: Labels: parquet (was: ) > [C++] Change parquet::arrow::FileReader::ReadRowGroups to read into > c

[jira] [Moved] (ARROW-3774) [C++] Change parquet::arrow::FileReader::ReadRowGroups to read into continuous arrays

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney moved PARQUET-1393 to ARROW-3774: -- Fix Version/s: (was: cpp-1.6.0) Component/s: (was: parque

[jira] [Updated] (ARROW-3773) [C++] Remove duplicated AssertArraysEqual code in parquet/arrow/arrow-reader-writer-test.cc

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3773: Summary: [C++] Remove duplicated AssertArraysEqual code in parquet/arrow/arrow-reader-writer-test.c

[jira] [Updated] (ARROW-3773) [C++] Fix AssertArraysEqual call

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3773: Labels: parquet (was: ) > [C++] Fix AssertArraysEqual call > > >

[jira] [Moved] (ARROW-3773) [C++] Fix AssertArraysEqual call

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney moved PARQUET-1127 to ARROW-3773: -- Fix Version/s: (was: cpp-1.5.0) 0.12.0

[jira] [Commented] (ARROW-3772) [C++] Read Parquet dictionary encoded ColumnChunks directly into an Arrow DictionaryArray

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684422#comment-16684422 ] Wes McKinney commented on ARROW-3772: - Moved issue to Arrow issue tracker > [C++] Re

[jira] [Moved] (ARROW-3772) [C++] Read Parquet dictionary encoded ColumnChunks directly into an Arrow DictionaryArray

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney moved PARQUET-1324 to ARROW-3772: -- Fix Version/s: (was: cpp-1.6.0) 0.13.0 Com

[jira] [Updated] (ARROW-3772) [C++] Read Parquet dictionary encoded ColumnChunks directly into an Arrow DictionaryArray

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3772: Labels: parquet (was: ) > [C++] Read Parquet dictionary encoded ColumnChunks directly into an Arro

[jira] [Updated] (ARROW-3770) [C++] Validate or add option to validate arrow::Table schema in parquet::arrow::FileWriter::WriteTable

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3770: Component/s: C++ > [C++] Validate or add option to validate arrow::Table schema in > parquet::arro

[jira] [Moved] (ARROW-3770) [C++] Validate or add option to validate arrow::Table schema in parquet::arrow::FileWriter::WriteTable

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney moved PARQUET-1362 to ARROW-3770: -- Fix Version/s: (was: cpp-1.6.0) Component/s: (was: parque

[jira] [Updated] (ARROW-3770) [C++] Validate or add option to validate arrow::Table schema in parquet::arrow::FileWriter::WriteTable

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3770: Labels: parquet (was: ) > [C++] Validate or add option to validate arrow::Table schema in > parqu

[jira] [Commented] (ARROW-3769) [C++] Support reading non-dictionary encoded binary Parquet columns directly as DictionaryArray

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684401#comment-16684401 ] Wes McKinney commented on ARROW-3769: - Moved this here from the Parquet JIRA > [C++]

[jira] [Moved] (ARROW-3769) [C++] Support reading non-dictionary encoded binary Parquet columns directly as DictionaryArray

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney moved PARQUET-1423 to ARROW-3769: -- Fix Version/s: (was: cpp-1.6.0) 0.13.0 Com

[jira] [Updated] (ARROW-3769) [C++] Support reading non-dictionary encoded binary Parquet columns directly as DictionaryArray

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3769: Labels: parquet (was: ) > [C++] Support reading non-dictionary encoded binary Parquet columns dire

[jira] [Commented] (ARROW-3738) [C++] Add CSV conversion option to parse ISO8601-like timestamp strings

2018-11-12 Thread Pindikura Ravindra (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684214#comment-16684214 ] Pindikura Ravindra commented on ARROW-3738: --- I'm fine with moving date.h to arr

[jira] [Commented] (ARROW-3738) [C++] Add CSV conversion option to parse ISO8601-like timestamp strings

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684184#comment-16684184 ] Wes McKinney commented on ARROW-3738: - It looks like this has already happened in ht

[jira] [Commented] (ARROW-3738) [C++] Add CSV conversion option to parse ISO8601-like timestamp strings

2018-11-12 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684181#comment-16684181 ] Antoine Pitrou commented on ARROW-3738: --- To keep things simple, I suggest we start

[jira] [Updated] (ARROW-3768) [Python] set classpath to hdfs not hadoop executable

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3768: Summary: [Python] set classpath to hdfs not hadoop executable (was: set classpath to hdfs not hado

[jira] [Commented] (ARROW-3768) [Python] set classpath to hdfs not hadoop executable

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684003#comment-16684003 ] Wes McKinney commented on ARROW-3768: - The issue will be resolved/closed once a patch

[jira] [Reopened] (ARROW-3768) set classpath to hdfs not hadoop executable

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reopened ARROW-3768: - > set classpath to hdfs not hadoop executable > --- > >

[jira] [Created] (ARROW-3768) set classpath to hdfs not hadoop executable

2018-11-12 Thread Andrew Harris (JIRA)
Andrew Harris created ARROW-3768: Summary: set classpath to hdfs not hadoop executable Key: ARROW-3768 URL: https://issues.apache.org/jira/browse/ARROW-3768 Project: Apache Arrow Issue Type:

[jira] [Updated] (ARROW-3439) [R] R language bindings for Feather format

2018-11-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-3439: -- Labels: pull-request-available (was: ) > [R] R language bindings for Feather format >

[jira] [Closed] (ARROW-3768) set classpath to hdfs not hadoop executable

2018-11-12 Thread Andrew Harris (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Harris closed ARROW-3768. Resolution: Fixed > set classpath to hdfs not hadoop executable > -

[jira] [Updated] (ARROW-2113) [Python] Incomplete CLASSPATH with "hadoop" contained in it can fool the classpath setting HDFS logic

2018-11-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-2113: -- Labels: pull-request-available (was: ) > [Python] Incomplete CLASSPATH with "hadoop" contained

[jira] [Commented] (ARROW-3766) pa.Table.from_pandas doesn't use schema ordering

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16683893#comment-16683893 ] Wes McKinney commented on ARROW-3766: - Sounds buggy to me. If you pass a schema to {{

[jira] [Updated] (ARROW-3766) pa.Table.from_pandas doesn't use schema ordering

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3766: Fix Version/s: 0.12.0 > pa.Table.from_pandas doesn't use schema ordering >

[jira] [Commented] (ARROW-3738) [C++] Add CSV conversion option to parse ISO8601-like timestamp strings

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16683886#comment-16683886 ] Wes McKinney commented on ARROW-3738: - Those sound like the right ones. date64 does

[jira] [Commented] (ARROW-3762) [C++] Arrow table reads error when overflowing capacity of BinaryArray

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16683875#comment-16683875 ] Wes McKinney commented on ARROW-3762: - No -- there have been lots of cases where peop

[jira] [Commented] (ARROW-3767) [C++] Add cast for Null to any type

2018-11-12 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16683866#comment-16683866 ] Wes McKinney commented on ARROW-3767: - Some memory buffers will have to be allocated

[jira] [Created] (ARROW-3767) [C++] Add cast for Null to any type

2018-11-12 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-3767: -- Summary: [C++] Add cast for Null to any type Key: ARROW-3767 URL: https://issues.apache.org/jira/browse/ARROW-3767 Project: Apache Arrow Issue Type: Improvement

[jira] [Commented] (ARROW-3738) [C++] Add CSV conversion option to parse ISO8601-like timestamp strings

2018-11-12 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16683527#comment-16683527 ] Antoine Pitrou commented on ARROW-3738: --- What formats exactly should we allow? I'm

[jira] [Updated] (ARROW-3766) pa.Table.from_pandas doesn't use schema ordering

2018-11-12 Thread Christian Thiel (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Thiel updated ARROW-3766: --- Description: Pyarrow is sensitive to the order of the columns upon load of partitioned Files.

[jira] [Created] (ARROW-3766) pa.Table.from_pandas doesn't use schema ordering

2018-11-12 Thread Christian Thiel (JIRA)
Christian Thiel created ARROW-3766: -- Summary: pa.Table.from_pandas doesn't use schema ordering Key: ARROW-3766 URL: https://issues.apache.org/jira/browse/ARROW-3766 Project: Apache Arrow Iss