[jira] [Updated] (ARROW-4739) [Rust] [DataFusion] It should be possible to share a logical plan between threads
[ https://issues.apache.org/jira/browse/ARROW-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4739: -- Labels: pull-request-available (was: ) > [Rust] [DataFusion] It should be possible to share a logical plan between > threads > - > > Key: ARROW-4739 > URL: https://issues.apache.org/jira/browse/ARROW-4739 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust, Rust - DataFusion >Affects Versions: 0.12.0 >Reporter: Andy Grove >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > I want to be able to compile sql to a logical plan and then share that plan > with other threads ( so I can run the same query in parallel on partitions of > my input relation). > > A/C > * LogicalPlan uses Arc instead of Rc > * ExecutionContext has a create_logical_plan method > * ExecutionContext.sql() is refactored to call create_logical_plan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4745) [C++][Documentation] Document process for replicating static_crt builds on windows
[ https://issues.apache.org/jira/browse/ARROW-4745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4745: -- Labels: pull-request-available (was: ) > [C++][Documentation] Document process for replicating static_crt builds on > windows > -- > > Key: ARROW-4745 > URL: https://issues.apache.org/jira/browse/ARROW-4745 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, Documentation >Reporter: Micah Kornfield >Assignee: Micah Kornfield >Priority: Minor > Labels: pull-request-available > > Based on collective wisdom of the mailing list. Give some step by step > instructions to getting things to build. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4749) [Rust] RecordBatch::new() should return result instead of panicking
[ https://issues.apache.org/jira/browse/ARROW-4749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4749: -- Labels: pull-request-available (was: ) > [Rust] RecordBatch::new() should return result instead of panicking > --- > > Key: ARROW-4749 > URL: https://issues.apache.org/jira/browse/ARROW-4749 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Affects Versions: 0.12.0 >Reporter: Andy Grove >Assignee: Neville Dipale >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > RecordBatch::new() has some good validation checks, but calls assert_eq > instead of returning a Result -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4754) [CI][Java] Flaky TestAuth Flight test
[ https://issues.apache.org/jira/browse/ARROW-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4754: -- Labels: pull-request-available (was: ) > [CI][Java] Flaky TestAuth Flight test > - > > Key: ARROW-4754 > URL: https://issues.apache.org/jira/browse/ARROW-4754 > Project: Apache Arrow > Issue Type: Improvement > Components: Continuous Integration, FlightRPC, Java >Reporter: Micah Kornfield >Assignee: Micah Kornfield >Priority: Blocker > Labels: pull-request-available > > org.apache.arrow.flight.auth.TestAuth > [ERROR] invalidAuth(org.apache.arrow.flight.auth.TestAuth) Time elapsed: > 0.013 s <<< ERROR! > java.io.IOException: Failed to bind > at org.apache.arrow.flight.auth.TestAuth.setup(TestAuth.java:108) > Caused by: java.net.BindException: Address already in use -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4756) [CI] document the procedure to update docker image for manylinux1 builds
[ https://issues.apache.org/jira/browse/ARROW-4756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4756: -- Labels: pull-request-available (was: ) > [CI] document the procedure to update docker image for manylinux1 builds > > > Key: ARROW-4756 > URL: https://issues.apache.org/jira/browse/ARROW-4756 > Project: Apache Arrow > Issue Type: Task > Components: Continuous Integration >Reporter: Pindikura Ravindra >Assignee: Pindikura Ravindra >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4758) [Flight] Build fails on Mac due to missing Schema_generated.h
[ https://issues.apache.org/jira/browse/ARROW-4758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4758: -- Labels: pull-request-available (was: ) > [Flight] Build fails on Mac due to missing Schema_generated.h > - > > Key: ARROW-4758 > URL: https://issues.apache.org/jira/browse/ARROW-4758 > Project: Apache Arrow > Issue Type: Task > Components: FlightRPC >Reporter: Pindikura Ravindra >Priority: Major > Labels: pull-request-available > > I saw this on CI, a retrigger of the build fixed the issue and I am not able > to get the link of the previous build failure. > The error happened for the file flight/client.cc, which includes > -ipc/metadata--internal.h, which includes arrow/ipc/Schema_generated.h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4707) [C++] move BitsetStack to bit-util.h
[ https://issues.apache.org/jira/browse/ARROW-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4707: -- Labels: pull-request-available (was: ) > [C++] move BitsetStack to bit-util.h > > > Key: ARROW-4707 > URL: https://issues.apache.org/jira/browse/ARROW-4707 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Benjamin Kietzman >Assignee: Benjamin Kietzman >Priority: Trivial > Labels: pull-request-available > Fix For: 0.13.0 > > > BitsetStack was written for use in the JSON parser, but it's useful enough > that it should be made available in bit-util.h -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4448) [JAVA][Flight] Flaky Flight java test
[
https://issues.apache.org/jira/browse/ARROW-4448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4448:
--
Labels: ci-failure pull-request-available (was: ci-failure)
> [JAVA][Flight] Flaky Flight java test
> -
>
> Key: ARROW-4448
> URL: https://issues.apache.org/jira/browse/ARROW-4448
> Project: Apache Arrow
> Issue Type: Improvement
> Components: FlightRPC, Java
>Reporter: Micah Kornfield
>Assignee: Francois Saint-Jacques
>Priority: Major
> Labels: ci-failure, pull-request-available
> Fix For: 0.13.0
>
>
> Pull requests rebeased off of the laster master seem to pass so this is
> probably a flake: [https://travis-ci.org/apache/arrow/jobs/487275065]
> {{[INFO] Running org.apache.arrow.flight.TestBackPressure
> [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 1.396
> s <<< FAILURE! - in org.apache.arrow.flight.TestBackPressure
> [ERROR] ensureIndependentSteams(org.apache.arrow.flight.TestBackPressure)
> Time elapsed: 1.394 s <<< ERROR!
> java.lang.IllegalStateException:
> Memory was leaked by query. Memory leaked: (131072)
> Allocator(perf-server) 0/131072/589824/9223372036854775807
> (res/actual/peak/limit)
> at
> org.apache.arrow.flight.TestBackPressure.ensureIndependentSteams(TestBackPressure.java:76)
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR] TestBackPressure.ensureIndependentSteams:76 » IllegalState Memory
> was leaked b...
> [INFO]
> [ERROR] Tests run: 13, Failures: 0, Errors: 1, Skipped: 3
> [INFO]
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Arrow Java Root POM 0.13.0-SNAPSHOT . SUCCESS [ 8.590
> s]
> [INFO] Arrow Format ... SUCCESS [ 5.985
> s]
> [INFO] Arrow Memory ... SUCCESS [ 12.750
> s]
> [INFO] Arrow Vectors .. SUCCESS [01:13
> min]
> [INFO] Arrow Tools SUCCESS [ 15.462
> s]
> [INFO] Arrow JDBC Adapter . SUCCESS [ 11.906
> s]
> [INFO] Arrow Plasma Client SUCCESS [ 3.967
> s]
> [INFO] Arrow Flight 0.13.0-SNAPSHOT ... FAILURE [ 18.237
> s]
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 02:30 min
> [INFO] Finished at: 2019-02-01T05:15:34Z
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test) on
> project arrow-flight: There are test failures.
> [ERROR]
> [ERROR] Please refer to
> /home/travis/build/apache/arrow/java/flight/target/surefire-reports for the
> individual test results.
> [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump,
> [date].dumpstream and [date]-jvmRun[N].dumpstream.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :arrow-flight
> }}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4719) [C#] Implement ChunkedArray, Column and Table in C#
[ https://issues.apache.org/jira/browse/ARROW-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4719: -- Labels: pull-request-available (was: ) > [C#] Implement ChunkedArray, Column and Table in C# > --- > > Key: ARROW-4719 > URL: https://issues.apache.org/jira/browse/ARROW-4719 > Project: Apache Arrow > Issue Type: Task > Components: C# >Reporter: Prashanth Govindarajan >Priority: Major > Labels: pull-request-available > > Implement "ChunkedArray", "Column" and "Table" in arrow/csharp. They are > useful types to represent pieces in a dataset -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4767) [C#] ArrowStreamReader crashes while reading the end of a stream
[ https://issues.apache.org/jira/browse/ARROW-4767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4767: -- Labels: pull-request-available (was: ) > [C#] ArrowStreamReader crashes while reading the end of a stream > > > Key: ARROW-4767 > URL: https://issues.apache.org/jira/browse/ARROW-4767 > Project: Apache Arrow > Issue Type: Bug > Components: C# >Reporter: Prashanth Govindarajan >Priority: Major > Labels: pull-request-available > > ReadRecordBatchAsync crashes at the end of a stream when messageLength is 0. > "0" indicates the end of the stream, so we should just return null. The call > Flatbug.Message.GetRootAsMessage seems to be crashing. The fix is simple and > safe. I'll have a PR up soon. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4768) [C++][CI] arrow-test-array sometimes gets stuck in MinGW build
[
https://issues.apache.org/jira/browse/ARROW-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4768:
--
Labels: pull-request-available (was: )
> [C++][CI] arrow-test-array sometimes gets stuck in MinGW build
> --
>
> Key: ARROW-4768
> URL: https://issues.apache.org/jira/browse/ARROW-4768
> Project: Apache Arrow
> Issue Type: Test
> Components: C++, Continuous Integration
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Minor
> Labels: pull-request-available
>
> Example:
> https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/22804493/job/6mbpslm97p4yj31c#L726
> {noformat}
> Start 2: arrow-array-test
> {noformat}
> isn't finished.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4335) [C++] Better document sparse tensor support
[
https://issues.apache.org/jira/browse/ARROW-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4335:
--
Labels: pull-request-available (was: )
> [C++] Better document sparse tensor support
> ---
>
> Key: ARROW-4335
> URL: https://issues.apache.org/jira/browse/ARROW-4335
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Affects Versions: 0.12.0
>Reporter: Antoine Pitrou
>Assignee: Kenta Murata
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> Currently the documentation (including docstrings) for the sparse tensor
> classes and methods is very... sparse. It would be nice to make those
> approachable.
> (also, a suggestion: rename {{SparseCSRIndex::indptr()}} to something else?
> perhaps {{SparseCSRIndex::row_indices()}}?)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4769) [Rust] Improve array limit function where max records > len
[ https://issues.apache.org/jira/browse/ARROW-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4769: -- Labels: pull-request-available (was: ) > [Rust] Improve array limit function where max records > len > --- > > Key: ARROW-4769 > URL: https://issues.apache.org/jira/browse/ARROW-4769 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Affects Versions: 0.12.0 >Reporter: Neville Dipale >Assignee: Neville Dipale >Priority: Trivial > Labels: pull-request-available > > When we have an array of n records, and we want to take a limit that's higher > or equat to n, we still iterate through the array values and create a new > array. > We could improve this by returning a copy of the array as-is. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-2022) [Format] Add custom metadata field specific to a RecordBatch message
[ https://issues.apache.org/jira/browse/ARROW-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-2022: -- Labels: pull-request-available (was: ) > [Format] Add custom metadata field specific to a RecordBatch message > > > Key: ARROW-2022 > URL: https://issues.apache.org/jira/browse/ARROW-2022 > Project: Apache Arrow > Issue Type: Improvement > Components: Format >Reporter: Wes McKinney >Assignee: Micah Kornfield >Priority: Major > Labels: pull-request-available > > While we can have schema- and field-level custom metadata, we cannot send > metadata at the record batch level. This could include things like statistics > (although statistics isn't a great example, because this might be something > we want to eventually standardize), but other things too > See message definitions in > https://github.com/apache/arrow/blob/master/format/Message.fbs -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4206) [Gandiva] Implement decimal divide
[ https://issues.apache.org/jira/browse/ARROW-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4206: -- Labels: pull-request-available (was: ) > [Gandiva] Implement decimal divide > -- > > Key: ARROW-4206 > URL: https://issues.apache.org/jira/browse/ARROW-4206 > Project: Apache Arrow > Issue Type: Task > Components: C++ - Gandiva >Reporter: Pindikura Ravindra >Assignee: Pindikura Ravindra >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4775) [Website] Site navbar cannot be expanded
[ https://issues.apache.org/jira/browse/ARROW-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4775: -- Labels: pull-request-available (was: ) > [Website] Site navbar cannot be expanded > > > Key: ARROW-4775 > URL: https://issues.apache.org/jira/browse/ARROW-4775 > Project: Apache Arrow > Issue Type: Bug > Components: Website >Reporter: Kenta Murata >Assignee: Kenta Murata >Priority: Major > Labels: pull-request-available > > I found that the navbar at the top of the page cannot be expanded when the > page is narrow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4766) [C++] Casting empty boolean array causes segfault
[
https://issues.apache.org/jira/browse/ARROW-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4766:
--
Labels: pull-request-available (was: )
> [C++] Casting empty boolean array causes segfault
> -
>
> Key: ARROW-4766
> URL: https://issues.apache.org/jira/browse/ARROW-4766
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.0
>Reporter: Keith Kraus
>Assignee: Francois Saint-Jacques
>Priority: Blocker
> Labels: pull-request-available
>
> Reproducer:
> {code:python}
> import pyarrow as pa
> test = pa.array([], type=pa.bool_())
> test2 = test.cast(pa.int8())
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3770) [C++] Validate or add option to validate arrow::Table schema in parquet::arrow::FileWriter::WriteTable
[ https://issues.apache.org/jira/browse/ARROW-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-3770: -- Labels: parquet pull-request-available (was: parquet) > [C++] Validate or add option to validate arrow::Table schema in > parquet::arrow::FileWriter::WriteTable > -- > > Key: ARROW-3770 > URL: https://issues.apache.org/jira/browse/ARROW-3770 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Priority: Major > Labels: parquet, pull-request-available > Fix For: 0.13.0 > > > Failing to validate will cause a segfault when the passed table does not > match the schema used to instantiate the writer. See ARROW-2926 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4777) [C++/Python] manylinux1: Update lz4 to 1.8.3
[ https://issues.apache.org/jira/browse/ARROW-4777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4777: -- Labels: pull-request-available (was: ) > [C++/Python] manylinux1: Update lz4 to 1.8.3 > > > Key: ARROW-4777 > URL: https://issues.apache.org/jira/browse/ARROW-4777 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, Packaging, Python >Reporter: Uwe L. Korn >Assignee: Uwe L. Korn >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4780) [JS] Package sourcemap files, update default package JS version
[ https://issues.apache.org/jira/browse/ARROW-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4780: -- Labels: pull-request-available (was: ) > [JS] Package sourcemap files, update default package JS version > --- > > Key: ARROW-4780 > URL: https://issues.apache.org/jira/browse/ARROW-4780 > Project: Apache Arrow > Issue Type: Improvement > Components: JavaScript >Affects Versions: JS-0.4.0 >Reporter: Paul Taylor >Assignee: Paul Taylor >Priority: Minor > Labels: pull-request-available > Fix For: JS-0.4.1 > > > The build should split the sourcemaps out to speed up client builds, and > include a "module" entry in the package.json for @pika/web, and the main > package should ship the latest ESNext JS versions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4781) [JS] Ensure empty data initializes empty typed arrays
[ https://issues.apache.org/jira/browse/ARROW-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4781: -- Labels: pull-request-available (was: ) > [JS] Ensure empty data initializes empty typed arrays > - > > Key: ARROW-4781 > URL: https://issues.apache.org/jira/browse/ARROW-4781 > Project: Apache Arrow > Issue Type: Bug > Components: JavaScript >Affects Versions: JS-0.4.0 >Reporter: Paul Taylor >Assignee: Paul Taylor >Priority: Major > Labels: pull-request-available > Fix For: JS-0.4.1 > > > Empty ArrayData instances should initialize with the appropriate 0-length > buffers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4783) [C++][CI] Mingw32 builds sometimes timeout
[ https://issues.apache.org/jira/browse/ARROW-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4783: -- Labels: ci-failure pull-request-available (was: ci-failure) > [C++][CI] Mingw32 builds sometimes timeout > -- > > Key: ARROW-4783 > URL: https://issues.apache.org/jira/browse/ARROW-4783 > Project: Apache Arrow > Issue Type: Bug > Components: C++, Continuous Integration >Reporter: Micah Kornfield >Assignee: Micah Kornfield >Priority: Major > Labels: ci-failure, pull-request-available > > From > [https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/22853367/job/e461la5roy7edpsb] > > It looks like the thread-pool test. I'm going to disable it for the time > being. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4782) [C++] Prototype scalar and array expression types for developing deferred operator algebra
[ https://issues.apache.org/jira/browse/ARROW-4782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4782: -- Labels: pull-request-available (was: ) > [C++] Prototype scalar and array expression types for developing deferred > operator algebra > -- > > Key: ARROW-4782 > URL: https://issues.apache.org/jira/browse/ARROW-4782 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > I am beginning to develop a C++ API for describing analytical expressions on > scalars, arrays, and tables in the general style of Ibis > (https://docs.ibis-project.org/sql.html) which has been used to successfully > model SQL relational algebra as well as a broader variety of analytical > operations. This is a large project so I'm starting small -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4778) [C++/Python] manylinux1: Update Thrift to 0.12.0
[ https://issues.apache.org/jira/browse/ARROW-4778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4778: -- Labels: pull-request-available (was: ) > [C++/Python] manylinux1: Update Thrift to 0.12.0 > > > Key: ARROW-4778 > URL: https://issues.apache.org/jira/browse/ARROW-4778 > Project: Apache Arrow > Issue Type: Task > Components: C++, Packaging, Python >Reporter: Uwe L. Korn >Assignee: Uwe L. Korn >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4786) [C++/Python] Support better parallelisation in manylinux1 base build
[ https://issues.apache.org/jira/browse/ARROW-4786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4786: -- Labels: pull-request-available (was: ) > [C++/Python] Support better parallelisation in manylinux1 base build > > > Key: ARROW-4786 > URL: https://issues.apache.org/jira/browse/ARROW-4786 > Project: Apache Arrow > Issue Type: Task > Components: C++, Packaging, Python >Reporter: Uwe L. Korn >Assignee: Uwe L. Korn >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > Currently we're building some dependencies single-threaded but could build > them with much higher parallelisation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4791) Unused dependencies in arrow and datafusion
[
https://issues.apache.org/jira/browse/ARROW-4791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4791:
--
Labels: pull-request-available (was: )
> Unused dependencies in arrow and datafusion
> ---
>
> Key: ARROW-4791
> URL: https://issues.apache.org/jira/browse/ARROW-4791
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Rust
>Reporter: Yu Ding
>Priority: Minor
> Labels: pull-request-available
>
> One unused dependency found in rust/arrow
> {{[dependencies]}}
> {{ bytes = "0.4"}}
> Another unused dependency found in rust/datafusion
> {{[dependencies]}}
> {{ clap = "2.31.2"}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4793) [Ruby] Suppress unused variable warning
[ https://issues.apache.org/jira/browse/ARROW-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4793: -- Labels: pull-request-available (was: ) > [Ruby] Suppress unused variable warning > --- > > Key: ARROW-4793 > URL: https://issues.apache.org/jira/browse/ARROW-4793 > Project: Apache Arrow > Issue Type: Test > Components: Ruby >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Minor > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4792) [Ruby] Don't raise an error by #==
[ https://issues.apache.org/jira/browse/ARROW-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4792: -- Labels: pull-request-available (was: ) > [Ruby] Don't raise an error by #== > -- > > Key: ARROW-4792 > URL: https://issues.apache.org/jira/browse/ARROW-4792 > Project: Apache Arrow > Issue Type: Improvement > Components: Ruby >Affects Versions: 0.12.1 >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4790) [Python/Packaging] Update manylinux docker image in crossbow task
[
https://issues.apache.org/jira/browse/ARROW-4790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4790:
--
Labels: pull-request-available (was: )
> [Python/Packaging] Update manylinux docker image in crossbow task
> -
>
> Key: ARROW-4790
> URL: https://issues.apache.org/jira/browse/ARROW-4790
> Project: Apache Arrow
> Issue Type: Task
> Components: Python
>Reporter: Krisztian Szucs
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> to {{ARROW - 4778}} see
> https://github.com/apache/arrow/pull/3823#issuecomment-470129575
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4789) [C++] Deprecate and and later remove arrow::io::ReadableFileInterface
[ https://issues.apache.org/jira/browse/ARROW-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4789: -- Labels: pull-request-available (was: ) > [C++] Deprecate and and later remove arrow::io::ReadableFileInterface > - > > Key: ARROW-4789 > URL: https://issues.apache.org/jira/browse/ARROW-4789 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > See arrow/io/interfaces.h. This is a legacy alias -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4645) [C++/Packaging] Ship Gandiva with OSX and Windows wheels
[ https://issues.apache.org/jira/browse/ARROW-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4645: -- Labels: pull-request-available (was: ) > [C++/Packaging] Ship Gandiva with OSX and Windows wheels > > > Key: ARROW-4645 > URL: https://issues.apache.org/jira/browse/ARROW-4645 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ - Gandiva, Packaging >Reporter: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > Gandiva is only installed via the linux wheels, We should support it on all > platforms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4640) [Python] Add docker-compose configuration to build and test the project without pandas installed
[ https://issues.apache.org/jira/browse/ARROW-4640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4640: -- Labels: pull-request-available (was: ) > [Python] Add docker-compose configuration to build and test the project > without pandas installed > > > Key: ARROW-4640 > URL: https://issues.apache.org/jira/browse/ARROW-4640 > Project: Apache Arrow > Issue Type: Improvement > Components: Python >Reporter: Wes McKinney >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > This will help detect problems where we have unintentionally introduced a > hard dependency -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4796) [Flight][Python] segfault in simple server implementation
[ https://issues.apache.org/jira/browse/ARROW-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4796: -- Labels: flight pull-request-available (was: flight) > [Flight][Python] segfault in simple server implementation > - > > Key: ARROW-4796 > URL: https://issues.apache.org/jira/browse/ARROW-4796 > Project: Apache Arrow > Issue Type: Bug > Components: FlightRPC, Python >Reporter: David Li >Assignee: David Li >Priority: Major > Labels: flight, pull-request-available > Fix For: 0.13.0 > > > Python segfaults if you implement a Flight server that returns a data stream > but does not keep a reference to the underlying data source (the Table, > RecordBatch, etc). The Flight bindings themselves do not keep a reference to > the object, so the server will segfault as the memory has been reclaimed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4797) [Plasma] Avoid store crash if not enough memory is available
[ https://issues.apache.org/jira/browse/ARROW-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4797: -- Labels: pull-request-available (was: ) > [Plasma] Avoid store crash if not enough memory is available > > > Key: ARROW-4797 > URL: https://issues.apache.org/jira/browse/ARROW-4797 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ - Plasma >Reporter: Philipp Moritz >Assignee: Philipp Moritz >Priority: Major > Labels: pull-request-available > > Currently, the plasma server exists with a fatal check if not enough memory > is available. This can lead to errors that are hard to diagnose, see > [https://github.com/ray-project/ray/issues/3670] > Instead, we should keep the store alive in these circumstances, taking up > some of the remaining memory and allow the client to check if enough memory > has been allocating. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4801) [GLib] Suppress pkgconfig.generate() warnings
[ https://issues.apache.org/jira/browse/ARROW-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4801: -- Labels: pull-request-available (was: ) > [GLib] Suppress pkgconfig.generate() warnings > - > > Key: ARROW-4801 > URL: https://issues.apache.org/jira/browse/ARROW-4801 > Project: Apache Arrow > Issue Type: Improvement > Components: GLib >Affects Versions: 0.12.1 >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Minor > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4807) [Rust] Fix csv_writer benchmark
[ https://issues.apache.org/jira/browse/ARROW-4807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4807: -- Labels: pull-request-available (was: ) > [Rust] Fix csv_writer benchmark > --- > > Key: ARROW-4807 > URL: https://issues.apache.org/jira/browse/ARROW-4807 > Project: Apache Arrow > Issue Type: Bug > Components: Rust >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Trivial > Labels: pull-request-available > Fix For: 0.13.0 > > > CSV writer benchmark suite isn't working due to `RecordBatch::try_new` > returns a `Result` now. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4713) [C++] Improve C++ Orc Adapter performance and memory footprint
[ https://issues.apache.org/jira/browse/ARROW-4713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4713: -- Labels: pull-request-available (was: ) > [C++] Improve C++ Orc Adapter performance and memory footprint > -- > > Key: ARROW-4713 > URL: https://issues.apache.org/jira/browse/ARROW-4713 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Yurui Zhou >Assignee: Yurui Zhou >Priority: Major > Labels: pull-request-available > > Currently the Arrow C++ provide a naive adapter implementation that allow > user to read orc file to Arrow RecordBatch. However, this implementation have > several drawbacks: > * Inefficient conversion that incurs huge memcpy overhead > ** currently the ORC adapter are performing byte to byte memcpy to move data > to ORC VectorBatch to Arrow RecordBatch regardless of the fact that ORC > VectorBatch shares the same memory layout with Arrow in most of the Data Types > * Huge memory footprint because the lack of TableReader implementation > ** The ORC adapter currently only allow user to read data with the unit of > stripe. However, as a columnar format with high compression ration, data read > from a ORC stripe can potential takes over gigabytes of memory, which makes > the ORC adapter not quite usable in production environment. > Here we propose a new ORC adapter implementation to fix the issues mentioned > above: > * To reduce conversion overhead, instead of performing naive data copy, the > new adapter would be able to fully taking advantage of the memory layout > similarity between ORC VectorBatch and Arrow RecordBatch. Namely the new > adapter will perform pointer manipulation to transfer the memory ownership > from VectorBatch to Arrow RecordBatch whenever possible. > * The new ORC Adapter would be able to provide user a row level granularity > when reading data from Orc File. The user should be able to specify how many > rows should be expected on output RecordBatch and the ORC Adapter should make > sure no more the requested number of rows would be returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4808) [Java][Vector] Convenience methods for setting decimal vector
[ https://issues.apache.org/jira/browse/ARROW-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4808: -- Labels: pull-request-available (was: ) > [Java][Vector] Convenience methods for setting decimal vector > - > > Key: ARROW-4808 > URL: https://issues.apache.org/jira/browse/ARROW-4808 > Project: Apache Arrow > Issue Type: Task >Reporter: Praveen Kumar Desabandu >Assignee: Praveen Kumar Desabandu >Priority: Major > Labels: pull-request-available > > Provide more convenience methods to set decimal vector, specifically > # Accept arrow buffers encode in little endian bytes that are of size < 16 > bytes > # Accept arrow buffers that are encoded in big endian and could of size <=16 > bytes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4774) [C++][Parquet] crash writing nested array to parquet
[
https://issues.apache.org/jira/browse/ARROW-4774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4774:
--
Labels: pull-request-available (was: )
> [C++][Parquet] crash writing nested array to parquet
>
>
> Key: ARROW-4774
> URL: https://issues.apache.org/jira/browse/ARROW-4774
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.11.1, 0.12.1
> Environment: Windows 10 16299.431, Python 3.6.4 64-bit, pyarrow 0.11.1
> Windows Linux (WSL) Ubuntu 18.04.1, Python 3.6.5 64-bit, pyarrow 0.12.1
>Reporter: Stephen Gallagher
>Assignee: Francois Saint-Jacques
>Priority: Blocker
> Labels: pull-request-available
>
> When writing a table to a parquet file that contains both flat arrays and
> nested array data it causes the Python to crash
> Reproducer:
> {code:python}
> import pyarrow as pa
> import pyarrow.parquet as pq
> import numpy as np
> array1 = np.array([0, 1, 2], dtype=np.uint8)
> array2 = np.array([[0,1,2], [3, 4, 5]], dtype=np.uint8).T
> t1 = pa.uint8()
> t2 = pa.list_(pa.uint8())
> fields = [
> pa.field('a1', t1),
> pa.field('a2', t2)
> ]
> myschema = pa.schema(fields)
> mytable = pa.Table.from_arrays([
> pa.array(array1, type=t1),
> pa.array([array2[:,0], array2[:,1]], type=t2)],
> schema=myschema)
> pq.write_table(mytable, 'example.parquet')
> {code}
> Windows 10 (Python 3.6.4 64-bit, pyarrow 0.11.1) crash code:
> {code:bash}
> Process finished with exit code -1073741819 (0xC005)
> {code}
> WSL (Python 3.6.5 64-bit, pyarrow 0.12.1) Crash code:
> {code:bash}
> Segmentation fault (core dumped)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4810: -- Labels: pull-request-available (was: ) > [Format][C++] Add "LargeList" type with 64-bit offsets > -- > > Key: ARROW-4810 > URL: https://issues.apache.org/jira/browse/ARROW-4810 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, Format >Reporter: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 0.14.0 > > > Mentioned in https://github.com/apache/arrow/issues/3845 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4681) [Rust] [DataFusion] Implement parallel query execution using threads
[ https://issues.apache.org/jira/browse/ARROW-4681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4681: -- Labels: pull-request-available (was: ) > [Rust] [DataFusion] Implement parallel query execution using threads > > > Key: ARROW-4681 > URL: https://issues.apache.org/jira/browse/ARROW-4681 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust, Rust - DataFusion >Affects Versions: 0.12.0 >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > I am planning on tackling this soon. The basic plan is to change the table > scan() method to return multiple scanners (one per partition) so that the > query execution can process each partition on a separate thread. > This will involve changing some of the DataFusion APIs to use Arc instead of > Rc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4486) [Python][CUDA] pyarrow.cuda.Context.foreign_buffer should have a `base=None` argument
[ https://issues.apache.org/jira/browse/ARROW-4486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4486: -- Labels: pull-request-available (was: ) > [Python][CUDA] pyarrow.cuda.Context.foreign_buffer should have a `base=None` > argument > - > > Key: ARROW-4486 > URL: https://issues.apache.org/jira/browse/ARROW-4486 > Project: Apache Arrow > Issue Type: Improvement > Components: Python >Reporter: Pearu Peterson >Assignee: Pearu Peterson >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > Time Spent: 2h > Remaining Estimate: 2h > > Similar to `pyarrow.foreign_buffer`, we need to keep the owner of cuda memory > alive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4466) [Rust] [DataFusion] Add support for Parquet data sources
[ https://issues.apache.org/jira/browse/ARROW-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4466: -- Labels: pull-request-available (was: ) > [Rust] [DataFusion] Add support for Parquet data sources > > > Key: ARROW-4466 > URL: https://issues.apache.org/jira/browse/ARROW-4466 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust, Rust - DataFusion >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Major > Labels: pull-request-available > Fix For: 0.14.0 > > > As a user, I would like to be able to run SQL queries against Parquet files. > For the initial implementation we can just support primitive types. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-2409) [Rust] Test for build warnings, remove current warnings
[ https://issues.apache.org/jira/browse/ARROW-2409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-2409: -- Labels: pull-request-available (was: ) > [Rust] Test for build warnings, remove current warnings > --- > > Key: ARROW-2409 > URL: https://issues.apache.org/jira/browse/ARROW-2409 > Project: Apache Arrow > Issue Type: Task > Components: Rust >Reporter: Maximilian Roos >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > Test for build warnings, remove current warnings -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4813) [Ruby] Add tests for #== and #!=
[ https://issues.apache.org/jira/browse/ARROW-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4813: -- Labels: pull-request-available (was: ) > [Ruby] Add tests for #== and #!= > > > Key: ARROW-4813 > URL: https://issues.apache.org/jira/browse/ARROW-4813 > Project: Apache Arrow > Issue Type: Test > Components: Ruby >Affects Versions: 0.12.1 >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Minor > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4664) [C++] DCHECK macro conditions are evaluated in release builds
[
https://issues.apache.org/jira/browse/ARROW-4664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4664:
--
Labels: pull-request-available (was: )
> [C++] DCHECK macro conditions are evaluated in release builds
> -
>
> Key: ARROW-4664
> URL: https://issues.apache.org/jira/browse/ARROW-4664
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Benjamin Kietzman
>Assignee: Wes McKinney
>Priority: Trivial
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> {{DCHECK(potentially_expensive())}} will evaluate the argument even in
> release mode, and is used in several places with the assumption that it will
> do so (which means removing the guarantee of evaluation causes numerous
> failures). By contrast, most debug assertion macros elide their arguments
> entirely ({{.assert}}, {{}}) in release mode
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3954) [Rust] Add Slice to Array and ArrayData
[
https://issues.apache.org/jira/browse/ARROW-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-3954:
--
Labels: pull-request-available (was: )
> [Rust] Add Slice to Array and ArrayData
> ---
>
> Key: ARROW-3954
> URL: https://issues.apache.org/jira/browse/ARROW-3954
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Rust
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> Similar to C++, we should be able to construct zero-copy slice from {{Array}}
> and {{ArrayData}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4822) [C++/Python] pyarrow.Table.equals segmentation fault on None
[
https://issues.apache.org/jira/browse/ARROW-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4822:
--
Labels: pull-request-available (was: )
> [C++/Python] pyarrow.Table.equals segmentation fault on None
>
>
> Key: ARROW-4822
> URL: https://issues.apache.org/jira/browse/ARROW-4822
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
>Affects Versions: 0.12.1
>Reporter: Andreas
>Assignee: Uwe L. Korn
>Priority: Trivial
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> Calling {{pyarrow.Table.equals}} with {{None}} causes a segmentation fault;
> this should be caught.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3735) [Python] Proper error handling in _ensure_type
[
https://issues.apache.org/jira/browse/ARROW-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-3735:
--
Labels: pull-request-available (was: )
> [Python] Proper error handling in _ensure_type
> --
>
> Key: ARROW-3735
> URL: https://issues.apache.org/jira/browse/ARROW-3735
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Reporter: Krisztian Szucs
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> We have multiple _ensure_type like functions, the in defined in array.pxi
> bypasses None which causes segfault in the following example:
> {code}
> pa.array([1, 2, 3]).cast(None)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-2905) [C++] Investigate if the *_data_ pointers used in Builder classes improve performance on hot paths
[ https://issues.apache.org/jira/browse/ARROW-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-2905: -- Labels: pull-request-available (was: ) > [C++] Investigate if the *_data_ pointers used in Builder classes improve > performance on hot paths > -- > > Key: ARROW-2905 > URL: https://issues.apache.org/jira/browse/ARROW-2905 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 0.14.0 > > > See [~alendit] comment in > https://github.com/apache/arrow/pull/2315#discussion_r204668176 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4828) [Python] manylinux1 docker-compose context should be python/manylinux1
[
https://issues.apache.org/jira/browse/ARROW-4828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4828:
--
Labels: pull-request-available (was: )
> [Python] manylinux1 docker-compose context should be python/manylinux1
> --
>
> Key: ARROW-4828
> URL: https://issues.apache.org/jira/browse/ARROW-4828
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
>
> Currently it doesn't find the {{scripts}} folder on running {{docker-compose
> build python-manylinux1}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4826) [Go] export Flush method for CSV writer
[ https://issues.apache.org/jira/browse/ARROW-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4826: -- Labels: pull-request-available (was: ) > [Go] export Flush method for CSV writer > --- > > Key: ARROW-4826 > URL: https://issues.apache.org/jira/browse/ARROW-4826 > Project: Apache Arrow > Issue Type: Improvement > Components: Go >Reporter: Sebastien Binet >Assignee: Anson Qian >Priority: Major > Labels: pull-request-available > > it should be possible to flush out to the underlying io.Writer the data that > has been passed to the arrow/csv.Writer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4734) [Go] Add option to write a header for CSV writer
[ https://issues.apache.org/jira/browse/ARROW-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4734: -- Labels: pull-request-available (was: ) > [Go] Add option to write a header for CSV writer > > > Key: ARROW-4734 > URL: https://issues.apache.org/jira/browse/ARROW-4734 > Project: Apache Arrow > Issue Type: Improvement > Components: Go >Reporter: Anson Qian >Assignee: Anson Qian >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4776) [C++] DictionaryBuilder should support bootstrapping from an existing dict type
[ https://issues.apache.org/jira/browse/ARROW-4776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4776: -- Labels: pull-request-available (was: ) > [C++] DictionaryBuilder should support bootstrapping from an existing dict > type > --- > > Key: ARROW-4776 > URL: https://issues.apache.org/jira/browse/ARROW-4776 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Francois Saint-Jacques >Priority: Minor > Labels: pull-request-available > Fix For: 0.14.0 > > > This would mean adding a new DictionaryBuilder constructor that receives a > dictionary type and performs a lazy deep copy if there's any modification. > We'll have to investigate how this translate in API ergonomics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-1639) [Python] More efficient serialization for RangeIndex in serialize_pandas
[ https://issues.apache.org/jira/browse/ARROW-1639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-1639: -- Labels: pull-request-available (was: ) > [Python] More efficient serialization for RangeIndex in serialize_pandas > > > Key: ARROW-1639 > URL: https://issues.apache.org/jira/browse/ARROW-1639 > Project: Apache Arrow > Issue Type: Improvement > Components: Python >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4724) [C++] Python not being built nor test under MinGW builds
[
https://issues.apache.org/jira/browse/ARROW-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4724:
--
Labels: pull-request-available (was: )
> [C++] Python not being built nor test under MinGW builds
>
>
> Key: ARROW-4724
> URL: https://issues.apache.org/jira/browse/ARROW-4724
> Project: Apache Arrow
> Issue Type: Test
> Components: C++
>Reporter: Javier Luraschi
>Priority: Major
> Labels: pull-request-available
>
> Follow up to needed for
> [arrow/pull/3693/files|https://github.com/apache/arrow/pull/3693/files].
> appveyor-cpp-build-mingw.bat has not yet enabled Python tests, need to revert,
> -DARROW_PYTHON=OFF
> Suggestion was to use,
> {code:java}
> diff --git a/ci/appveyor-cpp-build-mingw.bat b/ci/appveyor-cpp-build-mingw.bat
> index 06e8b7f7..3a853031 100644
> --- a/ci/appveyor-cpp-build-mingw.bat
> +++ b/ci/appveyor-cpp-build-mingw.bat
> @@ -24,6 +24,15 @@ set INSTALL_DIR=%HOMEDRIVE%%HOMEPATH%\install
> set PATH=%INSTALL_DIR%\bin;%PATH%
> set PKG_CONFIG_PATH=%INSTALL_DIR%\lib\pkgconfig
> +for /f "usebackq" %%v in (`python3 -c "import sys; print('.'.join(map(str,
> sys.version_info[0:2])))"`) do (
> + set PYTHON_VERSION=%%v
> +)
> +
> +set PYTHONHOME=%MINGW_PREFIX%\lib\python%PYTHON_VERSION%
> +set PYTHONPATH=%PYTHONHOME%
> +set
> PYTHONPATH=%PYTHONPATH%;%MINGW_PREFIX%\lib\python%PYTHON_VERSION%\lib-dynload
> +set
> PYTHONPATH=%PYTHONPATH%;%MINGW_PREFIX%\lib\python%PYTHON_VERSION%\site-packages
> +
> {code}
> However, this suggestion currently trigger a built error in Travis,
> {code:java}
> [ 43%] Building CXX object
> src/arrow/CMakeFiles/arrow_objlib.dir/ipc/json-simple.cc.obj
> [ 44%] Building CXX object
> src/arrow/CMakeFiles/arrow_objlib.dir/ipc/message.cc.obj
> [ 44%] Building CXX object
> src/arrow/CMakeFiles/arrow_objlib.dir/ipc/metadata-internal.cc.obj
> [ 45%] Building CXX object
> src/arrow/CMakeFiles/arrow_objlib.dir/ipc/reader.cc.obj
> [ 45%] Building CXX object
> src/arrow/CMakeFiles/arrow_objlib.dir/ipc/writer.cc.obj
> [ 45%] Built target arrow_objlib
> make: *** [Makefile:141: all] Error 2
> C:\projects\arrow\cpp\build>goto scriptexit{code}
> Therefore, additional investigation is needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4834) [R] Feature flag to disable parquet
[ https://issues.apache.org/jira/browse/ARROW-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4834: -- Labels: pull-request-available (was: ) > [R] Feature flag to disable parquet > --- > > Key: ARROW-4834 > URL: https://issues.apache.org/jira/browse/ARROW-4834 > Project: Apache Arrow > Issue Type: Improvement > Components: R >Reporter: Javier Luraschi >Priority: Major > Labels: pull-request-available > > Support for a ARROW_R_PARQUET_OFF feature flag that will disable building the > R package with parquet support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-2119) [C++][Java] Handle Arrow stream with zero record batch
[
https://issues.apache.org/jira/browse/ARROW-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-2119:
--
Labels: pull-request-available (was: )
> [C++][Java] Handle Arrow stream with zero record batch
> --
>
> Key: ARROW-2119
> URL: https://issues.apache.org/jira/browse/ARROW-2119
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Java
>Reporter: Jingyuan Wang
>Assignee: Wes McKinney
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> It looks like currently many places of the code assume that there needs to be
> at least one record batch for streaming format. Is zero-recordbatch not
> supported by design?
> e.g.
> [https://github.com/apache/arrow/blob/master/java/tools/src/main/java/org/apache/arrow/tools/StreamToFile.java#L45]
> {code:none}
> public static void convert(InputStream in, OutputStream out) throws
> IOException {
> BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
> try (ArrowStreamReader reader = new ArrowStreamReader(in, allocator)) {
> VectorSchemaRoot root = reader.getVectorSchemaRoot();
> // load the first batch before instantiating the writer so that we have
> any dictionaries
> if (!reader.loadNextBatch()) {
> throw new IOException("Unable to read first record batch");
> }
> ...
> {code}
> Pyarrow-0.8.0 does not load 0-recordbatch stream either. It would throw an
> exception originated from
> [https://github.com/apache/arrow/blob/a95465b8ce7a32feeaae3e13d0a64102ffa590d9/cpp/src/arrow/table.cc#L309:]
> {code:none}
> Status Table::FromRecordBatches(const
> std::vector>& batches,
> std::shared_ptr* table) {
> if (batches.size() == 0) {
> return Status::Invalid("Must pass at least one record batch");
> }
> ...{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4820) [Python] hadoop class path derived not correct
[ https://issues.apache.org/jira/browse/ARROW-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4820: -- Labels: pull-request-available (was: ) > [Python] hadoop class path derived not correct > -- > > Key: ARROW-4820 > URL: https://issues.apache.org/jira/browse/ARROW-4820 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Affects Versions: 0.12.0, 0.12.1 >Reporter: Tiger068 >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > Attachments: image-2019-03-11-16-39-25-195.png > > > in hdfs.py, the method _derive_hadoop_classpath add jar files under > $HADOOP_HOME into hadoop classpath,but the hadoop config directory is not > contained in classpath. > > when hadoop HA mode enabled,the hdfs uri like this: hdfs://ns > when the HADOOP_CONF_DIR directory is not in the hadoop classpath,the libhdfs > can not locate the right hdfs-site.xml, in the HA mode, hdfs service name > was parsed as host name ,it is not correct > !image-2019-03-11-16-39-25-195.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4835) [GLib] Add boolean operations
[ https://issues.apache.org/jira/browse/ARROW-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4835: -- Labels: pull-request-available (was: ) > [GLib] Add boolean operations > - > > Key: ARROW-4835 > URL: https://issues.apache.org/jira/browse/ARROW-4835 > Project: Apache Arrow > Issue Type: New Feature > Components: GLib >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4837) [C++] Support c++filt on a custom path in the run-test.sh script
[ https://issues.apache.org/jira/browse/ARROW-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4837: -- Labels: pull-request-available (was: ) > [C++] Support c++filt on a custom path in the run-test.sh script > > > Key: ARROW-4837 > URL: https://issues.apache.org/jira/browse/ARROW-4837 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Krisztian Szucs >Priority: Major > Labels: pull-request-available > > On conda this is CXXFILT=/opt/conda/bin/x86_64-conda_cos6-linux-gnu-c++filt -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4673) [C++] Implement AssertDatumEquals
[ https://issues.apache.org/jira/browse/ARROW-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4673: -- Labels: pull-request-available (was: ) > [C++] Implement AssertDatumEquals > - > > Key: ARROW-4673 > URL: https://issues.apache.org/jira/browse/ARROW-4673 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Francois Saint-Jacques >Assignee: Francois Saint-Jacques >Priority: Minor > Labels: pull-request-available > Fix For: 0.14.0 > > > Aggregate tests could benefit from this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4825) [Python][C++] MemoryPool is destructed before deallocating its buffers leads to segfault
[ https://issues.apache.org/jira/browse/ARROW-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4825: -- Labels: pull-request-available (was: ) > [Python][C++] MemoryPool is destructed before deallocating its buffers leads > to segfault > - > > Key: ARROW-4825 > URL: https://issues.apache.org/jira/browse/ARROW-4825 > Project: Apache Arrow > Issue Type: Bug > Components: C++, Python >Affects Versions: 0.13.0 >Reporter: Pearu Peterson >Priority: Major > Labels: pull-request-available > > Consider the following test function: > ``` > def test_memory_pool(): > import pyarrow as pa > pool = pa.logging_memory_pool(pa.default_memory_pool()) > buf = pa.allocate_buffer(10, memory_pool=pool) > ``` > that will fail with segfault when `pool` is garbage collected before `buf`. > However, the following test function succeeds: > ``` > def test_memory_pool(): > import pyarrow as pa > pool = pa.logging_memory_pool(pa.default_memory_pool()) > buf = pa.allocate_buffer(10, memory_pool=pool) > del buf > ``` > because all buffers are freed before `pool` destruction. > To fix this issue, the pool instance should be attached to buffer instances > that the pool is creating. This will ensure that `pool` will be alive until > all its buffers are destroyed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-2501) [Java] Remove Jackson from compile-time dependencies for arrow-vector
[ https://issues.apache.org/jira/browse/ARROW-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-2501: -- Labels: pull-request-available (was: ) > [Java] Remove Jackson from compile-time dependencies for arrow-vector > - > > Key: ARROW-2501 > URL: https://issues.apache.org/jira/browse/ARROW-2501 > Project: Apache Arrow > Issue Type: Improvement > Components: Java >Affects Versions: 0.9.0 >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Minor > Labels: pull-request-available > Fix For: 0.14.0 > > > I would like to upgrade Jackson to the latest version (2.9.5). If there are > no objections I will create a PR (it is literally just changing the version > number in the pom - no code changes required). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4421) [Flight][C++] Handle large Flight data messages
[ https://issues.apache.org/jira/browse/ARROW-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4421: -- Labels: pull-request-available (was: ) > [Flight][C++] Handle large Flight data messages > --- > > Key: ARROW-4421 > URL: https://issues.apache.org/jira/browse/ARROW-4421 > Project: Apache Arrow > Issue Type: Bug > Components: C++, FlightRPC >Reporter: Wes McKinney >Assignee: David Li >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > I believe the message payloads are currently limited to 4MB by default, see > one developer's discussion here: > https://nanxiao.me/en/message-length-setting-in-grpc/ > While it is a good idea to break large messages into smaller ones, we will > need to address how to gracefully send larger payloads that may be provided > by a user's server implementation. Either we can increase the limit or break > up the record batches into smaller chunks in the Flight server base (or both, > of course) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4735) [Go] Benchmark strconv.Format vs. fmt.Sprintf for CSV writer
[
https://issues.apache.org/jira/browse/ARROW-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4735:
--
Labels: pull-request-available (was: )
> [Go] Benchmark strconv.Format vs. fmt.Sprintf for CSV writer
>
>
> Key: ARROW-4735
> URL: https://issues.apache.org/jira/browse/ARROW-4735
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Go
>Reporter: Anson Qian
>Assignee: Sebastien Binet
>Priority: Major
> Labels: pull-request-available
>
> Need test out strconv.Format\{Bool,Float,Int,Uint} instead of fmt.Sprintf and
> see if we can improve write performance
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4846) [Java] Update Jackson to 2.9.8
[ https://issues.apache.org/jira/browse/ARROW-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4846: -- Labels: pull-request-available (was: ) > [Java] Update Jackson to 2.9.8 > -- > > Key: ARROW-4846 > URL: https://issues.apache.org/jira/browse/ARROW-4846 > Project: Apache Arrow > Issue Type: Improvement > Components: Java >Reporter: Wes McKinney >Assignee: Andy Grove >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > We are looking at removing Jackson from arrow-vector dependencies in > ARROW-2501 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-3364) [Doc] Document docker compose setup
[ https://issues.apache.org/jira/browse/ARROW-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-3364: -- Labels: pull-request-available (was: ) > [Doc] Document docker compose setup > --- > > Key: ARROW-3364 > URL: https://issues.apache.org/jira/browse/ARROW-3364 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Krisztian Szucs >Assignee: Francois Saint-Jacques >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > Introduced by https://github.com/apache/arrow/pull/2572 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4811) [C++] An incorrect dependency leads "ninja" to re-evaluate steps unnecessarily on subsequent calls
[
https://issues.apache.org/jira/browse/ARROW-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4811:
--
Labels: pull-request-available (was: )
> [C++] An incorrect dependency leads "ninja" to re-evaluate steps
> unnecessarily on subsequent calls
> --
>
> Key: ARROW-4811
> URL: https://issues.apache.org/jira/browse/ARROW-4811
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Wes McKinney
>Assignee: Wes McKinney
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> Not sure about the root cause yet but here are the 5 steps that are
> re-executing
> {code}
> $ ninja -v
> [1/5] /usr/bin/ccache /usr/bin/g++ -DARROW_EXTRA_ERROR_CONTEXT
> -DARROW_JEMALLOC
> -DARROW_JEMALLOC_INCLUDE_DIR=/home/wesm/code/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep/dist//include
> -DARROW_NO_DEPRECATED_API -DARROW_PYTHON_EXPORTING -DARROW_USE_GLOG
> -DARROW_USE_SIMD -DARROW_WITH_BROTLI -DARROW_WITH_BZ2 -DARROW_WITH_LZ4
> -DARROW_WITH_SNAPPY -DARROW_WITH_ZLIB -DARROW_WITH_ZSTD -Isrc -I../src
> -isystem /home/wesm/cpp-toolchain/include -isystem
> gbenchmark_ep/src/gbenchmark_ep-install/include -isystem
> jemalloc_ep-prefix/src -isystem ../thirdparty/hadoop/include -isystem
> orc_ep-install/include -isystem /home/wesm/cpp-toolchain/include/thrift
> -isystem
> /home/wesm/miniconda/envs/arrow-3.7/lib/python3.7/site-packages/numpy/core/include
> -isystem /home/wesm/miniconda/envs/arrow-3.7/include/python3.7m
> -Wno-noexcept-type -fdiagnostics-color=always -O3 -DNDEBUG -Wall
> -Wno-unused-variable -msse4.2 -fno-omit-frame-pointer -O3 -DNDEBUG -fPIC
> -std=gnu++11 -MD -MT
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/flight.cc.o -MF
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/flight.cc.o.d -o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/flight.cc.o -c
> ../src/arrow/python/flight.cc
> [2/5] : && /usr/bin/ccache /home/wesm/miniconda/envs/arrow-3.7/bin/cmake -E
> remove release/libarrow_python.a && /usr/bin/ccache /usr/bin/ar qc
> release/libarrow_python.a
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/arrow_to_pandas.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/benchmark.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/common.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/config.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/decimal.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/deserialize.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/helpers.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/inference.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/init.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/io.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/numpy_convert.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/numpy_to_arrow.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/python_to_arrow.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/pyarrow.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/serialize.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/flight.cc.o &&
> /usr/bin/ccache /usr/bin/ranlib release/libarrow_python.a && :
> [3/5] : && /usr/bin/ccache /usr/bin/g++ -fPIC -Wno-noexcept-type
> -fdiagnostics-color=always -O3 -DNDEBUG -Wall -Wno-unused-variable -msse4.2
> -fno-omit-frame-pointer -O3 -DNDEBUG -shared
> -Wl,-soname,libarrow_python.so.13 -o release/libarrow_python.so.13.0.0
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/arrow_to_pandas.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/benchmark.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/common.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/config.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/decimal.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/deserialize.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/helpers.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/inference.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/init.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/io.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/numpy_convert.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/numpy_to_arrow.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/python_to_arrow.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/pyarrow.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/serialize.cc.o
> src/arrow/python/CMakeFiles/arrow_python_objlib.dir/flight.cc.o
> -Wl,-rpath,/home/wesm/code/arrow/cpp/build/re
[jira] [Updated] (ARROW-4850) [CI] Integration test failures do not fail the Travis CI build
[ https://issues.apache.org/jira/browse/ARROW-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4850: -- Labels: pull-request-available (was: ) > [CI] Integration test failures do not fail the Travis CI build > -- > > Key: ARROW-4850 > URL: https://issues.apache.org/jira/browse/ARROW-4850 > Project: Apache Arrow > Issue Type: Bug > Components: Continuous Integration >Reporter: Wes McKinney >Assignee: Francois Saint-Jacques >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > See https://github.com/apache/arrow/pull/3871 > These changes fail the build, but it is reported as success > The errors can be seen in https://travis-ci.org/apache/arrow/jobs/505028161 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4831) [C++] CMAKE_AR is not passed to ZSTD thirdparty dependency
[ https://issues.apache.org/jira/browse/ARROW-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4831: -- Labels: pull-request-available (was: ) > [C++] CMAKE_AR is not passed to ZSTD thirdparty dependency > --- > > Key: ARROW-4831 > URL: https://issues.apache.org/jira/browse/ARROW-4831 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Krisztian Szucs >Priority: Major > Labels: pull-request-available > > ZSTD_CMAKE_ARGS should utilize > https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L359 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4855) [Packaging] Generate default package version based on cpp tags in crossbow.py
[ https://issues.apache.org/jira/browse/ARROW-4855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4855: -- Labels: pull-request-available (was: ) > [Packaging] Generate default package version based on cpp tags in crossbow.py > - > > Key: ARROW-4855 > URL: https://issues.apache.org/jira/browse/ARROW-4855 > Project: Apache Arrow > Issue Type: Improvement > Components: Packaging >Reporter: Krisztian Szucs >Assignee: Krisztian Szucs >Priority: Major > Labels: pull-request-available > > It picked up wrong version because of the recent JS release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4859) [GLib] Add garrow_numeric_array_mean()
[ https://issues.apache.org/jira/browse/ARROW-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4859: -- Labels: pull-request-available (was: ) > [GLib] Add garrow_numeric_array_mean() > -- > > Key: ARROW-4859 > URL: https://issues.apache.org/jira/browse/ARROW-4859 > Project: Apache Arrow > Issue Type: New Feature > Components: GLib >Reporter: Yosuke Shiro >Assignee: Yosuke Shiro >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4858) [Flight][Python] Enable custom FlightDataStream in Python
[ https://issues.apache.org/jira/browse/ARROW-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4858: -- Labels: flight pull-request-available (was: flight) > [Flight][Python] Enable custom FlightDataStream in Python > - > > Key: ARROW-4858 > URL: https://issues.apache.org/jira/browse/ARROW-4858 > Project: Apache Arrow > Issue Type: Improvement > Components: FlightRPC, Python >Reporter: David Li >Assignee: David Li >Priority: Minor > Labels: flight, pull-request-available > Fix For: 0.14.0 > > > We should be able to provide a custom data stream as the result of Flight > do_get in Python. In particular, when returning data produced on the fly, or > when returning a large Pandas DataFrame, it'd be nice to provide data in > chunks as it becomes available, rather than having to copy everything into a > Table first. > On the Python side, a FlightDataStream wrapper that accepts RecordBatches > from a Python generator should suffice. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4839) [C#] Add NuGet support
[ https://issues.apache.org/jira/browse/ARROW-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4839: -- Labels: pull-request-available (was: ) > [C#] Add NuGet support > -- > > Key: ARROW-4839 > URL: https://issues.apache.org/jira/browse/ARROW-4839 > Project: Apache Arrow > Issue Type: Improvement > Components: C# >Reporter: Eric Erhardt >Assignee: Eric Erhardt >Priority: Major > Labels: pull-request-available > > We should add the metadata to the .csproj so we can create a NuGet package > without changing any source code. > Also, we should add any scripts and documentation on how to create the NuGet > package to allow ease of creation at release time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4012) [Documentation][C++] Document how to install Apache Arrow on MSYS2
[ https://issues.apache.org/jira/browse/ARROW-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4012: -- Labels: pull-request-available (was: ) > [Documentation][C++] Document how to install Apache Arrow on MSYS2 > -- > > Key: ARROW-4012 > URL: https://issues.apache.org/jira/browse/ARROW-4012 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, Documentation >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4637) [Python] Avoid importing Pandas unless necessary
[
https://issues.apache.org/jira/browse/ARROW-4637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4637:
--
Labels: pull-request-available (was: )
> [Python] Avoid importing Pandas unless necessary
>
>
> Key: ARROW-4637
> URL: https://issues.apache.org/jira/browse/ARROW-4637
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Affects Versions: 0.12.0
>Reporter: Antoine Pitrou
>Assignee: Wes McKinney
>Priority: Minor
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> Importing PyArrow is more than twice slower when Pandas is installed:
> {code}
> $ time python -c "import pyarrow"
> real 0m0,360s
> user 0m0,305s
> sys 0m0,037s
> $ time python -c "import sys; sys.modules['pandas'] = None; import pyarrow"
> real 0m0,144s
> user 0m0,124s
> sys 0m0,020s
> {code}
> We should only import Pandas when necessary, e.g. when asked to ingest or
> create Pandas data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4862) [GLib] Add GArrowCastOptions::allow-invalid-utf8 property
[ https://issues.apache.org/jira/browse/ARROW-4862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4862: -- Labels: pull-request-available (was: ) > [GLib] Add GArrowCastOptions::allow-invalid-utf8 property > - > > Key: ARROW-4862 > URL: https://issues.apache.org/jira/browse/ARROW-4862 > Project: Apache Arrow > Issue Type: New Feature > Components: GLib >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4705) [Rust] CSV reader should show line number and error message when failing to parse a line
[ https://issues.apache.org/jira/browse/ARROW-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4705: -- Labels: newbie pull-request-available (was: newbie) > [Rust] CSV reader should show line number and error message when failing to > parse a line > > > Key: ARROW-4705 > URL: https://issues.apache.org/jira/browse/ARROW-4705 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Affects Versions: 0.12.0 >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Major > Labels: newbie, pull-request-available > Fix For: 0.13.0 > > > We currently throw away the original error and do not report line number, > making it very difficult to debug. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4705) [Rust] CSV reader should show line number and error message when failing to parse a line
[ https://issues.apache.org/jira/browse/ARROW-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4705: -- Labels: newbie pull-request-available (was: newbie) > [Rust] CSV reader should show line number and error message when failing to > parse a line > > > Key: ARROW-4705 > URL: https://issues.apache.org/jira/browse/ARROW-4705 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Affects Versions: 0.12.0 >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Major > Labels: newbie, pull-request-available > Fix For: 0.13.0 > > > We currently throw away the original error and do not report line number, > making it very difficult to debug. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4865) [Rust] Support casting lists and primitives to lists
[ https://issues.apache.org/jira/browse/ARROW-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4865: -- Labels: pull-request-available (was: ) > [Rust] Support casting lists and primitives to lists > > > Key: ARROW-4865 > URL: https://issues.apache.org/jira/browse/ARROW-4865 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Affects Versions: 0.12.0 >Reporter: Neville Dipale >Assignee: Neville Dipale >Priority: Minor > Labels: pull-request-available > > This adds support for casting between list arrays and from primitive arrays > to single-value list arrays -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4871) [Flight][Java] Handle large Flight messages
[ https://issues.apache.org/jira/browse/ARROW-4871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4871: -- Labels: flight pull-request-available (was: flight) > [Flight][Java] Handle large Flight messages > --- > > Key: ARROW-4871 > URL: https://issues.apache.org/jira/browse/ARROW-4871 > Project: Apache Arrow > Issue Type: Bug > Components: FlightRPC, Java >Reporter: David Li >Assignee: David Li >Priority: Major > Labels: flight, pull-request-available > Fix For: 0.14.0 > > > Similarly to ARROW-4421, Java/gRPC needs to be configured to allow large > messages. The integration tests should also be updated to cover this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4751) [C++] Add pkg-config to conda_env_cpp.yml
[
https://issues.apache.org/jira/browse/ARROW-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4751:
--
Labels: pull-request-available (was: )
> [C++] Add pkg-config to conda_env_cpp.yml
> -
>
> Key: ARROW-4751
> URL: https://issues.apache.org/jira/browse/ARROW-4751
> Project: Apache Arrow
> Issue Type: Task
> Components: C++, Continuous Integration
>Reporter: Uwe L. Korn
>Assignee: Wes McKinney
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> Once the CMake refactor has been merged, we should add {{pkg-config}} to the
> dependencies as it should be also available for Windows now:
> https://github.com/conda-forge/pkg-config-feedstock/pull/27 This will
> simplify some packaging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4873) [C++] ARROW_DEPENDENCY_SOURCE should not be overridden to CONDA if ARROW_PACKAGE_PREFIX is set by user
[
https://issues.apache.org/jira/browse/ARROW-4873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4873:
--
Labels: pull-request-available (was: )
> [C++] ARROW_DEPENDENCY_SOURCE should not be overridden to CONDA if
> ARROW_PACKAGE_PREFIX is set by user
> --
>
> Key: ARROW-4873
> URL: https://issues.apache.org/jira/browse/ARROW-4873
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Wes McKinney
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> I use conda to manage Python dependencies but keep my C++ toolchain in a
> separate directory. This organizational scheme is incompatible with the new
> options after the CMake refactor
> I think if you pass {{-DARROW_PREFIX_PATH=$MY_CPP_TOOLCHAIN}} then this
> should not be overridden with {{$CONDA_PREFIX}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4251) [C++] Add option to use vendored Boost in verify-release-candidate.sh
[ https://issues.apache.org/jira/browse/ARROW-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4251: -- Labels: pull-request-available (was: ) > [C++] Add option to use vendored Boost in verify-release-candidate.sh > - > > Key: ARROW-4251 > URL: https://issues.apache.org/jira/browse/ARROW-4251 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > Got burned by this while verifying 0.12 RC2. It is useful to have the option > to use the system Boost during verification, but it would also be nice to opt > in to using the vendored Boost build -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4866) [C++] zstd ExternalProject failing on Windows
[ https://issues.apache.org/jira/browse/ARROW-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4866: -- Labels: ci-failure pull-request-available (was: ci-failure) > [C++] zstd ExternalProject failing on Windows > - > > Key: ARROW-4866 > URL: https://issues.apache.org/jira/browse/ARROW-4866 > Project: Apache Arrow > Issue Type: Bug > Components: C++, Packaging >Reporter: Uwe L. Korn >Assignee: Wes McKinney >Priority: Major > Labels: ci-failure, pull-request-available > Fix For: 0.13.0 > > > After > [https://github.com/apache/arrow/pull/3885|https://github.com/apache/arrow/pull/3885,] > the zstd ExternalProject is failing in the Windows builds, see > [https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/23063072/job/bd0gom16atlkddtx] > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4876) Port MutableBuffer to csharp
[ https://issues.apache.org/jira/browse/ARROW-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4876: -- Labels: pull-request-available (was: ) > Port MutableBuffer to csharp > > > Key: ARROW-4876 > URL: https://issues.apache.org/jira/browse/ARROW-4876 > Project: Apache Arrow > Issue Type: Task > Components: C# >Reporter: Prashanth Govindarajan >Priority: Major > Labels: pull-request-available > > C++ has a "MutableBuffer" that exposes the underlying T*. Port it to csharp. > It's an easy port. ArrowBuffer at the moment is exposed as ReadOnlyMemory. > The builder actually hands it a "Memory" object, so it ought to be a simple > change -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4056) [C++] Upgrade to boost-cpp 1.69.0 again
[ https://issues.apache.org/jira/browse/ARROW-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4056: -- Labels: pull-request-available (was: ) > [C++] Upgrade to boost-cpp 1.69.0 again > --- > > Key: ARROW-4056 > URL: https://issues.apache.org/jira/browse/ARROW-4056 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > EDIT: the issue has been present for a large portion of 2018. I found this > when merging the macOS C++ builds and changed the build type to Xcode 8.3: > https://travis-ci.org/wesm/arrow/jobs/469297420#L2856 > I reported the issue into conda-forge at > https://github.com/conda-forge/boost-cpp-feedstock/issues/40 > It seems that the Ray project worked around this earlier this year: > https://github.com/ray-project/ray/pull/1688 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4878) [C++] ARROW_DEPENDENCY_SOURCE=CONDA does not work properly with MSVC
[
https://issues.apache.org/jira/browse/ARROW-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4878:
--
Labels: pull-request-available (was: )
> [C++] ARROW_DEPENDENCY_SOURCE=CONDA does not work properly with MSVC
>
>
> Key: ARROW-4878
> URL: https://issues.apache.org/jira/browse/ARROW-4878
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Wes McKinney
>Assignee: Wes McKinney
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> The prefix must have {{\Library}} added to it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4870) [Ruby] gemspec has wrong msys2 dependency listed
[ https://issues.apache.org/jira/browse/ARROW-4870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4870: -- Labels: pull-request-available (was: ) > [Ruby] gemspec has wrong msys2 dependency listed > > > Key: ARROW-4870 > URL: https://issues.apache.org/jira/browse/ARROW-4870 > Project: Apache Arrow > Issue Type: Bug > Components: Ruby >Affects Versions: 0.12.1 >Reporter: Dominic Sisneros >Assignee: Dominic Sisneros >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > ruby gemspec has wrong msys2 dependency listed > change mys2_mingw_dependencies to correct package > pacman -Ss arrow > mingw32/mingw-w64-i686-arrow 0.11.1-1 > Apache Arrow is a cross-language development platform for in-memory data > (mingw-w64) > mingw64/mingw-w64-x86_64-arrow 0.11.1-1 [installed] > Apache Arrow is a cross-language development platform for in-memory data > (mingw-w64) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-3822) [C++] parquet::arrow::FileReader::GetRecordBatchReader has logical error on row groups with chunked columns
[ https://issues.apache.org/jira/browse/ARROW-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-3822: -- Labels: pull-request-available (was: ) > [C++] parquet::arrow::FileReader::GetRecordBatchReader has logical error on > row groups with chunked columns > --- > > Key: ARROW-3822 > URL: https://issues.apache.org/jira/browse/ARROW-3822 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Wes McKinney >Assignee: Benjamin Kietzman >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > If a BinaryArray / StringArray overflows a single column when reading a row > group, the resulting table will have a ChunkedArray. Using TableBatchReader > in > https://github.com/apache/arrow/blob/master/cpp/src/parquet/arrow/reader.cc#L176 > will therefore only return a part of the row group, discarding the rest -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4887) [GLib] Add garrow_array_count()
[ https://issues.apache.org/jira/browse/ARROW-4887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4887: -- Labels: pull-request-available (was: ) > [GLib] Add garrow_array_count() > --- > > Key: ARROW-4887 > URL: https://issues.apache.org/jira/browse/ARROW-4887 > Project: Apache Arrow > Issue Type: New Feature > Components: GLib >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4889) [C++] Add STATUS messages for Protobuf in CMake
[
https://issues.apache.org/jira/browse/ARROW-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4889:
--
Labels: pull-request-available (was: )
> [C++] Add STATUS messages for Protobuf in CMake
> ---
>
> Key: ARROW-4889
> URL: https://issues.apache.org/jira/browse/ARROW-4889
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> With Protobuf it can easily happen that {{protoc}} and {{libprotobuf}}
> mismatch. We should have some output about this in CMake to better debug this
> when users report issues.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4891) [C++] ZLIB include directories not added
[ https://issues.apache.org/jira/browse/ARROW-4891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4891: -- Labels: pull-request-available (was: ) > [C++] ZLIB include directories not added > > > Key: ARROW-4891 > URL: https://issues.apache.org/jira/browse/ARROW-4891 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Uwe L. Korn >Assignee: Uwe L. Korn >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > This causes a failing centos-7 build. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4892) Move SQL parser and planner into sql package
[ https://issues.apache.org/jira/browse/ARROW-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4892: -- Labels: pull-request-available (was: ) > Move SQL parser and planner into sql package > > > Key: ARROW-4892 > URL: https://issues.apache.org/jira/browse/ARROW-4892 > Project: Apache Arrow > Issue Type: Sub-task > Components: Rust, Rust - DataFusion >Affects Versions: 0.12.0 >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4893) [C++] conda packages should use $PREFIX inside of conda-build
[ https://issues.apache.org/jira/browse/ARROW-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4893: -- Labels: pull-request-available (was: ) > [C++] conda packages should use $PREFIX inside of conda-build > - > > Key: ARROW-4893 > URL: https://issues.apache.org/jira/browse/ARROW-4893 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, Packaging >Reporter: Uwe L. Korn >Assignee: Uwe L. Korn >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4896) [Rust] [DataFusion] Remove all uses of panic! from tests
[ https://issues.apache.org/jira/browse/ARROW-4896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4896: -- Labels: pull-request-available (was: ) > [Rust] [DataFusion] Remove all uses of panic! from tests > > > Key: ARROW-4896 > URL: https://issues.apache.org/jira/browse/ARROW-4896 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust, Rust - DataFusion >Affects Versions: 0.12.0 >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > Tests should use assert!(false) rather than panic!() -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4898) [C++] Old versions of FindProtobuf.cmake use ALL-CAPS for variables
[
https://issues.apache.org/jira/browse/ARROW-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-4898:
--
Labels: pull-request-available (was: )
> [C++] Old versions of FindProtobuf.cmake use ALL-CAPS for variables
> ---
>
> Key: ARROW-4898
> URL: https://issues.apache.org/jira/browse/ARROW-4898
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> We only need to handle {{PROTOBUF_PROTOC_LIBRARY}} vs
> {{Protobuf_PROTOC_LIBRARY}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-4899) [Rust] [DataFusion] Remove all uses of panic! from expression.rs
[ https://issues.apache.org/jira/browse/ARROW-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4899: -- Labels: pull-request-available (was: ) > [Rust] [DataFusion] Remove all uses of panic! from expression.rs > > > Key: ARROW-4899 > URL: https://issues.apache.org/jira/browse/ARROW-4899 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust, Rust - DataFusion >Affects Versions: 0.12.0 >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4894) [Rust] [DataFusion] Remove all uses of panic! from aggregate.rs
[ https://issues.apache.org/jira/browse/ARROW-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-4894: -- Labels: pull-request-available (was: ) > [Rust] [DataFusion] Remove all uses of panic! from aggregate.rs > --- > > Key: ARROW-4894 > URL: https://issues.apache.org/jira/browse/ARROW-4894 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust, Rust - DataFusion >Affects Versions: 0.12.0 >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > > Code cleanup -- This message was sent by Atlassian JIRA (v7.6.3#76005)
