Re: [DISCUSS] Changing C++ build system default options to produce more barebones builds

2019-09-18 Thread Micah Kornfield
> > * Should optional components be "opt in", "out out", or a mix? > Currently it's a mix, and that's confusing for people. I think we > should make them all "opt in". Agreed they should all be opt in by default. I think active developer are quite adept at flipping the appropriate CMake flags.

Re: Timeline for 0.15.0 release

2019-09-18 Thread Micah Kornfield
> > The process should be well documented at this point but there are a > number of steps. Is [1] the up-to-date documentation for the release? Are there instructions for the adding the code signing Key to SVN? I will make a go of it. i will try to mitigate any internet issues by doing the

Draft blog post for 0.15 release

2019-09-18 Thread Neal Richardson
Hi all, In preparation for next week, I've started a release announcement blog post here: https://github.com/apache/arrow-site/pull/27 Please fill in the parts you know best. Committers can just push edits to my branch; also feel free to reply to this thread with content, or email me directly,

[jira] [Created] (ARROW-6616) [Website] Release annoucement blog post for 0.15

2019-09-18 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-6616: -- Summary: [Website] Release annoucement blog post for 0.15 Key: ARROW-6616 URL: https://issues.apache.org/jira/browse/ARROW-6616 Project: Apache Arrow

[jira] [Created] (ARROW-6615) [C++] Add filtering option to fs::Selector

2019-09-18 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6615: - Summary: [C++] Add filtering option to fs::Selector Key: ARROW-6615 URL: https://issues.apache.org/jira/browse/ARROW-6615 Project: Apache Arrow

[jira] [Created] (ARROW-6614) [C++][Dataset] Implement FileSystemDataSourceDiscovery

2019-09-18 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6614: - Summary: [C++][Dataset] Implement FileSystemDataSourceDiscovery Key: ARROW-6614 URL: https://issues.apache.org/jira/browse/ARROW-6614 Project:

[jira] [Created] (ARROW-6613) [C++] Remove dependency on boost::filesystem

2019-09-18 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-6613: - Summary: [C++] Remove dependency on boost::filesystem Key: ARROW-6613 URL: https://issues.apache.org/jira/browse/ARROW-6613 Project: Apache Arrow Issue

[jira] [Created] (ARROW-6612) [C++] Add ARROW_CSV CMake build flag

2019-09-18 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-6612: --- Summary: [C++] Add ARROW_CSV CMake build flag Key: ARROW-6612 URL: https://issues.apache.org/jira/browse/ARROW-6612 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-6611) [C++] Make ARROW_JSON=OFF the default

2019-09-18 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-6611: --- Summary: [C++] Make ARROW_JSON=OFF the default Key: ARROW-6611 URL: https://issues.apache.org/jira/browse/ARROW-6611 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-6610) [C++] Add ARROW_FILESYSTEM=ON/OFF CMake configuration flag

2019-09-18 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-6610: --- Summary: [C++] Add ARROW_FILESYSTEM=ON/OFF CMake configuration flag Key: ARROW-6610 URL: https://issues.apache.org/jira/browse/ARROW-6610 Project: Apache Arrow

[jira] [Created] (ARROW-6609) [C++] Add minimal build Dockerfile example

2019-09-18 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-6609: --- Summary: [C++] Add minimal build Dockerfile example Key: ARROW-6609 URL: https://issues.apache.org/jira/browse/ARROW-6609 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-6608) [C++] Make default for ARROW_HDFS to be OFF

2019-09-18 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-6608: --- Summary: [C++] Make default for ARROW_HDFS to be OFF Key: ARROW-6608 URL: https://issues.apache.org/jira/browse/ARROW-6608 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-6607) Support for set/list columns in python

2019-09-18 Thread Giora Simchoni (Jira)
Giora Simchoni created ARROW-6607: - Summary: Support for set/list columns in python Key: ARROW-6607 URL: https://issues.apache.org/jira/browse/ARROW-6607 Project: Apache Arrow Issue Type:

[DISCUSS] C-level in-process array protocol

2019-09-18 Thread Antoine Pitrou
Hello, One thing that was discussed in the sync call is the ability to easily pass arrays at runtime between Arrow implementations or Arrow-supporting libraries in the same process, without bearing the cost of linking to e.g. the C++ Arrow library. (for example: "Duckdb wants to provide an

[jira] [Created] (ARROW-6606) [C++] Construct tree structure from std::vector

2019-09-18 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6606: - Summary: [C++] Construct tree structure from std::vector Key: ARROW-6606 URL: https://issues.apache.org/jira/browse/ARROW-6606 Project: Apache

[jira] [Created] (ARROW-6605) [C++] Add recursion depth control to fs::Selector

2019-09-18 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6605: - Summary: [C++] Add recursion depth control to fs::Selector Key: ARROW-6605 URL: https://issues.apache.org/jira/browse/ARROW-6605 Project: Apache

[jira] [Created] (ARROW-6604) [C++] Add support for nested types to MakeArrayFromScalar

2019-09-18 Thread Benjamin Kietzman (Jira)
Benjamin Kietzman created ARROW-6604: Summary: [C++] Add support for nested types to MakeArrayFromScalar Key: ARROW-6604 URL: https://issues.apache.org/jira/browse/ARROW-6604 Project: Apache Arrow

[jira] [Created] (ARROW-6603) [C#] ArrayBuilder API to support writing nulls

2019-09-18 Thread Eric Erhardt (Jira)
Eric Erhardt created ARROW-6603: --- Summary: [C#] ArrayBuilder API to support writing nulls Key: ARROW-6603 URL: https://issues.apache.org/jira/browse/ARROW-6603 Project: Apache Arrow Issue

Re: Timeline for 0.15.0 release

2019-09-18 Thread Wes McKinney
The process should be well documented at this point but there are a number of steps. Note that you need to add your code signing key to the KEYS file in SVN (that's not very hard to do). I think it's fine to hand off the process to others after the VOTE but it would be tricky to have multiple RMs

[jira] [Created] (ARROW-6602) [Doc] Add feature / implementation matrix

2019-09-18 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-6602: - Summary: [Doc] Add feature / implementation matrix Key: ARROW-6602 URL: https://issues.apache.org/jira/browse/ARROW-6602 Project: Apache Arrow Issue Type:

Re: Arrow sync call September 19 at 12:00 US/Eastern, 16:00 UTC

2019-09-18 Thread Wes McKinney
I'm unable to join today but hope that participants can review the active DISCUSS threads On Tue, Sep 17, 2019 at 11:28 PM Neal Richardson wrote: > > Hi all, > Belated reminder that the biweekly Arrow call is coming up in less than 12 > hours at https://meet.google.com/vtm-teks-phx. All are

Re: [DISCUSS] Changing C++ build system default options to produce more barebones builds

2019-09-18 Thread Wes McKinney
To be clear I think we should make these changes right after 0.15.0 is released so we aren't playing whackamole with our packaging scripts. I'm happy to take the lead on the work... On Wed, Sep 18, 2019 at 9:54 AM Antoine Pitrou wrote: > > On Wed, 18 Sep 2019 09:46:54 -0500 > Wes McKinney

[jira] [Created] (ARROW-6600) [Java] Implement dictionary-encoded subfields for Union type

2019-09-18 Thread Ji Liu (Jira)
Ji Liu created ARROW-6600: - Summary: [Java] Implement dictionary-encoded subfields for Union type Key: ARROW-6600 URL: https://issues.apache.org/jira/browse/ARROW-6600 Project: Apache Arrow Issue

Re: [DISCUSS] Changing C++ build system default options to produce more barebones builds

2019-09-18 Thread Wes McKinney
I think these are both interesting areas to explore further. I'd like to focus on the couple of immediate items I think we should address * Should optional components be "opt in", "out out", or a mix? Currently it's a mix, and that's confusing for people. I think we should make them all "opt in".

[jira] [Created] (ARROW-6598) [Java] Sort the code for ApproxEqualsVisitor

2019-09-18 Thread Liya Fan (Jira)
Liya Fan created ARROW-6598: --- Summary: [Java] Sort the code for ApproxEqualsVisitor Key: ARROW-6598 URL: https://issues.apache.org/jira/browse/ARROW-6598 Project: Apache Arrow Issue Type:

[NIGHTLY] Arrow Build Report for Job nightly-2019-09-18-0

2019-09-18 Thread Crossbow
Arrow Build Report for Job nightly-2019-09-18-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-18-0 Failed Tasks: - docker-cpp-fuzzit: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-09-18-0-circle-docker-cpp-fuzzit -

[jira] [Created] (ARROW-6597) [Python] Segfault in test_pandas with Python 2.7

2019-09-18 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-6597: - Summary: [Python] Segfault in test_pandas with Python 2.7 Key: ARROW-6597 URL: https://issues.apache.org/jira/browse/ARROW-6597 Project: Apache Arrow

Re: How can I help?

2019-09-18 Thread SemanticBeeng .
Hi Weston, Documenting your use cases would be a great help, imo. If open then am interested to help with that. I am looking to build some advanced POC. Please advise Thanks Nick https://twitter.com/semanticbeeng On Tue, Sep 17, 2019 at 6:10 PM Weston Platter wrote: > Hey there, > > I’ve

Re: [DISCUSS] Changing C++ build system default options to produce more barebones builds

2019-09-18 Thread Uwe L. Korn
Hello Micah, I don't think we have explored using bazel yet. I would see it as a possible modular alternative but as you mention it will be a lot of work and we would probably need a mentor who is familiar with bazel, otherwise we probably end up spending too much time on this and get a

Re: [DISCUSS] Changing C++ build system default options to produce more barebones builds

2019-09-18 Thread Micah Kornfield
It has come up in the past, but I wonder if exploring Bazel as a build system with its a very explicit dependency graph might help (I'm not sure if something similar is available in CMake). This is also a lot of work, but could also potentially benefit the developer experience because we can make

[jira] [Created] (ARROW-6596) Getting "Cannot call io___MemoryMappedFile__Open()" error while reading a parquet file

2019-09-18 Thread Addhyan (Jira)
Addhyan created ARROW-6596: -- Summary: Getting "Cannot call io___MemoryMappedFile__Open()" error while reading a parquet file Key: ARROW-6596 URL: https://issues.apache.org/jira/browse/ARROW-6596 Project:

Re: [DISCUSS] Changing C++ build system default options to produce more barebones builds

2019-09-18 Thread Uwe L. Korn
Hello, I can think of two other alternatives that make it more visible what Arrow core is and what are the optional components: * Error out when no component is selected instead of building just the core Arrow. Here we could add an explanative message that list all components and for each