[jira] [Created] (ARROW-7683) [Packaging] Set 0.16.0 as the next version

2020-01-26 Thread Kouhei Sutou (Jira)
Kouhei Sutou created ARROW-7683: --- Summary: [Packaging] Set 0.16.0 as the next version Key: ARROW-7683 URL: https://issues.apache.org/jira/browse/ARROW-7683 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-7685) [Developer] Add support for GitHub Actions to Crossbow

2020-01-26 Thread Kouhei Sutou (Jira)
Kouhei Sutou created ARROW-7685: --- Summary: [Developer] Add support for GitHub Actions to Crossbow Key: ARROW-7685 URL: https://issues.apache.org/jira/browse/ARROW-7685 Project: Apache Arrow

[jira] [Created] (ARROW-7686) [Packaging][deb][RPM] Include more arrow-*.pc

2020-01-26 Thread Kouhei Sutou (Jira)
Kouhei Sutou created ARROW-7686: --- Summary: [Packaging][deb][RPM] Include more arrow-*.pc Key: ARROW-7686 URL: https://issues.apache.org/jira/browse/ARROW-7686 Project: Apache Arrow Issue Type:

ARM CI by ursa-labs/crossbow

2020-01-26 Thread Sutou Kouhei
Hi, We're building .deb/.rpm packages for ARM by using QEMU on x86-64 machine: https://github.com/apache/arrow/blob/master/dev/tasks/linux-packages/azure.linux.arm64.yml#L36-L50 It's slow. Some jobs can't finish in 6h, Azure Pipelines build time limit. You can see some *-aarch64 and *-arm64

Re: [Rust] Possible blocking issue for 0.16 release

2020-01-26 Thread Neville Dipale
Hi Andy, I think `cargo update` is the correct approach to resolve this issue. Yes, we've got quite a bit of backlog in Rust, but we should initiate the process to adopt parquet-format with Chao and Ivan's approval post 0.16. In the interim we could fix the thrift dependency issue upstream (as

Re: [Format] Array/RowBatch filters

2020-01-26 Thread Jacques Nadeau
At Dremio, we use four main types of selection vector/bitmaps: Dense Format (record valid or not, no ordering) - single bit (bitmap) Sparse formats (identifies valid records as well as their order) - 2 byte (for record batches up to 2^16 records). - 4 byte (for 2^16 batches of 2^16 records); - 6

[jira] [Created] (ARROW-7684) [Rust] Provide example of Flight server

2020-01-26 Thread Andy Grove (Jira)
Andy Grove created ARROW-7684: - Summary: [Rust] Provide example of Flight server Key: ARROW-7684 URL: https://issues.apache.org/jira/browse/ARROW-7684 Project: Apache Arrow Issue Type:

Re: 0.16 release preparation

2020-01-26 Thread Sutou Kouhei
Hi, https://github.com/apache/arrow/pull/6282 isn't a blocker. It just a nice-to-have feature. So we can cut 0.16.0-rc0 right now without the PR. If there are any problems in the PR, I think that we should not wait for it. If we can merge it as-is, I want us to merge the PR and cut 0.16.0-rc0.

Re: [Rust] Possible blocking issue for 0.16 release

2020-01-26 Thread Andy Grove
Running "cargo update" locally resolved the issue for me, so I guess this isn't a blocker after all. Cargo update actually downgraded the Thrift dependency back to 0.12 and I see that in master we are using 0.12 so I probably had a lock file that was generated due to using Thrift 0.13 on a branch

Re: [Rust] Possible blocking issue for 0.16 release

2020-01-26 Thread Krisztián Szűcs
Thanks for the update Andy! On Sun, Jan 26, 2020 at 5:21 PM Andy Grove wrote: > > Running "cargo update" locally resolved the issue for me, so I guess this > isn't a blocker after all. Cargo update actually downgraded the Thrift > dependency back to 0.12 and I see that in master we are using

Re: 0.16 release preparation

2020-01-26 Thread Krisztián Szűcs
Hi, I've just merged the last PR for 0.16 release. We have another packaging related improvement under [1], so theoretically we're good to go once that PR is merged. @Kou what do you think? [1]: https://github.com/apache/arrow/pull/6282 On Sat, Jan 18, 2020 at 3:00 AM Neal Richardson wrote:

[Rust] Possible blocking issue for 0.16 release

2020-01-26 Thread Andy Grove
Apologies for showing up at the last minute but I'm now re-engaged in the project after a bit of an absence and I noticed that we have some dependency conflicts due to the parquet-format crate (not controlled by Apache) using Thrift 0.12 whereas the parquet crate uses Thrift 0.13 and they require

Re: ARM CI by ursa-labs/crossbow

2020-01-26 Thread Wes McKinney
hi Kou, Sounds like a good idea. We have a couple ARM machines that we can try connecting to GHA * Jetson TX2 dev kit * Rockpro64 I don't know that I can personally spend much time on this this week but I'll make sure that the machines are up on the network and able to be SSH'd into by other