Re: post-release tasks (4.0.1)

2021-06-18 Thread Jorge Cardoso Leitão
Sutou was able to push them (we needed to login via yarn, not npm). I think that we are all good; the last item was done. I have created a short post [1] for it. Do we usually announce it anywhere else? Best, Jorge [1] https://github.com/apache/arrow-site/pull/122 On Sat, Jun 12, 2021 at

Re: [C++] Apache Arrow C++ Variadic Kernels Design

2021-06-18 Thread Wes McKinney
COUNT(DISTINCT varargs...) can be used either as a scalar aggregate function or a group aggregate function. For example SELECT COUNT(DISTINCT expr1, expr2, ...) FROM TABLE; returns a single value. It can be used with GROUP BY to produce a distinct count per group. I think it would be useful to

Re: [C++] Apache Arrow C++ Variadic Kernels Design

2021-06-18 Thread Ian Cook
> Aren't SELECT DISTINCT and COUNT DISTINCT just condensed variants of a GROUP > BY query? Do they need to be exposed as standalone kernels? I listed SELECT DISTINCT and COUNT DISTINCT in the document only as examples of SQL statements that take a variable number of arguments, not to imply that

Re: [INFO_REQUEST][FLIGHT] - Dynamic schema changes in ArrowFlight streams

2021-06-18 Thread Gosh Arzumanyan
Hi David, Thanks for poking me on this. I have been thinking it out but have not got to crafting a doc. Let me put together a rough proposal this weekend. Afterwards I'll do need your help for bringing it to a reviewable state. Cheers, Gosh On Fri., 18 Jun. 2021, 18:11 David Li, wrote: >

Re: [C++] Apache Arrow C++ Variadic Kernels Design

2021-06-18 Thread Antoine Pitrou
Aren't SELECT DISTINCT and COUNT DISTINCT just condensed variants of a GROUP BY query? Do they need to be exposed as standalone kernels? Le 18/06/2021 à 00:58, Ian Cook a écrit : Arrow developers, A couple of recent PRs have added new variadic scalar kernels to the Arrow C++ library

Re: [INFO_REQUEST][FLIGHT] - Dynamic schema changes in ArrowFlight streams

2021-06-18 Thread David Li
Following up here - Gosh, did you get a chance to put something together? Do you need/want help on this? This would also potentially be useful for FlightSQL. (See the discussion on GitHub: https://github.com/apache/arrow/pull/9368#discussion_r572941765) Best, David On Fri, Apr 16, 2021, at

Re: Future of Rust sync call

2021-06-18 Thread Wes McKinney
Sounds great! Thanks On Fri, Jun 18, 2021 at 9:29 AM Jorge Cardoso Leitão wrote: > > Hi Wes, > > Yes, on ASF Slack, #arrow-rust. Andy advertised it here some time ago. > > Most relevant topics there end up either as a github issue or over > this mailing list. On this note, hat tip to Andrew, who

Re: [C++] Apache Arrow C++ Variadic Kernels Design

2021-06-18 Thread Wes McKinney
hi Ian — I agree with implementing these functions with varargs/variadic inputs (this was my original intent when drafting compute/kernel.h and related machinery last year). As one nuance with the way that things work right now, the type matching infrastructure isn't necessarily able to determine

Re: Future of Rust sync call

2021-06-18 Thread Jorge Cardoso Leitão
Hi Wes, Yes, on ASF Slack, #arrow-rust. Andy advertised it here some time ago. Most relevant topics there end up either as a github issue or over this mailing list. On this note, hat tip to Andrew, who has been doing a lot of the curation. There are other informal discussions, more about Rust

Re: [Question] Rational for offsets instead of deltas

2021-06-18 Thread Wes McKinney
On Fri, Jun 18, 2021 at 1:12 AM Micah Kornfield wrote: > > > > > Is it to ensure O(1) random access (instead of having to sum all > > deltas up to the index)? > > > This is my understanding of why it was chosen. Yes, that's the reason. For example, certain columnar query processing patterns

Re: Future of Rust sync call

2021-06-18 Thread Wes McKinney
hi Jorge — there is a Rust Slack channel? On that, I would just say to be vigilant about what communication takes place there (since Slack is semi-private) versus on channels that are being archived / mirrored to mailing lists. It's useful for coordination and quick questions but not a place to

Re: [Question] Rational for offsets instead of deltas

2021-06-18 Thread Micah Kornfield
> > Is it to ensure O(1) random access (instead of having to sum all > deltas up to the index)? This is my understanding of why it was chosen. On Thu, Jun 17, 2021 at 10:32 PM Jorge Cardoso Leitão < jorgecarlei...@gmail.com> wrote: > Hi, > > (this has no direction; I am just genuinely curious)