Right, the situations where array and scalar inputs are mixed (e.g. Array + Scalar -> Array) are unaffected. The change will make more sense when you see in the PR how much code I've been able to delete.
On Wed, Jun 29, 2022 at 1:03 PM Weston Pace <weston.p...@gmail.com> wrote: > > This is only for the situation where ALL inputs and outputs are > scalar. Scalars, at the kernel level, do not have length. So in this > case there is nothing to repeat. It does build a buffer, but just > with a single value, so it is all O(1). > > On Wed, Jun 29, 2022 at 9:49 AM Antoine Pitrou <anto...@python.org> wrote: > > > > > > Does boxing a scalar into an array actually build a buffer with the > > repeated value, or is it more efficient than that? > > > > > > Le 29/06/2022 à 17:57, Wes McKinney a écrit : > > > I'm working on my next PR which addresses the "scalar output modality" > > > and removes usages of ValueDescr and output shapes in general from the > > > kernel execution machinery. The ability to provide all scalar input > > > and scalar output of course is preserved — it's just that the "boxing" > > > and "unboxing" of the inputs and outputs is happening at a higher > > > level (i.e. there is only one place that boxes scalars into arrays and > > > unboxes the output into a scalar) rather than being implemented on a > > > piecemeal basis in the kernel implementations. > > > > > > Please keep an eye out for any non-trivial PRs that affect > > > arrow/compute/kernels since it could cause some rebasing headache. I > > > am working as fast as I can to complete this next PR. > > > > > > After this, the next thing is porting the aggregate kernels to use > > > ExecSpan. Then I think the mass refactoring will cool off and we can > > > address follow-on improvements like rewriting expression evaluation to > > > utilize the span data structures to yield performance gains. > > > > > > On Mon, Jun 13, 2022 at 12:37 PM Wes McKinney <wesmck...@gmail.com> wrote: > > >> > > >> I merged the PR a little while ago — thanks for David, Sasha for > > >> helping review. If you have more comments or things you would like to > > >> fix from that PR, please comment there and I can help address them in > > >> follow up PRs. > > >> > > >> I'm going to work next on migrating the rest of the kernels to use > > >> ExecSpan (and thus ExecBatchIterator can be removed in favor of > > >> ExecSpanIterator — I'll add benchmarks so we can see the performance > > >> difference), and then my next priority will be removing the ValueDescr > > >> concept and the ValueDescr::SCALAR output path for the > > >> scalar/elementwise kernels which will help us delete a lot of code > > >> > > >> I'll attach related Jiras to this umbrella issue: > > >> > > >> https://issues.apache.org/jira/browse/ARROW-16755 > > >> > > >> On Fri, Jun 10, 2022 at 12:56 PM Wes McKinney <wesmck...@gmail.com> > > >> wrote: > > >>> > > >>> PR is up: https://github.com/apache/arrow/pull/13364 > > >>> > > >>> Look forward to getting this in since there's a bunch of follow on > > >>> work that I'd like to get started on ASAP! > > >>> > > >>> On Thu, Jun 9, 2022 at 7:34 AM Wes McKinney <wesmck...@gmail.com> wrote: > > >>>> > > >>>> I'm making good progress getting my branch PR-ready -- working through > > >>>> the compute-scalar-test suite and fixing the little things I broke. I > > >>>> hope I'll have it done by the end of the week. > > >>>> > > >>>> On Mon, Jun 6, 2022 at 3:21 PM Wes McKinney <wesmck...@gmail.com> > > >>>> wrote: > > >>>>> > > >>>>> I created https://issues.apache.org/jira/browse/ARROW-16755 as an > > >>>>> umbrella issue to track improvements to reduce overhead in the > > >>>>> expression and kernel execution machinery. Please help by attaching > > >>>>> related issues and creating new issues for specific individual efforts > > >>>>> here. I'll work as quickly as I can to have my initial patch > > >>>>> ARROW-16756 ready which will unblock the next few projects here > > >>>>> > > >>>>> On Mon, Jun 6, 2022 at 10:35 AM Wes McKinney <wesmck...@gmail.com> > > >>>>> wrote: > > >>>>>> > > >>>>>> This is definitely only the first stage of cleanup and > > >>>>>> streamlining — > > >>>>>> I anticipate multiple rounds of refactoring (maybe not as invasive > > >>>>>> and > > >>>>>> painful as this one), and this patch I'm not sure will do a lot to > > >>>>>> alleviate bottom line expression evaluation overhead but it creates > > >>>>>> the environment (i.e. where a whole chain of scalar functions that > > >>>>>> all > > >>>>>> write into preallocated memory can execute without having to touch > > >>>>>> shared_ptrs or deal with other objects with excess microperformance > > >>>>>> overhead) where such optimization can happen more easily. > > >>>>>> > > >>>>>> > > >>>>>> On Mon, Jun 6, 2022 at 4:08 AM Antoine Pitrou <anto...@python.org> > > >>>>>> wrote: > > >>>>>>> > > >>>>>>> > > >>>>>>> Le 06/06/2022 à 09:34, Sasha Krassovsky a écrit : > > >>>>>>>> Wow that's a lot of progress! > > >>>>>>>> Definitely agree on the scalar outputs point. > > >>>>>>>> > > >>>>>>>> One point about the ArraySpan - why does it need to know its data > > >>>>>>>> type? > > >>>>>>>> Once a kernel has been resolved by the registry, the kernel will > > >>>>>>>> only know > > >>>>>>>> how to execute on the specific type it was resolved for, right? > > >>>>>>> > > >>>>>>> Because of parametric types for example (e.g. timestamps with a > > >>>>>>> unit and > > >>>>>>> timezone, or decimals with a precision and scale). > > >>>>>>> > > >>>>>>> Regards > > >>>>>>> > > >>>>>>> Antoine.
