[
https://issues.apache.org/jira/browse/ARROW-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16766766#comment-16766766
]
Micah Kornfield commented on ARROW-4333:
----------------------------------------
[http://www.vldb.org/pvldb/vol4/p539-neumann.pdf] might be of interest. It
seems like this reverses the model to some extent (and then goes ahead and
disposes of it entirely to compile code). Caveat (I'm still ramping up on DB
systems in general).
> [C++] Sketch out design for kernels and "query" execution in compute layer
> --------------------------------------------------------------------------
>
> Key: ARROW-4333
> URL: https://issues.apache.org/jira/browse/ARROW-4333
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
> Reporter: Micah Kornfield
> Priority: Major
> Labels: analytics
> Fix For: 0.13.0
>
>
> It would be good to formalize the design of kernels and the controlling query
> execution layer (e.g. volcano batch model?) to understand the following:
> Contracts for kernels:
> * Thread safety of kernels?
> * When Kernels should allocate memory vs expect preallocated memory? How to
> communicate requirements for a kernels memory allocaiton?
> * How to communicate the whether a kernels execution is parallelizable
> across a ChunkedArray? How to determine if the order to execution across a
> ChunkedArray is important?
> * How to communicate when it is safe to re-use the same buffers and input
> and output to the same kernel?
> What does the threading model look like for the higher level of control?
> Where should synchronization happen?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
