[
https://issues.apache.org/jira/browse/CALCITE-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16697810#comment-16697810
]
Enrico Olivelli commented on CALCITE-2662:
------------------------------------------
Sorry for late reply.
I have updated the patch with a miminal "positive" test and change
InputStream+Charset to java.io.Reader.
The ideas around this change are the following:
* I have a Netty Direct ByteBuf which is holding the query (which is coming
from the network)
* I know that the parser will split the query and create the AST, I can't
prevent it from during so, we should stop using java.lang.String at all, and I
think this change won't be acceptable in the short/mid term
* If I have to create a java.lang.String (like not with Calcite 1.17) by sure
I will duplicate the memory used for the query one more time (the AST will
create many copied of part of the original String, and using a StringReader,
which is also bad), and this will be on the heap.
* Overall this change will save only an allocation of the byte[] in the heap
(+ UTF8 encoder stuff), if you have "long" queries it will be a
non-transcurable saving of resources.
*I will keep on hold this change* until I finish the full story on my project
and demonstrate that this change is usefull.
I have also to create the test about error reporting, which in my case is not
so important, but I understand that we have to be clear to what an user will
gain and what it will lose.
> Planner: allow parsing directly a stream instead of a java.lang.String
> ----------------------------------------------------------------------
>
> Key: CALCITE-2662
> URL: https://issues.apache.org/jira/browse/CALCITE-2662
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.17.0
> Reporter: Enrico Olivelli
> Assignee: Julian Hyde
> Priority: Major
>
> In 1.17.0 the org.apache.calcite.tools.Planner interface only accept a
> java.lang.String as input.
> In order to reduce memory allocations and copies it will be useful that the
> planner could accept the query in a more 'raw' format.
> Creating a java.lang.String is very expensive, and if your "query" is coming
> from the network (think about a Netty Direct memory ByteBuf) you are forced
> to create a copy of the text of the query.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
