[ https://issues.apache.org/jira/browse/CALCITE-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16697810#comment-16697810 ]
Enrico Olivelli commented on CALCITE-2662: ------------------------------------------ Sorry for late reply. I have updated the patch with a miminal "positive" test and change InputStream+Charset to java.io.Reader. The ideas around this change are the following: * I have a Netty Direct ByteBuf which is holding the query (which is coming from the network) * I know that the parser will split the query and create the AST, I can't prevent it from during so, we should stop using java.lang.String at all, and I think this change won't be acceptable in the short/mid term * If I have to create a java.lang.String (like not with Calcite 1.17) by sure I will duplicate the memory used for the query one more time (the AST will create many copied of part of the original String, and using a StringReader, which is also bad), and this will be on the heap. * Overall this change will save only an allocation of the byte[] in the heap (+ UTF8 encoder stuff), if you have "long" queries it will be a non-transcurable saving of resources. *I will keep on hold this change* until I finish the full story on my project and demonstrate that this change is usefull. I have also to create the test about error reporting, which in my case is not so important, but I understand that we have to be clear to what an user will gain and what it will lose. > Planner: allow parsing directly a stream instead of a java.lang.String > ---------------------------------------------------------------------- > > Key: CALCITE-2662 > URL: https://issues.apache.org/jira/browse/CALCITE-2662 > Project: Calcite > Issue Type: Improvement > Components: core > Affects Versions: 1.17.0 > Reporter: Enrico Olivelli > Assignee: Julian Hyde > Priority: Major > > In 1.17.0 the org.apache.calcite.tools.Planner interface only accept a > java.lang.String as input. > In order to reduce memory allocations and copies it will be useful that the > planner could accept the query in a more 'raw' format. > Creating a java.lang.String is very expensive, and if your "query" is coming > from the network (think about a Netty Direct memory ByteBuf) you are forced > to create a copy of the text of the query. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)