xubo245 commented on a change in pull request #3193: [CARBONDATA-3365]
Integrate apache arrow vector filling to carbon SDK
URL: https://github.com/apache/carbondata/pull/3193#discussion_r281077111
##########
File path:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReader.java
##########
@@ -94,6 +96,43 @@ public T readNextRow() throws IOException,
InterruptedException {
return currentReader.getCurrentValue();
}
+ /**
+ * Carbon reader will fill the arrow vector after reading the carbondata
files.
+ * This arrow byte[] can be used to create arrow table and used for in
memory analytics
+ *
+ * Note: create a reader at blocklet level, so that arrow byte[] will not
exceed INT_MAX
+ *
+ * @param carbonSchema
+ * @return
+ * @throws Exception
+ */
+ public byte[] readArrowBatch(Schema carbonSchema) throws Exception {
Review comment:
You already remove this method in python code, no other place use this,
right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
