Paul Rogers created DRILL-5279:
----------------------------------
Summary: JsonRecordReader inefficient: allocates temp direct
memory buffers
Key: DRILL-5279
URL: https://issues.apache.org/jira/browse/DRILL-5279
Project: Apache Drill
Issue Type: Improvement
Affects Versions: 1.10.0
Reporter: Paul Rogers
Priority: Minor
The {{JsonRecordReader}} class uses the {{VarCharWriter}} class to add a string
value to a VarChar vector. The {{VarCharWriter}} method to add a string has
this signature:
{code}
void writeVarChar(int start, int end, DrillBuf buffer);
{code}
This means that the {{JsonRecordReader}} has to convert the JSON string value
to a direct memory buffer, then write it into the buffer.
Direct memory buffers are expensive to create and should be done only to hold
large amounts of semi-persistent data (such as value vectors.)
Extend the {{VarCharWriter}} to accept a Java String and/or heap buffer to
avoid having to incur the overhead of the direct memory temporary buffer.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
