Paul Rogers created DRILL-5279: ---------------------------------- Summary: JsonRecordReader inefficient: allocates temp direct memory buffers Key: DRILL-5279 URL: https://issues.apache.org/jira/browse/DRILL-5279 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.10.0 Reporter: Paul Rogers Priority: Minor
The {{JsonRecordReader}} class uses the {{VarCharWriter}} class to add a string value to a VarChar vector. The {{VarCharWriter}} method to add a string has this signature: {code} void writeVarChar(int start, int end, DrillBuf buffer); {code} This means that the {{JsonRecordReader}} has to convert the JSON string value to a direct memory buffer, then write it into the buffer. Direct memory buffers are expensive to create and should be done only to hold large amounts of semi-persistent data (such as value vectors.) Extend the {{VarCharWriter}} to accept a Java String and/or heap buffer to avoid having to incur the overhead of the direct memory temporary buffer. -- This message was sent by Atlassian JIRA (v6.3.15#6346)