[
https://issues.apache.org/jira/browse/TRAFODION-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16502036#comment-16502036
]
Sandhya Sundaresan commented on TRAFODION-2949:
-----------------------------------------------
Additional issue uncovered on driver side :
While running batch tests from JDBC , this error occured with 16MB length :
{color:#000000}The length of lob is defined as 16777216, which will be prompted
when the batch insert:{color}
{color:#000000}ERROR[30050] Memory required by input rowset buffer is 1600 MB,
setup specified limit of 1024. Row length is 16777226 bytes and rowset has a
maximum of 100{color}
{color:#000000}It is {color}{color:#000000}saying that the limit we have set is
1024 MB . Since the rowsize is 16MB, the rowset size must be limited to 64 or
less. If you kept as 100, then 1600MB exceeds 1024 MB. So lets make it 50 to
be safe.{color}
{color:#000000}I am not sure if the rowset size is configurable in the JDBC
driver ? William , do you know ? I thought Rao told me
it{color}{color:#000000}’{color}{color:#000000}s configurable.{color}
{color:#000000}So in general whenever we want to try larger rowsizes, we must
limit the number of rows in the rowset. {color}
{color:#000000}Change needs to happen in driver.{color}
{color:#000000}-----{color}
> LOB: Address performance issues related to inserts/extracts in tables
> containing LOB columns
> --------------------------------------------------------------------------------------------
>
> Key: TRAFODION-2949
> URL: https://issues.apache.org/jira/browse/TRAFODION-2949
> Project: Apache Trafodion
> Issue Type: Improvement
> Reporter: Sandhya Sundaresan
> Assignee: Sandhya Sundaresan
> Priority: Major
>
> # Investigate bottlenecks in the insert and update code paths.
> A couple of bottlenecks identified initially were hdfsWrite and the LOBCLi
> call.
> # Investigate if lib hdfs write operation is signifincantly slower than a
> direct JNI call. If so consider refactoring code being done for hive inserts
> and reuse for LOB writes as well. Also investigate use of libhdfs3 , a
> lightweight interface by Pivotal for Hawk ,that doesn’t use JNI at all.
> # Investigate if LOB Cli calls that do prepare and execute of statements to
> insert/update the LOB descriptor tables are being cached and reused when
> multiple LOB inserts are being done in the same session.
> # Investigate external LOBs as a solution for faster inserts/extracts for
> usage compared to internal LOBs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
