[
https://issues.apache.org/jira/browse/TRAFODION-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168373#comment-16168373
]
ASF GitHub Bot commented on TRAFODION-2738:
-------------------------------------------
Github user sureshsubbiah commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/1232#discussion_r139228319
--- Diff: core/sql/sqlcomp/nadefaults.cpp ---
@@ -1336,6 +1336,8 @@ SDDkwd__(EXE_DIAGNOSTIC_EVENTS, "OFF"),
SDDui___(EXE_MEMORY_FOR_PROBE_CACHE_IN_MB, "100"),
+ SDDui___(EXE_MEMORY_FOR_UNPACK_ROWS_IN_MB, "100"),
--- End diff --
A default value of 100 MB maybe too small. I understand that this is good
for mixed workloads, but do consider than plans with Unpack, especially when
used for insert/upsert are simple. Unpack is always serial and part of master
exe. Often there is only one in a query. The cost of having a low value here
seems to be that queue length from unpack could be limited by this when we are
loading a table with wide rows (even if actual data in row is not wide). We
know from performance runs that queue length is critical to upsert load
performance. This is a hunch on my part and could be misguided. It will be good
to performance test for upsert of table with wide rows, if you think that is
worthwhile.
> Rowset buffer size during insert/upsert should be limited
> ---------------------------------------------------------
>
> Key: TRAFODION-2738
> URL: https://issues.apache.org/jira/browse/TRAFODION-2738
> Project: Apache Trafodion
> Issue Type: Bug
> Reporter: Selvaganesan Govindarajan
>
> Currently, the rowset size for upsert and insert can be given in a such way
> that it would be allocate unlimited buffer size to move the data from client
> to Trafodion SQL engine. This can cause memory corruption and unexpected
> behavior. Hence, we need a way to limit the number of rows in the rowset.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
