[
https://issues.apache.org/jira/browse/IMPALA-10611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325987#comment-17325987
]
Riza Suminto commented on IMPALA-10611:
---------------------------------------
Gerrit review submitted here:
https://gerrit.cloudera.org/c/17324/
> test_wide_row fails with 'Failed to allocate row batch'
> -------------------------------------------------------
>
> Key: IMPALA-10611
> URL: https://issues.apache.org/jira/browse/IMPALA-10611
> Project: IMPALA
> Issue Type: Bug
> Reporter: Abhishek Rawat
> Assignee: Riza Suminto
> Priority: Major
>
> It appears that we may not be accounting for memory properly somewhere and as
> a result we end up with allocations much more than reservations resulting in
> OOM error. It's also possible we may not be freeing memory properly (could be
> a leak or memory accumulation due to smart pointers etc).
> It is also possible that something (table data?) changed in the test case and
> as a result the memory limit is not sufficient.
> *Error:*
> {code:java}
> Messagequery_test/test_scanners.py:276: in test_wide_row
> self.run_test_case('QueryTest/wide-row', new_vector)
> common/impala_test_suite.py:665: in run_test_case result = exec_fn(query,
> user=test_section.get('USER', '').strip() or None)
> common/impala_test_suite.py:603: in __exec_in_impala result =
> self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:923: in __execute_query return
> impalad_client.execute(query, user=user) common/impala_connection.py:205: in
> execute return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:187: in execute handle =
> self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:365: in __execute_query
> self.wait_for_finished(handle) beeswax/impala_beeswax.py:386: in
> wait_for_finished raise ImpalaBeeswaxException("Query aborted:" +
> error_log, None) E ImpalaBeeswaxException: ImpalaBeeswaxException: E
> Query aborted:Memory limit exceeded: Failed to allocate row batch E
> EXCHANGE_NODE (id=1) could not allocate 16.00 MB without exceeding limit. E
> Error occurred on backend ip-172-31-14-211:27000 E Memory left in process
> limit: 9.09 GB E Memory left in query limit: 9.95 MB E
> Query(7647dab034192a59:e4f2972900000000): Limit=100.00 MB Reservation=40.00
> MB ReservationLimit=68.00 MB OtherMemory=50.05 MB Total=90.05 MB Peak=90.05
> MB E Unclaimed reservations: Reservation=32.00 MB OtherMemory=0
> Total=32.00 MB Peak=40.00 MB E Fragment
> 7647dab034192a59:e4f2972900000001: Reservation=8.00 MB OtherMemory=50.04 MB
> Total=58.04 MB Peak=58.05 MB E HDFS_SCAN_NODE (id=0): Reservation=8.00
> MB OtherMemory=30.01 MB Total=38.01 MB Peak=58.03 MB E
> KrpcDataStreamSender (dst_id=1): Total=9.84 KB Peak=9.84 KB E CodeGen:
> Total=1004.00 B Peak=118.00 KB E Fragment
> 7647dab034192a59:e4f2972900000000: Reservation=0 OtherMemory=16.00 KB
> Total=16.00 KB Peak=16.00 KB E EXCHANGE_NODE (id=1): Reservation=8.00
> KB OtherMemory=0 Total=8.00 KB Peak=8.00 KB E KrpcDeferredRpcs:
> Total=0 Peak=0 E PLAN_ROOT_SINK: Total=0 Peak=0 E CodeGen:
> Total=90.00 B Peak=65.00 KB{code}
> *StackTrace:*
> {code:java}
> query_test/test_scanners.py:276: in test_wide_row
> self.run_test_case('QueryTest/wide-row', new_vector)
> common/impala_test_suite.py:665: in run_test_case
> result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
> common/impala_test_suite.py:603: in __exec_in_impala
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:923: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:205: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:187: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:365: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:386: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:Memory limit exceeded: Failed to allocate row batch
> E EXCHANGE_NODE (id=1) could not allocate 16.00 MB without exceeding limit.
> E Error occurred on backend ip-172-31-14-211:27000
> E Memory left in process limit: 9.09 GB
> E Memory left in query limit: 9.95 MB
> E Query(7647dab034192a59:e4f2972900000000): Limit=100.00 MB
> Reservation=40.00 MB ReservationLimit=68.00 MB OtherMemory=50.05 MB
> Total=90.05 MB Peak=90.05 MB
> E Unclaimed reservations: Reservation=32.00 MB OtherMemory=0 Total=32.00
> MB Peak=40.00 MB
> E Fragment 7647dab034192a59:e4f2972900000001: Reservation=8.00 MB
> OtherMemory=50.04 MB Total=58.04 MB Peak=58.05 MB
> E HDFS_SCAN_NODE (id=0): Reservation=8.00 MB OtherMemory=30.01 MB
> Total=38.01 MB Peak=58.03 MB
> E KrpcDataStreamSender (dst_id=1): Total=9.84 KB Peak=9.84 KB
> E CodeGen: Total=1004.00 B Peak=118.00 KB
> E Fragment 7647dab034192a59:e4f2972900000000: Reservation=0
> OtherMemory=16.00 KB Total=16.00 KB Peak=16.00 KB
> E EXCHANGE_NODE (id=1): Reservation=8.00 KB OtherMemory=0 Total=8.00 KB
> Peak=8.00 KB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E PLAN_ROOT_SINK: Total=0 Peak=0
> E CodeGen: Total=90.00 B Peak=65.00 KB
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
