Zoram Thanga created IMPALA-6971:
------------------------------------
Summary: Log flooding with unhelpful parsing error message
Key: IMPALA-6971
URL: https://issues.apache.org/jira/browse/IMPALA-6971
Project: IMPALA
Issue Type: Improvement
Affects Versions: Impala 2.12.0, Impala 3.0
Reporter: Zoram Thanga
Once in a while I've seen Impala flood the log file with messages similar to
the following:
{noformat}
I0318 08:20:28.566603 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566643 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566676 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566709 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566742 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566787 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566820 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566856 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566890 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566923 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566959 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.566998 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567039 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567070 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567101 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567132 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567173 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567209 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567245 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567276 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567312 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567350 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567386 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567433 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567473 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567507 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567538 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567574 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567606 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567638 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567674 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567710 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567749 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567785 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567821 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567857 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567893 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567924 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567961 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.567993 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568038 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568078 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568117 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568150 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568186 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568222 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568255 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568291 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568327 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568367 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568399 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568430 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568470 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568506 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568542 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568578 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568619 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568662 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568699 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568738 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568771 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568807 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568840 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568883 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
I0318 08:20:28.568919 984564 runtime-state.cc:168] Error from query
cf41e4ee4f7e1273:bcc4103800000000: Error parsing row: file:
hdfs://nameservice/user/hive/warehouse/mydb/mytable/part-m-00000, before
offset: 2122317824
{noformat}
Two main issues with this log entry are:
# The line number printed is wrong. This line has nothing to do with processing
rows. We should try harder to show the real location of the problem.
# It is unnecessarily repeated, filling log files.
In this specific instance, the same entry is repeated more than 1,740,663 times!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
