Ken created IMPALA-8282:
---------------------------
Summary: Impala Catalog 'Failed to load metadata for table' and
'GC overhead limit exceeded'
Key: IMPALA-8282
URL: https://issues.apache.org/jira/browse/IMPALA-8282
Project: IMPALA
Issue Type: Question
Components: Catalog
Affects Versions: Impala 2.5.0
Environment: Centos6.9
Reporter: Ken
Hi all
Our hive has inner table test1,test2 ,now *we cannot use test2 throw impala
anyway*.Can you help me to locate the real causes and do you have some ?
details as follow:
*Cannot execute 'desc test2', 'refresh test2' , 'invalidate metedata test2' ,
'select * from test2' commonds throw impala-shell or jdbc connection.*
*But*
*1.we can show table test2 in hive .*
*2.we can use other tables(such as test1) normally throw impala-shell or jdbc
connection or hive.*
*exception as follows:*
[DEVICE001:21000] > show create table test2;
Query: show create table test2
ERROR: AnalysisException: java.lang.OutOfMemoryError: GC overhead limit
exceeded
CAUSED BY: ExecutionException: java.lang.OutOfMemoryError: GC overhead limit
exceeded
CAUSED BY: OutOfMemoryError: GC overhead limit exceeded
CAUSED BY: TableLoadingException: java.lang.OutOfMemoryError: GC overhead
limit exceeded
CAUSED BY: ExecutionException: java.lang.OutOfMemoryError: GC overhead limit
exceeded
CAUSED BY: OutOfMemoryError: GC overhead limit exceeded
[DEVICE001:21000] > select * from test2 limit 1;
Query: select * from test2 limit 1
ERROR: AnalysisException: Failed to load metadata for table: 'test2'
CAUSED BY: TableLoadingException: java.lang.OutOfMemoryError: GC overhead
limit exceeded
CAUSED BY: ExecutionException: java.lang.OutOfMemoryError: GC overhead limit
exceeded
CAUSED BY: OutOfMemoryError: GC overhead limit exceeded
*top:*
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
9522 impala 20 0 34.1g 21g 29m S 8.0 8.6 1355:06 catalogd
9449 impala 20 0 1793m 499m 10m S 0.3 0.2 2:19.99 statestored
*free -g*
total used free shared buffers cached
Mem: 251 107 144 0 0 27
*Analyzer.java*
I find the exception was thrown in
*/Impala-cdh5-2.5.0_5.7.0/fe/src/main/java/com/cloudera/impala/analysis/Analyzer.java
*
public Table getTable(String dbName, String tableName)
throws AnalysisException, TableLoadingException {
Table table = null;
try {
table = getCatalog().getTable(dbName, tableName);
} catch (DatabaseNotFoundException e) {
throw new AnalysisException(DB_DOES_NOT_EXIST_ERROR_MSG + dbName);
} catch (CatalogException e) {
String errMsg = String.format("Failed to load metadata for table: %s",
tableName);
// We don't want to log all AnalysisExceptions as ERROR, only failures due to
// TableLoadingExceptions.
LOG.error(String.format("%s\n%s", errMsg, e.getMessage()));
if (e instanceof TableLoadingException) throw (TableLoadingException) e;
throw new TableLoadingException(errMsg, e);
}
if (table == null) {
throw new AnalysisException(
TBL_DOES_NOT_EXIST_ERROR_MSG + dbName + "." + tableName);
}
if (!table.isLoaded()) {
missingTbls_.add(new TableName(table.getDb().getName(), table.getName()));
throw new AnalysisException(
"Table/view is missing metadata: " + table.getFullName());
}
return table;
}
*Now I trid setting*
1.'export JAVA_TOOL_OPTIONS=" -Xmx40g"' ,
2. 'IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} -mem_limit=-1b"'
3.'IMPALA_SERVER_ARGS=" -mem_limit=-1b"'
but still 'java.lang.OutOfMemoryError' .
*Can you give me some suggestion ?*
*Thanks & Best Regards.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
