[
https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854991#comment-16854991
]
Vihang Karajgaonkar commented on IMPALA-8567:
---------------------------------------------
While tracing the actual query which fails its not very clear why the the table
would be missing from the local catalog cache.
Following logs are helpful to understand what is happening for the table in
question \{{test_parquet_stats_209e5179.test_nan_true_predicate}}
Impala coordinator 0 receives the create table sql (query_id =
e2493bdcaf66a03c:a980f9df00000000)
{code:java}
I0601 15:54:28.168197 63083 impala-beeswax-server.cc:54] query(): query=create
table test_nan_true_predicate(val double) stored as parquet
I0601 15:54:28.168256 63083 impala-beeswax-server.cc:432] query: Query {
01: query (string) = "create table test_nan_true_predicate(val double) stored
as parquet",
03: configuration (list) = list<string>[7] {
[0] = "CLIENT_IDENTIFIE[...](273)",
[1] = "BATCH_SIZE=0",
[2] = "NUM_NODES=1",
[3] = "DISABLE_CODEGEN_ROWS_THRESHOLD=0",
[4] = "DISABLE_CODEGEN=False",
[5] = "ABORT_ON_ERROR=1",
[6] = "EXEC_SINGLE_NODE_ROWS_THRESHOLD=0",
},
04: hadoop_user (string) = "ubuntu",
}
{code}
There is no log in catalogd which tells that it is creating a table, but I can
see that table is created in HMS based on the following log:
{code:java}
2019-06-01T15:54:28,234 INFO [pool-8-thread-14] metastore.HiveMetaStore: 14:
source:172.18.0.3 create_table: Table(tableName:test_nan_true_predicate,
dbName:test_parquet_stats_209e5179, owner:ubuntu, createTime:0,
lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:val,
type:double, comment:null)], location:null,
inputFormat:org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat,
outputFormat:org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat,
compressed:false, numBuckets:0, serdeInfo:SerDeInfo(name:null,
serializationLib:org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe,
parameters:{}), bucketCols:null, sortCols:null, parameters:null),
partitionKeys:[], parameters:{}, viewOriginalText:null, viewExpandedText:null,
tableType:MANAGED_TABLE, ownerType:USER)
{code}
I think I will need some help to debug this further. Specifically, how is the
createTable update propagated to a v2 client. Does it take certain action which
can remove the cached entry in the local catalog cache?
> Many random catalog consistency issues with catalog v2/event processor
> ----------------------------------------------------------------------
>
> Key: IMPALA-8567
> URL: https://issues.apache.org/jira/browse/IMPALA-8567
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.3.0
> Reporter: Tim Armstrong
> Assignee: Vihang Karajgaonkar
> Priority: Blocker
> Labels: broken-build, catalog, flaky
>
> [~tlipcon] [~vihangk1] FYI. I'm not sure whether the local catalog or the
> event processor is likely to blame here so I'll let you look. The general
> theme is tables and databases not existing when they should.
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/289/testReport/junit/metadata.test_refresh_partition/TestRefreshPartition/test_drop_hive_partition_and_refresh_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/267/testReport/junit/query_test.test_kudu/TestKuduOperations/test_kudu_insert_protocol__beeswax___exec_option____kudu_read_mode____READ_AT_SNAPSHOT____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_metadata_query_statements/TestMetadataQueryStatements/test_describe_db_protocol__beeswax___exec_option____sync_ddl___0___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_hms_integration/TestHmsIntegrationSanity/test_sanity_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/288/testReport/junit/query_test.test_insert_parquet/TestHdfsParquetTableStatsWriter/test_write_statistics_multiple_row_groups_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__parquet_none_/
> I'll include the output of each job in a follow-on comment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
