[
https://issues.apache.org/jira/browse/IMPALA-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang resolved IMPALA-13009.
-------------------------------------
Fix Version/s: Impala 4.5.0
Resolution: Fixed
> Potential leak of partition deletions in the catalog topic
> ----------------------------------------------------------
>
> Key: IMPALA-13009
> URL: https://issues.apache.org/jira/browse/IMPALA-13009
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 4.0.0, Impala 4.1.0, Impala 4.2.0, Impala 4.1.1,
> Impala 4.1.2, Impala 4.3.0
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Fix For: Impala 4.5.0
>
>
> Catalogd might not send partition deletions to the catalog topic in the
> following scenario:
> * Some partitions of a table are dropped.
> * The HdfsTable object is removed sequentially before catalogd collects the
> dropped partitions.
> In such case, catalogd loses track of the dropped partitions so their updates
> keep existing in the catalog topic, until the partition names are reused
> again.
> Note that the HdfsTable object can be removed by commands like DropTable or
> INVALIDATE.
> The leaked partitions will be detected when a coordinator restarts. An
> IllegalStateException complaining stale partitions will be reported, causing
> the table not being added to the catalog cache of coordinator.
> {noformat}
> E0417 16:41:22.317298 20746 ImpaladCatalog.java:264] Error adding catalog
> object: Received stale partition in a statestore update:
> THdfsPartition(partitionKeyExprs:[TExpr(nodes:[TExprNode(node_type:INT_LITERAL,
> type:TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:INT))]), num_children:0, is_constant:true,
> int_literal:TIntLiteral(value:106), is_codegen_disabled:false)])],
> location:THdfsPartitionLocation(prefix_index:0, suffix:p=106), id:138,
> file_desc:[THdfsFileDesc(file_desc_data:18 00 00 00 00 00 00 00 00 00 0E 00
> 1C 00 18 00 10 00 00 00 08 00 04 00 0E 00 00 00 18 00 00 00 8B 0E 2D EB 8E 01
> 00 00 04 00 00 00 00 00 00 00 0C 00 00 00 01 00 00 00 4C 00 00 00 36 00 00 00
> 34 34 34 37 62 35 66 34 62 30 65 64 66 64 65 31 2D 32 33 33 61 64 62 38 35 30
> 30 30 30 30 30 30 30 5F 36 36 34 31 30 39 33 37 33 5F 64 61 74 61 2E 30 2E 74
> 78 74 00 00 0C 00 14 00 00 00 0C 00...)], access_level:READ_WRITE,
> stats:TTableStats(num_rows:-1), is_marked_cached:false,
> hms_parameters:{transient_lastDdlTime=1713342582, totalSize=4,
> numFilesErasureCoded=0, numFiles=1}, num_blocks:1, total_file_size_bytes:4,
> has_incremental_stats:false, write_id:0, db_name:default, tbl_name:my_part,
> partition_name:p=106,
> hdfs_storage_descriptor:THdfsStorageDescriptor(lineDelim:10, fieldDelim:1,
> collectionDelim:1, mapKeyDelim:1, escapeChar:0, quoteChar:1, fileFormat:TEXT,
> blockSize:0))
> Java exception follows:
> java.lang.IllegalStateException: Received stale partition in a statestore
> update:
> THdfsPartition(partitionKeyExprs:[TExpr(nodes:[TExprNode(node_type:INT_LITERAL,
> type:TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:INT))]), num_children:0, is_constant:true,
> int_literal:TIntLiteral(value:106), is_codegen_disabled:false)])],
> location:THdfsPartitionLocation(prefix_index:0, suffix:p=106), id:138,
> file_desc:[THdfsFileDesc(file_desc_data:18 00 00 00 00 00 00 00 00 00 0E 00
> 1C 00 18 00 10 00 00 00 08 00 04 00 0E 00 00 00 18 00 00 00 8B 0E 2D EB 8E 01
> 00 00 04 00 00 00 00 00 00 00 0C 00 00 00 01 00 00 00 4C 00 00 00 36 00 00 00
> 34 34 34 37 62 35 66 34 62 30 65 64 66 64 65 31 2D 32 33 33 61 64 62 38 35 30
> 30 30 30 30 30 30 30 5F 36 36 34 31 30 39 33 37 33 5F 64 61 74 61 2E 30 2E 74
> 78 74 00 00 0C 00 14 00 00 00 0C 00...)], access_level:READ_WRITE,
> stats:TTableStats(num_rows:-1), is_marked_cached:false,
> hms_parameters:{transient_lastDdlTime=1713342582, totalSize=4,
> numFilesErasureCoded=0, numFiles=1}, num_blocks:1, total_file_size_bytes:4,
> has_incremental_stats:false, write_id:0, db_name:default, tbl_name:my_part,
> partition_name:p=106,
> hdfs_storage_descriptor:THdfsStorageDescriptor(lineDelim:10, fieldDelim:1,
> collectionDelim:1, mapKeyDelim:1, escapeChar:0, quoteChar:1, fileFormat:TEXT,
> blockSize:0))
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:512)
> at
> org.apache.impala.catalog.ImpaladCatalog.addTable(ImpaladCatalog.java:523)
> at
> org.apache.impala.catalog.ImpaladCatalog.addCatalogObject(ImpaladCatalog.java:334)
> at
> org.apache.impala.catalog.ImpaladCatalog.updateCatalog(ImpaladCatalog.java:262)
> at
> org.apache.impala.service.FeCatalogManager$CatalogdImpl.updateCatalogCache(FeCatalogManager.java:120)
> at
> org.apache.impala.service.Frontend.updateCatalogCache(Frontend.java:565)
> at
> org.apache.impala.service.JniFrontend.updateCatalogCache(JniFrontend.java:196)
> {noformat}
> *Workaround*
> Add back the stale partitions to the table (if the table is also dropped, add
> back the table first) using the same partition names. Then drop them again.
> This gives catalogd chances to send deletions for them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
