[
https://issues.apache.org/jira/browse/IMPALA-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bharath v updated IMPALA-7717:
------------------------------
Description:
Concurrent invalidates with partial RPC on partitioned tables can throw this
exception.
{noformat}
I1016 15:49:03.438048 30197 jni-util.cc:256]
java.lang.IllegalArgumentException: Partition id 162 does not exist
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
at org.apache.impala.catalog.HdfsTable.getPartialInfo(HdfsTable.java:1711)
at
org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:2202)
at
org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:2141)
at
org.apache.impala.service.JniCatalog.getPartialCatalogObject(JniCatalog.java:237)
I1016 15:49:03.440939 30197 status.cc:129] IllegalArgumentException: Partition
id 162 does not exist
{noformat}
{noformat}
@Override
public TGetPartialCatalogObjectResponse getPartialInfo(
TGetPartialCatalogObjectRequest req) throws TableLoadingException {
if (partIds != null) {
resp.table_info.partitions =
Lists.newArrayListWithCapacity(partIds.size());
for (long partId : partIds) {
HdfsPartition part = partitionMap_.get(partId);
Preconditions.checkArgument(part != null, "Partition id %s does not
exist",
partId); <====
{noformat}
The issue is that the invalidate command can reset the partition IDs and the
RPCs could look up with older IDs.
We should wrap this into an inconsistent metadata fetch exception and retry
rather than throwing a RTE.
was:
Concurrent invalidates with partial RPC on partitioned tables can throw this
exception.
{noformat}
@Override
public TGetPartialCatalogObjectResponse getPartialInfo(
TGetPartialCatalogObjectRequest req) throws TableLoadingException {
if (partIds != null) {
resp.table_info.partitions =
Lists.newArrayListWithCapacity(partIds.size());
for (long partId : partIds) {
HdfsPartition part = partitionMap_.get(partId);
Preconditions.checkArgument(part != null, "Partition id %s does not
exist",
partId); <====
{noformat}
The issue is that the invalidate command can reset the partition IDs and the
RPCs could look up with older IDs.
We should wrap this into an inconsistent metadata fetch exception and retry
rather than throwing a RTE.
> Partition id does not exist exception - Catalog V2
> --------------------------------------------------
>
> Key: IMPALA-7717
> URL: https://issues.apache.org/jira/browse/IMPALA-7717
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: bharath v
> Priority: Major
>
> Concurrent invalidates with partial RPC on partitioned tables can throw this
> exception.
> {noformat}
> I1016 15:49:03.438048 30197 jni-util.cc:256]
> java.lang.IllegalArgumentException: Partition id 162 does not exist
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
> at org.apache.impala.catalog.HdfsTable.getPartialInfo(HdfsTable.java:1711)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:2202)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:2141)
> at
> org.apache.impala.service.JniCatalog.getPartialCatalogObject(JniCatalog.java:237)
> I1016 15:49:03.440939 30197 status.cc:129] IllegalArgumentException:
> Partition id 162 does not exist
> {noformat}
> {noformat}
> @Override
> public TGetPartialCatalogObjectResponse getPartialInfo(
> TGetPartialCatalogObjectRequest req) throws TableLoadingException {
>
> if (partIds != null) {
> resp.table_info.partitions =
> Lists.newArrayListWithCapacity(partIds.size());
> for (long partId : partIds) {
> HdfsPartition part = partitionMap_.get(partId);
> Preconditions.checkArgument(part != null, "Partition id %s does not
> exist",
> partId); <====
> {noformat}
> The issue is that the invalidate command can reset the partition IDs and the
> RPCs could look up with older IDs.
> We should wrap this into an inconsistent metadata fetch exception and retry
> rather than throwing a RTE.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
