[
https://issues.apache.org/jira/browse/IMPALA-10080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185774#comment-17185774
]
ASF subversion and git services commented on IMPALA-10080:
----------------------------------------------------------
Commit 568b3394b2945d684d8fdb6c4f4e1f33cbf98898 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=568b339 ]
IMPALA-10080: Skip loading HDFS cache pools for non-HDFS file systems
In global invalidate metadata, we always load HDFS cache pools using the
CachePoolReader. Actually, it only works for HDFS file systems, not for
other systems like S3 or local, etc. We already handle this in
CatalogServiceCatalog#CatalogServiceCatalog(). This patch adds a check
in CatalogServiceCatalog#reset() to skip loading cache pools if it's not
a true HDFS file system.
Tests
- Ran tests on S3. Verified that the IllegalStateException doesn't
exists anymore.
Change-Id: Ib243d349177e1b982b313dd6e87ecc2ef4dfc3d8
Reviewed-on: http://gerrit.cloudera.org:8080/16335
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Skip loading HDFS cache pools for non-HDFS file systems
> -------------------------------------------------------
>
> Key: IMPALA-10080
> URL: https://issues.apache.org/jira/browse/IMPALA-10080
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.0, Impala 2.12.0, Impala 3.1.0, Impala 3.2.0,
> Impala 3.3.0, Impala 3.4.0
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> Saw some IllegalStateExceptions when running on non-HDFS filesystesms:
> {code}
> I0812 01:48:31.159904 5375 CatalogServiceCatalog.java:1601] Invalidating all
> metadata. Version: 0
> E0812 01:48:31.164568 5375 CatalogServiceCatalog.java:519] Error loading
> cache pools:
> Java exception follows:
> java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:492)
> at
> org.apache.impala.common.FileSystemUtil.getDistributedFileSystem(FileSystemUtil.java:465)
> at
> org.apache.impala.catalog.CatalogServiceCatalog$CachePoolReader.run(CatalogServiceCatalog.java:512)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.reset(CatalogServiceCatalog.java:1620)
> at org.apache.impala.service.JniCatalog.<init>(JniCatalog.java:137)
> {code}
> This won't break anything. But it's good to skip this as what we have already
> done in CatalogServiceCatalog#CatalogServiceCatalog().
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
