Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19233
Change subject: IMPALA-11720: Deflake FileMetadataLoaderTest due to FileSystem closed ...................................................................... IMPALA-11720: Deflake FileMetadataLoaderTest due to FileSystem closed IMPALA-11699 recently reverted the changes in FileMetadataLoaderTest that it no longer extends to FrontendTestBase. This causes tests in FileMetadataLoaderTest to become flaky if run individually. This is a subtle bug happens in FileMetadataLoader#load(), where we first call FileSystem#getFileSystem() to get a FileSystem object, and then call methods of FileSystemUtil which will trigger class loading of it and running its static code. FileSystem#getFileSystem() will create a FileSystem object in the first get and cache it for follow-up usage. Without extending FrontendTestBase, BackendConfig.INSTANCE is not initialized when the test is run. So the static code in FileSystemUtil lazily initializes BackendConfig.INSTANCE, which initializes FeSupport and finally calls JniUtil::InitLibhdfs(). In this method, a FileSystem object is get and closed. This is exactly the FileSystem object creates in FileMetadataLoader#load(). So the following usage on it causes an IOException of "Filesystem closed". The purpose of JniUtil::InitLibhdfs() is to make a simple call on libhdfs to make it initialize the JVM. This is crucial when launching from c++ codes for impalad and catalogd to init the embedded JVM. So we should keep it unchanged. The fix for this bug is not initializing BackendConfig.INSTANCE in the static code of FileSystemUtil, which avoids the subtle side effects. Tests: - Verified tests in FileMetadataLoaderTest individually Change-Id: Ib6f96950210c9a0124fe03696ef334cb00b057ab --- M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java 1 file changed, 11 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/19233/1 -- To view, visit http://gerrit.cloudera.org:8080/19233 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib6f96950210c9a0124fe03696ef334cb00b057ab Gerrit-Change-Number: 19233 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <[email protected]>
