[
https://issues.apache.org/jira/browse/HBASE-26605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17462002#comment-17462002
]
Josh Elser commented on HBASE-26605:
------------------------------------
Also, have not evaluated other active branches for relevance. Will do that
after I have a fix in mind for 2.4.
> TestHStore#testRefreshStoreFiles broken due to unqualified and qualified paths
> ------------------------------------------------------------------------------
>
> Key: HBASE-26605
> URL: https://issues.apache.org/jira/browse/HBASE-26605
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 2.4.9
> Reporter: Josh Elser
> Assignee: Josh Elser
> Priority: Minor
>
> Was looking at a failures of this method where
> {noformat}
> [ERROR] org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles
> Time elapsed: 4.2 s <<< ERROR!
> java.util.NoSuchElementException
> at
> org.apache.hbase.thirdparty.com.google.common.collect.AbstractIndexedListIterator.next(AbstractIndexedListIterator.java:75)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.closeCompactedFile(TestHStore.java:962)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:1000)
> {noformat}
> This was on a branch where I had some HBASE-26067 changes backported, so I
> thought the problem _was_ those changes. After a bit of digging, I believe
> the test case itself is "broken" (the test passes, but for the wrong reasons).
> This test methods adds some files to a Store (via memstore flush or direct
> addition of a file) and eventually tries to get the first file which is
> candidate to be removed. The test {*}never compacted any files{*}. This was
> the first sign that the test itself was wrong.
> After lots of comparison with the HBASE-26067 logging to compare against, I
> found that the Store was listing a file which was created by the memstore
> flush as a file to retain AND a file to remove. Second warning. Upon closer
> inspection, I finally noticed that one of the files was qualified with the
> filesystem URI and the other was not.
> {noformat}
> 2021-12-18 16:57:10,903 INFO [Time-limited test] regionserver.HStore(675):
> toBeAddedFiles=[file:/Users/jelser/projects/cldr/hbase-copy.git/hbase-server/target/test-data/f4ed7913-e62a-8d5f-a968-bb4c94d5494a/TestStoretestRefreshStoreFiles/data/default/table/297ad8361c3326bfb1520dbc54b1c3bd/family/dd8a430b391546d8b9bdc39bb77d447b,
>
> file:/Users/jelser/projects/cldr/hbase-copy.git/hbase-server/target/test-data/f4ed7913-e62a-8d5f-a968-bb4c94d5494a/TestStoretestRefreshStoreFiles/data/default/table/297ad8361c3326bfb1520dbc54b1c3bd/family/d4c5442b772c43fd9ebdfed1a11c0e73],
>
> toBeRemovedFiles=[/Users/jelser/projects/cldr/hbase-copy.git/hbase-server/target/test-data/f4ed7913-e62a-8d5f-a968-bb4c94d5494a/TestStoretestRefreshStoreFiles/data/default/table/297ad8361c3326bfb1520dbc54b1c3bd/family/d4c5442b772c43fd9ebdfed1a11c0e73]
> {noformat}
> {{d4c5442b772c43fd9ebdfed1a11c0e73}} how are we both adding and removing this
> file! Turns out, this is because one of them is "/..." and the other is
> "file:/...". Either the problem is in TestHStore in how it is creating/adding
> these files behind the scenes or we should be qualifying the Path inside of
> StoreFileInfo with the filesystem that we're using.
> I remember too vividly the problems when trying to separate the rootdir and
> waldir from each other and am cautious against adding something to
> StoreFileInfo to {{{}fs.qualifyPath(p){}}}. Need to look some more, but will
> get a patch up to fix.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
