[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20401: --- Fix Version/s: (was: 1.5.0) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 1.3.3, 1.4.6, 2.2.0, 2.1.1, 2.0.2 > > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, > HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20401: --- Fix Version/s: 1.3.3 > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.4.6, 2.2.0, 2.1.1, 2.0.2 > > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, > HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-20401: -- Resolution: Resolved Fix Version/s: 1.4.6 1.5.0 Status: Resolved (was: Patch Available) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 1.5.0, 1.4.6, 2.0.2, 2.2.0, 2.1.1 > > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, > HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.branch-1.003.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1 > > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, > HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Status: Patch Available (was: Reopened) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 2.0.0, 1.4.4, 2.0.0-beta-1, 3.0.0, 1.5.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1 > > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.master.001.patch, > HBASE-20401.master.002.patch, HBASE-20401.master.003.patch, > HBASE-20401.master.004.patch, HBASE-20401.master.005.patch, > HBASE-20401.master.006.patch, HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.branch-1.002.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1 > > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.master.001.patch, > HBASE-20401.master.002.patch, HBASE-20401.master.003.patch, > HBASE-20401.master.004.patch, HBASE-20401.master.005.patch, > HBASE-20401.master.006.patch, HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: (was: HBASE-20401.branch-1.003.patch) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1 > > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, > HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.branch-1.003.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1 > > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, > HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-20401: -- Resolution: Resolved Hadoop Flags: Reviewed Fix Version/s: 2.1.1 2.2.0 2.0.2 3.0.0 Status: Resolved (was: Patch Available) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1 > > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, > HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.master.007.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, > HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Release Note: When oldwals (and hfile) cleaner cleans stale wals (and hfiles), it will periodically check and wait the clean results from filesystem, the total wait time will be no more than a max time. The periodically wait and check configurations are hbase.oldwals.cleaner.thread.check.interval.msec (default is 500 ms) and hbase.regionserver.hfilecleaner.thread.check.interval.msec (default is 1000 ms). Meanwhile, The max time configurations are hbase.oldwals.cleaner.thread.timeout.msec and hbase.regionserver.hfilecleaner.thread.timeout.msec, they are set to 60 seconds by default. All support dynamic configuration. e.g. in the oldwals cleaning scenario, one may consider tuning hbase.oldwals.cleaner.thread.timeout.msec and hbase.oldwals.cleaner.thread.check.interval.msec 1. While deleting a oldwal never complete (strange but possible), then delete file task needs to wait for a max of 60 seconds. Here, 60 seconds might be too long, or the opposite way is to increase more than 60 seconds in the use cases of slow file delete. 2. The check and wait of a file delete is set to default in the period of 500 milliseconds, one might want to tune this checking period to a short interval to check more frequently or to a longer interval to avoid checking too often to manage their delete file task checking period (the longer interval may be use to avoid checking too fast while using a high latency storage). was: Add thread-level timeout and check interval configurations in LogCleaner and HFileCleaner, such that one can tune the file cleaners to match their environment. 1. hbase.oldwals.cleaner.thread.timeout.msec 2. hbase.oldwals.cleaner.thread.check.interval.msec 3. hbase.regionserver.hfilecleaner.thread.timeout.msec 4. hbase.regionserver.hfilecleaner.thread.check.interval.msec For example in the LogCleaner scenario, one might consider tuning hbase.oldwals.cleaner.thread.timeout.msec and hbase.oldwals.cleaner.thread.check.interval.msec 1. fs.delete never complete (strange but possible), then we need to wait for a max of 60 seconds. here, 60 seconds might be too long. or the opposite way is to increase more than 60 seconds in the use cases of slow oldwals delete. 2. getResult is waiting in the period of 500 milliseconds, but the fs.delete has completed and setFromCleaner is set but yet notify(). one might want to tune this 500 milliseconds to 200 or less . > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.master.006.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Release Note: Add thread-level timeout and check interval configurations in LogCleaner and HFileCleaner, such that one can tune the file cleaners to match their environment. 1. hbase.oldwals.cleaner.thread.timeout.msec 2. hbase.oldwals.cleaner.thread.check.interval.msec 3. hbase.regionserver.hfilecleaner.thread.timeout.msec 4. hbase.regionserver.hfilecleaner.thread.check.interval.msec For example in the LogCleaner scenario, one might consider tuning hbase.oldwals.cleaner.thread.timeout.msec and hbase.oldwals.cleaner.thread.check.interval.msec 1. fs.delete never complete (strange but possible), then we need to wait for a max of 60 seconds. here, 60 seconds might be too long. or the opposite way is to increase more than 60 seconds in the use cases of slow oldwals delete. 2. getResult is waiting in the period of 500 milliseconds, but the fs.delete has completed and setFromCleaner is set but yet notify(). one might want to tune this 500 milliseconds to 200 or less . was: Add thread-level timeout and check interval configurations in LogCleaner and HFileCleaner, such that one can tune the file cleaners to match their environment. 1. hbase.oldwals.cleaner.thread.timeout.msec 2. hbase.oldwals.cleaner.thread.check.interval.msec 3. hbase.regionserver.hfilecleaner.thread.timeout.msec 4. hbase.regionserver.hfilecleaner.thread.check.interval.msec For example in the LogCleaner scenario, one might consider tuning hbase.oldwals.cleaner.thread.timeout.msec and hbase.oldwals.cleaner.thread.check.interval.msec 1. fs.delete never complete (strange but possible), then we need to wait for a max of 60 seconds. here, 60 seconds might be too long. or the opposite way is to increase more than 60 seconds in the use cases of slow oldwals delete. 2. getResult is waiting in the period of 500 milliseconds, but the fs.delete has completed and setFromClear is set but yet notify(). one might want to tune this 500 milliseconds to 200 or less . > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Release Note: Add thread-level timeout and check interval configurations in LogCleaner and HFileCleaner, such that one can tune the file cleaners to match their environment. 1. hbase.oldwals.cleaner.thread.timeout.msec 2. hbase.oldwals.cleaner.thread.check.interval.msec 3. hbase.regionserver.hfilecleaner.thread.timeout.msec 4. hbase.regionserver.hfilecleaner.thread.check.interval.msec For example in the LogCleaner scenario, one might consider tuning hbase.oldwals.cleaner.thread.timeout.msec and hbase.oldwals.cleaner.thread.check.interval.msec 1. fs.delete never complete (strange but possible), then we need to wait for a max of 60 seconds. here, 60 seconds might be too long. or the opposite way is to increase more than 60 seconds in the use cases of slow oldwals delete. 2. getResult is waiting in the period of 500 milliseconds, but the fs.delete has completed and setFromClear is set but yet notify(). one might want to tune this 500 milliseconds to 200 or less . was: Add thread-level timeout and check interval configurations in LogCleaner and HFileCleaner, such that one can tune the file cleaners to match their environment. 1. hbase.oldwals.cleaner.thread.timeout.msec 1. hbase.oldwals.cleaner.thread.check.interval.msec 1. hbase.regionserver.hfilecleaner.thread.timeout.msec 1. hbase.regionserver.hfilecleaner.thread.check.interval.msec For example in the LogCleaner scenario, one might consider tuning hbase.oldwals.cleaner.thread.timeout.msec and hbase.oldwals.cleaner.thread.check.interval.msec 1. fs.delete never complete (strange but possible), then we need to wait for a max of 60 seconds. here, 60 seconds might be too long. or the opposite way is to increase more than 60 seconds in the use cases of slow oldwals delete. 2. getResult is waiting in the period of 500 milliseconds, but the fs.delete has completed and setFromClear is set but yet notify(). one might want to tune this 500 milliseconds to 200 or less . > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Release Note: Add thread-level timeout and check interval configurations in LogCleaner and HFileCleaner, such that one can tune the file cleaners to match their environment. 1. hbase.oldwals.cleaner.thread.timeout.msec 1. hbase.oldwals.cleaner.thread.check.interval.msec 1. hbase.regionserver.hfilecleaner.thread.timeout.msec 1. hbase.regionserver.hfilecleaner.thread.check.interval.msec For example in the LogCleaner scenario, one might consider tuning hbase.oldwals.cleaner.thread.timeout.msec and hbase.oldwals.cleaner.thread.check.interval.msec 1. fs.delete never complete (strange but possible), then we need to wait for a max of 60 seconds. here, 60 seconds might be too long. or the opposite way is to increase more than 60 seconds in the use cases of slow oldwals delete. 2. getResult is waiting in the period of 500 milliseconds, but the fs.delete has completed and setFromClear is set but yet notify(). one might want to tune this 500 milliseconds to 200 or less . was: Add thread-level timeout and check interval configurations in LogCleaner and HFileCleaner, such that one can tune the file cleaners to match their environment. 1. hbase.oldwals.cleaner.thread.timeout.msec 2. hbase.oldwals.cleaner.thread.check.interval.msec 3. hbase.regionserver.hfilecleaner.thread.timeout.msec 4. hbase.regionserver.hfilecleaner.thread.check.interval.msec For example in the LogCleaner scenario, one might consider tuning hbase.oldwals.cleaner.thread.timeout.msec and hbase.oldwals.cleaner.thread.check.interval.msec 1. fs.delete never complete (strange but possible), then we need to wait for a max of 60 seconds. here, 60 seconds might be too long. or the opposite way is to increase more than 60 seconds in the use cases of slow oldwals delete. 2. getResult is waiting in the period of 500 milliseconds, but the fs.delete has completed and setFromClear is set but yet notify(). one might want to tune this 500 milliseconds to 200 or less . > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Release Note: Add thread-level timeout and check interval configurations in LogCleaner and HFileCleaner, such that one can tune the file cleaners to match their environment. 1. hbase.oldwals.cleaner.thread.timeout.msec 2. hbase.oldwals.cleaner.thread.check.interval.msec 3. hbase.regionserver.hfilecleaner.thread.timeout.msec 4. hbase.regionserver.hfilecleaner.thread.check.interval.msec For example in the LogCleaner scenario, one might consider tuning hbase.oldwals.cleaner.thread.timeout.msec and hbase.oldwals.cleaner.thread.check.interval.msec 1. fs.delete never complete (strange but possible), then we need to wait for a max of 60 seconds. here, 60 seconds might be too long. or the opposite way is to increase more than 60 seconds in the use cases of slow oldwals delete. 2. getResult is waiting in the period of 500 milliseconds, but the fs.delete has completed and setFromClear is set but yet notify(). one might want to tune this 500 milliseconds to 200 or less . was: Add thread-level timeout and check interval configurations in LogCleaner and HFileCleaner, such that one can tune the file cleaners to match their environment. 1. hbase.oldwals.cleaner.thread.timeout.msec 2. hbase.oldwals.cleaner.thread.check.interval.msec 3. hbase.regionserver.hfilecleaner.thread.timeout.msec 4. hbase.regionserver.hfilecleaner.thread.check.interval.msec > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.branch-1.001.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: (was: HBASE-20401.branch-1.001.patch) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.master.001.patch, > HBASE-20401.master.002.patch, HBASE-20401.master.003.patch, > HBASE-20401.master.004.patch, HBASE-20401.master.005.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: (was: HBASE-20401.branch-1.002.patch) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.master.001.patch, > HBASE-20401.master.002.patch, HBASE-20401.master.003.patch, > HBASE-20401.master.004.patch, HBASE-20401.master.005.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: (was: HBASE-20401.branch-2.001.patch) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.master.001.patch, > HBASE-20401.master.002.patch, HBASE-20401.master.003.patch, > HBASE-20401.master.004.patch, HBASE-20401.master.005.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Release Note: Add thread-level timeout and check interval configurations in LogCleaner and HFileCleaner, such that one can tune the file cleaners to match their environment. 1. hbase.oldwals.cleaner.thread.timeout.msec 2. hbase.oldwals.cleaner.thread.check.interval.msec 3. hbase.regionserver.hfilecleaner.thread.timeout.msec 4. hbase.regionserver.hfilecleaner.thread.check.interval.msec > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.master.005.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.master.004.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.master.003.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.branch-1.002.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.branch-2.001.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.master.002.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.master.001.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: (was: HBASE-20401.master.001.patch) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.branch-1.001.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.master.001.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Attachment: HBASE-20401.master.001.patch > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.master.001.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Status: Patch Available (was: Open) I will add branch-1 and branch-2 patches later today > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 2.0.0, 1.4.4, 2.0.0-beta-1, 3.0.0, 1.5.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Attachments: HBASE-20401.master.001.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20401: Labels: beginner (was: easyfix newbie) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Tags: performance, HMaster, LogCleaner, (was: performance) Labels: easyfix newbie (was: ) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Priority: Minor > Labels: easyfix, newbie > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-20401: - Tags: performance Component/s: master > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Priority: Minor > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
