[GitHub] [hbase] VicoWu commented on pull request #1743: HBASE-24403 FsDelegationToken Should Cache Token After Acquired A New One
VicoWu commented on pull request #1743: URL: https://github.com/apache/hbase/pull/1743#issuecomment-638133811 @Apache9 Thanks so much. I have fixed the checkstyle issue and also enhance the test case for my fix. Currently everything is fine now. Please help to re-review it. Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] VicoWu commented on pull request #1743: HBASE-24403 FsDelegationToken Should Cache Token After Acquired A New One
VicoWu commented on pull request #1743: URL: https://github.com/apache/hbase/pull/1743#issuecomment-635050379 > @jojochuang > This issue is found by our cluster incident. > But my current commit still has some problem which need to be resolved before finally adopted: > > In this PR, I cache the token between RS and NameNode instead of always applying for a new one every time processing a file, so the token should be confirmed that it is not expired every time we use it; > So I think I need to add a check logic to verify its expiration time , if it is already or going to be expired, I will have to apply for a new one and overwrite the cache; @jojochuang I have added the token expiration processing logic and help to review it; The logic has updated to the PR's description part; This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] VicoWu commented on pull request #1743: HBASE-24403 FsDelegationToken should cache token after acquired a new one
VicoWu commented on pull request #1743: URL: https://github.com/apache/hbase/pull/1743#issuecomment-632964520 > @jojochuang > This issue is found by our cluster incident. > But my current commit still has some problem which need to be resolved before finally adopted: > > In this PR, I cache the token between RS and NameNode instead of always applying for a new one every time processing a file, so the token should be confirmed that it is not expired every time before we use it; > So I think I need to add a check logic to verify its expiration time , if it is already or going to be expired, I will have to apply for a new one and overwrite the cache; @jojochuang Would you please have a review for my above fix planning? If you think it is ok, I will add the token expiration process logic to avoid the cached token expiration and apply for new token before it is expired; This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] VicoWu commented on pull request #1743: HBASE-24403 FsDelegationToken should cache token after acquired a new one
VicoWu commented on pull request #1743: URL: https://github.com/apache/hbase/pull/1743#issuecomment-632964260 > **-1 overall** > > Vote Subsystem Runtime Comment > +0 reexec 0m 37s Docker mode activated. > -0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck > _ Prechecks _ > _ master Compile Tests _ > +1 mvninstall 3m 49s master passed > +1 compile 0m 55s master passed > +1 shadedjars 5m 40s branch has no errors when building our shaded downstream artifacts. > +1 javadoc 0m 39s master passed > _ Patch Compile Tests _ > +1 mvninstall 3m 26s the patch passed > +1 compile 0m 55s the patch passed > +1 javac 0m 55s the patch passed > +1 shadedjars 5m 30s patch has no errors when building our shaded downstream artifacts. > +1 javadoc 0m 36s the patch passed > _ Other Tests _ > -1 ❌ unit143m 39shbase-server in the patch failed. > 167m 43s > Subsystem Report/Notes > Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1743/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile > GITHUB PR #1743 > Optional Tests javac javadoc unit shadedjars compile > uname Linux 6292c20d21b4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux > Build tool maven > Personalitydev-support/hbase-personality.sh > git revision master / [a8724e8](https://github.com/apache/hbase/commit/a8724e8120738488d863ac547495d3209b898e83) > Default Java 1.8.0_232 > unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1743/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt > Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1743/2/testReport/ > Max. process+thread count 3910 (vs. ulimit of 12500) > modulesC: hbase-server U: hbase-server > Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1743/2/console > versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) > Powered by Apache Yetus 0.11.1 https://yetus.apache.org > This message was automatically generated. This failure is caused by connection loss to zookeeper. has nothing to do with my patch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] VicoWu commented on pull request #1743: HBASE-24403 FsDelegationToken should cache token after acquired a new one
VicoWu commented on pull request #1743: URL: https://github.com/apache/hbase/pull/1743#issuecomment-632431531 > @VicoWu Do you have a Jira ticket related to this one? [HBASE-24403](https://issues.apache.org/jira/browse/HBASE-24403) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] VicoWu commented on pull request #1743: HBASE-24403 FsDelegationToken should cache token after acquired a new one
VicoWu commented on pull request #1743: URL: https://github.com/apache/hbase/pull/1743#issuecomment-632031498 @jojochuang This issue is found by our cluster incident. But my current commit still has some problem which need to be resolved before finally adopted: In this PR, I cache the token between RS and NameNode instead of always applying for a new one every time processing a file, so the token should be confirmed that it is not expired every time we use it; So I think I need to add a check logic to verify its expiration time , if it is already or going to be expired, I will have to apply for a new one and overwrite the cache; This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
