[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #1456: HDDS-4172. Implement Finalize command in Ozone Manager server.

2020-10-02 Thread GitBox
fapifta commented on a change in pull request #1456: URL: https://github.com/apache/hadoop-ozone/pull/1456#discussion_r498761138 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/StorageInfo.java ## @@ -150,6 +179,16 @@ private void verifyClusterId(

[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #1456: HDDS-4172. Implement Finalize command in Ozone Manager server.

2020-10-02 Thread GitBox
fapifta commented on a change in pull request #1456: URL: https://github.com/apache/hadoop-ozone/pull/1456#discussion_r498760977 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMUpgradeFinalizer.java ## @@ -0,0 +1,303 @@ +/** + * Lice

[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #1456: HDDS-4172. Implement Finalize command in Ozone Manager server.

2020-10-02 Thread GitBox
fapifta commented on a change in pull request #1456: URL: https://github.com/apache/hadoop-ozone/pull/1456#discussion_r498760785 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMUpgradeFinalizer.java ## @@ -0,0 +1,303 @@ +/** + * Lice

[GitHub] [hadoop-ozone] rakeshadr commented on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-10-02 Thread GitBox
rakeshadr commented on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-702608629 Thank you @bharatviswa504 and @linyiqun for help in reviews and provided really useful comments. Thi

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1460: HDDS-4298. Use an interface in Ozone client instead of XceiverClientManager

2020-10-02 Thread GitBox
adoroszlai commented on a change in pull request #1460: URL: https://github.com/apache/hadoop-ozone/pull/1460#discussion_r498684631 ## File path: hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientFactory.java ## @@ -0,0 +1,44 @@ +/* + * Licensed to the A

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1467: HDDS-4268. Prepare for Upgrade step should purge the log after waiting for the last txn to be applied.

2020-10-02 Thread GitBox
linyiqun commented on a change in pull request #1467: URL: https://github.com/apache/hadoop-ozone/pull/1467#discussion_r498649951 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerRatisServer.java ## @@ -432,7 +443,14 @@ privat

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-10-02 Thread GitBox
adoroszlai commented on pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#issuecomment-702565283 > ``` > Exception in thread "main" java.net.SocketException: Socket closed >at java.net.PlainSocketImpl.socketAccept(Native Method) >at java.net.Abstract

[GitHub] [hadoop-ozone] linyiqun commented on pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-10-01 Thread GitBox
linyiqun commented on pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#issuecomment-702553239 @prashantpogde , the CI failure still happened, due to below socket closed error. Seems not related. ``` Exception in thread "main" java.net.SocketException: Socket c

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-10-01 Thread GitBox
linyiqun commented on a change in pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#discussion_r498636258 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java ## @@ -240,8 +247,19 @@ public VersionRespon

[GitHub] [hadoop-ozone] linyiqun commented on pull request #1448: HDDS-4280. Document notable configurations for Recon.

2020-10-01 Thread GitBox
linyiqun commented on pull request #1448: URL: https://github.com/apache/hadoop-ozone/pull/1448#issuecomment-702548266 >Can we add a follow up JIRA to fix this inconsistent config key (recon.om.snapshot.task.interval.delay) ? Hi @avijayanhwx , I also find there is more other ozone re

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1448: HDDS-4280. Document notable configurations for Recon.

2020-10-01 Thread GitBox
adoroszlai commented on pull request #1448: URL: https://github.com/apache/hadoop-ozone/pull/1448#issuecomment-702526133 > Can we add a follow up JIRA to fix this inconsistent config key (recon.om.snapshot.task.interval.delay) ? @avijayanhwx if you mean it should be prefixed with `oz

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-10-01 Thread GitBox
prashantpogde commented on a change in pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#discussion_r498600324 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java ## @@ -240,8 +247,19 @@ public VersionR

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-10-01 Thread GitBox
prashantpogde commented on a change in pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#discussion_r498600362 ## File path: hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java ## @@ -170,6 +176,33 @@ public void

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-10-01 Thread GitBox
prashantpogde commented on a change in pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#discussion_r498600324 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java ## @@ -240,8 +247,19 @@ public VersionR

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-10-01 Thread GitBox
prashantpogde commented on a change in pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#discussion_r498600060 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java ## @@ -240,8 +247,19 @@ public VersionR

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-10-01 Thread GitBox
prashantpogde commented on a change in pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#discussion_r498599539 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/StorageContainerNodeProtocol.java ## @@ -58,7 +60,8

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-10-01 Thread GitBox
prashantpogde commented on a change in pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#discussion_r498599539 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/StorageContainerNodeProtocol.java ## @@ -58,7 +60,8

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #1441: HDDS-3966. Enable TestOMRatisSnapshots.

2020-10-01 Thread GitBox
bharatviswa504 merged pull request #1441: URL: https://github.com/apache/hadoop-ozone/pull/1441 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1441: HDDS-3966. Enable TestOMRatisSnapshots.

2020-10-01 Thread GitBox
bharatviswa504 commented on pull request #1441: URL: https://github.com/apache/hadoop-ozone/pull/1441#issuecomment-702469785 Thank You @aryangupta1998 for the contribution and @hanishakoneru for the info. This is an automate

[GitHub] [hadoop-ozone] umamaheswararao merged pull request #1469: HDDS-4302 : Shade the org.apache.common.lang3 package as this is coming from other hadoop packages as well.

2020-10-01 Thread GitBox
umamaheswararao merged pull request #1469: URL: https://github.com/apache/hadoop-ozone/pull/1469 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [hadoop-ozone] umamaheswararao commented on pull request #1469: HDDS-4302 : Shade the org.apache.common.lang3 package as this is coming from other hadoop packages as well.

2020-10-01 Thread GitBox
umamaheswararao commented on pull request #1469: URL: https://github.com/apache/hadoop-ozone/pull/1469#issuecomment-702468583 Thanks @elek for help on reproducing the issue locally on duplicate class verification test. Thanks @bharatviswa504 for the review!

[GitHub] [hadoop-ozone] umamaheswararao opened a new pull request #1469: HDDS-4302 : Shade the org.apache.common.lang3 package as this is coming from other hadoop packages as well.

2020-10-01 Thread GitBox
umamaheswararao opened a new pull request #1469: URL: https://github.com/apache/hadoop-ozone/pull/1469 ## What changes were proposed in this pull request? Shading the org.apache.hadoop.lang3 package. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HD

[GitHub] [hadoop-ozone] vivekratnavel commented on pull request #1448: HDDS-4280. Document notable configurations for Recon.

2020-10-01 Thread GitBox
vivekratnavel commented on pull request #1448: URL: https://github.com/apache/hadoop-ozone/pull/1448#issuecomment-702426915 @adoroszlai Thanks for working on this! This is an automated message from the Apache Git Service. To

[GitHub] [hadoop-ozone] xiaoyuyao opened a new pull request #1468: HDDS-4301. SCM CA certificate does not encode KeyUsage extension prop…

2020-10-01 Thread GitBox
xiaoyuyao opened a new pull request #1468: URL: https://github.com/apache/hadoop-ozone/pull/1468 …ertly. ## What changes were proposed in this pull request? Fix the encoding of KeyUsage in the CSR to ensure it can be used in FIPS environment. ## What is the link to the

[GitHub] [hadoop-ozone] hanishakoneru merged pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

2020-10-01 Thread GitBox
hanishakoneru merged pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-10-01 Thread GitBox
bharatviswa504 edited a comment on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-702342888 +1 LGTM. Thank You @rakeshadr for multiple updates and @linyiqun for the review. This

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-10-01 Thread GitBox
bharatviswa504 commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r498462050 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0 +1,

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-10-01 Thread GitBox
bharatviswa504 merged pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-10-01 Thread GitBox
bharatviswa504 commented on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-702342888 +1 LGTM. Thank You @rakeshadr for multiple updates. This is an automated message from the Apa

[GitHub] [hadoop-ozone] avijayanhwx opened a new pull request #1467: HDDS-4268. Prepare for Upgrade step should purge the log after waiting for the last txn to be applied.

2020-10-01 Thread GitBox
avijayanhwx opened a new pull request #1467: URL: https://github.com/apache/hadoop-ozone/pull/1467 ## What changes were proposed in this pull request? This is a follow up task from HDDS-4227 in which the prepare upgrade/downgrade task should purge the Raft log immediately after waiting f

[GitHub] [hadoop-ozone] hanishakoneru commented on pull request #1441: HDDS-3966. Enable TestOMRatisSnapshots.

2020-10-01 Thread GitBox
hanishakoneru commented on pull request #1441: URL: https://github.com/apache/hadoop-ozone/pull/1441#issuecomment-702225601 There were some changes made as part of HDDS-3741. That could have possibly resolved this. This is a

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1464: HDDS-4299. Display Ratis version with ozone version

2020-10-01 Thread GitBox
adoroszlai commented on a change in pull request #1464: URL: https://github.com/apache/hadoop-ozone/pull/1464#discussion_r498323795 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/RatisVersionInfo.java ## @@ -0,0 +1,61 @@ +/* + * Licensed to the Apa

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1456: HDDS-4172. Implement Finalize command in Ozone Manager server.

2020-10-01 Thread GitBox
linyiqun commented on a change in pull request #1456: URL: https://github.com/apache/hadoop-ozone/pull/1456#discussion_r498312712 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMUpgradeFinalizer.java ## @@ -0,0 +1,303 @@ +/** + * Lic

[GitHub] [hadoop-ozone] captainzmc edited a comment on pull request #1447: HDDS-4264. Uniform naming conventions of Ozone Shell Options.

2020-10-01 Thread GitBox
captainzmc edited a comment on pull request #1447: URL: https://github.com/apache/hadoop-ozone/pull/1447#issuecomment-702161601 Thanks for @adoroszlai 's reminding. I have solved the conflict. This PR can continue to be reviewed. ---

[GitHub] [hadoop-ozone] captainzmc commented on pull request #1447: HDDS-4264. Uniform naming conventions of Ozone Shell Options.

2020-10-01 Thread GitBox
captainzmc commented on pull request #1447: URL: https://github.com/apache/hadoop-ozone/pull/1447#issuecomment-702161601 > Thanks for working on this @captainzmc. Can you please resolve conflict with latest master when you get a chance? Thanks for @adoroszlai 's reminding. I have solved

[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #1466: HDDS-4271. Avoid logging chunk content in Ozone Insight

2020-10-01 Thread GitBox
adoroszlai opened a new pull request #1466: URL: https://github.com/apache/hadoop-ozone/pull/1466 ## What changes were proposed in this pull request? HDDS-2660 added an insight point for the datanode dispatcher. At trace level it logs all chunk content, which can be huge and contain

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1447: HDDS-4264. Uniform naming conventions of Ozone Shell Options.

2020-10-01 Thread GitBox
adoroszlai commented on pull request #1447: URL: https://github.com/apache/hadoop-ozone/pull/1447#issuecomment-702079340 Thanks for working on this @captainzmc. Can you please resolve conflict with latest master when you get a chance? -

[GitHub] [hadoop-ozone] sodonnel opened a new pull request #1465: Removed unneeded class DatanodeAdminNodeDetails

2020-10-01 Thread GitBox
sodonnel opened a new pull request #1465: URL: https://github.com/apache/hadoop-ozone/pull/1465 ## What changes were proposed in this pull request? DatanodeAdminNodeDetails was added earlier in the decommission branch, to track metrics and, the decommission state and maintenance end

[GitHub] [hadoop-ozone] elek opened a new pull request #1464: HDDS-4299. Display Ratis version with ozone version

2020-10-01 Thread GitBox
elek opened a new pull request #1464: URL: https://github.com/apache/hadoop-ozone/pull/1464 ## What changes were proposed in this pull request? During the development Ozone uses snapshot releases from Ratis. It can be useful to print out the exact version of the used Ratis as part of

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1456: HDDS-4172. Implement Finalize command in Ozone Manager server.

2020-10-01 Thread GitBox
linyiqun commented on a change in pull request #1456: URL: https://github.com/apache/hadoop-ozone/pull/1456#discussion_r498051832 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMUpgradeFinalizer.java ## @@ -0,0 +1,303 @@ +/** + * Lic

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1456: HDDS-4172. Implement Finalize command in Ozone Manager server.

2020-10-01 Thread GitBox
linyiqun commented on a change in pull request #1456: URL: https://github.com/apache/hadoop-ozone/pull/1456#discussion_r498056289 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMUpgradeFinalizer.java ## @@ -0,0 +1,303 @@ +/** + * Lic

[GitHub] [hadoop-ozone] elek commented on pull request #1461: Merge Master into HDDS-1880-Decom

2020-10-01 Thread GitBox
elek commented on pull request #1461: URL: https://github.com/apache/hadoop-ozone/pull/1461#issuecomment-701947184 /pending This merge should be committed manually instead of using the "squash and merge" button. This is an a

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-30 Thread GitBox
bharatviswa504 commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r497970366 ## File path: hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMDirectoryCreateRequestV1.java ## @@ -0,0

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-30 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r497961956 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0 +1,321 @

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-30 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r497962566 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0 +1,321 @

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-30 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r497961956 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0 +1,321 @

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-30 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r497958404 ## File path: hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMDirectoryCreateRequestV1.java ## @@ -0,0 +1,5

[GitHub] [hadoop-ozone] github-actions[bot] commented on pull request #1033: HDDS-3667. If we gracefully stop datanode it would be better to notify scm and r…

2020-09-30 Thread GitBox
github-actions[bot] commented on pull request #1033: URL: https://github.com/apache/hadoop-ozone/pull/1033#issuecomment-701719760 Thank you very much for the patch. I am closing this PR __temporarily__ as there was no activity recently and it is waiting for response from its author.

[GitHub] [hadoop-ozone] github-actions[bot] closed pull request #1033: HDDS-3667. If we gracefully stop datanode it would be better to notify scm and r…

2020-09-30 Thread GitBox
github-actions[bot] closed pull request #1033: URL: https://github.com/apache/hadoop-ozone/pull/1033 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1462: HDDS-4251. Update Ratis version to latest snapshot

2020-09-30 Thread GitBox
bharatviswa504 commented on pull request #1462: URL: https://github.com/apache/hadoop-ozone/pull/1462#issuecomment-701702951 Thank You @hanishakoneru for the contribution and @amaliujia for the review. This is an automated me

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #1462: HDDS-4251. Update Ratis version to latest snapshot

2020-09-30 Thread GitBox
bharatviswa504 merged pull request #1462: URL: https://github.com/apache/hadoop-ozone/pull/1462 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1463: HDDS-4292. Ozone Client not working with Hadoop Version < 3.2

2020-09-30 Thread GitBox
bharatviswa504 commented on pull request #1463: URL: https://github.com/apache/hadoop-ozone/pull/1463#issuecomment-701702259 Thank You @vivekratnavel for the review. This is an automated message from the Apache Git Service. T

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #1463: HDDS-4292. Ozone Client not working with Hadoop Version < 3.2

2020-09-30 Thread GitBox
bharatviswa504 merged pull request #1463: URL: https://github.com/apache/hadoop-ozone/pull/1463 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] vivekratnavel commented on a change in pull request #1463: HDDS-4292. Ozone Client not working with Hadoop Version < 3.2

2020-09-30 Thread GitBox
vivekratnavel commented on a change in pull request #1463: URL: https://github.com/apache/hadoop-ozone/pull/1463#discussion_r497857806 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/ha/OMFailoverProxyProvider.java ## @@ -141,15 +141,14 @@ protected

[GitHub] [hadoop-ozone] vivekratnavel commented on pull request #1463: HDDS-4292. Ozone Client not working with Hadoop Version < 3.2

2020-09-30 Thread GitBox
vivekratnavel commented on pull request #1463: URL: https://github.com/apache/hadoop-ozone/pull/1463#issuecomment-701701432 +1 LGTM This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-30 Thread GitBox
bharatviswa504 commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r497840299 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0 +1,

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1463: HDDS-4292. Ozone Client not working with Hadoop Version < 3.2

2020-09-30 Thread GitBox
bharatviswa504 commented on a change in pull request #1463: URL: https://github.com/apache/hadoop-ozone/pull/1463#discussion_r497854422 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/ha/OMFailoverProxyProvider.java ## @@ -141,15 +141,14 @@ protected

[GitHub] [hadoop-ozone] vivekratnavel commented on a change in pull request #1463: HDDS-4292. Ozone Client not working with Hadoop Version < 3.2

2020-09-30 Thread GitBox
vivekratnavel commented on a change in pull request #1463: URL: https://github.com/apache/hadoop-ozone/pull/1463#discussion_r497852683 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/ha/OMFailoverProxyProvider.java ## @@ -141,15 +141,14 @@ protected

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-09-30 Thread GitBox
avijayanhwx commented on a change in pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#discussion_r497840950 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java ## @@ -1149,4 +1155,8 @@ publi

[GitHub] [hadoop-ozone] amaliujia edited a comment on pull request #1462: HDDS-4251. Update Ratis version to latest snapshot

2020-09-30 Thread GitBox
amaliujia edited a comment on pull request #1462: URL: https://github.com/apache/hadoop-ozone/pull/1462#issuecomment-701647959 +1 and BTW, the exceptions changes in this PR is from https://github.com/apache/incubator-ratis/pull/203, in case anyone has an interest on the context.

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1462: HDDS-4251. Update Ratis version to latest snapshot

2020-09-30 Thread GitBox
amaliujia commented on pull request #1462: URL: https://github.com/apache/hadoop-ozone/pull/1462#issuecomment-701647959 +1 and BTW, the exceptions changes in this PR is from https://github.com/apache/incubator-ratis/pull/203. --

[GitHub] [hadoop-ozone] bharatviswa504 opened a new pull request #1463: HDDS-4292. Ozone Client not working with Hadoop Version < 3.2

2020-09-30 Thread GitBox
bharatviswa504 opened a new pull request #1463: URL: https://github.com/apache/hadoop-ozone/pull/1463 ## What changes were proposed in this pull request? The ozone client is not working with Hadoop Version < 3.2. The issue is due to not properly initialization of the proxy object.

[GitHub] [hadoop-ozone] hanishakoneru opened a new pull request #1462: HDDS-4251. Update Ratis version to latest snapshot

2020-09-30 Thread GitBox
hanishakoneru opened a new pull request #1462: URL: https://github.com/apache/hadoop-ozone/pull/1462 ## What changes were proposed in this pull request? This Jira aims to update ozone with latest Ratis snapshot which has a critical fix for OM HA - RATIS-1025. ## What is the li

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

2020-09-30 Thread GitBox
adoroszlai commented on pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#issuecomment-701520846 > Should we just pass down the token from the caller directly and bypass the UGI addToken/getTokens completely? > We can directly encode the pass in token for the spec

[GitHub] [hadoop-ozone] xiaoyuyao merged pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-30 Thread GitBox
xiaoyuyao merged pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] xiaoyuyao commented on pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-30 Thread GitBox
xiaoyuyao commented on pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#issuecomment-701500271 Thanks @timmylicheng for the update. LGTM, +1. I will merge it shortly. This is an automated message fr

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

2020-09-30 Thread GitBox
xiaoyuyao commented on a change in pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#discussion_r497641447 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/ContainerProtocolCalls.java ## @@ -84,7 +86,8 @@ private Contain

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

2020-09-30 Thread GitBox
xiaoyuyao commented on a change in pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#discussion_r497636613 ## File path: hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockInputStream.java ## @@ -193,10 +196,11 @@ public synchroniz

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-09-30 Thread GitBox
linyiqun commented on a change in pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#discussion_r497571389 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java ## @@ -240,8 +247,19 @@ public VersionRespon

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-09-30 Thread GitBox
linyiqun commented on a change in pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457#discussion_r497566797 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/StorageContainerNodeProtocol.java ## @@ -58,7 +60,8 @@

[GitHub] [hadoop-ozone] sodonnel opened a new pull request #1461: Merge Master into HDDS-1880-Decom

2020-09-30 Thread GitBox
sodonnel opened a new pull request #1461: URL: https://github.com/apache/hadoop-ozone/pull/1461 ## What changes were proposed in this pull request? This PR is to run the Github CI workflow on the HDDS-1880-Decom branch to validate it passes the checks after merging the current master

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1456: HDDS-4172. Implement Finalize command in Ozone Manager server.

2020-09-30 Thread GitBox
avijayanhwx commented on a change in pull request #1456: URL: https://github.com/apache/hadoop-ozone/pull/1456#discussion_r497516613 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMUpgradeFinalizer.java ## @@ -0,0 +1,303 @@ +/** + *

[GitHub] [hadoop-ozone] linyiqun edited a comment on pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

2020-09-30 Thread GitBox
linyiqun edited a comment on pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#issuecomment-701381759 >since it's a user token, suppose it has a much longer liveness than input and output stream object instance, right? A good point that @ChenSammi mentioned, t

[GitHub] [hadoop-ozone] linyiqun edited a comment on pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

2020-09-30 Thread GitBox
linyiqun edited a comment on pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#issuecomment-701381759 >since it's a user token, suppose it has a much longer liveness than input and output stream object instance, right? A good point that @ChenSammi mentioned, t

[GitHub] [hadoop-ozone] linyiqun commented on pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

2020-09-30 Thread GitBox
linyiqun commented on pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#issuecomment-701381759 >since it's a user token, suppose it has a much longer liveness than input and output stream object instance, right? A good point that @ChenSammi mentioned, there is a s

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1435: HDDS-4122. Implement OM Delete Expired Open Key Request and Response

2020-09-30 Thread GitBox
avijayanhwx commented on a change in pull request #1435: URL: https://github.com/apache/hadoop-ozone/pull/1435#discussion_r497488345 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRequest.java ## @@ -553,17 +554,43 @@ protect

[GitHub] [hadoop-ozone] elek opened a new pull request #1460: HDDS-4298. Use an interface in Ozone client instead of XceiverClientManager

2020-09-30 Thread GitBox
elek opened a new pull request #1460: URL: https://github.com/apache/hadoop-ozone/pull/1460 …anager ## What changes were proposed in this pull request? `XceiverClientManager` is used everywhere in the ozone client (Key/Block Input/OutputStream) to get a client when required.

[GitHub] [hadoop-ozone] bshashikant commented on pull request #1440: HDDS-2011. Enable TestRandomKeyGenerator Test Cases.

2020-09-30 Thread GitBox
bshashikant commented on pull request #1440: URL: https://github.com/apache/hadoop-ozone/pull/1440#issuecomment-701318613 > Thanks @aryangupta1998 for working on this. I'd vote for removing this integration test instead of enabling it. It adds 4-5 minutes of run time for no clear benefit.

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

2020-09-30 Thread GitBox
adoroszlai commented on pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#issuecomment-701266985 > thanks for identify this performance issue Credit goes to @elek who reported the problem, see HDDS-4285. > I'm not quite familar with tokens Neither

[GitHub] [hadoop-ozone] GlenGeng commented on a change in pull request #1274: HDDS-3810. Add the logic to distribute open containers among the pipelines of a datanode.

2020-09-30 Thread GitBox
GlenGeng commented on a change in pull request #1274: URL: https://github.com/apache/hadoop-ozone/pull/1274#discussion_r497350686 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java ## @@ -107,6 +107,14 @@ public SCM

[GitHub] [hadoop-ozone] bshashikant commented on a change in pull request #1274: HDDS-3810. Add the logic to distribute open containers among the pipelines of a datanode.

2020-09-30 Thread GitBox
bshashikant commented on a change in pull request #1274: URL: https://github.com/apache/hadoop-ozone/pull/1274#discussion_r497311610 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java ## @@ -101,12 +103,23 @@ public

[GitHub] [hadoop-ozone] lokeshj1703 opened a new pull request #1459: HDDS-4297. Allow multiple transactions per container to be sent for deletion by SCM.

2020-09-30 Thread GitBox
lokeshj1703 opened a new pull request #1459: URL: https://github.com/apache/hadoop-ozone/pull/1459 ## What changes were proposed in this pull request? Currently SCM Block Deleting Service allows only one transaction per container to be sent for deletion to the datanode. This can slow

[GitHub] [hadoop-ozone] captainzmc commented on pull request #1458: HDDS-3728. Bucket space: check quotaUsageInBytes when write key and allocate block.

2020-09-30 Thread GitBox
captainzmc commented on pull request #1458: URL: https://github.com/apache/hadoop-ozone/pull/1458#issuecomment-701207765 Hi @ChenSammi , this PR add judge whether we can be write if bucket space quota enable. Can you help to review this? ---

[GitHub] [hadoop-ozone] captainzmc opened a new pull request #1458: HDDS-3728. Bucket space: check quotaUsageInBytes when write key and allocate block.

2020-09-30 Thread GitBox
captainzmc opened a new pull request #1458: URL: https://github.com/apache/hadoop-ozone/pull/1458 ## What changes were proposed in this pull request? In addition, the current Quota setting does not take effect. HDDS-541 gives all the work needed to perfect Quota. This PR is a subt

[GitHub] [hadoop-ozone] prashantpogde opened a new pull request #1457: HDDS-4253. Add LayoutVersion request/response for DN registration.

2020-09-29 Thread GitBox
prashantpogde opened a new pull request #1457: URL: https://github.com/apache/hadoop-ozone/pull/1457 ## What changes were proposed in this pull request? Add LayoutVersion request/response for DN registration. ## What is the link to the Apache JIRA https://issues.apache.o

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1399: HDDS-3684. Add tests for replication annotation

2020-09-29 Thread GitBox
amaliujia commented on pull request #1399: URL: https://github.com/apache/hadoop-ozone/pull/1399#issuecomment-701150039 @timmylicheng conflicts resolved. This is an automated message from the Apache Git Service. To r

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

2020-09-29 Thread GitBox
ChenSammi commented on pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#issuecomment-701144316 @adoroszlai thanks for identify this performance issue. I'm not quite familar with tokens, since it's a user token, suppose it has a much longer liveness than input and

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

2020-09-29 Thread GitBox
ChenSammi commented on pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#issuecomment-701139655 Thanks @captainzmc for the contribution and @cxorm @adoroszlai @maobaolong for the review. This is an a

[GitHub] [hadoop-ozone] ChenSammi merged pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

2020-09-29 Thread GitBox
ChenSammi merged pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] amaliujia edited a comment on pull request #1444: HDDS-4242. Copy PrefixInfo proto to new project hadoop-ozone/interface-storage

2020-09-29 Thread GitBox
amaliujia edited a comment on pull request #1444: URL: https://github.com/apache/hadoop-ozone/pull/1444#issuecomment-700874303 @elek your suggestion make senses as the new util classes are dedicated to be used by `interface-storage`. If it is even dedicated to be used by a `codec` class, i

[GitHub] [hadoop-ozone] fapifta opened a new pull request #1456: Upgrade

2020-09-29 Thread GitBox
fapifta opened a new pull request #1456: URL: https://github.com/apache/hadoop-ozone/pull/1456 ## What changes were proposed in this pull request? Server side implementation of the finalization logic is implemented in this pull request. The initial idea in the client side implemen

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

2020-09-29 Thread GitBox
linyiqun commented on a change in pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#discussion_r497199494 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/ContainerProtocolCalls.java ## @@ -84,7 +86,8 @@ private Containe

[GitHub] [hadoop-ozone] umamaheswararao merged pull request #1455: HDDS-4287: Exclude protobuff classes from ozone-filesystem-hadoop3 jars

2020-09-29 Thread GitBox
umamaheswararao merged pull request #1455: URL: https://github.com/apache/hadoop-ozone/pull/1455 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1455: HDDS-4287: Exclude protobuff classes from ozone-filesystem-hadoop3 jars

2020-09-29 Thread GitBox
bharatviswa504 commented on pull request #1455: URL: https://github.com/apache/hadoop-ozone/pull/1455#issuecomment-701064102 +1 LGTM This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop-ozone] umamaheswararao opened a new pull request #1455: HDDS-4287: Exclude protobuff classes from ozone-filesystem-hadoop3 jars

2020-09-29 Thread GitBox
umamaheswararao opened a new pull request #1455: URL: https://github.com/apache/hadoop-ozone/pull/1455 ## What changes were proposed in this pull request? Excluded the protobuf classes from hadoop-ozone-filesystem-hadoop3/2. We already kept the dependency on hadoop anyway and hadoop

[GitHub] [hadoop-ozone] bharatviswa504 closed pull request #1331: HDDS-4117. Normalize Keypath for listKeys.

2020-09-29 Thread GitBox
bharatviswa504 closed pull request #1331: URL: https://github.com/apache/hadoop-ozone/pull/1331 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1331: HDDS-4117. Normalize Keypath for listKeys.

2020-09-29 Thread GitBox
bharatviswa504 commented on pull request #1331: URL: https://github.com/apache/hadoop-ozone/pull/1331#issuecomment-700952276 Closing this. Opened #1451 to track this issue. This is an automated message from the Apache Git Ser

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1451: HDDS-4117. Normalize Keypath for listKeys.

2020-09-29 Thread GitBox
bharatviswa504 commented on pull request #1451: URL: https://github.com/apache/hadoop-ozone/pull/1451#issuecomment-700951976 Rebased and fixed test. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1453: HDDS-4290. Enable insight point for SCM heartbeat protocol

2020-09-29 Thread GitBox
amaliujia commented on pull request #1453: URL: https://github.com/apache/hadoop-ozone/pull/1453#issuecomment-700941359 +1! This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] vivekratnavel merged pull request #1452: HDDS-4288. the icon of hadoop-ozone is bigger than ever

2020-09-29 Thread GitBox
vivekratnavel merged pull request #1452: URL: https://github.com/apache/hadoop-ozone/pull/1452 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

<    2   3   4   5   6   7   8   9   10   11   >