[jira] [Commented] (HDFS-12763) DataStreamer should heartbeat during flush

2017-11-29 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271159#comment-16271159 ] Kuhu Shukla commented on HDFS-12763: Would be nice to get some preliminary comments from the community

[jira] [Commented] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks

2017-11-28 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269265#comment-16269265 ] Kuhu Shukla commented on HDFS-11246: Test failures are unrelated. [~daryn], request for

[jira] [Updated] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks

2017-11-28 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11246: --- Attachment: HDFS-11246.005.patch Updated patch after talking to Daryn offline about what the locking

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-27 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Attachment: HDFS-12754-branch-2.patch Adding branch-2 version of the patch with a minor fix to

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-22 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Attachment: HDFS-12754.009.patch Thank you [~xiaochen], I missed putting up the version which had the

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-20 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Attachment: HDFS-12754.008.patch Attaching patch that improves the test for better coordination

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-15 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254237#comment-16254237 ] Kuhu Shukla commented on HDFS-12754: Thank you for the review [~xiaochen]. I will address the test

[jira] [Commented] (HDFS-11317) Modify DataNodeDescriptor() to not call UpdateHeartBeatState

2017-11-14 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251755#comment-16251755 ] Kuhu Shukla commented on HDFS-11317: [~kihwal], request for review. Thanks a lot! > Modify

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-14 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251748#comment-16251748 ] Kuhu Shukla commented on HDFS-12754: Request for review since test failures are unrelated. > Lease

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-10 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Attachment: HDFS-12754.007.patch Apologies the v6 patch had additional changes that are not relevant

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-10 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247594#comment-16247594 ] Kuhu Shukla commented on HDFS-12754: [~kihwal], [~yangjiandan] could you help review. Thank you! >

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-09 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Attachment: HDFS-12754.006.patch Fixing minor checkstyle issues except the one where it asks to make

[jira] [Commented] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks

2017-11-09 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246067#comment-16246067 ] Kuhu Shukla commented on HDFS-11246: Most test failures are due to OOM causing no new thread creations

[jira] [Commented] (HDFS-12763) DataStreamer should heartbeat during flush

2017-11-09 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246008#comment-16246008 ] Kuhu Shukla commented on HDFS-12763: Test failures are unrelated. [~kihwal], request for

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-09 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Attachment: HDFS-12754.005.patch Uploading the same patch again since the failures are not

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-08 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Attachment: HDFS-12754.004.patch v4 patch that makes a change to how the test sets the grace period.

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-08 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Attachment: HDFS-12754.003.patch v3 patch that fixes the test failures in testDistributedFileSystem

[jira] [Commented] (HDFS-12771) Add genstamp and block size to metasave Corrupt blocks list

2017-11-03 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16238268#comment-16238268 ] Kuhu Shukla commented on HDFS-12771: [~kihwal], could you share any review comments for this patch.

[jira] [Updated] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks

2017-11-03 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11246: --- Attachment: HDFS-11246.004.patch Revised patch addressing comments from Daryn. >

[jira] [Updated] (HDFS-12771) Add genstamp and block size to metasave Corrupt blocks list

2017-11-03 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12771: --- Attachment: HDFS-12771.001.patch v1 patch that uses block.toString() and getNumBytes() . > Add

[jira] [Updated] (HDFS-12771) Add genstamp and block size to metasave Corrupt blocks list

2017-11-03 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12771: --- Status: Patch Available (was: Open) > Add genstamp and block size to metasave Corrupt blocks list >

[jira] [Created] (HDFS-12771) Add genstamp and block size to metasave Corrupt blocks list

2017-11-03 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-12771: -- Summary: Add genstamp and block size to metasave Corrupt blocks list Key: HDFS-12771 URL: https://issues.apache.org/jira/browse/HDFS-12771 Project: Hadoop HDFS

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-03 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Attachment: HDFS-12754.002.patch Thank you for the comments [~yangjiandan]. I do an additional remove

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-02 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16236680#comment-16236680 ] Kuhu Shukla commented on HDFS-12754: Adding the test as it is causes a hang since the deadlock of the

[jira] [Updated] (HDFS-12763) DataStreamer should heartbeat during flush

2017-11-02 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12763: --- Attachment: HDFS-12763.001.patch v1 patch with a test that checks if heartbeats get sent during

[jira] [Updated] (HDFS-12763) DataStreamer should heartbeat during flush

2017-11-02 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12763: --- Status: Patch Available (was: Open) > DataStreamer should heartbeat during flush >

[jira] [Created] (HDFS-12763) DataStreamer should heartbeat during flush

2017-11-02 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-12763: -- Summary: DataStreamer should heartbeat during flush Key: HDFS-12763 URL: https://issues.apache.org/jira/browse/HDFS-12763 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-02 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16236308#comment-16236308 ] Kuhu Shukla commented on HDFS-12754: bq. I would add it here. Agree. CC: [~yangjiandan], adding the UT

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-01 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234905#comment-16234905 ] Kuhu Shukla commented on HDFS-12754: This deadlock was found during testing on our end about an year

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-01 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234646#comment-16234646 ] Kuhu Shukla commented on HDFS-12754: CC: [~kihwal]. > Lease renewal can hit a deadlock >

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-01 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Attachment: HDFS-12754.001.patch This patch calls removal only when necessary in endFileLease(). >

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-01 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-12754: --- Status: Patch Available (was: Open) > Lease renewal can hit a deadlock >

[jira] [Created] (HDFS-12754) Lease renewal can hit a deadlock

2017-11-01 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-12754: -- Summary: Lease renewal can hit a deadlock Key: HDFS-12754 URL: https://issues.apache.org/jira/browse/HDFS-12754 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-12674) hadoop fs -get does not throw permission denied in some cases

2017-10-17 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207740#comment-16207740 ] Kuhu Shukla commented on HDFS-12674: Tried the following on my MAC: {code} @Test (timeout = 1L)

[jira] [Created] (HDFS-12674) hadoop fs -get does not throw permission denied in some cases

2017-10-17 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-12674: -- Summary: hadoop fs -get does not throw permission denied in some cases Key: HDFS-12674 URL: https://issues.apache.org/jira/browse/HDFS-12674 Project: Hadoop HDFS

[jira] [Updated] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks

2017-10-16 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11246: --- Attachment: HDFS-11246.003.patch Rebased patch. > FSNameSystem#logAuditEvent should be called

[jira] [Comment Edited] (HDFS-12036) Add audit log for some erasure coding operations

2017-10-16 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206071#comment-16206071 ] Kuhu Shukla edited comment on HDFS-12036 at 10/16/17 3:58 PM: --

[jira] [Commented] (HDFS-12036) Add audit log for some erasure coding operations

2017-10-16 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206071#comment-16206071 ] Kuhu Shukla commented on HDFS-12036: [~HuafengWang], [~drankye], was the intent of this patch to log

[jira] [Commented] (HDFS-10743) MiniDFSCluster test runtimes can be drastically reduce

2017-10-11 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201188#comment-16201188 ] Kuhu Shukla commented on HDFS-10743: Apologies for missing your comment [~asuresh]. Thanks [~subru],

[jira] [Updated] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-09-22 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-5040: -- Attachment: HDFS-5040.009.patch Added audit logging to getDatanodeStorageReport() and added a test as

[jira] [Commented] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-09-13 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164707#comment-16164707 ] Kuhu Shukla commented on HDFS-5040: --- Test failures are related to bind exceptions and tests like

[jira] [Updated] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-09-12 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-5040: -- Attachment: HDFS-5040.008.patch Apologies for the delay [~brahmareddy], I somehow missed the last ping.

[jira] [Commented] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-08-14 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126420#comment-16126420 ] Kuhu Shukla commented on HDFS-5040: --- [~brahmareddy], thank you for the reviews. Request for one more

[jira] [Updated] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-08-10 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-5040: -- Attachment: HDFS-5040.007.patch Fixing checkstyle nits. Test failures are not related. Same goes for

[jira] [Updated] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-08-09 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-5040: -- Attachment: HDFS-5040.006.patch Fixing minor checkstyle warnings. Findbug warnings are unrelated and so

[jira] [Updated] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-08-09 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-5040: -- Attachment: HDFS-5040.005.patch bq. Add fail assertion for exception cases. you can use

[jira] [Updated] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-08-01 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-5040: -- Attachment: HDFS-5040.004.patch Added tests and incorporated the review changes. Attaching revised

[jira] [Commented] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-07-20 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094929#comment-16094929 ] Kuhu Shukla commented on HDFS-5040: --- Thank you [~brahmareddy]. bq. can we cover success cases here..? I

[jira] [Commented] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks

2017-07-20 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094879#comment-16094879 ] Kuhu Shukla commented on HDFS-11246: Thank you [~daryn], [~brahmareddy] for your comments. Request you

[jira] [Commented] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-07-19 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093893#comment-16093893 ] Kuhu Shukla commented on HDFS-5040: --- bq. Also Check for usages of checkSuperuserPrivilege(), wherever

[jira] [Updated] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks

2017-07-19 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11246: --- Summary: FSNameSystem#logAuditEvent should be called outside the read or write locks (was:

[jira] [Commented] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks during operations like getContentSummary

2017-07-19 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093317#comment-16093317 ] Kuhu Shukla commented on HDFS-11246: Thanks [~brahmareddy] for the review! Will update patch with

[jira] [Commented] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-07-19 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093313#comment-16093313 ] Kuhu Shukla commented on HDFS-5040: --- For {{refreshServiceAcl}}, {{refreshUserToGroupsMappings}},

[jira] [Commented] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-07-19 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093102#comment-16093102 ] Kuhu Shukla commented on HDFS-5040: --- Thank you so much [~brahmareddy] for the detailed review/comments.

[jira] [Updated] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-07-18 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-5040: -- Target Version/s: (was: ) Status: Patch Available (was: In Progress) > Audit log for

[jira] [Updated] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-07-18 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-5040: -- Attachment: HDFS-5040.001.patch First cut of the patch that audit logs allowed=false for

[jira] [Comment Edited] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks during operations like getContentSummary

2017-07-18 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091759#comment-16091759 ] Kuhu Shukla edited comment on HDFS-11246 at 7/18/17 4:05 PM: - Fixed checkstyle

[jira] [Updated] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks during operations like getContentSummary

2017-07-18 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11246: --- Attachment: HDFS-11246.002.patch Fixed checkstyle and whitespace issues. Findbugs warnings are

[jira] [Updated] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks during operations like getContentSummary

2017-07-17 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11246: --- Attachment: HDFS-11246.001.patch First cut of the patch which adds a try-finally block to unlock

[jira] [Updated] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks during operations like getContentSummary

2017-07-17 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11246: --- Status: Patch Available (was: Open) > FSNameSystem#logAuditEvent should be called outside the read

[jira] [Commented] (HDFS-12112) TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE

2017-07-11 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082500#comment-16082500 ] Kuhu Shukla commented on HDFS-12112: Thanks [~jojochuang] for the JIRA. The patch looks good to me.

[jira] [Commented] (HDFS-5040) Audit log for admin commands/ logging output of all DFS admin commands

2017-07-10 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080458#comment-16080458 ] Kuhu Shukla commented on HDFS-5040: --- [~brahmareddy], sure. Will update patch in a couple days. Let me

[jira] [Commented] (HDFS-11797) BlockManager#createLocatedBlocks() can throw ArrayIndexOutofBoundsException when corrupt replicas are inconsistent

2017-06-08 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042731#comment-16042731 ] Kuhu Shukla commented on HDFS-11797: The symptoms match exactly. Thanks [~jingzhao]. Will wait for

[jira] [Updated] (HDFS-11797) BlockManager#createLocatedBlocks() can throw ArrayIndexOutofBoundsException when corrupt replicas are inconsistent

2017-06-06 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11797: --- Attachment: HDFS-11797.001.patch For HDFS-9958 case we did manage to find a stack trace after some

[jira] [Commented] (HDFS-11797) BlockManager#createLocatedBlocks() can throw ArrayIndexOutofBoundsException when corrupt replicas are inconsistent

2017-06-06 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039585#comment-16039585 ] Kuhu Shukla commented on HDFS-11797: Thank you [~jnp], the inconsistent data structures are a rare

[jira] [Created] (HDFS-11797) BlockManager#createLocatedBlocks() can throw ArrayIndexOutofBoundsException when corrupt replicas are inconsistent

2017-05-10 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-11797: -- Summary: BlockManager#createLocatedBlocks() can throw ArrayIndexOutofBoundsException when corrupt replicas are inconsistent Key: HDFS-11797 URL:

[jira] [Commented] (HDFS-10788) fsck NullPointerException when it encounters corrupt replicas

2017-04-26 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985482#comment-15985482 ] Kuhu Shukla commented on HDFS-10788: [~jojochuang], Thank you for the the update on this. Please

[jira] [Comment Edited] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider

2017-03-16 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928561#comment-15928561 ] Kuhu Shukla edited comment on HDFS-11431 at 3/16/17 6:22 PM: - mvn install is

[jira] [Commented] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider

2017-03-16 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928561#comment-15928561 ] Kuhu Shukla commented on HDFS-11431: mvn install is breaking for me with the error that duplicate

[jira] [Commented] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2017-03-14 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924650#comment-15924650 ] Kuhu Shukla commented on HDFS-10601: TestHASafeMode passes locally and has an address bind exception.

[jira] [Updated] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2017-03-14 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-10601: --- Attachment: HDFS-10601.004.patch Rebasing with some modifications to address the test failures and

[jira] [Updated] (HDFS-11317) Modify DataNodeDescriptor() to not call UpdateHeartBeatState

2017-01-17 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11317: --- Description: HDFS 2.7+ DataNodeDescriptor constructor calls updateHeartBeat which updates only a

[jira] [Updated] (HDFS-11317) Modify DataNodeDescriptor() to not call UpdateHeartBeatState

2017-01-17 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11317: --- Status: Patch Available (was: Open) > Modify DataNodeDescriptor() to not call UpdateHeartBeatState >

[jira] [Updated] (HDFS-11317) Modify DataNodeDescriptor() to not call UpdateHeartBeatState

2017-01-17 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11317: --- Attachment: HDFS-11317.001.patch This moves the only possible operations that come from

[jira] [Created] (HDFS-11317) Modify DataNodeDescriptor() to not call UpdateHeartBeatState

2017-01-11 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-11317: -- Summary: Modify DataNodeDescriptor() to not call UpdateHeartBeatState Key: HDFS-11317 URL: https://issues.apache.org/jira/browse/HDFS-11317 Project: Hadoop HDFS

[jira] [Commented] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks during operations like getContentSummary

2016-12-15 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15751535#comment-15751535 ] Kuhu Shukla commented on HDFS-11246: bq. audit log can be async That is interesting. Will get back to

[jira] [Commented] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks during operations like getContentSummary

2016-12-15 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15751533#comment-15751533 ] Kuhu Shukla commented on HDFS-11246: Thanks a lot [~linyiqun]. bq. I see we can move

[jira] [Created] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks during operations like getContentSummary

2016-12-14 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-11246: -- Summary: FSNameSystem#logAuditEvent should be called outside the read or write locks during operations like getContentSummary Key: HDFS-11246 URL:

[jira] [Commented] (HDFS-8870) Lease is leaked on write failure

2016-10-27 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15612873#comment-15612873 ] Kuhu Shukla commented on HDFS-8870: --- Test failures are unrelated. TestPermission is broken by

[jira] [Commented] (HDFS-10776) Revisit Hdfs Audit Logging

2016-10-27 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15612803#comment-15612803 ] Kuhu Shukla commented on HDFS-10776: [~aw], Looking forward to the blog post and get things moving on

[jira] [Updated] (HDFS-8870) Lease is leaked on write failure

2016-10-27 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-8870: -- Status: Patch Available (was: Open) > Lease is leaked on write failure >

[jira] [Updated] (HDFS-8870) Lease is leaked on write failure

2016-10-27 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-8870: -- Attachment: HDFS-8870.001.patch Initial patch that moves {{endFileLease}} to {{setClosed()}}. Includes a

[jira] [Commented] (HDFS-11050) Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR

2016-10-25 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606660#comment-15606660 ] Kuhu Shukla commented on HDFS-11050: [~kihwal], request for comments/review.Thanks a lot! > Change

[jira] [Updated] (HDFS-11050) Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR

2016-10-25 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11050: --- Attachment: HDFS-11050.002.patch Fixing test to not rely on stdout. > Change log level to 'warn'

[jira] [Updated] (HDFS-11050) Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR

2016-10-25 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11050: --- Status: Patch Available (was: Open) > Change log level to 'warn' when ssl initialization fails and

[jira] [Updated] (HDFS-11050) Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR

2016-10-25 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11050: --- Attachment: HDFS-11050.001.patch Initial patch with the log level change. > Change log level to

[jira] [Commented] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-25 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15605547#comment-15605547 ] Kuhu Shukla commented on HDFS-11042: Thank you [~kihwal]. Opened HDFS-11050 for : bq. When the SSL

[jira] [Created] (HDFS-11050) Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR

2016-10-25 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-11050: -- Summary: Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR Key: HDFS-11050 URL:

[jira] [Commented] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-21 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596850#comment-15596850 ] Kuhu Shukla commented on HDFS-11042: [~kihwal], Request for comments/review. Thanks a lot! > Add

[jira] [Updated] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-21 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11042: --- Attachment: HDFS-11042.002.patch > Add missing cleanupSSLConfig() call for tests that use

[jira] [Updated] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-21 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11042: --- Component/s: test > Add missing cleanupSSLConfig() call for tests that use setupSSLConfig() >

[jira] [Updated] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-21 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11042: --- Status: Patch Available (was: Open) > Add missing cleanupSSLConfig() call for tests that use

[jira] [Updated] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-21 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11042: --- Attachment: HDFS-11042.001.patch Patch that addresses only the hdfs-project tests that need the fix.

[jira] [Created] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-21 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-11042: -- Summary: Add missing cleanupSSLConfig() call for tests that use setupSSLConfig() Key: HDFS-11042 URL: https://issues.apache.org/jira/browse/HDFS-11042 Project: Hadoop

[jira] [Updated] (HDFS-11019) Inconsistent number of corrupt replicas if a corrupt replica is reported multiple times

2016-10-19 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-11019: --- Attachment: HDFS-11019.test.patch [~jojochuang] Thank you reporting this. In Hadoop 2.6(CDH 5.7.2)

[jira] [Comment Edited] (HDFS-11019) Inconsistent number of corrupt replicas if a corrupt replica is reported multiple times

2016-10-19 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589190#comment-15589190 ] Kuhu Shukla edited comment on HDFS-11019 at 10/19/16 4:36 PM: -- [~jojochuang]

[jira] [Commented] (HDFS-10776) Revisit Hdfs Audit Logging

2016-09-19 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504549#comment-15504549 ] Kuhu Shukla commented on HDFS-10776: [~aw], Thank you! Could you point me to the blog post or share

[jira] [Updated] (HDFS-10842) Prevent IncrementalBlockReport processing before the first FullBlockReport has been received

2016-09-06 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-10842: --- Assignee: Daryn Sharp (was: Kuhu Shukla) > Prevent IncrementalBlockReport processing before the

[jira] [Created] (HDFS-10842) Prevent IncrementalBlockReport processing before the first FullBlockReport has been received

2016-09-06 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-10842: -- Summary: Prevent IncrementalBlockReport processing before the first FullBlockReport has been received Key: HDFS-10842 URL: https://issues.apache.org/jira/browse/HDFS-10842

[jira] [Commented] (HDFS-9395) Make HDFS audit logging consistant

2016-08-30 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449083#comment-15449083 ] Kuhu Shukla commented on HDFS-9395: --- Thank you [~andrew.wang] for updating the release note. I was under

  1   2   3   >