[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550451#comment-15550451
]
Fenghua Hu commented on HDFS-10690:
---
[~xyao], thank you for the great help!
> Optimize
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15541287#comment-15541287
]
Fenghua Hu commented on HDFS-10690:
---
Failed case has nothing to do with the patch, and they also passed
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15541287#comment-15541287
]
Fenghua Hu edited comment on HDFS-10690 at 10/3/16 1:27 AM:
Failed cases have
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15541287#comment-15541287
]
Fenghua Hu edited comment on HDFS-10690 at 10/3/16 1:26 AM:
Failed cases have
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: HDFS-10690.008.patch
> Optimize insertion/removal of replica in ShortCircuitCache.java
>
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: (was: HDFS-10690.008.patch)
> Optimize insertion/removal of replica in
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: HDFS-10690.008.patch
Fixed unit test issue.
> Optimize insertion/removal of replica in
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15537580#comment-15537580
]
Fenghua Hu commented on HDFS-10690:
---
Hi [~xyao], I should had run a clean compilation, sorry for your
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: HDFS-10690.007.patch
> Optimize insertion/removal of replica in ShortCircuitCache.java
>
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533068#comment-15533068
]
Fenghua Hu commented on HDFS-10690:
---
[~stack] thanks for reviewing the patch!
> Optimize
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531057#comment-15531057
]
Fenghua Hu commented on HDFS-10690:
---
[~xyao], thanks for the help!
> Optimize insertion/removal of
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Attachment: (was: HDFS-10804-003.patch)
> Use separate lock for ReplicaMap
>
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Attachment: HDFS-10804-003.patch
Re-attach to trigger Jenkins build.
> Use separate lock for
[
https://issues.apache.org/jira/browse/HDFS-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528731#comment-15528731
]
Fenghua Hu commented on HDFS-9668:
--
HI [~jingcheng...@intel.com], thanks for the great fix. I think this
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528661#comment-15528661
]
Fenghua Hu commented on HDFS-10804:
---
Thanks [~arpitagarwal] for updating the description to reflect the
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Attachment: HDFS-10804-003.patch
> Use separate lock for ReplicaMap
>
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528584#comment-15528584
]
Fenghua Hu commented on HDFS-10690:
---
This fix doesn't change any interface, and I have run the unit test
[
https://issues.apache.org/jira/browse/HDFS-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528580#comment-15528580
]
Fenghua Hu commented on HDFS-9668:
--
hi [~jingcheng...@intel.com], regarding ReplicaMap lock, I opened a
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: HDFS-10690.006.patch
Re-submit patch v6 so as to trigger Jenkins build.
> Optimize
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: (was: HDFS-10690.006.patch)
> Optimize insertion/removal of replica in
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528033#comment-15528033
]
Fenghua Hu commented on HDFS-10690:
---
looks like patch v6 hasn't been built and verified by Jenkins.
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Summary: Use separate lock for ReplicaMap (was: Use finer-granularity lock
for ReplicaMap)
> Use
[
https://issues.apache.org/jira/browse/HDFS-10828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524786#comment-15524786
]
Fenghua Hu commented on HDFS-10828:
---
[~arpitagarwal],
Actually, i opened another JIRA
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524750#comment-15524750
]
Fenghua Hu commented on HDFS-10690:
---
Removed unnessary "if (eldestKey == null)" statement, and updated
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: HDFS-10690.006.patch
> Optimize insertion/removal of replica in ShortCircuitCache.java
>
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524699#comment-15524699
]
Fenghua Hu commented on HDFS-10690:
---
Thanks [~xyao] for your suggestion. How about we just remove "if
[
https://issues.apache.org/jira/browse/HDFS-10828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523110#comment-15523110
]
Fenghua Hu commented on HDFS-10828:
---
[~arpitagarwal],
Currently datasetLock is a big lock, which is
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: HDFS-10690.005.patch
Fixed a few coding style issues.
> Optimize insertion/removal of
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522107#comment-15522107
]
Fenghua Hu commented on HDFS-10690:
---
Existing unit tests have been able to cover all the necessary
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: HDFS-10690.004.patch
Fixed two minor issues.
> Optimize insertion/removal of replica in
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15521943#comment-15521943
]
Fenghua Hu commented on HDFS-10690:
---
hi [~xyao], based on your suggestion, i have updated the patch
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: HDFS-10690.003.patch
> Optimize insertion/removal of replica in ShortCircuitCache.java
>
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484232#comment-15484232
]
Fenghua Hu commented on HDFS-10690:
---
[~xyao],
I just got environment to test your patch. Below is the
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450435#comment-15450435
]
Fenghua Hu commented on HDFS-10804:
---
[~vagarychen], thanks for the suggestion.
My intention is to avoid
[
https://issues.apache.org/jira/browse/HDFS-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446506#comment-15446506
]
Fenghua Hu commented on HDFS-10682:
---
[~vagarychen] Actually, I am not sure why ReplicaMap takes a so big
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15445531#comment-15445531
]
Fenghua Hu commented on HDFS-10804:
---
Failed case:
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Attachment: HDFS-10804-002.patch
> Use finer-granularity lock for ReplicaMap
>
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Attachment: HDFS-10804-001.patch
> Use finer-granularity lock for ReplicaMap
>
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Attachment: (was: HDFS-10804-003.patch)
> Use finer-granularity lock for ReplicaMap
>
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Attachment: (was: HDFS-10804-002.patch)
> Use finer-granularity lock for ReplicaMap
>
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Attachment: HDFS-10804-003.patch
> Use finer-granularity lock for ReplicaMap
>
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Attachment: HDFS-10804-002.patch
> Use finer-granularity lock for ReplicaMap
>
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Fix Version/s: 3.0.0-beta1
Status: Patch Available (was: Open)
> Use finer-granularity lock
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Status: Open (was: Patch Available)
> Use finer-granularity lock for ReplicaMap
>
[
https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10804:
--
Fix Version/s: 3.0.0-beta1
Release Note: Add a private object for synchronization as
[
https://issues.apache.org/jira/browse/HDFS-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444774#comment-15444774
]
Fenghua Hu edited comment on HDFS-10682 at 8/29/16 4:25 AM:
In
[
https://issues.apache.org/jira/browse/HDFS-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444774#comment-15444774
]
Fenghua Hu commented on HDFS-10682:
---
In FsDatasetImpl#FsDatasetImpl() and FsDatasetImpl#addVolume():
Fenghua Hu created HDFS-10804:
-
Summary: Use finer-granularity lock for ReplicaMap
Key: HDFS-10804
URL: https://issues.apache.org/jira/browse/HDFS-10804
Project: Hadoop HDFS
Issue Type:
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434077#comment-15434077
]
Fenghua Hu commented on HDFS-10690:
---
I got you. I'll run YCSB.
Regarding the patch, what do you mean "I
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433918#comment-15433918
]
Fenghua Hu commented on HDFS-10690:
---
[~xyao], thanks for your comments. I'll write a micro benchmark to
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433142#comment-15433142
]
Fenghua Hu edited comment on HDFS-10690 at 8/23/16 4:43 PM:
I would like to
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433142#comment-15433142
]
Fenghua Hu edited comment on HDFS-10690 at 8/23/16 4:41 PM:
I would like to
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433142#comment-15433142
]
Fenghua Hu edited comment on HDFS-10690 at 8/23/16 4:38 PM:
I would like to
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433142#comment-15433142
]
Fenghua Hu edited comment on HDFS-10690 at 8/23/16 4:32 PM:
I would like to
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433142#comment-15433142
]
Fenghua Hu edited comment on HDFS-10690 at 8/23/16 4:30 PM:
I would like to
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433142#comment-15433142
]
Fenghua Hu commented on HDFS-10690:
---
I would like to explain the solution here.
Currently, TreeMap is
[
https://issues.apache.org/jira/browse/HDFS-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427691#comment-15427691
]
Fenghua Hu commented on HDFS-9668:
--
[~jingcheng...@intel.com]，
We reviewed your lock patch, but didn't
[
https://issues.apache.org/jira/browse/HDFS-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427542#comment-15427542
]
Fenghua Hu commented on HDFS-9668:
--
[~jingcheng...@intel.com], Thanks for the great work.
I found you had
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423714#comment-15423714
]
Fenghua Hu commented on HDFS-10690:
---
[~xyao],
Thanks for your support!
I would like to clarify that
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420579#comment-15420579
]
Fenghua Hu edited comment on HDFS-10690 at 8/15/16 3:25 AM:
[~xyao], I have a
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420579#comment-15420579
]
Fenghua Hu commented on HDFS-10690:
---
[~xyao], I have a question about the patch:
According to
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416326#comment-15416326
]
Fenghua Hu commented on HDFS-10690:
---
Xiaoyu,
[~xyao]Thanks for the suggestion. I'll test the patch once
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407012#comment-15407012
]
Fenghua Hu edited comment on HDFS-10690 at 8/10/16 1:30 AM:
Xiaoyu,
[~xyao]I
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407012#comment-15407012
]
Fenghua Hu commented on HDFS-10690:
---
Xiaoyu,
[~xiaoyuyao] I tried to replace TreeMap with
[
https://issues.apache.org/jira/browse/HDFS-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403399#comment-15403399
]
Fenghua Hu commented on HDFS-10682:
---
Arpit/Liang,
Looks like there is one
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400336#comment-15400336
]
Fenghua Hu commented on HDFS-10690:
---
Xiaoyu, thanks for the reply. Regarding the bulletin 2, look like i
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398939#comment-15398939
]
Fenghua Hu edited comment on HDFS-10690 at 7/29/16 8:28 AM:
Performance test
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398939#comment-15398939
]
Fenghua Hu commented on HDFS-10690:
---
Performance test result against hadoop-2.6.4:
Test configuration:
*
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398762#comment-15398762
]
Fenghua Hu commented on HDFS-10690:
---
Patch updated.
> Optimize insertion/removal of replica in
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: HDFS-10690.002.patch
> Optimize insertion/removal of replica in ShortCircuitCache.java
>
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398429#comment-15398429
]
Fenghua Hu commented on HDFS-10690:
---
Xiaoyu, thanks for the suggestion. LinkedHashMap is another good
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: HDFS-10690.001.patch
> Optimize insertion/removal of replica in ShortCircuitCache.java
>
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: (was: lrulist.patch)
> Optimize insertion/removal of replica in ShortCircuitCache.java
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Affects Version/s: (was: 2.6.4)
3.0.0-alpha2
Target Version/s:
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397637#comment-15397637
]
Fenghua Hu commented on HDFS-10690:
---
Some comments about the patch:
1. LruList.java implements a
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Attachment: lrulist.patch
> Optimize insertion/removal of replica in ShortCircuitCache.java
>
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Status: Open (was: Patch Available)
> Optimize insertion/removal of replica in ShortCircuitCache.java
[
https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fenghua Hu updated HDFS-10690:
--
Affects Version/s: 2.6.4
Target Version/s: 3.0.0-alpha2
Tags: ShortCircuitCache
Fenghua Hu created HDFS-10690:
-
Summary: Optimize insertion/removal of replica in
ShortCircuitCache.java
Key: HDFS-10690
URL: https://issues.apache.org/jira/browse/HDFS-10690
Project: Hadoop HDFS
79 matches
Mail list logo
