[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17562354#comment-17562354 ] zhengchenyu commented on HDFS-14703: Is this ongoing? It is a great work indeed. But I doubt that why

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413096#comment-17413096 ] JiangHua Zhu commented on HDFS-14703: - Okay, I will continue to work. > NameNode Fine-Grained

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413087#comment-17413087 ] Renukaprasad C commented on HDFS-14703: --- Thanks [~jianghuazhu] for your interest & attention on

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412908#comment-17412908 ] JiangHua Zhu commented on HDFS-14703: - Thanks [~prasad-acit] for sharing. Yes, I have browsed through

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412801#comment-17412801 ] Renukaprasad C commented on HDFS-14703: --- Thanks [~jianghuazhu] for sharing your thoughts. Hope this

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412536#comment-17412536 ] JiangHua Zhu commented on HDFS-14703: - [~prasad-acit], I have some very curious questions. The first

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410871#comment-17410871 ] JiangHua Zhu commented on HDFS-14703: - Okay, I get it. Thanks [~prasad-acit] for the comment.

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410702#comment-17410702 ] Renukaprasad C commented on HDFS-14703: --- [~jianghuazhu] Initially there are 2 commits done as part

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410606#comment-17410606 ] JiangHua Zhu commented on HDFS-14703: - I noticed that the file PartitionedGSet.java has some

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17390818#comment-17390818 ] Konstantin Shvachko commented on HDFS-14703: Some thoughts on [~daryn]'s comment: * For small

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388217#comment-17388217 ] Renukaprasad C commented on HDFS-14703: --- Thanks [Daryn

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387797#comment-17387797 ] Xing Lin commented on HDFS-14703: - [~daryn] Thanks for your comments. I will address your last question

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387540#comment-17387540 ] Daryn Sharp commented on HDFS-14703: I applaud this effort but I have concerns about real world

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380757#comment-17380757 ] Konstantin Shvachko commented on HDFS-14703: ??Shall I raise separate Jira for Create and

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380424#comment-17380424 ] Renukaprasad C commented on HDFS-14703: --- Thanks [~shv] for review & feedback. Shall i raise

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379531#comment-17379531 ] Wei-Chiu Chuang commented on HDFS-14703: Added a version "Fine-Grained Locking". Please use it

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17378314#comment-17378314 ] Konstantin Shvachko commented on HDFS-14703: Great progress [~prasad-acit]. It proves the

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359651#comment-17359651 ] Xing Lin commented on HDFS-14703: - Hi [~prasad-acit], that is awesome! Konstantin is on vacation this

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359306#comment-17359306 ] Renukaprasad C commented on HDFS-14703: --- [~shv]/ [~xinglin] We have implemented FGL for Create API

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346379#comment-17346379 ] Konstantin Shvachko commented on HDFS-14703: Thanks [~prasad-acit] and [~xinglin] for

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346002#comment-17346002 ] Renukaprasad C commented on HDFS-14703: --- Thanks [~shv] & [~xinglin] I have tested on 48 Core

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345870#comment-17345870 ] Xing Lin commented on HDFS-14703: - I did some performance benchmarks using a physical server (a d430

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345069#comment-17345069 ] Xing Lin commented on HDFS-14703: - [~prasad-acit] try this command: use -fs file:///, instead of

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344975#comment-17344975 ] Renukaprasad C commented on HDFS-14703: --- Thanks [~xinglin], I tried with 8 core on laptop as well

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344970#comment-17344970 ] Xing Lin commented on HDFS-14703: - [~prasad-acit] how many CPU cores does your server have? The OPS per

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343725#comment-17343725 ] Renukaprasad C commented on HDFS-14703: --- [~shv] Thanks for sharing the patch. I tried to test the

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341089#comment-17341089 ] Konstantin Shvachko commented on HDFS-14703: Updated the POC patches. There were indeed some

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321979#comment-17321979 ] Renukaprasad C commented on HDFS-14703: --- [~shv] Thanks for sharing design and the patch. There are

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279473#comment-17279473 ] Hui Fei commented on HDFS-14703: [~shv] Great feature and look forward to it, Is it in progress? >

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213932#comment-17213932 ] Hemanth Boyina commented on HDFS-14703: --- thanks [~shv] for your work gone through the design doc ,

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191894#comment-17191894 ] Konstantin Shvachko commented on HDFS-14703: After HDFS-14731 the first two patches are

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188871#comment-17188871 ] Konstantin Shvachko commented on HDFS-14703: Hey guys. Glad to hear of your interest in this

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187451#comment-17187451 ] Xiaoqiao He commented on HDFS-14703: cc [~shv]. {quote}I want to do some work on this issue ,could

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187409#comment-17187409 ] junbiao chen commented on HDFS-14703: - Which  version does the patch base on?thanks > NameNode

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032095#comment-17032095 ] Ayush Saxena commented on HDFS-14703: - Thanx [~shv] for the design, Overall design seems interesting. 

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955302#comment-16955302 ] Konstantin Shvachko commented on HDFS-14703: Updated the design doc. Added a picture and some

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923782#comment-16923782 ] Konstantin Shvachko commented on HDFS-14703: Good questions guys, thanks. ??how to handle

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922937#comment-16922937 ] Anu Engineer commented on HDFS-14703: - [~shv] We ([~arp] , [~xyao] , [~jojochuang] , [~szetszwo] )

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922886#comment-16922886 ] Konstantin Shvachko commented on HDFS-14703: Hey [~arp] , thanks for reviewing. ??Do atomic

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922880#comment-16922880 ] Konstantin Shvachko commented on HDFS-14703: Hey [~hexiaoqiao], clarifying on your questions.

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919948#comment-16919948 ] Arpit Agarwal commented on HDFS-14703: -- Interesting proposal [~shv] . Thanks for sharing this and

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909681#comment-16909681 ] He Xiaoqiao commented on HDFS-14703: Thanks [~shv] for your POC patches. I have to state that this is

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907662#comment-16907662 ] Konstantin Shvachko commented on HDFS-14703: Attaching the POC patch. It consists of 4

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903374#comment-16903374 ] Konstantin Shvachko commented on HDFS-14703: Hi [~hexiaoqiao], thanks for reviewing the doc.

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901887#comment-16901887 ] He Xiaoqiao commented on HDFS-14703: Thanks [~shv] for file this JIRA and plan to push this feature

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900526#comment-16900526 ] Konstantin Shvachko commented on HDFS-14703: Attached the design document for review. >