[jira] [Commented] (HDFS-13752) fs.Path stores file path in java.net.URI causes big memory waste

2018-08-17 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584480#comment-16584480 ] Misha Dmitriev commented on HDFS-13752: --- [~b.maidics] thank you, from my prospective this looks

[jira] [Commented] (HDFS-13752) fs.Path stores file path in java.net.URI causes big memory waste

2018-08-10 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576932#comment-16576932 ] Misha Dmitriev commented on HDFS-13752: --- [~b.maidics] thank you for investigating this, your

[jira] [Commented] (HDFS-13752) fs.Path stores file path in java.net.URI causes big memory waste

2018-07-20 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551380#comment-16551380 ] Misha Dmitriev commented on HDFS-13752: --- Ok, I've downloaded and analyzed the heap dump with jxray

[jira] [Updated] (HDFS-13752) fs.Path stores file path in java.net.URI causes big memory waste

2018-07-20 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-13752: -- Attachment: heapdump-10partitions.html > fs.Path stores file path in java.net.URI causes

[jira] [Commented] (HDFS-13752) fs.Path stores file path in java.net.URI causes big memory waste

2018-07-20 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551229#comment-16551229 ] Misha Dmitriev commented on HDFS-13752: --- Hi [~b.maidics], Can you share more details of your

[jira] [Commented] (HDFS-13671) Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

2018-06-15 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514289#comment-16514289 ] Misha Dmitriev commented on HDFS-13671: --- [~linyiqun] did you check how much time NN was spending in

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-14 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365020#comment-16365020 ] Misha Dmitriev commented on HDFS-12051: --- [~atm] I've just submitted a patch where I've addressed

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-14 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Release Note: Addressed the @atm's comments Status: Patch Available (was: In

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-14 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.12.patch > Reimplement NameCache in NameNode: Intern duplicate byte[]

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-14 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Reimplement NameCache in NameNode: Intern

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-09 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358939#comment-16358939 ] Misha Dmitriev commented on HDFS-12051: --- Thank you for the review [~atm] Please see my answers

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-07 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) Addressed the latest commment by [~yzhangal]

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-07 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.11.patch > Reimplement NameCache in NameNode: Intern duplicate byte[]

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-07 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Reimplement NameCache in NameNode: Intern

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-06 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354588#comment-16354588 ] Misha Dmitriev commented on HDFS-12051: --- [~szetszwo] I've done more benchmarking per your request.

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-05 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) In this patch, the default NameCache size is

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-05 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.10.patch > Reimplement NameCache in NameNode: Intern duplicate byte[]

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-05 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Reimplement NameCache in NameNode: Intern

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-05 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353441#comment-16353441 ] Misha Dmitriev commented on HDFS-12051: --- I tested my change in a relatively small cluster that

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-01 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349174#comment-16349174 ] Misha Dmitriev commented on HDFS-12051: --- I've just attached the detailed document comparing the old

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-01 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051-NameCache-Rewrite.pdf > Reimplement NameCache in NameNode: Intern

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-30 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346080#comment-16346080 ] Misha Dmitriev commented on HDFS-12051: --- I ran another test in a cluster with ~30M HDFS files, where

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-30 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) I've just updated this patch to make NameNode

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-30 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.09.patch > Reimplement NameCache in NameNode: Intern duplicate byte[]

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-30 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Reimplement NameCache in NameNode: Intern

[jira] [Comment Edited] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-25 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340176#comment-16340176 ] Misha Dmitriev edited comment on HDFS-12051 at 1/25/18 10:29 PM: - Thank

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-25 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) Addressed the most recent comments by [~yzhangal]

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-25 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.08.patch > Reimplement NameCache in NameNode: Intern duplicate byte[]

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-25 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Reimplement NameCache in NameNode: Intern

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-25 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340176#comment-16340176 ] Misha Dmitriev commented on HDFS-12051: --- Thank you for the review, [~manojg] See my responses inline

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-23 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) > Reimplement NameCache in NameNode: Intern

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-23 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336416#comment-16336416 ] Misha Dmitriev commented on HDFS-12051: --- Thank you [~yzhangal], I've addressed your comment and

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-23 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Reimplement NameCache in NameNode: Intern

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-23 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.07.patch > Reimplement NameCache in NameNode: Intern duplicate byte[]

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-17 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329913#comment-16329913 ] Misha Dmitriev commented on HDFS-12051: --- [~szetszwo] regarding the patch name: I believe your

[jira] [Updated] (HDFS-12051) Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-16 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Description: When snapshot diff operation is performed in a NameNode that manages several

[jira] [Commented] (HDFS-12051) Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-16 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328189#comment-16328189 ] Misha Dmitriev commented on HDFS-12051: --- I've just modified the ticket name and description. >

[jira] [Updated] (HDFS-12051) Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-01-16 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Summary: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save

[jira] [Updated] (HDFS-12051) Intern duplicate byte[] arrays, mainly denoting file/directory names, to save memory

2018-01-16 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Summary: Intern duplicate byte[] arrays, mainly denoting file/directory names, to save memory

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2018-01-16 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328130#comment-16328130 ] Misha Dmitriev commented on HDFS-12051: --- [~szetszwo] did you see my previous response, by any

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2018-01-16 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327669#comment-16327669 ] Misha Dmitriev commented on HDFS-12051: --- [~szetszwo] I hope I've addressed all your concerns? >

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2018-01-10 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321758#comment-16321758 ] Misha Dmitriev commented on HDFS-12051: --- [~szetszwo] I have already provided you the numbers that

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2018-01-08 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317025#comment-16317025 ] Misha Dmitriev commented on HDFS-12051: --- >> The extra code on itself doesn't noticeably affect the

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2018-01-05 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314358#comment-16314358 ] Misha Dmitriev commented on HDFS-12051: --- >> ... it's nice to finally see someone other than Yongjun

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2018-01-05 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314339#comment-16314339 ] Misha Dmitriev commented on HDFS-12051: --- Hi [~szetszwo], it's nice to finally see someone other than

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-21 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) > Intern INOdeFileAttributes$SnapshotCopy.name

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-21 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.06.patch > Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-21 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16300664#comment-16300664 ] Misha Dmitriev commented on HDFS-12051: --- Addressed Yongjun's comments and submitted a new patch. >

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-21 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Intern INOdeFileAttributes$SnapshotCopy.name

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-15 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293428#comment-16293428 ] Misha Dmitriev commented on HDFS-12051: --- There are some test failures again. They seem unrelated -

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-15 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.05.patch > Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-15 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) > Intern INOdeFileAttributes$SnapshotCopy.name

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-15 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Intern INOdeFileAttributes$SnapshotCopy.name

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-15 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293109#comment-16293109 ] Misha Dmitriev commented on HDFS-12051: --- Test failures above (some with OOM) look rather strange. I

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-14 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Intern INOdeFileAttributes$SnapshotCopy.name

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-14 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.04.patch > Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-14 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) > Intern INOdeFileAttributes$SnapshotCopy.name

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-12-14 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291574#comment-16291574 ] Misha Dmitriev commented on HDFS-12051: --- In the most recent heap dump obtained from a very big HDFS

[jira] [Updated] (HDFS-12922) Arrays of length 1 cause 9.2% memory overhead

2017-12-12 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12922: -- Attachment: screenshot-1.png > Arrays of length 1 cause 9.2% memory overhead >

[jira] [Created] (HDFS-12922) Arrays of length 1 cause 9.2% memory overhead

2017-12-12 Thread Misha Dmitriev (JIRA)
Misha Dmitriev created HDFS-12922: - Summary: Arrays of length 1 cause 9.2% memory overhead Key: HDFS-12922 URL: https://issues.apache.org/jira/browse/HDFS-12922 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-11-30 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.03.patch > Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-11-30 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) > Intern INOdeFileAttributes$SnapshotCopy.name

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-11-30 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273322#comment-16273322 ] Misha Dmitriev commented on HDFS-12051: --- Thank you for the review [~yzhangal] Here are the answers

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-07-12 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Intern INOdeFileAttributes$SnapshotCopy.name

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-07-12 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.02.patch I've redesigned the new NameCache so that its size adjusts

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-07-12 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) > Intern INOdeFileAttributes$SnapshotCopy.name

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-07-06 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: Open) > Intern INOdeFileAttributes$SnapshotCopy.name byte[]

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-07-06 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.01.patch > Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-06-29 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068797#comment-16068797 ] Misha Dmitriev commented on HDFS-12051: --- I've just discovered that this problem has apparently been

[jira] [Commented] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-06-28 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067460#comment-16067460 ] Misha Dmitriev commented on HDFS-12051: --- Upon a closer analysis of the problem, it looks like we

[jira] [Updated] (HDFS-12042) Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption

2017-06-28 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Attachment: HDFS-12042.04.patch Addressed the last comment by [~jojochuang], submitted

[jira] [Updated] (HDFS-12042) Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption

2017-06-28 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Status: Patch Available (was: In Progress) > Lazy initialize AbstractINodeDiffList#diffs for

[jira] [Updated] (HDFS-12042) Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption

2017-06-28 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Status: In Progress (was: Patch Available) > Lazy initialize AbstractINodeDiffList#diffs for

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Description: When snapshot diff operation is performed in a NameNode that manages several

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Description: When snapshot diff operation is performed in a NameNode that manages several

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Description: When snapshot diff operation is performed in a NameNode that manages several

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Description: When snapshot diff operation is performed in a NameNode that manages several

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Description: When snapshot diff operation is performed in a NameNode that manages several

[jira] [Updated] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Description: When snapshot diff operation is performed in a NameNode that manages several

[jira] [Created] (HDFS-12051) Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory

2017-06-27 Thread Misha Dmitriev (JIRA)
Misha Dmitriev created HDFS-12051: - Summary: Intern INOdeFileAttributes$SnapshotCopy.name byte[] arrays to save memory Key: HDFS-12051 URL: https://issues.apache.org/jira/browse/HDFS-12051 Project:

[jira] [Commented] (HDFS-12042) Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065364#comment-16065364 ] Misha Dmitriev commented on HDFS-12042: --- [~manojg], I'll add/update the tests and will post another

[jira] [Updated] (HDFS-12042) Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Attachment: HDFS-12042.03.patch Addressed [~jojochuang]'s comments. > Lazy initialize

[jira] [Updated] (HDFS-12042) Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Status: Patch Available (was: In Progress) > Lazy initialize AbstractINodeDiffList#diffs for

[jira] [Updated] (HDFS-12042) Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Status: In Progress (was: Patch Available) > Lazy initialize AbstractINodeDiffList#diffs for

[jira] [Commented] (HDFS-12042) Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065266#comment-16065266 ] Misha Dmitriev commented on HDFS-12042: --- Hi [~manojg], With all the due respect, the measurements

[jira] [Commented] (HDFS-12042) Reduce memory used by snapshot diff data structures

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065184#comment-16065184 ] Misha Dmitriev commented on HDFS-12042: --- Yes [~jojochuang], you are correct. The essence of the

[jira] [Commented] (HDFS-12042) Reduce memory used by snapshot diff data structures

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064472#comment-16064472 ] Misha Dmitriev commented on HDFS-12042: --- Looks like the failed test are unrelated and flaky - at

[jira] [Updated] (HDFS-12042) Reduce memory used by snapshot diff data structures

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Status: Patch Available (was: In Progress) > Reduce memory used by snapshot diff data

[jira] [Updated] (HDFS-12042) Reduce memory used by snapshot diff data structures

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Attachment: HDFS-12042.02.patch > Reduce memory used by snapshot diff data structures >

[jira] [Updated] (HDFS-12042) Reduce memory used by snapshot diff data structures

2017-06-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Status: In Progress (was: Patch Available) > Reduce memory used by snapshot diff data

[jira] [Updated] (HDFS-12042) Reduce memory used by snapshot diff data structures

2017-06-26 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Status: Patch Available (was: Open) > Reduce memory used by snapshot diff data structures >

[jira] [Updated] (HDFS-12042) Reduce memory used by snapshot diff data structures

2017-06-26 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Attachment: HDFS-12042.01.patch > Reduce memory used by snapshot diff data structures >

[jira] [Updated] (HDFS-12042) Reduce memory used by snapshot diff data structures

2017-06-26 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12042: -- Description: When snapshot diff operation is performed in a NameNode that manages several

[jira] [Created] (HDFS-12042) Reduce memory used by snapshot diff data structures

2017-06-26 Thread Misha Dmitriev (JIRA)
Misha Dmitriev created HDFS-12042: - Summary: Reduce memory used by snapshot diff data structures Key: HDFS-12042 URL: https://issues.apache.org/jira/browse/HDFS-12042 Project: Hadoop HDFS

[jira] [Updated] (HDFS-11383) String duplication in org.apache.hadoop.fs.BlockLocation

2017-06-01 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-11383: -- Hi Andrew, I think this time I've addressed all your concerns and there is nothing from findbugs or

[jira] [Updated] (HDFS-11383) String duplication in org.apache.hadoop.fs.BlockLocation

2017-05-31 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-11383: -- Status: Patch Available (was: In Progress) > String duplication in

[jira] [Updated] (HDFS-11383) String duplication in org.apache.hadoop.fs.BlockLocation

2017-05-31 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-11383: -- Status: In Progress (was: Patch Available) > String duplication in

[jira] [Updated] (HDFS-11383) String duplication in org.apache.hadoop.fs.BlockLocation

2017-05-31 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-11383: -- Attachment: HDFS-11383.04.patch > String duplication in org.apache.hadoop.fs.BlockLocation >

[jira] [Commented] (HDFS-11383) String duplication in org.apache.hadoop.fs.BlockLocation

2017-05-31 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032008#comment-16032008 ] Misha Dmitriev commented on HDFS-11383: --- Ok, I've fixed checkstyles and added HashCodeBuilder. Will

[jira] [Updated] (HDFS-11383) String duplication in org.apache.hadoop.fs.BlockLocation

2017-05-30 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-11383: -- Status: Patch Available (was: In Progress) > String duplication in

  1   2   >