[
https://issues.apache.org/jira/browse/HDFS-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075157#comment-14075157
]
Daryn Sharp commented on HDFS-6709:
-----------------------------------
I sense a bit of condescension. We're all friends here. I'm just asking
leading questions to make sure this approach is sound. Y! stands to lose a lot
if this doesn't actually scale. To clarify the RTTI, I thought you meant more
than just a per-instance reference to the class would be saved - although
saving a reference is indeed great. Regarding atomicity/CAS, it's relevant
because using misalignment (over-optimization?) prevents adding concurrency to
data structures that aren't but should allow concurrency. I digress....
The important point for discussion is this:
bq. No, because every modern GC uses "generational collection." This means that
short-lived instances are quickly cleaned up, without any pauses.
I know about generational collection but I'm admittedly not an expert. Which
young gen GC method does not pause? ParNew+CMS definitively pauses... Here
are some quickly gathered 12-day observations from a moderately loaded,
multi-thousand node, non-production cluster:
* ParNew collections:
** frequency: every ~4s
** range: 66ms -> 468ms
** avg: 130ms
** collects: 2.5GB
* CMS old collections:
** frequency: < 4 per day
** range: 1.0s -> 2.9s
** avg: 1.5s
We have production clusters over 2.5X larger that sustained over 3X ops/sec.
This non-prod cluster is generating ~625MB of garbage/sec. How do you predict
dynamic instantiation of INode and BlockInfo objects will help? They generally
won't be promoted to old gen which should reduce the infrequent CMS collection
times. BUT, will it dramatically increase the frequency of young collection
and/or lead to premature tenuring?
I'm honestly asking for design specifics of how this will help. Are you
positive we won't get a little by giving up a lot? I'd love for my concerns to
be unfounded.
> Implement off-heap data structures for NameNode and other HDFS memory
> optimization
> ----------------------------------------------------------------------------------
>
> Key: HDFS-6709
> URL: https://issues.apache.org/jira/browse/HDFS-6709
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-6709.001.patch
>
>
> We should investigate implementing off-heap data structures for NameNode and
> other HDFS memory optimization. These data structures could reduce latency
> by avoiding the long GC times that occur with large Java heaps. We could
> also avoid per-object memory overheads and control memory layout a little bit
> better. This also would allow us to use the JVM's "compressed oops"
> optimization even with really large namespaces, if we could get the Java heap
> below 32 GB for those cases. This would provide another performance and
> memory efficiency boost.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
