[
https://issues.apache.org/jira/browse/HDFS-5451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827261#comment-13827261
]
Colin Patrick McCabe commented on HDFS-5451:
--------------------------------------------
bq. Can we hide these new PBCD Builder methods from users of DFS? They're
meaningless for create/modify, ideally we only see them when doing listing.
Seems well suited as a listing subclass?
as per our discussion earlier, I'd rather not create more subclasses. Users
should be able to get back a PBCD from listDirectives, modify one thing, and
then use it in modifyDirective.
bq.Extra newline at the end of the Builder
ok
bq. Seems like having #clear and #increment(long) methods would better suit the
new PBCE byte methods.
ok
bq. I feel like this should be a min then, so the repl 1 PBCE doesn't get
charged double....
Yeah, this was supposed to be min. Good call.
bq. BPOS#blockIdArrayToString, could this go in DFSUtil instead? Seems like a
better place for it. Also, you can use Guava's Joiner for doing this kind of
task, and Arrays.asList and List.subList could get the max behavior.
I don't want to do so much copying. This list could have arbitrary length.
This can't go in DFSUtil since it depends on the configuration value for
maximum blocks to print.
bq. Should this extra logging maybe be at DEBUG? It could be a rather large
message, even with the 1000 limit.
I think seeing what was cached is useful. This will be our only window into
what happened in production in a lot of cases.
bq. CacheAdmin, the row extension for printStatus is kinda ugly. Maybe use an
ArrayList so we can cleanly append?
It's kind of annoying, but {{ArrayList}} doesn't actually give you acess to the
backing array. Anyway, this array is tiny. Let's just use a List and then
call toArray at the end.
bq. Good catch on right justifying numeric columns. Mind doing the same for the
ID field?
ok
bq. Tests need to be rebased on trunk, but there's an above comment on the JIRA
about verifying uncache/cache races.
Let's do that later. It's not really related to the other changes here and we
want the stats in soon.
bq. Can we also get a test for the code snippet above, where we have multiple
things caching the same file with different repls?
I added a new test for this area.
bq. Test verifying stats for a cached directory as files are added to it?
the new test covers this, I think
> add more debugging for cache rescan
> -----------------------------------
>
> Key: HDFS-5451
> URL: https://issues.apache.org/jira/browse/HDFS-5451
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Affects Versions: 3.0.0
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-5451.001.patch, HDFS-5451.002.patch,
> HDFS-5451.003.patch
>
>
> It would be nice to have message at DEBUG level that described all the
> decisions we made for cache entries. That way we could turn on this
> debugging to get more information. We should also store the number of bytes
> each PBCE wanted, and the number of bytes it got, plus the number of inodes
> it got, and output those in {{listDirectives}}.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
