[
https://issues.apache.org/jira/browse/HDFS-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HDFS-3969:
------------------------------
Attachment: hdfs-3969.txt
Here's an initial rev of this patch. Enumerating the changes:
- Got rid of the VolumeId.isValid() API. Instead, we just put a null in that
slot of the locations array (a user of this API found the {{isValid()}} stuff
confusing)
- Changed stringification to use hex encoding instead of Base64, since it's
more readable by developers (eg "00000001" instead of "AAAAAQ==")
- Changed RPCs over to take a block pool ID and a packed array of longs, as
described above. The RPCs should be substantially more efficient now.
- Fixed a bug where, if any of the blocks in the list were non-existent, it
would previously throw an exception and return no data from the call. Now it
just fills in a null. Added a new test case for this.
- Added some extra trace logs, etc, which I found handy in debugging here.
- Added exception text to the INFO log when a call fails
- Made the timeout default to 1 second instead of 60 seconds -- since this is a
performance API, I expect a default timeout on this order is much more
reasonable. Also renamed the parameter to make it a millisecond unit like most
of our other timeouts, and fixed the timeout on the executor to match units.
This change is not wire-compatible, but since this API is marked experimental,
I think it's OK. I used a new protobuf tag ID so that, if an old client talks
to a new DN or vice versa, they'll get appropriate exceptions about missing
fields rather than incorrect data.
> Small bug fixes and improvements for disk locations API
> -------------------------------------------------------
>
> Key: HDFS-3969
> URL: https://issues.apache.org/jira/browse/HDFS-3969
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hdfs-3969.txt
>
>
> The new disk block locations API has a configurable timeout, but it's used
> inconsistently: the invokeAll() call to the thread pool assumes the timeout
> is in seconds, but the RPC timeout is set in milliseconds.
> Also, we can improve the wire protocol for this API to be a lot more
> efficient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
