[
https://issues.apache.org/jira/browse/HDFS-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931082#comment-13931082
]
Haohui Mai commented on HDFS-5978:
----------------------------------
{code}
echo " oiv apply the offline fsimage viewer to an fsimage"
+ echo " wiv run the web fsimage viewer to an fsimage"
{code}
It might be more better to put this functionality as a subtool of the offline
image viewer, since it is intended to be a successor of the lsr tool.
It might be cleaner to separate the logics of loading fsimage and handling
webhdfs requests in {{FSImageHandler}}. You can wrap the information into a
private static class:
{code}
+ private String[] stringTable;
+ private HashMap<Long, INode> inodes = Maps.newHashMap();
+ private HashMap<Long, long[]> dirmap = Maps.newHashMap();
+ private ArrayList<INodeReferenceSection.INodeReference> refList =
+ Lists.newArrayList();
{code}
{code}
+ public ChannelPipeline getPipeline() throws Exception {
+ ChannelPipeline pipeline = Channels.pipeline();
+ pipeline.addLast("httpDecoder", new HttpRequestDecoder());
+ pipeline.addLast("requestHandler", new FSImageHandler(inputFile));
+ pipeline.addLast("stringEncoder", new StringEncoder());
+ pipeline.addLast("httpEncoder", new HttpResponseEncoder());
+ return pipeline;
+ }
+ }
{code}
You might be able to create a static pipeline instead of a pipeline factory.
See {{setPipeline()}} for more details. I'm also unclear why {{StringEncoder}}
is required.
{code}
+ public void testWebImageViewer() throws IOException, InterruptedException {
+ final String port = "9001";
{code}
The command line needs to accept both the listening host and the port.
Otherwise it'll listen to all interfaces by default. In unit test, it is also
important to configure the port to zero to avoid intermediate failures.
{code}
+ // wait until the viewer starts
+ Thread.sleep(3000);
+
{code}
You can use a condition variable here instead of sleeping.
{code}
+ HttpClient client = new DefaultHttpClient();
+ HttpGet httpGet =
+ new HttpGet("http://localhost:"; + port + "/?op=LISTSTATUS");
+ HttpResponse response = client.execute(httpGet);
+ assertEquals(200, response.getStatusLine().getStatusCode());
+ assertEquals("application/json",
+ response.getEntity().getContentType().getValue());
{code}
Using the built-in {{HttpUrlConnection}} is sufficient. It reduces the
dependency of the unit tests.
{code}
+import com.google.gson.JsonArray;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
{code}
Please use jackson instead of gson. hadoop-hdfs does not depend on gson at all.
That way there is no additional dependency.
> Create a tool to take fsimage and expose read-only WebHDFS API
> --------------------------------------------------------------
>
> Key: HDFS-5978
> URL: https://issues.apache.org/jira/browse/HDFS-5978
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: tools
> Reporter: Akira AJISAKA
> Assignee: Akira AJISAKA
> Labels: newbie
> Attachments: HDFS-5978.patch
>
>
> Suggested in HDFS-5975.
> Add an option to exposes the read-only version of WebHDFS API for
> OfflineImageViewer. You can imagine it looks very similar to jhat.
> That way we can allow the operator to use the existing command-line tool, or
> even the web UI to debug the fsimage. It also allows the operator to
> interactively browsing the file system, figuring out what goes wrong.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
