I think you can create web service using Java, and then in .net using the web service to display the result.
On Thu, Feb 4, 2010 at 7:21 PM, Jeff Zhang <[email protected]> wrote: > Do you mean want to connect the JobTracker using .Net ? If so, I'm afraid I > have no idea how to this. The rpc of hadoop is language dependent. > > > > > On Thu, Feb 4, 2010 at 7:18 PM, Mark N <[email protected]> wrote: > >> could you please elaborate on this ( * hint to get started as am very >> new >> to hadoop? ) >> So far I could succesfully read all the default and custom counters. >> >> Currently we are having a .net client. >> >> thanks in advance. >> >> >> On Thu, Feb 4, 2010 at 4:53 PM, Jeff Zhang <[email protected]> wrote: >> >> > Well, you can create a proxy of JobTracker in client side, and then you >> can >> > use the API of JobTracker to get the information of jobs. The Proxy take >> > the >> > responsibility of communication with the Master Node. Read the source >> > code >> > of JobClient can help you. >> > >> > >> > On Thu, Feb 4, 2010 at 6:59 PM, Mark N <[email protected]> wrote: >> > >> > > Ye currently am using jobclient to read these counters. >> > > >> > > But We are not able to use *webservices *because the jar which is used >> to >> > > read the counters from running hadoop job is itself a Hadoop program >> > > >> > > If we could have pure Java Api which is run without hadoop command >> then >> > we >> > > could return the counter variable into webservices and show in UI. >> > > >> > > Any help or technique to show thsese counters in the UI would be >> > > appreciated ( not necessarily using web service ) >> > > >> > > >> > > I am using webservices because I am having .net VB client >> > > >> > > thanks >> > > >> > > >> > > >> > > On Wed, Feb 3, 2010 at 8:33 PM, Jeff Zhang <[email protected]> wrote: >> > > >> > > > I think you can use JobClient to get the counters in your web >> service. >> > > > If you look at the shell script bin/hadoop, you will find that >> actually >> > > > this >> > > > shell use the JobClient to get the counters. >> > > > >> > > > >> > > > >> > > > On Wed, Feb 3, 2010 at 4:34 AM, Mark N <[email protected]> >> wrote: >> > > > >> > > > > We have a hadoop job running and have used custom counters to >> track >> > > few >> > > > > counters ( like no of successfully processed documents matching >> > certain >> > > > > conditions) >> > > > > >> > > > > >> > > > > Since we need to get this counters even while the Hadoop job is >> > running >> > > , >> > > > > we >> > > > > wrote another Java program to read these counters >> > > > > >> > > > > >> > > > > *Counter reader program *will do the following : >> > > > > >> > > > > >> > > > > 1) List all the running jobs. >> > > > > >> > > > > 2) Get the running job using Job name >> > > > > >> > > > > 2) Get all the counter for individual running jobs >> > > > > >> > > > > 3) Set this counters in variables. >> > > > > We could successfully read these counters , but since we >> need >> > > to >> > > > > show these counters to custom UI , how can we show these counters? >> > > > > >> > > > > we looked into various options to read these counters to >> show >> > in >> > > > UI >> > > > > as following : >> > > > > >> > > > > 1. Dump these counters to database , however this may be >> > overhead >> > > > > 2. Write web service and UI will invoke the functions from >> > these >> > > > > service to show in UI ( However since we need to run "*Counter >> reader >> > > > > program " *with Hadoop command it might not be feasible to write >> web >> > > > > service ? ) >> > > > > >> > > > > so the question is can we achive to read the counters using >> > simple >> > > > > Java APIs ? Does anyone have idea how does the default jobtracker >> JSP >> > > > works >> > > > > ? we wanted to built something similar to this >> > > > > >> > > > > thanks >> > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > Nipen Mark >> > > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > Best Regards >> > > > >> > > > Jeff Zhang >> > > > >> > > >> > > >> > > >> > > -- >> > > Nipen Mark >> > > >> > >> > >> > >> > -- >> > Best Regards >> > >> > Jeff Zhang >> > >> >> >> >> -- >> Nipen Mark >> > > > > -- > Best Regards > > Jeff Zhang > -- Best Regards Jeff Zhang
