Dear Swetha,
We fixed the OOZIE_HTTP_HOSTNAME issue temporarily by hard-coding the
machine name and adding dns entries on every hadoop TT machine.
But the jobs still seem slower than expected. Almost all the time we
have free mapper/reducer slots available on hadoop but enough jobs are
not running.
Apart from setting the concurrency of the coordinator jobs to a higher
number, is there any other configuration that can be changed so that
idle time is minimum?
I read on a blog regarding maximum number of threads for jobs in each
state and even in the oozie configuration I can see a lot of threadpools
defined with max thread limits.
Is there any particular config that we need to change?
We have a backlog of jobs for one month and there are a lot of jobs
available that can be submitted on the cluster, so according to me all
the hadoop slots should be completely utilized.
Regards,
On 10/14/2014 11:56 AM, Shwetha GS wrote:
This is what oozie server does:
export OOZIE_HTTP_HOSTNAME=`hostname -f`
export OOZIE_BASE_URL="http://
${OOZIE_HTTP_HOSTNAME}:${OOZIE_HTTP_PORT}/oozie"
catalina_opts="${catalina_opts} -Doozie.base.url=${OOZIE_BASE_URL}";
Callback url is picked up from oozie.base.url config which is set to
http://:11000/oozie
in your case which means oozie failed to get hostname
The jobs run on TT, so the host resolution should work on TTs
-Shwetha
On Tue, Oct 14, 2014 at 11:45 AM, Harshal Vora <[email protected]> wrote:
Re-sending the shell output
------------------------------------------------------------------------------------------
[xxx@server-lp-4 ~]$ hostname -f
server-lp-4
[xxx@server-lp-4 ~]$ sudo su yyy
[sudo] password for xxx:
bash-4.1$ hostname -f
server-lp-4
------------------------------------------------------------------------------------------
On 10/14/2014 11:43 AM, Harshal Vora wrote:
Dear Shwetha,
Below is the output of shell on oozie machine for both regular user (xxx)
as well as oozie user (yyy) which is used to submit the jobs.
For security purpose I have changed the user names to xxx and yyy.
We use /etc/hosts for DNS mapping as we do not have a central DNS server.
So I believe we will have to put the oozie machines mapping i.e.
"server-lp-4" on jobtracker machine.
------------------------------------------------------------------------------------------
[xxx@server-lp-4 ~]$ hostname -f
server-lp-4
[xxx@packet-server-lp-4 ~]$ sudo su yyy
[sudo] password for xxx:
bash-4.1$ hostname -f
server-lp-4
------------------------------------------------------------------------------------------
On 10/14/2014 11:36 AM, Shwetha GS wrote:
If the command 'hostname -f' was resolving correctly, callback url would
have the host. So, I don't think the command is returning the host.
Once you resolve this, hadoop machine where the jobs runs should resolve
the host.
On Tue, Oct 14, 2014 at 11:14 AM, Harshal Vora <[email protected]>
wrote:
Dear Swetha,
On oozie machine, hostname points to "server-lp-4" which is the name we
have given to our machine.
Does this mean we need a mapping in DNS for this hostname on the hadoop
machine?
Regards,
On 10/14/2014 10:49 AM, Shwetha GS wrote:
Oozie uses 'hostname -f' to get the hostname which is used in setting
callback url. Check if its resolving correctly. This is the reason for
delay between job completion and oozie status update
-Shwetha
On Tue, Oct 14, 2014 at 10:17 AM, Harshal Vora <[email protected]>
wrote:
Logs on Jobtracker machine:
ERROR org.apache.hadoop.mapred.JobEndNotifier: Notification failure
[URL:
http://:11000/oozie/callback?id=0000443-141012054231695-oozie-oozi-W@
visitInfoProcessor-cascading&status=SUCCEEDED& remaining retries: 0
interval: 30000]
~/.bashrc on oozie machine.
export OOZIE_URL=http://localhost:11000/oozie
Is the hostname empty in the callback URL because it is configured to
localhost on oozie machine?
For some reason the transition between oozie actions and job status
change
is very very slow.
Regards,
--
Regards,
Harshal Vora
www.radiolocus.com
--
Regards,
Harshal Vora
www.radiolocus.com
--
Regards,
Harshal Vora
www.radiolocus.com
--
Regards,
Harshal Vora
www.radiolocus.com
