Re: [VOTE] Apache Ambari 2.5.2 RC1

[Sumit Mohanty]

Verified the release tag and tarball content.

Verified the signature and hashes.

Using the source tarball compiled on a clean VM and ran some unit tests.

LGTM, +1.

Thanks Aravindan.

From: Aravindan Vijayan 
Sent: Saturday, August 26, 2017 2:52 PM
To: dev@ambari.apache.org
Subject: [VOTE] Apache Ambari 2.5.2 RC1

Hello all,

I have created an apache-ambari-2.5.2 release candidate.

GIT source tag (release-2.5.2-rc1) 
https://git-wip-us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.5.2-rc1

Staging site: http://home.apache.org/~avijayan/apache-ambari-2.5.2-rc1/

PGP release keys (signed using 088372a9) - 
http://pgp.mit.edu:11371/pks/lookup?search=0x088372A9=vindex

One can look into the issues fixed in this release at 
https://issues.apache.org/jira/projects/AMBARI/versions/12340469

Vote will be open for 72 hours.
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

My vote: +1
--
Thanks and Regards,
Aravindan Vijayan

Re: Ambari upgrade 2.5.1.0 issue - database consistency issues for topology

[Sumit Mohanty]

The error is:


2017-08-11 17:31:34,665 ERROR - Your topology request hierarchy is not complete 
for each row in topology_host_request should exist at least one row in 
topology_host_task, topology_logical_task.


This looks like https://issues.apache.org/jira/browse/AMBARI-19929?


The fix will prevent the inconsistency but meanwhile you need to delete some 
rows manually.


If you are ok, can you open an Apache Ambari JIRA with the log file as 
attachment. Someone will be able to add instructions for cleaning up the DB. 
That way it remains searchable for others.


thanks

Sumit


From: Anandha L Ranganathan <analog.s...@gmail.com>
Sent: Friday, August 11, 2017 1:29 PM
To: dev@ambari.apache.org
Cc: u...@ambari.apache.org
Subject: Re: Ambari upgrade 2.5.1.0 issue - database consistency issues for 
topology

Sumit,

Please find the attached database-check  logs.

On Fri, Aug 11, 2017 at 12:57 PM, Sumit Mohanty 
<smoha...@hortonworks.com<mailto:smoha...@hortonworks.com>> wrote:
Anandha, can you share the content of 
/var/log/ambari-server/ambari-server-check-database.log

thanks
Sumit

From: Anandha L Ranganathan 
<analog.s...@gmail.com<mailto:analog.s...@gmail.com>>
Sent: Friday, August 11, 2017 11:57 AM
To: u...@ambari.apache.org<mailto:u...@ambari.apache.org>; 
dev@ambari.apache.org<mailto:dev@ambari.apache.org>
Subject: Ambari upgrade 2.5.1.0 issue - database consistency issues for topology


I am trying to upgrade from ambari-2.2.1 to ambari-2.5.1. During upgrade it 
didn't throw any error and it completed successfully.

  1.  INFO: Updating Ambari Server properties in ambari.properties ...
  2.  INFO: Updating Ambari Server properties in ambari-env.sh ...
  3.  WARNING: Original file ambari-env.sh kept
  4.  INFO: Fixing database objects owner
  5.  Ambari Server configured for Embedded Postgres. Confirm you have made a 
backup of the Ambari Server database [y/n] (y)? y
  6.  INFO: Upgrading database schema
  7.  INFO: Return code from schema upgrade command, retcode = 0
  8.  INFO: Schema upgrade completed
  9.  Adjusting ambari-server permissions and ownership...
  10. Ambari Server 'upgrade' completed successfully.
  11.
  12.

After upgrade , I restarted the ambari-server and it throws this error.

  1.  [root@usw2dxdpmn02 yum.repos.d]# ambari-server restart
  2.  Using python  /usr/bin/python
  3.  Restarting ambari-server
  4.  Waiting for server stop...
  5.  Ambari Server stopped
  6.  Ambari Server running with administrator privileges.
  7.  Organizing resource files at /var/lib/ambari-server/resources...
  8.  Ambari database consistency check started...
  9.  Server PID at: /var/run/ambari-server/ambari-server.pid
  10. Server out at: /var/log/ambari-server/ambari-server.out
  11. Server log at: /var/log/ambari-server/ambari-server.log
  12. Waiting for server start
  13. DB configs consistency check failed. Run "ambari-server start 
--skip-database-check" to skip. You may try --auto-fix-database flag to attempt 
to fix issues automatically. If you use this "--skip-database-check" option, do 
not make any changes to your cluster topology or perform a cluster upgrade 
until you correct the database consistency issues. See 
/var/log/ambari-server/ambari-server-check-database.log for more details on the 
consistency issues.
  14. ERROR: Exiting with exit code -1.
  15. REASON: Ambari Server java process has stopped. Please check the logs for 
more information.
  16.
  17.
  18.

I am able to restart ambari-server with --skip-database-check. What is the 
reason for this error and how do I fix it?

Re: Ambari upgrade 2.5.1.0 issue - database consistency issues for topology

[Sumit Mohanty]

Anandha, can you share the content of 
/var/log/ambari-server/ambari-server-check-database.log

thanks
Sumit

From: Anandha L Ranganathan 
Sent: Friday, August 11, 2017 11:57 AM
To: u...@ambari.apache.org; dev@ambari.apache.org
Subject: Ambari upgrade 2.5.1.0 issue - database consistency issues for topology


I am trying to upgrade from ambari-2.2.1 to ambari-2.5.1. During upgrade it 
didn't throw any error and it completed successfully.

  1.  INFO: Updating Ambari Server properties in ambari.properties ...
  2.  INFO: Updating Ambari Server properties in ambari-env.sh ...
  3.  WARNING: Original file ambari-env.sh kept
  4.  INFO: Fixing database objects owner
  5.  Ambari Server configured for Embedded Postgres. Confirm you have made a 
backup of the Ambari Server database [y/n] (y)? y
  6.  INFO: Upgrading database schema
  7.  INFO: Return code from schema upgrade command, retcode = 0
  8.  INFO: Schema upgrade completed
  9.  Adjusting ambari-server permissions and ownership...
  10. Ambari Server 'upgrade' completed successfully.
  11.
  12.

After upgrade , I restarted the ambari-server and it throws this error.

  1.  [root@usw2dxdpmn02 yum.repos.d]# ambari-server restart
  2.  Using python  /usr/bin/python
  3.  Restarting ambari-server
  4.  Waiting for server stop...
  5.  Ambari Server stopped
  6.  Ambari Server running with administrator privileges.
  7.  Organizing resource files at /var/lib/ambari-server/resources...
  8.  Ambari database consistency check started...
  9.  Server PID at: /var/run/ambari-server/ambari-server.pid
  10. Server out at: /var/log/ambari-server/ambari-server.out
  11. Server log at: /var/log/ambari-server/ambari-server.log
  12. Waiting for server start
  13. DB configs consistency check failed. Run "ambari-server start 
--skip-database-check" to skip. You may try --auto-fix-database flag to attempt 
to fix issues automatically. If you use this "--skip-database-check" option, do 
not make any changes to your cluster topology or perform a cluster upgrade 
until you correct the database consistency issues. See 
/var/log/ambari-server/ambari-server-check-database.log for more details on the 
consistency issues.
  14. ERROR: Exiting with exit code -1.
  15. REASON: Ambari Server java process has stopped. Please check the logs for 
more information.
  16.
  17.
  18.

I am able to restart ambari-server with --skip-database-check. What is the 
reason for this error and how do I fix it?

Re: Ambari 2.6.x Release Management

[Sumit Mohanty]

Thanks Swapan for volunteering. +1

From: Aravindan Vijayan 
Sent: Thursday, August 03, 2017 1:11 PM
To: dev@ambari.apache.org
Subject: Re: Ambari 2.6.x Release Management

+1. Thanks Swapan.

--
Thanks and Regards,
Aravindan Vijayan








On 8/3/17, 12:55 PM, "Jayush Luniya"  wrote:

>+1. Thanks Swapan for volunteering.
>
>On 8/3/17, 11:21 AM, "Swapan Shridhar"  wrote:
>
>>
>>Hi All,
>>
>>I would like to volunteer to be the Release Manager for the upcoming
>>Ambari 2.6.x release.
>>
>>Thanks.
>>
>>Regards,
>>Swapan.
>>
>

Re: [VOTE] Apache Ambari 2.5.1 RC0

[Sumit Mohanty]

Verified the hashes and the signature.

Downloaded the tar and verified that it builds. The content of the tar looks 
good.

Cloned the repo and verified the tag for the release.

+1

From: Robert Nettleton 
Sent: Monday, May 22, 2017 2:51 PM
To: dev@ambari.apache.org
Subject: Re: [VOTE] Apache Ambari 2.5.1 RC0

Hi Aravindan,

+1 for RC0

I’ve downloaded the tarball, verified that the hashes are correct, ran a diff 
check against the source, ran a rat check build (which passed), and also 
verified that the ambari/ambari-server code compiles successfully.

Thanks,
Bob

> On May 22, 2017, at 2:54 PM, Aravindan Vijayan  
> wrote:
>
> Hello all,
>
> I have created an apache-ambari-2.5.1 release candidate.
>
> GIT source tag (release-2.5.1-rc0) 
> https://git-wip-us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.5.1-rc0
>
> Staging site: http://home.apache.org/~avijayan/apache-ambari-2.5.1-rc0/
>
> PGP release keys (signed using 088372a9) - 
> http://pgp.mit.edu:11371/pks/lookup?search=0x088372A9=vindex
>
> One can look into the issues fixed in this release at 
> https://issues.apache.org/jira/browse/AMBARI/fixforversion/12340256/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel
>
> Vote will be open for 72 hours.
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> My vote: +1
>
> --
> Thanks and Regards,
> Aravindan Vijayan

Re: [VOTE] Apache Ambari 2.4.3 RC0

[Sumit Mohanty]

Verified the hashes and the signature.

Downloaded the tar and verified that it builds. The content of the tar looks 
good.

Cloned the repo and verified the tag for the release.

+1

thanks Aravindan.

-Sumit

From: Aravindan Vijayan 
Sent: Friday, May 05, 2017 3:21 PM
To: dev@ambari.apache.org
Subject: [VOTE] Apache Ambari 2.4.3 RC0

Hello all,

I have created an apache-ambari-2.4.3 release candidate.

GIT source tag (release-2.4.3-rc0) 
https://git-wip-us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.4.3-rc0

Staging site: http://home.apache.org/~avijayan/apache-ambari-2.4.3-rc0/

PGP release keys (signed using 088372a9) - 
http://pgp.mit.edu:11371/pks/lookup?search=0x088372A9=vindex

One can look into the issues fixed in this release at 
https://issues.apache.org/jira/browse/AMBARI/fixforversion/12340370/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel

Vote will be open for 72 hours.
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

My vote: +1

--
Thanks and Regards,
Aravindan Vijayan

Re: Plain Hadoop with Ambari

[Sumit Mohanty]

It is certainly possible.

For example, contrib/management-packs/odpi-ambari-mpack is used to deploy the 
ODPi build of hadoop.

From: Mohit Goel 
Sent: Tuesday, April 04, 2017 11:19 PM
To: dev@ambari.apache.org
Subject: Plain Hadoop with Ambari

Hi Team,
My aim is to setup plain vanilla hadoop cluster and use Ambari for the
purpose of hadoop service monitoring. As I browsed through the existing
stack implementations, I found that most of the services extends existing
stack definitions defined in common-services folder. I need to use my
custom versions of different components independent of HortonWorks
repositories. Will it be as simple as creating a new repository and
specifying it in stack ? Will the existing  stack definitions defined in
common-services folder work ?
--


*Mohit GoelComputer EngineeringNIT Kurukshetra*

Re: [VOTE] Apache Ambari 2.5.0 RC1

[Sumit Mohanty]

+1 for RC1

On a clean vagrant VM:
* verified signature, md5 and sha1 hashes
* keychain looks good
* expanded the source tarball, compiled, and ran unit tests - compilation 
succeeded along with RAT check. 

Two ambari-server unit tests failed but that should not hold up the release. I 
will open a bug for the next release after verifying that they are not due to 
my VM environment.

Thanks Aravindan.

-Sumit

From: Aravindan Vijayan 
Sent: Thursday, March 23, 2017 2:11 PM
To: dev@ambari.apache.org
Subject: [VOTE] Apache Ambari 2.5.0 RC1

Hello all,

I have created an apache-ambari-2.5.0 release candidate.

GIT source tag (release-2.5.0-rc1) 
https://git-wip-us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.5.0-rc1

Staging site:  http://home.apache.org/~avijayan/apache-ambari-2.5.0-rc1/

PGP release keys (signed using 088372a9) - 
http://pgp.mit.edu:11371/pks/lookup?search=0x088372A9=vindex

One can look into the issues fixed in this release at 
https://issues.apache.org/jira/browse/AMBARI/fixforversion/12336060/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel

Vote will be open for 72 hours.
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

My vote: +1

FYI, the list of commits between rc0 and rc1
70b4ba403dd7e890d7fcc5964b1cd231ee6e98c0 AMBARI-20516 create table failing with 
HiveAccessControlException (additional patch). (ababiichuk)
7bb1e3335d41ab7adaf0c66ae5fd8df454178cd7 Revert "AMBARI-20488 Config types are 
validated before the cluster resources are persisted"
95d7cc2c000c350dc0e5efa5a0660098f21da547 Revert "AMBARI-20542 Fixed 
configuration type validation in case of blueprint deployments."
50b192979eca936bdb8ea1ecb9a1bb624c28e7fc AMBARI-20542 Fixed configuration type 
validation in case of blueprint deployments.

--
Thanks and Regards,
Aravindan Vijayan

[ANNOUNCE] Laszlo Puskas as a new Apache Ambari committer

[Sumit Mohanty]

Hi all,

I'm happy to announce that Laszlo Puskas has become a new Apache Ambari 
committer.
Laszlo has been making significant contributions in various areas of the Apache 
Ambari project.

Congratulations, Laszlo!
We look forward to your continued involvement to push the project forward.

thanks
-Sumit

Re: Old Cluster name appering in logs even after Ambari-Server Reset Query!

[Sumit Mohanty]

Once you deploy the new cluster they will go away. Agent may respond as there 
could be inflight commands and alert definitions.

-Sumit

From: Sandy 
Sent: Thursday, February 02, 2017 8:42 PM
To: dev@ambari.apache.org
Subject: Old Cluster name appering in logs even after Ambari-Server Reset Query!

I created an AMBARI cluster using Vagrant VMs.  After cluster creation, I
stopped all the services and did ambari-server reset.

I assumed that It'll be clearing all the information about cluster from DB
and none of the AGENTS maintain the state about cluster.  They just send
heartbeat messages to the server and in return look for commands that may
have to execute.

But after restarting the server, I observed the log messages, that
mentioned my cluster name (before reset).  Just wondering, How is this name
persisted??

Do AGENTS remember state regarding clusters and services??  If yes, how do
I clear that off (Like ambari-server reset, is there something like
ambari-agent reset too)??

// A SNIP FROM LOG MESSAGES

03 Feb 2017 15:19:12,911 DEBUG [alert-event-bus-1]
AlertReceivedListener:477 - Unable to process alert
hbase_regionserver_process for an invalid cluster named *hdpCluster*
org.apache.ambari.server.ClusterNotFoundException: Cluster not found,
clusterName=*hdpCluster*
at
org.apache.ambari.server.state.cluster.ClustersImpl.getCluster(ClustersImpl.java:293)
at
org.apache.ambari.server.events.listeners.alerts.AlertReceivedListener.isValid(AlertReceivedListener.java:468)
at
org.apache.ambari.server.events.listeners.alerts.AlertReceivedListener.onAlertEvent(AlertReceivedListener.java:150)
at
org.apache.ambari.server.events.listeners.alerts.AlertReceivedListener$$EnhancerByGuice$$ea7961df.CGLIB$onAlertEvent$0()
at
org.apache.ambari.server.events.listeners.alerts.AlertReceivedListener$$EnhancerByGuice$$ea7961df$$FastClassByGuice$$a5084ed4.invoke()
at
com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at
com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at
org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
at
com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at
com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at
org.apache.ambari.server.events.listeners.alerts.AlertReceivedListener$$EnhancerByGuice$$ea7961df.onAlertEvent()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.google.common.eventbus.EventSubscriber.handleEvent(EventSubscriber.java:74)
at com.google.common.eventbus.EventBus.dispatch(EventBus.java:322)
at
com.google.common.eventbus.AsyncEventBus.access$001(AsyncEventBus.java:34)
at
com.google.common.eventbus.AsyncEventBus$1.run(AsyncEventBus.java:117)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

--

*Sandeep Kumar,*
 Mobile +91-9866507368

*“Happiness is not a destination, It is the journey”*

Re: [VOTE] Apache Ambari 2.4.2 RC1

[Sumit Mohanty]

Verified the signature and hashes.

Compiled the source tarball and verified that it cleanly compiled.

+1

thanks
-Sumit

From: Jayush Luniya 
Sent: Friday, November 18, 2016 9:29 PM
To: dev@ambari.apache.org
Subject: [VOTE] Apache Ambari 2.4.2 RC1

Hello,

I have created an apache-ambari-2.4.2 release candidate.

GIT source tag (release-2.4.2-rc1) 
https://git-wip-us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.4.2-rc1

Staging site:  http://home.apache.org/~jluniya/apache-ambari-2.4.2-rc1/

PGP release keys (signed using 
CD23CAAE)http://pgp.mit.edu:11371/pks/lookup?op=vindex=0x4BC59EDACD23CAAE

One can look into the issues fixed in this release at 
https://issues.apache.org/jira/browse/AMBARI/fixforversion/12338230/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel

Vote will be open for 72 hours.
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

My vote: +1

Regards,
Jayush Luniya
Apache Ambari 2.4.X Release Manager

Re: Ambari 2.5.x Release Management

[Sumit Mohanty]

+1.

Thanks for offering, Aravindan.

From: Aravindan Vijayan 
Sent: Wednesday, November 16, 2016 4:14 PM
To: dev@ambari.apache.org
Subject: Ambari 2.5.x Release Management

Hi all,

I would like to volunteer to be the Release Manager for the upcoming Ambari 
2.5.x release.

--
Thanks and Regards,
Aravindan Vijayan

Re: Ambari DB Consistency Check Failed to Complete

[Sumit Mohanty]

?Actually, lets open a JIRA.


From: Sumit Mohanty
Sent: Wednesday, October 26, 2016 12:20 PM
To: Ambari
Cc: sumit.moha...@gmail.com
Subject: Re: Ambari DB Consistency Check Failed to Complete


Hi Mithun,


It could be just me but the attachment is missing.


Can you forward it to sumit.moha...@gmail.com?


thanks

Sumit


From: Mithun Mathew <mithm...@apache.org>
Sent: Wednesday, October 26, 2016 12:17 PM
To: Ambari
Subject: Fwd: Ambari DB Consistency Check Failed to Complete

Hi Ambari-devs


I've been hitting this failure while running ambari-server start on
?a ?
branch-2.4
? build?
:
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
No errors were found.
ERROR: Exiting with exit code 1.
REASON: Database check failed to complete. Please check 
/var/log/ambari-server/ambari-server.log and 
/var/log/ambari-server/ambari-server-check-database.log for more information.

?I see that this started happening on branch-2.4 builds fairly recently. The 
same happened with my trunk build.
D
?oes ?
?
any
?one have?
thoughts on how to resolve this issue?

The log says Error preallocating sequence numbers
I've attached the
? log?
file ?/var/log/ambari-server/ambari-server-check-database.log


?Regards
Matt?

Re: Ambari DB Consistency Check Failed to Complete

[Sumit Mohanty]

Hi Mithun,


It could be just me but the attachment is missing.


Can you forward it to sumit.moha...@gmail.com?


thanks

Sumit


From: Mithun Mathew 
Sent: Wednesday, October 26, 2016 12:17 PM
To: Ambari
Subject: Fwd: Ambari DB Consistency Check Failed to Complete

Hi Ambari-devs


I've been hitting this failure while running ambari-server start on
?a ?
branch-2.4
? build?
:
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
No errors were found.
ERROR: Exiting with exit code 1.
REASON: Database check failed to complete. Please check 
/var/log/ambari-server/ambari-server.log and 
/var/log/ambari-server/ambari-server-check-database.log for more information.

?I see that this started happening on branch-2.4 builds fairly recently. The 
same happened with my trunk build.
D
?oes ?
?
any
?one have?
thoughts on how to resolve this issue?

The log says Error preallocating sequence numbers
I've attached the
? log?
file ?/var/log/ambari-server/ambari-server-check-database.log


?Regards
Matt?

Re: [VOTE] Apache Ambari 2.4.1 RC1

[Sumit Mohanty]

+1

Verified checksum and the signature.

Successfully ran "mvn clean install" on a clean VM.

Thanks Jayush.

-Sumit

From: Yusaku Sako 
Sent: Saturday, September 10, 2016 3:39 PM
To: dev@ambari.apache.org
Subject: Re: [VOTE] Apache Ambari 2.4.1 RC1

+1.
Verified that the checksums/PGP signature are good.  Also made sure "mvn clean 
apache-rat:check" passes.

Yusaku



On 9/9/16, 2:05 PM, "Jayush Luniya"  wrote:

>Hello,
>
>I have created an apache-ambari-2.4.1 release candidate.
>
>GIT source tag (release-2.4.1-rc1) 
>https://git-wip-us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.4.1-rc1
>
>Staging site:  http://home.apache.org/~jluniya/apache-ambari-2.4.1-rc1/
>
>PGP release keys (signed using 
>CD23CAAE)
> http://pgp.mit.edu:11371/pks/lookup?op=vindex=0x4BC59EDACD23CAAE
>
>One can look into the issues fixed in this release at 
>https://issues.apache.org/jira/browse/AMBARI/fixforversion/12335513/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel
>
>Vote will be open for 72 hours.
>[ ] +1 approve
>[ ] +0 no opinion
>[ ] -1 disapprove (and reason why)
>
>My vote: +1
>
>Regards,
>Jayush Luniya
>Apache Ambari 2.4.X Release Manager

Another blocker for 2.4.0 (https://issues.apache.org/jira/browse/AMBARI-18219)

[Sumit Mohanty]

Hi all,


A simple fix to modify the call to stop oozie as the earlier mechanism some 
times failed to stop the server but did not report failure to Ambari resulting 
in zombie processes.


thanks

Sumit

Approval for AMBARI-18142

[Sumit Mohanty]

Hi Jayush,

Original fix for AMBARI-18142 caused a regression by removing an existing kinit 
command. 
Bikas has provided an addendum patch to fix the immediate regression. We plan 
to commit the changes.

Patch - 
https://issues.apache.org/jira/secure/attachment/12824418/AMBARI-18142.addendum.patch

Original patch - 
https://github.com/apache/ambari/commit/3e3641d8ee3f71feef200802cc19adf2235c8fc6

thanks
Sumit

Re: Reminder on Code Review Best Practices

[Sumit Mohanty]

A large number of changes we see towards the end of the release cycle is 
modification to config properties and are also prime candidates to cause 
regression. To that end, a new page is added to the wiki - 
https://cwiki.apache.org/confluence/display/AMBARI/Adding+config+properties+to+service+definition
 to talk about what to look out for when making config property changes.

Hope this will help as a guideline to think through the scenarios that get 
impacted when a config property is added/modified/deleted.

Feel free to add to the above.

thanks
Sumit

From: Alejandro Fernandez 
Sent: Thursday, August 04, 2016 10:49 AM
To: dev@ambari.apache.org
Subject: Reminder on Code Review Best Practices

Hi all,

This is a friendly reminder on our code review best practices.
https://cwiki.apache.org/confluence/display/AMBARI/Code+Review+Guidelines

* Whenever possible, the reviewer should be the first to annotate the code 
review to make it easier for others to read, i.e., help them help you.
* For larger patches, please allow 24 hours in code review so other developers 
have gotten a chance to look at it, especially if they are in a different time 
zone.
* Include the right developers in the review, run git blame to see who last 
modified the changed files and take a look at the chart from the link above to 
include developers interested in that area.

Thank you,
Alejandro Fernandez

Re: Configuration files of optional components are still displayed on ambari ui

[Sumit Mohanty]

This is not a bug. Ambari manages config files as being part of a service as 
opposed to part of a component.

Get Outlook for iOS




On Wed, Jun 15, 2016 at 4:51 PM -0700, "Jeff Zhang" 
> wrote:


e.g. I didn't choose to install spark thrift server which is optional, but its 
configuration files are still displayed on the ambari ui. Is it expected or a 
bug ? If it is a bug, is there any way to remove the configuration files on 
ambari ui ?

--
Best Regards

Jeff Zhang

Re: Code Review groups

[Sumit Mohanty]

We can look into the already available component names in the JIRA for the 
initial list. 
We should not create fine-grained groups and aim to have at least 3-5 devs 
(more is better) in a single component/area.

Possible list:
ambari-web
ambari-views
ambari-server
ambari-agent
stacks-framework/extensibility
stack-definitions (this could break into separate services)
blueprints
alerts/metrics
logsearch
security/kerberos/ldap
stack-upgrade/RU/EU

Once the list is final lets make sure that the available list of components in 
the JIRA matches this list.

This is probably also a good opportunity to see if there are better 
alternatives to reviews.apache.org.

regards
Sumit

From: Jayush Luniya 
Sent: Thursday, June 02, 2016 1:47 PM
To: dev@ambari.apache.org
Subject: Re: Code Review groups

+1 on this


On 6/2/16, 1:44 PM, "Swapan Shridhar"  wrote:

>+1. Makes sense.
>
>Thanks.
>
>Regards,
>Swapan.
>
>
>
>
>
>
>
>
>
>On 6/2/16, 1:27 PM, "Robert Levas"  wrote:
>
>>Alejandro, I agree.  I just hope we (as a group) can manage the wiki
>>page without letting it get too stale over time.
>>
>>+1
>>
>>Rob
>>
>>
>>On 6/2/16, 12:55 PM, "Alejandro Fernandez" 
>>wrote:
>>
>>>Hi committers and contributors,
>>>
>>>I'm sure most of you have ran into this before; whenever I submit a
>>>code review I'm always curious to find out which reviewers I should
>>>include that are knowledgeable in that area.
>>>So I'll typically run git blame to find the last 2-3 people that worked
>>>on those files, which takes time and may include reviewers no longer
>>>interested in that code area or miss reviewers that are interested.
>>>I want to propose a wiki where developers sign up to be reviewers for a
>>>particular section, could be a feature, directory, etc.
>>>
>>>Thoughts?
>>>
>>>This allows developers to opt-in to areas of interest (even outside of
>>>their current expertise), should produce better code reviews, and make
>>>it easier for new contributors to find the right people.
>>>
>>>Thank you,
>>>Alejandro Fernandez
>>>
>>

Another blocker for 2.2.2 (AMBARI-15854)

[Sumit Mohanty]

AMBARI-15854 is a candidate fix for 2.2.2 that I am planning to commit. This is 
an isolated change to Slider View to set stack version so that the correct 
version of dependency tarball (includes Slider jars) is picked up when creating 
slider apps.


Without the change, Slider View creates a tar based on its own lib directory 
that includes jars that may conflict with libraries needed by the apps created 
by the Slider view. issues due to conflicting versions of jars are not easy to 
debug.


https://issues.apache.org/jira/browse/AMBARI-15854?


thanks

Sumit

[ANNOUNCE] Ajit Kumar as a new Apache Ambari Committer

[Sumit Mohanty]

It is my pleasure to announce that Ajit Kumar has become a committer for Apache 
Ambari. Welcome Ajit.


thanks

-Sumit

Re: Apache Ambari-2.2.2 release planning

[Sumit Mohanty]

+1. Thanks Srimanth.

Sent from Outlook Mobile




On Sun, Apr 3, 2016 at 1:45 PM -0700, "Srimanth Gunturi" 
> wrote:

?Hello,

With work on several important new features being committed and becoming 
stable, I think its time for us to start the release planning process for the 
??release of Ambari-2.2.2.


This will help us stabilize the branch faster and provide users access to the 
features in a formal release.


Towards this goal I would like to volunteer for the release manager role.?

Best regards,

Srimanth

Re: Support for Multiple Nameservices

[Sumit Mohanty]

A fall back logic will be good, perhaps required. Any documentation around 
dfs.namenode.shared.edits.dir that I can refer to in terms of its content and 
reliability of it?

My original thoughts were:

* For fresh NN/HA enablements and fresh blueprint deployments, it will work 
fine as Amabri can add and rely on the existence of "dfs.internal.nameservice"
* For existing clusters, during Ambari upgrade it can auto copy value from 
dfs.nameservice to dfs.internal.nameservice if the value not a comma separated 
list
** If the value is a comma separated list then we already a problem - so those 
deployments need to fix it manually after upgrade as Ambari can now understand 
dfs.internal.nameservice
* We may still need a fall back logic
** For fallback, I was thinking of falling back to the current logic (mostly, 
as I did not know about dfs.namenode.shared.edits.dir)
** Fallback should not be needed if we can fix the configs automatically during 
Ambari Upgrade

-Sumit

From: Henning Kropp <hkr...@microlution.de>
Sent: Friday, April 01, 2016 12:49 PM
To: dev@ambari.apache.org
Subject: Re: Support for Multiple Nameservices

Hi Sumit,

very interesting. I will certainly explore the possibility of using
dfs.inernal.nameservices (leaving federation for another day :)

You think a fallback to dfs.nameservices (the current behaivour), if
internal nameservices is not set, is needed?

Regards,
Henning


Am 01/04/16 um 20:30 schrieb Sumit Mohanty:
> Hi Henning.
>
> Its a very good problem that you brought up.
>
> https://community.hortonworks.com/questions/8989/how-to-use-name-service-id-between-to-clusters.html
>  talks about "dfs.internal.nameservices" and if Ambari populates this and 
> uses it for all its internal scripts/alerts then that may be a solution.
>
> This, as you notice also says "nameservices" (plural) which points to HDFS 
> federation. That is probably a problem for a different day.
> In any case, in absence of HDFS federation, we can have Ambari refer to 
> "dfs.internal.nameservices" for its own use. The requirement will be to have 
> NN HA wizard and Blueprint deployment populate this property.
>
> AMBARI-15615 modified the NN HA wizard to populate the property. But as far 
> as I can see no other changes have happened.
>
> I was not aware of hdfs_site['dfs.namenode.shared.edits.dir'] as a possible 
> solution but talking to few folks from HDFS it seems that populating/using 
> "dfs.internal.nameservices" will be more desirable.
>
> Do you want to explore the possibility of using "dfs.internal.nameservices"?
>
> thanks
> Sumit
> 
> From: Henning Kropp <hkr...@microlution.de>
> Sent: Friday, April 01, 2016 10:38 AM
> To: dev@ambari.apache.org
> Subject: Support for Multiple Nameservices
>
> Hi,
>
> we noticed that throughout Ambari hdfs_site['dfs.nameservices'] is
> treated as a string denoting just one nameservice. It is possible to
> configure multiple namservices for example to support seamless distcp
> between two HA clusters. In general this makes it possible to use
> multiple nameservices wiht hdfs.
>
> This makes dfs.nameservices a comma separated string holding mulitple
> namservices. When doing this with the current Ambari release this leads
> to multiple problems. One is that Ambari marks the restart of HDFS as
> failed, even though the restart was succesful. The reason for that is
> that hdfs_resources.py is not working this a comma separated list of
> nameservices AMBARI-15506.
>
> We created an umbrella JIRA to track the other issues AMBARI-15507.
> Problems with Blueprint install, because the DN's were registered with
> the other cluster AMBARI-15509. Web alerting for NNs does not work
> AMBARI-15508.
>
> There might be other places where dfs.namservices is treated just a
> string? How can web alerting be refactored to work with multiple
> nameservices?
>
> Also I would appreciate any feedback about the function to resolve the
> current nameservice for the current cluster.
>
> For AMBARI-15506 I defined the following nameservice resolution:
> 1. split names by ','
> 2. for all services check if string is also contained in
> dfs.namenode.shared.edits.dir . Typically this are
> jounalnode1,journalnode2,journalnode3:port/servicename. Here it would
> probably be better to verify the name with fs.defaultFS, but this is
> part of core-site not hdfs-site, which would add a separate dependency.
> For namenode_ha_utils.py for me this seemed like an issue, because
> refactoring all the python scripts to also include the core-site seemed
> much more involved.
> 3. A default fallback the first string in the list is used as the
> nameservice.
>
> Thanks and regards,
> Henning

Re: Support for Multiple Nameservices

[Sumit Mohanty]

Hi Henning.

Its a very good problem that you brought up.

https://community.hortonworks.com/questions/8989/how-to-use-name-service-id-between-to-clusters.html
 talks about "dfs.internal.nameservices" and if Ambari populates this and uses 
it for all its internal scripts/alerts then that may be a solution.

This, as you notice also says "nameservices" (plural) which points to HDFS 
federation. That is probably a problem for a different day. 
In any case, in absence of HDFS federation, we can have Ambari refer to 
"dfs.internal.nameservices" for its own use. The requirement will be to have NN 
HA wizard and Blueprint deployment populate this property.

AMBARI-15615 modified the NN HA wizard to populate the property. But as far as 
I can see no other changes have happened.

I was not aware of hdfs_site['dfs.namenode.shared.edits.dir'] as a possible 
solution but talking to few folks from HDFS it seems that populating/using 
"dfs.internal.nameservices" will be more desirable.

Do you want to explore the possibility of using "dfs.internal.nameservices"?

thanks
Sumit

From: Henning Kropp 
Sent: Friday, April 01, 2016 10:38 AM
To: dev@ambari.apache.org
Subject: Support for Multiple Nameservices

Hi,

we noticed that throughout Ambari hdfs_site['dfs.nameservices'] is
treated as a string denoting just one nameservice. It is possible to
configure multiple namservices for example to support seamless distcp
between two HA clusters. In general this makes it possible to use
multiple nameservices wiht hdfs.

This makes dfs.nameservices a comma separated string holding mulitple
namservices. When doing this with the current Ambari release this leads
to multiple problems. One is that Ambari marks the restart of HDFS as
failed, even though the restart was succesful. The reason for that is
that hdfs_resources.py is not working this a comma separated list of
nameservices AMBARI-15506.

We created an umbrella JIRA to track the other issues AMBARI-15507.
Problems with Blueprint install, because the DN's were registered with
the other cluster AMBARI-15509. Web alerting for NNs does not work
AMBARI-15508.

There might be other places where dfs.namservices is treated just a
string? How can web alerting be refactored to work with multiple
nameservices?

Also I would appreciate any feedback about the function to resolve the
current nameservice for the current cluster.

For AMBARI-15506 I defined the following nameservice resolution:
1. split names by ','
2. for all services check if string is also contained in
dfs.namenode.shared.edits.dir . Typically this are
jounalnode1,journalnode2,journalnode3:port/servicename. Here it would
probably be better to verify the name with fs.defaultFS, but this is
part of core-site not hdfs-site, which would add a separate dependency.
For namenode_ha_utils.py for me this seemed like an issue, because
refactoring all the python scripts to also include the core-site seemed
much more involved.
3. A default fallback the first string in the list is used as the
nameservice.

Thanks and regards,
Henning

Re: Apache Ambari-2.next release planning

[Sumit Mohanty]

+1. Thanks Jayush.

From: Jaimin Jetly <jai...@hortonworks.com>
Sent: Tuesday, March 22, 2016 4:50 PM
To: dev@ambari.apache.org
Subject: Re: Apache Ambari-2.next release planning

+1 for Jayush as Release Manager for the next 2.x release.
Thanks Jayush for volunteering.

-- Thanks
Jaimin

From: Jayush Luniya <jlun...@hortonworks.com>
Sent: Tuesday, March 22, 2016 4:47 PM
To: dev@ambari.apache.org
Subject: Re: Apache Ambari-2.next release planning

Hi all,

I would like to volunteer to handle the Release Manager role for the next
2.x release.

Regards
Jayush


On 3/22/16, 4:44 PM, "Sumit Mohanty" <smoha...@hortonworks.com> wrote:

>Folks,
>
>I feel, its time for us to start the release planning for the next 2.x
>release for Ambari.
>
>This is little sooner than what we have done for prior releases but with
>several big ticket items being worked on trunk, it makes sense to start
>sooner and decide on a date for creating the branch and then a duration
>for stabilizing features on the branch.
>
>This will free up the trunk for features being planned 1-2 releases out
>and help us stabilize the branch faster.
>
>Any volunteers for the release manager role?
>
>thanks
>Sumit
>

Apache Ambari-2.next release planning

[Sumit Mohanty]

Folks,

I feel, its time for us to start the release planning for the next 2.x release 
for Ambari.

This is little sooner than what we have done for prior releases but with 
several big ticket items being worked on trunk, it makes sense to start sooner 
and decide on a date for creating the branch and then a duration for 
stabilizing features on the branch.

This will free up the trunk for features being planned 1-2 releases out and 
help us stabilize the branch faster.

Any volunteers for the release manager role?

thanks
Sumit

Re: Please subscribe to reviews list if you use the Review Board

[Sumit Mohanty]

Hi,

I have been noticing a large number of emails asking review emails to be 
moderated. Thats the default setting if you are not subscribed to the mailing 
list. So its possible that folks may be missing review emails if no one 
"accepts" the moderate email (folks are trying to approve these emails ASAP but 
there will be misses).

As Yusaku pointed out, pls. go ahead and send a mail to 
reviews-subscr...@ambari.apache.org to subscribe.

thanks
Sumit

From: Yusaku Sako 
Sent: Tuesday, March 08, 2016 12:29 PM
To: dev@ambari.apache.org
Subject: Please subscribe to reviews list if you use the Review Board

Hi all,

Please subscribe to the revi...@ambari.apache.org mailing list if you are using 
ReviewBoard.
If you do not belong to this list and make a Review Board request against 
"ambari", your email will not be received by the subscribers of 
revi...@ambari.apache.org.

Yusaku

[jira] [Commented] (AMBARI-15328) Exception on manual start of AMS is misleading

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184147#comment-15184147
 ] 

Sumit Mohanty commented on AMBARI-15328:


+1

> Exception on manual start of AMS is misleading
> --
>
> Key: AMBARI-15328
> URL: https://issues.apache.org/jira/browse/AMBARI-15328
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-metrics
>Affects Versions: 2.2.0
>Reporter: Aravindan Vijayan
>Assignee: Aravindan Vijayan
> Fix For: 2.2.2
>
> Attachments: AMBARI-15328.patch
>
>
> On manual start of AMS in embedded mode, following stderr messages are logged 
> to console which is misleading since the start succeeds when the server is up.
> {code}
> 2016-01-27 19:15:21,487 ERROR [main] zookeeper.ZooKeeperWatcher: 
> hconnection-0x799ed4e80x0, quorum=localhost:61181, 
> baseZNode=/ams-hbase-unsecure Received unexpected KeeperException, 
> re-throwing exception
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode 
> = ConnectionLoss for /ams-hbase-unsecure
>   at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>   at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>   at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
>   at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:221)
>   at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:417)
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:902)
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.access$400(ConnectionManager.java:552)
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1490)
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1531)
> {code}
> We already log this to a file so not need to print stderr to console.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15304) HiveServerInteractive. Fix the 'hive.server2.thrift.port' and 'hive.server2.thrift.http.port' values.

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15304:
---
Attachment: AMBARI-15304.patch

> HiveServerInteractive. Fix the 'hive.server2.thrift.port' and 
> 'hive.server2.thrift.http.port' values.
> -
>
> Key: AMBARI-15304
> URL: https://issues.apache.org/jira/browse/AMBARI-15304
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.3.0
>Reporter: Swapan Shridhar
>Assignee: Swapan Shridhar
> Fix For: 2.4.0
>
> Attachments: AMBARI-15304.patch
>
>
> The values are follows:
> 'hive.server2.thrift.port' -> 10500
> 'hive.server2.thrift.http.port' -> 10501



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 44403: HiveServerInteractive. Correcting the 'hive.server2.thrift.port' and 'hive.server2.thrift.http.port' values.

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44403/#review122134
---


Ship it!




Ship It!

- Sumit Mohanty


On March 4, 2016, 7:06 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44403/
> ---
> 
> (Updated March 4, 2016, 7:06 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Sumit Mohanty, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15304
> https://issues.apache.org/jira/browse/AMBARI-15304
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - The values are following:
> 'hive.server2.thrift.port' -> 10500
> 'hive.server2.thrift.http.port' -> 10501
> 
> 
> - The values are taken as suggested by Sid in tracker : 
> https://hortonworks.jira.com/browse/BUG-50819
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/HIVE/configuration/hive-interactive-site.xml
>  76a9724 
> 
> Diff: https://reviews.apache.org/r/44403/diff/
> 
> 
> Testing
> ---
> 
> Not required at this point.
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 44397: New Alerts Do Not Honor Existing Maintenance Mode Setting

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44397/#review122112
---




ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java
 (line 248)
<https://reviews.apache.org/r/44397/#comment184021>

We did not have a helper method yet for this? This is useful - I will let 
Nahappan know. I think he needed something like this for something he is 
working on.


- Sumit Mohanty


On March 4, 2016, 5:46 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44397/
> ---
> 
> (Updated March 4, 2016, 5:46 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush 
> Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15303
> https://issues.apache.org/jira/browse/AMBARI-15303
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Alerts "suppress" maintenance mode by indicating a {{maintenance_state}} 
> attribute in addition to the actual state which is being reported:
> 
> {code}
>   "Alert": {
> "cluster_name": "c1",
> "component_name": "METRICS_COLLECTOR",
> "definition_id": 43,
> "definition_name": "ams_metrics_collector_process",
> "host_name": "c6401.ambari.apache.org",
> "id": 28,
> "instance": null,
> "label": "Metrics Collector Process",
> "latest_timestamp": 1457108946118,
> "maintenance_state": "ON",
> "original_timestamp": 1457108646099,
> "scope": "ANY",
> "service_name": "AMBARI_METRICS",
> "state": "CRITICAL",
> "text": "Connection failed: [Errno 111] Connection refused to 
> c6401.ambari.apache.org"
>   }
> {code}
> 
> When a host/service/component is placed into MM, the database is updated so 
> that all {{alert_current}} rows which are affected have their MM updated as 
> well.
> 
> However, this fails under two scenarios:
> - The alert hasn't been received yet in a brand new cluster
> - The alert definition was disabled, which removed all current alerts. Then, 
> it was re-enabled.
> 
> In both cases, when constructing a new {{AlertCurrentEntity}}, we need to 
> calculate the correct maintenance state.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java
>  cd49e76 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java
>  9bbfe37 
> 
> Diff: https://reviews.apache.org/r/44397/diff/
> 
> 
> Testing
> ---
> 
> PENDING: Writing UTs and running tests now... 
> 
> Verified fix in an existing cluster by disabling alerts, then re-enabling 
> them on a MM component with an active alert.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 44397: New Alerts Do Not Honor Existing Maintenance Mode Setting

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44397/#review122113
---


Ship it!




Ship It!

- Sumit Mohanty


On March 4, 2016, 5:46 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44397/
> ---
> 
> (Updated March 4, 2016, 5:46 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush 
> Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15303
> https://issues.apache.org/jira/browse/AMBARI-15303
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Alerts "suppress" maintenance mode by indicating a {{maintenance_state}} 
> attribute in addition to the actual state which is being reported:
> 
> {code}
>   "Alert": {
> "cluster_name": "c1",
> "component_name": "METRICS_COLLECTOR",
> "definition_id": 43,
> "definition_name": "ams_metrics_collector_process",
> "host_name": "c6401.ambari.apache.org",
> "id": 28,
> "instance": null,
> "label": "Metrics Collector Process",
> "latest_timestamp": 1457108946118,
> "maintenance_state": "ON",
> "original_timestamp": 1457108646099,
> "scope": "ANY",
> "service_name": "AMBARI_METRICS",
> "state": "CRITICAL",
> "text": "Connection failed: [Errno 111] Connection refused to 
> c6401.ambari.apache.org"
>   }
> {code}
> 
> When a host/service/component is placed into MM, the database is updated so 
> that all {{alert_current}} rows which are affected have their MM updated as 
> well.
> 
> However, this fails under two scenarios:
> - The alert hasn't been received yet in a brand new cluster
> - The alert definition was disabled, which removed all current alerts. Then, 
> it was re-enabled.
> 
> In both cases, when constructing a new {{AlertCurrentEntity}}, we need to 
> calculate the correct maintenance state.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java
>  cd49e76 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java
>  9bbfe37 
> 
> Diff: https://reviews.apache.org/r/44397/diff/
> 
> 
> Testing
> ---
> 
> PENDING: Writing UTs and running tests now... 
> 
> Verified fix in an existing cluster by disabling alerts, then re-enabling 
> them on a MM component with an active alert.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 44272: Install & Manage Zeppelin with Ambari

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44272/#review121940
---




ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/master.py
 (line 161)
<https://reviews.apache.org/r/44272/#comment183786>

For example I see these in 
/usr/hdp/2.5.0.0-56/zeppelin/lib/interpreter/spark/dep/zeppelin-spark-dependencies-0.6.0.2.5.0.0-56.jar.
 So if we need these in hdfs we can copy from this location.


- Sumit Mohanty


On March 2, 2016, 4:54 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44272/
> ---
> 
> (Updated March 2, 2016, 4:54 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15265
> https://issues.apache.org/jira/browse/AMBARI-15265
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - add service defnition for Apache Zeppelin
> - add support for Zeppelin view in Ambari
> - add alert config
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/alerts.json 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/configuration/zeppelin-ambari-config.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/configuration/zeppelin-config.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/configuration/zeppelin-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/kerberos.json 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/alert_check_zeppelin.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/master.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/setup_snapshot.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/status_params.py
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ZEPPELIN/metainfo.xml
>  PRE-CREATION 
>   contrib/views/pom.xml 627552a 
>   contrib/views/zeppelin/pom.xml PRE-CREATION 
>   
> contrib/views/zeppelin/src/main/java/org/apache/ambari/view/zeppelin/ZeppelinServlet.java
>  PRE-CREATION 
>   contrib/views/zeppelin/src/main/resources/WEB-INF/index.jsp PRE-CREATION 
>   contrib/views/zeppelin/src/main/resources/WEB-INF/web.xml PRE-CREATION 
>   contrib/views/zeppelin/src/main/resources/view.xml PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/44272/diff/
> 
> 
> Testing
> ---
> 
> - manual testing in vm
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

[jira] [Commented] (AMBARI-15237) Create Grafana user through Ambari and add user to Grafana

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177187#comment-15177187
 ] 

Sumit Mohanty commented on AMBARI-15237:


LGTM, for the additional patch - +1

> Create Grafana user through Ambari and add user to Grafana
> --
>
> Key: AMBARI-15237
> URL: https://issues.apache.org/jira/browse/AMBARI-15237
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-metrics
>Affects Versions: 2.2.2
>Reporter: Siddharth Wagle
>Assignee: Siddharth Wagle
> Fix For: 2.2.2
>
> Attachments: AMBARI-15237-1.patch, AMBARI-15237.patch
>
>
> - Grafana manages passwords and usernames with its own db.
> - Ambari should create initial user and password
> - User should change the password in Grafana and make sure the configs in 
> Ambari are updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (AMBARI-15237) Create Grafana user through Ambari and add user to Grafana

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177187#comment-15177187
 ] 

Sumit Mohanty edited comment on AMBARI-15237 at 3/3/16 5:12 AM:


LGTM, for the additional patch +1


was (Author: sumitmohanty):
LGTM, for the additional patch - +1

> Create Grafana user through Ambari and add user to Grafana
> --
>
> Key: AMBARI-15237
> URL: https://issues.apache.org/jira/browse/AMBARI-15237
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-metrics
>Affects Versions: 2.2.2
>Reporter: Siddharth Wagle
>Assignee: Siddharth Wagle
> Fix For: 2.2.2
>
> Attachments: AMBARI-15237-1.patch, AMBARI-15237.patch
>
>
> - Grafana manages passwords and usernames with its own db.
> - Ambari should create initial user and password
> - User should change the password in Grafana and make sure the configs in 
> Ambari are updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 44272: Install & Manage Zeppelin with Ambari

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44272/#review121773
---




ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/alert_check_zeppelin.py
 (line 4)
<https://reviews.apache.org/r/44272/#comment183602>

This alert is just checking for Zeppelin PID. This is not needed. status() 
command is enough. If Zepplin has a listen port then we need alerts that go 
against the port - see WEB or PORT type alerts in HBase and HDFS



ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/master.py
 (line 145)
<https://reviews.apache.org/r/44272/#comment183603>

Delete this line.



ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/master.py
 (line 157)
<https://reviews.apache.org/r/44272/#comment183604>

Why OSX?



ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/master.py
 (line 161)
<https://reviews.apache.org/r/44272/#comment183605>

How did these files reach /tmp folder?



ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/master.py
 (line 165)
<https://reviews.apache.org/r/44272/#comment183606>

Need to use HdfsResource function call. We cannot rely on hdfs calls 
directly through Execute.



ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/master.py
 (line 171)
<https://reviews.apache.org/r/44272/#comment183607>

Remove all echo statements. If you need to log then use Logger.info(). 
Anyway, we should not need to read/print pid value.



ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/master.py
 (line 186)
<https://reviews.apache.org/r/44272/#comment183608>

Does the zeppelin pid file name change on every start? Could we create the 
name statically in status_params.py and then use it everywhere.



ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/setup_snapshot.sh
 (line 1)
<https://reviews.apache.org/r/44272/#comment183600>

When is this sh script used? Is it needed for Zeppelin start or install?



ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/setup_snapshot.sh
 (line 68)
<https://reviews.apache.org/r/44272/#comment183601>

If Zeppelin needs hive-site then we should use the library in 
resource_management module to do it.



ambari-server/src/main/resources/stacks/HDP/2.5/services/ZEPPELIN/metainfo.xml 
(line 24)
<https://reviews.apache.org/r/44272/#comment183598>

It should be 0.6.0.2.5.0.0



ambari-server/src/main/resources/stacks/HDP/2.5/services/ZEPPELIN/metainfo.xml 
(line 26)
<https://reviews.apache.org/r/44272/#comment183599>

You need to add 2.5 specific package names such as zeppelin_2_5_* and 
zeppelin-2-5-.*.

See 
ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/metainfo.xml


- Sumit Mohanty


On March 2, 2016, 4:54 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44272/
> ---
> 
> (Updated March 2, 2016, 4:54 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15265
> https://issues.apache.org/jira/browse/AMBARI-15265
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - add service defnition for Apache Zeppelin
> - add support for Zeppelin view in Ambari
> - add alert config
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/alerts.json 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/configuration/zeppelin-ambari-config.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/configuration/zeppelin-config.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/configuration/zeppelin-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/kerberos.json 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/alert_check_zeppelin.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0/package/scripts/master.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-s

[ANNOUNCE] Sebastian Toader as a new Apache Ambari Committer

[Sumit Mohanty]

It is my pleasure to announce that Sebastian Toader has become a committer for 
Apache Ambari.

Welcome Sebastian.

thanks
Sumit

Re: Review Request 44283: AMBARI-15270 : Make AMS HBase initialization check optional through a config

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44283/#review121710
---


Ship it!




Ship It!

- Sumit Mohanty


On March 2, 2016, 8:18 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44283/
> ---
> 
> (Updated March 2, 2016, 8:18 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15270
> https://issues.apache.org/jira/browse/AMBARI-15270
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Making the AMS HBase table initialization check during AMS startup optional 
> through a config, as a trade-off between quick startup and reliable startup 
> command success notification.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector
>  f9d0e99 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml
>  9b4689a 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  29a09a8 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  294c025 
> 
> Diff: https://reviews.apache.org/r/44283/diff/
> 
> 
> Testing
> ---
> 
> Manual testing done.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Who reviews ambari-agent patches?

[Sumit Mohanty]

You can add DmytriL and AndrewO (in CC).

-Sumit

From: Greg Hill 
Sent: Wednesday, March 02, 2016 11:57 AM
To: Greg Hill; Ambari
Subject: Who reviews ambari-agent patches?

This is a behavior that has broken us multiple times.  Can I get some eyes on 
this patch?

Greg

From: Greg >
Reply-To: Greg >
Date: Wednesday, March 2, 2016 at 1:54 PM
To: Greg >, Ambari 
>
Subject: Review Request 44285: AMBARI-15266 - Add configurable download retries 
for cache misses

This is an automatically generated e-mail. To reply, visit: 
https://reviews.apache.org/r/44285/

Review request for Ambari.
By Greg Hill.
Bugs: AMBARI-15266
Repository: ambari
Description

Adds configuration options to tell ambari-agent to retry attempts to download 
the custom script cache from ambari-server


Testing

New unit tests cover the behavior.


Diffs

  *   ambari-agent/conf/unix/ambari-agent.ini (80afdb2)
  *   ambari-agent/conf/windows/ambari-agent.ini (1b24f25)
  *   ambari-agent/src/main/python/ambari_agent/AmbariConfig.py (2c82ca5)
  *   ambari-agent/src/main/python/ambari_agent/FileCache.py (b7c5dee)
  *   ambari-agent/src/test/python/ambari_agent/TestFileCache.py (5933daa)

View Diff

[jira] [Commented] (AMBARI-13216) Add a "Add" button to Repo management UI.

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176179#comment-15176179
 ] 

Sumit Mohanty commented on AMBARI-13216:


[~mithmatt] can you add me and [~u39kun] to the review request? This is a great 
enhancement.

> Add a "Add" button to Repo management UI.
> -
>
> Key: AMBARI-13216
> URL: https://issues.apache.org/jira/browse/AMBARI-13216
> Project: Ambari
>  Issue Type: Bug
>Reporter: jun aoki
>Assignee: Matt
> Fix For: 2.2.2
>
> Attachments: AMBARI-13216-branch-2.2-orig.patch, 
> Ambariaddrepositoryfeaturedesigndocument_v1.pdf, Untitled.png, 
> screenshot-1.png, screenshot-2.png
>
>
> From UI, there is no way to add a new repo URL. (You can modify but not add). 
> This is a blocker when a service plugin is added to ambari-server.
> This maybe depends on https://issues.apache.org/jira/browse/AMBARI-13215



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 44270: Python UT fail on trunk

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44270/#review121666
---


Ship it!




Ship It!

- Sumit Mohanty


On March 2, 2016, 4:15 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44270/
> ---
> 
> (Updated March 2, 2016, 4:15 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-15262
> https://issues.apache.org/jira/browse/AMBARI-15262
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> <https://builds.apache.org/job/Ambari-trunk-Commit/4416/consoleFull>
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/dbConfiguration_linux.py 
> 1690708 
>   ambari-server/src/test/python/TestAmbariServer.py 901867c 
> 
> Diff: https://reviews.apache.org/r/44270/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

[jira] [Commented] (AMBARI-15238) Deploying AMS datasource and default dashboards sometimes fails during cluster install

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173094#comment-15173094
 ] 

Sumit Mohanty commented on AMBARI-15238:


LGTM, +1

> Deploying AMS datasource and default dashboards sometimes fails during 
> cluster install
> --
>
> Key: AMBARI-15238
> URL: https://issues.apache.org/jira/browse/AMBARI-15238
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-metrics
>Affects Versions: 2.2.2
>Reporter: Siddharth Wagle
>Assignee: Siddharth Wagle
> Fix For: 2.2.2
>
> Attachments: AMBARI-15238.patch
>
>
> Exception trace:
> {code}
> stderr:
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana.py",
>  line 70, in 
> AmsGrafana().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 219, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana.py",
>  line 51, in start
> create_ams_datasource()
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py",
>  line 143, in create_ams_datasource
> response = perform_grafana_get_call(GRAFANA_DATASOURCE_URL, server)
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py",
>  line 50, in perform_grafana_get_call
> conn.request("GET", url)
>   File "/usr/lib64/python2.6/httplib.py", line 936, in request
> self._send_request(method, url, body, headers)
>   File "/usr/lib64/python2.6/httplib.py", line 973, in _send_request
> self.endheaders()
>   File "/usr/lib64/python2.6/httplib.py", line 930, in endheaders
> self._send_output()
>   File "/usr/lib64/python2.6/httplib.py", line 802, in _send_output
> self.send(msg)
>   File "/usr/lib64/python2.6/httplib.py", line 761, in send
> self.connect()
>   File "/usr/lib64/python2.6/httplib.py", line 742, in connect
> self.timeout)
>   File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
> raise error, msg
> socket.error: [Errno 111] Connection refused
>  stdout:
> 2016-02-29 18:49:46,185 - The hadoop conf dir 
> /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for 
> version 2.4.0.0-169
> 2016-02-29 18:49:46,185 - Checking if need to create versioned conf dir 
> /etc/hadoop/2.4.0.0-169/0
> 2016-02-29 18:49:46,186 - call['conf-select create-conf-dir --package hadoop 
> --stack-version 2.4.0.0-169 --conf-version 0'] {'logoutput': False, 'sudo': 
> True, 'quiet': False, 'stderr': -1}
> 2016-02-29 18:49:46,239 - call returned (1, '/etc/hadoop/2.4.0.0-169/0 exist 
> already', '')
> 2016-02-29 18:49:46,240 - checked_call['conf-select set-conf-dir --package 
> hadoop --stack-version 2.4.0.0-169 --conf-version 0'] {'logoutput': False, 
> 'sudo': True, 'quiet': False}
> 2016-02-29 18:49:46,296 - checked_call returned (0, 
> '/usr/hdp/2.4.0.0-169/hadoop/conf -> /etc/hadoop/2.4.0.0-169/0')
> 2016-02-29 18:49:46,296 - Ensuring that hadoop has the correct symlink 
> structure
> 2016-02-29 18:49:46,296 - Using hadoop conf dir: 
> /usr/hdp/current/hadoop-client/conf
> 2016-02-29 18:49:46,488 - The hadoop conf dir 
> /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for 
> version 2.4.0.0-169
> 2016-02-29 18:49:46,488 - Checking if need to create versioned conf dir 
> /etc/hadoop/2.4.0.0-169/0
> 2016-02-29 18:49:46,488 - call['conf-select create-conf-dir --package hadoop 
> --stack-version 2.4.0.0-169 --conf-version 0'] {'logoutput': False, 'sudo': 
> True, 'quiet': False, 'stderr': -1}
> 2016-02-29 18:49:46,514 - call returned (1, '/etc/hadoop/2.4.0.0-169/0 exist 
> already', '')
> 2016-02-29 18:49:46,515 - checked_call['conf-select set-conf-dir --package 
> hadoop --stack-version 2.4.0.0-169 --conf-version 0'] {'logoutput': False, 
> 'sudo': True, 'quiet': False}
> 2016-02-29 18:49:46,552 - checked_call returned (0, 
> '/usr/hdp/2.4.0.0-169/hadoop/conf -> /etc/hadoop/2.4.0.0-169/0')
> 2016-02-29 18:49:46,553 - Ensuring that hadoop has the correct symlink 
> structure
> 2016-02-29 18:49:46,553 - Using hadoop conf dir: 
> /usr/hdp/current/hadoop-client/conf
&g

Re: Review Request 44194: Create Grafana user through Ambari and add user to Grafana

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44194/#review121375
---


Ship it!




Ship It!

- Sumit Mohanty


On March 1, 2016, 1:29 a.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44194/
> ---
> 
> (Updated March 1, 2016, 1:29 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Sumit Mohanty, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-15237
> https://issues.apache.org/jira/browse/AMBARI-15237
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Grafana manages passwords and usernames with its own db.
> - Ambari should create initial user and password
> - User should change the password in Grafana and make sure the configs in 
> Ambari are updated.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-grafana/conf/unix/ams-grafana.ini c0ccf1a 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-grafana-env.xml
>  de32ead 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-grafana-ini.xml
>  0a1ff32 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
>  3309878 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  3bd98a2 
>   
> ambari-server/src/test/python/stacks/2.0.6/AMBARI_METRICS/test_metrics_grafana.py
>  38ce90f 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 7626056 
>   ambari-web/app/data/HDP2/site_properties.js e061c96 
> 
> Diff: https://reviews.apache.org/r/44194/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> --
> Ran 244 tests in 6.872s
> 
> OK
> --
> Total run:891
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 44181: Don't copy fast-hdfs-resource.jar if sys_prepped

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44181/#review121318
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 29, 2016, 9:12 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44181/
> ---
> 
> (Updated Feb. 29, 2016, 9:12 p.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15233
> https://issues.apache.org/jira/browse/AMBARI-15233
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams_service.py
>  c9188c2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  e60a471 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
>  3ee2178 
> 
> Diff: https://reviews.apache.org/r/44181/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 44127: Add ability to Create pre-built Grafana dashboards

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44127/#review121064
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 27, 2016, 2:38 a.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44127/
> ---
> 
> (Updated Feb. 27, 2016, 2:38 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15214
> https://issues.apache.org/jira/browse/AMBARI-15214
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - On Grafana start check if dashboard exists and has same version as ambari
> - Add tags to dashboard to indicate _builtin_ and version of ambari
> - Re-create dashboard if there is a mismatch or newly added definition found
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hbase-master.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hbase-rs.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hdfs-dn.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hdfs-nn.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-apps.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-nm.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-overview.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-rm.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana.py
>  d96309c 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
>  37d403d 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  58ff71c 
> 
> Diff: https://reviews.apache.org/r/44127/diff/
> 
> 
> Testing
> ---
> 
> --
> Ran 244 tests in 6.817s
> 
> OK
> --
> Total run:891
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 44127: Add ability to Create pre-built Grafana dashboards

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44127/#review121062
---




ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hbase-master.json
 (line 2)
<https://reviews.apache.org/r/44127/#comment182655>

There are two with id = null



ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hdfs-nn.json
 (line 2)
<https://reviews.apache.org/r/44127/#comment182656>

second one with id = null



ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-nm.json
 (line 2)
<https://reviews.apache.org/r/44127/#comment182654>

These ids have to be unique - I assume.



ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
 (line 172)
<https://reviews.apache.org/r/44127/#comment182653>

Will it fail Grafana start?


- Sumit Mohanty


On Feb. 27, 2016, 2:38 a.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44127/
> ---
> 
> (Updated Feb. 27, 2016, 2:38 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15214
> https://issues.apache.org/jira/browse/AMBARI-15214
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - On Grafana start check if dashboard exists and has same version as ambari
> - Add tags to dashboard to indicate _builtin_ and version of ambari
> - Re-create dashboard if there is a mismatch or newly added definition found
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hbase-master.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hbase-rs.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hdfs-dn.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hdfs-nn.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-apps.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-nm.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-overview.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-rm.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana.py
>  d96309c 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
>  37d403d 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  58ff71c 
> 
> Diff: https://reviews.apache.org/r/44127/diff/
> 
> 
> Testing
> ---
> 
> --
> Ran 244 tests in 6.817s
> 
> OK
> --
> Total run:891
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

[jira] [Commented] (AMBARI-14951) Tez Ambari View: Add protocol configuration for YARN

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169994#comment-15169994
 ] 

Sumit Mohanty commented on AMBARI-14951:


LGTM but some parts I am not familiar with. +1 but let other folks review.
[~u39kun]/[~dbhowmick] can you also check?

> Tez Ambari View: Add protocol configuration for YARN
> 
>
> Key: AMBARI-14951
> URL: https://issues.apache.org/jira/browse/AMBARI-14951
> Project: Ambari
>  Issue Type: Bug
>Affects Versions: 2.2.2
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Fix For: 2.2.2
>
> Attachments: AMBARI-14951.1.patch, AMBARI-14951.2.patch, 
> AMBARI-14951.3.patch, AMBARI-14951.4.patch
>
>
> - Value set in yarn.http.policy is required for creating task attempt log 
> links, and we need a mechanism to get that configuration value at the UI side.
> - As host points to Ambari proxy, there is no way to know the protocol at 
> which yarn daemons are configured to run. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 44086: Registering 700 hosts gets stuck without timeout on 8 hosts

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44086/#review120889
---




ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSRunner.java 
(line 43)
<https://reviews.apache.org/r/44086/#comment182414>

Why 320?



ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSRunner.java 
(line 160)
<https://reviews.apache.org/r/44086/#comment182415>

Too frequent? Why not 1 minute?


- Sumit Mohanty


On Feb. 26, 2016, 5:07 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44086/
> ---
> 
> (Updated Feb. 26, 2016, 5:07 p.m.)
> 
> 
> Review request for Ambari and Myroslav Papirkovskyy.
> 
> 
> Bugs: AMBARI-15205
> https://issues.apache.org/jira/browse/AMBARI-15205
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When adding 700 hosts through the UI's add-host-wizard, all hosts successfully
> registered except for 8 of them. These 8 hosts were stuck in RUNNING state and
> would not timeout even after 20 minutes.
> 
> I had to kill the bootstrap process on ambari-server to make the operation to
> complete.  
> On my second attempt I was able to recreate the problem
> 
> Below is the output from `/api/v1/bootstrap/2` for the failing hosts
> (bootstrap response attached)
> 
> 
> 
> 
>   RUNNING
>   .
> 
> 
>   
> perf-d-511.c.pramod-thangali.internal
> RUNNING
> 
> ==
> Creating target directory...
> ==
> 
> Command start time 2016-02-17 18:55:54
> chmod: cannot access `/var/lib/ambari-agent/data': No such file or 
> directory
> 
> @@@
> @   WARNING: POSSIBLE DNS SPOOFING DETECTED!  @
> @@@
> The RSA host key for perf-d-511.c.pramod-thangali.internal has changed,
> and the key for the corresponding IP address 10.240.5.174
> has a different value. This could either mean that
> DNS SPOOFING is happening or the IP address for the host
> and its host key have changed at the same time.
> Offending key for IP in /root/.ssh/known_hosts:1418
> @@@
> @WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
> @@@
> IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
> Someone could be eavesdropping on you right now (man-in-the-middle 
> attack)!
> It is also possible that the RSA host key has just been changed.
> The fingerprint for the RSA key sent by the remote host is
> 66:45:6d:a4:fe:13:5d:47:ee:df:45:cc:32:dc:6a:2f.
> Please contact your system administrator.
> Add correct host key in /root/.ssh/known_hosts to get rid of this message.
> Offending key in /root/.ssh/known_hosts:1417
> Password authentication is disabled to avoid man-in-the-middle attacks.
> Keyboard-interactive authentication is disabled to avoid 
> man-in-the-middle attacks.
> Connection to perf-d-511.c.pramod-thangali.internal closed.
> SSH command execution finished
> host=perf-d-511.c.pramod-thangali.internal, exitcode=0
> Command end time 2016-02-17 18:55:55
> 
> ==
> Copying common functions script...
> ==
> 
> Command start time 2016-02-17 18:55:55
> 
> @@@
> @   WARNING: POSSIBLE DNS SPOOFING DETECTED!  @
> @@@
> The RSA host key for perf-d-511.c.pramod-thangali.internal has changed,
> and the key for the corresponding IP address 10.240.5.174
> has a different value. This could either mean that
> DNS SPOOFING is happening or the IP address for the host
> and its host key have changed at the same time.
> Offending key for IP in /root/.ssh/known_hosts:1418
> @@@
> @WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
> @@@
> IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
> Someone could be eavesdroppin

Re: Review Request 43967: Express Upgrade Stuck At Manual Prompt Due To HRC Status Calculation Cache Problem

[Sumit Mohanty]

> On Feb. 25, 2016, 12:07 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java,
> >  line 16
> > <https://reviews.apache.org/r/43967/diff/1/?file=1268481#file1268481line16>
> >
> > During RU/EU do we change the state of some HRC - force change to 
> > HOLDING or something like that. Will that need cache invalidation?
> 
> Jonathan Hurley wrote:
> As far as I can see, all updates to HRC statuses are done through this 
> class and all of the methods have the invalidation on them. If there's one 
> that you see which I don't, please point it out so I can take a look.

I was thinking about when upgrade is paused. But I assume it will end up in the 
merge/mergeAll call.


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43967/#review120615
---


On Feb. 24, 2016, 11:53 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43967/
> ---
> 
> (Updated Feb. 24, 2016, 11:53 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Sumit Mohanty, 
> Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15173
> https://issues.apache.org/jira/browse/AMBARI-15173
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Seen while performing an upgrade, it's possible that the status of a 
> request/stage does not match that of its tasks. Essentially, the task could 
> be {{HOLDING}} while the request is still {{IN_PROGRESS}}.
> 
> I believe that AMBARI-15011 is responsible for this issue. AMBARI-15011 
> introduced, among other things, a cache to the 
> {{HostRoleCommandStatusSummaryDTO}} which is a aggregation of the number of 
> tasks a stage has in each state (PENDING, HOLDING, etc).
> 
> This {{HostRoleCommandStatusSummaryDTO}} is used by {{CalculatedState}} to 
> calculate a stage's and request's status based on the tasks. 
> 
> The problem is that {{ServerActionExecutor}} is moving a tasks's state to 
> {{HOLDING}} (reflected in the database correctly) but the cache invalidation 
> happens inside the uncommitted transaction. This causes stale data to be 
> re-cached. So, when we go to calculate the request and state status, we get 
> {{IN_PROGRESS}} instead of {{HOLDING}}.
> 
> {code}
> {
>   "href": 
> "http://172.22.72.13:8080/api/v1/clusters/cl1/requests/61/stages/1?fields=*,tasks/*;,
>   "Stage": {
> "cluster_name": "cl1",
> "context": "Stop YARN Queues",
> "display_status": "IN_PROGRESS",
> "end_time": -1,
> "progress_percent": 35,
> "request_id": 61,
> "skippable": true,
> "stage_id": 1,
> "start_time": 1456227329191,
> "status": "IN_PROGRESS"
>   },
>   "tasks": [
> {
>   "href": 
> "http://172.22.72.13:8080/api/v1/clusters/cl1/requests/61/stages/1/tasks/754;,
>   "Tasks": {
> "attempt_cnt": 1,
> "cluster_name": "cl1",
> "command": "EXECUTE",
> "command_detail": "Before continuing, please stop all YARN queues. If 
> yarn-site's yarn.resourcemanager.work-preserving-recovery.enabled is set to 
> true, then you can skip this step since the clients will retry on their own.",
> "custom_command_name": 
> "org.apache.ambari.server.serveraction.upgrades.ManualStageAction",
> "end_time": -1,
> "error_log": "errors-754.txt",
> "exit_code": 0,
> "host_name": "os-r6-mkqzcs-c10tom21unsecha-6.novalocal",
> "id": 754,
> "output_log": "output-754.txt",
> "request_id": 61,
> "role": "AMBARI_SERVER_ACTION",
> "stage_id": 1,
> "start_time": 1456227329191,
> "status": "HOLDING",
> "stderr": "",
> "stdout": "",
> "structured_out": {}
>   }
> }
>   ]
> }
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/com/google/inject/persist/jpa/AmbariJpaPersistModule.java
>  4e4dd35 
>   
> ambari-server/src/main/java/org/apache/ambari/annotations/TransactionalLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/AmbariJpaLocalTxnInterceptor.java
>  6d7901c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/TransactionalLockInterceptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/TransactionalLocks.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  deca9b1 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 29ebc1f 
> 
> Diff: https://reviews.apache.org/r/43967/diff/
> 
> 
> Testing
> ---
> 
> Pending unit tests...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 43967: Express Upgrade Stuck At Manual Prompt Due To HRC Status Calculation Cache Problem

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43967/#review120617
---


Ship it!




LGTM, two questions

- Sumit Mohanty


On Feb. 24, 2016, 11:53 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43967/
> ---
> 
> (Updated Feb. 24, 2016, 11:53 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Sumit Mohanty, 
> Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15173
> https://issues.apache.org/jira/browse/AMBARI-15173
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Seen while performing an upgrade, it's possible that the status of a 
> request/stage does not match that of its tasks. Essentially, the task could 
> be {{HOLDING}} while the request is still {{IN_PROGRESS}}.
> 
> I believe that AMBARI-15011 is responsible for this issue. AMBARI-15011 
> introduced, among other things, a cache to the 
> {{HostRoleCommandStatusSummaryDTO}} which is a aggregation of the number of 
> tasks a stage has in each state (PENDING, HOLDING, etc).
> 
> This {{HostRoleCommandStatusSummaryDTO}} is used by {{CalculatedState}} to 
> calculate a stage's and request's status based on the tasks. 
> 
> The problem is that {{ServerActionExecutor}} is moving a tasks's state to 
> {{HOLDING}} (reflected in the database correctly) but the cache invalidation 
> happens inside the uncommitted transaction. This causes stale data to be 
> re-cached. So, when we go to calculate the request and state status, we get 
> {{IN_PROGRESS}} instead of {{HOLDING}}.
> 
> {code}
> {
>   "href": 
> "http://172.22.72.13:8080/api/v1/clusters/cl1/requests/61/stages/1?fields=*,tasks/*;,
>   "Stage": {
> "cluster_name": "cl1",
> "context": "Stop YARN Queues",
> "display_status": "IN_PROGRESS",
> "end_time": -1,
> "progress_percent": 35,
> "request_id": 61,
> "skippable": true,
> "stage_id": 1,
> "start_time": 1456227329191,
> "status": "IN_PROGRESS"
>   },
>   "tasks": [
> {
>   "href": 
> "http://172.22.72.13:8080/api/v1/clusters/cl1/requests/61/stages/1/tasks/754;,
>   "Tasks": {
> "attempt_cnt": 1,
> "cluster_name": "cl1",
> "command": "EXECUTE",
> "command_detail": "Before continuing, please stop all YARN queues. If 
> yarn-site's yarn.resourcemanager.work-preserving-recovery.enabled is set to 
> true, then you can skip this step since the clients will retry on their own.",
> "custom_command_name": 
> "org.apache.ambari.server.serveraction.upgrades.ManualStageAction",
> "end_time": -1,
> "error_log": "errors-754.txt",
> "exit_code": 0,
> "host_name": "os-r6-mkqzcs-c10tom21unsecha-6.novalocal",
> "id": 754,
> "output_log": "output-754.txt",
> "request_id": 61,
> "role": "AMBARI_SERVER_ACTION",
> "stage_id": 1,
> "start_time": 1456227329191,
> "status": "HOLDING",
> "stderr": "",
> "stdout": "",
> "structured_out": {}
>   }
> }
>   ]
> }
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/com/google/inject/persist/jpa/AmbariJpaPersistModule.java
>  4e4dd35 
>   
> ambari-server/src/main/java/org/apache/ambari/annotations/TransactionalLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/AmbariJpaLocalTxnInterceptor.java
>  6d7901c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/TransactionalLockInterceptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/TransactionalLocks.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  deca9b1 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 29ebc1f 
> 
> Diff: https://reviews.apache.org/r/43967/diff/
> 
> 
> Testing
> ---
> 
> Pending unit tests...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 43967: Express Upgrade Stuck At Manual Prompt Due To HRC Status Calculation Cache Problem

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43967/#review120615
---




ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 (line 16)
<https://reviews.apache.org/r/43967/#comment182082>

During RU/EU do we change the state of some HRC - force change to HOLDING 
or something like that. Will that need cache invalidation?



ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 (line 242)
<https://reviews.apache.org/r/43967/#comment182081>

Any reason it was expireAfterAccess? Just to force a refresh?


- Sumit Mohanty


On Feb. 24, 2016, 11:53 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43967/
> ---
> 
> (Updated Feb. 24, 2016, 11:53 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Sumit Mohanty, 
> Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15173
> https://issues.apache.org/jira/browse/AMBARI-15173
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Seen while performing an upgrade, it's possible that the status of a 
> request/stage does not match that of its tasks. Essentially, the task could 
> be {{HOLDING}} while the request is still {{IN_PROGRESS}}.
> 
> I believe that AMBARI-15011 is responsible for this issue. AMBARI-15011 
> introduced, among other things, a cache to the 
> {{HostRoleCommandStatusSummaryDTO}} which is a aggregation of the number of 
> tasks a stage has in each state (PENDING, HOLDING, etc).
> 
> This {{HostRoleCommandStatusSummaryDTO}} is used by {{CalculatedState}} to 
> calculate a stage's and request's status based on the tasks. 
> 
> The problem is that {{ServerActionExecutor}} is moving a tasks's state to 
> {{HOLDING}} (reflected in the database correctly) but the cache invalidation 
> happens inside the uncommitted transaction. This causes stale data to be 
> re-cached. So, when we go to calculate the request and state status, we get 
> {{IN_PROGRESS}} instead of {{HOLDING}}.
> 
> {code}
> {
>   "href": 
> "http://172.22.72.13:8080/api/v1/clusters/cl1/requests/61/stages/1?fields=*,tasks/*;,
>   "Stage": {
> "cluster_name": "cl1",
> "context": "Stop YARN Queues",
> "display_status": "IN_PROGRESS",
> "end_time": -1,
> "progress_percent": 35,
> "request_id": 61,
> "skippable": true,
> "stage_id": 1,
> "start_time": 1456227329191,
> "status": "IN_PROGRESS"
>   },
>   "tasks": [
> {
>   "href": 
> "http://172.22.72.13:8080/api/v1/clusters/cl1/requests/61/stages/1/tasks/754;,
>   "Tasks": {
> "attempt_cnt": 1,
> "cluster_name": "cl1",
> "command": "EXECUTE",
> "command_detail": "Before continuing, please stop all YARN queues. If 
> yarn-site's yarn.resourcemanager.work-preserving-recovery.enabled is set to 
> true, then you can skip this step since the clients will retry on their own.",
> "custom_command_name": 
> "org.apache.ambari.server.serveraction.upgrades.ManualStageAction",
> "end_time": -1,
> "error_log": "errors-754.txt",
> "exit_code": 0,
> "host_name": "os-r6-mkqzcs-c10tom21unsecha-6.novalocal",
> "id": 754,
> "output_log": "output-754.txt",
> "request_id": 61,
> "role": "AMBARI_SERVER_ACTION",
> "stage_id": 1,
> "start_time": 1456227329191,
> "status": "HOLDING",
> "stderr": "",
> "stdout": "",
> "structured_out": {}
>   }
> }
>   ]
> }
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/com/google/inject/persist/jpa/AmbariJpaPersistModule.java
>  4e4dd35 
>   
> ambari-server/src/main/java/org/apache/ambari/annotations/TransactionalLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/AmbariJpaLocalTxnInterceptor.java
>  6d7901c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/TransactionalLockInterceptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/TransactionalLocks.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  deca9b1 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 29ebc1f 
> 
> Diff: https://reviews.apache.org/r/43967/diff/
> 
> 
> Testing
> ---
> 
> Pending unit tests...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 43965: Move logic involving HBase shell calls to enable normalization and FIFO compaction policy to Java code

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43965/#review120588
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 24, 2016, 9:56 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43965/
> ---
> 
> (Updated Feb. 24, 2016, 9:56 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15170
> https://issues.apache.org/jira/browse/AMBARI-15170
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, HBase shell calls are made through the metrics collector startup 
> script in a parallel thread along with the Collector start jvm, causing 
> performance issues. If the logic is moved to the collector Java code, we can 
> return from the startup script quicker and declare the action to be Success 
> to the agent once schema creating succeeds.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector
>  e319d73 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  b5ec6e8 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  1c86ebb 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultPhoenixDataSource.java
>  8283f7d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixConnectionProvider.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  cd1bfb3 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/AbstractMiniHBaseClusterTest.java
>  df4fc89 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java
>  0522f81 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  9838bca 
> 
> Diff: https://reviews.apache.org/r/43965/diff/
> 
> 
> Testing
> ---
> 
> Unit test pass.
> Manual test pass.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 43965: Move logic involving HBase shell calls to enable normalization and FIFO compaction policy to Java code

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43965/#review120587
---




ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector
 (line 307)
<https://reviews.apache.org/r/43965/#comment182058>

Delete these?


- Sumit Mohanty


On Feb. 24, 2016, 9:56 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43965/
> ---
> 
> (Updated Feb. 24, 2016, 9:56 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15170
> https://issues.apache.org/jira/browse/AMBARI-15170
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, HBase shell calls are made through the metrics collector startup 
> script in a parallel thread along with the Collector start jvm, causing 
> performance issues. If the logic is moved to the collector Java code, we can 
> return from the startup script quicker and declare the action to be Success 
> to the agent once schema creating succeeds.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector
>  e319d73 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  b5ec6e8 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  1c86ebb 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultPhoenixDataSource.java
>  8283f7d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixConnectionProvider.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  cd1bfb3 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/AbstractMiniHBaseClusterTest.java
>  df4fc89 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java
>  0522f81 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  9838bca 
> 
> Diff: https://reviews.apache.org/r/43965/diff/
> 
> 
> Testing
> ---
> 
> Unit test pass.
> Manual test pass.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 43950: UpgradeCatalog230 is not idempotent

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43950/#review120527
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 24, 2016, 4:09 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43950/
> ---
> 
> (Updated Feb. 24, 2016, 4:09 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15162
> https://issues.apache.org/jira/browse/AMBARI-15162
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If ambari-server upgrade is run again, the following error is encountered. 
> One easy way to test it is to set the version in the DB to an older version 
> and then call ambari-server upgrade.
> 
> e.g. `update metainfo set metainfo_value = '2.2.0' where neatinfo_key = 
> 'version';`
>  
> ```
> Error output from schema upgrade command:
> Exception in thread "main" org.apache.ambari.server.AmbariException: 
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 
> 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key 
> value violates unique constraint "adminpermission_pkey"
> Error Code: 0
> Call: INSERT INTO adminpermission (permission_id, permission_label, 
> permission_name, sort_order, resource_type_id) VALUES (?, ?, ?, ?, ?)
>   bind => [5 parameters bound]
>   at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:233)
>   at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:307)
> Caused by: javax.persistence.RollbackException: Exception [EclipseLink-4002] 
> (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): 
> org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key 
> value violates unique constraint "adminpermission_pkey"
> Error Code: 0
> Call: INSERT INTO adminpermission (permission_id, permission_label, 
> permission_name, sort_order, resource_type_id) VALUES (?, ?, ?, ?, ?)
>   bind => [5 parameters bound]
>   at 
> org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.commit(EntityTransactionImpl.java:157)
>   at 
> org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:91)
>   at 
> org.apache.ambari.server.upgrade.UpgradeCatalog230.addNewPermissions(UpgradeCatalog230.java:144)
>   at 
> org.apache.ambari.server.upgrade.UpgradeCatalog230.executeDMLUpdates(UpgradeCatalog230.java:127)
>   at 
> org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeData(AbstractUpgradeCatalog.java:659)
>   at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:230)
>   ... 1 more
> Caused by: Exception [EclipseLink-4002] (Eclipse Persistence Services - 
> 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key 
> value violates unique constraint "adminpermission_pkey"
> Error Code: 0
> Call: INSERT INTO adminpermission (permission_id, permission_label, 
> permission_name, sort_order, resource_type_id) VALUES (?, ?, ?, ?, ?)
>   bind => [5 parameters bound]
>   at 
> org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:340)
>   at 
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.processExceptionForCommError(DatabaseAccessor.java:1611)
>   at 
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:898)
>   at 
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeNoSelect(DatabaseAccessor.java:962)
>   at 
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:631)
>   at 
> org.eclipse.persistence.internal.databaseaccess.ParameterizedSQLBatchWritingMechanism.executeBatch(ParameterizedSQLBatchWritingMechanism.java:149)
>   at 
> org.eclipse.persistence.internal.databaseaccess.ParameterizedSQLBatchWritingMechanism.executeBatchedStatements(ParameterizedSQLBatchWritingMechanism.java:134)
>   at 
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccesso

Re: Review Request 43947: YARN restart icon appeared after 5 minutes after reconfig

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43947/#review120524
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 24, 2016, 3:54 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43947/
> ---
> 
> (Updated Feb. 24, 2016, 3:54 p.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15160
> https://issues.apache.org/jira/browse/AMBARI-15160
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:  
> 1) Add new host  
> 2) Add Zookeeper server to added host
> 
> Expected: Yarn show restart icon immediately  
> Actually: YARN restart icon was appeared after 5 minutes
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> aa30b48 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
>  4c9bd2e 
> 
> Diff: https://reviews.apache.org/r/43947/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 43909: Remove Result.STATUS enum as it is not being used

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43909/#review120392
---


Ship it!




Interesting.

- Sumit Mohanty


On Feb. 23, 2016, 9:58 p.m., Ajit Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43909/
> ---
> 
> (Updated Feb. 23, 2016, 9:58 p.m.)
> 
> 
> Review request for Ambari and Nahappan Somasundaram.
> 
> 
> Bugs: AMBARI-15149
> https://issues.apache.org/jira/browse/AMBARI-15149
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove Result.STATUS enum as it is not being used
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/Result.java 
> c827ac49ea204c25065e84ffbe4395a720a8dba9 
> 
> Diff: https://reviews.apache.org/r/43909/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ajit Kumar
> 
>

[jira] [Updated] (AMBARI-15139) Tracks work items related to log accumulation and searching support in Ambari

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15139:
---
Description: Ambari already deploys a number of services and is aware of 
the log file location for all the services. If there is a need to debug, 
currently, user needs to go to each host and browse through each file to debug 
issues. This can be greatly improved by creating a log search service (a stack 
service in itself) that can be configured to ingest existing log files and 
allowing easy search/query across all log files - include Ambari's own.  (was: 
Ambari already deploys a number of services and know of the log location for 
all the services. Currently, user needs to go to each host to look up for log 
files  and manually browse through each file to debug issues. This can be 
greatly improved by creating a log search service (a stack service in itself) 
that can be configured to ingest existing log files and allowing easy 
search/query across all log files - include Ambari's own.)

> Tracks work items related to log accumulation and searching support in Ambari
> -
>
> Key: AMBARI-15139
> URL: https://issues.apache.org/jira/browse/AMBARI-15139
> Project: Ambari
>  Issue Type: Epic
>  Components: stacks
>Affects Versions: 2.2.2
>    Reporter: Sumit Mohanty
>    Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.4.0
>
>
> Ambari already deploys a number of services and is aware of the log file 
> location for all the services. If there is a need to debug, currently, user 
> needs to go to each host and browse through each file to debug issues. This 
> can be greatly improved by creating a log search service (a stack service in 
> itself) that can be configured to ingest existing log files and allowing easy 
> search/query across all log files - include Ambari's own.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-15139) Tracks work items related to log accumulation and searching support in Ambari

[Sumit Mohanty (JIRA)]

Sumit Mohanty created AMBARI-15139:
--

 Summary: Tracks work items related to log accumulation and 
searching support in Ambari
 Key: AMBARI-15139
 URL: https://issues.apache.org/jira/browse/AMBARI-15139
 Project: Ambari
  Issue Type: Epic
  Components: stacks
Affects Versions: 2.2.2
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
Priority: Critical
 Fix For: 2.4.0


Ambari already deploys a number of services and know of the log location for 
all the services. Currently, user needs to go to each host to look up for log 
files  and manually browse through each file to debug issues. This can be 
greatly improved by creating a log search service (a stack service in itself) 
that can be configured to ingest existing log files and allowing easy 
search/query across all log files - include Ambari's own.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43852: Upgrade alert definition for App Timeline Web UI alert

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43852/#review120220
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 22, 2016, 10:02 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43852/
> ---
> 
> (Updated Feb. 22, 2016, 10:02 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15130
> https://issues.apache.org/jira/browse/AMBARI-15130
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update the alert definition of the App Timeline Web UI alert in Upgrade 
> catalog.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  2d0b556 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  08b38e3 
> 
> Diff: https://reviews.apache.org/r/43852/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

[jira] [Commented] (AMBARI-15130) Upgrade alert definition for App Timeline Web UI alert

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157345#comment-15157345
 ] 

Sumit Mohanty commented on AMBARI-15130:


Sample:  
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java#L1128

> Upgrade alert definition for App Timeline Web UI alert
> --
>
> Key: AMBARI-15130
> URL: https://issues.apache.org/jira/browse/AMBARI-15130
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.2.2
>    Reporter: Sumit Mohanty
> Fix For: 2.2.2
>
>
> Update the alert definition of the App Timeline Web UI alert. See 
> AMBARI-15123 for the change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15130) Upgrade alert definition for App Timeline Web UI alert

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15130:
---
Assignee: Vitaly Brodetskyi

> Upgrade alert definition for App Timeline Web UI alert
> --
>
> Key: AMBARI-15130
> URL: https://issues.apache.org/jira/browse/AMBARI-15130
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.2.2
>    Reporter: Sumit Mohanty
>Assignee: Vitaly Brodetskyi
> Fix For: 2.2.2
>
>
> Update the alert definition of the App Timeline Web UI alert. See 
> AMBARI-15123 for the change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43831: Ambari's App Timeline Web UI Alert tries make http heavy call timelineserver:httpPort which can also cause OOM

[Sumit Mohanty]

> On Feb. 22, 2016, 2:12 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/alerts.json,
> >  line 391
> > <https://reviews.apache.org/r/43831/diff/1/?file=1264104#file1264104line391>
> >
> > Jonathan, one question: It's not critical to update the definiton 
> > during Ambari upgrade. However, do we have any example of making alert 
> > definition changes during upgrade?
> 
> Jonathan Hurley wrote:
> Yeah; the upgrade packs have this already. Here's one: 
> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java#L1128

Thanks. I have opened https://issues.apache.org/jira/browse/AMBARI-15130 for 
the upgarde catalog change.


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43831/#review120139
-------


On Feb. 22, 2016, 2:10 p.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43831/
> ---
> 
> (Updated Feb. 22, 2016, 2:10 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-15123
> https://issues.apache.org/jira/browse/AMBARI-15123
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Change the path to call during alert
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/alerts.json 
> bbf3a7c 
> 
> Diff: https://reviews.apache.org/r/43831/diff/
> 
> 
> Testing
> ---
> 
> Manually tested by editing alert defintion of a deployed cluster
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Re: Review Request 43833: Oozie server unable to load JDBC driver

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43833/#review120147
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 22, 2016, 3:35 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43833/
> ---
> 
> (Updated Feb. 22, 2016, 3:35 p.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15129
> https://issues.apache.org/jira/browse/AMBARI-15129
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ERROR: Oozie could not be started
> 
> REASON: org.apache.oozie.service.ServiceException: E0103: Could not load 
> service classes, Cannot load JDBC driver class 
> 'com.sqlserver.jdbc.SQLServerDriver'
> 
> Stacktrace:
> -
> org.apache.oozie.service.ServiceException: E0103: Could not load service 
> classes, Cannot load JDBC driver class 'com.sqlserver.jdbc.SQLServerDriver'
> at 
> org.apache.oozie.service.Services.loadServices(Services.java:309)
> at org.apache.oozie.service.Services.init(Services.java:213)
> at 
> org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
> at 
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> at 
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> at 
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:802)
> at 
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
> at 
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
> at 
> org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:676)
> at 
> org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:602)
> at 
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:503)
> at 
> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322)
> at 
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325)
> at 
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> at 
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1068)
> at 
> org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
> at 
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1060)
> at 
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/scripts/Ambaripreupload.py fb0c3e1 
> 
> Diff: https://reviews.apache.org/r/43833/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

[jira] [Created] (AMBARI-15130) Upgrade alert definition for App Timeline Web UI alert

[Sumit Mohanty (JIRA)]

Sumit Mohanty created AMBARI-15130:
--

 Summary: Upgrade alert definition for App Timeline Web UI alert
 Key: AMBARI-15130
 URL: https://issues.apache.org/jira/browse/AMBARI-15130
 Project: Ambari
  Issue Type: Bug
  Components: stacks
Affects Versions: 2.2.2
Reporter: Sumit Mohanty
 Fix For: 2.2.2


Update the alert definition of the App Timeline Web UI alert. See AMBARI-15123 
for the change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43831: Ambari's App Timeline Web UI Alert tries make http heavy call timelineserver:httpPort which can also cause OOM

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43831/#review120139
---




ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/alerts.json 
(line 391)
<https://reviews.apache.org/r/43831/#comment181517>

Jonathan, one question: It's not critical to update the definiton during 
Ambari upgrade. However, do we have any example of making alert definition 
changes during upgrade?


- Sumit Mohanty


On Feb. 22, 2016, 2:10 p.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43831/
> ---
> 
> (Updated Feb. 22, 2016, 2:10 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-15123
> https://issues.apache.org/jira/browse/AMBARI-15123
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Change the path to call during alert
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/alerts.json 
> bbf3a7c 
> 
> Diff: https://reviews.apache.org/r/43831/diff/
> 
> 
> Testing
> ---
> 
> Manually tested by editing alert defintion of a deployed cluster
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Review Request 43831: Ambari's App Timeline Web UI Alert tries make http heavy call timelineserver:httpPort which can also cause OOM

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43831/
---

Review request for Ambari, Andrew Onischuk and Jonathan Hurley.


Bugs: AMBARI-15123
https://issues.apache.org/jira/browse/AMBARI-15123


Repository: ambari


Description
---

Change the path to call during alert


Diffs
-

  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/alerts.json 
bbf3a7c 

Diff: https://reviews.apache.org/r/43831/diff/


Testing
---

Manually tested by editing alert defintion of a deployed cluster


Thanks,

Sumit Mohanty

Re: Review Request 43804: Preupload.py should use hdfs_path_prefix

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43804/#review120053
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 20, 2016, 6:19 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43804/
> ---
> 
> (Updated Feb. 20, 2016, 6:19 p.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15125
> https://issues.apache.org/jira/browse/AMBARI-15125
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/scripts/Ambaripreupload.py cc6213e 
> 
> Diff: https://reviews.apache.org/r/43804/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

[jira] [Updated] (AMBARI-15123) Ambari's App Timeline Web UI Alert tries make http heavy call timelineserver:httpPort which can also cause OOM

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15123:
---
Attachment: AMBARI-15123.patch

> Ambari's App Timeline Web UI Alert tries make http heavy call 
> timelineserver:httpPort which can also cause OOM
> --
>
> Key: AMBARI-15123
> URL: https://issues.apache.org/jira/browse/AMBARI-15123
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.2.2
>    Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
> Fix For: 2.2.2
>
> Attachments: AMBARI-15123.patch
>
>
> Ambari's App Timeline Web UI Alert tries make http heavy call 
> timelineserver:httpPort which can also cause OOM
> Hit this issue while testing ATS v1.5 on 32 node cluster under load was going 
> OOM, we discovered there are instance where  is being hit.
> Looks like Ambari's App Timeline Web UI alerting system is trying call it
> As from Ambari Alerts we can see error like "Connection failed to 
> http://ATS_HOST:8188 (timed out)"
> Now If we try hit http://:, by default It load 
> http://:/applicationhistory 
> e.g.  Accessing http://ATS_HOST:8188 will try to access 
> http:/ATS_HOST:8188/applicationhistory page.
> If cluster has AHS/GHS enabled (under ATS), then this call result heavy html 
> containing large of application.
> We can make alert to access some lightweight call like: 
> http://:/applicationhistory/about, or 
> http://:/ws/v1/timeline
> e.g.  http://ATS_HOST:8188/applicationhistory/about
> or 
> http://ATS_HOST:8188/ws/v1/timeline



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-15123) Ambari's App Timeline Web UI Alert tries make http heavy call timelineserver:httpPort which can also cause OOM

[Sumit Mohanty (JIRA)]

Sumit Mohanty created AMBARI-15123:
--

 Summary: Ambari's App Timeline Web UI Alert tries make http heavy 
call timelineserver:httpPort which can also cause OOM
 Key: AMBARI-15123
 URL: https://issues.apache.org/jira/browse/AMBARI-15123
 Project: Ambari
  Issue Type: Bug
  Components: stacks
Affects Versions: 2.2.2
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
 Fix For: 2.2.2
 Attachments: AMBARI-15123.patch

Ambari's App Timeline Web UI Alert tries make http heavy call 
timelineserver:httpPort which can also cause OOM

Hit this issue while testing ATS v1.5 on 32 node cluster under load was going 
OOM, we discovered there are instance where  is being hit.

Looks like Ambari's App Timeline Web UI alerting system is trying call it
As from Ambari Alerts we can see error like "Connection failed to 
http://ATS_HOST:8188 (timed out)"

Now If we try hit http://:, by default It load 
http://:/applicationhistory 
e.g.  Accessing http://ATS_HOST:8188 will try to access 
http:/ATS_HOST:8188/applicationhistory page.
If cluster has AHS/GHS enabled (under ATS), then this call result heavy html 
containing large of application.

We can make alert to access some lightweight call like: 
http://:/applicationhistory/about, or 
http://:/ws/v1/timeline
e.g.  http://ATS_HOST:8188/applicationhistory/about
or 
http://ATS_HOST:8188/ws/v1/timeline



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-15006) Ambari Server Upgrade adds unneeded Atlas properties even though Atlas is not needed, causing forced Hive restart

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154967#comment-15154967
 ] 

Sumit Mohanty commented on AMBARI-15006:


[~aonishuk], can you commit it. Changes look good.

> Ambari Server Upgrade adds unneeded Atlas properties even though Atlas is not 
> needed, causing forced Hive restart
> -
>
> Key: AMBARI-15006
> URL: https://issues.apache.org/jira/browse/AMBARI-15006
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Yusaku Sako
>Assignee: Andrew Onischuk
>Priority: Critical
> Fix For: 2.2.2
>
> Attachments: AMBARI-15006.patch
>
>
> Upgrading Ambari Server from 2.1.2.1 to 2.2.1 caused unnecessary addition of 
> the following config properties in hive-site.xml and forced Hive to be 
> restarted:
> atlas.cluster.name: primary
> atlas.hook.hive.maxThreads: 1
> atlas.hook.hive.minThreads: 1
> atlas.rest.address: http://localhost:21000
> The cluster does not have Atlas installed.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15006) Ambari Server Upgrade adds unneeded Atlas properties even though Atlas is not needed, causing forced Hive restart

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15006:
---
Assignee: Andrew Onischuk  (was: Sumit Mohanty)

> Ambari Server Upgrade adds unneeded Atlas properties even though Atlas is not 
> needed, causing forced Hive restart
> -
>
> Key: AMBARI-15006
> URL: https://issues.apache.org/jira/browse/AMBARI-15006
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Yusaku Sako
>Assignee: Andrew Onischuk
>Priority: Critical
> Fix For: 2.2.2
>
> Attachments: AMBARI-15006.patch
>
>
> Upgrading Ambari Server from 2.1.2.1 to 2.2.1 caused unnecessary addition of 
> the following config properties in hive-site.xml and forced Hive to be 
> restarted:
> atlas.cluster.name: primary
> atlas.hook.hive.maxThreads: 1
> atlas.hook.hive.minThreads: 1
> atlas.rest.address: http://localhost:21000
> The cluster does not have Atlas installed.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43720: Ambari Server Upgrade adds unneeded Atlas properties even though Atlas is not needed, causing forced Hive restart

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43720/#review119983
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 19, 2016, 11:40 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43720/
> ---
> 
> (Updated Feb. 19, 2016, 11:40 a.m.)
> 
> 
> Review request for Ambari and Myroslav Papirkovskyy.
> 
> 
> Bugs: AMBARI-15006
> https://issues.apache.org/jira/browse/AMBARI-15006
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upgrading Ambari Server from 2.1.2.1 to 2.2.1 caused unnecessary addition of
> the following config properties in hive-site.xml and forced Hive to be
> restarted:
> 
> atlas.cluster.name: primary  
> atlas.hook.hive.maxThreads: 1  
> atlas.hook.hive.minThreads: 1  
> atlas.rest.address: <http://localhost:21000>
> 
> The cluster does not have Atlas installed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  88b3151 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  4087c72 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-site.xml
>  a611386 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  6061e06 
> 
> Diff: https://reviews.apache.org/r/43720/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 43770: Agent bootstrap/registration fails in multi-byte language environments

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43770/#review119979
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 19, 2016, 9:29 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43770/
> ---
> 
> (Updated Feb. 19, 2016, 9:29 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Dmytro Sen, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15115
> https://issues.apache.org/jira/browse/AMBARI-15115
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During cluster installation, Ambari Agent would output the error below and 
> host registration/set up would fail.
> It seems that this issue only happens in multi-byte language environments on 
> particular OS's.
> The issue occurred on CentOS / RHEL 6.3, 6.4 6.5, 7.1, and 7.2.
> CentOS / RHEL 6.6, 6.7 seem to be fine.
> Version: HDP 2.3.4, Ambari 2.2.0.0
> ambari-agent.out
> ==
> File "/usr/lib/python2.6/site-packages/resource_management/core/logger.py", 
> line 147, in get_function_repr
> return unicode("
> {0}
> {{
> {1}
> }}").format(name, arguments_str)
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 15: 
> ordinal not in range(128)
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/resource_management/core/logger.py f126f1e 
> 
> Diff: https://reviews.apache.org/r/43770/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Review Request 43781: Fix and enable UTs testMetricsLoaded and testServicesWithRangerPluginRoleCommandOrder

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43781/
---

Review request for Ambari, Alejandro Fernandez and Sid Wagle.


Bugs: AMBARI-15118
https://issues.apache.org/jira/browse/AMBARI-15118


Repository: ambari


Description
---

Fix and enable UTs testMetricsLoaded and 
testServicesWithRangerPluginRoleCommandOrder


Diffs
-

  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerTest.java
 455652b 

Diff: https://reviews.apache.org/r/43781/diff/


Testing
---

Ran the enabled unit tests locally and they succeeded (they were failing before 
the fixes).


Thanks,

Sumit Mohanty

[jira] [Updated] (AMBARI-15118) Fix and enable UTs testMetricsLoaded and testServicesWithRangerPluginRoleCommandOrder

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15118:
---
Attachment: AMBARI-15118.patch

> Fix and enable UTs testMetricsLoaded and 
> testServicesWithRangerPluginRoleCommandOrder
> -
>
> Key: AMBARI-15118
> URL: https://issues.apache.org/jira/browse/AMBARI-15118
> Project: Ambari
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.2.2
>    Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
> Fix For: 2.2.2
>
> Attachments: AMBARI-15118.patch
>
>
> Fix and enable UTs testMetricsLoaded and 
> testServicesWithRangerPluginRoleCommandOrder. AMBARI-15101 removed an RCO 
> dependency that breaks one of the unit tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-15118) Fix and enable UTs testMetricsLoaded and testServicesWithRangerPluginRoleCommandOrder

[Sumit Mohanty (JIRA)]

Sumit Mohanty created AMBARI-15118:
--

 Summary: Fix and enable UTs testMetricsLoaded and 
testServicesWithRangerPluginRoleCommandOrder
 Key: AMBARI-15118
 URL: https://issues.apache.org/jira/browse/AMBARI-15118
 Project: Ambari
  Issue Type: Bug
  Components: test
Affects Versions: 2.2.2
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
 Fix For: 2.2.2


Fix and enable UTs testMetricsLoaded and 
testServicesWithRangerPluginRoleCommandOrder. AMBARI-15101 removed an RCO 
dependency that breaks one of the unit tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15101) Remove the dependency between ZK start and Ranger user sync start

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15101:
---
Attachment: AMBARI-15101.patch

> Remove the dependency between ZK start and Ranger user sync start
> -
>
> Key: AMBARI-15101
> URL: https://issues.apache.org/jira/browse/AMBARI-15101
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.2.2
>    Reporter: Sumit Mohanty
>    Assignee: Sumit Mohanty
> Fix For: 2.2.2
>
> Attachments: AMBARI-15101.patch
>
>
> ZK start should not depend on Ranger user sync start. We should remove the 
> dependency.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-15101) Remove the dependency between ZK start and Ranger user sync start

[Sumit Mohanty (JIRA)]

Sumit Mohanty created AMBARI-15101:
--

 Summary: Remove the dependency between ZK start and Ranger user 
sync start
 Key: AMBARI-15101
 URL: https://issues.apache.org/jira/browse/AMBARI-15101
 Project: Ambari
  Issue Type: Bug
  Components: stacks
Affects Versions: 2.2.2
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
 Fix For: 2.2.2


ZK start should not depend on Ranger user sync start. We should remove the 
dependency.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43739: AMBARI-15099 : Ambari missing metrics in UI with Vip settings enabled

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43739/#review119724
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 18, 2016, 9:23 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43739/
> ---
> 
> (Updated Feb. 18, 2016, 9:23 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15099
> https://issues.apache.org/jira/browse/AMBARI-15099
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> When using a vip settings for a cluster with 2 Collectors (1 active at a 
> time), no metrics are seen unless Ambari Server is restarted.
> 
> BUG
> In a cluster with metrics collector Vip settings enabled, when a Metrics 
> Collector instance status changes from STOPPED to STARTED, it is not tracked 
> in the data structure that maintains the collector host in Ambari Server. 
> This causes the Ambari server to drop requests for metrics assuming there is 
> no Provider to satsify the request.
> 
> FIX
> Decouple the data structure used to track collector hosts and Vip host.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  5fb2cf0 
> 
> Diff: https://reviews.apache.org/r/43739/diff/
> 
> 
> Testing
> ---
> 
> Manual testing done.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 43675: Preupload.py should pre-create hdfs directories

[Sumit Mohanty]

> On Feb. 17, 2016, 9:39 p.m., Sumit Mohanty wrote:
> > ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py,
> >  line 460
> > <https://reviews.apache.org/r/43675/diff/1/?file=1253095#file1253095line460>
> >
> > Overall changes look good. Can we enable it unconditionally? What will 
> > happen when a path got created say "/usr/abc" then user changed the 
> > property to be "/usr/xyz" and then manually deleted "/usr/abc" in HDFS. 
> > Now, if user changes the path back to "/usr/abc" will Ambari re-create the 
> > path?
> 
> Andrew Onischuk wrote:
> Unconditionally? I don't get it, if we remove this condition it's gonna 
> ignore every single hdfsResource.
> 
> Also the usecase you mentioned will indeed cause a problem. 
> But the usecase is a bit weird, because if user deleted the directory he 
> did it for purpose obviously.

What I meant is if we need a config to enable/disable the ability for 
HdfsResourceProvider to track work using that file. I agree that my example is 
odd. A more real life scenario will be to Add HBase as a service, delete it and 
delete its data folders from HDFS. Then add it back and the data folders will 
not be created again.


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43675/#review119530
---


On Feb. 17, 2016, 9:07 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43675/
> -------
> 
> (Updated Feb. 17, 2016, 9:07 p.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15079
> https://issues.apache.org/jira/browse/AMBARI-15079
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This should eliminate any 'hadoop --config ... jar' fast-hdfs-resource.jar 
> calls during deploy.
> By executing them pre-deploy in preupload.py, and adding to ignore file.
> This should benefit ~6 minutes of time deploy, taking the data from 24min 
> cluster. And probably less for faster deployments.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
>  ebcf1a4 
>   
> ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py
>  7c12409 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
>  2bd2626 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  f3a97fc 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  b150464 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  6837bf1 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  29c4784 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  dc17dba 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py
>  47af240 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  7ec85b5 
>   
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/params.py
>  69c03ea 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  7a2f6f6 
>   
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/params_linux.py
>  f923723 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/params_linux.py
>  09b7876 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  68c4f37 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/params_linux.py
>  25da2a1 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py
>  25f867e 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  f0b6927 
>   ambari-server/src/main/resources/scripts/Ambaripreupload.py 61db286 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  2a9d7c5 
>   
> ambari-server/src/test/python/stacks/2.0.

Re: Review Request 43675: Preupload.py should pre-create hdfs directories

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43675/#review119530
---




ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
 (line 441)
<https://reviews.apache.org/r/43675/#comment180883>

Overall changes look good. Can we enable it unconditionally? What will 
happen when a path got created say "/usr/abc" then user changed the property to 
be "/usr/xyz" and then manually deleted "/usr/abc" in HDFS. Now, if user 
changes the path back to "/usr/abc" will Ambari re-create the path?


- Sumit Mohanty


On Feb. 17, 2016, 9:07 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43675/
> ---
> 
> (Updated Feb. 17, 2016, 9:07 p.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15079
> https://issues.apache.org/jira/browse/AMBARI-15079
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This should eliminate any 'hadoop --config ... jar' fast-hdfs-resource.jar 
> calls during deploy.
> By executing them pre-deploy in preupload.py, and adding to ignore file.
> This should benefit ~6 minutes of time deploy, taking the data from 24min 
> cluster. And probably less for faster deployments.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
>  ebcf1a4 
>   
> ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py
>  7c12409 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
>  2bd2626 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  f3a97fc 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  b150464 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  6837bf1 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  29c4784 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  dc17dba 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py
>  47af240 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  7ec85b5 
>   
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/params.py
>  69c03ea 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  7a2f6f6 
>   
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/params_linux.py
>  f923723 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/params_linux.py
>  09b7876 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  68c4f37 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/params_linux.py
>  25da2a1 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py
>  25f867e 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  f0b6927 
>   ambari-server/src/main/resources/scripts/Ambaripreupload.py 61db286 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  2a9d7c5 
>   
> ambari-server/src/test/python/stacks/2.0.6/AMBARI_METRICS/test_metrics_collector.py
>  96e2286 
>   ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 
> 13b2e33 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 39244ff 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_service_check.py 
> 0f5afa8 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 494d16c 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 
> ea17c27 
>   ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 
> ba1b84a 
>   ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 
> a6d0145 
>   ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_service_check.py 
> c5de4c3 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_

[jira] [Commented] (AMBARI-15007) Make amazon2015 to be part of redhat6 family

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149254#comment-15149254
 ] 

Sumit Mohanty commented on AMBARI-15007:


Functional tests should not run as part of the unit test run. [~smnaha] can you 
check if the behavior has changed?

> Make amazon2015 to be part of redhat6 family
> 
>
> Key: AMBARI-15007
> URL: https://issues.apache.org/jira/browse/AMBARI-15007
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.2.2
>
> Attachments: AMBARI-15007.patch
>
>
> We have discussed the solution for this with Sumit Mohanty.  
> We decided to avoid introducing new os family for amazon.  
> And to simply add aliases section to os_family.json like this:
> 
> 
>   "aliases": {
> "amazon2015": "amazon6"
>   }
> 
> The only disadvantage is that when choosing repo-url, user might be confused
> that centos6 url is for amazon 2015.  
> But that what initially what I was asked to do.
> Also this jira should include using /etc/system-release instead of /etc/issue
> to detect amazon os. (/etc/os-release seems to be absent on some centos
> distros, so I think it's not reliable)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-15035) Add support for drpc.servers config property in storm config for blueprint based deployments

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15146032#comment-15146032
 ] 

Sumit Mohanty commented on AMBARI-15035:


Another property with similar requirement:
mapreduce.job.hdfs-servers, needs to point to NN.

> Add support for drpc.servers config property in storm config for blueprint 
> based deployments
> 
>
> Key: AMBARI-15035
> URL: https://issues.apache.org/jira/browse/AMBARI-15035
> Project: Ambari
>  Issue Type: Bug
>  Components: blueprints, stacks
>Affects Versions: 2.2.1
>    Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
> Fix For: 2.2.2
>
> Attachments: AMBARI-15035.patch
>
>
> {{drpc.servers}} is a property in storm yaml config file that captures the 
> drpc server hostname. During blueprint based deployments this can be 
> specified by the user but some code changes are needed to have the blueprint 
> config processor resolve the value to actual hostname.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15035) Add support for drpc.servers config property in storm config for blueprint based deployments

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15035:
---
Attachment: AMBARI-15035.patch

> Add support for drpc.servers config property in storm config for blueprint 
> based deployments
> 
>
> Key: AMBARI-15035
> URL: https://issues.apache.org/jira/browse/AMBARI-15035
> Project: Ambari
>  Issue Type: Bug
>  Components: blueprints, stacks
>Affects Versions: 2.2.1
>    Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
> Fix For: 2.2.2
>
> Attachments: AMBARI-15035.patch
>
>
> {{drpc.servers}} is a property in storm yaml config file that captures the 
> drpc server hostname. During blueprint based deployments this can be 
> specified by the user but some code changes are needed to have the blueprint 
> config processor resolve the value to actual hostname.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15035) Add support for drpc.servers config property in storm config for blueprint based deployments

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15035:
---
Attachment: (was: AMBARI-15035.patch)

> Add support for drpc.servers config property in storm config for blueprint 
> based deployments
> 
>
> Key: AMBARI-15035
> URL: https://issues.apache.org/jira/browse/AMBARI-15035
> Project: Ambari
>  Issue Type: Bug
>  Components: blueprints, stacks
>Affects Versions: 2.2.1
>    Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
> Fix For: 2.2.2
>
> Attachments: AMBARI-15035.patch
>
>
> {{drpc.servers}} is a property in storm yaml config file that captures the 
> drpc server hostname. During blueprint based deployments this can be 
> specified by the user but some code changes are needed to have the blueprint 
> config processor resolve the value to actual hostname.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15035) Add support for drpc.servers config property in storm config for blueprint based deployments

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15035:
---
Description: 
{{drpc.servers}} is a property in storm yaml config file that captures the drpc 
server hostname. During blueprint based deployments this can be specified by 
the user but some code changes are needed to have the blueprint config 
processor resolve the value to actual hostname.

{{mapreduce.job.hdfs-servers}} support need to be added to resolve to NN host.

  was:{{drpc.servers}} is a property in storm yaml config file that captures 
the drpc server hostname. During blueprint based deployments this can be 
specified by the user but some code changes are needed to have the blueprint 
config processor resolve the value to actual hostname.


> Add support for drpc.servers config property in storm config for blueprint 
> based deployments
> 
>
> Key: AMBARI-15035
> URL: https://issues.apache.org/jira/browse/AMBARI-15035
> Project: Ambari
>  Issue Type: Bug
>  Components: blueprints, stacks
>Affects Versions: 2.2.1
>    Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
> Fix For: 2.2.2
>
> Attachments: AMBARI-15035.patch
>
>
> {{drpc.servers}} is a property in storm yaml config file that captures the 
> drpc server hostname. During blueprint based deployments this can be 
> specified by the user but some code changes are needed to have the blueprint 
> config processor resolve the value to actual hostname.
> {{mapreduce.job.hdfs-servers}} support need to be added to resolve to NN host.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 43559: Add blueprint support for drpc.servers and mapreduce.job.hdfs-servers

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43559/
---

Review request for Ambari, Mahadev Konar, Robert Nettleton, and Sid Wagle.


Bugs: AMBARI-15035
https://issues.apache.org/jira/browse/AMBARI-15035


Repository: ambari


Description
---

drpc.servers is a property in storm yaml config file that captures the drpc 
server hostname. During blueprint based deployments this can be specified by 
the user but some code changes are needed to have the blueprint config 
processor resolve the value to actual hostname.

mapreduce.job.hdfs-servers support need to be added to resolve to NN host.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 7fb2592 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 9c76e8a 

Diff: https://reviews.apache.org/r/43559/diff/


Testing
---

Tested manually and added unit tests for these two new properties.


Thanks,

Sumit Mohanty

Re: Review Request 43559: Add blueprint support for drpc.servers and mapreduce.job.hdfs-servers

[Sumit Mohanty]

> On Feb. 13, 2016, 5:26 p.m., Sid Wagle wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java,
> >  line 2258
> > <https://reviews.apache.org/r/43559/diff/1/?file=1241159#file1241159line2258>
> >
> > QQ: The updater handles NN HA properly ?

Blueprint expects users to put in the value of name-service if its a NN-HA 
cluster. So in that case users will specify "mapreduce.job.hdfs-servers":"nnha" 
and the code will not update the property value. It only updates if it finds 
pattern such as "%HOSTGROUP::headnode0%"


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43559/#review119150
-------


On Feb. 13, 2016, 4:43 p.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43559/
> ---
> 
> (Updated Feb. 13, 2016, 4:43 p.m.)
> 
> 
> Review request for Ambari, Mahadev Konar, Robert Nettleton, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15035
> https://issues.apache.org/jira/browse/AMBARI-15035
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> drpc.servers is a property in storm yaml config file that captures the drpc 
> server hostname. During blueprint based deployments this can be specified by 
> the user but some code changes are needed to have the blueprint config 
> processor resolve the value to actual hostname.
> 
> mapreduce.job.hdfs-servers support need to be added to resolve to NN host.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  7fb2592 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  9c76e8a 
> 
> Diff: https://reviews.apache.org/r/43559/diff/
> 
> 
> Testing
> ---
> 
> Tested manually and added unit tests for these two new properties.
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Re: Review Request 43557: App Timeline server failed to start due to java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.timeline.Entit yGroupFSTimelineStore

[Sumit Mohanty]

> On Feb. 13, 2016, 8:35 a.m., Matt wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-site.xml,
> >  line 73
> > <https://reviews.apache.org/r/43557/diff/1/?file=1241153#file1241153line73>
> >
> > Why does this parameter have to be changed?
> > 
> > It was changed in November 2015: 
> > https://issues.apache.org/jira/browse/AMBARI-13936 
> > 
> > Just wondering why the failure because of this parameter showed up now 
> > and not in the last 4 months.
> 
> bhuvnesh chaudhary wrote:
> @Dmytro may be able to guide us why this parameter was set to use 
> EntityGroupFSTimelineStore earlier, but its not working now.
> I believe this ATS has been failing since last couple of days and the 
> status was red, but since service check does not captures it, probably didn't 
> block anything and was overlooked.

I do not think we should make this change. Latest (HDP-2.3.4+) versions of HDP 
needs the value to be EntityGroupFSTimelineStore but older HDP-2.3 does not 
work with that. So depending on which version is being deployed you will see 
one or the other error.  Ambari does not have support for such changes when its 
in the 3rd digit releases. So we decided to favor newer releases.


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43557/#review119144
---


On Feb. 13, 2016, 6:03 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43557/
> ---
> 
> (Updated Feb. 13, 2016, 6:03 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, jun aoki, Jayush 
> Luniya, Oleksandr Diachenko, Richard Zang, Sumit Mohanty, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15042
> https://issues.apache.org/jira/browse/AMBARI-15042
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> App Timeline server failed to start due to java.lang.ClassNotFoundException: 
> Class org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore
> 
> #Snippet from logs:
> Server failed in state INITED; cause: java.lang.RuntimeException: 
> java.lang.RuntimeException: java.lang.ClassNotFoundException: Class 
> org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore not found
> ...
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: 
> Class org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore not 
> found
> 
> There is no class EntityGroupFSTimelineStore available, the value of 
> parameter yarn.timeline-service.store-class should be set as below: 
> org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-site.xml
>  8b6709d 
> 
> Diff: https://reviews.apache.org/r/43557/diff/
> 
> 
> Testing
> ---
> 
> yes. manual
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Re: Review Request 43557: App Timeline server failed to start due to java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.timeline.Entit yGroupFSTimelineStore

[Sumit Mohanty]

> On Feb. 13, 2016, 8:35 a.m., Matt wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-site.xml,
> >  line 73
> > <https://reviews.apache.org/r/43557/diff/1/?file=1241153#file1241153line73>
> >
> > Why does this parameter have to be changed?
> > 
> > It was changed in November 2015: 
> > https://issues.apache.org/jira/browse/AMBARI-13936 
> > 
> > Just wondering why the failure because of this parameter showed up now 
> > and not in the last 4 months.
> 
> bhuvnesh chaudhary wrote:
> @Dmytro may be able to guide us why this parameter was set to use 
> EntityGroupFSTimelineStore earlier, but its not working now.
> I believe this ATS has been failing since last couple of days and the 
> status was red, but since service check does not captures it, probably didn't 
> block anything and was overlooked.
> 
> Sumit Mohanty wrote:
> I do not think we should make this change. Latest (HDP-2.3.4+) versions 
> of HDP needs the value to be EntityGroupFSTimelineStore but older HDP-2.3 
> does not work with that. So depending on which version is being deployed you 
> will see one or the other error.  Ambari does not have support for such 
> changes when its in the 3rd digit releases. So we decided to favor newer 
> releases.
> 
> bhuvnesh chaudhary wrote:
> @Thanks Sumit for the clarification.
> Does it mean that https://reviews.apache.org/r/43556/ is also not 
> required and is done considering HDP-2.3.4+ version ?

Its the same root cause. YARN had to move to latest version of ATS 1.5 as its 
feature rich and highly available.


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43557/#review119144
---


On Feb. 13, 2016, 6:03 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43557/
> ---
> 
> (Updated Feb. 13, 2016, 6:03 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, jun aoki, Jayush 
> Luniya, Oleksandr Diachenko, Richard Zang, Sumit Mohanty, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15042
> https://issues.apache.org/jira/browse/AMBARI-15042
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> App Timeline server failed to start due to java.lang.ClassNotFoundException: 
> Class org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore
> 
> #Snippet from logs:
> Server failed in state INITED; cause: java.lang.RuntimeException: 
> java.lang.RuntimeException: java.lang.ClassNotFoundException: Class 
> org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore not found
> ...
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: 
> Class org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore not 
> found
> 
> There is no class EntityGroupFSTimelineStore available, the value of 
> parameter yarn.timeline-service.store-class should be set as below: 
> org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-site.xml
>  8b6709d 
> 
> Diff: https://reviews.apache.org/r/43557/diff/
> 
> 
> Testing
> ---
> 
> yes. manual
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Re: Review Request 43528: AMBARI-15030 : Fix unreasonable default heap settings for AMS HBase heapsize and xmn size.

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43528/#review119050
---




ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 (line 281)
<https://reviews.apache.org/r/43528/#comment180332>

Is this necessary? I am making a defensive argument. Ambari upgrade, 
without any basic change to AMS code base should not ned to modify AMS config. 
But yes, if we decide to do it then lets allow both with and without "m"


- Sumit Mohanty


On Feb. 12, 2016, 5:29 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43528/
> ---
> 
> (Updated Feb. 12, 2016, 5:29 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15030
> https://issues.apache.org/jira/browse/AMBARI-15030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Default values from the stack (Not stack advisor)
> regionserver_heapsize : 512MB
> regionserver_xmn_size : 256MB
> 
> xmn is 50% of rs heapsize which is unreasonable. Changing to 756MB and 128MB 
> respectively.
> 
> Made the changes in stack_Advisor and UpgradeCatalog too.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  88b3151 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
>  191e8b2 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> af21008 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  6061e06 
> 
> Diff: https://reviews.apache.org/r/43528/diff/
> 
> 
> Testing
> ---
> 
> UpgradeCatalog tests pass.
> Python unit tests pass.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

[jira] [Updated] (AMBARI-15023) Agents should not ask for auto-start command details if it has the details

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15023:
---
Description: 
agent keeps on asking for the details for auto execution commands even if it 
has the details. this results in server unnecessarily computing START command 
details when its not needed.

{code}
12 Feb 2016 04:20:13,092  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 - 
NIMBUS is at INSTALLED adding more payload per agent ask from 
c6401.ambari.apache.org
12 Feb 2016 04:20:13,096  INFO [ambari-hearbeat-monitor] 
AmbariManagementControllerImpl:2077 - 
AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
1-0, with cluster-env tags TOPOLOGY_RESOLVED
12 Feb 2016 04:21:13,103  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 - 
NIMBUS is at INSTALLED adding more payload per agent ask from 
c6401.ambari.apache.org
12 Feb 2016 04:21:13,109  INFO [ambari-hearbeat-monitor] 
AmbariManagementControllerImpl:2077 - 
AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
1-0, with cluster-env tags TOPOLOGY_RESOLVED
12 Feb 2016 04:22:13,112  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 - 
NIMBUS is at INSTALLED adding more payload per agent ask from 
c6401.ambari.apache.org
12 Feb 2016 04:22:13,116  INFO [ambari-hearbeat-monitor] 
AmbariManagementControllerImpl:2077 - 
AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
1-0, with cluster-env tags TOPOLOGY_RESOLVED
12 Feb 2016 04:23:13,121  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 - 
NIMBUS is at INSTALLED adding more payload per agent ask from 
c6401.ambari.apache.org
12 Feb 2016 04:23:13,124  INFO [ambari-hearbeat-monitor] 
AmbariManagementControllerImpl:2077 - 
AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
1-0, with cluster-env tags TOPOLOGY_RESOLVED
{code}

  was:agent keeps on asking for the details for auto execution commands even if 
it has the details. this results in server unnecessarily computing START 
command details when its not needed.


> Agents should not ask for auto-start command details if it has the details
> --
>
> Key: AMBARI-15023
> URL: https://issues.apache.org/jira/browse/AMBARI-15023
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-agent
>Affects Versions: 2.2.1
>    Reporter: Sumit Mohanty
>    Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.2.2
>
> Attachments: AMBARI-15023.patch
>
>
> agent keeps on asking for the details for auto execution commands even if it 
> has the details. this results in server unnecessarily computing START command 
> details when its not needed.
> {code}
> 12 Feb 2016 04:20:13,092  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:20:13,096  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> 12 Feb 2016 04:21:13,103  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:21:13,109  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> 12 Feb 2016 04:22:13,112  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:22:13,116  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> 12 Feb 2016 04:23:13,121  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload pe

[jira] [Updated] (AMBARI-15023) Agents should not ask for auto-start command details if it has the details

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15023:
---
Attachment: AMBARI-15023.patch

> Agents should not ask for auto-start command details if it has the details
> --
>
> Key: AMBARI-15023
> URL: https://issues.apache.org/jira/browse/AMBARI-15023
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-agent
>Affects Versions: 2.2.1
>    Reporter: Sumit Mohanty
>    Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.2.2
>
> Attachments: AMBARI-15023.patch
>
>
> agent keeps on asking for the details for auto execution commands even if it 
> has the details. this results in server unnecessarily computing START command 
> details when its not needed.
> {code}
> 12 Feb 2016 04:20:13,092  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:20:13,096  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> 12 Feb 2016 04:21:13,103  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:21:13,109  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> 12 Feb 2016 04:22:13,112  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:22:13,116  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> 12 Feb 2016 04:23:13,121  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:23:13,124  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15023) Agents should not ask for auto-start command details if it has the details

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15023:
---
Attachment: (was: AMBARI-15023.patch)

> Agents should not ask for auto-start command details if it has the details
> --
>
> Key: AMBARI-15023
> URL: https://issues.apache.org/jira/browse/AMBARI-15023
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-agent
>Affects Versions: 2.2.1
>    Reporter: Sumit Mohanty
>    Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.2.2
>
> Attachments: AMBARI-15023.patch
>
>
> agent keeps on asking for the details for auto execution commands even if it 
> has the details. this results in server unnecessarily computing START command 
> details when its not needed.
> {code}
> 12 Feb 2016 04:20:13,092  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:20:13,096  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> 12 Feb 2016 04:21:13,103  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:21:13,109  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> 12 Feb 2016 04:22:13,112  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:22:13,116  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> 12 Feb 2016 04:23:13,121  INFO [ambari-hearbeat-monitor] HeartbeatMonitor:306 
> - NIMBUS is at INSTALLED adding more payload per agent ask from 
> c6401.ambari.apache.org
> 12 Feb 2016 04:23:13,124  INFO [ambari-hearbeat-monitor] 
> AmbariManagementControllerImpl:2077 - 
> AmbariManagementControllerImpl.createHostAction: created ExecutionCommand for 
> host c6401.ambari.apache.org, role NIMBUS, roleCommand START, and command ID 
> 1-0, with cluster-env tags TOPOLOGY_RESOLVED
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 43539: Agents should not ask for auto-start command details if it has the details

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43539/
---

Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sid Wagle.


Bugs: AMBARI-15023
https://issues.apache.org/jira/browse/AMBARI-15023


Repository: ambari


Description
---

Agents should not ask for auto-start command details if it has the details


Diffs
-

  ambari-agent/src/main/python/ambari_agent/ActionQueue.py a4c433d 
  ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 35d048a 

Diff: https://reviews.apache.org/r/43539/diff/


Testing
---

mvn test and manual testing to verify that agent does not ask for command 
details when it already has them.


Thanks,

Sumit Mohanty

[jira] [Created] (AMBARI-15035) Add support for drpc.servers config property in storm config for blueprint based deployments

[Sumit Mohanty (JIRA)]

Sumit Mohanty created AMBARI-15035:
--

 Summary: Add support for drpc.servers config property in storm 
config for blueprint based deployments
 Key: AMBARI-15035
 URL: https://issues.apache.org/jira/browse/AMBARI-15035
 Project: Ambari
  Issue Type: Bug
  Components: blueprints, stacks
Affects Versions: 2.2.1
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
 Fix For: 2.2.2


{{drpc.servers}} is a property in storm yaml config file that captures the drpc 
server hostname. During blueprint based deployments this can be specified by 
the user but some code changes are needed to have the blueprint config 
processor resolve the value to actual hostname.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15035) Add support for drpc.servers config property in storm config for blueprint based deployments

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-15035:
---
Attachment: AMBARI-15035.patch

> Add support for drpc.servers config property in storm config for blueprint 
> based deployments
> 
>
> Key: AMBARI-15035
> URL: https://issues.apache.org/jira/browse/AMBARI-15035
> Project: Ambari
>  Issue Type: Bug
>  Components: blueprints, stacks
>Affects Versions: 2.2.1
>    Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
> Fix For: 2.2.2
>
> Attachments: AMBARI-15035.patch
>
>
> {{drpc.servers}} is a property in storm yaml config file that captures the 
> drpc server hostname. During blueprint based deployments this can be 
> specified by the user but some code changes are needed to have the blueprint 
> config processor resolve the value to actual hostname.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43538: Add checks and alerts when clusterconfigmapping has multiple selected entries for a config type

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43538/#review119121
---


Ship it!




Ship It!

- Sumit Mohanty


On Feb. 13, 2016, 12:51 a.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43538/
> ---
> 
> (Updated Feb. 13, 2016, 12:51 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Myroslav Papirkovskyy, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15034
> https://issues.apache.org/jira/browse/AMBARI-15034
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In clusterconfigmapping table we shouldn't have more than one entries for a 
> config_type with selected=1
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDatabaseHelper.java
>  a078c8a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/CheckDatabaseHelperTest.java
>  e329ab7 
> 
> Diff: https://reviews.apache.org/r/43538/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

[jira] [Created] (AMBARI-15023) Agents should not ask for auto-start command details if it has the details

[Sumit Mohanty (JIRA)]

Sumit Mohanty created AMBARI-15023:
--

 Summary: Agents should not ask for auto-start command details if 
it has the details
 Key: AMBARI-15023
 URL: https://issues.apache.org/jira/browse/AMBARI-15023
 Project: Ambari
  Issue Type: Bug
  Components: ambari-agent
Affects Versions: 2.2.1
Reporter: Sumit Mohanty
Assignee: Sumit Mohanty
Priority: Critical
 Fix For: 2.2.2
 Attachments: AMBARI-15023.patch

agent keeps on asking for the details for auto execution commands even if it 
has the details. this results in server unnecessarily computing START command 
details when its not needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)