Re: support multi-server node

[Di Li]

you can monitor the following jira -  (AMBARI-17126) Provide Ambari Server
HA (active-passive)

On Sat, May 12, 2018 at 4:07 AM,  wrote:

> Hi,all!
>
> Is there any plan to support multi-server node?
>
> When one server down, change to another server, the ambari UI still can
> work,
> which i think so cool.
>
> Thanks
>

Re: where can view zookeeper detail

[Di Li]

Services are not tied to Ambari releases but rather HDP releases. If you
look at Ambari 2.6.2 web UI during the install, it offers you various HDP
stacks to choose from. Because zookeeper is part of  HDP stacks, the
install location is fixed.

If you have your own full stack and expect it to exist under a different
directory than /usr/hdp, you should not use pre-built HDP Zookeeper RPM.
You need to build your own, you can try the Apache open source version.


On Wed, May 2, 2018 at 8:52 AM,  wrote:

> I found that zookeeper pkgs have updated after ambari 2.6.2 released.
>
> What is new about zookeeper pkgs? Where can i get detail about it?
>
> One difference i notice is that it install at /usr/hdp not /usr/bin.
>
> /usr/hdp in the stack_root defined, if i redefine stack_root, the
> installation would fail since wrong installation path.
>
> I think it would be better if install zookeeper pkgs can get stack_root.
>
> Besides, can i build zookeeper pkgs from source codes?
>
> Thanks
>
> Xiang
>

Re: question about service both has master and client

[Di Li]

Restart all = stop, re-install client, start non-client components
Stop then start is just what it indicates.

On Thu, Apr 12, 2018 at 10:30 PM,  wrote:

> Hi!
>
> I define a custom service which include master and client.
>
> In master script, it has install/status/configuare/start/stop function.
>
> In client script, it only has *install* function.
>
> For A node, i install master, and for B node, i install client.
>
> I install and stop as expected, but when exec "restart all" from web,
>
> A node restart as usual, but B node exec *install* function.
>
> In my thought, "restart all" should work like "stop" which only work for
> master/slave node.
>
> Is is a bug or someting import i had missed?
>
> Best Wishes
>

Re: Datanode does not need hdfs.headless.keytab ?

[Di Li]

Great ! Thanks Rob!  I will try it out today and reach out if I hit issues.

Thank you for your help.

Di

On Wed, Apr 4, 2018 at 4:15 PM, Robert Levas <rle...@hortonworks.com> wrote:

> If you would like the HDFS keytab file installed on the same host as your
> component, you can add a reference to that Kerberos identity in your
> Kerberos.json file. Ideally this reference would be added to the
> "identities" section for the specific component.   The declaration would
> look something like this:
>
> {
>   "name": "custom_component_hdfs",
>   "reference": "/HDFS/NAMENODE/hdfs"
> }
>
> For example:
>
>   "components": [
> {
>   "name":  "MY_COMPONENT",
>   ...
>   "identities": [
> ...
> {
>   "name": "custom_component_hdfs ",
>       "reference": "/HDFS/NAMENODE/hdfs"
> }
> ...
>   ],
>   ...
> },
>
> I hope this helps.
>
> Rob
>
>
> On 4/4/18, 4:02 PM, "Di Li" <osji...@gmail.com> wrote:
>
> Hi Rob,
>
> Thanks for the explanation. I don't have issues with DN per se. My case
> falls into the "*since then some services need to create directories
> and
> change permissions on them as the HDFS root user upon installation *
> category
> that you mentioned. I paired my service with DN assuming
> hdfs.headless.keytab would be available.
>
> Is that possible for my service's kerberos.json to define a dependency
> on
> hdfs.headless.keytab ? This way, I can still cohost my component with
> DN
> (hard requirement ...) but still have the keytab available (no need to
> modify HDFS kerberos.json)
>
> Thanks.
>
> Di
>
> On Wed, Apr 4, 2018 at 3:17 PM, Robert Levas <rle...@hortonworks.com>
> wrote:
>
> > The DN does not need to authenticate as the "root" HDFS user to
> perform
> > administrative tasks.
> >
> > A while back, we started an initiative to reduce the exposure of the
> HDFS
> > "root" user due to security concerns.  In doing so, we tightened up
> where
> > we distribute the HDFS keytab file. However since then some services
> need
> > to create directories and change permissions on them as the HDFS
> root user
> > upon installation; and thus, the keytab file is being distributed
> more than
> > some security-conscious people would like.  Until we find a way to
> > centralize the creation of these HDFS resources, we need to deal
> with this.
> >
> > You should not normally need the HDFS keytab file on DN hosts... are
> you
> > having an issue?
> >
> > Rob
> >
> >
> > On 4/4/18, 2:15 PM, "Di Li" <osji...@gmail.com> wrote:
> >
> > Hi folks,
> >
> > I noticed hdfs.headless.keytab only exists on NameNode and HDFS
> client
> > node.
> >
> > Could someone please share some details on why DN does not need
> the
> > hdfs.headless.keytab ? I thought we need it in order for DN to
> work
> > against
> > NN.
> >
> > Any negative impacts if I always include hdfs.headless.keytab on
> the DN
> > nodes  (such as ensure HDFS client always cohost with DNs) ?
> >
> > Thank you.
> >
> > Di
> >
> >
> >
>
>
>

Re: Datanode does not need hdfs.headless.keytab ?

[Di Li]

Hi Rob,

Thanks for the explanation. I don't have issues with DN per se. My case
falls into the "*since then some services need to create directories and
change permissions on them as the HDFS root user upon installation * category
that you mentioned. I paired my service with DN assuming
hdfs.headless.keytab would be available.

Is that possible for my service's kerberos.json to define a dependency on
hdfs.headless.keytab ? This way, I can still cohost my component with DN
(hard requirement ...) but still have the keytab available (no need to
modify HDFS kerberos.json)

Thanks.

Di

On Wed, Apr 4, 2018 at 3:17 PM, Robert Levas <rle...@hortonworks.com> wrote:

> The DN does not need to authenticate as the "root" HDFS user to perform
> administrative tasks.
>
> A while back, we started an initiative to reduce the exposure of the HDFS
> "root" user due to security concerns.  In doing so, we tightened up where
> we distribute the HDFS keytab file. However since then some services need
> to create directories and change permissions on them as the HDFS root user
> upon installation; and thus, the keytab file is being distributed more than
> some security-conscious people would like.  Until we find a way to
> centralize the creation of these HDFS resources, we need to deal with this.
>
> You should not normally need the HDFS keytab file on DN hosts... are you
> having an issue?
>
> Rob
>
>
> On 4/4/18, 2:15 PM, "Di Li" <osji...@gmail.com> wrote:
>
> Hi folks,
>
> I noticed hdfs.headless.keytab only exists on NameNode and HDFS client
> node.
>
> Could someone please share some details on why DN does not need the
> hdfs.headless.keytab ? I thought we need it in order for DN to work
> against
> NN.
>
> Any negative impacts if I always include hdfs.headless.keytab on the DN
> nodes  (such as ensure HDFS client always cohost with DNs) ?
>
> Thank you.
>
> Di
>
>
>

Datanode does not need hdfs.headless.keytab ?

[Di Li]

Hi folks,

I noticed hdfs.headless.keytab only exists on NameNode and HDFS client
node.

Could someone please share some details on why DN does not need the
hdfs.headless.keytab ? I thought we need it in order for DN to work against
NN.

Any negative impacts if I always include hdfs.headless.keytab on the DN
nodes  (such as ensure HDFS client always cohost with DNs) ?

Thank you.

Di

Re: Could custom client service add start/stop?

[Di Li]

Clients do not have start/stop. Start and stop shown on the UI are reserved
actions from Ambari, you can't overwrite them. You can add custom actions
(so long as they show up on the UI) to do similar things, but you will need
to give them  different names.

On Thu, Mar 29, 2018 at 3:07 AM,  wrote:

> Hi!
>
> I follow https://cwiki.apache.org/confluence/display/AMBARI/
> Defining+a+Custom+Service and add a custom client service.
>
> I want add start/stop as customCommand, but i do not know how and when to
> call them.
>
> Also, is there any possible to start/stop client service at web UI like
> master service?
>

Re: custom master service can not stop

[Di Li]

Not much to go with based on your example, as I assume you removed
sensitive code that actually does the stop. All I can suggest is to make
sure the stop actually stops the process and don't sit in a loop forever.
Also, it's not the stop that tells the UI a component is "stopped" or not,
it's the status call.

On Thu, Mar 29, 2018 at 8:33 AM,  wrote:

> After waiting about 1 hour, it always print log as below and failed
> finally:
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> 2018-03-29 20:15:35,835 - Waiting for actual component stop
>
> - Original Message -
> From: "xiang dai" 
> To: dev@ambari.apache.org
> Sent: Thursday, March 29, 2018 8:14:47 PM
> Subject: custom master service can not stop
>
> Hi!
>
> I add a master service as below:
>
> 2.0
> 
> 
> YUM
> YUM
> Yum Service
> 1.0.0
> 
> 
> YUM_MASTER
> Yum Master
> MASTER
> 1
> 
> scripts/master.py
> PYTHON
> 600
> 
> 
> 
> YUM_SLAVE
> Yum Slave
> SLAVE
> 1+
> 
> scripts/slave.py
> PYTHON
> 600
> 
> 
> 
> 
> 
> any
> 
> 
> 
> 
> 
>
> master.py:
>
> class Master(Script):
> def install(self, env):
> print 'Install Yum'
>
> def start(self, env):
> print 'Start Yum'
>
> def stop(self, env):
> print 'Stop Yum'
>
> def status(self, env):
> print 'Test Yum'
>
> if __name__ == "__main__":
> Master().execute()
>
> slave.py:
>
> class Slave(Script):
> def install(self, env):
> print 'Wait for Yum_Master install finish...'
>
> def start(self, env):
> print 'Wait for Yum_Master install finish...'
>
> def stop(self, env):
> print 'Wait for Yum_Master stop finish...'
>
> def status(self, env):
> print 'Wait for Yum_Master test finish...'
>
> if __name__ == "__main__":
> Slave().execute()
>
> I install it successfully, but when stop on the web UI, the master can not
> stop:
>
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> Test Yum
> 2018-03-29 19:12:54,068 - Waiting for actual component stop
>
> It hang at 35%.
>
> I follow https://cwiki.apache.org/confluence/display/AMBARI/
> Defining+a+Custom+Service, did i miss something import?
>
> Best wishes
>

Re: Bulk remove components from multiple hosts

[Di Li]

great, thx!

On Sat, Mar 24, 2018 at 6:08 AM, Attila Magyar <amag...@hortonworks.com>
wrote:

> Yes, for example this will delete all datanodes from host1,host2,host3
>
> curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE "
> http://AMBARI_SERVER/api/v1/clusters/CLUSTERNAME/host_
> components?HostRoles/component_name.in(DATANODE)&
> HostRoles/host_name.in(host1,host2,host3)"
>
> On 3/23/18, 8:58 PM, "Di Li" <osji...@gmail.com> wrote:
>
> Hello folks,
>
> Is there REST API support for bulk removing components from multiple
> hosts ?
>
> I know Ambari support bulk install components to multiple hosts and
> bulk
> removing of empty hosts from a cluster. It seems to me that it only
> makes
> sense if we can also bulk remove components as well.
>
> Thanks.
>
> Di
>
>
>

Bulk remove components from multiple hosts

[Di Li]

Hello folks,

Is there REST API support for bulk removing components from multiple hosts ?

I know Ambari support bulk install components to multiple hosts and bulk
removing of empty hosts from a cluster. It seems to me that it only makes
sense if we can also bulk remove components as well.

Thanks.

Di

Re: how to exec shell script and output its log

[Di Li]

and this is after you had tried the *logoutput *flag in Execute(... ?

On Fri, Mar 23, 2018 at 6:49 AM,  wrote:

> Hi!
>
> I have some shell scripts and want to call them in service python scripts.
>
> I saw an example which use Execute to call shell script like
> Execute("/path/do_something").
>
> But i notice that if the log level is INFO, the shekllscript own log can
> not be printed, only change log
> level to DEBUG can show log.
>
> Is there a better way to call shell script and print its log?
>

Re: could retry install do uninstall first

[Di Li]

I don't think scripts.py has the uninstall fun interface, so don't expect
Ambari to magically become aware of a func you wrote in your code and start
calling it. You can obviously implement the Decommission custom command or
include your own custom command in metainfo.xml

On Thu, Mar 22, 2018 at 9:29 PM, <xiang@sky-data.cn> wrote:

> So when and who call the uninstall function in the service script?
>
> - Original Message -----
> From: "Di Li" <osji...@gmail.com>
> To: dev@ambari.apache.org
> Sent: Thursday, March 22, 2018 9:17:32 PM
> Subject: Re: could retry install do uninstall first
>
> First of all, Ambari's "Delete" is more of "unregistering" a component from
> Ambari's management. It does not remove component binaries.
>
> Retry install does not uninstall. In the Ambari paradigm, "adding" a
> component to a host has three parts, install the binary, config, start. The
> "install" option on the host menu calls the install method of that
> component to install the binary and local config (you should try to make
> the config step idempotent).
>
> On Thu, Mar 22, 2018 at 8:48 AM, <xiang@sky-data.cn> wrote:
>
> > Hi!
> >
> > I add some service and want to use amabri to install and manage them, but
> > i have some question about retry install.
> >
> > Does retry install do uninstall first and then install?
> >
> > If not, i think failed install would make next install failed.
> >
> > Also, uninstall manually is not cool.
> >
>

Re: why stack_root does not exist

[Di Li]

It doesn't create that directory, as ambari does not know what could be
used. The service RPMs create directories upon installation and Ambari
reads its content. That's about it.

On Wed, Mar 21, 2018 at 10:11 PM,  wrote:

> HI!
>
> I define a new stack called DX and change the stack_root to {DX: /usr/dx}
> in the cluster_env.xml, the old is {HDP: /usr/hdp}.
>
> But when i start install, i find below error:
>
> WARNING 2018-03-22 10:05:49,709 base_alert.py:138 -
> [Alert][ambari_agent_disk_usage] Unable to execute alert. [Errno 2] No
> such file or directory: '/usr/dx'
>
> Does it mean ambari can not auto mkdir this one?
>
> If i miss something important, please tell me.
>
> Best Wishes
>

Re: could retry install do uninstall first

[Di Li]

First of all, Ambari's "Delete" is more of "unregistering" a component from
Ambari's management. It does not remove component binaries.

Retry install does not uninstall. In the Ambari paradigm, "adding" a
component to a host has three parts, install the binary, config, start. The
"install" option on the host menu calls the install method of that
component to install the binary and local config (you should try to make
the config step idempotent).

On Thu, Mar 22, 2018 at 8:48 AM,  wrote:

> Hi!
>
> I add some service and want to use amabri to install and manage them, but
> i have some question about retry install.
>
> Does retry install do uninstall first and then install?
>
> If not, i think failed install would make next install failed.
>
> Also, uninstall manually is not cool.
>

Re: Questions aboute install order

[Di Li]

metainfo only list direct dependencies. RCO only dictates what a dependent
should do when it INSTALL/START if its dependencies are also INSTALL-*ing*,
START-*ing*, etc.  You can verify this by running a start all, where high
level services waits for their dependencies to start.  It does not,
howerver, let Ambari install/start dependencies for you. You can easily
verify this by starting a high level service such as HBASE and has RCO
depdencies on HDFS.

On Wed, Mar 21, 2018 at 4:11 AM,  wrote:

> Hi!
>
> I know that when choose services, i can define service dependence in
> service's metainfo.xml,
> at the same time, i can define order in role_command_order.json.
>
>
> I have some questions:
>
> 1) why the dependence in metainfo can not auto covert into
> role_command_order which i think it is very cool?
> A depend on B, so install B before install A, isn't it?
>
> 2) If A depend on B and C, D depend on A and E, i define in
> role_command_order.json as below:
>
> A_CLIENT_INSTALL: [B_CLIENT_INSTALL, C_CLIETN_INSTALL],
> D_CLIENT_INSTALL: [A_CLIENT_INSTALL, E_CLIETN_INSTALL],
>
>
> so the install order is B,C,A,E,D , right?
>

Re: how to define stacks like 2.6.5.0

[Di Li]

If you look at the various HDP 2.6.x releases, you will see that nothing
version-wise was added to the HDP-2.6 directory. The stack directory
already covers the major and minor version, while all releases happened at
the patch and hotfix level, such as 2.6.'2'.'0'.  In other words, it's up
to your RPMs to install binaries to a new version dir under /usr/hdp.
Ambari just uses service definitions from the HDP-2.6 stack (this also
implies that HDP 2.6 stack Python scripts, config XML, etc are compatible
with all HDP 2.6.x releases)

On Tue, Mar 20, 2018 at 9:14 AM, <xiang@sky-data.cn> wrote:

> > HDP 2.6.x releases share the same stack definitions from the HDP-2.6
> directory
>
> Could you share more details about it? How can i add 2.6.x in XXX-2.6 dir?
>
> - Original Message -
> From: "Di Li" <osji...@gmail.com>
> To: dev@ambari.apache.org
> Sent: Tuesday, March 20, 2018 9:03:51 PM
> Subject: Re: how to define stacks like 2.6.5.0
>
> Hello,
>
> The stack directory can be name up to four digits if I am not mistaken
> (albeit I only tried 3 digit directory name in practice). HDP 2.6.x
> releases share the same stack definitions from the HDP-2.6 directory. For
> your need, simply add a 3.1.1 dir. If you meant to have 3.0.x, you can
> either update your 3.0 stack to make it compatible with the new release or
> create a new stack (but that'd mostly be an overkill)
>
> On Tue, Mar 20, 2018 at 2:01 AM, <xiang@sky-data.cn> wrote:
>
> > Hi!
> >
> > I has defined a new stack like 3.0.0 and there is a dir called 3.0.0, but
> > i find that HDP has 2.6.5.0 but only has 2.6 dir.
> >
> > How can i also defined a new stack like 3.1.1 in this way?
> >
>

Re: how to define stacks like 2.6.5.0

[Di Li]

Hello,

The stack directory can be name up to four digits if I am not mistaken
(albeit I only tried 3 digit directory name in practice). HDP 2.6.x
releases share the same stack definitions from the HDP-2.6 directory. For
your need, simply add a 3.1.1 dir. If you meant to have 3.0.x, you can
either update your 3.0 stack to make it compatible with the new release or
create a new stack (but that'd mostly be an overkill)

On Tue, Mar 20, 2018 at 2:01 AM,  wrote:

> Hi!
>
> I has defined a new stack like 3.0.0 and there is a dir called 3.0.0, but
> i find that HDP has 2.6.5.0 but only has 2.6 dir.
>
> How can i also defined a new stack like 3.1.1 in this way?
>

Re: how to enable warning span without click

[Di Li]

Hi, can you remove read-only and try again ?

On Mon, Mar 12, 2018 at 2:19 AM,  wrote:

>
> Hi !
>
> As the pic shows, the waring span can show without click, but i add a
> customized service with config, the waring span can not show.
>
> Here is my config xml:
>
> 
>   
> content
> Lvm Disk Config
> 
>
> 
> 
> 
>   content
>   true
>   true
>   true
>   false
>   true
> 
> 
>   
> 
>
>
> How can i do this? It seems that i miss something important.
>
> Could someone help me?
>

No Jenkins job results for pull request after 24 hours

[Di Li]

Hello folks,

My PR doesn't have any update of its jenkins job status after 24 hours
while many other PRs created around the same time as mine got their build
results back within hours of creation.

Is there a way to restart the jenkins job associated with a pull request or
at least check if it's run ?

The Jenkins job web UI only lists the most recent ones and many at the top
are just pending ones that do show associated PR id.
https://builds.apache.org/view/A/view/Ambari/job/Ambari-Github-PullRequest-Builder/

Re: Question on integrating Custom service / Management pack

[Di Li]

Hello Steve,

>From the context of Ambari (express upgrade, rolling upgrade), currently
you really can't upgrade a custom service (tied to a give HDP stack)
without upgrading the stack. This is because EU and RU run at the stack
level, your custom service , from Ambari's point of view, is part of the
stack.

Ambari 3.0 (much later this year) will support mpack based EU/RU, where you
can package your service as a stand along stack (single service stack ?)
that has a dependency on say a given HDP mpack, then you can upgrade your
mpack without touching HDP.

But as for Ambari 2.x, if your service is simply a custom service tied to
an HDP stack, you can't use Ambari RU/EU to *only *upgrade your service.



On Thu, Feb 15, 2018 at 1:30 PM, Steve Varnau 
wrote:

> Hi,
>
> I have worked on defining a custom service and the management pack to plug
> into Ambari. (In this case for Apache Trafodion project.)
>
> It is working pretty well, but I'm confused on how it can handle upgrading
> the custom service independently of the HDP stack. A given release of
> trafodion is compatible with multiple versions of HDP (2.3, 2.4, 2.6), so
> the mgmt pack is configured to say Traf2.1 works with all three, but then
> Traf2.2 is also compatible with all three HDP.
>
> In Ambari, trying to register a new version of the stack, I try to change
> only the URL for trafodion, leaving the URLs for HDP and HDP-UTILS the
> same, but that is not allowed.  So, I'm confused about how Ambari is
> expecting custom services to be upgraded.
>
> Thanks for any hints/pointers.
>
> --Steve
>
>

Re: [VOTE] Migrate to gitbox

[Di Li]

+1

On Fri, Dec 8, 2017 at 1:58 PM, Jayush Luniya 
wrote:

> +1
>
> On 12/8/17, 10:54 AM, "Swapan Shridhar"  wrote:
>
> >
> >+1.
> >
> >
> >Thanks.
> >
> >Regards,
> >Swapan.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >On 12/8/17, 8:00 AM, "Aravindan Vijayan" 
> wrote:
> >
> >>+1 for gitbox migration.
> >>
> >>--
> >>Thanks and Regards,
> >>Aravindan Vijayan
> >>
> >>On 12/8/17, 7:14 AM, "Nishant Bangarwa" 
> >>wrote:
> >>
> >>+1
> >>--
> >>Nishant Bangarwa
> >>Hortonworks
> >>(M): +91-9729200044
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>On 12/8/17, 5:50 PM, "Sean Roberts" 
> wrote:
> >>
> >>>+1billion!
> >>>
> >>>--
> >>>Sean Roberts
> >>>@seano
> >>>
> >>>
> >>>On 12/8/17, 12:09 PM, "Jonathan Hurley" 
> >>wrote:
> >>>
> >>>+1
> >>>
> >>>> On Dec 7, 2017, at 6:48 PM, Vivek Ratnavel
> >> wrote:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> I am starting this vote to migrate Apache Ambari project from
> >>git-wip
> >>>> repository to gitbox, which allows a deeper integration with
> >>github
> >>>> features. Moving to gitbox will allow committers to merge,
> >>close or edit
> >>>> pull requests in the github UI.
> >>>>
> >>>> Other advantages:
> >>>>
> >>>>   - JIRA linking can be enabled on request to the Apache
> >>Infra team, which
> >>>>   will automatically link a PR with its corresponding JIRA in
> >>>>   https://issues.apache.org/jira
> >>>>   - It will make our code review process cleaner and more
> >>efficient when
> >>>>   compared to the Review Board.
> >>>>   - Opens up a lot of opportunities on leveraging the rich
> >>set of github
> >>>>   APIs and web hooks to develop tools that can help monitor
> >>the open PRs,
> >>>>   trigger a jenkins job to run unit tests, etc..
> >>>>
> >>>>  [ ] +1, Migrate Ambari repository to gitbox
> >>>>  [ ] -1, Keep git-wip + github read-only mirror
> >>>>
> >>>> Regards,
> >>>> Vivek Ratnavel
> >>>
> >>>
> >>>
> >>
> >>
>
>

Design doc review for AMBARI-20944

[Di Li]

Hello folks,

I have posted a design doc on a GUI feature aiming at better user
experience on blueprint based cluster deployment. Please feel free to
review the design doc posted in the following JIRA and let me know of your
feedback.

   https://issues.apache.org/jira/browse/AMBARI-20944

Thank you.

Di Li

Re: Purpose a Titan/Ambari integration

[Di Li]

Could you please post your design doc to the JIRA to show what you are
trying to accomplish ?

On Tue, May 23, 2017 at 9:34 PM, Hugo Cao  wrote:

> Hi guys,
>
> I did some work for the Titan/Ambari integration in the common-service, to
> make Ambari install, uninstall and manage Titan which is designed to
> support the processing of the scaling graph. I would like to see if we are
> interested in putting it in the community.
>
> I have also opened a Jira here at https://issues.apache.org/
> jira/browse/AMBARI-21098
>
> Thanks
>
> Hugo
>

Re: [VOTE] Apache Ambari 2.5.1 RC0

[Di Li]

+1 for release-2.5.1-rc0

On Tue, May 23, 2017 at 11:39 AM, Nate Cole  wrote:

> +1 for rc0
>
> On 5/22/17, 2:54 PM, "Aravindan Vijayan"  wrote:
>
> Hello all,
>
> I have created an apache-ambari-2.5.1 release candidate.
>
> GIT source tag (release-2.5.1-rc0) https://git-wip-us.apache.org/
> repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.5.1-rc0
>
> Staging site: http://home.apache.org/~avijayan/apache-ambari-2.5.1-
> rc0/
>
> PGP release keys (signed using 088372a9) -
> http://pgp.mit.edu:11371/pks/lookup?search=0x088372A9=vindex
>
> One can look into the issues fixed in this release at
> https://issues.apache.org/jira/browse/AMBARI/fixforversion/12340256/?
> selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel
>
> Vote will be open for 72 hours.
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> My vote: +1
>
> --
> Thanks and Regards,
> Aravindan Vijayan
>
>
>

Design - Generate blueprint via Ambari Installer UI

[Di Li]

Hello folks,

I am proposing a design for a new UI feature that allows Ambari Installer
UI to generate a blueprint (so user has the options to Deploy a cluster or
Create a blueprint)

Please feel free to review the design doc attached to AMBARI-20698 and let
me know your feedback: https://issues.apache.org/jira/browse/AMBARI-20698

Thanks.

Di Li

Ambari Agent ignores Ambari server ssl cert by default ?

[Di Li]

Hello folks

I am trying to understand how Ambari agent handles the self-sign ssl cert
generated by the Ambari server during default installation.

Ambari agent security.py has this code here that sets
"cert_reqs=ssl.CERT_NONE"
>>> self.sock = ssl.wrap_socket(sock, cert_reqs=ssl.CERT_NONE)

Doesn't this mean Ambari Agent ignores Ambari server ssl cert unless
customers manually setup two-way ssl authentication ?  How does Ambari
Agent use the self-sign cert ( it looks like to me that it doesn';t use it
at all) ?



Thank you.

Di Li

Re: Crearting Hadoop Cluster with NameNode HA (unattended) using only REST APIs

[Di Li]

Apache WIKI has the following topic show casing how you may deploy a NN HA
cluster via blueprint.

Blueprint deployment is via REST API and can be automated so long as you
have a rest client.

https://cwiki.apache.org/confluence/display/AMBARI/Blueprint+Support+for+HA+Clusters#BlueprintSupportforHAClusters-ConfiguringActiveandStandbyNameNodes

On Fri, Apr 7, 2017 at 2:08 AM, Sandy  wrote:

> Currently through AMBARI UI we need to take following route to create a
> cluster
> 1.  Create a Cluster (only ZK and HDFS) with Non-HA NameNode
> 2.  Make NameNode HA (using some manual steps in-between)
>
> Is there a way to achieve the same using REST APIs so that we don't have to
> do any manual steps in-between (a solution that can be used for multiple
> environments where each env may have different number of machines)?
>
> --
>
> *Sandeep Kumar,*
>  Mobile +91-9866507368
>
> *“Happiness is not a destination, It is the journey”*
>

Why RU doesn't run update Kerberos descriptors as the first step ?

[Di Li]

Hello folks,

I am testing rolling ugprade in Ambari trunk code and I noticed that for
rolling upgrade, update kerberos descriptors step is after Zookeeper,
Ranger, Ranger KMS and Kafka.  For rolling downgrade, update Kerberos
descriptors step happens at the end of the entire downgrade just before the
finalize downgrade step.

I thought updating Kerberos descriptors should have happened before
starting components. For example, express upgrade/downgrade runs that step
before starting core and high level components.

Are there any particular reasons that RU runs Kerberos descriptor update
step the way it does ?

Thanks.

Di

Re: Why are AMS and LogSearch not part of RU/EU

[Di Li]

Got it thanks Jonathan and Alejandro.

On Fri, Feb 17, 2017 at 1:47 PM, Alejandro Fernandez <
afernan...@hortonworks.com> wrote:

> Only services that support side-by-side bits can be orchestrated.
> AMS and LogSearch don't have hdp-select to change the symlink of the
> version that is current and can only update the existing bits.
>
> Thanks,
> Alejandro
>
> On 2/17/17, 8:25 AM, "Jonathan Hurley" <jhur...@hortonworks.com> wrote:
>
> >They're not going to be part of those stacks for much longer. As Ambari
> >moves toward management packs in 3.0, they'll be their own stack
> >essentially.
> >
> >To your question - services can provide their own upgrade packs. Since
> >these services are not part of the actual HDP stack, they should not be a
> >part of the HDP pre-defined orchestration. Additionally, they are not
> >"version-able", so Ambari wouldn't know to schedule them. If they provide
> >their own upgrade pack, they can certainly be restarted as part of an HDP
> >orchestration.
> >
> >> On Feb 17, 2017, at 11:21 AM, Di Li <osji...@gmail.com> wrote:
> >>
> >> Hello fellow Ambari developers,
> >>
> >> I noticed no RU/EU upgrade XML files include AMS or LogSearch. I
> >>understand
> >> they are part of the Ambari repo not the HDP/HDP-UTILS repo, but they
> >>are
> >> still part of the HDP stacks.
> >>
> >> Could any one tell me whether there are any particular reasons that AMS
> >>and
> >> LogSearch are left out of RU/EU, please ?
> >>
> >>
> >> Thank you.
> >>
> >> Di
> >
> >
>
>

Why are AMS and LogSearch not part of RU/EU

[Di Li]

Hello fellow Ambari developers,

I noticed no RU/EU upgrade XML files include AMS or LogSearch. I understand
they are part of the Ambari repo not the HDP/HDP-UTILS repo, but they are
still part of the HDP stacks.

Could any one tell me whether there are any particular reasons that AMS and
LogSearch are left out of RU/EU, please ?


Thank you.

Di

Does Ambari web UI use JSON files from ambari-web/app/assets/data ?

[Di Li]

Hello folks,

Does Ambari web UI use JSON files from ambari-web/app/assets/data ? I
noticed they are installed to /usr/lib/ambari-server/web/data on a cluster.
For example, HBASE_SUMMARY.json from
ambari-web/app/assets/data/widget_layouts is installed
to /usr/lib/ambari-server/web/data/widget_layouts.

I thought they were for UI unit testing only as I see that the JSON files
in ambari-web/app/assets/data are used as mock URLs for the real REST API
URL  submitted by the web UI.

Does Ambari web UI uses those JSON files in the assets dir for any real UI
functions?

Thanks

Di.

Re: Can "Delete" button on Ambari UI be disabled ?

[Di Li]

I see, good to know, thanks Jaimin and Richard.

On Mon, Feb 6, 2017 at 7:50 PM, Richard Zang <rz...@hortonworks.com> wrote:

> Hi Di,
>
> From an end user perspective, you can also use the RBAC features in admin
> view to assign “Cluster Operator” or lower role to the users you don’t want
> add/delete service available.
>
> Richard
>
>
> On 2/6/17, 2:12 PM, "Jaimin Jetly" <jai...@hortonworks.com> wrote:
>
> Hi Di,
>
> Existence of "Delete" button for a service is not stack driven.
> Although on clicking delete action, UI asks end-user to first stop and
> delete any service that depends on "being deleted" service.
> This logic is stack driven.
>
> So when user tries to delete a zookeeper service, ambari-web will ask
> user to first delete other "many" services like HDFS, YARN, etc first since
> they are marked to require zookeeper service in their service definition.
>
> Again these restrictions are stack driven but these are made by
> ambari-web client and so user can always use API to delete a service
> directly. In this case all "Delete Service" API checks is that no host
> component of service should be in running state.
>
>
> -- Thanks
> Jaimin
> 
> From: Di Li <osji...@gmail.com>
> Sent: Monday, February 06, 2017 1:43 PM
> To: dev@ambari.apache.org
> Subject: Can "Delete" button on Ambari UI be disabled ?
>
> Hello folks,
>
> Can "Delete" button on Ambari UI be disabled (via some stack service
> configuration ) ?  For example, disable the Delete button for
> Zookeeper as
> many services depend on it ?
>
> Thanks.
>
> Di
>
>
>
>

Can "Delete" button on Ambari UI be disabled ?

[Di Li]

Hello folks,

Can "Delete" button on Ambari UI be disabled (via some stack service
configuration ) ?  For example, disable the Delete button for Zookeeper as
many services depend on it ?

Thanks.

Di

How to tell when Ambari server DB was upgraded

[Di Li]

Hello folks,

Are there any indications in Ambari server DB on when it has been upgraded
by the ambari-server upgrade command ?

For example, if I upgrade Ambari from version 2.2.0 to 2.4.2 at "Thu Feb  2
07:54:49 PST 2017" , will this timestamp/date-time info store in the
database ?

I would like to know this information in order to check if certain
operations had run (similar to the service check validation done by the
express upgrade prechecker)

Di

Re: the version number of the latest trunk

[Di Li]

you may use 2.5.0.0-SNAPSHOT to make it the same version as branch-2.5, or
you may also just omit *-DnewVersion* altogether so that maven will use
default value "2.0.0.0-SNAPSHOT" defined in pom.xml files.


On Tue, Dec 20, 2016 at 3:40 AM, Takanobu Asanuma 
wrote:

> Hi experts,
>
> I want to build the latest trunk branch. I read the Ambari Development
> document but I couldn't understand what version number should be set to
> "-DnewVersion". How to determine the "-DnewVersion" number for the latest
> trunk?
>
> Thanks,
> Takanobu Asanuma
>

Design doc review for JIRA AMBARI-18102

[Di Li]

Hello folks,

I have attached design doc and use cases for a blueprint feature that I am 
proposing via AMBARI-18102
[AMBARI-18102] Preinstall check for blueprint-based cluster deployment - ASF 
JIRA


Please feel free to review the document and let me know of your comments

Thank you.

Di

How to build ambari-logsearch-solr RPM

[Di Li]

Hello folks,

I am trying to build Ambari RPMs off the trunk code to install a dev cluster 
with the Log Search service. 

Can someone please tell me how to build ambari-logsearch-solr RPM ? I am able 
to build the rest of the ambari-logsearch-* RPMs as shown below.

  ambari-logsearch-2.4.0.0-SNAPSHOT1.noarch.rpm
  ambari-logsearch-appender-2.4.0.0-SNAPSHOT1.noarch.rpm
  ambari-logsearch-assembly-2.4.0.0-SNAPSHOT1.noarch.rpm
  ambari-logsearch-logfeeder-2.4.0.0-SNAPSHOT1.noarch.rpm
  ambari-logsearch-portal-2.4.0.0-SNAPSHOT1.noarch.rpm
  ambari-logsearch-solr-client-2.4.0.0-SNAPSHOT1.noarch.rpm
Based on the following piece of code in ambari-logsearch-assembly pom.xml, It 
seems to me that I need solr in order to build the ambari-logsearch-solr RPM. 
BUt I have no idea how to get the solr code.
         logsearch-solr
    package
    
  rpm
    
    
  Development
  ${solr.package.name}
  
    
  ${solr.mapping.path}
  
    
  ${project.build.directory}/solr

Thanks.

Di

AmbariJpaLocalTxnInterceptor throws warnings about synthetic methods in DAO classes

[Di Li]

Hello folks,

When I ran ambari-upgrade -v on an Ambari 2.1.0 cluster to upgrade Ambari to 
version 2.2.0, I got the following warnings in the ambari server log. Where it 
says Guice detected some warning related to AmbariJpaLocalTxnInterceptor.

Jun 08, 2016 8:33:18 AM com.google.inject.internal.ProxyFactory 
WARNING: Method [public void 
org.apache.ambari.server.orm.dao.ClusterVersionDAO.create(java.lang.Object)] is 
synthetic and is being intercepted by 
[org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor@44da7eb3]. This 
could indicate a bug.  The method may be intercepted twice, or may not be 
intercepted at all.

Is this something to be concerned with ? If not, what should I do if I want to 
modify AmbariJpaLocalTxnInterceptor to get rid of the warnings ?
Thanks.

Di

No agent heart beats after adding TLSv1 and TLSv1.1 to security.server.disabled.protocols

[Di Li]

Hello folks,

I installed Ambari 2.2.0 on a single node cluster. I added TLSv1 and TLSv1.1 to 
the security.server.disabled.protocols list in ambari.properties. Once I have 
done that I found agents couldn't connect to the ambari server anymore, with 
the following error in ambari server log.

Does anyone know what I need to do to only allow TLSv1.2 ?

DEBUG [qtp-ambari-agent-54] nio:714 - EOF
org.eclipse.jetty.io.EofException
    at org.eclipse.jetty.http.HttpParser.fill(HttpParser.java:1050)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:280)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at 
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
    at 
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
    at 
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:745)
javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1 not 
enabled or not supported
    at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1431)
    at 
sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
    at 
sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214)
    at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186)
    at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
    at org.eclipse.jetty.io.nio.SslConnection.wrap(SslConnection.java:465)
    at 
org.eclipse.jetty.io.nio.SslConnection.process(SslConnection.java:386)
    at 
org.eclipse.jetty.io.nio.SslConnection.access$900(SslConnection.java:48)
    at 
org.eclipse.jetty.io.nio.SslConnection$SslEndPoint.fill(SslConnection.java:715)
    at org.eclipse.jetty.http.HttpParser.fill(HttpParser.java:1044)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:280)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at 
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
    at 
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
    at 
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:745)


Thanks.

Di

Adding new utility scripts to "contrib" folder under the root ambari dir

[Di Li]

Hello folks,

I understand this is really a novice question. My question is: Should I add the 
scripts to the contrib folder (as a sub-project) if I would like to include 
some Python scripts that end users can use?

I would like to build a script that user can run manually for host checks prior 
to blueprint install. I wonder if I should create a project under the contrib 
dir and put the scripts there. I can't figure out a proper location in Ambari 
to include this type of utility scripts.

Thank you .
Di

Request Edit role to Ambari Wiki for editing privileges

[Di Li]

Hello folks,

I am an Ambari committer and I would like to request Edit role to Ambari Wiki 
for editing privileges.  My Confluence ID: dili

Thank you.
Di Li

Design review for AMBARI-12697

[Di Li]

Hello folks,

Could you please help review the design doc attached to 
https://issues.apache.org/jira/browse/AMBARI-12697 ?

It's a UI feature I am proposing that displays all the upgrade and downgrade 
attempts done against the cluster. So that customers can review the 
upgrade/downgrade history much the same as how they may review all the 
background requests that Ambari server handles. (Users can see the background 
requests list via the "ops" link at the top of the page, right besides the 
cluster name).

Please let me know of your comments. If all is good, I will start coding it.

Thanks

Di.

"detected unhandled Python exception" in RHEL's /var/log/messages log

[Di Li]

Hello folks,

I noticed that everytime Ambari Python script fails with unexpected exception. 
A "detected unhandled Python exception" error message is logged in 
/var/log/messages.  For example, the Install Package step may fail due to 
corrupted HDP yum repo cache. When that happened, I saw the following error 
message logged in /var/logs/messages.
Apr 11 11:17:55 bdavm1251 abrt: detected unhandled Python exception in 
'/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py'

It seemed to me that the "unhandled" exception was from the execute() method in 
script.py as shown below. where the method has "try" and "finally" but no 
"expect" part. The script used to catch the exception. The pieces was removed 
via AMBARI-11242 as part of the Ambari server/agent performance optimization.

Could someone explain to me why the "expect" was removed ? Should "expect" be 
added back so arbt on RHEL will not log errors into /var/log/messages ?

try:
  method = self.choose_method_to_execute(self.command_name)
  with Environment(self.basedir, tmp_dir=Script.tmp_dir) as env:
    env.config.download_path = Script.tmp_dir
    method(env)
    finally:
  if self.should_expose_component_version(self.command_name):
    self.save_component_version_to_structured_out()


Thanks.

Di

No killing topologies manual step for Storm in HDP 2.3 rolling upgrade

[Di Li]

Hello folks,

I noticed the manual step "Before continuing, please deactivate and kill any 
currently running topologies." is in upgrade-2.3.xml and 
nonrolling-upgrade-2.3.xml in HDP/2.2/upgrades. But starting HDP 2.3 stack (and 
HDP 2.4, etc) only the nonrolling-upgrade XML file has that manual step, the 
rolling ones don't have it anymore.

Is the step not needed  anymore by rolling upgrade?

Di

Re: Review Request 44457: AMBARI-15310: Host Service Summary Page does not display long service names well

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44457/#review122390
---




ambari-web/app/templates/main/host/details/host_component.hbs (line 40)
<https://reviews.apache.org/r/44457/#comment184361>

In additional to a fixed width (number of chars) field, it'd also be much 
more flexible to use a flyout. User simply mouses over the component name and 
the flyout  appears with the fullname. User mouses away from the component 
name, the flyout disappears.


- Di Li


On March 7, 2016, 9:57 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44457/
> ---
> 
> (Updated March 7, 2016, 9:57 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Di Li.
> 
> 
> Bugs: AMBARI-15310
> https://issues.apache.org/jira/browse/AMBARI-15310
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In Ambari UI, navigate to Hosts and click on the hostname where the service 
> is installed. The summary page displays the list of services installed.
> Observe that longer service names does not display well. The refresh icon 
> etc. spills to the next line (see attachment "Original spacing with long 
> service name").
> Also, note the indentation of the last button for Clients does not align well 
> with the buttons above it.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/styles/application.less 755dff2 
>   ambari-web/app/templates/main/host/details/host_component.hbs 60031de 
>   ambari-web/app/templates/main/host/summary.hbs 17a0b69 
> 
> Diff: https://reviews.apache.org/r/44457/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> This issue is simialr to AMBARI-11670 where the spacing for each component 
> was fixed in a  tag.
> The fix here also uses a similar approach where every component is surrounded 
> by a  tag and the width is fixed for them so that the appear in their 
> respective positions regardless of the length of service names. I have also 
> adjusted the width of the left  component with "span8" instead of 
> "span7" and the right  component of the button to "span4" instead of 
> "span5" to better adjust the spacing.
> 
> TESTS:
> No tests are written for these changes as the fix involves updates in the 
> stylesheet.
> However, the changes were tested by zooming in and out and checking if the 
> icons and names were contained in their respective blocks and not overlapping 
> with other  blocks.
> 
> 
> The following is the result of running ambari-web tests:
> 
>   24565 tests complete (28 seconds)
>   145 tests pending
> 
> 
> File Attachments
> 
> 
> AMBARI-15310 (patch-2)
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/07/5ade9968-bb9d-4d61-8362-740fb24c591f__AMBARI-15310_Mar07.patch
> 
> 
> Thanks,
> 
> Keta Patel
> 
>

Re: Review Request 44457: AMBARI-15310: Host Service Summary Page does not display long service names well

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44457/#review122388
---




ambari-web/app/templates/main/host/details/host_component.hbs (line 30)
<https://reviews.apache.org/r/44457/#comment184359>

How is the service name displayed when the component name is extra long, in 
relation to the new divs added ?

The service name should be visible so that user can navigate back to the 
corresponding service.

What about fixing the width of the component name and trancate it with ... 
when it reaches the limite.

For example, ExtraLongZookeeperComponentName would be dispalyed as 

ExtraLongZook.../Zookeeper

The ExtraLongZookeeperComponentName is still displayed on zookeeper's 
dashboard where user can hover over to reveal the host name, yes ?


- Di Li


On March 7, 2016, 9:57 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44457/
> ---
> 
> (Updated March 7, 2016, 9:57 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Di Li.
> 
> 
> Bugs: AMBARI-15310
> https://issues.apache.org/jira/browse/AMBARI-15310
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In Ambari UI, navigate to Hosts and click on the hostname where the service 
> is installed. The summary page displays the list of services installed.
> Observe that longer service names does not display well. The refresh icon 
> etc. spills to the next line (see attachment "Original spacing with long 
> service name").
> Also, note the indentation of the last button for Clients does not align well 
> with the buttons above it.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/styles/application.less 755dff2 
>   ambari-web/app/templates/main/host/details/host_component.hbs 60031de 
>   ambari-web/app/templates/main/host/summary.hbs 17a0b69 
> 
> Diff: https://reviews.apache.org/r/44457/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> This issue is simialr to AMBARI-11670 where the spacing for each component 
> was fixed in a  tag.
> The fix here also uses a similar approach where every component is surrounded 
> by a  tag and the width is fixed for them so that the appear in their 
> respective positions regardless of the length of service names. I have also 
> adjusted the width of the left  component with "span8" instead of 
> "span7" and the right  component of the button to "span4" instead of 
> "span5" to better adjust the spacing.
> 
> TESTS:
> No tests are written for these changes as the fix involves updates in the 
> stylesheet.
> However, the changes were tested by zooming in and out and checking if the 
> icons and names were contained in their respective blocks and not overlapping 
> with other  blocks.
> 
> 
> The following is the result of running ambari-web tests:
> 
>   24565 tests complete (28 seconds)
>   145 tests pending
> 
> 
> File Attachments
> 
> 
> AMBARI-15310 (patch-2)
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/07/5ade9968-bb9d-4d61-8362-740fb24c591f__AMBARI-15310_Mar07.patch
> 
> 
> Thanks,
> 
> Keta Patel
> 
>

Review Request 44387: AMBARI-15292: Text update for the Hive Server 2 port switch manual step in rolling upgrade

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44387/
---

Review request for Ambari and Alejandro Fernandez.


Bugs: AMBARI-15292
https://issues.apache.org/jira/browse/AMBARI-15292


Repository: ambari


Description
---

For the manual step in rolling upgrade where it tells users that Hiver Server 2 
port is to be changed, the message current reads "hive". Changing it to "Hive" 
in all the rolling upgrade xml files.


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 
953a731 
  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 
61540b7 
  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml 
5c91764 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
4622d3d 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
f775314 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
592a0ef 

Diff: https://reviews.apache.org/r/44387/diff/


Testing
---

patch a cluster with change, run rolling upgrade to hit the manual step and 
check the message.


Thanks,

Di Li

[jira] [Updated] (AMBARI-15292) Text update for the Hive Server 2 port switch manual step in rolling upgrade

[Di Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Di Li updated AMBARI-15292:
---
Component/s: ambari-web

> Text update for the Hive Server 2 port switch manual step in rolling upgrade
> 
>
> Key: AMBARI-15292
> URL: https://issues.apache.org/jira/browse/AMBARI-15292
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.2.0
>Reporter: Di Li
>Assignee: Di Li
> Fix For: trunk
>
> Attachments: AMBARI-15292.patch
>
>
> The message current reads "hive". Changing it to "Hive" in all the rolling 
> upgrade xml files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15292) Text update for the Hive Server 2 port switch manual step in rolling upgrade

[Di Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Di Li updated AMBARI-15292:
---
Attachment: AMBARI-15292.patch

> Text update for the Hive Server 2 port switch manual step in rolling upgrade
> 
>
> Key: AMBARI-15292
> URL: https://issues.apache.org/jira/browse/AMBARI-15292
> Project: Ambari
>  Issue Type: Bug
>    Reporter: Di Li
>Assignee: Di Li
> Attachments: AMBARI-15292.patch
>
>
> The message current reads "hive". Changing it to "Hive" in all the rolling 
> upgrade xml files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-15292) Text update for the Hive Server 2 port switch manual step in rolling upgrade

[Di Li (JIRA)]

Di Li created AMBARI-15292:
--

 Summary: Text update for the Hive Server 2 port switch manual step 
in rolling upgrade
 Key: AMBARI-15292
 URL: https://issues.apache.org/jira/browse/AMBARI-15292
 Project: Ambari
  Issue Type: Bug
Reporter: Di Li
Assignee: Di Li


The message current reads "hive". Changing it to "Hive" in all the rolling 
upgrade xml files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 44346: AMBARI-15285:AMS quicklink directory name is wrong causing errors in ambari server start

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44346/
---

Review request for Ambari and Jaimin Jetly.


Bugs: AMBARI-15285
https://issues.apache.org/jira/browse/AMBARI-15285


Repository: ambari


Description
---

The dir name should be called quicklinks, with lower case "l".
It's currently quickLinks (upper case L) in trunk code, causing the following 
error during ambari server start up...

03 Mar 2016 07:16:15,133 ERROR [main] QuickLinksConfigurationModule:64 - Quick 
links file not found
03 Mar 2016 07:16:15,134 ERROR [main] QuickLinksConfigurationModule:74 - Unable 
to parse quicklinks configuration fil
e 
/var/lib/ambari-server/resources/common-services/AMBARI_METRICS/0.1.0/quicklinks/quicklinks.json
java.io.EOFException: No content to map to Object due to end of input


The fix is to correct the name to be the default one quicklinks.


Diffs
-

  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/quickLinks/quicklinks.json
 1c276c0 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/quicklinks/quicklinks.json
 PRE-CREATION 

Diff: https://reviews.apache.org/r/44346/diff/


Testing
---

tested by patching a cluster installed with trunk code.


Thanks,

Di Li

[jira] [Commented] (AMBARI-15285) AMS quicklink directory name is wrong causing errors in ambari server start

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15178266#comment-15178266
 ] 

Di Li commented on AMBARI-15285:


tested by patching a cluster installed with trunk code.

> AMS quicklink directory name is wrong causing errors in ambari server start
> ---
>
> Key: AMBARI-15285
> URL: https://issues.apache.org/jira/browse/AMBARI-15285
> Project: Ambari
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Di Li
>Assignee: Di Li
> Fix For: trunk
>
> Attachments: AMBARI-15285.patch
>
>
> The dir name should be called quicklinks, with lower case "l".
> It's currently quickLinks (upper case L) in trunk code, causing the following 
> error during ambari server start up...
> 03 Mar 2016 07:16:15,133 ERROR [main] QuickLinksConfigurationModule:64 - 
> Quick links file not found
> 03 Mar 2016 07:16:15,134 ERROR [main] QuickLinksConfigurationModule:74 - 
> Unable to parse quicklinks configuration fil
> e 
> /var/lib/ambari-server/resources/common-services/AMBARI_METRICS/0.1.0/quicklinks/quicklinks.json
> java.io.EOFException: No content to map to Object due to end of input



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15285) AMS quicklink directory name is wrong causing errors in ambari server start

[Di Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Di Li updated AMBARI-15285:
---
Attachment: AMBARI-15285.patch

> AMS quicklink directory name is wrong causing errors in ambari server start
> ---
>
> Key: AMBARI-15285
> URL: https://issues.apache.org/jira/browse/AMBARI-15285
> Project: Ambari
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Di Li
>Assignee: Di Li
> Fix For: trunk
>
> Attachments: AMBARI-15285.patch
>
>
> The dir name should be called quicklinks, with lower case "l".
> It's currently quickLinks (upper case L) in trunk code, causing the following 
> error during ambari server start up...
> 03 Mar 2016 07:16:15,133 ERROR [main] QuickLinksConfigurationModule:64 - 
> Quick links file not found
> 03 Mar 2016 07:16:15,134 ERROR [main] QuickLinksConfigurationModule:74 - 
> Unable to parse quicklinks configuration fil
> e 
> /var/lib/ambari-server/resources/common-services/AMBARI_METRICS/0.1.0/quicklinks/quicklinks.json
> java.io.EOFException: No content to map to Object due to end of input



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-15285) AMS quicklink directory name is wrong causing errors in ambari server start

[Di Li (JIRA)]

Di Li created AMBARI-15285:
--

 Summary: AMS quicklink directory name is wrong causing errors in 
ambari server start
 Key: AMBARI-15285
 URL: https://issues.apache.org/jira/browse/AMBARI-15285
 Project: Ambari
  Issue Type: Bug
Reporter: Di Li
Assignee: Di Li


The dir name should be called quicklinks, with lower case "l".
It's currently quickLinks (upper case L) in trunk code, causing the following 
error during ambari server start up...

03 Mar 2016 07:16:15,133 ERROR [main] QuickLinksConfigurationModule:64 - Quick 
links file not found
03 Mar 2016 07:16:15,134 ERROR [main] QuickLinksConfigurationModule:74 - Unable 
to parse quicklinks configuration fil
e 
/var/lib/ambari-server/resources/common-services/AMBARI_METRICS/0.1.0/quicklinks/quicklinks.json
java.io.EOFException: No content to map to Object due to end of input




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-15132) in upgrade.xml to support set/replace/transfer

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172509#comment-15172509
 ] 

Di Li commented on AMBARI-15132:


failure in  https://builds.apache.org/job/Ambari-trunk-Commit/4406/ is 
unrelated to the commit from this jira. it was due to missing license header >> 
"Too many files with unapproved license: 2 See RAT report in: 
/home/jenkins/jenkins-slave/workspace/Ambari-trunk-Commit/ambari-server/target/rat.txt"

>  in upgrade.xml to support set/replace/transfer
> --
>
> Key: AMBARI-15132
> URL: https://issues.apache.org/jira/browse/AMBARI-15132
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server, ambari-web
>Affects Versions: 2.2.0
>        Reporter: Di Li
>Assignee: Di Li
> Fix For: trunk
>
> Attachments: AMBARI-15132.patch
>
>
>  in upgrade.xml  currently only supports single property set. this 
> is to improve it to to support set/replace/transfer. The code change must be 
> backward compatible in order to not break the existing update-***.xml and 
> nonrolling-upgrade-***.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

How hdp.version is calculated?

[Di Li]

Hello folks,

I notice many services uses ${hdp.version} in their Python scripts.  I yet 
failed to locate the logic in Ambari code that shows how hdp.version is 
calculated to an exact value based on the stack.

Could someone please give me some points on how hdp.version is calculated ?

Thanks.

Di.

[jira] [Commented] (AMBARI-15171) Parameterize distro-specific stack information for SLIDER

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172012#comment-15172012
 ] 

Di Li commented on AMBARI-15171:


Pushed to Apache Ambari AMBARI-13364 branch as 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=ec88341d12042241afcbef6ef320c7a5790ec9b7

> Parameterize distro-specific stack information for SLIDER
> -
>
> Key: AMBARI-15171
> URL: https://issues.apache.org/jira/browse/AMBARI-15171
> Project: Ambari
>  Issue Type: Technical task
>  Components: ambari-server
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Juanjo Marron
>Assignee: Juanjo Marron
> Fix For: 2.4.0
>
> Attachments: AMBARI-15171.patch
>
>
> Apply the prototype detailed on AMBARI-13364 to SLIDER common-service 
> defintion .
> Tokens such as: current version, upgrade version, stack name, install 
> path..., will be parameterized and distro-agnostic at common-service 
> definition level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-15114) Parameterize distro-specific stack information for HBASE

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172000#comment-15172000
 ] 

Di Li commented on AMBARI-15114:


Pushed to Apache Ambari AMBARI-13364 branch as 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=e9741e0e6b6c0696fa06eea549d28df4795d2a7e

> Parameterize distro-specific stack information for HBASE
> 
>
> Key: AMBARI-15114
> URL: https://issues.apache.org/jira/browse/AMBARI-15114
> Project: Ambari
>  Issue Type: Technical task
>  Components: ambari-server
>Affects Versions: 2.1.0, 2.2.0, 2.2.1
>Reporter: Juanjo Marron
>Assignee: Juanjo Marron
> Fix For: 2.4.0
>
> Attachments: AMBARI-15114.patch
>
>
> Apply the prototype detailed on AMBARI-13364 to HBASE common-service.
> Tokens such as: current version, upgrade version, stack name, install 
> path..., will be parameterized and make distro-agnostic at service definition 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-15137) Parameterize distro-specific stack information for TEZ

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172006#comment-15172006
 ] 

Di Li commented on AMBARI-15137:


Pushed to Apache Ambari AMBARI-13364 branch as 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=f83a07e360a9dd6717ce03a7b5d718e1fe8433c4

> Parameterize distro-specific stack information for TEZ
> --
>
> Key: AMBARI-15137
> URL: https://issues.apache.org/jira/browse/AMBARI-15137
> Project: Ambari
>  Issue Type: Technical task
>  Components: ambari-server
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Juanjo Marron
>Assignee: Juanjo Marron
> Attachments: AMBARI-15137.patch
>
>
> Apply the prototype detailed on AMBARI-13364 to TEZ .
> Tokens such as: current version, upgrade version, stack name, install 
> path..., will be parameterized and distro-agnostic at common-service 
> definition level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-14472) Parameterize distro-specific stack information for OOZIE

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-14472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171993#comment-15171993
 ] 

Di Li commented on AMBARI-14472:


Pushed to Apache Ambari AMBARI-13364 branch as 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=b3f886e13daea3641a8942ef5046f738d9809d8d

> Parameterize distro-specific stack information for OOZIE
> 
>
> Key: AMBARI-14472
> URL: https://issues.apache.org/jira/browse/AMBARI-14472
> Project: Ambari
>  Issue Type: Technical task
>  Components: ambari-server
>Affects Versions: 2.2.0, 2.2.1
>Reporter: Juanjo Marron
>Assignee: Juanjo Marron
> Fix For: 2.4.0
>
> Attachments: AMBARI-14472.patch
>
>
> Apply the prototype detailed on AMBARI-13364 to OOZIE service.
> Tokens such as: current version, upgrade version, stack name, install 
> path..., will be parameterized and distro-agnostic at service definition 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-15053) Parameterize distro-specific stack information for YARN/MR

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171989#comment-15171989
 ] 

Di Li commented on AMBARI-15053:


Pushed to Apache Ambari AMBARI-13364 branch as 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=7a77bfe52060920678d50aebc6d7f1e24ef01858

> Parameterize distro-specific stack information for YARN/MR
> --
>
> Key: AMBARI-15053
> URL: https://issues.apache.org/jira/browse/AMBARI-15053
> Project: Ambari
>  Issue Type: Technical task
>  Components: ambari-server
>Affects Versions: 2.2.0, 2.2.1, 2.2.2
>Reporter: Juanjo Marron
>Assignee: Juanjo Marron
> Fix For: 2.4.0
>
> Attachments: AMBARI-15053.patch
>
>
> Apply the prototype detailed on AMBARI-13364 to YARN/MR common-service.
> Tokens such as: current version, upgrade version, stack name, install 
> path..., will be parameterized and distro-agnostic at service definition 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-14451) Parameterize distro-specific stack information for HDFS

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-14451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171982#comment-15171982
 ] 

Di Li commented on AMBARI-14451:


pushed to Apache Ambari AMBARI-13364 branch as 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=3e02731abe04f4614108a6750371c5bf369d73de

> Parameterize distro-specific stack information for HDFS
> ---
>
> Key: AMBARI-14451
> URL: https://issues.apache.org/jira/browse/AMBARI-14451
> Project: Ambari
>  Issue Type: Technical task
>  Components: ambari-server
>Affects Versions: 2.2.0, 2.2.1
>Reporter: Juanjo Marron
>Assignee: Juanjo Marron
> Fix For: 2.4.0
>
> Attachments: AMBARI-14451.patch
>
>
> Apply the prototype detailed on AMBARI-13364 to HDFS service.
> Tokens such as: current version, upgrade version, stack name, install 
> path..., will be parameterized and distro-agnostic at service definition 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-14451) Parameterize distro-specific stack information for HDFS

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-14451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171978#comment-15171978
 ] 

Di Li commented on AMBARI-14451:


depends on AMBARI-14435 for various code changes.

> Parameterize distro-specific stack information for HDFS
> ---
>
> Key: AMBARI-14451
> URL: https://issues.apache.org/jira/browse/AMBARI-14451
> Project: Ambari
>  Issue Type: Technical task
>  Components: ambari-server
>Affects Versions: 2.2.0, 2.2.1
>Reporter: Juanjo Marron
>Assignee: Juanjo Marron
> Fix For: 2.4.0
>
> Attachments: AMBARI-14451.patch
>
>
> Apply the prototype detailed on AMBARI-13364 to HDFS service.
> Tokens such as: current version, upgrade version, stack name, install 
> path..., will be parameterized and distro-agnostic at service definition 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-14435) Parameterize distro-specific stack information for ZOOKEEPER

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-14435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171976#comment-15171976
 ] 

Di Li commented on AMBARI-14435:


Pushed to AMBARI-13364 branch as 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=7f4975fa5078eff6fda34c76b9830c263e03f859

> Parameterize distro-specific stack information for ZOOKEEPER
> 
>
> Key: AMBARI-14435
> URL: https://issues.apache.org/jira/browse/AMBARI-14435
> Project: Ambari
>  Issue Type: Technical task
>  Components: ambari-server
>Affects Versions: 2.2.0, 2.2.1
>Reporter: Juanjo Marron
>Assignee: Juanjo Marron
> Fix For: 2.4.0
>
> Attachments: AMBARI-14435.patch
>
>
> Apply the prototype detailed on AMBARI-13364 to ZOOKEEPER service.
> Tokens such as: current version, upgrade version, stack name, install 
> path..., will be parameterized and distro-agnostic at service definition 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-15223) Add ability to display messages on Upgrade UI in paragraphs

[Di Li (JIRA)]

Di Li created AMBARI-15223:
--

 Summary: Add ability to display messages on Upgrade UI in 
paragraphs
 Key: AMBARI-15223
 URL: https://issues.apache.org/jira/browse/AMBARI-15223
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server, ambari-web
Affects Versions: 2.2.0
Reporter: Di Li
Assignee: Di Li
 Fix For: trunk


Ambari web UI displays the messages defined in each manual step in 
upgrade-**.xml and nonrolling-upgrade-**.xml as a whole paragraph.  This is ok 
for short messages, but becomes less user friendly when we display a large 
paragraph of message.

This JIRA is an attempt to improve the readability of the message by allowing 
users to breaking down a single large section into smaller (cleaner) paragraphs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-15132) in upgrade.xml to support set/replace/transfer

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-15132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171880#comment-15171880
 ] 

Di Li commented on AMBARI-15132:


Committed to Apache Ambari trunk as
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=771deecd2a2f7d04e354f211ca64f3b2f41e6043

>  in upgrade.xml to support set/replace/transfer
> --
>
> Key: AMBARI-15132
> URL: https://issues.apache.org/jira/browse/AMBARI-15132
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server, ambari-web
>Affects Versions: 2.2.0
>Reporter: Di Li
>Assignee: Di Li
> Fix For: trunk
>
> Attachments: AMBARI-15132.patch
>
>
>  in upgrade.xml  currently only supports single property set. this 
> is to improve it to to support set/replace/transfer. The code change must be 
> backward compatible in order to not break the existing update-***.xml and 
> nonrolling-upgrade-***.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

> On Feb. 26, 2016, 10:13 p.m., Nate Cole wrote:
> > Nice work!  Just remove dead comments and I'm all set on this review.  
> > Thanks for taking this up, it really helps the consistency and readability 
> > of the upgrade packs.

Hello Nate,

Sorry about the dead comments. I was meant to remove them in the previous patch 
submit.

Please review the latest patch.

Thanks


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/#review120978
---


On Feb. 26, 2016, 9:47 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43944/
> ---
> 
> (Updated Feb. 26, 2016, 9:47 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-15132
> https://issues.apache.org/jira/browse/AMBARI-15132
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
>  in upgrade.xml currently only supports single property set. this 
> is to improve it to to support set/replace/transfer. The code change must be 
> backward compatible in order to not break the existing update-**.xml and 
> nonrolling-upgrade-**.xml.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
>  e77cf36 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
>  a4dff2e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
>  6b22f58 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java
>  ce87668 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  1a5d4e7 
>   
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
> f345d12 
>   ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
> 237da77 
> 
> Diff: https://reviews.apache.org/r/43944/diff/
> 
> 
> Testing
> ---
> 
> unit test
> ran upgrade from Ambari 2.2 to Ambari trunk.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/
---

(Updated Feb. 26, 2016, 10:28 p.m.)


Review request for Ambari, Alejandro Fernandez and Nate Cole.


Bugs: AMBARI-15132
https://issues.apache.org/jira/browse/AMBARI-15132


Repository: ambari


Description
---

 in upgrade.xml currently only supports single property set. this is 
to improve it to to support set/replace/transfer. The code change must be 
backward compatible in order to not break the existing update-**.xml and 
nonrolling-upgrade-**.xml.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
 e77cf36 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
 a4dff2e 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
 6b22f58 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java
 ce87668 
  
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
 1a5d4e7 
  ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
f345d12 
  ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
237da77 

Diff: https://reviews.apache.org/r/43944/diff/


Testing
---

unit test
ran upgrade from Ambari 2.2 to Ambari trunk.


Thanks,

Di Li

Re: Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

> On Feb. 26, 2016, 2:27 a.m., Nate Cole wrote:
> > This is very close - I think there's just one more thing to do (below).

Hello Nate,

Please review the lastest patch where the ConfigureTask only does basic if 
attribute validation like you suggested (like has if-value but no if-key). and 
the ConfigureAction.java to perform the allow/disallow check.


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/#review120815
---


On Feb. 25, 2016, 9:34 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43944/
> ---
> 
> (Updated Feb. 25, 2016, 9:34 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-15132
> https://issues.apache.org/jira/browse/AMBARI-15132
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
>  in upgrade.xml currently only supports single property set. this 
> is to improve it to to support set/replace/transfer. The code change must be 
> backward compatible in order to not break the existing update-**.xml and 
> nonrolling-upgrade-**.xml.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
>  a4dff2e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
>  6b22f58 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  1a5d4e7 
>   
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
> f345d12 
>   ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
> 237da77 
> 
> Diff: https://reviews.apache.org/r/43944/diff/
> 
> 
> Testing
> ---
> 
> unit test
> ran upgrade from Ambari 2.2 to Ambari trunk.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/
---

(Updated Feb. 26, 2016, 9:47 p.m.)


Review request for Ambari, Alejandro Fernandez and Nate Cole.


Bugs: AMBARI-15132
https://issues.apache.org/jira/browse/AMBARI-15132


Repository: ambari


Description
---

 in upgrade.xml currently only supports single property set. this is 
to improve it to to support set/replace/transfer. The code change must be 
backward compatible in order to not break the existing update-**.xml and 
nonrolling-upgrade-**.xml.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
 e77cf36 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
 a4dff2e 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
 6b22f58 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java
 ce87668 
  
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
 1a5d4e7 
  ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
f345d12 
  ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
237da77 

Diff: https://reviews.apache.org/r/43944/diff/


Testing
---

unit test
ran upgrade from Ambari 2.2 to Ambari trunk.


Thanks,

Di Li

Re: Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/
---

(Updated Feb. 25, 2016, 9:34 p.m.)


Review request for Ambari, Alejandro Fernandez and Nate Cole.


Bugs: AMBARI-15132
https://issues.apache.org/jira/browse/AMBARI-15132


Repository: ambari


Description
---

 in upgrade.xml currently only supports single property set. this is 
to improve it to to support set/replace/transfer. The code change must be 
backward compatible in order to not break the existing update-**.xml and 
nonrolling-upgrade-**.xml.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
 a4dff2e 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
 6b22f58 
  
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
 1a5d4e7 
  ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
f345d12 
  ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
237da77 

Diff: https://reviews.apache.org/r/43944/diff/


Testing
---

unit test
ran upgrade from Ambari 2.2 to Ambari trunk.


Thanks,

Di Li

Re: Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

> On Feb. 25, 2016, 7:49 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java,
> >  lines 445-460
> > <https://reviews.apache.org/r/43944/diff/2/?file=1268445#file1268445line445>
> >
> > These properties can all be abstracted to Masked - Transfer and 
> > ConfigurationKeyValue ("set") already extend that.
> > 
> > Also, probably want to remove  from XML.

Hello Nate,

I updated the code to mve the if-key, etc to the Masked. Please review.

Thanks.


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/#review120749
-------


On Feb. 24, 2016, 10:32 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43944/
> ---
> 
> (Updated Feb. 24, 2016, 10:32 p.m.)
> 
> 
> Review request for Ambari and Alejandro Fernandez.
> 
> 
> Bugs: AMBARI-15132
> https://issues.apache.org/jira/browse/AMBARI-15132
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
>  in upgrade.xml currently only supports single property set. this 
> is to improve it to to support set/replace/transfer. The code change must be 
> backward compatible in order to not break the existing update-**.xml and 
> nonrolling-upgrade-**.xml.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
>  a4dff2e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
>  6b22f58 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  1a5d4e7 
>   
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
> f345d12 
>   ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
> 237da77 
> 
> Diff: https://reviews.apache.org/r/43944/diff/
> 
> 
> Testing
> ---
> 
> unit test
> ran upgrade from Ambari 2.2 to Ambari trunk.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

> On Feb. 24, 2016, 7:35 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml,
> >  line 129
> > <https://reviews.apache.org/r/43944/diff/1/?file=1267519#file1267519line129>
> >
> > Do we already have granularity at the individual op level?
> >     E.g.,
> > 
> > 
> > 
> 
> Di Li wrote:
> Hello Alejandro,
> 
> Ambari trunk code currently only have conditions for transfter operation 
> "delete".
> 
> My initial thought was to do all the condition in the   
> section. Based on Nate and your comments , I am updating the patch to have 
> conditions done at operation level just like the ones you listed.

Hello Alejandro,

Could you please reveiw the new patch? It has the conditions in  
set/transfer/replace.


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/#review120556
---


On Feb. 24, 2016, 1:54 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43944/
> ---
> 
> (Updated Feb. 24, 2016, 1:54 p.m.)
> 
> 
> Review request for Ambari and Alejandro Fernandez.
> 
> 
> Bugs: AMBARI-15132
> https://issues.apache.org/jira/browse/AMBARI-15132
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
>  in upgrade.xml currently only supports single property set. this 
> is to improve it to to support set/replace/transfer. The code change must be 
> backward compatible in order to not break the existing update-**.xml and 
> nonrolling-upgrade-**.xml.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
>  a4dff2e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
>  6b22f58 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  4ea8f15 
>   
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
> f345d12 
>   ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
> 237da77 
> 
> Diff: https://reviews.apache.org/r/43944/diff/
> 
> 
> Testing
> ---
> 
> unit test
> ran upgrade from Ambari 2.2 to Ambari trunk.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/
---

(Updated Feb. 24, 2016, 10:32 p.m.)


Review request for Ambari and Alejandro Fernandez.


Bugs: AMBARI-15132
https://issues.apache.org/jira/browse/AMBARI-15132


Repository: ambari


Description
---

 in upgrade.xml currently only supports single property set. this is 
to improve it to to support set/replace/transfer. The code change must be 
backward compatible in order to not break the existing update-**.xml and 
nonrolling-upgrade-**.xml.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
 a4dff2e 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
 6b22f58 
  
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
 1a5d4e7 
  ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
f345d12 
  ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
237da77 

Diff: https://reviews.apache.org/r/43944/diff/


Testing
---

unit test
ran upgrade from Ambari 2.2 to Ambari trunk.


Thanks,

Di Li

Re: Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

> On Feb. 24, 2016, 7:35 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml,
> >  line 129
> > <https://reviews.apache.org/r/43944/diff/1/?file=1267519#file1267519line129>
> >
> > Do we already have granularity at the individual op level?
> > E.g.,
> > 
> > 
> > 

Hello Alejandro,

Ambari trunk code currently only have conditions for transfter operation 
"delete".

My initial thought was to do all the condition in the   section. 
Based on Nate and your comments , I am updating the patch to have conditions 
done at operation level just like the ones you listed.


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/#review120556
-------


On Feb. 24, 2016, 1:54 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43944/
> ---
> 
> (Updated Feb. 24, 2016, 1:54 p.m.)
> 
> 
> Review request for Ambari and Alejandro Fernandez.
> 
> 
> Bugs: AMBARI-15132
> https://issues.apache.org/jira/browse/AMBARI-15132
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
>  in upgrade.xml currently only supports single property set. this 
> is to improve it to to support set/replace/transfer. The code change must be 
> backward compatible in order to not break the existing update-**.xml and 
> nonrolling-upgrade-**.xml.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
>  a4dff2e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
>  6b22f58 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  4ea8f15 
>   
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
> f345d12 
>   ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
> 237da77 
> 
> Diff: https://reviews.apache.org/r/43944/diff/
> 
> 
> Testing
> ---
> 
> unit test
> ran upgrade from Ambari 2.2 to Ambari trunk.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

> On Feb. 24, 2016, 3:48 p.m., Nate Cole wrote:
> > It seems like we should be going the other way.  set/replace should support 
> > conditional attributes.  The  already supports this (see 
> > HDP/2.3/config-upgrade.xml for dfs.namenode.rpc-address).   
> > should not be used

Got it Nate. I will update the patch accordingly.


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/#review120509
---


On Feb. 24, 2016, 1:54 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43944/
> ---
> 
> (Updated Feb. 24, 2016, 1:54 p.m.)
> 
> 
> Review request for Ambari and Alejandro Fernandez.
> 
> 
> Bugs: AMBARI-15132
> https://issues.apache.org/jira/browse/AMBARI-15132
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
>  in upgrade.xml currently only supports single property set. this 
> is to improve it to to support set/replace/transfer. The code change must be 
> backward compatible in order to not break the existing update-**.xml and 
> nonrolling-upgrade-**.xml.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
>  a4dff2e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
>  6b22f58 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  4ea8f15 
>   
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
> f345d12 
>   ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
> 237da77 
> 
> Diff: https://reviews.apache.org/r/43944/diff/
> 
> 
> Testing
> ---
> 
> unit test
> ran upgrade from Ambari 2.2 to Ambari trunk.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Review Request 43944: AMBARI-15132 in upgrade.xml to support set/replace/transfer

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43944/
---

Review request for Ambari and Alejandro Fernandez.


Bugs: AMBARI-15132
https://issues.apache.org/jira/browse/AMBARI-15132


Repository: ambari


Description
---

 in upgrade.xml currently only supports single property set. this is 
to improve it to to support set/replace/transfer. The code change must be 
backward compatible in order to not break the existing update-**.xml and 
nonrolling-upgrade-**.xml.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigUpgradeChangeDefinition.java
 a4dff2e 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
 6b22f58 
  
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
 4ea8f15 
  ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/config-upgrade.xml 
f345d12 
  ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 
237da77 

Diff: https://reviews.apache.org/r/43944/diff/


Testing
---

unit test
ran upgrade from Ambari 2.2 to Ambari trunk.


Thanks,

Di Li

Re: Ru/EU not updating alert definitions in Ambari server db?

[Di Li]

Hello folks,

Yes, I would probably have had seen the error if I had looked at the UI (my 
bad, I didn't check until post-RU). and yes the issue (since it's about 
refactored stack structure) will be there the moment I upgrade the Ambari 
server.

I wanted to know if there is someway to have the alert definitions updated 
during ambari server upgrade or RU/EU. I am keeping RU/EU in the loop because I 
may have different versions of the same service in the common service mapped to 
service in stack. By that, I mean
  common-services/FLUME/1.2.3.4.5 mapped to MyStack/1.0/FLUME
  common-services/FLUME/1.3.4.5.1 mapped to MyStack/2.0/FLUMEwhere two versions 
of FLUME have different alert_definition Python script.

In this case,I 'd rather have RU/EU take care of updating the alert definition 
when the stack version is switched (from 1.0 to 2.0).

Understandably this may not be something currently in place in Ambari upgrade 
nor RU/EU, so I figured I'd better check about this behavior on the mail list.
Di

  From: Jonathan Hurley <jhur...@hortonworks.com>
 To: Ambari <dev@ambari.apache.org>; Di Li <lovelyco...@yahoo.com> 
 Sent: Tuesday, February 23, 2016 3:43 PM
 Subject: Re: Ru/EU not updating alert definitions in Ambari server db?
   
Based on this description, you should have seen the error before RU since you 
refactored your stacks and moved the script. RU and, for that matter, Ambari 
upgrade, has no built-in knowledge of your scripts. You can pretty much put 
them anywhere. So, if you change your directory structure around, you're going 
to also have to update your definitions to have the right path.

This is why you should probably always put them in common services, and use 
inheritance between stacks.

> On Feb 23, 2016, at 2:24 PM, Di Li <lovelyco...@yahoo.com.INVALID> wrote:
> 
> Hello folks,
> 
> Thanks for the reply. I have my own stack called MyStack 1.0 and MyStack 2.0. 
> I found the issue after I heavily refactored the MyStack directory structure.
> 
> 1. In Ambari 2.1/MyStack 1.0, I have Flume's alert_flume_agent_status.py 
> defined as 
> MyStack/1.0/services/FLUME/package/alerts/alert_flume_agent_status.py
> 2. In Ambari 2.2, I changed the structure of MyStack to make use of common 
> services, so 
>    a) The Flume directories are moved into common-services
>    b) MyStack 1.0 now contains a Flume dir with just a metainfo.xml file.
>    c) I also added MyStack 2.0 so I can run RU/EU. MyStack 2.0 also only 
>contains a Flume dir with just a metainfo.xml file.
>    d) common-services has Flume agent alert Python script as 
>FLUME/1.2.3.4.5/package/alerts/alert_flume_agent_status.py (note the custom 
>version I used here...). As you can see, I changed the path to the 
>alert_flume_agent_status.py...
> 3. Perform RU
> 
> -- Post RU, I saw Flume agent produced an "UNKNOWN" alert with an error 
> message
> Flume Agent Status
> Unable to find 
> 'MyStack/1.0/services/FLUME/package/alerts/alert_flume_agent_status.py' as an 
> absolute path or part of /var/lib/ambari-agent/cache/stacks or 
> /var/lib/ambari-agent/cache/host_scripts
> 
> -- I noticed tht in the Ambari server database, the Flume agent alert 
> definition had the wrong patch to the Python script as shown below. It still 
> has the old MyStack based path.
> 9 |          2 | flume_agent_status                            | FLUME        
>   | FLUME_HANDLER      |
> ANY    | Flume Agent Status                              | This host-level 
> alert is triggered if any of the expect
> ed flume agent processes are not available.
>        |      1 |                1 | SCRIPT      | 
>{"path":"MyStack/1.0/services/FLUME/package/alerts/alert_flu
> me_agent_status.py","parameters":[{"name":"run.directory","display_name":"Run 
> Directory","value":"/var/run/flume","de
> scription":"The directory where flume agent processes will place their PID 
> files.","type":"STRING"}],"type":"SCRIPT"}
> 
> -- This prompted me to ask the question about whether the alert definition 
> should have been updated by RU/EU (or ambari server upgrade maybe).
> 
> -- I was able to update Flume agent's alert definition with the correct path 
> via following REST API.
> curl -u admin:admin -H 'X-Requested-By:ambari' -X PUT -d '{"AlertDefinition" 
> : {"source" : {"path" : 
> "FLUME/1.2.3.4.5/package/alerts/alert_flume_agent_status.py",      "type" : 
> "SCRIPT" }}}' 
> http://myhost.mydomain.com:8080/api/v1/clusters/test/alert_definitions/9
> 
> 
>      From: Jonathan Hurley <jhur...@hortonworks.com>
> To: "dev@ambari.apache.org&qu

[jira] [Updated] (AMBARI-15132) in upgrade.xml to support set/replace/transfer

[Di Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Di Li updated AMBARI-15132:
---
Attachment: AMBARI-15132.patch

>  in upgrade.xml to support set/replace/transfer
> --
>
> Key: AMBARI-15132
> URL: https://issues.apache.org/jira/browse/AMBARI-15132
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server, ambari-web
>Affects Versions: 2.2.0
>Reporter: Di Li
>Assignee: Di Li
> Fix For: trunk
>
> Attachments: AMBARI-15132.patch
>
>
>  in upgrade.xml  currently only supports single property set. this 
> is to improve it to to support set/replace/transfer. The code change must be 
> backward compatible in order to not break the existing update-***.xml and 
> nonrolling-upgrade-***.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Ru/EU not updating alert definitions in Ambari server db?

[Di Li]

Hello folks,

Thanks for the reply. I have my own stack called MyStack 1.0 and MyStack 2.0. I 
found the issue after I heavily refactored the MyStack directory structure.

1. In Ambari 2.1/MyStack 1.0, I have Flume's alert_flume_agent_status.py 
defined as MyStack/1.0/services/FLUME/package/alerts/alert_flume_agent_status.py
2. In Ambari 2.2, I changed the structure of MyStack to make use of common 
services, so 
   a) The Flume directories are moved into common-services
   b) MyStack 1.0 now contains a Flume dir with just a metainfo.xml file.
   c) I also added MyStack 2.0 so I can run RU/EU. MyStack 2.0 also only 
contains a Flume dir with just a metainfo.xml file.
   d) common-services has Flume agent alert Python script as 
FLUME/1.2.3.4.5/package/alerts/alert_flume_agent_status.py (note the custom 
version I used here...). As you can see, I changed the path to the 
alert_flume_agent_status.py...
3. Perform RU

-- Post RU, I saw Flume agent produced an "UNKNOWN" alert with an error message
Flume Agent Status
Unable to find 
'MyStack/1.0/services/FLUME/package/alerts/alert_flume_agent_status.py' as an 
absolute path or part of /var/lib/ambari-agent/cache/stacks or 
/var/lib/ambari-agent/cache/host_scripts

-- I noticed tht in the Ambari server database, the Flume agent alert 
definition had the wrong patch to the Python script as shown below. It still 
has the old MyStack based path.
9 |  2 | flume_agent_status | FLUME 
 | FLUME_HANDLER   |
 ANY | Flume Agent Status   | This host-level 
alert is triggered if any of the expect
ed flume agent processes are not available.
   |   1 | 1 | SCRIPT  | 
{"path":"MyStack/1.0/services/FLUME/package/alerts/alert_flu
me_agent_status.py","parameters":[{"name":"run.directory","display_name":"Run 
Directory","value":"/var/run/flume","de
scription":"The directory where flume agent processes will place their PID 
files.","type":"STRING"}],"type":"SCRIPT"}

-- This prompted me to ask the question about whether the alert definition 
should have been updated by RU/EU (or ambari server upgrade maybe).

-- I was able to update Flume agent's alert definition with the correct path 
via following REST API.
curl -u admin:admin -H 'X-Requested-By:ambari' -X PUT -d '{"AlertDefinition" : 
{"source" : {"path" : 
"FLUME/1.2.3.4.5/package/alerts/alert_flume_agent_status.py",  "type" : 
"SCRIPT" }}}' 
http://myhost.mydomain.com:8080/api/v1/clusters/test/alert_definitions/9


  From: Jonathan Hurley <jhur...@hortonworks.com>
 To: "dev@ambari.apache.org" <dev@ambari.apache.org>; Di Li 
<lovelyco...@yahoo.com> 
 Sent: Tuesday, February 23, 2016 1:23 PM
 Subject: Re: Ru/EU not updating alert definitions in Ambari server db?
   
No, they are not updated; however, stack inheritance should allow you to 
reference the files from the new stack. We've performed many different upgrades 
both both inter-stack and intra-stack and have never seen any issues with alert 
definitions. 

> On Feb 23, 2016, at 1:13 PM, Di Li <lovelyco...@yahoo.com.INVALID> wrote:
> 
> Hello folks,
> 
> Do RU and EU handle updating alert definitions stored in the Ambari server 
> database? 
> 
> Based on what I tested, alert definitions, especially the paths to 
> alert_***.py Python scripts are not updated by neither RU and EU. meaning if 
> I changed the path to the alert_***.py scripts in a new release, the alert 
> definitions are broken and certain components would show with a UNKNOWN alert 
> state after I run RU/EU.
> 
> 
> P.S
>    I know that, post upgrade, I can manually update the alert definitions via 
>REST APIs.
> 
> Thanks.
> 
> 


  

Ru/EU not updating alert definitions in Ambari server db?

[Di Li]

Hello folks,

Do RU and EU handle updating alert definitions stored in the Ambari server 
database? 

Based on what I tested, alert definitions, especially the paths to alert_***.py 
Python scripts are not updated by neither RU and EU. meaning if I changed the 
path to the alert_***.py scripts in a new release, the alert definitions are 
broken and certain components would show with a UNKNOWN alert state after I run 
RU/EU.


P.S
I know that, post upgrade, I can manually update the alert definitions via 
REST APIs.

Thanks.

[jira] [Created] (AMBARI-15132) in upgrade.xml to support set/replace/transfer

[Di Li (JIRA)]

Di Li created AMBARI-15132:
--

 Summary:  in upgrade.xml to support set/replace/transfer
 Key: AMBARI-15132
 URL: https://issues.apache.org/jira/browse/AMBARI-15132
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server, ambari-web
Affects Versions: 2.2.0
Reporter: Di Li
Assignee: Di Li
 Fix For: trunk


 in upgrade.xml  currently only supports single property set. this 
is to improve it to to support set/replace/transfer. The code change must be 
backward compatible in order to not break the existing update-***.xml and 
nonrolling-upgrade-***.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Is HiveServer2 rolling restart wrongly deregister the new HiveServer2 instance?

[Di Li]

Hello there,

I am looking at Hive Server2 rolling ugprade code.

For the code here, Ambari issues post_upgrade_deregister() at the end of the 
start method call. The deregister calculates the "current version" of Hive via 
hive --version cmd.
Since the deregister is at the end of the start method, is the version 
calculated be the one for the update hive and thus the deregister is deregister 
the new hive server2 instance (the one started on 10010). I think the goal is 
to deregister the old instance running on 1 port.

def start(self, env, upgrade_type=None):
    import params
    env.set_params(params)
    self.configure(env) # FOR SECURITY

    hive_service( 'hiveserver2', action = 'start', upgrade_type=upgrade_type)

    # only perform this if upgrading and rolling; a non-rolling upgrade doesn't 
need
    # to do this since hive is already down
    if upgrade_type == UPGRADE_TYPE_ROLLING:
  hive_server_upgrade.post_upgrade_deregister()

Re: Review Request 43508: AMBARI-14932: "download zip" does not work

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43508/#review119611
---


Ship it!




Ship It!

- Di Li


On Feb. 17, 2016, 7:22 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43508/
> ---
> 
> (Updated Feb. 17, 2016, 7:22 p.m.)
> 
> 
> Review request for Ambari, Di Li, DIPAYAN BHOWMICK, Jaimin Jetly, Pallav 
> Kulshreshtha, and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-14932
> https://issues.apache.org/jira/browse/AMBARI-14932
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to Reproduce:
> 1. go to "View" - "FILES" - " Create Instance"
> 2. go to hdfs file view
> 3. click "download zip" icon
> The click doesn't download any file. The REST call returns 200 OK status.
> 
> Detailed steps on configuring Views can be found in the link below:
> http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/bk_ambari_views_guide-20150721.pdf
> 
> 
> Diffs
> -
> 
>   
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/DownloadService.java
>  749174a 
>   contrib/views/files/src/main/resources/ui/app/controllers/file.js 88fa5fb 
> 
> Diff: https://reviews.apache.org/r/43508/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> The check for "allowed" in the controller, from the response of 
> zipByRequestId() ("/zip" API call for downloading zip file) resulted in no 
> action after clicking the download zip button. This was because there was no 
> "allowed" paramter in the response of zip download. This "allowed" property 
> was added in the response of browse() ("/browse" API call for downloading 
> individual file) to check if the user had permissions for the file or not. 
> This was verified by opening the file. If that operation didn't throw an 
> error, then it would mean that the user had the required permissions to 
> download the individual file. But the fact that the point of execution 
> reaches past the statement of opening the file verifies that the user has the 
> permission. The check with "checkperm" and setting the response with 
> "allowed" attribute in the response is not necessary. 
> So, for the fix I simply removed the check for "allowed" from the controller 
> and also removed the check for "checkperm" which sets the "allowed" attribute 
> in the response.
> 
> TESTS:
> There are already exisitng tests for DownloadService, so I haven't added any 
> new ones.
> 
> 
> Thanks,
> 
> Keta Patel
> 
>

Re: Review Request 43508: AMBARI-14932: "download zip" does not work

[Di Li]

> On Feb. 17, 2016, 4:25 p.m., Di Li wrote:
> > Hello Keta, 
> > 
> > Why did you remove code change introduced via AMBARI-14228? Have you tested 
> > if that'd introduce regression?
> > 
> > If you look at DownloadService.java, methods mapped to   @Path("/zip")
> > neither is setting allowed to true. as the "allowed" is only set when url 
> > is /browse, have you tried distinguishing when to and not to check the 
> > "data.allowed" at the front end?
> > 
> > Please also add Jaimin Jetly as a reviewer?
> 
> Keta Patel wrote:
> Hello Di,
> I have added Jaimin Jetly as per your suggestion.
> 
> Regarding your question on when the front end distinguishes between 
> checking "data.allowed" in the response, I have understood the following from 
> the code:
> 1. For the API with @Path("/browse"), the following happens on the 
> server-side:
>- To check if the user has permission to read the file, a file-open 
> operation is performed. If the user has required permission, then the 
> operation goes through successfully.
>  But if the user does not have the permission, then a 
> "ServiceFormattedException" error is thrown. This error is caught in the 
> front-end by the controller and an error message is displayed.
>- If the user is able to open the file without throwing the error:
>  - then the next statement checks if "checkperm=true", and returns a 
> response with "allowed=true"
>  - else if "checkperm=false", the response returned only contains the 
> file to be downloaded.
>
> 2. For teh API with @Path("/concat") and @Path("/zip"), the following 
> happens on server-side:
>- "checkperm" is not used at all.
>- The response contains only the data of the file(s) to be downloaded.
>
> 3. Front-end controller:
>- It always calls the function "downloadFile" for any download action 
> in the FILES view.
>- This function always makes the server call with "checkperm=true" 
>- It then checks if the response contained "allowed=true" to proceed 
> with the download.
>- If the response doesn't contain "allowed=true", nothing is done.
>- If the response contains "allowed=true", a 2nd server call is made 
> with "checkperm=false" which returns the response with the file(s) to be 
> downloaded.
> 
> 
> The above code works fine for the API with @Path("/browse"), but fails to 
> download for the other 2 APIs.
> The proposed FIX in the patch does the following for the above 3 points:
> 1. For the API with @Path("/browse"):
>- In order to see if the user has permission for the file, if the file 
> open operation does not throw an error, we can conclude that the user has the 
> required permission.
>  We do not need to set "allowed=true" just to indicate that the user 
> has the permission. In the proposed fix, I have removed the clause checking 
> if "checkperm=true" 
>  and hence, response no longer contains "allowed=true"
>- If the user is able to open the file without throwing the error:
>  - the response returned only contains the file to be downloaded.
> 
> 2. For teh API with @Path("/concat") and @Path("/zip"):
>- No changes are made.
>
> 3. Front-end controller:
>- I have removed the check for "allowed=true" in the response.
>- If an error was thrown, it gets caught in the failureCallback 
> function.
>- Else, the file is downloaded.
> 
> Thank you!

I see. Thanks for the explanation.


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43508/#review119469
---


On Feb. 17, 2016, 7:22 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43508/
> ---
> 
> (Updated Feb. 17, 2016, 7:22 p.m.)
> 
> 
> Review request for Ambari, Di Li, DIPAYAN BHOWMICK, Jaimin Jetly, Pallav 
> Kulshreshtha, and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-14932
> https://issues.apache.org/jira/browse/AMBARI-14932
> 
> 
> Repository: ambari
> 
> 
> Description
> --

Re: Review Request 43508: AMBARI-14932: "download zip" does not work

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43508/#review119469
---



Hello Keta, 

Why did you remove code change introduced via AMBARI-14228? Have you tested if 
that'd introduce regression?

If you look at DownloadService.java, methods mapped to   @Path("/zip")
neither is setting allowed to true. as the "allowed" is only set when url is 
/browse, have you tried distinguishing when to and not to check the 
"data.allowed" at the front end?

Please also add Jaimin Jetly as a reviewer?

- Di Li


On Feb. 12, 2016, 5:34 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43508/
> ---
> 
> (Updated Feb. 12, 2016, 5:34 p.m.)
> 
> 
> Review request for Ambari, Di Li, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and 
> Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-14932
> https://issues.apache.org/jira/browse/AMBARI-14932
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to Reproduce:
> 1. go to "View" - "FILES" - " Create Instance"
> 2. go to hdfs file view
> 3. click "download zip" icon
> The click doesn't download any file. The REST call returns 200 OK status.
> 
> Detailed steps on configuring Views can be found in the link below:
> http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/bk_ambari_views_guide-20150721.pdf
> 
> 
> Diffs
> -
> 
>   
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/DownloadService.java
>  749174a 
>   contrib/views/files/src/main/resources/ui/app/controllers/file.js 88fa5fb 
> 
> Diff: https://reviews.apache.org/r/43508/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> The check for "allowed" in the controller, from the response of 
> zipByRequestId() ("/zip" API call for downloading zip file) resulted in no 
> action after clicking the download zip button. This was because there was no 
> "allowed" paramter in the response of zip download. This "allowed" property 
> was added in the response of browse() ("/browse" API call for downloading 
> individual file) to check if the user had permissions for the file or not. 
> This was verified by opening the file. If that operation didn't throw an 
> error, then it would mean that the user had the required permissions to 
> download the individual file. But the fact that the point of execution 
> reaches past the statement of opening the file verifies that the user has the 
> permission. The check with "checkperm" and setting the response with 
> "allowed" attribute in the response is not necessary. 
> So, for the fix I simply removed the check for "allowed" from the controller 
> and also removed the check for "checkperm" which sets the "allowed" attribute 
> in the response.
> 
> TESTS:
> There are already exisitng tests for DownloadService, so I haven't added any 
> new ones.
> 
> 
> Thanks,
> 
> Keta Patel
> 
>

[jira] [Commented] (AMBARI-13364) Parameterize stack information used by common services

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145249#comment-15145249
 ] 

Di Li commented on AMBARI-13364:


I just recreated the AMBARI-13364 branch based on the latest trunk code. 
Please find the branch here:  
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=shortlog;h=refs/heads/AMBARI-13364

>  Parameterize stack information used by common services
> ---
>
> Key: AMBARI-13364
> URL: https://issues.apache.org/jira/browse/AMBARI-13364
> Project: Ambari
>  Issue Type: Story
>  Components: ambari-server
>Reporter: Tuong Truong
>Assignee: Juanjo Marron
> Attachments: AMBARI-13364 Parameterize stack information used by 
> common services.pdf, AMBARI-13364.patch
>
>
> This feature will add a basic framework to remove hardcoded stack information 
> out of the common services and use parameter to get access to stack 
> information.  Currently, the common services hardcoded much information 
> specific to Hortonworks' HDP stack including name (HDP), specific versions 
> (2.0, 2.1, 2.2), and install location.   This feature will propose a way of 
> configuration these information and parameterize them into the services for 
> reference as require. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-14885) After exporting blueprint from existing cluster knox_master_secret is exported

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-14885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143263#comment-15143263
 ] 

Di Li commented on AMBARI-14885:


pushed to trunk as 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=7d1ab29c5860a8cab6f19eb98c0a7bd25529ec5e

> After exporting blueprint from existing cluster knox_master_secret is exported
> --
>
> Key: AMBARI-14885
> URL: https://issues.apache.org/jira/browse/AMBARI-14885
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server, blueprints
>Affects Versions: 2.2.0
>Reporter: Amruta Borkar
>Assignee: Amruta Borkar
> Attachments: AMBARI-14885.patch, AMBARI-14885_02_05.patch
>
>
> After exporting blueprint from existing cluster knox_master_secret is also 
> exported
> Due to this property, the same blueprint file cannot be used to create 
> another cluster, this property need to be removed to create a cluster from 
> the file.
> User needs to remove the property in order to use the blueprint.
> Error shown:
> {
>   "status" : 400,
>   "message" : "Blueprint configuration validation failed: Secret references 
> are not allowed in blueprints, replace following properties with real 
> passwords:\n  Config:knox-env Property:knox_master_secret\n"
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43281: AMBARI-14885: After exporting blueprint from existing cluster knox_master_secret is exported.

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43281/#review118715
---


Ship it!




Ship It!

- Di Li


On Feb. 10, 2016, 7:23 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43281/
> ---
> 
> (Updated Feb. 10, 2016, 7:23 p.m.)
> 
> 
> Review request for Ambari, Di Li and Robert Nettleton.
> 
> 
> Bugs: AMBARI-14885
> https://issues.apache.org/jira/browse/AMBARI-14885
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-14885: After exporting blueprint from existing cluster 
> knox_master_secret is exported. This causes error while registering the 
> blueprint. 
> Following error occurs while using and exported blueprint if a password or 
> secret property is present in the blueprint:
> { "status" : 400, "message" : "Blueprint configuration validation failed: 
> Secret references are not allowed in blueprints, replace following properties 
> with real passwords:\n Config:knox-env Property:knox_master_secret\n" }
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  de31a0d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  7a77a25 
> 
> Diff: https://reviews.apache.org/r/43281/diff/
> 
> 
> Testing
> ---
> 
> In this patch, isPropertyIncluded() method is modified to include the 
> properties marked as secret. Corresponding test cases are also added.
>  The fix excludes properties marked as 'secret' while export blueprint. 
>  Test cases are added to test various combination by which a property can be 
> marked as 'secret'.
>  Test cases verify that the properties that do not match the rule "*SECRET" 
> are still included.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 42367: AMBARI-14574: multiple clicks on "Next" button causes skipping of steps while installing a cluster

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42367/#review118712
---


Ship it!




Ship It!

- Di Li


On Jan. 22, 2016, 7:14 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42367/
> ---
> 
> (Updated Jan. 22, 2016, 7:14 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, and Jaimin Jetly.
> 
> 
> Bugs: AMBARI-14574
> https://issues.apache.org/jira/browse/AMBARI-14574
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> On installation wizard, multiple clicks on the "Next" button can cause 
> skipping of steps. The timing of the 2nd click decides if that click will be 
> processed again or not. The following JIRAs had the same issue:
> AMBARI-7195
> AMBARI-7315
> The fix for the above issues helps to resolve most part of the problem. But 
> if the user happened to click again at the point when the last callback 
> function has just completed (and thus, has made the "Next" button clickable 
> again) but the next step has not yet completed its rendering, then the 2nd 
> click gets processed again. The "current step" was already updated in the 
> code to point to the next step, from the 1st click. So when this 2nd click 
> calls the "next" function in the router, the subsequent step of the "current 
> step" gets set as the current step. As a result we skip steps.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/wizard/step0_controller.js 5f134b5 
>   ambari-web/app/controllers/wizard/step3_controller.js a43303b 
>   ambari-web/app/controllers/wizard/step4_controller.js 17ce7a6 
>   ambari-web/app/controllers/wizard/step7_controller.js 7b423b6 
>   ambari-web/app/mixins/wizard/assign_master_components.js 5545e92 
>   ambari-web/app/router.js 92b2bae 
>   ambari-web/app/routes/installer.js e37a05d 
>   ambari-web/app/views/wizard/step0_view.js 07d3414 
>   ambari-web/app/views/wizard/step10_view.js 858a4b2 
>   ambari-web/app/views/wizard/step1_view.js 452e676 
>   ambari-web/app/views/wizard/step2_view.js 1988f4e 
>   ambari-web/app/views/wizard/step3_view.js 0aa5460 
>   ambari-web/app/views/wizard/step4_view.js 3def0c5 
>   ambari-web/app/views/wizard/step5_view.js a3f57b5 
>   ambari-web/app/views/wizard/step6_view.js e8ae176 
>   ambari-web/app/views/wizard/step7_view.js 9cea52f 
>   ambari-web/app/views/wizard/step8_view.js c653fdd 
>   ambari-web/app/views/wizard/step9_view.js e58186b 
>   ambari-web/test/controllers/wizard/step0_test.js 4349d04 
>   ambari-web/test/controllers/wizard/step3_test.js 7b6684f 
>   ambari-web/test/controllers/wizard/step4_test.js 9227279 
>   ambari-web/test/controllers/wizard/step5_test.js e0f8b71 
>   ambari-web/test/controllers/wizard/step7_test.js 762cf1f 
> 
> Diff: https://reviews.apache.org/r/42367/diff/
> 
> 
> Testing
> ---
> 
> CAUSE & FIX:
> The issue is seen in most of the wizard installation steps because the 
> properties responsible to disable the duplicate submit calls are unable to 
> account for the transition between the 'actions' and 'render' queues used by 
> the Ember framework. 
> The installation steps use the 'submit' function from controllers or the 
> 'next' function from installer to set the properties used in the checks for 
> processing the Next button clicks. These properties are reset again in the 
> views of next step if it renders or in the same controller/installer in case 
> of failures. Hence, the property that is responsible for the checks must be 
> accessible across controllers, installer and views. 
> 
> The fix proposed for this issue involves creating a new property 
> 'nextBtnClickInProgress' in the router, hence accessible in all the above 
> places. This property is set at suitable places in the controller/installer. 
> It is reset in Views of all the 11 steps of the  installation wizard and also 
> at appropriate places in some controllers (resetting for failure cases).
> The following steps have been found to skip steps and have been fixed with 
> the required checks:
> Step 1
> Step 3
> Step 4
> Step 5
> Step 7
> 
> TEST CASES:
> The test cases check if multiple submit calls succeed in skipping subsequent 
> steps. This is verified by checking the status of router's transitions call 
> after each submit call. Since the router's 'next' function for a step is the 
> last step before the conte

[jira] [Created] (AMBARI-14975) RU: HiveServer2 port should be restored instead of being changed to hardcoded 10010

[Di Li (JIRA)]

Di Li created AMBARI-14975:
--

 Summary: RU: HiveServer2 port should be restored instead of being 
changed to hardcoded 10010
 Key: AMBARI-14975
 URL: https://issues.apache.org/jira/browse/AMBARI-14975
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server, ambari-web
Affects Versions: 2.2.0
Reporter: Di Li
 Fix For: trunk






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 43189: AMBARI-14871: Better messages when Ambari web UI can't resolve hosts to build quicklinks

[Di Li]

> On Feb. 5, 2016, 4:36 p.m., Alexandr Antonenko wrote:
> > ambari-web/app/views/common/quick_view_link_view.js, line 443
> > <https://reviews.apache.org/r/43189/diff/1/?file=1232165#file1232165line443>
> >
> > Why are you making a compare with null ? I do not see any posible way 
> > oozieHostsArray can get null value

Right, I removed the unecessary check. Thanks Alex.


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43189/#review118026
-----------


On Feb. 5, 2016, 2:39 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43189/
> ---
> 
> (Updated Feb. 5, 2016, 2:39 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Jaimin Jetly.
> 
> 
> Bugs: AMBARI-14871
> https://issues.apache.org/jira/browse/AMBARI-14871
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> It currently says "Hostname is undefined". This JIRA is to update the logic a 
> little bit in an attempt to display better error messages.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/messages.js 8861b2a 
>   ambari-web/app/templates/main/service/item.hbs f61695e 
>   ambari-web/app/views/common/quick_view_link_view.js c2f7dcc 
>   ambari-web/test/views/common/quick_link_view_test.js 93a235c 
> 
> Diff: https://reviews.apache.org/r/43189/diff/
> 
> 
> Testing
> ---
> 
> unit test
> 
> manually patch a cluster and test the error message when oozie server is 
> stopped.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 43189: AMBARI-14871: Better messages when Ambari web UI can't resolve hosts to build quicklinks

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43189/
---

(Updated Feb. 5, 2016, 4:56 p.m.)


Review request for Ambari, Alexandr Antonenko and Jaimin Jetly.


Bugs: AMBARI-14871
https://issues.apache.org/jira/browse/AMBARI-14871


Repository: ambari


Description
---

It currently says "Hostname is undefined". This JIRA is to update the logic a 
little bit in an attempt to display better error messages.


Diffs (updated)
-

  ambari-web/app/messages.js f24876b 
  ambari-web/app/templates/main/service/item.hbs f61695e 
  ambari-web/app/views/common/quick_view_link_view.js c2f7dcc 
  ambari-web/test/views/common/quick_link_view_test.js 93a235c 

Diff: https://reviews.apache.org/r/43189/diff/


Testing
---

unit test

manually patch a cluster and test the error message when oozie server is 
stopped.


Thanks,

Di Li

Re: Review Request 43154: AMBARI-14766: Selecting ALL groups does not work under Manage Alert Notifications

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43154/#review118077
---


Ship it!




Ship It!

- Di Li


On Feb. 4, 2016, 5:49 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43154/
> ---
> 
> (Updated Feb. 4, 2016, 5:49 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, Jonathan Hurley, and 
> Oleg Nechiporenko.
> 
> 
> Bugs: AMBARI-14766
> https://issues.apache.org/jira/browse/AMBARI-14766
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Selecting "ALL" groups does not work under Manage Alert Notifications.
> 
> Reproduction Steps:-
> 1. Go to Alerts Tab in Ambari GUI
> 2. Actions --> Manage Notifications
> 3. Create New Alert Notification (Click the custom radio button for groups 
> and select couple of options eg HDFS, PIG)
> 4. Save it.
> 5. Edit the newly created Alert notification by clicking on the All radio 
> button. Everything gets selected.
> 6. Save it
> 
> Expected results:
> On Saving the Alert notification, the dialog box "Manage Alert Notification" 
> should show Groups as ALL ( or Show all the values eg HDFS, PIG, MR2). 
> Basically all the values should be selected.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertTargetResourceProvider.java
>  d28987d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertTargetResourceProviderTest.java
>  f80b6f7 
>   
> ambari-web/app/controllers/main/alerts/manage_alert_notifications_controller.js
>  38e4e75 
> 
> Diff: https://reviews.apache.org/r/43154/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> 
> The UI did not show "All" even after selecting "ALL" for Groups in the Edit 
> Alert Target pop-up because the property "selectedAlertGroup" was not set to 
> true even after the selection. This is fixed in the 
> "manageAlertGroupsController" in ambari-web.
> 
> However, this only resolved the UI issue. The alert notifications were still 
> not sent for all services after this change. This was because there was no 
> corresponding update made in the server code for this "ALL" Groups selection 
> for a Target. The DB has a table for AlertTarget (alert_target) and 
> AlertGroup (alert_group). There is a many-to-many mapping between these 2 
> tables, which is tracked in the combination table alert_group_target. 
> 1. This combination table was not getting updated for the "ALL" Groups 
> selection. 
> 2. Also, the Target was not updated as global in the DB. 
> 
> The "updateAlertTargets()" in "AlertTargetResourceProvider" is updated with 
> the fix for both these issues.
> The global property in Target is set if the ALL selection is made.
> And, if the Target is global, all the groups in the DB are upadted with the 
> current Target in the alert_group_target table.
> 
> 
> TESTS:
> ==
> 2 test cases are added to test for Custom and All Group selections.
> In case of Custom, the request to the REST API has group IDs passed and the 
> global paramter is false.
> The Target after this update must have global as false and should be mapped 
> to only the selected groups passed in the request.
> 
> In case of All, the request to the REST API does not have group IDs passed 
> and the global paramter is true.
> The Target after this update must contain global as true and should be mapped 
> to all the groups in the DB.
> 
> 
> Thanks,
> 
> Keta Patel
> 
>

Re: RU: HiveServer2 port conflict when RU 2.2 to 2.3 then to 2.4 ?

[Di Li]

Hello folks,
Do we have a JIRA to track this one already?
Thanks
Di.

  From: Jonathan Hurley <jhur...@hortonworks.com>
 To: Ambari <dev@ambari.apache.org>; Di Li <lovelyco...@yahoo.com> 
Cc: Nate Cole <nc...@hortonworks.com>
 Sent: Thursday, February 4, 2016 6:23 PM
 Subject: Re: RU: HiveServer2 port conflict when RU 2.2 to 2.3 then to 2.4 ?
   
Yes. Most Ambari administrators actually change this back to 1 after the 
upgrade since they have hard coded scripts which they want to use. We've 
discussed adding the ability for a while now to increase by a value instead of 
setting the port directly. It's just never been prioritized. 

> On Feb 4, 2016, at 4:44 PM, Di Li <lovelyco...@yahoo.com.INVALID> wrote:
> 
> Hello folks,
> 
> Thanks for the reply.  I was thinking of more like automatically increasing 
> (for RU) or decreasing (for rolling downgrade) like by 10 against the base 
> port ( aka whatever port value a customer may specify during the cluster 
> installation time, may not be the default 1 though).
> 
> For example, if the customer set HiveServer2 port to 3, then I guess the 
> upgrade logic would ideally be able to change it to 30010 for RU (like 
> port_value + 10)  and set that back to 3 (like port_value - 10) for 
> downgrade.
> 
> 
> From: Alejandro Fernandez <afernan...@hortonworks.com>
> To: "dev@ambari.apache.org" <dev@ambari.apache.org>; Di Li 
> <lovelyco...@yahoo.com>; Nate Cole <nc...@hortonworks.com>; Jonathan Hurley 
> <jhur...@hortonworks.com> 
> Sent: Thursday, February 4, 2016 4:22 PM
> Subject: Re: RU: HiveServer2 port conflict when RU 2.2 to 2.3 then to 2.4 ?
> 
> That's a good observation. I think it should have logic to always swap.
> If hive.server2.transport.mode is set to "binary", then swap
> hive.server2.thrift.port between 10,000 and 10,010
> If hive.server2.transport.mode is set to "http", then swap
> hive.server2.http.port    between 10,001 and 10,011
> 
> 
> Thanks,
> Alejandro
> 
> On 2/4/16, 1:10 PM, "Di Li" <lovelyco...@yahoo.com.INVALID> wrote:
> 
>> Hello folks,
>> 
>> When rolling upgrade HDP 2.2 to 2.3, HiveServer2 port is changed from
>> 1 to 10010. The same port change logic is in upgrade-2.4.xml as well.
>> 
>> But wouldn't this make no difference if I rolling upgrade a 2.3 cluster
>> that was originally rolling upgrade from an HDP 2.2 cluster , where the
>> port had already been changed to 10010? It'd be like changing the port
>> from 10010 (on HDP 2.3) to 10010 (on HDP 2.4), right ?
>> 
>> How does Ambari  handle Hive Server 2 port value change in  this
>> continuously upgrade case?
>> 
>> 
>> Thank you.
>> 
>> Di Li
> 
> 


  

RU: HiveServer2 port conflict when RU 2.2 to 2.3 then to 2.4 ?

[Di Li]

Hello folks,

When rolling upgrade HDP 2.2 to 2.3, HiveServer2 port is changed from 1 to 
10010. The same port change logic is in upgrade-2.4.xml as well. 

But wouldn't this make no difference if I rolling upgrade a 2.3 cluster that 
was originally rolling upgrade from an HDP 2.2 cluster , where the port had 
already been changed to 10010? It'd be like changing the port from 10010 (on 
HDP 2.3) to 10010 (on HDP 2.4), right ?

How does Ambari  handle Hive Server 2 port value change in  this continuously 
upgrade case?


Thank you.

Di Li

Re: RU: HiveServer2 port conflict when RU 2.2 to 2.3 then to 2.4 ?

[Di Li]

Hello folks,
 
Thanks for the reply.  I was thinking of more like automatically increasing 
(for RU) or decreasing (for rolling downgrade) like by 10 against the base port 
( aka whatever port value a customer may specify during the cluster 
installation time, may not be the default 1 though).

For example, if the customer set HiveServer2 port to 3, then I guess the 
upgrade logic would ideally be able to change it to 30010 for RU (like 
port_value + 10)  and set that back to 3 (like port_value - 10) for 
downgrade.


 From: Alejandro Fernandez <afernan...@hortonworks.com>
 To: "dev@ambari.apache.org" <dev@ambari.apache.org>; Di Li 
<lovelyco...@yahoo.com>; Nate Cole <nc...@hortonworks.com>; Jonathan Hurley 
<jhur...@hortonworks.com> 
 Sent: Thursday, February 4, 2016 4:22 PM
 Subject: Re: RU: HiveServer2 port conflict when RU 2.2 to 2.3 then to 2.4 ?
   
That's a good observation. I think it should have logic to always swap.
If hive.server2.transport.mode is set to "binary", then swap
hive.server2.thrift.port between 10,000 and 10,010
If hive.server2.transport.mode is set to "http", then swap
hive.server2.http.port    between 10,001 and 10,011


Thanks,
Alejandro

On 2/4/16, 1:10 PM, "Di Li" <lovelyco...@yahoo.com.INVALID> wrote:

>Hello folks,
>
>When rolling upgrade HDP 2.2 to 2.3, HiveServer2 port is changed from
>1 to 10010. The same port change logic is in upgrade-2.4.xml as well.
>
>But wouldn't this make no difference if I rolling upgrade a 2.3 cluster
>that was originally rolling upgrade from an HDP 2.2 cluster , where the
>port had already been changed to 10010? It'd be like changing the port
>from 10010 (on HDP 2.3) to 10010 (on HDP 2.4), right ?
>
>How does Ambari  handle Hive Server 2 port value change in  this
>continuously upgrade case?
>
>
>Thank you.
>
>Di Li


  

Re: Review Request 43154: AMBARI-14766: Selecting ALL groups does not work under Manage Alert Notifications

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43154/#review117803
---



hello Keta, 

Please add Jonathan Hurley to the review.

- Di Li


On Feb. 3, 2016, 11:34 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43154/
> ---
> 
> (Updated Feb. 3, 2016, 11:34 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, and Oleg Nechiporenko.
> 
> 
> Bugs: AMBARI-14766
> https://issues.apache.org/jira/browse/AMBARI-14766
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Selecting "ALL" groups does not work under Manage Alert Notifications.
> 
> Reproduction Steps:-
> 1. Go to Alerts Tab in Ambari GUI
> 2. Actions --> Manage Notifications
> 3. Create New Alert Notification (Click the custom radio button for groups 
> and select couple of options eg HDFS, PIG)
> 4. Save it.
> 5. Edit the newly created Alert notification by clicking on the All radio 
> button. Everything gets selected.
> 6. Save it
> 
> Expected results:
> On Saving the Alert notification, the dialog box "Manage Alert Notification" 
> should show Groups as ALL ( or Show all the values eg HDFS, PIG, MR2). 
> Basically all the values should be selected.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertTargetResourceProvider.java
>  d28987d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertTargetResourceProviderTest.java
>  f80b6f7 
>   
> ambari-web/app/controllers/main/alerts/manage_alert_notifications_controller.js
>  38e4e75 
> 
> Diff: https://reviews.apache.org/r/43154/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> 
> The UI did not show "All" even after selecting "ALL" for Groups in the Edit 
> Alert Target pop-up because the property "selectedAlertGroup" was not set to 
> true even after the selection. This is fixed in the 
> "manageAlertGroupsController" in ambari-web.
> 
> However, this only resolved the UI issue. The alert notifications were still 
> not sent for all services after this change. This was because there was no 
> corresponding update made in the server code for this "ALL" Groups selection 
> for a Target. The DB has a table for AlertTarget (alert_target) and 
> AlertGroup (alert_group). There is a many-to-many mapping between these 2 
> tables, which is tracked in the combination table alert_group_target. 
> 1. This combination table was not getting updated for the "ALL" Groups 
> selection. 
> 2. Also, the Target was not updated as global in the DB. 
> 
> The "updateAlertTargets()" in "AlertTargetResourceProvider" is updated with 
> the fix for both these issues.
> The global property in Target is set if the ALL selection is made.
> And, if the Target is global, all the groups in the DB are upadted with the 
> current Target in the alert_group_target table.
> 
> 
> TESTS:
> ==
> 2 test cases are added to test for Custom and All Group selections.
> In case of Custom, the request to the REST API has group IDs passed and the 
> global paramter is false.
> The Target after this update must have global as false and should be mapped 
> to only the selected groups passed in the request.
> 
> In case of All, the request to the REST API does not have group IDs passed 
> and the global paramter is true.
> The Target after this update must contain global as true and should be mapped 
> to all the groups in the DB.
> 
> 
> Thanks,
> 
> Keta Patel
> 
>

Review Request 43189: AMBARI-14871: Better messages when Ambari web UI can't resolve hosts to build quicklinks

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43189/
---

Review request for Ambari and Jaimin Jetly.


Bugs: AMBARI-14871
https://issues.apache.org/jira/browse/AMBARI-14871


Repository: ambari


Description
---

It currently says "Hostname is undefined". This JIRA is to update the logic a 
little bit in an attempt to display better error messages.


Diffs
-

  ambari-web/app/messages.js 8861b2a 
  ambari-web/app/templates/main/service/item.hbs f61695e 
  ambari-web/app/views/common/quick_view_link_view.js c2f7dcc 
  ambari-web/test/views/common/quick_link_view_test.js 93a235c 

Diff: https://reviews.apache.org/r/43189/diff/


Testing
---

unit test

manually patch a cluster and test the error message when oozie server is 
stopped.


Thanks,

Di Li

Review Request 43147: AMBARI-14906: Text update for conf_select.py

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43147/
---

Review request for Ambari and Jaimin Jetly.


Bugs: AMBARI-14906
https://issues.apache.org/jira/browse/AMBARI-14906


Repository: ambari


Description
---

A small text update for conf_select.py


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 2bc01ee 

Diff: https://reviews.apache.org/r/43147/diff/


Testing
---

manually patch a cluster to test it out.


Thanks,

Di Li