[jira] [Commented] (AMBARI-18368) Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486256#comment-15486256
 ] 

Hadoop QA commented on AMBARI-18368:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828158/AMBARI-18368.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
ambari-server.

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8640//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8640//console

This message is automatically generated.

> Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas 
> Service
> -
>
> Key: AMBARI-18368
> URL: https://issues.apache.org/jira/browse/AMBARI-18368
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.4.0
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
>Priority: Critical
> Fix For: trunk, 2.5.0
>
> Attachments: AMBARI-18368.patch
>
>
> Steps to Reproduce:
> * Install Ambari 2.2.2 with HDP 2.4 (HBase, Solr)
> * Kerberize the cluster
> * Perform EU/RU to HDP 2.5
> * Add Atlas Service
> Atlas Server log contains,
> {code}
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not 
> find the specified config set: vertex_index
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.createCollectionIfNotExists(Solr5Index.java:901)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.register(Solr5Index.java:269)
> at 
> com.thinkaurelius.titan.diskstorage.indexing.IndexTransaction.register(IndexTransaction.java:83)
> at 
> com.thinkaurelius.titan.graphdb.database.IndexSerializer.register(IndexSerializer.java:92)
> at 
> com.thinkaurelius.titan.graphdb.database.management.ManagementSystem.addIndexKey(ManagementSystem.java:534)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.enhanceMixedIndex(GraphBackedSearchIndexer.java:405)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.createIndexes(GraphBackedSearchIndexer.java:334)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.initialize(GraphBackedSearchIndexer.java:103)
> ... 71 more
> {code}
> Atlas tables in HBase look ok.
> {code}
> su hbase
> kinit -kt /etc/security/keytabs/hbase.headless.keytab cstm-hb...@example.com
> hbase shell
> hbase(main):001:0> list
> TABLE
> ATLAS_ENTITY_AUDIT_EVENTS
> atlas_titan
> 2 row(s) in 1.4300 seconds
> => ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan"]
> {code}
> h4. Workaround
> 1. Stop Atlas Server
> 2. Copy solr xml files to correct config folder and chown as 
> $atlas_user:$hadoop_group
> {code}
> cp -R /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/solr/* /etc/atlas/conf/solr/
> cp: overwrite `/etc/atlas/conf/solr/solrconfig.xml'? n
> chown atlas:hadoop /etc/atlas/conf/solr/*
> cp 

[jira] [Updated] (AMBARI-18368) Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Alejandro Fernandez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alejandro Fernandez updated AMBARI-18368:
-
Attachment: (was: AMBARI-18368.patch)

> Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas 
> Service
> -
>
> Key: AMBARI-18368
> URL: https://issues.apache.org/jira/browse/AMBARI-18368
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.4.0
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
>Priority: Critical
> Fix For: trunk, 2.5.0
>
> Attachments: AMBARI-18368.patch
>
>
> Steps to Reproduce:
> * Install Ambari 2.2.2 with HDP 2.4 (HBase, Solr)
> * Kerberize the cluster
> * Perform EU/RU to HDP 2.5
> * Add Atlas Service
> Atlas Server log contains,
> {code}
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not 
> find the specified config set: vertex_index
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.createCollectionIfNotExists(Solr5Index.java:901)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.register(Solr5Index.java:269)
> at 
> com.thinkaurelius.titan.diskstorage.indexing.IndexTransaction.register(IndexTransaction.java:83)
> at 
> com.thinkaurelius.titan.graphdb.database.IndexSerializer.register(IndexSerializer.java:92)
> at 
> com.thinkaurelius.titan.graphdb.database.management.ManagementSystem.addIndexKey(ManagementSystem.java:534)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.enhanceMixedIndex(GraphBackedSearchIndexer.java:405)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.createIndexes(GraphBackedSearchIndexer.java:334)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.initialize(GraphBackedSearchIndexer.java:103)
> ... 71 more
> {code}
> Atlas tables in HBase look ok.
> {code}
> su hbase
> kinit -kt /etc/security/keytabs/hbase.headless.keytab cstm-hb...@example.com
> hbase shell
> hbase(main):001:0> list
> TABLE
> ATLAS_ENTITY_AUDIT_EVENTS
> atlas_titan
> 2 row(s) in 1.4300 seconds
> => ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan"]
> {code}
> h4. Workaround
> 1. Stop Atlas Server
> 2. Copy solr xml files to correct config folder and chown as 
> $atlas_user:$hadoop_group
> {code}
> cp -R /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/solr/* /etc/atlas/conf/solr/
> cp: overwrite `/etc/atlas/conf/solr/solrconfig.xml'? n
> chown atlas:hadoop /etc/atlas/conf/solr/*
> cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/users-credentials.properties 
> /etc/atlas/conf/
> cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/policy-store.txt /etc/atlas/conf/
> chown atlas:hadoop /etc/atlas/conf/users-credentials.properties
> chown atlas:hadoop /etc/atlas/conf/policy-store.txt
> {code}
> 3. Delete zookeeper znode,
> {code}
> # kinit -kt /etc/security/keytabs/atlas.service.keytab  atlas/@
> # cd /usr/hdp/current/zookeeper-client/bin/ 
> # ./zkCli.sh -server :
> [ .. (CONNECTED) ] rmr  /infra-solr/configs/atlas_configs
> {code}
> 4. Ensure Atlas application-properties are present for,
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> 5. Start Atlas



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18368) Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Alejandro Fernandez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alejandro Fernandez updated AMBARI-18368:
-
Attachment: AMBARI-18368.patch

> Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas 
> Service
> -
>
> Key: AMBARI-18368
> URL: https://issues.apache.org/jira/browse/AMBARI-18368
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.4.0
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
>Priority: Critical
> Fix For: trunk, 2.5.0
>
> Attachments: AMBARI-18368.patch
>
>
> Steps to Reproduce:
> * Install Ambari 2.2.2 with HDP 2.4 (HBase, Solr)
> * Kerberize the cluster
> * Perform EU/RU to HDP 2.5
> * Add Atlas Service
> Atlas Server log contains,
> {code}
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not 
> find the specified config set: vertex_index
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.createCollectionIfNotExists(Solr5Index.java:901)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.register(Solr5Index.java:269)
> at 
> com.thinkaurelius.titan.diskstorage.indexing.IndexTransaction.register(IndexTransaction.java:83)
> at 
> com.thinkaurelius.titan.graphdb.database.IndexSerializer.register(IndexSerializer.java:92)
> at 
> com.thinkaurelius.titan.graphdb.database.management.ManagementSystem.addIndexKey(ManagementSystem.java:534)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.enhanceMixedIndex(GraphBackedSearchIndexer.java:405)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.createIndexes(GraphBackedSearchIndexer.java:334)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.initialize(GraphBackedSearchIndexer.java:103)
> ... 71 more
> {code}
> Atlas tables in HBase look ok.
> {code}
> su hbase
> kinit -kt /etc/security/keytabs/hbase.headless.keytab cstm-hb...@example.com
> hbase shell
> hbase(main):001:0> list
> TABLE
> ATLAS_ENTITY_AUDIT_EVENTS
> atlas_titan
> 2 row(s) in 1.4300 seconds
> => ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan"]
> {code}
> h4. Workaround
> 1. Stop Atlas Server
> 2. Copy solr xml files to correct config folder and chown as 
> $atlas_user:$hadoop_group
> {code}
> cp -R /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/solr/* /etc/atlas/conf/solr/
> cp: overwrite `/etc/atlas/conf/solr/solrconfig.xml'? n
> chown atlas:hadoop /etc/atlas/conf/solr/*
> cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/users-credentials.properties 
> /etc/atlas/conf/
> cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/policy-store.txt /etc/atlas/conf/
> chown atlas:hadoop /etc/atlas/conf/users-credentials.properties
> chown atlas:hadoop /etc/atlas/conf/policy-store.txt
> {code}
> 3. Delete zookeeper znode,
> {code}
> # kinit -kt /etc/security/keytabs/atlas.service.keytab  atlas/@
> # cd /usr/hdp/current/zookeeper-client/bin/ 
> # ./zkCli.sh -server :
> [ .. (CONNECTED) ] rmr  /infra-solr/configs/atlas_configs
> {code}
> 4. Ensure Atlas application-properties are present for,
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> 5. Start Atlas



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18368) Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Alejandro Fernandez (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485944#comment-15485944
 ] 

Alejandro Fernandez commented on AMBARI-18368:
--

To summarize, Atlas is failing because config files do not have the correct 
permissions as they are not managed by Ambari.
My local cluster went through HDP 2.4, EU to HDP 2.5, and then added Atlas.

Before the EU, /usr/hdp/2.5.0.0-1245/etc/ doesn't contain any atlas dirs, as it 
should.
After EU, I ran "yum install atlas-metadata_2_5_0_0_1245" and this creates 
/usr/hdp/2.5.0.0-1245/etc/atlas/conf.dist/ , which contains the "solr" 
subdirectory, policy-store.txt, and users-credentials.properties

At this point in time, those 2 files and the solr dir are present in two places,
1.
{code}
/usr/hdp/current/atlas-client -> /usr/hdp/2.5.0.0-1245/atlas
/usr/hdp/current/atlas-client/conf -> /etc/atlas/conf

[root@c6404 etc]# ls -la /etc/atlas/conf  (this is a directory that will be 
backed up, since Ambari needs to make this a symlink instead)
-rw-r--r-- 1 root root 8054 Aug 26 04:49 atlas-application.properties
-rw-r--r-- 1 root root 3208 Aug 26 04:49 atlas-env.sh
-rw-r--r-- 1 root root 3912 Aug 26 04:49 atlas-log4j.xml
drwxr-xr-x 2 root root 4096 Sep 13 00:13 hbase
-rw-r--r-- 1 root root  623 Aug 26 04:49 policy-store.txt
drwxr-xr-x 3 root root 4096 Sep 13 00:13 solr
-rw-r--r-- 1 root root  207 Aug 26 04:49 users-credentials.properties
{code}

And 2 (this dir is not managed by Ambari),
{code}
[root@c6404 etc]# ls -la  /usr/hdp/2.5.0.0-1245/etc/atlas/conf.dist/
-rw-r--r-- 1 root root 8054 Aug 26 04:49 atlas-application.properties
-rw-r--r-- 1 root root 3208 Aug 26 04:49 atlas-env.sh
-rw-r--r-- 1 root root 3912 Aug 26 04:49 atlas-log4j.xml
drwxr-xr-x 2 root root 4096 Sep 13 00:13 hbase
-rw-r--r-- 1 root root  623 Aug 26 04:49 policy-store.txt
drwxr-xr-x 3 root root 4096 Sep 13 00:13 solr
-rw-r--r-- 1 root root  207 Aug 26 04:49 users-credentials.properties
{code}

During the Atlas Install command, Ambari log shows this,
{code}
2016-09-13 00:22:52,189 - Seeding versioned configuration directories for atlas
2016-09-13 00:22:52,189 - Execute['ambari-sudo.sh  -H -E cp -R -p -v 
/usr/hdp/current/atlas-client/conf/* /etc/atlas/2.5.0.0-1245/0'] {'logoutput': 
True}
`/usr/hdp/current/atlas-client/conf/atlas-application.properties' -> 
`/etc/atlas/2.5.0.0-1245/0/atlas-application.properties'
`/usr/hdp/current/atlas-client/conf/atlas-env.sh' -> 
`/etc/atlas/2.5.0.0-1245/0/atlas-env.sh'
`/usr/hdp/current/atlas-client/conf/atlas-log4j.xml' -> 
`/etc/atlas/2.5.0.0-1245/0/atlas-log4j.xml'
`/usr/hdp/current/atlas-client/conf/atlas_jaas.conf' -> 
`/etc/atlas/2.5.0.0-1245/0/atlas_jaas.conf'
`/usr/hdp/current/atlas-client/conf/hbase' -> `/etc/atlas/2.5.0.0-1245/0/hbase'
`/usr/hdp/current/atlas-client/conf/hbase/hbase-site.xml.template' -> 
`/etc/atlas/2.5.0.0-1245/0/hbase/hbase-site.xml.template'
`/usr/hdp/current/atlas-client/conf/policy-store.txt' -> 
`/etc/atlas/2.5.0.0-1245/0/policy-store.txt'
`/usr/hdp/current/atlas-client/conf/solr' -> `/etc/atlas/2.5.0.0-1245/0/solr'
`/usr/hdp/current/atlas-client/conf/solr/lang' -> 
`/etc/atlas/2.5.0.0-1245/0/solr/lang'
`/usr/hdp/current/atlas-client/conf/solr/lang/stopwords_en.txt' -> 
`/etc/atlas/2.5.0.0-1245/0/solr/lang/stopwords_en.txt'
`/usr/hdp/current/atlas-client/conf/solr/stopwords.txt' -> 
`/etc/atlas/2.5.0.0-1245/0/solr/stopwords.txt'
`/usr/hdp/current/atlas-client/conf/solr/synonyms.txt' -> 
`/etc/atlas/2.5.0.0-1245/0/solr/synonyms.txt'
`/usr/hdp/current/atlas-client/conf/solr/schema.xml' -> 
`/etc/atlas/2.5.0.0-1245/0/solr/schema.xml'
`/usr/hdp/current/atlas-client/conf/solr/currency.xml' -> 
`/etc/atlas/2.5.0.0-1245/0/solr/currency.xml'
`/usr/hdp/current/atlas-client/conf/solr/solrconfig.xml' -> 
`/etc/atlas/2.5.0.0-1245/0/solr/solrconfig.xml'
`/usr/hdp/current/atlas-client/conf/solr/protwords.txt' -> 
`/etc/atlas/2.5.0.0-1245/0/solr/protwords.txt'
`/usr/hdp/current/atlas-client/conf/users-credentials.properties' -> 
`/etc/atlas/2.5.0.0-1245/0/users-credentials.properties'
2016-09-13 00:22:52,195 - Execute['ambari-sudo.sh  -H -E cp -R -p 
/etc/atlas/conf/* /etc/atlas/2.5.0.0-1245/0'] {'only_if': 'ls -d 
/etc/atlas/conf/*'}
2016-09-13 00:22:52,204 - Checking if need to create versioned conf dir 
/etc/atlas/2.5.0.0-1245/0
2016-09-13 00:22:52,204 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
'create-conf-dir', '--package', 'atlas', '--stack-version', '2.5.0.0-1245', 
'--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
'stderr': -1}
2016-09-13 00:22:52,224 - call returned (1, '/etc/atlas/2.5.0.0-1245/0 exist 
already', '')
2016-09-13 00:22:52,225 - checked_call[('ambari-python-wrap', 
'/usr/bin/conf-select', 'set-conf-dir', '--package', 'atlas', 
'--stack-version', '2.5.0.0-1245', '--conf-version', '0')] {'logoutput': False, 
'sudo': True, 'quiet': False}
2016-09-13 00:22:52,243 - checked_call 

[jira] [Updated] (AMBARI-18368) Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Alejandro Fernandez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alejandro Fernandez updated AMBARI-18368:
-
Attachment: AMBARI-18368.patch

> Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas 
> Service
> -
>
> Key: AMBARI-18368
> URL: https://issues.apache.org/jira/browse/AMBARI-18368
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.4.0
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
>Priority: Critical
> Fix For: trunk, 2.5.0
>
> Attachments: AMBARI-18368.patch
>
>
> Steps to Reproduce:
> * Install Ambari 2.2.2 with HDP 2.4 (HBase, Solr)
> * Kerberize the cluster
> * Perform EU/RU to HDP 2.5
> * Add Atlas Service
> Atlas Server log contains,
> {code}
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not 
> find the specified config set: vertex_index
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.createCollectionIfNotExists(Solr5Index.java:901)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.register(Solr5Index.java:269)
> at 
> com.thinkaurelius.titan.diskstorage.indexing.IndexTransaction.register(IndexTransaction.java:83)
> at 
> com.thinkaurelius.titan.graphdb.database.IndexSerializer.register(IndexSerializer.java:92)
> at 
> com.thinkaurelius.titan.graphdb.database.management.ManagementSystem.addIndexKey(ManagementSystem.java:534)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.enhanceMixedIndex(GraphBackedSearchIndexer.java:405)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.createIndexes(GraphBackedSearchIndexer.java:334)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.initialize(GraphBackedSearchIndexer.java:103)
> ... 71 more
> {code}
> Atlas tables in HBase look ok.
> {code}
> su hbase
> kinit -kt /etc/security/keytabs/hbase.headless.keytab cstm-hb...@example.com
> hbase shell
> hbase(main):001:0> list
> TABLE
> ATLAS_ENTITY_AUDIT_EVENTS
> atlas_titan
> 2 row(s) in 1.4300 seconds
> => ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan"]
> {code}
> h4. Workaround
> 1. Stop Atlas Server
> 2. Copy solr xml files to correct config folder and chown as 
> $atlas_user:$hadoop_group
> {code}
> cp -R /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/solr/* /etc/atlas/conf/solr/
> cp: overwrite `/etc/atlas/conf/solr/solrconfig.xml'? n
> chown atlas:hadoop /etc/atlas/conf/solr/*
> cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/users-credentials.properties 
> /etc/atlas/conf/
> cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/policy-store.txt /etc/atlas/conf/
> chown atlas:hadoop /etc/atlas/conf/users-credentials.properties
> chown atlas:hadoop /etc/atlas/conf/policy-store.txt
> {code}
> 3. Delete zookeeper znode,
> {code}
> # kinit -kt /etc/security/keytabs/atlas.service.keytab  atlas/@
> # cd /usr/hdp/current/zookeeper-client/bin/ 
> # ./zkCli.sh -server :
> [ .. (CONNECTED) ] rmr  /infra-solr/configs/atlas_configs
> {code}
> 4. Ensure Atlas application-properties are present for,
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> 5. Start Atlas



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18368) Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Alejandro Fernandez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alejandro Fernandez updated AMBARI-18368:
-
Status: Patch Available  (was: Open)

> Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas 
> Service
> -
>
> Key: AMBARI-18368
> URL: https://issues.apache.org/jira/browse/AMBARI-18368
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.4.0
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
>Priority: Critical
> Fix For: trunk, 2.5.0
>
> Attachments: AMBARI-18368.patch
>
>
> Steps to Reproduce:
> * Install Ambari 2.2.2 with HDP 2.4 (HBase, Solr)
> * Kerberize the cluster
> * Perform EU/RU to HDP 2.5
> * Add Atlas Service
> Atlas Server log contains,
> {code}
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not 
> find the specified config set: vertex_index
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.createCollectionIfNotExists(Solr5Index.java:901)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.register(Solr5Index.java:269)
> at 
> com.thinkaurelius.titan.diskstorage.indexing.IndexTransaction.register(IndexTransaction.java:83)
> at 
> com.thinkaurelius.titan.graphdb.database.IndexSerializer.register(IndexSerializer.java:92)
> at 
> com.thinkaurelius.titan.graphdb.database.management.ManagementSystem.addIndexKey(ManagementSystem.java:534)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.enhanceMixedIndex(GraphBackedSearchIndexer.java:405)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.createIndexes(GraphBackedSearchIndexer.java:334)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.initialize(GraphBackedSearchIndexer.java:103)
> ... 71 more
> {code}
> Atlas tables in HBase look ok.
> {code}
> su hbase
> kinit -kt /etc/security/keytabs/hbase.headless.keytab cstm-hb...@example.com
> hbase shell
> hbase(main):001:0> list
> TABLE
> ATLAS_ENTITY_AUDIT_EVENTS
> atlas_titan
> 2 row(s) in 1.4300 seconds
> => ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan"]
> {code}
> h4. Workaround
> 1. Stop Atlas Server
> 2. Copy solr xml files to correct config folder and chown as 
> $atlas_user:$hadoop_group
> {code}
> cp -R /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/solr/* /etc/atlas/conf/solr/
> cp: overwrite `/etc/atlas/conf/solr/solrconfig.xml'? n
> chown atlas:hadoop /etc/atlas/conf/solr/*
> cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/users-credentials.properties 
> /etc/atlas/conf/
> cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/policy-store.txt /etc/atlas/conf/
> chown atlas:hadoop /etc/atlas/conf/users-credentials.properties
> chown atlas:hadoop /etc/atlas/conf/policy-store.txt
> {code}
> 3. Delete zookeeper znode,
> {code}
> # kinit -kt /etc/security/keytabs/atlas.service.keytab  atlas/@
> # cd /usr/hdp/current/zookeeper-client/bin/ 
> # ./zkCli.sh -server :
> [ .. (CONNECTED) ] rmr  /infra-solr/configs/atlas_configs
> {code}
> 4. Ensure Atlas application-properties are present for,
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> 5. Start Atlas



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18255) Add unique constraint to host_version table

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485702#comment-15485702
 ] 

Hudson commented on AMBARI-18255:
-

ABORTED: Integrated in Jenkins build Ambari-branch-2.5 #19 (See 
[https://builds.apache.org/job/Ambari-branch-2.5/19/])
AMBARI-18255. Add unique constraint to host_version table (ncole) (ncole: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=4e9ed5c0264b7283296fc7f66cb1b418e43977c5])
* (edit) 
ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql
* (edit) ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql
* (add) 
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
* (edit) 
ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostVersionDAOTest.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostVersionEntity.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java
* (edit) ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
* (add) 
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
* (edit) ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql
* (edit) ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql
* (edit) ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql
* (edit) ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql


> Add unique constraint to host_version table
> ---
>
> Key: AMBARI-18255
> URL: https://issues.apache.org/jira/browse/AMBARI-18255
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Reporter: Nate Cole
>Assignee: Nate Cole
>Priority: Critical
> Fix For: trunk
>
> Attachments: AMBARI-18255.patch
>
>
> The software is allowing duplicate records into the {{host_version}} table.  
> This should be constrained in the db.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-8458) Add support for "add hosts" specifying host name, blueprint name and host group name

[Amruta Borkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485695#comment-15485695
 ] 

Amruta Borkar commented on AMBARI-8458:
---

Hi Jeff,
In point 1. The blueprint that is created. The host groups 'worker' and 
'master' were already present in the previous blueprint or those are new 
hostgroups.

> Add support for "add hosts" specifying host name, blueprint name and host 
> group name 
> -
>
> Key: AMBARI-8458
> URL: https://issues.apache.org/jira/browse/AMBARI-8458
> Project: Ambari
>  Issue Type: Improvement
>Reporter: John Speidel
>Assignee: John Speidel
> Fix For: 2.0.0
>
>
> Provide a higher level api for host provisioning.
> This api will accept a host name, blueprint name and host group.
> The result of this api call will be fully operational hosts added to the 
> existing cluster with configuration and components which are defined in the 
> specified blueprint/host_group.  All components on the added hosts will be 
> installed and started.
> All hosts must be reachable via ambari server and have the ambari agent 
> running and registered with the ambari server prior to using this api.  
> This is an asynchronous api so it will return the standard asynchronous 
> response.
> {code}
> 202 - Accepted
> {
>   "href" : "http://AMBARI_HOST:8080/api/v1/clusters/c1/requests/2;,
>   "Requests" : {
> "id" : 2,
> "status" : "InProgress"
>   }
> }
> {code}
> This api will support adding a single host or multiple hosts.
> h4. Single Host:
> {code}
> POST http://AMBARI_HOST:8080/api/v1/clusters/c1/hosts/newHostName.domain
> {
> "blueprint" : "my_blueprint",
> "host_group" : "slave"
> }
> {code}
> h4. Multiple Hosts
> {code}
> POST http://AMBARI_HOST:8080/api/v1/clusters/c1/hosts
> [
>   {
>   "blueprint" : "my_blueprint"
>   "host_group" : "slave",
>   "host_name" : "host2.domain"
>   },
>   {
>   "blueprint" : "my_blueprint",
>   "host_group" : "slave",
>   "host_name" : "host5.domain"
>   }
> ]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18368) Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-18368:
---
Fix Version/s: 2.5.0

> Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas 
> Service
> -
>
> Key: AMBARI-18368
> URL: https://issues.apache.org/jira/browse/AMBARI-18368
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.4.0
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
>Priority: Critical
> Fix For: trunk, 2.5.0
>
>
> Steps to Reproduce:
> * Install Ambari 2.2.2 with HDP 2.4 (HBase, Solr)
> * Kerberize the cluster
> * Perform EU/RU to HDP 2.5
> * Add Atlas Service
> Atlas Server log contains,
> {code}
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not 
> find the specified config set: vertex_index
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
> at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
> at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.createCollectionIfNotExists(Solr5Index.java:901)
> at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.register(Solr5Index.java:269)
> at 
> com.thinkaurelius.titan.diskstorage.indexing.IndexTransaction.register(IndexTransaction.java:83)
> at 
> com.thinkaurelius.titan.graphdb.database.IndexSerializer.register(IndexSerializer.java:92)
> at 
> com.thinkaurelius.titan.graphdb.database.management.ManagementSystem.addIndexKey(ManagementSystem.java:534)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.enhanceMixedIndex(GraphBackedSearchIndexer.java:405)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.createIndexes(GraphBackedSearchIndexer.java:334)
> at 
> org.apache.atlas.repository.graph.GraphBackedSearchIndexer.initialize(GraphBackedSearchIndexer.java:103)
> ... 71 more
> {code}
> Atlas tables in HBase look ok.
> {code}
> su hbase
> kinit -kt /etc/security/keytabs/hbase.headless.keytab cstm-hb...@example.com
> hbase shell
> hbase(main):001:0> list
> TABLE
> ATLAS_ENTITY_AUDIT_EVENTS
> atlas_titan
> 2 row(s) in 1.4300 seconds
> => ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan"]
> {code}
> h4. Workaround
> 1. Stop Atlas Server
> 2. Copy solr xml files to correct config folder and chown as 
> $atlas_user:$hadoop_group
> {code}
> cp -R /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/solr/* /etc/atlas/conf/solr/
> cp: overwrite `/etc/atlas/conf/solr/solrconfig.xml'? n
> chown atlas:hadoop /etc/atlas/conf/solr/*
> cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/users-credentials.properties 
> /etc/atlas/conf/
> cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/policy-store.txt /etc/atlas/conf/
> chown atlas:hadoop /etc/atlas/conf/users-credentials.properties
> chown atlas:hadoop /etc/atlas/conf/policy-store.txt
> {code}
> 3. Delete zookeeper znode,
> {code}
> # kinit -kt /etc/security/keytabs/atlas.service.keytab  atlas/@
> # cd /usr/hdp/current/zookeeper-client/bin/ 
> # ./zkCli.sh -server :
> [ .. (CONNECTED) ] rmr  /infra-solr/configs/atlas_configs
> {code}
> 4. Ensure Atlas application-properties are present for,
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> 5. Start Atlas



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18361) All classes recompiled due to Maven bug, even if none changed

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485543#comment-15485543
 ] 

Hadoop QA commented on AMBARI-18361:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828058/AMBARI-18361.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
ambari-server.

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8639//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8639//console

This message is automatically generated.

> All classes recompiled due to Maven bug, even if none changed
> -
>
> Key: AMBARI-18361
> URL: https://issues.apache.org/jira/browse/AMBARI-18361
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: AMBARI-18361.patch
>
>
> maven-compiler-plugin version 3.0 has a 
> [bug|https://issues.apache.org/jira/browse/MCOMPILER-187] that causes all 
> classes to be recompiled even if no classes have changed.
> {noformat}
> [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ 
> ambari-server ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 1720 source files ...
> ...
> [INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @ 
> ambari-server ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 760 source files ...
> {noformat}
> version 3.1+ has [another, related 
> bug|https://issues.apache.org/jira/browse/MCOMPILER-209] that causes all 
> classes to be compiled even if only one has changed, although it seems to 
> correctly detect the "no change" case.
> {noformat}
> [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ 
> ambari-server ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 1720 source files ...
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18334) Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Gnanamalar Jebaraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anita Gnanamalar Jebaraj updated AMBARI-18334:
--
Status: Patch Available  (was: Open)

> Password in the configurations.json file in the ambari-agent cache is not 
> encrypted
> ---
>
> Key: AMBARI-18334
> URL: https://issues.apache.org/jira/browse/AMBARI-18334
> Project: Ambari
>  Issue Type: Bug
>Reporter: Anita Gnanamalar Jebaraj
>Assignee: Anita Gnanamalar Jebaraj
> Attachments: AMBARI-18334-Sep12.patch, AMBARI-18334.patch
>
>
> The  configurations.json  file loaded in the ambari-agent cache located at  
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18334) Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Gnanamalar Jebaraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anita Gnanamalar Jebaraj updated AMBARI-18334:
--
Attachment: AMBARI-18334-Sep12.patch

> Password in the configurations.json file in the ambari-agent cache is not 
> encrypted
> ---
>
> Key: AMBARI-18334
> URL: https://issues.apache.org/jira/browse/AMBARI-18334
> Project: Ambari
>  Issue Type: Bug
>Reporter: Anita Gnanamalar Jebaraj
>Assignee: Anita Gnanamalar Jebaraj
> Attachments: AMBARI-18334-Sep12.patch, AMBARI-18334.patch
>
>
> The  configurations.json  file loaded in the ambari-agent cache located at  
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18334) Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Gnanamalar Jebaraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anita Gnanamalar Jebaraj updated AMBARI-18334:
--
Attachment: (was: AMBARI-18334-Sep12.patch)

> Password in the configurations.json file in the ambari-agent cache is not 
> encrypted
> ---
>
> Key: AMBARI-18334
> URL: https://issues.apache.org/jira/browse/AMBARI-18334
> Project: Ambari
>  Issue Type: Bug
>Reporter: Anita Gnanamalar Jebaraj
>Assignee: Anita Gnanamalar Jebaraj
> Attachments: AMBARI-18334.patch
>
>
> The  configurations.json  file loaded in the ambari-agent cache located at  
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18332) Blueprints: API should make available "setting" property from blueprint

[Nahappan Somasundaram (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nahappan Somasundaram updated AMBARI-18332:
---
Description: 
The following APIs do not retrieve *setting* section from the blueprint that 
was used during deployment:

{code}
http://:/api/v1/blueprints/ 
{code}

  was:
The following APIs do not retrieve *setting* section from the blueprint that 
was used during deployment:

{code}
http://:/api/v1/blueprints/ 
http://:/api/v1/clusters/?format=blueprint 
{code}


> Blueprints: API should make available "setting" property from blueprint
> ---
>
> Key: AMBARI-18332
> URL: https://issues.apache.org/jira/browse/AMBARI-18332
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Nahappan Somasundaram
>Assignee: Nahappan Somasundaram
> Fix For: 2.5.0
>
>
> The following APIs do not retrieve *setting* section from the blueprint that 
> was used during deployment:
> {code}
> http://:/api/v1/blueprints/ 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-17457) Modify the AMS stack scripts to support distributed collector

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485490#comment-15485490
 ] 

Hadoop QA commented on AMBARI-17457:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828062/AMBARI-17457.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
ambari-server.

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8638//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8638//console

This message is automatically generated.

> Modify the AMS stack scripts to support distributed collector
> -
>
> Key: AMBARI-17457
> URL: https://issues.apache.org/jira/browse/AMBARI-17457
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Siddharth Wagle
>Assignee: Dmytro Sen
>Priority: Critical
> Fix For: trunk
>
> Attachments: AMBARI-17457.patch
>
>
> _Tasks_:
> - If collector mode = distributed:
> -- Do not start zookeeper and use the zk quorum property to find cluster zk.
> -- Start Master and RS as before
> -- Add metrics_collectors property to ams-site
> -- Add the same to the metrics_monitor.ini file template
> - Make changes to the stack advisor:
> -- Validate ZK quorum property
> -- Set metrics_collectors property in ams-site with comma separated collector 
> hostnames
> -- Validate hbase.rootdir is pointing to HDFS path



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18334) Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Gnanamalar Jebaraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anita Gnanamalar Jebaraj updated AMBARI-18334:
--
Status: Open  (was: Patch Available)

> Password in the configurations.json file in the ambari-agent cache is not 
> encrypted
> ---
>
> Key: AMBARI-18334
> URL: https://issues.apache.org/jira/browse/AMBARI-18334
> Project: Ambari
>  Issue Type: Bug
>Reporter: Anita Gnanamalar Jebaraj
>Assignee: Anita Gnanamalar Jebaraj
> Attachments: AMBARI-18334-Sep12.patch, AMBARI-18334.patch
>
>
> The  configurations.json  file loaded in the ambari-agent cache located at  
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18362) Update sinks to read multiple collector hostnames from configs

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485445#comment-15485445
 ] 

Hadoop QA commented on AMBARI-18362:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828065/AMBARI-18362.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
ambari-logsearch/ambari-logsearch-logfeeder 
ambari-logsearch/ambari-logsearch-portal ambari-metrics/ambari-metrics-common 
ambari-metrics/ambari-metrics-flume-sink 
ambari-metrics/ambari-metrics-hadoop-sink 
ambari-metrics/ambari-metrics-kafka-sink 
ambari-metrics/ambari-metrics-storm-sink 
ambari-metrics/ambari-metrics-storm-sink-legacy ambari-server.

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8637//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8637//console

This message is automatically generated.

> Update sinks to read multiple collector hostnames from configs
> --
>
> Key: AMBARI-18362
> URL: https://issues.apache.org/jira/browse/AMBARI-18362
> Project: Ambari
>  Issue Type: Task
>Reporter: Dmytro Sen
>Assignee: Dmytro Sen
>Priority: Critical
> Fix For: trunk
>
> Attachments: AMBARI-18362.patch
>
>
> Update sinks to read multiple collector hostnames from configs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18368) Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service

[Alejandro Fernandez (JIRA)]

Alejandro Fernandez created AMBARI-18368:


 Summary: Atlas web UI alert after performing stack upgrade to HDP 
2.5 and adding Atlas Service
 Key: AMBARI-18368
 URL: https://issues.apache.org/jira/browse/AMBARI-18368
 Project: Ambari
  Issue Type: Bug
  Components: stacks
Affects Versions: 2.4.0
Reporter: Alejandro Fernandez
Assignee: Alejandro Fernandez
Priority: Critical
 Fix For: trunk


Steps to Reproduce:
* Install Ambari 2.2.2 with HDP 2.4 (HBase, Solr)
* Kerberize the cluster
* Perform EU/RU to HDP 2.5
* Add Atlas Service

Atlas Server log contains,
{code}
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at 
http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not find 
the specified config set: vertex_index
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:577)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at 
com.thinkaurelius.titan.diskstorage.solr.Solr5Index.createCollectionIfNotExists(Solr5Index.java:901)
at 
com.thinkaurelius.titan.diskstorage.solr.Solr5Index.register(Solr5Index.java:269)
at 
com.thinkaurelius.titan.diskstorage.indexing.IndexTransaction.register(IndexTransaction.java:83)
at 
com.thinkaurelius.titan.graphdb.database.IndexSerializer.register(IndexSerializer.java:92)
at 
com.thinkaurelius.titan.graphdb.database.management.ManagementSystem.addIndexKey(ManagementSystem.java:534)
at 
org.apache.atlas.repository.graph.GraphBackedSearchIndexer.enhanceMixedIndex(GraphBackedSearchIndexer.java:405)
at 
org.apache.atlas.repository.graph.GraphBackedSearchIndexer.createIndexes(GraphBackedSearchIndexer.java:334)
at 
org.apache.atlas.repository.graph.GraphBackedSearchIndexer.initialize(GraphBackedSearchIndexer.java:103)
... 71 more
{code}

Atlas tables in HBase look ok.
{code}
su hbase
kinit -kt /etc/security/keytabs/hbase.headless.keytab cstm-hb...@example.com
hbase shell

hbase(main):001:0> list
TABLE
ATLAS_ENTITY_AUDIT_EVENTS
atlas_titan
2 row(s) in 1.4300 seconds

=> ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan"]
{code}

h4. Workaround
1. Stop Atlas Server
2. Copy solr xml files to correct config folder and chown as 
$atlas_user:$hadoop_group
{code}
cp -R /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/solr/* /etc/atlas/conf/solr/
cp: overwrite `/etc/atlas/conf/solr/solrconfig.xml'? n
chown atlas:hadoop /etc/atlas/conf/solr/*

cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/users-credentials.properties 
/etc/atlas/conf/
cp /usr/hdp/2.5.0.0-/etc/atlas/conf.dist/policy-store.txt /etc/atlas/conf/

chown atlas:hadoop /etc/atlas/conf/users-credentials.properties
chown atlas:hadoop /etc/atlas/conf/policy-store.txt
{code}
3. Delete zookeeper znode,
{code}
# kinit -kt /etc/security/keytabs/atlas.service.keytab  atlas/@
# cd /usr/hdp/current/zookeeper-client/bin/ 
# ./zkCli.sh -server :
[ .. (CONNECTED) ] rmr  /infra-solr/configs/atlas_configs
{code}

4. Ensure Atlas application-properties are present for,
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com

5. Start Atlas



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18367) Create authentication filter to encapsulate the various Ambari authentication methods

[Robert Levas (JIRA)]

Robert Levas created AMBARI-18367:
-

 Summary: Create authentication filter to encapsulate the various 
Ambari authentication methods
 Key: AMBARI-18367
 URL: https://issues.apache.org/jira/browse/AMBARI-18367
 Project: Ambari
  Issue Type: Task
  Components: ambari-server
Affects Versions: 2.5.0
Reporter: Robert Levas
Assignee: Robert Levas
 Fix For: 2.5.0


Create a Spring authentication filter to encapsulate the various Ambari 
authentication methods since the Spring filter chain allows for a single 
authentication filter and Ambari needs to allow for multiple, optional, 
authentication filters to handle one of (but not limited to) the following 
authentication methods:

* Basic Auth 
* SSO (JWT)
* Kerberos token





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15538) Support service-specific repo for add-on services

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Balázs Bence Sári updated AMBARI-15538:
---
Attachment: AMBARI-15538-custom-repos-patch3-trunk.diff

Latest patch.

> Support service-specific repo for add-on services
> -
>
> Key: AMBARI-15538
> URL: https://issues.apache.org/jira/browse/AMBARI-15538
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.0, 2.2.0, 2.4.0
>Reporter: Jayush Luniya
>Assignee: Balázs Bence Sári
> Fix For: 2.5.0, 2.4.2
>
> Attachments: AMBARI-15538-custom-repos-patch3-trunk.diff
>
>
> The approach for custom-services to specify their own repo location will be 
> to provide a {{/repos/repoinfo.xml}} inside the stack-version they will be 
> in. This repo file will be loaded by Ambari during startup into the 
> {{/api/v1/stacks/HDP/versions/2.4/repository_versions}} repos. *Service repo 
> files have a restriction that their (repo-name, base-url) locations should be 
> unique and not conflict*. When conflicts do occur, they will not be loaded 
> into the stacks model.
> Now the management-pack will provide such repos/ folder in 
> {{mpacks/custom-services/8.0.0/repos}} which will be linked into the stacks/ 
> folder.
> {{ambari/ambari-server/src/main/resources/stacks/HDP/2.3/services/SERVICE_NAME/repos
>  -> mpacks/custom-services/8.0.0/repos}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15538) Support service-specific repo for add-on services

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Balázs Bence Sári updated AMBARI-15538:
---
Status: Patch Available  (was: Open)

> Support service-specific repo for add-on services
> -
>
> Key: AMBARI-15538
> URL: https://issues.apache.org/jira/browse/AMBARI-15538
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.0, 2.2.0, 2.4.0
>Reporter: Jayush Luniya
>Assignee: Balázs Bence Sári
> Fix For: 2.5.0, 2.4.2
>
> Attachments: AMBARI-15538-custom-repos-patch3-trunk.diff
>
>
> The approach for custom-services to specify their own repo location will be 
> to provide a {{/repos/repoinfo.xml}} inside the stack-version they will be 
> in. This repo file will be loaded by Ambari during startup into the 
> {{/api/v1/stacks/HDP/versions/2.4/repository_versions}} repos. *Service repo 
> files have a restriction that their (repo-name, base-url) locations should be 
> unique and not conflict*. When conflicts do occur, they will not be loaded 
> into the stacks model.
> Now the management-pack will provide such repos/ folder in 
> {{mpacks/custom-services/8.0.0/repos}} which will be linked into the stacks/ 
> folder.
> {{ambari/ambari-server/src/main/resources/stacks/HDP/2.3/services/SERVICE_NAME/repos
>  -> mpacks/custom-services/8.0.0/repos}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15538) Support service-specific repo for add-on services

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Balázs Bence Sári updated AMBARI-15538:
---
Status: Open  (was: Patch Available)

> Support service-specific repo for add-on services
> -
>
> Key: AMBARI-15538
> URL: https://issues.apache.org/jira/browse/AMBARI-15538
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.0, 2.2.0, 2.4.0
>Reporter: Jayush Luniya
>Assignee: Balázs Bence Sári
> Fix For: 2.5.0, 2.4.2
>
>
> The approach for custom-services to specify their own repo location will be 
> to provide a {{/repos/repoinfo.xml}} inside the stack-version they will be 
> in. This repo file will be loaded by Ambari during startup into the 
> {{/api/v1/stacks/HDP/versions/2.4/repository_versions}} repos. *Service repo 
> files have a restriction that their (repo-name, base-url) locations should be 
> unique and not conflict*. When conflicts do occur, they will not be loaded 
> into the stacks model.
> Now the management-pack will provide such repos/ folder in 
> {{mpacks/custom-services/8.0.0/repos}} which will be linked into the stacks/ 
> folder.
> {{ambari/ambari-server/src/main/resources/stacks/HDP/2.3/services/SERVICE_NAME/repos
>  -> mpacks/custom-services/8.0.0/repos}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-15538) Support service-specific repo for add-on services

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Balázs Bence Sári updated AMBARI-15538:
---
Attachment: (was: AMBARI-15538-trunk-v1.patch)

> Support service-specific repo for add-on services
> -
>
> Key: AMBARI-15538
> URL: https://issues.apache.org/jira/browse/AMBARI-15538
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.0, 2.2.0, 2.4.0
>Reporter: Jayush Luniya
>Assignee: Balázs Bence Sári
> Fix For: 2.5.0, 2.4.2
>
>
> The approach for custom-services to specify their own repo location will be 
> to provide a {{/repos/repoinfo.xml}} inside the stack-version they will be 
> in. This repo file will be loaded by Ambari during startup into the 
> {{/api/v1/stacks/HDP/versions/2.4/repository_versions}} repos. *Service repo 
> files have a restriction that their (repo-name, base-url) locations should be 
> unique and not conflict*. When conflicts do occur, they will not be loaded 
> into the stacks model.
> Now the management-pack will provide such repos/ folder in 
> {{mpacks/custom-services/8.0.0/repos}} which will be linked into the stacks/ 
> folder.
> {{ambari/ambari-server/src/main/resources/stacks/HDP/2.3/services/SERVICE_NAME/repos
>  -> mpacks/custom-services/8.0.0/repos}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18237) Certain configuration files cannot be modified through Ambari api.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485220#comment-15485220
 ] 

Hudson commented on AMBARI-18237:
-

SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #5654 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/5654/])
Revert "AMBARI-18237. Certain configuration files cannot be modified (aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=181de178b270fd111f395b951ded7ae670a7d9ce])
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py


> Certain configuration files cannot be modified through Ambari api.
> --
>
> Key: AMBARI-18237
> URL: https://issues.apache.org/jira/browse/AMBARI-18237
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18237.patch
>
>
> Certain configuration files like hadoop-metrics2.properties are not exposed to
> Ambari api calls, therefore cannot be modified by Ambari REST api calls.
> Ambari QE team uses these configuration files to modify it for HDP stack
> tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18351) Preserve output while executing HSI/LLAP's "run.sh" command for launching LLAP.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485221#comment-15485221
 ] 

Hudson commented on AMBARI-18351:
-

SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #5654 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/5654/])
AMBARI-18351. Preserve output while executing HSI/LLAP's run.sh command 
(sshridhar: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=b44078cc30ac3c5d4fa86efbcaa9e5d68a37bc04])
* (edit) 
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
* (edit) ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py


> Preserve output while executing HSI/LLAP's "run.sh" command for launching 
> LLAP.
> ---
>
> Key: AMBARI-18351
> URL: https://issues.apache.org/jira/browse/AMBARI-18351
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Swapan Shridhar
>Assignee: Swapan Shridhar
>Priority: Critical
> Fix For: 2.4.2
>
> Attachments: AMBARI-18351.patch
>
>
> Ambari currently runs run.sh for LLAP as such:
> {noformat}
> 2016-08-29 20:34:47,770 - 
> Execute['/var/lib/ambari-agent/tmp/llap-slider2016-08-29_20-34-01/run.sh'] 
> {'user': 'hive'}
> 2016-08-29 20:35:51,856 - Submitted LLAP app name : llap0
> {noformat}
> Above is from Hive Interactive start operation output.
> Fix:
> - It would be good to include the output (stdout and stderr) of this command 
> in the operation logs, for debugging purposes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18365) Add Ambari configuration options to support Kerberos token authentication

[Robert Levas (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Levas updated AMBARI-18365:
--
Status: Patch Available  (was: Open)

> Add Ambari configuration options to support Kerberos token authentication
> -
>
> Key: AMBARI-18365
> URL: https://issues.apache.org/jira/browse/AMBARI-18365
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Robert Levas
>Assignee: Robert Levas
>  Labels: authentication, kerberos, security
> Fix For: 2.5.0
>
> Attachments: AMBARI-18365_branch-2.5_01.patch, 
> AMBARI-18365_trunk_01.patch
>
>
> Add the followng Ambari configuration options to support Kerberos token 
> authentication
> * {{authentication.kerberos.enabled}}
> ** Determines whether to use Kerberos (SPNEGO) authentication when connecting 
> Ambari:  {{true}} to enable this feature; {{false}}, otherwise
> * {{authentication.kerberos.spnego.principal}}
> ** The Kerberos principal name to use when verifying user-supplied Kerberos 
> tokens for authentication via SPNEGO
> * {{authentication.kerberos.spnego.keytab.file}}
> ** The Kerberos keytab file to use when verifying user-supplied Kerberos 
> tokens for authentication via SPNEGO
> * {{authentication.kerberos.user.types}}
> ** A comma-delimited (ordered) list of preferred user types to use when 
> finding the Ambari user account for the user-supplied Kerberos identity 
> during authentication via SPNEGO
> * {{authentication.kerberos.auth_to_local.rules}}
> ** The auth-to-local rules set to use when translating a user's principal 
> name to a local user name during authentication via SPNEGO.
> NOTE: These properties are in the {{ambari.properties}} file since this 
> feature may be enabled whether the rest of the cluster has Kerberos enabled 
> or not. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18365) Add Ambari configuration options to support Kerberos token authentication

[Robert Levas (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Levas updated AMBARI-18365:
--
Attachment: AMBARI-18365_trunk_01.patch
AMBARI-18365_branch-2.5_01.patch

> Add Ambari configuration options to support Kerberos token authentication
> -
>
> Key: AMBARI-18365
> URL: https://issues.apache.org/jira/browse/AMBARI-18365
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Robert Levas
>Assignee: Robert Levas
>  Labels: authentication, kerberos, security
> Fix For: 2.5.0
>
> Attachments: AMBARI-18365_branch-2.5_01.patch, 
> AMBARI-18365_trunk_01.patch
>
>
> Add the followng Ambari configuration options to support Kerberos token 
> authentication
> * {{authentication.kerberos.enabled}}
> ** Determines whether to use Kerberos (SPNEGO) authentication when connecting 
> Ambari:  {{true}} to enable this feature; {{false}}, otherwise
> * {{authentication.kerberos.spnego.principal}}
> ** The Kerberos principal name to use when verifying user-supplied Kerberos 
> tokens for authentication via SPNEGO
> * {{authentication.kerberos.spnego.keytab.file}}
> ** The Kerberos keytab file to use when verifying user-supplied Kerberos 
> tokens for authentication via SPNEGO
> * {{authentication.kerberos.user.types}}
> ** A comma-delimited (ordered) list of preferred user types to use when 
> finding the Ambari user account for the user-supplied Kerberos identity 
> during authentication via SPNEGO
> * {{authentication.kerberos.auth_to_local.rules}}
> ** The auth-to-local rules set to use when translating a user's principal 
> name to a local user name during authentication via SPNEGO.
> NOTE: These properties are in the {{ambari.properties}} file since this 
> feature may be enabled whether the rest of the cluster has Kerberos enabled 
> or not. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (AMBARI-18364) Ambari authentication with Kerberos token

[Robert Levas (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Levas reassigned AMBARI-18364:
-

Assignee: Robert Levas

> Ambari authentication with Kerberos token
> -
>
> Key: AMBARI-18364
> URL: https://issues.apache.org/jira/browse/AMBARI-18364
> Project: Ambari
>  Issue Type: Epic
>  Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Robert Levas
>Assignee: Robert Levas
>  Labels: authentication, kerberos, security
> Fix For: 2.5.0
>
>
> Users should be able to authenticate to use Ambari by providing a Kerberos 
> token using SPNEGO - Simple and Protected GSSAPI Negotiation Mechanism.  This 
> includes access to the Ambari REST API as well as the Ambari web-based UI. 
> The implementation should support the ability to perform the full SPNEGO 
> handshake as well as access requests directly providing the appropriate HTTP 
> header containing the Kerberos token. For example:
> {noformat}
> Authorization: Negotiate YIICcgY...r/vJcLO
> {noformat}
> In the full handshake model
> # The client requests access to a web resource
> # The server responds with an HTTP 401 status ({{Unauthorized}}), including 
> the header {{WWW-Authenticate: Negotiate}}
> # The client generates the Kerberos data and creates a new request containing 
> the authentication header - {{Authorization: Negotiate YIICcgY...r/vJcLO}}
> Since Ambari needs to generally return a HTTP status of 403 ({{Forbidden}}) 
> when authentication is needed, a _hint_ must be sent along with the request 
> indicate to Ambari that Kerberos authentication is desired.  If this _hint_ 
> is received, then Ambari will respond with the appropriate status and header 
> to initiate SPNEGO with the client. This _hint_ is an Ambari-specific header 
> named "X-Authentication-Type" with the value of "kerberos":
> {noformat}
> X-Authentication-Type: kerberos
> {noformat}
> No matter what the handshake mechanism is (or lack of), once the Kerberos 
> token is received by Ambari, Ambari is to parse and validate the token.  If a 
> failure occurs, Ambari is to respond with the appropriate HTTP status and 
> related header(s).  Upon success, the user's principal name is retrieved and 
> converted into a _local_ user name.  The use of an auth-to-local rule set 
> processor may be needed to perform this translation.  Using this _local_ 
> username, an appropriate Ambari user account is located and used as the 
> authenticated users identity - details, privileges, etc Failure to find 
> an appropriate Ambari user account is to result in an authentication failure 
> response.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18334) Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Gnanamalar Jebaraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anita Gnanamalar Jebaraj updated AMBARI-18334:
--
Attachment: AMBARI-18334-Sep12.patch

> Password in the configurations.json file in the ambari-agent cache is not 
> encrypted
> ---
>
> Key: AMBARI-18334
> URL: https://issues.apache.org/jira/browse/AMBARI-18334
> Project: Ambari
>  Issue Type: Bug
>Reporter: Anita Gnanamalar Jebaraj
>Assignee: Anita Gnanamalar Jebaraj
> Attachments: AMBARI-18334-Sep12.patch, AMBARI-18334.patch
>
>
> The  configurations.json  file loaded in the ambari-agent cache located at  
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18355) Introduce conditional dependencies in stack defition to handle blueprint validation gracefully

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485151#comment-15485151
 ] 

Hadoop QA commented on AMBARI-18355:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828076/AMBARI-18355_v1.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
ambari-server.

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8636//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8636//console

This message is automatically generated.

> Introduce conditional dependencies in stack defition to handle blueprint 
> validation gracefully
> --
>
> Key: AMBARI-18355
> URL: https://issues.apache.org/jira/browse/AMBARI-18355
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server
>Affects Versions: trunk
>Reporter: Amruta Borkar
>Assignee: Amruta Borkar
> Fix For: trunk
>
> Attachments: AMBARI-18355.patch, AMBARI-18355_v1.patch, Adding 
> Conditional Dependencies.pdf
>
>
> Currently stack definitions do not list conditional dependencies, adding 
> those to the stack definitions would make it easy to validate errors in case 
> of blueprint deployment. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18334) Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Gnanamalar Jebaraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anita Gnanamalar Jebaraj updated AMBARI-18334:
--
Attachment: (was: AMBARI-18334-Sep12.patch)

> Password in the configurations.json file in the ambari-agent cache is not 
> encrypted
> ---
>
> Key: AMBARI-18334
> URL: https://issues.apache.org/jira/browse/AMBARI-18334
> Project: Ambari
>  Issue Type: Bug
>Reporter: Anita Gnanamalar Jebaraj
>Assignee: Anita Gnanamalar Jebaraj
> Attachments: AMBARI-18334-Sep12.patch, AMBARI-18334.patch
>
>
> The  configurations.json  file loaded in the ambari-agent cache located at  
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18334) Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Gnanamalar Jebaraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anita Gnanamalar Jebaraj updated AMBARI-18334:
--
Status: Patch Available  (was: Open)

> Password in the configurations.json file in the ambari-agent cache is not 
> encrypted
> ---
>
> Key: AMBARI-18334
> URL: https://issues.apache.org/jira/browse/AMBARI-18334
> Project: Ambari
>  Issue Type: Bug
>Reporter: Anita Gnanamalar Jebaraj
>Assignee: Anita Gnanamalar Jebaraj
> Attachments: AMBARI-18334-Sep12.patch, AMBARI-18334.patch
>
>
> The  configurations.json  file loaded in the ambari-agent cache located at  
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18334) Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Gnanamalar Jebaraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anita Gnanamalar Jebaraj updated AMBARI-18334:
--
Status: Open  (was: Patch Available)

> Password in the configurations.json file in the ambari-agent cache is not 
> encrypted
> ---
>
> Key: AMBARI-18334
> URL: https://issues.apache.org/jira/browse/AMBARI-18334
> Project: Ambari
>  Issue Type: Bug
>Reporter: Anita Gnanamalar Jebaraj
>Assignee: Anita Gnanamalar Jebaraj
> Attachments: AMBARI-18334-Sep12.patch, AMBARI-18334.patch
>
>
> The  configurations.json  file loaded in the ambari-agent cache located at  
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18334) Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Gnanamalar Jebaraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anita Gnanamalar Jebaraj updated AMBARI-18334:
--
Attachment: AMBARI-18334-Sep12.patch

> Password in the configurations.json file in the ambari-agent cache is not 
> encrypted
> ---
>
> Key: AMBARI-18334
> URL: https://issues.apache.org/jira/browse/AMBARI-18334
> Project: Ambari
>  Issue Type: Bug
>Reporter: Anita Gnanamalar Jebaraj
>Assignee: Anita Gnanamalar Jebaraj
> Attachments: AMBARI-18334-Sep12.patch, AMBARI-18334.patch
>
>
> The  configurations.json  file loaded in the ambari-agent cache located at  
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18237) Certain configuration files cannot be modified through Ambari api.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485122#comment-15485122
 ] 

Hudson commented on AMBARI-18237:
-

SUCCESS: Integrated in Jenkins build Ambari-branch-2.5 #18 (See 
[https://builds.apache.org/job/Ambari-branch-2.5/18/])
Revert "AMBARI-18237. Certain configuration files cannot be modified (aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=44e64e5f675057da08a7e0d3a1fddd229a4707f5])
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py


> Certain configuration files cannot be modified through Ambari api.
> --
>
> Key: AMBARI-18237
> URL: https://issues.apache.org/jira/browse/AMBARI-18237
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18237.patch
>
>
> Certain configuration files like hadoop-metrics2.properties are not exposed to
> Ambari api calls, therefore cannot be modified by Ambari REST api calls.
> Ambari QE team uses these configuration files to modify it for HDP stack
> tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18366) YAML Maps For Storm Are Not Being Escaped Correctly

[Jonathan Hurley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hurley updated AMBARI-18366:
-
Attachment: AMBARI-18366.patch

> YAML Maps For Storm Are Not Being Escaped Correctly
> ---
>
> Key: AMBARI-18366
> URL: https://issues.apache.org/jira/browse/AMBARI-18366
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-agent
>Affects Versions: 2.0.0
>Reporter: Jonathan Hurley
>Assignee: Jonathan Hurley
>Priority: Critical
> Fix For: 2.5.0
>
> Attachments: AMBARI-18366.patch
>
>
> When attempting to add a custom {{storm-site}} property which is a YAML map 
> of maps, the {{storm.yaml}} file being generated is incorrectly escaping the 
> map. 
> {code:title="Expected YAML"}
> nimbus.impersonation.acl:
> storm:
>   hosts:
> [c6401.ambari.apache.org, c6402.ambari.apache.org]
>   groups:
> [hadoop, foo]
> {code}
> {code:title="Actual YAML"}
> nimbus.impersonation.acl: 'storm:
>   hosts:
> [c6401.ambari.apache.org, c6402.ambari.apache.org]
>   groups:
> [hadoop, foo]'
> {code}
> - We should not be escaping YAML maps
> - YAML maps must being on a newline with no leading whitespace



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18366) YAML Maps For Storm Are Not Being Escaped Correctly

[Jonathan Hurley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hurley updated AMBARI-18366:
-
Status: Patch Available  (was: Open)

> YAML Maps For Storm Are Not Being Escaped Correctly
> ---
>
> Key: AMBARI-18366
> URL: https://issues.apache.org/jira/browse/AMBARI-18366
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-agent
>Affects Versions: 2.0.0
>Reporter: Jonathan Hurley
>Assignee: Jonathan Hurley
>Priority: Critical
> Fix For: 2.5.0
>
> Attachments: AMBARI-18366.patch
>
>
> When attempting to add a custom {{storm-site}} property which is a YAML map 
> of maps, the {{storm.yaml}} file being generated is incorrectly escaping the 
> map. 
> {code:title="Expected YAML"}
> nimbus.impersonation.acl:
> storm:
>   hosts:
> [c6401.ambari.apache.org, c6402.ambari.apache.org]
>   groups:
> [hadoop, foo]
> {code}
> {code:title="Actual YAML"}
> nimbus.impersonation.acl: 'storm:
>   hosts:
> [c6401.ambari.apache.org, c6402.ambari.apache.org]
>   groups:
> [hadoop, foo]'
> {code}
> - We should not be escaping YAML maps
> - YAML maps must being on a newline with no leading whitespace



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18366) YAML Maps For Storm Are Not Being Escaped Correctly

[Jonathan Hurley (JIRA)]

Jonathan Hurley created AMBARI-18366:


 Summary: YAML Maps For Storm Are Not Being Escaped Correctly
 Key: AMBARI-18366
 URL: https://issues.apache.org/jira/browse/AMBARI-18366
 Project: Ambari
  Issue Type: Bug
  Components: ambari-agent
Affects Versions: 2.0.0
Reporter: Jonathan Hurley
Assignee: Jonathan Hurley
Priority: Critical
 Fix For: 2.5.0


When attempting to add a custom {{storm-site}} property which is a YAML map of 
maps, the {{storm.yaml}} file being generated is incorrectly escaping the map. 

{code:title="Expected YAML"}
nimbus.impersonation.acl:
storm:
  hosts:
[c6401.ambari.apache.org, c6402.ambari.apache.org]
  groups:
[hadoop, foo]
{code}

{code:title="Actual YAML"}
nimbus.impersonation.acl: 'storm:
  hosts:
[c6401.ambari.apache.org, c6402.ambari.apache.org]
  groups:
[hadoop, foo]'
{code}

- We should not be escaping YAML maps
- YAML maps must being on a newline with no leading whitespace



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-17728) Error message does not deliver when executing ambari-server command as a non-root user

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484945#comment-15484945
 ] 

Hudson commented on AMBARI-17728:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #5653 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/5653/])
AMBARI-17728: Error message does not deliver when executing (jluniya: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=e4cb41e0ab469788180f3ac5741d331706b46ea0])
* (edit) ambari-server/sbin/ambari-server


> Error message does not deliver when executing ambari-server command as a 
> non-root user
> --
>
> Key: AMBARI-17728
> URL: https://issues.apache.org/jira/browse/AMBARI-17728
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: trunk, 2.4.0
>Reporter: wangyaoxin
>Assignee: wangyaoxin
> Fix For: trunk
>
> Attachments: AMBARI-17728-1.patch, AMBARI-17728-2.patch, 
> AMBARI-17728.patch
>
>
> non-root user: like hdfs
> excute : ambari-server stop
> show :  Using python  /usr/bin/python2.6 Stopping ambari-server
> intent to: You can't perform this operation as non-sudoer user. Please, 
> re-login or configure sudo access for this user



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18365) Add Ambari configuration options to support Kerberos token authentication

[Robert Levas (JIRA)]

Robert Levas created AMBARI-18365:
-

 Summary: Add Ambari configuration options to support Kerberos 
token authentication
 Key: AMBARI-18365
 URL: https://issues.apache.org/jira/browse/AMBARI-18365
 Project: Ambari
  Issue Type: Task
  Components: ambari-server
Affects Versions: 2.5.0
Reporter: Robert Levas
Assignee: Robert Levas
 Fix For: 2.5.0


Add the followng Ambari configuration options to support Kerberos token 
authentication

* {{authentication.kerberos.enabled}}
** Determines whether to use Kerberos (SPNEGO) authentication when connecting 
Ambari:  {{true}} to enable this feature; {{false}}, otherwise
* {{authentication.kerberos.spnego.principal}}
** The Kerberos principal name to use when verifying user-supplied Kerberos 
tokens for authentication via SPNEGO
* {{authentication.kerberos.spnego.keytab.file}}
** The Kerberos keytab file to use when verifying user-supplied Kerberos tokens 
for authentication via SPNEGO
* {{authentication.kerberos.user.types}}
** A comma-delimited (ordered) list of preferred user types to use when finding 
the Ambari user account for the user-supplied Kerberos identity during 
authentication via SPNEGO
* {{authentication.kerberos.auth_to_local.rules}}
** The auth-to-local rules set to use when translating a user's principal name 
to a local user name during authentication via SPNEGO.

NOTE: These properties are in the {{ambari.properties}} file since this feature 
may be enabled whether the rest of the cluster has Kerberos enabled or not. 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18364) Ambari authentication with Kerberos token

[Robert Levas (JIRA)]

Robert Levas created AMBARI-18364:
-

 Summary: Ambari authentication with Kerberos token
 Key: AMBARI-18364
 URL: https://issues.apache.org/jira/browse/AMBARI-18364
 Project: Ambari
  Issue Type: Epic
  Components: ambari-server
Affects Versions: 2.5.0
Reporter: Robert Levas
 Fix For: 2.5.0


Users should be able to authenticate to use Ambari by providing a Kerberos 
token using SPNEGO - Simple and Protected GSSAPI Negotiation Mechanism.  This 
includes access to the Ambari REST API as well as the Ambari web-based UI. 

The implementation should support the ability to perform the full SPNEGO 
handshake as well as access requests directly providing the appropriate HTTP 
header containing the Kerberos token. For example:

{noformat}
Authorization: Negotiate YIICcgY...r/vJcLO
{noformat}

In the full handshake model
# The client requests access to a web resource
# The server responds with an HTTP 401 status ({{Unauthorized}}), including the 
header {{WWW-Authenticate: Negotiate}}
# The client generates the Kerberos data and creates a new request containing 
the authentication header - {{Authorization: Negotiate YIICcgY...r/vJcLO}}

Since Ambari needs to generally return a HTTP status of 403 ({{Forbidden}}) 
when authentication is needed, a _hint_ must be sent along with the request 
indicate to Ambari that Kerberos authentication is desired.  If this _hint_ is 
received, then Ambari will respond with the appropriate status and header to 
initiate SPNEGO with the client. This _hint_ is an Ambari-specific header named 
"X-Authentication-Type" with the value of "kerberos":

{noformat}
X-Authentication-Type: kerberos
{noformat}

No matter what the handshake mechanism is (or lack of), once the Kerberos token 
is received by Ambari, Ambari is to parse and validate the token.  If a failure 
occurs, Ambari is to respond with the appropriate HTTP status and related 
header(s).  Upon success, the user's principal name is retrieved and converted 
into a _local_ user name.  The use of an auth-to-local rule set processor may 
be needed to perform this translation.  Using this _local_ username, an 
appropriate Ambari user account is located and used as the authenticated users 
identity - details, privileges, etc Failure to find an appropriate Ambari 
user account is to result in an authentication failure response.








--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18363) Ambari authentication with Kerberos token

[Robert Levas (JIRA)]

Robert Levas created AMBARI-18363:
-

 Summary: Ambari authentication with Kerberos token
 Key: AMBARI-18363
 URL: https://issues.apache.org/jira/browse/AMBARI-18363
 Project: Ambari
  Issue Type: Epic
  Components: ambari-server
Affects Versions: 2.5.0
Reporter: Robert Levas
 Fix For: 2.5.0


Users should be able to authenticate to use Ambari by providing a Kerberos 
token using SPNEGO - Simple and Protected GSSAPI Negotiation Mechanism.  This 
includes access to the Ambari REST API as well as the Ambari web-based UI. 

The implementation should support the ability to perform the full SPNEGO 
handshake as well as access requests directly providing the appropriate HTTP 
header containing the Kerberos token. For example:

{noformat}
Authorization: Negotiate YIICcgY...r/vJcLO
{noformat}

In the full handshake model
# The client requests access to a web resource
# The server responds with an HTTP 401 status ({{Unauthorized}}), including the 
header {{WWW-Authenticate: Negotiate}}
# The client generates the Kerberos data and creates a new request containing 
the authentication header - {{Authorization: Negotiate YIICcgY...r/vJcLO}}

Since Ambari needs to generally return a HTTP status of 403 ({{Forbidden}}) 
when authentication is needed, a _hint_ must be sent along with the request 
indicate to Ambari that Kerberos authentication is desired.  If this _hint_ is 
received, then Ambari will respond with the appropriate status and header to 
initiate SPNEGO with the client. This _hint_ is an Ambari-specific header named 
"X-Authentication-Type" with the value of "kerberos":

{noformat}
X-Authentication-Type: kerberos
{noformat}

No matter what the handshake mechanism is (or lack of), once the Kerberos token 
is received by Ambari, Ambari is to parse and validate the token.  If a failure 
occurs, Ambari is to respond with the appropriate HTTP status and related 
header(s).  Upon success, the user's principal name is retrieved and converted 
into a _local_ user name.  The use of an auth-to-local rule set processor may 
be needed to perform this translation.  Using this _local_ username, an 
appropriate Ambari user account is located and used as the authenticated users 
identity - details, privileges, etc Failure to find an appropriate Ambari 
user account is to result in an authentication failure response.








--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18355) Introduce conditional dependencies in stack defition to handle blueprint validation gracefully

[Amruta Borkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amruta Borkar updated AMBARI-18355:
---
Attachment: AMBARI-18355_v1.patch

> Introduce conditional dependencies in stack defition to handle blueprint 
> validation gracefully
> --
>
> Key: AMBARI-18355
> URL: https://issues.apache.org/jira/browse/AMBARI-18355
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server
>Affects Versions: trunk
>Reporter: Amruta Borkar
>Assignee: Amruta Borkar
> Fix For: trunk
>
> Attachments: AMBARI-18355.patch, AMBARI-18355_v1.patch, Adding 
> Conditional Dependencies.pdf
>
>
> Currently stack definitions do not list conditional dependencies, adding 
> those to the stack definitions would make it easy to validate errors in case 
> of blueprint deployment. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18355) Introduce conditional dependencies in stack defition to handle blueprint validation gracefully

[Amruta Borkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amruta Borkar updated AMBARI-18355:
---
Status: Patch Available  (was: In Progress)

> Introduce conditional dependencies in stack defition to handle blueprint 
> validation gracefully
> --
>
> Key: AMBARI-18355
> URL: https://issues.apache.org/jira/browse/AMBARI-18355
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server
>Affects Versions: trunk
>Reporter: Amruta Borkar
>Assignee: Amruta Borkar
> Fix For: trunk
>
> Attachments: AMBARI-18355.patch, AMBARI-18355_v1.patch, Adding 
> Conditional Dependencies.pdf
>
>
> Currently stack definitions do not list conditional dependencies, adding 
> those to the stack definitions would make it easy to validate errors in case 
> of blueprint deployment. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18355) Introduce conditional dependencies in stack defition to handle blueprint validation gracefully

[Amruta Borkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amruta Borkar updated AMBARI-18355:
---
Status: Open  (was: Patch Available)

> Introduce conditional dependencies in stack defition to handle blueprint 
> validation gracefully
> --
>
> Key: AMBARI-18355
> URL: https://issues.apache.org/jira/browse/AMBARI-18355
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server
>Affects Versions: trunk
>Reporter: Amruta Borkar
>Assignee: Amruta Borkar
> Fix For: trunk
>
> Attachments: AMBARI-18355.patch, Adding Conditional Dependencies.pdf
>
>
> Currently stack definitions do not list conditional dependencies, adding 
> those to the stack definitions would make it easy to validate errors in case 
> of blueprint deployment. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18324) Externalize skip repo url check to ambari.properties instead of hardcoding it in Ambari Java code

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484657#comment-15484657
 ] 

Di Li commented on AMBARI-18324:


[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec 
(python-test) on project ambari-server: Command execution failed. Process 
exited with an error: 1 (Exit value: 1) -> [Help 1]

Build failed on kicking off Python unit tests. This failure has been there in 
previous builds as well...

> Externalize skip repo url check to ambari.properties instead of hardcoding it 
> in Ambari Java code
> -
>
> Key: AMBARI-18324
> URL: https://issues.apache.org/jira/browse/AMBARI-18324
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-trunk
>Affects Versions: trunk
>Reporter: Di Li
>Assignee: Di Li
> Fix For: trunk
>
> Attachments: AMBARI-18324.patch, AMBARI-18324_rebase.patch
>
>
> Externalize skip repo url check to ambari.properties instead of hardcoding it 
> in Ambari Java code, so that users can define what repo ids to skip repo urls 
> validation when adding repos for the given stack



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18324) Externalize skip repo url check to ambari.properties instead of hardcoding it in Ambari Java code

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484632#comment-15484632
 ] 

Hudson commented on AMBARI-18324:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #5652 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/5652/])
AMBARI-18324: Externalize skip repo url check to ambari.properties (dili: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=623c36d2e0832d9a0bc60bc9ff19b302d5e3cefa])
* (edit) 
ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java


> Externalize skip repo url check to ambari.properties instead of hardcoding it 
> in Ambari Java code
> -
>
> Key: AMBARI-18324
> URL: https://issues.apache.org/jira/browse/AMBARI-18324
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-trunk
>Affects Versions: trunk
>Reporter: Di Li
>Assignee: Di Li
> Fix For: trunk
>
> Attachments: AMBARI-18324.patch, AMBARI-18324_rebase.patch
>
>
> Externalize skip repo url check to ambari.properties instead of hardcoding it 
> in Ambari Java code, so that users can define what repo ids to skip repo urls 
> validation when adding repos for the given stack



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-17728) Error message does not deliver when executing ambari-server command as a non-root user

[Jayush Luniya (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484609#comment-15484609
 ] 

Jayush Luniya commented on AMBARI-17728:


commit e4cb41e0ab469788180f3ac5741d331706b46ea0
Author: Jayush Luniya 
Date:   Mon Sep 12 09:50:49 2016 -0700

AMBARI-17728: Error message does not deliver when executing ambari-server 
command as a non-root use (wang yaoxin via jluniya)

> Error message does not deliver when executing ambari-server command as a 
> non-root user
> --
>
> Key: AMBARI-17728
> URL: https://issues.apache.org/jira/browse/AMBARI-17728
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: trunk, 2.4.0
>Reporter: wangyaoxin
>Assignee: wangyaoxin
> Fix For: trunk
>
> Attachments: AMBARI-17728-1.patch, AMBARI-17728-2.patch, 
> AMBARI-17728.patch
>
>
> non-root user: like hdfs
> excute : ambari-server stop
> show :  Using python  /usr/bin/python2.6 Stopping ambari-server
> intent to: You can't perform this operation as non-sudoer user. Please, 
> re-login or configure sudo access for this user



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-17728) Error message does not deliver when executing ambari-server command as a non-root user

[Jayush Luniya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jayush Luniya updated AMBARI-17728:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Error message does not deliver when executing ambari-server command as a 
> non-root user
> --
>
> Key: AMBARI-17728
> URL: https://issues.apache.org/jira/browse/AMBARI-17728
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: trunk, 2.4.0
>Reporter: wangyaoxin
>Assignee: wangyaoxin
> Fix For: trunk
>
> Attachments: AMBARI-17728-1.patch, AMBARI-17728-2.patch, 
> AMBARI-17728.patch
>
>
> non-root user: like hdfs
> excute : ambari-server stop
> show :  Using python  /usr/bin/python2.6 Stopping ambari-server
> intent to: You can't perform this operation as non-sudoer user. Please, 
> re-login or configure sudo access for this user



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18051) Services should be able to provide their own pre-req checks by supplying a jar file

[Tim Thorpe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Thorpe updated AMBARI-18051:

Description: 
Services should be able to provide their own pre-req checks by supplying a jar 
file.

This would allow custom services to supply their own jar files to handle 
pre-req checks rather than forcing third party developers to make changes to 
ambari-server code.

  was:
Extension should be able to provide their own pre-req checks by supplying a jar 
file.

This would allow custom services packaged as extensions to supply their own jar 
files to handle their pre-req checks rather than forcing third party developers 
to make changes to ambari-server code.


> Services should be able to provide their own pre-req checks by supplying a 
> jar file
> ---
>
> Key: AMBARI-18051
> URL: https://issues.apache.org/jira/browse/AMBARI-18051
> Project: Ambari
>  Issue Type: New Feature
>  Components: ambari-server
>Affects Versions: trunk
>Reporter: Tim Thorpe
>Assignee: Tim Thorpe
>
> Services should be able to provide their own pre-req checks by supplying a 
> jar file.
> This would allow custom services to supply their own jar files to handle 
> pre-req checks rather than forcing third party developers to make changes to 
> ambari-server code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18051) Services should be able to provide their own pre-req checks by supplying a jar file

[Tim Thorpe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Thorpe updated AMBARI-18051:

Summary: Services should be able to provide their own pre-req checks by 
supplying a jar file  (was: Extension should be able to provide their own 
pre-req checks by supplying a jar file)

> Services should be able to provide their own pre-req checks by supplying a 
> jar file
> ---
>
> Key: AMBARI-18051
> URL: https://issues.apache.org/jira/browse/AMBARI-18051
> Project: Ambari
>  Issue Type: New Feature
>  Components: ambari-server
>Affects Versions: trunk
>Reporter: Tim Thorpe
>Assignee: Tim Thorpe
>
> Extension should be able to provide their own pre-req checks by supplying a 
> jar file.
> This would allow custom services packaged as extensions to supply their own 
> jar files to handle their pre-req checks rather than forcing third party 
> developers to make changes to ambari-server code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18362) Update sinks to read multiple collector hostnames from configs

[Dmytro Sen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmytro Sen updated AMBARI-18362:

Status: Patch Available  (was: Open)

> Update sinks to read multiple collector hostnames from configs
> --
>
> Key: AMBARI-18362
> URL: https://issues.apache.org/jira/browse/AMBARI-18362
> Project: Ambari
>  Issue Type: Task
>Reporter: Dmytro Sen
>Assignee: Dmytro Sen
>Priority: Critical
> Fix For: trunk
>
> Attachments: AMBARI-18362.patch
>
>
> Update sinks to read multiple collector hostnames from configs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18359) Get LDAP tests to run in concurrent forked JVMs

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484519#comment-15484519
 ] 

Hadoop QA commented on AMBARI-18359:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828030/AMBARI-18359.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
ambari-server:

  
org.apache.ambari.server.controller.internal.ConfigGroupResourceProviderTest

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8635//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8635//console

This message is automatically generated.

> Get LDAP tests to run in concurrent forked JVMs
> ---
>
> Key: AMBARI-18359
> URL: https://issues.apache.org/jira/browse/AMBARI-18359
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
> Fix For: 2.5.0
>
> Attachments: AMBARI-18359.patch
>
>
> Some LDAP tests fail to run with forkCount:>1, reuseForks:false. This blocks 
> increasing forkCount to speed up the unit tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18362) Update sinks to read multiple collector hostnames from configs

[Dmytro Sen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmytro Sen updated AMBARI-18362:

Attachment: AMBARI-18362.patch

> Update sinks to read multiple collector hostnames from configs
> --
>
> Key: AMBARI-18362
> URL: https://issues.apache.org/jira/browse/AMBARI-18362
> Project: Ambari
>  Issue Type: Task
>Reporter: Dmytro Sen
>Assignee: Dmytro Sen
>Priority: Critical
> Fix For: trunk
>
> Attachments: AMBARI-18362.patch
>
>
> Update sinks to read multiple collector hostnames from configs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18362) Update sinks to read multiple collector hostnames from configs

[Dmytro Sen (JIRA)]

Dmytro Sen created AMBARI-18362:
---

 Summary: Update sinks to read multiple collector hostnames from 
configs
 Key: AMBARI-18362
 URL: https://issues.apache.org/jira/browse/AMBARI-18362
 Project: Ambari
  Issue Type: Task
Reporter: Dmytro Sen
Assignee: Dmytro Sen
Priority: Critical
 Fix For: trunk


Update sinks to read multiple collector hostnames from configs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-17457) Modify the AMS stack scripts to support distributed collector

[Dmytro Sen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmytro Sen updated AMBARI-17457:

Attachment: AMBARI-17457.patch

> Modify the AMS stack scripts to support distributed collector
> -
>
> Key: AMBARI-17457
> URL: https://issues.apache.org/jira/browse/AMBARI-17457
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Siddharth Wagle
>Assignee: Dmytro Sen
>Priority: Critical
> Fix For: trunk
>
> Attachments: AMBARI-17457.patch
>
>
> _Tasks_:
> - If collector mode = distributed:
> -- Do not start zookeeper and use the zk quorum property to find cluster zk.
> -- Start Master and RS as before
> -- Add metrics_collectors property to ams-site
> -- Add the same to the metrics_monitor.ini file template
> - Make changes to the stack advisor:
> -- Validate ZK quorum property
> -- Set metrics_collectors property in ams-site with comma separated collector 
> hostnames
> -- Validate hbase.rootdir is pointing to HDFS path



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-17457) Modify the AMS stack scripts to support distributed collector

[Dmytro Sen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmytro Sen updated AMBARI-17457:

Fix Version/s: (was: 2.5.0)
   trunk

> Modify the AMS stack scripts to support distributed collector
> -
>
> Key: AMBARI-17457
> URL: https://issues.apache.org/jira/browse/AMBARI-17457
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Siddharth Wagle
>Assignee: Dmytro Sen
>Priority: Critical
> Fix For: trunk
>
> Attachments: AMBARI-17457.patch
>
>
> _Tasks_:
> - If collector mode = distributed:
> -- Do not start zookeeper and use the zk quorum property to find cluster zk.
> -- Start Master and RS as before
> -- Add metrics_collectors property to ams-site
> -- Add the same to the metrics_monitor.ini file template
> - Make changes to the stack advisor:
> -- Validate ZK quorum property
> -- Set metrics_collectors property in ams-site with comma separated collector 
> hostnames
> -- Validate hbase.rootdir is pointing to HDFS path



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-17457) Modify the AMS stack scripts to support distributed collector

[Dmytro Sen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmytro Sen updated AMBARI-17457:

Status: Patch Available  (was: Open)

> Modify the AMS stack scripts to support distributed collector
> -
>
> Key: AMBARI-17457
> URL: https://issues.apache.org/jira/browse/AMBARI-17457
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Siddharth Wagle
>Assignee: Dmytro Sen
>Priority: Critical
> Fix For: 2.5.0
>
> Attachments: AMBARI-17457.patch
>
>
> _Tasks_:
> - If collector mode = distributed:
> -- Do not start zookeeper and use the zk quorum property to find cluster zk.
> -- Start Master and RS as before
> -- Add metrics_collectors property to ams-site
> -- Add the same to the metrics_monitor.ini file template
> - Make changes to the stack advisor:
> -- Validate ZK quorum property
> -- Set metrics_collectors property in ams-site with comma separated collector 
> hostnames
> -- Validate hbase.rootdir is pointing to HDFS path



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18361) All classes recompiled due to Maven bug, even if none changed

[Doroszlai, Attila (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doroszlai, Attila updated AMBARI-18361:
---
Attachment: AMBARI-18361.patch

> All classes recompiled due to Maven bug, even if none changed
> -
>
> Key: AMBARI-18361
> URL: https://issues.apache.org/jira/browse/AMBARI-18361
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: AMBARI-18361.patch
>
>
> maven-compiler-plugin version 3.0 has a 
> [bug|https://issues.apache.org/jira/browse/MCOMPILER-187] that causes all 
> classes to be recompiled even if no classes have changed.
> {noformat}
> [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ 
> ambari-server ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 1720 source files ...
> ...
> [INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @ 
> ambari-server ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 760 source files ...
> {noformat}
> version 3.1+ has [another, related 
> bug|https://issues.apache.org/jira/browse/MCOMPILER-209] that causes all 
> classes to be compiled even if only one has changed, although it seems to 
> correctly detect the "no change" case.
> {noformat}
> [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ 
> ambari-server ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 1720 source files ...
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18361) All classes recompiled due to Maven bug, even if none changed

[Doroszlai, Attila (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doroszlai, Attila updated AMBARI-18361:
---
Status: Patch Available  (was: Open)

> All classes recompiled due to Maven bug, even if none changed
> -
>
> Key: AMBARI-18361
> URL: https://issues.apache.org/jira/browse/AMBARI-18361
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: AMBARI-18361.patch
>
>
> maven-compiler-plugin version 3.0 has a 
> [bug|https://issues.apache.org/jira/browse/MCOMPILER-187] that causes all 
> classes to be recompiled even if no classes have changed.
> {noformat}
> [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ 
> ambari-server ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 1720 source files ...
> ...
> [INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @ 
> ambari-server ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 760 source files ...
> {noformat}
> version 3.1+ has [another, related 
> bug|https://issues.apache.org/jira/browse/MCOMPILER-209] that causes all 
> classes to be compiled even if only one has changed, although it seems to 
> correctly detect the "no change" case.
> {noformat}
> [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ 
> ambari-server ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 1720 source files ...
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18050) Upgrade pre-req check code needs to be decoupled from CheckDescription class

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484381#comment-15484381
 ] 

Hudson commented on AMBARI-18050:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #5651 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/5651/])
AMBARI-18050 - Upgrade pre-req check code needs to be decoupled from (tthorpe: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=2961c480e05ff179abe96eee4e699662ed820c5c])
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java


> Upgrade pre-req check code needs to be decoupled from CheckDescription class
> 
>
> Key: AMBARI-18050
> URL: https://issues.apache.org/jira/browse/AMBARI-18050
> Project: Ambari
>  Issue Type: New Feature
>  Components: ambari-server
>Affects Versions: trunk
>Reporter: Tim Thorpe
>Assignee: Tim Thorpe
> Fix For: trunk
>
> Attachments: AMBARI-18050.patch
>
>
> Currently each pre-req check must add an entry in the CheckDescription enum.  
> This limits the ability for third party stacks and extensions to provide 
> their own pre-req checks.  
> The CheckDescription enum should be rewritten as a class and each pre-req 
> check class should create an instance of it.  This will allow stacks and 
> extensions to include their own pre-req checks in separate jar files without 
> requiring changes to ambari-server java code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18260) Journal node restart failing on RU from HDP 2.4.x to 2.5 on Wire Encrypted cluster

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484380#comment-15484380
 ] 

Hudson commented on AMBARI-18260:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #5651 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/5651/])
AMBARI-18260. Journal node restart failing on RU from dergM10 to erie on 
(aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=2205f9cbb6bec08f95735310c743e17267c3abec])
* (edit) 
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py


> Journal node restart failing on RU from HDP 2.4.x to 2.5 on Wire Encrypted 
> cluster
> --
>
> Key: AMBARI-18260
> URL: https://issues.apache.org/jira/browse/AMBARI-18260
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: trunk
>
> Attachments: AMBARI-18260.patch
>
>
> Type of upgrade : RU  
> Upgrade from HDP (2.4.2.0) to 2.5 (on secure, Wire encrypted
> cluster)
> Journal node logs show :
> 
> 
> 
> org.apache.hadoop.hdfs.qjournal.protocol.JournalOutOfSyncException: Can't 
> write, no segment open
>   at 
> org.apache.hadoop.hdfs.qjournal.server.Journal.checkSync(Journal.java:484)
>   at 
> org.apache.hadoop.hdfs.qjournal.server.Journal.journal(Journal.java:353)
>   at 
> org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.journal(JournalNodeRpcServer.java:152)
>   at 
> org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.journal(QJournalProtocolServerSideTranslatorPB.java:158)
>   at 
> org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos$QJournalProtocolService$2.callBlockingMethod(QJournalProtocolProtos.java:25421)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> ç2016-08-25 07:19:47,651 INFO  namenode.FileJournalManager 
> (FileJournalManager.java:finalizeLogSegment(142)) - Finalizing edits file 
> /grid/0/hadoop/hdfs/journal/nameservice/current/edits_inprogress_0073697
>  -> 
> /grid/0/hadoop/hdfs/journal/nameservice/current/edits_0073697-0073698
> 
> Error at the exact RU task:
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
>  line 198, in 
> JournalNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 731, in restart
> self.post_upgrade_restart(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
>  line 75, in post_upgrade_restart
> journalnode_upgrade.post_upgrade_check()
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py",
>  line 64, in post_upgrade_check
> namenode_ha.is_encrypted(), params.security_enabled)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py",
>  line 306, in get_jmx_data
> data = urllib2.urlopen(nn_address).read()
>   File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen
> return _opener.open(url, data, timeout)
>   File "/usr/lib64/python2.6/urllib2.py", line 391, in open
> response = self._open(req, data)
>   File "/usr/lib64/python2.6/urllib2.py", line 409, in _open
> '_open', req)
>   File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
> result = func(*args)
>   File "/usr/lib64/python2.6/urllib2.py", line 1194, in https_open
> return self.do_open(httplib.HTTPSConnection, req)
>   File "/usr/lib64/python2.6/urllib2.py", line 1161, in do_open
> raise URLError(err)
> urllib2.URLError:  routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure>
> 
> Live cluster :  
> 
> Artifacts: 

[jira] [Commented] (AMBARI-18260) Journal node restart failing on RU from HDP 2.4.x to 2.5 on Wire Encrypted cluster

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484337#comment-15484337
 ] 

Hudson commented on AMBARI-18260:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #17 (See 
[https://builds.apache.org/job/Ambari-branch-2.5/17/])
AMBARI-18260. Journal node restart failing on RU from dergM10 to erie on 
(aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=534aad3f042f887b5873439ea262375402ed3ff0])
* (edit) 
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py


> Journal node restart failing on RU from HDP 2.4.x to 2.5 on Wire Encrypted 
> cluster
> --
>
> Key: AMBARI-18260
> URL: https://issues.apache.org/jira/browse/AMBARI-18260
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: trunk
>
> Attachments: AMBARI-18260.patch
>
>
> Type of upgrade : RU  
> Upgrade from HDP (2.4.2.0) to 2.5 (on secure, Wire encrypted
> cluster)
> Journal node logs show :
> 
> 
> 
> org.apache.hadoop.hdfs.qjournal.protocol.JournalOutOfSyncException: Can't 
> write, no segment open
>   at 
> org.apache.hadoop.hdfs.qjournal.server.Journal.checkSync(Journal.java:484)
>   at 
> org.apache.hadoop.hdfs.qjournal.server.Journal.journal(Journal.java:353)
>   at 
> org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.journal(JournalNodeRpcServer.java:152)
>   at 
> org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.journal(QJournalProtocolServerSideTranslatorPB.java:158)
>   at 
> org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos$QJournalProtocolService$2.callBlockingMethod(QJournalProtocolProtos.java:25421)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> ç2016-08-25 07:19:47,651 INFO  namenode.FileJournalManager 
> (FileJournalManager.java:finalizeLogSegment(142)) - Finalizing edits file 
> /grid/0/hadoop/hdfs/journal/nameservice/current/edits_inprogress_0073697
>  -> 
> /grid/0/hadoop/hdfs/journal/nameservice/current/edits_0073697-0073698
> 
> Error at the exact RU task:
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
>  line 198, in 
> JournalNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 731, in restart
> self.post_upgrade_restart(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
>  line 75, in post_upgrade_restart
> journalnode_upgrade.post_upgrade_check()
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py",
>  line 64, in post_upgrade_check
> namenode_ha.is_encrypted(), params.security_enabled)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py",
>  line 306, in get_jmx_data
> data = urllib2.urlopen(nn_address).read()
>   File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen
> return _opener.open(url, data, timeout)
>   File "/usr/lib64/python2.6/urllib2.py", line 391, in open
> response = self._open(req, data)
>   File "/usr/lib64/python2.6/urllib2.py", line 409, in _open
> '_open', req)
>   File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
> result = func(*args)
>   File "/usr/lib64/python2.6/urllib2.py", line 1194, in https_open
> return self.do_open(httplib.HTTPSConnection, req)
>   File "/usr/lib64/python2.6/urllib2.py", line 1161, in do_open
> raise URLError(err)
> urllib2.URLError:  routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure>
> 
> Live cluster :  
> 
> Artifacts: 

[jira] [Commented] (AMBARI-17228) Blueprint deployments should support a "START_ONLY" provision_action for clusters

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-17228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484336#comment-15484336
 ] 

Hudson commented on AMBARI-17228:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #17 (See 
[https://builds.apache.org/job/Ambari-branch-2.5/17/])
AMBARI-17228. Blueprint deployments should support a START_ONLY (smagyari: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=0a3c839432473a0663f6917e662099f6c8935035])
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionAction.java
* (delete) 
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithHostsSyspreppedTest.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
* (add) 
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithStartOnlyTest.java
* (edit) 
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java


> Blueprint deployments should support a "START_ONLY" provision_action for 
> clusters
> -
>
> Key: AMBARI-17228
> URL: https://issues.apache.org/jira/browse/AMBARI-17228
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Robert Nettleton
>Assignee: Sandor Magyari
>Priority: Critical
> Fix For: trunk
>
> Attachments: AMBARI-17228_v3.patch
>
>
> This JIRA tracks an extension to the work started in:
> AMBARI-16220
> In that JIRA, support is being added to allow Blueprint deployments that skip 
> the installation steps, and only attempt to start all components in the 
> cluster after the configuration phase has completed.
> In AMBARI-16220, the feature is configured via a property in 
> "ambari.properties", which configures this feature on the ambari-server 
> itself.
> This current JIRA tracks a suggestion to extend this feature, such that the 
> "START_ONLY" mode can be treated as a new type of "provision_action". 
> Using the property specified in ambari.properties isn't incorrect, but may be 
> inconvenient in the future for enhancements and maintenance.  
> In multi-cluster scenarios, it might be better to configure this at the 
> cluster-level.
> There has already been some work done to configure provisioning in a more 
> fine-grained way.
> Sid's patch for the following Blueprints feature:
> https://issues.apache.org/jira/browse/AMBARI-14283
> Shows how a "provision_action" can be selected for a given deployment.  
> This notion of a "provision_action" has also been extended to the component 
> level as well:
> https://issues.apache.org/jira/browse/AMBARI-14555
> Currently, this "provision_action" configuration is only used to select an 
> INSTALL_ONLY deployment, but  it could certainly be extended to have a 
> "START_ONLY" action as well.  This would have the benefit of being able to 
> choose this option on a per-cluster basis, and not require an ambari-server 
> restart if this feature is desired but not enabled by default. 
> The following code reference shows how the "provision_action" configuration 
> option is used by a Blueprints deployment:
> org.apache.ambari.server.topology.HostRequest#createTasks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18323) Cluster creation complete is missing from ambari-server log

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484335#comment-15484335
 ] 

Hudson commented on AMBARI-18323:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #17 (See 
[https://builds.apache.org/job/Ambari-branch-2.5/17/])
AMBARI-18323. Cluster creation complete is missing from ambari-server 
(smagyari: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=f50c229602001eda025cdfbdeecb6dde6248c46e])
* (edit) 
ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java
* (add) 
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartOnComponentLevelTest.java
* (edit) 
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java


> Cluster creation complete is missing from ambari-server log
> ---
>
> Key: AMBARI-18323
> URL: https://issues.apache.org/jira/browse/AMBARI-18323
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Sandor Magyari
>Assignee: Sandor Magyari
> Fix For: trunk, 2.5.0
>
> Attachments: AMBARI-18323_v2.patch
>
>
> In case provision_action is set to INSTALL_ONLY on a given hostgroup compoent 
> in Blueprint, cluster completion message is not logged. Normally during 
> Blueprint deployments Ambari creates LogicalTasks bounded later to INSTALL 
> tasks (HostRoleCommand), however in this case there are  no INSTALL task 
> bounded to LogicalTasks, so they will never complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18260) Journal node restart failing on RU from HDP 2.4.x to 2.5 on Wire Encrypted cluster

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-18260:
---
Summary: Journal node restart failing on RU from HDP 2.4.x to 2.5 on Wire 
Encrypted cluster  (was: Journal node restart failing on RU from dergM10 to 
erie on Wire Encrypted cluster)

> Journal node restart failing on RU from HDP 2.4.x to 2.5 on Wire Encrypted 
> cluster
> --
>
> Key: AMBARI-18260
> URL: https://issues.apache.org/jira/browse/AMBARI-18260
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: trunk
>
> Attachments: AMBARI-18260.patch
>
>
> Type of upgrade : RU  
> Upgrade from HDP Derg M10 (2.4.2.0) to Erie (on secure, Wire encrypted
> cluster)
> Journal node logs show :
> 
> 
> 
> org.apache.hadoop.hdfs.qjournal.protocol.JournalOutOfSyncException: Can't 
> write, no segment open
>   at 
> org.apache.hadoop.hdfs.qjournal.server.Journal.checkSync(Journal.java:484)
>   at 
> org.apache.hadoop.hdfs.qjournal.server.Journal.journal(Journal.java:353)
>   at 
> org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.journal(JournalNodeRpcServer.java:152)
>   at 
> org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.journal(QJournalProtocolServerSideTranslatorPB.java:158)
>   at 
> org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos$QJournalProtocolService$2.callBlockingMethod(QJournalProtocolProtos.java:25421)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> ç2016-08-25 07:19:47,651 INFO  namenode.FileJournalManager 
> (FileJournalManager.java:finalizeLogSegment(142)) - Finalizing edits file 
> /grid/0/hadoop/hdfs/journal/nameservice/current/edits_inprogress_0073697
>  -> 
> /grid/0/hadoop/hdfs/journal/nameservice/current/edits_0073697-0073698
> 
> Error at the exact RU task:
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
>  line 198, in 
> JournalNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 731, in restart
> self.post_upgrade_restart(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
>  line 75, in post_upgrade_restart
> journalnode_upgrade.post_upgrade_check()
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py",
>  line 64, in post_upgrade_check
> namenode_ha.is_encrypted(), params.security_enabled)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py",
>  line 306, in get_jmx_data
> data = urllib2.urlopen(nn_address).read()
>   File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen
> return _opener.open(url, data, timeout)
>   File "/usr/lib64/python2.6/urllib2.py", line 391, in open
> response = self._open(req, data)
>   File "/usr/lib64/python2.6/urllib2.py", line 409, in _open
> '_open', req)
>   File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
> result = func(*args)
>   File "/usr/lib64/python2.6/urllib2.py", line 1194, in https_open
> return self.do_open(httplib.HTTPSConnection, req)
>   File "/usr/lib64/python2.6/urllib2.py", line 1161, in do_open
> raise URLError(err)
> urllib2.URLError:  routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure>
> 
> Live cluster :  
> 
> Artifacts:  dgm10toerienoha-s11/test-logs/ambariru-dgm10toerie-sec-noha/ambaritestartifact
> s/artifacts/screenshots/com.hw.ambari.ui.tests.monitoring.admin_page.TestQuick
> 

[jira] [Updated] (AMBARI-18260) Journal node restart failing on RU from HDP 2.4.x to 2.5 on Wire Encrypted cluster

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-18260:
---
Description: 
Type of upgrade : RU  
Upgrade from HDP (2.4.2.0) to 2.5 (on secure, Wire encrypted
cluster)

Journal node logs show :




org.apache.hadoop.hdfs.qjournal.protocol.JournalOutOfSyncException: Can't 
write, no segment open
at 
org.apache.hadoop.hdfs.qjournal.server.Journal.checkSync(Journal.java:484)
at 
org.apache.hadoop.hdfs.qjournal.server.Journal.journal(Journal.java:353)
at 
org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.journal(JournalNodeRpcServer.java:152)
at 
org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.journal(QJournalProtocolServerSideTranslatorPB.java:158)
at 
org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos$QJournalProtocolService$2.callBlockingMethod(QJournalProtocolProtos.java:25421)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
ç2016-08-25 07:19:47,651 INFO  namenode.FileJournalManager 
(FileJournalManager.java:finalizeLogSegment(142)) - Finalizing edits file 
/grid/0/hadoop/hdfs/journal/nameservice/current/edits_inprogress_0073697
 -> 
/grid/0/hadoop/hdfs/journal/nameservice/current/edits_0073697-0073698


Error at the exact RU task:




Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
 line 198, in 
JournalNode().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 731, in restart
self.post_upgrade_restart(env, upgrade_type=upgrade_type)
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
 line 75, in post_upgrade_restart
journalnode_upgrade.post_upgrade_check()
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py",
 line 64, in post_upgrade_check
namenode_ha.is_encrypted(), params.security_enabled)
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py",
 line 306, in get_jmx_data
data = urllib2.urlopen(nn_address).read()
  File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
  File "/usr/lib64/python2.6/urllib2.py", line 391, in open
response = self._open(req, data)
  File "/usr/lib64/python2.6/urllib2.py", line 409, in _open
'_open', req)
  File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
result = func(*args)
  File "/usr/lib64/python2.6/urllib2.py", line 1194, in https_open
return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib64/python2.6/urllib2.py", line 1161, in do_open
raise URLError(err)
urllib2.URLError: 


Live cluster :  


Artifacts: 



  was:
Type of upgrade : RU  
Upgrade from HDP Derg M10 (2.4.2.0) to Erie (on secure, Wire encrypted
cluster)

Journal node logs show :




org.apache.hadoop.hdfs.qjournal.protocol.JournalOutOfSyncException: Can't 
write, no segment open
at 
org.apache.hadoop.hdfs.qjournal.server.Journal.checkSync(Journal.java:484)
at 
org.apache.hadoop.hdfs.qjournal.server.Journal.journal(Journal.java:353)
at 
org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.journal(JournalNodeRpcServer.java:152)
at 
org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.journal(QJournalProtocolServerSideTranslatorPB.java:158)
at 

[jira] [Updated] (AMBARI-18050) Upgrade pre-req check code needs to be decoupled from CheckDescription class

[Tim Thorpe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Thorpe updated AMBARI-18050:

   Resolution: Fixed
Fix Version/s: trunk
   Status: Resolved  (was: Patch Available)

Pushed to trunk

> Upgrade pre-req check code needs to be decoupled from CheckDescription class
> 
>
> Key: AMBARI-18050
> URL: https://issues.apache.org/jira/browse/AMBARI-18050
> Project: Ambari
>  Issue Type: New Feature
>  Components: ambari-server
>Affects Versions: trunk
>Reporter: Tim Thorpe
>Assignee: Tim Thorpe
> Fix For: trunk
>
> Attachments: AMBARI-18050.patch
>
>
> Currently each pre-req check must add an entry in the CheckDescription enum.  
> This limits the ability for third party stacks and extensions to provide 
> their own pre-req checks.  
> The CheckDescription enum should be rewritten as a class and each pre-req 
> check class should create an instance of it.  This will allow stacks and 
> extensions to include their own pre-req checks in separate jar files without 
> requiring changes to ambari-server java code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18360) ambari-agent check for unset variables (AMBARI-18317)

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484246#comment-15484246
 ] 

Hadoop QA commented on AMBARI-18360:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828039/AMBARI-18360.patch
  against trunk revision .

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8634//console

This message is automatically generated.

> ambari-agent check for unset variables (AMBARI-18317)
> -
>
> Key: AMBARI-18360
> URL: https://issues.apache.org/jira/browse/AMBARI-18360
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.4.2
>
> Attachments: AMBARI-18360.patch
>
>
> Reported in community:
> 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18324) Externalize skip repo url check to ambari.properties instead of hardcoding it in Ambari Java code

[Di Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Di Li updated AMBARI-18324:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Externalize skip repo url check to ambari.properties instead of hardcoding it 
> in Ambari Java code
> -
>
> Key: AMBARI-18324
> URL: https://issues.apache.org/jira/browse/AMBARI-18324
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-trunk
>Affects Versions: trunk
>Reporter: Di Li
>Assignee: Di Li
> Fix For: trunk
>
> Attachments: AMBARI-18324.patch, AMBARI-18324_rebase.patch
>
>
> Externalize skip repo url check to ambari.properties instead of hardcoding it 
> in Ambari Java code, so that users can define what repo ids to skip repo urls 
> validation when adding repos for the given stack



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18324) Externalize skip repo url check to ambari.properties instead of hardcoding it in Ambari Java code

[Di Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484240#comment-15484240
 ] 

Di Li commented on AMBARI-18324:


pushed to trunk as 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=623c36d2e0832d9a0bc60bc9ff19b302d5e3cefa

> Externalize skip repo url check to ambari.properties instead of hardcoding it 
> in Ambari Java code
> -
>
> Key: AMBARI-18324
> URL: https://issues.apache.org/jira/browse/AMBARI-18324
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-trunk
>Affects Versions: trunk
>Reporter: Di Li
>Assignee: Di Li
> Fix For: trunk
>
> Attachments: AMBARI-18324.patch, AMBARI-18324_rebase.patch
>
>
> Externalize skip repo url check to ambari.properties instead of hardcoding it 
> in Ambari Java code, so that users can define what repo ids to skip repo urls 
> validation when adding repos for the given stack



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18356) ZEPPELIN user won't get created in ZEPPELIN group

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484110#comment-15484110
 ] 

Hudson commented on AMBARI-18356:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #5650 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/5650/])
AMBARI-18356. ZEPPELIN user won't get created in ZEPPELIN group (aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=e9b4f5298123d98a062f458604a2ae322405a282])
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py


> ZEPPELIN user won't get created in ZEPPELIN group
> -
>
> Key: AMBARI-18356
> URL: https://issues.apache.org/jira/browse/AMBARI-18356
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.4.2
>
> Attachments: AMBARI-18356.patch
>
>
> Zeppelin user should also be created with group = ZEPPELIN. We should also 
> look at other
> services. Further we should look at how this user to group linkage can be
> config property driven instead of hardcoded logic in the before-ANY hook.
>  service/blob/master/configuration/zeppelin-env.xml#L29-L41>  
>  server/src/main/resources/stacks/HDP/2.0.6/hooks/before-
> ANY/scripts/params.py#L209-L221>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18357) Ambari fails to start nodemanager due to unexpected return code from sudo su command on the pid file

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484112#comment-15484112
 ] 

Hudson commented on AMBARI-18357:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #5650 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/5650/])
AMBARI-18357. Ambari fails to start nodemanager due to unexpected return 
(aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=1a279102ebf5ec99a772ff3ca2c889df31ce8c60])
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py
* (edit) 
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py
* (edit) ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py


> Ambari fails to start nodemanager due to unexpected return code from sudo su 
> command on the pid file
> 
>
> Key: AMBARI-18357
> URL: https://issues.apache.org/jira/browse/AMBARI-18357
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18357.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18360) ambari-agent check for unset variables (AMBARI-18317)

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484091#comment-15484091
 ] 

Hadoop QA commented on AMBARI-18360:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828039/AMBARI-18360.patch
  against trunk revision .

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8633//console

This message is automatically generated.

> ambari-agent check for unset variables (AMBARI-18317)
> -
>
> Key: AMBARI-18360
> URL: https://issues.apache.org/jira/browse/AMBARI-18360
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.4.2
>
> Attachments: AMBARI-18360.patch
>
>
> Reported in community:
> 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18360) ambari-agent check for unset variables (AMBARI-18317)

[Andrew Onischuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Onischuk updated AMBARI-18360:
-
Status: Patch Available  (was: Open)

> ambari-agent check for unset variables (AMBARI-18317)
> -
>
> Key: AMBARI-18360
> URL: https://issues.apache.org/jira/browse/AMBARI-18360
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.4.2
>
> Attachments: AMBARI-18360.patch
>
>
> Reported in community:
> 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18360) ambari-agent check for unset variables (AMBARI-18317)

[Andrew Onischuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Onischuk updated AMBARI-18360:
-
Attachment: AMBARI-18360.patch

> ambari-agent check for unset variables (AMBARI-18317)
> -
>
> Key: AMBARI-18360
> URL: https://issues.apache.org/jira/browse/AMBARI-18360
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.4.2
>
> Attachments: AMBARI-18360.patch
>
>
> Reported in community:
> 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18360) ambari-agent check for unset variables (AMBARI-18317)

[Andrew Onischuk (JIRA)]

Andrew Onischuk created AMBARI-18360:


 Summary: ambari-agent check for unset variables (AMBARI-18317)
 Key: AMBARI-18360
 URL: https://issues.apache.org/jira/browse/AMBARI-18360
 Project: Ambari
  Issue Type: Bug
Reporter: Andrew Onischuk
Assignee: Andrew Onischuk
 Fix For: 2.4.2
 Attachments: AMBARI-18360.patch

Reported in community:







--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (AMBARI-17728) Error message does not deliver when executing ambari-server command as a non-root user

[wangyaoxin (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477199#comment-15477199
 ] 

wangyaoxin edited comment on AMBARI-17728 at 9/12/16 1:13 PM:
--

Hi [~afernandez],  [~jluniya]  ,[~smnaha]  ,  thank you for review my patch and 
ship it from the reviewboard , I have to  trouble you to  commit the patch!
Thank you !


was (Author: forestsissi):
Hi  [~jluniya]  ,[~smnaha]  ,   trouble you to  commit the patch!Thank you !

> Error message does not deliver when executing ambari-server command as a 
> non-root user
> --
>
> Key: AMBARI-17728
> URL: https://issues.apache.org/jira/browse/AMBARI-17728
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: trunk, 2.4.0
>Reporter: wangyaoxin
>Assignee: wangyaoxin
> Fix For: trunk
>
> Attachments: AMBARI-17728-1.patch, AMBARI-17728-2.patch, 
> AMBARI-17728.patch
>
>
> non-root user: like hdfs
> excute : ambari-server stop
> show :  Using python  /usr/bin/python2.6 Stopping ambari-server
> intent to: You can't perform this operation as non-sudoer user. Please, 
> re-login or configure sudo access for this user



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18356) ZEPPELIN user won't get created in ZEPPELIN group

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484062#comment-15484062
 ] 

Hudson commented on AMBARI-18356:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #16 (See 
[https://builds.apache.org/job/Ambari-branch-2.5/16/])
AMBARI-18356. ZEPPELIN user won't get created in ZEPPELIN group (aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=6d634522ff24851348cd079d1369c0fe5b2174d0])
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py


> ZEPPELIN user won't get created in ZEPPELIN group
> -
>
> Key: AMBARI-18356
> URL: https://issues.apache.org/jira/browse/AMBARI-18356
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.4.2
>
> Attachments: AMBARI-18356.patch
>
>
> Zeppelin user should also be created with group = ZEPPELIN. We should also 
> look at other
> services. Further we should look at how this user to group linkage can be
> config property driven instead of hardcoded logic in the before-ANY hook.
>  service/blob/master/configuration/zeppelin-env.xml#L29-L41>  
>  server/src/main/resources/stacks/HDP/2.0.6/hooks/before-
> ANY/scripts/params.py#L209-L221>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18357) Ambari fails to start nodemanager due to unexpected return code from sudo su command on the pid file

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484064#comment-15484064
 ] 

Hudson commented on AMBARI-18357:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #16 (See 
[https://builds.apache.org/job/Ambari-branch-2.5/16/])
AMBARI-18357. Ambari fails to start nodemanager due to unexpected return 
(aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=f0dd1b9303cfc7db7b761b4ce2d02114f7c2e6ef])
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py
* (edit) 
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py
* (edit) ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py


> Ambari fails to start nodemanager due to unexpected return code from sudo su 
> command on the pid file
> 
>
> Key: AMBARI-18357
> URL: https://issues.apache.org/jira/browse/AMBARI-18357
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18357.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18323) Cluster creation complete is missing from ambari-server log

[Sandor Magyari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandor Magyari updated AMBARI-18323:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Cluster creation complete is missing from ambari-server log
> ---
>
> Key: AMBARI-18323
> URL: https://issues.apache.org/jira/browse/AMBARI-18323
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Sandor Magyari
>Assignee: Sandor Magyari
> Fix For: trunk, 2.5.0
>
> Attachments: AMBARI-18323_v2.patch
>
>
> In case provision_action is set to INSTALL_ONLY on a given hostgroup compoent 
> in Blueprint, cluster completion message is not logged. Normally during 
> Blueprint deployments Ambari creates LogicalTasks bounded later to INSTALL 
> tasks (HostRoleCommand), however in this case there are  no INSTALL task 
> bounded to LogicalTasks, so they will never complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18323) Cluster creation complete is missing from ambari-server log

[Sandor Magyari (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484030#comment-15484030
 ] 

Sandor Magyari commented on AMBARI-18323:
-

Committed to trunk & 'branch-2.5'.

> Cluster creation complete is missing from ambari-server log
> ---
>
> Key: AMBARI-18323
> URL: https://issues.apache.org/jira/browse/AMBARI-18323
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Sandor Magyari
>Assignee: Sandor Magyari
> Fix For: trunk, 2.5.0
>
> Attachments: AMBARI-18323_v2.patch
>
>
> In case provision_action is set to INSTALL_ONLY on a given hostgroup compoent 
> in Blueprint, cluster completion message is not logged. Normally during 
> Blueprint deployments Ambari creates LogicalTasks bounded later to INSTALL 
> tasks (HostRoleCommand), however in this case there are  no INSTALL task 
> bounded to LogicalTasks, so they will never complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18359) Get LDAP tests to run in concurrent forked JVMs

[Doroszlai, Attila (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doroszlai, Attila updated AMBARI-18359:
---
Status: Patch Available  (was: Open)

> Get LDAP tests to run in concurrent forked JVMs
> ---
>
> Key: AMBARI-18359
> URL: https://issues.apache.org/jira/browse/AMBARI-18359
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
> Fix For: 2.5.0
>
> Attachments: AMBARI-18359.patch
>
>
> Some LDAP tests fail to run with forkCount:>1, reuseForks:false. This blocks 
> increasing forkCount to speed up the unit tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18359) Get LDAP tests to run in concurrent forked JVMs

[Doroszlai, Attila (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doroszlai, Attila updated AMBARI-18359:
---
Attachment: AMBARI-18359.patch

> Get LDAP tests to run in concurrent forked JVMs
> ---
>
> Key: AMBARI-18359
> URL: https://issues.apache.org/jira/browse/AMBARI-18359
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Doroszlai, Attila
>Assignee: Doroszlai, Attila
> Fix For: 2.5.0
>
> Attachments: AMBARI-18359.patch
>
>
> Some LDAP tests fail to run with forkCount:>1, reuseForks:false. This blocks 
> increasing forkCount to speed up the unit tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18359) Get LDAP tests to run in concurrent forked JVMs

[Doroszlai, Attila (JIRA)]

Doroszlai, Attila created AMBARI-18359:
--

 Summary: Get LDAP tests to run in concurrent forked JVMs
 Key: AMBARI-18359
 URL: https://issues.apache.org/jira/browse/AMBARI-18359
 Project: Ambari
  Issue Type: Task
  Components: ambari-server
Affects Versions: 2.5.0
Reporter: Doroszlai, Attila
Assignee: Doroszlai, Attila
 Fix For: 2.5.0


Some LDAP tests fail to run with forkCount:>1, reuseForks:false. This blocks 
increasing forkCount to speed up the unit tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18355) Introduce conditional dependencies in stack defition to handle blueprint validation gracefully

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483968#comment-15483968
 ] 

Hadoop QA commented on AMBARI-18355:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12827947/AMBARI-18355.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The test build failed in ambari-server 

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8632//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8632//console

This message is automatically generated.

> Introduce conditional dependencies in stack defition to handle blueprint 
> validation gracefully
> --
>
> Key: AMBARI-18355
> URL: https://issues.apache.org/jira/browse/AMBARI-18355
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server
>Affects Versions: trunk
>Reporter: Amruta Borkar
>Assignee: Amruta Borkar
> Fix For: trunk
>
> Attachments: AMBARI-18355.patch, Adding Conditional Dependencies.pdf
>
>
> Currently stack definitions do not list conditional dependencies, adding 
> those to the stack definitions would make it easy to validate errors in case 
> of blueprint deployment. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18260) Journal node restart failing on RU from dergM10 to erie on Wire Encrypted cluster

[Andrew Onischuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Onischuk updated AMBARI-18260:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk and branch-2.5

> Journal node restart failing on RU from dergM10 to erie on Wire Encrypted 
> cluster
> -
>
> Key: AMBARI-18260
> URL: https://issues.apache.org/jira/browse/AMBARI-18260
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: trunk
>
> Attachments: AMBARI-18260.patch
>
>
> Type of upgrade : RU  
> Upgrade from HDP Derg M10 (2.4.2.0) to Erie (on secure, Wire encrypted
> cluster)
> Journal node logs show :
> 
> 
> 
> org.apache.hadoop.hdfs.qjournal.protocol.JournalOutOfSyncException: Can't 
> write, no segment open
>   at 
> org.apache.hadoop.hdfs.qjournal.server.Journal.checkSync(Journal.java:484)
>   at 
> org.apache.hadoop.hdfs.qjournal.server.Journal.journal(Journal.java:353)
>   at 
> org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.journal(JournalNodeRpcServer.java:152)
>   at 
> org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.journal(QJournalProtocolServerSideTranslatorPB.java:158)
>   at 
> org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos$QJournalProtocolService$2.callBlockingMethod(QJournalProtocolProtos.java:25421)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> ç2016-08-25 07:19:47,651 INFO  namenode.FileJournalManager 
> (FileJournalManager.java:finalizeLogSegment(142)) - Finalizing edits file 
> /grid/0/hadoop/hdfs/journal/nameservice/current/edits_inprogress_0073697
>  -> 
> /grid/0/hadoop/hdfs/journal/nameservice/current/edits_0073697-0073698
> 
> Error at the exact RU task:
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
>  line 198, in 
> JournalNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 731, in restart
> self.post_upgrade_restart(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
>  line 75, in post_upgrade_restart
> journalnode_upgrade.post_upgrade_check()
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py",
>  line 64, in post_upgrade_check
> namenode_ha.is_encrypted(), params.security_enabled)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py",
>  line 306, in get_jmx_data
> data = urllib2.urlopen(nn_address).read()
>   File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen
> return _opener.open(url, data, timeout)
>   File "/usr/lib64/python2.6/urllib2.py", line 391, in open
> response = self._open(req, data)
>   File "/usr/lib64/python2.6/urllib2.py", line 409, in _open
> '_open', req)
>   File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
> result = func(*args)
>   File "/usr/lib64/python2.6/urllib2.py", line 1194, in https_open
> return self.do_open(httplib.HTTPSConnection, req)
>   File "/usr/lib64/python2.6/urllib2.py", line 1161, in do_open
> raise URLError(err)
> urllib2.URLError:  routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure>
> 
> Live cluster :  
> 
> Artifacts:  dgm10toerienoha-s11/test-logs/ambariru-dgm10toerie-sec-noha/ambaritestartifact
> s/artifacts/screenshots/com.hw.ambari.ui.tests.monitoring.admin_page.TestQuick
> RollingUpgradeApi/test060_StartPerformUpgrade/_24_22_9_0_One_step_of_upgrade_f
> 

[jira] [Commented] (AMBARI-18237) Certain configuration files cannot be modified through Ambari api.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483867#comment-15483867
 ] 

Hudson commented on AMBARI-18237:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #5649 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/5649/])
AMBARI-18237. Certain configuration files cannot be modified through (aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=4b141dd8842fe97d9cf8565af179ea2b68191729])
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py


> Certain configuration files cannot be modified through Ambari api.
> --
>
> Key: AMBARI-18237
> URL: https://issues.apache.org/jira/browse/AMBARI-18237
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18237.patch
>
>
> Certain configuration files like hadoop-metrics2.properties are not exposed to
> Ambari api calls, therefore cannot be modified by Ambari REST api calls.
> Ambari QE team uses these configuration files to modify it for HDP stack
> tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18286) Processes children are not killed on timeout

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483850#comment-15483850
 ] 

Hudson commented on AMBARI-18286:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #15 (See 
[https://builds.apache.org/job/Ambari-branch-2.5/15/])
AMBARI-18286. Processes children are not killed on timeout (aonishuk) 
(aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=7ef4fe57f86fd3047bae4f1326ac329d2121ba73])
* (edit) ambari-common/src/main/python/resource_management/core/shell.py
* (edit) ambari-common/src/main/python/resource_management/core/utils.py
* (edit) 
ambari-common/src/main/python/resource_management/core/resources/system.py
* (edit) ambari-agent/src/test/python/resource_management/TestGroupResource.py
* (edit) ambari-agent/src/test/python/resource_management/TestUserResource.py
* (edit) 
ambari-common/src/main/python/resource_management/core/providers/system.py


> Processes children are not killed on timeout
> 
>
> Key: AMBARI-18286
> URL: https://issues.apache.org/jira/browse/AMBARI-18286
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18286.patch
>
>
> This leads to spawning a lot of processes on machines where Hive alerts run.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18237) Certain configuration files cannot be modified through Ambari api.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483849#comment-15483849
 ] 

Hudson commented on AMBARI-18237:
-

FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #15 (See 
[https://builds.apache.org/job/Ambari-branch-2.5/15/])
AMBARI-18237. Certain configuration files cannot be modified through (aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=80a6b01415cba7f011be1551ea24bef7cbfe393c])
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
* (edit) 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py


> Certain configuration files cannot be modified through Ambari api.
> --
>
> Key: AMBARI-18237
> URL: https://issues.apache.org/jira/browse/AMBARI-18237
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18237.patch
>
>
> Certain configuration files like hadoop-metrics2.properties are not exposed to
> Ambari api calls, therefore cannot be modified by Ambari REST api calls.
> Ambari QE team uses these configuration files to modify it for HDP stack
> tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18357) Ambari fails to start nodemanager due to unexpected return code from sudo su command on the pid file

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483730#comment-15483730
 ] 

Hadoop QA commented on AMBARI-18357:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828007/AMBARI-18357.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The test build failed in ambari-server 

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8631//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8631//console

This message is automatically generated.

> Ambari fails to start nodemanager due to unexpected return code from sudo su 
> command on the pid file
> 
>
> Key: AMBARI-18357
> URL: https://issues.apache.org/jira/browse/AMBARI-18357
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18357.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18357) Ambari fails to start nodemanager due to unexpected return code from sudo su command on the pid file

[Andrew Onischuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Onischuk updated AMBARI-18357:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk and branch-2.5

> Ambari fails to start nodemanager due to unexpected return code from sudo su 
> command on the pid file
> 
>
> Key: AMBARI-18357
> URL: https://issues.apache.org/jira/browse/AMBARI-18357
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18357.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-18356) ZEPPELIN user won't get created in ZEPPELIN group

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483688#comment-15483688
 ] 

Hadoop QA commented on AMBARI-18356:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828006/AMBARI-18356.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The test build failed in ambari-server 

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8630//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/8630//console

This message is automatically generated.

> ZEPPELIN user won't get created in ZEPPELIN group
> -
>
> Key: AMBARI-18356
> URL: https://issues.apache.org/jira/browse/AMBARI-18356
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.4.2
>
> Attachments: AMBARI-18356.patch
>
>
> Zeppelin user should also be created with group = ZEPPELIN. We should also 
> look at other
> services. Further we should look at how this user to group linkage can be
> config property driven instead of hardcoded logic in the before-ANY hook.
>  service/blob/master/configuration/zeppelin-env.xml#L29-L41>  
>  server/src/main/resources/stacks/HDP/2.0.6/hooks/before-
> ANY/scripts/params.py#L209-L221>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18358) Deleting a service with node specific configuration makes Ambari unstable

[Gonzalo Herreros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gonzalo Herreros updated AMBARI-18358:
--
Description: 
Always reproduced.

On any service, change some setting to be node specific (instead of global to 
the cluster). 
To do so create a config group, switch to it and change some setting to be 
specific for that group.

Now, stop and delete the service. The configuration is deleted but the group 
configuration is left behind.

>From this moment the log will be full of " Config inconsistency exists: 
>unknown configType=dbks-site" (in my case the service I reproduced it was 
>Ranger KMS).
Any attempt to add a service will get a 500.

The only workaround is going to the DB and manually delete the group 
configuration from the table confgroupclusterconfigmapping

  was:
Always reproduced.

On any service, change some setting to be node specific (instead of global to 
the cluster). 
To do so create a config group, switch to it and change some setting to be 
specific for that group.

Now, stop and delete the service. The configuration is deleted but the group 
configuration is left behind.

>From this moment the log will be full of " Config inconsistency exists: 
>unknown configType=dbks-site" (in my case the service I reproduced it was 
>Ranger KMS).
Any attempt to add a service will get a 500.

The only workaround is going to the DB and manually delete the group 
configuration.


> Deleting a service with node specific configuration makes Ambari unstable
> -
>
> Key: AMBARI-18358
> URL: https://issues.apache.org/jira/browse/AMBARI-18358
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-admin
>Affects Versions: 2.4.0
>Reporter: Gonzalo Herreros
>
> Always reproduced.
> On any service, change some setting to be node specific (instead of global to 
> the cluster). 
> To do so create a config group, switch to it and change some setting to be 
> specific for that group.
> Now, stop and delete the service. The configuration is deleted but the group 
> configuration is left behind.
> From this moment the log will be full of " Config inconsistency exists: 
> unknown configType=dbks-site" (in my case the service I reproduced it was 
> Ranger KMS).
> Any attempt to add a service will get a 500.
> The only workaround is going to the DB and manually delete the group 
> configuration from the table confgroupclusterconfigmapping



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18356) ZEPPELIN user won't get created in ZEPPELIN group

[Andrew Onischuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Onischuk updated AMBARI-18356:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk and branch-2.5

> ZEPPELIN user won't get created in ZEPPELIN group
> -
>
> Key: AMBARI-18356
> URL: https://issues.apache.org/jira/browse/AMBARI-18356
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.4.2
>
> Attachments: AMBARI-18356.patch
>
>
> Zeppelin user should also be created with group = ZEPPELIN. We should also 
> look at other
> services. Further we should look at how this user to group linkage can be
> config property driven instead of hardcoded logic in the before-ANY hook.
>  service/blob/master/configuration/zeppelin-env.xml#L29-L41>  
>  server/src/main/resources/stacks/HDP/2.0.6/hooks/before-
> ANY/scripts/params.py#L209-L221>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18358) Deleting a service with node specific configuration makes Ambari unstable

[Gonzalo Herreros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gonzalo Herreros updated AMBARI-18358:
--
Description: 
Always reproduced.

On any service, change some setting to be node specific (instead of global to 
the cluster). 
To do so create a config group, switch to it and change some setting to be 
specific for that group.

Now, stop and delete the service. The configuration is deleted but the group 
configuration is left behind.

>From this moment the log will be full of " Config inconsistency exists: 
>unknown configType=dbks-site" (in my case the service I reproduced it was 
>Ranger KMS).
Any attempt to add a service will get a 500.

The only workaround is going to the DB and manually delete the group 
configuration.

  was:
Always reproduced.

On any service change some setting to be node specific (instead of global to 
the cluster). To do so create a config group, switch to it and change some 
setting to be specific for that group.

Now, stop and delete the service. The configuration is deleted but the group 
configuration is left behind.

>From this moment the log will be full of " Config inconsistency exists: 
>unknown configType=dbks-site" (in my case the service I reproduced it was 
>Ranger KMS).
Any attempt to add a service will get a 500.

The only workaround is going to the DB and manually delete the group 
configuration.


> Deleting a service with node specific configuration makes Ambari unstable
> -
>
> Key: AMBARI-18358
> URL: https://issues.apache.org/jira/browse/AMBARI-18358
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-admin
>Affects Versions: 2.4.0
>Reporter: Gonzalo Herreros
>
> Always reproduced.
> On any service, change some setting to be node specific (instead of global to 
> the cluster). 
> To do so create a config group, switch to it and change some setting to be 
> specific for that group.
> Now, stop and delete the service. The configuration is deleted but the group 
> configuration is left behind.
> From this moment the log will be full of " Config inconsistency exists: 
> unknown configType=dbks-site" (in my case the service I reproduced it was 
> Ranger KMS).
> Any attempt to add a service will get a 500.
> The only workaround is going to the DB and manually delete the group 
> configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18358) Deleting a service with node specific configuration makes Ambari unstable

[Gonzalo Herreros (JIRA)]

Gonzalo Herreros created AMBARI-18358:
-

 Summary: Deleting a service with node specific configuration makes 
Ambari unstable
 Key: AMBARI-18358
 URL: https://issues.apache.org/jira/browse/AMBARI-18358
 Project: Ambari
  Issue Type: Bug
  Components: ambari-admin
Affects Versions: 2.4.0
Reporter: Gonzalo Herreros


Always reproduced.

On any service change some setting to be node specific (instead of global to 
the cluster). To do so create a config group, switch to it and change some 
setting to be specific for that group.

Now, stop and delete the service. The configuration is deleted but the group 
configuration is left behind.

>From this moment the log will be full of " Config inconsistency exists: 
>unknown configType=dbks-site" (in my case the service I reproduced it was 
>Ranger KMS).
Any attempt to add a service will get a 500.

The only workaround is going to the DB and manually delete the group 
configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-18357) Ambari fails to start nodemanager due to unexpected return code from sudo su command on the pid file

[Andrew Onischuk (JIRA)]

Andrew Onischuk created AMBARI-18357:


 Summary: Ambari fails to start nodemanager due to unexpected 
return code from sudo su command on the pid file
 Key: AMBARI-18357
 URL: https://issues.apache.org/jira/browse/AMBARI-18357
 Project: Ambari
  Issue Type: Bug
Reporter: Andrew Onischuk
Assignee: Andrew Onischuk
 Fix For: 2.5.0
 Attachments: AMBARI-18357.patch





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18357) Ambari fails to start nodemanager due to unexpected return code from sudo su command on the pid file

[Andrew Onischuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Onischuk updated AMBARI-18357:
-
Status: Patch Available  (was: Open)

> Ambari fails to start nodemanager due to unexpected return code from sudo su 
> command on the pid file
> 
>
> Key: AMBARI-18357
> URL: https://issues.apache.org/jira/browse/AMBARI-18357
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18357.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-18357) Ambari fails to start nodemanager due to unexpected return code from sudo su command on the pid file

[Andrew Onischuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-18357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Onischuk updated AMBARI-18357:
-
Attachment: AMBARI-18357.patch

> Ambari fails to start nodemanager due to unexpected return code from sudo su 
> command on the pid file
> 
>
> Key: AMBARI-18357
> URL: https://issues.apache.org/jira/browse/AMBARI-18357
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.5.0
>
> Attachments: AMBARI-18357.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)