Re: cross-site vulnerability of APIs
Vulnerabilities should not be exposed on public mailing lists without giving a project the chance to patch vulnerable released versions. Please report such vulnerabilities to secur...@apache.org or priv...@ambari.apache.org. http://www.apache.org/security/#reporting-a-vulnerability thanks — Hitesh On Apr 5, 2016, at 1:31 PM, Keta Patelwrote: > Hello all, > I recently encountered a couple of APIs which were vulnerable to cross-site > script attacks through parameters like "description" or "name". These > parameters are passed in directly to server-side code and stored in the > database. The UI validation at present only checks for the length of the > input text. There needs to be a more robust server-side validation to > handle XSS attacks. > > Could somebody please help me by pointing out if there is an existing way > to handle > this vulnerability or whether it must be handled from scratch. > > Thanks in advance! > Keta
[jira] [Updated] (AMBARI-14951) Tez Ambari View: Add protocol configuration for YARN
[ https://issues.apache.org/jira/browse/AMBARI-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-14951: - Attachment: AMBARI-14951.4.patch Revert unintentional tez.version change in tez pom.xml > Tez Ambari View: Add protocol configuration for YARN > > > Key: AMBARI-14951 > URL: https://issues.apache.org/jira/browse/AMBARI-14951 > Project: Ambari > Issue Type: Bug >Affects Versions: 2.2.2 >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 2.2.2 > > Attachments: AMBARI-14951.1.patch, AMBARI-14951.2.patch, > AMBARI-14951.3.patch, AMBARI-14951.4.patch > > > - Value set in yarn.http.policy is required for creating task attempt log > links, and we need a mechanism to get that configuration value at the UI side. > - As host points to Ambari proxy, there is no way to know the protocol at > which yarn daemons are configured to run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-14951) Tez Ambari View: Add protocol configuration for YARN
[ https://issues.apache.org/jira/browse/AMBARI-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169983#comment-15169983 ] Hitesh Shah commented on AMBARI-14951: -- [~sumitmohanty] please review. Reactor order seems to be fine on my local setup: [INFO] [INFO] Reactor Build Order: [INFO] [INFO] Ambari Contrib Views [INFO] Ambari View Utils [INFO] Files [INFO] Jobs [INFO] Pig [INFO] Slider [INFO] Capacity Scheduler [INFO] Tez [INFO] Storm_Monitoring [INFO] Hive [INFO] > Tez Ambari View: Add protocol configuration for YARN > > > Key: AMBARI-14951 > URL: https://issues.apache.org/jira/browse/AMBARI-14951 > Project: Ambari > Issue Type: Bug >Affects Versions: 2.2.2 >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 2.2.2 > > Attachments: AMBARI-14951.1.patch, AMBARI-14951.2.patch, > AMBARI-14951.3.patch > > > - Value set in yarn.http.policy is required for creating task attempt log > links, and we need a mechanism to get that configuration value at the UI side. > - As host points to Ambari proxy, there is no way to know the protocol at > which yarn daemons are configured to run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-14951) Tez Ambari View: Add protocol configuration for YARN
[ https://issues.apache.org/jira/browse/AMBARI-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-14951: - Attachment: AMBARI-14951.3.patch Fix pom versions > Tez Ambari View: Add protocol configuration for YARN > > > Key: AMBARI-14951 > URL: https://issues.apache.org/jira/browse/AMBARI-14951 > Project: Ambari > Issue Type: Bug >Affects Versions: 2.2.2 >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 2.2.2 > > Attachments: AMBARI-14951.1.patch, AMBARI-14951.2.patch, > AMBARI-14951.3.patch > > > - Value set in yarn.http.policy is required for creating task attempt log > links, and we need a mechanism to get that configuration value at the UI side. > - As host points to Ambari proxy, there is no way to know the protocol at > which yarn daemons are configured to run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Debugging Views
Is this a secure or non-secure cluster? Secure clusters need some additional configs but usually the view should display back an error pointing to the problem. The view errors are usually logged at debug level in the ambari-server ( think there is an open jira to fix this at some point ). Changing the log level may shed some light on the issue if there were errors. The common errors are usually: - misconfiguration of the RM or timeline hosts in the view configuration - failure to connect/read data from the above servers. Are you seeing the view with the table correctly displayed but having no data in the rows? Try the following call: http://timelinehost:8188/ws/v1/timeline/TEZ_DAG_ID?limit=11. If this does not return any data, you will need to debug whether the tez jobs were configured to write data to ATS. Additionally, if your cluster has acls enabled ( search for acls in tez-site.xml ), it might not be showing you data based on which user ran the job and if you are logged in to ambari with a different user id. If you have specific issues with the Tez view, feel free to follow up at u...@tez.apache.org thanks — Hitesh On Feb 12, 2016, at 7:14 AM, Harp, Michaelwrote: > Anyone have a technique for debugging view issues? For instance I am not > seeing any job records displayed in the stock Tez view(ambari-2.2.2.0-64) and > so far have not found any clues in ambari-server.log or javascript errors on > the frontend. > Thanks, > Mike
Re: Epic for supporting downloadable stack definitions
Is there a plan to support extending such stack definitions after initial setup? i.e. say you have an HDP 2.3 stack but want to layer it with your own custom services at some point in the future. Is this new download capability being designed to handle incremental updates? — Hitesh On Jan 31, 2016, at 7:09 PM, Siddharth Waglewrote: > Hi all, > > > I have created an Epic for the next Ambari release for separating Ambari core > and stack definitions. The stack definitions give the management and > monitoring capabilities to a service and therefore the term stack management > pack has been coined instead. > > > The scope of work is still at high level however, Ambari's design already > caters to this model and thereby the changes should be limited to enhancing > the cluster setup, UI modifications, validation of stack definition, > inheritance and son. > > > https://issues.apache.org/jira/browse/AMBARI-14854 > > > Please feel free to comment. Contributions welcome. > > > Best Regards, > > Sid
[jira] [Commented] (AMBARI-11473) Fix Tez view to surface underlying errors resulting from failure to communicate with timeline
[ https://issues.apache.org/jira/browse/AMBARI-11473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15032703#comment-15032703 ] Hitesh Shah commented on AMBARI-11473: -- [~tbeerbower] does this jira need to be resolved? > Fix Tez view to surface underlying errors resulting from failure to > communicate with timeline > - > > Key: AMBARI-11473 > URL: https://issues.apache.org/jira/browse/AMBARI-11473 > Project: Ambari > Issue Type: Bug > Reporter: Hitesh Shah > Assignee: Hitesh Shah > Attachments: AMBARI-11473.1.noprefix.patch, AMBARI-11473.1.patch, > AMBARI-11473.2.patch, AMBARI-11473.3.patch > > > If there are any errors in communicating with timeline, the Tez view always > shows a 404. > \cc [~Sreenath] [~pramachandran] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12179) Tez View should automatically configuration settings based on http/https policy
[ https://issues.apache.org/jira/browse/AMBARI-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-12179: - Attachment: AMBARI-12179.4.patch Re-uploading view-xml unchanged as patch 4 to keep things clear. Tez View should automatically configuration settings based on http/https policy --- Key: AMBARI-12179 URL: https://issues.apache.org/jira/browse/AMBARI-12179 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Fix For: 2.1.1 Attachments: AMBARI-12179.1.patch, AMBARI-12179.2.patch, AMBARI-12179.3.patch, AMBARI-12179.3.view-xml_unchanged.patch, AMBARI-12179.4.patch The current tez view impl always defaults to the http addresses instead of relying on the http policy setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12179) Tez View should automatically configuration settings based on http/https policy
[ https://issues.apache.org/jira/browse/AMBARI-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-12179: - Attachment: AMBARI-12179.3.patch Patch updated to address [~Sreenath]'s comment. Tez View should automatically configuration settings based on http/https policy --- Key: AMBARI-12179 URL: https://issues.apache.org/jira/browse/AMBARI-12179 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Fix For: 2.1.1 Attachments: AMBARI-12179.1.patch, AMBARI-12179.2.patch, AMBARI-12179.3.patch The current tez view impl always defaults to the http addresses instead of relying on the http policy setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12179) Tez View should automatically configuration settings based on http/https policy
[ https://issues.apache.org/jira/browse/AMBARI-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-12179: - Attachment: AMBARI-12179.3.view-xml_unchanged.patch Same as patch 3 except for changes in view.xml reverted due to Ambari bug. Tez View should automatically configuration settings based on http/https policy --- Key: AMBARI-12179 URL: https://issues.apache.org/jira/browse/AMBARI-12179 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Fix For: 2.1.1 Attachments: AMBARI-12179.1.patch, AMBARI-12179.2.patch, AMBARI-12179.3.patch, AMBARI-12179.3.view-xml_unchanged.patch The current tez view impl always defaults to the http addresses instead of relying on the http policy setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-12179) Tez View should automatically configuration settings based on http/https policy
[ https://issues.apache.org/jira/browse/AMBARI-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658447#comment-14658447 ] Hitesh Shah commented on AMBARI-12179: -- [~Sreenath] Mind trying out the 3rd patch ( with view-xml unchanged ) ? Tez View should automatically configuration settings based on http/https policy --- Key: AMBARI-12179 URL: https://issues.apache.org/jira/browse/AMBARI-12179 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Fix For: 2.1.1 Attachments: AMBARI-12179.1.patch, AMBARI-12179.2.patch, AMBARI-12179.3.patch, AMBARI-12179.3.view-xml_unchanged.patch The current tez view impl always defaults to the http addresses instead of relying on the http policy setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12179) Tez View should automatically configuration settings based on http/https policy
[ https://issues.apache.org/jira/browse/AMBARI-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-12179: - Attachment: AMBARI-12179.3.view-xml_unchanged.patch Re-uploading patch Tez View should automatically configuration settings based on http/https policy --- Key: AMBARI-12179 URL: https://issues.apache.org/jira/browse/AMBARI-12179 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Fix For: 2.1.1 Attachments: AMBARI-12179.1.patch, AMBARI-12179.2.patch, AMBARI-12179.3.patch, AMBARI-12179.3.view-xml_unchanged.patch The current tez view impl always defaults to the http addresses instead of relying on the http policy setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12179) Tez View should automatically configuration settings based on http/https policy
[ https://issues.apache.org/jira/browse/AMBARI-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-12179: - Attachment: (was: AMBARI-12179.3.view-xml_unchanged.patch) Tez View should automatically configuration settings based on http/https policy --- Key: AMBARI-12179 URL: https://issues.apache.org/jira/browse/AMBARI-12179 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Fix For: 2.1.1 Attachments: AMBARI-12179.1.patch, AMBARI-12179.2.patch, AMBARI-12179.3.patch, AMBARI-12179.3.view-xml_unchanged.patch The current tez view impl always defaults to the http addresses instead of relying on the http policy setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [VOTE] Apache Ambari Release 2.0.2 RC2
-0. There are a bunch of issues that cause a cluster to come up as non-functional when using Ambari-2.0.1 and the 2.0.2 RC. For example, without AMBARI-11640, Hive on Tez does not work. It would be good to have these back ported into 2.0.2. There are probably a whole lot more issues that affect Ambari 2.0.x but were only fixed in Ambari-2.1.x. It might be good to backport all the critical ones. thanks — Hitesh On Aug 3, 2015, at 6:39 PM, Alejandro Fernandez alejan...@apache.org wrote: Hi, I have created an Apache Ambari 2.0.2 Release Candidate (RC2). This is a maintenance release for branch-2.0.maint GIT source tag: *https://git-wip-us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.0.2-rc2 https://git-wip-us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.0.2-rc2* Staging site: http://people.apache.org/~alejandro/apache-ambari-2.0.2-rc2/ PGP release key used (signed using 392F57AC): http://pgp.mit.edu:11371/pks/lookup?op=vindexsearch=0x2147F220392F57AC Updated KEYS file: https://dist.apache.org/repos/dist/release/ambari/KEYS One can look into the issues fixed in this release at https://issues.apache.org/jira/browse/AMBARI/fixforversion/12332747 Total 2.0.2 JIRAs: 13 Total JIRAs Resolved as Fixed: 13 Vote will be open for 72 hours. [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Here's my vote to start: +1 Here's how to verify the tarball hashes: git clone --branch release-2.0.2-rc2 https://git-wip-us.apache.org/repos/asf/ambari.git apache-ambari-2.0.2-src cd apache-ambari-2.0.2-src git clean -xdf cd ambari-web npm install ulimit -n 2048 brunch build rm -rf node_modules mv public public-static cd ../.. tar --exclude=.git --exclude=.gitignore --exclude=.gitattributes -zcvf apache-ambari-2.0.2-src.tar.gz apache-ambari-2.0.2-src # Verify my signature, gpg --verify apache-ambari-2.0.2-src.tar.gz.asc apache-ambari-2.0.2-src.tar.gz gpg: Signature made Fri Jul 31 18:36:49 2015 PDT using RSA key ID 392F57AC gpg: Good signature from Alejandro Fernandez (Apache Ambari Release Management) alejan...@apache.org openssl md5 apache-ambari-2.0.2-src.tar.gz apache-ambari-2.0.2-src.tar.gz.md5# and compare to the file I provided openssl sha1 apache-ambari-2.0.2-src.tar.gz apache-ambari-2.0.2-src.tar.gz.sha1 # and compare to the file I provided # I made sure ratcheck passes cp -R apache-ambari-2.0.2-src apache-ambari-2.0.2-ratcheck cd apache-ambari-2.0.2-ratcheck mvn clean apache-rat:check Thanks, Alejandro Fernandez Apache Ambari PMC
Re: [VOTE] Apache Ambari Release 2.0.2 RC1
-0 The source seems to include fontawesome fonts which are licensed using SIL Open Font License but the license does not account for it. Ref: http://www.apache.org/legal/resolved.html#category-b — Hitesh On Jul 31, 2015, at 6:47 PM, Alejandro Fernandez alejan...@apache.org wrote: Hi, I have created an Apache Ambari 2.0.2 Release Candidate (RC1). This is a maintenance release for branch-2.0.maint GIT source tag: *https://git-wip-us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.0.2-rc1 https://git-wip-us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=log;h=refs/tags/release-2.0.2-rc1* Staging site: http://people.apache.org/~alejandro/apache-ambari-2.0.2-rc1/ PGP release key used (signed using 392F57AC): http://pgp.mit.edu:11371/pks/lookup?op=vindexsearch=0x2147F220392F57AC Updated KEYS file: https://dist.apache.org/repos/dist/release/ambari/KEYS One can look into the issues fixed in this release at https://issues.apache.org/jira/browse/AMBARI/fixforversion/12332747 Total 2.0.2 JIRAs: 12 Total JIRAs Resolved as Fixed: 12 Vote will be open for 72 hours. [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Here's my vote to start: +1 Here's how to verify the tarball hashes: git clone --branch release-2.0.2-rc1 https://git-wip-us.apache.org/repos/asf/ambari.git apache-ambari-2.0.2-src cd apache-ambari-2.0.2-src git clean -xdf cd ambari-web npm install ulimit -n 2048 brunch build rm -rf node_modules mv public public-static cd ../.. tar --exclude=.git --exclude=.gitignore --exclude=.gitattributes -zcvf apache-ambari-2.0.2-src.tar.gz apache-ambari-2.0.2-src # Verify my signature, gpg --verify apache-ambari-2.0.2-src.tar.gz.asc apache-ambari-2.0.2-src.tar.gz gpg: Signature made Fri Jul 31 18:36:49 2015 PDT using RSA key ID 392F57AC gpg: Good signature from Alejandro Fernandez (Apache Ambari Release Management) alejan...@apache.org openssl md5 apache-ambari-2.0.2-src.tar.gz apache-ambari-2.0.2-src.tar.gz.md5# and compare to the file I provided openssl sha1 apache-ambari-2.0.2-src.tar.gz apache-ambari-2.0.2-src.tar.gz.sha1 # and compare to the file I provided # I made sure ratcheck passes cp -R apache-ambari-2.0.2-src apache-ambari-2.0.2-ratcheck cd apache-ambari-2.0.2-ratcheck mvn clean apache-rat:check Thanks, Alejandro Fernandez Apache Ambari PMC
[jira] [Updated] (AMBARI-12179) Tez View should automatically configuration settings based on http/https policy
[ https://issues.apache.org/jira/browse/AMBARI-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-12179: - Attachment: AMBARI-12179.2.patch Tez View should automatically configuration settings based on http/https policy --- Key: AMBARI-12179 URL: https://issues.apache.org/jira/browse/AMBARI-12179 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-12179.1.patch, AMBARI-12179.2.patch The current tez view impl always defaults to the http addresses instead of relying on the http policy setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-12179) Tez View should automatically configuration settings based on http/https policy
[ https://issues.apache.org/jira/browse/AMBARI-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-12179: - Attachment: AMBARI-12179.1.patch Tez View should automatically configuration settings based on http/https policy --- Key: AMBARI-12179 URL: https://issues.apache.org/jira/browse/AMBARI-12179 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-12179.1.patch The current tez view impl always defaults to the http addresses instead of relying on the http policy setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-12179) Tez View should automatically configuration settings based on http/https policy
Hitesh Shah created AMBARI-12179: Summary: Tez View should automatically configuration settings based on http/https policy Key: AMBARI-12179 URL: https://issues.apache.org/jira/browse/AMBARI-12179 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah The current tez view impl always defaults to the http addresses instead of relying on the http policy setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-11551) Additional patch for AMBARI-11473 to address review comment
[ https://issues.apache.org/jira/browse/AMBARI-11551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567960#comment-14567960 ] Hitesh Shah commented on AMBARI-11551: -- [~rlevas] If the patch is good to go, can this be committed to get it into the 2.1.0 release? Additional patch for AMBARI-11473 to address review comment Key: AMBARI-11551 URL: https://issues.apache.org/jira/browse/AMBARI-11551 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial Attachments: AMBARI-11473.addendum.patch To address comment in https://reviews.apache.org/r/34820/ \cc [~rlevas] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-11473) Fix Tez view to surface underlying errors resulting from failure to communicate with timeline
[ https://issues.apache.org/jira/browse/AMBARI-11473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14565201#comment-14565201 ] Hitesh Shah commented on AMBARI-11473: -- Created https://reviews.apache.org/r/34820/ . \cc [~tbeerbower] Fix Tez view to surface underlying errors resulting from failure to communicate with timeline - Key: AMBARI-11473 URL: https://issues.apache.org/jira/browse/AMBARI-11473 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-11473.1.noprefix.patch, AMBARI-11473.1.patch, AMBARI-11473.2.patch, AMBARI-11473.3.patch If there are any errors in communicating with timeline, the Tez view always shows a 404. \cc [~Sreenath] [~pramachandran] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-11473) Fix Tez view to surface underlying errors resulting from failure to communicate with timeline
[ https://issues.apache.org/jira/browse/AMBARI-11473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-11473: - Attachment: AMBARI-11473.3.patch Fix Tez view to surface underlying errors resulting from failure to communicate with timeline - Key: AMBARI-11473 URL: https://issues.apache.org/jira/browse/AMBARI-11473 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-11473.1.noprefix.patch, AMBARI-11473.1.patch, AMBARI-11473.2.patch, AMBARI-11473.3.patch If there are any errors in communicating with timeline, the Tez view always shows a 404. \cc [~Sreenath] [~pramachandran] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-11551) Additional patch for AMBARI-11473 to address review comment
Hitesh Shah created AMBARI-11551: Summary: Additional patch for AMBARI-11473 to address review comment Key: AMBARI-11551 URL: https://issues.apache.org/jira/browse/AMBARI-11551 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial To address comment in https://reviews.apache.org/r/34820/ \cc [~rlevas] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-11551) Additional patch for AMBARI-11473 to address review comment
[ https://issues.apache.org/jira/browse/AMBARI-11551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-11551: - Attachment: AMBARI-11473.addendum.patch [~rlevas] Patch attached for trivial change. Additional patch for AMBARI-11473 to address review comment Key: AMBARI-11551 URL: https://issues.apache.org/jira/browse/AMBARI-11551 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial Attachments: AMBARI-11473.addendum.patch To address comment in https://reviews.apache.org/r/34820/ \cc [~rlevas] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-11551) Additional patch for AMBARI-11473 to address review comment
[ https://issues.apache.org/jira/browse/AMBARI-11551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14565657#comment-14565657 ] Hitesh Shah commented on AMBARI-11551: -- [~rlevas] Done - updated reviewboard. Additional patch for AMBARI-11473 to address review comment Key: AMBARI-11551 URL: https://issues.apache.org/jira/browse/AMBARI-11551 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial Attachments: AMBARI-11473.addendum.patch To address comment in https://reviews.apache.org/r/34820/ \cc [~rlevas] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-11551) Additional patch for AMBARI-11473 to address review comment
[ https://issues.apache.org/jira/browse/AMBARI-11551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14565658#comment-14565658 ] Hitesh Shah commented on AMBARI-11551: -- Did not update the patch there as AMBARI-11473 patch was already committed. Additional patch for AMBARI-11473 to address review comment Key: AMBARI-11551 URL: https://issues.apache.org/jira/browse/AMBARI-11551 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial Attachments: AMBARI-11473.addendum.patch To address comment in https://reviews.apache.org/r/34820/ \cc [~rlevas] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-11473) Fix Tez view to surface underlying errors resulting from failure to communicate with timeline
[ https://issues.apache.org/jira/browse/AMBARI-11473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-11473: - Assignee: Tom Beerbower (was: Hitesh Shah) Fix Tez view to surface underlying errors resulting from failure to communicate with timeline - Key: AMBARI-11473 URL: https://issues.apache.org/jira/browse/AMBARI-11473 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Tom Beerbower Attachments: AMBARI-11473.1.noprefix.patch, AMBARI-11473.1.patch, AMBARI-11473.2.patch If there are any errors in communicating with timeline, the Tez view always shows a 404. \cc [~Sreenath] [~pramachandran] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-11473) Fix Tez view to surface underlying errors resulting from failure to communicate with timeline
Hitesh Shah created AMBARI-11473: Summary: Fix Tez view to surface underlying errors resulting from failure to communicate with timeline Key: AMBARI-11473 URL: https://issues.apache.org/jira/browse/AMBARI-11473 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah If there are any errors in communicating with timeline, the Tez view always shows a 404. \cc [~Sreenath] [~pramachandran] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-11473) Fix Tez view to surface underlying errors resulting from failure to communicate with timeline
[ https://issues.apache.org/jira/browse/AMBARI-11473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-11473: - Attachment: AMBARI-11473.1.patch AMBARI-11473.1.noprefix.patch [~tbeerbower] [~Sreenath] [~pramachandran] please review. Fix Tez view to surface underlying errors resulting from failure to communicate with timeline - Key: AMBARI-11473 URL: https://issues.apache.org/jira/browse/AMBARI-11473 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-11473.1.noprefix.patch, AMBARI-11473.1.patch If there are any errors in communicating with timeline, the Tez view always shows a 404. \cc [~Sreenath] [~pramachandran] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-11473) Fix Tez view to surface underlying errors resulting from failure to communicate with timeline
[ https://issues.apache.org/jira/browse/AMBARI-11473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14562311#comment-14562311 ] Hitesh Shah commented on AMBARI-11473: -- The URLConnectionProvider works for me. Fix Tez view to surface underlying errors resulting from failure to communicate with timeline - Key: AMBARI-11473 URL: https://issues.apache.org/jira/browse/AMBARI-11473 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-11473.1.noprefix.patch, AMBARI-11473.1.patch, AMBARI-11473.2.patch If there are any errors in communicating with timeline, the Tez view always shows a 404. \cc [~Sreenath] [~pramachandran] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-10664) Add new config for tez for HDP 2.3
[ https://issues.apache.org/jira/browse/AMBARI-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509994#comment-14509994 ] Hitesh Shah commented on AMBARI-10664: -- Minor ask: could you also add the following new config: Property name: tez.task.generate.counters.per.io Value: true Description: Whether to generate counters on a per-edge basis for a Tez DAG. Helpful for in-depth analysis. Add new config for tez for HDP 2.3 --- Key: AMBARI-10664 URL: https://issues.apache.org/jira/browse/AMBARI-10664 Project: Ambari Issue Type: Bug Reporter: Andrew Onischuk Assignee: Andrew Onischuk Fix For: 2.1.0 Need to add new config(s) for tez-site specific only to Dal and **should not** be set for previous releases. Configs: tez.runtime.optimize.local.fetch - default true Description: If the shuffle input is on the local host bypass the http fetch and access the files directly Advanced configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: reviews created - 32777, 32779, 32789
Hello David Thanks for reporting these issues. Would you giving your vote[1] on the currenty-open voting thread for the 2.0.0 RC3 release: http://mail-archives.apache.org/mod_mbox/ambari-dev/201504.mbox/%3CD140C400.1B87C%25jluniya%40hortonworks.com%3E thanks — Hitesh [1] https://www.apache.org/foundation/voting.html On Apr 2, 2015, at 1:20 PM, McWhorter, David david_mcwhor...@premierinc.com wrote: Hi all, I’ve created reviews for 3 issues I found testing Ambari 2.0.0-rc2 on ubuntu 12.04: https://reviews.apache.org/r/32780/ addressing https://issues.apache.org/jira/browse/AMBARI-10321 https://reviews.apache.org/r/32779/ addressing https://issues.apache.org/jira/browse/AMBARI-10320 https://reviews.apache.org/r/32777/ addressing https://issues.apache.org/jira/browse/AMBARI-10319 Please let me know if I need to do anything different procedurally to submit these fixes, I’ve tried to follow the instructions at https://cwiki.apache.org/confluence/display/AMBARI/How+to+Contribute. I also found https://issues.apache.org/jira/browse/AMBARI-10317 which I believe Alejandro Fernandez is looking into fixing. All of these issues kept me from either building or deploying components of Ambari 2.0.0-rc2 on ubuntu 12.04, so it may be advisable to include fixes in the 2.0.0 release. Thanks, David — David McWhorter Senior Developer, Foundations Informatics and Technology Services Office: 434.260.5232 | Mobile: 434.227.2551 david_mcwhor...@premierinc.com | Premier, Inc. (NASDAQ: PINC) From: McWhorter, David McWhorter david_mcwhor...@premierinc.commailto:david_mcwhor...@premierinc.com Date: Wednesday, April 1, 2015 at 12:40 PM To: dev@ambari.apache.orgmailto:dev@ambari.apache.org dev@ambari.apache.orgmailto:dev@ambari.apache.org Subject: error on Knox restart - ambari 2.0.0-rc2 Hello, we are testing deploying an HDP 2.2. Cluster using ambari 2.0.0-rc2 running on ubuntu 12.04. I’ve been able to set up a cluster running HDFS, MapReduce2, YARN, Zookeeper, Knox, Ranger, and Ambari Metrics. When I shut down the whole cluster using Actions - Stop All in Ambari, reboot the hosts, and then try to restart the cluster I see the error below restarting the Knox gateway. Can anyone tell me if this is a known error, or if something is obviously misconfigured. If not I’m happy to create an issue and/or take a look at what is going on if someone can point me at where to look. As a note, the directory /var/run/knox is indeed missing on the master host. Thank you, David Knox Gateway startup log: 2015-04-01 16:17:12,075 - Error while executing command 'start': Traceback (most recent call last): File /usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py, line 214, in execute method(env) File /usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py, line 89, in thunk return fn(*args, **kwargs) File /var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py, line 80, in start self.configure(env) File /var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py, line 64, in configure knox() File /usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py, line 89, in thunk return fn(*args, **kwargs) File /var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox.py, line 99, in knox sudo = True, File /usr/lib/python2.6/site-packages/resource_management/core/base.py, line 148, in __init__ self.env.run() File /usr/lib/python2.6/site-packages/resource_management/core/environment.py, line 152, in run self.run_action(resource, action) File /usr/lib/python2.6/site-packages/resource_management/core/environment.py, line 118, in run_action provider_action() File /usr/lib/python2.6/site-packages/resource_management/core/providers/system.py, line 274, in action_run raise ex Fail: Execution of 'chown -R knox:knox /var/lib/knox/data /var/log/knox /var/log/knox /var/run/knox /etc/knox/conf' returned 1. chown: cannot access `/var/run/knox': No such file or directory stdout: /var/lib/ambari-agent/data/output-107.txt 2015-04-01 16:17:06,744 - uGroup['hadoop'] {'ignore_failures': False} 2015-04-01 16:17:06,744 - Modifying group hadoop 2015-04-01 16:17:06,797 - uGroup['users'] {'ignore_failures': False} 2015-04-01 16:17:06,797 - Modifying group users 2015-04-01 16:17:06,839 - uGroup['knox'] {'ignore_failures': False} 2015-04-01 16:17:06,839 - Modifying group knox 2015-04-01 16:17:06,886 - uGroup['ranger'] {'ignore_failures': False} 2015-04-01 16:17:06,886 - Modifying group ranger 2015-04-01 16:17:06,930 - uUser['mapred'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']} 2015-04-01 16:17:06,930 - Modifying user mapred 2015-04-01 16:17:06,976 - uUser['root'] {'gid': 'hadoop', 'ignore_failures': False,
Re: [VOTE] Apache Ambari Release 2.0.0 RC2
-1 (binding) Comments: The pom.xml in https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=log;h=refs/tags/release-2.0.0-rc2 has a snapshot version. The source tarball also has 2.0.0-SNAPSHOT. Various bits of code ( views ) also have snapshot versions. Release contains a DISCLAIMER saying Ambari is still incubating. High time this was fixed. No CHANGES.txt or release notes in the release artifact? Release artifacts should ideally be named apache-ambari-*. Should the artifact be named *-src-* too ? sha file is named “.sha” but actually contains a sha1 sig. The release manager’s keys are not in the KEYS file. For future releases, you should really use dist for staging a release. Easier to have track and have history on what is being voted on. Likewise, the KEYS file should also be available on the download link for users to be able to verify the authenticity of the release. I remember there being a problem the last time around where the api jars were not deployed to nexus and verified as part of the release process. Have those steps been missed again? — Hitesh On Mar 26, 2015, at 1:52 AM, Jayush Luniya jlun...@hortonworks.com wrote: Hi, I have created an ambari-2.0.0 release candidate (RC2). GIT source tag: https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=log;h=refs/tags/release-2.0.0-rc2 Staging site: http://people.apache.org/~jluniya/ambari-2.0.0-rc2/ PGP release keys (signed using CD23CAAE): http://pgp.mit.edu:11371/pks/lookup?search=0xCD23CAAEop=vindex One can look into the issues fixed in this release at https://issues.apache.org/jira/browse/AMBARI/fixforversion/12327486 Total 2.0.0 JIRAs: 1723 Total JIRAs Resolved as Fixed: 1680 Released Feature List: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=30755705 Vote will be open for 72 hours. [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Here's my vote to start: +1 Thanks Jayush Luniya Ambari 2.0.0 Release Manager
[jira] [Updated] (AMBARI-9636) Modifications to configs for Tez and skipping deploy for Tez view
[ https://issues.apache.org/jira/browse/AMBARI-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-9636: Attachment: AMBARI-9636.1.withprefix.patch AMBARI-9636.1.patch Patch attached. [~srimanth.gunturi] [~afernandez] please review Modifications to configs for Tez and skipping deploy for Tez view -- Key: AMBARI-9636 URL: https://issues.apache.org/jira/browse/AMBARI-9636 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-9636.1.patch, AMBARI-9636.1.withprefix.patch There are some modifications need to get Tez to work correctly with the Tez view in Ambari. Also, the tez view artifact should not be deployed to a repository. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-9636) Modifications to configs for Tez and skipping deploy for Tez view
[ https://issues.apache.org/jira/browse/AMBARI-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14320762#comment-14320762 ] Hitesh Shah commented on AMBARI-9636: - Minor change on the way. Modifications to configs for Tez and skipping deploy for Tez view -- Key: AMBARI-9636 URL: https://issues.apache.org/jira/browse/AMBARI-9636 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-9636.1.patch, AMBARI-9636.1.withprefix.patch There are some modifications need to get Tez to work correctly with the Tez view in Ambari. Also, the tez view artifact should not be deployed to a repository. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-9636) Modifications to configs for Tez and skipping deploy for Tez view
[ https://issues.apache.org/jira/browse/AMBARI-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-9636: Attachment: AMBARI-9636.2.withprefix.patch AMBARI-9636.2.patch Modifications to configs for Tez and skipping deploy for Tez view -- Key: AMBARI-9636 URL: https://issues.apache.org/jira/browse/AMBARI-9636 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-9636.1.patch, AMBARI-9636.1.withprefix.patch, AMBARI-9636.2.patch, AMBARI-9636.2.withprefix.patch There are some modifications need to get Tez to work correctly with the Tez view in Ambari. Also, the tez view artifact should not be deployed to a repository. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 30899: Ambari does not upload Tez tarball to HDFS when Tez is installed
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30899/#review72051 --- I do not have the full context on the code so please bear with any howlers in the comments. ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py https://reviews.apache.org/r/30899/#comment117982 What does this line do? What is the relationship between mapreduce and hive-server2? Also, how does the tez.user come into play for mapreduce? ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py https://reviews.apache.org/r/30899/#comment117984 trailing whitespace needs cleanup ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py https://reviews.apache.org/r/30899/#comment117986 Why does the location of the tez tarball depend on the version of hadoop client? - Hitesh Shah On Feb. 11, 2015, 10:42 p.m., Alejandro Fernandez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30899/ --- (Updated Feb. 11, 2015, 10:42 p.m.) Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, Srimanth Gunturi, and Tom Beerbower. Bugs: AMBARI-9585 https://issues.apache.org/jira/browse/AMBARI-9585 Repository: ambari Description --- Today, the tez tarball is copied to HDFS only when HiveServer2 is installed, and when the Pig Service check runs. Instead, this should happen whenever Tez is installed. For a RU, the install of new bits should also copy the tarball to HDFS. Diffs - ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py abfde14 ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py 7137e60 ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 00375d7 ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 1fc3fbf ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py 3d74113 Diff: https://reviews.apache.org/r/30899/diff/ Testing --- Installed a 3-node cluster with HDFS, ZK, MR, YARN, and Tez. Then ensured that the tez tarball was copied to hdfs, su - hdfs -c 'hdfs dfs -ls /hdp/apps/2.2.0.0-2041/tez' When I installed Pig client, on the same host that had the tez client, I was able to run a Pig job, using the following example, http://hortonworks.com/hadoop-tutorial/faster-pig-tez/ I then performed a Rolling Upgrade and the newer tez tarball was indeed copied to HDFS. su - hdfs -c 'hdfs dfs -ls /hdp/apps/2.2.1.0-2260/tez' Unit tests are in progress. Thanks, Alejandro Fernandez
Re: Review Request 30899: Ambari does not upload Tez tarball to HDFS when Tez is installed
On Feb. 11, 2015, 11:36 p.m., Alejandro Fernandez wrote: ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py, line 57 https://reviews.apache.org/r/30899/diff/1/?file=861153#file861153line57 When we copy a tarball to HDFS, we need to figure out the destination, e.g., /hdp/apps/2.2.1.0-2260/tez That version number has to come from somewhere, so we use hdp-select status {component_name}, where component name is the 2nd argument. Hitesh Shah wrote: In that case, why is the version of mapreduce dependent on hive-server2? Also, I am not sure why the mapreduce tarball is being uploaded as user tez if that is what the 3rd param is used for. - Hitesh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30899/#review72057 --- On Feb. 11, 2015, 10:42 p.m., Alejandro Fernandez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30899/ --- (Updated Feb. 11, 2015, 10:42 p.m.) Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, Srimanth Gunturi, and Tom Beerbower. Bugs: AMBARI-9585 https://issues.apache.org/jira/browse/AMBARI-9585 Repository: ambari Description --- Today, the tez tarball is copied to HDFS only when HiveServer2 is installed, and when the Pig Service check runs. Instead, this should happen whenever Tez is installed. For a RU, the install of new bits should also copy the tarball to HDFS. Diffs - ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py abfde14 ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py 7137e60 ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 00375d7 ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 1fc3fbf ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py 3d74113 Diff: https://reviews.apache.org/r/30899/diff/ Testing --- Installed a 3-node cluster with HDFS, ZK, MR, YARN, and Tez. Then ensured that the tez tarball was copied to hdfs, su - hdfs -c 'hdfs dfs -ls /hdp/apps/2.2.0.0-2041/tez' When I installed Pig client, on the same host that had the tez client, I was able to run a Pig job, using the following example, http://hortonworks.com/hadoop-tutorial/faster-pig-tez/ I then performed a Rolling Upgrade and the newer tez tarball was indeed copied to HDFS. su - hdfs -c 'hdfs dfs -ls /hdp/apps/2.2.1.0-2260/tez' Unit tests are in progress. Thanks, Alejandro Fernandez
Re: Review Request 30899: Ambari does not upload Tez tarball to HDFS when Tez is installed
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30899/#review72097 --- Ship it! +1 assuming the comment on the pig client change is correct. ( not tested this change though ). Thanks for the patience in handling all the comments. ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py https://reviews.apache.org/r/30899/#comment118045 This change is fine as long as tez client is installed on same host as pig client. - Hitesh Shah On Feb. 12, 2015, 2:40 a.m., Alejandro Fernandez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30899/ --- (Updated Feb. 12, 2015, 2:40 a.m.) Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, Srimanth Gunturi, and Tom Beerbower. Bugs: AMBARI-9585 https://issues.apache.org/jira/browse/AMBARI-9585 Repository: ambari Description --- Today, the tez tarball is copied to HDFS only when HiveServer2 is installed, and when the Pig Service check runs. Instead, this should happen whenever Tez is installed. For a RU, the install of new bits should also copy the tarball to HDFS. Diffs - ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py 7137e60 ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 00375d7 ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py 3d74113 Diff: https://reviews.apache.org/r/30899/diff/ Testing --- Installed a 3-node cluster with HDFS, ZK, MR, YARN, and Tez. Then ensured that the tez tarball was copied to hdfs, su - hdfs -c 'hdfs dfs -ls /hdp/apps/2.2.0.0-2041/tez' When I installed Pig client, on the same host that had the tez client, I was able to run a Pig job, using the following example, http://hortonworks.com/hadoop-tutorial/faster-pig-tez/ I then performed a Rolling Upgrade and the newer tez tarball was indeed copied to HDFS. su - hdfs -c 'hdfs dfs -ls /hdp/apps/2.2.1.0-2260/tez' Unit tests are in progress. Thanks, Alejandro Fernandez
Re: Review Request 30899: Ambari does not upload Tez tarball to HDFS when Tez is installed
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30899/#review72090 --- Ship it! The changes seems fine in that it looks like it will fix the issue that was raised. However, there seem to be other problems with the existing code that can/should be looked at in a follow-up jira. - Hitesh Shah On Feb. 11, 2015, 10:42 p.m., Alejandro Fernandez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30899/ --- (Updated Feb. 11, 2015, 10:42 p.m.) Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, Srimanth Gunturi, and Tom Beerbower. Bugs: AMBARI-9585 https://issues.apache.org/jira/browse/AMBARI-9585 Repository: ambari Description --- Today, the tez tarball is copied to HDFS only when HiveServer2 is installed, and when the Pig Service check runs. Instead, this should happen whenever Tez is installed. For a RU, the install of new bits should also copy the tarball to HDFS. Diffs - ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py abfde14 ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py 7137e60 ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 00375d7 ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 1fc3fbf ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py 3d74113 Diff: https://reviews.apache.org/r/30899/diff/ Testing --- Installed a 3-node cluster with HDFS, ZK, MR, YARN, and Tez. Then ensured that the tez tarball was copied to hdfs, su - hdfs -c 'hdfs dfs -ls /hdp/apps/2.2.0.0-2041/tez' When I installed Pig client, on the same host that had the tez client, I was able to run a Pig job, using the following example, http://hortonworks.com/hadoop-tutorial/faster-pig-tez/ I then performed a Rolling Upgrade and the newer tez tarball was indeed copied to HDFS. su - hdfs -c 'hdfs dfs -ls /hdp/apps/2.2.1.0-2260/tez' Unit tests are in progress. Thanks, Alejandro Fernandez
[jira] [Updated] (AMBARI-9482) Add new config to tez-site needed to enable tez view in Ambari
[ https://issues.apache.org/jira/browse/AMBARI-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-9482: Attachment: AMBARI-9482.3.patch Same patch without --no-prefix. Add new config to tez-site needed to enable tez view in Ambari --- Key: AMBARI-9482 URL: https://issues.apache.org/jira/browse/AMBARI-9482 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-9482.1.patch, AMBARI-9482.2.patch, AMBARI-9482.3.patch A new config needs to be introduced to tez-site to enable display of configs in the tez view in Ambari. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-9484) Failed to execute goal org.vafer:jdeb:1.0.1:jdeb (default-cli) on project tez-ambari-view
[ https://issues.apache.org/jira/browse/AMBARI-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-9484: Attachment: AMBARI-9484.1.patch Failed to execute goal org.vafer:jdeb:1.0.1:jdeb (default-cli) on project tez-ambari-view - Key: AMBARI-9484 URL: https://issues.apache.org/jira/browse/AMBARI-9484 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-9484.1.patch tez view compile fails to build when using for deb packaging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-9484) Failed to execute goal org.vafer:jdeb:1.0.1:jdeb (default-cli) on project tez-ambari-view
Hitesh Shah created AMBARI-9484: --- Summary: Failed to execute goal org.vafer:jdeb:1.0.1:jdeb (default-cli) on project tez-ambari-view Key: AMBARI-9484 URL: https://issues.apache.org/jira/browse/AMBARI-9484 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah tez view compile fails to build when using for deb packaging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-9482) Add new config to tez-site needed to enable tez view in Ambari
Hitesh Shah created AMBARI-9482: --- Summary: Add new config to tez-site needed to enable tez view in Ambari Key: AMBARI-9482 URL: https://issues.apache.org/jira/browse/AMBARI-9482 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-9482.1.patch A new config needs to be introduced to tez-site to enable display of configs in the tez view in Ambari. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-9482) Add new config to tez-site needed to enable tez view in Ambari
[ https://issues.apache.org/jira/browse/AMBARI-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-9482: Attachment: AMBARI-9482.1.patch Add new config to tez-site needed to enable tez view in Ambari --- Key: AMBARI-9482 URL: https://issues.apache.org/jira/browse/AMBARI-9482 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-9482.1.patch A new config needs to be introduced to tez-site to enable display of configs in the tez view in Ambari. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-9482) Add new config to tez-site needed to enable tez view in Ambari
[ https://issues.apache.org/jira/browse/AMBARI-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-9482: Attachment: AMBARI-9482.2.patch Re-attaching same patch. Add new config to tez-site needed to enable tez view in Ambari --- Key: AMBARI-9482 URL: https://issues.apache.org/jira/browse/AMBARI-9482 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: AMBARI-9482.1.patch, AMBARI-9482.2.patch A new config needs to be introduced to tez-site to enable display of configs in the tez view in Ambari. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-9432) TEZ Ambari view build fixes
[ https://issues.apache.org/jira/browse/AMBARI-9432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-9432: Attachment: AMBARI-9432.1.patch TEZ Ambari view build fixes --- Key: AMBARI-9432 URL: https://issues.apache.org/jira/browse/AMBARI-9432 Project: Ambari Issue Type: Bug Reporter: Sreenath Somarajapuram Attachments: AMBARI-9432.1.patch 1. TEZ Ambari view is not getting build with other views 2. Make build point to TEZ 0.6.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-8760) TEZ UI View
[ https://issues.apache.org/jira/browse/AMBARI-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14260512#comment-14260512 ] Hitesh Shah commented on AMBARI-8760: - [~Sreenath] Looks like the precommit build that applies the patch and tests it ( from the previous commit ) does not like patches that have not been created using --no-prefix TEZ UI View --- Key: AMBARI-8760 URL: https://issues.apache.org/jira/browse/AMBARI-8760 Project: Ambari Issue Type: Bug Reporter: Sreenath Somarajapuram Assignee: Sreenath Somarajapuram Attachments: AMBARI-8760.1.patch, AMBARI-8760.2.patch, AMBARI-8760.3.patch, AMBARI-8760.4.patch Create an Ambari view for TEZ UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-8760) TEZ UI View
[ https://issues.apache.org/jira/browse/AMBARI-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14260515#comment-14260515 ] Hitesh Shah commented on AMBARI-8760: - Also, FWIW, patch no 3 applied for me on ambari trunk without any problems using patch -p0 TEZ UI View --- Key: AMBARI-8760 URL: https://issues.apache.org/jira/browse/AMBARI-8760 Project: Ambari Issue Type: Bug Reporter: Sreenath Somarajapuram Assignee: Sreenath Somarajapuram Attachments: AMBARI-8760.1.patch, AMBARI-8760.2.patch, AMBARI-8760.3.patch, AMBARI-8760.4.patch Create an Ambari view for TEZ UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-7178) Disclaimer file should be removed as Ambari is no longer incubating.
[ https://issues.apache.org/jira/browse/AMBARI-7178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238320#comment-14238320 ] Hitesh Shah commented on AMBARI-7178: - [~swagle] Review please? Disclaimer file should be removed as Ambari is no longer incubating. - Key: AMBARI-7178 URL: https://issues.apache.org/jira/browse/AMBARI-7178 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial Attachments: AMBARI-7178.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-966) Write controller unit tests to verify that config updates are allowed only in certain states like init/installed/start-failed
[ https://issues.apache.org/jira/browse/AMBARI-966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-966: --- Assignee: (was: Hitesh Shah) Write controller unit tests to verify that config updates are allowed only in certain states like init/installed/start-failed - Key: AMBARI-966 URL: https://issues.apache.org/jira/browse/AMBARI-966 Project: Ambari Issue Type: Sub-task Affects Versions: 1.2.0 Reporter: Hitesh Shah Labels: AMBARI-666 Fix For: 1.2.0 Config updates should not be allowed if the object for which configs are being updated is in a started or starting state. This also implies if there is a pending start, config updates should fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Ambari build failure on H4 due to RAT failure
@Yusaku, if you can modify the build job itself, running “git clean -f -d” at the start might be a useful step. — Hitesh On Sep 8, 2014, at 11:26 AM, Yusaku Sako yus...@hortonworks.com wrote: Jun, Thanks. I have yet tried to access Jenkins, so I guess your instructions will be clear to me once I do. I found this: http://wiki.apache.org/general/Jenkins So I will try it out and see. Yusaku On Mon, Sep 8, 2014 at 11:18 AM, jun aoki jun.aoki@gmail.com wrote: Hi Yusaku, thank you for responding. It is a bit tricky but if you have access to the job configuration, you can set 'rm' command in Execute Shell step. Something like {code} rm ${WORKSPACE}/* {code} Then run the job once (make sure it runs on H4) , remove the rm line from the Configuration, then it should run good. On Mon, Sep 8, 2014 at 11:09 AM, Yusaku Sako yus...@hortonworks.com wrote: Roman, Who gets to access these Jenkins servers? And for those who have the access privilege, how do we access them? Yusaku On Mon, Sep 8, 2014 at 11:06 AM, jun aoki jun.aoki@gmail.com wrote: Hi Ambari Commiters I submitted a ticket https://issues.apache.org/jira/browse/AMBARI-7205 It seems like there are some garbage on H4's workspace. Could you please remove the garbage (or possibly the entire git clone) to clean up the RAT failure? I don't have access to the Jenkins slaves. Thank you! -- -jun -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- -jun -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (AMBARI-7178) Disclaimer file should be removed as Ambari is no longer incubating.
Hitesh Shah created AMBARI-7178: --- Summary: Disclaimer file should be removed as Ambari is no longer incubating. Key: AMBARI-7178 URL: https://issues.apache.org/jira/browse/AMBARI-7178 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMBARI-7178) Disclaimer file should be removed as Ambari is no longer incubating.
[ https://issues.apache.org/jira/browse/AMBARI-7178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-7178: Attachment: AMBARI-7178.1.patch Disclaimer file should be removed as Ambari is no longer incubating. - Key: AMBARI-7178 URL: https://issues.apache.org/jira/browse/AMBARI-7178 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial Attachments: AMBARI-7178.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMBARI-4640) Update Tez Configs for integration with YARN Application Timeline Server
Hitesh Shah created AMBARI-4640: --- Summary: Update Tez Configs for integration with YARN Application Timeline Server Key: AMBARI-4640 URL: https://issues.apache.org/jira/browse/AMBARI-4640 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (AMBARI-4640) Update Tez Configs for integration with YARN Application Timeline Server
[ https://issues.apache.org/jira/browse/AMBARI-4640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated AMBARI-4640: Attachment: AMBARI-4640.1.patch Update Tez Configs for integration with YARN Application Timeline Server Key: AMBARI-4640 URL: https://issues.apache.org/jira/browse/AMBARI-4640 Project: Ambari Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial Attachments: AMBARI-4640.1.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)