[jira] [Commented] (AMBARI-25928) Continuous increase of websocket connections in Ambari web UI
[
https://issues.apache.org/jira/browse/AMBARI-25928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844143#comment-17844143
]
caijialiang commented on AMBARI-25928:
--
[~zrain] sorry, i can't assign this issue to you, due to permission problems
> Continuous increase of websocket connections in Ambari web UI
> -
>
> Key: AMBARI-25928
> URL: https://issues.apache.org/jira/browse/AMBARI-25928
> Project: Ambari
> Issue Type: Technical task
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-04-14-14-48-36-956.png,
> image-2023-04-14-14-54-25-386.png
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> How to reproduce: Open Ambari UI page and use Chrome to view the websocket
> connections. At the beginning, there is only one connection. After a few
> minutes, it is observed that the connections keep increasing infinitely. If
> the page is refreshed, the number of connections will return to one, and then
> continue to increase.
>
> {code:java}
> 2023-04-14 14:23:19,062 INFO [MessageBroker-1]
> WebSocketMessageBrokerStats:128 - WebSocketSession[2 current
> WS(2)-HttpStream(0)-HttpPoll(0), 258459 total, 0 closed abnormally (0 connect
> failure, 0 send limit, 171 transport error)], stompSubProtocol[processed
> CONNECT(258377)-CONNECTED(258377)-DISCONNECT(0)], stompBrokerRelay[null],
> inboundChannel[pool size = 4, active threads = 0, queued tasks = 0, completed
> tasks = 2109843], outboundChannel[pool size = 4, active threads = 3, queued
> tasks = 0, completed tasks = 1747340], sockJsScheduler[pool size = 2, active
> threads = 1, queued tasks = 0, completed tasks = 193]
> 2023-04-14 14:23:20,256 INFO [MessageBroker-1]
> WebSocketMessageBrokerStats:128 - WebSocketSession[3 current
> WS(3)-HttpStream(0)-HttpPoll(0), 18 total, 0 closed abnormally (0 connect
> failure, 0 send limit, 0 transport error)], stompSubProtocol[processed
> CONNECT(18)-CONNECTED(18)-DISCONNECT(15)], stompBrokerRelay[null],
> inboundChannel[pool size = 10, active threads = 0, queued tasks = 0,
> completed tasks = 415155], outboundChannel[pool size = 10, active threads =
> 0, queued tasks = 0, completed tasks = 138408], sockJsScheduler[pool size =
> 2, active threads = 1, queued tasks = 0, completed tasks = 193] {code}
>
>
>
> !image-2023-04-14-14-48-36-956.png!
>
> !image-2023-04-14-14-54-25-386.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org
For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-26071) Ambari 3 tasks
[ https://issues.apache.org/jira/browse/AMBARI-26071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840273#comment-17840273 ] caijialiang commented on AMBARI-26071: -- [~vjasani] Could you help discuss and establishing the related roadmap and epics? I have no permissions to do this. > Ambari 3 tasks > -- > > Key: AMBARI-26071 > URL: https://issues.apache.org/jira/browse/AMBARI-26071 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Priority: Major > > This issue primarily lists the content I envision needing for the Ambari 3 > roadmap, and contributions are welcome. > 1. Upgrade Apache Ambari project to Python 3 (Already Done). > 2. Upgrade Apache Ambari Infra project to Python 3 (waiting for review). > 3. Upgrade Apache Ambari Metrics project to Python 3 (waiting for review). > 4. Support for Hadoop Router (in progress) - This needs to be divided into > several tasks. > 5. Support for Ubuntu 22 (in progress) - Requires Bigtop to first support the > feature of custom prefix support in deb packaging (finished testing, waiting > to submit to Bigtop). > 6. Upgrade the Ambari Python unit testing framework. The default mock uses a > very old version of a mock library maintained in the Ambari server code, > which needs to be replaced with the official newer version. > 7. Repairing Ambari's CI/CD. > 8. Upgrading Ambari Frontend's Node.js and outdated frontend libraries - This > part is complex and massive. It needs discussion whether to include it in the > Ambari 3 roadmap and whether there are volunteers to undertake this task, > especially since community activity is relatively low at the moment. > 9. Fixing the bug where the frontend WebSocket leaks cause the page to freeze. > 10. Others - Not thought of yet, open for additions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-26071) Ambari 3 tasks
[ https://issues.apache.org/jira/browse/AMBARI-26071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-26071: - Description: This issue primarily lists the content I envision needing for the Ambari 3 roadmap, and contributions are welcome. 1. Upgrade Apache Ambari project to Python 3 (Already Done). 2. Upgrade Apache Ambari Infra project to Python 3 (waiting for review). 3. Upgrade Apache Ambari Metrics project to Python 3 (waiting for review). 4. Support for Hadoop Router (in progress) - This needs to be divided into several tasks. 5. Support for Ubuntu 22 (in progress) - Requires Bigtop to first support the feature of custom prefix support in deb packaging (finished testing, waiting to submit to Bigtop). 6. Upgrade the Ambari Python unit testing framework. The default mock uses a very old version of a mock library maintained in the Ambari server code, which needs to be replaced with the official newer version. 7. Repairing Ambari's CI/CD. 8. Upgrading Ambari Frontend's Node.js and outdated frontend libraries - This part is complex and massive. It needs discussion whether to include it in the Ambari 3 roadmap and whether there are volunteers to undertake this task, especially since community activity is relatively low at the moment. 9. Fixing the bug where the frontend WebSocket leaks cause the page to freeze. 10. Others - Not thought of yet, open for additions. > Ambari 3 tasks > -- > > Key: AMBARI-26071 > URL: https://issues.apache.org/jira/browse/AMBARI-26071 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Priority: Major > > This issue primarily lists the content I envision needing for the Ambari 3 > roadmap, and contributions are welcome. > 1. Upgrade Apache Ambari project to Python 3 (Already Done). > 2. Upgrade Apache Ambari Infra project to Python 3 (waiting for review). > 3. Upgrade Apache Ambari Metrics project to Python 3 (waiting for review). > 4. Support for Hadoop Router (in progress) - This needs to be divided into > several tasks. > 5. Support for Ubuntu 22 (in progress) - Requires Bigtop to first support the > feature of custom prefix support in deb packaging (finished testing, waiting > to submit to Bigtop). > 6. Upgrade the Ambari Python unit testing framework. The default mock uses a > very old version of a mock library maintained in the Ambari server code, > which needs to be replaced with the official newer version. > 7. Repairing Ambari's CI/CD. > 8. Upgrading Ambari Frontend's Node.js and outdated frontend libraries - This > part is complex and massive. It needs discussion whether to include it in the > Ambari 3 roadmap and whether there are volunteers to undertake this task, > especially since community activity is relatively low at the moment. > 9. Fixing the bug where the frontend WebSocket leaks cause the page to freeze. > 10. Others - Not thought of yet, open for additions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-26068) Fix python3 type error on trunk reader
[ https://issues.apache.org/jira/browse/AMBARI-26068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-26068: Assignee: caijialiang > Fix python3 type error on trunk reader > -- > > Key: AMBARI-26068 > URL: https://issues.apache.org/jira/browse/AMBARI-26068 > Project: Ambari > Issue Type: Bug >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > > During the adaptation process of Ambari for Ubuntu 22, encountered a type > error with Ambari trunk reader while installing deb pkg and > collections.Mapping error -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-26068) Fix python3 type error on trunk reader
[ https://issues.apache.org/jira/browse/AMBARI-26068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-26068: - Description: During the adaptation process of Ambari for Ubuntu 22, encountered a type error with Ambari trunk reader while installing deb pkg and collections.Mapping error > Fix python3 type error on trunk reader > -- > > Key: AMBARI-26068 > URL: https://issues.apache.org/jira/browse/AMBARI-26068 > Project: Ambari > Issue Type: Bug >Reporter: caijialiang >Priority: Major > > During the adaptation process of Ambari for Ubuntu 22, encountered a type > error with Ambari trunk reader while installing deb pkg and > collections.Mapping error -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26068) Fix python3 type error on trunk reader
caijialiang created AMBARI-26068: Summary: Fix python3 type error on trunk reader Key: AMBARI-26068 URL: https://issues.apache.org/jira/browse/AMBARI-26068 Project: Ambari Issue Type: Bug Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25929) Add ranger 2.4 support in ambari bigtop stack
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837076#comment-17837076
]
caijialiang commented on AMBARI-25929:
--
I agree, for future releases, we will maintain only the Python 3 version,
starting from the master branch, transitioning to 3.0 and using the bigtop
stack. Next, we can list some new features or tasks that need support and add
them to the roadmap, then proceed to tackle them one by one until completed.
> Add ranger 2.4 support in ambari bigtop stack
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: Sub-task
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Assignee: caijialiang
>Priority: Major
> Fix For: 2.9.0
>
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png, image-2023-05-09-11-09-14-373.png,
> image-2023-05-09-11-09-22-441.png
>
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> Add ranger 2.4 support in ambari bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces (merged)
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
>
> 2.AMBARI-25932: Wrong config file name in spark service advisor
> ({*}merged{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}merged{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for merge in next version)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (merged)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support {*}({*}{*}merged{*}{*}){*}
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger {*}({*}{*}merged{*}{*}){*}
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> 4.BIGTOP-3950: fix ranger etc conf dir (in review)
> [https://github.com/apache/bigtop/pull/1120]
>
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value
[jira] [Commented] (AMBARI-25929) Add ranger 2.4 support in ambari bigtop stack
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837026#comment-17837026
]
caijialiang commented on AMBARI-25929:
--
[~vjasani] I'm also quite confused, as there currently lacks a release plan and
roadmap for Ambari. It's uncertain whether to release the Python 2 version of
Ambari 2.8 or the Python 3 version of Ambari master.
> Add ranger 2.4 support in ambari bigtop stack
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: Sub-task
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Assignee: caijialiang
>Priority: Major
> Fix For: 2.9.0
>
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png, image-2023-05-09-11-09-14-373.png,
> image-2023-05-09-11-09-22-441.png
>
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> Add ranger 2.4 support in ambari bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces (merged)
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
>
> 2.AMBARI-25932: Wrong config file name in spark service advisor
> ({*}merged{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}merged{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for merge in next version)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (merged)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support {*}({*}{*}merged{*}{*}){*}
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger {*}({*}{*}merged{*}{*}){*}
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> 4.BIGTOP-3950: fix ranger etc conf dir (in review)
> [https://github.com/apache/bigtop/pull/1120]
>
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
>
[jira] [Updated] (AMBARI-26066) Add ambari mpack tools support
[ https://issues.apache.org/jira/browse/AMBARI-26066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-26066: - Description: In day-to-day operations, 99% of changes usually occur under the Ambari Stack rather than the Ambari Server itself. In such circumstances, using MPack to solely update the stack's content becomes highly essential. For instance, when updating the Ambari service code in an already existing environment, like the management code for the HDFS service, there's no need for manual distribution or synchronization, just package it into an MPack for the update. While MPacks are convenient, manually packaging an MPack can be cumbersome, hence a tool is proposed to automate the packaging of existing services into an MPack. > Add ambari mpack tools support > -- > > Key: AMBARI-26066 > URL: https://issues.apache.org/jira/browse/AMBARI-26066 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > > In day-to-day operations, 99% of changes usually occur under the Ambari Stack > rather than the Ambari Server itself. In such circumstances, using MPack to > solely update the stack's content becomes highly essential. For instance, > when updating the Ambari service code in an already existing environment, > like the management code for the HDFS service, there's no need for manual > distribution or synchronization, just package it into an MPack for the > update. While MPacks are convenient, manually packaging an MPack can be > cumbersome, hence a tool is proposed to automate the packaging of existing > services into an MPack. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-26066) Add ambari mpack tools support
[ https://issues.apache.org/jira/browse/AMBARI-26066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-26066: Assignee: caijialiang > Add ambari mpack tools support > -- > > Key: AMBARI-26066 > URL: https://issues.apache.org/jira/browse/AMBARI-26066 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26066) Add ambari mpack tools support
caijialiang created AMBARI-26066: Summary: Add ambari mpack tools support Key: AMBARI-26066 URL: https://issues.apache.org/jira/browse/AMBARI-26066 Project: Ambari Issue Type: New Feature Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25929) Add ranger 2.4 support in ambari bigtop stack
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835162#comment-17835162
]
caijialiang commented on AMBARI-25929:
--
[~bpatel] Ranger-KMS support finished ,already merged to master.
> Add ranger 2.4 support in ambari bigtop stack
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: Sub-task
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Assignee: caijialiang
>Priority: Major
> Fix For: 2.9.0
>
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png, image-2023-05-09-11-09-14-373.png,
> image-2023-05-09-11-09-22-441.png
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> Add ranger 2.4 support in ambari bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces (merged)
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
>
> 2.AMBARI-25932: Wrong config file name in spark service advisor
> ({*}merged{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}merged{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for merge in next version)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (merged)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support {*}({*}{*}merged{*}{*}){*}
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger {*}({*}{*}merged{*}{*}){*}
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> 4.BIGTOP-3950: fix ranger etc conf dir (in review)
> [https://github.com/apache/bigtop/pull/1120]
>
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
> https://issues.apache.org/jira/browse/RANGER-3992 wait for review
> manual test：
> before enable kerberos all compoent works smoonthly
>
[jira] [Updated] (AMBARI-26064) Ambari CI/CD failed
[
https://issues.apache.org/jira/browse/AMBARI-26064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-26064:
-
Description:
According to the logs, it's still a UI failure, unable to find Chrome. It
seems that besides the PhantomJS PR, there was an impact on CI/CD. It appears
that the CI/CD scripts were also modified, causing the failure.
CI/CD configuration:
stage('Parallel Unit Tests') {
parallel {
stage('Ambari WebUI Tests') {
steps {
sh 'lsb_release -a'
sh 'ls /usr/bin'
sh 'find / -name "chromium-browser"'
sh 'export CHROME_BIN=/usr/bin/chromium-browser'
sh 'mvn -X -T 2C -am test -pl ambari-web,ambari-admin
-Dmaven.artifact.threads=10 -Drat.skip'
}
}
stage('Ambari Agent Tests') {
steps {
sh 'pip3 install distro'
sh 'mvn -Dmaven.test.failure.ignore=true -am test -pl
ambari-agent -Dmaven.artifact.threads=10 -Drat.skip'
}
}
In Ubuntu, the installation location of chromium-browser can vary depending on
the method used for installation, such as directly from the Ubuntu repositories
or through a snap package. Common installation locations include:
Installed via apt (Ubuntu repositories):
/usr/bin/chromium-browser
Installed via snap:
/snap/bin/chromium
For modifying the Jenkins script, there are two suggested approaches:
Approach 1: Limit the Search Scope
Since chromium-browser is most likely installed in /usr/bin/ or /snap/bin/ for
snap installations, you can directly check these locations instead of the
entire filesystem to avoid unnecessary permission issues. The modified Jenkins
script could look like this:
stage('Parallel Unit Tests') {
parallel {
stage('Ambari WebUI Tests') {
steps {
sh 'lsb_release -a'
sh 'ls /usr/bin'
// Directly check possible installation locations
sh 'if [ -f /usr/bin/chromium-browser ]; then export
CHROME_BIN=/usr/bin/chromium-browser; fi'
sh 'if [ -f /snap/bin/chromium ]; then export
CHROME_BIN=/snap/bin/chromium; fi'
sh 'mvn -X -T 2C -am test -pl ambari-web,ambari-admin
-Dmaven.artifact.threads=10 -Drat.skip'
}
}
// Other test stages...
}
}
Approach 2: Using sudo
If you need to search the entire filesystem for chromium-browser and you have
permissions to use sudo in Jenkins scripts, you can run the find command with
sudo. However, this is generally not recommended for security reasons, as it
could expose sensitive parts of the system or unnecessarily elevate script
permissions. If you decide to use sudo, ensure the Jenkins user has appropriate
sudo privileges and the sudoers configuration is correct. The modified command
might look like:
sh 'sudo find / -name "chromium-browser" 2>/dev/null'
Here, 2>/dev/null is used to ignore error output, such as permission denied
messages.
Security Tip: In production environments, avoid using sudo for commands that
could affect system security and stability. Approach 1 is a safer, more focused
method and should be considered first.
> Ambari CI/CD failed
> ---
>
> Key: AMBARI-26064
> URL: https://issues.apache.org/jira/browse/AMBARI-26064
> Project: Ambari
> Issue Type: Bug
>Reporter: caijialiang
>Priority: Major
>
> According to the logs, it's still a UI failure, unable to find Chrome. It
> seems that besides the PhantomJS PR, there was an impact on CI/CD. It appears
> that the CI/CD scripts were also modified, causing the failure.
> CI/CD configuration:
> stage('Parallel Unit Tests') {
> parallel {
> stage('Ambari WebUI Tests') {
> steps {
> sh 'lsb_release -a'
> sh 'ls /usr/bin'
> sh 'find / -name "chromium-browser"'
> sh 'export CHROME_BIN=/usr/bin/chromium-browser'
> sh 'mvn -X -T 2C -am test -pl ambari-web,ambari-admin
> -Dmaven.artifact.threads=10 -Drat.skip'
> }
> }
> stage('Ambari Agent Tests') {
> steps {
> sh 'pip3 install distro'
> sh 'mvn -Dmaven.test.failure.ignore=true -am test -pl
> ambari-agent -Dmaven.artifact.threads=10 -Drat.skip'
> }
> }
> In Ubuntu, the installation location of chromium-browser can vary depending
> on the method used for installation, such as directly from the Ubuntu
> repositories or through a snap package. Common installation locations include:
> Installed via apt (Ubuntu repositories):
> /usr/bin/chromium-browser
> Installed via snap:
> /snap/bin/chromium
> For modifying the Jenkins script, there are two suggested approaches:
> Approach 1:
[jira] [Created] (AMBARI-26064) Ambari CI/CD failed
caijialiang created AMBARI-26064: Summary: Ambari CI/CD failed Key: AMBARI-26064 URL: https://issues.apache.org/jira/browse/AMBARI-26064 Project: Ambari Issue Type: Bug Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26058) Add trinodb support
caijialiang created AMBARI-26058: Summary: Add trinodb support Key: AMBARI-26058 URL: https://issues.apache.org/jira/browse/AMBARI-26058 Project: Ambari Issue Type: New Feature Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26057) Add apache knox support
caijialiang created AMBARI-26057: Summary: Add apache knox support Key: AMBARI-26057 URL: https://issues.apache.org/jira/browse/AMBARI-26057 Project: Ambari Issue Type: New Feature Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-26057) Add apache knox support
[ https://issues.apache.org/jira/browse/AMBARI-26057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-26057: Assignee: caijialiang > Add apache knox support > --- > > Key: AMBARI-26057 > URL: https://issues.apache.org/jira/browse/AMBARI-26057 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-26056) Add ranger kms support
[ https://issues.apache.org/jira/browse/AMBARI-26056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-26056: Assignee: caijialiang > Add ranger kms support > -- > > Key: AMBARI-26056 > URL: https://issues.apache.org/jira/browse/AMBARI-26056 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-26055) Add alluxio support
[ https://issues.apache.org/jira/browse/AMBARI-26055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-26055: Assignee: caijialiang > Add alluxio support > --- > > Key: AMBARI-26055 > URL: https://issues.apache.org/jira/browse/AMBARI-26055 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26056) Add ranger kms support
caijialiang created AMBARI-26056: Summary: Add ranger kms support Key: AMBARI-26056 URL: https://issues.apache.org/jira/browse/AMBARI-26056 Project: Ambari Issue Type: New Feature Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26055) Add alluxio support
caijialiang created AMBARI-26055: Summary: Add alluxio support Key: AMBARI-26055 URL: https://issues.apache.org/jira/browse/AMBARI-26055 Project: Ambari Issue Type: New Feature Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25928) Continuous increase of websocket connections in Ambari web UI
[
https://issues.apache.org/jira/browse/AMBARI-25928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820280#comment-17820280
]
caijialiang commented on AMBARI-25928:
--
[~skumar34] no, it's very difficult to solve .
> Continuous increase of websocket connections in Ambari web UI
> -
>
> Key: AMBARI-25928
> URL: https://issues.apache.org/jira/browse/AMBARI-25928
> Project: Ambari
> Issue Type: Technical task
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-04-14-14-48-36-956.png,
> image-2023-04-14-14-54-25-386.png
>
>
> How to reproduce: Open Ambari UI page and use Chrome to view the websocket
> connections. At the beginning, there is only one connection. After a few
> minutes, it is observed that the connections keep increasing infinitely. If
> the page is refreshed, the number of connections will return to one, and then
> continue to increase.
>
> {code:java}
> 2023-04-14 14:23:19,062 INFO [MessageBroker-1]
> WebSocketMessageBrokerStats:128 - WebSocketSession[2 current
> WS(2)-HttpStream(0)-HttpPoll(0), 258459 total, 0 closed abnormally (0 connect
> failure, 0 send limit, 171 transport error)], stompSubProtocol[processed
> CONNECT(258377)-CONNECTED(258377)-DISCONNECT(0)], stompBrokerRelay[null],
> inboundChannel[pool size = 4, active threads = 0, queued tasks = 0, completed
> tasks = 2109843], outboundChannel[pool size = 4, active threads = 3, queued
> tasks = 0, completed tasks = 1747340], sockJsScheduler[pool size = 2, active
> threads = 1, queued tasks = 0, completed tasks = 193]
> 2023-04-14 14:23:20,256 INFO [MessageBroker-1]
> WebSocketMessageBrokerStats:128 - WebSocketSession[3 current
> WS(3)-HttpStream(0)-HttpPoll(0), 18 total, 0 closed abnormally (0 connect
> failure, 0 send limit, 0 transport error)], stompSubProtocol[processed
> CONNECT(18)-CONNECTED(18)-DISCONNECT(15)], stompBrokerRelay[null],
> inboundChannel[pool size = 10, active threads = 0, queued tasks = 0,
> completed tasks = 415155], outboundChannel[pool size = 10, active threads =
> 0, queued tasks = 0, completed tasks = 138408], sockJsScheduler[pool size =
> 2, active threads = 1, queued tasks = 0, completed tasks = 193] {code}
>
>
>
> !image-2023-04-14-14-48-36-956.png!
>
> !image-2023-04-14-14-54-25-386.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org
For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-26049) Fix type error in ambari port alert
[ https://issues.apache.org/jira/browse/AMBARI-26049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-26049: - Description: This PR addresses a bug that was introduced after upgrading from Python 2 to Python 3. The original code was using byte strings for socket communication, which is the default string type in Python 2. However, in Python 3, strings are Unicode by default and need to be explicitly encoded to byte strings before being sent over a socket. Similarly, received data must be decoded back into a Unicode string. The changes in this PR ensure that the socket_command is properly encoded before sending and the received data is decoded back into a string for further processing. This fix restores proper socket communication functionality under Python 3. > Fix type error in ambari port alert > --- > > Key: AMBARI-26049 > URL: https://issues.apache.org/jira/browse/AMBARI-26049 > Project: Ambari > Issue Type: Bug >Reporter: caijialiang >Priority: Major > > This PR addresses a bug that was introduced after upgrading from Python 2 to > Python 3. The original code was using byte strings for socket communication, > which is the default string type in Python 2. However, in Python 3, strings > are Unicode by default and need to be explicitly encoded to byte strings > before being sent over a socket. Similarly, received data must be decoded > back into a Unicode string. The changes in this PR ensure that the > socket_command is properly encoded before sending and the received data is > decoded back into a string for further processing. This fix restores proper > socket communication functionality under Python 3. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26049) Fix type error in ambari port alert
caijialiang created AMBARI-26049: Summary: Fix type error in ambari port alert Key: AMBARI-26049 URL: https://issues.apache.org/jira/browse/AMBARI-26049 Project: Ambari Issue Type: Bug Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25929) Add ranger 2.4 support in ambari bigtop stack
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798847#comment-17798847
]
caijialiang commented on AMBARI-25929:
--
[~bpatel]
If the compatibility issue with Hadoop 3.6 is resolved, I will close the
related issue. After resolution, is there a plan to release a minor version of
Ranger? This would make it much more convenient for users.
> Add ranger 2.4 support in ambari bigtop stack
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: Sub-task
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Assignee: caijialiang
>Priority: Major
> Fix For: 2.9.0
>
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png, image-2023-05-09-11-09-14-373.png,
> image-2023-05-09-11-09-22-441.png
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Add ranger 2.4 support in ambari bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces (merged)
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
>
> 2.AMBARI-25932: Wrong config file name in spark service advisor
> ({*}merged{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}merged{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for merge in next version)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (merged)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support {*}({*}{*}merged{*}{*}){*}
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger {*}({*}{*}merged{*}{*}){*}
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> 4.BIGTOP-3950: fix ranger etc conf dir (in review)
> [https://github.com/apache/bigtop/pull/1120]
>
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
>
[jira] [Commented] (AMBARI-26009) Add Rocky8 operating system support
[ https://issues.apache.org/jira/browse/AMBARI-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795967#comment-17795967 ] caijialiang commented on AMBARI-26009: -- [~wilsonpenha] The Python 3 pull requests for infra solr and metrics are separate: * For metrics: [https://github.com/apache/ambari-metrics/pull/126/files] * For infra solr: [https://github.com/apache/ambari-infra/pull/53] > Add Rocky8 operating system support > --- > > Key: AMBARI-26009 > URL: https://issues.apache.org/jira/browse/AMBARI-26009 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > The PR that needs to be merged for Python3 is > [https://github.com/apache/ambari/pull/3751]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-26009) Add Rocky8 operating system support
[ https://issues.apache.org/jira/browse/AMBARI-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794896#comment-17794896 ] caijialiang commented on AMBARI-26009: -- [~wilsonpenha] Before you start ambari-server, use "ambari-server setup" to configure the running user for Ambari. This step requires sudo or root access. Ambari will automatically set the relevant permissions for all directories to the corresponding running user. > Add Rocky8 operating system support > --- > > Key: AMBARI-26009 > URL: https://issues.apache.org/jira/browse/AMBARI-26009 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > The PR that needs to be merged for Python3 is > [https://github.com/apache/ambari/pull/3751]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-26009) Add Rocky8 operating system support
[ https://issues.apache.org/jira/browse/AMBARI-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-26009: - Description: The PR that needs to be merged for Python3 is [https://github.com/apache/ambari/pull/3751]. > Add Rocky8 operating system support > --- > > Key: AMBARI-26009 > URL: https://issues.apache.org/jira/browse/AMBARI-26009 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > The PR that needs to be merged for Python3 is > [https://github.com/apache/ambari/pull/3751]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-26009) Add Rocky8 operating system support
[ https://issues.apache.org/jira/browse/AMBARI-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794483#comment-17794483 ] caijialiang commented on AMBARI-26009: -- [~wilsonpenha] I apologize for the unclear description of the PR. This PR is the adaptation I made based on Python3. The PR that needs to be merged for Python3 is [https://github.com/apache/ambari/pull/3751]. > Add Rocky8 operating system support > --- > > Key: AMBARI-26009 > URL: https://issues.apache.org/jira/browse/AMBARI-26009 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-26021) Upgrade Ambari Infra to python3
[ https://issues.apache.org/jira/browse/AMBARI-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-26021: - Summary: Upgrade Ambari Infra to python3 (was: upgrade Ambari Infra to python3) > Upgrade Ambari Infra to python3 > --- > > Key: AMBARI-26021 > URL: https://issues.apache.org/jira/browse/AMBARI-26021 > Project: Ambari > Issue Type: Improvement >Reporter: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-26020) upgrade Ambari Metrics to python3
[ https://issues.apache.org/jira/browse/AMBARI-26020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-26020: Assignee: caijialiang > upgrade Ambari Metrics to python3 > - > > Key: AMBARI-26020 > URL: https://issues.apache.org/jira/browse/AMBARI-26020 > Project: Ambari > Issue Type: Improvement >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26020) upgrade Ambari Metrics to python3
caijialiang created AMBARI-26020: Summary: upgrade Ambari Metrics to python3 Key: AMBARI-26020 URL: https://issues.apache.org/jira/browse/AMBARI-26020 Project: Ambari Issue Type: Improvement Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-26009) Add Rocky8 operating system support
[ https://issues.apache.org/jira/browse/AMBARI-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773875#comment-17773875 ] caijialiang commented on AMBARI-26009: -- [~wzhy] Great suggestion, we can support many operating systems, all those supported by Apache Bigtop can be supported by us too. However, we need to merge Python3 first, as many operating systems have dropped support for Python2. Both of these PRs of mine are based on Python3. And you're also welcome to contribute, to support and test more operating systems. > Add Rocky8 operating system support > --- > > Key: AMBARI-26009 > URL: https://issues.apache.org/jira/browse/AMBARI-26009 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-26010) Add Ubuntu 22 operating system support
[ https://issues.apache.org/jira/browse/AMBARI-26010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-26010: Assignee: caijialiang > Add Ubuntu 22 operating system support > -- > > Key: AMBARI-26010 > URL: https://issues.apache.org/jira/browse/AMBARI-26010 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26010) Add Ubuntu 22 operating system support
caijialiang created AMBARI-26010: Summary: Add Ubuntu 22 operating system support Key: AMBARI-26010 URL: https://issues.apache.org/jira/browse/AMBARI-26010 Project: Ambari Issue Type: New Feature Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-26009) Add Rocky8 operating system support
[ https://issues.apache.org/jira/browse/AMBARI-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-26009: Assignee: caijialiang > Add Rocky8 operating system support > --- > > Key: AMBARI-26009 > URL: https://issues.apache.org/jira/browse/AMBARI-26009 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-26008) Add openEuler operating system support
[ https://issues.apache.org/jira/browse/AMBARI-26008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-26008: Assignee: caijialiang > Add openEuler operating system support > -- > > Key: AMBARI-26008 > URL: https://issues.apache.org/jira/browse/AMBARI-26008 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26009) Add Rocky8 operating system support
caijialiang created AMBARI-26009: Summary: Add Rocky8 operating system support Key: AMBARI-26009 URL: https://issues.apache.org/jira/browse/AMBARI-26009 Project: Ambari Issue Type: New Feature Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26008) Add openEuler operating system support
caijialiang created AMBARI-26008: Summary: Add openEuler operating system support Key: AMBARI-26008 URL: https://issues.apache.org/jira/browse/AMBARI-26008 Project: Ambari Issue Type: New Feature Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-26001) ambari2.8 release
[ https://issues.apache.org/jira/browse/AMBARI-26001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-26001: Assignee: caijialiang > ambari2.8 release > - > > Key: AMBARI-26001 > URL: https://issues.apache.org/jira/browse/AMBARI-26001 > Project: Ambari > Issue Type: Task >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-26001) ambari2.8 release
[ https://issues.apache.org/jira/browse/AMBARI-26001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773499#comment-17773499 ] caijialiang commented on AMBARI-26001: -- ok > ambari2.8 release > - > > Key: AMBARI-26001 > URL: https://issues.apache.org/jira/browse/AMBARI-26001 > Project: Ambari > Issue Type: Task >Reporter: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-26000) Ambari supports Python3 environment
[
https://issues.apache.org/jira/browse/AMBARI-26000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-26000:
-
Description:
This patch is about upgrade python3 on trunk branch.
All unit tests are passed on python3 version.
We also deploy it in the production environment(Centos7) and it works fine.
Upgrade Instructions:
All major changes related to this upgrade are documented in the MD file of this
issue. Colleagues who need to review and understand the upgrade changes can
download and review it.
1. Required Dependencies:
After upgrading to Python 3, Ambari requires the 'distro' dependency. To
install it, run:
```
pip install distro
```
If you are using CentOS 7, manually install the following dependencies:
```
yum install python3 python3-libs python36-distro python36-rpm
```
2. Compilation and Testing:
To compile and package, use the following Maven command:
```
mvn -B clean install package rpm:rpm -Dbuild-rpm -Drat.skip=true -DskipTests
-Dmaven.test.skip=true
```
To run unit tests for Ambari server, use:
```
mvn -am test -pl ambari-server -DskipSurefireTests
-Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip
-Dcheckstyle.skip
```
To run unit tests for Ambari agent, use:
```
mvn -am test -pl ambari-agent -DskipSurefireTests
-Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip
-Dcheckstyle.skip
```
Note: Before compiling, ensure that you create a symbolic link from
'python3' to 'python':
```
rm -f /usr/bin/python
ln -s /usr/bin/python-3.x /usr/bin/python
```
This is necessary because the rpm-maven-plugin used by Ambari for RPM
packaging defaults to using Python 2, and it cannot be configured to use Python
3. This symbolic link change may affect the use of 'yum.' To address this,
modify the first line in the following files on CentOS 7:
```
/usr/libexec/urlgrabber-ext-down
/usr/bin/yum
```
Change
{code:java}
#!/usr/bin/python{code}
to
{code:java}
#!/usr/bin/python2 {code}
3. Testing:
All unit tests have passed, and we have tested manual installation and
blueprint testing in the production environment, including common component
operations.
4. How to Test Installation:
Currently, the trunk supports testing installation on CentOS 7. Pre-install
the Python 3 dependencies on CentOS 7:
```
yum install python3 python3-libs python36-distro python36-rpm
```
The installation steps are the same as before.
Installation Note: Ambari trunk defaults to using the Apache Bigtop stack
during installation, which relies on the 'bigtop-select' tool. This tool
currently defaults to Python 2, but we have updated it to Python 3 for testing
purposes. Below are the Ambari RPMs required for trunk testing. All Big Data
component RPMs (packaged using Apache Bigtop) are available for CentOS 7 since
trunk currently only supports CentOS 7. Download links are provided below. We
plan to support more operating systems after upgrading to Python 3, which will
make multi-OS support easier.
was:
This patch is about upgrade python3 on trunk branch.
All unit tests are passed on python3 version.
We also deploy it in the production environment(Centos7) and it works fine.
Upgrade Instructions:
All major changes related to this upgrade are documented in the MD file of this
issue. Colleagues who need to review and understand the upgrade changes can
download and review it.
1. Required Dependencies:
After upgrading to Python 3, Ambari requires the 'distro' dependency. To
install it, run:
```
pip install distro
```
If you are using CentOS 7, manually install the following dependencies:
```
yum install python3 python3-libs python36-distro python36-rpm
```
2. Compilation and Testing:
To compile and package, use the following Maven command:
```
mvn -B clean install package rpm:rpm -Dbuild-rpm -Drat.skip=true -DskipTests
-Dmaven.test.skip=true
```
To run unit tests for Ambari server, use:
```
mvn -am test -pl ambari-server -DskipSurefireTests
-Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip
-Dcheckstyle.skip
```
To run unit tests for Ambari agent, use:
```
mvn -am test -pl ambari-agent -DskipSurefireTests
-Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip
-Dcheckstyle.skip
```
Note: Before compiling, ensure that you create a symbolic link from
'python3' to 'python':
```
rm -f /usr/bin/python
ln -s /usr/bin/python-3.x /usr/bin/python
```
This is necessary because the rpm-maven-plugin used by Ambari for RPM
packaging defaults to using Python 2, and it cannot be configured to use Python
3. This symbolic link change may affect the use of 'yum.' To address this,
modify the first line in the following files on CentOS 7:
```
/usr/libexec/urlgrabber-ext-down
[jira] [Updated] (AMBARI-26000) Ambari supports Python3 environment
[
https://issues.apache.org/jira/browse/AMBARI-26000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-26000:
-
Description:
This patch is about upgrade python3 on trunk branch.
All unit tests are passed on python3 version.
We also deploy it in the production environment(Centos7) and it works fine.
Upgrade Instructions:
All major changes related to this upgrade are documented in the MD file of this
issue. Colleagues who need to review and understand the upgrade changes can
download and review it.
1. Required Dependencies:
After upgrading to Python 3, Ambari requires the 'distro' dependency. To
install it, run:
```
pip install distro
```
If you are using CentOS 7, manually install the following dependencies:
```
yum install python3 python3-libs python36-distro python36-rpm
```
2. Compilation and Testing:
To compile and package, use the following Maven command:
```
mvn -B clean install package rpm:rpm -Dbuild-rpm -Drat.skip=true -DskipTests
-Dmaven.test.skip=true
```
To run unit tests for Ambari server, use:
```
mvn -am test -pl ambari-server -DskipSurefireTests
-Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip
-Dcheckstyle.skip
```
To run unit tests for Ambari agent, use:
```
mvn -am test -pl ambari-agent -DskipSurefireTests
-Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip
-Dcheckstyle.skip
```
Note: Before compiling, ensure that you create a symbolic link from
'python3' to 'python':
```
rm -f /usr/bin/python
ln -s /usr/bin/python-3.x /usr/bin/python
```
This is necessary because the rpm-maven-plugin used by Ambari for RPM
packaging defaults to using Python 2, and it cannot be configured to use Python
3. This symbolic link change may affect the use of 'yum.' To address this,
modify the first line in the following files on CentOS 7:
```
/usr/libexec/urlgrabber-ext-down
/usr/bin/yum
Change "#!/usr/bin/python" to "#!/usr/bin/python2"
```
3. Testing:
All unit tests have passed, and we have tested manual installation and
blueprint testing in the production environment, including common component
operations.
4. How to Test Installation:
Currently, the trunk supports testing installation on CentOS 7. Pre-install
the Python 3 dependencies on CentOS 7:
```
yum install python3 python3-libs python36-distro python36-rpm
```
The installation steps are the same as before.
Installation Note: Ambari trunk defaults to using the Apache Bigtop stack
during installation, which relies on the 'bigtop-select' tool. This tool
currently defaults to Python 2, but we have updated it to Python 3 for testing
purposes. Below are the Ambari RPMs required for trunk testing. All Big Data
component RPMs (packaged using Apache Bigtop) are available for CentOS 7 since
trunk currently only supports CentOS 7. Download links are provided below. We
plan to support more operating systems after upgrading to Python 3, which will
make multi-OS support easier.
was:
This patch is about upgrade python3 on trunk branch.
All unit tests are passed on python3 version.
We also deploy it in the production environment(Centos7) and it works fine.
Upgrade Instructions:
All major changes related to this upgrade are documented in the MD file of this
issue. Colleagues who need to review and understand the upgrade changes can
download and review it.
1. Required Dependencies:
After upgrading to Python 3, Ambari requires the 'distro' dependency. To
install it, run:
{code:java}
pip install distro {code}
If you are using CentOS 7, manually install the following dependencies:
{code:java}
yum install python3 python3-libs python36-distro python36-rpm {code}
2. Compilation and Testing:
To compile and package, use the following Maven command:
{code:java}
mvn -B clean install package rpm:rpm -Dbuild-rpm -Drat.skip=true -DskipTests
-Dmaven.test.skip=true {code}
To run unit tests for Ambari server, use:
{code:java}
mvn -am test -pl ambari-server -DskipSurefireTests -Dmaven.test.failure.ignore
-Dmaven.artifact.threads=10 -Drat.skip -Dcheckstyle.skip {code}
To run unit tests for Ambari agent, use:
{code:java}
mvn -am test -pl ambari-agent -DskipSurefireTests -Dmaven.test.failure.ignore
-Dmaven.artifact.threads=10 -Drat.skip -Dcheckstyle.skip{code}
Note: Before compiling, ensure that you create a symbolic link from
'python3' to 'python':
{code:java}
rm -f /usr/bin/python
ln -s /usr/bin/python-3.x /usr/bin/python{code}
This is necessary because the rpm-maven-plugin used by Ambari for RPM
packaging defaults to using Python 2, and it cannot be configured to use Python
3. This symbolic link change may affect the use of 'yum.' To address this,
modify the first line in the following files on CentOS 7:
{code:java}
/usr/libexec/urlgrabber-ext-down
/usr/bin/yum
Change
[jira] [Updated] (AMBARI-26000) Ambari supports Python3 environment
[ https://issues.apache.org/jira/browse/AMBARI-26000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-26000: - Description: This patch is about upgrade python3 on trunk branch. All unit tests are passed on python3 version. We also deploy it in the production environment(Centos7) and it works fine. Upgrade Instructions: All major changes related to this upgrade are documented in the MD file of this issue. Colleagues who need to review and understand the upgrade changes can download and review it. 1. Required Dependencies: After upgrading to Python 3, Ambari requires the 'distro' dependency. To install it, run: ``` pip install distro ``` If you are using CentOS 7, manually install the following dependencies: ``` yum install python3 python3-libs python36-distro python36-rpm ``` 2. Compilation and Testing: To compile and package, use the following Maven command: ``` mvn -B clean install package rpm:rpm -Dbuild-rpm -Drat.skip=true -DskipTests -Dmaven.test.skip=true ``` To run unit tests for Ambari server, use: ``` mvn -am test -pl ambari-server -DskipSurefireTests -Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip -Dcheckstyle.skip ``` To run unit tests for Ambari agent, use: ``` mvn -am test -pl ambari-agent -DskipSurefireTests -Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip -Dcheckstyle.skip ``` Note: Before compiling, ensure that you create a symbolic link from 'python3' to 'python': ``` rm -f /usr/bin/python ln -s /usr/bin/python-3.x /usr/bin/python ``` This is necessary because the rpm-maven-plugin used by Ambari for RPM packaging defaults to using Python 2, and it cannot be configured to use Python 3. This symbolic link change may affect the use of 'yum.' To address this, modify the first line in the following files on CentOS 7: ``` /usr/libexec/urlgrabber-ext-down /usr/bin/yum Change '#!/usr/bin/python' to '#!/usr/bin/python2' ``` 3. Testing: All unit tests have passed, and we have tested manual installation and blueprint testing in the production environment, including common component operations. 4. How to Test Installation: Currently, the trunk supports testing installation on CentOS 7. Pre-install the Python 3 dependencies on CentOS 7: ``` yum install python3 python3-libs python36-distro python36-rpm ``` The installation steps are the same as before. Installation Note: Ambari trunk defaults to using the Apache Bigtop stack during installation, which relies on the 'bigtop-select' tool. This tool currently defaults to Python 2, but we have updated it to Python 3 for testing purposes. Below are the Ambari RPMs required for trunk testing. All Big Data component RPMs (packaged using Apache Bigtop) are available for CentOS 7 since trunk currently only supports CentOS 7. Download links are provided below. We plan to support more operating systems after upgrading to Python 3, which will make multi-OS support easier. was: This patch is about upgrade python3 on trunk branch. All unit tests are passed on python3 version. We also deploy it in the production environment(Centos7) and it works fine. More changes details can be found in the attachment "Ambari support python3.md" > Ambari supports Python3 environment > --- > > Key: AMBARI-26000 > URL: https://issues.apache.org/jira/browse/AMBARI-26000 > Project: Ambari > Issue Type: New Feature >Reporter: wenweijian >Priority: Major > Attachments: Ambari support python3.md > > > This patch is about upgrade python3 on trunk branch. > All unit tests are passed on python3 version. > We also deploy it in the production environment(Centos7) and it works fine. > Upgrade Instructions: > All major changes related to this upgrade are documented in the MD file of > this issue. Colleagues who need to review and understand the upgrade changes > can download and review it. > 1. Required Dependencies: > After upgrading to Python 3, Ambari requires the 'distro' dependency. To > install it, run: > ``` > pip install distro > ``` > If you are using CentOS 7, manually install the following dependencies: > ``` > yum install python3 python3-libs python36-distro python36-rpm > ``` > 2. Compilation and Testing: > To compile and package, use the following Maven command: > ``` > mvn -B clean install package rpm:rpm -Dbuild-rpm -Drat.skip=true > -DskipTests -Dmaven.test.skip=true > ``` > To run unit tests for Ambari server, use: > ``` > mvn -am test -pl ambari-server -DskipSurefireTests > -Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip > -Dcheckstyle.skip > ``` > To run unit tests for Ambari agent, use: > ``` > mvn -am
[jira] [Commented] (AMBARI-26001) ambari2.8 release
[ https://issues.apache.org/jira/browse/AMBARI-26001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761664#comment-17761664 ] caijialiang commented on AMBARI-26001: -- [~brahmareddy] This is currently an issue in branch 2.8, and I need your assistance to cherry-pick it into branch 2.8. Once you have completed it, please let me know, and I will proceed with packaging, installation, and testing. After I have finished testing, please update the Fix Version for these issues. > ambari2.8 release > - > > Key: AMBARI-26001 > URL: https://issues.apache.org/jira/browse/AMBARI-26001 > Project: Ambari > Issue Type: Task >Reporter: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-26001) ambari2.8 release
caijialiang created AMBARI-26001: Summary: ambari2.8 release Key: AMBARI-26001 URL: https://issues.apache.org/jira/browse/AMBARI-26001 Project: Ambari Issue Type: Task Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-25997) Improvement and Bug Fix in ambari_server.properties.py
[
https://issues.apache.org/jira/browse/AMBARI-25997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang reassigned AMBARI-25997:
Assignee: caijialiang
> Improvement and Bug Fix in ambari_server.properties.py
> --
>
> Key: AMBARI-25997
> URL: https://issues.apache.org/jira/browse/AMBARI-25997
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.8.0, 2.7.7
>Reporter: caijialiang
>Assignee: caijialiang
>Priority: Major
>
> File Location: ambari-server/src/main/python/ambari_server/properties.py
> Problem Description:
> 1. The `__getattr__` method is used for dynamic proxying of methods. In the
> case where `hasattr(self._props, name)` is false indicating the corresponding
> method or attribute cannot be found, an exception needs to be thrown. If not,
> it may confuse the user and obstruct code debugging.
> 2. The class `Properties` proxies the `has_key` method to the `dict` object
> through `__getattr__`, which is acceptable: `Properties().has_key("x")`.
> However, for the operation `"x" in Properties()`, it ends up in an infinite
> hang, never returning. Therefore, a `__contains__` method needs to be
> implemented to keep consistency with the logic of `has_key`, i.e., to
> determine whether the object is in `_props`:
> ```python
> def __contains__(self, key):
> return key in self._props
> ```
> This modification prevents the infinite hang problem from occurring.
> To reproduce the hang issue, we can make a simplified version of the
> `Properties` class for testing, as follows:
> ```python
> #!/usr/bin/env python
> import os
> import re
> import time
> class Properties(object):
> def __init__(self, props=None):
> self._props = {}
> self._origprops = {}
> self._keymap = {}
> def get_property(self, key):
> return self._props.get(key, '')
> def propertyNames(self):
> return self._props.keys()
> def getPropertyDict(self):
> return self._props
> def __getitem__(self, name):
> return self.get_property(name)
> def __getattr__(self, name):
> try:
> return self.__dict__[name]
> except KeyError:
> if hasattr(self._props, name):
> return getattr(self._props, name)
> else:
> raise NotImplementedError("The method '{}' is not
> implemented.".format(name))
> def __contains__(self, key):
> return key in self._props
> if __name__ == "__main__":
> p = Properties()
> "a" in p
> ```
> Upon running this program with either Python 3 or Python 2 python, it will
> indefinitely hang, never returning.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org
For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-25997) Improvement and Bug Fix in ambari_server.properties.py
caijialiang created AMBARI-25997:
Summary: Improvement and Bug Fix in ambari_server.properties.py
Key: AMBARI-25997
URL: https://issues.apache.org/jira/browse/AMBARI-25997
Project: Ambari
Issue Type: Bug
Affects Versions: 2.7.7, 2.8.0
Reporter: caijialiang
File Location: ambari-server/src/main/python/ambari_server/properties.py
Problem Description:
1. The `__getattr__` method is used for dynamic proxying of methods. In the
case where `hasattr(self._props, name)` is false indicating the corresponding
method or attribute cannot be found, an exception needs to be thrown. If not,
it may confuse the user and obstruct code debugging.
2. The class `Properties` proxies the `has_key` method to the `dict` object
through `__getattr__`, which is acceptable: `Properties().has_key("x")`.
However, for the operation `"x" in Properties()`, it ends up in an infinite
hang, never returning. Therefore, a `__contains__` method needs to be
implemented to keep consistency with the logic of `has_key`, i.e., to determine
whether the object is in `_props`:
```python
def __contains__(self, key):
return key in self._props
```
This modification prevents the infinite hang problem from occurring.
To reproduce the hang issue, we can make a simplified version of the
`Properties` class for testing, as follows:
```python
#!/usr/bin/env python
import os
import re
import time
class Properties(object):
def __init__(self, props=None):
self._props = {}
self._origprops = {}
self._keymap = {}
def get_property(self, key):
return self._props.get(key, '')
def propertyNames(self):
return self._props.keys()
def getPropertyDict(self):
return self._props
def __getitem__(self, name):
return self.get_property(name)
def __getattr__(self, name):
try:
return self.__dict__[name]
except KeyError:
if hasattr(self._props, name):
return getattr(self._props, name)
else:
raise NotImplementedError("The method '{}' is not
implemented.".format(name))
def __contains__(self, key):
return key in self._props
if __name__ == "__main__":
p = Properties()
"a" in p
```
Upon running this program with either Python 3 or Python 2 python, it will
indefinitely hang, never returning.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org
For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Comment Edited] (AMBARI-25986) Upgrade Ambari hadoop dependency version
[ https://issues.apache.org/jira/browse/AMBARI-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755753#comment-17755753 ] caijialiang edited comment on AMBARI-25986 at 8/18/23 1:52 AM: --- [~bpatel] [~vjasani] Great advice indeed! Hadoop 3.3.5 has fixed many CVE vulnerabilities. Therefore, the main content of the Apache BigTop 3.2.1 release is the upgrade from Hadoop 3.3.4 to Hadoop 3.3.5! Since our Hadoop version is tied to BigTop, currently, the only Hadoop version available in Ambari for BigTop 3.2.0 stack is Hadoop 3.3.4. Therefore, we use 3.3.4 for the release of Ambari 2.8, specifically targeting BigTop stack 3.2.0. In the upcoming BigTop 3.3, the Hadoop version will also be 3.3.5. However, the development of Ambiar BigTop stack 3.3 is currently in progress. Once the development of BigTop stack 3.3 is complete, we will upgrade to Hadoop 3.3.5. was (Author: JIRAUSER298676): [~bpatel] [~vjasani] Great advice indeed! Hadoop 3.3.5 has fixed many CBVE vulnerabilities. Therefore, the main content of the Apache BigTop 3.2.1 release is the upgrade from Hadoop 3.3.4 to Hadoop 3.3.5! Since our Hadoop version is tied to BigTop, currently, the only Hadoop version available in Ambari for BigTop 3.2.0 stack is Hadoop 3.3.4. Therefore, we use 3.3.4 for the release of Ambari 2.8, specifically targeting BigTop stack 3.2.0. In the upcoming BigTop 3.3, the Hadoop version will also be 3.3.5. However, the development of Ambiar BigTop stack 3.3 is currently in progress. Once the development of BigTop stack 3.3 is complete, we will upgrade to Hadoop 3.3.5. > Upgrade Ambari hadoop dependency version > - > > Key: AMBARI-25986 > URL: https://issues.apache.org/jira/browse/AMBARI-25986 > Project: Ambari > Issue Type: Bug >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Fix For: 2.9.0 > > Attachments: image-2023-08-14-17-54-52-257.png > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Due to Ambari 2.8's Bigtop stack, the default version of Hadoop is 3.3.4. > Starting from Hadoop 3.3, Jetty was upgraded to version 9.4. However, certain > parts of Ambari still retain Hadoop versions lower than 3.3, which use Jetty > versions below 9.4. This leads to an API incompatibility issue, causing > failures in Ambari's contributed tests that utilize the HDFS MiniCluster due > to conflicting Jetty versions. > Furthermore, the version of Hadoop should also align with the current default > stack's Hadoop version to ensure consistency. > !image-2023-08-14-17-54-52-257.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25986) Upgrade Ambari hadoop dependency version
[ https://issues.apache.org/jira/browse/AMBARI-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755753#comment-17755753 ] caijialiang commented on AMBARI-25986: -- [~bpatel] [~vjasani] Great advice indeed! Hadoop 3.3.5 has fixed many CBVE vulnerabilities. Therefore, the main content of the Apache BigTop 3.2.1 release is the upgrade from Hadoop 3.3.4 to Hadoop 3.3.5! Since our Hadoop version is tied to BigTop, currently, the only Hadoop version available in Ambari for BigTop 3.2.0 stack is Hadoop 3.3.4. Therefore, we use 3.3.4 for the release of Ambari 2.8, specifically targeting BigTop stack 3.2.0. In the upcoming BigTop 3.3, the Hadoop version will also be 3.3.5. However, the development of Ambiar BigTop stack 3.3 is currently in progress. Once the development of BigTop stack 3.3 is complete, we will upgrade to Hadoop 3.3.5. > Upgrade Ambari hadoop dependency version > - > > Key: AMBARI-25986 > URL: https://issues.apache.org/jira/browse/AMBARI-25986 > Project: Ambari > Issue Type: Bug >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Fix For: 2.9.0 > > Attachments: image-2023-08-14-17-54-52-257.png > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Due to Ambari 2.8's Bigtop stack, the default version of Hadoop is 3.3.4. > Starting from Hadoop 3.3, Jetty was upgraded to version 9.4. However, certain > parts of Ambari still retain Hadoop versions lower than 3.3, which use Jetty > versions below 9.4. This leads to an API incompatibility issue, causing > failures in Ambari's contributed tests that utilize the HDFS MiniCluster due > to conflicting Jetty versions. > Furthermore, the version of Hadoop should also align with the current default > stack's Hadoop version to ensure consistency. > !image-2023-08-14-17-54-52-257.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-25989) Remove not supported modules from views
[ https://issues.apache.org/jira/browse/AMBARI-25989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-25989: - Attachment: image-2023-08-17-10-35-19-845.png > Remove not supported modules from views > --- > > Key: AMBARI-25989 > URL: https://issues.apache.org/jira/browse/AMBARI-25989 > Project: Ambari > Issue Type: Bug >Reporter: caijialiang >Priority: Major > Attachments: image-2023-08-17-10-35-19-845.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-25989) Remove not supported modules from views
[
https://issues.apache.org/jira/browse/AMBARI-25989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25989:
-
Description:
Comment out the deprecated {{pig}} and {{wfmanager}} modules, as these two
modules are not supported by stacks after Ambari 2.8. Additionally, this also
impacts the compilation of Ambari views.
!image-2023-08-17-10-35-19-845.png!
> Remove not supported modules from views
> ---
>
> Key: AMBARI-25989
> URL: https://issues.apache.org/jira/browse/AMBARI-25989
> Project: Ambari
> Issue Type: Bug
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-08-17-10-35-19-845.png
>
>
> Comment out the deprecated {{pig}} and {{wfmanager}} modules, as these two
> modules are not supported by stacks after Ambari 2.8. Additionally, this also
> impacts the compilation of Ambari views.
> !image-2023-08-17-10-35-19-845.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org
For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-25989) Remove not supported modules from views
caijialiang created AMBARI-25989: Summary: Remove not supported modules from views Key: AMBARI-25989 URL: https://issues.apache.org/jira/browse/AMBARI-25989 Project: Ambari Issue Type: Bug Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-25988) Fix outdated ember-collection dependency
caijialiang created AMBARI-25988:
Summary: Fix outdated ember-collection dependency
Key: AMBARI-25988
URL: https://issues.apache.org/jira/browse/AMBARI-25988
Project: Ambari
Issue Type: Bug
Components: contrib
Reporter: caijialiang
Attachments: image-2023-08-17-10-01-42-130.png,
image-2023-08-17-10-02-52-235.png
In the {{contrib/views}} module, specifically within the {{views}}
subdirectory, there is a dependency on the outdated {{ember-collection}} in the
frontend code. This is causing the {{files}} view to fail during compilation.
This issue is occurring because the repository
{{git://github.com/emberjs/ember-collection.git}} has been moved or relocated.
The problem can be resolved by updating the following file:
{{contrib/views/files/src/main/resources/ui/package.json}}
!image-2023-08-17-10-01-42-130.png!
Once this fix is applied, successful compilation should be achieved.
!image-2023-08-17-10-02-52-235.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org
For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754367#comment-17754367 ] caijialiang commented on AMBARI-25951: -- [~brahmareddy] Upgrading Ambari to Python 3 is indeed a significant and complex feature that requires a lot of support, development, and review. Considering that there was a previous PR discussing the Python 3 upgrade based on the branch 2.7, we might consider building upon this foundation for our development efforts. Here's [the link to that PR: https://github.com/apache/ambari/pull/3681.|https://github.com/apache/ambari/pull/3681] We highly value your insights and support in this matter. We look forward to hearing your thoughts and receiving your guidance! > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-25986) Upgrade Ambari hadoop dependency version
[ https://issues.apache.org/jira/browse/AMBARI-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-25986: - Attachment: image-2023-08-14-17-54-52-257.png > Upgrade Ambari hadoop dependency version > - > > Key: AMBARI-25986 > URL: https://issues.apache.org/jira/browse/AMBARI-25986 > Project: Ambari > Issue Type: Bug >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Attachments: image-2023-08-14-17-54-52-257.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-25986) Upgrade Ambari hadoop dependency version
[ https://issues.apache.org/jira/browse/AMBARI-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-25986: - Description: Due to Ambari 2.8's Bigtop stack, the default version of Hadoop is 3.3.4. Starting from Hadoop 3.3, Jetty was upgraded to version 9.4. However, certain parts of Ambari still retain Hadoop versions lower than 3.3, which use Jetty versions below 9.4. This leads to an API incompatibility issue, causing failures in Ambari's contributed tests that utilize the HDFS MiniCluster due to conflicting Jetty versions. Furthermore, the version of Hadoop should also align with the current default stack's Hadoop version to ensure consistency. !image-2023-08-14-17-54-52-257.png! > Upgrade Ambari hadoop dependency version > - > > Key: AMBARI-25986 > URL: https://issues.apache.org/jira/browse/AMBARI-25986 > Project: Ambari > Issue Type: Bug >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > Attachments: image-2023-08-14-17-54-52-257.png > > > Due to Ambari 2.8's Bigtop stack, the default version of Hadoop is 3.3.4. > Starting from Hadoop 3.3, Jetty was upgraded to version 9.4. However, certain > parts of Ambari still retain Hadoop versions lower than 3.3, which use Jetty > versions below 9.4. This leads to an API incompatibility issue, causing > failures in Ambari's contributed tests that utilize the HDFS MiniCluster due > to conflicting Jetty versions. > Furthermore, the version of Hadoop should also align with the current default > stack's Hadoop version to ensure consistency. > !image-2023-08-14-17-54-52-257.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-25986) Upgrade Ambari hadoop dependency version
[ https://issues.apache.org/jira/browse/AMBARI-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-25986: Assignee: caijialiang > Upgrade Ambari hadoop dependency version > - > > Key: AMBARI-25986 > URL: https://issues.apache.org/jira/browse/AMBARI-25986 > Project: Ambari > Issue Type: Bug >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-25986) Upgrade Ambari hadoop dependency version
caijialiang created AMBARI-25986: Summary: Upgrade Ambari hadoop dependency version Key: AMBARI-25986 URL: https://issues.apache.org/jira/browse/AMBARI-25986 Project: Ambari Issue Type: Bug Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752595#comment-17752595 ] caijialiang commented on AMBARI-25951: -- [~yaolei] [~houyu] Could you please provide some advice on releasing a version of the Ambari Hadoop3 ecosystem? > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752593#comment-17752593 ] caijialiang commented on AMBARI-25951: -- [~FanYuping] Regarding Ambari's support for deploying and manage containerized applications, it's indeed a fantastic idea. Other members of the community are also considering similar concepts and exploring solutions for implementation. If you have any great ideas or approaches that you'd like to share, please feel free to contribute and share your insights. Collaboration and collective brainstorming can lead to innovative solutions and advancements in this area. Your input and contributions would be highly valuable. > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752592#comment-17752592 ] caijialiang commented on AMBARI-25951: -- [~FanYuping] You've thought this through comprehensively. Indeed, there's a lot of stack-related logic in Ambari Common, such as HDFS resources, Ranger, Solr, and others. However, these are abstracted as common logic for stacks and aren't tightly bound to specific stacks. What I mean is that stacks could be maintained as separate sub-projects, allowing Ambari to support various stacks without the need for rigid coupling during releases. This approach would enable those with expertise in big data operations to focus on stack development and releases, while others familiar with Ambari could concentrate on enhancing its core functionality. Ambari's release cycle might be longer, but stacks could have a more agile and frequent release schedule, primarily focused on service management logic. Ambari's releases would emphasize its own feature enhancements and new capabilities. The release and iteration of stacks could be influenced more by the Bigtop community. Of course, there are existing challenges, particularly in swiftly deploying and setting up a cluster for testing within the Bigtop and Ambari environments. I'm also working on addressing this issue by developing scripts for rapid and flexible deployment to meet various environment requirements. > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Comment Edited] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752556#comment-17752556 ] caijialiang edited comment on AMBARI-25951 at 8/10/23 1:49 AM: --- [~brahmareddy] Supporting Python 3 is a great idea. Once Python 3 is supported, it will be much easier to expand compatibility with various operating systems. However, this is a substantial feature, and we might need more volunteers for its implementation. I am in favor of supporting Python at ambari 3.3 release. Should we discuss which specific version of Python 3 to target? was (Author: JIRAUSER298676): [~brahmareddy] Supporting Python 3 is a great idea. Once Python 3 is supported, it will be much easier to expand compatibility with various operating systems. However, this is a substantial feature, and we might need more volunteers for its implementation. I am in favor of supporting Python 3.3. Should we discuss which specific version of Python 3 to target? > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752563#comment-17752563 ] caijialiang commented on AMBARI-25951: -- I have a few ideas, but I'm unsure whether they should be included in version 3.3. Perhaps we could initiate a separate discussion. Regarding the issue of decoupling the stack: I believe decoupling the stack from Ambari and releasing it independently is a sound idea, offering the following advantages: 1. Decoupling Ambari and stack releases allows for more flexible versioning of the stack, which can be released as an mpack. 2. This would significantly enhance development and debugging processes, as well as allow users to customize actions like adding new components or upgrading component versions. Often, Ambari releases involve minimal changes while most modifications occur in the stack. During the development and debugging of the stack, any changes require recompiling Ambari and reinstalling the cluster, which can be quite cumbersome. If we separate the stack and provide an mpack installation option within Ambari, both the installation and debugging of the stack, as well as user-initiated stack updates, would become much simpler. I've observed many people struggling with the need to recompile and reinstall Ambari for stack updates. 3. In fact, Bigtop already supports packaging the stack into mpacks, offering convenience. Going forward, by incorporating the features of mpack v2, we could entirely manage the stack using REST APIs. We could even consider creating a stack service plugin library. These are my thoughts on addressing the stack decoupling matter. It might be worth initiating a discussion to delve further into this topic. > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752556#comment-17752556 ] caijialiang commented on AMBARI-25951: -- [~brahmareddy] Supporting Python 3 is a great idea. Once Python 3 is supported, it will be much easier to expand compatibility with various operating systems. However, this is a substantial feature, and we might need more volunteers for its implementation. I am in favor of supporting Python 3.3. Should we discuss which specific version of Python 3 to target? > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-25928) Continuous increase of websocket connections in Ambari web UI
[
https://issues.apache.org/jira/browse/AMBARI-25928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25928:
-
Parent: AMBARI-25951
Issue Type: Technical task (was: Bug)
> Continuous increase of websocket connections in Ambari web UI
> -
>
> Key: AMBARI-25928
> URL: https://issues.apache.org/jira/browse/AMBARI-25928
> Project: Ambari
> Issue Type: Technical task
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-04-14-14-48-36-956.png,
> image-2023-04-14-14-54-25-386.png
>
>
> How to reproduce: Open Ambari UI page and use Chrome to view the websocket
> connections. At the beginning, there is only one connection. After a few
> minutes, it is observed that the connections keep increasing infinitely. If
> the page is refreshed, the number of connections will return to one, and then
> continue to increase.
>
> {code:java}
> 2023-04-14 14:23:19,062 INFO [MessageBroker-1]
> WebSocketMessageBrokerStats:128 - WebSocketSession[2 current
> WS(2)-HttpStream(0)-HttpPoll(0), 258459 total, 0 closed abnormally (0 connect
> failure, 0 send limit, 171 transport error)], stompSubProtocol[processed
> CONNECT(258377)-CONNECTED(258377)-DISCONNECT(0)], stompBrokerRelay[null],
> inboundChannel[pool size = 4, active threads = 0, queued tasks = 0, completed
> tasks = 2109843], outboundChannel[pool size = 4, active threads = 3, queued
> tasks = 0, completed tasks = 1747340], sockJsScheduler[pool size = 2, active
> threads = 1, queued tasks = 0, completed tasks = 193]
> 2023-04-14 14:23:20,256 INFO [MessageBroker-1]
> WebSocketMessageBrokerStats:128 - WebSocketSession[3 current
> WS(3)-HttpStream(0)-HttpPoll(0), 18 total, 0 closed abnormally (0 connect
> failure, 0 send limit, 0 transport error)], stompSubProtocol[processed
> CONNECT(18)-CONNECTED(18)-DISCONNECT(15)], stompBrokerRelay[null],
> inboundChannel[pool size = 10, active threads = 0, queued tasks = 0,
> completed tasks = 415155], outboundChannel[pool size = 10, active threads =
> 0, queued tasks = 0, completed tasks = 138408], sockJsScheduler[pool size =
> 2, active threads = 1, queued tasks = 0, completed tasks = 193] {code}
>
>
>
> !image-2023-04-14-14-48-36-956.png!
>
> !image-2023-04-14-14-54-25-386.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org
For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Resolved] (AMBARI-25879) Ambari server hang when test kerberos clients
[ https://issues.apache.org/jira/browse/AMBARI-25879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang resolved AMBARI-25879. -- Resolution: Invalid > Ambari server hang when test kerberos clients > - > > Key: AMBARI-25879 > URL: https://issues.apache.org/jira/browse/AMBARI-25879 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.8.0 >Reporter: caijialiang >Priority: Major > Attachments: ambari_hang.png > > > ambari server hang when test kerberos clients! > after restart the ambari server,it showed test failed . > i just click test client again , it test passed. > so all kerberos are right ,something wrong in ambari server. > !ambari_hang.png! > these are some log in ambari server > > 2023-03-01 12:13:46,408 ERROR [agent-command-publisher-2] > AgentCommandsPublisher:115 - Exception on sendAgentCommand > java.lang.NumberFormatException: null > at java.lang.Long.parseLong(Long.java:552) > at java.lang.Long.valueOf(Long.java:803) > at > org.apache.ambari.server.events.publishers.AgentCommandsPublisher.lambda$null$0(AgentCommandsPublisher.java:110) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) > at > com.google.common.collect.CollectSpliterators$1.lambda$forEachRemaining$1(CollectSpliterators.java:116) > at > java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) > at > com.google.common.collect.CollectSpliterators$1.forEachRemaining(CollectSpliterators.java:116) > at > com.google.common.collect.CollectSpliterators$1FlatMapSpliterator.lambda$forEachRemaining$1(CollectSpliterators.java:247) > at > java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1696) > at > com.google.common.collect.CollectSpliterators$1FlatMapSpliterator.forEachRemaining(CollectSpliterators.java:247) > at > java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) > at > java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291) > at > java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731) > at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) > at > java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) > at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) > at > java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) > > > 2023-03-01 12:13:59,599 INFO [ambari-client-thread-27] RoleGraph:175 - > Detecting cycle graphs > 2023-03-01 12:13:59,600 INFO [ambari-client-thread-27] RoleGraph:176 - Graph: > (KERBEROS_CLIENT, CUSTOM_COMMAND, 0) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751998#comment-17751998 ] caijialiang commented on AMBARI-25951: -- [~lvkaihua] Due to a limited number of contributing developers, support for multiple NameNodes is currently not available. If you're interested, you can develop, test, and contribute to the community to incorporate this feature into the Ambari 2.9 version. > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751996#comment-17751996 ] caijialiang commented on AMBARI-25951: -- [~wzhy] There are currently no plans to support RHEL/Rocky Linux 9 in Ambari 2.9. If you're interested in contributing, you can get involved in the community. > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Resolved] (AMBARI-25983) Add ambari infra support
[ https://issues.apache.org/jira/browse/AMBARI-25983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang resolved AMBARI-25983. -- Resolution: Implemented > Add ambari infra support > - > > Key: AMBARI-25983 > URL: https://issues.apache.org/jira/browse/AMBARI-25983 > Project: Ambari > Issue Type: Technical task >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751448#comment-17751448 ] caijialiang commented on AMBARI-25951: -- [~vjasani] Could you please provide some advice on releasing a version of the Ambari Hadoop3 ecosystem? > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Comment Edited] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751444#comment-17751444 ] caijialiang edited comment on AMBARI-25951 at 8/7/23 3:59 AM: -- [~brahmareddy] Could you please provide some advice on releasing a version of the Ambari Hadoop3 ecosystem? was (Author: JIRAUSER298676): [~brahma] Could you please provide some advice on releasing a version of the Ambari Hadoop3 ecosystem? > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Commented] (AMBARI-25951) Support Bigtop stack 3.3.0
[ https://issues.apache.org/jira/browse/AMBARI-25951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751444#comment-17751444 ] caijialiang commented on AMBARI-25951: -- [~brahma] Could you please provide some advice on releasing a version of the Ambari Hadoop3 ecosystem? > Support Bigtop stack 3.3.0 > -- > > Key: AMBARI-25951 > URL: https://issues.apache.org/jira/browse/AMBARI-25951 > Project: Ambari > Issue Type: Task >Reporter: Lei Yao >Priority: Major > Fix For: 2.9.0 > > > Since Apache Bigtop prepare for releasing 3.3.0 , Bigtop stack 3.3.0 should > be supported -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-25983) Add ambari infra support
[ https://issues.apache.org/jira/browse/AMBARI-25983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-25983: Assignee: caijialiang > Add ambari infra support > - > > Key: AMBARI-25983 > URL: https://issues.apache.org/jira/browse/AMBARI-25983 > Project: Ambari > Issue Type: Technical task >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-25983) Add ambari infra support
caijialiang created AMBARI-25983: Summary: Add ambari infra support Key: AMBARI-25983 URL: https://issues.apache.org/jira/browse/AMBARI-25983 Project: Ambari Issue Type: Technical task Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari bigtop stack
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Parent: AMBARI-25951
Issue Type: Sub-task (was: New Feature)
> Add ranger 2.4 support in ambari bigtop stack
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: Sub-task
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Assignee: caijialiang
>Priority: Major
> Fix For: 2.9.0
>
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png, image-2023-05-09-11-09-14-373.png,
> image-2023-05-09-11-09-22-441.png
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Add ranger 2.4 support in ambari bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces (merged)
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
>
> 2.AMBARI-25932: Wrong config file name in spark service advisor
> ({*}merged{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}merged{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for merge in next version)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (merged)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support {*}({*}{*}merged{*}{*}){*}
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger {*}({*}{*}merged{*}{*}){*}
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> 4.BIGTOP-3950: fix ranger etc conf dir (in review)
> [https://github.com/apache/bigtop/pull/1120]
>
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
> https://issues.apache.org/jira/browse/RANGER-3992 wait for review
> manual test：
> before enable kerberos all compoent works smoonthly
> !image-2023-05-09-11-08-47-864.png!
>
[jira] [Created] (AMBARI-25971) Add yarn TimelineService V2 support
caijialiang created AMBARI-25971: Summary: Add yarn TimelineService V2 support Key: AMBARI-25971 URL: https://issues.apache.org/jira/browse/AMBARI-25971 Project: Ambari Issue Type: New Feature Reporter: caijialiang Attachments: image-2023-07-21-17-21-14-621.png, image-2023-07-21-17-21-30-630.png ambari 2.8.0 does not support yarn timeline service v2 and registrydns. However, yarn ui2 is very user-friendly and has commonly used features. Therefore, we have integrated it. after integrated: !image-2023-07-21-17-21-14-621.png! !image-2023-07-21-17-21-30-630.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-25971) Add yarn TimelineService V2 and Registrydns support
[ https://issues.apache.org/jira/browse/AMBARI-25971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-25971: - Summary: Add yarn TimelineService V2 and Registrydns support (was: Add yarn TimelineService V2 support) > Add yarn TimelineService V2 and Registrydns support > --- > > Key: AMBARI-25971 > URL: https://issues.apache.org/jira/browse/AMBARI-25971 > Project: Ambari > Issue Type: New Feature >Reporter: caijialiang >Priority: Major > Attachments: image-2023-07-21-17-21-14-621.png, > image-2023-07-21-17-21-30-630.png > > > ambari 2.8.0 does not support yarn timeline service v2 and registrydns. > However, yarn ui2 is very user-friendly and has commonly used features. > Therefore, we have integrated it. > after integrated: > !image-2023-07-21-17-21-14-621.png! > !image-2023-07-21-17-21-30-630.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-25970) Fix incorrect ambari infra-solr service config
caijialiang created AMBARI-25970: Summary: Fix incorrect ambari infra-solr service config Key: AMBARI-25970 URL: https://issues.apache.org/jira/browse/AMBARI-25970 Project: Ambari Issue Type: Bug Reporter: caijialiang ambari infra-solr raised an error after started like this: solrconfig.xml contains more than one value for config path: X root cause is ： XML configuration parsing is now more strict about situations where a single setting is allowed but multiple values are found. In the past, one value would be chosen arbitrarily and silently. Starting with 4.5, configuration parsing will fail with an error -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari bigtop stack
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Description:
Add ranger 2.4 support in ambari bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part first. Otherwise, after enabling Ranger plugin in
Ambari, manual updates to the Ranger-related configurations would still be
required.
1.AMBARI-25894: Missing file service_advisor.py in some serivces (merged)
[https://github.com/apache/ambari/pull/3677
|https://github.com/apache/ambari/pull/3677]
2.AMBARI-25932: Wrong config file name in spark service advisor ({*}merged{*})
https://issues.apache.org/jira/browse/AMBARI-25932
h2. ambari ranger support related issues:
Here, we additionally adapted Ambari Infra because the Ambari Ranger service
relies on the Infra client to perform Solr-related automation settings for
Ranger.
h4. 1.ambari infra PR
1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}merged{*})
https://issues.apache.org/jira/browse/AMBARI-25933
h4. 2.ambari ranger service support (wait for merge in next version)
https://issues.apache.org/jira/browse/AMBARI-25929
h4. 3.ambari infra service support wait for review (merged)
[https://github.com/apache/ambari/pull/3696]
h2. apache bigtop related issues
To adapt Ranger to Ambari, we need to first build RPM packages related to
Ranger using Bigtop. The following are the pull requests required for Bigtop to
support Ranger.
1.BIGTOP-3925 ranger support {*}({*}{*}merged{*}{*}){*}
[https://github.com/apache/bigtop/pull/1100]
2.BIGTOP-3923: Add missing jars for Ranger {*}({*}{*}merged{*}{*}){*}
[https://github.com/apache/bigtop/pull/1099]
3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
[https://github.com/apache/bigtop/pull/1089]
4.BIGTOP-3950: fix ranger etc conf dir (in review)
[https://github.com/apache/bigtop/pull/1120]
Bigtop support for Ranger requires three PRs:
# BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
includes a patch that resolves problems with starting HBase after integrating
with Ranger.
# BIGTOP-3923: This PR addresses package dependency issues when running "java
-cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
# BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
h3. ranger related issues:
These are the PRs encountered during the process of adapting Ranger in Bigtop
Ambari. All 3 PRs have been made into patches and submitted to the
aforementioned Bigtop Ranger support-related PRs. The review of the related
issues on the Ranger side is also currently underway.
1.fix Kafka2.8 can't restart after enable ranger plugin
https://issues.apache.org/jira/browse/RANGER-4228 wait for review
2.addresses the issue of HBase not starting after integrating with Ranger due
to class loading order
https://issues.apache.org/jira/browse/RANGER-4201 wait for review
3.mainly addresses the missing dependency issue when running the Ranger command
"java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
https://issues.apache.org/jira/browse/RANGER-3992 wait for review
manual test：
before enable kerberos all compoent works smoonthly
!image-2023-05-09-11-08-47-864.png!
!image-2023-05-09-11-08-57-472.png!
after enable kerberos all compoent works smoonthly
!image-2023-05-09-11-09-14-373.png!
!image-2023-05-09-11-09-22-441.png!
was:
Add ranger 2.4 support in ambari bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part first. Otherwise, after enabling Ranger plugin in
Ambari, manual updates to the Ranger-related configurations would still be
required.
1.AMBARI-25894: Missing file service_advisor.py
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Description:
Add ranger 2.4 support in ambari 2.8 bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part first. Otherwise, after enabling Ranger plugin in
Ambari, manual updates to the Ranger-related configurations would still be
required.
1.AMBARI-25894: Missing file service_advisor.py in some serivces ({*}wait
for review{*})
[https://github.com/apache/ambari/pull/3677
|https://github.com/apache/ambari/pull/3677]
2.AMBARI-25932: Wrong config file name in spark service advisor ({*}merged{*})
https://issues.apache.org/jira/browse/AMBARI-25932
h2. ambari 2.8 ranger support related issues:
Here, we additionally adapted Ambari Infra because the Ambari Ranger service
relies on the Infra client to perform Solr-related automation settings for
Ranger.
h4. 1.ambari infra PR
1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25933
h4. 2.ambari ranger service support (wait for review)
https://issues.apache.org/jira/browse/AMBARI-25929
h4. 3.ambari infra service support wait for review (wait for review)
[https://github.com/apache/ambari/pull/3696]
h2. apache bigtop related issues
To adapt Ranger to Ambari, we need to first build RPM packages related to
Ranger using Bigtop. The following are the pull requests required for Bigtop to
support Ranger.
1.BIGTOP-3925 ranger support {*}({*}{*}merged{*}{*}){*}
[https://github.com/apache/bigtop/pull/1100]
2.BIGTOP-3923: Add missing jars for Ranger {*}({*}{*}merged{*}{*}){*}
[https://github.com/apache/bigtop/pull/1099]
3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
[https://github.com/apache/bigtop/pull/1089]
Bigtop support for Ranger requires three PRs:
# BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
includes a patch that resolves problems with starting HBase after integrating
with Ranger.
# BIGTOP-3923: This PR addresses package dependency issues when running "java
-cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
# BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
h3. ranger related issues:
These are the PRs encountered during the process of adapting Ranger in Bigtop
Ambari. All 3 PRs have been made into patches and submitted to the
aforementioned Bigtop Ranger support-related PRs. The review of the related
issues on the Ranger side is also currently underway.
1.fix Kafka2.8 can't restart after enable ranger plugin
https://issues.apache.org/jira/browse/RANGER-4228 wait for review
2.addresses the issue of HBase not starting after integrating with Ranger due
to class loading order
https://issues.apache.org/jira/browse/RANGER-4201 wait for review
3.mainly addresses the missing dependency issue when running the Ranger command
"java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
https://issues.apache.org/jira/browse/RANGER-3992 wait for review
manual test：
before enable kerberos all compoent works smoonthly
!image-2023-05-09-11-08-47-864.png!
!image-2023-05-09-11-08-57-472.png!
after enable kerberos all compoent works smoonthly
!image-2023-05-09-11-09-14-373.png!
!image-2023-05-09-11-09-22-441.png!
was:
Add ranger 2.4 support in ambari 2.8 bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part first. Otherwise, after enabling Ranger plugin in
Ambari, manual updates to the Ranger-related configurations would still be
required.
1.AMBARI-25894: Missing file service_advisor.py in some serivces ({*}wait
for review{*})
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Description:
Add ranger 2.4 support in ambari 2.8 bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part first. Otherwise, after enabling Ranger plugin in
Ambari, manual updates to the Ranger-related configurations would still be
required.
1.AMBARI-25894: Missing file service_advisor.py in some serivces ({*}wait
for review{*})
[https://github.com/apache/ambari/pull/3677
|https://github.com/apache/ambari/pull/3677]
2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25931
3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25932
h2. ambari 2.8 ranger support related issues:
Here, we additionally adapted Ambari Infra because the Ambari Ranger service
relies on the Infra client to perform Solr-related automation settings for
Ranger.
h4. 1.ambari infra PR
1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25933
h4. 2.ambari ranger service support (wait for review)
https://issues.apache.org/jira/browse/AMBARI-25929
h4. 3.ambari infra service support wait for review (wait for review)
[https://github.com/apache/ambari/pull/3696]
h2. apache bigtop related issues
To adapt Ranger to Ambari, we need to first build RPM packages related to
Ranger using Bigtop. The following are the pull requests required for Bigtop to
support Ranger.
1.BIGTOP-3925 ranger support {*}({*}{*}merged{*}{*}){*}
[https://github.com/apache/bigtop/pull/1100]
2.BIGTOP-3923: Add missing jars for Ranger {*}({*}{*}merged{*}{*}){*}
[https://github.com/apache/bigtop/pull/1099]
3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
[https://github.com/apache/bigtop/pull/1089]
Bigtop support for Ranger requires three PRs:
# BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
includes a patch that resolves problems with starting HBase after integrating
with Ranger.
# BIGTOP-3923: This PR addresses package dependency issues when running "java
-cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
# BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
h3. ranger related issues:
These are the PRs encountered during the process of adapting Ranger in Bigtop
Ambari. All 3 PRs have been made into patches and submitted to the
aforementioned Bigtop Ranger support-related PRs. The review of the related
issues on the Ranger side is also currently underway.
1.fix Kafka2.8 can't restart after enable ranger plugin
https://issues.apache.org/jira/browse/RANGER-4228 wait for review
2.addresses the issue of HBase not starting after integrating with Ranger due
to class loading order
https://issues.apache.org/jira/browse/RANGER-4201 wait for review
3.mainly addresses the missing dependency issue when running the Ranger command
"java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
https://issues.apache.org/jira/browse/RANGER-3992 wait for review
manual test：
before enable kerberos all compoent works smoonthly
!image-2023-05-09-11-08-47-864.png!
!image-2023-05-09-11-08-57-472.png!
after enable kerberos all compoent works smoonthly
!image-2023-05-09-11-09-14-373.png!
!image-2023-05-09-11-09-22-441.png!
was:
Add ranger 2.3 support in ambari 2.8 bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part first. Otherwise, after enabling Ranger plugin in
Ambari, manual updates to the Ranger-related
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Description:
Add ranger 2.3 support in ambari 2.8 bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part first. Otherwise, after enabling Ranger plugin in
Ambari, manual updates to the Ranger-related configurations would still be
required.
1.AMBARI-25894: Missing file service_advisor.py in some serivces ({*}wait
for review{*})
[https://github.com/apache/ambari/pull/3677
|https://github.com/apache/ambari/pull/3677]
yarn advisor 中 -Dhdp.version=${hdp.version}") The suggestion of a review is
still under discussion.
2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25931
3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25932
h2. ambari 2.8 ranger support related issues:
Here, we additionally adapted Ambari Infra because the Ambari Ranger service
relies on the Infra client to perform Solr-related automation settings for
Ranger.
h4. 1.ambari infra PR
1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25933
h4. 2.ambari ranger service support (wait for review)
https://issues.apache.org/jira/browse/AMBARI-25929
h4. 3.ambari infra service support wait for review (wait for review)
[https://github.com/apache/ambari/pull/3696]
h2. apache bigtop related issues
To adapt Ranger to Ambari, we need to first build RPM packages related to
Ranger using Bigtop. The following are the pull requests required for Bigtop to
support Ranger.
1.BIGTOP-3925 ranger support {*}({*}{*}merged{*}{*}){*}
[https://github.com/apache/bigtop/pull/1100]
2.BIGTOP-3923: Add missing jars for Ranger {*}({*}{*}merged{*}{*}){*}
[https://github.com/apache/bigtop/pull/1099]
3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
[https://github.com/apache/bigtop/pull/1089]
Bigtop support for Ranger requires three PRs:
# BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
includes a patch that resolves problems with starting HBase after integrating
with Ranger.
# BIGTOP-3923: This PR addresses package dependency issues when running "java
-cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
# BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
h3. ranger related issues:
These are the PRs encountered during the process of adapting Ranger in Bigtop
Ambari. All 3 PRs have been made into patches and submitted to the
aforementioned Bigtop Ranger support-related PRs. The review of the related
issues on the Ranger side is also currently underway.
1.fix Kafka2.8 can't restart after enable ranger plugin
https://issues.apache.org/jira/browse/RANGER-4228 wait for review
2.addresses the issue of HBase not starting after integrating with Ranger due
to class loading order
https://issues.apache.org/jira/browse/RANGER-4201 wait for review
3.mainly addresses the missing dependency issue when running the Ranger command
"java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
https://issues.apache.org/jira/browse/RANGER-3992 wait for review
manual test：
before enable kerberos all compoent works smoonthly
!image-2023-05-09-11-08-47-864.png!
!image-2023-05-09-11-08-57-472.png!
after enable kerberos all compoent works smoonthly
!image-2023-05-09-11-09-14-373.png!
!image-2023-05-09-11-09-22-441.png!
was:
Add ranger 2.3 support in ambari 2.8 bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part
[jira] [Created] (AMBARI-25948) Update hadoop related dependencies's version to hadoop3
caijialiang created AMBARI-25948: Summary: Update hadoop related dependencies's version to hadoop3 Key: AMBARI-25948 URL: https://issues.apache.org/jira/browse/AMBARI-25948 Project: Ambari Issue Type: Improvement Reporter: caijialiang Ambari 2.8 now supports Hadoop 3 ecosystem components, but most of the Hadoop version dependencies in the Ambari project are still based on Hadoop 2.7. This could potentially cause issues, so it is necessary to update the dependencies to Hadoop 3. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Assigned] (AMBARI-25948) Update hadoop related dependencies's version to hadoop3
[ https://issues.apache.org/jira/browse/AMBARI-25948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang reassigned AMBARI-25948: Assignee: caijialiang > Update hadoop related dependencies's version to hadoop3 > --- > > Key: AMBARI-25948 > URL: https://issues.apache.org/jira/browse/AMBARI-25948 > Project: Ambari > Issue Type: Improvement >Reporter: caijialiang >Assignee: caijialiang >Priority: Major > > Ambari 2.8 now supports Hadoop 3 ecosystem components, but most of the Hadoop > version dependencies in the Ambari project are still based on Hadoop 2.7. > This could potentially cause issues, so it is necessary to update the > dependencies to Hadoop 3. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Comment Edited] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720857#comment-17720857
]
caijialiang edited comment on AMBARI-25929 at 5/9/23 8:30 AM:
--
[~wzhy]
I didn't see any plans for the community to support Elasticsearch in Ambari,
but you can develop your own Elasticsearch mpack to easily install it in an
existing Ambari cluster.
Of course, if you're interested, you can also add support for Elasticsearch in
Ambari and submit it to the community.
was (Author: JIRAUSER298676):
[~wzhy]
I didn't see any plans for the community to support Elasticsearch in Ambari,
but you can develop your own Elasticsearch mpack to easily install it in an
existing Ambari cluster.
> Add ranger 2.4 support in ambari 2.8 bigtop stacks
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: New Feature
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png, image-2023-05-09-11-09-14-373.png,
> image-2023-05-09-11-09-22-441.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add ranger 2.3 support in ambari 2.8 bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces
> ({*}wait for review{*})
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
> yarn advisor 中 -Dhdp.version=${hdp.version}") The suggestion of a review is
> still under discussion.
>
> 2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25931
>
> 3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari 2.8 ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for review)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (wait for review)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support *(wait for review)*
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger *(wait for review)*
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses
[jira] [Commented] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720857#comment-17720857
]
caijialiang commented on AMBARI-25929:
--
[~wzhy]
I didn't see any plans for the community to support Elasticsearch in Ambari,
but you can develop your own Elasticsearch mpack to easily install it in an
existing Ambari cluster.
> Add ranger 2.4 support in ambari 2.8 bigtop stacks
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: New Feature
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png, image-2023-05-09-11-09-14-373.png,
> image-2023-05-09-11-09-22-441.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add ranger 2.3 support in ambari 2.8 bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces
> ({*}wait for review{*})
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
> yarn advisor 中 -Dhdp.version=${hdp.version}") The suggestion of a review is
> still under discussion.
>
> 2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25931
>
> 3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari 2.8 ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for review)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (wait for review)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support *(wait for review)*
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger *(wait for review)*
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password
[jira] [Commented] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720760#comment-17720760
]
caijialiang commented on AMBARI-25929:
--
[~wuzhiguo] [~houyu] [~leiyao]
The current progress of Ranger adaptation and all related issues are listed in
description.
> Add ranger 2.4 support in ambari 2.8 bigtop stacks
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: New Feature
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png, image-2023-05-09-11-09-14-373.png,
> image-2023-05-09-11-09-22-441.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add ranger 2.3 support in ambari 2.8 bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces
> ({*}wait for review{*})
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
> yarn advisor 中 -Dhdp.version=${hdp.version}") The suggestion of a review is
> still under discussion.
>
> 2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25931
>
> 3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari 2.8 ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for review)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (wait for review)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support *(wait for review)*
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger *(wait for review)*
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
>
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Description:
Add ranger 2.3 support in ambari 2.8 bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part first. Otherwise, after enabling Ranger plugin in
Ambari, manual updates to the Ranger-related configurations would still be
required.
1.AMBARI-25894: Missing file service_advisor.py in some serivces ({*}wait
for review{*})
[https://github.com/apache/ambari/pull/3677
|https://github.com/apache/ambari/pull/3677]
yarn advisor 中 -Dhdp.version=${hdp.version}") The suggestion of a review is
still under discussion.
2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25931
3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25932
h2. ambari 2.8 ranger support related issues:
Here, we additionally adapted Ambari Infra because the Ambari Ranger service
relies on the Infra client to perform Solr-related automation settings for
Ranger.
h4. 1.ambari infra PR
1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25933
h4. 2.ambari ranger service support (wait for review)
https://issues.apache.org/jira/browse/AMBARI-25929
h4. 3.ambari infra service support wait for review (wait for review)
[https://github.com/apache/ambari/pull/3696]
h2. apache bigtop related issues
To adapt Ranger to Ambari, we need to first build RPM packages related to
Ranger using Bigtop. The following are the pull requests required for Bigtop to
support Ranger.
1.BIGTOP-3925 ranger support *(wait for review)*
[https://github.com/apache/bigtop/pull/1100]
2.BIGTOP-3923: Add missing jars for Ranger *(wait for review)*
[https://github.com/apache/bigtop/pull/1099]
3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
[https://github.com/apache/bigtop/pull/1089]
Bigtop support for Ranger requires three PRs:
# BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
includes a patch that resolves problems with starting HBase after integrating
with Ranger.
# BIGTOP-3923: This PR addresses package dependency issues when running "java
-cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
# BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
h3. ranger related issues:
These are the PRs encountered during the process of adapting Ranger in Bigtop
Ambari. All 3 PRs have been made into patches and submitted to the
aforementioned Bigtop Ranger support-related PRs. The review of the related
issues on the Ranger side is also currently underway.
1.fix Kafka2.8 can't restart after enable ranger plugin
https://issues.apache.org/jira/browse/RANGER-4228 wait for review
2.addresses the issue of HBase not starting after integrating with Ranger due
to class loading order
https://issues.apache.org/jira/browse/RANGER-4201 wait for review
3.mainly addresses the missing dependency issue when running the Ranger command
"java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
https://issues.apache.org/jira/browse/RANGER-3992 wait for review
manual test：
before enable kerberos all compoent works smoonthly
!image-2023-05-09-11-08-47-864.png!
!image-2023-05-09-11-08-57-472.png!
after enable kerberos all compoent works smoonthly
!image-2023-05-09-11-09-14-373.png!
!image-2023-05-09-11-09-22-441.png!
was:
Add ranger 2.3 support in ambari 2.8 bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part first. Otherwise,
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Attachment: image-2023-05-09-11-09-14-373.png
> Add ranger 2.4 support in ambari 2.8 bigtop stacks
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: New Feature
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png, image-2023-05-09-11-09-14-373.png,
> image-2023-05-09-11-09-22-441.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add ranger 2.3 support in ambari 2.8 bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces
> ({*}wait for review{*})
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
> yarn advisor 中 -Dhdp.version=${hdp.version}") The suggestion of a review is
> still under discussion.
>
> 2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25931
>
> 3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari 2.8 ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for review)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (wait for review)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support *(wait for review)*
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger *(wait for review)*
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
> https://issues.apache.org/jira/browse/RANGER-3992 wait for review
> manual test：
>
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Attachment: image-2023-05-09-11-09-22-441.png
> Add ranger 2.4 support in ambari 2.8 bigtop stacks
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: New Feature
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png, image-2023-05-09-11-09-14-373.png,
> image-2023-05-09-11-09-22-441.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add ranger 2.3 support in ambari 2.8 bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces
> ({*}wait for review{*})
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
> yarn advisor 中 -Dhdp.version=${hdp.version}") The suggestion of a review is
> still under discussion.
>
> 2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25931
>
> 3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari 2.8 ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for review)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (wait for review)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support *(wait for review)*
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger *(wait for review)*
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
> https://issues.apache.org/jira/browse/RANGER-3992 wait for review
> manual test：
>
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Description:
Add ranger 2.3 support in ambari 2.8 bigtop stacks
h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
completed. Due to the large number of issues involved in the adaptation, they
are summarized as follows:
h2. apache ambari related issues
The main issues related to adapting Ranger in Ambari 2.8 are related to the
advisor functionality. Enabling Ranger would trigger the advisor to recommend
updates to the component's Ranger-related configurations, thus adapting Ranger
requires fixing this part first. Otherwise, after enabling Ranger plugin in
Ambari, manual updates to the Ranger-related configurations would still be
required.
1.AMBARI-25894: Missing file service_advisor.py in some serivces ({*}wait
for review{*})
[https://github.com/apache/ambari/pull/3677
|https://github.com/apache/ambari/pull/3677]
yarn advisor 中 -Dhdp.version=${hdp.version}") The suggestion of a review is
still under discussion.
2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25931
3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25932
h2. ambari 2.8 ranger support related issues:
Here, we additionally adapted Ambari Infra because the Ambari Ranger service
relies on the Infra client to perform Solr-related automation settings for
Ranger.
h4. 1.ambari infra PR
1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
review{*})
https://issues.apache.org/jira/browse/AMBARI-25933
h4. 2.ambari ranger service support (wait for review)
https://issues.apache.org/jira/browse/AMBARI-25929
h4. 3.ambari infra service support wait for review (wait for review)
[https://github.com/apache/ambari/pull/3696]
h2. apache bigtop related issues
To adapt Ranger to Ambari, we need to first build RPM packages related to
Ranger using Bigtop. The following are the pull requests required for Bigtop to
support Ranger.
1.BIGTOP-3925 ranger support *(wait for review)*
[https://github.com/apache/bigtop/pull/1100]
2.BIGTOP-3923: Add missing jars for Ranger *(wait for review)*
[https://github.com/apache/bigtop/pull/1099]
3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
[https://github.com/apache/bigtop/pull/1089]
Bigtop support for Ranger requires three PRs:
# BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
includes a patch that resolves problems with starting HBase after integrating
with Ranger.
# BIGTOP-3923: This PR addresses package dependency issues when running "java
-cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
# BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
h3. ranger related issues:
These are the PRs encountered during the process of adapting Ranger in Bigtop
Ambari. All 3 PRs have been made into patches and submitted to the
aforementioned Bigtop Ranger support-related PRs. The review of the related
issues on the Ranger side is also currently underway.
1.fix Kafka2.8 can't restart after enable ranger plugin
https://issues.apache.org/jira/browse/RANGER-4228 wait for review
2.addresses the issue of HBase not starting after integrating with Ranger due
to class loading order
https://issues.apache.org/jira/browse/RANGER-4201 wait for review
3.mainly addresses the missing dependency issue when running the Ranger command
"java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
org.apache.ranger.credentialapi.buildks create
ranger.usersync.policymgr.password -value [PROTECTED] -provider
jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
https://issues.apache.org/jira/browse/RANGER-3992 wait for review
manual test：
before enable kerberos all compoent works smoonthly
!image-2023-05-09-10-51-20-266.png!
!image-2023-05-09-10-51-44-693.png!
after enable kerberos all compoent works smoonthly
!image-2023-05-09-10-52-10-134.png!
!image-2023-05-09-10-52-22-888.png!
was:Add ranger 2.3 support in ambari 2.8 bigtop stacks
> Add ranger 2.4 support in ambari 2.8 bigtop stacks
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: New Feature
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-05-09-11-08-47-864.png,
>
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Attachment: image-2023-05-09-11-08-57-472.png
> Add ranger 2.4 support in ambari 2.8 bigtop stacks
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: New Feature
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add ranger 2.3 support in ambari 2.8 bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces
> ({*}wait for review{*})
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
> yarn advisor 中 -Dhdp.version=${hdp.version}") The suggestion of a review is
> still under discussion.
>
> 2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25931
>
> 3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari 2.8 ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for review)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (wait for review)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support *(wait for review)*
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger *(wait for review)*
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
> https://issues.apache.org/jira/browse/RANGER-3992 wait for review
> manual test：
> before enable kerberos all compoent works smoonthly
>
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[
https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang updated AMBARI-25929:
-
Attachment: image-2023-05-09-11-08-47-864.png
> Add ranger 2.4 support in ambari 2.8 bigtop stacks
> --
>
> Key: AMBARI-25929
> URL: https://issues.apache.org/jira/browse/AMBARI-25929
> Project: Ambari
> Issue Type: New Feature
> Components: ambari-server
>Affects Versions: 2.8.0
>Reporter: caijialiang
>Priority: Major
> Attachments: image-2023-05-09-11-08-47-864.png,
> image-2023-05-09-11-08-57-472.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add ranger 2.3 support in ambari 2.8 bigtop stacks
> h3. Currently, the work of adapting Ranger to Bigtop and Ambari has been
> completed. Due to the large number of issues involved in the adaptation, they
> are summarized as follows:
> h2. apache ambari related issues
> The main issues related to adapting Ranger in Ambari 2.8 are related to the
> advisor functionality. Enabling Ranger would trigger the advisor to recommend
> updates to the component's Ranger-related configurations, thus adapting
> Ranger requires fixing this part first. Otherwise, after enabling Ranger
> plugin in Ambari, manual updates to the Ranger-related configurations would
> still be required.
>
> 1.AMBARI-25894: Missing file service_advisor.py in some serivces
> ({*}wait for review{*})
> [https://github.com/apache/ambari/pull/3677
> |https://github.com/apache/ambari/pull/3677]
> yarn advisor 中 -Dhdp.version=${hdp.version}") The suggestion of a review is
> still under discussion.
>
> 2.AMBARI-25931: fix wrong hdp path in bigtop stack services ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25931
>
> 3.AMBARI-25932: Wrong config file name in spark service advisor ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25932
>
> h2. ambari 2.8 ranger support related issues:
> Here, we additionally adapted Ambari Infra because the Ambari Ranger service
> relies on the Infra client to perform Solr-related automation settings for
> Ranger.
>
> h4. 1.ambari infra PR
> 1.Add support for Ambari Infra in Ambari 2.8 wait for review ({*}wait for
> review{*})
> https://issues.apache.org/jira/browse/AMBARI-25933
> h4. 2.ambari ranger service support (wait for review)
> https://issues.apache.org/jira/browse/AMBARI-25929
> h4. 3.ambari infra service support wait for review (wait for review)
> [https://github.com/apache/ambari/pull/3696]
> h2. apache bigtop related issues
> To adapt Ranger to Ambari, we need to first build RPM packages related to
> Ranger using Bigtop. The following are the pull requests required for Bigtop
> to support Ranger.
>
> 1.BIGTOP-3925 ranger support *(wait for review)*
> [https://github.com/apache/bigtop/pull/1100]
> 2.BIGTOP-3923: Add missing jars for Ranger *(wait for review)*
> [https://github.com/apache/bigtop/pull/1099]
> 3.BIGTOP-3910: Bigtop-select support Ranger ({*}merged{*})
> [https://github.com/apache/bigtop/pull/1089]
> Bigtop support for Ranger requires three PRs:
> # BIGTOP-3925: This PR mainly addresses issues with Ranger RPM packaging and
> includes a patch that resolves problems with starting HBase after integrating
> with Ranger.
> # BIGTOP-3923: This PR addresses package dependency issues when running
> "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks" command.
> # BIGTOP-3910: This PR adds support for Ranger in Bigtop-select.
> h3. ranger related issues:
>
> These are the PRs encountered during the process of adapting Ranger in Bigtop
> Ambari. All 3 PRs have been made into patches and submitted to the
> aforementioned Bigtop Ranger support-related PRs. The review of the related
> issues on the Ranger side is also currently underway.
>
>
> 1.fix Kafka2.8 can't restart after enable ranger plugin
> https://issues.apache.org/jira/browse/RANGER-4228 wait for review
> 2.addresses the issue of HBase not starting after integrating with Ranger due
> to class loading order
> https://issues.apache.org/jira/browse/RANGER-4201 wait for review
> 3.mainly addresses the missing dependency issue when running the Ranger
> command "java -cp '/usr/bigtop/current/ranger-usersync/lib/*'
> org.apache.ranger.credentialapi.buildks create
> ranger.usersync.policymgr.password -value [PROTECTED] -provider
> jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks".
> https://issues.apache.org/jira/browse/RANGER-3992 wait for review
> manual test：
> before enable kerberos all compoent works smoonthly
>
[jira] [Updated] (AMBARI-25929) Add ranger 2.4 support in ambari 2.8 bigtop stacks
[ https://issues.apache.org/jira/browse/AMBARI-25929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-25929: - Summary: Add ranger 2.4 support in ambari 2.8 bigtop stacks (was: Add ranger 2.3 support in ambari 2.8 bigtop stacks) > Add ranger 2.4 support in ambari 2.8 bigtop stacks > -- > > Key: AMBARI-25929 > URL: https://issues.apache.org/jira/browse/AMBARI-25929 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: 2.8.0 >Reporter: caijialiang >Priority: Major > > Add ranger 2.3 support in ambari 2.8 bigtop stacks -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-25935) Upgrade Ambari Infra Solr to version 8 and ensure it is consistent with the Solr version in Ambari 2.8 stack
caijialiang created AMBARI-25935: Summary: Upgrade Ambari Infra Solr to version 8 and ensure it is consistent with the Solr version in Ambari 2.8 stack Key: AMBARI-25935 URL: https://issues.apache.org/jira/browse/AMBARI-25935 Project: Ambari Issue Type: Technical task Reporter: caijialiang Upgrade Ambari Infra Solr to version 8 and ensure it is consistent with the Solr version in Ambari 2.8 stack. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-25934) Add support for Ambari Infra in Ambari Server Bigtop Stack
caijialiang created AMBARI-25934: Summary: Add support for Ambari Infra in Ambari Server Bigtop Stack Key: AMBARI-25934 URL: https://issues.apache.org/jira/browse/AMBARI-25934 Project: Ambari Issue Type: Technical task Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-25933) Add support for Ambari Infra in Ambari 2.8
caijialiang created AMBARI-25933: Summary: Add support for Ambari Infra in Ambari 2.8 Key: AMBARI-25933 URL: https://issues.apache.org/jira/browse/AMBARI-25933 Project: Ambari Issue Type: New Feature Reporter: caijialiang -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Updated] (AMBARI-25932) wrong config file name in spark service advisor
[ https://issues.apache.org/jira/browse/AMBARI-25932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caijialiang updated AMBARI-25932: - Description: When using Spark 3.x, the "spark-defaults.conf" file is required to configure the Spark cluster, instead of "spark-default.xml" or "spark2-default.xml". In Ambari 2.8, all configurations for the Spark service are in the "spark-defaults.conf" file. However, the service advisor is still based on Spark 2, so the default Spark configuration file needs to be changed from "spark2-default" to "spark-default". In addition, in Ambari 2.8, the Spark 3 Thrift Server also uses "spark-defaults.conf" at startup, so the "spark2-thrift-sparkconf" configuration in the advisor needs to be deprecated. was:When using Spark 3.x, the "spark-defaults.conf" file is required to configure the Spark cluster, instead of "spark-default.xml" or "spark2-default.xml". In Ambari 2.8, all configurations for the Spark service are in the "spark-defaults.conf" file. However, the service advisor is still based on Spark 2, so the default Spark configuration file needs to be changed from "spark2-default" to "spark-default". > wrong config file name in spark service advisor > > > Key: AMBARI-25932 > URL: https://issues.apache.org/jira/browse/AMBARI-25932 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.8.0 >Reporter: caijialiang >Priority: Major > > When using Spark 3.x, the "spark-defaults.conf" file is required to configure > the Spark cluster, instead of "spark-default.xml" or "spark2-default.xml". In > Ambari 2.8, all configurations for the Spark service are in the > "spark-defaults.conf" file. However, the service advisor is still based on > Spark 2, so the default Spark configuration file needs to be changed from > "spark2-default" to "spark-default". > In addition, in Ambari 2.8, the Spark 3 Thrift Server also uses > "spark-defaults.conf" at startup, so the "spark2-thrift-sparkconf" > configuration in the advisor needs to be deprecated. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
[jira] [Created] (AMBARI-25932) wrong config file name in spark service advisor
caijialiang created AMBARI-25932: Summary: wrong config file name in spark service advisor Key: AMBARI-25932 URL: https://issues.apache.org/jira/browse/AMBARI-25932 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.8.0 Reporter: caijialiang When using Spark 3.x, the "spark-defaults.conf" file is required to configure the Spark cluster, instead of "spark-default.xml" or "spark2-default.xml". In Ambari 2.8, all configurations for the Spark service are in the "spark-defaults.conf" file. However, the service advisor is still based on Spark 2, so the default Spark configuration file needs to be changed from "spark2-default" to "spark-default". -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org
