[jira] [Commented] (AMBARI-25505) New version of isilon-onefs-mpack (1.0.3)
[
https://issues.apache.org/jira/browse/AMBARI-25505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110049#comment-17110049
]
Ivan Martynov commented on AMBARI-25505:
I ran local unit test.
Tests are located at:
~/ambari/contrib/management-packs/isilon-onefs-mpack/src/test/python
Ran "python unitTests.py"
Results:
test_core_site_validation_error_on_host_mismatch (TestServiceAdvisor.TestUri)
... ok
test_fix_host (TestServiceAdvisor.TestUri) ... ok
test_hdfs_site_no_validation_error (TestServiceAdvisor.TestUri) ... ok
test_skip_fixing_invalid_host (TestServiceAdvisor.TestUri) ... ok
test_skip_replacing_to_empty_host (TestServiceAdvisor.TestUri) ... ok
--
Ran 5 tests in 0.003s
OK
--
Total run:5
Total errors:0
Total failures:0
OK
although I got the error before test ran, it is not connected with failed
test-cases.
So, local unit testing succeeded
> New version of isilon-onefs-mpack (1.0.3)
> -
>
> Key: AMBARI-25505
> URL: https://issues.apache.org/jira/browse/AMBARI-25505
> Project: Ambari
> Issue Type: Improvement
> Components: contrib
>Affects Versions: trunk, 2.7.0
> Environment: HDP versions started from "2.6"
> Ambari started from version "2.7"
>Reporter: Ivan Martynov
>Priority: Critical
> Labels: performance
> Fix For: trunk, 2.7.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This issue is going under the direction of Dell Technologies.
> We are going to change the code of isilon-onefs-mpack
> (+[https://github.com/apache/ambari/tree/trunk/contrib/management-packs/isilon-onefs-mpack]+).
> The problem is that a new feature of HBase 2 (AsyncFSWAL) causes worse
> performance with OneFS. We need to disable this in the Ambari management pack
> by setting the property name: "hbase.wal.provider" with value: "filesystem"
> for HBASE. The way to change the property has been already introduced at:
> [https://jira.apache.org/jira/browse/HBASE-15536]
> In section about how to disable AsyncFSWAL:
> {quote}> ...add this in hbase-site.xml
> > \{code}
> >
> > hbase.wal.provider
> > filesystem
> >
> > \{code}
> {quote}
> We are going to issue a new version of isilon-onefs-mpack (1.0.3) where we
> will have some code lines which add this property to HBASE settings. Change
> of property is applied automatically after installation of HBASE service and
> installing a new version of mpack.
> Now our code change relates to one file of code: service_advisor.py, other
> changes are cosmetic (version numbering and comments).
> As well we are going to port the code to all necessary branches of Ambari of
> older versions which support isilon-onefs-mpack.
> Business Justification: The problem we are working on is that a new feature
> of HBase 2 (AsyncFSWAL) causes worse performance with OneFS.
> Some Isilon customers which use HBASE with Ambari with OneFS suffer from this
> problem. The fix should help these customers.
> Available Maintenance Resources: HDFS team of Dell Isilon supports Ambari
> mpack project.
> Impact: Improve interaction of Isilon with Hadoop
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AMBARI-25505) New version of isilon-onefs-mpack (1.0.3)
[
https://issues.apache.org/jira/browse/AMBARI-25505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096651#comment-17096651
]
Ivan Martynov commented on AMBARI-25505:
I've tested Case #4: "Customer has manually added property to hbase settings
through UI and installs new mpack with fix."> (4) For this use case I suspect
that nothing will happen as an mpack can't
> override already existing settings but may be worth checking out. In the
> past for specific settings controlled on the OneFS UI service page an mpack
> could overwrite a customer config. I performed the following steps:
a) New deployment of HDP 3 with with Hbase service installed with mpack
1.0.2.0. (with conductor)
Stack: HDP-3.1
Name: HDP-3.1.5.0
Type: STANDARD
Version: 3.1.5.0-152
b) Add property to hbase settings through UI
c) Install mpack with fix (1.0.3)
ambari-server install-mpack --mpack=isilon-onefs-mpack-1.0.3.0.tar.gz --force
--verbose
d) Restart Ambari cluster
e) Confirm property shows up in UI under Hbase Service settings
YES. It is in UI under custom HBASE site section.
Conclusions:
1. Test passed and behavior is expected.
2. In this scenario we confirmed that customers who has manually added property
to hbase settings through UI and installs new mpack with fix didn't loose the
property
3. Mpack v.1.0.3 is ready to work under this scenario.
> New version of isilon-onefs-mpack (1.0.3)
> -
>
> Key: AMBARI-25505
> URL: https://issues.apache.org/jira/browse/AMBARI-25505
> Project: Ambari
> Issue Type: Improvement
> Components: contrib
>Affects Versions: trunk, 2.7.0
> Environment: HDP versions started from "2.6"
> Ambari started from version "2.7"
>Reporter: Ivan Martynov
>Priority: Critical
> Labels: performance
> Fix For: trunk, 2.7.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This issue is going under the direction of Dell Technologies.
> We are going to change the code of isilon-onefs-mpack
> (+[https://github.com/apache/ambari/tree/trunk/contrib/management-packs/isilon-onefs-mpack]+).
> The problem is that a new feature of HBase 2 (AsyncFSWAL) causes worse
> performance with OneFS. We need to disable this in the Ambari management pack
> by setting the property name: "hbase.wal.provider" with value: "filesystem"
> for HBASE. The way to change the property has been already introduced at:
> [https://jira.apache.org/jira/browse/HBASE-15536]
> In section about how to disable AsyncFSWAL:
> {quote}> ...add this in hbase-site.xml
> > \{code}
> >
> > hbase.wal.provider
> > filesystem
> >
> > \{code}
> {quote}
> We are going to issue a new version of isilon-onefs-mpack (1.0.3) where we
> will have some code lines which add this property to HBASE settings. Change
> of property is applied automatically after installation of HBASE service and
> installing a new version of mpack.
> Now our code change relates to one file of code: service_advisor.py, other
> changes are cosmetic (version numbering and comments).
> As well we are going to port the code to all necessary branches of Ambari of
> older versions which support isilon-onefs-mpack.
> Business Justification: The problem we are working on is that a new feature
> of HBase 2 (AsyncFSWAL) causes worse performance with OneFS.
> Some Isilon customers which use HBASE with Ambari with OneFS suffer from this
> problem. The fix should help these customers.
> Available Maintenance Resources: HDFS team of Dell Isilon supports Ambari
> mpack project.
> Impact: Improve interaction of Isilon with Hadoop
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AMBARI-25505) New version of isilon-onefs-mpack (1.0.3)
[
https://issues.apache.org/jira/browse/AMBARI-25505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096648#comment-17096648
]
Ivan Martynov commented on AMBARI-25505:
I managed to test the case with manual install (case #3 "Customer installs new
mpack with fix and selects hbase service in the installation wizard")
1. I created a HDFS cluster with command without HDP and MPack.:
conductor -b onefs-hdp3.1-4clients-ranger-kms-nohbase.json --onefs
onefs19.west.isilon.com --ducttape-owner=imartynov --skip-hdp --no-mpack
--ducttape-location=seattle
2. I installed mpack 1.0.3 with fix
3. I ran Ambari, started wizard, selected to create HBASE service. Opened
service settings (Custom hbase-site) and see the property
hbase.wal.provider is set.
4. I installed the HBASE service, run HDP cluster - OK.
5. Checked hbase.wal.provider property after running - it is there.
Now all cases have tested, I'm working on publishing the project to Open Source.
> New version of isilon-onefs-mpack (1.0.3)
> -
>
> Key: AMBARI-25505
> URL: https://issues.apache.org/jira/browse/AMBARI-25505
> Project: Ambari
> Issue Type: Improvement
> Components: contrib
>Affects Versions: trunk, 2.7.0
> Environment: HDP versions started from "2.6"
> Ambari started from version "2.7"
>Reporter: Ivan Martynov
>Priority: Critical
> Labels: performance
> Fix For: trunk, 2.7.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This issue is going under the direction of Dell Technologies.
> We are going to change the code of isilon-onefs-mpack
> (+[https://github.com/apache/ambari/tree/trunk/contrib/management-packs/isilon-onefs-mpack]+).
> The problem is that a new feature of HBase 2 (AsyncFSWAL) causes worse
> performance with OneFS. We need to disable this in the Ambari management pack
> by setting the property name: "hbase.wal.provider" with value: "filesystem"
> for HBASE. The way to change the property has been already introduced at:
> [https://jira.apache.org/jira/browse/HBASE-15536]
> In section about how to disable AsyncFSWAL:
> {quote}> ...add this in hbase-site.xml
> > \{code}
> >
> > hbase.wal.provider
> > filesystem
> >
> > \{code}
> {quote}
> We are going to issue a new version of isilon-onefs-mpack (1.0.3) where we
> will have some code lines which add this property to HBASE settings. Change
> of property is applied automatically after installation of HBASE service and
> installing a new version of mpack.
> Now our code change relates to one file of code: service_advisor.py, other
> changes are cosmetic (version numbering and comments).
> As well we are going to port the code to all necessary branches of Ambari of
> older versions which support isilon-onefs-mpack.
> Business Justification: The problem we are working on is that a new feature
> of HBase 2 (AsyncFSWAL) causes worse performance with OneFS.
> Some Isilon customers which use HBASE with Ambari with OneFS suffer from this
> problem. The fix should help these customers.
> Available Maintenance Resources: HDFS team of Dell Isilon supports Ambari
> mpack project.
> Impact: Improve interaction of Isilon with Hadoop
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AMBARI-25505) New version of isilon-onefs-mpack (1.0.3)
[
https://issues.apache.org/jira/browse/AMBARI-25505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096646#comment-17096646
]
Ivan Martynov commented on AMBARI-25505:
Test case #2: "Customer performs an mpack upgrade with fix when hbase service
already installed".
Test steps:
a) new deployment of HDP 3 with Hbase service installed with mpack 1.0.2.0.
Stack: HDP-3.1
Name: HDP-3.1.5.0
Type: STANDARD
Version: 3.1.5.0-152
Cluster: hadoop0427
Check installed mpack version:
root@dtseacli629941:~ # ls -l /var/lib/ambari-server/resources/mpacks/
total 12
drwxr-xr-x. 2 root root 4096 Apr 16 06:07 cache
-rw-r--r--. 1 root root 223 Apr 16 06:07 mpacks_replay.log
drwxr-xr-x. 3 root root 4096 Nov 21 04:11 onefs-ambari-mpack-1.0.2.0
root@dtseacli629941:~ #
Do not set the hbase.wal.provider setting:
root@dtseacli629941:/home #
root@dtseacli629941:/home # python
/var/lib/ambari-server/resources/scripts/configs.py --user=admin
--password=admin --port=8080 --action=get --host=localhost --cluster=hadoop0427
--config-type=hbase-site | grep 'hbase.wal.provider'
#
I did not set the hbase.wal.provider setting.
b) upgrade to the new mpack 1.0.3.0
root@dtseacli629941:/home/install # ambari-server upgrade-mpack
--mpack=isilon-onefs-mpack-1.0.3.0.tar.gz --verbose
...
Ambari Server 'upgrade-mpack' completed successfully.
Check upgraded mpack version:
root@dtseacli629941:/home/install # ls -l
/var/lib/ambari-server/resources/mpacks/
total 12
drwxr-xr-x. 2 root root 4096 Apr 16 09:02 cache
-rwxr-xr-x. 1 root root 421 Apr 16 09:02 mpacks_replay.log
drwxr-xr-x. 3 root root 4096 Apr 11 13:40 onefs-ambari-mpack-1.0.3.0
Restart ambari server with the following ambari-server restart
root@dtseacli629941:/home/install # ambari-server restart -v
c) restart ambari server with the following:
ambari-server restart
d) confirm property shows up in UI under OneFS Service settings or Hbase
Service settings
The property hbase.wal.provider didn't show up in UI.
This is an expected behavior, as I read in
> (2) The suggested fix shown in the branch linked in use case above will not
> automatically inject the setting because the hbase service is already
> installed. At least this is base on my understanding based on observation
> and talks with Cloudera. I briefly remember testing this use case. The only
> solution to this use case then is to have the customer manually add the
> config through the UI. e) Add property manually in UI:
hbase.wal.provider
filesystem
f) Restarted services: 4 HBase Clients, 4 RegionServers, 1 HBase Master.
Ambari suggested to restart, so I did.
h) Confirm property shows up in UI under OneFS Service settings or Hbase
Service settings
YES. It is in UI.
Conclusions:
1. Test passed and behavior is expected.
2. In this scenario we recommend customers to add property hbase.wal.provider
with value filesystem manually to HBASE settings through UI after MPack upgrade.
3. Mpack v.1.0.3 is ready to work under this scenario.
> New version of isilon-onefs-mpack (1.0.3)
> -
>
> Key: AMBARI-25505
> URL: https://issues.apache.org/jira/browse/AMBARI-25505
> Project: Ambari
> Issue Type: Improvement
> Components: contrib
>Affects Versions: trunk, 2.7.0
> Environment: HDP versions started from "2.6"
> Ambari started from version "2.7"
>Reporter: Ivan Martynov
>Priority: Critical
> Labels: performance
> Fix For: trunk, 2.7.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This issue is going under the direction of Dell Technologies.
> We are going to change the code of isilon-onefs-mpack
> (+[https://github.com/apache/ambari/tree/trunk/contrib/management-packs/isilon-onefs-mpack]+).
> The problem is that a new feature of HBase 2 (AsyncFSWAL) causes worse
> performance with OneFS. We need to disable this in the Ambari management pack
> by setting the property name: "hbase.wal.provider" with value: "filesystem"
> for HBASE. The way to change the property has been already introduced at:
> [https://jira.apache.org/jira/browse/HBASE-15536]
> In section about how to disable AsyncFSWAL:
> {quote}> ...add this in hbase-site.xml
> > \{code}
> >
> > hbase.wal.provider
> > filesystem
> >
> > \{code}
> {quote}
> We are going to issue a new version of isilon-onefs-mpack (1.0.3) where we
> will have some code lines which add this property to HBASE settings. Change
> of property is applied automatically after installation of HBASE service and
> installing a new version of mpack.
> Now our code change relates to one file of code: service_advisor.py, other
> changes are cosmetic (version numbering and comments).
> As well we are going to port the code to all necessary branches of Ambari of
> older versions which support
[jira] [Commented] (AMBARI-25505) New version of isilon-onefs-mpack (1.0.3)
[
https://issues.apache.org/jira/browse/AMBARI-25505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096641#comment-17096641
]
Ivan Martynov commented on AMBARI-25505:
I've tested Case #1: "Customer performs an mpack upgrade with fix and adds
hbase service"
It worked With mpack version 1.0.3 with updated service_advisory.py code to
inject the property when the hbase service is
installed.
I performed the following steps:
a) new deployment of HDP 3 with without Hbase service installed with mpack
1.0.2.0.
Stack: HDP-3.1
Name: HDP-3.1.5.0
Type: STANDARD
Version: 3.1.5.0-152
b) upgrade to the new mpack 1.0.3.0
ambari-server upgrade-mpack --mpack=isilon-onefs-mpack-1.0.3.0.tar.gz --verbose
c) Install HBASE service though UI
I installed a service in Ambari UI.
I didn't add any specific properties in UI
d) Confirm property shows up in UI under Hbase Service settings
YES. It is in UI under custom HBASE site section.
Conclusions:
1. Test passed and behavior is expected.
2. In this scenario we recommend customers to install MPack 1.0.3 before
installing HBASE service
3. Mpack v.1.0.3 is ready to work under this scenario.
> New version of isilon-onefs-mpack (1.0.3)
> -
>
> Key: AMBARI-25505
> URL: https://issues.apache.org/jira/browse/AMBARI-25505
> Project: Ambari
> Issue Type: Improvement
> Components: contrib
>Affects Versions: trunk, 2.7.0
> Environment: HDP versions started from "2.6"
> Ambari started from version "2.7"
>Reporter: Ivan Martynov
>Priority: Critical
> Labels: performance
> Fix For: trunk, 2.7.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This issue is going under the direction of Dell Technologies.
> We are going to change the code of isilon-onefs-mpack
> (+[https://github.com/apache/ambari/tree/trunk/contrib/management-packs/isilon-onefs-mpack]+).
> The problem is that a new feature of HBase 2 (AsyncFSWAL) causes worse
> performance with OneFS. We need to disable this in the Ambari management pack
> by setting the property name: "hbase.wal.provider" with value: "filesystem"
> for HBASE. The way to change the property has been already introduced at:
> [https://jira.apache.org/jira/browse/HBASE-15536]
> In section about how to disable AsyncFSWAL:
> {quote}> ...add this in hbase-site.xml
> > \{code}
> >
> > hbase.wal.provider
> > filesystem
> >
> > \{code}
> {quote}
> We are going to issue a new version of isilon-onefs-mpack (1.0.3) where we
> will have some code lines which add this property to HBASE settings. Change
> of property is applied automatically after installation of HBASE service and
> installing a new version of mpack.
> Now our code change relates to one file of code: service_advisor.py, other
> changes are cosmetic (version numbering and comments).
> As well we are going to port the code to all necessary branches of Ambari of
> older versions which support isilon-onefs-mpack.
> Business Justification: The problem we are working on is that a new feature
> of HBase 2 (AsyncFSWAL) causes worse performance with OneFS.
> Some Isilon customers which use HBASE with Ambari with OneFS suffer from this
> problem. The fix should help these customers.
> Available Maintenance Resources: HDFS team of Dell Isilon supports Ambari
> mpack project.
> Impact: Improve interaction of Isilon with Hadoop
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AMBARI-25505) New version of isilon-onefs-mpack (1.0.3)
[
https://issues.apache.org/jira/browse/AMBARI-25505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096638#comment-17096638
]
Ivan Martynov commented on AMBARI-25505:
Tests provided:
There are four use cases:
1) Customer performs an mpack upgrade with fix and adds hbase service
2) Customer performs an mpack upgrade with fix when hbase service already
installed
3) Customer installs new mpack with fix and selects hbase service in the
installation wizard
4) Customer has manually added property to hbase settings through UI and
installs new mpack with fix.
(1) This use case should hopefully be addressed by updating the
service_advisory.py code to inject the property when the hbase service is
installed. The service_advisory code essentially allows us to make custom
modifications to a service when it is first installed.
When testing this case it will be important to also check, after the
injecting the hbase service setting, whether or not the hbase config shows
up in the UI.
(2) The suggested fix shown in the branch linked in use case above will not
automatically inject the setting because the hbase service is already
installed. At least this is base on my understanding based on observation
and talks with Cloudera. I briefly remember testing this use case. The only
solution to this use case then is to have the customer manually add the
config through the UI.
(3) This use case should be addressed by solution to (1) but is necessary
to check as this is what a new customer would go through who already wants
hbase at wizard installation time.
(4) For this use case I suspect that nothing will happen as an mpack can't
override already existing settings but may be worth checking out. In the
past for specific settings controlled on the OneFS UI service page an mpack
could overwrite a customer config.
While testing this, whenever an mpack is upgraded ambari-server does a
check to see if the new mpack has a higher version. When debugging and
trying mpacks in succession (say you wanted to add logging to
service_advisory.py to see what was happening) it is necessary to bump the
mpack version here
[https://github.com/apache/ambari/blob/trunk/contrib/management-packs/isilon-onefs-mpack/src/main/resources/mpack.json#L4]
for new mpacks to take effect. Note that if you do want to add extra
logging to the service_advisory.py file to debug, they too should show up
/var/log/ambari-server/ambari-server.log (if I remember correctly).
> New version of isilon-onefs-mpack (1.0.3)
> -
>
> Key: AMBARI-25505
> URL: https://issues.apache.org/jira/browse/AMBARI-25505
> Project: Ambari
> Issue Type: Improvement
> Components: contrib
>Affects Versions: trunk, 2.7.0
> Environment: HDP versions started from "2.6"
> Ambari started from version "2.7"
>Reporter: Ivan Martynov
>Priority: Critical
> Labels: performance
> Fix For: trunk, 2.7.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This issue is going under the direction of Dell Technologies.
> We are going to change the code of isilon-onefs-mpack
> (+[https://github.com/apache/ambari/tree/trunk/contrib/management-packs/isilon-onefs-mpack]+).
> The problem is that a new feature of HBase 2 (AsyncFSWAL) causes worse
> performance with OneFS. We need to disable this in the Ambari management pack
> by setting the property name: "hbase.wal.provider" with value: "filesystem"
> for HBASE. The way to change the property has been already introduced at:
> [https://jira.apache.org/jira/browse/HBASE-15536]
> In section about how to disable AsyncFSWAL:
> {quote}> ...add this in hbase-site.xml
> > \{code}
> >
> > hbase.wal.provider
> > filesystem
> >
> > \{code}
> {quote}
> We are going to issue a new version of isilon-onefs-mpack (1.0.3) where we
> will have some code lines which add this property to HBASE settings. Change
> of property is applied automatically after installation of HBASE service and
> installing a new version of mpack.
> Now our code change relates to one file of code: service_advisor.py, other
> changes are cosmetic (version numbering and comments).
> As well we are going to port the code to all necessary branches of Ambari of
> older versions which support isilon-onefs-mpack.
> Business Justification: The problem we are working on is that a new feature
> of HBase 2 (AsyncFSWAL) causes worse performance with OneFS.
> Some Isilon customers which use HBASE with Ambari with OneFS suffer from this
> problem. The fix should help these customers.
> Available Maintenance Resources: HDFS team of Dell Isilon supports Ambari
> mpack project.
> Impact: Improve interaction of Isilon with Hadoop
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
