Ivan Martynov commented on AMBARI-25505:

Tests provided:
There are four use cases:

1) Customer performs an mpack upgrade with fix and adds hbase service

2) Customer performs an mpack upgrade with fix when hbase service already 

3) Customer installs new mpack with fix and selects hbase service in the 
installation wizard

4) Customer has manually added property to hbase settings through UI and 
installs new mpack with fix. 

(1) This use case should hopefully be addressed by updating the 
service_advisory.py code to inject the property when the hbase service is 
installed. The service_advisory code essentially allows us to make custom 
modifications to a service when it is first installed.  
When testing this case it will be important to also check, after the 
injecting the hbase service setting, whether or not the hbase config shows 
up in the UI.

(2) The suggested fix shown in the branch linked in use case above will not 
automatically inject the setting because the hbase service is already 
installed. At least this is base on my understanding based on observation 
and talks with Cloudera. I briefly remember testing this use case. The only 
solution to this use case then is to have the customer manually add the 
config through the UI. 

(3) This use case should be addressed by solution to (1) but is necessary 
to check as this is what a new customer would go through who already wants 
hbase at wizard installation time. 

(4) For this use case I suspect that nothing will happen as an mpack can't 
override already existing settings but may be worth checking out. In the 
past for specific settings controlled on the OneFS UI service page an mpack 
could overwrite a customer config. 

While testing this, whenever an mpack is upgraded ambari-server does a 
check to see if the new mpack has a higher version. When debugging and 
trying mpacks in succession (say you wanted to add logging to 
service_advisory.py to see what was happening) it is necessary to bump the 
mpack version here 
 for new mpacks to take effect. Note that if you do want to add extra 
logging to the service_advisory.py file to debug, they too should show up 
/var/log/ambari-server/ambari-server.log (if I remember correctly).

> New version of isilon-onefs-mpack (1.0.3)
> -----------------------------------------
>                 Key: AMBARI-25505
>                 URL: https://issues.apache.org/jira/browse/AMBARI-25505
>             Project: Ambari
>          Issue Type: Improvement
>          Components: contrib
>    Affects Versions: trunk, 2.7.0
>         Environment: HDP versions started from  "2.6"
> Ambari started from version "2.7"
>            Reporter: Ivan Martynov
>            Priority: Critical
>              Labels: performance
>             Fix For: trunk, 2.7.0
>   Original Estimate: 168h
>  Remaining Estimate: 168h
> This issue is going under the direction of Dell Technologies.
> We are going to change the code of isilon-onefs-mpack 
> (+[https://github.com/apache/ambari/tree/trunk/contrib/management-packs/isilon-onefs-mpack]+).
> The problem is that a new feature of HBase 2 (AsyncFSWAL) causes worse 
> performance with OneFS. We need to disable this in the Ambari management pack 
> by setting the property name: "hbase.wal.provider" with value: "filesystem" 
> for HBASE. The way to change the property has been already introduced at:
> [https://jira.apache.org/jira/browse/HBASE-15536]
> In section about how to disable AsyncFSWAL:
> {quote}> ...add this in hbase-site.xml
> > \{code}
> >  <property>
> >   <name>hbase.wal.provider</name>
> >   <value>filesystem</value>
> >  </property>
> > \{code}
> {quote}
>  We are going to issue a new version of isilon-onefs-mpack (1.0.3) where we 
> will have some code lines which add this property to HBASE settings. Change 
> of property is applied automatically after installation of HBASE service and 
> installing a new version of mpack.
> Now our code change relates to one file of code: service_advisor.py, other 
> changes are cosmetic (version numbering and comments).
> As well we are going to port the code to all necessary branches of Ambari of 
> older versions which support isilon-onefs-mpack.
> Business Justification: The problem we are working on is that a new feature 
> of HBase 2 (AsyncFSWAL) causes worse performance with OneFS.
> Some Isilon customers which use HBASE with Ambari with OneFS suffer from this 
> problem. The fix should help these customers.
> Available Maintenance Resources: HDFS team of Dell Isilon supports Ambari 
> mpack project.
> Impact: Improve interaction of Isilon with Hadoop

This message was sent by Atlassian Jira

Reply via email to