Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-21 Thread Jeff Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/
---

(Updated June 22, 2016, 12:06 a.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Changes
---

update the unit test


Bugs: AMBARI-16920
https://issues.apache.org/jira/browse/AMBARI-16920


Repository: ambari


Description
---

This a followup ticket for add spark2 stack definition. There's serveral issues:
1.  Spark2 thrift server can not started due to miss of 
spark-thrift-fairscheduler.xml
2.  Miss of add spark2 cache file in copy_barball.py
3.  Miss the role_commnad_order of spark2
4.  conf_select is missign for spark2


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 4eb0015 
  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 286df8d 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 be99edd 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 c2385df 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f6011b0 
  
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
 6c7fe18 

Diff: https://reviews.apache.org/r/47941/diff/


Testing
---

Manually verified.


Thanks,

Jeff Zhang

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-17 Thread Jayush Luniya 


> On June 17, 2016, 2:57 p.m., Jayush Luniya wrote:
> > Ship It!
> 
> Jayush Luniya wrote:
> Update patch in CR with 
> https://issues.apache.org/jira/secure/attachment/12811383/AMBARI-16920-7.patch

Also close CR once done


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review138226
---


On June 16, 2016, 11:06 p.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 16, 2016, 11:06 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 4.  conf_select is missign for spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  4eb0015 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  be99edd 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f6011b0 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
>  6c7fe18 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-17 Thread Jayush Luniya 


> On June 17, 2016, 2:57 p.m., Jayush Luniya wrote:
> > Ship It!

Update patch in CR with 
https://issues.apache.org/jira/secure/attachment/12811383/AMBARI-16920-7.patch


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review138226
---


On June 16, 2016, 11:06 p.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 16, 2016, 11:06 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 4.  conf_select is missign for spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  4eb0015 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  be99edd 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f6011b0 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
>  6c7fe18 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-17 Thread Jayush Luniya 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review138226
---


Ship it!




Ship It!

- Jayush Luniya


On June 16, 2016, 11:06 p.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 16, 2016, 11:06 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 4.  conf_select is missign for spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  4eb0015 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  be99edd 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f6011b0 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
>  6c7fe18 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-17 Thread Jayush Luniya 


> On June 16, 2016, 6:15 p.m., Jayush Luniya wrote:
> > This is causing unit test failures. Please take a look.
> > 
> > ---
> > Failed tests:
> > FAIL: test_hook_default_conf_select 
> > (test_after_install.TestHookAfterInstall)
> > --
> > Traceback (most recent call last):
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
> >  line 1199, in patched
> > return func(*args, **keywargs)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
> >  line 109, in test_hook_default_conf_select
> > sudo = True,)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
> >  line 278, in assertResourceCalled
> > self.assertEquals(name, resource.name)
> > AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
> > '-R', '-p', '/etc/kafka...
> > 
> > First differing element 3:
> > /etc/pig/conf
> > /etc/kafka/conf
> > 
> > - ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
> > ?  ^^^  ^^^
> > 
> > + ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
> > ?  ^  ^
> > 
> > 
> > FAIL: test_hook_default_conf_select_suspended 
> > (test_after_install.TestHookAfterInstall)
> > --
> > Traceback (most recent call last):
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
> >  line 1199, in patched
> > return func(*args, **keywargs)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
> >  line 695, in test_hook_default_conf_select_suspended
> > sudo = True,)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
> >  line 278, in assertResourceCalled
> > self.assertEquals(name, resource.name)
> > AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
> > '-R', '-p', '/etc/kafka...
> > 
> > First differing element 3:
> > /etc/pig/conf
> > /etc/kafka/conf
> > 
> > - ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
> > ?  ^^^  ^^^
> > 
> > + ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
> > ?  ^  ^
> > 
> > 
> > FAIL: test_hook_default_conf_select_with_error 
> > (test_after_install.TestHookAfterInstall)
> > --
> > Traceback (most recent call last):
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
> >  line 1199, in patched
> > return func(*args, **keywargs)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
> >  line 366, in test_hook_default_conf_select_with_error
> > sudo = True,)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
> >  line 278, in assertResourceCalled
> > self.assertEquals(name, resource.name)
> > AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
> > '-R', '-p', '/etc/kafka...
> > 
> > First differing element 3:
> > /etc/pig/conf
> > /etc/kafka/conf
> > 
> > - ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
> > ?  ^^^  ^^^
> > 
> > + ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
> > ?  ^  ^
> > 
> > 
> > --
> > Total run:1062
> > Total errors:0
> > Total failures:3
> > ERROR
> > INFO: AMBARI_SERVER_LIB is not set, using default /usr/lib/ambari-server
> > INFO: Return code from stack upgrade command, retcode = 0
> > StackAdvisor implementation for stack HDP1, version 2.0.6 was not found
> > Returning DefaultStackAdvisor implementation
> > StackAdvisor implementation for stack XYZ, version 1.0.0 was loaded
> > StackAdvisor implementation for stack XYZ, version 1.0.1 was loaded
> > Returning XYZ101StackAdvisor implementation
> > [INFO] 
> > 
> > [INFO] BUILD FAILURE
> > [INFO] 
> > 
> > [INFO] Total time: 1:15.808s
> > [INFO] Finished at: Thu Jun 16 09:56:48 PDT 2016
> > [INFO] Final Memory: 60M/837M
> > [INFO] 
> > 
> 
> Jeff Zhang wrote:
> I don't have clues why

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-17 Thread Jayush Luniya 


> On June 16, 2016, 6:15 p.m., Jayush Luniya wrote:
> > This is causing unit test failures. Please take a look.
> > 
> > ---
> > Failed tests:
> > FAIL: test_hook_default_conf_select 
> > (test_after_install.TestHookAfterInstall)
> > --
> > Traceback (most recent call last):
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
> >  line 1199, in patched
> > return func(*args, **keywargs)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
> >  line 109, in test_hook_default_conf_select
> > sudo = True,)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
> >  line 278, in assertResourceCalled
> > self.assertEquals(name, resource.name)
> > AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
> > '-R', '-p', '/etc/kafka...
> > 
> > First differing element 3:
> > /etc/pig/conf
> > /etc/kafka/conf
> > 
> > - ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
> > ?  ^^^  ^^^
> > 
> > + ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
> > ?  ^  ^
> > 
> > 
> > FAIL: test_hook_default_conf_select_suspended 
> > (test_after_install.TestHookAfterInstall)
> > --
> > Traceback (most recent call last):
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
> >  line 1199, in patched
> > return func(*args, **keywargs)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
> >  line 695, in test_hook_default_conf_select_suspended
> > sudo = True,)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
> >  line 278, in assertResourceCalled
> > self.assertEquals(name, resource.name)
> > AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
> > '-R', '-p', '/etc/kafka...
> > 
> > First differing element 3:
> > /etc/pig/conf
> > /etc/kafka/conf
> > 
> > - ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
> > ?  ^^^  ^^^
> > 
> > + ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
> > ?  ^  ^
> > 
> > 
> > FAIL: test_hook_default_conf_select_with_error 
> > (test_after_install.TestHookAfterInstall)
> > --
> > Traceback (most recent call last):
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
> >  line 1199, in patched
> > return func(*args, **keywargs)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
> >  line 366, in test_hook_default_conf_select_with_error
> > sudo = True,)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
> >  line 278, in assertResourceCalled
> > self.assertEquals(name, resource.name)
> > AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
> > '-R', '-p', '/etc/kafka...
> > 
> > First differing element 3:
> > /etc/pig/conf
> > /etc/kafka/conf
> > 
> > - ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
> > ?  ^^^  ^^^
> > 
> > + ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
> > ?  ^  ^
> > 
> > 
> > --
> > Total run:1062
> > Total errors:0
> > Total failures:3
> > ERROR
> > INFO: AMBARI_SERVER_LIB is not set, using default /usr/lib/ambari-server
> > INFO: Return code from stack upgrade command, retcode = 0
> > StackAdvisor implementation for stack HDP1, version 2.0.6 was not found
> > Returning DefaultStackAdvisor implementation
> > StackAdvisor implementation for stack XYZ, version 1.0.0 was loaded
> > StackAdvisor implementation for stack XYZ, version 1.0.1 was loaded
> > Returning XYZ101StackAdvisor implementation
> > [INFO] 
> > 
> > [INFO] BUILD FAILURE
> > [INFO] 
> > 
> > [INFO] Total time: 1:15.808s
> > [INFO] Finished at: Thu Jun 16 09:56:48 PDT 2016
> > [INFO] Final Memory: 60M/837M
> > [INFO] 
> > 
> 
> Jeff Zhang wrote:
> I don't have clues why

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-16 Thread Jeff Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/
---

(Updated June 16, 2016, 11:06 p.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Changes
---

Fix unit test, but not sure how to run.


Bugs: AMBARI-16920
https://issues.apache.org/jira/browse/AMBARI-16920


Repository: ambari


Description
---

This a followup ticket for add spark2 stack definition. There's serveral issues:
1.  Spark2 thrift server can not started due to miss of 
spark-thrift-fairscheduler.xml
2.  Miss of add spark2 cache file in copy_barball.py
3.  Miss the role_commnad_order of spark2
4.  conf_select is missign for spark2


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 4eb0015 
  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 286df8d 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 be99edd 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 c2385df 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f6011b0 
  
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
 6c7fe18 

Diff: https://reviews.apache.org/r/47941/diff/


Testing
---

Manually verified.


Thanks,

Jeff Zhang

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-16 Thread Jeff Zhang 


> On June 16, 2016, 6:15 p.m., Jayush Luniya wrote:
> > This is causing unit test failures. Please take a look.
> > 
> > ---
> > Failed tests:
> > FAIL: test_hook_default_conf_select 
> > (test_after_install.TestHookAfterInstall)
> > --
> > Traceback (most recent call last):
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
> >  line 1199, in patched
> > return func(*args, **keywargs)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
> >  line 109, in test_hook_default_conf_select
> > sudo = True,)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
> >  line 278, in assertResourceCalled
> > self.assertEquals(name, resource.name)
> > AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
> > '-R', '-p', '/etc/kafka...
> > 
> > First differing element 3:
> > /etc/pig/conf
> > /etc/kafka/conf
> > 
> > - ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
> > ?  ^^^  ^^^
> > 
> > + ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
> > ?  ^  ^
> > 
> > 
> > FAIL: test_hook_default_conf_select_suspended 
> > (test_after_install.TestHookAfterInstall)
> > --
> > Traceback (most recent call last):
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
> >  line 1199, in patched
> > return func(*args, **keywargs)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
> >  line 695, in test_hook_default_conf_select_suspended
> > sudo = True,)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
> >  line 278, in assertResourceCalled
> > self.assertEquals(name, resource.name)
> > AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
> > '-R', '-p', '/etc/kafka...
> > 
> > First differing element 3:
> > /etc/pig/conf
> > /etc/kafka/conf
> > 
> > - ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
> > ?  ^^^  ^^^
> > 
> > + ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
> > ?  ^  ^
> > 
> > 
> > FAIL: test_hook_default_conf_select_with_error 
> > (test_after_install.TestHookAfterInstall)
> > --
> > Traceback (most recent call last):
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
> >  line 1199, in patched
> > return func(*args, **keywargs)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
> >  line 366, in test_hook_default_conf_select_with_error
> > sudo = True,)
> >   File 
> > "/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
> >  line 278, in assertResourceCalled
> > self.assertEquals(name, resource.name)
> > AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
> > '-R', '-p', '/etc/kafka...
> > 
> > First differing element 3:
> > /etc/pig/conf
> > /etc/kafka/conf
> > 
> > - ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
> > ?  ^^^  ^^^
> > 
> > + ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
> > ?  ^  ^
> > 
> > 
> > --
> > Total run:1062
> > Total errors:0
> > Total failures:3
> > ERROR
> > INFO: AMBARI_SERVER_LIB is not set, using default /usr/lib/ambari-server
> > INFO: Return code from stack upgrade command, retcode = 0
> > StackAdvisor implementation for stack HDP1, version 2.0.6 was not found
> > Returning DefaultStackAdvisor implementation
> > StackAdvisor implementation for stack XYZ, version 1.0.0 was loaded
> > StackAdvisor implementation for stack XYZ, version 1.0.1 was loaded
> > Returning XYZ101StackAdvisor implementation
> > [INFO] 
> > 
> > [INFO] BUILD FAILURE
> > [INFO] 
> > 
> > [INFO] Total time: 1:15.808s
> > [INFO] Finished at: Thu Jun 16 09:56:48 PDT 2016
> > [INFO] Final Memory: 60M/837M
> > [INFO] 
> > 

I don't have clues why my changes cause unit test

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-16 Thread Jayush Luniya 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review138042
---



This is causing unit test failures. Please take a look.

---
Failed tests:
FAIL: test_hook_default_conf_select (test_after_install.TestHookAfterInstall)
--
Traceback (most recent call last):
  File 
"/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
 line 1199, in patched
return func(*args, **keywargs)
  File 
"/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
 line 109, in test_hook_default_conf_select
sudo = True,)
  File 
"/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
 line 278, in assertResourceCalled
self.assertEquals(name, resource.name)
AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
'-R', '-p', '/etc/kafka...

First differing element 3:
/etc/pig/conf
/etc/kafka/conf

- ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
?  ^^^  ^^^

+ ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
?  ^  ^


FAIL: test_hook_default_conf_select_suspended 
(test_after_install.TestHookAfterInstall)
--
Traceback (most recent call last):
  File 
"/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
 line 1199, in patched
return func(*args, **keywargs)
  File 
"/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
 line 695, in test_hook_default_conf_select_suspended
sudo = True,)
  File 
"/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
 line 278, in assertResourceCalled
self.assertEquals(name, resource.name)
AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
'-R', '-p', '/etc/kafka...

First differing element 3:
/etc/pig/conf
/etc/kafka/conf

- ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
?  ^^^  ^^^

+ ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
?  ^  ^


FAIL: test_hook_default_conf_select_with_error 
(test_after_install.TestHookAfterInstall)
--
Traceback (most recent call last):
  File 
"/Users/jluniya/release/trunk2/ambari/ambari-common/src/test/python/mock/mock.py",
 line 1199, in patched
return func(*args, **keywargs)
  File 
"/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py",
 line 366, in test_hook_default_conf_select_with_error
sudo = True,)
  File 
"/Users/jluniya/release/trunk2/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py",
 line 278, in assertResourceCalled
self.assertEquals(name, resource.name)
AssertionError: Tuples differ: ('cp', '-R', '-p', '/etc/pig/c... != ('cp', 
'-R', '-p', '/etc/kafka...

First differing element 3:
/etc/pig/conf
/etc/kafka/conf

- ('cp', '-R', '-p', '/etc/pig/conf', '/etc/pig/conf.backup')
?  ^^^  ^^^

+ ('cp', '-R', '-p', '/etc/kafka/conf', '/etc/kafka/conf.backup')
?  ^  ^


--
Total run:1062
Total errors:0
Total failures:3
ERROR
INFO: AMBARI_SERVER_LIB is not set, using default /usr/lib/ambari-server
INFO: Return code from stack upgrade command, retcode = 0
StackAdvisor implementation for stack HDP1, version 2.0.6 was not found
Returning DefaultStackAdvisor implementation
StackAdvisor implementation for stack XYZ, version 1.0.0 was loaded
StackAdvisor implementation for stack XYZ, version 1.0.1 was loaded
Returning XYZ101StackAdvisor implementation
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 1:15.808s
[INFO] Finished at: Thu Jun 16 09:56:48 PDT 2016
[INFO] Final Memory: 60M/837M
[INFO] 

- Jayush Luniya


On June 15, 2016, 11:06 p.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 15, 2016, 11:06 p.m.)
> 
> 
>

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-15 Thread Jeff Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/
---

(Updated June 15, 2016, 11:06 p.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Changes
---

Patch rebased


Bugs: AMBARI-16920
https://issues.apache.org/jira/browse/AMBARI-16920


Repository: ambari


Description
---

This a followup ticket for add spark2 stack definition. There's serveral issues:
1.  Spark2 thrift server can not started due to miss of 
spark-thrift-fairscheduler.xml
2.  Miss of add spark2 cache file in copy_barball.py
3.  Miss the role_commnad_order of spark2
4.  conf_select is missign for spark2


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 4eb0015 
  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 286df8d 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 be99edd 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 c2385df 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f6011b0 

Diff: https://reviews.apache.org/r/47941/diff/


Testing
---

Manually verified.


Thanks,

Jeff Zhang

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-15 Thread Jayush Luniya 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review137788
---



@Jeff
Please rebase patch with latest in trunk. There are conflicts. Will commit once 
I have the latest patch.

- Jayush Luniya


On June 13, 2016, 10:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 13, 2016, 10:45 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 4.  conf_select is missign for spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  4eb0015 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  6925ab5 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f6011b0 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-15 Thread Jayush Luniya 


> On June 2, 2016, 4:42 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py,
> >  line 1
> > 
> >
> > Please add unit tests as mentioned in AMBARI-16864 for SPARK2 service.

There is a patch for UTs


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review135955
---


On June 13, 2016, 10:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 13, 2016, 10:45 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 4.  conf_select is missign for spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  4eb0015 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  6925ab5 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f6011b0 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-13 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review137374
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 13, 2016, 10:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 13, 2016, 10:45 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 4.  conf_select is missign for spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  4eb0015 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  6925ab5 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f6011b0 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-13 Thread Jeff Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/
---

(Updated June 13, 2016, 10:45 a.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Bugs: AMBARI-16920
https://issues.apache.org/jira/browse/AMBARI-16920


Repository: ambari


Description (updated)
---

This a followup ticket for add spark2 stack definition. There's serveral issues:
1.  Spark2 thrift server can not started due to miss of 
spark-thrift-fairscheduler.xml
2.  Miss of add spark2 cache file in copy_barball.py
3.  Miss the role_commnad_order of spark2
4.  conf_select is missign for spark2


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 4eb0015 
  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 286df8d 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 6925ab5 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 c2385df 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f6011b0 

Diff: https://reviews.apache.org/r/47941/diff/


Testing
---

Manually verified.


Thanks,

Jeff Zhang

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-13 Thread Jeff Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/
---

(Updated June 13, 2016, 10:28 a.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Changes
---

address comments and rebase patch


Summary (updated)
-

[AMBARI-16920] Follow up issue for Spark2 stack definition


Bugs: AMBARI-16920
https://issues.apache.org/jira/browse/AMBARI-16920


Repository: ambari


Description
---

This a followup ticket for add spark2 stack definition. There's serveral issues:
1.  Spark2 thrift server can not started due to miss of 
spark-thrift-fairscheduler.xml
2.  Miss of add spark2 cache file in copy_barball.py
3.  Miss the role_commnad_order of spark2


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 4eb0015 
  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 286df8d 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 6925ab5 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 c2385df 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f6011b0 

Diff: https://reviews.apache.org/r/47941/diff/


Testing
---

Manually verified.


Thanks,

Jeff Zhang