Review Request 63138: Update Spark2 log4j default settings to latest

2017-10-18 Thread Saisai Shao
/diff/1/ Testing --- Manually verification on refresh install of HDP 2.6 and upgrade from HDP 2.5. Thanks, Saisai Shao

Re: Review Request 59992: AMBARI-21226. Add two new Spark thrift server related configurations in Spark stack

2017-06-13 Thread Saisai Shao
> On June 13, 2017, 3:18 p.m., Sumit Mohanty wrote: > > One more thing: If user upgrades from HDP-2.6.x to 2.6.2, do you want the > > configs to be added? If yes then that upgrade xml change is needed. > > Saisai Shao wrote: > Yes, I want. So do I only need to chang

Re: Review Request 59992: AMBARI-21226. Add two new Spark thrift server related configurations in Spark stack

2017-06-13 Thread Saisai Shao
> On June 13, 2017, 3:18 p.m., Sumit Mohanty wrote: > > One more thing: If user upgrades from HDP-2.6.x to 2.6.2, do you want the > > configs to be added? If yes then that upgrade xml change is needed. > > Saisai Shao wrote: > Yes, I want. So do I only need to chang

Re: Review Request 59992: AMBARI-21226. Add two new Spark thrift server related configurations in Spark stack

2017-06-13 Thread Saisai Shao
on: updating HDP from 2.3 -> 2.6, 2.4 -> 2.6, 2.5 -> 2.6, and fresh install of 2.6 Thanks, Saisai Shao

Re: Review Request 59992: AMBARI-21226. Add two new Spark thrift server related configurations in Spark stack

2017-06-13 Thread Saisai Shao
des? - Saisai --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59992/#review177746 --- On June 13, 2017, 5:44 a.m., Sa

Re: Review Request 59992: AMBARI-21226. Add two new Spark thrift server related configurations in Spark stack

2017-06-12 Thread Saisai Shao
--- Test is done manually on: updating HDP from 2.3 -> 2.6, 2.4 -> 2.6, 2.5 -> 2.6, and fresh install of 2.6 Thanks, Saisai Shao

Re: Review Request 59992: AMBARI-21226. Add two new Spark thrift server related configurations in Spark stack

2017-06-12 Thread Saisai Shao
uring upgrading? - Saisai --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59992/#review177621 --- On June 12, 2017, 8

Review Request 59992: AMBARI-21226. Add two new Spark thrift server related configurations in Spark stack

2017-06-12 Thread Saisai Shao
/spark2-thrift-sparkconf.xml PRE-CREATION Diff: https://reviews.apache.org/r/59992/diff/1/ Testing --- Test is done manually on: updating HDP from 2.3 -> 2.6, 2.4 -> 2.6, 2.5 -> 2.6, and fresh install of 2.6 Thanks, Saisai Shao

Re: Review Request 59305: Fix Livy service check and alerts script with SSL enabled

2017-05-16 Thread Saisai Shao
ly, visit: https://reviews.apache.org/r/59305/#review175106 ------- On May 16, 2017, 5:58 a.m., Saisai Shao wrote: > > --- > This is an automatically generated e-mail. To re

Review Request 59305: Fix Livy service check and alerts script with SSL enabled

2017-05-15 Thread Saisai Shao
ces/SPARK2/2.0.0/package/scripts/service_check.py 8e7a766 Diff: https://reviews.apache.org/r/59305/diff/1/ Testing --- Local verification is done. Thanks, Saisai Shao

Review Request 56641: Change Livy recovery folder permission to 0700

2017-02-13 Thread Saisai Shao
--- Manual verification. Thanks, Saisai Shao

Re: Review Request 55609: Add Livy session recovery configurations in Ambari

2017-01-25 Thread Saisai Shao
--- Manual verification. Thanks, Saisai Shao

Re: Review Request 55609: Add Livy session recovery configurations in Ambari

2017-01-25 Thread Saisai Shao
; > Does this need the attribute of "directory"? > > Saisai Shao wrote: > I'm not sure the usage of attribute "directory", but this configuration > do means a directory, and will be created in livy setup. But one thing should be noted that this directory is a

Re: Review Request 55609: Add Livy session recovery configurations in Ambari

2017-01-25 Thread Saisai Shao
"directory", but this configuration do means a directory, and will be created in livy setup. - Saisai --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55609/#review1

Re: Review Request 55609: Add Livy session recovery configurations in Ambari

2017-01-24 Thread Saisai Shao
--- Manual verification. Thanks, Saisai Shao

Re: Review Request 55609: Add Livy session recovery configurations in Ambari

2017-01-23 Thread Saisai Shao
> On Jan. 19, 2017, 4:59 a.m., Bikas Saha wrote: > > Should the URL for recovery be something like > > /apps/{livy|spark}/livy-recovery? Having /livy-recovery as a top level HDFS > > dir seems distracting. > > Saisai Shao wrote: > It will not

Re: Review Request 55609: Add Livy session recovery configurations in Ambari

2017-01-23 Thread Saisai Shao
> On Jan. 19, 2017, 4:59 a.m., Bikas Saha wrote: > > Should the URL for recovery be something like > > /apps/{livy|spark}/livy-recovery? Having /livy-recovery as a top level HDFS > > dir seems distracting. > > Saisai Shao wrote: > It will not

Re: Review Request 55609: Add Livy session recovery configurations in Ambari

2017-01-18 Thread Saisai Shao
-- On Jan. 18, 2017, 2:25 a.m., Saisai Shao wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55609/ > ---

Review Request 55609: Add Livy session recovery configurations in Ambari

2017-01-17 Thread Saisai Shao
/HDP/2.6/services/SPARK2/configuration/livy2-conf.xml 457869d Diff: https://reviews.apache.org/r/55609/diff/ Testing --- Manual verification. Thanks, Saisai Shao

Re: Review Request 49484: Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition (2.4/2.5 branch)

2016-10-19 Thread Saisai Shao
--- Local verification. Thanks, Saisai Shao

Re: Review Request 49484: Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition

2016-10-19 Thread Saisai Shao
-upgrade-2.5.xml 3478603 ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 826cf16 Diff: https://reviews.apache.org/r/49484/diff/ Testing --- Local verification. Thanks, Saisai Shao

Re: Review Request 49484: Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition

2016-10-17 Thread Saisai Shao
-upgrade-2.5.xml 0a14581 ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml fa1a555 Diff: https://reviews.apache.org/r/49484/diff/ Testing --- Local verification. Thanks, Saisai Shao

Review Request 50945: Fix Spark2 service check failure when WE is enabled

2016-08-09 Thread Saisai Shao
/resources/common-services/SPARK2/2.0.0/package/scripts/service_check.py 565f924 Diff: https://reviews.apache.org/r/50945/diff/ Testing --- Manual verification is done. Thanks, Saisai Shao

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

2016-08-02 Thread Saisai Shao
. Thanks, Saisai Shao

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

2016-08-01 Thread Saisai Shao
with my hotfix the config > > will remain -Dhdp.version={{hdp_full_version}}. > > Saisai Shao wrote: > From my understanding, you mean that in the params.py we should also take > care of {{hdp_full_version}} if amabri is upgraded from lower version. Can > you please explain more

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

2016-08-01 Thread Saisai Shao
get the specific version of Ambari and how to upgrade to the specific version? - Saisai --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50594/#review144425 --

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

2016-08-01 Thread Saisai Shao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50594/#review144418 ------- On Aug. 1, 2016, 1:22 a.m., Saisai Shao wrote: > > -

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

2016-08-01 Thread Saisai Shao
e addition of -Dhdp.version should also be under condition > > check_stack_feature(StackFeature.SPARK_JAVA_OPTS_SUPPORT, > > effective_version). > > > > I assume -Dhdp.version is to be added only for HDP-2.3 and below. > > Saisai Shao wrote: >

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

2016-08-01 Thread Saisai Shao
------- On Aug. 1, 2016, 1:22 a.m., Saisai Shao wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50594/ > ---

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

2016-07-31 Thread Saisai Shao
/test/python/stacks/2.3/SPARK/test_spark_thrift_server.py a1abdfa Diff: https://reviews.apache.org/r/50594/diff/ Testing --- Manual test with different scenarios: 1. Fresh install of HDP 2.3.6, 2.4.3, 2.5.0 2. Upgrade for 2.3.6 to 2.5.0. 3. Downgrade from 2.5.0 to 2.3.6. Thanks, Saisai

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

2016-07-31 Thread Saisai Shao
>action="delete" > >) Done - Saisai ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50594/#review144288

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

2016-07-31 Thread Saisai Shao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50594/#review144288 ------- On July 31, 2016, 3:44 a.m., Saisai Shao wrote: > > -

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

2016-07-30 Thread Saisai Shao
/test/python/stacks/2.3/SPARK/test_spark_thrift_server.py a1abdfa Diff: https://reviews.apache.org/r/50594/diff/ Testing --- Manual test with different scenarios: 1. Fresh install of HDP 2.3.6, 2.4.3, 2.5.0 2. Upgrade for 2.3.6 to 2.5.0. 3. Downgrade from 2.5.0 to 2.3.6. Thanks, Saisai

Review Request 49584: Remove spark.driver.extraJavaOptions and spark.yarn.am.extraJavaOptions property in upgrading to HDP 2.5

2016-07-03 Thread Saisai Shao
/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml 00dbaa7 ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 68e8fc2 Diff: https://reviews.apache.org/r/49584/diff/ Testing --- N/A Thanks, Saisai Shao

Re: Review Request 49295: remove spark.driver.extraJavaOptions and spark.yarn.am.extraJavaOptions property in HDP2.5 definition

2016-06-28 Thread Saisai Shao
/stacks/HDP/2.5/services/SPARK/configuration/spark-defaults.xml PRE-CREATION Diff: https://reviews.apache.org/r/49295/diff/ Testing --- Local verification. Thanks, Saisai Shao

Re: Review Request 49295: remove spark.driver.extraJavaOptions and spark.yarn.am.extraJavaOptions property in HDP2.5 definition

2016-06-28 Thread Saisai Shao
------ On June 28, 2016, 6:01 a.m., Saisai Shao wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49295/ > ---

Review Request 49295: remove spark.driver.extraJavaOptions and spark.yarn.am.extraJavaOptions property in HDP2.5 definition

2016-06-28 Thread Saisai Shao
/spark-defaults.xml PRE-CREATION Diff: https://reviews.apache.org/r/49295/diff/ Testing --- Local verification. Thanks, Saisai Shao

Review Request 48999: Fix Spark2 history server quick link still point to spark1 url

2016-06-20 Thread Saisai Shao
://reviews.apache.org/r/48999/diff/ Testing --- Local verification is done. Thanks, Saisai Shao

Re: Review Request 48651: Add unit tests for Spark2 service definition

2016-06-15 Thread Saisai Shao
> patch does not apply Sure, I will. - Saisai --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48651/#review137765 --- On June 13, 2016, 4:28 p

Review Request 48659: Fix Spark2 thriftserver Ambari definition bug

2016-06-13 Thread Saisai Shao
/ Testing --- Unit test is done. Thanks, Saisai Shao

Review Request 48651: Add unit tests for Spark2 service definition

2016-06-13 Thread Saisai Shao
-server/src/test/python/stacks/2.5/configs/spark2_default.json PRE-CREATION ambari-server/src/test/python/stacks/2.5/configs/spark2_thriftserver.json PRE-CREATION Diff: https://reviews.apache.org/r/48651/diff/ Testing --- Local unit test is done. Thanks, Saisai Shao

Re: Review Request 47579: Spark2 service definition for Ambari

2016-05-22 Thread Saisai Shao
spark2-env Same as above. - Saisai --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47579/#review134269 ------- On May

Re: Review Request 47579: Spark2 service definition for Ambari

2016-05-22 Thread Saisai Shao
/metainfo.xml PRE-CREATION Diff: https://reviews.apache.org/r/47579/diff/ Testing --- Local manual test Thanks, Saisai Shao

Re: Review Request 47579: Spark2 service definition for Ambari

2016-05-22 Thread Saisai Shao
blem from my understanding. - Saisai --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47579/#review134269 --- On May 19, 2016

Re: Review Request 47583: Remove useless log dir definition in Spark Ambari definition

2016-05-19 Thread Saisai Shao
Thanks, Saisai Shao

Review Request 47583: Remove useless log dir definition in Spark Ambari definition

2016-05-19 Thread Saisai Shao
/common-services/SPARK/1.2.1/package/scripts/setup_spark.py eca8534 Diff: https://reviews.apache.org/r/47583/diff/ Testing --- Local manual test Thanks, Saisai Shao

Review Request 47579: Spark2 service definition for Ambari

2016-05-18 Thread Saisai Shao
://reviews.apache.org/r/47579/diff/ Testing --- Local manual test Thanks, Saisai Shao