[jira] [Commented] (HIVE-17090) spark.only.query.files are not being run by ptest
[
https://issues.apache.org/jira/browse/HIVE-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094850#comment-16094850
]
Sahil Takiar commented on HIVE-17090:
-
Committed to master, thanks Sergio!
> spark.only.query.files are not being run by ptest
> -
>
> Key: HIVE-17090
> URL: https://issues.apache.org/jira/browse/HIVE-17090
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Fix For: 3.0.0
>
> Attachments: HIVE-17090.patch
>
>
> Checked a recent run of Hive QA and it doesn't look like qtests specified in
> spark.only.query.files are being run.
> I think some modifications to ptest config files are required to get this
> working - e.g. the deployed master-m2.properties file for ptest should
> contain mainProperties.$\{spark.only.query.files} in the
> qFileTest.miniSparkOnYarn.groups.normal key.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17090) spark.only.query.files are not being run by ptest
[
https://issues.apache.org/jira/browse/HIVE-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093918#comment-16093918
]
Sergio Peña commented on HIVE-17090:
Thanks. Feel free to commit this then.
> spark.only.query.files are not being run by ptest
> -
>
> Key: HIVE-17090
> URL: https://issues.apache.org/jira/browse/HIVE-17090
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-17090.patch
>
>
> Checked a recent run of Hive QA and it doesn't look like qtests specified in
> spark.only.query.files are being run.
> I think some modifications to ptest config files are required to get this
> working - e.g. the deployed master-m2.properties file for ptest should
> contain mainProperties.$\{spark.only.query.files} in the
> qFileTest.miniSparkOnYarn.groups.normal key.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17090) spark.only.query.files are not being run by ptest
[
https://issues.apache.org/jira/browse/HIVE-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093912#comment-16093912
]
Sahil Takiar commented on HIVE-17090:
-
[~spena], yes I already applied this profile to the ptest master. After I
applied it, all the spark tests above started failing because they haven't been
run in a long time and they all have outdated .q.out files. I'm working on
fixing those in HIVE-17099
> spark.only.query.files are not being run by ptest
> -
>
> Key: HIVE-17090
> URL: https://issues.apache.org/jira/browse/HIVE-17090
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-17090.patch
>
>
> Checked a recent run of Hive QA and it doesn't look like qtests specified in
> spark.only.query.files are being run.
> I think some modifications to ptest config files are required to get this
> working - e.g. the deployed master-m2.properties file for ptest should
> contain mainProperties.$\{spark.only.query.files} in the
> qFileTest.miniSparkOnYarn.groups.normal key.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17090) spark.only.query.files are not being run by ptest
[
https://issues.apache.org/jira/browse/HIVE-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093904#comment-16093904
]
Sergio Peña commented on HIVE-17090:
I forgot this patch is not executed. [~stakiar] YOu already applied this
profile on the server, rigth?
LGTM
+1
> spark.only.query.files are not being run by ptest
> -
>
> Key: HIVE-17090
> URL: https://issues.apache.org/jira/browse/HIVE-17090
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-17090.patch
>
>
> Checked a recent run of Hive QA and it doesn't look like qtests specified in
> spark.only.query.files are being run.
> I think some modifications to ptest config files are required to get this
> working - e.g. the deployed master-m2.properties file for ptest should
> contain mainProperties.$\{spark.only.query.files} in the
> qFileTest.miniSparkOnYarn.groups.normal key.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17090) spark.only.query.files are not being run by ptest
[
https://issues.apache.org/jira/browse/HIVE-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093900#comment-16093900
]
Sergio Peña commented on HIVE-17090:
I see some spark failures. Are those related to this change?
> spark.only.query.files are not being run by ptest
> -
>
> Key: HIVE-17090
> URL: https://issues.apache.org/jira/browse/HIVE-17090
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-17090.patch
>
>
> Checked a recent run of Hive QA and it doesn't look like qtests specified in
> spark.only.query.files are being run.
> I think some modifications to ptest config files are required to get this
> working - e.g. the deployed master-m2.properties file for ptest should
> contain mainProperties.$\{spark.only.query.files} in the
> qFileTest.miniSparkOnYarn.groups.normal key.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17090) spark.only.query.files are not being run by ptest
[
https://issues.apache.org/jira/browse/HIVE-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088385#comment-16088385
]
Hive QA commented on HIVE-17090:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12877396/HIVE-17090.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 10907 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_2]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_use_op_stats]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_use_ts_stats_for_mapjoin]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=167)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=233)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=233)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=178)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=178)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=178)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6045/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6045/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6045/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12877396 - PreCommit-HIVE-Build
> spark.only.query.files are not being run by ptest
> -
>
> Key: HIVE-17090
> URL: https://issues.apache.org/jira/browse/HIVE-17090
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-17090.patch
>
>
> Checked a recent run of Hive QA and it doesn't look like qtests specified in
> spark.only.query.files are being run.
> I think some modifications to ptest config files are required to get this
> working - e.g. the deployed master-m2.properties file for ptest should
> contain mainProperties.$\{spark.only.query.files} in the
> qFileTest.miniSparkOnYarn.groups.normal key.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17090) spark.only.query.files are not being run by ptest
[
https://issues.apache.org/jira/browse/HIVE-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087911#comment-16087911
]
Sahil Takiar commented on HIVE-17090:
-
Ok looks like that changed worked! The spark.only.query.files files are now
being run by ptest, the bad news is that lots of them are failing:
{code}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_2]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_use_op_stats]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_use_ts_stats_for_mapjoin]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=167)
{code}
Filed HIVE-17099 to fix these tests.
> spark.only.query.files are not being run by ptest
> -
>
> Key: HIVE-17090
> URL: https://issues.apache.org/jira/browse/HIVE-17090
> Project: Hive
> Issue Type: Bug
> Components: Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>
> Checked a recent run of Hive QA and it doesn't look like qtests specified in
> spark.only.query.files are being run.
> I think some modifications to ptest config files are required to get this
> working - e.g. the deployed master-m2.properties file for ptest should
> contain mainProperties.$\{spark.only.query.files} in the
> qFileTest.miniSparkOnYarn.groups.normal key.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
