Peter Vary created HIVE-17270:

             Summary: Qtest results show wrong number of executors
                 Key: HIVE-17270
             Project: Hive
          Issue Type: Bug
          Components: Spark
    Affects Versions: 3.0.0
            Reporter: Peter Vary
            Assignee: Peter Vary

The hive-site.xml shows, that the TestMiniSparkOnYarnCliDriver uses 2 cores, 
and 2 executor instances to run the queries. See:

When reading the log files for the query tests, I see the following:
2017-08-08T07:41:03,315  INFO [0381325d-2c8c-46fb-ab51-423defaddd84 main] 
session.SparkSession: Spark cluster current has executors: 1, total cores: 2, 
memory per executor: 512M, memoryFraction: 0.4

When running the tests against a real cluster, I found that running an explain 
query for the first time I see 1 executor, but running it for the second time I 
see 2 executors.

Also setting some spark configuration on the cluster resets this behavior. For 
the first time I will see 1 executor, and for the second time I will see 2 
executors again.

This message was sent by Atlassian JIRA

Reply via email to