Sahil Takiar created HIVE-18684:

             Summary: Race condition in RemoteSparkJobMonitor
                 Key: HIVE-18684
             Project: Hive
          Issue Type: Sub-task
          Components: Spark
            Reporter: Sahil Takiar
            Assignee: Sahil Takiar

There is a race condition in {{RemoteSparkJobMonitor}}. Sometimes the info in 
{{RemoteSparkJobMonitor#startMonitor.STARTED}} gets printed out, sometimes it 
doesn't. This can be easily verified by running a qtest on 
{{TestMiniSparkOnYarnCliDriver}} and counting the number of times {{Query Hive 
on Spark job}} is printed vs. the number of times {{Finished successfully in}} 
gets printed.

The issue is that {{RemoteSparkJobMonitor}} runs every one second, and checks 
the state of {{JobHandle}}. Depending on the state, it prints out some logging 
info. The content of the logs contain an implicit assumption that logs in the 
{{STARTED}} state are printed before the logs in the {{SUCCEEDED}} state. 
However, this isn't always the case. The state transitions are driven by how 
long the remote Spark job takes to run, and it it finishes within one second 
then the logs in the {{STARTED}} state never printed.

This can be confusing to users, and there is key debugging information that is 
printed in the {{STARTED}} state.

This message was sent by Atlassian JIRA

Reply via email to