> On Oct. 16, 2018, 1:47 p.m., Sahil Takiar wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java > > Line 352 (original) > > <https://reviews.apache.org/r/69022/diff/2/?file=2097991#file2097991line361> > > > > why remove this? > > denys kuzmenko wrote: > it's not required. close() method is covered with the lock, and > activeJobs is a concurrent collection > > Sahil Takiar wrote: > what happens if a job is submitted after `hasTimedOut` returns true? > > denys kuzmenko wrote: > I see. However existing lock won't help as it doesn't prevent other > threads from adding new queries. > > public void onQuerySubmission(String queryId) { > activeJobs.add(queryId); > } > > we might need to cover this with separate lock (onQueryCompletion, > onQuerySubmission, triggerTimeout) > What do you think? > > denys kuzmenko wrote: > what happens if a job is submitted after hasTimedOut returns true? > current Session will be closed and a new one will be opened. > > denys kuzmenko wrote: > there might be an issue when 2nd session checkes state and get isOpen and > before it's reaching the submit phase, 1st one calls the close. > I think we need synchronization for active sessions manipulation. > > denys kuzmenko wrote: > fixed. reverted back to the original locking. Above tricky case will be > handled by preventing new queries to execute open() before close() is > complete.
@before triggerTimeout() is complete - denys ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69022/#review209617 ----------------------------------------------------------- On Oct. 16, 2018, 4:49 p.m., denys kuzmenko wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69022/ > ----------------------------------------------------------- > > (Updated Oct. 16, 2018, 4:49 p.m.) > > > Review request for hive, Sahil Takiar and Adam Szita. > > > Bugs: HIVE-20737 > https://issues.apache.org/jira/browse/HIVE-20737 > > > Repository: hive-git > > > Description > ------- > > 1. Local SparkContext is shared between user sessions and should be closed > only when there is no active. > 2. Possible race condition in SparkSession.open() in case when user queries > run in parallel within the same session. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java > 72ff53e3bd > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java > bb50129518 > > ql/src/test/org/apache/hadoop/hive/ql/exec/spark/TestLocalHiveSparkClient.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/69022/diff/3/ > > > Testing > ------- > > Added TestLocalHiveSparkClient test > > > File Attachments > ---------------- > > HIVE-20737.7.patch > > https://reviews.apache.org/media/uploaded/files/2018/10/15/9cf8a2b3-9ec1-4316-81d0-3cd124b1a9fd__HIVE-20737.7.patch > > > Thanks, > > denys kuzmenko > >
