GitHub user zhzhan opened a pull request:
https://github.com/apache/spark/pull/2967
Spark 4103
Currently the sessionState in HiveContext is lazy evaluated, and to start
the session, SessionState.start() has to be invoked to initialize it. Using
ThreadLocal variable natively in hive to track the availability of SessionState
and initialize it if required. It avoid multiple thread sharing the same
SessionState or single thread invoke SessionState.start with the same
SessionState multiple times, which seems to cause unexpected behavior, and
break the contract defined in hive. Refer below comments form SessionState in
hive.
/**
* set current session to existing session object if a thread is running
* multiple sessions - it must call this method with the new session
object
* when switching from one session to another.
* @throws HiveException
*/
public static SessionState start(SessionState startSs) {
...
}
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zhzhan/spark spark-4103
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2967.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2967
----
commit ba14f283d8a68cc90ee828eee5dcc9a95e6dd5a4
Author: Zhan Zhang <[email protected]>
Date: 2014-08-08T17:47:18Z
test
commit f6a8a40baa588539727d2cd450f6b638756b9b29
Author: Zhan Zhang <[email protected]>
Date: 2014-08-08T17:57:06Z
revert
commit cb53a2cc4b25878c9ce1e8629f13739d0e26b2bf
Author: Zhan Zhang <[email protected]>
Date: 2014-08-30T18:01:37Z
Merge branch 'master' of https://github.com/apache/spark
commit 789ea21617178ea039b306070a69fcc9c807a053
Author: Zhan Zhang <[email protected]>
Date: 2014-09-02T20:31:23Z
Merge branch 'master' of https://github.com/apache/spark
commit 921e914083929aeccf95c7c6876fe182ccc9ee74
Author: Zhan Zhang <[email protected]>
Date: 2014-09-02T23:07:30Z
Merge branch 'master' of https://github.com/apache/spark
commit e4c19828a26cf0ae69521b2244be382dc20661ba
Author: Zhan Zhang <[email protected]>
Date: 2014-09-05T23:48:43Z
Merge branch 'master' of https://github.com/apache/spark
commit af9feb92848bb8102144095e0226f23f056a56b3
Author: Zhan Zhang <[email protected]>
Date: 2014-09-10T20:19:31Z
Merge branch 'master' of https://github.com/apache/spark
commit 1ccd7ccf06c7a15b52430d6f3b01ae196e3dd30a
Author: Zhan Zhang <[email protected]>
Date: 2014-09-16T20:30:28Z
Merge branch 'master' of https://github.com/apache/spark
commit 2b0d513f2193d36bfc6c74ffca08018dc01e7714
Author: Zhan Zhang <[email protected]>
Date: 2014-09-17T16:24:33Z
Merge branch 'master' of https://github.com/apache/spark
commit 3ee3b2bffc51e109ce38e0dee711288623a8978c
Author: Zhan Zhang <[email protected]>
Date: 2014-09-22T06:55:23Z
Merge branch 'master' of https://github.com/apache/spark
commit 68deb1163b542cef337c30215de32ede752d2935
Author: Zhan Zhang <[email protected]>
Date: 2014-09-22T18:52:10Z
Merge branch 'master' of https://github.com/apache/spark
commit 7e0cc362e2a6f237969e1d917c7c9b35382d5dea
Author: Zhan Zhang <[email protected]>
Date: 2014-09-25T17:14:19Z
Merge branch 'master' of https://github.com/apache/spark
commit d10bf009589ec6bf4d416c9499c1797efb397226
Author: Zhan Zhang <[email protected]>
Date: 2014-09-30T16:16:32Z
Merge branch 'master' of https://github.com/apache/spark
commit adf4924f3a79d5dd3b4f4a4fe486995e8f4c9338
Author: Zhan Zhang <[email protected]>
Date: 2014-10-01T06:05:40Z
Merge branch 'master' of https://github.com/apache/spark
commit cedcc6f42230331c992116797640e9468ec4e0db
Author: Zhan Zhang <[email protected]>
Date: 2014-10-01T22:33:29Z
Merge branch 'master' of https://github.com/apache/spark
commit 301eb4a3cb177b80855cdc206704d389eb9af907
Author: Zhan Zhang <[email protected]>
Date: 2014-10-05T07:49:21Z
Merge branch 'master' of https://github.com/apache/spark
commit a72c0d43c4b5fb8de7e456e0c07bca04dd23b463
Author: Zhan Zhang <[email protected]>
Date: 2014-10-09T19:00:11Z
Merge branch 'master' of https://github.com/apache/spark
commit 4a2e36d9eb35168064e7ae7645f2d3ace1880391
Author: Zhan Zhang <[email protected]>
Date: 2014-10-13T17:17:57Z
Merge branch 'master' of https://github.com/apache/spark
commit 497b0f45b11a9468c472799f567954aa9809a2cc
Author: Zhan Zhang <[email protected]>
Date: 2014-10-13T20:55:17Z
Merge branch 'master' of https://github.com/apache/spark
commit a00f60ffe9e4b7ae6f8beec6ea85f2b5b5d9aa9c
Author: Zhan Zhang <[email protected]>
Date: 2014-10-16T21:31:15Z
Merge branch 'master' of https://github.com/apache/spark
commit a9d372b6c82d401e91a3ba18620036d42e67068d
Author: Zhan Zhang <[email protected]>
Date: 2014-10-16T22:14:37Z
Merge branch 'master' of https://github.com/zhzhan/spark
commit 3764505dfd8934708ab4766a7ab09df6f0261f00
Author: Zhan Zhang <[email protected]>
Date: 2014-10-17T19:51:06Z
Merge branch 'master' of https://github.com/apache/spark
commit 93f3081dcdfc70c009a7caf7942cf78a1a639702
Author: Zhan Zhang <[email protected]>
Date: 2014-10-22T18:46:39Z
Merge branch 'master' of https://github.com/apache/spark
commit ce0ca7b98776ef9083596fb494c6992efa92dcc1
Author: Zhan Zhang <[email protected]>
Date: 2014-10-24T19:27:19Z
Merge branch 'master' of https://github.com/apache/spark
commit 12e1be5b89920a4c08de8653313a076d919560d7
Author: Zhan Zhang <[email protected]>
Date: 2014-10-27T16:22:02Z
Merge branch 'master' of https://github.com/apache/spark
commit 53a26ae4c3f671014e5d020a1fc6d3502003f598
Author: Zhan Zhang <[email protected]>
Date: 2014-10-27T22:09:52Z
clean up session state in HiveContext. Currently the sessionState in
HiveContext is lazy evaluated, and to start the session, SessionState.start()
has to be invoked to initialize it. Using ThreadLocal variable natively in hive
to track the availability of SessionState and initialize it if required. It
avoid multiple thread sharing the same SessionState or single thread invoking
SessionState.start with the same SessionState multiple times, which seems to
cause unexpected behavior.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
