[
https://issues.apache.org/jira/browse/SPARK-11199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14966206#comment-14966206
]
Felix Cheung commented on SPARK-11199:
--------------------------------------
[~falaki]Nice - I have a slightly different scope in mind - I hope this JIRA
will address the inconsistent R API. I think that is complementary to your PR.
My proposal:
- No need to return SparkContext from sparkR.init()
- Initialize sqlContext, hiveCtx with sparkR.init()
- Remove/deprecate sparkRSQL.init(), sparkRHive.init()
- Make sqlContext, hiveCtx singleton and "hidden" (since they are not required
as discussed in SPARK-10903)
This way it is very simple to manage contexts (sparkR.init(), sparkR.stop())
Would you like to update this JIRA to say getOrCreate for SQLContext and I'll
create another JIRA for above?
> Improve R context management story and add getOrCreate
> ------------------------------------------------------
>
> Key: SPARK-11199
> URL: https://issues.apache.org/jira/browse/SPARK-11199
> Project: Spark
> Issue Type: Sub-task
> Components: SparkR
> Affects Versions: 1.5.1
> Reporter: Felix Cheung
> Assignee: Felix Cheung
> Priority: Minor
>
> Similar to SPARK-11114
> Also from discussion in SPARK-10903:
> "
> Hossein Falaki added a comment - 08/Oct/15 13:06
> +1 We have seen a lot of questions from new SparkR users about the life cycle
> of the context.
> My question is: are we going to remove or deprecate sparkRSQL.init()? I
> suggest we should, because right now calling that method creates a new Java
> SQLContext object, and having two of them prevents users form viewing temp
> tables.
> Felix Cheung added a comment - 08/Oct/15 17:13
> +1 perhaps sparkR.init() should create sqlContext and/or hiveCtx together.
> But Hossein Falaki, as of now calling sparkRSQL.init() should return the same
> one as you can see
> https://github.com/apache/spark/blob/master/R/pkg/R/sparkR.R#L224
> Hossein Falaki added a comment - 08/Oct/15 17:16
> I meant the SQL Context:
> https://github.com/apache/spark/blob/master/R/pkg/R/sparkR.R#L236
> This call should have been "getOrCreate."
> "
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
