[
https://issues.apache.org/jira/browse/TOREE-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300694#comment-15300694
]
poplav commented on TOREE-315:
------------------------------
[~gurvinder], I believe your are hitting an issue where markSuccess on the
sparkr_runner is hung.
https://github.com/apache/incubator-toree/blob/master/sparkr-interpreter/src/main/resources/kernelR/sparkr_runner.R#L116
is this not completing for you or is it something else? As an example, so you
can see in the notebook that it received code, evaluated it, but it hangs on
marking it as success? I'm trying to figure out if the issue was introduced
when manually upgrading the sparkR fork or from this markSuccess issue.
> SparkR interpreter hangs when running on Spark 1.6.1
> ----------------------------------------------------
>
> Key: TOREE-315
> URL: https://issues.apache.org/jira/browse/TOREE-315
> Project: TOREE
> Issue Type: Bug
> Affects Versions: 0.1.0
> Environment: Spark 1.6.1
> Jupyter notebook 4.2.0
> R 3.2.2
> Toree 0.1.0.dev7
> Reporter: Gurvinder
> Priority: Minor
> Fix For: 0.1.0
>
>
> Create notebook with scala interpreter and once notebook started with spark
> context, running SparkR magic as
> %%SparkR sc
> cause notebook to hang and no further execution of any cells. Need to restart
> the notebook. Issue seems to be caused by patching/updating install SparkR
> package in spark directory (/usr/local/spark/R/libs/SparkR , in my case).
> Here is the diff from default installed SparkR and updated one
> diff SparkR/DESCRIPTION SparkR.bak/DESCRIPTION
>
> 4c4
>
> < Version: 1.6.1
>
> ---
>
> > Version: 1.5.0
> >
> 16,18c16,17
>
> < 'functions.R' 'mllib.R' 'serialize.R' 'sparkR.R' 'stats.R'
>
> < 'types.R' 'utils.R'
>
> < Built: R 3.1.1; ; 2016-02-27 04:45:19 UTC; unix
>
> ---
>
> > 'functions.R' 'mllib.R' 'serialize.R' 'sparkR.R' 'utils.R'
> >
> > Built: R 3.2.1; ; 2015-10-01 18:38:50 UTC; unix
> >
> Only in SparkR: INDEX
>
> Common subdirectories: SparkR/Meta and SparkR.bak/Meta
>
> diff SparkR/NAMESPACE SparkR.bak/NAMESPACE
>
> 12a13,22
>
> > # Needed exports for runner
> >
> > export("sparkR.connect")
> >
> > export("isInstanceOf")
> >
> > export("callJMethod")
> >
> > export("callJStatic")
> >
> > export("newJObject")
> >
> > export("removeJObject")
> >
> > export("isRemoveMethod")
> >
> > export("invokeJava")
> >
> >
> >
> 26,27d35
>
> < "as.data.frame",
>
> < "attach",
>
> 30,33d37
>
> < "colnames",
>
> < "colnames<-",
>
> < "coltypes",
>
> < "coltypes<-",
>
> 36,37d39
>
> < "cov",
>
> < "corr",
>
> 49d50
>
> < "freqItems",
> Also once the default version is patched, running sparkR from terminal also
> give this error which worked earlier
> Launching java with spark-submit command /usr/local/spark/bin/spark-submit
> "sparkr-shell" /tmp/RtmpbfR7Ea/backend_port27b11cd7f3f
> 16/05/17 18:26:02 ERROR RBackendHandler: createSparkContext on
> org.apache.spark.api.r.RRDD failed
> Error in invokeJava(isStatic = TRUE, className, methodName, ...) :
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
