[
https://issues.apache.org/jira/browse/TOREE-315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gino Bustelo updated TOREE-315:
-------------------------------
Fix Version/s: 0.1.0
> SparkR interpreter hangs when running on Spark 1.6.1
> ----------------------------------------------------
>
> Key: TOREE-315
> URL: https://issues.apache.org/jira/browse/TOREE-315
> Project: TOREE
> Issue Type: Bug
> Affects Versions: 0.1.0
> Environment: Spark 1.6.1
> Jupyter notebook 4.2.0
> R 3.2.2
> Toree 0.1.0.dev7
> Reporter: Gurvinder
> Priority: Minor
> Fix For: 0.1.0
>
>
> Create notebook with scala interpreter and once notebook started with spark
> context, running SparkR magic as
> %%SparkR sc
> cause notebook to hang and no further execution of any cells. Need to restart
> the notebook. Issue seems to be caused by patching/updating install SparkR
> package in spark directory (/usr/local/spark/R/libs/SparkR , in my case).
> Here is the diff from default installed SparkR and updated one
> diff SparkR/DESCRIPTION SparkR.bak/DESCRIPTION
>
> 4c4
>
> < Version: 1.6.1
>
> ---
>
> > Version: 1.5.0
> >
> 16,18c16,17
>
> < 'functions.R' 'mllib.R' 'serialize.R' 'sparkR.R' 'stats.R'
>
> < 'types.R' 'utils.R'
>
> < Built: R 3.1.1; ; 2016-02-27 04:45:19 UTC; unix
>
> ---
>
> > 'functions.R' 'mllib.R' 'serialize.R' 'sparkR.R' 'utils.R'
> >
> > Built: R 3.2.1; ; 2015-10-01 18:38:50 UTC; unix
> >
> Only in SparkR: INDEX
>
> Common subdirectories: SparkR/Meta and SparkR.bak/Meta
>
> diff SparkR/NAMESPACE SparkR.bak/NAMESPACE
>
> 12a13,22
>
> > # Needed exports for runner
> >
> > export("sparkR.connect")
> >
> > export("isInstanceOf")
> >
> > export("callJMethod")
> >
> > export("callJStatic")
> >
> > export("newJObject")
> >
> > export("removeJObject")
> >
> > export("isRemoveMethod")
> >
> > export("invokeJava")
> >
> >
> >
> 26,27d35
>
> < "as.data.frame",
>
> < "attach",
>
> 30,33d37
>
> < "colnames",
>
> < "colnames<-",
>
> < "coltypes",
>
> < "coltypes<-",
>
> 36,37d39
>
> < "cov",
>
> < "corr",
>
> 49d50
>
> < "freqItems",
> Also once the default version is patched, running sparkR from terminal also
> give this error which worked earlier
> Launching java with spark-submit command /usr/local/spark/bin/spark-submit
> "sparkr-shell" /tmp/RtmpbfR7Ea/backend_port27b11cd7f3f
> 16/05/17 18:26:02 ERROR RBackendHandler: createSparkContext on
> org.apache.spark.api.r.RRDD failed
> Error in invokeJava(isStatic = TRUE, className, methodName, ...) :
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
