I forgot to mention that if you need to access these functions for some reason, you can prepend the function call with the SparkR private namespace, like so, SparkR:::lapply(rdd, func).
On 6/25/15, 9:30 AM, "Felix C" <[email protected]> wrote: >Thanks! It's good to know > >--- Original Message --- > >From: "Eskilson,Aleksander" <[email protected]> >Sent: June 25, 2015 5:57 AM >To: "Felix C" <[email protected]>, [email protected] >Subject: Re: SparkR parallelize not found with 1.4.1? > >Hi there, > >Parallelize is part of the RDD API which was made private for Spark v. >1.4.0. Some functions in the RDD API were considered too low-level to >expose, so only most of the DataFrame API is currently public. The >original rationale for this decision can be found on the issue's JIRA [1]. >The devs are still considering which parts of the RDD API, if any, should >be made public for later releases. If you have some use case that you feel >is most easily addressed by the functions currently private in the RDD >API, go ahead and let the dev mailing list know. > >Alek >[1] -- https://issues.apache.org/jira/browse/SPARK-7230 > >On 6/25/15, 12:24 AM, "Felix C" <[email protected]> wrote: > >>Hi, >> >>It must be something very straightforward... >> >>Not working: >>parallelize(sc) >>Error: could not find function "parallelize" >> >>Working: >>df <- createDataFrame(sqlContext, localDF) >> >>What did I miss? >>Thanks >>?B�KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB >>� >>?�?[��X��ܚX�K??K[XZ[?�?\�\�][��X��ܚX�P?�?\�˘\?X�?K�ܙ�B��܈?Y??]?[ۘ[??��[X[ >>� >>?�??K[XZ[?�?\�\�Z?[???�?\�˘\?X�?K�ܙ�B�B > > >CONFIDENTIALITY NOTICE This message and any included attachments are from >Cerner Corporation and are intended only for the addressee. The >information contained in this message is confidential and may constitute >inside or non-public information under international, federal, or state >securities laws. Unauthorized forwarding, printing, copying, >distribution, or use of such information is strictly prohibited and may >be unlawful. If you are not the addressee, please promptly delete this >message and notify the sender of the delivery error by e-mail or you may >call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) >(816)221-1024. >?B�KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB� >?�?[��X��ܚX�K??K[XZ[?�?\�\�][��X��ܚX�P?�?\�˘\?X�?K�ܙ�B��܈?Y??]?[ۘ[??��[X[� >?�??K[XZ[?�?\�\�Z?[???�?\�˘\?X�?K�ܙ�B�B
