Re: [EXTERNAL] RDD.pipe() for binary data

2022-07-16 Thread Andrew Melo
can use pyspark as entrypoint to your application. With py4j you >>>> can call Java/Scala functions from the python application. There's no need >>>> to use the pipe() function for that. >>>> >>>> >>>> Shay >>>>

Re: [EXTERNAL] RDD.pipe() for binary data

2022-07-16 Thread Sebastian Piu
application. With py4j you can >>> call Java/Scala functions from the python application. There's no need to >>> use the pipe() function for that. >>> >>> >>> Shay >>> -- >>> *From:* Yuhao Zhang >>> *S

Re: [EXTERNAL] RDD.pipe() for binary data

2022-07-16 Thread Sean Owen
you can >> call Java/Scala functions from the python application. There's no need to >> use the pipe() function for that. >> >> >> Shay >> -- >> *From:* Yuhao Zhang >> *Sent:* Saturday, July 9, 2022 4:13:42 AM >&

Re: [EXTERNAL] RDD.pipe() for binary data

2022-07-16 Thread Yuhao Zhang
e() function for that. > > > Shay > -- > *From:* Yuhao Zhang > *Sent:* Saturday, July 9, 2022 4:13:42 AM > *To:* user@spark.apache.org > *Subject:* [EXTERNAL] RDD.pipe() for binary data > > > *ATTENTION:* This email originated from outside of GM. > > > Hi All, > &g

Re: [EXTERNAL] RDD.pipe() for binary data

2022-07-10 Thread Shay Elbaz
AM To: user@spark.apache.org Subject: [EXTERNAL] RDD.pipe() for binary data ATTENTION: This email originated from outside of GM. Hi All, I'm currently working on a project involving transferring between Spark 3.x (I use Scala) and a Python runtime. In Spark, data is stored in an RDD as floa