can use pyspark as entrypoint to your application. With py4j you
>>>> can call Java/Scala functions from the python application. There's no need
>>>> to use the pipe() function for that.
>>>>
>>>>
>>>> Shay
>>>>
application. With py4j you can
>>> call Java/Scala functions from the python application. There's no need to
>>> use the pipe() function for that.
>>>
>>>
>>> Shay
>>> --
>>> *From:* Yuhao Zhang
>>> *S
you can
>> call Java/Scala functions from the python application. There's no need to
>> use the pipe() function for that.
>>
>>
>> Shay
>> --
>> *From:* Yuhao Zhang
>> *Sent:* Saturday, July 9, 2022 4:13:42 AM
>&
e() function for that.
>
>
> Shay
> --
> *From:* Yuhao Zhang
> *Sent:* Saturday, July 9, 2022 4:13:42 AM
> *To:* user@spark.apache.org
> *Subject:* [EXTERNAL] RDD.pipe() for binary data
>
>
> *ATTENTION:* This email originated from outside of GM.
>
>
> Hi All,
>
&g
AM
To: user@spark.apache.org
Subject: [EXTERNAL] RDD.pipe() for binary data
ATTENTION: This email originated from outside of GM.
Hi All,
I'm currently working on a project involving transferring between Spark 3.x (I
use Scala) and a Python runtime. In Spark, data is stored in an RDD as
floa