Hi all,
We have 4 HPC nodes and installed spark individually in all nodes.
Spark is used as local mode(each driver/executor will have 8 cores and 65
GB) in Sparklyr/pyspark using Rstudio/Posit workbench. Slurm is used as
scheduler.
As this is local mode, we are facing performance issue(as only o
Hi Russell et al,Acknowledging receipt; we’ll get these answers back to the group.Follow-up forthcoming.CraigOn Sep 14, 2023, at 6:38 PM, russell.spit...@gmail.com wrote:Exactly once should be output sink dependent, what sink was being used?Sent from my iPhoneOn Sep 14, 2023, at 4:52 PM, Jerry Pen
Exactly once should be output sink dependent, what sink was being used?Sent from my iPhoneOn Sep 14, 2023, at 4:52 PM, Jerry Peng wrote:Craig,Thanks! Please let us know the result!Best,JerryOn Thu, Sep 14, 2023 at 12:22 PM Mich Talebzadeh wrote:Hi Craig,Can you please
Craig,
Thanks! Please let us know the result!
Best,
Jerry
On Thu, Sep 14, 2023 at 12:22 PM Mich Talebzadeh
wrote:
>
> Hi Craig,
>
> Can you please clarify what this bug is and provide sample code causing
> this issue?
>
> HTH
>
> Mich Talebzadeh,
> Distinguished Technologist, Solutions Archit
Hi Jerry- This is exactly the type of help we're seeking, to confirm the
FilestreamSink was not utilized on our test runs.
Our team is going to work towards implementing this and re-running our
experiments across the versions.
If everything comes back with similar results, we will reach back ou
Thanks Holden and Martin for the nice words and feedback :)
On Wed, Sep 13, 2023 at 8:22 AM Martin Grund wrote:
> This is absolutely awesome! Thank you so much for dedicating your time to
> this project!
>
>
> On Wed, Sep 13, 2023 at 6:04 AM Holden Karau wrote:
>
>> That’s so cool! Great work y