Re: Question related to parallelism using structed streaming parallelism

[Mich Talebzadeh]

or download it from here

https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf

Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Tue, 21 Mar 2023 at 15:38, Mich Talebzadeh 
wrote:

> Hi Emmanouil and anyone else interested
>
> Sounds like you may benefit from this booklet.Not the latest but good
> enough.
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 21 Mar 2023 at 12:21, Sean Owen  wrote:
>
>> Yes more specifically, you can't ask for executors once the app starts,
>> in SparkConf like that. You set this when you launch it against a Spark
>> cluster in spark-submit or otherwise.
>>
>> On Tue, Mar 21, 2023 at 4:23 AM Mich Talebzadeh <
>> mich.talebza...@gmail.com> wrote:
>>
>>> Hi Emmanouil,
>>>
>>> This means that your job is running on the driver as a single JVM, hence
>>> active(1)
>>>
>>>

Re: Question related to parallelism using structed streaming parallelism

[Sean Owen]

Yes more specifically, you can't ask for executors once the app starts,
in SparkConf like that. You set this when you launch it against a Spark
cluster in spark-submit or otherwise.

On Tue, Mar 21, 2023 at 4:23 AM Mich Talebzadeh 
wrote:

> Hi Emmanouil,
>
> This means that your job is running on the driver as a single JVM, hence
> active(1)
>
>

Topics for Spark online classes & webinars, next steps

[Mich Talebzadeh]

Hi all,



As you may be aware we are proposing to set-up community classes and
webinars for Spark interest group or simply for those who could benefit
from them.



@Denny Lee   and myself had a discussion on how to
put this framework forward. The idea is first and foremost getting support
from peers (whether presenters or participants) and with the notion of
contributions from everyone.



We propose that these presentations could be at the starter, intermediate
and advanced levels to appeal to everyone. We thus welcome all candidates
who want to contribute and present, regardless of their depth of Spark
knowledge. You have already seen the list of topics of interest.



https://www.linkedin.com/posts/apachespark_topics-for-spark-online-classes-webinars-activity-7041754636860411904-wkfj?utm_source=share_medium=member_desktop



 If you would like to contribute, you can send me or @Denny Lee
  an email stating which topic and at what level you
would like to take part. We propose to do a peer review of the draft
presentation so no worries.



Looking forward to hearing from you.



HTH


Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.