This is a very good idea-would love to read such a confluence page. Adding a section “common mistakes/misconceptions” might be useful for many of these sections. It would describe undesired behaviour/errors one would get in case of not following some best practices.
On 13 Mar 2023, at 17:20, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: Apologies I missed the list. To move forward I selected these topics from the thread "Online classes for spark topics". To take this further I propose a confluence page to be seup. 1. Spark UI 2. Dynamic allocation 3. Tuning of jobs 4. Collecting spark metrics for monitoring and alerting 5. For those who prefer to use Pandas API on Spark since the release of Spark 3.2, What are some important notes for those users? For example, what are the additional factors affecting the Spark performance using Pandas API on Spark? How to tune them in addition to the conventional Spark tuning methods applied to Spark SQL users. 6. Spark internals and/or comparing spark 3 and 2 7. Spark Streaming & Spark Structured Streaming 8. Spark on notebooks 9. Spark on serverless (for example Spark on Google Cloud) 10. Spark on k8s Opinions and how to is welcome [https://ci3.googleusercontent.com/mail-sig/AIorK4zholKucR2Q9yMrKbHNn-o1TuS4mYXyi2KO6Xmx6ikHPySa9MLaLZ8t2hrA6AUcxSxDgHIwmKE] view my Linkedin profile<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fmich-talebzadeh-ph-d-5205b2%2F&data=05%7C01%7Cjoris.billen%40bigindustries.be%7C33df48307f1049c3932508db23df1022%7C49c3d703357947bfa8887c913fbdced9%7C0%7C0%7C638143213181129126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZP0m80Q1NGAMFzD4FFIW8kC%2FUaInAZwRM9HVV1UoJtE%3D&reserved=0> https://en.everybodywiki.com/Mich_Talebzadeh<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.everybodywiki.com%2FMich_Talebzadeh&data=05%7C01%7Cjoris.billen%40bigindustries.be%7C33df48307f1049c3932508db23df1022%7C49c3d703357947bfa8887c913fbdced9%7C0%7C0%7C638143213181129126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LOs7tXiyHRNXc0ViAFRu13ltXXjzob7lENq0MvDDOxs%3D&reserved=0> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mich.talebza...@gmail.com<mailto:mich.talebza...@gmail.com>> wrote: Hi guys To move forward I selected these topics from the thread "Online classes for spark topics". To take this further I propose a confluence page to be seup. Opinions and how to is welcome Cheers [https://ci3.googleusercontent.com/mail-sig/AIorK4zholKucR2Q9yMrKbHNn-o1TuS4mYXyi2KO6Xmx6ikHPySa9MLaLZ8t2hrA6AUcxSxDgHIwmKE] view my Linkedin profile<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fmich-talebzadeh-ph-d-5205b2%2F&data=05%7C01%7Cjoris.billen%40bigindustries.be%7C33df48307f1049c3932508db23df1022%7C49c3d703357947bfa8887c913fbdced9%7C0%7C0%7C638143213181285354%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2t5OqcDn7%2B8Tv%2FBtuRGmEzPyJwbVcKaIuKZ4dzsigHs%3D&reserved=0> https://en.everybodywiki.com/Mich_Talebzadeh<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.everybodywiki.com%2FMich_Talebzadeh&data=05%7C01%7Cjoris.billen%40bigindustries.be%7C33df48307f1049c3932508db23df1022%7C49c3d703357947bfa8887c913fbdced9%7C0%7C0%7C638143213181285354%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Zk11t0gp%2Fl3iOgx3dvKWMl5OniuErrrsxCPCVM6L9JM%3D&reserved=0> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
