Re: Improved Structured Streaming Documentation Proof-of-Concept

2024-03-25 Thread Neil Ramaswamy
I'm glad you think it's generally a good idea! I will mention, though, that with these better docs I've almost finished, I'm hoping that Structured Streaming no longer stays a specialist topic that requires "trench warfare." With good pedagogy, I think that it's very approachable. The Knowledge

Re: Improved Structured Streaming Documentation Proof-of-Concept

2024-03-25 Thread Mich Talebzadeh
Hi, Your intended work on improving the Structured Streaming documentation is great! Clear and well-organized instructions are important for everyone using Spark, beginners and experts alike. Having said that, Spark Structured Streaming much like other specialist topics with Spark say (k8s) or

Improved Structured Streaming Documentation Proof-of-Concept

2024-03-25 Thread Neil Ramaswamy
Hi all, I recently started an effort to improve the Structured Streaming documentation. I thought that the current documentation, while very comprehensive, could be improved in terms of organization, clarity, and presence of examples. You can view the repo here