Hi folks, The next Bay Area Stream Processing meetup <https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/264589317/> will be held on *Oct 3rd, 2019*. This meetup will focus on Apache Kafka, Apache Samza and related streaming technologies.
*Where*: Unify Conference room, 950 W Maude Ave, Sunnyvale. *When: *5:30-8:30 PM. *RSVP*: link <https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/264589317/> *Summary*: We will have 3 talks and our first speaker will be from the *Azure Streaming team*. The talk will cover Azure Stream Analytics, its architecture, capabilities and use cases. The second talk will be about Samza’s recently built capability to do Stream processing using Python. *Samza-Python *opens up multiple possibilities including an ability to use Python libraries for ML applications. LinkedIn's Kafka team will talk about the development workflow that allows them to handle LinkedIn’s scale by building a LinkedIn version of Kafka while also contributing back to the open source community. Additional details are below. *Agenda*: - 5:30 PM: Doors open - 5:30-6:00 PM: Networking - *6:00-6:30 PM: Azure Stream Analytics : * *Sasha Alperovich & Sid Ramadoss, Microsoft *Azure Stream Analytics (ASA) is a fully managed near real-time data processing service on Azure. In this talk we will highlight the unique value propositions that ASA brings to the table, and show a demo of how Azure customers can utilize the power of ASA to gain insights in near real-time with the NYC taxi scenario. We will then dive deeper into how the service is built, covering resiliency, dataflow and other technical aspects of the ASA runtime. We will also discuss how ASA’s unique design choices compare and contrast with other streaming technologies, namely Spark Structured Streaming and Flink - *6:30 - 7:00PM:* *Stream Processing in Python using Apache Samza Hai Lu, LinkedIn * Apache Samza is the streaming engine being used at LinkedIn that processes ~ 2 trillion messages daily. A while back we announced Samza's integration with Apache Beam, a great success which leads to our Samza Beam API. Now an UPGRADE of our APIs - we're now supporting Stream Processing in Python! This work has made stream processing more accessible and enabled many interesting use cases, particularly in the area of machine learning.The Python API is based on our work of Samza runner for Apache Beam. In this talk, we will quickly review our work on Samza runner, and then how we extended it to support portability in Beam (Python specifically). In addition to technical and architectural details, we will also talk about how we bridged Python and Java ecosystems at LinkedIn with the Python API, together with different use cases. - *7:00-7:30 PM: Apache Kafka and LinkedIn: How LinkedIn customizes Kafka to work at the trillion scale Jon Lee & Wesley Wu, LinkedIn *At LinkedIn, we operate thousands of brokers to handle trillions of messages per day. Running at such a large scale constantly raises various scalability and operability challenges for the Kafka ecosystem. While we try to maintain our internal releases as close as possible to upstream, we maintain a version of Kafka which includes patches for addressing our production and feature requirements. In this presentation we will share the Kafka that LinkedIn runs in production, the workflow process we follow to develop new patches, the way we upstream the changes we make, some of the patches we maintain in our branch and how we generate releases. - *7:30-8:30PM: Additional networking and Q&A* If you are interested in attending, please RSVP via this meetup.com link <https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/264589317/> . Hope to see you there! - Prateek Maheshwari
