Re: Samza jdk11 support
Hi Yan Can you give me some pointer on running all the integrations tests that are present on samza 1.5 release branch in a local environment with Open JDK 11 ? I found the below doc but that talks about samza 0.10. https://samza.apache.org/contribute/tests.html Thanks, On Thu, Jun 4, 2020 at 12:18 AM Yi Pan wrote: > Hi, Debraj and Jordan, > > Thanks a lot for the ping. I dug a bit deeper in the past email thread on > this topic and talked with our internal team. Unfortunately, LI currently > does not have a short term plan to migrate to JDK 11. However, we encourage > you to contribute to the matter if that is important to you. Here is an > earlier investigation on this matter from this mailing list that I want to > share: > > http://mail-archives.apache.org/mod_mbox/samza-dev/201810.mbox/%3C1538433805053.85238%40helixeducation.com%3E > . > > Best! > > -Yi > > On Mon, Jun 1, 2020 at 9:09 AM Jordan Messec > wrote: > > > This is a pressing issue for our team as well. > > > > Jordan > > > > > On May 31, 2020, at 7:15 PM, Debraj Manna > > wrote: > > > > > > Thanks Yi. > > > > > > Did you get any update on the JDK 11 roadmap? > > > > > > > > > On Wed, May 27, 2020 at 1:04 AM Yi Pan wrote: > > > > > >> Hi, Debraj, > > >> > > >> Thanks for the reminder. We did discussed about JDK11 before. > > >> Unfortunately, I don't know whether JDK11 is up to the roadmap as of > > now. > > >> Let me sync up with the team and get back to you. > > >> > > >> -Yi > > >> > > >> On Mon, May 25, 2020 at 9:53 AM Debraj Manna < > subharaj.ma...@gmail.com> > > >> wrote: > > >> > > >>> Hi > > >>> > > >>> I have seen a few earlier discussions on the email group which > > >>> suggested that samza officially does not support jdk11. But those > > >>> discussions seem to be old. > > >>> > > >>> Do the latest samza support jdk11? > > >>> > > >>> Thanks, > > >>> > > >> > > > > >
Draft board report for Samza
Hi, all, Here is the draft report for Samza. Please let me know if I missed anything. Thanks! == ## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ## Issues: [Insert your own data here] ## Membership Data: Apache Samza was founded 2015-01-22 (6 years ago) There are currently 26 committers and 17 PMC members in this project. The Committer-to-PMC ratio is roughly 7:5. Community changes, past quarter: - No new PMC members. Last addition was Bharath Kumarasubramanian on 2020-02-13. - No new committers. Last addition was Rayman Preet Singh on 2019-07-08. ## Project Activity: - New version 1.5.1 was released on 2020-08-28 ## Community Health: - We held an online stream processing meetup on July 21, 2020 - We presented Python stream processing on Beam Samza Runner in Beam Summit on Aug 24-28, 2020 - We presented Fast SamzaSQL in ApacheCon on Sept 30, 2020 - Beam Samza runner performance improvement was published in LinkedIn Engineering blog on Oct 1, 2020
Re: Issues with Samza-Kafka compatibility while upgrading to latest versions
Hi, Choudhary,
Thanks for reporting your issues. Samza manages its dependencies in a few
gradle files under ${ROOT}/gradle/dependency-versions-*.gradle.
Specifically, the Kafka dependency is defined in the file
${ROOT}/gradle/dependency-versions.gradle. I quickly checked the versions
listed in 1.4.0 and it is still at 2.0.1. Please try to use 2.0.1 instead
of 2.5.0. Let me know if that works for you.
Thanks!
-Yi
On Thu, Oct 8, 2020 at 1:48 PM Choudhary, Suraj
wrote:
> Hi,
>
> I am working on updating our infrastructure by upgrading Samza, Kafka and
> Hadoop(our container manager for samza).
>
> I am upgrading from/to these versions:
> Samza:from 2.11-0.12.0 to 2.12-1.4.0
> Kafka: from 2.11-0.10.0.1 to 2.12-2.5.0
> Hadoop: from 2.7.2 to 2.9.2
>
> After updating the versions on starting the Samza application I am getting
> the following error:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> kafka/common/TopicAndPartition
> at
> org.apache.samza.system.kafka.KafkaSystemConsumer.toTopicAndPartition(KafkaSystemConsumer.java:317)
> at
> org.apache.samza.system.kafka.KafkaSystemConsumer.register(KafkaSystemConsumer.java:284)
> at
> org.apache.samza.coordinator.stream.CoordinatorStreamSystemConsumer.register(CoordinatorStreamSystemConsumer.java:130)
> at
> org.apache.samza.util.CoordinatorStreamUtil$.writeConfigToCoordinatorStream(CoordinatorStreamUtil.scala:159)
> at org.apache.samza.job.JobRunner.run(JobRunner.scala:80)
> at org.apache.samza.job.JobRunner$.doOperation(JobRunner.scala:52)
> at org.apache.samza.job.JobRunner$.main(JobRunner.scala:47)
> at org.apache.samza.job.JobRunner.main(JobRunner.scala)
> Caused by: java.lang.ClassNotFoundException: kafka.common.TopicAndPartition
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 8 more
>
> On further checking I found that ‘KafkaSystemConsumer’ is part of
> samza-kafka_2.12-1.4.0.jar. And I looked into our deploy jars and I am
> finding only one copy of this class.
> The reference class ‘TopicAndPartition’ is not present anywhere in the
> deploy jars or kafka_2.12-2.5.0 jar in particular.
>
> I tried to find a Samza-Kafka version compatibility documentation. But I
> didn’t find any. On investigating our import of samza-kafka_2.12-1.4.0.jar,
> I found that it is including the kafka_2.12-2.5.0 dependency, which leads
> me to think that Samza should be compatible the Kafka version I am
> referencing.
>
> I am also including all the required maven dependencies such as kafka,
> kafka-clients, samza-core, samza-api of these latest versions and made sure
> no other version of the same is present in my deploy jars.
>
> Please let me know If I am missing something. And what can do to resolve
> this issue.
>
> Thanks in Advance,
> Suraj
>
>
>
