Yes, I explicitly commit via code for this job as an effort to ensure only once
processing.
Thanks for taking the time to look into our concerns.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Yi Pan
Samza container still running after you see those WARN logs?
Yes.
I am thinking this is a timeout issue. We've never seen the issue before. The
warning first appeared after testing Confluent's Cloud kafka offering. We had
no issues when running our own kafka clusters in aws.
Jeremiah A
Can anyone take a look at the message below? We are trying to gauge our risk
before moving forward.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Jeremiah Adams
Sent: Wednesday, March 4, 2020 2:28 PM
ted before
a response was received.. Going to request metadata update now
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com/HelixEducation> |
Facebook<https://
Thanks Brett.
In the short term, can I fix this by specifying an alternate directory? It will
still be empty on initial startup.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Brett Konold
Sent
RocksDbTableDescriptor(tableName, KVSerde.of(new StringSerde(), new
StringSerde(.withTtl(this.rocksDbTtl);
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Brett Konold
Sent: Thursday, January 23, 2020
(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com/HelixEducation> |
Facebook<htt
and copy it into the executable jar? All of my
attempts to do this have failed.
Thanks for any advice.
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com/HelixEducation>
is concerned. What
exactly do you mean by 'submitting multiple jobs at once'? I start ours one at
a time though do not verify they are 'Running' until I've submitted all three
instances of the jar.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
Thanks for the clarification. I completely missed that there
LocalApplicationRunner could accept a TaskApplication. The javadocs are
actually hard to find.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
, config);
localApplicationRunner.run();
localApplicationRunner.waitForFinish();
}?
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com/HelixEducation>
to 0 to consume from beginning
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com/HelixEducation> |
Facebook<https://www.facebook.com/HelixEducation> |
LinkedIn
Is zookeeper required for the tech stack? If samza does all of its
checkpointing on kafka, I may be able to save us some money by eliminating the
ZK cluster.
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation
Prateek,
Migrating to v1.0.0 went off without a hitch. Thanks.
Jeremiah Adams
Software Engineer
www.helixeducation.com <http://www.helixeducation.com/>
Blog <http://www.helixeducation.com/blog/> | Twitter
<https://twitter.com/HelixEducation> | Facebook
<htt
to
containers.
Also, does Samza have any hard requirements for Zookeeper since checkpointing
was implemented?
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com
Thanks for checking Prateek. There’s a similar thread in the group. I decided
to follow advice there and am working on migrating to V1.0.0 instead of 0.14.
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation
dException:
org.apache.samza.container.grouper.task.GroupByContainerCount.()
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.newInstance(Class.java:412)
... 8 more
Jeremiah Adams
Software Engineer
www.helixeducation.com <http://www.helixeducation.com
] Caught throwable and stopping run loop
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com/HelixEducation> |
Facebook<https://www.facebook.com/HelixEducat
Thanks you Jagadish!
This is exactly what was looking for. I've been behind samza versions due to
long dev time on a product so I often miss the v.1 documentation.
Jeremiah Adams
Software Engineer
www.helixeducation.com <http://www.helixeducation.com/>
Blog <http://www.helixeduc
of partitions I can
get away with? Any advice on determining the appropriate number of partitions?
Kafka, Zookeeper and Secor are also costs we could potentially reduce via
partition count.
Thanks for any input.
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.
Thanks for the information Tom and Jagadish.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Jagadish Venkatraman
Sent: Tuesday, November 27, 2018 4:43 PM
To: dev@samza.apache.org
Subject: Re: Alerting
din/Burrow) for a test
drive locally, hoping it would solve my problem due to it looking at the
internal consumers. However, I have the same problem - can't get data on a
consumer group that doesn't exist.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook
for
examining a topic's checkpoint aren't readily useful for application
consumption.
Can some of you share your approaches to monitoring and alerting on consumer
lag?
Regards.
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeduc
No luck Prateek. Same problem.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Prateek Maheshwari
Sent: Thursday, October 11, 2018 10:54 AM
To: dev@samza.apache.org
Subject: Re: Need Some Help w/ Gradle
Anyone have a few cycles to help me out with Apache Rat failing the build on
OpenJDK 11?
Some things I’ve tried:
Bumped Gradle to 4.10.2 (required for Java11). Bumped the Rat version to 0.12.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Jeremiah Adams
Sent: Tuesday, October 2, 2018 12:55 PM
To: dev@samza.apache.org
Subject: [POSSIBLE PHISHING] Re: Open JDK 11 Early Results
I am trying both building
%20%3D%20KAFKA%20AND%20text%20~%20%22java%2011%22
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Yi Pan
Sent: Tuesday, October 2, 2018 11:16 AM
To: dev@samza.apache.org
Subject: Re: Open JDK 11 Early
I know very little about Gradle. I got a response on the issue I opened
regarding Gradle builds failing on OpenJDK 11. I've since upgraded the Gradle
version in gradle-wrapper.properties to 4.10.2. Build now gets past the java11
issue but is dying on the Apache rat task. The build/plugin
not start on Open JDK 11
Kafka: - 2.11-2.0.0 - Starts on Open JDK 11
Zookeeper: 3.4.13 - Starts on Open JDK 11
I can’t really test anything interesting until Yarn can play nice on JDK 11.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
Yes it does. Thanks Yi.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Yi Pan
Sent: Tuesday, September 25, 2018 9:09 AM
To: dev@samza.apache.org
Subject: Re: Java 11, Oracle Java and OpenJDK Release
Has anyone had a chance to digest this information regarding Java11 and OpenJDK
release cadences?
What would be next steps to drive adoption of the OpenJDK release cadence
(never participated in an apache proj) ?
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter
ion
http://www.oracle.com/technetwork/java/javaseproducts/javasesubscription-data-sheet-4891969.pdf
https://www.infoq.com/presentations/java-10-11
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twit
/update-and-faq-on-the-java-se-release-cadence
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Jagadish Venkatraman
Sent: Tuesday, September 11, 2018 1:09 AM
To: dev@samza.apache.org
Subject: Re: Path
How does the java 11 changes to Oracle Java and Open JDK affect Samza's release
schedules? When can I upgrade to java 11?
Are there any docs for this information yet?
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeduc
as a side-effect of processing a normal message.
Keep in mind our data is nowhere near 'big'.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Jagadish Venkatraman
Sent: Tuesday, June 5, 2018 10:22 AM
To: dev
ocessing the window()?
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com/HelixEducation> |
Facebook<https://www.facebook.com/HelixEducation> |
LinkedIn<http://www.linkedin.com/company/3609946>
Thanks Prateek. That got me pointed in the right direction.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: prateek...@gmail.com <prateek...@gmail.com> on behalf of Prateek
Maheshwari
org.apache.samza.metrics.JvmMetrics -
updating jvm metrics?
What log4j file do I need to edit to correct this?
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com/HelixEducation>
10.201.10.222:9092
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Jacob Maes <jacob.m...@gmail.com>
Sent: Tuesday, August 15, 2017 9:23 AM
To: dev@samza.apache.org
Subject: Re:
, the directory is empty at appcache/ and
filecache/ both:
[centos@porter-yarn-slave-1 container_1502753192195_0007_02_01]$ ls
/tmp/hadoop-centos/nm-local-dir/usercache/centos/appcache/
[centos@porter-yarn-slave-1 container_1502753192195_0007_02_01]$
Jeremiah Adams
Software Engineer
6b1f500013 : Unable to read additional data from
server sessionid 0x25de16b1f500013, likely server has closed socket
[10.201.9.105] out: 17:18:49.349 [main-EventThread] INFO
org.apache.zookeeper.ClientCnxn - EventThread shut down?
Jeremiah Adams
Software Engineer
www.helixeducatio
am instrumenting a method via Timer.update() which is called n-number of
times in 60 seconds am I seeing an aggregation of the time spent in said method
during a 60 second period or am I seeing the last value sent through
Timer.update()?
Jeremiah Adams
Software Engineer
www.helixeducation.com
? If not it seems
better to implement the Histogram?
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
From: Jagadish Venkatraman <jagadish1...@gmail.com>
Sent: Friday, June 30, 2017 9:08 AM
To: dev@samza.apac
(ApplicationMasterProtocolPBClientImpl.java:91)
... 16 more
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com/HelixEducation> |
Facebook<https://www.facebook.com/HelixEducat
this with docker compose but
this approach seems heavy handed. I am hoping someone has already plowed this
ground.
"No matter how far you have gone on the wrong road, turn back."
- Turkish
proverb
Jeremiah Adams
Softwar
Thank you for the info.
Is there any way to 'pause' the job or stop processing kafka from inside a
StreamTask.process() method? That would work for me too.
Jeremiah Adams
Software Engineer
www.helixeducation.com
Blog | Twitter | Facebook | LinkedIn
right direction?
Thanks in advance.
Jeremiah Adams
Software Engineer
www.helixeducation.com<http://www.helixeducation.com/>
Blog<http://www.helixeducation.com/blog/> |
Twitter<https://twitter.com/HelixEducation> |
Facebook<https://www.facebook.com/HelixEducation> |
LinkedIn&l
We have a samza job configured to run in a yarn cluster. This job consumes
multiple kafka topics and routes the messages to elasticsearch for
indexing. When enough batch-updates to elasticsearch fail using the
ElasticsearchSystemProducer, the entire samza job dies. Due to
checkpointing + yarn, the
48 matches
Mail list logo
