Re: [VOTE] Apache Samza 1.7.0 RC0

-1 I am getting an error when running the integ tests. See logs attached. On Mon, Feb 28, 2022 at 5:28 PM Daniel Chen wrote: > Hey all, This is a call for a vote on a release of Apache Samza 1.7.0. > Thanks to everyone who has contributed to this release. > > The release candidate can be

Apache Samza 1.2.0 is released

Documentation and Blog are published. Thanks everyone.

Re: REMINDER. [VOTE] Apache Samza 1.2.0 RC4

Hi, Vote passed. Thanks. On Sun, Jun 2, 2019 at 5:00 PM Boris Shkolnik wrote: > Hi, > > This is a call for a vote on a release of Apache Samza 1.2.0. Thanks to > everyone who has contributed to this release. > > > The release candidate can be downloaded from here: >

REMINDER. [VOTE] Apache Samza 1.2.0 RC4

Hi, This is a call for a vote on a release of Apache Samza 1.2.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~boryas/samza-1.2.0-rc 4 (this release has a fix

Canceled: [VOTE] Apache Samza 1.2.0 RC3

On Mon, May 27, 2019 at 2:48 PM Boris Shkolnik wrote: > Hi, > > This is a call for a vote on a release of Apache Samza 1.2.0. Thanks to > everyone who has contributed to this release. > > > In this release we reverted commit: > > "SAMZA-2155: Remove log4j l

[VOTE] Apache Samza 1.2.0 RC4

Hi, This is a call for a vote on a release of Apache Samza 1.2.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~boryas/samza-1.2.0-rc 4 (this release has a fix

[VOTE] Apache Samza 1.2.0 RC1

Hi, This is a call for a vote on a release of Apache Samza 1.2.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~boryas/samza-1.2.0-rc1/ The release candidate

[VOTE] Apache Samza 1.2.0 RC0

Hi, This is a call for a vote on a release of Apache Samza 1.2.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~boryas/samza-1.2.0-rc0/ The release candidate is signed with pgp key 0x7D74D0CD5B5EB041, which can

Re: Review Request 68867: Update hell0-samza with latest code

are provided in code and removed from the configs. We need to clarify that users still can use configs and the values of the configs will override values given in the code (to allow redeployments without recompilation). - Boris Shkolnik On Sept. 28, 2018, 11:55 p.m., Xinyu Liu wrote

Re: Review Request 58866: fixed SAMZA-1248. use processor id for stand alone barrier

ally generated e-mail. To reply, visit: https://reviews.apache.org/r/58866/#review173409 --- On April 28, 2017, 11:54 p.m., Boris Shkolnik wrote: > > --- > This is an automatically generated e

Re: Review Request 58866: fixed SAMZA-1248. use processor id for stand alone barrier

/java/org/apache/samza/zk/ZkUtils.java fee840511fbc19da2e19525a97fcfb5812a70a53 samza-core/src/test/java/org/apache/samza/zk/TestZkUtils.java b8dc2953ead2fb11fa22db5ec30b19a74a779830 Diff: https://reviews.apache.org/r/58866/diff/1/ Testing --- Thanks, Boris Shkolnik

Review Request 58866: fixed SAMZA-1248. use processor id for stand alone barrier

fee840511fbc19da2e19525a97fcfb5812a70a53 samza-core/src/test/java/org/apache/samza/zk/TestZkUtils.java b8dc2953ead2fb11fa22db5ec30b19a74a779830 Diff: https://reviews.apache.org/r/58866/diff/1/ Testing --- Thanks, Boris Shkolnik

Re: [DISCUSS] Samza 0.12.0 release

Looks good. +1 to cut the new release. On Fri, Dec 23, 2016 at 10:44 AM, santhosh venkat < santhoshvenkat1...@gmail.com> wrote: > Hi All, > > There have been quite a lot of new features added to master since 0.11 > release to warrant a new major release. At LinkedIn, we've done functional > and

Re: [DISCUSS] Samza 0.12.0 release

Please add SAMZA-1061 (it is docs for 1060) On Fri, Dec 23, 2016 at 10:44 AM, santhosh venkat < santhoshvenkat1...@gmail.com> wrote: > Hi All, > > There have been quite a lot of new features added to master since 0.11 > release to warrant a new

Re: Review Request 54563: added new config for job.changelog.system

/test/scala/org/apache/samza/config/TestStorageConfig.scala 81a35ec719975052f50af249dfcc17249e8342d1 Diff: https://reviews.apache.org/r/54563/diff/ Testing --- added unit test. Thanks, Boris Shkolnik

Review Request 54563: added new config for job.changelog.system

/TestStorageConfig.scala 81a35ec719975052f50af249dfcc17249e8342d1 Diff: https://reviews.apache.org/r/54563/diff/ Testing --- added unit test. Thanks, Boris Shkolnik

Re: Review Request 53826: added user document for Checkpoint callbacks

t; > > > Multiple "and"s in the sentence. Maybe "... for a task. These are the > > ..."? done. - Boris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53826/#review156607

Re: Review Request 53826: added user document for Checkpoint callbacks

://reviews.apache.org/r/53826/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 53826: added user document for Checkpoint callbacks

- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53826/#review156276 --- On Nov. 16, 2016, 11:28 p.m., Boris Shkolnik wrote: > > -

Re: Review Request 53826: added user document for Checkpoint callbacks

iew156279 --- On Nov. 16, 2016, 11:28 p.m., Boris Shkolnik wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://revie

Review Request 53826: added user document for Checkpoint callbacks

--- Thanks, Boris Shkolnik

Re: Review Request 53453: Add optional interface for SystemConsumer checkpontListener() for checkpoint notifications

. Thanks, Boris Shkolnik

Re: Review Request 53453: Add optional interface for SystemConsumer checkpontListener() for checkpoint notifications

. Thanks, Boris Shkolnik

Re: Review Request 53453: Add optional interface for SystemConsumer checkpontListener() for checkpoint notifications

in.. - Boris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53453/#review155495 --- On Nov. 4, 2016, 11:23 p.m., Boris

Re: Review Request 53453: Add optional interface for SystemConsumer checkpontListener() for checkpoint notifications

se having a parameter without a default between parameters with default will confuse compiler when only some of the arguments are provided. - Boris --- This is an automatically generated e-mail. To reply, visit: https://re

Re: Producer stopped during leader switch

Just a follow up question - was a container failing with this error? And if it was - when it was restarted somewhere else, was it still getting the error? On Fri, Oct 28, 2016 at 10:12 AM, David Yu wrote: > Hi, > > We recently experienced a Kafka broker crash. When a

Re: Review Request 53453: Add optional interface for SystemConsumer checkpontListener() for checkpoint notifications

, Boris Shkolnik

Review Request 53453: Add optional interface for SystemConsumer checkpontListener() for checkpoint notifications

cb78223f1b59a78bbeb1e42b5974670a53def504 Diff: https://reviews.apache.org/r/53453/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 52492: Adding monitor to clean up stale local stores of jobs/tasks.

2/#comment223704> Seems like this class may be usefull elsewhere. Is it worth moving it to a top-level class? samza-rest/src/main/java/org/apache/samza/monitor/LocalStoreMonitorConfig.java (line 40) <https://reviews.apache.org/r/52492/#comment223697> can we add a check for it and

[DISCUSS] Offset checkpoint notification to the System Consumer

Currently we keep track of the progress of each consumer using checkpointing mechanism. These checkpoints are controlled by Samza, and the system consumer is not aware of it. But some systems may require knowledge of the current checkpointed position in the stream. For this purpose we propose an

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

lis directly makes it difficult to unit test. samza-core/src/test/scala/org/apache/samza/storage/TestTaskStorageManager.scala (line 323) <https://reviews.apache.org/r/52476/#comment222800> isn't it easier just to make method package private? - Boris Shkolnik On Oct. 19, 2016, 10:0

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52476/#review153476 --- Ship it! - Boris Shkolnik On Oct. 19, 2016, 10:04 p.m

Re: Review Request 53028: SAMZA-1040: Revert the ClassLoaderHelper change in SamzaContainer

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53028/#review153466 --- Ship it! Ship It! - Boris Shkolnik On Oct. 19, 2016, 6:30

Re: Review Request 53000: SAMZA-1038: Update hello-samza master to use Samza 0.11.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53000/#review153462 --- Ship it! Ship It! - Boris Shkolnik On Oct. 18, 2016, 10:17

Re: Review Request 52962: SAMZA-1029: Prepare release candidate for 0.11.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52962/#review153105 --- Ship it! - Boris Shkolnik On Oct. 18, 2016, midnight, Xinyu

Re: [VOTE] Apache Samza 0.11.0 RC2

+1 (non-binding) built, ran tests, verified MD5 and the signature. Boris. On Mon, Oct 10, 2016 at 4:07 PM, xinyu liu wrote: > Hey all, > > This is a call for a vote on a release of Apache Samza 0.11.0. Thanks to > everyone who has contributed to this release. We are very

Re: [DISCUSS] [VOTE] Apache Samza 0.11.0 RC0

+1 On Fri, Sep 30, 2016 at 1:39 PM, xinyu liu wrote: > Subject correction: [VOTE] Apache Samza 0.11.0 RC0. > > Thanks, > Xinyu > > On Fri, Sep 30, 2016 at 12:00 PM, xinyu liu wrote: > > > Hey all, > > > > This is a call for a vote on a release of

Re: [Discuss] Moving Samza to Java 1.8 source compatibility.

+1 for moving to 1.8. On Wed, Apr 27, 2016 at 6:27 PM, Jacob Maes wrote: > Hey everyone, > > I wanted to start a discussion to see what folks think about moving to Java > 1.8 source compatibility at some point after the 10.1 release. > > Java 8 has a number of nice

Re: Review Request 52140: added docs for split deployment

a997445db2af40e3ac87dc01d510a5e9045d73f4 docs/learn/documentation/versioned/jobs/split-deployment.md PRE-CREATION Diff: https://reviews.apache.org/r/52140/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 52140: added docs for split deployment

--- Thanks, Boris Shkolnik

Re: Review Request 52140: added docs for split deployment

: samza Description --- added docs for split deployment Diffs (updated) - docs/learn/documentation/versioned/operations/split-deployment.md PRE-CREATION Diff: https://reviews.apache.org/r/52140/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 52140: added docs for split deployment

urrent deployment). - Boris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52140/#review149931 --- On Sept. 22, 2016, 12:48 a.m.,

Re: Review Request 52133: reopen a Kafka system producer if send called again

://reviews.apache.org/r/52133/diff/ Testing --- manual. Thanks, Boris Shkolnik

Review Request 52140: added docs for split deployment

for split deployment Diffs - docs/learn/documentation/versioned/operations/split-deployment.md PRE-CREATION Diff: https://reviews.apache.org/r/52140/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 51934: reopen a Kafka system producer if send called again

/KafkaSystemProducer.scala 5ff6d3caf54ed148aa40c7c752c587e556a4f34a Diff: https://reviews.apache.org/r/51934/diff/ Testing --- manual testing. Thanks, Boris Shkolnik

Review Request 52133: reopen a Kafka system producer if send called again

a Kafka system producer if send called again Diffs - samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemProducer.scala 5ff6d3caf54ed148aa40c7c752c587e556a4f34a Diff: https://reviews.apache.org/r/52133/diff/ Testing --- manual. Thanks, Boris Shkolnik

Review Request 51934: reopen a Kafka system producer if send called again

/diff/ Testing --- manual testing. Thanks, Boris Shkolnik

Re: Review Request 50583: SAMZA-954 Improve logging for Samza

/Logging.scala (line 31) <https://reviews.apache.org/r/50583/#comment212368> Suggestion: what if we need a message at a WARN level. May be you can have another parameter 'level' with the default set to INFO. - Boris Shkolnik On July 28, 2016, 8:41 p.m., Jake Maes

Re: [DISCUSS] [VOTE] Apache Samza 0.10.1 RC0

+1 (non-binding). Boris. On Mon, Aug 1, 2016 at 11:39 AM, Navina Ramesh wrote: > Hey all, > > This is a call for a vote on a release of Apache Samza 0.10.0. Thanks > to everyone > who has contributed to this release. We are very glad to see some new >

Re: Review Request 50318: SAMZA-979: Remove KafkaCheckpointMigration

a version before migration to version 11 directly? - Boris Shkolnik On July 25, 2016, 9:29 p.m., Xinyu Liu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache

Re: Review Request 48862: allow empty serde for SystemStream

://reviews.apache.org/r/48862/diff/ Testing --- Manual. Thanks, Boris Shkolnik

Re: Review Request 48862: allow empty serde for SystemStream

samza-core/src/main/scala/org/apache/samza/util/Util.scala fc3d085d7fff9f7dcec766ba48e550eb0052e99d Diff: https://reviews.apache.org/r/48862/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 48862: allow empty serde for SystemStream

samza-core/src/main/scala/org/apache/samza/util/Util.scala fc3d085d7fff9f7dcec766ba48e550eb0052e99d Diff: https://reviews.apache.org/r/48862/diff/ Testing --- Thanks, Boris Shkolnik

Review Request 48862: allow empty serde for SystemStream

: https://reviews.apache.org/r/48862/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 48352: Added java friendly api to KafkaMetrics

straighforward Java api to this class. Diffs (updated) - samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumerMetrics.scala 741a176259b9a1cf2e5f7d64b0be98497f1c9242 Diff: https://reviews.apache.org/r/48352/diff/ Testing --- manual testing. Thanks, Boris Shkolnik

Review Request 48352: Added java friendly api to KafkaMetrics

- samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumerMetrics.scala 741a176259b9a1cf2e5f7d64b0be98497f1c9242 Diff: https://reviews.apache.org/r/48352/diff/ Testing --- manual testing. Thanks, Boris Shkolnik

Re: Review Request 47251: SAMZA-852 - Better logging when system can not be created Always log the exception when we cant instantiate a producer or consumer

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47251/#review133835 --- Ship it! - Boris Shkolnik On May 11, 2016, 6:53 p.m., Jake

Re: Review Request 47029: SAMZA-932 port collisions in JmxServer

/JmxServer.scala (line 92) <https://reviews.apache.org/r/47029/#comment196122> Can you please add a log line, so we can know what port was chosen. - Boris Shkolnik On May 5, 2016, 4:50 p.m., Jake Maes wrote: > > --- > This is a

Re: Review Request 47029: SAMZA-932 port collisions in JmxServer

doesn't work with SSL? - Boris Shkolnik On May 5, 2016, 4:50 p.m., Jake Maes wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache

Re: How to implement broadcasted KeyValueStorage with 0.9

I may be missing something - but doesn't that sounds exactly as broadcast stream? You can make all the tasks read from all the partitions. ( https://samza.apache.org/learn/documentation/0.10/container/samza-container.html, search for broadcast) On Tue, May 3, 2016 at 12:46 PM, Jacob Maes

Re: Review Request 46732: SAMZA-930 fix issue with json deserialisation in YarnUtil

and the fix. - Boris Shkolnik On April 29, 2016, 11:13 a.m., Alex Buck wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache

Re: Review Request 46129: SAMZA 910 fix the container allocator tests

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46129/#review128698 --- test ship it - don't! - Boris Shkolnik On April 13, 2016, 1:51

Re: Review Request 46129: SAMZA 910 fix the container allocator tests

129/#comment192165> Test comment! - Boris Shkolnik On April 13, 2016, 1:51 a.m., Jake Maes wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.

Re: Review Request 45464: SAMZA-922 Host Affinity - Bug in SamzaContainerRequest causes (recoverable) exceptions in YARN

/TestSamzaContainerRequest.java (line 45) <https://reviews.apache.org/r/45464/#comment191932> But it is passed!? Or the name of the test is confusing.. - Boris Shkolnik On March 30, 2016, 12:30 a.m., Jake Maes

Re: Review Request 45504: SAMZA-924: Add disk space monitoring

May be it would be cleaner to add a getStoreLocation() method to the TaskStorageManager and call it for each taskInstance? - Boris Shkolnik On March 30, 2016, 7:43 p.m., Chris Pettitt wrote: > > --- > This is an automatically g

Re: Review Request 44604: split deployment logic

s an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44604/#review125429 --- On March 15, 2016, 5:33 p.m., Boris Shkolnik wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https:

Re: Review Request 44418: SAMZA-887: Use cached JobModel everywhere in the Samza AM container

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44418/#review124688 --- Ship it! Ship It! - Boris Shkolnik On March 5, 2016, 7:08

Re: Review Request 44604: split deployment logic

/YarnJob.scala 1aa26bba00d2c3d42ef3b8951fa72314d410 samza-yarn/src/test/java/org/apache/samza/job/yarn/TestSamzaTaskManager.java 9da1edf6ff165ef0306de8730853ad30551a9831 Diff: https://reviews.apache.org/r/44604/diff/ Testing --- Thanks, Boris Shkolnik

Review Request 44604: split deployment logic

/yarn/TestSamzaTaskManager.java 88d9f24d16fc3d9842b387cfc22edaf1dfa6fd06 Diff: https://reviews.apache.org/r/44604/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 44241: SAMZA-883 Improve logging for container handling and kafka refresh

nly for container allocations. So it should not add too much noise and should be quite helpfull. - Boris Shkolnik On March 2, 2016, 2:24 a.m., Jake Maes wrote: > > --- > This is an automatically generated e-mail. To reply, vi

Re: checkpoint example?

To add to Jacob's and Jagadish's answers. If you want to to read from 24 hours before (not from the beginning or the end of the stream) you can set the checkpoint interval (see Jagadish's comment) to 24 hours. It is kind of unusual, but should work :). On Tue, Mar 1, 2016 at 4:32 PM, Jacob Maes

Re: Review Request 43550: SAMZA-836: fix unit test failure w/ FlushOptions() in rocksdbjni-3.13.1

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43550/#review121371 --- Ship it! I wish we understood the reason for it. - Boris

Re: Review Request 43766: SAMZA-872: little change in Logging docs

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43766/#review120637 --- Ship it! after addressing Jagadish's comment. - Boris

Re: Review Request 43053: allow warning instead of fail in case of invalid num of partitions in the checkpoint partition

--- Thanks, Boris Shkolnik

Re: Review Request 43053: allow warning instead of fail in case of invalid num of partitions in the checkpoint partition

/test/scala/org/apache/samza/checkpoint/kafka/TestKafkaCheckpointManager.scala af4051b28df5eeaeaee527a24907a8e66441f743 Diff: https://reviews.apache.org/r/43053/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 43648: SAMZA-872: removed unsafe log4j DaillyRollingFileAppender from hello-samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43648/#review119545 --- Ship it! Ship It! - Boris Shkolnik On Feb. 17, 2016, 11:01

Re: Review Request 39032: SAMZA-787: task.log4j.system should not be guessed if not configured

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39032/#review118471 --- Ship it! - Boris Shkolnik On Oct. 6, 2015, 12:44 a.m

Re: Review Request 43053: allow warning instead of fail in case of invalid num of partitions in the checkpoint partition

/TestKafkaCheckpointManager.scala af4051b28df5eeaeaee527a24907a8e66441f743 Diff: https://reviews.apache.org/r/43053/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 43053: allow warning instead of fail in case of invalid num of partitions in the checkpoint partition

t; nit: failOnTopicValidation. tnx, fixed. - Boris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43053/#review117554 ------- On Feb. 3

Re: ChangeLog Question for TTL rocksDB stores

As Jacob mentioned there is not direct relationship between the rocksdb tts (internal to rocksdb) and changelog (done by Samza). The problem may arise if the store is restored from the changelog, since the log will have the expired entries, and they will be entered with the NEW date (and as Yi

Re: Review Request 43053: allow warning instead of fail in case of invalid num of partitions in the checkpoint partition

/TestKafkaCheckpointManager.scala af4051b28df5eeaeaee527a24907a8e66441f743 Diff: https://reviews.apache.org/r/43053/diff/ Testing --- Thanks, Boris Shkolnik

Review Request 42815: issue a warn() instead of error() in case of rocksdb store with both TTL and changelog enabled

211fc3be1e168f1f92812406785b39b5a3fd9174 Diff: https://reviews.apache.org/r/42815/diff/ Testing --- Thanks, Boris Shkolnik

Review Request 42619: move public kv api from samza-kv to samza-api package

/apache/samza/storage/kv/KeyValueStore.java Diff: https://reviews.apache.org/r/42619/diff/ Testing --- Thanks, Boris Shkolnik

Re: Review Request 41884: SAMZA-802: KafkaSystemAdmin needs to handle empty topic offsets

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41884/#review112954 --- Ship it! Logic seems correct. - Boris Shkolnik On Jan. 4, 2016

Re: Review Request 41663: SAMZA-843 : Slow start of Samza jobs with large number of containers

<https://reviews.apache.org/r/41663/#comment173183> Where is the change to JobCoordinator constructor? Did I miss it? - Boris Shkolnik On Jan. 4, 2016, 10:40 p.m., Navina Ramesh wrote: > > --- > This is an automatically gener

Re: Review Request 41874: SAMZA-837: Support Gradle 2.9

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41874/#review112707 --- Ship it! Ship It! - Boris Shkolnik On Jan. 4, 2016, 9:59 a.m

Re: Review Request 41106: SAMZA-833: ProcessJob mishandling containers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41106/#review109619 --- Ship it! - Boris Shkolnik On Dec. 9, 2015, 6:41 a.m., Tao Feng

Re: Passing Java system properties to YARN containers of Samza Job

I am not sure about AWS credentials, but, in general, you can use task.opts config value to pass any java level settings, which will be passed to each container jvm. On Tue, Dec 8, 2015 at 9:57 AM, Gordon Tai wrote: > Hi, > > Is there any convenient way to pass custom Java

Re: Review Request 40933: SAMZA-829: Add YARN configure doc to allow clean shutdown for large state jobs

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40933/#review109673 --- Ship it! Ship It! - Boris Shkolnik On Dec. 3, 2015, 11:10 p.m

Re: [VOTE] Samza 0.10.0 Release Candidate 2

I've opened a ticket to track this rocksdb:test failure: 1. SAMZA-836 <https://issues.apache.org/jira/browse/SAMZA-836> On Wed, Dec 9, 2015 at 7:11 PM, Boris Shkolnik <bor...@gmail.com> wrote: > +1 (non-binding). > One note. I failed for me couple of times w

Re: Review Request 40857: SAMZA 826 Fix string format issue with log message in ContainerUtil

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40857/#review109674 --- Ship it! Ship It! - Boris Shkolnik On Dec. 2, 2015, 7:53 a.m

Re: Review Request 41071: SAMZA-843: fix heap usage increase caused by container timer change

be using preallocated storage for timer, to avoid allocation and deallocation of the map values. - Boris Shkolnik On Dec. 8, 2015, 2:30 a.m., Xinyu Liu wrote: > > --- > This is an automatically generated e-mail. To reply, visit

Re: Review Request 40485: SAMZA-767 yarn.queue option is not used anywhere

/YarnJob.scala (line 76) <https://reviews.apache.org/r/40485/#comment166262> nit. add a new line here. - Boris Shkolnik On Nov. 19, 2015, 2:56 p.m., Aleksandar Pejakovic wrote: > > --- > This is an automatically generated e

Re: Review Request 40313: SAMZA-785

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40313/#review106697 --- Ship it! - Boris Shkolnik On Nov. 13, 2015, 10:51 p.m., VENKATA

Re: Monitoring consumer lag

Just to add to Jagasish's suggestion - you can configure MetricsSnapshotRecorder to output the metrics into a stream and read them from there if it works better for you. Boris. On Mon, Nov 16, 2015 at 12:32 PM, Jordan Shaw wrote: > Michael, > I should have added if your

Re: API for containers?

Here is couple of ideas (not sure if that exactly what you want). You can read JobModel of a job (form JobCoordinator server) and this job model will contain container to tasks (and to partitions) mapping. In order to find containers and the nodes - you can use yarn web api. http://RM/cluster,

Re: Review Request 39806: SAMZA-798 : Performance and stability issue after combining checkpoint and coordinator stream

ration). Or am I missing something? - Boris Shkolnik On Nov. 2, 2015, 9:56 p.m., Navina Ramesh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.

Re: Review Request 35397: Fix SAMZA-697

? This - Boris Shkolnik On June 18, 2015, 6:42 p.m., Guozhang Wang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35397