Re: Review Request 52476: SAMZA-1083 : Do not load task store which are older than delete tombstones.

/StorageConfig.scala (line 32) <https://reviews.apache.org/r/52476/#comment236506> Doesn't look like this is fixed? Did you miss updating with a patch? - Prateek Maheshwari On Feb. 8, 2017, 11:09 a.m., Shanthoosh Venkataraman

Re: Review Request 52476: SAMZA-1083 : Do not load task store which are older than delete tombstones.

a/org/apache/samza/storage/TaskStorageManager.scala (line 202) <https://reviews.apache.org/r/52476/#comment236564> Thanks! - Prateek Maheshwari On Feb. 8, 2017, 1:37 p.m., Shanthoosh Venkataraman wrote: > > ---

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

":" here and elsewhere, including method type annotations (e.g. line 169, 185). samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala (line 152) <https://reviews.apache.org/r/52476/#comment234450> We should log which directory we're using for the store here at

Re: Review Request 52168: Tasks endpoint to list the complete details of all tasks related to a job

ructs a bad request (HTTP 400) response" samza-rest/src/main/java/org/apache/samza/rest/resources/TasksResource.java (line 40) <https://reviews.apache.org/r/52168/#comment238534> If this resource is always for a particular job, should "/{jobName}/{jobId}" shouldn't go here instead o

Re: Review Request 51252: SAMZA-1004: Fix some logging and javadoc issues for AsyncStreamTask

<https://reviews.apache.org/r/51252/#comment214493> s/serialized/serial? - Prateek Maheshwari On Aug. 30, 2016, 3:01 p.m., Xinyu Liu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://re

Re: Review Request 51142: SAMZA-967: HDFS System Consumer

ame, but the hdfs producer configs do. The kafka convention is better IMHO. Either way, we should at least be consistent between this and the new partitioner/reader configs which don't have the hdfs prefix. - Prateek Maheshwari On Sept. 28, 201

Re: Review Request 51142: SAMZA-967: HDFS System Consumer

> On Sept. 29, 2016, 10:56 a.m., Prateek Maheshwari wrote: > > samza-hdfs/src/main/scala/org/apache/samza/system/hdfs/HdfsConfig.scala, > > line 66 > > <https://reviews.apache.org/r/51142/diff/5/?file=1493810#file1493810line66> > > > > "systems.%s

Re: Review Request 52403: SAMZA-1028: Moving logline before closing kafka producer and making exception thrown AtomicReference

ilure, so the source is a moot point). - Prateek Maheshwari On Sept. 29, 2016, 2:23 p.m., Xinyu Liu wrote: > > --- > This is an automatically generated e-mail. To reply

Re: Review Request 52403: SAMZA-1028: Moving logline before closing kafka producer and making exception thrown AtomicReference

> On Sept. 29, 2016, 12:33 p.m., Prateek Maheshwari wrote: > > samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemProducer.scala, > > line 202 > > <https://reviews.apache.org/r/52403/diff/1/?file=1516363#file1516363line202> > > > >

Re: Review Request 51346: SAMZA-974 - Support finite datasources in Samza that have a notion of End-Of-Stream

samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala (line 354) <https://reviews.apache.org/r/51346/#comment213533> Just wondering, is 'checkX' our naming convention for methods with a check + side effects? - Prateek Maheshwari On Aug. 24, 2

Re: Review Request 52403: SAMZA-1028: Moving logline before closing kafka producer and making exception thrown AtomicReference

e callback exception, the user will not get a chance to ignore the exception. - Prateek Maheshwari On Sept. 29, 2016, 2:23 p.m., Xinyu Liu wrote: > > --- > This is an automatically

Re: Review Request 53297: Initial version of adding metrics into samza rest.

xist yet). samza-rest/src/main/java/org/apache/samza/rest/SamzaRestConfig.java (line 55) <https://reviews.apache.org/r/53297/#comment225267> What is this config, and what does container refer to here? The samza-rest service container? - Prateek Maheshwari On Nov. 8, 2016, 3:13

Re: Review Request 53453: Add optional interface for SystemConsumer checkpontListener() for checkpoint notifications

views.apache.org/r/53453/#comment225260> Indent by 2 Space before and after => Space b/w map and { samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala (line 366) <https://reviews.apache.org/r/53453/#comment225261> Indent by 2. -

Re: Review Request 53282: SAMZA-1043: Samza performance improvements

dependencies: StorageEngine is reading the Metrics configuration out of the entire container context, so that it can choose what clock the Timer should use. Haven't thought this through, but maybe we should refactor things to move the clock to the Timer class instead? - Prateek Maheshwari

Re: Review Request 53282: SAMZA-1043: Samza performance improvements

> On Nov. 2, 2016, 12:57 p.m., Prateek Maheshwari wrote: > > Sorry for the late reply, didn't get an email notification for your replies. > On Nov. 2, 2016, 12:57 p.m., Prateek Maheshwari wrote: > > samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.s

Re: Review Request 52168: Tasks endpoint to list the complete details of all tasks related to a job

> On Nov. 2, 2016, 11:32 p.m., Prateek Maheshwari wrote: > > docs/learn/documentation/versioned/rest/resources/tasks.md, line 55 > > <https://reviews.apache.org/r/52168/diff/8/?file=1552720#file1552720line55> > > > > What's the difference b/w containerId a

Re: Review Request 53282: SAMZA-1043: Samza performance improvements

; samza-core/src/main/scala/org/apache/samza/util/Util.scala (line 393) <https://reviews.apache.org/r/53282/#comment225451> s/TimerClock/HighResolutionClock - Prateek Maheshwari On Nov. 9, 2016, 11:10 a.m., Xinyu Liu wrote: > > --

Re: Review Request 53297: Initial version of adding metrics into samza rest.

> On Nov. 5, 2016, 11:47 a.m., Jake Maes wrote: > > samza-rest/src/main/java/org/apache/samza/rest/SamzaRestService.java, line > > 75 > > > > > > I don't think the MetricsConfig constructure takes a subset. > >

Re: Review Request 53297: Initial version of adding metrics into samza rest.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53297/#review155689 --- Ship it! Ship It! - Prateek Maheshwari On Nov. 10, 2016, 4

Re: Review Request 53297: Initial version of adding metrics into samza rest.

a/job/yarn/SamzaAppMasterMetrics.scala (line 46) <https://reviews.apache.org/r/53297/#comment225641> Maybe use JavaCoverters._ and .asScala? - Prateek Maheshwari On Nov. 9, 2016, 5:04 p.m., Shanthoosh Venkataraman wrote: > > --

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

er if fileOffset was updated explicitly (after staleness checks etc.) and not as a side effect of reading the file. If you don't, let's add return type to method signature. samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala (line 155) <https://reviews.apache.o

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

greater than/is older than. - Prateek Maheshwari On Oct. 22, 2016, 3:06 p.m., Shanthoosh Venkataraman wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://re

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

.apache.org/r/52476/#comment222830> implicit conversion should probably work. - Prateek Maheshwari On Oct. 19, 2016, 3:04 p.m., Shanthoosh Venkataraman wrote: > > --- > Thi

Review Request 53027: SAMZA-1017 - Added disk quota based throttling to AsyncRunLoop.

.scala cff6b96 Diff: https://reviews.apache.org/r/53027/diff/ Testing --- Tested locally with a hello world app. Thanks, Prateek Maheshwari

Re: Review Request 53027: SAMZA-1017 - Added disk quota based throttling to AsyncRunLoop.

ff6b96 Diff: https://reviews.apache.org/r/53027/diff/ Testing --- Tested locally with a hello world app. Thanks, Prateek Maheshwari

Re: Review Request 52403: SAMZA-1028: Moving logline before closing kafka producer and making exception thrown AtomicReference

> On Sept. 29, 2016, 2:30 p.m., Prateek Maheshwari wrote: > > samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemProducer.scala, > > line 220 > > <https://reviews.apache.org/r/52403/diff/2/?file=1516385#file1516385line220> > > > > If t

Re: Review Request 54020: Operator API refactoring

://reviews.apache.org/r/54020/diff/ Testing --- ./gradlew clean build works. Thanks, Prateek Maheshwari

Re: Review Request 53826: added user document for Checkpoint callbacks

"push system" mean? Maybe clarify/remove? docs/learn/documentation/versioned/container/checkpointing.md (line 138) <https://reviews.apache.org/r/53826/#comment226781> Multiple "and"s in the sentence. Maybe "... for a task. These are the ..."? - Prateek

Re: Review Request 53826: added user document for Checkpoint callbacks

restart) attempt to `seek()` from the checkpointed offset for all systems? "Note: The" or "Note that the" - Prateek Maheshwari On Nov. 16, 2016, 3:28 p.m., Boris Shkolnik wrote: > > --

Re: Review Request 53282: SAMZA-1043: Samza performance improvements

d method to have. Would prefer to remove. samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala (line 137) <https://reviews.apache.org/r/53282/#comment224249> See commen

Re: Review Request 52168: Tasks endpoint to list the complete details of all tasks related to a job

s://reviews.apache.org/r/52168/#comment224341> "partition id" samza-rest/src/main/java/org/apache/samza/rest/proxy/job/SimpleYarnJobProxy.java (line 51) <https://reviews.apache.org/r/52168/#comment224344> config.getConfig

Re: [DISCUSS] SEP-1: Semantics of ProcessorId in Samza

rocessor in the job? If so, probably worth calling > out in the SEP or configuration docs. We can also document it as deprecated > and a candidate for removal in near future (maybe 0.14?). > Yes. That is still a requirement. I think I updated the document regarding > deprecating and removing i

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

gt; > > > regarding to which processorId we are getting. > > > > > > > > Otherwise, the proposal looks good. > > > > > > > > -Yi > > > > > > > > On Wed, Mar 29, 2017 at 7:57 PM, Navina Ramesh > > > >

Re: [DISCUSS] Support Scala 2.12

Hi Maksim, Thanks for the PR comments and updates. Looks good to me too. - Prateek On Wed, Mar 29, 2017 at 12:17 PM, Prateek Maheshwari < pmaheshw...@linkedin.com> wrote: > Hi Maksim, > > I'm in favor of adding Scala 2.12 support as well, thanks for the PR. > I have a f

Re: [DISCUSS] Support Scala 2.12

Hi Maksim, I'm in favor of adding Scala 2.12 support as well, thanks for the PR. I have a few questions about the way JavaConverter APIs and some of the conversions in the PR work. I'll try it out locally and update the PR with feedback/questions soon. Thanks, Prateek On Tue, Mar 28, 2017 at

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

+1 (non binding) from me. - Prateek On Tue, Mar 28, 2017 at 2:17 PM, Boris S wrote: > +1 Looks good to me. > > On Tue, Mar 28, 2017 at 2:00 PM, xinyu liu wrote: > > > +1 on my side. Very happy to see this proposal. This is a blocker for > > integrating

Re: [DISCUSS] SEP-1: Semantics of ProcessorId in Samza

Hi Navina, Thanks for SEP-1, looks pretty good to me. A few questions/comments: Implementation/Interface related: 1. Do you have any examples of custom processor IDs? Wondering what information/classes ProcessorIdGenerator would need to be able to generate one. 2. The default "static"

Re: [DISCUSS] SEP-2: ApplicationRunner Design

Thanks for putting this together Yi! I agree with Jake, it does seem like there are a few too many moving parts here. That said, the problem being solved is pretty broad, so let me try to summarize my current understanding of the requirements. Please correct me if I'm wrong or missing something.

Re: Kafka client.id collision

+1 for adding system name to the client id. - Prateek On Thu, Jul 20, 2017 at 10:43 AM, Navina Ramesh (Apache) wrote: > Hi David, > > I think this is expected to occur as a warning since we spin up all kafka > clients with the same client-id, which is $job.name + $job.id. >

Re: [VOTE] Apache Samza 0.13.0 RC0

Resent the CANCEL email, hopefully it makes it this time. - Prateek On Wed, May 17, 2017 at 2:08 PM, Navina Ramesh (Apache) wrote: > Prateek told me that he sent out a cancel email. It didn't reach the > mail-archive I think. Lately, we have this kind of issues where the

Re: [DISCUSS] SEP-2: ApplicationRunner Design

gt; > > > > For this version of API, seems we don't need the > StreamApplication > >>>> > > > wrapper > >>>> > > > > as well as exposing the StreamGraph. Do you think we are on > the > >>>> right &g

Re: [DISCUSS] SEP-6: Support Watermark Across Intermediate Streams for Batch Processing

Hi Xinyu, Thanks for the proposal. Some requests for clarifications. Let's update the SEP directly instead of replying here. E.g., in "For any following intermediate stream whose input streams are all end-of-stream, it will be marked as pending EOS" - Should clarify that (IIUC) something is

[DISCUSS] Samza 0.13.0 release

Hi all, There have been quite a lot of new features added to master since 0.12 release to warrant a new major release. At LinkedIn, we've done functional and performance testing against master in the past weeks, and deployed jobs with the latest build in production. We will continue to test for

Re: Urgent : Help with latency / backlog / topic lag

d.mode = true > > And actually I think we did catch up with that setting. I have since > completed also the merge of 0.14.1 and we are able to keep up with the input > now. > > Thanks again for the pointers and the fast response! > > -Original Message- > From: Prateek Maheshw

Re: Urgent : Help with latency / backlog / topic lag

Hi Thunder, > What we believe may be happening is that most of the topics have no backlog, but one topic has all the backlog (this is because one of the topics accounts for ~60% of the total message rate). Could there be something inducing extra latency on processing the one topic with a backlog

Re: Urgent : Help with latency / backlog / topic lag

chieve ~700/sec > > > -Original Message- > From: Prateek Maheshwari [mailto:prateek...@gmail.com] > Sent: Friday, June 8, 2018 13:54 > To: dev@samza.apache.org > Subject: Re: Urgent : Help with latency / backlog / topic lag > > Hi Thunder, > >> What we believe may be hap

Re: Welcome Xinyu as new Samza PMC!

This is great news. Congrats Xinyu, and thanks for your contributions! > On Jan 17, 2018, at 10:39 AM, Srinivasulu Punuru wrote: > > Congrats Xinyu, Very well deserved! > > > From: Jagadish Venkatraman > Sent:

[DISCUSS] SEP-14: System and Stream Descriptors

Hi all, Here's the proposal for System and Stream Descriptors - a way of specifying systems, input and output streams properties in application code instead of configurations. https://cwiki.apache.org/confluence/display/SAMZA/SEP-14%3A+System+and+Stream+Descriptors Here's the PR with an

Re: [DISCUSS] SEP-14: System and Stream Descriptors

g 6, 2018, at 4:20 PM, Prateek Maheshwari wrote: > > Hi all, > > Here's the proposal for System and Stream Descriptors - a way of > specifying systems, input and output streams properties in application > code instead of configurations. > https://cwiki.apache.org/confluence

Re: Samza 0.14.1 : OffsetOutOfRangeException even with auto.offset.reset=smallest

Hi Thunder, Can you provide debug level logs from KafkaSystemConsumer with the stack trace for the exception? It'll help figure out why the auto.offset.reset property isn't taking effect. If this error is due to an older checkpoint for the stream, you can try resetting the checkpoint using the

Re: Old style "low level" Tasks with alternative deployment model(s)

Hi Thunder, Can you please take and attach a thread dump with this? Thanks, Prateek On Fri, Mar 16, 2018 at 4:47 PM, Thunder Stumpges wrote: > It appears it IS hung while serializing the JobModel... very strange! I > added some debug statements around the calls: > >

Re: Old style "low level" Tasks with alternative deployment model(s)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ > ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at java.util.concurrent.ScheduledThreadPoolExecutor$ > DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) > at java.util.concurrent.ScheduledThreadPoolExecut

Re: Need Some Help w/ Gradle Build on OpenJDK 11

Hi Jeremiah, We fixed a Rat related issue yesterday in https://github.com/apache/samza/pull/703/. I don't know if this is the same issue you were running into, but might be worth trying again with the latest master. Thanks, Prateek On Wed, Oct 10, 2018 at 6:58 AM Jeremiah Adams wrote: > >

[VOTE] SEP-14: System and Stream Descriptors

Hi folks, Now that SAMZA-1804 has been implemented and reviewed, we've updated SEP-14 with the latest APIs and design decisions. Please vote for accepting SEP-14 in its current form for the upcoming Samza 1.0 release.

Re: [VOTE] SEP-13: unified ApplicationDescriptor and ApplicationRunner APIs for high and low- level APIs in YARN and standalone deployment

+1 (non-binding) from me. Thanks for making the changes and updating the SEP! - Prateek On Fri, Oct 12, 2018 at 12:15 PM Yi Pan wrote: > > Hi, all, > > Given SAMZA-1789 has been reviewed and implemented, SEP-13 has been updated > to the latest API classes as well. Please vote on whether there

[VOTE] Apache Samza 1.0.0 RC1

Hi all, This is a call for a vote on a release of Apache Samza 1.0.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~pmaheshwari/samza-1.0.0-rc1/ The release candidate is signed with pgp key 6585B3D7, which can

[CANCEL][VOTE] Apache Samza 1.0.0 RC0

Hi all, This is the CANCEL notification for the 1.0.0 RC0. We found an integration test setup issue that we will fix. We will also include the following PR in the new RC: SAMZA-1901: Implementation of Samza SQL Shell, Thanks, Prateek

Re: [VOTE] Apache Samza 1.0.0 RC0

gt; > -- Jagadish > > > > On Fri, Oct 19, 2018 at 6:59 PM Prateek Maheshwari > wrote: > > > Hi all, > > > > This is a call for a vote on a release of Apache Samza 1.0.0. Thanks to > > everyone who has contributed to this release. > > &g

Re: [VOTE] SEP-15: New Runtime Context API

gt; > jagadish1...@gmail.com> wrote: > > > > > +1 (binding) from my side. > > > > > > LGTM > > > > > > On Mon, Oct 15, 2018 at 10:44 AM Prateek Maheshwari > > > > > wrote: > > > > > > > +1 (non-bindi

[VOTE] Apache Samza 1.0.0 RC0

Hi all, This is a call for a vote on a release of Apache Samza 1.0.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~pmaheshwari/samza-1.0.0-rc0/ The release candidate is signed with pgp key 6585B3D7, which can

Re: [VOTE] SEP-14: System and Stream Descriptors

On Fri, Oct 12, 2018 at 12:30 PM Prateek Maheshwari > wrote: > > > Hi folks, > > > > Now that SAMZA-1804 has been implemented and reviewed, we've updated > > SEP-14 with the latest APIs and design decisions. > > > > Please vote for accepting SEP-14 in i

Re: [VOTE] SEP-15: New Runtime Context API

+1 (non-binding) for these changes. - Prateek > On Oct 12, 2018, at 3:27 PM, Cameron Lee wrote: > > Hi all, > > SEP-15 has been updated now that SAMZA-1714 has been reviewed and implemented. > Please vote on whether there are further breaking changes needed in the API > or we can accept this

Re: [VOTE] SEP-15: New Runtime Context API

+1 (non-binding) for these changes. (Resending from a non-LI email due to email delivery issues) - Prateek On Fri, Oct 12, 2018 at 3:28 PM Cameron Lee wrote: > > Hi all, > > SEP-15 has been updated now that SAMZA-1714 has been reviewed and implemented. > Please vote on whether there are further

Re: [VOTE] Apache Samza 1.0.0 RC2

> > On Tue, Oct 23, 2018 at 2:50 PM Boris S wrote: > > > Ran build, test and integration test on Linux. > > Verified the signatures. > > > > +1 > > > > On Tue, Oct 23, 2018 at 11:55 AM Prateek Maheshwari > > wrote: > > > > > Hi all, &g

[CANCEL] [VOTE] Apache Samza 1.0.0 RC1

Hi all, This is the CANCEL notification for the 1.0.0 RC1. We found a checkstyle issue that we will fix in the new RC. Thanks, Prateek

Re: [VOTE] Apache Samza 1.0.0 RC1

/samza-test/src/main/java/org/apache/samza/test/integration/TestStandaloneIntegrationApplication.java:44: > 'method def' child have incorrect indentation level 5, expected level > should be 4. > > Thanks. > > On Mon, Oct 22, 2018 at 9:09 PM Prateek Maheshwari > wrote: > &g

[VOTE] Apache Samza 1.0.0 RC2

Hi all, This is a call for a vote on a release of Apache Samza 1.0.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~pmaheshwari/samza-1.0.0-rc2/ The release candidate is signed with pgp key 6585B3D7, which can

[VOTE] Apache Samza 1.0.0 RC3

Hi all, This is a call for a vote on a release of Apache Samza 1.0.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~pmaheshwari/samza-1.0.0-rc3/ The release candidate is signed with pgp key 6585B3D7, which can

Re: [VOTE] Apache Samza 1.0.0 RC3

We found an issue with Samza SQL integration with the new ApplicationRunners APIs. We'll cancel this vote and create a new RC. Thanks, Prateek On Tue, Oct 30, 2018 at 10:14 AM Jake Maes wrote: > > +1 binding > > Ran check-all on OSX with Gradle 2.8 > > On Mon, Oct 29, 2018 a

[CANCEL] [VOTE] Apache Samza 1.0.0 RC2

Hi all, This is the CANCEL notification for the 1.0.0 RC3. We found an issue with Samza SQL integration with the new ApplicationRunners API that we will fix in the new RC. Thanks, Prateek

Re: [VOTE] Apache Samza 1.0.0 RC2

ll and deployed locally with the test jobs. All tests passed. > > > > +1 (binding) from my end. > > > > Thanks for push the release! > > > > -Yi > > > > On Wed, Oct 24, 2018 at 8:53 AM Prateek Maheshwari > > wrote: > > > > > Hi

[CANCEL] [VOTE] Apache Samza 1.0.0 RC2

Hi all, This is the CANCEL notification for the 1.0.0 RC2. We found a test framework message serialization issue that we will fix in the new RC. Thanks, Prateek

Re: [VOTE] Apache Samza 1.0.0 RC4

Verified signatures and successfully ran check-all and integration tests. +1 (binding) from me. Thanks, Prateek On Fri, Nov 2, 2018 at 2:39 PM Boris S wrote: > > ran check-all and integration tests. All passed. > verified signatures. > +1 > > On Wed, Oct 31, 2018 at 7:15 PM Jagadish

Re: Welcome Hai Lu and Aditya Toomla as committers to Apache Samza!

Congrats Hai and Aditya, and thanks for your contributions! - Prateek > On Nov 6, 2018, at 10:40 AM, Wei Song wrote: > > Congrats Hai and Aditya! > > > > On 11/6/18, 10:20 AM, "Yi Pan" wrote: > >Hi, all, > >All official steps are completed and please join me to welcome Hai and >

Re: [DISCUSS] Mandatory migration of Samza git repo to gitbox.apache.org

Thanks for starting the discussion Pawas. I'm +1 (binding) for the migration. - Prateek On Tue, Jan 15, 2019 at 11:44 AM Pawas Chhokra wrote: > > Hi all, > > As mandated by the Apache Infrastructure Team, all git repositories must be > migrated from git-wip-us.apache.org URL to

Re: [VOTE] Migration of Samza git repo to gitbox.apache.org

+1 (binding) again - Prateek On Wed, Jan 23, 2019 at 11:50 AM Pawas Chhokra wrote: > > Hi all, > > This is a call for a vote on migrating Samza git repo to gitbox.apache.org, on > 11 AM, Jan 29, 2019. As mandated by the Apache Infrastructure Team, all git > repositories must be migrated from

Re: https://issues.apache.org/jira/browse/SAMZA-2039

For notifying others, you can leave a comment on the ticket that you're working on it. Additionally, you can assign the ticket to yourself if you have the permissions to do so. Thanks for your interest, and please let us know if you need any help. - Prateek > On Dec 17, 2018, at 8:27 PM,

Re: Beam Samza Runner - java.lang.UnsupportedOperationException: Cannot create a producer for an input system

+ Xinyu > On Jan 4, 2019, at 9:58 PM, Deshpande, Omkar > wrote: > > Hello, > > I am getting following exception while running Beam Samza Runner – > > java.lang.UnsupportedOperationException: Cannot create a producer for an > input system > > at >

Re: app.class or task.class for beam samza runner

Hi Omkar, I think it's only possible to get that exception with Samza 1.0. Can you verify that the deployment is indeed using samza 0.14.1? Thanks, Prateek On Wed, Jan 2, 2019 at 11:40 PM Deshpande, Omkar wrote: > > Hello, > > I have been able to execute my Samza-Beam application in Local

Re: Draft report to board - Jan 2019

- Beam Samza runner upgrade to use Samza 1.0 > > - Go and Python support via Beam Samza runner > > > > ## Health report: > > - Project is in healthy status with 1.0 released in Nov 2018 > > > > ## PMC changes: > > > > - Currently 15 PMC members.

Re: Running w/ multiple CPUs/container on YARN

And just to double check, you also changed the yarn.resourcemanager.scheduler.class to CapacityScheduler? On Tue, Apr 2, 2019 at 9:49 AM Prateek Maheshwari wrote: > Is it still the same message from the AM? The one that says: "Got AM > register response. The YARN RM supports contain

Re: Running w/ multiple CPUs/container on YARN

.opts=-Daws.accessKeyId= -Daws.secretKey=, but > >> since this seemed very workaround-ish, I just assumed that I would > >> eventually hit other problems using a version of Hadoop not pinned in > >> the Samza repo. If you're running 2.7.x at LinkedIn, however, I'll >

Re: Error handling

and verifying if the issue is already fixed. Thanks, Prateek On Fri, Mar 22, 2019 at 7:23 AM Tom Davis wrote: > > Prateek Maheshwari writes: > > > Hi Tom, > > > > This would depend on what your k8s container orchestration logic looks > > like. For example, in YA

Fwd: SSL with Samza 0.14.1?

Forwarding again. Original email did not show up on the OSS mailing list. -- Forwarded message - From: Deshpande, Omkar Date: Fri, Mar 22, 2019 at 5:08 PM Subject: Fwd: SSL with Samza 0.14.1? To: prateek...@gmail.com ++Prateek gmail -- *From:*

Re: Running w/ multiple CPUs/container on YARN

about it. > > Cheers, > Malcolm > > On Mon, Apr 1, 2019 at 1:51 PM Prateek Maheshwari > wrote: > > > Hi Malcolm, > > > > Yes, the AM is just reporting what the RM specified as the maximum > allowed > > request size. > > &

Re: [DISCUSS] Change Apache Samza git comments/merge email recipient to commits@samza

Thanks Xinyu. Separating discussions and commit messages sounds good to me. I'm +1, but happy to keep it as-is if others find the commit emails useful. - Prateek On Thu, Apr 4, 2019 at 3:14 PM Xinyu Liu wrote: > Hi, All, > > Our dev mailing list has been flooded with github comments/merges so

Re: SSL with Samza 0.14.1?

pache.samza > samza-core_2.12 > ${samza.version} > > > > org.apache.samza > samza-kafka_2.12 > ${samza.version} > runtime > > > > org.apache.samza > samza-kv_2.12 > ${samza.version} > runtime > > > >

Re: Running w/ multiple CPUs/container on YARN

Hi Malcolm, Just setting that configuration should be sufficient. We haven't seen this issue before. What Samza/YARN versions are you using? Can you also include the logs from where you get the error and your yarn configuration? - Prateek On Mon, Apr 1, 2019 at 2:33 AM Malcolm McFarland wrote:

Re: Running w/ multiple CPUs/container on YARN

ion-mb > > 256 > > > > > > yarn.scheduler.maximum-allocation-mb > > 14336 > > > > > > yarn.scheduler.minimum-allocation-vcores > > 1 > > > > > > yarn.scheduler.maximum-allocation-vcores

Re: SSL with Samza 0.14.1?

for the job. 3. Stacktrace where you see the SSL connect errors. Thanks, Prateek On Mon, Mar 25, 2019 at 9:47 AM Prateek Maheshwari wrote: > Forwarding again. Original email did not show up on the OSS mailing list. > > -- Forwarded message - > From: Deshpande, Omkar

Re: "send to" ordering is inconsistent

ep all of this functionality in a sink function. Does this work for you? Thanks, Prateek On Wed, Feb 27, 2019 at 2:54 PM Tom Davis wrote: > > Prateek Maheshwari writes: > > > Hi Tom, > > > > I'm assuming that the two sub-DAGs you're talking about are the two M

Re: "send to" ordering is inconsistent

Hi Tom, I'm assuming that the two sub-DAGs you're talking about are the two Map -> Send To chains acting on the "audit-report-requests" input and sending their results to the "audit-report-status" output. Although processing within each Task is in-order, the framework does not guarantee the

Re: [POSSIBLE PHISHING] Task Partition Commit Failed After Upgrade

Hi Jeremiah, The configuration you want to look for is: 'job.systemstreampartition.grouper.factory'. It should default to: 'org.apache.samza.container.grouper.stream.GroupByPartitionFactory'. Can you check if you see this value in the configuration logged by SamzaContainer during container start?

Re: Backing Kafka/Yarn/Zookeeper version for Samza 1.0.0?

://www.helixeducation.com/blog/> | Twitter > <https://twitter.com/HelixEducation> | Facebook > <https://www.facebook.com/HelixEducation> | LinkedIn > <http://www.linkedin.com/company/3609946> > > > > > > *From: *Prateek Maheshwari

Re: "send to" ordering is inconsistent

nks, Prateek! Yes, the workaround will be fine for the time being. > Thank you again! > > Prateek Maheshwari writes: > > > Hi Tom, > > > > Thanks for reporting this. I created a ticket (SAMZA-2116 > > <https://issues.apache.org/jira/browse/SAMZA-2116>

Re: [VOTE] Apache Samza 1.1.0 RC2

1. Verified checksum and signatures for the binaries. 2. Ran ./check-all.sh 3. Ran YARN and Standalone integration tests with the config patch successfully. +1(binding) from my side as well. Thanks, Prateek On Mon, Mar 18, 2019 at 2:06 PM Jagadish Venkatraman wrote: > 1. Verified check-sum

Re: Backing Kafka/Yarn/Zookeeper version for Samza 1.0.0?

Hi Jeremiah, We're in the process of upgrading Samza to use Kafka clients version 2.0, and YARN client version 2.9. This should be available in the next release (version 1.2). In the mean time, Kafka 0.11 and YARN 2.6 / 2.7 are the recommended versions. Can you clarify what you mean about hard

Re: Error handling

Hi Tom, This would depend on what your k8s container orchestration logic looks like. For example, in YARN, 'status' returns 'not running' after 'start' until all the containers requested from the AM are 'running'. We also leverage YARN to restart containers/job automatically on failures (within

Re: [DISCUSS] Samza 1.1.0 release

Daniel, let's try to include the following change in the release as well. SAMZA-2116: Make sendTo and sink operators non-terminal Other than that, +1 (binding). - Prateek On Thu, Mar 7, 2019 at 9:22 AM Xinyu Liu wrote: > +1 (binding) > > Thanks, > Xinyu > > On Thu, Mar 7, 2019 at 12:43 AM

Re: [POSSIBLE PHISHING] Task Partition Commit Failed After Upgrade

Jeremiah, were you able to resolve this issue? - Prateek On Wed, Mar 6, 2019 at 10:08 AM Prateek Maheshwari wrote: > Hi Jeremiah, > > The configuration you want to look for is: > 'job.systemstreampartition.grouper.factory'. It sh

  1   2   >