Re: Testing dev@samza.apache.org

Yep! On 1/23/15, 9:56 AM, "Yi Pan" wrote: >Yes! > >On Fri, Jan 23, 2015 at 9:49 AM, Chris Riccomini >wrote: > >> Hey all, >> >> Could you please confirm that you're seeing this? I'm trying to verify >>the >> TLP migration for: >> >> https://issues.apache.org/jira/browse/INFRA-9055 >> >> Cheer

Re: Review Request 29899: Changes for Kafka producer upgrade

-- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29899/#review69423 --- On Jan. 23, 2015, 12:42 a.m., Navina Ramesh wrote: > > ---

Re: Review Request 29899: Changes for Kafka producer upgrade

sure no other > > configs need to be changed? I already did a grep for this config and replaced them. - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2989

Re: Review Request 29899: Changes for Kafka producer upgrade

clean build - SUCCESSFUL Thanks, Navina Ramesh

Re: Review Request 29899: Changes for Kafka producer upgrade

.kafka.clients.producer.MockProducer be configured to > > satisfy the requests here? > > Navina Ramesh wrote: > I have addresssed the same question in > https://reviews.apache.org/r/29899/#comment113812 Created a follow-up task here - ht

Re: Review Request 29899: Changes for Kafka producer upgrade

negate-number.properties 4989b279a22ffd8acdb2c31ca4813f6768edc75b samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala ca25258217e5ebc44b34fbc4d69ecb28c81df618 Diff: https://reviews.apache.org/r/29899/diff/ Testing --- ./gradlew clean build - SUCCESSFUL Thanks, Navina Ramesh

Review Request 30634: Serde for Stream Tuples

/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/30634/diff/ Testing --- Thanks, Navina Ramesh

Re: Review Request 30634: Serde for Stream Tuples

://reviews.apache.org/r/30627/ . Main changes are: SqlStringSerde, SqlStringSerdeFactory, StringData and StringSchema - Navina Ramesh On Feb. 4, 2015, 7:39 p.m., Navina Ramesh wrote: > > --- > This is an automatically generated e-mail.

Re: [DISCUSS] SQL workflow

I have updates SAMZA-484 and RB is here - https://reviews.apache.org/r/30634/diff/# Navina On 2/4/15, 11:13 AM, "Yi Pan" wrote: >Just did the update w/ SAMZA-482. > >On Wed, Feb 4, 2015 at 10:31 AM, Chris Riccomini >wrote: > >> I think so. There was some RB downtime, but it just got fixed. Yi,

Re: Review Request 30634: Serde for Stream Tuples

rializers.StringSerdeFactory", the key is string and the value is an instance of StringSerdeFactory. - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30634/#review71121 --

Re: Review Request 30634: Serde for Stream Tuples

hanks, Navina Ramesh

Re: Review Request 30634: Serde for Stream Tuples

ATION settings.gradle bb07a3b84b14dcef94da1bb166eab6aa3d0026bb Diff: https://reviews.apache.org/r/30634/diff/ Testing --- Thanks, Navina Ramesh

Re: Review Request 30781: SAMZA-548

tps://reviews.apache.org/r/30781/#comment117547> What are the final job states that is returned by Yarn? samza-test/src/main/python/tests/performance_tests.py <https://reviews.apache.org/r/30781/#comment117545> What is the purpose of this if statement ? - Navina Ramesh On Feb. 9

Re: Terminology: Tumbling and sliding windows

+1 on using the right terminology. Navina On 2/17/15, 9:33 AM, "Yi Pan" wrote: >+1 on consolidating the terminology as well. Azure's definition looks good >to me. > >On Tue, Feb 17, 2015 at 9:13 AM, Chris Riccomini >wrote: > >> Hey Julian, >> >> +1 I'm not sure if we actually *are* using the r

Re: Samza commit guarantees

Hi Richard, SystemProducer is an abstraction for producers. If you are specifically looking for Kafka as a system, you should be looking into KafkaSystemProducer. Depending on the Samza version you are using, you should be able to see the code related to buffering. If you check the RunLoop class,

Re: Review Request 31405: Serde Implementation or AvroData (SAMZA-575)

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31405/#review74065 --- Looks good! - Navina Ramesh On Feb. 25, 2015, 5:54 a.m., Milinda

Review Request 32188: Disable WAL in RocksDB KV store

/ Testing --- Thanks, Navina Ramesh

Re: Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

;t you do the failedSources empty check here instead of Line 153 ? - Navina Ramesh On March 17, 2015, 9:37 a.m., Yan Fang wrote: > > --- > This is an automatically generated e-mail. To reply, visit:

Re: Review Request 32188: Disable WAL in RocksDB KV store

://reviews.apache.org/r/32188/diff/ Testing (updated) --- ./bin/check-all.sh ran successfully on MAC Thanks, Navina Ramesh

Re: Review Request 32188: Disable WAL in RocksDB KV store

reviews.apache.org/r/32188/#review76954 --- On March 18, 2015, 9:48 p.m., Navina Ramesh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32188/ > ---

Re: Review Request 32188: Disable WAL in RocksDB KV store

Diff: https://reviews.apache.org/r/32188/diff/ Testing --- ./bin/check-all.sh ran successfully on MAC Thanks, Navina Ramesh

Re: Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32155/#review77293 --- Ship it! Ship It! - Navina Ramesh On March 20, 2015, 9:40 p.m

Re: New to Samza/Yarn and having Kafka issues

Hi Ash, I just ran wikipedia-parser with your patch. Looks like you have set the message serde correctly in the configs. However, the original code still converts it into a Map for consumption in the WikipediaFeedEvent. I am seeing the following (expected): 2015-03-23 19:17:49 SamzaContainerExcept

Tags for Samza in stack overflow

Hi all, I think it is good idea to improve our presence in stackoverflow.com because: * any new user/developer mostly googles/searches stackoverflow for resolving issues before hitting us on the mailing list; This requires the user to subscribe to the mailing list -> unnecessary overhead, apart

Re: New to Samza/Yarn and having Kafka issues

reference to >Map parsedJsonObject = parse(event.getRawEvent()); >in the source java file. > > >On Mon, Mar 23, 2015 at 7:42 PM, Ash W Matheson >wrote: > >> I'm in transit right now but if memory serves me everything should be >> commented out of that meth

Re: Tags for Samza in stack overflow

ittle starting guide >available with all the steps I followed to get it working. As I believe >it is an easy process but I lack the references to do it, I hope it will >make it easier for the rest with it. > >best regards, > > Jordi > >___

Re: Review Request 33305: [SAMZA-655] Add EnvironmentConfigRewriter to samza-api.

://samza.apache.org/learn/documentation/latest/jobs/configuration-table.html) with info about this re-writer ? - Navina Ramesh On April 17, 2015, 1:12 p.m., Dan Harvey wrote: > > --- > This is an automatically generated e-mail. To reply, visit

Re: Review Request 33297: [SAMZA-654] Added ElasticsearchSystemProducer and Factory to output messages into Elasticseach indexes.

/elasticsearch/ElasticsearchSystemProducer.java <https://reviews.apache.org/r/33297/#comment131450> Is this flush a blocking call in elasticsearch? - Navina Ramesh On April 22, 2015, 5:12 p.m., Dan Harvey wrote: > > ---

Re: Review Request 33297: [SAMZA-654] Added ElasticsearchSystemProducer and Factory to output messages into Elasticseach indexes.

> On April 22, 2015, 5:49 p.m., Naveen Somasundaram wrote: > > samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/ElasticsearchSystemProducer.java, > > line 116 > > > > > > If the previous send ha

Review Request 33453: SAMZA-557 initial working draft - needs refactoring

/RocksDbKeyValueStore.scala 66c2a0dc2e38e21f951727a30f0987776ac52fe2 Diff: https://reviews.apache.org/r/33453/diff/ Testing --- Thanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

. Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state. Thanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state. Thanks, Navina Ramesh

Re: Review Request 33297: [SAMZA-654] Added ElasticsearchSystemProducer and Factory to output messages into Elasticseach indexes.

s here? - Navina Ramesh On April 23, 2015, 6:18 p.m., Dan Harvey wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

titionDirs. True. That is exactly what I meant in comment above. We can probably modify the Util.rm method to be a no-op when the file path doesn't exist. - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33453/#review81838 -

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

Question: shouldn't the base dir to be determined by container ID, not > > jobId? > > Navina Ramesh wrote: > I think containerId is not really relevant because the store partition > directories are uniquely named across the containers. The ownership of > containers

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

hanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

s there any checksum in the offset file to make sure that we don't > > read in a corrupted value? > > Navina Ramesh wrote: > No. As of now, we don't have any checksum. Do you think it is better to > add it now than later ? Yi, I created SAMZA

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

--- Tested locally using hello-samza. Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state. Thanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33453/#review82593 --- On May 5, 2015, 6:47 p.m., Navina Ramesh wrote: > >

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

store with changelog, I think it should remain as is. - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33453/#review82593 -------

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

May 6, 2015, 12:10 a.m., Yan Fang wrote: > > samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala, > > lines 110-112 > > <https://reviews.apache.org/r/33453/diff/4/?file=950516#file950516line110> > > > > personal o

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

locally using hello-samza. Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state. Thanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

ould this logic be another method, such as > > readOffsetFromFile, not "cleanBaseDir"? A little misleading. > > Navina Ramesh wrote: > I can modularize it further if it helps. Since we need to load offsets > (if available) before cleaning directories, it became a p

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

ngeLogSystemStreamPartition matter here? > > 2. prefer the storePartitionDir, because it's the dir for each > > partition/task, right? > > Navina Ramesh wrote: > 1. why does the changeLogSystemStreamPartition matter here? > >> changeLogSystemStreamParti

Re: Review Request 34009: SAMZA-552 window store implementation

sk/sql/RandomWindowOperatorTask.java <https://reviews.apache.org/r/34009/#comment134678> Shouldn't the window id be system generated? - Navina Ramesh On May 13, 2015, 5:36 p.m., Yi Pan (Data Infrastructure) wrote: > > --- > This is an automati

Re: Review Request 34009: SAMZA-552 window store implementation

> On May 13, 2015, 9:56 p.m., Navina Ramesh wrote: > > samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java, > > line 96 > > <https://reviews.apache.org/r/34009/diff/1/?file=954320#file954320line96> > > > > Shouldn'

Re: Yarn jobs in accepted state

Hi Shekar, Actually, I am not able to view even after signing in. Like Yan suggested, please try to share it elsewhere on the internet. Should G drive work? Thanks! Navina On 5/21/15, 11:21 AM, "Yan Fang" wrote: >Hi Shekar, > >This website requires signed in. Could you past to another more ope

Re: Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

names, irrespective of whether it is storename or taskname. Just my two cents. - Navina Ramesh On May 20, 2015, 10:04 p.m., Yan Fang wrote: > > --- > This is an automaticall

Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

g up YARN on 3 machines. Verified that the message is correctly written and consumed from the Coordinator Stream Thanks, Navina Ramesh

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

omment137337> Why do we need 2 instances of the TopologyBuilder here? I think this occurs because stream() and sink() method return OperatorSource type rather than a TopologyBuilder instance. How differently does the TopologyBuilder handle the OperatorSource and the OperatorS

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

ttps://reviews.apache.org/r/34746/diff/ Testing --- Used a sample job to test it locally and also, by setting up YARN on 3 machines. Verified that the message is correctly written and consumed from the Coordinator Stream Thanks, Navina Ramesh

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

> On May 29, 2015, 1:28 a.m., Navina Ramesh wrote: > > samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java, > > line 120 > > <https://reviews.apache.org/r/34500/diff/1/?file=965740#file965740line120> > > > > I thought T

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

73286#file973286line629> > > > > catch the UnknownHostException ? Mmm.. makes sense. It could be thrown. I will change it to handle the exception with a no-op. - Navina --- This is an automatically generated e-m

Re: Reprocessing old events no longer in Kafka

Hi Zach, It sounds like you are asking for a SystemConsumer for hdfs. Does SAMZA-263 match your requirements? Thanks! Navina On 5/29/15, 2:23 PM, "Zach Cox" wrote: >(continuing from previous email) in addition to not wanting to duplicate >code, say that some of the Samza jobs need to build up

Re: Reprocessing old events no longer in Kafka

messages from the other Kafka topic. Thanks! Navina On 5/29/15, 2:26 PM, "Navina Ramesh" wrote: >Hi Zach, > >It sounds like you are asking for a SystemConsumer for hdfs. Does >SAMZA-263 match your requirements? > >Thanks! >Navina > >On 5/29/15, 2:23 PM, &q

Re: Reprocessing old events no longer in Kafka

ild a composite stream, that would first read old events >from hdfs and then switch over to kafka. > >Do you know if there has been any movement on treating hdfs as a samza >stream? > >Thanks, >Zach > >On Fri, May 29, 2015 at 4:27 PM Navina Ramesh > >wrote:

Re: Reprocessing old events no longer in Kafka

t;historical events. Then standard Samza reprocessing goes through all old >events. > >I'm hesitant to pursue that though, as those topic partitions then grow >unbounded over time, which seems problematic. > >Thanks, >Zach > >On Fri, May 29, 2015 at 4:34 PM Navina Ra

Re: Reprocessing old events no longer in Kafka

oo >tstrapping > > >On Fri, May 29, 2015 at 4:53 PM Navina Ramesh > >wrote: > >> Hi Zach, >> >> I agree. It is not a good idea to keep the entire set of historical data >> in Kafka. >> The retention period in Kafka does make it trickier to synchronize w

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

Diff: https://reviews.apache.org/r/34746/diff/ Testing --- Used a sample job to test it locally and also, by setting up YARN on 3 machines. Verified that the message is correctly written and consumed from the Coordinator Stream Thanks, Navina Ramesh

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

tioned above, I will add this refactoring as a part of [SAMZA-678](https://issues.apache.org/jira/browse/SAMZA-678) - Navina ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34746/#review85855 --

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

Diff: https://reviews.apache.org/r/34746/diff/ Testing --- Used a sample job to test it locally and also, by setting up YARN on 3 machines. Verified that the message is correctly written and consumed from the Coordinator Stream Thanks, Navina Ramesh

Re: [2/2] samza git commit: Yi's TopologyBuilder RB 34500

Sorry about the confusion, Milinda. Thanks for reverting, Yi! On 6/1/15, 10:53 PM, "Yi Pan" wrote: >Hi, Milinda, > >That was an accidental mistake. I have reverted the check-in. I am still >working on that. Thanks! > >-Yi > >On Mon, Jun 1, 2015 at 9:34 PM, Milinda Pathirage >wrote: > >> Hi Nav

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

rg/apache/samza/config/KafkaConfig.scala <https://reviews.apache.org/r/35067/#comment139191> We need a log message when this value is overriden (to < INT.MAX) and that Samza cannot handle producer failures. Hence, the onus of retries is left up to the kafka producer. - Navina Ramesh

Re: Issues with certain characters in config values

Hi Tommy, I wouldn't think Util.envVarEscape() is totally unnecessary. Are you running into this issue with a version <= 0.9 ? With 0.10, I think we no longer pass all the environment variables through command line. We only pass the Coordinator system information via command line. While Util.env

Re: Missing artifact? org.apache.samza:samza-serializers_2.10:jar:0.9.0

Hi Mart, Tommy is right. We moved the serializers into samza-core. I think that documentation requires an update. Please remove the dependency section for samza_serializers. Thanks! Navina On 6/10/15, 4:36 AM, "Tommy Becker" wrote: >I'm sure others will chime in, but I think that jar was elimin

Re: Missing artifact? org.apache.samza:samza-serializers_2.10:jar:0.9.0

again. -Mart On Jun 10, 2015, at 1:06 PM, Navina Ramesh mailto:nram...@linkedin.com.INVALID>> wrote: Hi Mart, Tommy is right. We moved the serializers into samza-core. I think that documentation requires an update. Please remove the dependency section for samza_serializers. Thanks!

Re: Review Request 35071: SAMZA-572 SamzaContainer checkpoints and windows immediately on startup

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35071/#review87507 --- Ship it! lgtm! - Navina Ramesh On June 4, 2015, 2:42 p.m

Re: Review Request 34585: SAMZA-681: create a unit test harness to easily test samza tasks

ess. ? samza-test/src/test/java/org/apache/samza/test/harness/InMemoryTaskContextTest.java <https://reviews.apache.org/r/34585/#comment138585> Same here - Navina Ramesh On May 22, 2015, 5:23 a.m., Luis De Pombo wrote: > > --

Re: Review Request 35348: SAMZA-707 Remove dependency on samza-serializers in the samza.apache.org/startup/download page

348/#comment140198> Actually, this dependency is required if the samza version used is < 0.9. So, can you add a comment below the "Artifacts" section mentioning this additional dependency for samza < 0.9 ? - Navina Ramesh On June 11, 2015, 8:49 a.m., Aleks

Re: Combine two web servlets in the AM?

+1 for adding to the documentation. I didn¹t know about this difference. This information is perhaps part of the JobCoordinator/CoordinatorStream documentation - SAMZA-614? On 6/15/15, 11:27 AM, "Yan Fang" wrote: >All right. Sorry for bothering. I figured it >out. ApplicationMasterRestServlet is

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35067/#review87952 --- Ship it! Ship It! - Navina Ramesh On June 15, 2015, 1:42 p.m

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

> On June 8, 2015, 2:29 a.m., Navina Ramesh wrote: > > samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala, line > > 182 > > <https://reviews.apache.org/r/35067/diff/1/?file=979072#file979072line182> > > > > This warning is v

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

verify that these properties get overridden correctly ? Sorry about this afterthought request. - Navina Ramesh On June 15, 2015, 1:42 p.m., Robert Zuljevic wrote: > > --- > This is an automatically generated e-mail. To reply, visit

Re: [DISCUSS] Samza 0.9.1 release

+1 for the release! On 6/16/15, 11:03 AM, "Yi Pan" wrote: >+1 Agreed. > >Thanks! > >On Tue, Jun 16, 2015 at 10:15 AM, Yan Fang wrote: > >> Agreed on this. >> >> Thanks, >> >> Fang, Yan >> yanfang...@gmail.com >> >> On Tue, Jun 16, 2015 at 10:14 AM, Guozhang Wang >> wrote: >> >> > Hi all, >> >

Re: Review Request 35241: refactoring the code for coordinator stream writer

coordinator/stream/MockCoordinatorStreamSystemFactory.java (line 60) <https://reviews.apache.org/r/35241/#comment140583> nit: spacing - Navina Ramesh On June 9, 2015, 11:53 p.m., Shadi A. Noghabi wrote: > > --- > This is an automatically generated

Re: Review Request 35348: SAMZA-707 Remove dependency on samza-serializers in the samza.apache.org/startup/download page

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35348/#review88287 --- Ship it! Ship It! - Navina Ramesh On June 17, 2015, 8:15 p.m

Re: Review Request 34974: SAMZA-676: implement broadcast stream

(line 98) <https://reviews.apache.org/r/34974/#comment140973> nit: typo 'resiter' - Navina Ramesh On June 16, 2015, 9:23 p.m., Yan Fang wrote: > > --- > This is a

Re: Deploying samza job to a remote yarn resource manager

Hi George, Looks like there is a formatting issue on the samza website. The environment variable name should be HADOOP_YARN_HOME. In your case, it should point to hadoop home: $HOME/hadoop-2.6.0/ Please let me know if it works. Cheers! Navina On 6/22/15, 2:11 PM, "George Li" wrote: >Hi, > >Ac

Re: Deploying samza job to a remote yarn resource manager

me host, but how do I pass the remote >resource manager address? Maybe I should include HADOOP_YARN_HOME and >HADOOP_CONF_DIR as classpath? > >George > > > > >From: Navina Ramesh >To: "dev@samza.apache.org" , >Date: 22/06/2015 05:57 PM >Subject:

Re: [VOTE] Apache Samza 0.9.1 RC1

+1 Verified MD5 & asc signatures. Local build & check-all.sh passed. Thanks! Navina On 6/30/15, 7:13 PM, "Chris Riccomini" wrote: >+1 > >Verified MD5, and asc signature. Build locally, and all tests pass. > >Cheers, >Chris > >On Tue, Jun 30, 2015 at 1:26 PM, Milinda Pathirage >wrote: > >> +1

Review Request 36224: SAMZA-728: Samza job fails due to null pointer in JobCoordinator refreshJobModel

://reviews.apache.org/r/36224/diff/ Testing --- Tested with a simple job deployed a multi-node yarn cluster with kafka/zookeeper deployments from hello-samza Thanks, Navina Ramesh

Re: Review Request 36224: SAMZA-728: Samza job fails due to null pointer in JobCoordinator refreshJobModel

8ee034a642a13af1b0fdb4ebbb3b2592bb8e2be1 Diff: https://reviews.apache.org/r/36224/diff/ Testing (updated) --- Tested with a simple job deployed a multi-node yarn cluster with kafka/zookeeper deployments from hello-samza ./bin/check-all.sh passed Thanks, Navina Ramesh

Re: Review Request 36224: SAMZA-728: Samza job fails due to null pointer in JobCoordinator refreshJobModel

/samza/coordinator/JobCoordinator.scala 8ee034a642a13af1b0fdb4ebbb3b2592bb8e2be1 Diff: https://reviews.apache.org/r/36224/diff/ Testing --- Tested with a simple job deployed a multi-node yarn cluster with kafka/zookeeper deployments from hello-samza Thanks, Navina Ramesh

Re: Review Request 35241: refactoring the code for coordinator stream writer

ews.apache.org/r/35241/#comment145212> Is it possible to simplify Lines 59-65? Looks like you are doing a null check in Line 61 and then, creating another handle for the field in Line 64. - Navina Ramesh On June 29, 2015, 11:08 p.m., Shadi A. Noghabi wrote: > > ---

Re: hello-samza git repo moved

Hi Guozhang, I tried sometime back. It works. nramesh-mn1:Projects nramesh$ git clone https://github.com/apache/samza-hello-samza open-source-hello-samza Cloning into 'open-source-hello-samza'... remote: Counting objects: 510, done. remote: Total 510 (delta 0), reused 0 (delta 0), pack-reused 510

Re: hello-samza git repo moved

apache/samza-hello-samza link works, as showed in >> my >> email as well. >> >> But the git://git.apache.org/samza-hello-samza.git does not work, which >>is >> stated in the wiki page >>(http://samza.apache.org/startup/hello-samza/0.9/ >> ). >> >> G

Re: Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

(line 42) <https://reviews.apache.org/r/33419/#comment145435> nit: SYSTEM_FACTORY can be defined in this class rather than from the scala class. Perhaps a future work of cleanup can do this as well. - Navina Ramesh On June 21, 2015, 6:1

Re: Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33419/#review91865 --- Ship it! Ship It! - Navina Ramesh On July 16, 2015, 12:56 a.m

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

687> Code here looks kind of arbitary here. Looks like you want to write the jobcoordinator url to the coordinator stream. You should perhaps move this to the init method in SamzaAppMasterService?? This way you don't hav

Re: Review Request 36769: SAMZA-742 Add a Contribution section in README of samza and hello-samza

tps://reviews.apache.org/r/36769/#comment147196> You can emphasize with "**" as in : **Samza git repository does not support git pull request** - Navina Ramesh On July 24, 2015, 8:51 a.m., Aleksandar Pejakovic wrote: > > --

Re: Review Request 36770: SAMZA-742 Add a Contribution section in README of samza and hello-samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36770/#review92936 --- Ship it! Ship It! - Navina Ramesh On July 24, 2015, 8:57 a.m

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

e/src/main/scala/org/apache/samza/container/SamzaContainer.scala (line 627) <https://reviews.apache.org/r/36545/#comment147198> LocalityManager maintains "container to host"-level mapping. Not a task to host mapping. Please change this back to containerId. - Navina Ramesh On

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

samza-hello-samza. I believe ElasticsearchProducer is going to be part of the 0.10 release in Samza. So, this patch should be against the "latest" branch in hello-samza. This diff looks like it is against the master branch. Can you please make the changes against latest branch ? - Navin

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

to setup and run the example Samza job that uses ElasticSearchProducer. - Navina Ramesh On July 24, 2015, 8:31 a.m., Stuart Davidson wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://review

Re: kafka producer failed

Hi Selina, Looks like the attachment got filtered. Can you past the config in the email or at pastebin? Thanks! Navina On Fri, Jul 24, 2015 at 1:18 PM, Job-Selina Wu wrote: > Hi, Yi: > > I am wondering if the problem can be fixed by the parameter " > max.message.size" at kafka.producer.P

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

> On July 24, 2015, 6:19 p.m., Navina Ramesh wrote: > > It will also be nice to have a tutorial added to the website on how to > > setup and run the example Samza job that uses ElasticSearchProducer. > > Stuart Davidson wrote: > Happy to write something up. Do

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

> On July 21, 2015, 9:53 p.m., Navina Ramesh wrote: > > Ideally, the job coordinator should act as the config manager. Since the > > chain of control is still from AppMaster to JobCoordinator (instead of the > > other way around), the only reasonable way to get this worki

Re: [DISCUSS] Release 0.10.0

+1 for StreamAppender bug fix as a mandatory in 0.10. @Yan: will review SAMZA-676 today and test it out locally. Thanks! Navina On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman wrote: > I can take care of SAMZA-340, SAMZA-683 and will follow up with Luis for > SAMZA-401,2,3,4 > > On Wed, Jul 29

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

> On July 24, 2015, 6:01 p.m., Navina Ramesh wrote: > > Thanks for picking this up! It feels good to look at a refactored code. > > > > One suggestion: Please run all the intergration test (including the zopkio > > tests) before checking in this patch. I don&#

Re: Required vs. optional methods for KeyValueStore

Hi Ken, We use putAll(list) when restoring from changelog. So, unless you don't want your store to have support for changelog, the implementation is required. I only have a high-level overview of what Solr is. Perhaps, others on the mailing list have experience with Solr and can provide more usef

  1   2   3   4   5   6   >