Re: Testing dev@samza.apache.org

2015-01-23 Thread Navina Ramesh
Yep! On 1/23/15, 9:56 AM, "Yi Pan" wrote: >Yes! > >On Fri, Jan 23, 2015 at 9:49 AM, Chris Riccomini >wrote: > >> Hey all, >> >> Could you please confirm that you're seeing this? I'm trying to verify >>the >> TLP migration for: >> >> https://issues.apache.org/jira/browse/INFRA-9055 >> >> Cheer

Re: Review Request 29899: Changes for Kafka producer upgrade

2015-01-23 Thread Navina Ramesh
-- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29899/#review69423 --- On Jan. 23, 2015, 12:42 a.m., Navina Ramesh wrote: > > ---

Re: Review Request 29899: Changes for Kafka producer upgrade

2015-01-23 Thread Navina Ramesh
sure no other > > configs need to be changed? I already did a grep for this config and replaced them. - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2989

Re: Review Request 29899: Changes for Kafka producer upgrade

2015-01-23 Thread Navina Ramesh
clean build - SUCCESSFUL Thanks, Navina Ramesh

Re: Review Request 29899: Changes for Kafka producer upgrade

2015-01-23 Thread Navina Ramesh
.kafka.clients.producer.MockProducer be configured to > > satisfy the requests here? > > Navina Ramesh wrote: > I have addresssed the same question in > https://reviews.apache.org/r/29899/#comment113812 Created a follow-up task here - ht

Re: Review Request 29899: Changes for Kafka producer upgrade

2015-01-27 Thread Navina Ramesh
negate-number.properties 4989b279a22ffd8acdb2c31ca4813f6768edc75b samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala ca25258217e5ebc44b34fbc4d69ecb28c81df618 Diff: https://reviews.apache.org/r/29899/diff/ Testing --- ./gradlew clean build - SUCCESSFUL Thanks, Navina Ramesh

Review Request 30634: Serde for Stream Tuples

2015-02-04 Thread Navina Ramesh
/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/30634/diff/ Testing --- Thanks, Navina Ramesh

Re: Review Request 30634: Serde for Stream Tuples

2015-02-04 Thread Navina Ramesh
://reviews.apache.org/r/30627/ . Main changes are: SqlStringSerde, SqlStringSerdeFactory, StringData and StringSchema - Navina Ramesh On Feb. 4, 2015, 7:39 p.m., Navina Ramesh wrote: > > --- > This is an automatically generated e-mail.

Re: [DISCUSS] SQL workflow

2015-02-04 Thread Navina Ramesh
I have updates SAMZA-484 and RB is here - https://reviews.apache.org/r/30634/diff/# Navina On 2/4/15, 11:13 AM, "Yi Pan" wrote: >Just did the update w/ SAMZA-482. > >On Wed, Feb 4, 2015 at 10:31 AM, Chris Riccomini >wrote: > >> I think so. There was some RB downtime, but it just got fixed. Yi,

Re: Review Request 30634: Serde for Stream Tuples

2015-02-04 Thread Navina Ramesh
rializers.StringSerdeFactory", the key is string and the value is an instance of StringSerdeFactory. - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30634/#review71121 --

Re: Review Request 30634: Serde for Stream Tuples

2015-02-05 Thread Navina Ramesh
hanks, Navina Ramesh

Re: Review Request 30634: Serde for Stream Tuples

2015-02-05 Thread Navina Ramesh
ATION settings.gradle bb07a3b84b14dcef94da1bb166eab6aa3d0026bb Diff: https://reviews.apache.org/r/30634/diff/ Testing --- Thanks, Navina Ramesh

Re: Review Request 30781: SAMZA-548

2015-02-09 Thread Navina Ramesh
tps://reviews.apache.org/r/30781/#comment117547> What are the final job states that is returned by Yarn? samza-test/src/main/python/tests/performance_tests.py <https://reviews.apache.org/r/30781/#comment117545> What is the purpose of this if statement ? - Navina Ramesh On Feb. 9

Re: Terminology: Tumbling and sliding windows

2015-02-17 Thread Navina Ramesh
+1 on using the right terminology. Navina On 2/17/15, 9:33 AM, "Yi Pan" wrote: >+1 on consolidating the terminology as well. Azure's definition looks good >to me. > >On Tue, Feb 17, 2015 at 9:13 AM, Chris Riccomini >wrote: > >> Hey Julian, >> >> +1 I'm not sure if we actually *are* using the r

Re: Samza commit guarantees

2015-02-19 Thread Navina Ramesh
Hi Richard, SystemProducer is an abstraction for producers. If you are specifically looking for Kafka as a system, you should be looking into KafkaSystemProducer. Depending on the Samza version you are using, you should be able to see the code related to buffering. If you check the RunLoop class,

Re: Review Request 31405: Serde Implementation or AvroData (SAMZA-575)

2015-02-25 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31405/#review74065 --- Looks good! - Navina Ramesh On Feb. 25, 2015, 5:54 a.m., Milinda

Review Request 32188: Disable WAL in RocksDB KV store

2015-03-17 Thread Navina Ramesh
/ Testing --- Thanks, Navina Ramesh

Re: Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

2015-03-18 Thread Navina Ramesh
;t you do the failedSources empty check here instead of Line 153 ? - Navina Ramesh On March 17, 2015, 9:37 a.m., Yan Fang wrote: > > --- > This is an automatically generated e-mail. To reply, visit:

Re: Review Request 32188: Disable WAL in RocksDB KV store

2015-03-18 Thread Navina Ramesh
://reviews.apache.org/r/32188/diff/ Testing (updated) --- ./bin/check-all.sh ran successfully on MAC Thanks, Navina Ramesh

Re: Review Request 32188: Disable WAL in RocksDB KV store

2015-03-18 Thread Navina Ramesh
reviews.apache.org/r/32188/#review76954 --- On March 18, 2015, 9:48 p.m., Navina Ramesh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32188/ > ---

Re: Review Request 32188: Disable WAL in RocksDB KV store

2015-03-18 Thread Navina Ramesh
Diff: https://reviews.apache.org/r/32188/diff/ Testing --- ./bin/check-all.sh ran successfully on MAC Thanks, Navina Ramesh

Re: Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

2015-03-20 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32155/#review77293 --- Ship it! Ship It! - Navina Ramesh On March 20, 2015, 9:40 p.m

Re: New to Samza/Yarn and having Kafka issues

2015-03-23 Thread Navina Ramesh
Hi Ash, I just ran wikipedia-parser with your patch. Looks like you have set the message serde correctly in the configs. However, the original code still converts it into a Map for consumption in the WikipediaFeedEvent. I am seeing the following (expected): 2015-03-23 19:17:49 SamzaContainerExcept

Tags for Samza in stack overflow

2015-03-23 Thread Navina Ramesh
Hi all, I think it is good idea to improve our presence in stackoverflow.com because: * any new user/developer mostly googles/searches stackoverflow for resolving issues before hitting us on the mailing list; This requires the user to subscribe to the mailing list -> unnecessary overhead, apart

Re: New to Samza/Yarn and having Kafka issues

2015-03-23 Thread Navina Ramesh
reference to >Map parsedJsonObject = parse(event.getRawEvent()); >in the source java file. > > >On Mon, Mar 23, 2015 at 7:42 PM, Ash W Matheson >wrote: > >> I'm in transit right now but if memory serves me everything should be >> commented out of that meth

Re: Tags for Samza in stack overflow

2015-03-24 Thread Navina Ramesh
ittle starting guide >available with all the steps I followed to get it working. As I believe >it is an easy process but I lack the references to do it, I hope it will >make it easier for the rest with it. > >best regards, > > Jordi > >___

Re: Review Request 33305: [SAMZA-655] Add EnvironmentConfigRewriter to samza-api.

2015-04-20 Thread Navina Ramesh
://samza.apache.org/learn/documentation/latest/jobs/configuration-table.html) with info about this re-writer ? - Navina Ramesh On April 17, 2015, 1:12 p.m., Dan Harvey wrote: > > --- > This is an automatically generated e-mail. To reply, visit

Re: Review Request 33297: [SAMZA-654] Added ElasticsearchSystemProducer and Factory to output messages into Elasticseach indexes.

2015-04-22 Thread Navina Ramesh
/elasticsearch/ElasticsearchSystemProducer.java <https://reviews.apache.org/r/33297/#comment131450> Is this flush a blocking call in elasticsearch? - Navina Ramesh On April 22, 2015, 5:12 p.m., Dan Harvey wrote: > > ---

Re: Review Request 33297: [SAMZA-654] Added ElasticsearchSystemProducer and Factory to output messages into Elasticseach indexes.

2015-04-22 Thread Navina Ramesh
> On April 22, 2015, 5:49 p.m., Naveen Somasundaram wrote: > > samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/ElasticsearchSystemProducer.java, > > line 116 > > > > > > If the previous send ha

Review Request 33453: SAMZA-557 initial working draft - needs refactoring

2015-04-22 Thread Navina Ramesh
/RocksDbKeyValueStore.scala 66c2a0dc2e38e21f951727a30f0987776ac52fe2 Diff: https://reviews.apache.org/r/33453/diff/ Testing --- Thanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-04-22 Thread Navina Ramesh
. Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state. Thanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-04-22 Thread Navina Ramesh
environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state. Thanks, Navina Ramesh

Re: Review Request 33297: [SAMZA-654] Added ElasticsearchSystemProducer and Factory to output messages into Elasticseach indexes.

2015-04-23 Thread Navina Ramesh
s here? - Navina Ramesh On April 23, 2015, 6:18 p.m., Dan Harvey wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-04-28 Thread Navina Ramesh
titionDirs. True. That is exactly what I meant in comment above. We can probably modify the Util.rm method to be a no-op when the file path doesn't exist. - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33453/#review81838 -

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-04-29 Thread Navina Ramesh
Question: shouldn't the base dir to be determined by container ID, not > > jobId? > > Navina Ramesh wrote: > I think containerId is not really relevant because the store partition > directories are uniquely named across the containers. The ownership of > containers

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-04 Thread Navina Ramesh
hanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-04 Thread Navina Ramesh
s there any checksum in the offset file to make sure that we don't > > read in a corrupted value? > > Navina Ramesh wrote: > No. As of now, we don't have any checksum. Do you think it is better to > add it now than later ? Yi, I created SAMZA

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-05 Thread Navina Ramesh
--- Tested locally using hello-samza. Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state. Thanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-05 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33453/#review82593 --- On May 5, 2015, 6:47 p.m., Navina Ramesh wrote: > >

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-05 Thread Navina Ramesh
store with changelog, I think it should remain as is. - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33453/#review82593 -------

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-05 Thread Navina Ramesh
May 6, 2015, 12:10 a.m., Yan Fang wrote: > > samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala, > > lines 110-112 > > <https://reviews.apache.org/r/33453/diff/4/?file=950516#file950516line110> > > > > personal o

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-05 Thread Navina Ramesh
locally using hello-samza. Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state. Thanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-05 Thread Navina Ramesh
ould this logic be another method, such as > > readOffsetFromFile, not "cleanBaseDir"? A little misleading. > > Navina Ramesh wrote: > I can modularize it further if it helps. Since we need to load offsets > (if available) before cleaning directories, it became a p

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-06 Thread Navina Ramesh
ngeLogSystemStreamPartition matter here? > > 2. prefer the storePartitionDir, because it's the dir for each > > partition/task, right? > > Navina Ramesh wrote: > 1. why does the changeLogSystemStreamPartition matter here? > >> changeLogSystemStreamParti

Re: Review Request 34009: SAMZA-552 window store implementation

2015-05-13 Thread Navina Ramesh
sk/sql/RandomWindowOperatorTask.java <https://reviews.apache.org/r/34009/#comment134678> Shouldn't the window id be system generated? - Navina Ramesh On May 13, 2015, 5:36 p.m., Yi Pan (Data Infrastructure) wrote: > > --- > This is an automati

Re: Review Request 34009: SAMZA-552 window store implementation

2015-05-13 Thread Navina Ramesh
> On May 13, 2015, 9:56 p.m., Navina Ramesh wrote: > > samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java, > > line 96 > > <https://reviews.apache.org/r/34009/diff/1/?file=954320#file954320line96> > > > > Shouldn'

Re: Yarn jobs in accepted state

2015-05-21 Thread Navina Ramesh
Hi Shekar, Actually, I am not able to view even after signing in. Like Yan suggested, please try to share it elsewhere on the internet. Should G drive work? Thanks! Navina On 5/21/15, 11:21 AM, "Yan Fang" wrote: >Hi Shekar, > >This website requires signed in. Could you past to another more ope

Re: Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

2015-05-21 Thread Navina Ramesh
names, irrespective of whether it is storename or taskname. Just my two cents. - Navina Ramesh On May 20, 2015, 10:04 p.m., Yan Fang wrote: > > --- > This is an automaticall

Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

2015-05-27 Thread Navina Ramesh
g up YARN on 3 machines. Verified that the message is correctly written and consumed from the Coordinator Stream Thanks, Navina Ramesh

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-05-28 Thread Navina Ramesh
omment137337> Why do we need 2 instances of the TopologyBuilder here? I think this occurs because stream() and sink() method return OperatorSource type rather than a TopologyBuilder instance. How differently does the TopologyBuilder handle the OperatorSource and the OperatorS

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

2015-05-29 Thread Navina Ramesh
ttps://reviews.apache.org/r/34746/diff/ Testing --- Used a sample job to test it locally and also, by setting up YARN on 3 machines. Verified that the message is correctly written and consumed from the Coordinator Stream Thanks, Navina Ramesh

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-05-29 Thread Navina Ramesh
> On May 29, 2015, 1:28 a.m., Navina Ramesh wrote: > > samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java, > > line 120 > > <https://reviews.apache.org/r/34500/diff/1/?file=965740#file965740line120> > > > > I thought T

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

2015-05-29 Thread Navina Ramesh
73286#file973286line629> > > > > catch the UnknownHostException ? Mmm.. makes sense. It could be thrown. I will change it to handle the exception with a no-op. - Navina --- This is an automatically generated e-m

Re: Reprocessing old events no longer in Kafka

2015-05-29 Thread Navina Ramesh
Hi Zach, It sounds like you are asking for a SystemConsumer for hdfs. Does SAMZA-263 match your requirements? Thanks! Navina On 5/29/15, 2:23 PM, "Zach Cox" wrote: >(continuing from previous email) in addition to not wanting to duplicate >code, say that some of the Samza jobs need to build up

Re: Reprocessing old events no longer in Kafka

2015-05-29 Thread Navina Ramesh
messages from the other Kafka topic. Thanks! Navina On 5/29/15, 2:26 PM, "Navina Ramesh" wrote: >Hi Zach, > >It sounds like you are asking for a SystemConsumer for hdfs. Does >SAMZA-263 match your requirements? > >Thanks! >Navina > >On 5/29/15, 2:23 PM, &q

Re: Reprocessing old events no longer in Kafka

2015-05-29 Thread Navina Ramesh
ild a composite stream, that would first read old events >from hdfs and then switch over to kafka. > >Do you know if there has been any movement on treating hdfs as a samza >stream? > >Thanks, >Zach > >On Fri, May 29, 2015 at 4:27 PM Navina Ramesh > >wrote:

Re: Reprocessing old events no longer in Kafka

2015-05-29 Thread Navina Ramesh
t;historical events. Then standard Samza reprocessing goes through all old >events. > >I'm hesitant to pursue that though, as those topic partitions then grow >unbounded over time, which seems problematic. > >Thanks, >Zach > >On Fri, May 29, 2015 at 4:34 PM Navina Ra

Re: Reprocessing old events no longer in Kafka

2015-05-29 Thread Navina Ramesh
oo >tstrapping > > >On Fri, May 29, 2015 at 4:53 PM Navina Ramesh > >wrote: > >> Hi Zach, >> >> I agree. It is not a good idea to keep the entire set of historical data >> in Kafka. >> The retention period in Kafka does make it trickier to synchronize w

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

2015-05-29 Thread Navina Ramesh
Diff: https://reviews.apache.org/r/34746/diff/ Testing --- Used a sample job to test it locally and also, by setting up YARN on 3 machines. Verified that the message is correctly written and consumed from the Coordinator Stream Thanks, Navina Ramesh

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

2015-05-30 Thread Navina Ramesh
tioned above, I will add this refactoring as a part of [SAMZA-678](https://issues.apache.org/jira/browse/SAMZA-678) - Navina ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34746/#review85855 --

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

2015-05-30 Thread Navina Ramesh
Diff: https://reviews.apache.org/r/34746/diff/ Testing --- Used a sample job to test it locally and also, by setting up YARN on 3 machines. Verified that the message is correctly written and consumed from the Coordinator Stream Thanks, Navina Ramesh

Re: [2/2] samza git commit: Yi's TopologyBuilder RB 34500

2015-06-01 Thread Navina Ramesh
Sorry about the confusion, Milinda. Thanks for reverting, Yi! On 6/1/15, 10:53 PM, "Yi Pan" wrote: >Hi, Milinda, > >That was an accidental mistake. I have reverted the check-in. I am still >working on that. Thanks! > >-Yi > >On Mon, Jun 1, 2015 at 9:34 PM, Milinda Pathirage >wrote: > >> Hi Nav

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

2015-06-07 Thread Navina Ramesh
rg/apache/samza/config/KafkaConfig.scala <https://reviews.apache.org/r/35067/#comment139191> We need a log message when this value is overriden (to < INT.MAX) and that Samza cannot handle producer failures. Hence, the onus of retries is left up to the kafka producer. - Navina Ramesh

Re: Issues with certain characters in config values

2015-06-09 Thread Navina Ramesh
Hi Tommy, I wouldn't think Util.envVarEscape() is totally unnecessary. Are you running into this issue with a version <= 0.9 ? With 0.10, I think we no longer pass all the environment variables through command line. We only pass the Coordinator system information via command line. While Util.env

Re: Missing artifact? org.apache.samza:samza-serializers_2.10:jar:0.9.0

2015-06-10 Thread Navina Ramesh
Hi Mart, Tommy is right. We moved the serializers into samza-core. I think that documentation requires an update. Please remove the dependency section for samza_serializers. Thanks! Navina On 6/10/15, 4:36 AM, "Tommy Becker" wrote: >I'm sure others will chime in, but I think that jar was elimin

Re: Missing artifact? org.apache.samza:samza-serializers_2.10:jar:0.9.0

2015-06-10 Thread Navina Ramesh
again. -Mart On Jun 10, 2015, at 1:06 PM, Navina Ramesh mailto:nram...@linkedin.com.INVALID>> wrote: Hi Mart, Tommy is right. We moved the serializers into samza-core. I think that documentation requires an update. Please remove the dependency section for samza_serializers. Thanks!

Re: Review Request 35071: SAMZA-572 SamzaContainer checkpoints and windows immediately on startup

2015-06-10 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35071/#review87507 --- Ship it! lgtm! - Navina Ramesh On June 4, 2015, 2:42 p.m

Re: Review Request 34585: SAMZA-681: create a unit test harness to easily test samza tasks

2015-06-10 Thread Navina Ramesh
ess. ? samza-test/src/test/java/org/apache/samza/test/harness/InMemoryTaskContextTest.java <https://reviews.apache.org/r/34585/#comment138585> Same here - Navina Ramesh On May 22, 2015, 5:23 a.m., Luis De Pombo wrote: > > --

Re: Review Request 35348: SAMZA-707 Remove dependency on samza-serializers in the samza.apache.org/startup/download page

2015-06-12 Thread Navina Ramesh
348/#comment140198> Actually, this dependency is required if the samza version used is < 0.9. So, can you add a comment below the "Artifacts" section mentioning this additional dependency for samza < 0.9 ? - Navina Ramesh On June 11, 2015, 8:49 a.m., Aleks

Re: Combine two web servlets in the AM?

2015-06-15 Thread Navina Ramesh
+1 for adding to the documentation. I didnĀ¹t know about this difference. This information is perhaps part of the JobCoordinator/CoordinatorStream documentation - SAMZA-614? On 6/15/15, 11:27 AM, "Yan Fang" wrote: >All right. Sorry for bothering. I figured it >out. ApplicationMasterRestServlet is

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

2015-06-15 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35067/#review87952 --- Ship it! Ship It! - Navina Ramesh On June 15, 2015, 1:42 p.m

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

2015-06-15 Thread Navina Ramesh
> On June 8, 2015, 2:29 a.m., Navina Ramesh wrote: > > samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala, line > > 182 > > <https://reviews.apache.org/r/35067/diff/1/?file=979072#file979072line182> > > > > This warning is v

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

2015-06-15 Thread Navina Ramesh
verify that these properties get overridden correctly ? Sorry about this afterthought request. - Navina Ramesh On June 15, 2015, 1:42 p.m., Robert Zuljevic wrote: > > --- > This is an automatically generated e-mail. To reply, visit

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Navina Ramesh
+1 for the release! On 6/16/15, 11:03 AM, "Yi Pan" wrote: >+1 Agreed. > >Thanks! > >On Tue, Jun 16, 2015 at 10:15 AM, Yan Fang wrote: > >> Agreed on this. >> >> Thanks, >> >> Fang, Yan >> yanfang...@gmail.com >> >> On Tue, Jun 16, 2015 at 10:14 AM, Guozhang Wang >> wrote: >> >> > Hi all, >> >

Re: Review Request 35241: refactoring the code for coordinator stream writer

2015-06-16 Thread Navina Ramesh
coordinator/stream/MockCoordinatorStreamSystemFactory.java (line 60) <https://reviews.apache.org/r/35241/#comment140583> nit: spacing - Navina Ramesh On June 9, 2015, 11:53 p.m., Shadi A. Noghabi wrote: > > --- > This is an automatically generated

Re: Review Request 35348: SAMZA-707 Remove dependency on samza-serializers in the samza.apache.org/startup/download page

2015-06-17 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35348/#review88287 --- Ship it! Ship It! - Navina Ramesh On June 17, 2015, 8:15 p.m

Re: Review Request 34974: SAMZA-676: implement broadcast stream

2015-06-18 Thread Navina Ramesh
(line 98) <https://reviews.apache.org/r/34974/#comment140973> nit: typo 'resiter' - Navina Ramesh On June 16, 2015, 9:23 p.m., Yan Fang wrote: > > --- > This is a

Re: Deploying samza job to a remote yarn resource manager

2015-06-22 Thread Navina Ramesh
Hi George, Looks like there is a formatting issue on the samza website. The environment variable name should be HADOOP_YARN_HOME. In your case, it should point to hadoop home: $HOME/hadoop-2.6.0/ Please let me know if it works. Cheers! Navina On 6/22/15, 2:11 PM, "George Li" wrote: >Hi, > >Ac

Re: Deploying samza job to a remote yarn resource manager

2015-06-23 Thread Navina Ramesh
me host, but how do I pass the remote >resource manager address? Maybe I should include HADOOP_YARN_HOME and >HADOOP_CONF_DIR as classpath? > >George > > > > >From: Navina Ramesh >To: "dev@samza.apache.org" , >Date: 22/06/2015 05:57 PM >Subject:

Re: [VOTE] Apache Samza 0.9.1 RC1

2015-07-01 Thread Navina Ramesh
+1 Verified MD5 & asc signatures. Local build & check-all.sh passed. Thanks! Navina On 6/30/15, 7:13 PM, "Chris Riccomini" wrote: >+1 > >Verified MD5, and asc signature. Build locally, and all tests pass. > >Cheers, >Chris > >On Tue, Jun 30, 2015 at 1:26 PM, Milinda Pathirage >wrote: > >> +1

Review Request 36224: SAMZA-728: Samza job fails due to null pointer in JobCoordinator refreshJobModel

2015-07-06 Thread Navina Ramesh
://reviews.apache.org/r/36224/diff/ Testing --- Tested with a simple job deployed a multi-node yarn cluster with kafka/zookeeper deployments from hello-samza Thanks, Navina Ramesh

Re: Review Request 36224: SAMZA-728: Samza job fails due to null pointer in JobCoordinator refreshJobModel

2015-07-13 Thread Navina Ramesh
8ee034a642a13af1b0fdb4ebbb3b2592bb8e2be1 Diff: https://reviews.apache.org/r/36224/diff/ Testing (updated) --- Tested with a simple job deployed a multi-node yarn cluster with kafka/zookeeper deployments from hello-samza ./bin/check-all.sh passed Thanks, Navina Ramesh

Re: Review Request 36224: SAMZA-728: Samza job fails due to null pointer in JobCoordinator refreshJobModel

2015-07-13 Thread Navina Ramesh
/samza/coordinator/JobCoordinator.scala 8ee034a642a13af1b0fdb4ebbb3b2592bb8e2be1 Diff: https://reviews.apache.org/r/36224/diff/ Testing --- Tested with a simple job deployed a multi-node yarn cluster with kafka/zookeeper deployments from hello-samza Thanks, Navina Ramesh

Re: Review Request 35241: refactoring the code for coordinator stream writer

2015-07-14 Thread Navina Ramesh
ews.apache.org/r/35241/#comment145212> Is it possible to simplify Lines 59-65? Looks like you are doing a null check in Line 61 and then, creating another handle for the field in Line 64. - Navina Ramesh On June 29, 2015, 11:08 p.m., Shadi A. Noghabi wrote: > > ---

Re: hello-samza git repo moved

2015-07-14 Thread Navina Ramesh
Hi Guozhang, I tried sometime back. It works. nramesh-mn1:Projects nramesh$ git clone https://github.com/apache/samza-hello-samza open-source-hello-samza Cloning into 'open-source-hello-samza'... remote: Counting objects: 510, done. remote: Total 510 (delta 0), reused 0 (delta 0), pack-reused 510

Re: hello-samza git repo moved

2015-07-14 Thread Navina Ramesh
apache/samza-hello-samza link works, as showed in >> my >> email as well. >> >> But the git://git.apache.org/samza-hello-samza.git does not work, which >>is >> stated in the wiki page >>(http://samza.apache.org/startup/hello-samza/0.9/ >> ). >> >> G

Re: Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

2015-07-15 Thread Navina Ramesh
(line 42) <https://reviews.apache.org/r/33419/#comment145435> nit: SYSTEM_FACTORY can be defined in this class rather than from the scala class. Perhaps a future work of cleanup can do this as well. - Navina Ramesh On June 21, 2015, 6:1

Re: Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

2015-07-15 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33419/#review91865 --- Ship it! Ship It! - Navina Ramesh On July 16, 2015, 12:56 a.m

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

2015-07-21 Thread Navina Ramesh
687> Code here looks kind of arbitary here. Looks like you want to write the jobcoordinator url to the coordinator stream. You should perhaps move this to the init method in SamzaAppMasterService?? This way you don't hav

Re: Review Request 36769: SAMZA-742 Add a Contribution section in README of samza and hello-samza

2015-07-24 Thread Navina Ramesh
tps://reviews.apache.org/r/36769/#comment147196> You can emphasize with "**" as in : **Samza git repository does not support git pull request** - Navina Ramesh On July 24, 2015, 8:51 a.m., Aleksandar Pejakovic wrote: > > --

Re: Review Request 36770: SAMZA-742 Add a Contribution section in README of samza and hello-samza

2015-07-24 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36770/#review92936 --- Ship it! Ship It! - Navina Ramesh On July 24, 2015, 8:57 a.m

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

2015-07-24 Thread Navina Ramesh
e/src/main/scala/org/apache/samza/container/SamzaContainer.scala (line 627) <https://reviews.apache.org/r/36545/#comment147198> LocalityManager maintains "container to host"-level mapping. Not a task to host mapping. Please change this back to containerId. - Navina Ramesh On

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

2015-07-24 Thread Navina Ramesh
samza-hello-samza. I believe ElasticsearchProducer is going to be part of the 0.10 release in Samza. So, this patch should be against the "latest" branch in hello-samza. This diff looks like it is against the master branch. Can you please make the changes against latest branch ? - Navin

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

2015-07-24 Thread Navina Ramesh
to setup and run the example Samza job that uses ElasticSearchProducer. - Navina Ramesh On July 24, 2015, 8:31 a.m., Stuart Davidson wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://review

Re: kafka producer failed

2015-07-24 Thread Navina Ramesh
Hi Selina, Looks like the attachment got filtered. Can you past the config in the email or at pastebin? Thanks! Navina On Fri, Jul 24, 2015 at 1:18 PM, Job-Selina Wu wrote: > Hi, Yi: > > I am wondering if the problem can be fixed by the parameter " > max.message.size" at kafka.producer.P

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

2015-07-26 Thread Navina Ramesh
> On July 24, 2015, 6:19 p.m., Navina Ramesh wrote: > > It will also be nice to have a tutorial added to the website on how to > > setup and run the example Samza job that uses ElasticSearchProducer. > > Stuart Davidson wrote: > Happy to write something up. Do

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

2015-07-28 Thread Navina Ramesh
> On July 21, 2015, 9:53 p.m., Navina Ramesh wrote: > > Ideally, the job coordinator should act as the config manager. Since the > > chain of control is still from AppMaster to JobCoordinator (instead of the > > other way around), the only reasonable way to get this worki

Re: [DISCUSS] Release 0.10.0

2015-07-29 Thread Navina Ramesh
+1 for StreamAppender bug fix as a mandatory in 0.10. @Yan: will review SAMZA-676 today and test it out locally. Thanks! Navina On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman wrote: > I can take care of SAMZA-340, SAMZA-683 and will follow up with Luis for > SAMZA-401,2,3,4 > > On Wed, Jul 29

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

2015-07-29 Thread Navina Ramesh
> On July 24, 2015, 6:01 p.m., Navina Ramesh wrote: > > Thanks for picking this up! It feels good to look at a refactored code. > > > > One suggestion: Please run all the intergration test (including the zopkio > > tests) before checking in this patch. I don&#

Re: Required vs. optional methods for KeyValueStore

2015-07-29 Thread Navina Ramesh
Hi Ken, We use putAll(list) when restoring from changelog. So, unless you don't want your store to have support for changelog, the implementation is required. I only have a high-level overview of what Solr is. Perhaps, others on the mailing list have experience with Solr and can provide more usef

  1   2   3   4   5   6   >