Re: Terminology: Tumbling and sliding windows

+1 on using the right terminology. Navina On 2/17/15, 9:33 AM, Yi Pan nickpa...@gmail.com wrote: +1 on consolidating the terminology as well. Azure's definition looks good to me. On Tue, Feb 17, 2015 at 9:13 AM, Chris Riccomini criccom...@apache.org wrote: Hey Julian, +1 I'm not sure if we

Re: Review Request 29899: Changes for Kafka producer upgrade

4989b279a22ffd8acdb2c31ca4813f6768edc75b samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala ca25258217e5ebc44b34fbc4d69ecb28c81df618 Diff: https://reviews.apache.org/r/29899/diff/ Testing --- ./gradlew clean build - SUCCESSFUL Thanks, Navina Ramesh

Re: Tags for Samza in stack overflow

to get it working. As I believe it is an easy process but I lack the references to do it, I hope it will make it easier for the rest with it. best regards, Jordi De: Navina Ramesh [nram...@linkedin.com.INVALID] Enviado: martes, 24 de marzo de 2015 3:40

Re: Review Request 32188: Disable WAL in RocksDB KV store

://reviews.apache.org/r/32188/diff/ Testing (updated) --- ./bin/check-all.sh ran successfully on MAC Thanks, Navina Ramesh

Re: New to Samza/Yarn and having Kafka issues

Hi Ash, I just ran wikipedia-parser with your patch. Looks like you have set the message serde correctly in the configs. However, the original code still converts it into a Map for consumption in the WikipediaFeedEvent. I am seeing the following (expected): 2015-03-23 19:17:49

Tags for Samza in stack overflow

Hi all, I think it is good idea to improve our presence in stackoverflow.com because: * any new user/developer mostly googles/searches stackoverflow for resolving issues before hitting us on the mailing list; This requires the user to subscribe to the mailing list - unnecessary overhead, apart

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

by container ID, not jobId? Navina Ramesh wrote: I think containerId is not really relevant because the store partition directories are uniquely named across the containers. The ownership of containers and partition directory is persisted in the config stream (going forward). I will verify

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33453/#review81838 --- On April 22, 2015, 9:54 p.m., Navina Ramesh wrote

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

://reviews.apache.org/r/33453/#review82593 --- On May 5, 2015, 6:47 p.m., Navina Ramesh wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

locally using hello-samza. Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state. Thanks, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

. prefer the storePartitionDir, because it's the dir for each partition/task, right? Navina Ramesh wrote: 1. why does the changeLogSystemStreamPartition matter here? changeLogSystemStreamPartition indicates whether a store has a changelog or not. Depending on the case, the base directory

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

that we don't read in a corrupted value? Navina Ramesh wrote: No. As of now, we don't have any checksum. Do you think it is better to add it now than later ? Yi, I created SAMZA-666 to work on adding checksum data to the offset file. - Navina

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

, Navina Ramesh

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

--- Tested locally using hello-samza. Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state. Thanks, Navina Ramesh

Re: Review Request 34009: SAMZA-552 window store implementation

On May 13, 2015, 9:56 p.m., Navina Ramesh wrote: samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java, line 96 https://reviews.apache.org/r/34009/diff/1/?file=954320#file954320line96 Shouldn't the window id be system generated? Yi Pan (Data

Re: Issues with certain characters in config values

Hi Tommy, I wouldn't think Util.envVarEscape() is totally unnecessary. Are you running into this issue with a version = 0.9 ? With 0.10, I think we no longer pass all the environment variables through command line. We only pass the Coordinator system information via command line. While

Re: Deploying samza job to a remote yarn resource manager

Hi George, Looks like there is a formatting issue on the samza website. The environment variable name should be HADOOP_YARN_HOME. In your case, it should point to hadoop home: $HOME/hadoop-2.6.0/ Please let me know if it works. Cheers! Navina On 6/22/15, 2:11 PM, George Li g...@ca.ibm.com

Re: Deploying samza job to a remote yarn resource manager

address? Maybe I should include HADOOP_YARN_HOME and HADOOP_CONF_DIR as classpath? George From: Navina Ramesh nram...@linkedin.com.INVALID To: dev@samza.apache.org dev@samza.apache.org, Date: 22/06/2015 05:57 PM Subject:Re: Deploying samza job to a remote yarn resource manager Hi

Re: Review Request 34974: SAMZA-676: implement broadcast stream

' - Navina Ramesh On June 16, 2015, 9:23 p.m., Yan Fang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34974/ --- (Updated

Re: Missing artifact? org.apache.samza:samza-serializers_2.10:jar:0.9.0

Hi Mart, Tommy is right. We moved the serializers into samza-core. I think that documentation requires an update. Please remove the dependency section for samza_serializers. Thanks! Navina On 6/10/15, 4:36 AM, Tommy Becker tobec...@tivo.com wrote: I'm sure others will chime in, but I think that

Re: [DISCUSS] Samza 0.9.1 release

+1 for the release! On 6/16/15, 11:03 AM, Yi Pan nickpa...@gmail.com wrote: +1 Agreed. Thanks! On Tue, Jun 16, 2015 at 10:15 AM, Yan Fang yanfang...@gmail.com wrote: Agreed on this. Thanks, Fang, Yan yanfang...@gmail.com On Tue, Jun 16, 2015 at 10:14 AM, Guozhang Wang

Re: Review Request 35348: SAMZA-707 Remove dependency on samza-serializers in the samza.apache.org/startup/download page

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35348/#review88287 --- Ship it! Ship It! - Navina Ramesh On June 17, 2015, 8:15 p.m

Re: Review Request 35241: refactoring the code for coordinator stream writer

/MockCoordinatorStreamSystemFactory.java (line 60) https://reviews.apache.org/r/35241/#comment140583 nit: spacing - Navina Ramesh On June 9, 2015, 11:53 p.m., Shadi A. Noghabi wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 35348: SAMZA-707 Remove dependency on samza-serializers in the samza.apache.org/startup/download page

/#comment140198 Actually, this dependency is required if the samza version used is 0.9. So, can you add a comment below the Artifacts section mentioning this additional dependency for samza 0.9 ? - Navina Ramesh On June 11, 2015, 8:49 a.m., Aleksandar Bircakovic wrote

Re: Combine two web servlets in the AM?

+1 for adding to the documentation. I didn¹t know about this difference. This information is perhaps part of the JobCoordinator/CoordinatorStream documentation - SAMZA-614? On 6/15/15, 11:27 AM, Yan Fang yanfang...@gmail.com wrote: All right. Sorry for bothering. I figured it out.

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

to verify that these properties get overridden correctly ? Sorry about this afterthought request. - Navina Ramesh On June 15, 2015, 1:42 p.m., Robert Zuljevic wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 34585: SAMZA-681: create a unit test harness to easily test samza tasks

/test/java/org/apache/samza/test/harness/InMemoryTaskContextTest.java https://reviews.apache.org/r/34585/#comment138585 Same here - Navina Ramesh On May 22, 2015, 5:23 a.m., Luis De Pombo wrote: --- This is an automatically

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

this occurs because stream() and sink() method return OperatorSource type rather than a TopologyBuilder instance. How differently does the TopologyBuilder handle the OperatorSource and the OperatorSink ? - Navina Ramesh On May 20, 2015, 11:13 p.m., Yi Pan (Data Infrastructure) wrote

Re: Review Request 34746: Adding new CoordinatorStreamMessage SetContainerHostMapping and LocalityManager (SAMZA-618)

--- On May 29, 2015, 11:26 p.m., Navina Ramesh wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34746

Re: Review Request 34746: Adding new CoordinatorStreamMessage SetContainerHostMapping and LocalityManager (SAMZA-618)

: https://reviews.apache.org/r/34746/diff/ Testing --- Used a sample job to test it locally and also, by setting up YARN on 3 machines. Verified that the message is correctly written and consumed from the Coordinator Stream Thanks, Navina Ramesh

Re: Review Request 34746: Adding new CoordinatorStreamMessage SetContainerHostMapping and LocalityManager (SAMZA-618)

://reviews.apache.org/r/34746/diff/ Testing --- Used a sample job to test it locally and also, by setting up YARN on 3 machines. Verified that the message is correctly written and consumed from the Coordinator Stream Thanks, Navina Ramesh

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

On May 29, 2015, 1:28 a.m., Navina Ramesh wrote: samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java, line 120 https://reviews.apache.org/r/34500/diff/1/?file=965740#file965740line120 I thought TopologyBuilder was to abstract away the spec and provide

Re: Reprocessing old events no longer in Kafka

Hi Zach, It sounds like you are asking for a SystemConsumer for hdfs. Does SAMZA-263 match your requirements? Thanks! Navina On 5/29/15, 2:23 PM, Zach Cox zcox...@gmail.com wrote: (continuing from previous email) in addition to not wanting to duplicate code, say that some of the Samza jobs

Re: Reprocessing old events no longer in Kafka

messages from the other Kafka topic. Thanks! Navina On 5/29/15, 2:26 PM, Navina Ramesh nram...@linkedin.com wrote: Hi Zach, It sounds like you are asking for a SystemConsumer for hdfs. Does SAMZA-263 match your requirements? Thanks! Navina On 5/29/15, 2:23 PM, Zach Cox zcox...@gmail.com wrote

Re: Reprocessing old events no longer in Kafka

a composite stream, that would first read old events from hdfs and then switch over to kafka. Do you know if there has been any movement on treating hdfs as a samza stream? Thanks, Zach On Fri, May 29, 2015 at 4:27 PM Navina Ramesh nram...@linkedin.com.invalid wrote: Hi Zach, It sounds like you

Re: [VOTE] Apache Samza 0.9.1 RC1

+1 Verified MD5 asc signatures. Local build check-all.sh passed. Thanks! Navina On 6/30/15, 7:13 PM, Chris Riccomini criccom...@apache.org wrote: +1 Verified MD5, and asc signature. Build locally, and all tests pass. Cheers, Chris On Tue, Jun 30, 2015 at 1:26 PM, Milinda Pathirage

Review Request 36224: SAMZA-728: Samza job fails due to null pointer in JobCoordinator refreshJobModel

://reviews.apache.org/r/36224/diff/ Testing --- Tested with a simple job deployed a multi-node yarn cluster with kafka/zookeeper deployments from hello-samza Thanks, Navina Ramesh

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36545/#review93826 --- Ship it! Thanks for testing with Zopkio ! - Navina Ramesh

Re: Review Request 34974: SAMZA-676: implement broadcast stream

can add that if there is enough user interest. samza-core/src/main/java/org/apache/samza/config/TaskConfigJava.java (line 53) https://reviews.apache.org/r/34974/#comment151212 Broadcast sounds much better! :) - Navina Ramesh On Aug. 20, 2015, 8:13 p.m., Yan Fang wrote

Re: Review Request 37536: SAMZA-710 Update WebServlet and RestServlet to read coordinatorStream information

On Aug. 20, 2015, 5:36 p.m., Yan Fang wrote: samza-core/src/main/java/org/apache/samza/container/ConfigManager.java, line 29 https://reviews.apache.org/r/37536/diff/2/?file=1044899#file1044899line29 How about renaming to LatestConfigManager? Then it is more specific and does

Re: Required vs. optional methods for KeyValueStore

Hi Ken, We use putAll(list) when restoring from changelog. So, unless you don't want your store to have support for changelog, the implementation is required. I only have a high-level overview of what Solr is. Perhaps, others on the mailing list have experience with Solr and can provide more

Re: Required vs. optional methods for KeyValueStore

notes about which methods are required (used by the Samza system) for changelog support vs. optional (only used by task-specific code as needed) would be very helpful. Thanks! -- Ken From: Navina Ramesh Sent: July 29, 2015 10:38:45am PDT To: dev@samza.apache.org Subject: Re: Required

Re: Keep samza container logs from being deleted

Hi Chen, You can set the yarn.nodemanage.delete.debug-delay-sec config in yarn-site.xml, in order to retain the logs after the job crashes. Refer https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml (This was intended for debug purpose). If you need to retain

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

please add a one-line comment about the shouldRewriteConfigToCoordinatorStream variable? Someone without the context of ConfigManager may not understand the motivation behind it. Overall, looks good. Please address Yi's comments as well and put up a patch. - Navina Ramesh On July 28, 2015, 2

Re: [DISCUSS] Release 0.10.0

on Linux boxes Anything that I missed? Thanks! -Yi On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh nram...@linkedin.com.invalid wrote: +1 for StreamAppender bug fix as a mandatory in 0.10. @Yan: will review SAMZA-676 today and test it out locally. Thanks! Navina

Re: [DISCUSS] Release 0.10.0

On Thu, Jul 30, 2015 at 5:10 PM, Navina Ramesh nram...@linkedin.com.invalid wrote: Hi Yi, Thanks for summarizing. But why are we excluding SAMZA-723 from the current release ? Doesn't this break the existing StreamAppender functionality in 0.9? Thanks! Navina On Thu, Jul 30

Re: Coordinator URL always 127.0.0.1

Hi Tommy, Yi is right. Container start is coordinated by the AppMaster using an NMClient. Container host name and port is provided by the RM during allocation. In Yarn (at least, afaik), when the node joins a cluster, the NM registers itself with the RM. So, the NM might still be using

Re: Coordinator URL always 127.0.0.1

. Thanks, Fang, Yan yanfang...@gmail.com On Thu, Jul 30, 2015 at 6:06 PM, Navina Ramesh nram...@linkedin.com.invalid wrote: The NM (and hence, by extension the container) heartbeats to the RM, not the AM. AM gets notified of container status from the RM. The AM starts / stops /releases

Re: Review Request 36471: Autoscaling for samza (work in progress)

how much memory to free up per container that you scale up?? Does autoscaling.optimizer.percent.memory.revoke or autoscaling.optimizer.memory.revoke sound correct? - Navina Ramesh On July 31, 2015, 4:24 a.m., Shadi A. Noghabi wrote

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

On July 24, 2015, 6:01 p.m., Navina Ramesh wrote: Thanks for picking this up! It feels good to look at a refactored code. One suggestion: Please run all the intergration test (including the zopkio tests) before checking in this patch. I don't think we cleanly start and stop

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

On July 21, 2015, 9:53 p.m., Navina Ramesh wrote: Ideally, the job coordinator should act as the config manager. Since the chain of control is still from AppMaster to JobCoordinator (instead of the other way around), the only reasonable way to get this working is to have

Re: Review Request 36903: SAMZA-744: shutdown stores before shutdown producers

. remove ';' - Navina Ramesh On Aug. 6, 2015, 11:24 a.m., Yi Pan (Data Infrastructure) wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36903

Re: Review Request 34974: SAMZA-676: implement broadcast stream

SystemAdmin of type (x$1: String, x$2: String)Integer is not defined class HdfsSystemAdmin extends SystemAdmin with Logging { ^ one error found I guess you can just throw an UnsupportedMethodException for now. - Navina Ramesh On July 29, 2015, 10:49 p.m., Yan Fang wrote

Re: Review Request 34974: SAMZA-676: implement broadcast stream

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34974/#review94980 --- Ship it! Ship It! - Navina Ramesh On July 29, 2015, 10:49 p.m

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

On July 24, 2015, 6:19 p.m., Navina Ramesh wrote: It will also be nice to have a tutorial added to the website on how to setup and run the example Samza job that uses ElasticSearchProducer. Stuart Davidson wrote: Happy to write something up. Do you want it on the hello-samza

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

move this to the init method in SamzaAppMasterService?? This way you don't have to change the interface of the run method. - Navina Ramesh On July 18, 2015, 2:52 a.m., Shadi A. Noghabi wrote: --- This is an automatically generated

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

believe ElasticsearchProducer is going to be part of the 0.10 release in Samza. So, this patch should be against the latest branch in hello-samza. This diff looks like it is against the master branch. Can you please make the changes against latest branch ? - Navina Ramesh On July 24, 2015, 8:31

Re: Review Request 36770: SAMZA-742 Add a Contribution section in README of samza and hello-samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36770/#review92936 --- Ship it! Ship It! - Navina Ramesh On July 24, 2015, 8:57 a.m

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

/main/scala/org/apache/samza/container/SamzaContainer.scala (line 627) https://reviews.apache.org/r/36545/#comment147198 LocalityManager maintains container to host-level mapping. Not a task to host mapping. Please change this back to containerId. - Navina Ramesh On July 24, 2015, 12:27 p.m

Re: Review Request 35241: refactoring the code for coordinator stream writer

like you are doing a null check in Line 61 and then, creating another handle for the field in Line 64. - Navina Ramesh On June 29, 2015, 11:08 p.m., Shadi A. Noghabi wrote: --- This is an automatically generated e-mail. To reply, visit

Re: Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33419/#review91865 --- Ship it! Ship It! - Navina Ramesh On July 16, 2015, 12:56 a.m

Re: hello-samza git repo moved

Hi Guozhang, I tried sometime back. It works. nramesh-mn1:Projects nramesh$ git clone https://github.com/apache/samza-hello-samza open-source-hello-samza Cloning into 'open-source-hello-samza'... remote: Counting objects: 510, done. remote: Total 510 (delta 0), reused 0 (delta 0), pack-reused

Re: hello-samza git repo moved

email as well. But the git://git.apache.org/samza-hello-samza.git does not work, which is stated in the wiki page (http://samza.apache.org/startup/hello-samza/0.9/ ). Guozhang On Tue, Jul 14, 2015 at 3:57 PM, Navina Ramesh nram...@linkedin.com.invalid wrote: Hi Guozhang, I tried

Re: Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

://reviews.apache.org/r/33419/#comment145435 nit: SYSTEM_FACTORY can be defined in this class rather than from the scala class. Perhaps a future work of cleanup can do this as well. - Navina Ramesh On June 21, 2015, 6:10 a.m., Yan Fang wrote

Re: Review Request 39252: SAMZA-626 - tool to read the RocksDb in a running job (Yan's patch)

, Navina Ramesh

Re: Review Request 39464: SAMZA-723 - hello-samza hangs when we use StreamAppender (Yan's patch)

to track this - https://issues.apache.org/jira/browse/SAMZA-800 - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39464/#review103838 -------

Re: Review Request 39252: SAMZA-626 - tool to read the RocksDb in a running job (Yan's patch)

I noticed that we are creating new JavaSerializerConfig, > > JavaStorageConfig, etc. instead of change the scala classes directly to > > Java. Why is it? It seems to be a good opportunity to move part of the code > > to Java. > > Navina Ramesh wrote: > Yeah. I think

Re: Review Request 39252: SAMZA-626 - tool to read the RocksDb in a running job (Yan's patch)

, Navina Ramesh

Re: Review Request 39252: SAMZA-626 - tool to read the RocksDb in a running job (Yan's patch)

ges in other parts of the code. I will work on that now! - Navina ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39252/#review103702 ---

Review Request 39558: Fix parsing errors in broadcast stream config values

/TaskConfigJava.java 015e99449f045fa578059f8b23a401274b0503ba samza-core/src/test/java/org/apache/samza/config/TestTaskConfigJava.java 2d6060ef41ba3105fb9f954d3f12065d699469b8 Diff: https://reviews.apache.org/r/39558/diff/ Testing --- ./gradlew clean build Thanks, Navina Ramesh

Re: How to set up RocksDB

ocksDB.(RocksDB.java:28) > > ... 27 more > > > > It should be this line in code: > > > https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/RocksDB.java#L51 > > > > Is there a way how to change the output directory for ThreadJobFac

Re: How to set up RocksDB

Hi Lukas, Are you running the Samza job on Yarn? If so, the state stores can be found in the working directory of a yarn application. Samza doesn't configure any special location. Unless you have configured the local-dir location in yarn, it should use the default, which is

Re: Samza hdfs

Hi Aram, I think a Cassandra producer will definitely be useful for some of the Samza users. It will be great if you can contribute back to Samza! I am also curious to understand what motivated you to move off from Spark Streaming. Thanks! Navina On Tue, Nov 10, 2015 at 11:21 PM, Bae, Jae Hyeon

Re: Review Request 39575: SAMZA-793 static config rewriter should be invoked in JobRunner

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39575/#review106318 --- Ship it! Lgtm! - Navina Ramesh On Oct. 23, 2015, 12:13 a.m

Re: question on yarn.container.cpu.cores

Hi Chen, Samza container is still single threaded. In case of yarn based deployment, Samza uses this config value to verify that the cluster has sufficient capacity to support running your job. Apart from this verification, I don't believe we utilize this config value. If you set it to > 1, it

Review Request 39806: SAMZA-798 : Performance and stability issue after combining checkpoint and coordinator stream

java b12ae5c1eaaee8e94d6e62a925a98d2c952fdb72 samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterLifecycle.scala ec5a8533c7a31b9790504e18e0528be28c77d496 Diff: https://reviews.apache.org/r/39806/diff/ Testing --- ./gradlew clean build Thanks, Navina Ramesh

Re: Checkpoint tool not working

Hi Lucas, What is the format that you are using for re-specifying the offsets? Do you specify it with --new-offsets ? If you can share that, it will be great. It seems to work for me. Thanks! Navina On Fri, Oct 30, 2015 at 9:11 AM, Roger Hoover wrote: > I tried it

Re: Review Request 39806: SAMZA-798 : Performance and stability issue after combining checkpoint and coordinator stream

orks as expected Thanks, Navina Ramesh

Re: Review Request 39806: SAMZA-798 : Performance and stability issue after combining checkpoint and coordinator stream

int Tool works as expected 4. Broadcast Stream works as expected 5. FileSystemCheckpointManager works as expected Thanks, Navina Ramesh

Re: Review Request 39806: SAMZA-798 : Performance and stability issue after combining checkpoint and coordinator stream

NE 5. FileSystemCheckpointManager works as expected - DONE Thanks, Navina Ramesh

Re: Review Request 39806: SAMZA-798 : Performance and stability issue after combining checkpoint and coordinator stream

gration - DONE 2. Second time job deployment only performs migration check and doesn't actually migrate anything - DONE 3. Checkpoint Tool works as expected - DONE 4. Broadcast Stream works as expected - DONE 5. FileSystemCheckpointManager works as expected Thanks, Navina Ramesh

Re: Review Request 39806: SAMZA-798 : Performance and stability issue after combining checkpoint and coordinator stream

ala prefers to use the companion object as the factory to > > create new instance (code before change). Is there any reason for this > > change? > > Navina Ramesh wrote: > I didn't know that about scala. But what is the advantage behind it? It > only seems to obscure the func

Re: Review Request 39806: SAMZA-798 : Performance and stability issue after combining checkpoint and coordinator stream

rate anything - DONE 3. Checkpoint Tool works as expected - DONE 4. Broadcast Stream works as expected - DONE 5. FileSystemCheckpointManager works as expected - DONE Thanks, Navina Ramesh

Re: 0.10.0 Release?

ocally I get links like: > > http://localhost:4000/learn/documentation/0.9/jobs/configuration-table.html > > > > > On Nov 5, 2015, at 3:43 PM, Navina Ramesh <nram...@linkedin.com> wrote: > > What is not working? generating documentations from source or are there >

Re: 0.10.0 Release?

t links like: > > http://localhost:4000/learn/documentation/0.9/jobs/configuration-table.html > > > > > On Nov 5, 2015, at 3:43 PM, Navina Ramesh <nram...@linkedin.com> wrote: > > What is not working? generating documentations from source or are there > missing configur

Re: Review Request 39806: SAMZA-798 : Performance and stability issue after combining checkpoint and coordinator stream

y. I don't mind either way. :) - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39806/#review104583 --- On Oct. 30, 2015,

Re: Review Request 39422: SAMZA-449 Expose RocksDB statistic

optional. What do you think? - Navina Ramesh On Oct. 18, 2015, 6:05 p.m., Gustavo Anatoly F. V. Solís wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > http

Review Request 39119: SAMZA-792: SamzaAppMaster Java code needs to pass the requested container memory size to RM

4c1eaa9354e3e3cfed9bf5e032d6d9e89a9bd8b5 samza-yarn/src/test/java/org/apache/samza/job/yarn/util/MockContainerAllocator.java 85f871a85b8fced212c7418d4c9a7f0de702811e Diff: https://reviews.apache.org/r/39119/diff/ Testing --- Thanks, Navina Ramesh

Re: Relaunching init

Hi Jordi, init() method is called only once during container start-up and not called again until the container is restarted. Semantically, it doesn't seem fit to make "init" callable more than once. With Samza 0.10 (that is, the current master branch), you can pass configuration via a broadcast

Re: Review Request 37994: SAMZA 368 AM UI should show failed container information

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37994/#review103155 --- Ship it! Ship It! - Navina Ramesh On Oct. 12, 2015, 11:59 a.m

Re: Review Request 37994: SAMZA 368 AM UI should show failed container information

> On Oct. 19, 2015, 8:48 p.m., Navina Ramesh wrote: > > Ship It! Thanks for re-patching this after SAMZA-619! - Navina --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37994/#rev

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

> On July 24, 2015, 6:18 p.m., Navina Ramesh wrote: > > bin/grid-elastic, line 1 > > <https://reviews.apache.org/r/36768/diff/1/?file=1020797#file1020797line1> > > > > Can you make changes to bin/grid by adding options to install > > elastic_sea

Review Request 39464: SAMZA-723 - hello-samza hangs when we use StreamAppender (Yan's patch)

, Navina Ramesh

Re: How to deploy Hello Samza app on ubuntu 15.04?

Hi Alex, I think the output you pasted is from the application-master.log. Can you send the link to the container logs? specifically, logs for container - container_1445285902612_0001_01_07 If a job fails in Yarn, the working directory usually gets deleted by NM's deletion service, unless you

Re: How to deploy Hello Samza app on ubuntu 15.04?

, > > Thank you for a prompt response! I updated yarn config and gathered some > logs. See attached. > > Alex > > > On Tue, Oct 20, 2015 at 12:17 PM, Navina Ramesh < > nram...@linkedin.com.invalid> wrote: > >> Hi Alex, >> I think the output you past

Re: Review Request 39252: SAMZA-626 - tool to read the RocksDb in a running job (Yan's patch)

/kv/TestRocksDbKeyValueReader.java PRE-CREATION samza-shell/src/main/bash/read-rocksdb-tool.sh PRE-CREATION Diff: https://reviews.apache.org/r/39252/diff/ Testing (updated) --- ./bin/check-all.sh Tested and verified with sample job on Yarn. Thanks, Navina Ramesh

Review Request 39355: Updating to rocksdb 3.13.1

: https://reviews.apache.org/r/39355/diff/ Testing --- Ran both these commands successfully on my Linux box & Mac ./gradlew clean test --refresh-dependencies ./bin/check-all.sh Thanks, Navina Ramesh

Re: Review Request 39119: SAMZA-792: SamzaAppMaster Java code needs to pass the requested container memory size to RM

://reviews.apache.org/r/39119/diff/ Testing --- Thanks, Navina Ramesh

Review Request 39252: SAMZA-626 - Yan's working patch

samza-shell/src/main/bash/read-rocksdb-tool.sh PRE-CREATION Diff: https://reviews.apache.org/r/39252/diff/ Testing --- Thanks, Navina Ramesh

Re: Review Request 39252: SAMZA-626 - Yan's working patch

/TestRocksDbKeyValueReader.java PRE-CREATION samza-shell/src/main/bash/read-rocksdb-tool.sh PRE-CREATION Diff: https://reviews.apache.org/r/39252/diff/ Testing --- Thanks, Navina Ramesh

  1   2   3   4   5   >