samza git commit: SAMZA-1435: Changed samza-api Serde implementations from Scala to Java.

2017-10-06 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master b989e51ae -> 357d6ca72 SAMZA-1435: Changed samza-api Serde implementations from Scala to Java. Author: Prateek Maheshwari Reviewers: Jacob Maes Closes #316 from prateekm/java-serdes Project:

samza git commit: Added self to committers list.

2017-10-03 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master db2f121e2 -> 1de13f765 Added self to committers list. vjagadish1989 Author: Prateek Maheshwari Closes #311 from prateekm/commiters-update Project: http://git-wip-us.apache.org/repos/asf/samza/repo Commit:

samza git commit: SAMZA-1444: Removed circular dependency b/w samza-core test and samza-kv-rocksdb test

2017-10-05 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 7ad6631bb -> 7f1ec6492 SAMZA-1444: Removed circular dependency b/w samza-core test and samza-kv-rocksdb test Also added an implementation for KVSerde. vjagadish1989, thanks for the TestInMemoryStore implementation! The only usage of

samza git commit: SAMZA-1461; Expose RocksDB properties as metrics

2017-10-16 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master c93dd8f60 -> 343712e30 SAMZA-1461; Expose RocksDB properties as metrics Automatically build gauges for RocksDB properties via configuration: `stores..rocksdb.telemetry.list=, ` Author: Janek Lasocki-Biczysko

samza git commit: SAMZA-1292: Merge operator can be no-op when there are no streams to merge

2017-10-09 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master f8cce6e15 -> 7b0a65b14 SAMZA-1292: Merge operator can be no-op when there are no streams to merge Author: Prateek Maheshwari Reviewers: Jacob Maes Closes #320 from prateekm/small-fixes

samza git commit: SAMZA-1109: Updated High Level API serde impl with Yi's feedback

2017-10-04 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 8b3fe5d26 -> ad80cf9f1 SAMZA-1109: Updated High Level API serde impl with Yi's feedback nickpan47 for review. Author: Prateek Maheshwari Reviewers: "Jagadish Venkatraman " Closes #310 from

samza git commit: Added Prateek's code signing public key to KEYS

2017-10-03 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 1de13f765 -> 471f81d78 Added Prateek's code signing public key to KEYS Author: Prateek Maheshwari Closes #312 from prateekm/commiters-update Project: http://git-wip-us.apache.org/repos/asf/samza/repo Commit:

[2/2] samza git commit: SAMZA-1056: Added wiring for High Level API state stores, their serdes and changelogs.

2017-10-04 Thread pmaheshwari
SAMZA-1056: Added wiring for High Level API state stores, their serdes and changelogs. Provided join operator access to durable state stores. Author: Prateek Maheshwari Reviewers: Jagadish Venkatraman Closes #309 from

[1/2] samza git commit: SAMZA-1056: Added wiring for High Level API state stores, their serdes and changelogs.

2017-10-04 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master ad80cf9f1 -> a671288e1 http://git-wip-us.apache.org/repos/asf/samza/blob/a671288e/samza-core/src/test/java/org/apache/samza/operators/impl/store/TestTimeSeriesStoreImpl.java

samza-hello-samza git commit: SAMZA-1441: Updated High Level API examples in hello-samza to provide serdes in code.

2017-10-04 Thread pmaheshwari
Repository: samza-hello-samza Updated Branches: refs/heads/latest 901c3a390 -> e5943a000 SAMZA-1441: Updated High Level API examples in hello-samza to provide serdes in code. Author: Chris Riccomini Author: Yan Fang Author: Prateek

samza git commit: SAMZA-272, SAMZA-1440, SAMZA-1269: Fixed thread interrupt tests in TestExponentialSleepStrategy.

2017-10-06 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 357d6ca72 -> f8cce6e15 SAMZA-272, SAMZA-1440, SAMZA-1269: Fixed thread interrupt tests in TestExponentialSleepStrategy. Author: Prateek Maheshwari Reviewers: Jacob Maes Closes #318 from

samza git commit: SAMZA-1451: Disable integration tests conditionally in build.

2017-10-11 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 24170389c -> e6eb13f85 SAMZA-1451: Disable integration tests conditionally in build. Remove runIntegrationTests gradle property added as a part of SAMZA-1355 and introduce skipIntegrationTests property(which inverts it). If

samza git commit: SAMZA-1466: Flaky test: TestRocksDbKeyValueStore suite

2017-10-19 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master fd9f0802f -> 62904334b SAMZA-1466: Flaky test: TestRocksDbKeyValueStore suite TestRocksDbKeyValueStore tests are failing intermittently with errors like: ``` testFlush FAILED org.apache.samza.SamzaException: Error opening RocksDB store

samza git commit: Minor fixes to KeyValueStore and RocksDBKeyValueStore

2017-10-19 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 62904334b -> 958edc42f Minor fixes to KeyValueStore and RocksDBKeyValueStore 1. Replaced extension class in KeyValueStore with default methods. 2. Fixed formatting in RocksDBKeyValueStore#openDB. 3. Now logs original RocksDBException on

samza git commit: SAMZA-1501: Validate operator IDs so that they don't contain special characters and spaces

2017-11-15 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 6fcf7f3f4 -> 4ac62231c SAMZA-1501: Validate operator IDs so that they don't contain special characters and spaces Author: Prateek Maheshwari Reviewers: Jacob Maes , Jagadish Venkatraman

samza git commit: SAMZA-1495: Set intermediate streams as higher priority by default

2017-11-15 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 4ac62231c -> fc6c70684 SAMZA-1495: Set intermediate streams as higher priority by default Most changes in StreamConfig are formatting fixes. Author: Prateek Maheshwari Reviewers: Jagadish Venkatraman

samza git commit: Updated Serde related documentation and error messages for High Level API

2017-12-04 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 9c9bbc45c -> 32d2af5dd Updated Serde related documentation and error messages for High Level API Updated and clarified the documentation and error messages related to Serdes for Input/Output/PartitionBy streams. Author: Prateek

samza git commit: Updated Serde related documentation and error messages for High Level API

2017-12-04 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/0.14.0 66e249d87 -> b6d0e2e02 Updated Serde related documentation and error messages for High Level API Updated and clarified the documentation and error messages related to Serdes for Input/Output/PartitionBy streams. Author: Prateek

samza git commit: Added document for table API to feature preview

2017-12-14 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 2e5ceec19 -> 54dc4a03e Added document for table API to feature preview Added document for table API to feature preview - Brief description of table - sendTo() operator for table - join() operator for stream-table-join Author: Wei Song

samza-hello-samza git commit: Added stream-table join sample to cookbook

2017-12-18 Thread pmaheshwari
Repository: samza-hello-samza Updated Branches: refs/heads/latest cb4365359 -> befe934c4 Added stream-table join sample to cookbook As per subject, it's a demo of stream-to-table join using RocksDb. Author: Wei Song Reviewers: Prateek Maheshwari

samza git commit: SAMZA-1538: Disabled Flaky Tests in TestStreamProcessor

2017-12-18 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master ed8dad54b -> ea2b6fa91 SAMZA-1538: Disabled Flaky Tests in TestStreamProcessor Author: Prateek Maheshwari Reviewers: Shanthoosh Venkataraman Closes #389 from prateekm/disable-flaky-test

samza git commit: SAMZA-1490: Fix TestRepartitionJoinWindowApp

2017-11-14 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 26dd9c14c -> 42c194040 SAMZA-1490: Fix TestRepartitionJoinWindowApp Author: Dong Lin Reviewers: Prateek Maheshwari Closes #356 from lindong28/SAMZA-1490 Project:

samza git commit: SAMZA-1487: Disable Flaky Zk Integration tests.

2017-11-14 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 42c194040 -> 20fe6ec61 SAMZA-1487: Disable Flaky Zk Integration tests. Author: Shanthoosh Venkataraman Reviewers: Yi Pan Closes #355 from shanthoosh/master Project:

samza git commit: SAMZA-1474: Bump up rocksdb version to 5.7.3 to include licensing changes

2017-11-14 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 20fe6ec61 -> 3750f5e24 SAMZA-1474: Bump up rocksdb version to 5.7.3 to include licensing changes You can find more details on the bug fixes and API changes [here](https://github.com/facebook/rocksdb/releases). I upgraded to 5.7.3 since

samza git commit: SAMZA-1487: Disable Flaky Zk Integration tests.

2017-11-09 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master d806e9dab -> 33010a731 SAMZA-1487: Disable Flaky Zk Integration tests. Author: Shanthoosh Venkataraman Reviewers: Prateek Maheshwari Closes #353 from shanthoosh/master Project:

[2/3] samza git commit: Initial version of Table API

2017-12-05 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/e74998c5/samza-core/src/main/java/org/apache/samza/operators/spec/StreamTableJoinOperatorSpec.java -- diff --git

[1/3] samza git commit: Initial version of Table API

2017-12-05 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master e3efdf5c8 -> e74998c5e http://git-wip-us.apache.org/repos/asf/samza/blob/e74998c5/samza-kv/src/main/java/org/apache/samza/storage/kv/LocalStoreBackedReadableTable.java --

[3/3] samza git commit: Initial version of Table API

2017-12-05 Thread pmaheshwari
Initial version of Table API Initial version of table API, it includes - Core table API (Table, TableDescriptor, TableSpec) - Local table implementation for in-memory and RocksDb - The writeTo() and stream-table join operators nickpan47 xinyuiscool prateekm could you help review? Author: Wei

samza git commit: SAMZA-1464: Flushing a closed RocksDB store causes SIGSEGVs

2017-10-25 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 85184d05b -> 711dd8dc3 SAMZA-1464: Flushing a closed RocksDB store causes SIGSEGVs Made RocksDB operations check if DB is still open to avoid segfaults. Author: Prateek Maheshwari Reviewers: Jagadish Venkatraman

[2/2] samza git commit: SAMZA-1454: Globally unique and user settable IDs for stateful operators

2017-10-25 Thread pmaheshwari
SAMZA-1454: Globally unique and user settable IDs for stateful operators Author: Prateek Maheshwari Reviewers: Jagadish Venkatraman , Yi Pan Closes #324 from prateekm/operator-id-uniqueness Project:

[1/2] samza git commit: SAMZA-1454: Globally unique and user settable IDs for stateful operators

2017-10-25 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 711dd8dc3 -> 1296c7ff9 http://git-wip-us.apache.org/repos/asf/samza/blob/1296c7ff/samza-core/src/test/java/org/apache/samza/operators/TestMessageStreamImpl.java -- diff

samza git commit: SAMZA-1700: Clean up SystemAdmins instance creation flows

2018-05-10 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 345f9f233 -> fed8212bd SAMZA-1700: Clean up SystemAdmins instance creation flows The SystemAdmins class has a special "test" constructor for building SystemAdmins instances for unit tests. However, that test constructor has leaked into

samza git commit: SAMZA-1696: Delete TestKeyValueStores flaky test

2018-05-11 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master e204d298e -> 15e0ababb SAMZA-1696: Delete TestKeyValueStores flaky test `testParallelReadWriteSameKey` is a flaky test that was failing due to a race condition between the main test thread and the thread manipulating the key-value store

samza git commit: SAMZA-1722: Upgrading avro from 1.7.0 to 1.7.1

2018-05-21 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master ebdb5b2f7 -> b0ce5db9b SAMZA-1722: Upgrading avro from 1.7.0 to 1.7.1 Jira: SAMZA-1722: https://issues.apache.org/jira/browse/SAMZA-1722 Author: sanil15 Reviewers: Prateek Maheshwari

samza git commit: Convert a put to to a delete operation in ReadWriteTable and TableWriteFunction when input value is null

2018-06-07 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 36c4c4b02 -> 161d1c47a Convert a put to to a delete operation in ReadWriteTable and TableWriteFunction when input value is null Currently, the behavior of putting a null value is inconsistent: it is a delete for RocksDB, and not

samza git commit: SAMZA-1736: Add counters and timers for batch get/put/delete operations in KeyValueStorageEngine

2018-06-01 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master ae056ff7b -> 2227f9f1a SAMZA-1736: Add counters and timers for batch get/put/delete operations in KeyValueStorageEngine Author: Prateek Maheshwari Reviewers: Cameron Lee , Shanthoosh Venkatraman Closes #539 from

samza git commit: Minor: KafkaConfig should treat empty changelog name as no changelog.

2018-06-06 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 4c0e50581 -> 36c4c4b02 Minor: KafkaConfig should treat empty changelog name as no changelog. If a store changelog stream name is empty, treat is as a non-changelogged store instead of throwing an exception. Author: Prateek Maheshwari

samza git commit: SAMZA-1476: Fix TestStatefulTask flaky test

2018-05-01 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 445d1e26c -> 94ff28c56 SAMZA-1476: Fix TestStatefulTask flaky test Unable to reproduce the issue even after constraining CPU/memory resources available to the JVM, this fix addresses a potential root cause — a `CountDownLatch` shared

samza git commit: SAMZA-1243: Fix flaky tests in TestLocalStoreMonitor

2018-05-01 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 94ff28c56 -> 906aa6b88 SAMZA-1243: Fix flaky tests in TestLocalStoreMonitor Having attempted to reproduce this issue without success, careful analysis of the code indicates the underlying issue is most likely incomplete cleanup of file

samza git commit: Fix for the TestSamzaSqlApplicationConfig.testConfigInit

2018-01-10 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 9893069cb -> 267dfc6ba Fix for the TestSamzaSqlApplicationConfig.testConfigInit Currently testConfigInit checks for a hardcoded number for udfs. Whenever a new UDF is added, This test is going to fail if it is not updated. Changed the

[1/2] samza git commit: SAMZA-1548; Add start() and stop() to SystemAdmin

2018-01-25 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 2e04e1772 -> 75e70e569 http://git-wip-us.apache.org/repos/asf/samza/blob/75e70e56/samza-core/src/test/java/org/apache/samza/clustermanager/TestClusterBasedJobCoordinator.java

[2/2] samza git commit: SAMZA-1548; Add start() and stop() to SystemAdmin

2018-01-25 Thread pmaheshwari
SAMZA-1548; Add start() and stop() to SystemAdmin This patch adds start() and stop() to SystemAdmin interface. This can be useful for e.g. kafka.admin.AdminClient which needs to be started before it can be used. Since we add this method in interface and expect AdminClient to be stateful and

samza git commit: SAMZA-1588: Add random jitter to monitor’s scheduling interval.

2018-02-16 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master c3e6b5517 -> 718c66612 SAMZA-1588: Add random jitter to monitor’s scheduling interval. We’ve observed in LinkedIn execution environments that, all the monitors running on the YARN node-manager machines hitting an external service at

samza git commit: SAMZA-1791: Fixing problem with Serde of StackTraceElement, was causing test failure

2018-07-27 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master e0ff4c523 -> 3e7f2e52b SAMZA-1791: Fixing problem with Serde of StackTraceElement, was causing test failure Root cause of the failing test (after constructor param addition) is that java's StackTraceElement does not serialize/deserialize

samza git commit: NO-JIRA: Change instances of "is catched up" to "is caught up"

2018-07-26 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 3235929bc -> c4dfbd347 NO-JIRA: Change instances of "is catched up" to "is caught up" Replaces any use of "catched" with "caught" Author: Tom Butterwith Reviewers: Jagadish Venkatraman , Shanthoosh Venkatraman This patch had

samza git commit: SAMZA-1790: LocalContainerRunner should not extend AbstractApplicationRunner.

2018-08-02 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 0c488bbb8 -> 8ce1bd556 SAMZA-1790: LocalContainerRunner should not extend AbstractApplicationRunner. LocalContainerRunner is the launcher for the process running SamzaContainer in YARN. It extends the AbstractApplicationRunner since the

[3/4] samza git commit: SAMZA-1745: Remove all usages of StreamSpec and ApplicationRunner from the operator spec and impl layers.

2018-07-27 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/440a25c9/samza-core/src/main/scala/org/apache/samza/metrics/reporter/MetricsSnapshotReporterFactory.scala -- diff --git

[4/4] samza git commit: SAMZA-1745: Remove all usages of StreamSpec and ApplicationRunner from the operator spec and impl layers.

2018-07-27 Thread pmaheshwari
SAMZA-1745: Remove all usages of StreamSpec and ApplicationRunner from the operator spec and impl layers. This PR is a pre-requisite for adding support for user-provided SystemDescriptors and StreamDescriptors to the High Level API. It removes all usages of StreamSpec and ApplicationRunner

[2/4] samza git commit: SAMZA-1745: Remove all usages of StreamSpec and ApplicationRunner from the operator spec and impl layers.

2018-07-27 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/440a25c9/samza-core/src/test/java/org/apache/samza/operators/impl/TestOperatorImplGraph.java -- diff --git

[1/4] samza git commit: SAMZA-1745: Remove all usages of StreamSpec and ApplicationRunner from the operator spec and impl layers.

2018-07-27 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 43c36e6f2 -> 440a25c97 http://git-wip-us.apache.org/repos/asf/samza/blob/440a25c9/samza-kafka/src/test/scala/org/apache/samza/checkpoint/kafka/TestKafkaCheckpointManager.scala

samza git commit: SAMZA-1733: Populating ListGauge metric using DiagnosticsAppender for exceptions

2018-07-27 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 440a25c97 -> e0ff4c523 SAMZA-1733: Populating ListGauge metric using DiagnosticsAppender for exceptions This PR shows how the ListGauge can be used to emit exceptions using a DiagnosticsAppender. 1. DiagnosticsAppender is enabled using a

samza git commit: SAMZA-1752: Pass full config to the IO resolver

2018-08-10 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master b706ecebe -> a8ddede18 SAMZA-1752: Pass full config to the IO resolver SQL IO Resolver needs full configs so that it can filter out the configs specific to the source that the SQL application is interested in. This change provides the IO

samza git commit: SAMZA-1809: Add more logging to intermediate topic partition count inference

2018-08-15 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master ce5765796 -> 8f7326be7 SAMZA-1809: Add more logging to intermediate topic partition count inference Author: bharathkk Reviewers: Ahmed Abdul Hamid Closes #607 from bharathkk/execution-planner-log-fix Project:

samza git commit: SAMZA-1733: Create diagnostic topic if diagnostics enabled

2018-08-15 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master a8ddede18 -> ce5765796 SAMZA-1733: Create diagnostic topic if diagnostics enabled * Current SnapshotReporter semantics are to specify stream as . * We create topic in JobRunner so that AM can also emit metrics (if desired). Author:

samza git commit: SAMZA-1813: ApplicationRunner should use Planner generated configs for StreamManager

2018-08-21 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 06488bf7a -> a9ff09373 SAMZA-1813: ApplicationRunner should use Planner generated configs for StreamManager Author: Prateek Maheshwari Reviewers: Jagadish Venkatraman , Bharath Kumarasubramanian Closes #612 from

[3/5] samza git commit: SAMZA-1804: System and Stream Descriptors

2018-08-27 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/2a71baf7/samza-core/src/test/java/org/apache/samza/execution/TestJobNode.java -- diff --git a/samza-core/src/test/java/org/apache/samza/execution/TestJobNode.java

[4/5] samza git commit: SAMZA-1804: System and Stream Descriptors

2018-08-27 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/2a71baf7/samza-api/src/test/java/org/apache/samza/operators/descriptors/TestSimpleInputDescriptor.java -- diff --git

[1/5] samza git commit: SAMZA-1804: System and Stream Descriptors

2018-08-27 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 2d6b19953 -> 2a71baf7c http://git-wip-us.apache.org/repos/asf/samza/blob/2a71baf7/samza-test/src/test/java/org/apache/samza/test/framework/TestTimerApp.java -- diff --git

[2/5] samza git commit: SAMZA-1804: System and Stream Descriptors

2018-08-27 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/2a71baf7/samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaInputDescriptor.java -- diff --git

[5/5] samza git commit: SAMZA-1804: System and Stream Descriptors

2018-08-27 Thread pmaheshwari
SAMZA-1804: System and Stream Descriptors Design details: https://cwiki.apache.org/confluence/display/SAMZA/SEP-14%3A+System+and+Stream+Descriptors Author: Prateek Maheshwari Reviewers: Yi Pan , Cameron Lee Closes #603 from prateekm/stream-descriptor Project:

samza git commit: SAMZA-1782: Making getTableSpecs API in TableConfigGenerator util class public

2018-07-24 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master fa56b15dc -> 025f61710 SAMZA-1782: Making getTableSpecs API in TableConfigGenerator util class public Author: Aditya Toomula Reviewers: Prateek Maheshwari Closes #577 from atoomula/table3 Project:

samza git commit: Side inputs: Bug fixes

2018-07-24 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 7a8927de5 -> de5d92e51 Side inputs: Bug fixes - Use the correct regex in Util#getSystemStreamFromNameOrId - Handle end of stream during dispatch of message to side input processor - Fix validation to check for presence of side input

samza git commit: SAMZA-1677: Make httpcore and httpclient dependencies consistent

2018-07-25 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master de5d92e51 -> c98d7b0d8 SAMZA-1677: Make httpcore and httpclient dependencies consistent Samza currently depends on httpclient 4.5.2 and httpcore 4.4.5. However, httpclient 4.5.2 also has a direct dependency on httpcore 4.4.4, which is not

samza git commit: SAMZA-1773: Side inputs for local stores

2018-07-24 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 9f8d593c4 -> fa56b15dc SAMZA-1773: Side inputs for local stores prateekm vjagadish Please take a look. I will update the PR with the unit tests for SideInputStorageManager and the util functions. Author: Bharath Kumarasubramanian

samza git commit: SAMZA-1786: Introduce metadata store abstraction.

2018-08-29 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 46685406f -> 9eadfa059 SAMZA-1786: Introduce metadata store abstraction. As a part of SEP-11, this patch adds MetadataStore interface to store task and container locality in both yarn and standalone deployment models. Please refer to

samza git commit: Upgrade to latest event hub version (1.0.1)

2018-04-10 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 8e000f398 -> fda1e37d0 Upgrade to latest event hub version (1.0.1) * Upgrade to the latest event hub version 1.0.1 * Adding configs for prefetchCount and maxEventPerPoll * Fix the high cpu usage issue in SamzaHistogram * Fixing a race

samza git commit: SAMZA-1584: Improve logging in StreamProcessor.

2018-04-12 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 3895a9070 -> 76de840c7 SAMZA-1584: Improve logging in StreamProcessor. Add the processorID in the log lines wherever necessary(since we support running multiple stream applications in a JVM) and improving logging in general in

[3/3] samza git commit: Misc. Util cleanup

2018-04-18 Thread pmaheshwari
Misc. Util cleanup Major changes: 1. Broke up 'Util' class into multiple classes: 'FileUtil', 'HttpUtil', 'CoordinatorStreamUtil'. 2. Consolidated some Util classes: MathUtil, ScalaJavaUtil 3. Removed redundant Util classes: ClassloaderUtil, ScalaToJavaUtil 4. Renamed some Util classes for

[2/3] samza git commit: Misc. Util cleanup

2018-04-18 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/5d73ecda/samza-core/src/main/scala/org/apache/samza/util/FileUtil.scala -- diff --git a/samza-core/src/main/scala/org/apache/samza/util/FileUtil.scala

[1/3] samza git commit: Misc. Util cleanup

2018-04-18 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master bca978eb2 -> 5d73ecdad http://git-wip-us.apache.org/repos/asf/samza/blob/5d73ecda/samza-rest/src/main/java/org/apache/samza/rest/proxy/task/SamzaTaskProxy.java -- diff

samza git commit: SAMZA-1686: Set finite operation timeout when creating zkClient.

2018-04-24 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master e63d27c0c -> c89841366 SAMZA-1686: Set finite operation timeout when creating zkClient. Currently zkClient is created with operationRetryTimeOut of -1. This causes zkClient to retry indefinitely in case of irrecoverable exceptions thereby

samza git commit: SAMZA-1478: Delete unneeded data from intermediate Kafka topic on offset commit

2018-04-18 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 5d73ecdad -> 3cc2a05f7 SAMZA-1478: Delete unneeded data from intermediate Kafka topic on offset commit Author: Dong Lin Author: Dong Lin Reviewers: Prateek Maheshwari ,

samza git commit: SAMZA-1630: Move thread dump from stdout to logs

2018-03-30 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master e58e3cadc -> aba739307 SAMZA-1630: Move thread dump from stdout to logs Author: Prateek Maheshwari Reviewers: Jagadish Venkatraman Closes #462 from prateekm/thread-dump-on-timeout

samza git commit: SAMZA-1630: Log a thread dump on timeouts

2018-03-28 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master c99e57133 -> 13ff09024 SAMZA-1630: Log a thread dump on timeouts Would be useful to get a thread dump on timeouts, e.g. for AsyncStreamTask callback timeout, container shutdown timeout, heartbeat monitor graceful shutdown timeout etc.

samza git commit: SAMZA-1631: Improve logging on Task callback timeout

2018-03-28 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 5431350b7 -> c99e57133 SAMZA-1631: Improve logging on Task callback timeout Author: Prateek Maheshwari Reviewers: Jacob Maes Closes #461 from prateekm/task-callback-logging Project:

samza git commit: ScheduleAfterDebounceTime should catch all Throwable to avoid losing unhandled Errors

2018-03-26 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 272aa32e3 -> b0603c3c7 ScheduleAfterDebounceTime should catch all Throwable to avoid losing unhandled Errors If an Action executed in the scheduler throws an Error (or other Throwable?) besides Exception, it is silently lost since the

samza git commit: SAMZA-1608 : Add hidden config to enable explicit stream creation in StreamAppender due to bug.

2018-03-16 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 924a78dab -> 30c6a89b3 SAMZA-1608 : Add hidden config to enable explicit stream creation in StreamAppender due to bug. Due to a intermittent bug that causes the explicit stream creation in `StreamAppender` to hang, a hidden config is

samza git commit: SAMZA-1845: Require setting serde for intermediate streams in High Level API

2018-10-05 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master ba03d599e -> d48cf5f74 SAMZA-1845: Require setting serde for intermediate streams in High Level API Author: Prateek Maheshwari Reviewers: Yi Pan , Jagadish Venkataraman Closes #693 from prateekm/intermediate-stream-serde Project:

samza git commit: SAMZA-1933: Fix NPE in LocalityManager.

2018-10-05 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master d48cf5f74 -> b101ff91a SAMZA-1933: Fix NPE in LocalityManager. Author: Shanthoosh Venkataraman Reviewers: Prateek Maheshwari Closes #684 from shanthoosh/fix_NPE_in_task_assignment_manager Project:

[1/2] samza git commit: SAMZA-1856: eventhub descriptors

2018-10-11 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 482c6a227 -> da88096a0 http://git-wip-us.apache.org/repos/asf/samza/blob/da88096a/samza-azure/src/test/java/org/apache/samza/system/eventhub/TestEventHubsSystemDescriptor.java

[2/2] samza git commit: SAMZA-1856: eventhub descriptors

2018-10-11 Thread pmaheshwari
SAMZA-1856: eventhub descriptors prateekm lhaiesp Add system descriptors and input/output descriptors for Eventhubs. We should make an effort to deprecate the `systems.%s.stream.list`. Author: Daniel Chen Reviewers: Prateek Maheshwari Closes #696 from dxichen/eventhub-system-descriptors

samza git commit: SAMZA-1900: Add restore logging info

2018-10-11 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master da88096a0 -> 3c78e06ac SAMZA-1900: Add restore logging info prateekm let me know if there are other places that logging should also be improved in this patch. Author: Daniel Chen Author: Daniel Chen Reviewers: Prateek Maheshwari

samza git commit: Fix broken tests on master.

2018-10-12 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 17113e163 -> 954fb25e7 Fix broken tests on master. Author: Prateek Maheshwari Reviewers: Prateek Maheshwari Closes #722 from prateekm/test-fixes Project: http://git-wip-us.apache.org/repos/asf/samza/repo Commit:

samza git commit: Use different paths for logged and non-logged stores in test runner

2018-10-12 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 0b0a0cabf -> 9a5094d68 Use different paths for logged and non-logged stores in test runner We have scattered logic to determine the store paths between SamzaContainer and StorageManager. Ideally, it would nice to consolidate them into

[07/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-core/src/main/java/org/apache/samza/runtime/LocalApplicationRunner.java -- diff --git

[01/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 9a5094d68 -> 74675cea5 http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-test/src/test/java/org/apache/samza/test/operator/RepartitionWindowApp.java -- diff

[08/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-core/src/main/java/org/apache/samza/application/LegacyTaskApplication.java -- diff --git

[10/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-api/src/test/java/org/apache/samza/operators/descriptors/TestExpandingInputDescriptor.java -- diff --git

[05/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-core/src/test/java/org/apache/samza/application/descriptors/TestStreamApplicationDescriptorImpl.java -- diff --git

[02/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-kv/src/test/java/org/apache/samza/storage/kv/TestBaseLocalStoreBackedTableProvider.java -- diff --git

[09/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-azure/src/main/java/org/apache/samza/system/eventhub/EventHubsOutputDescriptor.java -- diff --git

[03/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-kv-inmemory/src/test/java/org/apache/samza/storage/kv/inmemory/descriptors/TestInMemoryTableProvider.java -- diff --git

[12/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
Consolidating package names for System, Stream, Application and Table descriptors. Everything in this PR is either: 1. A package name change and a corresponding file move. 2. Javadoc changes to use FQN in link tags to fix checkstyle complaints about unused imports, and corresponding fixes to

[04/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaInputDescriptor.java -- diff --git

[06/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-core/src/main/java/org/apache/samza/table/remote/descriptors/RemoteTableDescriptor.java -- diff --git

[11/12] samza git commit: Consolidating package names for System, Stream, Application and Table descriptors.

2018-10-12 Thread pmaheshwari
http://git-wip-us.apache.org/repos/asf/samza/blob/74675cea/samza-api/src/main/java/org/apache/samza/operators/descriptors/base/system/SimpleInputDescriptorProvider.java -- diff --git

[1/2] samza git commit: Application Descriptor implementation cleanup.

2018-10-12 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 74675cea5 -> ef3599721 http://git-wip-us.apache.org/repos/asf/samza/blob/ef359972/samza-core/src/test/java/org/apache/samza/operators/spec/TestPartitionByOperatorSpec.java

[2/2] samza git commit: Application Descriptor implementation cleanup.

2018-10-12 Thread pmaheshwari
Application Descriptor implementation cleanup. Major changes: 1. Move withDefaultSystem to the base ApplicationDescriptor interface so that it can be used for TaskApplication as well. 2. Move some of the common state (input/output/system/table descriptors) and corresponding helper methods from

samza git commit: SAMZA-1771: Test Framework support for stateful testing and assertions over state

2018-10-11 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master cc490ea89 -> 5c5afb82a SAMZA-1771: Test Framework support for stateful testing and assertions over state - Gives StateAssert for stateful assertions - Gives examples of Testing stateful jobs for TaskApplication and StreamApplication

samza git commit: System and Stream Descriptor API cleanup.

2018-10-11 Thread pmaheshwari
Repository: samza Updated Branches: refs/heads/master 9903539a2 -> 378fd8a76 System and Stream Descriptor API cleanup. Major changes: 1. Made withPhysicalName(String) a protected method in StreamDescriptor. The primary reason to set the physical name is to use stream names with special

  1   2   3   >