[GitHub] [samza] rmatharu commented on issue #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
rmatharu commented on issue #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#issuecomment-503402564 @vjagadish1989 addressed all comments. This is an au

[GitHub] [samza] rmatharu commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
rmatharu commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295105359 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManager.ja

[GitHub] [samza] rmatharu commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
rmatharu commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295105368 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManager.ja

[jira] [Created] (SAMZA-2257) Simplify checkSideInputCaughtup logic in ContainerStorageManager

2019-06-18 Thread Rayman (JIRA)
Rayman created SAMZA-2257: - Summary: Simplify checkSideInputCaughtup logic in ContainerStorageManager Key: SAMZA-2257 URL: https://issues.apache.org/jira/browse/SAMZA-2257 Project: Samza Issue Type:

[GitHub] [samza] rmatharu commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
rmatharu commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295105359 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManager.ja

[GitHub] [samza] vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295084685 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManag

[GitHub] [samza] vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295083551 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManag

[GitHub] [samza] vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295086283 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManag

[GitHub] [samza] vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295078168 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManag

[GitHub] [samza] vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295084978 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManag

[GitHub] [samza] vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295084357 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManag

[GitHub] [samza] vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295086657 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManag

[GitHub] [samza] vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
vjagadish1989 commented on a change in pull request #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#discussion_r295077506 ## File path: samza-core/src/main/java/org/apache/samza/storage/TaskSideInputStorageMa

[GitHub] [samza] rmatharu commented on issue #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
rmatharu commented on issue #1086: SAMZA-2256: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#issuecomment-503364705 done This is an automated message from the Apache Gi

[jira] [Created] (SAMZA-2256) Hotstandby fix for exception while reading side inputs

2019-06-18 Thread Rayman (JIRA)
Rayman created SAMZA-2256: - Summary: Hotstandby fix for exception while reading side inputs Key: SAMZA-2256 URL: https://issues.apache.org/jira/browse/SAMZA-2256 Project: Samza Issue Type: Improvemen

[GitHub] [samza] shanthoosh edited a comment on issue #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
shanthoosh edited a comment on issue #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#issuecomment-503360594 @sborya I addressed both of your comments. Can you please take a look when you get a chance. -

[GitHub] [samza] shanthoosh commented on issue #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
shanthoosh commented on issue #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#issuecomment-503360594 @sborya I addressed both of your comments. Can you please take a look. ---

[GitHub] [samza] shanthoosh edited a comment on issue #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
shanthoosh edited a comment on issue #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#issuecomment-503359306 @prateekm Added checksum to the end of metadata-value in `ZkMetadataStore`. It is used for verification during read an

[GitHub] [samza] shanthoosh commented on issue #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
shanthoosh commented on issue #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#issuecomment-503359306 @prateekm Added checksum to the end of metadata-value in `ZkMetadataStore`. It is used for verification during read and publis

[GitHub] [samza] vjagadish1989 commented on issue #1086: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
vjagadish1989 commented on issue #1086: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086#issuecomment-503348070 consider creating a jira and linking it? This is an automat

[GitHub] [samza] rmatharu opened a new pull request #1086: Hotstandby fix for exception while reading side inputs

2019-06-18 Thread GitBox
rmatharu opened a new pull request #1086: Hotstandby fix for exception while reading side inputs URL: https://github.com/apache/samza/pull/1086 This is an automated message from the Apache Git Service. To respond to the mes

[jira] [Updated] (SAMZA-2255) Optimize value writes in TaskSideInputStorageManager

2019-06-18 Thread Rayman (JIRA)
[ https://issues.apache.org/jira/browse/SAMZA-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rayman updated SAMZA-2255: -- Summary: Optimize value writes in TaskSideInputStorageManager (was: Smart value writes in TaskSideInputStorage

[jira] [Updated] (SAMZA-2255) Smart value writes in TaskSideInputStorageManager

2019-06-18 Thread Rayman (JIRA)
[ https://issues.apache.org/jira/browse/SAMZA-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rayman updated SAMZA-2255: -- Description: TaskSideInputStorageManager converts each IME into the desired set of records to be written by in

[jira] [Created] (SAMZA-2255) Smart value writes in TaskSideInputStorageManager

2019-06-18 Thread Rayman (JIRA)
Rayman created SAMZA-2255: - Summary: Smart value writes in TaskSideInputStorageManager Key: SAMZA-2255 URL: https://issues.apache.org/jira/browse/SAMZA-2255 Project: Samza Issue Type: Improvement

[samza] branch master updated: Samza-2253: Upgrading calcite to 1.19 (#1085)

2019-06-18 Thread srinivasulu
This is an automated email from the ASF dual-hosted git repository. srinivasulu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/samza.git The following commit(s) were added to refs/heads/master by this push: new 1908622 Samza-2253: Upgrading calcite to 1.

[jira] [Created] (SAMZA-2254) Add Relevant Calcite functions to Samza SQL operator table

2019-06-18 Thread Srinivasulu Punuru (JIRA)
Srinivasulu Punuru created SAMZA-2254: - Summary: Add Relevant Calcite functions to Samza SQL operator table Key: SAMZA-2254 URL: https://issues.apache.org/jira/browse/SAMZA-2254 Project: Samza

[GitHub] [samza] srinipunuru merged pull request #1085: Samza-2253: Upgrading calcite to 1.19

2019-06-18 Thread GitBox
srinipunuru merged pull request #1085: Samza-2253: Upgrading calcite to 1.19 URL: https://github.com/apache/samza/pull/1085 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [samza] atoomula commented on a change in pull request #1085: Samza-2253: Upgrading calcite to 1.19

2019-06-18 Thread GitBox
atoomula commented on a change in pull request #1085: Samza-2253: Upgrading calcite to 1.19 URL: https://github.com/apache/samza/pull/1085#discussion_r295033114 ## File path: samza-test/src/test/java/org/apache/samza/test/samzasql/TestSamzaSqlEndToEnd.java ## @@ -417,6 +4

[GitHub] [samza] srinipunuru commented on a change in pull request #1085: Samza-2253: Upgrading calcite to 1.19

2019-06-18 Thread GitBox
srinipunuru commented on a change in pull request #1085: Samza-2253: Upgrading calcite to 1.19 URL: https://github.com/apache/samza/pull/1085#discussion_r295032003 ## File path: samza-test/src/test/java/org/apache/samza/test/samzasql/TestSamzaSqlEndToEnd.java ## @@ -417,6

[GitHub] [samza] prateekm commented on issue #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
prateekm commented on issue #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#issuecomment-503311465 @shanthoosh If checksum is useful (as we think it is), why does it need to be a separate patch? Should be trivial to implement here?

[GitHub] [samza] dnishimura commented on issue #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
dnishimura commented on issue #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#issuecomment-503309799 In its current implementation, there are `ZkMetadataStore` and `CoordinatorStreamStore` metadata store types used in the `ZkJobCoor

[GitHub] [samza] atoomula commented on a change in pull request #1085: Samza-2253: Upgrading calcite to 1.19

2019-06-18 Thread GitBox
atoomula commented on a change in pull request #1085: Samza-2253: Upgrading calcite to 1.19 URL: https://github.com/apache/samza/pull/1085#discussion_r295025582 ## File path: samza-test/src/test/java/org/apache/samza/test/samzasql/TestSamzaSqlEndToEnd.java ## @@ -417,6 +4

[GitHub] [samza] shanthoosh commented on issue #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
shanthoosh commented on issue #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#issuecomment-503309436 @sborya 1. I've moved the chunking logic into the zookeeper metadata store implementation. 2. I've already replied to checks

[GitHub] [samza] prateekm commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
prateekm commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#discussion_r295022999 ## File path: samza-core/src/main/java/org/apache/samza/job/model/JobModelUtil.java ## @@ -19,20

[GitHub] [samza] shanthoosh commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
shanthoosh commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#discussion_r295021975 ## File path: samza-core/src/main/java/org/apache/samza/job/model/JobModelUtil.java ## @@ -19,2

[GitHub] [samza] shanthoosh commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
shanthoosh commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#discussion_r295021274 ## File path: samza-core/src/main/java/org/apache/samza/job/model/JobModelUtil.java ## @@ -57,6

[GitHub] [samza] srinipunuru opened a new pull request #1085: Samza-2253: Upgrading calcite to 1.19

2019-06-18 Thread GitBox
srinipunuru opened a new pull request #1085: Samza-2253: Upgrading calcite to 1.19 URL: https://github.com/apache/samza/pull/1085 Upgrading calcite to latest build to include bug fixes with compound boolean checks and others. Subqueries with flatten doesnt seem to work after this upgrade

[jira] [Created] (SAMZA-2253) Upgrade calcite to 1.19

2019-06-18 Thread Srinivasulu Punuru (JIRA)
Srinivasulu Punuru created SAMZA-2253: - Summary: Upgrade calcite to 1.19 Key: SAMZA-2253 URL: https://issues.apache.org/jira/browse/SAMZA-2253 Project: Samza Issue Type: Bug R

[jira] [Created] (SAMZA-2252) Samza SQL subquery with flatten doesn't work after calcite 1.19

2019-06-18 Thread Srinivasulu Punuru (JIRA)
Srinivasulu Punuru created SAMZA-2252: - Summary: Samza SQL subquery with flatten doesn't work after calcite 1.19 Key: SAMZA-2252 URL: https://issues.apache.org/jira/browse/SAMZA-2252 Project: Samz

[GitHub] [samza] sborya commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
sborya commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#discussion_r294936960 ## File path: samza-core/src/main/java/org/apache/samza/job/model/JobModelUtil.java ## @@ -57,6 +70

[GitHub] [samza] sborya commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
sborya commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#discussion_r294907655 ## File path: samza-core/src/main/java/org/apache/samza/job/model/JobModelUtil.java ## @@ -19,20 +1

[GitHub] [samza] sborya commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
sborya commented on a change in pull request #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#discussion_r294930395 ## File path: samza-core/src/main/java/org/apache/samza/job/model/JobModelUtil.java ## @@ -57,6 +70

[GitHub] [samza] sborya commented on issue #1079: SAMZA-2250: Support large job models in standalone.

2019-06-18 Thread GitBox
sborya commented on issue #1079: SAMZA-2250: Support large job models in standalone. URL: https://github.com/apache/samza/pull/1079#issuecomment-503235236 I am still not clear on the two questions brought up by @prateekm : why we don't need checksum and why the invocation of the chunking s