Re: Parquet Writer Operator

2016-06-14 Thread Devendra Tagare
Hi All, We can focus on the below 2 problems, 1.Avoid the small files problem which could arise due a flush at every endWindow, since there wouldn't be significant data in a window. 2.Fault Tolerance. *Proposal* : Create a module in which there are 2 operators, *Operator 1 :

[jira] [Commented] (APEXCORE-470) New Api for setting the attribute on the operator ( setOperatorAttribute )

2016-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330752#comment-15330752 ] ASF GitHub Bot commented on APEXCORE-470: - Github user PramodSSImmaneni commented on a diff in

[GitHub] apex-core pull request #349: APEXCORE-470 New API in DAG - setOperatorAttrib...

2016-06-14 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on a diff in the pull request: https://github.com/apache/apex-core/pull/349#discussion_r67067938 --- Diff: api/src/main/java/com/datatorrent/api/DAG.java --- @@ -236,11 +236,22 @@ public abstract void setAttribute(Attribute key, T

[jira] [Commented] (APEXCORE-470) New Api for setting the attribute on the operator ( setOperatorAttribute )

2016-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330750#comment-15330750 ] ASF GitHub Bot commented on APEXCORE-470: - Github user PramodSSImmaneni commented on a diff in

[jira] [Commented] (APEXCORE-470) New Api for setting the attribute on the operator ( setOperatorAttribute )

2016-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330747#comment-15330747 ] ASF GitHub Bot commented on APEXCORE-470: - Github user PramodSSImmaneni commented on a diff in

[GitHub] apex-core pull request #349: APEXCORE-470 New API in DAG - setOperatorAttrib...

2016-06-14 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on a diff in the pull request: https://github.com/apache/apex-core/pull/349#discussion_r67067785 --- Diff: api/src/main/java/com/datatorrent/api/DAG.java --- @@ -236,11 +236,22 @@ public abstract void setAttribute(Attribute key, T

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330733#comment-15330733 ] ASF GitHub Bot commented on APEXCORE-222: - Github user PramodSSImmaneni commented on a diff in

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-14 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r67067166 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java --- @@ -769,7 +769,11 @@ public void

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330718#comment-15330718 ] ASF GitHub Bot commented on APEXCORE-222: - Github user sandeshh commented on a diff in the pull

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-14 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r67065992 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java --- @@ -769,7 +769,11 @@ public void

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-14 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r67063171 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java --- @@ -769,7 +769,11 @@ public void

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330672#comment-15330672 ] ASF GitHub Bot commented on APEXCORE-222: - Github user sandeshh commented on a diff in the pull

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-14 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r67062269 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java --- @@ -769,7 +769,11 @@ public void

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330644#comment-15330644 ] ASF GitHub Bot commented on APEXCORE-222: - Github user PramodSSImmaneni commented on a diff in

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-14 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r67060191 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java --- @@ -769,7 +769,11 @@ public void

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-14 Thread sandeshh
GitHub user sandeshh reopened a pull request: https://github.com/apache/apex-core/pull/350 APEXCORE-222 Purging the stale data present in the bufferserver, dire… …ctly from the streaming container. You can merge this pull request into a Git repository by running: $ git

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330620#comment-15330620 ] ASF GitHub Bot commented on APEXCORE-222: - Github user sandeshh closed the pull request at:

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330621#comment-15330621 ] ASF GitHub Bot commented on APEXCORE-222: - GitHub user sandeshh reopened a pull request:

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-14 Thread sandeshh
Github user sandeshh closed the pull request at: https://github.com/apache/apex-core/pull/350 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Created] (APEXCORE-472) Enhance Kafka 0.9 input operator to support starting from custom offsets in multiple partitions

2016-06-14 Thread Ananth (JIRA)
Ananth created APEXCORE-472: --- Summary: Enhance Kafka 0.9 input operator to support starting from custom offsets in multiple partitions Key: APEXCORE-472 URL: https://issues.apache.org/jira/browse/APEXCORE-472

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-14 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329626#comment-15329626 ] Sandesh commented on APEXCORE-222: -- Here is the change for fixing this issue,

[jira] [Commented] (APEXMALHAR-2117) JDBCLoader for Enricher should generate default queryStmt even when value is empty string

2016-06-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328991#comment-15328991 ] ASF GitHub Bot commented on APEXMALHAR-2117: Github user asfgit closed the pull request

[GitHub] apex-malhar pull request #317: APEXMALHAR-2117 Generate default queryStmt ev...

2016-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/317 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature