Re: Megh operator library

2016-09-26 Thread Pramod Immaneni
Added a section for flume based on the feedback. Thanks On Mon, Sep 26, 2016 at 8:51 AM, Pramod Immaneni wrote: > Hi Thomas, > > My responses are inline > > On Sun, Sep 25, 2016 at 11:39 AM, Thomas Weise > wrote: > >> Thanks for putting it

[GitHub] apex-site pull request #51: Mentioned enforcing no trailing whitespaces by c...

2016-09-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-site/pull/51 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] apex-site pull request #51: Mentioned enforcing no trailing whitespaces by c...

2016-09-26 Thread vrozov
GitHub user vrozov opened a pull request: https://github.com/apache/apex-site/pull/51 Mentioned enforcing no trailing whitespaces by checkstyle. @tweise Please review You can merge this pull request into a Git repository by running: $ git pull

[GitHub] apex-core pull request #398: Add pointer for removing trailing whitespace in...

2016-09-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-core/pull/398 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (APEXMALHAR-2271) Special SpillableSetMultimap<K, Window.SessionWindow> handling in ManagedState in SpillableSessionWindowedStorage

2016-09-26 Thread David Yan (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Yan updated APEXMALHAR-2271: -- Description: The problem with this data structure is that the key is not time based. This

[jira] [Created] (APEXMALHAR-2271) Special SpillableSetMultimap<K, Window.SessionWindow> handling in ManagedState in SpillableSessionWindowedStorage

2016-09-26 Thread David Yan (JIRA)
David Yan created APEXMALHAR-2271: - Summary: Special SpillableSetMultimap handling in ManagedState in SpillableSessionWindowedStorage Key: APEXMALHAR-2271 URL:

[GitHub] apex-core pull request #398: Add pointer for removing trailing whitespace in...

2016-09-26 Thread tweise
GitHub user tweise opened a pull request: https://github.com/apache/apex-core/pull/398 Add pointer for removing trailing whitespace in Eclipse. @vrozov please merge You can merge this pull request into a Git repository by running: $ git pull https://github.com/tweise/apex-core

[GitHub] apex-malhar pull request #429: Apexmalhar 2181.cassandra upsert operator

2016-09-26 Thread ananthc
GitHub user ananthc opened a pull request: https://github.com/apache/apex-malhar/pull/429 Apexmalhar 2181.cassandra upsert operator @sanjaypujare , @ashwinchandrap : Guess the features that we targetted for are passing all unit tests in my local environments and hence ready for a

[jira] [Commented] (APEXMALHAR-2190) Use reusable buffer to serial spillable data structure

2016-09-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524244#comment-15524244 ] ASF GitHub Bot commented on APEXMALHAR-2190: Github user brightchen closed the pull

[GitHub] apex-malhar pull request #404: APEXMALHAR-2190 #resolve #comment Use reusabl...

2016-09-26 Thread brightchen
Github user brightchen closed the pull request at: https://github.com/apache/apex-malhar/pull/404 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (APEXMALHAR-2190) Use reusable buffer to serial spillable data structure

2016-09-26 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524245#comment-15524245 ] ASF GitHub Bot commented on APEXMALHAR-2190: GitHub user brightchen reopened a pull

[GitHub] apex-malhar pull request #404: APEXMALHAR-2190 #resolve #comment Use reusabl...

2016-09-26 Thread brightchen
GitHub user brightchen reopened a pull request: https://github.com/apache/apex-malhar/pull/404 APEXMALHAR-2190 #resolve #comment Use reusable buffer to serial spill… …able data structure You can merge this pull request into a Git repository by running: $ git pull

Re: Watermark generation in Windowed Operators

2016-09-26 Thread David Yan
Actually on a second thought, the join accumulation interface for the non-keyed join operator can support theta joins if you have the proper accumulation implementation for that. David On Mon, Sep 26, 2016 at 12:42 PM, David Yan wrote: > Chinmay, > > Just to clarify, the

[jira] [Assigned] (APEXCORE-505) setup and activate calls in operator block heartbeat loop in container

2016-09-26 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh reassigned APEXCORE-505: Assignee: Sandesh > setup and activate calls in operator block heartbeat loop in container >

[jira] [Created] (APEXMALHAR-2270) AbstractFileInputOperator: During replay, inputStream should skip tuples

2016-09-26 Thread Munagala V. Ramanath (JIRA)
Munagala V. Ramanath created APEXMALHAR-2270: Summary: AbstractFileInputOperator: During replay, inputStream should skip tuples Key: APEXMALHAR-2270 URL:

[jira] [Created] (APEXMALHAR-2269) AbstractFileInputOperator: During replay, IO errors not handled

2016-09-26 Thread Munagala V. Ramanath (JIRA)
Munagala V. Ramanath created APEXMALHAR-2269: Summary: AbstractFileInputOperator: During replay, IO errors not handled Key: APEXMALHAR-2269 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2269

[jira] [Commented] (APEXCORE-510) Enforce DefaultOutputPort.emit() or Sink.put() thread affinity

2016-09-26 Thread Sanjay M Pujare (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523550#comment-15523550 ] Sanjay M Pujare commented on APEXCORE-510: -- DefaultOutputPort.setup is overridden in multiple

Re: Megh operator library

2016-09-26 Thread Pramod Immaneni
Hi Thomas, My responses are inline On Sun, Sep 25, 2016 at 11:39 AM, Thomas Weise wrote: > Thanks for putting it together. It looks like there are really only 2 > operators? > There were others but looked like they were already good implementations or alternatives for

Re: checkpoint statistics

2016-09-26 Thread Venkatesh Kottapalli
+1 for this feature. The size and time to checkpoint the state at operator level will help in tuning and understanding the overheads if any. -Venkatesh. > On Sep 25, 2016, at 10:56 PM, Chinmay Kolhatkar > wrote: > > +1. very useful feature. We should also provide