[jira] [Commented] (APEXMALHAR-2276) ManagedState: value of a key does not get over-written in the same time bucket

[ https://issues.apache.org/jira/browse/APEXMALHAR-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546598#comment-15546598 ] Siyuan Hua commented on APEXMALHAR-2276: Time will be mapped to a time bucket anyways,

[jira] [Commented] (APEXMALHAR-2254) File input operator is not idempotent with closing files on replay

[ https://issues.apache.org/jira/browse/APEXMALHAR-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546572#comment-15546572 ] Munagala V. Ramanath commented on APEXMALHAR-2254: -- Here is a list of other JIRAs

[jira] [Commented] (APEXMALHAR-2276) ManagedState: value of a key does not get over-written in the same time bucket

[ https://issues.apache.org/jira/browse/APEXMALHAR-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546592#comment-15546592 ] Chandni Singh commented on APEXMALHAR-2276: --- As [~chaithu] pointed out, we need to compare

[jira] [Commented] (APEXMALHAR-2220) Move the FunctionOperator to Malhar library

[ https://issues.apache.org/jira/browse/APEXMALHAR-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545968#comment-15545968 ] Dongming Liang commented on APEXMALHAR-2220: [~hsy541] There seems to be a circular

[jira] [Comment Edited] (APEXMALHAR-2254) File input operator is not idempotent with closing files on replay

[ https://issues.apache.org/jira/browse/APEXMALHAR-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546182#comment-15546182 ] Munagala V. Ramanath edited comment on APEXMALHAR-2254 at 10/4/16 6:45 PM:

[jira] [Commented] (APEXMALHAR-2254) File input operator is not idempotent with closing files on replay

[ https://issues.apache.org/jira/browse/APEXMALHAR-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546182#comment-15546182 ] Munagala V. Ramanath commented on APEXMALHAR-2254: -- It would be useful to define

[jira] [Commented] (APEXMALHAR-2276) ManagedState: value of a key does not get over-written in the same time bucket

[ https://issues.apache.org/jira/browse/APEXMALHAR-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546623#comment-15546623 ] Chandni Singh commented on APEXMALHAR-2276: --- Yes time gets mapped to a time bucket, however

[jira] [Created] (APEXCORE-547) Strict & lenient Physical Plan checking

Sandesh created APEXCORE-547: Summary: Strict & lenient Physical Plan checking Key: APEXCORE-547 URL: https://issues.apache.org/jira/browse/APEXCORE-547 Project: Apache Apex Core Issue Type:

[jira] [Updated] (APEXCORE-547) Strict & lenient Physical Plan checking

[ https://issues.apache.org/jira/browse/APEXCORE-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh updated APEXCORE-547: - Description: This will need a bigger discussion, filing here so that it can be tracked. Example:

[jira] [Issue Comment Deleted] (APEXMALHAR-2276) ManagedState: value of a key does not get over-written in the same time bucket

[ https://issues.apache.org/jira/browse/APEXMALHAR-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated APEXMALHAR-2276: -- Comment: was deleted (was: Yes time gets mapped to a time bucket, however the

Re: [VOTE] Hadoop upgrade

+1 for 2.6.x On Tuesday, October 4, 2016, David Yan wrote: > Hi all, > > Thomas created this ticket for upgrading our Hadoop dependency version a > couple weeks ago: > > https://issues.apache.org/jira/browse/APEXCORE-536 > > We'd like to get the ball rolling and would

Re: Generic Malhar operator to get formatted data.

Hi Priyanka, Yes it is template engine. Initial use case to target is to generate automated emails using template for alerts, logging might be also for monitoring. Also to convert POJO's or Map or List in XML or JSON format to further absorb by other operators depending on templates. Thanks,

Re: Writing to external systems in reconciled fashion

Hi Priyanka, tuples stored in HFile will not be replayed in same order at the output as HFile will save tuples in different order. If order is important then you could use org.apache.apex.malhar.lib.wal.FileSystemWAL which is like a on-disk queue. Or your could directly use more high level

Re: [VOTE] Hadoop upgrade

+1 for 2.6 - Tushar. On Tue, Oct 4, 2016 at 3:18 PM, Pradeep A. Dalvi wrote: > +1 for 2.6.x > > On Tuesday, October 4, 2016, David Yan wrote: > >> Hi all, >> >> Thomas created this ticket for upgrading our Hadoop dependency version a >> couple weeks

[jira] [Assigned] (APEXMALHAR-2270) AbstractFileInputOperator: During replay, inputStream should skip tuples

[ https://issues.apache.org/jira/browse/APEXMALHAR-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Zhang reassigned APEXMALHAR-2270: -- Assignee: Matt Zhang > AbstractFileInputOperator: During replay, inputStream

[jira] [Assigned] (APEXMALHAR-2263) Offsets in AbstractFileInputOperator should be long rather than int

[ https://issues.apache.org/jira/browse/APEXMALHAR-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Zhang reassigned APEXMALHAR-2263: -- Assignee: Matt Zhang > Offsets in AbstractFileInputOperator should be long rather

[jira] [Assigned] (APEXMALHAR-2269) AbstractFileInputOperator: During replay, IO errors not handled

[ https://issues.apache.org/jira/browse/APEXMALHAR-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Zhang reassigned APEXMALHAR-2269: -- Assignee: Matt Zhang > AbstractFileInputOperator: During replay, IO errors not

[GitHub] apex-core pull request #404: APEXCORE-536 #resolve Upgrade Hadoop dependency...

GitHub user davidyan74 reopened a pull request: https://github.com/apache/apex-core/pull/404 APEXCORE-536 #resolve Upgrade Hadoop dependency version from 2.2.0 to 2.6.0 You can merge this pull request into a Git repository by running: $ git pull

[jira] [Commented] (APEXCORE-536) Upgrade Hadoop dependency

[ https://issues.apache.org/jira/browse/APEXCORE-536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547170#comment-15547170 ] ASF GitHub Bot commented on APEXCORE-536: - Github user davidyan74 closed the pull request at:

[jira] [Commented] (APEXCORE-536) Upgrade Hadoop dependency

[ https://issues.apache.org/jira/browse/APEXCORE-536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547171#comment-15547171 ] ASF GitHub Bot commented on APEXCORE-536: - GitHub user davidyan74 reopened a pull request:

[jira] [Assigned] (APEXCORE-536) Upgrade Hadoop dependency

[ https://issues.apache.org/jira/browse/APEXCORE-536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Yan reassigned APEXCORE-536: -- Assignee: David Yan > Upgrade Hadoop dependency > - > >

[GitHub] apex-core pull request #404: APEXCORE-536 #resolve Upgrade Hadoop dependency...

Github user davidyan74 closed the pull request at: https://github.com/apache/apex-core/pull/404 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (APEXCORE-536) Upgrade Hadoop dependency

[ https://issues.apache.org/jira/browse/APEXCORE-536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547010#comment-15547010 ] ASF GitHub Bot commented on APEXCORE-536: - GitHub user davidyan74 opened a pull request:

[GitHub] apex-core pull request #404: APEXCORE-536 #resolve Upgrade Hadoop dependency...

GitHub user davidyan74 opened a pull request: https://github.com/apache/apex-core/pull/404 APEXCORE-536 #resolve Upgrade Hadoop dependency version from 2.2.0 to 2.6.0 You can merge this pull request into a Git repository by running: $ git pull

Re: [VOTE] Hadoop upgrade

+1 for 2.6.x Vlad On 10/3/16 13:47, David Yan wrote: Hi all, Thomas created this ticket for upgrading our Hadoop dependency version a couple weeks ago: https://issues.apache.org/jira/browse/APEXCORE-536 We'd like to get the ball rolling and would like to take a vote from the community which

Re: [VOTE] Hadoop upgrade

+1 for 2.6.x Ram On Mon, Oct 3, 2016 at 1:47 PM, David Yan wrote: > Hi all, > > Thomas created this ticket for upgrading our Hadoop dependency version a > couple weeks ago: > > https://issues.apache.org/jira/browse/APEXCORE-536 > > We'd like to get the ball rolling and

[jira] [Commented] (APEXCORE-505) setup and activate calls in operator block heartbeat loop in container

[ https://issues.apache.org/jira/browse/APEXCORE-505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547380#comment-15547380 ] ASF GitHub Bot commented on APEXCORE-505: - GitHub user sandeshh opened a pull request:

[GitHub] apex-core pull request #405: APEXCORE-505 Heartbeat loop was blocked waiting...

GitHub user sandeshh opened a pull request: https://github.com/apache/apex-core/pull/405 APEXCORE-505 Heartbeat loop was blocked waiting for operator activati… …on, the reason for this is that Stream activation(Only BufferServerSubscriber and WindowGenerator) waits for operator

[jira] [Commented] (APEXCORE-505) setup and activate calls in operator block heartbeat loop in container

[ https://issues.apache.org/jira/browse/APEXCORE-505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547463#comment-15547463 ] ASF GitHub Bot commented on APEXCORE-505: - GitHub user sandeshh reopened a pull request:

[jira] [Commented] (APEXCORE-505) setup and activate calls in operator block heartbeat loop in container

[ https://issues.apache.org/jira/browse/APEXCORE-505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547462#comment-15547462 ] ASF GitHub Bot commented on APEXCORE-505: - Github user sandeshh closed the pull request at:

[GitHub] apex-core pull request #405: APEXCORE-505 Heartbeat loop was blocked waiting...

GitHub user sandeshh reopened a pull request: https://github.com/apache/apex-core/pull/405 APEXCORE-505 Heartbeat loop was blocked waiting for operator activati… …on, the reason for this is that Stream activation(Only BufferServerSubscriber and WindowGenerator) waits for

[GitHub] apex-core pull request #404: APEXCORE-536 #resolve Upgrade Hadoop dependency...

GitHub user davidyan74 reopened a pull request: https://github.com/apache/apex-core/pull/404 APEXCORE-536 #resolve Upgrade Hadoop dependency version from 2.2.0 to 2.6.0 You can merge this pull request into a Git repository by running: $ git pull

[jira] [Commented] (APEXCORE-536) Upgrade Hadoop dependency

[ https://issues.apache.org/jira/browse/APEXCORE-536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547333#comment-15547333 ] ASF GitHub Bot commented on APEXCORE-536: - GitHub user davidyan74 reopened a pull request:

[GitHub] apex-core pull request #404: APEXCORE-536 #resolve Upgrade Hadoop dependency...

Github user davidyan74 closed the pull request at: https://github.com/apache/apex-core/pull/404 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (APEXCORE-536) Upgrade Hadoop dependency

[ https://issues.apache.org/jira/browse/APEXCORE-536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547313#comment-15547313 ] ASF GitHub Bot commented on APEXCORE-536: - Github user davidyan74 closed the pull request at:

[jira] [Reopened] (APEXMALHAR-2276) ManagedState: value of a key does not get over-written in the same time bucket

[ https://issues.apache.org/jira/browse/APEXMALHAR-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh reopened APEXMALHAR-2276: --- > ManagedState: value of a key does not get over-written in the same time bucket >

Re: Fixed Width Record Parser

Hi All, Thank you for thee feedback. I will use univocity for *parsing (only) , *will do the type checking/validation manually. Input schema is similar to that of CSV so will have to create another base class Schema (having common elements of Delimited and Fixed width Schema)and Delimited and

Re: Fixed Width Record Parser

Hi Hitesh, I agree with Chinmay. -1 for creating our own library. +1 for using Univocity. For input schema, I suggest we use the same one as used by Delimited Parser. We would need to add fields to accept padding character, startingCharacterPosition and endingCharacterPosition. To construct the

Re: [VOTE] Hadoop upgrade

+1 for 2.6 On Tue, Oct 4, 2016 at 6:19 AM, Bhupesh Chawda wrote: > +1 for 2.6 > > ~ Bhupesh > > On Oct 4, 2016 4:14 AM, "Siyuan Hua" wrote: > > > +1 for 2.6.x > > > > On Mon, Oct 3, 2016 at 3:41 PM, Pramod Immaneni > >

Generic Malhar operator to get formatted data.

Hi Folks, Planning to write an malhar operator which will take Map or POJO as input and provide output as formatted data string as per specified template data. Use Cases: Get data in output data format like XML, HTML etc. Generate automated emails etc. Generate configuration files also source