[jira] [Created] (APEXMALHAR-2346) DocumentBuilder.parse() should take InputSource as an argument instead of String

Hitesh Kapoor created APEXMALHAR-2346: - Summary: DocumentBuilder.parse() should take InputSource as an argument instead of String Key: APEXMALHAR-2346 URL:

[GitHub] apex-malhar pull request #502: review only: Windowed Operator benchmark

GitHub user brightchen opened a pull request: https://github.com/apache/apex-malhar/pull/502 review only: Windowed Operator benchmark You can merge this pull request into a Git repository by running: $ git pull https://github.com/brightchen/apex-malhar APEXMALHAR-2339

[jira] [Updated] (APEXMALHAR-2335) StateTracker can cause bucket memory leak

[ https://issues.apache.org/jira/browse/APEXMALHAR-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bright chen updated APEXMALHAR-2335: Summary: StateTracker can cause bucket memory leak (was: StateTracker has memory

[jira] [Updated] (APEXMALHAR-2335) StateTracker has memory leak

[ https://issues.apache.org/jira/browse/APEXMALHAR-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bright chen updated APEXMALHAR-2335: Summary: StateTracker has memory leak (was: Problems on StateTracker) > StateTracker

[jira] [Resolved] (APEXMALHAR-2321) Improve Buckets memory management

[ https://issues.apache.org/jira/browse/APEXMALHAR-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bright chen resolved APEXMALHAR-2321. - Resolution: Fixed see https://issues.apache.org/jira/browse/APEXMALHAR-2335 >

[jira] [Updated] (APEXMALHAR-2321) Improve Buckets memory management

[ https://issues.apache.org/jira/browse/APEXMALHAR-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bright chen updated APEXMALHAR-2321: Description: Currently buckets were managed as an array. Each bucket have memory

[jira] [Issue Comment Deleted] (APEXMALHAR-2335) Problems on StateTracker

[ https://issues.apache.org/jira/browse/APEXMALHAR-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bright chen updated APEXMALHAR-2335: Comment: was deleted (was: see commit on

[jira] [Closed] (APEXMALHAR-2331) StateTracker#bucketAccessed should add bucket to bucketAccessTimes

[ https://issues.apache.org/jira/browse/APEXMALHAR-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bright chen closed APEXMALHAR-2331. --- Resolution: Invalid The whole logic of bucketAccessed() changed. So this issue not valid

Re: Malhar release 3.6

Hi Thomas, We would like to finish the following tickets if possible for the 3.6.0 release: https://issues.apache.org/jira/browse/APEXMALHAR-2301 (ETA: 11/18) https://issues.apache.org/jira/browse/APEXMALHAR-2339 (ETA: 11/21) https://issues.apache.org/jira/browse/APEXMALHAR-2345 (ETA: 11/21)

Re: Proposing an operator for log parsing.

Check for pull request https://github.com/apache/apex-malhar/pull/396, This tries to use java regex with group syntax to populate fields of POJO. may be you can use some of the ideas from this pull request. On Fri, Nov 18, 2016 at 12:30 AM, Pradeep A. Dalvi wrote: > +1 for the

Re: Proposing an operator for log parsing.

+1 for the feature On Thu, 17 Nov 2016 at 16:56 Shraddha Jog wrote: > Dear community, > > We would like to add operator in malhar for parsing different types of > logs. > Idea of this operator is to read log data records of known formats such as > Syslog, common log,

[jira] [Commented] (APEXMALHAR-2342) Fix null pointer exception in AbstractFileOutputOperator setup

[ https://issues.apache.org/jira/browse/APEXMALHAR-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15674521#comment-15674521 ] Tushar Gosavi commented on APEXMALHAR-2342: --- please provide stack strace in the Jira for

[jira] [Resolved] (APEXMALHAR-2342) Fix null pointer exception in AbstractFileOutputOperator setup

[ https://issues.apache.org/jira/browse/APEXMALHAR-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tushar Gosavi resolved APEXMALHAR-2342. --- Resolution: Fixed Fix Version/s: 3.6.0 > Fix null pointer exception in

[jira] [Commented] (APEXMALHAR-2342) Fix null pointer exception in AbstractFileOutputOperator setup

[ https://issues.apache.org/jira/browse/APEXMALHAR-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15674516#comment-15674516 ] ASF GitHub Bot commented on APEXMALHAR-2342: Github user asfgit closed the pull request

[GitHub] apex-malhar pull request #496: APEXMALHAR-2342 Check file exists to prevent ...

Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/496 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Re: Visitor API for DAG

The code will execute before application master is launched, it is just one time activity during application startup. Few use cases I could think are - Operator validation/configuration validator jdbc operator could check if database is accessible with given credentials. file output operator

Re: Proposing an operator for log parsing.

+1 for the feature If possible may be you could include grok format for specifying log structure. https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html - Tushar. On Thu, Nov 17, 2016 at 11:48 PM, Sanjay Pujare wrote: > +1, I like this feature. > >

Re: Proposing an operator for log parsing.

+1, I like this feature. On 11/17/16, 3:26 AM, "Shraddha Jog" wrote: Dear community, We would like to add operator in malhar for parsing different types of logs. Idea of this operator is to read log data records of known formats such as

[jira] [Created] (APEXMALHAR-2345) Purge the time buckets from managed state for all time buckets that fall beyond the lateness horizon

David Yan created APEXMALHAR-2345: - Summary: Purge the time buckets from managed state for all time buckets that fall beyond the lateness horizon Key: APEXMALHAR-2345 URL:

Re: Visitor API for DAG

As Tushar mentions, properties/attributes can be injected from external sources. We've already had multiple questions on the mailing list asking how to do this. Ram On Thu, Nov 17, 2016 at 10:05 AM, Sanjay Pujare wrote: > There is a risk if the user written code blocks

Re: Visitor API for DAG

There is a risk if the user written code blocks the thread or crashes the process. What are the real life examples of this use case? On 11/17/16, 9:21 AM, "amol kekre" wrote: +1. Opening up the API for users to put in their own code is good. In general we should

[jira] [Updated] (APEXCORE-519) Add support for DIGEST enabled hadoop web services environment

[ https://issues.apache.org/jira/browse/APEXCORE-519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-519: Fix Version/s: 3.5.0 > Add support for DIGEST enabled hadoop web services environment >

[jira] [Updated] (APEXCORE-476) Window Id calculation in ToString method in SubscribeRequestTuple & GenericRequestTuple is wrong

[ https://issues.apache.org/jira/browse/APEXCORE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-476: Fix Version/s: 3.5.0 > Window Id calculation in ToString method in SubscribeRequestTuple & >

[jira] [Updated] (APEXCORE-510) Enforce DefaultOutputPort.emit() or Sink.put() thread affinity

[ https://issues.apache.org/jira/browse/APEXCORE-510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-510: Fix Version/s: 3.5.0 > Enforce DefaultOutputPort.emit() or Sink.put() thread affinity >

[jira] [Updated] (APEXCORE-505) setup and activate calls in operator block heartbeat loop in container

[ https://issues.apache.org/jira/browse/APEXCORE-505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-505: Fix Version/s: 3.5.0 > setup and activate calls in operator block heartbeat loop in container >

[jira] [Resolved] (APEXCORE-557) Upgrade netlet dependency to 1.3.0

[ https://issues.apache.org/jira/browse/APEXCORE-557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov resolved APEXCORE-557. - Resolution: Fixed Fix Version/s: 3.5.0 > Upgrade netlet dependency to 1.3.0 >

Re: Visitor API for DAG

+1. Opening up the API for users to put in their own code is good. In general we should enable users to register their code in a lot of scenerios. Thks Amol On Thu, Nov 17, 2016 at 9:06 AM, Tushar Gosavi wrote: > Yes, It could happen after current DAG validation and

Re: Visitor API for DAG

Yes, It could happen after current DAG validation and before the application master is launched. - Tushar. On Thu, Nov 17, 2016 at 8:32 PM, Munagala Ramanath wrote: > When would the visits happen ? Just before normal validation ? > > Ram > > On Wed, Nov 16, 2016 at 9:50

Re: Visitor API for DAG

When would the visits happen ? Just before normal validation ? Ram On Wed, Nov 16, 2016 at 9:50 PM, Tushar Gosavi wrote: > Hi All, > > How about adding visitor like API for DAG in Apex, and an api to > register visitor for the DAG. > Possible use cases are > - Validator

Proposing an operator for log parsing.

Dear community, We would like to add operator in malhar for parsing different types of logs. Idea of this operator is to read log data records of known formats such as Syslog, common log, combined log, extended log etc from the upstream in a DAG, parse/validate it based on the configured format

[jira] [Commented] (APEXMALHAR-2340) Initialize the list of JdbcFieldInfo in JdbcPOJOInsertOutput from properties.xml

[ https://issues.apache.org/jira/browse/APEXMALHAR-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673454#comment-15673454 ] ASF GitHub Bot commented on APEXMALHAR-2340: Github user Hitesh-Scorpio closed the pull

[GitHub] apex-malhar pull request #501: APEXMALHAR-2340 code changes to initialize th...

Github user Hitesh-Scorpio closed the pull request at: https://github.com/apache/apex-malhar/pull/501 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

Re: Malhar release 3.6

David, Any update WRT APEXMALHAR-2130 / APEXMALHAR-2301 ? I would like to cut first RC by end of the week. Thanks On Wed, Nov 16, 2016 at 12:44 AM, Thomas Weise

Re: Visitor API for DAG

+1 Regards, Mohit On Thu, Nov 17, 2016 at 11:20 AM, Tushar Gosavi wrote: > Hi All, > > How about adding visitor like API for DAG in Apex, and an api to > register visitor for the DAG. > Possible use cases are > - Validator visitor which could validate the dag > - Visitor