Re: Adding new log4j appender to Apex core

Vlad, The feature only works *if the log file name at error does not change later* . In this case, Priyanka proposes a default appender that the user can use that has this behavior while log rotation is still supported. If the user has a custom log appender, the feature can still work if it

Re: Adding new log4j appender to Apex core

Additionally, I think that it is necessary to re-evaluate the requirements. Custom logging is quite common and many enterprises/Devops have own preferences/policy for log rotation and logging format. I saw instances when logging was redirected to stdout. By enforcing specific rotation policy

[jira] [Commented] (APEXCORE-462) StramAgent class initiates a lot of unnecessary YARN connections

[ https://issues.apache.org/jira/browse/APEXCORE-462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691870#comment-15691870 ] ASF GitHub Bot commented on APEXCORE-462: - Github user davidyan74 closed the pull request at:

Re: Proposing a new feature to persist logical and physical plan snapshots in HDFS

Persisted plan on DFS is good. I am +1 for it. This could be both of the following 1. Attribute : If set, then upon change in plan persist to DFS 2. On demand Thks Amol On Wed, Nov 23, 2016 at 4:15 PM, Sanjay Pujare wrote: > Okay, but this “state” is gone after the

Re: Proposing a new feature to persist logical and physical plan snapshots in HDFS

Okay, but this “state” is gone after the app is “dead” isn’t that true? Also the reason for this enhancement is debuggability/troubleshooting of Apex apps so it is good to have separate explicit user visible files that contain the plan information instead of overloading the state for this

Re: Proposing a new feature to persist logical and physical plan snapshots in HDFS

Good idea. Stram does save state, and maybe a script that translates may work. But explicit plan saving is also a good idea. Could this be "on demand"? a rest call that writes out the plan(s) to specifid hdfs files? We could do both (write on any change/set call) and/or on-demand. Thks Amol On

Proposing a new feature to persist logical and physical plan snapshots in HDFS

To help Apex developers/users with debugging or troubleshooting “dead” applications, I am proposing a new feature to persist logical and physical plan snapshots in HDFS. Similar to how the Apex engine persists container data per application attempt in HDFS as containers_NNN.json (where NNN

Re: Adding new log4j appender to Apex core

Both approaches look quite "hacky" to me. Thank you, Vlad On 11/23/16 00:01, Mohit Jotwani wrote: +1 - Approach 2 Regards, Mohit On Wed, Nov 23, 2016 at 12:35 PM, AJAY GUPTA wrote: +1 for approach 2. Regards, Ajay On Wed, Nov 23, 2016 at 12:16 PM, David Yan

[jira] [Commented] (APEXMALHAR-2352) Improve performance of keyed windowed operators

[ https://issues.apache.org/jira/browse/APEXMALHAR-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691426#comment-15691426 ] bright chen commented on APEXMALHAR-2352: - I am not very sure about the reason. Maybe in

FOSDEM 2017 HPC, Bigdata and Data Science DevRoom CFP is closing soon

Hi! apologies for the extra wide distribution (this exhausts my once a year ASF mail-to-all-bigdata-projects quota ;-)) but I wanted to suggest that all of you should consider submitting talks to FOSDEM 2017 HPC, Bigdata and Data Science DevRoom: https://hpc-bigdata-fosdem17.github.io/ It

[jira] [Updated] (APEXCORE-563) Have a pointer to container log file name and offset in stram events that deliver a container or operator failure event.

[ https://issues.apache.org/jira/browse/APEXCORE-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-563: Issue Type: New Feature (was: Bug) > Have a pointer to container log file name and offset in

[jira] [Commented] (APEXCORE-526) Publish javadoc for releases on ASF infrastructure

[ https://issues.apache.org/jira/browse/APEXCORE-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690911#comment-15690911 ] Munagala V. Ramanath commented on APEXCORE-526: --- Anything that can be generated with a

[jira] [Comment Edited] (APEXCORE-526) Publish javadoc for releases on ASF infrastructure

[ https://issues.apache.org/jira/browse/APEXCORE-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690911#comment-15690911 ] Munagala V. Ramanath edited comment on APEXCORE-526 at 11/23/16 6:04 PM:

[jira] [Commented] (APEXCORE-563) Have a pointer to container log file name and offset in stram events that deliver a container or operator failure event.

[ https://issues.apache.org/jira/browse/APEXCORE-563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689980#comment-15689980 ] ASF GitHub Bot commented on APEXCORE-563: - GitHub user DT-Priyanka opened a pull request:

[GitHub] apex-core pull request #421: APEXCORE-563: Adding APEXRFA appender to roll l...

GitHub user DT-Priyanka opened a pull request: https://github.com/apache/apex-core/pull/421 APEXCORE-563: Adding APEXRFA appender to roll log files without renam… …ing them, also adding log file name and offset in events You can merge this pull request into a Git repository by

[jira] [Resolved] (APEXMALHAR-2353) timeExpression should not be null for time based Dedup

[ https://issues.apache.org/jira/browse/APEXMALHAR-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar resolved APEXMALHAR-2353. --- Resolution: Fixed Fix Version/s: 3.6.0 > timeExpression should

[jira] [Commented] (APEXMALHAR-2353) timeExpression should not be null for time based Dedup

[ https://issues.apache.org/jira/browse/APEXMALHAR-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689945#comment-15689945 ] ASF GitHub Bot commented on APEXMALHAR-2353: Github user asfgit closed the pull request

[GitHub] apex-malhar pull request #508: APEXMALHAR-2353 Made timeExpression as @NotNu...

Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/508 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (APEXMALHAR-2340) Initialize the list of JdbcFieldInfo in JdbcPOJOInsertOutput from properties.xml

[ https://issues.apache.org/jira/browse/APEXMALHAR-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689355#comment-15689355 ] ASF GitHub Bot commented on APEXMALHAR-2340: GitHub user Hitesh-Scorpio reopened a pull

[jira] [Commented] (APEXMALHAR-2340) Initialize the list of JdbcFieldInfo in JdbcPOJOInsertOutput from properties.xml

[ https://issues.apache.org/jira/browse/APEXMALHAR-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689356#comment-15689356 ] ASF GitHub Bot commented on APEXMALHAR-2340: Github user Hitesh-Scorpio closed the pull

[GitHub] apex-malhar pull request #507: APEXMALHAR-2340 code changes to initialize th...

GitHub user Hitesh-Scorpio reopened a pull request: https://github.com/apache/apex-malhar/pull/507 APEXMALHAR-2340 code changes to initialize the list of JdbcFieldInfo … …in JdbcPOJOInsertOutput from properties.xml @bhupeshchawda please review You can merge this pull request

[GitHub] apex-malhar pull request #507: APEXMALHAR-2340 code changes to initialize th...

Github user Hitesh-Scorpio closed the pull request at: https://github.com/apache/apex-malhar/pull/507 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Updated] (APEXMALHAR-2353) timeExpression should not be null for time based Dedup

[ https://issues.apache.org/jira/browse/APEXMALHAR-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhupesh Chawda updated APEXMALHAR-2353: --- Description: Time Based Dedup has timeExpression as optional. It has to be

[jira] [Commented] (APEXMALHAR-2353) timeExpression should not be null for time based Dedup

[ https://issues.apache.org/jira/browse/APEXMALHAR-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689342#comment-15689342 ] ASF GitHub Bot commented on APEXMALHAR-2353: GitHub user bhupeshchawda opened a pull

[GitHub] apex-malhar pull request #508: APEXMALHAR-2353 Made timeExpression as @NotNu...

GitHub user bhupeshchawda opened a pull request: https://github.com/apache/apex-malhar/pull/508 APEXMALHAR-2353 Made timeExpression as @NotNull @chinmaykolhatkar Please review and merge You can merge this pull request into a Git repository by running: $ git pull

[jira] [Created] (APEXMALHAR-2353) timeExpression should not be null for time based Dedup

Bhupesh Chawda created APEXMALHAR-2353: -- Summary: timeExpression should not be null for time based Dedup Key: APEXMALHAR-2353 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2353 Project:

Re: Adding new log4j appender to Apex core

+1 - Approach 2 Regards, Mohit On Wed, Nov 23, 2016 at 12:35 PM, AJAY GUPTA wrote: > +1 for approach 2. > > > Regards, > Ajay > > On Wed, Nov 23, 2016 at 12:16 PM, David Yan wrote: > > > The goal of this log4j appender is to provide a log offset