[jira] [Commented] (FALCON-2148) Entries not removed from sla list when instance succeeds
[
https://issues.apache.org/jira/browse/FALCON-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15511959#comment-15511959
]
Pallavi Rao commented on FALCON-2148:
-
{quote}
Let's say we have metadata loading job or some other job in which only the last
instance matters i.e if the last instance has completed on time there is no
backlog.
{quote}
The Backlog emitter needs to have that intelligence. The execution order,
LAST_ONLY in this case, that is part of process definition and nominal time of
the "latest" instance will need to be used to determine the backlog.
> Entries not removed from sla list when instance succeeds
>
>
> Key: FALCON-2148
> URL: https://issues.apache.org/jira/browse/FALCON-2148
> Project: Falcon
> Issue Type: Bug
> Components: prism
>Affects Versions: trunk
>Reporter: Pragya Mittal
>Assignee: Praveen Adlakha
>
> Entries corresponding to a process sla should be removed if the process goes
> from WAITING to SUCCEEDED state. The same is not happening.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (FALCON-2148) Entries not removed from sla list when instance succeeds
[ https://issues.apache.org/jira/browse/FALCON-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510352#comment-15510352 ] Praveen Adlakha edited comment on FALCON-2148 at 9/21/16 3:57 PM: -- Hi [~sandeep.samudrala] Few more things we need to consider here what if a process get killed before its SLAhigh is missed? If it gets deleted from the pending_instances we will never be able to report the SLA miss to the BacklogEmitterService so we cannot delete it just like that. I believe it should only get deleted in case of success rest of the states should account for the backlog. Edited the comment as there were two separate issues, that were being addressed. Thanks Praveen was (Author: praveen): Hi [~sandeep.samudrala] Few more things we need to consider here what if a process get killed before its SLAhigh is missed? If it gets deleted from the pending_instances we will never be able to report the SLA miss to the BacklogEmitterService so we cannot delete it just like that. I believe it should only get deleted in case of success rest of the states should account for the backlog. Thanks Praveen > Entries not removed from sla list when instance succeeds > > > Key: FALCON-2148 > URL: https://issues.apache.org/jira/browse/FALCON-2148 > Project: Falcon > Issue Type: Bug > Components: prism >Affects Versions: trunk >Reporter: Pragya Mittal >Assignee: Praveen Adlakha > > Entries corresponding to a process sla should be removed if the process goes > from WAITING to SUCCEEDED state. The same is not happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-2148) Entries not removed from sla list when instance succeeds
[ https://issues.apache.org/jira/browse/FALCON-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510360#comment-15510360 ] Praveen Adlakha commented on FALCON-2148: - Hi [~pallavi.rao] One case that comes to my mind in this scenario: Let's say we have metadata loading job or some other job in which only the last instance matters i.e if the last instance has completed on time there is no backlog. In these kinds of case's we will keep on reporting backlog for the failed instances which is not true how are we handling this? Thanks Praveen > Entries not removed from sla list when instance succeeds > > > Key: FALCON-2148 > URL: https://issues.apache.org/jira/browse/FALCON-2148 > Project: Falcon > Issue Type: Bug > Components: prism >Affects Versions: trunk >Reporter: Pragya Mittal >Assignee: Praveen Adlakha > > Entries corresponding to a process sla should be removed if the process goes > from WAITING to SUCCEEDED state. The same is not happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-2148) Entries not removed from sla list when instance succeeds
[ https://issues.apache.org/jira/browse/FALCON-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510352#comment-15510352 ] Praveen Adlakha commented on FALCON-2148: - Hi [~sandeep.samudrala] Few more things we need to consider here what if a process get killed before its SLAhigh is missed? If it gets deleted from the pending_instances we will never be able to report the SLA miss to the BacklogEmitterService so we cannot delete it just like that. I believe it should only get deleted in case of success rest of the states should account for the backlog. Thanks Praveen > Entries not removed from sla list when instance succeeds > > > Key: FALCON-2148 > URL: https://issues.apache.org/jira/browse/FALCON-2148 > Project: Falcon > Issue Type: Bug > Components: prism >Affects Versions: trunk >Reporter: Pragya Mittal >Assignee: Praveen Adlakha > > Entries corresponding to a process sla should be removed if the process goes > from WAITING to SUCCEEDED state. The same is not happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (FALCON-2148) Entries not removed from sla list when instance succeeds
[ https://issues.apache.org/jira/browse/FALCON-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Adlakha updated FALCON-2148: Comment: was deleted (was: Hi [~sandeep.samudrala] Few more things we need to consider here what if a process get killed before its SLAhigh is missed? If it gets deleted from the pending_instances we will never be able to report the SLA miss to the BacklogEmitterService so we cannot delete it just like that. I believe it should only get deleted in case of success rest of the states should account for the backlog. [~pallavi.rao] One case that comes to my mind in this scenario: Let's say we have metadata loading job or some other job in which only the last instance matters i.e if the last instance has completed on time there is no backlog. In these kinds of case's we will keep on reporting backlog for the failed instances which is not true how do you suggest we should handle this? Thanks Praveen ) > Entries not removed from sla list when instance succeeds > > > Key: FALCON-2148 > URL: https://issues.apache.org/jira/browse/FALCON-2148 > Project: Falcon > Issue Type: Bug > Components: prism >Affects Versions: trunk >Reporter: Pragya Mittal >Assignee: Praveen Adlakha > > Entries corresponding to a process sla should be removed if the process goes > from WAITING to SUCCEEDED state. The same is not happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-2148) Entries not removed from sla list when instance succeeds
[ https://issues.apache.org/jira/browse/FALCON-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509977#comment-15509977 ] Praveen Adlakha commented on FALCON-2148: - Hi [~sandeep.samudrala] Few more things we need to consider here what if a process get killed before its SLAhigh is missed? If it gets deleted from the pending_instances we will never be able to report the SLA miss to the BacklogEmitterService so we cannot delete it just like that. I believe it should only get deleted in case of success rest of the states should account for the backlog. [~pallavi.rao] One case that comes to my mind in this scenario: Let's say we have metadata loading job or some other job in which only the last instance matters i.e if the last instance has completed on time there is no backlog. In these kinds of case's we will keep on reporting backlog for the failed instances which is not true how do you suggest we should handle this? Thanks Praveen > Entries not removed from sla list when instance succeeds > > > Key: FALCON-2148 > URL: https://issues.apache.org/jira/browse/FALCON-2148 > Project: Falcon > Issue Type: Bug > Components: prism >Affects Versions: trunk >Reporter: Pragya Mittal >Assignee: Praveen Adlakha > > Entries corresponding to a process sla should be removed if the process goes > from WAITING to SUCCEEDED state. The same is not happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-2150) Instances run in random order when dependencies are met
[ https://issues.apache.org/jira/browse/FALCON-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pragya Mittal updated FALCON-2150: -- Description: If 5 instances of process are in WAITING state and at some point their dependencies are met, then they should ideally run in defined order. But this is not happening as of now. Please note that the db used is mysql and not derby. > Instances run in random order when dependencies are met > --- > > Key: FALCON-2150 > URL: https://issues.apache.org/jira/browse/FALCON-2150 > Project: Falcon > Issue Type: Bug > Components: prism >Affects Versions: trunk >Reporter: Pragya Mittal >Assignee: Praveen Adlakha > > If 5 instances of process are in WAITING state and at some point their > dependencies are met, then they should ideally run in defined order. But this > is not happening as of now. Please note that the db used is mysql and not > derby. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-2150) Instances run in random order when dependencies are met
Pragya Mittal created FALCON-2150: - Summary: Instances run in random order when dependencies are met Key: FALCON-2150 URL: https://issues.apache.org/jira/browse/FALCON-2150 Project: Falcon Issue Type: Bug Components: prism Affects Versions: trunk Reporter: Pragya Mittal -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-2150) Instances run in random order when dependencies are met
[ https://issues.apache.org/jira/browse/FALCON-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pragya Mittal updated FALCON-2150: -- Assignee: Praveen Adlakha > Instances run in random order when dependencies are met > --- > > Key: FALCON-2150 > URL: https://issues.apache.org/jira/browse/FALCON-2150 > Project: Falcon > Issue Type: Bug > Components: prism >Affects Versions: trunk >Reporter: Pragya Mittal >Assignee: Praveen Adlakha > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-2148) Entries not removed from sla list when instance succeeds
[ https://issues.apache.org/jira/browse/FALCON-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509691#comment-15509691 ] sandeep samudrala commented on FALCON-2148: --- Just a NIT to the change that should be applied. It has to be removed even if the process state is killed/failed/succeeded/running > Entries not removed from sla list when instance succeeds > > > Key: FALCON-2148 > URL: https://issues.apache.org/jira/browse/FALCON-2148 > Project: Falcon > Issue Type: Bug > Components: prism >Affects Versions: trunk >Reporter: Pragya Mittal >Assignee: Praveen Adlakha > > Entries corresponding to a process sla should be removed if the process goes > from WAITING to SUCCEEDED state. The same is not happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-2148) Entries not removed from sla list when instance succeeds
[ https://issues.apache.org/jira/browse/FALCON-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pragya Mittal updated FALCON-2148: -- Assignee: Praveen Adlakha > Entries not removed from sla list when instance succeeds > > > Key: FALCON-2148 > URL: https://issues.apache.org/jira/browse/FALCON-2148 > Project: Falcon > Issue Type: Bug > Components: prism >Affects Versions: trunk >Reporter: Pragya Mittal >Assignee: Praveen Adlakha > > Entries corresponding to a process sla should be removed if the process goes > from WAITING to SUCCEEDED state. The same is not happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-2149) On deleting process entries still exist in BACKLOG_METRIC
[ https://issues.apache.org/jira/browse/FALCON-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pragya Mittal updated FALCON-2149: -- Assignee: Praveen Adlakha > On deleting process entries still exist in BACKLOG_METRIC > - > > Key: FALCON-2149 > URL: https://issues.apache.org/jira/browse/FALCON-2149 > Project: Falcon > Issue Type: Bug > Components: prism >Affects Versions: trunk >Reporter: Pragya Mittal >Assignee: Praveen Adlakha > > On deleting process entries still exist in BACKLOG_METRIC which should not be > the case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-2149) On deleting process entries still exist in BACKLOG_METRIC
Pragya Mittal created FALCON-2149: - Summary: On deleting process entries still exist in BACKLOG_METRIC Key: FALCON-2149 URL: https://issues.apache.org/jira/browse/FALCON-2149 Project: Falcon Issue Type: Bug Components: prism Affects Versions: trunk Reporter: Pragya Mittal On deleting process entries still exist in BACKLOG_METRIC which should not be the case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-2148) Entries not removed from sla list when instance succeeds
Pragya Mittal created FALCON-2148: - Summary: Entries not removed from sla list when instance succeeds Key: FALCON-2148 URL: https://issues.apache.org/jira/browse/FALCON-2148 Project: Falcon Issue Type: Bug Components: prism Affects Versions: trunk Reporter: Pragya Mittal Entries corresponding to a process sla should be removed if the process goes from WAITING to SUCCEEDED state. The same is not happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-2147) Excess logging due to SlaMonitoring service and Backlog emitter service
Pragya Mittal created FALCON-2147: - Summary: Excess logging due to SlaMonitoring service and Backlog emitter service Key: FALCON-2147 URL: https://issues.apache.org/jira/browse/FALCON-2147 Project: Falcon Issue Type: Bug Components: prism Affects Versions: trunk Reporter: Pragya Mittal With the SlaMonitoring service clubbed with Backlog emitter service, there has been too much logging information which is filling up the disk space. Please minimize the same and add relevant logging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-2146) EntitySlaAlertService runs only once irrespective of the configured frequency
[
https://issues.apache.org/jira/browse/FALCON-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509258#comment-15509258
]
ASF GitHub Bot commented on FALCON-2146:
GitHub user PraveenAdlakha opened a pull request:
https://github.com/apache/falcon/pull/283
FALCON-2146 EntitySlaAlertService runs only once irrespective of the …
…configured frequency
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/PraveenAdlakha/falcon 2146
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/falcon/pull/283.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #283
commit 2d2bba751f448fb75e2d7a32686dda6e02aa14f3
Author: Praveen Adlakha
Date: 2016-09-17T09:54:36Z
FALCON-2146 EntitySlaAlertService runs only once irrespective of the
configured frequency
> EntitySlaAlertService runs only once irrespective of the configured frequency
> -
>
> Key: FALCON-2146
> URL: https://issues.apache.org/jira/browse/FALCON-2146
> Project: Falcon
> Issue Type: Bug
> Components: prism
>Affects Versions: trunk
>Reporter: Pragya Mittal
>Assignee: Praveen Adlakha
>
> Initially I have 20 pending instances which have missed their sla. The logs
> say the same.
> After some time I have 33 instances of a process which have missed their sla
> {noformat}
> dataqa@falcon1001:/mnt/users/pragya/functionality/sla-alerts$ falcon entity
> -type process -slaAlert -name ProcessSLATest-agregator-coord16-dcd226b3
> -start 2016-09-16T08:50Z -end 2016-09-16T12:10Z| grep -i name | wc -l
> 33
> {noformat}
> But server logs show only 20 instances and also it is running just once
> {noformat}
> dataqa@falcon1001:/mnt/falcon$ cat server/logs/falcon.application.log| grep
> processSLACandidates
> 2016-09-16 10:01:20,117 DEBUG - [pool-13-thread-1:] ~ In processSLACandidates
> :20 (EntitySLAAlertService:115)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] falcon pull request #283: FALCON-2146 EntitySlaAlertService runs only once i...
GitHub user PraveenAdlakha opened a pull request: https://github.com/apache/falcon/pull/283 FALCON-2146 EntitySlaAlertService runs only once irrespective of the ⦠â¦configured frequency You can merge this pull request into a Git repository by running: $ git pull https://github.com/PraveenAdlakha/falcon 2146 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/falcon/pull/283.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #283 commit 2d2bba751f448fb75e2d7a32686dda6e02aa14f3 Author: Praveen Adlakha Date: 2016-09-17T09:54:36Z FALCON-2146 EntitySlaAlertService runs only once irrespective of the configured frequency --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
