[jira] [Assigned] (FALCON-2059) BacklogMetricEmitter Service for Falcon
[ https://issues.apache.org/jira/browse/FALCON-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri reassigned FALCON-2059: Assignee: pavan kumar kolamuri > BacklogMetricEmitter Service for Falcon > > > Key: FALCON-2059 > URL: https://issues.apache.org/jira/browse/FALCON-2059 > Project: Falcon > Issue Type: Task >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > > Backlog Metric Emitter service evaluates backlog for processes and emits > backlog metrics to graphite -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-2059) BacklogMetricEmitter Service for Falcon
pavan kumar kolamuri created FALCON-2059: Summary: BacklogMetricEmitter Service for Falcon Key: FALCON-2059 URL: https://issues.apache.org/jira/browse/FALCON-2059 Project: Falcon Issue Type: Task Reporter: pavan kumar kolamuri Backlog Metric Emitter service evaluates backlog for processes and emits backlog metrics to graphite -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1893) Add documentation and examples for spark workflow engine
[ https://issues.apache.org/jira/browse/FALCON-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1893: - Summary: Add documentation and examples for spark workflow engine (was: Add documentation and examples to spark workflow engine) > Add documentation and examples for spark workflow engine > > > Key: FALCON-1893 > URL: https://issues.apache.org/jira/browse/FALCON-1893 > Project: Falcon > Issue Type: Sub-task >Reporter: Peeyush Bishnoi >Assignee: Peeyush Bishnoi > Attachments: FALCON-1893.patch > > > Add documentation with Spark examples that will illustrate how to execute > the Spark Java/Python application through Falcon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1893) Add documentation and examples for Spark workflow engine
[ https://issues.apache.org/jira/browse/FALCON-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1893: - Summary: Add documentation and examples for Spark workflow engine (was: Add documentation with Spark examples ) > Add documentation and examples for Spark workflow engine > > > Key: FALCON-1893 > URL: https://issues.apache.org/jira/browse/FALCON-1893 > Project: Falcon > Issue Type: Sub-task >Reporter: Peeyush Bishnoi >Assignee: Peeyush Bishnoi > Attachments: FALCON-1893.patch > > > Add documentation with Spark examples that will illustrate how to execute > the Spark Java/Python application through Falcon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FALCON-1935) Can't start Falcon with default startup.properties (FalconJPAService fails)
[
https://issues.apache.org/jira/browse/FALCON-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pavan kumar kolamuri resolved FALCON-1935.
--
Resolution: Fixed
Fix Version/s: trunk
Issue resolved by pull request 128
[https://github.com/apache/falcon/pull/128]
> Can't start Falcon with default startup.properties (FalconJPAService fails)
> ---
>
> Key: FALCON-1935
> URL: https://issues.apache.org/jira/browse/FALCON-1935
> Project: Falcon
> Issue Type: Bug
>Reporter: Ying Zheng
>Assignee: Praveen Adlakha
>Priority: Blocker
> Fix For: trunk
>
>
> When starting Falcon with default startup.properties, it fails when starting
> FalconJPAService. See errors below:
> {code}
> 2016-05-03 21:51:16,756 ERROR - [Thread-0:] ~ Failed to destroy service
> org.apache.falcon.service.FalconJPAService (ServiceInitializer:62)
> java.lang.NullPointerException
> at
> org.apache.falcon.service.FalconJPAService.destroy(FalconJPAService.java:156)
> at
> org.apache.falcon.service.ServiceInitializer.destroy(ServiceInitializer.java:60)
> at
> org.apache.falcon.listener.ContextStartupListener.contextDestroyed(ContextStartupListener.java:86)
> at
> org.mortbay.jetty.handler.ContextHandler.doStop(ContextHandler.java:601)
> at org.mortbay.jetty.webapp.WebAppContext.doStop(WebAppContext.java:537)
> at
> org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStop(HandlerWrapper.java:142)
> at org.mortbay.jetty.Server.doStop(Server.java:283)
> at
> org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
> at org.apache.falcon.util.EmbeddedServer.stop(EmbeddedServer.java:62)
> at org.apache.falcon.FalconServer$ShutDown.run(FalconServer.java:76)
> 2016-05-03 21:51:16,757 WARN - [Thread-0:] ~ Error destroying services
> (ContextStartupListener:88)
> org.apache.falcon.FalconException: java.lang.NullPointerException
> at
> org.apache.falcon.service.ServiceInitializer.destroy(ServiceInitializer.java:63)
> at
> org.apache.falcon.listener.ContextStartupListener.contextDestroyed(ContextStartupListener.java:86)
> at
> org.mortbay.jetty.handler.ContextHandler.doStop(ContextHandler.java:601)
> at org.mortbay.jetty.webapp.WebAppContext.doStop(WebAppContext.java:537)
> at
> org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStop(HandlerWrapper.java:142)
> at org.mortbay.jetty.Server.doStop(Server.java:283)
> at
> org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
> at org.apache.falcon.util.EmbeddedServer.stop(EmbeddedServer.java:62)
> at org.apache.falcon.FalconServer$ShutDown.run(FalconServer.java:76)
> Caused by: java.lang.NullPointerException
> at
> org.apache.falcon.service.FalconJPAService.destroy(FalconJPAService.java:156)
> at
> org.apache.falcon.service.ServiceInitializer.destroy(ServiceInitializer.java:60)
> ... 9 more
> {code}
> If I disable FalconJPAService in startup.properties, it still throws error:
> {code}
> java.lang.RuntimeException: org.apache.falcon.FalconException: Unable to get
> instance for #
> org.apache.falcon.service.FalconJPAService
> at
> org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:59)
> at
> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:550)
> at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:519)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:57)
> at org.apache.falcon.FalconServer.main(FalconServer.java:118)
> Caused by: org.apache.falcon.FalconException: Unable to get instance for #
> org.apache.falcon.service.FalconJPAService
> at
> org.apache.falcon.util.ReflectionUtils.getInstanceByClassName(ReflectionUtils.java:52)
> at
> org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:43)
> at
>
[jira] [Resolved] (FALCON-1956) Graphite Plugin for monitoring
[ https://issues.apache.org/jira/browse/FALCON-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri resolved FALCON-1956. -- Resolution: Fixed Fix Version/s: trunk Issue resolved by pull request 141 [https://github.com/apache/falcon/pull/141] > Graphite Plugin for monitoring > -- > > Key: FALCON-1956 > URL: https://issues.apache.org/jira/browse/FALCON-1956 > Project: Falcon > Issue Type: New Feature >Reporter: Praveen Adlakha >Assignee: Praveen Adlakha > Fix For: trunk > > > Hi All, > To monitor the performance of a process we should have Graphite plugin > through which we can plot the graphs how the process has been performing over > a period of time. > Monitoring frame work is already there with all the data.We just have to add > a plugin to push data to graphite. > Thanks > Praveen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-1910) Optional Input should be truly optional in case of Native Scheduler as well
pavan kumar kolamuri created FALCON-1910: Summary: Optional Input should be truly optional in case of Native Scheduler as well Key: FALCON-1910 URL: https://issues.apache.org/jira/browse/FALCON-1910 Project: Falcon Issue Type: Bug Components: scheduler Reporter: pavan kumar kolamuri https://issues.apache.org/jira/browse/FALCON-1852 We should do the same in case of Native Scheduler as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1867) hardcoded query names in JDBCStateStore
[ https://issues.apache.org/jira/browse/FALCON-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15215566#comment-15215566 ] pavan kumar kolamuri commented on FALCON-1867: -- [~ajayyadava] Make sense. [~Praveen] We are not hardcoding string in multiple places, we are just using the query name. Anyway it doesn't have compile time check. Can you remove extra spaces in patch . Otherwise patch looks good +1 > hardcoded query names in JDBCStateStore > --- > > Key: FALCON-1867 > URL: https://issues.apache.org/jira/browse/FALCON-1867 > Project: Falcon > Issue Type: Bug >Reporter: Praveen Adlakha >Assignee: Praveen Adlakha > > Hi All, > All the query names in JDBC statestore are hardcoded string across two > different classes which is not a good practice and can causes run time errors. > Need to change that. > Thanks > Praveen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1867) hardcoded query names in JDBCStateStore
[ https://issues.apache.org/jira/browse/FALCON-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214700#comment-15214700 ] pavan kumar kolamuri commented on FALCON-1867: -- [~Praveen] Can you explain what kind of run time errors possible ? > hardcoded query names in JDBCStateStore > --- > > Key: FALCON-1867 > URL: https://issues.apache.org/jira/browse/FALCON-1867 > Project: Falcon > Issue Type: Bug >Reporter: Praveen Adlakha >Assignee: Praveen Adlakha > > Hi All, > All the query names in JDBC statestore are hardcoded string across two > different classes which is not a good practice and can causes run time errors. > Need to change that. > Thanks > Praveen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1817) Update xsd for Spark execution engine
[ https://issues.apache.org/jira/browse/FALCON-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199394#comment-15199394 ] pavan kumar kolamuri commented on FALCON-1817: -- [~peeyushb] All this spark options are defined in oozie workflow. With Spark Engine in Falcon won't we use oozie spark action ? > Update xsd for Spark execution engine > - > > Key: FALCON-1817 > URL: https://issues.apache.org/jira/browse/FALCON-1817 > Project: Falcon > Issue Type: Sub-task >Reporter: Peeyush Bishnoi >Assignee: Peeyush Bishnoi > Fix For: trunk > > Attachments: FALCON-1817.patch > > > Update xsd to add Spark execution engine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1406) Effective time in Entity updates.
[ https://issues.apache.org/jira/browse/FALCON-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184631#comment-15184631 ] pavan kumar kolamuri commented on FALCON-1406: -- Just started review, had some initial comments. Will review and update again. > Effective time in Entity updates. > - > > Key: FALCON-1406 > URL: https://issues.apache.org/jira/browse/FALCON-1406 > Project: Falcon > Issue Type: New Feature >Reporter: sandeep samudrala >Assignee: sandeep samudrala > Labels: syncup > Attachments: FALCON-1406-initial.patch, > effective_time_in_entity_updates.pdf > > > Effective time with entity updates needs to be provided even with past time > too. There was effective time capability provided in the past which gives the > functionality to set an effective time for an entity with only current or > future time(now + delay), which could not solve all the issues. > Following are few scenarios which would require effective time to be > available with time back in past. > a) New code being deployed for an incompatible input data set which would > leave instances with old code and new data. > b) Bad code being pushed for which, the entity should be able to go back in > time to replay(rerun) with new code. > c) Orchestration level changes(good/bad) would need functionality to go back > in time to start with. > For reference: Linking all the Jiras that have been worked upon around > effective time . > https://issues.apache.org/jira/browse/FALCON-374 > https://issues.apache.org/jira/browse/FALCON-297 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1794) Behaviour mismatch in running Api when scheduled on native instead of oozie
[
https://issues.apache.org/jira/browse/FALCON-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182819#comment-15182819
]
pavan kumar kolamuri commented on FALCON-1794:
--
[~deepak.barr] Sure no problem Thanks for the update.
> Behaviour mismatch in running Api when scheduled on native instead of oozie
> ---
>
> Key: FALCON-1794
> URL: https://issues.apache.org/jira/browse/FALCON-1794
> Project: Falcon
> Issue Type: Improvement
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: Deepak Barr
>
> Running api shows following result for native scheduler
> {noformat}
> dataqa@lda01:~$ falcon instance -type process -name processMerlinNative
> -running
> Consolidated Status: SUCCEEDED
> Instances:
> Instance Cluster SourceCluster Status
> Start End Details Log
> ---
> 2016-01-29T09:37:00.000Z ProcessMultipleClustersTest-corp-ed7e77d1
> - RUNNING 2016-01-29T10:27Z -- -
> 2016-01-29T09:38:00.000Z ProcessMultipleClustersTest-corp-ed7e77d1
> - RUNNING 2016-01-29T10:27Z -- -
> 2016-01-29T09:37:00.000Z ProcessMultipleClustersTest-corp-f62f3b66
> - RUNNING 2016-01-29T10:27Z -- -
> 2016-01-29T09:38:00.000Z ProcessMultipleClustersTest-corp-f62f3b66
> - RUNNING 2016-01-29T10:27Z -- -
> {noformat}
> While oozie scheduled process shows following result
> {noformat}
> dataqa@lda01:~$ falcon instance -type process -name processMerlinOozie
> -running
> Consolidated Status: SUCCEEDED
> Instances:
> Instance Cluster SourceCluster Status
> Start End Details Log
> ---
> 2016-01-29T09:40ZProcessMultipleClustersTest-corp-ed7e77d1
> - RUNNING 2016-01-29T10:28Z - --
> 2016-01-29T09:39Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> RUNNING 2016-01-29T10:28Z - --
> 2016-01-29T09:38Z ProcessMultipleClustersTest-corp-f62f3b66
> - RUNNING 2016-01-29T10:27Z - --
> 2016-01-29T09:37Z ProcessMultipleClustersTest-corp-f62f3b66 -
> RUNNING 2016-01-29T10:27Z - --
> {noformat}
> There are two concerns :
> 1. Instance time format :
> Native - 2016-01-29T09:37:00.000Z
> Oozie - 2016-01-29T09:40Z
> 2. Order of instances -
> Native - Ascending
> Oozie - Descending
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1794) Behaviour mismatch in running Api when scheduled on native instead of oozie
[
https://issues.apache.org/jira/browse/FALCON-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182582#comment-15182582
]
pavan kumar kolamuri commented on FALCON-1794:
--
[~deepak.barr] Can you please update patch based on [~pallavi.rao] comments.
> Behaviour mismatch in running Api when scheduled on native instead of oozie
> ---
>
> Key: FALCON-1794
> URL: https://issues.apache.org/jira/browse/FALCON-1794
> Project: Falcon
> Issue Type: Improvement
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: Deepak Barr
>
> Running api shows following result for native scheduler
> {noformat}
> dataqa@lda01:~$ falcon instance -type process -name processMerlinNative
> -running
> Consolidated Status: SUCCEEDED
> Instances:
> Instance Cluster SourceCluster Status
> Start End Details Log
> ---
> 2016-01-29T09:37:00.000Z ProcessMultipleClustersTest-corp-ed7e77d1
> - RUNNING 2016-01-29T10:27Z -- -
> 2016-01-29T09:38:00.000Z ProcessMultipleClustersTest-corp-ed7e77d1
> - RUNNING 2016-01-29T10:27Z -- -
> 2016-01-29T09:37:00.000Z ProcessMultipleClustersTest-corp-f62f3b66
> - RUNNING 2016-01-29T10:27Z -- -
> 2016-01-29T09:38:00.000Z ProcessMultipleClustersTest-corp-f62f3b66
> - RUNNING 2016-01-29T10:27Z -- -
> {noformat}
> While oozie scheduled process shows following result
> {noformat}
> dataqa@lda01:~$ falcon instance -type process -name processMerlinOozie
> -running
> Consolidated Status: SUCCEEDED
> Instances:
> Instance Cluster SourceCluster Status
> Start End Details Log
> ---
> 2016-01-29T09:40ZProcessMultipleClustersTest-corp-ed7e77d1
> - RUNNING 2016-01-29T10:28Z - --
> 2016-01-29T09:39Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> RUNNING 2016-01-29T10:28Z - --
> 2016-01-29T09:38Z ProcessMultipleClustersTest-corp-f62f3b66
> - RUNNING 2016-01-29T10:27Z - --
> 2016-01-29T09:37Z ProcessMultipleClustersTest-corp-f62f3b66 -
> RUNNING 2016-01-29T10:27Z - --
> {noformat}
> There are two concerns :
> 1. Instance time format :
> Native - 2016-01-29T09:37:00.000Z
> Oozie - 2016-01-29T09:40Z
> 2. Order of instances -
> Native - Ascending
> Oozie - Descending
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (FALCON-1825) Process end time inclusive in case of Native Scheduler
[ https://issues.apache.org/jira/browse/FALCON-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri resolved FALCON-1825. -- Resolution: Fixed Fix Version/s: trunk Issue resolved by pull request 59 [https://github.com/apache/falcon/pull/59] > Process end time inclusive in case of Native Scheduler > --- > > Key: FALCON-1825 > URL: https://issues.apache.org/jira/browse/FALCON-1825 > Project: Falcon > Issue Type: Bug > Components: scheduler >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Fix For: trunk > > > In case of process scheduling through oozie as scheduler end time is taken as > exclusive and where as in case of Native Scheduler end time is inclusive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1845) Retries Stopped happening for all entities when one entity was deleted during rerun of instance
[ https://issues.apache.org/jira/browse/FALCON-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179468#comment-15179468 ] pavan kumar kolamuri commented on FALCON-1845: -- There won't be any stack trace we are actually returning from thread which makes RetryHandler thread to stop. I just removed return statement in the thread. > Retries Stopped happening for all entities when one entity was deleted > during rerun of instance > > > Key: FALCON-1845 > URL: https://issues.apache.org/jira/browse/FALCON-1845 > Project: Falcon > Issue Type: Bug > Components: rerun >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > > Whenever entity was deleted during rerun of instance of that entity , there > is chance that RetryHandler thread was stopped. Further retries won't happen > in FalconServer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (FALCON-1845) Retries Stopped happening for all entities when one entity was deleted during rerun of instance
[ https://issues.apache.org/jira/browse/FALCON-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179468#comment-15179468 ] pavan kumar kolamuri edited comment on FALCON-1845 at 3/4/16 6:56 AM: -- [~ajayyadava]] There won't be any stack trace we are actually returning from thread which makes RetryHandler thread to stop. I just removed return statement in the thread. was (Author: pavan kumar): There won't be any stack trace we are actually returning from thread which makes RetryHandler thread to stop. I just removed return statement in the thread. > Retries Stopped happening for all entities when one entity was deleted > during rerun of instance > > > Key: FALCON-1845 > URL: https://issues.apache.org/jira/browse/FALCON-1845 > Project: Falcon > Issue Type: Bug > Components: rerun >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > > Whenever entity was deleted during rerun of instance of that entity , there > is chance that RetryHandler thread was stopped. Further retries won't happen > in FalconServer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FALCON-1826) Execution order not honoured when instances are KILLED
[ https://issues.apache.org/jira/browse/FALCON-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri resolved FALCON-1826. -- Resolution: Fixed Fix Version/s: trunk Issue resolved by pull request 58 [https://github.com/apache/falcon/pull/58] > Execution order not honoured when instances are KILLED > -- > > Key: FALCON-1826 > URL: https://issues.apache.org/jira/browse/FALCON-1826 > Project: Falcon > Issue Type: Bug > Components: scheduler >Affects Versions: trunk, 0.9 >Reporter: Pragya Mittal >Assignee: Pallavi Rao > Fix For: trunk > > > If any instance(entity scheduled on native scheduler) is killed (irrespective > of its state - READY/RUNNING) then instance execution order is not preserved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1826) Execution order not honoured when instances are KILLED
[ https://issues.apache.org/jira/browse/FALCON-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179437#comment-15179437 ] pavan kumar kolamuri commented on FALCON-1826: -- +1 will commit shortly. > Execution order not honoured when instances are KILLED > -- > > Key: FALCON-1826 > URL: https://issues.apache.org/jira/browse/FALCON-1826 > Project: Falcon > Issue Type: Bug > Components: scheduler >Affects Versions: trunk, 0.9 >Reporter: Pragya Mittal >Assignee: Pallavi Rao > > If any instance(entity scheduled on native scheduler) is killed (irrespective > of its state - READY/RUNNING) then instance execution order is not preserved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-1845) Retries Stopped happening for all entities when one entity was deleted during rerun of instance
pavan kumar kolamuri created FALCON-1845: Summary: Retries Stopped happening for all entities when one entity was deleted during rerun of instance Key: FALCON-1845 URL: https://issues.apache.org/jira/browse/FALCON-1845 Project: Falcon Issue Type: Bug Components: rerun Reporter: pavan kumar kolamuri Assignee: pavan kumar kolamuri Whenever entity was deleted during rerun of instance of that entity , there is chance that RetryHandler thread was stopped. Further retries won't happen in FalconServer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-1825) Process end time not exclusive in case of Native Scheduler
pavan kumar kolamuri created FALCON-1825: Summary: Process end time not exclusive in case of Native Scheduler Key: FALCON-1825 URL: https://issues.apache.org/jira/browse/FALCON-1825 Project: Falcon Issue Type: Bug Components: scheduler Reporter: pavan kumar kolamuri Assignee: pavan kumar kolamuri In case of process scheduling through oozie as scheduler end time is taken as exclusive and where as in case of Native Scheduler end time is not exclusive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1803) SkipDryRun and properties options are not taken care in case of Native Scheduler during Schedule
[ https://issues.apache.org/jira/browse/FALCON-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136153#comment-15136153 ] pavan kumar kolamuri commented on FALCON-1803: -- [~Praveen] Did you start working on this ? I have to fix this as part of data based support forgot to assign this to me . > SkipDryRun and properties options are not taken care in case of Native > Scheduler during Schedule > > > Key: FALCON-1803 > URL: https://issues.apache.org/jira/browse/FALCON-1803 > Project: Falcon > Issue Type: Bug > Components: scheduler >Reporter: pavan kumar kolamuri >Assignee: Praveen Adlakha > > skipDryRun option and properties option while scheduling an entity are not > honored in case of Native Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1795) Kill api does not kill waiting/ready instances
[
https://issues.apache.org/jira/browse/FALCON-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131907#comment-15131907
]
pavan kumar kolamuri commented on FALCON-1795:
--
+1 for the patch. Will commit it if there are no further comments.
> Kill api does not kill waiting/ready instances
> --
>
> Key: FALCON-1795
> URL: https://issues.apache.org/jira/browse/FALCON-1795
> Project: Falcon
> Issue Type: Bug
> Components: oozie
>Affects Versions: trunk, 0.9
>Reporter: Pragya Mittal
>Assignee: sandeep samudrala
>
> If instance is in WAITING/READY state, kill api should kill those instances.
> {noformat}
> dataqa@lda01:~$ falcon instance -type process -kill -name processMerlinOozie
> -start 2016-01-29T10:50Z -end 2016-01-29T10:53Z
> Consolidated Status: SUCCEEDED
> Instances:
> Instance Cluster SourceCluster Status
> Start End Details Log
> ---
> 2016-01-29T10:52Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> WAITING - - - -
> 2016-01-29T10:51Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> READY - - - -
> 2016-01-29T10:50Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> READY - - - -
> 2016-01-29T10:52Z ProcessMultipleClustersTest-corp-f62f3b66 -
> WAITING - - - -
> 2016-01-29T10:51Z ProcessMultipleClustersTest-corp-f62f3b66 -
> READY - - - -
> 2016-01-29T10:50Z ProcessMultipleClustersTest-corp-f62f3b66 -
> READY - - - -
> Additional Information:
> Response: ua1/KILL
> ua2/KILL
> Request Id: ua1/467103859@qtp-1816147548-8 -
> 0645af64-a84e-453b-b339-af8ba6eb128a
> ua2/1459704390@qtp-1583159071-4 - 635cf4c0-620a-4a44-a535-8a35924f66d9
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1795) Kill api does not kill waiting/ready instances
[
https://issues.apache.org/jira/browse/FALCON-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131947#comment-15131947
]
pavan kumar kolamuri commented on FALCON-1795:
--
[~ajayyadava] Sometimes during review we want to apply patch locally and see
code flow in IDEA or Eclipse, for that patch is useful. But anyway we are
getting diff from https://github.com/apache/falcon/pull/22.diff.
> Kill api does not kill waiting/ready instances
> --
>
> Key: FALCON-1795
> URL: https://issues.apache.org/jira/browse/FALCON-1795
> Project: Falcon
> Issue Type: Bug
> Components: oozie
>Affects Versions: trunk, 0.9
>Reporter: Pragya Mittal
>Assignee: sandeep samudrala
>
> If instance is in WAITING/READY state, kill api should kill those instances.
> {noformat}
> dataqa@lda01:~$ falcon instance -type process -kill -name processMerlinOozie
> -start 2016-01-29T10:50Z -end 2016-01-29T10:53Z
> Consolidated Status: SUCCEEDED
> Instances:
> Instance Cluster SourceCluster Status
> Start End Details Log
> ---
> 2016-01-29T10:52Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> WAITING - - - -
> 2016-01-29T10:51Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> READY - - - -
> 2016-01-29T10:50Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> READY - - - -
> 2016-01-29T10:52Z ProcessMultipleClustersTest-corp-f62f3b66 -
> WAITING - - - -
> 2016-01-29T10:51Z ProcessMultipleClustersTest-corp-f62f3b66 -
> READY - - - -
> 2016-01-29T10:50Z ProcessMultipleClustersTest-corp-f62f3b66 -
> READY - - - -
> Additional Information:
> Response: ua1/KILL
> ua2/KILL
> Request Id: ua1/467103859@qtp-1816147548-8 -
> 0645af64-a84e-453b-b339-af8ba6eb128a
> ua2/1459704390@qtp-1583159071-4 - 635cf4c0-620a-4a44-a535-8a35924f66d9
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1795) Kill api does not kill waiting/ready instances
[
https://issues.apache.org/jira/browse/FALCON-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131908#comment-15131908
]
pavan kumar kolamuri commented on FALCON-1795:
--
[~sandeep.samudrala] Please attach patch here as well
> Kill api does not kill waiting/ready instances
> --
>
> Key: FALCON-1795
> URL: https://issues.apache.org/jira/browse/FALCON-1795
> Project: Falcon
> Issue Type: Bug
> Components: oozie
>Affects Versions: trunk, 0.9
>Reporter: Pragya Mittal
>Assignee: sandeep samudrala
>
> If instance is in WAITING/READY state, kill api should kill those instances.
> {noformat}
> dataqa@lda01:~$ falcon instance -type process -kill -name processMerlinOozie
> -start 2016-01-29T10:50Z -end 2016-01-29T10:53Z
> Consolidated Status: SUCCEEDED
> Instances:
> Instance Cluster SourceCluster Status
> Start End Details Log
> ---
> 2016-01-29T10:52Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> WAITING - - - -
> 2016-01-29T10:51Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> READY - - - -
> 2016-01-29T10:50Z ProcessMultipleClustersTest-corp-ed7e77d1 -
> READY - - - -
> 2016-01-29T10:52Z ProcessMultipleClustersTest-corp-f62f3b66 -
> WAITING - - - -
> 2016-01-29T10:51Z ProcessMultipleClustersTest-corp-f62f3b66 -
> READY - - - -
> 2016-01-29T10:50Z ProcessMultipleClustersTest-corp-f62f3b66 -
> READY - - - -
> Additional Information:
> Response: ua1/KILL
> ua2/KILL
> Request Id: ua1/467103859@qtp-1816147548-8 -
> 0645af64-a84e-453b-b339-af8ba6eb128a
> ua2/1459704390@qtp-1583159071-4 - 635cf4c0-620a-4a44-a535-8a35924f66d9
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (FALCON-1798) Evaluation of input paths and check for their availability in case of Native Scheduler
pavan kumar kolamuri created FALCON-1798: Summary: Evaluation of input paths and check for their availability in case of Native Scheduler Key: FALCON-1798 URL: https://issues.apache.org/jira/browse/FALCON-1798 Project: Falcon Issue Type: Task Reporter: pavan kumar kolamuri Assignee: pavan kumar kolamuri As part of Data based support to Native Scheduler, input paths evaluation and check their availability and schedule the process if all paths available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-1803) SkipDryRun and properties options are not taken care in case of Native Scheduler during Schedule
pavan kumar kolamuri created FALCON-1803: Summary: SkipDryRun and properties options are not taken care in case of Native Scheduler during Schedule Key: FALCON-1803 URL: https://issues.apache.org/jira/browse/FALCON-1803 Project: Falcon Issue Type: Bug Components: scheduler Reporter: pavan kumar kolamuri skipDryRun option and properties option while scheduling an entity are not honored in case of Native Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1435) EL Expression Evaluation of Entities in case of Native Scheduler
[ https://issues.apache.org/jira/browse/FALCON-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125813#comment-15125813 ] pavan kumar kolamuri commented on FALCON-1435: -- Refactoring of DataNotificationRequestBuilder and DataNotificationRequest was done as part of FALCON-1230 itself. > EL Expression Evaluation of Entities in case of Native Scheduler > > > Key: FALCON-1435 > URL: https://issues.apache.org/jira/browse/FALCON-1435 > Project: Falcon > Issue Type: Sub-task > Components: scheduler >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > > All the EL Expressions which are supported in oozie and falcon, should be > added in falcon as part of native scheduler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1798) Evaluation of input paths and check for their availability in case of Native Scheduler
[ https://issues.apache.org/jira/browse/FALCON-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1798: - Attachment: FALCON-1798-initial.patch > Evaluation of input paths and check for their availability in case of Native > Scheduler > > > Key: FALCON-1798 > URL: https://issues.apache.org/jira/browse/FALCON-1798 > Project: Falcon > Issue Type: Task >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Attachments: FALCON-1798-initial.patch > > > As part of Data based support to Native Scheduler, input paths evaluation and > check their availability and schedule the process if all paths available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-1802) Workflow Builder for scheduling based on Data for Process in case of Native Scheduler
pavan kumar kolamuri created FALCON-1802: Summary: Workflow Builder for scheduling based on Data for Process in case of Native Scheduler Key: FALCON-1802 URL: https://issues.apache.org/jira/browse/FALCON-1802 Project: Falcon Issue Type: Task Reporter: pavan kumar kolamuri Assignee: pavan kumar kolamuri In case of Native Scheduler to support Data based scheduling, evaluation of input paths, output paths and other properties is needed as part of Workflow Builder. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1774) Falcon to honour PRISM_URL env var
[
https://issues.apache.org/jira/browse/FALCON-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123408#comment-15123408
]
pavan kumar kolamuri commented on FALCON-1774:
--
[~ajayyadava] Why to add API just to say this is not supported. Even i am not
sure whats the best way, will explore and update.
> Falcon to honour PRISM_URL env var
> --
>
> Key: FALCON-1774
> URL: https://issues.apache.org/jira/browse/FALCON-1774
> Project: Falcon
> Issue Type: Improvement
>Reporter: Sanjeev T
>Assignee: Praveen Adlakha
> Attachments: FALCON-1774-v0.patch
>
>
> When we submit/delete/update process or feed, and we have falcon server url
> in client.properties, we cannot do the operation, and the output, doesn't
> make sense
> {noformat}
> $ falcon entity -type process -name bad-process -delete
> ERROR: Not Found;
> {noformat}
> In such cases we need to pass -url $PRISM_URL, where PRISM_URL points to
> prism host service.
> Requirement:
> * error message should say what is not found or expected PRISM_URL
> * Allow to set env var PRISM_URL
> * When we do submit/delete/update it should use the env variable
> * In case of schedule/suspend it should honour as specified in the
> client.properties
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1778) Check licenses and copyright information
[ https://issues.apache.org/jira/browse/FALCON-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123180#comment-15123180 ] pavan kumar kolamuri commented on FALCON-1778: -- +1 looks good > Check licenses and copyright information > > > Key: FALCON-1778 > URL: https://issues.apache.org/jira/browse/FALCON-1778 > Project: Falcon > Issue Type: Sub-task >Reporter: Pallavi Rao >Assignee: Pallavi Rao > Fix For: 0.9 > > Attachments: FALCON-1778-0.9.patch, FALCON-1778-trunk.patch, > FALCON-1778-v1-0.9.patch, FALCON-1778-v1-trunk.patch, > FALCON-1778-v2-0.9.patch, FALCON-1778-v2-trunk.patch, > FALCON-1778-v3-0.9.patch, FALCON-1778-v3-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1774) Falcon to honour PRISM_URL env var
[
https://issues.apache.org/jira/browse/FALCON-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121006#comment-15121006
]
pavan kumar kolamuri commented on FALCON-1774:
--
[~Praveen] Did u try to search any other alternate ways to solve this ? I go
through your solution but thinking any better way to solve this.
> Falcon to honour PRISM_URL env var
> --
>
> Key: FALCON-1774
> URL: https://issues.apache.org/jira/browse/FALCON-1774
> Project: Falcon
> Issue Type: Improvement
>Reporter: Sanjeev T
>Assignee: Praveen Adlakha
> Attachments: FALCON-1774-v0.patch
>
>
> When we submit/delete/update process or feed, and we have falcon server url
> in client.properties, we cannot do the operation, and the output, doesn't
> make sense
> {noformat}
> $ falcon entity -type process -name bad-process -delete
> ERROR: Not Found;
> {noformat}
> In such cases we need to pass -url $PRISM_URL, where PRISM_URL points to
> prism host service.
> Requirement:
> * error message should say what is not found or expected PRISM_URL
> * Allow to set env var PRISM_URL
> * When we do submit/delete/update it should use the env variable
> * In case of schedule/suspend it should honour as specified in the
> client.properties
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1778) Check licenses and copyright information
[ https://issues.apache.org/jira/browse/FALCON-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121016#comment-15121016 ] pavan kumar kolamuri commented on FALCON-1778: -- Thanks [~pallavi.rao] small question : We are also copying mysql-connector jar to client/lib, i think we should remove there also right ? > Check licenses and copyright information > > > Key: FALCON-1778 > URL: https://issues.apache.org/jira/browse/FALCON-1778 > Project: Falcon > Issue Type: Sub-task >Reporter: Pallavi Rao >Assignee: Pallavi Rao > Fix For: 0.9 > > Attachments: FALCON-1778-0.9.patch, FALCON-1778-trunk.patch, > FALCON-1778-v1-0.9.patch, FALCON-1778-v1-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1787) Ooozie pig-action.xml requires hive sharelib for HCatalog use
[ https://issues.apache.org/jira/browse/FALCON-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122992#comment-15122992 ] pavan kumar kolamuri commented on FALCON-1787: -- +1 Looks Good > Ooozie pig-action.xml requires hive sharelib for HCatalog use > - > > Key: FALCON-1787 > URL: https://issues.apache.org/jira/browse/FALCON-1787 > Project: Falcon > Issue Type: Bug > Components: oozie >Affects Versions: 0.6.1 > Environment: HDP-2.3.2.0-2950 > Pig 0.15.0.2.3 > Hive 1.2.1.2.3 > Oozie 4.2.0.2.3 > Falcon0.6.1.2.3 >Reporter: Mark Greene >Assignee: Sowmya Ramesh > Fix For: trunk > > Attachments: EDL-COMMON-CERTIFIED-PRF-OF-DLVR.xml, > EDL-COMMON-LOAD-DLVR-RCPT.xml, EDL-COMMON-PRF-OF-DLVR-LOAD.xml, > FALCON-1787.V0.patch, Stack Trace.txt, Workflow Job Configuration.txt, > Workflow Pig Action Configuration.txt, prf_of_dlvr_lz_to_cz.pig > > > I have a Pig script that I am using as the workflow for my Falcon process. > The pig script uses HCatalogStorer to write to a HCatalog URI that is the > output feed defined in my Falcon Process Entity. The Pig action in the > resulting Ooozie Workflow generated by Falcon fails with the attached stack > trace. The root is that it is missing a class definitions of > org/apache/hadoop/hive/shims/ShimLoader. > Running the script manually using pig -x tex -useHCatalog passed by Oozie> results in a successful execution. It's > only once this is called as a Pig activity in the Falcon-generated Oozie > workflow that the missing class definitions manifests. > After some investigation I found that the Oozie workflow.xml is missing a > required sharelib decleration. > From the workflow.xml generated by Falcon: > > oozie.action.sharelib.for.pig > pig,hcatalog > > If I modify the value to include hive sharelib then the Pig action succeeds > and does not throw a missing class definition error. > Modified workflow.xml property (works): > > oozie.action.sharelib.for.pig > hive,pig,hcatalog > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1609) Add entity commands to falcon spring-shell CLI
[ https://issues.apache.org/jira/browse/FALCON-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120836#comment-15120836 ] pavan kumar kolamuri commented on FALCON-1609: -- Thanks [~prongs] will look into it. > Add entity commands to falcon spring-shell CLI > -- > > Key: FALCON-1609 > URL: https://issues.apache.org/jira/browse/FALCON-1609 > Project: Falcon > Issue Type: Sub-task >Reporter: Rajat Khandelwal >Assignee: Rajat Khandelwal > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1609) Add entity commands to falcon spring-shell CLI
[ https://issues.apache.org/jira/browse/FALCON-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119861#comment-15119861 ] pavan kumar kolamuri commented on FALCON-1609: -- [~prongs] Thanks for working on this. Can you please rebase the patch and also upload the patch here as well. > Add entity commands to falcon spring-shell CLI > -- > > Key: FALCON-1609 > URL: https://issues.apache.org/jira/browse/FALCON-1609 > Project: Falcon > Issue Type: Sub-task >Reporter: Rajat Khandelwal >Assignee: Rajat Khandelwal > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1778) Check licenses and copyright information
[ https://issues.apache.org/jira/browse/FALCON-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119231#comment-15119231 ] pavan kumar kolamuri commented on FALCON-1778: -- [~pallavi.rao] I think with deepak's patch we need to also check the same for postgres. Can we also document the same in statestore.properties about this my-sql connector jar , it will be useful to users. > Check licenses and copyright information > > > Key: FALCON-1778 > URL: https://issues.apache.org/jira/browse/FALCON-1778 > Project: Falcon > Issue Type: Sub-task >Reporter: Pallavi Rao >Assignee: Pallavi Rao > Fix For: 0.9 > > Attachments: FALCON-1778-0.9.patch, FALCON-1778-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1548) Refactoring for Instance Execution operations in Native Scheduler
[ https://issues.apache.org/jira/browse/FALCON-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114949#comment-15114949 ] pavan kumar kolamuri commented on FALCON-1548: -- [~Praveen] Assigned to you. Thanks for taking it . > Refactoring for Instance Execution operations in Native Scheduler > - > > Key: FALCON-1548 > URL: https://issues.apache.org/jira/browse/FALCON-1548 > Project: Falcon > Issue Type: Improvement >Reporter: pavan kumar kolamuri >Assignee: Praveen Adlakha > Labels: newbie > > Refactoring for Instance Execution operations. Separate out InstanceExecutor > and InstanceBean -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1548) Refactoring for Instance Execution operations in Native Scheduler
[ https://issues.apache.org/jira/browse/FALCON-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1548: - Assignee: Praveen Adlakha (was: pavan kumar kolamuri) > Refactoring for Instance Execution operations in Native Scheduler > - > > Key: FALCON-1548 > URL: https://issues.apache.org/jira/browse/FALCON-1548 > Project: Falcon > Issue Type: Improvement >Reporter: pavan kumar kolamuri >Assignee: Praveen Adlakha > Labels: newbie > > Refactoring for Instance Execution operations. Separate out InstanceExecutor > and InstanceBean -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1758) APIs fail when oozie workflow entries are deleted
[
https://issues.apache.org/jira/browse/FALCON-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pavan kumar kolamuri updated FALCON-1758:
-
Attachment: FALCON-1758.patch
> APIs fail when oozie workflow entries are deleted
> -
>
> Key: FALCON-1758
> URL: https://issues.apache.org/jira/browse/FALCON-1758
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Fix For: 0.9
>
> Attachments: FALCON-1758.patch
>
>
> Whenever a process is scheduled in Falcon Native Scheduler and instances are
> running , later if those workflow entries got deleted , then entity deletion
> fails with following exception
> {noformat}
> 2016-01-19 14:41:42,261 INFO - [ActiveMQ Session Task-56:] ~ Logging in
> pragya (CurrentUser:65)
> 2016-01-19 14:41:42,261 INFO - [ActiveMQ Session Task-56:] ~ Creating Oozie
> client object for http://192.168.138.236:11000/oozie/ (OozieClientFactory:50)
> 2016-01-19 14:41:42,267 DEBUG - [ActiveMQ Session Task-56:] ~ Error while
> retrieving JMS connection info (OozieWorkflowEngine:613)
> E1601 : E1601: Cannot retrieve JMS connection info [JMSTopicService is not
> initialized. JMS notificationmay not be enabled]
> at
> org.apache.oozie.client.OozieClient.handleError(OozieClient.java:542)
> at
> org.apache.oozie.client.OozieClient$JMSInfo.call(OozieClient.java:869)
> at
> org.apache.oozie.client.OozieClient$JMSInfo.call(OozieClient.java:856)
> at
> org.apache.oozie.client.OozieClient$ClientCallable.call(OozieClient.java:514)
> at
> org.apache.oozie.client.OozieClient.getJMSConnectionInfo(OozieClient.java:912)
> at
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.isNotificationEnabled(OozieWorkflowEngine.java:604)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:214)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifySuccess(WorkflowJobEndNotificationService.java:105)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:218)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:114)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-19 14:41:42,267 INFO - [ActiveMQ Session Task-56:] ~ Creating Oozie
> client object for http://192.168.138.236:11000/oozie/ (OozieClientFactory:50)
> {noformat}
> However if the same is done for entities scheduled with oozie scheduler,
> entity deletion happens successfully.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1758) APIs fail when oozie workflow entries are deleted
[
https://issues.apache.org/jira/browse/FALCON-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114218#comment-15114218
]
pavan kumar kolamuri commented on FALCON-1758:
--
After an offline discussion with [~pallavi.rao] [~ajayyadava] [~pragya.mittal]
regarding delete API. Updated the patch with corresponding changes. Delete API
might fail in case of oozie not responding or activemq slow or down. In case
of failure in delete API , there is no guarantee that process runs as it is
further . It's user responsibility to delete completely later.
> APIs fail when oozie workflow entries are deleted
> -
>
> Key: FALCON-1758
> URL: https://issues.apache.org/jira/browse/FALCON-1758
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Fix For: 0.9
>
> Attachments: FALCON-1758.patch
>
>
> Whenever a process is scheduled in Falcon Native Scheduler and instances are
> running , later if those workflow entries got deleted , then entity deletion
> fails with following exception
> {noformat}
> 2016-01-19 14:41:42,261 INFO - [ActiveMQ Session Task-56:] ~ Logging in
> pragya (CurrentUser:65)
> 2016-01-19 14:41:42,261 INFO - [ActiveMQ Session Task-56:] ~ Creating Oozie
> client object for http://192.168.138.236:11000/oozie/ (OozieClientFactory:50)
> 2016-01-19 14:41:42,267 DEBUG - [ActiveMQ Session Task-56:] ~ Error while
> retrieving JMS connection info (OozieWorkflowEngine:613)
> E1601 : E1601: Cannot retrieve JMS connection info [JMSTopicService is not
> initialized. JMS notificationmay not be enabled]
> at
> org.apache.oozie.client.OozieClient.handleError(OozieClient.java:542)
> at
> org.apache.oozie.client.OozieClient$JMSInfo.call(OozieClient.java:869)
> at
> org.apache.oozie.client.OozieClient$JMSInfo.call(OozieClient.java:856)
> at
> org.apache.oozie.client.OozieClient$ClientCallable.call(OozieClient.java:514)
> at
> org.apache.oozie.client.OozieClient.getJMSConnectionInfo(OozieClient.java:912)
> at
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.isNotificationEnabled(OozieWorkflowEngine.java:604)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:214)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifySuccess(WorkflowJobEndNotificationService.java:105)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:218)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:114)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-19 14:41:42,267 INFO - [ActiveMQ Session Task-56:] ~ Creating Oozie
> client object for http://192.168.138.236:11000/oozie/ (OozieClientFactory:50)
> {noformat}
> However if the same is done for entities scheduled with oozie scheduler,
> entity deletion happens successfully.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1716) API fails with CommunicationsException when mysql interaction time is longer than 53,434,939 milliseconds
[
https://issues.apache.org/jira/browse/FALCON-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15112277#comment-15112277
]
pavan kumar kolamuri commented on FALCON-1716:
--
Updated the patch please review [~pallavi.rao] [~ajayyadava]
> API fails with CommunicationsException when mysql interaction time is longer
> than 53,434,939 milliseconds
> -
>
> Key: FALCON-1716
> URL: https://issues.apache.org/jira/browse/FALCON-1716
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: FALCON-1716-v1.patch, FALCON-1716.patch,
> jdbc-timedout.log
>
>
> If mysql is not interacted for less than 53,434,939 milliseconds, the query
> fails with CommunicationsException for first few attempts after which
> connection is set. We should configure communication window for the same.
> Error is :
> {noformat}
> 2015-12-30 06:54:47,294 ERROR - [357665612@qtp-1816147548-10 -
> 8ebf22cf-4f82-4280-a248-fa3ca6048042:dataqa:GET//instance/params/process/ProcessInstanceStatusTest-agregator-coord16-50a6b241]
> ~ Failed to display params of an instance (AbstractInstanceManager:553)
>
> org.apache.openjpa.persistence.PersistenceException: The last packet
> successfully received from the server was 53,434,939 milliseconds ago. The
> last packet sent successfully to the server was 53,434,939 milliseconds ago.
> is longer than the server configured value of 'wait_timeout'. You should
> consider either expiring and/or testing connection validity before use in
> your application, increasing the server configured values for client
> timeouts, or using the Connector/J connection property 'autoReconnect=true'
> to avoid this problem.
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:5001)
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4961)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:133)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:107)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:59)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.connect(JDBCStoreManager.java:967)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.getConnection(JDBCStoreManager.java:241)
> at
> org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:104)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1004)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:862)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:793)
> at
> org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:541)
> at
> org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:274)
> at
> org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:290)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.getEntityByKey(JDBCStateStore.java:95)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.entityExists(JDBCStateStore.java:105)
> at
> org.apache.falcon.workflow.engine.FalconWorkflowEngine.isActive(FalconWorkflowEngine.java:98)
> at
> org.apache.falcon.workflow.WorkflowEngineFactory.getWorkflowEngine(WorkflowEngineFactory.java:54)
> at
> org.apache.falcon.resource.AbstractEntityManager.getWorkflowEngine(AbstractEntityManager.java:1132)
> at
> org.apache.falcon.resource.AbstractInstanceManager.getInstanceParams(AbstractInstanceManager.java:550)
> at
> org.apache.falcon.resource.InstanceManager.getInstanceParams(InstanceManager.java:178)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1664) Add Postgres support for native scheduler
[ https://issues.apache.org/jira/browse/FALCON-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110488#comment-15110488 ] pavan kumar kolamuri commented on FALCON-1664: -- +1 for the patch > Add Postgres support for native scheduler > - > > Key: FALCON-1664 > URL: https://issues.apache.org/jira/browse/FALCON-1664 > Project: Falcon > Issue Type: Improvement > Components: scheduler >Reporter: Ajay Yadava >Assignee: Deepak Barr > Labels: newbie > Attachments: FALCON-1664-v1.patch, FALCON-1664-v2.patch > > > Currently native scheduler doesn't have support for Postgres database. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1757) EntityNotRegisteredException when entity is deleted from falcon
[
https://issues.apache.org/jira/browse/FALCON-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pavan kumar kolamuri updated FALCON-1757:
-
Attachment: FALCON-1757.patch
> EntityNotRegisteredException when entity is deleted from falcon
> ---
>
> Key: FALCON-1757
> URL: https://issues.apache.org/jira/browse/FALCON-1757
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Fix For: 0.9
>
> Attachments: FALCON-1757.patch
>
>
> When entity is deleted from falcon (which is scheduled using native
> scheduler), the following exception is traced in logs :
> {noformat}
> 2016-01-19 13:07:42,975 INFO - [RetryHandler:] ~ Logging in pragya
> (CurrentUser:65)
> 2016-01-19 13:07:42,975 ERROR - [RetryHandler:] ~ Error in rerun consumer
> (AbstractRerunConsumer:86)
> org.apache.falcon.entity.EntityNotRegisteredException: processMerlinNative
> (PROCESS) not found
> at org.apache.falcon.entity.EntityUtil.getEntity(EntityUtil.java:121)
> at
> org.apache.falcon.rerun.handler.AbstractRerunHandler.getWfEngine(AbstractRerunHandler.java:67)
> at
> org.apache.falcon.rerun.handler.AbstractRerunConsumer.run(AbstractRerunConsumer.java:78)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-19 13:09:03,241 DEBUG - [RetryHandler:] ~ Dequeued Message:
> clusterName=ProcessMultipleClustersTest-corp-40de481b*wfId=0001377-160118061527782-oozie-oozi-W*msgInsertTime=1453208763241*delayInMilliSec=18*entityType=PROCESS*entityName=processMerlinNative*instance=2016-01-19T12:07Z*runId=1*attempts=3*failRetryCount=0*workflowUser=dataqa
> (DelayedQueue:70)
> {noformat}
> This happens because retry is enabled for the process and oozie tries to
> rerun the job while it is deleted from falcon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1757) EntityNotRegisteredException when entity is deleted from falcon
[
https://issues.apache.org/jira/browse/FALCON-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110237#comment-15110237
]
pavan kumar kolamuri commented on FALCON-1757:
--
By the time falcon get failure notifications of process instances from oozie
there is chance that entity was deleted before re-running the workflow. In that
case Falcon is throwing exception saying rerun consumer failed. We fixed it by
catching EntityNotRegistered Exception and logged it as warning.
> EntityNotRegisteredException when entity is deleted from falcon
> ---
>
> Key: FALCON-1757
> URL: https://issues.apache.org/jira/browse/FALCON-1757
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Fix For: 0.9
>
> Attachments: FALCON-1757.patch
>
>
> When entity is deleted from falcon (which is scheduled using native
> scheduler), the following exception is traced in logs :
> {noformat}
> 2016-01-19 13:07:42,975 INFO - [RetryHandler:] ~ Logging in pragya
> (CurrentUser:65)
> 2016-01-19 13:07:42,975 ERROR - [RetryHandler:] ~ Error in rerun consumer
> (AbstractRerunConsumer:86)
> org.apache.falcon.entity.EntityNotRegisteredException: processMerlinNative
> (PROCESS) not found
> at org.apache.falcon.entity.EntityUtil.getEntity(EntityUtil.java:121)
> at
> org.apache.falcon.rerun.handler.AbstractRerunHandler.getWfEngine(AbstractRerunHandler.java:67)
> at
> org.apache.falcon.rerun.handler.AbstractRerunConsumer.run(AbstractRerunConsumer.java:78)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-19 13:09:03,241 DEBUG - [RetryHandler:] ~ Dequeued Message:
> clusterName=ProcessMultipleClustersTest-corp-40de481b*wfId=0001377-160118061527782-oozie-oozi-W*msgInsertTime=1453208763241*delayInMilliSec=18*entityType=PROCESS*entityName=processMerlinNative*instance=2016-01-19T12:07Z*runId=1*attempts=3*failRetryCount=0*workflowUser=dataqa
> (DelayedQueue:70)
> {noformat}
> This happens because retry is enabled for the process and oozie tries to
> rerun the job while it is deleted from falcon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1682) Falcon server starts successfully even if application services fail to start
[
https://issues.apache.org/jira/browse/FALCON-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106426#comment-15106426
]
pavan kumar kolamuri commented on FALCON-1682:
--
Looks like it's not a small change to change jetty to latest version as
configuring ssl and other things were changed a lot and there might be other
things as well, we can go with this hack for now . We can raise a jira and we
can fix this cleanly in next version.
[~pallavi.rao] [~pragya.mittal] does it make sense ?
> Falcon server starts successfully even if application services fail to start
>
>
> Key: FALCON-1682
> URL: https://issues.apache.org/jira/browse/FALCON-1682
> Project: Falcon
> Issue Type: Bug
> Components: general
>Affects Versions: 0.8
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: FALCON-1682.patch
>
>
> If falcon is configured to run with mysql db, and user does not create db
> then server start should fail and throw error for the same. But server starts
> successfully as of now. Although error is logged in server logs saying :
> {noformat}
> 2015-12-21 13:41:01,899 ERROR - [main:] ~ Failed to initialize service
> org.apache.falcon.state.store.service.FalconJPAService (ServiceInitializer:49)
>
> org.apache.openjpa.persistence.PersistenceException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1518)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.falcon.state.store.service.FalconJPAService.getEntityManager(FalconJPAService.java:169)
> at
> org.apache.falcon.state.store.service.FalconJPAService.init(FalconJPAService.java:91)
> at
> org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:47)
> at
> org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:56)
> at
> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:550)
> at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:519)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:57)
> at org.apache.falcon.FalconServer.main(FalconServer.java:102)
> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (FALCON-1760) Upgrade jetty version and fix Falcon start issue while Application initialization fails
pavan kumar kolamuri created FALCON-1760: Summary: Upgrade jetty version and fix Falcon start issue while Application initialization fails Key: FALCON-1760 URL: https://issues.apache.org/jira/browse/FALCON-1760 Project: Falcon Issue Type: Improvement Reporter: pavan kumar kolamuri https://issues.apache.org/jira/browse/FALCON-1682 for more details -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1716) API fails with CommunicationsException when mysql interaction time is longer than 53,434,939 milliseconds
[
https://issues.apache.org/jira/browse/FALCON-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106715#comment-15106715
]
pavan kumar kolamuri commented on FALCON-1716:
--
In Connection Pool of JDBC, there are pool of connections min and maximum .
Whenever we made a call to DB, it will take object from that connection pool
and do db operation. There are few properties which we can configure and avoid
socketTimeout exception one of those is .
testOnBorrow: The indication of whether objects will be validated before being
borrowed from the pool. If the object fails to validate, it will be dropped
from the pool, and we will attempt to borrow another.
It was set to false because of that property which i changed in
statestore.properties. If it is set to true it will validate connection and try
to get the new connection when connection is idle for long time.
Test query is very inexpensive and oozie also doing same.
> API fails with CommunicationsException when mysql interaction time is longer
> than 53,434,939 milliseconds
> -
>
> Key: FALCON-1716
> URL: https://issues.apache.org/jira/browse/FALCON-1716
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: FALCON-1716.patch, jdbc-timedout.log
>
>
> If mysql is not interacted for less than 53,434,939 milliseconds, the query
> fails with CommunicationsException for first few attempts after which
> connection is set. We should configure communication window for the same.
> Error is :
> {noformat}
> 2015-12-30 06:54:47,294 ERROR - [357665612@qtp-1816147548-10 -
> 8ebf22cf-4f82-4280-a248-fa3ca6048042:dataqa:GET//instance/params/process/ProcessInstanceStatusTest-agregator-coord16-50a6b241]
> ~ Failed to display params of an instance (AbstractInstanceManager:553)
>
> org.apache.openjpa.persistence.PersistenceException: The last packet
> successfully received from the server was 53,434,939 milliseconds ago. The
> last packet sent successfully to the server was 53,434,939 milliseconds ago.
> is longer than the server configured value of 'wait_timeout'. You should
> consider either expiring and/or testing connection validity before use in
> your application, increasing the server configured values for client
> timeouts, or using the Connector/J connection property 'autoReconnect=true'
> to avoid this problem.
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:5001)
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4961)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:133)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:107)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:59)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.connect(JDBCStoreManager.java:967)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.getConnection(JDBCStoreManager.java:241)
> at
> org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:104)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1004)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:862)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:793)
> at
> org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:541)
> at
> org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:274)
> at
> org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:290)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.getEntityByKey(JDBCStateStore.java:95)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.entityExists(JDBCStateStore.java:105)
> at
> org.apache.falcon.workflow.engine.FalconWorkflowEngine.isActive(FalconWorkflowEngine.java:98)
> at
> org.apache.falcon.workflow.WorkflowEngineFactory.getWorkflowEngine(WorkflowEngineFactory.java:54)
> at
> org.apache.falcon.resource.AbstractEntityManager.getWorkflowEngine(AbstractEntityManager.java:1132)
> at
> org.apache.falcon.resource.AbstractInstanceManager.getInstanceParams(AbstractInstanceManager.java:550)
> at
> org.apache.falcon.resource.InstanceManager.getInstanceParams(InstanceManager.java:178)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
[jira] [Commented] (FALCON-1723) Rerun with skip fail actions won't work in few cases
[ https://issues.apache.org/jira/browse/FALCON-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106719#comment-15106719 ] pavan kumar kolamuri commented on FALCON-1723: -- [~pallavi.rao] Can you try re-basing now. > Rerun with skip fail actions won't work in few cases > > > Key: FALCON-1723 > URL: https://issues.apache.org/jira/browse/FALCON-1723 > Project: Falcon > Issue Type: Bug > Components: process >Affects Versions: 0.8 >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Attachments: FALCON-1723-v1.patch, FALCON-1723-v2.patch, > FALCON-1723.patch > > > Normal Rerun - Rerunning an instance without force, it will rerun only failed > actions > Force Rerun - Rerunning an instance forcefully either by using force option > or passing props , it will run all actions irrespective of status > Normal Rerun won't work if force done once. And also we should check whether > force option is ignored if user also related properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1716) API fails with CommunicationsException when mysql interaction time is longer than 53,434,939 milliseconds
[
https://issues.apache.org/jira/browse/FALCON-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pavan kumar kolamuri updated FALCON-1716:
-
Attachment: FALCON-1716.patch
> API fails with CommunicationsException when mysql interaction time is longer
> than 53,434,939 milliseconds
> -
>
> Key: FALCON-1716
> URL: https://issues.apache.org/jira/browse/FALCON-1716
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: FALCON-1716.patch, jdbc-timedout.log
>
>
> If mysql is not interacted for less than 53,434,939 milliseconds, the query
> fails with CommunicationsException for first few attempts after which
> connection is set. We should configure communication window for the same.
> Error is :
> {noformat}
> 2015-12-30 06:54:47,294 ERROR - [357665612@qtp-1816147548-10 -
> 8ebf22cf-4f82-4280-a248-fa3ca6048042:dataqa:GET//instance/params/process/ProcessInstanceStatusTest-agregator-coord16-50a6b241]
> ~ Failed to display params of an instance (AbstractInstanceManager:553)
>
> org.apache.openjpa.persistence.PersistenceException: The last packet
> successfully received from the server was 53,434,939 milliseconds ago. The
> last packet sent successfully to the server was 53,434,939 milliseconds ago.
> is longer than the server configured value of 'wait_timeout'. You should
> consider either expiring and/or testing connection validity before use in
> your application, increasing the server configured values for client
> timeouts, or using the Connector/J connection property 'autoReconnect=true'
> to avoid this problem.
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:5001)
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4961)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:133)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:107)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:59)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.connect(JDBCStoreManager.java:967)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.getConnection(JDBCStoreManager.java:241)
> at
> org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:104)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1004)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:862)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:793)
> at
> org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:541)
> at
> org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:274)
> at
> org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:290)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.getEntityByKey(JDBCStateStore.java:95)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.entityExists(JDBCStateStore.java:105)
> at
> org.apache.falcon.workflow.engine.FalconWorkflowEngine.isActive(FalconWorkflowEngine.java:98)
> at
> org.apache.falcon.workflow.WorkflowEngineFactory.getWorkflowEngine(WorkflowEngineFactory.java:54)
> at
> org.apache.falcon.resource.AbstractEntityManager.getWorkflowEngine(AbstractEntityManager.java:1132)
> at
> org.apache.falcon.resource.AbstractInstanceManager.getInstanceParams(AbstractInstanceManager.java:550)
> at
> org.apache.falcon.resource.InstanceManager.getInstanceParams(InstanceManager.java:178)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1716) API fails with CommunicationsException when mysql interaction time is longer than 53,434,939 milliseconds
[
https://issues.apache.org/jira/browse/FALCON-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1510#comment-1510
]
pavan kumar kolamuri commented on FALCON-1716:
--
I am not putting this on review board since it is very small patch. Please
review
> API fails with CommunicationsException when mysql interaction time is longer
> than 53,434,939 milliseconds
> -
>
> Key: FALCON-1716
> URL: https://issues.apache.org/jira/browse/FALCON-1716
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: FALCON-1716.patch, jdbc-timedout.log
>
>
> If mysql is not interacted for less than 53,434,939 milliseconds, the query
> fails with CommunicationsException for first few attempts after which
> connection is set. We should configure communication window for the same.
> Error is :
> {noformat}
> 2015-12-30 06:54:47,294 ERROR - [357665612@qtp-1816147548-10 -
> 8ebf22cf-4f82-4280-a248-fa3ca6048042:dataqa:GET//instance/params/process/ProcessInstanceStatusTest-agregator-coord16-50a6b241]
> ~ Failed to display params of an instance (AbstractInstanceManager:553)
>
> org.apache.openjpa.persistence.PersistenceException: The last packet
> successfully received from the server was 53,434,939 milliseconds ago. The
> last packet sent successfully to the server was 53,434,939 milliseconds ago.
> is longer than the server configured value of 'wait_timeout'. You should
> consider either expiring and/or testing connection validity before use in
> your application, increasing the server configured values for client
> timeouts, or using the Connector/J connection property 'autoReconnect=true'
> to avoid this problem.
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:5001)
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4961)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:133)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:107)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:59)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.connect(JDBCStoreManager.java:967)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.getConnection(JDBCStoreManager.java:241)
> at
> org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:104)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1004)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:862)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:793)
> at
> org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:541)
> at
> org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:274)
> at
> org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:290)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.getEntityByKey(JDBCStateStore.java:95)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.entityExists(JDBCStateStore.java:105)
> at
> org.apache.falcon.workflow.engine.FalconWorkflowEngine.isActive(FalconWorkflowEngine.java:98)
> at
> org.apache.falcon.workflow.WorkflowEngineFactory.getWorkflowEngine(WorkflowEngineFactory.java:54)
> at
> org.apache.falcon.resource.AbstractEntityManager.getWorkflowEngine(AbstractEntityManager.java:1132)
> at
> org.apache.falcon.resource.AbstractInstanceManager.getInstanceParams(AbstractInstanceManager.java:550)
> at
> org.apache.falcon.resource.InstanceManager.getInstanceParams(InstanceManager.java:178)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FALCON-1723) Rerun with skip fail actions won't work in few cases
[ https://issues.apache.org/jira/browse/FALCON-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1723: - Attachment: FALCON-1723-v2.patch > Rerun with skip fail actions won't work in few cases > > > Key: FALCON-1723 > URL: https://issues.apache.org/jira/browse/FALCON-1723 > Project: Falcon > Issue Type: Bug > Components: process >Affects Versions: 0.8 >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Attachments: FALCON-1723-v1.patch, FALCON-1723-v2.patch, > FALCON-1723.patch > > > Normal Rerun - Rerunning an instance without force, it will rerun only failed > actions > Force Rerun - Rerunning an instance forcefully either by using force option > or passing props , it will run all actions irrespective of status > Normal Rerun won't work if force done once. And also we should check whether > force option is ignored if user also related properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1716) API fails with CommunicationsException when mysql interaction time is longer than 53,434,939 milliseconds
[
https://issues.apache.org/jira/browse/FALCON-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106665#comment-15106665
]
pavan kumar kolamuri commented on FALCON-1716:
--
Because of wrong default value of create.db.schema=false idle connections are
not checked. Now idle connections are checked every 5 mins with inexpensive
query.
> API fails with CommunicationsException when mysql interaction time is longer
> than 53,434,939 milliseconds
> -
>
> Key: FALCON-1716
> URL: https://issues.apache.org/jira/browse/FALCON-1716
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: FALCON-1716.patch, jdbc-timedout.log
>
>
> If mysql is not interacted for less than 53,434,939 milliseconds, the query
> fails with CommunicationsException for first few attempts after which
> connection is set. We should configure communication window for the same.
> Error is :
> {noformat}
> 2015-12-30 06:54:47,294 ERROR - [357665612@qtp-1816147548-10 -
> 8ebf22cf-4f82-4280-a248-fa3ca6048042:dataqa:GET//instance/params/process/ProcessInstanceStatusTest-agregator-coord16-50a6b241]
> ~ Failed to display params of an instance (AbstractInstanceManager:553)
>
> org.apache.openjpa.persistence.PersistenceException: The last packet
> successfully received from the server was 53,434,939 milliseconds ago. The
> last packet sent successfully to the server was 53,434,939 milliseconds ago.
> is longer than the server configured value of 'wait_timeout'. You should
> consider either expiring and/or testing connection validity before use in
> your application, increasing the server configured values for client
> timeouts, or using the Connector/J connection property 'autoReconnect=true'
> to avoid this problem.
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:5001)
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4961)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:133)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:107)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:59)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.connect(JDBCStoreManager.java:967)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.getConnection(JDBCStoreManager.java:241)
> at
> org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:104)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1004)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:862)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:793)
> at
> org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:541)
> at
> org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:274)
> at
> org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:290)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.getEntityByKey(JDBCStateStore.java:95)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.entityExists(JDBCStateStore.java:105)
> at
> org.apache.falcon.workflow.engine.FalconWorkflowEngine.isActive(FalconWorkflowEngine.java:98)
> at
> org.apache.falcon.workflow.WorkflowEngineFactory.getWorkflowEngine(WorkflowEngineFactory.java:54)
> at
> org.apache.falcon.resource.AbstractEntityManager.getWorkflowEngine(AbstractEntityManager.java:1132)
> at
> org.apache.falcon.resource.AbstractInstanceManager.getInstanceParams(AbstractInstanceManager.java:550)
> at
> org.apache.falcon.resource.InstanceManager.getInstanceParams(InstanceManager.java:178)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> {noformat}
--
This message was sent by Atlassian JIRA
[jira] [Commented] (FALCON-1723) Rerun with skip fail actions won't work in few cases
[ https://issues.apache.org/jira/browse/FALCON-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106816#comment-15106816 ] pavan kumar kolamuri commented on FALCON-1723: -- I have submit patch for 0.9 branch. Please use FALCON-1723-v2.patch for trunk and FALCON-1723-0.9-branch.patch for 0.9 branch. > Rerun with skip fail actions won't work in few cases > > > Key: FALCON-1723 > URL: https://issues.apache.org/jira/browse/FALCON-1723 > Project: Falcon > Issue Type: Bug > Components: process >Affects Versions: 0.8 >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Attachments: FALCON-1723-0.9-branch.patch, FALCON-1723-v1.patch, > FALCON-1723-v2.patch, FALCON-1723.patch > > > Normal Rerun - Rerunning an instance without force, it will rerun only failed > actions > Force Rerun - Rerunning an instance forcefully either by using force option > or passing props , it will run all actions irrespective of status > Normal Rerun won't work if force done once. And also we should check whether > force option is ignored if user also related properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1723) Rerun with skip fail actions won't work in few cases
[ https://issues.apache.org/jira/browse/FALCON-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1723: - Attachment: FALCON-1723-0.9-branch.patch > Rerun with skip fail actions won't work in few cases > > > Key: FALCON-1723 > URL: https://issues.apache.org/jira/browse/FALCON-1723 > Project: Falcon > Issue Type: Bug > Components: process >Affects Versions: 0.8 >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Attachments: FALCON-1723-0.9-branch.patch, FALCON-1723-v1.patch, > FALCON-1723-v2.patch, FALCON-1723.patch > > > Normal Rerun - Rerunning an instance without force, it will rerun only failed > actions > Force Rerun - Rerunning an instance forcefully either by using force option > or passing props , it will run all actions irrespective of status > Normal Rerun won't work if force done once. And also we should check whether > force option is ignored if user also related properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (FALCON-1758) APIs fail when oozie workflow entries are deleted
[
https://issues.apache.org/jira/browse/FALCON-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pavan kumar kolamuri reassigned FALCON-1758:
Assignee: pavan kumar kolamuri
> APIs fail when oozie workflow entries are deleted
> -
>
> Key: FALCON-1758
> URL: https://issues.apache.org/jira/browse/FALCON-1758
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Fix For: 0.9
>
>
> Whenever a process is scheduled in Falcon Native Scheduler and instances are
> running , later if those workflow entries got deleted , then entity deletion
> fails with following exception
> {noformat}
> 2016-01-19 14:41:42,280 ERROR - [ActiveMQ Session Task-56:] ~ Error in
> listener org.apache.falcon.handler.SLAMonitoringHandler
> (WorkflowJobEndNotificationService:245)
> java.lang.NullPointerException
> at
> org.apache.falcon.service.FeedSLAMonitoringService.makeFeedInstanceAvailable(FeedSLAMonitoringService.java:273)
> at
> org.apache.falcon.handler.SLAMonitoringHandler.updateSLAMonitoring(SLAMonitoringHandler.java:63)
> at
> org.apache.falcon.handler.SLAMonitoringHandler.onSuccess(SLAMonitoringHandler.java:48)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:235)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifySuccess(WorkflowJobEndNotificationService.java:105)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:218)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:114)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-19 14:41:42,280 INFO - [ActiveMQ Session Task-56:] ~ Late date
> handling not applicable for entityType: PROCESS, entityName:
> LogMoverTest-agregator-coord16-3c67a0aa operation: GENERATE
> (LateRerunHandler:231)
> {noformat}
> However if the same is done for entities scheduled with oozie scheduler,
> entity deletion happens successfully.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1758) APIs fail when oozie workflow entries are deleted
[
https://issues.apache.org/jira/browse/FALCON-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15107003#comment-15107003
]
pavan kumar kolamuri commented on FALCON-1758:
--
[~pragya.mittal] Looks like stacktrace copied was not related can you confirm
is it correct or not ?
> APIs fail when oozie workflow entries are deleted
> -
>
> Key: FALCON-1758
> URL: https://issues.apache.org/jira/browse/FALCON-1758
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Fix For: 0.9
>
>
> Whenever a process is scheduled in Falcon Native Scheduler and instances are
> running , later if those workflow entries got deleted , then entity deletion
> fails with following exception
> {noformat}
> 2016-01-19 14:41:42,280 ERROR - [ActiveMQ Session Task-56:] ~ Error in
> listener org.apache.falcon.handler.SLAMonitoringHandler
> (WorkflowJobEndNotificationService:245)
> java.lang.NullPointerException
> at
> org.apache.falcon.service.FeedSLAMonitoringService.makeFeedInstanceAvailable(FeedSLAMonitoringService.java:273)
> at
> org.apache.falcon.handler.SLAMonitoringHandler.updateSLAMonitoring(SLAMonitoringHandler.java:63)
> at
> org.apache.falcon.handler.SLAMonitoringHandler.onSuccess(SLAMonitoringHandler.java:48)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:235)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifySuccess(WorkflowJobEndNotificationService.java:105)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:218)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:114)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-19 14:41:42,280 INFO - [ActiveMQ Session Task-56:] ~ Late date
> handling not applicable for entityType: PROCESS, entityName:
> LogMoverTest-agregator-coord16-3c67a0aa operation: GENERATE
> (LateRerunHandler:231)
> {noformat}
> However if the same is done for entities scheduled with oozie scheduler,
> entity deletion happens successfully.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (FALCON-1757) EntityNotRegisteredException when entity is deleted from falcon
[
https://issues.apache.org/jira/browse/FALCON-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pavan kumar kolamuri reassigned FALCON-1757:
Assignee: pavan kumar kolamuri
> EntityNotRegisteredException when entity is deleted from falcon
> ---
>
> Key: FALCON-1757
> URL: https://issues.apache.org/jira/browse/FALCON-1757
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Fix For: 0.9
>
>
> When entity is deleted from falcon (which is scheduled using native
> scheduler), the following exception is traced in logs :
> {noformat}
> 2016-01-19 13:07:42,975 INFO - [RetryHandler:] ~ Logging in pragya
> (CurrentUser:65)
> 2016-01-19 13:07:42,975 ERROR - [RetryHandler:] ~ Error in rerun consumer
> (AbstractRerunConsumer:86)
> org.apache.falcon.entity.EntityNotRegisteredException: processMerlinNative
> (PROCESS) not found
> at org.apache.falcon.entity.EntityUtil.getEntity(EntityUtil.java:121)
> at
> org.apache.falcon.rerun.handler.AbstractRerunHandler.getWfEngine(AbstractRerunHandler.java:67)
> at
> org.apache.falcon.rerun.handler.AbstractRerunConsumer.run(AbstractRerunConsumer.java:78)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-19 13:09:03,241 DEBUG - [RetryHandler:] ~ Dequeued Message:
> clusterName=ProcessMultipleClustersTest-corp-40de481b*wfId=0001377-160118061527782-oozie-oozi-W*msgInsertTime=1453208763241*delayInMilliSec=18*entityType=PROCESS*entityName=processMerlinNative*instance=2016-01-19T12:07Z*runId=1*attempts=3*failRetryCount=0*workflowUser=dataqa
> (DelayedQueue:70)
> {noformat}
> This happens because retry is enabled for the process and oozie tries to
> rerun the job while it is deleted from falcon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1682) Falcon server starts successfully even if application services fail to start
[
https://issues.apache.org/jira/browse/FALCON-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105407#comment-15105407
]
pavan kumar kolamuri commented on FALCON-1682:
--
I have explored about them we need to bump up the jetty version to do this
cleanly. Thoughts please
> Falcon server starts successfully even if application services fail to start
>
>
> Key: FALCON-1682
> URL: https://issues.apache.org/jira/browse/FALCON-1682
> Project: Falcon
> Issue Type: Bug
> Components: general
>Affects Versions: 0.8
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: FALCON-1682.patch
>
>
> If falcon is configured to run with mysql db, and user does not create db
> then server start should fail and throw error for the same. But server starts
> successfully as of now. Although error is logged in server logs saying :
> {noformat}
> 2015-12-21 13:41:01,899 ERROR - [main:] ~ Failed to initialize service
> org.apache.falcon.state.store.service.FalconJPAService (ServiceInitializer:49)
>
> org.apache.openjpa.persistence.PersistenceException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1518)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.falcon.state.store.service.FalconJPAService.getEntityManager(FalconJPAService.java:169)
> at
> org.apache.falcon.state.store.service.FalconJPAService.init(FalconJPAService.java:91)
> at
> org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:47)
> at
> org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:56)
> at
> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:550)
> at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:519)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:57)
> at org.apache.falcon.FalconServer.main(FalconServer.java:102)
> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FALCON-1723) Rerun with skip fail actions won't work in few cases
[ https://issues.apache.org/jira/browse/FALCON-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1723: - Attachment: FALCON-1723-v1.patch > Rerun with skip fail actions won't work in few cases > > > Key: FALCON-1723 > URL: https://issues.apache.org/jira/browse/FALCON-1723 > Project: Falcon > Issue Type: Bug > Components: process >Affects Versions: 0.8 >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Attachments: FALCON-1723-v1.patch, FALCON-1723.patch > > > Normal Rerun - Rerunning an instance without force, it will rerun only failed > actions > Force Rerun - Rerunning an instance forcefully either by using force option > or passing props , it will run all actions irrespective of status > Normal Rerun won't work if force done once. And also we should check whether > force option is ignored if user also related properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1682) Falcon server starts successfully even if application services fail to start
[
https://issues.apache.org/jira/browse/FALCON-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106297#comment-15106297
]
pavan kumar kolamuri commented on FALCON-1682:
--
Even i agree same, i will go ahead and change jetty version and fix this.
> Falcon server starts successfully even if application services fail to start
>
>
> Key: FALCON-1682
> URL: https://issues.apache.org/jira/browse/FALCON-1682
> Project: Falcon
> Issue Type: Bug
> Components: general
>Affects Versions: 0.8
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: FALCON-1682.patch
>
>
> If falcon is configured to run with mysql db, and user does not create db
> then server start should fail and throw error for the same. But server starts
> successfully as of now. Although error is logged in server logs saying :
> {noformat}
> 2015-12-21 13:41:01,899 ERROR - [main:] ~ Failed to initialize service
> org.apache.falcon.state.store.service.FalconJPAService (ServiceInitializer:49)
>
> org.apache.openjpa.persistence.PersistenceException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1518)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.falcon.state.store.service.FalconJPAService.getEntityManager(FalconJPAService.java:169)
> at
> org.apache.falcon.state.store.service.FalconJPAService.init(FalconJPAService.java:91)
> at
> org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:47)
> at
> org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:56)
> at
> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:550)
> at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:519)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:57)
> at org.apache.falcon.FalconServer.main(FalconServer.java:102)
> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1723) Rerun with skip fail actions won't work in few cases
[ https://issues.apache.org/jira/browse/FALCON-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106301#comment-15106301 ] pavan kumar kolamuri commented on FALCON-1723: -- Added doc regarding prioritization over rerun can you please review [~pallavi.rao] > Rerun with skip fail actions won't work in few cases > > > Key: FALCON-1723 > URL: https://issues.apache.org/jira/browse/FALCON-1723 > Project: Falcon > Issue Type: Bug > Components: process >Affects Versions: 0.8 >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Attachments: FALCON-1723-v1.patch, FALCON-1723.patch > > > Normal Rerun - Rerunning an instance without force, it will rerun only failed > actions > Force Rerun - Rerunning an instance forcefully either by using force option > or passing props , it will run all actions irrespective of status > Normal Rerun won't work if force done once. And also we should check whether > force option is ignored if user also related properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1538) Prism status gives wrong info.
[
https://issues.apache.org/jira/browse/FALCON-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103132#comment-15103132
]
pavan kumar kolamuri commented on FALCON-1538:
--
[~Praveen] Can you please paste result how it is coming now when you do
./bin/prism-status
> Prism status gives wrong info.
> ---
>
> Key: FALCON-1538
> URL: https://issues.apache.org/jira/browse/FALCON-1538
> Project: Falcon
> Issue Type: Bug
>Reporter: sandeep samudrala
>Assignee: Praveen Adlakha
> Labels: newbie
> Attachments: FALCON-1538-v0.patch
>
>
> Upon doing prism-status it gives Falcon server is running.
> {noformat}
> [sandeep@/usr/local/falcon/falcon-distributed-0.8-SNAPSHOT$]./bin/prism-status
>
> Hadoop is installed, adding hadoop classpath to falcon classpath
> Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
> Falcon server is running (on
> https://localhost:16443/)[sandeep@/usr/local/falcon/falcon-distributed-0.8-SNAPSHOT$]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1662) Ensure entity can be scheduled on multiple clusters on same colo
[ https://issues.apache.org/jira/browse/FALCON-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103149#comment-15103149 ] pavan kumar kolamuri commented on FALCON-1662: -- +1 for the patch > Ensure entity can be scheduled on multiple clusters on same colo > > > Key: FALCON-1662 > URL: https://issues.apache.org/jira/browse/FALCON-1662 > Project: Falcon > Issue Type: Sub-task > Components: scheduler >Reporter: Pallavi Rao >Assignee: Pallavi Rao > Fix For: 0.9 > > Attachments: FALCON-1662-v1.patch, FALCON-1662.patch > > > A single Falcon Server (colo) can have multiple clusters, hence, the cluster > information should be part of the State store -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1715) IllegalStateException in MetadataMappingService when entity is scheduled via native scheduler
[
https://issues.apache.org/jira/browse/FALCON-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097914#comment-15097914
]
pavan kumar kolamuri commented on FALCON-1715:
--
+1 looks good
> IllegalStateException in MetadataMappingService when entity is scheduled via
> native scheduler
> -
>
> Key: FALCON-1715
> URL: https://issues.apache.org/jira/browse/FALCON-1715
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: Pallavi Rao
> Fix For: 0.9
>
> Attachments: FALCON-1715.patch
>
>
> When entity is scheduled via native scheduler, MetadataMappingServicefails
> with the following error :
> {noformat}
> 2015-12-31 09:01:34,818 ERROR - [ActiveMQ Session Task-123:] ~ Error in
> listener org.apache.falcon.metadata.MetadataMappingService
> (WorkflowJobEndNotificationService:245)
> org.apache.falcon.FalconException: java.lang.RuntimeException:
> java.lang.IllegalStateException: USER entity vertex must exist dataqa
> at
> org.apache.falcon.metadata.MetadataMappingService.onSuccess(MetadataMappingService.java:274)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:235)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifySuccess(WorkflowJobEndNotificationService.java:105)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:218)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:121)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: USER
> entity vertex must exist dataqa
> at
> com.tinkerpop.blueprints.util.TransactionRetryStrategy$AbstractRetryStrategy.execute(TransactionRetryStrategy.java:176)
> at
> com.tinkerpop.blueprints.util.TransactionRetryHelper.use(TransactionRetryHelper.java:115)
> at
> com.tinkerpop.blueprints.util.TransactionRetryHelper.exponentialBackoff(TransactionRetryHelper.java:94)
> at
> org.apache.falcon.metadata.MetadataMappingService.onSuccess(MetadataMappingService.java:263)
> ... 12 more
> Caused by: java.lang.IllegalStateException: USER entity vertex must exist
> dataqa
> at
> org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.addInstanceToEntity(InstanceRelationshipGraphBuilder.java:161)
> at
> org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.addInstanceToEntity(InstanceRelationshipGraphBuilder.java:151)
> at
> org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.addProcessInstance(InstanceRelationshipGraphBuilder.java:85)
> at
> org.apache.falcon.metadata.MetadataMappingService.onProcessInstanceExecuted(MetadataMappingService.java:320)
> at
> org.apache.falcon.metadata.MetadataMappingService.onSuccessfulExecution(MetadataMappingService.java:282)
> at
> org.apache.falcon.metadata.MetadataMappingService.access$100(MetadataMappingService.java:53)
> at
> org.apache.falcon.metadata.MetadataMappingService$3.execute(MetadataMappingService.java:267)
> at
> org.apache.falcon.metadata.MetadataMappingService$3.execute(MetadataMappingService.java:264)
> at
> com.tinkerpop.blueprints.util.TransactionRetryStrategy$AbstractRetryStrategy.execute(TransactionRetryStrategy.java:146)
> ... 15 more
> 2015-12-31 09:01:34,819 DEBUG - [ActiveMQ Session Task-123:] ~ Retrieving
> details for job 706-151230090106575-oozie-oozi-W (OozieDAGEngine:284)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FALCON-1723) Rerun with skip fail actions won't work in few cases
[ https://issues.apache.org/jira/browse/FALCON-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1723: - Attachment: FALCON-1723.patch > Rerun with skip fail actions won't work in few cases > > > Key: FALCON-1723 > URL: https://issues.apache.org/jira/browse/FALCON-1723 > Project: Falcon > Issue Type: Bug > Components: process >Affects Versions: 0.8 >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Attachments: FALCON-1723.patch > > > Normal Rerun - Rerunning an instance without force, it will rerun only failed > actions > Force Rerun - Rerunning an instance forcefully either by using force option > or passing props , it will run all actions irrespective of status > Normal Rerun won't work if force done once. And also we should check whether > force option is ignored if user also related properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1741) Rerun API behaviour different in case of succeeded instances
[
https://issues.apache.org/jira/browse/FALCON-1741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15091663#comment-15091663
]
pavan kumar kolamuri commented on FALCON-1741:
--
+1 for the patch.
> Rerun API behaviour different in case of succeeded instances
>
>
> Key: FALCON-1741
> URL: https://issues.apache.org/jira/browse/FALCON-1741
> Project: Falcon
> Issue Type: Sub-task
> Components: oozie, scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: Pallavi Rao
> Fix For: 0.9
>
>
> In case of succeeded instances , rerun without -force API has different
> behaviour based on the scheduler choice.
> For oozie :
> The state of instances changes to RUNNING.
> {noformat}
> dataqa@lda01:~$ fiprerun processMerlinOozie -start 2016-01-08T12:13Z -end
> 2016-01-08T12:15Z
> Consolidated Status: SUCCEEDED
> Instances:
> Instance Cluster SourceCluster Status
> Start End Details Log
> ---
> 2016-01-08T12:13Z ProcessMultipleClustersTest-corp-9706f068 -
> RUNNING 2016-01-08T13:03Z 2016-01-08T13:03Z -
> http://8RPCG32.corp.inmobi.com:11000/oozie?job=0001811-160104160825636-oozie-oozi-W
> 2016-01-08T12:13Z ProcessMultipleClustersTest-corp-0b270a1d -
> RUNNING 2016-01-08T13:03Z 2016-01-08T13:03Z -
> http://lda01:11000/oozie?job=0002247-160104115615658-oozie-oozi-W
> Additional Information:
> Response: ua1/RERUN
> ua2/RERUN
> Request Id: ua1/871377866@qtp-630572412-35 -
> 7190c4c8-bacb-4639-8d48-c9e639f544da
> ua2/1554129706@qtp-536122141-13 - bc18127b-1bf8-4ea1-99e6-b1f10ba3a441
> {noformat}
> For native :
> The state of instances changes to READY. Also params are present as a part of
> rerun API>
> {noformat}
> dataqa@lda01:~$ fiprerun
> ProcessMultipleClustersTest-agregator-coord16-8f55f59b -start
> 2016-01-08T12:13Z -end 2016-01-08T12:15Z
> Consolidated Status: SUCCEEDED
> Instances:
> Instance Cluster SourceCluster Status
> Start End Details Log
> ---
> 2016-01-08T12:13Z ProcessMultipleClustersTest-corp-9706f068 -
> READY 2016-01-08T13:03Z 2016-01-08T13:03Z -
> http://8RPCG32.corp.inmobi.com:11000/oozie?job=0001812-160104160825636-oozie-oozi-W
> Workflow params
> shouldRecord=false
> timeStamp=2016-01-08-12-13
> mapreduce.job.user.name=dataqa
> falconInPaths=NONE
> feedNames=NONE
> oozie.wf.rerun.failnodes=true
> falcon.libpath=/tmp/falcon-regression-working/lib
> ENTITY_PATH=/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/3cd7594ed13b218460df53a9e827eb28_1452252282546
> entityType=PROCESS
> systemJMSNotificationEnabled=false
> nominalTime=2016-01-08-12-13
> feedInstancePaths=NONE
> field2=value2
> logDir=hdfs://10.14.114.202:8020/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/logs
> userWorkflowEngine=oozie
> oozie.use.system.libpath=true
> userJMSNotificationEnabled=true
> queueName=default
> falconDataOperation=GENERATE
> falcon.system.rerun=true
> fileTime=${formatTime(dateOffset(instanceTime(), 1, 'DAY'), '-MMM-dd')}
> field1=value1
> workflowEngineUrl=http://10.14.114.202:11000/oozie/
> datasource=NA
> brokerImplClass=org.apache.activemq.ActiveMQConnectionFactory
> userBrokerImplClass=org.apache.activemq.ActiveMQConnectionFactory
> brokerUrl=tcp://localhost:61616
> ENTITY_NAME=FALCON_PROCESS_DEFAULT_ProcessMultipleClustersTest-agregator-coord16-8f55f59b
> availabilityFlag=NA
> userWorkflowName=ProcessMultipleClustersTest-agregator-coord16-8f55f59b-workflow
> brokerTTL=4320
> entityName=ProcessMultipleClustersTest-agregator-coord16-8f55f59b
> srcClusterName=NA
> userBrokerUrl=tcp://10.14.114.202:61616?daemon=true
> falconInputFeeds=NONE
> userWorkflowVersion=1.0
> oozie.libpath=/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/3cd7594ed13b218460df53a9e827eb28_1452252282546/lib
> cluster=ProcessMultipleClustersTest-corp-9706f068
> jobPriority=NORMAL
> colo.name=ua1
> nameNode=hdfs://10.14.114.202:8020
> user=${user()}
> jobTracker=10.14.114.202:8021
> oozie.wf.application.path=hdfs://10.14.114.202:8020/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/3cd7594ed13b218460df53a9e827eb28_1452252282546
> 2016-01-08T12:13Z
[jira] [Commented] (FALCON-1230) Data based notification Service to notify execution instances when data becomes available
[ https://issues.apache.org/jira/browse/FALCON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15091706#comment-15091706 ] pavan kumar kolamuri commented on FALCON-1230: -- Thanks [~pallavi.rao] [~ajayyadava] for reviewing. > Data based notification Service to notify execution instances when data > becomes available > - > > Key: FALCON-1230 > URL: https://issues.apache.org/jira/browse/FALCON-1230 > Project: Falcon > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Pallavi Rao >Assignee: pavan kumar kolamuri > Fix For: trunk > > Attachments: FALCON-1230-v1.patch, FALCON-1230-v2.patch, > FALCON-1230-v4.patch, FALCON-1230-v5.patch, FALCON-1230-v6.patch, > FALCON-1230-v7.patch, FALCON-1230-v8.patch, FALCON-1230.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1601) Make Falcon StateStore more secure by not disclosing imp params in startup.props
[ https://issues.apache.org/jira/browse/FALCON-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15091710#comment-15091710 ] pavan kumar kolamuri commented on FALCON-1601: -- Thanks [~pallavi.rao] [~ajayyadava] for review and suggestions. > Make Falcon StateStore more secure by not disclosing imp params in > startup.props > - > > Key: FALCON-1601 > URL: https://issues.apache.org/jira/browse/FALCON-1601 > Project: Falcon > Issue Type: Improvement >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Fix For: 0.9 > > Attachments: FALCON-1601-v1.patch, FALCON-1601.patch > > > We are bringing Falcon state Store DB for Native Scheduler as part of > https://issues.apache.org/jira/browse/FALCON-1234, We need to add more > secureness by removing password properties from startup props and also made > this Statestore supports Mysql as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (FALCON-1716) API fails with CommunicationsException when mysql interaction time is longer than 53,434,939 milliseconds
[
https://issues.apache.org/jira/browse/FALCON-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pavan kumar kolamuri reassigned FALCON-1716:
Assignee: pavan kumar kolamuri
> API fails with CommunicationsException when mysql interaction time is longer
> than 53,434,939 milliseconds
> -
>
> Key: FALCON-1716
> URL: https://issues.apache.org/jira/browse/FALCON-1716
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: jdbc-timedout.log
>
>
> If mysql is not interacted for less than 53,434,939 milliseconds, the query
> fails with CommunicationsException for first few attempts after which
> connection is set. We should configure communication window for the same.
> Error is :
> {noformat}
> 2015-12-30 06:54:47,294 ERROR - [357665612@qtp-1816147548-10 -
> 8ebf22cf-4f82-4280-a248-fa3ca6048042:dataqa:GET//instance/params/process/ProcessInstanceStatusTest-agregator-coord16-50a6b241]
> ~ Failed to display params of an instance (AbstractInstanceManager:553)
>
> org.apache.openjpa.persistence.PersistenceException: The last packet
> successfully received from the server was 53,434,939 milliseconds ago. The
> last packet sent successfully to the server was 53,434,939 milliseconds ago.
> is longer than the server configured value of 'wait_timeout'. You should
> consider either expiring and/or testing connection validity before use in
> your application, increasing the server configured values for client
> timeouts, or using the Connector/J connection property 'autoReconnect=true'
> to avoid this problem.
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:5001)
> at
> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4961)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:133)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:107)
> at
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:59)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.connect(JDBCStoreManager.java:967)
> at
> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.getConnection(JDBCStoreManager.java:241)
> at
> org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:104)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1004)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:862)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:793)
> at
> org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:541)
> at
> org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:274)
> at
> org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:290)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.getEntityByKey(JDBCStateStore.java:95)
> at
> org.apache.falcon.state.store.jdbc.JDBCStateStore.entityExists(JDBCStateStore.java:105)
> at
> org.apache.falcon.workflow.engine.FalconWorkflowEngine.isActive(FalconWorkflowEngine.java:98)
> at
> org.apache.falcon.workflow.WorkflowEngineFactory.getWorkflowEngine(WorkflowEngineFactory.java:54)
> at
> org.apache.falcon.resource.AbstractEntityManager.getWorkflowEngine(AbstractEntityManager.java:1132)
> at
> org.apache.falcon.resource.AbstractInstanceManager.getInstanceParams(AbstractInstanceManager.java:550)
> at
> org.apache.falcon.resource.InstanceManager.getInstanceParams(InstanceManager.java:178)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FALCON-1715) IllegalStateException in MetadataMappingService when entity is scheduled via native scheduler
[
https://issues.apache.org/jira/browse/FALCON-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pavan kumar kolamuri updated FALCON-1715:
-
Assignee: Pallavi Rao
> IllegalStateException in MetadataMappingService when entity is scheduled via
> native scheduler
> -
>
> Key: FALCON-1715
> URL: https://issues.apache.org/jira/browse/FALCON-1715
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: Pallavi Rao
>
> When entity is scheduled via native scheduler, MetadataMappingServicefails
> with the following error :
> {noformat}
> 2015-12-31 09:01:34,818 ERROR - [ActiveMQ Session Task-123:] ~ Error in
> listener org.apache.falcon.metadata.MetadataMappingService
> (WorkflowJobEndNotificationService:245)
> org.apache.falcon.FalconException: java.lang.RuntimeException:
> java.lang.IllegalStateException: USER entity vertex must exist dataqa
> at
> org.apache.falcon.metadata.MetadataMappingService.onSuccess(MetadataMappingService.java:274)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:235)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifySuccess(WorkflowJobEndNotificationService.java:105)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:218)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:121)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: USER
> entity vertex must exist dataqa
> at
> com.tinkerpop.blueprints.util.TransactionRetryStrategy$AbstractRetryStrategy.execute(TransactionRetryStrategy.java:176)
> at
> com.tinkerpop.blueprints.util.TransactionRetryHelper.use(TransactionRetryHelper.java:115)
> at
> com.tinkerpop.blueprints.util.TransactionRetryHelper.exponentialBackoff(TransactionRetryHelper.java:94)
> at
> org.apache.falcon.metadata.MetadataMappingService.onSuccess(MetadataMappingService.java:263)
> ... 12 more
> Caused by: java.lang.IllegalStateException: USER entity vertex must exist
> dataqa
> at
> org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.addInstanceToEntity(InstanceRelationshipGraphBuilder.java:161)
> at
> org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.addInstanceToEntity(InstanceRelationshipGraphBuilder.java:151)
> at
> org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.addProcessInstance(InstanceRelationshipGraphBuilder.java:85)
> at
> org.apache.falcon.metadata.MetadataMappingService.onProcessInstanceExecuted(MetadataMappingService.java:320)
> at
> org.apache.falcon.metadata.MetadataMappingService.onSuccessfulExecution(MetadataMappingService.java:282)
> at
> org.apache.falcon.metadata.MetadataMappingService.access$100(MetadataMappingService.java:53)
> at
> org.apache.falcon.metadata.MetadataMappingService$3.execute(MetadataMappingService.java:267)
> at
> org.apache.falcon.metadata.MetadataMappingService$3.execute(MetadataMappingService.java:264)
> at
> com.tinkerpop.blueprints.util.TransactionRetryStrategy$AbstractRetryStrategy.execute(TransactionRetryStrategy.java:146)
> ... 15 more
> 2015-12-31 09:01:34,819 DEBUG - [ActiveMQ Session Task-123:] ~ Retrieving
> details for job 706-151230090106575-oozie-oozi-W (OozieDAGEngine:284)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FALCON-1682) Falcon server starts successfully even if mysql db is not created.
[
https://issues.apache.org/jira/browse/FALCON-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pavan kumar kolamuri updated FALCON-1682:
-
Attachment: FALCON-1682.patch
> Falcon server starts successfully even if mysql db is not created.
> --
>
> Key: FALCON-1682
> URL: https://issues.apache.org/jira/browse/FALCON-1682
> Project: Falcon
> Issue Type: Bug
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: FALCON-1682.patch
>
>
> If falcon is configured to run with mysql db, and user does not create db
> then server start should fail and throw error for the same. But server starts
> successfully as of now. Although error is logged in server logs saying :
> {noformat}
> 2015-12-21 13:41:01,899 ERROR - [main:] ~ Failed to initialize service
> org.apache.falcon.state.store.service.FalconJPAService (ServiceInitializer:49)
>
> org.apache.openjpa.persistence.PersistenceException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1518)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.falcon.state.store.service.FalconJPAService.getEntityManager(FalconJPAService.java:169)
> at
> org.apache.falcon.state.store.service.FalconJPAService.init(FalconJPAService.java:91)
> at
> org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:47)
> at
> org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:56)
> at
> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:550)
> at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:519)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:57)
> at org.apache.falcon.FalconServer.main(FalconServer.java:102)
> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1682) Falcon server starts successfully even if mysql db is not created.
[
https://issues.apache.org/jira/browse/FALCON-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15091847#comment-15091847
]
pavan kumar kolamuri commented on FALCON-1682:
--
Based on discussion with [~pallavi.rao], i took approach to change in
falcon-start script and updated patch. Please review.
> Falcon server starts successfully even if mysql db is not created.
> --
>
> Key: FALCON-1682
> URL: https://issues.apache.org/jira/browse/FALCON-1682
> Project: Falcon
> Issue Type: Bug
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Attachments: FALCON-1682.patch
>
>
> If falcon is configured to run with mysql db, and user does not create db
> then server start should fail and throw error for the same. But server starts
> successfully as of now. Although error is logged in server logs saying :
> {noformat}
> 2015-12-21 13:41:01,899 ERROR - [main:] ~ Failed to initialize service
> org.apache.falcon.state.store.service.FalconJPAService (ServiceInitializer:49)
>
> org.apache.openjpa.persistence.PersistenceException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1518)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.falcon.state.store.service.FalconJPAService.getEntityManager(FalconJPAService.java:169)
> at
> org.apache.falcon.state.store.service.FalconJPAService.init(FalconJPAService.java:91)
> at
> org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:47)
> at
> org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:56)
> at
> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:550)
> at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:519)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:57)
> at org.apache.falcon.FalconServer.main(FalconServer.java:102)
> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1678) SLA Monitoring does not honour entity end date
[ https://issues.apache.org/jira/browse/FALCON-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15091450#comment-15091450 ] pavan kumar kolamuri commented on FALCON-1678: -- +1 for the patch > SLA Monitoring does not honour entity end date > -- > > Key: FALCON-1678 > URL: https://issues.apache.org/jira/browse/FALCON-1678 > Project: Falcon > Issue Type: Bug > Components: feed >Affects Versions: 0.9 > Environment: QA >Reporter: Pragya Mittal >Assignee: Ajay Yadava > Fix For: 0.9 > > Attachments: FALCON-1678-v1.patch, FALCON-1678-v2.patch, > FALCON-1678.patch > > > Currently SLA monitoring shows all the SLA miss till the given endTime > (default is now) irrespective of entity end date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1230) Data based notification Service to notify execution instances when data becomes available
[ https://issues.apache.org/jira/browse/FALCON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1230: - Attachment: FALCON-1230-v8.patch > Data based notification Service to notify execution instances when data > becomes available > - > > Key: FALCON-1230 > URL: https://issues.apache.org/jira/browse/FALCON-1230 > Project: Falcon > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Pallavi Rao >Assignee: pavan kumar kolamuri > Attachments: FALCON-1230-v1.patch, FALCON-1230-v2.patch, > FALCON-1230-v4.patch, FALCON-1230-v5.patch, FALCON-1230-v6.patch, > FALCON-1230-v7.patch, FALCON-1230-v8.patch, FALCON-1230.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1230) Data based notification Service to notify execution instances when data becomes available
[ https://issues.apache.org/jira/browse/FALCON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15091541#comment-15091541 ] pavan kumar kolamuri commented on FALCON-1230: -- [~ajayyadava] Addressed your comments please review it > Data based notification Service to notify execution instances when data > becomes available > - > > Key: FALCON-1230 > URL: https://issues.apache.org/jira/browse/FALCON-1230 > Project: Falcon > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Pallavi Rao >Assignee: pavan kumar kolamuri > Attachments: FALCON-1230-v1.patch, FALCON-1230-v2.patch, > FALCON-1230-v4.patch, FALCON-1230-v5.patch, FALCON-1230-v6.patch, > FALCON-1230-v7.patch, FALCON-1230-v8.patch, FALCON-1230.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1230) Data based notification Service to notify execution instances when data becomes available
[ https://issues.apache.org/jira/browse/FALCON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088902#comment-15088902 ] pavan kumar kolamuri commented on FALCON-1230: -- I refactored most of things in next patch like Removing INVALID Status and renaming variables and make builder pattern, ok i will be bring all those there and submit new patch > Data based notification Service to notify execution instances when data > becomes available > - > > Key: FALCON-1230 > URL: https://issues.apache.org/jira/browse/FALCON-1230 > Project: Falcon > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Pallavi Rao >Assignee: pavan kumar kolamuri > Attachments: FALCON-1230-v1.patch, FALCON-1230-v2.patch, > FALCON-1230-v4.patch, FALCON-1230-v5.patch, FALCON-1230-v6.patch, > FALCON-1230.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1677) Support re-tries for timed-out instances
[ https://issues.apache.org/jira/browse/FALCON-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089016#comment-15089016 ] pavan kumar kolamuri commented on FALCON-1677: -- Completely agree with [~pallavi.rao] +1 for 2nd approach > Support re-tries for timed-out instances > > > Key: FALCON-1677 > URL: https://issues.apache.org/jira/browse/FALCON-1677 > Project: Falcon > Issue Type: Bug >Reporter: Pallavi Rao > > Currently, Falcon retries only on failure. We should extend support in case > of timed-out instances too. Earlier, since we were relying on post-processing > to notify the instance status, this was not possible. Now that Falcon relies > on Oozie JMS notifications, we can support retries for timed out instances > too. > If a dataset is expected to get delayed for a long time, the user is > currently forced to supply a large timeout value. This is an overhead in > terms of Oozie having to poll for that long. If we introduce retries, the > timeout can be a reasonable value with periodic/exponential back-off retries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1230) Data based notification Service to notify execution instances when data becomes available
[ https://issues.apache.org/jira/browse/FALCON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089139#comment-15089139 ] pavan kumar kolamuri commented on FALCON-1230: -- [~ajayyadava] Please review now, i have addressed comments > Data based notification Service to notify execution instances when data > becomes available > - > > Key: FALCON-1230 > URL: https://issues.apache.org/jira/browse/FALCON-1230 > Project: Falcon > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Pallavi Rao >Assignee: pavan kumar kolamuri > Attachments: FALCON-1230-v1.patch, FALCON-1230-v2.patch, > FALCON-1230-v4.patch, FALCON-1230-v5.patch, FALCON-1230-v6.patch, > FALCON-1230-v7.patch, FALCON-1230.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1230) Data based notification Service to notify execution instances when data becomes available
[ https://issues.apache.org/jira/browse/FALCON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089137#comment-15089137 ] pavan kumar kolamuri commented on FALCON-1230: -- Raised https://issues.apache.org/jira/browse/FALCON-1738 based on review comments and also fix the timeout scenario > Data based notification Service to notify execution instances when data > becomes available > - > > Key: FALCON-1230 > URL: https://issues.apache.org/jira/browse/FALCON-1230 > Project: Falcon > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Pallavi Rao >Assignee: pavan kumar kolamuri > Attachments: FALCON-1230-v1.patch, FALCON-1230-v2.patch, > FALCON-1230-v4.patch, FALCON-1230-v5.patch, FALCON-1230-v6.patch, > FALCON-1230-v7.patch, FALCON-1230.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-1738) Support Retry for failure Data-based Notifications and fix timeout test case in FalconExecutionServiceTest
pavan kumar kolamuri created FALCON-1738: Summary: Support Retry for failure Data-based Notifications and fix timeout test case in FalconExecutionServiceTest Key: FALCON-1738 URL: https://issues.apache.org/jira/browse/FALCON-1738 Project: Falcon Issue Type: Improvement Reporter: pavan kumar kolamuri Whenever there are failures while notifying Data Availability Event, this should be handled and has to retried. And also need to fix whether timeout can be zero or not and enable test case in FalconExecutionService -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1230) Data based notification Service to notify execution instances when data becomes available
[ https://issues.apache.org/jira/browse/FALCON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1230: - Attachment: FALCON-1230-v7.patch > Data based notification Service to notify execution instances when data > becomes available > - > > Key: FALCON-1230 > URL: https://issues.apache.org/jira/browse/FALCON-1230 > Project: Falcon > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Pallavi Rao >Assignee: pavan kumar kolamuri > Attachments: FALCON-1230-v1.patch, FALCON-1230-v2.patch, > FALCON-1230-v4.patch, FALCON-1230-v5.patch, FALCON-1230-v6.patch, > FALCON-1230-v7.patch, FALCON-1230.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1230) Data based notification Service to notify execution instances when data becomes available
[ https://issues.apache.org/jira/browse/FALCON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1230: - Attachment: FALCON-1230-v6.patch > Data based notification Service to notify execution instances when data > becomes available > - > > Key: FALCON-1230 > URL: https://issues.apache.org/jira/browse/FALCON-1230 > Project: Falcon > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Pallavi Rao >Assignee: pavan kumar kolamuri > Attachments: FALCON-1230-v1.patch, FALCON-1230-v2.patch, > FALCON-1230-v4.patch, FALCON-1230-v5.patch, FALCON-1230-v6.patch, > FALCON-1230.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1230) Data based notification Service to notify execution instances when data becomes available
[ https://issues.apache.org/jira/browse/FALCON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087276#comment-15087276 ] pavan kumar kolamuri commented on FALCON-1230: -- I will refactor DataNotificationRequest class in next patch as part of https://issues.apache.org/jira/browse/FALCON-1435 > Data based notification Service to notify execution instances when data > becomes available > - > > Key: FALCON-1230 > URL: https://issues.apache.org/jira/browse/FALCON-1230 > Project: Falcon > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Pallavi Rao >Assignee: pavan kumar kolamuri > Attachments: FALCON-1230-v1.patch, FALCON-1230-v2.patch, > FALCON-1230-v4.patch, FALCON-1230-v5.patch, FALCON-1230-v6.patch, > FALCON-1230.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1728) Process entity definition allows multiple clusters when it has output Feed defined.
[ https://issues.apache.org/jira/browse/FALCON-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088724#comment-15088724 ] pavan kumar kolamuri commented on FALCON-1728: -- Yes [~bvellanki] you are right . Falcon should not allow the scenario you described in last comment. When cluster is defined as target in feed that means it meant for replication and process shouldn't run in target cluster with this feed as output feed where as input feed is fine. > Process entity definition allows multiple clusters when it has output Feed > defined. > > > Key: FALCON-1728 > URL: https://issues.apache.org/jira/browse/FALCON-1728 > Project: Falcon > Issue Type: Bug > Components: process >Affects Versions: 0.9 >Reporter: Balu Vellanki >Assignee: Balu Vellanki >Priority: Critical > > Process XSD allows user to specify multiple clusters per process entity. I am > guessing this would allow a user to run duplicate instance of the process on > multiple clusters at the same time (I do not really see a need for this). > When the process has an output feed defined, you can have duplicate process > instances writing to same feed instance, causing data corruption/failures. > The solution is to > 1. Do not allow multiple clusters per process. Let the user define a > duplicate process if user wants to run duplicate instances. > OR > 2. Allow multiple clusters, but only when there is no output feed defined. > [~sriksun] please let me know if there is any other reason for allowing > multiple clusters in a process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1720) Rerun API does not rerun succeeded instances
[ https://issues.apache.org/jira/browse/FALCON-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085359#comment-15085359 ] pavan kumar kolamuri commented on FALCON-1720: -- Thanks [~pragya.mittal] for raising the jira , we will add asap > Rerun API does not rerun succeeded instances > > > Key: FALCON-1720 > URL: https://issues.apache.org/jira/browse/FALCON-1720 > Project: Falcon > Issue Type: Bug > Components: scheduler >Affects Versions: 0.9 >Reporter: Pragya Mittal >Assignee: pavan kumar kolamuri > Attachments: FALCON-1720-v1.patch, FALCON-1720.patch > > > When entity is scheduled via native scheduler, rerun api does not force rerun > the succeeded instances. Although oozie run count gets incremented but state > of instance does not change from 'SUCCEEDED' to 'RUNNING' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1230) Data based notification Service to notify execution instances when data becomes available
[ https://issues.apache.org/jira/browse/FALCON-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085269#comment-15085269 ] pavan kumar kolamuri commented on FALCON-1230: -- Please review , my other patches blocked on this > Data based notification Service to notify execution instances when data > becomes available > - > > Key: FALCON-1230 > URL: https://issues.apache.org/jira/browse/FALCON-1230 > Project: Falcon > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Pallavi Rao >Assignee: pavan kumar kolamuri > Attachments: FALCON-1230-v1.patch, FALCON-1230-v2.patch, > FALCON-1230-v4.patch, FALCON-1230-v5.patch, FALCON-1230.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1601) Make Falcon StateStore more secure by not disclosing imp params in startup.props
[ https://issues.apache.org/jira/browse/FALCON-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1601: - Attachment: FALCON-1601-v1.patch > Make Falcon StateStore more secure by not disclosing imp params in > startup.props > - > > Key: FALCON-1601 > URL: https://issues.apache.org/jira/browse/FALCON-1601 > Project: Falcon > Issue Type: Improvement >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Attachments: FALCON-1601-v1.patch, FALCON-1601.patch > > > We are bringing Falcon state Store DB for Native Scheduler as part of > https://issues.apache.org/jira/browse/FALCON-1234, We need to add more > secureness by removing password properties from startup props and also made > this Statestore supports Mysql as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1601) Make Falcon StateStore more secure by not disclosing imp params in startup.props
[ https://issues.apache.org/jira/browse/FALCON-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085261#comment-15085261 ] pavan kumar kolamuri commented on FALCON-1601: -- I have addressed comments based on discussion please review. > Make Falcon StateStore more secure by not disclosing imp params in > startup.props > - > > Key: FALCON-1601 > URL: https://issues.apache.org/jira/browse/FALCON-1601 > Project: Falcon > Issue Type: Improvement >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > Attachments: FALCON-1601-v1.patch, FALCON-1601.patch > > > We are bringing Falcon state Store DB for Native Scheduler as part of > https://issues.apache.org/jira/browse/FALCON-1234, We need to add more > secureness by removing password properties from startup props and also made > this Statestore supports Mysql as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1720) Rerun API does not rerun succeeded instances
[ https://issues.apache.org/jira/browse/FALCON-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085356#comment-15085356 ] pavan kumar kolamuri commented on FALCON-1720: -- Thanks [~pallavi.rao] [~pragya.mittal] [~ajayyadava] [~bvellanki] [~sowmyaramesh] for reviewing the patch. Soumya i have explained about the comment if you think i am missing something please comment it , i will raise jira and fix it . > Rerun API does not rerun succeeded instances > > > Key: FALCON-1720 > URL: https://issues.apache.org/jira/browse/FALCON-1720 > Project: Falcon > Issue Type: Bug > Components: scheduler >Affects Versions: 0.9 >Reporter: Pragya Mittal >Assignee: pavan kumar kolamuri > Attachments: FALCON-1720-v1.patch, FALCON-1720.patch > > > When entity is scheduled via native scheduler, rerun api does not force rerun > the succeeded instances. Although oozie run count gets incremented but state > of instance does not change from 'SUCCEEDED' to 'RUNNING' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1725) Falcon API shows results in ascending order in native scheduler
[ https://issues.apache.org/jira/browse/FALCON-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085321#comment-15085321 ] pavan kumar kolamuri commented on FALCON-1725: -- Just had minor nit in review board . Patch looks good, +1 for the patch. > Falcon API shows results in ascending order in native scheduler > --- > > Key: FALCON-1725 > URL: https://issues.apache.org/jira/browse/FALCON-1725 > Project: Falcon > Issue Type: Bug > Components: scheduler >Affects Versions: 0.9 >Reporter: Pragya Mittal >Assignee: Pallavi Rao > Fix For: 0.9 > > Attachments: FALCON-1725.patch > > > Currently falcon shows results in descending order(by default) as long as > order is not explicitly defined. But native scheduler shows the results in > ascending order. Since behaviour of both native scheduler and oozie should be > same, the order should be consistent irrespective of the scheduler choice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1682) Falcon server starts successfully even if mysql db is not created.
[
https://issues.apache.org/jira/browse/FALCON-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085641#comment-15085641
]
pavan kumar kolamuri commented on FALCON-1682:
--
[~pallavi.rao] [~sandeep.samudrala]
I have looked into issue further,
1) Before starting server itself this context Initialized method will execute
so there is no chance to call server.stop() . Even i tried System.exit() that's
also not working.
In fact Falcon server looks like started but it will throw exception for all
requests so no issue in correctness.
Please suggest me any better approach for stopping server if there is any.
> Falcon server starts successfully even if mysql db is not created.
> --
>
> Key: FALCON-1682
> URL: https://issues.apache.org/jira/browse/FALCON-1682
> Project: Falcon
> Issue Type: Bug
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
>
> If falcon is configured to run with mysql db, and user does not create db
> then server start should fail and throw error for the same. But server starts
> successfully as of now. Although error is logged in server logs saying :
> {noformat}
> 2015-12-21 13:41:01,899 ERROR - [main:] ~ Failed to initialize service
> org.apache.falcon.state.store.service.FalconJPAService (ServiceInitializer:49)
>
> org.apache.openjpa.persistence.PersistenceException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1518)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.falcon.state.store.service.FalconJPAService.getEntityManager(FalconJPAService.java:169)
> at
> org.apache.falcon.state.store.service.FalconJPAService.init(FalconJPAService.java:91)
> at
> org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:47)
> at
> org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:56)
> at
> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:550)
> at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:519)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:57)
> at org.apache.falcon.FalconServer.main(FalconServer.java:102)
> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> {noformat}
--
This message was sent by
[jira] [Commented] (FALCON-1728) Process entity definition allows multiple clusters when it has output Feed defined.
[
https://issues.apache.org/jira/browse/FALCON-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086815#comment-15086815
]
pavan kumar kolamuri commented on FALCON-1728:
--
Just adding more points to what [~ajayyadava] said
{noformat}
{noformat}
In the feed definition we can specify partition, in the target cluster
replication will happen based on partitions so data override won't happen
{noformat}
ex: /{partition1}/cluster1-data
/{partition2}/cluster2-data
{noformat}
> Process entity definition allows multiple clusters when it has output Feed
> defined.
>
>
> Key: FALCON-1728
> URL: https://issues.apache.org/jira/browse/FALCON-1728
> Project: Falcon
> Issue Type: Bug
> Components: process
>Affects Versions: 0.9
>Reporter: Balu Vellanki
>Assignee: Balu Vellanki
>Priority: Critical
>
> Process XSD allows user to specify multiple clusters per process entity. I am
> guessing this would allow a user to run duplicate instance of the process on
> multiple clusters at the same time (I do not really see a need for this).
> When the process has an output feed defined, you can have duplicate process
> instances writing to same feed instance, causing data corruption/failures.
> The solution is to
> 1. Do not allow multiple clusters per process. Let the user define a
> duplicate process if user wants to run duplicate instances.
> OR
> 2. Allow multiple clusters, but only when there is no output feed defined.
> [~sriksun] please let me know if there is any other reason for allowing
> multiple clusters in a process.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1682) Falcon server starts successfully even if mysql db is not created.
[
https://issues.apache.org/jira/browse/FALCON-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086969#comment-15086969
]
pavan kumar kolamuri commented on FALCON-1682:
--
Even i will prefer option-2 will look into it and update if there any issues in
that.
> Falcon server starts successfully even if mysql db is not created.
> --
>
> Key: FALCON-1682
> URL: https://issues.apache.org/jira/browse/FALCON-1682
> Project: Falcon
> Issue Type: Bug
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
>
> If falcon is configured to run with mysql db, and user does not create db
> then server start should fail and throw error for the same. But server starts
> successfully as of now. Although error is logged in server logs saying :
> {noformat}
> 2015-12-21 13:41:01,899 ERROR - [main:] ~ Failed to initialize service
> org.apache.falcon.state.store.service.FalconJPAService (ServiceInitializer:49)
>
> org.apache.openjpa.persistence.PersistenceException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> at
> org.apache.openjpa.jdbc.sql.DBDictionaryFactory.newDBDictionary(DBDictionaryFactory.java:106)
> at
> org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getDBDictionaryInstance(JDBCConfigurationImpl.java:603)
> at
> org.apache.openjpa.jdbc.meta.MappingRepository.endConfiguration(MappingRepository.java:1518)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:533)
> at
> org.apache.openjpa.lib.conf.Configurations.configureInstance(Configurations.java:458)
> at
> org.apache.openjpa.lib.conf.PluginValue.instantiate(PluginValue.java:121)
> at
> org.apache.openjpa.conf.MetaDataRepositoryValue.instantiate(MetaDataRepositoryValue.java:68)
> at
> org.apache.openjpa.lib.conf.ObjectValue.instantiate(ObjectValue.java:83)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.newMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:967)
> at
> org.apache.openjpa.conf.OpenJPAConfigurationImpl.getMetaDataRepositoryInstance(OpenJPAConfigurationImpl.java:958)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:642)
> at
> org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:202)
> at
> org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:154)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:226)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:153)
> at
> org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:59)
> at
> org.apache.falcon.state.store.service.FalconJPAService.getEntityManager(FalconJPAService.java:169)
> at
> org.apache.falcon.state.store.service.FalconJPAService.init(FalconJPAService.java:91)
> at
> org.apache.falcon.service.ServiceInitializer.initialize(ServiceInitializer.java:47)
> at
> org.apache.falcon.listener.ContextStartupListener.contextInitialized(ContextStartupListener.java:56)
> at
> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:550)
> at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:519)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.falcon.util.EmbeddedServer.start(EmbeddedServer.java:57)
> at org.apache.falcon.FalconServer.main(FalconServer.java:102)
> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create
> PoolableConnectionFactory (Access denied for user 'sa'@'localhost' (using
> password: NO))
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FALCON-1719) Retry does not update the state of the instance in the database
[
https://issues.apache.org/jira/browse/FALCON-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086955#comment-15086955
]
pavan kumar kolamuri commented on FALCON-1719:
--
[~sowmyaramesh] [~bvellanki] Thanks for raising concern i have raised the jira
to fix the same https://issues.apache.org/jira/browse/FALCON-1735
> Retry does not update the state of the instance in the database
> ---
>
> Key: FALCON-1719
> URL: https://issues.apache.org/jira/browse/FALCON-1719
> Project: Falcon
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 0.9
>Reporter: Pragya Mittal
>Assignee: pavan kumar kolamuri
> Fix For: 0.9
>
> Attachments: FALCON-1719-v1.patch, FALCON-1719.patch
>
>
> Falcon retries the instance if it gets killed. Although the same state is not
> reflected in database in case of native scheduler.
> Instance status
> {noformat}
> dataqa@lda01:~$ falcon instance -type process -name
> ProcessWfTest-agregator-coord16-a7c562ac -status -start 2016-01-04T12:04Z
> -end 2016-01-04T12:05Z
> Consolidated Status: SUCCEEDED
> Instances:
> Instance Cluster SourceCluster Status
> Start End Details Log
> ---
> 2016-01-04T12:04Z ProcessWfTest-corp-bb2ecac1 - SUCCEEDED
> 2016-01-04T12:13Z 2016-01-04T12:13Z -
> http://lda01:11000/oozie?job=089-160104115615658-oozie-oozi-W
> actions:
> hdfscommands OK -
> aggregator SUCCEEDED
> http://lda01:8088/proxy/application_1451903998946_0133/
> Additional Information:
> Response: ua1/STATUS
> Request Id: ua1/512026219@qtp-1816147548-5 -
> 0dc22236-75fc-4a0b-a699-60266c304329
> {noformat}
> Oozie info :
> {noformat}
> dataqa@lda01:~$ oozie job -info 089-160104115615658-oozie-oozi-W
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/mnt/oozie/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/mnt/oozie/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/mnt/oozie/libext/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
> Job ID : 089-160104115615658-oozie-oozi-W
>
> Workflow Name :
> FALCON_PROCESS_DEFAULT_ProcessWfTest-agregator-coord16-a7c562ac
> App Path :
> hdfs://192.168.138.236:8020/tmp/falcon-regression-staging/falcon/workflows/process/ProcessWfTest-agregator-coord16-a7c562ac/11642dd2e8532508c97721c51dbb28ce_1451909324381
> Status: SUCCEEDED
> Run : 1
> User : dataqa
> Group : -
> Created : 2016-01-04 12:09 GMT
> Started : 2016-01-04 12:13 GMT
> Last Modified : 2016-01-04 12:13 GMT
> Ended : 2016-01-04 12:13 GMT
> CoordAction ID: -
> Actions
>
> ID
> StatusExt ID Ext Status Err Code
>
> 089-160104115615658-oozie-oozi-W@:start:
> OK- OK -
>
> 089-160104115615658-oozie-oozi-W@failed-post-processing
> OKjob_1451903998946_0108 SUCCEEDED -
>
> 089-160104115615658-oozie-oozi-W@fail
> OK- OK E0729
>
> 089-160104115615658-oozie-oozi-W@user-action
> OK090-160104115615658-oozie-oozi-WSUCCEEDED -
>
>
[jira] [Commented] (FALCON-1720) Rerun API does not rerun succeeded instances
[ https://issues.apache.org/jira/browse/FALCON-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082678#comment-15082678 ] pavan kumar kolamuri commented on FALCON-1720: -- I am explaining all cases here Normal Rerun - Rerunning an instance without force, it will rerun only failed actions Force Rerun - Rerunning an instance forcefully either by using force option or passing props , it will run all actions irrespective of status Initially if you do force rerun or normal rerun then rerun will work fine without any issues, issue will come only when you do rerun twice, i will explain all those scenarios here. 1) Normal Rerun and Normal Rerun - Both Current Falcon and Native Scheduler will work in this case. 2) Normal Rerun and Force Rerun - Current Falcon will work in this case and Native Scheduler won't work in this case 3) Force Rerun and Force Rerun - Both Current Falcon and Native Scheduler will work in this case. 4) Force Rerun and Normal Rerun - Both Current Falcon and Native Scheduler won't work in this case, it will always do force rerun Apart from these cases there is also properties option where user can pass workflow properties. 1) If Force flag and user properties are contradictory which should be prioritized ? In code it says force flag will be ignored in current but it is not happening with current falcon. I am thinking to make ignoring force option if user has set any of these props. > Rerun API does not rerun succeeded instances > > > Key: FALCON-1720 > URL: https://issues.apache.org/jira/browse/FALCON-1720 > Project: Falcon > Issue Type: Bug > Components: scheduler >Affects Versions: 0.9 >Reporter: Pragya Mittal >Assignee: pavan kumar kolamuri > > When entity is scheduled via native scheduler, rerun api does not force rerun > the succeeded instances. Although oozie run count gets incremented but state > of instance does not change from 'SUCCEEDED' to 'RUNNING' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FALCON-1720) Rerun API does not rerun succeeded instances
[ https://issues.apache.org/jira/browse/FALCON-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri updated FALCON-1720: - Attachment: FALCON-1720.patch > Rerun API does not rerun succeeded instances > > > Key: FALCON-1720 > URL: https://issues.apache.org/jira/browse/FALCON-1720 > Project: Falcon > Issue Type: Bug > Components: scheduler >Affects Versions: 0.9 >Reporter: Pragya Mittal >Assignee: pavan kumar kolamuri > Attachments: FALCON-1720.patch > > > When entity is scheduled via native scheduler, rerun api does not force rerun > the succeeded instances. Although oozie run count gets incremented but state > of instance does not change from 'SUCCEEDED' to 'RUNNING' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FALCON-1723) Rerun with skip fail actions won't work in few cases
pavan kumar kolamuri created FALCON-1723: Summary: Rerun with skip fail actions won't work in few cases Key: FALCON-1723 URL: https://issues.apache.org/jira/browse/FALCON-1723 Project: Falcon Issue Type: Bug Components: process Reporter: pavan kumar kolamuri Normal Rerun - Rerunning an instance without force, it will rerun only failed actions Force Rerun - Rerunning an instance forcefully either by using force option or passing props , it will run all actions irrespective of status Normal Rerun won't work if force done once. And also we should check whether force option is ignored if user also related properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (FALCON-1723) Rerun with skip fail actions won't work in few cases
[ https://issues.apache.org/jira/browse/FALCON-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavan kumar kolamuri reassigned FALCON-1723: Assignee: pavan kumar kolamuri > Rerun with skip fail actions won't work in few cases > > > Key: FALCON-1723 > URL: https://issues.apache.org/jira/browse/FALCON-1723 > Project: Falcon > Issue Type: Bug > Components: process >Reporter: pavan kumar kolamuri >Assignee: pavan kumar kolamuri > > Normal Rerun - Rerunning an instance without force, it will rerun only failed > actions > Force Rerun - Rerunning an instance forcefully either by using force option > or passing props , it will run all actions irrespective of status > Normal Rerun won't work if force done once. And also we should check whether > force option is ignored if user also related properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FALCON-1720) Rerun API does not rerun succeeded instances
[ https://issues.apache.org/jira/browse/FALCON-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082769#comment-15082769 ] pavan kumar kolamuri commented on FALCON-1720: -- All the four cases mentioned here are fixed with this patch and also forced option will be ignoring if user passed properties. All these are fixed in case of Native Scheduler, to fix in oozie and documentation it was tracked in FALCON-1723. > Rerun API does not rerun succeeded instances > > > Key: FALCON-1720 > URL: https://issues.apache.org/jira/browse/FALCON-1720 > Project: Falcon > Issue Type: Bug > Components: scheduler >Affects Versions: 0.9 >Reporter: Pragya Mittal >Assignee: pavan kumar kolamuri > Attachments: FALCON-1720.patch > > > When entity is scheduled via native scheduler, rerun api does not force rerun > the succeeded instances. Although oozie run count gets incremented but state > of instance does not change from 'SUCCEEDED' to 'RUNNING' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
