[
https://issues.apache.org/jira/browse/FALCON-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suhas Vasu reassigned FALCON-494:
---------------------------------
Assignee: Suhas Vasu
> update feature broken
> ----------------------
>
> Key: FALCON-494
> URL: https://issues.apache.org/jira/browse/FALCON-494
> Project: Falcon
> Issue Type: Bug
> Components: general
> Affects Versions: 0.6
> Environment: QA
> Reporter: Samarth Gupta
> Assignee: Suhas Vasu
> Priority: Blocker
> Fix For: 0.4, 0.5
>
>
> when we update an process / feed , the update take place from current time,
> and all the future instance of the old coordinator are moved to new coord
> with new definition.
> However with the latest falcon, the coord which is getting created is having
> all instances which have have been already processed by old coodinator.
> Resulting in rerunning on those instances which are already in RUNNING /
> READY / WAITING state in old coord.
> in the logs, following lines can be seen which says the new start time :
> {code}
> 2014-06-25 06:36:40,819 INFO -
> [1871593525@qtp-1222207508-0:dataqa:POST//sync/update/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395
> 1118a58a-4a4e-4052-afea-40d4960470f3] ~ Submitted
> 0000328-140624044727425-oozie-oozi-B on cluster
> corp-2babc92c-129f-422e-b836-17e286eeaf0111-9b497e34-e892-4e9b-a5ee-f9a417e55f1e
> with properties: {nameNode=hdfs://192.168.138.182:8020,
> oozie.use.system.libpath=true, field2=value2,
> jobTracker=192.168.138.182:8021, field1=value1,
> falcon.libpath=/projectsTest/ivory/working/lib, colo.name=ua3,
> oozie.bundle.application.path=${nameNode}/projects/ivory/staging/falcon/workflows/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395/1403678197513,
> user.name=dataqa} (OozieWorkflowEngine:1316)
> 2014-06-25 06:36:40,827 INFO -
> [1871593525@qtp-1222207508-0:dataqa:POST//sync/update/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395
> 1118a58a-4a4e-4052-afea-40d4960470f3] ~ New bundle
> 0000328-140624044727425-oozie-oozi-B scheduled successfully with start time
> 2014-06-25T06:39Z (OozieWorkflowEngine:1131)
> 2014-06-25 06:36:40,828 DEBUG -
> [1871593525@qtp-1222207508-0:dataqa:POST//sync/update/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395
> 1118a58a-4a4e-4052-afea-40d4960470f3] ~ Updating endtime of coord
> 0000323-140624044727425-oozie-oozi-C to 2014-06-25T06:39Z on cluster
> corp-2babc92c-129f-422e-b836-17e286eeaf0111-9b497e34-e892-4e9b-a5ee-f9a417e55f1e
> (OozieWorkflowEngine:1055)
> {code}
> but in the coord xml the start time is till the original process time :
> {code}
> 2014-06-25 06:36:39,925 DEBUG -
> [1871593525@qtp-1222207508-0:dataqa:POST//sync/update/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395
> 1118a58a-4a4e-4052-afea-40d4960470f3] ~ Writing definition to
> /projects/ivory/staging/falcon/workflows/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395/1403678197513/DEFAULT/coordinator.xml
> on cluster
> corp-2babc92c-129f-422e-b836-17e286eeaf0111-9b497e34-e892-4e9b-a5ee-f9a417e55f1e
> (OozieWorkflowBuilder:338)
> 2014-06-25 06:36:39,925 DEBUG -
> [1871593525@qtp-1222207508-0:dataqa:POST//sync/update/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395
> 1118a58a-4a4e-4052-afea-40d4960470f3] ~ <?xml version="1.0" encoding="UTF-8"
> standalone="yes"?>
> <coordinator-app
> name="FALCON_PROCESS_DEFAULT_agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395"
> frequency="${coord:minutes(1)}" start="2014-06-25T06:34Z"
> end="2014-06-25T06:44Z" timezone="UTC" xmlns="uri:oozie:coordinator:0.3">
> <controls>
> <timeout>30</timeout>
> <concurrency>1</concurrency>
> <execution>FIFO</execution>
> <throttle>60</throttle>
> </controls>
> <datasets>
> <dataset name="inputData" frequency="${coord:minutes(1)}"
> initial-instance="2009-02-01T00:00Z" timezone="UTC">
>
> <uri-template>hdfs://192.168.138.182:8020/tmp/falcon-regression/NewPrismProcessUpdateTest/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}</uri-template>
> <done-flag></done-flag>
> </dataset>
> <dataset name="inputData2" frequency="${coord:minutes(1)}"
> initial-instance="2009-02-01T00:00Z" timezone="UTC">
>
> <uri-template>hdfs://192.168.138.182:8020/tmp/falcon-regression/NewPrismProcessUpdateTest/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}</uri-template>
> <done-flag></done-flag>
> </dataset>
> {code}
> most of the regression NewPrismProcessUpdateTest is failing coz of this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
