[
https://issues.apache.org/jira/browse/FALCON-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687308#comment-15687308
]
Ajay Yadava commented on FALCON-1406:
-------------------------------------
Hi [~sriksun]
I know that you want to see this feature go live for a long time, but with all
sincerest intentions I believe that Falcon is better off without this change :)
I think we all can agree that this change doesn't provide any new remarkable
capabilities (capabilities that would have helped someone evaluating falcon for
their use case and found it short) but just a way of doing things without
creating temporary entities and probably more conveniently.
Temporary entities are not created just for the reasons solved by this feature,
they will be created even after this change e.g. during colo migrations,
avoiding restrictions of moving start date to an earlier time and many more. It
doesn't impact the system in any way, however, if you prefer to keep system
"clean", then providing a utiliity to "cleanse" the system of such entities is
much better and something which was proposed by the operations people who are
the most affected by this "mess".
The fact that an entity definition can change for a past instance is a rewrite
of history and in my view is a violation of fundamental "assumption". Apart
from just being a general bad practice IMHO, a lot of features developed in
falcon took the immutability of entity definition for past instances as given.
I suspect the lack of details in design doc has kept you from the true
nature/impact of the change. I hope I am able to surface some of them once
[~sandeep.samudrala] updates the design doc with other qustions that I have
asked.
Now, we can start fixing those issues on a case by case basis or we can just
reevaluate the trade off of this change in favor of a command to clean
up(addresses much wider use cases) or versioning of the entities.
> Effective time in Entity updates.
> ---------------------------------
>
> Key: FALCON-1406
> URL: https://issues.apache.org/jira/browse/FALCON-1406
> Project: Falcon
> Issue Type: New Feature
> Reporter: sandeep samudrala
> Assignee: sandeep samudrala
> Attachments: FALCON-1406-initial.patch,
> effective_time_in_entity_updates.pdf
>
>
> Effective time with entity updates needs to be provided even with past time
> too. There was effective time capability provided in the past which gives the
> functionality to set an effective time for an entity with only current or
> future time(now + delay), which could not solve all the issues.
> Following are few scenarios which would require effective time to be
> available with time back in past.
> a) New code being deployed for an incompatible input data set which would
> leave instances with old code and new data.
> b) Bad code being pushed for which, the entity should be able to go back in
> time to replay(rerun) with new code.
> c) Orchestration level changes(good/bad) would need functionality to go back
> in time to start with.
> For reference: Linking all the Jiras that have been worked upon around
> effective time .
> https://issues.apache.org/jira/browse/FALCON-374
> https://issues.apache.org/jira/browse/FALCON-297
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
