GitHub user jernejfrank added a comment to the discussion: APIs for updating
underlying function of a node (2 part question)
Ok I think I got it now. Atm i don't think we have any mechanism to modify the
graph after it has been created, just a lot of helper decorators for creating
the DAGs (that let you re-use functions). The closest you can come is with
config/overrides during the execution to switch things out dynamically, but
IIRC overrides can only inject values instead of nodes not actually replace the
callable in the node.
A pseudo code example to achieve this at the moment would be
```python
def _D(): ...
def X(...): ...
@pipe_input(
step(_D, ...),
)
def B(X,...): ...
@pipe_input(
step(_D, ...),
)
def C(X,...): ...
```
Now, if you need to update `X-->X'` in path through B, you have
```python
def _D(): ...
def X(...): ...
def X'(...): ...
@pipe_input(
step(_D, ...),
)
def B(X',...): ...
@pipe_input(
step(_D, ...),
)
def C(X,...): ...
```
Your request is very interesting tho. To my understanding, we could modify
override that it does 2 more things:
- accepts a callable that replaces the current callable in the node (i.e.
override a whole node with need functionality)
- add a second flag `path_through` that looks at the existing graph and creates
a new path from the override node to the target node specified by
`path_through` and duplicates those nodes and wires them correctly to not
conflict with the other branches.
The only thing I am not sure about is how this will play with caching. Am I
missing something else?
GitHub link:
https://github.com/apache/hamilton/discussions/1397#discussioncomment-14582769
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to:
[email protected]
