GitHub user gidhubuser255 created a discussion: APIs for updating underlying
function of a node (2 part question)
### (1) Updating function logic
For reference consider the following DAG
```
def A(B: int, C: int) -> int:
return B + C
def B(D: int) -> int:
return D + 1
def C(D: int) -> int:
return D + 2
def D() -> int:
return 7
dr = driver.Builder().with_modules(__import__('__main__')).build()
dr.execute('A') # returns 17
```
If I want to update the logic of `B` in the DAG with some function
`alternative_B`, the best way I've found to do so is to do the following:
```
@config.when(use_alternative_b=False)
def B(D: int) -> int:
return D + 1
@config.when(name='B', use_alternative_b=True)
def alternative_B(C: int, D: int) -> int:
return C + D
dr = (
driver.Builder()
.with_modules(__import__('__main__'))
.with_config(use_alternative_b=True)
.build()
)
dr.execute('A') # returns 25
```
Which is a bit cumbersome, since it requires modifying source code and adding
configuration primitives to enable the ability.
An alternative way to do this is how the fn_graph library handles this
(https://fn-graph.readthedocs.io/en/latest/usage.html#building-up-logic), which
provides a similar ability to update the implementation of a node by allowing
you to make updates on your "driver" (called composer in fn_graph). For example
to achieve the same ability in fn_graph you can do this, no decorators or
additional config entities needed:
```
composer = Composer().update(A, B, C, D) # these are all the same defined
functions as above
composer2 = composer.update(B=alternative_B) # just plain defined function, no
decorators
composer3 = composer2.update(B=alternative_B2)
composer2.A()
composer3.A()
```
**Questions:**
1. Is there a similar existing mechanism to this in Hamilton?
2. If not could it be added?
### (2) Updating function logic in a specific branch
Similar to the above consider when you want to update the implementation for a
function, but only when the path from that function to the root executed
function passes through another specific node/function.
For example if I want to update `D` to `D__prime` but only in the path that
includes `B`, in Hamilton I could achieve that like so:
```
def A(B: int, C: int) -> int:
return B + C
def B(D_thru_B: int) -> int:
return D + 1
def C(D: int) -> int:
return D + 2
@config.when(use_d_prime=False)
def D_thru_B(D: int) -> int:
return D
@config.when(name='D_thru_B', use_d_prime=True)
def D__prime(D: int) -> int:
return D * 2
def D() -> int:
return 7
```
But this has a couple problems: (a) if I want to later update all references to
`D` with `D__prime` (not just through `B`) I'll need to add another `D__prime2`
with a `config.when(name='D', ...)` and a new config parameter, which is again
pretty cumbersome. And more importantly (b) this method breaks down when the
branch specific replacement happens behind a converging point, for example:
```
def A(B: int, C: int) -> int:
return B + C
def B(D: int) -> int:
return D + 1
def C(D: int) -> int:
return D + 2
def D(X: int) -> int:
return X + 1
def X() -> int:
return 9
```
If I want to replace `X` in the path of `B` with this method I would need to
bifurcate all my functions starting at the converging point (`D_thru_B`,
`X_thru_B`, `D_thru_C`, `X_thru_C`, etc), which would be prohibitive for
anything more than a very simple example.
**Questions:**
1. Is there a good existing way to achieve this? I'm thinking there's probably
a way using the existing `Parallelizable` and `Collect` mechanism which can
already dynamically generate new nodes to add to the graph, and which could be
used to do the bifurcation I mentioned?
2. Would it be possible to add the ability to do this with something more like
the fn_graph style api, something like `driver.update(X=X__prime,
in_path=['B']) `
GitHub link: https://github.com/apache/hamilton/discussions/1397
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to:
[email protected]
