damccorm commented on code in PR #29782:
URL: https://github.com/apache/beam/pull/29782#discussion_r1446204612
##########
sdks/python/apache_beam/transforms/enrichment.py:
##########
@@ -15,47 +15,48 @@
# limitations under the License.
#
+from typing import Any
from typing import Callable
-from typing import Generic
+from typing import Dict
from typing import Optional
+from typing import Tuple
from typing import TypeVar
import apache_beam as beam
from apache_beam.io.requestresponse import DEFAULT_TIMEOUT_SECS
-from apache_beam.io.requestresponse import CacheReader
-from apache_beam.io.requestresponse import CacheWriter
from apache_beam.io.requestresponse import Caller
-from apache_beam.io.requestresponse import PreCallThrottler
-from apache_beam.io.requestresponse import Repeater
from apache_beam.io.requestresponse import RequestResponseIO
-from apache_beam.io.requestresponse import ShouldBackOff
__all__ = [
"EnrichmentSourceHandler",
"Enrichment",
+ "cross_join",
]
InputT = TypeVar('InputT')
OutputT = TypeVar('OutputT')
+JoinFn = Callable[[Tuple[Dict[str, Any], Dict[str, Any]]], beam.Row]
Review Comment:
Nit: Since we're exposing this to users, it's probably better to make the
JoinFn of the form:
`def join_fn(left: Dict[str, Any], right: Dict[str, Any]) -> beam.Row:`
That's easier to understand and doesn't force every user to repeat the
`left, right = element` line. This also then requires flipping the map below to:
`return fetched_data | beam.Map(lambda x: self._join_fn(x[0], x[1]))`
##########
sdks/python/apache_beam/io/requestresponse.py:
##########
@@ -111,7 +109,7 @@ class
RequestResponseIO(beam.PTransform[beam.PCollection[RequestT],
"""
def __init__(
self,
- caller: [Caller],
+ caller: Caller,
Review Comment:
To maintain stronger typing, should this be `Caller[RequestT, ResponseT]`?
That way we get a construction time error if the input pcollection type doesn't
match the caller input type. We probably need some additional plumbing to make
this work, but I think we'd benefit from stronger typing here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
