damccorm commented on code in PR #35952:
URL: https://github.com/apache/beam/pull/35952#discussion_r2301893947
##########
sdks/python/apache_beam/yaml/tests/assign_timestamps.yaml:
##########
@@ -82,3 +82,118 @@ pipelines:
config:
elements:
- {user: bob, timestamp: 3}
+
+# Assign timestamp to beam row element with error handling and output schema
+# check.
+ - pipeline:
+ type: composite
+ transforms:
+ - type: Create
+ name: CreateVisits
+ config:
+ elements:
+ - {user: alice, timestamp: "not-valid"}
+ - {user: bob, timestamp: 3}
+ - type: AssignTimestamps
+ input: CreateVisits
+ config:
+ timestamp: timestamp
+ error_handling:
+ output: invalid_rows
+ output_schema:
+ type: object
+ properties:
+ user:
+ type: string
+ timestamp:
+ type: integer
+ error_handling:
+ output: invalid_schema_rows
Review Comment:
In practice, they're both probably going to be data issues which users want
to pipe to some exception handling sink, so I think treating them as the same
collection probably makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
