charlespnh opened a new issue, #35719:
URL: https://github.com/apache/beam/issues/35719
### What happened?
Run the following pipeline `tmp.yaml`:
```
pipeline:
transforms:
- type: Create
name: Create1
config:
elements:
- ride_id: "1"
a: "pickup"
- type: Create
name: Create2
config:
elements:
- ride_id: "1"
b: "dropoff"
- type: Join
name: Join
input:
pickup: Create1
dropoff: Create2
config:
equalities: ride_id
type: inner
fields:
pickup: [ride_id, a]
dropoff: [b]
- type: LogForTesting
name: LogForTesting
input: Join
tests:
- name: MyUnitTest
expected_inputs:
- name: LogForTesting
elements:
- ride_id: '1'
a: 'pickup'
b: 'dropoff'
```
... with the following command:
```
python -m apache_beam.yaml.main --test \
--yaml_pipeline_file tmp.yaml
```
... gives the following error:
```
...
File
"/Users/charlesnguyen/.virtualenvs/beam/lib/python3.11/site-packages/apache_beam/internal/cloudpickle/cloudpickle.py",
line 1304, in dump
return super().dump(obj)
^^^^^^^^^^^^^^^^^
File "stringsource", line 2, in
grpc._cython.cygrpc.Channel.__reduce_cython__
TypeError: no default __reduce__ due to non-trivial __cinit__
...
```
This is using Beam `2.66.0`. There's something going on with YAML Join
transform that doesn't go well with the YAML testing framework.
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Infrastructure
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
