[
https://issues.apache.org/jira/browse/BEAM-12391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17388237#comment-17388237
]
Ahmet Altay commented on BEAM-12391:
------------------------------------
This could be a good starter bug.
As the description explained, `fo` is not always a public member of Writer. It
is with fastavro cython implementation
(https://github.com/fastavro/fastavro/blob/e4ce5237dd8e378fb8874e2f8a9ebb587c2b550f/fastavro/_write.pyx#L576),
and it is not with native pyhton implementation
(https://github.com/fastavro/fastavro/blob/e4ce5237dd8e378fb8874e2f8a9ebb587c2b550f/fastavro/_write_py.py#L433)
This line in beam avroio: (at
https://github.com/aaltay/beam/blob/eb8ceeb297a4ccaf0fd8a411a9486f2af962f052/sdks/python/apache_beam/io/avroio.py#L621)
`writer.fo.close()` assumes that `fo` is always a public member but it is not.
One option would be to check that `fo` exists then call close, or a better
option would be close `file_handle` created in `open` instead.
> WriteToAvro fails if fastavro loads its python implementation of writer
> -----------------------------------------------------------------------
>
> Key: BEAM-12391
> URL: https://issues.apache.org/jira/browse/BEAM-12391
> Project: Beam
> Issue Type: Bug
> Components: io-py-avro
> Affects Versions: 2.25.0, 2.26.0, 2.27.0, 2.28.0, 2.29.0
> Reporter: Chris Chandler
> Priority: P2
> Labels: stale-P2, starter
>
> It's possible for fastavro to fail to correctly load its cython
> implementation of the Writer class in which case it will silently fall back
> to a pure python implementation. If this happens there's no outward
> indication, but line 621 in io/avroio.py will fail because writer.fo is only
> present on the cython implementation.
> To reproduce you can modify fastavro's write.py to only use its fallback:
> {code}
> #from . import _write
> from . import _write_py as _write
> {code}
> And then run a workflow that sinks to WriteToAvro(use_fastavro=True).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
