Lakshmi Manasa Gaduputi created SAMZA-2526:
----------------------------------------------
Summary: Azure blob system producer: do not commit blobs if avro
DataFileWriter.close fails
Key: SAMZA-2526
URL: https://issues.apache.org/jira/browse/SAMZA-2526
Project: Samza
Issue Type: Bug
Reporter: Lakshmi Manasa Gaduputi
Assignee: Lakshmi Manasa Gaduputi
Currently, when Avro DataFileWriter.close fails, the blob is committed with
blocks uploaded so far for that blob. This could lead to invalid blobs because,
the DataFileWriter.close does a flush which writes some magic bytes (the sync
marker to signal end of block) which when absent might lead to invalid blob.
Hence, prevent commit of blob in this scenario. This will lead to uploaded
blocks being discarded. Hence, the user of the AzureBlobSystemProducer should
now be aware of this exception bubbling up and retry producing the whole blob
again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
