[
https://issues.apache.org/jira/browse/SAMZA-2526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17143210#comment-17143210
]
Lakshmi Manasa Gaduputi commented on SAMZA-2526:
------------------------------------------------
Fixed in PR: [https://github.com/apache/samza/pull/1362]
> Azure blob system producer: do not commit blobs if avro DataFileWriter.close
> fails
> ----------------------------------------------------------------------------------
>
> Key: SAMZA-2526
> URL: https://issues.apache.org/jira/browse/SAMZA-2526
> Project: Samza
> Issue Type: Bug
> Reporter: Lakshmi Manasa Gaduputi
> Assignee: Lakshmi Manasa Gaduputi
> Priority: Minor
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, when Avro DataFileWriter.close fails, the blob is committed with
> blocks uploaded so far for that blob. This could lead to invalid blobs
> because, the DataFileWriter.close does a flush which writes some magic bytes
> (the sync marker to signal end of block) which when absent might lead to
> invalid blob.
> Hence, prevent commit of blob in this scenario. This will lead to uploaded
> blocks being discarded. Hence, the user of the AzureBlobSystemProducer should
> now be aware of this exception bubbling up and retry producing the whole blob
> again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
