There are 2 execution times when using Beam. The first execution is local,
when a pipeline is constructed, and the second is remote on the runner,
processing data.
Based on what you said, it sounds like you are wrapping pipeline
construction in a try-catch, and constructing FileIO isn't failing.
e.g.
try {
FileIO.write().someOtherconfigs()
} catch ...
this will catch any exceptions in constructing fileio, but the running
pipeline won't propagate exceptions through this exception block.
On Tue, Nov 7, 2023 at 5:21 PM Robert Bradshaw via dev <dev@beam.apache.org>
wrote:
> File write failures should be throwing exceptions that will terminate the
> pipeline on failure. (Generally a distributed runner will make multiple
> attempts before abandoning the entire pipeline of course.)
>
> Are you seeing files failing to be written but no exceptions being thrown?
> If so, this is definitely a bug that we want to resolve.
>
>
> On Tue, Nov 7, 2023 at 11:17 AM Ramya Prasad via dev <dev@beam.apache.org>
> wrote:
>
>> Hello,
>>
>> I am a developer using Apache Beam in my Java application, and I need
>> some help on how to handle exceptions when writing a file to S3. I have
>> tried wrapping my code within a try-catch block, but no exception is being
>> thrown within the try block. I'm assuming that FileIO doesn't throw any
>> exceptions upon failure. Is there a way in which I can either terminate the
>> program on failure or at least be made aware of if any of my write
>> operations fail?
>>
>> Thanks and sincerely,
>> Ramya
>> ------------------------------
>>
>> The information contained in this e-mail may be confidential and/or
>> proprietary to Capital One and/or its affiliates and may only be used
>> solely in performance of work or services for Capital One. The information
>> transmitted herewith is intended only for use by the individual or entity
>> to which it is addressed. If the reader of this message is not the intended
>> recipient, you are hereby notified that any review, retransmission,
>> dissemination, distribution, copying or other use of, or taking of any
>> action in reliance upon this information is strictly prohibited. If you
>> have received this communication in error, please contact the sender and
>> delete the material from your computer.
>>
>>
>>
>>
>>
