Hello, By default the sink will not fail, the underlying connector has a flag "failOnError" which defaults to false. Unfortunately this cannot be set for Flink 1.14 in the Table API, however in 1.15 it can via 'sink.fail-on-error: true'
Thanks On Wed, Nov 30, 2022 at 5:41 AM Dan Hill <quietgol...@gmail.com> wrote: > My text logs don't have a stack trace with this exception. I'm doing this > inside Flink SQL with a standard Kinesis connector and JSON formatter. > > On Tue, Nov 29, 2022 at 6:38 PM yuxia <luoyu...@alumni.sjtu.edu.cn> wrote: > >> Which code line the error message happens? Maybe it will swallow the >> exception and then log the error message, in which case Flink job won't >> fail since it seems like no exception happens. >> >> Best regards, >> Yuxia >> >> ------------------------------ >> *发件人: *"Dan Hill" <quietgol...@gmail.com> >> *收件人: *"User" <user@flink.apache.org> >> *发送时间: *星期三, 2022年 11 月 30日 上午 8:06:52 >> *主题: *Flink Table Kinesis sink not failing when sink fails >> >> I set up a simple Flink SQL job (Flink v1.14.4) that writes to Kinesis. >> The job looks healthy but the records are not being written. I did not >> give enough IAM permissions to write to Kinesis. However, the Flink SQL >> job acts like it's healthy and checkpoints even though the Kinesis PutRecords >> call fails. I'd expect this error to kill the Flink job. >> I looked through Flink Jira and the Flink user group but didn't see a >> similar issue. >> >> Is the silent failure a known issue? If the Flink job doesn't fail, >> it'll be hard to detect production issues. >> >> ``` >> >> 2022-11-29 23:30:27,587 ERROR >> org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.LogInputStreamReader >> [] - [2022-11-29 23:30:27.578072] [0x00001e3b][0x00007f12ef8fc700] [error] >> [AWS Log: ERROR](AWSClient)HTTP response code: 400 >> Exception name: AccessDeniedException >> Error message: User: >> arn:aws:sts::055315558257:assumed-role/dev-workers-us-east-1b-2022031014331389150000000a/i-09e4f747a4bdbb1f0 >> is not authorized to perform: kinesis:ListShards on resource: >> arn:aws:kinesis:us-east-1:055315558257:stream/dan-dev-content-metrics >> because no identity-based policy allows the kinesis:ListShards action >> 6 response headers: >> connection : close >> content-length : 379 >> content-type : application/x-amz-json-1.1 >> date : Tue, 29 Nov 2022 23:30:27 GMT >> x-amz-id-2 : >> q8kuplUOMJILzVU97YA+TYSyy6aozeoST+yws26rOkyzEUUZT0zKBdcMWUAjV/8RrnMeed/+em7CbjpwzGYEANgkwCihZWdC >> x-amzn-requestid : e4a39674-66fa-4dcd-b8a3-0e273e5e628a >> >> ``` >> >>
