[
https://issues.apache.org/jira/browse/FLINK-38725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karan Makhija updated FLINK-38725:
----------------------------------
Description:
Avro schema which with BigDecimal gives the following exception. The cluster
configuration included a flink-conf property: "env.java.opts":
"-Dorg.apache.flink.avro.shaded.org.apache.avro.SERIALIZABLE_PACKAGESS=java.math
-Dorg.apache.avro.SERIALIZABLE_PACKAGES=java.math".
Caused by: java.lang.SecurityException: Forbidden class java.math.BigDecimal!
This class is not trusted to be included in Avro schema using java-class.
Please set org.apache.avro.SERIALIZABLE_PACKAGES system property with the
packages you trust. at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.checkSecurity([SpecificDatumReader.java:145|http://specificdatumreader.java:145/])
at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.getPropAsClass([SpecificDatumReader.java:119|http://specificdatumreader.java:119/])
at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.findStringClass([SpecificDatumReader.java:102|http://specificdatumreader.java:102/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader$ReaderCache.lambda$getStringClass$1([GenericDatumReader.java:567|http://genericdatumreader.java:567/])
at
java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent([ConcurrentHashMap.java:1705|http://concurrenthashmap.java:1705/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader$ReaderCache.getStringClass([GenericDatumReader.java:567|http://genericdatumreader.java:567/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readString([GenericDatumReader.java:455|http://genericdatumreader.java:455/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readWithoutConversion([GenericDatumReader.java:192|http://genericdatumreader.java:192/])
at
org.apache.flink.avro.shaded.org.apache.avro.reflect.ReflectDatumReader.readField([ReflectDatumReader.java:301|http://reflectdatumreader.java:301/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readRecord([GenericDatumReader.java:248|http://genericdatumreader.java:248/])
at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.readRecord([SpecificDatumReader.java:168|http://specificdatumreader.java:168/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readWithoutConversion([GenericDatumReader.java:180|http://genericdatumreader.java:180/])
at
[org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.read|http://org.apache.flink.avro.shaded.org.apache.avro.generic.genericdatumreader.read/]([GenericDatumReader.java:161|http://genericdatumreader.java:161/])
at
[org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.read|http://org.apache.flink.avro.shaded.org.apache.avro.generic.genericdatumreader.read/]([GenericDatumReader.java:154|http://genericdatumreader.java:154/])
at
[org.apache.flink.avro.shaded.org.apache.avro.file.DataFileStream.next|http://org.apache.flink.avro.shaded.org.apache.avro.file.datafilestream.next/]([DataFileStream.java:263|http://datafilestream.java:263/])
at
org.apache.flink.formats.avro.AvroInputFormat.nextRecord([AvroInputFormat.java:170|http://avroinputformat.java:170/])
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.readAndCollectRecord([ContinuousFileReaderOperator.java:390|http://continuousfilereaderoperator.java:390/])
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.processRecord([ContinuousFileReaderOperator.java:352|http://continuousfilereaderoperator.java:352/])
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.lambda$new$0([ContinuousFileReaderOperator.java:240|http://continuousfilereaderoperator.java:240/])
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing([StreamTaskActionExecutor.java:50|http://streamtaskactionexecutor.java:50/])
at
[org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run|http://org.apache.flink.streaming.runtime.tasks.mailbox.mail.run/]([Mail.java:101|http://mail.java:101/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail([MailboxProcessor.java:414|http://mailboxprocessor.java:414/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking([MailboxProcessor.java:399|http://mailboxprocessor.java:399/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail([MailboxProcessor.java:361|http://mailboxprocessor.java:361/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop([MailboxProcessor.java:229|http://mailboxprocessor.java:229/])
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop([StreamTask.java:973|http://streamtask.java:973/])
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke([StreamTask.java:917|http://streamtask.java:917/])
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring([Task.java:970|http://task.java:970/])
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke([Task.java:949|http://task.java:949/])
at
org.apache.flink.runtime.taskmanager.Task.doRun([Task.java:763|http://task.java:763/])
at
[org.apache.flink.runtime.taskmanager.Task.run|http://org.apache.flink.runtime.taskmanager.task.run/]([Task.java:575|http://task.java:575/])
at java.base/java.lang.Thread.run([Thread.java:829|http://thread.java:829/])
The same is working after I remove the relocations from the flink-sql-avro.jar.
{code:java}
flink run flink-job.jar --source s3://<S3-BUCKET>/data/
2025-11-20 11:12:32,971 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] -
Found Yarn properties file under /var/lib/flink/yarn/.yarn-properties-hadoop.
2025-11-20 11:12:32,971 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] -
Found Yarn properties file under /var/lib/flink/yarn/.yarn-properties-hadoop.
2025-11-20 11:12:33,665 INFO org.apache.hadoop.metrics2.impl.MetricsConfig [] -
Loaded properties from hadoop-metrics2.properties
2025-11-20 11:12:33,678 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl
[] - Scheduled Metric snapshot period at 300 second(s).
2025-11-20 11:12:33,678 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl
[] - s3a-file-system metrics system started
Reading data source 1: }}s3://<S3-BUCKET>/data/{} Skipping filter step Adding
execution job 2025-11-20 11:12:34,847 INFO
org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider [] -
Connecting to ResourceManager at ip-172-31-16-8.ec2.internal/172.31.16.8:8032
2025-11-20 11:12:34,932 INFO org.apache.hadoop.yarn.client.AHSProxy [] -
Connecting to Application History server at
ip-172-31-16-8.ec2.internal/172.31.16.8:10200 2025-11-20 11:12:34,938 INFO
org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar
passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor
to locate the jar 2025-11-20 11:12:35,026 INFO
org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface
ip-172-31-30-122.ec2.internal:41139 of application
'application_1763613799717_0008'. Job has been submitted with JobID
1bad2e49ee0da8894a65a98133b894ce Program execution finished Job with JobID
1bad2e49ee0da8894a65a98133b894ce has finished.
Job Runtime: 21012 ms Total input records: 0 Total output records: 0 timestamp:
2025-11-20T11:12:58.596487112Z inputTotal: 0 outputTotal: 0 parameters:
{source=s3://<S3-BUCKET>/data/}{code}
was:
Avro schema which with BigDecimal gives the following exception:
Caused by: java.lang.SecurityException: Forbidden class java.math.BigDecimal!
This class is not trusted to be included in Avro schema using java-class.
Please set org.apache.avro.SERIALIZABLE_PACKAGES system property with the
packages you trust. at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.checkSecurity([SpecificDatumReader.java:145|http://specificdatumreader.java:145/])
at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.getPropAsClass([SpecificDatumReader.java:119|http://specificdatumreader.java:119/])
at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.findStringClass([SpecificDatumReader.java:102|http://specificdatumreader.java:102/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader$ReaderCache.lambda$getStringClass$1([GenericDatumReader.java:567|http://genericdatumreader.java:567/])
at
java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent([ConcurrentHashMap.java:1705|http://concurrenthashmap.java:1705/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader$ReaderCache.getStringClass([GenericDatumReader.java:567|http://genericdatumreader.java:567/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readString([GenericDatumReader.java:455|http://genericdatumreader.java:455/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readWithoutConversion([GenericDatumReader.java:192|http://genericdatumreader.java:192/])
at
org.apache.flink.avro.shaded.org.apache.avro.reflect.ReflectDatumReader.readField([ReflectDatumReader.java:301|http://reflectdatumreader.java:301/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readRecord([GenericDatumReader.java:248|http://genericdatumreader.java:248/])
at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.readRecord([SpecificDatumReader.java:168|http://specificdatumreader.java:168/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readWithoutConversion([GenericDatumReader.java:180|http://genericdatumreader.java:180/])
at
[org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.read|http://org.apache.flink.avro.shaded.org.apache.avro.generic.genericdatumreader.read/]([GenericDatumReader.java:161|http://genericdatumreader.java:161/])
at
[org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.read|http://org.apache.flink.avro.shaded.org.apache.avro.generic.genericdatumreader.read/]([GenericDatumReader.java:154|http://genericdatumreader.java:154/])
at
[org.apache.flink.avro.shaded.org.apache.avro.file.DataFileStream.next|http://org.apache.flink.avro.shaded.org.apache.avro.file.datafilestream.next/]([DataFileStream.java:263|http://datafilestream.java:263/])
at
org.apache.flink.formats.avro.AvroInputFormat.nextRecord([AvroInputFormat.java:170|http://avroinputformat.java:170/])
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.readAndCollectRecord([ContinuousFileReaderOperator.java:390|http://continuousfilereaderoperator.java:390/])
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.processRecord([ContinuousFileReaderOperator.java:352|http://continuousfilereaderoperator.java:352/])
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.lambda$new$0([ContinuousFileReaderOperator.java:240|http://continuousfilereaderoperator.java:240/])
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing([StreamTaskActionExecutor.java:50|http://streamtaskactionexecutor.java:50/])
at
[org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run|http://org.apache.flink.streaming.runtime.tasks.mailbox.mail.run/]([Mail.java:101|http://mail.java:101/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail([MailboxProcessor.java:414|http://mailboxprocessor.java:414/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking([MailboxProcessor.java:399|http://mailboxprocessor.java:399/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail([MailboxProcessor.java:361|http://mailboxprocessor.java:361/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop([MailboxProcessor.java:229|http://mailboxprocessor.java:229/])
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop([StreamTask.java:973|http://streamtask.java:973/])
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke([StreamTask.java:917|http://streamtask.java:917/])
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring([Task.java:970|http://task.java:970/])
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke([Task.java:949|http://task.java:949/])
at
org.apache.flink.runtime.taskmanager.Task.doRun([Task.java:763|http://task.java:763/])
at
[org.apache.flink.runtime.taskmanager.Task.run|http://org.apache.flink.runtime.taskmanager.task.run/]([Task.java:575|http://task.java:575/])
at java.base/java.lang.Thread.run([Thread.java:829|http://thread.java:829/])
The cluster configuration included a flink-conf property: "env.java.opts":
"-Dorg.apache.flink.avro.shaded.org.apache.avro.SERIALIZABLE_PACKAGESS=java.math
-Dorg.apache.avro.SERIALIZABLE_PACKAGES=java.math". The same is working after
I remove the relocations from the flink-sql-avro.jar.
{{[hadoop@ip-172-31-16-8 ~]$ flink run flink-job.jar --source
s3://<S3-BUCKET>/data/
2025-11-20 11:12:32,971 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
[] - Found Yarn properties file under
/var/lib/flink/yarn/.yarn-properties-hadoop.
2025-11-20 11:12:32,971 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
[] - Found Yarn properties file under
/var/lib/flink/yarn/.yarn-properties-hadoop. 2025-11-20 11:12:33,665 INFO
org.apache.hadoop.metrics2.impl.MetricsConfig [] - Loaded
properties from hadoop-metrics2.properties
2025-11-20 11:12:33,678 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl
[] - Scheduled Metric snapshot period at 300 second(s).
2025-11-20 11:12:33,678 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl
[] - s3a-file-system metrics system started
Reading data source 1: }}{{s3://<S3-BUCKET>/data/}}{{{} Skipping filter step
Adding execution job 2025-11-20 11:12:34,847 INFO
org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider [] -
Connecting to ResourceManager at ip-172-31-16-8.ec2.internal/172.31.16.8:8032
2025-11-20 11:12:34,932 INFO org.apache.hadoop.yarn.client.AHSProxy [] -
Connecting to Application History server at
ip-172-31-16-8.ec2.internal/172.31.16.8:10200 2025-11-20 11:12:34,938 INFO
org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar
passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor
to locate the jar 2025-11-20 11:12:35,026 INFO
org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface
ip-172-31-30-122.ec2.internal:41139 of application
'application_1763613799717_0008'. Job has been submitted with JobID
1bad2e49ee0da8894a65a98133b894ce Program execution finished Job with JobID
1bad2e49ee0da8894a65a98133b894ce has finished. Job Runtime: 21012 ms Total
input records: 0 Total output records: 0 timestamp:
2025-11-20T11:12:58.596487112Z inputTotal: 0 outputTotal: 0 parameters:
{source={}}}{{{}s3://<S3-BUCKET>/data/{}}}{{{}}{}}}
> Flink avro gives Security Exception with flink-sql-avro.jar
> -----------------------------------------------------------
>
> Key: FLINK-38725
> URL: https://issues.apache.org/jira/browse/FLINK-38725
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.20.0
> Reporter: Karan Makhija
> Priority: Major
>
> Avro schema which with BigDecimal gives the following exception. The cluster
> configuration included a flink-conf property: "env.java.opts":
> "-Dorg.apache.flink.avro.shaded.org.apache.avro.SERIALIZABLE_PACKAGESS=java.math
> -Dorg.apache.avro.SERIALIZABLE_PACKAGES=java.math".
> Caused by: java.lang.SecurityException: Forbidden class java.math.BigDecimal!
> This class is not trusted to be included in Avro schema using java-class.
> Please set org.apache.avro.SERIALIZABLE_PACKAGES system property with the
> packages you trust. at
> org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.checkSecurity([SpecificDatumReader.java:145|http://specificdatumreader.java:145/])
> at
> org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.getPropAsClass([SpecificDatumReader.java:119|http://specificdatumreader.java:119/])
> at
> org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.findStringClass([SpecificDatumReader.java:102|http://specificdatumreader.java:102/])
> at
> org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader$ReaderCache.lambda$getStringClass$1([GenericDatumReader.java:567|http://genericdatumreader.java:567/])
> at
> java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent([ConcurrentHashMap.java:1705|http://concurrenthashmap.java:1705/])
> at
> org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader$ReaderCache.getStringClass([GenericDatumReader.java:567|http://genericdatumreader.java:567/])
> at
> org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readString([GenericDatumReader.java:455|http://genericdatumreader.java:455/])
> at
> org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readWithoutConversion([GenericDatumReader.java:192|http://genericdatumreader.java:192/])
> at
> org.apache.flink.avro.shaded.org.apache.avro.reflect.ReflectDatumReader.readField([ReflectDatumReader.java:301|http://reflectdatumreader.java:301/])
> at
> org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readRecord([GenericDatumReader.java:248|http://genericdatumreader.java:248/])
> at
> org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.readRecord([SpecificDatumReader.java:168|http://specificdatumreader.java:168/])
> at
> org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readWithoutConversion([GenericDatumReader.java:180|http://genericdatumreader.java:180/])
> at
> [org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.read|http://org.apache.flink.avro.shaded.org.apache.avro.generic.genericdatumreader.read/]([GenericDatumReader.java:161|http://genericdatumreader.java:161/])
> at
> [org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.read|http://org.apache.flink.avro.shaded.org.apache.avro.generic.genericdatumreader.read/]([GenericDatumReader.java:154|http://genericdatumreader.java:154/])
> at
> [org.apache.flink.avro.shaded.org.apache.avro.file.DataFileStream.next|http://org.apache.flink.avro.shaded.org.apache.avro.file.datafilestream.next/]([DataFileStream.java:263|http://datafilestream.java:263/])
> at
> org.apache.flink.formats.avro.AvroInputFormat.nextRecord([AvroInputFormat.java:170|http://avroinputformat.java:170/])
> at
> org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.readAndCollectRecord([ContinuousFileReaderOperator.java:390|http://continuousfilereaderoperator.java:390/])
> at
> org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.processRecord([ContinuousFileReaderOperator.java:352|http://continuousfilereaderoperator.java:352/])
> at
> org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.lambda$new$0([ContinuousFileReaderOperator.java:240|http://continuousfilereaderoperator.java:240/])
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing([StreamTaskActionExecutor.java:50|http://streamtaskactionexecutor.java:50/])
> at
> [org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run|http://org.apache.flink.streaming.runtime.tasks.mailbox.mail.run/]([Mail.java:101|http://mail.java:101/])
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail([MailboxProcessor.java:414|http://mailboxprocessor.java:414/])
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking([MailboxProcessor.java:399|http://mailboxprocessor.java:399/])
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail([MailboxProcessor.java:361|http://mailboxprocessor.java:361/])
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop([MailboxProcessor.java:229|http://mailboxprocessor.java:229/])
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop([StreamTask.java:973|http://streamtask.java:973/])
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke([StreamTask.java:917|http://streamtask.java:917/])
> at
> org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring([Task.java:970|http://task.java:970/])
> at
> org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke([Task.java:949|http://task.java:949/])
> at
> org.apache.flink.runtime.taskmanager.Task.doRun([Task.java:763|http://task.java:763/])
> at
> [org.apache.flink.runtime.taskmanager.Task.run|http://org.apache.flink.runtime.taskmanager.task.run/]([Task.java:575|http://task.java:575/])
> at java.base/java.lang.Thread.run([Thread.java:829|http://thread.java:829/])
> The same is working after I remove the relocations from the
> flink-sql-avro.jar.
>
> {code:java}
> flink run flink-job.jar --source s3://<S3-BUCKET>/data/
> 2025-11-20 11:12:32,971 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli []
> - Found Yarn properties file under
> /var/lib/flink/yarn/.yarn-properties-hadoop.
> 2025-11-20 11:12:32,971 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli []
> - Found Yarn properties file under
> /var/lib/flink/yarn/.yarn-properties-hadoop. 2025-11-20 11:12:33,665 INFO
> org.apache.hadoop.metrics2.impl.MetricsConfig [] - Loaded properties from
> hadoop-metrics2.properties
> 2025-11-20 11:12:33,678 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl [] - Scheduled Metric
> snapshot period at 300 second(s).
> 2025-11-20 11:12:33,678 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl [] - s3a-file-system
> metrics system started
> Reading data source 1: }}s3://<S3-BUCKET>/data/{} Skipping filter step Adding
> execution job 2025-11-20 11:12:34,847 INFO
> org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider [] -
> Connecting to ResourceManager at ip-172-31-16-8.ec2.internal/172.31.16.8:8032
> 2025-11-20 11:12:34,932 INFO org.apache.hadoop.yarn.client.AHSProxy [] -
> Connecting to Application History server at
> ip-172-31-16-8.ec2.internal/172.31.16.8:10200 2025-11-20 11:12:34,938 INFO
> org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar
> passed. Using the location of class
> org.apache.flink.yarn.YarnClusterDescriptor to locate the jar 2025-11-20
> 11:12:35,026 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web
> Interface ip-172-31-30-122.ec2.internal:41139 of application
> 'application_1763613799717_0008'. Job has been submitted with JobID
> 1bad2e49ee0da8894a65a98133b894ce Program execution finished Job with JobID
> 1bad2e49ee0da8894a65a98133b894ce has finished.
> Job Runtime: 21012 ms Total input records: 0 Total output records: 0
> timestamp: 2025-11-20T11:12:58.596487112Z inputTotal: 0 outputTotal: 0
> parameters: {source=s3://<S3-BUCKET>/data/}{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
