Karan Makhija created FLINK-38725:
-------------------------------------
Summary: Flink avro gives Security Exception with
flink-sql-avro.jar
Key: FLINK-38725
URL: https://issues.apache.org/jira/browse/FLINK-38725
Project: Flink
Issue Type: Bug
Affects Versions: 1.20.0
Reporter: Karan Makhija
Avro schema which with BigDecimal gives the following exception:
Caused by: java.lang.SecurityException: Forbidden class java.math.BigDecimal!
This class is not trusted to be included in Avro schema using java-class.
Please set org.apache.avro.SERIALIZABLE_PACKAGES system property with the
packages you trust. at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.checkSecurity([SpecificDatumReader.java:145|http://specificdatumreader.java:145/])
at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.getPropAsClass([SpecificDatumReader.java:119|http://specificdatumreader.java:119/])
at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.findStringClass([SpecificDatumReader.java:102|http://specificdatumreader.java:102/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader$ReaderCache.lambda$getStringClass$1([GenericDatumReader.java:567|http://genericdatumreader.java:567/])
at
java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent([ConcurrentHashMap.java:1705|http://concurrenthashmap.java:1705/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader$ReaderCache.getStringClass([GenericDatumReader.java:567|http://genericdatumreader.java:567/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readString([GenericDatumReader.java:455|http://genericdatumreader.java:455/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readWithoutConversion([GenericDatumReader.java:192|http://genericdatumreader.java:192/])
at
org.apache.flink.avro.shaded.org.apache.avro.reflect.ReflectDatumReader.readField([ReflectDatumReader.java:301|http://reflectdatumreader.java:301/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readRecord([GenericDatumReader.java:248|http://genericdatumreader.java:248/])
at
org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.readRecord([SpecificDatumReader.java:168|http://specificdatumreader.java:168/])
at
org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readWithoutConversion([GenericDatumReader.java:180|http://genericdatumreader.java:180/])
at
[org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.read|http://org.apache.flink.avro.shaded.org.apache.avro.generic.genericdatumreader.read/]([GenericDatumReader.java:161|http://genericdatumreader.java:161/])
at
[org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.read|http://org.apache.flink.avro.shaded.org.apache.avro.generic.genericdatumreader.read/]([GenericDatumReader.java:154|http://genericdatumreader.java:154/])
at
[org.apache.flink.avro.shaded.org.apache.avro.file.DataFileStream.next|http://org.apache.flink.avro.shaded.org.apache.avro.file.datafilestream.next/]([DataFileStream.java:263|http://datafilestream.java:263/])
at
org.apache.flink.formats.avro.AvroInputFormat.nextRecord([AvroInputFormat.java:170|http://avroinputformat.java:170/])
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.readAndCollectRecord([ContinuousFileReaderOperator.java:390|http://continuousfilereaderoperator.java:390/])
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.processRecord([ContinuousFileReaderOperator.java:352|http://continuousfilereaderoperator.java:352/])
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.lambda$new$0([ContinuousFileReaderOperator.java:240|http://continuousfilereaderoperator.java:240/])
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing([StreamTaskActionExecutor.java:50|http://streamtaskactionexecutor.java:50/])
at
[org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run|http://org.apache.flink.streaming.runtime.tasks.mailbox.mail.run/]([Mail.java:101|http://mail.java:101/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail([MailboxProcessor.java:414|http://mailboxprocessor.java:414/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking([MailboxProcessor.java:399|http://mailboxprocessor.java:399/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail([MailboxProcessor.java:361|http://mailboxprocessor.java:361/])
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop([MailboxProcessor.java:229|http://mailboxprocessor.java:229/])
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop([StreamTask.java:973|http://streamtask.java:973/])
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke([StreamTask.java:917|http://streamtask.java:917/])
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring([Task.java:970|http://task.java:970/])
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke([Task.java:949|http://task.java:949/])
at
org.apache.flink.runtime.taskmanager.Task.doRun([Task.java:763|http://task.java:763/])
at
[org.apache.flink.runtime.taskmanager.Task.run|http://org.apache.flink.runtime.taskmanager.task.run/]([Task.java:575|http://task.java:575/])
at java.base/java.lang.Thread.run([Thread.java:829|http://thread.java:829/])
The cluster configuration included a flink-conf property: "env.java.opts":
"-Dorg.apache.flink.avro.shaded.org.apache.avro.SERIALIZABLE_PACKAGESS=java.math
-Dorg.apache.avro.SERIALIZABLE_PACKAGES=java.math". The same is working after
I remove the relocations from the flink-sql-avro.jar.
{{[hadoop@ip-172-31-16-8 ~]$ flink run flink-job.jar --source
s3://<S3-BUCKET>/data/
2025-11-20 11:12:32,971 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
[] - Found Yarn properties file under
/var/lib/flink/yarn/.yarn-properties-hadoop.
2025-11-20 11:12:32,971 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
[] - Found Yarn properties file under
/var/lib/flink/yarn/.yarn-properties-hadoop. 2025-11-20 11:12:33,665 INFO
org.apache.hadoop.metrics2.impl.MetricsConfig [] - Loaded
properties from hadoop-metrics2.properties
2025-11-20 11:12:33,678 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl
[] - Scheduled Metric snapshot period at 300 second(s).
2025-11-20 11:12:33,678 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl
[] - s3a-file-system metrics system started
Reading data source 1: }}{{s3://<S3-BUCKET>/data/}}{{{} Skipping filter step
Adding execution job 2025-11-20 11:12:34,847 INFO
org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider [] -
Connecting to ResourceManager at ip-172-31-16-8.ec2.internal/172.31.16.8:8032
2025-11-20 11:12:34,932 INFO org.apache.hadoop.yarn.client.AHSProxy [] -
Connecting to Application History server at
ip-172-31-16-8.ec2.internal/172.31.16.8:10200 2025-11-20 11:12:34,938 INFO
org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar
passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor
to locate the jar 2025-11-20 11:12:35,026 INFO
org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface
ip-172-31-30-122.ec2.internal:41139 of application
'application_1763613799717_0008'. Job has been submitted with JobID
1bad2e49ee0da8894a65a98133b894ce Program execution finished Job with JobID
1bad2e49ee0da8894a65a98133b894ce has finished. Job Runtime: 21012 ms Total
input records: 0 Total output records: 0 timestamp:
2025-11-20T11:12:58.596487112Z inputTotal: 0 outputTotal: 0 parameters:
{source={}}}{{{}s3://<S3-BUCKET>/data/{}}}{{{}}{}}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
