Maxim Gekk created SPARK-30599:
----------------------------------
Summary: SparkFunSuite.LogAppender throws
java.lang.IllegalStateException
Key: SPARK-30599
URL: https://issues.apache.org/jira/browse/SPARK-30599
Project: Spark
Issue Type: Test
Components: Spark Core, SQL, Tests
Affects Versions: 3.0.0
Reporter: Maxim Gekk
Attachments: success_trace.txt
The LogAppender has a limit for the number of logged event. By default, it is
100. All tests that use LogAppender finish successfully with this limit but
sometime the test fail with the exception like:
{code:java}
java.lang.IllegalStateException: Number of events reached the limit of 100
while logging CSV header matches to schema w/ enforceSchema.
at
org.apache.spark.SparkFunSuite$LogAppender.append(SparkFunSuite.scala:200)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at
org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
{code}
For example, the CSV test "SPARK-23786: warning should be printed if CSV header
doesn't conform to schema" uses 2 log appenders.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
