cchighman removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-664700136
Interesting, they seem to pass in the logs in most places. It looks like something about this test is making it flaky but I'm not sure off-hand what it could be since the date is so far into the future. Going to keep looking. **Passsing** ``` 2020-07-27T08:42:10.6518723Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified with a past date (156 milliseconds)[0m[0m 2020-07-27T08:42:10.7814288Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified with a past date and pathGlobalFilter returning results (132 milliseconds)[0m[0m 2020-07-27T08:42:10.7895191Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified with past date and pathGlobFilter filtering results (8 milliseconds)[0m[0m 2020-07-27T08:42:10.7967338Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified after future date and pathGlobFilter returning results (7 milliseconds)[0m[0m 2020-07-27T08:42:10.8050729Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified with future date and pathGlobFilter filtering results (6 milliseconds)[0m[0m 2020-07-27T08:42:10.8085823Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedBefore and modifiedAfterare specified out of range and pathGlobFilter returning results (6 milliseconds)[0m[0m 2020-07-27T08:42:10.9358582Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedBefore and modifiedAfterare specified in range and pathGlobFilter returning results (127 milliseconds)[0m[0m 2020-07-27T08:42:10.9428714Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedBefore and modifiedAfterare specified in range and pathGlobFilter filtering results (7 milliseconds)[0m[0m 2020-07-27T08:42:10.9559416Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter is specified with invalid date (5 milliseconds)[0m[0m 2020-07-27T08:42:10.9597076Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - PathFilterStrategies - modifiedAfter option (1 millisecond)[0m[0m 2020-07-27T08:42:10.9609838Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - PathFilterStrategies - modifiedBefore option (0 milliseconds)[0m[0m 2020-07-27T08:42:10.9619697Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - PathFilterStrategies - pathGlobFilter option (0 milliseconds)[0m[0m 2020-07-27T08:42:10.9634071Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - PathFilterStrategies - multiple options (1 millisecond)[0m[0m 2020-07-27T08:42:10.9642494Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - PathFilterStrategies - no options (0 milliseconds)[0m[0m 2020-07-27T08:42:10.9704999Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedBefore - empty option (6 milliseconds)[0m[0m 2020-07-27T08:42:10.9780622Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedAfter - empty option (6 milliseconds)[0m[0m 2020-07-27T08:42:10.9809164Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedAfter filter takes into account local timezone when specified as an option. After UTC. (3 milliseconds)[0m[0m 2020-07-27T08:42:10.9830762Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedAfter filter takes into account local timezone when specified as an option. Before UTC. (3 milliseconds)[0m[0m 2020-07-27T08:42:10.9860404Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedBefore filter takes into account local timezone when specified as an option. After UTC. (2 milliseconds)[0m[0m 2020-07-27T08:42:10.9896541Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedBefore filter takes into account local timezone when specified as an option. Before UTC. (3 milliseconds)[0m[0m 2020-07-27T08:42:11.3068941Z [0m[[0minfo[0m] [0m[32m- Option pathGlobFilter: filter files correctly (317 milliseconds)[0m[0m 2020-07-27T08:42:11.7598903Z [0m[[0minfo[0m] [0m[32m- Option pathGlobFilter: simple extension filtering should contains partition info (452 milliseconds)[0m[0m 2020-07-27T08:42:11.8681609Z [0m[[0minfo[0m] [0m[32m- Option recursiveFileLookup: recursive loading correctly (108 milliseconds)[0m[0m 2020-07-27T08:42:11.9066082Z [0m[[0minfo[0m] [0m[32m- Option recursiveFileLookup: disable partition inferring (39 milliseconds)[0m[0m ``` **Failing** ``` 2020-07-27T08:42:10.2387647Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified before future date (133 milliseconds)[0m[0m 2020-07-27T08:42:10.3701490Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedBefore specified with a future date (131 milliseconds)[0m[0m 2020-07-27T08:42:10.3870283Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedBefore specified with a past date (6 milliseconds)[0m[0m 2020-07-27T08:42:10.4830967Z [0m[[0minfo[0m] [0m[31m- SPARK-31962 - when modifiedAfter specified with a future date *** FAILED *** (91 milliseconds)[0m[0m 2020-07-27T08:42:10.4832099Z [0m[[0minfo[0m] [0m[31m Expected exception org.apache.spark.sql.AnalysisException to be thrown, but no exception was thrown (FileBasedDataSourceSuite.scala:620)[0m[0m 2020-07-27T08:42:10.4832863Z [0m[[0minfo[0m] [0m[31m org.scalatest.exceptions.TestFailedException:[0m[0m 2020-07-27T08:42:10.4833467Z [0m[[0minfo[0m] [0m[31m at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)[0m[0m 2020-07-27T08:42:10.4834013Z [0m[[0minfo[0m] [0m[31m at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)[0m[0m 2020-07-27T08:42:10.4834733Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1562)[0m[0m 2020-07-27T08:42:10.4835258Z [0m[[0minfo[0m] [0m[31m at org.scalatest.Assertions.intercept(Assertions.scala:766)[0m[0m 2020-07-27T08:42:10.4835922Z [0m[[0minfo[0m] [0m[31m at org.scalatest.Assertions.intercept$(Assertions.scala:746)[0m[0m 2020-07-27T08:42:10.4836494Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuite.intercept(AnyFunSuite.scala:1562)[0m[0m 2020-07-27T08:42:10.4837153Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.sql.FileBasedDataSourceSuite.$anonfun$new$126(FileBasedDataSourceSuite.scala:620)[0m[0m 2020-07-27T08:42:10.4837805Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.sql.FileBasedDataSourceSuite.$anonfun$new$126$adapted(FileBasedDataSourceSuite.scala:615)[0m[0m 2020-07-27T08:42:10.4838769Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.sql.test.SQLTestUtils.$anonfun$withTempDir$1(SQLTestUtils.scala:78)[0m[0m 2020-07-27T08:42:10.4839910Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.sql.test.SQLTestUtils.$anonfun$withTempDir$1$adapted(SQLTestUtils.scala:77)[0m[0m 2020-07-27T08:42:10.4840959Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.SparkFunSuite.withTempDir(SparkFunSuite.scala:170)[0m[0m 2020-07-27T08:42:10.4871149Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.sql.FileBasedDataSourceSuite.org$apache$spark$sql$test$SQLTestUtils$$super$withTempDir(FileBasedDataSourceSuite.scala:50)[0m[0m 2020-07-27T08:42:10.4872261Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.sql.test.SQLTestUtils.withTempDir(SQLTestUtils.scala:77)[0m[0m 2020-07-27T08:42:10.4872652Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.sql.test.SQLTestUtils.withTempDir$(SQLTestUtils.scala:76)[0m[0m 2020-07-27T08:42:10.4873295Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.sql.FileBasedDataSourceSuite.withTempDir(FileBasedDataSourceSuite.scala:50)[0m[0m 2020-07-27T08:42:10.4873727Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.sql.FileBasedDataSourceSuite.$anonfun$new$125(FileBasedDataSourceSuite.scala:615)[0m[0m 2020-07-27T08:42:10.4874112Z [0m[[0minfo[0m] [0m[31m at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)[0m[0m 2020-07-27T08:42:10.4874630Z [0m[[0minfo[0m] [0m[31m at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)[0m[0m 2020-07-27T08:42:10.4875047Z [0m[[0minfo[0m] [0m[31m at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)[0m[0m 2020-07-27T08:42:10.4875562Z [0m[[0minfo[0m] [0m[31m at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)[0m[0m 2020-07-27T08:42:10.4875930Z [0m[[0minfo[0m] [0m[31m at org.scalatest.Transformer.apply(Transformer.scala:22)[0m[0m 2020-07-27T08:42:10.4876275Z [0m[[0minfo[0m] [0m[31m at org.scalatest.Transformer.apply(Transformer.scala:20)[0m[0m 2020-07-27T08:42:10.4876645Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:189)[0m[0m 2020-07-27T08:42:10.4877004Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:158)[0m[0m 2020-07-27T08:42:10.4877382Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:187)[0m[0m 2020-07-27T08:42:10.4877782Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:199)[0m[0m 2020-07-27T08:42:10.4878135Z [0m[[0minfo[0m] [0m[31m at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)[0m[0m 2020-07-27T08:42:10.4878655Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:199)[0m[0m 2020-07-27T08:42:10.4879194Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:181)[0m[0m 2020-07-27T08:42:10.4879605Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:60)[0m[0m 2020-07-27T08:42:10.4880003Z [0m[[0minfo[0m] [0m[31m at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)[0m[0m 2020-07-27T08:42:10.4880379Z [0m[[0minfo[0m] [0m[31m at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)[0m[0m 2020-07-27T08:42:10.4880779Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:60)[0m[0m 2020-07-27T08:42:10.4881171Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:232)[0m[0m 2020-07-27T08:42:10.4881661Z [0m[[0minfo[0m] [0m[31m at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)[0m[0m 2020-07-27T08:42:10.4882420Z [0m[[0minfo[0m] [0m[31m at scala.collection.immutable.List.foreach(List.scala:392)[0m[0m 2020-07-27T08:42:10.4882988Z [0m[[0minfo[0m] [0m[31m at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)[0m[0m 2020-07-27T08:42:10.4883818Z [0m[[0minfo[0m] [0m[31m at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)[0m[0m 2020-07-27T08:42:10.4884606Z [0m[[0minfo[0m] [0m[31m at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)[0m[0m 2020-07-27T08:42:10.4885206Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:232)[0m[0m 2020-07-27T08:42:10.4885824Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:231)[0m[0m 2020-07-27T08:42:10.4886424Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1562)[0m[0m 2020-07-27T08:42:10.4887017Z [0m[[0minfo[0m] [0m[31m at org.scalatest.Suite.run(Suite.scala:1112)[0m[0m 2020-07-27T08:42:10.4888379Z [0m[[0minfo[0m] [0m[31m at org.scalatest.Suite.run$(Suite.scala:1094)[0m[0m 2020-07-27T08:42:10.4888984Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1562)[0m[0m 2020-07-27T08:42:10.4889608Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:236)[0m[0m 2020-07-27T08:42:10.4890176Z [0m[[0minfo[0m] [0m[31m at org.scalatest.SuperEngine.runImpl(Engine.scala:535)[0m[0m 2020-07-27T08:42:10.4890836Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:236)[0m[0m 2020-07-27T08:42:10.4891454Z [0m[[0minfo[0m] [0m[31m at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:235)[0m[0m 2020-07-27T08:42:10.4892327Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:60)[0m[0m 2020-07-27T08:42:10.4893315Z [0m[[0minfo[0m] [0m[31m at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)[0m[0m 2020-07-27T08:42:10.4894053Z [0m[[0minfo[0m] [0m[31m at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)[0m[0m 2020-07-27T08:42:10.4894649Z [0m[[0minfo[0m] [0m[31m at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)[0m[0m 2020-07-27T08:42:10.4895635Z [0m[[0minfo[0m] [0m[31m at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:60)[0m[0m 2020-07-27T08:42:10.4896403Z [0m[[0minfo[0m] [0m[31m at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)[0m[0m 2020-07-27T08:42:10.4897877Z [0m[[0minfo[0m] [0m[31m at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)[0m[0m 2020-07-27T08:42:10.4899022Z [0m[[0minfo[0m] [0m[31m at sbt.ForkMain$Run$2.call(ForkMain.java:296)[0m[0m 2020-07-27T08:42:10.4900074Z [0m[[0minfo[0m] [0m[31m at sbt.ForkMain$Run$2.call(ForkMain.java:286)[0m[0m 2020-07-27T08:42:10.4901231Z [0m[[0minfo[0m] [0m[31m at java.util.concurrent.FutureTask.run(FutureTask.java:266)[0m[0m 2020-07-27T08:42:10.4902402Z [0m[[0minfo[0m] [0m[31m at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[0m[0m 2020-07-27T08:42:10.4903548Z [0m[[0minfo[0m] [0m[31m at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[0m[0m 2020-07-27T08:42:10.4904632Z [0m[[0minfo[0m] [0m[31m at java.lang.Thread.run(Thread.java:748)[0m[0m 2020-07-27T08:42:10.6518723Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified with a past date (156 milliseconds)[0m[0m 2020-07-27T08:42:10.7814288Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified with a past date and pathGlobalFilter returning results (132 milliseconds)[0m[0m 2020-07-27T08:42:10.7895191Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified with past date and pathGlobFilter filtering results (8 milliseconds)[0m[0m 2020-07-27T08:42:10.7967338Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified after future date and pathGlobFilter returning results (7 milliseconds)[0m[0m 2020-07-27T08:42:10.8050729Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter specified with future date and pathGlobFilter filtering results (6 milliseconds)[0m[0m 2020-07-27T08:42:10.8085823Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedBefore and modifiedAfterare specified out of range and pathGlobFilter returning results (6 milliseconds)[0m[0m 2020-07-27T08:42:10.9358582Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedBefore and modifiedAfterare specified in range and pathGlobFilter returning results (127 milliseconds)[0m[0m 2020-07-27T08:42:10.9428714Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedBefore and modifiedAfterare specified in range and pathGlobFilter filtering results (7 milliseconds)[0m[0m 2020-07-27T08:42:10.9559416Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - when modifiedAfter is specified with invalid date (5 milliseconds)[0m[0m 2020-07-27T08:42:10.9597076Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - PathFilterStrategies - modifiedAfter option (1 millisecond)[0m[0m 2020-07-27T08:42:10.9609838Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - PathFilterStrategies - modifiedBefore option (0 milliseconds)[0m[0m 2020-07-27T08:42:10.9619697Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - PathFilterStrategies - pathGlobFilter option (0 milliseconds)[0m[0m 2020-07-27T08:42:10.9634071Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - PathFilterStrategies - multiple options (1 millisecond)[0m[0m 2020-07-27T08:42:10.9642494Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - PathFilterStrategies - no options (0 milliseconds)[0m[0m 2020-07-27T08:42:10.9704999Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedBefore - empty option (6 milliseconds)[0m[0m 2020-07-27T08:42:10.9780622Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedAfter - empty option (6 milliseconds)[0m[0m 2020-07-27T08:42:10.9809164Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedAfter filter takes into account local timezone when specified as an option. After UTC. (3 milliseconds)[0m[0m 2020-07-27T08:42:10.9830762Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedAfter filter takes into account local timezone when specified as an option. Before UTC. (3 milliseconds)[0m[0m 2020-07-27T08:42:10.9860404Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedBefore filter takes into account local timezone when specified as an option. After UTC. (2 milliseconds)[0m[0m 2020-07-27T08:42:10.9896541Z [0m[[0minfo[0m] [0m[32m- SPARK-31962 - modifiedBefore filter takes into account local timezone when specified as an option. Before UTC. (3 milliseconds)[0m[0m ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
