[jira] [Created] (SPARK-39633) Dataframe options for time travel via `timestampAsOf` should respect both formats of specifying timestamp

[Prashant Singh (Jira)]

Title: Message Title

Prashant Singh created an issue   Spark / SPARK-39633 Dataframe options for time travel via `timestampAsOf` should respect both formats of specifying timestamp Issue Type: Bug Affects Versions: 3.3.0 Assignee: Unassigned Components: SQL Created: 29/Jun/22 14:14 Priority: Minor Reporter: Prashant Singh presently spark sql query for time travel like : {{SELECT * from {table} TIMESTAMP AS OF 1548751078 }} works correctly, which is what is specified in sql grammar as well (((FOR SYSTEM_VERSION) | VERSION) AS OF version=(INTEGER_VALUE | STRING)),  but when trying to do the same via dataframe option `timestampAsOf` the code fails with : [info]   org.apache.spark.sql.AnalysisException: '1548751078' is not a valid timestamp _expression_ for time travel. [info]   at org.apache.spark.sql.errors.QueryCompilationErrors$.invalidTimestampExprForTimeTravel(QueryCompilationErrors.scala:2413) [info]   at org.apache.spark.sql.catalyst.analysis.TimeTravelSpec$.create(TimeTravelSpec.scala:55) [info]   at org.apache.spark.sql.execution.datasources.v2.DataSourceV2Utils$.loadV2Source(DataSourceV2Utils.scala:128) [info]   at org.apache.spark.sql.DataFrameReader.$anonfun$load$1(DataFrameReader.scala:209) [info]   at scala.Option.flatMap(Option.scala:271) [info]   at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:207) [info]   at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:171) [info]   at org.apache.spark.sql.connector.SupportsCatalogOptionsSuite.load(SupportsCatalogOptionsSuite.scala:365) [info]   at org.apache.spark.sql.connector.SupportsCatalogOptionsSuite.$anonfun$new$33(SupportsCatalogOptionsSuite.scala:329) [info]   at org.apache.spark.sql.QueryTest.checkAnswer(QueryTest.scala:133) [info]   at org.apache.spark.sql.QueryTest.checkAnswer(QueryTest.scala:158) [info]   at org.apache.spark.sql.connector.SupportsCatalogOptionsSuite.$anonfun$new$30(SupportsCatalogOptionsSuite.scala:329) [info]   at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) [info]   at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1490) [info]   at org.apache.spark.sql.test.SQLTestUtilsBase.withTable(SQLTestUtils.scala:306) [info]   at org.apache.spark.sql.test.SQLTestUtilsBase.withTable$(SQLTestUtils.scala:304) [info]   at org.apache.spark.sql.connector.SupportsCatalogOptionsSuite.withTable(SupportsCatalogOptionsSuite.scala:44) [info]   at org.apache.spark.sql.connector.SupportsCatalogOptionsSuite.$anonfun$new$26(SupportsCatalogOptionsSuite.scala:309) [info]   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) [info]   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) [info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) [info]   at org.scalatest.Transformer.apply(Transformer.scala:22) [info]   at org.scalatest.Transformer.apply(Transformer.scala:20) [info]   at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:190) [info]   at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:203) [info]   at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:188) [info]   at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:200) [info]   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306) [info]   at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:200) [info]   at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:182) Add Comment   This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)