wenwj0 opened a new issue, #9556:
URL: https://github.com/apache/incubator-gluten/issues/9556
### Backend
VL (Velox)
### Bug description
Running gluten-1.4.0 on debian 10.
Got error:
```
org.apache.gluten.exception.GlutenException: corrupt tzdb: expected
character '#', got 'ÿ' instead
at
org.apache.gluten.vectorized.PlanEvaluatorJniWrapper.nativeValidateWithFailureReason(Native
Method)
at
org.apache.gluten.vectorized.NativePlanEvaluator.doNativeValidateWithFailureReason(NativePlanEvaluator.java:51)
at
org.apache.gluten.backendsapi.velox.VeloxValidatorApi.$anonfun$doNativeValidateWithFailureReason$1(VeloxValidatorApi.scala:42)
at
org.apache.spark.task.TaskResources$.runUnsafe(TaskResources.scala:99)
at
org.apache.gluten.backendsapi.velox.VeloxValidatorApi.doNativeValidateWithFailureReason(VeloxValidatorApi.scala:40)
at
org.apache.gluten.execution.TransformSupport.doNativeValidation(WholeStageTransformer.scala:169)
at
org.apache.gluten.execution.TransformSupport.doNativeValidation$(WholeStageTransformer.scala:165)
at
org.apache.gluten.execution.ProjectExecTransformerBase.doNativeValidation(BasicPhysicalOperatorTransformer.scala:171)
at
org.apache.gluten.execution.ProjectExecTransformerBase.$anonfun$doValidateInternal$1(BasicPhysicalOperatorTransformer.scala:190)
at
org.apache.gluten.execution.ValidatablePlan.failValidationWithException(WholeStageTransformer.scala:76)
at
org.apache.gluten.execution.ValidatablePlan.failValidationWithException$(WholeStageTransformer.scala:73)
at
org.apache.gluten.execution.ProjectExecTransformerBase.failValidationWithException(BasicPhysicalOperatorTransformer.scala:171)
at
org.apache.gluten.execution.ProjectExecTransformerBase.doValidateInternal(BasicPhysicalOperatorTransformer.scala:186)
at
org.apache.gluten.execution.ValidatablePlan.$anonfun$doValidate$4(WholeStageTransformer.scala:114)
at
org.apache.gluten.execution.ValidatablePlan.failValidationWithException(WholeStageTransformer.scala:76)
at
org.apache.gluten.execution.ValidatablePlan.failValidationWithException$(WholeStageTransformer.scala:73)
at
org.apache.gluten.execution.ProjectExecTransformerBase.failValidationWithException(BasicPhysicalOperatorTransformer.scala:171)
at
org.apache.gluten.execution.ValidatablePlan.doValidate(WholeStageTransformer.scala:116)
at
org.apache.gluten.execution.ValidatablePlan.doValidate$(WholeStageTransformer.scala:97)
at
org.apache.gluten.execution.ProjectExecTransformerBase.doValidate(BasicPhysicalOperatorTransformer.scala:171)
at
org.apache.gluten.extension.columnar.validator.Validators$FallbackByNativeValidation.validate(Validators.scala:235)
at
org.apache.gluten.extension.columnar.validator.Validator$Builder$ValidatorPipeline.$anonfun$validate$1(Validator.scala:90)
at
scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
at
scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
at scala.collection.immutable.List.foldLeft(List.scala:91)
at
org.apache.gluten.extension.columnar.validator.Validator$Builder$ValidatorPipeline.validate(Validator.scala:87)
at
org.apache.gluten.extension.columnar.heuristic.AddFallbackTags.addFallbackTag(AddFallbackTags.scala:36)
at
org.apache.gluten.extension.columnar.heuristic.AddFallbackTags.$anonfun$apply$1(AddFallbackTags.scala:29)
at
org.apache.gluten.extension.columnar.heuristic.AddFallbackTags.$anonfun$apply$1$adapted(AddFallbackTags.scala:28)
at
org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:263)
at
org.apache.gluten.extension.columnar.heuristic.AddFallbackTags.apply(AddFallbackTags.scala:28)
at
org.apache.gluten.extension.columnar.heuristic.AddFallbackTags.apply(AddFallbackTags.scala:26)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicTransform$WithRewrites.$anonfun$apply$3(HeuristicTransform.scala:112)
at
scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
at
scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
at scala.collection.immutable.List.foldLeft(List.scala:91)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicTransform$WithRewrites.apply(HeuristicTransform.scala:110)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicTransform$WithRewrites.apply(HeuristicTransform.scala:100)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicTransform.$anonfun$apply$1(HeuristicTransform.scala:58)
at
scala.collection.IndexedSeqOptimized.foldLeft(IndexedSeqOptimized.scala:60)
at
scala.collection.IndexedSeqOptimized.foldLeft$(IndexedSeqOptimized.scala:68)
at scala.collection.mutable.ArrayBuffer.foldLeft(ArrayBuffer.scala:49)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicTransform.apply(HeuristicTransform.scala:56)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicTransform.apply(HeuristicTransform.scala:52)
at
org.apache.gluten.extension.columnar.ColumnarRuleExecutor$LoggedRule.$anonfun$apply$1(ColumnarRuleExecutor.scala:59)
at
org.apache.gluten.metrics.GlutenTimeMetric$.withNanoTime(GlutenTimeMetric.scala:41)
at
org.apache.gluten.metrics.GlutenTimeMetric$.withMillisTime(GlutenTimeMetric.scala:46)
at
org.apache.gluten.metrics.GlutenTimeMetric$.recordMillisTime(GlutenTimeMetric.scala:50)
at
org.apache.gluten.extension.columnar.ColumnarRuleExecutor$LoggedRule.apply(ColumnarRuleExecutor.scala:59)
at
org.apache.gluten.extension.columnar.ColumnarRuleExecutor$LoggedRule.apply(ColumnarRuleExecutor.scala:41)
at
org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$2(RuleExecutor.scala:211)
at
scala.collection.IndexedSeqOptimized.foldLeft(IndexedSeqOptimized.scala:60)
at
scala.collection.IndexedSeqOptimized.foldLeft$(IndexedSeqOptimized.scala:68)
at scala.collection.mutable.ArrayBuffer.foldLeft(ArrayBuffer.scala:49)
at
org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$1(RuleExecutor.scala:208)
at
org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$1$adapted(RuleExecutor.scala:200)
at scala.collection.immutable.List.foreach(List.scala:431)
at
org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:200)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicApplier.transformPlan(HeuristicApplier.scala:69)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicApplier.org$apache$gluten$extension$columnar$heuristic$HeuristicApplier$$$anonfun$makeRule$1(HeuristicApplier.scala:49)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicApplier$$anonfun$makeRule$3.apply(HeuristicApplier.scala:48)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicApplier$$anonfun$makeRule$3.apply(HeuristicApplier.scala:48)
at
org.apache.gluten.extension.columnar.heuristic.HeuristicApplier.apply(HeuristicApplier.scala:44)
at
org.apache.gluten.extension.GlutenColumnarRule.org$apache$gluten$extension$GlutenColumnarRule$$$anonfun$postColumnarTransitions$1(GlutenColumnarRule.scala:102)
at
org.apache.gluten.extension.GlutenColumnarRule$$anonfun$postColumnarTransitions$2.apply(GlutenColumnarRule.scala:85)
at
org.apache.gluten.extension.GlutenColumnarRule$$anonfun$postColumnarTransitions$2.apply(GlutenColumnarRule.scala:85)
at
org.apache.gluten.extension.injector.InjectorControl$Disabler$DisablerOps$$anon$5$$anon$7.apply(InjectorControl.scala:149)
at
org.apache.gluten.extension.injector.InjectorControl$Disabler$DisablerOps$$anon$5$$anon$7.apply(InjectorControl.scala:144)
at
org.apache.spark.sql.execution.ApplyColumnarRulesAndInsertTransitions.$anonfun$apply$2(Columnar.scala:548)
at
org.apache.spark.sql.execution.ApplyColumnarRulesAndInsertTransitions.$anonfun$apply$2$adapted(Columnar.scala:547)
at
scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at
scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at
org.apache.spark.sql.execution.ApplyColumnarRulesAndInsertTransitions.apply(Columnar.scala:547)
at
org.apache.spark.sql.execution.ApplyColumnarRulesAndInsertTransitions.apply(Columnar.scala:505)
at
org.apache.spark.sql.execution.QueryExecution$.$anonfun$prepareForExecution$1(QueryExecution.scala:434)
at
scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
at
scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
at scala.collection.immutable.List.foldLeft(List.scala:91)
at
org.apache.spark.sql.execution.QueryExecution$.prepareForExecution(QueryExecution.scala:433)
at
org.apache.spark.sql.execution.QueryExecution.$anonfun$executedPlan$1(QueryExecution.scala:157)
at
org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
at
org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:183)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
at
org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:183)
at
org.apache.spark.sql.execution.QueryExecution.executedPlan$lzycompute(QueryExecution.scala:157)
at
org.apache.spark.sql.execution.QueryExecution.executedPlan(QueryExecution.scala:150)
at
org.apache.spark.sql.execution.QueryExecution.simpleString(QueryExecution.scala:201)
at
org.apache.spark.sql.execution.QueryExecution.org$apache$spark$sql$execution$QueryExecution$$explainString(QueryExecution.scala:246)
at
org.apache.spark.sql.execution.QueryExecution.explainString(QueryExecution.scala:215)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:102)
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:183)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:94)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:68)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3704)
at org.apache.spark.sql.Dataset.isEmpty(Dataset.scala:603)
at
org.apache.kyuubi.engine.spark.KyuubiSparkUtil$.$anonfun$initializeSparkSession$1(KyuubiSparkUtil.scala:53)
at
org.apache.kyuubi.engine.spark.KyuubiSparkUtil$.$anonfun$initializeSparkSession$1$adapted(KyuubiSparkUtil.scala:46)
at
scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at
scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at
org.apache.kyuubi.engine.spark.KyuubiSparkUtil$.initializeSparkSession(KyuubiSparkUtil.scala:46)
at
org.apache.kyuubi.engine.spark.SparkSQLEngine$.createSpark(SparkSQLEngine.scala:317)
at
org.apache.kyuubi.engine.spark.SparkSQLEngine$.main(SparkSQLEngine.scala:422)
at
org.apache.kyuubi.engine.spark.SparkSQLEngine.main(SparkSQLEngine.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:737)
```
It seems that there is no `tzdata.zi` file in `/usr/share/zoneinfo/`
Related code:
http://github.com/facebookincubator/velox/blob/main/velox/external/tzdb/tzdb.cpp#L837
### Gluten version
Gluten-1.4
### Spark version
Spark-3.2.x
### Spark configurations
_No response_
### System information
OS: debian10.
Linux 22b50d614525 4.9.0-14-amd64 #1 SMP Debian 4.9.246-2 (2020-12-17)
x86_64 GNU/Linux
### Relevant logs
```bash
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
