[
https://issues.apache.org/jira/browse/BAHIR-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Dixit closed BAHIR-238.
--------------------------------
Fix Version/s: Spark-2.4.0
Resolution: Duplicate
Raised another issue which has link to PR for this
> Incomplete Error Message on Failure to Create AmazonSQS Client
> --------------------------------------------------------------
>
> Key: BAHIR-238
> URL: https://issues.apache.org/jira/browse/BAHIR-238
> Project: Bahir
> Issue Type: Bug
> Components: Spark Structured Streaming Connectors
> Reporter: Abhishek Dixit
> Assignee: Abhishek Dixit
> Priority: Major
> Fix For: Spark-2.4.0
>
>
> Users have reported incomplete error message with null on failure to create
> AmazonSQSClient.
> {code:java}
> org.apache.spark.SparkException: Error occured while creating Amazon SQS
> Client null
> at
> org.apache.spark.sql.streaming.sqs.SqsClient.createSqsClient(SqsClient.scala:227)
> at org.apache.spark.sql.streaming.sqs.SqsClient.(SqsClient.scala:54)
> at org.apache.spark.sql.streaming.sqs.SqsSource.(SqsSource.scala:53)
> at
> org.apache.spark.sql.streaming.sqs.SqsSourceProvider.createSource(SqsSourceProvider.scala:47)
> at
> org.apache.spark.sql.execution.datasources.DataSource.createSource(DataSource.scala:255)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution$$anonfun$1$$anonfun$applyOrElse$1.apply(MicroBatchExecution.scala:88)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution$$anonfun$1$$anonfun$applyOrElse$1.apply(MicroBatchExecution.scala:85)
> at scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:79)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution$$anonfun$1.applyOrElse(MicroBatchExecution.scala:85)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution$$anonfun$1.applyOrElse(MicroBatchExecution.scala:83)
> at
> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$2.apply(TreeNode.scala:286)
> at
> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$2.apply(TreeNode.scala:286)
> at
> org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:71)
> at
> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:285)
> at
> org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDown(LogicalPlan.scala:29)
> at
> org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$class.transformDown(AnalysisHelper.scala:149)
> at
> org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDown(LogicalPlan.scala:29)
> at
> org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDown(LogicalPlan.scala:29)
> at org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:275)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.logicalPlan$lzycompute(MicroBatchExecution.scala:83)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.logicalPlan(MicroBatchExecution.scala:65)
> at
> org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.scala:269)
> at
> org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:193)
> {code}
> I'll create a PR for this fix.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
