TomXiaoYZ opened a new issue, #3502:
URL: https://github.com/apache/incubator-seatunnel/issues/3502

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22)
 and found no similar issues.
   
   
   ### What happened
   
   I've tried the new seatunnel S3File source connector. 
   But it shows the following error. I can confirm the path exists.
   
   ### SeaTunnel Version
   
   2.3.0-beta
   
   ### SeaTunnel Config
   
   ```conf
   env {
       # see available properties defined by spark: 
https://spark.apache.org/docs/latest/configuration.html#available-properties
       spark.app.name = "SeaTunnel S3 DA Synchronization To Hive"
       spark.executor.cores = 5
       spark.executor.memory = "5g"
       spark.dynamicAllocation.minExecutors = "1"
       spark.dynamicAllocation.maxExecutors = "5"
       spark.sql.catalogImplementation = "hive"
   }
   
   source {
       S3File {
           path = "/hive/da/da_coin_px/"${partition_end_date}"/*"
           access_key = ""${username}""
           secret_key = ""${password}""
           bucket = ""${url}""
           type = "parquet"
           result_table_name = "da_coin_px"
       }
   }
   
   transform {
   }
   
   sink {
       Hive {
           source_table_name = "da_coin_px"
           result_table_name = "risk_data_superset_temp.ods_s3_da_coin_px_df"
           save_mode = "overwrite"
       }
   }
   ```
   
   
   ### Running Command
   
   ```shell
   sh ./bin/start-seatunnel-spark-connector-v2.sh \
     --config "$1" \
     --master yarn \
     --deploy-mode cluster \
     -i url="$2" \
     -i username="$3" \
     -i password="$4" \
     -i partition_start_date="$5" \
     -i partition_end_date="$6"
   ```
   
   
   ### Error Exception
   
   ```log
   org.apache.seatunnel.api.common.PrepareFailException: PluginName: S3File, 
PluginType: source, Message: Check file path fail.
   ```
   
   
   ### Flink or Spark Version
   
   I've tried both spark2 & 3
   
   ### Java or Scala Version
   
   1.8.0
   
   ### Screenshots
   
   
![企业微信截图_20221122112532](https://user-images.githubusercontent.com/38311335/203213934-0d39575e-8889-4f7a-b573-8804c6870f87.png)
   
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to