xiehanghang commented on issue #4129:
URL:
https://github.com/apache/incubator-seatunnel/issues/4129#issuecomment-1431736609
> Change the job.mode from `STEAMING` to `BATCH`
After modifying the job.mode I got another error about package dependency.
I want to know which jar packages I need to get the S3 source.
this is the jar package I put in the lib directory
```
aws-java-sdk-bundle-1.12.262.jar hadoop-aws-2.10.2.jar
hadoop-shaded-guava-1.1.1.jar
aws-java-sdk-core-1.11.1000.jar hadoop-client-2.10.2.jar
aws-java-sdk-s3-1.11.1000.jar hadoop-common-2.10.2.jar
seatunnel-hadoop3-3.1.4-uber-2.3.0-optional.jar
```
```
Exception in thread "main" java.lang.NoSuchMethodError:
com.amazonaws.http.HttpResponse.getHttpRequest()Lorg/apache/http/client/methods/HttpRequestBase;
at
com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:57)
at
com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:29)
at
com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:69)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1794)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleSuccessResponse(AmazonHttpClient.java:1477)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1384)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5445)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5392)
at
com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1519)
at
org.apache.hadoop.fs.s3a.S3AInputStream.reopen(S3AInputStream.java:169)
at
org.apache.hadoop.fs.s3a.S3AInputStream.lazySeek(S3AInputStream.java:305)
at org.apache.hadoop.fs.s3a.S3AInputStream.read(S3AInputStream.java:330)
at java.io.FilterInputStream.read(FilterInputStream.java:83)
at
org.apache.seatunnel.shade.connector.file.org.apache.parquet.io.DelegatingSeekableInputStream.read(DelegatingSeekableInputStream.java:61)
at
org.apache.seatunnel.shade.connector.file.org.apache.parquet.bytes.BytesUtils.readIntLittleEndian(BytesUtils.java:83)
at
org.apache.seatunnel.shade.connector.file.org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:548)
at
org.apache.seatunnel.shade.connector.file.org.apache.parquet.hadoop.ParquetFileReader.<init>(ParquetFileReader.java:777)
at
org.apache.seatunnel.shade.connector.file.org.apache.parquet.hadoop.ParquetFileReader.open(ParquetFileReader.java:646)
at
org.apache.seatunnel.connectors.seatunnel.file.source.reader.ParquetReadStrategy.getSeaTunnelRowTypeInfo(ParquetReadStrategy.java:209)
at
org.apache.seatunnel.connectors.seatunnel.file.s3.source.S3FileSource.prepare(S3FileSource.java:96)
at
org.apache.seatunnel.engine.core.parse.ConnectorInstanceLoader.loadSourceInstance(ConnectorInstanceLoader.java:60)
at
org.apache.seatunnel.engine.core.parse.JobConfigParser.sampleAnalyze(JobConfigParser.java:314)
at
org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:125)
at
org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:129)
at
org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:121)
at
org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:91)
at org.apache.seatunnel.core.starter.Seatunnel.run(Seatunnel.java:39)
at
org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:31)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
