itziklavon opened a new issue #5835:
URL: https://github.com/apache/incubator-pinot/issues/5835
having problem syncing data from s3
```
Caused by: java.lang.IllegalArgumentException: Parameter 'Bucket' must not
be null
at
software.amazon.awssdk.protocols.xml.internal.marshall.SimpleTypePathMarshaller.lambda$static$0(SimpleTypePathMarshaller.java:43)
~[pinot-s3-0.5.0-SNAPSHOT-shaded.jar:0.5.0-SNAPSHOT-1d4d47adfe7abf0c3ed8a3a14929de084e979968]
at
software.amazon.awssdk.protocols.xml.internal.marshall.XmlProtocolMarshaller.doMarshall(XmlProtocolMarshaller.java:104)
~[pinot-s3-0.5.0-SNAPSHOT-shaded.jar:0.5.0-SNAPSHOT-1d4d47adfe7abf0c3ed8a3a14929de084e979968]
at
software.amazon.awssdk.protocols.xml.internal.marshall.XmlProtocolMarshaller.marshall(XmlProtocolMarshaller.java:80)
~[pinot-s3-0.5.0-SNAPSHOT-shaded.jar:0.5.0-SNAPSHOT-1d4d47adfe7abf0c3ed8a3a14929de084e979968]
at
software.amazon.awssdk.protocols.xml.internal.marshall.XmlProtocolMarshaller.marshall(XmlProtocolMarshaller.java:49)
~[pinot-s3-0.5.0-SNAPSHOT-shaded.jar:0.5.0-SNAPSHOT-1d4d47adfe7abf0c3ed8a3a14929de084e979968]
at
software.amazon.awssdk.services.s3.transform.ListObjectsV2RequestMarshaller.marshall(ListObjectsV2RequestMarshaller.java:51)
~[pinot-s3-0.5.0-SNAPSHOT-shaded.jar:0.5.0-SNAPSHOT-1d4d47adfe7abf0c3ed8a3a14929de084e979968]
at
software.amazon.awssdk.services.s3.transform.ListObjectsV2RequestMarshaller.marshall(ListObjectsV2RequestMarshaller.java:31)
~[pinot-s3-0.5.0-SNAPSHOT-shaded.jar:0.5.0-SNAPSHOT-1d4d47adfe7abf0c3ed8a3a14929de084e979968]
at
software.amazon.awssdk.core.internal.handler.BaseClientHandler.finalizeSdkHttpFullRequest(BaseClientHandler.java:68)
~[pinot-s3-0.5.0-SNAPSHOT-shaded.jar:0.5.0-SNAPSHOT-1d4d47adfe7abf0c3ed8a3a14929de084e979968]
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:131)
~[pinot-s3-0.5.0-SNAPSHOT-shaded.jar:0.5.0-SNAPSHOT-1d4d47adfe7abf0c3ed8a3a14929de084e979968]
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:101)
~[pinot-s3-0.5.0-SNAPSHOT-shaded.jar:0.5.0-SNAPSHOT-1d4d47adfe7abf0c3ed8a3a14929de084e979968]
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
~[pinot-s3-0.5.0-SNAPSHOT-shaded.jar:0.5.0-SNAPSHOT-1d4d47adfe7abf0c3ed8a3a14929de084e979968]`
using docker image with following info:
executionFrameworkSpec:
name: 'standalone'
segmentGenerationJobRunnerClassName:
'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner'
segmentTarPushJobRunnerClassName:
'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner'
segmentUriPushJobRunnerClassName:
'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentUriPushJobRunner'
jobType: SegmentCreationAndTarPush
inputDirURI: 's3://mybucket/'
includeFileNamePattern: 'glob:**/*.json'
outputDirURI: '/tmp/pinot-quick-start/segments/'
overwriteOutput: true
pinotFSSpecs:
- scheme: s3
className: org.apache.pinot.plugin.filesystem.S3PinotFS
configs:
region: your_bucket_region_here
- scheme: file
className: org.apache.pinot.spi.filesystem.LocalPinotFS
recordReaderSpec:
dataFormat: 'json'
className: 'org.apache.pinot.plugin.inputformat.csv.CSVRecordReader'
##TODO change format
configClassName:
'org.apache.pinot.plugin.inputformat.csv.CSVRecordReaderConfig' ##TODO change
format
tableSpec:
tableName: 'transcript'
schemaURI: 'http://pinot-quickstart:9000/tables/transcript/schema'
tableConfigURI: 'http://pinot-quickstart:9000/tables/transcript'
pinotClusterSpecs:
- controllerURI: 'http://pinot-quickstart:9000'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
