Re: EMC ECS Configuration with Apache Drill
I am able to connect to the http endpoint using boto3 from python (able to
retrieve files/store files), from IE with https and port 9021 , it comes
back with 403 Forbidden indicating that it was able to connect to website
but does not have permission to view the webpage ( may be due to
credentials ).
Regards,
Prabu
On Mon, Aug 19, 2019 at 12:59 AM Sorabh Apache
wrote:
> Are you able to use the same configured endpoint *http://:9020* from your
> browser ?
>
> Thanks,
>
> Sorabh.
>
> On Mon, Aug 19, 2019 at 12:19 AM Prabu Mohan
wrote:
> I'm trying to configure Apache Drill with EMC ECS, it was quite easy
> configuring with AWS S3 and GCP Cloud Storage, but I'm facing issues
> configuring with EMC ECS,
>
> When I use http with port 9020 as endpoint I face this error
>
> Error:SYSTEM ERROR: AmazonS3Exception: Status Code: 405, AWS
> Service:Amazon S3, AWS Request ID:, AWS Error Code:null, AWS Error Message:
> Method Not Allowed
>
> core-site.xml
>
>
>
>
> fs.s3a.access.key
> accesskey
>
>
>
> fs.s3a.secret.key
> secretkey
>
>
>
> fs.s3a.endpoint
> http://:9020
>
>
>
> fs.s3a.connection.ssl.enabled
> false
>
>
>
>
> and for connecting to the bucket i use the following in the storage plugin
>
> {
> "type" : "file",
> "connection" : "s3a://bucketname/",
> "config" : {
> "fs.s3a.impl.disable.cache" : "true"
> },
> "workspaces" : {
> "tmp" : {
> "location" : "/tmp",
> "writable" : true,
> "defaultInputFormat" : null,
> "allowAccessOutsideWorkspace" : false
> },
> "root" : {
> "location" : "/",
> "writable" : false,
> "defaultInputFormat" : null,
> "allowAccessOutsideWorkspace" : false
> }
> },
> "formats" : {
> "psv" : {
> "type" : "text",
> "extensions" : [ "tbl" ],
> "delimiter" : "|"
> },
> "csv" : {
> "type" : "text",
> "extensions" : [ "csv" ],
> "delimiter" : ","
> },
> "tsv" : {
> "type" : "text",
> "extensions" : [ "tsv" ],
> "delimiter" : "\t"
> },
> "parquet" : {
> "type" : "parquet"
> },
> "json" : {
> "type" : "json",
> "extensions" : [ "json" ]
> },
> "avro" : {
> "type" : "avro"
> },
> "sequencefile" : {
> "type" : "sequencefile",
> "extensions" : [ "seq" ]
> },
> "csvh" : {
> "type" : "text",
> "extensions" : [ "csvh" ],
> "extractHeader" : true,
> "delimiter" : ","
> }
> },
> "enabled" : true
> }
>
> Any suggestions on how to get this working ?
>
Re: EMC ECS Configuration with Apache Drill
Are you able to use the same configured endpoint *http://:9020* from your
browser ?
Thanks,
Sorabh
On Sun, Aug 18, 2019 at 11:55 AM Prabu Mohan wrote:
> I'm trying to configure Apache Drill with EMC ECS, it was quite easy
> configuring with AWS S3 and GCP Cloud Storage, but I'm facing issues
> configuring with EMC ECS,
>
> When I use http with port 9020 as endpoint I face this error
>
> Error:SYSTEM ERROR: AmazonS3Exception: Status Code: 405, AWS
> Service:Amazon S3, AWS Request ID:, AWS Error Code:null, AWS Error Message:
> Method Not Allowed
>
> core-site.xml
>
>
>
>
> fs.s3a.access.key
> accesskey
>
>
>
> fs.s3a.secret.key
> secretkey
>
>
>
> fs.s3a.endpoint
> http://:9020
>
>
>
> fs.s3a.connection.ssl.enabled
> false
>
>
>
>
> and for connecting to the bucket i use the following in the storage plugin
>
> {
> "type" : "file",
> "connection" : "s3a://bucketname/",
> "config" : {
> "fs.s3a.impl.disable.cache" : "true"
> },
> "workspaces" : {
> "tmp" : {
> "location" : "/tmp",
> "writable" : true,
> "defaultInputFormat" : null,
> "allowAccessOutsideWorkspace" : false
> },
> "root" : {
> "location" : "/",
> "writable" : false,
> "defaultInputFormat" : null,
> "allowAccessOutsideWorkspace" : false
> }
> },
> "formats" : {
> "psv" : {
> "type" : "text",
> "extensions" : [ "tbl" ],
> "delimiter" : "|"
> },
> "csv" : {
> "type" : "text",
> "extensions" : [ "csv" ],
> "delimiter" : ","
> },
> "tsv" : {
> "type" : "text",
> "extensions" : [ "tsv" ],
> "delimiter" : "\t"
> },
> "parquet" : {
> "type" : "parquet"
> },
> "json" : {
> "type" : "json",
> "extensions" : [ "json" ]
> },
> "avro" : {
> "type" : "avro"
> },
> "sequencefile" : {
> "type" : "sequencefile",
> "extensions" : [ "seq" ]
> },
> "csvh" : {
> "type" : "text",
> "extensions" : [ "csvh" ],
> "extractHeader" : true,
> "delimiter" : ","
> }
> },
> "enabled" : true
> }
>
> Any suggestions on how to get this working ?
>
EMC ECS Configuration with Apache Drill
I'm trying to configure Apache Drill with EMC ECS, it was quite easy
configuring with AWS S3 and GCP Cloud Storage, but I'm facing issues
configuring with EMC ECS,
When I use http with port 9020 as endpoint I face this error
Error:SYSTEM ERROR: AmazonS3Exception: Status Code: 405, AWS
Service:Amazon S3, AWS Request ID:, AWS Error Code:null, AWS Error Message:
Method Not Allowed
core-site.xml
fs.s3a.access.key
accesskey
fs.s3a.secret.key
secretkey
fs.s3a.endpoint
http://:9020
fs.s3a.connection.ssl.enabled
false
and for connecting to the bucket i use the following in the storage plugin
{
"type" : "file",
"connection" : "s3a://bucketname/",
"config" : {
"fs.s3a.impl.disable.cache" : "true"
},
"workspaces" : {
"tmp" : {
"location" : "/tmp",
"writable" : true,
"defaultInputFormat" : null,
"allowAccessOutsideWorkspace" : false
},
"root" : {
"location" : "/",
"writable" : false,
"defaultInputFormat" : null,
"allowAccessOutsideWorkspace" : false
}
},
"formats" : {
"psv" : {
"type" : "text",
"extensions" : [ "tbl" ],
"delimiter" : "|"
},
"csv" : {
"type" : "text",
"extensions" : [ "csv" ],
"delimiter" : ","
},
"tsv" : {
"type" : "text",
"extensions" : [ "tsv" ],
"delimiter" : "\t"
},
"parquet" : {
"type" : "parquet"
},
"json" : {
"type" : "json",
"extensions" : [ "json" ]
},
"avro" : {
"type" : "avro"
},
"sequencefile" : {
"type" : "sequencefile",
"extensions" : [ "seq" ]
},
"csvh" : {
"type" : "text",
"extensions" : [ "csvh" ],
"extractHeader" : true,
"delimiter" : ","
}
},
"enabled" : true
}
Any suggestions on how to get this working ?
