Was able to connect to N Virginia, thanks. But to be able to use the Drill as a standard tool, would need to be able to connect to all regions, of course
On Thu, Jun 15, 2017 at 7:20 AM, Jack Ingoldsby <[email protected]> wrote: > Thx for this. Sounds like a combination of AWS/Drill factors. > Are we likely to address the Drill side in a subsequent release? > > On Thu, Jun 15, 2017, 01:39 Uwe L. Korn <[email protected]> wrote: > >> The current Drill releases use the hadoop-io libraries from the 2.7.x >> series. Locally I have built against the 3.0.0 alpha (2.8 should also work) >> and can access the regions with newer signature versions. But you should be >> careful with that as I had to do some code changes to have it built with >> the 3.0 jars and there were some breaking unit tests afterwards. >> >> Also note that 2.8/3.0 greatly improves on S3 performance if you select >> the new (and experimental) random-access mode in s3a. This resulted for me >> in massive improvements for queries that only access a fraction of all >> columns or that have multiple RowGroups inside each Parquet file. >> >> > Am 15.06.2017 um 06:36 schrieb Shankar Mane <[email protected] >> >: >> > >> > aws new regions uses only signature version 4 protocol for S3. Other >> > regions has both V2 and V4 compatible. Drill works very well if regions >> has >> > both signature versions. >> > >> > By adding endpoints, same problem persists. May be Drill API doesn't >> have >> > support to V4 protocol yet. >> > >> > This V4 problems is also with native hadoop versions prior to 2.8.0. >> > >> > >> > >> >> On 15-Jun-2017 9:49 AM, "Jack Ingoldsby" <[email protected]> >> wrote: >> >> >> >> Useful to know, thanks. Also having problems with Ohio. Will try >> another >> >> region >> >> >> >>> On Wed, Jun 14, 2017, 19:46 Сергей Боровик <[email protected]> >> wrote: >> >>> >> >>> Hi! >> >>> I have an AWS EC2 instance with apache drill 1-10.0.and configured IAM >> >>> Role. >> >>> >> >>> And I am able to access and query S3 bucket in US East (N. Virginia) >> >>> region, >> >>> but not able to access/query buckets in US East (Ohio) region, it >> fails >> >>> with >> >>> "error: system error: amazons3exception: status code 400, AWS Service: >> >>> Amazon S3, >> >>> AWS Request ID:9D54A8310F26582B, AWS Error Code: null, AWS Error >> Message: >> >>> Bad Request" >> >>> >> >>> >> >>> I've tried set conf/core-site.xml property to: >> >>> >> >>> <property> >> >>> <name>fs.s3a.endpoint</name> >> >>> <value>s3.us-east-2.amazonaws.com</value> >> >>> </property> >> >>> >> >>> in this case Ohio fails with the same error, >> >>> and N. Virginia has error status code 301, AWS Error Code: >> >>> PermanentRedirect, >> >>> AWS Error message: The bucket you are attempting to access must be >> >>> addressed using the specified endpoint >> >>> >> >>> 1) Is there any specific configuration that needs to be enabled on >> Drill >> >>> for Ohio region? >> >>> 2) Does Drill not work on aws signature version 4? >> >>> >> >>> Thank you in advance. >> >>> Any advice is much appreciated! >> >>> >> >> >> >>
