RE: Unable to connect to S3 parquet data using Drill

2016-04-20 Thread Trang Nguyen 
Hi Jason,

I am seeing the same error. The files are stored in parquet using native s3n 
format and we don't have issues to read the data via SparkSQL.
However, one difference is that with SparkSQL, I can drill down to the folder I 
need, whereas in Drill, I need to configure the storage plugin to point to the 
top level bucket, which has subfolders that could be causing the issue 
depending on how Drill is scanning it.
Is there a workaround to configure only a subfolder in Drill?
I've tried changing the workspace to "/data" as well as further nested 
subfolders but I would get the error: 
IllegalArgumentException: Can not create a Path from an empty string

Any advice would be appreciated.

Thanks,
Trang

-Original Message-
From: Nick Monetta 
Sent: Wednesday, April 20, 2016 3:19 PM
To: user@drill.apache.org; Trang Nguyen <trang.ngu...@inrix.com>
Cc: abhishek.gir...@gmail.com
Subject: RE: Unable to connect to S3 parquet data using Drill

+ Trang - see Jason's comments below. 

Jason - I updated the core-site file to have all s3,s3n, and s3a configs.

NowI'm getting an  "IllegalArgumentException: Can not create a Path from an 
empty string" error now from all my queries below. 


My core-site file is edited with s3, s3n, and s3a configs - each with 
corresponding plugins on the web interface. 


  fs.s3.awsAccessKeyId
   



  fs.s3.awsSecretAccessKey
   



  fs.s3n.awsAccessKeyId
   



  fs.s3n.awsSecretAccessKey
   



  fs.s3a.awsAccessKeyId
   



  fs.s3a.awsSecretAccessKey
   



SELECT * FROM s3a. 
`/data/year=2016/month=04/day=01/part-r-0-2e33bf90-9e25-41a9-9a71-8ab758295686.gz.parquet`
 LIMIT 3; SELECT * FROM s3n. 
`/data/year=2016/month=04/day=01/part-r-0-2e33bf90-9e25-41a9-9a71-8ab758295686.gz.parquet`
 LIMIT 3; SELECT * FROM s3. 
`/data/year=2016/month=04/day=01/part-r-0-2e33bf90-9e25-41a9-9a71-8ab758295686.gz.parquet`
 LIMIT 3; SELECT * FROM s3a. `/data/year=2016/month=04/day=01/` LIMIT 3;

RESPONSE: IllegalArgumentException: Can not create a Path from an empty string. 




Nick Monetta | INRIX |ni...@inrix.com |Movement Intelligence | www.inrix.com  | 
mobile +1 646-248-4105 |  
 

-Original Message-
From: Jason Altekruse [mailto:ja...@dremio.com]
Sent: Wednesday, April 20, 2016 5:43 PM
To: user@drill.apache.org
Cc: abhishek.gir...@gmail.com
Subject: Re: Unable to connect to S3 parquet data using Drill

Looking here it appears you need to set up an empty bucket to store a 
filesystem if you are going to use s3:// [1]. Are you trying to connect to a 
bucket you have populated with the normal S3 bucket APIs and not just the HDFS 
FileSystem API calls? Have you tried connecting instead with s3a? It looks like 
from this doc page that s3n and s3a are designed to connect to existing buckets 
filled with files, with s3a being a complete replacement for s3n.

I believe the error you are seeing means it cannot find the path "/". It is 
probably trying to look up the root of the filesystem wherever it puts metadata 
in the bucket (maybe a hidden file or something?) and it isn't finding it. This 
makes me think that your bucket isn't set up as it is expected to be for a 
connection using s3://.

[1] - https://wiki.apache.org/hadoop/AmazonS3

Jason Altekruse
Software Engineer at Dremio
Apache Drill Committer

On Wed, Apr 20, 2016 at 2:34 PM, Nick Monetta <ni...@inrix.com> wrote:

> Thanks for the quick responses!
>
> I'm using drill 1.4.  I think I may have sorted out my S3 connections 
> issues, but I'm not sure because I'm having trouble executing a query:
>
>
> My s3 connection (named "s3"):
> {
>   "type": "file",
>   "enabled": true,
>   "connection": "s3://inrixprod-tapp/",
>   "workspaces": {
> "root": {
>   "location": "/",
>   "writable": false,
>   "defaultInputFormat": null
> }
>
> Query:
> SELECT * FROM
> s3.`data/year=2016/month=02/day=28/part-r-0-f2b42e00-ff01-4d82-84e
> 3-c75aafa007ae.gz.parquet`
> LIMIT 3;
>
>  Response:
> Query Failed: An Error Occurred
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> IOException: / doesn't exist [Error Id:
> 9e076a2b-c4fa-4020-af2e-4d43c2e9588c on 
> NickM-LPT02.inrix.corpnet.local:31010]
>
>
>
> Nick Monetta | INRIX |ni...@inrix.com |Movement Intelligence | 
> www.inrix.com  | mobile +1 646-248-4105 |
>
>
> -Original Message-
> From: Jason Altekruse [mailto:ja...@dremio.com]
> Sent: Wednesday, April 20, 2016 4:45 PM
> To: user@drill.apache.org
> Cc: abhishek.gir...@gmail.com
> Subject: Re: Unable to connect to S3 parquet data using Drill
>
> Which version of Drill are you running? The config block for adding 
> your credentials was added in a recent release,

RE: Unable to connect to S3 parquet data using Drill

2016-04-20 Thread Nick Monetta 
+ Trang - see Jason's comments below. 

Jason - I updated the core-site file to have all s3,s3n, and s3a configs.

NowI'm getting an  "IllegalArgumentException: Can not create a Path from an 
empty string" error now from all my queries below. 


My core-site file is edited with s3, s3n, and s3a configs - each with 
corresponding plugins on the web interface. 


  fs.s3.awsAccessKeyId
   



  fs.s3.awsSecretAccessKey
   



  fs.s3n.awsAccessKeyId
   



  fs.s3n.awsSecretAccessKey
   



  fs.s3a.awsAccessKeyId
   



  fs.s3a.awsSecretAccessKey
   



SELECT * FROM s3a. 
`/data/year=2016/month=04/day=01/part-r-0-2e33bf90-9e25-41a9-9a71-8ab758295686.gz.parquet`
 LIMIT 3;
SELECT * FROM s3n. 
`/data/year=2016/month=04/day=01/part-r-0-2e33bf90-9e25-41a9-9a71-8ab758295686.gz.parquet`
 LIMIT 3;
SELECT * FROM s3. 
`/data/year=2016/month=04/day=01/part-r-0-2e33bf90-9e25-41a9-9a71-8ab758295686.gz.parquet`
 LIMIT 3;
SELECT * FROM s3a. `/data/year=2016/month=04/day=01/` LIMIT 3;

RESPONSE: IllegalArgumentException: Can not create a Path from an empty string. 




Nick Monetta | INRIX |ni...@inrix.com |Movement Intelligence | www.inrix.com  | 
mobile +1 646-248-4105 |  
 

-Original Message-
From: Jason Altekruse [mailto:ja...@dremio.com] 
Sent: Wednesday, April 20, 2016 5:43 PM
To: user@drill.apache.org
Cc: abhishek.gir...@gmail.com
Subject: Re: Unable to connect to S3 parquet data using Drill

Looking here it appears you need to set up an empty bucket to store a 
filesystem if you are going to use s3:// [1]. Are you trying to connect to a 
bucket you have populated with the normal S3 bucket APIs and not just the HDFS 
FileSystem API calls? Have you tried connecting instead with s3a? It looks like 
from this doc page that s3n and s3a are designed to connect to existing buckets 
filled with files, with s3a being a complete replacement for s3n.

I believe the error you are seeing means it cannot find the path "/". It is 
probably trying to look up the root of the filesystem wherever it puts metadata 
in the bucket (maybe a hidden file or something?) and it isn't finding it. This 
makes me think that your bucket isn't set up as it is expected to be for a 
connection using s3://.

[1] - https://wiki.apache.org/hadoop/AmazonS3

Jason Altekruse
Software Engineer at Dremio
Apache Drill Committer

On Wed, Apr 20, 2016 at 2:34 PM, Nick Monetta <ni...@inrix.com> wrote:

> Thanks for the quick responses!
>
> I'm using drill 1.4.  I think I may have sorted out my S3 connections 
> issues, but I'm not sure because I'm having trouble executing a query:
>
>
> My s3 connection (named "s3"):
> {
>   "type": "file",
>   "enabled": true,
>   "connection": "s3://inrixprod-tapp/",
>   "workspaces": {
> "root": {
>   "location": "/",
>   "writable": false,
>   "defaultInputFormat": null
> }
>
> Query:
> SELECT * FROM
> s3.`data/year=2016/month=02/day=28/part-r-0-f2b42e00-ff01-4d82-84e
> 3-c75aafa007ae.gz.parquet`
> LIMIT 3;
>
>  Response:
> Query Failed: An Error Occurred
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> IOException: / doesn't exist [Error Id:
> 9e076a2b-c4fa-4020-af2e-4d43c2e9588c on 
> NickM-LPT02.inrix.corpnet.local:31010]
>
>
>
> Nick Monetta | INRIX |ni...@inrix.com |Movement Intelligence | 
> www.inrix.com  | mobile +1 646-248-4105 |
>
>
> -Original Message-
> From: Jason Altekruse [mailto:ja...@dremio.com]
> Sent: Wednesday, April 20, 2016 4:45 PM
> To: user@drill.apache.org
> Cc: abhishek.gir...@gmail.com
> Subject: Re: Unable to connect to S3 parquet data using Drill
>
> Which version of Drill are you running? The config block for adding 
> your credentials was added in a recent release, I believe 1.5.
>
> Jason Altekruse
> Software Engineer at Dremio
> Apache Drill Committer
>
> On Wed, Apr 20, 2016 at 1:38 PM, Nick Monetta <ni...@inrix.com> wrote:
>
> > Copying and pasting your JSON directly into a new configuration gets 
> > me “Error (invalid JSON Mapping)”.
> >
> >
> >
> > What am I doing wrong?
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Nick Monetta | INRIX |ni...@inrix.com |Movement Intelligence | 
> > www.inrix.com  | mobile +1 646-248-4105 |
> >
> >
> >
> >
> >
> > -Original Message-
> > From: Jason Altekruse [mailto:ja...@dremio.com]
> > Sent: Wednesday, April 20, 2016 4:27 PM
> > To: user@drill.apache.org
> > Cc: abhishek.gir...@gmail.com
> > Subject: Re: Unable to connect to S3 parquet data using Drill
> >
> >
> >
> &g

Re: Unable to connect to S3 parquet data using Drill

2016-04-20 Thread Jason Altekruse 
Looking here it appears you need to set up an empty bucket to store a
filesystem if you are going to use s3:// [1]. Are you trying to connect to
a bucket you have populated with the normal S3 bucket APIs and not just the
HDFS FileSystem API calls? Have you tried connecting instead with s3a? It
looks like from this doc page that s3n and s3a are designed to connect to
existing buckets filled with files, with s3a being a complete replacement
for s3n.

I believe the error you are seeing means it cannot find the path "/". It is
probably trying to look up the root of the filesystem wherever it puts
metadata in the bucket (maybe a hidden file or something?) and it isn't
finding it. This makes me think that your bucket isn't set up as it is
expected to be for a connection using s3://.

[1] - https://wiki.apache.org/hadoop/AmazonS3

Jason Altekruse
Software Engineer at Dremio
Apache Drill Committer

On Wed, Apr 20, 2016 at 2:34 PM, Nick Monetta <ni...@inrix.com> wrote:

> Thanks for the quick responses!
>
> I'm using drill 1.4.  I think I may have sorted out my S3 connections
> issues, but I'm not sure because I'm having trouble executing a query:
>
>
> My s3 connection (named "s3"):
> {
>   "type": "file",
>   "enabled": true,
>   "connection": "s3://inrixprod-tapp/",
>   "workspaces": {
> "root": {
>   "location": "/",
>   "writable": false,
>   "defaultInputFormat": null
> }
>
> Query:
> SELECT * FROM
> s3.`data/year=2016/month=02/day=28/part-r-0-f2b42e00-ff01-4d82-84e3-c75aafa007ae.gz.parquet`
> LIMIT 3;
>
>  Response:
> Query Failed: An Error Occurred
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> IOException: / doesn't exist [Error Id:
> 9e076a2b-c4fa-4020-af2e-4d43c2e9588c on
> NickM-LPT02.inrix.corpnet.local:31010]
>
>
>
> Nick Monetta | INRIX |ni...@inrix.com |Movement Intelligence |
> www.inrix.com  | mobile +1 646-248-4105 |
>
>
> -Original Message-
> From: Jason Altekruse [mailto:ja...@dremio.com]
> Sent: Wednesday, April 20, 2016 4:45 PM
> To: user@drill.apache.org
> Cc: abhishek.gir...@gmail.com
> Subject: Re: Unable to connect to S3 parquet data using Drill
>
> Which version of Drill are you running? The config block for adding your
> credentials was added in a recent release, I believe 1.5.
>
> Jason Altekruse
> Software Engineer at Dremio
> Apache Drill Committer
>
> On Wed, Apr 20, 2016 at 1:38 PM, Nick Monetta <ni...@inrix.com> wrote:
>
> > Copying and pasting your JSON directly into a new configuration gets
> > me “Error (invalid JSON Mapping)”.
> >
> >
> >
> > What am I doing wrong?
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Nick Monetta | INRIX |ni...@inrix.com |Movement Intelligence |
> > www.inrix.com  | mobile +1 646-248-4105 |
> >
> >
> >
> >
> >
> > -Original Message-
> > From: Jason Altekruse [mailto:ja...@dremio.com]
> > Sent: Wednesday, April 20, 2016 4:27 PM
> > To: user@drill.apache.org
> > Cc: abhishek.gir...@gmail.com
> > Subject: Re: Unable to connect to S3 parquet data using Drill
> >
> >
> >
> > {
> >
> >   "type": "file",
> >
> >   "enabled": true,
> >
> >   "connection": "s3a://PATH.TO.BUCKET/",
> >
> >   "config": {
> >
> > "fs.s3a.access.key": "",
> >
> > "fs.s3a.secret.key": ""
> >
> >   },
> >
> >   "workspaces": {
> >
> > "root": {
> >
> >   "location": "/",
> >
> >   "writable": false,
> >
> >   "defaultInputFormat": null
> >
> > },
> >
> > "tmp": {
> >
> >   "location": "/tmp",
> >
> >   "writable": true,
> >
> >   "defaultInputFormat": null
> >
> > }
> >
> >   },
> >
> >   "formats": {
> >
> > "psv": {
> >
> >   "type": "text",
> >
> >   "extensions": [
> >
> > "tbl"
> >
> >   ],
> >
> >   "delimiter": "|"
> >
> > },
> >
> > "csv": {
> >
> >   "type": "text",
> >
> >

Re: Unable to connect to S3 parquet data using Drill

2016-04-20 Thread Jason Altekruse 
Which version of Drill are you running? The config block for adding your
credentials was added in a recent release, I believe 1.5.

Jason Altekruse
Software Engineer at Dremio
Apache Drill Committer

On Wed, Apr 20, 2016 at 1:38 PM, Nick Monetta <ni...@inrix.com> wrote:

> Copying and pasting your JSON directly into a new configuration gets me
> “Error (invalid JSON Mapping)”.
>
>
>
> What am I doing wrong?
>
>
>
>
>
>
>
>
>
> Nick Monetta | INRIX |ni...@inrix.com |Movement Intelligence |
> www.inrix.com  | mobile +1 646-248-4105 |
>
>
>
>
>
> -Original Message-
> From: Jason Altekruse [mailto:ja...@dremio.com]
> Sent: Wednesday, April 20, 2016 4:27 PM
> To: user@drill.apache.org
> Cc: abhishek.gir...@gmail.com
> Subject: Re: Unable to connect to S3 parquet data using Drill
>
>
>
> {
>
>   "type": "file",
>
>   "enabled": true,
>
>   "connection": "s3a://PATH.TO.BUCKET/",
>
>   "config": {
>
> "fs.s3a.access.key": "",
>
> "fs.s3a.secret.key": ""
>
>   },
>
>   "workspaces": {
>
> "root": {
>
>   "location": "/",
>
>   "writable": false,
>
>   "defaultInputFormat": null
>
> },
>
> "tmp": {
>
>   "location": "/tmp",
>
>   "writable": true,
>
>   "defaultInputFormat": null
>
> }
>
>   },
>
>   "formats": {
>
> "psv": {
>
>   "type": "text",
>
>   "extensions": [
>
> "tbl"
>
>   ],
>
>   "delimiter": "|"
>
> },
>
> "csv": {
>
>   "type": "text",
>
>   "extensions": [
>
> "csv"
>
>   ],
>
>   "delimiter": ","
>
> },
>
> "tsv": {
>
>   "type": "text",
>
>   "extensions": [
>
> "tsv"
>
>   ],
>
>   "delimiter": "\t"
>
> },
>
> "parquet": {
>
>   "type": "parquet"
>
> },
>
> "json": {
>
>   "type": "json",
>
>   "extensions": [
>
> "json"
>
>   ]
>
> },
>
> "avro": {
>
>   "type": "avro"
>
> },
>
>     "sequencefile": {
>
>   "type": "sequencefile",
>
>   "extensions": [
>
> "seq"
>
>   ]
>
> },
>
> "csvh": {
>
>   "type": "text",
>
>   "extensions": [
>
> "csvh"
>
>   ],
>
>   "extractHeader": true,
>
>   "delimiter": ","
>
> }
>
>   }
>
> }
>
>
>
> Jason Altekruse
>
> Software Engineer at Dremio
>
> Apache Drill Committer
>
>
>
> On Wed, Apr 20, 2016 at 1:24 PM, Nick Monetta <ni...@inrix.com> wrote:
>
>
>
> > Can you send me the full JSON for the new config example you provided?
>
> > I keep getting JSON errors.
>
> >
>
> >
>
> > Nick Monetta | INRIX |ni...@inrix.com |Movement Intelligence |
>
> > www.inrix.com  | mobile +1 646-248-4105 |
>
> >
>
> >
>
> > -Original Message-
>
> > From: Abhishek Girish [mailto:abhishek.gir...@gmail.com
> <abhishek.gir...@gmail.com>]
>
> > Sent: Wednesday, April 20, 2016 12:57 PM
>
> > To: user <user@drill.apache.org>
>
> > Subject: Re: Unable to connect to S3 parquet data using Drill
>
> >
>
> > Hey Trang,
>
> >
>
> > A similar issue related to S3 config was discussed today on the
>
> > mailing list [1]. Can you see if that helps resolve the issue?
>
> >
>
> > [1]
>
> >
>
> > http://mail-archives.apache.org/mod_mbox/drill-dev/201604.mbox/%3CCAN6
>
> > ttnukzsAKgQE-RTF0RNCvBr1uWsB9SaxnS_7y-v0yBdUj%3Dw%40mail.gmail.com%3E
>
> >
>
> >
>
> > -Abhishek
>
> >
>
> > On Tue, Apr 19, 2016 at 6:38 PM, Trang Nguyen <trang.ngu...@inrix.com>
>
> > wrote:
>
> >
>
> > > Hi,
>
> > >
>
> > > I am having trouble to connect to an Amazon S3 bucket containing
>
> > > parquet files.
>
> > > I followed the instructions on
>
> > > https://drill.apache.org/docs/s3-storage-plugin/ to download
>
> > > jets3_0.9.3 on my Ubuntu VM.
>
> > > My storage configs:
>
> > > {
>
> > >   "type": "file",
>
> > >   "enabled": true,
>
> > >   "connection": "s3://inrixprod-tapp",
>
> > >   "config": null,
>
> > >   "workspaces": {
>
> > > "root": {
>
> > >   "location": "/",
>
> > >   "writable": false,
>
> > >   "defaultInputFormat": null
>
> > > },
>
> > > "tmp": {
>
> > >   "location": "/tmp",
>
> > >   "writable": true,
>
> > >   "defaultInputFormat": null
>
> > > }
>
> > >   },
>
> > > ...
>
> > > }
>
> > >
>
> > > I've started the embedded-drill instance but get the following error
>
> > > trying to connect:
>
> > > 0: jdbc:drill:zk=local> use s3-trips.`root`;
>
> > > Error: SYSTEM ERROR: IOException: / doesn't exist
>
> > >
>
> > >
>
> > > [Error Id: 081c66e6-177d-48fa-8eca-4ee1370ae785 on
>
> > > ubuntu-VirtualBox:31010] (state=,code=0)
>
> > >
>
> > > Any advice would be appreciated!
>
> > >
>
> > > Thanks,
>
> > > Trang
>
> > >
>
> >
>

RE: Unable to connect to S3 parquet data using Drill

2016-04-20 Thread Nick Monetta 
Copying and pasting your JSON directly into a new configuration gets me “Error 
(invalid JSON Mapping)”.



What am I doing wrong?







[cid:image003.jpg@01D19B22.E22C4340]



Nick Monetta | INRIX |ni...@inrix.com |Movement Intelligence | www.inrix.com  | 
mobile +1 646-248-4105 |





-Original Message-
From: Jason Altekruse [mailto:ja...@dremio.com]
Sent: Wednesday, April 20, 2016 4:27 PM
To: user@drill.apache.org
Cc: abhishek.gir...@gmail.com
Subject: Re: Unable to connect to S3 parquet data using Drill



{

  "type": "file",

  "enabled": true,

  "connection": "s3a://PATH.TO.BUCKET/",

  "config": {

"fs.s3a.access.key": "",

"fs.s3a.secret.key": ""

  },

  "workspaces": {

"root": {

  "location": "/",

  "writable": false,

  "defaultInputFormat": null

},

"tmp": {

  "location": "/tmp",

  "writable": true,

  "defaultInputFormat": null

}

  },

  "formats": {

"psv": {

  "type": "text",

  "extensions": [

"tbl"

  ],

  "delimiter": "|"

},

"csv": {

  "type": "text",

  "extensions": [

"csv"

  ],

  "delimiter": ","

},

"tsv": {

  "type": "text",

  "extensions": [

"tsv"

  ],

  "delimiter": "\t"

},

"parquet": {

  "type": "parquet"

},

"json": {

  "type": "json",

  "extensions": [

"json"

  ]

},

"avro": {

  "type": "avro"

},

"sequencefile": {

  "type": "sequencefile",

  "extensions": [

"seq"

  ]

},

"csvh": {

  "type": "text",

  "extensions": [

"csvh"

  ],

  "extractHeader": true,

  "delimiter": ","

}

  }

}



Jason Altekruse

Software Engineer at Dremio

Apache Drill Committer



On Wed, Apr 20, 2016 at 1:24 PM, Nick Monetta 
<ni...@inrix.com<mailto:ni...@inrix.com>> wrote:



> Can you send me the full JSON for the new config example you provided?

> I keep getting JSON errors.

>

>

> Nick Monetta | INRIX |ni...@inrix.com<mailto:|ni...@inrix.com> |Movement 
> Intelligence |

> www.inrix.com<http://www.inrix.com>  | mobile +1 646-248-4105 |

>

>

> -Original Message-

> From: Abhishek Girish [mailto:abhishek.gir...@gmail.com]

> Sent: Wednesday, April 20, 2016 12:57 PM

> To: user <user@drill.apache.org<mailto:user@drill.apache.org>>

> Subject: Re: Unable to connect to S3 parquet data using Drill

>

> Hey Trang,

>

> A similar issue related to S3 config was discussed today on the

> mailing list [1]. Can you see if that helps resolve the issue?

>

> [1]

>

> http://mail-archives.apache.org/mod_mbox/drill-dev/201604.mbox/%3CCAN6

> ttnukzsAKgQE-RTF0RNCvBr1uWsB9SaxnS_7y-v0yBdUj%3Dw%40mail.gmail.com%3E

>

>

> -Abhishek

>

> On Tue, Apr 19, 2016 at 6:38 PM, Trang Nguyen 
> <trang.ngu...@inrix.com<mailto:trang.ngu...@inrix.com>>

> wrote:

>

> > Hi,

> >

> > I am having trouble to connect to an Amazon S3 bucket containing

> > parquet files.

> > I followed the instructions on

> > https://drill.apache.org/docs/s3-storage-plugin/ to download

> > jets3_0.9.3 on my Ubuntu VM.

> > My storage configs:

> > {

> >   "type": "file",

> >   "enabled": true,

> >   "connection": "s3://inrixprod-tapp",

> >   "config": null,

> >   "workspaces": {

> > "root": {

> >   "location": "/",

> >   "writable": false,

> >   "defaultInputFormat": null

> > },

> > "tmp": {

> >   "location": "/tmp",

> >   "writable": true,

> >   "defaultInputFormat": null

> > }

> >   },

> > ...

> > }

> >

> > I've started the embedded-drill instance but get the following error

> > trying to connect:

> > 0: jdbc:drill:zk=local> use s3-trips.`root`;

> > Error: SYSTEM ERROR: IOException: / doesn't exist

> >

> >

> > [Error Id: 081c66e6-177d-48fa-8eca-4ee1370ae785 on

> > ubuntu-VirtualBox:31010] (state=,code=0)

> >

> > Any advice would be appreciated!

> >

> > Thanks,

> > Trang

> >

>

Re: Unable to connect to S3 parquet data using Drill

2016-04-20 Thread Abhishek Girish 
Hey Trang,

A similar issue related to S3 config was discussed today on the mailing
list [1]. Can you see if that helps resolve the issue?

[1]
http://mail-archives.apache.org/mod_mbox/drill-dev/201604.mbox/%3CCAN6ttnukzsAKgQE-RTF0RNCvBr1uWsB9SaxnS_7y-v0yBdUj%3Dw%40mail.gmail.com%3E


-Abhishek

On Tue, Apr 19, 2016 at 6:38 PM, Trang Nguyen 
wrote:

> Hi,
>
> I am having trouble to connect to an Amazon S3 bucket containing parquet
> files.
> I followed the instructions on
> https://drill.apache.org/docs/s3-storage-plugin/ to download jets3_0.9.3
> on my Ubuntu VM.
> My storage configs:
> {
>   "type": "file",
>   "enabled": true,
>   "connection": "s3://inrixprod-tapp",
>   "config": null,
>   "workspaces": {
> "root": {
>   "location": "/",
>   "writable": false,
>   "defaultInputFormat": null
> },
> "tmp": {
>   "location": "/tmp",
>   "writable": true,
>   "defaultInputFormat": null
> }
>   },
> ...
> }
>
> I've started the embedded-drill instance but get the following error
> trying to connect:
> 0: jdbc:drill:zk=local> use s3-trips.`root`;
> Error: SYSTEM ERROR: IOException: / doesn't exist
>
>
> [Error Id: 081c66e6-177d-48fa-8eca-4ee1370ae785 on
> ubuntu-VirtualBox:31010] (state=,code=0)
>
> Any advice would be appreciated!
>
> Thanks,
> Trang
>

Unable to connect to S3 parquet data using Drill

2016-04-19 Thread Trang Nguyen 
Hi,

I am having trouble to connect to an Amazon S3 bucket containing parquet files.
I followed the instructions on https://drill.apache.org/docs/s3-storage-plugin/ 
to download jets3_0.9.3 on my Ubuntu VM.
My storage configs:
{
  "type": "file",
  "enabled": true,
  "connection": "s3://inrixprod-tapp",
  "config": null,
  "workspaces": {
"root": {
  "location": "/",
  "writable": false,
  "defaultInputFormat": null
},
"tmp": {
  "location": "/tmp",
  "writable": true,
  "defaultInputFormat": null
}
  },
...
}

I've started the embedded-drill instance but get the following error trying to 
connect:
0: jdbc:drill:zk=local> use s3-trips.`root`;
Error: SYSTEM ERROR: IOException: / doesn't exist


[Error Id: 081c66e6-177d-48fa-8eca-4ee1370ae785 on ubuntu-VirtualBox:31010] 
(state=,code=0)

Any advice would be appreciated!

Thanks,
Trang