Re: Reading Parquet files with array or list columns

2017-06-30 Thread rahul challapalli
HmmI too see no simple workaround for the second case. Can you also file a jira for the CTAS case? Drill could have been running short on heap memory. - Rahul On Fri, Jun 30, 2017 at 11:46 AM, David Kincaid wrote: > The view only works for the first example in the Jira I created. That was >

Re: Reading Parquet files with array or list columns

2017-06-30 Thread David Kincaid
The view only works for the first example in the Jira I created. That was the workaround we have been using since January. Recently we've had a use case where we are running a Spark script to pre-join some data before we try to use it in Drill. That was the subject of the initial e-mail in this th

Re: Reading Parquet files with array or list columns

2017-06-30 Thread rahul challapalli
Like I suggested in the comment for DRILL-5183, can you try using a view as a workaround until the issue gets resolved? On Fri, Jun 30, 2017 at 10:41 AM, David Kincaid wrote: > As far as I was able to discern it is not possible to actually use this > column as an array in Drill at all. It just d

Re: Reading Parquet files with array or list columns

2017-06-30 Thread David Kincaid
As far as I was able to discern it is not possible to actually use this column as an array in Drill at all. It just does not correctly read the Parquet. I have had a very similar defect I created in Jira back in January that has had no attention at all. So we are moving on to other tools. I underst

Re: Reading Parquet files with array or list columns

2017-06-30 Thread François Méthot
Hi, Have you tried: select column['list'][0]['element'] from ... should return "My First Value". or try: select flatten(column['list'])['element] from ... Hope it helps, in our data we have a column that looks like this: [{"NAME:":"Aname", "DATA":"thedata"},{"NAME:":"Aname2", "DATA