Yes, parquet is always better for multiple reasons. With JSON, we have to read
the whole file
from a single reader thread and have to parse to read individual columns.
Parquet compresses and encodes data on disk. So, we read much less data from
disk.
Drill can read individual columns with in each rowgroup in parallel. Also, we
can leverage
features like filter pushdown, partition pruning, metadata cache for better
query performance.
Thanks
Padma
> On Jun 10, 2018, at 8:22 PM, Abhishek Girish wrote:
>
> I would suggest converting the JSON files to parquet for better
> performance. JSON supports a more free form data model, so that's a
> trade-off you need to consider, in my opinion.
> On Sun, Jun 10, 2018 at 8:08 PM Divya Gehlot
> wrote:
>
>> Hi,
>> I am looking for the advise regarding the performance for below :
>> 1. keep the JSON as is
>> 2. Convert the JSON file to parquet files
>>
>> My JSON files data is not in fixed format and file size varies from 10 KB
>> to 1 MB.
>>
>> Appreciate the community users advise on above !
>>
>>
>> Thanks,
>> Divya
>>
