My guess is that spilling to S3 will be disastrously slow.
On Fri, Aug 16, 2019 at 9:37 AM Paul Rogers
wrote:
> Hi Manu,
>
> To add a bit more background... Drill uses local storage only for spilling
> result sets when they are too large for memory. Otherwise, data never
> touches disk once
Hi Divya,
Would be great to add to the docs. First step is to gather the info in a JIRA
ticket. Then, someone (thanks for volunteering) can update the various tables
in the docs.
I believe that the documentation is generated via some process other than a
direct edit of the MD files in the
Hi Manu,
To add a bit more background... Drill uses local storage only for spilling
result sets when they are too large for memory. Otherwise, data never touches
disk once read from S3.
Unlike Snowflake, Drill does not cache S3 data locally. This means that, if you
query the same file
Hi Dave,
Charles and I added the sqlTypeOf() function while writing the book so we could
clearly explain types. Drill also has a SQL-standard function, typeOf(), but
this tends to report "NULL" if the value is null, regardless of type, which
made it hard to explain the behavior of nullable
Can we add this as part of Drill Function docs , will be useful for other
ppl as well ?
I can help to add , unsure if I have access to edit Drill doc
Thanks ,
Divya
On Fri, 16 Aug 2019 at 01:04, Paul Rogers wrote:
> Hi Dave,
>
> As it turns out, improving the detail in function documentation
Thanks Paul, I hadn't seen sqlTypeOf before, that looks perfect for
checking this sort of thing.
Dave
On Thu, 15 Aug 2019 at 18:04, Paul Rogers wrote:
> Hi Dave,
>
> As it turns out, improving the detail in function documentation is a
> long-standing request. The historical answer has been to
>From my learning and I could be wrong in few things but wait for others to
answer as well
1. When stetting up the drill cluster in prod environment to query data
ranging from several gigabytes to few terabytes hosted in s3/blob
storage/cloud storage, what are the considerations for disk space
