[
https://issues.apache.org/jira/browse/DRILL-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669597#comment-16669597
]
Kunal Khatua commented on DRILL-6814:
-------------------------------------
10K records every batch might not be too much, depending on what the average
row size is. If you can save the profile even after a 10-15 minute run, that
helps (though, a complete 30 min run will help most).
When the query completes (or is running after a reasonable amount of time), you
can add a {{.json}} to the URL to get the profile's JSON and save that to your
computer to share on this jira.
e.g.
from http://ashish.drill.com:8047/profiles/24270d0f-249a-128d-b007-1b4602cba4f4
save the link
http://ashish.drill.com:8047/profiles/24270d0f-249a-128d-b007-1b4602cba4f4.json
Also, it might be worth taking a stack trace of the Drillbits occassionally.
Luckily, 1.14 has an option to copy to clipboard on the /threads page.
e.g.
http://ashish.drill.com:8047/threads
Paste the contents to a file. You can do this every 30-60 secs... to crudely
sample what the Drillbit threads are doing. Sharing that would help narrow down
the issue as well.
> Query performance on S3 files
> -----------------------------
>
> Key: DRILL-6814
> URL: https://issues.apache.org/jira/browse/DRILL-6814
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Other
> Affects Versions: 1.14.0
> Environment: Amazon EC2 instances-
> 4 Linux Redhat machines -version 7.5
> RAM- 32GB
> Reporter: Ashish Shukla
> Assignee: Robert Hou
> Priority: Major
>
> I have installed 4 Node drill cluster on Amazon EC2 and trying to execute a
> simple count on one Amazon S3 file. File type is CSV and size is approx- 14GB.
> The query returns expected count after the execution of approx 30 minutes.
> If we keep the same file in hdfs or create a table in postgres, execution
> time is relatively very less (approx 2-3 minutes).
> Is it normal behavior or something can be done for S3 files to make
> execution time comparable ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
