[
https://issues.apache.org/jira/browse/DRILL-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833051#comment-16833051
]
mehran commented on DRILL-6737:
-------------------------------
I checked the problem in 1.16 release and it is not the parquet writer issue.
there is two step at the end of plan that takes each 2 minutes to finish:
PROJECT_ALLOW_DUP, PROJECT.
h3.
{panel:bgColor=#ffffff}
{panel:bgColor=#f5f5f5}
[Overview|http://10.233.50.111:8047/profiles/23329968-0346-e658-b1c9-92fd7dc60d2a#operator-overview]{panel}
{panel}
||Operator ID||Type||Avg Setup Time||Max Setup Time||Avg Process Time||Max
Process Time||Min Wait Time||Avg Wait Time||Max Wait Time||% Fragment Time||%
Query Time||Rows||Avg Peak Memory||Max Peak Memory||
|00-xx-00|SCREEN|0.000s|0.000s|1.262s|2.510s|0.004s|0.075s|0.145s|0.82%|0.82%|111,491|10MB|20MB|
|00-xx-01|PROJECT|0.002s|0.002s|0.001s|0.001s|0.000s|0.000s|0.000s|0.00%|0.00%|1|-|-|
|00-xx-02|PARQUET_WRITER|0.293s|0.293s|50.750s|50.750s|0.000s|0.000s|0.000s|16.44%|16.44%|111,490|-|-|
|00-xx-03|PROJECT_ALLOW_DUP|0.032s|0.032s|2m0s|2m0s|0.000s|0.000s|0.000s|39.01%|39.01%|111,490|13MB|13MB|
|00-xx-04|PROJECT|16.092s|16.092s|2m15s|2m15s|0.000s|0.000s|0.000s|43.73%|43.73%|111,490|13MB|13MB|
{panel}
{panel}
I do not know what these steps do after parquet writer is finished
But it takes strangely long time to run.
> Ctas json to Parquet is very very slow
> --------------------------------------
>
> Key: DRILL-6737
> URL: https://issues.apache.org/jira/browse/DRILL-6737
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Affects Versions: 1.14.0
> Reporter: mehran
> Assignee: salim achouche
> Priority: Critical
> Attachments: drill.bmp
>
>
> 5 minute takes to insert a json file to parquet, where in 1.13 it takes 10
> seconds. it seems to be a blocker bug.
> In plan it is Parquet writer that takes this duration.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
