[
https://issues.apache.org/jira/browse/DRILL-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sudheesh Katkam updated DRILL-2666:
-----------------------------------
Attachment: small_cancelled_query2.json
This does not anything to do with the query profile containing large number of
fragments.
This query on a smaller data set fails
{code}
SELECT FLATTEN(t0.fragmentprofile) col FROM dfs.`small_cancelled_query2.json`
t0;
{code}
The "doubleValue" value in one of the operator profiles' metrics is an integer,
which is incorrect. This is because the cancelled_query2.json profile is from
"Full JSON Profile" in the profile page. This profile is subject to JavaScript
[weirdness |
http://stackoverflow.com/questions/5520399/how-to-prevent-removing-decimal-point-when-parsing-json]
that removes the decimal point.
So the "real" query profile that should be used is in the tmp directory
mentioned in the corresponding storage plugin.
In any case, drill should not have failed with _You tried to write a Float8
type when you are using a ValueWriter of type NullableBigIntWriterImpl_. But
there is already a [JIRA | https://issues.apache.org/jira/browse/DRILL-1460]
for this.
> Query against json profile with large number of fragments fails
> ---------------------------------------------------------------
>
> Key: DRILL-2666
> URL: https://issues.apache.org/jira/browse/DRILL-2666
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 0.8.0
> Reporter: Krystal
> Assignee: Sudheesh Katkam
> Fix For: 0.9.0
>
> Attachments: cancelled_query2.json.gz, drill2666.log,
> small_cancelled_query2.json
>
>
> git.commit.id=417ad38f7239c6dc1ff0421cbd5a2358d1437c87
> I have a json profile that contains about 300 fragments. When I try to run
> queries against it, the query would fail with "You tried to do a batch data
> read operation when you were in a state of STOP" error. Below is an example:
> {code}
> 0: jdbc:drill:schema=dfs.hbase> select t3.majorId from (select
> t2.majorFragmentId majorId,t2.minorFragmentProfile.minorFragmentId
> minorId,flatten(t2.minorFragmentProfile.operatorProfile) oProfile from
> (select t1.col.majorFragmentId
> majorFragmentId,flatten(t1.col.minorFragmentProfile) minorFragmentProfile
> from (select flatten(t.fragmentprofile) col from
> `dfs.drillTestDir`.`/ui_profiles/cancelled_query2.json`t) t1) t2) t3 group by
> t3.majorId,t3.oProfile.operatorId,t3.oProfile.operatorType order by
> t3.majorId,t3.oProfile.operatorId;
> Query failed: RemoteRpcException: Failure while running fragment., You tried
> to do a batch data read operation when you were in a state of STOP. You can
> only do this type of operation when you are in a state of OK or
> OK_NEW_SCHEMA. [ 4de479a5-ce93-4a3f-89b4-3aa55481869c on
> qa-node114.qa.lab:31010 ]
> [ 4de479a5-ce93-4a3f-89b4-3aa55481869c on qa-node114.qa.lab:31010 ]
> {code}
> For json profile files that contain smaller number of fragments, the queries
> run fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
