Actually picking N_NAME field works for me: select _MAP['N_NAME'] from "sample-data/nation.parquet" order by _MAP['N_NAME'] desc;
Running Ted's query on REGION_KEY gives me this error instead: at org.apache.drill.exec.expr.TypeHelper.getNewVector(TypeHelper.java:504) ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:102) ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:36) ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.sort.SortBatch.next(SortBatch.java:86) ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:25) ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77) ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79) ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_25] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_25] at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25] Which is another problem as you can see. Tim On Wed, Sep 4, 2013 at 3:33 PM, Timothy Chen <[email protected]> wrote: > Hi Jacques, > > Do you know what query with order by will work using the sample data? I'm > trying to find one that can actually work. > > Tim > > > On Wed, Sep 4, 2013 at 1:59 PM, Jacques Nadeau <[email protected]> wrote: > >> In the plan here, the problem is this: >> >> { order : "asc", expr : "_MAP", nullCollation : "NULLS_LAST" } >> >> Two issues: there is no such thing as ordering by a map. Additionally, we >> don't support direct map references yet. >> >> What query were these plans for? We need to make sure everybody has the >> same expectation of output. >> >> J >> >> >> On Wed, Sep 4, 2013 at 1:03 PM, Timothy Chen <[email protected]> wrote: >> >> > Just as Ted pointed out I tried just columns and ordering by different >> > fields and none of them work. >> > >> > It fails on the tip of master too. >> > >> > Logical plan: >> > >> > Logical { head : { type : "APACHE_DRILL_LOGICAL", version : 1, >> generator : >> > { type : "optiq", info : "na" } }, storage : { parquet-local : { type : >> > "parquet", dfsName : "file:///" } }, query : [ { op : "scan", @id : 1, >> memo >> > : "initial_scan", storageengine : "parquet-local", selection : [ { path >> : >> > "sample-data/region.parquet" } ], ref : "_MAP" }, { op : "order", @id : >> 2, >> > input : 1, within : null, orderings : [ { order : "asc", expr : "_MAP", >> > nullCollation : "NULLS_LAST" } ] }, { op : "store", @id : 3, memo : >> "output >> > sink", input : 2, target : { number : 0 }, partition : null, >> storageEngine >> > : "queue" } ] } >> > >> > Physical plan: >> > >> > Physical { head : { type : "APACHE_DRILL_PHYSICAL", version : 1, >> generator >> > : { type : "optiq", info : "na" } }, graph : [ { pop : "parquet-scan", >> @id >> > : 1, entries : [ { path : "sample-data/region.parquet" } ], ref : >> "_MAP", >> > fragmentPointer : 0 }, { pop : "sort", @id : 2, child : 1, orderings : >> [ { >> > order : "asc", expr : "_MAP" } ], reverse : false }, { pop : >> > "selection-vector-remover", @id : 3, child : 2 }, { pop : "screen", @id >> : >> > 4, child : 3 } ] } >> > >> > Tim >> > >> > >> > On Wed, Sep 4, 2013 at 12:42 PM, Jacques Nadeau <[email protected]> >> > wrote: >> > >> > > Good point. Do you want to take a look at the plans and see what are >> > pinch >> > > point is? >> > > >> > > J >> > > On Sep 4, 2013 12:03 PM, "Ted Dunning" <[email protected]> wrote: >> > > >> > > > Hmm... I don't understand the limitation. >> > > > >> > > > I also see these: >> > > > >> > > > *fails*: >> > > > >> > > > select _MAP['REGION_KEY'] from "sample-data/nation.parquet" >> > > > order by cast( _MAP['NATION_KEY'] as INTEGER); >> > > > >> > > > >> > > > *fails*: select _MAP['REGION_KEY'] from "sample-data/nation.parquet" >> > > order >> > > > by _MAP['NATION_KEY']; >> > > > >> > > > >> > > > *fails: *select _MAP['REGION_KEY'] from "sample-data/nation.parquet" >> > > order >> > > > by 1; >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > On Wed, Sep 4, 2013 at 11:53 AM, Jacques Nadeau <[email protected] >> > >> > > > wrote: >> > > > >> > > > > Two problems in this query. >> > > > > >> > > > > First is you're sorting by a map field. We don't currently direct >> > map >> > > > > field expressions. You need to qualify to a particular scalar or >> > > > repeated >> > > > > field. >> > > > > >> > > > > The second problem is that the current code requires resolution >> of a >> > > > field >> > > > > rather than using * when you're trying to reference it elsewhere. >> > > > > >> > > > > therefore: >> > > > > not supported: select * from "sample-data/region.parquet" order >> by 1 >> > ; >> > > > (map >> > > > > keys are not currently referenceable and that is what this >> currently >> > > > means) >> > > > > not supported yet: select * from "sample-data/region.parquet" >> order >> > by >> > > > > _MAP['R_REGIONKEY']; (known issue we should fix in m2 or m3) >> > > > > supported: select _MAP['R_REGIONKEY'] from >> > "sample-data/region.parquet" >> > > > > order by _MAP['R_REGIONKEY']; >> > > > > >> > > > > >> > > > > >> > > > > On Wed, Sep 4, 2013 at 11:39 AM, Timothy Chen <[email protected]> >> > > wrote: >> > > > > >> > > > > > Hi Jacques, >> > > > > > >> > > > > > It i rebased on the top of latest master. >> > > > > > >> > > > > > I am using a newer optiq release (0.4.11) so not sure if master >> has >> > > > that >> > > > > > problem. >> > > > > > >> > > > > > I will try later once I get a chance. >> > > > > > >> > > > > > Tim >> > > > > > >> > > > > > Sent from my iPhone >> > > > > > >> > > > > > On Sep 4, 2013, at 11:04 AM, Jacques Nadeau <[email protected] >> > >> > > > wrote: >> > > > > > >> > > > > > > Is this on the tip of master? >> > > > > > > >> > > > > > > >> > > > > > > On Wed, Sep 4, 2013 at 10:55 AM, Timothy Chen < >> [email protected] >> > > >> > > > > wrote: >> > > > > > > >> > > > > > >> Hi all, >> > > > > > >> >> > > > > > >> I'm getting a exception running a simple select * from >> > > > > > >> "sample-data/region.parquet" order by 1 ; (or any field >> > actually) >> > > > > > >> >> > > > > > >> error_type: 0 >> > > > > > >> message: "Failure while running fragment. < >> > > > > IndexOutOfBoundsException:[ >> > > > > > >> dstIndex: 222 ]" >> > > > > > >> ] >> > > > > > >> at >> > > > org.apache.drill.sql.client.full.ResultEnumerator.moveNext(R >> > > > > > >> esultEnumerator.java:44) >> > > > > > >> at >> > > > net.hydromatic.optiq.runtime.ArrayEnumeratorCursor.next(Arra >> > > > > > >> yEnumeratorCursor.java:44) >> > > > > > >> >> > > > > > >> A bit more digging it looks like it's from the >> > > RemovingBatchCreator >> > > > > > trying >> > > > > > >> to do a copy and getting a index out of bounds in the data >> > > > > valuevector. >> > > > > > >> >> > > > > > >> Jacques suggested someone might be looking into this? Is >> someone >> > > > > > lookling >> > > > > > >> at fixing this arleady? >> > > > > > >> >> > > > > > >> Tim >> > > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
