Can you file one or more bugs for this with more details? We need to figure out if we should include it in m1.
On Wed, Sep 4, 2013 at 3:39 PM, Timothy Chen <[email protected]> wrote: > Actually picking N_NAME field works for me: > > select _MAP['N_NAME'] from "sample-data/nation.parquet" order by > _MAP['N_NAME'] desc; > > Running Ted's query on REGION_KEY gives me this error instead: > > at org.apache.drill.exec.expr.TypeHelper.getNewVector(TypeHelper.java:504) > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > at > > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:102) > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > at > > org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:36) > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.sort.SortBatch.next(SortBatch.java:86) > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > at > > org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:25) > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > at > > org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77) > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > at org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79) > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_25] > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ~[na:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25] > > Which is another problem as you can see. > > Tim > > > On Wed, Sep 4, 2013 at 3:33 PM, Timothy Chen <[email protected]> wrote: > > > Hi Jacques, > > > > Do you know what query with order by will work using the sample data? I'm > > trying to find one that can actually work. > > > > Tim > > > > > > On Wed, Sep 4, 2013 at 1:59 PM, Jacques Nadeau <[email protected]> > wrote: > > > >> In the plan here, the problem is this: > >> > >> { order : "asc", expr : "_MAP", nullCollation : "NULLS_LAST" } > >> > >> Two issues: there is no such thing as ordering by a map. Additionally, > we > >> don't support direct map references yet. > >> > >> What query were these plans for? We need to make sure everybody has the > >> same expectation of output. > >> > >> J > >> > >> > >> On Wed, Sep 4, 2013 at 1:03 PM, Timothy Chen <[email protected]> wrote: > >> > >> > Just as Ted pointed out I tried just columns and ordering by different > >> > fields and none of them work. > >> > > >> > It fails on the tip of master too. > >> > > >> > Logical plan: > >> > > >> > Logical { head : { type : "APACHE_DRILL_LOGICAL", version : 1, > >> generator : > >> > { type : "optiq", info : "na" } }, storage : { parquet-local : { type > : > >> > "parquet", dfsName : "file:///" } }, query : [ { op : "scan", @id : 1, > >> memo > >> > : "initial_scan", storageengine : "parquet-local", selection : [ { > path > >> : > >> > "sample-data/region.parquet" } ], ref : "_MAP" }, { op : "order", @id > : > >> 2, > >> > input : 1, within : null, orderings : [ { order : "asc", expr : > "_MAP", > >> > nullCollation : "NULLS_LAST" } ] }, { op : "store", @id : 3, memo : > >> "output > >> > sink", input : 2, target : { number : 0 }, partition : null, > >> storageEngine > >> > : "queue" } ] } > >> > > >> > Physical plan: > >> > > >> > Physical { head : { type : "APACHE_DRILL_PHYSICAL", version : 1, > >> generator > >> > : { type : "optiq", info : "na" } }, graph : [ { pop : "parquet-scan", > >> @id > >> > : 1, entries : [ { path : "sample-data/region.parquet" } ], ref : > >> "_MAP", > >> > fragmentPointer : 0 }, { pop : "sort", @id : 2, child : 1, orderings : > >> [ { > >> > order : "asc", expr : "_MAP" } ], reverse : false }, { pop : > >> > "selection-vector-remover", @id : 3, child : 2 }, { pop : "screen", > @id > >> : > >> > 4, child : 3 } ] } > >> > > >> > Tim > >> > > >> > > >> > On Wed, Sep 4, 2013 at 12:42 PM, Jacques Nadeau <[email protected]> > >> > wrote: > >> > > >> > > Good point. Do you want to take a look at the plans and see what > are > >> > pinch > >> > > point is? > >> > > > >> > > J > >> > > On Sep 4, 2013 12:03 PM, "Ted Dunning" <[email protected]> > wrote: > >> > > > >> > > > Hmm... I don't understand the limitation. > >> > > > > >> > > > I also see these: > >> > > > > >> > > > *fails*: > >> > > > > >> > > > select _MAP['REGION_KEY'] from "sample-data/nation.parquet" > >> > > > order by cast( _MAP['NATION_KEY'] as INTEGER); > >> > > > > >> > > > > >> > > > *fails*: select _MAP['REGION_KEY'] from > "sample-data/nation.parquet" > >> > > order > >> > > > by _MAP['NATION_KEY']; > >> > > > > >> > > > > >> > > > *fails: *select _MAP['REGION_KEY'] from > "sample-data/nation.parquet" > >> > > order > >> > > > by 1; > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > On Wed, Sep 4, 2013 at 11:53 AM, Jacques Nadeau < > [email protected] > >> > > >> > > > wrote: > >> > > > > >> > > > > Two problems in this query. > >> > > > > > >> > > > > First is you're sorting by a map field. We don't currently > direct > >> > map > >> > > > > field expressions. You need to qualify to a particular scalar > or > >> > > > repeated > >> > > > > field. > >> > > > > > >> > > > > The second problem is that the current code requires resolution > >> of a > >> > > > field > >> > > > > rather than using * when you're trying to reference it > elsewhere. > >> > > > > > >> > > > > therefore: > >> > > > > not supported: select * from "sample-data/region.parquet" order > >> by 1 > >> > ; > >> > > > (map > >> > > > > keys are not currently referenceable and that is what this > >> currently > >> > > > means) > >> > > > > not supported yet: select * from "sample-data/region.parquet" > >> order > >> > by > >> > > > > _MAP['R_REGIONKEY']; (known issue we should fix in m2 or m3) > >> > > > > supported: select _MAP['R_REGIONKEY'] from > >> > "sample-data/region.parquet" > >> > > > > order by _MAP['R_REGIONKEY']; > >> > > > > > >> > > > > > >> > > > > > >> > > > > On Wed, Sep 4, 2013 at 11:39 AM, Timothy Chen < > [email protected]> > >> > > wrote: > >> > > > > > >> > > > > > Hi Jacques, > >> > > > > > > >> > > > > > It i rebased on the top of latest master. > >> > > > > > > >> > > > > > I am using a newer optiq release (0.4.11) so not sure if > master > >> has > >> > > > that > >> > > > > > problem. > >> > > > > > > >> > > > > > I will try later once I get a chance. > >> > > > > > > >> > > > > > Tim > >> > > > > > > >> > > > > > Sent from my iPhone > >> > > > > > > >> > > > > > On Sep 4, 2013, at 11:04 AM, Jacques Nadeau < > [email protected] > >> > > >> > > > wrote: > >> > > > > > > >> > > > > > > Is this on the tip of master? > >> > > > > > > > >> > > > > > > > >> > > > > > > On Wed, Sep 4, 2013 at 10:55 AM, Timothy Chen < > >> [email protected] > >> > > > >> > > > > wrote: > >> > > > > > > > >> > > > > > >> Hi all, > >> > > > > > >> > >> > > > > > >> I'm getting a exception running a simple select * from > >> > > > > > >> "sample-data/region.parquet" order by 1 ; (or any field > >> > actually) > >> > > > > > >> > >> > > > > > >> error_type: 0 > >> > > > > > >> message: "Failure while running fragment. < > >> > > > > IndexOutOfBoundsException:[ > >> > > > > > >> dstIndex: 222 ]" > >> > > > > > >> ] > >> > > > > > >> at > >> > > > org.apache.drill.sql.client.full.ResultEnumerator.moveNext(R > >> > > > > > >> esultEnumerator.java:44) > >> > > > > > >> at > >> > > > net.hydromatic.optiq.runtime.ArrayEnumeratorCursor.next(Arra > >> > > > > > >> yEnumeratorCursor.java:44) > >> > > > > > >> > >> > > > > > >> A bit more digging it looks like it's from the > >> > > RemovingBatchCreator > >> > > > > > trying > >> > > > > > >> to do a copy and getting a index out of bounds in the data > >> > > > > valuevector. > >> > > > > > >> > >> > > > > > >> Jacques suggested someone might be looking into this? Is > >> someone > >> > > > > > lookling > >> > > > > > >> at fixing this arleady? > >> > > > > > >> > >> > > > > > >> Tim > >> > > > > > >> > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > > > >
