Created DRILL-215 and DRILL-216 for both exceptions. Tim
On Thu, Sep 5, 2013 at 4:04 PM, Jacques Nadeau <[email protected]> wrote: > Can you file one or more bugs for this with more details? We need to > figure out if we should include it in m1. > > > On Wed, Sep 4, 2013 at 3:39 PM, Timothy Chen <[email protected]> wrote: > > > Actually picking N_NAME field works for me: > > > > select _MAP['N_NAME'] from "sample-data/nation.parquet" order by > > _MAP['N_NAME'] desc; > > > > Running Ted's query on REGION_KEY gives me this error instead: > > > > at > org.apache.drill.exec.expr.TypeHelper.getNewVector(TypeHelper.java:504) > > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > > at > > > > > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:102) > > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > > at > > > > > org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:36) > > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > > at > > > org.apache.drill.exec.physical.impl.sort.SortBatch.next(SortBatch.java:86) > > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > > at > > > > > org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:25) > > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > > at > > > > > org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77) > > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > > at org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79) > > ~[java-exec-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] > > at > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > ~[na:1.7.0_25] > > at > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > ~[na:1.7.0_25] > > at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25] > > > > Which is another problem as you can see. > > > > Tim > > > > > > On Wed, Sep 4, 2013 at 3:33 PM, Timothy Chen <[email protected]> wrote: > > > > > Hi Jacques, > > > > > > Do you know what query with order by will work using the sample data? > I'm > > > trying to find one that can actually work. > > > > > > Tim > > > > > > > > > On Wed, Sep 4, 2013 at 1:59 PM, Jacques Nadeau <[email protected]> > > wrote: > > > > > >> In the plan here, the problem is this: > > >> > > >> { order : "asc", expr : "_MAP", nullCollation : "NULLS_LAST" } > > >> > > >> Two issues: there is no such thing as ordering by a map. > Additionally, > > we > > >> don't support direct map references yet. > > >> > > >> What query were these plans for? We need to make sure everybody has > the > > >> same expectation of output. > > >> > > >> J > > >> > > >> > > >> On Wed, Sep 4, 2013 at 1:03 PM, Timothy Chen <[email protected]> > wrote: > > >> > > >> > Just as Ted pointed out I tried just columns and ordering by > different > > >> > fields and none of them work. > > >> > > > >> > It fails on the tip of master too. > > >> > > > >> > Logical plan: > > >> > > > >> > Logical { head : { type : "APACHE_DRILL_LOGICAL", version : 1, > > >> generator : > > >> > { type : "optiq", info : "na" } }, storage : { parquet-local : { > type > > : > > >> > "parquet", dfsName : "file:///" } }, query : [ { op : "scan", @id : > 1, > > >> memo > > >> > : "initial_scan", storageengine : "parquet-local", selection : [ { > > path > > >> : > > >> > "sample-data/region.parquet" } ], ref : "_MAP" }, { op : "order", > @id > > : > > >> 2, > > >> > input : 1, within : null, orderings : [ { order : "asc", expr : > > "_MAP", > > >> > nullCollation : "NULLS_LAST" } ] }, { op : "store", @id : 3, memo : > > >> "output > > >> > sink", input : 2, target : { number : 0 }, partition : null, > > >> storageEngine > > >> > : "queue" } ] } > > >> > > > >> > Physical plan: > > >> > > > >> > Physical { head : { type : "APACHE_DRILL_PHYSICAL", version : 1, > > >> generator > > >> > : { type : "optiq", info : "na" } }, graph : [ { pop : > "parquet-scan", > > >> @id > > >> > : 1, entries : [ { path : "sample-data/region.parquet" } ], ref : > > >> "_MAP", > > >> > fragmentPointer : 0 }, { pop : "sort", @id : 2, child : 1, > orderings : > > >> [ { > > >> > order : "asc", expr : "_MAP" } ], reverse : false }, { pop : > > >> > "selection-vector-remover", @id : 3, child : 2 }, { pop : "screen", > > @id > > >> : > > >> > 4, child : 3 } ] } > > >> > > > >> > Tim > > >> > > > >> > > > >> > On Wed, Sep 4, 2013 at 12:42 PM, Jacques Nadeau <[email protected] > > > > >> > wrote: > > >> > > > >> > > Good point. Do you want to take a look at the plans and see what > > are > > >> > pinch > > >> > > point is? > > >> > > > > >> > > J > > >> > > On Sep 4, 2013 12:03 PM, "Ted Dunning" <[email protected]> > > wrote: > > >> > > > > >> > > > Hmm... I don't understand the limitation. > > >> > > > > > >> > > > I also see these: > > >> > > > > > >> > > > *fails*: > > >> > > > > > >> > > > select _MAP['REGION_KEY'] from "sample-data/nation.parquet" > > >> > > > order by cast( _MAP['NATION_KEY'] as INTEGER); > > >> > > > > > >> > > > > > >> > > > *fails*: select _MAP['REGION_KEY'] from > > "sample-data/nation.parquet" > > >> > > order > > >> > > > by _MAP['NATION_KEY']; > > >> > > > > > >> > > > > > >> > > > *fails: *select _MAP['REGION_KEY'] from > > "sample-data/nation.parquet" > > >> > > order > > >> > > > by 1; > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > On Wed, Sep 4, 2013 at 11:53 AM, Jacques Nadeau < > > [email protected] > > >> > > > >> > > > wrote: > > >> > > > > > >> > > > > Two problems in this query. > > >> > > > > > > >> > > > > First is you're sorting by a map field. We don't currently > > direct > > >> > map > > >> > > > > field expressions. You need to qualify to a particular scalar > > or > > >> > > > repeated > > >> > > > > field. > > >> > > > > > > >> > > > > The second problem is that the current code requires > resolution > > >> of a > > >> > > > field > > >> > > > > rather than using * when you're trying to reference it > > elsewhere. > > >> > > > > > > >> > > > > therefore: > > >> > > > > not supported: select * from "sample-data/region.parquet" > order > > >> by 1 > > >> > ; > > >> > > > (map > > >> > > > > keys are not currently referenceable and that is what this > > >> currently > > >> > > > means) > > >> > > > > not supported yet: select * from "sample-data/region.parquet" > > >> order > > >> > by > > >> > > > > _MAP['R_REGIONKEY']; (known issue we should fix in m2 or m3) > > >> > > > > supported: select _MAP['R_REGIONKEY'] from > > >> > "sample-data/region.parquet" > > >> > > > > order by _MAP['R_REGIONKEY']; > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > On Wed, Sep 4, 2013 at 11:39 AM, Timothy Chen < > > [email protected]> > > >> > > wrote: > > >> > > > > > > >> > > > > > Hi Jacques, > > >> > > > > > > > >> > > > > > It i rebased on the top of latest master. > > >> > > > > > > > >> > > > > > I am using a newer optiq release (0.4.11) so not sure if > > master > > >> has > > >> > > > that > > >> > > > > > problem. > > >> > > > > > > > >> > > > > > I will try later once I get a chance. > > >> > > > > > > > >> > > > > > Tim > > >> > > > > > > > >> > > > > > Sent from my iPhone > > >> > > > > > > > >> > > > > > On Sep 4, 2013, at 11:04 AM, Jacques Nadeau < > > [email protected] > > >> > > > >> > > > wrote: > > >> > > > > > > > >> > > > > > > Is this on the tip of master? > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > On Wed, Sep 4, 2013 at 10:55 AM, Timothy Chen < > > >> [email protected] > > >> > > > > >> > > > > wrote: > > >> > > > > > > > > >> > > > > > >> Hi all, > > >> > > > > > >> > > >> > > > > > >> I'm getting a exception running a simple select * from > > >> > > > > > >> "sample-data/region.parquet" order by 1 ; (or any field > > >> > actually) > > >> > > > > > >> > > >> > > > > > >> error_type: 0 > > >> > > > > > >> message: "Failure while running fragment. < > > >> > > > > IndexOutOfBoundsException:[ > > >> > > > > > >> dstIndex: 222 ]" > > >> > > > > > >> ] > > >> > > > > > >> at > > >> > > > org.apache.drill.sql.client.full.ResultEnumerator.moveNext(R > > >> > > > > > >> esultEnumerator.java:44) > > >> > > > > > >> at > > >> > > > net.hydromatic.optiq.runtime.ArrayEnumeratorCursor.next(Arra > > >> > > > > > >> yEnumeratorCursor.java:44) > > >> > > > > > >> > > >> > > > > > >> A bit more digging it looks like it's from the > > >> > > RemovingBatchCreator > > >> > > > > > trying > > >> > > > > > >> to do a copy and getting a index out of bounds in the > data > > >> > > > > valuevector. > > >> > > > > > >> > > >> > > > > > >> Jacques suggested someone might be looking into this? Is > > >> someone > > >> > > > > > lookling > > >> > > > > > >> at fixing this arleady? > > >> > > > > > >> > > >> > > > > > >> Tim > > >> > > > > > >> > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > >
