Hi Jacques, Do you know what query with order by will work using the sample data? I'm trying to find one that can actually work.
Tim On Wed, Sep 4, 2013 at 1:59 PM, Jacques Nadeau <[email protected]> wrote: > In the plan here, the problem is this: > > { order : "asc", expr : "_MAP", nullCollation : "NULLS_LAST" } > > Two issues: there is no such thing as ordering by a map. Additionally, we > don't support direct map references yet. > > What query were these plans for? We need to make sure everybody has the > same expectation of output. > > J > > > On Wed, Sep 4, 2013 at 1:03 PM, Timothy Chen <[email protected]> wrote: > > > Just as Ted pointed out I tried just columns and ordering by different > > fields and none of them work. > > > > It fails on the tip of master too. > > > > Logical plan: > > > > Logical { head : { type : "APACHE_DRILL_LOGICAL", version : 1, generator > : > > { type : "optiq", info : "na" } }, storage : { parquet-local : { type : > > "parquet", dfsName : "file:///" } }, query : [ { op : "scan", @id : 1, > memo > > : "initial_scan", storageengine : "parquet-local", selection : [ { path : > > "sample-data/region.parquet" } ], ref : "_MAP" }, { op : "order", @id : > 2, > > input : 1, within : null, orderings : [ { order : "asc", expr : "_MAP", > > nullCollation : "NULLS_LAST" } ] }, { op : "store", @id : 3, memo : > "output > > sink", input : 2, target : { number : 0 }, partition : null, > storageEngine > > : "queue" } ] } > > > > Physical plan: > > > > Physical { head : { type : "APACHE_DRILL_PHYSICAL", version : 1, > generator > > : { type : "optiq", info : "na" } }, graph : [ { pop : "parquet-scan", > @id > > : 1, entries : [ { path : "sample-data/region.parquet" } ], ref : "_MAP", > > fragmentPointer : 0 }, { pop : "sort", @id : 2, child : 1, orderings : [ > { > > order : "asc", expr : "_MAP" } ], reverse : false }, { pop : > > "selection-vector-remover", @id : 3, child : 2 }, { pop : "screen", @id : > > 4, child : 3 } ] } > > > > Tim > > > > > > On Wed, Sep 4, 2013 at 12:42 PM, Jacques Nadeau <[email protected]> > > wrote: > > > > > Good point. Do you want to take a look at the plans and see what are > > pinch > > > point is? > > > > > > J > > > On Sep 4, 2013 12:03 PM, "Ted Dunning" <[email protected]> wrote: > > > > > > > Hmm... I don't understand the limitation. > > > > > > > > I also see these: > > > > > > > > *fails*: > > > > > > > > select _MAP['REGION_KEY'] from "sample-data/nation.parquet" > > > > order by cast( _MAP['NATION_KEY'] as INTEGER); > > > > > > > > > > > > *fails*: select _MAP['REGION_KEY'] from "sample-data/nation.parquet" > > > order > > > > by _MAP['NATION_KEY']; > > > > > > > > > > > > *fails: *select _MAP['REGION_KEY'] from "sample-data/nation.parquet" > > > order > > > > by 1; > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 4, 2013 at 11:53 AM, Jacques Nadeau <[email protected]> > > > > wrote: > > > > > > > > > Two problems in this query. > > > > > > > > > > First is you're sorting by a map field. We don't currently direct > > map > > > > > field expressions. You need to qualify to a particular scalar or > > > > repeated > > > > > field. > > > > > > > > > > The second problem is that the current code requires resolution of > a > > > > field > > > > > rather than using * when you're trying to reference it elsewhere. > > > > > > > > > > therefore: > > > > > not supported: select * from "sample-data/region.parquet" order by > 1 > > ; > > > > (map > > > > > keys are not currently referenceable and that is what this > currently > > > > means) > > > > > not supported yet: select * from "sample-data/region.parquet" order > > by > > > > > _MAP['R_REGIONKEY']; (known issue we should fix in m2 or m3) > > > > > supported: select _MAP['R_REGIONKEY'] from > > "sample-data/region.parquet" > > > > > order by _MAP['R_REGIONKEY']; > > > > > > > > > > > > > > > > > > > > On Wed, Sep 4, 2013 at 11:39 AM, Timothy Chen <[email protected]> > > > wrote: > > > > > > > > > > > Hi Jacques, > > > > > > > > > > > > It i rebased on the top of latest master. > > > > > > > > > > > > I am using a newer optiq release (0.4.11) so not sure if master > has > > > > that > > > > > > problem. > > > > > > > > > > > > I will try later once I get a chance. > > > > > > > > > > > > Tim > > > > > > > > > > > > Sent from my iPhone > > > > > > > > > > > > On Sep 4, 2013, at 11:04 AM, Jacques Nadeau <[email protected]> > > > > wrote: > > > > > > > > > > > > > Is this on the tip of master? > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 4, 2013 at 10:55 AM, Timothy Chen < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > >> Hi all, > > > > > > >> > > > > > > >> I'm getting a exception running a simple select * from > > > > > > >> "sample-data/region.parquet" order by 1 ; (or any field > > actually) > > > > > > >> > > > > > > >> error_type: 0 > > > > > > >> message: "Failure while running fragment. < > > > > > IndexOutOfBoundsException:[ > > > > > > >> dstIndex: 222 ]" > > > > > > >> ] > > > > > > >> at > > > > org.apache.drill.sql.client.full.ResultEnumerator.moveNext(R > > > > > > >> esultEnumerator.java:44) > > > > > > >> at > > > > net.hydromatic.optiq.runtime.ArrayEnumeratorCursor.next(Arra > > > > > > >> yEnumeratorCursor.java:44) > > > > > > >> > > > > > > >> A bit more digging it looks like it's from the > > > RemovingBatchCreator > > > > > > trying > > > > > > >> to do a copy and getting a index out of bounds in the data > > > > > valuevector. > > > > > > >> > > > > > > >> Jacques suggested someone might be looking into this? Is > someone > > > > > > lookling > > > > > > >> at fixing this arleady? > > > > > > >> > > > > > > >> Tim > > > > > > >> > > > > > > > > > > > > > > > > > > > > >
