Hi Qiang, were you seeing this same issue with the prior HBase version also ? (I would think this is not a regression). It would be best to create a new JIRA and attach the EXPLAIN plans for the successful and failed runs. With more nodes some minor fragments of the hash join may be getting empty input batches and I am guessing that has something to do with the SchemaChangeException. Someone would need to debug once you create the JIRA with relevant details.
-Aman On Mon, Jun 20, 2016 at 5:13 AM, qiang li <[email protected]> wrote: > Thanks Aditya. > > By the way, I found another issue. > > Let say I have two tables. > > offers_ref0 : rowkey salt(1byte)+long uid(8 byte ) , family: v, qualifier: > v(string) > offers_nation_idx: rowkey salt(1byte) + string, family:v, qualifier: v(long > 8 byte) > > there is the SQL: > > select CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') as uid, > convert_from(`ref0`.`v`.`v`,'UTF8') as v from hbase.`offers_nation_idx` as > `nation` join hbase.offers_ref0 as `ref0` on > CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') = > CONVERT_FROM(nation.`v`.`v`,'BIGINT_BE') where `nation`.row_key > '0br' > and `nation`.row_key < '0bs' limit 10 > > When I execute the query with single node or less than 5 nodes, its working > good. But when I execute it in cluster which have about 14 nodes, its throw > a exception: > > First time will throw this exception: > *Caused by: java.sql.SQLException: SYSTEM ERROR: SchemaChangeException: > Hash join does not support schema changes* > > Then if I query again, it will always throw below exception: > *Query Failed: An Error Occurred* > *org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: > IllegalStateException: Failure while reading vector. Expected vector class > of org.apache.drill.exec.vector.NullableIntVector but was holding vector > class org.apache.drill.exec.vector.complex.MapVector, field= > v(MAP:REQUIRED)[v(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), > v(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]] Fragment 12:4 [Error Id: > 06c6eae4-0822-4714-b0bf-a6e04ebfec79 on xxx:31010]* > > Its very strange, and I do not know how to solve it. > I tried add node to the cluster one by one, it will reproduce when I added > 5 nodes. Can anyone help me solve this issue? > > > > > 2016-06-17 4:39 GMT+08:00 Aditya <[email protected]>: > > > https://issues.apache.org/jira/browse/DRILL-4727 > > > > On Thu, Jun 16, 2016 at 11:39 AM, Aman Sinha <[email protected]> > wrote: > > > >> Qiang/Aditya can you create a JIRA for this and mark it for 1.7. > thanks. > >> > >> On Thu, Jun 16, 2016 at 11:25 AM, Aditya <[email protected]> > wrote: > >> > >> > Thanks for reporting, I'm looking into it and will post a patch soon. > >> > > >> > On Wed, Jun 15, 2016 at 7:27 PM, qiang li <[email protected]> > wrote: > >> > > >> > > Hi Aditya, > >> > > > >> > > I tested the latest version and got this exception and the drillbit > >> fail > >> > > to startup . > >> > > > >> > > Exception in thread "main" java.lang.NoSuchMethodError: > >> > > io.netty.util.UniqueName.<init>(Ljava/lang/String;)V > >> > > at > >> io.netty.channel.ChannelOption.<init>(ChannelOption.java:136) > >> > > at > >> io.netty.channel.ChannelOption.valueOf(ChannelOption.java:99) > >> > > at > >> io.netty.channel.ChannelOption.<clinit>(ChannelOption.java:42) > >> > > at > >> > > org.apache.drill.exec.rpc.BasicServer.<init>(BasicServer.java:63) > >> > > at > >> > > org.apache.drill.exec.rpc.user.UserServer.<init>(UserServer.java:74) > >> > > at > >> > > > >> > org.apache.drill.exec.service.ServiceEngine.<init>(ServiceEngine.java:78) > >> > > at > >> > org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:108) > >> > > at > >> org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285) > >> > > at > >> org.apache.drill.exec.server.Drillbit.start(Drillbit.java:271) > >> > > at > >> org.apache.drill.exec.server.Drillbit.main(Drillbit.java:267) > >> > > > >> > > It will working if I remove > jars/3rdparty/netty-all-4.0.23.Final.jar, > >> the > >> > > drill can startup. I think there have some package dependency > version > >> > > issue, do you think so ? > >> > > > >> > > > >> > > > >> > > 2016-06-15 8:14 GMT+08:00 Aditya <[email protected]>: > >> > > > >> > >> HBase 1.x support has been merged and is available in latest > >> > >> 1.7.0-SNAPSHOT > >> > >> builds. > >> > >> > >> > >> On Wed, Jun 1, 2016 at 1:23 PM, Aditya <[email protected]> > >> wrote: > >> > >> > >> > >> > Thanks Jacques for promptly reviewing my long series of patches! > >> > >> > > >> > >> > I'm planning to merge the HBase 1.x support some time in next 48 > >> > hours. > >> > >> > > >> > >> > If anyone else is interested and willing, please review the > latest > >> > patch > >> > >> > here[1]. > >> > >> > > >> > >> > aditya... > >> > >> > > >> > >> > [1] https://github.com/apache/drill/pull/443/files > >> > >> > > >> > >> > >> > > > >> > > > >> > > >> > > > > >
