hi shaofeng, one more question, suppose that we have SQL like: select *
from t1 where col1 in {A} group by..
set {A} is a subset of set {C}, say {B} = {C} - {A}, then {B} is
the difference set, and suppose that |A| > |B|, |A| indicates
A's cardinality, then tests show the performance of select * from t1 where
col1 NOT IN {B} group by.. is much better that the performance of select *
from t1 where col1 IN {A} group by.. especially when |A| is much greater
than |B|, so, could anyone please have a explain?
|
2015-03-16 16:11 GMT+08:00 Shi, Shaofeng <[email protected]>:
> Awesome! Thanks for the sharing, we will compose these tips into wiki for
> all Kylin users;
>
> On 3/16/15, 4:03 PM, "dong wang" <[email protected]> wrote:
>
> >it works, thanks~
> >
> >adding more, if someone deploys kylin on linux server but wants to debug
> >kylin from Windows' source code in eclipse, one can just add the following
> >codes in kylin.sh, and then start eclipse java remote debug feature~
> >
> >if [ -z "$JPDA_TRANSPORT" ]; then
> > JPDA_TRANSPORT="dt_socket"
> >fi
> >if [ -z "$JPDA_ADDRESS" ]; then
> > JPDA_ADDRESS="8084" #remote debug port
> >fi
> >if [ -z "$JPDA_SUSPEND" ]; then
> > JPDA_SUSPEND="n"
> >fi
> >if [ -z "$JPDA_OPTS" ]; then
> >
> >JPDA_OPTS="-agentlib:jdwp=transport=$JPDA_TRANSPORT,address=$JPDA_ADDRESS,
> >server=y,suspend=$JPDA_SUSPEND"
> >fi
> >
> >export JAVA_OPTS="-Xms2048M -Xmx2048M -Xss4M"
> >
> >
> >hbase ${JAVA_OPTS} $JPDA_OPTS \
> >-Djava.util.logging.config.file=${tomcat_root}/conf/logging.properties \
> >
> >2015-03-16 15:43 GMT+08:00 Shi, Shaofeng <[email protected]>:
> >
> >> Seems the JVM default stack size couldn¹t fulfill your need; You can try
> >> to give a bigger stack size when start Kylin:
> >>
> >> in kylin.sh:
> >>
> >> export JAVA_OPTS="-Xms2048M -Xmx2048M -Xss4M"
> >>
> >> hbase ${JAVA_OPTS}
> >> -Djava.util.logging.config.file=${CATALINA_HOME}/conf/logging.properties
> >> -Djava.library.path=${KYLIN_LD_LIBRARY_PATH}
> >> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> >> -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
> >> -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true
> >> -Dspring.profiles.active=sandbox
> >> -Djava.endorsed.dirs=${CATALINA_HOME}/endorsed
> >> -Dcatalina.base=${CATALINA_HOME} -Dcatalina.home=${CATALINA_HOME}
> >> -Djava.io.tmpdir=${CATALINA_HOME}/temp org.apache.hadoop.util.RunJar
> >> ${CATALINA_HOME}/bin/bootstrap.jar
> >>org.apache.catalina.startup.Bootstrap
> >> start > ${CATALINA_HOME}/logs/kylin.log 2>&1 &
> >>
> >>
> >> Please let us know whether it can solve the problem, so that it can help
> >> other guys;
> >>
> >> On 3/16/15, 3:25 PM, "dong wang" <[email protected]> wrote:
> >>
> >> >when there are about 3500 values in the "select ...from.. where
> >>..in(v1,
> >> >v2, ...v3500)" syntax, it throw the error below, I'm debugging it now,
> >> >since the problem is urgent, thus, hope that someone can also do a
> >>help
> >> >to
> >> >take a look?
> >> >
> >> >in addition, if the only a few values(for example, 10 values, and
> >>etc), it
> >> >can return correct result!
> >> >
> >> >[[http-bio-8080-exec-7]:[2015-03-16
> >>
> >>>15:17:19,301][INFO][org.apache.kylin.query.routing.RoutingRule.applyRule
> >>>s(
> >> >RoutingRule.java:60)]
> >> >- ===================================================
> >> >[http-bio-8080-exec-7]:[2015-03-16
> >>
> >>>15:17:19,301][INFO][org.apache.kylin.query.routing.QueryRouter.selectRea
> >>>li
> >> >zation(QueryRouter.java:54)]
> >> >- The realizations remaining:
> >> >[http-bio-8080-exec-7]:[2015-03-16
> >>
> >>>15:17:19,301][INFO][org.apache.kylin.query.routing.QueryRouter.selectRea
> >>>li
> >> >zation(QueryRouter.java:55)]
> >> >- [table1]
> >> >[http-bio-8080-exec-7]:[2015-03-16
> >>
> >>>15:17:19,301][INFO][org.apache.kylin.query.routing.QueryRouter.selectRea
> >>>li
> >> >zation(QueryRouter.java:56)]
> >> >- The realization being chosen: table1
> >> >[http-bio-8080-exec-7]:[2015-03-16
> >>
> >>>15:17:22,589][ERROR][
> org.apache.kylin.rest.controller.QueryController.do
> >>>Qu
> >> >ery(QueryController.java:226)]
> >> >- Exception when execute sql
> >> >java.lang.StackOverflowError
> >> > at
> >>
> >>>org.codehaus.janino.UnitCompiler.getConstantValue2(UnitCompiler.java:445
> >>>7)
> >> > at
> >>org.codehaus.janino.UnitCompiler.access$8900(UnitCompiler.java:183)
> >> > at
> >>
> >>>org.codehaus.janino.UnitCompiler$11.visitBinaryOperation(UnitCompiler.ja
> >>>va
> >> >:4371)
> >> > at org.codehaus.janino.Java$BinaryOperation.accept(Java.java:3768)
> >>
> >>
>
>
