The description of the problem seems similar (Kylin doesn't show up the
table/columns in "Query" tab after cube be built) but I don't see the
exception mentioned in my log:
$ grep OLAPSchemaFactory tomcat/logs/*
tomcat/logs/kylin.log:[http-bio-7070-exec-10]:[2015-06-10
01:55:06,407][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicService.java:109)]
- "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
tomcat/logs/kylin.log:[http-bio-7070-exec-2]:[2015-06-10
01:59:21,381][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicService.java:109)]
- "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
tomcat/logs/kylin.log.2015-06-05:[http-bio-7070-exec-3]:[2015-06-05
05:36:33,311][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicService.java:109)]
- "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
tomcat/logs/kylin.log.2015-06-05:[http-bio-7070-exec-6]:[2015-06-09
08:18:56,643][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicService.java:109)]
- "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
tomcat/logs/kylin.log.2015-06-09:[http-bio-7070-exec-6]:[2015-06-09
08:18:56,643][DEBUG][org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicService.java:109)]
- "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
(no Error / Exception related to that)
the bin/find-hive-dependency.sh I use already includes this fix:
hive_lib=`find "$hdp_home/hive/lib" -name '*.jar' ! -name '*calcite*'
-printf '%p:' | sed 's/:$//'`
hive_dependency=${hive_conf_path}:${hive_lib}:${hcatalog}
On Wed, Jun 10, 2015 at 3:18 PM, Shi, Shaofeng <[email protected]> wrote:
> Please check whether this matches your problem:
> https://issues.apache.org/jira/browse/KYLIN-761
>
> If yes then check whether your Kylin installation includes this fix (in
> bin/find-hive-dependency.sh):
>
> https://github.com/apache/incubator-kylin/commit/2439b9b5289d6c1cefe6dd49fc
> 3403b2d1644ff6
>
>
> On 6/10/15, 7:21 PM, "alex schufo" <[email protected]> wrote:
>
> >So like I said in my previous email after copying the jars and restarting
> >Kylin I could see the tables and query them.
> >
> >But yesterday I built a new cube and the tables disappeared for both
> >projects. I restart Kylin and then I could see the tables for both
> >projects
> >and query them.
> >
> >Is there a Jira for this?
> >
> >On Fri, Jun 5, 2015 at 2:34 PM, divye sheth <[email protected]> wrote:
> >
> >> My bad just realised it when compared it with my working copy. This has
> >>to
> >> do something with the loading of jars though they are available in the
> >>lib
> >> but somehow are not loaded. I believe there is already a jira for this.
> >> Will send the link once I get it.
> >> On Jun 5, 2015 18:49, "alex schufo" <[email protected]> wrote:
> >>
> >> > Awesome!
> >> >
> >> > I tried this and it solved my issue. I now the the table / database
> >>and
> >> can
> >> > query the cube with second latency, thanks a lot!
> >> >
> >> > Just one thing, the source and destination are actually reversed, here
> >> are
> >> > the commands:
> >> >
> >> > cp tomcat/webapps/kylin/WEB-INF/lib/calcite-core-${version}.jar
> >> tomcat/lib/
> >> >
> >> > cp tomcat/webapps/kylin/WEB-INF/lib/kylin-query-${version}.jar
> >> tomcat/lib/
> >> >
> >> > On Fri, Jun 5, 2015 at 11:47 AM, divye sheth <[email protected]>
> >> wrote:
> >> >
> >> > > Hi Alex,
> >> > >
> >> > > I had faced the same issue and I believe it was addressed by one of
> >>the
> >> > > developers of Kylin, cant get hold of the link though. Here is the
> >> > solution
> >> > > that was posted there and I was able to get this working.
> >> > >
> >> > > Copy the kylin-query-${version}.jar and calcite-core-${version}.jar
> >>the
> >> > > source and destination are given below:
> >> > > sourcce location: $KYLIN_HOME/tomcat/lib
> >> > > destination: $KYLIN_HOME/tomcat/webapps/kylin/WEB-INF/lib
> >> > >
> >> > > after you have copied please restart the kylin. Let me know if this
> >> > solves
> >> > > your issue.
> >> > >
> >> > > Thanks
> >> > > Divye Sheth
> >> > >
> >> > > On Fri, Jun 5, 2015 at 4:04 PM, alex schufo <[email protected]>
> >> > wrote:
> >> > >
> >> > > > I checked that but there isn't any dependency error, see below the
> >> > > output:
> >> > > >
> >> > > > $ bin/check-env.sh
> >> > > >
> >> > > > KYLIN_HOME is set to
> >> > /home/username/kylin/kylin-0.7.1-incubating-SNAPSHOT
> >> > > >
> >> > > > On Fri, Jun 5, 2015 at 11:16 AM, Shi, Shaofeng <[email protected]>
> >> > wrote:
> >> > > >
> >> > > > > Please run $KYLIN_HOME/bin/find-hive-dependency.sh and then copy
> >> the
> >> > > > > output here; Kylin need add the Hcatalog to MR class path, if
> >>the
> >> > > > Hcatalog
> >> > > > > jar wasn¹t found, the "HCatInputFormat not found² error will be
> >> > > reported;
> >> > > > >
> >> > > > > On 6/5/15, 4:58 PM, "alex schufo" <[email protected]> wrote:
> >> > > > >
> >> > > > > >Hi all,
> >> > > > > >
> >> > > > > >I built a cube on a Hive fact table of about 300 millions rows,
> >> all
> >> > > the
> >> > > > > >steps are FINISHED with success but it seems I cannot do any
> >> query.
> >> > > > > >
> >> > > > > >select * from myFactTable; (or any other query)
> >> > > > > >
> >> > > > > >returns From line 1, column 15 to line 1, column 43: Table
> >> > > 'MYFACTTABLE'
> >> > > > > >not found while executing SQL: "select * from myFactTable LIMIT
> >> > 50000"
> >> > > > > >
> >> > > > > >I tried rebuilding completely the cube after a purge, created a
> >> new
> >> > > > > >project, etc. but it doesn't solve the issue.
> >> > > > > >
> >> > > > > >It seems like a new KYLIN_ table has been created and contains
> >> data.
> >> > > How
> >> > > > > >can I know which HBase table corresponds to which cube?
> >> > > > > >
> >> > > > > >The "Convert Cuboid Data to HFile" step took a while, the
> >> MapReduce
> >> > > job
> >> > > > > >stayed in Undefined status for a long time but finally
> >>Succeeded.
> >> > > When I
> >> > > > > >click on the link from the Kylin UI it send me to
> >>7070/kylin/N/A
> >> > > instead
> >> > > > > >of
> >> > > > > >the MapReduce job info, is that a bad sign?
> >> > > > > >
> >> > > > > >In the logs I don't see any error in the Build N-Dimension
> >>Cuboid
> >> > > Data,
> >> > > > > >Calculate HTable Region Splits, Create HTable
> >> > > > > >
> >> > > > > >I am not sure if it's event related but in the logs can this:
> >> > > > > >
> >> > > > > >2015-06-04 10:17:10,910 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 1%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >2015-06-04 10:17:11,936 INFO [pool-7-thread-10] mapreduce.Job:
> >> Task
> >> > > Id
> >> > > > :
> >> > > > > >attempt_1432057232815_49758_m_000003_2, Status : FAILED
> >> > > > > >
> >> > > > > >Error: java.lang.RuntimeException:
> >> java.lang.ClassNotFoundException:
> >> > > > Class
> >> > > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> >> > > > > >
> >> > > > > > at
> >> > > > >
> >> >
> >>>org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1961)
> >> > > > > >
> >> > > > > > at
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>>org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobC
> >>>on
> >> > > > > >textImpl.java:174)
> >> > > > > >
> >> > > > > > at
> >> > > > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
> >> > > > > >
> >> > > > > > at
> >>org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> >> > > > > >
> >> > > > > > at
> >> > > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> >> > > > > >
> >> > > > > > at java.security.AccessController.doPrivileged(Native
> >> > Method)
> >> > > > > >
> >> > > > > > at javax.security.auth.Subject.doAs(Subject.java:415)
> >> > > > > >
> >> > > > > > at
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformatio
> >>>n.
> >> > > > > >java:1594)
> >> > > > > >
> >> > > > > > at
> >> > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> >> > > > > >
> >> > > > > >Caused by: java.lang.ClassNotFoundException: Class
> >> > > > > >org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
> >> > > > > >
> >> > > > > > at
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>>org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1
> >>>86
> >> > > > > >7)
> >> > > > > >
> >> > > > > > at
> >> > > > >
> >> >
> >>>org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1959)
> >> > > > > >
> >> > > > > > ... 8 more
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >but then it seems to go on anyway:
> >> > > > > >
> >> > > > > >2015-06-04 10:17:15,127 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 2%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >2015-06-04 10:17:18,174 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 3%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >2015-06-04 10:17:20,202 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 4%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >2015-06-04 10:17:22,230 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 5%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >2015-06-04 10:17:23,244 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > 6%
> >> > > > > >reduce 0%
> >> > > > > >
> >> > > > > >...
> >> > > > > >
> >> > > > > >2015-06-04 10:19:33,458 INFO [pool-7-thread-10] mapreduce.Job:
> >> map
> >> > > > 100%
> >> > > > > >reduce 100%
> >> > > > > >
> >> > > > > >2015-06-04 10:19:33,481 INFO [pool-7-thread-10] mapreduce.Job:
> >> Job
> >> > > > > >job_1432057232815_49758 completed successfully
> >> > > > > >
> >> > > > > >
> >> > > > > >This comes after
> >> > > > > >
> >> > > > > >
> >> > > > > >Hive Column Cardinality calculation for table
> >> > > > > >
> >> > > > > >and a step called :
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>>+-----------------------------------------------------------------------
> >>>--
> >> > > > > >-----------------------------+
> >> > > > > >
> >> > > > > >| null
> >> > > > > > |
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>>+-----------------------------------------------------------------------
> >>>--
> >> > > > > >-----------------------------+
> >> > > > > >
> >> > > > > >
> >> > > > > >Thanks for your help, I would really like to make this PoC
> >>work!
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
>
