Hi,
I am trying to test kylin by using as hadoop CLI "BigInsights IBM Open
Platform with Apache Hadoop 4.0".
Environment:
* kylin-0.7.1-incubating-SNAPSHOT
* BigInsights IBM Open Platform with Apache Hadoop 4.0
* KYLIN_HOME=/opt/apache/kylin/kylin-0.7.1-incubating-SNAPSHOT ==>
Directory where I put kylin
* $ java -version
java version "1.7.0_75"
OpenJDK Runtime Environment (rhel-2.5.4.0.el6_6-x86_64 u75-b13)
OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)
NOTE: IBM Open Platform with Apache Hadoop 4.0 works with the
openJDK. Anyway, I tried also with SUN JDK but result is the same
* KYLIN_CONF=/etc/kylin ==> I set also
KYLIN_EXTRA_START_OPTS="-DKYLIN_CONF=/etc/kylin" end copied the cfg
files also in /etc/kylin
* Both hadoop CLI and kylin run on the same server
Kylin dosen's start properly and I see the following log related to
problems with ldap.
[localhost-startStop-1]:[2015-06-12
17:25:25,827][INFO][org.apache.kylin.common.KylinConfig.getKylinProperties(KylinConfig.java:544)]
- Use KYLIN_CONF=/etc/kylin
[localhost-startStop-1]:[2015-06-12
17:25:25,863][INFO][org.springframework.core.io.support.PropertiesLoaderSupport.loadProperties(PropertiesLoaderSupport.java:177)]
- Loading properties file from resource loaded through InputStream
[localhost-startStop-1]:[2015-06-12
17:25:25,905][ERROR][org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:307)]
- Context initialization failed
org.springframework.beans.factory.BeanDefinitionStoreException: Invalid
bean definition with name 'ldapSource' defined in class path resource
[kylinSecurity.xml]: Input length must be multiple of 16 when decrypting
with padded cipher
at
org.springframework.beans.factory.config.PlaceholderConfigurerSupport.doProcessProperties(PlaceholderConfigurerSupport.java:209)
at
org.springframework.beans.factory.config.PropertyPlaceholderConfigurer.processProperties(PropertyPlaceholderConfigurer.java:220)
at
org.springframework.beans.factory.config.PropertyResourceConfigurer.postProcessBeanFactory(PropertyResourceConfigurer.java:84)
at
org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:686)
at
org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:661)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:451)
at
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
at
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
at
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5016)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5524)
at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1081)
at
org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1877)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jun 12, 2015 5:25:25 PM org.apache.catalina.core.StandardContext
startInternal
SEVERE: Error listenerStart
Jun 12, 2015 5:25:25 PM org.apache.catalina.core.StandardContext
startInternal
SEVERE: Context [/kylin] startup failed due to previous errors
Jun 12, 2015 5:25:25 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deployment of web application archive
/opt/apache/kylin/kylin-0.7.1-incubating-SNAPSHOT/tomcat/webapps/kylin.war
has finished in 11,533 ms
Jun 12, 2015 5:25:25 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-7070"]
Jun 12, 2015 5:25:25 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-9009"]
Jun 12, 2015 5:25:25 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 11574 ms
rceConfigurer.java:84)
19,1 48%
Is it possible to disable LDAP usage ?
Could you please provide some feedbacks ?
In attach the log file and the cfg files.
Thanks in advance.
BR,
-- gas
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
## Config for Kylin Engine ##
# List of web servers in use, this enables one web server instance to sync up
with other servers.
kylin.rest.servers=localhost:7070
# The metadata store in hbase
kylin.metadata.url=kylin_metadata@hbase
# The storage for final cube file in hbase
kylin.storage.url=hbase
# Temp folder in hdfs
kylin.hdfs.working.dir=/tmp
kylin.job.mapreduce.default.reduce.input.mb=500
# If true, job engine will not assume that hadoop CLI reside on the same server
as it self
# you will have to specify kylin.job.remote.cli.hostname,
kylin.job.remote.cli.username and kylin.job.remote.cli.password
#kylin.job.run.as.remote.cmd=false
kylin.job.run.as.remote.cmd=true
# Only necessary when kylin.job.run.as.remote.cmd=true
kylin.job.remote.cli.hostname=gas
# Only necessary when kylin.job.run.as.remote.cmd=true
kylin.job.remote.cli.username=hdfs
# Only necessary when kylin.job.run.as.remote.cmd=true
kylin.job.remote.cli.password=hdfs
# Used by test cases to prepare synthetic data for sample cube
kylin.job.remote.cli.working.dir=/var/opt/data/apache/kylin
# Max count of concurrent jobs running
kylin.job.concurrent.max.limit=10
# Time interval to check hadoop job status
kylin.job.yarn.app.rest.check.interval.seconds=10
## Config for Restful APP ##
# database connection settings:
ldap.server=ldap://gas:33389
ldap.username=tom
ldap.password=tom-password
ldap.user.searchBase=
ldap.user.searchPattern=
ldap.user.groupSearchBase=
ldap.service.searchBase=OU=
ldap.service.searchPattern=
ldap.service.groupSearchBase=
acl.adminRole=
acl.defaultRole=
ganglia.group=
ganglia.port=8664
## Config for mail service
# If true, will send email notification;
mail.enabled=false
mail.host=
mail.username=
mail.password=
mail.sender=
###########################config info for web#######################
#help info ,format{name|displayName|link} ,optional
kylin.web.help.length=4
kylin.web.help.0=start|Getting Started|
kylin.web.help.1=odbc|ODBC Driver|
kylin.web.help.2=tableau|Tableau Guide|
kylin.web.help.3=onboard|Cube Design Tutorial|
#hadoop url link ,optional
kylin.web.hadoop=
#job diagnostic url link ,optional
kylin.web.diagnostic=
#contact mail on web page ,optional
kylin.web.contact_mail=
###########################config info for front#######################
#env DEV|QA|PROD
deploy.env=DEV
###########################config info for sandbox#######################
kylin.sandbox=true
#kylin.sandbox=false
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<property>
<name>mapreduce.job.split.metainfo.maxsize</name>
<value>-1</value>
<description>The maximum permissible size of the split metainfo file.
The JobTracker won't attempt to read split metainfo files bigger than
the configured value. No limits if set to -1.
</description>
</property>
<!-- uncomment the following 5 properties to enable lzo compressing
<property>
<name>mapred.compress.map.output</name>
<value>true</value>
<description>Compress map outputs</description>
</property>
<property>
<name>mapred.map.output.compression.codec</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
<description>The compression codec to use for map outputs
</description>
</property>
<property>
<name>mapred.output.compress</name>
<value>true</value>
<description>Compress the output of a MapReduce job</description>
</property>
<property>
<name>mapred.output.compression.codec</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
<description>The compression codec to use for job outputs
</description>
</property>
<property>
<name>mapred.output.compression.type</name>
<value>BLOCK</value>
<description>The compression type to use for job outputs</description>
</property>
!-->
<property>
<name>mapreduce.job.max.split.locations</name>
<value>2000</value>
<description>No description</description>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
<description>Block replication</description>
</property>
</configuration>usage: java org.apache.catalina.startup.Catalina [ -config {pathname} ] [ -nonaming ] { -help | start | stop }
Jun 12, 2015 5:25:14 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: :/usr/iop/current/hadoop-client/lib/native/Linux-amd64-64:/usr/iop/4.0.0.0/hadoop/lib/native
Jun 12, 2015 5:25:14 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-7070"]
Jun 12, 2015 5:25:14 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-9009"]
Jun 12, 2015 5:25:14 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 632 ms
Jun 12, 2015 5:25:14 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Jun 12, 2015 5:25:14 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.59
Jun 12, 2015 5:25:14 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive /opt/apache/kylin/kylin-0.7.1-incubating-SNAPSHOT/tomcat/webapps/kylin.war
Jun 12, 2015 5:25:14 PM org.apache.catalina.loader.WebappClassLoader validateJarFile
INFO: validateJarFile(/opt/apache/kylin/kylin-0.7.1-incubating-SNAPSHOT/tomcat/webapps/kylin/WEB-INF/lib/servlet-api-2.5.jar) - jar not loaded. See Servlet Spec 3.0, section 10.7.2. Offending class: javax/servlet/Servlet.class
Jun 12, 2015 5:25:24 PM org.apache.catalina.startup.TaglibUriRule body
INFO: TLD skipped. URI: urn:com:sun:jersey:api:view is already defined
Jun 12, 2015 5:25:24 PM org.apache.catalina.startup.TaglibUriRule body
INFO: TLD skipped. URI: urn:com:sun:jersey:api:view is already defined
Jun 12, 2015 5:25:24 PM org.apache.catalina.startup.TaglibUriRule body
INFO: TLD skipped. URI: urn:com:sun:jersey:api:view is already defined
Jun 12, 2015 5:25:24 PM org.apache.catalina.startup.TaglibUriRule body
INFO: TLD skipped. URI: urn:com:sun:jersey:api:view is already defined
Jun 12, 2015 5:25:24 PM org.apache.catalina.startup.TaglibUriRule body
INFO: TLD skipped. URI: urn:com:sun:jersey:api:view is already defined
Jun 12, 2015 5:25:24 PM org.apache.catalina.startup.TaglibUriRule body
INFO: TLD skipped. URI: urn:com:sun:jersey:api:view is already defined
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/iop/4.0.0.0/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/iop/4.0.0.0/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/iop/4.0.0.0/hive/lib/hive-jdbc-0.14.0_IBM_8-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/apache/kylin/kylin-0.7.1-incubating-SNAPSHOT/tomcat/webapps/kylin/WEB-INF/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
[localhost-startStop-1]:[2015-06-12 17:25:25,827][INFO][org.apache.kylin.common.KylinConfig.getKylinProperties(KylinConfig.java:544)] - Use KYLIN_CONF=/etc/kylin
[localhost-startStop-1]:[2015-06-12 17:25:25,863][INFO][org.springframework.core.io.support.PropertiesLoaderSupport.loadProperties(PropertiesLoaderSupport.java:177)] - Loading properties file from resource loaded through InputStream
[localhost-startStop-1]:[2015-06-12 17:25:25,905][ERROR][org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:307)] - Context initialization failed
org.springframework.beans.factory.BeanDefinitionStoreException: Invalid bean definition with name 'ldapSource' defined in class path resource [kylinSecurity.xml]: Input length must be multiple of 16 when decrypting with padded cipher
at org.springframework.beans.factory.config.PlaceholderConfigurerSupport.doProcessProperties(PlaceholderConfigurerSupport.java:209)
at org.springframework.beans.factory.config.PropertyPlaceholderConfigurer.processProperties(PropertyPlaceholderConfigurer.java:220)
at org.springframework.beans.factory.config.PropertyResourceConfigurer.postProcessBeanFactory(PropertyResourceConfigurer.java:84)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:686)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:661)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:451)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5016)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5524)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1081)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1877)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jun 12, 2015 5:25:25 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Error listenerStart
Jun 12, 2015 5:25:25 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [/kylin] startup failed due to previous errors
Jun 12, 2015 5:25:25 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deployment of web application archive /opt/apache/kylin/kylin-0.7.1-incubating-SNAPSHOT/tomcat/webapps/kylin.war has finished in 11,533 ms
Jun 12, 2015 5:25:25 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-7070"]
Jun 12, 2015 5:25:25 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-9009"]
Jun 12, 2015 5:25:25 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 11574 ms
rceConfigurer.java:84)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:686)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:661)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:451)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5016)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5524)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1081)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1877)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
