I built the jar from the source, as checked, only, there is a file named *.SF under job/target/kylin-job-0.7.1-SNAPSHOT-job.jar, then I tried to update the pom.xml with adding "<exclude>META-INF/*.SF</exclude>" mentioned above under "job" directory for the job.jar, then build again, however, currently, there is stranger problem with few job info:
Task failed task_1425461099463_0005_m_000000 Job failed as tasks failed. failedMaps:1 failedReduces:0 2015-03-04 18:03 GMT+08:00 Zhou, Qianhao <[email protected]>: > @Dong > Are you running kylin from source code or kylin-0.7.1-SNAPSHOT.tgz? > > If from source code, you will see > job/target/kylin-job-0.7.1-SNAPSHOT-job.jar > storage/target/kylin-storage-0.7.1-SNAPSHOT-coprocessor.jar > > If from tgz > lib/kylin-job-0.7.1-SNAPSHOT.jar > lib/kylin-coprocessor-0.7.1-SNAPSHOT.jar > > Please check if there are *.SF *.RSA *.DSA files in these two jars. If > they do exist, as a workaround, you can remove these files from the jar. > Please let us know if it works, and then we will fix it. > > Best Regard > Zhou QianHao > > > > > > On 3/4/15, 5:48 PM, "dong wang" <[email protected]> wrote: > > >I'm not sure whether the problem is similar to: > >http://www.cnblogs.com/fuxinci/p/3356087.html > > > >2015-03-04 17:38 GMT+08:00 dong wang <[email protected]>: > > > >> As checked, it says: > >> > >> Job init failed : java.lang.SecurityException: Invalid signature file > >> digest for Manifest main attributes > >> at > >> > >>sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier > >>.java:286) > >> at > >> > >>sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.jav > >>a:239) > >> at java.util.jar.JarVerifier.processEntry(JarVerifier.java:317) > >> at java.util.jar.JarVerifier.update(JarVerifier.java:228) > >> at java.util.jar.JarFile.initializeVerifier(JarFile.java:348) > >> at java.util.jar.JarFile.getInputStream(JarFile.java:415) > >> at > >>sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:775) > >> at sun.misc.Resource.cachedInputStream(Resource.java:77) > >> at sun.misc.Resource.getByteBuffer(Resource.java:160) > >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:436) > >> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > >> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > >> at java.security.AccessController.doPrivileged(Native Method) > >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > >> at java.lang.Class.forName0(Native Method) > >> at java.lang.Class.forName(Class.java:191) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.isChainJob(JobImpl.ja > >>va:1305) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.makeUberDecision(JobI > >>mpl.java:1247) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.access$3800(JobImpl.j > >>ava:139) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transi > >>tion(JobImpl.java:1451) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transi > >>tion(JobImpl.java:1388) > >> at > >> > >>org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTr > >>ansition(StateMachineFactory.java:385) > >> at > >> > >>org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachin > >>eFactory.java:302) > >> at > >> > >>org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineF > >>actory.java:46) > >> at > >> > >>org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doT > >>ransition(StateMachineFactory.java:448) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:9 > >>96) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:1 > >>38) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle( > >>MRAppMaster.java:1272) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.j > >>ava:1045) > >> at > >> > >>org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:147 > >>8) > >> at java.security.AccessController.doPrivileged(Native Method) > >> at javax.security.auth.Subject.doAs(Subject.java:415) > >> at > >> > >>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation > >>.java:1642) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAp > >>pMaster.java:1474) > >> at > >> > >>org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1407 > >>) > >> > >> 2015-03-04 17:03 GMT+08:00 Zhou, Qianhao <[email protected]>: > >> > >>> @Dong > >>> The map-reduce job may encounter some error, can you please look > >>>into > >>> the job detail. > >>> Just FYI, you can use http://$job history > >>> server$/jobhistory/job/job_1425448727678_0003/ to see what is really > >>>going > >>> on. > >>> > >>> Best Regard > >>> Zhou QianHao > >>> > >>> > >>> > >>> > >>> > >>> On 3/4/15, 3:21 PM, "dong wang" <[email protected]> wrote: > >>> > >>> >more error info: > >>> > > >>> > > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:41,023][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.che > >>>>ckS > >>> >tatus(HadoopStatusChecker.java:57)] > >>> >- State of Hadoop job: job_1425448727678_0003:ACCEPTED-UNDEFINED > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:41,030][DEBUG][org.apache.kylin.common.persistence.ResourceStore. > >>>>put > >>> >Resource(ResourceStore.java:171)] > >>> >- Saving resource > >>>/execute_output/99cbf62c-d0d4-4427-b075-e848cd8d83e9-01 > >>> >(Store kylin_metadata@hbase) > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:51,046][WARN][org.apache.commons.httpclient.HttpMethodBase.getRes > >>>>pon > >>> >seBody(HttpMethodBase.java:682)] > >>> >- Going to buffer response body of large or unknown size. Using > >>> >getResponseBodyAsStream instead is recommended. > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:51,047][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getH > >>>>ttp > >>> >Response(HadoopStatusGetter.java:90)] > >>> >- Job job_1425448727678_0003 get status check result. > >>> > > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:51,047][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.che > >>>>ckS > >>> >tatus(HadoopStatusChecker.java:57)] > >>> >- State of Hadoop job: job_1425448727678_0003:FINISHED-FAILED > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:51,058][DEBUG][org.apache.kylin.common.persistence.ResourceStore. > >>>>put > >>> >Resource(ResourceStore.java:171)] > >>> >- Saving resource > >>>/execute_output/99cbf62c-d0d4-4427-b075-e848cd8d83e9-01 > >>> >(Store kylin_metadata@hbase) > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:51,068][DEBUG][org.apache.kylin.common.persistence.ResourceStore. > >>>>put > >>> >Resource(ResourceStore.java:171)] > >>> >- Saving resource > >>>/execute_output/99cbf62c-d0d4-4427-b075-e848cd8d83e9-01 > >>> >(Store kylin_metadata@hbase) > >>> >2015-03-04 14:19:51,074 INFO [pool-7-thread-3] > >>> >mapred.ClientServiceDelegate: Application state is completed. > >>> >FinalApplicationStatus=FAILED. Redirecting to job history server > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:51,135][WARN][org.apache.kylin.job.common.HadoopCmdOutput.updateJ > >>>>obC > >>> >ounter(HadoopCmdOutput.java:89)] > >>> >- no counters for job job_1425448727678_0003 > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:51,141][DEBUG][org.apache.kylin.common.persistence.ResourceStore. > >>>>put > >>> >Resource(ResourceStore.java:171)] > >>> >- Saving resource > >>>/execute_output/99cbf62c-d0d4-4427-b075-e848cd8d83e9-01 > >>> >(Store kylin_metadata@hbase) > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:51,149][DEBUG][org.apache.kylin.common.persistence.ResourceStore. > >>>>put > >>> >Resource(ResourceStore.java:171)] > >>> >- Saving resource > >>>/execute_output/99cbf62c-d0d4-4427-b075-e848cd8d83e9-01 > >>> >(Store kylin_metadata@hbase) > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:51,162][DEBUG][org.apache.kylin.common.persistence.ResourceStore. > >>>>put > >>> >Resource(ResourceStore.java:171)] > >>> >- Saving resource > >>>/execute_output/99cbf62c-d0d4-4427-b075-e848cd8d83e9-01 > >>> >(Store kylin_metadata@hbase) > >>> >[pool-7-thread-3]:[2015-03-04 > >>> > >>> > >>>>14:19:51,165][INFO][org.apache.kylin.job.manager.ExecutableManager.upda > >>>>teJ > >>> >obOutput(ExecutableManager.java:222)] > >>> >- job id:99cbf62c-d0d4-4427-b075-e848cd8d83e9-01 from RUNNING to ERROR > >>> > > >>> >2015-03-04 14:42 GMT+08:00 dong wang <[email protected]>: > >>> > > >>> >> when check the log shown in web GUI of this step: > >>> >> > >>> >> 2015-03-03 22:39:39.495 - State of Hadoop job: > >>> >> job_1425448727678_0004:ACCEPTED - UNDEFINED > >>> >> 2015-03-03 22:39:49.518 - State of Hadoop job: > >>> >> job_1425448727678_0004:FINISHED - FAILED > >>> >> no counters for job job_1425448727678_0004 > >>> >> > >>> >> > >>> >> with the same cube, it can be built successfully with kylin-0.6* > >>>before > >>> >> > >>> > >>> > >> > >
