[
https://issues.apache.org/jira/browse/PIG-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974234#comment-13974234
]
srinivas commented on PIG-3897:
-------------------------------
Update:
It looks like issue is with temp files getting created and not getting cleaned
up, since we run the job for multiple hours in one jvm using spring, the temp
jar files created by pigserver are not getting deleted and hence it is filling
up disk space and jobs errors out silently.
We tried calling pigServer.shutdown(), still some temp jar files are not
getting removed.
Any thoughts?
> very generic stack trace and root cause missing
> -----------------------------------------------
>
> Key: PIG-3897
> URL: https://issues.apache.org/jira/browse/PIG-3897
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.12.0
> Environment: linux
> Reporter: srinivas
>
> We noticed following error randomly while upgrading to yarn, we use pig0.12
> and hadoop 2.3
> Stack trace is generic and missing root cause
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException:
> ERROR 2017: Internal error creating job configuration.
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:873)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:298)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:190)
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
> at org.apache.pig.PigServer.execute(PigServer.java:1297)
> at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
> at
> com.shopzilla.fasttrack.sessions.PigExecutor.executePigScriptBatchMode(PigExecutor.java:40)
> at
> com.shopzilla.fasttrack.sessions.SessionFileterRulesSyncTasklet.execute(SessionFileterRulesSyncTasklet.java:49)
> at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
> at
> org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
> at $Proxy28.execute(Unknown Source)
> at
> org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:386)
> at
> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
> at
> org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:264)
> at
> org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:76)
> at org.springframework.batch.r
--
This message was sent by Atlassian JIRA
(v6.2#6252)
