ok great, so it's indeed an issue of file system permissions ("[...] is not
publicly accessable [...]"). Could you please recursively set 755
permissions for everything under systemml (temporarily) just to see if this
fixes the issue. Thanks.
Regards,
Matthias
From: 281165...@qq.com
To: "dev" <dev@systemml.incubator.apache.org>
Date: 04/11/2016 11:26 PM
Subject: Re: machine learning - Some tests failure when build systemML
project -StackOverflow
Stacktrace for
org.apache.sysml.test.integration.functions.data.FullReblockTest, I just
run command 'mvn test -Dtest=FullReblockTest -e'.
org.apache.sysml.runtime.DMLRuntimeException:
org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in
program block generated from statement block between lines 22 and 25 --
Error evaluating instruction: jobtype = CSV_REBLOCK
input labels = [pREADA1, pREADA2]
recReader inst =
rand inst =
mapper inst =
shuffle inst = MR°csvrblk°0·MATRIX·DOUBLE°2·MATRIX·DOUBLE°1000°1000°
false°,°true°0.0‡MR°csvrblk°1·MATRIX·DOUBLE°3·MATRIX·DOUBLE°1000°1000°
false°,°true°0.0
agg inst =
other inst =
output labels = [pVar89, pVar90]
result indices = ,2,3
num reducers = 10
replication = 1
at org.apache.sysml.runtime.controlprogram.Program.execute
(Program.java:152)
at org.apache.sysml.api.DMLScript.execute(DMLScript.java:677)
at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:335)
at org.apache.sysml.api.DMLScript.main(DMLScript.java:190)
at org.apache.sysml.test.integration.AutomatedTestBase.runTest
(AutomatedTestBase.java:1171)
at
org.apache.sysml.test.integration.functions.data.FullReblockTest.runReblockTest
(FullReblockTest.java:459)
at
org.apache.sysml.test.integration.functions.data.FullReblockTest.testCSVMultipleMSparseMR
(FullReblockTest.java:403)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall
(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run
(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively
(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate
(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate
(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate
(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild
(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild
(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute
(JUnit4Provider.java:283)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun
(JUnit4Provider.java:173)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet
(JUnit4Provider.java:153)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke
(JUnit4Provider.java:128)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader
(ForkedBooter.java:203)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess
(ForkedBooter.java:155)
at org.apache.maven.surefire.booter.ForkedBooter.main
(ForkedBooter.java:103)
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime
error in program block generated from statement block between lines 22 and
25 -- Error evaluating instruction: jobtype = CSV_REBLOCK
input labels = [pREADA1, pREADA2]
recReader inst =
rand inst =
mapper inst =
shuffle inst = MR°csvrblk°0·MATRIX·DOUBLE°2·MATRIX·DOUBLE°1000°1000°
false°,°true°0.0‡MR°csvrblk°1·MATRIX·DOUBLE°3·MATRIX·DOUBLE°1000°1000°
false°,°true°0.0
agg inst =
other inst =
output labels = [pVar89, pVar90]
result indices = ,2,3
num reducers = 10
replication = 1
at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction
(ProgramBlock.java:333)
at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions
(ProgramBlock.java:222)
at org.apache.sysml.runtime.controlprogram.ProgramBlock.execute
(ProgramBlock.java:166)
at org.apache.sysml.runtime.controlprogram.Program.execute
(Program.java:145)
... 32 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException:
java.io.IOException: java.util.concurrent.ExecutionException:
java.io.IOException: Resource
file:/home/grapebaba/Documents/projects/incubator-systemml/target/testTemp/functions/data/FullReblockTest/MultipleReblockTest/scratch_space/_p20807_127.0.0.1/TmpOutput28/part-00000
is not publicly accessable and as such cannot be part of the public cache.
at org.apache.sysml.lops.runtime.RunMRJobs.submitJob
(RunMRJobs.java:334)
at org.apache.sysml.lops.runtime.RunMRJobs.prepareAndSubmitJob
(RunMRJobs.java:142)
at
org.apache.sysml.runtime.instructions.MRJobInstruction.processInstruction
(MRJobInstruction.java:1499)
at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction
(ProgramBlock.java:303)
... 35 more
Caused by: java.io.IOException: java.util.concurrent.ExecutionException:
java.io.IOException: Resource
file:/home/grapebaba/Documents/projects/incubator-systemml/target/testTemp/functions/data/FullReblockTest/MultipleReblockTest/scratch_space/_p20807_127.0.0.1/TmpOutput28/part-00000
is not publicly accessable and as such cannot be part of the public cache.
at org.apache.hadoop.mapred.LocalDistributedCacheManager.setup
(LocalDistributedCacheManager.java:149)
at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>
(LocalJobRunner.java:163)
at org.apache.hadoop.mapred.LocalJobRunner.submitJob
(LocalJobRunner.java:731)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal
(JobSubmitter.java:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs
(UserGroupInformation.java:1556)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs
(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.JobClient.submitJobInternal
(JobClient.java:557)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:833)
at org.apache.sysml.runtime.matrix.CSVReblockMR.runCSVReblockJob
(CSVReblockMR.java:468)
at org.apache.sysml.runtime.matrix.CSVReblockMR.runJob
(CSVReblockMR.java:280)
at org.apache.sysml.lops.runtime.RunMRJobs.submitJob
(RunMRJobs.java:256)
... 38 more
Caused by: java.util.concurrent.ExecutionException: java.io.IOException:
Resource
file:/home/grapebaba/Documents/projects/incubator-systemml/target/testTemp/functions/data/FullReblockTest/MultipleReblockTest/scratch_space/_p20807_127.0.0.1/TmpOutput28/part-00000
is not publicly accessable and as such cannot be part of the public cache.
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.hadoop.mapred.LocalDistributedCacheManager.setup
(LocalDistributedCacheManager.java:145)
... 58 more
Caused by: java.io.IOException: Resource
file:/home/grapebaba/Documents/projects/incubator-systemml/target/testTemp/functions/data/FullReblockTest/MultipleReblockTest/scratch_space/_p20807_127.0.0.1/TmpOutput28/part-00000
is not publicly accessable and as such cannot be part of the public cache.
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:257)
at org.apache.hadoop.yarn.util.FSDownload.access$000
(FSDownload.java:60)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:355)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:353)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs
(UserGroupInformation.java:1556)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:352)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:59)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
------------------ Original ------------------
From: "Matthias Boehm";<mbo...@us.ibm.com>;
Date: Tue, Apr 12, 2016 01:18 PM
To: "dev"<dev@systemml.incubator.apache.org>;
Cc: "葡萄??"<281165...@qq.com>;
Subject: Re: machine learning - Some tests failure when build systemML
project -StackOverflow
could you please pick one failing test to start with and provide the
respective full stacktrace (potentially rerun it individually w/
maven/junit)? This would already reveal common setup issues like file
permissions, missing directories, etc. Thanks.
Regards,
Matthias
281165273---04/11/2016 10:11:21 PM---Thanks your quick response. But I saw
many different errors occur on different test case. I archived
From: 281165...@qq.com
To: "dev" <dev@systemml.incubator.apache.org>
Date: 04/11/2016 10:11 PM
Subject: Re: machine learning - Some tests failure when build
systemML project -Stack Overflow
Thanks your quick response. But I saw many different errors occur on
different test case. I archived the failsafe-report. Would you like help to
see it?
------------------ Original ------------------
From: "Matthias Boehm";<mbo...@us.ibm.com>;
Date: Tue, Apr 12, 2016 11:31 AM
To: "dev"<dev@systemml.incubator.apache.org>;
Cc: "葡萄??"<281165...@qq.com>;
Subject: Re: machine learning - Some tests failure when build systemML
project -Stack Overflow
well the error is not coming from R but from SystemML's runtime. Could you
please provide the full stacktrace to see what is going on here?
Regards,
Matthias
281165273---04/11/2016 08:22:18 PM---Sorry to bother you guys, I am a
developer from IBM and interesting in this project. I just ask a qu
From: 281165...@qq.com
To: "dev" <dev@systemml.incubator.apache.org>
Date: 04/11/2016 08:22 PM
Subject: machine learning - Some tests failure when build systemML project
- Stack Overflow
Sorry to bother you guys, I am a developer from IBM and interesting in this
project. I just ask a question (
http://stackoverflow.com/questions/36562951/some-tests-failure-when-build-systemml-project
) on stackoverflow, but I don't find any tag for this project, could I post
it in this mail list?
