[jira] [Created] (SYSTEMML-2468) Extended matrix histograms for matrix product chains
Matthias Boehm created SYSTEMML-2468: Summary: Extended matrix histograms for matrix product chains Key: SYSTEMML-2468 URL: https://issues.apache.org/jira/browse/SYSTEMML-2468 Project: SystemML Issue Type: Sub-task Reporter: Matthias Boehm -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (SYSTEMML-2458) Add experiment on spark paramserv
[ https://issues.apache.org/jira/browse/SYSTEMML-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556472#comment-16556472 ] Matthias Boehm edited comment on SYSTEMML-2458 at 7/26/18 2:47 AM: --- Thanks - I just gave it a try and the script failed due to invalid name bindings on function invocations (with the recently introduced named function arguments we now check for correct name bindings). I just pushed the fix. Subsequently, it ran into SYSTEMML-2466 - maybe you could have a look [~Guobao]? was (Author: mboehm7): Thanks - I just gave it a try and the script failed due to invalid name bindings on function invocations. I just pushed the fix. Subsequently, it ran into SYSTEMML-2466 - maybe you could have a look [~Guobao]? > Add experiment on spark paramserv > - > > Key: SYSTEMML-2458 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2458 > Project: SystemML > Issue Type: Sub-task >Reporter: LI Guobao >Assignee: LI Guobao >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SYSTEMML-2467) Wrong IPA size propagation w/ permuted named function arguments
Matthias Boehm created SYSTEMML-2467: Summary: Wrong IPA size propagation w/ permuted named function arguments Key: SYSTEMML-2467 URL: https://issues.apache.org/jira/browse/SYSTEMML-2467 Project: SystemML Issue Type: Bug Reporter: Matthias Boehm -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SYSTEMML-2458) Add experiment on spark paramserv
[ https://issues.apache.org/jira/browse/SYSTEMML-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556472#comment-16556472 ] Matthias Boehm commented on SYSTEMML-2458: -- Thanks - I just gave it a try and the script failed due to invalid name bindings on function invocations. I just pushed the fix. Subsequently, it ran into SYSTEMML-2466 - maybe you could have a look [~Guobao]? > Add experiment on spark paramserv > - > > Key: SYSTEMML-2458 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2458 > Project: SystemML > Issue Type: Sub-task >Reporter: LI Guobao >Assignee: LI Guobao >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SYSTEMML-2466) Distributed paramserv fails on newer Spark version > 2.1
Matthias Boehm created SYSTEMML-2466:
Summary: Distributed paramserv fails on newer Spark version > 2.1
Key: SYSTEMML-2466
URL: https://issues.apache.org/jira/browse/SYSTEMML-2466
Project: SystemML
Issue Type: Task
Reporter: Matthias Boehm
{code}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/spark/network/util/SystemPropertyConfigProvider
at
org.apache.sysml.runtime.instructions.cp.ParamservBuiltinCPInstruction.runOnSpark(ParamservBuiltinCPInstruction.java:163)
at
org.apache.sysml.runtime.instructions.cp.ParamservBuiltinCPInstruction.processInstruction(ParamservBuiltinCPInstruction.java:113)
at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:252)
at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:210)
at
org.apache.sysml.runtime.controlprogram.ProgramBlock.execute(ProgramBlock.java:161)
at
org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:116)
at
org.apache.sysml.api.ScriptExecutorUtils.executeRuntimeProgram(ScriptExecutorUtils.java:106)
at org.apache.sysml.api.DMLScript.execute(DMLScript.java:487)
at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:272)
at org.apache.sysml.api.DMLScript.main(DMLScript.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:782)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException:
org.apache.spark.network.util.SystemPropertyConfigProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (SYSTEMML-2465) Keep data consistency for a pre-trained model
LI Guobao created SYSTEMML-2465: --- Summary: Keep data consistency for a pre-trained model Key: SYSTEMML-2465 URL: https://issues.apache.org/jira/browse/SYSTEMML-2465 Project: SystemML Issue Type: Sub-task Reporter: LI Guobao Assignee: LI Guobao In distributed spark backend, pass a given pre-trained model to the paramserv function may cause the data inconsistency. Because the pre-trained model would be cached in driver's memory. In this case, when kicking off the paramserv func, the workers firstly will try to read the data from HDFS where the dirty data in pre-trained model has not been persisted. This leads to a inconsistency. So the idea is to export the dirty data to HDFS before kicking off the remote workers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SYSTEMML-2090) Documentation of language extension
[ https://issues.apache.org/jira/browse/SYSTEMML-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LI Guobao reassigned SYSTEMML-2090: --- Assignee: LI Guobao > Documentation of language extension > --- > > Key: SYSTEMML-2090 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2090 > Project: SystemML > Issue Type: Sub-task >Reporter: Matthias Boehm >Assignee: LI Guobao >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (SYSTEMML-2423) Implementation of spark ps
[ https://issues.apache.org/jira/browse/SYSTEMML-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LI Guobao resolved SYSTEMML-2423. - Resolution: Fixed Fix Version/s: SystemML 1.2 > Implementation of spark ps > -- > > Key: SYSTEMML-2423 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2423 > Project: SystemML > Issue Type: Sub-task >Reporter: LI Guobao >Assignee: LI Guobao >Priority: Major > Fix For: SystemML 1.2 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SYSTEMML-2457) Error handling and add statistics for spark backend
[ https://issues.apache.org/jira/browse/SYSTEMML-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LI Guobao updated SYSTEMML-2457: Summary: Error handling and add statistics for spark backend (was: Error handling and add statistic for spark backend) > Error handling and add statistics for spark backend > --- > > Key: SYSTEMML-2457 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2457 > Project: SystemML > Issue Type: Sub-task >Reporter: LI Guobao >Assignee: LI Guobao >Priority: Major > Fix For: SystemML 1.2 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
