[jira] [Closed] (SYSTEMML-2525) Add RESTful prediction service to deploy SystemML models
[ https://issues.apache.org/jira/browse/SYSTEMML-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare closed SYSTEMML-2525. - Resolution: Fixed Fix Version/s: SystemML 1.3 Closed by the commit [https://github.com/apache/systemml/commit/863c9d5cb1752b0e50140f5c6673968b57c2f9d0] > Add RESTful prediction service to deploy SystemML models > > > Key: SYSTEMML-2525 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2525 > Project: SystemML > Issue Type: Project >Reporter: Niketan Pansare >Priority: Major > Fix For: SystemML 1.3 > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SYSTEMML-2525) Add RESTful prediction service to deploy SystemML models
Niketan Pansare created SYSTEMML-2525: - Summary: Add RESTful prediction service to deploy SystemML models Key: SYSTEMML-2525 URL: https://issues.apache.org/jira/browse/SYSTEMML-2525 Project: SystemML Issue Type: Project Reporter: Niketan Pansare -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SYSTEMML-2514) Hadoop gives java.lang.UnsatisfiedLinkError
[ https://issues.apache.org/jira/browse/SYSTEMML-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16797676#comment-16797676 ] Niketan Pansare commented on SYSTEMML-2514: --- I usually use SystemML with Spark. To verify whether Spark is correctly installed, please follow the below steps: 1. Java === The Java version should be > 1.8. > java -version Set JAVA_HOME environment variable and include %JAVA_HOME%\bin in the environment variable PATH > ls "%JAVA_HOME%" 2. Spark Download and extract Spark from https://spark.apache.org/downloads.html, > tar -xzf spark-2.1.0-bin-hadoop2.7.tgz and set environment variable SPARK_HOME to point to the extracted directory. Next step, install winutils: - Download winutils.exe from http://github.com/steveloughran/winutils/raw/master/hadoop-2.6.0/bin/winutils.exe - Place it in c:\winutils\bin - Set environment variable HADOOP_HOME to point to c:\winutils - Add c:\winutils\bin to the environment variable PATH. - Finally, modify permission of hive directory that will be used by spark > winutils.exe chmod 777 /tmp/hive Finally, check if Spark is correctly installed: > %SPARK_HOME%\bin\spark-shell > %SPARK_HOME%\bin\pyspark > Hadoop gives java.lang.UnsatisfiedLinkError > --- > > Key: SYSTEMML-2514 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2514 > Project: SystemML > Issue Type: Sub-task > Components: Build >Reporter: Janardhan >Priority: Minor > Attachments: hadoop_on_win.pdf > > > > {code:java} > java.lang.UnsatisfiedLinkError: > org.apache.hadoop.io.nativeio.NativeIO$Windows.createFileWithMode0(Ljava/lang/String;JJJI)Ljava/io/FileDescriptor; > at org.apache.hadoop.io.nativeio.NativeIO$Windows.createFileWithMode0(Native > Method) > at > org.apache.hadoop.io.nativeio.NativeIO$Windows.createFileOutputStreamWithMode(NativeIO.java:559) > at > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:219) > at > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:209) > at > org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:307) > at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:295) > at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:328) > at > org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.(ChecksumFileSystem.java:398) > at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:461) > at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:440) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:910) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788) > at org.apache.sysml.test.utils.TestUtils.writeTestMatrix(TestUtils.java:1458) > at org.apache.sysml.test.utils.TestUtils.writeTestMatrix(TestUtils.java:1514) > at > org.apache.sysml.test.integration.AutomatedTestBase.writeInputMatrix(AutomatedTestBase.java:530) > at > org.apache.sysml.test.integration.AutomatedTestBase.writeInputMatrix(AutomatedTestBase.java:587) > at > org.apache.sysml.test.integration.functions.blocks.VariableTest.testVariablePassing1(VariableTest.java:80) > {code} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SYSTEMML-2523) Update SystemML to Support Spark 2.3.0
Niketan Pansare created SYSTEMML-2523: - Summary: Update SystemML to Support Spark 2.3.0 Key: SYSTEMML-2523 URL: https://issues.apache.org/jira/browse/SYSTEMML-2523 Project: SystemML Issue Type: Project Reporter: Niketan Pansare Spark 2.3 (released on February 28, 2018) has updated Antlr version, which throws warning every time we invoke SystemML. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (SYSTEMML-2505) Support generation of DML script using Keras2DML and Caffe2DML
[ https://issues.apache.org/jira/browse/SYSTEMML-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-2505. --- Resolution: Fixed Fixed by https://github.com/apache/systemml/commit/7019f3bc805aaae67ef32e281cf99e26cbd26b29 > Support generation of DML script using Keras2DML and Caffe2DML > -- > > Key: SYSTEMML-2505 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2505 > Project: SystemML > Issue Type: Test > Components: APIs >Reporter: Niketan Pansare >Assignee: Niketan Pansare >Priority: Major > Fix For: SystemML 1.3 > > > > {code} > from keras.applications.vgg16 import VGG16 > keras_model = VGG16(weights="imagenet", pooling="max") > from systemml.mllearn import Keras2DML > sysml_model = Keras2DML(spark, keras_model, input_shape=(3,224,224), > weights='weights_dir') > sysml_model.set(test_algo='batch', train_algo='minibatch') > print(sysml_model.get_training_script()) > print(sysml_model.get_prediction_script()) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SYSTEMML-2505) Support generation of DML script using Keras2DML and Caffe2DML
Niketan Pansare created SYSTEMML-2505: - Summary: Support generation of DML script using Keras2DML and Caffe2DML Key: SYSTEMML-2505 URL: https://issues.apache.org/jira/browse/SYSTEMML-2505 Project: SystemML Issue Type: Test Components: APIs Reporter: Niketan Pansare Assignee: Niketan Pansare Fix For: SystemML 1.3 {code} from keras.applications.vgg16 import VGG16 keras_model = VGG16(weights="imagenet", pooling="max") from systemml.mllearn import Keras2DML sysml_model = Keras2DML(spark, keras_model, input_shape=(3,224,224), weights='weights_dir') sysml_model.set(test_algo='batch', train_algo='minibatch') print(sysml_model.get_training_script()) print(sysml_model.get_prediction_script()) {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (SYSTEMML-2496) Skip MapReduce JUnit tests
[ https://issues.apache.org/jira/browse/SYSTEMML-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-2496. --- Resolution: Fixed Fix Version/s: SystemML 1.3 Fixed by the commit https://github.com/apache/systemml/commit/95bf8cfe6c0ae4f45b460a03c397203b7b354fc5 > Skip MapReduce JUnit tests > --- > > Key: SYSTEMML-2496 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2496 > Project: SystemML > Issue Type: Test > Components: Test >Reporter: Niketan Pansare >Assignee: Niketan Pansare >Priority: Minor > Fix For: SystemML 1.3 > > > Since MR backend is in maintenance mode, skip MR tests to allow the JUnit > test to complete in less time. This will also enable us to turn on the larger > number of Spark tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SYSTEMML-2496) Skip MapReduce JUnit tests
Niketan Pansare created SYSTEMML-2496: - Summary: Skip MapReduce JUnit tests Key: SYSTEMML-2496 URL: https://issues.apache.org/jira/browse/SYSTEMML-2496 Project: SystemML Issue Type: Test Components: Test Reporter: Niketan Pansare Assignee: Niketan Pansare Since MR backend is in maintenance mode, skip MR tests to allow the JUnit test to complete in less time. This will also enable us to turn on the larger number of Spark tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (SYSTEMML-880) Push-down loop structures in Python DSL
[ https://issues.apache.org/jira/browse/SYSTEMML-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539267#comment-16539267 ] Niketan Pansare edited comment on SYSTEMML-880 at 7/10/18 9:54 PM: --- Pushdown of loop avoids invocation overhead and also enables additional optimization. Here is a simple pyspark script that demonstrates the overhead: {code:java} from systemml import MLContext, dml import numpy as np import time numpyX = np.ones((1,100)) ml = MLContext(sc) # Execute with pushdown of loop script_with_loop = dml('s = 0; for(i in 1:1000) { s = s + sum(X); } ') t0 = time.time() ml.execute(script_with_loop.input(X=numpyX).output('s')).get('s') print('Total time with loop:' + str(time.time()-t0)) # Total time with loop:2.50334095955 # Execute without pushdown of loop pythonS = 0 totalTime = 0 script_without_loop = dml('s = s + sum(X)').input(X=numpyX).output('s') for i in range(1000): t0 = time.time() pythonS = ml.execute(script_without_loop.input(s=pythonS)).get('s') totalTime = totalTime + time.time()-t0 print('Total time without loop:' + str(totalTime)) # Total time without loop:1008.73590732 {code} One way to go about doing this is to define the boundaries using a decorator (for example: parallelize) and try by first supporting simple expression and a loop structure. Few related links: [https://greentreesnakes.readthedocs.io/en/latest/nodes.html#control-flow] [https://eli.thegreenplace.net/2009/11/28/python-internals-working-with-python-asts] was (Author: niketanpansare): Pushdown of loop avoids invocation overhead and also enables additional optimization. Here is a simple pyspark script that demonstrates the overhead: {code:java} from systemml import MLContext, dml import numpy as np import time numpyX = np.ones((1,100)) ml = MLContext(sc) # Execute with pushdown of loop script_with_loop = dml('s = 0; for(i in 1:1000) { s = s + sum(X); } ') t0 = time.time() ml.execute(script_with_loop.input(X=numpyX).output('s')).get('s') print('Total time with loop:' + str(time.time()-t0)) # Total time with loop:2.50334095955 # Execute without pushdown of loop pythonS = 0 totalTime = 0 script_without_loop = dml('s = s + sum(X)').input(X=numpyX).output('s') for i in range(1000): t0 = time.time() pythonS = ml.execute(script_without_loop.input(s=pythonS)).get('s') totalTime = totalTime + time.time()-t0 print('Total time without loop:' + str(totalTime)) # Total time without loop:1008.73590732 {code} One way to go about doing this is to define the boundaries using a decorator (for example: parallelize) and try by first supporting simple expression and a loop structure. > Push-down loop structures in Python DSL > --- > > Key: SYSTEMML-880 > URL: https://issues.apache.org/jira/browse/SYSTEMML-880 > Project: SystemML > Issue Type: Task >Reporter: Niketan Pansare >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SYSTEMML-880) Push-down loop structures in Python DSL
[ https://issues.apache.org/jira/browse/SYSTEMML-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539267#comment-16539267 ] Niketan Pansare commented on SYSTEMML-880: -- Pushdown of loop avoids invocation overhead and also enables additional optimization. Here is a simple pyspark script that demonstrates the overhead: {code:java} from systemml import MLContext, dml import numpy as np import time numpyX = np.ones((1,100)) ml = MLContext(sc) # Execute with pushdown of loop script_with_loop = dml('s = 0; for(i in 1:1000) { s = s + sum(X); } ') t0 = time.time() ml.execute(script_with_loop.input(X=numpyX).output('s')).get('s') print('Total time with loop:' + str(time.time()-t0)) # Total time with loop:2.50334095955 # Execute without pushdown of loop pythonS = 0 totalTime = 0 script_without_loop = dml('s = s + sum(X)').input(X=numpyX).output('s') for i in range(1000): t0 = time.time() pythonS = ml.execute(script_without_loop.input(s=pythonS)).get('s') totalTime = totalTime + time.time()-t0 print('Total time without loop:' + str(totalTime)) # Total time without loop:1008.73590732 {code} One way to go about doing this is to define the boundaries using a decorator (for example: parallelize) and try by first supporting simple expression and a loop structure. > Push-down loop structures in Python DSL > --- > > Key: SYSTEMML-880 > URL: https://issues.apache.org/jira/browse/SYSTEMML-880 > Project: SystemML > Issue Type: Task >Reporter: Niketan Pansare >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SYSTEMML-880) Push-down loop structures in Python DSL
[ https://issues.apache.org/jira/browse/SYSTEMML-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517551#comment-16517551 ] Niketan Pansare commented on SYSTEMML-880: -- Since the below steps are not necessarily subtask, I am listing them: # Read the documentation and try the code: [http://apache.github.io/systemml/python-reference#matrix-class] # Understand the core problem of pushing down the loops: [http://apache.github.io/systemml/python-reference#dealing-with-the-loops] # Then, read the discussion in [PR 270|https://github.com/apache/systemml/pull/270] and understand why a [simple solution|https://github.com/niketanpansare/systemml/blob/6e4c0a15a597034010ce80439aaee87960e836e1/src/main/python/systemml/pushdown.py#L43-L47] is not preferred. # The solution that is preferred is to walk the [Python AST|https://docs.python.org/2/library/ast.html] of the given function and replace loops with PyDML script. > Push-down loop structures in Python DSL > --- > > Key: SYSTEMML-880 > URL: https://issues.apache.org/jira/browse/SYSTEMML-880 > Project: SystemML > Issue Type: Task >Reporter: Niketan Pansare >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SYSTEMML-2313) Add custom kernel for sparse matrix dense matrix multiplication
Niketan Pansare created SYSTEMML-2313: - Summary: Add custom kernel for sparse matrix dense matrix multiplication Key: SYSTEMML-2313 URL: https://issues.apache.org/jira/browse/SYSTEMML-2313 Project: SystemML Issue Type: Sub-task Reporter: Niketan Pansare -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SYSTEMML-2312) Add custom kernel for sparse matrix dense vector multiplication
Niketan Pansare created SYSTEMML-2312: - Summary: Add custom kernel for sparse matrix dense vector multiplication Key: SYSTEMML-2312 URL: https://issues.apache.org/jira/browse/SYSTEMML-2312 Project: SystemML Issue Type: Sub-task Reporter: Niketan Pansare -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (SYSTEMML-1092) Integrate python test cases into junit test case
[ https://issues.apache.org/jira/browse/SYSTEMML-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425815#comment-16425815 ] Niketan Pansare edited comment on SYSTEMML-1092 at 4/4/18 4:43 PM: --- Subtasks for this JIRAs are: # Investigate whether it is possible to package pyspark dependencies into our maven test suite without spark installation. # If not, then come-up with simplest spark installation: ## Download spark and set SPARK_HOME ## Add pyspark as pip dependency # Enable tests [https://github.com/apache/systemml/blob/master/src/test/java/org/apache/sysml/test/integration/functions/python/PythonTestRunner.java#L41] For reference, I believe PySpark test suite is kicked using the shell script: [https://github.com/apache/spark/blob/master/dev/run-tests-jenkins] was (Author: niketanpansare): Before closing this issue, one needs to investigate whether it is possible to package pyspark dependencies into our maven test suite. If not, the following solution needs to be discussed in the mailing list before adding SPARK_HOME requirement - The Python tests are currently disabled as it requires the environment variable SPARK_HOME to be set: https://github.com/apache/systemml/blob/master/src/test/java/org/apache/sysml/test/integration/functions/python/PythonTestRunner.java#L41 - For reference, I believe PySpark test suite is kicked using the shell script: https://github.com/apache/spark/blob/master/dev/run-tests-jenkins > Integrate python test cases into junit test case > > > Key: SYSTEMML-1092 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1092 > Project: SystemML > Issue Type: Task >Reporter: Niketan Pansare >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SYSTEMML-1092) Integrate python test cases into junit test case
[ https://issues.apache.org/jira/browse/SYSTEMML-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425815#comment-16425815 ] Niketan Pansare commented on SYSTEMML-1092: --- Before closing this issue, one needs to investigate whether it is possible to package pyspark dependencies into our maven test suite. If not, the following solution needs to be discussed in the mailing list before adding SPARK_HOME requirement - The Python tests are currently disabled as it requires the environment variable SPARK_HOME to be set: https://github.com/apache/systemml/blob/master/src/test/java/org/apache/sysml/test/integration/functions/python/PythonTestRunner.java#L41 - For reference, I believe PySpark test suite is kicked using the shell script: https://github.com/apache/spark/blob/master/dev/run-tests-jenkins > Integrate python test cases into junit test case > > > Key: SYSTEMML-1092 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1092 > Project: SystemML > Issue Type: Task >Reporter: Niketan Pansare >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SYSTEMML-2165) Support ARC eviction policy for the GPU backend
Niketan Pansare created SYSTEMML-2165: - Summary: Support ARC eviction policy for the GPU backend Key: SYSTEMML-2165 URL: https://issues.apache.org/jira/browse/SYSTEMML-2165 Project: SystemML Issue Type: Sub-task Reporter: Niketan Pansare Link to the paper: [https://dbs.uni-leipzig.de/file/ARC.pdf|https://dbs.uni-leipzig.de/file/ARC.pdf] There might be patent related issues with ARC algorithm. If so, we cannot support it in SystemML: [https://news.ycombinator.com/item?id=9898430|https://news.ycombinator.com/item?id=9898430] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SYSTEMML-1975) Fix GPU tests for single precision
[ https://issues.apache.org/jira/browse/SYSTEMML-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221207#comment-16221207 ] Niketan Pansare commented on SYSTEMML-1975: --- Following tests are failing for double precision: {code} Failed tests: MatrixMultiplicationOpTest.matrixMatrixTest2:161->assertMatrixMultiplication:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:267 Relative error(1.00) is more than threshold (0.10). Expected = -0.002456, Actual = 0.00, differed at [0, 0] MatrixMultiplicationOpTest.transposeSelfMatrixMultiply:235->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:272 expected:<0.0> but was:<1.929108490094312> {code} My guess is there is likely a bug caught by these two tests and other tests are likely related to precision. > Fix GPU tests for single precision > -- > > Key: SYSTEMML-1975 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1975 > Project: SystemML > Issue Type: Bug >Reporter: Niketan Pansare > > The current GPU tests for single precision compares the results of double > precision operation on CPU with single precision operation on GPU. So, it is > difficult to compare the results. When we support single precision on CPU, we > can rework the GPU tests to fix the following tests: > {code} > Failed tests: > > NeuralNetworkOpTests.testConv2dBackwardFilter:324->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 > Relative error(0.12) is more than threshold (0.10). Expected = > 25311.00, Actual = 25310.687500, differed at [0, 50] > > MatrixMultiplicationOpTest.matrixMatrixTest2:161->assertMatrixMultiplication:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 > Relative error(1.00) is more than threshold (0.10). Expected = > -0.002456, Actual = 0.00, differed at [0, 0] > > MatrixMultiplicationOpTest.matrixMatrixTest4:209->assertMatrixMultiplication:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 > Relative error(0.24) is more than threshold (0.10). Expected = > 0.573809, Actual = 0.573822, differed at [5, 127] > > MatrixMultiplicationOpTest.commonCaseDLMatrixMatrixTest2:137->assertMatrixMultiplication:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 > Relative error(0.13) is more than threshold (0.10). Expected = > -0.922950, Actual = -0.922939, differed at [0, 13] > > MatrixMultiplicationOpTest.transposeSelfMatrixMultiply:235->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:272 > expected:<0.0> but was:<1.9291085004806519> > > ScalarMatrixElementwiseOpTests.testModulusLeftScalar:167->runScalarMatrixElementWiseTests:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 > Relative error(1.00) is more than threshold (0.10). Expected = > 0.078125, Actual = 0.00, differed at [1500, 22] > > ScalarMatrixElementwiseOpTests.testIntegerDivideLeftScalar:157->runScalarMatrixElementWiseTests:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 > Relative error(0.015873) is more than threshold (0.10). Expected = > 63.00, Actual = 64.00, differed at [1500, 22] > {code} > [~nakul02] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (SYSTEMML-1969) Support single-precision on GPU
[ https://issues.apache.org/jira/browse/SYSTEMML-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare closed SYSTEMML-1969. - Resolution: Fixed Fix Version/s: SystemML 1.0 Fixed in the commit https://github.com/apache/systemml/commit/abbffc55ef8f47f10b6e59b0ae5e1f311f4a8f3e > Support single-precision on GPU > --- > > Key: SYSTEMML-1969 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1969 > Project: SystemML > Issue Type: New Feature >Reporter: Niketan Pansare >Assignee: Niketan Pansare > Fix For: SystemML 1.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1975) Fix GPU tests for single precision
Niketan Pansare created SYSTEMML-1975: - Summary: Fix GPU tests for single precision Key: SYSTEMML-1975 URL: https://issues.apache.org/jira/browse/SYSTEMML-1975 Project: SystemML Issue Type: Bug Reporter: Niketan Pansare The current GPU tests for single precision compares the results of double precision operation on CPU with single precision operation on GPU. So, it is difficult to compare the results. When we support single precision on CPU, we can rework the GPU tests to fix the following tests: {code} Failed tests: NeuralNetworkOpTests.testConv2dBackwardFilter:324->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 Relative error(0.12) is more than threshold (0.10). Expected = 25311.00, Actual = 25310.687500, differed at [0, 50] MatrixMultiplicationOpTest.matrixMatrixTest2:161->assertMatrixMultiplication:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 Relative error(1.00) is more than threshold (0.10). Expected = -0.002456, Actual = 0.00, differed at [0, 0] MatrixMultiplicationOpTest.matrixMatrixTest4:209->assertMatrixMultiplication:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 Relative error(0.24) is more than threshold (0.10). Expected = 0.573809, Actual = 0.573822, differed at [5, 127] MatrixMultiplicationOpTest.commonCaseDLMatrixMatrixTest2:137->assertMatrixMultiplication:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 Relative error(0.13) is more than threshold (0.10). Expected = -0.922950, Actual = -0.922939, differed at [0, 13] MatrixMultiplicationOpTest.transposeSelfMatrixMultiply:235->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:272 expected:<0.0> but was:<1.9291085004806519> ScalarMatrixElementwiseOpTests.testModulusLeftScalar:167->runScalarMatrixElementWiseTests:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 Relative error(1.00) is more than threshold (0.10). Expected = 0.078125, Actual = 0.00, differed at [1500, 22] ScalarMatrixElementwiseOpTests.testIntegerDivideLeftScalar:157->runScalarMatrixElementWiseTests:261->GPUTests.assertEqualObjects:367->GPUTests.assertEqualMatrices:269 Relative error(0.015873) is more than threshold (0.10). Expected = 63.00, Actual = 64.00, differed at [1500, 22] {code} [~nakul02] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1969) Support single-precision on GPU
Niketan Pansare created SYSTEMML-1969: - Summary: Support single-precision on GPU Key: SYSTEMML-1969 URL: https://issues.apache.org/jira/browse/SYSTEMML-1969 Project: SystemML Issue Type: New Feature Reporter: Niketan Pansare Assignee: Niketan Pansare -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1965) Refactor nn layers to move the computation in forward/backward function known at compile time to init function
[ https://issues.apache.org/jira/browse/SYSTEMML-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16208004#comment-16208004 ] Niketan Pansare commented on SYSTEMML-1965: --- My two cents: I think both engine improvement as well as good DML practice go hand in hand. If `init` is not the right place, then we should add `compute_height` and `compute_width` functions. Having a loop-invariant code (as part of reusable function) in the for loop is not advisable even if we fix the constant propagation portion of the engine. > Refactor nn layers to move the computation in forward/backward function known > at compile time to init function > -- > > Key: SYSTEMML-1965 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1965 > Project: SystemML > Issue Type: Bug >Reporter: Niketan Pansare > > Ideally, we should move the computation known at compile time to init layer, > rather than keep in the forward function. This reduces recompilation time and > also potentially unnecessary instructions. Here is an example snippet from > our conv2d layer: > {code} > Hout = as.integer(floor((Hin + 2*padh - Hf)/strideh + 1)) > Wout = as.integer(floor((Win + 2*padw - Wf)/stridew + 1)) > {code} > [~prithvi_r_s] [~dusenberrymw] [~reinwald] do you have any comments or > concerns ? > [~dusenberrymw] do you have free cycles to take this over ? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1965) Refactor nn layers to move the computation in forward/backward function known at compile time to init function
Niketan Pansare created SYSTEMML-1965: - Summary: Refactor nn layers to move the computation in forward/backward function known at compile time to init function Key: SYSTEMML-1965 URL: https://issues.apache.org/jira/browse/SYSTEMML-1965 Project: SystemML Issue Type: Bug Reporter: Niketan Pansare Ideally, we should move the computation known at compile time to init layer, rather than keep in the forward function. This reduces recompilation time and also potentially unnecessary instructions. Here is an example snippet from our conv2d layer: {code} Hout = as.integer(floor((Hin + 2*padh - Hf)/strideh + 1)) Wout = as.integer(floor((Win + 2*padw - Wf)/stridew + 1)) {code} [~prithvi_r_s] [~dusenberrymw] [~reinwald] do you have any comments or concerns ? [~dusenberrymw] do you have free cycles to take this over ? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1962) Support model-selection via mllearn APIs
[ https://issues.apache.org/jira/browse/SYSTEMML-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16206706#comment-16206706 ] Niketan Pansare commented on SYSTEMML-1962: --- Interesting paper somewhat related to this PR: https://arxiv.org/abs/1601.00024 > Support model-selection via mllearn APIs > > > Key: SYSTEMML-1962 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1962 > Project: SystemML > Issue Type: New Feature >Reporter: Niketan Pansare >Assignee: Janardhan > > The end goal of this JIRA is to support model selection facility similar to > [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. > Currently, we support model selection using MLPipeline's cross-validator. For > example: please replace `from pyspark.ml.classification import > LogisticRegression` with `from systemml.mllearn import LogisticRegression` in > the example > http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation. > > However, this invokes k-seperate and independent mlcontext calls. This PR > proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly > bayesian optimization which like mllearn has methods `fit` and `predict`. > These methods internally generate a script that wraps the external script > with a `parfor` when the fit method is called. For example: > {code} > from sklearn import datasets > from systemml.mllearn import GridSearchCV, SVM > iris = datasets.load_iris() > parameters = {'C':[1, 10]} > svm = SVM() > clf = GridSearchClassifierCV(svm, parameters) > clf.fit(iris.data, iris.target) > {code} > would execute the script: > {code} > CVals = matrix("1; 10", rows=2, cols=1) > parfor(i in seq(1, nrow(CVals))) { >C = CVals[i, 1] >reg = 1 / C > # SVM script > } > {code} > This will require: > 1. Functionization of the script (for example: L2SVM) > {code} > svm = function(matrix[double] X, matrix[double] Y, double icpt, double tol, > double reg, double maxiter) returns (matrix[double] w) { >if(nrow(X) < 2) > stop("Stopping due to invalid inputs: Not possible to learn a binary > class classifier without at least 2 rows") >check_min = min(Y) > >w = t(cbind(t(w), t(extra_model_params))) > } > {code} > 2. Adding two new java classes in the package `org.apache.sysml.api.ml` > called `GridSearchClassifierCV` which extends > `Estimator[GridSearchClassifierCVModel]` and `GridSearchClassifierCVModel` > which `extends Model[GridSearchClassifierCVModel] with > BaseSystemMLClassifierModel`. Then you will have to implement the abstract > methods: fit and transform respectively. > 3. Add a python class GridSearchClassifierCV that invokes the above java > classes. > For more details on step 2 and step 3, please read the design documentation > of mllearn API: > https://github.com/apache/systemml/blob/master/src/main/scala/org/apache/sysml/api/ml/BaseSystemMLClassifier.scala#L42 > [~dusenberrymw] may be, this can be part of > https://issues.apache.org/jira/browse/SYSTEMML-1159 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1962) Support model-selection via mllearn APIs
[ https://issues.apache.org/jira/browse/SYSTEMML-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare updated SYSTEMML-1962: -- Description: The end goal of this JIRA is to support model selection facility similar to [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. Currently, we support model selection using MLPipeline's cross-validator. For example: please replace `from pyspark.ml.classification import LogisticRegression` with `from systemml.mllearn import LogisticRegression` in the example http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation. However, this invokes k-seperate and independent mlcontext calls. This PR proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly bayesian optimization which like mllearn has methods `fit` and `predict`. These methods internally generate a script that wraps the external script with a `parfor` when the fit method is called. For example: {code} from sklearn import datasets from systemml.mllearn import GridSearchCV, SVM iris = datasets.load_iris() parameters = {'C':[1, 10]} svm = SVM() clf = GridSearchClassifierCV(svm, parameters) clf.fit(iris.data, iris.target) {code} would execute the script: {code} CVals = matrix("1; 10", rows=2, cols=1) parfor(i in seq(1, nrow(CVals))) { C = CVals[i, 1] reg = 1 / C # SVM script } {code} This will require: 1. Functionization of the script (for example: L2SVM) {code} svm = function(matrix[double] X, matrix[double] Y, double icpt, double tol, double reg, double maxiter) returns (matrix[double] w) { if(nrow(X) < 2) stop("Stopping due to invalid inputs: Not possible to learn a binary class classifier without at least 2 rows") check_min = min(Y) w = t(cbind(t(w), t(extra_model_params))) } {code} 2. Adding two new java classes in the package `org.apache.sysml.api.ml` called `GridSearchClassifierCV` which extends `Estimator[GridSearchClassifierCVModel]` and `GridSearchClassifierCVModel` which `extends Model[GridSearchClassifierCVModel] with BaseSystemMLClassifierModel`. Then you will have to implement the abstract methods: fit and transform respectively. 3. Add a python class GridSearchClassifierCV that invokes the above java classes. For more details on step 2 and step 3, please read the design documentation of mllearn API: https://github.com/apache/systemml/blob/master/src/main/scala/org/apache/sysml/api/ml/BaseSystemMLClassifier.scala#L42 [~dusenberrymw] may be, this can be part of https://issues.apache.org/jira/browse/SYSTEMML-1159 was: The end goal of this JIRA is to support model selection facility similar to [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. Currently, we support model selection using MLPipeline's cross-validator. For example: please replace `from pyspark.ml.classification import LogisticRegression` with `from systemml.mllearn import LogisticRegression` in the example http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation. However, this invokes k-seperate and independent mlcontext calls. This PR proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly bayesian optimization which like mllearn has methods `fit` and `predict`. These methods internally generate a script that wraps the external script with a `parfor` when the fit method is called. For example: {code} from sklearn import datasets from systemml.mllearn import GridSearchCV, SVM iris = datasets.load_iris() parameters = {'C':[1, 10]} svm = SVM() clf = GridSearchClassifierCV(svm, parameters) clf.fit(iris.data, iris.target) {code} would execute the script: {code} CVals = matrix("1; 10", rows=2, cols=1) parfor(i in seq(1, nrow(CVals))) { C = CVals[i, 1] # SVM script } {code} This will require: 1. Functionization of the script (for example: L2SVM) {code} svm = function(matrix[double] X, matrix[double] Y, double icpt, double tol, double reg, double maxiter) returns (matrix[double] w) { if(nrow(X) < 2) stop("Stopping due to invalid inputs: Not possible to learn a binary class classifier without at least 2 rows") check_min = min(Y) w = t(cbind(t(w), t(extra_model_params))) } {code} 2. Adding two new java classes in the package `org.apache.sysml.api.ml` called `GridSearchClassifierCV` which extends `Estimator[GridSearchClassifierCVModel]` and `GridSearchClassifierCVModel` which `extends Model[GridSearchClassifierCVModel] with BaseSystemMLClassifierModel`. Then you will have to implement the abstract methods: fit and transform respectively. 3. Add a python class GridSearchClassifierCV that invokes the above java classes. For more details on step 2 and step 3, please read the design documentation of mllearn API: https://github.com/apache/systemml/blob/master/src
[jira] [Updated] (SYSTEMML-1962) Support model-selection via mllearn APIs
[ https://issues.apache.org/jira/browse/SYSTEMML-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare updated SYSTEMML-1962: -- Description: The end goal of this JIRA is to support model selection facility similar to [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. Currently, we support model selection using MLPipeline's cross-validator. For example: please replace `from pyspark.ml.classification import LogisticRegression` with `from systemml.mllearn import LogisticRegression` in the example http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation. However, this invokes k-seperate and independent mlcontext calls. This PR proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly bayesian optimization which like mllearn has methods `fit` and `predict`. These methods internally generate a script that wraps the external script with a `parfor` when the fit method is called. For example: {code} from sklearn import datasets from systemml.mllearn import GridSearchCV, SVM iris = datasets.load_iris() parameters = {'C':[1, 10]} svm = SVM() clf = GridSearchClassifierCV(svm, parameters) clf.fit(iris.data, iris.target) {code} would execute the script: {code} CVals = matrix("1; 10", rows=2, cols=1) parfor(i in seq(1, nrow(CVals))) { C = CVals[i, 1] # SVM script } {code} This will require: 1. Functionization of the script (for example: L2SVM) {code} svm = function(matrix[double] X, matrix[double] Y, double icpt, double tol, double reg, double maxiter) returns (matrix[double] w) { if(nrow(X) < 2) stop("Stopping due to invalid inputs: Not possible to learn a binary class classifier without at least 2 rows") check_min = min(Y) w = t(cbind(t(w), t(extra_model_params))) } {code} 2. Adding two new java classes in the package `org.apache.sysml.api.ml` called `GridSearchClassifierCV` which extends `Estimator[GridSearchClassifierCVModel]` and `GridSearchClassifierCVModel` which `extends Model[GridSearchClassifierCVModel] with BaseSystemMLClassifierModel`. Then you will have to implement the abstract methods: fit and transform respectively. 3. Add a python class GridSearchClassifierCV that invokes the above java classes. For more details on step 2 and step 3, please read the design documentation of mllearn API: https://github.com/apache/systemml/blob/master/src/main/scala/org/apache/sysml/api/ml/BaseSystemMLClassifier.scala#L42 [~dusenberrymw] may be, this can be part of https://issues.apache.org/jira/browse/SYSTEMML-1159 was: The end goal of this JIRA is to support model selection facility similar to [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. Currently, we support model selection using MLPipeline's cross-validator. For example: please replace `from pyspark.ml.classification import LogisticRegression` with `from systemml.mllearn import LogisticRegression` in the example http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation. However, this invokes k-seperate and independent mlcontext calls. This PR proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly bayesian optimization which like mllearn has methods `fit` and `predict`. These methods internally generate a script that wraps the external script with a `parfor` when the fit method is called. For example: {code} from sklearn import datasets from systemml.mllearn import GridSearchCV, SVM iris = datasets.load_iris() parameters = {'C':[1, 10]} svm = SVM() clf = GridSearchClassifierCV(svm, parameters) clf.fit(iris.data, iris.target) {code} would execute the script: {code} CVals = matrix("1; 10", rows=2, cols=1) parfor(i in seq(1, nrow(CVals))) { C = CVals[i, 1] # SVM script } {code} This will require: 1. Functionization of the script (for example: L2SVM) {code} svm = function(matrix[double] X, matrix[double] Y, double icpt, double tol, double reg, double maxiter) returns (matrix[double] w) { if(nrow(X) < 2) stop("Stopping due to invalid inputs: Not possible to learn a binary class classifier without at least 2 rows") check_min = min(Y) w = t(cbind(t(w), t(extra_model_params))) } {code} 2. Adding two new java classes in the package `org.apache.sysml.api.ml` called `GridSearchClassifierCV` which extends `Estimator[GridSearchClassifierCVModel]` and `GridSearchClassifierCVModel` which `extends Model[GridSearchClassifierCVModel] with BaseSystemMLClassifierModel`. Then you will have to implement the abstract methods: fit and transform respectively. 3. Add a python class GridSearchClassifierCV that invokes the above java classes. For more details on step 2 and step 3, please read the design documentation of mllearn API: https://github.com/apache/systemml/blob/master/src/main/scala/org
[jira] [Updated] (SYSTEMML-1962) Support model-selection via mllearn APIs
[ https://issues.apache.org/jira/browse/SYSTEMML-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare updated SYSTEMML-1962: -- Description: The end goal of this JIRA is to support model selection facility similar to [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. Currently, we support model selection using MLPipeline's cross-validator. For example: please replace `from pyspark.ml.classification import LogisticRegression` with `from systemml.mllearn import LogisticRegression` in the example http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation. However, this invokes k-seperate and independent mlcontext calls. This PR proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly bayesian optimization which like mllearn has methods `fit` and `predict`. These methods internally generate a script that wraps the external script with a `parfor` when the fit method is called. For example: {code} from sklearn import datasets from systemml.mllearn import GridSearchCV, SVM iris = datasets.load_iris() parameters = {'C':[1, 10]} svm = SVM() clf = GridSearchClassifierCV(svm, parameters) clf.fit(iris.data, iris.target) {code} would execute the script: {code} CVals = matrix("1; 10", rows=2, cols=1) parfor(i in seq(1, nrow(CVals))) { C = CVals[i, 1] # SVM script } {code} This will require: 1. Functionization of the script (for example: L2SVM) {code} svm = function(matrix[double] X, matrix[double] Y, double icpt, double tol, double reg, double maxiter) returns (matrix[double] w) { if(nrow(X) < 2) stop("Stopping due to invalid inputs: Not possible to learn a binary class classifier without at least 2 rows") check_min = min(Y) w = t(cbind(t(w), t(extra_model_params))) } {code} 2. Adding two new java classes in the package `org.apache.sysml.api.ml` called `GridSearchClassifierCV` which extends `Estimator[GridSearchClassifierCVModel]` and `GridSearchClassifierCVModel` which `extends Model[GridSearchClassifierCVModel] with BaseSystemMLClassifierModel`. Then you will have to implement the abstract methods: fit and transform respectively. 3. Add a python class GridSearchClassifierCV that invokes the above java classes. For more details on step 2 and step 3, please read the design documentation of mllearn API: https://github.com/apache/systemml/blob/master/src/main/scala/org/apache/sysml/api/ml/BaseSystemMLClassifier.scala#L42 [~dusenberrymw] was: The end goal of this JIRA is to support model selection facility similar to [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. Currently, we support model selection using MLPipeline's cross-validator. For example: please replace `from pyspark.ml.classification import LogisticRegression` with `from systemml.mllearn import LogisticRegression` in the example http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation. However, this invokes k-seperate and independent mlcontext calls. This PR proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly bayesian optimization which like mllearn has methods `fit` and `predict`. These methods internally generate a script that wraps the external script with a `parfor` when the fit method is called. For example: {code} from sklearn import datasets from systemml.mllearn import GridSearchCV, SVM iris = datasets.load_iris() parameters = {'C':[1, 10]} svm = SVM() clf = GridSearchCV(svm, parameters) clf.fit(iris.data, iris.target) {code} would execute the script: {code} CVals = matrix("1; 10", rows=2, cols=1) parfor(i in seq(1, nrow(CVals))) { C = CVals[i, 1] # SVM script } {code} > Support model-selection via mllearn APIs > > > Key: SYSTEMML-1962 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1962 > Project: SystemML > Issue Type: New Feature >Reporter: Niketan Pansare > > The end goal of this JIRA is to support model selection facility similar to > [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. > Currently, we support model selection using MLPipeline's cross-validator. For > example: please replace `from pyspark.ml.classification import > LogisticRegression` with `from systemml.mllearn import LogisticRegression` in > the example > http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation. > > However, this invokes k-seperate and independent mlcontext calls. This PR > proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly > bayesian optimization which like mllearn has methods `fit` and `predict`. > These methods internally generate a script that wraps the external script > with a `
[jira] [Updated] (SYSTEMML-1962) Support model-selection via mllearn APIs
[ https://issues.apache.org/jira/browse/SYSTEMML-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare updated SYSTEMML-1962: -- Description: The end goal of this JIRA is to support model selection facility similar to [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. Currently, we support model selection using MLPipeline's cross-validator. For example: please replace `from pyspark.ml.classification import LogisticRegression` with `from systemml.mllearn import LogisticRegression` in the example http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation. However, this invokes k-seperate and independent mlcontext calls. This PR proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly bayesian optimization which like mllearn has methods `fit` and `predict`. These methods internally generate a script that wraps the external script with a `parfor` when the fit method is called. For example: {code} from sklearn import datasets from systemml.mllearn import GridSearchCV, SVM iris = datasets.load_iris() parameters = {'C':[1, 10]} svm = SVM() clf = GridSearchCV(svm, parameters) clf.fit(iris.data, iris.target) {code} would execute the script: {code} CVals = matrix("1; 10", rows=2, cols=1) parfor(i in seq(1, nrow(CVals))) { C = CVals[i, 1] # SVM script } {code} was:The end goal of this JIRA is to support model selection facility similar to [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. > Support model-selection via mllearn APIs > > > Key: SYSTEMML-1962 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1962 > Project: SystemML > Issue Type: New Feature >Reporter: Niketan Pansare > > The end goal of this JIRA is to support model selection facility similar to > [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. > Currently, we support model selection using MLPipeline's cross-validator. For > example: please replace `from pyspark.ml.classification import > LogisticRegression` with `from systemml.mllearn import LogisticRegression` in > the example > http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation. > > However, this invokes k-seperate and independent mlcontext calls. This PR > proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly > bayesian optimization which like mllearn has methods `fit` and `predict`. > These methods internally generate a script that wraps the external script > with a `parfor` when the fit method is called. For example: > {code} > from sklearn import datasets > from systemml.mllearn import GridSearchCV, SVM > iris = datasets.load_iris() > parameters = {'C':[1, 10]} > svm = SVM() > clf = GridSearchCV(svm, parameters) > clf.fit(iris.data, iris.target) > {code} > would execute the script: > {code} > CVals = matrix("1; 10", rows=2, cols=1) > parfor(i in seq(1, nrow(CVals))) { >C = CVals[i, 1] > # SVM script > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1962) Support model-selection via mllearn APIs
[ https://issues.apache.org/jira/browse/SYSTEMML-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare updated SYSTEMML-1962: -- Description: The end goal of this JIRA is to support model selection facility similar to [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. (was: The end goal of this JIRA is to support model selection facility similar to [scikit-learn](http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection).) > Support model-selection via mllearn APIs > > > Key: SYSTEMML-1962 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1962 > Project: SystemML > Issue Type: New Feature >Reporter: Niketan Pansare > > The end goal of this JIRA is to support model selection facility similar to > [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1962) Support model-selection via mllearn APIs
[ https://issues.apache.org/jira/browse/SYSTEMML-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare updated SYSTEMML-1962: -- Description: The end goal of this JIRA is to support model selection facility similar to [scikit-learn](http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection). > Support model-selection via mllearn APIs > > > Key: SYSTEMML-1962 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1962 > Project: SystemML > Issue Type: New Feature >Reporter: Niketan Pansare > > The end goal of this JIRA is to support model selection facility similar to > [scikit-learn](http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1962) Support model-selection via mllearn APIs
Niketan Pansare created SYSTEMML-1962: - Summary: Support model-selection via mllearn APIs Key: SYSTEMML-1962 URL: https://issues.apache.org/jira/browse/SYSTEMML-1962 Project: SystemML Issue Type: New Feature Reporter: Niketan Pansare -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (SYSTEMML-1940) Modify the default directory of performance test to avoid maven compilation issue
[ https://issues.apache.org/jira/browse/SYSTEMML-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-1940. --- Resolution: Fixed Fix Version/s: SystemML 1.0 Fixed in the commit https://github.com/apache/systemml/commit/355373990c478293090c90557383ab24c2c01c6b > Modify the default directory of performance test to avoid maven compilation > issue > - > > Key: SYSTEMML-1940 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1940 > Project: SystemML > Issue Type: Test >Reporter: Niketan Pansare >Assignee: Niketan Pansare > Fix For: SystemML 1.0 > > > Currently, the performance test generates it data in scripts folder. This > causes maven to take too long for compilation as it tries to create jar with > all the data. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1940) Modify the default directory of performance test to avoid maven compilation issue
[ https://issues.apache.org/jira/browse/SYSTEMML-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare reassigned SYSTEMML-1940: - Assignee: Niketan Pansare > Modify the default directory of performance test to avoid maven compilation > issue > - > > Key: SYSTEMML-1940 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1940 > Project: SystemML > Issue Type: Test >Reporter: Niketan Pansare >Assignee: Niketan Pansare > > Currently, the performance test generates it data in scripts folder. This > causes maven to take too long for compilation as it tries to create jar with > all the data. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1940) Modify the default directory of performance test to avoid maven compilation issue
Niketan Pansare created SYSTEMML-1940: - Summary: Modify the default directory of performance test to avoid maven compilation issue Key: SYSTEMML-1940 URL: https://issues.apache.org/jira/browse/SYSTEMML-1940 Project: SystemML Issue Type: Test Reporter: Niketan Pansare Currently, the performance test generates it data in scripts folder. This causes maven to take too long for compilation as it tries to create jar with all the data. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1500) Add missing loss layers to Caffe2DML
[ https://issues.apache.org/jira/browse/SYSTEMML-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare reassigned SYSTEMML-1500: - Assignee: Niketan Pansare > Add missing loss layers to Caffe2DML > > > Key: SYSTEMML-1500 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1500 > Project: SystemML > Issue Type: Sub-task >Reporter: Niketan Pansare >Assignee: Niketan Pansare > > Multinomial Logistic Loss > Infogain Loss - a generalization of MultinomialLogisticLossLayer. > Softmax with Loss - computes the multinomial logistic loss of the softmax of > its inputs. It’s conceptually identical to a softmax layer followed by a > multinomial logistic loss layer, but provides a more numerically stable > gradient. > Sum-of-Squares / Euclidean - computes the sum of squares of differences of > its two inputs, 12N∑Ni=1∥x1i−x2i∥2212N∑i=1N‖xi1−xi2‖22. > Hinge / Margin - The hinge loss layer computes a one-vs-all hinge (L1) or > squared hinge loss (L2). > Sigmoid Cross-Entropy Loss - computes the cross-entropy (logistic) loss, > often used for predicting targets interpreted as probabilities. > Accuracy / Top-k layer - scores the output as an accuracy with respect to > target – it is not actually a loss and has no backward step. > Contrastive Loss -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1500) Add missing loss layers to Caffe2DML
[ https://issues.apache.org/jira/browse/SYSTEMML-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184560#comment-16184560 ] Niketan Pansare commented on SYSTEMML-1500: --- - Euclidean loss added in the commit: https://github.com/apache/systemml/commit/61dcc85e48a390c1bb63ee4c42aad9a3fade7d06 > Add missing loss layers to Caffe2DML > > > Key: SYSTEMML-1500 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1500 > Project: SystemML > Issue Type: Sub-task >Reporter: Niketan Pansare > > Multinomial Logistic Loss > Infogain Loss - a generalization of MultinomialLogisticLossLayer. > Softmax with Loss - computes the multinomial logistic loss of the softmax of > its inputs. It’s conceptually identical to a softmax layer followed by a > multinomial logistic loss layer, but provides a more numerically stable > gradient. > Sum-of-Squares / Euclidean - computes the sum of squares of differences of > its two inputs, 12N∑Ni=1∥x1i−x2i∥2212N∑i=1N‖xi1−xi2‖22. > Hinge / Margin - The hinge loss layer computes a one-vs-all hinge (L1) or > squared hinge loss (L2). > Sigmoid Cross-Entropy Loss - computes the cross-entropy (logistic) loss, > often used for predicting targets interpreted as probabilities. > Accuracy / Top-k layer - scores the output as an accuracy with respect to > target – it is not actually a loss and has no backward step. > Contrastive Loss -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (SYSTEMML-1493) Add Tanh layer in nn library and Caffe2DML
[ https://issues.apache.org/jira/browse/SYSTEMML-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-1493. --- Resolution: Fixed Assignee: Niketan Pansare Fix Version/s: SystemML 1.0 Fixed in the commit https://github.com/apache/systemml/commit/61dcc85e48a390c1bb63ee4c42aad9a3fade7d06 > Add Tanh layer in nn library and Caffe2DML > -- > > Key: SYSTEMML-1493 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1493 > Project: SystemML > Issue Type: Sub-task >Reporter: Niketan Pansare >Assignee: Niketan Pansare > Fix For: SystemML 1.0 > > > http://caffe.berkeleyvision.org/tutorial/layers/tanh.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1930) Fix protoc compilation error for TensorFlow proto files and restore Tensorboard functionality
[ https://issues.apache.org/jira/browse/SYSTEMML-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare updated SYSTEMML-1930: -- Description: To use Tensorboard in Caffe2DML, we were compiling src/main/proto/tensorflow/event.proto and src/main/proto/tensorflow/summary.proto files using protoc 3.0 mvn plugin (please see [https://github.com/apache/systemml/commit/a4962f39a065bce89dd1c45635e318b9440e5491#diff-600376dffeb79835ede4a0b285078036L369]). These proto files match the ones that are available in TensorFlow repository. To compile SystemML on Power, we have to switch to a newer version of protoc mvn plugin which now only supports 3.4 version (not 3.0). The problem is that the above mentioned proto files are giving compilation error on protoc 3.4 version. So, for time being, we are removed tensorboard integration from SystemML in commit: https://github.com/apache/systemml/commit/a4962f39a065bce89dd1c45635e318b9440e5491#diff-600376dffeb79835ede4a0b285078036L369 This task involves debugging compilation error on protoc 3.4 version and restoring the above commit. [~freiss] [~nakul02] [~reinwald] was: To use Tensorboard in Caffe2DML, we were compiling src/main/proto/tensorflow/event.proto and src/main/proto/tensorflow/summary.proto files using protoc 3.0 mvn plugin (please see [https://github.com/apache/systemml/commit/a4962f39a065bce89dd1c45635e318b9440e5491#diff-600376dffeb79835ede4a0b285078036L369]). These proto files match the ones that are available in TensorFlow repository. To compile SystemML on Power, we have to switch to a newer version of protoc mvn plugin which now only supports 3.4 version (not 3.0). The problem is that the above mentioned proto files are giving compilation error. So, for time being, we are removed tensorboard integration from SystemML in commit: https://github.com/apache/systemml/commit/a4962f39a065bce89dd1c45635e318b9440e5491#diff-600376dffeb79835ede4a0b285078036L369 This task involves debugging compilation error on protoc 3.4 version and restoring the above commit. [~freiss] [~nakul02] [~reinwald] > Fix protoc compilation error for TensorFlow proto files and restore > Tensorboard functionality > - > > Key: SYSTEMML-1930 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1930 > Project: SystemML > Issue Type: Bug >Reporter: Niketan Pansare > Original Estimate: 24h > Remaining Estimate: 24h > > To use Tensorboard in Caffe2DML, we were compiling > src/main/proto/tensorflow/event.proto and > src/main/proto/tensorflow/summary.proto files using protoc 3.0 mvn plugin > (please see > [https://github.com/apache/systemml/commit/a4962f39a065bce89dd1c45635e318b9440e5491#diff-600376dffeb79835ede4a0b285078036L369]). > These proto files match the ones that are available in TensorFlow > repository. > To compile SystemML on Power, we have to switch to a newer version of protoc > mvn plugin which now only supports 3.4 version (not 3.0). The problem is that > the above mentioned proto files are giving compilation error on protoc 3.4 > version. So, for time being, we are removed tensorboard integration from > SystemML in commit: > https://github.com/apache/systemml/commit/a4962f39a065bce89dd1c45635e318b9440e5491#diff-600376dffeb79835ede4a0b285078036L369 > This task involves debugging compilation error on protoc 3.4 version and > restoring the above commit. > [~freiss] [~nakul02] [~reinwald] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1928) Remove tensorboard visualization code
[ https://issues.apache.org/jira/browse/SYSTEMML-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176784#comment-16176784 ] Niketan Pansare commented on SYSTEMML-1928: --- I have created a new JIRA for this feature: https://issues.apache.org/jira/browse/SYSTEMML-1930 > Remove tensorboard visualization code > - > > Key: SYSTEMML-1928 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1928 > Project: SystemML > Issue Type: Task >Reporter: Niketan Pansare >Assignee: Niketan Pansare > Fix For: SystemML 1.0 > > > This avoids protoc compilation error on power. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1930) Fix protoc compilation error for TensorFlow proto files and restore Tensorboard functionality
Niketan Pansare created SYSTEMML-1930: - Summary: Fix protoc compilation error for TensorFlow proto files and restore Tensorboard functionality Key: SYSTEMML-1930 URL: https://issues.apache.org/jira/browse/SYSTEMML-1930 Project: SystemML Issue Type: Bug Reporter: Niketan Pansare To use Tensorboard in Caffe2DML, we were compiling src/main/proto/tensorflow/event.proto and src/main/proto/tensorflow/summary.proto files using protoc 3.0 mvn plugin (please see [https://github.com/apache/systemml/commit/a4962f39a065bce89dd1c45635e318b9440e5491#diff-600376dffeb79835ede4a0b285078036L369]). These proto files match the ones that are available in TensorFlow repository. To compile SystemML on Power, we have to switch to a newer version of protoc mvn plugin which now only supports 3.4 version (not 3.0). The problem is that the above mentioned proto files are giving compilation error. So, for time being, we are removed tensorboard integration from SystemML in commit: https://github.com/apache/systemml/commit/a4962f39a065bce89dd1c45635e318b9440e5491#diff-600376dffeb79835ede4a0b285078036L369 This task involves debugging compilation error on protoc 3.4 version and restoring the above commit. [~freiss] [~nakul02] [~reinwald] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (SYSTEMML-1928) Remove tensorboard visualization code
[ https://issues.apache.org/jira/browse/SYSTEMML-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-1928. --- Resolution: Fixed Fix Version/s: SystemML 1.0 Fixed by the commit https://github.com/apache/systemml/commit/a4962f39a065bce89dd1c45635e318b9440e5491 > Remove tensorboard visualization code > - > > Key: SYSTEMML-1928 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1928 > Project: SystemML > Issue Type: Task >Reporter: Niketan Pansare >Assignee: Niketan Pansare > Fix For: SystemML 1.0 > > > This avoids protoc compilation error on power. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (SYSTEMML-1923) Support sinh, cosh and tanh as built-in functions
[ https://issues.apache.org/jira/browse/SYSTEMML-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-1923. --- Resolution: Fixed Fix Version/s: SystemML 1.0 Fixed in the commit https://github.com/apache/systemml/commit/50a895f86167d5766b249a3db89f9c2b06f7a0d0 > Support sinh, cosh and tanh as built-in functions > - > > Key: SYSTEMML-1923 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1923 > Project: SystemML > Issue Type: New Feature >Reporter: Niketan Pansare >Assignee: Niketan Pansare > Fix For: SystemML 1.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1928) Remove tensorboard visualization code
Niketan Pansare created SYSTEMML-1928: - Summary: Remove tensorboard visualization code Key: SYSTEMML-1928 URL: https://issues.apache.org/jira/browse/SYSTEMML-1928 Project: SystemML Issue Type: Task Reporter: Niketan Pansare Assignee: Niketan Pansare This avoids protoc compilation error on power. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1923) Support sinh, cosh and tanh as built-in functions
Niketan Pansare created SYSTEMML-1923: - Summary: Support sinh, cosh and tanh as built-in functions Key: SYSTEMML-1923 URL: https://issues.apache.org/jira/browse/SYSTEMML-1923 Project: SystemML Issue Type: New Feature Reporter: Niketan Pansare Assignee: Niketan Pansare -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1907) Rename python package from systemml-*-python.tgz to systemml-*-python.tar.gz
[ https://issues.apache.org/jira/browse/SYSTEMML-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166772#comment-16166772 ] Niketan Pansare commented on SYSTEMML-1907: --- thanks [~gweidner] :) > Rename python package from systemml-*-python.tgz to systemml-*-python.tar.gz > > > Key: SYSTEMML-1907 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1907 > Project: SystemML > Issue Type: Improvement > Components: Build >Reporter: Niketan Pansare >Assignee: Glenn Weidner > > I encountered this issue because pypi has migrated to a new process: > https://packaging.python.org/guides/migrating-to-pypi-org/#uploading > As noted in the above document, the recommended way to upload python packages > to pypi is now via `twine`. However, if we use `twine` with our current > package naming scheme (i.e. tgz), then it complains `ValueError: Unknown > distribution format: 'systemml-0.15.0-python.tgz'`. Hence, I would recommend > to use suffix `tar.gz` in our subsequent releases. This way we also are > compatible with default package naming convention of pypi: `tar.gz`. > [~acs_s] [~gweidner] [~deron] [~dusenberrymw] [~reinwald] Suggestions ? Any > takers ? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1907) Rename python package from systemml-*-python.tgz to systemml-*-python.tar.gz
Niketan Pansare created SYSTEMML-1907: - Summary: Rename python package from systemml-*-python.tgz to systemml-*-python.tar.gz Key: SYSTEMML-1907 URL: https://issues.apache.org/jira/browse/SYSTEMML-1907 Project: SystemML Issue Type: Improvement Components: Build Reporter: Niketan Pansare I encountered this issue because pypi has migrated to a new process: https://packaging.python.org/guides/migrating-to-pypi-org/#uploading As noted in the above document, the recommended way to upload python packages to pypi is now via `twine`. However, if we use `twine` with our current package naming scheme (i.e. tgz), then it complains `ValueError: Unknown distribution format: 'systemml-0.15.0-python.tgz'`. Hence, I would recommend to use suffix `tar.gz` in our subsequent releases. This way we also are compatible with default package naming convention of pypi: `tar.gz`. [~acs_s] [~gweidner] [~deron] [~dusenberrymw] [~reinwald] Suggestions ? Any takers ? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-976) Add explain and stats option to Python DSL
[ https://issues.apache.org/jira/browse/SYSTEMML-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158911#comment-16158911 ] Niketan Pansare commented on SYSTEMML-976: -- Yes, absolutely. Please let me know if you need any help. Another interesting JIRA is https://issues.apache.org/jira/browse/SYSTEMML-880 :) > Add explain and stats option to Python DSL > -- > > Key: SYSTEMML-976 > URL: https://issues.apache.org/jira/browse/SYSTEMML-976 > Project: SystemML > Issue Type: Improvement > Components: APIs >Reporter: Niketan Pansare > Labels: Hacktoberfest, starter > > This would allow users to see the plans and statistics for a given evaluation > of Python DSL. Since this is a starter task, I would give some hints: > 1. Extend "def eval(outputs, outputDF=False, execute=True):" method to accept > explain and statistics options and pass them before "results = > matrix.ml.execute(matrix.script)". > Here are the relevant links: > https://github.com/apache/incubator-systemml/blob/master/src/main/python/systemml/defmatrix.py#L290 > https://github.com/apache/incubator-systemml/blob/master/src/main/python/systemml/mlcontext.py#L223 > 2. Expose these options through user facing APIs: eval() and methods that > invoke eval (such as toPandas, toNumPyArray, toDataFrame, etc) > 3. Update the documentation: > https://apache.github.io/incubator-systemml/beginners-guide-python -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-984) Add mllearn and scala wrappers for PCA
[ https://issues.apache.org/jira/browse/SYSTEMML-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144306#comment-16144306 ] Niketan Pansare commented on SYSTEMML-984: -- If we implement scalable PCA and do reasonable accuracy/performance testing, then I would recommend that we add scala/python wrapper :) > Add mllearn and scala wrappers for PCA > -- > > Key: SYSTEMML-984 > URL: https://issues.apache.org/jira/browse/SYSTEMML-984 > Project: SystemML > Issue Type: Task > Components: APIs >Reporter: Niketan Pansare > Labels: Hacktoberfest, starter > > See > https://apache.github.io/incubator-systemml/algorithms-matrix-factorization.html#principle-component-analysis > for usage -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1870) Include avro dependency in the bin and standalone-jar artifacts
Niketan Pansare created SYSTEMML-1870: - Summary: Include avro dependency in the bin and standalone-jar artifacts Key: SYSTEMML-1870 URL: https://issues.apache.org/jira/browse/SYSTEMML-1870 Project: SystemML Issue Type: Bug Reporter: Niketan Pansare Assignee: Deron Eriksson [~prithvi_r_s] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1808) Visualize Hop DAG for explaining the optimizer
Niketan Pansare created SYSTEMML-1808: - Summary: Visualize Hop DAG for explaining the optimizer Key: SYSTEMML-1808 URL: https://issues.apache.org/jira/browse/SYSTEMML-1808 Project: SystemML Issue Type: New Feature Reporter: Niketan Pansare Assignee: Niketan Pansare -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (SYSTEMML-1798) provide getScriptExecutionString and getScriptString for Python Script object
[ https://issues.apache.org/jira/browse/SYSTEMML-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-1798. --- Resolution: Fixed Fix Version/s: SystemML 1.0 Fixed in the commit https://github.com/apache/systemml/commit/8f412ac5cd2a06453a55efbe4afe60e838cd33b1 > provide getScriptExecutionString and getScriptString for Python Script object > - > > Key: SYSTEMML-1798 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1798 > Project: SystemML > Issue Type: Sub-task > Components: APIs >Reporter: Berthold Reinwald >Assignee: Niketan Pansare > Fix For: SystemML 1.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (SYSTEMML-1802) Fix build failure for distribution profile on Windows
[ https://issues.apache.org/jira/browse/SYSTEMML-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-1802. --- Resolution: Fixed Fix Version/s: SystemML 1.0 Fixed by the commit https://github.com/apache/systemml/commit/11b689d49b78a73f44b5944cfe0d14bb7c05c3a7 > Fix build failure for distribution profile on Windows > - > > Key: SYSTEMML-1802 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1802 > Project: SystemML > Issue Type: Bug >Reporter: Niketan Pansare >Assignee: Niketan Pansare > Fix For: SystemML 1.0 > > > To test please run `mvn package -P distribution` on Windows machine -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1802) Fix build failure for distribution profile on Windows
Niketan Pansare created SYSTEMML-1802: - Summary: Fix build failure for distribution profile on Windows Key: SYSTEMML-1802 URL: https://issues.apache.org/jira/browse/SYSTEMML-1802 Project: SystemML Issue Type: Bug Reporter: Niketan Pansare Assignee: Niketan Pansare To test please run `mvn package -P distribution` on Windows machine -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1798) provide getScriptExecutionString and getScriptString for Python Script object
[ https://issues.apache.org/jira/browse/SYSTEMML-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare reassigned SYSTEMML-1798: - Assignee: Niketan Pansare > provide getScriptExecutionString and getScriptString for Python Script object > - > > Key: SYSTEMML-1798 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1798 > Project: SystemML > Issue Type: Sub-task > Components: APIs >Reporter: Berthold Reinwald >Assignee: Niketan Pansare > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (SYSTEMML-1796) Provide fine-grained statistics for SystemML
[ https://issues.apache.org/jira/browse/SYSTEMML-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-1796. --- Resolution: Fixed Fix Version/s: SystemML 1.0 Fixed by the commit https://github.com/apache/systemml/commit/648eb21d66f9cd8727090cdf950986765a7e6ee8 > Provide fine-grained statistics for SystemML > > > Key: SYSTEMML-1796 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1796 > Project: SystemML > Issue Type: New Feature >Reporter: Niketan Pansare >Assignee: Niketan Pansare > Fix For: SystemML 1.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1796) Provide fine-grained statistics for SystemML
Niketan Pansare created SYSTEMML-1796: - Summary: Provide fine-grained statistics for SystemML Key: SYSTEMML-1796 URL: https://issues.apache.org/jira/browse/SYSTEMML-1796 Project: SystemML Issue Type: New Feature Reporter: Niketan Pansare Assignee: Niketan Pansare -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1793) Support left and right indexing on GPU
Niketan Pansare created SYSTEMML-1793: - Summary: Support left and right indexing on GPU Key: SYSTEMML-1793 URL: https://issues.apache.org/jira/browse/SYSTEMML-1793 Project: SystemML Issue Type: Sub-task Reporter: Niketan Pansare Assignee: Niketan Pansare -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1774) Improve Parfor parallelism for deep learning
[ https://issues.apache.org/jira/browse/SYSTEMML-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092019#comment-16092019 ] Niketan Pansare commented on SYSTEMML-1774: --- Thanks [~mboehm7] I will look into it :) > Improve Parfor parallelism for deep learning > > > Key: SYSTEMML-1774 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1774 > Project: SystemML > Issue Type: Improvement > Components: Algorithms, Compiler, ParFor >Affects Versions: SystemML 1.0 >Reporter: Fei Hu > Labels: deeplearning > Attachments: Explain_For_HYBRID_SPARK_Mode_With_ErrorInfo.txt, > Explain_For_Spark_Mode.txt, MNIST_Distrib_Sgd.scala, > mnist_lenet_distrib_sgd.dml > > > When running the [distributed MNIST LeNet example | > https://github.com/apache/systemml/blob/master/scripts/nn/examples/mnist_lenet_distrib_sgd.dml], > each mini-batch could ideally run in parallel without interaction. We try to > force {{parfor (j in 1:parallel_batches)}} at line 137 of > {{nn/examples/mnist_lenet_distrib_sgd.dml}} to be {{parfor (j in > 1:parallel_batches, mode=REMOTE_SPARK, opt=CONSTRAINED)}} use > {{REMOTE_SPARK}} mode, but got some errors about > {{org.apache.sysml.runtime.DMLRuntimeException: Not supported: Instructions > of type other than CP instructions}} using the mode {{SPARK}}, and the error > {{java.lang.NullPointerException}} using the mode {{HYBRID_SPARK}}. More log > information can be found at the following comments. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1695) Create design/investigation document for parameter server
[ https://issues.apache.org/jira/browse/SYSTEMML-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088255#comment-16088255 ] Niketan Pansare commented on SYSTEMML-1695: --- Personally, I would go through https://github.com/apache/systemml/blob/master/scripts/nn/examples/Example%20-%20MNIST%20LeNet.ipynb and think how can we modify https://github.com/apache/systemml/blob/master/scripts/nn/examples/mnist_lenet.dml#L190-L198 to enable HogWild-style training. > Create design/investigation document for parameter server > - > > Key: SYSTEMML-1695 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1695 > Project: SystemML > Issue Type: Sub-task >Reporter: Niketan Pansare >Assignee: Janardhan > > Here are relevant links: > - https://issues.apache.org/jira/browse/SPARK-4590 > - https://issues.apache.org/jira/browse/SPARK-6932 > - > https://docs.google.com/document/d/1SX3nkmF41wFXAAIr9BgqvrHSS5mW362fJ7roBXJm06o/edit > This document need not cover how to integrate a parameter server in SystemML > (which comes later), but should discuss following questions: > 1. Should we integrate existing parameter server implementation or build one > from scratch ? What are the pros and cons of each approaches ? For example: > https://github.com/rjagerman/glint > 2. If we plan to integrate existing parameter server implementation, what is > the interface ? (for exampe: update(matrix, new matrix) or update(matrix, > delta)). > 3. Can parameter server help in ML algorithms ? Some experiments might be > helpful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-979) Add support for bayesian optimization
[ https://issues.apache.org/jira/browse/SYSTEMML-979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16081284#comment-16081284 ] Niketan Pansare commented on SYSTEMML-979: -- Also, notified Sasha regarding this JIRA :) > Add support for bayesian optimization > - > > Key: SYSTEMML-979 > URL: https://issues.apache.org/jira/browse/SYSTEMML-979 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Reporter: Niketan Pansare >Assignee: Janardhan > > Main paper: > https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf > supplements: > * selecting the next point to evaluate > [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] > * sobol sequence generator [P Bratley, BL > Fox|http://dl.acm.org/citation.cfm?id=214372] > * Handling Sparsity via the Horseshoe [Carlos M. > Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1695) Create design/investigation document for parameter server
[ https://issues.apache.org/jira/browse/SYSTEMML-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050871#comment-16050871 ] Niketan Pansare commented on SYSTEMML-1695: --- This is a good start. Once you are ready with your draft, you may want to send it over the mailing list for feedback from other committers :) > Create design/investigation document for parameter server > - > > Key: SYSTEMML-1695 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1695 > Project: SystemML > Issue Type: Sub-task >Reporter: Niketan Pansare > > Here are relevant links: > - https://issues.apache.org/jira/browse/SPARK-4590 > - https://issues.apache.org/jira/browse/SPARK-6932 > - > https://docs.google.com/document/d/1SX3nkmF41wFXAAIr9BgqvrHSS5mW362fJ7roBXJm06o/edit > This document need not cover how to integrate a parameter server in SystemML > (which comes later), but should discuss following questions: > 1. Should we integrate existing parameter server implementation or build one > from scratch ? What are the pros and cons of each approaches ? For example: > https://github.com/rjagerman/glint > 2. If we plan to integrate existing parameter server implementation, what is > the interface ? (for exampe: update(matrix, new matrix) or update(matrix, > delta)). > 3. Can parameter server help in ML algorithms ? Some experiments might be > helpful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1695) Create design/investigation document for parameter server
Niketan Pansare created SYSTEMML-1695: - Summary: Create design/investigation document for parameter server Key: SYSTEMML-1695 URL: https://issues.apache.org/jira/browse/SYSTEMML-1695 Project: SystemML Issue Type: Sub-task Reporter: Niketan Pansare Here are relevant links: - https://issues.apache.org/jira/browse/SPARK-4590 - https://issues.apache.org/jira/browse/SPARK-6932 - https://docs.google.com/document/d/1SX3nkmF41wFXAAIr9BgqvrHSS5mW362fJ7roBXJm06o/edit This document need not cover how to integrate a parameter server in SystemML (which comes later), but should discuss following questions: 1. Should we integrate existing parameter server implementation or build one from scratch ? What are the pros and cons of each approaches ? For example: https://github.com/rjagerman/glint 2. If we plan to integrate existing parameter server implementation, what is the interface ? (for exampe: update(matrix, new matrix) or update(matrix, delta)). 3. Can parameter server help in ML algorithms ? Some experiments might be helpful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-739) Explore model-parallel constructs in DML
[ https://issues.apache.org/jira/browse/SYSTEMML-739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043214#comment-16043214 ] Niketan Pansare commented on SYSTEMML-739: -- Hi Janardhan, Welcome to SystemML community !! Here is a light-weight parameter server implementation that you can look at: https://github.com/dmlc/ps-lite and the corresponding paper: https://www.cs.cmu.edu/~muli/file/parameter_server_osdi14.pdf To integrate a parameter server with SystemML, you might want to look into how SystemML's bufferpool work and how one can extend DML to support model parallelism. You can begin by reading the papers http://www.vldb.org/pvldb/vol9/p1425-boehm.pdf and https://arxiv.org/pdf/1503.06384.pdf. Thanks, Niketan. > Explore model-parallel constructs in DML > > > Key: SYSTEMML-739 > URL: https://issues.apache.org/jira/browse/SYSTEMML-739 > Project: SystemML > Issue Type: Task >Reporter: Niketan Pansare > > An example of such construct is providing access to the parameter server. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SYSTEMML-1661) Builtin functions bias_add and bias_multiply not documented
[ https://issues.apache.org/jira/browse/SYSTEMML-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036174#comment-16036174 ] Niketan Pansare commented on SYSTEMML-1661: --- You are welcome :) > Builtin functions bias_add and bias_multiply not documented > --- > > Key: SYSTEMML-1661 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1661 > Project: SystemML > Issue Type: Bug >Reporter: Matthias Boehm >Assignee: Niketan Pansare >Priority: Minor > Fix For: SystemML 1.0 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (SYSTEMML-1661) Builtin functions bias_add and bias_multiply not documented
[ https://issues.apache.org/jira/browse/SYSTEMML-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-1661. --- Resolution: Fixed Fix Version/s: SystemML 1.0 Resolved by the commit https://github.com/apache/incubator-systemml/commit/88f4a468f48081031d926d917ebc4f3e9014fc7f > Builtin functions bias_add and bias_multiply not documented > --- > > Key: SYSTEMML-1661 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1661 > Project: SystemML > Issue Type: Bug >Reporter: Matthias Boehm >Assignee: Niketan Pansare >Priority: Minor > Fix For: SystemML 1.0 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SYSTEMML-1643) Extra jar does not include license information
[ https://issues.apache.org/jira/browse/SYSTEMML-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030631#comment-16030631 ] Niketan Pansare commented on SYSTEMML-1643: --- Thanks Glenn for looking into this. You may want to take a look at: https://github.com/apache/incubator-systemml/blob/master/pom.xml#L296 > Extra jar does not include license information > -- > > Key: SYSTEMML-1643 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1643 > Project: SystemML > Issue Type: Bug >Reporter: Arvind Surve >Assignee: Glenn Weidner > > File systemml-x.y.z-incubating-SNAPSHOT-extra.jar has default LICENSE file > but it does not contain license information for files such as generated > classes from Caffe and TensorFlow. > It will be better to have separate LICENSE file in src/assembly/extra-jar > location of repository and add it in > systemml-x.y.z-incubating-SNAPSHOT-extra.jar file. > Besides that, this jar file contains some extra files which should not be > included e.g. pom.xml etc. > [~niketanpansare] Can you please look at this? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SYSTEMML-1650) GPU cudnn produces worrisome amount of numerical instability
[ https://issues.apache.org/jira/browse/SYSTEMML-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030627#comment-16030627 ] Niketan Pansare commented on SYSTEMML-1650: --- Let's consider conv2d builtin function. Can you please try following two things: 1. First, let's validate whether the results are due to bugs in SystemML GPU backend. Eg: incorrect parameters passed to CuDNN, incorrect memory transfer, etc. A simple way to validate this is to edit the following lines: https://github.com/apache/incubator-systemml/blob/master/scripts/nn/test/grad_check.dml#L447 and replace them with {code} # Try modifying N, C, ... for further testing if you want X = matrix(seq(1, N*C*Hin*Win), rows=N, cols=C*Hin*Win) y = matrix(seq(N*F*Hin*Win, 1), rows=N, cols=F*Hin*Win) {code} 2. Second, assuming that the above test succeeds, we need to quantify how much precision we lose when using CuDNN as opposed to implementing numerically-stable conv2d using custom kernel. > GPU cudnn produces worrisome amount of numerical instability > > > Key: SYSTEMML-1650 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1650 > Project: SystemML > Issue Type: Bug > Components: Runtime >Affects Versions: SystemML 0.14 >Reporter: Nakul Jindal > Fix For: SystemML 1.0 > > > When running GPU tests (mike's run_tests.dml in the nn directory) > {code} > 17/05/30 17:24:19 INFO api.DMLScript: BEGIN DML run 05/30/2017 17:24:19 > 17/05/30 17:24:20 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 17/05/30 17:24:21 INFO context.GPUContext: Initializing CUDA > Starting grad checks. > --- > 17/05/30 17:24:22 INFO context.GPUContext: GPU memory - Total: 2096.300032 > MB, Available: 1295.97438 MB on GPUContext{deviceNum=0} > 17/05/30 17:24:22 INFO context.GPUContext: Total number of GPUs on the > machine: 1 > Grad checking the cross-entropy loss function. > Grad checking the L1 loss function. > Grad checking the L1 regularization function. > Grad checking the L2 loss function. > Grad checking the L2 regularization function. > Grad checking the log loss function. > Grad checking the affine layer with L2 loss. > - Grad checking X. > - Grad checking W. > - Grad checking b. > Grad checking the 1D batch normalization layer with L2 loss. > - Grad checking the 'train' mode. >- Grad checking X. >- Grad checking gamma. >- Grad checking beta. > - Grad checking the 'test' mode. >- Grad checking X. >- Grad checking gamma. >- Grad checking beta. > Grad checking the 2D (spatial) batch normalization layer with L2 loss. > - Grad checking the 'train' mode. >- Grad checking X. >- Grad checking gamma. >- Grad checking beta. > - Grad checking the 'test' mode. >- Grad checking X. >- Grad checking gamma. >- Grad checking beta. > Grad checking the `im2col` 2D convolutional layer with L2 loss. > 17/05/30 17:24:28 INFO Configuration.deprecation: session.id is deprecated. > Instead, use dfs.metrics.session-id > 17/05/30 17:24:28 INFO jvm.JvmMetrics: Initializing JVM Metrics with > processName=JobTracker, sessionId= > - Grad checking X. > - Grad checking W. > - Grad checking b. > Grad checking the built-in 2D convolutional layer with L2 loss. > - Grad checking X. > - Grad checking W. > WARNING: Relative error 3.063931109511093E-4 > 1.0E-4 & <= 0.01 with > -11.682479557456533 analytical vs -11.689640614065409 numerical, with lossph > 40.510115394324195 and lossmh 40.51034918713648 > WARNING: Relative error 6.785572589631694E-4 > 1.0E-4 & <= 0.01 with > -14.363880156229683 analytical vs -14.383386822913733 numerical, with lossph > 40.510088543924184 and lossmh 40.51037621166064 > WARNING: Relative error 8.117464157218959E-4 > 1.0E-4 & <= 0.01 with > -13.400658690617757 analytical vs -13.378920463225084 numerical, with lossph > 40.51009898805432 and lossmh 40.51036656646358 > WARNING: Relative error 6.785567321010216E-4 > 1.0E-4 & <= 0.01 with > -14.37300870216048 analytical vs -14.39252775057298 numerical, with lossph > 40.510088452456074 and lossmh 40.510376303011085 > WARNING: Relative error 0.0023065358169588085 > 1.0E-4 & <= 0.01 with > -15.081214796672182 analytical vs -15.011804170583785 numerical, with lossph > 40.510081360786614 and lossmh 40.510381596870026 > WARNING: Relative error 1.2020843619724922E-4 > 1.0E-4 & <= 0.01 with > -14.602099111310885 analytical vs -14.60561012436301 numerical, with lossph > 40.51008637609418 and lossmh 40.51037848829 > WARNING: Relative error 3.063921242335014E-4 > 1.0E-4 & <= 0.01 with > -11.654549775926586 analytical vs -11.66169368929104 numerical, with lossph > 40.51011567395115 and lossmh 40.510348907824934 > - Grad
[jira] [Resolved] (SYSTEMML-1630) Remove confusing warning for disabled native BLAS
[ https://issues.apache.org/jira/browse/SYSTEMML-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-1630. --- Resolution: Fixed Fix Version/s: SystemML 1.0 Fixed by the commit https://github.com/apache/incubator-systemml/commit/fd41f0253810a8d0ea62e4fb21854229dfe526ef > Remove confusing warning for disabled native BLAS > - > > Key: SYSTEMML-1630 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1630 > Project: SystemML > Issue Type: Bug >Affects Versions: SystemML 1.0 >Reporter: Matthias Boehm >Assignee: Niketan Pansare >Priority: Trivial > Fix For: SystemML 1.0 > > > We should not raise BLAS-related warnings if it's explicitly or by default > disabled. Let's use this warning (after rewording) only if BLAS is enabled > but cannot be loaded. > {code} > WARN utils.NativeHelper: Using internal Java BLAS as native BLAS support the > configuration 'native.blas'=none. > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (SYSTEMML-1632) Support loading and saving models via mllearn
[ https://issues.apache.org/jira/browse/SYSTEMML-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niketan Pansare resolved SYSTEMML-1632. --- Resolution: Fixed Fix Version/s: SystemML 1.0 Fixed by the commit https://github.com/apache/incubator-systemml/commit/d69f3441c8243ddd13dd3da6aab9c2d5701c6e50 > Support loading and saving models via mllearn > - > > Key: SYSTEMML-1632 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1632 > Project: SystemML > Issue Type: New Feature >Reporter: Niketan Pansare >Assignee: Niketan Pansare > Fix For: SystemML 1.0 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (SYSTEMML-1632) Support loading and saving models via mllearn
Niketan Pansare created SYSTEMML-1632: - Summary: Support loading and saving models via mllearn Key: SYSTEMML-1632 URL: https://issues.apache.org/jira/browse/SYSTEMML-1632 Project: SystemML Issue Type: New Feature Reporter: Niketan Pansare Assignee: Niketan Pansare -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (SYSTEMML-1611) Support fused weight update operators (similar to codegen on CP)
Niketan Pansare created SYSTEMML-1611: - Summary: Support fused weight update operators (similar to codegen on CP) Key: SYSTEMML-1611 URL: https://issues.apache.org/jira/browse/SYSTEMML-1611 Project: SystemML Issue Type: Sub-task Reporter: Niketan Pansare -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (SYSTEMML-1610) Support alternative algorithms for CuDNN operators such as convolution
Niketan Pansare created SYSTEMML-1610: - Summary: Support alternative algorithms for CuDNN operators such as convolution Key: SYSTEMML-1610 URL: https://issues.apache.org/jira/browse/SYSTEMML-1610 Project: SystemML Issue Type: Sub-task Reporter: Niketan Pansare Assignee: Niketan Pansare https://devblogs.nvidia.com/parallelforall/cudnn-v2-higher-performance-deep-learning-gpus/ -- This message was sent by Atlassian JIRA (v6.3.15#6346)