Baunsgaard closed pull request #992:
URL: https://github.com/apache/systemds/pull/992
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
Baunsgaard commented on pull request #992:
URL: https://github.com/apache/systemds/pull/992#issuecomment-687754049
Closing, because of plans to update Spark version after 2.0 release of
systemds.
This is an
Baunsgaard closed pull request #1016:
URL: https://github.com/apache/systemds/pull/1016
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
Baunsgaard commented on pull request #1047:
URL: https://github.com/apache/systemds/pull/1047#issuecomment-687735766
agree! also really like the PR!
This is an automated message from the Apache Git Service.
To respond to the
Baunsgaard opened a new pull request #1049:
URL: https://github.com/apache/systemds/pull/1049
Minor changes in Python API startup for ease of startup if systemds is
installed somewhere else it will use that systemds.
This practically means that if you have systemds home set, it will
Baunsgaard commented on pull request #1047:
URL: https://github.com/apache/systemds/pull/1047#issuecomment-687736854
But the issue is that it is hard / impossible to debug the processes while
it is possible when inside the same JVM using threads, and in that context if
we really want to
mboehm7 commented on pull request #1036:
URL: https://github.com/apache/systemds/pull/1036#issuecomment-687819591
LGTM - thanks for the additional builtin function @Shafaq-Siddiqi. I only
slightly modified the formatting and changed the for loops over columns to
parfor loops.
asfgit closed pull request #1036:
URL: https://github.com/apache/systemds/pull/1036
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
mboehm7 commented on pull request #1046:
URL: https://github.com/apache/systemds/pull/1046#issuecomment-687847007
LGTM - thanks @kev-inn for the added impute, and general clean. I modified
the Encoder interface though to keep it independent of data structures from the
federated backend.
asfgit closed pull request #1046:
URL: https://github.com/apache/systemds/pull/1046
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
kev-inn opened a new pull request #1047:
URL: https://github.com/apache/systemds/pull/1047
Adds a new function to start multiple federated workers. The function first
starts multiple processes and then waits for all of them to be ready for a
connection by pinging (by sending a `CLEAR`)
Shafaq-Siddiqi closed pull request #988:
URL: https://github.com/apache/systemds/pull/988
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486439575
##
File path: src/main/java/org/apache/sysds/parser/DataExpression.java
##
@@ -2239,9 +2251,44 @@ public boolean isRead()
* Sets privacy of
Baunsgaard opened a new pull request #1052:
URL: https://github.com/apache/systemds/pull/1052
Fix bug in right multiplication OLE where the disabled skiplist would
break the execution, because it was disabled.
This is an
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486439247
##
File path:
src/main/java/org/apache/sysds/runtime/instructions/cp/MultiReturnBuiltinCPInstruction.java
##
@@ -47,6 +47,10 @@ public CPOperand
sebwrede opened a new pull request #1053:
URL: https://github.com/apache/systemds/pull/1053
The privacy handling of UDFs is moved from the individual instruction
classes to the FederatedWorkerHandler.
This will handle the privacy constraints of the input before any execution
of the UDF
sebwrede opened a new pull request #1054:
URL: https://github.com/apache/systemds/pull/1054
Exceptions added to the FederatedResponse risk exposing data from the
federated worker. Exceptions need to be caught and then a new exception could
be created and added to the FederatedResponse
Baunsgaard merged pull request #1052:
URL: https://github.com/apache/systemds/pull/1052
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
sebwrede opened a new pull request #1051:
URL: https://github.com/apache/systemds/pull/1051
This PR replaces the [Fine-Grained Privacy Constraints 2
PR](https://github.com/apache/systemds/pull/985).
This PR adds fine-grained privacy constraints and adapts the privacy
constraint
sebwrede closed pull request #985:
URL: https://github.com/apache/systemds/pull/985
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
sebwrede commented on pull request #985:
URL: https://github.com/apache/systemds/pull/985#issuecomment-689635208
This PR has been replaced by https://github.com/apache/systemds/pull/1051.
This is an automated message from
tobiasrieger opened a new pull request #1050:
URL: https://github.com/apache/systemds/pull/1050
To transmit the parameter ListObject of MatrixObjects to the federated
Workers it was necessary to serialize a list. This pull request also contains a
unit test, with a recursive call by
kev-inn commented on pull request #1047:
URL: https://github.com/apache/systemds/pull/1047#issuecomment-688257975
> But the issue is that it is hard / impossible to debug the processes while
it is possible when inside the same JVM using threads, and in that context if
we really want to
kev-inn edited a comment on pull request #1047:
URL: https://github.com/apache/systemds/pull/1047#issuecomment-688257975
> But the issue is that it is hard / impossible to debug the processes while
it is possible when inside the same JVM using threads, and in that context if
we really
Baunsgaard merged pull request #1052:
URL: https://github.com/apache/systemds/pull/1052
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
sebwrede commented on a change in pull request #1054:
URL: https://github.com/apache/systemds/pull/1054#discussion_r487786099
##
File path:
src/main/java/org/apache/sysds/runtime/controlprogram/federated/FederatedWorkerHandler.java
##
@@ -155,7 +155,7 @@ private
Shafaq-Siddiqi opened a new pull request #1055:
URL: https://github.com/apache/systemds/pull/1055
This commit contains,
1. Optimizer for cleaning pipelines
2. Minor built-ins imputeByMean, imputeByMedian, frameSort, vectorToCsv.dml
3. minor fixes for resolving warnings in different
asfgit closed pull request #1051:
URL: https://github.com/apache/systemds/pull/1051
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
mboehm7 commented on pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#issuecomment-691663906
LGTM. Thanks for the patch @sebwrede and reconciling the tests with the new
federated backend. I only made some minor changes: removed warnings, fixed few
formatting issues,
mboehm7 commented on pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#issuecomment-691664020
Also thanks to @Baunsgaard for the earlier review.
This is an automated message from the Apache Git Service.
mboehm7 commented on pull request #1053:
URL: https://github.com/apache/systemds/pull/1053#issuecomment-691666709
LGTM - thanks @sebwrede
This is an automated message from the Apache Git Service.
To respond to the message,
asfgit closed pull request #1053:
URL: https://github.com/apache/systemds/pull/1053
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
mboehm7 commented on a change in pull request #1054:
URL: https://github.com/apache/systemds/pull/1054#discussion_r487525755
##
File path:
src/main/java/org/apache/sysds/runtime/controlprogram/federated/FederatedWorkerHandler.java
##
@@ -155,7 +155,7 @@ private
phaniarnab merged pull request #1048:
URL: https://github.com/apache/systemds/pull/1048
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
phaniarnab opened a new pull request #1048:
URL: https://github.com/apache/systemds/pull/1048
PR to check if tests are passing.
This is an automated message from the Apache Git Service.
To respond to the message, please log
kev-inn opened a new pull request #1046:
URL: https://github.com/apache/systemds/pull/1046
Adds support for federated execution for the final encoder
`EncoderMVImpute`. This should finish support for federated transform
operations (perf and improvements being TODO).
##
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486407377
##
File path:
src/main/java/org/apache/sysds/runtime/instructions/spark/SPInstruction.java
##
@@ -41,16 +41,15 @@
}
protected final
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486373173
##
File path: src/main/java/org/apache/sysds/runtime/instructions/Instruction.java
##
@@ -38,7 +41,21 @@
FEDERATED
}
-
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486376414
##
File path: src/main/java/org/apache/sysds/parser/DataExpression.java
##
@@ -2239,9 +2251,44 @@ public boolean isRead()
* Sets privacy of
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486379692
##
File path:
src/main/java/org/apache/sysds/runtime/controlprogram/federated/FederatedWorkerHandler.java
##
@@ -280,8 +281,9 @@ private
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486383020
##
File path:
src/main/java/org/apache/sysds/runtime/instructions/cp/MultiReturnBuiltinCPInstruction.java
##
@@ -47,6 +47,10 @@ public CPOperand
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486385202
##
File path:
src/main/java/org/apache/sysds/runtime/instructions/spark/SPInstruction.java
##
@@ -41,16 +41,15 @@
}
protected final
Baunsgaard commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486391285
##
File path:
src/main/java/org/apache/sysds/runtime/instructions/spark/SPInstruction.java
##
@@ -41,16 +41,15 @@
}
protected
Baunsgaard commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486393184
##
File path:
src/main/java/org/apache/sysds/runtime/instructions/cp/MultiReturnBuiltinCPInstruction.java
##
@@ -47,6 +47,10 @@ public CPOperand
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486393369
##
File path:
src/main/java/org/apache/sysds/runtime/privacy/FineGrained/FineGrainedPrivacyList.java
##
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the
Baunsgaard commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486393950
##
File path: src/main/java/org/apache/sysds/parser/DataExpression.java
##
@@ -2239,9 +2251,44 @@ public boolean isRead()
* Sets privacy of
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486393369
##
File path:
src/main/java/org/apache/sysds/runtime/privacy/FineGrained/FineGrainedPrivacyList.java
##
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486396715
##
File path: src/main/java/org/apache/sysds/runtime/privacy/PrivacyPropagator.java
##
@@ -19,15 +19,19 @@
package org.apache.sysds.runtime.privacy;
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486396082
##
File path:
src/main/java/org/apache/sysds/runtime/privacy/FineGrained/FineGrainedPrivacyList.java
##
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486399379
##
File path: src/main/java/org/apache/sysds/runtime/privacy/PrivacyPropagator.java
##
@@ -414,4 +591,25 @@ else if ( inst instanceof SqlCPInstruction
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486400993
##
File path:
src/test/java/org/apache/sysds/test/functions/privacy/BuiltinGLMTest.java
##
@@ -228,6 +229,9 @@ public void
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486402441
##
File path: src/test/java/org/apache/sysds/test/functions/privacy/GLMTest.java
##
@@ -154,7 +155,10 @@ public GLMTest (int numRecords_, int
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486403288
##
File path:
src/main/java/org/apache/sysds/runtime/instructions/cp/MultiReturnBuiltinCPInstruction.java
##
@@ -47,6 +47,10 @@ public CPOperand
Baunsgaard commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486337594
##
File path: src/main/java/org/apache/sysds/runtime/instructions/Instruction.java
##
@@ -38,7 +41,21 @@
FEDERATED
}
sebwrede commented on a change in pull request #1051:
URL: https://github.com/apache/systemds/pull/1051#discussion_r486369426
##
File path: src/main/java/org/apache/sysds/runtime/instructions/Instruction.java
##
@@ -38,7 +41,21 @@
FEDERATED
}
-
Baunsgaard commented on a change in pull request #1050:
URL: https://github.com/apache/systemds/pull/1050#discussion_r486332722
##
File path:
src/test/java/org/apache/sysds/test/component/paramserv/SerializationTest.java
##
@@ -19,7 +19,9 @@
package
tobiasrieger commented on a change in pull request #1050:
URL: https://github.com/apache/systemds/pull/1050#discussion_r486364637
##
File path:
src/test/java/org/apache/sysds/test/component/paramserv/SerializationTest.java
##
@@ -19,7 +19,9 @@
package
OlgaOvcharenko commented on pull request #1040:
URL: https://github.com/apache/systemds/pull/1040#issuecomment-706165102
@Baunsgaard I also pushed modified FederationUtils (with fed min, max, sum,
mean), because it seems like privacy tests are failing without this changes
mboehm7 commented on pull request #1079:
URL: https://github.com/apache/systemds/pull/1079#issuecomment-711149500
LGTM - thanks @XorSum for catching this issue and the related fix. I'll
merge this in, as we indeed removed unnecessary entries in this instruction (in
the past this
asfgit closed pull request #1079:
URL: https://github.com/apache/systemds/pull/1079
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
XorSum opened a new pull request #1079:
URL: https://github.com/apache/systemds/pull/1079
When i use `CostEstimator` to estimate the time cost of a programblock, an
error occurred:
``` java
java.lang.NumberFormatException: For input string: "copy"
at
Baunsgaard merged pull request #1080:
URL: https://github.com/apache/systemds/pull/1080
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
Baunsgaard closed pull request #1076:
URL: https://github.com/apache/systemds/pull/1076
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
Baunsgaard commented on pull request #1076:
URL: https://github.com/apache/systemds/pull/1076#issuecomment-712105055
Hi @manushree635
Thanks for the guide, I have taken a look and decided to change the install
to use Brew.
Therefore i changed it completely, while also following
OlgaOvcharenko opened a new pull request #1081:
URL: https://github.com/apache/systemds/pull/1081
This PR adds federated right indexing.
This is an automated message from the Apache Git Service.
To respond to the
Baunsgaard merged pull request #1077:
URL: https://github.com/apache/systemds/pull/1077
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
Baunsgaard commented on pull request #1078:
URL: https://github.com/apache/systemds/pull/1078#issuecomment-711303063
Closing based on comments
This is an automated message from the Apache Git Service.
To respond to the
Baunsgaard opened a new pull request #1080:
URL: https://github.com/apache/systemds/pull/1080
[SYSTEMDS-2686] Compressed overlapping column groups
This commit change the compressed right multiplication to perform in
compressed space, resulting in significantly faster execution.
Baunsgaard opened a new pull request #1082:
URL: https://github.com/apache/systemds/pull/1082
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
Baunsgaard commented on pull request #1040:
URL: https://github.com/apache/systemds/pull/1040#issuecomment-706551018
@OlgaOvcharenko Thanks for the PR. :1st_place_medal:
While merging i fixed the indentation in the test files, and reduced the
number of tests run on RowCol aggregate
Baunsgaard opened a new pull request #1078:
URL: https://github.com/apache/systemds/pull/1078
This PR is sort of an question.
Is it okay for our MKL to assume that the output matrix is fully dense with
all values !=0?
If so then we have an potential improved performance.
Baunsgaard opened a new pull request #1077:
URL: https://github.com/apache/systemds/pull/1077
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
manushree635 opened a new pull request #1076:
URL: https://github.com/apache/systemds/pull/1076
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
Baunsgaard closed pull request #1040:
URL: https://github.com/apache/systemds/pull/1040
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
mboehm7 commented on pull request #1070:
URL: https://github.com/apache/systemds/pull/1070#issuecomment-706555293
LGTM - thanks for the new builtin functions @gabrielaozegovic. This
completes the AMLS project.
Just for the record, during the merge, I made the following
asfgit closed pull request #1070:
URL: https://github.com/apache/systemds/pull/1070
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
mboehm7 commented on pull request #1078:
URL: https://github.com/apache/systemds/pull/1078#issuecomment-706566376
The current runtime/compiler requires the non-zeros to be exact or unknown
(-1) - so we cannot simply set it to fully dense. This information could be
mistakenly used (e.g.,
asfgit closed pull request #1071:
URL: https://github.com/apache/systemds/pull/1071
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
gabrielaozegovic commented on pull request #1070:
URL: https://github.com/apache/systemds/pull/1070#issuecomment-706074537
I pushed my latest changes to the branch.
I did output verification, which I think should be fine since it's matrix
factorization.
For some reason, one test is
mboehm7 commented on pull request #1078:
URL: https://github.com/apache/systemds/pull/1078#issuecomment-706680533
well, for now I would recommend to properly maintain the non-zeros. It can
only become a moderate overhead if the output is large compared to the compute
(cubic compute and
OlgaOvcharenko commented on pull request #1040:
URL: https://github.com/apache/systemds/pull/1040#issuecomment-706165102
@Baunsgaard I also pushed modified FederationUtils (with fed min, max, sum,
mean), because it seems like privacy tests are failing without this changes
phaniarnab closed pull request #1074:
URL: https://github.com/apache/systemds/pull/1074
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
gabrielaozegovic commented on pull request #1070:
URL: https://github.com/apache/systemds/pull/1070#issuecomment-706074537
I pushed my latest changes to the branch.
I did output verification, which I think should be fine since it's matrix
factorization.
For some reason, one test is
Baunsgaard closed pull request #1038:
URL: https://github.com/apache/systemds/pull/1038
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
Shafaq-Siddiqi opened a new pull request #1034:
URL: https://github.com/apache/systemds/pull/1034
…on a Frame.
The built-in use the Janino compiler for run-time code generation and
compilation and accepts a frame and string (containing Java code) as input and
execute the code in
OlgaOvcharenko opened a new pull request #1035:
URL: https://github.com/apache/systemds/pull/1035
New univar and bivar builtins. Also fixed federated min and max.
This is an automated message from the Apache Git Service.
To
OlgaOvcharenko closed pull request #1039:
URL: https://github.com/apache/systemds/pull/1039
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
kev-inn opened a new pull request #1027:
URL: https://github.com/apache/systemds/pull/1027
Adds decode support for recode, pass-through and composite (containing only
those two).
This is an automated message from the Apache
asfgit closed pull request #1028:
URL: https://github.com/apache/systemds/pull/1028
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
mboehm7 opened a new pull request #1028:
URL: https://github.com/apache/systemds/pull/1028
This patch fixes two major issues of request interference from multiple
coordinator threads.
First, we now properly maintain separate execution context at the
federated site for different
Baunsgaard opened a new pull request #1029:
URL: https://github.com/apache/systemds/pull/1029
Trials for caching in tests using custom actions
This is an automated message from the Apache Git Service.
To respond to the
Baunsgaard opened a new pull request #1042:
URL: https://github.com/apache/systemds/pull/1042
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
Baunsgaard opened a new pull request #1044:
URL: https://github.com/apache/systemds/pull/1044
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
Baunsgaard merged pull request #1042:
URL: https://github.com/apache/systemds/pull/1042
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
Baunsgaard closed pull request #1043:
URL: https://github.com/apache/systemds/pull/1043
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
Baunsgaard commented on pull request #1043:
URL: https://github.com/apache/systemds/pull/1043#issuecomment-682534142
merged
This is an automated message from the Apache Git Service.
To respond to the message, please log on
Baunsgaard opened a new pull request #1043:
URL: https://github.com/apache/systemds/pull/1043
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
Baunsgaard merged pull request #1044:
URL: https://github.com/apache/systemds/pull/1044
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
mboehm7 commented on pull request #1041:
URL: https://github.com/apache/systemds/pull/1041#issuecomment-683141558
LGTM - thanks @gilgenbergg for the update and tests, as well as
incorporating the licenses.
This is an
asfgit closed pull request #1031:
URL: https://github.com/apache/systemds/pull/1031
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
1 - 100 of 3467 matches
Mail list logo
