sebwrede commented on a change in pull request #1525:
URL: https://github.com/apache/systemds/pull/1525#discussion_r793524481
##########
File path:
src/main/java/org/apache/sysds/runtime/controlprogram/federated/FederatedWorkerHandler.java
##########
@@ -417,9 +417,11 @@ private FederatedResponse execUDF(FederatedRequest
request, ExecutionContextMap
ExecutionContext ec = ecm.get(request.getTID());
// get function and input parameters
+ // FIXME: handle HE privacy constraints
Review comment:
When the `-federatedCompilation` flag is added to the execution of the
DML script, the rewriter `RewriteFederatedExecution.java` will be called where
the privacy constraints will be retrieved from the federated workers for all
federated data inputs in the script. You need to add a check for when the
privacy level "Private" is set for an input to the parameter server so that you
can switch to the encrypted version of the federated parameter server.
In the FederatedWorkerHandler, the call to the PrivacyMonitor needs to check
if the UDF received is the UDF from the encrypted version of the parameter
server. If this is the case, the PrivacyMonitor should not block data inputs
with the privacy level "Private". For all other UDFs or instructions, the
PrivacyMonitor should still block requests for the "Private" data elements,
since the data elements will be transferred unencrypted to the coordinator
otherwise.
The idea behind the PrivacyMonitor check in the FederatedWorkerHandler is
that the UDF has some input data and the privacy constraints of that input is
checked. It is the responsibility of the SystemDS developer to ensure that no
other data is exposed when running the UDF since it is possible to access other
data in the UDF execution without checking the privacy level of those data
elements.
Regarding the propagation of the privacy constraints, this happens in the
`PrivacyPropagator.java` class. During runtime, CPInstructions will, before
execution, propagate the privacy constraints from the inputs to the outputs. In
your case, I don't think that you execute any CPInstructions since you only
send a UDF. In the UDF, you need to handle the propagation if you leave any
output on the federated workers. You can add such propagation logic to the
PrivacyPropagator class and reuse the logic in the corePropagation method by
calling that method. If you do not leave any output on the federated workers
then you do not have to think about the propagation, but you still have to
think about which data elements you send back to the coordinator. If the data
has the privacy level "Private" then you should only transfer the encrypted
version of it. The encrypted data does not need the privacy constraint unless
the decryption key is transferred also (which it is not because only the fede
rated workers can fully decrypt it). When the federated workers decrypt the
aggregated data sent from the coordinator, you need to add a privacy constraint
to that output so that it is not sent back to the coordinator without being
encrypted again. The output should again have the privacy level "Private".
This is now a long comment :smile: . Please ask if anything is unclear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
