mboehm7 commented on a change in pull request #1525:
URL: https://github.com/apache/systemds/pull/1525#discussion_r793032449
##########
File path:
src/main/java/org/apache/sysds/runtime/controlprogram/paramserv/homomorphicEncryption/SEALServer.java
##########
@@ -0,0 +1,84 @@
+package
org.apache.sysds.runtime.controlprogram.paramserv.homomorphicEncryption;
+
+import org.apache.sysds.common.Types;
+import org.apache.sysds.hops.OptimizerUtils;
+import org.apache.sysds.runtime.controlprogram.caching.MatrixObject;
+import org.apache.sysds.runtime.data.DenseBlock;
+import org.apache.sysds.runtime.data.DenseBlockFactory;
+import org.apache.sysds.runtime.instructions.cp.CiphertextMatrix;
+import org.apache.sysds.runtime.instructions.cp.Encrypted;
+import org.apache.sysds.runtime.instructions.cp.PlaintextMatrix;
+import org.apache.sysds.runtime.matrix.data.MatrixBlock;
+import org.apache.sysds.runtime.meta.DataCharacteristics;
+import org.apache.sysds.runtime.meta.MetaDataFormat;
+import org.apache.sysds.utils.NativeHelper;
+
+import java.util.Arrays;
+
+public class SEALServer {
+ public SEALServer() {
+ // TODO take params here, like slot_count etc.
+ ctx = NativeHelper.initServer();
+ }
+
+ // this is a pointer to the context used by all native methods of this
class
+ private final long ctx;
+ private byte[] _a;
+
+ // NOTICE: all long[] arys here have to be of size SEAL slot_count
+ // they represent the data of one Ciphertext object
+ // NOTICE: all double[] arys here have to be half the size of SEAL
slot_count
+ // they represent the data of one Plaintext object
+
+ // this generates the a constant. in a future version we want to generate
this together with the clients to prevent misuse
+ public synchronized byte[] generateA() {
+ if (_a == null) {
+ _a = NativeHelper.generateA(ctx);
+ }
+ return _a;
+ }
+
+ // accumulates the given partial public keys into a public key, stores it
in ctx and returns it
+ public PublicKey aggregatePartialPublicKeys(PublicKey[]
partial_public_keys) {
+ return new PublicKey(NativeHelper.aggregatePartialPublicKeys(ctx,
extractRawData(partial_public_keys)));
+ }
+
+ // accumulates the given ciphertext blocks into a sum ciphertext and
returns it
+ // stores c0 of the sum to be used in averageBlocks()
+ public CiphertextMatrix accumulateCiphertexts(CiphertextMatrix[]
ciphertexts) {
+ return new CiphertextMatrix(ciphertexts[0].getDims(),
ciphertexts[0].getDataCharacteristics(),
NativeHelper.accumulateCiphertexts(ctx, extractRawData(ciphertexts)));
+ }
+
+ // averages the partial decryptions and stores the result in old_mo
+ // encrypted_sum is the result of accumulateCiphertexts() and
partial_plaintexts is the result of partiallyDecryptBlock
+ // of each ciphertext fed into accumulateCiphertexts
+ public MatrixObject average(CiphertextMatrix encrypted_sum,
PlaintextMatrix[] partial_plaintexts) {
+ double[] raw_result = NativeHelper.average(ctx,
encrypted_sum.getData(), extractRawData(partial_plaintexts));
+ int[] dims = encrypted_sum.getDims();
+ int result_len = Arrays.stream(dims).reduce(1, (x,y) -> x*y);
+ DataCharacteristics dc = encrypted_sum.getDataCharacteristics();
+
+ DenseBlock new_dense_block =
DenseBlockFactory.createDenseBlock(Arrays.copyOf(raw_result, result_len), dims);
+ MatrixBlock new_matrix_block = new MatrixBlock((int)dc.getRows(),
(int)dc.getCols(), new_dense_block);
+ MatrixObject new_mo = new MatrixObject(Types.ValueType.FP64,
OptimizerUtils.getUniqueTempFileName(), new MetaDataFormat(dc,
Types.FileFormat.BINARY), new_matrix_block);
+ new_mo.exportData(); // write data, otherwise it might get evicted and
thus get lost
Review comment:
Great progress, for now I just quickly answer the question before doing
a full code review in the next days.
The constructor that takes a MatrixBlock bypassed our default eviction
logic. Please, create a MatrixObject new_mo only with the meta data, and then
call new_mo.acquireModify(new_matrix_block) which correctly hands it over to
the buffer pool, where it is evicted and restored as needed. An acquireRead
then pins the object in memory, and release() unpins it, while
acquireReadAndRelease is a shorthand to get the MatrixBlock. Conceptually a
MatrixObject is an abstraction for hiding local vs distributed vs federated
data, evictions, and multi-device replication; while a MatrixBlock is a plain
old java object for in-memory matrices of different sparse/dense/compressed
formats.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
