damccorm commented on code in PR #36346:
URL: https://github.com/apache/beam/pull/36346#discussion_r2406503209
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/options/PipelineOptions.java:
##########
@@ -413,6 +414,40 @@ public Long create(PipelineOptions options) {
void setUserAgent(String userAgent);
+ /**
+ * A string defining whether GroupByKey transforms should be replaced by
GroupByEncryptedKey
+ *
+ * <p>Beam will infer the secret type and value based on the secret itself.
This guarantees that
+ * any data at rest during the performing a GBK, so this can be used to
guarantee that data is not
+ * unencrypted. Runners with this behavior include the Dataflow, Flink, and
Spark runners. The
+ * option should be structured like:
+ *
+ * <pre><code>
+ * --encrypt=type:<secret_type>;<secret_param>:<value>
+ * </code></pre>
+ *
+ * for example:
+ *
+ * <pre><code>
+ * --encrypt=type:GcpSecret;version_name:my_secret/versions/latest"
+ * </code></pre>
+ *
+ * All variables should use snake case to allow consistency across languages.
+ */
+ @Description(
+ "When set, will replace all GroupByKey transforms in the pipeline the
option. Beam will"
+ + " infer the secret type and value based on the secret itself. This
guarantees that"
+ + " any data at rest during the performing a GBK, so this can be
used to guarantee"
+ + " that data is not unencrypted. Runners with this behavior include
the Dataflow,"
+ + " Flink, and Spark runners. The option should be structured like:"
+ + " --encrypt=type:<secret_type>;<secret_param>:<value>, for example
"
+ + " --encrypt=type:GcpSecret;version_name:my_secret/versions/latest.
All variables "
+ + " should use snake case to allow consistency across languages.")
Review Comment:
Fixed
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupByEncryptedKey.java:
##########
@@ -67,7 +77,26 @@ private GroupByEncryptedKey(Secret hmacKey) {
* @return A {@link GroupByEncryptedKey} transform.
*/
public static <K, V> GroupByEncryptedKey<K, V> create(Secret hmacKey) {
- return new GroupByEncryptedKey<>(hmacKey);
+ GroupByKey<byte[], KV<byte[], byte[]>> gbk = GroupByKey.create();
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
