damccorm commented on code in PR #36346:
URL: https://github.com/apache/beam/pull/36346#discussion_r2409082514
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupByKey.java:
##########
@@ -244,6 +264,20 @@ public PCollection<KV<K, Iterable<V>>>
expand(PCollection<KV<K, V>> input) {
throw new IllegalStateException("the keyCoder of a GroupByKey must be
deterministic", e);
}
+ PipelineOptions options = input.getPipeline().getOptions();
+ String gbekOveride = options.getGBEK();
+ if (!this.insideGBEK && gbekOveride != null &&
!gbekOveride.trim().isEmpty()) {
Review Comment:
If I do this here (and in DataflowGroupByKey), it doesn't compile because we
don't have the explicit null check:
```
/Users/dannymccormick/beam/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupByKey.java:272:
error: [argument] incompatible argument for parameter secretOption of
Secret.parseSecretOption.
Secret hmacSecret = Secret.parseSecretOption(gbekOveride);
```
It seems cleaner to me to leave as is vs working around it.
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/internal/DataflowGroupByKey.java:
##########
@@ -117,6 +140,20 @@ public PCollection<KV<K, Iterable<V>>>
expand(PCollection<KV<K, V>> input) {
"the keyCoder of a DataflowGroupByKey must be deterministic", e);
}
+ PipelineOptions options = input.getPipeline().getOptions();
+ String gbekOveride = options.getGBEK();
+ if (!this.insideGBEK && gbekOveride != null &&
!gbekOveride.trim().isEmpty()) {
Review Comment:
Same thing here as in GroupByKey above
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/util/Secret.java:
##########
@@ -33,4 +38,48 @@ public interface Secret extends Serializable {
* @return The secret as a byte array.
*/
byte[] getSecretBytes();
+
+ static Secret parseSecretOption(String secretOption) {
+ Map<String, String> paramMap = new HashMap<>();
+ for (String param : secretOption.split(";", -1)) {
+ String[] parts = param.split(":", 2);
+ if (parts.length == 2) {
+ paramMap.put(parts[0], parts[1]);
+ }
+ }
+
+ if (!paramMap.containsKey("type")) {
+ throw new RuntimeException("Secret string must contain a valid type
parameter");
+ }
+
+ String secretType = paramMap.get("type");
+ paramMap.remove("type");
+
+ if (secretType == null) {
Review Comment:
If I don't have the explicit null check, this will not compile unfortunately:
```
/Users/dannymccormick/beam/sdks/java/core/src/main/java/org/apache/beam/sdk/util/Secret.java:61:
error: [dereference.of.nullable] dereference of possibly-null reference
secretType
switch (secretType.toLowerCase()) {
```
It seems cleaner to just leave it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
