[jira] [Work logged] (BEAM-7246) Create a Spanner IO for Python
[ https://issues.apache.org/jira/browse/BEAM-7246?focusedWorklogId=387971=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387971 ] ASF GitHub Bot logged work on BEAM-7246: Author: ASF GitHub Bot Created on: 15/Feb/20 18:37 Start Date: 15/Feb/20 18:37 Worklog Time Spent: 10m Work Description: mszb commented on issue #10712: [BEAM-7246] Added Google Spanner Write Transform URL: https://github.com/apache/beam/pull/10712#issuecomment-586629057 @chamikaramj @nielm could you please verify the changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387971) Time Spent: 21h 40m (was: 21.5h) > Create a Spanner IO for Python > -- > > Key: BEAM-7246 > URL: https://issues.apache.org/jira/browse/BEAM-7246 > Project: Beam > Issue Type: Bug > Components: io-py-gcp >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 21h 40m > Remaining Estimate: 0h > > Add I/O support for Google Cloud Spanner for the Python SDK (Batch Only). > Testing in this work item will be in the form of DirectRunner tests and > manual testing. > Integration and performance tests are a separate work item (not included > here). > See https://beam.apache.org/documentation/io/built-in/. The goal is to add > Google Clound Spanner to the Database column for the Python/Batch row. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9314) dot output should be deterministic
[ https://issues.apache.org/jira/browse/BEAM-9314?focusedWorklogId=387953=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387953 ] ASF GitHub Bot logged work on BEAM-9314: Author: ASF GitHub Bot Created on: 15/Feb/20 17:21 Start Date: 15/Feb/20 17:21 Worklog Time Spent: 10m Work Description: happygiraffe commented on issue #10864: [BEAM-9314] Make dot output deterministic URL: https://github.com/apache/beam/pull/10864#issuecomment-586622050 Thanks. I suspected there is more to do here, but this worked for my (simple) use case. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387953) Time Spent: 40m (was: 0.5h) > dot output should be deterministic > -- > > Key: BEAM-9314 > URL: https://issues.apache.org/jira/browse/BEAM-9314 > Project: Beam > Issue Type: Sub-task > Components: sdk-go >Reporter: Dominic Mitchell >Priority: Trivial > Time Spent: 40m > Remaining Estimate: 0h > > Presently the dot output changes each time it's executed. The output should > be stable (given the inputs) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8847) Handle the registration request synchronously in the Java SDK harness
[ https://issues.apache.org/jira/browse/BEAM-8847?focusedWorklogId=387922=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387922 ] ASF GitHub Bot logged work on BEAM-8847: Author: ASF GitHub Bot Created on: 15/Feb/20 13:07 Start Date: 15/Feb/20 13:07 Worklog Time Spent: 10m Work Description: sunjincheng121 commented on issue #10243: [BEAM-8847] Handle the registration request synchronously in the Java… URL: https://github.com/apache/beam/pull/10243#issuecomment-586588696 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387922) Time Spent: 1h 20m (was: 1h 10m) > Handle the registration request synchronously in the Java SDK harness > - > > Key: BEAM-8847 > URL: https://issues.apache.org/jira/browse/BEAM-8847 > Project: Beam > Issue Type: Bug > Components: sdk-java-harness >Affects Versions: 2.18.0 >Reporter: sunjincheng >Assignee: sunjincheng >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently the registration request is handled asynchronously in the Java SDK > harness. As discussed in BEAM-8733, this JIRA. tries to change it to > synchronous. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9035) Typed options for Row Schema and Fields
[
https://issues.apache.org/jira/browse/BEAM-9035?focusedWorklogId=387887=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387887
]
ASF GitHub Bot logged work on BEAM-9035:
Author: ASF GitHub Bot
Created on: 15/Feb/20 08:34
Start Date: 15/Feb/20 08:34
Worklog Time Spent: 10m
Work Description: alexvanboxel commented on pull request #10413:
[BEAM-9035] Typed options for Row Schema and Field
URL: https://github.com/apache/beam/pull/10413#discussion_r379804929
##
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/Schema.java
##
@@ -950,6 +1007,338 @@ public int hashCode() {
}
}
+ public static class Options implements Serializable {
+private Map options;
+
+@Override
+public String toString() {
+ TreeMap sorted = new TreeMap(options);
+ return "{" + sorted + '}';
+}
+
+Map getAllOptions() {
+ return options;
+}
+
+public Set getOptionNames() {
+ return options.keySet();
+}
+
+public boolean hasOptions() {
+ return options.size() > 0;
+}
+
+@Override
+public boolean equals(Object o) {
+ if (this == o) {
+return true;
+ }
+ if (o == null || getClass() != o.getClass()) {
+return false;
+ }
+ Options options1 = (Options) o;
+ if (!options.keySet().equals(options1.options.keySet())) {
+return false;
+ }
+ for (Map.Entry optionEntry : options.entrySet()) {
+Option thisOption = optionEntry.getValue();
+Option otherOption = options1.options.get(optionEntry.getKey());
+if (!thisOption.getType().equals(otherOption.getType())) {
+ return false;
+}
+switch (thisOption.getType().getTypeName()) {
+ case BYTE:
+ case INT16:
+ case INT32:
+ case INT64:
+ case DECIMAL:
+ case FLOAT:
+ case DOUBLE:
+ case STRING:
+ case DATETIME:
+ case BOOLEAN:
+ case ARRAY:
+ case ITERABLE:
+ case MAP:
+ case ROW:
+ case LOGICAL_TYPE:
+if (!thisOption.getValue().equals(otherOption.getValue())) {
+ return false;
+}
+break;
+ case BYTES:
+if (!Arrays.equals((byte[]) thisOption.getValue(),
otherOption.getValue())) {
+ return false;
+}
+}
+ }
+ return true;
+}
+
+@Override
+public int hashCode() {
+ return Objects.hash(options);
+}
+
+static class Option implements Serializable {
+ Option(FieldType type, Object value) {
+this.type = type;
+this.value = value;
+ }
+
+ private FieldType type;
+ private Object value;
+
+ @SuppressWarnings("TypeParameterUnusedInFormals")
+ T getValue() {
+return (T) value;
+ }
+
+ FieldType getType() {
+return type;
+ }
+
+ @Override
+ public String toString() {
+return "Option{type=" + type + ", value=" + value + '}';
+ }
+
+ @Override
+ public boolean equals(Object o) {
+if (this == o) {
+ return true;
+}
+if (o == null || getClass() != o.getClass()) {
+ return false;
+}
+Option option = (Option) o;
+return Objects.equals(type, option.type) && Objects.equals(value,
option.value);
+ }
+
+ @Override
+ public int hashCode() {
+return Objects.hash(type, value);
+ }
+}
+
+public static class Builder {
+ private Map options;
+
+ Builder(Map init) {
+this.options = new HashMap<>(init);
+ }
+
+ Builder() {
+this(new HashMap<>());
+ }
+
+ public Builder setByteOption(String optionName, Byte value) {
+setOption(optionName, FieldType.BYTE, value);
+return this;
+ }
+
+ public Builder setBytesOption(String optionName, byte[] value) {
+setOption(optionName, FieldType.BYTES, value);
+return this;
+ }
+
+ public Builder setInt16Option(String optionName, Short value) {
+setOption(optionName, FieldType.INT16, value);
+return this;
+ }
+
+ public Builder setInt32Option(String optionName, Integer value) {
+setOption(optionName, FieldType.INT32, value);
+return this;
+ }
+
+ public Builder setInt64Option(String optionName, Long value) {
+setOption(optionName, FieldType.INT64, value);
+return this;
+ }
+
+ public Builder setDecimalOption(String optionName, BigDecimal value) {
+setOption(optionName, FieldType.DECIMAL, value);
+return this;
+ }
+
+ public Builder setFloatOption(String optionName, Float value) {
+setOption(optionName, FieldType.FLOAT, value);
+return this;
+ }
+
+
