[GitHub] [samza] shanthoosh commented on issue #970: SAMZA-2143: Fix NPE in `CoordinatorStreamMessage#equals` and some clean-up to CoordinatorStreamSystemConsumer
shanthoosh commented on issue #970: SAMZA-2143: Fix NPE in `CoordinatorStreamMessage#equals` and some clean-up to CoordinatorStreamSystemConsumer URL: https://github.com/apache/samza/pull/970#issuecomment-477889117 @dnishimura Thanks for the changes. Merged to trunk. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza] shanthoosh merged pull request #970: SAMZA-2143: Fix NPE in `CoordinatorStreamMessage#equals` and some clean-up to CoordinatorStreamSystemConsumer
shanthoosh merged pull request #970: SAMZA-2143: Fix NPE in `CoordinatorStreamMessage#equals` and some clean-up to CoordinatorStreamSystemConsumer URL: https://github.com/apache/samza/pull/970 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza] srinipunuru merged pull request #979: SAMZA-2110 : Support for deleting records from a table
srinipunuru merged pull request #979: SAMZA-2110 : Support for deleting records from a table URL: https://github.com/apache/samza/pull/979 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Re: [VOTE] SEP-21: Samza Async API for High Level
Looks great! +1 (binding). On Thu, Mar 28, 2019 at 12:03 PM Jake Maes wrote: > +1 (binding) > > On Mon, Mar 25, 2019 at 6:09 PM Jagadish Venkatraman < > jagadish1...@gmail.com> > wrote: > > > +1 (binding); > > > > thanks Bharath for the proposal and the implementation. > > > > LGTM; > > > > On Monday, March 25, 2019, Bharath Kumara Subramanian < > > codin.mart...@gmail.com> wrote: > > > > > Hi all, > > > > > > > > > This is a call for a vote for SEP-21: Samza Async API for High Level > > > > > > > > > SEP-21 has been discussed and implemented using SAMZA-2055. For > > reference, > > > the design document can be found - > > > https://cwiki.apache.org/confluence/display/SAMZA/SEP- > > > 21%3A+Samza+Async+API+for+High+Level > > > > > > > > > Thanks, > > > Bharath > > > > > > > > > -- > > Jagadish V, > > Graduate Student, > > Department of Computer Science, > > Stanford University > > >
Re: [VOTE] SEP-21: Samza Async API for High Level
+1 (binding) On Mon, Mar 25, 2019 at 6:09 PM Jagadish Venkatraman wrote: > +1 (binding); > > thanks Bharath for the proposal and the implementation. > > LGTM; > > On Monday, March 25, 2019, Bharath Kumara Subramanian < > codin.mart...@gmail.com> wrote: > > > Hi all, > > > > > > This is a call for a vote for SEP-21: Samza Async API for High Level > > > > > > SEP-21 has been discussed and implemented using SAMZA-2055. For > reference, > > the design document can be found - > > https://cwiki.apache.org/confluence/display/SAMZA/SEP- > > 21%3A+Samza+Async+API+for+High+Level > > > > > > Thanks, > > Bharath > > > > > -- > Jagadish V, > Graduate Student, > Department of Computer Science, > Stanford University >
[GitHub] [samza-hello-samza] shanthoosh closed pull request #39: Sample application to illustrate side-inputs using stream table join
shanthoosh closed pull request #39: Sample application to illustrate side-inputs using stream table join URL: https://github.com/apache/samza-hello-samza/pull/39 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza-hello-samza] shanthoosh commented on issue #39: Sample application to illustrate side-inputs using stream table join
shanthoosh commented on issue #39: Sample application to illustrate side-inputs using stream table join URL: https://github.com/apache/samza-hello-samza/pull/39#issuecomment-477723517 Since @mynameborat has lost access to his account, I'm closing this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza] mynameborat commented on a change in pull request #905: SAMZA-2055: Async high level api
mynameborat commented on a change in pull request #905: SAMZA-2055: Async high
level api
URL: https://github.com/apache/samza/pull/905#discussion_r270129948
##
File path:
samza-core/src/main/java/org/apache/samza/operators/impl/AsyncStreamOperatorImpl.java
##
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.samza.operators.impl;
+
+import java.util.Collection;
+import java.util.Collections;
+import java.util.concurrent.CompletionStage;
+import java.util.concurrent.ExecutionException;
+import org.apache.samza.SamzaException;
+import org.apache.samza.context.Context;
+import org.apache.samza.operators.functions.AsyncFlatMapFunction;
+import org.apache.samza.operators.spec.AsyncOperatorSpec;
+import org.apache.samza.operators.spec.OperatorSpec;
+import org.apache.samza.task.MessageCollector;
+import org.apache.samza.task.TaskCoordinator;
+
+
+public class AsyncStreamOperatorImpl extends OperatorImpl {
+ private final AsyncOperatorSpec asyncOperatorSpec;
+ private final AsyncFlatMapFunction transformFn;
+
+ AsyncStreamOperatorImpl(AsyncOperatorSpec asyncOperatorSpec) {
+this.asyncOperatorSpec = asyncOperatorSpec;
+this.transformFn = asyncOperatorSpec.getTransformFn();
+ }
+ @Override
+ protected void handleInit(Context context) {
+this.transformFn.init(context);
+ }
+
+ @Override
+ protected Collection handleMessage(M message, MessageCollector
collector, TaskCoordinator coordinator) {
Review comment:
Fixed it. Although I left the `onMessage` and `handleMessage` as package
private test helper methods otherwise, tests would need to convert the
`CompletionStage` to `CompletableFuture` and do a get on them.
I can follow up with another PR just targeting the tests cleanup if we feel
tests should be okay doing the above and we don't need a package private helper
method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza] srinipunuru edited a comment on issue #979: SAMZA-2110 : Support for deleting records from a table
srinipunuru edited a comment on issue #979: SAMZA-2110 : Support for deleting records from a table URL: https://github.com/apache/samza/pull/979#issuecomment-477700449 @atoomula > * delete events from cc streams. eg: "select * from brooklin.stream", "select f1, f2 from brooklin.stream" With this new model there needs to be separate sql statement for delete. I think this model may be better longer term, because When we have delete from support. users need to write two sql statements on for propagating inserts and updates and other one for deletes. This model is inline with that. > * Non-explicit delete events. eg: "select f1, f2 from kafka.topic". In this case both f1 and f2 could be null. Deleting such an event from destination store is unclear. When user does select f1, f2 and when both f1 and f2 are null, it is possible that user wants us to insert a record with each fields being null. Assuming that the behavior in this case should be delete is non-intuitive and also problematic because we may never be able to support the above requested behavior (inserting a record with all fields being null). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza] srinipunuru commented on issue #979: SAMZA-2110 : Support for deleting records from a table
srinipunuru commented on issue #979: SAMZA-2110 : Support for deleting records from a table URL: https://github.com/apache/samza/pull/979#issuecomment-477700449 @atoomula > * delete events from cc streams. eg: "select * from brooklin.stream", "select f1, f2 from brooklin.stream" With this new model there needs to be separate sql statement for delete. I think this model may be better longer term, because When we have delete from support. users need to write two sql statements on for propagating inserts and updates and other one for deletes. This model is inline with that. > * Non-explicit delete events. eg: "select f1, f2 from kafka.topic". In this case both f1 and f2 could be null. Deleting such an event from destination store is unclear. When user does select f1, f2 and when both f1 and f2 are null, it is possible that user wants us to insert a record with each fields being null. Assuming that the behavior in this case should be delete is non-intuitive and also problematic because we may never be able to support the above requested behavior. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza] dnishimura commented on issue #971: SAMZA-2144: In TaskInstance, bypass "SSP caught-up" check if offset is set by a startpoint
dnishimura commented on issue #971: SAMZA-2144: In TaskInstance, bypass "SSP caught-up" check if offset is set by a startpoint URL: https://github.com/apache/samza/pull/971#issuecomment-477700203 Closing since incoming API modifications will not require this PR. FYI - @shanthoosh This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza] dnishimura closed pull request #971: SAMZA-2144: In TaskInstance, bypass "SSP caught-up" check if offset is set by a startpoint
dnishimura closed pull request #971: SAMZA-2144: In TaskInstance, bypass "SSP caught-up" check if offset is set by a startpoint URL: https://github.com/apache/samza/pull/971 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza] mynameborat commented on a change in pull request #905: SAMZA-2055: Async high level api
mynameborat commented on a change in pull request #905: SAMZA-2055: Async high
level api
URL: https://github.com/apache/samza/pull/905#discussion_r270075454
##
File path:
samza-api/src/main/java/org/apache/samza/operators/functions/AsyncFlatMapFunction.java
##
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.samza.operators.functions;
+
+import java.io.Serializable;
+import java.util.Collection;
+import java.util.concurrent.CompletionStage;
+import org.apache.samza.annotation.InterfaceStability;
+
+
+/**
+ * Transforms an input message into a collection of 0 or more messages,
possibly of a different type.
+ *
+ * @param type of the input message
+ * @param type of the transformed messages
+ */
+@InterfaceStability.Unstable
+public interface AsyncFlatMapFunction extends InitableFunction,
ClosableFunction, Serializable {
+
+ /**
+ * Transforms the provided message into a collection of 0 or more messages.
+ *
+ * @param message the input message to be transformed
+ * @return a collection of 0 or more transformed messages
+ */
+ CompletionStage> apply(M message);
Review comment:
I too initially had CompletableFuture and then moved to CompletionStage.
CompletionStage offers much stricter async programming semantics and
CompletableFuture is concrete implementation of the CompletionStage interface.
I chose to go with the interface.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] mynameborat commented on issue #39: Sample application to illustrate side-inputs using stream table join
mynameborat commented on issue #39: Sample application to illustrate side-inputs using stream table join URL: https://github.com/apache/samza-hello-samza/pull/39#issuecomment-477643676 I lost access to this account due to 2FC issues. Will see if I can close it somehow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
