[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[
https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643495#comment-16643495
]
ASF GitHub Bot commented on FLINK-10310:
dawidwys closed pull request #6732: [FLINK-10310] Cassandra Sink - Handling
failing requests.
URL: https://github.com/apache/flink/pull/6732
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/dev/connectors/cassandra.md b/docs/dev/connectors/cassandra.md
index 6e1470a4367a..c4c3e3a47517 100644
--- a/docs/dev/connectors/cassandra.md
+++ b/docs/dev/connectors/cassandra.md
@@ -75,7 +75,10 @@ The following configuration methods can be used:
5. _enableWriteAheadLog([CheckpointCommitter committer])_
* An __optional__ setting
* Allows exactly-once processing for non-deterministic algorithms.
-6. _build()_
+6. _setFailureHandler([CassandraFailureHandler failureHandler])_
+* An __optional__ setting
+* Sets the custom failur handler.
+7. _build()_
* Finalizes the configuration and constructs the CassandraSink instance.
### Write-ahead Log
diff --git
a/flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/AbstractCassandraTupleSink.java
b/flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/AbstractCassandraTupleSink.java
index fda739e2aeb4..41826f58203f 100644
---
a/flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/AbstractCassandraTupleSink.java
+++
b/flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/AbstractCassandraTupleSink.java
@@ -32,8 +32,8 @@
private final String insertQuery;
private transient PreparedStatement ps;
- public AbstractCassandraTupleSink(String insertQuery, ClusterBuilder
builder) {
- super(builder);
+ public AbstractCassandraTupleSink(String insertQuery, ClusterBuilder
builder, CassandraFailureHandler failureHandler) {
+ super(builder, failureHandler);
this.insertQuery = insertQuery;
}
diff --git
a/flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraFailureHandler.java
b/flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraFailureHandler.java
new file mode 100644
index ..478ba47129e2
--- /dev/null
+++
b/flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraFailureHandler.java
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.connectors.cassandra;
+
+import org.apache.flink.annotation.PublicEvolving;
+
+import java.io.IOException;
+import java.io.Serializable;
+
+/**
+ * An implementation of {@link CassandraFailureHandler} is provided by the
user to define how
+ * {@link Throwable Throwable} should be handled, e.g. dropping them if the
failure is only temporary.
+ *
+ * Example:
+ *
+ * {@code
+ *
+ * private static class ExampleFailureHandler implements
CassandraFailureHandler {
+ *
+ * @Override
+ * void onFailure(Throwable failure) throws IOException {
+ * if (ExceptionUtils.containsThrowable(failure,
WriteTimeoutException.class)) {
+ * // drop exception
+ * } else {
+ * // for all other failures, fail the sink;
+ * // here the failure is simply rethrown, but
users can also choose to throw custom exceptions
+ * throw failure;
+ * }
+ * }
+ * }
+ *
+ * }
+ *
+ * The above example will let the sink ignore the WriteTimeoutException,
without failing the sink. For all other
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[ https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643492#comment-16643492 ] ASF GitHub Bot commented on FLINK-10310: dawidwys commented on issue #6732: [FLINK-10310] Cassandra Sink - Handling failing requests. URL: https://github.com/apache/flink/pull/6732#issuecomment-428210451 +1, merging This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Cassandra Sink - Handling failing requests > -- > > Key: FLINK-10310 > URL: https://issues.apache.org/jira/browse/FLINK-10310 > Project: Flink > Issue Type: Improvement > Components: Cassandra Connector >Reporter: Jayant Ameta >Priority: Major > Labels: pull-request-available > > The cassandra sink fails for any kind of error. For some exceptions (e.g > WriteTimeoutException), ignoring the exception may be acceptable as well. > Can we discuss having a FailureHandler on the lines of > ActionRequestFailureHandler? > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[
https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626819#comment-16626819
]
ASF GitHub Bot commented on FLINK-10310:
yanghua commented on a change in pull request #6732: [FLINK-10310] Cassandra
Sink - Handling failing requests.
URL: https://github.com/apache/flink/pull/6732#discussion_r220067824
##
File path:
flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/NoOpCassandraFailureHandler.java
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.connectors.cassandra;
+
+import org.apache.flink.annotation.Internal;
+
+import java.io.IOException;
+
+/**
+ * A {@link CassandraFailureHandler} that simply fails the sink on any
failures.
+ */
+@Internal
+public class NoOpCassandraFailureHandler implements CassandraFailureHandler {
Review comment:
@wittyameta You don't have to modify this naming. This is just my personal
thought and does not represent the final reviewer's opinion. In addition, PR
review may not be so fast, please be patient.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Cassandra Sink - Handling failing requests
> --
>
> Key: FLINK-10310
> URL: https://issues.apache.org/jira/browse/FLINK-10310
> Project: Flink
> Issue Type: Improvement
> Components: Cassandra Connector
>Reporter: Jayant Ameta
>Priority: Major
> Labels: pull-request-available
>
> The cassandra sink fails for any kind of error. For some exceptions (e.g
> WriteTimeoutException), ignoring the exception may be acceptable as well.
> Can we discuss having a FailureHandler on the lines of
> ActionRequestFailureHandler?
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[
https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626808#comment-16626808
]
ASF GitHub Bot commented on FLINK-10310:
wittyameta commented on a change in pull request #6732: [FLINK-10310] Cassandra
Sink - Handling failing requests.
URL: https://github.com/apache/flink/pull/6732#discussion_r220066503
##
File path:
flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/NoOpCassandraFailureHandler.java
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.connectors.cassandra;
+
+import org.apache.flink.annotation.Internal;
+
+import java.io.IOException;
+
+/**
+ * A {@link CassandraFailureHandler} that simply fails the sink on any
failures.
+ */
+@Internal
+public class NoOpCassandraFailureHandler implements CassandraFailureHandler {
Review comment:
@yanghua Hi, should I just commit with the other indentation fix, or should
I wait for consensus on naming of NoOpFailureHandler?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Cassandra Sink - Handling failing requests
> --
>
> Key: FLINK-10310
> URL: https://issues.apache.org/jira/browse/FLINK-10310
> Project: Flink
> Issue Type: Improvement
> Components: Cassandra Connector
>Reporter: Jayant Ameta
>Priority: Major
> Labels: pull-request-available
>
> The cassandra sink fails for any kind of error. For some exceptions (e.g
> WriteTimeoutException), ignoring the exception may be acceptable as well.
> Can we discuss having a FailureHandler on the lines of
> ActionRequestFailureHandler?
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[
https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625628#comment-16625628
]
ASF GitHub Bot commented on FLINK-10310:
wittyameta commented on a change in pull request #6732: [FLINK-10310] Cassandra
Sink - Handling failing requests.
URL: https://github.com/apache/flink/pull/6732#discussion_r219792995
##
File path:
flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/NoOpCassandraFailureHandler.java
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.connectors.cassandra;
+
+import org.apache.flink.annotation.Internal;
+
+import java.io.IOException;
+
+/**
+ * A {@link CassandraFailureHandler} that simply fails the sink on any
failures.
+ */
+@Internal
+public class NoOpCassandraFailureHandler implements CassandraFailureHandler {
Review comment:
@tillrohrmann any suggestions?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Cassandra Sink - Handling failing requests
> --
>
> Key: FLINK-10310
> URL: https://issues.apache.org/jira/browse/FLINK-10310
> Project: Flink
> Issue Type: Improvement
> Components: Cassandra Connector
>Reporter: Jayant Ameta
>Priority: Major
> Labels: pull-request-available
>
> The cassandra sink fails for any kind of error. For some exceptions (e.g
> WriteTimeoutException), ignoring the exception may be acceptable as well.
> Can we discuss having a FailureHandler on the lines of
> ActionRequestFailureHandler?
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[ https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625532#comment-16625532 ] ASF GitHub Bot commented on FLINK-10310: wittyameta commented on a change in pull request #6732: [FLINK-10310] Cassandra Sink - Handling failing requests. URL: https://github.com/apache/flink/pull/6732#discussion_r219762573 ## File path: flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraTupleSink.java ## @@ -17,7 +17,7 @@ package org.apache.flink.streaming.connectors.cassandra; -import org.apache.flink.api.java.tuple.Tuple; + import org.apache.flink.api.java.tuple.Tuple; Review comment: Will fix the indent. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Cassandra Sink - Handling failing requests > -- > > Key: FLINK-10310 > URL: https://issues.apache.org/jira/browse/FLINK-10310 > Project: Flink > Issue Type: Improvement > Components: Cassandra Connector >Reporter: Jayant Ameta >Priority: Major > Labels: pull-request-available > > The cassandra sink fails for any kind of error. For some exceptions (e.g > WriteTimeoutException), ignoring the exception may be acceptable as well. > Can we discuss having a FailureHandler on the lines of > ActionRequestFailureHandler? > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[
https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625512#comment-16625512
]
ASF GitHub Bot commented on FLINK-10310:
wittyameta commented on a change in pull request #6732: [FLINK-10310] Cassandra
Sink - Handling failing requests.
URL: https://github.com/apache/flink/pull/6732#discussion_r219756690
##
File path:
flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/NoOpCassandraFailureHandler.java
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.connectors.cassandra;
+
+import org.apache.flink.annotation.Internal;
+
+import java.io.IOException;
+
+/**
+ * A {@link CassandraFailureHandler} that simply fails the sink on any
failures.
+ */
+@Internal
+public class NoOpCassandraFailureHandler implements CassandraFailureHandler {
Review comment:
Hm, I read NoOp as not doing anything at all and allowing all exceptions to
pass through :)
I can wait for other reviewers to suggest better notation.
Thanks!
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Cassandra Sink - Handling failing requests
> --
>
> Key: FLINK-10310
> URL: https://issues.apache.org/jira/browse/FLINK-10310
> Project: Flink
> Issue Type: Improvement
> Components: Cassandra Connector
>Reporter: Jayant Ameta
>Priority: Major
> Labels: pull-request-available
>
> The cassandra sink fails for any kind of error. For some exceptions (e.g
> WriteTimeoutException), ignoring the exception may be acceptable as well.
> Can we discuss having a FailureHandler on the lines of
> ActionRequestFailureHandler?
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[
https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625497#comment-16625497
]
ASF GitHub Bot commented on FLINK-10310:
yanghua commented on a change in pull request #6732: [FLINK-10310] Cassandra
Sink - Handling failing requests.
URL: https://github.com/apache/flink/pull/6732#discussion_r219752224
##
File path:
flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/NoOpCassandraFailureHandler.java
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.connectors.cassandra;
+
+import org.apache.flink.annotation.Internal;
+
+import java.io.IOException;
+
+/**
+ * A {@link CassandraFailureHandler} that simply fails the sink on any
failures.
+ */
+@Internal
+public class NoOpCassandraFailureHandler implements CassandraFailureHandler {
Review comment:
Oh, I didn't go to see the related implementation of `ElasticsearchSink `.
This is just a suggestion, because when I first saw the name, I thought it was
an empty method, didn't do anything because you called it "NoOp", and in fact
you throw an exception, it will cause the sink task to fail if the outer layer
does not handle it. Maybe you can listen to other people's opinions and see if
it is a better choice to keep up with the implementation of `ElasticsearchSink
`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Cassandra Sink - Handling failing requests
> --
>
> Key: FLINK-10310
> URL: https://issues.apache.org/jira/browse/FLINK-10310
> Project: Flink
> Issue Type: Improvement
> Components: Cassandra Connector
>Reporter: Jayant Ameta
>Priority: Major
> Labels: pull-request-available
>
> The cassandra sink fails for any kind of error. For some exceptions (e.g
> WriteTimeoutException), ignoring the exception may be acceptable as well.
> Can we discuss having a FailureHandler on the lines of
> ActionRequestFailureHandler?
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[
https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625492#comment-16625492
]
ASF GitHub Bot commented on FLINK-10310:
wittyameta commented on a change in pull request #6732: [FLINK-10310] Cassandra
Sink - Handling failing requests.
URL: https://github.com/apache/flink/pull/6732#discussion_r219750666
##
File path:
flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/NoOpCassandraFailureHandler.java
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.connectors.cassandra;
+
+import org.apache.flink.annotation.Internal;
+
+import java.io.IOException;
+
+/**
+ * A {@link CassandraFailureHandler} that simply fails the sink on any
failures.
+ */
+@Internal
+public class NoOpCassandraFailureHandler implements CassandraFailureHandler {
Review comment:
There is a `NoOpFailureHandler` used in ElasticsearchSink which throws the
exception as well. I referred to that class while naming the failureHandler in
CassandraSink.
Should I rename the class?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Cassandra Sink - Handling failing requests
> --
>
> Key: FLINK-10310
> URL: https://issues.apache.org/jira/browse/FLINK-10310
> Project: Flink
> Issue Type: Improvement
> Components: Cassandra Connector
>Reporter: Jayant Ameta
>Priority: Major
> Labels: pull-request-available
>
> The cassandra sink fails for any kind of error. For some exceptions (e.g
> WriteTimeoutException), ignoring the exception may be acceptable as well.
> Can we discuss having a FailureHandler on the lines of
> ActionRequestFailureHandler?
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[ https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625470#comment-16625470 ] ASF GitHub Bot commented on FLINK-10310: yanghua commented on a change in pull request #6732: [FLINK-10310] Cassandra Sink - Handling failing requests. URL: https://github.com/apache/flink/pull/6732#discussion_r219744959 ## File path: flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraTupleSink.java ## @@ -17,7 +17,7 @@ package org.apache.flink.streaming.connectors.cassandra; -import org.apache.flink.api.java.tuple.Tuple; + import org.apache.flink.api.java.tuple.Tuple; Review comment: Need to indent here? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Cassandra Sink - Handling failing requests > -- > > Key: FLINK-10310 > URL: https://issues.apache.org/jira/browse/FLINK-10310 > Project: Flink > Issue Type: Improvement > Components: Cassandra Connector >Reporter: Jayant Ameta >Priority: Major > Labels: pull-request-available > > The cassandra sink fails for any kind of error. For some exceptions (e.g > WriteTimeoutException), ignoring the exception may be acceptable as well. > Can we discuss having a FailureHandler on the lines of > ActionRequestFailureHandler? > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[
https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625469#comment-16625469
]
ASF GitHub Bot commented on FLINK-10310:
yanghua commented on a change in pull request #6732: [FLINK-10310] Cassandra
Sink - Handling failing requests.
URL: https://github.com/apache/flink/pull/6732#discussion_r219744404
##
File path:
flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/NoOpCassandraFailureHandler.java
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.connectors.cassandra;
+
+import org.apache.flink.annotation.Internal;
+
+import java.io.IOException;
+
+/**
+ * A {@link CassandraFailureHandler} that simply fails the sink on any
failures.
+ */
+@Internal
+public class NoOpCassandraFailureHandler implements CassandraFailureHandler {
Review comment:
Because this handler throws an exception, I think the name "NoOp" may not be
accurate, maybe named `FailingCassandraFailureHandler` (here, I just refer to
`FailingCheckpointExceptionHandler`) or some other name is more appropriate .
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Cassandra Sink - Handling failing requests
> --
>
> Key: FLINK-10310
> URL: https://issues.apache.org/jira/browse/FLINK-10310
> Project: Flink
> Issue Type: Improvement
> Components: Cassandra Connector
>Reporter: Jayant Ameta
>Priority: Major
> Labels: pull-request-available
>
> The cassandra sink fails for any kind of error. For some exceptions (e.g
> WriteTimeoutException), ignoring the exception may be acceptable as well.
> Can we discuss having a FailureHandler on the lines of
> ActionRequestFailureHandler?
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[ https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623422#comment-16623422 ] ASF GitHub Bot commented on FLINK-10310: wittyameta opened a new pull request #6732: [FLINK-10310] Cassandra Sink - Handling failing requests. URL: https://github.com/apache/flink/pull/6732 ## What is the purpose of the change This pull request provides support to optionally handle cassandra sink errors. A user may choose to ignore an exception instead of allowing the sink to fail. The handler is similar to `ActionRequestFailureHandler` in Elastic Sink. ## Brief change log - *Added interface `CassandraFailureHandler` and a no-op implementation `NoOpCassandraFailureHandler`* - *`CassandraSinkBase` has a new field `failureHandler` and updated constructor* - *`checkAsyncErrors` method in `CassandraSinkBase` calls the `failureHandler` instead of throwing an `IOException`* - *`CassandraSinkBuilder` has a new optional field `failureHandler` and updated setter. Uses no-op implementation as default* ## Verifying this change This change added tests and can be verified as follows: - *Added test in class `CassandraSinkBaseTest` that validates that failureHandler is called and the error is ignored* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **yes** - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **yes** - If yes, how is the feature documented? **docs** This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Cassandra Sink - Handling failing requests > -- > > Key: FLINK-10310 > URL: https://issues.apache.org/jira/browse/FLINK-10310 > Project: Flink > Issue Type: Improvement > Components: Cassandra Connector >Reporter: Jayant Ameta >Priority: Major > Labels: pull-request-available > > The cassandra sink fails for any kind of error. For some exceptions (e.g > WriteTimeoutException), ignoring the exception may be acceptable as well. > Can we discuss having a FailureHandler on the lines of > ActionRequestFailureHandler? > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[ https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621604#comment-16621604 ] Till Rohrmann commented on FLINK-10310: --- Looks good to me. I think we can proceed with the implementation. > Cassandra Sink - Handling failing requests > -- > > Key: FLINK-10310 > URL: https://issues.apache.org/jira/browse/FLINK-10310 > Project: Flink > Issue Type: Improvement > Components: Cassandra Connector >Reporter: Jayant Ameta >Priority: Major > > The cassandra sink fails for any kind of error. For some exceptions (e.g > WriteTimeoutException), ignoring the exception may be acceptable as well. > Can we discuss having a FailureHandler on the lines of > ActionRequestFailureHandler? > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[ https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618447#comment-16618447 ] Jayant Ameta commented on FLINK-10310: -- Hey [~till.rohrmann], do you want to discuss this further? > Cassandra Sink - Handling failing requests > -- > > Key: FLINK-10310 > URL: https://issues.apache.org/jira/browse/FLINK-10310 > Project: Flink > Issue Type: Improvement > Components: Cassandra Connector >Reporter: Jayant Ameta >Priority: Major > > The cassandra sink fails for any kind of error. For some exceptions (e.g > WriteTimeoutException), ignoring the exception may be acceptable as well. > Can we discuss having a FailureHandler on the lines of > ActionRequestFailureHandler? > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[
https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614371#comment-16614371
]
Jayant Ameta commented on FLINK-10310:
--
Hi [~till.rohrmann]
{{CassandraSinkBase}} would have a field similar to
{{ActionRequestFailureHandler}}.
In the {{checkAsyncErrors}} method, the failureHandler would be called instead
of throwing the {{IOException}}
Current code snippet
{code:java}
private void checkAsyncErrors() throws Exception {
Throwable error = exception;
if (error != null) {
// prevent throwing duplicated error
exception = null;
throw new IOException("Error while sending value.",
error);
}
}
{code}
would change to:
{code:java}
private void checkAsyncErrors() throws Exception {
Throwable error = exception;
if (error != null) {
failureHandler.onFailure(error);
}
}
{code}
Here the {{failureHandler}} can decide what steps to take based on the
{{Throwable}}.
> Cassandra Sink - Handling failing requests
> --
>
> Key: FLINK-10310
> URL: https://issues.apache.org/jira/browse/FLINK-10310
> Project: Flink
> Issue Type: Improvement
> Components: Cassandra Connector
>Reporter: Jayant Ameta
>Priority: Major
>
> The cassandra sink fails for any kind of error. For some exceptions (e.g
> WriteTimeoutException), ignoring the exception may be acceptable as well.
> Can we discuss having a FailureHandler on the lines of
> ActionRequestFailureHandler?
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10310) Cassandra Sink - Handling failing requests
[
https://issues.apache.org/jira/browse/FLINK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610162#comment-16610162
]
Till Rohrmann commented on FLINK-10310:
---
Having something similar to the {{ActionRequestFailureHandler}} in the
{{CassandraSink}} could work. Could you outline the required changes a little
bit more in detail [~wittyameta]?
> Cassandra Sink - Handling failing requests
> --
>
> Key: FLINK-10310
> URL: https://issues.apache.org/jira/browse/FLINK-10310
> Project: Flink
> Issue Type: Improvement
> Components: Cassandra Connector
>Reporter: Jayant Ameta
>Priority: Major
>
> The cassandra sink fails for any kind of error. For some exceptions (e.g
> WriteTimeoutException), ignoring the exception may be acceptable as well.
> Can we discuss having a FailureHandler on the lines of
> ActionRequestFailureHandler?
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/WriteTimeoutException-in-Cassandra-sink-kill-the-job-tp22706p22707.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
