Github user zentol commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-220975120
I finally was able to fix the restart issue. there were 2 massive bugs in
the CassandraCommitter:
- within open() the checkpoint entry was always overridden
-
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r64217845
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericAtLeastOnceSink.java
---
@@ -0,0 +1,192 @@
+/**
+ *
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63871737
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
@@
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63869055
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63856510
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
@@
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63856337
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
@@
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63855468
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericAtLeastOnceSink.java
---
@@ -0,0 +1,192 @@
+/**
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63855104
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63855149
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63854905
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63678427
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericAtLeastOnceSink.java
---
@@ -0,0 +1,192 @@
+/**
+ *
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63678203
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
@@
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63678205
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericAtLeastOnceSink.java
---
@@ -0,0 +1,192 @@
+/**
+ *
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63675917
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63675682
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericAtLeastOnceSink.java
---
@@ -0,0 +1,192 @@
+/**
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63675532
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericAtLeastOnceSink.java
---
@@ -0,0 +1,192 @@
+/**
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r63673048
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericAtLeastOnceSink.java
---
@@ -0,0 +1,192 @@
+/**
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-217171304
I just tried the PR, but the recovery after a failure doesn't seem to work:
```
java.lang.RuntimeException: Error triggering a checkpoint as the result of
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-217160848
I'll test the PR again
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user alkagin commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-217159043
Hi guys :) any update on this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r61421879
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSink.java
---
@@ -0,0
Github user alkagin commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r61419752
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSink.java
---
@@ -0,0
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-209882761
Thank you for the fix. I'll take a look at it soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user zentol commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-209400438
@rmetzger I hav efixed the issue you encountered. The problem was that the
JM tries to discard all state belonging to pending checkpoints upon job
failure, which
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-207421723
I've tested the change again on a cluster and locally. The mode without WAL
works fine, the WAL-variant fails on (or quickly after) recovery.
It seems to be an
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r59004103
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58999599
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
@@
Github user zentol commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-206309053
@rmetzger addressed all your comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58675302
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/test/resources/cassandra.yaml
---
@@ -0,0 +1,43 @@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58572834
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/test/resources/cassandra.yaml
---
@@ -0,0 +1,43 @@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58571353
--- Diff:
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58568963
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSink.java
---
@@ -0,0
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58562758
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSink.java
---
@@ -0,0
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58562221
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/test/java/org/apache/flink/batch/connectors/cassandra/example/BatchExample.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58562019
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/ClusterBuilder.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58562121
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/test/java/org/apache/flink/batch/connectors/cassandra/example/BatchExample.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58561933
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraTupleWriteAheadSink.java
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58561402
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraTupleWriteAheadSink.java
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58561096
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSinkBase.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58557862
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSink.java
---
@@ -0,0
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58556585
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSink.java
---
@@ -0,0
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58556061
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSink.java
---
@@ -0,0
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58548774
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58548996
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraCommitter.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r58548581
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/batch/connectors/cassandra/CassandraOutputFormat.java
---
@@
Github user alkagin commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r56825863
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericAtLeastOnceSink.java
---
@@ -0,0 +1,197 @@
+/**
+ *
Github user zentol commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-199298767
Just pushed the following changes:
* Fixed AT_LEAST_ONCE type
* CassandraInputFormat.close properly propagates exceptions in close()
* cassandra pom now uses
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r56825317
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericAtLeastOnceSink.java
---
@@ -0,0 +1,197 @@
+/**
+ *
Github user alkagin commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r56823697
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericAtLeastOnceSink.java
---
@@ -0,0 +1,197 @@
+/**
+ *
Github user alkagin commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r56807419
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSink.java
---
@@ -0,0
Github user zentol commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-199229037
@alkagin Thank you for reporting this, will look into it. I think i may
already know how to fix it...
---
If your project is set up for it, you can reply to this email
Github user alkagin commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-199224407
Hi,
I am testing the Cassandra connector and I may have found a bug using
CassandraSink-DSL. During the execution it throws
`java.io.NotSerializableException:
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r56156903
--- Diff: flink-streaming-connectors/flink-connector-cassandra/pom.xml ---
@@ -0,0 +1,175 @@
+
+
+http://maven.apache.org/POM/4.0.0;
+
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1771#discussion_r56156990
--- Diff:
flink-streaming-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/batch/connectors/cassandra/CassandraInputFormat.java
---
@@
Github user alkagin commented on the pull request:
https://github.com/apache/flink/pull/1771#issuecomment-194294832
Great work :+1:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/1771
[FLINK-3311/FLINK-3332] Add Cassandra connector
This PR is a combination of #1640 and #1660, essentially providing a
mid-merge view of both PR's. As they reside in the same module and expose
56 matches
Mail list logo
