Odd error

Getting this: DataStream stream =                env.addSource(new FlinkKafkaConsumer08<>("raw", schema, properties)                ).setParallelism(30).flatMap(new RecordSplit()).setParallelism(30).                        name("Raw splitter").keyBy("id","keyByHelper","schema"); Field

RE: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

That worked.. Thanks Chesnay, [Verizon] Kanagaraj Vengidasamy RTCI 7701 E Telecom PKWY Temple Terrace, FL 33637 O 813.978.4372 | M 813.455.9757 [Twitter] [LinkedIn]

Re: Kinesis connector SHARD_GETRECORDS_MAX default value

Hi Steffan, I have to admit that I didn’t put too much thoughts in the default values for the Kinesis consumer. I’d say it would be reasonable to change the default values to follow KCL’s settings. Could you file a JIRA for this? In general, we might want to reconsider all the default values

Kinesis connector SHARD_GETRECORDS_MAX default value

Hi there, I recently ran into problems with a Flink job running on an EMR cluster consuming events from a Kinesis stream receiving roughly 15k event/second. Although the EMR cluster was substantially scaled and CPU utilization and system load were well below any alarming threshold, the

Re: Cassandra Sink version

Hi Nancy, For both Flink 1.2 and Flink 1.3, our tests are written against Cassandra 2.2.5. We use the version 3.0 of this https://github.com/datastax/java-driver/tree/3.0.x driver. So please check there to see which Cassandra versions they

Re: RocksDB segfaults

Hi, for the first checkpoint, from the stacktrace I assume that the backend is not accessed as part of processing an element, but by another thread. Is that correct? RocksDB requires accessing threads to hold the task’s checkpointing lock, otherwise they might call methods on an instance that

Cassandra Sink version

Hi all, Which Cassandra version is currently supported by the Flink 1.2 connector? and I am wondering which version Flink 1.3 will be supporting? Thank you! -- View this message in context:

Re: RocksDB segfaults

Hi Stephen, you are right, the second stack trace is indeed from a run of Flink 1.1.4. Sorry, my bad. That leaves us with the first trace of a segfault for which I can guarantee that it brought down a 1.2.0 instance. Unfortunately I cannot reproduce the problem. It has happened twice so far,

Re: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

I guess that's because the grouping is wrong. ^server\.*([0-9])+*[[:space:]]*\=([^: \#]+) should probably be ^server\.*([0-9]+)*[[:space:]]*\=([^: \#]+) Could you modify the .sh script as such and try again? Regards, Chesnay On 22.03.2017 16:10, kanagaraj.vengidas...@verizon.com wrote:

RE: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Greg, Sorry about that.. when I copy the config and replaced the real server names . I don't have space in my configurations. The issue is when I have server.10=, server.11= And tried to start the zookeeper, Myid replaced with 0 and 1 for these servers which is conflicting with server.0 and

Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Side note: It's quite uncommon to have more than 9 ZK nodes (at least if the ZK is dedicated to Flink alone). The ZooKeeper docs suggest to add "observers" to scale out ZooKeeper further, not quorum peers, because many quorum peers make ZooKeeper transactions slower (because the quorum that has

RocksDB segfaults

Hi, I have experienced two crashes of Flink 1.2 by SIGSEGV, caused by something in RocksDB. What is the preferred way to report them? All I got at the moment are two hs_err_pid12345.log files. They are over 4000 lines long each. Is there anything significant that I should extract to help you