Re: Flink SQL: Execute DELETE queries

Hi @Papadopoulos, Konstantinos I think you can try something like this: JDBCAppendTableSink sink = JDBCAppendTableSink.builder() .setDrivername("foo") .setDBUrl("bar") .setQuery("delete from %s where id = ?)") .setParameterTypes(FIELD_TYPES) .build(); Or you can build your own Sink c

Building Flink distribution with Scala2.12

Hi, I am trying to build Flink distribution for Scala 2.12 using the following command: mvn clean package -pl flink-dist -am -Pscala-2.12 -DskipTests , but I am getting the following error: [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ flink-runtime_2.12 --- [WARNING] Ru

Re: Upgrading from 1.4 to 1.8, losing Kafka consumer state

I checked the logs thanks to Paul's suggestion. I see a couple interesting things. Restoring into 1.8 from a 1.4 savepoint, some TMs receive partial state (e.g. only a partition/offset pair or two per TM -- we have 8 partitions on this topic). I'm not sure if this is normal (e.g. maybe TMs only use

RE: Flink SQL: Execute DELETE queries

The case I have in mind was to have an external JDBC table sink and try to delete a number of or all rows of the target DB table. Is it possible using Flink SQL? From: Vasyl Bervetskyi Sent: Tuesday, May 28, 2019 5:36 PM To: Papadopoulos, Konstantinos Cc: user@flink.apache.org Subject: RE: Fli

RE: Flink SQL: Execute DELETE queries

Hi Papadopoulos, Unfortunately no, there is no DELETE or MODIFY queries, you should create new table as a result of query which will filter records from existing one From: Papadopoulos, Konstantinos Sent: Tuesday, May 28, 2019 5:25 PM To: user@flink.apache.org Subject: Flink SQL: Execute DELETE

Re: [External] Re: How many task managers can a cluster reasonably handle?

Thanks for the info, Xintong Song! Cheers, Antonio On Fri, May 24, 2019 at 3:38 AM Xintong Song wrote: > Hi Antonio， > > According to experience in our production, Flink totally can handle 150 > TaskManagers per cluster. Actually, we have encountered much larger jobs > with thousands that each

Flink SQL: Execute DELETE queries

Hi all, I experiment on Flink Table API & SQL and I have the following question; is there any way to execute DELETE queries using Flink SQL? Thanks in advance, Konstantinos

Re: Restore state class not found exception in 1.8

Hi Gordon We have found a solution but not why it happens on 1.8. For it to work we need to call Env.registertype(Reportmessage.class) Reportmessage extends ReportmessageBase and the state operator use ReportmessageBase. So we need to register all the class’s that extends a class used in state

Flink CLI distributed cache fault

Hi there, I faced with issue in adding file to distributed cache in Flink. My setup: - Java 1.8 - Flink 1.8 - OS: Windows, Linux Test scenario: 1. Create simple stream environment 2. Add to distributed cache local file 3. Add simple source functio

Re: Restore state class not found exception in 1.8

Hi Gordon Thanks for the reply. No we haven’t moved it around namespaces. The only thing we have done is to add a new attribute to the object in another branch of our code and it could be we by mistake has used it but it should still not give a class not found exception. We have the save point

[ANNOUNCE] Munich meetup: "Let's talk about "Stream Processing with Apache Flink"

Hi folks, Next Tuesday (June, 4th), Vasia and I will be speaking at a meetup in Munich about Flink and how we wrote our book "Stream Processing with Apache Flink". We will also raffle a few copies of the book. Please RSVP if you'd like to attend: -> https://www.meetup.com/inovex-munich/events/26

Re: Distributed cache fault

Hi Vasyl, please post these kind of question to Flink's user ML since the dev ML is used for development discussions. For the failure on Windows could you share the complete stack trace to see where exactly it fails? It looks as if on Windows the scheme part of the URI makes problems. Looking at

Re: [SURVEY] Usage of flink-ml and [DISCUSS] Delete flink-ml

+1 for removing it. I think it is effectively dead by now. Cheers, Till On Mon, May 27, 2019 at 4:00 PM Hequn Cheng wrote: > Hi Shaoxuan, > > Thanks a lot for driving this. +1 to remove the module. > > The git log of this module shows that it has been inactive for a long > time. I think it's ok

Flink 1.8: Job manager redirection not happening in High Availability mode

Hi all, Prior to upgrading to 1.8, there was one active job manager and when I try to access the inactive job manager's web UI, the page used to get redirected to the active job manager. But now there is no redirection happening from the inactive JM to active JM. Did something change to the red

Re: Re: How can I add config file as classpath in taskmgr node when submitting a flink job?

Thanks. Let me have a try wangl...@geekplus.com.cn From: Yang Wang Date: 2019-05-28 09:47 To: wangl...@geekplus.com.cn CC: user Subject: Re: How can I add config file as classpath in taskmgr node when submitting a flink job? Hi, wangleiYou could use the flink distributed cache to register som

Re: Restore state class not found exception in 1.8

Hi Lasse, Did you move the class to a different namespace / package or changed to be a nested class, across the Flink versions? That would be the only cause I could reason about at the moment. If possible, could you also have a very minimal snippet / instructions on how I can maybe reproduce this