Re: Joining multiple temporal tables

2019-12-06 Thread Chris Miller
Hi Fabian, Thanks for confirming the issue and suggesting a workaround - I'll give that a try. I've created a JIRA issue as you suggested, https://issues.apache.org/jira/browse/FLINK-15112 Many thanks, Chris -- Original Message -- From: "Fabian Hueske" To: "Ch

Joining multiple temporal tables

2019-12-05 Thread Chris Miller
I want to decorate/enrich a stream by joining it with "lookups" to the most recent data available in other streams. For example, suppose I have a stream of product orders. For each order, I want to add price and FX rate information based on the order's product ID and order currency. Is it

Re: Need help using AggregateFunction instead of FoldFunction

2019-12-05 Thread Chris Miller
I hit the same problem, as far as I can tell it should be fixed in Pulsar 2.4.2. The release of this has already passed voting so I hope it should be available in a day or two. https://github.com/apache/pulsar/pull/5068 -- Original Message -- From: "devinbost" To:

Re: Accessing fields in a POJO stream

2019-12-04 Thread Chris Miller
al Message -- From: "Jingsong Lee" To: "Chris Miller" Cc: "user" Sent: 04/12/2019 03:41:05 Subject: Re: Add time attribute column to POJO stream Hi Chris, First thing, FxRate is not POJO, a POJO should have a constructor without arguments. In this way, you can read fr

Add time attribute column to POJO stream

2019-12-03 Thread Chris Miller
I'm having trouble dealing with a DataStream of POJOs. In particular, when I perform SQL operations on it I can't figure out the syntax for referring to individual fields within the POJO. Below is an example that illustrates the problem and the various approaches I've tried. Can anyone please

Re: low performance in running queries

2019-10-30 Thread Chris Miller
I haven't run any benchmarks with Flink or even used it enough to directly help with your question, however I suspect that the following article might be relevant: http://dsrg.pdos.csail.mit.edu/2016/06/26/scalability-cost/ Given the computation you're performing is trivial, it's possible

Re: AssertionError: mismatched type $5 TIMESTAMP(3)

2019-02-06 Thread Chris Miller
com/dataArtisans/sql-training/wiki/SQL-on-streams I hope this helps. Feel free to ask further questions. Regards, Timo Am 05.02.19 um 11:30 schrieb Chris Miller: Exception in thread "main" java.lang.AssertionError: mismatched type $5 TIMESTAMP(3) at

Re: AssertionError: mismatched type $5 TIMESTAMP(3)

2019-02-05 Thread Chris Miller
Sorry to reply to my own post but I wasn't able to figure out a solution for this. Does anyone have any suggestions I could try? -- Original Message -- From: "Chris Miller" To: "Timo Walther" ; "user" Sent: 29/01/2019 10:06:47 Subject: Re: AssertionError

Re: AssertionError: mismatched type $5 TIMESTAMP(3)

2019-01-29 Thread Chris Miller
quired for the temporal table join. Any other ideas on how I can work around this problem? Many thanks, Chris -- Original Message -- From: "Timo Walther" To: "Chris Miller" ; "user" Sent: 29/01/2019 09:44:14 Subject: Re: AssertionError: mismatched type $5 TIM

AssertionError: mismatched type $5 TIMESTAMP(3)

2019-01-25 Thread Chris Miller
I'm trying to group some data and then enrich it by joining with a temporal table function, however my test code (attached) is failing with the error shown below. Can someone please give me a clue as to what I'm doing wrong? Exception in thread "main" java.lang.AssertionError: mismatched type

Re: Temporal tables not behaving as expected

2019-01-22 Thread Chris Miller
Message -- From: "Fabian Hueske" To: "Chris Miller" Cc: "user" Sent: 22/01/2019 11:23:23 Subject: Re: Temporal tables not behaving as expected Hi, The problem is that you are using processing time which is non-deterministic. Both inputs are consumed at the same

Temporal tables not behaving as expected

2019-01-21 Thread Chris Miller
Hi all, I'm new to Flink so am probably missing something simple. I'm using Flink 1.7.1 and am trying to use temporal table functions but aren't getting the results I expect. With the example code below, I would expect 4 records to be output (one for each order), but instead I'm only seeing

Where are the TaskManagers IPs and Ports stored?

2018-10-15 Thread Chris Miller
Hi, I'm looking for the property file where IP and Port of TaskManagers are stored in Flink. Does anyone know where it is located and when it's updated? (And for the case that there should not be such a file, where does the JobManager and TaskManagers take this information from?) Thanks.

Partitions vs. Subpartitions

2018-10-10 Thread Chris Miller
Hi, what's the difference between partitions and subpartitions? Thanks. CM