Re: How to user ParameterTool.fromPropertiesFile() to get resource file inside my jar

Hi Aljoscha, I found the root cause of my problem from this reference https://stackoverflow.com/questions/18151072/cant-find-resource-file-after-exporting-to-a-runnable-jar . So I changed the way to use ParameterTool. I read the configurations from InputStream, construct them as argument format

RE: ETL with changing reference data

Hi Peter, I think what you referred is typical amendment process where partial or all results need to modified. I think it is definitely interesting topic! Here is my two cents In ideal world, reference data source can ingest updated used values as events and join with buffered events in

ETL with changing reference data

hi, We are building an ETL style application in Flink that consumes records from a file or a message bus as a DataStream. We are transforming records using SQL and UDFs. The UDF loads reference data in the open method and currently the data loaded remains in memory until the job is cancelled.

Best way to deriving streams from another one

Hi, I have a data stream resulting from an operation executed on a data stream of data. Essentially I want to obtain two different streams from that one to send their to different cassandra tables. I.e.: datastream 0 composed by Tuple3 I want to have: a datastream 1 composed

FLIP-17: Side Inputs

A bit late to this discussion, but I wanted to reiterate something that others also said. Side input readiness, and blocking until that is the case, is an important feature. This is specially true when the side input is used as a configuration stream. You don't want the main stream to be

Is State access synchronized?

Hi, as per the mail subject I wanted to ask you if a State access (read and write) is synchronized. I have the following stream: val airtrafficEvents = stream .keyBy(_.flightInfo.flight) .map(new UpdateIdFunction()) where UpdateIdFunction is a RichMapFunction with a ValueState and a MapState,

Queryable State

Hi All, I'm running a streaming job on flink 1.3.2 with few queryable states. There are 3 task managers and a job manager. I'm getting timeout exception when trying to query a state and also a warning message in the job manager log. *Client:* final Configuration config = new Configuration();

Re: State Issue

Sorry my bad, figured out it was a change done at our end which created different keys. Thanks. On Fri, Sep 8, 2017 at 5:32 PM, Navneeth Krishnan wrote: > Hi, > > I'm experiencing a wired issue where any data put into map state when > retrieved with the same key is

HeapInternalTimerService#advanceWatermark

Hi, i have read this snip of code again and again, but i cant understand the usage of it , can anyone explain it for me. thank you very much. public void advanceWatermark(long time) throws Exception { currentWatermark = time; InternalTimer timer;

Re: LatencyMarker

Hi, You are right, i have proved that the backpressure will increase the latency,and in have another question, now the value of the latency is not Visualization in the dashboard，is there any plan to do this, i have add this by remove the other operator and only keep the source -> end latency and