Re: My task managers are not starting

2018-07-31 Thread Felipe Gutierrez
Strange. I decreased the max value (TM_MAX_OFFHEAP_SIZE) even it says the slave will not use all space and it worked. # Long.MAX_VALUE in TB: This is an upper bound, much less direct memory will be used # TM_MAX_OFFHEAP_SIZE="8388607T" *--* *-- Felipe Gutierrez*

My task managers are not starting

2018-07-31 Thread Felipe Gutierrez
no success. Does anyone could help me to start the cluster? thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

Re: connection failed when running flink in a cluster

2018-08-07 Thread Felipe Gutierrez
Gary. Best Regards, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Mon, Aug 6, 2018 at 9:57 PM Gary Yao wrote: > Hi, > > Can you try submitting with: > > ./b

Re: connection failed when running flink in a cluster

2018-08-06 Thread Felipe Gutierrez
:745) 2017-05-30 05:10:40,016 INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: Socket Stream -> Flat Map (1/1) (d5e3d87395995d3977d2f472de896e23). *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blo

Re: connection failed when running flink in a cluster

2018-08-06 Thread Felipe Gutierrez
didn`t include in my deploy. Do you have any clue? I am following the original quickstart ( https://ci.apache.org/projects/flink/flink-docs-master/quickstart/setup_quickstart.html ) Kind Regards, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https

connection failed when running flink in a cluster

2018-08-06 Thread Felipe Gutierrez
PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1) (aa1a0e7ee3a1d3ad8f99b2608bd64c5b) switched from CANCELING to CANCELED. Thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

Re: connection failed when running flink in a cluster

2018-08-06 Thread Felipe Gutierrez
yes. when I execute the jps command on the master node I see TaskManagerRunner and StandaloneSessionClusterEntrypoint (which I believe it is the jobManager). On the slave nodes I see TaskManagerRunner when I run jps command *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https

Re: connection failed when running flink in a cluster

2018-08-06 Thread Felipe Gutierrez
completed? > > If it throws the exception you provided, the job's status should be failed. > > Thanks, vino. > > 2018-08-06 23:42 GMT+08:00 Felipe Gutierrez > : > >> yes. with this example (examples/streaming/WordCount.jar) my cluster >> worked. >> >> the

Implement a sort inside the WindowFunction

2018-03-09 Thread Felipe Gutierrez
Hi all, I have a word count using flink stream and mey reduce transformations is applying a WindowFunction. I would like that this WindowFunction sort the output of the reduce. Is that possible? So I will sort by key the data set inside the window. Thanks for your ideas! Here is my code:

Operators in Flink

2018-04-20 Thread Felipe Gutierrez
Hi, I have a doubt about Flink operators implementation and I am trying to search some link on the internet about this Do the operators in Flink are compiled and fused at runtime? Do you guys have a link that can explain to me how it is implemented in Flink? Kind Regards, Felipe *--* *-- Felipe

Re: Operators in Flink

2018-04-25 Thread Felipe Gutierrez
thanks Robert, I will definitely have a look at it. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Wed, Apr 25, 2018 at 9:45 AM, Robert Metzger <rmetz...@apache.org> wrote:

How to correct use timeWindow() with DataStream?

2018-03-16 Thread Felipe Gutierrez
Hi all, I am building an example with DataStream using Flink that has a fake source generator of LogLine(Date d, String line). I want to work with Watermarks on it so I created a class that implements AssignerWithPeriodicWatermarks. If I don't use the monad ".timeWindow(Time.seconds(2))" on the

Re: Implement a sort inside the WindowFunction

2018-03-16 Thread Felipe Gutierrez
have to add a non-keyed window (allWindow) for your sorting > WindowFunction. > Note that this function cannot run in parallel. > > Best, Fabian > > 2018-03-09 22:07 GMT+01:00 Felipe Gutierrez <felipe.o.gutier...@gmail.com> > : > >> Hi all, >> >> I have

Re: How to correct use timeWindow() with DataStream?

2018-03-19 Thread Felipe Gutierrez
t want to partition by key, i.e., have a single result for each > time window, you should not use keyBy and an allWindow. > However, this will only be executed with a parallelism of 1. > > 2018-03-19 13:54 GMT+01:00 Felipe Gutierrez <felipe.o.gutier...@gmail.com> > : > >&g

Re: How to correct use timeWindow() with DataStream?

2018-03-19 Thread Felipe Gutierrez
e (i.e., from 0 to 60). You should use Date.getTime() instead of > Date.getSeconds(). > > Best, Fabian > > 2018-03-16 18:08 GMT+01:00 Felipe Gutierrez <felipe.o.gutier...@gmail.com> > : > >> Hi all, >> >> I am building an example with DataStream using Flink th

Flink join operator after sorting seems to group fields (Scala)

2018-03-03 Thread Felipe Gutierrez
Hi all, I have implemented a simple Scala object using Flink to play with joins operator. After that, I put the join operator show my results I decided to sort the output by the first field (.sortPartition(0, Order.ASCENDING)). It seems that the output is ordered by group. The output shows two

Re: Custom scheduler in Flink

2018-11-30 Thread Felipe Gutierrez
thanks. I will check it out *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Fri, Nov 30, 2018 at 3:51 PM Till Rohrmann wrote: > Hi Felipe, > > https://issues.apache.org/jira/bro

Re: Custom scheduler in Flink

2018-11-30 Thread Felipe Gutierrez
thanks. I saw the google docs just now. I am gonna print and study it at the weekend. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Fri, Nov 30, 2018 at 4:42 PM Felipe Gutierrez < felipe

Custom scheduler in Flink

2018-11-28 Thread Felipe Gutierrez
saw some related question on this post from 2016 ( http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/custom-scheduler-in-Flink-td7804.html) and I wonder if there is something more actual or better explained about a custom scheduler. Thanks, Felipe *--* *-- Felipe Gutierrez

Re: Looking for relevant sources related to connecting Apache Flink and Edgent.

2018-11-30 Thread Felipe Gutierrez
this description as actual of latest Flink source code? *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Nov 29, 2018 at 12:01 PM Felipe Gutierrez < felipe.o.gutier...@gmail.com> wrot

Looking for relevant sources related to connecting Apache Flink and Edgent.

2018-11-29 Thread Felipe Gutierrez
that is interesting regarding my prototype? Thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

Re: Looking for relevant sources related to connecting Apache Flink and Edgent.

2018-11-29 Thread Felipe Gutierrez
nent on Flink I can get the operations on the fly that are running inside a query? thanks *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Nov 29, 2018 at 11:18 AM Kostas Kloudas wrote:

Re: Looking for relevant sources related to connecting Apache Flink and Edgent.

2018-11-30 Thread Felipe Gutierrez
> > Best, Fabian > > Am Fr., 30. Nov. 2018 um 10:13 Uhr schrieb Felipe Gutierrez < > felipe.o.gutier...@gmail.com>: > >> I guess this message from 2016 is very related of what I am looking for ( >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabbl

Re: Custom scheduler in Flink

2018-11-28 Thread Felipe Gutierrez
Thanks, I'll check it out. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Wed, Nov 28, 2018 at 2:44 PM Chesnay Schepler wrote: > There's no *reasonable *way to implement a custom Sched

Question about Flink optimizer on Stream API

2018-12-19 Thread Felipe Gutierrez
https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/SensorsReadingMqttJoinQEP.java *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

How can I visualize the metrics Counter and Meter at VisualVM?

2019-03-29 Thread Felipe Gutierrez
9-8790 metrics.reporter.jmx.interval: 30 SECONDS Thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

print() method does not always print on the taskmanager.out file

2019-04-04 Thread Felipe Gutierrez
eyDAG.java#L48 Thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

Re: print() method does not always print on the taskmanager.out file

2019-04-05 Thread Felipe Gutierrez
I guess there is something to do with the parallelism of the cluster. When I set "taskmanager.numberOfTaskSlots" to 1 and do not use "setParallelism()" I can see the logs. And on Eclipse I can see the logs. Does anybody have a clue? Thanks *--* *-- Felipe Gutierrez* *-- skype

Re: print() method does not always print on the taskmanager.out file

2019-04-05 Thread Felipe Gutierrez
is a virtual machine. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Fri, Apr 5, 2019 at 2:50 PM Chesnay Schepler wrote: > This kind of sounds like a Outputstream flushing issue. Tr

Re: print() method does not always print on the taskmanager.out file

2019-04-08 Thread Felipe Gutierrez
machine I cannot see the output. Thanks for your help anyway Chesnay! *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Fri, Apr 5, 2019 at 6:01 PM Felipe Gutierrez < felipe.o.gutier...@

How do I compute the average and keep track of a state over a window in DataStream?

2019-02-28 Thread Felipe Gutierrez
ense/flink/examples/stream/MultiSensorMultiStationsReadingMqtt2.java#L70 How would I implement this in Flink? Thanks for your ideas Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

Re: How do I compute the average and keep track of a state over a window in DataStream?

2019-03-01 Thread Felipe Gutierrez
thanks Congxian. I will check Process Function over windows. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Fri, Mar 1, 2019 at 8:16 AM Congxian Qiu wrote: > Hi Felipe > > Mayb

Re: What is the best way to handle data skew processing in Data Stream applications?

2019-04-11 Thread Felipe Gutierrez
ormId, skewParameter); But it is still a static solution =(. I mean, the developer has to set on the Mapper which key is skewed. Best, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Ap

What is the best way to handle data skew processing in Data Stream applications?

2019-04-10 Thread Felipe Gutierrez
indowFunction)).name(metricWindowFunction) .map(new StationPlatformMapper(metricSkewedMapper)).name(metricSkewedMapper) .addSink(new MqttStationPlatformPublisher(ipAddressSink, topic)).name(metricSinkFunction) ; Thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez

Re: How would I use OneInputStreamOperator to deal with data skew?

2019-04-15 Thread Felipe Gutierrez
ink flink-table-planner_2.11 1.8.0 org.apache.flink flink-table-api-java-bridge_2.11 1.8.0 org.apache.flink flink-streaming-scala_2.11 1.8.0 org.apache.flink flink-table-common 1.8.0 org.apache.flink flink-table 1.8.0 pom provided org.apache.flink flink-table-runtime-blink 1.8.0 *--* *--

Re: How would I use OneInputStreamOperator to deal with data skew?

2019-04-15 Thread Felipe Gutierrez
Cool, thanks Kurt! *-* *- Felipe Gutierrez* *- skype: felipe.o.gutierrez* *- **https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* * <https://felipeogutierrez.blogspot.com>* On Mon, Apr 15, 2019 at 6:06 AM Kurt Young wrote: > Hi, > > You ca

Re: How would I use OneInputStreamOperator to deal with data skew?

2019-04-15 Thread Felipe Gutierrez
oh, yes. I just saw. I will use 1.9 then. thanks *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Mon, Apr 15, 2019 at 3:23 PM Kurt Young wrote: > It's because all blink codes are no

Re: How would I use OneInputStreamOperator to deal with data skew?

2019-04-15 Thread Felipe Gutierrez
org/codehaus/janino/** org/codehaus/commons/** org.sense.flink.App false Thanks *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Mon, Apr 15, 2019 at 3:25 PM Felipe Gutierre

How would I use OneInputStreamOperator to deal with data skew?

2019-04-12 Thread Felipe Gutierrez
be, is there someone that have an example of a use case similar to this? [1] https://youtu.be/mSLesPzWplA?t=835 [2] https://ci.apache.org/projects/flink/flink-docs-master/api/java/index.html?org/apache/flink/streaming/api/functions/source/ContinuousFileReaderOperator.html Thanks! Felipe *--* *-

Re: How would I use OneInputStreamOperator to deal with data skew?

2019-04-16 Thread Felipe Gutierrez
m/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/MqttSensorDataSkewedCombinerByKeySkewedDAG.java#L86> . Thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.

Re: How would I use OneInputStreamOperator to deal with data skew?

2019-04-17 Thread Felipe Gutierrez
ors/StreamFlatMap.java#L29>" class implementation. Best, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Wed, Apr 17, 2019 at 4:13 AM Kurt Young wrote: > I think you might mixed som

Re: How can I improve this Flink application for "Distinct Count of elements" in the data stream?

2019-06-12 Thread Felipe Gutierrez
-master/api/java/index.html?org/apache/flink/streaming/api/functions/windowing/ProcessWindowFunction.html Kind Regards, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Wed, Jun 12, 201

How can I improve this Flink application for "Distinct Count of elements" in the data stream?

2019-06-11 Thread Felipe Gutierrez
the words using "Tuple2(0, word)", so I will have always the same key (0). As I understand, all the events will be processed on the same TaskManager which will not achieve parallelism if I am in a cluster. Kind Regards, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *

Why the current time of my window never reaches the end time when I use KeyedProcessFunction?

2019-06-17 Thread Felipe Gutierrez
/WordHLLKeyedProcessWindowSocket.java [3] https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/WordHLLKeyedProcessWindowTwitter.java Kind Regards, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com

Re: Why the current time of my window never reaches the end time when I use KeyedProcessFunction?

2019-06-18 Thread Felipe Gutierrez
I am sorry, I wanted to point this reference https://stackoverflow.com/a/47071833/2096986 which implements a window on a ProcessFunction in Flink. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.

Re: How can I improve this Flink application for "Distinct Count of elements" in the data stream?

2019-06-13 Thread Felipe Gutierrez
*--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Jun 13, 2019 at 8:10 AM Hequn Cheng wrote: > Hi Felipe, > > From your code, I think you want to get the "count distin

Re: How can I improve this Flink application for "Distinct Count of elements" in the data stream?

2019-06-13 Thread Felipe Gutierrez
humm.. it seems that it is my turn to implement all this stuff using Table API. Thanks Rong! *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Jun 13, 2019 at 6:00 PM Rong Rong wrote:

Re: How can I improve this Flink application for "Distinct Count of elements" in the data stream?

2019-06-12 Thread Felipe Gutierrez
some improvements? [1] https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/WordDistinctCountProcessTimeWindowSocket.java Thanks! *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <ht

Re: Calculate a 4 hour time window for sum,count with sum and count output every 5 mins

2019-06-18 Thread Felipe Gutierrez
k/examples/stream/WordHLLKeyedProcessWindowTwitter.java Kind Regards, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Mon, Jun 17, 2019 at 8:57 PM Rafi Aroch wrote: > Hi Vijay, >

Re: Why the current time of my window never reaches the end time when I use KeyedProcessFunction?

2019-06-18 Thread Felipe Gutierrez
/a/53646529/2096986 [2] https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/WordHLLKeyedProcessWindowTwitter.java *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez

Hello-world example of Flink Table API using a edited Calcite rule

2019-06-26 Thread Felipe Gutierrez
ble/MqttSensorDataAverageTableAPI.java#L40 Kind Regards, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

Error when creating InMemoryExternalCatalog to populate using another stream.

2019-06-26 Thread Felipe Gutierrez
link.table.api.TableEnvironment.scan(TableEnvironment.scala:620) at org.sense.flink.examples.stream.table.HelloWordExternalCatalogTableAPI.(HelloWordExternalCatalogTableAPI.java:55) at org.sense.flink.examples.stream.table.HelloWordExternalCatalogTableAPI.main(HelloWordExternalCatalogTableAPI.java:32) [1] https://

Re: Hello-world example of Flink Table API using a edited Calcite rule

2019-06-26 Thread Felipe Gutierrez
com/questions/54278508/how-can-i-create-an-external-catalog-table-in-apache-flink > [4] > https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/table/MqttSensorDataAverageTableAPI.java#L40 > > Kind Regards, > Felipe > *--* > *-- Felipe Gutierrez* > > *-- skype: felipe.o.gutierrez* > *--* *https://felipeogutierrez.blogspot.com > <https://felipeogutierrez.blogspot.com>* > > >

Re: Add Logical filter on a query plan from Flink Table API

2019-07-09 Thread Felipe Gutierrez
ilder().addLogicalOptRuleSet(), .addNormRuleSet(), .addPhysicalOptRuleSet()". [1] https://calcite.apache.org/apidocs/org/apache/calcite/rel/core/RelFactories.html#LOGICAL_BUILDER *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.

Re: Add Logical filter on a query plan from Flink Table API

2019-07-09 Thread Felipe Gutierrez
o the code... Thanks anyway! [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/FilterJoinRule.java *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Tue,

Add Logical filter on a query plan from Flink Table API

2019-07-08 Thread Felipe Gutierrez
/blob/master/src/main/java/org/sense/calcite/rules/MyFilterRule.java#L14 *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

How to implement custom stream operator over a window? And after the Count-Min Sketch?

2019-04-23 Thread Felipe Gutierrez
/api/functions/windowing/AggregateApplyWindowFunction.html [3] https://github.com/tillrohrmann/custom-join [4] https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/operator/AbstractRichMapStreamBundleOperator.java Thanks, Felipe *--* *-- Felipe Gutierrez* *-- s

Re: What is the best way to handle data skew processing in Data Stream applications?

2019-04-17 Thread Felipe Gutierrez
*--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Apr 11, 2019 at 2:21 PM Felipe Gutierrez < felipe.o.gutier...@gmail.com> wrote: > thanks All for your suggestions! > > I am not

Re: How to implement custom stream operator over a window? And after the Count-Min Sketch?

2019-04-24 Thread Felipe Gutierrez
-2147 Best, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Wed, Apr 24, 2019 at 2:06 AM Rong Rong wrote: > Hi Felipe, > > In a short glance, the question can depend on

Re: How to implement custom stream operator over a window? And after the Count-Min Sketch?

2019-04-26 Thread Felipe Gutierrez
wse/FLINK-2147> JIRA ticket. Doing so I can collect more opinions =) Thanks! Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Fri, Apr 26, 2019 at 4:10 AM Rong Rong wrote: > Hi F

Re: Hello-world example of Flink Table API using a edited Calcite rule

2019-06-27 Thread Felipe Gutierrez
? ps.: I am gonna look at the links that you mentioned. Thanks for that! Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Jun 27, 2019 at 7:23 AM JingsongLee wrote: > Hi Felipe:

Re: Calculate a 4 hour time window for sum,count with sum and count output every 5 mins

2019-07-01 Thread Felipe Gutierrez
No, there is no specific reason. I am using it because I am computing the HyperLogLog over a window. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Mon, Jul 1, 2019 at 12:34 AM Vijay Balakr

Implementing a low level join

2019-08-13 Thread Felipe Gutierrez
/projects/flink/flink-docs-release-1.8/dev/stream/operators/process_function.html#low-level-joins [2] https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com

Re: Implementing a low level join

2019-08-14 Thread Felipe Gutierrez
ream/state/broadcast_state.html > [2] > https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/table/sql.html#joins > [3] > https://github.com/apache/flink/tree/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/runtime/join > > > On Tue, Aug 1

Re: Implementing a low level join

2019-08-15 Thread Felipe Gutierrez
, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Aug 15, 2019 at 9:42 AM Fabian Hueske wrote: > Hi, > > Just to clarify. You cannot dynamically switch the join strateg

Re: Implementing a low level join

2019-08-15 Thread Felipe Gutierrez
I see, I am gonna try this. Thanks Hequn *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Aug 15, 2019 at 4:01 AM Hequn Cheng wrote: > Hi Felipe, > > If I understand correctly

Re: Implementing a low level join

2019-08-15 Thread Felipe Gutierrez
Thanks for the advice. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Aug 15, 2019 at 9:59 AM Fabian Hueske wrote: > Hi Felipe, > > No, this is not possible (with re

How to implement different Join strategies for a Flink stream application?

2019-08-09 Thread Felipe Gutierrez
inExercise.java ) If I want to decide whether I use BradCastJoin or HashJoin or any other Join algorithm, which way do you think it is better? is there any other example code that I could borrow? Thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierr

Re: Can I use watermarkers to have a global trigger of different ProcessFunction's?

2019-08-22 Thread Felipe Gutierrez
thanks for the detail explanation! I removed my implementation of the watermark which is not necessary in my case. I will only use Watermarkers if I am dealing with out of order events. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <ht

Can I use watermarkers to have a global trigger of different ProcessFunction's?

2019-08-21 Thread Felipe Gutierrez
merService().registerEventTimeTimer() and what is the logic that I should use in the onTimer() method? [1] https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/valencia/ValenciaDataSkewedBloomFilterJoinExample.java#L47 Thanks, Felipe *--* *-- Felipe Gutierrez*

Flink & Mesos don't launch Job and Task managers

2019-09-05 Thread Felipe Gutierrez
: 3500 [1] https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/deployment/mesos.html#mesos-without-dcos Thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

How to access a file with Flink application running on Mesos?

2019-09-05 Thread Felipe Gutierrez
) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530) at java.lang.Thread.run(Thread.java:748) *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>*

Re: Flink & Mesos don't launch Job and Task managers

2019-09-05 Thread Felipe Gutierrez
my bad. Flink allocates task managers dynamically. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Thu, Sep 5, 2019 at 5:24 PM Felipe Gutierrez < felipe.o.gutier...@gmail.com> wrote:

How do I start a Flink application on my Flink+Mesos cluster?

2019-09-06 Thread Felipe Gutierrez
.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.

Re: How do I start a Flink application on my Flink+Mesos cluster?

2019-09-12 Thread Felipe Gutierrez
om/apache/flink/blob/0a405251b297109fde1f9a155eff14be4d943887/flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java#L344 > > On Tue, Sep 10, 2019 at 10:41 AM Felipe Gutierrez < > felipe.o.gutier...@gmail.com> wrote: > >>

Error "Failed to load native Mesos library from" when I run Flink on a compiled version of Apache Mesos

2019-09-17 Thread Felipe Gutierrez
to load native Mesos library from /usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib Thanks, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez

Re: Client for Monitoring API!

2019-09-18 Thread Felipe Gutierrez
yes. you can use prometheus+Grafana. https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter https://felipeogutierrez.blogspot.com/2019/04/monitoring-apache-flink-with-prometheus.html Felipe On 2019/09/18

Re: How do I start a Flink application on my Flink+Mesos cluster?

2019-09-10 Thread Felipe Gutierrez
.cpus" to be equal or less the available cores on a single node of the cluster. I am not sure about this parameter, but only after this configuration it worked. Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez

Re: Error "Failed to load native Mesos library from" when I run Flink on a compiled version of Apache Mesos

2019-09-19 Thread Felipe Gutierrez
BlobServer - Stopped BLOB server at 0.0.0.0:37375 *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Wed, Sep 18, 2019 at 4:53 AM Rui Li wrote: > Hey Felipe, > > I haven't tr