[jira] [Created] (FLINK-4101) Calculating average in Flink DataStream on window time
Akshay Shingote created FLINK-4101:
--
Summary: Calculating average in Flink DataStream on window time
Key: FLINK-4101
URL: https://issues.apache.org/jira/browse/FLINK-4101
Project: Flink
Issue Type: Task
Components: DataStream API
Affects Versions: 1.0.2
Reporter: Akshay Shingote
I am using Flink DataStream API where there where racks are available & I want
to calculate "average"of temperature group by rack IDs. My window duration is
of 40 seconds & my window is sliding every 10 seconds...Following is my code
where I am calculating sum of temperatures every 10 seconds for every
rackID,but now I want to calculate average temperatures::
static Properties properties=new Properties();
public static Properties getProperties()
{
properties.setProperty("bootstrap.servers", "54.164.200.104:9092");
properties.setProperty("zookeeper.connect", "54.164.200.104:2181");
//properties.setProperty("deserializer.class",
"kafka.serializer.StringEncoder");
//properties.setProperty("group.id", "akshay");
properties.setProperty("auto.offset.reset", "earliest");
return properties;
}
@SuppressWarnings("rawtypes")
public static void main(String[] args) throws Exception
{
StreamExecutionEnvironment
env=StreamExecutionEnvironment.getExecutionEnvironment();
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
Properties props=Program.getProperties();
DataStream dstream=env.addSource(new
FlinkKafkaConsumer09("TemperatureEvent", new
TemperatureEventSchema(), props)).assignTimestampsAndWatermarks(new
IngestionTimeExtractor<>());
DataStream
ds1=dstream.keyBy("rackId").timeWindow(Time.seconds(40),
Time.seconds(10)).sum("temperature");
env.execute("Temperature Consumer");
}
How can I calcluate average temperature for the above example ??
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] Akshay Shingote shared "FLINK-4059: Requirement of Streaming layer for Complex Event Processing" with you
Akshay Shingote shared an issue with you > Requirement of Streaming layer for Complex Event Processing > --- > > Key: FLINK-4059 > URL: https://issues.apache.org/jira/browse/FLINK-4059 > Project: Flink > Issue Type: Test > Components: CEP >Affects Versions: 1.0.1 >Reporter: Akshay Shingote > > I am trying to get this Flink CEP Example : > https://github.com/tillrohrmann/cep-monitoring > One question which arises here is that they have made one single application > which does the work of Pattern Matching or CEP. (Like ingest & process is > done by 1 application)...They didn't use any streaming layer in between like > Kafka or any...I just want to know why any streaming layer is not use in > between the producer & the consumer?? Also,I want to know what will be the > advantages and disadvantages if I put a streaming layer in between ?? Thank > You Also shared with u...@flink.apache.org -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-4059) Requirement of Streaming layer for Complex Event Processing
Akshay Shingote created FLINK-4059: -- Summary: Requirement of Streaming layer for Complex Event Processing Key: FLINK-4059 URL: https://issues.apache.org/jira/browse/FLINK-4059 Project: Flink Issue Type: Test Components: CEP Affects Versions: 1.0.1 Reporter: Akshay Shingote I am trying to get this Flink CEP Example : https://github.com/tillrohrmann/cep-monitoring One question which arises here is that they have made one single application which does the work of Pattern Matching or CEP. (Like ingest & process is done by 1 application)...They didn't use any streaming layer in between like Kafka or any...I just want to know why any streaming layer is not use in between the producer & the consumer?? Also,I want to know what will be the advantages and disadvantages if I put a streaming layer in between ?? Thank You -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4008) Hello,I am not able to create jar of flink-streaming-connectors ...I am able to create jar of others like twitter,kafka,flume but I am not able to create jar of flink-s
[ https://issues.apache.org/jira/browse/FLINK-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319984#comment-15319984 ] Akshay Shingote commented on FLINK-4008: How can I build the snapshot ?? Can you please help it out ?? Thank You > Hello,I am not able to create jar of flink-streaming-connectors ...I am able > to create jar of others like twitter,kafka,flume but I am not able to create > jar of flink-streaming connectors ?? How can I create this jar ?? > --- > > Key: FLINK-4008 > URL: https://issues.apache.org/jira/browse/FLINK-4008 > Project: Flink > Issue Type: Bug > Components: Streaming, Streaming Connectors >Affects Versions: 1.1.0 >Reporter: Akshay Shingote > > I filed an issue here https://github.com/apache/flink/pull/2058 ... I want to > know how can we create jar of Flink-Streaming-Connectors ?? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (FLINK-4008) Hello,I am not able to create jar of flink-streaming-connectors ...I am able to create jar of others like twitter,kafka,flume but I am not able to create jar of fl
[ https://issues.apache.org/jira/browse/FLINK-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312070#comment-15312070 ] Akshay Shingote edited comment on FLINK-4008 at 6/2/16 10:03 AM: - I am trying to put streaming data to Kinesis hence I am using flink-streaming-kinesis connector jar which requires flink-streaming-connector jar Error is Could not find artifact org.apache.flink:flink-streaming-connectors:pom:1.1-SNAPSHOT ?? Hence I want flink-streaming-connector jar..I need to create the jar or I need to resolve that dependency...How can I do this ?? was (Author: mrakki3110): I am using flink-streaming-kinesis connector jar which requires flink-streaming-connector jar Error is Could not find artifact org.apache.flink:flink-streaming-connectors:pom:1.1-SNAPSHOT ?? Hence I want flink-streaming-connector jar..I need to create the jar or I need to resolve that dependency...How can I do this ?? > Hello,I am not able to create jar of flink-streaming-connectors ...I am able > to create jar of others like twitter,kafka,flume but I am not able to create > jar of flink-streaming connectors ?? How can I create this jar ?? > --- > > Key: FLINK-4008 > URL: https://issues.apache.org/jira/browse/FLINK-4008 > Project: Flink > Issue Type: Bug > Components: Streaming, Streaming Connectors >Affects Versions: 1.1.0 >Reporter: Akshay Shingote > > I filed an issue here https://github.com/apache/flink/pull/2058 ... I want to > know how can we create jar of Flink-Streaming-Connectors ?? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4008) Hello,I am not able to create jar of flink-streaming-connectors ...I am able to create jar of others like twitter,kafka,flume but I am not able to create jar of flink-s
[ https://issues.apache.org/jira/browse/FLINK-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312070#comment-15312070 ] Akshay Shingote commented on FLINK-4008: I am using flink-streaming-kinesis connector jar which requires flink-streaming-connector jar Error is Could not find artifact org.apache.flink:flink-streaming-connectors:pom:1.1-SNAPSHOT ?? Hence I want flink-streaming-connector jar..I need to create the jar or I need to resolve that dependency...How can I do this ?? > Hello,I am not able to create jar of flink-streaming-connectors ...I am able > to create jar of others like twitter,kafka,flume but I am not able to create > jar of flink-streaming connectors ?? How can I create this jar ?? > --- > > Key: FLINK-4008 > URL: https://issues.apache.org/jira/browse/FLINK-4008 > Project: Flink > Issue Type: Bug > Components: Streaming, Streaming Connectors >Affects Versions: 1.1.0 >Reporter: Akshay Shingote > > I filed an issue here https://github.com/apache/flink/pull/2058 ... I want to > know how can we create jar of Flink-Streaming-Connectors ?? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-4008) Hello,I am not able to create jar of flink-streaming-connectors ...I am able to create jar of others like twitter,kafka,flume but I am not able to create jar of flink-str
Akshay Shingote created FLINK-4008: -- Summary: Hello,I am not able to create jar of flink-streaming-connectors ...I am able to create jar of others like twitter,kafka,flume but I am not able to create jar of flink-streaming connectors ?? How can I create this jar ?? Key: FLINK-4008 URL: https://issues.apache.org/jira/browse/FLINK-4008 Project: Flink Issue Type: Bug Components: Streaming, Streaming Connectors Affects Versions: 1.1.0 Reporter: Akshay Shingote I filed an issue here https://github.com/apache/flink/pull/2058 ... I want to know how can we create jar of Flink-Streaming-Connectors ?? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-4007) Flink Kafka Consumer throws Null Pointer Exception when using DataStream keyBy
Akshay Shingote created FLINK-4007: -- Summary: Flink Kafka Consumer throws Null Pointer Exception when using DataStream keyBy Key: FLINK-4007 URL: https://issues.apache.org/jira/browse/FLINK-4007 Project: Flink Issue Type: Bug Components: CEP, DataStream API Affects Versions: 1.0.1 Reporter: Akshay Shingote Hello,yesterday I filed an issue here http://stackoverflow.com/questions/37568822/flink-kafka-consumer-throws-null-pointer-exception-when-using-datastream-key-by I want to know how to resolve this issue ?? I am not finding any other example of Flink Kafka Consumer...Thank You -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-3954) Installing Flink on EMR using EMR bootstrap action
Akshay Shingote created FLINK-3954: -- Summary: Installing Flink on EMR using EMR bootstrap action Key: FLINK-3954 URL: https://issues.apache.org/jira/browse/FLINK-3954 Project: Flink Issue Type: Bug Affects Versions: 1.0.2 Environment: AWS EMR 4.6 Reporter: Akshay Shingote Hello, I want to know can Flink be installed on AWS EMR through EMR's bootstrap action?? This will help getting Flink installed on EMR's master & core nodes..so is there any way through which we can install Flink on AWS EMR through bootstrap action ?? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-3915) Updating a Dataset in Flink using Flink's Table API
[ https://issues.apache.org/jira/browse/FLINK-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286627#comment-15286627 ] Akshay Shingote commented on FLINK-3915: Ok..I am not able to copy jar using this dependency : org.apache.flink flink-table_2.10 1.1-SNAPSHOT Can you please help me out in getting Flink 1.1-SNAPSHOT jar ?? > Updating a Dataset in Flink using Flink's Table API > --- > > Key: FLINK-3915 > URL: https://issues.apache.org/jira/browse/FLINK-3915 > Project: Flink > Issue Type: Wish > Components: Table API >Affects Versions: 1.0.1 >Reporter: Akshay Shingote > > I am using Flink Table API. I want to update the table in Flink through > Pattern Detection..I am using 3 fields : routeno,source,distance,category . > Now I want to update the category based on the value of distance for every > routeno...For ex : if routeno=1 and distance<=200 then category='daily' ..How > can I update this using Flink's Table API?? Can we update dataset or Table > using Flink's Table API ?? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-3915) Updating a Dataset in Flink using Flink's Table API
[ https://issues.apache.org/jira/browse/FLINK-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286477#comment-15286477 ] Akshay Shingote commented on FLINK-3915: Can you give any estimated date when Flink 1.1.0 will be released ?? > Updating a Dataset in Flink using Flink's Table API > --- > > Key: FLINK-3915 > URL: https://issues.apache.org/jira/browse/FLINK-3915 > Project: Flink > Issue Type: Wish > Components: Table API >Affects Versions: 1.0.1 >Reporter: Akshay Shingote > > I am using Flink Table API. I want to update the table in Flink through > Pattern Detection..I am using 3 fields : routeno,source,distance,category . > Now I want to update the category based on the value of distance for every > routeno...For ex : if routeno=1 and distance<=200 then category='daily' ..How > can I update this using Flink's Table API?? Can we update dataset or Table > using Flink's Table API ?? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-3915) Updating a Dataset in Flink using Flink's Table API
[ https://issues.apache.org/jira/browse/FLINK-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286470#comment-15286470 ] Akshay Shingote commented on FLINK-3915: Ok...Thanks Fabian > Updating a Dataset in Flink using Flink's Table API > --- > > Key: FLINK-3915 > URL: https://issues.apache.org/jira/browse/FLINK-3915 > Project: Flink > Issue Type: Wish > Components: Table API >Affects Versions: 1.0.1 >Reporter: Akshay Shingote > > I am using Flink Table API. I want to update the table in Flink through > Pattern Detection..I am using 3 fields : routeno,source,distance,category . > Now I want to update the category based on the value of distance for every > routeno...For ex : if routeno=1 and distance<=200 then category='daily' ..How > can I update this using Flink's Table API?? Can we update dataset or Table > using Flink's Table API ?? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-3915) Updating a Dataset in Flink using Flink's Table API
Akshay Shingote created FLINK-3915: -- Summary: Updating a Dataset in Flink using Flink's Table API Key: FLINK-3915 URL: https://issues.apache.org/jira/browse/FLINK-3915 Project: Flink Issue Type: Wish Components: Table API Affects Versions: 1.0.1 Reporter: Akshay Shingote I am using Flink Table API. I want to update the table in Flink through Pattern Detection..I am using 3 fields : routeno,source,distance,category . Now I want to update the category based on the value of distance for every routeno...For ex : if routeno=1 and distance<=200 then category='daily' ..How can I update this using Flink's Table API?? Can we update dataset or Table using Flink's Table API ?? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
