[jira] [Created] (FLINK-7962) Add built-in support for min/max aggregation for Timestamp

[Dian Fu (JIRA)]

Dian Fu created FLINK-7962:
--

 Summary: Add built-in support for min/max aggregation for Timestamp
 Key: FLINK-7962
 URL: https://issues.apache.org/jira/browse/FLINK-7962
 Project: Flink
  Issue Type: Task
Reporter: Dian Fu
Assignee: Dian Fu
Priority: Major


This JIRA adds the built-in support for min/max aggregation for Timestamp.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New committer: Haohui Mai

[Dawid Wysakowicz]

Congratulations!

01.11.2017 7:45 PM "Stephan Ewen"  napisał(a):

> Congrats and welcome!
>
> On Wed, Nov 1, 2017 at 6:47 PM, Chen Qin  wrote:
>
> > Congratulations!
> >
> > On Wed, Nov 1, 2017 at 2:41 AM, Aljoscha Krettek 
> > wrote:
> >
> > > Congratulations! 
> > >
> > > > On 1. Nov 2017, at 10:13, Shaoxuan Wang  wrote:
> > > >
> > > > Congratulations!
> > > >
> > > > On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann 
> > > wrote:
> > > >
> > > >> Congrats and welcome on board :-)
> > > >>
> > > >> On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske 
> > > wrote:
> > > >>
> > > >>> Hi everybody,
> > > >>>
> > > >>> On behalf of the PMC I am delighted to announce Haohui Mai as a new
> > > Flink
> > > >>> committer!
> > > >>>
> > > >>> Haohui has been an active member of our community for several
> months.
> > > >>> Among other things, he made major contributions in ideas and code
> to
> > > the
> > > >>> SQL and Table APIs.
> > > >>>
> > > >>> Please join me in congratulating Haohui for becoming a Flink
> > committer!
> > > >>>
> > > >>> Cheers,
> > > >>> Fabian
> > > >>>
> > > >>
> > >
> > >
> >
>

Re: [ANNOUNCE] New committer: Haohui Mai

[Stephan Ewen]

Congrats and welcome!

On Wed, Nov 1, 2017 at 6:47 PM, Chen Qin  wrote:

> Congratulations!
>
> On Wed, Nov 1, 2017 at 2:41 AM, Aljoscha Krettek 
> wrote:
>
> > Congratulations! 
> >
> > > On 1. Nov 2017, at 10:13, Shaoxuan Wang  wrote:
> > >
> > > Congratulations!
> > >
> > > On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann 
> > wrote:
> > >
> > >> Congrats and welcome on board :-)
> > >>
> > >> On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske 
> > wrote:
> > >>
> > >>> Hi everybody,
> > >>>
> > >>> On behalf of the PMC I am delighted to announce Haohui Mai as a new
> > Flink
> > >>> committer!
> > >>>
> > >>> Haohui has been an active member of our community for several months.
> > >>> Among other things, he made major contributions in ideas and code to
> > the
> > >>> SQL and Table APIs.
> > >>>
> > >>> Please join me in congratulating Haohui for becoming a Flink
> committer!
> > >>>
> > >>> Cheers,
> > >>> Fabian
> > >>>
> > >>
> >
> >
>

Re: [ANNOUNCE] New committer: Haohui Mai

[Chen Qin]

Congratulations!

On Wed, Nov 1, 2017 at 2:41 AM, Aljoscha Krettek 
wrote:

> Congratulations! 
>
> > On 1. Nov 2017, at 10:13, Shaoxuan Wang  wrote:
> >
> > Congratulations!
> >
> > On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann 
> wrote:
> >
> >> Congrats and welcome on board :-)
> >>
> >> On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske 
> wrote:
> >>
> >>> Hi everybody,
> >>>
> >>> On behalf of the PMC I am delighted to announce Haohui Mai as a new
> Flink
> >>> committer!
> >>>
> >>> Haohui has been an active member of our community for several months.
> >>> Among other things, he made major contributions in ideas and code to
> the
> >>> SQL and Table APIs.
> >>>
> >>> Please join me in congratulating Haohui for becoming a Flink committer!
> >>>
> >>> Cheers,
> >>> Fabian
> >>>
> >>
>
>

[jira] [Created] (FLINK-7961) Docker-Flink with Docker Swarm doesn't work when machines are in different clouds

[Thalita Vergilio (JIRA)]

Thalita Vergilio created FLINK-7961:
---

 Summary: Docker-Flink with Docker Swarm doesn't work when machines 
are in different clouds
 Key: FLINK-7961
 URL: https://issues.apache.org/jira/browse/FLINK-7961
 Project: Flink
  Issue Type: Bug
  Components: Docker
Affects Versions: 1.3.0
Reporter: Thalita Vergilio
Priority: Normal


Task Managers can't find Job Manager by name. Maybe some additional Docker 
configuration is needed?

I am running the standard setup and create-docker-swarm-service.sh script from 
the Docker Flink project:
https://github.com/apache/flink/blob/master/flink-contrib/docker-flink/create-docker-swarm-service.sh

This is the log from one of the Task Manager's containers:

{{Starting Task Manager
config file:
jobmanager.rpc.address: flink-jobmanager
jobmanager.rpc.port: 6123
jobmanager.heap.mb: 1024
taskmanager.heap.mb: 1024
taskmanager.numberOfTaskSlots: 2
taskmanager.memory.preallocate: false
parallelism.default: 1
jobmanager.web.port: 8081
blob.server.port: 6124
query.server.port: 6125
Starting taskmanager as a console application on host c42a6093f7bb.
2017-11-01 11:20:51,459 WARN  org.apache.hadoop.util.NativeCodeLoader   
- Unable to load native-hadoop library for your platform... using 
builtin-java classes where applicable
2017-11-01 11:20:51,522 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- 

2017-11-01 11:20:51,522 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
-  Starting TaskManager (Version: 1.3.2, Rev:0399bee, 
Date:03.08.2017 @ 10:23:11 UTC)
2017-11-01 11:20:51,522 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
-  Current user: flink
2017-11-01 11:20:51,522 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
-  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 
1.8/25.141-b15
2017-11-01 11:20:51,522 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
-  Maximum heap size: 1024 MiBytes
2017-11-01 11:20:51,522 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
-  JAVA_HOME: /docker-java-home/jre
2017-11-01 11:20:51,526 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
-  Hadoop version: 2.7.2
2017-11-01 11:20:51,526 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
-  JVM Options:
2017-11-01 11:20:51,526 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- -XX:+UseG1GC
2017-11-01 11:20:51,526 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- -Xms1024M
2017-11-01 11:20:51,526 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- -Xmx1024M
2017-11-01 11:20:51,526 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- -XX:MaxDirectMemorySize=8388607T
2017-11-01 11:20:51,526 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- 
-Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties
2017-11-01 11:20:51,526 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- 
-Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml
2017-11-01 11:20:51,526 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
-  Program Arguments:
2017-11-01 11:20:51,527 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- --configDir
2017-11-01 11:20:51,527 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- /opt/flink/conf
2017-11-01 11:20:51,527 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
-  Classpath: 
/opt/flink/lib/flink-python_2.11-1.3.2.jar:/opt/flink/lib/flink-shaded-hadoop2-uber-1.3.2.jar:/opt/flink/lib/log4j-1.2.17.jar:/opt/flink/lib/slf4j-log4j12-1.7.7.jar:/opt/flink/lib/flink-dist_2.11-1.3.2.jar:::
2017-11-01 11:20:51,527 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- 

2017-11-01 11:20:51,528 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- Registered UNIX signal handlers for [TERM, HUP, INT]
2017-11-01 11:20:51,532 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- Maximum number of open file descriptors is 1048576
2017-11-01 11:20:51,548 INFO  org.apache.flink.runtime.taskmanager.TaskManager  
- Loading configuration from /opt/flink/conf
2017-11-01 11:20:51,551 INFO  
org.apache.flink.configuration.GlobalConfiguration- Loading 
configuration property: jobmanager.rpc.address, flink-jobmanager
2017-11-01 11:20:51,551 INFO  
org.apache.flink.configuration.GlobalConfiguration- Loading 
configuration property: jobmanager.rpc.port, 6123
2017-11-01 11:20:51,551 INFO  

Re: [ANNOUNCE] New committer: Haohui Mai

[Peter Huang]

Congratulations!

On Wed, Nov 1, 2017 at 10:04 AM, Bowen Li  wrote:

> Congratulations!
>
> On Wed, Nov 1, 2017 at 8:39 AM, Xingcan Cui  wrote:
>
> > Congratulations!
> >
> > On Wed, Nov 1, 2017 at 9:37 PM, Kurt Young  wrote:
> >
> > > Congrats and welcome on board!
> > >
> > > Best,
> > > Kurt
> > >
> > > On Wed, Nov 1, 2017 at 8:15 PM, Hai Zhou  wrote:
> > >
> > >> Congratulations!
> > >>
> > >> On 1. Nov 2017, at 10:13, Shaoxuan Wang  wrote:
> > >>
> > >> Congratulations!
> > >>
> > >> On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann 
> > >> wrote:
> > >>
> > >> Congrats and welcome on board :-)
> > >>
> > >> On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske 
> > wrote:
> > >>
> > >> Hi everybody,
> > >>
> > >> On behalf of the PMC I am delighted to announce Haohui Mai as a new
> > Flink
> > >> committer!
> > >>
> > >> Haohui has been an active member of our community for several months.
> > >> Among other things, he made major contributions in ideas and code to
> the
> > >> SQL and Table APIs.
> > >>
> > >> Please join me in congratulating Haohui for becoming a Flink
> committer!
> > >>
> > >> Cheers,
> > >> Fabian
> > >>
> > >>
> > >>
> > >>
> > >>
> > >
> >
>

Re: [ANNOUNCE] New committer: Haohui Mai

[Bowen Li]

Congratulations!

On Wed, Nov 1, 2017 at 8:39 AM, Xingcan Cui  wrote:

> Congratulations!
>
> On Wed, Nov 1, 2017 at 9:37 PM, Kurt Young  wrote:
>
> > Congrats and welcome on board!
> >
> > Best,
> > Kurt
> >
> > On Wed, Nov 1, 2017 at 8:15 PM, Hai Zhou  wrote:
> >
> >> Congratulations!
> >>
> >> On 1. Nov 2017, at 10:13, Shaoxuan Wang  wrote:
> >>
> >> Congratulations!
> >>
> >> On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann 
> >> wrote:
> >>
> >> Congrats and welcome on board :-)
> >>
> >> On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske 
> wrote:
> >>
> >> Hi everybody,
> >>
> >> On behalf of the PMC I am delighted to announce Haohui Mai as a new
> Flink
> >> committer!
> >>
> >> Haohui has been an active member of our community for several months.
> >> Among other things, he made major contributions in ideas and code to the
> >> SQL and Table APIs.
> >>
> >> Please join me in congratulating Haohui for becoming a Flink committer!
> >>
> >> Cheers,
> >> Fabian
> >>
> >>
> >>
> >>
> >>
> >
>

Re: [ANNOUNCE] New committer: Haohui Mai

[Xingcan Cui]

Congratulations!

On Wed, Nov 1, 2017 at 9:37 PM, Kurt Young  wrote:

> Congrats and welcome on board!
>
> Best,
> Kurt
>
> On Wed, Nov 1, 2017 at 8:15 PM, Hai Zhou  wrote:
>
>> Congratulations!
>>
>> On 1. Nov 2017, at 10:13, Shaoxuan Wang  wrote:
>>
>> Congratulations!
>>
>> On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann 
>> wrote:
>>
>> Congrats and welcome on board :-)
>>
>> On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske  wrote:
>>
>> Hi everybody,
>>
>> On behalf of the PMC I am delighted to announce Haohui Mai as a new Flink
>> committer!
>>
>> Haohui has been an active member of our community for several months.
>> Among other things, he made major contributions in ideas and code to the
>> SQL and Table APIs.
>>
>> Please join me in congratulating Haohui for becoming a Flink committer!
>>
>> Cheers,
>> Fabian
>>
>>
>>
>>
>>
>

Re: Get Window Time

[miki haiat]

Can you post the code please

On 1 Nov 2017 16:58, "Erdem erdfem"  wrote:

> Hello,
>
> I have a datastream with sliding windows. I want to ask how can i get
> which window pattern matches?
> ex window 3 : 11m-21m
>
> [image: Satır içi resim 1]
>

Get Window Time

[Erdem erdfem]

Hello,

I have a datastream with sliding windows. I want to ask how can i get which
window pattern matches?
ex window 3 : 11m-21m

[image: Satır içi resim 1]

[jira] [Created] (FLINK-7960) ExecutionGraphRestartTest fails sporadically

[Gary Yao (JIRA)]

Gary Yao created FLINK-7960:
---

 Summary: ExecutionGraphRestartTest fails sporadically
 Key: FLINK-7960
 URL: https://issues.apache.org/jira/browse/FLINK-7960
 Project: Flink
  Issue Type: Bug
  Components: Distributed Coordination, Tests
Affects Versions: 1.4.0, 1.3.2
Reporter: Gary Yao
Assignee: Till Rohrmann
Priority: Critical
 Fix For: 1.4.0


Recently, I received timeouts from some tests in {{ExecutionGraphRestartTest}} 
like this

{code}
Tests in error: 
  ExecutionGraphRestartTest.testConcurrentLocalFailAndRestart:638 » Timeout
{code}

This particular instance is from 1.3.2 RC2 and stuck in 
{{ExecutionGraphTestUtils#waitUntilDeployedAndSwitchToRunning()}} but I also 
had instances stuck in {{ExecutionGraphTestUtils#waitUntilJobStatus}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New committer: Haohui Mai

[Kurt Young]

Congrats and welcome on board!

Best,
Kurt

On Wed, Nov 1, 2017 at 8:15 PM, Hai Zhou  wrote:

> Congratulations!
>
> On 1. Nov 2017, at 10:13, Shaoxuan Wang  wrote:
>
> Congratulations!
>
> On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann 
> wrote:
>
> Congrats and welcome on board :-)
>
> On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske  wrote:
>
> Hi everybody,
>
> On behalf of the PMC I am delighted to announce Haohui Mai as a new Flink
> committer!
>
> Haohui has been an active member of our community for several months.
> Among other things, he made major contributions in ideas and code to the
> SQL and Table APIs.
>
> Please join me in congratulating Haohui for becoming a Flink committer!
>
> Cheers,
> Fabian
>
>
>
>
>

[jira] [Created] (FLINK-7959) Split CodeGenerator into CodeGeneratorContext and ExprCodeGenerator

[Kurt Young (JIRA)]

Kurt Young created FLINK-7959:
-

 Summary: Split CodeGenerator into CodeGeneratorContext and 
ExprCodeGenerator
 Key: FLINK-7959
 URL: https://issues.apache.org/jira/browse/FLINK-7959
 Project: Flink
  Issue Type: Improvement
  Components: Table API & SQL
Reporter: Kurt Young
Assignee: Kurt Young
Priority: Major


Right now {{CodeGenerator}} actually acts two roles, one is responsible for 
generating codes from RexNode, and the other one is keeping lots of reusable 
statements. It makes more sense to split these logic into two dedicated class. 

The new {{CodeGeneratorContext}} will keep all the reusable statements, while 
the new {{ExprCodeGenerator}} will only do generating codes from RexNode.

And classes like {{AggregationCodeGenerator}} or {{FunctionCodeGenerator}}, I 
think the should not be subclass of the {{CodeGenerator}}, but all as 
standalone classes. They can create {{ExprCodeGenerator}} when the need to 
generating codes from RexNode, and they can also generating codes by 
themselves. The {{CodeGeneratorContext}} can be passed around to collect all 
reusable statements, and list them in the final generated class.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New committer: Haohui Mai

[Hai Zhou]

Congratulations! 

>> On 1. Nov 2017, at 10:13, Shaoxuan Wang  wrote:
>> 
>> Congratulations!
>> 
>> On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann  wrote:
>> 
>>> Congrats and welcome on board :-)
>>> 
>>> On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske  wrote:
>>> 
 Hi everybody,
 
 On behalf of the PMC I am delighted to announce Haohui Mai as a new Flink
 committer!
 
 Haohui has been an active member of our community for several months.
 Among other things, he made major contributions in ideas and code to the
 SQL and Table APIs.
 
 Please join me in congratulating Haohui for becoming a Flink committer!
 
 Cheers,
 Fabian
 
>>> 
> 

[ANNOUNCE] Apache Flink-shaded 2.0 released

[Chesnay Schepler]

The Apache Flink community is pleased to announce the release of Apache 
Flink-shaded 2.0.


The flink-shaded project contains a number of shaded dependencies for 
Apache Flink.



This release adds jackson-core/databind/annotations:2.7.9 as a single 
flink-shaded-jackson dependency,


and a build profile to build shaded source jars.


For more information about flink-shaded, see 
https://issues.apache.org/jira/browse/FLINK-6529.


The release is available for download at:

   https://dist.apache.org/repos/dist/release/flink/flink-shaded-2.0/
   

[jira] [Created] (FLINK-7956) Add support for scheduling with slot sharing

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7956:


 Summary: Add support for scheduling with slot sharing
 Key: FLINK-7956
 URL: https://issues.apache.org/jira/browse/FLINK-7956
 Project: Flink
  Issue Type: Sub-task
  Components: Scheduler
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Major


In order to reach feature equivalence with the old code base, we should add 
support for scheduling with slot sharing to the {{SlotPool}}. This will also 
allow us to run all the IT cases based on the {{AbstractTestBase}} on the 
Flip-6 {{MiniCluster}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New committer: Haohui Mai

[Aljoscha Krettek]

Congratulations! 

> On 1. Nov 2017, at 10:13, Shaoxuan Wang  wrote:
> 
> Congratulations!
> 
> On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann  wrote:
> 
>> Congrats and welcome on board :-)
>> 
>> On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske  wrote:
>> 
>>> Hi everybody,
>>> 
>>> On behalf of the PMC I am delighted to announce Haohui Mai as a new Flink
>>> committer!
>>> 
>>> Haohui has been an active member of our community for several months.
>>> Among other things, he made major contributions in ideas and code to the
>>> SQL and Table APIs.
>>> 
>>> Please join me in congratulating Haohui for becoming a Flink committer!
>>> 
>>> Cheers,
>>> Fabian
>>> 
>> 

[jira] [Created] (FLINK-7955) Rowtime attribute cannot be aggregated in group window of stream Table API

[Fabian Hueske (JIRA)]

Fabian Hueske created FLINK-7955:


 Summary: Rowtime attribute cannot be aggregated in group window of 
stream Table API
 Key: FLINK-7955
 URL: https://issues.apache.org/jira/browse/FLINK-7955
 Project: Flink
  Issue Type: Bug
  Components: Table API & SQL
Affects Versions: 1.3.2, 1.4.0
Reporter: Fabian Hueske


The following query fails with a {{NullPointerException}}:

{code}
val windowedTable = table
  .window(Tumble over 5.milli on 'rowtime as 'w)
  .groupBy('w, 'word)
  .select('word, 'rowtime.count, 'w.start, 'w.end)
{code}

Equivalent SQL queries or Table API queries that use a processing time 
attribute are working.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New committer: Haohui Mai

[Shaoxuan Wang]

Congratulations!

On Wed, Nov 1, 2017 at 4:36 PM, Till Rohrmann  wrote:

> Congrats and welcome on board :-)
>
> On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske  wrote:
>
> > Hi everybody,
> >
> > On behalf of the PMC I am delighted to announce Haohui Mai as a new Flink
> > committer!
> >
> > Haohui has been an active member of our community for several months.
> > Among other things, he made major contributions in ideas and code to the
> > SQL and Table APIs.
> >
> > Please join me in congratulating Haohui for becoming a Flink committer!
> >
> > Cheers,
> > Fabian
> >
>

Re: [ANNOUNCE] New committer: Haohui Mai

[Till Rohrmann]

Congrats and welcome on board :-)

On Wed, Nov 1, 2017 at 9:14 AM, Fabian Hueske  wrote:

> Hi everybody,
>
> On behalf of the PMC I am delighted to announce Haohui Mai as a new Flink
> committer!
>
> Haohui has been an active member of our community for several months.
> Among other things, he made major contributions in ideas and code to the
> SQL and Table APIs.
>
> Please join me in congratulating Haohui for becoming a Flink committer!
>
> Cheers,
> Fabian
>

[ANNOUNCE] New committer: Haohui Mai

[Fabian Hueske]

Hi everybody,

On behalf of the PMC I am delighted to announce Haohui Mai as a new Flink
committer!

Haohui has been an active member of our community for several months.
Among other things, he made major contributions in ideas and code to the
SQL and Table APIs.

Please join me in congratulating Haohui for becoming a Flink committer!

Cheers,
Fabian