[jira] [Created] (FLINK-6753) Flaky SqlITCase

2017-05-28 Thread Chesnay Schepler (JIRA) 
Chesnay Schepler created FLINK-6753:
---

 Summary: Flaky SqlITCase
 Key: FLINK-6753
 URL: https://issues.apache.org/jira/browse/FLINK-6753
 Project: Flink
  Issue Type: Bug
  Components: Table API & SQL, Tests
Affects Versions: 1.4.0
Reporter: Chesnay Schepler


{code}
Tests run: 11, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6.674 sec <<< 
FAILURE! - in org.apache.flink.table.api.scala.stream.sql.SqlITCase
testUnnestArrayOfArrayFromTable(org.apache.flink.table.api.scala.stream.sql.SqlITCase)
  Time elapsed: 0.289 sec  <<< ERROR!
org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
at 
org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply$mcV$sp(JobManager.scala:933)
at 
org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:876)
at 
org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:876)
at 
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at 
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
at 
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at 
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at 
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at 
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: org.apache.flink.api.common.InvalidProgramException: Table program 
cannot be compiled. This is a bug. Please file an issue.
at 
org.apache.flink.table.codegen.Compiler$class.compile(Compiler.scala:36)
at 
org.apache.flink.table.runtime.CRowCorrelateProcessRunner.compile(CRowCorrelateProcessRunner.scala:35)
at 
org.apache.flink.table.runtime.CRowCorrelateProcessRunner.open(CRowCorrelateProcessRunner.scala:59)
at 
org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
at 
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:111)
at 
org.apache.flink.streaming.api.operators.ProcessOperator.open(ProcessOperator.java:56)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:377)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:254)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: null
at org.codehaus.janino.IClass.isAssignableFrom(IClass.java:652)
at 
org.codehaus.janino.UnitCompiler.isWideningReferenceConvertible(UnitCompiler.java:10844)
at 
org.codehaus.janino.UnitCompiler.isMethodInvocationConvertible(UnitCompiler.java:9010)
at 
org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:8799)
at 
org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:8657)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8539)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8441)
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4609)
at org.codehaus.janino.UnitCompiler.access$8200(UnitCompiler.java:209)
at 
org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(UnitCompiler.java:3969)
at 
org.codehaus.janino.UnitCompiler$12.visitMethodInvocation(UnitCompiler.java:3942)
at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4874)
at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:3942)
at 
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5125)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3343)
at org.codehaus.janino.UnitCompiler.access$5000(UnitCompiler.java:209)
at 
org.codehaus.janino.UnitCompiler$9.visitMethodInvocation(UnitCompiler.java:3322)
at 
org.codehaus.janino.UnitCompiler$9.visitMethodInvocation(UnitCompiler.java:3294)
at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:4874)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3294)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2214)
at org.codehaus.janino.UnitCompiler.access$1700(UnitCompiler.java:209)
at 
org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1445)
at

Re: [VOTE] Release Apache Flink 1.3.0 (RC3)

2017-05-28 Thread Robert Metzger 
@Shaoxuan, I don't think missing documentation is a release blocker, but
it's something we should fix asap and with high priority :)
Since the documentation is not bundled with the release, and the docs are
build off the "release-x.y" branch, we can always update them (even after
the release).

However, it makes sense to have the docs ready when we announce the release
so that users can understand how to use the newly released features.

On Sat, May 27, 2017 at 6:14 PM, Chesnay Schepler 
wrote:

> I've responded in the JIRA. In my opinion isn't a functional issue, but
> more about improving
> error messages/documentation.
>
>
> On 27.05.2017 18:07, Gyula Fóra wrote:
>
>> Hi!
>> I have found this issue: https://issues.apache.org/jira/browse/FLINK-6742
>>
>> Not sure if it's a blocker or not (not even completely sure what causes it
>> at the moment)
>>
>> Gyula
>>
>> Shaoxuan Wang  ezt írta (időpont: 2017. máj. 27.,
>> Szo,
>> 6:30):
>>
>> Hi Robert.
>>> Will doc update be a blocker for release?
>>> Release 1.3 has many updates on tableAPI and SQL, and the docs are kind
>>> of
>>> lagging. We are trying the efforts to update the doc as much as possible
>>> before the release is officially published, but I am hoping this is not a
>>> blocker for the release.
>>>
>>> Shaoxuan
>>>
>>>
>>>
>>> On Sat, May 27, 2017 at 12:58 AM, Robert Metzger 
>>> wrote:
>>>
>>> Hi all,

 this is the second VOTEing release candidate for Flink 1.3.0

 The commit to be voted on:
 760eea8a 
 (*http://git-wip-us.apache.org/repos/asf/flink/commit/760eea8a
 *)

 Branch:
 release-1.3.0-rc3

 The release artifacts to be voted on can be found at:
 http://people.apache.org/~rmetzger/flink-1.3.0-rc3


 The release artifacts are signed with the key with fingerprint D9839159:
 http://www.apache.org/dist/flink/KEYS

 The staging repository for this release can be found at:
 *https://repository.apache.org/content/repositories/orgapacheflink-1122

Re: [DISCUSS] Reorganize Table API / SQL documentation

2017-05-28 Thread Fabian Hueske 
Hi everybody,

I added a couple of subissues to
https://issues.apache.org/jira/browse/FLINK-5354 and started to work on the
first two.
I'll open PRs for those soon.

Please pick an issue and assign it to yourself if you want to help with the
Table API / SQL documentation.

Thanks,
Fabian


2017-05-23 17:49 GMT+02:00 Fabian Hueske :

> Hi everybody,
>
> I pushed the branch to the ASF Flink repository as a feature branch to
> keep all PRs in one place:
>
> -->  https://github.com/apache/flink/tree/tableDocs
>
> Thanks,
> Fabian
>
> 2017-05-23 16:25 GMT+01:00 Fabian Hueske :
>
>> Hi everybody,
>>
>> I prepared a branch that creates the proposed structure and copied the
>> existing documentation into the corresponding pages / sections.
>> There are plenty of gaps that need to be filled or reworked.
>>
>> --> https://github.com/fhueske/flink/tree/tableDocs
>>
>> How do we go on from here?
>> I think the easiest would be if everybody who's interested in working on
>> the documentation picks a page and prepares a PR against my branch (we
>> could also push this into a feature branch in the Flink repository if
>> somebody prefers that). The PRs are cross-checked and we merge everything
>> into to the master when the docs are ready.
>>
>> Any opinions or other proposals?
>>
>> Cheers, Fabian
>>
>> 2017-05-23 10:31 GMT+01:00 Fabian Hueske :
>>
>>> Hi everybody,
>>>
>>> Thanks for the feedback. I'll go ahead and create the proposed structure
>>> and move the content of the existing docs with comments of what needs to be
>>> adapted.
>>> I'll put this into branch of my Github repo and let you know when I'm
>>> done.
>>> From there, we can distribute working on the missing parts / parts that
>>> need adaption.
>>>
>>> Cheers, Fabian
>>>
>>> 2017-05-19 9:44 GMT+01:00 jincheng sun :
>>>
 Hi, Fabian,

   Thanks for the sketch. The structure is pretty well to me, And glad to
 join in the discussion in google doc.

 Cheers,
 SunJincheng

 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang :

 > Hello Fabian,
 > Thanks for drafting the proposal. I like the entire organization in
 general
 > and left a few comments. I think this will be a very good kick off to
 > reorganize the tableAPI doc.
 >
 > -shaoxuan
 >
 > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske 
 wrote:
 >
 > > Hi everybody,
 > >
 > > I came up with a proposal for the structure of the Table API / SQL
 > > documentation:
 > >
 > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
 > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
 > >
 > > Feedback and comments are very welcome.
 > > Once we agree on a structure, we can create skeletons and
 distribute the
 > > work.
 > >
 > > Cheers,
 > > Fabian
 > >
 > > 2017-05-18 21:01 GMT+02:00 Haohui Mai :
 > >
 > > > +1
 > > >
 > > > The Table / SQL component has made significant progress in the
 last few
 > > > months (kudos to all contributors).
 > > >
 > > > It is a good time to have a documentation to reflect all the
 changes in
 > > the
 > > > Table / SQL side.
 > > >
 > > >
 > > >
 > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <
 rmetz...@apache.org>
 > > > wrote:
 > > >
 > > > > Thank you Fabian for working on the proposal.
 > > > >
 > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <
 fhue...@gmail.com>
 > > > wrote:
 > > > >
 > > > > > Thanks for starting this discussion Robert.
 > > > > >
 > > > > > I think with the next release the Table API / SQL should be
 moved
 > up
 > > in
 > > > > the
 > > > > > Application Development menu.
 > > > > > I also though about restructuring the docs, but it won't be
 trivial
 > > to
 > > > do
 > > > > > this, IMO because there are many orthogonal aspects:
 > > > > > - Stream/Batch
 > > > > > - Table/SQL
 > > > > > - Scala/Java
 > > > > >
 > > > > > and sometimes also common concepts.
 > > > > > At the moment there are also many new features missing like
 OVER
 > > > windows,
 > > > > > UDAGGs, retraction, StreamTableSinks, time indicator
 attributes,
 > > filter
 > > > > > pushdown, ...
 > > > > >
 > > > > > I will try to sketch a new structure in a Google Doc in the
 next
 > days
 > > > and
 > > > > > share it in this thread.
 > > > > >
 > > > > > Cheers, Fabian
 > > > > >
 > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
 > > k.klou...@data-artisans.com
 > > > >:
 > > > > >
 > > > > > > A big +1 as well.
 > > > > > >
 > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi 
 > wrote:
 > > > > > >

[jira] [Created] (FLINK-6751) Table API / SQL Docs: UDFs Page

2017-05-28 Thread Fabian Hueske (JIRA) 
Fabian Hueske created FLINK-6751:


 Summary: Table API / SQL Docs: UDFs Page
 Key: FLINK-6751
 URL: https://issues.apache.org/jira/browse/FLINK-6751
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation, Table API & SQL
Affects Versions: 1.3.0
Reporter: Fabian Hueske


Update and refine {{./docs/dev/table/udfs.md}} in feature branch 
https://github.com/apache/flink/tree/tableDocs



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (FLINK-6750) Table API / SQL Docs: Table Sources & Sinks Page

2017-05-28 Thread Fabian Hueske (JIRA) 
Fabian Hueske created FLINK-6750:


 Summary: Table API / SQL Docs: Table Sources & Sinks Page
 Key: FLINK-6750
 URL: https://issues.apache.org/jira/browse/FLINK-6750
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation, Table API & SQL
Affects Versions: 1.3.0
Reporter: Fabian Hueske


Update and refine {{./docs/dev/table/sourceSinks.md}} in feature branch 
https://github.com/apache/flink/tree/tableDocs



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (FLINK-6747) Table API / SQL Docs: Streaming Page

2017-05-28 Thread Fabian Hueske (JIRA) 
Fabian Hueske created FLINK-6747:


 Summary: Table API / SQL Docs: Streaming Page
 Key: FLINK-6747
 URL: https://issues.apache.org/jira/browse/FLINK-6747
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation, Table API & SQL
Affects Versions: 1.3.0
Reporter: Fabian Hueske


Update and refine {{./docs/dev/table/streaming.md}} in feature branch 
https://github.com/apache/flink/tree/tableDocs



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (FLINK-6749) Table API / SQL Docs: SQL Page

2017-05-28 Thread Fabian Hueske (JIRA) 
Fabian Hueske created FLINK-6749:


 Summary: Table API / SQL Docs: SQL Page
 Key: FLINK-6749
 URL: https://issues.apache.org/jira/browse/FLINK-6749
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation, Table API & SQL
Affects Versions: 1.3.0
Reporter: Fabian Hueske


Update and refine {{./docs/dev/table/sql.md}} in feature branch 
https://github.com/apache/flink/tree/tableDocs



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (FLINK-6748) Table API / SQL Docs: Table API Page

2017-05-28 Thread Fabian Hueske (JIRA) 
Fabian Hueske created FLINK-6748:


 Summary: Table API / SQL Docs: Table API Page
 Key: FLINK-6748
 URL: https://issues.apache.org/jira/browse/FLINK-6748
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation, Table API & SQL
Affects Versions: 1.3.0
Reporter: Fabian Hueske


Update and refine {{./docs/dev/table/tableApi.md}} in feature branch 
https://github.com/apache/flink/tree/tableDocs



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (FLINK-6745) Table API / SQL Docs: Overview Page

2017-05-28 Thread Fabian Hueske (JIRA) 
Fabian Hueske created FLINK-6745:


 Summary: Table API / SQL Docs: Overview Page
 Key: FLINK-6745
 URL: https://issues.apache.org/jira/browse/FLINK-6745
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation, Table API & SQL
Affects Versions: 1.3.0
Reporter: Fabian Hueske
Assignee: Fabian Hueske


Update and refine ./docs/dev/tableApi.md in feature branch 
https://github.com/apache/flink/tree/tableDocs



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (FLINK-6746) Table API / SQL Docs: Common Page

2017-05-28 Thread Fabian Hueske (JIRA) 
Fabian Hueske created FLINK-6746:


 Summary: Table API / SQL Docs: Common Page
 Key: FLINK-6746
 URL: https://issues.apache.org/jira/browse/FLINK-6746
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation, Table API & SQL
Affects Versions: 1.3.0
Reporter: Fabian Hueske
Assignee: Fabian Hueske


Update and refine ./docs/dev/table/common.md in feature branch 
https://github.com/apache/flink/tree/tableDocs



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (FLINK-6744) Flaky ExecutionGraphSchedulingTest

2017-05-28 Thread Chesnay Schepler (JIRA) 
Chesnay Schepler created FLINK-6744:
---

 Summary: Flaky ExecutionGraphSchedulingTest
 Key: FLINK-6744
 URL: https://issues.apache.org/jira/browse/FLINK-6744
 Project: Flink
  Issue Type: Bug
  Components: Tests
Affects Versions: 1.4.0
Reporter: Chesnay Schepler


{code}

Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.013 sec <<< 
FAILURE! - in 
org.apache.flink.runtime.executiongraph.ExecutionGraphSchedulingTest

testDeployPipelinedConnectedComponentsTogether(org.apache.flink.runtime.executiongraph.ExecutionGraphSchedulingTest)
  Time elapsed: 0.121 sec  <<< FAILURE!

org.mockito.exceptions.verification.WantedButNotInvoked: 

Wanted but not invoked:

taskManagerGateway.submitTask(, );

-> at 
org.apache.flink.runtime.executiongraph.ExecutionGraphSchedulingTest.testDeployPipelinedConnectedComponentsTogether(ExecutionGraphSchedulingTest.java:246)

Actually, there were zero interactions with this mock.

at 
org.apache.flink.runtime.executiongraph.ExecutionGraphSchedulingTest.testDeployPipelinedConnectedComponentsTogether(ExecutionGraphSchedulingTest.java:246)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (FLINK-6743) .window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8))) doesn't work

2017-05-28 Thread Lix (JIRA) 
Lix created FLINK-6743:
--

 Summary: .window(TumblingEventTimeWindows.of(Time.days(1), 
Time.hours(-8))) doesn't work
 Key: FLINK-6743
 URL: https://issues.apache.org/jira/browse/FLINK-6743
 Project: Flink
  Issue Type: Bug
  Components: DataStream API
 Environment: Flink 1.2.0 and above.
Reporter: Lix


The tutorial on the official website says that we can use

```
// daily tumbling event-time windows offset by -8 hours.
input
.keyBy()
.window(TumblingEventTimeWindows.of(Time.days(1), Time.hours(-8)))
.();
```

when our timezone is UTC+8, which is in China. But when I tried to run this 
code, it just reported an error:

```
Exception in thread "main" java.lang.IllegalArgumentException: 
TumblingProcessingTimeWindows parameters must satisfy  0 <= offset < size
at 
org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows.(TumblingProcessingTimeWindows.java:54)
at 
org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows.of(TumblingProcessingTimeWindows.java:111)
```

How then, should I write my code to make a tumbling window which clears every 
day at 00:00, when my timezone is UTC+8?




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)