Hi Andrey, I noticed that 1.10 has changed to enabling background cleanup by default just after I posted to this email. So it won't affect 1.10 any more, just affect 1.9.x. We can move to the Jira ticket to discuss further more.
Andrey Zagrebin <azagre...@apache.org> 于2020年2月6日周四 下午11:30写道： > Hi Benchao, > > Do you observe this issue FLINK-15938 with 1.9 or 1.10? > If with 1.9, I suggest to check with 1.10. > > Thanks, > Andrey > > On Thu, Feb 6, 2020 at 4:07 PM Benchao Li <libenc...@gmail.com> wrote: > > > Hi all, > > > > I found another issue[1], I don't know if it should be a blocker. But it > > does affects joins without window in blink planner. > > > > [1] https://issues.apache.org/jira/browse/FLINK-15938 > > > > Jeff Zhang <zjf...@gmail.com> 于2020年2月6日周四 下午5:05写道： > > > > > Hi Jingsong, > > > > > > Thanks for the suggestion. It works for running it in IDE, but for > > > downstream project like Zeppelin where I will include flink jars in > > > classpath. > > > it only works when I specify the jars one by one explicitly in > classpath, > > > using * doesn't work. > > > > > > e.g. > > > > > > The following command where I use * to specify classpath doesn't work, > > > jzhang 4794 1.2 6.3 8408904 1048828 s007 S 4:30PM 0:33.90 > > > > /Library/Java/JavaVirtualMachines/jdk1.8.0_211.jdk/Contents/Home/bin/java > > > -Dfile.encoding=UTF-8 > > > > > > > > > -Dlog4j.configuration=file:///Users/jzhang/github/zeppelin/conf/log4j.properties > > > > > > > > > -Dzeppelin.log.file=/Users/jzhang/github/zeppelin/logs/zeppelin-interpreter-flink-shared_process-jzhang-JeffdeMacBook-Pro.local.log > > > -Xms1024m -Xmx2048m -XX:MaxPermSize=512m -cp > > > > > > > > > *:/Users/jzhang/github/flink/build-target/lib/**:/Users/jzhang/github/zeppelin/interpreter/flink/*:/Users/jzhang/github/zeppelin/zeppelin-interpreter-api/target/*:/Users/jzhang/github/zeppelin/zeppelin-zengine/target/test-classes/*::/Users/jzhang/github/zeppelin/interpreter/zeppelin-interpreter-api-0.9.0-SNAPSHOT.jar:/Users/jzhang/github/zeppelin/zeppelin-interpreter/target/classes:/Users/jzhang/github/zeppelin/zeppelin-zengine/target/test-classes:/Users/jzhang/github/flink/build-target/opt/flink-python_2.11-1.10-SNAPSHOT.jar > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 0.0.0.0 > > > 52395 flink-shared_process : > > > > > > > > > While this command where I specify jar one by one explicitly in > classpath > > > works > > > > > > jzhang 5660 205.2 6.1 8399420 1015036 s005 R 4:43PM > > > 0:24.82 > > > > /Library/Java/JavaVirtualMachines/jdk1.8.0_211.jdk/Contents/Home/bin/java > > > -Dfile.encoding=UTF-8 > > > > > > > > > -Dlog4j.configuration=file:///Users/jzhang/github/zeppelin/conf/log4j.properties > > > > > > > > > -Dzeppelin.log.file=/Users/jzhang/github/zeppelin/logs/zeppelin-interpreter-flink-shared_process-jzhang-JeffdeMacBook-Pro.local.log > > > -Xms1024m -Xmx2048m -XX:MaxPermSize=512m -cp > > > > > > > > > *:/Users/jzhang/github/flink/build-target/lib/slf4j-log4j12-1.7.15.jar:/Users/jzhang/github/flink/build-target/lib/flink-connector-hive_2.11-1.10-SNAPSHOT.jar:/Users/jzhang/github/flink/build-target/lib/hive-exec-2.3.4.jar:/Users/jzhang/github/flink/build-target/lib/flink-shaded-hadoop-2-uber-2.7.5-7.0.jar:/Users/jzhang/github/flink/build-target/lib/log4j-1.2.17.jar:/Users/jzhang/github/flink/build-target/lib/flink-table-blink_2.11-1.10-SNAPSHOT.jar:/Users/jzhang/github/flink/build-target/lib/flink-table_2.11-1.10-SNAPSHOT.jar:/Users/jzhang/github/flink/build-target/lib/flink-dist_2.11-1.10-SNAPSHOT.jar:/Users/jzhang/github/flink/build-target/lib/flink-python_2.11-1.10-SNAPSHOT.jar:/Users/jzhang/github/flink/build-target/lib/flink-hadoop-compatibility_2.11-1.9.1.jar*:/Users/jzhang/github/zeppelin/interpreter/flink/*:/Users/jzhang/github/zeppelin/zeppelin-interpreter-api/target/*:/Users/jzhang/github/zeppelin/zeppelin-zengine/target/test-classes/*::/Users/jzhang/github/zeppelin/interpreter/zeppelin-interpreter-api-0.9.0-SNAPSHOT.jar:/Users/jzhang/github/zeppelin/zeppelin-interpreter/target/classes:/Users/jzhang/github/zeppelin/zeppelin-zengine/target/test-classes:/Users/jzhang/github/flink/build-target/opt/flink-python_2.11-1.10-SNAPSHOT.jar > > > > > > > > > /Users/jzhang/github/flink/build-target/opt/tmp/flink-python_2.11-1.10-SNAPSHOT.jar > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 0.0.0.0 > > > 52603 flink-shared_process : > > > > > > > > > Jingsong Li <jingsongl...@gmail.com> 于2020年2月6日周四 下午4:10写道： > > > > > > > Hi Jeff, > > > > > > > > > > > > For FLINK-15935 [1], > > > > > > > > I try to think of it as a non blocker. But it's really an important > > > issue. > > > > > > > > > > > > The problem is the class loading order. We want to load the class in > > the > > > > blink-planner.jar, but actually load the class in the > > flink-planner.jar. > > > > > > > > > > > > First of all, the order of class loading is based on the order of > > > > classpath. > > > > > > > > > > > > I just tried, the order of classpath of the folder is depends on the > > > order > > > > of file names. > > > > > > > > -That is to say, our order is OK now: because > > > > flink-table-blink_2.11-1.11-SNAPSHOT.jar is before the name order of > > > > flink-table_2.11-1.11-snapshot.jar. > > > > > > > > -But once I change the name of flink-table_2.11-1.11-SNAPSHOT.jar to > > > > aflink-table_2.11-1.11-SNAPSHOT.jar, an error will be reported. > > > > > > > > > > > > The order of classpaths should be influenced by the ls of Linux. By > > > default > > > > the ls command is listing the files in alphabetical order. [1] > > > > > > > > > > > > IDE seems to have another rule, because it is directly load classes. > > Not > > > > jars. > > > > > > > > > > > > So the possible impact of this problem is: > > > > > > > > -IDE with two planners to use watermark, which will report an error. > > > > > > > > -The real production environment, under Linux, will not report errors > > > > because of the above reasons. > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-15935 > > > > > > > > [2] > > > > > > > > > > https://linuxize.com/post/how-to-list-files-in-linux-using-the-ls-command/ > > > > > > > > > > > > Best, > > > > > > > > Jingsong Lee > > > > > > > > On Thu, Feb 6, 2020 at 3:09 PM Jeff Zhang <zjf...@gmail.com> wrote: > > > > > > > > > -1, I just found one critical issue > > > > > https://issues.apache.org/jira/browse/FLINK-15935 > > > > > This ticket means user unable to use watermark in sql if he specify > > > both > > > > > flink planner and blink planner in pom.xml > > > > > > > > > > <dependency> > > > > > <groupId>org.apache.flink</groupId> > > > > > > > <artifactId>flink-table-planner_${scala.binary.version}</artifactId> > > > > > <version>${project.version}</version> > > > > > </dependency> > > > > > <dependency> > > > > > <groupId>org.apache.flink</groupId> > > > > > > > > > > > > > > > > > > > > <artifactId>flink-table-planner-blink_${scala.binary.version}</artifactId> > > > > > <version>${project.version}</version> > > > > > </dependency> > > > > > > > > > > > > > > > > > > > > Thomas Weise <t...@apache.org> 于2020年2月6日周四 上午5:16写道： > > > > > > > > > > > I deployed commit 81cf2f9e59259389a6549b07dcf822ec63c899a4 and > can > > > > > confirm > > > > > > that the dataformat-cbor and checkpoint alignment metric issues > are > > > > > > resolved. > > > > > > > > > > > > > > > > > > On Wed, Feb 5, 2020 at 11:26 AM Gary Yao <g...@apache.org> > wrote: > > > > > > > > > > > > > Note that there is currently an ongoing discussion about > whether > > > > > > > FLINK-15917 > > > > > > > and FLINK-15918 should be fixed in 1.10.0 [1]. > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/flink-dev/202002.mbox/%3CCA%2B5xAo3D21-T5QysQg3XOdm%3DL9ipz3rMkA%3DqMzxraJRgfuyg2A%40mail.gmail.com%3E > > > > > > > > > > > > > > On Wed, Feb 5, 2020 at 8:00 PM Gary Yao <g...@apache.org> > wrote: > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > Please review and vote on the release candidate #2 for the > > > version > > > > > > > 1.10.0, > > > > > > > > as follows: > > > > > > > > [ ] +1, Approve the release > > > > > > > > [ ] -1, Do not approve the release (please provide specific > > > > comments) > > > > > > > > > > > > > > > > > > > > > > > > The complete staging area is available for your review, which > > > > > includes: > > > > > > > > * JIRA release notes [1], > > > > > > > > * the official Apache source release and binary convenience > > > > releases > > > > > to > > > > > > > be > > > > > > > > deployed to dist.apache.org [2], which are signed with the > key > > > > with > > > > > > > > fingerprint BB137807CEFBE7DD2616556710B12A1F89C115E8 [3], > > > > > > > > * all artifacts to be deployed to the Maven Central > Repository > > > [4], > > > > > > > > * source code tag "release-1.10.0-rc2" [5], > > > > > > > > * website pull request listing the new release and adding > > > > > announcement > > > > > > > > blog post [6][7]. > > > > > > > > > > > > > > > > The vote will be open for at least 24 hours. It is adopted by > > > > > majority > > > > > > > > approval, with at least 3 PMC affirmative votes. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Yu & Gary > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12345845 > > > > > > > > [2] > > > https://dist.apache.org/repos/dist/dev/flink/flink-1.10.0-rc2/ > > > > > > > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS > > > > > > > > [4] > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapacheflink-1332 > > > > > > > > [5] > > > > https://github.com/apache/flink/releases/tag/release-1.10.0-rc2 > > > > > > > > [6] https://github.com/apache/flink-web/pull/302 > > > > > > > > [7] https://github.com/apache/flink-web/pull/301 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best Regards > > > > > > > > > > Jeff Zhang > > > > > > > > > > > > > > > > > -- > > > > Best, Jingsong Lee > > > > > > > > > > > > > -- > > > Best Regards > > > > > > Jeff Zhang > > > > > > > > > -- > > > > Benchao Li > > School of Electronics Engineering and Computer Science, Peking University > > Tel:+86-15650713730 > > Email: libenc...@gmail.com; libenc...@pku.edu.cn > > > -- Benchao Li School of Electronics Engineering and Computer Science, Peking University Tel:+86-15650713730 Email: libenc...@gmail.com; libenc...@pku.edu.cn
