Hi Justin,
thank you for reporting your issues. I never tried the Table API with
SBT but `flink-table` should not declare dependencies to core modules,
this is only done in `test` scope, maybe you have to specify the right
scope manually? You are right, the mentioned Jira should be fixed asap,
I added it to my personal TODO list. Regarding the missing Janino
artifacts in the Jar file, I created a Jira issue
(https://issues.apache.org/jira/browse/FLINK-5994). This is very strange
as it actually a dependency of flink-table.
Thanks again for the feedback. If you experience any further issues with
the Table API feel free to post them here.
Regards,
Timo
Am 08/03/17 um 04:50 schrieb Justin Yan:
Of course, 15 minutes after I give up and decide to email the mailing
list, I figure it out - my flink App was using the
CollectionsEnvironment and not the proper RemoteEnvironment.
It is still the case, however, that the `flink-table` JAR built by the
standard commands doesn't include the dependencies it requires, and so
I'd be curious to hear what the proper procedure is for linking
against `flink-table` if you want to avoid the bug I highlighted in
the aforementioned JIRA.
Thank you and sorry for the extra noise!
Justin
On Tue, Mar 7, 2017 at 7:21 PM, Justin Yan <[email protected]
<mailto:[email protected]>> wrote:
Hello!
We are attempting to use the Flink Table API, but are running into
a few issues.
We initially started with our dependencies looking something like
this:
libraryDependencies ++=Seq(
"org.apache.flink" %%"flink-scala" %"1.2.0" %"provided", "org.apache.flink" %%"flink-clients" %"1.2.0"
%"provided", "org.apache.flink" %%"flink-table" %"1.2.0", Libraries.specs2, ...)
However, this is mildly annoying since flink-table declares
dependencies on the flink core modules, and thus brings everything
in /anyway/. On top of that, we saw this JIRA:
https://issues.apache.org/jira/browse/FLINK-5227
<https://issues.apache.org/jira/browse/FLINK-5227>, which we found
concerning, so we decided to follow the advice - we downloaded and
built Flink-1.2 from source (java 7, mvn 3.3) using the following
(We're also using the Kinesis connector):
tools/change-scala-version.sh 2.11 mvn clean install
-Pinclude-kinesis -DskipTests cd flink-dist mvn clean install
-Pinclude-kinesis -DskipTests
Once this was done, we took the JAR in
"/flink-libraries/flink-table/target/" and copied it over to the
taskManager "/lib" directory. Finally, we marked our `flink-table`
dependency as "provided". Everything compiles, but when I try to
actually run a simple job, I get the following error at runtime:
java.lang.NoClassDefFoundError:
org/codehaus/commons/compiler/CompileException
Indeed, when I peek inside of the `flink-table` JAR, I can't find
that particular package (and similarly, it isn't in the flink-dist
JAR either)
$ jar tf flink-table_2.11-1.2.0.jar | grep codehaus
$
I then attempted to include this library in my user code by adding:
"org.codehaus.janino" %"janino" %"3.0.6",
to my list of dependencies. When I run a `jar tf myjar.jar | grep
CompileException` - I see the class. However, when I run my flink
application in this fat JAR, I /continue to get the same error/,
even though I am positive this class is included in the fat JAR.
I eventually got around this by placing this jar in the
`flink/lib` directory, but I am very confused as to how this class
cannot be found when I have included it in the fat JAR that I am
submitting with the Flink CLI to a YARN cluster. I mostly wanted
to mention this in case it is a bug, but mostly to see if anyone
else has had trouble with the Table API, and if not, if I have
structured my project incorrectly to cause these troubles.
Thanks!
Justin
