Hello,
after having a version mismatch between Avro in Flink 1.3.2 I decided
to see how things work with Flink 1.4.0.
The pipeline I am building runs now, deployed as standalone on YARN
with Flink 1.3.2 and putting it "FIRST" on the classpath (to use Avro
1.8.2 instead of an 1.7.x version).
The default setting for yarn.per-job-cluster.include-user-jar is
ORDERED which in my case was a bit tricky, because I first deployed a
pipeline having a jar starting with "A" so it was placed before the
flink jars, later I worked on a mostly identical pipeline with a name
a bit further down in the alphabet, and that was placed behind the
flink jars causing various issues. I would recommend to change this as
default to LAST. Then the jar name has no impact on how things are
loaded.
Should I open a jira for this?
Now after updating to Flink 1.4.0 and also running on YARN as
standalone I am getting null values in a map field I did set in
earlier steps in the pipeline on Avro objects. The thing is a bit
weird, because avroObject.toString would show the values just fine,
but I get null on retrieval.
I have my own TableInputFormat to read from HBase and output a Tuple2
containing an Avro object.
Next comes a map which tries to access it.
The failing piece looks like this:
ao.getTheMap().get("theKey") (returns null)
Doing an ao.toString would show the "theMap" and the value of "theKey".
Is this a bug in Flink 1.4.0 or is there something wrong with my dependencies?
Thanks,
Jörn
