Hi yang
Thank you for your comment.
I read document and got an idea.
Flink uses custom serializers on known types, and the fallback is kryo.
The other arbitary objects is serialized by kryo.
On 2018/07/12 12:14, vino yang wrote:
Hi Yuta,
It seems Chesnay is right. The "fallback" in flink's documentation is in
terms of the types flink supported. But for all the other arbitrary
types kryo is the first choice.
2018-07-12 9:55 GMT+08:00 Yuta Morisawa <yu-moris...@kddi-research.jp
<mailto:yu-moris...@kddi-research.jp>>:
Thank you for your answer.
> For POJOs Flink has a custom serializer. For arbitrary objects we use
> kryo, and can use Avro as a fallback.
https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html#serialization-of-pojo-types
<https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html#serialization-of-pojo-types>
It may be the reverse.
Kryo is for fallback, right?
On 2018/07/11 19:00, Chesnay Schepler wrote:
1) TypeInformation are used to create serializers, comparators
and to verify correctness of certain operations (like
projections on tuple datasets).
2) see
https://flink.apache.org/news/2015/05/11/Juggling-with-Bits-and-Bytes.html
<https://flink.apache.org/news/2015/05/11/Juggling-with-Bits-and-Bytes.html>
3) Flink comes with a number of serializers for varying types as
outlined here
<https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html#flinks-typeinformation-class
<https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html#flinks-typeinformation-class>>.
For POJOs Flink has a custom serializer. For arbitrary objects
we use kryo, and can use Avro as a fallback.
On 11.07.2018 09:24, Yuta Morisawa wrote:
Hi all
Now, I'm reading Flink document and I have some points to
feel difficult to get an idea.
I'd appreciate if you tell it me.
1,TypeInformation
I understand TypeInformation is used for selecting
relevant serializer and comparator.
But, the ducument doesn't specify if it has another way to
be used.
So, what I want to know is that what kinds of process gets
benefit from TypeInformation other than serializer and
comparator.
2, Managed Memory
The word "Managed memory" is appeared several time in the
document but I can't find any detail description.
This is the only document I found
(https://www.slideshare.net/sbaltagi/overview-of-apacheflinkbyslimbaltagi
<https://www.slideshare.net/sbaltagi/overview-of-apacheflinkbyslimbaltagi>)
If anyone has document that explains managed memory,
please let me know.
3, Serializer
What do the words in the document "serializers we ship
with Flink" mean? I know Flink uses avro for POJOs, is it
the same thing?
https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html>
Regards,
Yuta
