Thanks, Yu sharing more background on this.
Jark,
We were able to sync with Yu a bit offline. I think we should reuse Jira
and the future on how to reuse code when we get into the implementation
phase.
and continue the discussion maybe share a google doc detail list of work
and options so folks c
Thanks for the discussion. In https://github.com/apache/flink/pull/8067 we
made an initial version on adding thrift-format support in flink, and
haven't got time to finish it. Feel free to take it over and make changes.
I've also linked this discussion thread in
https://issues.apache.org/jira/brows
Thanks Dawid for the link. I have a glance at the PR.
I think we can continue the thrift format based on the PR (would be better
to reach out to the author).
Best,
Jark
On Tue, 21 Jul 2020 at 15:58, Dawid Wysakowicz
wrote:
> Hi,
>
> I've just spotted this PR that might be helpful in the discus
Hi,
I've just spotted this PR that might be helpful in the discussion:
https://github.com/apache/flink/pull/8067
Best,
Dawid
On 20/07/2020 04:30, Benchao Li wrote:
> Hi Chen,
>
> Thanks for bringing up this discussion. We are doing something similar
> internally recently.
>
> Our use case is th
Hi Chen,
Your listed items sound great to me. I think we can start from the thrift
format, could you open an issue for it?
The community also planned to support PB format in the next version, maybe
can work together.
Deriving table schema out of thrift struct is also an interesting topic,
and is
Hi Chen,
- adding support in flink-format (e.g flink-thrift)
Sure. We should have a flink-thrift format to do the (de)ser work.
- evaluate if TBaseSeralizaer (Kryo) need extra work
I don't known if I understand it correctly, I think we don't need to
transfer thrift data inside Flink, we just
Jeff
A sample would be you have a Kafka topic stores record in thrift format,
- Flink SQL will not work because it doesn't support thrift format out of
the box,
- table schema can't be inferred so the user might end up handcrafting
field by field mapping
- thrift object serialization fall back to
Hi Chen,
Thanks for bringing up this discussion. We are doing something similar
internally recently.
Our use case is that many services in our company are built with
thrift protocol, and we
want to support accessing these RPC services natively with Flink SQL.
Currently, there are two ways that we
Hi Chen,
Right, this is what I mean. Could you provide more details about the
desr/ser work ? Giving a concrete example or usage scenario would be
helpful.
Chen Qin 于2020年7月18日周六 下午11:09写道：
> Jeff,
>
> Are you referring something like this SPIP?
>
> https://docs.google.com/document/d/1ug4K5e2
Jeff,
Are you referring something like this SPIP?
https://docs.google.com/document/d/1ug4K5e2okF5Q2Pzi3qJiUILwwqkn0fVQaQ-Q95HEcJQ/edit#heading=h.x97c6tj78zo0
Not at this moment, we are working on desr/ser work at the moment. Would be
good to starts discussion and learn if folks working on related
Hi Chen,
Are building something like hive thrift server ?
Chen Qin 于2020年7月18日周六 上午8:50写道：
> Hi there,
>
> Here in Pinterest, we utilize thrift end to end in our tech stack. As we
> have been building Flink as a service platform, the team spent time working
> on supporting Flink jobs with thrif
11 matches
Mail list logo
