Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

I see, alright. Thank you for all the help! On Mon, Dec 28, 2020 at 3:57 PM Timo Walther wrote: > Hi Yuval, > > FLIP-136 will resolve the remaining legacy type issues for DataStream > API. After that we can drop legacy types because everything can be > expressed with DataType. DataType is a sup

Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

Hi Yuval, FLIP-136 will resolve the remaining legacy type issues for DataStream API. After that we can drop legacy types because everything can be expressed with DataType. DataType is a super type of TypeInformation. Registering types (e.g. giving a RAW type backed by io.circe.Json a name su

Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

Hi Timo, I will look at the Catalog interface and see what it requires. For the meanwhile I'll go back to the old API. Do you expect FLIP-136 to resolve the issue around legacy types? Will it's implementation allow to register LEGACY types? or a new variation of them? On Mon, Dec 28, 2020 at 12:4

Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

I would recommend to use the old UDF stack for now. You can simply call `StreamTableEnvironment.registerFunction` instead of `createTemporarySystemFunction`. Then the UDF returns a legacy type that the DataStream API understands. Have you thought about implementing your own catalog instead of

Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

Timo, an additional question. I am currently using TypeConversions.fromLegacyInfoToDataType. However, this still does not allow me to register this table with the catalog since any representation of LEGACY isn't supported (I don't see it generating any other DataType other than RAW(.., LEGACY). Is

Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

Hi Timo, Thanks for the explanation. Passing around a byte array is not possible since I need to know the concrete type later for serialization in my sink, so I need to keep the type. What I'm trying to achieve is the following: I have a scalar UDF function: [image: image.png] This function is

Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

Hi Yuval, the legacy type has no string representation that can be used in a SQL DDL statement. The current string representation LEGACY(...) is only a temporary work around to persist the old types in catalogs. Until FLIP-136 is fully implemented, toAppendStream/toRetractStream support only

Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

Hi Danny, Yes, I tried implementing the DataTypeFactory for the UDF using TypeInformationRawType (which is deprecated BTW, and there's no support for RawType in the conversion), didn't help. I did manage to get the conversion working using TableEnvironment.toAppendStream (I was previously directl

Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

> SQL parse failed. Encount What syntax did you use ? > TypeConversions.fromDataTypeToLegacyInfo cannot convert a plain RAW type back to TypeInformation. Did you try to construct type information by a new fresh TypeInformationRawType ? Yuval Itzchakov 于2020年12月24日周四 下午7:24写道： > An expansion to

Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

An expansion to my question: What I really want is for the UDF to return `RAW(io.circe.Json, ?)` type, but I have to do a conversion between Table and DataStream, and TypeConversions.fromDataTypeToLegacyInfo cannot convert a plain RAW type back to TypeInformation. On Thu, Dec 24, 2020 at 12:59 PM

Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

Hi, I have a UDF which returns a type of MAP')>. When I try to register this type with Flink via the CREATE TABLE DDL, I encounter an exception: - SQL parse failed. Encountered "(" at line 2, column 256. Was expecting one of: "NOT" ... "NULL" ... ">" ... "MULTISET" ... "ARRAY"