Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

gt;>>>>>> >>>>>>> As for the ConnectTableDescriptor I agree this is very similar to >>>>>>> CatalogTable. I am not sure though if we should get rid of it. In the >>>>>>> end I see it as a builder for a Cat

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

. Please take a look. I would like to > >>>>> start a vote on this FLIP as soon as the vote on FLIP-57 goes > through. > >>>>> > >>>>> Best, > >>>>> > >>>>> Dawid > >>>>> > >>>

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

dNames Maybe it's not a good idea to add these interfaces to TableSource/TableSink. What do you think? Best, Jingsong Lee -- From:Kurt Young Send Time:2019年9月18日(星期三) 17:54 To:dev Subject:Re: [DISCUSS] FLIP-64: Support for Te

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

deal with > >>> ConnectTableDescriptor? > >>> It's actually very similar with serializable CatalogTable, both carry > >>> > >>> some > >>> > >>> text > >>> properties which even are the same. Is there any chance

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

SQL world. I am OK to have a deprecated registerTemporarySource/Sink to compatible with old ways. Best, Jingsong Lee -- From:Timo Walther Send Time:2019年9月17日(星期二) 08:00 To:dev Subject:Re: [DISCUSS] FLIP-64: Support for T

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

s > > > > on TableSource/TableSink: getPartitions, getPartitionFieldNames > > Maybe it's not a good idea to add these interfaces to > > TableSource/TableSink. What do you think? > > > > Best, > > Jingsong Lee > > > > > > -- > > From:Kurt Young

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

gt; > have > > a > > separate methods for that. In the end the temporary tables are held > > in > > memory as CatalogTables. > Best, > Dawid > > On Thu, 19 Sep 2019, 10:08 JingsongLee, > .invalid> > > wrote: > > > Hi dawid: > Can temporary tables achieve the same capabilities as catalog > > table? > >

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

;> Thanks Dawid for the design doc. >>>>>>>> >>>>>>>> In general, I’m +1 to the FLIP. >>>>>>>> >>>>>>>> >>>>>>>> +1 to the single-string and parse way to express object path. >>>>>>>> >>>>>&

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

Sink & registerTableSource. > >>>>>> But I would suggest to provide an easy way to register a custom > >>>>>> source/sink before we drop them (this is another story). > >>>>>> Currently, it’s not easy to implement a custom connector descriptor. > >>>>>> > >>>&

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

9年9月18日(星期三) 17:54 To:dev Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module Hi all, Sorry to join this party late. Big +1 to this flip, especially for the dropping "registerTableSink & registerTableSource" part. These are indeed legacy and we sho

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

an. Underneath they will be > CatalogTables. > >>>> The > >>>>> difference is the lifetime of the tables. Plus some of the user > facing > >>>>> interfaces cannot be persisted e.g. datastream. Therefore we must > have > >> a > >>>>> separate meth

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

ngsong Lee -- From:Kurt Young Send Time:2019年9月18日(星期三) 17:54 To:dev Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module Hi all, Sorry to join this party late. Big +1 to this flip, especially for the dropping "registerTableSink & registerTableSource" part.

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

gt; and we should try to unify them through CatalogTable after we > introduce > >>>> the concept of Catalog. > >>>> > >>>> From my understanding, what we can registered should all be metadata, > >>>> TableSource/TableSink shoul

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

either use connect() or DDL API which can be validated and stored in >>>>> catalog. >>>>> >>>>> Also from a concept perspective, registering a source/sink does not fit >>>>> into the SQL world. SQL does not know about source/sinks but only about >>>>> tables. If the responsibility of a TableS

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

nt interfaces > >> on TableSource/TableSink: getPartitions, getPartitionFieldNames > >> Maybe it's not a good idea to add these interfaces to > >> TableSource/TableSink. What do you think? > >> > >> Best, > >> Jingsong Lee > >> &g

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

tring parsing in the code, which seems to an anti-pattern to OOP. >>> Have >>>> we considered using ObjectIdenitifier with optional catalog and db >> parts, >>>> esp. if we are worried about arguments of variable length or method >>>> overloadin

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

k? > > Best, > Jingsong Lee > > > -- > From:Kurt Young > Send Time:2019年9月18日(星期三) 17:54 > To:dev > Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table > module > > Hi all, > > Sorry to join this party late. Big +1 to this flip, especi

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

. > > Best, > Jingsong Lee > > > -- > From:Timo Walther > Send Time:2019年9月17日(星期二) 08:00 > To:dev > Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table > module > > Hi Dawid, > > thanks for the

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

ays. > > Best, > Jingsong Lee > > > -- > From:Timo Walther > Send Time:2019年9月17日(星期二) 08:00 > To:dev > Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table > module > > Hi Dawid, > > thanks for the design document.

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

-- From:Timo Walther Send Time:2019年9月17日(星期二) 08:00 To:dev Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module Hi Dawid, thanks for the design document. It fixes big concept gaps due to historical reasons with proper support for serializability and catalog

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

-- From:Dawid Wysakowicz Send Time:2019年9月6日(星期五) 16:21 To:dev Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module Hi all, @Jingsong Could you elaborate a bit more what do you mean by "some Connectors are difficult to convert all states to properties

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

Thanks david for pushing this forward. I have one concern about temporary objects and non-persistent catalog(e.g., GenericInMemoryCatalog). In SQL, temporary objects exist at the session level. They are only visible to the session in which they were created and are automatically dropped when that

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

can't use TableSource-related features, like project & filter push down, partition support, etc.. Any idea about this? Best, Jingsong Lee -- From:Dawid Wysakowicz Send Time:2019年9月4日(星期三) 22:20 To:dev Subject:[DISCUSS] FLIP-6

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

; >> Any idea about this? >> >> Best, >> Jingsong Lee >> >> >> -- >> From:Dawid Wysakowicz >> Send Time:2019年9月4日(星期三) 22:20 >> To:dev >> Subject:[DISCUSS] FLIP-64: Suppo

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

rtition support, etc.. > > Any idea about this? > > Best, > Jingsong Lee > > > -- > From:Dawid Wysakowicz > Send Time:2019年9月4日(星期三) 22:20 > To:dev > Subject:[DISCUSS] FLIP-64: Support for Temporary Objects

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

sakowicz Send Time:2019年9月4日(星期三) 22:20 To:dev Subject:[DISCUSS] FLIP-64: Support for Temporary Objects in Table module Hi all, As part of FLIP-30 a Catalog API was introduced that enables storing table meta objects permanently. At the same time the majority of current APIs create temporary

[DISCUSS] FLIP-64: Support for Temporary Objects in Table module

Hi all, As part of FLIP-30 a Catalog API was introduced that enables storing table meta objects permanently. At the same time the majority of current APIs create temporary objects that cannot be serialized. We