Got it, thanks Julian! That makes sense. Sorry if the question was a bit pedantic.
Madhav On Tue, Nov 19, 2019 at 5:22 PM Julian Hyde <[email protected]> wrote: > Make your own JDBC connection. Calcite doesn't have any "magic" that > allows it to see into a data source's metadata. > > If the RelNode in question is a JdbcTableScan then you should have the > means to create a JDBC connection. > > On Tue, Nov 19, 2019 at 1:43 PM Madhav Suresh <[email protected]> > wrote: > > > > We have a handler implemented, for example we have implemented > > getPrivacyMode() method in the RelMetadataQuery as a proof of > > concept and it seems to do the right thing. Our question was regarding > > how to read the underlying data from the source database. We can > > open a JDBC connection and read in the data for the metadata provider to > > use - however we were not sure if there was existing functionality to > bring > > in > > table metadata. > > > > If, for example, we wanted to make a metadata provider which provides > > histograms > > for cardinality estimates, where/how should the provider read in the > > histograms from > > the underlying database? > > > > On Tue, Nov 19, 2019 at 1:00 PM Julian Hyde <[email protected]> wrote: > > > > > > is there existing functionality to read in external > > > > statistics and metadata - for example things like table statistics. > > > > > > Yes. Write a handler for that kind of metadata, and put it in the > > > RelMetadataProvider that Calcite uses to prepare the query. Then when > > > it asks for (say) the row count of a table it will call your handler > > > rather than the built-in one. > > > > > > On Tue, Nov 19, 2019 at 6:08 AM Madhav Suresh < > [email protected]> > > > wrote: > > > > > > > > Public/private columns are attributes we created for our > implementation > > > of > > > > a private query processor. Right now we store those attributes in > > > > information_schema. We want to read those into calcite for Query > planning > > > > > > > > Madhav > > > > > > > > On Tue, Nov 19, 2019 at 4:47 AM Muhammad Gelbana < > [email protected]> > > > > wrote: > > > > > > > > > To me, your question is still unclear. Are you asking about system > > > columns > > > > > [1] ? > > > > > > > > > > What are public/private attributes ? > > > > > > > > > > [1] > https://www.postgresql.org/docs/current/ddl-system-columns.html > > > > > > > > > > > > > > > On Tue, Nov 19, 2019 at 5:12 AM Madhav Suresh < > > > [email protected]> > > > > > wrote: > > > > > > > > > > > Thanks for the reply Julian! > > > > > > > > > > > > I should post more details: right now we have a Metadata class > which > > > > > > implements the Metadata spec. All of that seems to work fine. > Where > > > we > > > > > are > > > > > > stuck is in how to read the information_schema metadata from JDBC > > > into > > > > > the > > > > > > planner. One hackish solution is to open a DB connection when our > > > > > Metadata > > > > > > getter is called on TableScan, and run a query against the > underlying > > > > > > database. However, is there existing functionality to read in > > > external > > > > > > statistics and metadata - for example things like table > statistics. > > > > > > > > > > > > Madhav > > > > > > > > > > > > On Mon, Nov 18, 2019 at 8:42 PM Julian Hyde <[email protected]> > > > wrote: > > > > > > > > > > > > > May need to be a new type of metadata (i.e. a class that > implements > > > > > > > Metadata and is acquired via RelMetadataQuery) [1] [2]. > > > > > > > > > > > > > > Julian > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > https://calcite.apache.org/apidocs/org/apache/calcite/rel/metadata/Metadata.html > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > https://calcite.apache.org/apidocs/org/apache/calcite/rel/metadata/Metadata.html > > > > > > > > > > > > > > > > > > > > > > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > https://calcite.apache.org/apidocs/org/apache/calcite/rel/metadata/package-summary.html > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > https://calcite.apache.org/apidocs/org/apache/calcite/rel/metadata/package-summary.html > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Nov 18, 2019, at 5:40 PM, Madhav Suresh < > > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > We have security type information that we want to bring into > the > > > > > > > optimizer. > > > > > > > > Consider the query: > > > > > > > > SELECT a FROM a,b where a.id=b.id AND a.p > 10; > > > > > > > > We want the optimizer to know that a.id, b.id are "public" > > > > > attributes, > > > > > > > and > > > > > > > > a.p is a private attribute. Currently we have type > information > > > stored > > > > > > > > inform information_schema in postgres. Right now our > prototype > > > > > > considers > > > > > > > > privacy as a RelTrait. We are stuck on what the best way to > read > > > this > > > > > > > > information is. Is there a similar example, or template on > how to > > > > > read > > > > > > in > > > > > > > > table metadata into calcite? > > > > > > > > > > > > > > > > Madhav > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
