Hi Bhavya, Thanks for your explanation.
I do not think a rule like ProjectFilterTableScan solves the problem. To see this, please note that the parent operator of the LogicalProject may need other columns from the table (e.g. the parent operator may be a join, which uses some other columns as the join key). So the column pruning of table scan should be a global decision, depending on the whole plan. Calcite has RelMdColumnOrigins, which can help in this scenario. In particular, given an output column in the output operator, it gives the dependent columns in the underlying source tables. So columns not referenced from the final output can be pruned from the source table. Best, Liya Fan On Fri, Dec 18, 2020 at 4:43 PM Bhavya Aggarwal <[email protected]> wrote: > Hi Liya, > > Please see below , I have a query as given below > > SELECT t_1.ID FROM USERS t_1 where NAME = 'HELLO'; > > Now I know that I have to fetch two columns from my columnar database i.e. > ID, Name, so need to pass these parameters in my custom Table Scan. > But when I convert the LogicalTableScan to my custom Table scan I do not > have that information, please see the tree below, in the Logical Table Scan > the Input is null, I was hoping that I will have the columns in the > LogicalTableScan, The question is what do I need to do in Calcite to pass > that information to the LogicalTableScan or in my physical operator. The > way I have found is that I create a Rule like ProjectFilterTableScan rule > and then extract what information I need. Is there a better way to do this? > > -----------------Logical JSON Plan -------------------- > > { > > "rels": [ > > { > > "id": "0", > > "relOp": "LogicalTableScan", > > "table": [ > > "USERS" > > ], > > "inputs": [] > > }, > > { > > "id": "1", > > "relOp": "LogicalFilter", > > "condition": { > > "op": { > > "name": "=", > > "kind": "EQUALS", > > "syntax": "BINARY" > > }, > > "operands": [ > > { > > "op": { > > "name": "CAST", > > "kind": "CAST", > > "syntax": "SPECIAL" > > }, > > "operands": [ > > { > > "input": 1, > > "name": "$1" > > } > > ], > > "type": { > > "type": "CHAR", > > "nullable": true, > > "precision": 5 > > } > > }, > > { > > "literal": "HELLO", > > "type": { > > "type": "CHAR", > > "nullable": false, > > "precision": 5 > > } > > } > > ] > > } > > }, > > { > > "id": "2", > > "relOp": "LogicalProject", > > "fields": [ > > "ID" > > ], > > "exprs": [ > > { > > "input": 0, > > "name": "$0" > > } > > ] > > } > > ] > > } > > > >> ----------------------------------------------------------- > > > > > Thanks and regards > Bhavya > > On Fri, Dec 18, 2020 at 11:54 AM Fan Liya <[email protected]> wrote: > > > Hi Bhavya, > > > > Sorry I do not understand your question. Why is it difficult to pass sth. > > to child operators? > > > > Best, > > Liya Fan > > > > > > On Fri, Dec 18, 2020 at 12:38 PM Bhavya Aggarwal <[email protected]> > > wrote: > > > > > Hi Liya, > > > > > > Actually the question is how to pass the projection columns to Table > Scan > > > as right now in the LogicalTableScan there are no projection columns > > being > > > passed so when I am trying to create my custom JSON , I do not have > those > > > projected columns or columns that are being used in the query. I want > to > > > understand what is the calcite mechanism for passing it to child > > operators > > > without using Rules as it will be a lot of rule that we need to > > implement. > > > > > > Thanks and Regards > > > Bhavya > > > > > > On Fri, Dec 18, 2020 at 9:47 AM Fan Liya <[email protected]> wrote: > > > > > > > Hi Bhavya, > > > > > > > > It seems the problem is that the json format provided by Calcite does > > not > > > > include properties specific to your operator? > > > > If so, I think you can override RelNode#explainTerms in your operator > > to > > > > attach the properties. > > > > > > > > Best, > > > > Liya Fan > > > > > > > > On Thu, Dec 17, 2020 at 10:45 PM Bhavya Aggarwal <[email protected] > > > > > > wrote: > > > > > > > > > Please let me know if we have an example for writing a custom > visitor > > > > that > > > > > I can use. I see a RelVisitor class that I can extend but how to > make > > > it > > > > to > > > > > visit the specific Physical Operator. > > > > > > > > > > Thanks and Regards > > > > > Bhavya > > > > > > > > > > On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal < > [email protected]> > > > > > wrote: > > > > > > > > > > > Thanks, > > > > > > > > > > > > I tried using the RelJsonWriter and tried to add my custom > > > > implementation > > > > > > in the my custom Rel Nodes but I am not getting all the data in > my > > > > child > > > > > > operators. We are trying to use Calcite for a columnar data and > we > > > need > > > > > the > > > > > > information of the projections in the TableScan as well because > we > > > need > > > > > to > > > > > > pass those projections to read specific columns but I am not able > > to > > > > find > > > > > > the projections in my Table scan. As per my understanding Calcite > > > > should > > > > > be > > > > > > passing this information as it is required for me reading the > > table. > > > > > Please > > > > > > let me know if I need to do something special to get that > > information > > > > in > > > > > > the child operator. > > > > > > > > > > > > Regards > > > > > > Bhavya > > > > > > > > > > > > On Sun, Dec 13, 2020 at 11:32 PM Michael Mior <[email protected]> > > > > wrote: > > > > > > > > > > > >> That would likely be the best approach if you have some specific > > > JSON > > > > > >> format you're trying to generate. If you're happy with a JSON > > format > > > > > >> defined by Calcite, you can look at RelJsonWriter. > > > > > >> > > > > > >> -- > > > > > >> Michael Mior > > > > > >> [email protected] > > > > > >> > > > > > >> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana < > > [email protected] > > > > > > > > a > > > > > >> écrit : > > > > > >> > > > > > > >> > I would use a visitor to traverse the optimized/physical plan. > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal < > > > [email protected] > > > > > > > > > > >> wrote: > > > > > >> > > > > > > >> > > Hi All, > > > > > >> > > > > > > > >> > > We need to generate a JSON object for the physical execution > > > tree > > > > > >> that has > > > > > >> > > been created. Is there an option in Calcite that we can use > to > > > do > > > > > >> this. I > > > > > >> > > am not sure what is the right approach to do it. Please let > me > > > > know > > > > > if > > > > > >> > > there are different ways to achieve this. > > > > > >> > > > > > > > >> > > Regards > > > > > >> > > Bhavya > > > > > >> > > > > > > > >> > > -- > > > > > >> > > Your feedback matters - At Knoldus we aim to be very > > > professional > > > > in > > > > > >> our > > > > > >> > > quality of work, commitment to results, and proactive > > > > communication. > > > > > >> If > > > > > >> > > you > > > > > >> > > feel otherwise please share your feedback > > > > > >> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on > > it. > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > -- > > > > > > *Bhavya Aggarwal* > > > > > > CTO & Partner > > > > > > Knoldus Inc. <http://www.knoldus.com/> > > > > > > +91-9910483067 > > > > > > Canada - USA - India - Singapore > > > > > > <https://in.linkedin.com/company/knoldus> < > > > > https://twitter.com/Knolspeak > > > > > > > > > > > > <https://www.facebook.com/KnoldusSoftware/> < > > > https://blog.knoldus.com/ > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > *Bhavya Aggarwal* > > > > > CTO & Partner > > > > > Knoldus Inc. <http://www.knoldus.com/> > > > > > +91-9910483067 > > > > > Canada - USA - India - Singapore > > > > > <https://in.linkedin.com/company/knoldus> < > > > https://twitter.com/Knolspeak > > > > > > > > > > <https://www.facebook.com/KnoldusSoftware/> < > > https://blog.knoldus.com/ > > > > > > > > > > > > > > -- > > > > > Your feedback matters - At Knoldus we aim to be very professional > in > > > our > > > > > quality of work, commitment to results, and proactive > communication. > > If > > > > > you > > > > > feel otherwise please share your feedback > > > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it. > > > > > > > > > > > > > > > > > > -- > > > *Bhavya Aggarwal* > > > CTO & Partner > > > Knoldus Inc. <http://www.knoldus.com/> > > > +91-9910483067 > > > Canada - USA - India - Singapore > > > <https://in.linkedin.com/company/knoldus> < > https://twitter.com/Knolspeak > > > > > > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/ > > > > > > > > -- > > > Your feedback matters - At Knoldus we aim to be very professional in > our > > > quality of work, commitment to results, and proactive communication. If > > > you > > > feel otherwise please share your feedback > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it. > > > > > > > > -- > *Bhavya Aggarwal* > CTO & Partner > Knoldus Inc. <http://www.knoldus.com/> > +91-9910483067 > Canada - USA - India - Singapore > <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak> > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/> > > -- > Your feedback matters - At Knoldus we aim to be very professional in our > quality of work, commitment to results, and proactive communication. If > you > feel otherwise please share your feedback > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it. >
