Hi Jacques, thanks for your quick reply - the goal of all this is to get some information from the where clause e.g. "t between 2016-01-01 and 2016-01-20" and make it available to the record reader that I am returning from EasyFormatPlugin.getRecordReader() which is only provided a FragmentContext, and the file paths to read. I couldn't see a way to get from the FragmentContext back to the original where clause so start looking at trying to modify the FileWork objects the reader is given to include the extra information.
cheers Lee [image: MahiFX] <https://mahifx.com/> Lee Butts Technology Level 3, 50 Victoria Street Christchurch 8013 New Zealand Mobile: +64 226 908315 Skype: leemahifx <https://www.facebook.com/mahifx> <https://twitter.com/mahiforex> <https://www.youtube.com/channel/UCxCiKv8XEM8GATA8kFFqsAw> <https://www.linkedin.com/company/mahifx> <https://mahifx.com/blog/feed.xml> <https://plus.google.com/+Mahifx/posts> [image: MFX Compass] <http://mfxcompass.mahifx.com/> On 5 April 2016 at 17:25, Jacques Nadeau <[email protected]> wrote: > On your strategy, generally, rules should be independent and rel nodes > should be immutable. I don't fully understand what you are trying to do but > your description seems like a bit of an anti-pattern. > > You'll also need to figure out what phase or phases should include your > rules. > On Apr 4, 2016 9:55 PM, "Lee Butts" <[email protected]> wrote: > > > Hi, > > > > I'm trying to add some custom optimizer rules for my storage format > plugin > > and am having trouble understanding the process the query planner goes > > through when building logical and physical query plans. > > > > What I'd like to do is the following: > > > > 1) Add a rule which matches all filters and collects some information > about > > restrictions being put on a known column (call it T). This rule does not > > change any nodes in the logical or physical plan > > > > 2) Add a second rule which uses the information collected by rule #1 and > > modifies all scan operations by changing the file paths that will be > > scanned. In this rule I want to append some information to the path that > my > > new record reader can use to be smarter about how much data it reads from > > the given file. This rule would replace all Scan (or DrillScan? Not sure > > which one is correct) nodes with modified versions referencing the > modified > > GroupScan. > > > > What I'm finding is that my rules are matching and the calls to > > transformTo() are executing but for some reason a plan with the original > > Scan objects is being picked as the cheapest and my reader does not > > received my modified GroupScan info. > > > > Am I going about this all wrong? Is there an easier way for me to pass > > information from the where clause out to a record reader via some other > > context object? > > > > cheers > > > > Lee > > > > [image: MahiFX] <https://mahifx.com/> > > > > Lee Butts > > Technology > > Level 3, 50 Victoria Street > > Christchurch 8013 > > New Zealand > > Mobile: +64 226 908315 > > Skype: leemahifx > > > > <https://www.facebook.com/mahifx> <https://twitter.com/mahiforex> > > <https://www.youtube.com/channel/UCxCiKv8XEM8GATA8kFFqsAw> > > <https://www.linkedin.com/company/mahifx> > > <https://mahifx.com/blog/feed.xml> < > > https://plus.google.com/+Mahifx/posts> > > > > > > [image: MFX Compass] <http://mfxcompass.mahifx.com/> > > > > -- > > > > > > ------------------------------ > > > > *IMPORTANT NOTICE*: MahiFX Limited (*MahiFX*), (company no. 2446590) is > > registered in New Zealand. MahiFX is authorised and regulated by the > > Australian Securities and Investment Commission ARBN 152 535 085, AFSL > > 414198. This email, its attachments and any rights attaching hereto are > > confidential and intended exclusively for the person to whom the email is > > addressed. If you are not the intended recipient, do not read, copy, > > disclose or use the contents in any way. Please notify the sender by > return > > email and destroy the email and attachments immediately. MahiFX does not > > accept any liability for any changes made to this email or attachments > > after sending by MahiFX. You must scan this email and attachments for > > viruses. The opinions expressed are not necessarily those of MahiFX. > MahiFX > > accepts no liability for any loss, damage or consequence, whether caused > by > > our own negligence or not, resulting directly or indirectly from the use > of > > this email and attachments. > > > > For more information about MahiFX Limited see mahifx.com. > > > -- ------------------------------ *IMPORTANT NOTICE*: MahiFX Limited (*MahiFX*), (company no. 2446590) is registered in New Zealand. MahiFX is authorised and regulated by the Australian Securities and Investment Commission ARBN 152 535 085, AFSL 414198. This email, its attachments and any rights attaching hereto are confidential and intended exclusively for the person to whom the email is addressed. If you are not the intended recipient, do not read, copy, disclose or use the contents in any way. Please notify the sender by return email and destroy the email and attachments immediately. MahiFX does not accept any liability for any changes made to this email or attachments after sending by MahiFX. You must scan this email and attachments for viruses. The opinions expressed are not necessarily those of MahiFX. MahiFX accepts no liability for any loss, damage or consequence, whether caused by our own negligence or not, resulting directly or indirectly from the use of this email and attachments. For more information about MahiFX Limited see mahifx.com.
