The objective is to create a universal way for Beam SQL IO APIs to support filter/project push-down. A proposed way to achieve that is by introducing an interface responsible for identifying what portion(s) of a Calc can be moved down to IO layer. Also, adding following methods to a BeamSqlTable interface to pass necessary parameters to IO APIs: - BeamSqlTableFilter supportsFilter(RexNode program, RexNode filter) - Boolean supportsProjects() - PCollection<Row> buildIOReader(PBegin begin, BeamSqlTableFilter filters, List<String> fieldNames)
Please feel free to provide feedback and suggestions on this proposal. Thank you! Here is a more complete design doc: https://docs.google.com/document/d/1-ysD7U7qF3MAmSfkbXZO_5PLJBevAL9bktlLCerd_jE/edit?usp=sharing -- Kirill Kozlov
