Hello Spark community,

My name is Scott Schenkein, and I am the maintainer of IndexTables
<https://www.indextables.io/> which is an open table format that provides
fast search services, either as a standalone table type or as a companion
index to Iceberg or Delta Lake.

In my implementation, I've needed to build a parallel pushdown structure
because several operators I rely on are not currently supported by Spark,
and there is no existing extensibility mechanism for predicate pushdown.

Last month I opened an issue
<https://issues.apache.org/jira/browse/SPARK-55869> and a pull request
<https://github.com/apache/spark/pull/54669>to add support for these
capabilities, but I've had difficulty finding a reviewer. A committer from
the Iceberg project suggested I reach out to this list as a good place to
connect with potential reviewers.

If anyone has bandwidth to take a look, I would greatly appreciate it.
Happy to provide any additional context about the motivation or design.

Thank you,
Scott Schenkein

Reply via email to