Sunitha Kambhampati created SPARK-26249:
-------------------------------------------
Summary: Extension Points Enhancements to inject a rule in order
and to add a batch
Key: SPARK-26249
URL: https://issues.apache.org/jira/browse/SPARK-26249
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.4.0
Reporter: Sunitha Kambhampati
+Motivation:+
Spark has extension points API to allow third parties to extend Spark with
custom optimization rules. The current API does not allow fine grain control
on when the optimization rule will be exercised.
In our use cases, we have optimization rules that we want to add as extensions
to a batch in a specific order.
In the current API, there is no way to add a batch to the optimization using
the SparkSessionExtensions API, similar to the postHocOptimizationBatches in
SparkOptimizer.
+Proposal:+
Add 2 new API's to the existing Extension Points to allow for more flexibility
for third party users of Spark.
# Inject a optimizer rule to a batch in order
# Inject a optimizer batch in order
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
