[
https://issues.apache.org/jira/browse/SPARK-36525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17450224#comment-17450224
]
Huaxin Gao commented on SPARK-36525:
------------------------------------
The major reason I work on index support is because I have customers who need
this in iceberg. I don't have any plan to make FileTable implement SupportIndex
because parquet or ORC doesn't support index.
> DS V2 Index Support
> -------------------
>
> Key: SPARK-36525
> URL: https://issues.apache.org/jira/browse/SPARK-36525
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Huaxin Gao
> Priority: Major
>
> Many data sources support index to improvement query performance. In order to
> take advantage of the index support in data source, the following APIs will
> be added for working with indexes:
> {code:java}
> public interface SupportsIndex extends Table {
> /**
> * Creates an index.
> *
> * @param indexName the name of the index to be created
> * @param indexType the type of the index to be created. If this is not
> specified, Spark
> * will use empty String.
> * @param columns the columns on which index to be created
> * @param columnsProperties the properties of the columns on which index to
> be created
> * @param properties the properties of the index to be created
> * @throws IndexAlreadyExistsException If the index already exists.
> */
> void createIndex(String indexName,
> String indexType,
> NamedReference[] columns,
> Map<NamedReference, Map<String, String>> columnsProperties,
> Map<String, String> properties)
> throws IndexAlreadyExistsException;
> /**
> * Drops the index with the given name.
> *
> * @param indexName the name of the index to be dropped.
> * @throws NoSuchIndexException If the index does not exist.
> */
> void dropIndex(String indexName) throws NoSuchIndexException;
> /**
> * Checks whether an index exists in this table.
> *
> * @param indexName the name of the index
> * @return true if the index exists, false otherwise
> */
> boolean indexExists(String indexName);
> /**
> * Lists all the indexes in this table.
> */
> TableIndex[] listIndexes();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
