[
https://issues.apache.org/jira/browse/HUDI-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar updated HUDI-8534:
---------------------------------
Description:
We need to ensure the user can execute `CREATE INDEX name ...`. with same name,
failing many times and then be able to eventually succeed with the right
syntax/parameters.
{code:java}
spark-sql (default)> create index idx_bloom on hudi_table using
bloom_filter(city) options(func='lower');
24/11/16 09:58:44 ERROR SparkSQLDriver: Failed in [create index idx_bloom on
hudi_table using bloom_filter(city) options(func='lower')]
java.lang.IllegalArgumentException: The value of hoodie.functional.index.type
should be one of COLUMN_STATS,BLOOM_FILTERS,SECONDARY_INDEX, but was
bloom_filter
spark-sql (default)> create index idx_bloom on hudi_table using
bloom_filters(city) options(func='lower');
24/11/16 09:59:11 ERROR SparkSQLDriver: Failed in [create index idx_bloom on
hudi_table using bloom_filters(city) options(func='lower')]
java.lang.IllegalArgumentException: The value of hoodie.functional.index.type
should be one of COLUMN_STATS,BLOOM_FILTERS,SECONDARY_INDEX, but was
bloom_filter
{code}
was:
is this expected behavior? We should be able to create a bloom_filters index on
any column without having to specify any options, including a conversion
function. Also lets allow for users to configure the bloom filter - number of
bits, fp ratio etc?
{code:java}
spark-sql (default)> create index idx_bloom1 on hudi_table using
bloom_filters(city);
24/11/16 10:04:44 ERROR SparkSQLDriver: Failed in [create index idx_bloom1 on
hudi_table using bloom_filters(city)]
org.apache.hudi.exception.HoodieIndexException: Unknown hoodie index
type:bloom_filters
at
org.apache.hudi.index.secondary.SecondaryIndexType.lambda$of$3(SecondaryIndexType.java:55)
at java.util.Optional.orElseThrow(Optional.java:290)
at
org.apache.hudi.index.secondary.SecondaryIndexType.of(SecondaryIndexType.java:54)
at
org.apache.hudi.index.secondary.HoodieSecondaryIndex$Builder.setIndexType(HoodieSecondaryIndex.java:112)
at
org.apache.hudi.index.secondary.SecondaryIndexManager.create(SecondaryIndexManager.java:110)
at
org.apache.spark.sql.hudi.command.CreateIndexCommand.run(IndexCommands.scala:60)
{code}
> Ensure index creation is idempotent in face of failures
> -------------------------------------------------------
>
> Key: HUDI-8534
> URL: https://issues.apache.org/jira/browse/HUDI-8534
> Project: Apache Hudi
> Issue Type: Task
> Reporter: Vinoth Chandar
> Assignee: Lokesh Jain
> Priority: Blocker
> Fix For: 1.0.0
>
>
> We need to ensure the user can execute `CREATE INDEX name ...`. with same
> name, failing many times and then be able to eventually succeed with the
> right syntax/parameters.
> {code:java}
> spark-sql (default)> create index idx_bloom on hudi_table using
> bloom_filter(city) options(func='lower');
> 24/11/16 09:58:44 ERROR SparkSQLDriver: Failed in [create index idx_bloom on
> hudi_table using bloom_filter(city) options(func='lower')]
> java.lang.IllegalArgumentException: The value of hoodie.functional.index.type
> should be one of COLUMN_STATS,BLOOM_FILTERS,SECONDARY_INDEX, but was
> bloom_filter
> spark-sql (default)> create index idx_bloom on hudi_table using
> bloom_filters(city) options(func='lower');
> 24/11/16 09:59:11 ERROR SparkSQLDriver: Failed in [create index idx_bloom on
> hudi_table using bloom_filters(city) options(func='lower')]
> java.lang.IllegalArgumentException: The value of hoodie.functional.index.type
> should be one of COLUMN_STATS,BLOOM_FILTERS,SECONDARY_INDEX, but was
> bloom_filter
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
