[
https://issues.apache.org/jira/browse/IGNITE-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289447#comment-16289447
]
Nikolay Izhikov commented on IGNITE-3084:
-----------------------------------------
> I'm not sure I understand the value of custom catalog implementation. Can you
> please elaborate what exactly it provides to a user?
It provide ability of automatic tables resolving.
One don't need to register every single Ignite table to make Spark queries to
it.
Without Catalog you has to execute following piece of code for each Ignite
table you want to use:
{{code:scala}}
val df = spark.read
.format(IGNITE) //Data source type.
.option(TABLE, "person") //Table to read.
.option(CONFIG_FILE, CONFIG) //Ignite config.
.load()
//Registering DataFrame as Spark view.
df.createOrReplaceTempView("person")
{{code}}
> Spark Data Frames Support in Apache Ignite
> ------------------------------------------
>
> Key: IGNITE-3084
> URL: https://issues.apache.org/jira/browse/IGNITE-3084
> Project: Ignite
> Issue Type: Task
> Components: spark
> Affects Versions: 1.5.0.final
> Reporter: Vladimir Ozerov
> Assignee: Nikolay Izhikov
> Priority: Critical
> Labels: bigdata, important
> Fix For: 2.4
>
>
> Apache Spark already benefits from integration with Apache Ignite. The latter
> provides shared RDDs, an implementation of Spark RDD, that help Spark to
> share a state between Spark workers and execute SQL queries much faster. The
> next logical step is to enable support for modern Spark Data Frames API in a
> similar way.
> As a contributor, you will be fully in charge of the integration of Spark
> Data Frame API and Apache Ignite.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
