I think the JDBC one is more inefficient, slower requires too much development effort. You can also check the integration of Alluxio with Spark. Then, in general I think JDBC has never designed for large data volumes. It is for executing queries and getting a small or aggregated result set back. Alternatively for inserting / updating single rows.
> On 3. Aug 2017, at 08:17, Dmitriy Setrakyan <dsetrak...@apache.org> wrote: > > Jorn, thanks for your feedback! > > Can you explain how the direct support would be different from the JDBC > support? > > Thanks, > D. > >> On Thu, Aug 3, 2017 at 7:40 AM, Jörn Franke <jornfra...@gmail.com> wrote: >> >> These are two different things. Spark applications themselves do not use >> JDBC - it is more for non-spark applications to access Spark DataFrames. >> >> A direct support by Ignite would make more sense. Although you have in >> theory IGFS, if the user is using HDFS, which might not be the case. It is >> now also very common to use Object stores, such as S3. >> Direct support could be leverage for interactive analysis or different >> Spark applications sharing data. >> >>> On 3. Aug 2017, at 05:12, Dmitriy Setrakyan <dsetrak...@apache.org> >> wrote: >>> >>> Igniters, >>> >>> We have had the integration with Spark Data Frames on our roadmap for a >>> while: >>> https://issues.apache.org/jira/browse/IGNITE-3084 >>> >>> However, while browsing Spark documentation, I cam across the generic >> JDBC >>> data frame support in Spark: >>> https://spark.apache.org/docs/latest/sql-programming-guide. >> html#jdbc-to-other-databases >>> >>> Given that Ignite has a JDBC driver, does it mean that it transitively >> also >>> supports Spark data frames? If yes, we should document it. >>> >>> D. >>
