On Thu, Aug 3, 2017 at 8:45 AM, Jörn Franke <jornfra...@gmail.com> wrote:

> I think the JDBC one is more inefficient, slower requires too much
> development effort. You can also check the integration of Alluxio with
> Spark.
>

As far as I know, Alluxio is a file system, so it cannot use JDBC. Ignite,
on the other hand, is an SQL system and works well with JDBC. As far as the
development effort, we are dealing with SQL, so I am not sure why JDBC
would be harder.

Generally speaking, until Ignite provides native data frame integration,
having JDBC-based integration out of the box is minimally acceptable.


> Then, in general I think JDBC has never designed for large data volumes.
> It is for executing queries and getting a small or aggregated result set
> back. Alternatively for inserting / updating single rows.
>

Agree in general. However, Ignite JDBC is designed to work with larger data
volumes and supports data pagination automatically.


> > On 3. Aug 2017, at 08:17, Dmitriy Setrakyan <dsetrak...@apache.org>
> wrote:
> >
> > Jorn, thanks for your feedback!
> >
> > Can you explain how the direct support would be different from the JDBC
> > support?
> >
> > Thanks,
> > D.
> >
> >> On Thu, Aug 3, 2017 at 7:40 AM, Jörn Franke <jornfra...@gmail.com>
> wrote:
> >>
> >> These are two different things. Spark applications themselves do not use
> >> JDBC - it is more for non-spark applications to access Spark DataFrames.
> >>
> >> A direct support by Ignite would make more sense. Although you have in
> >> theory IGFS, if the user is using HDFS, which might not be the case. It
> is
> >> now also very common to use Object stores, such as S3.
> >> Direct support could be leverage for interactive analysis or different
> >> Spark applications sharing data.
> >>
> >>> On 3. Aug 2017, at 05:12, Dmitriy Setrakyan <dsetrak...@apache.org>
> >> wrote:
> >>>
> >>> Igniters,
> >>>
> >>> We have had the integration with Spark Data Frames on our roadmap for a
> >>> while:
> >>> https://issues.apache.org/jira/browse/IGNITE-3084
> >>>
> >>> However, while browsing Spark documentation, I cam across the generic
> >> JDBC
> >>> data frame support in Spark:
> >>> https://spark.apache.org/docs/latest/sql-programming-guide.
> >> html#jdbc-to-other-databases
> >>>
> >>> Given that Ignite has a JDBC driver, does it mean that it transitively
> >> also
> >>> supports Spark data frames? If yes, we should document it.
> >>>
> >>> D.
> >>
>

Reply via email to