I think that is probably a question for the Spark Connector forum:
https://groups.google.com/a/lists.datastax.com/forum/#!forum/spark-connector-user
as
it’s much more related to the function of the connector than functionality
of Cassandra itself.
Cheers
Ben
On Sat, 17 Mar 2018 at 21:18 onmstester onmstester <onmstes...@zoho.com>
wrote:
>
> I'm querying a single cassandra partition using sqlContext and Its temView
> which creates more than 2000 tasks on spark and took about 360 seconds:
>
>
> sqlContext.read().format("org.apache.spark.sql.cassandra).options(ops).load.createOrReplaceTempView("tableName")
>
> But using javaFunctions(sc).cassandraTable().where() it creates only one
> task which response in 200 ms!
> I'm using exactly the same where clause for both scenarios.
> Spark UI shows like 60 GB input for sqlcontext scenario and only a few KBs
> for javaFunctions scenario
>
> Sent using Zoho Mail <https://www.zoho.com/mail/>
>
>
>
> --
*Ben Slater*
*Chief Product Officer <https://www.instaclustr.com/>*
<https://www.facebook.com/instaclustr> <https://twitter.com/instaclustr>
<https://www.linkedin.com/company/instaclustr>
Read our latest technical blog posts here
<https://www.instaclustr.com/blog/>.
This email has been sent on behalf of Instaclustr Pty. Limited (Australia)
and Instaclustr Inc (USA).
This email and any attachments may contain confidential and legally
privileged information. If you are not the intended recipient, do not copy
or disclose its content, but please reply to this email immediately and
highlight the error to the sender and then immediately delete the message.
