Re: cassandra spark-connector-sqlcontext too many tasks

2018-03-17 Thread Ben Slater
I think that is probably a question for the Spark Connector forum: https://groups.google.com/a/lists.datastax.com/forum/#!forum/spark-connector-user as it’s much more related to the function of the connector than functionality of Cassandra itself. Cheers Ben On Sat, 17 Mar 2018 at 21:18

cassandra spark-connector-sqlcontext too many tasks

2018-03-17 Thread onmstester onmstester
I'm querying a single cassandra partition using sqlContext and Its temView which creates more than 2000 tasks on spark and took about 360 seconds: sqlContext.read().format("org.apache.spark.sql.cassandra).options(ops).load.createOrReplaceTempView("tableName") But using