sessionization in spark 1.6 would
also help (couldn't find anything that helped me)
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Spark-SQL-Design-suggestions-needed-for-sessionization-tp28480.html
Sent from the Apache Spark User List mailing list
Hi all,
I am using Spark Streaming to monitor an S3 bucket for objects that contain
JSON. I want
to import that JSON into Spark SQL DataFrame.
Here's my current code:
*from pyspark import SparkContext, SparkConf*
*from pyspark.streaming import StreamingContext*
*import json*
*from pyspark.sql
Hi all,
I figured it out! The DataFrames and SQL example in Spark Streaming docs
were useful.
Best,
Vadim
ᐧ
On Wed, Apr 8, 2015 at 2:38 PM, Vadim Bichutskiy vadim.bichuts...@gmail.com
wrote:
Hi all,
I am using Spark Streaming to monitor an S3 bucket for objects that
contain JSON. I want
You probably want to mark the HiveContext as @transient as its not valid to
use it on the slaves anyway.
On Mon, Feb 16, 2015 at 1:58 AM, Haopu Wang hw...@qilinsoft.com wrote:
I have a streaming application which registered temp table on a
HiveContext for each batch duration.
The
I have a streaming application which registered temp table on a
HiveContext for each batch duration.
The application runs well in Spark 1.1.0. But I get below error from
1.1.1.
Do you have any suggestions to resolve it? Thank you!
java.io.NotSerializableException:
