The example query didn't get posted in the previous email.
Adding it here:
> spark.sql("select cast((id || 'name') as int) from range(10) ").show
> +-+
|CAST(concat(CAST(id AS STRING), name) AS INT)|
+-+
|
I noticed that Spark handles CAST operation in fail-safe manner. i.e. if the
casting operation will fail for some record(s), Spark doesn't fail the
entire query instead it returns null data value for those failures.
For example following query:
Looking at the code it seems that this behavior
spark-core_2.11
- AVRO VERSION - 1.8.2
Please help us understand, why this issue is coming?
Thanks,
Vatsal
Is there any configurable timeout which controls queuing of the driver in Mesos
cluster mode or the driver will remain in queue for indefinite until it find
resource on cluster?
From: Michael Gummelt [mailto:mgumm...@mesosphere.io]
Sent: Friday, May 26, 2017 11:33 PM
To: Mevada, Vatsal <
of queuing up) if sufficient resources are not found on
Mesos cluster?
Regards,
Vatsal
In my Spark Streaming application I am reading data from certain Kafka topic.
While reading from topic whenever I encounter certain message (for example:
"poison") I want to stop the streaming. Currently I am achieving this using
following code: jsc is instance of JavaStreamingContext and
In my Spark Streaming application I am reading data from certain Kafka topic.
While reading from topic whenever I encounter certain message (for example:
"poison") I want to stop the streaming. Currently I am achieving this using
following code: jsc is instance of JavaStreamingContext and
