olumn("time",col("time").cast("timestamp"))\
.orderBy("dev_id")
I want to know how data is distributed over multiple executors. I want the
data to be distributed on the basis of dev_id, each executor gets all data
from one dev_id. Later I group by dev_id an
> at
> org.apache.spark.sql.execution.datasources.DataSource$$anonfun$23.apply(
> DataSource.scala:618)
> at scala.util.Try.orElse(Try.scala:84)
> at
> org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(
> DataSource.scala:618)
> ... 12 more
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>
--
Regards,
Arbab Khalil
Software Design Engineer
Why are you reading batch from kafka and writing it as stream?
On Fri, Jul 27, 2018, 1:40 PM dddaaa wrote:
> No, I just made sure I'm not doing it.
> changed the path in .start() to another path and the same still occurs.
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.co
om_json(col("value").cast("string"), schema)/
>
>
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> ---------
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>
--
Regards,
Arbab Khalil
Software Design Engineer
%40znv.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%220049003208%40znv.com%22%5D>
>> 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制
>> - To
>> unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
> --
>
> --
> Daniel de Oliveira Mantovani
> Perl Evangelist/Data Hacker
> +1 786 459 1341
>
>
--
Regards,
Arbab Khalil
Software Design Engineer
this without putting dataframe B in a
> broadcast variable or doing a broadcast join ?
>
>
--
Regards,
Arbab Khalil
Software Design Engineer
