I do think Kafka is an overkill in this case. There are no streaming use-
cases that needs a queue to do pub-sub.
On 16-Mar-2017 11:47 AM, "vvshvv" wrote:
> Hi,
>
> >> A slightly over-kill solution may be Spark to Kafka to ElasticSearch?
>
> I do not think so, in this case you will be able to pr
Hi,
>> A slightly over-kill solution may be Spark to Kafka to ElasticSearch?
I do not think so, in this case you will be able to process Parquet files as usual, but Kafka will allow your Elasticsearch cluster to be stable and survive regarding the number of rows.
Regards,
Uladzimir
On jasbir.
