Gko1geGkhurMfq7watcAdgoE=mBMkukfF49HDvJdBwhtMyMmZzj3W-AC-0gVkjmuGHb4=>
>
>
>
>
>
>
>
>
>
>
> On Wed, Jan 9, 2019 at 4:51 AM wrote:
>
> I have a data frame for which I apply an UDF that calls a REST web
> service. This web service is distributed in only
libraries?
Thank you
From: Ramandeep Singh
Sent: Thursday, January 10, 2019 1:48 AM
To: Sonal Goyal
Cc: em...@yeikel.com; user
Subject: Re: Is it possible to rate limit an UDP?
Backpressure is the suggested way out here and is the correct approach, it rate
limits at the source itself
>
> On Wed, Jan 9, 2019 at 4:51 AM wrote:
>
>> I have a data frame for which I apply an UDF that calls a REST web
>> service. This web service is distributed in only a few nodes and it won’t
>> be able to handle a massive load from Spark.
>>
>>
>
istributed in only a few nodes and it won’t
> be able to handle a massive load from Spark.
>
>
>
> Is it possible to rate limit this UDP? For example , something like 100
> op/s.
>
>
>
> If not , what are the options? Is splitting the df an option?
>
>
I have a data frame for which I apply an UDF that calls a REST web service.
This web service is distributed in only a few nodes and it won't be able to
handle a massive load from Spark.
Is it possible to rate limit this UDP? For example , something like 100
op/s.
If not , what
