Hey, Cameron,
Thanks for the detailed answers. It would be good to add this explanation
to the SEP page as well.
Otherwise, +1 from my side. Thanks!
-Yi
On Mon, Mar 16, 2020 at 10:06 AM Cameron Lee
wrote:
> You have the correct understanding about the "yarn.resources.*"
> configuration, and
You have the correct understanding about the "yarn.resources.*"
configuration, and your question is a good one. Currently, the
implementation is that Samza will look in a specific place on the file
system (i.e. /__samzaFrameworkApi and /__samzaFrameworkInfrastructure) to get the
API/infrastructure
OK. If I understand correctly, your answer is the following:
yarn.resources.* configuration variables are used by YARN localizer to make
API and infrastructure classpath available, together with the application's
own classpath, which is also determined by the YARN localizer.
The question here is:
The configuration variables are only used by the YARN localizer. The Samza
application will look for the framework resources in certain places in the
application's working directory when it needs to access them. My aim is to
do something similar to how "yarn.package.path" works. In other execution
Hi, Cameron,
Thanks for the quick responses! Appreciate it.
I am still having a concern on a): are those configuration variables used
by YARN localizer or by Samza applications? If those are used only by the
YARN localizer, I agree that we should keep those as yarn specific.
Otherwise, I think
a) The "yarn.resources.*" configs are for localizing the necessary
resources into the working directory for the process. I felt that the
specific configuration format to specify these resources might be
YARN-specific (e.g. YARN has type and visibility configs for each of its
resources), so a
Hi, Cameron,
Sorry to chime in late. Overall, looks great! I do have a few
suggestions/questions before I can cast my vote here:
a) for the configuration variable names, why are we limiting ourselves to
yarn.resource.*? We have changed some of the configuration variables from
yarn specific to
+1.
Thanks for driving this effort.
Best,
Ke
> On Mar 3, 2020, at 6:28 PM, Jagadish Venkatraman
> wrote:
>
> +1 binding.
>
> Thanks Cameron. I look forward to this feature taking our "Stream
> Processing as a service" offering to the next level.
>
> Cheers
>
> On Tuesday, March 3, 2020,
+1 binding.
Thanks Cameron. I look forward to this feature taking our "Stream
Processing as a service" offering to the next level.
Cheers
On Tuesday, March 3, 2020, Prateek Maheshwari wrote:
> +1 (binding) from me. Thanks for contributing this feature. Looking forward
> to having dependency
+1 (binding) from me. Thanks for contributing this feature. Looking forward
to having dependency isolation and to the ability to upgrade the framework
independently from an application.
Thanks,
Prateek
On Fri, Feb 28, 2020 at 10:48 AM Cameron Lee
wrote:
> Hi all,
>
> This is a call for a vote
10 matches
Mail list logo
