Re: Looking for YARN Alternative

The standalone mode was introduced for this exact reason for customers who don’t want to run YARN. Have you considered running Samza in stand-alone mode? In this mode, Samza is an embedded library - very similar to Kafka Streams.

Looking for YARN Alternative

We are finding YARN and AWS Ec2 to be too costly for us. We are having to scale the cluster to support more jobs and have plans to write more jobs. We are scaling because cluster doesn’t have enough VCores to support all the Containers, not enough RAM for jobs, etc. Has anyone had luck running

[GitHub] cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer.

cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer. URL: https://github.com/apache/samza/pull/918#discussion_r258302217 ## File path:

[GitHub] cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer.

cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer. URL: https://github.com/apache/samza/pull/918#discussion_r258283701 ## File path:

[GitHub] cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer.

cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer. URL: https://github.com/apache/samza/pull/918#discussion_r258284619 ## File path:

[GitHub] shanthoosh commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

shanthoosh commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor URL: https://github.com/apache/samza/pull/920#discussion_r258306761 ## File path:

[GitHub] shanthoosh commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

shanthoosh commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor URL: https://github.com/apache/samza/pull/920#discussion_r258307442 ## File path: samza-core/src/main/java/org/apache/samza/execution/JobPlanner.java ## @@ -155,4 +158,33

[GitHub] Sanil15 commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

Sanil15 commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor URL: https://github.com/apache/samza/pull/920#discussion_r258288145 ## File path: samza-core/src/main/java/org/apache/samza/execution/JobPlanner.java ## @@ -155,4 +158,33 @@

[GitHub] cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer.

cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer. URL: https://github.com/apache/samza/pull/918#discussion_r258295838 ## File path:

[GitHub] cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer.

cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer. URL: https://github.com/apache/samza/pull/918#discussion_r258286825 ## File path:

[GitHub] cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer.

cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer. URL: https://github.com/apache/samza/pull/918#discussion_r258281900 ## File path: samza-core/src/main/java/org/apache/samza/system/SystemAdmins.java

[GitHub] shanthoosh commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

shanthoosh commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor URL: https://github.com/apache/samza/pull/920#discussion_r258307896 ## File path: samza-test/src/main/java/org/apache/samza/test/framework/TestRunner.java ## @@ -104,7

[GitHub] shanthoosh commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

shanthoosh commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor URL: https://github.com/apache/samza/pull/920#discussion_r258308009 ## File path: samza-core/src/test/java/org/apache/samza/runtime/TestLocalApplicationRunner.java ## @@

[GitHub] vjagadish1989 commented on issue #925: SAMZA-2109: Reduce per-partition default buffer sizes

vjagadish1989 commented on issue #925: SAMZA-2109: Reduce per-partition default buffer sizes URL: https://github.com/apache/samza/pull/925#issuecomment-465323526 Since, hot-fixes are reserved for critical bug fixes - probably not worth back-porting new features/enhancements. I'd wait for

[GitHub] asfgit closed pull request #925: SAMZA-2109: Reduce per-partition default buffer sizes

asfgit closed pull request #925: SAMZA-2109: Reduce per-partition default buffer sizes URL: https://github.com/apache/samza/pull/925 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258138159 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258147011 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258142667 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258171369 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258231454 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258148518 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258224159 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258149441 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258169902 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258228794 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258167734 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258248810 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258221771 ## File path:

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r258161946 ## File path:

[GitHub] rmatharu commented on a change in pull request #924: SAMZA-2108: Check for host affinity config before resolving preferred host matching

rmatharu commented on a change in pull request #924: SAMZA-2108: Check for host affinity config before resolving preferred host matching URL: https://github.com/apache/samza/pull/924#discussion_r258203228 ## File path:

[GitHub] vjagadish1989 opened a new pull request #925: SAMZA-2109: Reduce per-partition default buffer sizes

vjagadish1989 opened a new pull request #925: SAMZA-2109: Reduce per-partition default buffer sizes URL: https://github.com/apache/samza/pull/925 This is an automated message from the Apache Git Service. To respond to the

[GitHub] shanthoosh commented on issue #925: SAMZA-2109: Reduce per-partition default buffer sizes

shanthoosh commented on issue #925: SAMZA-2109: Reduce per-partition default buffer sizes URL: https://github.com/apache/samza/pull/925#issuecomment-465307020 Is it possible to back-port this patch to samza internal releases done at linkedin?

[GitHub] lhaiesp commented on issue #923: Adding a bound on EventHubSystemConsumer's initalizeEventHubsManagers

lhaiesp commented on issue #923: Adding a bound on EventHubSystemConsumer's initalizeEventHubsManagers URL: https://github.com/apache/samza/pull/923#issuecomment-465251446 Looks good This is an automated message from the

[GitHub] rmatharu commented on a change in pull request #924: SAMZA-2108: Check for host affinity config before resolving preferred host matching

rmatharu commented on a change in pull request #924: SAMZA-2108: Check for host affinity config before resolving preferred host matching URL: https://github.com/apache/samza/pull/924#discussion_r258203228 ## File path:

[GitHub] dxichen opened a new pull request #924: SAMZA-2108: Check for host affinity config before resolving preferred host matching

dxichen opened a new pull request #924: SAMZA-2108: Check for host affinity config before resolving preferred host matching URL: https://github.com/apache/samza/pull/924 @vjagadish @rmatharu Added check in the ContainerAllocator to check the host-affinity config before using the

[GitHub] shanthoosh commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer.

shanthoosh commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer. URL: https://github.com/apache/samza/pull/918#discussion_r257523191 ## File path:

[GitHub] shanthoosh commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

shanthoosh commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor URL: https://github.com/apache/samza/pull/920#discussion_r258141427 ## File path: samza-core/src/main/java/org/apache/samza/execution/JobPlanner.java ## @@ -155,4 +158,33