[RESULT][VOTE] Apache Samza 1.6.0 RC1

[Boris S]

The vote for 1.6.0 release has been out for more than 72 hours and we got
+1 (binding) x3 (Yi, Bharath, Boris)
+1(non-binding) x1 (Stuart Perks)

Samza 1.6.0 officially passed the VOTE!

Thanks for your contribution to the release and help with validation.

--
Boris

Re: [VOTE] Apache Samza 1.6.0 RC1

[Boris S]

Quick reminder,
Please take some time to validate the release and vote on it.

Thanks,
Boris.

On Wed, Jan 6, 2021 at 11:13 PM Boris S  wrote:

> Hi all,
>
> This is a call for a vote on a release of Apache Samza 1.6.0. We are
> excited to see some new features and improvements in this release.
>
> The release candidate can be downloaded from here:
> http://people.apache.org/~boryas/samza-1.6.0-rc1/
> <http://people.apache.org/~boryas/samza-1.6.0-rc0/>
>
> The release candidate is signed with pgp key D2103453, which is
> included in the repository's KEYS file:
> https://github.com/apache/samza/blob/master/KEYS
>
> or to directly see the public key here:
>
> https://keyserver.ubuntu.com/pks/lookup?search=Boris+Shkolnik=on=index
>
> The git tag is release-1.6.0-rc1 and signed with the same pgp key above:
>
> https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.6.0-rc1
> <https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.6.0-rc0>
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1088/
> <https://repository.apache.org/content/repositories/orgapachesamza-1080/>
> and
> https://repository.apache.org/content/repositories/orgapachesamza-1089/
> <https://repository.apache.org/content/repositories/orgapachesamza-1080/>
> (for Scala 2.12)
>
> The vote will be open for 72 hours (end at 06:00pm, Mon 1/11/2021).
> Please download the release candidate, check the hashes/signature, build it
> and test it, and vote:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> I ran check-all.sh and integration tests (both YARN and standalone) passed.
>
> +1 on my end for the release.
>
> Thanks,
> Boris
>

[VOTE] Apache Samza 1.6.0 RC1

[Boris S]

Hi all,

This is a call for a vote on a release of Apache Samza 1.6.0. We are
excited to see some new features and improvements in this release.

The release candidate can be downloaded from here:
http://people.apache.org/~boryas/samza-1.6.0-rc1/


The release candidate is signed with pgp key D2103453, which is
included in the repository's KEYS file:
https://github.com/apache/samza/blob/master/KEYS

or to directly see the public key here:
https://keyserver.ubuntu.com/pks/lookup?search=Boris+Shkolnik=on=index

The git tag is release-1.6.0-rc1 and signed with the same pgp key above:
https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.6.0-rc1


Test binaries have been published to Maven's staging repository, and are
available here:
https://repository.apache.org/content/repositories/orgapachesamza-1088/

and
https://repository.apache.org/content/repositories/orgapachesamza-1089/

(for Scala 2.12)

The vote will be open for 72 hours (end at 06:00pm, Mon 1/11/2021).
Please download the release candidate, check the hashes/signature, build it
and test it, and vote:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

I ran check-all.sh and integration tests (both YARN and standalone) passed.

+1 on my end for the release.

Thanks,
Boris

Re: [VOTE] Apache Samza 1.6.0 RC0

[Boris S]

Hi all,
We found an issue with one of the fixes in the current release.
This Release candidate is canceled.
We will send another release candidate shortly.

Boris.

On Wed, Dec 9, 2020 at 6:04 PM Boris S  wrote:

> Hi all,
>
> This is a call for a vote on a release of Apache Samza 1.6.0. We are
> excited to see some new features and improvements in this release.
>
> The release candidate can be downloaded from here:
> http://people.apache.org/~boryas/samza-1.6.0-rc0/
>
> The release candidate is signed with pgp key D2103453, which is
> included in the repository's KEYS file:
> https://github.com/apache/samza/blob/master/KEYS
>
> or to directly see the public key here:
>
> https://keyserver.ubuntu.com/pks/lookup?search=Boris+Shkolnik=on=index
>
> The git tag is release-1.6.0-rc0 and signed with the same pgp key above:
>
> https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.6.0-rc0
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1086/
> <https://repository.apache.org/content/repositories/orgapachesamza-1080/>
> and
> https://repository.apache.org/content/repositories/orgapachesamza-1086/
> <https://repository.apache.org/content/repositories/orgapachesamza-1080/>
> (for Scala 2.12)
>
> The vote will be open for 72 hours (end at 06:00pm, Fri 12/11/2020).
> Please download the release candidate, check the hashes/signature, build it
> and test it, and vote:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> I ran check-all.sh and integration tests (both YARN and standalone) passed.
>
> +1 on my end for the release.
>
> Thanks,
> Boris
>

Re: [VOTE] Apache Samza 1.6.0 RC0

[Boris S]

Hi all,
A quick reminder to vote on the 1.6 release for Samza.
There are a lot of fixes and improvements in this release and it should be
available for all to use.

Thanks,
Boris.

On Wed, Dec 9, 2020 at 6:04 PM Boris S  wrote:

> Hi all,
>
> This is a call for a vote on a release of Apache Samza 1.6.0. We are
> excited to see some new features and improvements in this release.
>
> The release candidate can be downloaded from here:
> http://people.apache.org/~boryas/samza-1.6.0-rc0/
>
> The release candidate is signed with pgp key D2103453, which is
> included in the repository's KEYS file:
> https://github.com/apache/samza/blob/master/KEYS
>
> or to directly see the public key here:
>
> https://keyserver.ubuntu.com/pks/lookup?search=Boris+Shkolnik=on=index
>
> The git tag is release-1.6.0-rc0 and signed with the same pgp key above:
>
> https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.6.0-rc0
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1086/
> <https://repository.apache.org/content/repositories/orgapachesamza-1080/>
> and
> https://repository.apache.org/content/repositories/orgapachesamza-1086/
> <https://repository.apache.org/content/repositories/orgapachesamza-1080/>
> (for Scala 2.12)
>
> The vote will be open for 72 hours (end at 06:00pm, Fri 12/11/2020).
> Please download the release candidate, check the hashes/signature, build it
> and test it, and vote:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> I ran check-all.sh and integration tests (both YARN and standalone) passed.
>
> +1 on my end for the release.
>
> Thanks,
> Boris
>

[VOTE] Apache Samza 1.6.0 RC0

[Boris S]

Hi all,

This is a call for a vote on a release of Apache Samza 1.6.0. We are
excited to see some new features and improvements in this release.

The release candidate can be downloaded from here:
http://people.apache.org/~boryas/samza-1.6.0-rc0/

The release candidate is signed with pgp key D2103453, which is
included in the repository's KEYS file:
https://github.com/apache/samza/blob/master/KEYS

or to directly see the public key here:
https://keyserver.ubuntu.com/pks/lookup?search=Boris+Shkolnik=on=index

The git tag is release-1.6.0-rc0 and signed with the same pgp key above:
https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.6.0-rc0

Test binaries have been published to Maven's staging repository, and are
available here:
https://repository.apache.org/content/repositories/orgapachesamza-1086/

and
https://repository.apache.org/content/repositories/orgapachesamza-1086/

(for Scala 2.12)

The vote will be open for 72 hours (end at 06:00pm, Fri 12/11/2020).
Please download the release candidate, check the hashes/signature, build it
and test it, and vote:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

I ran check-all.sh and integration tests (both YARN and standalone) passed.

+1 on my end for the release.

Thanks,
Boris

[DISCUSS] Apache Samza 1.6.0 RC0

[Boris S]

** with correct subject this time**

Hi all,

We have added a number of major features and changes to master since
1.5, that warrants a major 1.6 release.

Within LinkedIn, some of these features have already been tested as
part of our test suites. We plan to continue our testing the coming
weeks to validate the stability prior to release.


We wanted to kick off the discussion in the open source forum to keep
the momentum flowing.


Here is a selected list of features that are part of the new release:
  SAMZA-2600: Extract constants for string literals used in AM and
container (#1439)

SAMZA-2596: Replace String.format() calls to avoid
MissingFormatArgumentException

SAMZA-2595: Updated MonitorService to use separate
ScheduleExecutor for each monitor (#1434)

SAMZA-2587: IntermediateMessageSerde exception handling (#1426)

SAMZA-2593: Update task callback to store only necessary fields
instead of the message envelope (#1433)

SAMZA-2574 : improve flexibility of SystemFactory interface

SAMZA-2589: Consolidate Beam and High/Low Samza Apps launch workflow (#1428)

SAMZA-2558: Refactor app.runner.class

SAMZA-2424: AM should cache and serve serialized Job Model to
containers (#1241)

SAMZA-2584: Refactor ClusterBasedJobCoordinator (#1424)

SAMZA-2585: Modify shutdown sequence to handle orphaned AMs (#1422)

SAMZA-2439: Remove LocalityManager and container location
information from JobModel (#1421)

SAMZA-2579: Force restart feature for Container Placements (#1414)


You can find a concrete list of the features, bug-fixes, upgrades here
https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20and%20fixVersion%20%3D%201.6






Here is my proposal on our release schedule and timelines.

   1. Cut the 1.6.0 release branch.

   2. Target a release vote on the week of Nov 23rd.

--

Thanks

Boris

[DISCUSS] Apache Samza 1.5.0 RC0

[Boris S]

Hi all,

We have added a number of major features and changes to master since
1.5, that warrants a major 1.6 release.

Within LinkedIn, some of these features have already been tested as
part of our test suites. We plan to continue our testing the coming
weeks to validate the stability prior to release.


We wanted to kick off the discussion in the open source forum to keep
the momentum flowing.


Here is a selected list of features that are part of the new release:
  SAMZA-2600: Extract constants for string literals used in AM and
container (#1439)

SAMZA-2596: Replace String.format() calls to avoid
MissingFormatArgumentException

SAMZA-2595: Updated MonitorService to use separate
ScheduleExecutor for each monitor (#1434)

SAMZA-2587: IntermediateMessageSerde exception handling (#1426)

SAMZA-2593: Update task callback to store only necessary fields
instead of the message envelope (#1433)

SAMZA-2574 : improve flexibility of SystemFactory interface

SAMZA-2589: Consolidate Beam and High/Low Samza Apps launch workflow (#1428)

SAMZA-2558: Refactor app.runner.class

SAMZA-2424: AM should cache and serve serialized Job Model to
containers (#1241)

SAMZA-2584: Refactor ClusterBasedJobCoordinator (#1424)

SAMZA-2585: Modify shutdown sequence to handle orphaned AMs (#1422)

SAMZA-2439: Remove LocalityManager and container location
information from JobModel (#1421)

SAMZA-2579: Force restart feature for Container Placements (#1414)


You can find a concrete list of the features, bug-fixes, upgrades here
https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20and%20fixVersion%20%3D%201.6






Here is my proposal on our release schedule and timelines.

   1. Cut the 1.6.0 release branch.

   2. Target a release vote on the week of Nov 23rd.

--

Thanks

Boris

Re: [VOTE] Apache Samza 1.5.1 RC0

[Boris S]

I've ran check-all.sh
and validated all the signatures and hashes.
All passed.
+1.

On Sat, Aug 22, 2020 at 8:09 PM Bharath Kumara Subramanian <
codin.mart...@gmail.com> wrote:

> Hi all,
>
> This is a call for a vote on a release of Apache Samza 1.5.1. We are
> releasing 1.5.1 to address a critical bug related transaction state
> feature.
>
> More details on the bug can be found here:
> https://issues.apache.org/jira/browse/SAMZA-2578
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~bharathkk/samza-1.5.1-rc0/
>
> The release candidate is signed with pgp key F3B965A6B192DAB7, which is
> included in the repository's KEYS file:
> https://github.com/apache/samza/blob/master/KEYS
>
> or to directly see the public key here:
>
> https://keyserver.ubuntu.com/pks/lookup?search=Bharath+Kumarasubramanian=on=index
>
> The git tag is release-1.5.1-rc0 and signed with the same pgp key above:
>
> https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.5.1-rc0
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1081
>
> The vote will be open for 72 hours (end at 07:15pm Wednesday, 08/26/2020).
> Please download the release candidate, check the hashes/signature, build it
> and test it, and vote:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> I ran check-all.sh and integration tests (both YARN and standalone) passed.
>
> +1 on my end for the release.
>
> Thanks,
> Bharath
>

Re: [DISCUSS] Samza 1.5.1 release

[Boris S]

+1

On Tue, Aug 18, 2020 at 9:02 PM Bharath Kumara Subramanian <
codin.mart...@gmail.com> wrote:

> Hi all,
>
> In 1.5 release, we enabled transactional state by default for all samza
> jobs. We identified a critical bug related to trimming the state which
> requires a minor release.
>
> I wanted to kick off the discussion on the open source forum as the bug fix
> has been validated internally at LinkedIn.
>
> More details on the bug can be found in SAMZA-2578
> .
> The patch that contains the fix: samza/pull/1413
> 
>
> I'd like to target early next week for voting.
>
> Cheers,
> Bharath
>

Re: [VOTE] Apache Samza 1.5.0 RC1

[Boris S]

+1 (binding)

   - verified signatures
   - ran check-all
   - ran integration tests.


On Mon, Jun 8, 2020 at 5:16 PM Bharath Kumara Subramanian <
codin.mart...@gmail.com> wrote:

> Hi all,
>
> This is a call for a vote on a release of Apache Samza 1.5.0. We are
> excited to see some new features and improvements in this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~bharathkk/samza-1.5.0-rc1/
>
> The release candidate is signed with pgp key F3B965A6B192DAB7, which is
> included in the repository's KEYS file:
> https://github.com/apache/samza/blob/master/KEYS
>
> or to directly see the public key here:
>
> https://keyserver.ubuntu.com/pks/lookup?search=Bharath+Kumarasubramanian=on=index
>
> The git tag is release-1.5.0-rc0 and signed with the same pgp key above:
>
> https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.5.0-rc1
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1080/
>
> The vote will be open for 72 hours (end at 05:15pm Tuesday, 06/11/2021).
> Please download the release candidate, check the hashes/signature, build it
> and test it, and vote:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> I ran check-all.sh and integration tests (both YARN and standalone) passed.
>
> +1 on my end for the release.
>
> Thanks,
> Bharath
>

Re: [VOTE] SEP-22: Container Placements in Samza

[Boris S]

Hi,
LGTM.
+1.

On Wed, May 27, 2020 at 11:28 AM Sanil Jain  wrote:

> Hi all,
>
> This is a call for a vote on SEP-22: Container Placements in Samza
>
> Thanks to everyone who reviewed the proposal and
> provided feedback.
>
> I have addressed comments on the SEP, and I am not aware of any further
> major questions or objections, so I am starting this vote.
>
> *SEP link: *
>
> https://cwiki.apache.org/confluence/display/SAMZA/SEP-22%3A+Container+Placements+in+Samza
>
>
> *Discuss thread:*
>
>
> http://mail-archives.apache.org/mod_mbox/samza-dev/202001.mbox/%3CCAKkRg%3D94NY8cLn89u%3DVeL1K52R3XuOimzxXsy7BLzS7fpS%3DLfg%40mail.gmail.com%3E
>
> There was also some discussion through comments on the SEP page (see
> Resolved Comments).
>
> Please vote:
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>

Re: [VOTE] Apache Samza 1.4.0 RC1

[Boris S]

Tests and integration tests (on linux) passed.
+1 (binding)

On Wed, Mar 11, 2020 at 3:28 PM Bharath Kumara Subramanian <
codin.mart...@gmail.com> wrote:

> +1(binding).
>
> Check-all.sh and the integration tests passed.
>
> Thanks,
> Bharath
>
> On Tue, Mar 10, 2020 at 3:35 PM Xinyu Liu  wrote:
>
> > +1 (binding).
> >
> > Run check-all.sh and integration tests for both yarn and standalone. All
> > passed.
> >
> > Thanks,
> > Xinyu
> >
> >
> > On Fri, Mar 6, 2020 at 6:46 PM Yi Pan  wrote:
> >
> > > Have downloaded the files, build with check-all.sh, and ran both YARN
> and
> > > standalone integration tests. All passed.
> > >
> > > +1 (binding).
> > >
> > > Thanks!
> > >
> > > -Yi
> > >
> > > On Tue, Mar 3, 2020 at 3:03 PM Cameron Lee 
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > This is a call for a vote on a release of Apache Samza 1.4.0. Thanks
> to
> > > > everyone who has contributed to this release.
> > > >
> > > > The release candidate can be downloaded from here:
> > > > https://home.apache.org/~cameronlee/samza-1.4.0-rc1/
> > > >
> > > > The release candidate is signed with pgp key 0x54CB3CE3, which can be
> > > found
> > > > here:
> > > >
> > > >
> > >
> >
> https://keyserver.ubuntu.com/pks/lookup?search=0x54CB3CE3=on=index
> > > > or to directly see the public key here:
> > > >
> > > >
> > >
> >
> https://keyserver.ubuntu.com/pks/lookup?op=get=0x71b0145290ecdbfa5caea6dbd786a7ba54cb3ce3
> > > >
> > > > The git tag is release-1.4.0-rc1, signed by the same pgp key above:
> > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=samza.git;a=commit;h=5327fafb8502b126482ec0c4efc8d1aa9b96ba44
> > > >
> > > > Test binaries have been published to Maven's staging repository, and
> > are
> > > > available here:
> > > >
> https://repository.apache.org/content/repositories/orgapachesamza-1077
> > > >
> > > > The vote will be open for 72 hours (until Friday, March 6, 2020 at
> 3pm
> > > > PST).
> > > >
> > > > Please download the release candidate, check the hashes/signature,
> > build
> > > it
> > > > and test it, and then please vote:
> > > > [ ] +1 approve
> > > > [ ] +0 no opinion
> > > > [ ] -1 disapprove (and reason why)
> > > >
> > > > I ran check-all.sh and integration tests.
> > > >
> > > > +1 (non-binding) from my side.
> > > >
> > > > Thank you,
> > > > Cameron
> > > >
> > >
> >
>

Re: [VOTE] Apache Samza 1.4.0 RC0

[Boris S]

Hi,
Currently it doesn't build if one to follow README file.
We discussed it with Cameron, and the reason is that running 'gradle -b
bootstrap.gradle'
brings in gradle5.*, while the gradle files we have are for 4.*
Cameron is looking into fixing this situation.
 -1

On Fri, Feb 28, 2020 at 3:59 PM Cameron Lee  wrote:

> Hi all,
>
> This is a call for a vote on a release of Apache Samza 1.4.0. Thanks to
> everyone who has contributed to this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~cameronlee/samza-1.4.0-rc0/
>
> The release candidate is signed with pgp key 0x54CB3CE3, which can be found
> here:
>
> https://keyserver.ubuntu.com/pks/lookup?search=0x54CB3CE3=on=index
> or to directly see the public key here:
>
> https://keyserver.ubuntu.com/pks/lookup?op=get=0x71b0145290ecdbfa5caea6dbd786a7ba54cb3ce3
>
> The git tag is release-1.4.0-rc0, signed by the same pgp key above:
>
> https://gitbox.apache.org/repos/asf?p=samza.git;a=commit;h=6d411a54666e9132f4cf501445bf7dd17c2e81b4
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1075
>
> The vote will be open until Friday, March 6, 2020 at 10am PST.
>
> Please download the release candidate, check the hashes/signature, build it
> and test it, and then please vote:
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> I ran check-all.sh and integration tests.
>
> +1 (non-binding) from my side.
>
> Thank you,
> Cameron
>

Re: MessageStream with multiple concurrent operation stacks

[Boris S]

Yes, to answer your second question , you should be able to fork the
streams. Message Streams are idempotent and can be used in different
pipelines.

With regards to group id, Samza does set the group ids, but, I think, they
are the same for the whole job. The idea behind it is, that Samza is not
using Kafka level partitioning. Samza itself distributes the partitions
directly to its tasks.

Boris.

On Tue, Oct 22, 2019 at 11:55 AM Bharath Kumara Subramanian <
codin.mart...@gmail.com> wrote:

> Hi Eric,
>
> Based on the source code, it appears that each job designates a unique
> > group id when subscribing to kafka topic, is my understanding correct?
> >
>
> Yes. Samza uses a combination of job name and job id to generate the group
> id.
>
>
> > is it possible to have 2 independent stack
>
> of operations applied on the same InputStream?
>
>
> Yes. The code snippet provided in the email should work as expected.
>
> Hope that helps.
>
> Thanks,
> Bharath
>
> On Fri, Oct 18, 2019 at 5:55 AM Eric Shieh  wrote:
>
> > Hi,
> >
> > Based on the source code, it appears that each job designates a unique
> > group id when subscribing to kafka topic, is my understanding correct?
> If
> > so, since one cannot call appDescriptor.getInputStream with the
> > same KafkaInputDescriptor twice, is it possible to have 2 independent
> stack
> > of operations applied on the same InputStream?  In essence, I have a
> > requirement to process a message from 1 InputStream and write to 2
> > OutputStreams or sinks after 2 different independent stacks of operations
> > applied.  One way to solve this is to deploy 2 independent jobs but the
> > downside of it is it would be difficult to synchronize the 2 jobs.  Is it
> > possible to do the following:
> >
> > MessageStream ms = appDescriptor.getInputStream(kid);
> > MessageStream msForkPoint = ms.map(mapping_logic1);
> > msForkPoint.filter(filter_logic_1).sendTo(outputSream1);
> > msForkPoint.map(mapping_logic2).sink(write_to_DB);
> >
> > Based on the source code, each operation instantiates a new instance of
> > MessageStream and registers the new StreamOperatorSpec with the previous
> > MessageStream instance's StreamOperatorSpec essentially forming a "linked
> > list" of parent-child StreamOperatorSpecs.  Since each parent
> OperatorSpec
> > maintains a LinkedHashSet of next OperatorSpecs, the above code of
> forking
> > 2 independent operation stacks after the initial map seems to be
> feasible.
> >
> > Regards,
> >
> > Eric
> >
>

Canceled. [VOTE] Apache Samza 1.2.0 RC0

[Boris S]

-- Forwarded message -
From: Bharath Kumara Subramanian 
Date: Wed, May 22, 2019 at 12:53 PM
Subject: Re: [VOTE] Apache Samza 1.2.0 RC0
To: 


-1 disapprove.

We have a bug fix  for
Async API.

The fix has been committed to master and we need to include it in 1.2.
The commit hash c087f8
<
https://github.com/apache/samza/commit/c087f80cfe2ae576f30585894a5d907f7c5b2cee
>
to cherry pick.

Thanks,
Bharath

On Wed, May 22, 2019 at 11:03 AM Boris Shkolnik  wrote:

> Hi,
>
> This is a call for a vote on a release of Apache Samza 1.2.0. Thanks to
> everyone who has contributed to this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~boryas/samza-1.2.0-rc0/
>
> The release candidate is signed with pgp key 0x7D74D0CD5B5EB041, which can
> be found
> http://keyserver.ubuntu.com/pks/lookup?op=get=0x7d74d0cd5b5eb041
> 
> The git tag is release-1.2.0-rc0 and signed with the same pgp key:
>
>
https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.2.0-rc
> <
>
https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.1.0-rc1
> >
> 0
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1065
> <
>
https://repository.apache.org/content/repositories/orgapachesamza-1065/org/
> >
>
> The vote will be open for 56 hours (ending at 06:00 PM PST Friday,
> 05/24/2019).
>
> Please download the release candidate, check the hashes/signature, build
it
> and test it, and then please vote:
>
> [ ] +1 approve
>
> [ ] +0 no opinion
>
> [ ] -1 disapprove (and reason why)
>
> I ran check-all.sh and integration tests.
>
> +1 (non-binding) from my side.
>
> Thanks,
>

Re: [DISCUSS] 1.2 release

[Boris S]

@jmakes, Hmm, the git log actually shows a commit related to this JIRA, so
there is a code change associated with this ticket. And the ticket is
resolved as FIXED. I can remove it the tag from the jira, but I cannot
remove it from the git log.

On Mon, May 20, 2019 at 9:39 AM Jake Maes  wrote:

> I don't think we did anything for "Making sendTo(table), sendTo(stream)
> non-terminal". The ticket was just closed as a "won't fix" IIRC.
>
> Nevertheless, I think the Kafka 2.0 upgrade warrants a release by itself.
>
> Let's do it.
>
> On Fri, May 17, 2019 at 12:17 PM Boris S  wrote:
>
> > Hi all,
> >
> > We have added a number of major features and changes to master since
> > 1.1 that warrants a new 1.2 release.
> >
> > Within LinkedIn, some of these features have already been tested as
> > part of our test suites. We plan to continue our testing in coming
> > week to validate the stability prior to release.
> >
> > We wanted to kick off the discussion in the open source forum to keep
> > the momentum flowing.
> > Here is a selected list of features that are part of the new release
> >
> >   Kafka 2.0 upgrade
> >
> >   Couchbase support for Samza Table API
> >   Making sendTo(table), sendTo(stream) non-terminal
> >
> > We have also worked on the following upgrades and bugfixes.
> > You can find a concrete list of the features, bug-fixes, upgrades
> > herehttps://
> >
> issues.apache.org/jira/issues/?jql=project%20%3D%20%22SAMZA%22%20and%20fixVersion%20in%20(1.2)
> >
> >
> > Some of these Jiras are not marked as fixed (but they are marked as
> > committed in the git log). Please close the Jiras is they are fixed.
> >
> > Here is my proposal on our release schedule and timelines.
> >1. Cut the 1.2 release branch.
> >2. Target a release vote on the week of May 20, 2019
> >
> >
> > Thanks
> > Boris
> >
>

[DISCUSS] 1.2 release

[Boris S]

Hi all,

We have added a number of major features and changes to master since
1.1 that warrants a new 1.2 release.

Within LinkedIn, some of these features have already been tested as
part of our test suites. We plan to continue our testing in coming
week to validate the stability prior to release.

We wanted to kick off the discussion in the open source forum to keep
the momentum flowing.
Here is a selected list of features that are part of the new release

  Kafka 2.0 upgrade

  Couchbase support for Samza Table API
  Making sendTo(table), sendTo(stream) non-terminal

We have also worked on the following upgrades and bugfixes.
You can find a concrete list of the features, bug-fixes, upgrades
herehttps://issues.apache.org/jira/issues/?jql=project%20%3D%20%22SAMZA%22%20and%20fixVersion%20in%20(1.2)


Some of these Jiras are not marked as fixed (but they are marked as
committed in the git log). Please close the Jiras is they are fixed.

Here is my proposal on our release schedule and timelines.
   1. Cut the 1.2 release branch.
   2. Target a release vote on the week of May 20, 2019


Thanks
Boris

Re: [VOTE] Apache Samza 1.1.0 RC2

[Boris S]

Sorry for the late reply. I've ran into a problem with the performance
tests:
./gradlew samza-shell:kvPerformanceTest
-PconfigPath=file://$PWD/samza-test/src/main/config/perf/kv-perf.properties

I think it is not a code problem, but a test problem, specifically test
configuration problem.

On Mon, Mar 18, 2019 at 5:10 PM santhosh venkat <
santhoshvenkat1...@gmail.com> wrote:

> Hi,
>
> The vote of Samza 1.1.0 has been open for more than 72 hours. We got +1
> (binding)x 3 and +1 (non-binding) x 3 and no vetos.
>
> *Binding +1: Prateek M, Jagadish V, Jake Maes*
> *Non-binding +1: Rayman P, Daniel C, Shanthoosh V*
>
> Thanks everyone for helping validate the release. Samza 1.1.0 has
> officially passed the VOTE.
>
> Thanks,
> Shanthoosh
>
>
> On Mon, Mar 18, 2019 at 4:32 PM Prateek Maheshwari 
> wrote:
>
> > 1. Verified checksum and signatures for the binaries.
> > 2. Ran ./check-all.sh
> > 3. Ran YARN and Standalone integration tests with the config patch
> > successfully.
> >
> > +1(binding) from my side as well.
> >
> > Thanks,
> > Prateek
> >
> > On Mon, Mar 18, 2019 at 2:06 PM Jagadish Venkatraman <
> > jagadish1...@gmail.com>
> > wrote:
> >
> > > 1. Verified check-sum and signatures for the release binaries.
> > > 2. Ran ./check-all.sh successfully
> > > 3. Ran YARN integration tests successfully
> > > 4. Encountered an error on the standalone integration test, but it
> > > succeeded after setting Kafka's replication factor config to 1.
> > >
> > > +1(binding) from my side.
> > >
> > > Thanks Daniel Chen and Shanthoosh for shepherding Samza 1.0.1!
> > >
> > > On Mon, Mar 18, 2019 at 9:47 AM Jake Maes  wrote:
> > >
> > > > Verified with check-all on RHEL 7
> > > >
> > > > Verified pgp and sha.
> > > >
> > > > +1 (binding)
> > > >
> > > > On Fri, Mar 15, 2019 at 11:39 AM rayman preet 
> > > > wrote:
> > > >
> > > > > +1 (Non-binding)
> > > > >
> > > > > --
> > > > > thanks
> > > > > rayman
> > > > >
> > > > > On Wed, Mar 13, 2019 at 7:17 PM Daniel Chen 
> > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I performed the following verifications:
> > > > > >
> > > > > > 1. ./bin/check-all.sh succeeded.
> > > > > >
> > > > > > 2. Verified both ./bin/integration-tests.sh
> yarn-integration-tests
> > > and
> > > > > > ./bin/integration-tests.sh standalone-integration-tests
> succeeded.
> > > > > >
> > > > > > 3. Verified that SQL console available in samza-tool.tgz.
> > > > > >
> > > > > > +1 (Non-binding)
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Daniel
> > > > > >
> > > > > >
> > > > > > On Tue, Mar 12, 2019 at 4:11 PM santhosh venkat <
> > > > > > santhoshvenkat1...@gmail.com> wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > This is a call for a vote on a release of Apache Samza 1.1.0.
> > > Thanks
> > > > to
> > > > > > > everyone who has contributed to this release.
> > > > > > >
> > > > > > > The release candidate can be downloaded from here:
> > > > > > > http://home.apache.org/~shanthoosh/samza-1.1.0-rc2/
> > > > > > >
> > > > > > > The release candidate is signed with pgp key
> 0xF8B95961A401BF0F,
> > > > which
> > > > > > can
> > > > > > > be found
> > > > > > >
> > > > >
> > >
> http://keyserver.ubuntu.com/pks/lookup?op=get=0xF8B95961A401BF0F
> > > > > > >
> > > > > > > The git tag is release-1.1.0-rc0 and signed with the same pgp
> > key:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.1.0-rc2
> > > > > > >
> > > > > > > Test binaries have been published to Maven's staging
> repository,
> > > and
> > > > > are
> > > > > > > available here:
> > > > > > >
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachesamza-1060/
> > > > > > >
> > > > > > > The vote will be open for 72 hours (ending at 16:30 PM PST
> > > Thursday,
> > > > > > > 03/15/2018).
> > > > > > >
> > > > > > > Please download the release candidate, check the
> > hashes/signature,
> > > > > build
> > > > > > it
> > > > > > > and test it, and then please vote:
> > > > > > >
> > > > > > > [ ] +1 approve
> > > > > > >
> > > > > > > [ ] +0 no opinion
> > > > > > >
> > > > > > > [ ] -1 disapprove (and reason why)
> > > > > > >
> > > > > > > I ran check-all.sh, integration tests and verified the SQL
> > console
> > > > > > > in samza-tool tgz.
> > > > > > >
> > > > > > > +1 (non-binding) from my side.
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > thanks
> > > > > rayman
> > > > >
> > > >
> > >
> > >
> > > --
> > > Jagadish V,
> > > Graduate Student,
> > > Department of Computer Science,
> > > Stanford University
> > >
> >
>

Re: [VOTE] Migration of Samza git repo to gitbox.apache.org

[Boris S]

+1. Thanks!

On Thu, Jan 24, 2019 at 8:53 AM santhosh venkat <
santhoshvenkat1...@gmail.com> wrote:

> +1 (non-binding).
>
> On Thu, Jan 24, 2019 at 7:10 AM Jake Maes  wrote:
>
> > +1 (binding)
> >
> > On Wed, Jan 23, 2019 at 10:35 PM santhosh venkat <
> > santhoshvenkat1...@gmail.com> wrote:
> >
> > > +1 (binding).
> > >
> > > Thanks,
> > >
> > > On Wed, Jan 23, 2019 at 2:43 PM Jagadish Venkatraman <
> > > jagadish1...@gmail.com>
> > > wrote:
> > >
> > > > +1 (binding). Thank you Pawas for driving this!
> > > >
> > > > On Wed, Jan 23, 2019 at 2:40 PM Xinyu Liu 
> > wrote:
> > > >
> > > > > +1 (binding).
> > > > >
> > > > > On Wed, Jan 23, 2019 at 2:39 PM Prateek Maheshwari <
> > > prateek...@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > +1 (binding) again
> > > > > >
> > > > > > - Prateek
> > > > > >
> > > > > > On Wed, Jan 23, 2019 at 11:50 AM Pawas Chhokra <
> > pawas2...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > This is a call for a vote on migrating Samza git repo to
> > > > > > gitbox.apache.org, on
> > > > > > > 11 AM, Jan 29, 2019. As mandated by the Apache Infrastructure
> > Team,
> > > > all
> > > > > > git
> > > > > > > repositories must be migrated from git-wip-us.apache.org URL
> to
> > > > > > > gitbox.apache.org, as the old service is being decommissioned.
> > > > > > > The vote will be open for 72 hours (ending at 12:00 PM PST
> > Monday,
> > > > > > > January 28). You can vote as follows:
> > > > > > >
> > > > > > > [ ] +1 approve
> > > > > > >
> > > > > > > [ ] +0 no opinion
> > > > > > >
> > > > > > > [ ] -1 disapprove (and reason why)
> > > > > > >
> > > > > > > The vote is +1 from my side.
> > > > > > >
> > > > > > > Thanks & Regards,
> > > > > > > Pawas Chhokra
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Jagadish V,
> > > > Graduate Student,
> > > > Department of Computer Science,
> > > > Stanford University
> > > >
> > >
> >
>

Re: [VOTE] Apache Samza 1.0.0 RC4

[Boris S]

ran check-all and integration tests. All passed.
verified signatures.
+1

On Wed, Oct 31, 2018 at 7:15 PM Jagadish Venkatraman 
wrote:

> Hi all,
>
> This is a call for a vote on a release of Apache Samza 1.0.0. Thanks to
> everyone who has contributed to this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~jagadish/samza-1.0.0-rc4/
>
> The release candidate is signed with pgp key AF81FFBF, which can be found
> on keyservers:
> http://pgp.mit.edu/pks/lookup?op=get=0xAF81FFBF
>
> The git tag is release-1.0.0-rc4 and signed with the same pgp key:
>
> https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.0.0-rc4
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1055/
>
> The vote will be open for 72 hours (ending at 7:00 PM PST Saturday,
> November 3).
>
> Please download the release candidate, check the hashes/signature, build it
> and test it, and then please vote:
>
> [ ] +1 approve
>
> [ ] +0 no opinion
>
> [ ] -1 disapprove (and reason why)
>
> For me, I ran check-all.sh, integration tests and verified the SQL console
> in samza-tool tgz. So +1 (binding) from my side.
>
> Thanks,
> Jagadish
>
> --
> Jagadish V
>

Re: [VOTE] Apache Samza 1.0.0 RC2

[Boris S]

Ran build, test and integration test on Linux.
Verified the signatures.

+1

On Tue, Oct 23, 2018 at 11:55 AM Prateek Maheshwari 
wrote:

> Hi all,
>
> This is a call for a vote on a release of Apache Samza 1.0.0. Thanks to
> everyone who has contributed to this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~pmaheshwari/samza-1.0.0-rc2/
>
> The release candidate is signed with pgp key 6585B3D7, which can be found
> on keyservers: https://pgp.mit.edu/pks/lookup?op=get=0x6585B3D7
>
> The git tag is release-1.0.0-rc2 and signed with the same pgp key:
>
> https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.0.0-rc2
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1053/
>
> The vote will be open for 72 hours (ending at 12:00 PM PST Friday,
> 10/26/2018).
>
> Please download the release candidate, check the hashes/signature, build it
> and test it, and then please vote:
>
> [ ] +1 approve
>
> [ ] +0 no opinion
>
> [ ] -1 disapprove (and reason why)
>
> For me, I ran check-all.sh, integration tests and verified the SQL console
> in samza-tool tgz. So +1 (non-binding) from my side.
>
> Thanks,
> Prateek
>

Re: [VOTE] Apache Samza 0.14.1 RC2

[Boris S]

verified signature and md5
ran check-all.sh successfully
ran few SQL queries from the tutorial

+1 (binding)

On Fri, May 18, 2018 at 2:31 PM, Xinyu Liu  wrote:

> Hi, All,
>
> This is a call for a vote on a release of Apache Samza 0.14.1. Thanks to
> everyone who has contributed to this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~xinyu/samza-0.14.1-rc2/
>
> The release candidate is signed with pgp key C31D7061, which can be found
> on keyservers: http://pgp.mit.edu/pks/lookup?op=get=0xC31D7061
>
> The git tag is release-0.14.1-rc2 and signed with the same pgp key:
> https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=
> refs/tags/release-0.14.1-rc2
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1049/
>
> 46 issues were resolved for this release: https://issues.
> apache.org/jira/projects/SAMZA/versions/12343155
>
> The vote will be open for 72 hours (ending at 3:00PM Wednesday,
> 05/23/2018).
>
> Please download the release candidate, check the hashes/signature, build it
> and test it, and then please vote:
>
> [ ] +1 approve
>
> [ ] +0 no opinion
>
> [ ] -1 disapprove (and reason why)
>
> For me, I ran check-all.sh, integration tests and verified the SQL console
> in samza-tool tgz. So +1 (binding) from my side.
>
> Thanks,
> Xinyu
>

Re: [Discuss] Samza 0.14.1 release

[Boris S]

Sure. Let's do it.

On Wed, May 2, 2018 at 9:58 AM, Xinyu Liu  wrote:

> Thanks everyone for the discussion! I marked the Kafka 0.11 upgrade ticket
> as 0.14.1 and will include it in this release. Now we have 3 binding votes
> from Jagadish, Yi and Jake, so I will go ahead and start the release
> procedure.
>
> Thanks,
> Xinyu
>
> On Tue, May 1, 2018 at 4:31 PM, Jake Maes  wrote:
>
> > Yes, lets do it.
> >
> > On Tue, May 1, 2018 at 11:55 AM, Tom Davis 
> wrote:
> >
> > > Yeah, I'd love that!
> > >
> > >
> > > On Tue, May 1, 2018, at 1:05 PM, Yi Pan wrote:
> > > > Yes, for Kafka 0.11 change, it mainly just affects the Kafka system
> > > > factory, admin, consumer, and producer's implementation changes.
> > > > There is> no API changes that affects the user code. The current
> > version
> > > > on master> should be source-compatible w/ existing Samza applications
> > on
> > > > Samza 0.14.0> and only needs re-compile and re-package with the new
> > > Kafka system
> > > > implementations.
> > > >
> > > > @Tom, does that sounds good to you if we include that as 0.14.1?
> > > >
> > > > -Yi
> > > >
> > > > On Mon, Apr 30, 2018 at 6:09 PM, Xinyu Liu
> > > >  wrote:>
> > > >> I chatted with Yi offline and seems for 0.11.0 Kafka upgrade,
> > > >> there is no>> user code change required. The changes we are aware of
> > > are that
> > > >> some of the>> old producer configs deprecated in 0.10 are finally
> > > removed in this
> > > >> version>> (e.g. block.on.buffer.full). If there are no objections, I
> > am
> > > >> going to>> include it in this release too.
> > > >>
> > > >> Thanks,
> > > >> Xinyu
> > > >>
> > > >> On Mon, Apr 30, 2018 at 3:49 PM, Jagadish Venkatraman <
> > > >> jagadish1...@gmail.com> wrote:
> > > >>
> > > >>> Xinyu,
> > > >>> This comes up with an exciting lineup of features. +1 (binding)
> > > >>> from my>>> side.
> > > >>>
> > > >>> Tom,
> > > >>>
> > > > These all look like excellent changes to me, I'd be happy to have
> > > > more>>> stability in Standalone!
> > > >>>
> > > >>> We are actively investing in standalone and have fixed some
> > > >>> edge-cases>> that
> > > >>> surfaced when running it at scale. Thanks much for the feedback!
> > > >>>
> > > > Any timeline on getting Kafka 0.11 upgrade in?
> > > >>>
> > > >>> We have historically been doing Kafka upgrades on major releases
> > > >>> (instead>>> of
> > > >>> minor ones). So, it will likely be in 0.15.0. I'll provide an
> > > >>> update on>> the
> > > >>> timeline
> > > >>> soon.
> > > >>>
> > > >>> Best,
> > > >>> Jagadish
> > > >>>
> > > >>>
> > > >>> On Mon, Apr 30, 2018 at 3:18 PM, Tom Davis  > >>>
> > > wrote:
> > > >>>
> > >  These all look like excellent changes to me, I'd be happy to
> > >  have more stability in Standalone!
> > >  Any timeline on getting Kafka 0.11 upgrade in? It seems to
> > >  have been done a while, but waiting for 0.15 I presume.
> > >  Best,
> > > 
> > >  Tom
> > > 
> > > 
> > >  On Mon, Apr 30, 2018, at 3:10 PM, Xinyu Liu wrote:
> > > > Hi, All,
> > > >
> > > > We have been adding many improvements and critical bug fixes in
> > > > the areas> of Samza sql, standalone, eventhub system consumer
> and>>
> > > host-
> > > > affinity since> 0.14.0 release. The changes should warrant a new
> > > >> minor
> > >  release:
> > > >
> > > > SQL
> > > > - SAMZA-1681: Samza-sql: Add support for handling older record
> > > >> schema>
> > >  versions in AvroRelConverter
> > > > - SAMZA-1671: SamzaSQL: add insert into table support
> > > > - SAMZA-1651: Samza-sql: Implement GROUP BY SQL operator
> > > >
> > > > Standalone
> > > > - SAMZA-1689: Add validations before state transitions in
> > > > ZkBarrierForVersionUpgrade.
> > > > - SAMZA-1686: Set finite operation timeout when creating
> > > zkClient.> - SAMZA-1667: Skip storing configuration as a part of
> > > JobModel in> zookeeper> data nodes.
> > > > - SAMZA-1647: Fix NPE in JobModelExpired event handler.
> > > >
> > > > Eventhub
> > > > - SAMZA-1688: use per partition eventhubs client
> > > > - SAMZA-1676: miscellaneous fix and improvement for eventhubs
> > > >   system> - SAMZA-1656: EventHubSystemAdmin does not fetch
> > > metadata for
> > > >   valid> streams.>
> > > > host-affinity
> > > > - SAMZA-1687: Prioritize preferred host requests over ANY-HOST
> > > > requests> - SAMZA-1649: Improve host-aware allocation to account
> > > >> for
> > >  strict
> > > > locality>
> > > > The complete list of changes are here:
> > > > https://issues.apache.org/jira/browse/SAMZA-1624?jql=
> > >  project%20%3D%2012314526%20AND%20fixVersion%20%3D%200.
> > >  14.1%20%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC.> Most
> JIRAs
> > > in>>> the
> > 

SEP-11. Host Affinity in standalone discussion.

[Boris S]

Shanthoosh,
Thank you for suggesting and submitting this SEP:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957309

Couple of things I would want to point out so far:

   1.  Kudos on cleaning up the interface and introducing new ones
   (LocalityInfo and LocalityManager). I think we also need MetadataStorage
   one (details may be worked out later) to hide the locality storage
   implementation details.
   2. Instead of using physical hostname we should stick to the LocationId,
   since some VMs may be running multiple processors on a single physical host.
   3. Thank you for adding the diagrams. I think we can improve them little
   bit.
   - First diagram describes how local storage works. Please label it as
  such.
  - Second diagram describes the flow of JobModel generation. I am not
  sure if actual pictures help here. Consider writing it as a list.
  - Third diagram. Host affinity implementation flow. This is very
  helpful. I think, though, using function names doesn't give
enough clarity
  on what is going on. May be we should add more explanation. For example:
  group(InputSSP) -> generate list of SSPs from the list of input
  streams/partitions.
  readTaskLocalityInfo() -> read locality mapping from the
  MetaDataStorage.
   Also we should add another step there - each processor will update
  locality information based on its mapping in the current JobModel.
   4. Some time the perfect mapping to the same Locality is not possible
   (especially when a task dies and is distributed between other tasks). What
   should we do in this case?


Thanks again. I will keep reading the document.

Re: [VOTE] Apache Samza 0.14.0 RC5

[Boris S]

Verified the signature.
Ran build, tests and integration tests on Unix.
All passed (as before requires python 2.7, neither higher nor lower).

+1
Thanks guys !!

On Fri, Dec 22, 2017 at 2:50 PM, xinyu liu  wrote:

> This is a call for a vote on a release of Apache Samza 0.14.0. Thanks
> to everyone
> who has contributed to this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~xinyu/samza-0.14.0-rc5/
>
> The release candidate is signed with pgp key C31D7061, which can be found
> on
> keyservers:
> http://pgp.mit.edu/pks/lookup?op=get=0x35964389C31D7061
>
> The git tag is release-0.14.1-rc5 and signed with the same pgp key:
> https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=
> refs/tags/release-0.14.0-rc5
>
> Test binaries have been published to Maven's staging repository, and
> are available
> here:
> https://repository.apache.org/content/repositories/orgapachesamza-1042
>
> 61 issues have been resolved as part of this release
> https://issues.apache.org/jira/browse/SAMZA-1519?jql=project
> %20%3D%20SAMZA%20AND%20fixVersion%20%3D%200.14.0%20AND%
> 20status%20%3D%20Resolved
>
> The vote will be open for 72 hours (ending at 15:00 PM Thursday,
> 12/28/2017).
>
> Please download the release candidate, check the hashes/signature, build it
> and test it, and then please vote:
>
> [ ] +1 approve
>
> [ ] +0 no opinion
>
> [ ] -1 disapprove (and reason why)
>
> Thanks,
> Xinyu
>

Re: [VOTE] Apache Samza 0.14.0 RC3

[Boris S]

Verified signature.
Ran unit and integration tests.
As usual had to force python to 2.7 for integration tests to run on Linux.

+1

On Mon, Dec 18, 2017 at 11:01 AM, xinyu liu  wrote:

> Correction: This is a call for a vote on a release of Apache Samza
> *0.14.0*.
>
> On Mon, Dec 18, 2017 at 10:57 AM, xinyu liu  wrote:
>
> > This is a call for a vote on a release of Apache Samza 0.13.1. Thanks to
> everyone
> > who has contributed to this release.
> >
> > The release candidate can be downloaded from here:
> > http://home.apache.org/~xinyu/samza-0.14.0-rc3/
> >
> > The release candidate is signed with pgp key C31D7061, which can be
> found on
> > keyservers:
> > http://pgp.mit.edu/pks/lookup?op=get=0x35964389C31D7061
> >
> > The git tag is release-0.13.1-rc0 and signed with the same pgp key:
> > https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=
> > refs/tags/release-0.14.0-rc3
> >
> > Test binaries have been published to Maven's staging repository, and are
> available
> > here:
> > https://repository.apache.org/content/repositories/orgapachesamza-1036
> >
> > 61 issues have been resolved as part of this release
> > https://issues.apache.org/jira/browse/SAMZA-1519?jql=
> > project%20%3D%20SAMZA%20AND%20fixVersion%20%3D%200.14.0%
> > 20AND%20status%20%3D%20Resolved
> >
> > The vote will be open for 72 hours (ending at 11:00 AM Thursday,
> > 12/21/2017).
> >
> > Please download the release candidate, check the hashes/signature, build
> > it and test it, and then please vote:
> >
> > [ ] +1 approve
> >
> > [ ] +0 no opinion
> >
> > [ ] -1 disapprove (and reason why)
> >
> > Thanks,
> > Xinyu
> >
>

Re: [VOTE] Apache Samza 0.13.1 RC0

[Boris S]

Hi,
I've downloaded and built it on Linux.
Ran unit and integration tests. All passed.
The only issue I've encountered was a version of python.
Version below 2.5 and above 3 all failed. 2.7 was the only version that
worked.

+1 (non binding)

On Fri, Aug 18, 2017 at 11:59 AM, Fred Haifeng Ji 
wrote:

> This is a call for a vote on a release of Apache Samza 0.13.1. Thanks to
> everyone who has contributed to this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~navina/samza-0.13.1-rc0/
>
>
> The release candidate is signed with pgp key A211312E, which can be found
> on keyservers:
> http://pgp.mit.edu/pks/lookup?op=get=0xEDFD8F9AA211312E
>
>
> The git tag is release-0.13.1-rc0 and signed with the same pgp key:
> https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=
> refs/tags/release-0.13.1-rc0
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> *https://repository.apache.org/content/repositories/orgapachesamza-1030/
> *
>
>
> 29 issues were resolved for this release: https://issues.apache
> .org/jira/issues/?jql=project%20%3D%2012314526%20AND%20fixVe
> rsion%20%3D%2012340845%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
>
>
> The vote will be open for 72 hours (ending at 1:00PM Monday, 08/21/2017).
>
> Please download the release candidate, check the hashes/signature, build it
> and test it, and then please vote:
>
>
> [ ] +1 approve
>
> [ ] +0 no opinion
>
> [ ] -1 disapprove (and reason why)
>
>
> --
> Fred Ji
>

Re: [Discuss] Samza 0.13.1 release

[Boris S]

+1 for the release.

On Mon, Aug 14, 2017 at 10:38 AM, Yi Pan  wrote:

> +1 for the list! Let's proceed!
>
> On Fri, Aug 11, 2017 at 6:13 PM, Ignacio Solis  wrote:
>
> > +1
> >
> > On Fri, Aug 11, 2017 at 3:52 PM, Jacob Maes 
> wrote:
> > > Looks good!
> > >
> > > +1
> > >
> > > On Thu, Aug 10, 2017 at 6:53 PM, Jagadish Venkatraman <
> > > jagadish1...@gmail.com> wrote:
> > >
> > >> +1 for the release. thanks for the summary and for driving this Fred!
> > >>
> > >> On Thu, Aug 10, 2017 at 5:15 PM Fred Haifeng Ji  >
> > >> wrote:
> > >>
> > >> > The format was messed up when sent from my yahoo mail to
> > >> > dev@samza.apache.org. I am resending it from my gmail account.
> Sorry
> > for
> > >> > inconvenience!
> > >> >
> > >> > Hi all,
> > >> >
> > >> > There have been some new features and critical bug fixes added to
> > master
> > >> > since 0.13.0 release, which makes Samza Standalone features more
> > stable.
> > >> It
> > >> > is now good enough to warrant *a new minor release*. We will
> continue
> > to
> > >> > test for stability and performance in the next few weeks.
> > >> >
> > >> > Here are the main JIRA tickets that will be included in this release
> > (but
> > >> > not limited to):
> > >> > SAMZA-1165: Cleanup data created by ZkStandalone in ZK;
> > >> > SAMZA-1324: Add a metricsreporter lifecycle for JobCoordinator
> > component
> > >> of
> > >> > StreamProcessor;
> > >> > SAMZA-1336: Standalone session expiration propagation;
> > >> > SAMZA-1337: LocalApplicationRunner needs to support StreamTask;
> > >> > SAMZA-1339: Add standalone integration tests;
> > >> > …
> > >> >
> > >> > There are also quite a few bug fixes in 0.13.1, *please check the
> > >> complete
> > >> > list of changes in 0.13.1 here
> > >> > <
> > >> > https://issues.apache.org/jira/browse/SAMZA-1165?jql=
> > >> project%20%3D%2012314526%20AND%20fixVersion%20%3D%
> > 2012340845%20ORDER%20BY%
> > >> 20priority%20DESC%2C%20key%20ASC
> > >> > >*
> > >> > .
> > >> >
> > >> > Most JIRAs in the list have been completed and merged, with the
> > following
> > >> > one remaining, but we should try to get it completed before 0.13.1
> is
> > >> > released.
> > >> > SAMZA-1385: Coordination utils in LocalApplicationRunner uses same
> Zk
> > >> node
> > >> > as ZkJobCoordinatorFactory for leader election
> > >> >
> > >> > Here's what I propose:
> > >> > 1. Cut an 0.13.1 release branch.
> > >> > 2. Work on getting the remaining open JIRA done.
> > >> > 3. Target a release vote by Aug 18.
> > >> >
> > >> > Thoughts?
> > >> >
> > >> > Fred
> > >> >
> > >> --
> > >> Sent from my iphone.
> > >>
> >
> >
> >
> > --
> > Nacho - Ignacio Solis - iso...@igso.net
> >
>

Re: [VOTE] Apache Samza 0.13.0 RC6

[Boris S]

+1 (non-binding)
build and tested on Linux (with python 2.7; 2.4 and 3.5 - didn't work)

On Tue, Jun 6, 2017 at 2:49 PM, Jacob Maes  wrote:

> +1 (non-binding)
>
> Built and tested on both OSX and RHEL with gradle 2.0 and 2.2 respectively.
>
> Also verified the high level API + YARN host affinity on a test job with 32
> containers.
>
>
>
> On Tue, Jun 6, 2017 at 9:14 AM, xinyu liu  wrote:
>
> > +1 (non-binding).
> >
> > Downloaded the source tar, built it and run check-all.sh on REHL6 with
> > gradle 2.8. All passed.
> >
> > As a side note to Jagadish's comments, the build doesn't work on a higher
> > gradle version either (gradle 3.5). Seems "-language:implicitConversions
> > -language:reflectiveCalls" is not a valid build option anymore.
> >
> > Thanks,
> > Xinyu
> >
> > On Mon, Jun 5, 2017 at 10:06 AM, Jagadish Venkatraman <
> > jagadish1...@gmail.com> wrote:
> >
> > > Checked out, ran tests, and all of them pass.
> > >
> > > +1 (non-binding)
> > >
> > > I did get an error when running with gradle 2.4:
> > > >>Could not resolve all dependencies for configuration
> > > ':samza-kafka_2.11:compile'. > java.lang.UnsupportedOperationException
> > (no
> > > error message)
> > >
> > > However, when I used gradle 2.8, it was resolved.
> > >
> > > *gradle wrapper --gradle-version 2.8*
> > >
> > > Best,
> > > Jagadish
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Jun 5, 2017 at 8:37 AM, Jake Maes  wrote:
> > >
> > > > This is a call for a vote on a release of Apache Samza 0.13.0. Thanks
> > to
> > > > everyone who has contributed to this release. We are very glad to see
> > > some
> > > > new contributors and features in this release.
> > > >
> > > > The release candidate can be downloaded from here:
> > > > http://home.apache.org/~jmakes/samza-0.13.0-rc6/
> > > >
> > > > The release candidate is signed with pgp key 940AFC5A, which can be
> > found
> > > > on keyservers:
> > > > *http://pgp.mit.edu/pks/lookup?op=get=0x940AFC5A
> > > > *
> > > >
> > > > The git tag is release-0.13.0-rc6 and signed with the same pgp key:
> > > > https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=
> > > > refs/tags/release-0.13.0-rc6
> > > >
> > > > Test binaries have been published to Maven's staging repository, and
> > are
> > > > available here:
> > > > https://repository.apache.org/content/repositories/
> orgapachesamza-1026
> > > >
> > > > 144 issues were resolved for this release:
> > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%
> > > > 20SAMZA%20AND%20fixVersion%20in%20(0.13%2C%200.13.0)%
> > > > 20AND%20status%20in%20(
> > > > Resolved%2C%20Closed)
> > > >
> > > > The vote will be open for 72 hours (ending at 9:00AM Thursday,
> > > 06/08/2017).
> > > >
> > > > Please download the release candidate, check the hashes/signature,
> > build
> > > it
> > > > and test it, and then please vote:
> > > >
> > > >
> > > > [ ] +1 approve
> > > >
> > > > [ ] +0 no opinion
> > > >
> > > > [ ] -1 disapprove (and reason why)
> > > >
> > >
> > >
> > >
> > > --
> > > Jagadish V,
> > > Graduate Student,
> > > Department of Computer Science,
> > > Stanford University
> > >
> >
>

Re: [VOTE] Apache Samza 0.13.0 RC2

[Boris S]

Downloaded, ran build and test and integration tests on Linux RedHat.
Integration tests passed (using python 2.7).

+1 (non-binding)

On Wed, May 31, 2017 at 7:33 PM, Jagadish Venkatraman 
wrote:

> This is a call for a vote on a release of Apache Samza 0.13.0. Thanks to
> everyone who has contributed to this release. We are very glad to see some
> new contributors and features in this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~jagadish/samza-0.13.0-rc2/
>
> The release candidate is signed with pgp key AF81FFBF, which can be found
> on keyservers:
> http://pgp.mit.edu/pks/lookup?op=get=0xAF81FFBF
>
> The git tag is release-0.13.0-rc2 and signed with the same pgp key:
> https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=
> refs/tags/release-0.13.0-rc2
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1022
>
> 142 issues were resolved for this release:
> https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20SAMZA%20AND%20fixVersion%20in%20(0.13%2C%200.13.0)%
> 20AND%20status%20in%20(Resolved%2C%20Closed)
>
> The vote will be open for 72 hours (ending at 8:00PM Saturday, 06/03/2017).
>
> Please download the release candidate, check the hashes/signature, build it
> and test it, and then please vote:
>
>
> [ ] +1 approve
>
> [ ] +0 no opinion
>
> [ ] -1 disapprove (and reason why)
>
>
> --
> Jagadish V,
>

Re: [VOTE][RESULT] Apache Samza 0.12.0 RC2

[Boris S]

I've run the build and all the regular tests and they passed.
But when I ran integration tests I got the following results:

python 2.6 doesn't work - version too low.
python 3.7 doesn't work - version too high ;-). It fails with a syntax
error in zopkio.

python 2.7 seems to work.

Integration tests ran for 21 minutes. It posted and error at the end (after
the tests finished):
2017-05-31 17:35:49,919 zopkio.test_runner [ERROR] single execution failed
teardown_suite(). Traceback (most recent call last):


3 tests passed in 21 minutes.

+1 for release (non-binding)

On Wed, Feb 15, 2017 at 1:45 PM, Jagadish Venkatraman <
jagadish1...@gmail.com> wrote:

> The vote of 0.12.0 RC2 has been more than 72 hours and we got +1
> (binding) x 3 and +1 (non-binding) x 6.
>
> Samza 0.12.0 officially passed the VOTE!
>
> Cheers,
> Jagadish
>
> On Wed, Feb 15, 2017 at 11:55 AM, Yi Pan <nickpa...@gmail.com> wrote:
>
> > Ran check-all and integration tests on Mac. Passed and verified the pgp
> > key.
> >
> > P.S. I do see a non-consistent test hanging issue on my Mac in
> > TestStreamProcessor test. Drilled in a bit and found out that the Kafka
> > broker is not started serving correctly during the job initialization. It
> > passed in Linux build and also passed when I ran it in Intellij. Given
> that
> > others does not see this issue on Mac build either, I won't consider this
> > as a blocker for the new release. I created a JIRA reporting that issue:
> > SAMZA-1088.
> >
> > +1 (binding).
> >
> > On Tue, Feb 14, 2017 at 5:43 PM, Chinmay Soman <
> chinmay.cere...@gmail.com>
> > wrote:
> >
> > > Downloaded release (on Mac), checked build test (and checkall).
> > >
> > > Verified the pgp key (although with the warning).
> > >
> > > +1 Binding.
> > >
> > > On Mon, Feb 13, 2017 at 3:26 PM, Renato Marroquín Mogrovejo <
> > > renatoj.marroq...@gmail.com> wrote:
> > >
> > > > I also run check-all against Debian and build was successful,
> although
> > I
> > > > saw a bunch of this error:
> > > >
> > > > apache-samza-0.12.0-src/samza-hdfs/src/main/java/org/apache/
> > > > samza/system/hdfs/HdfsSystemConsumer.java:59:
> > > > error: unmappable character for encoding ASCII
> > > >  *
> > > >  
> > > > ?
> > > >
> > > > I don't think they are a blocker, it's some characters not being able
> > to
> > > be
> > > > parsed when doing the scalaCompile task for the samza-hdfs component.
> > Is
> > > it
> > > > worth opening a JIRA to fix this?
> > > >
> > > >
> > > > Best,
> > > >
> > > > Renato M.
> > > >
> > > >
> > > > 2017-02-13 22:20 GMT+01:00 Navina Ramesh
> <nram...@linkedin.com.invalid
> > >:
> > > >
> > > > > I ran check-all against Mac and integration tests on Linux. Looks
> > good
> > > > with
> > > > > no concerning issues.
> > > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks!
> > > > > Navina
> > > > >
> > > > > On Fri, Feb 10, 2017 at 9:25 AM, Boris S <bor...@gmail.com> wrote:
> > > > >
> > > > > > I also successfully ran the integration tests on Linux. All
> passed.
> > > > > > +1 non-binding
> > > > > >
> > > > > > On Wed, Feb 8, 2017 at 4:57 PM, Jacob Maes <jacob.m...@gmail.com
> >
> > > > wrote:
> > > > > >
> > > > > > > Build and integration tests were successful for me.
> > > > > > >
> > > > > > > +1 non-binding
> > > > > > >
> > > > > > > On Wed, Feb 8, 2017 at 4:48 PM, xinyu liu <
> xinyuliu...@gmail.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Ran build, checkAll and integration tests. All passed.
> > > > > > > >
> > > > > > > > +1 non-binding.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Xinyu
> > > > > > > >
> > > > > > > > On Wed, Feb 8, 2017 at 4:18 PM, Boris S <bor...@gmail.com>
> > > wrote:
> > > > 

Re: [VOTE] Apache Samza 0.13.0 RC0

[Boris S]

I think we need to add SAMZA-1286 and
SAMZA-1279 to the release .

On Wed, May 10, 2017 at 7:51 PM, Jagadish Venkatraman 
wrote:

> This is a call for a vote on a release of Apache Samza 0.13.0. Thanks to
> everyone who has contributed to this release. We are very glad to see some
> new contributors and features in this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~jagadish/samza-0.13.0-rc0/
>
> The release candidate is signed with pgp key AF81FFBF, which can be found
> on keyservers:
> http://pgp.mit.edu/pks/lookup?op=get=0xAF81FFBF
>
> The git tag is release-0.13.0-rc0 and signed with the same pgp key:
> https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=
> refs/tags/release-0.13.0-rc0
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1020
>
> 127 issues were resolved for this release:
> https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20SAMZA%20AND%20fixVersion%20in%20(0.13%2C%200.13.0)%
> 20AND%20status%20in%20(Resolved%2C%20Closed)
>
> The vote will be open for 72 hours (ending at 8:00PM Saturday, 05/13/2017).
>
> Please download the release candidate, check the hashes/signature, build it
> and test it, and then please vote:
>
>
> [ ] +1 approve
>
> [ ] +0 no opinion
>
> [ ] -1 disapprove (and reason why)
>
>
> +1 from my side for the release.
>
> Cheers!
>

Re: [VOTE] Samza Logo

[Boris S]

Voted. I like the orbital one and the dotted 'S'. Red is not my preferred
color, but Samza has been associated with red for a long time, so I don't
see any reason for changing it.

On Tue, Apr 18, 2017 at 11:38 AM, Navina Ramesh <
nram...@linkedin.com.invalid> wrote:

> @Renato:
> Thanks for your feedback. Always appreciate help from our contributors :)
>
> @Jagadish:
> Thanks for stating your points. It is clear that you are against the
> butterfly ones. But are you in support of any of the others? Please vote :)
>
> Thanks!
> Navina
>
> On Tue, Apr 18, 2017 at 10:38 AM, Jagadish Venkatraman <
> jagadish1...@gmail.com> wrote:
>
> > FWIW, I have a contrarian perspective on this one. Here's my 2 cents:
> >
> > I'm -1 for having for our logo to do anything with a butterfly.
> >
> > - Samza and Kafka are separate top-level projects. I do not think the
> > connection to Franz Kafka's novel on "metamorphosis", and the fact that a
> > salesman named "Samsa" in the novel was transformed to an "insect" should
> > dictate our logo.  Agreed, the butterfly is a cute insect remotely
> > relatable(?) to stream processing via a convoluted story.
> >
> > - For a choice of a mascot, I'd much rather have something that signifies
> > scale, sturdiness or swiftness instead of a cute butterfly :-)
> >
> > - The 2 other non-butterfly logos at-least have a "node", "stream",
> > "edges", "graph" like feel which I like.
> >
> > Thanks,
> > Jagadish
> >
> > On Sat, Apr 15, 2017 at 12:24 PM, Jacob Maes 
> wrote:
> >
> > > I think I voted the exact opposite to everyone else in this thread.
> > >
> > > I don't want anything to do with a butterfly. The metaphor is even
> > further
> > > removed from the Samsa story than a cockroach, so I think we should
> give
> > up
> > > on that. I don't want a mascot; we're not building a university
> football
> > > team. And as animals go, the only slower one I can think of is a sloth,
> > so
> > > I don't feel a butterfly says "scalable stream processing". This,
> > combined
> > > with my preference to eschew the color red for logos, puts the red
> > > butterfly last.
> > >
> > > The blue butterfly is a little more abstract and formal looking, but
> > still
> > > a butterfly, so that is second to last.
> > >
> > > The other 2 are very close, in my opinion.
> > >
> > > The one with the circles is reminiscent of orbital loops, which gives
> me
> > > the feeling of scale. It also has the dots at varying places along the
> > > lines, which to me conveys the different proportions of input/output
> > stream
> > > sizes/TTLs. And the cyclical shape could also be used for animations
> > > portraying the concept of "reprocessing"
> > >
> > > The one with the "S" dots reminds me of the Kafka logo without the
> lines.
> > > If the lines are the streams and the dots are processing nodes, then I
> > > think it's clever for the Samza logo to be a "negative" of the Kafka
> one.
> > > That's not to say samza is any more related to Kafka than it is; but if
> > the
> > > Kafka logo says "streams" then to me this Samza logo says "processors"
> > >
> > > My 2 cents.
> > >
> > > On Fri, Apr 14, 2017 at 11:05 PM, Ignacio Solis 
> wrote:
> > >
> > > > You're making me feel bad for linking that one! :-)
> > > >
> > > > I don't see it as a maze. To me, that one is like circles that turn,
> > > > representing the processing. Like cogs on an engine. The little
> > > > circles are like the messages. The concentric circles are like the
> > > > streams.
> > > >
> > > > The red butterfly is my second favorite.
> > > >
> > > > Vote note:
> > > >
> > > > Once we close voting we'll look at the actual results.  The way the
> > > > ranking gets calculated it you don't vote for a design at all, that
> > > > vote does not get factored in. It assumes you have no opinion. So if
> > > > somebody votes 5 stars on A and 1 start on B.  And second person only
> > > > votes 5 stars on B, then the ranking would be  A-5 stars, B-3 stars.
> > > > (or something along those lines).   So if you only vote on the 5 star
> > > > ones, you're missing your vote on the ones you don't like.
> > > >
> > > > So, once we close, we'll see how people voted.
> > > >
> > > > Nacho
> > > >
> > > >
> > > > On Fri, Apr 14, 2017 at 9:21 PM, Yi Pan  wrote:
> > > > > Really? The one with the maze on the left currently is top one? I
> > can't
> > > > > relate to that either. My favorite was the logo w/ Taiji symbol.
> > Since
> > > > that
> > > > > did not make the top 4, I am voting for the red bufferfly one, same
> > as
> > > > > Navina.
> > > > >
> > > > > -Yi
> > > > >
> > > > > On Fri, Apr 14, 2017 at 3:33 PM, Navina Ramesh
> > > >  > > > >> wrote:
> > > > >
> > > > >> I prefer to have open discussions in the official mailing list or
> > JIRA
> > > > >> since it is an open-community. It also helps track the
> discussions.
> > > > >>
> > > > >> Fwiw, I am in favor of the 

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

[Boris S]

+1 Looks good to me.

On Tue, Mar 28, 2017 at 2:00 PM, xinyu liu  wrote:

> +1 on my side. Very happy to see this proposal. This is a blocker for
> integrating fluent API with StreamProcessor, and hopefully we can get it
> resolved soon :).
>
> Thanks,
> Xinyu
>
> On Tue, Mar 28, 2017 at 11:28 AM, Navina Ramesh (Apache) <
> nav...@apache.org>
> wrote:
>
> > Hi everyone,
> >
> > This is a voting thread for SEP-1: Semantics of ProcessorId in Samza.
> > For reference, here is the wiki link:
> > https://cwiki.apache.org/confluence/display/SAMZA/SEP-
> > 1%3A+Semantics+of+ProcessorId+in+Samza
> >
> > Link to discussion mail thread:
> > http://mail-archives.apache.org/mod_mbox/samza-dev/201703.
> > mbox/%3CCANazzuuHiO%3DvZQyFbTiYU-0Sfh3riK%3Dz4j_
> AdCicQ8rBO%3DXuYQ%40mail.
> > gmail.com%3E
> >
> > Please vote on this SEP asap. :)
> >
> > Thanks!
> > Navina
> >
>

Re: [VOTE] Apache Samza 0.12.0 RC2

[Boris S]

I also successfully ran the integration tests on Linux. All passed.
+1 non-binding

On Wed, Feb 8, 2017 at 4:57 PM, Jacob Maes <jacob.m...@gmail.com> wrote:

> Build and integration tests were successful for me.
>
> +1 non-binding
>
> On Wed, Feb 8, 2017 at 4:48 PM, xinyu liu <xinyuliu...@gmail.com> wrote:
>
> > Ran build, checkAll and integration tests. All passed.
> >
> > +1 non-binding.
> >
> > Thanks,
> > Xinyu
> >
> > On Wed, Feb 8, 2017 at 4:18 PM, Boris S <bor...@gmail.com> wrote:
> >
> > > Cloned the release and ran build, test and checkAll.sh
> > > All passed.
> > > Verified MD5 and the signature.
> > > Got warning - "this key is not certified with a trusted signature". I
> > guess
> > > it is ok.
> > >
> > > +1
> > >
> > > On Mon, Feb 6, 2017 at 5:32 PM, Jagadish Venkatraman <
> > > jagadish1...@gmail.com
> > > > wrote:
> > >
> > > > This is a call for a vote on a release of Apache Samza 0.12.0. Thanks
> > to
> > > > everyone who has contributed to this release. We are very glad to see
> > > some
> > > > new contributors in this release.
> > > >
> > > > The release candidate can be downloaded from here:
> > > > http://home.apache.org/~jagadish/samza-0.12.0-rc2/
> > > >
> > > > The release candidate is signed with pgp key AF81FFBF, which can be
> > found
> > > > on keyservers:
> > > > http://pgp.mit.edu/pks/lookup?op=get=0xAF81FFBF
> > > >
> > > > The git tag is release-0.12.0-rc2 and signed with the same pgp key:
> > > > https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=
> > > > refs/tags/release-0.12.0-rc2
> > > >
> > > > Test binaries have been published to Maven's staging repository, and
> > are
> > > > available here:
> > > > https://repository.apache.org/content/repositories/
> orgapachesamza-1018
> > > >
> > > > Note that the binaries were built with JDK8 without incident.
> > > >
> > > > 26 issues were resolved for this release:
> > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20S
> > > > AMZA%20AND%20fixVersion%20in%20(0.12%2C%200.12.0)%20AND%20st
> > > > atus%20in%20(Resolved%2C%20Closed)
> > > >
> > > > The vote will be open for 72 hours (end in 6PM Thursday, 02/09/2017
> ).
> > > >
> > > > Please download the release candidate, check the hashes/signature,
> > build
> > > it
> > > > and test it, and then please vote:
> > > >
> > > >
> > > > [ ] +1 approve
> > > >
> > > > [ ] +0 no opinion
> > > >
> > > > [ ] -1 disapprove (and reason why)
> > > >
> > > >
> > > > +1 from my side for the release.
> > > >
> > > > Cheers!
> > > >
> > > > --
> > > > Jagadish V,
> > > > Graduate Student,
> > > > Department of Computer Science,
> > > > Stanford University
> > > >
> > >
> >
>

Re: [VOTE] Apache Samza 0.12.0 RC2

[Boris S]

Cloned the release and ran build, test and checkAll.sh
All passed.
Verified MD5 and the signature.
Got warning - "this key is not certified with a trusted signature". I guess
it is ok.

+1

On Mon, Feb 6, 2017 at 5:32 PM, Jagadish Venkatraman  wrote:

> This is a call for a vote on a release of Apache Samza 0.12.0. Thanks to
> everyone who has contributed to this release. We are very glad to see some
> new contributors in this release.
>
> The release candidate can be downloaded from here:
> http://home.apache.org/~jagadish/samza-0.12.0-rc2/
>
> The release candidate is signed with pgp key AF81FFBF, which can be found
> on keyservers:
> http://pgp.mit.edu/pks/lookup?op=get=0xAF81FFBF
>
> The git tag is release-0.12.0-rc2 and signed with the same pgp key:
> https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=
> refs/tags/release-0.12.0-rc2
>
> Test binaries have been published to Maven's staging repository, and are
> available here:
> https://repository.apache.org/content/repositories/orgapachesamza-1018
>
> Note that the binaries were built with JDK8 without incident.
>
> 26 issues were resolved for this release:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20S
> AMZA%20AND%20fixVersion%20in%20(0.12%2C%200.12.0)%20AND%20st
> atus%20in%20(Resolved%2C%20Closed)
>
> The vote will be open for 72 hours (end in 6PM Thursday, 02/09/2017 ).
>
> Please download the release candidate, check the hashes/signature, build it
> and test it, and then please vote:
>
>
> [ ] +1 approve
>
> [ ] +0 no opinion
>
> [ ] -1 disapprove (and reason why)
>
>
> +1 from my side for the release.
>
> Cheers!
>
> --
> Jagadish V,
> Graduate Student,
> Department of Computer Science,
> Stanford University
>

Re: Questions about checkpointing and Kinesis

[Boris S]

If you are implementing your own consumer, on start Samza will call
register method of the consumer and will pass the offsets.
If you are using CheckpointListener (like Jagadish mentioned) you will get
a callback on each checkpoint (so you can ignore the register call),
otherwise you can use checkpoint passed to you in the register call.

On Tue, Feb 7, 2017 at 9:11 AM, Chad Greenberg 
wrote:

> Thanks Jagadish for the reply. A few comments.
>
>
> I was under the impression that using the KCL would mean recording the
> offsets in Kinesis/DynanoDB and therefore not Samza. Avoiding the KCL so
> that I can save state in Samza.
>
>
> I assumed that the checkpoint values were based on the values of the put()
> method, but I did not see any explicit documentation to that effect (I have
> been reading a ton, so I could have missed something). What I do not see
> however, is how to retrieve those values upon start-up. Is this the role of
> the SystemAdmin? Is there any documentation about the use of SystemAdmin?
>
> 
> From: Jagadish Venkatraman 
> Sent: Tuesday, February 7, 2017 1:48:00 AM
> To: dev@samza.apache.org
> Subject: Re: Questions about checkpointing and Kinesis
>
> Great to hear this development on the kinesis consumer!
>
> Let me answer some of your questions here.
>
> *1. "Kinesis does not have a listener/push framework (unless I missed
> something)"*
>
>  Let me point out that Kinesis has both a push and a pull based model. You
> can choose to implement either for your use-case.
>
>- *Pull:* The pull based model supports obtaining an *ShardIterator* for
>a shard and iterating on it. Please refer the docs for the
>*ShardIteratorRequest* here
> amazonaws/services/kinesis/model/GetShardIteratorRequest.html>.
>It's important that you are aware of re-sharding - The
>record.getNextShardIterator() can return null if there was a merge / a
>split for the shard. (You can trivially handle re-shards by re-starting)
>- *Push:* The push based model directly uses KCL (The Kinesis Client
>Library) to subscribe to events. KCL will handle sharding, re-balancing,
>checkpointing internally.
>
> Your consumer can implement the *CheckpointListener
>  src/main/java/org/apache/samza/checkpoint/CheckpointListener.java>*
> interface to get notified when Samza performs a checkpoint. You can save
> the offsets of the shards to kinesis by invoking
> IRecordCheckpointer.checkpoint
>  master/src/main/java/com/amazonaws/services/kinesis/
> clientlibrary/interfaces/IRecordProcessorCheckpointer.java>
> APIs.
>
>
> *2. "My plan was to use the default KafkaCheckpointManagerFactory on an
> timed interval basis"*
>
> The checkpoint manager merely provides persistence for the checkpoints. (In
> that sense, it's actually a checkpoint writer). You probably don't want to
> implement a custom checkpoint manager.
>
>
> *3. "What exactly is being checkpointed? What value can I retrieve to use
> as an offset for my Kinesis stream? Or is this something I need to keep
> track of in a store? If so, what is the point of checkpointing? Can I use
> RocksDb to save the Kinesis offset at every document (efficiently that
> is)?"*
>
> *- *Samza checkpoints [ssp, offset] pairs for your tasks.
> - Kinesis has an implicit notion of sequence numbers for every shard in a
> stream. You can use that as offsets.
> - You don't want to record offsets in a separate store. If you want Samza
> to manage offsets, Samza will use Kafka internally. If you want Kinesis to
> manage offsets (KCL) , Kinesis will use DynamoDb to store its offsets.
>
>
>
>
> On Mon, Feb 6, 2017 at 2:57 PM, Chad Greenberg 
> wrote:
>
> > Starting on an integration project between a Kinesis stream and Samza,
> > despite have no background in either, but I am familiar with most other
> > messaging/queuing systems.
> >
> >
> > Decided to keep all state management within Samza instead of using
> > Kinesis' client library. My plan was to use the default
> > KafkaCheckpointManagerFactory on an timed interval basis, but I have a
> few
> > questions.
> >
> >
> > What exactly is being checkpointed? What value can I retrieve to use as
> an
> > offset for my Kinesis stream? Or is this something I need to keep track
> of
> > in a store? If so, what is the point of checkpointing? Can I use RocksDb
> to
> > save the Kinesis offset at every document (efficiently that is)?
> >
> >
> > Related to Kinesis and not quite Samza, it does not have a listener/push
> > framework, but it requires constant polling (unless I missed something).
> > First of all, I was going to have a partition for each Kinesis shard
> > partition. But the next question is, should I simply have a while(true)
> > polling