Re: Samza jdk11 support

2020-10-16 Thread Yi Pan
found the below doc but that talks about samza 0.10. > > https://samza.apache.org/contribute/tests.html > > Thanks, > > On Thu, Jun 4, 2020 at 12:18 AM Yi Pan wrote: > > > Hi, Debraj and Jordan, > > > > Thanks a lot for the ping. I dug a bit deeper in the pa

Re: SAMZA-2612: Kafka topic naming not supported fully

2020-12-28 Thread Yi Pan
Hey, Stuart, Sounds great that you have found the way around it! Thanks! -Yi On Sat, Dec 19, 2020 at 12:28 PM Stuart Perks wrote: > This can be done using withPhysicalName > > Closed the JIRA > > On 2020/12/17 12:19:27, Stuart Perks wrote: > > https://issues.apache.org/jira/browse/SAMZA-2612>

Re: Zookeeper and Client upgrade

2021-01-11 Thread Yi Pan
Hi, Stuart, Please feel free to raise tickets for update requests like these. Thanks for reporting! -Yi On Fri, Jan 8, 2021 at 9:58 PM Stuart Perks wrote: > The Zookeeper version sits at 3.4.6 with 3.6.2 now available bringing > security enhancements. > > > https://github.com/apache/samza/blo

[REPORT] Apache Samza - Jan 2021

2021-01-13 Thread Yi Pan
## Description Apache Samza is a distributed stream processing engine that is highly configurable to process events from various data sources, including real-time messaging system (e.g. Kafka) and distributed file systems (e.g. HDFS). ## Issues: - There are no issues requiring board attention. ##

Re: [VOTE] Apache Samza 1.6.0 RC1

2021-01-13 Thread Yi Pan
Verified signature and sha1. Ran check-all and integration tests. All passed. +1 (binding). -Yi On Mon, Jan 11, 2021 at 4:08 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > Thanks for driving this release Boris! Verified signatures and ran all the > tests. > check-all and inte

Re: Zookeeper and Client upgrade

2021-01-14 Thread Yi Pan
/jira/browse/SAMZA-2617 > > Happy to start looking onto one of these. > > On 2021/01/11 23:04:08, Yi Pan wrote: > > Hi, Stuart,> > > > > Please feel free to raise tickets for update requests like these.> > > > > Thanks for reporting!> > > >

Welcome Sanil Jain as Apache Samza committer

2021-02-02 Thread Yi Pan
Hi, everyone, I am glad to announce that Sanil Jain has officially accepted our invitation and become an Apache Samza committer now. Please join me to give him a warm welcome! Cheers! -Yi

Welcome Ke Wu as Apache Samza committer

2021-02-23 Thread Yi Pan
Hi, everyone, I am glad to announce that Ke Wu has officially accepted our invitation and become an Apache Samza committer now. Please join me to give him a warm welcome! Cheers! -Yi

Re: How are samza.container.id generated in yarn?

2021-04-22 Thread Yi Pan
Hi, Debraj, In YARN environment, Samza uses YARN generated containerIds as environmental variables to set each container process's samza.container.id. i.e. when containers are requested by Samza AM process in YARN, YARN RM will reply with a set of allocated container objects, which is of class org

Upcoming streams meetup event

2021-05-25 Thread Yi Pan
& Yuhong Cheng, LinkedIn Scaling Kafka Audit Service - Abhishek Mendhekar, LinkedIn Please visit our Meetup site for access to more details & the live event link (TBD), and to let us know you're coming! https://lnkd.in/g7Fq-3K Hope to see you there! Best, - Yi Pan

Reminder: LinkedIn Stream Processing Meetup on Jun 24

2021-06-15 Thread Yi Pan
The meet up is coming in less than 10 days! Please remember to sign up and join us! https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/278266182/ Short abstracts for the contents: https://www.linkedin.com/posts/celiakkung_stream-processing-with-apache-kafka-apache-activity-68027065437

Re: [VOTE] SEP-28: Samza State Backend Interface and Checkpointing Improvement

2021-06-22 Thread Yi Pan
+1 (binding) this is going to improve our state recovery story significantly! -Yi On Mon, Jun 21, 2021 at 1:03 PM Daniel Chen wrote: > Hi all, > > This is a call for a vote on SEP-28: Samza State Backend Interface and > Checkpointing Improvements. Thanks to everyone who was involved with the >

Re: [VOTE] SEP-29: Blob Store Based State Backup And Restore

2021-06-22 Thread Yi Pan
+1 (binding). Thanks for rolling out this big feature! -Yi On Tue, Jun 22, 2021 at 1:42 PM Sanil Jain wrote: > +1 (non-binding) Thanks for this contribution! > > -Sanil > > On Tue, 22 Jun 2021 at 13:13, Daniel Chen wrote: > > > +1 (non-binding), thanks! > > > > On Tue, Jun 22, 2021 at 1:10 PM

[Draft] [REPORT] Samza Report July 2021

2021-07-14 Thread Yi Pan
## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ## Issues: - There are no issues requiring board attention. ## Membership Data: Apache Samza was founded 2015-01-22 (6 years ago) There are currently 28 committers a

Re: Malformed URL issue while deploying Samza on a pure IPv6 VM

2021-08-26 Thread Yi Pan
Hi Vishal, Could you open a JIRA to track this one? I will circle back to our internal team to do a quick assessment on IPv6 related issues. Meanwhile, I strongly encourage you to submit the patch and continue your test, since that's the best way to discover any hidden issues on that front. Thank

Re: [ANNOUNCE] Welcome Daniel Chen as Samza Committer

2021-09-17 Thread Yi Pan
Congrats, Daniel, well deserved!!! -Yi On Fri, Sep 17, 2021 at 11:23 AM Xinyu Liu wrote: > Hi, all, > > I am glad to announce that Daniel Chen has officially accepted our > invitation and become an Apache Samza Committer now. > > Daniel has contributed to many areas of Samza, from his early wor

[Draft][REPORT] Apache Samza Oct 2021

2021-10-11 Thread Yi Pan
Hi, team, Please read the following draft report for Oct 2021. Let me know if I missed anything. Thanks a lot! ## Description: The mission of Samza is the creation and maintenance of software related to distributed

[Draft] [REPORT] Apache Samza Jan 2022

2022-01-12 Thread Yi Pan
Hi, team, Please read the draft report below and let me know if I missed anything. Thanks! - Yi == ## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ## I

Re: [VOTE] SEP-30: Support Updates in Table API

2022-01-24 Thread Yi Pan
Discussed and resolved the minor concerns offline. +1 (binding) for this one. Thanks! -Yi On Tue, Dec 21, 2021 at 1:28 PM Xinyu Liu wrote: > +1 on my side. > > Glad to see this feature coming. Please make sure the api changes are > reflected in the documents, e.g. > https://samza.apache.org/le

Re: [DISCUSS] Apache Samza 1.7.0 RC0

2022-01-26 Thread Yi Pan
Huge +1! Can't wait to see this list of features coming out! -Yi On Wed, Jan 26, 2022 at 2:36 PM Daniel Chen wrote: > Hi folks, > > We have added a number of major features and changes to master since > > 1.6, that warrants a major 1.7 release. > > Within LinkedIn, some of these features have a

[REPORT] Samza - Feb 2022

2022-02-09 Thread Yi Pan
## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ## Issues: - There are no issues requiring board attention. ## Membership Data: Apache Samza was founded 2015-01-22 (7 years ago) There are currently 29 committers a

Re: [VOTE] Apache Samza 1.7.0 RC1

2022-03-15 Thread Yi Pan
+1 (binding). Ran check-all, verified the signature and checksums. All passed. Thanks for pushing 1.7.0 out of the door! Yi On Fri, Mar 11, 2022 at 2:31 PM Xinyu Liu wrote: > +1 (binding). > > Verified the signature and checksums, and also ran check-all tests which > all passed. > > Thanks, >

Re: [RESULT][VOTE] Apache Samza 1.7.0 RC1

2022-03-15 Thread Yi Pan
Thanks, Daniel! Just want to mention that Boris also voted +1 (binding). Best! -Yi On Tue, Mar 15, 2022 at 9:22 AM Daniel Chen wrote: > Hey all, > > The vote for 1.7.0 release has been out for more than 72 hours and we got > +1(binding) x3 from Yi, Xinyu, Daniel > > Samza 1.7.0 officially pa

[REPORT] Samza - April 2022

2022-04-13 Thread Yi Pan
## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ## Issues: - There are no issues requiring board attention. ## Membership Data: Apache Samza was founded 2015-01-22 (7 years ago) There are currently 29 committers a

Re: Java 11 Checkin again

2022-09-02 Thread Yi Pan
Hey, James, Thanks for the ping. @prateek, can we have someone to review this change? One question: have you tested the change w/ the older YARN cluster version (running 2.10.1)? If this change requires YARN cluster upgrade to 3.3.4 as well, that may be a breaking change to existing Samza users (

Re: Running v1.7.0 locally

2022-09-02 Thread Yi Pan
Hey, Malcolm, Thanks for reporting this issue. Could you open a JIRA to track that? Best! -Yi On Mon, Aug 29, 2022 at 5:53 PM Malcolm McFarland wrote: > Hey folks, > > I've recently been attempting to upgrade our legacy application from Samza > 1.5.1 to 1.7.0. With version 1.5.1, I've had no

Re: Java 11 Checkin again

2022-09-14 Thread Yi Pan
Samza app running against 3.3.4 but I am wary it > >> would > >> > work. Yarn itself is not compatible at 2.10.1 with Java 11 so you > would > >> > have to update yarn even if the Java library here wasn't updated. > >> > > >> > Could we

Re: Java 11 Checkin again

2022-09-14 Thread Yi Pan
the Java 11 and yarn 3 module need to only use the Scala 2.12 > > version of the build. > > > > Jamie > > > > > > On Wed, Sep 14, 2022, 2:38 AM Yi Pan wrote: > > > >> Hi, James, > >> > >> Sorry to reply late. I just came

Re: Java 11 Checkin again

2022-09-19 Thread Yi Pan
me/us, but just wanted to call it out. > > -Jamie > > > > On Wed, Sep 14, 2022 at 5:52 PM Yi Pan wrote: > > > Hey, James, > > > > In order to merge your PR without breaking the jdk8 older modules, we > will > > need the changes proposed here. Can you tr

Re: Java 11 Checkin again

2022-09-19 Thread Yi Pan
mode) > > > > > > ./gradlew build -PscalaSuffix=2.11 > > > > > > The build fails with this command using that version of Java 8 ^. > > > > > > Anyway, just wanted to point that out since I hit this in my branch > > > trying to utilize

Re: Request for new release

2022-10-14 Thread Yi Pan
Hi, James, Thanks for the reminder. We are preparing the new 1.8 release. It is expected to be the end of this quarter. Best, -Yi On Mon, Oct 10, 2022 at 12:52 PM James DeMichele wrote: > Hello, we just had a pr merged to main in the Samza app that now supports > Java 11 runtime environments.

[REPORT] Samza - Nov 2022

2022-11-09 Thread Yi Pan
## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ## Issues: - There are no issues requiring board attention. ## Membership Data: Apache Samza was founded 2015-01-22 (8 years ago) There are currently 29 committers a

Re: SEP-31: Pipeline Drain: Support the ability to drain pipelines to allow incompatible intermediate schema changes

2022-12-01 Thread Yi Pan
Hi, Ajo, Sorry to reply this late. Could you clarify one thing in the design: For watermark triggered window draining, is the infinitive watermark trigger happen first, or the drain token in all source SSP happen first? Shouldn't it be the following sequence: a) all drain token from all input sour

Re: SEP-31: Pipeline Drain: Support the ability to drain pipelines to allow incompatible intermediate schema changes

2022-12-08 Thread Yi Pan
ready to shutdown, RunLoop > shuts down. > > Do you see any issues with it ? > > - Ajo > > > On Thu, 1 Dec 2022 at 20:06, Yi Pan wrote: > > > Hi, Ajo, > > > > Sorry to reply this late. Could you clarify one thing in the design: For > > watermark triggered

Re: [VOTE] SEP-31: Pipeline Drain- Support the ability to drain pipelines to allow incompatible intermediate schema changes

2022-12-08 Thread Yi Pan
+1. Long awaited feature! Thanks! -Yi On Tue, Nov 29, 2022 at 11:46 AM Xinyu Liu wrote: > +1. > > Overall the design looks good. Thanks for contributing to this feature. > > Thanks, > Xinyu > > On Tue, Nov 29, 2022 at 10:44 AM Ajo Thomas > wrote: > > > Hi All, > > > > This is a call for a vote

Re: [ANNOUNCE] Welcome Ajo Thomas as Samza Committer

2022-12-16 Thread Yi Pan
Welcome and congrats, Ajo! - Yi On Wed, Dec 14, 2022 at 3:42 PM Xinyu Liu wrote: > Hi, All, > > I am glad to announce that Ajo Thomas has officially accepted our > invitation and become an Apache Samza Committer now. > > Ajo has made contributions to improve both Samza user experience and > ope

Re: [VOTE] Apache Samza 1.8.0 RC0

2023-01-06 Thread Yi Pan
(+1) binding, Downloaded the src tarball, run check-all.sh and passed all tests. One thing noticed: there are configurations for your personal keys used for publishing the jars checked in gradle.properties. I don't think that we need to include that in the published src tarball. Otherwise, lgtm.

Re: [DISCUSS] SEP-32: Elasticity for Samza

2023-01-19 Thread Yi Pan
Hey, Manasa, Sorry to chime in late. A few questions: a) how are states for the virtual tasks managed during split/merge? b) what's perf impact when we have 2 virtual tasks on the same SSP in the same container, while one virtual task is much faster than the other? c) what's the reason that a virt

Re: [DISCUSS] SEP-32: Elasticity for Samza

2023-02-06 Thread Yi Pan
due to all keys (single task) in key ordered executor >> sitting in the same host will increase the load on the host and (c) if one >> or more of the parallel units (threads here) needs more resources, it will >> result in large container which makes scheduling harder as finding lar

Re: [VOTE] SEP-32: Elasticity for Samza

2023-02-08 Thread Yi Pan
+1 (binding) Thanks! -Yi On Tue, Feb 7, 2023 at 2:14 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > +1 (binding) > > Cheers, > Bharath > > On Tue, Feb 7, 2023 at 12:56 PM Lakshmi Manasa > wrote: > > > Hi folks, > > > > This is a call for vote on SEP-32: Elasticity for Samza

Re: Review Request 29994: WIP: first draft of Schema API and one example implementation via Avro

2015-01-23 Thread Yi Pan (Data Infrastructure)
/test/java/org/apache/samza/sql/task/StoredRuntimeContext.java PRE-CREATION Diff: https://reviews.apache.org/r/29994/diff/ Testing --- Thanks, Yi Pan (Data Infrastructure)

Review Request 30241: [SAMZA-520] TLP changes

2015-01-23 Thread Yi Pan (Data Infrastructure)
025f0557bfcec85b325db4b2475eff633cfe2f4a Diff: https://reviews.apache.org/r/30241/diff/ Testing --- All links tested. Travis/build related links and download links are not updated yet. Need to follow up with that. Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 30241: [SAMZA-520] TLP changes

2015-01-23 Thread Yi Pan (Data Infrastructure)
/watcher.samsa 025f0557bfcec85b325db4b2475eff633cfe2f4a Diff: https://reviews.apache.org/r/30241/diff/ Testing --- All links tested. Travis/build related links and download links are not updated yet. Need to follow up with that. Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 30287: Stream SQL Object Model Draft

2015-01-27 Thread Yi Pan (Data Infrastructure)
d on the output entity that they represent. i.e. essentially, window expression and join expression are expressions that all generate a table-type data source entity. - Yi Pan (Data Infrastructure) On Jan. 26, 2015, 10:04 p.m., Milinda Pathirage wrote: > >

Re: Review Request 29754: StreamSQL operator API draft

2015-01-29 Thread Yi Pan (Data Infrastructure)
ql/src/test/java/org/apache/samza/sql/task/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/29754/diff/ Testing (updated) --- run ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 29754: StreamSQL operator API draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
ile=840328#file840328line37> > > > > StoreMessageCollector? Sounds good to me. - Yi ----------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29754/#review70380 ---

Re: Review Request 29754: StreamSQL operator API draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
pache/samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/29754/diff/ Testing --- run ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 30287: Stream SQL Object Model Draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
t; ------- > > (Updated Jan. 26, 2015, 10:04 p.m.) > > > Review request for samza, Chris Riccomini and Yi Pan (Data Infrastructure). > > > Repository: samza > > > Description > --- > > WIP: Stream SQL Obje

Re: Review Request 29754: StreamSQL operator API draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
ava/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/29754/diff/ Testing --- run ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 29754: StreamSQL operator API draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
rg/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/29754/diff/ Testing --- run ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 29754: StreamSQL operator API draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/29754/diff/ Testing --- run ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Review Request 30627: REVISION: operator APIs: rebased and cleaned up

2015-02-04 Thread Yi Pan (Data Infrastructure)
/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle bb07a3b84b14dcef94da1bb166eab6aa3d0026bb Diff: https://reviews.apache.org/r/30627/diff/ Testing --- ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 30634: Serde for Stream Tuples

2015-02-04 Thread Yi Pan (Data Infrastructure)
ackage name from org.apache.samza.sql.task to org.apache.samza.task.sql since both tasks are extended from org.apache.samza.task.StreamTask. The same for the StreamSqlTask. - Yi Pan (Data Infrastructure) On Feb. 4, 2015, 7:39 p.m., Navina Ramesh wrote: > > --

Review Request 30800: WIP: improving the manual operator interface

2015-02-09 Thread Yi Pan (Data Infrastructure)
/RandomOperatorTask.java PRE-CREATION samza-sql/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION Diff: https://reviews.apache.org/r/30800/diff/ Testing --- ./gradlew clean build passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 31909: SAMZA-590

2015-03-11 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31909/#review76120 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On March 10

Re: Review Request 32006: SAMZA-597

2015-03-13 Thread Yi Pan (Data Infrastructure)
g4j/src/main/java/org/apache/samza/logging/log4j/serializers/LoggingEventJsonSerde.java <https://reviews.apache.org/r/32006/#comment123934> nit: Serde - Yi Pan (Data Infrastructure) On March 13, 2015, 12:57 a.m., Chris Riccomini wrote: > > -

Re: Review Request 32052: SAMZA-592

2015-03-13 Thread Yi Pan (Data Infrastructure)
n metadata is not inserted in the cache, getOffsets would raise exception? And how do we capture that case? - Yi Pan (Data Infrastructure) On March 13, 2015, 7:56 p.m., Chris Riccomini wrote: > > --- > This is an automatically generat

Re: Review Request 32006: SAMZA-597

2015-03-13 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32006/#review76430 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On March 13

Re: Review Request 32052: SAMZA-592

2015-03-13 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32052/#review76437 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On March 13

Re: Review Request 32052: SAMZA-592

2015-03-13 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32052/#review76442 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On March 13

Re: Review Request 32188: Disable WAL in RocksDB KV store

2015-03-18 Thread Yi Pan (Data Infrastructure)
/samza/test/performance/TestKeyValuePerformance.scala <https://reviews.apache.org/r/32188/#comment124695> nit: do we need multiple instances of ByteSerde here? - Yi Pan (Data Infrastructure) On March 18, 2015, 9:48 p.m., Navina Ramesh

Re: Review Request 32188: Disable WAL in RocksDB KV store

2015-03-18 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32188/#review76989 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On March 18

Re: Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

2015-03-20 Thread Yi Pan (Data Infrastructure)
ated? 2. In the code block from 153, wouldn't it be easier to set a noFlushStop flag s.t. the stop() method can avoid calling the flush again when stop() is triggered from failed flush anyways? - Yi Pan (Data Infrastructure) On March 17, 2015, 9:37 a.m.

Re: Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

2015-03-20 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32155/#review77282 --- Ship it! Looks good to me. - Yi Pan (Data Infrastructure) On

Review Request 32407: SAMZA-571: add suppression interface for uncaught exceptions

2015-03-23 Thread Yi Pan (Data Infrastructure)
://reviews.apache.org/r/32407/diff/ Testing --- Unit test added. Pass with ./gradlew clean build Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 32407: SAMZA-571: add suppression interface for uncaught exceptions

2015-03-23 Thread Yi Pan (Data Infrastructure)
f we can live w/ the fact that if user implements ExceptionTask automatically disables all configured suppressions, I would go ahead w/ your suggestion. Thanks! - Yi --- This is an automatically generated e-mail. To reply, visit: https://revi

Re: Review Request 32407: SAMZA-571: add suppression interface for uncaught exceptions

2015-03-24 Thread Yi Pan (Data Infrastructure)
f6521 samza-core/src/test/scala/org/apache/samza/container/TestTaskInstance.scala 54b4df84f47f818d62ac0361196567ad1f430fde Diff: https://reviews.apache.org/r/32407/diff/ Testing (updated) --- Unit tests added. Pass with ./bin/check-all.sh Thanks, Yi Pan (Data Infrastructure)

Review Request 32874: SAMZA-608: fix system consumer stuck issue when discarding the serde exception

2015-04-06 Thread Yi Pan (Data Infrastructure)
/SystemConsumers.scala 125d37602e2c0a9da75674f37580a1ac02f94796 samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala 3fdc781c1275f928f4b51b01869e1122502a2c08 Diff: https://reviews.apache.org/r/32874/diff/ Testing --- passed local unit tests. Thanks, Yi Pan (Data

Review Request 32872: SAMZA-571: add test to capture RecordTooLargeException

2015-04-06 Thread Yi Pan (Data Infrastructure)
d66b3bd070a4cef4b1d3dded1d79a33cbe3fa09b Diff: https://reviews.apache.org/r/32872/diff/ Testing --- Passed local test suite Thanks, Yi Pan (Data Infrastructure)

Review Request 32892: DOCS; updating wiki links to point to cwiki

2015-04-06 Thread Yi Pan (Data Infrastructure)
s.apache.org/r/32892/diff/ Testing --- Passed ./bin/check-all.sh Thanks, Yi Pan (Data Infrastructure)

Review Request 33142: [SAMZA-561] Review in progress

2015-04-13 Thread Yi Pan (Data Infrastructure)
fd41b samza-test/src/main/python/tests/sql_tests.py PRE-CREATION samza-test/src/main/resources/orders.avsc PRE-CREATION samza-test/src/main/resources/orders.json PRE-CREATION Diff: https://reviews.apache.org/r/33142/diff/ Testing --- Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 33170: Renamed samza-sql to samza-sql-core

2015-04-14 Thread Yi Pan (Data Infrastructure)
s://reviews.apache.org/r/33170/#comment129814> Move to samza-sql-calcite samza-sql-core/src/test/java/org/apache/samza/sql/test/metadata/TestAvroSchemaConverter.java <https://reviews.apache.org/r/33170/#comment129816> Same here. - Yi Pan (Data Infrastructure)

Re: Review Request 33142: [SAMZA-561] Review in progress

2015-04-14 Thread Yi Pan (Data Infrastructure)
in/java/org/apache/samza/sql/operators/insert/InsertToStreamOp.java <https://reviews.apache.org/r/33142/#comment129689> I think that we may be able to combine these two operators. Let me think about it a bit more. samza-sql/src/main/java/org/apache/samza/sql/operators/scan/Projectabl

Re: Review Request 33219: [SAMZA-649] Create samza-sql-calcite module for Calcite SQL front end

2015-04-15 Thread Yi Pan (Data Infrastructure)
ma/types in samza-sql-core model to Calcite's RelDataType? In that case, can we use the generic Schema class in samza-sql-core instead of implementation specific for Avro? - Yi Pan (Data Infrastructure) On April 15, 2015, 2:49 p.m.,

Re: Review Request 33219: [SAMZA-649] Create samza-sql-calcite module for Calcite SQL front end

2015-04-15 Thread Yi Pan (Data Infrastructure)
> On April 15, 2015, 6:20 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-sql-calcite/src/main/java/org/apache/samza/sql/calcite/schema/AvroSchemaConverter.java, > > line 37 > > <https://reviews.apache.org/r/33219/diff/1/?file=930371#file930371line37> > > &g

Re: Review Request 33219: [SAMZA-649] Create samza-sql-calcite module for Calcite SQL front end

2015-04-15 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33219/#review80237 --- Ship it! +1 - Yi Pan (Data Infrastructure) On April 15, 2015, 2

Re: Review Request 33146: Adding a new KV store contract: BatchingKeyValueStore

2015-04-15 Thread Yi Pan (Data Infrastructure)
> On April 14, 2015, 9:14 p.m., Chris Riccomini wrote: > > I'm concerned that there might be an issue with this approach. In > > BaseKeyValueStorageEngineFactory, we compose stores by nesting them. If > > this is the case, I think that the top-most store will implement the > > batching key val

Re: Review Request 33219: [SAMZA-649] Create samza-sql-calcite module for Calcite SQL front end

2015-04-15 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33219/#review80248 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On April 15

Re: Review Request 33146: New KeyValueStore Features

2015-04-21 Thread Yi Pan (Data Infrastructure)
test/scala/org/apache/samza/storage/kv/TestKeyValueStores.scala <https://reviews.apache.org/r/33146/#comment131257> nit: prefer not to re-order the methods if not necessary. - Yi Pan (Data Infrastructure) On April 16, 2015,

Re: Review Request 33146: New KeyValueStore Features

2015-04-21 Thread Yi Pan (Data Infrastructure)
> On April 21, 2015, 6:49 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-kv/src/main/java/org/apache/samza/storage/kv/KeyValueStore.java, line > > 33 > > <https://reviews.apache.org/r/33146/diff/2/?file=931566#file931566line33> > > > > The sig

Re: Review Request 33199: Coordinator stream

2015-04-22 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33199/#review81242 --- Ship it! Went through the latest patch w/ Naveen. LGTM - Yi Pan

Re: Review Request 33146: New KeyValueStore Features

2015-04-23 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33146/#review81381 --- Ship it! LGTM. Thanks! - Yi Pan (Data Infrastructure) On April

Re: Review Request 33146: New KeyValueStore Features

2015-04-24 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33146/#review81497 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On April 24

Re: Review Request 33146: New KeyValueStore Features

2015-04-26 Thread Yi Pan (Data Infrastructure)
> On April 24, 2015, 5:01 p.m., Yi Pan (Data Infrastructure) wrote: > > Ship It! > > Mohamed Mahmoud (El-Geish) wrote: > I don't have access to commit. Can you please grant me access or commit > for me? Thanks! Hi, MOhamed, I was trying to go through all the test

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-04-28 Thread Yi Pan (Data Infrastructure)
, here we may be deleting empty/non-existing storagePartitionDirs. - Yi Pan (Data Infrastructure) On April 22, 2015, 9:54 p.m., Navina Ramesh wrote: > > --- > This is an automatically generated e-mail.

Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-01 Thread Yi Pan (Data Infrastructure)
-CREATION Diff: https://reviews.apache.org/r/33749/diff/ Testing --- ./gradlew clean :samza-sql-core:build Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-03 Thread Yi Pan (Data Infrastructure)
--- ./gradlew clean :samza-sql-core:build Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 33761: Fix SAMZA-658

2015-05-04 Thread Yi Pan (Data Infrastructure)
a/storage/kv/KeyValueStorageEngine.scala <https://reviews.apache.org/r/33761/#comment133095> nit: same here. - Yi Pan (Data Infrastructure) On May 1, 2015, 6:43 p.m., Guozhang Wang wrote: > > --- > This is an automatically genera

Re: Review Request 33146: New KeyValueStore Features

2015-05-04 Thread Yi Pan (Data Infrastructure)
DB and call getAll() vs many get() that directly hitting RocksDB APIs? - Yi Pan (Data Infrastructure) On May 4, 2015, 4:27 a.m., Mohamed Mahmoud (El-Geish) wrote: > > --- > This is an automatically generated e-mail. To reply,

Re: Review Request 33146: New KeyValueStore Features

2015-05-04 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33146/#review82432 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On May 4, 2015

Re: Review Request 33146: New KeyValueStore Features

2015-05-04 Thread Yi Pan (Data Infrastructure)
> On May 4, 2015, 8:14 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-test/src/main/scala/org/apache/samza/test/performance/TestKeyValuePerformance.scala, > > line 320 > > <https://reviews.apache.org/r/33146/diff/5-6/?file=943969#file943969line320> > > >

Re: Review Request 33761: Fix SAMZA-658

2015-05-05 Thread Yi Pan (Data Infrastructure)
to: K) = { metrics.ranges.inc flush() getIter(store.range(from, to)) } samza-kv/src/test/scala/org/apache/samza/storage/kv/TestCachedStore.scala <https://reviews.apache.org/r/33761/#comment133362> It would be better to test iter.remove() for both range() and all(

Re: Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-06 Thread Yi Pan (Data Infrastructure)
e-mail. To reply, visit: https://reviews.apache.org/r/33749/#review82673 ------- On May 4, 2015, 6:58 a.m., Yi Pan (Data Infrastructure) wrote: > > --- > This is an auto

Re: Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-06 Thread Yi Pan (Data Infrastructure)
parameter in doc comment. Thanks! Will fix. - Yi --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33749/#review82674 ----------- On May 4, 2015,

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-06 Thread Yi Pan (Data Infrastructure)
-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala <https://reviews.apache.org/r/33453/#comment133550> Should be "default" here. - Yi Pan (Data Infrastructure) On May 6, 2015, 6:22 a.m., Nav

Re: Review Request 33761: Fix SAMZA-658

2015-05-06 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33761/#review82777 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On May 6, 2015

Re: Review Request 33488: SAMZA-657

2015-05-06 Thread Yi Pan (Data Infrastructure)
ain/java/org/apache/samza/test/integration/join/Emitter.java <https://reviews.apache.org/r/33488/#comment133628> nit: There are still many trailing white spaces. We should remove them. - Yi Pan (Data Infrastructure) On April 27, 2015, 7:59 p.m., G

Re: Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-07 Thread Yi Pan (Data Infrastructure)
g to field X in A and field Y in B, the validation should fail. - Yi --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33749/#review82821 --------

Re: Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-07 Thread Yi Pan (Data Infrastructure)
ze. - Yi --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33749/#review82824 ----------- On May 4, 2015, 6:58 a.m., Yi Pan (Data Infrastructure) w

Re: Review Request 33735: RocksDB TTL support

2015-05-07 Thread Yi Pan (Data Infrastructure)
ry catch the exception here and log an error, in case the db open failed. - Yi Pan (Data Infrastructure) On May 6, 2015, 8:55 p.m., Naveen Somasundaram wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > ht

<    1   2   3   4   5   6   7   8   9   10   >