Re: Review Request 29994: WIP: first draft of Schema API and one example implementation via Avro

2015-01-23 Thread Yi Pan (Data Infrastructure)
/test/java/org/apache/samza/sql/task/StoredRuntimeContext.java PRE-CREATION Diff: https://reviews.apache.org/r/29994/diff/ Testing --- Thanks, Yi Pan (Data Infrastructure)

Review Request 30241: [SAMZA-520] TLP changes

2015-01-23 Thread Yi Pan (Data Infrastructure)
025f0557bfcec85b325db4b2475eff633cfe2f4a Diff: https://reviews.apache.org/r/30241/diff/ Testing --- All links tested. Travis/build related links and download links are not updated yet. Need to follow up with that. Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 30241: [SAMZA-520] TLP changes

2015-01-23 Thread Yi Pan (Data Infrastructure)
/watcher.samsa 025f0557bfcec85b325db4b2475eff633cfe2f4a Diff: https://reviews.apache.org/r/30241/diff/ Testing --- All links tested. Travis/build related links and download links are not updated yet. Need to follow up with that. Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 30287: Stream SQL Object Model Draft

2015-01-27 Thread Yi Pan (Data Infrastructure)
d on the output entity that they represent. i.e. essentially, window expression and join expression are expressions that all generate a table-type data source entity. - Yi Pan (Data Infrastructure) On Jan. 26, 2015, 10:04 p.m., Milinda Pathirage wrote: > >

Re: Review Request 29754: StreamSQL operator API draft

2015-01-29 Thread Yi Pan (Data Infrastructure)
ql/src/test/java/org/apache/samza/sql/task/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/29754/diff/ Testing (updated) --- run ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 29754: StreamSQL operator API draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
ile=840328#file840328line37> > > > > StoreMessageCollector? Sounds good to me. - Yi --------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29754/#review70380 ---

Re: Review Request 29754: StreamSQL operator API draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
pache/samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/29754/diff/ Testing --- run ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 30287: Stream SQL Object Model Draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
t; ------- > > (Updated Jan. 26, 2015, 10:04 p.m.) > > > Review request for samza, Chris Riccomini and Yi Pan (Data Infrastructure). > > > Repository: samza > > > Description > --- > > WIP: Stream SQL Obje

Re: Review Request 29754: StreamSQL operator API draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
ava/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/29754/diff/ Testing --- run ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 29754: StreamSQL operator API draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
rg/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/29754/diff/ Testing --- run ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 29754: StreamSQL operator API draft

2015-01-30 Thread Yi Pan (Data Infrastructure)
samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4 Diff: https://reviews.apache.org/r/29754/diff/ Testing --- run ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Review Request 30627: REVISION: operator APIs: rebased and cleaned up

2015-02-04 Thread Yi Pan (Data Infrastructure)
/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION settings.gradle bb07a3b84b14dcef94da1bb166eab6aa3d0026bb Diff: https://reviews.apache.org/r/30627/diff/ Testing --- ./bin/check-all.sh passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 30634: Serde for Stream Tuples

2015-02-04 Thread Yi Pan (Data Infrastructure)
ackage name from org.apache.samza.sql.task to org.apache.samza.task.sql since both tasks are extended from org.apache.samza.task.StreamTask. The same for the StreamSqlTask. - Yi Pan (Data Infrastructure) On Feb. 4, 2015, 7:39 p.m., Navina Ramesh wrote: > > --

Review Request 30800: WIP: improving the manual operator interface

2015-02-09 Thread Yi Pan (Data Infrastructure)
/RandomOperatorTask.java PRE-CREATION samza-sql/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION Diff: https://reviews.apache.org/r/30800/diff/ Testing --- ./gradlew clean build passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 31909: SAMZA-590

2015-03-11 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31909/#review76120 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On March 10

Re: Review Request 32006: SAMZA-597

2015-03-13 Thread Yi Pan (Data Infrastructure)
g4j/src/main/java/org/apache/samza/logging/log4j/serializers/LoggingEventJsonSerde.java <https://reviews.apache.org/r/32006/#comment123934> nit: Serde - Yi Pan (Data Infrastructure) On March 13, 2015, 12:57 a.m., Chris Riccomini wrote: > > -

Re: Review Request 32052: SAMZA-592

2015-03-13 Thread Yi Pan (Data Infrastructure)
n metadata is not inserted in the cache, getOffsets would raise exception? And how do we capture that case? - Yi Pan (Data Infrastructure) On March 13, 2015, 7:56 p.m., Chris Riccomini wrote: > > --- > This is an automatically generat

Re: Review Request 32006: SAMZA-597

2015-03-13 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32006/#review76430 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On March 13

Re: Review Request 32052: SAMZA-592

2015-03-13 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32052/#review76437 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On March 13

Re: Review Request 32052: SAMZA-592

2015-03-13 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32052/#review76442 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On March 13

Re: Review Request 32188: Disable WAL in RocksDB KV store

2015-03-18 Thread Yi Pan (Data Infrastructure)
/samza/test/performance/TestKeyValuePerformance.scala <https://reviews.apache.org/r/32188/#comment124695> nit: do we need multiple instances of ByteSerde here? - Yi Pan (Data Infrastructure) On March 18, 2015, 9:48 p.m., Navina Ramesh

Re: Review Request 32188: Disable WAL in RocksDB KV store

2015-03-18 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32188/#review76989 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On March 18

Re: Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

2015-03-20 Thread Yi Pan (Data Infrastructure)
ated? 2. In the code block from 153, wouldn't it be easier to set a noFlushStop flag s.t. the stop() method can avoid calling the flush again when stop() is triggered from failed flush anyways? - Yi Pan (Data Infrastructure) On March 17, 2015, 9:37 a.m.

Re: Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

2015-03-20 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32155/#review77282 --- Ship it! Looks good to me. - Yi Pan (Data Infrastructure) On

Review Request 32407: SAMZA-571: add suppression interface for uncaught exceptions

2015-03-23 Thread Yi Pan (Data Infrastructure)
://reviews.apache.org/r/32407/diff/ Testing --- Unit test added. Pass with ./gradlew clean build Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 32407: SAMZA-571: add suppression interface for uncaught exceptions

2015-03-23 Thread Yi Pan (Data Infrastructure)
f we can live w/ the fact that if user implements ExceptionTask automatically disables all configured suppressions, I would go ahead w/ your suggestion. Thanks! - Yi --- This is an automatically generated e-mail. To reply, visit: https://revi

Re: Review Request 32407: SAMZA-571: add suppression interface for uncaught exceptions

2015-03-24 Thread Yi Pan (Data Infrastructure)
f6521 samza-core/src/test/scala/org/apache/samza/container/TestTaskInstance.scala 54b4df84f47f818d62ac0361196567ad1f430fde Diff: https://reviews.apache.org/r/32407/diff/ Testing (updated) --- Unit tests added. Pass with ./bin/check-all.sh Thanks, Yi Pan (Data Infrastructure)

Review Request 32874: SAMZA-608: fix system consumer stuck issue when discarding the serde exception

2015-04-06 Thread Yi Pan (Data Infrastructure)
/SystemConsumers.scala 125d37602e2c0a9da75674f37580a1ac02f94796 samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala 3fdc781c1275f928f4b51b01869e1122502a2c08 Diff: https://reviews.apache.org/r/32874/diff/ Testing --- passed local unit tests. Thanks, Yi Pan (Data

Review Request 32872: SAMZA-571: add test to capture RecordTooLargeException

2015-04-06 Thread Yi Pan (Data Infrastructure)
d66b3bd070a4cef4b1d3dded1d79a33cbe3fa09b Diff: https://reviews.apache.org/r/32872/diff/ Testing --- Passed local test suite Thanks, Yi Pan (Data Infrastructure)

Review Request 32892: DOCS; updating wiki links to point to cwiki

2015-04-06 Thread Yi Pan (Data Infrastructure)
s.apache.org/r/32892/diff/ Testing --- Passed ./bin/check-all.sh Thanks, Yi Pan (Data Infrastructure)

Review Request 33142: [SAMZA-561] Review in progress

2015-04-13 Thread Yi Pan (Data Infrastructure)
fd41b samza-test/src/main/python/tests/sql_tests.py PRE-CREATION samza-test/src/main/resources/orders.avsc PRE-CREATION samza-test/src/main/resources/orders.json PRE-CREATION Diff: https://reviews.apache.org/r/33142/diff/ Testing --- Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 33170: Renamed samza-sql to samza-sql-core

2015-04-14 Thread Yi Pan (Data Infrastructure)
s://reviews.apache.org/r/33170/#comment129814> Move to samza-sql-calcite samza-sql-core/src/test/java/org/apache/samza/sql/test/metadata/TestAvroSchemaConverter.java <https://reviews.apache.org/r/33170/#comment129816> Same here. - Yi Pan (Data Infrastructure)

Re: Review Request 33142: [SAMZA-561] Review in progress

2015-04-14 Thread Yi Pan (Data Infrastructure)
eFilterableStreamScanSpec.java <https://reviews.apache.org/r/33142/#comment129692> These calcite specific class should be moved out-of samza-sql-core module. - Yi Pan (Data Infrastructure) On April 13, 2015, 9:04 p.m., Yi Pan (Data I

Re: Review Request 33219: [SAMZA-649] Create samza-sql-calcite module for Calcite SQL front end

2015-04-15 Thread Yi Pan (Data Infrastructure)
ma/types in samza-sql-core model to Calcite's RelDataType? In that case, can we use the generic Schema class in samza-sql-core instead of implementation specific for Avro? - Yi Pan (Data Infrastructure) On April 15, 2015, 2:49 p.m.,

Re: Review Request 33219: [SAMZA-649] Create samza-sql-calcite module for Calcite SQL front end

2015-04-15 Thread Yi Pan (Data Infrastructure)
> On April 15, 2015, 6:20 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-sql-calcite/src/main/java/org/apache/samza/sql/calcite/schema/AvroSchemaConverter.java, > > line 37 > > <https://reviews.apache.org/r/33219/diff/1/?file=930371#file930371line37> > > &g

Re: Review Request 33219: [SAMZA-649] Create samza-sql-calcite module for Calcite SQL front end

2015-04-15 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33219/#review80237 --- Ship it! +1 - Yi Pan (Data Infrastructure) On April 15, 2015, 2

Re: Review Request 33146: Adding a new KV store contract: BatchingKeyValueStore

2015-04-15 Thread Yi Pan (Data Infrastructure)
> On April 14, 2015, 9:14 p.m., Chris Riccomini wrote: > > I'm concerned that there might be an issue with this approach. In > > BaseKeyValueStorageEngineFactory, we compose stores by nesting them. If > > this is the case, I think that the top-most store will implement the > > batching key val

Re: Review Request 33219: [SAMZA-649] Create samza-sql-calcite module for Calcite SQL front end

2015-04-15 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33219/#review80248 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On April 15

Re: Review Request 33146: New KeyValueStore Features

2015-04-21 Thread Yi Pan (Data Infrastructure)
test/scala/org/apache/samza/storage/kv/TestKeyValueStores.scala <https://reviews.apache.org/r/33146/#comment131257> nit: prefer not to re-order the methods if not necessary. - Yi Pan (Data Infrastructure) On April 16, 2015,

Re: Review Request 33146: New KeyValueStore Features

2015-04-21 Thread Yi Pan (Data Infrastructure)
> On April 21, 2015, 6:49 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-kv/src/main/java/org/apache/samza/storage/kv/KeyValueStore.java, line > > 33 > > <https://reviews.apache.org/r/33146/diff/2/?file=931566#file931566line33> > > > > The sig

Re: Review Request 33199: Coordinator stream

2015-04-22 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33199/#review81242 --- Ship it! Went through the latest patch w/ Naveen. LGTM - Yi Pan

Re: Review Request 33146: New KeyValueStore Features

2015-04-23 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33146/#review81381 --- Ship it! LGTM. Thanks! - Yi Pan (Data Infrastructure) On April

Re: Review Request 33146: New KeyValueStore Features

2015-04-24 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33146/#review81497 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On April 24

Re: Review Request 33146: New KeyValueStore Features

2015-04-26 Thread Yi Pan (Data Infrastructure)
> On April 24, 2015, 5:01 p.m., Yi Pan (Data Infrastructure) wrote: > > Ship It! > > Mohamed Mahmoud (El-Geish) wrote: > I don't have access to commit. Can you please grant me access or commit > for me? Thanks! Hi, MOhamed, I was trying to go through all the test

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-04-28 Thread Yi Pan (Data Infrastructure)
, here we may be deleting empty/non-existing storagePartitionDirs. - Yi Pan (Data Infrastructure) On April 22, 2015, 9:54 p.m., Navina Ramesh wrote: > > --- > This is an automatically generated e-mail.

Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-01 Thread Yi Pan (Data Infrastructure)
-CREATION Diff: https://reviews.apache.org/r/33749/diff/ Testing --- ./gradlew clean :samza-sql-core:build Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-03 Thread Yi Pan (Data Infrastructure)
--- ./gradlew clean :samza-sql-core:build Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 33761: Fix SAMZA-658

2015-05-04 Thread Yi Pan (Data Infrastructure)
a/storage/kv/KeyValueStorageEngine.scala <https://reviews.apache.org/r/33761/#comment133095> nit: same here. - Yi Pan (Data Infrastructure) On May 1, 2015, 6:43 p.m., Guozhang Wang wrote: > > --- > This is an automatically genera

Re: Review Request 33146: New KeyValueStore Features

2015-05-04 Thread Yi Pan (Data Infrastructure)
DB and call getAll() vs many get() that directly hitting RocksDB APIs? - Yi Pan (Data Infrastructure) On May 4, 2015, 4:27 a.m., Mohamed Mahmoud (El-Geish) wrote: > > --- > This is an automatically generated e-mail. To reply,

Re: Review Request 33146: New KeyValueStore Features

2015-05-04 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33146/#review82432 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On May 4, 2015

Re: Review Request 33146: New KeyValueStore Features

2015-05-04 Thread Yi Pan (Data Infrastructure)
> On May 4, 2015, 8:14 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-test/src/main/scala/org/apache/samza/test/performance/TestKeyValuePerformance.scala, > > line 320 > > <https://reviews.apache.org/r/33146/diff/5-6/?file=943969#file943969line320> > > >

Re: Review Request 33761: Fix SAMZA-658

2015-05-05 Thread Yi Pan (Data Infrastructure)
to: K) = { metrics.ranges.inc flush() getIter(store.range(from, to)) } samza-kv/src/test/scala/org/apache/samza/storage/kv/TestCachedStore.scala <https://reviews.apache.org/r/33761/#comment133362> It would be better to test iter.remove() for both range() and all(

Re: Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-06 Thread Yi Pan (Data Infrastructure)
e-mail. To reply, visit: https://reviews.apache.org/r/33749/#review82673 --------------- On May 4, 2015, 6:58 a.m., Yi Pan (Data Infrastructure) wrote: > > --- > This is an auto

Re: Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-06 Thread Yi Pan (Data Infrastructure)
parameter in doc comment. Thanks! Will fix. - Yi --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33749/#review82674 ----------- On May 4, 2015,

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

2015-05-06 Thread Yi Pan (Data Infrastructure)
-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala <https://reviews.apache.org/r/33453/#comment133550> Should be "default" here. - Yi Pan (Data Infrastructure) On May 6, 2015, 6:22 a.m., Nav

Re: Review Request 33761: Fix SAMZA-658

2015-05-06 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33761/#review82777 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On May 6, 2015

Re: Review Request 33488: SAMZA-657

2015-05-06 Thread Yi Pan (Data Infrastructure)
ain/java/org/apache/samza/test/integration/join/Emitter.java <https://reviews.apache.org/r/33488/#comment133628> nit: There are still many trailing white spaces. We should remove them. - Yi Pan (Data Infrastructure) On April 27, 2015, 7:59 p.m., G

Re: Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-07 Thread Yi Pan (Data Infrastructure)
g to field X in A and field Y in B, the validation should fail. - Yi --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33749/#review82821 ----------------

Re: Review Request 33749: WIP: SAMZA-650 window store implementation

2015-05-07 Thread Yi Pan (Data Infrastructure)
ze. - Yi --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33749/#review82824 --------------- On May 4, 2015, 6:58 a.m., Yi Pan (Data Infrastructure) w

Re: Review Request 33735: RocksDB TTL support

2015-05-07 Thread Yi Pan (Data Infrastructure)
ry catch the exception here and log an error, in case the db open failed. - Yi Pan (Data Infrastructure) On May 6, 2015, 8:55 p.m., Naveen Somasundaram wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > ht

Review Request 34009: WIP: SAMZA-650 window store implementation

2015-05-08 Thread Yi Pan (Data Infrastructure)
/StreamSqlTask.java PRE-CREATION samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION Diff: https://reviews.apache.org/r/34009/diff/ Testing --- Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 34009: WIP: SAMZA-650 window store implementation

2015-05-08 Thread Yi Pan (Data Infrastructure)
/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION Diff: https://reviews.apache.org/r/34009/diff/ Testing (updated) --- ./gradlew clean build passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 33280: [SAMZA-561] Basic streaming SQL query planning support

2015-05-11 Thread Yi Pan (Data Infrastructure)
Question: isn't this operator just a FilterableStreamScanOp? There is no projection expression in the spec. samza-test/src/main/java/org/apache/samza/test/integration/sql/OrdersStreamFactory.java <https://reviews.apache.org/r/33280/#comment134288> Question: do we need to in

Re: Review Request 34009: WIP: SAMZA-650 window store implementation

2015-05-12 Thread Yi Pan (Data Infrastructure)
I use this name to indicate that this is a task that users of operator APIs are implementing a customized OperatorCallback in the SQL task. - Yi --------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3

Re: Review Request 34009: WIP: SAMZA-650 window store implementation

2015-05-12 Thread Yi Pan (Data Infrastructure)
-- On May 9, 2015, 1:52 a.m., Yi Pan (Data Infrastructure) wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/34009/ > ---

Re: Review Request 34009: WIP: SAMZA-650 window store implementation

2015-05-12 Thread Yi Pan (Data Infrastructure)
t: https://reviews.apache.org/r/34009/#review83405 --------------- On May 9, 2015, 1:52 a.m., Yi Pan (Data Infrastructure) wrote: > > --- > This is an automatically generated e-

Re: Review Request 34009: SAMZA-552 window store implementation

2015-05-13 Thread Yi Pan (Data Infrastructure)
/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION Diff: https://reviews.apache.org/r/34009/diff/ Testing --- ./gradlew clean build passed Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 34009: SAMZA-552 window store implementation

2015-05-13 Thread Yi Pan (Data Infrastructure)
e/src/main/java/org/apache/samza/task/sql/RouterMessageCollector.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/task/sql/SimpleMessageCollector.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/task/sql/SqlMessageCollector.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java > PRE-CREATION > > samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java > PRE-CREATION > > samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java > PRE-CREATION > samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java > PRE-CREATION > > samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/34009/diff/ > > > Testing > --- > > ./gradlew clean build passed > > > Thanks, > > Yi Pan (Data Infrastructure) > >

Review Request 34206: WIP: update operator API to allow callbacks and allow a single API to trigger OperatorRouter execution w/ user callbacks

2015-05-14 Thread Yi Pan (Data Infrastructure)
/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION Diff: https://reviews.apache.org/r/34206/diff/ Testing --- Thanks, Yi Pan (Data Infrastructure)

Review Request 34207: SAMZA-552: use OperatorCallback to allow implementation of callbacks w/o inheriting and creating many sub-classes from operators

2015-05-14 Thread Yi Pan (Data Infrastructure)
/samza/task/sql/StreamSqlTask.java PRE-CREATION samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION Diff: https://reviews.apache.org/r/34207/diff/ Testing --- Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 33735: RocksDB TTL support

2015-05-14 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33735/#review83847 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On May 13

Re: Review Request 34206: WIP: update operator API to allow callbacks and allow a single API to trigger OperatorRouter execution w/ user callbacks

2015-05-14 Thread Yi Pan (Data Infrastructure)
/UserCallbacksSqlTask.java PRE-CREATION Diff: https://reviews.apache.org/r/34206/diff/ Testing --- Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 34206: WIP: update operator API to allow callbacks and allow a single API to trigger OperatorRouter execution w/ user callbacks

2015-05-14 Thread Yi Pan (Data Infrastructure)
--- Thanks, Yi Pan (Data Infrastructure)

Re: Review Request 34206: WIP: update operator API to allow callbacks and allow a single API to trigger OperatorRouter execution w/ user callbacks

2015-05-15 Thread Yi Pan (Data Infrastructure)
. To reply, visit: https://reviews.apache.org/r/34206/#review83937 ----------- On May 15, 2015, 2:16 a.m., Yi Pan (Data Infrastructure) wrote: > > --- > This is an au

Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-05-20 Thread Yi Pan (Data Infrastructure)
/samza/task/sql/StreamSqlTask.java PRE-CREATION samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION Diff: https://reviews.apache.org/r/34500/diff/ Testing --- ./gradlew clean build passed Thanks, Yi Pan (Data Infrastructure)

Review Request 34574: SAMZA-608; don't hange on serde errors in system consumers

2015-05-21 Thread Yi Pan (Data Infrastructure)
ystem/SystemConsumers.scala 125d37602e2c0a9da75674f37580a1ac02f94796 samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala 3fdc781c1275f928f4b51b01869e1122502a2c08 Diff: https://reviews.apache.org/r/34574/diff/ Testing --- Thanks, Yi Pan (Data Infrastructure)

Review Request 34664: SAMZA-552 Operator API change: builder and simplified operator classes

2015-05-25 Thread Yi Pan (Data Infrastructure)
commit of the following: commit fad81106901e494d3950eeaafaeefef482ac0125 Author: Yi Pan (Data Infrastructure) Date: Mon May 25 23:40:00 2015 -0700 SAMZA-650 window message store and window store implementation commit 58c2eeebf4bb0975f70aeba733379e1104f3a7de Author: Yi Pan (Data

Re: Review Request 33280: [SAMZA-561] Basic streaming SQL query planning support

2015-05-26 Thread Yi Pan (Data Infrastructure)
il. To reply, visit: > https://reviews.apache.org/r/33280/ > --- > > (Updated May 23, 2015, 4:27 p.m.) > > > Review request for samza, Guozhang Wang and Yi Pan (Data Infrastructure). > > > Bugs: SAMZA-561 > https://issues.apache.org/jira/browse/SAMZA-561 >

Re: Review Request 34626: SAMZA-688 Fix for YarnJobFactory not supporting 'https' filesystem

2015-05-26 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34626/#review85155 --- Ship it! LGTM. Thanks! - Yi Pan (Data Infrastructure) On May 23

Re: Review Request 33280: [SAMZA-561] Basic streaming SQL query planning support

2015-05-27 Thread Yi Pan (Data Infrastructure)
> On May 26, 2015, 7:40 a.m., Yi Pan (Data Infrastructure) wrote: > > build.gradle, line 408 > > <https://reviews.apache.org/r/33280/diff/5/?file=970932#file970932line408> > > > > You might need to re-base this change. > > Milinda Pathirage wrote:

Re: Review Request 34664: SAMZA-650 window store implementation

2015-05-27 Thread Yi Pan (Data Infrastructure)
sub-classes from operators SAMZA-552 update the operator API SAMZA-552: operator builder API update Squashed commit of the following: commit fad81106901e494d3950eeaafaeefef482ac0125 Author: Yi Pan (Data Infrastructure) Date: Mon May 25 23:40:00 2015 -0700 SAMZA-650 window message store

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-05-29 Thread Yi Pan (Data Infrastructure)
This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34500/#review85672 --------------- On May 20, 2015, 11:13 p.m., Yi Pan (Data Infrastructure) wrote: > > -

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-05-29 Thread Yi Pan (Data Infrastructure)
gt; > samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java, > > line 123 > > <https://reviews.apache.org/r/34500/diff/1/?file=965740#file965740line123> > > > > Why do we need 2 instances of the TopologyBuilder here? > &

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-05-29 Thread Yi Pan (Data Infrastructure)
nt OperatorSpec based API > is you need to create intermediate streams (EntityName)s to wire operators > together. I think we should try to hide that complexity through the builder > API. Even though source and sink hides that complexity to some extent, its > better if we

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

2015-05-30 Thread Yi Pan (Data Infrastructure)
ree managers that all requires to initialize the coordinatorSystemProducer/coordinatorSystemConsumer in a single CoordinatorStreamManager? And CoordinatorStreamManager.getCheckpointManager()/getChangelogManager()/getLocalityManager() would return the specific management function handler? - Yi Pan (

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

2015-06-01 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34746/#review86125 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On May 30

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

2015-06-01 Thread Yi Pan (Data Infrastructure)
> On May 30, 2015, 8:58 a.m., Yi Pan (Data Infrastructure) wrote: > > samza-core/src/main/java/org/apache/samza/container/LocalityManager.java, > > line 62 > > <https://reviews.apache.org/r/34746/diff/2/?file=974783#file974783line62> > > > > This woul

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-06-03 Thread Yi Pan (Data Infrastructure)
------------ On May 20, 2015, 11:13 p.m., Yi Pan (Data Infrastructure) wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/34500/ > --

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-06-03 Thread Yi Pan (Data Infrastructure)
will update the RB accordingly. - Yi --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34500/#review86504 --- On May 20,

Re: Review Request 35325: SAMZA-698: update Samza and Spark Streaming Comparison

2015-06-12 Thread Yi Pan (Data Infrastructure)
rivial to claim exact-once under the asumption that the system running healthy, - Yi Pan (Data Infrastructure) On June 12, 2015, 11:54 p.m., Yan Fang wrote: > > --- > This is an automatically generated e-mail. To reply,

Re: Review Request 35325: SAMZA-698: update Samza and Spark Streaming Comparison

2015-06-12 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35325/#review87789 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On June 13

Re: Review Request 35577: Fix SAMZA-714

2015-06-17 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35577/#review88262 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On June 17

Re: Review Request 35397: Fix SAMZA-697

2015-06-19 Thread Yi Pan (Data Infrastructure)
pClassloader { exceptionHandler.maybeHandle { task.process(envelope, collector, coordinator) } } ... - Yi Pan (Data Infrastructure) On June 18, 2015, 6:42 p.m., Guozhang Wang wrote: > > --- > This is a

Re: Review Request 35601: SAMZA-418 Additional information on AM web ui

2015-06-19 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35601/#review88512 --- Ship it! LGTM. Thanks! - Yi Pan (Data Infrastructure) On June

Re: Review Request 35723: SAMZA-720: fix bootstrap hangs when container number > 1

2015-06-22 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35723/#review88820 --- Ship it! +1. LGTM. Thanks for the quick fix, Yan. - Yi Pan (Data

Re: Review Request 35676: Checkpoint migration

2015-07-08 Thread Yi Pan (Data Infrastructure)
the offsets 2. remove all checkpoint.factory configuration 3. restart the job - Yi Pan (Data Infrastructure) On July 8, 2015, 1:41 a.m., Naveen Somasundaram wrote: > > --- > This is an automatically generated e-m

Re: Review Request 35676: Checkpoint migration

2015-07-08 Thread Yi Pan (Data Infrastructure)
Pan (Data Infrastructure) On July 8, 2015, 1:41 a.m., Naveen Somasundaram wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache

Re: Review Request 35676: Checkpoint migration

2015-07-08 Thread Yi Pan (Data Infrastructure)
> On July 8, 2015, 7:34 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-core/src/main/scala/org/apache/samza/migration/JobRunnerMigration.scala, > > line 46 > > <https://reviews.apache.org/r/35676/diff/3/?file=1001916#file1001916line46> > > > > Th

Re: Review Request 35676: Checkpoint migration

2015-07-08 Thread Yi Pan (Data Infrastructure)
review comment. Thanks! samza-kafka/src/main/scala/old/checkpoint/KafkaCheckpointManager.scala (line 236) <https://reviews.apache.org/r/35676/#comment144132> nit: "reading both changelog partition mapping and checkpoint"? - Yi Pan (Data Infrastructure) On July 8, 2015, 1

Re: Review Request 36224: SAMZA-728: Samza job fails due to null pointer in JobCoordinator refreshJobModel

2015-07-09 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36224/#review91248 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On July 6

  1   2   3   4   5   >