Just found a paper from Google in this year's SIGMOD accepted papers about
joining continuous data streams:
Photon: Fault-tolerant and scalable joining of continuous data streams.
[slides: http://cloud.berkeley.edu/data/photon.pdf].
It uses sharding + Paxos to ensure scalability and
Hi Tim,
If you want to just change the status to patch available after you have
upload the patch file, you can just click the submit patch option to
change the status.
Guozhang
On Mon, Sep 2, 2013 at 9:39 PM, Jun Rao jun...@gmail.com wrote:
Hi, Tim,
Thanks for the filing the jira. Seems
/14041/diff/
Testing
---
unit tests
Thanks,
Guozhang Wang
e7a692a1d23ca5a9ecf86e3cb34be418b9c0c943
Diff: https://reviews.apache.org/r/14041/diff/
Testing
---
unit tests
Thanks,
Guozhang Wang
/ZookeeperConsumerConnector.scala
e7a692a1d23ca5a9ecf86e3cb34be418b9c0c943
Diff: https://reviews.apache.org/r/14041/diff/
Testing
---
unit tests
Thanks,
Guozhang Wang
/ClientUtils.scala
cc526ec933052b239f0e7ce43e76cd9d011d5bd9
core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala
e7a692a1d23ca5a9ecf86e3cb34be418b9c0c943
Diff: https://reviews.apache.org/r/14041/diff/
Testing
---
unit tests
Thanks,
Guozhang Wang
Same concern. It seems that ReviewBoard would not, say, mark the JIRA as
patch available when request is published. Does that mean we have to keep
doing this manually still?
Guozhang
On Sun, Sep 8, 2013 at 10:04 AM, Neha Narkhede neha.narkh...@gmail.comwrote:
This is great. So far I reviewed
/consumer/ZookeeperConsumerConnector.scala
e7a692a1d23ca5a9ecf86e3cb34be418b9c0c943
Diff: https://reviews.apache.org/r/14041/diff/
Testing
---
unit tests
Thanks,
Guozhang Wang
., Guozhang Wang wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14041/
---
(Updated Sept. 10, 2013, 6:29 p.m.)
Review
if one is
specified.
-Jay
On Mon, Sep 9, 2013 at 2:32 PM, Guozhang Wang wangg...@gmail.com
wrote:
Same concern. It seems that ReviewBoard would not, say, mark the JIRA
as
patch available when request is published. Does that mean we have to
keep
doing this manually still
can at least force -v to
be required and then if -r is not set ignore this parameter.
- Guozhang Wang
On Sept. 12, 2013, 9:23 p.m., Neha Narkhede wrote:
---
This is an automatically generated e-mail. To reply, visit:
https
Hi Anuj,
The LeaderNotLocalException could be transient:
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-WhydoIseelotsofLeadernotlocalexceptionsonthebrokerduringcontrolledshutdown%3F
Guozhang
On Fri, Sep 13, 2013 at 12:21 AM, Anuj Mehta mehta.r...@gmail.com wrote:
Hi
I am a
/#review26022
---
On Sept. 10, 2013, 6:29 p.m., Guozhang Wang wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14041
/browse/KAFKA-955
Repository: kafka
Description
---
Diffs
-
core/src/main/scala/kafka/api/ProducerRequest.scala
fda3e39184307798f52083a42cf8468b3fb134d7
Diff: https://reviews.apache.org/r/14140/diff/
Testing
---
Thanks,
Guozhang Wang
For 1. I ran into this also and I was using Python 2.6.6, so I downloaded
argparse and then it works:
https://pypi.python.org/pypi/argparse
Guozhang
On Sun, Sep 15, 2013 at 6:04 PM, Neha Narkhede (JIRA) j...@apache.orgwrote:
[
a rebase is needed.
- Guozhang Wang
On Sept. 16, 2013, 5:03 p.m., Swapnil Ghike wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14149
81bf0bda3229e94ecb6b6aff3ffc9fde852df61b
Diff: https://reviews.apache.org/r/14041/diff/
Testing
---
unit tests
Thanks,
Guozhang Wang
Hello Sam,
I agree that with the fix we still should sort the partition list before
hand it to the assignment algorithm. I will try to make a follow-up patch
to fix this.
Guozhang
On Thu, Sep 19, 2013 at 8:17 AM, Sam Meder sam.me...@jivesoftware.comwrote:
The latest consumer changes to read
I think Joe's suggesting that we can remove the checking logic for
key==null in DefaultEventHandler, and do that in partitioner.
One thing about this idea is any customized partitioner also has to
consider key == null case then.
Guozhang
On Fri, Sep 27, 2013 at 9:12 PM, Jun Rao
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14399/#review26469
---
Ship it!
Ship It!
- Guozhang Wang
On Sept. 29, 2013, 11:25 p.m
://reviews.apache.org/r/14497/diff/
Testing
---
Thanks,
Guozhang Wang
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14496/#review26703
---
Ship it!
Agree with Swapnil.
- Guozhang Wang
On Oct. 4, 2013, 9
On Oct. 6, 2013, 8 p.m., Swapnil Ghike wrote:
core/src/main/scala/kafka/admin/ReassignPartitionsCommand.scala, lines 61-65
https://reviews.apache.org/r/14496/diff/2/?file=361683#file361683line61
Instead of asking the user to store the output of a dryrun into a JSON
file, should
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14497/#review26705
---
On Oct. 4, 2013, 10:23 p.m., Guozhang Wang wrote
path is only set in reinitializeConsumer() which is invoked
on startup and when new topics are discovered through wildcards. Doesn't
zkclient automatically re-register watches once they are fired ?
Guozhang Wang wrote:
I am not sure if the zkClient auto re-register upon watcher fired
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14497/#review26712
---
On Oct. 4, 2013, 10:23 p.m., Guozhang Wang wrote
08b4b7218f62f876eb95628bd6db78c0956c7f04
Diff: https://reviews.apache.org/r/14497/diff/
Testing
---
Thanks,
Guozhang Wang
08b4b7218f62f876eb95628bd6db78c0956c7f04
Diff: https://reviews.apache.org/r/14497/diff/
Testing
---
Thanks,
Guozhang Wang
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14534/#review26774
---
Ship it!
Ship It!
- Guozhang Wang
On Oct. 8, 2013, 3:25 p.m
anymore
core/src/test/scala/unit/kafka/admin/AddPartitionsTest.scala
https://reviews.apache.org/r/14554/#comment52176
Why need to increment the added # partitions by 1 here?
- Guozhang Wang
On Oct. 9, 2013, 6:48 a.m., Sriram Subramanian wrote
://reviews.apache.org/r/14641/#comment52584
I thought mkString is used to transfer an iterable to String, not vice
versa?
- Guozhang Wang
On Oct. 15, 2013, 12:31 a.m., Neha Narkhede wrote:
---
This is an automatically generated e-mail
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14676/#review27074
---
Ship it!
Ship It!
- Guozhang Wang
On Oct. 16, 2013, 5:19 p.m
ee1cc0cf451b691eb91d9158ca765aeb60fc3dc8
Diff: https://reviews.apache.org/r/14730/diff/
Testing
---
Thanks,
Guozhang Wang
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14730/#review27195
---
On Oct. 18, 2013, 2:15 a.m., Guozhang Wang wrote
this be included in this RB?
- Guozhang Wang
On Oct. 23, 2013, 4:34 a.m., Neha Narkhede wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14865
://reviews.apache.org/r/14898/diff/
Testing
---
Thanks,
Guozhang Wang
/ReplicaManager.scala
ee1cc0cf451b691eb91d9158ca765aeb60fc3dc8
Diff: https://reviews.apache.org/r/14730/diff/
Testing
---
Thanks,
Guozhang Wang
/14898/diff/
Testing
---
Thanks,
Guozhang Wang
/kafka/server/KafkaRequestHandler.scala
6d562c213b0f55d70341f9d657a7d0c8b212338e
Diff: https://reviews.apache.org/r/14898/diff/
Testing
---
Thanks,
Guozhang Wang
/ReplicaManager.scala
ee1cc0cf451b691eb91d9158ca765aeb60fc3dc8
Diff: https://reviews.apache.org/r/14730/diff/
Testing
---
Thanks,
Guozhang Wang
not actually..
- Guozhang
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14730/#review27553
---
On Oct. 25, 2013, 6:27 p.m., Guozhang Wang wrote
/server/ReplicaManager.scala
ee1cc0cf451b691eb91d9158ca765aeb60fc3dc8
Diff: https://reviews.apache.org/r/14730/diff/
Testing
---
Thanks,
Guozhang Wang
core/src/main/scala/kafka/server/ReplicaManager.scala
ee1cc0cf451b691eb91d9158ca765aeb60fc3dc8
Diff: https://reviews.apache.org/r/14730/diff/
Testing
---
Thanks,
Guozhang Wang
, and hence
step 1/2 will be skipped, is that right?
core/src/main/scala/kafka/controller/KafkaController.scala
https://reviews.apache.org/r/14865/#comment53850
Will the LeaderAndISR request be rejected by the current leader due to its
leader epoch?
- Guozhang Wang
On Oct. 29, 2013, 5:49
Hello Joe,
Is there way I (as a non-committer) can create/edit a new wiki page today?
What is the process I need to follow to be authorized?
Guozhang
On Mon, Oct 7, 2013 at 5:39 PM, Neha Narkhede neha.narkh...@gmail.comwrote:
+1 on Sriram's suggestion. I'm in favor of all registered kafka
/allthingshadoop
/
On Tue, Oct 29, 2013 at 5:23 PM, Guozhang Wang wangg...@gmail.com wrote:
Hello Joe,
Is there way I (as a non-committer) can create/edit a new wiki page
today?
What is the process I need to follow to be authorized?
Guozhang
The default behavior of the consumer is to consume from the tail of the
partition log, did you continue processing data after the consumers have
created their streams?
Guozhang
On Tue, Oct 29, 2013 at 10:52 AM, Shafaq s.abdullah...@gmail.com wrote:
Hi,
I see the following scenario:
1.
https://reviews.apache.org/r/14865/#comment54324
One thing worth mention here is that previously we are printing
TopicAndPartition.toString and now we are printing Partition.toString, which
will be more verbose. This is good for debugging but may flood state-change-log.
- Guozhang Wang
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15137/#review27969
---
Ship it!
Ship It!
- Guozhang Wang
On Oct. 31, 2013, 10:22 p.m
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14865/#review28030
---
Ship it!
Ship It!
- Guozhang Wang
On Nov. 1, 2013, 4:55 p.m
://reviews.apache.org/r/15274/#comment55061
Shall we use null instead of for deleted config values? Sometime in the
future we may have with some meanings.
- Guozhang Wang
On Nov. 6, 2013, 6:13 p.m., Neha Narkhede wrote
.
- Guozhang Wang
On Nov. 6, 2013, 6:13 p.m., Neha Narkhede wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15274/
---
(Updated
On Nov. 6, 2013, 9:41 p.m., Guozhang Wang wrote:
Topic config change as well as create-topic, add-partition,
partition-reassignment and preferred leader election are all asynchronous
in the sense that the admin command would return immediately and one has to
check himself
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15201/#review28715
---
Ship it!
Ship It!
- Guozhang Wang
On Nov. 11, 2013, 4:44 p.m
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15201/#review28872
---
Ship it!
Ship It!
- Guozhang Wang
On Nov. 14, 2013, 4:24 p.m
,
Guozhang Wang
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15665/#review29159
---
Ship it!
- Guozhang Wang
On Nov. 19, 2013, 3:21 a.m., Swapnil
On Nov. 19, 2013, 5:21 p.m., Guozhang Wang wrote:
core/src/main/scala/kafka/server/ZookeeperLeaderElector.scala, line 52
https://reviews.apache.org/r/15665/diff/3/?file=388317#file388317line52
Can we make the version number a global variable, so that when we
upgrade in the future
.
Done.
- Guozhang
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15659/#review29175
---
On Nov. 18, 2013, 11:30 p.m., Guozhang Wang
/test/scala/unit/kafka/consumer/TopicFilterTest.scala
40a2bf7a9277eb5f94bc07b40d7726d81860cefc
system_test/migration_tool_testsuite/0.7/config/test-log4j.properties
a3ae33f20e4b7cff87d8cf8368d0639b8bea73a6
Diff: https://reviews.apache.org/r/15659/diff/
Testing
---
Thanks,
Guozhang
/test/scala/unit/kafka/consumer/TopicFilterTest.scala
40a2bf7a9277eb5f94bc07b40d7726d81860cefc
system_test/migration_tool_testsuite/0.7/config/test-log4j.properties
a3ae33f20e4b7cff87d8cf8368d0639b8bea73a6
Diff: https://reviews.apache.org/r/15659/diff/
Testing
---
Thanks,
Guozhang Wang
to batch election? Also this can help make the handling
function very light so the chance of missing an event can be reduced.
- Guozhang Wang
On Nov. 21, 2013, 5:42 p.m., Sriram Subramanian wrote:
---
This is an automatically generated
/browse/KAFKA-1140
Repository: kafka
Description
---
KAFKA-1140.v1
Dummy
Diffs
-
core/src/main/scala/kafka/consumer/ConsumerIterator.scala
a4227a49684c7de08e07cb1f3a10d2f76ba28da7
Diff: https://reviews.apache.org/r/15805/diff/
Testing
---
Thanks,
Guozhang Wang
a4227a49684c7de08e07cb1f3a10d2f76ba28da7
Diff: https://reviews.apache.org/r/15805/diff/
Testing
---
Thanks,
Guozhang Wang
/kafka/consumer/ConsumerIteratorTest.scala
ef1de8321c713cd9d27ef937216f5b76a5d8c574
Diff: https://reviews.apache.org/r/15805/diff/
Testing
---
Thanks,
Guozhang Wang
Does this enforce the key/value decoder to be passed also into
MessageAndMetadata?
Guozhang
From: Jun Rao [nore...@reviews.apache.org] on behalf of Jun Rao
[jun...@gmail.com]
Sent: Tuesday, November 26, 2013 9:30 AM
To: kafka; Jun Rao; Guozhang Wang
Subject: Re
://reviews.apache.org/r/15805/#review29412
---
On Nov. 25, 2013, 8:55 p.m., Guozhang Wang wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org
/main/scala/kafka/perf/ConsumerPerformance.scala
ec3cd295266e162cd884799b491ae48a51ac8f10
Diff: https://reviews.apache.org/r/15805/diff/
Testing
---
Thanks,
Guozhang Wang
,
Guozhang Wang
/browse/KAFKA-1157
Repository: kafka
Description
---
KAFKA-1157.v1
Diffs
-
core/src/main/scala/kafka/server/KafkaConfig.scala
b324344d0a383398db8bfe2cbeec2c1378fe13c9
Diff: https://reviews.apache.org/r/15950/diff/
Testing
---
Thanks,
Guozhang Wang
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15953/#review29636
---
On Dec. 3, 2013, 12:58 a.m., Guozhang Wang wrote
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15964/#review29777
---
Ship it!
Ship It!
- Guozhang Wang
On Dec. 4, 2013, 12:22 a.m
://reviews.apache.org/r/16005/#comment57258
Could we make the printing of each topic in one line and also in the format
that can be re-used in the topic tool?
- Guozhang Wang
On Dec. 4, 2013, 6:38 p.m., Neha Narkhede wrote
waitUntilInProgressFlusherIsDone passed
another flushing procedure starts?
- Guozhang Wang
On Dec. 4, 2013, 6:15 p.m., Jun Rao wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15674
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16005/#review29828
---
Could we add a unit test for this usage?
- Guozhang Wang
On Dec
Hello Joe,
On the Jira page, you can check to which branch it gets applied to by
checking Fix Version/s.
Guozhang
On Fri, Dec 6, 2013 at 6:15 AM, Joe Stein joe.st...@stealth.ly wrote:
That patch is only on the trunk, will be in 0.8.1
/***
Joe Stein
715845b167c44268bd7e4b76dfb69199bfb2fad4
Diff: https://reviews.apache.org/r/16092/diff/
Testing
---
Thanks,
Guozhang Wang
://reviews.apache.org/r/16095/#comment57415
Same here
- Guozhang Wang
On Dec. 6, 2013, 10:49 p.m., Neha Narkhede wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16095
, visit:
https://reviews.apache.org/r/16092/#review29947
---
On Dec. 6, 2013, 10:34 p.m., Guozhang Wang wrote:
---
This is an automatically generated e-mail. To reply, visit:
https
a7e5b739454aec796070d95b003f20f79c84ef89
Diff: https://reviews.apache.org/r/16092/diff/
Testing
---
Thanks,
Guozhang Wang
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16095/#review29963
---
Ship it!
Ship It!
- Guozhang Wang
On Dec. 7, 2013, 12:22 a.m
/KafkaConfig.scala
a7e5b739454aec796070d95b003f20f79c84ef89
Diff: https://reviews.apache.org/r/16092/diff/
Testing
---
Thanks,
Guozhang Wang
https://reviews.apache.org/r/15953/#comment57578
This it not required, but just to eliminate log flushing due to
fire-all-watcher-at-zk-expiration.
Neha, I think you reviewed the wrong diff, it is diff between r1 and r2.
- Guozhang Wang
On Dec. 5, 2013, 7:13 p.m., Guozhang Wang wrote
/browse/KAFKA-1142
Repository: kafka
Description
---
Dummy
Diffs
-
kafka-patch-review.py 7fa6cb5165d0d497ec3004dc2c98b60fb8d0436d
Diff: https://reviews.apache.org/r/16140/diff/
Testing
---
Thanks,
Guozhang Wang
/browse/KAFKA-1142
Repository: kafka
Description
---
Dummy
Diffs
-
kafka-patch-review.py 7fa6cb5165d0d497ec3004dc2c98b60fb8d0436d
Diff: https://reviews.apache.org/r/16141/diff/
Testing
---
Thanks,
Guozhang Wang
/browse/KAFKA-1142
Repository: kafka
Description
---
KAFKA-1142.v2
Diffs
-
kafka-patch-review.py 7fa6cb5165d0d497ec3004dc2c98b60fb8d0436d
Diff: https://reviews.apache.org/r/16143/diff/
Testing
---
Thanks,
Guozhang Wang
/
Testing (updated)
---
Test passed by
1. Checkout an old version, make some changes.
2. Commit, and call kafka-path-review, it outputs:
ERROR: Your current working branch is from an older version of origin/trunk.
Please rebase first by using git pull --rebase
Thanks,
Guozhang Wang
://reviews.apache.org/r/16092/#review30101
---
On Dec. 9, 2013, 5:14 p.m., Guozhang Wang wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r
://reviews.apache.org/r/16092/diff/
Testing
---
Thanks,
Guozhang Wang
://reviews.apache.org/r/16175/#comment57813
Under which condition the broker will not be in the assigned replica list?
Since the controller have full control of sending LeaderAndIsr request to
brokers, this case sounds more like a bug to me.
- Guozhang Wang
On Dec. 11, 2013, 6:30 a.m., Neha
for a long time, the
connectToBroker can still throw an exception and hence not being able to send
the request. Will this be an issue?
- Guozhang Wang
On Dec. 18, 2013, 12:29 a.m., Neha Narkhede wrote:
---
This is an automatically
://reviews.apache.org/r/16356/#comment58738
We need to check if the value is a single topic string here.
- Guozhang Wang
On Dec. 18, 2013, 9:33 p.m., Neha Narkhede wrote:
---
This is an automatically generated e-mail. To reply
registration path data
is deleted, no matter whether the current broker is the elected leader. That is
not natural to be used as a resignation API do you think?
- Guozhang Wang
On Dec. 18, 2013, 11:29 p.m., Neha Narkhede wrote
://reviews.apache.org/r/16360/diff/
Testing
---
Thanks,
Guozhang Wang
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16357/#review30724
---
Ship it!
- Guozhang Wang
On Dec. 19, 2013, 9:57 p.m., Neha
/ReplicaManager.scala
https://reviews.apache.org/r/16360/#comment58917
This logging for all the partitions.
- Guozhang Wang
On Dec. 19, 2013, 3:01 a.m., Guozhang Wang wrote:
---
This is an automatically generated e-mail. To reply, visit
5c9307d71641ccc6c09a54b69d5aa2b4bc2a4cde
core/src/main/scala/kafka/server/ReplicaManager.scala
242c18d47828b7c5e6b1fc219a0f1199fb1f9512
Diff: https://reviews.apache.org/r/16360/diff/
Testing
---
Thanks,
Guozhang Wang
5c9307d71641ccc6c09a54b69d5aa2b4bc2a4cde
core/src/main/scala/kafka/server/ReplicaManager.scala
242c18d47828b7c5e6b1fc219a0f1199fb1f9512
Diff: https://reviews.apache.org/r/16360/diff/
Testing
---
Thanks,
Guozhang Wang
/browse/KAFKA-1198
Repository: kafka
Description
---
Dummy
KAFKA-1198.v1
Diffs
-
core/src/main/scala/kafka/admin/TopicCommand.scala
083fd632f66c0b48d3d8fe75f3820f8b0e40d291
Diff: https://reviews.apache.org/r/16579/diff/
Testing
---
Thanks,
Guozhang Wang
/r/16579/diff/
Testing
---
Thanks,
Guozhang Wang
1 - 100 of 6729 matches
Mail list logo
