Re: Change default min.insync.replicas for cluster

2015-04-10 Thread Bryan Baugher
To answer my own question via testing, setting min.insync.replicas on the
broker does not change the default. The only way I can find to change this
is by editing the topic config.

On Fri, Apr 10, 2015 at 12:57 PM Kamal C kamaltar...@gmail.com wrote:

 
 
 
  Similarly my producers are currently using request.required.acks to 2
 with
  our topics having a replication factor of 3 to kind of get the same thing
  as min.insync.replicas = 2. I know this is no longer allowed but will my
  producers fail to write with request.required.acks = 2 in a 0.8.2.1
  cluster?
 
 
 There was a https://issues.apache.org/jira/browse/KAFKA-1697 JIRA filed to
 remove this support. Kafka 0.8.2.1 will log a warning if ack1. In the new
 Shiny Producer 0.8.2.1, you can wait on the returned future to know whether
 the message / record successfully reached to Kafka broker or not.

 producer.send(new ProducerRecordbyte[], byte[](the-topic,
 key.getBytes(), value.getBytes()).get();


 --
 Kamal



Re: [DISCUSS] KIP-14 Tools Standardization

2015-04-10 Thread Ewen Cheslack-Postava
Command line tools are definitely public interfaces. They should get the
same treatment as any other public interface like the APIs or protocols.
Improving and standardizing them is the right thing to do, but
compatibility is still important. Changes should really come with a
well-documented period of deprecation before any old flags are removed.

On Fri, Apr 10, 2015 at 10:26 AM, Andrew Otto ao...@wikimedia.org wrote:

 Col, that looks great!


  On Apr 10, 2015, at 12:48, Joe Stein joe.st...@stealth.ly wrote:
 
  With KIP-4 the ability to write you admin client in any language you want
  (including building your own REST interface) is possible.
 
  The new Apache project's tools admin client is both an Interactive Shell
  and CLI
 
 https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-4.InteractiveShell/CLItool
  that was geared to solve exaclty what I think the heart of KIP-14 is
  getting at here... so... maybe it already has everything you are looking
  for?
 
  ~ Joe Stein
  - - - - - - - - - - - - - - - - -
 
   http://www.stealth.ly
  - - - - - - - - - - - - - - - - -
 
  On Fri, Apr 10, 2015 at 10:28 AM, Wesley Chow w...@chartbeat.com wrote:
 
  +1 on the wikimedia kafka tool. We use it exclusively.
 
  Wes
  On Apr 10, 2015 9:32 AM, Krishna Kumar kku...@nanigans.com wrote:
 
  That seems like a better idea. It preserves the backward compatibility
 of
  existing tools, which can be updated with a warning to use the new
  commands, and that they will be retired in version 1.x - so that there
 is
  no confusion. Also avoids the one-off problem since there will only be
  one
  tool. Documentation also can be simplified.
 
 
  On 4/10/15, 9:21 AM, Andrew Otto ao...@wikimedia.org wrote:
 
  (WARNING: Unrelated but kinda related post below!)
 
 
  Cough cough, ditch the myriad of individual scripts and standardize in
  just one, or a few, that take subcommands, cough cough
  :)
 
 
  E.g.
 
 
 
 
 https://github.com/wikimedia/operations-debs-kafka/blob/debian/debian/bin/
  kafka
 
  kafka console-consumer --topic foo
 
  with ZOOKEEPER_URL as an env var is so much nicer!
 
  -Ao
 
 
  On Apr 10, 2015, at 06:
 
  00, Steve Miller st...@idrathernotsay.com wrote:
 
   I think people will thank you for fixing the inconsistent names,
  sure, but even if you just break their test tools they won't thank
 you
  for the firedrill while they revamp the testing or monitoring stuff
  they
  did on top of the current tools.  I'd rather have a somewhat-icky
 usage
  string than have to drop everything that needs tweaking, all at once.
 
  -Steve
 
  On Thu, Apr 09, 2015 at 08:56:31PM -0700, Jay Kreps wrote:
  Personally I think this is one where most people would thank us for
  fixing
  the random inconsistent names, and aside from MM most of the tools
  effected
  are just test tools.
 
  I do think jopt-simple supports providing multiple names for the
 same
  option so we could retain the old names, not sure if that screws up
  the
  usage message though.
 
  -Jay
 
  On Thu, Apr 9, 2015 at 12:40 AM, Steve Miller
  st...@idrathernotsay.com
  wrote:
 
  FWIW I like the standardization idea but just making the old
  switches
  fail
  seems like it's not the best plan.  People wrap this sort of thing
  for any
  number of reasons, and breaking all of their stuff all at once is
  not
  going
  to make them happy.  And it's not like keeping the old switches
  working for
  a while is all that challenging from a technical standpoint.
 
  Even if all this does is break stuff when you finally phase out the
  old
  switches, telling people that will happen and giving them time to
  adjust
  will make them a lot less annoyed with the Kafka community when
 that
  happens.  They may still be annoyed, mind you, just not at you.
 (-:
 
-Steve
 
 
 
  On Apr 8, 2015, at 10:56 PM, Matthew Warhaftig 
  mwarhaf...@gmail.com
 
  wrote:
 
  The Tool Standardization KIP that Jiangjie started has been
 updated
  to
  contain proposal details:
 
 
 
 
 
 https://cwiki.apache.org/confluence/display/KAFKA/KIP-14+-+Tools+Standa
  rdization
 
  Any feedback is appreciated.
 
  Thanks,
  Matt
 
 
 
 
 




-- 
Thanks,
Ewen