Note that the documentation that you linked to for upgrades specifically lists configs that you need to be careful to adjust in your server.properties.
In fact, the server.properties shipped with Kafka is meant for testing only. There are some configs in the example server.properties that are not safe to use in production, for example log.dirs, which I see you've adjusted. That said, we are careful about not changing default values in the actual service without warning. If we change them, it would be during major version changes and we would still warn you about it in the release notes. You are making a big jump (0.8.2-beta to 0.10.x) so you'd want to check the 0.8.2, 0.9, and 0.10 release notes. The most obvious thing I'd expect to be an issue would be the log format changes and inter protocol version settings. These are settings you need to be careful about setting correctly during upgrades and are clearly described in the docs you already linked: http://kafka.apache.org/0100/documentation.html#upgrade -Ewen On Thu, Jan 26, 2017 at 3:22 PM, Fernando Vega <fernando.v...@turn.com> wrote: > I have a few questions regarding an upgrade that Im attempting to perform. > > - Currently we are running version 0.8.2_beta. > - We are trying to upgrade to 10.1.1 > - Our setup uses the following path > /server/kafka > where kafka is a symlink to kafka-{version} > - I attempt to perform the upgrade as you guys specify in the > documentation: > http://kafka.apache.org/0100/documentation.html#upgrade > > However when I attempt this my server.properties file that was in > kafka-0.8.2_beta was gone and a new file was under kafka-0.10.1.1. > > I started the server without noticing this at first and it started > normally. > When I realize about it I stop it and copy the server.properties file from > another host, running the old version into this server in the new directory > and adding the 2 lines mentioned in the instructions above as well as > changing its broker id of course, when I do this the server does not start. > > So my questions are: > > Are any of the config files different from one version to another? > Does the options from one version to another are the same? > > This is a sample of the file that the old cluster uses and I will like to > know if this options are still valid for the new version: > > ## See http://kafka.apache.org/documentation.html#brokerconfigs for > default > values. > > # The id of the broker. This must be set to a unique integer for each > broker. > broker.id=6 > > # The port the socket server listens on > port=9092 > > # A comma seperated list of directories under which to store log files > log.dirs=/data/1/kafka/datalog,/data/2/kafka/datalog,/data/ > 3/kafka/datalog,/data/4/kafka/datalog,/data/5/kafka/datalog, > /data/6/kafka/datalog,/data/7/kafka/datalog,/data/8/kafka/datalog > > # Zookeeper connection string (see zookeeper docs for details). > # This is a comma separated host:port pairs, each corresponding to a zk > # server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002". > # You can also append an optional chroot string to the urls to specify the > # root directory for all kafka znodes. > > zookeeper.connect=dwh-pipeline001:2181,dwh-pipeline002:2181, > dwh-pipeline003:2181,dwh-pipeline004:2181,dwh-pipeline005:2181/kafka/data > > # Additional configuration options may follow here > auto.leader.rebalance.enable=true > delete.topic.enable=true > log.retention.size=20000000000 > socket.receive.buffer.bytes=1048576 > socket.send.buffer.bytes=1048576 > default.replication.factor=2 > auto.create.topics.enable=true > num.partitions=100 > num.network.threads=3 > num.io.threads=40 > log.retention.hours=24 > log.roll.hours=1 > num.replica.fetchers=8 > zookeeper.connection.timeout.ms=30000 > zookeeper.session.timeout.ms=30000 > > Also What are the steps or ways I can check the update work fine and that > Im running x or y version or to determine things are working fine > > Thank you > > [image: Turn] <http://www.turn.com/> > > *Fernando Vega* > Sr. Operations Engineer > *cell* (415) 810-0242 > 901 Marshall St, Suite 200, Redwood City, CA 94063 > > > turn.com <http://www.turn.com/> | @TurnPlatform > <https://twitter.com/@TurnPlatform> > > This message is Turn Confidential, except for information included that is > already available to the public. If this message was sent to you > accidentally, please delete it. >
