just VPN'ed into my workstation:
the answer to 5 is [*yes*]
answer to 1,2:
the error's I see on the python client are first timeouts and then message
send failures, using sync send.
on the controller log:
ontroller.log.2014-08-26-13:[2014-08-26 13:40:44,317] ERROR
[Controller-1-to-broker-3-send-thread], Controller 1 epoch 3 failed to send
StopReplica request with correlation id 519 to broker
id:3,host:shlomi-kafka-broker-3,port:9092. Reconnecting to broker.
(kafka.controller.RequestSendThread)
controller.log.2014-08-26-13:[2014-08-26 13:40:44,319] ERROR
[Controller-1-to-broker-3-send-thread], Controller 1's connection to broker
id:3,host:shlomi-kafka-broker-3,port:9092 was unsuccessful
(kafka.controller.RequestSendThread)
on the server log (selected greps):
...
server.log.2014-08-27-01:[2014-08-27 01:44:23,143] ERROR
[ReplicaFetcherThread-4-2], Error for partition [vpq_android_gcm_h,270] to
broker 2:class kafka.common.NotLeaderForPartitionException
(kafka.server.ReplicaFetcherThread)
...
server.log.2014-08-27-12:[2014-08-27 12:08:34,638] ERROR Closing socket for
/10.184.150.54 because of error (kafka.network.Processor)
...
server.log.2014-08-28-07:[2014-08-28 07:57:35,944] ERROR [KafkaApi-1] Error
when processing fetch request for partition [vpq_android_gcm_h,184] offset
8798 from follower with correlation id 0 (kafka.server.KafkaApis)
...
erver.log.2014-09-03-15:[2014-09-03 15:46:18,220] ERROR
[ReplicaFetcherThread-2-3], Error in fetch Name: FetchRequest; Version: 0;
CorrelationId: 177593; ClientId: ReplicaFetcherThread-2-3; ReplicaId: 1;
MaxWait: 1000 ms; MinBytes: 1 bytes; RequestInfo: [vpq_android_gcm_h,196]
-> PartitionFetchInfo(65283,8388608),[vpq_android_gcm_h,76] ->
PartitionFetchInfo(262787,8388608),[vpq_android_gcm_h,460] ->
PartitionFetchInfo(285709,8388608),[vpq_android_gcm_h,100] ->
PartitionFetchInfo(199405,8388608),[vpq_android_gcm_h,148] ->
PartitionFetchInfo(339032,8388608),[vpq_android_gcm_h,436] ->
PartitionFetchInfo(0,8388608),[vpq_android_gcm_h,124] ->
PartitionFetchInfo(484447,8388608),[vpq_android_gcm_h,484] ->
PartitionFetchInfo(105945,8388608),[vpq_android_gcm_h,340] ->
PartitionFetchInfo(0,8388608),[vpq_android_gcm_h,388] ->
PartitionFetchInfo(9,8388608),[vpq_android_gcm_h,316] ->
PartitionFetchInfo(194766,8388608),[vpq_android_gcm_h,364] ->
PartitionFetchInfo(139897,8388608),[vpq_android_gcm_h,292] ->
PartitionFetchInfo(195408,8388608),[vpq_android_gcm_h,28] ->
PartitionFetchInfo(329961,8388608),[vpq_android_gcm_h,172] ->
PartitionFetchInfo(436959,8388608),[vpq_android_gcm_h,268] ->
PartitionFetchInfo(59827,8388608),[vpq_android_gcm_h,244] ->
PartitionFetchInfo(259731,8388608),[vpq_android_gcm_h,220] ->
PartitionFetchInfo(61669,8388608),[vpq_android_gcm_h,412] ->
PartitionFetchInfo(563609,8388608),[vpq_android_gcm_h,4] ->
PartitionFetchInfo(360336,8388608),[vpq_android_gcm_h,52] ->
PartitionFetchInfo(378533,8388608) (kafka.server.ReplicaFetcherThread)
...
server.log.2014-09-03-14:[2014-09-03 14:04:18,548] ERROR Error in acceptor
(kafka.network.Acceptor)
...
On Sat, Sep 6, 2014 at 5:48 PM, Shlomi Hazan <shl...@viber.com> wrote:
> Hi and sorry for the late response I just got into the weekend and still
> Satdurday here...
> Well, not at my desk but will answer what I can:
> 1. what else on the logs? [*will vpn and check*]
> 2. other broker failure reason? [*"*]
> 3. other broker failure after taking leadership? [*how can I be sure? ask
> another to describe topic?*]
> 4. how do I measure number of connections? [*ls -l /proc/<pid>/fd | grep
> socket | wc -l, also did watch on that*]
> 5. is that number equals the number of {new Producer}?
> 6. how many topics? [*1*] how many partitions [*504*]
> 7. Are u using a partition key? [*yes, I use the python client with* ]
>
>
>
>
>
>
>
>
>
> *class ProducerIdPartitioner(Partitioner): """ Implements a
> partitioner which selects the target partition based on the sending
> producer ID """ def partition(self, key, partitions): size =
> len(partitions) prod_id = int(key) idx = prod_id %
> size return partitions[idx]*
> 8. maybe running into over partitioned topic? [*producer instances is 6
> machines * 84 procs * 24 threads, but never got to start them all*,*b/c
> of errors*]
> 9. r u running anything else? [*yes, zookeeper*]
>
> additional:
> do you want broker or other config? EC2 instance types? anythying else?....
> Thanks,
> Shlomi
>
>
> On Thu, Sep 4, 2014 at 10:02 PM, Joe Stein <joe.st...@stealth.ly> wrote:
>
>> I think it sounds more like another issue than your thinking...the broker
>> should not be failing like that especially another broker being affected
>> doesn't make sense.
>>
>> What else is in the logs on failure? Is the other broker failing because
>> of number of files too? Is it happening after it becomes the leader?
>>
>> How are you measuring number of connections? Is this how many producer
>> connections you are opening up yourself (new Producer())?
>>
>> How many topics do you have? How many partitions?
>>
>> Are you using a partition key? Maybe you are running into an over
>> partitioned topic type of scenario where your # of producer instances
>> multiplied by # topics is the problem (where the producer will open up
>> it's
>> own connection over time to brokers for each partition leader).
>>
>> Are you running anything else besides kafka broker on the box?
>>
>> /*******************************************
>> Joe Stein
>> Founder, Principal Consultant
>> Big Data Open Source Security LLC
>> http://www.stealth.ly
>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> ********************************************/
>>
>>
>> On Thu, Sep 4, 2014 at 2:06 PM, Shlomi Hazan <shl...@viber.com> wrote:
>>
>> > I sure did. the reason I am building is trying to patch some.
>> specifically
>> > this : KAFKA-1623.
>> > actually if I felt more confident about scala, I would happily send you
>> a
>> > patch.
>> > If you don't care screening, just tell me how to prep it for ya and i
>> will.
>> > The bigger problem is running into too many open files when ramping up
>> to
>> > several thousand connection.
>> > going above 4k is hard and around 6K the broker says goodbye. sometimes
>> > taking a broker friend for a ride....
>> > This was what led me to the acceptor and it's being slow..
>> > Have you got an idea why 10K connection should pose a problem. ulimit
>> > checked. not that. ??
>> > Shlomi
>> >
>> >
>> > On Thu, Sep 4, 2014 at 7:00 PM, Joe Stein <joe.st...@stealth.ly> wrote:
>> >
>> > > Have you tried using a binary release
>> > > http://kafka.apache.org/downloads.html
>> > > this way you don't have to-do a build?
>> > >
>> > > We build using JDK 6 you should be able to run in 8 (I know for sure
>> 6 &
>> > 7
>> > > work honestly never tried 8).
>> > >
>> > > I just did a quick test with a broker running on 8 and
>> produced/consumed
>> > a
>> > > few message didn't run into issues...
>> > >
>> > > As for building in JDK 8 => I reproduced your issue and created a
>> ticket
>> > > https://issues.apache.org/jira/browse/KAFKA-1624 patches are
>> welcomed.
>> > >
>> > > /*******************************************
>> > > Joe Stein
>> > > Founder, Principal Consultant
>> > > Big Data Open Source Security LLC
>> > > http://www.stealth.ly
>> > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> > > ********************************************/
>> > >
>> > >
>> > > On Thu, Sep 4, 2014 at 11:22 AM, Shlomi Hazan <shl...@viber.com>
>> wrote:
>> > >
>> > > > ./gradlew -PscalaVersion=2.9.2 clean jar failed with JDK 8. (error:
>> > error
>> > > > while loading CharSequence, class file
>> > > >
>> '/usr/java/jdk1.8.0_20/jre/lib/rt.jar(java/lang/CharSequence.class)' is
>> > > > broken)
>> > > > I understand there's no escape from installing JDK 7?
>> > > > 10x
>> > > > Shlomi
>> > > >
>> > > >
>> > > > On Thu, Sep 4, 2014 at 6:11 PM, Joe Stein <joe.st...@stealth.ly>
>> > wrote:
>> > > >
>> > > > > When building you need to use the ./gradelw script as Harsha said.
>> > > > Please
>> > > > > take a look at the README for specific commands and how to run
>> them.
>> > > > >
>> > > > > /*******************************************
>> > > > > Joe Stein
>> > > > > Founder, Principal Consultant
>> > > > > Big Data Open Source Security LLC
>> > > > > http://www.stealth.ly
>> > > > > Twitter: @allthingshadoop <
>> http://www.twitter.com/allthingshadoop>
>> > > > > ********************************************/
>> > > > >
>> > > > >
>> > > > > On Thu, Sep 4, 2014 at 10:59 AM, Shlomi Hazan <shl...@viber.com>
>> > > wrote:
>> > > > >
>> > > > > > it failed with JDK 8 so I hoped a newer gradle will maybe do the
>> > > magic,
>> > > > > and
>> > > > > > stepped into this other problem.
>> > > > > > I assume you will say :
>> > > > > > install JDK 7 and build with our gradle 1.6.
>> > > > > > is it so?
>> > > > > > Shlomi
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Sep 4, 2014 at 5:41 PM, Harsha <ka...@harsha.io> wrote:
>> > > > > >
>> > > > > > > Did you tried "gradlew" script in kafka source dir.
>> > > > > > > -Harsha
>> > > > > > >
>> > > > > > > On Thu, Sep 4, 2014, at 07:32 AM, Shlomi Hazan wrote:
>> > > > > > > > what gradle version is used to build kafka_2.9.2-0.8.1.1 ?
>> > > > > > > >
>> > > > > > > > tried with v2 and failed with :
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > gradle --stacktrace clean
>> > > > > > > >
>> > > > > > > > FAILURE: Build failed with an exception.
>> > > > > > > >
>> > > > > > > > * Where:
>> > > > > > > > Build file
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> '/home/shlomi/0dec0xb/project/vpmb/master/3rdparty/kafka/code/kafka-0.8.1.1-src/build.gradle'
>> > > > > > > > line: 34
>> > > > > > > >
>> > > > > > > > * What went wrong:
>> > > > > > > > A problem occurred evaluating root project
>> 'kafka-0.8.1.1-src'.
>> > > > > > > > > Could not find method add() for arguments [licenseMain,
>> class
>> > > > > > > > nl.javadude.gradle.plugins.license.License] on task set.
>> > > > > > > >
>> > > > > > > > * Try:
>> > > > > > > > Run with --info or --debug option to get more log output.
>> > > > > > > >
>> > > > > > > > * Exception is:
>> > > > > > > > org.gradle.api.GradleScriptException: A problem occurred
>> > > evaluating
>> > > > > > root
>> > > > > > > > project 'kafka-0.8.1.1-src'.
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:54)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.apply(DefaultScriptPluginFactory.java:187)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:39)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:26)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.configuration.project.ConfigureActionsProjectEvaluator.evaluate(ConfigureActionsProjectEvaluator.java:34)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.configuration.project.LifecycleProjectEvaluator.evaluate(LifecycleProjectEvaluator.java:55)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:470)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:79)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.configuration.DefaultBuildConfigurer.configure(DefaultBuildConfigurer.java:31)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:128)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:105)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:85)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:81)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:39)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:29)
>> > > > > > > > at
>> > > > > >
>> org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:50)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
>> > > > > > > > at org.gradle.launcher.Main.doAction(Main.java:33)
>> > > > > > > > at
>> > > > > org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
>> > > > > > > > at
>> org.gradle.launcher.GradleMain.main(GradleMain.java:23)
>> > > > > > > > Caused by: org.gradle.api.internal.MissingMethodException:
>> > Could
>> > > > not
>> > > > > > find
>> > > > > > > > method add() for arguments [licenseMain, class
>> > > > > > > > nl.javadude.gradle.plugins.license.License] on task set.
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.AbstractDynamicObject.methodMissingException(AbstractDynamicObject.java:68)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.AbstractDynamicObject.invokeMethod(AbstractDynamicObject.java:56)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.CompositeDynamicObject.invokeMethod(CompositeDynamicObject.java:172)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.tasks.DefaultTaskContainer_Decorated.invokeMethod(Unknown
>> > > > > > > > Source)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> nl.javadude.gradle.plugins.license.LicensePlugin$_configureSourceSetRule_closure6_closure18.doCall(LicensePlugin.groovy:117)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.ClosureBackedAction.execute(ClosureBackedAction.java:59)
>> > > > > > > > at
>> > > > > > > >
>> > > > > >
>> > org.gradle.listener.ActionBroadcast.execute(ActionBroadcast.java:39)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.DefaultDomainObjectCollection.doAdd(DefaultDomainObjectCollection.java:164)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.DefaultDomainObjectCollection.add(DefaultDomainObjectCollection.java:159)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.DefaultNamedDomainObjectCollection.add(DefaultNamedDomainObjectCollection.java:71)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.AbstractNamedDomainObjectContainer.create(AbstractNamedDomainObjectContainer.java:58)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.AbstractNamedDomainObjectContainer.create(AbstractNamedDomainObjectContainer.java:40)
>> > > > > > > > at
>> > > > > > > >
>> > > > > >
>> > > >
>> >
>> org.gradle.api.plugins.JavaPlugin.configureSourceSets(JavaPlugin.java:80)
>> > > > > > > > at
>> > > org.gradle.api.plugins.JavaPlugin.apply(JavaPlugin.java:68)
>> > > > > > > > at
>> > > org.gradle.api.plugins.JavaPlugin.apply(JavaPlugin.java:46)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.plugins.DefaultPluginContainer.providePlugin(DefaultPluginContainer.java:110)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.plugins.DefaultPluginContainer.addPluginInternal(DefaultPluginContainer.java:69)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.plugins.DefaultPluginContainer.apply(DefaultPluginContainer.java:35)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.plugins.DefaultObjectConfigurationAction.applyPlugin(DefaultObjectConfigurationAction.java:117)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.plugins.DefaultObjectConfigurationAction.access$200(DefaultObjectConfigurationAction.java:36)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.plugins.DefaultObjectConfigurationAction$3.run(DefaultObjectConfigurationAction.java:85)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.plugins.DefaultObjectConfigurationAction.execute(DefaultObjectConfigurationAction.java:130)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.project.AbstractPluginAware.apply(AbstractPluginAware.java:41)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> build_6i3s72vsr5r4u0gb7m8oo0piuq$_run_closure2.doCall(/home/shlomi/0dec0xb/project/vpmb/master/3rdparty/kafka/code/kafka-0.8.1.1-src/build.gradle:34)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.ClosureBackedAction.execute(ClosureBackedAction.java:59)
>> > > > > > > > at
>> > > > > org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:130)
>> > > > > > > > at
>> > > > org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:91)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.project.AbstractProject.configure(AbstractProject.java:785)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.project.AbstractProject.configure(AbstractProject.java:790)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.project.AbstractProject.subprojects(AbstractProject.java:773)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:225)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.BeanDynamicObject.invokeMethod(BeanDynamicObject.java:129)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.api.internal.CompositeDynamicObject.invokeMethod(CompositeDynamicObject.java:147)
>> > > > > > > > at
>> > > > > > > >
>> > > > > >
>> > > >
>> > org.gradle.groovy.scripts.BasicScript.methodMissing(BasicScript.java:79)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> build_6i3s72vsr5r4u0gb7m8oo0piuq.run(/home/shlomi/0dec0xb/project/vpmb/master/3rdparty/kafka/code/kafka-0.8.1.1-src/build.gradle:33)
>> > > > > > > > at
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:52)
>> > > > > > > > ... 31 more
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > 10x
>> > > > > > > > Shlomi
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
