Pretty weird, I have the same build.sbt than you :
scalaVersion := "2.11.7"
val kafkaV = "0.10.0.1"
val reactiveKafkaV = "0.13"
libraryDependencies ++= Seq(
"org.apache.kafka" % "kafka-clients" % kafkaV,
"org.apache.kafka" % "kafka_2.11" % kafkaV intransitive,
"com.typesafe.akka" %% "akka-stream-kafka" % reactiveKafkaV
)
publishMavenStyle in ThisBuild := false
But I get fatal exceptions when running my consumer :
[info] Loading project definition from /home/admin/reactive-hosted-kafka-
benchmark/project
[info] Updating {file:/home/admin/reactive-hosted-kafka-benchmark/project/}
reactive-hosted-kafka-benchmark-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] downloading
https://repo1.maven.org/maven2/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar
...
[info] [SUCCESSFUL ]
org.apache.commons#commons-compress;1.4.1!commons-compress.jar
(110ms)
[info] downloading
https://repo1.maven.org/maven2/org/apache/ant/ant/1.9.6/ant-1.9.6.jar
...
[info] [SUCCESSFUL ] org.apache.ant#ant;1.9.6!ant.jar (278ms)
[info] downloading
https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.9.6/ant-launcher-1.9.6.jar
...
[info] [SUCCESSFUL ] org.apache.ant#ant-launcher;1.9.6!ant-launcher.jar
(47ms)
[info] Done updating.
[info] Set current project to reactive-hosted-kafka-benchmark (in build file
:/home/admin/reactive-hosted-kafka-benchmark/)
[info] Updating {file:/home/admin/reactive-hosted-kafka-benchmark/}reactive-
hosted-kafka-benchmark...
[info] Resolving jline#jline;2.12.1 ...
[info] downloading
https://repo1.maven.org/maven2/org/apache/kafka/kafka-clients/0.10.0.1/kafka-clients-0.10.0.1.jar
...
[info] [SUCCESSFUL ] org.apache.kafka#kafka-clients;0.10.0.1!kafka-clients.jar
(103ms)
[info] downloading
https://repo1.maven.org/maven2/org/apache/kafka/kafka_2.11/0.10.0.1/kafka_2.11-0.10.0.1.jar
...
[info] [SUCCESSFUL ] org.apache.kafka#kafka_2.11;0.10.0.1!kafka_2.11.jar
(267ms)
[info] downloading
https://repo1.maven.org/maven2/com/typesafe/ssl-config-akka_2.11/0.2.1/ssl-config-akka_2.11-0.2.1.jar
...
[info] [SUCCESSFUL ]
com.typesafe#ssl-config-akka_2.11;0.2.1!ssl-config-akka_2.11.jar(bundle)
(54ms)
[info] downloading
https://repo1.maven.org/maven2/com/typesafe/config/1.3.0/config-1.3.0.jar
...
[info] [SUCCESSFUL ] com.typesafe#config;1.3.0!config.jar(bundle) (58ms)
[info] downloading
https://repo1.maven.org/maven2/com/typesafe/ssl-config-core_2.11/0.2.1/ssl-config-core_2.11-0.2.1.jar
...
[info] [SUCCESSFUL ]
com.typesafe#ssl-config-core_2.11;0.2.1!ssl-config-core_2.11.jar(bundle)
(56ms)
[info] Done updating.
[warn] Scala version was updated by one of library dependencies:
[warn] * org.scala-lang:scala-library:(2.11.7, 2.11.6) -> 2.11.8
[warn] To force scalaVersion, add the following:
[warn] ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true)
}
[warn] Run 'evicted' to see detailed eviction warnings
[info] Compiling 4 Scala sources to /home/admin/reactive-hosted-kafka-
benchmark/target/scala-2.11/classes...
[info] Running bioserenity_kafka_benchmark.Main
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
[WARN] [06/23/2017 21:56:41.413] [toto-akka.kafka.default-dispatcher-12] [
akka://toto/system/kafka-consumer-1] Consumer interrupted with
WakeupException after timeout. Message: null. Current value of
akka.kafka.consumer.wakeup-timeout is 3000 milliseconds
[WARN] [06/23/2017 21:56:44.500] [toto-akka.kafka.default-dispatcher-14] [
akka://toto/system/kafka-consumer-1] Consumer interrupted with
WakeupException after timeout. Message: null. Current value of
akka.kafka.consumer.wakeup-timeout is 3000 milliseconds
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(
0x00000007b4100000, 139460608, 0) failed; error='Cannot allocate memory' (
errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 139460608 bytes for
committing reserved memory.
# An error report file with more information is saved as:
# /home/admin/reactive-hosted-kafka-benchmark/hs_err_pid1000.log
Did you meet the same issue than me ?
Le vendredi 23 juin 2017 23:38:17 UTC+2, Kilic Ali-Firat a écrit :
>
> Thank you Alex,
>
> I will give a try :)
>
> Le vendredi 23 juin 2017 23:30:54 UTC+2, Kilic Ali-Firat a écrit :
>>
>> Did you do something like this ?
>>
>> libraryDependencies ++= Seq(
>> "com.typesafe.akka" %% "akka-stream-kafka" % "0.16"
>> )
>>
>> dependencyOverrides += (
>> "org.apache.kafka" % "kafka-clients" % "0.10.0.1"
>> )
>>
>> Using dependencyOverrides on kafka-clients should not be enough to get
>> the same speedup ..
>>
>> Le vendredi 23 juin 2017 23:27:43 UTC+2, Alex Cozzi a écrit :
>>>
>>> yes, in our dependencies we do not only include reactive kafka, but also
>>> explicitly the dependencies on kafka libs so that we can force 0.10.0.1.
>>>
>>>
>>> On Friday, June 23, 2017 at 1:50:33 PM UTC-7, Kilic Ali-Firat wrote:
>>>>
>>>> Oh so I'm not the only one to observe a such throughput.
>>>>
>>>> Maybe a stupid question but how did you force the compilation against
>>>> the 0.10.0.1 release of kafka libs ?
>>>>
>>>> Le vendredi 23 juin 2017 22:24:21 UTC+2, Alex Cozzi a écrit :
>>>>>
>>>>> We observed a similar slowdown when i use reactive kafka (which uses
>>>>> kafka 0.10.2.0 as dependency) connecting to a 0.10.0.0 server, but we
>>>>> solved by forcing compiling against the 0.10.0.1 release of the kafka
>>>>> libraries (kafka-clients and kafka_2.11) we get the same speedup. I think
>>>>> it has to do with 0.10.2.0 client libraries running in compatibility
>>>>> mode).
>>>>>
>>>>> On Friday, June 23, 2017 at 10:10:20 AM UTC-7, Kilic Ali-Firat wrote:
>>>>>>
>>>>>> *kafka broker is kafka_2.11-0.10.0.0
>>>>>
>>>>>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
