[GitHub] storm issue #1935: STORM-2353: Replace kafka-unit by kafka_2.11 and kafka-cl...
Github user lukess commented on the issue: https://github.com/apache/storm/pull/1935 @srdo the issue is if I clone a storm repo from github, and follow [doc](https://github.com/apache/storm/blob/master/docs/storm-kafka-client.md#using-storm-kafka-client-with-different-versions-of-kafka) would fail if mvn clean package -Dstorm.kafka.client.version=0.10.1.0 mvn clean package -Dstorm.kafka.client.version=0.10.1.1 ` Running org.apache.storm.kafka.bolt.KafkaBoltTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.175 sec - in org.apache.storm.kafka.bolt.KafkaBoltTest Running org.apache.storm.kafka.spout.KafkaSpoutRebalanceTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.055 sec - in org.apache.storm.kafka.spout.KafkaSpoutRebalanceTest Running org.apache.storm.kafka.spout.KafkaSpoutConfigTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in org.apache.storm.kafka.spout.KafkaSpoutConfigTest Running org.apache.storm.kafka.spout.ByTopicRecordTranslatorTest Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - in org.apache.storm.kafka.spout.ByTopicRecordTranslatorTest Running org.apache.storm.kafka.spout.DefaultRecordTranslatorTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec - in org.apache.storm.kafka.spout.DefaultRecordTranslatorTest Running org.apache.storm.kafka.spout.SingleTopicKafkaSpoutTest Tests run: 4, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 7.591 sec <<< FAILURE! - in org.apache.storm.kafka.spout.SingleTopicKafkaSpoutTest shouldReplayInOrderFailedMessages(org.apache.storm.kafka.spout.SingleTopicKafkaSpoutTest) Time elapsed: 1.826 sec <<< ERROR! kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries. at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:96) at kafka.producer.Producer.send(Producer.scala:78) at kafka.javaapi.producer.Producer.send(Producer.scala:35) at info.batey.kafka.unit.KafkaUnit.sendMessages(KafkaUnit.java:263) at org.apache.storm.kafka.spout.SingleTopicKafkaSpoutTest.lambda$populateTopicData$0(SingleTopicKafkaSpoutTest.java:102) shouldContinueWithSlowDoubleAcks(org.apache.storm.kafka.spout.SingleTopicKafkaSpoutTest) Time elapsed: 1.651 sec <<< ERROR! kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries. at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:96) at kafka.producer.Producer.send(Producer.scala:78) at kafka.javaapi.producer.Producer.send(Producer.scala:35) at info.batey.kafka.unit.KafkaUnit.sendMessages(KafkaUnit.java:263) at org.apache.storm.kafka.spout.SingleTopicKafkaSpoutTest.lambda$populateTopicData$0(SingleTopicKafkaSpoutTest.java:102) shouldEmitAllMessages(org.apache.storm.kafka.spout.SingleTopicKafkaSpoutTest) Time elapsed: 2.465 sec <<< ERROR! kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries. at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:96) at kafka.producer.Producer.send(Producer.scala:78) at kafka.javaapi.producer.Producer.send(Producer.scala:35) at info.batey.kafka.unit.KafkaUnit.sendMessages(KafkaUnit.java:263) at org.apache.storm.kafka.spout.SingleTopicKafkaSpoutTest.lambda$populateTopicData$0(SingleTopicKafkaSpoutTest.java:102) shouldReplayFirstTupleFailedOutOfOrder(org.apache.storm.kafka.spout.SingleTopicKafkaSpoutTest) Time elapsed: 1.648 sec <<< ERROR! kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries. at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:96) at kafka.producer.Producer.send(Producer.scala:78) at kafka.javaapi.producer.Producer.send(Producer.scala:35) at info.batey.kafka.unit.KafkaUnit.sendMessages(KafkaUnit.java:263) at org.apache.storm.kafka.spout.SingleTopicKafkaSpoutTest.lambda$populateTopicData$0(SingleTopicKafkaSpoutTest.java:102) Results : Tests in error: SingleTopicKafkaSpoutTest.shouldContinueWithSlowDoubleAcks:129->initializeSpout:107->populateTopicData:97->lambda$populateTopicData$0:102 » FailedToSendMessage SingleTopicKafkaSpoutTest.shouldEmitAllMessages:168->initializeSpout:107->populateTopicData:97->lambda$populateTopicData$0:102 » FailedToSendMessage SingleTopicKafkaSpoutTest.shouldReplayFirstTupleFailedOutOfOrder:237->initializeSpout:107->populateTopicData:97->lambda$populateTopicData$0:102 » FailedToSendMessage SingleTopicKafkaSpoutTest.shouldReplayInOrderFailedMessages:196->initializeSpout:107->populateTopicData:97->lambda$populateTopicData$0:102 » FailedToSendMessage Tests run: 12, Failures: 0,
[VOTE] Release Apache Storm 1.1.0 (RC1)
This is a call to vote on releasing Apache Storm 1.1.0 (rc1) Full list of changes in this release: https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=17a2017fb644e353fb2a0f5bf50d400ee28036ba The tag/commit to be voted upon is v1.1.0: https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=75b7d808d10674ff580beb031ae970fe94e97615;hb=17a2017fb644e353fb2a0f5bf50d400ee28036ba The source archive being voted upon can be found here: https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.0-rc1/apache-storm-1.1.0-src.tar.gz Other release files, signatures and digests can be found here: https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.0-rc1/ The release artifacts are signed with the following key: https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd The Nexus staging repository for this release is: https://repository.apache.org/content/repositories/orgapachestorm-1045 Please vote on releasing this package as Apache Storm 1.1.0. When voting, please list the actions taken to verify the release. This vote will be open for at least 72 hours. [ ] +1 Release this package as Apache Storm 1.1.0 [ ] 0 No opinion [ ] -1 Do not release this package because... Thanks to everyone who contributed to this release. -Taylor
[GitHub] storm issue #1935: STORM-2353: Replace kafka-unit by kafka_2.11 and kafka-cl...
Github user srdo commented on the issue: https://github.com/apache/storm/pull/1935 I'm not sure what you mean about not being able to release on master/1.x. The next step is that someone with write access needs to review. @HeartSaVioR do you have time to take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1935: STORM-2353: Replace kafka-unit by kafka_2.11 and kafka-cl...
Github user lukess commented on the issue: https://github.com/apache/storm/pull/1935 May I know the next step for review? :) My Kafka server is on 0.10.1, and I'm releasing storm-kafka-client with kafka_client 0.10.1 from my branch. I cannot release on origin master/1.x branch without skip tests. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
How to debug the storm1.0.0 source code?
Hi, stormers: I have recently studied on the apache storm-1.0.0, I have a question: How to debug the storm1.0.0 source code with IDEA or Eclipse? Could you please help me ? I need your kindly help, thanks very much. Best wishes. Haihui Xu
storm WordCountTopology problems
Hello, I am tanchengbing. I am reading your storm project in github. I am a great fan of it. I like it very much. However, there are some issues I am a little confused. For example, I work with Storm for a while already, but want to get started with development. As suggested, I am using Intellij (up to now, I was using Eclipse and did only write Topologies against Java API).First I got the following error: java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read. Serializer Exception: Traceback (most recent call last): File "splitsentence.py", line 16, in import storm ImportError: No module named storm Update: installing pyhton-storm is not required to make it work Unfortunately, I am getting a different error now: File "splitsentence.py", line 18, in class SplitSentenceBolt(storm.BasicBolt): AttributeError: 'module' object has no attribute 'BasicBolt' Could you please help me to solve them out? Thank you very much. I am looking forward for your answers. Thank you. Best wishes/Regards. Yours sincerely, tanchengbing
[GitHub] storm pull request #1944: [STORM-2365] Support for specifying output stream ...
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1944 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1942: STORM-2358: Update storm hdfs spout to remove specific im...
Github user pasalkarsachin1 commented on the issue: https://github.com/apache/storm/pull/1942 I have reverted class renaming. Other than this I don't see break in backward compatibility --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1942: STORM-2358: Update storm hdfs spout to remove specific im...
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1942 First of all, thanks for the contribution. I'm seeing backward incompatible as @roshannaik stated, and like renaming class is not strictly necessary to. Huge part of maintaining open source is fighting with backward compatibility. What you change will affect number of developers. It's joyful thing for open source developers but also huge pain point. Unless there are critical places so need to change or get rid of, breaking backward compatibility would not be accepted, and even it is really needed, it would be better to discuss widely. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1942: STORM-2358: Update storm hdfs spout to remove spec...
Github user roshannaik commented on a diff in the pull request:
https://github.com/apache/storm/pull/1942#discussion_r101704609
--- Diff:
external/storm-hdfs/src/main/java/org/apache/storm/hdfs/spout/HDFSSpout.java ---
@@ -45,11 +46,12 @@
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.tuple.Fields;
-public class HdfsSpout extends BaseRichSpout {
+public class HDFSSpout extends BaseRichSpout {
--- End diff --
Do you realize what you are doing here ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1942: STORM-2358: Update storm hdfs spout to remove spec...
Github user pasalkarsachin1 commented on a diff in the pull request:
https://github.com/apache/storm/pull/1942#discussion_r101704350
--- Diff:
external/storm-hdfs/src/main/java/org/apache/storm/hdfs/spout/HDFSSpout.java ---
@@ -105,87 +105,87 @@
private String configKey = Configs.DEFAULT_HDFS_CONFIG_KEY; // key for
hdfs Kerberos configs
- public HdfsSpout() {
+ public HDFSSpout() {
}
- public HdfsSpout setHdfsUri(String hdfsUri) {
+ public HDFSSpout setHdfsUri(String hdfsUri) {
this.hdfsUri = hdfsUri;
return this;
}
- public HdfsSpout setReaderType(String readerType) {
+ public HDFSSpout setReaderType(Class readerType) {
--- End diff --
I have deprecated as per comments, not removed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1942: STORM-2358: Update storm hdfs spout to remove spec...
Github user roshannaik commented on a diff in the pull request:
https://github.com/apache/storm/pull/1942#discussion_r101703840
--- Diff:
external/storm-hdfs/src/main/java/org/apache/storm/hdfs/spout/HDFSSpout.java ---
@@ -105,87 +105,87 @@
private String configKey = Configs.DEFAULT_HDFS_CONFIG_KEY; // key for
hdfs Kerberos configs
- public HdfsSpout() {
+ public HDFSSpout() {
}
- public HdfsSpout setHdfsUri(String hdfsUri) {
+ public HDFSSpout setHdfsUri(String hdfsUri) {
this.hdfsUri = hdfsUri;
return this;
}
- public HdfsSpout setReaderType(String readerType) {
+ public HDFSSpout setReaderType(Class readerType) {
--- End diff --
Looks like you are removing support for aliases. Is that right ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
