GitHub user liujkang opened a pull request:
https://github.com/apache/storm/pull/2388
storm-kafka-client 1.1.1 lag still is 1
when i use storm-kafka-client kafkaspout to consume message,
still has 1 message
[http://note.youdao.com/yws/public/resource/93a58cfe26d2c6f9f97f37e50913e108/xmlnote/A32584036A744581B2E7B12E2734A811/2909](http://note.youdao.com/yws/public/resource/93a58cfe26d2c6f9f97f37e50913e108/xmlnote/A32584036A744581B2E7B12E2734A811/2909)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/storm 1.x-branch
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/2388.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2388
----
commit 979b5b03955dad00e7db657c4fc083478b520b5f
Author: Sriharsha Chintalapani <[email protected]>
Date: 2017-05-01T20:50:11Z
Merge branch 'STORM-2343-1.x' of https://github.com/srdo/storm into
1.x-branch
commit e56dd29bdaa6775a8bbb8336e60978234e298ead
Author: Sriharsha Chintalapani <[email protected]>
Date: 2017-05-01T20:51:16Z
Added STORM-2343 to CHANGELOG.
commit 4f240f2667059866f6a982844a2c5b9d83f33bcb
Author: Sriharsha Chintalapani <[email protected]>
Date: 2017-05-02T16:51:06Z
Merge branch 'STORM-2482-1.x' of https://github.com/arunmahadevan/storm
into 1.x-branch
commit 95ec555e8e9ce91797076ef6000c56c43b6c0a72
Author: Sriharsha Chintalapani <[email protected]>
Date: 2017-05-02T21:57:23Z
Added STORM-2482 to CHANGELOG.
commit 0fda4f20462192b1338c646abf3275691e0179d1
Author: Arun Mahadevan <[email protected]>
Date: 2017-05-03T07:53:24Z
[STORM-2498] Fix Download Full File link in 1.x branch
commit 9767255b2627119f14f930f6b39fe5d5e7fd663d
Author: Erik Weathers <[email protected]>
Date: 2017-04-28T07:50:18Z
[STORM-2191] shorten classpaths by using wildcards
This commit resolves cherry-pick conflicts for backporting change from
storm master branch.
Related commit in master: 2ceef0dce21895c4ef0f0b8eb5bc8248fad25e13
Instead of fully enumerating all JARs in the lib directories, we just use
a Java classpath wildcard to allow the JVM to autodiscover all JARs in the
classpath. This affects both the Worker and LogWriter processes, as well
as Storm daemons such as the Nimbus, UI, Logviewer, and Supervisor.
This change results in shorter commands, so that you can actually see
the full content of the Worker command in `ps` output on Linux.
Prior to this change Worker commands were easily longer than 4096 bytes,
which is the default Linux kernel limit for commands being recorded into
the process table. Longer commands get truncated, though they do get
executed.
An example of the change in Worker classpath length can be seen here:
Before:
```
-cp
STORM_DIR/lib-worker/asm-5.0.3.jar:STORM_DIR/lib-worker/carbonite-1.5.0.jar:STORM_DIR/lib-worker/chill-java-0.8.0.jar:STORM_DIR/lib-worker/clojure-1.7.0.jar:STORM_DIR/lib-worker/commons-codec-1.6.jar:STORM_DIR/lib-worker/commons-collections-3.2.2.jar:STORM_DIR/lib-worker/commons-io-2.5.jar:STORM_DIR/lib-worker/commons-lang-2.5.jar:STORM_DIR/lib-worker/commons-logging-1.1.3.jar:STORM_DIR/lib-worker/curator-client-2.12.0.jar:STORM_DIR/lib-worker/curator-framework-2.12.0.jar:STORM_DIR/lib-worker/disruptor-3.3.2.jar:STORM_DIR/lib-worker/guava-16.0.1.jar:STORM_DIR/lib-worker/httpclient-4.3.3.jar:STORM_DIR/lib-worker/httpcore-4.4.1.jar:STORM_DIR/lib-worker/jgrapht-core-0.9.0.jar:STORM_DIR/lib-worker/jline-0.9.94.jar:STORM_DIR/lib-worker/json-simple-1.1.jar:STORM_DIR/lib-worker/kryo-3.0.3.jar:STORM_DIR/lib-worker/kryo-shaded-3.0.3.jar:STORM_DIR/lib-worker/libthrift-0.9.3.jar:STORM_DIR/lib-worker/log4j-api-2.8.jar:STORM_DIR/lib-worker/log4j-core-2.8.jar:STORM_DIR/lib-worker/log4j-ove
r-slf4j-1.6.6.jar:STORM_DIR/lib-worker/log4j-slf4j-impl-2.8.jar:STORM_DIR/lib-worker/minlog-1.3.0.jar:STORM_DIR/lib-worker/netty-3.9.0.Final.jar:STORM_DIR/lib-worker/objenesis-2.1.jar:STORM_DIR/lib-worker/reflectasm-1.10.1.jar:STORM_DIR/lib-worker/servlet-api-2.5.jar:STORM_DIR/lib-worker/slf4j-api-1.7.21.jar:STORM_DIR/lib-worker/snakeyaml-1.11.jar:STORM_DIR/lib-worker/storm-client-2.0.0-SNAPSHOT.jar:STORM_DIR/lib-worker/sysout-over-slf4j-1.0.2.jar:STORM_DIR/lib-worker/zookeeper-3.4.6.jar:STORM_DIR/conf:STORM_DIR/storm-local/supervisor/stormdist/foo-topology-1-1-1493359573/stormjar.jar
```
After:
```
-cp
STORM_DIR/lib-worker/*:STORM_DIR/extlib/*:STORM_DIR/conf:STORM_DIR/storm-local/supervisor/stormdist/foo-topology-1-1-1493359573/stormjar.jar
```
This change also includes additional documentation about the use of
classpaths in
Storm and provides some guidance for using the various features for using
external
libraries.
For more details on this problem and a discussion about this solution's
merits, please see
[STORM-2191](https://issues.apache.org/jira/browse/STORM-2191).
commit b04011ee8c40b8971592e74e85c18778989bb932
Author: Erik Weathers <[email protected]>
Date: 2017-05-04T21:57:56Z
[STORM-2191] address revans2's suggestions, and fix typo (extra ']') in
markdown in Setting-up-a-Storm-cluster.md
commit b780df2312835021de4c29c673696cfac7dff767
Author: Erik Weathers <[email protected]>
Date: 2017-05-04T22:01:12Z
[STORM-2191] add link to the new Classpath-handling doc page from the index
page
commit cca9ba71ea595b79c16842180029e8475bbcd096
Author: Jungtaek Lim <[email protected]>
Date: 2017-05-08T03:14:25Z
STORM-2496 Dependency artifacts should be uploaded to blobstore with READ
permission for all
* When uploading dependencies, set ACL properly so that it can be shared to
other users as well
* but allows only READ so that it can't be deleted from others
commit b396d4c610dbf7036fedc46f45ae2bf7ea99a5d0
Author: Jungtaek Lim <[email protected]>
Date: 2017-05-08T06:34:49Z
Merge branch 'STORM-2191__1.x-branch' of https://github.com/erikdw/storm
into STORM-2191-1.x-merge
commit 31e75bf5b1ac8e248f5ab03596eb39b199e22c52
Author: Jungtaek Lim <[email protected]>
Date: 2017-05-08T06:39:41Z
STORM-2191: CHANGELOG
commit e36bf4b6e43af8b3d330d07f6c8d692fb7d67301
Author: Jungtaek Lim <[email protected]>
Date: 2017-05-08T07:22:50Z
Merge branch 'STORM-2498' of https://github.com/arunmahadevan/storm into
STORM-2498-1.x-merge
commit ea7e7f75493868bdb4daf0fa65590a520602e0e9
Author: Jungtaek Lim <[email protected]>
Date: 2017-05-08T07:23:20Z
STORM-2498: CHANGELOG
commit a2dde20cc4313ec373946e0845742eabdce67017
Author: Vivek Mittal <[email protected]>
Date: 2017-05-08T05:13:36Z
[STORM-2505] Spout to support topic compaction
[STORM-2505] Maintaining a emitted set in OffsetManager to handle the voids
in the topic
[STORM-2505] Handling NPE in Boxed Long to primitive type comparison
[STORM-2505] Rephrased the log message when a non contiguous offset is
acked by the spout
[STORM-2505] Updated comment
[STORM-2505] Renamed the methods ack/emit to addToAckMsgs and addToEmitMsgs
in OffsetManager
commit 958f53f013363128189c0dd022d8d2bdec8e1078
Author: Sriharsha Chintalapani <[email protected]>
Date: 2017-05-13T05:03:32Z
Merge branch '1.x-branch' of https://github.com/vivekmittal/storm into
1.x-branch
commit 979153be8ae09ecdc368d10a5aa01db7685ef02c
Author: Sriharsha Chintalapani <[email protected]>
Date: 2017-05-13T05:04:18Z
Added STORM-2505 to CHANGELOG.
commit 5ff7865cf0b86f40e99b54e789fa60b8843191aa
Author: Priyank <[email protected]>
Date: 2017-05-13T06:32:08Z
STORM-2512: Make constructor public and add one more builder constructor
commit 6f8bdccea3383de9ac5d784180d0d25a721652f5
Author: Sriharsha Chintalapani <[email protected]>
Date: 2017-05-14T15:52:18Z
Added STORM-2512 to CHANGELOG.
commit 3785dead38b4be93c04b95b6431729472b5390f9
Author: Sriharsha Chintalapani <[email protected]>
Date: 2017-05-15T15:14:49Z
Merge branch 'STORM-2496-1.x' of https://github.com/HeartSaVioR/storm into
1.x-branch
commit c2950e490c95cb56c5a04088a1e07754ad5929fe
Author: Sriharsha Chintalapani <[email protected]>
Date: 2017-05-15T15:15:38Z
Added STORM-2496 to CHANGELOG.
commit 24437acc57235d29ef8e9632f678c9c53f7ff7d4
Author: Robert Evans <[email protected]>
Date: 2017-05-15T17:27:52Z
Merge branch 'STORM-2511' of https://github.com/satishd/storm into
STORM-2511
STORM-2511: Submitting a topology with name containing unicode getting
failed
commit aa6a559e6731051b36101d53213358ef6b6fa794
Author: Robert Evans <[email protected]>
Date: 2017-05-15T18:05:05Z
Added STORM-2511 to Changelog
commit cfc72a240873e465908dd8e012676a35d5096319
Author: Arun Mahadevan <[email protected]>
Date: 2017-05-12T08:57:12Z
Fix minor typos in storm-hdfs and storm-hbase docs that shows usage of
configKeys
And make sure docs/storm-hdfs.md docs/storm-hbase.md are in sync with
external/storm-hdfs/README.md and
external/storm-hbase/README.md
commit 40586234b9a739634a2aaf06ba1085ffb48ed306
Author: Arun Mahadevan <[email protected]>
Date: 2017-05-16T11:11:03Z
Added STORM-2493 to CHANGELOG.md
commit 3744b4b6b3370248f770f498263b69f67e382bd9
Author: Jungtaek Lim <[email protected]>
Date: 2017-05-17T08:45:44Z
STORM-2518 Handles empty name for "USER type" ACL when normalizing ACLs
* if ACLs have both empty name and user, discard empty name ACLs
* add default ACL for user when neither empty name nor user ACL exists
* populate empty name ACLs to users in Subject: set name to user for each
commit 97f8ba016a47f141e2665d3413733f65fc3adfcb
Author: Arun Mahadevan <[email protected]>
Date: 2017-05-17T09:09:12Z
STORM-2519: Modify AbstractAutoCreds to look for configKeys in both nimbus
and topology configs.
commit 7a749a1051d77e5b267c0801e83f169961ee90d1
Author: Jungtaek Lim <[email protected]>
Date: 2017-05-17T14:45:42Z
STORM-2520 AutoHDFS should prefer cluster-wise hdfs kerberos principal to
global hdfs kerberos principal
commit ad5ac422833a857a311563a2a529c15a65353613
Author: Manikumar Reddy O <[email protected]>
Date: 2017-04-25T11:54:06Z
STORM-2501: Auto populate Hive credentials using Hive MetaStore delegation
tokens
commit 4d95363563edf28dea8aeeeca118e81e2f959053
Author: Arun Mahadevan <[email protected]>
Date: 2017-05-18T07:39:10Z
Merge branch 'STORM-2519' of https://github.com/arunmahadevan/storm into
STORM-2519-1.x
commit fa48720e5ac6a35d277fbcedd87929ea81f00343
Author: Arun Mahadevan <[email protected]>
Date: 2017-05-18T07:40:03Z
Added STORM-2519 to CHANGELOG.md
----
---
