[jira] [Commented] (FLUME-2736) issue in exporting flipkart data from web url to HDFS
[ https://issues.apache.org/jira/browse/FLUME-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614818#comment-14614818 ] swayam commented on FLUME-2736: --- Here is my config file info .Please help me to get the data from flipkart : ### # Name the components on this agent agent1.sources = source1 agent1.sinks = sink1 agent1.channels = channel1 # Describe Source # Source Http agent1.sources.source1.type = org.apache.flume.source.http.HTTPSource agent1.sources.source1.port = 5140 agent1.sources.source1.handler = org.apache.flume.http.JSONHandler agent1.sources.source1.handler.nickname = json handler # Describe Sink # Sink Logger agent1.sinks.sink1.type = logger # Describe Channel # Channel Memory agent1.channels.channel1.type = memory agent1.channels.channel1.capacity = 1000 agent1.channels.channel1.transactionCapactiy = 100 # Bind the source and sink to the channel agent1.sources.source1.channels = channel1 agent1.sinks.sink1.channel = channel1 issue in exporting flipkart data from web url to HDFS - Key: FLUME-2736 URL: https://issues.apache.org/jira/browse/FLUME-2736 Project: Flume Issue Type: Task Components: Sinks+Sources Affects Versions: v1.5.0 Reporter: swayam Original Estimate: 24h Remaining Estimate: 24h Dear Team, I am trying to export the flipkart's json formatted data from URL https://affiliate-api.flipkart.net/affiliate/api/31873d12df9e46499e535333fdee9dad.json;. Please help us . Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2729) Allow pollableSource backoff times to be configurable
[ https://issues.apache.org/jira/browse/FLUME-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615185#comment-14615185 ] Johny Rufus commented on FLUME-2729: +1, Will run tests and commit Allow pollableSource backoff times to be configurable -- Key: FLUME-2729 URL: https://issues.apache.org/jira/browse/FLUME-2729 Project: Flume Issue Type: Improvement Reporter: Ted Malaska Assignee: Ted Malaska Priority: Minor Attachments: FLUME-2729.patch.2, FLUME-2729.patch.4, FLUME-2729.patch.7, FLUME_2729.patch There are use cases that require more control of the backoff times setting. This jira will allow those times to be set through config of the source -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2736) issue in exporting flipkart data from web url to HDFS
[ https://issues.apache.org/jira/browse/FLUME-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615357#comment-14615357 ] Ashish Paliwal commented on FLUME-2736: --- HTTP Source is not a pull source, you need to push events to it via Get or put. It won't pull data from a URL. This what I could understood so far. As I said pls ask questions on User Mailing List not JIRA. I am closing this JIRA, please send email to ML. issue in exporting flipkart data from web url to HDFS - Key: FLUME-2736 URL: https://issues.apache.org/jira/browse/FLUME-2736 Project: Flume Issue Type: Task Components: Sinks+Sources Affects Versions: v1.5.0 Reporter: swayam Original Estimate: 24h Remaining Estimate: 24h Dear Team, I am trying to export the flipkart's json formatted data from URL https://affiliate-api.flipkart.net/affiliate/api/31873d12df9e46499e535333fdee9dad.json;. Please help us . Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [ANNOUNCE] New Flume committer - Johny Rufus
Congratulations Johny! Jarcec On Jun 19, 2015, at 1:38 PM, Hari Shreedharan hshreedha...@apache.org wrote: On behalf of the Apache Flume PMC, I am excited to welcome Johny Rufus as a committer on the Apache Flume project. Johny has actively contributed several patches to the Flume project, including bug fixes, authentication and other new features. Congratulations and Welcome, Johny! Cheers, Hari Shreedharan
[jira] [Resolved] (FLUME-2736) issue in exporting flipkart data from web url to HDFS
[ https://issues.apache.org/jira/browse/FLUME-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-2736. --- Resolution: Invalid User ML Question issue in exporting flipkart data from web url to HDFS - Key: FLUME-2736 URL: https://issues.apache.org/jira/browse/FLUME-2736 Project: Flume Issue Type: Task Components: Sinks+Sources Affects Versions: v1.5.0 Reporter: swayam Original Estimate: 24h Remaining Estimate: 24h Dear Team, I am trying to export the flipkart's json formatted data from URL https://affiliate-api.flipkart.net/affiliate/api/31873d12df9e46499e535333fdee9dad.json;. Please help us . Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2672) NPE in KafkaSourceCounter
[ https://issues.apache.org/jira/browse/FLUME-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615958#comment-14615958 ] Hari Shreedharan commented on FLUME-2672: - [~ashishpaliwal] - If this looks ok to you, do you mind committing it? Thanks! NPE in KafkaSourceCounter - Key: FLUME-2672 URL: https://issues.apache.org/jira/browse/FLUME-2672 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v1.6.0 Environment: Mac OS 10.10.3, Java 1.7.0_60 Reporter: Rigo MacTaggart Priority: Trivial Labels: easyfix Attachments: FLUME-2672-with-test.patch, FLUME-2672.patch Original Estimate: 0h Remaining Estimate: 0h A NPE is thrown when KafkaSource calls counter.incrementKafkaEmptyCount() because it expects MonitoredCounterGroup.counterMap to contain key source.kafka.empty.count. A patch is included which adds this key to the ATTRIBUTES string array, which is used to pre-populate keys with an initial value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2729) Allow pollableSource backoff times to be configurable
[ https://issues.apache.org/jira/browse/FLUME-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615976#comment-14615976 ] ASF subversion and git services commented on FLUME-2729: Commit e7a8b80a0e5115e68fd545824e6b71082c89eb06 in flume's branch refs/heads/flume-1.7 from [~jrufus] [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=e7a8b80 ] FLUME-2729. Allow pollableSource backoff times to be configurable (Ted Malaska via Johny Rufus) Allow pollableSource backoff times to be configurable -- Key: FLUME-2729 URL: https://issues.apache.org/jira/browse/FLUME-2729 Project: Flume Issue Type: Improvement Reporter: Ted Malaska Assignee: Ted Malaska Priority: Minor Attachments: FLUME-2729.patch.2, FLUME-2729.patch.4, FLUME-2729.patch.7, FLUME_2729.patch There are use cases that require more control of the backoff times setting. This jira will allow those times to be set through config of the source -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2729) Allow pollableSource backoff times to be configurable
[ https://issues.apache.org/jira/browse/FLUME-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615973#comment-14615973 ] ASF subversion and git services commented on FLUME-2729: Commit b5e102bee4d4b2783f3f65aab403a53c1ae8e401 in flume's branch refs/heads/trunk from [~jrufus] [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=b5e102b ] FLUME-2729. Allow pollableSource backoff times to be configurable (Ted Malaska via Johny Rufus) Allow pollableSource backoff times to be configurable -- Key: FLUME-2729 URL: https://issues.apache.org/jira/browse/FLUME-2729 Project: Flume Issue Type: Improvement Reporter: Ted Malaska Assignee: Ted Malaska Priority: Minor Attachments: FLUME-2729.patch.2, FLUME-2729.patch.4, FLUME-2729.patch.7, FLUME_2729.patch There are use cases that require more control of the backoff times setting. This jira will allow those times to be set through config of the source -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2729) Allow pollableSource backoff times to be configurable
[ https://issues.apache.org/jira/browse/FLUME-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615983#comment-14615983 ] Ted Malaska commented on FLUME-2729: Wow thank u so much. Allow pollableSource backoff times to be configurable -- Key: FLUME-2729 URL: https://issues.apache.org/jira/browse/FLUME-2729 Project: Flume Issue Type: Improvement Reporter: Ted Malaska Assignee: Ted Malaska Priority: Minor Fix For: v1.7.0 Attachments: FLUME-2729.patch.2, FLUME-2729.patch.4, FLUME-2729.patch.7, FLUME_2729.patch There are use cases that require more control of the backoff times setting. This jira will allow those times to be set through config of the source -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLUME-2737) Documentation for Pollable Source config parameters introduced in FLUME-2729
Johny Rufus created FLUME-2737: -- Summary: Documentation for Pollable Source config parameters introduced in FLUME-2729 Key: FLUME-2737 URL: https://issues.apache.org/jira/browse/FLUME-2737 Project: Flume Issue Type: Documentation Reporter: Johny Rufus Assignee: Ted Malaska Documentation needs to be updated in Flume User Guide, for all Pollable Sources, that would use the new config parameters introduced in FLUME-2729 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2729) Allow pollableSource backoff times to be configurable
[ https://issues.apache.org/jira/browse/FLUME-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616039#comment-14616039 ] Hudson commented on FLUME-2729: --- UNSTABLE: Integrated in Flume-trunk-hbase-1 #107 (See [https://builds.apache.org/job/Flume-trunk-hbase-1/107/]) FLUME-2729. Allow pollableSource backoff times to be configurable (johnyrufus: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.gita=commith=b5e102bee4d4b2783f3f65aab403a53c1ae8e401) * flume-ng-core/src/main/java/org/apache/flume/PollableSource.java * flume-ng-sources/flume-kafka-source/src/main/java/org/apache/flume/source/kafka/KafkaSource.java * flume-ng-core/src/main/java/org/apache/flume/source/SequenceGeneratorSource.java * flume-ng-core/src/test/java/org/apache/flume/source/TestStressSource.java * flume-ng-core/src/main/java/org/apache/flume/source/PollableSourceRunner.java * flume-ng-core/src/main/java/org/apache/flume/source/AbstractPollableSource.java * flume-ng-core/src/test/java/org/apache/flume/source/TestAbstractPollableSource.java * flume-ng-core/src/test/java/org/apache/flume/source/TestPollableSourceRunner.java * flume-ng-core/src/main/java/org/apache/flume/source/PollableSourceConstants.java * flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestIntegration.java * flume-ng-core/src/test/java/org/apache/flume/source/TestSequenceGeneratorSource.java * flume-ng-core/src/main/java/org/apache/flume/source/StressSource.java Allow pollableSource backoff times to be configurable -- Key: FLUME-2729 URL: https://issues.apache.org/jira/browse/FLUME-2729 Project: Flume Issue Type: Improvement Reporter: Ted Malaska Assignee: Ted Malaska Priority: Minor Fix For: v1.7.0 Attachments: FLUME-2729.patch.2, FLUME-2729.patch.4, FLUME-2729.patch.7, FLUME_2729.patch There are use cases that require more control of the backoff times setting. This jira will allow those times to be set through config of the source -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Jenkins build became unstable: Flume-trunk-hbase-1 #107
See https://builds.apache.org/job/Flume-trunk-hbase-1/107/changes
[jira] [Resolved] (FLUME-2729) Allow pollableSource backoff times to be configurable
[ https://issues.apache.org/jira/browse/FLUME-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johny Rufus resolved FLUME-2729. Resolution: Fixed Fix Version/s: v1.7.0 Committed. Thanks Ted! Allow pollableSource backoff times to be configurable -- Key: FLUME-2729 URL: https://issues.apache.org/jira/browse/FLUME-2729 Project: Flume Issue Type: Improvement Reporter: Ted Malaska Assignee: Ted Malaska Priority: Minor Fix For: v1.7.0 Attachments: FLUME-2729.patch.2, FLUME-2729.patch.4, FLUME-2729.patch.7, FLUME_2729.patch There are use cases that require more control of the backoff times setting. This jira will allow those times to be set through config of the source -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2672) NPE in KafkaSourceCounter
[ https://issues.apache.org/jira/browse/FLUME-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615981#comment-14615981 ] Ashish Paliwal commented on FLUME-2672: --- Let me check and commit. NPE in KafkaSourceCounter - Key: FLUME-2672 URL: https://issues.apache.org/jira/browse/FLUME-2672 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v1.6.0 Environment: Mac OS 10.10.3, Java 1.7.0_60 Reporter: Rigo MacTaggart Priority: Trivial Labels: easyfix Attachments: FLUME-2672-with-test.patch, FLUME-2672.patch Original Estimate: 0h Remaining Estimate: 0h A NPE is thrown when KafkaSource calls counter.incrementKafkaEmptyCount() because it expects MonitoredCounterGroup.counterMap to contain key source.kafka.empty.count. A patch is included which adds this key to the ATTRIBUTES string array, which is used to pre-populate keys with an initial value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
