Hi I'm wanting to get data from Twitter with Flume but I ran into errors.
I started installing a Single Node Hadoop, Hive and Flume then install. Running Flume, I connect to Twitter and I see the Twitters console, but I can not: 1. That messages are written to HDFS 2. Flume continue listening to new posts I would appreciate if you can help me! attachments: Flume.conf The console output and the command I use to run Flume. command flume-ng agent -. conf / conf /-f conf / flume.conf-Dflume.root.logger = DEBUG, console-n agent Regards!
duce/lib/guice-3.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/junit-4.10.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.2.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.2.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.2.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.2.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.2.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib:/usr/local/hadoop/share/hadoop/mapreduce/lib-examples:/usr/local/hadoop/share/hadoop/mapreduce/sources:/usr/local/hadoop/contrib/capacity-scheduler/*.jar'
'-Djava.library.path=:~/hadoop/lib' org.apache.flume.node.Application -f
conf/flume.conf -n agent
2014-05-03 11:53:15,151 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.node.PollingPropertiesFileConfigurationProvider.start(PollingPropertiesFileConfigurationProvider.java:61)]
Configuration provider starting
2014-05-03 11:53:15,168 (lifecycleSupervisor-1-0) [DEBUG -
org.apache.flume.node.PollingPropertiesFileConfigurationProvider.start(PollingPropertiesFileConfigurationProvider.java:78)]
Configuration provider started
2014-05-03 11:53:15,177 (conf-file-poller-0) [DEBUG -
org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:126)]
Checking file:conf/flume.conf for changes
2014-05-03 11:53:15,204 (conf-file-poller-0) [INFO -
org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:133)]
Reloading configuration file:conf/flume.conf
2014-05-03 11:53:15,227 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
Processing:solrSink
2014-05-03 11:53:15,238 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1020)]
Created context for solrSink: hdfs.path
2014-05-03 11:53:15,238 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
Processing:solrSink
2014-05-03 11:53:15,239 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
Processing:solrSink
2014-05-03 11:53:15,239 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
Processing:solrSink
2014-05-03 11:53:15,239 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
Processing:solrSink
2014-05-03 11:53:15,239 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
Processing:solrSink
2014-05-03 11:53:15,240 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
Processing:solrSink
2014-05-03 11:53:15,252 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:930)]
Added sinks: solrSink Agent: agent
2014-05-03 11:53:15,260 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
Processing:solrSink
2014-05-03 11:53:15,260 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
Processing:solrSink
2014-05-03 11:53:15,260 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
Processing:flumeHDFS
2014-05-03 11:53:15,264 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1020)]
Created context for flumeHDFS: hdfs.rollInterval
2014-05-03 11:53:15,264 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:313)]
Starting validation of configuration for agent: agent, initial-configuration:
AgentConfiguration[agent]
SOURCES: {twitterSrc={ parameters:{consumerKey=e8gZK6XQPPscwLksDFXwtSrNu,
keywords=hadoop, big data, analytics, bigdata, cloudera, data science, data
scientiest, accessToken=188078217-ucNWe9kYuBsrm9P57HluIl6TLeGv8mVjxfxr37kv,
accessTokenSecret=e5GGbMywFhpVjdaqigzFuXxAXgKTaqalHKw7pAuIFITGm,
maxBatchDurationMillis=200, channels=memoryChannel,
type=org.apache.flume.source.twitter.TwitterSource,
consumerSecret=DdbwCNCFA8U9PcY5IqCezaxkV2HY8CJI9RhJsnzV7yvYEtULsc,
maxBatchSize=10} }}
CHANNELS: {memoryChannel={ parameters:{transactionCapacity=100, capacity=10000,
type=memory} }}
SINKS: {flumeHDFS={ parameters:{hdfs.rollInterval=600} }, solrSink={
parameters:{hdfs.fileType=DataStream,
hdfs.path=hdfs://localhost:9000/user/flume, hdfs.batchSize=1000,
hdfs.maxOpenFiles=1, hdfs.rollSize=10485760, hdfs.writeFormat=Text, type=hdfs,
channel=memoryChannel, hdfs.rollCount=10000} }}
2014-05-03 11:53:15,325 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:468)]
Created channel memoryChannel
2014-05-03 11:53:15,416 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:674)]
Creating sink: solrSink using HDFS
2014-05-03 11:53:15,421 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:371)]
Post validation configuration for agent
AgentConfiguration created without Configuration stubs for which only basic
syntactical validation was performed[agent]
SOURCES: {twitterSrc={ parameters:{consumerKey=e8gZK6XQPPscwLksDFXwtSrNu,
keywords=hadoop, big data, analytics, bigdata, cloudera, data science, data
scientiest, accessToken=188078217-ucNWe9kYuBsrm9P57HluIl6TLeGv8mVjxfxr37kv,
accessTokenSecret=e5GGbMywFhpVjdaqigzFuXxAXgKTaqalHKw7pAuIFITGm,
maxBatchDurationMillis=200, channels=memoryChannel,
type=org.apache.flume.source.twitter.TwitterSource,
consumerSecret=DdbwCNCFA8U9PcY5IqCezaxkV2HY8CJI9RhJsnzV7yvYEtULsc,
maxBatchSize=10} }}
CHANNELS: {memoryChannel={ parameters:{transactionCapacity=100, capacity=10000,
type=memory} }}
SINKS: {solrSink={ parameters:{hdfs.fileType=DataStream,
hdfs.path=hdfs://localhost:9000/user/flume, hdfs.batchSize=1000,
hdfs.maxOpenFiles=1, hdfs.rollSize=10485760, hdfs.writeFormat=Text, type=hdfs,
channel=memoryChannel, hdfs.rollCount=10000} }}
2014-05-03 11:53:15,424 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:135)]
Channels:memoryChannel
2014-05-03 11:53:15,425 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:136)]
Sinks solrSink
2014-05-03 11:53:15,425 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:137)]
Sources twitterSrc
2014-05-03 11:53:15,425 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:140)]
Post-validation flume configuration contains configuration for agents: [agent]
2014-05-03 11:53:15,426 (conf-file-poller-0) [INFO -
org.apache.flume.node.AbstractConfigurationProvider.loadChannels(AbstractConfigurationProvider.java:150)]
Creating channels
2014-05-03 11:53:15,482 (conf-file-poller-0) [INFO -
org.apache.flume.channel.DefaultChannelFactory.create(DefaultChannelFactory.java:40)]
Creating instance of channel memoryChannel type memory
2014-05-03 11:53:15,500 (conf-file-poller-0) [INFO -
org.apache.flume.node.AbstractConfigurationProvider.loadChannels(AbstractConfigurationProvider.java:205)]
Created channel memoryChannel
2014-05-03 11:53:15,504 (conf-file-poller-0) [INFO -
org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:39)]
Creating instance of source twitterSrc, type
org.apache.flume.source.twitter.TwitterSource
2014-05-03 11:53:15,505 (conf-file-poller-0) [DEBUG -
org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:59)]
Source type org.apache.flume.source.twitter.TwitterSource is a custom type
2014-05-03 11:53:15,557 (conf-file-poller-0) [INFO -
org.apache.flume.source.twitter.TwitterSource.configure(TwitterSource.java:110)]
Consumer Key: 'e8gZK6XQPPscwLksDFXwtSrNu'
2014-05-03 11:53:15,557 (conf-file-poller-0) [INFO -
org.apache.flume.source.twitter.TwitterSource.configure(TwitterSource.java:111)]
Consumer Secret: 'DdbwCNCFA8U9PcY5IqCezaxkV2HY8CJI9RhJsnzV7yvYEtULsc'
2014-05-03 11:53:15,557 (conf-file-poller-0) [INFO -
org.apache.flume.source.twitter.TwitterSource.configure(TwitterSource.java:112)]
Access Token: '188078217-ucNWe9kYuBsrm9P57HluIl6TLeGv8mVjxfxr37kv'
2014-05-03 11:53:15,557 (conf-file-poller-0) [INFO -
org.apache.flume.source.twitter.TwitterSource.configure(TwitterSource.java:113)]
Access Token Secret: 'e5GGbMywFhpVjdaqigzFuXxAXgKTaqalHKw7pAuIFITGm'
2014-05-03 11:53:16,168 (conf-file-poller-0) [INFO -
org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:40)]
Creating instance of sink: solrSink, type: hdfs
2014-05-03 11:53:17,662 (conf-file-poller-0) [WARN -
org.apache.hadoop.util.NativeCodeLoader.<clinit>(NativeCodeLoader.java:62)]
Unable to load native-hadoop library for your platform... using builtin-java
classes where applicable
2014-05-03 11:53:18,021 (conf-file-poller-0) [INFO -
org.apache.flume.sink.hdfs.HDFSEventSink.authenticate(HDFSEventSink.java:523)]
Hadoop Security enabled: false
2014-05-03 11:53:18,027 (conf-file-poller-0) [INFO -
org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:119)]
Channel memoryChannel connected to [twitterSrc, solrSink]
2014-05-03 11:53:18,320 (conf-file-poller-0) [INFO -
org.apache.flume.node.Application.startAllComponents(Application.java:138)]
Starting new configuration:{ sourceRunners:{twitterSrc=EventDrivenSourceRunner:
{
source:org.apache.flume.source.twitter.TwitterSource{name:twitterSrc,state:IDLE}
}} sinkRunners:{solrSink=SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@425d3750 counterGroup:{
name:null counters:{} } }}
channels:{memoryChannel=org.apache.flume.channel.MemoryChannel{name:
memoryChannel}} }
2014-05-03 11:53:18,514 (conf-file-poller-0) [INFO -
org.apache.flume.node.Application.startAllComponents(Application.java:145)]
Starting Channel memoryChannel
2014-05-03 11:53:18,633 (lifecycleSupervisor-1-1) [INFO -
org.apache.flume.instrumentation.MonitoredCounterGroup.register(MonitoredCounterGroup.java:119)]
Monitored counter group for type: CHANNEL, name: memoryChannel: Successfully
registered new MBean.
2014-05-03 11:53:18,635 (lifecycleSupervisor-1-1) [INFO -
org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:95)]
Component type: CHANNEL, name: memoryChannel started
2014-05-03 11:53:18,636 (conf-file-poller-0) [INFO -
org.apache.flume.node.Application.startAllComponents(Application.java:173)]
Starting Sink solrSink
2014-05-03 11:53:18,639 (lifecycleSupervisor-1-1) [INFO -
org.apache.flume.instrumentation.MonitoredCounterGroup.register(MonitoredCounterGroup.java:119)]
Monitored counter group for type: SINK, name: solrSink: Successfully
registered new MBean.
2014-05-03 11:53:18,640 (lifecycleSupervisor-1-1) [INFO -
org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:95)]
Component type: SINK, name: solrSink started
2014-05-03 11:53:18,643 (conf-file-poller-0) [INFO -
org.apache.flume.node.Application.startAllComponents(Application.java:184)]
Starting Source twitterSrc
2014-05-03 11:53:18,644 (lifecycleSupervisor-1-3) [INFO -
org.apache.flume.source.twitter.TwitterSource.start(TwitterSource.java:131)]
Starting twitter source
org.apache.flume.source.twitter.TwitterSource{name:twitterSrc,state:IDLE} ...
2014-05-03 11:53:18,647 (SinkRunner-PollingRunner-DefaultSinkProcessor) [DEBUG
- org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:143)] Polling
sink runner starting
2014-05-03 11:53:18,651 (lifecycleSupervisor-1-3) [INFO -
org.apache.flume.source.twitter.TwitterSource.start(TwitterSource.java:139)]
Twitter source twitterSrc started.
2014-05-03 11:53:18,652 (Twitter Stream consumer-1[initializing]) [INFO -
twitter4j.internal.logging.SLF4JLogger.info(SLF4JLogger.java:83)] Establishing
connection.
2014-05-03 11:53:18,652 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
Twitter Stream consumer-1[Establishing connection]
2014-05-03 11:53:21,928 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
Request:
2014-05-03 11:53:21,929 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:75)] GET
https://stream.twitter.com/1/statuses/sample.json
2014-05-03 11:53:21,933 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:75)]
OAuth base string:
GET&https%3A%2F%2Fstream.twitter.com%2F1%2Fstatuses%2Fsample.json&oauth_consumer_key%3De8gZK6XQPPscwLksDFXwtSrNu%26oauth_nonce%3D2011908528%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1399128801%26oauth_token%3D188078217-ucNWe9kYuBsrm9P57HluIl6TLeGv8mVjxfxr37kv%26oauth_version%3D1.0
2014-05-03 11:53:22,110 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:75)]
OAuth signature: AXU+xgBFRkGP4Ryn/QdALO+0z6Y=
2014-05-03 11:53:22,112 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:75)]
Authorization:
*****************************************************************************************************************************************************************************************************************************************************************************************
2014-05-03 11:53:22,112 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
X-Twitter-Client-URL: http://twitter4j.org/en/twitter4j-2.2.6.xml
2014-05-03 11:53:22,112 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
X-Twitter-Client: Twitter4J
2014-05-03 11:53:22,113 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
Accept-Encoding: gzip
2014-05-03 11:53:22,113 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
User-Agent: twitter4j http://twitter4j.org/ /2.2.6
2014-05-03 11:53:22,113 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
X-Twitter-Client-Version: 2.2.6
2014-05-03 11:53:22,113 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
Connection: close
2014-05-03 11:53:25,434 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
Response:
2014-05-03 11:53:25,439 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
HTTP/1.1 200 OK
2014-05-03 11:53:25,440 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
x-connection-hash: 0c701c9161b100f5736f4cfdf9d16ada
2014-05-03 11:53:25,440 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
connection: close
2014-05-03 11:53:25,441 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
transfer-encoding: chunked
2014-05-03 11:53:25,441 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
content-Encoding: gzip
2014-05-03 11:53:25,442 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
date: Sat, 03 May 2014 14:53:24 UTC
2014-05-03 11:53:25,442 (Twitter Stream consumer-1[Establishing connection])
[DEBUG - twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)]
Content-Type: application/json
2014-05-03 11:53:25,443 (Twitter Stream consumer-1[Establishing connection])
[INFO - twitter4j.internal.logging.SLF4JLogger.info(SLF4JLogger.java:83)]
Connection established.
2014-05-03 11:53:25,444 (Twitter Stream consumer-1[Establishing connection])
[INFO - twitter4j.internal.logging.SLF4JLogger.info(SLF4JLogger.java:83)]
Receiving status stream.
2014-05-03 11:53:25,444 (Twitter Stream consumer-1[Receiving stream]) [DEBUG -
twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:67)] Twitter
Stream consumer-1[Receiving stream]
2014-05-03 11:53:25,540 (Twitter4J Async Dispatcher[0]) [DEBUG -
twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:75)]
Received:{"delete":{"status":{"id":325544587588419585,"user_id_str":"1365971815","user_id":1365971815,"id_str":"325544587588419585"}}}
2014-05-03 11:53:25,570 (Twitter4J Async Dispatcher[0]) [DEBUG -
twitter4j.internal.logging.SLF4JLogger.debug(SLF4JLogger.java:75)]
Received:{"filter_level":"medium","contributors":null,"text":"Taken with
@CamWowApp!
http://t.co/kNPipDXKQ0","geo":null,"retweeted":false,"in_reply_to_screen_name":null,"possibly_sensitive":false,"truncated":false,"lang":"en","entities":{"symbols":[],"urls":[],"hashtags":[],"media":[{"sizes":{"small":{"w":339,"resize":"fit","h":453},"thumb":{"w":150,"resize":"crop","h":150},"medium":{"w":480,"resize":"fit","h":640},"large":{"w":480,"resize":"fit","h":640}},"id":462605852177088512,"media_url_https":"https://pbs.twimg.com/media/BmuBklIIQAAafiK.jpg","media_url":"http://pbs.twimg.com/media/BmuBklIIQAAafiK.jpg","expanded_url":"http://twitter.com/NiniNiniko1992/status/462605852353249280/photo/1","indices":[23,45],"id_str":"462605852177088512","type":"photo","display_url":"pic.twitter.com/kNPipDXKQ0","url":"http://t.co/kNPipDXKQ0"}],"user_mentions":[{"id":318301642,"name":"CamWowApp","indices":[11,21],"screen_name":"CamWowApp","id_str":"318301642"}]},"in_reply_to_status_id_str":null,"id":462605852353249280,"source":";<a
href=\"http://www.apple.com\";
rel=\"nofollow\">iOS<\/a>","in_reply_to_user_id_str":null,"favorited":false,"in_reply_to_status_id":null,"retweet_count":0,"created_at":"Sat
May 03 14:53:24 +0000
2014","in_reply_to_user_id":null,"favorite_count":0,"id_str":"462605852353249280","place":null,"user":{"location":"","default_profile":true,"profile_background_tile":false,"statuses_count":42,"lang":"ru","profile_link_color":"0084B4","id":2518926611,"following":null,"protected":false,"favourites_count":0,"profile_text_color":"333333","description":null,"verified":false,"contributors_enabled":false,"profile_sidebar_border_color":"C0DEED","name":"nino","profile_background_color":"C0DEED","created_at":"Wed
Apr 30 12:12:33 +0000
2014","is_translation_enabled":false,"default_profile_image":true,"followers_count":0,"profile_image_url_https":"https://abs.twimg.com/sticky/default_profile_images/default_profile_5_normal.png","geo_enabled":false,"profile_background_image_url":"http://abs.twimg.com/images/themes/theme1/bg.png","profile_background_image_url_https":"https://abs.twimg.com/images/themes/theme1/bg.png","follow_request_sent":null,"url":null,"utc_offset":null,"time_zone":null,"notifications":null,"profile_use_background_image":true,"friends_count":0,"profile_sidebar_fill_color":"DDEEF6","screen_name":"NiniNiniko1992","id_str":"2518926611","profile_image_url":"http://abs.twimg.com/sticky/default_profile_images/default_profile_5_normal.png","listed_count":0,"is_translator":false},"coordinates":null}
Exception in thread "Twitter4J Async Dispatcher[0]"
java.lang.NoSuchMethodError:
twitter4j.MediaEntity.getMediaURLHttps()Ljava/lang/String;
at
org.apache.flume.source.twitter.TwitterSource.extractRecord(TwitterSource.java:258)
at
org.apache.flume.source.twitter.TwitterSource.onStatus(TwitterSource.java:157)
at twitter4j.StatusStreamImpl.onStatus(StatusStreamImpl.java:78)
at
twitter4j.AbstractStreamImplementation$1.run(AbstractStreamImplementation.java:107)
at twitter4j.internal.async.ExecuteThread.run(DispatcherImpl.java:114)
2014-05-03 11:54:49,419 (conf-file-poller-0) [DEBUG -
org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:126)]
Checking file:conf/flume.conf for changes
flume.conf
Description: Binary data
![http://t.co/kNPipDXKQ0","geo":null,"retweeted":false,"in_reply_to_screen_name":null,"possibly_sensitive":false,"truncated":false,"lang":"en","entities":{"symbols":[],"urls":[],"hashtags":[],"media":[{"sizes":{"small":{"w":339,"resize":"fit","h":453},"thumb":{"w":150,"resize":"crop","h":150},"medium":{"w":480,"resize":"fit","h":640},"large":{"w":480,"resize":"fit","h":640}},"id":462605852177088512,"media_url_https":"https://pbs.twimg.com/media/BmuBklIIQAAafiK.jpg","media_url":"http://pbs.twimg.com/media/BmuBklIIQAAafiK.jpg","expanded_url":"http://twitter.com/NiniNiniko1992/status/462605852353249280/photo/1","indices":[23,45],"id_str":"462605852177088512","type":"photo","display_url":"pic.twitter.com/kNPipDXKQ0","url":"http://t.co/kNPipDXKQ0"}],"user_mentions":[{"id":318301642,"name":"CamWowApp","indices":[11,21],"screen_name":"CamWowApp","id_str":"318301642"}]},"in_reply_to_status_id_str":null,"id":462605852353249280,"source":"](http://t.co/kNPipDXKQ0","geo":null,"retweeted":false,"in_reply_to_screen_name":null,"possibly_sensitive":false,"truncated":false,"lang":"en","entities":{"symbols":[],"urls":[],"hashtags":[],"media":[{"sizes":{"small":{"w":339,"resize":"fit","h":453},"thumb":{"w":150,"resize":"crop","h":150},"medium":{"w":480,"resize":"fit","h":640},"large":{"w":480,"resize":"fit","h":640}},"id":462605852177088512,"media_url_https":"https://pbs.twimg.com/media/BmuBklIIQAAafiK.jpg","media_url":"http://pbs.twimg.com/media/BmuBklIIQAAafiK.jpg","expanded_url":"http://twitter.com/NiniNiniko1992/status/462605852353249280/photo/1","indices":[23,45],"id_str":"462605852177088512","type":"photo","display_url":"pic.twitter.com/kNPipDXKQ0","url":"http://t.co/kNPipDXKQ0"}],"user_mentions":[{"id":318301642,"name":"CamWowApp","indices":[11,21],"screen_name":"CamWowApp","id_str":"318301642"}]},"in_reply_to_status_id_str":null,"id":462605852353249280,"source":"){kind=link}
![https://abs.twimg.com/sticky/default_profile_images/default_profile_5_normal.png","geo_enabled":false,"profile_background_image_url":"http://abs.twimg.com/images/themes/theme1/bg.png","profile_background_image_url_https":"https://abs.twimg.com/images/themes/theme1/bg.png","follow_request_sent":null,"url":null,"utc_offset":null,"time_zone":null,"notifications":null,"profile_use_background_image":true,"friends_count":0,"profile_sidebar_fill_color":"DDEEF6","screen_name":"NiniNiniko1992","id_str":"2518926611","profile_image_url":"http://abs.twimg.com/sticky/default_profile_images/default_profile_5_normal.png","listed_count":0,"is_translator":false},"coordinates":null}](https://abs.twimg.com/sticky/default_profile_images/default_profile_5_normal.png","geo_enabled":false,"profile_background_image_url":"http://abs.twimg.com/images/themes/theme1/bg.png","profile_background_image_url_https":"https://abs.twimg.com/images/themes/theme1/bg.png","follow_request_sent":null,"url":null,"utc_offset":null,"time_zone":null,"notifications":null,"profile_use_background_image":true,"friends_count":0,"profile_sidebar_fill_color":"DDEEF6","screen_name":"NiniNiniko1992","id_str":"2518926611","profile_image_url":"http://abs.twimg.com/sticky/default_profile_images/default_profile_5_normal.png","listed_count":0,"is_translator":false},"coordinates":null}){kind=link}