I am not sure if there is a Flume Hive Sink for you to put your records into Hive directly. https://issues.apache.org/jira/browse/FLUME-1008
Also, another approach can be to push the events to HBase with Flume-HBase Sink. [Just in case if none of your other options work for you]. From: Manu Moncy K [mailto:[email protected]] Sent: Thursday, August 30, 2012 3:49 PM To: [email protected] Subject: Details needed about RADIUS LOGS Tue Aug 7 00:00:00 2012 User-Name = "xxxxxxxx" NAS-Port = xxxxxxxx NAS-IP-Address = xxxxxxxx Framed-IP-Address = xxxxxxxx Filter-Id = " xxxxxxxx " Class = " xxxxxxxx " NAS-Identifier = " xxxxxxxx " Acct-Status-Type = xxxxxxxx Acct-Delay-Time = 0 Acct-Session-Id = " xxxxxxxx " Acct-Authentic = RADIUS Event-Timestamp = 1344286800 NAS-Port-Type = Ethernet Calling-Station-Id = " xxxxxxxx " NAS-Port-Id = " xxxxxxxx " Service-Type = Framed-User Framed-Protocol = PPP Acct-Link-Count = 0 RB-Agent-Circuit-Id = " xxxxxxxx " DSLForum-Agent-Circuit-Id = " xxxxxxxx " DSLForum-Access-Loop-Encapsulation = "" Timestamp = 1344286800 OSC-Service-Identifier = "DSLUsers" Proxy-State = OSC-Extended-Id=40682 Timestamp = 1344286800 Tue Aug 7 00:00:00 2012 User-Name = " xxxxxxxx " NAS-Port = xxxxxxxx NAS-IP-Address = xxxxxxxx Framed-IP-Address = xxxxxxxx Class = "44620232:04:" NAS-Identifier = " xxxxxxxx " Acct-Status-Type = Stop Acct-Delay-Time = 0 Acct-Input-Octets = 6021 Acct-Output-Octets = 323749 Acct-Session-Id = " xxxxxxxx " Acct-Authentic = RADIUS Acct-Session-Time = 1348 Acct-Input-Packets = 53 Acct-Output-Packets = 3187 Acct-Terminate-Cause = User-Request Acct-Input-Gigawords = 0 Acct-Output-Gigawords = 0 Event-Timestamp = 1344286800 NAS-Port-Type = Ethernet Calling-Station-Id = " xxxxxxxx " NAS-Port-Id = " xxxxxxxx " Service-Type = Framed-User Framed-Protocol = PPP Acct-Link-Count = 0 Timestamp = 1344286800 OSC-Service-Identifier = "DSLUsers" Proxy-State = OSC-Extended-Id=24386 Timestamp = 1344286800 Above given log format (2 events given) is the RADIUS LOG I am working on, I wanted to know if there is a way i can use flume and put this log into hive in JSON format and take the required fields for each event. -- Manu K Moncy Data Scientist Flutura Business Solutions Pvt. Ltd Electronics and Communication Engineering(2008-2012) Govt. Model Engineering College, Cochin - 21 ☎: +91-9740245341 ☎: +91-9895163190 ✉: [email protected]<mailto:[email protected]> ✉: [email protected]<mailto:[email protected]> ______________________________________________________________________ Disclaimer:This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding
