My aim is to collect data as and when it appears inside a log file. For that I
am using the below configuration
exec config node1 default-Flow
'tail("/var/log/flume/test1.log",startFromEnd="true")'
'value("LogType","Test1Log") agentBESink("ip-10-40-222-77.ec2.internal",35853)'
Can you please tell me if I am correct in using below mentioned configuration
from Flume Master web console, where I will configure node:
"ip-10-40-222-77.ec2.internal"
And in source => tail("/var/log/flume/test1.log",startFromEnd="true")
And in sink => value("LogType","Test1Log")
agentBESink("ip-10-40-222-77.ec2.internal",35853)
After that, will a new hbase table get created? And will data get inserted
inside HBase table?
Please help
Regards,
Shouvanik
From: Kartashov, Andy [mailto:[email protected]]
Sent: 14 December 2012 03:42
To: [email protected]
Subject: RE: greetings - Flume on Windows
Thanks Jeff,
Got stuck now. Need to bypass the proxy firewall somehow.
From: Jeff Lord [mailto:[email protected]]
Sent: Thursday, December 13, 2012 5:09 PM
To: [email protected]
Subject: Re: greetings - Flume on Windows
Andy,
The current stable release of flume is 1.3.0 and you can always check which
release is current on this page:
http://flume.apache.org/releases/index.html
In order to checkout this release from git you can issue the following command:
git clone https://git-wip-us.apache.org/repos/asf/flume.git -b flume-1.3.0
That will clone the branch into a directory of the same name in your cwd.
Once you have that you can simply follow the next steps.
- Change into the previously outchecked directory
- Build with "mvn clean" and "mvn package -DskipTests"
-Jeff
On Thu, Dec 13, 2012 at 12:04 PM, Kartashov, Andy
<[email protected]<mailto:[email protected]>> wrote:
Alex,
Thanks.
- download JDK 1.6 - check
- download Maven - check
- set env variables - check
- download msysgit -check
Now have no idea how to do the next step:
- checkout a git repo
- Change into the previously outchecked directory
- Build with "mvn clean" and "mvn package -DskipTests"
Please help.
Thanks,
-----Original Message-----
From: Alexander Alten-Lorenz
[mailto:[email protected]<mailto:[email protected]>]
Sent: Thursday, December 13, 2012 12:49 PM
To: [email protected]<mailto:[email protected]>
Subject: Re: greetings - Flume on Windows
You can follow our wiki guide:
https://cwiki.apache.org/confluence/display/FLUME/Build+Flume+1.3.x+up+on+Windows
to build Flume on Windows.
Best,
Alex
On Dec 13, 2012, at 6:35 PM, Hari Shreedharan
<[email protected]<mailto:[email protected]>> wrote:
> The installation instructions you have referenced here is for Flume OG, not
> Flume 1.x. Flume 1.x currently does not have a Windows installer or Windows
> daemons.
>
>
> Hari
> --
> Hari Shreedharan
>
>
> On Thursday, December 13, 2012 at 8:53 AM, Kartashov, Andy wrote:
>
>> Flummers,
>>
>> I successfully ran flume-ng 1.2 on two boxes running Unix and would now like
>> to try to connect to one box running Windows.
>>
>> The installation instructions on Cloudera site are too ambiguous for me to
>> grasp.
>>
>> 1.1. Installation
>> Flume for Windows is installed using an executable installer generated by
>> Nullsoft Scriptable Install System (NSIS). It provides a simple wizard which
>> presents basic options for a node's configuration settings. It will install
>> the software and optionally start a Flume node as a Windows service (daemon).
>> I could not get this to work as described.
>>
>> I downloaded NSIS installer and downloaded flume 0.9x zip file from github.
>> When I tried to drag flume.nsis into to the script compiler window I got
>> this error:
>> !insertmacro: MUI_PAGE_LICENSE
>> LicenseData: open failed "LICENSE"
>> Usage: LicenseData local_file_that_has_license_text |
>> license_lang_string Error in macro MUI_PAGEDECLARATION_LICENSE on
>> macroline 17 Error in macro MUI_PAGE_LICENSE on macroline 6 Error in
>> script
>> ".....\Flume\flume-master\flume-windows-dist\src\main\nsis\flume.nsi "
>> on line 96 -- aborting creation process
>>
>> Then I tried another option, "installing based on zip file". Installation
>> went well but there was no prompts for configuration, so when I ran
>> /bin/flume-node.exe I got this:
>>
>> Sorry for this silly question but how to do it?
>>
>> Thanks,
>>
>>
>> NOTICE: This e-mail message and any attachments are confidential,
>> subject to copyright and may be privileged. Any unauthorized use,
>> copying or disclosure is prohibited. If you are not the intended
>> recipient, please delete and contact the sender immediately. Please
>> consider the environment before printing this e-mail. AVIS : le
>> présent courriel et toute pièce jointe qui l'accompagne sont
>> confidentiels, protégés par le droit d'auteur et peuvent être
>> couverts par le secret professionnel. Toute utilisation, copie ou
>> divulgation non autorisée est interdite. Si vous n'êtes pas le
>> destinataire prévu de ce courriel, supprimez-le et contactez
>> immédiatement l'expéditeur. Veuillez penser à l'environnement avant
>> d'imprimer le présent courriel
>>
>
>
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
NOTICE: This e-mail message and any attachments are confidential, subject to
copyright and may be privileged. Any unauthorized use, copying or disclosure is
prohibited. If you are not the intended recipient, please delete and contact
the sender immediately. Please consider the environment before printing this
e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont
confidentiels, protégés par le droit d'auteur et peuvent être couverts par le
secret professionnel. Toute utilisation, copie ou divulgation non autorisée est
interdite. Si vous n'êtes pas le destinataire prévu de ce courriel,
supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à
l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to
copyright and may be privileged. Any unauthorized use, copying or disclosure is
prohibited. If you are not the intended recipient, please delete and contact
the sender immediately. Please consider the environment before printing this
e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont
confidentiels, protégés par le droit d'auteur et peuvent être couverts par le
secret professionnel. Toute utilisation, copie ou divulgation non autorisée est
interdite. Si vous n'êtes pas le destinataire prévu de ce courriel,
supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à
l'environnement avant d'imprimer le présent courriel
________________________________
This message is for the designated recipient only and may contain privileged,
proprietary, or otherwise private information. If you have received it in
error, please notify the sender immediately and delete the original. Any other
use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its
affiliates, including e-mail and instant messaging (including content), may be
scanned by our systems for the purposes of information security and assessment
of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com
