Hello,
I'm not sure how to perform this load balancing, could you please give
me an example ?
Regards,
Smaine
On 02/18/15 17:39, Smaine KAHLOUCH wrote:
Thank you Ahmed, i'll try that
Regards,
Smaine
------------------------------------------------------------------------
*From:* Ahmed Vila <[email protected]>
*Sent:* Wednesday, February 18, 2015 5:33 PM
*To:* [email protected]
*Subject:* Re: Spreading multiple file-channels over distinct disks
Hi Smaine,
There is always an overhead of running multiple flume instances, i.e.
multiple Java VMs. Thus, I wouldn't recommend running more of them if
that's not due to distribution across servers.
But, you are able to do round-robin events from source into multiple
channels and those channels could have one disk per channel.
However, in that case you'll also need at least one sink per channel.
Guess that's even desirable if you're sinking your data into the HDFS.
Regards
On Wed, Feb 18, 2015 at 10:14 AM, Smaine Kahlouch
<[email protected] <mailto:[email protected]>>
wrote:
Hi all!
I would like to know what is the best way to spread the i/o on my
server.
Actually it has several disks and i was wondering how to spread
Flume's work over them :
* One Flume instance with multiple working directories (dataDirs),
each working directory on a single disk
* Multiple flume instances with a file channel on a distinct disk
(using containers would be the best way to do so)
Could you please helpe me to find out the best architecture which
allows us to keep file-channel for data persistency ?
Regards,
--
Smaine Kahlouch - Engineer, Research & Engineering
Arkena | T: +33 1 5868 6196 <tel:%2B33%201%205868%206196>
27 Blvd Hippolyte Marquès, 94200 Ivry-sur-Seine, France
arkena.com <http://arkena.com>
--
Best regards,
Ahmed Vila | Senior software developer
DevLogic | Sarajevo | Bosnia and Herzegovina
Office : +387 33 942 123
Mobile: +387 62 139 348
Website: www.devlogic.eu <http://www.devlogic.eu>
E-mail : [email protected] <mailto:[email protected]>
---------------------------------------------------------------------
This e-mail and any attachment is for authorised use by the intended
recipient(s) only. This email contains confidential information. It
should not be copied, disclosed to, retained or used by, any party
other than the intended recipient. Any unauthorised distribution,
dissemination or copying of this E-mail or its attachments, and/or any
use of any information contained in them, is strictly prohibited and
may be illegal. If you are not an intended recipient then please
promptly delete this e-mail and any attachment and all copies and
inform the sender directly via email. Any emails that you send to us
may be monitored by systems or persons other than the named
communicant for the purposes of ascertaining whether the communication
complies with the law and company policies.
---------------------------------------------------------------------
This e-mail and any attachment is for authorised use by the intended
recipient(s) only. This email contains confidential information. It
should not be copied, disclosed to, retained or used by, any party
other than the intended recipient. Any unauthorised distribution,
dissemination or copying of this E-mail or its attachments, and/or any
use of any information contained in them, is strictly prohibited and
may be illegal. If you are not an intended recipient then please
promptly delete this e-mail and any attachment and all copies and
inform the sender directly via email. Any emails that you send to us
may be monitored by systems or persons other than the named
communicant for the purposes of ascertaining whether the communication
complies with the law and company policies.
--
Smaine Kahlouch - Engineer, Research & Engineering
Arkena | T: +33 1 5868 6196
27 Blvd Hippolyte Marquès, 94200 Ivry-sur-Seine, France
arkena.com
