JMS (Tibco) Source - input messages status during transaction

Hi, When leveraging a JMS (Tibco implementation) source to consume messages from a queue, are such messages immediately consumed , hence removed from the queue, even if the flume transaction hasn't been closed yet? What happens if the transaction is not committed (e.g. in case of errors)? Fo

Understand JMS source + HDFS sink batch management

Hello folks, I'm testing a Flume agent defined by a topology made of : *JMS source* (Tibco implementation) -> *memory channel* -> *hdfs sink* The *JMS source* has: * my_agent.sources.my_source.batchSize = 100 The *memory channel* has: * my_agent.channels.my_channel.capacity = 100 The *HDF

Define and leverage custom constants in agent configuration

Hello folks, I was wondering if it's possible to define custom constants in a Flume agent configuration file, so to avoid errors and replicated stuff when not needed. An example to make my point clear: assume I want to launch an agent of type /myAgent/ configured with file channel and hdfs s

Re: Define and leverage custom constants in agent configuration

pe it is useful have nice day Silvio 2016-11-28 18:26 GMT+01:00 Roberto Coluccio <mailto:roberto.coluc...@eng.it>>: Hello folks, I was wondering if it's possible to define custom constants in a Flume agent configuration file, so to avoid errors and replicated stuff

Re: Define and leverage custom constants in agent configuration

ng Wu (CS)" <mailto:shang...@symantec.com>> wrote: Hi Robert, Looks like what you want is more like a global configuration, and based on my limited knowledge in Apache Flume, it is missing now. BR, Shang *From: *Roberto Coluccio mailto:roberto.coluc...@eng

duplicated events with kafka channel and hdfs sink

Hello again guys, sorry to bother, I'm kinda new to the Flume world and I'm running experiments to evaluate pros and cons about diverse topologies and settings. I'm currently experiencing the following issue: Agent topology made of: JMS source --> Kafka Channel --> HDFS sink Source and Sink b

AWS SQS Flume Source

Hello community, I was wondering if is there an "official" AWS SQS Source component to leverage w/ Flume 1.6 or 1.7. I couldn't find any references in the documentation. I just found an open-source project: https://github.com/plumbee/flume-sqs-source. Did this project have an "official" f

Re: AWS SQS Flume Source

Sad about that. Thanks anyway, Denes. Best, Roberto On 06/04/2017 17:20, Denes Arvay wrote: Hi Roberto, No, unfortunately the Flume codebase doesn't contain AWS SQS source. Best, Denes On Thu, Apr 6, 2017 at 5:13 PM Roberto Coluccio mailto:roberto.coluc...@eng.it>> wrote: