[jira] [Commented] (FLUME-2933) Does Flume support Kafka 0.10 version

[Lior Zeno (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338357#comment-15338357
 ] 

Lior Zeno commented on FLUME-2933:
--

Flume uses Kafka 0.9.0.1 client, it should work with 0.10 brokers. 
http://kafka.apache.org/documentation.html#upgrade

> Does Flume support Kafka 0.10 version
> -
>
> Key: FLUME-2933
> URL: https://issues.apache.org/jira/browse/FLUME-2933
> Project: Flume
>  Issue Type: Question
>  Components: Docs
>Reporter: Joseph Aliase
>
> Does Flume support Kafka 0.10 version or there is a plan to support Kafka 0.10



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [jira] [Updated] (FLUME-2933) Does Flume support Kafka 0.10 version

[Lior Zeno]

Flume uses Kafka 0.9.0.1 client, it should work with 0.10 brokers.
http://kafka.apache.org/documentation.html#upgrade
On Jun 19, 2016 2:32 AM, "Joseph Aliase (JIRA)"  wrote:

>
>  [
> https://issues.apache.org/jira/browse/FLUME-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
>
> Joseph Aliase updated FLUME-2933:
> -
> Description: Does Flume support Kafka 0.10 version or there is a plan
> to support Kafka 0.10  (was: Does Flume support Kafka 0.10 version or there
> is a plan support Kafka 0.10)
>
> > Does Flume support Kafka 0.10 version
> > -
> >
> > Key: FLUME-2933
> > URL: https://issues.apache.org/jira/browse/FLUME-2933
> > Project: Flume
> >  Issue Type: Question
> >  Components: Docs
> >Reporter: Joseph Aliase
> >
> > Does Flume support Kafka 0.10 version or there is a plan to support
> Kafka 0.10
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>

[jira] [Updated] (FLUME-2933) Does Flume support Kafka 0.10 version

[Joseph Aliase (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Aliase updated FLUME-2933:
-
Description: Does Flume support Kafka 0.10 version or there is a plan to 
support Kafka 0.10  (was: Does Flume support Kafka 0.10 version or there is a 
plan support Kafka 0.10)

> Does Flume support Kafka 0.10 version
> -
>
> Key: FLUME-2933
> URL: https://issues.apache.org/jira/browse/FLUME-2933
> Project: Flume
>  Issue Type: Question
>  Components: Docs
>Reporter: Joseph Aliase
>
> Does Flume support Kafka 0.10 version or there is a plan to support Kafka 0.10



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (FLUME-2933) Does Flume support Kafka 0.10 version

[Joseph Aliase (JIRA)]

Joseph Aliase created FLUME-2933:


 Summary: Does Flume support Kafka 0.10 version
 Key: FLUME-2933
 URL: https://issues.apache.org/jira/browse/FLUME-2933
 Project: Flume
  Issue Type: Question
  Components: Docs
Reporter: Joseph Aliase


Does Flume support Kafka 0.10 version or there is a plan support Kafka 0.10



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2897) AsyncHBase sink NPE when Channel.getTransaction() fails

[Mike Percy (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338196#comment-15338196
 ] 

Mike Percy commented on FLUME-2897:
---

It's alright. Thanks for your efforts and for working on the 1.7 release.


> AsyncHBase sink NPE when Channel.getTransaction() fails
> ---
>
> Key: FLUME-2897
> URL: https://issues.apache.org/jira/browse/FLUME-2897
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Mike Percy
>Assignee: Mike Percy
> Fix For: v1.7.0
>
> Attachments: FLUME-2897-1.patch
>
>
> There is a possibility for a NPE in the AsyncHBaseSink when a channel 
> getTransaction() call fails. This is possible when the FileChannel is out of 
> disk space.
> Patch forthcoming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2897) AsyncHBase sink NPE when Channel.getTransaction() fails

[Lior Zeno (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338192#comment-15338192
 ] 

Lior Zeno commented on FLUME-2897:
--

The JIRA workflow is open for interpretations and usually there are many 
different views, one of them is as you state - testers (or QA) close issues 
after they verified that the patch satisfies the issue. However, another view 
is that a closed issue is an issue that has been resolved and there are no 
further actions that need to be taken. One of the benefits of closing an issue 
is disallowing comments. Since a resolved issue is committed to the repo, in 
case there is a bug with the provided patch, a new issue should be created. 

After having a look at the reports, the majority of issues are resolved but not 
closed, so I guess we will continue in that path. Sorry for the noise.

> AsyncHBase sink NPE when Channel.getTransaction() fails
> ---
>
> Key: FLUME-2897
> URL: https://issues.apache.org/jira/browse/FLUME-2897
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Mike Percy
>Assignee: Mike Percy
> Fix For: v1.7.0
>
> Attachments: FLUME-2897-1.patch
>
>
> There is a possibility for a NPE in the AsyncHBaseSink when a channel 
> getTransaction() call fails. This is possible when the FileChannel is out of 
> disk space.
> Patch forthcoming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2798) Malformed Syslog messages can lead to OutOfMemoryException

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2798.


> Malformed Syslog messages can lead to OutOfMemoryException
> --
>
> Key: FLUME-2798
> URL: https://issues.apache.org/jira/browse/FLUME-2798
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.4.0, v1.5.0, v1.6.0
>Reporter: Phil D'Amore
>Assignee: Phil D'Amore
>Priority: Critical
> Fix For: v1.7.0
>
> Attachments: FLUME-2798.patch
>
>
> It's possible for a client submitting syslog data which is malformed in 
> various ways to convince SyslogUtils.extractEvent to continually fill the 
> ByteArrayOutputStream it uses to collect the event until the agent runs out 
> of memory.  Since the OOM condition affects the whole agent, it's possible 
> that a client sending such data (due to accident or malicious intent) to 
> disable the agent, as long as it remains connected.
> Note that this is probably only possible using SyslogTcpSource although the 
> fix touches common code in SyslogUtils.java.
> The issue can happen in two ways:
> Scenario 1: Send a message like this:
> {{<> some more stuff here}}
> This causes a NumberFormatException:
> {code}
> Sep 11, 2015 2:27:07 AM org.jboss.netty.channel.SimpleChannelHandler
> WARNING: EXCEPTION, please implement 
> org.apache.flume.source.SyslogTcpSource$syslogTcpHandler.exceptionCaught() 
> for proper handling.
> java.lang.NumberFormatException: For input string: ""
>   at 
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>   at java.lang.Integer.parseInt(Integer.java:504)
>   at java.lang.Integer.parseInt(Integer.java:527)
>   at org.apache.flume.source.SyslogUtils.buildEvent(SyslogUtils.java:198)
>   at 
> org.apache.flume.source.SyslogUtils.extractEvent(SyslogUtils.java:344)
>   at 
> org.apache.flume.source.SyslogTcpSource$syslogTcpHandler.messageReceived(SyslogTcpSource.java:76)
>   at 
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
>   at 
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
>   at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
>   at 
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
>   at 
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
>   at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> {code}
> This exception does not get handled, and it happens before reset() can be 
> called.  The result is that the state machine in SyslogUtils gets stuck in 
> the DATA state, and all subsequent data just gets appended to the baos, while 
> the above exception streams to the log.  Eventually the agent runs out of 
> memory.
> Scenario 2: Send some data like this:
> {{<123...}}
> No length checking is done in the PRIO state so you could potentially fill 
> the agent memory this way too.
> I'm attaching a patch which handles both of these issues and adds more 
> exception handling to buildEvent to make sure that reset() is called in 
> future unforeseen situations.
> Thanks also to [~roshan_naik] for helping to make this patch better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2763) flume_env script should handle jvm parameters like -javaagent -agentpath -agentlib

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2763.


> flume_env script should handle jvm parameters like -javaagent -agentpath 
> -agentlib
> --
>
> Key: FLUME-2763
> URL: https://issues.apache.org/jira/browse/FLUME-2763
> Project: Flume
>  Issue Type: Bug
>Reporter: Johny Rufus
>Assignee: Neerja Khattar
> Fix For: v1.7.0
>
> Attachments: FLUME-2763.patch
>
>
> flume_env script should handle jvm parameters like -javaagent -agentpath and 
> -agentlib when given as command line arguments [Currently only -X -D are 
> handled as java properties]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2753) Error when specifying empty replace string in Search and Replace Interceptor

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2753.


> Error when specifying empty replace string in Search and Replace Interceptor
> 
>
> Key: FLUME-2753
> URL: https://issues.apache.org/jira/browse/FLUME-2753
> Project: Flume
>  Issue Type: Bug
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.7.0
>
> Attachments: FLUME-2753-1.patch, FLUME-2753.patch
>
>
> The user guide specifies that the following configuration
> a1.sources.avroSrc.interceptors.search-replace.replaceString =
> will replace the search pattern with an empty string, but the following error 
> is observed and there is no way to specify an empty replace string
> java.lang.NullPointerException: Must supply a replacement string 
> replaceString (empty is ok)
> at 
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at 
> org.apache.flume.interceptor.SearchAndReplaceInterceptor$Builder.configure(SearchAndReplaceInterceptor.java:108)
> at 
> org.apache.flume.channel.ChannelProcessor.configureInterceptors(ChannelProcessor.java:110)
> We need to modify the SearchAndReplaceInterceptor to accept null values for 
> replacement string and use empty string in that case



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2754) Hive Sink skipping first transaction in each Batch of Hive Transactions

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2754.


> Hive Sink skipping first transaction in each Batch of Hive Transactions
> ---
>
> Key: FLUME-2754
> URL: https://issues.apache.org/jira/browse/FLUME-2754
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.0, v1.6.0
>Reporter: Roshan Naik
>Assignee: Deepesh Khandelwal
> Fix For: v1.7.0
>
> Attachments: FLUME-2754.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2751) Upgrade Derby version to 10.11.1.1

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2751.


> Upgrade Derby version to 10.11.1.1
> --
>
> Key: FLUME-2751
> URL: https://issues.apache.org/jira/browse/FLUME-2751
> Project: Flume
>  Issue Type: Bug
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.7.0
>
> Attachments: FLUME-2751.patch
>
>
> Derby has lot has bug fixes in the past releases over 3 years, we should 
> upgrade to the latest stable release to 10.11.1.1



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2749) Kerberos configuration error when using short names in multiple HDFS Sinks

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2749.


> Kerberos configuration error when using short names in multiple HDFS Sinks
> --
>
> Key: FLUME-2749
> URL: https://issues.apache.org/jira/browse/FLUME-2749
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.6.0
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.7.0
>
> Attachments: FLUME-2749.patch
>
>
> When we have more thank one HDFS Sink, configured in kerberos mode, and 
> principal is configured with a short name like 'flume' (without the @REALM 
> information), we get a 
> java.lang.IllegalStateException: Cannot use multiple kerberos principals in 
> the same agent.  Must restart agent to use new principal or keytab. Previous 
> = fl...@example.com (auth:KERBEROS), New = flume
>   at 
> com.google.common.base.Preconditions.checkState(Preconditions.java:172)
>   at 
> org.apache.flume.auth.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:131)
>   at 
> org.apache.flume.auth.FlumeAuthenticationUtil.getAuthenticator(FlumeAuthenticationUtil.java:67)
>   at 
> org.apache.flume.sink.hdfs.HDFSEventSink.configure(HDFSEventSink.java:261)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2732) Make maximum tolerated failures before shutting down and recreating client in AsyncHbaseSink configurable

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2732.


> Make maximum tolerated failures before shutting down and recreating client in 
> AsyncHbaseSink configurable
> -
>
> Key: FLUME-2732
> URL: https://issues.apache.org/jira/browse/FLUME-2732
> Project: Flume
>  Issue Type: Bug
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.7.0
>
> Attachments: FLUME-2732.patch
>
>
> In AsyncHbaseSink, the maximum consecutive transaction failures, after which 
> we shutdown the HbaseClient  and recreate the client, is currently hardcoded 
> to 10. (This change was introduced to overcome a Memory leak in 
> AsyncHbaseClient)
> This needs to be configurable, and defaulted to 0 (Unlimited)
> The reason for this change is to overcome a bug in the AsyncHbaseClient, that 
> starts leaking File Descriptors on shutdown. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2738) Async HBase sink FD leak on client shutdown

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2738.


> Async HBase sink FD leak on client shutdown
> ---
>
> Key: FLUME-2738
> URL: https://issues.apache.org/jira/browse/FLUME-2738
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.7.0
>
> Attachments: FLUME-2738-1.patch, FLUME-2738-2.patch, 
> FLUME-2738-3.patch, FLUME-2738.patch
>
>
> Currently every time Async Hbase Sink calls HBaseSink.shutdown, there is FD 
> leak due to HBaseSink using CustomChannelFactory where 
> releaseExternalResources() is overridden to a No-op. Need to replace this 
> with a standard NioClientSocketChannelFactory, that releases external 
> resources properly



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2734) Kafka Channel timeout property is overridden by default value

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2734.


> Kafka Channel timeout property is overridden by default value
> -
>
> Key: FLUME-2734
> URL: https://issues.apache.org/jira/browse/FLUME-2734
> Project: Flume
>  Issue Type: Bug
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.7.0
>
> Attachments: FLUME-2734.patch
>
>
> Kafka Channel timeout property which will be passed to the Kafka Consumer 
> internally, does not  work as expected, as in the code, it is overridden by 
> default value (or value specified by .timeout property which is undocumented)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2737) Documentation for Pollable Source config parameters introduced in FLUME-2729

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2737.


> Documentation for Pollable Source config parameters introduced in FLUME-2729
> 
>
> Key: FLUME-2737
> URL: https://issues.apache.org/jira/browse/FLUME-2737
> Project: Flume
>  Issue Type: Documentation
>Reporter: Johny Rufus
>Assignee: Ted Malaska
> Fix For: v1.7.0
>
> Attachments: FLUME-2737.patch, FLUME-2737.patch.1
>
>
> Documentation needs to be updated in Flume User Guide, for all Pollable 
> Sources, that would use the new config parameters introduced in FLUME-2729



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2718) HTTP Source to support generic Stream Handler

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2718.


> HTTP Source to support generic Stream Handler
> -
>
> Key: FLUME-2718
> URL: https://issues.apache.org/jira/browse/FLUME-2718
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Reporter: Hari
>Assignee: Hari
> Fix For: v1.7.0
>
> Attachments: 
> 0001-FLUME-2718-HTTP-Source-to-support-generic-Stream-Han.patch, 
> 0002-FLUME-2718-HTTP-Source-to-support-generic-Stream-Han.patch
>
>
> Currently the HTTP Source supports JSONHandler as the default implementation. 
> A more generic approach will be having a BLOBHandler which accepts any 
> request input stream (that loads the stream as Event payload). Furthermore, 
> this Handler lets you define mandatory request parameters and maps those 
> parameters into Event Headers. 
> This way HTTPSource can be used as a generic Data Ingress endpoint for any 
> sink, where one can specify attributes run like basepath, filename & 
> timestamp as request parameters and access those values via HEADER values in 
> sink properties.
> All this can be done without developing any custom Handler code.
> For e.g.
> With the below agent configuration, you can send any type of data 
> (JSON/CSV/TSV) and store it in any sink, HDFS in this case. 
> {code:title=sample command|borderStyle=solid}
> curl -v -X POST 
> "http://testHost:8080/?basepath=/data/=test.json=1434101498275;
>  --data @test.json
> {code}
> {code:title=HDFS data path |borderStyle=solid}
> /data/2015/06/12/test.json.1434101498275.lzo
> {code}
> {code:title=agent.conf|borderStyle=solid}
> #Agent configuration
> #HTTP Source configuration
> agent.sources = httpSrc
> agent.channels = memChannel
> agent.sources.httpSrc.type = http
> agent.sources.httpSrc.channels = memChannel
> agent.sources.httpSrc.bind = testHost
> agent.sources.httpSrc.port = 8080
> agent.sources.httpSrc.handler = org.apache.flume.source.http.BLOBHandler
> agent.sources.httpSrc.handler.mandatoryParameters = basepath, filename
> #Memory channel with default configuration
> agent.channels.memChannel.type = memory
> agent.channels.memChannel.capacity = 10
> agent.channels.memChannel.transactionCapacity = 1000
> #HDFS Sink configuration
> agent.sinks.hdfsSink.type = hdfs
> agent.sinks.hdfsSink.hdfs.path = %{basepath}/%Y/%m/%d
> agent.sinks.hdfsSink.hdfs.useLocalTimeStamp = true
> agent.sinks.hdfsSink.hdfs.filePrefix = %{filename}
> agent.sinks.hdfsSink.hdfs.fileType = CompressedStream
> agent.sinks.hdfsSink.hdfs.codeC = lzop
> agent.sinks.hdfsSink.channel = memChannel
> #Finally, activate.
> agent.channels = memChannel
> agent.sources = httpSrc
> agent.sinks = hdfsSink
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2729) Allow pollableSource backoff times to be configurable

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2729.


> Allow pollableSource backoff times to be configurable 
> --
>
> Key: FLUME-2729
> URL: https://issues.apache.org/jira/browse/FLUME-2729
> Project: Flume
>  Issue Type: Improvement
>Reporter: Ted Malaska
>Assignee: Ted Malaska
>Priority: Minor
> Fix For: v1.7.0
>
> Attachments: FLUME-2729.patch.2, FLUME-2729.patch.4, 
> FLUME-2729.patch.7, FLUME_2729.patch
>
>
> There are use cases that require more control of the backoff times setting.  
> This jira will allow those times to be set through config of the source



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2713) Document Fault Tolerant Config parameters in FlumeUserGuide

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2713.


> Document Fault Tolerant Config parameters in FlumeUserGuide
> ---
>
> Key: FLUME-2713
> URL: https://issues.apache.org/jira/browse/FLUME-2713
> Project: Flume
>  Issue Type: Documentation
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.7.0
>
> Attachments: FLUME-2713.patch
>
>
> The following FaultTolerance related parameters in MorphlineSolrSink need to 
> be documented in Flume user guide
> FaultTolerance.IS_PRODUCTION_MODE
> FaultTolerance.IS_IGNORING_RECOVERABLE_EXCEPTIONS
> FaultTolerance.RECOVERABLE_EXCEPTION_CLASSES



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2676) Update flume docs and release notes to 1.7-SNAPSHOT in trunk and 1.7 branch

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2676.


> Update flume docs and release notes to 1.7-SNAPSHOT in trunk and 1.7 branch
> ---
>
> Key: FLUME-2676
> URL: https://issues.apache.org/jira/browse/FLUME-2676
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.7.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2675) Update pom.xml version in trunk and 1.7 branch

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2675.


> Update pom.xml version in trunk and 1.7 branch
> --
>
> Key: FLUME-2675
> URL: https://issues.apache.org/jira/browse/FLUME-2675
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.7.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2704) Configurable poll delay for spooling directory source

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2704.


> Configurable poll delay for spooling directory source
> -
>
> Key: FLUME-2704
> URL: https://issues.apache.org/jira/browse/FLUME-2704
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: v1.6.0, v1.5.2
>Reporter: Somin Mithraa
>Assignee: Somin Mithraa
>Priority: Minor
>  Labels: SpoolDir, pollDelay, sources
> Fix For: v1.7.0
>
> Attachments: FLUME-2704.patch
>
>
> SpoolDir source polls a directory for new files at specific interval. This 
> interval(or poll delay) is currently hardcoded as 500ms.
> 500ms may be too fast for some applications. This JIRA is to make this 
> property configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2712) Optional channel errors slows down the Source to Main channel event rate

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2712.


> Optional channel errors slows down the Source to Main channel event rate
> 
>
> Key: FLUME-2712
> URL: https://issues.apache.org/jira/browse/FLUME-2712
> Project: Flume
>  Issue Type: Bug
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.7.0
>
> Attachments: FLUME-2712-1.patch, FLUME-2712-2.patch, 
> FLUME-2712.patch, FLUME-2712.patch
>
>
> When we have a source configured to deliver events to a main channel and an 
> optional channel, and if the delivery to optional channel fails, this 
> significantly slows down the rate at which events are delivered to the main 
> channel by the source.
> We need to evaluate async means of delivering events to the optional channel 
> and isolate the errors happening in optional channel from the delivery to the 
> main channel



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2660) Add documentation for EventValidator

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2660.


> Add documentation for EventValidator
> 
>
> Key: FLUME-2660
> URL: https://issues.apache.org/jira/browse/FLUME-2660
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.1
>Reporter: Hari Shreedharan
>Assignee: Ashish Paliwal
> Fix For: v1.7.0
>
> Attachments: FLUME-2660-0.patch, FLUME-2660-1.patch
>
>
> [~paliwalashish] - Assigning this to you. Please add docs for the 
> functionality you contributed in FLUME-2613



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2672) NPE in KafkaSourceCounter

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2672.


> NPE in KafkaSourceCounter
> -
>
> Key: FLUME-2672
> URL: https://issues.apache.org/jira/browse/FLUME-2672
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
> Environment: Mac OS 10.10.3, Java 1.7.0_60
>Reporter: Rigo MacTaggart
>Assignee: Rigo MacTaggart
>Priority: Trivial
>  Labels: easyfix
> Fix For: v1.7.0
>
> Attachments: FLUME-2672-with-test-git-diff.patch, 
> FLUME-2672-with-test.patch, FLUME-2672.patch
>
>   Original Estimate: 0h
>  Remaining Estimate: 0h
>
> A NPE is thrown when KafkaSource calls counter.incrementKafkaEmptyCount() 
> because it expects MonitoredCounterGroup.counterMap to contain key 
> "source.kafka.empty.count". A patch is included which adds this key to the 
> ATTRIBUTES string array, which is used to pre-populate keys with an initial 
> value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2652) Documented transaction handling semantics incorrect

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2652.


> Documented transaction handling semantics incorrect
> ---
>
> Key: FLUME-2652
> URL: https://issues.apache.org/jira/browse/FLUME-2652
> Project: Flume
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: v1.5.1
>Reporter: Graham Gear
>Assignee: Graham Gear
>Priority: Minor
> Fix For: v1.7.0
>
> Attachments: FLUME-2652-1.patch
>
>
> The Developer Guide Transaction and Source descriptions and Source example 
> are misleading, stating that transactions must be managed within the Source 
> space, when in fact they are managed within the ChannelProcessor. The Source 
> example code does not compile against the current API, nor is the intention 
> Source space transaction management possible with the current 
> ChannelProcessor.processEvent() and BasicTransactionSemantics.begin() 
> implementations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2628) Add an optional parameter to specify the expected input text encoding for the netcat sourcef the netcat source

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2628.


> Add an optional parameter to specify the expected input text encoding for the 
> netcat sourcef the netcat source
> --
>
> Key: FLUME-2628
> URL: https://issues.apache.org/jira/browse/FLUME-2628
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: v1.5.1
>Reporter: Lionel Herbet
>Assignee: Lionel Herbet
>Priority: Trivial
>  Labels: newbie, patch
> Fix For: v1.7.0
>
> Attachments: FLUME-2628.patch, FLUME-2628.patch, FLUME-2628.patch, 
> Netcat_Encoding.patch, Netcat_Encoding_WithUnitTest.patch, 
> Netcat_WithUnitTest.patch
>
>   Original Estimate: 0h
>  Remaining Estimate: 0h
>
> Add the 'encoding' configuration parameter to specify the expected text 
> encoding of the netcat source in input.
> This parameter is optional and defaulted to "utf-8" (same value as the one 
> hardcoded before the patch)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2632) High CPU on KafkaSink

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2632.


> High CPU on KafkaSink
> -
>
> Key: FLUME-2632
> URL: https://issues.apache.org/jira/browse/FLUME-2632
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.5.1
>Reporter: Gwen Shapira
>Assignee: Ashish Paliwal
>  Labels: kafka
> Fix For: v1.7.0
>
> Attachments: FLUME-2632-0.patch
>
>
> Reported here: https://github.com/harishreedharan/flume/issues/1
> "I tried flume-ng-kafka-sink and it worked fine. But I noticed that the cpu 
> utilization stay at 100% and never dropped down all the time even at the time 
> the channel is empty.
> I looked into the source code and found that "process" function in KafkaSink 
> always return Status.READY even if no events available in channel. That 
> causes the SinkRunner keep running achieving event from channel and get 
> nothing.
> Do we need to change to return Status.BACKOFF in "process" function in 
> KafkaSink when it notices that there is no events processed in current round? 
> So that the SinkRunner has a chance to take a rest when there is no event in 
> channel. If this proposal feasible, function "testEmptyChannel" in 
> TestKafkaSink also need to be changed. "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2575) FLUME-2548 brings SSLv2Hello back for Avro Sink, but UG says it is one of the protocols to exclude

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2575.


> FLUME-2548 brings SSLv2Hello back for Avro Sink, but UG says it is one of the 
> protocols to exclude
> --
>
> Key: FLUME-2575
> URL: https://issues.apache.org/jira/browse/FLUME-2575
> Project: Flume
>  Issue Type: Documentation
>  Components: Docs
>Reporter: li xiang
>Assignee: li xiang
>Priority: Minor
>  Labels: documentation
> Fix For: v1.7.0
>
> Attachments: FLUME-2575-0.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> FLUME-2548 changes NettyAvroRpcClient, to remove SSLv2Hello from the default 
> exclude protocol list. 
> But in flume UG(http://flume.apache.org/FlumeUserGuide.html), the chapter for 
> "Avro Sink", SSLv2Hello is still mentioned as the default for 
> "exclude-protocols"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2498) Implement Taildir Source

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2498.


> Implement Taildir Source
> 
>
> Key: FLUME-2498
> URL: https://issues.apache.org/jira/browse/FLUME-2498
> Project: Flume
>  Issue Type: New Feature
>  Components: Sinks+Sources
>Reporter: Satoshi Iijima
> Fix For: v1.7.0
>
> Attachments: FLUME-2498-2.patch, FLUME-2498-3.patch, 
> FLUME-2498-4.patch, FLUME-2498-5.patch, FLUME-2498.patch
>
>
> This is the proposal of implementing a new tailing source.
> This source watches the specified files, and tails them in nearly real-time 
> once appends are detected to these files.
> * This source is reliable and will not miss data even when the tailing files 
> rotate.
> * It periodically writes the last read position of each file in a position 
> file using the JSON format.
> * If Flume is stopped or down for some reason, it can restart tailing from 
> the position written on the existing position file.
> * It can add event headers to each tailing file group. 
> A attached patch includes a config documentation of this.
> This source requires Unix-style file system and Java 1.7 or later.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2485) Thrift Source tests fail on Oracle JDK 8

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2485.


> Thrift Source tests fail on Oracle JDK 8
> 
>
> Key: FLUME-2485
> URL: https://issues.apache.org/jira/browse/FLUME-2485
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Santiago M. Mola
>Assignee: Miroslav Holubec
>  Labels: jdk8
> Fix For: v1.7.0
>
> Attachments: 
> 0001-FLUME-2485-too-fast-processing-leads-to-failed-jUnit.patch, 
> FLUME-2485.patch
>
>
> Thrift Source tests fail on Oracle JDK 8:
> https://travis-ci.org/Stratio/flume/jobs/36817396#L6245
> testAppendBatch(org.apache.flume.source.TestThriftSource)  Time elapsed: 6083 
> sec  <<< FAILURE!
> java.lang.AssertionError
>   at org.junit.Assert.fail(Assert.java:92)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at org.junit.Assert.assertTrue(Assert.java:54)
>   at 
> org.apache.flume.source.TestThriftSource.testAppendBatch(TestThriftSource.java:144)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:483)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>   at 
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
>   at 
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
>   at 
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:483)
>   at 
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
>   at 
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
>   at 
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
>   at 
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
>   at 
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2215) ResettableFileInputStream can't support ucs-4 character

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2215.


> ResettableFileInputStream can't support  ucs-4 character
> 
>
> Key: FLUME-2215
> URL: https://issues.apache.org/jira/browse/FLUME-2215
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.0
>Reporter: syntony liu
>Assignee: Alexandre Dutra
>Priority: Critical
>  Labels: patch
> Fix For: v1.7.0
>
> Attachments: 
> 0001-FLUME-2215-Fixes-reading-surrogate-based-chars.patch, 
> FLUME-2215-0-README.txt, FLUME-2215-0.patch, FLUME-2215-1-README.txt, 
> FLUME-2215-1.patch
>
>
> ResettableFileInputStream.java:readChar() not handle ucs-4 character. it need 
> 2 charBuf. it cause an unexpected termination。
>  a temporary solution:
>  if (res.isOverflow() && !charBuf.hasRemaining()){ 
>  logger.warn("decoder ucs-4 at postion: {}" , buf.position()); 
> tmpBuf.clear();  
> res = decoder.decode(buf, tmpBuf, isEndOfInput); 
> incrPosition( buf.position() - start, false); 
>return '?'; 
>  } 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2801) Performance improvement on TailDir source

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2801.


> Performance improvement on TailDir source
> -
>
> Key: FLUME-2801
> URL: https://issues.apache.org/jira/browse/FLUME-2801
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: v1.7.0
>Reporter: Jun Seok Hong
>Assignee: Jun Seok Hong
> Fix For: v1.7.0
>
> Attachments: FLUME-2801-1.patch, FLUME-2801-2.patch, FLUME-2801.patch
>
>
> This a proposal of performance improvement for new tailing source FLUME-2498.
> Taildir source reads a file by 1byte, so the performance is very low compared 
> to tailing on exec source.
> I tested lot's of ways to improve performance and implemented the best one.
> Changes.
> * Reading a file by a 8k block instead of 1 byte.
> * Use byte[] for handling data instead of 
> ByteArrayDataOutput/ByteBuffer(direct)/.. for better performance.
> * Don't convert byte[] to string and vice verse.
> Simple file reading test results.
> {quote}
>  File size: 100 MB, 
>  Line size: 500 byte
> Estimated time to read the file:
> |Reading 1byte(Using the code in Taildir)|32544 ms|
> |Reading 8K Block|431 ms|
> {quote}
> Testing on flume, it catches up the performance of tailing on exec source. 
> (30x performance boost)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2820) Support New Kafka APIs

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2820.


> Support New Kafka APIs
> --
>
> Key: FLUME-2820
> URL: https://issues.apache.org/jira/browse/FLUME-2820
> Project: Flume
>  Issue Type: Improvement
>Reporter: Jeff Holoman
>Assignee: Jeff Holoman
> Fix For: v1.7.0
>
>
> Flume utilizes the older Kafka producer and consumer APIs. We should add 
> implementations of the Source, Channel and Sink that utilize the new APIs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2804) Hive sink - abort remaining transactions on shutdown

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2804.


> Hive sink - abort remaining transactions on shutdown
> 
>
> Key: FLUME-2804
> URL: https://issues.apache.org/jira/browse/FLUME-2804
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>  Labels: Hive
> Fix For: v1.7.0
>
> Attachments: FLUME-2804.patch
>
>
> Currently the hive sink does not explicitly abort unused transactions. 
> Although these eventually timeout on the hive side, it is preferable to 
> explicitly abort them so that the associated locks on the hive 
> table/partition are released. As long as the locks stay open, the 
> table/partition cannot be dropped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2821) Flume-Kafka Source with new Consumer

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2821.


> Flume-Kafka Source with new Consumer
> 
>
> Key: FLUME-2821
> URL: https://issues.apache.org/jira/browse/FLUME-2821
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Jeff Holoman
>Assignee: Grigoriy Rozhkov
> Fix For: v1.7.0
>
> Attachments: FLUME-2821.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2835) Hive Sink tests need to create table with transactional property set

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2835.


>  Hive Sink tests need to create table with transactional property set
> -
>
> Key: FLUME-2835
> URL: https://issues.apache.org/jira/browse/FLUME-2835
> Project: Flume
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
> Fix For: v1.7.0
>
> Attachments: FLUME-hive-test.patch
>
>
> As per Hive streaming wiki the  transactional=true property needs to be set 
> on the table for streaming. 
> https://cwiki.apache.org/confluence/display/Hive/Streaming+Data+Ingest



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2841) Upgrade commons-collections to 3.2.2

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2841.


> Upgrade commons-collections to 3.2.2
> 
>
> Key: FLUME-2841
> URL: https://issues.apache.org/jira/browse/FLUME-2841
> Project: Flume
>  Issue Type: Bug
>  Components: Build
>Affects Versions: v1.6.0
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.7.0
>
> Attachments: FLUME-2841.patch
>
>
> Refer: COLLECTIONS-580



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2823) Flume-Kafka-Channel with new APIs

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2823.


> Flume-Kafka-Channel with new APIs
> -
>
> Key: FLUME-2823
> URL: https://issues.apache.org/jira/browse/FLUME-2823
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Jeff Holoman
>Assignee: Jeff Holoman
> Fix For: v1.7.0
>
> Attachments: FLUME-2823v4.patch, FLUME-2823v6.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2822) Flume-Kafka-Sink with new Producer

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2822.


> Flume-Kafka-Sink with new Producer
> --
>
> Key: FLUME-2822
> URL: https://issues.apache.org/jira/browse/FLUME-2822
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Jeff Holoman
>Assignee: Jeff Holoman
> Fix For: v1.7.0
>
> Attachments: FLUME-2822.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2852) Kafka Source/Sink should optionally read/write Flume records

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2852.


> Kafka Source/Sink should optionally read/write Flume records
> 
>
> Key: FLUME-2852
> URL: https://issues.apache.org/jira/browse/FLUME-2852
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Tristan Stevens
>Assignee: Tristan Stevens
> Fix For: v1.7.0
>
> Attachments: FLUME-2852.patch, FLUME-2852v4.patch, FLUME-2852v5.patch
>
>
> Currently the Kafka Sink only writes the event body to Kafka rather than an 
> Avro Datum. This works fine when being used with a Kafka Source, or when 
> being used with Kafka Channel, however it does mean that any Flume headers 
> are lost when transported via Kafka.
> Request is to implement an equivalent of the Kafka Channel's 
> parseAsFlumeEvent parameter to the sink/source so that optionally Avro Datums 
> can be read from and written to Kafka.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2868) Kafka Channel partition topic by key

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2868.


> Kafka Channel partition topic by key
> 
>
> Key: FLUME-2868
> URL: https://issues.apache.org/jira/browse/FLUME-2868
> Project: Flume
>  Issue Type: Improvement
>  Components: Channel
>Affects Versions: v1.6.0
>Reporter: James Kinley
> Fix For: v1.7.0
>
>
> Add the ability to partition a Kafka channel topic by key. Users can 
> currently partition topics by key using a file/memory channel with a Kafka 
> sink. Although this approach adds unnecessary latency and the Kafka sink 
> drops Flume event headers, which can be undesirable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2881) Windows Launch Script fails in plugins dir code

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2881.


> Windows Launch Script fails in plugins dir code
> ---
>
> Key: FLUME-2881
> URL: https://issues.apache.org/jira/browse/FLUME-2881
> Project: Flume
>  Issue Type: Bug
>  Components: Configuration, Windows
>Affects Versions: v1.6.0
> Environment: Tested on Windows 7 and Windows 8
>Reporter: Jonathan Smith
>Assignee: Jonathan Smith
>  Labels: easyfix, patch, windows
> Fix For: v1.7.0
>
> Attachments: fix_windows_launch.patch, op-addition-not-found.log
>
>
> Running flume-ng.cmd results in the attached error from the Windows command 
> line.
> The problem seems to originate in flume-ng.ps1, line 323 where the plugins 
> are added to the class path. Adding together directory information does not 
> seem to be supported on windows 7 or 8. I was able to fix the problem by 
> separating out the two plugin directories in the attached patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2875) Allow RollingFileSink to specify a file prefix and a file extension.

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2875.


> Allow RollingFileSink to specify a file prefix and a file extension.
> 
>
> Key: FLUME-2875
> URL: https://issues.apache.org/jira/browse/FLUME-2875
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Ralph Goers
>Assignee: Ralph Goers
> Fix For: v1.7.0
>
> Attachments: FLUME-2875.patch
>
>
> Currently the RollingFileSink is hard-wired to use a specific PathManager 
> that creates file based on a timestamp and and incrementing value. User's 
> should have the ability to add a prefix to the file name and to add a file 
> extension to properly identify the type of data in the file.  In addition, 
> user's should have the ability to provide their own PathManager 
> implementation to allow them to construct the file names and locations 
> however they desire.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2886) Optional Channels can cause OOMs

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2886.


> Optional Channels can cause OOMs
> 
>
> Key: FLUME-2886
> URL: https://issues.apache.org/jira/browse/FLUME-2886
> Project: Flume
>  Issue Type: Bug
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.7.0
>
> Attachments: FLUME-2886.patch
>
>
> If an optional channel is full, the queue backing the executor that is 
> asynchronously submitting the events to the channel can grow indefinitely in 
> size leading to a huge number of events on the heap and causing OOMs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2889) Fixes to DateTime computations

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2889.


> Fixes to  DateTime computations
> ---
>
> Key: FLUME-2889
> URL: https://issues.apache.org/jira/browse/FLUME-2889
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.6.0
>Reporter: Roshan Naik
>Assignee: Tristan Stevens
> Fix For: v1.7.0
>
> Attachments: FLUME-2889-2.patch, FLUME-2889-4.patch, 
> FLUME-2889.3.patch, FLUME-2889.patch
>
>
> date.withYear(year+1)  can lead to incorrect date calculations .. for example 
> if  the date is Feb 29th.   need to use date.plusYears(1) instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2891) Revert FLUME-2712 and FLUME-2886

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2891.


> Revert FLUME-2712 and FLUME-2886
> 
>
> Key: FLUME-2891
> URL: https://issues.apache.org/jira/browse/FLUME-2891
> Project: Flume
>  Issue Type: Bug
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.7.0
>
> Attachments: FLUME-2891.patch
>
>
> FLUME-2712 can be fixed by simply setting keep-alive to 0. I think it added 
> additional complexity, which we can probably avoid. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2897) AsyncHBase sink NPE when Channel.getTransaction() fails

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2897.


> AsyncHBase sink NPE when Channel.getTransaction() fails
> ---
>
> Key: FLUME-2897
> URL: https://issues.apache.org/jira/browse/FLUME-2897
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Mike Percy
>Assignee: Mike Percy
> Fix For: v1.7.0
>
> Attachments: FLUME-2897-1.patch
>
>
> There is a possibility for a NPE in the AsyncHBaseSink when a channel 
> getTransaction() call fails. This is possible when the FileChannel is out of 
> disk space.
> Patch forthcoming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2908) NetcatSource - SocketChannel not closed when session is broken

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2908.


> NetcatSource - SocketChannel not closed when session is broken
> --
>
> Key: FLUME-2908
> URL: https://issues.apache.org/jira/browse/FLUME-2908
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.3.0
>Reporter: Grant Henke
>Assignee: Grant Henke
> Fix For: v1.7.0
>
> Attachments: FLUME-2908-1.patch, FLUME-2908.patch
>
>
> When the NetcatSource socketChannel receives an IOException due to an 
> abruptly closed session the exception is caught but the socketChannel is 
> never closed. This results in leaked sockets and ever growing file 
> descriptors. 
> Thanks to [~pdvorak] for finding the leak. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2909) Bump Rat version

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2909.


> Bump Rat version
> 
>
> Key: FLUME-2909
> URL: https://issues.apache.org/jira/browse/FLUME-2909
> Project: Flume
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: v1.7.0
>Reporter: Bessenyei Balázs Donát
>Assignee: Bessenyei Balázs Donát
>Priority: Minor
> Fix For: v1.7.0
>
> Attachments: FLUME-2909.patch
>
>
> The current version of Rat ( http://creadur.apache.org/rat/index.html ) is 
> outdated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2910) AsyncHBaseSink - Failure callbacks should log the exception that caused them

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2910.


> AsyncHBaseSink - Failure callbacks should log the exception that caused them
> 
>
> Key: FLUME-2910
> URL: https://issues.apache.org/jira/browse/FLUME-2910
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Abraham Fine
>Assignee: Abraham Fine
> Fix For: v1.7.0
>
> Attachments: FLUME-2910.patch
>
>
> Failure callbacks in the AsyncHBaseSink currently do not log the exception 
> that causes them to be called, this should be fixed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2915) The kafka channel using new APIs will be stuck when the sink is avro sink

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2915.


> The kafka channel using new APIs will be stuck when the sink is avro sink
> -
>
> Key: FLUME-2915
> URL: https://issues.apache.org/jira/browse/FLUME-2915
> Project: Flume
>  Issue Type: Bug
>  Components: Channel
>Affects Versions: v1.7.0
>Reporter: Julian Zhang
>Assignee: Jeff Holoman
> Fix For: v1.7.0
>
> Attachments: FLUME-2915.patch, FLUME-2915v2.patch
>
>
> The avro sink was stuck when I using the kafka channel which using the new 
> APIs.
> After couple of hours I found the issue at KafkaChannel.java#L384:
> e.getHeaders().put(KEY_HEADER, record.key());
> and change it to:
> if (record.key() != null) {
> e.getHeaders().put(KEY_HEADER, record.key());
> }
> The reason is:
> record.key() could be null if the user didn't set it. And the avro serialize 
> the event will throw a NullPointerException.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2920) Kafka Channel Should Not Commit Offsets When Stopping

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2920.


> Kafka Channel Should Not Commit Offsets When Stopping
> -
>
> Key: FLUME-2920
> URL: https://issues.apache.org/jira/browse/FLUME-2920
> Project: Flume
>  Issue Type: Bug
>  Components: Channel
>Affects Versions: v1.6.0
>Reporter: Kevin Conaway
>Assignee: Kevin Conaway
>Priority: Critical
> Fix For: v1.7.0
>
> Attachments: FLUME-2920.patch
>
>
> The Kafka channel commits the consumer offsets when shutting down (via stop() 
> -> decommissionConsumerAndRecords())
> This can lead to data loss if the channel is shut down while messages have 
> been fetched in a transaction but the transaction has not yet been committed.
> The only time that the offsets should be committed is when a transaction is 
> complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2913) Flume classpath too long

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2913.


> Flume classpath too long
> 
>
> Key: FLUME-2913
> URL: https://issues.apache.org/jira/browse/FLUME-2913
> Project: Flume
>  Issue Type: Bug
>Reporter: Bessenyei Balázs Donát
>Assignee: Bessenyei Balázs Donát
> Fix For: v1.7.0
>
> Attachments: FLUME-2913.patch
>
>
> Starting Flume generates the FLUME_CLASSPATH variable dynamically, adding 
> jars one by one.
> In some cases, the classpath can grow too long, sometimes exceeding ARG_MAX, 
> maximum length of arguments for a new process ( 
> http://www.in-ulm.de/~mascheck/various/argmax/ )
> Thus, being unable to start Flume:
> flume-ng/bin/flume-ng: line 239: /usr/java/jdk1.7.0_67/bin/java: Argument 
> list too long



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (FLUME-2921) Support Elasticsearch 2.0+

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno updated FLUME-2921:
-
Attachment: (was: FLUME-2921-1.patch)

> Support Elasticsearch 2.0+
> --
>
> Key: FLUME-2921
> URL: https://issues.apache.org/jira/browse/FLUME-2921
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Reporter: Lior Zeno
> Fix For: v1.7.0
>
> Attachments: FLUME-2921-0.patch, FLUME-2921-1.patch
>
>
> Elasticsearch sink supports an ancient version of ES. We should make the sink 
> work with newer versions of Elasticsearch.
> I attached a patch for that. Please note that this involves upgrading netty 
> and guava.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (FLUME-2921) Support Elasticsearch 2.0+

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno updated FLUME-2921:
-
Attachment: FLUME-2921-1.patch

> Support Elasticsearch 2.0+
> --
>
> Key: FLUME-2921
> URL: https://issues.apache.org/jira/browse/FLUME-2921
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Reporter: Lior Zeno
> Fix For: v1.7.0
>
> Attachments: FLUME-2921-0.patch, FLUME-2921-1.patch
>
>
> Elasticsearch sink supports an ancient version of ES. We should make the sink 
> work with newer versions of Elasticsearch.
> I attached a patch for that. Please note that this involves upgrading netty 
> and guava.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2921) Support Elasticsearch 2.0+

[Lior Zeno (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338025#comment-15338025
 ] 

Lior Zeno commented on FLUME-2921:
--

I just updated the patch. FLUME-2923 already bumped netty version. In addition, 
I simplified TestElasticSearchIndexRequestBuilderFactory.  

> Support Elasticsearch 2.0+
> --
>
> Key: FLUME-2921
> URL: https://issues.apache.org/jira/browse/FLUME-2921
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Reporter: Lior Zeno
> Fix For: v1.7.0
>
> Attachments: FLUME-2921-0.patch, FLUME-2921-1.patch
>
>
> Elasticsearch sink supports an ancient version of ES. We should make the sink 
> work with newer versions of Elasticsearch.
> I attached a patch for that. Please note that this involves upgrading netty 
> and guava.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (FLUME-2921) Support Elasticsearch 2.0+

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno updated FLUME-2921:
-
Attachment: FLUME-2921-1.patch

> Support Elasticsearch 2.0+
> --
>
> Key: FLUME-2921
> URL: https://issues.apache.org/jira/browse/FLUME-2921
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Reporter: Lior Zeno
> Fix For: v1.7.0
>
> Attachments: FLUME-2921-0.patch, FLUME-2921-1.patch
>
>
> Elasticsearch sink supports an ancient version of ES. We should make the sink 
> work with newer versions of Elasticsearch.
> I attached a patch for that. Please note that this involves upgrading netty 
> and guava.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLUME-2923) Bump AsyncHBase version

[Lior Zeno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lior Zeno closed FLUME-2923.


> Bump AsyncHBase version
> ---
>
> Key: FLUME-2923
> URL: https://issues.apache.org/jira/browse/FLUME-2923
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.7.0
>Reporter: Bessenyei Balázs Donát
>Assignee: Bessenyei Balázs Donát
>Priority: Minor
> Fix For: v1.7.0
>
> Attachments: FLUME-2923-2.patch, FLUME-2923-3.patch, FLUME-2923.patch
>
>
> The current version of AsyncHBase ( http://opentsdb.github.io/asynchbase/ ) 
> is outdated.
> Reason for the upgrade would be many bug fixes and some new public APIs. (See 
> https://github.com/OpenTSDB/asynchbase/blob/master/NEWS )



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)