[jira] [Commented] (HADOOP-12556) KafkaSink jar files are created but not copied to target dist

2016-01-08 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089578#comment-15089578
 ] 

Babak Behzad commented on HADOOP-12556:
---

Sure, sounds good to me! Thanks [~steve_l]. What do you think [~raviprak] and 
[~aw]?

> KafkaSink jar files are created but not copied to target dist
> -
>
> Key: HADOOP-12556
> URL: https://issues.apache.org/jira/browse/HADOOP-12556
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Babak Behzad
>Assignee: Babak Behzad
> Attachments: HADOOP-12556.patch
>
>
> There is a hadoop-kafka artifact missing from hadoop-tools-dist's pom.xml 
> which was causing the compiled Kafka jar files not to be copied to the target 
> dist directory. The new patch adds this in order to complete this fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HADOOP-12556) KafkaSink jar files are created but not copied to target dist

2015-11-06 Thread Babak Behzad (JIRA)
Babak Behzad created HADOOP-12556:
-

 Summary: KafkaSink jar files are created but not copied to target 
dist
 Key: HADOOP-12556
 URL: https://issues.apache.org/jira/browse/HADOOP-12556
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Babak Behzad
Assignee: Babak Behzad


There is a hadoop-kafka artifact missing from hadoop-tools-dist's pom.xml which 
was causing the compiled Kafka jar files not to be copied to the target dist 
directory. The new patch adds this in order to complete this fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HADOOP-12556) KafkaSink jar files are created but not copied to target dist

2015-11-06 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-12556:
--
Attachment: HADOOP-12556.patch

> KafkaSink jar files are created but not copied to target dist
> -
>
> Key: HADOOP-12556
> URL: https://issues.apache.org/jira/browse/HADOOP-12556
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Babak Behzad
>Assignee: Babak Behzad
> Attachments: HADOOP-12556.patch
>
>
> There is a hadoop-kafka artifact missing from hadoop-tools-dist's pom.xml 
> which was causing the compiled Kafka jar files not to be copied to the target 
> dist directory. The new patch adds this in order to complete this fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HADOOP-10949) metrics2 sink plugin for Apache Kafka

2015-11-06 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994028#comment-14994028
 ] 

Babak Behzad commented on HADOOP-10949:
---

Since this ticket is already fixed, we created a new JIRA for the above fix. 
Here's the link: [https://issues.apache.org/jira/browse/HADOOP-12556]

> metrics2 sink plugin for Apache Kafka
> -
>
> Key: HADOOP-10949
> URL: https://issues.apache.org/jira/browse/HADOOP-10949
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: metrics
>Reporter: Babak Behzad
>Assignee: Babak Behzad
> Fix For: 3.0.0
>
> Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
> HADOOP-10949-4.patch, HADOOP-10949-5.patch, HADOOP-10949-6-1.patch, 
> HADOOP-10949-6.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch
>
>
> Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
> Kafka in addition to the current, Graphite 
> ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
> and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HADOOP-12556) KafkaSink jar files are created but not copied to target dist

2015-11-06 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994039#comment-14994039
 ] 

Babak Behzad commented on HADOOP-12556:
---

This is a short fix related to the KafkaSink 
([https://issues.apache.org/jira/browse/HADOOP-10949]). Since the ticket was 
closed, we created this new ticket, as [~aw] instructed me to do so. The patch 
is attached.

> KafkaSink jar files are created but not copied to target dist
> -
>
> Key: HADOOP-12556
> URL: https://issues.apache.org/jira/browse/HADOOP-12556
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Babak Behzad
>Assignee: Babak Behzad
> Attachments: HADOOP-12556.patch
>
>
> There is a hadoop-kafka artifact missing from hadoop-tools-dist's pom.xml 
> which was causing the compiled Kafka jar files not to be copied to the target 
> dist directory. The new patch adds this in order to complete this fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HADOOP-10949) metrics2 sink plugin for Apache Kafka

2015-10-26 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14975192#comment-14975192
 ] 

Babak Behzad commented on HADOOP-10949:
---

There was a hadoop-kafka artifact missing from hadoop-tools-dist's pom.xml 
which was causing the compiled Kafka jar files to be copied to the target dist 
directory. The new patch adds this in order to complete this fix.

> metrics2 sink plugin for Apache Kafka
> -
>
> Key: HADOOP-10949
> URL: https://issues.apache.org/jira/browse/HADOOP-10949
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: metrics
>Reporter: Babak Behzad
>Assignee: Babak Behzad
> Fix For: 3.0.0
>
> Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
> HADOOP-10949-4.patch, HADOOP-10949-5.patch, HADOOP-10949-6-1.patch, 
> HADOOP-10949-6.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch
>
>
> Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
> Kafka in addition to the current, Graphite 
> ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
> and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HADOOP-10949) metrics2 sink plugin for Apache Kafka

2015-10-26 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10949:
--
Attachment: HADOOP-10949-6-1.patch

> metrics2 sink plugin for Apache Kafka
> -
>
> Key: HADOOP-10949
> URL: https://issues.apache.org/jira/browse/HADOOP-10949
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: metrics
>Reporter: Babak Behzad
>Assignee: Babak Behzad
> Fix For: 3.0.0
>
> Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
> HADOOP-10949-4.patch, HADOOP-10949-5.patch, HADOOP-10949-6-1.patch, 
> HADOOP-10949-6.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch
>
>
> Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
> Kafka in addition to the current, Graphite 
> ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
> and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2015-09-08 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735411#comment-14735411
 ] 

Babak Behzad commented on HADOOP-10949:
---

Ok, based on comments from [~raviprak], [~aw], [~steve_l] we moved KafkaSink 
code and its test file into a new directory under 
{{/hadoop-tools/hadoop-kafka}}. This will avoid the problem of having 
hadoop-common depend on any Kafka libraries. The new patch file uploaded 
({{HADOOP-10949-6.patch}}) has all the corresponding updated version of the 
{{pom.xml}} files and code.

> Write metrics2 sink plugin for Apache Kafka -- KafkaSink
> 
>
> Key: HADOOP-10949
> URL: https://issues.apache.org/jira/browse/HADOOP-10949
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: metrics
>Reporter: Babak Behzad
>Assignee: Babak Behzad
> Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
> HADOOP-10949-4.patch, HADOOP-10949-5.patch, HADOOP-10949-6.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch
>
>
> Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
> Kafka in addition to the current, Graphite 
> ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
> and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2015-09-08 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10949:
--
Attachment: HADOOP-10949-6.patch

> Write metrics2 sink plugin for Apache Kafka -- KafkaSink
> 
>
> Key: HADOOP-10949
> URL: https://issues.apache.org/jira/browse/HADOOP-10949
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: metrics
>Reporter: Babak Behzad
>Assignee: Babak Behzad
> Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
> HADOOP-10949-4.patch, HADOOP-10949-5.patch, HADOOP-10949-6.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch
>
>
> Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
> Kafka in addition to the current, Graphite 
> ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
> and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2015-09-03 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10949:
--
Attachment: HADOOP-10949-5.patch

> Write metrics2 sink plugin for Apache Kafka -- KafkaSink
> 
>
> Key: HADOOP-10949
> URL: https://issues.apache.org/jira/browse/HADOOP-10949
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: metrics
>Reporter: Babak Behzad
>Assignee: Babak Behzad
> Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
> HADOOP-10949-4.patch, HADOOP-10949-5.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch
>
>
> Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
> Kafka in addition to the current, Graphite 
> ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
> and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2015-09-03 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730110#comment-14730110
 ] 

Babak Behzad commented on HADOOP-10949:
---

I have attached the patch with [~steve_l]'s comment of using SLF4J instead of 
commons-logging.

The only decision to make now seems to be where to put this code so that 
everyone is happy. I am going to discuss it more and will put a comment here 
soon.

> Write metrics2 sink plugin for Apache Kafka -- KafkaSink
> 
>
> Key: HADOOP-10949
> URL: https://issues.apache.org/jira/browse/HADOOP-10949
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: metrics
>Reporter: Babak Behzad
>Assignee: Babak Behzad
> Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
> HADOOP-10949-4.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
> HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch
>
>
> Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
> Kafka in addition to the current, Graphite 
> ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
> and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HADOOP-12360) Create StatsD metrics2 sink

2015-09-01 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-12360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14725758#comment-14725758
 ] 

Babak Behzad commented on HADOOP-12360:
---

[~dlmarion], it seems that there is an interest in putting all the new sinks 
into a separate module as discussed in 
[HADOOP-10949|https://issues.apache.org/jira/browse/HADOOP-10949]. The main 
reason being there may be some dependencies that these sinks have, but 
hadoop-common should not have.

> Create StatsD metrics2 sink
> ---
>
> Key: HADOOP-12360
> URL: https://issues.apache.org/jira/browse/HADOOP-12360
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: metrics
>Affects Versions: 2.7.1
>Reporter: Dave Marion
>Assignee: Dave Marion
>Priority: Minor
> Attachments: HADOOP-12360.001.patch, HADOOP-12360.002.patch, 
> HADOOP-12360.003.patch, HADOOP-12360.004.patch
>
>
> Create a metrics sink that pushes to a StatsD daemon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2015-08-27 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717339#comment-14717339
 ] 

Babak Behzad commented on HADOOP-10949:
---

The patch that I just uploaded (HADOOP-10949-4.patch) should pass everything, 
as I just checked on my own machine. [~raviprak] and I are working on the 
logistics of putting these sinks into a separate module as [~wheat9] suggested.

 Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 

 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
  Components: metrics
Reporter: Babak Behzad
Assignee: Babak Behzad
 Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
 HADOOP-10949-4.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch


 Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
 Kafka in addition to the current, Graphite 
 ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
 and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2015-08-27 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10949:
--
Attachment: HADOOP-10949-4.patch

 Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 

 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
  Components: metrics
Reporter: Babak Behzad
Assignee: Babak Behzad
 Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
 HADOOP-10949-4.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch


 Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
 Kafka in addition to the current, Graphite 
 ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
 and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2015-08-26 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10949:
--
Attachment: HADOOP-10949-3.patch

 Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 

 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
  Components: metrics
Reporter: Babak Behzad
Assignee: Babak Behzad
 Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
 HADOOP-10949-3.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch


 Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
 Kafka in addition to the current, Graphite 
 ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
 and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2015-08-26 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10949:
--
Attachment: (was: HADOOP-10949-3.patch)

 Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 

 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
  Components: metrics
Reporter: Babak Behzad
Assignee: Babak Behzad
 Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, 
 HADOOP-10949.patch, HADOOP-10949.patch


 Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
 Kafka in addition to the current, Graphite 
 ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
 and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2014-10-06 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160393#comment-14160393
 ] 

Babak Behzad commented on HADOOP-10949:
---

[~aw]: The reason that it does not compile is that it needs Kafka jar 
libraries! How can we add those? Should I also upload pom.xml patch here?

 Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 

 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
  Components: metrics
Reporter: Babak Behzad
Assignee: Babak Behzad
 Fix For: 2.2.0

 Attachments: HADOOP-10949.patch


 Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
 Kafka in addition to the current, Graphite 
 ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
 and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2014-10-05 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10949:
--
Fix Version/s: 2.2.0
   Status: Patch Available  (was: Open)

 Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 

 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
  Components: metrics
Reporter: Babak Behzad
Assignee: Babak Behzad
 Fix For: 2.2.0

 Attachments: HADOOP-10949.patch


 Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
 Kafka in addition to the current, Graphite 
 ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
 and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2014-10-05 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10949:
--
Attachment: HADOOP-10949.patch

 Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 

 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
  Components: metrics
Reporter: Babak Behzad
Assignee: Babak Behzad
 Fix For: 2.2.0

 Attachments: HADOOP-10949.patch


 Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
 Kafka in addition to the current, Graphite 
 ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
 and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2014-10-05 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159703#comment-14159703
 ] 

Babak Behzad commented on HADOOP-10949:
---

I attached my first patch available for this. It has been tested, but needs 
reviews and also a test. I will start working on the test for it soon.

 Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 

 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
  Components: metrics
Reporter: Babak Behzad
Assignee: Babak Behzad
 Fix For: 2.2.0

 Attachments: HADOOP-10949.patch


 Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
 Kafka in addition to the current, Graphite 
 ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
 and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2014-08-08 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10949:
--

Description: Write a metrics2 sink plugin for Hadoop to send metrics 
directly to Apache Kafka in additional to the current, Graphite 
([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia and 
File sinks.  (was: Write a metrics2 sink plugin for Hadoop to send metrics 
directly to Apache Kafka in additional to the current, 
Graphite([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), 
Ganglia and File sinks.)

 Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 

 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
Reporter: Babak Behzad
Assignee: Babak Behzad

 Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
 Kafka in additional to the current, Graphite 
 ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
 and File sinks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2014-08-08 Thread Babak Behzad (JIRA)
Babak Behzad created HADOOP-10949:
-

 Summary: Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
Reporter: Babak Behzad
Assignee: Babak Behzad


Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
Kafka in additional to the current, 
Graphite([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), 
Ganglia and File sinks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink

2014-08-08 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10949:
--

Description: Write a metrics2 sink plugin for Hadoop to send metrics 
directly to Apache Kafka in addition to the current, Graphite 
([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia and 
File sinks.  (was: Write a metrics2 sink plugin for Hadoop to send metrics 
directly to Apache Kafka in additional to the current, Graphite 
([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia and 
File sinks.)

 Write metrics2 sink plugin for Apache Kafka -- KafkaSink
 

 Key: HADOOP-10949
 URL: https://issues.apache.org/jira/browse/HADOOP-10949
 Project: Hadoop Common
  Issue Type: New Feature
  Components: metrics
Reporter: Babak Behzad
Assignee: Babak Behzad

 Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache 
 Kafka in addition to the current, Graphite 
 ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]), Ganglia 
 and File sinks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-10727) GraphiteSink metric names should not contain .

2014-07-08 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10727:
--

Attachment: HADOOP-10727-v2.patch

 GraphiteSink metric names should not contain .
 

 Key: HADOOP-10727
 URL: https://issues.apache.org/jira/browse/HADOOP-10727
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Babak Behzad
Priority: Minor
 Attachments: HADOOP-10727-v2.patch, HADOOP-10727.patch


 Sometimes the names of the metrics sent to Graphite contain . in them (such 
 as hostnames). Graphite interpret these dots as a separator for directories 
 causing long directory hierarchies. It would be easier to replace them to _ 
 in order to have easier to read metric names.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-10727) GraphiteSink metric names should not contain .

2014-07-08 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055394#comment-14055394
 ] 

Babak Behzad commented on HADOOP-10727:
---

Thanks Ravi. Both of your comments are incorporated in the new patch I uploaded.

 GraphiteSink metric names should not contain .
 

 Key: HADOOP-10727
 URL: https://issues.apache.org/jira/browse/HADOOP-10727
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Babak Behzad
Priority: Minor
 Attachments: HADOOP-10727-v2.patch, HADOOP-10727.patch


 Sometimes the names of the metrics sent to Graphite contain . in them (such 
 as hostnames). Graphite interpret these dots as a separator for directories 
 causing long directory hierarchies. It would be easier to replace them to _ 
 in order to have easier to read metric names.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-10727) GraphiteSink metric names should not contain .

2014-07-01 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10727:
--

Attachment: HADOOP-10727.patch

 GraphiteSink metric names should not contain .
 

 Key: HADOOP-10727
 URL: https://issues.apache.org/jira/browse/HADOOP-10727
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Babak Behzad
Priority: Minor
 Attachments: HADOOP-10727.patch


 Sometimes the names of the metrics sent to Graphite contain . in them (such 
 as hostnames). Graphite interpret these dots as a separator for directories 
 causing long directory hierarchies. It would be easier to replace them to _ 
 in order to have easier to read metric names.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-10727) GraphiteSink metric names should not contain .

2014-07-01 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049274#comment-14049274
 ] 

Babak Behzad commented on HADOOP-10727:
---

I attached my first shot on this. It would be great if someone can review it 
and let me know what they think.

 GraphiteSink metric names should not contain .
 

 Key: HADOOP-10727
 URL: https://issues.apache.org/jira/browse/HADOOP-10727
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Babak Behzad
Priority: Minor
 Attachments: HADOOP-10727.patch


 Sometimes the names of the metrics sent to Graphite contain . in them (such 
 as hostnames). Graphite interpret these dots as a separator for directories 
 causing long directory hierarchies. It would be easier to replace them to _ 
 in order to have easier to read metric names.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-10727) GraphiteSink metric names should not contain .

2014-06-27 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046155#comment-14046155
 ] 

Babak Behzad commented on HADOOP-10727:
---

Thanks [~cwimmer]. I like your idea of making it configurable. We can put it as 
a new option in hadoop-metrics2.properties file along with other options such 
as server_host, server_port and metrics_prefix. Following the naming convention 
of Graphite we can have the following line to replace all the . to ::
*.sink.graphite.escape_character=:
Default is _ (Underscore).

 GraphiteSink metric names should not contain .
 

 Key: HADOOP-10727
 URL: https://issues.apache.org/jira/browse/HADOOP-10727
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Babak Behzad
Priority: Minor

 Sometimes the names of the metrics sent to Graphite contain . in them (such 
 as hostnames). Graphite interpret these dots as a separator for directories 
 causing long directory hierarchies. It would be easier to replace them to _ 
 in order to have easier to read metric names.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-10715) Make GraphiteSink#setWriter() private

2014-06-25 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10715:
--

Attachment: HADOOP-10715.patch

 Make GraphiteSink#setWriter() private
 -

 Key: HADOOP-10715
 URL: https://issues.apache.org/jira/browse/HADOOP-10715
 Project: Hadoop Common
  Issue Type: Task
Reporter: Ted Yu
Priority: Minor
 Attachments: HADOOP-10715.patch


 During review of HADOOP-10660, Ravi brought up the notion of making 
 GraphiteSink#setWriter() private.
 This JIRA is to address Ravi's comment.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-10715) Make GraphiteSink#setWriter() private

2014-06-19 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038107#comment-14038107
 ] 

Babak Behzad commented on HADOOP-10715:
---

I just uploaded the patch. Thanks [~raviprak] and [~tedyu].

 Make GraphiteSink#setWriter() private
 -

 Key: HADOOP-10715
 URL: https://issues.apache.org/jira/browse/HADOOP-10715
 Project: Hadoop Common
  Issue Type: Task
Reporter: Ted Yu
Priority: Minor
 Attachments: HADOOP-10715.patch


 During review of HADOOP-10660, Ravi brought up the notion of making 
 GraphiteSink#setWriter() private.
 This JIRA is to address Ravi's comment.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-10715) Make GraphiteSink#setWriter() private

2014-06-19 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10715:
--

Attachment: HADOOP-10715.patch

 Make GraphiteSink#setWriter() private
 -

 Key: HADOOP-10715
 URL: https://issues.apache.org/jira/browse/HADOOP-10715
 Project: Hadoop Common
  Issue Type: Task
Reporter: Ted Yu
Priority: Minor
 Attachments: HADOOP-10715.patch


 During review of HADOOP-10660, Ravi brought up the notion of making 
 GraphiteSink#setWriter() private.
 This JIRA is to address Ravi's comment.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HADOOP-10727) GraphiteSink metric names should not contain .

2014-06-19 Thread Babak Behzad (JIRA)
Babak Behzad created HADOOP-10727:
-

 Summary: GraphiteSink metric names should not contain .
 Key: HADOOP-10727
 URL: https://issues.apache.org/jira/browse/HADOOP-10727
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Babak Behzad
Priority: Minor


Sometimes the names of the metrics sent to Graphite contain . in them (such 
as hostnames). Graphite interpret these dots as a separator for directories 
causing long directory hierarchies. It would be easier to replace them to _ 
in order to have easier to read metric names.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-10715) Make GraphiteSink#setWriter() private

2014-06-19 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-10715:
--

Attachment: (was: HADOOP-10715.patch)

 Make GraphiteSink#setWriter() private
 -

 Key: HADOOP-10715
 URL: https://issues.apache.org/jira/browse/HADOOP-10715
 Project: Hadoop Common
  Issue Type: Task
Reporter: Ted Yu
Priority: Minor

 During review of HADOOP-10660, Ravi brought up the notion of making 
 GraphiteSink#setWriter() private.
 This JIRA is to address Ravi's comment.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-10715) Make GraphiteSink#setWriter() private

2014-06-19 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038189#comment-14038189
 ] 

Babak Behzad commented on HADOOP-10715:
---

The tests are not compilable with that patch and I need to re-structure them. 
Let me work on it and I'll apply a new patch soon. Deleting the current one for 
now, since it has problems.

 Make GraphiteSink#setWriter() private
 -

 Key: HADOOP-10715
 URL: https://issues.apache.org/jira/browse/HADOOP-10715
 Project: Hadoop Common
  Issue Type: Task
Reporter: Ted Yu
Priority: Minor

 During review of HADOOP-10660, Ravi brought up the notion of making 
 GraphiteSink#setWriter() private.
 This JIRA is to address Ravi's comment.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-10660) GraphiteSink should implement Closeable

2014-06-12 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029452#comment-14029452
 ] 

Babak Behzad commented on HADOOP-10660:
---

No, thanks [~ted_yu], I am currently in the process of testing this. I will let 
you know soon.

 GraphiteSink should implement Closeable
 ---

 Key: HADOOP-10660
 URL: https://issues.apache.org/jira/browse/HADOOP-10660
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Chen He
 Attachments: HADOOP-10660-v2.patch, HADOOP-10660-v3.patch, 
 HADOOP-10660.patch


 GraphiteSink wraps OutputStreamWriter around socket's output stream.
 Currently the socket is never closed.
 GraphiteSink should implement Closeable such that MetricsSystem can close the 
 socket when it is stopped.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-10660) GraphiteSink should implement Closeable

2014-06-12 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029687#comment-14029687
 ] 

Babak Behzad commented on HADOOP-10660:
---

I just finished testing this patch on my own local Hadoop environment. 
Everything seems to be working correctly. Did the pull request in order to 
deploy it. Thanks [~ted_yu] and [~airbots].

 GraphiteSink should implement Closeable
 ---

 Key: HADOOP-10660
 URL: https://issues.apache.org/jira/browse/HADOOP-10660
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Chen He
 Attachments: HADOOP-10660-v2.patch, HADOOP-10660-v3.patch, 
 HADOOP-10660.patch


 GraphiteSink wraps OutputStreamWriter around socket's output stream.
 Currently the socket is never closed.
 GraphiteSink should implement Closeable such that MetricsSystem can close the 
 socket when it is stopped.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-10660) GraphiteSink should implement Closeable

2014-06-09 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025359#comment-14025359
 ] 

Babak Behzad commented on HADOOP-10660:
---

Thanks [~airbots] and [~ted_yu]. We have a system set up for using this and I 
will give this patch a test today. I will let you guys know about these 
experiments.

 GraphiteSink should implement Closeable
 ---

 Key: HADOOP-10660
 URL: https://issues.apache.org/jira/browse/HADOOP-10660
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Chen He
 Attachments: HADOOP-10660-v2.patch, HADOOP-10660-v3.patch, 
 HADOOP-10660.patch


 GraphiteSink wraps OutputStreamWriter around socket's output stream.
 Currently the socket is never closed.
 GraphiteSink should implement Closeable such that MetricsSystem can close the 
 socket when it is stopped.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-10660) GraphiteSink should implement Closeable

2014-06-05 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-10660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018904#comment-14018904
 ] 

Babak Behzad commented on HADOOP-10660:
---

A couple of colleagues and I contributed the GraphiteSink metric and it got 
into the trunk a couple of days ago. Thanks for the bug report. I see the 
problem, but was wondering why none of the other sinks (FileSink and 
GangliaSink) implement Closeable as well? I also would like to bring [~vicaya] 
and [~raviprak] attention to this issue.

 GraphiteSink should implement Closeable
 ---

 Key: HADOOP-10660
 URL: https://issues.apache.org/jira/browse/HADOOP-10660
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Chen He
 Attachments: HADOOP-10660.patch


 GraphiteSink wraps OutputStreamWriter around socket's output stream.
 Currently the socket is never closed.
 GraphiteSink should implement Closeable such that MetricsSystem can close the 
 socket when it is stopped.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-9704) Write metrics sink plugin for Hadoop/Graphite

2014-05-30 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-9704:
-

Attachment: HADOOP-9704.patch

 Write metrics sink plugin for Hadoop/Graphite
 -

 Key: HADOOP-9704
 URL: https://issues.apache.org/jira/browse/HADOOP-9704
 Project: Hadoop Common
  Issue Type: New Feature
Affects Versions: 2.0.3-alpha
Reporter: Chu Tong
 Attachments: 
 0001-HADOOP-9704.-Write-metrics-sink-plugin-for-Hadoop-Gr.patch, 
 HADOOP-9704.patch, HADOOP-9704.patch, HADOOP-9704.patch, Hadoop-9704.patch


 Write a metrics sink plugin for Hadoop to send metrics directly to Graphite 
 in additional to the current ganglia and file ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-9704) Write metrics sink plugin for Hadoop/Graphite

2014-05-30 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14013912#comment-14013912
 ] 

Babak Behzad commented on HADOOP-9704:
--

Thanks both Ravi and Luke. I fixed the tabs and the formatting and attached a 
new patch. Hopefully it is correct and sorry about that.

 Write metrics sink plugin for Hadoop/Graphite
 -

 Key: HADOOP-9704
 URL: https://issues.apache.org/jira/browse/HADOOP-9704
 Project: Hadoop Common
  Issue Type: New Feature
Affects Versions: 2.0.3-alpha
Reporter: Chu Tong
 Attachments: 
 0001-HADOOP-9704.-Write-metrics-sink-plugin-for-Hadoop-Gr.patch, 
 HADOOP-9704.patch, HADOOP-9704.patch, HADOOP-9704.patch, Hadoop-9704.patch


 Write a metrics sink plugin for Hadoop to send metrics directly to Graphite 
 in additional to the current ganglia and file ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-9704) Write metrics sink plugin for Hadoop/Graphite

2014-05-30 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-9704:
-

Attachment: (was: HADOOP-9704.patch)

 Write metrics sink plugin for Hadoop/Graphite
 -

 Key: HADOOP-9704
 URL: https://issues.apache.org/jira/browse/HADOOP-9704
 Project: Hadoop Common
  Issue Type: New Feature
Affects Versions: 2.0.3-alpha
Reporter: Chu Tong
 Attachments: 
 0001-HADOOP-9704.-Write-metrics-sink-plugin-for-Hadoop-Gr.patch, 
 HADOOP-9704.patch, HADOOP-9704.patch, Hadoop-9704.patch


 Write a metrics sink plugin for Hadoop to send metrics directly to Graphite 
 in additional to the current ganglia and file ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-9704) Write metrics sink plugin for Hadoop/Graphite

2014-05-30 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-9704:
-

Attachment: (was: Hadoop-9704.patch)

 Write metrics sink plugin for Hadoop/Graphite
 -

 Key: HADOOP-9704
 URL: https://issues.apache.org/jira/browse/HADOOP-9704
 Project: Hadoop Common
  Issue Type: New Feature
Affects Versions: 2.0.3-alpha
Reporter: Chu Tong
 Attachments: 
 0001-HADOOP-9704.-Write-metrics-sink-plugin-for-Hadoop-Gr.patch, 
 HADOOP-9704.patch, HADOOP-9704.patch, HADOOP-9704.patch


 Write a metrics sink plugin for Hadoop to send metrics directly to Graphite 
 in additional to the current ganglia and file ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-9704) Write metrics sink plugin for Hadoop/Graphite

2014-05-30 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-9704:
-

Attachment: HADOOP-9704.patch

 Write metrics sink plugin for Hadoop/Graphite
 -

 Key: HADOOP-9704
 URL: https://issues.apache.org/jira/browse/HADOOP-9704
 Project: Hadoop Common
  Issue Type: New Feature
Affects Versions: 2.0.3-alpha
Reporter: Chu Tong
 Attachments: 
 0001-HADOOP-9704.-Write-metrics-sink-plugin-for-Hadoop-Gr.patch, 
 HADOOP-9704.patch, HADOOP-9704.patch, HADOOP-9704.patch


 Write a metrics sink plugin for Hadoop to send metrics directly to Graphite 
 in additional to the current ganglia and file ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-9704) Write metrics sink plugin for Hadoop/Graphite

2014-05-28 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011294#comment-14011294
 ] 

Babak Behzad commented on HADOOP-9704:
--

Thanks Ravi. So, in order to enable this one has to put the following three 
lines in etc/hadoop-metrics2.properties for each of the contexts (namenode, 
datanode, resource manager, etc):
namenode.sink.graphite.server_host=localhost
namenode.sink.graphite.server_port=2003
namenode.sink.graphite.metrics_prefix=test.namenode

Looking at the logs and source code, the socket connection is only created once 
in the init() function for each of these contexts that you enable them, so at 
most 5 new connections are created. This is exactly the same as other sinks 
such as FileSink and GangliaSink and I don't think it is an issue.

Regarding a slow Graphite server, that's a good point. We will be able to test 
this soon, but again looking at the current Hadoop code, Ganglia's sink is 
using DatagramSocket instead of Socket. I am not sure if we need to do that 
later if we see problems with the current code.

 Write metrics sink plugin for Hadoop/Graphite
 -

 Key: HADOOP-9704
 URL: https://issues.apache.org/jira/browse/HADOOP-9704
 Project: Hadoop Common
  Issue Type: New Feature
Affects Versions: 2.0.3-alpha
Reporter: Chu Tong
 Attachments: 
 0001-HADOOP-9704.-Write-metrics-sink-plugin-for-Hadoop-Gr.patch, 
 HADOOP-9704.patch, HADOOP-9704.patch, Hadoop-9704.patch


 Write a metrics sink plugin for Hadoop to send metrics directly to Graphite 
 in additional to the current ganglia and file ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (HADOOP-9704) Write metrics sink plugin for Hadoop/Graphite

2014-05-27 Thread Babak Behzad (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Babak Behzad updated HADOOP-9704:
-

Attachment: Hadoop-9704.patch

 Write metrics sink plugin for Hadoop/Graphite
 -

 Key: HADOOP-9704
 URL: https://issues.apache.org/jira/browse/HADOOP-9704
 Project: Hadoop Common
  Issue Type: New Feature
Affects Versions: 2.0.3-alpha
Reporter: Chu Tong
 Attachments: 
 0001-HADOOP-9704.-Write-metrics-sink-plugin-for-Hadoop-Gr.patch, 
 HADOOP-9704.patch, HADOOP-9704.patch, Hadoop-9704.patch


 Write a metrics sink plugin for Hadoop to send metrics directly to Graphite 
 in additional to the current ganglia and file ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HADOOP-9704) Write metrics sink plugin for Hadoop/Graphite

2014-05-27 Thread Babak Behzad (JIRA)

[ 
https://issues.apache.org/jira/browse/HADOOP-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009925#comment-14009925
 ] 

Babak Behzad commented on HADOOP-9704:
--

We need this feature in our company and I got the patch and used it. There was 
a small bug which caused Graphite to ignore a lot of metrics (some of the 
metrics have space in their name and Graphite do not handle them). I fixed that 
and I also fixed all the three feedbacks that [~vicaya] mentioned in the above 
comment. Can someone please review this patch before I submit it?

 Write metrics sink plugin for Hadoop/Graphite
 -

 Key: HADOOP-9704
 URL: https://issues.apache.org/jira/browse/HADOOP-9704
 Project: Hadoop Common
  Issue Type: New Feature
Affects Versions: 2.0.3-alpha
Reporter: Chu Tong
 Attachments: 
 0001-HADOOP-9704.-Write-metrics-sink-plugin-for-Hadoop-Gr.patch, 
 HADOOP-9704.patch, HADOOP-9704.patch, Hadoop-9704.patch


 Write a metrics sink plugin for Hadoop to send metrics directly to Graphite 
 in additional to the current ganglia and file ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)