[jira] [Updated] (MAPREDUCE-6931) Remove TestDFSIO "Total Throughput" calculation

2017-09-01 Thread Konstantin Shvachko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Shvachko updated MAPREDUCE-6931:
---
Priority: Critical  (was: Trivial)

> Remove TestDFSIO "Total Throughput" calculation
> ---
>
> Key: MAPREDUCE-6931
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6931
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks, test
>Affects Versions: 2.8.0
>Reporter: Dennis Huo
>Assignee: Dennis Huo
>Priority: Critical
> Fix For: 2.9.0, 3.0.0-beta1, 2.7.5, 2.8.3
>
> Attachments: MAPREDUCE-6931-001.patch
>
>
> The new "Total Throughput" line added in 
> https://issues.apache.org/jira/browse/HDFS-9153 is currently calculated as 
> {{toMB(size) / ((float)execTime)}} and claims to be in units of "MB/s", but 
> {{execTime}} is in milliseconds; thus, the reported number is 1/1000x the 
> actual value:
> {code:java}
> String resultLines[] = {
> "- TestDFSIO - : " + testType,
> "Date & time: " + new Date(System.currentTimeMillis()),
> "Number of files: " + tasks,
> " Total MBytes processed: " + df.format(toMB(size)),
> "  Throughput mb/sec: " + df.format(size * 1000.0 / (time * 
> MEGA)),
> "Total Throughput mb/sec: " + df.format(toMB(size) / 
> ((float)execTime)),
> " Average IO rate mb/sec: " + df.format(med),
> "  IO rate std deviation: " + df.format(stdDev),
> " Test exec time sec: " + df.format((float)execTime / 1000),
> "" };
> {code}
> The different calculated fields can also use toMB and a shared 
> milliseconds-to-seconds conversion to make it easier to keep units consistent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6931) Remove TestDFSIO "Total Throughput" calculation

2017-08-31 Thread Junping Du (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Junping Du updated MAPREDUCE-6931:
--
Fix Version/s: (was: 2.8.2)
   2.8.3

> Remove TestDFSIO "Total Throughput" calculation
> ---
>
> Key: MAPREDUCE-6931
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6931
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks, test
>Affects Versions: 2.8.0
>Reporter: Dennis Huo
>Assignee: Dennis Huo
>Priority: Trivial
> Fix For: 2.9.0, 3.0.0-beta1, 2.7.5, 2.8.3
>
> Attachments: MAPREDUCE-6931-001.patch
>
>
> The new "Total Throughput" line added in 
> https://issues.apache.org/jira/browse/HDFS-9153 is currently calculated as 
> {{toMB(size) / ((float)execTime)}} and claims to be in units of "MB/s", but 
> {{execTime}} is in milliseconds; thus, the reported number is 1/1000x the 
> actual value:
> {code:java}
> String resultLines[] = {
> "- TestDFSIO - : " + testType,
> "Date & time: " + new Date(System.currentTimeMillis()),
> "Number of files: " + tasks,
> " Total MBytes processed: " + df.format(toMB(size)),
> "  Throughput mb/sec: " + df.format(size * 1000.0 / (time * 
> MEGA)),
> "Total Throughput mb/sec: " + df.format(toMB(size) / 
> ((float)execTime)),
> " Average IO rate mb/sec: " + df.format(med),
> "  IO rate std deviation: " + df.format(stdDev),
> " Test exec time sec: " + df.format((float)execTime / 1000),
> "" };
> {code}
> The different calculated fields can also use toMB and a shared 
> milliseconds-to-seconds conversion to make it easier to keep units consistent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6931) Remove TestDFSIO "Total Throughput" calculation

2017-08-30 Thread Konstantin Shvachko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Shvachko updated MAPREDUCE-6931:
---
   Resolution: Fixed
 Assignee: Dennis Huo
 Hadoop Flags: Reviewed
Fix Version/s: 2.7.5
   2.8.2
   3.0.0-beta1
   2.9.0
   Status: Resolved  (was: Patch Available)

I just committed this to trunk, branch-2, branch-2.8, and branch-2.7.
Thank you [~dennishuo].

> Remove TestDFSIO "Total Throughput" calculation
> ---
>
> Key: MAPREDUCE-6931
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6931
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks, test
>Affects Versions: 2.8.0
>Reporter: Dennis Huo
>Assignee: Dennis Huo
>Priority: Trivial
> Fix For: 2.9.0, 3.0.0-beta1, 2.8.2, 2.7.5
>
> Attachments: MAPREDUCE-6931-001.patch
>
>
> The new "Total Throughput" line added in 
> https://issues.apache.org/jira/browse/HDFS-9153 is currently calculated as 
> {{toMB(size) / ((float)execTime)}} and claims to be in units of "MB/s", but 
> {{execTime}} is in milliseconds; thus, the reported number is 1/1000x the 
> actual value:
> {code:java}
> String resultLines[] = {
> "- TestDFSIO - : " + testType,
> "Date & time: " + new Date(System.currentTimeMillis()),
> "Number of files: " + tasks,
> " Total MBytes processed: " + df.format(toMB(size)),
> "  Throughput mb/sec: " + df.format(size * 1000.0 / (time * 
> MEGA)),
> "Total Throughput mb/sec: " + df.format(toMB(size) / 
> ((float)execTime)),
> " Average IO rate mb/sec: " + df.format(med),
> "  IO rate std deviation: " + df.format(stdDev),
> " Test exec time sec: " + df.format((float)execTime / 1000),
> "" };
> {code}
> The different calculated fields can also use toMB and a shared 
> milliseconds-to-seconds conversion to make it easier to keep units consistent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6931) Remove TestDFSIO "Total Throughput" calculation

2017-08-25 Thread Konstantin Shvachko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Shvachko updated MAPREDUCE-6931:
---
Attachment: MAPREDUCE-6931-001.patch

Attaching patch to trigger Jenkins

> Remove TestDFSIO "Total Throughput" calculation
> ---
>
> Key: MAPREDUCE-6931
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6931
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks, test
>Affects Versions: 2.8.0
>Reporter: Dennis Huo
>Priority: Trivial
> Attachments: MAPREDUCE-6931-001.patch
>
>
> The new "Total Throughput" line added in 
> https://issues.apache.org/jira/browse/HDFS-9153 is currently calculated as 
> {{toMB(size) / ((float)execTime)}} and claims to be in units of "MB/s", but 
> {{execTime}} is in milliseconds; thus, the reported number is 1/1000x the 
> actual value:
> {code:java}
> String resultLines[] = {
> "- TestDFSIO - : " + testType,
> "Date & time: " + new Date(System.currentTimeMillis()),
> "Number of files: " + tasks,
> " Total MBytes processed: " + df.format(toMB(size)),
> "  Throughput mb/sec: " + df.format(size * 1000.0 / (time * 
> MEGA)),
> "Total Throughput mb/sec: " + df.format(toMB(size) / 
> ((float)execTime)),
> " Average IO rate mb/sec: " + df.format(med),
> "  IO rate std deviation: " + df.format(stdDev),
> " Test exec time sec: " + df.format((float)execTime / 1000),
> "" };
> {code}
> The different calculated fields can also use toMB and a shared 
> milliseconds-to-seconds conversion to make it easier to keep units consistent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6931) Remove TestDFSIO "Total Throughput" calculation

2017-08-25 Thread Konstantin Shvachko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Shvachko updated MAPREDUCE-6931:
---
Status: Patch Available  (was: Open)

> Remove TestDFSIO "Total Throughput" calculation
> ---
>
> Key: MAPREDUCE-6931
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6931
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks, test
>Affects Versions: 2.8.0
>Reporter: Dennis Huo
>Priority: Trivial
> Attachments: MAPREDUCE-6931-001.patch
>
>
> The new "Total Throughput" line added in 
> https://issues.apache.org/jira/browse/HDFS-9153 is currently calculated as 
> {{toMB(size) / ((float)execTime)}} and claims to be in units of "MB/s", but 
> {{execTime}} is in milliseconds; thus, the reported number is 1/1000x the 
> actual value:
> {code:java}
> String resultLines[] = {
> "- TestDFSIO - : " + testType,
> "Date & time: " + new Date(System.currentTimeMillis()),
> "Number of files: " + tasks,
> " Total MBytes processed: " + df.format(toMB(size)),
> "  Throughput mb/sec: " + df.format(size * 1000.0 / (time * 
> MEGA)),
> "Total Throughput mb/sec: " + df.format(toMB(size) / 
> ((float)execTime)),
> " Average IO rate mb/sec: " + df.format(med),
> "  IO rate std deviation: " + df.format(stdDev),
> " Test exec time sec: " + df.format((float)execTime / 1000),
> "" };
> {code}
> The different calculated fields can also use toMB and a shared 
> milliseconds-to-seconds conversion to make it easier to keep units consistent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Updated] (MAPREDUCE-6931) Remove TestDFSIO "Total Throughput" calculation

2017-08-12 Thread Dennis Huo (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dennis Huo updated MAPREDUCE-6931:
--
Summary: Remove TestDFSIO "Total Throughput" calculation  (was: Fix 
TestDFSIO "Total Throughput" calculation)

> Remove TestDFSIO "Total Throughput" calculation
> ---
>
> Key: MAPREDUCE-6931
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6931
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks, test
>Affects Versions: 2.8.0
>Reporter: Dennis Huo
>Priority: Trivial
>
> The new "Total Throughput" line added in 
> https://issues.apache.org/jira/browse/HDFS-9153 is currently calculated as 
> {{toMB(size) / ((float)execTime)}} and claims to be in units of "MB/s", but 
> {{execTime}} is in milliseconds; thus, the reported number is 1/1000x the 
> actual value:
> {code:java}
> String resultLines[] = {
> "- TestDFSIO - : " + testType,
> "Date & time: " + new Date(System.currentTimeMillis()),
> "Number of files: " + tasks,
> " Total MBytes processed: " + df.format(toMB(size)),
> "  Throughput mb/sec: " + df.format(size * 1000.0 / (time * 
> MEGA)),
> "Total Throughput mb/sec: " + df.format(toMB(size) / 
> ((float)execTime)),
> " Average IO rate mb/sec: " + df.format(med),
> "  IO rate std deviation: " + df.format(stdDev),
> " Test exec time sec: " + df.format((float)execTime / 1000),
> "" };
> {code}
> The different calculated fields can also use toMB and a shared 
> milliseconds-to-seconds conversion to make it easier to keep units consistent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org