[jira] [Updated] (HADOOP-16011) OsSecureRandom very slow compared to other SecureRandom implementations
[
https://issues.apache.org/jira/browse/HADOOP-16011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HADOOP-16011:
-
Resolution: Fixed
Fix Version/s: 3.3.0
Status: Resolved (was: Patch Available)
Pushed to trunk.
Thanks [~smeng] for the patch and [~tlipcon] for reporting the issue!
> OsSecureRandom very slow compared to other SecureRandom implementations
> ---
>
> Key: HADOOP-16011
> URL: https://issues.apache.org/jira/browse/HADOOP-16011
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Todd Lipcon
>Assignee: Siyao Meng
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HADOOP-16011.001.patch, HADOOP-16011.002.patch,
> MyBenchmark.java
>
>
> In looking at performance of a workload which creates a lot of short-lived
> remote connections to a secured DN, [~philip] and I found very high system
> CPU usage. We tracked it down to reads from /dev/random, which are incurred
> by the DN using CryptoCodec.generateSecureRandom to generate a transient
> session key and IV for AES encryption.
> In the case that the OpenSSL codec is not enabled, the above code falls
> through to the JDK SecureRandom implementation, which performs reasonably.
> However, OpenSSLCodec defaults to using OsSecureRandom, which reads all
> random data from /dev/random rather than doing something more efficient like
> initializing a CSPRNG from a small seed.
> I wrote a simple JMH benchmark to compare various approaches when running
> with concurrency 10:
> testHadoop - using CryptoCodec
> testNewSecureRandom - using 'new SecureRandom()' each iteration
> testSha1PrngNew - using the SHA1PRNG explicitly, new instance each iteration
> testSha1PrngShared - using a single shared instance of SHA1PRNG
> testSha1PrngThread - using a thread-specific instance of SHA1PRNG
> {code:java}
> Benchmark Mode CntScore Error Units
> MyBenchmark.testHadoop thrpt 1293.000 ops/s
> [with libhadoop.so]
> MyBenchmark.testHadoop thrpt461515.697 ops/s
> [without libhadoop.so]
> MyBenchmark.testNewSecureRandom thrpt 43413.640 ops/s
> MyBenchmark.testSha1PrngNew thrpt395515.000 ops/s
> MyBenchmark.testSha1PrngShared thrpt164488.713 ops/s
> MyBenchmark.testSha1PrngThread thrpt 4295123.210 ops/s
> {code}
> In other words, the presence of the OpenSSL acceleration slows down this code
> path by 356x. And, compared to the optimal (thread-local Sha1Prng) it's 3321x
> slower.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16011) OsSecureRandom very slow compared to other SecureRandom implementations
[
https://issues.apache.org/jira/browse/HADOOP-16011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-16011:
Release Note:
The default RNG is now OpensslSecureRandom instead of OsSecureRandom.The
high-performance hardware random number generator (RDRAND instruction) will be
used if available. If not, it will fall back to OpenSSL secure random generator.
If you insist on using OsSecureRandom, set hadoop.security.secure.random.impl
in core-site.xml to org.apache.hadoop.crypto.random.OsSecureRandom.
was:The default RNG is now OpensslSecureRandom instead of OsSecureRandom: if
available, the high-performance hardware random number generator (RDRAND
instruction) will be used will be used; if not, it will fall back to OpenSSL
secure random generator. If you insist on using OsSecureRandom set
hadoop.security.secure.random.impl in core-site.xml to
org.apache.hadoop.crypto.random.OsSecureRandom.
> OsSecureRandom very slow compared to other SecureRandom implementations
> ---
>
> Key: HADOOP-16011
> URL: https://issues.apache.org/jira/browse/HADOOP-16011
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Todd Lipcon
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HADOOP-16011.001.patch, HADOOP-16011.002.patch,
> MyBenchmark.java
>
>
> In looking at performance of a workload which creates a lot of short-lived
> remote connections to a secured DN, [~philip] and I found very high system
> CPU usage. We tracked it down to reads from /dev/random, which are incurred
> by the DN using CryptoCodec.generateSecureRandom to generate a transient
> session key and IV for AES encryption.
> In the case that the OpenSSL codec is not enabled, the above code falls
> through to the JDK SecureRandom implementation, which performs reasonably.
> However, OpenSSLCodec defaults to using OsSecureRandom, which reads all
> random data from /dev/random rather than doing something more efficient like
> initializing a CSPRNG from a small seed.
> I wrote a simple JMH benchmark to compare various approaches when running
> with concurrency 10:
> testHadoop - using CryptoCodec
> testNewSecureRandom - using 'new SecureRandom()' each iteration
> testSha1PrngNew - using the SHA1PRNG explicitly, new instance each iteration
> testSha1PrngShared - using a single shared instance of SHA1PRNG
> testSha1PrngThread - using a thread-specific instance of SHA1PRNG
> {code:java}
> Benchmark Mode CntScore Error Units
> MyBenchmark.testHadoop thrpt 1293.000 ops/s
> [with libhadoop.so]
> MyBenchmark.testHadoop thrpt461515.697 ops/s
> [without libhadoop.so]
> MyBenchmark.testNewSecureRandom thrpt 43413.640 ops/s
> MyBenchmark.testSha1PrngNew thrpt395515.000 ops/s
> MyBenchmark.testSha1PrngShared thrpt164488.713 ops/s
> MyBenchmark.testSha1PrngThread thrpt 4295123.210 ops/s
> {code}
> In other words, the presence of the OpenSSL acceleration slows down this code
> path by 356x. And, compared to the optimal (thread-local Sha1Prng) it's 3321x
> slower.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16011) OsSecureRandom very slow compared to other SecureRandom implementations
[
https://issues.apache.org/jira/browse/HADOOP-16011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-16011:
Attachment: HADOOP-16011.002.patch
Status: Patch Available (was: In Progress)
Submitted patch rev 002. Added default RNG impl in core-site.xml.
> OsSecureRandom very slow compared to other SecureRandom implementations
> ---
>
> Key: HADOOP-16011
> URL: https://issues.apache.org/jira/browse/HADOOP-16011
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Todd Lipcon
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HADOOP-16011.001.patch, HADOOP-16011.002.patch,
> MyBenchmark.java
>
>
> In looking at performance of a workload which creates a lot of short-lived
> remote connections to a secured DN, [~philip] and I found very high system
> CPU usage. We tracked it down to reads from /dev/random, which are incurred
> by the DN using CryptoCodec.generateSecureRandom to generate a transient
> session key and IV for AES encryption.
> In the case that the OpenSSL codec is not enabled, the above code falls
> through to the JDK SecureRandom implementation, which performs reasonably.
> However, OpenSSLCodec defaults to using OsSecureRandom, which reads all
> random data from /dev/random rather than doing something more efficient like
> initializing a CSPRNG from a small seed.
> I wrote a simple JMH benchmark to compare various approaches when running
> with concurrency 10:
> testHadoop - using CryptoCodec
> testNewSecureRandom - using 'new SecureRandom()' each iteration
> testSha1PrngNew - using the SHA1PRNG explicitly, new instance each iteration
> testSha1PrngShared - using a single shared instance of SHA1PRNG
> testSha1PrngThread - using a thread-specific instance of SHA1PRNG
> {code:java}
> Benchmark Mode CntScore Error Units
> MyBenchmark.testHadoop thrpt 1293.000 ops/s
> [with libhadoop.so]
> MyBenchmark.testHadoop thrpt461515.697 ops/s
> [without libhadoop.so]
> MyBenchmark.testNewSecureRandom thrpt 43413.640 ops/s
> MyBenchmark.testSha1PrngNew thrpt395515.000 ops/s
> MyBenchmark.testSha1PrngShared thrpt164488.713 ops/s
> MyBenchmark.testSha1PrngThread thrpt 4295123.210 ops/s
> {code}
> In other words, the presence of the OpenSSL acceleration slows down this code
> path by 356x. And, compared to the optimal (thread-local Sha1Prng) it's 3321x
> slower.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16011) OsSecureRandom very slow compared to other SecureRandom implementations
[
https://issues.apache.org/jira/browse/HADOOP-16011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-16011:
Status: In Progress (was: Patch Available)
> OsSecureRandom very slow compared to other SecureRandom implementations
> ---
>
> Key: HADOOP-16011
> URL: https://issues.apache.org/jira/browse/HADOOP-16011
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Todd Lipcon
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HADOOP-16011.001.patch, MyBenchmark.java
>
>
> In looking at performance of a workload which creates a lot of short-lived
> remote connections to a secured DN, [~philip] and I found very high system
> CPU usage. We tracked it down to reads from /dev/random, which are incurred
> by the DN using CryptoCodec.generateSecureRandom to generate a transient
> session key and IV for AES encryption.
> In the case that the OpenSSL codec is not enabled, the above code falls
> through to the JDK SecureRandom implementation, which performs reasonably.
> However, OpenSSLCodec defaults to using OsSecureRandom, which reads all
> random data from /dev/random rather than doing something more efficient like
> initializing a CSPRNG from a small seed.
> I wrote a simple JMH benchmark to compare various approaches when running
> with concurrency 10:
> testHadoop - using CryptoCodec
> testNewSecureRandom - using 'new SecureRandom()' each iteration
> testSha1PrngNew - using the SHA1PRNG explicitly, new instance each iteration
> testSha1PrngShared - using a single shared instance of SHA1PRNG
> testSha1PrngThread - using a thread-specific instance of SHA1PRNG
> {code:java}
> Benchmark Mode CntScore Error Units
> MyBenchmark.testHadoop thrpt 1293.000 ops/s
> [with libhadoop.so]
> MyBenchmark.testHadoop thrpt461515.697 ops/s
> [without libhadoop.so]
> MyBenchmark.testNewSecureRandom thrpt 43413.640 ops/s
> MyBenchmark.testSha1PrngNew thrpt395515.000 ops/s
> MyBenchmark.testSha1PrngShared thrpt164488.713 ops/s
> MyBenchmark.testSha1PrngThread thrpt 4295123.210 ops/s
> {code}
> In other words, the presence of the OpenSSL acceleration slows down this code
> path by 356x. And, compared to the optimal (thread-local Sha1Prng) it's 3321x
> slower.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16011) OsSecureRandom very slow compared to other SecureRandom implementations
[
https://issues.apache.org/jira/browse/HADOOP-16011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-16011:
Release Note: The default RNG is now OpensslSecureRandom instead of
OsSecureRandom: if available, the high-performance hardware random number
generator (RDRAND instruction) will be used will be used; if not, it will fall
back to OpenSSL secure random generator. If you insist on using OsSecureRandom
set hadoop.security.secure.random.impl in core-site.xml to
org.apache.hadoop.crypto.random.OsSecureRandom. (was: The default RNG is now
OpensslSecureRandom instead of OsSecureRandom: if available, the
high-performance hardware random number generator (RDRAND instruction) will be
used will be used; if not, it will fall back to OpenSSL secure random
generator.)
> OsSecureRandom very slow compared to other SecureRandom implementations
> ---
>
> Key: HADOOP-16011
> URL: https://issues.apache.org/jira/browse/HADOOP-16011
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Todd Lipcon
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HADOOP-16011.001.patch, MyBenchmark.java
>
>
> In looking at performance of a workload which creates a lot of short-lived
> remote connections to a secured DN, [~philip] and I found very high system
> CPU usage. We tracked it down to reads from /dev/random, which are incurred
> by the DN using CryptoCodec.generateSecureRandom to generate a transient
> session key and IV for AES encryption.
> In the case that the OpenSSL codec is not enabled, the above code falls
> through to the JDK SecureRandom implementation, which performs reasonably.
> However, OpenSSLCodec defaults to using OsSecureRandom, which reads all
> random data from /dev/random rather than doing something more efficient like
> initializing a CSPRNG from a small seed.
> I wrote a simple JMH benchmark to compare various approaches when running
> with concurrency 10:
> testHadoop - using CryptoCodec
> testNewSecureRandom - using 'new SecureRandom()' each iteration
> testSha1PrngNew - using the SHA1PRNG explicitly, new instance each iteration
> testSha1PrngShared - using a single shared instance of SHA1PRNG
> testSha1PrngThread - using a thread-specific instance of SHA1PRNG
> {code:java}
> Benchmark Mode CntScore Error Units
> MyBenchmark.testHadoop thrpt 1293.000 ops/s
> [with libhadoop.so]
> MyBenchmark.testHadoop thrpt461515.697 ops/s
> [without libhadoop.so]
> MyBenchmark.testNewSecureRandom thrpt 43413.640 ops/s
> MyBenchmark.testSha1PrngNew thrpt395515.000 ops/s
> MyBenchmark.testSha1PrngShared thrpt164488.713 ops/s
> MyBenchmark.testSha1PrngThread thrpt 4295123.210 ops/s
> {code}
> In other words, the presence of the OpenSSL acceleration slows down this code
> path by 356x. And, compared to the optimal (thread-local Sha1Prng) it's 3321x
> slower.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16011) OsSecureRandom very slow compared to other SecureRandom implementations
[
https://issues.apache.org/jira/browse/HADOOP-16011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-16011:
Release Note: The default RNG is now OpensslSecureRandom instead of
OsSecureRandom: if available, the high-performance hardware random number
generator (RDRAND instruction) will be used will be used; if not, it will fall
back to OpenSSL secure random generator.
> OsSecureRandom very slow compared to other SecureRandom implementations
> ---
>
> Key: HADOOP-16011
> URL: https://issues.apache.org/jira/browse/HADOOP-16011
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Todd Lipcon
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HADOOP-16011.001.patch, MyBenchmark.java
>
>
> In looking at performance of a workload which creates a lot of short-lived
> remote connections to a secured DN, [~philip] and I found very high system
> CPU usage. We tracked it down to reads from /dev/random, which are incurred
> by the DN using CryptoCodec.generateSecureRandom to generate a transient
> session key and IV for AES encryption.
> In the case that the OpenSSL codec is not enabled, the above code falls
> through to the JDK SecureRandom implementation, which performs reasonably.
> However, OpenSSLCodec defaults to using OsSecureRandom, which reads all
> random data from /dev/random rather than doing something more efficient like
> initializing a CSPRNG from a small seed.
> I wrote a simple JMH benchmark to compare various approaches when running
> with concurrency 10:
> testHadoop - using CryptoCodec
> testNewSecureRandom - using 'new SecureRandom()' each iteration
> testSha1PrngNew - using the SHA1PRNG explicitly, new instance each iteration
> testSha1PrngShared - using a single shared instance of SHA1PRNG
> testSha1PrngThread - using a thread-specific instance of SHA1PRNG
> {code:java}
> Benchmark Mode CntScore Error Units
> MyBenchmark.testHadoop thrpt 1293.000 ops/s
> [with libhadoop.so]
> MyBenchmark.testHadoop thrpt461515.697 ops/s
> [without libhadoop.so]
> MyBenchmark.testNewSecureRandom thrpt 43413.640 ops/s
> MyBenchmark.testSha1PrngNew thrpt395515.000 ops/s
> MyBenchmark.testSha1PrngShared thrpt164488.713 ops/s
> MyBenchmark.testSha1PrngThread thrpt 4295123.210 ops/s
> {code}
> In other words, the presence of the OpenSSL acceleration slows down this code
> path by 356x. And, compared to the optimal (thread-local Sha1Prng) it's 3321x
> slower.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16011) OsSecureRandom very slow compared to other SecureRandom implementations
[
https://issues.apache.org/jira/browse/HADOOP-16011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-16011:
Assignee: Siyao Meng
Attachment: HADOOP-16011.001.patch
Status: Patch Available (was: Open)
Uploaded patch rev 001: Change default RNG in OpensslAesCtrCryptoCodec from
OsSecureRandom to OpensslSecureRandom.
> OsSecureRandom very slow compared to other SecureRandom implementations
> ---
>
> Key: HADOOP-16011
> URL: https://issues.apache.org/jira/browse/HADOOP-16011
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Todd Lipcon
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HADOOP-16011.001.patch, MyBenchmark.java
>
>
> In looking at performance of a workload which creates a lot of short-lived
> remote connections to a secured DN, [~philip] and I found very high system
> CPU usage. We tracked it down to reads from /dev/random, which are incurred
> by the DN using CryptoCodec.generateSecureRandom to generate a transient
> session key and IV for AES encryption.
> In the case that the OpenSSL codec is not enabled, the above code falls
> through to the JDK SecureRandom implementation, which performs reasonably.
> However, OpenSSLCodec defaults to using OsSecureRandom, which reads all
> random data from /dev/random rather than doing something more efficient like
> initializing a CSPRNG from a small seed.
> I wrote a simple JMH benchmark to compare various approaches when running
> with concurrency 10:
> testHadoop - using CryptoCodec
> testNewSecureRandom - using 'new SecureRandom()' each iteration
> testSha1PrngNew - using the SHA1PRNG explicitly, new instance each iteration
> testSha1PrngShared - using a single shared instance of SHA1PRNG
> testSha1PrngThread - using a thread-specific instance of SHA1PRNG
> {code:java}
> Benchmark Mode CntScore Error Units
> MyBenchmark.testHadoop thrpt 1293.000 ops/s
> [with libhadoop.so]
> MyBenchmark.testHadoop thrpt461515.697 ops/s
> [without libhadoop.so]
> MyBenchmark.testNewSecureRandom thrpt 43413.640 ops/s
> MyBenchmark.testSha1PrngNew thrpt395515.000 ops/s
> MyBenchmark.testSha1PrngShared thrpt164488.713 ops/s
> MyBenchmark.testSha1PrngThread thrpt 4295123.210 ops/s
> {code}
> In other words, the presence of the OpenSSL acceleration slows down this code
> path by 356x. And, compared to the optimal (thread-local Sha1Prng) it's 3321x
> slower.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16011) OsSecureRandom very slow compared to other SecureRandom implementations
[
https://issues.apache.org/jira/browse/HADOOP-16011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HADOOP-16011:
-
Attachment: MyBenchmark.java
> OsSecureRandom very slow compared to other SecureRandom implementations
> ---
>
> Key: HADOOP-16011
> URL: https://issues.apache.org/jira/browse/HADOOP-16011
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Todd Lipcon
>Priority: Major
> Attachments: MyBenchmark.java
>
>
> In looking at performance of a workload which creates a lot of short-lived
> remote connections to a secured DN, [~philip] and I found very high system
> CPU usage. We tracked it down to reads from /dev/random, which are incurred
> by the DN using CryptoCodec.generateSecureRandom to generate a transient
> session key and IV for AES encryption.
> In the case that the OpenSSL codec is not enabled, the above code falls
> through to the JDK SecureRandom implementation, which performs reasonably.
> However, OpenSSLCodec defaults to using OsSecureRandom, which reads all
> random data from /dev/random rather than doing something more efficient like
> initializing a CSPRNG from a small seed.
> I wrote a simple JMH benchmark to compare various approaches when running
> with concurrency 10:
> testHadoop - using CryptoCodec
> testNewSecureRandom - using 'new SecureRandom()' each iteration
> testSha1PrngNew - using the SHA1PRNG explicitly, new instance each iteration
> testSha1PrngShared - using a single shared instance of SHA1PRNG
> testSha1PrngThread - using a thread-specific instance of SHA1PRNG
> {code:java}
> Benchmark Mode CntScore Error Units
> MyBenchmark.testHadoop thrpt 1293.000 ops/s
> [with libhadoop.so]
> MyBenchmark.testHadoop thrpt461515.697 ops/s
> [without libhadoop.so]
> MyBenchmark.testNewSecureRandom thrpt 43413.640 ops/s
> MyBenchmark.testSha1PrngNew thrpt395515.000 ops/s
> MyBenchmark.testSha1PrngShared thrpt164488.713 ops/s
> MyBenchmark.testSha1PrngThread thrpt 4295123.210 ops/s
> {code}
> In other words, the presence of the OpenSSL acceleration slows down this code
> path by 356x. And, compared to the optimal (thread-local Sha1Prng) it's 3321x
> slower.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
