[jira] [Comment Edited] (HBASE-16463) Improve transparent table/CF encryption with Commons Crypto

2016-09-06 Thread Dapeng Sun (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-16463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466862#comment-15466862
 ] 

Dapeng Sun edited comment on HBASE-16463 at 9/6/16 8:52 AM:


Just checked the error log of {{hbaseprotoc}}, I think the error is caused by 
the linux image didn't install protobuf library, and the command {{mvn 
-DHBasePatchProcess compile -DskipTests -Pcompile-protobuf -X 
-DHBasePatchProcess}} is passed at my local env.
{noformat}
 [ERROR] Failed to execute goal 
org.apache.hadoop:hadoop-maven-plugins:2.7.1:protoc (compile-protoc) on project 
hbase-protocol: org.apache.maven.plugin.MojoExecutionException: 'protoc 
--version' did not return a version
{noformat}

Is there any comments?


was (Author: dapengsun):
Just checked the error log of {{hbaseprotoc}}, I think the error is caused by 
the linux image didn't install protobuf library, and the command {{ mvn 
-DHBasePatchProcess compile -DskipTests -Pcompile-protobuf -X 
-DHBasePatchProcess}} is passed at my local env.
{noformat}
 [ERROR] Failed to execute goal 
org.apache.hadoop:hadoop-maven-plugins:2.7.1:protoc (compile-protoc) on project 
hbase-protocol: org.apache.maven.plugin.MojoExecutionException: 'protoc 
--version' did not return a version
{noformat}

Is there any comments?

> Improve transparent table/CF encryption with Commons Crypto
> ---
>
> Key: HBASE-16463
> URL: https://issues.apache.org/jira/browse/HBASE-16463
> Project: HBase
>  Issue Type: New Feature
>  Components: encryption
>Affects Versions: 2.0.0
>Reporter: Dapeng Sun
> Attachments: HBASE-16463.001.patch, HBASE-16463.002.patch, 
> HBASE-16463.003.patch
>
>
> Apache Commons Crypto 
> (https://commons.apache.org/proper/commons-crypto/index.html) is a 
> cryptographic library optimized with AES-NI.
> HBASE-7544 introduces a framework for transparent encryption feature for 
> protecting HFile and WAL data at rest. Currently JCE cipher is used bu 
> default, the improvement will use Commons Crypto to accelerate the 
> transparent encryption of HBase. new crypto provider with Commons CRYPTO will 
> be provided for Transparent encryption.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HBASE-16463) Improve transparent table/CF encryption with Commons Crypto

2016-09-06 Thread Dapeng Sun (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-16463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466862#comment-15466862
 ] 

Dapeng Sun edited comment on HBASE-16463 at 9/6/16 8:53 AM:


Just checked the error log of {{hbaseprotoc}}, I think the error is caused by 
the linux image didn't install protobuf library, and the command {{mvn 
-DHBasePatchProcess compile -DskipTests -Pcompile-protobuf -X 
-DHBasePatchProcess}} is passed at my local env.
{noformat}
 [ERROR] Failed to execute goal 
org.apache.hadoop:hadoop-maven-plugins:2.7.1:protoc (compile-protoc) on project 
hbase-protocol: org.apache.maven.plugin.MojoExecutionException: 'protoc 
--version' did not return a version
{noformat}

Is there any comments about the latest patch?


was (Author: dapengsun):
Just checked the error log of {{hbaseprotoc}}, I think the error is caused by 
the linux image didn't install protobuf library, and the command {{mvn 
-DHBasePatchProcess compile -DskipTests -Pcompile-protobuf -X 
-DHBasePatchProcess}} is passed at my local env.
{noformat}
 [ERROR] Failed to execute goal 
org.apache.hadoop:hadoop-maven-plugins:2.7.1:protoc (compile-protoc) on project 
hbase-protocol: org.apache.maven.plugin.MojoExecutionException: 'protoc 
--version' did not return a version
{noformat}

Is there any comments?

> Improve transparent table/CF encryption with Commons Crypto
> ---
>
> Key: HBASE-16463
> URL: https://issues.apache.org/jira/browse/HBASE-16463
> Project: HBase
>  Issue Type: New Feature
>  Components: encryption
>Affects Versions: 2.0.0
>Reporter: Dapeng Sun
> Attachments: HBASE-16463.001.patch, HBASE-16463.002.patch, 
> HBASE-16463.003.patch
>
>
> Apache Commons Crypto 
> (https://commons.apache.org/proper/commons-crypto/index.html) is a 
> cryptographic library optimized with AES-NI.
> HBASE-7544 introduces a framework for transparent encryption feature for 
> protecting HFile and WAL data at rest. Currently JCE cipher is used bu 
> default, the improvement will use Commons Crypto to accelerate the 
> transparent encryption of HBase. new crypto provider with Commons CRYPTO will 
> be provided for Transparent encryption.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HBASE-16463) Improve transparent table/CF encryption with Commons Crypto

2016-08-30 Thread Dapeng Sun (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-16463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448275#comment-15448275
 ] 

Dapeng Sun edited comment on HBASE-16463 at 8/30/16 7:06 AM:
-

I just finished the test of {{HFilePerformanceEvaluation}}, here is the result 
of cost time.

||codec||cipher||SequentialWriteBenchmark(ms)||UniformRandomSmallScan||UniformRandomReadBenchmark||GaussianRandomReadBenchmark||SequentialReadBenchmark||
|none|none|960|2776|58131|65449|355|
|none|Commons|1023|4690|82826|91713|402|
|none|JCE|1106|6693|99451|110057|530|
|gz|none|1273|7126|104380|108270|349|
|gz|Commons|1457|8670|127350|137593|632|
|gz|JCE|1665|9880|132759|145476|673|


Here is the percentage increase of Commons Crypto compared with default JCE.
||codec||SequentialWriteBenchmark||UniformRandomSmallScan||UniformRandomReadBenchmark||GaussianRandomReadBenchmark||SequentialReadBenchmark||
|none|0.08113392|0.427078891|0.200721996|0.200015265|0.31840796
|gz|0.142759094|0.139561707|0.042473498|0.057292159|0.064873418


We can see Commons CRYPTO compared with JCE, it shows 4%~42% gain.


was (Author: dapengsun):
I just finished the test of {{HFilePerformanceEvaluation}}, here is the result 
of cost time.

||codec||cipher||SequentialWriteBenchmark(ms)||UniformRandomSmallScan||UniformRandomReadBenchmark||GaussianRandomReadBenchmark||SequentialReadBenchmark||
|none|none|960|2776|58131|65449|355|
|none|Commons|1023|4690|82826|91713|402|
|none|JCE|1106|6693|99451|110057|530|
|gz|none|1273|7126|104380|108270|349|
|gz|Commons|1457|8670|127350|137593|632|
|gz|JCE|1665|9880|132759|145476|673|


Here is the percentage increase table.
||codec||SequentialWriteBenchmark||UniformRandomSmallScan||UniformRandomReadBenchmark||GaussianRandomReadBenchmark||SequentialReadBenchmark||
|none|0.08113392|0.427078891|0.200721996|0.200015265|0.31840796
|gz|0.142759094|0.139561707|0.042473498|0.057292159|0.064873418


We can see Commons CRYPTO compared with JCE, it shows 4%~42% gain.

> Improve transparent table/CF encryption with Commons Crypto
> ---
>
> Key: HBASE-16463
> URL: https://issues.apache.org/jira/browse/HBASE-16463
> Project: HBase
>  Issue Type: New Feature
>  Components: encryption
>Affects Versions: 2.0.0
>Reporter: Dapeng Sun
> Attachments: HBASE-16463.001.patch, HBASE-16463.002.patch
>
>
> Apache Commons Crypto 
> (https://commons.apache.org/proper/commons-crypto/index.html) is a 
> cryptographic library optimized with AES-NI.
> HBASE-7544 introduces a framework for transparent encryption feature for 
> protecting HFile and WAL data at rest. Currently JCE cipher is used bu 
> default, the improvement will use Commons Crypto to accelerate the 
> transparent encryption of HBase. new crypto provider with Commons CRYPTO will 
> be provided for Transparent encryption.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)