Github user winningsix commented on the issue:
https://github.com/apache/spark/pull/11045
@yaooqinn yes, the validation is working on server side.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user winningsix commented on the issue:
https://github.com/apache/spark/pull/11045
@yaooqinn thank you for your interest. It's required by Hive side at
SessionState. And it's not used for privilege check. Some companies have
already used this pull request in their product
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/15172#discussion_r79746570
--- Diff: common/network-common/pom.xml ---
@@ -97,6 +97,11 @@
slf4j-log4j12
test
+
--- End diff
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/15172#discussion_r79746732
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/sasl/SaslServerBootstrap.java
---
@@ -29,7 +29,6 @@
* authentication
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r76175174
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnIOEncryptionSuite.scala ---
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r75794260
--- Diff:
core/src/main/scala/org/apache/spark/crypto/CryptoStreamUtils.scala ---
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r75631172
--- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoConf.scala ---
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-190014141
@vanzin I missed the point that non-spark configurations in the
conf/spark-defaults will be
ignored(https://github.com/apache/spark/blob/master/core/src/main/scala
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-186069349
Hi @vanzin, I have a suggestion about the usage of configuration.
Currently we convert the Spark crypto related configuration to Chimera
configuration
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r52838315
--- Diff:
core/src/test/scala/org/apache/spark/crypto/ShuffleEncryptionSuite.scala ---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/10144#issuecomment-179571823
Hi @hvanhovell @rxin , could you help me review this changes? I have
updated the original pull request since changes in #10905.
---
If your project is set up
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/11045#issuecomment-179572731
Hi @rxin, could you help review this PR? This PR is kind of POC code and
serve as illustrating how to enable Hive authorization V2 in Spark SQL side. We
can divide
GitHub user winningsix opened a pull request:
https://github.com/apache/spark/pull/11045
[SPARK-8321][SQL][WIP] Authorization Support(on all operations not only
DDL) in Spark Sql
This PR enables Hive authorization V2(SQL based authorization). It adds
authorization check after
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/10144#issuecomment-172700529
Hi @hvanhovell , I am still working on this. It would be great if you or
some one can review this PR. Thank you!
---
If your project is set up for it, you can
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-169181174
In fact, native optimization(using Openssl cipher) with AES-NI is very
necessary.
Regards the performance, I have some rough data about the performance
impacts
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-165009204
The failed reason is as follows which is not related to the patch:
> \> git fetch --tags --progress https://github.com/apache/spark.git
+refs/pull/8880/
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-164617083
@vanzin, I have updated the patch addressing your comments. The following
changes are included:
* Mock the ```SparkEnv``` instead of instantiating
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r47583388
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnShuffleEncryptionSuite.scala
---
@@ -0,0 +1,289 @@
+/*
+ * Licensed to the Apache
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-164317758
>>we have to do it in the level of Yarn module since we need to obtain the
security key from the credentials stored in user group information
>Ye
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r47466929
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnShuffleEncryptionSuite.scala
---
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-164366860
Hi @vanzin, could you help me review the updated patches? In the updated
patch, the following changes come:
1. I added test for shuffle encryption
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-163466926
>>Yes, I agree that we should encrypt the compressed stream when writing.
However, BlockStoreShuffleReader should be opposite to the write path
>
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-163075127
Hi @vanzin
Thanks for your comments. I mean to ping you and add reply some of your
previous comments but the build failed since the Chimera jar is not updated. I
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/10144#issuecomment-163087290
Hi @vanzin
This PR includes two parts:
1. Add user information to SessionState which is required to initialize the
authorizer.
2. Add parser rules
GitHub user winningsix opened a pull request:
https://github.com/apache/spark/pull/10144
[SPARK-12145][SQL] Command 'Set Role [ADMIN|NONE|ALL]' doesn't work iâ¦
â¦n SQL based authorization
"Set role [ADMIN|NONE|ALL]" is not working since it's treated as "
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-160554121
Hi @vanzin , sorry for my late response. Some failed Jenkins tasks distract
me for some time. Could you please help me review the latest update? Thank you
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-160014509
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-159804799
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-157954855
Hi @vanzin, I changed the title to the SPARK-5682 since this patch include
both Openssl and JCE codec supports by using Chimera library. PR has been
updated as well
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r44354373
--- Diff:
core/src/test/scala/org/apache/spark/crypto/JceAesCtrCryptoCodecSuite.scala ---
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-148256354
Hi @vanzin, thank you for your review and comments. I will work on the code
refactory of crypto input/output stream part. Before that, I will update the
patch
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r41608824
--- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoCodec.scala ---
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r41605310
--- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoConf.scala ---
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r41593638
--- Diff:
core/src/main/scala/org/apache/spark/crypto/CommonConfigurationKeys.scala ---
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r41594247
--- Diff:
core/src/main/scala/org/apache/spark/crypto/CommonConfigurationKeys.scala ---
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r41597185
--- Diff:
core/src/main/scala/org/apache/spark/crypto/JceAesCtrCryptoCodec.scala ---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-14639
Hi @vanzin, I have just updated the code based the latest upstream. Could
you please take a look at the updated patch? Thank you!
---
If your project is set up
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r40750017
--- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoConf.scala ---
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-144240319
Hi @vanzin, thank you for your reviews. I have updated the patch addressing
your comments and do some code clean up.
---
If your project is set up for it, you can
GitHub user winningsix opened a pull request:
https://github.com/apache/spark/pull/8880
SPARK-10771: Implement the shuffle encryption with AES-CTR crypto using JCE
key provider.
This patch is using JCE key provider and it's a subtask of SPARK-5682. We
will add openssl
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/5307#issuecomment-142516961
@vanzin I create a new PR(https://github.com/apache/spark/pull/8880)
addressing the common part and JCE key provider support. Could you help me
review it? Ticket
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/8880#issuecomment-142793076
@rxin, the design document is available in
https://issues.apache.org/jira/secure/attachment/12730704/Design%20Document%20of%20Encrypted%20Spark%20Shuffle_20150506
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/5307#issuecomment-141892023
Hi @JoshRosen and @vanzin, do you have any further comments about this PR?
The library Chimera is maintained by Intel. One way to address your comments is
to move
Github user winningsix commented on the pull request:
https://github.com/apache/spark/pull/5307#issuecomment-142151286
Thanks @vanzin for your prompt reply. In the current patch it consists of
two parts. One is the encryption framework using JCE encryption and another is
performance
44 matches
Mail list logo
