Re: [PR] [RANGER-4307] Allow users customize the time interval [ranger]
zhutong6688 commented on PR #269: URL: https://github.com/apache/ranger/pull/269#issuecomment-2121831198 @zachzzq 不过我们在生产上已经用了好久了，对生产的促进也挺大的。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [RANGER-4307] Allow users customize the time interval [ranger]
zhutong6688 commented on PR #269: URL: https://github.com/apache/ranger/pull/269#issuecomment-2121829394 @zachzzq 你好，你是apache ranger的负责人吗？这个提交有一定的时间了，没有看到项目负责人的回复。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-XXXX: Fix problem with ElasticSearch as Audit database [ranger]
FerArribas14 commented on PR #314: URL: https://github.com/apache/ranger/pull/314#issuecomment-2110298498 Thank you @ognjen-it. I had a problem with the configuration of a plugin. Now it works correctly :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-XXXX: Fix problem with ElasticSearch as Audit database [ranger]
ognjen-it commented on PR #314: URL: https://github.com/apache/ranger/pull/314#issuecomment-2109464015 Hi @FerArribas14, The version of ElasticSearch is 7.10.2 (latest with Apache 2.0 license). Everything looks good on your screenshots, ranger-admin successfully created the index and now some plugins need to start writing data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-XXXX: Fix problem with ElasticSearch as Audit database [ranger]
FerArribas14 commented on PR #314: URL: https://github.com/apache/ranger/pull/314#issuecomment-2109057763 Hi, what version of elasticsearch are you testing on localhost:9200? I can't get it to write audit events to elasticsearch. There are no errors in the ranger admin logs and the index has been created correctly. I have everything deployed in docker. I have created an elasticsearch service in docker compose. Thanks, -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-XXXX: Fix problem with ElasticSearch as Audit database [ranger]
ognjen-it opened a new pull request, #314:
URL: https://github.com/apache/ranger/pull/314
## What changes were proposed in this pull request?
I tried to create a user on Apache Jira but I've not received confirmation
yet. Until that, I'm opening this pull request for all whos seeing problems
with audit. I'm not sure if I need to do the same on the master branch?
1. Lucene version - fixed problem with writing data to ElasticSearch
**Error**: java.lang.NoSuchFieldError: LUCENE_8_5_1
I tried to change minor version one by one, but only latest version fit for
me.
Changes:
- agents-audit/pom.xml: 311
- pom.xml: 241
2. Elastic search protocol - fixed problem with changing protocol
Even though I changed ranger.audit.elasticsearch.protocol from http to
https, audit plugin still using http protocol.
Changes:
- security-admin/scripts/ranger-admin-site-template.xml: 167-170
- security-admin/scripts/setup.sh: 79, 794-797
- security-admin/scripts/upgrade_admin.py: 116
- security-admin/src/main/resources/conf.dist/ranger-admin-site.xml: 53-57
-
security-admin/src/test/java/org/apache/ranger/elasticsearch/ElasticSearchAccessAuditsServiceTest.java:
56
3. Audit plugin - cannot write audit to ES
**Error**: bootstrap method initialization exception
After changing the version of httpcomponents I started seeing audit
Changes:
- pom.xml: 137, 138, 140
4. Ranger admin console - Audit show 1-1-1970
**Erro**: Error converting value to date. Value = 2024-05-13T13:08:47.905Z
Even though evtTime was ok in ElasticSearch, ranger couldn't show it on GUI.
Changes:
-
security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchAccessAuditsService.java:
260
-
security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java:
239
## How was this patch tested?
I tested it manually.
ElasticSearch:
https://github.com/apache/ranger/assets/44586606/0ca72983-4cea-4484-9cf4-845ee5bd0a91;>
Ranger:
https://github.com/apache/ranger/assets/44586606/d52071d7-fe04-4199-a45b-3f7d1ec55974;>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4761: Add License [ranger]
mneethiraj merged PR #313: URL: https://github.com/apache/ranger/pull/313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4761: Add License [ranger]
fateh288 opened a new pull request, #313: URL: https://github.com/apache/ranger/pull/313 ## What changes were proposed in this pull request? Add license ## How was this patch tested? mvn clean install -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4783: remove duplicates for users/groups/roles during policy validation [ranger]
mneethiraj merged PR #311: URL: https://github.com/apache/ranger/pull/311 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
kumaab merged PR #307: URL: https://github.com/apache/ranger/pull/307 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4783: remove duplicates for users/groups/roles during policy validation [ranger]
fateh288 commented on code in PR #311:
URL: https://github.com/apache/ranger/pull/311#discussion_r1591489218
##
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerPolicyValidator.java:
##
@@ -958,6 +958,22 @@ private String getDuplicate(List values) {
}
return duplicate;
}
+ private void removeDuplicates(List values){
+ if (values==null || values.isEmpty()){
+ return;
+ }
+ Iterator itr = values.iterator();
+ HashSet uniqueElements = new HashSet<>();
Review Comment:
Fixed in latest patch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-3409: remove org.codehaus.jackson ver 1.9 / add com.fasterxml.jackson ver 2.17 [ranger]
nblagodarnyi commented on PR #312: URL: https://github.com/apache/ranger/pull/312#issuecomment-2095991206 Hello Sercan! Thanks for your reply. I didn't manage to reproduce your issue with UI. I used this command to rebuild ranger from my branch and everything works fine in Docker. `export RANGER_REBUILD=1 && export DOCKER_MAVEN_BUILD=0 && ./ranger_in_docker up` https://github.com/apache/ranger/assets/57439616/1766488e-a74e-47aa-8a9f-25bef67aac74;> https://github.com/apache/ranger/assets/57439616/46cf3436-7ce7-48bc-9291-33ff31edd188;> https://github.com/apache/ranger/assets/57439616/f452c3aa-c895-468a-b094-1fddae7042ec;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [RANGER-4307] Allow users customize the time interval [ranger]
zachzzq commented on PR #269: URL: https://github.com/apache/ranger/pull/269#issuecomment-2094310693 > This PR can greatly improve the production efficiency of enterprises and save a lot of costs for them. 你好 我看你提交修改的代码中引入了新的配置文件属性，但配置文件中并没有这个属性，pr中也找不到。 是我找的有问题吗？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4783: remove duplicates for users/groups/roles during policy validation [ranger]
mneethiraj commented on code in PR #311:
URL: https://github.com/apache/ranger/pull/311#discussion_r1589733127
##
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerPolicyValidator.java:
##
@@ -958,6 +958,22 @@ private String getDuplicate(List values) {
}
return duplicate;
}
+ private void removeDuplicates(List values){
+ if (values==null || values.isEmpty()){
+ return;
+ }
+ Iterator itr = values.iterator();
+ HashSet uniqueElements = new HashSet<>();
Review Comment:
Consider this version as well:
```
Set uniqueElements = new HashSet<>();
values.removeIf(e -> !uniqueElements.add(e));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4783: remove duplicates for users/groups/roles during policy validation [ranger]
mneethiraj commented on code in PR #311:
URL: https://github.com/apache/ranger/pull/311#discussion_r1589731450
##
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerPolicyValidator.java:
##
@@ -958,6 +958,22 @@ private String getDuplicate(List values) {
}
return duplicate;
}
+ private void removeDuplicates(List values){
+ if (values==null || values.isEmpty()){
+ return;
+ }
+ Iterator itr = values.iterator();
+ HashSet uniqueElements = new HashSet<>();
Review Comment:
Consider replacing #965 to #975 with the following:
```
Set uniqueElements = new HashSet<>();
for (Iterator iter = values.iterator(); iter.hasNext(); ) {
if (!uniqueElements.add(iter.next())) {
iter.remove();
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-3409: remove org.codehaus.jackson ver 1.9 / add com.fasterxml.jackson ver 2.17 [ranger]
sercanCyberVision commented on PR #312: URL: https://github.com/apache/ranger/pull/312#issuecomment-2093374126 I have build your branch with Docker option and checked the UI, I see below error in multiple endpoints:  For example: http://:/index.html#/users/usertab http://:/index.html#/reports/audit/admin I have seen jackson related serialization issues before which affect UI, you can see the details here: https://issues.apache.org/jira/browse/RANGER-4225 https://github.com/apache/ranger/pull/252 Could you please check the UI in your end just to make sure? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-3409: remove org.codehaus.jackson ver 1.9 / add com.fasterxml.jackson ver 2.17 [ranger]
nblagodarnyi opened a new pull request, #312: URL: https://github.com/apache/ranger/pull/312 ## What changes were proposed in this pull request? According to https://issues.apache.org/jira/browse/RANGER-3409 all appearances of old org.codehaus.jackson dependencies (ver.1.9.13) were replaced by corresponding new versions from com.fasterxml.jackson. ## How was this patch tested? - Locally tested with existing unit/intergration tests in project with different Hadoop versions. ``` export JAVA_HOME=/path/to/java11; mvn clean package -Dhadoop.version=3.3.6; # with latest stable release mvn clean package; # with default Hadoop version ``` - In physical test environment where Hadoop 3.3.6 / Hive 4 are installed on Ubutnu 20. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4783: remove duplicates for users/groups/roles during policy validation [ranger]
fateh288 opened a new pull request, #311: URL: https://github.com/apache/ranger/pull/311 ## What changes were proposed in this pull request? De-duplicate the users/groups/roles during policy validation when policy created/updated via REST (duplicates are prevented via ranger UI already) ## How was this patch tested? Local Docker container with master branch Unzip the attached scripts folder - the folder contains some payloads which have duplicate users/groups/roles. Run `bash create-policies.sh "/service/public/v2/api/policy"` command and wait for the script to finish Login to Ranger Admin and check policies created in Hive - duplicate users/groups/roles are de-duplicated. [scripts 3.zip](https://github.com/apache/ranger/files/15121965/scripts.3.zip) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
fateh288 commented on code in PR #307: URL: https://github.com/apache/ranger/pull/307#discussion_r1575414974 ## ranger-tools/src/main/python/stress/stress-hbase-loadgenerator.py: ## @@ -0,0 +1,125 @@ +import subprocess +import time +import argparse +import os +from datetime import datetime + +def increase_memory_for_loadgenerator(): +try: +cmd = "export HBASE_OPTS='-Xmx10g'" +print(cmd) +op = subprocess.call(cmd, shell=True) Review Comment: Fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
fateh288 commented on code in PR #307:
URL: https://github.com/apache/ranger/pull/307#discussion_r1575414848
##
ranger-tools/src/main/python/stress/stress-hbase-loadgenerator.py:
##
@@ -0,0 +1,125 @@
+import subprocess
+import time
+import argparse
+import os
+from datetime import datetime
+
+def increase_memory_for_loadgenerator():
+try:
+cmd = "export HBASE_OPTS='-Xmx10g'"
+print(cmd)
+op = subprocess.call(cmd, shell=True)
+print("Output:", op)
+except subprocess.CalledProcessError as e:
+print("Error in setting HBASE_HEAPSIZE:", e)
+exit(1)
+def login(keytab_path, user_name):
+try:
+cmd = f"kinit -kt {keytab_path} {user_name}"
+print(cmd)
+login_op = subprocess.call(cmd, shell=True)
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
fateh288 commented on code in PR #307:
URL: https://github.com/apache/ranger/pull/307#discussion_r1575408160
##
ranger-tools/src/main/python/stress/stress-hbase-loadgenerator.py:
##
@@ -0,0 +1,125 @@
+import subprocess
+import time
+import argparse
+import os
+from datetime import datetime
+
+def increase_memory_for_loadgenerator():
+try:
+cmd = "export HBASE_OPTS='-Xmx10g'"
+print(cmd)
+op = subprocess.call(cmd, shell=True)
+print("Output:", op)
+except subprocess.CalledProcessError as e:
+print("Error in setting HBASE_HEAPSIZE:", e)
+exit(1)
+def login(keytab_path, user_name):
+try:
+cmd = f"kinit -kt {keytab_path} {user_name}"
+print(cmd)
+login_op = subprocess.call(cmd, shell=True)
+print("Login output:", login_op)
+except subprocess.CalledProcessError as e:
+print("Error in login:", e)
+exit(1)
+
+def create_ltt_command_multiput(num_cols_per_cf=1000, num_threads=10,
num_keys=100, table_name = "multitest",avg_data_size=2, num_col_families=3,
col_family_pattern="cf%d", num_regions_per_server=1):
+def get_column_families():
+col_families = []
+for i in range(num_col_families):
+col_families.append(col_family_pattern % i)
+return ','.join(col_families)
+#Sample: hbase ltt -tn multitest -families f1,f2,f3 -write 2:2:20
-multiput -num_keys 1000 -num_regions_per_server 1
+cmd = f"hbase ltt -tn {table_name} -families {get_column_families()}
-write {num_cols_per_cf}:{avg_data_size}:{num_threads}" \
+ f" -multiput -num_keys {num_keys} -num_regions_per_server
{num_regions_per_server}"
+return cmd
+
+
+def create_pe_command_multiget(multiget_batchsize=500, num_threads=10,
num_keys=100, table_name="multitest", num_col_families=3):
+#Sample: hbase pe --table=multitest --families=3 --columns=1
--multiGet=10 --rows=1000 --nomapred randomRead 5
+
+cmd = f"hbase pe --table={table_name} --families={num_col_families}
--columns={num_cols_per_cf} " \
+ f"--multiGet={multiget_batchsize} --rows={num_keys} --nomapred
randomRead {num_threads}"
+return cmd
+
+
+
+def generate_hbase_load(op_type, multiget_batchsize, num_cf, num_keys_list,
num_cols_per_cf, num_threads_list, metadata,
csv_outfile="/root/ltt_output.csv", ):
Review Comment:
Command line argument -csv_output can be used in that case to provide a
custom path where one has appropriate access
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
kumaab commented on code in PR #307:
URL: https://github.com/apache/ranger/pull/307#discussion_r1575399634
##
ranger-tools/src/main/python/stress/stress-hbase-loadgenerator.py:
##
@@ -0,0 +1,125 @@
+import subprocess
+import time
+import argparse
+import os
+from datetime import datetime
+
+def increase_memory_for_loadgenerator():
+try:
+cmd = "export HBASE_OPTS='-Xmx10g'"
+print(cmd)
+op = subprocess.call(cmd, shell=True)
Review Comment:
subprocess.call() -> subprocess.run()
##
ranger-tools/src/main/python/stress/stress-hbase-loadgenerator.py:
##
@@ -0,0 +1,125 @@
+import subprocess
+import time
+import argparse
+import os
+from datetime import datetime
+
+def increase_memory_for_loadgenerator():
+try:
+cmd = "export HBASE_OPTS='-Xmx10g'"
+print(cmd)
+op = subprocess.call(cmd, shell=True)
+print("Output:", op)
+except subprocess.CalledProcessError as e:
+print("Error in setting HBASE_HEAPSIZE:", e)
+exit(1)
+def login(keytab_path, user_name):
+try:
+cmd = f"kinit -kt {keytab_path} {user_name}"
+print(cmd)
+login_op = subprocess.call(cmd, shell=True)
Review Comment:
subprocess.call() -> subprocess.run()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
kumaab commented on code in PR #307:
URL: https://github.com/apache/ranger/pull/307#discussion_r1575399048
##
ranger-tools/src/main/python/stress/stress-hbase-loadgenerator.py:
##
@@ -0,0 +1,125 @@
+import subprocess
+import time
+import argparse
+import os
+from datetime import datetime
+
+def increase_memory_for_loadgenerator():
+try:
+cmd = "export HBASE_OPTS='-Xmx10g'"
+print(cmd)
+op = subprocess.call(cmd, shell=True)
+print("Output:", op)
+except subprocess.CalledProcessError as e:
+print("Error in setting HBASE_HEAPSIZE:", e)
+exit(1)
+def login(keytab_path, user_name):
+try:
+cmd = f"kinit -kt {keytab_path} {user_name}"
+print(cmd)
+login_op = subprocess.call(cmd, shell=True)
+print("Login output:", login_op)
+except subprocess.CalledProcessError as e:
+print("Error in login:", e)
+exit(1)
+
+def create_ltt_command_multiput(num_cols_per_cf=1000, num_threads=10,
num_keys=100, table_name = "multitest",avg_data_size=2, num_col_families=3,
col_family_pattern="cf%d", num_regions_per_server=1):
+def get_column_families():
+col_families = []
+for i in range(num_col_families):
+col_families.append(col_family_pattern % i)
+return ','.join(col_families)
+#Sample: hbase ltt -tn multitest -families f1,f2,f3 -write 2:2:20
-multiput -num_keys 1000 -num_regions_per_server 1
+cmd = f"hbase ltt -tn {table_name} -families {get_column_families()}
-write {num_cols_per_cf}:{avg_data_size}:{num_threads}" \
+ f" -multiput -num_keys {num_keys} -num_regions_per_server
{num_regions_per_server}"
+return cmd
+
+
+def create_pe_command_multiget(multiget_batchsize=500, num_threads=10,
num_keys=100, table_name="multitest", num_col_families=3):
+#Sample: hbase pe --table=multitest --families=3 --columns=1
--multiGet=10 --rows=1000 --nomapred randomRead 5
+
+cmd = f"hbase pe --table={table_name} --families={num_col_families}
--columns={num_cols_per_cf} " \
+ f"--multiGet={multiget_batchsize} --rows={num_keys} --nomapred
randomRead {num_threads}"
+return cmd
+
+
+
+def generate_hbase_load(op_type, multiget_batchsize, num_cf, num_keys_list,
num_cols_per_cf, num_threads_list, metadata,
csv_outfile="/root/ltt_output.csv", ):
Review Comment:
User might not have write permissions writing to the /root dir, please check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] RANGER-4776: SolrAuditDestination uses local SSLContext instead of se… [ranger]
turcsanyip opened a new pull request, #310: URL: https://github.com/apache/ranger/pull/310 …tting the system-wide default ## What changes were proposed in this pull request? In order to avoid setting the system-wide `SSLContext` in `SolrAuditDestination` (which may affect other components running in the same JVM negatively), use Solr's `HttpClientUtil.setSocketFactoryRegistryProvider()` to pass the `SSLContext` to the Solr client locally. ## How was this patch tested? We use the Ranger Plugin in Apache NiFi. Before the change, Azure components in NiFi failed with "unable to find valid certification path to requested target" error. After the change, the Ranger - Solr connection still works and the Azure components can also access the needed certificate in the system default truststore (cacerts). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4746: Change default docker build to use JDK 8 [ranger]
mneethiraj merged PR #309: URL: https://github.com/apache/ranger/pull/309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4750: Optimizations in the docker image build process [ranger]
mneethiraj commented on code in PR #302:
URL: https://github.com/apache/ranger/pull/302#discussion_r1558348942
##
dev-support/ranger-docker/Dockerfile.ranger-hbase:
##
@@ -22,14 +22,15 @@ ARG HBASE_PLUGIN_VERSION
COPY ./dist/version
/home/ranger/dist/
COPY ./dist/ranger-${HBASE_PLUGIN_VERSION}-hbase-plugin.tar.gz
/home/ranger/dist/
-COPY ./downloads/hbase-${HBASE_VERSION}-bin.tar.gz
/home/ranger/dist/
-COPY ./scripts/ranger-hbase-setup.sh /home/ranger/scripts/
-COPY ./scripts/ranger-hbase.sh /home/ranger/scripts/
-COPY ./scripts/ranger-hbase-plugin-install.properties/home/ranger/scripts/
-COPY ./scripts/hbase-site.xml/home/ranger/scripts/
+COPY ./scripts/ranger-hbase-setup.sh
/home/ranger/scripts/
+COPY ./scripts/ranger-hbase.sh
/home/ranger/scripts/
+COPY ./scripts/ranger-hbase-plugin-install.properties
/home/ranger/scripts/
+COPY ./scripts/hbase-site.xml
/home/ranger/scripts/
-RUN tar xvfz /home/ranger/dist/hbase-${HBASE_VERSION}-bin.tar.gz
--directory=/opt/ && \
+RUN wget -O /home/ranger/dist/hbase-${HBASE_VERSION}-bin.tar.gz \
Review Comment:
files like hbase--bin.tar.gz could be large. It will help to avoid
downloading such files for every build. Please review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] RANGER-4746: Change default docker build to use JDK 8 [ranger]
kumaab opened a new pull request, #309: URL: https://github.com/apache/ranger/pull/309 ## What changes were proposed in this pull request? Default build currently uses JDK 11, changing the default to refer to JDK 8. ## How was this patch tested? Ongoing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4762:Prevent duplicate values for resource while validating policy [ranger]
fateh288 commented on PR #308: URL: https://github.com/apache/ranger/pull/308#issuecomment-2035122830 The changes have been merged : https://github.com/apache/ranger/commit/ffda775366ac8ac8a6869991dbab5e12d307a423 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4762:Prevent duplicate values for resource while validating policy [ranger]
fateh288 closed pull request #308: RANGER-4762:Prevent duplicate values for resource while validating policy URL: https://github.com/apache/ranger/pull/308 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
kumaab commented on code in PR #307:
URL: https://github.com/apache/ranger/pull/307#discussion_r1548609407
##
ranger-tools/src/main/python/stress/stress-hbase-loadgenerator.py:
##
@@ -0,0 +1,106 @@
+import subprocess
+import time
+import argparse
+import os
+from datetime import datetime
+
+def increase_memory_for_loadgenerator():
+try:
+cmd = "export HBASE_OPTS='-Xmx10g'"
+print(cmd)
+op = subprocess.call(cmd, shell=True)
+print("Output:", op)
+except subprocess.CalledProcessError as e:
+print("Error in setting HBASE_HEAPSIZE:", e)
+exit(1)
+def login():
+try:
+cmd = "kinit -kt systest"
Review Comment:
Instead of hardcoding the value it is better to provide the keytab as an
argument. Alternatively, doing the kinit externally and documenting the steps
in readme also works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
kumaab commented on code in PR #307:
URL: https://github.com/apache/ranger/pull/307#discussion_r1548604094
##
ranger-tools/src/main/python/stress/stress-hbase-loadgenerator.py:
##
@@ -0,0 +1,106 @@
+import subprocess
+import time
+import argparse
+import os
+from datetime import datetime
+
+def increase_memory_for_loadgenerator():
+try:
+cmd = "export HBASE_OPTS='-Xmx10g'"
+print(cmd)
+op = subprocess.call(cmd, shell=True)
+print("Output:", op)
+except subprocess.CalledProcessError as e:
+print("Error in setting HBASE_HEAPSIZE:", e)
+exit(1)
+def login():
+try:
+cmd = "kinit -kt systest"
+print(cmd)
+login_op = subprocess.call(cmd, shell=True)
+print("Login output:", login_op)
+except subprocess.CalledProcessError as e:
+print("Error in login:", e)
+exit(1)
+
+def create_ltt_command_multiput(num_cols_per_cf=1000, num_threads=10,
num_keys=100, table_name = "multitest",avg_data_size=2, num_col_families=3,
col_family_pattern="cf%d", num_regions_per_server=1):
+def get_column_families():
+col_families = []
+for i in range(num_col_families):
+col_families.append(col_family_pattern % i)
+return ','.join(col_families)
+#Sample: hbase ltt -tn multitest -families f1,f2,f3 -write 2:2:20
-multiput -num_keys 1000 -num_regions_per_server 1
+cmd = f"hbase ltt -tn {table_name} -families {get_column_families()}
-write {num_cols_per_cf}:{avg_data_size}:{num_threads}" \
+ f" -multiput -num_keys {num_keys} -num_regions_per_server
{num_regions_per_server}"
+return cmd
+
+
+def create_pe_command_multiget(multiget_batchsize=500, num_threads=10,
num_keys=100, table_name="multitest", num_col_families=3):
+#Sample: hbase pe --table=multitest --families=3 --columns=1
--multiGet=10 --rows=1000 --nomapred randomRead 5
+
+cmd = f"hbase pe --table={table_name} --families={num_col_families}
--columns={num_cols_per_cf} " \
+ f"--multiGet={multiget_batchsize} --rows={num_keys} --nomapred
randomRead {num_threads}"
+return cmd
+
+
+
+def generate_hbase_load(op_type, multiget_batchsize, num_cf, num_rows_list,
num_cols_per_cf, num_threads_list, metadata,
csv_outfile="/root/ltt_output.csv", ):
+#if output file does not exist only then write the header
+if(not os.path.exists(csv_outfile)):
+with open(csv_outfile, "w") as f:
+
f.write("op,num_cf,num_keys,num_cols_per_cf,num_threads,time_taken,command,metadata,date_start,time_start,date_end,time_end\n")
+assert type(num_threads_list) == list
+assert type(num_rows_list) == list
+for num_keys in num_rows_list:
+for num_threads in num_threads_list:
+if op_type == "multiput":
+cmd =
create_ltt_command_multiput(num_cols_per_cf=num_cols_per_cf,
+ num_threads=num_threads,
+ num_keys=num_keys,
+ num_col_families=num_cf)
+elif op_type == "multiget":
+cmd =
create_pe_command_multiget(multiget_batchsize=multiget_batchsize,
+ num_threads=num_threads,
+ num_keys=num_keys,
+ num_col_families=num_cf)
+else:
+print("Invalid op_type")
+exit(1)
+
+datetime_start = datetime.now()
+date_start_str = datetime_start.date()
+time_start_str = str(datetime_start.time()).split(".")[0]
+time_start = time.time()
+ltt_out = subprocess.call(cmd, shell=True)
Review Comment:
Consider adding error handling with hbase commands.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
kumaab commented on code in PR #307:
URL: https://github.com/apache/ranger/pull/307#discussion_r1548582891
##
ranger-tools/src/main/python/stress/stress-hbase-loadgenerator.py:
##
@@ -0,0 +1,106 @@
+import subprocess
+import time
+import argparse
+import os
+from datetime import datetime
+
+def increase_memory_for_loadgenerator():
+try:
+cmd = "export HBASE_OPTS='-Xmx10g'"
+print(cmd)
+op = subprocess.call(cmd, shell=True)
+print("Output:", op)
+except subprocess.CalledProcessError as e:
+print("Error in setting HBASE_HEAPSIZE:", e)
+exit(1)
+def login():
+try:
+cmd = "kinit -kt systest"
+print(cmd)
+login_op = subprocess.call(cmd, shell=True)
+print("Login output:", login_op)
+except subprocess.CalledProcessError as e:
+print("Error in login:", e)
+exit(1)
+
+def create_ltt_command_multiput(num_cols_per_cf=1000, num_threads=10,
num_keys=100, table_name = "multitest",avg_data_size=2, num_col_families=3,
col_family_pattern="cf%d", num_regions_per_server=1):
+def get_column_families():
+col_families = []
+for i in range(num_col_families):
+col_families.append(col_family_pattern % i)
+return ','.join(col_families)
+#Sample: hbase ltt -tn multitest -families f1,f2,f3 -write 2:2:20
-multiput -num_keys 1000 -num_regions_per_server 1
+cmd = f"hbase ltt -tn {table_name} -families {get_column_families()}
-write {num_cols_per_cf}:{avg_data_size}:{num_threads}" \
+ f" -multiput -num_keys {num_keys} -num_regions_per_server
{num_regions_per_server}"
+return cmd
+
+
+def create_pe_command_multiget(multiget_batchsize=500, num_threads=10,
num_keys=100, table_name="multitest", num_col_families=3):
+#Sample: hbase pe --table=multitest --families=3 --columns=1
--multiGet=10 --rows=1000 --nomapred randomRead 5
+
+cmd = f"hbase pe --table={table_name} --families={num_col_families}
--columns={num_cols_per_cf} " \
+ f"--multiGet={multiget_batchsize} --rows={num_keys} --nomapred
randomRead {num_threads}"
+return cmd
+
+
+
+def generate_hbase_load(op_type, multiget_batchsize, num_cf, num_rows_list,
num_cols_per_cf, num_threads_list, metadata,
csv_outfile="/root/ltt_output.csv", ):
+#if output file does not exist only then write the header
+if(not os.path.exists(csv_outfile)):
+with open(csv_outfile, "w") as f:
+
f.write("op,num_cf,num_keys,num_cols_per_cf,num_threads,time_taken,command,metadata,date_start,time_start,date_end,time_end\n")
+assert type(num_threads_list) == list
+assert type(num_rows_list) == list
+for num_keys in num_rows_list:
+for num_threads in num_threads_list:
+if op_type == "multiput":
+cmd =
create_ltt_command_multiput(num_cols_per_cf=num_cols_per_cf,
+ num_threads=num_threads,
+ num_keys=num_keys,
+ num_col_families=num_cf)
+elif op_type == "multiget":
+cmd =
create_pe_command_multiget(multiget_batchsize=multiget_batchsize,
+ num_threads=num_threads,
+ num_keys=num_keys,
+ num_col_families=num_cf)
+else:
+print("Invalid op_type")
+exit(1)
+
+datetime_start = datetime.now()
+date_start_str = datetime_start.date()
+time_start_str = str(datetime_start.time()).split(".")[0]
+time_start = time.time()
+ltt_out = subprocess.call(cmd, shell=True)
Review Comment:
subprocess.run() is a recommended approach, please see:
https://docs.python.org/3/library/subprocess.html#using-the-subprocess-module
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
kumaab commented on code in PR #307:
URL: https://github.com/apache/ranger/pull/307#discussion_r1548578831
##
ranger-tools/src/main/python/stress/stress-hbase-loadgenerator.py:
##
@@ -0,0 +1,106 @@
+import subprocess
+import time
+import argparse
+import os
+from datetime import datetime
+
+def increase_memory_for_loadgenerator():
+try:
+cmd = "export HBASE_OPTS='-Xmx10g'"
+print(cmd)
+op = subprocess.call(cmd, shell=True)
+print("Output:", op)
+except subprocess.CalledProcessError as e:
+print("Error in setting HBASE_HEAPSIZE:", e)
+exit(1)
+def login():
+try:
+cmd = "kinit -kt systest"
+print(cmd)
+login_op = subprocess.call(cmd, shell=True)
+print("Login output:", login_op)
+except subprocess.CalledProcessError as e:
+print("Error in login:", e)
+exit(1)
+
+def create_ltt_command_multiput(num_cols_per_cf=1000, num_threads=10,
num_keys=100, table_name = "multitest",avg_data_size=2, num_col_families=3,
col_family_pattern="cf%d", num_regions_per_server=1):
+def get_column_families():
+col_families = []
+for i in range(num_col_families):
+col_families.append(col_family_pattern % i)
+return ','.join(col_families)
+#Sample: hbase ltt -tn multitest -families f1,f2,f3 -write 2:2:20
-multiput -num_keys 1000 -num_regions_per_server 1
+cmd = f"hbase ltt -tn {table_name} -families {get_column_families()}
-write {num_cols_per_cf}:{avg_data_size}:{num_threads}" \
+ f" -multiput -num_keys {num_keys} -num_regions_per_server
{num_regions_per_server}"
+return cmd
+
+
+def create_pe_command_multiget(multiget_batchsize=500, num_threads=10,
num_keys=100, table_name="multitest", num_col_families=3):
+#Sample: hbase pe --table=multitest --families=3 --columns=1
--multiGet=10 --rows=1000 --nomapred randomRead 5
+
+cmd = f"hbase pe --table={table_name} --families={num_col_families}
--columns={num_cols_per_cf} " \
+ f"--multiGet={multiget_batchsize} --rows={num_keys} --nomapred
randomRead {num_threads}"
+return cmd
+
+
+
+def generate_hbase_load(op_type, multiget_batchsize, num_cf, num_rows_list,
num_cols_per_cf, num_threads_list, metadata,
csv_outfile="/root/ltt_output.csv", ):
+#if output file does not exist only then write the header
+if(not os.path.exists(csv_outfile)):
+with open(csv_outfile, "w") as f:
+
f.write("op,num_cf,num_keys,num_cols_per_cf,num_threads,time_taken,command,metadata,date_start,time_start,date_end,time_end\n")
+assert type(num_threads_list) == list
+assert type(num_rows_list) == list
+for num_keys in num_rows_list:
+for num_threads in num_threads_list:
+if op_type == "multiput":
+cmd =
create_ltt_command_multiput(num_cols_per_cf=num_cols_per_cf,
+ num_threads=num_threads,
+ num_keys=num_keys,
+ num_col_families=num_cf)
+elif op_type == "multiget":
+cmd =
create_pe_command_multiget(multiget_batchsize=multiget_batchsize,
+ num_threads=num_threads,
+ num_keys=num_keys,
+ num_col_families=num_cf)
+else:
+print("Invalid op_type")
+exit(1)
+
+datetime_start = datetime.now()
+date_start_str = datetime_start.date()
+time_start_str = str(datetime_start.time()).split(".")[0]
+time_start = time.time()
+ltt_out = subprocess.call(cmd, shell=True)
+time_end = time.time()
+datetime_end = datetime.now()
+date_end_str = datetime_end.date()
+time_end_str = str(datetime_end.time()).split(".")[0]
+time_taken = time_end - time_start
+
+print("cmd:", cmd)
+print("LTT output:", ltt_out)
+print("Time taken:", time_taken)
+with open(csv_outfile, "a") as f:
+if ltt_out != 0:
+time_taken = "non_zero_exit_code"
+
f.write(f'{op_type},{num_cf},{num_keys},{num_cols_per_cf},{num_threads},{time_taken},"{cmd}",{metadata},{date_start_str},{time_start_str},{date_end_str},{time_end_str}\n')
+print(f"Written to file: {csv_outfile}")
+# Sleep added so that the next command does not start immediately
and any metric measurement such as heap useage can be captured more accurately
+time.sleep(90)
+
+if __name__ == '__main__':
+argparser = argparse.ArgumentParser("Generate LTT load and create report")
+argparser.add_argument('-csv_output', '--csv_output', help='Full path to
the csv output file',
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
fateh288 commented on PR #307: URL: https://github.com/apache/ranger/pull/307#issuecomment-2032639005 @mneethiraj @rameeshm Following up for review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4623:Change the http client of the knox plug-in to avoid conne… [ranger]
upczsh commented on PR #295: URL: https://github.com/apache/ranger/pull/295#issuecomment-2029097463 Requesting review for this PR. @kumaab @bhavikpatel9977 @mneethiraj -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4622:property ranger.accesslog.rotate.rename_on_rotate, the de… [ranger]
upczsh commented on PR #294: URL: https://github.com/apache/ranger/pull/294#issuecomment-2029097118 Requesting review for this PR. @kumaab @bhavikpatel9977 @mneethiraj -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4454. KMS: adds support for the SM4 encryption algorithm. [ranger]
xiaojunxiang2023 commented on PR #287: URL: https://github.com/apache/ranger/pull/287#issuecomment-2026279526 @bhavikpatel9977 @vikaskr22 , hi, I just responded to you on issues “https://issues.apache.org/jira/browse/RANGER-4454”. Please help me review it, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4762:Prevent duplicate values for resource while validating policy [ranger]
fateh288 opened a new pull request, #308:
URL: https://github.com/apache/ranger/pull/308
Previously: Via REST, policy creation possible when duplicate values
specified for a resource as a list e.g. for a resource, multiple dbs can be
mentioned like db:[test_db1,test_db1] and this policy is considered different
from a policy already created for the resource [test_db1]
The same scenario is already prevented by Ranger UI as it prevents duplicate
values for a resource (in a given policy)
## What changes were proposed in this pull request?
After this patch: Error is thrown for duplicate resource values like
[test_db1,test_db1] - {"statusCode":1,"msgDesc":"(0) Validation failure: error
code[3056], reason[Values for the resource=database contained a duplicate
value=test_db1. Ensure all values for a resource are unique],
field[resource-values], subfield[database], type[semantically incorrect] "}
## How was this patch tested?
Unit tests added for the scenario
Manually tested via REST end points for policy creation that duplicate
resource values for a resource throws above error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
fateh288 commented on PR #307: URL: https://github.com/apache/ranger/pull/307#issuecomment-2023262368 Requesting review for this PR. @kumaab @bhavikpatel9977 @mneethiraj -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4761: make lazy memory allocation for family map instead … [ranger]
fateh288 opened a new pull request, #307: URL: https://github.com/apache/ranger/pull/307 …of ahead of time memory allocation for family map of type Map>. Removed ColumnFailyCache. Impact: Memory and computational benefit - Cache memory saved & huge reduction in memory when large number of columns accessed. Since ColumnFamilyCache is always a miss because of non deterministic access patterns and also a bug wherein address of byte array is used as key in cache, we get computational benefit by removing ColumnFamilyCache. Memory footprint will get reduced even further when enabling column auth optimization supported by RANGER-4670 ## What changes were proposed in this pull request? Map> getColumnFamilies(Map> families) function was the original bottleneck for resulting in Ranger Authz CP taking 60% of Put RPC time — it is a computationally heavy function converting bytes to string and type-casting Collection to set of strings. ColumnFamilyCache was introduced to fix this issue. But this caching does not work because the columns and column families accessed for a table are inconsistent and results in many many cache misses and thus results in getColumnFamilies() getting called – so additional memory requirement for cache (grows exponentially due to large number of columns) and also double computation (serialization of new family map to check in cache and also getColumnFamilies is computed). E.g. First call comes for cf1:c1,c3,c4 Second call comes for cf1:c1,c2,c5 then the second call is a cache miss because the set of columns accessed is different from the first call. Both these entries get added to the cache. Validation: Added debug statements for cache hit and miss counts. word count in log file for Cache Miss and Cache Hit, Cache size reaches max default size of 1024 ``` root@ccycloud-2 hbase]# grep 'Cache Miss' hbase-cmf-HBASE-1-REGIONSERVER-ccycloud-2.fs2-7192.root.comops.site.log.out | wc -l 10584 [root@ccycloud-2 hbase]# grep 'Cache Hit' hbase-cmf-HBASE-1-REGIONSERVER-ccycloud-2.fs2-7192.root.comops.site.log.out | wc -l 0 2024-02-09 17:12:31,093 DEBUG org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor: evaluateAccess: Cache Size:1024 2024-02-09 17:12:31,096 DEBUG org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor: evaluateAccess: Cache Size:1024 ``` In the patch, make lazy memory allocation for family map instead of ahead of time memory allocation for family map of type Map>. Removed ColumnFamilyCache from implementation since it was always a miss. ## How was this patch tested? Unit test cases pass for hbase plugin -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4278:Update query not working on column level access for Trino… [ranger]
respon007 commented on PR #306: URL: https://github.com/apache/ranger/pull/306#issuecomment-2016822387 This is my testCase and screenshot. https://github.com/apache/ranger/assets/36159052/72e0641f-de6b-4399-b96a-5dbca959f403;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4278:Update query not working on column level access for Trino… [ranger]
respon007 opened a new pull request, #306: URL: https://github.com/apache/ranger/pull/306 … plugin ## What changes were proposed in this pull request? Fix Update query not working on column level access for Trino ## How was this patch tested? you should add 'update' policy first by Ranger UI, and you can test SQL like 'update table1 set col2='bbb' where col1 = 1 ' https://github.com/apache/ranger/assets/36159052/2dc7d2f8-f85e-4c57-aa75-5e05a5ee630a;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4752: Enable jdk 11 builds on PRs and merges to master [ranger]
kumaab merged PR #305: URL: https://github.com/apache/ranger/pull/305 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4752: Enable jdk 11 builds on PRs and merges to master [ranger]
kumaab closed pull request #304: RANGER-4752: Enable jdk 11 builds on PRs and merges to master URL: https://github.com/apache/ranger/pull/304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4752: Enable jdk 11 builds on PRs and merges to master [ranger]
kumaab opened a new pull request, #305: URL: https://github.com/apache/ranger/pull/305 ## What changes were proposed in this pull request? Enable JDK 11 builds on master and PRs. ## How was this patch tested? In Progress -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4746: Fix ranger-jdk11 build profile [ranger]
kumaab merged PR #301: URL: https://github.com/apache/ranger/pull/301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4511: Enable Parallel Compilation for Multiple Modules in Ranger [ranger]
kumaab merged PR #289: URL: https://github.com/apache/ranger/pull/289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4746: Fix ranger-jdk11 build profile [ranger]
kumaab commented on PR #301: URL: https://github.com/apache/ranger/pull/301#issuecomment-2014398637 Verified that all tests pass with : `mvn clean package -P ranger-jdk11` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Update maven.yml [ranger]
kumaab opened a new pull request, #304: URL: https://github.com/apache/ranger/pull/304 ## What changes were proposed in this pull request? Enable JDK 11 builds on master and PRs. ## How was this patch tested? In Progress -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4511: Enable Parallel Compilation for Multiple Modules in Ranger [ranger]
kumaab commented on PR #289: URL: https://github.com/apache/ranger/pull/289#issuecomment-2005327035 + 1 The changes look good, this is helpful in reducing the time taken for mvn builds significantly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Resolve ranger-examples-distro module compile error [ranger]
vikaskumareinstein commented on PR #71: URL: https://github.com/apache/ranger/pull/71#issuecomment-2002525114 Actually, I was getting this issue. I didn't mean to merge it. Thanks for the clarification though. Will try with your fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Resolve ranger-examples-distro module compile error [ranger]
shubhluck commented on PR #71: URL: https://github.com/apache/ranger/pull/71#issuecomment-2002343984 This is not an issue that needs to be merged, it happens when there is a mismatch in main Ranger version specified in ranger/pom.xml and distro/pom.xml or ranger-examples/pom.xml, this causes unable to identify right project parent directory causing file path or no such file exception issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Update maven.yml [ranger]
kumaab opened a new pull request, #303: URL: https://github.com/apache/ranger/pull/303 ## What changes were proposed in this pull request? Enable docker builds with PRs/merges. ## How was this patch tested? In progress -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4750: Optimizations in the docker image build process [ranger]
kumaab opened a new pull request, #302:
URL: https://github.com/apache/ranger/pull/302
## What changes were proposed in this pull request?
Restructure the downloads of service tarballs from `download-archives.sh` to
`Dockerfile.ranger-service` as part of the image build for respective services.
Key highlights:
- When building specific images (eg: ranger-hadoop, ranger-hive), only
hadoop and hive tarballs are downloaded bringing down the time taken to bring
up the containers effectively.
- Allows to take advantage of docker parallelization thereby allowing
parallel service tarball downloads. Currently, the service tarball downloads
happen in a sequential manner with `download-archives.sh`.
- Although subject to network speeds, this also has shown minor improvements
in overall build times for all images.
## How was this patch tested?
Command to bring up containers:
- `docker-compose -f docker-compose.ranger-base.yml -f
docker-compose.ranger.yml -f docker-compose.ranger-${RANGER_DB_TYPE}.yml -f
docker-compose.ranger-usersync.yml -f docker-compose.ranger-tagsync.yml -f
docker-compose.ranger-kms.yml -f docker-compose.ranger-hadoop.yml -f
docker-compose.ranger-hbase.yml -f docker-compose.ranger-kafka.yml -f
docker-compose.ranger-hive.yml -f docker-compose.ranger-trino.yml -f
docker-compose.ranger-knox.yml build`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4747: Add exception handling in docker based service creation [ranger]
mneethiraj merged PR #300: URL: https://github.com/apache/ranger/pull/300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4746: Fix ranger-jdk11 profile to only have the required modules [ranger]
kumaab opened a new pull request, #301: URL: https://github.com/apache/ranger/pull/301 ## What changes were proposed in this pull request? The ranger profile `ranger-jdk11` has been added to build trino with jdk 11 as currently trino plugin is the only plugin to have a compile time dependency on jdk 11. However, it currently builds the full ranger repo. The PR aims to fix this. When modules have a jdk 11 requirement they can be added to this profile. ## How was this patch tested? - Bring up the containers - Build with JDK 11: `mvn clean package -DskipTests -P ranger-jdk11` - `cp target/ranger-admin.tar.gz dev-support/ranger-docker/dist` - `cp target/ranger-trino-plugin.tar.gz dev-support/ranger-docker/dist/` - `cd dev-support/ranger-docker` - `docker-compose -f docker-compose.ranger.yml -f docker-compose.ranger-postgres.yml -f docker-compose.ranger-trino.yml build` - `docker-compose -f docker-compose.ranger.yml -f docker-compose.ranger-postgres.yml up -d` - `docker-compose -f docker-compose.ranger.yml -f docker-compose.ranger-postgres.yml -f docker-compose.ranger-trino.yml up -d` - Verified ranger and ranger-trino are successfully up in containers. - Verified ranger policy creation, policies created for trino are working. - Verified audits to Solr are visible in ranger UI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4747: Add exception handling in docker based service creation [ranger]
kumaab opened a new pull request, #300: URL: https://github.com/apache/ranger/pull/300 ## What changes were proposed in this pull request? create-ranger-services.py creates ranger services after ranger is up in a docker container. If service creation for hdfs fails, subsequent service creation is aborted, the PR aims to fix this behavior. ## How was this patch tested? Tested the changes by bringing up ranger in a container and below are the installation logs seen when only trino is packaged with ranger. (Please explain how this patch was tested. Ex: unit tests, manual tests) (If this patch involves UI changes, please attach a screen-shot; otherwise, remove this) https://github.com/apache/ranger/assets/16475454/a0b8effe-2fdf-4bea-a19e-f1c274b0784a;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4626: Support trino to enable kerberos [ranger]
everypp commented on PR #296: URL: https://github.com/apache/ranger/pull/296#issuecomment-1989833162 @mneethiraj Can you help review it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4734: Trino Ranger Integration in Docker [ranger]
mneethiraj commented on PR #299: URL: https://github.com/apache/ranger/pull/299#issuecomment-1987698374 @kumaab - thank you for the enhancement. This patch is merged in master branch. ``` commit 74ac212fc7e90ca6ea78ff2af01e4ddceb8b6a92 (HEAD -> master, origin/master, origin/HEAD) Author: abhishek-kumar Date: Sat Mar 9 00:37:01 2024 -0800 RANGER-4734: Docker setup to run Ranger enabled Trino Signed-off-by: Madhan Neethiraj ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4734: Trino Ranger Integration in Docker [ranger]
mneethiraj closed pull request #299: RANGER-4734: Trino Ranger Integration in Docker URL: https://github.com/apache/ranger/pull/299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4734: Trino Ranger Integration in Docker [ranger]
kumaab opened a new pull request, #299: URL: https://github.com/apache/ranger/pull/299 ## What changes were proposed in this pull request? Introduces Docker Image for Trino in Ranger ## How was this patch tested? - Bring up the containers - Build with JDK 11: `mvn clean package -DskipTests -P ranger-jdk11 -pl '!unixauthnative'` - `cp target/ranger-admin.tar.gz dev-support/ranger-docker/dist` - `cp target/ranger-trino-plugin.tar.gz dev-support/ranger-docker/dist/` - `cd dev-support/ranger-docker` - `docker-compose -f docker-compose.ranger.yml -f docker-compose.ranger-postgres.yml -f docker-compose.ranger-trino.yml build` - `docker-compose -f docker-compose.ranger.yml -f docker-compose.ranger-postgres.yml up -d` - `docker-compose -f docker-compose.ranger.yml -f docker-compose.ranger-postgres.yml -f docker-compose.ranger-trino.yml up -d` - Audits to Solr visible in Ranger UI for Trino https://github.com/apache/ranger/assets/16475454/5a6fe81f-ed54-4a97-9579-c1e4371c45c1;> - Policies synced to Plugin https://github.com/apache/ranger/assets/16475454/8a2eba6a-a45c-406f-abf1-a230d76dd53d;> - Configuring Ranger Policies for Trino and accessing via Trino CLI https://github.com/apache/ranger/assets/16475454/602817b0-2324-4591-a7cc-93f4d8a59597;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Resolve ranger-examples-distro module compile error [ranger]
vikaskumareinstein commented on PR #71: URL: https://github.com/apache/ranger/pull/71#issuecomment-1983558014 While trying to build the Apache Ranger on Ubuntu 20.04, Using Java 1.8, I came across this issue. Not sure, whether this issue is related to maven or Ranger. It is failing on **Apache Ranger Distribution ** step. I am attaching the screenshot for reference.  ``` [ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.6:single (default) on project ranger-distro: Failed to create assem bly: Error creating assembly archive schema-registry-plugin: Problem creating jar: jar:file:/home/swapnendu_b/apache-ranger-2.4.0/distro/target/ran ger-distro-2.4.0.jar!/META-INF/maven/org.apache.ranger/ranger-distro/pom.properties: JAR entry META-INF/maven/org.apache.ranger/ranger-distro/pom.p roperties not found in /home/swapnendu_b/apache-ranger-2.4.0/distro/target/ranger-distro-2.4.0.jar -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :ranger-distro ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4390: creating another row batch causes null pointer exception [ranger]
rameeshm merged PR #298: URL: https://github.com/apache/ranger/pull/298 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
nil1729 commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1928029729 Hi @tarekabouzeid, I built the ranger locally with the JDK version `openjdk-17.0.8` and maven version `3.6.3`. There was some error when building on MAC due to the `shadow.h` header file not being found. Then I have used those jars to build Dockerfile and tested `trino` with docker only -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
tarekabouzeid commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1925902577 Hi @nil1729 Thank you for sharing, I guess its related to different JDK versions between Trino and Ranger, which version have you used to build ranger and which one you are using to run Trino ? For me, I build ranger using : maven:3.8.5-openjdk-17-slim, it was built fine but ES was not working in Trino. When I tried to build ranger using : maven:3.9.6-eclipse-temurin-17-focal, built failed I got this error while building, ```bash #7 191.6 [ERROR] /opt/ranger/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/ElasticSearchIndexBootStrapper.java:[53,36] error: cannot find symbol #7 191.6 [ERROR] symbol: class TimeValue #7 191.6 [ERROR] location: package org.elasticsearch.common.unit #7 191.6 [ERROR] /opt/ranger/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/ElasticSearchIndexBootStrapper.java:[54,40] error: cannot find symbol #7 191.6 [ERROR] symbol: class XContentType #7 191.6 [ERROR] location: package org.elasticsearch.common.xcontent #7 191.6 [ERROR] /opt/ranger/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/ElasticSearchIndexBootStrapper.java:[253,49] error: cannot find symbol ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
nil1729 commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1925422218 Hello @tarekabouzeid, How did you build the ranger plugin, and have you setup TLS on elasticsearch ? I was getting connection error initially with ranger when there is TLS enabled on elasticsearch. I have setup ranger with elasticsearch and trino 433, you can take a look once https://github.com/nil1729/trino-ranger-noob -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
tarekabouzeid commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1916909057 Hi, Thanks so much for this contribution, It helped me a lot. I have tested it with Trino 433 and Trino server is up and running and there seems to be an issue in Trino ranger raudits initialization when using Elastic search, so audits are not sent to ranger. I am going to look more into this issue and will report my findings. Elastic search version used : **7.16.2** Trino version: **433** Ranger: **2.4.0** Below is the error I am getting: ``` 024-01-30T13:32:13.562ZINFOmainstdout INFO - Connecting to ElasticSearch: User:ranger, http://es-internal.infra:80/ranger_audits 2024-01-30T13:32:13.895ZINFOmainstdout WARN - request [POST http://es-internal.infra:80/ranger_audits/_open?master_timeout=30s_throttled=false_unavailable=false_wildcards=closed_no_indices=true=30s] returned 1 warnings: [299 Elasticsearch-7.17.16-2b23fa076334f8d4651aeebe458a955a2ae23218 "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."] 2024-01-30T13:32:13.899ZINFOmainstdout ERROR - Can't connect to ElasticSearch server: User:ranger, http://es-internal.infra:80/ranger_auditsjava.lang.BootstrapMethodError: bootstrap method initialization exception at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:188) at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:316) at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:274) at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:264) at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:312) at org.elasticsearch.client.RestClient.performRequest(RestClient.java:296) at org.elasticsearch.client.RestClient.performRequest(RestClient.java:270) at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1632) at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1602) at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1572) at org.elasticsearch.client.IndicesClient.open(IndicesClient.java:614) at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.newClient(ElasticSearchAuditDestination.java:267) at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.getClient(ElasticSearchAuditDestination.java:187) at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.init(ElasticSearchAuditDestination.java:101) at org.apache.ranger.audit.provider.AuditProviderFactory.init(AuditProviderFactory.java:183) at org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:234) at org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControl.(RangerSystemAccessControl.java:120) at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) at org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControl.(RangerSystemAccessControl.java:69) at org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControl$$FastClassByGuice$$1345336.GUICE$TRAMPOLINE() at org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControl$$FastClassByGuice$$1345336.apply() at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82) at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114) at com.google.inject.internal.ConstructorInjector.access$000(ConstructorInjector.java:33) at com.google.inject.internal.ConstructorInjector$1.call(ConstructorInjector.java:98) at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:109) at io.airlift.bootstrap.LifeCycleModule.provision(LifeCycleModule.java:54) at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:117) at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:66) at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:93) at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:300) at
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
respon007 commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1915959805 > I omitted the part where I had to manually modify RANGER's BACK-END DB. I'm using POSTGRESQL and used DBeaver. > >   > >  > > I hope it helps you as it helped me. @origin0099 Thanks for your reply! I modify the file agents-common/src/main/resources/service-defs/ranger-servicedef-trino.json and re-compile ranger. Now I have solved the problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
origin0099 commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1913964046 I omitted the part where I had to manually modify RANGER's BACK-END DB. I'm using POSTGRESQL and used DBeaver.   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4390: creating another row batch causes null pointer exception [ranger]
fateh288 opened a new pull request, #298:
URL: https://github.com/apache/ranger/pull/298
Creating another row batch causes null pointer exception as
vectorizedRowBatchMap has old batch references and gets garbage collected.
## What changes were proposed in this pull request?
In `log(Writer writer, Collection events)`, creating new
row batch causes an exception since the new batch renders references in
vectorizedRowBatchMap invalid.
The creation of new batch is also not necessary as batch of same size was
allocated in initORCAuditSchema(). Also, creating a batch larger than orc
buffer size is also not useful since the batch is written to writer as soon as
its size reaches orc buffer size.
` if (batch.size == orcBufferSize) {
writer.addRowBatch(batch);
batch.reset();
}`
## How was this patch tested?
The unit test case TestAuditQueue.testAuditFileQueueSpoolORC failed when
xasecure.audit.destination.hdfs.batch.filequeue.filespool.buffer.size and
xasecure.audit.destination.hdfs.orc.buffersize were unequal. After the fix, the
test case passes and now different values (e.g. 10 and 5 respectively as shown
below) can be used for these properties.
```
props = {
xasecure.audit.destination.hdfs.orc.buffersize=5,
xasecure.audit.destination.hdfs.batch.filequeue.filespool.dir=target/spool,
xasecure.audit.destination.hdfs.batch.queuetype=filequeue,
xasecure.audit.destination.hdfs.batch.filequeue.filespool.buffer.size=10,
xasecure.audit.destination.hdfs.batch.filequeue.filetype=orc,
xasecure.audit.is.enabled=true,
xasecure.audit.destination.hdfs.filename.format=%app-type%_ranger_audit.orc,
xasecure.audit.destination.hdfs=enable,
xasecure.audit.destination.hdfs.orc.stripesize=10,
xasecure.audit.destination.hdfs.dir=target/testAuditFileQueueSpoolORC,
xasecure.audit.destination.hdfs.orc.compression=none,
xasecure.audit.destination.hdfs.batch.filequeue.filespool.file.rollover.sec=5
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Support Java 11 and add trino plugin [ranger]
tarekabouzeid commented on PR #106: URL: https://github.com/apache/ranger/pull/106#issuecomment-1911943982 > Hi @esselius ! Your solution is very close to what I need! Unfortunately this is not working for me on 2.4.0 ranger version with 17ojdk. Maybe you have some ideas how I can fix that. I tried build trino plugin with 414 version of trino with the next command: `JAVA_HOME=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home && mvn clean package -DskipTests -P ranger-trino-plugin,-linux(or -mac or -"mac os x") -am -pl agents-audit,agents-common,agents-cred,agents-installer,credentialbuilder,ranger-plugin-classloader,ranger-util,plugin-trino,ranger-trino-plugin-shim` > > ``` > [INFO] Reactor Summary for ranger 2.4.0: > [INFO] > [INFO] ranger . SUCCESS [ 0.897 s] > [INFO] Credential Support . SUCCESS [ 1.974 s] > [INFO] Audit Component SUCCESS [ 2.799 s] > [INFO] ranger-plugin-classloader .. SUCCESS [ 0.906 s] > [INFO] Common library for Plugins . SUCCESS [ 5.663 s] > [INFO] Installer Support Component SUCCESS [ 0.513 s] > [INFO] Credential Builder . SUCCESS [ 1.403 s] > [INFO] Ranger Util SUCCESS [ 1.196 s] > [INFO] Trino Security Plugin .. SUCCESS [ 2.398 s] > [INFO] Trino Security Plugin Shim . SUCCESS [ 1.300 s] > [INFO] > [INFO] BUILD SUCCESS > ``` > > But archive was missing into target directory. Thanks a lot for your ideas! Hi, Did you manage to get it to work with this Trino version? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
respon007 commented on PR #291:
URL: https://github.com/apache/ranger/pull/291#issuecomment-1910220709
> ZooKeeper 3.5 has a transitive dependency on a zookeeper-jute artifact.
./distro/src/main/assembly/plugin-trino.xml should have
"org.apache.zookeeper:zookeeper-jute:jar:${zookeeper.version}"
>
>
> The latest version of Trino supports the update statement. vim
./ranger-trino-plugin-shim/src/main/java/org/apache/ranger/authorization/trino/authorizer/RangerSystemAccessControl.java
>
> 246 //kimdy 247 @OverRide 248 public void
checkCanUpdateTableColumns(SystemSecurityContext securityContext,
CatalogSchemaTableName table, Set updatedColumnNames) { 249 try { 250
activatePluginClassLoader(); 251
systemAccessControlImpl.checkCanUpdateTableColumns(securityContext, table,
updatedColumnNames); 252 } finally { 253 deactivatePluginClassLoader(); 254 }
255 }
>
> vim
./plugin-trino/src/main/java/org/apache/ranger/authorization/trino/authorizer/RangerSystemAccessControl.java
>
> 631 // kimdy 632 @OverRide 633 public void
checkCanUpdateTableColumns(SystemSecurityContext securityContext,
CatalogSchemaTableName table, Set updatedColumnNames){ 634 for
(RangerTrinoResource res : createResource(table, updatedColumnNames)) { 635 if
(!hasPermission(res, securityContext, TrinoAccessType.UPDATE)) { 636
LOG.debug("RangerSystemAccessControl.checkCanUpdateTableColumns(" +
table.getSchemaTableName().getTableName() + ") denied"); 637
AccessDeniedException.denyUpdateTableColumns(table.getSchemaTableName().getTableName(),
updatedColumnNames); 638 } 639 } 640 }
>
> // UDPATE 추가 923 enum TrinoAccessType { 924 CREATE, DROP, SELECT, INSERT,
DELETE, USE, ALTER, ALL, GRANT, REVOKE, SHOW, IMPERSONATE, EXECUTE, UPDATE; 925
}
>
> 3. 나의 빌드 환경
>java -version
>openjdk version "17.0.7" 2023-04-18 LTS
>OpenJDK Runtime Environment Zulu17.42+19-CA (build 17.0.7+7-LTS)
>OpenJDK 64-Bit Server VM Zulu17.42+19-CA (build 17.0.7+7-LTS, mixed
mode, sharing)
>
> mvn -v Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39) Maven
home: /opt/apps/maven Java version: 17.0.7, vendor: Azul Systems, Inc.,
runtime: /opt/apps/zulu17.42.19-ca-jdk17.0.7-linux_x64 Default locale: en_US,
platform encoding: UTF-8 OS name: "linux", version: "3.10.0-1160.el7.x86_64",
arch: "amd64", family: "unix"
>
> mvn clean package -DskipTests -P ranger-trino-plugin,-linux -am -pl
distro,plugin-trino,ranger-trino-plugin-shim,agents-installer,credentialbuilder
>
> 4. Lastly, ranger's group policy doesn't seem to apply well to trino. The
role policy applies well to trino.
> 5. I'm an old school poor IT guy who's not familiar with GIT, so I don't
know what form and procedure I should use to submit a proposal. I hope this was
of some help. Your efforts have been of great help to me. thank you!
@origin0099 I have tried your way,but the "Permissions" had no "Update"
option. Could you help me?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
shreyas-dview commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1909439755 Give me some time over the weekend, will try to resolve the issues (if exists) or will answer them with proper explanation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
respon007 commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1905848596 > @shreyas-dview - this PR includes binary file ranger-3.0.0-SNAPSHOT-trino-plugin.tar.gz. Can you please remove this file and update the PR? > > Would 17 be the minimum JDK version after this patch? Given many existing plugins require JDK8 support, it is critical to be able to build with earlier version JDK as well. I had resolved the problem by maven-tool-chains plugin (https://stackoverflow.com/questions/12414209/compile-maven-module-with-different-java-version), but my ranger version is 2.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
origin0099 commented on PR #291:
URL: https://github.com/apache/ranger/pull/291#issuecomment-1903411043
1.
ZooKeeper 3.5 has a transitive dependency on a zookeeper-jute artifact.
./distro/src/main/assembly/plugin-trino.xml should have
"org.apache.zookeeper:zookeeper-jute:jar:${zookeeper.version}"
2.
The latest version of Treeno supports the update statement.
vim
./ranger-trino-plugin-shim/src/main/java/org/apache/ranger/authorization/trino/authorizer/RangerSystemAccessControl.java
246 //kimdy
247 @Override
248 public void checkCanUpdateTableColumns(SystemSecurityContext
securityContext, CatalogSchemaTableName table, Set updatedColumnNames) {
249 try {
250 activatePluginClassLoader();
251
systemAccessControlImpl.checkCanUpdateTableColumns(securityContext, table,
updatedColumnNames);
252 } finally {
253 deactivatePluginClassLoader();
254 }
255 }
vim
./plugin-trino/src/main/java/org/apache/ranger/authorization/trino/authorizer/RangerSystemAccessControl.java
631 // kimdy
632 @Override
633 public void checkCanUpdateTableColumns(SystemSecurityContext
securityContext, CatalogSchemaTableName table, Set updatedColumnNames){
634 for (RangerTrinoResource res : createResource(table,
updatedColumnNames)) {
635 if (!hasPermission(res, securityContext, TrinoAccessType.UPDATE)) {
636
LOG.debug("RangerSystemAccessControl.checkCanUpdateTableColumns(" +
table.getSchemaTableName().getTableName() + ") denied");
637
AccessDeniedException.denyUpdateTableColumns(table.getSchemaTableName().getTableName(),
updatedColumnNames);
638 }
639 }
640 }
// UDPATE 추가
923 enum TrinoAccessType {
924 CREATE, DROP, SELECT, INSERT, DELETE, USE, ALTER, ALL, GRANT,
REVOKE, SHOW, IMPERSONATE, EXECUTE, UPDATE;
925 }
3. 나의 빌드 환경
java -version
openjdk version "17.0.7" 2023-04-18 LTS
OpenJDK Runtime Environment Zulu17.42+19-CA (build 17.0.7+7-LTS)
OpenJDK 64-Bit Server VM Zulu17.42+19-CA (build 17.0.7+7-LTS, mixed mode,
sharing)
mvn -v
Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39)
Maven home: /opt/apps/maven
Java version: 17.0.7, vendor: Azul Systems, Inc., runtime:
/opt/apps/zulu17.42.19-ca-jdk17.0.7-linux_x64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-1160.el7.x86_64", arch: "amd64", family:
"unix"
mvn clean package -DskipTests -P ranger-trino-plugin,-linux -am -pl
distro,plugin-trino,ranger-trino-plugin-shim,agents-installer,credentialbuilder
4.
I'm an old school IT guy who's not familiar with GIT, so I don't know what
form and procedure I should use to submit a proposal. I hope this was of some
help. Your efforts have been of great help to me. thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
bhavikpatel9977 commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1903129680 Nashron is not available from jdk15+. You can exclude from the build and use graalvm for jdk-17. The dependency is already added in the plugins-common module, you can enable that dependency and try to build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
mneethiraj commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1899216773 > if you look at pom.xml changes I have removed trino-plugin from default and jdk-11 profile , so if anyone wants to build trino-plugin they have to use -Pranger-jdk17 profile @shreyas-dview - build fails with the following error due to changes to agents-common module. Can you please review and update? > [INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ ranger-plugins-common --- > [INFO] Compiling 250 source files to /home/build/Apache/git/ranger/agents-common/target/classes > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] /home/build/Apache/git/ranger/agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java:[26,40] error: cannot access ClassFilter > [INFO] 1 error > [INFO] - -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
shreyas-dview commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1898136958 @mneethiraj removed ranger-3.0.0-SNAPSHOT-trino-plugin.tar.gz, thanks for pointing that out. Would 17 be the minimum JDK version after this patch? -> if you look at pom.xml changes I have removed trino-plugin from default and jdk-11 profile , so if anyone wants to build trino-plugin they have to use -Pranger-jdk17 profile -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
mneethiraj commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1896670319 @shreyas-dview - this PR includes binary file ranger-3.0.0-SNAPSHOT-trino-plugin.tar.gz. Can you please remove this file and update the PR? Would 17 be the minimum JDK version after this patch? Given many existing plugins require JDK8 support, it is critical to be able to build with earlier version JDK as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4675: Using the ozone service id specified in the config instead of hardcoded value for resource lookup [ranger]
kumaab merged PR #297: URL: https://github.com/apache/ranger/pull/297 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4675: Using the ozone service id specified in the config instead of hardcoded value for resource lookup [ranger]
fateh288 opened a new pull request, #297: URL: https://github.com/apache/ranger/pull/297 ## What changes were proposed in this pull request? Using the ozone service id specified in the config instead of hardcoded value for resource lookup ## How was this patch tested? Tested in dev environment. DIfferent values in ozone conf for ozone service id can be used to test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
lozbrown commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1892168147 is this likely to get reviewed/merged soon? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4640: Trino ranger plugin for 433 snapshot [ranger]
origin0099 commented on PR #291: URL: https://github.com/apache/ranger/pull/291#issuecomment-1888336851 Your efforts have been of great help to me. thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Update trino-plugin to be compatible with the latest Trino SPI (version 433) [ranger]
nghiabuilvb commented on PR #290: URL: https://github.com/apache/ranger/pull/290#issuecomment-1884448532 > @okayhooni can you share your maven build command for ranger and trino plugin? I tried to build your fork with JDK 17 but getting errors > > `[ERROR] Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.6:exploded (prepare) on project security-admin-web: Execution prepare of goal org.apache.maven.plugins:maven-war-plugin:2.6:exploded failed: Unable to load the mojo 'exploded' in the plugin 'org.apache.maven.plugins:maven-war-plugin:2.6' due to an API incompatibility: org.codehaus.plexus.component.repository.exception.ComponentLookupException: null` > > I build by the following command: > > `mvn -DskipTests=true clean compile package install` You should build only trino plugin, you can try to run this command: mvn clean package -DskipTests -P ranger-trino-plugin,-linux -am -pl distro,plugin-trino,ranger-trino-plugin-shim,agents-installer,credentialbuilder -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Update trino-plugin to be compatible with the latest Trino SPI (version 433) [ranger]
deebify commented on PR #290: URL: https://github.com/apache/ranger/pull/290#issuecomment-1882389974 @okayhooni can you share your maven build command for ranger and trino plugin? I tried to build your fork with JDK 17 but getting errors `[ERROR] Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.6:exploded (prepare) on project security-admin-web: Execution prepare of goal org.apache.maven.plugins:maven-war-plugin:2.6:exploded failed: Unable to load the mojo 'exploded' in the plugin 'org.apache.maven.plugins:maven-war-plugin:2.6' due to an API incompatibility: org.codehaus.plexus.component.repository.exception.ComponentLookupException: null` I build by the following command: `mvn -DskipTests=true clean compile package install` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] RANGER-4029 Ranger cannot build with HBase 2.5.x versions because pre… [ranger]
Jetiaime commented on PR #207: URL: https://github.com/apache/ranger/pull/207#issuecomment-1876963519 Has it been completely modified? It seems that it has only been partially modified. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Update trino-plugin to be compatible with the latest Trino SPI (version 433) [ranger]
okayhooni commented on PR #290: URL: https://github.com/apache/ranger/pull/290#issuecomment-1868894879 @leeyh0216 Sorry for the late response..! - We built and tested all the packages of Trino and Ranger with JDK 17 - I already committed the fixes on the `ranger-trino-plugin-shim` on this MR (https://github.com/apache/ranger/pull/290/files#diff-afa003f80796be360546cf8a2827aaf0842ca3c6f296554e88ab87f5056107dc) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4626: Support trino to enable kerberos [ranger]
everypp opened a new pull request, #296: URL: https://github.com/apache/ranger/pull/296 ## What changes were proposed in this pull request? issues: https://issues.apache.org/jira/projects/RANGER/issues/RANGER-4626 When trino to enable kerberos, the range Test Connection success ## How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4623:Change the http client of the knox plug-in to avoid conne… [ranger]
upczsh opened a new pull request, #295: URL: https://github.com/apache/ranger/pull/295 …ctivity failure caused by SSL links. ## What changes were proposed in this pull request? The ranger plug-in client does not need SSL handshake when calling knox, so it is replaced with apache http client to solve the SSL handshake problem and adjust the knox interface. The ranger knox test connection error is : Unable to retrieve any files using given parameters, You can still save the repository and start creating policies, but you would not be able to use autocomplete for resource names. Check ranger_admin.log for more info. org.apache.ranger.plugin.client.HadoopException: Exception on REST call to KnoxUrl : https://xxx:8443/gateway/admin/api/v1/topologies.. javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. ## How was this patch tested? Click ranger knox service connectivity test, and pull down to view policy files, etc. You can see it : https://issues.apache.org/jira/projects/RANGER/issues/RANGER-4623?filter=allissues -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Change the http client of the knox plug-in to avoid conne… [ranger]
upczsh closed pull request #293: Change the http client of the knox plug-in to avoid conne… URL: https://github.com/apache/ranger/pull/293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4622:property ranger.accesslog.rotate.rename_on_rotate, the de… [ranger]
upczsh opened a new pull request, #294: URL: https://github.com/apache/ranger/pull/294 …fault value is false, not 15 ## What changes were proposed in this pull request? Change the default value of the configuration ranger.accesslog.rotate.rename_on_rotate in ranger-admin-default-site.xml.    the issuse links is :https://issues.apache.org/jira/projects/RANGER/issues/RANGER-4622?filter=allissues -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] property ranger.accesslog.rotate.rename_on_rotate, the de… [ranger]
upczsh closed pull request #292: property ranger.accesslog.rotate.rename_on_rotate, the de… URL: https://github.com/apache/ranger/pull/292 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Update trino-plugin to be compatible with the latest Trino SPI (version 433) [ranger]
leeyh0216 commented on PR #290: URL: https://github.com/apache/ranger/pull/290#issuecomment-1862864010 Hello, @okayhooni Although we are not doing Ranger Contributing, we are using the same upgraded version of Trino internally. How did you test these changes? I remember that Trino Dependency built with JDK 17 could not be used on the Ranger Admin side (built with JDK8), so I was curious how you got it running. (In our case, Ranger Admin was also built based on JDK 17.) Additionally, there doesn't seem to be any code modification for `ranger-trino-plugin-shim`, but I think it needs to be modified. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4619:Change the http client of the knox plug-in to avoid conne… [ranger]
upczsh opened a new pull request, #293: URL: https://github.com/apache/ranger/pull/293 …ctivity failure caused by SSL links. ## What changes were proposed in this pull request? The ranger knox plug-in will report SSLHandhakeException during the connectivity test. SSL should not be required for the plug-in, so the http client is changed to avoid errors caused by the SSL handshake. The error is reported as follows：   ## How was this patch tested? After changing the http client, the test is as follows:     -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] RANGER-4618:property ranger.accesslog.rotate.rename_on_rotate, the de… [ranger]
upczsh opened a new pull request, #292: URL: https://github.com/apache/ranger/pull/292 …fault value is false, not 15 ## What changes were proposed in this pull request? Fix the default value of ranger.accesslog.rotate.rename_on_rotate configuration in ranger--admin-default-site.xml. The default value of this configuration is false instead of 15.   ## How was this patch tested? (Please explain how this patch was tested. Ex: unit tests, manual tests) (If this patch involves UI changes, please attach a screen-shot; otherwise, remove this) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Trino ranger plugin for 433 snapshot [ranger]
shreyas-dview opened a new pull request, #291: URL: https://github.com/apache/ranger/pull/291 ## What changes were proposed in this pull request? Will create a JIRA once I get the access. Changes made to support Trino Ranger Plugin with 433 version. Also made ranger compatible by adding nashorn-core to agent-commons to support build it with java 11 and java 17. ## How was this patch tested? Yes this is been deployed with our prod env and is working fine for more than an month now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Update trino-plugin to be compatibe with the latest Trino SPI (version 433) [ranger]
okayhooni opened a new pull request, #290: URL: https://github.com/apache/ranger/pull/290 ## What changes were proposed in this pull request? - Update trino-plugin to be compatible with the latest Trino SPI (version 433) - I hope it helps someone in trouble with Trino SPI interface changing too fast.. - Related issue: https://github.com/aakashnand/trino-ranger-demo/issues/7 ## How was this patch tested? - I tested this updated plugin with Trino v433(latest) and Ranger v2.4.0 - I guess it will work well with the higher version of Ranger(it may also v3.0.0..?) @aakashnand , would you please review this update..? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [RANGER-4307] Allow users customize the time interval [ranger]
yanghongwei655 commented on PR #269: URL: https://github.com/apache/ranger/pull/269#issuecomment-1817417462 This PR is very practical and helpful to improve development efficiency, thanks for sharing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [RANGER-4307] Allow users customize the time interval [ranger]
qinping416 commented on PR #269: URL: https://github.com/apache/ranger/pull/269#issuecomment-1817413769 This pull request can improve the production efficiency of enterprises and save costs for enterprises -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [RANGER-4307] Allow users customize the time interval [ranger]
zhutong6688 commented on PR #269: URL: https://github.com/apache/ranger/pull/269#issuecomment-1817048269 This PR can greatly improve the production efficiency of enterprises and save a lot of costs for them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
