yuqi1129 commented on code in PR #8102:
URL: https://github.com/apache/gravitino/pull/8102#discussion_r2332783726
##########
core/src/main/java/org/apache/gravitino/cache/CaffeineEntityCache.java:
##########
@@ -408,12 +413,17 @@ private boolean invalidateEntities(
* @param action The action to run with the lock
*/
private void withLock(Runnable action) {
+ segmentedLock.withAllLocks(action);
Review Comment:
Why do we need to get all locks here to operate the action?
##########
server-common/src/main/java/org/apache/gravitino/server/authorization/jcasbin/JcasbinAuthorizer.java:
##########
@@ -76,6 +81,21 @@ public class JcasbinAuthorizer implements
GravitinoAuthorizer {
*/
private Set<Long> loadedRoles = ConcurrentHashMap.newKeySet();
+ /**
+ * loadedOwners is used to cache owners that have loaded permissions. When
the permissions of a
+ * role are updated, they should be removed from it.
+ */
+ private Set<Long> loadedOwners = ConcurrentHashMap.newKeySet();
+
+ private static Executor executor =
+ Executors.newFixedThreadPool(
+ 50,
Review Comment:
50 is not a nice number. Please use the CPU cores to set it exactly.
##########
core/src/main/java/org/apache/gravitino/cache/SegmentedLock.java:
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.cache;
+
+import com.google.common.base.Preconditions;
+import java.util.concurrent.locks.ReentrantLock;
+import org.apache.gravitino.NameIdentifier;
+
+/**
+ * A segmented lock manager that reduces lock contention by distributing locks
across multiple
+ * segments based on the hash code of a {@link NameIdentifier}.
+ *
+ * <p>Instead of using a single global lock, this class uses an array of
{@link ReentrantLock}
+ * instances. Each identifier is mapped to a specific lock using its hash code
modulo the number of
+ * segments. This allows concurrent operations on identifiers that map to
different segments,
+ * improving overall throughput in high-concurrency scenarios.
+ *
+ * <p>This implementation is thread-safe and immutable after construction. The
number of segments
+ * remains fixed for the lifetime of the instance.
+ */
+public class SegmentedLock {
+ private final ReentrantLock[] locks;
+
+ public SegmentedLock(int numSegments) {
+ locks = new ReentrantLock[numSegments];
+ for (int i = 0; i < numSegments; i++) {
+ locks[i] = new ReentrantLock();
+ }
+ }
+
+ /**
+ * Returns the lock associated with the given identifier. The lock is
determined by computing the
+ * hash code of the identifier and mapping it to a segment using modulo
arithmetic.
+ *
+ * <p>Note: {@link NameIdentifier#hashCode()} is assumed to provide a
reasonably uniform
+ * distribution to ensure balanced lock utilization.
+ *
+ * @param ident the identifier to map to a lock; must not be null
+ * @return the {@link ReentrantLock} associated with the identifier
+ */
+ public ReentrantLock getLock(NameIdentifier ident) {
Review Comment:
To make it more general, I believe the parameter type can be an object.
##########
core/src/main/java/org/apache/gravitino/cache/SegmentedLock.java:
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.cache;
+
+import com.google.common.base.Preconditions;
+import java.util.concurrent.locks.ReentrantLock;
+import org.apache.gravitino.NameIdentifier;
+
+/**
+ * A segmented lock manager that reduces lock contention by distributing locks
across multiple
+ * segments based on the hash code of a {@link NameIdentifier}.
+ *
+ * <p>Instead of using a single global lock, this class uses an array of
{@link ReentrantLock}
+ * instances. Each identifier is mapped to a specific lock using its hash code
modulo the number of
+ * segments. This allows concurrent operations on identifiers that map to
different segments,
+ * improving overall throughput in high-concurrency scenarios.
+ *
+ * <p>This implementation is thread-safe and immutable after construction. The
number of segments
+ * remains fixed for the lifetime of the instance.
+ */
+public class SegmentedLock {
+ private final ReentrantLock[] locks;
+
+ public SegmentedLock(int numSegments) {
+ locks = new ReentrantLock[numSegments];
+ for (int i = 0; i < numSegments; i++) {
+ locks[i] = new ReentrantLock();
Review Comment:
I believe we can initialize the lock when we attempt to fetch them, and
there is no need to create them initially.
##########
core/src/main/java/org/apache/gravitino/cache/SegmentedLock.java:
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.cache;
+
+import com.google.common.base.Preconditions;
+import java.util.concurrent.locks.ReentrantLock;
+import org.apache.gravitino.NameIdentifier;
+
+/**
+ * A segmented lock manager that reduces lock contention by distributing locks
across multiple
+ * segments based on the hash code of a {@link NameIdentifier}.
+ *
+ * <p>Instead of using a single global lock, this class uses an array of
{@link ReentrantLock}
+ * instances. Each identifier is mapped to a specific lock using its hash code
modulo the number of
+ * segments. This allows concurrent operations on identifiers that map to
different segments,
+ * improving overall throughput in high-concurrency scenarios.
+ *
+ * <p>This implementation is thread-safe and immutable after construction. The
number of segments
+ * remains fixed for the lifetime of the instance.
+ */
+public class SegmentedLock {
Review Comment:
What's the difference between segment lock that strip<lock> in PR
https://github.com/apache/gravitino/pull/8452
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
