This is an automated email from the ASF dual-hosted git repository.
yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 157b1e3ba5d5 [SPARK-48612][SQL][SS] Cleanup deprecated api usage
related to commons-pool2
157b1e3ba5d5 is described below
commit 157b1e3ba5d5d5e75eb79805eaa3ea14fa876f5b
Author: yangjie01 <[email protected]>
AuthorDate: Fri Jun 14 12:36:22 2024 +0800
[SPARK-48612][SQL][SS] Cleanup deprecated api usage related to commons-pool2
### What changes were proposed in this pull request?
This pr make the following changes
- o.a.c.pool2.impl.BaseObjectPoolConfig#setMinEvictableIdleTime ->
o.a.c.pool2.impl.BaseObjectPoolConfig#setMinEvictableIdleDuration
- o.a.c.pool2.impl.BaseObjectPoolConfig#setSoftMinEvictableIdleTime ->
o.a.c.pool2.impl.BaseObjectPoolConfig#setSoftMinEvictableIdleDuration
to fix the following compilation warnings related to 'commons-pool2':
```
[WARNING] [Warn]
/Users/yangjie01/SourceCode/git/spark-mine-13/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/consumer/InternalKafkaConsumerPool.scala:186:
method setMinEvictableIdleTime in class BaseObjectPoolConfig is deprecated
Applicable -Wconf / nowarn filters for this warning: msg=<part of the
message>, cat=deprecation,
site=org.apache.spark.sql.kafka010.consumer.InternalKafkaConsumerPool.PoolConfig.init,
origin=org.apache.commons.pool2.impl.BaseObjectPoolConfig.setMinEvictableIdleTime
[WARNING] [Warn]
/Users/yangjie01/SourceCode/git/spark-mine-13/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/consumer/InternalKafkaConsumerPool.scala:187:
method setSoftMinEvictableIdleTime in class BaseObjectPoolConfig is deprecated
Applicable -Wconf / nowarn filters for this warning: msg=<part of the
message>, cat=deprecation,
site=org.apache.spark.sql.kafka010.consumer.InternalKafkaConsumerPool.PoolConfig.init,
origin=org.apache.commons.pool2.impl.BaseObjectPoolConfig.setSoftMinEvictableIdleTime
```
The fix refers to:
-
https://github.com/apache/commons-pool/blob/e5c44f5184a55a58fef4a1efec8124d162a348bd/src/main/java/org/apache/commons/pool2/impl/BaseObjectPoolConfig.java#L765-L789
-
https://github.com/apache/commons-pool/blob/e5c44f5184a55a58fef4a1efec8124d162a348bd/src/main/java/org/apache/commons/pool2/impl/BaseObjectPoolConfig.java#L815-L839
```java
/**
* Sets the value for the {code minEvictableIdleTime} configuration
attribute for pools created with this configuration instance.
*
* param minEvictableIdleTime The new setting of {code
minEvictableIdleTime} for this configuration instance
* see GenericObjectPool#getMinEvictableIdleDuration()
* see GenericKeyedObjectPool#getMinEvictableIdleDuration()
* since 2.10.0
* deprecated Use {link #setMinEvictableIdleDuration(Duration)}.
*/
Deprecated
public void setMinEvictableIdleTime(final Duration
minEvictableIdleTime) {
this.minEvictableIdleDuration =
PoolImplUtils.nonNull(minEvictableIdleTime, DEFAULT_MIN_EVICTABLE_IDLE_TIME);
}
/**
* Sets the value for the {code minEvictableIdleTime} configuration
attribute for pools created with this configuration instance.
*
* param minEvictableIdleTime The new setting of {code
minEvictableIdleTime} for this configuration instance
* see GenericObjectPool#getMinEvictableIdleDuration()
* see GenericKeyedObjectPool#getMinEvictableIdleDuration()
* since 2.12.0
*/
public void setMinEvictableIdleDuration(final Duration
minEvictableIdleTime) {
this.minEvictableIdleDuration =
PoolImplUtils.nonNull(minEvictableIdleTime, DEFAULT_MIN_EVICTABLE_IDLE_TIME);
}
/**
* Sets the value for the {code softMinEvictableIdleTime} configuration
attribute for pools created with this configuration instance.
*
* param softMinEvictableIdleTime The new setting of {code
softMinEvictableIdleTime} for this configuration instance
* see GenericObjectPool#getSoftMinEvictableIdleDuration()
* see GenericKeyedObjectPool#getSoftMinEvictableIdleDuration()
* since 2.10.0
* deprecated Use {link #setSoftMinEvictableIdleDuration(Duration)}.
*/
Deprecated
public void setSoftMinEvictableIdleTime(final Duration
softMinEvictableIdleTime) {
this.softMinEvictableIdleDuration =
PoolImplUtils.nonNull(softMinEvictableIdleTime,
DEFAULT_SOFT_MIN_EVICTABLE_IDLE_TIME);
}
/**
* Sets the value for the {code softMinEvictableIdleTime} configuration
attribute for pools created with this configuration instance.
*
* param softMinEvictableIdleTime The new setting of {code
softMinEvictableIdleTime} for this configuration instance
* see GenericObjectPool#getSoftMinEvictableIdleDuration()
* see GenericKeyedObjectPool#getSoftMinEvictableIdleDuration()
* since 2.12.0
*/
public void setSoftMinEvictableIdleDuration(final Duration
softMinEvictableIdleTime) {
this.softMinEvictableIdleDuration =
PoolImplUtils.nonNull(softMinEvictableIdleTime,
DEFAULT_SOFT_MIN_EVICTABLE_IDLE_TIME);
}
```
### Why are the changes needed?
Cleanup deprecated api usage related to `commons-pool2`
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass GitHub Actions
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #46967 from LuciferYang/commons-pool2-deprecated.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
---
.../spark/sql/kafka010/consumer/InternalKafkaConsumerPool.scala | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/consumer/InternalKafkaConsumerPool.scala
b/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/consumer/InternalKafkaConsumerPool.scala
index 661fe731b97b..edd5121cfbee 100644
---
a/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/consumer/InternalKafkaConsumerPool.scala
+++
b/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/consumer/InternalKafkaConsumerPool.scala
@@ -183,8 +183,8 @@ private[consumer] object InternalKafkaConsumerPool {
setMaxTotal(-1)
// Set minimum evictable idle time which will be referred from evictor
thread
- setMinEvictableIdleTime(Duration.ofMillis(minEvictableIdleTimeMillis))
-
setSoftMinEvictableIdleTime(BaseObjectPoolConfig.DEFAULT_SOFT_MIN_EVICTABLE_IDLE_DURATION)
+
setMinEvictableIdleDuration(Duration.ofMillis(minEvictableIdleTimeMillis))
+
setSoftMinEvictableIdleDuration(BaseObjectPoolConfig.DEFAULT_SOFT_MIN_EVICTABLE_IDLE_DURATION)
// evictor thread will run test with ten idle objects
setTimeBetweenEvictionRuns(Duration.ofMillis(evictorThreadRunIntervalMillis))
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
