This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 442fdb8be42 [SPARK-43476][PYTHON][TESTS] Enable
SeriesStringTests.test_string_replace for pandas 2.0.0
442fdb8be42 is described below
commit 442fdb8be42789d9a3fac8361f339f4e2a304fb8
Author: panbingkun <[email protected]>
AuthorDate: Mon Jul 3 15:30:03 2023 +0900
[SPARK-43476][PYTHON][TESTS] Enable SeriesStringTests.test_string_replace
for pandas 2.0.0
### What changes were proposed in this pull request?
The pr aims to enable SeriesStringTests.test_string_replace for pandas
2.0.0.
### Why are the changes needed?
Improve UT coverage.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
- Pass GA.
- Manually test:
'''
(base) panbingkun:~/Developer/spark/spark-community$python/run-tests
--testnames 'pyspark.pandas.tests.test_series_string
SeriesStringTests.test_string_replace'
Running PySpark tests. Output is in
/Users/panbingkun/Developer/spark/spark-community/python/unit-tests.log
Will test against the following Python executables: ['python3.9']
Will test the following Python tests:
['pyspark.pandas.tests.test_series_string
SeriesStringTests.test_string_replace']
python3.9 python_implementation is CPython
python3.9 version is: Python 3.9.13
Starting test(python3.9): pyspark.pandas.tests.test_series_string
SeriesStringTests.test_string_replace (temp output:
/Users/panbingkun/Developer/spark/spark-community/python/target/d51a913a-b400-4d1b-adb3-97765bb463bd/python3.9__pyspark.pandas.tests.test_series_string_SeriesStringTests.test_string_replace__izk1fx8o.log)
Finished test(python3.9): pyspark.pandas.tests.test_series_string
SeriesStringTests.test_string_replace (13s)
Tests passed in 13 seconds
'''
Closes #41823 from panbingkun/SPARK-43476.
Authored-by: panbingkun <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/pandas/tests/test_series_string.py | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/python/pyspark/pandas/tests/test_series_string.py
b/python/pyspark/pandas/tests/test_series_string.py
index 3c2bd58da1a..956567bc5a4 100644
--- a/python/pyspark/pandas/tests/test_series_string.py
+++ b/python/pyspark/pandas/tests/test_series_string.py
@@ -246,10 +246,6 @@ class SeriesStringTestsMixin:
with self.assertRaises(TypeError):
self.check_func(lambda x: x.str.repeat(repeats=[0, 1, 2, 3, 4, 5,
6, 7, 8, 9]))
- @unittest.skipIf(
- LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),
- "TODO(SPARK-43476): Enable SeriesStringTests.test_string_replace for
pandas 2.0.0.",
- )
def test_string_replace(self):
self.check_func(lambda x: x.str.replace("a.", "xx", regex=True))
self.check_func(lambda x: x.str.replace("a.", "xx", regex=False))
@@ -259,10 +255,11 @@ class SeriesStringTestsMixin:
def repl(m):
return m.group(0)[::-1]
- self.check_func(lambda x: x.str.replace(r"[a-z]+", repl))
+ regex_pat = re.compile(r"[a-z]+")
+ self.check_func(lambda x: x.str.replace(regex_pat, repl, regex=True))
# compiled regex with flags
regex_pat = re.compile(r"WHITESPACE", flags=re.IGNORECASE)
- self.check_func(lambda x: x.str.replace(regex_pat, "---"))
+ self.check_func(lambda x: x.str.replace(regex_pat, "---", regex=True))
def test_string_rfind(self):
self.check_func(lambda x: x.str.rfind("a"))
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
