This is an automated email from the ASF dual-hosted git repository.
garydgregory pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-lang.git
The following commit(s) were added to refs/heads/master by this push:
new 9c86ffe30 Clarify StringUtils.normalizeSpace contract and preserve
NBSP handling
9c86ffe30 is described below
commit 9c86ffe308e7c682df3b1cf76f27dfc5bb6a2243
Author: Gary Gregory <[email protected]>
AuthorDate: Fri Oct 16 17:19:23 2026 -0400
Clarify StringUtils.normalizeSpace contract and preserve NBSP handling
Document XPath normalize-space as the inspiration rather than an exact
behavioral contract. Explain Character.isWhitespace semantics, explicit
U+00A0 conversion, preservation of other non-breaking spaces, and final
trimming. Clarify that adjacent converted NBSPs are not collapsed.
The method was introduced for LANG-640 in commit 823f2603a. LANG-910
added NBSP support to the regex implementation in f9b486a17, released
in Commons Lang 3.2. LANG-1020 introduced the current manual whitespace
normalization and per-character NBSP conversion in bc8e23808 (PR #27).
LANG-1226 restored final trimming in 7e85d1cf5 (PR #150).
PR #1809 proposes removing NBSP conversion. Preserve the established
behavior here and resolve its mismatch with the old Javadoc explicitly.
Add regression tests for standalone, boundary, repeated, and mixed
NBSPs, other non-breaking spaces, and Unicode whitespace.
Validation: three focused tests passed; Javadoc generation with full
doclint and git diff --check passed.
References:
https://issues.apache.org/jira/browse/LANG-640
https://issues.apache.org/jira/browse/LANG-910
https://issues.apache.org/jira/browse/LANG-1020
https://issues.apache.org/jira/browse/LANG-1226
https://github.com/apache/commons-lang/pull/27
https://github.com/apache/commons-lang/pull/150
https://github.com/apache/commons-lang/pull/1809
---
.../java/org/apache/commons/lang3/StringUtils.java | 42 ++++++---------
.../org/apache/commons/lang3/StringUtilsTest.java | 62 ++++++++++++++--------
2 files changed, 57 insertions(+), 47 deletions(-)
diff --git a/src/main/java/org/apache/commons/lang3/StringUtils.java
b/src/main/java/org/apache/commons/lang3/StringUtils.java
index aaf30c3cc..832a063d1 100644
--- a/src/main/java/org/apache/commons/lang3/StringUtils.java
+++ b/src/main/java/org/apache/commons/lang3/StringUtils.java
@@ -5484,37 +5484,27 @@ public static String mid(final String str, int pos,
final int len) {
}
/**
- * Similar to <a
href="https://www.w3.org/TR/xpath/#function-normalize-space";>https://www.w3.org/TR/xpath/#function-normalize
-space</a>
+ * Normalizes whitespace in a String.
*
- * <p>
- * This function returns the argument string with whitespace normalized by
using {@code {@link #trim(String)}} to remove leading and trailing whitespace
and
- * then replacing sequences of whitespace characters by a single space.
- * </p>
- * In XML, whitespace characters are the same as those allowed by the <a
href="https://www.w3.org/TR/REC-xml/#NT-S";>S</a> production, which is S ::=
(#x20 |
- * #x9 | #xD | #xA)+
- * <p>
- * Java's regexp pattern \s defines whitespace as [ \t\n\x0B\f\r]
- * </p>
- * <p>
- * For reference:
- * </p>
- * <ul>
- * <li>\x0B = vertical tab</li>
- * <li>\f = #xC = form feed</li>
- * <li>#x20 = space</li>
- * <li>#x9 = \t</li>
- * <li>#xA = \n</li>
- * <li>#xD = \r</li>
- * </ul>
+ * <p>Removes leading and trailing whitespace and replaces each remaining
sequence of characters recognized by {@link Character#isWhitespace(char)}
+ * with a single ordinary space ({@code U+0020}). This includes Unicode
whitespace beyond the characters matched by the default regular expression
+ * {@code \s}, such as EM SPACE ({@code U+2003}).</p>
*
- * <p>
- * The difference is that Java's whitespace includes vertical tab and form
feed, which this function will also normalize. Additionally {@code {@link
- * #trim(String)}} removes control characters (char <= 32) from both
ends of this String.
- * </p>
+ * <p>Additionally, converts each non-breaking space ({@code U+00A0}) to
an ordinary space, even though {@link Character#isWhitespace(char)} excludes it.
+ * These conversions do not collapse adjacent spaces: for example,
<code>"a\u00A0\u00A0b"</code> becomes {@code "a b"}. Finally,
+ * {@link String#trim()} removes
+ * leading and trailing characters whose values are at most {@code
U+0020}, including converted non-breaking spaces. Other non-breaking spaces,
such as
+ * FIGURE SPACE ({@code U+2007}) and NARROW NO-BREAK SPACE ({@code
U+202F}), are preserved.</p>
+ *
+ * <p>This method was inspired by XPath's
+ * <a
href="https://www.w3.org/TR/xpath/#function-normalize-space";>normalize-space</a>
function, but does not reproduce its behavior.
+ * XPath normalizes only XML
+ * whitespace ({@code U+0020}, {@code U+0009}, {@code U+000D}, and {@code
U+000A}) and preserves non-breaking spaces. This method additionally normalizes
+ * Java whitespace, converts {@code U+00A0}, and trims leading and
trailing control characters as described above.</p>
*
* @param str The source String to normalize whitespaces from, may be null.
* @return The modified string with whitespace normalized, {@code null} if
null String input.
- * @see Pattern
+ * @see Character#isWhitespace(char)
* @see #trim(String)
* @see <a
href="https://www.w3.org/TR/xpath/#function-normalize-space";>https://www.w3.org/TR/xpath/#function-normalize-space</a>
* @since 3.0
diff --git a/src/test/java/org/apache/commons/lang3/StringUtilsTest.java
b/src/test/java/org/apache/commons/lang3/StringUtilsTest.java
index df4bae511..21e59b617 100644
--- a/src/test/java/org/apache/commons/lang3/StringUtilsTest.java
+++ b/src/test/java/org/apache/commons/lang3/StringUtilsTest.java
@@ -141,6 +141,26 @@ public String toString() {
private static final boolean[] ARRAY_FALSE_TRUE = {false, true};
private static final boolean[] ARRAY_FALSE_TRUE_FALSE = {false, true,
false};
+ private static void assertNoLoneSurrogates(final String result) {
+ for (int i = 0; i < result.length(); i++) {
+ final char ch = result.charAt(i);
+ if (Character.isHighSurrogate(ch)) {
+ assertTrue(i + 1 < result.length() &&
Character.isLowSurrogate(result.charAt(i + 1)),
+ "lone high surrogate in: " + result);
+ i++; // skip the paired low surrogate
+ } else {
+ assertFalse(Character.isLowSurrogate(ch), "lone low surrogate
in: " + result);
+ }
+ }
+ }
+
+ static Stream<TriFunction<String, Integer, String, String>>
testPadSurrogatePair() {
+ return Stream.of(
+ StringUtils::center,
+ StringUtils::leftPad,
+ StringUtils::rightPad);
+ }
+
private void innerTestSplit(final char separator, final String sepStr,
final char noMatch) {
final String msg = "Failed on separator hex(" +
Integer.toHexString(separator) +
"), noMatch hex(" + Integer.toHexString(noMatch) + "),
sepStr(" + sepStr + ")";
@@ -226,19 +246,6 @@ private void innerTestSplitPreserveAllTokens(final char
separator, final String
assertEquals(str.substring(2), res[1], msg);
}
- private static void assertNoLoneSurrogates(final String result) {
- for (int i = 0; i < result.length(); i++) {
- final char ch = result.charAt(i);
- if (Character.isHighSurrogate(ch)) {
- assertTrue(i + 1 < result.length() &&
Character.isLowSurrogate(result.charAt(i + 1)),
- "lone high surrogate in: " + result);
- i++; // skip the paired low surrogate
- } else {
- assertFalse(Character.isLowSurrogate(ch), "lone low surrogate
in: " + result);
- }
- }
- }
-
/**
* Tests {@code appendIfMissing}.
*/
@@ -1444,7 +1451,7 @@ void testLowerCase() {
@Test
void testNormalizeSpace() {
- // Java says a non-breaking whitespace is not a whitespace.
+ // Character.isWhitespace excludes non-breaking space; normalizeSpace
converts it explicitly.
assertFalse(Character.isWhitespace('\u00A0'));
//
assertNull(StringUtils.normalizeSpace(null));
@@ -1469,6 +1476,26 @@ void testNormalizeSpace() {
assertEquals("b", StringUtils.normalizeSpace("b\u0000"));
}
+ @Test
+ void testNormalizeSpaceNonBreakingSpace() {
+ assertEquals("", StringUtils.normalizeSpace("\u00A0"));
+ assertEquals("", StringUtils.normalizeSpace("\u00A0\u00A0"));
+ assertEquals("a", StringUtils.normalizeSpace("\u00A0a"));
+ assertEquals("a", StringUtils.normalizeSpace("a\u00A0"));
+ assertEquals("a", StringUtils.normalizeSpace(" \u00A0a\u00A0 "));
+ assertEquals("a b", StringUtils.normalizeSpace("a\u00A0b"));
+ assertEquals("a b", StringUtils.normalizeSpace("a\u00A0\u00A0b"));
+ assertEquals("a b", StringUtils.normalizeSpace("a\u00A0 b"));
+ assertEquals("a b", StringUtils.normalizeSpace("a \u00A0b"));
+ assertEquals("\u2007a\u2007",
StringUtils.normalizeSpace("\u2007a\u2007"));
+ assertEquals("\u202Fa\u202F",
StringUtils.normalizeSpace("\u202Fa\u202F"));
+ }
+
+ @Test
+ void testNormalizeSpaceUnicodeWhitespace() {
+ assertEquals("a b",
StringUtils.normalizeSpace("\u2003a\u2003\u2003b\u2003"));
+ }
+
@Test
void testOverlay_StringStringIntInt() {
assertNull(StringUtils.overlay(null, null, 2, 4));
@@ -1541,13 +1568,6 @@ void testPadMixedSurrogatePair() {
assertEquals("a" + grin + "xa" + grin, StringUtils.center("x", 7,
pad));
}
- static Stream<TriFunction<String, Integer, String, String>>
testPadSurrogatePair() {
- return Stream.of(
- StringUtils::center,
- StringUtils::leftPad,
- StringUtils::rightPad);
- }
-
@ParameterizedTest
@MethodSource
void testPadSurrogatePair(final TriFunction<String, Integer, String,
String> padFunction) {
