deepakpanda93 commented on issue #19589:
URL: https://github.com/apache/hudi/issues/19589#issuecomment-5358667598
Confirmed, and it is broader than the title suggests — this affects **all
Spark 3.x bundles in 1.2.0**, and it is a regression from 1.1.1.
**Verified against the published artifact** (SHA1 checked against Maven
Central's `.sha1`, so this is the exact released bytes):
```python
import zipfile, struct, collections, urllib.request
u = ("https://repo1.maven.org/maven2/org/apache/hudi/";
"hudi-spark3.3-bundle_2.12/1.2.0/hudi-spark3.3-bundle_2.12-1.2.0.jar")
urllib.request.urlretrieve(u, "b.jar")
z = zipfile.ZipFile("b.jar"); d = collections.Counter(); hir = None
for n in z.namelist():
if not n.endswith(".class"): continue
b = z.open(n).read(8)
if len(b) < 8 or b[:4] != b'\xca\xfe\xba\xbe': continue
m = struct.unpack(">HH", b[4:8])[1]; d[m] += 1
if n.endswith("client/model/HoodieInternalRow.class"): hir = m
print(sorted(d.items())); print("HoodieInternalRow major:", hir)
```
Output:
```
[(48, 30), (49, 37), (50, 2214), (51, 157), (52, 7866), (53, 1), (55, 1418)]
HoodieInternalRow major: 55
```
**1,418 classes at major 55 (Java 11)**, including
`org/apache/hudi/client/model/HoodieInternalRow` — exactly the class and
version in the reported `UnsupportedClassVersionError`.
| bundle | Java-11 classes | `HoodieInternalRow` |
|---|---|---|
| `hudi-spark3.3-bundle_2.12-1.1.1` | 0 | major 52 |
| `hudi-spark3.5-bundle_2.12-1.1.1` | 0 | major 52 |
| `hudi-spark3.3-bundle_2.12-1.2.0` | 1,418 | **major 55** |
| `hudi-spark3.4-bundle_2.12-1.2.0` | 1,418 | **major 55** |
| `hudi-spark3.5-bundle_2.12-1.2.0` | 1,418 | **major 55** |
**Root cause.** The root `pom.xml` raised `<java.version>` from `8` (1.1.1)
to `11` (1.2.0). Only `hudi-common`, `hudi-io` and `hudi-hadoop-common` carry
an explicit `<release>8</release>` pin. `hudi-client/hudi-spark-client` — which
owns `HoodieInternalRow` — inherits 11, as do `hudi-client-common`,
`hudi-spark-common`, `hudi-aws` and `hudi-sync-common`. The result is a
single jar with `org/apache/hudi/common` at Java 8 (752 classes) and
`org/apache/hudi/client` at Java 11 (155 classes), loaded by one classloader.
**Why this looks like a defect rather than an intentional Java 8 drop:** the
1.2.0 release notes, under *"Java 11 is the Default Build/Runtime"*, state that
*"Spark 3.x bundles continue to support Java 8+."* The shipped Spark 3.x
artifacts do not.
It also fails unconditionally rather than on some code path — the class
loads during `SparkAdapterSupport` initialisation at `SparkSession` creation,
before any Hudi table is touched, which matches the reported stack.
Can you try running the Spark 3.x bundle on Java 11, or stay on 1.1.1 for
Java 8.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
