Hyukjin Kwon created SPARK-61991:
------------------------------------

             Summary: Check the session_id of the connection string as 
PySpark's uuid.UUID(session_id, version=4), with its messages
                 Key: SPARK-61991
                 URL: https://issues.apache.org/jira/browse/SPARK-61991
             Project: Spark
          Issue Type: Bug
          Components: Connect
    Affects Versions: 5.0.0
            Reporter: Hyukjin Kwon


Problem
PySpark's ChannelBuilder.session_id (connect/client/core.py:438-447) checks the 
session_id of
the connection string with Python's uuid.UUID(session_id, version=4). That 
takes any text which,
with every "urn:" and "uuid:" removed, the braces stripped from both ends and 
every "-" removed,
is 32 characters that int(hex, 16) reads, such as "uuid:<uuid>", "urn:<uuid>", 
braces around the
32 digits, or hyphens anywhere. Its error is INVALID_SESSION_UUID_ID of the 
text of the ValueError,
such as "badly formed hexadecimal UUID string". The session ID is then sent as 
given, and the
server refuses those other forms as a handle at the first request.

The core (crates/spark-connect-core/src/channel.rs, ChannelBuilder::session_id) 
checked it with
the uuid crate's Uuid::parse_str. SparkSession.builder.remote(...).create() and 
the Rust API take
that path. It refused some of the forms that Python's uuid reads, with the 
origin "invalid UUID".
spark.client's ChannelBuilder is Python's and was already right.

Repro (u = "550e8400-e29b-41d4-a716-446655440000")
    
SparkSession.builder.remote(f"sc://localhost:15002/;session_id=uuid:{u}").create()
    
SparkSession.builder.remote("sc://localhost:15002/;session_id={550e8400e29b41d4a716446655440000}").create()
    SparkSession.builder.remote("sc://localhost:15002/;session_id=x").create()
Actual (drop-in):
- PySparkValueError [INVALID_SESSION_UUID_ID] Parameter value session_id must 
be a valid UUID
  format: invalid UUID, at create(), for all three.
Expected (pyspark-client 4.3.0):
- the first two create the session; its first request fails with the server's
  INVALID_HANDLE.FORMAT;
- the third raises "... must be a valid UUID format: badly formed hexadecimal 
UUID string".




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to