[
https://issues.apache.org/jira/browse/SPARK-61991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-61991:
------------------------------------
Assignee: Hyukjin Kwon
> Check the session_id of the connection string as PySpark's
> uuid.UUID(session_id, version=4), with its messages
> --------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61991
> URL: https://issues.apache.org/jira/browse/SPARK-61991
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: 5.0.0
> Reporter: Hyukjin Kwon
> Assignee: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
>
> Problem
> PySpark's ChannelBuilder.session_id (connect/client/core.py:438-447) checks
> the session_id of
> the connection string with Python's uuid.UUID(session_id, version=4). That
> takes any text which,
> with every "urn:" and "uuid:" removed, the braces stripped from both ends and
> every "-" removed,
> is 32 characters that int(hex, 16) reads, such as "uuid:<uuid>",
> "urn:<uuid>", braces around the
> 32 digits, or hyphens anywhere. Its error is INVALID_SESSION_UUID_ID of the
> text of the ValueError,
> such as "badly formed hexadecimal UUID string". The session ID is then sent
> as given, and the
> server refuses those other forms as a handle at the first request.
> The core (crates/spark-connect-core/src/channel.rs,
> ChannelBuilder::session_id) checked it with
> the uuid crate's Uuid::parse_str. SparkSession.builder.remote(...).create()
> and the Rust API take
> that path. It refused some of the forms that Python's uuid reads, with the
> origin "invalid UUID".
> spark.client's ChannelBuilder is Python's and was already right.
> Repro (u = "550e8400-e29b-41d4-a716-446655440000")
>
> SparkSession.builder.remote(f"sc://localhost:15002/;session_id=uuid:{u}").create()
>
> SparkSession.builder.remote("sc://localhost:15002/;session_id={550e8400e29b41d4a716446655440000}").create()
> SparkSession.builder.remote("sc://localhost:15002/;session_id=x").create()
> Actual (drop-in):
> - PySparkValueError [INVALID_SESSION_UUID_ID] Parameter value session_id must
> be a valid UUID
> format: invalid UUID, at create(), for all three.
> Expected (pyspark-client 4.3.0):
> - the first two create the session; its first request fails with the server's
> INVALID_HANDLE.FORMAT;
> - the third raises "... must be a valid UUID format: badly formed hexadecimal
> UUID string".
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]