Messages by Thread
-
[jira] [Updated] (SPARK-62092) LimitPushDown drops the top-N guarantee of ORDER BY ... LIMIT when it pushes a smaller limit into it
ASF GitHub Bot (Jira) via issues
-
[jira] [Updated] (SPARK-62089) Make the messages of the Rust protobuf runtime of pyspark.sql.connect.proto, and of the plugins' generated modules, behave as upb's where protobuf is installed, without requiring it
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62093) Add structured OBJECT type
Serge Rielau (Jira) via issues
-
[jira] [Created] (SPARK-62092) LimitPushDown drops the top-N guarantee of ORDER BY ... LIMIT when it pushes a smaller limit into it
Luka Zdravic (Jira) via issues
-
[jira] [Updated] (SPARK-60664) Remap metadata columns in runtime group filters of group-based UPDATE with subqueries
Dongjoon Hyun (Jira) via issues
-
[jira] [Updated] (SPARK-62091) Fix query failures when moves a filter condition that can fail ahead of a throwable condition of an inner join
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62091) Fix query failures when moves a filter condition that can fail ahead of a throwable condition of an inner join
Dongjoon Hyun (Jira) via issues
-
[jira] [Updated] (SPARK-60565) Fix reading and writing collated strings in Hive serde tables
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-62051) Fix wrong results by Dataset.unpivot when an outer join makes a value column nullable
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-62051) Fix wrong results by Dataset.unpivot when an outer join makes a value column nullable
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-62084) Make pyspark.sql.Column, DataFrameNaFunctions, DataFrameStatFunctions, Window, WindowSpec and TableArg PySpark's classic classes, of which the native Connect classes are virtual subclasses, as in pyspark-client
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62084) Make pyspark.sql.Column, DataFrameNaFunctions, DataFrameStatFunctions, Window, WindowSpec and TableArg PySpark's classic classes, of which the native Connect classes are virtual subclasses, as in pyspark-client
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-60545) Raise a clear error for `TIMESTAMP_NTZ` columns in Hive serde tables
Dongjoon Hyun (Jira) via issues
-
[jira] [Commented] (SPARK-45099) SortMergeExec with Outer using join forgets sort information
Saath (Jira) via issues
-
[jira] [Resolved] (SPARK-61878) BIN BY: a join keyword after the BIN BY clause is parsed as a table alias
Wenchen Fan (Jira) via issues
-
[jira] [Assigned] (SPARK-61878) BIN BY: a join keyword after the BIN BY clause is parsed as a table alias
Wenchen Fan (Jira) via issues
-
[jira] [Resolved] (SPARK-62052) Fix DPP evaluating a pruning key's Python or external UDF twice without pruning any partition
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-62052) Fix DPP evaluating a pruning key's Python or external UDF twice without pruning any partition
Dongjoon Hyun (Jira) via issues
-
[jira] [Created] (SPARK-62090) Use SQL type names for unmapped integral targets of pushed JDBC casts
Aleksandar Trajković (Jira) via issues
-
[jira] [Resolved] (SPARK-61506) Mark a CALL as bound only after BindProcedures binds it
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-61506) Mark a CALL as bound only after BindProcedures binds it
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-61482) Evaluate only aggregate functions on zero rows when DecorrelateInnerQuery handles the COUNT bug
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-61482) Evaluate only aggregate functions on zero rows when DecorrelateInnerQuery handles the COUNT bug
Dongjoon Hyun (Jira) via issues
-
[jira] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.
Minreng Wu (Jira) via issues
-
[jira] [Commented] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.
Minreng Wu (Jira) via issues
-
[jira] [Created] (SPARK-62087) Rework position mapper for error messages with parameter marker pre parser
Serge Rielau (Jira) via issues
-
[jira] [Created] (SPARK-62089) Make the messages of the Rust protobuf runtime of pyspark.sql.connect.proto, and of the plugins' generated modules, behave as upb's where protobuf is installed, without requiring it
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62088) Move DDL table and view lookup into RelationResolution
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62088) Move DDL table and view lookup into RelationResolution
Vladan Vasić (Jira) via issues
-
[jira] [Updated] (SPARK-62087) Error positions for parameterized queries refer to the substituted SQL, not the original SQL
Serge Rielau (Jira) via issues
-
[jira] [Resolved] (SPARK-62085) The types of pyspark.sql.types and other modules of pyspark.sql and pyspark.errors have docstrings of their own, not PySpark's, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62087) Rework position mapper for error messages with parameter marker pre parser
Serge Rielau (Jira) via issues
-
[jira] [Assigned] (SPARK-62085) The types of pyspark.sql.types and other modules of pyspark.sql and pyspark.errors have docstrings of their own, not PySpark's, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62084) Make pyspark.sql.Column, DataFrameNaFunctions, DataFrameStatFunctions, Window, WindowSpec and TableArg PySpark's classic classes, of which the native Connect classes are virtual subclasses, as in pyspark-client
ASF GitHub Bot (Jira) via issues
-
[jira] [Updated] (SPARK-62059) Bound transpiler early-return normalization and emit the NULL-guard advisory once
ASF GitHub Bot (Jira) via issues
-
[jira] [Updated] (SPARK-62085) The types of pyspark.sql.types and other modules of pyspark.sql and pyspark.errors have docstrings of their own, not PySpark's, in the Python client
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62086) Support a custom driver log URL in the History Server
Tarik Beganović (Jira) via issues
-
[jira] [Resolved] (SPARK-62082) Make pyspark.sql.functions PySpark's package of the modules builtin and partitioning, whose functions run those of pyspark.sql.connect.functions, as in pyspark-client
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62082) Make pyspark.sql.functions PySpark's package of the modules builtin and partitioning, whose functions run those of pyspark.sql.connect.functions, as in pyspark-client
Hyukjin Kwon (Jira) via issues
-
[jira] [Created] (SPARK-62085) The types of pyspark.sql.types and other modules of pyspark.sql and pyspark.errors have docstrings of their own, not PySpark's, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62083) Add golden tests for functions whose rewrite uses an argument twice
Johannes Ehls (Jira) via issues
-
[jira] [Created] (SPARK-62084) Make pyspark.sql.Column, DataFrameNaFunctions, DataFrameStatFunctions, Window, WindowSpec and TableArg PySpark's classic classes, of which the native Connect classes are virtual subclasses, as in pyspark-client
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-56076) Types Framework - Phase 1b - Generic Row Access
Uroš Bojanić (Jira) via issues
-
[jira] [Resolved] (SPARK-56076) Types Framework - Phase 1b - Generic Row Access
Uroš Bojanić (Jira) via issues
-
[jira] [Updated] (SPARK-62083) Single-pass resolver resolves a function argument again when its rewrite uses the argument twice
ASF GitHub Bot (Jira) via issues
-
[jira] [Updated] (SPARK-62082) Make pyspark.sql.functions PySpark's package of the modules builtin and partitioning, whose functions run those of pyspark.sql.connect.functions, as in pyspark-client
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62083) Single-pass resolver resolves a function argument again when its rewrite uses the argument twice
Johannes Ehls (Jira) via issues
-
[jira] [Created] (SPARK-62082) Make pyspark.sql.functions PySpark's package of the modules builtin and partitioning, whose functions run those of pyspark.sql.connect.functions, as in pyspark-client
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62081) Send the metadata of the fields of a type as PySpark's json.dumps writes it, and raise the errors of createDataFrame of a user-defined type of no JSON in PySpark's order
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62081) Send the metadata of the fields of a type as PySpark's json.dumps writes it, and raise the errors of createDataFrame of a user-defined type of no JSON in PySpark's order
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62080) The catalog, conf, readers and writers, grouped data, merge writer, Observation and TVF classes lack the docstrings that PySpark Connect copies of PySpark's classic classes, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62080) The catalog, conf, readers and writers, grouped data, merge writer, Observation and TVF classes lack the docstrings that PySpark Connect copies of PySpark's classic classes, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-61993) Resolve GAP FILL into a Generate with a GapFillGenerator
ASF GitHub Bot (Jira) via issues
-
[jira] [Resolved] (SPARK-61458) Skip master post-merge documentation builds covered by Pages
Ruifeng Zheng (Jira) via issues
-
[jira] [Updated] (SPARK-62081) Send the metadata of the fields of a type as PySpark's json.dumps writes it, and raise the errors of createDataFrame of a user-defined type of no JSON in PySpark's order
ASF GitHub Bot (Jira) via issues
-
[jira] [Updated] (SPARK-60381) Account the late-materialization survivor buffer to the task's execution memory
Peter Toth (Jira) via issues
-
[jira] [Updated] (SPARK-60107) Trim the storage-filter comments and conf docs
Peter Toth (Jira) via issues
-
[jira] [Updated] (SPARK-60034) Skip a storage filter on the rows the Parquet reader already checked
Peter Toth (Jira) via issues
-
[jira] [Updated] (SPARK-59620) Late-materialization storage-filter pushdown in the vectorized Parquet reader
Peter Toth (Jira) via issues
-
[jira] [Updated] (SPARK-62080) The catalog, conf, readers and writers, grouped data, merge writer, Observation and TVF classes lack the docstrings that PySpark Connect copies of PySpark's classic classes, in the Python client
ASF GitHub Bot (Jira) via issues
-
[jira] [Resolved] (SPARK-62079) The retry jitter of the Rust core is nearly the same for the retries of a burst of calls, where PySpark draws random.uniform(0, jitter) each time
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62079) The retry jitter of the Rust core is nearly the same for the retries of a burst of calls, where PySpark draws random.uniform(0, jitter) each time
Hyukjin Kwon (Jira) via issues
-
[jira] [Created] (SPARK-62081) Send the metadata of the fields of a type as PySpark's json.dumps writes it, and raise the errors of createDataFrame of a user-defined type of no JSON in PySpark's order
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-61279) Keep GroupedPartitionCoalescer's groups out of tasks and improve its location choice
Peter Toth (Jira) via issues
-
[jira] [Assigned] (SPARK-61279) Keep GroupedPartitionCoalescer's groups out of tasks and improve its location choice
Peter Toth (Jira) via issues
-
[jira] [Resolved] (SPARK-62078) The methods of the UDF and UDTF registrations and of the streaming classes lack the docstrings that PySpark Connect copies of PySpark's classic classes, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62078) The methods of the UDF and UDTF registrations and of the streaming classes lack the docstrings that PySpark Connect copies of PySpark's classic classes, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Created] (SPARK-62080) The catalog, conf, readers and writers, grouped data, merge writer, Observation and TVF classes lack the docstrings that PySpark Connect copies of PySpark's classic classes, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62074) pyspark.sql.utils lacks 17 of PySpark's names, and the package pyspark.sql.connect lacks check_dependencies and utils and has udf and udtf, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62074) pyspark.sql.utils lacks 17 of PySpark's names, and the package pyspark.sql.connect lacks check_dependencies and utils and has udf and udtf, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62074) pyspark.sql.utils lacks 17 of PySpark's names, and the package pyspark.sql.connect lacks check_dependencies and utils and has udf and udtf, in the Python client
ASF GitHub Bot (Jira) via issues
-
[jira] [Updated] (SPARK-62079) The retry jitter of the Rust core is nearly the same for the retries of a burst of calls, where PySpark draws random.uniform(0, jitter) each time
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62079) The retry jitter of the Rust core is nearly the same for the retries of a burst of calls, where PySpark draws random.uniform(0, jitter) each time
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62078) The methods of the UDF and UDTF registrations and of the streaming classes lack the docstrings that PySpark Connect copies of PySpark's classic classes, in the Python client
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62078) The methods of the UDF and UDTF registrations and of the streaming classes lack the docstrings that PySpark Connect copies of PySpark's classic classes, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Created] (SPARK-62077) Do not push a filter condition over a column that can fail through a projection ahead of the conditions before it
Kousuke Saruta (Jira) via issues
-
[jira] [Updated] (SPARK-62040) Improve hypothesis testing by batching examples when it makes sense through df creation
ASF GitHub Bot (Jira) via issues
-
[jira] [Assigned] (SPARK-62062) Fix JSON and row-based Parquet and ORC readers re-analyzing every existence default value for each row
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-62062) Fix JSON and row-based Parquet and ORC readers re-analyzing every existence default value for each row
Dongjoon Hyun (Jira) via issues
-
[jira] [Updated] (SPARK-62076) Transpile str in and not in to contains
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62076) Transpile str in and not in to contains
Guan-Ming Chiu (Jira) via issues
-
[jira] [Resolved] (SPARK-62071) Make the classic modules of the objects of a session PySpark's, so that they are not instances of the classic classes, as in pyspark-client
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62071) Make the classic modules of the objects of a session PySpark's, so that they are not instances of the classic classes, as in pyspark-client
Hyukjin Kwon (Jira) via issues
-
[jira] [Created] (SPARK-62075) Stream-stream outer join with state format 4 never emits rows of an input without its own watermark that a legacy-mode run stored
Jungtaek Lim (Jira) via issues
-
[jira] [Created] (SPARK-62074) pyspark.sql.utils lacks 17 of PySpark's names, and the package pyspark.sql.connect lacks check_dependencies and utils and has udf and udtf, in the Python client
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62073) Fix ClassCastException in classic randomSplit with int weights
ASF GitHub Bot (Jira) via issues
-
[jira] [Assigned] (SPARK-62065) [Connect][Python] Drop-in: hold PySpark's user-defined type objects in the cached schemas of set operations and pickles, raise the conversion errors of the schema of the server where PySpark reads it, and make DataType.simpleString the typeName
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62065) [Connect][Python] Drop-in: hold PySpark's user-defined type objects in the cached schemas of set operations and pickles, raise the conversion errors of the schema of the server where PySpark reads it, and make DataType.simpleString the typeName
Hyukjin Kwon (Jira) via issues
-
[jira] [Created] (SPARK-62073) Fix ClassCastException in classic randomSplit with int weights
Anshul Baliga (Jira) via issues
-
[jira] [Resolved] (SPARK-62072) An Observation's _name, _result and _error are its attributes, its plan is built of its _name each time, its metrics are set at the action, and a signal handler's Exception is caught by the plan compression lookup, as PySpark Connect
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62072) An Observation's _name, _result and _error are its attributes, its plan is built of its _name each time, its metrics are set at the action, and a signal handler's Exception is caught by the plan compression lookup, as PySpark Connect
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62071) Make the classic modules of the objects of a session PySpark's, so that they are not instances of the classic classes, as in pyspark-client
ASF GitHub Bot (Jira) via issues
-
[jira] [Updated] (SPARK-62072) An Observation's _name, _result and _error are its attributes, its plan is built of its _name each time, its metrics are set at the action, and a signal handler's Exception is caught by the plan compression lookup, as PySpark Connect
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62072) An Observation's _name, _result and _error are its attributes, its plan is built of its _name each time, its metrics are set at the action, and a signal handler's Exception is caught by the plan compression lookup, as PySpark Connect
Hyukjin Kwon (Jira) via issues
-
[jira] [Created] (SPARK-62071) Make the classic modules of the objects of a session PySpark's, so that they are not instances of the classic classes, as in pyspark-client
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62070) Fix CHECK constraints accepting rows that violate them when they compare a CHAR column with a string literal
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62070) Fix CHECK constraints accepting rows that violate them when they compare a CHAR column with a string literal
Dongjoon Hyun (Jira) via issues
-
[jira] [Updated] (SPARK-62069) Fix query failures when ReorderJoin evaluates a condition after a NULL condition of a lower join
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62069) Fix query failures when ReorderJoin evaluates a condition after a NULL condition of a lower join
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-62066) [Connect][Python] Drop-in: take local rows apart by the cached names of a StructType changed in place, raise PySpark's errors of the types that proto_schema_to_pyspark_data_type does not convert, and raise the MALFORMED conditions of no parameters
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62060) Fix CSV reader re-analyzing every existence default value for each missing column in each row
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-62060) Fix CSV reader re-analyzing every existence default value for each missing column in each row
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-62066) [Connect][Python] Drop-in: take local rows apart by the cached names of a StructType changed in place, raise PySpark's errors of the types that proto_schema_to_pyspark_data_type does not convert, and raise the MALFORMED conditions of no parameters
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-61938) Fix query errors by BooleanSimplification when an operand of AND or OR guards a throwable expression
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-61938) Fix query errors by BooleanSimplification when an operand of AND or OR guards a throwable expression
Dongjoon Hyun (Jira) via issues
-
[jira] [Updated] (SPARK-60775) getOrCreateDataFrame for DataFrame to allow to resuming after a failed stage
Amogh Desai (Jira) via issues
-
[jira] [Created] (SPARK-62068) Support SparkSession.getOrCreateDataFrame in Spark Connect
Amogh Desai (Jira) via issues
-
[jira] [Updated] (SPARK-60775) getOrCreate for Dataset / DataFrame to allow to resuming after a failed stage
Amogh Desai (Jira) via issues
-
[jira] [Assigned] (SPARK-62056) Take the arguments of the methods of the maps and repeated containers of the Rust protobuf runtime as upb's, with upb's errors
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62056) Take the arguments of the methods of the maps and repeated containers of the Rust protobuf runtime as upb's, with upb's errors
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62064) Send the relations of DataFrames that the extensions of expressions hold, and with the changes a plan makes to them, and resolve the messages of extensions in a time of their length, as PySpark sends its plans
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62064) Send the relations of DataFrames that the extensions of expressions hold, and with the changes a plan makes to them, and resolve the messages of extensions in a time of their length, as PySpark sends its plans
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62055) Fix vectorized Parquet and ORC readers re-analyzing every existence default value for each column
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-62055) Fix vectorized Parquet and ORC readers re-analyzing every existence default value for each column
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-62050) Tag more DSv2-only test suites with `DataSourceV2Test`
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-62050) Tag more DSv2-only test suites with `DataSourceV2Test`
Dongjoon Hyun (Jira) via issues
-
[jira] [Updated] (SPARK-62067) Report Python worker timing metrics in microseconds
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62067) Report Python worker timing metrics in microseconds
Haotian Sun (Jira) via issues
-
[jira] [Assigned] (SPARK-62063) Report the option keys and values that are objects, and the arguments that the writers keep, to Python's garbage collector in the readers, writers and session builder, also of their shallow copies
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62063) Report the option keys and values that are objects, and the arguments that the writers keep, to Python's garbage collector in the readers, writers and session builder, also of their shallow copies
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62066) [Connect][Python] Drop-in: take local rows apart by the cached names of a StructType changed in place, raise PySpark's errors of the types that proto_schema_to_pyspark_data_type does not convert, and raise the MALFORMED conditions of no parameters
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62066) [Connect][Python] Drop-in: take local rows apart by the cached names of a StructType changed in place, raise PySpark's errors of the types that proto_schema_to_pyspark_data_type does not convert, and raise the MALFORMED conditions of no parameters
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62065) [Connect][Python] Drop-in: hold PySpark's user-defined type objects in the cached schemas of set operations and pickles, raise the conversion errors of the schema of the server where PySpark reads it, and make DataType.simpleString the typeName
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62065) [Connect][Python] Drop-in: hold PySpark's user-defined type objects in the cached schemas of set operations and pickles, raise the conversion errors of the schema of the server where PySpark reads it, and make DataType.simpleString the typeName
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62064) Send the relations of DataFrames that the extensions of expressions hold, and with the changes a plan makes to them, and resolve the messages of extensions in a time of their length, as PySpark sends its plans
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62064) Send the relations of DataFrames that the extensions of expressions hold, and with the changes a plan makes to them, and resolve the messages of extensions in a time of their length, as PySpark sends its plans
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-59869) Update GitHub Actions to the latest approved versions
Dongjoon Hyun (Jira) via issues
-
[jira] [Updated] (SPARK-60528) Fix `ExpressionEncoder` assertion failure on nested empty structs
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-61704) Fix ConstantPropagation propagating an EqualTo with a NULL literal under a NOT
Kousuke Saruta (Jira) via issues
-
[jira] [Assigned] (SPARK-61704) Fix ConstantPropagation propagating an EqualTo with a NULL literal under a NOT
Kousuke Saruta (Jira) via issues
-
[jira] [Updated] (SPARK-62063) Report the option keys and values that are objects, and the arguments that the writers keep, to Python's garbage collector in the readers, writers and session builder, also of their shallow copies
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62063) Report the option keys and values that are objects, and the arguments that the writers keep, to Python's garbage collector in the readers, writers and session builder, also of their shallow copies
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62061) SparkSession.addArtifacts reads a path that is not a str, and pyspark.sql.connect.client exports its names, as PySpark Connect
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62061) SparkSession.addArtifacts reads a path that is not a str, and pyspark.sql.connect.client exports its names, as PySpark Connect
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62049) Use `splitMovable` in `CombineFilters`
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-62049) Use `splitMovable` in `CombineFilters`
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-62058) Log the events of a pipeline run at the exact times of their timestamps, as PySpark's Timestamp.ToDatetime() gives them
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62058) Log the events of a pipeline run at the exact times of their timestamps, as PySpark's Timestamp.ToDatetime() gives them
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62062) Fix JSON and row-based Parquet and ORC readers re-analyzing every existence default value for each row
Dongjoon Hyun (Jira) via issues
-
[jira] [Created] (SPARK-62062) Fix JSON reader and row-based Parquet and ORC readers re-analyzing every existence default value for each row
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-62047) Pickle the Any and Timestamp messages of the drop-in's protobuf runtime, and fail each request of a user context extension that is not an Any, as PySpark
Hyukjin Kwon (Jira) via issues
-
[jira] [Assigned] (SPARK-62047) Pickle the Any and Timestamp messages of the drop-in's protobuf runtime, and fail each request of a user context extension that is not an Any, as PySpark
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62061) SparkSession.addArtifacts reads a path that is not a str, and pyspark.sql.connect.client exports its names, as PySpark Connect
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62061) SparkSession.addArtifacts reads a path that is not a str, and pyspark.sql.connect.client exports its names, as PySpark Connect
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62060) Fix CSV reader re-analyzing every existence default value for each missing column in each row
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62060) Fix CSV reader re-analyzing every existence default value for each missing column in each row
Dongjoon Hyun (Jira) via issues
-
[jira] [Created] (SPARK-62059) Bound transpiler early-return normalization and emit the NULL-guard advisory once
Holden Karau (Jira) via issues
-
[jira] [Updated] (SPARK-62058) Log the events of a pipeline run at the exact times of their timestamps, as PySpark's Timestamp.ToDatetime() gives them
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62058) Log the events of a pipeline run at the exact times of their timestamps, as PySpark's Timestamp.ToDatetime() gives them
Hyukjin Kwon (Jira) via issues
-
[jira] [Created] (SPARK-62057) Stream-stream left semi join over a streaming union fails with a missing state file when a union input gets its first data after the first micro-batch
Jungtaek Lim (Jira) via issues
-
[jira] [Updated] (SPARK-62057) Stream-stream left semi join over a streaming union fails with a missing state file when a union input gets its first data after the first micro-batch
Jungtaek Lim (Jira) via issues
-
[jira] [Updated] (SPARK-62056) Take the arguments of the methods of the maps and repeated containers of the Rust protobuf runtime as upb's, with upb's errors
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62056) Take the arguments of the methods of the maps and repeated containers of the Rust protobuf runtime as upb's, with upb's errors
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62055) Fix vectorized Parquet and ORC readers re-analyzing every existence default value for each column
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62055) Fix vectorized Parquet and ORC readers re-analyzing every existence default value for each column
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-60301) Spark doesn't respect session timezone when casting nested variant to string
Wenchen Fan (Jira) via issues
-
[jira] [Resolved] (SPARK-60301) Spark doesn't respect session timezone when casting nested variant to string
Wenchen Fan (Jira) via issues
-
[jira] [Commented] (SPARK-59718) Add opt-in in-process Python UDF execution via Arrow C Data Interface
Yicong Huang (Jira) via issues
-
[jira] [Updated] (SPARK-62054) Fix wrong results and query failures when cost-based join reorder drops or moves join conditions
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62054) Fix wrong results and query failures when cost-based join reorder drops or moves join conditions
Dongjoon Hyun (Jira) via issues
-
[jira] [Created] (SPARK-62053) Remove outdated unsupported `ArrayType of TimestampType` note from `spark.sql.execution.arrow.pyspark.enabled` documentation
Zhou Yifan (Jira) via issues
-
[jira] [Assigned] (SPARK-62041) Warn of a window in turn with the picklings of UDFs, send the clauses of a merge as they were built, keep the execution info of a failed toPandas, and pickle subquery Columns and Column._expr, as PySpark
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62041) Warn of a window in turn with the picklings of UDFs, send the clauses of a merge as they were built, keep the execution info of a failed toPandas, and pickle subquery Columns and Column._expr, as PySpark
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62047) Pickle the Any and Timestamp messages of the drop-in's protobuf runtime, and fail each request of a user context extension that is not an Any, as PySpark
ASF GitHub Bot (Jira) via issues
-
[jira] [Updated] (SPARK-62052) Fix DPP evaluating a pruning key's Python or external UDF twice without pruning any partition
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62052) Fix DPP evaluating a pruning key's Python or external UDF twice without pruning any partition
Dongjoon Hyun (Jira) via issues
-
[jira] [Updated] (SPARK-62051) Fix wrong results by Dataset.unpivot when an outer join makes a value column nullable
ASF GitHub Bot (Jira) via issues
-
[jira] [Created] (SPARK-62051) Fix wrong results by Dataset.unpivot when an outer join makes a value column nullable
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-61935) Fix DSv2 writes failing or writing NULL into NOT NULL columns for a Scala UDF with a primitive input
Dongjoon Hyun (Jira) via issues
-
[jira] [Assigned] (SPARK-61935) Fix DSv2 writes failing or writing NULL into NOT NULL columns for a Scala UDF with a primitive input
Dongjoon Hyun (Jira) via issues
-
[jira] [Resolved] (SPARK-61301) Improve the FILE type error messages and their builders
Daniel Tenedorio (Jira) via issues
-
[jira] [Assigned] (SPARK-62037) Share the options and hooks of a builder's shallow copies, hold PySpark's session and builder locks, let SparkSession, Catalog and DataSourceRegistration be subclassed, and load PySpark's pickles of readers
Hyukjin Kwon (Jira) via issues
-
[jira] [Resolved] (SPARK-62037) Share the options and hooks of a builder's shallow copies, hold PySpark's session and builder locks, let SparkSession, Catalog and DataSourceRegistration be subclassed, and load PySpark's pickles of readers
Hyukjin Kwon (Jira) via issues
-
[jira] [Updated] (SPARK-62050) Tag more DSv2-only test suites with `DataSourceV2Test`
Dongjoon Hyun (Jira) via issues
-
[jira] [Updated] (SPARK-62050) Tag more DSv2-only test suites with `DataSourceV2Test`
ASF GitHub Bot (Jira) via issues