commits  

Messages by Thread

• (spark) branch branch-4.0 updated: [SPARK-53167][DEPLOY] Spark launcher isRemote also respects properties file dongjoon
• (spark) branch master updated: [SPARK-53167][DEPLOY] Spark launcher isRemote also respects properties file dongjoon
• (spark) branch master updated: [SPARK-53163][PYTHON][INFRA] Upgrade PyArrow to 21.0.0 dongjoon
• (spark) branch master updated (598f310aed5d -> 89973d057699) dongjoon
• (spark) branch master updated (4430cd8544b1 -> 598f310aed5d) dongjoon
• (spark) branch master updated (88dbe4297103 -> 4430cd8544b1) dongjoon
• (spark-connect-go) branch master updated (defb852 -> bc07d65) mgrund
• (spark) branch master updated: [SPARK-53066][SQL] Improve EXPLAIN output for DSv2 Join pushdown wenchen
• (spark) branch master updated: [SPARK-52828][SQL] Make hashing for collated strings collation agnostic wenchen
• (spark) branch branch-4.0 updated: [SPARK-52828][SQL] Make hashing for collated strings collation agnostic wenchen
• (spark-connect-rust) branch master updated: [SPARK-52964] Add `merge_connect_rust_pr.py` and `PULL_REQUEST_TEMPLATE` (#3) liyuanjian
• (spark) branch master updated: [SPARK-53166][CORE] Use `SparkExitCode.EXIT_FAILURE` in `SparkPipelines` object dongjoon
• (spark) branch master updated: [SPARK-53165][CORE] Add `SparkExitCode.CLASS_NOT_FOUND` dongjoon
• (spark) branch master updated: [SPARK-53162][SQL][TESTS] Mark `DynamicPartitionPruningHive*Suite*` as `SlowHiveTest` dongjoon
• (spark) branch master updated (e4ded0810119 -> a72fd418242d) ruifengz
• (spark) branch branch-4.0 updated: [MINOR][CORE] Remove a wrong comment inside `SparkTestUtils.SparkTestUtils` dongjoon
• (spark) branch master updated: [MINOR][CORE] Remove a wrong comment inside `SparkTestUtils.SparkTestUtils` dongjoon
• (spark) branch master updated: [SPARK-52980][FOLLOWUP] Fix Python Arrow UDTF test cases ruifengz
• (spark) branch master updated: [SPARK-53134][PS] Clean up unused ANSI imports in tests xinrong
• (spark) branch master updated (a09ea772e374 -> 6b4c3138c85e) dongjoon
• (spark) branch master updated: [SPARK-53098][SQL][TEST-ONLY][FOLLOWUP] Fix a test when returning single-pass result wenchen
• (spark) branch master updated (9a458eb8b6f0 -> 49d542d1d951) xinrong
• (spark) branch master updated (7db6c8287b08 -> 9a458eb8b6f0) dongjoon
• (spark) branch master updated: [SPARK-53153][CORE][TESTS] Use Java `Files.readAllLines` in `PluginContainerSuite` dongjoon
• (spark) branch master updated (19ea6ff95ef3 -> 01d695365d3d) dongjoon
• (spark) branch master updated (f6cd38598078 -> 19ea6ff95ef3) wenchen
• (spark) branch branch-4.0 updated: [SPARK-53130][SQL][PYTHON] Fix `toJson` behavior of collated string types wenchen
• (spark) branch master updated: [SPARK-52943][PYTHON] Enable arrow_cast for all pandas UDF eval types ruifengz
• (spark) branch master updated: [SPARK-53107][SQL] Implement the time_trunc function in Scala wenchen
• (spark) branch master updated (cf3ab766c8c2 -> 70c20087b709) wenchen
• (spark) branch master updated: [SPARK-53136][CORE] tryWithResource & tryInitializeResource shall close resource quietly yao
• (spark) branch branch-4.0 updated: [SPARK-52989][SS][4.0] Add explicit close() API to State Store iterators kabhwan
• (spark) branch master updated: [SPARK-51585][SQL] Oracle dialect supports pushdown datetime functions beliefer
• (spark) branch master updated (2f1fde501227 -> 95a96891ac6e) dongjoon
• (spark) branch master updated (bca5f3a0f09e -> 2f1fde501227) dongjoon
• (spark) branch branch-3.5 updated (2524c0a6d578 -> 665ccb34dd9e) ptoth
• (spark) branch branch-4.0 updated (3c88f32583e9 -> a4ed4c83147e) ptoth
• (spark) branch master updated: [SPARK-53132][CORE][TESTS][FOLLOWUP] Use `Utils.listFiles` in `BlockManagerDecommissionIntegrationSuite` dongjoon
• (spark) branch branch-4.0 updated: [SPARK-52942][YARN][BUILD] YARN External Shuffle Service jar should include `scala-library` yangjie01
• (spark) branch master updated: [SPARK-52942][YARN][BUILD] YARN External Shuffle Service jar should include `scala-library` yangjie01
• (spark) branch master updated (d2187f4b4ea1 -> 08e734e6fb71) mridulm80
• (spark) branch master updated: [SPARK-53135][CORE][SQL] Support `copyURLToFile` in `SparkFileUtils` and `JavaUtils` dongjoon
• (spark) branch master updated: [SPARK-52989][SS] Add explicit close() API to State Store iterators kabhwan
• (spark) branch master updated: [SPARK-53133][SQL][TESTS] Use Java `System.getProperty` instead of `FileUtils.getTempDirectory` dongjoon
• (spark) branch master updated (ca02481b6be2 -> f4640734ef8b) dongjoon
• (spark-kubernetes-operator) branch main updated: [SPARK-53026] Fix deserialize error for spec with DriverServiceIngressSpec field ptoth
• (spark) branch master updated: [SPARK-52971][PYTHON] Limit idle Python worker queue size ueshin
• (spark) branch master updated: [SPARK-53094][SQL] Fix CUBE with aggregate containing HAVING clauses ptoth
• (spark-kubernetes-operator) branch main updated: [SPARK-53027] Operator sa should be able to create ingress ptoth
• (spark) branch master updated (5180c0c51de1 -> af4e1d91b76a) ueshin
• (spark) branch master updated (0491d7e32bb9 -> 5180c0c51de1) ashrigondekar
• (spark) branch master updated: [SPARK-53131][SHELL] Improve `SparkShell` to import `java.nio.file._` by default dongjoon
• (spark) branch master updated (0c885bfd14b1 -> 48367c125b9d) dongjoon
• (spark) branch master updated: [SPARK-53126][SQL][CONNECT] Refactor `SparkErrorUtils#stackTraceToString` to handle `null` input and use it to replace `ExceptionUtils#getStackTrace` yangjie01
• (spark) branch master updated: [SPARK-53077][CORE][TESTS][FOLLOWUP] Reduce insertion count in SparkBloomFilterSuite dongjoon
• (spark) branch master updated: [SPARK-51462][SPARK-51564][SQL][FOLLOWUP][TESTS] Add E2E SQL tests for TIME literals in 24hr and 12hr formats wenchen
• (spark) branch master updated: [SPARK-53123][CORE][SQL] Support `getRootCause` in `SparkErrorUtils` yangjie01
• (spark) branch master updated (4a2ce9f7b0df -> ef30c3aaa63e) yangjie01
• (spark) branch master updated (9932e8ccf715 -> 4a2ce9f7b0df) wenchen
• (spark) branch master updated (faa6723125d0 -> 9932e8ccf715) dongjoon
• (spark) branch master updated: [SPARK-53121][SQL] Use `deleteRecursively` instead of `FileUtils.forceDelete` dongjoon
• (spark) branch master updated (bccbb5130d17 -> 84becfbb73a6) dongjoon
• (spark) branch branch-3.5 updated: Revert "[SPARK-49182][DOCS][PYTHON] Stop publish site/docs/{version}/api/python/_sources dir" gurwls223
• (spark) branch master updated: [SPARK-53120][DOCS] Recover _source directory for PySpark documentation gurwls223
• (spark) branch branch-4.0 updated: [SPARK-53120][DOCS] Recover _source directory for PySpark documentation gurwls223
• (spark) branch r0DK6b6xRWqc5bnV deleted (was 6add9c89855f) gurwls223
• (spark) branch master updated: [SPARK-53118][CORE][TESTS] Use `Files.write` in `GenericFileInputStreamSuite` dongjoon
• (spark) branch master updated (1f4f5401d072 -> 93fc2ed9fb1b) allisonwang
• (spark) branch master updated (50882d2d8886 -> 1f4f5401d072) dongjoon
• (spark) branch master updated: [SPARK-53115][TESTS] Ban `org.apache.commons.lang3.StringUtils` dongjoon
• (spark) branch master updated (8e99e4ee3d5d -> ff06fa2b6e27) dongjoon
• (spark) branch master updated (14c62c0dd6d4 -> 8e99e4ee3d5d) dongjoon
• (spark) branch master updated (35c299a1e3e3 -> 14c62c0dd6d4) ashrigondekar
• (spark) branch master updated: [SPARK-53002][SS] Show accurate memory usage statistics when bounded memory is enabled for RocksDB ashrigondekar
• (spark) branch master updated: [SPARK-53102][BUILD] Upgrade `commons-cli` to 1.10.0 dongjoon
• (spark) branch master updated (ba85b25e3867 -> c28b51b4dae3) dongjoon
• (spark) branch master updated: [SPARK-53100][SQL] Use Java `String.substring` instead of `StringUtils.substring` dongjoon
• (spark) branch master updated: [SPARK-52974][SQL] Don't broadcast dynamicpruning InSubqueries ptoth
• (spark) branch master updated: [SPARK-53040][SQL] Ban self references inside topmost CTEs in Recursive CTEs wenchen
• (spark) branch master updated: [SPARK-52451][CONNECT][SQL] Make WriteOperation in SparkConnectPlanner side effect free wenchen
• (spark) branch master updated: [SPARK-53092][CORE][CONNECT] Ban `org.apache.commons.lang3.SystemUtils` dongjoon
• (spark) branch master updated (ced87308a23e -> 635c24295c85) dongjoon
• (spark) branch master updated: [SPARK-53077][CORE][TESTS] Add an env to control the execution of the `SparkBloomFilterSuite` yangjie01
• (spark) branch master updated: [SPARK-51555][SQL][FOLLOWUP] Fix the error message for invalid unit in the time_diff() function wenchen
• (spark) branch master updated (c9b85a2d6008 -> 09d2765e56ca) wenchen
• (spark) branch xyz deleted (was 6b99ccbabc16) ruifengz
• (spark) branch xyz created (now 6b99ccbabc16) ruifengz
  • (spark) 02/03: xyz ruifengz
  • (spark) 01/03: Your commit message here ruifengz
  • (spark) 03/03: xyz ruifengz
• (spark) branch master updated: [SPARK-53063][CORE] Implement and call new APIs in FileCommitProtocol instead of the deprecated yao
• (spark) branch master updated: [SPARK-53093][K8S][DOCS] Drop K8s v1.31 Support dongjoon
• (spark) branch master updated: [SPARK-52975][SQL] Simplify field names in pushdown join sql wenchen
• (spark) branch master updated: [SPARK-53091][SQL] Ban `org.apache.commons.lang3.Strings` dongjoon
• (spark) branch master updated: [SPARK-53032] Fix parquet format of shredded timestamp values within arrays wenchen
• (spark) branch master updated (a32bfd7be493 -> a7e9d93b2a2c) dongjoon
• (spark) branch master updated: [SPARK-53008][SQL] Infer SQL Data Access from SQL UDF body wenchen
• (spark) branch master updated: [SPARK-53088][BUILD] Remove `scala-collection-compat` dependency dongjoon
• (spark) branch master updated: [SPARK-53090][CORE][SS][TESTS] Use Java `OutputStream.write` instead of `IOUtils.write` dongjoon
• (spark) branch master updated: [SPARK-53087][BUILD] Upgrade `Jackson` to 2.19.2 dongjoon
• (spark-website) branch asf-site updated: Remove Spark 3.4.4 release from download dropdown list (#622) dongjoon
• (spark) branch master updated: [SPARK-53086][TESTS] Revise Scalastyle RegexChecker patterns dongjoon
• (spark) branch master updated: [SPARK-53079][K8S][DOCS] Update `YuniKorn` docs with `1.7.0` dongjoon
• (spark) branch master updated: [SPARK-53080][CORE][CONNECT] Support `cleanDirectory` in `SparkFileUtils` and `JavaUtils` dongjoon
• (spark) branch master updated: [SPARK-53081][CORE][SQL][CONNECT] Support `contentEquals` in `SparkFileUtils` dongjoon
• (spark) branch master updated: [SPARK-53083][CORE][SQL][K8S][SS][TESTS] Use Java `Files.writeString` instead of `FileUtils.write` dongjoon
• (spark) branch master updated: [SPARK-53082][CORE][SQL][TESTS] Use Java `Files.readString` instead of `FileUtils.readFileToString` dongjoon
• (spark) branch master updated: [SPARK-53078][CORE][TESTS] Remove `commons-(io|lang3)` test dependencies from `kvstore` yangjie01
• (spark) branch master updated: [SPARK-53076][CORE][TESTS] Disable `SparkBloomFilterSuite` yangjie01
• (spark) branch r0DK6b6xRWqc5bnV created (now 6add9c89855f) holden
• (spark) branch master updated (322b3d06d3f8 -> 4a8c4be79918) dongjoon
• (spark) branch master updated: [SPARK-53075][CORE][TESTS] Use Java `Files.readAllLines/write` instead of `FileUtils.(read|write)Lines` yangjie01
• [PR] Remove Spark 3.4.4 release from download dropdown list [spark-website] via GitHub
  • Re: [PR] Remove Spark 3.4.4 release from download dropdown list [spark-website] via GitHub
  • Re: [PR] Remove Spark 3.4.4 release from download dropdown list [spark-website] via GitHub
• (spark) branch master updated (443e170397a5 -> 2513f4123539) dongjoon
• (spark) branch master updated (fd5ed10d01dc -> 443e170397a5) dongjoon
• (spark) branch master updated (bd76faf83bff -> fd5ed10d01dc) gurwls223
• (spark) branch master updated: [SPARK-53068][SQL][TESTS] Mark `TransformWith*Suite` as `SlowSQLTest` in `sql/core` dongjoon
• (spark) branch master updated (0225d4cf76cc -> 6e7c67fef7af) dongjoon
• (spark) branch master updated: [SPARK-53001][CORE][SQL][FOLLOW-UP] Disable `spark.memory.unmanagedMemoryPollingInterval` by default dongjoon
• (spark) branch master updated: [SPARK-53035][TESTS][FOLLOWUP] Use `String.repeat` in `tests` too dongjoon
• (spark) branch master updated: [SPARK-52968][SS] Emit additional state store metrics ashrigondekar
• (spark) branch master updated: [SPARK-53061][CORE][SQL] Support `copyFileToDirectory` in `SparkFileUtils` dongjoon
• (spark) branch branch-3.5 updated: [SPARK-53054][CONNECT][3.5] Fix the connect.DataFrameReader default format behavior ruifengz
• (spark) branch master updated: [SPARK-53059][PYTHON] Arrow UDF no need to depend on pandas ruifengz
• (spark) branch master updated (032677cb329d -> 6b0d2ca00121) dongjoon
• (spark) branch master updated: [SPARK-52928][PYTHON][FOLLOW-UP] Remove unreachable code after upgrading pyarrow minimum version to 15.0.0 dongjoon
• (spark) branch master updated (0dd0e022b411 -> 3839fed74c20) wenchen
• (spark) branch branch-4.0 updated: [SPARK-53054][CONNECT] Fix the connect.DataFrameReader default format behavior gurwls223
• (spark) branch master updated (92df232addc5 -> 0dd0e022b411) gurwls223
• [PR] Fix broken Pandas On PySpark link doc link [spark-website] via GitHub
• (spark) branch master updated (fbebc20e61d1 -> 92df232addc5) dongjoon
• (spark) branch master updated (d0a8b18c1adb -> fbebc20e61d1) dongjoon
• (spark) branch master updated (57dfc0a21bf2 -> d0a8b18c1adb) ruifengz
• (spark) branch master updated: [SPARK-53039][PYTHON][TESTS] Add unit test for complex arrow UDF used in window dongjoon
• (spark) branch master updated (8c90e6fa77c4 -> 133d9f60e72f) dongjoon
• (spark) branch master updated (a75e1aa1cac2 -> 8c90e6fa77c4) dongjoon
• (spark) branch master updated: [SPARK-53001] Integrate RocksDB Memory Usage with the Unified Memory Manager ashrigondekar
• (spark) branch master updated (fd17a8d06e23 -> 3024c7c21db8) dongjoon
• (spark) branch master updated (707d9eef6bfa -> fd17a8d06e23) gurwls223
• (spark) branch master updated (c5789579de20 -> 707d9eef6bfa) dongjoon
• (spark) branch master updated: [SPARK-53036][PYTHON][DOCS][TESTS] Enable doctest `pyspark.sql.pandas.functions` gurwls223
• (spark) branch master updated: [SPARK-53043][CORE][SQL][K8S] Use Java `transferTo` instead of `IOUtils.copy` dongjoon
• (spark) branch master updated (e9a285eaf273 -> 9c35c43eaf3b) xinrong
• (spark) branch master updated: Revert "[SPARK-52610][BUILD] Upgrade rocksdbjni to 10.2.1" dongjoon
• (spark) branch master updated: [MINOR][CORE][TESTS] Rename `TestMemoryManager.(markconsequentOOM -> markConsequentOOM)` dongjoon
• (spark) branch master updated (6a6317d6fd2f -> 1ce28ccfb7de) yangjie01
• (spark) branch branch-4.0 updated: [SPARK-53020][DEPLOY][4.0] JPMS args should also apply to non-SparkSubmit process dongjoon
• (spark) branch master updated (293342da1185 -> 6a6317d6fd2f) dongjoon
• (spark) branch master updated: [SPARK-53017][SQL][FOLLOWUP] Add logs in DSv2 Join pushdown rule wenchen
• (spark) branch master updated: [SPARK-52926][SQL] Added SQLMetric for remote schema fetching time duration wenchen
• (spark) branch master updated: [SPARK-53035][CORE][SQL][K8S][MLLIB] Use `String.repeat` instead of Scala string multiplication dongjoon
• (spark) branch master updated (b03a86923c0d -> d892ca40fa66) dongjoon
• (spark) branch master updated: [SPARK-51555][SQL] Add the time_diff() function wenchen
• (spark) branch master updated: [SPARK-52940][PYTHON][TESTS][FOLLOW-UP] Correct the return type in test UDFs gurwls223
• (spark) branch master updated: [MINOR][PYTHON][DOCS] Fix a pandas UDF example gurwls223
• (spark) branch master updated: [SPARK-53031][CORE] Support `getFile` in `SparkFileUtils` dongjoon
• (spark) branch master updated (3738cb3dd701 -> 1f425c7433b4) dongjoon
• (spark) branch master updated (dd6525acd927 -> 3738cb3dd701) dongjoon
• (spark) branch master updated: [SPARK-53023][SQL] Remove `commons-io` dependency from `sql/api` module dongjoon
• (spark) branch master updated: [SPARK-53014][PYTHON][DOCS] Make Arrow UDF public gurwls223
• (spark) branch master updated: [SPARK-53013][PYTHON] Fix Arrow-optimized Python UDTF returning no rows on lateral join gurwls223
• (spark) branch master updated: [SPARK-53024][BUILD] Upgrade `commons-io` to 2.20.0 dongjoon
• (spark) branch master updated: [SPARK-52622][PS] Avoid CAST_INVALID_INPUT of `DataFrame.melt` in ANSI mode xinrong
• (spark) branch master updated (9a1c1ab4710a -> dc8fba647ac1) dongjoon
• (spark) branch master updated: [SPARK-52008] Throwing an error if State Stores do not commit at the end of a batch when ForeachBatch is used ashrigondekar
• (spark) branch master updated: [SPARK-53020][DEPLOY] JPMS args should also apply to non-SparkSubmit process dongjoon
• (spark) branch master updated: [SPARK-52967][BUILD] Upgrade ORC to 2.2.0 dongjoon
• (spark) branch master updated: [SPARK-52954][PYTHON][TESTS][FOLLOW-UP] Alway set safe_check=True in Arrow UDFs gurwls223
• (spark) branch master updated: [SPARK-53018][PYTHON] ArrowStreamArrowUDFSerializer should respect argument arrow_cast gurwls223
• (spark) branch master updated: [SPARK-51554][SQL] Add the time_trunc() function wenchen
• (spark) branch master updated: [SPARK-53003][CORE][FOLLOWUP] Handle null input values dongjoon
• (spark) branch master updated: [SPARK-53004][CORE] Support `abbreviate` in `SparkStringUtils` dongjoon
• (spark) branch master updated: [SPARK-53010][MLLIB][YARN] Ban `com.google.common.base.Strings` dongjoon
• (spark) branch master updated (161f447d6432 -> 783beb648d0e) dongjoon
• (spark) branch master updated (42553b94e368 -> 161f447d6432) dongjoon
• (spark) branch master updated: [SPARK-52992][PYTHON][DOCS] Restore API reference of `pandas_udf` gurwls223
• (spark) branch master updated: [SPARK-52985][PS] Raise TypeError for pandas numpy operand in comparison operators xinrong
• (spark) branch master updated (c3fa01032fcc -> 2d1f77f9f744) xinrong
• (spark) branch master updated (475d72f61c92 -> c3fa01032fcc) dongjoon
• (spark) branch master updated: [SPARK-52999][K8S][TESTS] Clean up the deprecated APIs usage in `kubernetes-integration-tests` module dongjoon
• (spark) branch master updated: [SPARK-52995][YARN] Use Buffered I/O for creating spark jar archive dongjoon
• (spark) branch master updated: [SPARK-52959][PYTHON] Support UDT in Arrow-optimized Python UDTF dongjoon
• (spark) branch master updated: [SPARK-52990][CORE] Support `StringSubstitutor` dongjoon
• (spark) branch master updated: [SPARK-52952][PYTHON] Add PySpark UDF Type Coercion Dev Script gurwls223
• (spark) branch master updated: [SPARK-52889][PYTHON] Implement the current_time function in PySpark ruifengz
• (spark) branch master updated (8274b7c78455 -> 7a6a6e66df20) wenchen
• (spark) branch master updated: [SPARK-52993][BUILD] Bump Snappy 1.1.10.8 dongjoon
• (spark) branch master updated: [SPARK-52977][TESTS] Fix npm vulnerabilities by `npm audit fix` yangjie01
• (spark) branch master updated (8348f2a845f1 -> 633ffe4f3744) ruifengz
• (spark) branch master updated: [SPARK-52983][BUILD] Upgrade Netty to 4.1.123.Final dongjoon
• (spark) branch master updated (c7d780b0bb3f -> 9eee6bf6be85) yao
• (spark) branch master updated: [SPARK-52987][SQL][K8S] Use Java `String.(equals|replace)` method instead of `commons-lang3` dongjoon
• (spark) branch master updated (13c43bc4fd2c -> e65341397b2e) wenchen
• (spark) branch master updated: [SPARK-52890][SPARK-52891][PYTHON] Implement the to_time and try_to_time functions in PySpark gurwls223
• (spark) branch branch-3.5 updated: [SPARK-52945][SQL][TESTS] Split `CastSuiteBase#checkInvalidCastFromNumericType` into three methods and guarantee assertions are valid yangjie01
• (spark) branch branch-4.0 updated: [SPARK-52945][SQL][TESTS] Split `CastSuiteBase#checkInvalidCastFromNumericType` into three methods and guarantee assertions are valid yangjie01
• (spark) branch master updated (2817654d439b -> 9a452f81dbdd) yangjie01
• (spark) branch master updated (cdb4f713402c -> 2817654d439b) gurwls223

• Earlier messages