commits  

Messages by Thread

• (spark) branch master updated (d823ccfc3c5d -> 5fb072e4f25f) ruifengz
• (spark) branch master updated: [SPARK-44571][SQL][FOLLOWUP][DOCS] Update Scaladoc of `MergeSubplans` yangjie01
• (spark) branch branch-4.1 updated: [SPARK-54375][CONNECT][TESTS][FOLLOWUP] Make `PythonPipelineSuite` perform a default check for `PythonTestDepsChecker.isConnectDepsAvailable` yangjie01
• (spark) branch master updated: [SPARK-54375][CONNECT][TESTS][FOLLOWUP] Make `PythonPipelineSuite` perform a default check for `PythonTestDepsChecker.isConnectDepsAvailable` yangjie01
• (spark) branch master updated (a6fda315f8e1 -> dff06206e211) wenchen
• (spark) branch dependabot/npm_and_yarn/ui-test/glob-10.5.0 deleted (was d7c117c172ac) github-bot
• (spark) branch dependabot/npm_and_yarn/ui-test/glob-10.5.0 created (now d7c117c172ac) github-bot
• (spark) branch master updated (2d7e9a77db80 -> a6fda315f8e1) ruifengz
• (spark) branch master updated (cd1601bb8b1b -> 2d7e9a77db80) ruifengz
• (spark) branch master updated (6227fbab0308 -> cd1601bb8b1b) ashrigondekar
• (spark) branch branch-4.1 updated: Revert "[SPARK-54349][PYTHON] Refactor code a bit to simplify faulthandler integration extension" ueshin
• (spark) branch branch-4.1 updated: [SPARK-54349][PYTHON] Refactor code a bit to simplify faulthandler integration extension ueshin
  • (spark) branch branch-4.1 updated: [SPARK-54349][PYTHON] Refactor code a bit to simplify faulthandler integration extension dongjoon
• (spark) branch master updated: [SPARK-54349][PYTHON] Refactor code a bit to simplify faulthandler integration extension ueshin
• (spark) branch master updated (13fea4fa02c6 -> 894a7e8993ed) dongjoon
• (spark) branch master updated (1012a5ffa51b -> 13fea4fa02c6) sunchao
• (spark) branch branch-4.1 updated: [SPARK-54163][SQL] Scan canonicalization for partitioning and ordering info gengliang
• (spark) branch master updated (05bc5d408fa9 -> 1012a5ffa51b) gengliang
• (spark) branch branch-4.1 updated: [SPARK-54350][SQL][STS] SparkGetColumnsOperation ORDINAL_POSITION should be 1-based dongjoon
• (spark) branch master updated (dce992b7708b -> 05bc5d408fa9) dongjoon
• (spark-connect-swift) branch main updated: [SPARK-54402] Upgrade `gRPC Swift NIO Transport` to 2.3.0 dongjoon
• (spark-connect-swift) branch main updated: [SPARK-54401] Upgrade `grpc-swift-2` to 2.2.0 dongjoon
• (spark-connect-swift) branch main updated: [SPARK-54398] Use 4.1.0-preview4 in `integration-test-(token|mac-spark41)` dongjoon
• (spark) branch master updated (aa387f32158a -> dce992b7708b) dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54387][SQL] Fix recaching of DSv2 tables dongjoon
• (spark) branch master updated: [SPARK-52767][SQL] Optimize maxRows and maxRowsPerPartition for join and union wenchen
• (spark) branch branch-4.1 updated: [SPARK-52767][SQL] Optimize maxRows and maxRowsPerPartition for join and union wenchen
• (spark) branch master updated (e8f0a67e248d -> 78fcc934d314) ptoth
• (spark) branch branch-4.1 updated: [SPARK-54377][SQL] Fix COMMENT ON TABLE IS NULL to properly remove table comment wenchen
• (spark) branch master updated (b03352b99bb9 -> e8f0a67e248d) wenchen
• (spark) branch master updated: [SPARK-54394][CORE] Move `isJavaVersionAtMost17` and `isJavaVersionAtLeast21` from `core` to `common/utils` yangjie01
• (spark) branch branch-4.1 updated: [SPARK-53924][FOLLOWUP][TESTS] Add tests for cached temp view detecting schema changes gengliang
• (spark) branch master updated: [SPARK-53924][FOLLOWUP][TESTS] Add tests for cached temp view detecting schema changes gengliang
• (spark) branch master updated (b7127004358a -> 35a99f84a352) yangjie01
• (spark) branch master updated: [SPARK-54317][PYTHON][CONNECT] Unify Arrow conversion logic for Classic and Connect toPandas ruifengz
• (spark) branch branch-4.1 updated: [SPARK-54339][SQL] Fix AttributeMap non-determinism wenchen
• (spark) branch master updated (a9c1fba603a2 -> 78d1d52601e4) wenchen
• (spark) branch master updated (1c283c01493a -> a9c1fba603a2) yangjie01
• (spark) branch master updated (19ff85950c35 -> 1c283c01493a) ashrigondekar
• (spark) branch master updated: [SPARK-54379] [SQL] Move lambda binding to separate `LambdaBinder` object dtenedor
• (spark-website) branch asf-site updated: Add RDD Programming Guide to llms.txt (#648) allisonwang
• [PR] Add RDD Programming Guide to llms.txt [spark-website] via GitHub
  • Re: [PR] Add RDD Programming Guide to llms.txt [spark-website] via GitHub
  • Re: [PR] Add RDD Programming Guide to llms.txt [spark-website] via GitHub
• (spark) branch branch-4.1 updated: [SPARK-54344][PYTHON] Kill the worker if flush fails in daemon.py gurwls223
• (spark) branch master updated: [SPARK-54344][PYTHON] Kill the worker if flush fails in daemon.py gurwls223
• (spark) branch master updated (1db267e3bd02 -> 21150230f455) gurwls223
• (spark) branch branch-4.1 updated: [SPARK-54376][SDP] Mark most pipeline configuration options as internal dongjoon
• (spark) branch master updated: [SPARK-54376][SDP] Mark most pipeline configuration options as internal dongjoon
• (spark) branch master updated: [SPARK-54378][SQL] Remove `CreateXmlParser.scala` from `catalyst` module yangjie01
• (spark) branch master updated (3757091e1c51 -> 722bcc0f0d15) yangjie01
• (spark) branch branch-4.1 updated: [SPARK-54375][CONNECT][TESTS] Add `assume` to cases in `PythonPipelineSuite` to skip tests when PyConnect dependencies is not available yangjie01
• (spark) branch branch-4.1 updated: [SPARK-54319][SQL] BHJ LeftAnti update numOutputRows wrong when codegen is disabled wenchen
• (spark) branch master updated: [SPARK-54319][SQL] BHJ LeftAnti update numOutputRows wrong when codegen is disabled wenchen
• (spark) branch master updated (0a42f557a8d2 -> 96e0d4b94247) wenchen
• svn commit: r80749 - in dev/spark/v4.1.0-preview4-rc1-docs: . _site _site/api _site/api/R _site/api/R/articles _site/api/R/articles/sparkr-vignettes_files _site/api/R/articles/sparkr-vignettes_files/accessible-code-block-0.0.1 _site/api/R/deps _site/ap... dongjoon
• svn commit: r80748 - dev/spark/v4.1.0-preview4-rc1-bin dongjoon
• (spark) tag v4.1.0-preview4-rc1 created (now c125aea395b3) dongjoon
  • (spark) 01/02: Removing test jars and class files dongjoon
  • (spark) 02/02: Preparing Spark release v4.1.0-preview4-rc1 dongjoon
• (spark) branch master updated: [SPARK-54370][INFRA] Limit the Maven GitHub Action job timeout to 150 minutes dongjoon
• (spark) branch branch-4.0 updated: [SPARK-54370][INFRA] Limit the Maven GitHub Action job timeout to 150 minutes dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54370][INFRA] Limit the Maven GitHub Action job timeout to 150 minutes dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54369][CONNECT][TESTS] Fix `PythonPipelineSuite` flakiness via `Set` instead of `Seq` dongjoon
• (spark) branch master updated: [SPARK-54369][CONNECT][TESTS] Fix `PythonPipelineSuite` flakiness via `Set` instead of `Seq` dongjoon
• (spark) branch branch-4.0 updated: [SPARK-54371][INFRA] Fix `spark-rm` Dockefile to install `pkgdown` version at the end dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54371][INFRA] Fix `spark-rm` Dockefile to install `pkgdown` version at the end dongjoon
• (spark) branch master updated: [SPARK-54371][INFRA] Fix `spark-rm` Dockefile to install `pkgdown` version at the end dongjoon
• (spark) branch branch-4.1 updated: [SPARK-53924] Reload DSv2 tables in views created using plans on each view access dongjoon
• (spark) branch master updated: [SPARK-53924] Reload DSv2 tables in views created using plans on each view access dongjoon
• (spark) branch branch-3.5 updated: [SPARK-54366][INFRA] Add `free_disk_space` step to K8s integration test GitHub Action job dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54366][INFRA] Add `free_disk_space` step to K8s integration test GitHub Action job dongjoon
• (spark) branch branch-4.0 updated: [SPARK-54366][INFRA] Add `free_disk_space` step to K8s integration test GitHub Action job dongjoon
• (spark) branch master updated (d02a6d490f1f -> 0311f44e33e5) dongjoon
• (spark) branch master updated (e3e986373735 -> d02a6d490f1f) dongjoon
• (spark) branch master updated (e09c99949107 -> e3e986373735) dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54355][CONNECT] Make `spark.connect.session.planCompression.defaultAlgorithm` to support `NONE` dongjoon
• (spark) branch dependabot/npm_and_yarn/dev/multi-9491a2a7cf deleted (was 38f0b0144dc5) github-bot
• (spark) branch master updated (cc72c647b6f9 -> e09c99949107) dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54358][SDP] Checkpoint dirs collide when streaming tables in different schemas have same name dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54020] Support `spark.sql(...)` Python API inside query functions for Spark Declarative Pipeline dongjoon
• (spark) branch master updated: [SPARK-54020] Support `spark.sql(...)` Python API inside query functions for Spark Declarative Pipeline dongjoon
• (spark) branch master updated (c5c65d2bf358 -> bb089184426d) dtenedor
• (spark) branch master updated (1214830dc16d -> c5c65d2bf358) dtenedor
• (spark) branch master updated (cf513611bf79 -> 1214830dc16d) dongjoon
• (spark) branch master updated: [SPARK-50906][SQL] Fix Avro nullability check for reordered struct fields gengliang
• (spark) branch branch-4.1 updated: [SPARK-50906][SQL] Fix Avro nullability check for reordered struct fields gengliang
• (spark) branch master updated (dc5bb9641b6e -> e739b7d349db) dongjoon
• (spark) branch branch-3.5 updated: [SPARK-53337][UI] XSS: Ensure the application name in historypage get escaped dongjoon
• (spark) branch branch-4.0 updated: [SPARK-53337][UI] XSS: Ensure the application name in historypage get escaped dongjoon
• (spark) branch branch-4.1 updated: [SPARK-53337][UI] XSS: Ensure the application name in historypage get escaped dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54172][SQL][FOLLOW-UP] Simplify attribute re-resolution and validate action assignment value resolution for schema evolution dongjoon
• (spark) branch master updated (32a3244c45da -> dc5bb9641b6e) dongjoon
• (spark) branch master updated (a916690d6101 -> 32a3244c45da) dtenedor
• (spark) branch branch-4.1 updated: [SPARK-54348][INFRA] Recover Python unit tests CI by installing `zstandard==0.25.0` dongjoon
• (spark) branch master updated (053617c4ba1e -> a916690d6101) dongjoon
• (spark) branch master updated (7599b2ffb28a -> 053617c4ba1e) dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54356][SDP] Fix EndToEndAPISuite caused by missing storage root schema dongjoon
• (spark) branch master updated (13e2765eb7ff -> 7599b2ffb28a) dongjoon
• (spark) branch master updated (59759f3646cf -> 13e2765eb7ff) dtenedor
• (spark) branch master updated: [SPARK-54157][SQL] Fix refresh of DSv2 tables in Dataset wenchen
• (spark) branch branch-4.1 updated: [SPARK-54157][SQL] Fix refresh of DSv2 tables in Dataset wenchen
• (spark) branch branch-4.1 updated: [SPARK-54114][CONNECT] Support getColumns for SparkConnectDatabaseMetaData dongjoon
• (spark) branch master updated (83d25b9ae588 -> be98a29875ce) dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54351][BUILD] Upgrade Dropwizard metrics to 4.2.37 dongjoon
• (spark) branch master updated (6cb88c10126b -> 83d25b9ae588) dongjoon
• (spark) branch dependabot/npm_and_yarn/dev/multi-9491a2a7cf created (now 38f0b0144dc5) github-bot
• (spark-kubernetes-operator) branch main updated: [SPARK-54328] Add configurable startupProbe and enhance liveness/readiness probes in Helm chart ptoth
• (spark) branch branch-4.1 updated: [SPARK-54194][CONNECT][FOLLOWUP] Spark Connect Proto Plan Compression - Scala Client hvanhovell
• (spark) branch master updated: [SPARK-54194][CONNECT][FOLLOWUP] Spark Connect Proto Plan Compression - Scala Client hvanhovell
• (spark) branch master updated (9f1bd47bab15 -> 551b922a53ac) ruifengz
• (spark) branch branch-4.1 updated: [SPARK-54332][PYTHON][CONNECT] No need to attach PlanId in grouping column names in rollup/cube/groupingSets wenchen
• (spark) branch master updated: [SPARK-54332][PYTHON][CONNECT] No need to attach PlanId in grouping column names in rollup/cube/groupingSets wenchen
• (spark) branch branch-4.1 updated: [SPARK-54341][SQL] Remember TimeTravelSpec for tables loaded via TableProvider dongjoon
• (spark) branch master updated: [SPARK-54341][SQL] Remember TimeTravelSpec for tables loaded via TableProvider dongjoon
• (spark) branch master updated (9f1dc1ccf8b7 -> 87b3b9423243) dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54209][CONNECT] [Followup] Fix linter error in `SparkConnectJdbcDataTypeSuite` dongjoon
• (spark-website) branch asf-site updated: Move llms.txt under docs gurwls223
• (spark) branch branch-4.1 updated: [SPARK-54209][CONNECT] Support TIMESTAMP type in SparkConnectResultSet dongjoon
• (spark) branch master updated: [SPARK-54209][CONNECT] Support TIMESTAMP type in SparkConnectResultSet dongjoon
• [PR] Move llms.txt under docs [spark-website] via GitHub
  • Re: [PR] Move llms.txt under docs [spark-website] via GitHub
  • Re: [PR] Move llms.txt under docs [spark-website] via GitHub
  • Re: [PR] Move llms.txt under docs [spark-website] via GitHub
  • Re: [PR] Move llms.txt under docs [spark-website] via GitHub
  • Re: [PR] Move llms.txt under docs [spark-website] via GitHub
  • Re: [PR] Move llms.txt under docs [spark-website] via GitHub
  • Re: [PR] Move llms.txt under docs [spark-website] via GitHub
• (spark) branch branch-4.1 updated: [SPARK-54280][SDP] Require pipeline checkpoint storage dir to be absolute path dongjoon
• (spark) branch master updated: [SPARK-54280][SDP] Require pipeline checkpoint storage dir to be absolute path dongjoon
• (spark-website) branch asf-site updated: Add llms.txt into the site's config gurwls223
• [PR] Add llms.txt into the site's config [spark-website] via GitHub
  • Re: [PR] Add llms.txt into the site's config [spark-website] via GitHub
  • Re: [PR] Add llms.txt into the site's config [spark-website] via GitHub
• (spark-website) branch add-committer-dtenedor deleted (was 22710d792b) gurwls223
• (spark-website) branch setEnv deleted (was 31da95c6aa) gurwls223
• (spark-website) branch fixSearch deleted (was 1e707b389e) gurwls223
• (spark-website) branch llms-txt deleted (was 92c12b5409) gurwls223
• (spark) branch branch-4.1 updated: [SPARK-54340][PYTHON] Add the capability to use viztracer on pyspark daemon/workers dongjoon
• (spark) branch master updated (7a6d3e18b132 -> e162b9b0007b) dongjoon
• (spark) branch master updated (2a6e6e55cea4 -> 7a6d3e18b132) dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54208][CONNECT] Support TIME type in SparkConnectResultSet dongjoon
• (spark) branch master updated: [SPARK-54208][CONNECT] Support TIME type in SparkConnectResultSet dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54331][PYTHON][TESTS] Optimize `pyspark.sql.tests.connect.test_connect_plan` dongjoon
• (spark) branch master updated: [SPARK-54331][PYTHON][TESTS] Optimize `pyspark.sql.tests.connect.test_connect_plan` dongjoon
• (spark) branch branch-3.5 updated: [SPARK-54336][SQL] Fix `BloomFilterMightContain` input type check with `ScalarSubqueryReference` dongjoon
• (spark) branch branch-4.0 updated: [SPARK-54336][SQL] Fix `BloomFilterMightContain` input type check with `ScalarSubqueryReference` dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54336][SQL] Fix `BloomFilterMightContain` input type check with `ScalarSubqueryReference` dongjoon
• (spark) branch master updated: [SPARK-54336][SQL] Fix `BloomFilterMightContain` input type check with `ScalarSubqueryReference` dongjoon
• (spark) branch dependabot/maven/hadoop-cloud/com.squareup.okhttp3-okhttp-4.9.2 deleted (was 68aa3b835287) github-bot
• (spark) branch branch-4.1 updated: [SPARK-54333][BUILD] Upgrade `commons-io` to 2.21.0 dongjoon
• (spark) branch master updated (840349ee51c8 -> 5e46f4c5fec4) dongjoon
• (spark) branch master updated: [SPARK-54334][SQL] Move the validation of subquery expressions under lambda and higher order functions to `SubqueryExpressionInLambdaOrHigherOrderFunctionValidator` wenchen
• (spark) branch branch-4.1 updated: [SPARK-54194][PYTHON][FOLLOWUP] Fix `connectutils.py` to import `pb2` conditionally dongjoon
• (spark) branch master updated: [SPARK-54194][PYTHON][FOLLOWUP] Fix `connectutils.py` to import `pb2` conditionally dongjoon
• (spark) branch dependabot/maven/hadoop-cloud/com.squareup.okhttp3-okhttp-4.9.2 created (now 68aa3b835287) github-bot
• (spark) branch branch-4.0 updated: [SPARK-54320][UI][4.0] Fix Job DAG overlapping sarutak
• (spark) branch master updated: [SPARK-54182][SQL][PYTHON] Optimize non-arrow conversion of `df.toPandas` ruifengz
• (spark) branch master updated: [SPARK-54318][PYTHON][DOCS] Fix doctests in `pyspark.sql.dataframe` ruifengz
• (spark) branch master updated (daa29f6ede29 -> d2188059d195) ruifengz
• (spark) branch master updated: [SPARK-54326][INFRA] Recover MacOS CIs by installing `zstandard==0.25.0` dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54326][INFRA] Recover MacOS CIs by installing `zstandard==0.25.0` dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54315][PYTHON][TESTS] Optimize test `ApplyInArrowTests.test_arrow_batch_slicing` ruifengz
• (spark) branch master updated: [SPARK-54315][PYTHON][TESTS] Optimize test `ApplyInArrowTests.test_arrow_batch_slicing` ruifengz
• (spark) branch master updated: [SPARK-54310][SQL] Add `numSourceRows` metric for `MergeIntoExec` dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54310][SQL] Add `numSourceRows` metric for `MergeIntoExec` dongjoon
• (spark-website) branch asf-site updated: Add llms.txt file to Spark documentation root (#645) allisonwang
• Re: [PR] Add llms.txt file to Spark documentation root [spark-website] via GitHub
  • Re: [PR] Add llms.txt file to Spark documentation root [spark-website] via GitHub
  • Re: [PR] Add llms.txt file to Spark documentation root [spark-website] via GitHub
  • Re: [PR] Add llms.txt file to Spark documentation root [spark-website] via GitHub
• (spark) branch branch-4.1 updated: [SPARK-54323][PYTHON] Change the way to access logs to TVF instead of system view dongjoon
• (spark) branch master updated: [SPARK-54323][PYTHON] Change the way to access logs to TVF instead of system view dongjoon
• (spark) branch master updated (4eb56bb65419 -> 05b054315b84) wenchen
• (spark) branch branch-4.1 updated: [SPARK-54240] Translate get array item catalyst expression to connector expression wenchen
• (spark) branch master updated (ecaec3d1b013 -> 4eb56bb65419) dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54206][CONNECT] Support BINARY type data in SparkConnectResultSet dongjoon
• (spark) branch master updated (03eb023c3a99 -> ecaec3d1b013) dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54172][SQL] Merge Into Schema Evolution should only add referenced columns dongjoon
• (spark) branch master updated (1a802e36ed76 -> 03eb023c3a99) dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54320][UI] Fix Job DAG overlapping dongjoon
• (spark) branch master updated: [SPARK-54320][UI] Fix Job DAG overlapping dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54306] Annotate Variant columns with Variant logical type annotation wenchen
• (spark) branch master updated: [SPARK-54306] Annotate Variant columns with Variant logical type annotation wenchen
• (spark) branch master updated: [SPARK-54301][SQL][TESTS] Enhance Spark SQL test suites for easier integration with other projects yumwang
• (spark) branch branch-4.1 updated: [SPARK-53917][CONNECT] Support large local relations - follow-ups hvanhovell
• (spark) branch master updated: [SPARK-53917][CONNECT] Support large local relations - follow-ups hvanhovell
• (spark) branch branch-4.1 updated: [SPARK-54130][SQL] Add detailed error messages for catalog assertion failures wenchen
• (spark) branch master updated: [SPARK-54130][SQL] Add detailed error messages for catalog assertion failures wenchen
• (spark) branch master updated: [SPARK-54183][PYTHON][CONNECT] Avoid one intermediate temp data frame during spark connect toPandas() ruifengz
• (spark) branch master updated: [SPARK-54207][CONNECT] Supports Date type data in SparkConnectResultSet dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54207][CONNECT] Supports Date type data in SparkConnectResultSet dongjoon
• (spark) branch branch-4.1 updated: [SPARK-54113][CONNECT] Support getTables for SparkConnectDatabaseMetaData dongjoon
• (spark) branch master updated: [SPARK-54113][CONNECT] Support getTables for SparkConnectDatabaseMetaData dongjoon
• (spark) branch master updated: [SPARK-54307][SS] Throw an error if streaming query is restarted with stateful op but there is empty state dir ashrigondekar
• (spark) branch master updated: [SPARK-54300][PYTHON] Optimize Py4J calls in `df.toPandas` ruifengz
• (spark) branch master updated (03a0e05acfcf -> c21d5a45073f) yangjie01
• (spark) branch branch-4.1 updated: [SPARK-54270][CONNECT] SparkConnectResultSet get* methods should call checkOpen and check index boundary yangjie01
• (spark) branch master updated: [SPARK-52439][SQL] Support check constraint with null value dongjoon
• (spark) branch branch-4.1 updated: [SPARK-52439][SQL] Support check constraint with null value dongjoon

• Later messages