Mailing lists matching spark.apache.org
commits spark.apache.orgdev spark.apache.org
issues spark.apache.org
reviews spark.apache.org
user spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34742: set the setContextClassLoader before using the `addJars` in `HiveClient`
HyukjinKwon commented on pull request #34742: URL: https://github.com/apache/spark/pull/34742#issuecomment-982181025 @kevincmchen mind filing a JIRA? see also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to the
[GitHub] [spark] HyukjinKwon commented on pull request #34163: Pin actions to a full length commit SHA
HyukjinKwon commented on pull request #34163: URL: https://github.com/apache/spark/pull/34163#issuecomment-932855811 Let's also file a JIRA to track these security issues (see also https://spark.apache.org/contributing.html). Otherwise, LGTM -- This is an automated message fro
Re: [PR] Make some corrections in the docstring of pyspark DataStreamReader methods [spark]
HyukjinKwon commented on PR #46416: URL: https://github.com/apache/spark/pull/46416#issuecomment-2099607939 Thanks for the fix. Mind filing a JIRA please? see also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to the
[GitHub] [spark] HyukjinKwon commented on pull request #40622: fix typo in ResourceRequest.equals()
HyukjinKwon commented on PR #40622: URL: https://github.com/apache/spark/pull/40622#issuecomment-1491509670 And please file a JIRA if this is an issue, see also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to the
[GitHub] [spark] HyukjinKwon commented on pull request #40637: Modify yarn client application report logging frequency to reduce noise
HyukjinKwon commented on PR #40637: URL: https://github.com/apache/spark/pull/40637#issuecomment-1493483696 Mind filing a JIRA please? See also https://spark.apache.org/contributing.html. -- This is an automated message from the Apache Git Service. To respond to the message, please log
[GitHub] [spark] HyukjinKwon commented on pull request #41686: Change handling of iterable objects for `on` field in joins
HyukjinKwon commented on PR #41686: URL: https://github.com/apache/spark/pull/41686#issuecomment-1600611015 Mind creating a JIRA please? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to the message, please log
Re: [PR] Refactor file listing with ScanFileListing interface [spark]
HyukjinKwon commented on PR #45224: URL: https://github.com/apache/spark/pull/45224#issuecomment-1963105731 Seems fine but mind filing a JIRA and put it into the PR title? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To
Re: [PR] Integrate range scan encoder changes with timer implementation [spark]
HyukjinKwon commented on PR #45709: URL: https://github.com/apache/spark/pull/45709#issuecomment-2019244232 @jingz-db mind liking the JIRA into the PR title? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to the
[GitHub] [spark] HyukjinKwon commented on pull request #42554: Make StreamingRelationV2 support metadata column
HyukjinKwon commented on PR #42554: URL: https://github.com/apache/spark/pull/42554#issuecomment-1685525915 Mind filing a JIRA please? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on
[GitHub] [spark] HyukjinKwon commented on pull request #40798: fix: name docker users
HyukjinKwon commented on PR #40798: URL: https://github.com/apache/spark/pull/40798#issuecomment-1510632838 @DerekTBrown Mind creating a JIRA and add it to the PR title please? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service
[GitHub] [spark] HyukjinKwon commented on pull request #40993: Make it possible to extend `ChannelBuilder` for `SparkConnectClient`
HyukjinKwon commented on PR #40993: URL: https://github.com/apache/spark/pull/40993#issuecomment-1531054320 @nfx Mind filing a JIRA please? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to the message, please
[GitHub] [spark] HyukjinKwon commented on pull request #41067: [KUBERNETES] Add configuration for pod memory limits
HyukjinKwon commented on PR #41067: URL: https://github.com/apache/spark/pull/41067#issuecomment-1537619340 Mind filing a JIRA? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to
Re: [PR] Do not convert array type string retrieved from jdbc driver [spark]
HyukjinKwon commented on PR #44459: URL: https://github.com/apache/spark/pull/44459#issuecomment-1869181617 Thanks for the PR. Mind creating a JIRA please? (see also https://spark.apache.org/contributing.html). -- This is an automated message from the Apache Git Service. To respond to
Re: Override Logging with spark-streaming
Any ideas ? Tks Nicolas - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org - To unsubscri
Re: Save DataFrame to HBase
Thu, Apr 21, 2016 at 6:52 AM, Benjamin Kim <mailto:bbuil...@gmail.com>> wrote: > Has anyone found an easy way to save a DataFrame into HBase? > > Thanks, > Ben > > > - > To unsubscribe, e-mail
Re: Streaming json records from kafka ... how can I process ... help please :)
Akhil wrote > You can do it like this: > > lines.foreachRDD(jsonRDD =>{ > > val data = sqlContext.read.json(jsonRDD) > data.registerTempTable("mytable") > sqlContext.sql("SELECT * FROM mytable") >
Re: Removing empty partitions before we write to HDFS
/Removing-empty-partitions-before-we-write-to-HDFS-tp24156.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user
Re: Categorical Features for K-Means Clustering
spark-user-list.1001560.n3.nabble.com/Categorical-Features-for-K-Means-Clustering-tp9416p14394.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubsc
Re: Unable to use HiveContext in spark-shell
/Unable-to-use-HiveCont >ext-in-spark-shell-tp18261p18280.html >Sent from the Apache Spark User List mailing list archive at Nabble.com. > >- >To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >For additi
RE: Spark Streaming Python APIs?
Cool thanks! Xiaoyong From: Shao, Saisai [mailto:saisai.s...@intel.com] Sent: Monday, December 15, 2014 10:57 AM To: Xiaoyong Zhu Cc: user@spark.apache.org Subject: RE: Spark Streaming Python APIs? AFAIK, this will be a new feature in version 1.2, you can check out the master branch or 1.2
[spark] branch branch-3.3 updated: [SPARK-40547][DOCS] Fix dead links in sparkr-vignettes.Rmd
. Should use tools::R_user_dir(). Found the following (possibly) invalid URLs: URL: https://spark.apache.org/docs/latest/api/R/column_aggregate_functions.html From: inst/doc/sparkr-vignettes.html Status: 404 Message: Not Found URL: https
[spark] branch master updated: [SPARK-40547][DOCS] Fix dead links in sparkr-vignettes.Rmd
. Should use tools::R_user_dir(). Found the following (possibly) invalid URLs: URL: https://spark.apache.org/docs/latest/api/R/column_aggregate_functions.html From: inst/doc/sparkr-vignettes.html Status: 404 Message: Not Found URL: https
[beam-site] 06/11: Updates to streaming connectors blog post
file:// URI) - https://beam.apache.org/documentation/sdks/javadoc/2.5.0/org/apache/beam/sdk/io/TextIO.html";>TextIO + https://beam.apache.org/documentation/sdks/javadoc/2.6.0/org/apache/beam/sdk/io/TextIO.html";>TextIO https://spark.apache.org/docs/latest/api/java/
[beam] branch asf-site updated: Publishing website 2019/03/21 15:32:43 at commit 133c56d
/index.html +++ b/website/generated-content/documentation/runners/spark/index.html @@ -224,8 +224,8 @@ The Spark Runner can execute Spark pipelines just like a native Spark applicatio Batch and streaming (and combined) pipelines. - The same fault-tolerance http://spark.apache.org/docs/1.6.3
spark-website git commit: Add pyspark instructions
a/site/sitemap.xml b/site/sitemap.xml index 0ce546f..73d60ea 100644 --- a/site/sitemap.xml +++ b/site/sitemap.xml @@ -652,27 +652,27 @@ weekly - https://spark.apache.org/graphx/ + https://spark.apache.org/sql/ weekly - https://spark.apache.org/mllib/ + https://spark.apache.org/stream
Re: spark on kubernetes
;>> information about their endpoint when application/worker register with >>>>> master, so when a user initiate a request to access the information, >>>>> master can proxy the request to corresponding endpoint. >>>>> >>>>&g
RE: NullWritable not serializable
amp;& finalStage.parents.isEmpty && partitions.length == 1 I'm wondering whether by default "running locally" is disabled. Yan From: Du Li [mailto:l...@yahoo-inc.com.INVALID] Sent: Tuesday, September 16, 2014 12:26 PM To: Matei Zaharia Cc: u...@spark.apache.org; dev@spark.apache.or
Re: RE : Re: HDFS small file generation problem
Thanks a lot, why you said "the most recent version" ? - Mail original - De: "Jörn Franke" À: "nibiau" Cc: banto...@gmail.com, user@spark.apache.org Envoyé: Samedi 3 Octobre 2015 13:56:43 Objet: Re: RE : Re: HDFS small file generation problem Yes the m
Re: spark on kubernetes
e with the Spark master Web UI to access the worker and >>>>>>>>application UI links. In brief, kubernetes service model allows me >>>>>>>> to >>>>>>>>expose the master service to internet, but accessing the >>&g
[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos
Github user Lemonjing commented on the issue: https://github.com/apache/spark/pull/20897 see http://spark.apache.org/docs/latest/ml-features.html#elementwiseproduct --- - To unsubscribe, e-mail: reviews-unsubscr
Re: unsubscribe
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org Thanks Best Regards On Mon, Nov 3, 2014 at 5:53 PM, Karthikeyan Arcot Kuppusamy < karthikeyan...@zanec.com> wrote: > hi > > - > To unsubscribe, e-m
[beam-site] 08/11: Update doc links in blog post to point to latest release
pport for input streaming connectors in [Apache Beam](https://beam.apache.org/). For more context, you'll also learn about the corresponding state of support in [Apache Spark](https://spark.apache.org/). +In this post, you'll learn about the current state of support for input streamin
git commit: Remove remaining references to incubation
om.xml +++ b/assembly/pom.xml @@ -21,14 +21,14 @@ org.apache.spark spark-parent -1.0.0-incubating-SNAPSHOT +1.0.0-SNAPSHOT ../pom.xml org.apache.spark spark-assembly_2.10 Spark Project Assembly - http://spark.incubator.apache.org/ + http://spark.apac
[spark] branch branch-3.1 updated: [SPARK-34021][R] Fix hyper links in SparkR documentation for CRAN submission
redirected now. This PR checked all hyperlinks in the docs such as `href{...}` and `url{...}`, and fixed all in SparkR: - Fix two problems above. - Fix http to https - Fix `https://www.apache.org/ https://spark.apache.org/` -> `https://www.apache.org https://spark.apache.
[GitHub] spark issue #20212: Update rdd-programming-guide.md
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20212 OK consider that and http://spark.apache.org/contributing.html for the future. I'll just merge this. --- - To unsubscri
[GitHub] spark issue #20372: Improved block merging logic for partitions
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20372 please see https://spark.apache.org/contributing.html open a JIRA and update this PR? --- - To unsubscribe, e-mail
[GitHub] spark issue #21893: Support selecting from partitioned tabels with partition...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21893 Please review http://spark.apache.org/contributing.html before opening a pull request. --- - To unsubscribe, e-mail
[GitHub] spark issue #21893: Support selecting from partitioned tabels with partition...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21893 Mind filing a JIRA please? Please see http://spark.apache.org/contributing.html --- - To unsubscribe, e-mail: reviews
[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21092 @lucashu1 please send your question to stackoverflow or u...@spark.apache.org! --- - To unsubscribe, e-mail: reviews
[GitHub] spark issue #21597: [SPARK-24603] Fix findTightestCommonType reference in co...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21597 @Fokko, thanks for bearing with it. (see also https://spark.apache.org/contributing.html). --- - To unsubscribe, e-mail
[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21669 btw, have you sent out this + doc to d...@spark.apache.org? --- - To unsubscribe, e-mail: reviews-unsubscr
[GitHub] spark issue #19489: The declared package "org.apache.hive.service.cli.thrift...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19489 This should be closed no matter what. Please start at the web site. http://spark.apache.org/community.html --- - To unsubscribe
[GitHub] spark issue #23107: small question in Spillable class
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23107 Hi, @Charele . Could you read http://spark.apache.org/community.html ? You had better close this. :) --- - To
[GitHub] spark issue #23107: small question in Spillable class
Github user srowen commented on the issue: https://github.com/apache/spark/pull/23107 Please send questions to u...@spark.apache.org; this should be closed. --- - To unsubscribe, e-mail: reviews-unsubscr
Berlin Apache Spark Meetup
Hi, there is a small Spark Meetup group in Berlin, Germany :-) http://www.meetup.com/Berlin-Apache-Spark-Meetup/ Plaes add this group to the Meetups list at https://spark.apache.org/community.html Ralph - To unsubscribe, e
[GitHub] [spark-website] srowen commented on a change in pull request #350: [SPARK-36335] Add local-cluster docs to developer-tools.md
srowen commented on a change in pull request #350: URL: https://github.com/apache/spark-website/pull/350#discussion_r679162872 ## File path: site/sitemap.xml ## @@ -907,10 +907,6 @@ https://spark.apache.org/mllib/ weekly - Review comment: (Revert this change
[GitHub] [spark-website] yutoacts commented on a change in pull request #350: [SPARK-36335] Add local-cluster docs to developer-tools.md
yutoacts commented on a change in pull request #350: URL: https://github.com/apache/spark-website/pull/350#discussion_r679593398 ## File path: site/sitemap.xml ## @@ -907,10 +907,6 @@ https://spark.apache.org/mllib/ weekly - Review comment: Reverted. Thank you
[GitHub] [spark-website] dongjoon-hyun commented on a change in pull request #341: Update release process
dongjoon-hyun commented on a change in pull request #341: URL: https://github.com/apache/spark-website/pull/341#discussion_r633878110 ## File path: site/sitemap.xml ## @@ -876,27 +876,27 @@ weekly - https://spark.apache.org/screencasts/ + https://spark.apache.org
Re: Welcome two new Apache Spark committers
! >>> > >>> > Thanks, >>> > Cheng Pan >>> > >>> > >>> > > On Aug 7, 2023, at 10:58, Gengliang Wang >> > > <mailto:ltn...@gmail.com>> wrote: >>> > &
Re: Welcoming Tejas Patil as a Spark committer
Congratulations , Tejas! -- Dilip - Original message -From: Suresh Thalamati To: "dev@spark.apache.org" Cc:Subject: Re: Welcoming Tejas Patil as a Spark committerDate: Tue, Oct 3, 2017 12:01 PM Congratulations , Tejas!-suresh> On Sep 29, 2017, at 12:58 PM, Matei Zaharia
Re: [SPARK-5100][SQL] Spark Thrift server monitor page
ing a monitor page for Thrift server. https://issues.apache.org/jira/browse/SPARK-5100 Anyone could review the design doc, and give some advises? - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e
Re: Jenkins install reference
want to set up a CI env for spark in our team, is there any reference of how to install jenkins over spark? Thanks Fei - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h
Re: Welcoming three new committers
; past year: Cheng on Spark SQL, Joseph on MLlib, and Sean on ML and many > pieces throughout Spark Core. Join me in welcoming them as committers! > > Matei > - > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.o
Re: Welcoming three new committers
------ > > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > > For additional commands, e-mail: dev-h...@spark.apache.org > > > > > > ----- > To un
[ANNOUNCE] Announcing Spark 1.3!
atures, or download [2] the release today. For errata in the contributions or release notes, please e-mail me *directly* (not on-list). Thanks to everyone who helped work on this release! [1] http://spark.apache.org/releases/spark-release-1-3-0.html [2] http://spark.apache.org/down
Re: parallelize method v.s. textFile method
- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > ----- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
[jira] [Commented] (SPARK-21498) quick start -> one py demo have some bug in code
[ https://issues.apache.org/jira/browse/SPARK-21498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16097172#comment-16097172 ] Sean Owen commented on SPARK-21498: --- http://spark.apache.org/contributing.
[jira] [Resolved] (SPARK-38831) How to enable encryption for checkpoint data?
>Reporter: zoli >Priority: Major > > Setting spark.io.encryption.enabled to true as described here: > [https://spark.apache.org/docs/latest/security.html#local-storage-encryption > |https://spark.apache.org/docs/latest/security.html#local-storage-encryption]has > no
[jira] [Resolved] (SPARK-38831) How to enable encryption for checkpoint data?
>Reporter: zoli >Priority: Major > > Setting spark.io.encryption.enabled to true as described here: > [https://spark.apache.org/docs/latest/security.html#local-storage-encryption > |https://spark.apache.org/docs/latest/security.html#local-storage-encryption]has > no effe
[jira] [Resolved] (SPARK-38831) How to enable encryption for checkpoint data?
>Reporter: zoli >Priority: Major > > Setting spark.io.encryption.enabled to true as described here: > [https://spark.apache.org/docs/latest/security.html#local-storage-encryption > |https://spark.apache.org/docs/latest/security.html#local-storage-encryption]has > no
[jira] [Created] (SPARK-24499) Documentation improvement of Spark core and SQL
Feature Components: SQL Affects Versions: 2.3.0 Reporter: Xiao Li The current documentation in Apache Spark lacks enough code examples and tips. If needed, we should also split the page of https://spark.apache.org/docs/latest/sql-programming-guide.html to multiple separate
[jira] [Assigned] (SPARK-11395) Support over and window specification in SparkR
1.5.1 >Reporter: Sun Rui > > 1. implement over() in Column class. > 2. support window spec > (http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.expressions.WindowSpec) > 3. support utility functions for defining window in DataFrames. >
[jira] [Updated] (SPARK-12144) Support more external data source API in SparkR
Issue Type: New Feature > Components: SparkR >Affects Versions: 1.5.2 >Reporter: Sun Rui > > DataFrameReader API: > http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameReader > DataFrameWriter API: > http://spar
[jira] [Assigned] (SPARK-10759) Missing Python code example in ML Programming guide
on >Affects Versions: 1.5.0 >Reporter: Raela Wang >Assignee: Apache Spark >Priority: Minor > Labels: starter > > http://spark.apache.org/docs/latest/ml-guide.html#example-model-selection-via-cross-validation > http://spark.ap
[jira] [Created] (SPARK-33181) SQL Reference: Run SQL on files directly
Components: SQL Affects Versions: 3.0.1 Reporter: Xiao Li Currently, SQL reference ([https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select.html] ) does not show the feature "Run SQL on files directly", which is documented in [https://spark.apache.org/
[jira] [Updated] (SPARK-27800) Example for xor function has a wrong answer
[ https://issues.apache.org/jira/browse/SPARK-27800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-27800: - Description: See [https://spark.apache.org/docs/latest/api/sql/index.html#_14] 3 ^ 5 should be 6 rather
[jira] [Commented] (SPARK-10759) Missing Python code example in ML Programming guide
Components: Documentation >Affects Versions: 1.5.0 >Reporter: Raela Wang > Priority: Minor > > http://spark.apache.org/docs/latest/ml-guide.html#example-model-selection-via-cross-validation > http://spark.apache.org/docs/latest/ml-guide.html#example-model-s
[jira] [Updated] (SPARK-10759) Missing Python code example in ML Programming guide
1.5.0 >Reporter: Raela Wang >Assignee: Lauren Moos > Priority: Minor > > http://spark.apache.org/docs/latest/ml-guide.html#example-model-selection-via-cross-validation > http://spark.apache.org/docs/latest/ml-guide.html#example-model-selection-via-
[jira] [Updated] (SPARK-10759) Missing Python code example in ML Programming guide
1.5.0 >Reporter: Raela Wang >Assignee: Lauren Moos >Priority: Minor > Labels: starter > > http://spark.apache.org/docs/latest/ml-guide.html#example-model-selection-via-cross-validation > http://spark.apache.org/docs/latest
[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR
cre"), License: Apache License (== 2.0) URL: http://www.apache.org/ http://spark.apache.org/ BugReports: http://spark.apache.org/contributing.html +SystemRequirements: Java (== 8) Depends: R (>= 3.0), --- End diff -- Thats a valid point. Lets discu
Re: [PR] SPARK-48380: SerDeUtil.javaToPython to support batchSize parameter [spark]
HyukjinKwon commented on PR #46691: URL: https://github.com/apache/spark/pull/46691#issuecomment-2123609711 Let's: 1. Create a PR against `master` branch 2. Add a test case 3. Fix the PR title (see also https://spark.apache.org/contributing.html) -- This is an automated me
Re: [PR] [PYTHON] Fix pyspark.sql.functions.reduce docstring typo [spark]
zhengruifeng commented on PR #46923: URL: https://github.com/apache/spark/pull/46923#issuecomment-2167652003 thanks for this fix. would you mind enabling the github action? you may refer to the `Pull request` section in https://spark.apache.org/contributing.html -- This is an
Re: [PR] Add width attribute for shuffle-write-time checkbox. [spark]
HyukjinKwon commented on PR #47798: URL: https://github.com/apache/spark/pull/47798#issuecomment-2295471514 @xunxunmimi5577 mind adding a screenshot, and filing a JIRA with adding to the PR title? See also https://spark.apache.org/contributing.html -- This is an automated message from
Re: [PR] [SPARK-49294]Add width attribute for shuffle-write-time checkbox. [spark]
xunxunmimi5577 commented on PR #47798: URL: https://github.com/apache/spark/pull/47798#issuecomment-2295591297 > @xunxunmimi5577 mind adding a screenshot, and filing a JIRA with adding to the PR title? See also https://spark.apache.org/contributing.html Ok,all done. -- This is
Re: [PR] To solve the issue of generating excessively large execution plans when encountering multiple levels of subqueries while enabling DynamicPartitionPruning. [spark]
HyukjinKwon commented on PR #47911: URL: https://github.com/apache/spark/pull/47911#issuecomment-2316972323 Mind filing a JIRA please? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on
[GitHub] [spark] HyukjinKwon commented on pull request #35667: [K8S] Avoid possible errors due to incorrect file size or type supplied in hadoop conf
HyukjinKwon commented on pull request #35667: URL: https://github.com/apache/spark/pull/35667#issuecomment-1053752532 @lyssg mind linking the JIRA into the PR title please? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service
[GitHub] [spark] dongjoon-hyun commented on pull request #35110: [SPARK-37820][SQL] Replace ApacheCommonBase64 with JavaBase64 for string funcs
dongjoon-hyun commented on pull request #35110: URL: https://github.com/apache/spark/pull/35110#issuecomment-1006371310 Yes, Here it is, `Running benchmarks in your forked repository`. It's easy. - https://spark.apache.org/developer-tools.html -- This is an automated message fro
[GitHub] [spark] HyukjinKwon commented on pull request #35466: Use error classes in the execution errors of pivoting
HyukjinKwon commented on pull request #35466: URL: https://github.com/apache/spark/pull/35466#issuecomment-1034360957 Thanks @ivoson. Do you mind linking the JIRA ticket to the PR title? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache
[GitHub] [spark] ivoson commented on pull request #35466: [SPARK-38113][SQL]Use error classes in the execution errors of pivoting
ivoson commented on pull request #35466: URL: https://github.com/apache/spark/pull/35466#issuecomment-1034420777 > Thanks @ivoson. Do you mind linking the JIRA ticket to the PR title? See also https://spark.apache.org/contributing.html Thanks @HyukjinKwon . Done. -- This
[GitHub] [spark] HyukjinKwon commented on pull request #39021: Last metrics system report should have a timeout, avoid to lead shutdown hook timeout
HyukjinKwon commented on PR #39021: URL: https://github.com/apache/spark/pull/39021#issuecomment-1345751613 @packyan mind creating a JIRA and linking it to PR title? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To
[GitHub] [spark] panbingkun commented on pull request #36188: [SPARK-38725][SQL][TESTS] Test the error class: DUPLICATE_KEY
panbingkun commented on PR #36188: URL: https://github.com/apache/spark/pull/36188#issuecomment-1098643675 old pr: https://github.com/apache/spark/pull/36160, i will close it new pr follow the guide: https://spark.apache.org/contributing.html -- This is an automated message from the
[GitHub] [spark] bjornjorgensen commented on pull request #38262: [SPARK-40801][BUILD] Upgrade `Apache commons-text` to 1.10
bjornjorgensen commented on PR #38262: URL: https://github.com/apache/spark/pull/38262#issuecomment-1319181705 @vitas I think it is best to ask questions like that on the mailing list https://spark.apache.org/community.html under Mailing lists -- This is an automated message from the
[GitHub] [spark] HyukjinKwon commented on pull request #33332: [SQL] Warn if less files visible after stats write
HyukjinKwon commented on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-879571129 @tooptoop4 please refer to https://spark.apache.org/contributing.html and make the PR description and title properly with a jira. -- This is an automated message from the
[GitHub] [spark] HyukjinKwon commented on pull request #38381: Fix the LogicalRelation computeStats for Row-level Runtime Filtering cannot be applied
HyukjinKwon commented on PR #38381: URL: https://github.com/apache/spark/pull/38381#issuecomment-1289904503 @chenminghua8 mind linking the JIRA ticket into the PR title? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To
[GitHub] [spark] carlfu-db commented on pull request #38404: [WIP] Replace Where
carlfu-db commented on PR #38404: URL: https://github.com/apache/spark/pull/38404#issuecomment-1294192968 > Mind adding a test, filing a JIRA, etc? See also https://spark.apache.org/contributing.html Will do. Still in progress :) -- This is an automated message from the Apa
[GitHub] [spark] srowen commented on pull request #37795: fix the question of SparkSQL call iceberg's expire_snapshots procedur…
srowen commented on PR #37795: URL: https://github.com/apache/spark/pull/37795#issuecomment-1236386281 You're going to need to back up and do more homework here https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond t
[GitHub] [spark] itholic commented on pull request #37817: [SPARK-40376] Avoid Numpy deprecation warning
itholic commented on PR #37817: URL: https://github.com/apache/spark/pull/37817#issuecomment-1240139388 Can we add a `[PYTHON]` tag for the title ? Also check the https://spark.apache.org/contributing.html out when you find some time. -- This is an automated message from the
[GitHub] [spark] dongjoon-hyun commented on pull request #34948: [SPARK-37684][BUILD] Upgrade log4j to 2.17
dongjoon-hyun commented on pull request #34948: URL: https://github.com/apache/spark/pull/34948#issuecomment-1000121841 @Niranjan-R-S . Apache Spark has a release cycle and this is for Apache Spark 3.3 on 2022 April+. - https://spark.apache.org/versioning-policy.html -- This is an
[GitHub] [spark] c21 commented on pull request #33711: Fix bug in disable unnecessary bucketed scan
c21 commented on pull request #33711: URL: https://github.com/apache/spark/pull/33711#issuecomment-897086852 I confirm this is a valid bug, thanks for flagging this @IonutBoicuAms. Could you file a JIRA as well (more info - https://spark.apache.org/contributing.html)? Thanks. -- This
[GitHub] [spark] HyukjinKwon commented on pull request #34613: [SPARK-37342][BUILD] Upgrade Apache Arrow to 6.0.0
HyukjinKwon commented on pull request #34613: URL: https://github.com/apache/spark/pull/34613#issuecomment-969620513 Oh, it should better be tested @sunchao. Running it regularly should work (https://spark.apache.org/developer-tools.html) but with `pip install pyarrow==6.0.0` -- This
[GitHub] [spark] AngersZhuuuu commented on pull request #34715: [WIP][SPARK-37445][BUILD] Rename the maven hadoop profile to hadoop-3 and hadoop-2
AngersZh commented on pull request #34715: URL: https://github.com/apache/spark/pull/34715#issuecomment-979677759 We may need to clarify this change in https://spark.apache.org/docs/latest/api/python/getting_started/install.html#manually-downloading ? -- This is an automated
[GitHub] [spark] naveensrinivasan commented on pull request #34163: Pin actions to a full length commit SHA
naveensrinivasan commented on pull request #34163: URL: https://github.com/apache/spark/pull/34163#issuecomment-932989471 > Let's also file a JIRA to track these security issues (see also https://spark.apache.org/contributing.html). Otherwise, LGTM Do you want me to cr
[GitHub] [spark] HyukjinKwon commented on pull request #33948: Use sed instead of profile to enable scala-parallel-collections on 2.13
HyukjinKwon commented on pull request #33948: URL: https://github.com/apache/spark/pull/33948#issuecomment-916551358 Thanks for contribution, @lrytz. Would you mind filing a JIRA and linking it to the PR title? See also https://spark.apache.org/contributing.html -- This is an automated
Re: [PR] [SPARK-47932][SQL][TEST] Avoid using legacy commons-lang [spark]
pan3793 commented on PR #46154: URL: https://github.com/apache/spark/pull/46154#issuecomment-2068608865 @smileyboy2019 Please STOP spamming on this thread. You can refer to https://spark.apache.org/community.html to subscribe to Spark Mailing lists and ask questions. -- This is an
[GitHub] [spark] srowen commented on pull request #41801: SSH Environment Manager
srowen commented on PR #41801: URL: https://github.com/apache/spark/pull/41801#issuecomment-1614717648 We can't see that notebook. You need a JIRA and a bit more context about how this arises. https://spark.apache.org/contributing.html -- This is an automated message from the Apach
Re: [PR] Allow spark-class script to be ran in an environment where /dev/fd is unavailable (AWS Lambda) [spark]
HyukjinKwon commented on PR #45441: URL: https://github.com/apache/spark/pull/45441#issuecomment-1987424381 Mind filing a JIRA at https://issues.apache.org/jira/projects/SPARK? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service
Re: [PR] [SPARK-47570][SS] Integrate range scan encoder changes with timer implementation [spark]
jingz-db commented on PR #45709: URL: https://github.com/apache/spark/pull/45709#issuecomment-2021101002 > @jingz-db mind liking the JIRA into the PR title? See also https://spark.apache.org/contributing.html Just did. Thanks! -- This is an automated message from the Apache
Re: [PR] [SPARK-46894][PYTHON] Move PySpark error conditions into standalone JSON file [spark]
HyukjinKwon commented on PR #44920: URL: https://github.com/apache/spark/pull/44920#issuecomment-2026650601 People can actually directly use PySpark via importing `pyspark.zip`, see https://spark.apache.org/docs/latest/api/python/getting_started/install.html?highlight=pythonpath#manually
[GitHub] [spark] srowen commented on pull request #42207: [SPARK-XXXX][MLLIB] Fix warning condition in MLLib RankingMetrics ndcgAk
srowen commented on PR #42207: URL: https://github.com/apache/spark/pull/42207#issuecomment-1655539263 Also, this needs a JIRA, and needs to be connected in the title https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to
[GitHub] [spark] HyukjinKwon commented on pull request #42541: Spark 44854
HyukjinKwon commented on PR #42541: URL: https://github.com/apache/spark/pull/42541#issuecomment-1685546533 Seems pretty good - mind retriggering https://github.com/hdaly0/spark/runs/15986658618 please? Also please fix the PR title (see also https://spark.apache.org
