[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550266#comment-16550266 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] I'm still not su

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550182#comment-16550182 ] Saisai Shao commented on SPARK-24723: - Hi [~mengxr], I don't think YARN

[jira] [Assigned] (SPARK-24195) sc.addFile for local:/ path is broken

[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24195: --- Assignee: Li Yuanjian > sc.addFile for local:/ path is bro

[jira] [Resolved] (SPARK-24195) sc.addFile for local:/ path is broken

[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24195. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21533 [https

[jira] [Commented] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550167#comment-16550167 ] Saisai Shao commented on SPARK-24307: - Issue resolved by pull request 21440 [h

[jira] [Resolved] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24307. - Resolution: Fixed Assignee: Imran Rashid > Support sending messages over 2GB from mem

[jira] [Updated] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24307: Fix Version/s: 2.4.0 > Support sending messages over 2GB from mem

[jira] [Updated] (SPARK-24037) stateful operators

[ https://issues.apache.org/jira/browse/SPARK-24037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24037: Fix Version/s: (was: 2.4.0) > stateful operators > -- > >

[jira] [Updated] (SPARK-24037) stateful operators

[ https://issues.apache.org/jira/browse/SPARK-24037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24037: Fix Version/s: 2.4.0 > stateful operators > -- > >

Re: [VOTE] SPARK 2.3.2 (RC3)

PR is to get rid of AnalysisBarrier. That is better than multiple > patches we added for AnalysisBarrier after 2.3.0 release. We can target it > to 2.4. > > Thanks, > > Xiao > > 2018-07-19 17:48 GMT-07:00 Saisai Shao : > >> I see, thanks Reynold. >> >> R

Re: [VOTE] SPARK 2.3.2 (RC3)

hink my ticket should block this release. It's a big general >> refactoring. >> >> Xiao do you have a ticket for the bug you found? >> >> >> On Thu, Jul 19, 2018 at 5:24 PM Saisai Shao >> wrote: >> >>> Hi Xiao, >>> >>> Are y

Re: [VOTE] SPARK 2.3.2 (RC3)

Hi Xiao, Are you referring to this JIRA ( https://issues.apache.org/jira/browse/SPARK-24865)? Xiao Li 于2018年7月20日周五 上午2:41写道： > dfWithUDF.cache() > dfWithUDF.write.saveAsTable("t") > dfWithUDF.write.saveAsTable("t1") > > > Cached data is not being used. It causes a big performance regression. >

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16549276#comment-16549276 ] Saisai Shao commented on SPARK-24615: - Thanks [~tgraves] for the sugges

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16548662#comment-16548662 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] I'm rewriting the d

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547263#comment-16547263 ] Saisai Shao commented on SPARK-24615: - Sure, I will also add it as Xiangrui

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16546590#comment-16546590 ] Saisai Shao commented on SPARK-24615: - Yes, currently the user is responsible

Re: [VOTE] SPARK 2.3.2 (RC3)

nd > should not block release. > > On Sun, Jul 15, 2018 at 9:39 PM Saisai Shao > wrote: > >> Hi Sean, >> >> I just did a clean build with mvn/sbt on 2.3.2, I didn't meet the errors >> you pasted here. I'm not sure how it happens. >> >&

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545962#comment-16545962 ] Saisai Shao commented on SPARK-24615: - Sorry [~tgraves] for the late response.

Re: [VOTE] SPARK 2.3.2 (RC3)

with > `-Ylog-classpath` to see the problematic classpath.) > > [error] A full rebuild may help if 'MetricsSystem.class' was compiled > against an incompatible version of org.eclipse > > On Sun, Jul 15, 2018 at 3:09 AM Saisai Shao > wrote: > >> Please

[VOTE] SPARK 2.3.2 (RC3)

Please vote on releasing the following candidate as Apache Spark version 2.3.2. The vote is open until July 20 PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 2.3.2 [ ] -1 Do not release this package because ... To lea

[jira] [Comment Edited] (SPARK-24804) There are duplicate words in the title in the DatasetSuite

[ https://issues.apache.org/jira/browse/SPARK-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16544386#comment-16544386 ] Saisai Shao edited comment on SPARK-24804 at 7/15/18 1:2

[jira] [Commented] (SPARK-24804) There are duplicate words in the title in the DatasetSuite

[ https://issues.apache.org/jira/browse/SPARK-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16544386#comment-16544386 ] Saisai Shao commented on SPARK-24804: - Please don't set target versi

[jira] [Updated] (SPARK-24804) There are duplicate words in the title in the DatasetSuite

[ https://issues.apache.org/jira/browse/SPARK-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24804: Priority: Trivial (was: Minor) > There are duplicate words in the title in the DatasetSu

[jira] [Updated] (SPARK-24804) There are duplicate words in the title in the DatasetSuite

[ https://issues.apache.org/jira/browse/SPARK-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24804: Target Version/s: (was: 2.3.2) > There are duplicate words in the title in the DatasetSu

[jira] [Updated] (SPARK-24804) There are duplicate words in the title in the DatasetSuite

[ https://issues.apache.org/jira/browse/SPARK-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24804: Fix Version/s: (was: 2.3.2) > There are duplicate words in the title in the DatasetSu

Re: [VOTE] SPARK 2.3.2 (RC1)

hon docs are usable or not in > this RC. They looked reasonable to me but I don't know enough to know what > the issue was. If the result is usable, then there's no problem here, even > if something could be fixed/improved later. > > On Sun, Jul 8, 2018 at 7:25 PM Saisai Shao

Re: Some questions about cached data in Livy

Hi Wandong, Livy's shared object mechanism mainly used to share objects between different Livy jobs, this is mainly used for Job API. For example job A create a object Foo which wants to be accessed by Job B, then user could store this object Foo into JobContext with a provided name, after that Jo

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539635#comment-16539635 ] Saisai Shao commented on SPARK-24781: - I see. I will wait for this before cutti

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539630#comment-16539630 ] Saisai Shao commented on SPARK-24781: - Thanks Felix. Does this have to be in 2

Re: [VOTE] SPARK 2.3.2 (RC1)

https://issues.apache.org/jira/browse/SPARK-24530 is just merged, I will cancel this vote and prepare a new RC2 cut with doc fixed. Thanks Saisai Wenchen Fan 于2018年7月11日周三 下午12:25写道： > +1 > > On Wed, Jul 11, 2018 at 1:31 AM John Zhuge wrote: > >> +1 >> >> On Sun,

[jira] [Assigned] (SPARK-24678) We should use 'PROCESS_LOCAL' first for Spark-Streaming

[ https://issues.apache.org/jira/browse/SPARK-24678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24678: --- Assignee: sharkd tu > We should use 'PROCESS_LOCAL' first for Sp

[jira] [Resolved] (SPARK-24678) We should use 'PROCESS_LOCAL' first for Spark-Streaming

[ https://issues.apache.org/jira/browse/SPARK-24678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24678. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21658 [https

[jira] [Resolved] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24646. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21633 [https

[jira] [Assigned] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24646: --- Assignee: Saisai Shao > Support wildcard '*' for to spark.yarn.dist.forceDo

Re: Podling report this month

Updated. Please review and sign off. Thanks Saisai Shao 于2018年7月9日周一 上午8:27写道： > Sorry I forgot about this, will draft this today. > > Thanks > Saisai > > Jean-Baptiste Onofré 于2018年7月8日周日 下午1:17写道： > >> Hi guys, >> >> we have to report this month:

Re: [VOTE] SPARK 2.3.2 (RC1)

https://github.com/apache/spark/pull/21659, and fix the release doc too. > > > 2018년 7월 9일 (월) 오전 8:25, Saisai Shao 님이 작성: > >> Hi Sean, >> >> SPARK-24530 is not included in this RC1 release. Actually I'm so familiar >> with this issue so still using python2

Re: Podling report this month

Sorry I forgot about this, will draft this today. Thanks Saisai Jean-Baptiste Onofré 于2018年7月8日周日 下午1:17写道： > Hi guys, > > we have to report this month: > > https://wiki.apache.org/incubator/July2018 > > Did someone already start a report ? I can bootstrap one if you want. > > Thanks, > Regards

Re: [VOTE] SPARK 2.3.2 (RC1)

not. >> >> >> Otherwise nothing is open for 2.3.2, sigs and license look good, tests >> pass as last time, etc. >> >> +1 >> >> On Sun, Jul 8, 2018 at 3:30 AM Saisai Shao >> wrote: >> >>> Please vote on releasing the following candidate

[VOTE] SPARK 2.3.2 (RC1)

Please vote on releasing the following candidate as Apache Spark version 2.3.2. The vote is open until July 11th PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 2.3.2 [ ] -1 Do not release this package because ... To l

[jira] [Comment Edited] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534565#comment-16534565 ] Saisai Shao edited comment on SPARK-24723 at 7/6/18 8:2

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534565#comment-16534565 ] Saisai Shao commented on SPARK-24723: - [~mengxr] [~jiangxb1987] There&#

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534529#comment-16534529 ] Saisai Shao commented on SPARK-24723: - After discussed with Xiangrui off

[jira] [Commented] (LIVY-480) Apache livy and Ajax

[ https://issues.apache.org/jira/browse/LIVY-480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533241#comment-16533241 ] Saisai Shao commented on LIVY-480: -- Please ask question in the mail list, to se

[jira] [Resolved] (LIVY-480) Apache livy and Ajax

[ https://issues.apache.org/jira/browse/LIVY-480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-480. -- Resolution: Invalid > Apache livy and Ajax > > > K

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533205#comment-16533205 ] Saisai Shao commented on SPARK-24723: - Hi [~mengxr], I would like to know the

[jira] [Commented] (SPARK-24739) PySpark does not work with Python 3.7.0

[ https://issues.apache.org/jira/browse/SPARK-24739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533189#comment-16533189 ] Saisai Shao commented on SPARK-24739: - Do we have to fix it in 2.3.2? I don

Re: Time for 2.3.2?

FYI, currently we have one block issue ( https://issues.apache.org/jira/browse/SPARK-24535), will start the release after this is fixed. Also please let me know if there're other blocks or fixes want to land to 2.3.2 release. Thanks Saisai Saisai Shao 于2018年7月2日周一 下午1:16写道： > I wi

[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR on Windows

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532145#comment-16532145 ] Saisai Shao commented on SPARK-24535: - Hi [~felixcheung] what's the curre

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532143#comment-16532143 ] Saisai Shao commented on SPARK-24530: - Hi [~hyukjin.kwon] what is the cur

Re: Time for 2.3.2?

t; +1, I heard some Spark users have skipped v2.3.1 because of these bugs. >>>> >>>> On Thu, Jun 28, 2018 at 3:09 PM Xingbo Jiang >>>> wrote: >>>> >>>>> +1 >>>>> >>>>> Wenchen Fan 于2018年6月28日 周四下午2:06写道： >>

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529352#comment-16529352 ] Saisai Shao commented on SPARK-24530: - [~hyukjin.kwon], Spark 2.1.3 and 2.2.2

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24530: Target Version/s: 2.1.3, 2.2.2, 2.3.2, 2.4.0 (was: 2.4.0) > Sphinx doesn'

Re: Does Livy Support Basic Auth?

No, it doesn't support, you should change the code to add a specific Filter to make it work. Harsch, Tim 于2018年6月30日周六 上午12:49写道： > I thought I read somewhere that Livy supports basic auth, but don't see > configurations for it. Does it? >

Re: Time for 2.3.2?

+1, like mentioned by Marcelo, these issues seems quite severe. I can work on the release if short of hands :). Thanks Jerry Marcelo Vanzin 于2018年6月28日周四 上午11:40写道： > +1. SPARK-24589 / SPARK-24552 are kinda nasty and we should get fixes > for those out. > > (Those are what delayed 2.2.2 and 2

[jira] [Commented] (LIVY-479) Enable livy.rsc.launcher.address configuration

[ https://issues.apache.org/jira/browse/LIVY-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16525775#comment-16525775 ] Saisai Shao commented on LIVY-479: -- Yes, I've left the comments on the PR.

Re: A new link request to my project and one question

at it allows, but it > would be better to add finer grained permissions. > > On Mon, Jun 25, 2018 at 6:30 PM, Saisai Shao > wrote: > > Yes, has a configuration "livy.superusers". Here in this case, the sql > > server user should be added as a superuser, who can impers

[jira] [Resolved] (SPARK-24418) Upgrade to Scala 2.11.12

[ https://issues.apache.org/jira/browse/SPARK-24418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24418. - Resolution: Fixed Issue resolved by pull request 21495 [https://github.com/apache/spark/pull

Re: A new link request to my project and one question

Yes, has a configuration "livy.superusers". Here in this case, the sql server user should be added as a superuser, who can impersonate other different users. Marcelo Vanzin 于2018年6月26日周二 上午9:12写道： > You're talking about another service between the user and the application. > > In that case a par

[jira] [Commented] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523022#comment-16523022 ] Saisai Shao commented on SPARK-24646: - Hi [~vanzin], here is a specific exa

[jira] [Created] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

Saisai Shao created SPARK-24646: --- Summary: Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes Key: SPARK-24646 URL: https://issues.apache.org/jira/browse/SPARK-24646 Proj

Re: Can livy execute code when session is in busy state?

No, busy means currently there's job running in Spark, so the follow-up code will wait until the previous job is done. JF Chen 于2018年6月22日周五 上午11:53写道： > Can livy execute code when session is in busy state? > > Regard, > Junfeng Chen >

[jira] [Commented] (SPARK-24374) SPIP: Support Barrier Scheduling in Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-24374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519934#comment-16519934 ] Saisai Shao commented on SPARK-24374: - Hi [~mengxr] [~jiangxb1987] SPARK-2

[jira] [Commented] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)

[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519931#comment-16519931 ] Saisai Shao commented on HIVE-16391: Gently ping [~hagleitn], would you please

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24615: Labels: SPIP (was: ) > Accelerator aware task scheduling for Sp

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24615: Description: In the machine learning area, accelerator card (GPU, FPGA, TPU) is predominant

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24615: Description: In the machine learning area, accelerator card (GPU, FPGA, TPU) is predominant

[jira] [Created] (SPARK-24615) Accelerator aware task scheduling for Spark

Saisai Shao created SPARK-24615: --- Summary: Accelerator aware task scheduling for Spark Key: SPARK-24615 URL: https://issues.apache.org/jira/browse/SPARK-24615 Project: Spark Issue Type

[jira] [Commented] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517720#comment-16517720 ] Saisai Shao commented on SPARK-24493: - This BUG is fixed in HDFS-12670, S

Re: A new link request to my project and one question

Sure, I will merge the website code, thanks! For proxyUser thing, I think there's no particular reason not adding it, maybe we just forgot to add the proxyUser support. It would be better if you could create a JIRA to track this issue. If you're familiar with Livy code, you can also submit a PR a

[jira] [Updated] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)

[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated HIVE-16391: --- Attachment: HIVE-16391.2.patch > Publish proper Hive 1.2 jars (without including all dependencies

[jira] [Commented] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)

[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511851#comment-16511851 ] Saisai Shao commented on HIVE-16391: I see. I can keep the "core" clas

[jira] [Resolved] (LIVY-477) Upgrade Livy Scala version to 2.11.12

[ https://issues.apache.org/jira/browse/LIVY-477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-477. -- Resolution: Fixed Fix Version/s: 0.6.0 > Upgrade Livy Scala version to 2.11

[jira] [Commented] (LIVY-477) Upgrade Livy Scala version to 2.11.12

[ https://issues.apache.org/jira/browse/LIVY-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509600#comment-16509600 ] Saisai Shao commented on LIVY-477: -- Issue resolved by pull request 100 [h

[jira] [Updated] (SPARK-24518) Using Hadoop credential provider API to store password

[ https://issues.apache.org/jira/browse/SPARK-24518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24518: Description: Current Spark configs password in a plaintext way, like putting in the

[jira] [Updated] (SPARK-24518) Using Hadoop credential provider API to store password

[ https://issues.apache.org/jira/browse/SPARK-24518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24518: Environment: (was: Current Spark configs password in a plaintext way, like putting in the

[jira] [Created] (SPARK-24518) Using Hadoop credential provider API to store password

Saisai Shao created SPARK-24518: --- Summary: Using Hadoop credential provider API to store password Key: SPARK-24518 URL: https://issues.apache.org/jira/browse/SPARK-24518 Project: Spark Issue

[jira] [Created] (LIVY-477) Upgrade Livy Scala version to 2.11.12

Saisai Shao created LIVY-477: Summary: Upgrade Livy Scala version to 2.11.12 Key: LIVY-477 URL: https://issues.apache.org/jira/browse/LIVY-477 Project: Livy Issue Type: Improvement

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506023#comment-16506023 ] Saisai Shao commented on SPARK-23534: - Yes, it is a Hadoop 2.8+ issue, but we d

Re: Spark YARN job submission error (code 13)

In Spark on YARN, error code 13 means SparkContext doesn't initialize in time. You can check the yarn application log to get more information. BTW, did you just write a plain python script without creating SparkContext/SparkSession? Aakash Basu 于2018年6月8日周五 下午4:15写道： > Hi, > > I'm trying to run

[jira] [Resolved] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24487. - Resolution: Won't Fix > Add support for

[jira] [Commented] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505786#comment-16505786 ] Saisai Shao commented on SPARK-24487: - I'm sure Spark community will not m

[jira] [Updated] (SPARK-23151) Provide a distribution of Spark with Hadoop 3.0

[ https://issues.apache.org/jira/browse/SPARK-23151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23151: Issue Type: Sub-task (was: Dependency upgrade) Parent: SPARK-23534 > Provid

[jira] [Commented] (SPARK-23151) Provide a distribution of Spark with Hadoop 3.0

[ https://issues.apache.org/jira/browse/SPARK-23151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505783#comment-16505783 ] Saisai Shao commented on SPARK-23151: - I will convert this as a subtask of S

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505771#comment-16505771 ] Saisai Shao commented on SPARK-23534: - Spark with Hadoop 3 will be failed in t

[jira] [Comment Edited] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505768#comment-16505768 ] Saisai Shao edited comment on SPARK-24493 at 6/8/18 6:5

[jira] [Commented] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505768#comment-16505768 ] Saisai Shao commented on SPARK-24493: - Adding more background, the issu

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Summary: Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3 (was: Kerberos Ticket

[jira] [Commented] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505760#comment-16505760 ] Saisai Shao commented on SPARK-24487: - You can add it in your own package

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in long running Spark job

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Component/s: Spark Core > Kerberos Ticket Renewal is failing in long running Spark

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in long running Spark job

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Component/s: (was: Spark Core) YARN > Kerberos Ticket Renewal is failing

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in long running Spark job

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Priority: Major (was: Blocker) > Kerberos Ticket Renewal is failing in long running Spark

[jira] [Commented] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505608#comment-16505608 ] Saisai Shao commented on SPARK-24487: - What's the usage and purpose to

[jira] [Commented] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)

[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505607#comment-16505607 ] Saisai Shao commented on HIVE-16391: Any comment [~vanzin] [~ste...@apache

[jira] [Reopened] (LIVY-476) Changing log-level INFO

[ https://issues.apache.org/jira/browse/LIVY-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reopened LIVY-476: -- > Changing log-level INFO > --- > > Key: LIVY-476 >

[jira] [Resolved] (LIVY-476) Changing log-level INFO

[ https://issues.apache.org/jira/browse/LIVY-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved LIVY-476. -- Resolution: Not A Problem > Changing log-level INFO > --- > >

[jira] [Commented] (LIVY-476) Changing log-level INFO

[ https://issues.apache.org/jira/browse/LIVY-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504545#comment-16504545 ] Saisai Shao commented on LIVY-476: -- Not super sure about your issues, can you pl

[jira] [Commented] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)

[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503188#comment-16503188 ] Saisai Shao commented on HIVE-16391: Uploaded a new patch [^HIVE-16391.1.patc

[jira] [Updated] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)

[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated HIVE-16391: --- Attachment: HIVE-16391.1.patch > Publish proper Hive 1.2 jars (without including all dependencies

[jira] [Commented] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)

[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16502976#comment-16502976 ] Saisai Shao commented on HIVE-16391: [~vanzin] one problem about your prop

[jira] [Commented] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)

[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16502756#comment-16502756 ] Saisai Shao commented on HIVE-16391: {quote}The problem with that is that it cha

<    1   2   3   4   5   6   7   8   9   10   >