[
https://issues.apache.org/jira/browse/SPARK-10063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15644092#comment-15644092
]
Steve Loughran commented on SPARK-10063:
HADOOP-13786 covers adding a committer for direct output
[
https://issues.apache.org/jira/browse/SPARK-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643819#comment-15643819
]
Steve Loughran commented on SPARK-7344:
---
I've been doing lots of work with S3a and not seeing
[
https://issues.apache.org/jira/browse/SPARK-13044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643827#comment-15643827
]
Steve Loughran commented on SPARK-13044:
This is HADOOP-13325; jets3t doesn't support v4 APIs.
[
https://issues.apache.org/jira/browse/SPARK-13044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-13044:
---
Summary: saveAsTextFile(s3n://) doesn't support s3 Signature Version 4
(was:
[
https://issues.apache.org/jira/browse/SPARK-13044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-13044.
Resolution: Won't Fix
> saveAsTextFile(s3n://) doesn't support s3 Signature Version 4
>
[
https://issues.apache.org/jira/browse/SPARK-12378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-12378.
Resolution: Cannot Reproduce
> CREATE EXTERNAL TABLE AS SELECT EXPORT AWS S3 ERROR
>
[
https://issues.apache.org/jira/browse/SPARK-12378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643851#comment-15643851
]
Steve Loughran commented on SPARK-12378:
This is amazon EMR; they've got their own releases of
[
https://issues.apache.org/jira/browse/SPARK-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643864#comment-15643864
]
Steve Loughran commented on SPARK-18017:
you can check what's been picked up by grabbing a copy
[
https://issues.apache.org/jira/browse/SPARK-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15584969#comment-15584969
]
Steve Loughran commented on SPARK-5925:
---
looking at this, I'm confused about what I'd written
[
https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602160#comment-15602160
]
Steve Loughran commented on SPARK-2984:
---
Alexy, can you describe your layout a bit more
# are you
[
https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591647#comment-15591647
]
Steve Loughran commented on SPARK-10673:
This may be related to SPARK-17179; Hive can do a lot
[
https://issues.apache.org/jira/browse/SPARK-18402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15657769#comment-15657769
]
Steve Loughran commented on SPARK-18402:
I've seen this before, somewhere. it's usually a
[
https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15702373#comment-15702373
]
Steve Loughran commented on SPARK-18512:
one question: what's the size of data being committed
[
https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15702368#comment-15702368
]
Steve Loughran commented on SPARK-18512:
This looks like a consistency problem; s3 listing always
[
https://issues.apache.org/jira/browse/SPARK-18262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15702340#comment-15702340
]
Steve Loughran commented on SPARK-18262:
~tdunning has done a mostly-compatible org.json
[
https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15702373#comment-15702373
]
Steve Loughran edited comment on SPARK-18512 at 11/28/16 5:54 PM:
--
one
[
https://issues.apache.org/jira/browse/SPARK-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703081#comment-15703081
]
Steve Loughran commented on SPARK-18551:
hmm. If you are running HDFS unsecure, then all I need
[
https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15679196#comment-15679196
]
Steve Loughran commented on SPARK-2984:
---
That sounds like a separate issue...could you open a new
[
https://issues.apache.org/jira/browse/SPARK-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15683998#comment-15683998
]
Steve Loughran commented on SPARK-14222:
Hadoop 2.9 just went to Java 2.7.8; latest update that
[
https://issues.apache.org/jira/browse/SPARK-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15635835#comment-15635835
]
Steve Loughran commented on SPARK-14222:
In Hadoop we're looking at -> 2.7.x to (a) be more
[
https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568180#comment-15568180
]
Steve Loughran commented on SPARK-15343:
this is a tough problem with Hadoop core, as if it moves
[
https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568187#comment-15568187
]
Steve Loughran commented on SPARK-15343:
There is a very quick fix here, to stop the problem
[
https://issues.apache.org/jira/browse/SPARK-12571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572442#comment-15572442
]
Steve Loughran commented on SPARK-12571:
Means the credentials aren't at the far end, either in
[
https://issues.apache.org/jira/browse/SPARK-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572468#comment-15572468
]
Steve Loughran commented on SPARK-8437:
---
Just came across by way of comments in the source. This
[
https://issues.apache.org/jira/browse/SPARK-14561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572401#comment-15572401
]
Steve Loughran commented on SPARK-14561:
To clarify: it's not changes in existing files that
[
https://issues.apache.org/jira/browse/SPARK-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572414#comment-15572414
]
Steve Loughran commented on SPARK-9004:
---
HADOOP-13605 added a whole new set of counters for HDFS, S3
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15582210#comment-15582210
]
Steve Loughran commented on SPARK-7481:
---
For anyone watching this; the code is pretty much ready to
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-7481:
--
Summary: Add spark-cloud module to pull in object store support; test
(was: Add spark-cloud
[
https://issues.apache.org/jira/browse/SPARK-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15754213#comment-15754213
]
Steve Loughran commented on SPARK-18883:
if it surfaces on HDFS it's not an S3 consistency issue,
[
https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726310#comment-15726310
]
Steve Loughran commented on SPARK-18512:
It'd be good to get some more details from people who
[
https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726135#comment-15726135
]
Steve Loughran commented on SPARK-18512:
ah, the "when will 2.8 ship" question. Really close, Jun
[
https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15751492#comment-15751492
]
Steve Loughran commented on SPARK-18512:
yes, file a SPARK one and that can be a base for blame
[
https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15742480#comment-15742480
]
Steve Loughran commented on SPARK-17593:
Marking as a dependency of HADOOP-13208, which fixes it
[
https://issues.apache.org/jira/browse/SPARK-19111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15821773#comment-15821773
]
Steve Loughran commented on SPARK-19111:
Just realised one more thing
If the allocated threads
[
https://issues.apache.org/jira/browse/SPARK-11353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-11353.
Resolution: Duplicate
This is a duplicate of SPARK-13044; that's transitive a WONTFIX due
[
https://issues.apache.org/jira/browse/SPARK-11353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-11353:
---
Summary: Writing to S3 buckets, which only support AWS4-HMAC-SHA256 fails
with s3n URLs
[
https://issues.apache.org/jira/browse/SPARK-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15705081#comment-15705081
]
Steve Loughran commented on SPARK-18551:
I'm not sure about a WONTFIX; it just needs the SHS to
[
https://issues.apache.org/jira/browse/SPARK-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15708360#comment-15708360
]
Steve Loughran commented on SPARK-18551:
Which JIRA are you using here?
FWIW, one issue with
[
https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-18512:
---
Environment:
AWS EMR 5.0.1
Spark 2.0.1
S3 EU-West-1 (S3A)
was:
AWS EMR 5.0.1
Spark 2.0.1
[
https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15722304#comment-15722304
]
Steve Loughran commented on SPARK-18512:
no. What you are seeing is an eventual consistency
[
https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15722320#comment-15722320
]
Steve Loughran commented on SPARK-18512:
Actually, this is the problem whcih MAPREDUCE-6478 deals
[
https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15722354#comment-15722354
]
Steve Loughran commented on SPARK-18512:
of course, if you do switch to EMRFS, you should get
[
https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15711735#comment-15711735
]
Steve Loughran commented on SPARK-18673:
HADOOP-13852 provides a quick workaround, limited DF
[
https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15711754#comment-15711754
]
Steve Loughran commented on SPARK-13446:
building against Hive 2.x is going to be hard; Spark's
[
https://issues.apache.org/jira/browse/SPARK-14694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-14694.
Resolution: Duplicate
stack trace marks this as a duplicate of SPARK-11851
> Thrift
Steve Loughran created SPARK-18673:
--
Summary: Dataframes doesn't work on Hadoop 3.x; Hive rejects
Hadoop version
Key: SPARK-18673
URL: https://issues.apache.org/jira/browse/SPARK-18673
Project:
[
https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15711675#comment-15711675
]
Steve Loughran commented on SPARK-18673:
{code}
java.lang.IllegalArgumentException:
[
https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801990#comment-15801990
]
Steve Loughran commented on SPARK-2356:
---
I'm sorry you are suffering; it's a pain for all of us who
[
https://issues.apache.org/jira/browse/SPARK-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795069#comment-15795069
]
Steve Loughran commented on SPARK-18917:
looking at the code being optionally disabled, the
[
https://issues.apache.org/jira/browse/SPARK-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795018#comment-15795018
]
Steve Loughran commented on SPARK-19013:
Re-opening this as it may deserve a bit of a closer
[
https://issues.apache.org/jira/browse/SPARK-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801059#comment-15801059
]
Steve Loughran edited comment on SPARK-19013 at 1/5/17 11:26 AM:
-
ok,
[
https://issues.apache.org/jira/browse/SPARK-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801120#comment-15801120
]
Steve Loughran commented on SPARK-19013:
Note also that a config based documentation option would
[
https://issues.apache.org/jira/browse/SPARK-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801059#comment-15801059
]
Steve Loughran commented on SPARK-19013:
ok, that is potentially the problem. One thing here,
[
https://issues.apache.org/jira/browse/SPARK-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801113#comment-15801113
]
Steve Loughran commented on SPARK-19013:
+ [~Thomas Demoor] for his opinion
>
[
https://issues.apache.org/jira/browse/SPARK-19100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811786#comment-15811786
]
Steve Loughran commented on SPARK-19100:
it's hard to imagine any dataset where large input sizes
[
https://issues.apache.org/jira/browse/SPARK-19111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811767#comment-15811767
]
Steve Loughran edited comment on SPARK-19111 at 1/9/17 1:26 PM:
What's
[
https://issues.apache.org/jira/browse/SPARK-19111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811767#comment-15811767
]
Steve Loughran commented on SPARK-19111:
What's happening here is that (a) S3n isn't uploading
[
https://issues.apache.org/jira/browse/SPARK-19100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811797#comment-15811797
]
Steve Loughran commented on SPARK-19100:
Relevant citations
* Grover and Carey, 2011: *Extending
[
https://issues.apache.org/jira/browse/SPARK-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785634#comment-15785634
]
Steve Loughran commented on SPARK-18883:
thanks, good to know
> FileNotFoundException on
[
https://issues.apache.org/jira/browse/SPARK-10294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944887#comment-15944887
]
Steve Loughran commented on SPARK-10294:
consider it a failure in the exception logic; it tries
[
https://issues.apache.org/jira/browse/SPARK-20061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-20061.
Resolution: Duplicate
> Reading a file with colon (:) from S3 fails with
[
https://issues.apache.org/jira/browse/SPARK-20061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943008#comment-15943008
]
Steve Loughran commented on SPARK-20061:
":" is one of those "implicitly forbidden characters in
[
https://issues.apache.org/jira/browse/SPARK-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15938321#comment-15938321
]
Steve Loughran commented on SPARK-19013:
One thing that code be done here would be to worry about
Steve Loughran created SPARK-19978:
--
Summary: spark thrift server to switch to normative hadoop 2.2+
service lifecycle
Key: SPARK-19978
URL: https://issues.apache.org/jira/browse/SPARK-19978
Steve Loughran created SPARK-20038:
--
Summary: FileFormatWriter.ExecuteWriteTask.releaseResources()
implementations to be re-entrant
Key: SPARK-20038
URL: https://issues.apache.org/jira/browse/SPARK-20038
[
https://issues.apache.org/jira/browse/SPARK-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933324#comment-15933324
]
Steve Loughran commented on SPARK-10109:
I think the cause is actually that in some codepaths, if
[
https://issues.apache.org/jira/browse/SPARK-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933340#comment-15933340
]
Steve Loughran commented on SPARK-10109:
This is a bit related to the execution/commit mechanism;
[
https://issues.apache.org/jira/browse/SPARK-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955956#comment-15955956
]
Steve Loughran commented on SPARK-20153:
I'm glad we are both in agreement about not using
[
https://issues.apache.org/jira/browse/SPARK-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259963#comment-15259963
]
Steve Loughran edited comment on SPARK-6527 at 4/1/17 12:41 PM:
I've not
[
https://issues.apache.org/jira/browse/SPARK-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15952201#comment-15952201
]
Steve Loughran commented on SPARK-6527:
---
Hadoop 2.8.0 is out the door, try against those JARs before
[
https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956879#comment-15956879
]
Steve Loughran commented on SPARK-20202:
# the ugliness need to inset the spark thrift stuff
[
https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962903#comment-15962903
]
Steve Loughran commented on SPARK-20202:
One thing I do recall as trouble here was that ivy
[
https://issues.apache.org/jira/browse/SPARK-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968934#comment-15968934
]
Steve Loughran commented on SPARK-10109:
SPARK-20038 should stop the failure being so dramatic
>
[
https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-17593.
Resolution: Fixed
closing as fixed now that Hadoop 2.8.0 is out the door. Upgrade your
[
https://issues.apache.org/jira/browse/SPARK-19790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897863#comment-15897863
]
Steve Loughran commented on SPARK-19790:
The only time a task output committer should be making
[
https://issues.apache.org/jira/browse/SPARK-19790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898003#comment-15898003
]
Steve Loughran commented on SPARK-19790:
Thinking some more & looking at code snippets
#
[
https://issues.apache.org/jira/browse/SPARK-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954233#comment-15954233
]
Steve Loughran commented on SPARK-20153:
This is fixed in Hadoop 2.8 with [per-bucket
[
https://issues.apache.org/jira/browse/SPARK-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954233#comment-15954233
]
Steve Loughran edited comment on SPARK-20153 at 4/3/17 10:13 PM:
-
This is
[
https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960925#comment-15960925
]
Steve Loughran commented on SPARK-2984:
---
For s3a commits, HADOOP-13786 is going to be the fix. This
[
https://issues.apache.org/jira/browse/SPARK-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973541#comment-15973541
]
Steve Loughran commented on SPARK-20153:
[~tafra...@gmail.com] : thanks for discovering that. I
[
https://issues.apache.org/jira/browse/SPARK-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15981552#comment-15981552
]
Steve Loughran commented on SPARK-20107:
This does not solve the problem you think it does, not
[
https://issues.apache.org/jira/browse/SPARK-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15981552#comment-15981552
]
Steve Loughran edited comment on SPARK-20107 at 4/24/17 5:30 PM:
-
This
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15981602#comment-15981602
]
Steve Loughran commented on SPARK-7481:
---
One thing I want to emphasise here is: I have no loyalty to
[
https://issues.apache.org/jira/browse/SPARK-21618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112511#comment-16112511
]
Steve Loughran commented on SPARK-21618:
It may depend on HADOOP-14383; I wouldn't recommend
[
https://issues.apache.org/jira/browse/SPARK-21618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112529#comment-16112529
]
Steve Loughran commented on SPARK-21618:
If you're relying on hadoop-common to provide the FS
[
https://issues.apache.org/jira/browse/SPARK-21618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112520#comment-16112520
]
Steve Loughran commented on SPARK-21618:
BTW, we haven't backported HADOOP-14383 into HDP; don't
[
https://issues.apache.org/jira/browse/SPARK-21618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112529#comment-16112529
]
Steve Loughran edited comment on SPARK-21618 at 8/3/17 10:09 AM:
-
If
[
https://issues.apache.org/jira/browse/SPARK-21618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114227#comment-16114227
]
Steve Loughran commented on SPARK-21618:
Thinking about this some more: what's happening in
[
https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128891#comment-16128891
]
Steve Loughran commented on SPARK-20952:
out of curiosity, what "filesystem games" are you
[
https://issues.apache.org/jira/browse/SPARK-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127061#comment-16127061
]
Steve Loughran commented on SPARK-21702:
This is interesting. What may be happening is that
[
https://issues.apache.org/jira/browse/SPARK-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127062#comment-16127062
]
Steve Loughran commented on SPARK-21702:
ps, you shouldn't need to set the s3a.impl field; that's
[
https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123063#comment-16123063
]
Steve Loughran commented on SPARK-21697:
# I don't see anything which can be done in HDFS here;
[
https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123193#comment-16123193
]
Steve Loughran commented on SPARK-21697:
PS: right now, probably doesn't work at all
> NPE &
[
https://issues.apache.org/jira/browse/SPARK-12868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123192#comment-16123192
]
Steve Loughran commented on SPARK-12868:
SPARK-21697: harder than it would initially seem
> ADD
[
https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123208#comment-16123208
]
Steve Loughran commented on SPARK-21697:
What would a test to replicate look like?
# Create
[
https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122195#comment-16122195
]
Steve Loughran commented on SPARK-21697:
{code}
Have u tried it in yarn-client mode? i add this
[
https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122195#comment-16122195
]
Steve Loughran edited comment on SPARK-21697 at 8/10/17 7:48 PM:
-
Text of
Steve Loughran created SPARK-21697:
--
Summary: NPE & ExceptionInInitializerError trying to load UTF from
HDFS
Key: SPARK-21697
URL: https://issues.apache.org/jira/browse/SPARK-21697
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084355#comment-16084355
]
Steve Loughran commented on SPARK-20703:
this has just added a whole new stack trace for my
[
https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085488#comment-16085488
]
Steve Loughran commented on SPARK-20703:
yeah, I'm not worrying too much about the new
301 - 400 of 828 matches
Mail list logo
