[
https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085559#comment-16085559
]
Steve Loughran commented on SPARK-20703:
Regarding a patch for this, what do people suggest as a
[
https://issues.apache.org/jira/browse/SPARK-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085645#comment-16085645
]
Steve Loughran commented on SPARK-21374:
This is possibly a sign that your new configuration
[
https://issues.apache.org/jira/browse/SPARK-19790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088630#comment-16088630
]
Steve Loughran commented on SPARK-19790:
I've now summarised the FileOutputCommitter v1 and v2
[
https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088632#comment-16088632
]
Steve Loughran commented on SPARK-20703:
..got a patch for this, but want to see if I can create
[
https://issues.apache.org/jira/browse/SPARK-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073951#comment-16073951
]
Steve Loughran commented on SPARK-20107:
If you are curious, I've just written out the v1 and v2
[
https://issues.apache.org/jira/browse/SPARK-12868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068301#comment-16068301
]
Steve Loughran commented on SPARK-12868:
It's actually not the cause of that, merely the
[
https://issues.apache.org/jira/browse/SPARK-21137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065423#comment-16065423
]
Steve Loughran commented on SPARK-21137:
Looking at this.
something is trying to get the
[
https://issues.apache.org/jira/browse/SPARK-21137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065445#comment-16065445
]
Steve Loughran commented on SPARK-21137:
Filed HADOOP-14600.
Looks like a v. old codepath that's
[
https://issues.apache.org/jira/browse/SPARK-21137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065428#comment-16065428
]
Steve Loughran commented on SPARK-21137:
ps, for now, do it in parallel:
[
https://issues.apache.org/jira/browse/SPARK-12868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065448#comment-16065448
]
Steve Loughran commented on SPARK-12868:
I think this is the case of HADOOP-14598: once the FS
[
https://issues.apache.org/jira/browse/SPARK-21137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065515#comment-16065515
]
Steve Loughran commented on SPARK-21137:
bq. so it is something that could be optimized in the
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984771#comment-15984771
]
Steve Loughran commented on SPARK-7481:
---
I think we ended up going in circles on that PR. Sean has
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985040#comment-15985040
]
Steve Loughran commented on SPARK-7481:
---
(This is a fairly long comment, but it tries to summarise
[
https://issues.apache.org/jira/browse/SPARK-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15981081#comment-15981081
]
Steve Loughran commented on SPARK-17159:
pulled out documentation into separate JIRA,
Steve Loughran created SPARK-20448:
--
Summary: Document how FileInputDStream works with object storage
Key: SPARK-20448
URL: https://issues.apache.org/jira/browse/SPARK-20448
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108897#comment-16108897
]
Steve Loughran commented on SPARK-21374:
I understand...the patch shows the issue. Its only
[
https://issues.apache.org/jira/browse/SPARK-21514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108882#comment-16108882
]
Steve Loughran commented on SPARK-21514:
Can you link this JIRA to the specific HIVE work?
>
[
https://issues.apache.org/jira/browse/SPARK-21618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114379#comment-16114379
]
Steve Loughran commented on SPARK-21618:
yes, and that 2.9+ feature breaks things, because when
Steve Loughran created SPARK-21762:
--
Summary: FileFormatWriter metrics collection fails if a newly
close()d file isn't yet visible
Key: SPARK-21762
URL: https://issues.apache.org/jira/browse/SPARK-21762
[
https://issues.apache.org/jira/browse/SPARK-20370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993405#comment-15993405
]
Steve Loughran commented on SPARK-20370:
Is this the bit under the PR tagged "!! HACK ALERT !!"
[
https://issues.apache.org/jira/browse/SPARK-20560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993323#comment-15993323
]
Steve Loughran commented on SPARK-20560:
To follow this up, I've now got a test which verifies
[
https://issues.apache.org/jira/browse/SPARK-20608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006454#comment-16006454
]
Steve Loughran commented on SPARK-20608:
One thing to consider here is starting with a test to
[
https://issues.apache.org/jira/browse/SPARK-21074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056309#comment-16056309
]
Steve Loughran commented on SPARK-21074:
Given this is an s3 URL, it may be amplifying the
[
https://issues.apache.org/jira/browse/SPARK-19111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056516#comment-16056516
]
Steve Loughran commented on SPARK-19111:
Followup: [~drcrallen]; Hadoop 2.8 is out the door with
[
https://issues.apache.org/jira/browse/SPARK-11373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059141#comment-16059141
]
Steve Loughran commented on SPARK-11373:
metrics might help with understanding the s3 load issues
[
https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017258#comment-16017258
]
Steve Loughran commented on SPARK-20799:
bq. Spark does output the S3xLoginHelper:90 - The
[
https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022807#comment-16022807
]
Steve Loughran commented on SPARK-20799:
If what I think is happening is, then it's the security
[
https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-20799:
---
Summary: Unable to infer schema for ORC on S3N when secrets are in the URL
(was: Unable to
[
https://issues.apache.org/jira/browse/SPARK-19669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16026219#comment-16026219
]
Steve Loughran commented on SPARK-19669:
thanks for this, very nice to have Logging usable
[
https://issues.apache.org/jira/browse/SPARK-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024526#comment-16024526
]
Steve Loughran commented on SPARK-8578:
---
Given SPARK-10063 has pulled the
[
https://issues.apache.org/jira/browse/SPARK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024886#comment-16024886
]
Steve Loughran commented on SPARK-20886:
Stack trace: after
{code}
2017-05-25 16:22:10,807
[
https://issues.apache.org/jira/browse/SPARK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024885#comment-16024885
]
Steve Loughran commented on SPARK-20886:
Stack trace: before
{code}
Driver stacktrace:
at
Steve Loughran created SPARK-20886:
--
Summary: HadoopMapReduceCommitProtocol to fail with message if
FileOutputCommitter.getWorkPath==null
Key: SPARK-20886
URL: https://issues.apache.org/jira/browse/SPARK-20886
[
https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-20799:
---
Priority: Minor (was: Major)
> Unable to infer schema for ORC on S3N when secrets are in
[
https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-20799:
---
Summary: Unable to infer schema for ORC/Parquet on S3N when secrets are in
the URL (was:
[
https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-20799:
---
Environment: Hadoop 2.8.0 binaries
> Unable to infer schema for ORC/Parquet on S3N when
[
https://issues.apache.org/jira/browse/SPARK-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16051680#comment-16051680
]
Steve Loughran commented on SPARK-21077:
like people say, this is inevitably a config problem.
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000502#comment-16000502
]
Steve Loughran commented on SPARK-7481:
---
thank you!
> Add spark-hadoop-cloud module to pull in
[
https://issues.apache.org/jira/browse/SPARK-20608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998150#comment-15998150
]
Steve Loughran commented on SPARK-20608:
Probably good to pull in someone who understands HDFS
Steve Loughran created SPARK-20560:
--
Summary: Review Spark's handling of filesystems returning
"localhost" in getFileBlockLocations
Key: SPARK-20560
URL: https://issues.apache.org/jira/browse/SPARK-20560
[
https://issues.apache.org/jira/browse/SPARK-20560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993008#comment-15993008
]
Steve Loughran commented on SPARK-20560:
{{FileSystem.getFileBlockLocations(path)}} is only
[
https://issues.apache.org/jira/browse/SPARK-19582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992985#comment-15992985
]
Steve Loughran commented on SPARK-19582:
All spark is doing is taking a URL To data, mapping that
[
https://issues.apache.org/jira/browse/SPARK-20560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-20560.
Resolution: Invalid
"localhost" is filtered, been done in
[
https://issues.apache.org/jira/browse/SPARK-21137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-21137:
---
Summary: Spark reads many small files slowly off local filesystem (was:
Spark reads many
[
https://issues.apache.org/jira/browse/SPARK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171386#comment-16171386
]
Steve Loughran commented on SPARK-20886:
Not, but related. This is handling the situation where
[
https://issues.apache.org/jira/browse/SPARK-21549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171393#comment-16171393
]
Steve Loughran commented on SPARK-21549:
Linking to SPARK-20045, which highlights the commit
[
https://issues.apache.org/jira/browse/SPARK-21549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171420#comment-16171420
]
Steve Loughran commented on SPARK-21549:
# you can't rely on the committers having output and
[
https://issues.apache.org/jira/browse/SPARK-21549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16173449#comment-16173449
]
Steve Loughran commented on SPARK-21549:
The {{newTaskTempFileAbsPath()}} method is an
[
https://issues.apache.org/jira/browse/SPARK-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-17159.
Resolution: Won't Fix
Based on the feedback of https://github.com/apache/spark/pull/14731
[
https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-22163:
---
Priority: Major (was: Critical)
> Design Issue of Spark Streaming that Causes Random
[
https://issues.apache.org/jira/browse/SPARK-22217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-22217:
---
Priority: Minor (was: Major)
> ParquetFileFormat to support arbitrary OutputCommitters
>
Steve Loughran created SPARK-22217:
--
Summary: ParquetFileFormat to support arbitrary OutputCommitters
if parquet.enable.summary-metadata is false
Key: SPARK-22217
URL:
[
https://issues.apache.org/jira/browse/SPARK-22217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-22217:
---
Summary: ParquetFileFormat to support arbitrary OutputCommitters (was:
ParquetFileFormat to
[
https://issues.apache.org/jira/browse/SPARK-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203495#comment-16203495
]
Steve Loughran commented on SPARK-22240:
We've got a test in HADOOP-14943 which looks @ part
[
https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192732#comment-16192732
]
Steve Loughran edited comment on SPARK-21999 at 10/5/17 1:39 PM:
-
Apache
[
https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192732#comment-16192732
]
Steve Loughran commented on SPARK-21999:
Apache projects are all open source, with an open
[
https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194332#comment-16194332
]
Steve Loughran commented on SPARK-21999:
Telling a project "their design is wrong" and expecting
[
https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-21999.
Resolution: Won't Fix
> ConcurrentModificationException - Spark Streaming
>
[
https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207573#comment-16207573
]
Steve Loughran commented on SPARK-2984:
---
bq. multiple batches writing to same location
[
https://issues.apache.org/jira/browse/SPARK-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203709#comment-16203709
]
Steve Loughran commented on SPARK-22240:
[~hyukjin.kwon]: we now see that on s3a, you only ever
[
https://issues.apache.org/jira/browse/SPARK-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201863#comment-16201863
]
Steve Loughran commented on SPARK-22240:
thanks. Now for a question which is probably obvious to
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201917#comment-16201917
]
Steve Loughran commented on SPARK-21797:
Update, in HADOOP-14874 I've noted we could use the
[
https://issues.apache.org/jira/browse/SPARK-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203966#comment-16203966
]
Steve Loughran commented on SPARK-22240:
Point me at a simple test suite for the multiline & I'll
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146193#comment-16146193
]
Steve Loughran commented on SPARK-21797:
No> That's a shame. I only came across the option when I
[
https://issues.apache.org/jira/browse/SPARK-20448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178236#comment-16178236
]
Steve Loughran commented on SPARK-20448:
thanks!
> Document how FileInputDStream works with
[
https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-2356.
---
Resolution: Duplicate
> Exception: Could not locate executable null\bin\winutils.exe in the
[
https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185744#comment-16185744
]
Steve Loughran commented on SPARK-2356:
---
[~Vasilina], that probably means you're running with Hadoop
[
https://issues.apache.org/jira/browse/SPARK-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138683#comment-16138683
]
Steve Loughran commented on SPARK-21817:
I think it's a regression in HDFS-6984; the superclass
[
https://issues.apache.org/jira/browse/SPARK-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138664#comment-16138664
]
Steve Loughran commented on SPARK-21817:
This a regression in HDFS?
> Pass FSPermissions to
[
https://issues.apache.org/jira/browse/SPARK-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138685#comment-16138685
]
Steve Loughran commented on SPARK-21817:
API is tagged as stable/evolving; it's clearly in use
[
https://issues.apache.org/jira/browse/SPARK-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139898#comment-16139898
]
Steve Loughran commented on SPARK-21702:
IF this is just "directories", then there are no
[
https://issues.apache.org/jira/browse/SPARK-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved SPARK-21702.
Resolution: Invalid
> Structured Streaming S3A SSE Encryption Not Visible through AWS S3
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140138#comment-16140138
]
Steve Loughran commented on SPARK-21797:
I was talking about the cost and time of getting data
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140147#comment-16140147
]
Steve Loughran commented on SPARK-21797:
Note that if it is just during spark partition
[
https://issues.apache.org/jira/browse/SPARK-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140161#comment-16140161
]
Steve Loughran commented on SPARK-21817:
FYI, this is now fixed in hadoop trunk/3.0-beta-1
>
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139919#comment-16139919
]
Steve Loughran commented on SPARK-21797:
If you are using S3// URLs then its the AWS team's
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140408#comment-16140408
]
Steve Loughran commented on SPARK-21797:
This is happening deep the Amazon EMR team's closed
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-21797:
---
Environment: Amazon EMR
> spark cannot read partitioned data in S3 that are partly in
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140408#comment-16140408
]
Steve Loughran edited comment on SPARK-21797 at 8/24/17 5:56 PM:
-
This is
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141483#comment-16141483
]
Steve Loughran commented on SPARK-21797:
bq. According to our test, it is 20% slower maximum to
[
https://issues.apache.org/jira/browse/SPARK-21762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated SPARK-21762:
---
Summary: FileFormatWriter/BasicWriteTaskStatsTracker metrics collection
fails if a new file
[
https://issues.apache.org/jira/browse/SPARK-21762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131190#comment-16131190
]
Steve Loughran edited comment on SPARK-21762 at 8/17/17 7:41 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-21762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131190#comment-16131190
]
Steve Loughran commented on SPARK-21762:
SPARK-20703 simplifies this, especially testing, as it's
[
https://issues.apache.org/jira/browse/SPARK-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200367#comment-16200367
]
Steve Loughran commented on SPARK-22240:
Amazon EMR is amazon's own fork of Spark & Hadoop, with
[
https://issues.apache.org/jira/browse/SPARK-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200485#comment-16200485
]
Steve Loughran commented on SPARK-22240:
What's the link to the multiline JIRA? As that could
[
https://issues.apache.org/jira/browse/SPARK-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182372#comment-16182372
]
Steve Loughran commented on SPARK-9103:
---
If it helps, most uses of ByteBuffer in hadoop core & HDFS
[
https://issues.apache.org/jira/browse/SPARK-22587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266697#comment-16266697
]
Steve Loughran commented on SPARK-22587:
See also
[
https://issues.apache.org/jira/browse/SPARK-22587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266674#comment-16266674
]
Steve Loughran commented on SPARK-22587:
Jerry had already pulled me in for this; it's one of
[
https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267015#comment-16267015
]
Steve Loughran commented on SPARK-22526:
HADOOP-15071 updates the s3a troubleshooting with what
[
https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265167#comment-16265167
]
Steve Loughran commented on SPARK-22526:
it says it in the javadocs for
[
https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264095#comment-16264095
]
Steve Loughran commented on SPARK-22526:
# Fix the code you invoke
#. wrap the code you invoke
[
https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264095#comment-16264095
]
Steve Loughran edited comment on SPARK-22526 at 11/23/17 3:47 PM:
--
# Fix
[
https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264496#comment-16264496
]
Steve Loughran commented on SPARK-22526:
I'm not giving a permanent fix. It's a bug in your code
[
https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272974#comment-16272974
]
Steve Loughran commented on SPARK-22657:
Hadoop FileSystem service introspection for FS binding
[
https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273191#comment-16273191
]
Steve Loughran commented on SPARK-22657:
if you look at HADOOP-14138 you can see why we cut s3a
[
https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263021#comment-16263021
]
Steve Loughran commented on SPARK-22526:
If the input stream doesn't get closed, there probably
[
https://issues.apache.org/jira/browse/SPARK-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292595#comment-16292595
]
Steve Loughran commented on SPARK-18294:
Following up on this, one question: Why support the
[
https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256789#comment-16256789
]
Steve Loughran commented on SPARK-14959:
Came across a reference to this while scanning for
[
https://issues.apache.org/jira/browse/SPARK-16996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255329#comment-16255329
]
Steve Loughran commented on SPARK-16996:
[~maver1ck]
Spark hive is custom as it was modified to
[
https://issues.apache.org/jira/browse/SPARK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247886#comment-16247886
]
Steve Loughran commented on SPARK-17593:
Hey nick,
yes, need to move to FileSystem.list(path,
401 - 500 of 828 matches
Mail list logo
