a the s3a client, you will need
>>> the SDK version to match the hadoop-aws JAR of the same version of Hadoop
>>> your JARs have. Similarly, if you were using spark-kinesis, it needs to be
>>> in sync there.
>>>
>>>
>>> *From:* Steve Loughra
u were using spark-kinesis, it needs to be
>> in sync there.
>>
>>
>> *From:* Steve Loughran [mailto:ste...@hortonworks.com
>> ]
>> *Sent:* Tuesday, October 03, 2017 2:20 PM
>> *To:* JG Perrin
>> *Cc:* user@spark.apache.org
>> *Subject:* Re: Quick on
gt; JARs have. Similarly, if you were using spark-kinesis, it needs to be in
>>> sync there.
>>>>
>>>> From: Steve Loughran [mailto:ste...@hortonworks.com]
>>>> Sent: Tuesday, October 03, 2017 2:20 PM
>>>> To: JG Perrin
>>>> Cc:
op
>> your JARs have. Similarly, if you were using spark-kinesis, it needs to be
>> in sync there.
>>
>>
>> *From:* Steve Loughran [mailto:ste...@hortonworks.com
>> ]
>> *Sent:* Tuesday, October 03, 2017 2:20 PM
>> *To:* JG Perrin
>> *Cc:* user@
om:* Steve Loughran [mailto:ste...@hortonworks.com
> ]
> *Sent:* Tuesday, October 03, 2017 2:20 PM
> *To:* JG Perrin
> *Cc:* user@spark.apache.org
> *Subject:* Re: Quick one... AWS SDK version?
>
>
>
> On 3 Oct 2017, at 02:28, JG Perrin wrote:
>
> Hey Sparkians,
>
&g
is.com>>
Cc: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Re: Quick one... AWS SDK version?
On 3 Oct 2017, at 02:28, JG Perrin
mailto:jper...@lumeris.com>> wrote:
Hey Sparkians,
What version of AWS Java SDK do you use with Spark 2.2? Do you stick with the
: Quick one... AWS SDK version?
On 3 Oct 2017, at 02:28, JG Perrin
mailto:jper...@lumeris.com>> wrote:
Hey Sparkians,
What version of AWS Java SDK do you use with Spark 2.2? Do you stick with the
Hadoop 2.7.3 libs?
You generally to have to stick with the version which hadoop was built wi
Thanks Yash… this is helpful!
From: Yash Sharma [mailto:yash...@gmail.com]
Sent: Tuesday, October 03, 2017 1:02 AM
To: JG Perrin ; user@spark.apache.org
Subject: Re: Quick one... AWS SDK version?
Hi JG,
Here are my cluster configs if it helps.
Cheers.
EMR: emr-5.8.0
Hadoop distribution
On 3 Oct 2017, at 02:28, JG Perrin
mailto:jper...@lumeris.com>> wrote:
Hey Sparkians,
What version of AWS Java SDK do you use with Spark 2.2? Do you stick with the
Hadoop 2.7.3 libs?
You generally to have to stick with the version which hadoop was built with I'm
afraid...very brittle depende
Hi JG,
Here are my cluster configs if it helps.
Cheers.
EMR: emr-5.8.0
Hadoop distribution: Amazon 2.7.3
AWS sdk: /usr/share/aws/aws-java-sdk/aws-java-sdk-1.11.160.jar
Applications:
Hive 2.3.0
Spark 2.2.0
Tez 0.8.4
On Tue, 3 Oct 2017 at 12:29 JG Perrin wrote:
> Hey Sparkians,
>
>
>
> What ve
10 matches
Mail list logo