[jira] [Commented] (SQOOP-3397) Java version issues with Sqoop on AWS EMR
[
https://issues.apache.org/jira/browse/SQOOP-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657555#comment-16657555
]
Devin G. Bost commented on SQOOP-3397:
--
Turns out that the solution was simpler than I expected. Just ensure that both
Java versions are installed on the machine (e.g. install Java 7 via: sudo yum
install java-1.7.0 java-1.7.0-openjdk-devel -y ) and then put:
{{ export JAVA_HOME=/usr/lib/jvm/java-1.7.0}}
into:
{{ /usr/lib/sqoop/conf/sqoop-env.sh}}
and ensure that your defaults are set back to Java 8 for everything else, like:
{{NR_OF_JRE_OPTIONS=$(echo 0 | alternatives --config java 2>/dev/null | grep
'There ' | awk '\{print $3}' | tail -1)}}
{{NR_OF_SDK_OPTIONS=$(echo 0 | alternatives --config javac 2>/dev/null | grep
'There ' | awk '\{print $3}' | tail -1)}}{{echo $(($NR_OF_SDK_OPTIONS - 1)) |
sudo alternatives --config javac}}
{{echo $(($NR_OF_JRE_OPTIONS - 1)) | sudo alternatives --config java}}
(Modified from here: [https://gist.github.com/pstorch/c217d8324c4133a003c4] )
> Java version issues with Sqoop on AWS EMR
> -
>
> Key: SQOOP-3397
> URL: https://issues.apache.org/jira/browse/SQOOP-3397
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.7
> Environment: Amazon EMR emr-5.14.0 and up.
>Reporter: Devin G. Bost
>Priority: Major
>
> The current build of Sqoop 1.4.7 appears to require Java 7.
> The version of Sqoop 1.4.7 on Amazon's Elastic Map Reduce (EMR) platform
> appears to be running on Java 8.
> We need a way to be able to build Sqoop extensions on Java 8 so that they are
> compatible with the version of Java running on the EMR versions of Sqoop.
> Otherwise, when attempts to run the extension of Sqoop are made (such as for
> a custom Manager class), Java version mismatch errors occur that block
> compilation of the class files that need to be generated for imports.
> How can we build Sqoop 1.4.7 on Java 8 so that extensions and plugins can be
> built with Java 8 so that they will still be compatible with the architecture
> of the other Apache tools (such as Hadoop) that otherwise break on EMR
> instances when the Java version must be switched to Java 7 to support Sqoop
> extensions and plugins?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SQOOP-3394) External Hive table tests should use unique external dir names
[ https://issues.apache.org/jira/browse/SQOOP-3394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656576#comment-16656576 ] Hudson commented on SQOOP-3394: --- FAILURE: Integrated in Jenkins build Sqoop-hadoop200 #1218 (See [https://builds.apache.org/job/Sqoop-hadoop200/1218/]) SQOOP-3394: External Hive table tests should use unique external dir (vasas: [https://git-wip-us.apache.org/repos/asf?p=sqoop.git=commit=15097756c7fc694ec86d003843f40a793bcd9633]) * (edit) src/test/org/apache/sqoop/testutil/S3TestUtils.java > External Hive table tests should use unique external dir names > -- > > Key: SQOOP-3394 > URL: https://issues.apache.org/jira/browse/SQOOP-3394 > Project: Sqoop > Issue Type: Sub-task >Affects Versions: 1.4.7 >Reporter: Boglarka Egyed >Assignee: Boglarka Egyed >Priority: Major > Fix For: 3.0.0 > > Attachments: SQOOP-3394.patch, SQOOP-3394.patch > > > Current external Hive table tests on S3 uses the same external directory name > in every unit test cases which can cause problems during running them in an > automated environment. These names should be unique in every test cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SQOOP-3394) External Hive table tests should use unique external dir names
[ https://issues.apache.org/jira/browse/SQOOP-3394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656495#comment-16656495 ] ASF subversion and git services commented on SQOOP-3394: Commit 15097756c7fc694ec86d003843f40a793bcd9633 in sqoop's branch refs/heads/trunk from [~vasas] [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=1509775 ] SQOOP-3394: External Hive table tests should use unique external dir names (Boglarka Egyed via Szabolcs Vasas) > External Hive table tests should use unique external dir names > -- > > Key: SQOOP-3394 > URL: https://issues.apache.org/jira/browse/SQOOP-3394 > Project: Sqoop > Issue Type: Sub-task >Affects Versions: 1.4.7 >Reporter: Boglarka Egyed >Assignee: Boglarka Egyed >Priority: Major > Attachments: SQOOP-3394.patch, SQOOP-3394.patch > > > Current external Hive table tests on S3 uses the same external directory name > in every unit test cases which can cause problems during running them in an > automated environment. These names should be unique in every test cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 69070: SQOOP-3394: External Hive table tests should use unique external dir names
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69070/#review209779 --- Ship it! Lgtm - Fero Szabo On Oct. 18, 2018, 5:49 p.m., Boglarka Egyed wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69070/ > --- > > (Updated Oct. 18, 2018, 5:49 p.m.) > > > Review request for Sqoop, Fero Szabo and Szabolcs Vasas. > > > Bugs: SQOOP-3394 > https://issues.apache.org/jira/browse/SQOOP-3394 > > > Repository: sqoop-trunk > > > Description > --- > > Current external Hive table tests on S3 uses the same external directory name > in every unit test cases which can cause problems during running them in an > automated environment. These names should be unique in every test cases. > > > Diffs > - > > src/test/org/apache/sqoop/testutil/S3TestUtils.java > 97d53bbaa7c72d2ad1b890d7a8367c45a3e2b95c > > > Diff: https://reviews.apache.org/r/69070/diff/2/ > > > Testing > --- > > ./gradlew test -Ds3.bucket.url= > -Ds3.generator.command= > > > Thanks, > > Boglarka Egyed > >
