[jira] [Commented] (SQOOP-3455) Sqoop job fails while importing to S3 as Parquet
[
https://issues.apache.org/jira/browse/SQOOP-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110959#comment-17110959
]
Boglarka Egyed commented on SQOOP-3455:
---
Hi [~shailendra...@gmail.com], there is no active effort around putting
together a next release at the moment, thus I'm not able to provide any ETA.
This may worth a question at the @dev mailing list if it is urgent, to see if
there is someone in the community who would take the lead of a next release.
> Sqoop job fails while importing to S3 as Parquet
>
>
> Key: SQOOP-3455
> URL: https://issues.apache.org/jira/browse/SQOOP-3455
> Project: Sqoop
> Issue Type: Bug
> Components: sqoop2-kite-connector
>Affects Versions: 1.4.7
>Reporter: Kriti Jha
>Priority: Blocker
>
> A Sqoop job to import data from a MySQL database into S3 fails on using
> --as-parquetfile with the error as shown below:
>
> {{ERROR sqoop.Sqoop: Got exception running Sqoop:
> org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern:
> dataset:s3://sqoop-trial-bucket/sqoop-trial/trial
> Check that JARs for s3 datasets are on the classpath
> org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern:
> dataset:s3://}}{{sqoop-trial-bucket}}{{/sqoop-trial/trial Check that JARs for
> s3 datasets are on the classpath at
> org.kitesdk.data.spi.Registration.lookupDatasetUri(Registration.java:128) at
> org.kitesdk.data.Datasets.exists(Datasets.java:624) at
> org.kitesdk.data.Datasets.exists(Datasets.java:646) at
> org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:118)
> at
> org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:132)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264) at
> org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692) at
> org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127) at
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520) at
> org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) at
> org.apache.sqoop.Sqoop.run(Sqoop.java:147) at
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at
> org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at
> org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at
> org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at
> org.apache.sqoop.Sqoop.main(Sqoop.java:252)}}
>
> {{}}
> {{All the JARs for S3 are present in the classpath. Further, the same works
> on simply removing the argument --as-parquetfile, i.e. with any other
> format.}}
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (SQOOP-3455) Sqoop job fails while importing to S3 as Parquet
[
https://issues.apache.org/jira/browse/SQOOP-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110139#comment-17110139
]
Shailendra Kumar commented on SQOOP-3455:
-
[~BoglarkaEgyed] Hi, When is the official release for the above mentioned bugs.
> Sqoop job fails while importing to S3 as Parquet
>
>
> Key: SQOOP-3455
> URL: https://issues.apache.org/jira/browse/SQOOP-3455
> Project: Sqoop
> Issue Type: Bug
> Components: sqoop2-kite-connector
>Affects Versions: 1.4.7
>Reporter: Kriti Jha
>Priority: Blocker
>
> A Sqoop job to import data from a MySQL database into S3 fails on using
> --as-parquetfile with the error as shown below:
>
> {{ERROR sqoop.Sqoop: Got exception running Sqoop:
> org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern:
> dataset:s3://sqoop-trial-bucket/sqoop-trial/trial
> Check that JARs for s3 datasets are on the classpath
> org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern:
> dataset:s3://}}{{sqoop-trial-bucket}}{{/sqoop-trial/trial Check that JARs for
> s3 datasets are on the classpath at
> org.kitesdk.data.spi.Registration.lookupDatasetUri(Registration.java:128) at
> org.kitesdk.data.Datasets.exists(Datasets.java:624) at
> org.kitesdk.data.Datasets.exists(Datasets.java:646) at
> org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:118)
> at
> org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:132)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264) at
> org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692) at
> org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127) at
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520) at
> org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) at
> org.apache.sqoop.Sqoop.run(Sqoop.java:147) at
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at
> org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at
> org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at
> org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at
> org.apache.sqoop.Sqoop.main(Sqoop.java:252)}}
>
> {{}}
> {{All the JARs for S3 are present in the classpath. Further, the same works
> on simply removing the argument --as-parquetfile, i.e. with any other
> format.}}
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (SQOOP-3455) Sqoop job fails while importing to S3 as Parquet
[
https://issues.apache.org/jira/browse/SQOOP-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110132#comment-17110132
]
Shailendra Kumar commented on SQOOP-3455:
-
[~BoglarkaEgyed] when can we expect this fixes .
My sqoop version - 1.4.7
> Sqoop job fails while importing to S3 as Parquet
>
>
> Key: SQOOP-3455
> URL: https://issues.apache.org/jira/browse/SQOOP-3455
> Project: Sqoop
> Issue Type: Bug
> Components: sqoop2-kite-connector
>Affects Versions: 1.4.7
>Reporter: Kriti Jha
>Priority: Blocker
>
> A Sqoop job to import data from a MySQL database into S3 fails on using
> --as-parquetfile with the error as shown below:
>
> {{ERROR sqoop.Sqoop: Got exception running Sqoop:
> org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern:
> dataset:s3://sqoop-trial-bucket/sqoop-trial/trial
> Check that JARs for s3 datasets are on the classpath
> org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern:
> dataset:s3://}}{{sqoop-trial-bucket}}{{/sqoop-trial/trial Check that JARs for
> s3 datasets are on the classpath at
> org.kitesdk.data.spi.Registration.lookupDatasetUri(Registration.java:128) at
> org.kitesdk.data.Datasets.exists(Datasets.java:624) at
> org.kitesdk.data.Datasets.exists(Datasets.java:646) at
> org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:118)
> at
> org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:132)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264) at
> org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692) at
> org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127) at
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520) at
> org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) at
> org.apache.sqoop.Sqoop.run(Sqoop.java:147) at
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at
> org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at
> org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at
> org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at
> org.apache.sqoop.Sqoop.main(Sqoop.java:252)}}
>
> {{}}
> {{All the JARs for S3 are present in the classpath. Further, the same works
> on simply removing the argument --as-parquetfile, i.e. with any other
> format.}}
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (SQOOP-3455) Sqoop job fails while importing to S3 as Parquet
[
https://issues.apache.org/jira/browse/SQOOP-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966604#comment-16966604
]
Boglarka Egyed commented on SQOOP-3455:
---
[~kritijha] this is the same issue as SQOOP-3453.
Sqoop import into S3 is not supported with this version, it has been introduced
only by SQOOP-3345 but has not been included into any official release yet. It
works in trunk version though. (There are some use cases that are already
working with 1.4.7 too but these are not tested at all.)
Furthermore, Parquet import into S3 will require some more options to set as
traditionally Sqoop used Kite SDK to read/write Parquet that has been changed
in SQOOP-3313 because it has many limitations. This change has also not been
released yet but can be found in trunk.
> Sqoop job fails while importing to S3 as Parquet
>
>
> Key: SQOOP-3455
> URL: https://issues.apache.org/jira/browse/SQOOP-3455
> Project: Sqoop
> Issue Type: Bug
> Components: sqoop2-kite-connector
>Affects Versions: 1.4.7
>Reporter: Kriti Jha
>Priority: Blocker
>
> A Sqoop job to import data from a MySQL database into S3 fails on using
> --as-parquetfile with the error as shown below:
>
> {{ERROR sqoop.Sqoop: Got exception running Sqoop:
> org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern:
> dataset:s3://sqoop-trial-bucket/sqoop-trial/trial
> Check that JARs for s3 datasets are on the classpath
> org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern:
> dataset:s3://}}{{sqoop-trial-bucket}}{{/sqoop-trial/trial Check that JARs for
> s3 datasets are on the classpath at
> org.kitesdk.data.spi.Registration.lookupDatasetUri(Registration.java:128) at
> org.kitesdk.data.Datasets.exists(Datasets.java:624) at
> org.kitesdk.data.Datasets.exists(Datasets.java:646) at
> org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:118)
> at
> org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:132)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264) at
> org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692) at
> org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127) at
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520) at
> org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) at
> org.apache.sqoop.Sqoop.run(Sqoop.java:147) at
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at
> org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at
> org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at
> org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at
> org.apache.sqoop.Sqoop.main(Sqoop.java:252)}}
>
> {{}}
> {{All the JARs for S3 are present in the classpath. Further, the same works
> on simply removing the argument --as-parquetfile, i.e. with any other
> format.}}
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
