[
https://issues.apache.org/jira/browse/SQOOP-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
William Watson updated SQOOP-1333:
----------------------------------
Description:
I see some issues resolved with importing from MySQL to S3 (SQOOP-891), but I
can't find any information on the following command and error:
{code}
sqoop import -Dfs.defaultFS=s3n:// --connect
"jdbc:postgresql://ip_address/cleanroom_matching?extra_options=-Dfs.defaultFS%3Ds3n%3A%2F%2F"
--fields-terminated-by \\t --username [omitted] --password [omitted]
--split-by cr_user_id --query "SELECT * FROM table WHERE (\$CONDITIONS)"
--direct --delete-target-dir --target-dir
's3n://[omitted]:[omitted]@[omitted]/sqoop/' --verbose
{code}
{code}
14/06/03 09:49:53 ERROR tool.ImportTool: Imported Failed: null
{code}
That's it, there's no stack trace. The query works on its own, if I import to
disk it works just fine. It's when I change to S3 that it fails. It was failing
because I didn't set the default file system after I did that, I started
getting the confusing error.
was:
I see some issues resolved with importing from MySQL to S3 (SQOOP-891), but I
can't find any information on the following command and error:
{code}
sqoop import -Dfs.defaultFS=s3n:// --connect
"jdbc:postgresql://33.33.33.12/cleanroom_matching?extra_options=-Dfs.defaultFS%3Ds3n%3A%2F%2F"
--fields-terminated-by \\t --username cleanroom_user --password waterfall
--split-by cr_user_id --query "SELECT cr_user_id, ad_day, ad_hour,
agency_organization_id, advertiser_organization_id, pixel_id, av1, av2, av3,
av4, av5, av6, av7, av8, av9, number_of_actions, lvl FROM (SELECT
cr_user_id,
ad_day,
ad_hour,
agency_organization_id,
advertiser_organization_id,
pixel_id,
av1,
av2,
av3,
av4,
av5,
av6,
av7,
av8,
av9,
number_of_actions,
lvl
FROM match_id_to_cr_user_id_mapping
JOIN match_manifest USING (match_id)
JOIN cleanroom_actions USING (match_id)) as transform_columns WHERE
(\$CONDITIONS)" --direct --delete-target-dir --target-dir
's3n://AKIAIACJB644REMVVE7Q:ehmI6SGPzARCGf0+m+ModcKQxnK80u5ujCs3NQDf@korrelate-test/sqoop/ko2o.action-event-matched-users-test'
--map-column-java
cr_user_id=String,ad_day=String,ad_hour=String,agency_organization_id=String,advertiser_organization_id=String,pixel_id=String,av1=String,av2=String,av3=String,av4=String,av5=String,av6=String,av7=String,av8=String,av9=String,number_of_actions=String,lvl=String
--null-string '' --null-non-string '' --verbose
{code}
{code}
14/06/03 09:49:53 ERROR tool.ImportTool: Imported Failed: null
{code}
That's it, there's no stack trace. The query works on its own, if I import to
disk it works just fine. It's when I change to S3 that it fails. It was failing
because I didn't set the default file system after I did that, I started
getting the confusing error.
> Sqoop Fails to Import from PostgreSQL to S3 with Confusing "Imported Failed:
> null" exception
> --------------------------------------------------------------------------------------------
>
> Key: SQOOP-1333
> URL: https://issues.apache.org/jira/browse/SQOOP-1333
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.4
> Environment: CentOS 6, Hadoop 2, Sqoop 1.4.4
> Reporter: William Watson
>
> I see some issues resolved with importing from MySQL to S3 (SQOOP-891), but I
> can't find any information on the following command and error:
> {code}
> sqoop import -Dfs.defaultFS=s3n:// --connect
> "jdbc:postgresql://ip_address/cleanroom_matching?extra_options=-Dfs.defaultFS%3Ds3n%3A%2F%2F"
> --fields-terminated-by \\t --username [omitted] --password [omitted]
> --split-by cr_user_id --query "SELECT * FROM table WHERE (\$CONDITIONS)"
> --direct --delete-target-dir --target-dir
> 's3n://[omitted]:[omitted]@[omitted]/sqoop/' --verbose
> {code}
> {code}
> 14/06/03 09:49:53 ERROR tool.ImportTool: Imported Failed: null
> {code}
> That's it, there's no stack trace. The query works on its own, if I import to
> disk it works just fine. It's when I change to S3 that it fails. It was
> failing because I didn't set the default file system after I did that, I
> started getting the confusing error.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
