Re: Review Request 67407: SQOOP-3327: Mainframe FTP needs to Include "Migrated" datasets when parsing the FTP list

2018-10-09 Thread Chris Teoh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67407/
---

(Updated Oct. 9, 2018, 10:44 p.m.)


Review request for Sqoop.


Changes
---

Updated indentation to match original file.


Repository: sqoop-trunk


Description
---

Mainframe FTP needs to Include "Migrated" datasets when parsing the FTP list. 
Initially, these were excluded out of the regular expression.


Diffs (updated)
-

  
src/java/org/apache/sqoop/mapreduce/mainframe/MainframeFTPFileEntryParser.java 
f0b87868 
  
src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileEntryParser.java
 eb0f8c00 


Diff: https://reviews.apache.org/r/67407/diff/6/

Changes: https://reviews.apache.org/r/67407/diff/5-6/


Testing
---

Unit testing.


Thanks,

Chris Teoh



Re: Review Request 67408: SQOOP-3326: Mainframe FTP listing for GDG should filter out non-GDG datasets in a heterogeneous listing

2018-10-09 Thread Chris Teoh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67408/
---

(Updated Oct. 9, 2018, 10:36 p.m.)


Review request for Sqoop.


Changes
---

Updated as per review feedback. Refactored some repeated strings to constants.


Repository: sqoop-trunk


Description
---

Mainframe FTP listing for GDG should filter out non-GDG datasets in a 
heterogeneous listing as it grabs the last file and in the case where there are 
other datasets mixed in, the latest file may not be the desired dataset.


Diffs (updated)
-

  build.xml cd2e9e29 
  src/java/org/apache/sqoop/mapreduce/mainframe/MainframeConfiguration.java 
9d6a2fe7 
  
src/java/org/apache/sqoop/mapreduce/mainframe/MainframeFTPFileGdgEntryParser.java
 PRE-CREATION 
  src/java/org/apache/sqoop/util/MainframeFTPClientUtils.java 654721e3 
  
src/scripts/thirdpartytest/docker-compose/sqoop-thirdpartytest-db-services.yml 
4648f545 
  src/test/org/apache/sqoop/manager/mainframe/MainframeManagerImportTest.java 
3b8ed236 
  src/test/org/apache/sqoop/manager/mainframe/MainframeTestUtil.java 9f86f6cd 
  
src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeFTPFileGdgEntryParser.java
 PRE-CREATION 
  src/test/org/apache/sqoop/util/TestMainframeFTPClientUtils.java 90a85194 


Diff: https://reviews.apache.org/r/67408/diff/4/

Changes: https://reviews.apache.org/r/67408/diff/3-4/


Testing
---

Unit tests. Integration testing locally on developer machine.


Thanks,

Chris Teoh



[jira] [Created] (SQOOP-3390) Document S3Guard usage with Sqoop

2018-10-09 Thread Boglarka Egyed (JIRA)
Boglarka Egyed created SQOOP-3390:
-

 Summary: Document S3Guard usage with Sqoop
 Key: SQOOP-3390
 URL: https://issues.apache.org/jira/browse/SQOOP-3390
 Project: Sqoop
  Issue Type: Sub-task
Reporter: Boglarka Egyed
Assignee: Boglarka Egyed






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SQOOP-3389) Unable to build Sqoop 1.4.7 due to upstream TLS 1.2 issue

2018-10-09 Thread Boglarka Egyed (JIRA)


[ 
https://issues.apache.org/jira/browse/SQOOP-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642951#comment-16642951
 ] 

Boglarka Egyed commented on SQOOP-3389:
---

[~devin.bost] your error log seems very odd to me, all the URLs are like this:
{noformat}
url=https://repository.cloudera.com/content/repositories/releases/x{noformat}
I don't understand why it tries to resolve every dependency from 
{{repository.cloudera.com.}} Do you maybe use some extra configuration in your 
environment? Where did you get the Sqoop source from?

> Unable to build Sqoop 1.4.7 due to upstream TLS 1.2 issue
> -
>
> Key: SQOOP-3389
> URL: https://issues.apache.org/jira/browse/SQOOP-3389
> Project: Sqoop
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.4.7
> Environment: Sqoop 1.4.7
> Java SDK 1.6.0_45 
> Ant 1.7.1
> Windows 10 via MinGW64
>Reporter: Devin G. Bost
>Priority: Blocker
>  Labels: build
> Attachments: sqoop_build_errors.txt
>
>
> When building Sqoop 1.4.7 with Java SDK 1.6.0_45 and Ant 1.7.1 on Windows in 
> MinGW64, I obtain these build errors:
> [ivy:resolve] Server access Error: Remote host closed connection during 
> handshake 
> url=https://repository.cloudera.com/content/repositories/releases/org/apache/avro/avro-mapred/1.8.1/avro-mapred-1.8.1-hadoop2.pom
> [ivy:resolve] Server access Error: Remote host closed connection during 
> handshake 
> url=https://repository.cloudera.com/content/repositories/releases/org/apache/avro/avro-mapred/1.8.1/avro-mapred-1.8.1-hadoop2.jar
> [ivy:resolve] Server access Error: Remote host closed connection during 
> handshake 
> url=https://repository.cloudera.com/content/repositories/staging/org/apache/avro/avro-mapred/1.8.1/avro-mapred-1.8.1-hadoop2.pom
> [ivy:resolve] Server access Error: Remote host closed connection during 
> handshake 
> url=[https://repository.cloudera.com/content/repositories/staging/org/apache/avro/avro-mapred/1.8.1/avro-mapred-1.8.1-hadoop2.jar]
>  
> I experienced similar build errors on a different project, which I traced to 
> this issue: 
> [https://stackoverflow.com/questions/21245796/javax-net-ssl-sslhandshakeexception-remote-host-closed-connection-during-handsh/22629008]
> The problem, however, is that because this particular build of Sqoop requires 
> Java 1.6, TLS v1.2 is unsupported, according to here: 
> [https://stackoverflow.com/questions/33364100/how-to-use-tls-1-2-in-java-6]
> which is a problem because some public repositories have dropped support for 
> TLS versions prior to 1.2, as reported here: 
> [https://github.com/Microsoft/vcpkg/issues/2969]
> and here: 
> [https://blog.github.com/2018-02-23-weak-cryptographic-standards-removed/]
> If it is impossible now to pull the upstream dependencies when building via 
> Ant due to TLS 1.2 being unsupported in Java 1.6, then this is a critical 
> dependency conflict. 
> If additional configuration steps are required to be able to successfully 
> build, they are not documented in any of the Sqoop documentation that I have 
> found. 
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)