Help to add me as contributor

[Ying Cao]

Hi all,

Anyone could kindly add me as contributor?

Thanks
Angela

[jira] [Updated] (SQOOP-1904) support for DB2 XML data type when importing to hdfs

[Ying Cao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SQOOP-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ying Cao updated SQOOP-1904:

Attachment: SQOOP-1904.7.patch

> support for DB2 XML data type when importing to hdfs
> 
>
> Key: SQOOP-1904
> URL: https://issues.apache.org/jira/browse/SQOOP-1904
> Project: Sqoop
>  Issue Type: Improvement
>  Components: connectors
>Affects Versions: 1.4.5
> Environment: RedHat6.4 + Sqoop1.4.5 + Hadoop 2.4.1 + Hive 0.14.0 + 
> DB2 10.5 
>Reporter: xieshiju
>Assignee: xieshiju
>  Labels: features
> Fix For: 1.4.7
>
> Attachments: SQOOP-1904.2.patch, SQOOP-1904.3.patch, 
> SQOOP-1904.4.patch, SQOOP-1904.5.patch, SQOOP-1904.6.patch, 
> SQOOP-1904.7.patch, SQOOP-1904.patch
>
>   Original Estimate: 504h
>  Remaining Estimate: 504h
>
> The DB2 connector should add XML data type support otherwise you will get an 
> error that Hive does not support the SQL type.
> Example with column DESCRIPTION data type XML:
> $SQOOP_HOME/bin/sqoop create-hive-table --connect 
> 'jdbc:db2://xxx.svl.ibm.com:6/SAMPLE' --username db2inst1 --password 
> db2inst1 --hive-table product_ct --table PRODUCT



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (SQOOP-3142) Restore fail messages removed in SQOOP-3092

[Boglarka Egyed (JIRA)]

Boglarka Egyed created SQOOP-3142:
-

 Summary: Restore fail messages removed in SQOOP-3092
 Key: SQOOP-3142
 URL: https://issues.apache.org/jira/browse/SQOOP-3142
 Project: Sqoop
  Issue Type: Sub-task
Affects Versions: 1.4.6
Reporter: Boglarka Egyed
Assignee: Boglarka Egyed
Priority: Minor


Restore fail messages on excepted exceptions using 
reportMissingExceptionWithMessage functionality added in JUnit 4.12



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (SQOOP-3100) Clean up expected exception logic in tests - part III.

[Boglarka Egyed (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SQOOP-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Boglarka Egyed updated SQOOP-3100:
--
Attachment: SQOOP-3100.patch

> Clean up expected exception logic in tests - part III.
> --
>
> Key: SQOOP-3100
> URL: https://issues.apache.org/jira/browse/SQOOP-3100
> Project: Sqoop
>  Issue Type: Sub-task
>Affects Versions: 1.4.6
>Reporter: Boglarka Egyed
>Assignee: Boglarka Egyed
>Priority: Minor
> Attachments: SQOOP-3100.patch, SQOOP-3100.patch
>
>
> Test classes affected by this ticket:
> * TestIncrementalImport
> * TestClassWriter
> * TestRecordParser
> * TestSacedJobs
> * LobFileStressTest



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (SQOOP-3143) Restore fail messages removed in SQOOP-3091

[Boglarka Egyed (JIRA)]

Boglarka Egyed created SQOOP-3143:
-

 Summary: Restore fail messages removed in SQOOP-3091
 Key: SQOOP-3143
 URL: https://issues.apache.org/jira/browse/SQOOP-3143
 Project: Sqoop
  Issue Type: Sub-task
Affects Versions: 1.4.6
Reporter: Boglarka Egyed
Assignee: Boglarka Egyed
Priority: Minor


Restore fail messages on excepted exceptions using 
reportMissingExceptionWithMessage functionality added in JUnit 4.12



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 55466: Clean up expected exception logic in tests - part III.

[Boglarka Egyed]

> On Feb. 26, 2017, 6:02 a.m., Attila Szabo wrote:
> > Hey Bogi,
> > 
> > The change looks good. I've only raised one concern:
> > The mechanism won't be indentical as before b/c  we will lose the text 
> > messages were included in the fail calls.
> > 
> > Would you please consider the usage of reportMissingExceptionWithMessage 
> > method?
> > 
> > Also: IMHO it would make sense to open JIRAs for your previous cleanup 
> > changes to add back those information in the same way.
> > 
> > Thanks,
> > Attila

Hey Attila,

Thanks for suggesting reportMissingExceptionWithMessage, I have included it in 
the logic. I also needed to upgrade to JUnit 4.12 (from 4.11) as this 
functionality exists from that version.

I'm opening a JIRA ticket for the previous changes to add these fail messages 
back.

Thanks,
Bogi


- Boglarka


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55466/#review166822
---


On Feb. 27, 2017, 8:01 p.m., Boglarka Egyed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55466/
> ---
> 
> (Updated Feb. 27, 2017, 8:01 p.m.)
> 
> 
> Review request for Sqoop, Attila Szabo, Anna Szonyi, and Liz Szilagyi.
> 
> 
> Bugs: SQOOP-3100
> https://issues.apache.org/jira/browse/SQOOP-3100
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> ---
> 
> Normalizing test cases where we except an exception using JUnit's 
> ExpectedException rule to make the code more clean and self-explanatory.
> 
> 
> Diffs
> -
> 
>   ivy/libraries.properties 05baa5cb46a841bbd686e950e575129820e7fcf2 
>   src/test/com/cloudera/sqoop/TestIncrementalImport.java 
> 57f4433d8f50b37f102a8a8ada7196d9287b5557 
>   src/test/com/cloudera/sqoop/lib/TestRecordParser.java 
> 57bdb5fe057f74dcaef9e2e3d6e1d26d3af7833a 
>   src/test/com/cloudera/sqoop/metastore/TestSavedJobs.java 
> 1fb732439dd2c8f187af726935a335c993895bc5 
>   src/test/com/cloudera/sqoop/orm/TestClassWriter.java 
> a27353df4ed84a7dfe7612662f3170c00951d2cf 
> 
> Diff: https://reviews.apache.org/r/55466/diff/
> 
> 
> Testing
> ---
> 
> ant test, ant clean test
> 
> 
> Thanks,
> 
> Boglarka Egyed
> 
>

Re: Review Request 55466: Clean up expected exception logic in tests - part III.

[Boglarka Egyed]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55466/
---

(Updated Feb. 27, 2017, 8:01 p.m.)


Review request for Sqoop, Attila Szabo, Anna Szonyi, and Liz Szilagyi.


Changes
---

Upgrading to JUnit 4.12 and using report reportMissingExceptionWithMessage 
where needed


Bugs: SQOOP-3100
https://issues.apache.org/jira/browse/SQOOP-3100


Repository: sqoop-trunk


Description
---

Normalizing test cases where we except an exception using JUnit's 
ExpectedException rule to make the code more clean and self-explanatory.


Diffs (updated)
-

  ivy/libraries.properties 05baa5cb46a841bbd686e950e575129820e7fcf2 
  src/test/com/cloudera/sqoop/TestIncrementalImport.java 
57f4433d8f50b37f102a8a8ada7196d9287b5557 
  src/test/com/cloudera/sqoop/lib/TestRecordParser.java 
57bdb5fe057f74dcaef9e2e3d6e1d26d3af7833a 
  src/test/com/cloudera/sqoop/metastore/TestSavedJobs.java 
1fb732439dd2c8f187af726935a335c993895bc5 
  src/test/com/cloudera/sqoop/orm/TestClassWriter.java 
a27353df4ed84a7dfe7612662f3170c00951d2cf 

Diff: https://reviews.apache.org/r/55466/diff/


Testing
---

ant test, ant clean test


Thanks,

Boglarka Egyed

Re: Review Request 55466: Clean up expected exception logic in tests - part III.

[Szabolcs Vasas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55466/#review166858
---


Ship it!




Ship It!

- Szabolcs Vasas


On Feb. 25, 2017, 5:46 p.m., Boglarka Egyed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55466/
> ---
> 
> (Updated Feb. 25, 2017, 5:46 p.m.)
> 
> 
> Review request for Sqoop, Attila Szabo, Anna Szonyi, and Liz Szilagyi.
> 
> 
> Bugs: SQOOP-3100
> https://issues.apache.org/jira/browse/SQOOP-3100
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> ---
> 
> Normalizing test cases where we except an exception using JUnit's 
> ExpectedException rule to make the code more clean and self-explanatory.
> 
> 
> Diffs
> -
> 
>   src/test/com/cloudera/sqoop/TestIncrementalImport.java 
> 57f4433d8f50b37f102a8a8ada7196d9287b5557 
>   src/test/com/cloudera/sqoop/lib/TestRecordParser.java 
> 57bdb5fe057f74dcaef9e2e3d6e1d26d3af7833a 
>   src/test/com/cloudera/sqoop/metastore/TestSavedJobs.java 
> 1fb732439dd2c8f187af726935a335c993895bc5 
>   src/test/com/cloudera/sqoop/orm/TestClassWriter.java 
> a27353df4ed84a7dfe7612662f3170c00951d2cf 
> 
> Diff: https://reviews.apache.org/r/55466/diff/
> 
> 
> Testing
> ---
> 
> ant test, ant clean test
> 
> 
> Thanks,
> 
> Boglarka Egyed
> 
>

[jira] [Commented] (SQOOP-3136) Sqoop should work well with not default file systems

[Anna Szonyi (JIRA)]

[ 
https://issues.apache.org/jira/browse/SQOOP-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885723#comment-15885723
 ] 

Anna Szonyi commented on SQOOP-3136:


Hey [~yalovyyi],

Understandable, most of the time so do I, unfortunately Sqoop still 
supports/uses Java 7 - we are planning to propose a switch to java 8 in the 
near future (that will have to be a community decision), in the mean time we're 
stuck with 7. We have source and target enforcements in  ant, though it is to 
enforce the related syntax more than the compiler/compilation itself afair. 

[~maugli] added a small patch to remove the offending method, so there is not 
much else to do :).

Thanks again for the contribution!

Thanks,
Anna

> Sqoop should work well with not default file systems
> 
>
> Key: SQOOP-3136
> URL: https://issues.apache.org/jira/browse/SQOOP-3136
> Project: Sqoop
>  Issue Type: Improvement
>  Components: connectors/hdfs
>Affects Versions: 1.4.5
>Reporter: Illya Yalovyy
>Assignee: Illya Yalovyy
> Attachments: SQOOP-3136.patch
>
>
> Currently Sqoop assumes default file system when it comes to IO operations. 
> It makes it hard to use other FileSystem implementations as source or 
> destination. Here is an example:
> {code}
> sqoop import --connect  --table table1 --driver  DRIVER> --username root --password  --delete-target-dir --target-dir 
> s3a://some-bucket/tmp/sqoop
> ...
> 17/02/15 19:16:59 ERROR tool.ImportTool: Imported Failed: Wrong FS: 
> s3a://some-bucket/tmp/sqoop, expected: hdfs://:8020
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: sqoop hbase incremental import - Sqoop 1.4.6

[Boglarka Egyed]

Hi Jilani,

To get your change committed please do the following:
* Open a JIRA ticket for your change in Apache's JIRA system
 for project Sqoop
* Create a review request at Apache's review board
 for project Sqoop and link it to the JIRA
ticket

Please consider the guidelines below:

Review board
* Summary: generate your summary using the issue's jira key + jira title
* Groups: add the relevant group so everyone on the project will know about
your patch (Sqoop)
* Bugs: add the issue's jira key so it's easy to navigate to the jira side
* Repository: sqoop-trunk for Sqoop1 or sqoop-sqoop2 for Sqoop2
* And as soon as the patch gets committed, it's very useful for the
community if you close the review and mark it as "Submitted" at the Review
board. The button to do this is top right at your own tickets, right next
to  the Download Diff button.

Jira
* Link: please add the link of the review as an external/web link so it's
easy to navigate to the reviews side
* Status: mark it as "patch available"

Sqoop community will receive emails about your new ticket and review
request and will review your change.

Thanks,
Bogi


On Sat, Feb 25, 2017 at 2:14 AM, Jilani Shaik  wrote:

> Do we have any update?
>
> I did checkout of the 1.4.6 code and done code changes to achieve this and
> tested in cluster and it is working as expected. Is there a way I can
> contribute this as a patch and then the committers can validate further and
> suggest if any changes required to move further. Please suggest the
> approach.
>
> Thanks,
> Jilani
>
> On Sun, Feb 5, 2017 at 10:41 PM, Jilani Shaik 
> wrote:
>
> > Hi Liz,
> >
> > lets say we inserted data in a table with initial import, that looks like
> > this in hbase shell
> >
> >  1 column=pay:amount,
> > timestamp=1485129654025, value=4.99
> >  1 column=pay:customer_id,
> > timestamp=1485129654025, value=1
> >  1 column=pay:last_update,
> > timestamp=1485129654025, value=2017-01-23 05:29:09.0
> >  1 column=pay:payment_date,
> > timestamp=1485129654025, value=2005-05-25 11:30:37.0
> >  1 column=pay:rental_id,
> > timestamp=1485129654025, value=573
> >  1 column=pay:staff_id,
> > timestamp=1485129654025, value=1
> >  10column=pay:amount,
> > timestamp=1485129504390, value=5.99
> >  10column=pay:customer_id,
> > timestamp=1485129504390, value=1
> >  10column=pay:last_update,
> > timestamp=1485129504390, value=2006-02-15 22:12:30.0
> >  10column=pay:payment_date,
> > timestamp=1485129504390, value=2005-07-08 03:17:05.0
> >  10column=pay:rental_id,
> > timestamp=1485129504390, value=4526
> >  10column=pay:staff_id,
> > timestamp=1485129504390, value=2
> >
> >
> > now assume that in source rental_id becomes NULL for rowkey "1", and then
> > we are doing incremental import into HBase. With current import the final
> > HBase data after incremental import will look like this.
> >
> >  1 column=pay:amount,
> > timestamp=1485129654025, value=4.99
> >  1 column=pay:customer_id,
> > timestamp=1485129654025, value=1
> >  1 column=pay:last_update,
> > timestamp=1485129654025, value=2017-02-05 05:29:09.0
> >  1 column=pay:payment_date,
> > timestamp=1485129654025, value=2005-05-25 11:30:37.0
> >  1 column=pay:rental_id,
> > timestamp=1485129654025, value=573
> >  1 column=pay:staff_id,
> > timestamp=1485129654025, value=1
> >  10column=pay:amount,
> > timestamp=1485129504390, value=5.99
> >  10column=pay:customer_id,
> > timestamp=1485129504390, value=1
> >  10column=pay:last_update,
> > timestamp=1485129504390, value=2017-02-05 05:12:30.0
> >  10column=pay:payment_date,
> > timestamp=1485129504390, value=2005-07-08 03:17:05.0
> >  10column=pay:rental_id,
> > timestamp=1485129504390, value=126
> >  10column=pay:staff_id,
> > timestamp=1485129504390, value=2
> >
> >
> >
> > As source column "rental_id" becomes NULL for rowkey "1", the final HBase
> > should not have the "rental_id" for this rowkey "1". I am expecting below
> > data for these rowkeys.
> >
> >
> >  1 column=pay:amount,
> >