Re: Review Request 28595: Date and DateTime is not encoded in Single Quotes

[Jarek Cecho]

> On Dec. 2, 2014, 6:04 a.m., Jarek Cecho wrote:
> > The patch looks good to me.
> > 
> > Integration test PartitionerTest is failing on my local computer, so I have 
> > to investigate whether it's environmental or not before I can commit it 
> > myself.
> 
> Veena Basavaraj wrote:
>     This is one test I usually skip since it eats away my CPU ..!
>     
>     looks like the tests used date time so I missed changing the expected. . 
> I am updating the tests and a new patch
> 
> Veena Basavaraj wrote:
>     Looks like this is more convuluted change that I expected. The generic 
> JDBC connector needs to encode dates in quotes I suppose.
> 
> Veena Basavaraj wrote:
>     Oh well lost my last comment.
>     
>     so traced this from start
>     
>     we do this in the test call to load test data
>         provider.insertRow(tableBaseName,  1, "Warty Warthog",    4.10,  
> "2004-10-20", false);
>     
>     Then GenericJdbcExtractor
>             context.getDataWriter().writeArrayRecord(array);
>     it calls the SqoopMapDataWriter
>      @Override
>         public void writeArrayRecord(Object[] array) {
>           fromDataFormat.setObjectData(array);
>           writeContent();
>         }
>     it then calls CSVIDF wich encodes the date in single quotes
>     @Override
>       public void setObjectData(Object[] data) {
>         Column[] columnArray = schema.getColumns().toArray(new 
> Column[data.length]);
>         encodeCSVStringElements(data, columnArray);
>         this.data = StringUtils.join(data, SEPARATOR_CHARACTER);
>       }
>       
>       on the TO side HDFSLoader
>       
>             while ((csv = reader.readTextRecord()) != null) {
>             filewriter.write(csv);
>        which calls SqoopOutputFormatDataReader
>        
>         public String readTextRecord() throws InterruptedException {
>           acquireSema();
>           // If the writer has finished, there is definitely no data remaining
>           if (writerFinished) {
>             return null;
>           }
>           try {
>             return dataFormat.getTextData();
>           } finally {
>             releaseSema();
>           }
>         }
>         
>         
>       which then calls CSVIDF
>       
>         */
>       @Override
>       public String getTextData() {
>         return data;
>       }
>     
>     the output should be have dates in single quotes, but it does not.
>     
>         <failure message="expected:&lt;3&gt; but was:&lt;10&gt;" 
> type="java.lang.AssertionError">java.lang.AssertionError: expected:&lt;3&gt; 
> but was:&lt;10&gt;

Nice work investigating Veena, I'm running the integration tests and I'll 
commit them if they passes.


- Jarek


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28595/#review63493
-----------------------------------------------------------


On Dec. 2, 2014, 3 p.m., Veena Basavaraj wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28595/
> -----------------------------------------------------------
> 
> (Updated Dec. 2, 2014, 3 p.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-1815
>     https://issues.apache.org/jira/browse/SQOOP-1815
> 
> 
> Repository: sqoop-sqoop2
> 
> 
> Description
> -------
> 
> see jira
> 
> 
> Diffs
> -----
> 
>   
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
>  bd0fbf0 
>   
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/idf/TestCSVIntermediateDataFormat.java
>  75fe429 
>   
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/PartitionerTest.java
>  824a51d 
> 
> Diff: https://reviews.apache.org/r/28595/diff/
> 
> 
> Testing
> -------
> 
> yes
> 
> 
> Thanks,
> 
> Veena Basavaraj
> 
>