Am not facing any problem. Am checking to see what are the reservations against not supporting complex joins with OR conditions. I would like to know when it could create a problem and would the problem be solvable by using a "view" or limiting the number of mappers to just 1. I would like to know if the problem if any is due to the parallelism which comes with increasing the number of mappers?
~Pratik On Thu, Sep 18, 2014 at 1:23 PM, Sambit Tripathy (RBEI/PJ-NBS) < [email protected]> wrote: > Pratik, > > > > Are you facing a problem or trying to make a recommendation? > > > > > > Regards, > > Sambit. > > > > > > *From:* pratik khadloya [mailto:[email protected]] > *Sent:* Thursday, September 18, 2014 1:09 PM > *To:* [email protected] > *Subject:* Complex free form queries > > > > The sqoop docs say: > > > > The facility of using free-form query in the current version of Sqoop is > limited to simple queries where there are no ambiguous projections and no > OR conditions in the WHERE clause. Use of complex queries such as queries > that have sub-queries or joins leading to ambiguous projections can lead to > unexpected results. > > > > Does anyone know why such is case is not supported and can it be avoided > by: > > > > a) Using only 1 mapper > > or > > b) Creating a view out of the complex query > > > > I have tested a hive textfile import for a very complex query and verified > the data and it seems to be correct. I checked the number of words, number > of lines and file sizes of the dump from mysql vs the text file imported > onto hdfs by sqoop. > > My query does have OR conditions. I have attached an obfuscated version of > the query, and that screenprint is still 1/2 of the complete query. > > > > Any info on this will be helpful. > > > > Thanks, > > Pratik >
