The fix in the https://issues.apache.org/jira/browse/HIVE-7164. does not works.
On Tue, Mar 1, 2016 at 10:51 AM, Richa Sharma <mailtorichasha...@gmail.com> wrote: > Great! > > So what is the interim fix you are implementing > > Richa > On Mar 1, 2016 4:06 PM, "Biswajit Nayak" <biswa...@altiscale.com> wrote: > >> Thanks Richa. >> >> The issue was suppose to be fixed in Hive 0.12 version as per the jira >> https://issues.apache.org/jira/browse/HIVE-7164. >> >> Even raised a ticket in sqoop jira [SQOOP-2840] for this . >> >> Thanks >> Biswa >> >> >> >> >> >> On Tue, Mar 1, 2016 at 9:56 AM, Richa Sharma <mailtorichasha...@gmail.com >> > wrote: >> >>> Hi, >>> >>> The values should still persist if partition column data type in Hive is >>> a string. >>> >>> I am checking HCatalog documentation for support of int data type in >>> partition column. >>> >>> Cheers >>> Richa >>> >>> On Tue, Mar 1, 2016 at 3:06 PM, Biswajit Nayak <biswa...@altiscale.com> >>> wrote: >>> >>>> Hi Richa, >>>> >>>> Thats a work around. But how to handle the columns with INT type. >>>> Changing the type will be the last option for me. >>>> >>>> Regards >>>> Biswa >>>> >>>> >>>> >>>> On Tue, Mar 1, 2016 at 9:31 AM, Richa Sharma < >>>> mailtorichasha...@gmail.com> wrote: >>>> >>>>> Hi Biswajit >>>>> >>>>> The answer is in the last line of the error message. Change the data >>>>> type of partition column to string in hive and try again. >>>>> >>>>> Hope it helps ! >>>>> >>>>> Richa >>>>> >>>>> 16/02/12 08:04:12 ERROR tool.ExportTool: Encountered IOException running >>>>> export job: java.io.IOException: The table provided default.emp_details1 >>>>> uses unsupported partitioning key type for column salary : int. Only >>>>> string fields are allowed in partition columns in Catalog >>>>> >>>>> >>>>> On Tue, Mar 1, 2016 at 2:19 PM, Biswajit Nayak <biswa...@altiscale.com >>>>> > wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> I am trying to do a SQOOP export from hive( integer type partition) >>>>>> to mysql through HCAT and it fails with the following error. >>>>>> >>>>>> Versions:- >>>>>> >>>>>> Hadoop :- 2.7.1 >>>>>> Hive :- 1.2.0 >>>>>> Sqoop :- 1.4.5 >>>>>> >>>>>> Table in Hive :- >>>>>> >>>>>> >>>>>> hive> use default; >>>>>> OK >>>>>> Time taken: 0.028 seconds >>>>>> hive> describe emp_details1; >>>>>> OK >>>>>> id int >>>>>> name string >>>>>> deg string >>>>>> dept string >>>>>> salary int >>>>>> >>>>>> # Partition Information >>>>>> # col_name data_type comment >>>>>> >>>>>> salary int >>>>>> Time taken: 0.125 seconds, Fetched: 10 row(s) >>>>>> hive> >>>>>> >>>>>> hive> select * from emp_details1; >>>>>> OK >>>>>> 1201 gopal 50000 >>>>>> 1202 manisha 50000 >>>>>> 1203 kalil 50000 >>>>>> 1204 prasanth 50000 >>>>>> 1205 kranthi 50000 >>>>>> 1206 satish 50000 >>>>>> Time taken: 0.195 seconds, Fetched: 6 row(s) >>>>>> hive> >>>>>> >>>>>> >>>>>> Conf added to Hive metastore site.xml >>>>>> >>>>>> >>>>>> [alti-test-01@hdpnightly271-ci-91-services ~]$ grep -A5 -B2 -i >>>>>> "hive.metastore.integral.jdo.pushdown" /etc/hive-metastore/hive-site.xml >>>>>> </property> >>>>>> <property> >>>>>> <name>hive.metastore.integral.jdo.pushdown</name> >>>>>> <value>TRUE</value> >>>>>> </property> >>>>>> >>>>>> </configuration> >>>>>> [alti-test-01@hdpnightly271-ci-91-services ~]$ >>>>>> >>>>>> >>>>>> The issue remains same >>>>>> >>>>>> >>>>>> [alti-test-01@hdpnightly271-ci-91-services ~]$ >>>>>> /opt/sqoop-1.4.5/bin/sqoop export --connect >>>>>> jdbc:mysql://localhost:3306/test --username hive --password ********* >>>>>> --table employee --hcatalog-database default --hcatalog-table >>>>>> emp_details1 >>>>>> Warning: /opt/sqoop-1.4.5/bin/../../hbase does not exist! HBase imports >>>>>> will fail. >>>>>> Please set $HBASE_HOME to the root of your HBase installation. >>>>>> Warning: /opt/sqoop-1.4.5/bin/../../accumulo does not exist! Accumulo >>>>>> imports will fail. >>>>>> Please set $ACCUMULO_HOME to the root of your Accumulo installation. >>>>>> Warning: /opt/sqoop-1.4.5/bin/../../zookeeper does not exist! Accumulo >>>>>> imports will fail. >>>>>> Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. >>>>>> 16/02/12 08:04:00 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5 >>>>>> 16/02/12 08:04:00 WARN tool.BaseSqoopTool: Setting your password on the >>>>>> command-line is insecure. Consider using -P instead. >>>>>> 16/02/12 08:04:00 INFO manager.MySQLManager: Preparing to use a MySQL >>>>>> streaming resultset. >>>>>> 16/02/12 08:04:00 INFO tool.CodeGenTool: Beginning code generation >>>>>> 16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement: >>>>>> SELECT t.* FROM `employee` AS t LIMIT 1 >>>>>> 16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement: >>>>>> SELECT t.* FROM `employee` AS t LIMIT 1 >>>>>> 16/02/12 08:04:01 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is >>>>>> /opt/hadoop >>>>>> Note: >>>>>> /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.java >>>>>> uses or overrides a deprecated API. >>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>> 16/02/12 08:04:07 INFO orm.CompilationManager: Writing jar file: >>>>>> /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.jar >>>>>> 16/02/12 08:04:07 INFO mapreduce.ExportJobBase: Beginning export of >>>>>> employee >>>>>> 16/02/12 08:04:08 INFO mapreduce.ExportJobBase: Configuring HCatalog for >>>>>> export job >>>>>> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Configuring HCatalog >>>>>> specific details for job >>>>>> 16/02/12 08:04:08 INFO manager.SqlManager: Executing SQL statement: >>>>>> SELECT t.* FROM `employee` AS t LIMIT 1 >>>>>> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column names >>>>>> projected : [id, name, deg, salary, dept] >>>>>> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column name - >>>>>> info map : >>>>>> id : [Type : 4,Precision : 11,Scale : 0] >>>>>> name : [Type : 12,Precision : 20,Scale : 0] >>>>>> deg : [Type : 12,Precision : 20,Scale : 0] >>>>>> salary : [Type : 4,Precision : 11,Scale : 0] >>>>>> dept : [Type : 12,Precision : 10,Scale : 0] >>>>>> >>>>>> 16/02/12 08:04:10 INFO hive.metastore: Trying to connect to metastore >>>>>> with URI thrift://hive-hdpnightly271-ci-91.test.altiscale.com:9083 >>>>>> 16/02/12 08:04:10 INFO hive.metastore: Connected to metastore. >>>>>> 16/02/12 08:04:11 INFO hcat.SqoopHCatUtilities: HCatalog full table >>>>>> schema fields = [id, name, deg, dept, salary] >>>>>> 16/02/12 08:04:12 ERROR tool.ExportTool: Encountered IOException running >>>>>> export job: java.io.IOException: The table provided default.emp_details1 >>>>>> uses unsupported partitioning key type for column salary : int. Only >>>>>> string fields are allowed in partition columns in Catalog >>>>>> >>>>>> >>>>>> Stuck with this issue. Any one had conquered this before. >>>>>> >>>>>> Regards >>>>>> Biswa >>>>>> >>>>>> >>>>> >>>> >>> >>
