[
https://issues.apache.org/jira/browse/PHOENIX-5452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916000#comment-16916000
]
sujit potluri edited comment on PHOENIX-5452 at 8/26/19 5:46 PM:
-----------------------------------------------------------------
Team,
I am facing following problem when trying to use lzo file:
{code:java}
//hdfs dfs -put data.csv.lzo /tmp{code}
{code:java}
HADOOP_CLASSPATH=/usr/hdp/2.6.4.0-91/hbase/lib/hbase-protocol-1.1.2.2.6.4.0-91.jar:/etc/hbase/conf
hadoop jar /usr/hdp/current/phoenix-client/phoenix-4.7.0.2.6.4.0-91-client.jar
org.apache.phoenix.mapreduce.CsvBulkLoadTool --table EXAMPLE --input
/tmp/data.csv.lzo{code}
{code:java}
19/08/22 15:28:47 INFO mapreduce.AbstractBulkLoadTool: Running MapReduce
import job from /tmp/data.csv.lzo to
/tmp/0d55520c-2db7-44a0-b2d6-926d4e4b70df{code}
{code:java}
19/08/22 15:29:54 INFO input.FileInputFormat: Total input paths to process : 1
19/08/22 15:29:54 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
19/08/22 15:29:54 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev c3bf57bd7310159c984a56403f3570d6ccb70f1c]
19/08/22 15:29:54 INFO mapreduce.JobSubmitter: number of splits:1 19/08/22
15:29:54 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1566486875553_0001{code}
{code:java}
19/08/22 15:29:55 INFO mapreduce.Job: Running job:
job_1566486875553_000119/08/22 15:29:55 INFO mapreduce.Job: Running job:
job_1566486875553_000119/08/22 15:30:08 INFO mapreduce.Job: Job
job_1566486875553_0001 running in uber mode : false19/08/22 15:30:08 INFO
mapreduce.Job: map 0% reduce 0%19/08/22 15:30:20 INFO mapreduce.Job: Task Id :
attempt_1566486875553_0001_m_000000_0, Status : FAILEDError:
java.lang.RuntimeException: java.lang.IllegalArgumentException: CSV record does
not have enough values (has 1, but needs 3) at
org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:201)
at
org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:73)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)Caused by:
java.lang.IllegalArgumentException: CSV record does not have enough values (has
1, but needs 3) at
org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:81)
at
org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:51)
at org.apache.phoenix.util.UpsertExecutor.execute(UpsertExecutor.java:133) at
org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:170)
... 9 more 19/08/22 15:30:29 INFO mapreduce.Job: Task Id :
attempt_1566486875553_0001_m_000000_1, Status : FAILEDError:
java.lang.RuntimeException: java.lang.IllegalArgumentException: CSV record does
not have enough values (has 1, but needs 3) at
org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:201)
at
org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:73)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)Caused by:
java.lang.IllegalArgumentException: CSV record does not have enough values (has
1, but needs 3) at
org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:81)
at
org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:51)
at org.apache.phoenix.util.UpsertExecutor.execute(UpsertExecutor.java:133) at
org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:170)
... 9 more 19/08/22 15:30:36 INFO mapreduce.Job: Task Id :
attempt_1566486875553_0001_m_000000_2, Status : FAILEDError:
java.lang.RuntimeException: java.lang.IllegalArgumentException: CSV record does
not have enough values (has 1, but needs 3) at
org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:201)
at
org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:73)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)Caused by:
java.lang.IllegalArgumentException: CSV record does not have enough values (has
1, but needs 3) at
org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:81)
at
org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:51)
at org.apache.phoenix.util.UpsertExecutor.execute(UpsertExecutor.java:133) at
org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:170)
... 9 more 19/08/22 15:30:49 INFO mapreduce.Job: map 100% reduce 100%19/08/22
15:30:49 INFO mapreduce.Job: Job job_1566486875553_0001 failed with state
FAILED due to: Task failed task_1566486875553_0001_m_000000Job failed as tasks
failed. failedMaps:1 failedReduces:0 19/08/22 15:30:49 INFO mapreduce.Job:
Counters: 13 Job Counters Failed map tasks=4 Killed reduce tasks=1 Launched
map tasks=4 Other local map tasks=3 Data-local map tasks=1 Total time spent by
all maps in occupied slots (ms)=34657 Total time spent by all reduces in
occupied slots (ms)=0 Total time spent by all map tasks (ms)=34657 Total time
spent by all reduce tasks (ms)=0 Total vcore-milliseconds taken by all map
tasks=34657 Total vcore-milliseconds taken by all reduce tasks=0 Total
megabyte-milliseconds taken by all map tasks=35488768 Total
megabyte-milliseconds taken by all reduce tasks=0
{code}
any ideas why it may not be working.
Regards
Sujit P
was (Author: sujitp0123):
Team,
I am facing following problem when trying to use lzo file:
{code:java}
//
{code}
> Document CsvBulkLoadTool can accept compressed files
> ----------------------------------------------------
>
> Key: PHOENIX-5452
> URL: https://issues.apache.org/jira/browse/PHOENIX-5452
> Project: Phoenix
> Issue Type: Improvement
> Affects Versions: 5.0.0, 4.15.0, 4.14.2, 4.14.3
> Reporter: Artem Ervits
> Assignee: Artem Ervits
> Priority: Major
> Labels: newbie
> Attachments: PHOENIX-5452-site.patch, PHOENIX-5452.v0.patch
>
>
> today, the bulk import tool only accepts uncompressed csv files, this Jira is
> to add ability to import compressed data.
> {code:java}
> Error: java.lang.RuntimeException:
> org.apache.phoenix.schema.IllegalDataException: java.sql.SQLException: ERROR
> 201 (22000): Illegal data.Error: java.lang.RuntimeException:
> org.apache.phoenix.schema.IllegalDataException: java.sql.SQLException: ERROR
> 201 (22000): Illegal data. at
> org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:205)
> at
> org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:77)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)Caused by:
> org.apache.phoenix.schema.IllegalDataException: java.sql.SQLException: ERROR
> 201 (22000): Illegal data. at
> org.apache.phoenix.schema.types.PDataType.newIllegalDataException(PDataType.java:310)
> at org.apache.phoenix.schema.types.PLong.toObject(PLong.java:228) at
> org.apache.phoenix.util.csv.CsvUpsertExecutor$SimpleDatatypeConversionFunction.apply(CsvUpsertExecutor.java:204)
> at
> org.apache.phoenix.util.csv.CsvUpsertExecutor$SimpleDatatypeConversionFunction.apply(CsvUpsertExecutor.java:120)
> at
> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:85)
> at
> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:52)
> at org.apache.phoenix.util.UpsertExecutor.execute(UpsertExecutor.java:133)
> at
> org.apache.phoenix.mapreduce.FormatToBytesWritableMapper.map(FormatToBytesWritableMapper.java:174)
> ... 9 moreCaused by: java.sql.SQLException: ERROR 201 (22000): Illegal data.
> at
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:498)
> at
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:150)
> ... 17 moreCaused by: java.lang.NumberFormatException: For input string:
> "data.csv0000664000175000017500000005065313527306733012416 0ustar
> vagrantvagrant71027" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Long.parseLong(Long.java:589) at
> java.lang.Long.parseLong(Long.java:631) at
> org.apache.phoenix.schema.types.PLong.toObject(PLong.java:226) ... 15 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
