[
https://issues.apache.org/jira/browse/HBASE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12856119#action_12856119
]
Ted Yu commented on HBASE-2432:
-------------------------------
The motivation was due to row size limit of 32767.
The initial patch targeted String which represents Float. So it is not suitable
for restoring to arbitrary data types. For multibyte character strings, I
checked
StringCoding.encode(String charsetName, char[] ca, int off, int len) which
resorts to StringEncoder, a private class.
So more work is needed for multibyte character strings.
> enhance hbase.util.Bytes.toBytes() with length limit
> ----------------------------------------------------
>
> Key: HBASE-2432
> URL: https://issues.apache.org/jira/browse/HBASE-2432
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.20.1
> Reporter: Ted Yu
> Attachments: HBASE-2432.patch, HBASE-2432.patch
>
>
> The following stack trace is seen in our hadoop log:
> java.lang.IllegalArgumentException: Row > 32767
> at org.apache.hadoop.hbase.KeyValue.createByteArray(KeyValue.java:437)
> at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:405)
> at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:374)
> at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:353)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:137)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:108)
> at
> org.apache.nutch.scoring.webgraph.ScoreUpdater$ScoreUpdaterReducer.reduce(ScoreUpdater.java:170)
> at
> org.apache.nutch.scoring.webgraph.ScoreUpdater$ScoreUpdaterReducer.reduce(ScoreUpdater.java:127)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:563)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Bytes.toBytes(Float.valueOf(score).toString()) may return an array longer
> than 32767 bytes.
> We should enhance Bytes.toBytes() to include length limit:
> public static byte[] toBytes(String s, int length)
> String.getBytes() doesn't have length limit
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
