Yes, RowResult seems too large. But, I don't think "increasing the
child heap" is a good solution.
Let's see the bigTable paper.
Each row in the imagery table corresponds to a single
geographic segment. Rows are named to ensure that
adjacent geographic segments are stored near each other.
The table contains a column family to keep track of the
sources of data for each segment. This column family
has a large number of columns: essentially one for each
raw data image.
Yes, We can have a large number of columns in the one column-family.
In above case, I think ......
column:miles image: ...
=================================================
segment(x, y) column:1 miles <segment(x',y')>
column:2 miles <segment(x^,y^)>
.......
Then, we can search something within a N-mile radius. Right?
Finally, ... I need another solution.
On Sat, Nov 29, 2008 at 12:35 AM, Thibaut_ <[email protected]> wrote:
>
> Your application uses too much memory. Try increasing the child heap space
> for mapreduce applications. (It's in the hadoop configuration file,
> mapred.child.java.opts)
>
> Thibaut
>
>
> Edward J. Yoon-2 wrote:
>>
>> While run mapred, I received below error. The size of RowResult seems
>> too large. What do you think?
>>
>> ----
>> 08/11/27 13:42:49 INFO mapred.JobClient: map 0% reduce 0%
>> 08/11/27 13:42:55 INFO mapred.JobClient: map 50% reduce 0%
>> 08/11/27 13:43:09 INFO mapred.JobClient: map 50% reduce 8%
>> 08/11/27 13:43:13 INFO mapred.JobClient: map 50% reduce 16%
>> 08/11/27 13:43:15 INFO mapred.JobClient: Task Id :
>> attempt_200811271320_0006_m_000000_0, Status : FAILED
>> java.lang.OutOfMemoryError: Java heap space
>> at java.util.Arrays.copyOf(Arrays.java:2786)
>> at
>> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>> at java.io.DataOutputStream.write(DataOutputStream.java:90)
>> at
>> org.apache.hadoop.hbase.util.Bytes.writeByteArray(Bytes.java:65)
>> at org.apache.hadoop.hbase.io.Cell.write(Cell.java:152)
>> at
>> org.apache.hadoop.hbase.io.HbaseMapWritable.write(HbaseMapWritable.java:196)
>> at org.apache.hadoop.hbase.io.RowResult.write(RowResult.java:245)
>> at
>> org.apache.hadoop.hbase.util.Writables.getBytes(Writables.java:49)
>> at
>> org.apache.hadoop.hbase.util.Writables.copyWritable(Writables.java:134)
>>
>> --
>> Best Regards, Edward J. Yoon @ NHN, corp.
>> [email protected]
>> http://blog.udanax.org
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/java.lang.OutOfMemoryError%3A-Java-heap-space-tp20714065p20736470.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
[email protected]
http://blog.udanax.org
