Thanks, setting hbase.bulkload.retries.retryOnIOException to true in the
configuration worked. My Hbase cluster is colocated with the Yarn cluster
on EMR.
On Thu, Sep 7, 2017 at 4:08 AM, Ankit Singhal
wrote:
> bq. This runs successfully if I split this into 2 files,
bq. This runs successfully if I split this into 2 files, but I'd like to
avoid doing that.
do you run a different job for each file?
if your HBase cluster is not co-located with your yarn cluster then it may
be possible that copying of large HFile is timing out(this may happen due
to the fewer
bq. hbase.bulkload.retries.retryOnIOException is disabled. Unable to recover
The above is from HBASE-17165.
See if the load can pass after enabling the config.
On Wed, Sep 6, 2017 at 3:11 PM, Sriram Nookala wrote:
> It finally times out with these exceptions
>
> ed Sep
It finally times out with these exceptions
ed Sep 06 21:38:07 UTC 2017,
RpcRetryingCaller{globalStartTime=1504731276347, pause=100, retries=35},
java.io.IOException: Call to ip-10-123-0-60.ec2.internal/10.123.0.60:16020
failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException:
Phoenix 4.11.0, HBase 1.3.1
This is what I get from jstack
"main" #1 prio=5 os_prio=0 tid=0x7fb3d0017000 nid=0x5de7 waiting on
condition [0x7fb3d75f7000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xf588> (a
Do you have more details on the version of Phoenix/HBase you are using as
well as how it hangs (Exceptions/messages that may help to understand the
problem)?
Thanks,
Sergey
On Wed, Sep 6, 2017 at 1:13 PM, Sriram Nookala wrote:
> I'm trying to load a 3.5G file with 60
I'm trying to load a 3.5G file with 60 million rows using CsvBulkLoadTool.
It hangs while loading HFiles. This runs successfully if I split this into
2 files, but I'd like to avoid doing that. This is on Amazon EMR, is this
an issue due to disk space or memory. I have a single master and 2 region