yeah Best Regards, Tariq +91-9741563634 https://mtariq.jux.com/
On Sat, Dec 22, 2012 at 6:57 AM, Mohit Anchlia <[email protected]>wrote: > You mean batch multiple put? > > On Fri, Dec 21, 2012 at 4:16 PM, Mohammad Tariq <[email protected]> > wrote: > > > It might be the RS which could not complete the operation in time. The > > appropriate way to find out is to monitor that RS's metrics and see if > > anything unusual is happening there. What type of keys are you using? It > is > > time-series data?You might be a victim of RS hotspotting in that case or > > perhaps some other processes are eating up resources there. Try using > > "put(List<Put> puts)" instead of "put(Put put)" and see if it makes any > > difference. > > > > I'm afraid, I can't say anything with 100% confidence as there could be > 'n' > > reasons which are not traceable from here. Some of the possible reasons > > could be : > > hotspotting region > > too much I/O wait due to Swapping > > overloaded disk > > slowness due to high cpu consumption > > > > Best Regards, > > Tariq > > +91-9741563634 > > https://mtariq.jux.com/ > > > > > > On Sat, Dec 22, 2012 at 5:23 AM, Mohit Anchlia <[email protected] > > >wrote: > > > > > I am just doing a put. This operation generally takes 10ms but in this > > case > > > it took more than 10sec. Nothing out of ordinary in the logs > > > > > > On Fri, Dec 21, 2012 at 3:26 PM, Mohammad Tariq <[email protected]> > > > wrote: > > > > > > > what exactly is the operation your trying to do?how is your network's > > > > health?is swapping too high at RS side?anything odd in your RS logs? > > > > > > > > Best Regards, > > > > Tariq > > > > +91-9741563634 > > > > https://mtariq.jux.com/ > > > > > > > > > > > > On Sat, Dec 22, 2012 at 4:36 AM, Mohit Anchlia < > [email protected] > > > > >wrote: > > > > > > > > > I looked at that link, but couldn't find anything useful. How do I > > > check > > > > if > > > > > it was client who didn't write data within that time or if it was > > > region > > > > > server that didn't finish operation in time. > > > > > > > > > > On Fri, Dec 21, 2012 at 2:54 PM, Mohammad Tariq < > [email protected]> > > > > > wrote: > > > > > > > > > > > The socket through which your client is communicating is getting > > > closed > > > > > > before the operation could get finished. May be it is taking > longer > > > > than > > > > > > usual or something. > > > > > > > > > > > > Best Regards, > > > > > > Tariq > > > > > > +91-9741563634 > > > > > > https://mtariq.jux.com/ > > > > > > > > > > > > > > > > > > On Sat, Dec 22, 2012 at 4:08 AM, Mohammad Tariq < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > Hello Mohit, > > > > > > > > > > > > > > You might this link< > > > > > > http://hbase.apache.org/book/ops.monitoring.html>useful. > > > > > > > > > > > > > > Best Regards, > > > > > > > Tariq > > > > > > > +91-9741563634 > > > > > > > https://mtariq.jux.com/ > > > > > > > > > > > > > > > > > > > > > On Sat, Dec 22, 2012 at 2:09 AM, Mohit Anchlia < > > > > [email protected] > > > > > > >wrote: > > > > > > > > > > > > > >> Could someone help me understand what this really means. Is > this > > > the > > > > > > >> network transfer taking long from client -> server or region > > > server > > > > > > taking > > > > > > >> long time writing to the memory? > > > > > > >> > > > > > > >> 2012-12-21 10:54:21,980 WARN > org.apache.hadoop.ipc.HBaseServer: > > > > > > >> (responseTooSlow): {"processingtimems":135652,"call":"multi( > > > > > > >> org.apache.hadoop.hbase.client.MultiAction@28338472), rpc > > > > version=1, > > > > > > >> client > > > > > > >> version=29, methodsFingerPrint=54742778","client":" > > > 10.18.3.80:48218 > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > ","starttimems":1356115926326,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"} > > > > > > >> 2012-12-21 10:54:21,985 WARN > org.apache.hadoop.ipc.HBaseServer: > > > IPC > > > > > > Server > > > > > > >> handler 26 on 60020 caught: > > > java.nio.channels.ClosedChannelException > > > > > > >> at > > > > > > >> > > > > > > > > > sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133) > > > > > > >> at > > > > > > sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324) > > > > > > >> at > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653) > > > > > > >> at > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924) > > > > > > >> at > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003) > > > > > > >> at > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409) > > > > > > >> at > > > > > > >> > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346) > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
