[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130383#comment-13130383 ] Ted Yu commented on HBASE-3581: --- If I understand the patch correctly, I think the following addendum should be applied: {code} Index: src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java === --- src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java (revision 1185920) +++ src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java (working copy) @@ -42,6 +42,6 @@ } static byte getErrorAndLengthSet() { -return LENGTH_BIT ERROR_BIT; +return LENGTH_BIT | ERROR_BIT; } } {code} hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130413#comment-13130413 ] gaojinchao commented on HBASE-3581: --- Ted's analysis makes sense. Our local test cases always fail, I will apply Ted's patch and run all testcases. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130416#comment-13130416 ] Jonathan Gray commented on HBASE-3581: -- should rename method to getErrorOrLengthSet()? hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130639#comment-13130639 ] ramkrishna.s.vasudevan commented on HBASE-3581: --- {code} + static byte getErrorAndLengthSet() { +return LENGTH_BIT ERROR_BIT; + } {code} This will always make the flag = 0. {code} + static boolean isError(final byte flag) { +return (flag ERROR_BIT) != 0; + } {code} {code} +// Read the flag byte +byte flag = in.readByte(); +boolean isError = ResponseFlag.isError(flag); +if (ResponseFlag.isLength(flag)) { + // Currently length if present is unused. {code} If the flag that we are reading here is the same flag that we have set while writing then the isError() method will return true only because flag = 0 ERROR_BIT = 0x1 So even if we make it {code} +return LENGTH_BIT | ERROR_BIT; } {code} will we have any change? If the flag that we try to read is not the one that we have set then the {code} +return LENGTH_BIT | ERROR_BIT; } {code} is valid. Correct me if am wrong hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130762#comment-13130762 ] stack commented on HBASE-3581: -- Should I open a new issue? hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130761#comment-13130761 ] stack commented on HBASE-3581: -- Thanks lads for digging in on this. Stuff seemed to pass for me (I might have messed up tests though). Ram, looks like I made a mistake in the above if thats the test that I'm doing. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130765#comment-13130765 ] Ted Yu commented on HBASE-3581: --- New issue would be nice. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130887#comment-13130887 ] Hudson commented on HBASE-3581: --- Integrated in HBase-TRUNK #2340 (See [https://builds.apache.org/job/HBase-TRUNK/2340/]) HBASE-4620 I broke the build when I submitted HBASE-3581 (Send length of the rpc response) stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129834#comment-13129834 ] Hudson commented on HBASE-3581: --- Integrated in HBase-TRUNK #2336 (See [https://builds.apache.org/job/HBase-TRUNK/2336/]) HBASE-3581 hbase rpc should send size of response HBASE-3581 hbase rpc should send size of response stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129922#comment-13129922 ] Ted Yu commented on HBASE-3581: --- TestMultiParallel#testFlushCommitsWithAbort hangs on MacBook. I see this in output: {code} 2011-10-18 10:55:39,063 INFO [Thread-333] client.TestMultiParallel(231): Aborted=10.246.204.195,60919,1318960526880 2011-10-18 10:55:39,081 WARN [IPC Client (47) connection to /10.246.204.195:60919 from an unknown user] ipc.HBaseClient$Connection(493): Unexpected exception receiving call responses java.lang.NullPointerException at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:522) at org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:296) at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:575) at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:490) {code} hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129973#comment-13129973 ] Hudson commented on HBASE-3581: --- Integrated in HBase-0.92 #73 (See [https://builds.apache.org/job/HBase-0.92/73/]) HBASE-3581 hbase rpc should send size of response stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130280#comment-13130280 ] Jean-Daniel Cryans commented on HBASE-3581: --- I see the same stack trace trying to start a 0.92 master (I think my env is dirty tho): {quote} 2011-10-19 00:34:48,529 WARN org.apache.hadoop.ipc.HBaseClient: Unexpected exception receiving call responses java.lang.NullPointerException at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:519) at org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:293) at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:575) at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:490) {quote} After that the master can't assign regions. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120689#comment-13120689 ] stack commented on HBASE-3581: -- Tried it on 0.92 branch head and got these failures. Look into it: {code} Results : Failed tests: testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling): New log file should have the default replication Tests in error: testMultiSlaveReplication(org.apache.hadoop.hbase.replication.TestMultiSlaveReplication): test timed out after 30 milliseconds testCyclicReplication(org.apache.hadoop.hbase.replication.TestMasterReplication): test timed out after 30 milliseconds testSimplePutDelete(org.apache.hadoop.hbase.replication.TestMasterReplication): Cluster already running at /home/stack/0.92/target/test-data/18b3c603-4c68-434f-8701-7cfcf558129b testExceptionFromCoprocessorWhenCreatingTable(org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove): test timed out after 3 milliseconds testClusterRestart(org.apache.hadoop.hbase.master.TestRestartCluster): test timed out after 30 milliseconds testBasicRollingRestart(org.apache.hadoop.hbase.master.TestRollingRestart): test timed out after 30 milliseconds testUsingMetaAndBinary(org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore): Cannot lock storage /home/stack/0.92/build/hbase/test/dfs/name1. The directory is already locked. testGetClosestRowBefore3(org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore): Cannot lock storage /home/stack/0.92/build/hbase/test/dfs/name1. The directory is already locked. testGetClosestRowBefore2(org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore): Cannot lock storage /home/stack/0.92/build/hbase/test/dfs/name1. The directory is already locked. testWideScanBatching(org.apache.hadoop.hbase.regionserver.TestWideScanner): Cannot lock storage /home/stack/0.92/build/hbase/test/dfs/name1. The directory is already locked. testBasicHalfMapFile(org.apache.hadoop.hbase.regionserver.TestStoreFile): Call to localhost/127.0.0.1:0 failed on connection exception: java.net.ConnectException: Connection refused testReference(org.apache.hadoop.hbase.regionserver.TestStoreFile): Call to localhost/127.0.0.1:0 failed on connection exception: java.net.ConnectException: Connection refused testBloomFilter(org.apache.hadoop.hbase.regionserver.TestStoreFile): Call to localhost/127.0.0.1:0 failed on connection exception: java.net.ConnectException: Connection refused testBloomTypes(org.apache.hadoop.hbase.regionserver.TestStoreFile): Call to localhost/127.0.0.1:0 failed on connection exception: java.net.ConnectException: Connection refused testBloomEdgeCases(org.apache.hadoop.hbase.regionserver.TestStoreFile): Call to localhost/127.0.0.1:0 failed on connection exception: java.net.ConnectException: Connection refused testFlushTimeComparator(org.apache.hadoop.hbase.regionserver.TestStoreFile): Call to localhost/127.0.0.1:0 failed on connection exception: java.net.ConnectException: Connection refused testMROnTableWithCustomMapper(org.apache.hadoop.hbase.mapreduce.TestImportTsv): java.io.IOException: File /tmp/hadoop-stack/mapred/system/job_local_0002/libjars/zookeeper-3.3.3.jar could only be replicated to 0 nodes, instead of 1 testSimpleLoad(org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles): No such file or directory testReconstruction(org.apache.hadoop.hbase.TestFullLogReconstruction): test timed out after 30 milliseconds testFlushCommitsWithAbort(org.apache.hadoop.hbase.client.TestMultiParallel): test timed out after 30 milliseconds testFlushCommitsNoAbort(org.apache.hadoop.hbase.client.TestMultiParallel): test timed out after 30 milliseconds test2481(org.apache.hadoop.hbase.client.TestScannerTimeout): test timed out after 30 milliseconds test2772(org.apache.hadoop.hbase.client.TestScannerTimeout): test timed out after 30 milliseconds Tests run: 897, Failures: 1, Errors: 20, Skipped: 18 {code} hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105894#comment-13105894 ] Benoit Sigoure commented on HBASE-3581: --- +1, thank you Stack. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13106962#comment-13106962 ] stack commented on HBASE-3581: -- Hmmm... tests hang. Need to figure whats up here. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13072916#comment-13072916 ] stack commented on HBASE-3581: -- ugh we should do this for tsdb and asynchbase I'm going to pull it in again (sorry Ted)... we can punt later if it don't make cut. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Priority: Critical Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13072951#comment-13072951 ] ryan rawson commented on HBASE-3581: Lets just commit the patch as is. Optimizing the deployment complexities of users isn't our job, and at the same time it's preventing a extremely useful patch from going in. Take the flag day, and in the future everything else will be easier. (until there is another flag day, but you want a job dont you?) hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Priority: Critical Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13005994#comment-13005994 ] stack commented on HBASE-3581: -- Yes. We should do BenoƮt's first suggestion. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003104#comment-13003104 ] Benoit Sigoure commented on HBASE-3581: --- Yeah I also like the flag suggestion better. It's easy to implement both in the server and in the client. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002228#comment-13002228 ] Jean-Daniel Cryans commented on HBASE-3581: --- +1 hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002281#comment-13002281 ] Benoit Sigoure commented on HBASE-3581: --- -1: Can we *please* find a way of making this change such that clients can still support the old protocol and the new one at the same time? For instance, after sending the hello header (hrpc3 + creds) maybe the server could reply (right now it doesn't) and say hey, here's my version info. Because you can't use the {{getProtocolVersion}} RPC to find about this incompatibility. Other possibility: swap the errorFlag and the size, and use an extra bit in the error flag, not to indicate an error but to indicate that the size follows. If we do this, we should refer to errorFlag as just flag since it's not going to be only about errors anymore. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002283#comment-13002283 ] Benoit Sigoure commented on HBASE-3581: --- I just wanna say I'm very strongly in favor of this change, but I just want it to be done in such a way that clients can determine what variation of the protocol the server speaks, so they can adjust themselves. This way a single {{.jar}} can talk to different HBase versions with no code change and no configuration setting. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002471#comment-13002471 ] stack commented on HBASE-3581: -- Hmmm... We should do your first suggestion too. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira