[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[
https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130383#comment-13130383
]
Ted Yu commented on HBASE-3581:
---
If I understand the patch correctly, I think the following addendum should be
applied:
{code}
Index: src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java
===
--- src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java (revision
1185920)
+++ src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java (working copy)
@@ -42,6 +42,6 @@
}
static byte getErrorAndLengthSet() {
-return LENGTH_BIT ERROR_BIT;
+return LENGTH_BIT | ERROR_BIT;
}
}
{code}
hbase rpc should send size of response
--
Key: HBASE-3581
URL: https://issues.apache.org/jira/browse/HBASE-3581
Project: HBase
Issue Type: Improvement
Reporter: ryan rawson
Assignee: stack
Priority: Critical
Fix For: 0.92.0
Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt,
HBASE-rpc-response.txt
The RPC reply from Server-Client does not include the size of the payload,
it is framed like so:
i32 callId
byte errorFlag
byte[] data
The data segment would contain enough info about how big the response is so
that it could be decoded by a writable reader.
This makes it difficult to write buffering clients, who might read the entire
'data' then pass it to a decoder. While less memory efficient, if you want to
easily write block read clients (eg: nio) it would be necessary to send the
size along so that the client could snarf into a local buf.
The new proposal is:
i32 callId
i32 size
byte errorFlag
byte[] data
the size being sizeof(data) + sizeof(errorFlag).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130413#comment-13130413 ] gaojinchao commented on HBASE-3581: --- Ted's analysis makes sense. Our local test cases always fail, I will apply Ted's patch and run all testcases. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130416#comment-13130416 ] Jonathan Gray commented on HBASE-3581: -- should rename method to getErrorOrLengthSet()? hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[
https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130639#comment-13130639
]
ramkrishna.s.vasudevan commented on HBASE-3581:
---
{code}
+ static byte getErrorAndLengthSet() {
+return LENGTH_BIT ERROR_BIT;
+ }
{code}
This will always make the flag = 0.
{code}
+ static boolean isError(final byte flag) {
+return (flag ERROR_BIT) != 0;
+ }
{code}
{code}
+// Read the flag byte
+byte flag = in.readByte();
+boolean isError = ResponseFlag.isError(flag);
+if (ResponseFlag.isLength(flag)) {
+ // Currently length if present is unused.
{code}
If the flag that we are reading here is the same flag that we have set while
writing then
the isError() method will return true only because flag = 0
ERROR_BIT = 0x1
So even if we make it
{code}
+return LENGTH_BIT | ERROR_BIT;
}
{code}
will we have any change?
If the flag that we try to read is not the one that we have set then the
{code}
+return LENGTH_BIT | ERROR_BIT;
}
{code}
is valid. Correct me if am wrong
hbase rpc should send size of response
--
Key: HBASE-3581
URL: https://issues.apache.org/jira/browse/HBASE-3581
Project: HBase
Issue Type: Improvement
Reporter: ryan rawson
Assignee: stack
Priority: Critical
Fix For: 0.92.0
Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt,
HBASE-rpc-response.txt
The RPC reply from Server-Client does not include the size of the payload,
it is framed like so:
i32 callId
byte errorFlag
byte[] data
The data segment would contain enough info about how big the response is so
that it could be decoded by a writable reader.
This makes it difficult to write buffering clients, who might read the entire
'data' then pass it to a decoder. While less memory efficient, if you want to
easily write block read clients (eg: nio) it would be necessary to send the
size along so that the client could snarf into a local buf.
The new proposal is:
i32 callId
i32 size
byte errorFlag
byte[] data
the size being sizeof(data) + sizeof(errorFlag).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130762#comment-13130762 ] stack commented on HBASE-3581: -- Should I open a new issue? hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130761#comment-13130761 ] stack commented on HBASE-3581: -- Thanks lads for digging in on this. Stuff seemed to pass for me (I might have messed up tests though). Ram, looks like I made a mistake in the above if thats the test that I'm doing. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130765#comment-13130765 ] Ted Yu commented on HBASE-3581: --- New issue would be nice. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130887#comment-13130887 ] Hudson commented on HBASE-3581: --- Integrated in HBase-TRUNK #2340 (See [https://builds.apache.org/job/HBase-TRUNK/2340/]) HBASE-4620 I broke the build when I submitted HBASE-3581 (Send length of the rpc response) stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129834#comment-13129834 ] Hudson commented on HBASE-3581: --- Integrated in HBase-TRUNK #2336 (See [https://builds.apache.org/job/HBase-TRUNK/2336/]) HBASE-3581 hbase rpc should send size of response HBASE-3581 hbase rpc should send size of response stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[
https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129922#comment-13129922
]
Ted Yu commented on HBASE-3581:
---
TestMultiParallel#testFlushCommitsWithAbort hangs on MacBook.
I see this in output:
{code}
2011-10-18 10:55:39,063 INFO [Thread-333] client.TestMultiParallel(231):
Aborted=10.246.204.195,60919,1318960526880
2011-10-18 10:55:39,081 WARN [IPC Client (47) connection to
/10.246.204.195:60919 from an unknown user] ipc.HBaseClient$Connection(493):
Unexpected exception receiving call responses
java.lang.NullPointerException
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:522)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:296)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:575)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:490)
{code}
hbase rpc should send size of response
--
Key: HBASE-3581
URL: https://issues.apache.org/jira/browse/HBASE-3581
Project: HBase
Issue Type: Improvement
Reporter: ryan rawson
Assignee: stack
Priority: Critical
Fix For: 0.92.0
Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt,
HBASE-rpc-response.txt
The RPC reply from Server-Client does not include the size of the payload,
it is framed like so:
i32 callId
byte errorFlag
byte[] data
The data segment would contain enough info about how big the response is so
that it could be decoded by a writable reader.
This makes it difficult to write buffering clients, who might read the entire
'data' then pass it to a decoder. While less memory efficient, if you want to
easily write block read clients (eg: nio) it would be necessary to send the
size along so that the client could snarf into a local buf.
The new proposal is:
i32 callId
i32 size
byte errorFlag
byte[] data
the size being sizeof(data) + sizeof(errorFlag).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129973#comment-13129973 ] Hudson commented on HBASE-3581: --- Integrated in HBase-0.92 #73 (See [https://builds.apache.org/job/HBase-0.92/73/]) HBASE-3581 hbase rpc should send size of response stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/ResponseFlag.java hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[
https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130280#comment-13130280
]
Jean-Daniel Cryans commented on HBASE-3581:
---
I see the same stack trace trying to start a 0.92 master (I think my env is
dirty tho):
{quote}
2011-10-19 00:34:48,529 WARN org.apache.hadoop.ipc.HBaseClient: Unexpected
exception receiving call responses
java.lang.NullPointerException
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:519)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:293)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:575)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:490)
{quote}
After that the master can't assign regions.
hbase rpc should send size of response
--
Key: HBASE-3581
URL: https://issues.apache.org/jira/browse/HBASE-3581
Project: HBase
Issue Type: Improvement
Reporter: ryan rawson
Assignee: stack
Priority: Critical
Fix For: 0.92.0
Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt,
HBASE-rpc-response.txt
The RPC reply from Server-Client does not include the size of the payload,
it is framed like so:
i32 callId
byte errorFlag
byte[] data
The data segment would contain enough info about how big the response is so
that it could be decoded by a writable reader.
This makes it difficult to write buffering clients, who might read the entire
'data' then pass it to a decoder. While less memory efficient, if you want to
easily write block read clients (eg: nio) it would be necessary to send the
size along so that the client could snarf into a local buf.
The new proposal is:
i32 callId
i32 size
byte errorFlag
byte[] data
the size being sizeof(data) + sizeof(errorFlag).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[
https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13120689#comment-13120689
]
stack commented on HBASE-3581:
--
Tried it on 0.92 branch head and got these failures. Look into it:
{code}
Results :
Failed tests:
testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling):
New log file should have the default replication
Tests in error:
testMultiSlaveReplication(org.apache.hadoop.hbase.replication.TestMultiSlaveReplication):
test timed out after 30 milliseconds
testCyclicReplication(org.apache.hadoop.hbase.replication.TestMasterReplication):
test timed out after 30 milliseconds
testSimplePutDelete(org.apache.hadoop.hbase.replication.TestMasterReplication):
Cluster already running at
/home/stack/0.92/target/test-data/18b3c603-4c68-434f-8701-7cfcf558129b
testExceptionFromCoprocessorWhenCreatingTable(org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove):
test timed out after 3 milliseconds
testClusterRestart(org.apache.hadoop.hbase.master.TestRestartCluster): test
timed out after 30 milliseconds
testBasicRollingRestart(org.apache.hadoop.hbase.master.TestRollingRestart):
test timed out after 30 milliseconds
testUsingMetaAndBinary(org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore):
Cannot lock storage /home/stack/0.92/build/hbase/test/dfs/name1. The directory
is already locked.
testGetClosestRowBefore3(org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore):
Cannot lock storage /home/stack/0.92/build/hbase/test/dfs/name1. The directory
is already locked.
testGetClosestRowBefore2(org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore):
Cannot lock storage /home/stack/0.92/build/hbase/test/dfs/name1. The directory
is already locked.
testWideScanBatching(org.apache.hadoop.hbase.regionserver.TestWideScanner):
Cannot lock storage /home/stack/0.92/build/hbase/test/dfs/name1. The directory
is already locked.
testBasicHalfMapFile(org.apache.hadoop.hbase.regionserver.TestStoreFile):
Call to localhost/127.0.0.1:0 failed on connection exception:
java.net.ConnectException: Connection refused
testReference(org.apache.hadoop.hbase.regionserver.TestStoreFile): Call to
localhost/127.0.0.1:0 failed on connection exception:
java.net.ConnectException: Connection refused
testBloomFilter(org.apache.hadoop.hbase.regionserver.TestStoreFile): Call to
localhost/127.0.0.1:0 failed on connection exception:
java.net.ConnectException: Connection refused
testBloomTypes(org.apache.hadoop.hbase.regionserver.TestStoreFile): Call to
localhost/127.0.0.1:0 failed on connection exception:
java.net.ConnectException: Connection refused
testBloomEdgeCases(org.apache.hadoop.hbase.regionserver.TestStoreFile): Call
to localhost/127.0.0.1:0 failed on connection exception:
java.net.ConnectException: Connection refused
testFlushTimeComparator(org.apache.hadoop.hbase.regionserver.TestStoreFile):
Call to localhost/127.0.0.1:0 failed on connection exception:
java.net.ConnectException: Connection refused
testMROnTableWithCustomMapper(org.apache.hadoop.hbase.mapreduce.TestImportTsv):
java.io.IOException: File
/tmp/hadoop-stack/mapred/system/job_local_0002/libjars/zookeeper-3.3.3.jar
could only be replicated to 0 nodes, instead of 1
testSimpleLoad(org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles):
No such file or directory
testReconstruction(org.apache.hadoop.hbase.TestFullLogReconstruction): test
timed out after 30 milliseconds
testFlushCommitsWithAbort(org.apache.hadoop.hbase.client.TestMultiParallel):
test timed out after 30 milliseconds
testFlushCommitsNoAbort(org.apache.hadoop.hbase.client.TestMultiParallel):
test timed out after 30 milliseconds
test2481(org.apache.hadoop.hbase.client.TestScannerTimeout): test timed out
after 30 milliseconds
test2772(org.apache.hadoop.hbase.client.TestScannerTimeout): test timed out
after 30 milliseconds
Tests run: 897, Failures: 1, Errors: 20, Skipped: 18
{code}
hbase rpc should send size of response
--
Key: HBASE-3581
URL: https://issues.apache.org/jira/browse/HBASE-3581
Project: HBase
Issue Type: Improvement
Reporter: ryan rawson
Assignee: stack
Priority: Critical
Fix For: 0.92.0
Attachments: 3581-v2.txt, 3581-v3.txt, 3581-v4.txt,
HBASE-rpc-response.txt
The RPC reply from Server-Client does not include the size of the payload,
it is framed like so:
i32 callId
byte errorFlag
byte[] data
The data segment would contain enough info about how big the response is so
that it could be decoded by a writable reader.
This makes it difficult to write buffering clients, who might read the entire
'data' then
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105894#comment-13105894 ] Benoit Sigoure commented on HBASE-3581: --- +1, thank you Stack. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13106962#comment-13106962 ] stack commented on HBASE-3581: -- Hmmm... tests hang. Need to figure whats up here. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: stack Priority: Critical Fix For: 0.92.0 Attachments: 3581-v2.txt, HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13072916#comment-13072916 ] stack commented on HBASE-3581: -- ugh we should do this for tsdb and asynchbase I'm going to pull it in again (sorry Ted)... we can punt later if it don't make cut. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Priority: Critical Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13072951#comment-13072951 ] ryan rawson commented on HBASE-3581: Lets just commit the patch as is. Optimizing the deployment complexities of users isn't our job, and at the same time it's preventing a extremely useful patch from going in. Take the flag day, and in the future everything else will be easier. (until there is another flag day, but you want a job dont you?) hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Priority: Critical Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13005994#comment-13005994 ] stack commented on HBASE-3581: -- Yes. We should do Benoît's first suggestion. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003104#comment-13003104 ] Benoit Sigoure commented on HBASE-3581: --- Yeah I also like the flag suggestion better. It's easy to implement both in the server and in the client. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002228#comment-13002228 ] Jean-Daniel Cryans commented on HBASE-3581: --- +1 hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[
https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002281#comment-13002281
]
Benoit Sigoure commented on HBASE-3581:
---
-1: Can we *please* find a way of making this change such that clients can
still support the old protocol and the new one at the same time?
For instance, after sending the hello header (hrpc3 + creds) maybe the
server could reply (right now it doesn't) and say hey, here's my version
info. Because you can't use the {{getProtocolVersion}} RPC to find about this
incompatibility.
Other possibility: swap the errorFlag and the size, and use an extra bit in the
error flag, not to indicate an error but to indicate that the size follows. If
we do this, we should refer to errorFlag as just flag since it's not going
to be only about errors anymore.
hbase rpc should send size of response
--
Key: HBASE-3581
URL: https://issues.apache.org/jira/browse/HBASE-3581
Project: HBase
Issue Type: Improvement
Reporter: ryan rawson
Assignee: ryan rawson
Fix For: 0.92.0
Attachments: HBASE-rpc-response.txt
The RPC reply from Server-Client does not include the size of the payload,
it is framed like so:
i32 callId
byte errorFlag
byte[] data
The data segment would contain enough info about how big the response is so
that it could be decoded by a writable reader.
This makes it difficult to write buffering clients, who might read the entire
'data' then pass it to a decoder. While less memory efficient, if you want to
easily write block read clients (eg: nio) it would be necessary to send the
size along so that the client could snarf into a local buf.
The new proposal is:
i32 callId
i32 size
byte errorFlag
byte[] data
the size being sizeof(data) + sizeof(errorFlag).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[
https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002283#comment-13002283
]
Benoit Sigoure commented on HBASE-3581:
---
I just wanna say I'm very strongly in favor of this change, but I just want it
to be done in such a way that clients can determine what variation of the
protocol the server speaks, so they can adjust themselves. This way a single
{{.jar}} can talk to different HBase versions with no code change and no
configuration setting.
hbase rpc should send size of response
--
Key: HBASE-3581
URL: https://issues.apache.org/jira/browse/HBASE-3581
Project: HBase
Issue Type: Improvement
Reporter: ryan rawson
Assignee: ryan rawson
Fix For: 0.92.0
Attachments: HBASE-rpc-response.txt
The RPC reply from Server-Client does not include the size of the payload,
it is framed like so:
i32 callId
byte errorFlag
byte[] data
The data segment would contain enough info about how big the response is so
that it could be decoded by a writable reader.
This makes it difficult to write buffering clients, who might read the entire
'data' then pass it to a decoder. While less memory efficient, if you want to
easily write block read clients (eg: nio) it would be necessary to send the
size along so that the client could snarf into a local buf.
The new proposal is:
i32 callId
i32 size
byte errorFlag
byte[] data
the size being sizeof(data) + sizeof(errorFlag).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3581) hbase rpc should send size of response
[ https://issues.apache.org/jira/browse/HBASE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002471#comment-13002471 ] stack commented on HBASE-3581: -- Hmmm... We should do your first suggestion too. hbase rpc should send size of response -- Key: HBASE-3581 URL: https://issues.apache.org/jira/browse/HBASE-3581 Project: HBase Issue Type: Improvement Reporter: ryan rawson Assignee: ryan rawson Fix For: 0.92.0 Attachments: HBASE-rpc-response.txt The RPC reply from Server-Client does not include the size of the payload, it is framed like so: i32 callId byte errorFlag byte[] data The data segment would contain enough info about how big the response is so that it could be decoded by a writable reader. This makes it difficult to write buffering clients, who might read the entire 'data' then pass it to a decoder. While less memory efficient, if you want to easily write block read clients (eg: nio) it would be necessary to send the size along so that the client could snarf into a local buf. The new proposal is: i32 callId i32 size byte errorFlag byte[] data the size being sizeof(data) + sizeof(errorFlag). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
