Use base64 or similar encodings maybe? Fi.
On Sat, Feb 27, 2010 at 6:02 PM, Jonathan Ellis <jbel...@gmail.com> wrote: > Keys are strings. That means they have to be UTF8-encoded, although > thrift bindings for many languages (including python) don't help you > with this. > > On Sat, Feb 27, 2010 at 7:43 PM, Robert Edmonds <edmo...@debian.org> > wrote: > > hi, > > > > i'm using cassandra 0.5.0 and pycassa 0.1. > > > > i'd like to store binary data (specifically, wire-format DNS names) in > > key names. is this possible with cassandra? > > > > i have a keyspace set up like this: > > > > <Keyspace Name="DNS"> > > <KeysCachedFraction>0.01</KeysCachedFraction> > > <ColumnFamily CompareWith="BytesType" > > CompareSubcolumnsWith="BytesType" > > ColumnType="Super" > > Name="RRsets"/> > > </Keyspace> > > > > when i insert a key with a printable ASCII name, everything works fine, > > but when i insert a key with a name containing binary data the insertion > > appears to succeed, but a subsequent get for the same key name fails. > > > > here's a test case: > > > > import pycassa > > conn = pycassa.connect() > > cf = pycassa.ColumnFamily(conn, 'DNS', 'RRsets', super=True) > > > > cf.insert('\x03www\x07example\x03com\x00', {'foo': {'bar' : 'baz'}}) > > cf.get('\x03www\x07example\x03com\x00') # <-- this fails > > > > cf.insert('www.example.com.', {'foo': {'bar' : 'baz'}}) > > cf.get('www.example.com.') # <-- this succeeds > > > > here's the same code being executed in ipython, with each python line > > interleaved with the hex dump of the TCP conversation. (the client > request hex > > dump is flush, the server response hex dump is indented.) > > > > (i don't know how the cassandra thrift protocol works, but the exact same > > response was sent on the wire for the successful and the unsuccessful > > insertions, so i would assume that either the insertion succeeded or the > > protocol doesn't indicate insertion success?) > > > > In [1]: import pycassa > > > > In [2]: conn = pycassa.connect() > > > > In [3]: cf = pycassa.ColumnFamily(conn, 'DNS', 'RRsets', super=True) > > > > > ---------------------------------------------------------------------------- > > > > In [4]: cf.insert('\x03www\x07example\x03com\x00', {'foo': {'bar' : > 'baz'}}) > > Out[4]: 1267319528 > > > > 00000000 80 01 00 01 00 00 00 0c 62 61 74 63 68 5f 69 6e ........ > batch_in > > 00000010 73 65 72 74 00 00 00 00 0b 00 01 00 00 00 03 44 sert.... > .......D > > 00000020 4e 53 0b 00 02 00 00 00 11 03 77 77 77 07 65 78 NS...... > ..www.ex > > 00000030 61 6d 70 6c 65 03 63 6f 6d 00 0d 00 03 0b 0f 00 ample.com....... > > 00000040 00 00 01 00 00 00 06 52 52 73 65 74 73 0c 00 00 .......R > Rsets... > > 00000050 00 01 0c 00 02 0b 00 01 00 00 00 03 66 6f 6f 0f ........ > ....foo. > > 00000060 00 02 0c 00 00 00 01 0b 00 01 00 00 00 03 62 61 ........ > ......ba > > 00000070 72 0b 00 02 00 00 00 03 62 61 7a 0a 00 03 00 00 r....... > baz..... > > 00000080 00 00 4b 89 c2 e8 00 00 00 08 00 04 00 00 00 00 ..K..... > ........ > > 00000090 00 . > > > > 00000000 80 01 00 02 00 00 00 0c 62 61 74 63 68 5f 69 6e ........ > batch_in > > 00000010 73 65 72 74 00 00 00 00 00 sert.... . > > > > > ---------------------------------------------------------------------------- > > > > In [5]: cf.get('\x03www\x07example\x03com\x00') > > NotFoundException: NotFoundException() > > > > 00000091 80 01 00 01 00 00 00 09 67 65 74 5f 73 6c 69 63 ........ > get_slic > > 000000A1 65 00 00 00 00 0b 00 01 00 00 00 03 44 4e 53 0b e....... > ....DNS. > > 000000B1 00 02 00 00 00 11 03 77 77 77 07 65 78 61 6d 70 .......w > ww.examp > > 000000C1 6c 65 03 63 6f 6d 00 0c 00 03 0b 00 03 00 00 00 le.com.. > ........ > > 000000D1 06 52 52 73 65 74 73 00 0c 00 04 0c 00 02 0b 00 .RRsets. > ........ > > 000000E1 01 00 00 00 00 0b 00 02 00 00 00 00 02 00 03 00 ........ > ........ > > 000000F1 08 00 04 00 00 00 64 00 00 08 00 05 00 00 00 01 ......d. > ........ > > 00000101 00 . > > > > 00000019 80 01 00 02 00 00 00 09 67 65 74 5f 73 6c 69 63 ........ > get_slic > > 00000029 65 00 00 00 00 0f 00 00 0c 00 00 00 00 00 e....... > ...... > > > > > ---------------------------------------------------------------------------- > > > > In [6]: cf.insert('www.example.com.', {'foo': {'bar' : 'baz'}}) > > Out[6]: 1267319533 > > > > 00000102 80 01 00 01 00 00 00 0c 62 61 74 63 68 5f 69 6e ........ > batch_in > > 00000112 73 65 72 74 00 00 00 00 0b 00 01 00 00 00 03 44 sert.... > .......D > > 00000122 4e 53 0b 00 02 00 00 00 10 77 77 77 2e 65 78 61 NS...... > .www.exa > > 00000132 6d 70 6c 65 2e 63 6f 6d 2e 0d 00 03 0b 0f 00 00 mple.com........ > > 00000142 00 01 00 00 00 06 52 52 73 65 74 73 0c 00 00 00 ......RR > sets.... > > 00000152 01 0c 00 02 0b 00 01 00 00 00 03 66 6f 6f 0f 00 ........ > ...foo.. > > 00000162 02 0c 00 00 00 01 0b 00 01 00 00 00 03 62 61 72 ........ > .....bar > > 00000172 0b 00 02 00 00 00 03 62 61 7a 0a 00 03 00 00 00 .......b > az...... > > 00000182 00 4b 89 c2 ed 00 00 00 08 00 04 00 00 00 00 00 .K...... > ........ > > > > 00000037 80 01 00 02 00 00 00 0c 62 61 74 63 68 5f 69 6e ........ > batch_in > > 00000047 73 65 72 74 00 00 00 00 00 sert.... . > > > > > ---------------------------------------------------------------------------- > > > > In [7]: cf.get('www.example.com.') > > Out[7]: {'foo': {'bar': 'baz'}} > > > > 00000192 80 01 00 01 00 00 00 09 67 65 74 5f 73 6c 69 63 ........ > get_slic > > 000001A2 65 00 00 00 00 0b 00 01 00 00 00 03 44 4e 53 0b e....... > ....DNS. > > 000001B2 00 02 00 00 00 10 77 77 77 2e 65 78 61 6d 70 6c ......ww > w.exampl > > 000001C2 65 2e 63 6f 6d 2e 0c 00 03 0b 00 03 00 00 00 06 e.com... > ........ > > 000001D2 52 52 73 65 74 73 00 0c 00 04 0c 00 02 0b 00 01 RRsets.. > ........ > > 000001E2 00 00 00 00 0b 00 02 00 00 00 00 02 00 03 00 08 ........ > ........ > > 000001F2 00 04 00 00 00 64 00 00 08 00 05 00 00 00 01 00 .....d.. > ........ > > > > 00000050 80 01 00 02 00 00 00 09 67 65 74 5f 73 6c 69 63 ........ > get_slic > > 00000060 65 00 00 00 00 0f 00 00 0c 00 00 00 01 0c 00 02 e....... > ........ > > 00000070 0b 00 01 00 00 00 03 66 6f 6f 0f 00 02 0c 00 00 .......f > oo...... > > 00000080 00 01 0b 00 01 00 00 00 03 62 61 72 0b 00 02 00 ........ > .bar.... > > 00000090 00 00 03 62 61 7a 0a 00 03 00 00 00 00 4b 89 c2 ...baz.. > .....K.. > > 000000A0 ed 00 00 00 00 > > > > > ---------------------------------------------------------------------------- > > > > -- > > Robert Edmonds > > edmo...@debian.org > > > > >