Hi Locke, Memcached itself only handles byte-arrays, with one exception which is the incr and decr commands. Other than that, the server has no real idea about what you stored. Memcached also allows you to store a byte-flag along with your data so that your client can know what is in the byte-array when it gets it. Not all clients do this, some old ones from before the flag existed use the first byte of the byte-array as their flag. Also, there is no convention around what different flag values should mean. There is also the problem that different languages have different sets of native data types that can be serialized, and there is of course the problem that different languages serialize complex objects completely different, and you can't use a C# object in Java and vice versa.
I completely disagree with you that clients should not serialize data, most people are simply not interested in that part, and most people use only one platform, and providing an efficient standard serializer is therefore a good idea. If you use my client (BeITMemcached) you can store byte-arrays, handle the serialization yourself, and my client won't do anything to the data. It might compress it, but you can switch that behaviour off. When I wrote my client, I looked around what flag values other clients were using, and it's all different, so I made my own version. :-) As a sort of minimum client interoperability, if all clients used the same flag value for byte-arrays, that'd be enough. I'm using flag value 0, I think some other clients do this as well. To solve your immediate problems, take the three clients you have, and modify them to use the same flag value for byte-arrays, and handle all serialization yourself. For BeITMemcached, there's an enum with the flags in Serializer.cs, it's really easy to change. To the rest of the people on this list, don't you think it'd be a good idea to get some standardization going for the flags with the release of the binary protocol? The actively supported clients will make implementation changes for it, and there will be new clients made for languages that have no actively supported client, so there will be a lot of implementation going on, and if we can standardize the flags, we can get everyone on the same page, which would be pretty nice. The minimum thing would be a flag for unmodified byte-arrays. If everyone agrees on a flag value for that, we technically get full client interoperability because users can then build custom serialization on top of that. To make life a bit easier we should have a standard flag value for UTF8-encoded strings, and maybe also define a flag value for string-encoded numbers, the ones that the incr and decr commands work on. Finally, JSON objects for data structures like arrays and associative arrays of strings and numbers. I checked the latest version of the Binary protocol spec at http://code.sixapart.com/svn/memcached/branches/binary/server/doc/protocol-binary.txtand it mentions "Data Type (reserved for future use)". Well. How about we define it now? It's one byte (unlike the current flags which is two bytes), so just reserve the upper half for client-specific values, but define a few common ones in the lower half. Is Aaron Stone reading this list? (He's listed as the author of the spec). Is "Sean" reading it? It says in the spec that he will use it soon. :-) /Henrik Schröder On Fri, Oct 17, 2008 at 10:17 AM, LockeVN <[EMAIL PROTECTED]> wrote: > > Thanks Dustin again, for your working Java ClientAPI ;) . > > Because of our solution has complex parts, several programs in several > programming languages, SO I want they (programs, websites, tools, > bots) talk the same language on the same channel. The common channel > is memcache, and we use native format for values, the common language > is STRING format, only string. All values we use to test are string. > We do the seriallization (simple ini-like, XML or JSON, all encode in > UTF-8) before put it to our MemcacheHelper to write to memcacheD. > > @Dustin: I think you should left the seriallization jobs to whom want > to use memcache. They have several systems, they need 'em talk > together, so they know what the most suitable seriallization way to > implement, NOT YOU. They (the wild wide world who use memcache) > develop program (maybe big program) know how to serialize :D , don't > worry. Thanks again. > > Memcache must operate well, fast, reliable and simple. Memcache is > useful and famous because of that feature. Add more complex > transparent general enterprise blah blah features will make memcache > become something like "Enterprise General Object Storing and Messaging > System" (or some names big-guys at MS, Sun, IBM can introduce). We > love memcache, only. > > But, last but not least: I think all ClientAPI version (C Java .NET, > PHP, Perl, ...) MUST understand (get and set and incr/decr) native > data formats, at least 32bit int and string. I think we need it, and > I'll waiting. > > >
