Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-08 Thread Benjamin Stiglitz
When doing force_encoding, convert to a ByteString in the old encoding, then try to convert to an NSString in the new encoding. If we succeed, great. If not, leave as a tagged ByteString (and probably whine about it). That's actually wrong. All force_encoding does is change the encoding a

Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-08 Thread Charles Oliver Nutter
Vincent Isambart wrote: the test_string tests in Ruby 1.9 repository do seem to mostly function You mean test/ruby/test_m17n.rb, test/ruby/test_m17n_comb.rb, test/ruby/test_io_m17n.rb and test/ruby/enc/test_*.rb? test/ruby/test_string.rb does not contain anything m17n related. We simply could

Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-08 Thread Vincent Isambart
> the test_string tests in Ruby 1.9 repository do seem to mostly function You mean test/ruby/test_m17n.rb, test/ruby/test_m17n_comb.rb, test/ruby/test_io_m17n.rb and test/ruby/enc/test_*.rb? test/ruby/test_string.rb does not contain anything m17n related. > We simply could not match Ruby regex exa

Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-08 Thread Charles Oliver Nutter
Vincent Isambart wrote: I think everyone agrees that having a Ruby 1.9 String specs will be necessity. And we'll also need to decide what parts of it to follow and what parts we do not need to. For example handling access to characters in a string with a partly invalid encoding exactly the same w

Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-08 Thread Vincent Isambart
> That's actually wrong. All force_encoding does is change the encoding > attribute of the string, it shouldn't change the internal encoding of the > bytes. The encoding attribute is basically a switch to describe which set of > string methods should be used on the bytes. That's what force_encodin

Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-08 Thread Manfred Stienstra
On Apr 8, 2009, at 7:23 AM, Benjamin Stiglitz wrote: When doing force_encoding, convert to a ByteString in the old encoding, then try to convert to an NSString in the new encoding. If we succeed, great. If not, leave as a tagged ByteString (and probably whine about it). That's actually w

Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-07 Thread Benjamin Stiglitz
So plan B: We emulate Ruby 1.9 strings behavior on top of of NSString/NSData. I'm really interested in this discussion too. A little background for JRuby: Thanks for the background, Charlie. This sort of history is very instructive. * Java's strings are all UTF-16. In order to represent

Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-07 Thread Charles Oliver Nutter
Vincent Isambart wrote: Hi again, So plan B: We emulate Ruby 1.9 strings behavior on top of of NSString/NSData. I'm really interested in this discussion too. A little background for JRuby: We started out (or really, the original authors started out) with JRuby using all Java strings and stri

Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-07 Thread Vincent Isambart
Hi again, So plan B: We emulate Ruby 1.9 strings behavior on top of of NSString/NSData. Internally we would use a NSData when the encoding is not valid or it's binary, and NSString in the other cases (never both at the same time). We would have a Ruby encoding that may be completely different fro

Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-06 Thread Vincent Isambart
> Finally; I don't think it's a good idea to discuss this a great length > without actual code but in order to write a compatible implementation most > (if not all) of the String awkwardness will have to be implemented. Thank you very much for the remarks. We will indeed need to go to actual code

Re: [MacRuby-devel] Strings, Encodings and IO

2009-04-06 Thread Manfred Stienstra
On Apr 7, 2009, at 7:47 AM, Vincent Isambart wrote: I have two small comments and a general statement about your essay; A few functions of 1.9 may also be disabled (like force_encoding). Of course it would be possible to add the full functionality of Ruby 1.9 strings on ByteString but it would

[MacRuby-devel] Strings, Encodings and IO

2009-04-06 Thread Vincent Isambart
Hi, MacRuby is getting better and better at great speed, but there is one point where MacRuby still has much to do: strings, encodings and IO. If you have some interest in that, but do not know Ruby 1.9's strings well, I recommend you check at least the 2 last posts of James Gray II's series abou