Re: Policy for exposing range structs

2016-04-20 Thread Johan Engelen via Digitalmars-d
On Tuesday, 19 April 2016 at 17:59:48 UTC, Steven Schveighoffer wrote: On 4/19/16 10:44 AM, Johan Engelen wrote: The feature is experimental, and has been tested on Weka.io's codebase. Compilation with -hashthres=1000 results in a binary that is half the size of the original (201MB vs.

Re: Policy for exposing range structs

2016-04-19 Thread Steven Schveighoffer via Digitalmars-d
On 4/19/16 4:50 PM, Marco Leise wrote: What about using old string compression algorithms on the mangled name? That should effectively get rid of all the repeated words. It's somewhat recursive, so I don't know how well string compression will work. It's possible, definitely. But I think a

Re: Policy for exposing range structs

2016-04-19 Thread Marco Leise via Digitalmars-d
What about using old string compression algorithms on the mangled name? That should effectively get rid of all the repeated words. -- Marco

Re: Policy for exposing range structs

2016-04-19 Thread Steven Schveighoffer via Digitalmars-d
On 4/19/16 10:44 AM, Johan Engelen wrote: The feature is experimental, and has been tested on Weka.io's codebase. Compilation with -hashthres=1000 results in a binary that is half the size of the original (201MB vs. 461MB). I did not observe a significant difference in total build times. I'd

Re: Policy for exposing range structs

2016-04-19 Thread Jack Stouffer via Digitalmars-d
On Tuesday, 19 April 2016 at 14:44:12 UTC, Johan Engelen wrote: What is very unfortunate is that std.traits depends on the mangled name, doing string parsing of the mangled name of symbols to obtain symbol traits. I'm surprised that's how it works. I just assumed there was some __traits call

Re: Policy for exposing range structs

2016-04-19 Thread Johan Engelen via Digitalmars-d
On Friday, 1 April 2016 at 14:46:42 UTC, Johan Engelen wrote: Meanwhile, I've implemented hashing of function names and other symbols *for the backend*, giving an object file size reduction of ~25% (hashing everything larger than 100 chars) for my current testcase (251MB -> 189MB). Hashing

Re: Policy for exposing range structs

2016-04-01 Thread Johan Engelen via Digitalmars-d
On Friday, 1 April 2016 at 11:00:29 UTC, Johan Engelen wrote: Unfortunately, and perhaps expectedly, it did not give a large size reduction / speed boost. Not very statistically sound results though ;) The times I measured are not including linking (!). But I think the small difference in

Re: Policy for exposing range structs

2016-04-01 Thread Johan Engelen via Digitalmars-d
On Friday, 25 March 2016 at 19:04:46 UTC, Anon wrote: Two changes to the mangling: 1) `LName`s of length 0 (which currently cannot exist) mean to repeat the previous `LName` of the current symbol. 2) N `Number` is added as a valid `Type`, meaning "Type Back Reference". Basically, all

Re: Policy for exposing range structs

2016-03-31 Thread Jonathan M Davis via Digitalmars-d
On Thursday, March 31, 2016 09:10:49 Steven Schveighoffer via Digitalmars-d wrote: > On 3/30/16 3:19 PM, Liran Zvibel wrote: > > On Sunday, 27 March 2016 at 17:01:39 UTC, David Nadlinger wrote: > >> Compression in the usual sense won't help. Sure, it might reduce the > >> object file size, but

Re: Policy for exposing range structs

2016-03-31 Thread Anon via Digitalmars-d
On Thursday, 31 March 2016 at 20:40:03 UTC, Adam D. Ruppe wrote: meh, if it is allowed, it is going to happen. Why make it worse when there's so little cost in making it better? Define "little cost". Whatever compression algorithm chosen will need support added to any/all tools that want to

Re: Policy for exposing range structs

2016-03-31 Thread Adam D. Ruppe via Digitalmars-d
On Thursday, 31 March 2016 at 20:12:34 UTC, Anon wrote: Having thought about it a bit more, I am now of the opinion that super-long strings have no business being in template args, so we shouldn't cater to them. meh, if it is allowed, it is going to happen. Why make it worse when there's so

Re: Policy for exposing range structs

2016-03-31 Thread Anon via Digitalmars-d
On Thursday, 31 March 2016 at 17:52:43 UTC, Adam D. Ruppe wrote: Yeah, but my thought is the typical use case isn't actually the problem - it is OK as it is. Longer strings are where it gets concerning to me. Doubling the size of UTF-8 (the effect of the current base16 encoding) bothers me

Re: Policy for exposing range structs

2016-03-31 Thread Adam D. Ruppe via Digitalmars-d
On Thursday, 31 March 2016 at 17:30:44 UTC, Anon wrote: My encoding is shorter in the typical use case Yeah, but my thought is the typical use case isn't actually the problem - it is OK as it is. Longer strings are where it gets concerning to me. Would a hybrid approach (my encoding,

Re: Policy for exposing range structs

2016-03-31 Thread Anon via Digitalmars-d
On Thursday, 31 March 2016 at 16:46:42 UTC, Adam D. Ruppe wrote: On Thursday, 31 March 2016 at 16:38:59 UTC, Anon wrote: I've been spending my D time thinking about potential changes to how template string value parameters are encoded. How does it compare to simply gzipping the string and

Re: Policy for exposing range structs

2016-03-31 Thread Adam D. Ruppe via Digitalmars-d
On Thursday, 31 March 2016 at 16:38:59 UTC, Anon wrote: I've been spending my D time thinking about potential changes to how template string value parameters are encoded. How does it compare to simply gzipping the string and writing it out with base62?

Re: Policy for exposing range structs

2016-03-31 Thread Anon via Digitalmars-d
On Thursday, 31 March 2016 at 11:15:18 UTC, Johan Engelen wrote: Hi Anon, I've started implementing your idea. But perhaps you already have a beginning of an implementation? If so, please contact me :) https://github.com/JohanEngelen Thanks, Johan No, I haven't started implemented

Re: Policy for exposing range structs

2016-03-31 Thread Steven Schveighoffer via Digitalmars-d
On 3/31/16 10:30 AM, Adam D. Ruppe wrote: On Thursday, 31 March 2016 at 14:00:38 UTC, Steven Schveighoffer wrote: Ugh, let's try the huffman coding thing first :) Do that after and along with! Stack traces would be unusable. That's why I'd keep the function name outside the hash. The

Re: Policy for exposing range structs

2016-03-31 Thread jmh530 via Digitalmars-d
On Thursday, 31 March 2016 at 13:10:49 UTC, Steven Schveighoffer wrote: I too like Voldemort types, but I actually found moving the types outside the functions quite straightforward. It's just annoying to have to repeat the template parameters. If you make them private, then you can simply

Re: Policy for exposing range structs

2016-03-31 Thread Adam D. Ruppe via Digitalmars-d
On Thursday, 31 March 2016 at 14:00:38 UTC, Steven Schveighoffer wrote: Ugh, let's try the huffman coding thing first :) Do that after and along with! Stack traces would be unusable. That's why I'd keep the function name outside the hash. The inner spam wouldn't be readable (not like a

Re: Policy for exposing range structs

2016-03-31 Thread Steven Schveighoffer via Digitalmars-d
On 3/31/16 9:38 AM, Adam D. Ruppe wrote: Of course, a chain of voldemort types will still include that type as a template argument in the next call, so names as a whole can still be very long, but how long are your chains? I can see this becoming a 10 KB long name in extreme circumstances

Re: Policy for exposing range structs

2016-03-31 Thread Steven Schveighoffer via Digitalmars-d
On 3/31/16 9:38 AM, Adam D. Ruppe wrote: On Thursday, 31 March 2016 at 13:10:49 UTC, Steven Schveighoffer wrote: Voldemort types are what cause the bloat, templates inside templates aren't as much of a problem. So here's an idea of other things don't work out: voldemort types don't have a

Re: Policy for exposing range structs

2016-03-31 Thread Adam D. Ruppe via Digitalmars-d
On Thursday, 31 March 2016 at 13:10:49 UTC, Steven Schveighoffer wrote: Voldemort types are what cause the bloat, templates inside templates aren't as much of a problem. So here's an idea of other things don't work out: voldemort types don't have a name that can be said... that could be true

Re: Policy for exposing range structs

2016-03-31 Thread Steven Schveighoffer via Digitalmars-d
On 3/30/16 3:19 PM, Liran Zvibel wrote: On Sunday, 27 March 2016 at 17:01:39 UTC, David Nadlinger wrote: Compression in the usual sense won't help. Sure, it might reduce the object file size, but the full string will again have to be generated first, still requiring absurd amounts time and

Re: Policy for exposing range structs

2016-03-31 Thread Johan Engelen via Digitalmars-d
On Saturday, 26 March 2016 at 17:42:06 UTC, Anon wrote: The (conceptually) simple change I suggested brings the mangled name length down to O(n). Hi Anon, I've started implementing your idea. But perhaps you already have a beginning of an implementation? If so, please contact me :)

Re: Policy for exposing range structs

2016-03-30 Thread Liran Zvibel via Digitalmars-d
On Sunday, 27 March 2016 at 17:01:39 UTC, David Nadlinger wrote: Compression in the usual sense won't help. Sure, it might reduce the object file size, but the full string will again have to be generated first, still requiring absurd amounts time and space. The latter is definitely not

Re: Policy for exposing range structs

2016-03-27 Thread David Nadlinger via Digitalmars-d
On Saturday, 26 March 2016 at 05:22:56 UTC, Andrei Alexandrescu wrote: Compression of template names. -- Andrei Compression in the usual sense won't help. Sure, it might reduce the object file size, but the full string will again have to be generated first, still requiring absurd amounts

Re: Policy for exposing range structs

2016-03-26 Thread Anon via Digitalmars-d
On Saturday, 26 March 2016 at 05:22:56 UTC, Andrei Alexandrescu wrote: On 03/25/2016 11:40 AM, Steven Schveighoffer wrote: On 3/25/16 11:07 AM, Andrei Alexandrescu wrote: On 3/25/16 10:07 AM, Steven Schveighoffer wrote: We should actually be moving *away from* voldemort types:

Re: Policy for exposing range structs

2016-03-26 Thread Steven Schveighoffer via Digitalmars-d
On 3/26/16 1:24 AM, Andrei Alexandrescu wrote: On 03/25/2016 10:30 AM, Adam D. Ruppe wrote: On Friday, 25 March 2016 at 14:07:41 UTC, Steven Schveighoffer wrote: We should actually be moving *away from* voldemort types: Indeed, they basically suck. I love those things! -- Andrei I

Re: Policy for exposing range structs

2016-03-25 Thread Jonathan M Davis via Digitalmars-d
On Saturday, March 26, 2016 01:24:11 Andrei Alexandrescu via Digitalmars-d wrote: > On 03/25/2016 10:30 AM, Adam D. Ruppe wrote: > > On Friday, 25 March 2016 at 14:07:41 UTC, Steven Schveighoffer wrote: > >> We should actually be moving *away from* voldemort types: > > Indeed, they basically

Re: Policy for exposing range structs

2016-03-25 Thread Andrei Alexandrescu via Digitalmars-d
On 03/25/2016 10:30 AM, Adam D. Ruppe wrote: On Friday, 25 March 2016 at 14:07:41 UTC, Steven Schveighoffer wrote: We should actually be moving *away from* voldemort types: Indeed, they basically suck. I love those things! -- Andrei

Re: Policy for exposing range structs

2016-03-25 Thread Andrei Alexandrescu via Digitalmars-d
On 03/25/2016 11:40 AM, Steven Schveighoffer wrote: On 3/25/16 11:07 AM, Andrei Alexandrescu wrote: On 3/25/16 10:07 AM, Steven Schveighoffer wrote: We should actually be moving *away from* voldemort types: https://forum.dlang.org/post/n96k3g$ka5$1...@digitalmars.com Has this bug been

Re: Policy for exposing range structs

2016-03-25 Thread Steven Schveighoffer via Digitalmars-d
On 3/25/16 3:04 PM, Anon wrote: These repetitions could be eliminated relatively easily (from a user's perspective, anyway; things might be more difficult in the actual implementation). Two changes to the mangling: [snip] Please add these ideas to the bug report! I'm not sure if it

Re: Policy for exposing range structs

2016-03-25 Thread Anon via Digitalmars-d
On Friday, 25 March 2016 at 18:20:12 UTC, Steven Schveighoffer wrote: On 3/25/16 12:18 PM, H. S. Teoh via Digitalmars-d wrote: On Fri, Mar 25, 2016 at 11:40:11AM -0400, Steven Schveighoffer via Digitalmars-d wrote: On 3/25/16 11:07 AM, Andrei Alexandrescu wrote: On 3/25/16 10:07 AM, Steven

Re: Policy for exposing range structs

2016-03-25 Thread Steven Schveighoffer via Digitalmars-d
On 3/25/16 12:18 PM, H. S. Teoh via Digitalmars-d wrote: On Fri, Mar 25, 2016 at 11:40:11AM -0400, Steven Schveighoffer via Digitalmars-d wrote: On 3/25/16 11:07 AM, Andrei Alexandrescu wrote: On 3/25/16 10:07 AM, Steven Schveighoffer wrote: We should actually be moving *away from*

Re: Policy for exposing range structs

2016-03-25 Thread H. S. Teoh via Digitalmars-d
On Fri, Mar 25, 2016 at 11:40:11AM -0400, Steven Schveighoffer via Digitalmars-d wrote: > On 3/25/16 11:07 AM, Andrei Alexandrescu wrote: > >On 3/25/16 10:07 AM, Steven Schveighoffer wrote: > >> > >>We should actually be moving *away from* voldemort types: > >> >

Re: Policy for exposing range structs

2016-03-25 Thread Steven Schveighoffer via Digitalmars-d
On 3/25/16 11:07 AM, Andrei Alexandrescu wrote: On 3/25/16 10:07 AM, Steven Schveighoffer wrote: We should actually be moving *away from* voldemort types: https://forum.dlang.org/post/n96k3g$ka5$1...@digitalmars.com Has this bug been submitted? -- Andrei I'm not sure it's a bug that can

Re: Policy for exposing range structs

2016-03-25 Thread Andrei Alexandrescu via Digitalmars-d
On 3/25/16 10:07 AM, Steven Schveighoffer wrote: We should actually be moving *away from* voldemort types: https://forum.dlang.org/post/n96k3g$ka5$1...@digitalmars.com -Steve Has this bug been submitted? -- Andrei

Re: Policy for exposing range structs

2016-03-25 Thread H. S. Teoh via Digitalmars-d
On Fri, Mar 25, 2016 at 11:37:24AM +, Seb via Digitalmars-d wrote: > If I understand it correctly, the current policy in Phobos is that > range methods should use static nested structs to avoid the name > clutter and document the capabilities of the returned ranges in the > documentation.

Re: Policy for exposing range structs

2016-03-25 Thread Adam D. Ruppe via Digitalmars-d
On Friday, 25 March 2016 at 14:07:41 UTC, Steven Schveighoffer wrote: We should actually be moving *away from* voldemort types: Indeed, they basically suck. Really, I think the structs ought to be where the bulk of the documentation is, and the function is just listed as a convenience

Re: Policy for exposing range structs

2016-03-25 Thread Steven Schveighoffer via Digitalmars-d
On 3/25/16 7:37 AM, Seb wrote: If I understand it correctly, the current policy in Phobos is that range methods should use static nested structs to avoid the name clutter and document the capabilities of the returned ranges in the documentation. However there are a lot of old functions that

Policy for exposing range structs

2016-03-25 Thread Seb via Digitalmars-d
If I understand it correctly, the current policy in Phobos is that range methods should use static nested structs to avoid the name clutter and document the capabilities of the returned ranges in the documentation. However there are a lot of old functions that still use public structs, which