Re: Code review: JSON unmarshaller
On 2012-10-15 22:35, Tyler Jameson Little wrote:
I'm basically trying to reproduce other JSON marshallers, like Go's, but
using compile-time reflection. Go uses runtime reflection, which D
notably does not support. I like the idea of compile-time reflection
better anyway. There are a few things that would make it easier (like a
__traits call like allMembers that excludes functions).
Most other languages are not as complicated as D, it's basically only C
and C++ that are. Implementing a marshaller in Ruby would be dead
simple. No pointers, no array slices (in the same way as D), support for
full runtime reflection.
I use a lot of JSON, so a JSON marshaller/unmarshaller is going to save
a lot of time, and make my code a lot cleaner.
Most of these points are when unmarshalling. I haven't actually looked
if your marshaller can handle these cases but looking at the small
amount of code I would guess no.
* Pointers
I've done this, but haven't fully tested it. Basic pointers work.
Are they correctly setup when unmarshaling. Example:
int a = 3; // global/TLS
class Foo
{
int b = 4;
int* c;
int* d;
}
auto foo = new Foo;
foo.c = &a;
foo.d = &foo.b;
When unmarshaling will "foo.d" point to "foo.b"?
* Array slices
I think this is handled.
This is basically the same as pointers:
class Foo
{
int[] a;
int[] b;
}
auto foo = new Foo;
foo.a = [3, 4, 5, 6];
foo.b = foo.a[1 .. 3];
When unmarshaling will "foo.b" point to "foo.a"?
* Serializing through base class references
Doesn't __traits(allMembers, T) give everything from all super classes?
__traits only work at compile time.
class A
{
int a;
}
class B : A
{
int b;
}
A b = new B;
The static type of "b" is "A" so all information about "B" is lost at
compile time. You either need to provide a way to register all
subclasses that should be be marshaled through a base class reference or
you need to implement proper runtime reflection.
* const/immutable fields
Hmm, not sure to handle this. These have to be set in the constructor,
right?
You shouldn't call the constructor when unmarshaling. That's another
problem. Do you want to limit your marshaller to only work with classes
that have a default constructor or none.
You need to create the class instances without calling the constructor.
Then you could provide a method that will be called before/after
unmarshaling.
Have a look that this post:
http://www.digitalmars.com/d/archives/digitalmars/D/Deserializing_const_fields_175774.html
* Any reference type (not really hard but it's more work)
Are you talking about aliases? What other kind of reference types are
there in structs/classes? I'm assuming this will have more to do with
marshalling as opposed to unmarshalling.
Yes, you don't want to marshal the same object twice. References types
in D are: objects, pointers, associative arrays and arrays. These are
the ones I can think of for now.
Have a look at for a basically fully working serialization library
Orange:
https://github.com/jacob-carlborg/orange
Hmm, looks interesting. This looks like it only supports XML, which I
don't use, but I'm sure you've already solved a lot of the corner cases.
Thanks, I'll take a look!
I have solved a lot of corner cases but there are a few left. I have a
branch for handling const/immutable fields but it needs more testing
before merging it with the master branch.
I'm also not really happy about the deserializing of arrays. It's quite
slow.
Apparently it's also breaks as soon as you turn on some kind of
optimization when compiling.
The goal of Orange was to be able serialize basically everything found
in D. Also to support multiple archive types, i.e. XML, JSON, binary and
so on.
--
/Jacob Carlborg
Re: clear array
Damian: destroy and clear work nicely, except when using large arrays dmd is taking quite a while to compile int[1000][1000] arr; destroy(arr); Unfortunately currently DMD/druntime is not designed to handle large fixed-sized arrays well, in general (large static floating point or char 2D arrays are the worst case). In such cases you sometimes have to use a dynamic array, or sometimes a little more efficiently a fixed sized array of dynamic arrays. In Bugzilla there are two ore more bug entries that ask for speeder compilations or better handling of such large fixed-sized arrays. My dmd isn't even running the program that "int[1000][1000] arr;" array inside the main, I think I have to increase stack size. Maybe you have found another case where dmd or its druntime are not well optimized, I have filed it in Bugzilla for you: http://d.puremagic.com/issues/show_bug.cgi?id=8828 See there for more info. Bye, bearophile
Re: clear array
On Tuesday, 16 October 2012 at 00:44:15 UTC, Jonathan M Davis wrote: On Tuesday, October 16, 2012 02:37:52 Andrej Mitrovic wrote: On 10/16/12, Damian wrote: > Does D have a built-in way to clear arrays dynamic and > static? clear(arr). arr.clear() should work too. Use destroy, not clear. clear is now an alias for clear and should be going away eventually. There were too many problems with the ambiguity of the name clear, so it was renamed. - Jonathan M Davis destroy and clear work nicely, except when using large arrays dmd is taking quite a while to compile int[1000][1000] arr; destroy(arr);
Re: clear array
On Tuesday, 16 October 2012 at 00:47:44 UTC, H. S. Teoh wrote:
On Tue, Oct 16, 2012 at 02:33:03AM +0200, Damian wrote:
I know this is a bit of a dumb question and I have searched
and not
found anything concrete surprisingly :/
Does D have a built-in way to clear arrays dynamic and static?
I don't want to iterate over every element and set a default
value.
In C++ I would just use memset, I don't know if I should be
using
this for D?
What about:
import std.algorithm;
void main() {
int[] arr = [1,2,3,4];
fill(arr, 0);
writeln(arr); // should print [0,0,0,0]
}
?
T
This does not handle multi-dimensional arrays
Re: Operator overloading through UFCS doesn't work
On 10/15/12 17:49, Maxim Fomin wrote:
> On Monday, 15 October 2012 at 11:01:13 UTC, Artur Skawina wrote:
>> UFCS has pros and cons. I could agree that it has problems and should be
>> removed
>> from the language completely. But if the feature is there, it should work,
>> w/o any
>> unnecessary special cases.
>
> Special cases would be created by any decision, the only question is which
> feature is discriminated - alias this, UFCS or something else (currently UFCS
> is).
>
>> An overloaded operator is just another normal method; you get the same type
>> of
>> problems when dealing with "normal" methods - eg in types having an "alias
>> this" -
>> an UFCS "method" must take precedence over one reachable via the alias -
>> just like
>> in the overloaded op case. The only sane alternative would be disallowing
>> UFCS
>> for types with an "alias this" (which would be a severe limitation).
>
> You seem to be in the second camp (UFCS free function takes precedence over
> alias this, if declared). I am not against, just to note.
Actually, I'm not really in any camp. UFCS has several obvious problems plus
likely
quite a few more subtle ones. Ignoring the issues does not make them go away and
the
some-compiler-happens-to-implement-it-like-that-today-therefore-thats-how-it-
-must-work arguments, that often appear here, do not really help.
Note that my above "UFCS method must take precedence" statement only describes
the
required functionality; handling it like that /by default/ wouldn't probably be
a
good idea, as that would make accidental method hijacking possible.
The lookup should be more like
- T.method
- ufcs_method(T) marked with 'override'
- while (T = alias this) {
- T.method
- ufcs_method(T) marked with 'override'
}
- ufcs_method(T) w/o 'override'
- while (T = alias this)
- ufcs_method(T) w/o 'override'
with the compiler enforcing the obvious 'override' rules for ufcs_method
declarations
(requiring that T isn't opaque when declaring (or calling) UFCS functions is
reasonable,
i don't think having 'foo(a)' and 'a.foo()' mean completely different things
would work
well together with UFCS). And yes, it wouldn't completely eliminate the
possibility of
hijacking - but you'd need three components interacting for it to happen, which
would
make it much less likely to occur.
[Note i came up w/ this design while writing this email - it's not necessarily
perfect.]
>> And the purpose of UFCS is?... "operator overloading methods" are /not/
>> special.
>
> The point is that when you want to define UFCS free functions like opUnary,
> you want not only to call them like a.opUnary!"++"() but to code like ++a.
> That is the key issue here and that makes the whole case special.
>
> In other words: with UFCS you have an option: to call your function as it was
> a method of some type. And anyone has this option. The only problem is
> namespace conflict which can be easily avoided. But you still has the option.
> With UFCS operator overloaded functions you have *two* options: to call free
> functions as methods as usual *and* to use struct/class with many operators
> in a manner you want. But if anyone of that type users define his set of
> operator overloaded functions *you lose the second option* which makes the
> proposal to allow simultaneous access to single resource pointless.
This is how UFCS works - for "normal" methods. There's no reason to handle op
overloads or
other special methods differently. You are arguing for introducing arbitrary
restrictions,
which would bring no gain (that i can see right now), but disallow useful
functionality.
Yes, there are some issues, but those are not op-overload specific, but
UFCS-related.
> Yes, it also may happen with regular function, when you lose ability to give
> a function some specific name you want (like "create", "foo" etc.). But in
> case of UFCS operators you lose not only some function name ("opUnary") but
> corresponding expression as well (++).
> This means that it makes sense to allow only one set of opUnary/opBinary/..
> etc. of functions (anyway, only one can define them and use with operators)
> and the most suitable place is declaration of their type.
Well, i don't think anybody wants to /override/ existing operators - it's only
about
allowing /extending/ the functionality of non-local types, by adding support
for additional
ops and/or types. While being able to override existing methods with UFCS would
have some
uses, allowing that would also introduce additional problems.
Anyway, if you need to modify how a type's existing ops work you can always
sub-type -
this is also true in the (non-virtual) method case; UFCS is basically just
syntax sugar
(which allows you to transparently locally inject methods w/o creating a new
type,
downcasting and handling the (implicit) upcasts (which could be problematic
when eg
working with (pointers-to-)structs)).
artur
Re: clear array
On Tue, Oct 16, 2012 at 02:33:03AM +0200, Damian wrote:
> I know this is a bit of a dumb question and I have searched and not
> found anything concrete surprisingly :/
>
> Does D have a built-in way to clear arrays dynamic and static?
> I don't want to iterate over every element and set a default value.
>
> In C++ I would just use memset, I don't know if I should be using
> this for D?
What about:
import std.algorithm;
void main() {
int[] arr = [1,2,3,4];
fill(arr, 0);
writeln(arr); // should print [0,0,0,0]
}
?
T
--
My program has no bugs! Only undocumented features...
Re: clear array
On Tuesday, October 16, 2012 02:37:52 Andrej Mitrovic wrote: > On 10/16/12, Damian wrote: > > Does D have a built-in way to clear arrays dynamic and static? > > clear(arr). > arr.clear() should work too. Use destroy, not clear. clear is now an alias for clear and should be going away eventually. There were too many problems with the ambiguity of the name clear, so it was renamed. - Jonathan M Davis
Re: clear array
On Tuesday, 16 October 2012 at 00:33:04 UTC, Damian wrote: In C++ I would just use memset, I don't know if I should be using this for D? The direct D analog to memset is: int[] arr; arr[] = 0; // or whatever You can assign a value to a whole slice in a single go. arr[] means return a slice of the entire array. You can also do a range: arr[0 .. 4] = 0;, though, of course, you don't want to go out of bounds then. I believe arr[] = 0 actually compiles as memset, but however it is implemented, it does the same thing. Note that this sets the content, but leaves the length the same. If you want to get rid of it all, you can just assign null: arr = null; assert(arr.length == 0);
clear array
I know this is a bit of a dumb question and I have searched and not found anything concrete surprisingly :/ Does D have a built-in way to clear arrays dynamic and static? I don't want to iterate over every element and set a default value. In C++ I would just use memset, I don't know if I should be using this for D?
Re: how to create a local copy of shared pointer?
Thanks!
Re: Sorting algorithms
On 15-Oct-12 23:15, Era Scarecrow wrote: On Monday, 15 October 2012 at 15:49:51 UTC, thedeemon wrote: On Monday, 15 October 2012 at 09:18:12 UTC, Era Scarecrow wrote: Big-O notation doesn't give you actual numbers, O(n) = O(25*n). If you're interested in a practical method, look at TimSort and similar ones that combine different algorithms. Yeah I know it's more of a generalized number of steps, but it still gives you a good idea of sorting time. I'll give TimSort a look over. Currently I'm estimating this will be a variant of merge-sort. A hybrid. I'm currently trying to get into Phobos: https://github.com/D-Programming-Language/phobos/pull/787 -- Dmitry Olshansky
Re: Code review: JSON unmarshaller
I'm not sure what your goal with this marshaller is but I would say it's a lot harder than you think if you want to have a complete serialization library. A couple of things making it harder to create a fully working serialization library: I'm basically trying to reproduce other JSON marshallers, like Go's, but using compile-time reflection. Go uses runtime reflection, which D notably does not support. I like the idea of compile-time reflection better anyway. There are a few things that would make it easier (like a __traits call like allMembers that excludes functions). I use a lot of JSON, so a JSON marshaller/unmarshaller is going to save a lot of time, and make my code a lot cleaner. * Pointers I've done this, but haven't fully tested it. Basic pointers work. * Array slices I think this is handled. * Serializing through base class references Doesn't __traits(allMembers, T) give everything from all super classes? * const/immutable fields Hmm, not sure to handle this. These have to be set in the constructor, right? * Any reference type (not really hard but it's more work) Are you talking about aliases? What other kind of reference types are there in structs/classes? I'm assuming this will have more to do with marshalling as opposed to unmarshalling. Have a look at for a basically fully working serialization library Orange: https://github.com/jacob-carlborg/orange Hmm, looks interesting. This looks like it only supports XML, which I don't use, but I'm sure you've already solved a lot of the corner cases. Thanks, I'll take a look!
Re: Code review: JSON unmarshaller
On 2012-10-15 21:03, Tyler Jameson Little wrote: https://gist.github.com/3894337 This is my first non-trivial D code, and I'd eventually like to get this into Phobos as part of std.json. I haven't written the marshaller yet, but that shouldn't be too hard. I wanted to get some feedback on whether this code is up to the quality standards of Phobos. I'm not sure what your goal with this marshaller is but I would say it's a lot harder than you think if you want to have a complete serialization library. A couple of things making it harder to create a fully working serialization library: * Pointers * Array slices * Serializing through base class references * const/immutable fields * Any reference type (not really hard but it's more work) Have a look at for a basically fully working serialization library Orange: https://github.com/jacob-carlborg/orange -- /Jacob Carlborg
Re: More automated interfacing of D with C codebases
On 2012-10-15 19:20, CapitalLetterC wrote: But there are all sorts of issues in that same vein, like how certain structs get put together, what types are aliased with what, etc. That's why I was wondering if you could handle this analogously to how C does, with a config file that gets setup by a kind of configuration system. I think that would be possible but I haven't seen anyone program in D like that. -- /Jacob Carlborg
Re: Sorting algorithms
On Monday, 15 October 2012 at 15:49:51 UTC, thedeemon wrote: On Monday, 15 October 2012 at 09:18:12 UTC, Era Scarecrow wrote: Big-O notation doesn't give you actual numbers, O(n) = O(25*n). If you're interested in a practical method, look at TimSort and similar ones that combine different algorithms. Yeah I know it's more of a generalized number of steps, but it still gives you a good idea of sorting time. I'll give TimSort a look over. Currently I'm estimating this will be a variant of merge-sort.
Code review: JSON unmarshaller
https://gist.github.com/3894337 This is my first non-trivial D code, and I'd eventually like to get this into Phobos as part of std.json. I haven't written the marshaller yet, but that shouldn't be too hard. I wanted to get some feedback on whether this code is up to the quality standards of Phobos. I used a lot of templates, so I hope I didn't break any cardinal sins, especially in terms of readability. I did my best in grokking std.traits, but I may have missed some subtleties about what the templates are actually testing. I used asserts and contracts to validate input, so the following would throw an AssertError: int x = unmarshalJSON!int(`"5"`); I wasn't sure if this is bad style, since AssertError is in core.exception. If this is considered bad style in D, I can create a JSONMarshalException and throw that instead.
Re: templated static array
On Monday, 15 October 2012 at 17:05:30 UTC, Simen Kjaeraas wrote: On 2012-35-15 17:10, Namespace wrote: But bar([1, 2, 3]); not. The compiler does not realize that [1, 2, 3] means a static array in this context. You have to write bar(cast(int[3]) [1, 2, 3]); but I think the compiler have to recognize this on it's own. This is true. The problem is, as you say, that the compiler treats array literals as dynamic rather than static arrays. I would argue this is the correct default, but it's obviously not the default you want here. bearophile has posted about this on numerous occasions, and it's among his top thousand wanted features. :p I think this is a bug in IFTE. Please file it into bugzilla. Kenji Hara
Re: std.stream, BOM, and deprecation
On Sat, 13 Oct 2012 21:53:48 -0400, Charles Hixson wrote: If std.stream is being deprecated, what is the correct way to deal with file BOMs. This is particularly concerning utf8 files, which I understand to be a bit problematic, as there isn't, actually, a utf8 BOM, merely a convention which isn't a part of a standard. But the std.stdio documentation doesn't so much as mention byte order marks (BOMs). If this should wait until std.io is released, then I could use std.stream until them, but the documentation is already warning to avoid using it. When std.io is released, it will be fully BOM-aware by default (as long as you use the purely D versions). The plan from my point of view is for std.io be be a replacement backend for std.stdio, with the C version being the default (as it must be for compatibility purposes). -Steve
Re: templated static array
On Mon, Oct 15, 2012 at 07:05:13PM +0200, Simen Kjaeraas wrote:
> On 2012-35-15 17:10, Namespace wrote:
>
> >But bar([1, 2, 3]); not. The compiler does not realize that [1, 2, 3]
> >means a static array in this context. You have to write
> >bar(cast(int[3]) [1, 2, 3]); but I think the compiler have to
> >recognize this on it's own.
>
> This is true. The problem is, as you say, that the compiler treats
> array literals as dynamic rather than static arrays. I would argue
> this is the correct default, but it's obviously not the default you
> want here.
>
> bearophile has posted about this on numerous occasions, and it's among
> his top thousand wanted features. :p
[...]
I have to chime in here and say that I'm not particularly happy with the
way dmd currently processes array literals. I think dmd currently
doesn't take type inference far enough. If a particular array literal is
being passed to a function that expects a static array, say, then it
should be able to interpret the literal as a static array of the
matching type.
Similarly, when you have, say, a template function:
func(T)(T[] args) if (is(T==short)) { ... }
Supposing there is no other overload that accepts an integral array, the
compiler should know that when you write func([1,2,3]) the array literal
should be a short[]. Currently, it treats [1,2,3] as int[], and so fails
to match the template. Lots of ugly hacks and workarounds are necessary
to make things work correctly.
I say this is one area where things need improvement. Especially since
type inference is one of D's major selling points (what with voldemort
types, etc., that doesn't require the user to spell out a long
complicated name just to use a range function in std.range, among many
other nice perks).
T
--
Never ascribe to malice that which is adequately explained by incompetence. --
Napoleon Bonaparte
Re: More automated interfacing of D with C codebases
On Monday, 15 October 2012 at 12:54:51 UTC, Jacob Carlborg wrote: On 2012-10-15 13:51, CapitalLetterC wrote: This will be my first posting here, but I've been obsessed with D since before there was a D2 standard. Despite that rather long period of obsession, it's only just now that I've started seriously using D as I attempt to code-up some POC projects to demonstrate that the language is mature enough for serious development, or at least the kind of development we're doing, anyway. So, that out of the way, I suppose I should apologize in advance for what must amount to a pretty dumb question, since I truly couldn't have been the first to think of this: with regard to interfacing D with C libraries, why would we require a project like Deimos which ensures that each codebase is ported, almost entirely, by hand? Theoretically, shouldn't there be some kind of autotools-like method to establish something like, config.d.in which is then used via abstractions to properly configure the rest of the D wrapper? Obviously, D doesn't have a preprocessor, but couldn't one synthesize that with a series of judicious version() blocks? Then, whenever someone goes to build the application on their system, you'd have something to a series of autoconf-like tests to fill in the proper sections of of config.d.in and get out a config.d suitable for their system. I'm really sorry if there's some stupidly obvious reason this couldn't work and I'm just not seeing it, or if this is some n00b question you get all the time. The header files need to be ported so D can find the declarations of the functions and types. There are a couple of tools that can do most of the work, but not completely. Macros are specially hard to port automatically. https://github.com/jacob-carlborg/dstep Actually, I understand perfectly well why the headers need to be ported to D code, what I was more wondering is if anybody knew of or had ever tried to use some kind of build system to automate the process to the greatest extent possible. So, for example, as a result of the configuration process certain features may be implicitly or explicitly enabled/disabled, without probing the individual user's configuration, you wouldn't know that, so how does one maximize portability, in that case? If you simply go to support the maximal set of features, you may wind up with a bunch of codepaths that go nowhere, likewise, if you choose only to support the minimal set of functionality, you may not wind up with code that's not useful to anyone, yourself included. Granted, I hadn't thought of the difficulty in porting things like macros, but I'd imagine that, in the absolute worst case, you could synthesize their behavior with templates. But there are all sorts of issues in that same vein, like how certain structs get put together, what types are aliased with what, etc. That's why I was wondering if you could handle this analogously to how C does, with a config file that gets setup by a kind of configuration system.
Re: templated static array
On 2012-35-15 17:10, Namespace wrote: But bar([1, 2, 3]); not. The compiler does not realize that [1, 2, 3] means a static array in this context. You have to write bar(cast(int[3]) [1, 2, 3]); but I think the compiler have to recognize this on it's own. This is true. The problem is, as you say, that the compiler treats array literals as dynamic rather than static arrays. I would argue this is the correct default, but it's obviously not the default you want here. bearophile has posted about this on numerous occasions, and it's among his top thousand wanted features. :p -- Simen
Re: How many std.concurrency receivers?
On Oct 15, 2012, at 9:35 AM, Dmitry Olshansky wrote: > > I'm wondering if it will be possible to (sort of) intercept all common I/O > calls in 3rd party C libraries. Something like using our own "wrapper" on top > of C runtime but that leaves BSD sockets and a ton of WinAPI/Posix primitives > to care about. It's possible, but I don't know that I want to inject our own behavior into what users think is a C system call. I'd probably put the behavior into whatever networking API is added to Phobos though. Still not sure if this should be opt-out or not though, or how that would work.
Re: How many std.concurrency receivers?
On 15-Oct-12 05:58, Sean Kelly wrote: On Oct 14, 2012, at 9:59 AM, Dmitry Olshansky wrote: On 14-Oct-12 20:19, Sean Kelly wrote: On Oct 12, 2012, at 2:29 AM, Russel Winder wrote: On Thu, 2012-10-11 at 20:30 -0700, Charles Hixson wrote: […] I'm not clear on what Fibers are. From Ruby they seem to mean co-routines, and that doesn't have much advantage. But it also seems as […] I think the emerging consensus is that threads allow for pre-emptive scheduling whereas fibres do not. So yes as in Ruby, fibres are collaborative co-routines. Stackless Python is similar. Yep. If fibers were used in std.concurrency there would basically be an implicit yield in send and receive. Makes me wonder how it will work with blocking I/O and the like. If all of (few of) threads get blocked this way that going to stall all of (thousands of) fibers. Ideally, IO would be nonblocking with a yield there too, at least if the operation would block. I'm wondering if it will be possible to (sort of) intercept all common I/O calls in 3rd party C libraries. Something like using our own "wrapper" on top of C runtime but that leaves BSD sockets and a ton of WinAPI/Posix primitives to care about. -- Dmitry Olshansky
Re: how to create a local copy of shared pointer?
On Monday, 15 October 2012 at 15:34:43 UTC, denizzzka wrote: Because shared(S) and S are different types. Either declare s as shared too or use a cast. Why it was made in the language? This can be a safe automatic conversion I think. Theoretically compiler can work with shared and non-shared types very differently, with the latter it can do all the work in registers without always updating the memory because it knows no other threads will look at this data. Shared data is much more volatile. So actually not only automatic but also explicit cast should be illegal. I don't know how much differently the compiler deals with these types right now though. Currently the whole story of separating shared and local types is not fully thought, imho, we need some kind of shared-polymorphism, without it we have to use casts quite often and it's not a right thing to do.
Re: Sorting algorithms
On 10/15/12 8:11 AM, Philippe Sigaud wrote: On Mon, Oct 15, 2012 at 1:04 PM, Era Scarecrow wrote: On Monday, 15 October 2012 at 09:18:12 UTC, Era Scarecrow wrote: So an example area to be sorted with 16 elements would take on average about 100 compares while theoretically you can do it in half that number. Correction. 16 numbers would be solved in about 49 compares while an optimal sorting takes about 45. And for 21 numbers about 74 compares while optimally about 63. These numbers don't seem that large, but at the same time they do. Somewhat related: I once played with sorting networks and how to generate them at compile time in D. It's in template tutorial on Github. Here are some results: https://github.com/PhilippeSigaud/D-templates-tutorial/blob/master/templates_around.tex#L560 Discarding LaTeX markup: nSortingStandardratio networksort 5324532 1.642 10 55510961.975 15 80316792.091 20 1154 23142.005 25 1538 32442.109 30 2173 35081.614 35 4075 41201.011 40 5918 52690.890 45 7479 59590.797 50 9179 64350.701 Were n is the (predetermined) number of elements in an array and a ratio of 2.0 means sorting networks are twice faster than std.algo.sort in this particular case. I wanted to investigate small sorts using sorting networks for ages, but never got around to it. That's important for quicksort because it produces many small arrays that need sorting. Could you also test for very small sizes (2 to 4) and essentially test for 1-increment speed up to, say, 30 elements? I assume that's where the major wins come. I think we could use CT-generated sorting networks for arrays below a specific size. The converse risk is growth of generated code. Andrei
Re: Operator overloading through UFCS doesn't work
On Monday, 15 October 2012 at 11:01:13 UTC, Artur Skawina wrote:
UFCS has pros and cons. I could agree that it has problems and
should be removed
from the language completely. But if the feature is there, it
should work, w/o any
unnecessary special cases.
Special cases would be created by any decision, the only question
is which feature is discriminated - alias this, UFCS or something
else (currently UFCS is).
An overloaded operator is just another normal method; you get
the same type of
problems when dealing with "normal" methods - eg in types
having an "alias this" -
an UFCS "method" must take precedence over one reachable via
the alias - just like
in the overloaded op case. The only sane alternative would be
disallowing UFCS
for types with an "alias this" (which would be a severe
limitation).
You seem to be in the second camp (UFCS free function takes
precedence over alias this, if declared). I am not against, just
to note.
And the purpose of UFCS is?... "operator overloading methods"
are /not/ special.
artur
The point is that when you want to define UFCS free functions
like opUnary, you want not only to call them like
a.opUnary!"++"() but to code like ++a. That is the key issue here
and that makes the whole case special.
In other words: with UFCS you have an option: to call your
function as it was a method of some type. And anyone has this
option. The only problem is namespace conflict which can be
easily avoided. But you still has the option.
With UFCS operator overloaded functions you have *two* options:
to call free functions as methods as usual *and* to use
struct/class with many operators in a manner you want. But if
anyone of that type users define his set of operator overloaded
functions *you lose the second option* which makes the proposal
to allow simultaneous access to single resource pointless.
Consider this:
---somelib.d---
struct A { void foo() {} }
---otherlib.d---
void bar(A a) {}
---mycode.d---
// blah, foo and bar are taken
// solution - choose other name
void baz(A a) {}
---
Now assume, UFCS operator overload is possible.
---somelib.d---
struct A { int i; int j; }
---mycode.d---
int opUnary(string T : "++")()
{
return ++i;
}
...
++a;
...
-
At some point of time the owner of somelib.d changes code (or
anyone whom code you import define such functions):
---somelib.d---
struct A {
int i; int j;
int opUnary(string T : "++")()
{
return ++j;
}
}
---mycode.d---
int opUnary(string T : "++")()
{
return ++i;
}
So, you lost your option to use A in expressions and call your
function which is the point here. You cannot invent +my_unary+
operator. Neither you can rebind ++ to some function other than
opUnary.
Yes, it also may happen with regular function, when you lose
ability to give a function some specific name you want (like
"create", "foo" etc.). But in case of UFCS operators you lose not
only some function name ("opUnary") but corresponding expression
as well (++).
This means that it makes sense to allow only one set of
opUnary/opBinary/.. etc. of functions (anyway, only one can
define them and use with operators) and the most suitable place
is declaration of their type.
Re: Sorting algorithms
On Monday, 15 October 2012 at 09:18:12 UTC, Era Scarecrow wrote: Been watching online lectures that's going into sorting and searching, and from what I'm seeing most sorting algorithms (by using comparison; merge sort, quicksort, etc) and even tree algorithms peak at O(n log n). So an example area to be sorted with 16 elements would take on average about 100 compares while theoretically you can do it in half that number. Big-O notation doesn't give you actual numbers, O(n) = O(25*n). If you're interested in a practical method, look at TimSort and similar ones that combine different algorithms.
Re: LockingTextWriter/Reader
On Friday, 12 October 2012 at 17:48:40 UTC, Steven Schveighoffer wrote: This speeds up output tremendously, but still isn't as good as it could be with native D code. I see. What platform? On Windows, there is the whole issue of CRLF -> LF translation. I tried it on Linux. CRLF was my first thought too, but I didn't look too much into it since I wasn't sure if was was supposed to work with binary files at all! In my update to stdio (long overdue), I have support for reading/writing 5 forms of UTF -- UTF8, UTF16, UTF16LE, UTF32, UTF32LE, along with binary read/write support using native D buffers, and avoiding locking altogether if your object isn't shared. Yeah, that might be good. (I worry about breaking existing code, but meh, that's a separate issue.)
Re: templated static array
But bar([1, 2, 3]); not. The compiler does not realize that [1, 2, 3] means a static array in this context. You have to write bar(cast(int[3]) [1, 2, 3]); but I think the compiler have to recognize this on it's own.
Re: how to create a local copy of shared pointer?
On Monday, 15 October 2012 at 15:27:03 UTC, thedeemon wrote: On Monday, 15 October 2012 at 15:15:57 UTC, denizzzka wrote: S* s = new shared (S); // Why this is a illegal? Error: cannot implicitly convert expression (new shared(S)) of type shared(S)* to S* Because shared(S) and S are different types. Either declare s as shared too or use a cast. Why it was made in the language? This can be a safe automatic conversion I think.
Re: how to create a local copy of shared pointer?
On Monday, 15 October 2012 at 15:15:57 UTC, denizzzka wrote: S* s = new shared (S); // Why this is a illegal? Error: cannot implicitly convert expression (new shared(S)) of type shared(S)* to S* Because shared(S) and S are different types. Either declare s as shared too or use a cast.
Re: Pegged: error I am unable to fix
Hi,
Of course I can. But I "evaluated" this as my failure coming from fact
that D is new to me. So I was not sure if this is a bug in Pegged. So I
wrote here.
Dňa 15. 10. 2012 14:01 Philippe Sigaud wrote / napísal(a):
On Mon, Oct 15, 2012 at 12:24 PM, Lubos Pintes wrote:
SQValue <~ :"'" (! "'" .)* :"'"
DQValue <~ :'"' (! '"' .)* :'"'
Hi,
that came from the code generated for the quote and double-quote
literals. I have trouble finding a resilient way to produce code
dealing with ", when some template arguments are strings themselves.
Even r" " and q{ } do not work perfectly.
Anyway, I chose ` ` this time and your example now works with the
latest Git HEAD (I pushed a few minutes ago)
You can also use the predefined 'quote' and 'doublequote' (also,
'backquote' and 'backslash') rules in your own rules.
To get my attention more efficiently, could you please post any
trouble using the git issues system?
https://github.com/PhilippeSigaud/Pegged/issues?direction=desc&sort=created&state=open
Philippe
Re: templated static array
On 2012-23-15 16:10, Simen Kjaeraas wrote:
On 2012-05-15 16:10, Namespace wrote:
How can I do this?
I have this code: http://dpaste.dzfl.pl/d9165502
And as you can see, the templated function 'receive2' take
automatically dynamic arrays. But how can I tell the compiler, that
this function takes (preferably) static arrays?
My little "hack" function 'receive' take the type and the number of
elements. So the compiler know: it's a static array. But is there no
simpler trick to do this?
Maybe something like 'void receive(T)(static T vals) {'.
Nope. That's the way to do it.
No, wait, sorry. You don't need to specify those things when calling the
function. This works:
void bar(T, size_t n)(T[n] a) {}
void main(){
int[3] a;
bar(a);
}
--
Simen
how to create a local copy of shared pointer?
void main()
{
struct S { int payload; }
S* s = new shared (S); // Why this is a illegal?
}
Error: cannot implicitly convert expression (new shared(S)) of
type shared(S)* to S*
Re: templated static array
On Monday, 15 October 2012 at 15:10:43 UTC, bearophile wrote: Namespace: So there is no way that the compiler knows by himself how many elements are in the array? The syntax I have suggested doesn't have the problem you fear. Why don't you compile and run a little test program? Bye, bearophile I have: http://dpaste.dzfl.pl/661e4fb3 But I still have to specify the number of elements and the type. That is what I try to avoid.
Re: templated static array
Namespace: So there is no way that the compiler knows by himself how many elements are in the array? The syntax I have suggested doesn't have the problem you fear. Why don't you compile and run a little test program? Bye, bearophile
Re: templated static array
So there is no way that the compiler knows by himself how many
elements are in the array? something like this:
void receive(T, size_t n = vals.length)(T[n] vals) {
writeln(typeof(vals).stringof);
}
or:
void receive(T)(T[vals.length] vals) {
writeln(typeof(vals).stringof);
}
Re: templated static array
On 2012-05-15 16:10, Namespace wrote:
How can I do this?
I have this code: http://dpaste.dzfl.pl/d9165502
And as you can see, the templated function 'receive2' take automatically
dynamic arrays. But how can I tell the compiler, that this function
takes (preferably) static arrays?
My little "hack" function 'receive' take the type and the number of
elements. So the compiler know: it's a static array. But is there no
simpler trick to do this?
Maybe something like 'void receive(T)(static T vals) {'.
Nope. That's the way to do it.
--
Simen
templated static array
How can I do this?
I have this code: http://dpaste.dzfl.pl/d9165502
And as you can see, the templated function 'receive2' take
automatically dynamic arrays. But how can I tell the compiler,
that this function takes (preferably) static arrays?
My little "hack" function 'receive' take the type and the number
of elements. So the compiler know: it's a static array. But is
there no simpler trick to do this?
Maybe something like 'void receive(T)(static T vals) {'.
Re: More automated interfacing of D with C codebases
On 2012-10-15 13:51, CapitalLetterC wrote: This will be my first posting here, but I've been obsessed with D since before there was a D2 standard. Despite that rather long period of obsession, it's only just now that I've started seriously using D as I attempt to code-up some POC projects to demonstrate that the language is mature enough for serious development, or at least the kind of development we're doing, anyway. So, that out of the way, I suppose I should apologize in advance for what must amount to a pretty dumb question, since I truly couldn't have been the first to think of this: with regard to interfacing D with C libraries, why would we require a project like Deimos which ensures that each codebase is ported, almost entirely, by hand? Theoretically, shouldn't there be some kind of autotools-like method to establish something like, config.d.in which is then used via abstractions to properly configure the rest of the D wrapper? Obviously, D doesn't have a preprocessor, but couldn't one synthesize that with a series of judicious version() blocks? Then, whenever someone goes to build the application on their system, you'd have something to a series of autoconf-like tests to fill in the proper sections of of config.d.in and get out a config.d suitable for their system. I'm really sorry if there's some stupidly obvious reason this couldn't work and I'm just not seeing it, or if this is some n00b question you get all the time. The header files need to be ported so D can find the declarations of the functions and types. There are a couple of tools that can do most of the work, but not completely. Macros are specially hard to port automatically. https://github.com/jacob-carlborg/dstep -- /Jacob Carlborg
Re: Sorting algorithms
On Mon, Oct 15, 2012 at 1:04 PM, Era Scarecrow wrote: > On Monday, 15 October 2012 at 09:18:12 UTC, Era Scarecrow wrote: >> >> So an example area to be sorted with 16 elements would take on average >> about 100 compares while theoretically you can do it in half that number. > > > Correction. 16 numbers would be solved in about 49 compares while an > optimal sorting takes about 45. And for 21 numbers about 74 compares while > optimally about 63. > > These numbers don't seem that large, but at the same time they do. Somewhat related: I once played with sorting networks and how to generate them at compile time in D. It's in template tutorial on Github. Here are some results: https://github.com/PhilippeSigaud/D-templates-tutorial/blob/master/templates_around.tex#L560 Discarding LaTeX markup: nSortingStandardratio networksort 5324532 1.642 10 55510961.975 15 80316792.091 20 1154 23142.005 25 1538 32442.109 30 2173 35081.614 35 4075 41201.011 40 5918 52690.890 45 7479 59590.797 50 9179 64350.701 Were n is the (predetermined) number of elements in an array and a ratio of 2.0 means sorting networks are twice faster than std.algo.sort in this particular case.
Re: Pegged: error I am unable to fix
On Mon, Oct 15, 2012 at 12:24 PM, Lubos Pintes wrote:
> SQValue <~ :"'" (! "'" .)* :"'"
>
> DQValue <~ :'"' (! '"' .)* :'"'
Hi,
that came from the code generated for the quote and double-quote
literals. I have trouble finding a resilient way to produce code
dealing with ", when some template arguments are strings themselves.
Even r" " and q{ } do not work perfectly.
Anyway, I chose ` ` this time and your example now works with the
latest Git HEAD (I pushed a few minutes ago)
You can also use the predefined 'quote' and 'doublequote' (also,
'backquote' and 'backslash') rules in your own rules.
To get my attention more efficiently, could you please post any
trouble using the git issues system?
https://github.com/PhilippeSigaud/Pegged/issues?direction=desc&sort=created&state=open
Philippe
Re: toStringz note about keeping references
It's also possible that if a function reserves stack without clearing it, some old values may remain in the stack which were passed to other functions, so you'll have false pointers.
Re: toStringz note about keeping references
Probably a bug in GC, immutable data should be collected if not needed.
Re: Sorting algorithms
On Monday, 15 October 2012 at 09:18:12 UTC, Era Scarecrow wrote: So an example area to be sorted with 16 elements would take on average about 100 compares while theoretically you can do it in half that number. Correction. 16 numbers would be solved in about 49 compares while an optimal sorting takes about 45. And for 21 numbers about 74 compares while optimally about 63. These numbers don't seem that large, but at the same time they do.
Re: Operator overloading through UFCS doesn't work
On 10/15/12 10:17, Maxim Fomin wrote: > On Sunday, 14 October 2012 at 19:50:54 UTC, Artur Skawina wrote: >> On 10/14/12 08:13, Maxim Fomin wrote: >>> The only mentioned reason is to allow writing operator overloading methods >>> outside type scope - just because somebody (currently two people) consider >>> it logical to broaden UFCS usage. >> >> It's more than two people... Also, it's not about "broadening UFCS usage", >> it's about making UFCS work properly. > > When UFCS was added to the language its purpose was to call free functions > pretending you are invoking methods. It does it job pretty well and actually > works properly. But some questions arise: how this addition interacts with > other parts of the language: > - with calling through pointer (8603) > - with template alias parameters (8692) > - with function imports (6185) > - with typeof operator (8661) > - with operator overloading > - ... > - probably other issues which are not encountered yet. UFCS has pros and cons. I could agree that it has problems and should be removed from the language completely. But if the feature is there, it should work, w/o any unnecessary special cases. An overloaded operator is just another normal method; you get the same type of problems when dealing with "normal" methods - eg in types having an "alias this" - an UFCS "method" must take precedence over one reachable via the alias - just like in the overloaded op case. The only sane alternative would be disallowing UFCS for types with an "alias this" (which would be a severe limitation). > Each time there should be a decision to choose which language feature has > higher priority. That is why this is broadening UFCS usage on areas of the > language where it has never been before rather than making work properly in > existing areas of usage. > >>> This doesn't solve ay practical issue. >> >> Obviously, it does. Otherwise this issue wouldn't come up repeatedly. > > Actually not - the only purpose mentioned in the thread was to place operator > overloading methods outside scope of declaration. And the purpose of UFCS is?... "operator overloading methods" are /not/ special. There have been several threads in the past where this missing functionality was mentioned. It's how it should work, if UFCS is here to stay. If you think that would causes problems and UFCS should instead be removed from the language then I can understand that - UFCS /does/ have issues. But it's also useful and I'm not yet convinced that the problems it introduces justifies killing the feature. Nothing justifies special casing just certain combinations like op-overloading and UFCS however, especially when it's not necessary. artur
Pegged: error I am unable to fix
Hi,
I wanted to write very simple Markup lexer, that would recognize
HTML/XML tags, not structurally, only lexically.
DMD generates some error which I don't understand, either there is a
problem in my grammar, or possibly a bug in Pegged:
import pegged.grammar;
import pegged.peg;
enum mp=`
MarkupParser:
Markup <- Spacing (Tag / Text)* :eoi
Text <~ (! '<' .)*
Tag <- StartTag / EndTag
StartTag <- '<' Name Attr* '>'
EndTag < "'
Attr < Name ('=' Value)?
Name <- (alpha / Alpha) (alpha / Alpha / digit / '-' / '_' / ':' / '.')*
Value <- SQValue / DQValue / CharValue
SQValue <~ :"'" (! "'" .)* :"'"
DQValue <~ :'"' (! '"' .)* :'"'
CharValue <~ (!(Spacing / '>') .)*
`;
mixin(grammar(mp));
void main() {
}
Re: Operator overloading through UFCS doesn't work
On Sunday, 14 October 2012 at 20:15:15 UTC, Tommi wrote:
On Sunday, 14 October 2012 at 07:14:25 UTC, Maxim Fomin wrote:
If this request is approved and compiler has opUnary
definition outside type (which suits better then alias
this) such function would hijack alias this.
Free functions cannot and must not ever hijack, i.e. modify
existing functionality of a type. Free functions should only be
able to add new functionality to a type. This is what currently
happens with alias this vs free function which is accessed
through UFCS:
This shows current behavior. The issue is future behavior of code
like this:
---foo.d---
struct A
{
int i;
alias i this;
}
---bar.d---
int opUnary(string T)(A a) { ... }
...
{
++a;
}
---
I. i is incremented, opUnary is not called. However opUnary
matches better to the actual type and if it were a method, it
would be called - another special issue in the language which
breaks usual logic. And if you declared opUnary in bar.d when
alias this was absent in foo.d and later added - hijacking also
occurs but now it happens from another side. Bad.
II. opUnary is called, i is not incremented. On the one hand you
get what you wanted - you supplied opUnary and it is called. At
least this is consistent with matching behavior. On the other
hand, it is hijacking from position of foo. Bad.
III. Compiler issues error if you try to define some free
functions which are similar to existing methods (code above is
particular case) or if you declare operator overloading methods
in the presence of alias this. This prevents from making
confusion but if you link to some library, update code and its
author defines new method, which is similar to your UFCS
function, you get errors and have to rewrite code.
IV. Do nothing and leave things as they are. Presence of opUnary
function doesn't affect operator overloading. While current UFCS
behavior falls in the first category (newly created foo.d methods
hijack bar's free functions) there are no such problems with
operator overloading methods. And operator overloading requires
methods, not just free functions. Although methods and free
functions may be called similar in source code, they still are
different - in runtime calling, in mangling, in requiring
contract invocation, argument passing, etc.
Sorting algorithms
Been watching online lectures that's going into sorting and searching, and from what I'm seeing most sorting algorithms (by using comparison; merge sort, quicksort, etc) and even tree algorithms peak at O(n log n). So an example area to be sorted with 16 elements would take on average about 100 compares while theoretically you can do it in half that number. What algorithms get you closest to that optimal value? If there isn't any I have an idea that may just do it. Either way I'll be trying to implement it and see how it performs. I wonder what happens if I succeed and become famous... O.O
Re: Operator overloading through UFCS doesn't work
On Sunday, 14 October 2012 at 19:50:54 UTC, Artur Skawina wrote: On 10/14/12 08:13, Maxim Fomin wrote: The only mentioned reason is to allow writing operator overloading methods outside type scope - just because somebody (currently two people) consider it logical to broaden UFCS usage. It's more than two people... Also, it's not about "broadening UFCS usage", it's about making UFCS work properly. When UFCS was added to the language its purpose was to call free functions pretending you are invoking methods. It does it job pretty well and actually works properly. But some questions arise: how this addition interacts with other parts of the language: - with calling through pointer (8603) - with template alias parameters (8692) - with function imports (6185) - with typeof operator (8661) - with operator overloading - ... - probably other issues which are not encountered yet. Each time there should be a decision to choose which language feature has higher priority. That is why this is broadening UFCS usage on areas of the language where it has never been before rather than making work properly in existing areas of usage. This doesn't solve ay practical issue. Obviously, it does. Otherwise this issue wouldn't come up repeatedly. artur Actually not - the only purpose mentioned in the thread was to place operator overloading methods outside scope of declaration.
Re: Do we have GC-free hash map implementation sitting somewhere?
On 2012-10-15 08:49, Alex Rønne Petersen wrote: Hi, Is there a GC-free hash map implementation for D somewhere on the intertubes? (Preferably in a Git repository and under a liberal/non-viral license.) It looks like Martin Nowak has that in his SharedRuntime branch of druntime: https://github.com/dawgfoto/druntime/blob/SharedRuntime/src/rt/util/container.d#L428 Don't know if you can extract that. -- /Jacob Carlborg
Re: toStringz note about keeping references
On 2012-10-15 01:56, Jonathan M Davis wrote: However, I believe that another option is to explicitly tell the GC not collect a chunk of memory (glancing at core.memory, I suspect that removeRoot is the function to use for that, but I've never done it before, so I'm not well acquainted with the details). It's the other way around. The GC will collect anything not reachable from the roots. The correct way is to use "addRoot". -- /Jacob Carlborg
