[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 Johannes Pfau johannesp...@gmail.com changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution|--- |WONTFIX --- Comment #22 from Johannes Pfau johannesp...@gmail.com --- The volatileLoad/Store intrinsics will have to suffice. -- You are receiving this mail because: You are watching all bug changes.
Re: [Bug 126] Add support for attribute to mark data as volatile.
Am Wed, 16 Jul 2014 14:41:42 +0100
schrieb Iain Buclaw via D.gnu d.gnu@puremagic.com:
On 16 July 2014 14:10, Johannes Pfau via D.gnu d.gnu@puremagic.com
wrote:
Am Wed, 16 Jul 2014 05:53:30 +
schrieb Timo Sintonen t.sinto...@luukku.com:
On Tuesday, 24 June 2014 at 14:14:18 UTC, Johannes Pfau wrote:
I think we should at least try to bring this to the main
newsgroup,
I told you this is not going to work. The decision seems to be
made even when the conversion is still going on. Lets just make
this into gdc so we can continue the real work. When this is
ready and working, we may try again.
Unfortunately I am not skilled enough to implement this into the
compiler so I have to ask somebody else to do it. I will test it
as soon it is possible.
This has even more importance now because I have understood that
dmd will disallow read-modify-write access to shared variables. I
hope this feature is not brought to gdc before volatile is
working.
I didn't have any high expectations, nevertheless we had to try.
Well I could implement the DIP for GDC, but this is against the
vision of a shared frontend. In the end Iain has to decide what to
do.
I'm not sure if I'd implement Volatile!T though. It's probably lots
of work and there's no guarantee it'll work at all in the end. The
more I think about it the more corner cases come to mind which all
need to be worked around.
Keeping only peek/poke as Walter suggests of course also has
drawbacks. peek/poke plus Mike's wrapper is probably the best we'll
get from DMD.
Btw: @Iain if we implement these peek/poke things I think
asm{ : : :memory;} is not good enough. At least it's being
removed in some tests and according to some sources it only affect
operations on volatile variables... So we'd have to implement
peek/poke on top of C/GCC volatile.
Wouldn't a reinterpret cast (VIEW_CONVERT_EXPR) to volatile be enough?
Something akin to:
T peek(T var) { return *(volatile T*)var; }
T poke(T var, T val) { return *(volatile T*)var = *(volatile
T*)val; }
Yes I guess this should work. I just meant that a library solution with
compiler barriers would probably not work.
Re: [Bug 126] Add support for attribute to mark data as volatile.
On Tuesday, 24 June 2014 at 14:14:18 UTC, Johannes Pfau wrote: I think we should at least try to bring this to the main newsgroup, I told you this is not going to work. The decision seems to be made even when the conversion is still going on. Lets just make this into gdc so we can continue the real work. When this is ready and working, we may try again. Unfortunately I am not skilled enough to implement this into the compiler so I have to ask somebody else to do it. I will test it as soon it is possible. This has even more importance now because I have understood that dmd will disallow read-modify-write access to shared variables. I hope this feature is not brought to gdc before volatile is working.
Re: [Bug 126] Add support for attribute to mark data as volatile.
On 30 Jun 2014 23:20, bearophile via D.gnu d.gnu@puremagic.com wrote: Iain Buclaw: http://bugzilla.gdcproject.org/show_bug.cgi?id=126 The D1 deprecation warning to replace volatile with synchronized: https://github.com/D-Programming-Language/dmd/commit/c8d580aea687f16b56ff4ce935f618b41a74c6e7 Bye, bearophile That's Volatile statements. Not volatile types. Regards Iain
Re: [Bug 126] Add support for attribute to mark data as volatile.
To keep this thread going, I had a quick look at the reference material of the dip and picked some thoughts. In some languages volatile has a stronger meaning, like guaranteeing an atomic access. In some languages it may not guarantee anything. In this proposal volatile is only for optimization, not for protection. It does not add any code, it just prevents the optimizer removing some code. Read-modify-write and multi-word access are not guaranteed. The user should be aware of possible failures and for example avoid longer data types than the word size of the processor. Volatile is never recommended to use for communicatin between threads. It has been mentioned to be a reasonably good method to exchange data with an interrupt program. It is the only reasonable way to access hardware registers. The only others I have seen are library functions and inline assembly and they are not acceptable. Walter has been against this and now also Martin. I think there is no use to bring this to the main forum. I understand the point that it is not very good to have something in the language specs that can not be guaranteed. We just need this to access registers. Dmd is more for desktop and servers. It will never support all the targets gcc can. Could we make this a gdc extension? While writing this it just popped to my mind: if volatile is not good, could we reuse the 'system' word? Then it would be clear that this is for accessing system resources and not for application level. There seems not to be much documentation about system. Tdpl says it may omit some checks and the website says it is quite the same than not having any other attributes. So: What system means in general? What it currently means in gdc? Could we use it instead of volatile?
Re: [Bug 126] Add support for attribute to mark data as volatile.
Am Tue, 24 Jun 2014 10:46:11 + schrieb Timo Sintonen t.sinto...@luukku.com: To keep this thread going, I had a quick look at the reference material of the dip and picked some thoughts. In some languages volatile has a stronger meaning, like guaranteeing an atomic access. In some languages it may not guarantee anything. In this proposal volatile is only for optimization, not for protection. It does not add any code, it just prevents the optimizer removing some code. Read-modify-write and multi-word access are not guaranteed. The user should be aware of possible failures and for example avoid longer data types than the word size of the processor. Volatile is never recommended to use for communicatin between threads. It has been mentioned to be a reasonably good method to exchange data with an interrupt program. It is the only reasonable way to access hardware registers. The only others I have seen are library functions and inline assembly and they are not acceptable. Yes, that's exactly how I envision 'volatile'. Walter has been against this and now also Martin. I think there is no use to bring this to the main forum. I understand the point that it is not very good to have something in the language specs that can not be guaranteed. I think we should at least try to bring this to the main newsgroup, however I got distracted with other things and I want to extend the DIP a little (Only rationale stuff, no technical changes). The newsgroups have been quite busy lately and there've been discussions about two new DIPs. Neither Andrei nor Walter even posted a response in these threads so now is probably not a good time to start the discussion on this DIP. Volatile loses much of it's use when it's compiler specific. Although we could 'ignore' dmd there's also ldc. (However, as we already have incompatible inline asm compared to dmd and ldc it's probably not that bad) We just need this to access registers. Dmd is more for desktop and servers. It will never support all the targets gcc can. Could we make this a gdc extension? Probably, however I'd make it an UDA then in core.gcc, similar to @attribute. Main drawback here is that we add gdc specific code to the frontend. But as we need that code for @attribute anyway we could add some generic code to the frontend to allow backends to use UDAs. While writing this it just popped to my mind: if volatile is not good, could we reuse the 'system' word? Then it would be clear that this is for accessing system resources and not for application level. volatile is still a (deprecated) keyword, so we don't break code by reintroducing it. However, if we have to make it a gdc only extension I wouldn't use a keyword with such a prominent name. An @volatile UDA is much better then. Of course there are also non-technical issues regarding the name 'volatile'. C/C++ coders will already know what it means, but Java/C# coder could be confused.
Re: [Bug 126] Add support for attribute to mark data as volatile.
On Tuesday, 24 June 2014 at 14:14:18 UTC, Johannes Pfau wrote: Am Tue, 24 Jun 2014 10:46:11 + schrieb Timo Sintonen t.sinto...@luukku.com: To keep this thread going, I had a quick look at the reference material of the dip and picked some thoughts. In some languages volatile has a stronger meaning, like guaranteeing an atomic access. In some languages it may not guarantee anything. In this proposal volatile is only for optimization, not for protection. It does not add any code, it just prevents the optimizer removing some code. Agreed. In fact if this proposal reaches the main forum, we may hear proposals to introduce a don't optimize pragma or some other workaround. I considered it myself :( Walter has been against this and now also Martin. I think there is no use to bring this to the main forum. I understand the point that it is not very good to have something in the language specs that can not be guaranteed. I think we should at least try to bring this to the main newsgroup, however I got distracted with other things and I want to extend the DIP a little (Only rationale stuff, no technical changes). The newsgroups have been quite busy lately and there've been discussions about two new DIPs. Neither Andrei nor Walter even posted a response in these threads so now is probably not a good time to start the discussion on this DIP. Agreed, it would be a shame to not try. And looking for a low in controversial topics on the main forum would probably be a good idea. It would be bad for the passions of one discussion to spill over into this one. Walter told me at DConf that he was in favor of compiler intrinsic peek/poke functions as proposed in DIP20 and he said, in the meantime, I could get by with Martin's volatileGet/volatileSet assembly workaround. This tells me two things that are working against DIP62: 1) Walter believes volatile is a property of the load/store operation. 2) He doesn't consider volatile semantics a big priority. DIP20 has Walter's support, but has been collecting mold on the DIP list for a year and a half. Getting DIP62 approved will be a challenge, to put it mildly, but it will still need to be implemented and accepted into DMD. I was in favor of Walter's suggestion (DIP20) until DIP62. What sold me on DIP62 was the fact that one would never want to access volatile memory with non-volatile semantics. This is an irrefutable truth, and, as I can tell, only a type qualifier can enforce provide this enforcement. DIP62 will also be a difficult sell given the simple assembly workaround. I concede, the workaround will solve the problem and is a trivial implementation, but as Iain said, it is ...an excuse to *not* implement a feature that is rather essential for a systems language. I define a systems programming language as one that is generally accepted and used to implement operating systems (kernels and hardware drivers), and I think this is the traditional definition before the language marketers redefined it. D currently requires the help of C and/or other techniques to do this and lacks a runtime suitable for such development, so it is not a systems programming language IMO. But, out of all the other languages I have encountered, D has the most potential of any to be the systems programming language of choice in the future, and DIP62 is a step in that direction. The work of all those currently participating on this thread also shows some encouraging momentum in that direction. Unfortunately, there just doesn't seem to be very many people using D for systems programming, likely because of the reasons I gave above, and this poses another hurdle for DIP62: volatile has very little use outside of systems programming. DIP62 has my support, but I know that doesn't mean much. It will likely need the support of those with some clout in the D community. Mike
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #17 from Mike slavo5...@yahoo.com --- I hope we can all agree that volatile semantics are necessary for this kind of programming regardless of whether it is implemented as an asm workaround, compiler intrinsic functions, or a type qualifier. I also hope we can all agree that neither shared nor shared+atomic, when properly implemented, provides volatile semantics (See Iain's last comment [http://bugzilla.gdcproject.org/show_bug.cgi?id=126#c10] and DIP 4.2.4 - [http://wiki.dlang.org/DIP62#Why_volatile_and_shared_can.27t_be_merged]). Please correct me if I'm wrong. I think DIP62 4.2.2 justifies why it should be a type qualifier [http://wiki.dlang.org/DIP62#Why_a_type_qualifier]: because one would never want to access a volatile memory location without volatile semantics. Using a type qualifier would give the programmer this guarantee but volatile get/set (i.e. peek/poke) intrinsic functions would not. I have yet to hear an equally strong argument in favor of get/set intrinsics. Until such time, I support DIP62. -- You are receiving this mail because: You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #18 from Mike slavo5...@yahoo.com --- I should also add that the inline asm volatileGet/volatileSet workaround is just that: a workaround. For the programmer to have to resort to these techniques is an indication that the language is lacking. If I wanted to use my mmio.d for other architectures (which I actually intend to do), I would have to resort to writing a custom volatileGet/volatileSet pair for each architecture, and that doesn't scale well. -- You are receiving this mail because: You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #20 from Iain Buclaw ibuc...@gdcproject.org --- (In reply to Mike from comment #19) Oh, and about atomic access... I agree that memory-mapped IO is shared, global state, but I also agree that it is wasteful to wrap each and every access in atomic accessors. A large amount of memory mapped I/O is done during core, peripheral, and board initialization, before any threads or interrupts are even possible. There is absolutely no need to wrap these in atomic accessors. I believe the only one capable of making the best decision about when and where to add atomic access is the programmer, not the compiler. FYI: This change is in 2.066 out next month. https://github.com/D-Programming-Language/dmd/commit/7535f3dc86e9e88c4ee609dd98b5305224c3b88a Having such a restriction in place makes a need for volatile types even more so crucial for any future in embedded development. It's worth noting that shared has been in a persistent state of slow definition for the last two years, for instance there was a time when it was thought that shared should imply volatile ( https://issues.dlang.org/show_bug.cgi?id=9163#c1 ). Infact in retrospect, part of the reasoning behind removing volatile was that shared was expected to replace volatile, but come with safer guarantees. Yet two years on, this viewpoint has changed. Just from reading discussions of this even in the last months, people are questioning the usefulness of shared as a suitable replacement for the now removed volatile. The way I see it, we have two camps. On the one side, there's a real need to have a specialised type for use of hardware access. On the other, it's trying to bend a type that has been tailored for safe inter-thread communication. In other words, shared is not just a property of memory, but a property its actions, to which a volatile memory access may or may not uphold. I certainly do understand where you are coming from when you talk about scaling volatileSet/volatileGet - however it is hard to get that across sometimes when it seems the core developer only see it as porting a 2 line function between systems. Recently, I stumbled upon a previously rejected DIP which proposed to keep volatile and why (written by Zor) http://wiki.dlang.org/DIP17 He had some interesting points that I thought be worth examining two years on. In summary, he explains that the shared type qualifier has been suggested as a solution to the problems volatile tries to solve. He notes however the following drawbacks: It is not implemented in any compiler, so practically using it now is not possible at all. This is true, atomic types are a relatively new concept in a compiler, but this is has changed over time since C++x11 atomic support has been introduced. Compilers are much better quipped today to handle at least some aspects of what shared tries to solve. It does not have any well-defined semantics yet. I think the change in the upcoming 2.066 release changes this, in that we are now saying shared is clearly defined in both its transitivity (which will improve with library support), and it's thread @safety by disallowing operations that perform more than one operation without the safety of core.atomic memory barriers/fences. It will most likely not be portable because it's designed for the x86 memory model. Again with C++x11 atomics, this has been disproved. Infact it's probably of no coincidence that the way core.atomic's memory model is designed, it maps pretty much 1:1 to C++x11 atomics. If ever implemented, it will result in memory fences and/or atomic operations, which is **not** what volatile memory operations are about. This will severely affect pipelining and performance in general. This is the number one reason why volatile is needed. And why shared is wholly unsuitable. Further to his arguments, he also tries to disprove the use of two-liner inline assembly to solve the problem, I won't bother reviewing these, the whole point of the matter why volatile is needed is that I am asking you (politely) to not use the inline assembler as an excuse to *not* implement a feature that is rather essential for a systems language. -- You are receiving this mail because: You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #16 from Martin Nowak c...@dawg.eu --- (In reply to Johannes Pfau from comment #15) The classical hello world for these devices is usually blinking a LED (http://www.micahcarrick.com/tutorials/avr-microcontroller-tutorial/getting- started.html). This usually requires accessing two registers. Now if somebody asks us 'How does Hello World look like in D?' and we present a small D main function + 10 lines of ASM they'll laugh at us and will immediately stop considering D. It's the same for peek/poke. Nope, that would only be in the header/library that defines all those MM I/O registers. Those indirections are used for any embedded programming. For example in avr-libc you'll find this macro expansion. GPIOA - _SFR_MEM8(0x000A) - _MMIO_BYTE(0x000A) - (*(volatile uint8_t *)(mem_addr)). This expands to volatile, because volatile provides the correct semantics. But you could as well expand it to some template in D which implements the correct accesses. This is what Michael did for ARM [1]. The only problem here is how to implement volatile reads/writes semantically correct and with as little overhead as possible. Currently he used shared for reading [2] and writing [3] and those few places could as well be replaced with intrinsics or asm until intrinsics are available. https://github.com/JinShil/stm32_registers/blob/master/source/stm32/registers/rcc.d https://github.com/JinShil/memory_mapped_io/blob/d19cefb42000cd06605ddf4d4d6b120670400144/source/mmio.d#L335 https://github.com/JinShil/memory_mapped_io/blob/d19cefb42000cd06605ddf4d4d6b120670400144/source/mmio.d#L402 -- You are receiving this mail because: You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #15 from Johannes Pfau johannesp...@gmail.com --- Didn't knew about this proposal It's not yet been announced/discussed, though I'll probably start a discussion on the newsgroup this week. but it's flawed IMO, because read-modify-write operations might get interrupted. So you do need atomic updates for volatile data that is accessed from an interrupt handler. Adding a type qualifier for memory mapped I/O is overkill IMO. That's partially true, but in embedded programming you know exactly what the interrupt handler does, as you write it. And if an interrupt handler gets invoked on timer updates and you only rearm the timer and probably write a variable, then there's no need / it's wasteful to use atomic access to other registers (like GPIO control, ADC) in normal code. And if you modify the same registers in interrupt handlers and normal code, atomic access usually won't suffice, you often need critical sections anyway. Some architectures do not even provide atomic instructions (AVR). Instead you globally disable all interrupts if you want to do something atomic, then enable them again. Now think about how wasteful this gets if every register access is causing this. Obviously you want to leave atomic access to the programmer in this case. Also as this DIP tries to explain shared does not provide enough guarantees to replace volatile(4.2.4). Adding these guarantees to shared would prevent possible valid optimizations for real shared data. Whether ASM solutions or peek/poke are acceptable is probably a point of view thing. If the amount of code dealing with volatile/MMIO registers is low you might get away writing ASM. But for small microcontrollers doing only simple tasks you might end up writing ASM or peek/poke all the time and it'd be very hard for D to compete with C then. The classical hello world for these devices is usually blinking a LED (http://www.micahcarrick.com/tutorials/avr-microcontroller-tutorial/getting-started.html). This usually requires accessing two registers. Now if somebody asks us 'How does Hello World look like in D?' and we present a small D main function + 10 lines of ASM they'll laugh at us and will immediately stop considering D. It's the same for peek/poke. For small, embedded devices D hasn't got much to offer. Cleaner syntax, a little bit CTFE but accessing MMIO in a comfortable way is a deal breaker here. So from this point of view I'd say a type qualifier is well justified. I could also say a 'shared' type qualifier is not justified, because I don't even have multiple threads on embedded devices - as you see it's only a point of view thing. -- You are receiving this mail because: You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126
Martin Nowak c...@dawg.eu changed:
What|Removed |Added
CC||c...@dawg.eu
--- Comment #13 from Martin Nowak c...@dawg.eu ---
My answer to the topic from a mail exchange with Michael.
On 06/11/2014 12:45 PM, Mike Franklin wrote:
Martin,
If you are interested in this low level programming in D, you might also be
interested in DIP62 (http://wiki.dlang.org/DIP62).
If you remember from my presentation, there was a glaring problem with my
implementation: using 'shared' for 'volatile' semantics. This is
fundamentally incorrect. There is a discussion about it here
(http://forum.dlang.org/post/mailman.1081.1400818840.2907.d@puremagic.com).
There aren't many people in the D community doing such low-level work, so I
fear that this issue will be marginalized. I would like to see more
discussion about it, and hopefully some action. Your input would be quite
valuable.
Mike
Didn't knew about this proposal, but it's flawed IMO, because read-modify-write
operations might get interrupted. So you do need atomic updates for volatile
data that is accessed from an interrupt handler. Adding a type qualifier for
memory mapped I/O is overkill IMO.
The simple solution is to use shared+core.atomic for interrupt sensitive data
and a few functions for volatile.
void volatileSet(T)(ref shared(T) t, HeadUnshared!T val) if (T.sizeof == 4)
{ asm { naked; str r1, [r0]; bx lr; } }
HeadUnshared!T volatileGet(T)(ref shared(T) t) if (T.sizeof == 4)
{ asm { naked; ldr r0, [r0]; bx lr; } }
And because it's very difficult to ensure that certain memory-mapped I/O (which
is essentially shared global state) isn't affected by an interrupt routine,
it's safer to use core.atomic all the time. I don't think that atomic ops
(using MemoryOrder.raw) will a big performance penalty, because memory-mapped
I/O is rare.
If it is, the volatile workaround is simple enough to be implemented in a
library.
--
You are receiving this mail because:
You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126
--- Comment #14 from Martin Nowak c...@dawg.eu ---
Bit-bending I/O is a counter-example against the no performance needed
argument, but I still think this can be achieved easily enough with the
existing tools, i.e. by inlining the asm code in your hot loop or writing an
asm function.
for (uint i = 0; i 32; ++i)
{
if (data 1)
asm { mov r0, #1; str r0, [r3]; }
else
asm { mov r0, #2; str r0, [r3]; }
data = 1;
}
Compiler intrinsics for volatile loading/storing could be implemented with zero
overhead and it would be useful for at least one other cases (forced float
rounding to fix excess precision). So I favor this solution, but until then
core.atomic can be used.
--
You are receiving this mail because:
You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #11 from Mike slavo5...@yahoo.com --- Ok, clearly I have not fully understood shared semantics. In future, the compiler would memoize the loop and go straight for the assignment. mov$12, _D4test4globi(%rip) That would be very bad for my memory-mapped I/O needs. Regarding peek/poke functions: Don't you think that's too cumbersome? I also think shared + peek/poke has the drawback that you can still accidentally access it like a normal shared variable, without the peek/poke guarantees. I do find it cumbersome, but I'm ok with it because I'll be wrapping it in a mixin or a template. But you make a good point about accidental access. BTW: I finally finished the volatile DIP, see http://wiki.dlang.org/DIP62. It'd be great to get some early feedback from you and Iain, and feel free to edit the DIP :-) The DIP is extremely well written. I've read it a couple of times and I'm currently studying some of the references. I think you've made a very compelling case for adding volatile semantics, and I support it, but I must be honest, peek() and poke() intrinsics would also be fine for me (more about that later). I'm under the impression, however, that this DIP will be a very tough sell to Walter. I've created a design discussion around this debate on the D wiki: http://wiki.dlang.org/Language_design_discussions#volatile If we are to lobby the core design team to accept this DIP it would probably be wise to review past discussions and prepare offensive and defensive arguments. Walter said in the past that there is debate about what 'volatile' really means (http://forum.dlang.org/post/l4afr7$2pj8$1...@digitalmars.com) and argues that peek() and poke() intrinsics is the correct and guaranteed way to do memory-mapped I/O (http://forum.dlang.org/post/l4abnd$2met$1...@digitalmars.com) Daniel Murphy argued that it is property of the load/store operation and not the variable (http://forum.dlang.org/post/l4b1j4$acl$1...@digitalmars.com) and, I think this is the core of the debate. DIP62 makes a compelling case for why 'volatile' should be a property of the type, but I think it would help to justify why 'volatile' is not a property of the load/store operation. I can actually see it both ways, and am therefore somewhat on the fence. -- You are receiving this mail because: You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #12 from Johannes Pfau johannesp...@gmail.com --- I've created a design discussion around this debate on the D wiki: http://wiki.dlang.org/Language_design_discussions#volatile Thanks, that's very useful. I did not even remember that we already had DIPs for peek/poke functions. I'm under the impression, however, that this DIP will be a very tough sell to Walter. Regarding the old discussions I have to agree. But I think we need to make using volatile memory as simple or even simpler as in C. The Hello World of embedded programming is blinking a LED. If a C programmer sees something like poke(addr, peek(addr) | 0b1); to set a bit in a register they'll probably discard D immediately. I think this is a really important point. If we are to lobby the core design team to accept this DIP it would probably be wise to review past discussions and prepare offensive and defensive arguments. Yes, that's a good point. I'll revisit these old discussions and make some notes or update the DIP. Walter said in the past that there is debate about what 'volatile' really means (http://forum.dlang.org/post/l4afr7$2pj8$1...@digitalmars.com) Fortunately with this DIP this should no longer be a valid point. In the end what C does or did does not matter as long as we clearly specify what volatile is supposed to do in D and I think the Effects of volatile on code generation should describe all guarantees a compiler needs to provide. and argues that peek() and poke() intrinsics is the correct and guaranteed way to do memory-mapped I/O (http://forum.dlang.org /post/l4abnd$2met$1...@digitalmars.com) Well, that's not an argument ;-) It's also easy to prove wrong, nobody uses peek/poke on AVR, MSP430, ARM so it's probably not as undisputed as Walter claims. Daniel Murphy argued that it is property of the load/store operation and not the variable (http://forum.dlang.org/post/l4b1j4$acl$1...@digitalmars.com) and, I think this is the core of the debate. DIP62 makes a compelling case for why 'volatile' should be a property of the type, but I think it would help to justify why 'volatile' is not a property of the load/store operation. I can actually see it both ways, and am therefore somewhat on the fence. Yes, this is an important point and I'll extend the DIP a little in this regard. I think the point that you must _always_ access such memory obeying volatile rules shows why it is a property of the memory and not the access. There's no reasonable example where you want one access to the same location to be volatile and another time not. It's even dangerous sometimes (DIP 4.2.2). One other way to think about it is that all 'only the access is volatile' arguments apply in exactly the same way to 'shared' or 'immutable' variables. Only the access is ever affected, cause in the end that's the only thing you can do with variables. But we nevertheless have shared and immutable qualifiers, simply because we want this memory to be _always_ accessed in threadsafe/readonly ways, and it's exactly the same for volatile. Another point is that using peek/poke without a special qualifier relies only on conventions to ask programmers to always use the correct functions to access a pointer, the type system doesn't help. People already admitted this was the biggest mistake of D's volatile statement and I don't see how peek/poke would be different in this regard. And of course without a type qualifier there can't be transitivity. The programmer always has to be careful to access struct members, array members, and other types 'connected' via indirection with peek/poke. -- You are receiving this mail because: You are watching all bug changes.
Re: [Bug 126] Add support for attribute to mark data as volatile.
On Monday, 2 June 2014 at 17:27:52 UTC, Johannes Pfau wrote: And of course without a type qualifier there can't be transitivity. The programmer always has to be careful to access struct members, array members, and other types 'connected' via indirection with peek/poke. I too think that a) volatile is necessary, and b) that it should apply to variables, not operations. However, I'm not convinced of transitivity. It makes sense to treat members of a volatile struct as volatile, too, but I don't see why this needs to be the case for pointers. Are there even cases of volatile pointers at all? Usually, hardware registers don't contain pointers, and when they do (DMA-like things maybe, but those typically use physical addresses, not (virtual) pointers), what they point to would probably be normal memory, wouldn't it?
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #9 from Johannes Pfau johannesp...@gmail.com --- @Mike: AFAIK shared has some additional requirements which are not necessary for volatile: 1.) Atomic access: Accessing shared variables must always use atomic access (This is not yet enforced, but see https://github.com/D-Programming-Language/dmd/pull/3070) 2.) Accessing shared variables should also prevent reordering by the CPU (I'm not 100% sure about this though), not only by the compiler, as volatile does. I guess in the end shared variables will always be accessed via some function (atomicOp, synchronized statement + casting, ...). Also shared variables could ignore requirement 1. At least in C++/11 multiple accesses to atomics can be merged (http://stackoverflow.com/a/6681505/471401). Regarding peek/poke functions: Don't you think that's too cumbersome? I also think shared + peek/poke has the drawback that you can still accidentally access it like a normal shared variable, without the peek/poke guarantees. I'd prefer introducing a volatile type qualifier that enforces only one property: The compiler does not optimize access but we don't guarantee anything about the actual execution on hardware. Volatile memory also sometimes has properties which don't match traditional memory: I think I've seen architectures where reading a register might return a completely different value than the value written in a previous write. Most functions dealing with 'standard' memory will produce incorrect results if they operate on such memory. By using an additional 'volatile' type qualifier we can make sure that only functions which explicitly accept volatile memory can be used with volatile variables. BTW: I finally finished the volatile DIP, see http://wiki.dlang.org/DIP62 . It'd be great to get some early feedback from you and Iain, and feel free to edit the DIP :-) -- You are receiving this mail because: You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #10 from Iain Buclaw ibuc...@gdcproject.org --- (In reply to Johannes Pfau from comment #9) @Mike: AFAIK shared has some additional requirements which are not necessary for volatile: 1.) Atomic access: Accessing shared variables must always use atomic access (This is not yet enforced, but see https://github.com/D-Programming-Language/dmd/pull/3070) 2.) Accessing shared variables should also prevent reordering by the CPU (I'm not 100% sure about this though), not only by the compiler, as volatile does. I guess in the end shared variables will always be accessed via some function (atomicOp, synchronized statement + casting, ...). Also shared variables could ignore requirement 1. At least in C++/11 multiple accesses to atomics can be merged (http://stackoverflow.com/a/6681505/471401). That is a re-order by the compiler, which is (2) - there is still a guarantee that the var isn't cached in some way. Mike, this is the most common example of reordering I was mentioning about at the conference. --- shared int myshared; myshared = 0; foreach (i; 0 .. 12) myshared += 1; --- Under volatile semantics, the loop would be unrolled and ordering kept. mov$0, _D4test4globOi(%rip); myshared = 0 mov_D4test4globOi(%rip), %eax ; myshared += 1 add$1, %eax mov%eax, _D4test4globOi(%rip) ; And so on... --- In future, the compiler would memoize the loop and go straight for the assignment. mov$12, _D4test4globi(%rip) --- Pull #3070 is interesting, as it would disallow most cases where the compiler may get away with this kind of reordering behaviour. The current implementation of core.atomic does mean that all atomicOps are fully sequenced. As soon as we switch over from the old-style __sync intrinsics to C++x11 __atomic intrinsics, then this may change (MemoryOrder.raw will be adhered to). mov$0, _D4test4globOi; glob = 0 mov$12, %eax ; count = 12 xor%edx, %edx jmp.L2 .L1 mov_D4test4globOi, %edx ; atomicOp!+=(glob, 1) .L2 lock add $1, (%edx) ; subl $1, %eax ; count-- jne.L1 --- -- You are receiving this mail because: You are watching all bug changes.
Re: [Bug 126] Add support for attribute to mark data as volatile.
I did not yet read the dip but here are some of my thoughts:
At the old days peripherals were simple. An uart might have a
control register, a status register and a data register, 8 bit
each. It just did not matter how they were accessed. Now a
peripheral like usb or ethernet may have tens of 32 bit registers.
The Basic language did not have pointers or any way to address a
certain location of memory. Several extensions were made to get
access to system locations. One common extension was poke and
peek functions. They had 16 bit address and 8 bit data. Basic did
not have any data types or stuctures.
D has pointers that can be used to access memory. It also has
several data types. A library function does not know if it should
do 8/16/32/64 bit access without templates. That would be too
complicated for such a low level operation like register access.
The registers of a peripheral may be defined as a struct and a
pointer of this struct type is used to access the registers.
There are individual registers but there may also be some sub
register sets inside the register set. A peripheral may have
common registers and then per channel registers. The register
struct may then have substructs or an array of register sets that
may be accessed as structs or arrays.
Yes, there are different kind of registers.
- Normal registers can be read and written. These are used as
normal control and status registers. Some bits may be changed by
hardware at any time. This may be a problem because it is
impossible to have a fully atomic access. The time between read
and write should be as short as possible.
- Read only registers may be used to represent the capabilities
of the peripheral or calibration values. They always return the
same data. Status registers represent the current state of the
hardware and may change any time when the conditions change.
Write to these registers has no effect.
- Write only registers are used to send data. The data packet is
written byte by byte to this same address. These type of
registers are also used to clear status. Reading the register may
return the last data or zero or anything else and the value
should be ignored.
- Bidirectional registers are used as data registers. A read will
return the last received byte and a write will transmit the byte
written.
Usually it does not matter if these registers are accessed wrong
way (write a read only or read a write only) so there is no need
to mark them different. They can all be volatile.
It is also common that one register has mixed read/write, read
only and write only bits. Many registers have also
undefined/reserved bits, which sometimes should be written with
zeros and sometimes left as they are.
One of the most common operations is to wait some status:
while ((regs.status0x80)==0) { /* check timeout here */ }
The way to clear the status may be one of:
- write directly to the status bit
regs.status = 0xff7f;
- write a 1 to the bit
regs.status |= 0x80;
- sometimes writing 0 to other bits has no effect and there is no
need to read-modify-write
regs.status = 0x80;
- sometimes status is cleared by writing to another bit
regs.status |= 0x200;
- sometimes there is a separate clear register
regs.statusclear = 0x80;
- sometimes accessing the data register clears status
automatically
- sometimes reading the status register clears the status. In
this case all status bits have to be checked at once.
Many of these have the result that reading the register does not
give back the data that was written.
And no, I did not read this on Wikipedia. All these forms of
access exist in the processor I use (STM32F407) It seems that
several teams have made the peripherals on the chip and every
peripheral has its own way to access it.
Another thing is: do I need to mark every member in a struct
volatile or is it enough to mark the struct definition or the
struct pointer. Will it go transitively to every member of an
array of substructs or do I need mark the substructs volatile?
One thing is the struct pointer. The peripherals have a fixed
address. If there is only one peripheral, the address can be a
compile time constant. If there are several similar peripherals,
the address may be known at compile time or it could be immutable
that is initialized at start.
Now I have to make the pointer shared to have the struct members
shared. This means the pointer is also mutable and volatile. The
pointer can not be cached and has to be fetched again every time
the struct variables are accessed. This decreases performance.
D has been marketed as a system language. Accessing registers is
an essential part of system programming. Whatever the method is,
it has to be in the language, not an external library function.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #8 from Mike slavo5...@yahoo.com --- But regardless of the solution, it would be nice if it were a front-end feature implemented by all compilers. -- You are receiving this mail because: You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #3 from Iain Buclaw ibuc...@gdcproject.org --- Yes this has been raised at DConf. Michael gave a good talk on Bare Metal programming, and mentioned that he's relying on the use of shared currently meaning volatile to do a lot of low level work in D, but wasn't sure if it was the correct approach. I am quite open to solutions, other than I don't think 'volatile' as a keyword would be something reintroduced to the language am afaid. I can certainly point Walter and Michael in the direction of your DIP if that helps. :) -- You are receiving this mail because: You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #5 from Johannes Pfau johannesp...@gmail.com --- Oh, and back to topic: I am quite open to solutions, other than I don't think 'volatile' as a keyword would be something reintroduced to the language am afaid. Yes, that's what I fear as well. But thinking about it volatility is a property of the memory location and that best maps to a type qualifier. (It's not exactly the same thing, but we also have this conflation for immutable type / read only memory and in practice it should work fine). That's one reason why peek/poke are dangerous, just like the old D1 volatile statements: If a volatile memory area is typed as a normal pointer you can still pass it to functions which access it in 'non-volatile' ways. OTOH one of the biggest problems with volatile in C is that it's not properly standardized. If we invent GDC/LDC only solutions these will likely be slightly different causing the same mess as in C/C++ (or even worse, if we don't have a standard at all). So this is something which would really benefit from being part of the D standard. Maybe there's some chance we can introduce a new type qualifier, if not we'll have to do the next best thing, whatever that may be. -- You are receiving this mail because: You are watching all bug changes.
[Bug 126] Add support for attribute to mark data as volatile.
http://bugzilla.gdcproject.org/show_bug.cgi?id=126 --- Comment #1 from Iain Buclaw ibuc...@gdcproject.org --- peek() and poke() intrinsics that are guaranteed to be volatile accessors are also on the table. -- You are receiving this mail because: You are watching all bug changes.
