Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 09/16/2016 12:07 PM, Jacob Carlborg wrote:

On 2016-09-16 16:39, Andrei Alexandrescu wrote:


Jacob, would you want to try your hand at tweaking the defaults? They
are in a string in https://github.com/dlang/dmd/blob/master/src/doc.d.
Clearly there's agreement these can be improved, and it's incredibly
more productive to just improve the content instead of arguing how it
could. Thanks!! -- Andrei


My design skills are quite limited. The theme I'm using for TextMate is
basically the same as Apple is using for their documentation. But now
when I tried to find a link to it, I see that they've changed the design.

If I can use what I already have it might not be that much work.


That would be awesome!


Two questions:

* Can we have DMD output a separate CSS file? What will avoid the colon
issue mentioned before and avoid the need to have 300-500 lines of CSS
code in each HTML document


I'd say we start without a separate CSS file. When the need for it 
becomes obvious, we can do that. What would be great is that your 
initiative isn't forestalled by the need for some other work.



* Can we create a new macro which will be used for each source code line
in examples enclosed in "---"? That would allow to add line numbers with
CSS


We can, and we should (there are js solutions too). Again, let's get the 
ball rolling and amplify this as needed.



Thanks!

Andrei

Re: colour lib needs reviewers

[Jacob Carlborg via Digitalmars-d]

On 2016-09-16 16:39, Andrei Alexandrescu wrote:


Jacob, would you want to try your hand at tweaking the defaults? They
are in a string in https://github.com/dlang/dmd/blob/master/src/doc.d.
Clearly there's agreement these can be improved, and it's incredibly
more productive to just improve the content instead of arguing how it
could. Thanks!! -- Andrei


My design skills are quite limited. The theme I'm using for TextMate is 
basically the same as Apple is using for their documentation. But now 
when I tried to find a link to it, I see that they've changed the design.


If I can use what I already have it might not be that much work.

Two questions:

* Can we have DMD output a separate CSS file? What will avoid the colon 
issue mentioned before and avoid the need to have 300-500 lines of CSS 
code in each HTML document


* Can we create a new macro which will be used for each source code line 
in examples enclosed in "---"? That would allow to add line numbers with CSS


--
/Jacob Carlborg

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 9/16/16 7:30 AM, Jacob Carlborg wrote:

I don't see how that's relevant. Currently one needs to have knowledge
of HTML and CSS because the default output is not usable.


Jacob, would you want to try your hand at tweaking the defaults? They 
are in a string in https://github.com/dlang/dmd/blob/master/src/doc.d. 
Clearly there's agreement these can be improved, and it's incredibly 
more productive to just improve the content instead of arguing how it 
could. Thanks!! -- Andrei

Re: colour lib needs reviewers

[Johan Engelen via Digitalmars-d]

On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
I think I'm about as happy with my colour lib as I'm going to 
be. It really needs reviews.


I don't know enough about colors and what people do with it to 
give you helpful feedback, but I was wondering if you had looked 
at the (auto) vectorization of the code.
I see some matrix math code, which perhaps would benefit from 
LDC's @fastmath (or similar). It's just a thought. You can shoot 
me some example that you think should vectorize.


Anyway, I think you are mostly interested in API feedback, 
implementation optimization can come later.


(others: let's stop depressing Manu further and move all ddoc and 
other irrelevant discussions to other topics! :)

Re: colour lib needs reviewers

[Jacob Carlborg via Digitalmars-d]

On 2016-09-16 02:02, Walter Bright wrote:


The default styling was set up to produce legible output while not
requiring any configuration file, nor a .css style file nor any
knowledge of HTML at all.


I don't see how that's relevant. Currently one needs to have knowledge 
of HTML and CSS because the default output is not usable.


With Yard (documentation generator for Ruby) one just invokes "yard -f 
html -o output intput.rb" and it generates all files and put them into 
"output". The result looks something like this [1].


How is that requiring any HTML or CSS knowledge?

[1] 
http://www.rubydoc.info/gems/rails-bootstrap-helpers/0.1.0/RailsBootstrapHelpers/Helpers/ButtonHelper


--
/Jacob Carlborg

Re: colour lib needs reviewers

[Jacob Carlborg via Digitalmars-d]

On 2016-09-15 20:55, Andrei Alexandrescu wrote:


That's fine because "Returns" is special-cased. Random words at the
beginning of a line followed by ":" should not receive special treatment.


Ah, so you mean all non-standard non-special sections? Basically any 
section that is not listed in the ddoc documentation.



Those DDOC_XXX things that are used for formatting declarations.


Hmm, I don't think there's a DDOC_ macro that evaluates to the full 
source code a given symbol. For a function that would include the source 
code of the body.



Guess we can improve the (d)docs.


But then we need to have a standardized macro for that.


That'd be neat. As always, somebody's got to do it...


As always.

--
/Jacob Carlborg

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/15/2016 4:59 AM, Nemanja Boric wrote:

On Thursday, 15 September 2016 at 09:45:26 UTC, Marco Leise wrote:

Am Wed, 14 Sep 2016 23:44:16 -0700
schrieb Walter Bright :

I suspect that adding Markdown would help a lot (yes, I changed my mind about
that).




https://github.com/kiith-sa/harbored-mod provides support for mixing ddoc and
markdown. That's what we're using, and it's nice (especially because docs in
source files look very close to generated ones), but it would be nicer if we
could just use dmd for that, of course.


It looks like you've done some great work with this! And Boost licensed, too! I 
think we can leverage this for Ddoc markdown support. Thanks!

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/15/2016 7:37 AM, Jacob Carlborg wrote:

Yeah, if you already have defined all the standard ddoc macros because the
default styling is not useful at all.


The default styling was set up to produce legible output while not requiring any 
configuration file, nor a .css style file nor any knowledge of HTML at all.

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

Started a new thread on this, "Ddoc macro syntax". Please direct further 
discussion there.

Re: colour lib needs reviewers

[Bastiaan Veelo via Digitalmars-d]

On Thursday, 15 September 2016 at 09:46:07 UTC, Patrick Schluter 
wrote:
There are also some niceties like auto brief and 
autoreferencing (when a symbol is written in a text will be 
clickable if its body was reachable during generation of the 
doc).


Not only that, by using tag files [1] Doxygen even generates 
clickable symbols to third party libraries and other online 
documentation, without having access to the respective code.


[1] 
http://www.stack.nl/~dimitri/doxygen/manual/config.html#config_external

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 09/15/2016 02:34 PM, Jacob Carlborg wrote:

On 2016-09-15 14:54, Andrei Alexandrescu wrote:


Yah, this is terrible (it happens with http: all the time). We need to
suppress the special meaning of ":" to only the cases when it's preceded
by only one word and followed by a newline, e,g, "\ncolor:\n".


It's useful for many of the sections to allow the section name on the
same line as the content:

/// Returns: some useful value
int foo();

See the Ddoc documentation for more examples [1].


That's fine because "Returns" is special-cased. Random words at the 
beginning of a line followed by ":" should not receive special treatment.



Can this be done by defining the expansion macros appropriately?


What are "the expansion macros" ?


Those DDOC_XXX things that are used for formatting declarations.


Slashes are tolerable. The underscore I agree it's a quirk.


There are several (other) problems:

* How do I know I should use slashes? There's no documentation


"Monkey see, monkey do." :o)


* It's not a standard macro. If it's not here [2] nobody will find it or
use it


Guess we can improve the (d)docs.


That's a fair assessment, and an actionable one.


Perhaps the current implementation of Ddoc could be moved to a separate
tool that would focus on being a macro system. That can still be used
for the site and other documents. Ddoc in the compiler could be improved
to be a better tool for documenting code. I think these are two separate
use cases that might not be best accomplished with the same tool.


That'd be neat. As always, somebody's got to do it...


Andrei

Re: colour lib needs reviewers

[Jacob Carlborg via Digitalmars-d]

On 2016-09-15 16:45, Andrei Alexandrescu wrote:


Glad to hear that. It becomes a matter of improving the default-provided
macros and/or distribute some .ddoc batteries.

Jacob, I'd also like to thank for the great constructive ideas for
improving things and for formulating them as enhancement requests on
bugsilla. This is the kind of communication that pushes things forward.


The oldest bugzilla issue I referred to is more than four years old and 
this is not the first discussion on the topic :(


--
/Jacob Carlborg

Re: colour lib needs reviewers

[Jacob Carlborg via Digitalmars-d]

On 2016-09-15 14:54, Andrei Alexandrescu wrote:


Yah, this is terrible (it happens with http: all the time). We need to
suppress the special meaning of ":" to only the cases when it's preceded
by only one word and followed by a newline, e,g, "\ncolor:\n".


It's useful for many of the sections to allow the section name on the 
same line as the content:


/// Returns: some useful value
int foo();

See the Ddoc documentation for more examples [1].


Can this be done by defining the expansion macros appropriately?


What are "the expansion macros" ?


Slashes are tolerable. The underscore I agree it's a quirk.


There are several (other) problems:

* How do I know I should use slashes? There's no documentation
* It's not a standard macro. If it's not here [2] nobody will find it or 
use it



That's a fair assessment, and an actionable one.


Perhaps the current implementation of Ddoc could be moved to a separate 
tool that would focus on being a macro system. That can still be used 
for the site and other documents. Ddoc in the compiler could be improved 
to be a better tool for documenting code. I think these are two separate 
use cases that might not be best accomplished with the same tool.


[1] http://dlang.org/spec/ddoc.html#standard_srction
[2] http://dlang.org/spec/ddoc.html#predefined_macros

--
/Jacob Carlborg

Re: colour lib needs reviewers

[Meta via Digitalmars-d]

On Wednesday, 14 September 2016 at 20:14:48 UTC, Andrei 
Alexandrescu wrote:

On 09/14/2016 09:47 AM, Meta wrote:

On Wednesday, 14 September 2016 at 13:28:23 UTC, Manu wrote:

Cheers.
Yeah, I need to do better with ddoc.

... I'm just gonna go on the record and say that I am really, 
really

not enjoying ddoc ;)


I can't say I'm a fan either.


Why? -- Andrei


The syntax is clunky and it's poorly documented. I've been 
lurking this mailing list since late 2012 and sporadically 
contributing since late 2013, and I had no idea you could do 
something like your example for Manu using DDOC. It seems like a 
real case of NiH syndrome; there are plenty of more widely used 
systems that would probably fit D just fine, but instead we have 
our own special macro documentation language. Really the only 
thing DDOC has going for it in comparison to something like 
Doxygen, IMO, is momentum. It's just too much of a pain to switch 
now.

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 09/15/2016 11:25 AM, bachmeier wrote:

On Thursday, 15 September 2016 at 00:42:57 UTC, Andrei Alexandrescu wrote:


Do you mean like a macro $(VERBATIM ... text with balanced parens
...)? -- Andrei


Yes. Unfortunately I cannot find any mention of it on this page:
http://dlang.org/spec/ddoc.html


It doesn't exist :o). I agree it's needed, I've hit the matter a few 
times. See https://issues.dlang.org/show_bug.cgi?id=16498. -- Andrei

Re: colour lib needs reviewers

[bachmeier via Digitalmars-d]

On Thursday, 15 September 2016 at 00:42:57 UTC, Andrei 
Alexandrescu wrote:


Do you mean like a macro $(VERBATIM ... text with balanced 
parens ...)? -- Andrei


Yes. Unfortunately I cannot find any mention of it on this page:
http://dlang.org/spec/ddoc.html

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 09/15/2016 10:37 AM, Jacob Carlborg wrote:

Yeah, if you already have defined all the standard ddoc macros because
the default styling is not useful at all.


Glad to hear that. It becomes a matter of improving the default-provided 
macros and/or distribute some .ddoc batteries.


Jacob, I'd also like to thank for the great constructive ideas for 
improving things and for formulating them as enhancement requests on 
bugsilla. This is the kind of communication that pushes things forward.



Andrei

Re: colour lib needs reviewers

[Jacob Carlborg via Digitalmars-d]

On 2016-09-15 08:44, Walter Bright wrote:


The original idea was to be able to write documentation without needing
many macros, if any. Such as this I wrote recently:

/**
 * Provides a safe interface to the Posix stat() function.
 *
 * Detailed description that I didn't write for this function,
 * but this is where it goes.
 *
 * Another paragraph.
 *
 * Params:
 *path = file name
 *buf = stat instance to be filled in
 * Returns:
 *0 for success, system errno for error
 */

I suspect that adding Markdown would help a lot (yes, I changed my mind
about that). Also auto-detecting email addresses and urls and turning
them into clickable links would help.


Yeah, if you already have defined all the standard ddoc macros because 
the default styling is not useful at all.


This is an example of the documentation generator for Ruby called Yard 
[1]. The default output view/style might not be the best in the world 
but I think it's pretty good. Here's an example of how to write the 
documentation [2].


[1] http://www.rubydoc.info/gems/httparty/0.14.0/HTTParty/ClassMethods
[2] http://www.rubydoc.info/gems/yard/file/docs/GettingStarted.md

--
/Jacob Carlborg

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 09/15/2016 02:55 AM, Jacob Carlborg wrote:

On 2016-09-14 22:14, Andrei Alexandrescu wrote:


Why? -- Andrei


I've recently worked on a theme for Ddoc that will be used in TextMate.
For convince I started by adding the CSS inline in the head tag. Turns
out if you have CSS looking like this:

.foo {
  font-weight: bold;
  background-color: white;
}

Everything works perfectly fine. Then I added text color as well:

color: black;

Guess what happens? It cuts, it cuts everything after "color". The
solution is to use a colon macro:

Macros:
  COLON = :

color$(COLON) black;

It took me quite a while and a lot of frustration until I figured that
out. But I guess that's an inherit problem with a macro system.


Yah, this is terrible (it happens with http: all the time). We need to 
suppress the special meaning of ":" to only the cases when it's preceded 
by only one word and followed by a newline, e,g, "\ncolor:\n".



That is what I'm missing from Ddoc:

* Inline source code for each symbol [1]. The [1] example both has a
button to show the source code inline and a direct link to GitHub


Can this be done by defining the expansion macros appropriately?


* Link to GitHub for each symbol [1] and for the whole module. The way
it's done now with $(PHOBOSSRC std/net/_isemail.d) is not standardized
and feels like a hack. Why slashes and why the underscore?


Slashes are tolerable. The underscore I agree it's a quirk.


If anything
it should be the fully qualified name.


That'd be nice.


Ideally it should be automatic,
or a compiler recognized macro that inserts the link


Ironically the underscore is necessary because ddoc does too much 
automatically.



* Ddoc should organize symbols by visibility [2]
* Ddoc should generate index table [3]
* Ddoc should organize a module by symbol category [4]
* Add a way to inherit documentation from the parent class [5]

* Ddoc should give a list of inherited members [6]. JavaDoc does this by
default [7]

* Consistent way of cross-referencing. Ideally one would just pass the
fully qualified name (or local name for symbols in the same module) to a
compiler recognized macro and it would generate the appropriate links

* An index page with all symbols, including a filter [8]. This is super
convenient when you know the symbol to use but you need to double check
how it's used or the behavior. I use it almost every day with Ruby

All this should be done within Ddoc without the need for any post
processing.


I think we should start with a postprocessing-based approach and see 
what capabilities need to be added to obviate the need for postprocessing.



In general I feel that Ddoc is too much focused on a generic macro
system instead of focusing on being a good tool for writing
documentation for code.


That's a fair assessment, and an actionable one.


Andrei

Re: colour lib needs reviewers

[Nemanja Boric via Digitalmars-d]

On Thursday, 15 September 2016 at 09:45:26 UTC, Marco Leise wrote:

Am Wed, 14 Sep 2016 23:44:16 -0700
schrieb Walter Bright :

I suspect that adding Markdown would help a lot (yes, I changed 
my mind about that).





https://github.com/kiith-sa/harbored-mod provides support for 
mixing ddoc and markdown. That's what we're using, and it's nice 
(especially because docs in source files look very close to 
generated ones), but it would be nicer if we could just use dmd 
for that, of course.

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 9/15/16 2:11 AM, Walter Bright wrote:

On 9/14/2016 2:31 PM, Andrei Alexandrescu wrote:

The resulting look is alien too, as opposed to the more familiar
$MACRO(a1, a2,
a3) or the CPP-style MACRO(a1, a2, a3).


The $(MACRO a1, a2, a3) syntax comes from makefiles.


In the words of Seinfeld, "And that doesn't help either!"

I was thinking more about it and figured we could introduce a special 
macro at the very beginning of a .ddoc file or "Macros:" section called 
DDOC_MACRO_CALL_SYNTAX. It may be defined one of the following:


DDOC_MACRO_CALL_SYNTAX = DDOC

That's today's syntax.

DDOC_MACRO_CALL_SYNTAX = CPP

A macro will be invoked as MACRO(a1, a2).

DDOC_MACRO_CALL_SYNTAX = $CPP

A macro will be invoked as $MACRO(a1, a2)

DDOC_MACRO_CALL_SYNTAX = LaTeX

Macro invocation is \MACRO{a1}{a2}.

This definition is modular, i.e. it doesn't carry beyond the current 
document (otherwise the semantics of a .dd file depends on the .ddoc 
file(s) it's used with).


Macro definitions stay the same using $1, $2 etc. but macros invoked by 
them would need to obey the DDOC_MACRO_CALL_SYNTAX defined at the top.


Thoughts?


Andrei

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 15 September 2016 at 19:23, Jacob Carlborg via Digitalmars-d
 wrote:
> On 2016-09-15 09:11, Manu via Digitalmars-d wrote:
>>
>> Do we mind moving this discussion to the other topic I started
>> actually on this topic?
>>
>> It's slightly upsetting that this discussion about ddoc is more
>> popular than the colour lib I spent months workinng on :(
>
>
> Sorry.
>
> Is it the "ddoc latex/formulas?" topic? That's the only recent topic by you
> related to ddoc. But I have no interest in either latex or formulas.

Yeah, sorry I forgot that detail. That thread meandered in the same
direction though.

Re: colour lib needs reviewers

[Patrick Schluter via Digitalmars-d]

On Thursday, 15 September 2016 at 03:49:55 UTC, Andrei 
Alexandrescu wrote:

On 9/14/16 9:28 PM, Manu via Digitalmars-d wrote:
Like, I really just don't care enough to try and understand 
ddoc
sufficiently to have a bag of tricks like those you 
demonstrated above
to workaround these issues. It's not a skill I *want* to 
possess,

rather, in this case, it's a skill I'm being forced into.
You're welcome to call me lazy, I'd suggest I'm being 
realistic.
Perhaps a phobos contributor will be required to work it out 
(as seems

to be a requirement for me right now), but normal programmers
wouldn't. In 7 years, I've never been motivated to find 
workarounds to
ddoc shortcomings before now, and now, it's only because 
people are
hassling me. There's no intrinsic motivation here... one 
reason I use

D is because I hate the C preprocessor ;)


I don't see those workarounds to ddoc shortcomings. m4 and all 
macro systems I know of have similar idioms. It's the nature of 
macro processing. Are you simply saying you're familiar with 
other documentation tools and are not motivated to get into 
another one? That's entirely fair.


If I had to suggest, I'd introduce doxygen style \tags 
alongside the
macros, then when people try and type docs in the way they've 
been
doing for decades, it'll just work, and they can get on with 
their
code. Nobody likes writing documentation, it needs to have the 
minimum

possible friction or people just won't.


I have difficulty understanding this. I haven't looked at 
Doxygen in a long time and never really used it, but from what 
I see at 
https://www.stack.nl/~dimitri/doxygen/manual/commands.html it 
seems the \tags you refer to are just a form of macros. The 
syntax is different, i.e. you'd write \a hello whereas in html 
you'd write hello, in latex \textit{hello}, in ddoc $(I 
hello). It's a matter of syntax and though I agree syntax 
matters (and it would be nice to make ddoc's more 
configurable), is it right to assume you simply want a syntax 
you're more familiar with?


How do you mean people "type docs in the way they've been doing 
for decades"? Are you implying doxygen not only has been around 
for decades, but it's been some sort of ubiquitous standard? 
Honest question, I'm definitely not getting that.


doxygen is mostly javadoc compatible and that one is the standard.
(in fact the \ can be replaced by @ every where) and it allows 
direct html tags. The good thing about doxygen is that it 
supports also markdown.
There are also some niceties like auto brief and autoreferencing 
(when a symbol is written in a text will be clickable if its body 
was reachable during generation of the doc).




Does this boil down to - if we replace the macro syntax with 
one closer to doxygen things will just click? (That may as well 
be the case.)


Probably. But be reminded of your remark in the SDLang vs. JSON 
thread from last year:
"...this is a strategic error. Please throw s/SDL/DDOC/ away and 
use a standardized file format. -- Andrei

Re: colour lib needs reviewers

[Marco Leise via Digitalmars-d]

Am Wed, 14 Sep 2016 23:44:16 -0700
schrieb Walter Bright :

> I suspect that adding Markdown would help a lot (yes, I changed my mind about 
> that).

+1

-- 
Marco

Re: colour lib needs reviewers

[Jacob Carlborg via Digitalmars-d]

On 2016-09-15 09:11, Manu via Digitalmars-d wrote:

Do we mind moving this discussion to the other topic I started
actually on this topic?

It's slightly upsetting that this discussion about ddoc is more
popular than the colour lib I spent months workinng on :(


Sorry.

Is it the "ddoc latex/formulas?" topic? That's the only recent topic by 
you related to ddoc. But I have no interest in either latex or formulas.


--
/Jacob Carlborg

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 15 September 2016 at 17:38, Walter Bright via Digitalmars-d
 wrote:
> On 9/15/2016 12:06 AM, Manu via Digitalmars-d wrote:
>>
>> I think I have a sense of uneasy-ness that comes from my intuition
>> that macros are macros, and I don't know which ones are 'special', or
>> which are just macros.
>
>
> That's easy, they are all just macros, and can all be redefined.

Yeah... I suspected this was the case ;)

Re: colour lib needs reviewers

[Paolo Invernizzi via Digitalmars-d]

On Thursday, 15 September 2016 at 06:44:16 UTC, Walter Bright 
wrote:

On 9/14/2016 11:11 PM, Walter Bright wrote:

I suspect that adding Markdown would help a lot (yes, I changed 
my mind about that). Also auto-detecting email addresses and 
urls and turning them into clickable links would help.


I **really** love the pragmatism of this man!

;-)

/Paolo

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/15/2016 12:06 AM, Manu via Digitalmars-d wrote:

I think I have a sense of uneasy-ness that comes from my intuition
that macros are macros, and I don't know which ones are 'special', or
which are just macros.


That's easy, they are all just macros, and can all be redefined.

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

Do we mind moving this discussion to the other topic I started
actually on this topic?

It's slightly upsetting that this discussion about ddoc is more
popular than the colour lib I spent months workinng on :(


On 15 September 2016 at 16:55, Jacob Carlborg via Digitalmars-d
 wrote:
> On 2016-09-14 22:14, Andrei Alexandrescu wrote:
>
>> Why? -- Andrei
>
>
> I've recently worked on a theme for Ddoc that will be used in TextMate. For
> convince I started by adding the CSS inline in the head tag. Turns out if
> you have CSS looking like this:
>
> .foo {
>   font-weight: bold;
>   background-color: white;
> }
>
> Everything works perfectly fine. Then I added text color as well:
>
> color: black;
>
> Guess what happens? It cuts, it cuts everything after "color". The solution
> is to use a colon macro:
>
> Macros:
>   COLON = :
>
> color$(COLON) black;
>
> It took me quite a while and a lot of frustration until I figured that out.
> But I guess that's an inherit problem with a macro system.
>
> That is what I'm missing from Ddoc:
>
> * Inline source code for each symbol [1]. The [1] example both has a button
> to show the source code inline and a direct link to GitHub
>
> * Link to GitHub for each symbol [1] and for the whole module. The way it's
> done now with $(PHOBOSSRC std/net/_isemail.d) is not standardized and feels
> like a hack. Why slashes and why the underscore? If anything it should be
> the fully qualified name. Ideally it should be automatic, or a compiler
> recognized macro that inserts the link
>
> * Ddoc should organize symbols by visibility [2]
> * Ddoc should generate index table [3]
> * Ddoc should organize a module by symbol category [4]
> * Add a way to inherit documentation from the parent class [5]
>
> * Ddoc should give a list of inherited members [6]. JavaDoc does this by
> default [7]
>
> * Consistent way of cross-referencing. Ideally one would just pass the fully
> qualified name (or local name for symbols in the same module) to a compiler
> recognized macro and it would generate the appropriate links
>
> * An index page with all symbols, including a filter [8]. This is super
> convenient when you know the symbol to use but you need to double check how
> it's used or the behavior. I use it almost every day with Ruby
>
> All this should be done within Ddoc without the need for any post
> processing.
>
> In general I feel that Ddoc is too much focused on a generic macro system
> instead of focusing on being a good tool for writing documentation for code.
>
> [1]
> http://api.rubyonrails.org/classes/ActiveRecord/FinderMethods.html#method-i-first
>
> [2] https://issues.dlang.org/show_bug.cgi?id=11893
> [3] https://issues.dlang.org/show_bug.cgi?id=11891
> [4] https://issues.dlang.org/show_bug.cgi?id=11892
> [5] https://issues.dlang.org/show_bug.cgi?id=10399
> [6] https://issues.dlang.org/show_bug.cgi?id=7688
>
> [7]
> http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.jdt.doc.isv%2Freference%2Fapi%2Forg%2Feclipse%2Fjdt%2Fcore%2Fdom%2FSimpleName.html
>
> [8] http://ruby-doc.org/core-2.3.1
>
> --
> /Jacob Carlborg

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 15 September 2016 at 16:44, Walter Bright via Digitalmars-d
 wrote:
> On 9/14/2016 11:11 PM, Walter Bright wrote:
>>
>> On 9/14/2016 2:31 PM, Andrei Alexandrescu wrote:
>>>
>>> The resulting look is alien too, as opposed to the more familiar
>>> $MACRO(a1, a2,
>>> a3) or the CPP-style MACRO(a1, a2, a3).
>>
>>
>> The $(MACRO a1, a2, a3) syntax comes from makefiles.
>
>
> The original idea was to be able to write documentation without needing many
> macros, if any. Such as this I wrote recently:
>
> /**
>  * Provides a safe interface to the Posix stat() function.
>  *
>  * Detailed description that I didn't write for this function,
>  * but this is where it goes.
>  *
>  * Another paragraph.
>  *
>  * Params:
>  *  path = file name
>  *  buf = stat instance to be filled in
>  * Returns:
>  *  0 for success, system errno for error
>  */
>
> I suspect that adding Markdown would help a lot (yes, I changed my mind
> about that). Also auto-detecting email addresses and urls and turning them
> into clickable links would help.

+1

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 15 September 2016 at 13:49, Andrei Alexandrescu via Digitalmars-d
 wrote:
> On 9/14/16 9:28 PM, Manu via Digitalmars-d wrote:
>>
>> Like, I really just don't care enough to try and understand ddoc
>> sufficiently to have a bag of tricks like those you demonstrated above
>> to workaround these issues. It's not a skill I *want* to possess,
>> rather, in this case, it's a skill I'm being forced into.
>> You're welcome to call me lazy, I'd suggest I'm being realistic.
>> Perhaps a phobos contributor will be required to work it out (as seems
>> to be a requirement for me right now), but normal programmers
>> wouldn't. In 7 years, I've never been motivated to find workarounds to
>> ddoc shortcomings before now, and now, it's only because people are
>> hassling me. There's no intrinsic motivation here... one reason I use
>> D is because I hate the C preprocessor ;)
>
>
> I don't see those workarounds to ddoc shortcomings. m4 and all macro systems
> I know of have similar idioms. It's the nature of macro processing. Are you
> simply saying you're familiar with other documentation tools and are not
> motivated to get into another one? That's entirely fair.

That's absolutely what I'm saying, but not only that, given that I'm
being forced into it now, I'm actively not enjoying it.
I understand it's a macro system, and I'm saying that's the problem as
I perceive it :)

I clearly recall when I learned doxygen. I generally liked it, and I
was motivated to populate my code with docs.
Perhaps there's an obvious counter-pressure in this case since I
already know doxygen, but I think that's insignificant compared to the
fact that I just don't like ddoc.
This macro system is not what I would have done, and it's just not
compelling. I don't want documentation-realted macro definitions in my
code. I also hate the paren spam.


>> If I had to suggest, I'd introduce doxygen style \tags alongside the
>> macros, then when people try and type docs in the way they've been
>> doing for decades, it'll just work, and they can get on with their
>> code. Nobody likes writing documentation, it needs to have the minimum
>> possible friction or people just won't.
>
>
> I have difficulty understanding this. I haven't looked at Doxygen in a long
> time and never really used it, but from what I see at
> https://www.stack.nl/~dimitri/doxygen/manual/commands.html it seems the
> \tags you refer to are just a form of macros. The syntax is different, i.e.
> you'd write \a hello whereas in html you'd write hello, in latex
> \textit{hello}, in ddoc $(I hello). It's a matter of syntax and though I
> agree syntax matters (and it would be nice to make ddoc's more
> configurable), is it right to assume you simply want a syntax you're more
> familiar with?

I think of those examples, doxygen is objectively the least annoying
syntax! It's 2 bytes, and doesn't introduce delimiters or parens of
any kind.

Sure, that's definitely an influence, I hate paren spam, but it's not
all. If in practise I was able to make my docs be how I want as easily
as I was able to the first time I encountered doxygen, I wouldn't be
bothered. But I've had a number of contacts with ddoc now, and it's
just not intuitive how to make my docs how I want, whereas with
doxygen, I was walking within minutes, seconds even.

I think I have a sense of uneasy-ness that comes from my intuition
that macros are macros, and I don't know which ones are 'special', or
which are just macros.
Ie, some macros are obviously not actually macros, but keywords in disguise.
Doxygen says things like "\a word" (style this as an argument), ddoc
seems to be $(I word) (italic this), which says nothing about the
semantic meaning of the thing allowing styling to be deferred.
I refer to this page: https://dlang.org/spec/ddoc.html
There's nothing there that says how to style a template argument for
instance, it's all 'italic, big, small, bold' etc, and skimming other
ddoc code, I see people using raw styling like $(I ..) (as you just
demonstrated above).
At face value, it feels clumsy and poorly defined. I'm sure I'm
incorrect, but I just didn't have this same first-impression with
doxygen, or C#'s doco standard (they're the only 2 others I've used
before).

The only reference I've ever looked at for doxygen is this page:
https://www.stack.nl/~dimitri/doxygen/manual/commands.html
I never read another line from their manuals, everything else emerged
via osmosis, and I think osmosis is the correct way a programmer
should learn this stuff, which means it needs to be as obvious and
intuitive as possible.


> How do you mean people "type docs in the way they've been doing for
> decades"? Are you implying doxygen not only has been around for decades, but
> it's been some sort of ubiquitous standard? Honest question, I'm definitely
> not getting that.

Do you feel that doxygen is not a ubiquitous standard? It's been
around for like 20 years.
I can't think of any non-doxygen docs I 

Re: colour lib needs reviewers

[Jacob Carlborg via Digitalmars-d]

On 2016-09-14 22:14, Andrei Alexandrescu wrote:


Why? -- Andrei


I've recently worked on a theme for Ddoc that will be used in TextMate. 
For convince I started by adding the CSS inline in the head tag. Turns 
out if you have CSS looking like this:


.foo {
  font-weight: bold;
  background-color: white;
}

Everything works perfectly fine. Then I added text color as well:

color: black;

Guess what happens? It cuts, it cuts everything after "color". The 
solution is to use a colon macro:


Macros:
  COLON = :

color$(COLON) black;

It took me quite a while and a lot of frustration until I figured that 
out. But I guess that's an inherit problem with a macro system.


That is what I'm missing from Ddoc:

* Inline source code for each symbol [1]. The [1] example both has a 
button to show the source code inline and a direct link to GitHub


* Link to GitHub for each symbol [1] and for the whole module. The way 
it's done now with $(PHOBOSSRC std/net/_isemail.d) is not standardized 
and feels like a hack. Why slashes and why the underscore? If anything 
it should be the fully qualified name. Ideally it should be automatic, 
or a compiler recognized macro that inserts the link


* Ddoc should organize symbols by visibility [2]
* Ddoc should generate index table [3]
* Ddoc should organize a module by symbol category [4]
* Add a way to inherit documentation from the parent class [5]

* Ddoc should give a list of inherited members [6]. JavaDoc does this by 
default [7]


* Consistent way of cross-referencing. Ideally one would just pass the 
fully qualified name (or local name for symbols in the same module) to a 
compiler recognized macro and it would generate the appropriate links


* An index page with all symbols, including a filter [8]. This is super 
convenient when you know the symbol to use but you need to double check 
how it's used or the behavior. I use it almost every day with Ruby


All this should be done within Ddoc without the need for any post 
processing.


In general I feel that Ddoc is too much focused on a generic macro 
system instead of focusing on being a good tool for writing 
documentation for code.


[1] 
http://api.rubyonrails.org/classes/ActiveRecord/FinderMethods.html#method-i-first


[2] https://issues.dlang.org/show_bug.cgi?id=11893
[3] https://issues.dlang.org/show_bug.cgi?id=11891
[4] https://issues.dlang.org/show_bug.cgi?id=11892
[5] https://issues.dlang.org/show_bug.cgi?id=10399
[6] https://issues.dlang.org/show_bug.cgi?id=7688

[7] 
http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.jdt.doc.isv%2Freference%2Fapi%2Forg%2Feclipse%2Fjdt%2Fcore%2Fdom%2FSimpleName.html


[8] http://ruby-doc.org/core-2.3.1

--
/Jacob Carlborg

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/14/2016 11:11 PM, Walter Bright wrote:

On 9/14/2016 2:31 PM, Andrei Alexandrescu wrote:

The resulting look is alien too, as opposed to the more familiar $MACRO(a1, a2,
a3) or the CPP-style MACRO(a1, a2, a3).


The $(MACRO a1, a2, a3) syntax comes from makefiles.


The original idea was to be able to write documentation without needing many 
macros, if any. Such as this I wrote recently:


/**
 * Provides a safe interface to the Posix stat() function.
 *
 * Detailed description that I didn't write for this function,
 * but this is where it goes.
 *
 * Another paragraph.
 *
 * Params:
 *  path = file name
 *  buf = stat instance to be filled in
 * Returns:
 *  0 for success, system errno for error
 */

I suspect that adding Markdown would help a lot (yes, I changed my mind about 
that). Also auto-detecting email addresses and urls and turning them into 
clickable links would help.

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/14/2016 2:31 PM, Andrei Alexandrescu wrote:

The resulting look is alien too, as opposed to the more familiar $MACRO(a1, a2,
a3) or the CPP-style MACRO(a1, a2, a3).


The $(MACRO a1, a2, a3) syntax comes from makefiles.

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 9/14/16 9:28 PM, Manu via Digitalmars-d wrote:

Like, I really just don't care enough to try and understand ddoc
sufficiently to have a bag of tricks like those you demonstrated above
to workaround these issues. It's not a skill I *want* to possess,
rather, in this case, it's a skill I'm being forced into.
You're welcome to call me lazy, I'd suggest I'm being realistic.
Perhaps a phobos contributor will be required to work it out (as seems
to be a requirement for me right now), but normal programmers
wouldn't. In 7 years, I've never been motivated to find workarounds to
ddoc shortcomings before now, and now, it's only because people are
hassling me. There's no intrinsic motivation here... one reason I use
D is because I hate the C preprocessor ;)


I don't see those workarounds to ddoc shortcomings. m4 and all macro 
systems I know of have similar idioms. It's the nature of macro 
processing. Are you simply saying you're familiar with other 
documentation tools and are not motivated to get into another one? 
That's entirely fair.



If I had to suggest, I'd introduce doxygen style \tags alongside the
macros, then when people try and type docs in the way they've been
doing for decades, it'll just work, and they can get on with their
code. Nobody likes writing documentation, it needs to have the minimum
possible friction or people just won't.


I have difficulty understanding this. I haven't looked at Doxygen in a 
long time and never really used it, but from what I see at 
https://www.stack.nl/~dimitri/doxygen/manual/commands.html it seems the 
\tags you refer to are just a form of macros. The syntax is different, 
i.e. you'd write \a hello whereas in html you'd write hello, in 
latex \textit{hello}, in ddoc $(I hello). It's a matter of syntax and 
though I agree syntax matters (and it would be nice to make ddoc's more 
configurable), is it right to assume you simply want a syntax you're 
more familiar with?


How do you mean people "type docs in the way they've been doing for 
decades"? Are you implying doxygen not only has been around for decades, 
but it's been some sort of ubiquitous standard? Honest question, I'm 
definitely not getting that.


Does this boil down to - if we replace the macro syntax with one closer 
to doxygen things will just click? (That may as well be the case.)



Andrei

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 15 September 2016 at 07:31, Andrei Alexandrescu via Digitalmars-d
 wrote:
> On 09/14/2016 09:28 AM, Manu via Digitalmars-d wrote:
>>
>> ... I'm just gonna go on the record and say that I am really, really
>> not enjoying ddoc ;)
>
>
> I was using ddoc today for the Greeks in DIP1000 and was musing about what
> makes it annoying. I think a major part is the macro invocation syntax:
> $(MACRO a1, a2, a3). After writing it for a few dozen times today, I figured
> it's objectively awkward to type (at least on the US keyboard): the dollar
> sign followed by the left paren is just unpleasant for mechanical reasons.
>
> The resulting look is alien too, as opposed to the more familiar $MACRO(a1,
> a2, a3) or the CPP-style MACRO(a1, a2, a3).
>
> There are a couple of un(der)documented tricks without which life is very
> difficult. For example, people commonly believe that in order to embed a
> literal comma they need to go like this:
>
> COMMA = ,
>
> and then $(MACRO Hello$(COMMA) world, a2). That scales poorly with the size
> of the argument. In fact, there's a much better solution:
>
> ARGS = $0
>
> and then $(MACRO $(ARGS Hello, world), a2) allows any amount of text with
> commas. Sadly no similar solution for unpaired parens.
>
> Also, the fact that the doc opens with the text and then has an obscure
> "Macros:" line at a point is unpleasant. The macro definitions should come
> first, followed by a separator and then the content. That would make for a
> much nicer flow.
>
> There are a few other macros I never leave home without:
>
> ARGS = $0
> COMMA = ,
> DOLLAR = $
> LPAREN = (
> RPAREN = )
> TAIL = $+
> COLON = :
> LF = $(LF)
> $(LF)
> COMMENT =
> SP = $(COMMENT space coming) $(COMMENT that was a space)
>
> TAIL sadly doesn't work properly.
>
> Overall: I'm trying to build a list of ddoc grievances that can be fixed.
> It's the de jure and de facto standard for D documentation (with the ddoc
> processor or ddox), so we'd do good to improve it.
>
> Please add your own ideas.

I haven't had enough experience for meaningful input. I have
experienced the comma and parens things myself, and I generally just
hate that everything is a macro, like you said yourself above.
It didn't occur to me to attempt the sort of workarounds you
illustrated here, I rather just factored the commas and parens out of
my docs and moved on ;)

Like, I really just don't care enough to try and understand ddoc
sufficiently to have a bag of tricks like those you demonstrated above
to workaround these issues. It's not a skill I *want* to possess,
rather, in this case, it's a skill I'm being forced into.
You're welcome to call me lazy, I'd suggest I'm being realistic.
Perhaps a phobos contributor will be required to work it out (as seems
to be a requirement for me right now), but normal programmers
wouldn't. In 7 years, I've never been motivated to find workarounds to
ddoc shortcomings before now, and now, it's only because people are
hassling me. There's no intrinsic motivation here... one reason I use
D is because I hate the C preprocessor ;)

If I had to suggest, I'd introduce doxygen style \tags alongside the
macros, then when people try and type docs in the way they've been
doing for decades, it'll just work, and they can get on with their
code. Nobody likes writing documentation, it needs to have the minimum
possible friction or people just won't.

Added bonus; when you cut C code to adapt bindings, the existing
docs on the C code would remain valid documentation without additional
amendment.
I bind C code regularly, and my goto 'solution' is to just remove the
doxygen doco comments from the translated code and point them to the
C/C++ docs instead.
Again, call me what you like, I think my behaviour is probably typical
though. I'm just being honest.

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 9/14/16 7:04 PM, bachmeier wrote:

On Wednesday, 14 September 2016 at 21:31:51 UTC, Andrei Alexandrescu wrote:


Please add your own ideas.


Andrei


One thing that I didn't like when I used DDOC was that I couldn't find a
way to add a block of text directly to the output file. I know that can
be done with html, but it's not the same. When I am programming I want
to make notes (the way I make comments) but not get bogged down with all
the macros/html stuff. Unformatted text is way better than unwritten
text. Blocks of markdown would be better, but blocks of text would have
helped.

[Maybe this is possible but I was a new D user at that time and haven't
had a reason to look into it.]


Do you mean like a macro $(VERBATIM ... text with balanced parens ...)? 
-- Andrei

Re: colour lib needs reviewers

[bachmeier via Digitalmars-d]

On Wednesday, 14 September 2016 at 21:31:51 UTC, Andrei 
Alexandrescu wrote:



Please add your own ideas.


Andrei


One thing that I didn't like when I used DDOC was that I couldn't 
find a way to add a block of text directly to the output file. I 
know that can be done with html, but it's not the same. When I am 
programming I want to make notes (the way I make comments) but 
not get bogged down with all the macros/html stuff. Unformatted 
text is way better than unwritten text. Blocks of markdown would 
be better, but blocks of text would have helped.


[Maybe this is possible but I was a new D user at that time and 
haven't had a reason to look into it.]

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 09/14/2016 05:31 PM, Andrei Alexandrescu wrote:

After writing it for a few dozen times today, I figured it's objectively
awkward to type (at least on the US keyboard): the dollar sign followed
by the left paren is just unpleasant for mechanical reasons.

The resulting look is alien too, as opposed to the more familiar
$MACRO(a1, a2, a3) or the CPP-style MACRO(a1, a2, a3).


I meant to add here: we can improve this in a backward-compatible way by 
allowing users to choose syntax by means of some notation, e.g. this 
thing as the first macro:


DD0C_SYNTAX = $(X)

for the existing syntax, vs.

DD0C_SYNTAX = $X()

for the $MACRO(a1, a2) syntax. We may also allow users to define the 
argument separator, also on a per-document basis. By default:


DDOC_ARGUMENT_SEPARATOR = ,

We may also allow custom parens and so on.


Andrei

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 09/14/2016 09:28 AM, Manu via Digitalmars-d wrote:

... I'm just gonna go on the record and say that I am really, really
not enjoying ddoc ;)


I was using ddoc today for the Greeks in DIP1000 and was musing about 
what makes it annoying. I think a major part is the macro invocation 
syntax: $(MACRO a1, a2, a3). After writing it for a few dozen times 
today, I figured it's objectively awkward to type (at least on the US 
keyboard): the dollar sign followed by the left paren is just unpleasant 
for mechanical reasons.


The resulting look is alien too, as opposed to the more familiar 
$MACRO(a1, a2, a3) or the CPP-style MACRO(a1, a2, a3).


There are a couple of un(der)documented tricks without which life is 
very difficult. For example, people commonly believe that in order to 
embed a literal comma they need to go like this:


COMMA = ,

and then $(MACRO Hello$(COMMA) world, a2). That scales poorly with the 
size of the argument. In fact, there's a much better solution:


ARGS = $0

and then $(MACRO $(ARGS Hello, world), a2) allows any amount of text 
with commas. Sadly no similar solution for unpaired parens.


Also, the fact that the doc opens with the text and then has an obscure 
"Macros:" line at a point is unpleasant. The macro definitions should 
come first, followed by a separator and then the content. That would 
make for a much nicer flow.


There are a few other macros I never leave home without:

ARGS = $0
COMMA = ,
DOLLAR = $
LPAREN = (
RPAREN = )
TAIL = $+
COLON = :
LF = $(LF)
$(LF)
COMMENT =
SP = $(COMMENT space coming) $(COMMENT that was a space)

TAIL sadly doesn't work properly.

Overall: I'm trying to build a list of ddoc grievances that can be 
fixed. It's the de jure and de facto standard for D documentation (with 
the ddoc processor or ddox), so we'd do good to improve it.


Please add your own ideas.


Andrei

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/14/2016 5:47 AM, Meta wrote:

DDOC sections such as Returns, Args, etc.


s/Args/Params/

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 09/14/2016 09:47 AM, Meta wrote:

On Wednesday, 14 September 2016 at 13:28:23 UTC, Manu wrote:

Cheers.
Yeah, I need to do better with ddoc.

... I'm just gonna go on the record and say that I am really, really
not enjoying ddoc ;)


I can't say I'm a fan either.


Why? -- Andrei

Re: colour lib needs reviewers

[Marco Leise via Digitalmars-d]

Am Wed, 14 Sep 2016 13:47:43 +
schrieb Meta :

> On Wednesday, 14 September 2016 at 13:28:23 UTC, Manu wrote:
> > Cheers.
> > Yeah, I need to do better with ddoc.
> >
> > ... I'm just gonna go on the record and say that I am really, 
> > really not enjoying ddoc ;)  
> 
> I can't say I'm a fan either.

You can replace $(D …) with `…` nowadays. It made me a bit more
of a fan. :)

-- 
Marco

Re: colour lib needs reviewers

[Meta via Digitalmars-d]

On Wednesday, 14 September 2016 at 13:28:23 UTC, Manu wrote:

Cheers.
Yeah, I need to do better with ddoc.

... I'm just gonna go on the record and say that I am really, 
really not enjoying ddoc ;)


I can't say I'm a fan either.

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 14 September 2016 at 22:47, Meta via Digitalmars-d
 wrote:
> On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
>>
>> I think I'm about as happy with my colour lib as I'm going to be. It
>> really needs reviews.
>>
>> I added packed-RGB support, including weird micro-float and
>> shared-exponent formats.
>> They're important for interacting with any real-time rendering libraries.
>> There is only one texture format I'm aware of that isn't supported,
>> and that is u7e3 floats, only available on Xbox360 ;)
>>
>> Chromatic adaptation functions are in.
>> I've done a tidy/reorg pass.
>>
>> I'm not sure what else should be in the scope of this lib.
>>
>> PR: https://github.com/dlang/phobos/pull/2845
>> dub: https://code.dlang.org/packages/color
>> docs:
>> http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html
>>
>> - Manu
>
>
> Looks like a nice and simple module. My only complaint is that DDOC sections
> such as Returns, Args, etc. don't seem to be used at all. Also maybe a bit
> more detailed of an overview in package.d, which I assume is the entry point
> for the documentation.

Cheers.
Yeah, I need to do better with ddoc.

... I'm just gonna go on the record and say that I am really, really
not enjoying ddoc ;)

If I could get people starting to review the code, I'll polish up the
docs over the next few days.

Re: colour lib needs reviewers

[Meta via Digitalmars-d]

On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
I think I'm about as happy with my colour lib as I'm going to 
be. It really needs reviews.


I added packed-RGB support, including weird micro-float and
shared-exponent formats.
They're important for interacting with any real-time rendering 
libraries.
There is only one texture format I'm aware of that isn't 
supported,

and that is u7e3 floats, only available on Xbox360 ;)

Chromatic adaptation functions are in.
I've done a tidy/reorg pass.

I'm not sure what else should be in the scope of this lib.

PR: https://github.com/dlang/phobos/pull/2845
dub: https://code.dlang.org/packages/color
docs: 
http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html


- Manu


Looks like a nice and simple module. My only complaint is that 
DDOC sections such as Returns, Args, etc. don't seem to be used 
at all. Also maybe a bit more detailed of an overview in 
package.d, which I assume is the entry point for the 
documentation.

Re: colour lib needs reviewers

[Marco Leise via Digitalmars-d]

Am Wed, 14 Sep 2016 11:36:12 +1000
schrieb Manu via Digitalmars-d :

> On 14 September 2016 at 04:34, Marco Leise via Digitalmars-d
>  wrote:
> > JavaScript's canvas works that way for
> > example. I.e. the only pixel format is RGBA for simplicity's
> > sake and I'm not surprised it actually draws something if I
> > load it with a 24-bit graphic. ;)  
> 
> Given this example, isn't it the job of the image loader to populate
> the image with data?2
> […]

I admit is was a constructed example. You can't load an image
directly into a HTML5 canvas. Shame on me.

> You'll notice I didn't add arithmetic operators to the HSx type ;)
> If you have HSx colors,  and want to do arithmetic, cast it to RGB first.

Now that you say it, yes I was wondering how the arithmetics
work since I couldn't find any opBinary.
 
> I went through a brief phase where I thought about adding an angle
> type (beside normint), but I felt it was overkill.
> I still wonder if it's the right thing to do though... some type that
> understands a circle, and making angle arithmetic work as expected.

I see. I'm also wondering what equality means for a color with
floating point hue 10° and another with hue 730°. I guess it
is ok the way it is now, because wrapping the values doesn't
guarantee they map to the same value either. Floating point
equality is flawed either way.

-- 
Marco

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 14 September 2016 at 04:10, Random D user via Digitalmars-d
 wrote:
>
> In general, I think basics should be dead simple (even over simplified for
> the very basic case)

I think they are:

  import std.experimental.color;
  RGB8 color("red");

I don't think i's possible to make it simpler than that?


> but the API and the docs should provide me layers of
> increasing detail and sophistication, which I can study if (or when) I
> really need control and care about the details.

I'll work on the docs some more.


> Deprecated by who? Shouldn't phobos grade lib include all reasonable
> platforms?

They're all comprehensively supported, I just didn't pre-instantiate
every possible format imaginable. That would be devastating to compile
times, and have absolutely no value.

BGR really just exists because WINAPI. Microsoft haven't recommended
people write GDI based software for decades.
Look at DX10(,11,12), BGR isn't even supported in modern DirectX.
Microsoft left BGR to die years ago (and good riddance).
That said, there is an instantiation for BGR in package.d... the
reversed versions (ie, alpha first) are completely unusual; just some
big-endian games consoles.

Modern formats have the channels in order; RGB, this is true for all
data types, ubyte, short, float, etc.


> I agree that you probably don't see too much variation within windows APIs,
> images (BMP etc.) or with D3D GPU textures (they're compressed anyway and
> asset pipelines to do the conversions beforehand), but I was more of
> thinking image libraries and the numerous, sometimes old and quirky, image
> formats. Or perhaps someone wants to implement a software blitter for their
> favorite toy device/embedded project.

"favorite toy device/embedded project" <- this is the definition of
specialist task. I don't mind if that person has to type 'alias' at
the top of their app.


>>> For 16 bits fairly common are:
>>> RGB565 and RGBA5551, also sometimes you see one of RGBA permutations
>>> (like RGBA8 above).
>>
>>
>> Nobody uses 16bit textures anymore. Compressed textures are both MUCH
>> smaller, and generally higher quality.
>
>
> Sure, agreed. These are not too useful as GPU textures these days (even on
> mobile), but if you do software 2d, load older image formats (image viewer
> etc.) or something even more specialized, these are pretty useful.
>
> I guess I was going for comprehensive (within reason), where as you were
> going for "90% of 2016 colors/image data", which is fine.

There is comprehensive support for all these formats... I just didn't
pre-instantiate the templates.
It's not hard to do at the top of your program:
  alias MyRGBType = RGB!(...details...);


> Anyways, it would be nice to see color (and hopefully image + simple
> blitter) in phobos.

I think that'll come. This is a pre-requisite.


> There's no need to write the same convert/copy/stretch/blend loop over and
> over again. And cpu-side code is nice to have for quick prototyping and
> small-scale work.

Sure.


>>> Just as a completely random idea - How about aliases for HDR formats like
>>> HDR10 and Dolby Vision? Kind of looks like they're just combinations what
>>> you already have.
>>
>>
>> This is very specialist. Instantiations are expensive. If they're not
>> used, it's a waste of compiler effort.
>
>
> HDR TVs and displays are coming fast and with that I'm sure HDR editing and
> photography will be common, which of course means HDR color formats and
> apps. But these can be added later.

There is already support for HDR.
Use a floating point RGB with UHDTV colour space: alias HDR =
RGB!("rgb", float, linear(?), RGBColorSpace.UHDTV);
CompressedRGB also has comprehensive support for all the HDR formats
I've ever encountered.

My point was, I don't want to provide instantiations for these
types... most people don't need it. People writing HDR rendering
software will instantiate the types they need themselves. It's very
specialist work.

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 14 September 2016 at 04:25, Random D user via Digitalmars-d
 wrote:
> On Tuesday, 13 September 2016 at 02:00:44 UTC, Manu wrote:
>>
>> On 13 September 2016 at 07:00, Marco Leise via Digitalmars-d
>>  wrote:
>>>
>>> Am Tue, 13 Sep 2016 00:37:22 +1000
>>> schrieb Manu via Digitalmars-d :
>>> Alright, but hybrid gamma is really not something that can be googled. Or
>>> rather I end up at Toyota's Gamma Hybrid product page. :D
>>
>>
>> True. I'm not even sure what the technical term for this sort of gamma
>> function is... I just made that up! :/
>> As Walter and others have asked, I'll have to start adding links to
>> reference material I guess, although that still feels really weird to
>> me for some reason.
>
>
> FWIW I stumbled into this while double-checking HDR standards for my
> previous post. (I'm not a HDR expert, only somewhat interested because it's
> the future of displays/graphics)
>
> https://en.wikipedia.org/wiki/Hybrid_Log-Gamma

Perfect, thanks!

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 14 September 2016 at 04:34, Marco Leise via Digitalmars-d
 wrote:
> Am Tue, 13 Sep 2016 12:00:44 +1000
> schrieb Manu via Digitalmars-d :
>
>> What is the worth of storing alpha data if it's uniform 0xFF anyway?
>> It sounds like you mean rgbx, not rgba (ie, 32bit, but no alpha).
>> There should only be an alpha channel if there's actually alpha data... 
>> right?
>
> I don't mean RGBX. JavaScript's canvas works that way for
> example. I.e. the only pixel format is RGBA for simplicity's
> sake and I'm not surprised it actually draws something if I
> load it with a 24-bit graphic. ;)

Given this example, isn't it the job of the image loader to populate
the image with data?
If you ask for an RGBA image loaded from a 24bit BMP or something,
sure, it should put 0xFF in there, but I don't know if it's right that
that logic belongs here...?

>> > […] An additive one may be:
>> >
>> >   color = color_dst + color_src * alpha_src
>> >   alpha = alpha_dst
>>
>> I thought about adding blend's, but I stopped short on that. I think
>> that's firmly entering image processing territory, and I felt that was
>> one step beyond the MO of this particular lib... no?
>> Blending opens up a whole world.
>
> I agree with that decision, and that it entails that
> arithmetics are undefined for alpha channels. :-( Yeah bummer.
> The idea that basic (saturating) arithmetics work on colors is
> a great simplification that works for the most part, but let's
> be fair, multiplying two HSV colors isn't exactly going to
> yield a well defined hue either,

You'll notice I didn't add arithmetic operators to the HSx type ;)
If you have HSx colors,  and want to do arithmetic, cast it to RGB first.


> just as multiplying two
> angles doesn't give you a new angle. I.e.:
> http://math.stackexchange.com/a/47880

I went through a brief phase where I thought about adding an angle
type (beside normint), but I felt it was overkill.
I still wonder if it's the right thing to do though... some type that
understands a circle, and making angle arithmetic work as expected.
I like my solution of not providing arithmetic operators for HSx and
LCh for now though ;)


>> > […]
>> […]
>> From which functions? Link me?
>> I'd love to see more precedents.
>
> Yep, that's better than arguing :)
> So here are all graphics APIs I know and what they do when
> they encounter colors without alpha and need a default value:
>
> SDL:
> https://wiki.libsdl.org/SDL_MapRGB
> "If the specified pixel format has an alpha component it will be returned as 
> all 1 bits (fully opaque)."
>
> Allegro:
> https://github.com/liballeg/allegro5/blob/master/include/allegro5/internal/aintern_pixels.h#L59
> (No docs, just source code that defaults to 255 for alpha when
> converting a color from a bitmap with non-alpha pixel format
> to an ALLEGRO_COLOR.)
>
> Cairo:
> https://www.cairographics.org/manual/cairo-cairo-t.html#cairo-set-source-rgb
> "Sets the source pattern within cr to an opaque color."
>
> Microsoft GDI+:
> https://msdn.microsoft.com/de-de/library/windows/desktop/ms536255%28v=vs.85%29.aspx
> "The default value of the alpha component for this Color
> object is 255."
>
> Gnome GDK:
> https://developer.gnome.org/gdk-pixbuf/2.33/gdk-pixbuf-Utilities.html#gdk-pixbuf-add-alpha
> "[…] the alpha channel is initialized to 255 (full opacity)."
>
> Qt:
> http://doc.qt.io/qt-4.8/qcolor.html#alpha-blended-drawing
> "By default, the alpha-channel is set to 255 (opaque)."
>
> OpenGL:
> https://www.opengl.org/sdk/docs/man2/xhtml/glColor.xml
> "glColor3 variants specify new red, green, and blue values
> explicitly and set the current alpha value to 1.0 (full
> intensity) implicitly."
> (Note: The color can be queried and shows a=1.0 without
> blending operations setting it internally if needed.)
>
> Java (AWT):
> https://docs.oracle.com/javase/7/docs/api/java/awt/Color.html#Color%28int,%20boolean%29
> "If the hasalpha argument is false, alpha is defaulted to 255."
>
> Apple's Quartz does not seem to provide color space
> conversions and always requires the user to give the alpha
> value for new colors, so there is no default:
> https://developer.apple.com/library/tvos/documentation/GraphicsImaging/Reference/CGColor/index.html#//apple_ref/c/func/CGColorCreate

Touche! ;)


> One thing I noticed is that many of those strictly separate
> color spaces from alpha as concepts. For example in Quartz
> *all* color spaces have alpha. In Allegro color space
> conversions are ignorant of alpha. That begs the question
> what should happen when you convert RGBA to a HLx
> color space and back to RGBA. Can you retain the alpha value?
> CSS3 for example has HSLA colors that raise the bar a bit.

I've actually had the thought that I should remove alpha from RGB, and
instead provide an aggregate colour type where you could make RGBA by
RGB +  alpha, and each element in the aggregate would follow its own
natural arithmetic rules... again, it feels 

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 14 September 2016 at 00:56, John Colvin via Digitalmars-d
 wrote:
> On Tuesday, 13 September 2016 at 09:31:53 UTC, Manu wrote:
>>
>> On 13 September 2016 at 17:47, John Colvin via Digitalmars-d
>>  wrote:
>>>
>>> On Tuesday, 13 September 2016 at 01:05:56 UTC, Manu wrote:
>
>
> Also can I swizzle channels directly?



 I could add something like:
   auto brg =  c.swizzle!"brg";

 The result would be strongly typed to the new arrangement.
>>>
>>>
>>>
>>> Perfect use-case for opDispatch like in gl3n.
>>
>>
>> One trouble with arbitrary swizzling is that people often want to
>> write expressions like "c.xyzy", or whatever, but repeating
>> components... and that's not something my colours can do. It's useful
>> in realtime code, but it doesn't mean anything, and you can't
>> interpret that value as a colour anymore after you do that.
>> This sort of thing is used when you're not actually storing colours in
>> textures, but instead just some arbitrary data. Don't use a colour
>> type for that, use a vector type instead.
>> What that sort of swizzling really is, are vector operations, not
>> colour operations. Maybe I could add an API to populate a vector from
>> the components of colours, in-order... but then we don't actually have
>> linear algebra functions in phobos either! So colour->vectors and
>> arbitrary swizzling is probably not something we need immediately.
>>
>> In my lib, colours are colours. If you have `BGR8 x` and `RGB8 y`, and add
>> them, you don't get x.b+y.r, x.g+y.g, x.r+y.b... that's not a colour
>> operation, that's an element-wise vector operation.
>
>
> Fair enough, you know much better than me how useful it would be. I was just
> suggesting that if you do support some sorts of swizzling then opDispatch
> would allow you to avoid users having to use strings. It would be as simple
> to implement as `alias opDispatch = swizzle;` given the swizzle function you
> were using before.

Oh, it would be super-useful! Just that it's not a colour operation,
its liner algebra... I think it's a job for another lib. Getting a
colour as a vector is all that's needed.

Re: colour lib needs reviewers

[Basile B. via Digitalmars-d]

On Tuesday, 13 September 2016 at 02:00:44 UTC, Manu wrote:
What is the worth of storing alpha data if it's uniform 0xFF 
anyway?


He he, that's the big problem with classic image formats. When a 
graphic is described in term of vector, fill, stroke, gradient, 
layer, etc (not to name SVG) there's no more waste.

Re: colour lib needs reviewers

[Marco Leise via Digitalmars-d]

Am Tue, 13 Sep 2016 12:00:44 +1000
schrieb Manu via Digitalmars-d :

> What is the worth of storing alpha data if it's uniform 0xFF anyway?
> It sounds like you mean rgbx, not rgba (ie, 32bit, but no alpha).
> There should only be an alpha channel if there's actually alpha data... right?

I don't mean RGBX. JavaScript's canvas works that way for
example. I.e. the only pixel format is RGBA for simplicity's
sake and I'm not surprised it actually draws something if I
load it with a 24-bit graphic. ;)

> > […] An additive one may be:
> >
> >   color = color_dst + color_src * alpha_src
> >   alpha = alpha_dst  
> 
> I thought about adding blend's, but I stopped short on that. I think
> that's firmly entering image processing territory, and I felt that was
> one step beyond the MO of this particular lib... no?
> Blending opens up a whole world.

I agree with that decision, and that it entails that
arithmetics are undefined for alpha channels. :-( Yeah bummer.
The idea that basic (saturating) arithmetics work on colors is
a great simplification that works for the most part, but let's
be fair, multiplying two HSV colors isn't exactly going to
yield a well defined hue either, just as multiplying two
angles doesn't give you a new angle. I.e.:
http://math.stackexchange.com/a/47880

> > […]
> […]
> From which functions? Link me?  
> I'd love to see more precedents.

Yep, that's better than arguing :)
So here are all graphics APIs I know and what they do when
they encounter colors without alpha and need a default value:

SDL:
https://wiki.libsdl.org/SDL_MapRGB
"If the specified pixel format has an alpha component it will be returned as 
all 1 bits (fully opaque)."

Allegro:
https://github.com/liballeg/allegro5/blob/master/include/allegro5/internal/aintern_pixels.h#L59
(No docs, just source code that defaults to 255 for alpha when
converting a color from a bitmap with non-alpha pixel format
to an ALLEGRO_COLOR.)

Cairo:
https://www.cairographics.org/manual/cairo-cairo-t.html#cairo-set-source-rgb
"Sets the source pattern within cr to an opaque color."

Microsoft GDI+:
https://msdn.microsoft.com/de-de/library/windows/desktop/ms536255%28v=vs.85%29.aspx
"The default value of the alpha component for this Color
object is 255."

Gnome GDK:
https://developer.gnome.org/gdk-pixbuf/2.33/gdk-pixbuf-Utilities.html#gdk-pixbuf-add-alpha
"[…] the alpha channel is initialized to 255 (full opacity)."

Qt:
http://doc.qt.io/qt-4.8/qcolor.html#alpha-blended-drawing
"By default, the alpha-channel is set to 255 (opaque)."

OpenGL:
https://www.opengl.org/sdk/docs/man2/xhtml/glColor.xml
"glColor3 variants specify new red, green, and blue values
explicitly and set the current alpha value to 1.0 (full
intensity) implicitly."
(Note: The color can be queried and shows a=1.0 without
blending operations setting it internally if needed.)

Java (AWT):
https://docs.oracle.com/javase/7/docs/api/java/awt/Color.html#Color%28int,%20boolean%29
"If the hasalpha argument is false, alpha is defaulted to 255."

Apple's Quartz does not seem to provide color space
conversions and always requires the user to give the alpha
value for new colors, so there is no default:
https://developer.apple.com/library/tvos/documentation/GraphicsImaging/Reference/CGColor/index.html#//apple_ref/c/func/CGColorCreate


One thing I noticed is that many of those strictly separate
color spaces from alpha as concepts. For example in Quartz
*all* color spaces have alpha. In Allegro color space
conversions are ignorant of alpha. That begs the question
what should happen when you convert RGBA to a HLx
color space and back to RGBA. Can you retain the alpha value?
CSS3 for example has HSLA colors that raise the bar a bit.

-- 
Marco

Re: colour lib needs reviewers

[Random D user via Digitalmars-d]

On Tuesday, 13 September 2016 at 02:00:44 UTC, Manu wrote:
On 13 September 2016 at 07:00, Marco Leise via Digitalmars-d 
 wrote:

Am Tue, 13 Sep 2016 00:37:22 +1000
schrieb Manu via Digitalmars-d :
Alright, but hybrid gamma is really not something that can be 
googled. Or rather I end up at Toyota's Gamma Hybrid product 
page. :D


True. I'm not even sure what the technical term for this sort 
of gamma

function is... I just made that up! :/
As Walter and others have asked, I'll have to start adding 
links to
reference material I guess, although that still feels really 
weird to

me for some reason.


FWIW I stumbled into this while double-checking HDR standards for 
my previous post. (I'm not a HDR expert, only somewhat interested 
because it's the future of displays/graphics)


https://en.wikipedia.org/wiki/Hybrid_Log-Gamma

Re: colour lib needs reviewers

[Random D user via Digitalmars-d]

On Tuesday, 13 September 2016 at 01:05:56 UTC, Manu wrote:


Can you describe what  you  perceive to be hard?


Well, I just skimmed through the docs and I didn't look at the 
code, so that sense it was an "honest" view for phobos proposal. 
Also I was trying to convey that based on the docs it "looks 
hard", although I was suspecting that it isn't really.


So the list of things was a list of examples that I would've 
wanted to see in the docs. Maybe in an overview or "here's how 
you use it" page. I can read the details if I really need them, 
but I think it's important to have a cook book examples for quick 
start.


In general, I think basics should be dead simple (even over 
simplified for the very basic case), but the API and the docs 
should provide me layers of increasing detail and sophistication, 
which I can study if (or when) I really need control and care 
about the details.



Few basic things I'd be interested in (examples):
1. How to pack/unpack color into uint/rgba channels.


auto c  = RGBA8(r, g, b, a);
uint r = c.r.value; // 'r' is a 'NormalizedInt", so you need 
'.value'

to get the raw integer value
float g = cast(float)c.g; // cast to float knows about the 
normalized 0-1 range


Let's put these in the docs.




Also can I swizzle channels directly?


I could add something like:
  auto brg =  c.swizzle!"brg";


It's just a nicety. I guess I could just define exact color 
formats and convert (let the lib do the rest), or just unpack and 
repack.


Also the docs might want to mention which way is the 
preferred/designed way.


2. Can I just cast an array of bytes into a RGBA array and 
operate on that?


Yes. Of course, if the image library returned an array of 
colors, you wouldn't have an array of raw bytes in the first 
place.


Cool. I just wanted to make sure something like this possible.

Also should be in the docs as an example, so that people don't 
have to try this out themselves.

Or search through github for details.



3. How to scale color channels e.g. 5 bits to 8 bits and vice 
versa. How to convert color formats? Can I combine 
scale/conversion with single channel or channel swizzling?


alias CustomPackedColor = PackedRGB!("BGR_5_6_5", ubyte); // <- 
ordered bgr

CustomPackedColor[] image = loadPackedImage();

auto unpacked = cast(RGB8)image[0]; // unpack and swizzle

I'm not sure what you mean by "with single channel"?


For example if you first unpack RGBA_uint32 into R,G,B,A then you 
want to take G (or just swizzle color.g) and scale that to 5 bits 
or 16 bits for what ever use case you might have.


Looks to me like this can be handled with something like 
PackedRGB!("G_5") and then do color conversion into that.




4. Which way are RGBA format names defined ("byte array order" 
r is first byte or "uint order" r is in high bits)?


I describe formats in byte (bit) order. It's the only 
reasonable way

that works outside of a uint.


Agreed. This should be in the docs, so that it's clear to users. 
(especially, since RGBA is the common case and it does fit 
into uint)




I'm showing that parsing colors from string's follow the web
convention. I can't redefine how colours are expressed on the 
web for

decades.
I think you've just highlighted that I need to remove the '0x'
notation though, that is definitely misleading! Use '#', that 
should

be clear.


Right. I was interpreting 0x as c hex literal. You should 
document that it's following the web convention.


6. Perhaps you should define all the usual color formats, 
since everybody

and their cat has their own preference for rgba channel orders.
In my experience all these 4 variants for RGBA are pretty 
common:

RGBA
BGRA
ARGB
ABGR
and their X and 24bit variants.


I don't think they're common at all. Only 2 are common (one of 
which

is deprecated), the other 2 are very niche.


Deprecated by who? Shouldn't phobos grade lib include all 
reasonable platforms?


I agree that you probably don't see too much variation within 
windows APIs, images (BMP etc.) or with D3D GPU textures (they're 
compressed anyway and asset pipelines to do the conversions 
beforehand), but I was more of thinking image libraries and the 
numerous, sometimes old and quirky, image formats. Or perhaps 
someone wants to implement a software blitter for their favorite 
toy device/embedded project.


I was trying to balance a tasteful amount of instantiations. I 
don't

really want to provide an instantiation of every single realtime
format I've ever encountered out of the box.

Seems reasonable.




For 16 bits fairly common are:
RGB565 and RGBA5551, also sometimes you see one of RGBA 
permutations

(like RGBA8 above).


Nobody uses 16bit textures anymore. Compressed textures are 
both MUCH smaller, and generally higher quality.


Sure, agreed. These are not too useful as GPU textures these days 
(even on mobile), but if you do software 2d, load older image 
formats (image viewer etc.) or something even 

Re: colour lib needs reviewers

[John Colvin via Digitalmars-d]

On Tuesday, 13 September 2016 at 09:31:53 UTC, Manu wrote:
On 13 September 2016 at 17:47, John Colvin via Digitalmars-d 
 wrote:

On Tuesday, 13 September 2016 at 01:05:56 UTC, Manu wrote:


Also can I swizzle channels directly?



I could add something like:
  auto brg =  c.swizzle!"brg";

The result would be strongly typed to the new arrangement.



Perfect use-case for opDispatch like in gl3n.


One trouble with arbitrary swizzling is that people often want 
to

write expressions like "c.xyzy", or whatever, but repeating
components... and that's not something my colours can do. It's 
useful

in realtime code, but it doesn't mean anything, and you can't
interpret that value as a colour anymore after you do that.
This sort of thing is used when you're not actually storing 
colours in
textures, but instead just some arbitrary data. Don't use a 
colour

type for that, use a vector type instead.
What that sort of swizzling really is, are vector operations, 
not
colour operations. Maybe I could add an API to populate a 
vector from
the components of colours, in-order... but then we don't 
actually have
linear algebra functions in phobos either! So colour->vectors 
and
arbitrary swizzling is probably not something we need 
immediately.


In my lib, colours are colours. If you have `BGR8 x` and `RGB8 
y`, and add them, you don't get x.b+y.r, x.g+y.g, x.r+y.b... 
that's not a colour operation, that's an element-wise vector 
operation.


Fair enough, you know much better than me how useful it would be. 
I was just suggesting that if you do support some sorts of 
swizzling then opDispatch would allow you to avoid users having 
to use strings. It would be as simple to implement as `alias 
opDispatch = swizzle;` given the swizzle function you were using 
before.

Re: colour lib needs reviewers

[Guillaume Piolat via Digitalmars-d]

On Tuesday, 13 September 2016 at 09:31:53 UTC, Manu wrote:


In my lib, colours are colours. If you have `BGR8 x` and `RGB8 
y`, and add them, you don't get x.b+y.r, x.g+y.g, x.r+y.b... 
that's not a colour operation, that's an element-wise vector 
operation.


I think swizzling is way overrated, especially swizzling 
assignment.



Which is the most readable?

color.rb = color.br; // swizzling
swap(color.r, color.b);  // no swizzling

auto color = RGBA8(c.rgb, 255); // swizzling
auto color = RGBA8(c.r, c.g, c.b, 255); // no swizzling

And finally:

auto rb = color.rgrb; // When do you ever need that?

Re: colour lib needs reviewers

[Guillaume Piolat via Digitalmars-d]

On Monday, 12 September 2016 at 22:51:14 UTC, Walter Bright wrote:


My last comment is what can I do with a color library that 
would not require some additional library?


Having correct conversions.

Converting sRGB to linear RGB is typically not done correctly, 
pow isn't enough.
It also dissipates most misunderstanding about color space (yes 
Lab also are dependent on the color point, it was something I 
wasn't sure about and this library says it clearly).


Right now I rely on the OS for Apple RGB to sRGB conversion, this 
library would allow me to do it without using a system call.


Other uses I can imagine:
- Downsampling an image preserving colors (compressed RGB isn't 
necessarily the right sep to do those)

- Perceptually correct color gradients. In font rendering,
- HSV is very useful for colouring stuff in video games, it's 
what the color selection does in painter-programs like GIMP. So 
lots of library out there have a HSV <-> RGB conversions
- converting to packed RGB is useful for programs using GPUs. 
Same reason than half-float support is useful.

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 13 September 2016 at 20:19, ketmar via Digitalmars-d
 wrote:
> On Tuesday, 13 September 2016 at 02:50:02 UTC, Manu wrote:
>>
>> std.conv.to  seems to 'just work' for color type conversion.
>> I'd like to support string -> color conversion via std.conv.to; ie:
>> auto c = "#00".to!RGB8;
>> Is there a simple way to do that?
>
>
> yes. just define a ctor that accepts string, and you're there. it works both
> for classes and structs.

Oh... it already works! :P

Re: colour lib needs reviewers

[Andrea Fontana via Digitalmars-d]

On Monday, 12 September 2016 at 22:51:14 UTC, Walter Bright wrote:


My last comment is what can I do with a color library that 
would not require some additional library?


I think it's easy to do some image processing using netbpm [1] as 
format.

Reading/Writing images doesn't require any other library.

And using ndslices a lot of things can be done :)

Andrea

[1] https://en.wikipedia.org/wiki/Netpbm_format

Re: colour lib needs reviewers

[ketmar via Digitalmars-d]

On Tuesday, 13 September 2016 at 02:50:02 UTC, Manu wrote:

std.conv.to  seems to 'just work' for color type conversion.
I'd like to support string -> color conversion via std.conv.to; 
ie:

auto c = "#00".to!RGB8;
Is there a simple way to do that?


yes. just define a ctor that accepts string, and you're there. it 
works both for classes and structs.

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 9/12/16 10:50 PM, Manu via Digitalmars-d wrote:

std.conv.to  seems to 'just work' for color type conversion.
I'd like to support string -> color conversion via std.conv.to; ie:
auto c = "#00".to!RGB8;
Is there a simple way to do that?


That was the purpose of toImpl, which I recall has been recently 
removed... -- Andrei

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 13 September 2016 at 17:47, John Colvin via Digitalmars-d
 wrote:
> On Tuesday, 13 September 2016 at 01:05:56 UTC, Manu wrote:
>>>
>>> Also can I swizzle channels directly?
>>
>>
>> I could add something like:
>>   auto brg =  c.swizzle!"brg";
>>
>> The result would be strongly typed to the new arrangement.
>
>
> Perfect use-case for opDispatch like in gl3n.

One trouble with arbitrary swizzling is that people often want to
write expressions like "c.xyzy", or whatever, but repeating
components... and that's not something my colours can do. It's useful
in realtime code, but it doesn't mean anything, and you can't
interpret that value as a colour anymore after you do that.
This sort of thing is used when you're not actually storing colours in
textures, but instead just some arbitrary data. Don't use a colour
type for that, use a vector type instead.
What that sort of swizzling really is, are vector operations, not
colour operations. Maybe I could add an API to populate a vector from
the components of colours, in-order... but then we don't actually have
linear algebra functions in phobos either! So colour->vectors and
arbitrary swizzling is probably not something we need immediately.

In my lib, colours are colours. If you have `BGR8 x` and `RGB8 y`, and
add them, you don't get x.b+y.r, x.g+y.g, x.r+y.b... that's not a
colour operation, that's an element-wise vector operation.

Re: colour lib needs reviewers

[John Colvin via Digitalmars-d]

On Tuesday, 13 September 2016 at 01:05:56 UTC, Manu wrote:

Also can I swizzle channels directly?


I could add something like:
  auto brg =  c.swizzle!"brg";

The result would be strongly typed to the new arrangement.


Perfect use-case for opDispatch like in gl3n.

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 13 September 2016 at 12:05, Walter Bright via Digitalmars-d
 wrote:
> On 9/12/2016 2:08 PM, John Colvin wrote:
>>
>> On Monday, 12 September 2016 at 19:59:58 UTC, Edwin van Leeuwen wrote:
>>>
>>> On Monday, 12 September 2016 at 19:55:57 UTC, Guillaume Piolat wrote:

 - I've wanted a function like colorFromString many times. It's
 especially
 nice with the added #RGBA and #RRGGBBAA syntax that eg. SVG lacks.
>>>
>>>
>>> What happens when the string is invalid? Does it throw an error?
>>
>>
>> It should be integrated with std.conv.parse and std.conv.to and follow
>> their
>> conventions.
>
>
> I'm not so sure. Follow the conventions, ok, but integrating them in - they
> are already large and complex.

std.conv.to  seems to 'just work' for color type conversion.
I'd like to support string -> color conversion via std.conv.to; ie:
auto c = "#00".to!RGB8;
Is there a simple way to do that?

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/12/2016 7:03 PM, Walter Bright wrote:

On 9/12/2016 6:11 PM, Manu via Digitalmars-d wrote:

Okay. I'll work on this feedback.


Wonderful!


BTW, linking to authoritative reference material has the effect of reassuring 
the user that the library is designed by someone who knows his stuff. It has the 
same effect as notes and references appendices have in any good technical book.

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/12/2016 2:08 PM, John Colvin wrote:

On Monday, 12 September 2016 at 19:59:58 UTC, Edwin van Leeuwen wrote:

On Monday, 12 September 2016 at 19:55:57 UTC, Guillaume Piolat wrote:

- I've wanted a function like colorFromString many times. It's especially
nice with the added #RGBA and #RRGGBBAA syntax that eg. SVG lacks.


What happens when the string is invalid? Does it throw an error?


It should be integrated with std.conv.parse and std.conv.to and follow their
conventions.


I'm not so sure. Follow the conventions, ok, but integrating them in - they are 
already large and complex.

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/12/2016 6:11 PM, Manu via Digitalmars-d wrote:

Okay. I'll work on this feedback.


Wonderful!



My last comment is what can I do with a color library that would not require
some additional library?


With color, ndslice, and std.algorithm, you can do quite a lot of
useful image processing already.


Ok, but I have no idea how. Perhaps include something as an example? Phobos in 
general lacks short, pithy, and *useful* examples. Such can go a long way 
towards selling the color library to users.




But my point before was that I see this primarily as an enabler. We
really need a graph/plot library for instance! Also, multimedia
libraries will all become naturally compatible when the primitive
working type is defined in phobos.


I had read your post on this topic after making my post. I think it's a strong 
answer.

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 13 September 2016 at 07:00, Marco Leise via Digitalmars-d
 wrote:
> Am Tue, 13 Sep 2016 00:37:22 +1000
> schrieb Manu via Digitalmars-d :
>> I flip-flopped on this multiple times.
>> It's not so simple.
>> 1. Alpha doesn't necessarily mean transparently, that's just one
>> (extremely common) use
>> 2. 1 is the multiplication identity, but 0 is the additive identity. I
>> cant find either addition or multiplication to take precedence over
>> eachother.
>> 3. With those 2 points above already making me feel a bit worried
>> about it, I think that appearing numbers out of nowhere is just
>> generally bad, so I went with a=0.
>
>> If alpha defaults to 1, then a*b works as expected (object does not
>> become transparent by this operation).
>> If alpha defaults to 0, then b-a works as expected (object does not
>> become transparent by this operation).
>
> Additive color components can work like that, but the alpha
> component just doesn't represent an amount of light. Don't try
> too hard to make the arithmetics make sense, when the
> practical use lies in 100% alpha by default.

I understand the argument, I have stressed over this to no end.
I still think it's just not 'right'. :/


> Converting every
> color to full transparent is not helpful when you want to save
> a 24-bit BMP as 32-bit PNG or get an RGBA color from an RGB
> texture.

What is the worth of storing alpha data if it's uniform 0xFF anyway?
It sounds like you mean rgbx, not rgba (ie, 32bit, but no alpha).
There should only be an alpha channel if there's actually alpha data... right?


> I don't know who declared that 100% alpha means opaque. For
> what it's worth it could have been the other way around and
> all the (1-a) and (a) in blending functions swapped.
> And a blending function is what is needed here to give any
> meaning at all to arithmetics on alpha. An additive one may be:
>
>   color = color_dst + color_src * alpha_src
>   alpha = alpha_dst

I thought about adding blend's, but I stopped short on that. I think
that's firmly entering image processing territory, and I felt that was
one step beyond the MO of this particular lib... no?
Blending opens up a whole world.


>> With that in mind, you realise alpha will *always* require explicit
>> handling... so I think 0 is a better default at that stage. It may be
>> possible to convince me otherwise ;)
>
> I fully agree with the first line, I just think that for
> practical uses alpha is considered to be 100% in pixel formats
> without alpha channel.

'rgbx' is for image formats without alpha channels...


> Otherwise the most popular blending
> functions would do nothing on RGB colors.

'rgbx' colors fed through a blend function would always properly imply alpha=1.


> SDL for example also uses 100% alpha by default: "If the
> specified pixel format has an alpha component it will be
> returned as all 1 bits (fully opaque)."

>From which functions? Link me?
I'd love to see more precedents.


>> > It seems inconsistent that you can specify all kinds of
>> > exponents when the exponent is shared but are forced to 5 bit
>> > exponents when mixed into each component, while the mantissa
>> > can still be adjusted. Would m7e3 have worked? (With f* being
>> > the IEEE standard formats.)
>>
>> I was thinking that, and mentioned as such in the comments near those lines 
>> ;)
>> Thing is, there are no realtime small floats with exponent != 5bits
>> except the unique 7e3 format on xbox 360.
>> I can extend this in an unbreaking way.
>> I might: f## =  ieee-like, s##e# = signed float, (u)##e# = unsgned float.
>> It's just pretty verbose is all: "rgb_11e5_11e5_10e5". I mean, it's
>> fine... but maybe people wouldn't find this intuitive? Many
>> programmers don't know what an 'exponent' or 'mantissa' is... :/
>
> Yeah I thought that the common formats would be the IEEE
> ones, especially f16 and that anything less are very niche
> custom formats that come and go and need verbose descriptions.
> Well, either way works.

The verbose idea I said above is my best idea. I'll roll with that if
enough people agree.
There was the other fellow who posted before who feels like it's
already too complicated for him though.

I've been very carefully trying to put the complicated bits just one
level back from arms reach, so a casual user isn't confronted with
those details.
Putting that complexity in the format string makes it very
user-facing. That's the part I'm not wild about.


>> > The final package should contain a fair amount of
>> > documentation, in my opinion also on the topics of sRGB vs.
>> > linear and what "hybrid gamma" is.
>>
>> Yeah, but is it really my job to teach someone about colour space?
>> Wikipedia can do that much better than I can... :/
>> I agree, but I don't really know where to draw the line, or how to organise 
>> it.
>>
>> Perhaps people with strong opinions on how this should be presented
>> can create a PR?
>
> Alright, 

Re: colour lib needs reviewers

[jmh530 via Digitalmars-d]

On Monday, 12 September 2016 at 22:55:01 UTC, Walter Bright wrote:


Thanks for the rationale. I'm glad to see you're planning 
follow-on libraries! I'd use a graph/plot library myself. Some 
of the rationale above should go into the documentation for the 
library.


You might try
https://github.com/BlackEdder/ggplotd

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 13 September 2016 at 08:51, Walter Bright via Digitalmars-d
 wrote:
> On 9/11/2016 9:14 PM, Manu via Digitalmars-d wrote:
>>
>> I think I'm about as happy with my colour lib as I'm going to be.
>> It really needs reviews.
>
>
> I have zero expertise in color programming, and have no idea what a color
> library might do or what it would be used for.
>
> I looked at the front page of the documentation. I would like to see some
> sort of explanation of who, what, where, why and how of a color library.
> Like what is a color space, etc. Maybe add some links to explanatory
> Wikipedia pages?
>
> "Set of colors defined by X11, adopted by the W3C, SVG, and other popular
> libraries."
>
> Would be a great place to add clickable links for the relevant X11, W3C,
> SVG, etc. specification pages.
>
> It really, really needs a "References:" section, with links like:
>
> https://en.wikipedia.org/wiki/RGB_color_model
>
> Are there any definitive books on the topic?
>
> What is a BGR color type? What is a UV color type? What is a UVW color type?
> What is an alpha color type? What is luminance?
>
> "This module implements HSV, HSL, HSI, HCY, HWB, HCG color types."
> "angular color spaces"
>
> Some references to what these are would be nice.
>
> Contrast with:
>
> http://dlang.org/phobos/std_regex.html
>
> where general information is given and links for more are provided (although
> that is weak and could be improved).

Okay. I'll work on this feedback.


> My last comment is what can I do with a color library that would not require
> some additional library?

With color, ndslice, and std.algorithm, you can do quite a lot of
useful image processing already.
But my point before was that I see this primarily as an enabler. We
really need a graph/plot library for instance! Also, multimedia
libraries will all become naturally compatible when the primitive
working type is defined in phobos.

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 13 September 2016 at 06:29, Random D user via Digitalmars-d
 wrote:
> On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
>>
>> I think I'm about as happy with my colour lib as I'm going to be. It
>> really needs reviews.
>>
>> - Manu
>
>
> Hi. I'm just a random forum lurker, but here's my feedback.
>
> It needs more docs/examples for the basic usage cases (i.e. how to skip docs
> and start working).
>
> Now at first glance it looks like working with RGBA (most common case (with
> srgb)) is complicated and while reading the docs I kind of felt like I'd
> just write my own.

You'll get it wrong though!

Can you describe what  you  perceive to be hard?

  import std.experimental.color;

  RGBA8 a(0xFF808000);
  RGBA8 b("GoldenRodYellow");
  auto r = a + b; // do normal operations

Or:
  auto image = new RGBA8[height][width];
  image[x][y] = RGBA8("Aquamarine");


> Few basic things I'd be interested in (examples):
> 1. How to pack/unpack color into uint/rgba channels.

auto c  = RGBA8(r, g, b, a);
uint r = c.r.value; // 'r' is a 'NormalizedInt", so you need '.value'
to get the raw integer value
float g = cast(float)c.g; // cast to float knows about the normalized 0-1 range

> Also can I swizzle channels directly?

I could add something like:
  auto brg =  c.swizzle!"brg";

The result would be strongly typed to the new arrangement.

> How would I extract a single channel from color?

uint r = c.r.value; // like this?

> 2. Can I just cast an array of bytes into a RGBA array and operate on that?

Yes. Of course, if the image library returned an array of colors, you
wouldn't have an array of raw bytes in the first place.

> 3. How to scale color channels e.g. 5 bits to 8 bits and vice versa. How to
> convert color formats? Can I combine scale/conversion with single channel or
> channel swizzling?

alias CustomPackedColor = PackedRGB!("BGR_5_6_5", ubyte); // <- ordered bgr
CustomPackedColor[] image = loadPackedImage();

auto unpacked = cast(RGB8)image[0]; // unpack and swizzle

I'm not sure what you mean by "with single channel"?

> 4. Which way are RGBA format names defined ("byte array order" r is first
> byte or "uint order" r is in high bits)?

I describe formats in byte (bit) order. It's the only reasonable way
that works outside of a uint.

> 5. Is it the same for packed rgba formats?

Yes.

> I'm not sure if packedrgb is a necessary distinction. Aren't they all kind
> of packed (small structs)? There's no guarantee that 1 byte is not just a
> part of a bigger field.

RGBA8 has 4 ubytes, and you can operate on them.
  ubyte a, b; a += b; // <- is valid D code
Unpacked types can do maths. Packed types can't do maths, they need to
be unpacked before you can do anything, and you don't generally want
the result to be immediately packed again.
If I allowed maths on PackedRGB for instance, it would be expected
that the result is re-packed at the end of every operation, and that
would be useless.

PackedRGB is for storage, not for doing work. This is why they're distinct.

> Btw. Your RGBA8 example is a bit weird:
> static assert(RGBA8("0x908000FF") == RGBA8(0x80,0x00,0xFF,0x90));
> I'd assume RGBA8(r,g,b,a) would be either "uint order" r(0x90), g(0x80),
> b(0x00), a(0xFF) or "byte array order" (le) r(0xFF), g(0x00), b(0x80),
> a(0x90)

I'm showing that parsing colors from string's follow the web
convention. I can't redefine how colours are expressed on the web for
decades.
I think you've just highlighted that I need to remove the '0x'
notation though, that is definitely misleading! Use '#', that should
be clear.

> 6. Perhaps you should define all the usual color formats, since everybody
> and their cat has their own preference for rgba channel orders.
> In my experience all these 4 variants for RGBA are pretty common:
> RGBA
> BGRA
> ARGB
> ABGR
> and their X and 24bit variants.

I don't think they're common at all. Only 2 are common (one of which
is deprecated), the other 2 are very niche.
I was trying to balance a tasteful amount of instantiations. I don't
really want to provide an instantiation of every single realtime
format I've ever encountered out of the box.


> For 16 bits fairly common are:
> RGB565 and RGBA5551, also sometimes you see one of RGBA permutations
> (like RGBA8 above).

Nobody uses 16bit textures anymore. Compressed textures are both MUCH
smaller, and generally higher quality.

> Also L16 is pretty common for heightfields or other data images/textures.

I could probably put that one.

> I guess there are also planar formats, but I think those are more of a Image
> problem space.

I think if people want to provide instantiations for convenience, it
should be in the libs that interact with those formats.

For instance, I imagine in OpenGL binding there might be:
  alias C = ColorFor!GL_R11F_G11F_B10F; // <- instantiation of the
proper type for the GL enum.

Most people really just want RGB8, or RGBA8. If you're interacting
with 

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 9/12/16 6:51 PM, Walter Bright wrote:

My last comment is what can I do with a color library that would not
require some additional library?


Great question and well asked. It was mine as well but I could not put 
it as succinctly.  -- Andrei

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/12/2016 8:06 AM, Manu via Digitalmars-d wrote:

So, the main reason I wanted to push for this in phobos is because
it's an enabler.
There is no multimedia in phobos at all. I think that's a shame, and
this is a starting point. Any conceivable image based work depends on
this as foundation. If pixel format types are phobos-supplied, then D
image libraries will naturally be compatible, as opposed to naturally
incompatible since every image library will otherwise re-implement
their own colour types (and almost always incorrectly).
The thing about colours is, they're HARD to get right, but more
importantly, most programmers have no idea that it's actually a hard
problem and won't tend to go looking for a lib for a colour type (Ie,
struct { ubyte r, g, b;}... simple! *cough*). They probably will use
it if it's in phobos though for compatibility reasons.
Image processing is super easy by contrast.

One of my first port-of-calls after this work would be a graph/plot
library... I'd use the hell out of that. I constantly want to output
data visualisations!

It also enables seamless interaction with realtime rendering api's.
Ie, if OpenGL/D3D/Vulkan bindings support phobos supplied colour
types, then image decoding libraries, and other sources of image data
become easy to render without any requirement for adaptation.
Rendering libs and image/video data source libs are almost always
separate, independent libraries. We need to maximise probability that
they are compatible out of the gate.

I didn't want to touch 'images' in this work, because there are
countless subjective designs to present those API's, but I don't think
there's many competing designs for a colour library, so it felt it was
the best and most bang-for-buck starting place overall.

Also, I think this has very high synergy with ndslice. I think ndslice
+  color is the foundation for image processing, or at least image
data transport.


Thanks for the rationale. I'm glad to see you're planning follow-on libraries! 
I'd use a graph/plot library myself. Some of the rationale above should go into 
the documentation for the library.

Re: colour lib needs reviewers

[Walter Bright via Digitalmars-d]

On 9/11/2016 9:14 PM, Manu via Digitalmars-d wrote:

I think I'm about as happy with my colour lib as I'm going to be.
It really needs reviews.


I have zero expertise in color programming, and have no idea what a color 
library might do or what it would be used for.


I looked at the front page of the documentation. I would like to see some sort 
of explanation of who, what, where, why and how of a color library. Like what is 
a color space, etc. Maybe add some links to explanatory Wikipedia pages?


"Set of colors defined by X11, adopted by the W3C, SVG, and other popular 
libraries."


Would be a great place to add clickable links for the relevant X11, W3C, SVG, 
etc. specification pages.


It really, really needs a "References:" section, with links like:

https://en.wikipedia.org/wiki/RGB_color_model

Are there any definitive books on the topic?

What is a BGR color type? What is a UV color type? What is a UVW color type? 
What is an alpha color type? What is luminance?


"This module implements HSV, HSL, HSI, HCY, HWB, HCG color types."
"angular color spaces"

Some references to what these are would be nice.

Contrast with:

http://dlang.org/phobos/std_regex.html

where general information is given and links for more are provided (although 
that is weak and could be improved).


---

My last comment is what can I do with a color library that would not require 
some additional library?

Re: colour lib needs reviewers

[John Colvin via Digitalmars-d]

On Monday, 12 September 2016 at 19:59:58 UTC, Edwin van Leeuwen 
wrote:
On Monday, 12 September 2016 at 19:55:57 UTC, Guillaume Piolat 
wrote:
- I've wanted a function like colorFromString many times. It's 
especially nice with the added #RGBA and #RRGGBBAA syntax that 
eg. SVG lacks.


What happens when the string is invalid? Does it throw an error?


It should be integrated with std.conv.parse and std.conv.to and 
follow their conventions.

Re: colour lib needs reviewers

[Marco Leise via Digitalmars-d]

Am Tue, 13 Sep 2016 00:37:22 +1000
schrieb Manu via Digitalmars-d :
> I flip-flopped on this multiple times.
> It's not so simple.
> 1. Alpha doesn't necessarily mean transparently, that's just one
> (extremely common) use
> 2. 1 is the multiplication identity, but 0 is the additive identity. I
> cant find either addition or multiplication to take precedence over
> eachother.
> 3. With those 2 points above already making me feel a bit worried
> about it, I think that appearing numbers out of nowhere is just
> generally bad, so I went with a=0.

> If alpha defaults to 1, then a*b works as expected (object does not
> become transparent by this operation).
> If alpha defaults to 0, then b-a works as expected (object does not
> become transparent by this operation).

Additive color components can work like that, but the alpha
component just doesn't represent an amount of light. Don't try
too hard to make the arithmetics make sense, when the
practical use lies in 100% alpha by default. Converting every
color to full transparent is not helpful when you want to save
a 24-bit BMP as 32-bit PNG or get an RGBA color from an RGB
texture.
I don't know who declared that 100% alpha means opaque. For
what it's worth it could have been the other way around and
all the (1-a) and (a) in blending functions swapped.
And a blending function is what is needed here to give any
meaning at all to arithmetics on alpha. An additive one may be:

  color = color_dst + color_src * alpha_src
  alpha = alpha_dst

> With that in mind, you realise alpha will *always* require explicit
> handling... so I think 0 is a better default at that stage. It may be
> possible to convince me otherwise ;)

I fully agree with the first line, I just think that for
practical uses alpha is considered to be 100% in pixel formats
without alpha channel. Otherwise the most popular blending
functions would do nothing on RGB colors.
SDL for example also uses 100% alpha by default: "If the
specified pixel format has an alpha component it will be
returned as all 1 bits (fully opaque)."

> > Cool stuff there in the colorspace module, to get started with
> > getting photos or DV/MPEG into a linear color space. Should
> > BT.709 also have prefab conversion methods? It would seem HD
> > video is more wide spread nowadays.  
> 
> BT.709's gamma func isn't changed from 601. Rec.2020 introduced a
> higher precision version.

> > There is a typo in "Function that converts a linear luminance
> > to gamme space." (colorspace.d)  
> 
> Thx. But please log comments like this on the PR so I don't lose track
> of them? :)

Will do!

> > It seems inconsistent that you can specify all kinds of
> > exponents when the exponent is shared but are forced to 5 bit
> > exponents when mixed into each component, while the mantissa
> > can still be adjusted. Would m7e3 have worked? (With f* being
> > the IEEE standard formats.)  
> 
> I was thinking that, and mentioned as such in the comments near those lines ;)
> Thing is, there are no realtime small floats with exponent != 5bits
> except the unique 7e3 format on xbox 360.
> I can extend this in an unbreaking way.
> I might: f## =  ieee-like, s##e# = signed float, (u)##e# = unsgned float.
> It's just pretty verbose is all: "rgb_11e5_11e5_10e5". I mean, it's
> fine... but maybe people wouldn't find this intuitive? Many
> programmers don't know what an 'exponent' or 'mantissa' is... :/

Yeah I thought that the common formats would be the IEEE
ones, especially f16 and that anything less are very niche
custom formats that come and go and need verbose descriptions.
Well, either way works.

> > The final package should contain a fair amount of
> > documentation, in my opinion also on the topics of sRGB vs.
> > linear and what "hybrid gamma" is.  
> 
> Yeah, but is it really my job to teach someone about colour space?
> Wikipedia can do that much better than I can... :/
> I agree, but I don't really know where to draw the line, or how to organise 
> it.
>
> Perhaps people with strong opinions on how this should be presented
> can create a PR?

Alright, but hybrid gamma is really not something that can be
googled. Or rather I end up at Toyota's Gamma Hybrid product
page. :D

And as always, don't let the turkeys get you down.

-- 
Marco

Re: colour lib needs reviewers

[Random D user via Digitalmars-d]

On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
I think I'm about as happy with my colour lib as I'm going to 
be. It really needs reviews.


- Manu


Hi. I'm just a random forum lurker, but here's my feedback.

It needs more docs/examples for the basic usage cases (i.e. how 
to skip docs and start working).


Now at first glance it looks like working with RGBA (most common 
case (with srgb)) is complicated and while reading the docs I 
kind of felt like I'd just write my own.


Few basic things I'd be interested in (examples):
1. How to pack/unpack color into uint/rgba channels. Also can I 
swizzle channels directly? How would I extract a single channel 
from color?
2. Can I just cast an array of bytes into a RGBA array and 
operate on that?
3. How to scale color channels e.g. 5 bits to 8 bits and vice 
versa. How to convert color formats? Can I combine 
scale/conversion with single channel or channel swizzling?
4. Which way are RGBA format names defined ("byte array order" r 
is first byte or "uint order" r is in high bits)?

5. Is it the same for packed rgba formats?
I'm not sure if packedrgb is a necessary distinction. Aren't they 
all kind of packed (small structs)? There's no guarantee that 1 
byte is not just a part of a bigger field.


Btw. Your RGBA8 example is a bit weird:
static assert(RGBA8("0x908000FF") == RGBA8(0x80,0x00,0xFF,0x90));
I'd assume RGBA8(r,g,b,a) would be either "uint order" r(0x90), 
g(0x80), b(0x00), a(0xFF) or "byte array order" (le) r(0xFF), 
g(0x00), b(0x80), a(0x90)


6. Perhaps you should define all the usual color formats, since 
everybody and their cat has their own preference for rgba channel 
orders.
In my experience all these 4 variants for RGBA are pretty 
common:

RGBA
BGRA
ARGB
ABGR
and their X and 24bit variants.

For 16 bits fairly common are:
RGB565 and RGBA5551, also sometimes you see one of RGBA 
permutations (like RGBA8 above).
Also L16 is pretty common for heightfields or other data 
images/textures.


I guess there are also planar formats, but I think those are more 
of a Image problem space.


Just as a completely random idea - How about aliases for HDR 
formats like HDR10 and Dolby Vision? Kind of looks like they're 
just combinations what you already have.

Re: colour lib needs reviewers

[Guillaume Piolat via Digitalmars-d]

On Monday, 12 September 2016 at 19:59:58 UTC, Edwin van Leeuwen 
wrote:
On Monday, 12 September 2016 at 19:55:57 UTC, Guillaume Piolat 
wrote:
- I've wanted a function like colorFromString many times. It's 
especially nice with the added #RGBA and #RRGGBBAA syntax that 
eg. SVG lacks.


What happens when the string is invalid? Does it throw an error?


It throws an Exception, like it should.

Re: colour lib needs reviewers

[Edwin van Leeuwen via Digitalmars-d]

On Monday, 12 September 2016 at 19:55:57 UTC, Guillaume Piolat 
wrote:
- I've wanted a function like colorFromString many times. It's 
especially nice with the added #RGBA and #RRGGBBAA syntax that 
eg. SVG lacks.


What happens when the string is invalid? Does it throw an error?

Re: colour lib needs reviewers

[Guillaume Piolat via Digitalmars-d]

On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
I think I'm about as happy with my colour lib as I'm going to 
be. It really needs reviews.



- Manu


My thoughts:
- I've wanted a function like colorFromString many times. It's 
especially nice with the added #RGBA and #RRGGBBAA syntax that 
eg. SVG lacks.

- The knowledge encoded in this library is great in itself.
- This could form the basis of the next-gen DbI image library so 
it is much needed.

=> I would certainly vote for inclusion.

Re: colour lib needs reviewers

[Guillaume Piolat via Digitalmars-d]

On Monday, 12 September 2016 at 14:12:35 UTC, Manu wrote:
Note also that I didn't do any YUV formats (this case you 
discuss is usually related to older broadcast formats)... 
didn't know if they'd be particularly useful to people. At 
least, it's not in my initial offering.


As someone who had to write lots of code involving such video 
color spaces, I'd say to don't bother.
- Video engineers will write their own highly-optimized version 
anyway,
- Dynamic range is indeed very often 16-235 / 16-240 but often 
with no meta-date allowing to know if this is the case,

- you often don't want clamping on each intermediate results

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 13 September 2016 at 01:30, John Colvin via Digitalmars-d
 wrote:
> On Monday, 12 September 2016 at 13:04:49 UTC, Manu wrote:
>>
>>
>>> 2. Q: is there anything to convert a color to grey scale ?
>>
>>
>> This isn't a short-answer question :) .. There are many ways depending on
>> the job.
>>
>> A simple way might be:
>>   L8 grey = cast(L8)RGB8(r, g, b); // L8 is luminance only, casting
>> will do the right thing
>>
>> This is the 'normal' way, but there are better ways...
>>
>> A superior way if you want a top-quality result would be to use the Lab
>> luminance value, since Lab is perceptually uniform, when you desaturate, it
>> maintains the colors relative luminance correctly, something like:
>>
>>   Lab!float lab = cast(Lab!float)RGB8(r, g, b); // cast your color to Lab
>>   lab.a = 0; // eliminate saturation on a axis
>>   lab.b = 0; // eliminate saturation on b axis
>>   // lab.L is now pure luminance
>>   RGB8 c = cast(RGB8)lab; // cast it back to RGB, and your RGB will be
>> greyscale, but with more perceptually correct conversion.
>>
>> Lots of pow's in that conversion, so it's slower.
>>
>> This link demonstrates some competing techniques:
>>
>> https://en.wikipedia.org/wiki/HSL_and_HSV#/media/File:Fire-breather_CIELAB_L*.jpg
>> Press left/right to cycle through them.
>> The first method (cast to L8) is: Rec. 601 luma Y′.  (or more
>> accurately, my code above would be: sRGB luma Y')
>> The second method (Lab.L) is: CIELAB L*.
>>
>> L* performs much better in low light.
>
>
> Is there some uniform policy w.r.t. casts here? If I take one colour type
> and cast it to another will it always do something unambiguously right or
> not compile?

I think, yes?
So, the logic here is, if you cast, and then cast back, you will have
the same thing barring loss of precision.
Casts in this lib just change format (+precision), they don't change
value at all. I feel it's compatible with casting an int to a float...
it's the same sort of conversion.


> Seems to me that a function that takes a target type would be
> more obvious for the user.

Really? I like the cast this way. There is an explicit function in
package.d though...

Re: colour lib needs reviewers

[Guillaume Chatelet via Digitalmars-d]

On Monday, 12 September 2016 at 15:06:29 UTC, Manu wrote:

On 13 September 2016 at 00:37, Andrei Alexandrescu via
Regarding Phobos suitability: One thing I'd like to hear from 
the community is the applicability of this work. I know very 
little about colors (only the RGB was familiar from HTML etc, 
but then there was no YUL), but that doesn't mean much. The 
library is quite isolated from the rest of Phobos, e.g. a 
stronger case would be made if there were some 
std.experimental.canvas package which uses the color package. 
Or at least a std.experimental.terminal package or something. 
A good case for inclusion would motivate the presence of the 
library as a catalyst for many user-level abstractions. -- 
Andrei


So, the main reason I wanted to push for this in phobos is 
because

it's an enabler.
There is no multimedia in phobos at all. I think that's a 
shame, and
this is a starting point. Any conceivable image based work 
depends on
this as foundation. If pixel format types are phobos-supplied, 
then D
image libraries will naturally be compatible, as opposed to 
naturally
incompatible since every image library will otherwise 
re-implement

their own colour types (and almost always incorrectly).
The thing about colours is, they're HARD to get right, but more
importantly, most programmers have no idea that it's actually a 
hard
problem and won't tend to go looking for a lib for a colour 
type (Ie,
struct { ubyte r, g, b;}... simple! *cough*). They probably 
will use

it if it's in phobos though for compatibility reasons.
Image processing is super easy by contrast.

One of my first port-of-calls after this work would be a 
graph/plot library... I'd use the hell out of that. I 
constantly want to output data visualisations!


It also enables seamless interaction with realtime rendering 
api's. Ie, if OpenGL/D3D/Vulkan bindings support phobos 
supplied colour types, then image decoding libraries, and other 
sources of image data become easy to render without any 
requirement for adaptation. Rendering libs and image/video data 
source libs are almost always separate, independent libraries. 
We need to maximise probability that they are compatible out of 
the gate.


I didn't want to touch 'images' in this work, because there are 
countless subjective designs to present those API's, but I 
don't think there's many competing designs for a colour 
library, so it felt it was the best and most bang-for-buck 
starting place overall.


Also, I think this has very high synergy with ndslice. I think 
ndslice +  color is the foundation for image processing, or at 
least image data transport.


+1

It's simply amazing how the concept of color is well understood 
by everyone and yet utterly hard to understand in the details.


Color is a Universe in itself! Just have a look at Metamerism(1) 
(colors that look the same but aren't), Human Tetrachromacy(2) 
(some people - mostly women - perceive more colors than others). 
I'm not even scratching the surface: it connects with everything: 
Physics, Physiology, Art, Entertainement, Evolution(3)...


Having a high quality Color API in Phobos is a differentiator and 
an enabler: think Raytracer, image batch processing, scientific 
analysis, interactions with GUI or rendering APIs.
I think it's an important addition to Phobos - I might be biased 
though :-P


1. https://en.wikipedia.org/wiki/Metamerism_(color)
2. https://en.wikipedia.org/wiki/Tetrachromacy#Humans
3. https://www.youtube.com/watch?v=qrKZBh8BL_U

Re: colour lib needs reviewers

[John Colvin via Digitalmars-d]

On Monday, 12 September 2016 at 13:04:49 UTC, Manu wrote:



2. Q: is there anything to convert a color to grey scale ?


This isn't a short-answer question :) .. There are many ways 
depending on the job.


A simple way might be:
  L8 grey = cast(L8)RGB8(r, g, b); // L8 is luminance only, 
casting

will do the right thing

This is the 'normal' way, but there are better ways...

A superior way if you want a top-quality result would be to use 
the Lab luminance value, since Lab is perceptually uniform, 
when you desaturate, it maintains the colors relative luminance 
correctly, something like:


  Lab!float lab = cast(Lab!float)RGB8(r, g, b); // cast your 
color to Lab

  lab.a = 0; // eliminate saturation on a axis
  lab.b = 0; // eliminate saturation on b axis
  // lab.L is now pure luminance
  RGB8 c = cast(RGB8)lab; // cast it back to RGB, and your RGB 
will be

greyscale, but with more perceptually correct conversion.

Lots of pow's in that conversion, so it's slower.

This link demonstrates some competing techniques:
https://en.wikipedia.org/wiki/HSL_and_HSV#/media/File:Fire-breather_CIELAB_L*.jpg
Press left/right to cycle through them.
The first method (cast to L8) is: Rec. 601 luma Y′.  (or more
accurately, my code above would be: sRGB luma Y')
The second method (Lab.L) is: CIELAB L*.

L* performs much better in low light.


Is there some uniform policy w.r.t. casts here? If I take one 
colour type and cast it to another will it always do something 
unambiguously right or not compile? Seems to me that a function 
that takes a target type would be more obvious for the user.


I think the multiple different meanings of cast are a mistake in 
D. We can't go back now, but I wish there was only 
reinterpet_cast and everything else was a function.

Re: colour lib needs reviewers

[Edwin van Leeuwen via Digitalmars-d]

On Monday, 12 September 2016 at 15:06:29 UTC, Manu wrote:
One of my first port-of-calls after this work would be a 
graph/plot library... I'd use the hell out of that. I 
constantly want to output data visualisations!


Just for your information. ggplotd currently uses an old version 
of your library on the backend. I'm planning on porting it soon 
to the latest (dub) version.

Re: colour lib needs reviewers

[Adam D. Ruppe via Digitalmars-d]

On Monday, 12 September 2016 at 15:06:29 UTC, Manu wrote:
So, the main reason I wanted to push for this in phobos is 
because it's an enabler.


Indeed. I know there's a window library that was waiting on this 
work to try for Phobos.


Also, worth noting that the one module I import more than any 
other in my libs is color.d. Image load/save depends on it. 
Windowing display depends on it. Graphing depends on one or the 
other, GUI depends on windowing, terminal I managed to keep 
independent... but it is REALLY tempting to make it depend on 
color too.


It is a very useful base building block, and moreover, having it 
in Phobos means User X's graphing lib and my image libs could 
conceivably be able to interchange data with ease since we both 
use the same underlying data structure!

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 13 September 2016 at 00:37, Andrei Alexandrescu via Digitalmars-d
 wrote:
> On 09/12/2016 12:14 AM, Manu via Digitalmars-d wrote:
>>
>> I think I'm about as happy with my colour lib as I'm going to be.
>> It really needs reviews.
>
>
> Upon a quick look it seems to have the right amount of "notes", as Emperor
> Joseph II would say. Well balanced, nothing jumping at you. Nice work.

Thanks. It has taken a lot more work than I actually expected to make
this not suck!

> Regarding Phobos suitability: One thing I'd like to hear from the community
> is the applicability of this work. I know very little about colors (only the
> RGB was familiar from HTML etc, but then there was no YUL), but that doesn't
> mean much. The library is quite isolated from the rest of Phobos, e.g. a
> stronger case would be made if there were some std.experimental.canvas
> package which uses the color package. Or at least a
> std.experimental.terminal package or something. A good case for inclusion
> would motivate the presence of the library as a catalyst for many user-level
> abstractions. -- Andrei

So, the main reason I wanted to push for this in phobos is because
it's an enabler.
There is no multimedia in phobos at all. I think that's a shame, and
this is a starting point. Any conceivable image based work depends on
this as foundation. If pixel format types are phobos-supplied, then D
image libraries will naturally be compatible, as opposed to naturally
incompatible since every image library will otherwise re-implement
their own colour types (and almost always incorrectly).
The thing about colours is, they're HARD to get right, but more
importantly, most programmers have no idea that it's actually a hard
problem and won't tend to go looking for a lib for a colour type (Ie,
struct { ubyte r, g, b;}... simple! *cough*). They probably will use
it if it's in phobos though for compatibility reasons.
Image processing is super easy by contrast.

One of my first port-of-calls after this work would be a graph/plot
library... I'd use the hell out of that. I constantly want to output
data visualisations!

It also enables seamless interaction with realtime rendering api's.
Ie, if OpenGL/D3D/Vulkan bindings support phobos supplied colour
types, then image decoding libraries, and other sources of image data
become easy to render without any requirement for adaptation.
Rendering libs and image/video data source libs are almost always
separate, independent libraries. We need to maximise probability that
they are compatible out of the gate.

I didn't want to touch 'images' in this work, because there are
countless subjective designs to present those API's, but I don't think
there's many competing designs for a colour library, so it felt it was
the best and most bang-for-buck starting place overall.

Also, I think this has very high synergy with ndslice. I think ndslice
+  color is the foundation for image processing, or at least image
data transport.

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 12 September 2016 at 21:13, Guillaume Chatelet via Digitalmars-d
 wrote:
> On Monday, 12 September 2016 at 09:09:29 UTC, Manu wrote:
>>
>> I thought about it, but it added (even more) complexity, and I'm not
>> really sure it's the MO of a color lib to worry about the endian-ness of
>> storage.
>>
>> I could support "_le"/"_be" suffix on the PackedRGB format string, but
>> then you have to also specify the word size, ie, is the packed color
>> 32bit words (eg, 11,11,10) or 16bit words (16f,16f,16f,16f)?
>> It's only really practically applicable to PackedRGB, so I wonder why
>> just one color type should worry about it, but not all the others...
>>
>> Consider an unpacked RGB with 16bit elements... that may also suffer
>> endian problems, but RGB is a runtime type, not a storage type, which means
>> it doesn't have a pack/unpack process... so there's no meaningful palace to
>> do it.
>
>
> Yep I totally understand the rationale - I've been there too :)
> I don't think it belong to Color but maybe Buffer(1) could be templated on
> endianness? Buffer is indeed the place where you deal with storage.
> If you don't want to add it to the library I think you should add a
> paragraph in the documentation explaining why it's not in there and how
> people can overcome it (copy + change endianness or use an adapter?).
>
> Also, it would be nice to have some code snippets in the documentation.
>
> Last thing, for the white points you didn't specify which CIE standard
> observer(2) version you used: CIE 1931 2° Standard Observer or CIE 1964 10°
> Standard Observer? IIRC the standard illuminant are not the same depending
> on the observer.

That's very true. The data is all CIE 1931 reference. I haven't seen
deployed use of the updated standard... I don't know why.

Re: colour lib needs reviewers

[Andrei Alexandrescu via Digitalmars-d]

On 09/12/2016 12:14 AM, Manu via Digitalmars-d wrote:

I think I'm about as happy with my colour lib as I'm going to be.
It really needs reviews.


Upon a quick look it seems to have the right amount of "notes", as 
Emperor Joseph II would say. Well balanced, nothing jumping at you. Nice 
work. Regarding Phobos suitability: One thing I'd like to hear from the 
community is the applicability of this work. I know very little about 
colors (only the RGB was familiar from HTML etc, but then there was no 
YUL), but that doesn't mean much. The library is quite isolated from the 
rest of Phobos, e.g. a stronger case would be made if there were some 
std.experimental.canvas package which uses the color package. Or at 
least a std.experimental.terminal package or something. A good case for 
inclusion would motivate the presence of the library as a catalyst for 
many user-level abstractions. -- Andrei

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 12 September 2016 at 21:28, Marco Leise via Digitalmars-d
 wrote:
> Am Mon, 12 Sep 2016 14:14:27 +1000
> schrieb Manu via Digitalmars-d :
>
>> I think I'm about as happy with my colour lib as I'm going to be.
>> It really needs reviews.
>>
>> I added packed-RGB support, including weird micro-float and
>> shared-exponent formats.
>> They're important for interacting with any real-time rendering libraries.
>> There is only one texture format I'm aware of that isn't supported,
>> and that is u7e3 floats, only available on Xbox360 ;)
>>
>> Chromatic adaptation functions are in.
>> I've done a tidy/reorg pass.
>>
>> I'm not sure what else should be in the scope of this lib.
>>
>> PR: https://github.com/dlang/phobos/pull/2845
>> dub: https://code.dlang.org/packages/color
>> docs: 
>> http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html
>>
>> - Manu
>
> Nice work. What's u7e3? 3*10-bit-floats + 2-bit padding?

Yup.


> In the example for convert color you show that converting from
> a color space without alpha (XYZ) to one with alpha (RGBA)
> turns the pixel transparent. I believe you meant this as
> something to lookout for when converting colors. What's the
> reasoning for not setting maximum alpha as a more useful and
> natural default?

I flip-flopped on this multiple times.
It's not so simple.
1. Alpha doesn't necessarily mean transparently, that's just one
(extremely common) use
2. 1 is the multiplication identity, but 0 is the additive identity. I
cant find either addition or multiplication to take precedence over
eachother.
3. With those 2 points above already making me feel a bit worried
about it, I think that appearing numbers out of nowhere is just
generally bad, so I went with a=0.

If alpha defaults to 1, then a*b works as expected (object does not
become transparent by this operation).
If alpha defaults to 0, then b-a works as expected (object does not
become transparent by this operation).
With that in mind, you realise alpha will *always* require explicit
handling... so I think 0 is a better default at that stage. It may be
possible to convince me otherwise ;)


> Cool stuff there in the colorspace module, to get started with
> getting photos or DV/MPEG into a linear color space. Should
> BT.709 also have prefab conversion methods? It would seem HD
> video is more wide spread nowadays.

BT.709's gamma func isn't changed from 601. Rec.2020 introduced a
higher precision version.


> There is a typo in "Function that converts a linear luminance
> to gamme space." (colorspace.d)

Thx. But please log comments like this on the PR so I don't lose track
of them? :)


> I like how RGB color spaces are described as primaries in xyY
> + gamma functions.

Seems natural :)

So, I was initially planning to make that struct the RGB template
argument, but then I realised that would make the symbols gigantic
(heaps of CS data in the symbol names), so I introduced the enum...
sadly though, this means you can't define a custom colour without
introducing a new custom-colour type to do what I originally intended.
I'm upset about this either way...
I think I should leave it how it is... custom colours are probably
more rare than gigantic symbol names ;)


> It seems inconsistent that you can specify all kinds of
> exponents when the exponent is shared but are forced to 5 bit
> exponents when mixed into each component, while the mantissa
> can still be adjusted. Would m7e3 have worked? (With f* being
> the IEEE standard formats.)

I was thinking that, and mentioned as such in the comments near those lines ;)
Thing is, there are no realtime small floats with exponent != 5bits
except the unique 7e3 format on xbox 360.
I can extend this in an unbreaking way.
I might: f## =  ieee-like, s##e# = signed float, (u)##e# = unsgned float.
It's just pretty verbose is all: "rgb_11e5_11e5_10e5". I mean, it's
fine... but maybe people wouldn't find this intuitive? Many
programmers don't know what an 'exponent' or 'mantissa' is... :/


> The final package should contain a fair amount of
> documentation, in my opinion also on the topics of sRGB vs.
> linear and what "hybrid gamma" is.

Yeah, but is it really my job to teach someone about colour space?
Wikipedia can do that much better than I can... :/
I agree, but I don't really know where to draw the line, or how to organise it.

Perhaps people with strong opinions on how this should be presented
can create a PR?


> What is your stance on pixel formats that are arranged in
> planes? I assume that e.g. YUV is handled by a single channel
> "sRGB luminance" + 2 signed 8-bit color planes? It may be
> worth adding some hints in that direction.

I haven't supported YUV, but it is as you say. Nitpick: most of those
YUV sinals are probably broadcast signals, with rec.601 luminance ;)
YUV is a pretty big family, 

Re: colour lib needs reviewers

[Guillaume Chatelet via Digitalmars-d]

On Monday, 12 September 2016 at 14:12:35 UTC, Manu wrote:
On 12 September 2016 at 21:31, Guillaume Chatelet via 
Digitalmars-d  wrote:

[...]


Actually, I spent quite some time stressing about dynamic range.
Trouble is, there's no real format spec's relating to dynamic 
range,

it tends to be by convention.
I'm not sure what the API would look like, so I haven't 
approached that problem.
It's very easy to do the transform yourself, I don't really 
think it

needs to be part of the type, particularly since there's no
conventional way to express it.

Note also that I didn't do any YUV formats (this case you 
discuss is usually related to older broadcast formats)... 
didn't know if they'd be particularly useful to people. At 
least, it's not in my initial offering.


Yes I agree, as you and Marco said: I think it's best to leave it 
to clients of the API.

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 12 September 2016 at 21:31, Guillaume Chatelet via Digitalmars-d
 wrote:
> On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
>>
>> I think I'm about as happy with my colour lib as I'm going to be. It
>> really needs reviews.
>>
>> I added packed-RGB support, including weird micro-float and
>> shared-exponent formats.
>> They're important for interacting with any real-time rendering libraries.
>> There is only one texture format I'm aware of that isn't supported,
>> and that is u7e3 floats, only available on Xbox360 ;)
>>
>> Chromatic adaptation functions are in.
>> I've done a tidy/reorg pass.
>>
>> I'm not sure what else should be in the scope of this lib.
>>
>> PR: https://github.com/dlang/phobos/pull/2845
>> dub: https://code.dlang.org/packages/color
>> docs:
>> http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html
>>
>> - Manu
>
>
> And also, do you handle limited (16-235) to full dynamic range (0-255)
> conversions?

Actually, I spent quite some time stressing about dynamic range.
Trouble is, there's no real format spec's relating to dynamic range,
it tends to be by convention.
I'm not sure what the API would look like, so I haven't approached that problem.
It's very easy to do the transform yourself, I don't really think it
needs to be part of the type, particularly since there's no
conventional way to express it.

Note also that I didn't do any YUV formats (this case you discuss is
usually related to older broadcast formats)... didn't know if they'd
be particularly useful to people. At least, it's not in my initial
offering.

Re: colour lib needs reviewers

[Manu via Digitalmars-d]

On 12 September 2016 at 19:30, Basile B. via Digitalmars-d
 wrote:
> On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
>>
>> I think I'm about as happy with my colour lib as I'm going to be. It
>> really needs reviews.
>>
>> I'm not sure what else should be in the scope of this lib.
>>
>> PR: https://github.com/dlang/phobos/pull/2845
>> dub: https://code.dlang.org/packages/color
>> docs:
>> http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html
>>
>> [...]
>> - Manu
>
>
> 1. Maybe that the each package ddoc comment should mention that everything
> is always @safe nothrow @nogc...For me it wasn't directly clear:
> - seen that there's opCast: Q: safe ?
> - seen in the sources that opCast call a templatized function
> - seen that the unittest for this function are not annotated
> - then only seen that the leading "@safe nothrow @nogc:" at the beg of the
> package.

How do you think this should this look? (it's best to leave these
comments on the PR)


> 2. Q: is there anything to convert a color to grey scale ?

This isn't a short-answer question :) .. There are many ways depending
on the job.

A simple way might be:
  L8 grey = cast(L8)RGB8(r, g, b); // L8 is luminance only, casting
will do the right thing

This is the 'normal' way, but there are better ways...

A superior way if you want a top-quality result would be to use the
Lab luminance value, since Lab is perceptually uniform, when you
desaturate, it maintains the colors relative luminance correctly,
something like:

  Lab!float lab = cast(Lab!float)RGB8(r, g, b); // cast your color to Lab
  lab.a = 0; // eliminate saturation on a axis
  lab.b = 0; // eliminate saturation on b axis
  // lab.L is now pure luminance
  RGB8 c = cast(RGB8)lab; // cast it back to RGB, and your RGB will be
greyscale, but with more perceptually correct conversion.

Lots of pow's in that conversion, so it's slower.

This link demonstrates some competing techniques:
https://en.wikipedia.org/wiki/HSL_and_HSV#/media/File:Fire-breather_CIELAB_L*.jpg
Press left/right to cycle through them.
The first method (cast to L8) is: Rec. 601 luma Y′.  (or more
accurately, my code above would be: sRGB luma Y')
The second method (Lab.L) is: CIELAB L*.

L* performs much better in low light.

Re: colour lib needs reviewers

[Marco Leise via Digitalmars-d]

Am Mon, 12 Sep 2016 11:31:13 +
schrieb Guillaume Chatelet :

> On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
> > I think I'm about as happy with my colour lib as I'm going to 
> > be. It really needs reviews.
> >
> > I added packed-RGB support, including weird micro-float and
> > shared-exponent formats.
> > They're important for interacting with any real-time rendering 
> > libraries.
> > There is only one texture format I'm aware of that isn't 
> > supported,
> > and that is u7e3 floats, only available on Xbox360 ;)
> >
> > Chromatic adaptation functions are in.
> > I've done a tidy/reorg pass.
> >
> > I'm not sure what else should be in the scope of this lib.
> >
> > PR: https://github.com/dlang/phobos/pull/2845
> > dub: https://code.dlang.org/packages/color
> > docs: 
> > http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html
> >
> > - Manu  
> 
> And also, do you handle limited (16-235) to full dynamic range 
> (0-255) conversions?
> 
> It's a setting some acquisition/graphic card allow.
> https://wiki.videolan.org/VSG:Video:Color_washed_out/
> 
>  [...]

Practically all video content is encoded with the limited
range, from VHS to HDTV, IIRC and HiFi equipment uses it as
well, only computers are not so familiar with it and when
connected to a TV over HDMI one or the other may think that
the other part thinks ... well colors are often washed out,
in effect.
On the other hand this luminance compression can apply to any
color format, so it is probably best to leave it to the user
to compress colors if the output is a video file.

-- 
Marco

Re: colour lib needs reviewers

[Guillaume Chatelet via Digitalmars-d]

On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
I think I'm about as happy with my colour lib as I'm going to 
be. It really needs reviews.


I added packed-RGB support, including weird micro-float and
shared-exponent formats.
They're important for interacting with any real-time rendering 
libraries.
There is only one texture format I'm aware of that isn't 
supported,

and that is u7e3 floats, only available on Xbox360 ;)

Chromatic adaptation functions are in.
I've done a tidy/reorg pass.

I'm not sure what else should be in the scope of this lib.

PR: https://github.com/dlang/phobos/pull/2845
dub: https://code.dlang.org/packages/color
docs: 
http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html


- Manu


And also, do you handle limited (16-235) to full dynamic range 
(0-255) conversions?


It's a setting some acquisition/graphic card allow.
https://wiki.videolan.org/VSG:Video:Color_washed_out/

From this page as well 
http://www.digitalpreservation.gov/formats/fdd/fdd000352.shtml


"Video and wide range. The ITU-R specifications support values 
for Y, Cb, and Cr that conform to what is sometimes called "video 
range," "legal range," or "studio swing" levels. Expressed in 
terms of 8-bit tonal range values, video range has a 16-235 
levels for Y and 16-240 levels for Cr and Cb. The term video 
range is used to contrast with "wide range" or "super white " 
values from 0 to 255, sometimes used when video signals are 
created using computer-based graphics applications."

Re: colour lib needs reviewers

[Marco Leise via Digitalmars-d]

Am Mon, 12 Sep 2016 14:14:27 +1000
schrieb Manu via Digitalmars-d :

> I think I'm about as happy with my colour lib as I'm going to be.
> It really needs reviews.
> 
> I added packed-RGB support, including weird micro-float and
> shared-exponent formats.
> They're important for interacting with any real-time rendering libraries.
> There is only one texture format I'm aware of that isn't supported,
> and that is u7e3 floats, only available on Xbox360 ;)
> 
> Chromatic adaptation functions are in.
> I've done a tidy/reorg pass.
> 
> I'm not sure what else should be in the scope of this lib.
> 
> PR: https://github.com/dlang/phobos/pull/2845
> dub: https://code.dlang.org/packages/color
> docs: 
> http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html
> 
> - Manu

Nice work. What's u7e3? 3*10-bit-floats + 2-bit padding?

In the example for convert color you show that converting from
a color space without alpha (XYZ) to one with alpha (RGBA)
turns the pixel transparent. I believe you meant this as
something to lookout for when converting colors. What's the
reasoning for not setting maximum alpha as a more useful and
natural default?

Cool stuff there in the colorspace module, to get started with
getting photos or DV/MPEG into a linear color space. Should
BT.709 also have prefab conversion methods? It would seem HD
video is more wide spread nowadays.

There is a typo in "Function that converts a linear luminance
to gamme space." (colorspace.d)

I like how RGB color spaces are described as primaries in xyY
+ gamma functions.

It seems inconsistent that you can specify all kinds of
exponents when the exponent is shared but are forced to 5 bit
exponents when mixed into each component, while the mantissa
can still be adjusted. Would m7e3 have worked? (With f* being
the IEEE standard formats.)

The final package should contain a fair amount of
documentation, in my opinion also on the topics of sRGB vs.
linear and what "hybrid gamma" is.

What is your stance on pixel formats that are arranged in
planes? I assume that e.g. YUV is handled by a single channel
"sRGB luminance" + 2 signed 8-bit color planes? It may be
worth adding some hints in that direction.

All in all good work with a scientific feel to it.

-- 
Marco

Re: colour lib needs reviewers

[Guillaume Chatelet via Digitalmars-d]

On Monday, 12 September 2016 at 09:09:29 UTC, Manu wrote:
I thought about it, but it added (even more) complexity, and 
I'm not really sure it's the MO of a color lib to worry about 
the endian-ness of storage.


I could support "_le"/"_be" suffix on the PackedRGB format 
string, but
then you have to also specify the word size, ie, is the packed 
color

32bit words (eg, 11,11,10) or 16bit words (16f,16f,16f,16f)?
It's only really practically applicable to PackedRGB, so I 
wonder why
just one color type should worry about it, but not all the 
others...


Consider an unpacked RGB with 16bit elements... that may also 
suffer endian problems, but RGB is a runtime type, not a 
storage type, which means it doesn't have a pack/unpack 
process... so there's no meaningful palace to do it.


Yep I totally understand the rationale - I've been there too :)
I don't think it belong to Color but maybe Buffer(1) could be 
templated on endianness? Buffer is indeed the place where you 
deal with storage.
If you don't want to add it to the library I think you should add 
a paragraph in the documentation explaining why it's not in there 
and how people can overcome it (copy + change endianness or use 
an adapter?).


Also, it would be nice to have some code snippets in the 
documentation.


Last thing, for the white points you didn't specify which CIE 
standard observer(2) version you used: CIE 1931 2° Standard 
Observer or CIE 1964 10° Standard Observer? IIRC the standard 
illuminant are not the same depending on the observer.




1. 
http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color/packedrgb/buffer.html
2. 
https://en.wikipedia.org/wiki/CIE_1931_color_space#CIE_standard_observer

Re: colour lib needs reviewers

[Basile B. via Digitalmars-d]

On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
I think I'm about as happy with my colour lib as I'm going to 
be. It really needs reviews.


I'm not sure what else should be in the scope of this lib.

PR: https://github.com/dlang/phobos/pull/2845
dub: https://code.dlang.org/packages/color
docs: 
http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html


[...]
- Manu


1. Maybe that the each package ddoc comment should mention that 
everything is always @safe nothrow @nogc...For me it wasn't 
directly clear:

- seen that there's opCast: Q: safe ?
- seen in the sources that opCast call a templatized function
- seen that the unittest for this function are not annotated
- then only seen that the leading "@safe nothrow @nogc:" at the 
beg of the package.


2. Q: is there anything to convert a color to grey scale ?

Re: colour lib needs reviewers

[Ilya Yaroshenko via Digitalmars-d]

On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
I think I'm about as happy with my colour lib as I'm going to 
be. It really needs reviews.


[...]


Looks great!

1. packedrgb has not enough docs to understand what it is.
2. colorFromString should have scope argument.

- Ilya