parallel build fixes for 7.4.2
Hi, recently we've been tweaking our internal build system at Tsuru to handle parallel builds of both cabal packages via 'cabal-sort --makefile' and our local code tree via 'ghc -M'. In addition to the recompilation checker fixes of #5878, the following would be great to have in 7.4.2: 1) http://hackage.haskell.org/trac/ghc/ticket/5891 -- the patch fixes a race condition in creating parent directories for built object files 2) master commit b6f94b5 Compile link .note section separately from main.c -- I think this is the patch that fixes link errors we've seen during parallel builds (via ghc -M) with 7.4.1, such as: /x/y/z.o: file not recognized: File truncated and: /usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.20.51-system.20100908 internal error, aborting at ../../bfd/merge.c line 872 in _bfd_merged_section_offset Will everything currently in master already be included in the next release or is it a separate branch? (If it's a separate branch I'll do some more testing to confirm that b6f94b5 is the patch that fixes the link error). Conrad. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: parallel build fixes for 7.4.2
By the way, it looks like the documentation for createDirectoryIfMissing doesn't mention anything about its safety under concurrent invocation. Looking at the code, it looks like it is safe (it handles the already exist exception): http://www.haskell.org/ghc/docs/latest/html/libraries/directory/src/System-Directory.html#createDirectoryIfMissing But maybe the docs could reflect that -Ryan On Fri, Feb 24, 2012 at 3:16 AM, Conrad Parker con...@metadecks.org wrote: Hi, recently we've been tweaking our internal build system at Tsuru to handle parallel builds of both cabal packages via 'cabal-sort --makefile' and our local code tree via 'ghc -M'. In addition to the recompilation checker fixes of #5878, the following would be great to have in 7.4.2: 1) http://hackage.haskell.org/trac/ghc/ticket/5891 -- the patch fixes a race condition in creating parent directories for built object files 2) master commit b6f94b5 Compile link .note section separately from main.c -- I think this is the patch that fixes link errors we've seen during parallel builds (via ghc -M) with 7.4.1, such as: /x/y/z.o: file not recognized: File truncated and: /usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.20.51-system.20100908 internal error, aborting at ../../bfd/merge.c line 872 in _bfd_merged_section_offset Will everything currently in master already be included in the next release or is it a separate branch? (If it's a separate branch I'll do some more testing to confirm that b6f94b5 is the patch that fixes the link error). Conrad. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
I looked at the DORF implementers view
http://hackage.haskell.org/trac/ghc/wiki/Records/DeclaredOverloadedRecordFields/ImplementorsView
It appears that we still have no solution for record updates that
change the type of a polymorphic field.
I think this means we need to look to ways to solve this update issue
other than just de-sugaring to typeclasses.
On Thu, Feb 23, 2012 at 5:01 PM, Greg Weber g...@gregweber.info wrote:
On Thu, Feb 23, 2012 at 4:25 PM, AntC anthony_clay...@clear.net.nz wrote:
Greg Weber greg at gregweber.info writes:
Thanks to Anthony for his DORF proposal, and spending time to clearly
explain it on the wiki.
I have a big issue with the approach as presented - it assumes that
record fields with the same name should automatically be shared. As I
have stated before, to avoid circular dependencies, one is forced to
declare types in a single module in Haskell that would ideally be
declared in a multiple modules. ...
Thanks Greg, but I'm struggling to understand what the difficulty is with
sharing the same name, or why your dependencies are circular. Would you be
able to post some code snippets that illustrate what's going on (or what's
failing to go on)?
Or perhaps this is an experience from some application where you weren't
using
Haskell? Could you at least describe what was in each record type?
You can get an idea of things in the section 'Problems with using the
module namespace mechanism' here:
http://hackage.haskell.org/trac/ghc/wiki/Records?version=4
The attachment that Chris Done left to demonstrate his types seems to
be overwritten.
I will bring back his text as it seems his point does need to be driven home.
A lot of Haskell projects have a separate Types module to avoid issues
with circular dependencies.
Continuing the database example, I
will have multiple tables with a 'name' column, but they do not have
the same meaning.
If I have a function:
helloName person = Hello, ++ person.name
The compiler could infer that I want to say hello to inanimate objects!
So the first question is:
* do your fields labelled `name` all have the same type? (Perhaps all
String?)
* what meaning does a name have beyond being a String?
Your code snippet doesn't give the types, but if I guess that you intend
`person` to be of type `Person`. Then you can give a signature:
helloName :: Person - String
If person can be 'anything' then the type inferred from the bare function
equation would be:
helloName :: r{ name :: String } = r - String
So you could say hello to your cat, and your pet rock. You couldn't say hello
to a pile of bricks (unless it's been given a name as an art installation in
the Tate Gallery ;-)
Of course we know that we can always add type annotations to clarify
things. The question is whether we want to be opt-out and have to
explain people that they can end up with weakly typed code when they
don't want to share fields.
Note that I am not completely against abstraction over fields, I just
don't think it is the best default behavior.
So what is the best default behaviour, and what is the range of other
behaviours you want to support?
I believe the best default is to not share fields, but instead have
the programmer indicate at or outside of the record definition that
they want to share fields. Basically just use type-classes how they
are used now - as opt-in. But I am OK with making an especially easy
way to do this with records if the current techniques for defining
typeclasses are seen as to verbose.
And the section Modules and qualified names for records shows that
the proposal doesn't fully solve the name-spacing issue.
I think it shows only that record field labels can run into accidental name
clash in exactly the same way as everything else in Haskell (or indeed in any
programming language). And that Haskell has a perfectly good way for dealing
with that; and that DORF fits in with it.
Greg, please give some specific examples! I'm trying to understand, but I'm
only guessing from the fragments of info you're giving.
AntC
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
Actually, I looked at the SORF page again: http://hackage.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields There is now an 'Alternative Proposal' section that claims to solve polymorphic update. If anyone has comments on this please let us know! So my proposal would be to somehow use the SORF Alternative Proposal to allow for name-spaced records. This should be used to generate internal constraints and not be exposed to the end user and not automatically abstract over fields. This leaves all of our future options open while satisfying the narrow issue at hand. On Fri, Feb 24, 2012 at 9:27 AM, Greg Weber g...@gregweber.info wrote: I looked at the DORF implementers view http://hackage.haskell.org/trac/ghc/wiki/Records/DeclaredOverloadedRecordFields/ImplementorsView It appears that we still have no solution for record updates that change the type of a polymorphic field. I think this means we need to look to ways to solve this update issue other than just de-sugaring to typeclasses. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
Actually, I looked at the SORF page again: http://hackage.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields There is now an 'Alternative Proposal' section that claims to solve polymorphic update. If anyone has comments on this please let us know! Yes, Greg the quasifunctor stuff has been there for some time. You seem to be confused between polymorphic fields vs type-changing update. Quasifunctor is talking about type-changing update. (Look at the example given for `data R a`: there's no polymorphic field.) DORF already has a working approach to type-changing update. I make reference to quasifunctor in the DORF proposal. Essentially, if quasifunctor can be made to work for SORF, it can also be made to work for DORF. (Because what underlies both is to use a Has class with methods get/set.) No, I don't think anybody has a satisfactory approach to updating polymorphic/higher-ranked fields. (DORF mentions one, but it's a ghastly hack. In addition to higher-ranked types, SPJ is also concerned about h-r's with constraints. I've added a very speculative piece in DORF's comparison to SORF that considers using Constraint Kinds. (It's probably piling ghastliness upon ghastliness.) ... and not automatically abstract over fields. This leaves all of our future options open while satisfying the narrow issue at hand. What on earth do you mean by not automatically abstract over fields? AntC ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
This should be used to generate internal constraints and not be exposed to the end user and not automatically abstract over fields. Every one of your messages about records stresses your dislike for polymorphic projections, and your insistence that the Has class should be hidden from the user. I've read all of your explanations, but I'm still totally unconvinced. All your arguments about the semantics of labels are based on the way you want to use them, not on what they are. They are projection functions! Semantically, the only difference between them is the types. Polymorphism makes perfect sense and is completely natural. There is nothing untyped about it. I feel you are pushing a narrow personal agenda here. I think the Has class would be useful to programmers and no harder to understand than other Haskel classes. It should not be hidden. Barney. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
On Fri, Feb 24, 2012 at 2:00 PM, Barney Hilken b.hil...@ntlworld.com wrote: This should be used to generate internal constraints and not be exposed to the end user and not automatically abstract over fields. Every one of your messages about records stresses your dislike for polymorphic projections, and your insistence that the Has class should be hidden from the user. I've read all of your explanations, but I'm still totally unconvinced. All your arguments about the semantics of labels are based on the way you want to use them, not on what they are. They are projection functions! Semantically, the only difference between them is the types. Polymorphism makes perfect sense and is completely natural. There is nothing untyped about it. I feel you are pushing a narrow personal agenda here. I think the Has class would be useful to programmers and no harder to understand than other Haskel classes. It should not be hidden. Barney, these kinds of statements about personal agenda are counterproductive. I have put in a lot of work that nobody, including yourself has been willing to do for years to push Haskell's records forward. I would appreciate respectful disagreement - I think I have earned that much. Please just stick to logical arguments. Greg Weber ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
No, I don't think anybody has a satisfactory approach to updating polymorphic/higher-ranked fields. (DORF mentions one, but it's a ghastly hack. So are the proposals dead until this is tackled, or should SORF/DORF propose not to allow that? What on earth do you mean by not automatically abstract over fields? Abstraction over fields is the ability to write a function that works on two records with the same field label. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
I'm sorry Greg, I didn't mean to be offensive. I just feel that all your arguments in favour of this restriction are based on one particular application of records, rather than a general notion of what records are. Obviously this application is what motivates you to do all the valuable work you have done, and I appreciate that. But people are going to use records in many different ways, and I don't think that a restriction which makes perfect sense in your application should be allowed to restrict the ways other people want to write programs. Barney. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
Hi Barney, On Fri, Feb 24, 2012 at 2:00 PM, Barney Hilken b.hil...@ntlworld.com wrote: Every one of your messages about records stresses your dislike for polymorphic projections, and your insistence that the Has class should be hidden from the user. I've read all of your explanations, but I'm still totally unconvinced. All your arguments about the semantics of labels are based on the way you want to use them, not on what they are. They are projection functions! Semantically, the only difference between them is the types. Polymorphism makes perfect sense and is completely natural. There is nothing untyped about it. I share Greg's concerns about polymorphic projections. For example, given a function sort :: Ord a = ... we don't allow any 'a' that happens to export a operator that's spelled = to be passed to 'sort'. We have the user explicitly create an instance and thereby defining that their = is e.g. a strict weak ordering and thus make sense when used with 'sort'. This explicitness is useful, it communicates the contract of the function to the reader and lets us catch mistakes in a way that automatically polymorphic projections don't. Automatically polymorphic projections feels like Go's structural polymorphism, C++'s templates or C's automatic numeric coercions, and I'm worried it'll lead to problems when used at scale. They're not required to solve the problem we're trying to solve, so lets hurry slowly and don't bake them in together with the namespacing problem. At the very least use two different LANGUAGE pragmas so users can have one without the other. Cheers, Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
I share Greg's concerns about polymorphic projections. For example, given a function sort :: Ord a = ... we don't allow any 'a' that happens to export a operator that's spelled = to be passed to 'sort'. We have the user explicitly create an instance and thereby defining that their = is e.g. a strict weak ordering and thus make sense when used with 'sort'. This explicitness is useful, it communicates the contract of the function to the reader and lets us catch mistakes in a way that automatically polymorphic projections don't. But the difference is that = is (potentially) an arbitrary function, so we need to be careful that the instances make sense. A formally correct system would insist that all instances are (at least) reflexive and transitive. But in the case of records, we know what the instances are: they are projection functions. Every single (automatically generated) instance does exactly the same thing: it projects out one component of a record. This isn't like OO polymorphism, where messages are actually arbitrary functions which could do anything, the polymorphism is exactly the same as that of fst and snd. At the very least use two different LANGUAGE pragmas so users can have one without the other. This I can agree with. It was the way that Greg mentioned it in every single email which was starting to worry me. Barney. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
Sorry for getting offended! I agree with your sentiment, which is why I thought having to write an extra word in a Record declaration to get automatic abstraction over fields was a good solution to the problem. But now I am in the position of being the lone dissenter on automatic abstraction over fields. And I am in disagreement with SPJ, and I would gladly drop this entire process and just ask him to be a benevolent dictator on the subject. So I am ok with automatic abstraction over fields. Ideally we should have a way to disable it, just as I was proposing we have a way to enable it. But perhaps having this will help force a solution to the circular references issues that makes automatic field abstraction problematic in the first place. After all, Haskell did start with barely any way to do IO. On Fri, Feb 24, 2012 at 2:26 PM, Barney Hilken b.hil...@ntlworld.com wrote: I'm sorry Greg, I didn't mean to be offensive. I just feel that all your arguments in favour of this restriction are based on one particular application of records, rather than a general notion of what records are. Obviously this application is what motivates you to do all the valuable work you have done, and I appreciate that. But people are going to use records in many different ways, and I don't think that a restriction which makes perfect sense in your application should be allowed to restrict the ways other people want to write programs. Barney. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
On Fri, Feb 24, 2012 at 3:07 PM, Barney Hilken b.hil...@ntlworld.com wrote:
But the difference is that = is (potentially) an arbitrary function, so we
need to be careful that the instances make sense. A formally correct system
would insist that all instances are (at least) reflexive and transitive. But
in the case of records, we know what the instances are: they are projection
functions. Every single (automatically generated) instance does exactly the
same thing: it projects out one component of a record. This isn't like OO
polymorphism, where messages are actually arbitrary functions which could
do anything, the polymorphism is exactly the same as that of fst and snd.
I appreciate the difference and it might be enough of a difference
(from e.g. OO systems) that the problems seen there won't show up in
Haskell under a new record system.
Aside: It is possible to have no scalar fields in records of course.
data R = C { compare :: (a - a - Ordering) }
-- Johan
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
On Fri, Feb 24, 2012 at 4:15 PM, Johan Tibell johan.tib...@gmail.com wrote:
Aside: It is possible to have no scalar fields in records of course.
data R = C { compare :: (a - a - Ordering) }
Has x f has no semantic content besides type x having an f field; Ord
has (at least in the programmer's mind, even if the language can't check
it) meaning beyond the simple presence of a compare function.
/g
--
Would you be so kind as to remove the apricots from the mashed potatoes?
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Records in Haskell
On Fri, Feb 24, 2012 at 11:40 PM, Johan Tibell johan.tib...@gmail.com wrote:
Hi Barney,
On Fri, Feb 24, 2012 at 2:00 PM, Barney Hilken b.hil...@ntlworld.com wrote:
Every one of your messages about records stresses your dislike for
polymorphic projections, and your insistence that the Has class should be
hidden from the user. I've read all of your explanations, but I'm still
totally unconvinced. All your arguments about the semantics of labels are
based on the way you want to use them, not on what they are. They are
projection functions! Semantically, the only difference between them is the
types. Polymorphism makes perfect sense and is completely natural. There is
nothing untyped about it.
I share Greg's concerns about polymorphic projections. For example,
given a function
sort :: Ord a = ...
we don't allow any 'a' that happens to export a operator that's
spelled = to be passed to 'sort'. We have the user explicitly create
an instance and thereby defining that their = is e.g. a strict weak
ordering and thus make sense when used with 'sort'. This explicitness
is useful, it communicates the contract of the function to the reader
and lets us catch mistakes in a way that automatically polymorphic
projections don't.
Automatically polymorphic projections feels like Go's structural
polymorphism, C++'s templates or C's automatic numeric coercions, and
I'm worried it'll lead to problems when used at scale. They're not
required to solve the problem we're trying to solve, so lets hurry
slowly and don't bake them in together with the namespacing problem.
At the very least use two different LANGUAGE pragmas so users can have
one without the other.
I agree completely. This is what I like about DORF: the D stands for
Declared, which is referring to the fact that the contracts are
explicit. Record fields aren't automatically polymorphic based on
their name and type, as with SORF, rather they are scoped and
disambiguated in the same way as classes. My only discontentments are
that it requires top-level declarations for each record field, which
feels excessive, and that the polymorphism is opt-out, in other words
if you declare a record with a given field and a field with that
name/type is already in scope, it is automatically considered to be an
instance of the same field. (Not that this is not the same as SORF,
because if more than one field with the same name/type is in scope you
can (and have to) use the usual explicit module qualification to
disambiguate them, and you can write a new top-level field declaration
to make it explicit that you're not re-using the field which was in
scope.)
I think both of these problems could be solved at the same time if (a)
instead of requiring explicit top-level declarations for fields,
declaring a record would also automatically declare its fields, as
distinct from any other fields which may have been in scope, and (b)
there would be some lightweight syntax you could use within record
declarations to specify that you do want to re-use the record field in
scope instead of declaring a new one. This has the added benefit that
record declarations as currently written would continue to have the
same meaning as they currently have. (For the record, I don't see any
harm in also allowing explicit top-level field declarations, outside
of records, it's the requirement for them which seems onerous.)
So in DORF, if you want to declare a Contact record with a name, a
phone number, and an address, you would write:
fieldLabel name :: Text
fieldLabel phoneNumber :: PhoneNumber
fieldLabel address :: Address
data Contact = Contact { name :: Text, phoneNumber :: PhoneNumber,
address :: Address }
-- it's unclear whether the type annotations would be belong in the
field declarations, in the record, or in both
then if you also want to keep track of people as employees, you write
fieldLabel position :: Position
fieldLabel salary :: Word
data Employee = Employee { name :: Text, position :: Position, salary :: Word }
And the name field would automatically be shared between them, and
could be used polymorphically with either record.
but then if you later write...
data City = City { name :: Text}
that would also automatically re-use the name field, but that would
clearly be wrong. It could be avoided by explicitly declaring a new
name field beforehand. (I suppose this aspect of the complaint might
be overblown, because as you can see when want a new field you always
write a fieldLabel declaration, and if you don't you're implying that
you're intending to use the existing one. But it's still very
verbose.)
In my variant of the proposal, declaring the Contact record would look
like this:
data Contact = Contact { name :: Text, position :: Position, salary :: Int }
This would automatically declare the name, position, and salary fields
with their associated types (equivalently to the fieldLabel
declarations from the previous example).
Then for Employee
