Re: [go-nuts] Generics: type-list vs method-based constraints -- not orthogonal?

[Steven Blenkinsop]

Passing in two type parameters, one which is the type used by the caller
and one which implements the interface, would almost work if conversions
were allowed between type parameters with the same underlying types:

https://go2goplay.golang.org/p/ylchcyACuV7

You have to double parameterize everywhere explicitly, though. I'm not
quite clear what the advantage would be compared to passing in the
comparator, other than the tradeoffs for static vs. dynamic.

Another option for abstracting over builtin and custom type cases would be
to add namespaced extension methods. Basically, a package could add methods
to a builtin type which are namespaced by the package, and then interfaces
could qualify their method names with the particular package whose methods
they want to use. Downstream packages would be able to satisfy these
methods only for the types they define, like with normal "free" methods.
This change would potentially have a far reaching impact, but it would
sidestep needing to add a lot of names to the language.

type CustomOrdered(type T) interface {
package.Less(other T) bool
}

func (x int) package.Less(y int) bool {
return x < y
}

It would be nice if you could do this for all types that implement
constraints.Ordered in one definition, but that's a whole other can of
worms.

One way to address this would be if you could
On Thu, Jul 2, 2020 at 7:59 PM, ben...@gmail.com  wrote:

> Hi folks,
>
> This thread  on
> lobste.rs made me wonder if the difference between type-list constraints
> and method-interface constraints is going to cause a bunch of issues or
> code duplication. There's a lack of orthogonality here that makes me uneasy.
>
> For example, the Smallest() function in this proposal takes a slice of
> []T, where T is "type T constraints.Ordered", a type-list based constraint.
> That means Smallest() will only work with the built-in types specified in
> the type list (or types with one of those as an underlying type), and can't
> be used with more complex struct types.
>
> For it to work for other types, you'd have to write a second version of
> Smallest() -- code duplication -- that took a method-interface based
> constraint like so:
>
> type CustomOrdered(type T) interface {
> Less(other T) bool
> }
>
> Arguably Smallest() would still be quite useful even if it only works for
> builtin integer and float point types, but with the type-list Ordered it
> wouldn't work for (say) big.Int, a custom Decimal type, or a more complex
> struct type.
>
> But I think this is worse for ordering-based *data structures* like Tree
> of T. If done with the type-list Ordered, the tree could only store
> built-in types, which isn't that useful. Using a constraint like
> CustomOrdered would work and be more flexible ... but then a basic
> Tree(int) wouldn't work. You'd have to do Tree(MyInt) and convert
> everything from int to MyInt on the way in, and MyInt back to int on the
> way out -- a bunch of type conversion boilerplate.
>
> Or you'd end up writing two versions of your generic Tree: BuiltinTree
> that uses Ordered (type lists), and CustomTree that uses CustomOrdered
> (interface with Less). Not very nice.
>
> (Here's some Go2Go code which shows this for Smallest:
> https://go2goplay.golang.org/p/Rbs374BqPWw)
>
> I'm not sure how the proposal solves this for something like Smallest()
> ... I don't think it does. For ordered data structures, like the Map
> example, it passes in a custom "compare" function to the initializer and
> kind of side-steps the issue. Which isn't bad for data structures as it'd
> still eliminate a lot of code, but it would be kind of a pain for little
> algorithms like Smallest().
>
> Thoughts?
>
> -Ben
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7f471eba-be08-4d32-a50a-22939f63b76dn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANjmGJs8F3m1DrGh6mzGngvCP%3DYBr4EPbiKfc8XAoRgK%2BrZY6A%40mail.gmail.com.

Re: [go-nuts] Generics: type-list vs method-based constraints -- not orthogonal?

[Ben Hoyt]

Thanks. You're right about the Map with the comparison function being
more flexible. That helps allay my concerns for ordered data
structures.

I guess what I'm worried about is for someone writing a "slices"
library and wants to include a function like slices.Smallest(). Do
they give it an Ordered type-list constraint or an constraint with a
Less method? How will they know which of the following to do?

1) Only include Smallest() and hope nobody who uses big.Int comes along.
2) Include two versions, Smallest(type T Ordered)(s []T) for int and
float64 etc, and SmallestCmp(type T)(s []T, func (x, y T) int) so
people can use SmallestCmp(bigs, big.Int.Cmp)?
3) Something else, like Smallest() plus a SmallestCustom() that uses
CustomOrdered

Can you flesh out what you're thinking of w.r.t Smallest() and taking
"an approach like the sort example"?

-Ben


On Fri, Jul 3, 2020 at 3:37 PM Ian Lance Taylor  wrote:
>
> On Thu, Jul 2, 2020 at 4:59 PM ben...@gmail.com  wrote:
> >
> > This thread on lobste.rs made me wonder if the difference between type-list 
> > constraints and method-interface constraints is going to cause a bunch of 
> > issues or code duplication. There's a lack of orthogonality here that makes 
> > me uneasy.
> >
> > For example, the Smallest() function in this proposal takes a slice of []T, 
> > where T is "type T constraints.Ordered", a type-list based constraint. That 
> > means Smallest() will only work with the built-in types specified in the 
> > type list (or types with one of those as an underlying type), and can't be 
> > used with more complex struct types.
> >
> > For it to work for other types, you'd have to write a second version of 
> > Smallest() -- code duplication -- that took a method-interface based 
> > constraint like so:
> >
> > type CustomOrdered(type T) interface {
> > Less(other T) bool
> > }
> >
> > Arguably Smallest() would still be quite useful even if it only works for 
> > builtin integer and float point types, but with the type-list Ordered it 
> > wouldn't work for (say) big.Int, a custom Decimal type, or a more complex 
> > struct type.
> >
> > But I think this is worse for ordering-based *data structures* like Tree of 
> > T. If done with the type-list Ordered, the tree could only store built-in 
> > types, which isn't that useful. Using a constraint like CustomOrdered would 
> > work and be more flexible ... but then a basic Tree(int) wouldn't work. 
> > You'd have to do Tree(MyInt) and convert everything from int to MyInt on 
> > the way in, and MyInt back to int on the way out -- a bunch of type 
> > conversion boilerplate.
> >
> > Or you'd end up writing two versions of your generic Tree: BuiltinTree that 
> > uses Ordered (type lists), and CustomTree that uses CustomOrdered 
> > (interface with Less). Not very nice.
> >
> > (Here's some Go2Go code which shows this for Smallest: 
> > https://go2goplay.golang.org/p/Rbs374BqPWw)
> >
> > I'm not sure how the proposal solves this for something like Smallest() ... 
> > I don't think it does. For ordered data structures, like the Map example, 
> > it passes in a custom "compare" function to the initializer and kind of 
> > side-steps the issue. Which isn't bad for data structures as it'd still 
> > eliminate a lot of code, but it would be kind of a pain for little 
> > algorithms like Smallest().
> >
> > Thoughts?
>
> What you say is true.  I don't know of a way around the basic problem
> other than introducing methods on basic types.  That would be a big
> change to the language, and would require introducing a large number
> of new names.
>
> The way I would suggest handling your Tree example is by making the
> basic data structure take a comparison function.  Then you can write
> tiny wrappers that create the function for any ordered type, or for
> any type with a Less method.  Or people can just pass their own
> comparison function.  You suggest that the Map example sidesteps the
> issue by using a comparison function, which is true, but the Map data
> structure is also more flexible with a comparison function.  For
> example, it lets you store the same type in multiple types with
> different sorting.
>
> The Smallest example is a case where we don't need to store the value,
> so you can take an approach like the sort example in the design draft
> (https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md#sort).
>
> I'm not going to claim that these are perfect solutions, but they seem
> workable to me.  It would be interesting to hear of real cases where
> they are problematic.
>
> Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAL9jXCEmCLnxJE-uOF10ihC6eTXYZNV%2B%3DqBYLyTHYoT4z3Zycw%40mail.gmail.com.

Re: [go-nuts] Generics: type-list vs method-based constraints -- not orthogonal?

[Ian Lance Taylor]

On Thu, Jul 2, 2020 at 4:59 PM ben...@gmail.com  wrote:
>
> This thread on lobste.rs made me wonder if the difference between type-list 
> constraints and method-interface constraints is going to cause a bunch of 
> issues or code duplication. There's a lack of orthogonality here that makes 
> me uneasy.
>
> For example, the Smallest() function in this proposal takes a slice of []T, 
> where T is "type T constraints.Ordered", a type-list based constraint. That 
> means Smallest() will only work with the built-in types specified in the type 
> list (or types with one of those as an underlying type), and can't be used 
> with more complex struct types.
>
> For it to work for other types, you'd have to write a second version of 
> Smallest() -- code duplication -- that took a method-interface based 
> constraint like so:
>
> type CustomOrdered(type T) interface {
> Less(other T) bool
> }
>
> Arguably Smallest() would still be quite useful even if it only works for 
> builtin integer and float point types, but with the type-list Ordered it 
> wouldn't work for (say) big.Int, a custom Decimal type, or a more complex 
> struct type.
>
> But I think this is worse for ordering-based *data structures* like Tree of 
> T. If done with the type-list Ordered, the tree could only store built-in 
> types, which isn't that useful. Using a constraint like CustomOrdered would 
> work and be more flexible ... but then a basic Tree(int) wouldn't work. You'd 
> have to do Tree(MyInt) and convert everything from int to MyInt on the way 
> in, and MyInt back to int on the way out -- a bunch of type conversion 
> boilerplate.
>
> Or you'd end up writing two versions of your generic Tree: BuiltinTree that 
> uses Ordered (type lists), and CustomTree that uses CustomOrdered (interface 
> with Less). Not very nice.
>
> (Here's some Go2Go code which shows this for Smallest: 
> https://go2goplay.golang.org/p/Rbs374BqPWw)
>
> I'm not sure how the proposal solves this for something like Smallest() ... I 
> don't think it does. For ordered data structures, like the Map example, it 
> passes in a custom "compare" function to the initializer and kind of 
> side-steps the issue. Which isn't bad for data structures as it'd still 
> eliminate a lot of code, but it would be kind of a pain for little algorithms 
> like Smallest().
>
> Thoughts?

What you say is true.  I don't know of a way around the basic problem
other than introducing methods on basic types.  That would be a big
change to the language, and would require introducing a large number
of new names.

The way I would suggest handling your Tree example is by making the
basic data structure take a comparison function.  Then you can write
tiny wrappers that create the function for any ordered type, or for
any type with a Less method.  Or people can just pass their own
comparison function.  You suggest that the Map example sidesteps the
issue by using a comparison function, which is true, but the Map data
structure is also more flexible with a comparison function.  For
example, it lets you store the same type in multiple types with
different sorting.

The Smallest example is a case where we don't need to store the value,
so you can take an approach like the sort example in the design draft
(https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md#sort).

I'm not going to claim that these are perfect solutions, but they seem
workable to me.  It would be interesting to hear of real cases where
they are problematic.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXH9oMLS-0yJeZYmOP3S3vPhDS-17w9oz21ACnD%3DADBvw%40mail.gmail.com.

Re: [go-nuts] maptype questions

[Ian Lance Taylor]

On Thu, Jul 2, 2020 at 5:34 PM arthurwil...@gmail.com
 wrote:
>
> On Monday, June 29, 2020 at 10:41:15 PM UTC-5 Ian Lance Taylor wrote:
>>
>> On Mon, Jun 29, 2020 at 6:33 PM arthurwil...@gmail.com
>>  wrote:
>> >
>> > On Monday, June 29, 2020 at 8:16:14 PM UTC-5 Ian Lance Taylor wrote:
>> >>
>> >> On Mon, Jun 29, 2020 at 5:32 PM Bill Morgan
>> >>  wrote:
>> >> >
>> >> > for this code:
>> >> >
>> >> > m := make(map[int]int, 9)
>> >> >
>> >> > I think the compiler creates a maptype that is stored at 
>> >> > type.*+60480(SB) that it uses for the call to runtime.makemap:
>> >> >
>> >> > m := make(map[int]int, 9)
>> >> > 0x10d0b81 488d05f8ec LEAQ type.*+60480(SB), AX
>> >> > 0x10d0b88 48890424 MOVQ AX, 0(SP)
>> >> > 0x10d0b8c 48c74424080900 MOVQ $0x9, 0x8(SP)
>> >> > 0x10d0b95 48c7442410 MOVQ $0x0, 0x10(SP)
>> >> > 0x10d0b9e 6690 NOPW
>> >> > 0x10d0ba0 e8fbdff3ff CALL runtime.makemap(SB)
>> >> >
>> >> > Where is the code in the compiler that creates the maptype? I'm 
>> >> > wondering how the bucket size is computed and how the rest of the 
>> >> > maptype structure is filled out.
>> >>
>> >> https://golang.org/src/cmd/compile/internal/gc/reflect.go#L189
>> >>
>> >> > Is there a way to print out the type info stored in the binary?
>> >>
>> >> I'm not sure if this is what you want, but you can use reflect.TypeOf:
>> >> https://play.golang.org/p/FEUIoqc6SMU .
>> >>
>> >> Ian
>> >
>> > Thanks Ian, that is pretty cool. I think that is printing out the 
>> > map[int]int type though instead of the *maptype that is generated by the 
>> > compiler and passed to runtime.makemap.
>> >
>> > I think this is loading a pointer to a maptype into AX to pass as the 
>> > first arg to runtime.makemap. The maptype seems to be stored at 
>> > type.*+60480(SB).
>> >
>> > LEAQ type.*+60480(SB), AX
>> >
>> > here's the type I'm asking about: where is this guy created in the 
>> > compiler?
>> >
>> > type maptype struct {
>> > typ _type
>> > key *_type
>> > elem *_type
>> > bucket *_type // internal type representing a hash bucket
>> > // function for hashing keys (ptr to key, seed) -> hash
>> > hasher func(unsafe.Pointer, uintptr) uintptr
>> > keysize uint8 // size of key slot
>> > elemsize uint8 // size of elem slot
>> > bucketsize uint16 // size of bucket
>> > flags uint32
>> > }
>>
>> https://golang.org/src/cmd/compile/internal/gc/reflect.go#L1285
>>
>> Ian
>
>
> Where is the maptype.bucket.size computed?
>
> It is used here in makemap (t.bucket.size):
>
> https://golang.org/src/runtime/map.go#L304
>
> I'm assuming the bucket is created here in bmap but I couldn't figure out how 
> the bucket.size is computed and set:
>
> https://golang.org/src/cmd/compile/internal/gc/reflect.go#L82

The bucket type is an ordinary struct, and the runtime code is looking
at the size field of the bucket struct's type descriptor.  That is the
value returned by bmap in cmd/compile/internal/gc/reflect.go.  The
size field is set by the call to dowidth(bucket), which the size as is
done for any struct type.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcV9d5530q3u_96Vps87gdcQJH%2BXFY5Cf9gigrpN4BMvxg%40mail.gmail.com.

Re: [go-nuts] maptype questions

[arthurwil...@gmail.com]

On Monday, June 29, 2020 at 10:41:15 PM UTC-5 Ian Lance Taylor wrote:

> On Mon, Jun 29, 2020 at 6:33 PM arthurwil...@gmail.com 
>  wrote: 
> > 
> > On Monday, June 29, 2020 at 8:16:14 PM UTC-5 Ian Lance Taylor wrote: 
> >> 
> >> On Mon, Jun 29, 2020 at 5:32 PM Bill Morgan 
> >>  wrote: 
> >> > 
> >> > for this code: 
> >> > 
> >> > m := make(map[int]int, 9) 
> >> > 
> >> > I think the compiler creates a maptype that is stored at 
> type.*+60480(SB) that it uses for the call to runtime.makemap: 
> >> > 
> >> > m := make(map[int]int, 9) 
> >> > 0x10d0b81 488d05f8ec LEAQ type.*+60480(SB), AX 
> >> > 0x10d0b88 48890424 MOVQ AX, 0(SP) 
> >> > 0x10d0b8c 48c74424080900 MOVQ $0x9, 0x8(SP) 
> >> > 0x10d0b95 48c7442410 MOVQ $0x0, 0x10(SP) 
> >> > 0x10d0b9e 6690 NOPW 
> >> > 0x10d0ba0 e8fbdff3ff CALL runtime.makemap(SB) 
> >> > 
> >> > Where is the code in the compiler that creates the maptype? I'm 
> wondering how the bucket size is computed and how the rest of the maptype 
> structure is filled out. 
> >> 
> >> https://golang.org/src/cmd/compile/internal/gc/reflect.go#L189 
> >> 
> >> > Is there a way to print out the type info stored in the binary? 
> >> 
> >> I'm not sure if this is what you want, but you can use reflect.TypeOf: 
> >> https://play.golang.org/p/FEUIoqc6SMU . 
> >> 
> >> Ian 
> > 
> > Thanks Ian, that is pretty cool. I think that is printing out the 
> map[int]int type though instead of the *maptype that is generated by the 
> compiler and passed to runtime.makemap. 
> > 
> > I think this is loading a pointer to a maptype into AX to pass as the 
> first arg to runtime.makemap. The maptype seems to be stored at 
> type.*+60480(SB). 
> > 
> > LEAQ type.*+60480(SB), AX 
> > 
> > here's the type I'm asking about: where is this guy created in the 
> compiler? 
> > 
> > type maptype struct { 
> > typ _type 
> > key *_type 
> > elem *_type 
> > bucket *_type // internal type representing a hash bucket 
> > // function for hashing keys (ptr to key, seed) -> hash 
> > hasher func(unsafe.Pointer, uintptr) uintptr 
> > keysize uint8 // size of key slot 
> > elemsize uint8 // size of elem slot 
> > bucketsize uint16 // size of bucket 
> > flags uint32 
> > } 
>
> https://golang.org/src/cmd/compile/internal/gc/reflect.go#L1285 
>
> Ian 
>

Where is the maptype.bucket.size computed? 

It is used here in makemap (t.bucket.size):

https://golang.org/src/runtime/map.go#L304

I'm assuming the bucket is created here in bmap but I couldn't figure out 
how the bucket.size is computed and set:

https://golang.org/src/cmd/compile/internal/gc/reflect.go#L82

 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/eec936f0-731b-4093-a75c-3e452380df54n%40googlegroups.com.

[go-nuts] Generics: type-list vs method-based constraints -- not orthogonal?

[ben...@gmail.com]

Hi folks,

This thread  on 
lobste.rs made me wonder if the difference between type-list constraints 
and method-interface constraints is going to cause a bunch of issues or 
code duplication. There's a lack of orthogonality here that makes me uneasy.

For example, the Smallest() function in this proposal takes a slice of []T, 
where T is "type T constraints.Ordered", a type-list based constraint. That 
means Smallest() will only work with the built-in types specified in the 
type list (or types with one of those as an underlying type), and can't be 
used with more complex struct types.

For it to work for other types, you'd have to write a second version of 
Smallest() -- code duplication -- that took a method-interface based 
constraint like so:

type CustomOrdered(type T) interface {
Less(other T) bool
}

Arguably Smallest() would still be quite useful even if it only works for 
builtin integer and float point types, but with the type-list Ordered it 
wouldn't work for (say) big.Int, a custom Decimal type, or a more complex 
struct type.

But I think this is worse for ordering-based *data structures* like Tree of 
T. If done with the type-list Ordered, the tree could only store built-in 
types, which isn't that useful. Using a constraint like CustomOrdered would 
work and be more flexible ... but then a basic Tree(int) wouldn't work. 
You'd have to do Tree(MyInt) and convert everything from int to MyInt on 
the way in, and MyInt back to int on the way out -- a bunch of type 
conversion boilerplate.

Or you'd end up writing two versions of your generic Tree: BuiltinTree that 
uses Ordered (type lists), and CustomTree that uses CustomOrdered 
(interface with Less). Not very nice.

(Here's some Go2Go code which shows this for Smallest: 
https://go2goplay.golang.org/p/Rbs374BqPWw)

I'm not sure how the proposal solves this for something like Smallest() ... 
I don't think it does. For ordered data structures, like the Map example, 
it passes in a custom "compare" function to the initializer and kind of 
side-steps the issue. Which isn't bad for data structures as it'd still 
eliminate a lot of code, but it would be kind of a pain for little 
algorithms like Smallest().

Thoughts?

-Ben

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7f471eba-be08-4d32-a50a-22939f63b76dn%40googlegroups.com.

[go-nuts] go/build - how to locate source without making network calls?

[James Lawrence]

I've been playing with the go/build package for resolving go package 
information for a few projects of mine and noticed some unexpected behavior.

mainly it seems to reach out to the network even when all the information 
is available locally. if I turn off my network my code fails whereas go 
install seems to continue to function normally even without a network.

is there a better way to resolve go packages locally than 
`build.Import(...)`?

importer looks promising but it seems that it is currently impossible to 
provide a build.Context to an importer because the srcimporter is inside an 
internal package.

https://github.com/golang/go/blob/master/src/go/importer/importer.go#L65


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0f620046-1f51-44bc-bc3d-0c017c7d97ben%40googlegroups.com.

[go-nuts] Re: Go program getting SIGSEGV as it panics and runs down after a SIGTERM or a SIGINT signal

[Charles Hathaway]

Hey Steve,

I'm no expert here, but the thing that sticks out to me is the garbage 
collection stacks in there, which suggests things are crashing while GC is 
running. Does the frequency change if you adjust the garbage collection 
frequency?

Charles

On Thursday, July 2, 2020 at 2:18:31 PM UTC-7 est...@yottadb.com wrote:

> We have developed a Go API (which we call a Go wrapper - 
> https://pkg.go.dev/lang.yottadb.com/go/yottadb?tab=doc) to the YottaDB 
> hierarchical key-value database (https://yottadb.com). It works well for 
> the most part, but there are some edge cases during process shutdown that 
> are standing between us and a full production-grade API. We have a test 
> program called threeenp1C2 that is an implementation of the classic 3n+1 
> problem that does some database-intensive activity. It comes in two 
> flavors: a multi-process version (1 [application] goroutine per process) 
> and a single process version (multiple [application] goroutines in one 
> process). The latter runs fine; the discussion below is about the 
> multi-process version.
>
> A Go main spawns 15 copies of itself as workers, each of which runs an 
> internal routine. Each process computes the lengths of 3n+1 sequences for a 
> block of integers, and then returns for another block of integers. The 
> results are stored in the database so that processes can use the results of 
> other processes' work. When they finish computing the results for the 
> problem, they shut down. So, for example, the overall problem may be to 
> compute the 3n+1 sequences for all integers from one through a million, 
> with processes working on blocks of 10,000 initial integers. There is 
> nothing special about the computation other than that it generates a lot of 
> database activity.
>
> This version runs fine, and always shuts down cleanly if allowed to run to 
> completion. But since the database is used for mission critical 
> applications, we have a number of stress tests. The threeenp1C2 test 
> involves starting 15 cooperative worker processes, and then sending each 
> process a SIGTERM or SIGINT, depending on the test. Sporadically, one of 
> the processes receiving the signal shuts down generating a core file due to 
> a SIGSEGV instead of shutting down cleanly. That's the 10,000 ft view. Here 
> are more details:
>
>- The database engine is daemonless, and runs in the address space of 
>each process, with processes cooperating with one another to manage the 
>database using a variety of semaphores and data structures in shared 
> memory 
>segments, as well as OS semaphores and mutexes. The database engine is 
>single-threaded, and when multiple threads in a process call the database 
>engine, there are mechanisms to ensure that non-reentrant code is called 
>without reentrancy.
>- The database engine is written in C, and traditionally has had a 
>heavy reliance on signals but with the Go wrapper calling the engine 
>through cgo, things were a bit dicey. So code was reworked for use with Go 
>such that Go now handles the signals and lets the YottaDB engine know 
> about 
>them. To that end, a goroutine for each signal type we want to know about 
>(around 17 of them) is started up each of which then call into a "signal 
>dispatcher" in the C code to drive signal handlers for those signals we 
> are 
>notifed of.
>- When a fatal signal such as a SIGTERM occurs, the goroutines started 
>for signal handling are all told to close down and we wait for them to 
> shut 
>down before driving a panic to stop the world (doing this reduced the 
>failures from their previous failure rate of nearly 100%). The current 
>failure rate with the core dumps now occurs 3-10% of the time. These 
>strange failures are in the bowels of Go (usually in either a futex call 
> or 
>something called newstack?). Empirical evidence suggests the failure 
>rate increases when the system is loaded - I guess thus affecting the 
>timing of how/when things shutdown though proof is hard to come by.
>- The database engine uses optimistic concurrency control to implement 
>ACID transactions. What this means is that Go application code (say 
> Routine 
>A) calls the database engine through CGO, passing it a Go entry point (say 
>Routine B) that the database engine calls one or more times till it 
>completes the transaction (to avoid live-locks, in a final retry, should 
>one be required, the engine locks out all other accesses, essentially 
>single-threading the database). Routine B itself calls into the YottaDB 
> API 
>through cgo. So mixed stacks of C and Go code are common.
>- To avoid endangering database integrity, the engine attempts to shut 
>down at “safe”points. If certain fatal signals are received at an unsafe 
>spot, we defer handling the signal till it is in a safe place. To ensure 
>that everything stops when it 

[go-nuts] Go program getting SIGSEGV as it panics and runs down after a SIGTERM or a SIGINT signal

[estess]

We have developed a Go API (which we call a Go wrapper - 
https://pkg.go.dev/lang.yottadb.com/go/yottadb?tab=doc) to the YottaDB 
hierarchical key-value database (https://yottadb.com). It works well for 
the most part, but there are some edge cases during process shutdown that 
are standing between us and a full production-grade API. We have a test 
program called threeenp1C2 that is an implementation of the classic 3n+1 
problem that does some database-intensive activity. It comes in two 
flavors: a multi-process version (1 [application] goroutine per process) 
and a single process version (multiple [application] goroutines in one 
process). The latter runs fine; the discussion below is about the 
multi-process version.

A Go main spawns 15 copies of itself as workers, each of which runs an 
internal routine. Each process computes the lengths of 3n+1 sequences for a 
block of integers, and then returns for another block of integers. The 
results are stored in the database so that processes can use the results of 
other processes' work. When they finish computing the results for the 
problem, they shut down. So, for example, the overall problem may be to 
compute the 3n+1 sequences for all integers from one through a million, 
with processes working on blocks of 10,000 initial integers. There is 
nothing special about the computation other than that it generates a lot of 
database activity.

This version runs fine, and always shuts down cleanly if allowed to run to 
completion. But since the database is used for mission critical 
applications, we have a number of stress tests. The threeenp1C2 test 
involves starting 15 cooperative worker processes, and then sending each 
process a SIGTERM or SIGINT, depending on the test. Sporadically, one of 
the processes receiving the signal shuts down generating a core file due to 
a SIGSEGV instead of shutting down cleanly. That's the 10,000 ft view. Here 
are more details:

   - The database engine is daemonless, and runs in the address space of 
   each process, with processes cooperating with one another to manage the 
   database using a variety of semaphores and data structures in shared memory 
   segments, as well as OS semaphores and mutexes. The database engine is 
   single-threaded, and when multiple threads in a process call the database 
   engine, there are mechanisms to ensure that non-reentrant code is called 
   without reentrancy.
   - The database engine is written in C, and traditionally has had a heavy 
   reliance on signals but with the Go wrapper calling the engine through cgo, 
   things were a bit dicey. So code was reworked for use with Go such that Go 
   now handles the signals and lets the YottaDB engine know about them. To 
   that end, a goroutine for each signal type we want to know about (around 17 
   of them) is started up each of which then call into a "signal dispatcher" 
   in the C code to drive signal handlers for those signals we are notifed of.
   - When a fatal signal such as a SIGTERM occurs, the goroutines started 
   for signal handling are all told to close down and we wait for them to shut 
   down before driving a panic to stop the world (doing this reduced the 
   failures from their previous failure rate of nearly 100%). The current 
   failure rate with the core dumps now occurs 3-10% of the time. These 
   strange failures are in the bowels of Go (usually in either a futex call or 
   something called newstack?). Empirical evidence suggests the failure 
   rate increases when the system is loaded - I guess thus affecting the 
   timing of how/when things shutdown though proof is hard to come by.
   - The database engine uses optimistic concurrency control to implement 
   ACID transactions. What this means is that Go application code (say Routine 
   A) calls the database engine through CGO, passing it a Go entry point (say 
   Routine B) that the database engine calls one or more times till it 
   completes the transaction (to avoid live-locks, in a final retry, should 
   one be required, the engine locks out all other accesses, essentially 
   single-threading the database). Routine B itself calls into the YottaDB API 
   through cgo. So mixed stacks of C and Go code are common.
   - To avoid endangering database integrity, the engine attempts to shut 
   down at “safe”points. If certain fatal signals are received at an unsafe 
   spot, we defer handling the signal till it is in a safe place. To ensure 
   that everything stops when it reaches a safe place, the engine calls back 
   into Go and drives a panic of choice to shut down the entire process.  I 
   find myself wondering if the "sandwich stack" of C and Go routines is 
   somehow causing the panic to generate cores.
   
These failures are like nothing I've seen. Sometimes one of the threads 
that we create in the C code are still around (it's been told to shutdown 
but evidently hasn't gotten around to it yet - like thread 11 in the gdb 
trace below). 

[go-nuts] No http.ServeStream() ?

[Liam]

Wondering why there's no http.ServeStream() API to do chunked 
transfer-encoding (or the http2 equivalent), in addition to the nice 
features of ServeContent()...

Has it been considered? Does it merit a proposal?

If not, some example code in the docs would be helpful...

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ccb73194-b810-4a83-88f1-8216bdfe6e4fo%40googlegroups.com.

[go-nuts] proposal: Go 2: error handling: set error as return value by default from function

[Александр Аверьянов]

Set Error as return value by default from function 



   
Go 2 language change template:
I am a novice Go programmer.
I have also some experience with JS, Java, SQL.
These changes simplifies working with error, do cleaner code and allows to 
not remember to pass the error up the stack.
I don't find idea one like it from proposed before.
This proposal is without implementation.

Orthogonality: how does this change interact or overlap with existing 
features? - complements existing.

Is the goal of this change a performance improvement? - for developers.

What quantifiable improvement should we expect? - Reduce work with errors 
only to determine the place of their processing.

Does this affect error handling? - Yes.

How does this differ from previous error handling proposals? - complements 
the existing implementation of functions.

Is this about generics? - No.


In "What to avoid":

Other languages have not it.

My proposal states a idea rather than a solution.


  Idea

All functions can be return Error by default. 

Its mean that all functions always return error = nil and if catch, than 
error != nil. 

In the place where error handling is required, then call the handling 
function (handling() or ifError() or checkError() or 
someNameFunctionErrorHandling() and etc.), here let it be checkError(). 


  Overview 

func anyFunc() error {var err error; return err} == func anyFunc() {}
 

so then


main()
|
˅
func() // if need error processing - "Any Error"
| ^
˅ |
func() // default forwarding error
| ^
˅ |
func() // getting an error - "Any Error"



Please see examples for understanding.



  Examples

Example 1 

Old

func doAny() (int, error) {
  if true{  // simulate getting an error
return 0, Errors.New("Any Error")
  } 
  return 1, nil
}
main() {
  _, err := doAny()
  if err != nil {  // error processing
fmt.Println(err)  // "Any Error"
  }
}

New (at first glance, in the simple case, almost nothing has changed)

func doAny() int {
  if true{  // simulate getting an error
return Errors.New("Any Error"), 0  // allways error is the first returned 
value
  }
  return 1
}
main() {
  i := doAny()
  checkError() {  // error processing
fmt.Println(err)  // "Any Error"
  }
}

Example 2 

Old

func doAny() (int, error) {
  if true{  // simulate getting an error
return 0, Errors.New("First Error")
  } 
  return 1, nil
}
main() {defer func() {
  if err != nil {  // last error processing
fmt.Println(err)  // "Second Error"
  }
}()
  _, err := doAny()
  if err != nil {  // current error processing from doAny()
fmt.Println(err)  // "First Error"
err = Errors.New("Second Error")
  }
}

New (we are able to immediately process the first error encountered)

func doAny() int {
  if true{  // simulate getting an error
return Errors.New("First Error"), 0
  }
  return 1
}
main() {
  checkError() {  // first error processing
fmt.Println(err)  // "First Error"
  }

  defer checkError() {  // last error processing
fmt.Println(err)  // "Second Error"
  }
  i := doAny()
  checkError() {  // current error processing from doAny()
fmt.Println(err)  //  "First Error"
  }
  err = Errors.New("Second Error")
}

Example 3 

Old

func doAny() (int, error) {
  if true{  // simulate getting an error
return 0, Errors.New("Any Error")
  } 
  return 1, nil
}
func doNext() error {
  ...
  _, err := doAny()
  if err != nil {
return err
  }
  return nil
}
main() {
  err := doNext()
  if err != nil {  // error processing
fmt.Println(err)  // "Any Error"
  }
}

New (we can forget to forward the error, but not here)

func doAny() int {
  if true{  // simulate getting an error
return Errors.New("Any Error"), 0
  }
  return 1
}
func doNext() {
  ...
  doAny()
}  // just forwarding last error
main() {
  _ = doNext()

  checkError() {  // error processing
fmt.Println(err)  // "Any Error"
err = nil  // and can forwarding nil as error
  }
}



Thank you.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/68ce2a90-8f30-4141-bb15-67d793b21b95o%40googlegroups.com.

[go-nuts] Why does html.UnescapeString not look for the trailing semicolon when finding HTML entities?

[Akash]

html.UnescapeString("Should this word, , be unescaped?")  

https://play.golang.org/p/vN5bvfooq8H

Aren't HTML entities supposed to end with a semicolon? See 
https://developer.mozilla.org/en-US/docs/Glossary/Entity

I couldn't see any edge cases mentioned in the source 
 .

Thanks.


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/baea05b9-6634-495b-a45f-78f02ec7a20bn%40googlegroups.com.

Re: [go-nuts] godoc.org pkg.go.dev banner feedback

[Russ Cox]

Thanks for the feedback Jan. Google is on an extended holiday weekend for
the US holiday but we'll definitely chase this down when people are back.

Best,
Russ


On Thu, Jul 2, 2020 at 11:38 AM Jan Mercl <0xj...@gmail.com> wrote:

> godoc.org displays packages from the domain modernc.org just fine. It
> also adds a banner sayings, for example at
> https://godoc.org/modernc.org/cc/v3
>
> "Pkg.go.dev is a new destination for Go discovery & docs.
> Check it out at  pkg.go.dev/modernc.org/cc/v3 and share your
> feedback."
>
> However, clicking that link (ie. https://pkg.go.dev/modernc.org/cc/v3)
> is a 404. In fact, all modernc.org packages I have tried just minutes
> ago, about 20 of them, are 404.
>
> The feedback is: Please either fix pkg.go.dev or do not put the banner
> in godoc that does not work and may confuse package users. FTR, it
> seems all the modernc.org packages are still present in the pkg.do.dev
> search index and I remember that the packages used to be shown there.
> I have not asked anyone to remove them, so I suppose some glitch must
> have happened somewhere.
>
> I have tried to follow the instructions found here:
> https://go.dev/about#adding-a-package but I see no change. However,
> it's not clear if there's some delay involved between triggering the
> process and the results being available. I suggest adding that
> particular piece of information to the instructions.
>
> I'd appreciate a reply from anyone having insight into this. Thanks in
> advance.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAA40n-Uaq651HywVteubB6CCre_UbfXofPcOZaESJ3V_y1gswA%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA8EjDQ9kP-4_j_RvWZq-Eu5Nf0ZTssaN0fueuMFd_2D9QC%3DJA%40mail.gmail.com.

Re: [go-nuts] Is it necessary to start function names within main package with capitalized letter?

[Jake Montgomery]

On Wednesday, June 24, 2020 at 12:46:42 PM UTC-4, Axel Wagner wrote:
>
> Personally, I tend to do it. Note that it doesn't actually make a 
> difference from a technical perspective - main can't be imported, so it 
> makes no real sense to export any identifiers. 
>
 
There is a case where main can be imported. You can import main from a test 
in the same directory, but with a different package name. AFAICT, this 
allows 'external' testing of executable code. In that case what identifiers 
are exported does matter. But since most of us will probably never use this 
feature, it probably does not matter much in practice. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7e4c307f-cd98-453d-b8bd-e685062830b8o%40googlegroups.com.

[go-nuts] godoc.org pkg.go.dev banner feedback

[Jan Mercl]

godoc.org displays packages from the domain modernc.org just fine. It
also adds a banner sayings, for example at
https://godoc.org/modernc.org/cc/v3

"Pkg.go.dev is a new destination for Go discovery & docs.
Check it out at  pkg.go.dev/modernc.org/cc/v3 and share your
feedback."

However, clicking that link (ie. https://pkg.go.dev/modernc.org/cc/v3)
is a 404. In fact, all modernc.org packages I have tried just minutes
ago, about 20 of them, are 404.

The feedback is: Please either fix pkg.go.dev or do not put the banner
in godoc that does not work and may confuse package users. FTR, it
seems all the modernc.org packages are still present in the pkg.do.dev
search index and I remember that the packages used to be shown there.
I have not asked anyone to remove them, so I suppose some glitch must
have happened somewhere.

I have tried to follow the instructions found here:
https://go.dev/about#adding-a-package but I see no change. However,
it's not clear if there's some delay involved between triggering the
process and the results being available. I suggest adding that
particular piece of information to the instructions.

I'd appreciate a reply from anyone having insight into this. Thanks in advance.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA40n-Uaq651HywVteubB6CCre_UbfXofPcOZaESJ3V_y1gswA%40mail.gmail.com.