Re: [go-nuts] Type func binding and interfaces

[Louki Sumirniy]

https://stackoverflow.com/questions/6531543/efficient-implementation-of-binary-heaps

Pretty much what I'm working on here is this, except with left to right 
sort instead of vertical. I think this guy's work will help me iron out the 
performance issues.

Another thing, that is more on topic more specifically, is that collections 
of interface methods introduce a significant overhead, compared to simply 
passing the pointer to the structure as a parameter. I am thinking that 
maybe a way to hide this parameter passing is by using closures, which bind 
in the namespace from a hypothetical initialiser function, without actually 
having to specify the pointer passing across. The structure is created and 
allocated by the initialising function (constructor, if you like) and 
because all of the functions are declared within the namespace as closures, 
the compiler implicitly passes the pointer to the struct without having to 
specify it.

I don't know exactly what FP paradigm says about structures and collections 
of functions exactly, as prima facie it looks like OOP. But closures are a 
uniquely FP mechanism, and really they are just a way to merge namespaces, 
and by doing this we don't have to pass the pointer to the function 
explicitly, as it is already accessible.

At least this is the tack I'm taking for today's outing into attempting to 
create a binary search tree with cache data locality. To some extent the 
benefits of this have been demonstrated with some other, relatively new 
algorithms like the one linked to above, and many of the same principles 
will apply. Index starting at shortens the walk functions significantly. 
Also, there is a question about the nil value for nodes. As someone pointed 
out, a non-trivial amount of hash results will be zero. Unless I'm 
mistaken, this also means that at that point, a hashchain would repeat, as 
this common originating value necessarily produces a common output. So in 
fact, I can use the zero sentinel or nil node payload, because we can 
consider the hashchain terminated if it does make a zero, and there is no 
point going any further as the remainder of elements of such a hashchain 
will be the exact same pattern as starting from a zero nonce.

On Tuesday, 24 April 2018 17:30:30 UTC+3, matthe...@gmail.com wrote:
>
> I'd suggest starting with the basic algorithm without any abstraction 
>> (just hard-code in the type you want to store), then benchmark/tweak 
>> the algorithm, and only then try to make it general. 
>
>
> This is my conclusion too. Abstracting the code is a lot of churn if we’re 
> not sure performance without abstraction works. We’ll be able to help on 
> the Go front better with a foundation.
>
> Matt
>
> On Tuesday, April 24, 2018 at 9:22:21 AM UTC-5, Louki Sumirniy wrote:
>>
>> Reading through the wikipedia description of a heap, and especially a 
>> binary heap... it's a heap. But that's not a sexy name! Technically it's 
>> not a heap because it sorts left to right, heaps sort bottom to top.
>>
>> I am stripping down my code and directly declaring the struct variables 
>> as function types, and I will be removing the method-interface completely, 
>> as I don't need it. I will be able to then isolate the external interface 
>> functions and 'hide' the internals while being able to easily link the 
>> scope of functions to the structure.
>>
>> If I really need a secondary value to flag empty or not, then the memory 
>> efficient way would be to have a bit-indexed flag for this property, so for 
>> every 32 bits of data one bit of flag and keep this in an orthogonal entry 
>> in the structure. You are correct, though the odds are small, probably 
>> 1/2^40 for 32 bit hashes making zeros, that kinda works out to 2^32/2^40 
>> frequency of all zeroes and I think that amounts to 0.390625% being zero. 
>> That would have been a problem! So for 30 rows of 32 bits I will need 2^30 
>> array elements, 2^30/8 bytes to store the empty/fill. I can't think of a 
>> more space efficient way to store this property. About 134Mb would be 
>> required for this, based on 30 rows. It can, however, be extended at the 
>> same time as the tree so its size will keep this proportion linear to the 
>> size of the tree array, it's not a big overhead compared to the 4Gb the 30 
>> row store will use.
>>
>> If it seems like this is excessive numbers, this stores 1/4 of the total 
>> magnitude of possible values of 32 bit hashes, or half hashes. In the 
>> application I have in mind, probably most of the time trees won't grow to 
>> more than about 15 rows anyway, which is not that much, before the required 
>> pattern is found in a hashchain, at least until the number of miners on the 
>> network gets really big. I am writing a Proof of Work algorithm and I was 
>> aiming for something that would scale down to short time periods at low 
>> difficulty.
>>
>> This tree store would satisfy that by allowing solutions to pop out at a 
>> particular sequence in a 

Re: [go-nuts] Type func binding and interfaces

[roger peppe]

On 25 April 2018 at 08:05, Louki Sumirniy
 wrote:
> https://stackoverflow.com/questions/6531543/efficient-implementation-of-binary-heaps
>
> Pretty much what I'm working on here is this, except with left to right sort
> instead of vertical. I think this guy's work will help me iron out the
> performance issues.

You do know that heaps aren't a great data structure for searching, right?

> Another thing, that is more on topic more specifically, is that collections
> of interface methods introduce a significant overhead, compared to simply
> passing the pointer to the structure as a parameter. I am thinking that
> maybe a way to hide this parameter passing is by using closures, which bind
> in the namespace from a hypothetical initialiser function, without actually
> having to specify the pointer passing across. The structure is created and
> allocated by the initialising function (constructor, if you like) and
> because all of the functions are declared within the namespace as closures,
> the compiler implicitly passes the pointer to the struct without having to
> specify it.

You don't need to do this. You're still thinking in traditional OO
terms. I'd suggest trying to think in a more Go like way instead. FWIW
I tried to point you in a more sensible direction with this code:
https://play.golang.org/p/wv0T8T8Ynns

> I don't know exactly what FP paradigm says about structures and collections
> of functions exactly, as prima facie it looks like OOP. But closures are a
> uniquely FP mechanism, and really they are just a way to merge namespaces,
> and by doing this we don't have to pass the pointer to the function
> explicitly, as it is already accessible.

What gives you the idea that closures are a uniquely FP mechanism?

  rog.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Type func binding and interfaces

[Louki Sumirniy]

I think that it's not necessarily non-idiomatic to use closures instead of 
interfaces in Go, it's more that Go has had interfaces longer than it's had 
closures, and so more code has been written this way.

In Angular 2+ you have the option of embedding HTML, CSS and TS code inside 
one file, or instead having 4, with the main file importing the other 
three. I don't like this for several reasons, but mainly because it makes a 
more complex interlinking between them, and I think even you can say that 
if your application is big enough, all this extra import work will also 
cost a lot of time, though in Go of course that time is not so big in the 
first place due to how efficient its compiler is.

The way I see it is that closures and interfaces are two ways to do exactly 
the same thing, binding namespaces together. One requires more accessory 
declarations and lines of code, not a huge extra overhead, but then for 
exported stuff - well, this is the one upside of it but I don't think it is 
that great, gofmt wants to see header comments on every exported function. 
Which is a nice idea, in theory, but in my opinion if you need comments 
your naming scheme sucks.

That's also why I created distinct comparators with meaningful names 
instead of creating named return values. b.IsEqual(data, cursor) compared 
to b.Compare(data, cursor) == EQUAL is not just a lot longer, but harder to 
read. I think technically it may also consume more code cache space to 
implement this extra operation, though on the other side, there is a small 
extra overhead for having three instead of 1. The compare function has to 
run three cases, most concisely expressed as 

switch{
case b.Store(c.index)>d: 
return GREATER; 
case b.Store(c.index)https://queue.acm.org/detail.cfm?id=1814327 and I 
found this story from this stackexchange 
topic: 
https://stackoverflow.com/questions/6531543/efficient-implementation-of-binary-heaps

The benefit of exploiting the properties of cpu and memory caching yielded 
a net boost of nearly 10x. I can't see how, at bare minimum, based on the 
ratios of cache and memory write speed on my system, I won't see close to 
or around 3x improvement compared to reference based binary trees, and 
potentially a similar kind of improvement compared to bucket sorting (which 
is how Cuckoo Cycle searches for cycles in hashchains). 

I don't know anything about what actual result it will have, but I am 
building it anyway, and I will use closures because I personally prefer the 
notation.

On Wednesday, 25 April 2018 10:48:28 UTC+3, rog wrote:
>
> On 25 April 2018 at 08:05, Louki Sumirniy 
>  wrote: 
> > 
> https://stackoverflow.com/questions/6531543/efficient-implementation-of-binary-heaps
>  
> > 
> > Pretty much what I'm working on here is this, except with left to right 
> sort 
> > instead of vertical. I think this guy's work will help me iron out the 
> > performance issues. 
>
> You do know that heaps aren't a great data structure for searching, right? 
>
> > Another thing, that is more on topic more specifically, is that 
> collections 
> > of interface methods introduce a significant overhead, compared to 
> simply 
> > passing the pointer to the structure as a parameter. I am thinking that 
> > maybe a way to hide this parameter passing is by using closures, which 
> bind 
> > in the namespace from a hypothetical initialiser function, without 
> actually 
> > having to specify the pointer passing across. The structure is created 
> and 
> > allocated by the initialising function (constructor, if you like) and 
> > because all of the functions are declared within the namespace as 
> closures, 
> > the compiler implicitly passes the pointer to the struct without having 
> to 
> > specify it. 
>
> You don't need to do this. You're still thinking in traditional OO 
> terms. I'd suggest trying to think in a more Go like way instead. FWIW 
> I tried to point you in a more sensible direction with this code: 
> https://play.golang.org/p/wv0T8T8Ynns 
>
> > I don't know exactly what FP paradigm says about structures and 
> collections 
> > of functions exactly, as prima facie it looks like OOP. But closures are 
> a 
> > uniquely FP mechanism, and really they are just a way to merge 
> namespaces, 
> > and by doing this we don't have to pass the pointer to the function 
> > explicitly, as it is already accessible. 
>
> What gives you the idea that closures are a uniquely FP mechanism? 
>
>   rog. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Tcp connection reset

[Binu Ps]

below code i am trying to reset the tcp connection which coming from 
192.168.1.3 address, now its not working its not resetting the connection , 
anythink wrong in my source code ? please help


package main

import (
   "fmt"
   "time"
   "github.com/google/gopacket"
   "github.com/google/gopacket/layers"
   "github.com/google/gopacket/pcap"
   "reflect"
   "net"
   "log"
)

//var started_at time.Time
//var rest_root_requests int64 = 0


func main() {
   fmt.Println("live test")

   defender_loop()
}



var (
   device   string = "eth0"
   snapshot_len int32  = 1024
   promiscuous  bool   = false
   err  error
   timeout  time.Duration = 5
   handle   *pcap.Handle
   // Will reuse these for each packet
   ethLayer layers.Ethernet
   ipLayer  layers.IPv4
   tcpLayer layers.TCP
)


func defender_loop() {

   //var dstIPstr string = "127.0.0.1"
   //
   //if handle, err := pcap.OpenOffline("test.pcap"); err != nil {
   // //panic()
   // fmt.Print("kjhk")
   //}


   if handle, err := pcap.OpenLive("eth0", 1600, false, 5); err != nil {
  panic(err)
  fmt.Println(err);
   } else {
fmt.Print(handle)
  var filter string = "tcp and port 80"
  err = handle.SetBPFFilter(filter)
  if err != nil {
 log.Fatal(err)
  }
  fmt.Println("Only capturing TCP port 22 packets.")
  packetSource := gopacket.NewPacketSource(handle, 
handle.LinkType())
  for packet := range packetSource.Packets() {
 //handlePacket(packet)  // Do something with a packet here.

 parser := gopacket.NewDecodingLayerParser(
layers.LayerTypeEthernet,
,
,
,
 )
 foundLayerTypes := []gopacket.LayerType{}

 err := parser.DecodeLayers(packet.Data(), )
 if err != nil {
//fmt.Println("Trouble decoding layers: ", err)
continue
 }
 //fmt.Printf("Founder Layer>%+v\n", foundLayerTypes)
 for _, layerType := range foundLayerTypes {

//fmt.Println();
//fmt.Printf("ipLayer-->%+v\n", ipLayer)
//fmt.Println();
if(ipLayer.SrcIP.String() =="192.168.1.3") {
 //ipLayer.SrcIP=ipLayer.DstIP;
   //ipLayer.DstIP=ipLayer.SrcIP;
   //if(tcpLayer.SrcPort==61585) {
fmt.Println("Inside  IP");
   if(tcpLayer.RST==false){
  //fmt.Println("Before Update");
  fmt.Printf("%+v\n", tcpLayer)

  //fmt.Println();
  tcpLayer.RST=true;
  tcpLayer.Ack=0;
tcpLayer.ACK=false;
  //fmt.Println("After Update");

   }
   //fmt.Printf("%+v\n", tcpLayer)
}
if layerType == layers.LayerTypeIPv4 {
   //fmt.Println("IPv4: ", ipLayer.SrcIP, "->", 
ipLayer.DstIP)
   // fmt.Printf("ipLayer-->%+v\n", ipLayer)

}


payload := gopacket.Payload([]byte("testing"))
buf := gopacket.NewSerializeBuffer()
opts := gopacket.SerializeOptions{
   //FixLengths:   true,
   ComputeChecksums: true,
}

tcpLayer.SetNetworkLayerForChecksum()
err := gopacket.SerializeLayers(buf, opts,
   ,
   ,
   payload)
if err != nil {
   panic(err)
}
packetData := buf.Bytes()
ipConn, err := net.ListenPacket("ip4:tcp", 
"0.0.0.0")
if err != nil {
   panic(err)
}

dstIPaddr := net.IPAddr{
   IP: ipLayer.SrcIP,
 

Re: [go-nuts] Type func binding and interfaces

[Bakul Shah]

Roger is right. A heap can be a good structure for a priority queue but not for 
search. That is because it is partially ordered and siblings are not in any 
sorted order. In any case heaps are typically implemented with a vector. Go 
already has a pkg for it. go doc heap.

Seems like you’re doing something with blockchains. If you can clearly describe 
the precise problem you’re trying to solve (or point to an existing 
description), without getting into implementation issues, we may be able to 
help. Right now it is hard to reverse engineer the problem from your code and 
messages to this list.

> On Apr 25, 2018, at 2:08 AM, Louki Sumirniy 
>  wrote:
> 
> I think that it's not necessarily non-idiomatic to use closures instead of 
> interfaces in Go, it's more that Go has had interfaces longer than it's had 
> closures, and so more code has been written this way.
> 
> In Angular 2+ you have the option of embedding HTML, CSS and TS code inside 
> one file, or instead having 4, with the main file importing the other three. 
> I don't like this for several reasons, but mainly because it makes a more 
> complex interlinking between them, and I think even you can say that if your 
> application is big enough, all this extra import work will also cost a lot of 
> time, though in Go of course that time is not so big in the first place due 
> to how efficient its compiler is.
> 
> The way I see it is that closures and interfaces are two ways to do exactly 
> the same thing, binding namespaces together. One requires more accessory 
> declarations and lines of code, not a huge extra overhead, but then for 
> exported stuff - well, this is the one upside of it but I don't think it is 
> that great, gofmt wants to see header comments on every exported function. 
> Which is a nice idea, in theory, but in my opinion if you need comments your 
> naming scheme sucks.
> 
> That's also why I created distinct comparators with meaningful names instead 
> of creating named return values. b.IsEqual(data, cursor) compared to 
> b.Compare(data, cursor) == EQUAL is not just a lot longer, but harder to 
> read. I think technically it may also consume more code cache space to 
> implement this extra operation, though on the other side, there is a small 
> extra overhead for having three instead of 1. The compare function has to run 
> three cases, most concisely expressed as 
> 
> switch{
> case b.Store(c.index)>d: 
> return GREATER; 
> case b.Store(c.index) return LESSER
> }
> return EQUAL 
> 
> If my function only needs to know if it's equal (for example a search tree 
> walk), it's just done two comparison/branch operations for absolutely no 
> reason, and if I switch the order to benefit search, I raise the cost of 
> determining which direction to step next after no match on a node.
> 
> I worked for a little while on the C++ server application for the Steem 
> network node, and I was intending to remove a whole swathe of code relating 
> to protocol changes at various hard forks. The number of times I ran across 
> poorly ordered if/then (not even using switch!) that would perform 
> unnecessary comparisons in more cases than not, to me it explained the 
> ballooning amount of time that was required to play the blockchain into a 
> database shared file. Literally, over a week, at that point, almost 9 months 
> ago, and last I heard 270Gb of memory is required because this playback takes 
> a stupid amount of time (effectively, eternity) unless the shared file is 
> stored in a ramdisk.
> 
> So, just to be clear, I am not using OOPish techniques because I am a 
> believer, and I don't think that convention should dictate effective 
> programming either. Closures are a more compact notation than interfaces, and 
> for me this is equally important as writing code that does not waste cycles 
> doing things for the sake of some arbitrary model that does not greatly 
> improve maintainability and readability at the cost of overhead that will 
> stack up the more this approach is used.
> 
> Heaps have certainly got some disadvantages compared to bucket sorts and 
> reference based trees but the one thing they have is data locality. Data 
> locality can be a huge advantage because of CPU cache misses being avoided. 
> Cache memory on my i5 CPU is 14Gb/s write speed. The DDR4-2400 memory in my 
> system writes at 4Gb/s. This is linear writing, in the case of the main 
> memory, so not only does conventional binary tree architecture cause a very 
> high bandwidth load on the main memory, it also seeks the data randomly which 
> adds even more delays due to the linearity of the memory cells.
> 
> To illustrate my point, here is a couple of articles I found relating to this 
> and how data locality has brought massive performance benefits in reverse 
> proxy servers: https://queue.acm.org/detail.cfm?id=1814327 and I found this 
> story from this 

Re: [go-nuts] Type func binding and interfaces

[Louki Sumirniy]

As you look deeper into the link discussing the B-heap you can see that 
actually I am pretty much exactly following the same general structure in 
my algorithm. The structure will align neatly with page boundaries and that 
means less page faults and reduced pressure on the virtual memory mapping 
system that means actually the tree can go a lot larger while cutting down 
delays on retrieving data. I intuitively understood this in my own visual 
model and this picture in particular is exactly how I visualised the data 
storage map:



 (this necessarily also requires a 1-root)

but then the B-heap reduces the fragmentation of the memory through the 
page even further:



S, I probably will be considering revising the mapping to correspond to 
this as I can see why it would reduce paging during both searches and 
'bubbling up' (in my case, bunching up, as in inwards) required to maintain 
a high fill ratio.

On Wednesday, 25 April 2018 12:08:17 UTC+3, Louki Sumirniy wrote:
>
> I think that it's not necessarily non-idiomatic to use closures instead of 
> interfaces in Go, it's more that Go has had interfaces longer than it's had 
> closures, and so more code has been written this way.
>
> In Angular 2+ you have the option of embedding HTML, CSS and TS code 
> inside one file, or instead having 4, with the main file importing the 
> other three. I don't like this for several reasons, but mainly because it 
> makes a more complex interlinking between them, and I think even you can 
> say that if your application is big enough, all this extra import work will 
> also cost a lot of time, though in Go of course that time is not so big in 
> the first place due to how efficient its compiler is.
>
> The way I see it is that closures and interfaces are two ways to do 
> exactly the same thing, binding namespaces together. One requires more 
> accessory declarations and lines of code, not a huge extra overhead, but 
> then for exported stuff - well, this is the one upside of it but I don't 
> think it is that great, gofmt wants to see header comments on every 
> exported function. Which is a nice idea, in theory, but in my opinion if 
> you need comments your naming scheme sucks.
>
> That's also why I created distinct comparators with meaningful names 
> instead of creating named return values. b.IsEqual(data, cursor) compared 
> to b.Compare(data, cursor) == EQUAL is not just a lot longer, but harder to 
> read. I think technically it may also consume more code cache space to 
> implement this extra operation, though on the other side, there is a small 
> extra overhead for having three instead of 1. The compare function has to 
> run three cases, most concisely expressed as 
>
> switch{
> case b.Store(c.index)>d: 
> return GREATER; 
> case b.Store(c.index) return LESSER
> }
> return EQUAL 
>
> If my function only needs to know if it's equal (for example a search tree 
> walk), it's just done two comparison/branch operations for absolutely no 
> reason, and if I switch the order to benefit search, I raise the cost of 
> determining which direction to step next after no match on a node.
>
> I worked for a little while on the C++ server application for the Steem 
> network node, and I was intending to remove a whole swathe of code relating 
> to protocol changes at various hard forks. The number of times I ran across 
> poorly ordered if/then (not even using switch!) that would perform 
> unnecessary comparisons in more cases than not, to me it explained the 
> ballooning amount of time that was required to play the blockchain into a 
> database shared file. Literally, over a week, at that point, almost 9 
> months ago, and last I heard 270Gb of memory is required because this 
> playback takes a stupid amount of time (effectively, eternity) unless the 
> shared file is stored in a ramdisk.
>
> So, just to be clear, I am not using OOPish techniques because I am a 
> believer, and I don't think that convention should dictate effective 
> programming either. Closures are a more compact notation than interfaces, 
> and for me this is equally important as writing code that does not waste 
> cycles doing things for the sake of some arbitrary model that does not 
> greatly improve maintainability and readability at the cost of overhead 
> that will stack up the more this approach is used.
>
> Heaps have certainly got some disadvantages compared to bucket sorts and 
> reference based trees but the one thing they have is data locality. Data 
> locality can be a huge advantage because of CPU cache misses being avoided. 
> Cache memory on my i5 CPU is 14Gb/s write speed. The DDR4-2400 memory in my 
> system writes at 4Gb/s. This is linear writing, in the case of the main 
> memory, so not only does conventional binary tree architecture cause a very 
> high bandwidth load on the main memory, it also seeks the data randomly 
> which adds even more delays due to the 

Re: [go-nuts] Type func binding and interfaces

[roger peppe]

On 25 April 2018 at 10:08, Louki Sumirniy
 wrote:
> I think that it's not necessarily non-idiomatic to use closures instead of
> interfaces in Go, it's more that Go has had interfaces longer than it's had
> closures, and so more code has been written this way.

Sorry, that's just not true. Go had both interfaces and closures on
the first day
of its public release.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Type func binding and interfaces

[roger peppe]

On 25 April 2018 at 10:24, Louki Sumirniy
 wrote:
> As you look deeper into the link discussing the B-heap you can see that 
> actually I am pretty much exactly following the same general structure in my 
> algorithm. The structure will align neatly with page boundaries and that 
> means less page faults and reduced pressure on the virtual memory mapping 
> system that means actually the tree can go a lot larger while cutting down 
> delays on retrieving data. I intuitively understood this in my own visual 
> model and this picture in particular is exactly how I visualised the data 
> storage map:

To expand slightly on why this data structure isn't good for
searching, imagine you're searching for the number 40 in either of
those pictured data structures. When you're looking at the root, you
have to make a decision whether to move to the left or the right
child, but you can only see 2 and 3. How do you decide that you need
to go left? Now imagine you're searching for 31 - how do you decide
that the correct direction is right?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go's memory model for unbuffered channels

[threebearsdan]

So it always guarantees to print "hello world" for the unbuffered channel, 
doesn't it?

package main
*var c = make(chan int)*
var a string

func f() {
 a = "hello, world"
 *c <- 0*
}

func main() {
 go f()
 *<-c*
 print(a)
} 
it will guarantee to print "hello, world".

package main
*var c = make(chan int)*
var a string

func f() {
 a = "hello, world"
  *<-c*
}

func main() {
 go f() 
* c <- 0*
 print(a)
} 
it will also guarantee to print "hello, world".

 A send on a channel happens before the corresponding receive from that 
channel completes.
For the unbuffered channel and the buffered channel.

*A receive from an unbuffered channel happens before the send on that 
channel completes.*
Only for the unbuffered channel.


在 2014年5月29日星期四 UTC+8下午10:00:29，Ian Lance Taylor写道：
>
> On Thu, May 29, 2014 at 1:32 AM, liming  
> wrote: 
> > 
> > From go’s documentation 
> > 
> > If the channel is unbuffered, the sender blocks until the receiver has 
> > received the value. If the channel has a buffer, the sender blocks only 
> > until the value has been copied to the buffer; if the buffer is full, 
> this 
> > means waiting until some receiver has retrieved a value. 
> > 
> > The following code is gaurantined to print “hello, world” 
> > 
> > package main 
> > var c = make(chan int, 10) 
> > var a string 
> > 
> > func f() { 
> >  a = "hello, world" 
> >  c <- 0 
> > } 
> > 
> > func main() { 
> >  go f() 
> >  <-c 
> >  print(a) 
> > } 
>
>
> Right. 
>
>
> > if we change channel c to unbuffered channel: 
> > 
> > package main 
> > var c = make(chan int) 
> > var a string 
> > 
> > func f() { 
> >  a = "hello, world" 
> >  c <- 0 
> > } 
> > 
> > func main() { 
> >  go f() 
> >  <-c 
> >  print(a) 
> > } 
> > 
> > Is it gaurantined to print “hello, world” 
>
> Yes. 
>
> > c<-0 will bock until <-c has received the value, 
> > so c<-0 happens before <-c, as the assignment to a happens before c<-0, 
> > so in the main function, it will print “hello, world” right? 
>
> Right. 
>
> > But from go memory model, it says 
> > A receive from an unbuffered channel happens before the send on that 
> channel 
> > completes. 
> > 
> > so print(a) in main function is not gaurantined to print “hello, world” 
>
> No.  The memory model also says "A send on a channel happens before 
> the corresponding receive from that channel completes."  So the send 
> starts to happen, then the receive completes, then the send completes. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Type func binding and interfaces

[matthewjuran]

>
> I worked for a little while on the C++ server application for the Steem 
> network node, and I was intending to remove a whole swathe of code relating 
> to protocol changes at various hard forks. The number of times I ran across 
> poorly ordered if/then (not even using switch!) that would perform 
> unnecessary comparisons in more cases than not, to me it explained the 
> ballooning amount of time that was required to play the blockchain into a 
> database shared file. 


What I’ve been saying is you should look at a performance measure to prove 
a hypothesis like this one. Removing that code may or may not work.

I think you’ve mixed optimizing the algorithm and code, and we have to look 
at one at a time to effectively reach your goal. A program that measures 
progress toward the goal (performance) is a foundation I think we can more 
seriously start from for the code part. Can you share something like this 
with us?

Matt

On Wednesday, April 25, 2018 at 4:08:17 AM UTC-5, Louki Sumirniy wrote:
>
> I think that it's not necessarily non-idiomatic to use closures instead of 
> interfaces in Go, it's more that Go has had interfaces longer than it's had 
> closures, and so more code has been written this way.
>
> In Angular 2+ you have the option of embedding HTML, CSS and TS code 
> inside one file, or instead having 4, with the main file importing the 
> other three. I don't like this for several reasons, but mainly because it 
> makes a more complex interlinking between them, and I think even you can 
> say that if your application is big enough, all this extra import work will 
> also cost a lot of time, though in Go of course that time is not so big in 
> the first place due to how efficient its compiler is.
>
> The way I see it is that closures and interfaces are two ways to do 
> exactly the same thing, binding namespaces together. One requires more 
> accessory declarations and lines of code, not a huge extra overhead, but 
> then for exported stuff - well, this is the one upside of it but I don't 
> think it is that great, gofmt wants to see header comments on every 
> exported function. Which is a nice idea, in theory, but in my opinion if 
> you need comments your naming scheme sucks.
>
> That's also why I created distinct comparators with meaningful names 
> instead of creating named return values. b.IsEqual(data, cursor) compared 
> to b.Compare(data, cursor) == EQUAL is not just a lot longer, but harder to 
> read. I think technically it may also consume more code cache space to 
> implement this extra operation, though on the other side, there is a small 
> extra overhead for having three instead of 1. The compare function has to 
> run three cases, most concisely expressed as 
>
> switch{
> case b.Store(c.index)>d: 
> return GREATER; 
> case b.Store(c.index) return LESSER
> }
> return EQUAL 
>
> If my function only needs to know if it's equal (for example a search tree 
> walk), it's just done two comparison/branch operations for absolutely no 
> reason, and if I switch the order to benefit search, I raise the cost of 
> determining which direction to step next after no match on a node.
>
> I worked for a little while on the C++ server application for the Steem 
> network node, and I was intending to remove a whole swathe of code relating 
> to protocol changes at various hard forks. The number of times I ran across 
> poorly ordered if/then (not even using switch!) that would perform 
> unnecessary comparisons in more cases than not, to me it explained the 
> ballooning amount of time that was required to play the blockchain into a 
> database shared file. Literally, over a week, at that point, almost 9 
> months ago, and last I heard 270Gb of memory is required because this 
> playback takes a stupid amount of time (effectively, eternity) unless the 
> shared file is stored in a ramdisk.
>
> So, just to be clear, I am not using OOPish techniques because I am a 
> believer, and I don't think that convention should dictate effective 
> programming either. Closures are a more compact notation than interfaces, 
> and for me this is equally important as writing code that does not waste 
> cycles doing things for the sake of some arbitrary model that does not 
> greatly improve maintainability and readability at the cost of overhead 
> that will stack up the more this approach is used.
>
> Heaps have certainly got some disadvantages compared to bucket sorts and 
> reference based trees but the one thing they have is data locality. Data 
> locality can be a huge advantage because of CPU cache misses being avoided. 
> Cache memory on my i5 CPU is 14Gb/s write speed. The DDR4-2400 memory in my 
> system writes at 4Gb/s. This is linear writing, in the case of the main 
> memory, so not only does conventional binary tree architecture cause a very 
> high bandwidth load on the main memory, it also seeks the data randomly 
> which adds even 

Re: [go-nuts] go's memory model for unbuffered channels

[Ian Lance Taylor]

On Wed, Apr 25, 2018 at 2:27 AM,   wrote:
>
> So it always guarantees to print "hello world" for the unbuffered channel,
> doesn't it?
>
> package main
> var c = make(chan int)
> var a string
>
> func f() {
>  a = "hello, world"
>  c <- 0
> }
>
> func main() {
>  go f()
>  <-c
>  print(a)
> }
> it will guarantee to print "hello, world".
>
> package main
> var c = make(chan int)
> var a string
>
> func f() {
>  a = "hello, world"
>   <-c
> }
>
> func main() {
>  go f()
>  c <- 0
>  print(a)
> }
> it will also guarantee to print "hello, world".


Yes.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Type func binding and interfaces

[Louki Sumirniy]

Always the elements are inserted according to greater than and less than. 
equal can't happen. The first value inserted will be the root to begin 
with, but if the tree gets heavy on one side, you rotate the root to 
rebalance. from any given node, you know that you will find the element you 
are looking for according to the node you are at, and its greater or less 
than test. You can know this is always going to be true because that's how 
the inserts will populate the tree, and the 'completeness' constraint 
enforces a requirement for the 'bunching up' of data. So long as there is 
space between two nodes, they can be as high up in the tree as the distance 
between them, each row downwards allows a power of two sequence of 
increasing numbers. The 'completeness' constraint is extended by not 
allowing orphans, every child has a parent, until you get to the root, so 
vice versa, you know that you will find every element in the tree by 
walking downwards and going left or right depending on greater than and 
less than, this is the comparability constraint. You should be able to see 
that a number of architectural features in this data storage system imply 
heuristics for optimisation.

Just to update, reading about B-heaps and how it structures storage, I 
realised that I need to think of the tree in terms of 4kb blocks (well, for 
most platforms) as this is the amount of memory that will be accessed in 
one operation. However many rows a payload size fits into 4kb is the size 
of each subtree. So for 32 bit values, I have one page at the top, and then 
1024 pages that are the left and right pairs of the 512 elements at the 
bottom of the first page, and, of course, this repeats again. This changes 
how I have to implement the walk functions, as when it overflows past 512, 
I know I have moved to the second page-row.

So... I have to kinda go back to the drawing board a little on the 
structuring of the algorithm since it is working on 4kb page sizes. I may 
have to switch up the design so that the pages are the primary indices of 
the first dimension, and then the pages themselves, the subtrees, can be 
allocated as fixed length arrays, which also simplifies how Go will deal 
with them. Further, instead of allocating whole rows at a time, only those 
rows that have been breached require allocation of a new page for a 
subtree. In this respect, then the question about search cost becomes a 
little different, since we use 4k pages, and we know walking them goes on 
inside 14Gb/s SRAM cache, performance optimisation by tree balancing has a 
somewhat different character than the Binary Heap linear mapping.

Why do I seem so confident about the performance of this, potentially? Look 
up B-heaps and Varnish reverse proxy. It is designed at the low level with 
the same basic concept in mind, using B-heaps - structure the heap segments 
according to memory pages. If you are familiar with filesystems, you may 
know about block alignment. It's not quite so important for spinning disks 
but for flash and S/DRAM, memory is only ever handled on the hardware level 
in these block sizes. Disks usually use 512 byte blocks in the FS level and 
generally most disks have 512 byte blocks, some have 1k, 2k, 4k or 8k. Most 
current generation FS's use 4k blocks as the default size, for the reason 
it is a 1:1 mapping with memory pages.

The difference with what I am designing is that it does not order 
vertically, it orders horizontally. It's basically fractal, each subtree 
has the same number of potential subtrees below it.

Any code that keeps data aligned to memory page and disk page sizes is 
automatically significantly faster, because misalignment automatically 
doubles the amount of memory that has to be accessed to satisfy a request. 
This is why Binary Heaps are way slower than B-heaps.

Anyway, because many of my assumptions based on my conceptual model have 
changed, and big thanks to you guys, I have to rework the design to account 
for this. Thanks :p

On Wednesday, 25 April 2018 15:11:15 UTC+3, rog wrote:
>
> On 25 April 2018 at 10:24, Louki Sumirniy 
>  wrote: 
> > As you look deeper into the link discussing the B-heap you can see that 
> actually I am pretty much exactly following the same general structure in 
> my algorithm. The structure will align neatly with page boundaries and that 
> means less page faults and reduced pressure on the virtual memory mapping 
> system that means actually the tree can go a lot larger while cutting down 
> delays on retrieving data. I intuitively understood this in my own visual 
> model and this picture in particular is exactly how I visualised the data 
> storage map: 
>
> To expand slightly on why this data structure isn't good for 
> searching, imagine you're searching for the number 40 in either of 
> those pictured data structures. When you're looking at the root, you 
> have to make a decision whether to move to the left or the right 
> 

Re: [go-nuts] [ANN] oksvg and rasterx; SVG 2.0 path compliant renderer and rasterizer

[Andy Balholm]

So it sounds like the AGPL is a good license to choose if you want to keep your 
code from being used by big companies… ;-)

> On Apr 25, 2018, at 8:48 AM, 'David Chase' via golang-nuts 
>  wrote:
> 
> 
> 
> On Tuesday, April 24, 2018 at 10:45:35 AM UTC-4, matthe...@gmail.com wrote:
> I’m curious if some companies juggle the GPL. I guess if the app is used 
> internally only then there’s no problem with accidentally requiring a 
> proprietary program to be released as source code to the world. I’d have 
> thought the case would be the same with the AGPL. Do people count as 
> individuals in a corporate license with the ability to freely redistribute?
> 
> I can understand completely avoiding the issue. Language is interpretable and 
> only a court or whatever would decide what was really agreed to. The FSF 
> seems to put a lot of work into building up their licenses with legal 
> precedence.
> 
> I have never worked anywhere that could touch AGPL code.
> It was at the level of "just don't, and don't waste anyone's time asking.  
> Don't."
> This is not legal advice, I am just telling you what the policy is/was at all 
> these companies.
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Isn't it a good idea to allow "local import" internal packages, even if the internal pacakges are located under GOPATH/src?

[T L]

However, now this is forbidden.
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Type func binding and interfaces

[matthewjuran]

>
> Any code that keeps data aligned to memory page and disk page sizes is 
> automatically significantly faster, because misalignment automatically 
> doubles the amount of memory that has to be accessed to satisfy a request. 
> This is why Binary Heaps are way slower than B-heaps.


My opinion is without measurement you will miss necessary knowledge found 
between assumptions. The package testing benchmark has non-obvious features 
for repeatable data: 
https://dave.cheney.net/2013/06/30/how-to-write-benchmarks-in-go

I see you have logic tests 
(https://github.com/calibrae-project/bast/blob/master/pkg/bast/bast_test.go), 
and benchmarks are another step.

Matt

On Wednesday, April 25, 2018 at 9:37:48 AM UTC-5, Louki Sumirniy wrote:
>
> Always the elements are inserted according to greater than and less than. 
> equal can't happen. The first value inserted will be the root to begin 
> with, but if the tree gets heavy on one side, you rotate the root to 
> rebalance. from any given node, you know that you will find the element you 
> are looking for according to the node you are at, and its greater or less 
> than test. You can know this is always going to be true because that's how 
> the inserts will populate the tree, and the 'completeness' constraint 
> enforces a requirement for the 'bunching up' of data. So long as there is 
> space between two nodes, they can be as high up in the tree as the distance 
> between them, each row downwards allows a power of two sequence of 
> increasing numbers. The 'completeness' constraint is extended by not 
> allowing orphans, every child has a parent, until you get to the root, so 
> vice versa, you know that you will find every element in the tree by 
> walking downwards and going left or right depending on greater than and 
> less than, this is the comparability constraint. You should be able to see 
> that a number of architectural features in this data storage system imply 
> heuristics for optimisation.
>
> Just to update, reading about B-heaps and how it structures storage, I 
> realised that I need to think of the tree in terms of 4kb blocks (well, for 
> most platforms) as this is the amount of memory that will be accessed in 
> one operation. However many rows a payload size fits into 4kb is the size 
> of each subtree. So for 32 bit values, I have one page at the top, and then 
> 1024 pages that are the left and right pairs of the 512 elements at the 
> bottom of the first page, and, of course, this repeats again. This changes 
> how I have to implement the walk functions, as when it overflows past 512, 
> I know I have moved to the second page-row.
>
> So... I have to kinda go back to the drawing board a little on the 
> structuring of the algorithm since it is working on 4kb page sizes. I may 
> have to switch up the design so that the pages are the primary indices of 
> the first dimension, and then the pages themselves, the subtrees, can be 
> allocated as fixed length arrays, which also simplifies how Go will deal 
> with them. Further, instead of allocating whole rows at a time, only those 
> rows that have been breached require allocation of a new page for a 
> subtree. In this respect, then the question about search cost becomes a 
> little different, since we use 4k pages, and we know walking them goes on 
> inside 14Gb/s SRAM cache, performance optimisation by tree balancing has a 
> somewhat different character than the Binary Heap linear mapping.
>
> Why do I seem so confident about the performance of this, potentially? 
> Look up B-heaps and Varnish reverse proxy. It is designed at the low level 
> with the same basic concept in mind, using B-heaps - structure the heap 
> segments according to memory pages. If you are familiar with filesystems, 
> you may know about block alignment. It's not quite so important for 
> spinning disks but for flash and S/DRAM, memory is only ever handled on the 
> hardware level in these block sizes. Disks usually use 512 byte blocks in 
> the FS level and generally most disks have 512 byte blocks, some have 1k, 
> 2k, 4k or 8k. Most current generation FS's use 4k blocks as the default 
> size, for the reason it is a 1:1 mapping with memory pages.
>
> The difference with what I am designing is that it does not order 
> vertically, it orders horizontally. It's basically fractal, each subtree 
> has the same number of potential subtrees below it.
>
> Any code that keeps data aligned to memory page and disk page sizes is 
> automatically significantly faster, because misalignment automatically 
> doubles the amount of memory that has to be accessed to satisfy a request. 
> This is why Binary Heaps are way slower than B-heaps.
>
> Anyway, because many of my assumptions based on my conceptual model have 
> changed, and big thanks to you guys, I have to rework the design to account 
> for this. Thanks :p
>
> On Wednesday, 25 April 2018 15:11:15 UTC+3, rog wrote:
>>
>> On 25 April 2018 at 10:24, Louki Sumirniy 
>> 

Re: [go-nuts] Go list returning directory starting with underscore on Ubuntu

[rungta via golang-nuts]

Not accurate. 

$ cat ~/code/scratch/repro-underscore-issue.sh
echo "GOPATH: $GOPATH"
mkdir -p $GOPATH/src/github.com/testcase/testdata
echo "package testdata" > 
$GOPATH/src/github.com/testcase/testdata/testdata.go
echo "" >> $GOPATH/src/github.com/testcase/testdata/testdata.go
cd $GOPATH/src/github.com/testcase/testdata
echo "Running go list ./..."
go list ./...

$ ~/code/scratch/repro-underscore-issue.sh
GOPATH: /Users/prungta/code/gocode
Running go list ./...
_/Users/prungta/code/gocode/src/github.com/testcase/testdata


Should this be a bug report? 

On Wednesday, April 25, 2018 at 1:05:10 AM UTC-4, Dave Cheney wrote:
>
> If the path start with _ then it is not within the list of directories in 
> your GOPATH. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] oksvg and rasterx; SVG 2.0 path compliant renderer and rasterizer

['David Chase' via golang-nuts]

On Tuesday, April 24, 2018 at 10:45:35 AM UTC-4, matthe...@gmail.com wrote:
>
> I’m curious if some companies juggle the GPL. I guess if the app is used 
> internally only then there’s no problem with accidentally requiring a 
> proprietary program to be released as source code to the world. I’d have 
> thought the case would be the same with the AGPL. Do people count as 
> individuals in a corporate license with the ability to freely redistribute?
>
> I can understand completely avoiding the issue. Language is interpretable 
> and only a court or whatever would decide what was really agreed to. The 
> FSF seems to put a lot of work into building up their licenses with legal 
> precedence.
>

I have never worked anywhere that could touch AGPL code.
It was at the level of "just don't, and don't waste anyone's time asking.  
Don't."
This is not legal advice, I am just telling you what the policy is/was at 
all these companies.


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] oksvg and rasterx; SVG 2.0 path compliant renderer and rasterizer

[matthewjuran]

>
> So it sounds like the AGPL is a good license to choose if you want to keep 
> your code from being used by big companies… ;-)


If your project is secret software with a public network interface then 
don’t apply the AGPL to it. That’s not the only kind of software used at 
big companies.

Matt

On Wednesday, April 25, 2018 at 12:25:24 PM UTC-5, Andy Balholm wrote:

> So it sounds like the AGPL is a good license to choose if you want to keep 
> your code from being used by big companies… ;-)
>
> On Apr 25, 2018, at 8:48 AM, 'David Chase' via golang-nuts <
> golan...@googlegroups.com > wrote:
>
>
>
> On Tuesday, April 24, 2018 at 10:45:35 AM UTC-4, matthe...@gmail.com 
> wrote:
>>
>> I’m curious if some companies juggle the GPL. I guess if the app is used 
>> internally only then there’s no problem with accidentally requiring a 
>> proprietary program to be released as source code to the world. I’d have 
>> thought the case would be the same with the AGPL. Do people count as 
>> individuals in a corporate license with the ability to freely redistribute?
>>
>> I can understand completely avoiding the issue. Language is interpretable 
>> and only a court or whatever would decide what was really agreed to. The 
>> FSF seems to put a lot of work into building up their licenses with legal 
>> precedence.
>>
>
> I have never worked anywhere that could touch AGPL code.
> It was at the level of "just don't, and don't waste anyone's time asking.  
> Don't."
> This is not legal advice, I am just telling you what the policy is/was at 
> all these companies.
>
>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] using ginkgo

[Keith Brown]

thank for the response John.

My intention is to run integration test where I specify my test in english 
(acceptance test driven tests). I dont mind generating the binary but I 
would like to use the binary to test components such as. 1) if the web 
server is alive. 2) if a certain url gives me a 202 for a POST/GET etc...

On Friday, April 20, 2018 at 8:26:24 AM UTC-4, John Shahid wrote:
>
>
> Keith Brown  writes: 
>
> > I would like to have a tool for testing infrastructure components. I 
> > stumbled across Ginkgo. Does anyone use it? 
>
> Yes, it is being used by multiple CloudFoundry projects [1] 
>
> > Does it need a go compiler for tests? 
>
> Yes. 
>
> > My intention is to write simple tests, "Is server up?". "Does the file 
> > contain abc". But I don't want to have the go compiler installed to 
> > run the library. Does such a tool exist? 
>
> I'm not sure what you mean by library, but you can pre-compile a test 
> binary by running `go test -c  -o a.out'. You loose certain 
> features by doing so. For example, you won't be able to run the tests in 
> parallel. 
>
> Hope I answered your question. If not, may be adding more context on 
> what you're trying to do would help. 
>
> cheers, 
>
> -js 
>
> [1]: https://github.com/cloudfoundry?utf8=%E2%9C%93===go 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.