[go-nuts] Re: create a .rpm file for go app.

[Rajanikanth Jammalamadaka]

Build your binary and then use fpm? https://github.com/jordansissel/fpm

On Tuesday, September 20, 2016 at 2:53:39 PM UTC-4, sujith...@gmail.com 
wrote:
>
>
> Right now, to build my .rpm package, I'm using this website: 
> https://packager.io/
>
> However, I would like to automate this in our prod enviroment. Whats the 
> best way? Any plugins or tools available to build the .rpm file from the GO 
> App code? I have Dependencies saved using GODEP.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] net/http: In the infinite loop, refused all requests.

[Justin Israel]

http://dave.cheney.net/2015/08/08/performance-without-the-event-loop
Goroutines

Rather than relying on the kernel to manage their time sharing, goroutines
are cooperatively scheduled. The switch between goroutines only happens at
well defined points, when an explicit call is made to the Go runtime
scheduler. The major points where a goroutine will yield to the scheduler
include:

   - Channel send and receive operations, if those operations would block.
   - The go statement, although there is no guarantee that new goroutine
   will be scheduled immediately.
   - Blocking syscalls like file and network operations.
   - After being stopped for a garbage collection cycle.


On Wed, Sep 21, 2016 at 2:48 PM Justin Israel 
wrote:

> I'm not an expert, but this looks to be the expected behaviour to me. If
> you have GOMAXPROCS=8 and you tie up 8 goroutines with infinite loops, then
> there is nothing left to service requests. They are spinning the cpus
> without any kind of context switching points. This kind of behaviour has
> been mentioned in the past. There have to be points in your code for the
> runtime to switch to another goroutine, such as function calls, chan
> send/receive, syscalls (?), etc.
>
> On Wed, Sep 21, 2016 at 2:42 PM Tony Chen  wrote:
>
>> ### What version of Go are you using (`go version`)?
>> go 1.6.2 mac/amd64
>>
>> ### What operating system and processor architecture are you using (`go
>> env`)?
>> GOARCH="amd64"
>> GOBIN=""
>> GOEXE=""
>> GOHOSTARCH="amd64"
>> GOHOSTOS="darwin"
>> GOOS="darwin"
>> GOPATH="/data/apps/go"
>> GORACE=""
>> GOROOT="/usr/local/go"
>> GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
>> GO15VENDOREXPERIMENT="1"
>> CC="clang"
>> GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments
>> -fmessage-length=0 -fno-common"
>> CXX="clang++"
>> CGO_ENABLED="1"
>>
>> ### What did you do?
>>
>> package main
>>
>> import (
>> "net/http"
>> "strconv"
>> "fmt"
>> )
>>
>> func main() {
>> http.HandleFunc("/", httpHandler)
>> http.ListenAndServe(":8001", nil)
>> }
>>
>> func httpHandler(w http.ResponseWriter, r *http.Request) {
>> b, _ := strconv.ParseBool(r.URL.Query().Get("loop"))
>> if b {
>> loop()
>> }
>> fmt.Fprintf(w, "ok")
>> }
>>
>> func loop() {
>> i := 0
>> for {
>> if i == 1 {
>> i = 0
>> }
>> i++
>> }
>> }
>> 1. curl http://127.0.0.1:8001/?loop=true
>> 2. curl http://127.0.0.1:8001/?loop=false
>> When the loop to the CPU processing and about 8 times the request,
>> refused to all the HTTP request.
>> ### What did you expect to see?
>>
>> export GODEBUG=schedtrace=1000
>> SCHED 1010ms: gomaxprocs=8 idleprocs=8 threads=6 spinningthreads=0
>> idlethreads=3 runqueue=0 [0 0 0 0 0 0 0 0]
>> SCHED 2016ms: gomaxprocs=8 idleprocs=8 threads=6 spinningthreads=0
>> idlethreads=3 runqueue=0 [0 0 0 0 0 0 0 0]
>> SCHED 3019ms: gomaxprocs=8 idleprocs=8 threads=6 spinningthreads=0
>> idlethreads=3 runqueue=0 [0 0 0 0 0 0 0 0]
>> SCHED 4028ms: gomaxprocs=8 idleprocs=7 threads=6 spinningthreads=0
>> idlethreads=2 runqueue=0 [0 0 0 0 0 0 0 0]
>> SCHED 5033ms: gomaxprocs=8 idleprocs=5 threads=6 spinningthreads=0
>> idlethreads=0 runqueue=0 [0 0 0 0 0 0 0 0]
>> SCHED 6033ms: gomaxprocs=8 idleprocs=3 threads=8 spinningthreads=0
>> idlethreads=0 runqueue=0 [0 0 0 0 0 0 0 0]
>> SCHED 7035ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
>> idlethreads=0 runqueue=0 [0 0 0 0 0 0 0 0]
>> SCHED 8037ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
>> idlethreads=0 runqueue=1 [0 0 0 0 0 0 0 0]
>> SCHED 9044ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
>> idlethreads=0 runqueue=1 [0 0 0 0 0 0 0 0]
>> SCHED 10050ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
>> idlethreads=0 runqueue=1 [0 0 0 0 0 0 0 0]
>> SCHED 11059ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
>> idlethreads=0 runqueue=1 [0 0 0 0 0 0 0 0]
>> SCHED 12060ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
>> idlethreads=0 runqueue=1 [0 0 0 0 0 0 0 0]
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] net/http: In the infinite loop, refused all requests.

[Justin Israel]

I'm not an expert, but this looks to be the expected behaviour to me. If
you have GOMAXPROCS=8 and you tie up 8 goroutines with infinite loops, then
there is nothing left to service requests. They are spinning the cpus
without any kind of context switching points. This kind of behaviour has
been mentioned in the past. There have to be points in your code for the
runtime to switch to another goroutine, such as function calls, chan
send/receive, syscalls (?), etc.

On Wed, Sep 21, 2016 at 2:42 PM Tony Chen  wrote:

> ### What version of Go are you using (`go version`)?
> go 1.6.2 mac/amd64
>
> ### What operating system and processor architecture are you using (`go
> env`)?
> GOARCH="amd64"
> GOBIN=""
> GOEXE=""
> GOHOSTARCH="amd64"
> GOHOSTOS="darwin"
> GOOS="darwin"
> GOPATH="/data/apps/go"
> GORACE=""
> GOROOT="/usr/local/go"
> GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
> GO15VENDOREXPERIMENT="1"
> CC="clang"
> GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments
> -fmessage-length=0 -fno-common"
> CXX="clang++"
> CGO_ENABLED="1"
>
> ### What did you do?
>
> package main
>
> import (
> "net/http"
> "strconv"
> "fmt"
> )
>
> func main() {
> http.HandleFunc("/", httpHandler)
> http.ListenAndServe(":8001", nil)
> }
>
> func httpHandler(w http.ResponseWriter, r *http.Request) {
> b, _ := strconv.ParseBool(r.URL.Query().Get("loop"))
> if b {
> loop()
> }
> fmt.Fprintf(w, "ok")
> }
>
> func loop() {
> i := 0
> for {
> if i == 1 {
> i = 0
> }
> i++
> }
> }
> 1. curl http://127.0.0.1:8001/?loop=true
> 2. curl http://127.0.0.1:8001/?loop=false
> When the loop to the CPU processing and about 8 times the request, refused
> to all the HTTP request.
> ### What did you expect to see?
>
> export GODEBUG=schedtrace=1000
> SCHED 1010ms: gomaxprocs=8 idleprocs=8 threads=6 spinningthreads=0
> idlethreads=3 runqueue=0 [0 0 0 0 0 0 0 0]
> SCHED 2016ms: gomaxprocs=8 idleprocs=8 threads=6 spinningthreads=0
> idlethreads=3 runqueue=0 [0 0 0 0 0 0 0 0]
> SCHED 3019ms: gomaxprocs=8 idleprocs=8 threads=6 spinningthreads=0
> idlethreads=3 runqueue=0 [0 0 0 0 0 0 0 0]
> SCHED 4028ms: gomaxprocs=8 idleprocs=7 threads=6 spinningthreads=0
> idlethreads=2 runqueue=0 [0 0 0 0 0 0 0 0]
> SCHED 5033ms: gomaxprocs=8 idleprocs=5 threads=6 spinningthreads=0
> idlethreads=0 runqueue=0 [0 0 0 0 0 0 0 0]
> SCHED 6033ms: gomaxprocs=8 idleprocs=3 threads=8 spinningthreads=0
> idlethreads=0 runqueue=0 [0 0 0 0 0 0 0 0]
> SCHED 7035ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
> idlethreads=0 runqueue=0 [0 0 0 0 0 0 0 0]
> SCHED 8037ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
> idlethreads=0 runqueue=1 [0 0 0 0 0 0 0 0]
> SCHED 9044ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
> idlethreads=0 runqueue=1 [0 0 0 0 0 0 0 0]
> SCHED 10050ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
> idlethreads=0 runqueue=1 [0 0 0 0 0 0 0 0]
> SCHED 11059ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
> idlethreads=0 runqueue=1 [0 0 0 0 0 0 0 0]
> SCHED 12060ms: gomaxprocs=8 idleprocs=0 threads=10 spinningthreads=0
> idlethreads=0 runqueue=1 [0 0 0 0 0 0 0 0]
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] go-scp: a scp client library in go

['Paul Borman' via golang-nuts]

For cancellation you run the actual operation in a goroutine and return
results on a channel.  You can then select on the Deadline from the context
and the returned result.  If the deadline hits, you just shut down the SSH
session.  Does that make sense?

-Paul

On Tue, Sep 20, 2016 at 3:36 PM, Hiroaki Nakamura 
wrote:

> 2016-09-21 0:59 GMT+09:00 Paul Borman :
> > I would suggest just putting your package out there and see how it is
> > received.  At some point, if it becomes the defacto package, moving it
> might
> > make sense.
>
> OK, I understood.
>
> >
> > I actually wrote an internal SCP package that works with a ssh
> multiplexer
> > built on top of crypto/ssh.  For various reasons, I am not in a position
> to
> > OpenSource the package.
>
> I hope your SCP package will be released as an OpenSource.
>
> >
> > It provides a NewSCP that returns an SCP object.  You provide NewSCP
> with a
> > configuration structure that specifies optional default timestamps,
> mode, if
> > -p should be used, and if Start or Shell should be used to make the
> session
> > (some SSH servers do not support exec mode (Start)).
> >
> > The SCP type only has 3 methods (and one is actually a convenience
> wrapper):
> >
> > func (*SCP) Send(ctx context.Context, dst string, srcs ...string) error
> >
> > Send sends the srcs to dst. If srcs has a length greater than 1, or any
> of
> > the srcs is a Directory, dst must reference a directory on the remote
> host.
> > Send returns any errors encountered. If an error is returned, it may
> contain
> > multiple errors. User Errors(err) to retrieve the list of errors.
> >
> > func (*SCP) Receive(ctx context.Context, src string, f func(*Incoming)
> > error) error
> >
> > Receive requests the directory or file src from the remote host. For each
> > directory and file in src, f is called with a populated Incoming
> structure.
> > If IsDir is not set then f must either populate W or return an error. In
> any
> > event, if f returns an error then that file or directory's transfer is
> > stopped. If the returned error is wrapped by Fatal{}, the entire SCP
> session
> > is shutdown.
> >
> > type Incoming struct {
> > Pathstring  // Name of file on remote host
> > Modeos.FileMode // Mode of the file on the remote host
> > MTime   time.Time   // Last modified time from the remote host
> > ATime   time.Time   // Last access time from the remote host
> > Length  int64   // Length of incoming file
> > IsDir   bool// Set if receiving a directory
> > WErrerror   // Error encountered during write, if any
> > W   io.Writer   // Destination for read data
> > NoClose bool// Do not close W when finished.
> > }
> >
> > Incoming is where to send information about a file or directory. When an
> > Incoming is provided to ReceiveFile, W must be set and IsDir must not be
> > set. If the set W also implements io.Closer, W.Close is called after the
> > fimal write to W, unless NoClose is set. NoClose is normally only set
> when
> > using a single io.Writer for all data (such as os.Stdout).
> >
> > The convenience function is:
> >
> > func (*SCP) ReceiveFile(ctx context.Context, in *Incoming, src string)
> error
> >
> > ReceiveFile requests the remote host send the file named src. ReceiveFile
> > writes the received contents to in.W. Path, Mode, MTime and ATime are set
> > prior to the first write to in.W. MTime and ATime may be the zero value
> for
> > time.Time if not provided by the remote host.
> >
> > If there is an error writing to in.W, in.WErr is set to the error.
> in.WErr
> > is only valid after ReceiveFile returns. If there is an error receiving
> the
> > file from the remote host, it is returned.
> >
> > You could easily imagine a wrapper that called Receive and provided an
> > internal function that did whatever needed to be done, or even a stock
> > function that can be passed to Receive.
> >
> > I hope this is helpful.
>
> Thanks for the spec about your SCP package.
> I think it is more general than mine.
>
> However most of my cases are copy a single file or a directory,
> so I think I'd keep my APIs for the moment.
>
> I added NewSCP and SCP
> https://github.com/hnakamur/go-scp/commit/b43795fc10ef52b02a76653b643fa5
> aabb7b85a7
>
> One thing I would like to add is the cancellation support using
> context.Context.
> However ssh.Run does not take a context.Context as an argument,
> so I don't know how to achieve that.
> https://godoc.org/golang.org/x/crypto/ssh#Session.Run
>
> Could you tell me how you implement the cancellation?
>
> Thanks,
> Hiroaki Nakamura
>
> >
> > -Paul
> >
> > On Tue, Sep 20, 2016 at 6:47 AM, Hiroaki Nakamura 
> > wrote:
> >>
> >> Hi Paul,
> >> Thanks for your feedback!
> >>
> >> 2016-09-20 0:02 GMT+09:00 Paul Borman :
> >> > Adding an scp package is a nice addition.
> >>
> >> I agree.
> >> Should I send a pull request to 

Re: [go-nuts] Calling a Go function from asm ("fatal error: missing stackmap")

[Caleb Spare]

Thanks very much for your reply, Ian. I had hoped this would not be
the answer, though :)

I think I can arrange to avoid calling from asm to Go at a minor cost.
That sounds like the best solution here.

Creating the stack map manually might be interesting exercise for
learning about this corner of the compiler/runtime integration.

-Caleb

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] go-scp: a scp client library in go

[Hiroaki Nakamura]

2016-09-21 0:59 GMT+09:00 Paul Borman :
> I would suggest just putting your package out there and see how it is
> received.  At some point, if it becomes the defacto package, moving it might
> make sense.

OK, I understood.

>
> I actually wrote an internal SCP package that works with a ssh multiplexer
> built on top of crypto/ssh.  For various reasons, I am not in a position to
> OpenSource the package.

I hope your SCP package will be released as an OpenSource.

>
> It provides a NewSCP that returns an SCP object.  You provide NewSCP with a
> configuration structure that specifies optional default timestamps, mode, if
> -p should be used, and if Start or Shell should be used to make the session
> (some SSH servers do not support exec mode (Start)).
>
> The SCP type only has 3 methods (and one is actually a convenience wrapper):
>
> func (*SCP) Send(ctx context.Context, dst string, srcs ...string) error
>
> Send sends the srcs to dst. If srcs has a length greater than 1, or any of
> the srcs is a Directory, dst must reference a directory on the remote host.
> Send returns any errors encountered. If an error is returned, it may contain
> multiple errors. User Errors(err) to retrieve the list of errors.
>
> func (*SCP) Receive(ctx context.Context, src string, f func(*Incoming)
> error) error
>
> Receive requests the directory or file src from the remote host. For each
> directory and file in src, f is called with a populated Incoming structure.
> If IsDir is not set then f must either populate W or return an error. In any
> event, if f returns an error then that file or directory's transfer is
> stopped. If the returned error is wrapped by Fatal{}, the entire SCP session
> is shutdown.
>
> type Incoming struct {
> Pathstring  // Name of file on remote host
> Modeos.FileMode // Mode of the file on the remote host
> MTime   time.Time   // Last modified time from the remote host
> ATime   time.Time   // Last access time from the remote host
> Length  int64   // Length of incoming file
> IsDir   bool// Set if receiving a directory
> WErrerror   // Error encountered during write, if any
> W   io.Writer   // Destination for read data
> NoClose bool// Do not close W when finished.
> }
>
> Incoming is where to send information about a file or directory. When an
> Incoming is provided to ReceiveFile, W must be set and IsDir must not be
> set. If the set W also implements io.Closer, W.Close is called after the
> fimal write to W, unless NoClose is set. NoClose is normally only set when
> using a single io.Writer for all data (such as os.Stdout).
>
> The convenience function is:
>
> func (*SCP) ReceiveFile(ctx context.Context, in *Incoming, src string) error
>
> ReceiveFile requests the remote host send the file named src. ReceiveFile
> writes the received contents to in.W. Path, Mode, MTime and ATime are set
> prior to the first write to in.W. MTime and ATime may be the zero value for
> time.Time if not provided by the remote host.
>
> If there is an error writing to in.W, in.WErr is set to the error. in.WErr
> is only valid after ReceiveFile returns. If there is an error receiving the
> file from the remote host, it is returned.
>
> You could easily imagine a wrapper that called Receive and provided an
> internal function that did whatever needed to be done, or even a stock
> function that can be passed to Receive.
>
> I hope this is helpful.

Thanks for the spec about your SCP package.
I think it is more general than mine.

However most of my cases are copy a single file or a directory,
so I think I'd keep my APIs for the moment.

I added NewSCP and SCP
https://github.com/hnakamur/go-scp/commit/b43795fc10ef52b02a76653b643fa5aabb7b85a7

One thing I would like to add is the cancellation support using context.Context.
However ssh.Run does not take a context.Context as an argument,
so I don't know how to achieve that.
https://godoc.org/golang.org/x/crypto/ssh#Session.Run

Could you tell me how you implement the cancellation?

Thanks,
Hiroaki Nakamura

>
> -Paul
>
> On Tue, Sep 20, 2016 at 6:47 AM, Hiroaki Nakamura 
> wrote:
>>
>> Hi Paul,
>> Thanks for your feedback!
>>
>> 2016-09-20 0:02 GMT+09:00 Paul Borman :
>> > Adding an scp package is a nice addition.
>>
>> I agree.
>> Should I send a pull request to https://github.com/golang/crypto?
>> If yes, what package? golang.org/x/crypto/scp or
>> golang.org/x/crypto/ssh/scp?
>>
>> > You might want to consider simple names like:
>> >
>> > Send - Sends from []byte to file on remote host
>> > SendDir - Send files in dir to a remote host
>> > SendFile - Sends the contents of a file to the remote host
>> > Fetch - Fetches the contents of a file on remote host into memory
>> > FetchFile - Fetches a file from remote host into file on local host
>> > FetchDir - Fetches the files in a directory from the remote host
>> >
>> > These would translate in code to names like 

Re: [go-nuts] Pop out first element of a slice.

['Axel Wagner' via golang-nuts]

FWIW, I whipped up a short benchmark for this and it seems to confirm my
intuition. For small N (depending on the exact setup, on my machine, the
threshold was around N ~ 16 to 20) the copy approach is significantly
faster, but for growing N the quadratic behavior of the copy quickly makes
it much worse.

http://sprunge.us/hZOj

The test-setup here mirrors my assumptions from the last mail. We allocate
a fixed slice of length N and then do N pop-pushes, reflecting the
conditions after the queue has been used for a while in a balanced manner
and looking at long-term-ish performance characteristics to amortize the
allocations.

I also looked at individual pop-push sequences (instead of doing N in a
row, so this code: http://sprunge.us/UGdI) and this favors the append-way
even more. Which makes sense, because then the allocations happen only very
rarely for large N, so the disadvantage get's compensated.


Also, the usual caveats apply: This is a microbenchmark and benchmarking is
hard and even though there is a relative difference, the absolute numbers
are small enough that it likely doesn't make an actual difference in any
realistic program.

On Tue, Sep 20, 2016 at 11:12 PM, Gabriel Adumitrachioaiei <
gabi.adumitrachioa...@gmail.com> wrote:

> Very good mathematical explanation. Thanks!
>
> On Tuesday, September 20, 2016 at 7:24:39 PM UTC+3, Axel Wagner wrote:
>>
>> if we assume, that the number of elements N is roughly stable (so pops
>> and pushes are roughly balanced):
>>
>> • If we append and reslice, we need to reallocate every N pops, because
>> when space runs out, append will allocate 2N elements, so it has space for
>> N new ones. After N pop/push sequences, it will run out of space and the
>> capacity is again N, so it will allocate a new array with 2N elements, copy
>> over N and continue. So every N pops, we need to allocate N elements and
>> copy N elements, but don't incur any other costs.
>> • If we shift down, then we ~never need to reallocate, but need to copy N
>> elements on each pop, so over N pops we need to shift N² elements
>>
>> So, really, the question is how (N•copy(N)) compares  (alloc(N) +
>> copy(N)). Which, as it seems to me, heavily depends on N, the memory you
>> have, the GC pressure you have and your requirements.
>>
>> The answer, as always with these kinds of question is: Benchmark both on
>> your specific machine and your specific workload. If there is a clear
>> winner, pick that one. Otherwise, choose either.
>> In general, I'd indeed assume that for somewhat filled fifo-queues the
>> append-approach is faster. But I'd be willing to be surprised.
>>
>> On Tue, Sep 20, 2016 at 6:03 PM, Ian Davis  wrote:
>>
>>> On Tue, Sep 20, 2016, at 04:15 PM, Gabriel Adumitrachioaiei wrote:
>>>
>>> You might be right, but I just don't realize how. Since capacity will be
>>> 2x or 1.5x as before, reallocating the slice will not happen often. Or do
>>> you think that this would still be worse than copying almost all slice
>>> everytime there is a pop ?
>>>
>>>
>>> I think it depends on your use case. For the sql package I suspect the
>>> use case is a long running process with only a few members in the slice and
>>> a roughly level number over time. In that case the cost of copying will be
>>> small and offset by the savings in not needing to allocate.
>>>
>>> However, if your use case is a a large slice that is iterated over via a
>>> pop operation, with few or no pushes then it makes more sense to simply
>>> reslice as you were doing in your original version.
>>>
>>> Ian
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Wordbreak and word extraction in Go?

['Ingo Oeser' via golang-nuts]

Thanks for the suggestion, but I am looking for an implementation of 
http://unicode.org/reports/tr29/

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Govendor and Docker Build Container issues

[Daniel Theophanes]

I would run
govendor list
Does it report any external packages? If so run "govendor add +external"

On Tuesday, September 20, 2016 at 8:12:16 AM UTC-7, sc28 wrote:
>
> I have a application structure as follows in my GoPath:
>
> --/ myproject
> |
> |-- apps
>|-- app1
>  |-- main.go
>|-- app2
>  |-- main.go
>|-- appN
>  |-- main.go
> |-- libs
>   |-- myCommonPackage
>|-- gofile1.go
>|-- gofile2.go
>
>
> If I go into each "app" and execute the following:
>
>>govendor init
>>govendor add +external +local
>
> I get the external (github) packages and my local (libs/myCommonPackage) 
> vendored int he /vendor folder and everything builds fine locally.
>
>
> Now I'm trying to use docker to build the apps, using the following 
> command:
>
>  docker run --rm -it  -v "$PWD":/usr/src/app -w /usr/src/app 
> golang:1.7 go get (the container's GOPATH is at /go/app)
>
>
> The goal here is to allow my DevOps guy to pull the source and build the 
> executable inside a docker container (containing GOlang). 
>
> The build inside the 'build' container fails on my local package (all the 
> vendored github/golang.org) are found, but it errors on the 
> /libs/myCommonPackage saying it cannot be found.
>
>
> What is a better structure that would allow me to use 'local' packages 
> that are vendored, but that could be found by the docker build container?
>
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Wordbreak and word extraction in Go?

[Shawn Milochik]

 How about strings.Fields?

https://golang.org/pkg/strings/#Fields

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Wordbreak and word extraction in Go?

['Ingo Oeser' via golang-nuts]

Hi all,

I am pretty sure I am overlooking something in the repository 
https://godoc.org/golang.org/x/text but I cannot find something to split text 
into words according to the next Unicode word splitting algorithm.

Has anyone examples or can point me to the right direction? Can anyone confirm 
that this is missing? If missing, I would like to file an issue against the 
text repository for this.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Pop out first element of a slice.

[Gabriel Adumitrachioaiei]

Very good mathematical explanation. Thanks!

On Tuesday, September 20, 2016 at 7:24:39 PM UTC+3, Axel Wagner wrote:
>
> if we assume, that the number of elements N is roughly stable (so pops and 
> pushes are roughly balanced):
>
> • If we append and reslice, we need to reallocate every N pops, because 
> when space runs out, append will allocate 2N elements, so it has space for 
> N new ones. After N pop/push sequences, it will run out of space and the 
> capacity is again N, so it will allocate a new array with 2N elements, copy 
> over N and continue. So every N pops, we need to allocate N elements and 
> copy N elements, but don't incur any other costs.
> • If we shift down, then we ~never need to reallocate, but need to copy N 
> elements on each pop, so over N pops we need to shift N² elements
>
> So, really, the question is how (N•copy(N)) compares  (alloc(N) + 
> copy(N)). Which, as it seems to me, heavily depends on N, the memory you 
> have, the GC pressure you have and your requirements.
>
> The answer, as always with these kinds of question is: Benchmark both on 
> your specific machine and your specific workload. If there is a clear 
> winner, pick that one. Otherwise, choose either.
> In general, I'd indeed assume that for somewhat filled fifo-queues the 
> append-approach is faster. But I'd be willing to be surprised.
>
> On Tue, Sep 20, 2016 at 6:03 PM, Ian Davis  > wrote:
>
>> On Tue, Sep 20, 2016, at 04:15 PM, Gabriel Adumitrachioaiei wrote:
>>
>> You might be right, but I just don't realize how. Since capacity will be 
>> 2x or 1.5x as before, reallocating the slice will not happen often. Or do 
>> you think that this would still be worse than copying almost all slice 
>> everytime there is a pop ?
>>
>>
>> I think it depends on your use case. For the sql package I suspect the 
>> use case is a long running process with only a few members in the slice and 
>> a roughly level number over time. In that case the cost of copying will be 
>> small and offset by the savings in not needing to allocate.
>>
>> However, if your use case is a a large slice that is iterated over via a 
>> pop operation, with few or no pushes then it makes more sense to simply 
>> reslice as you were doing in your original version.
>>
>> Ian
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Sorting slice of structs

[xingtao zhao]

Or:

sort.Sort(Courses(course))

On Tuesday, September 20, 2016 at 10:50:15 AM UTC-7, Jan Mercl wrote:
>
> Types []Course and Courses are distinct types. courses have type []Courses 
> because that's what the parameter to make is. Change it to Courses, ie.
>
> courses := make(Courses, maxrow)
>
> On Tue, Sep 20, 2016, 19:44 Arie van Wingerden  > wrote:
>
>> Hi,
>>
>> Still a beginner ... first take at a bit more difficult thing.
>>
>> I cannot get this https://play.golang.org/p/qgBLH0r_vq to work.
>>
>> Compiler says:
>> cannot use courses (type []Course) as type sort.Interface in argument to 
>> sort.Sort:
>> []Course does not implement sort.Interface (missing Len method)
>>
>> However, I thought I need the Len method for []Courses instead of 
>> []Course!
>>
>> Pleas help.
>>
>> TIA
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: create a .rpm file for go app.

[mhhcbon]

hi,

if that can help you i did this package,
https://github.com/mh-cbon/go-bin-rpm

i m not pro packager, but that did work for me.

hope this help.

Le mardi 20 septembre 2016 20:53:39 UTC+2, sujith...@gmail.com a écrit :
>
>
> Right now, to build my .rpm package, I'm using this website: 
> https://packager.io/
>
> However, I would like to automate this in our prod enviroment. Whats the 
> best way? Any plugins or tools available to build the .rpm file from the GO 
> App code? I have Dependencies saved using GODEP.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Govendor and Docker Build Container issues

[Tamás Gulácsi]

Is your local files accessible to your devops?
Maybe set "-e GOPATH=/go/app:/local -v $GOPATH/src/lib:/local/src/lib:ro" or 
something?
Provide the host (local) GOPATH to the container, accessible under the 
container's GOPATH.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Sorting slice of structs

[Jan Mercl]

Types []Course and Courses are distinct types. courses have type []Courses
because that's what the parameter to make is. Change it to Courses, ie.

courses := make(Courses, maxrow)

On Tue, Sep 20, 2016, 19:44 Arie van Wingerden  wrote:

> Hi,
>
> Still a beginner ... first take at a bit more difficult thing.
>
> I cannot get this https://play.golang.org/p/qgBLH0r_vq to work.
>
> Compiler says:
> cannot use courses (type []Course) as type sort.Interface in argument to
> sort.Sort:
> []Course does not implement sort.Interface (missing Len method)
>
> However, I thought I need the Len method for []Courses instead of []Course!
>
> Pleas help.
>
> TIA
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
-- 

-j

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Sorting slice of structs

[Arie van Wingerden]

Hi,

Still a beginner ... first take at a bit more difficult thing.

I cannot get this https://play.golang.org/p/qgBLH0r_vq to work.

Compiler says:
cannot use courses (type []Course) as type sort.Interface in argument to
sort.Sort:
[]Course does not implement sort.Interface (missing Len method)

However, I thought I need the Len method for []Courses instead of []Course!

Pleas help.

TIA

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Crypto/rsa : OAEP documentation can lead to misunderstanding regarding max message length, is it a bug?

[AnomalRoil]

Hi there,

I was reading the Crypto/RSA documentation, since I'm using OAEP encryption 
for testing purposes.
So I read this  which ends 
with:

> The message must be no longer than the length of the public modulus less 
twice the hash length *plus *2.

Which I directly *translated *(maybe me being no native speaker is at fault 
there) into: 
> len(msg) <= len(public_modulus) - 2*len(hash) + 2
which I found puzzling since I thought it should have been -2, as I had 
checked the standard recently, so I double-checked the RFC 
 which says:

> RSAES-OAEP can operate on messages of length up to k - 2hLen - 2 octets, 
where hLen is the length of the output from the underlying hash function 
and k is the length in octets of the recipient's RSA modulus.

I guess it was intended to mean len(msg) <= len(public_modulus) - 
(2*len(hash) + 2), but those parens are not enough implied by such 
phrasing, in my opinion.
I've checked and the code is correct 
, so in the worse case a 
user may once end up getting an error after trying this "false-positive" 
border-case, if I may say so.

So here am I: *should I open an issue for a trivial sign mistake in a 
sentence in the documentation?*
I don't know what the usage is, I took a look at the guidelines for 
contributing and did not see any mention of such minor contribution.

I think it would be clearer written as follows (but maybe my usage of the 
comma is not correct in English?):
> The message must be no longer than the length of the public modulus less 
twice the hash length, less 2.
Or maybe it would even be better to write it in a more mathematical 
fashion? Like:
> The message must be no longer than the length of the public modulus - 2 * 
the hash length - 2. 

However, any opinion would be appreciated.
Best regards,
Y.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: http ssl "can't identify protocol"

[Greg Saylor]

Yeah I'm trying to figure out how to get a concise example of this - my 
deepest apologies.

I did try it with default ListenAndServeTLS and it exhibited the same 
problem.  There is no Hijack calls, x/net/ipv4.  This is really pretty 
bare-bones http service.

I'd say its about 100 reqs/sec on one interface and 20 on the other.

- Greg


On Tuesday, September 20, 2016 at 7:44:40 AM UTC-7, James Bardin wrote:
>
> It would really help if we had some complete code to reproduce the issue. 
>
> It sounds like the server isn't closing connections, but with only these 
> simple handlers I don't see where that could happen. Are there any Hijack 
> calls, x/net/ipv4, or network related syscalls anywhere in your code?
>
> Does this only happen with your custom tls.Config, or can you reproduce 
> this with the default ListenAndServeTLS?
>
> What is the request rate you're seeing when things start to fail?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] pprof: what are runtime.scanobject/gentraceback/pcvalue/etc

[Ian Lance Taylor]

On Tue, Sep 20, 2016 at 8:17 AM, mhhcbon  wrote:
>
> on my way to learn to use pprof, i now face some intriguing results with
> lots of runtime.* calls,
>
> can you help me to understand what those call are and how i should care
> about ?
>
> (pprof) top30
> 8720ms of 14380ms total (60.64%)
> Dropped 118 nodes (cum <= 71.90ms)
> Showing top 30 nodes out of 143 (cum >= 3800ms)
>   flat  flat%   sum%cum   cum%
>  720ms  5.01%  5.01% 1050ms  7.30%  runtime.scanobject
>  700ms  4.87%  9.87% 3640ms 25.31%  runtime.gentraceback
>  670ms  4.66% 14.53% 1780ms 12.38%  runtime.pcvalue
>  490ms  3.41% 17.94%  490ms  3.41%  runtime.readvarint
>  420ms  2.92% 20.86%  910ms  6.33%  runtime.step
>  420ms  2.92% 23.78%  420ms  2.92%  runtime/internal/atomic.Xchg
>  380ms  2.64% 26.43%  380ms  2.64%  runtime.heapBitsForObject
>  340ms  2.36% 28.79% 1580ms 10.99%  runtime.mallocgc
>  310ms  2.16% 30.95%  360ms  2.50%  runtime.gopark
>  270ms  1.88% 32.82%  270ms  1.88%  runtime.heapBitsSetType
>  260ms  1.81% 34.63%  510ms  3.55%  runtime.scanblock
>  250ms  1.74% 36.37%  250ms  1.74%  runtime/internal/atomic.Cas
>  240ms  1.67% 38.04% 4040ms 28.09%
> github.com/mh-cbon/chan/stream.(*Transform).Push
>  240ms  1.67% 39.71% 1210ms  8.41%  runtime.schedule
>  230ms  1.60% 41.31%  230ms  1.60%  runtime.newdefer
>  220ms  1.53% 42.84%  320ms  2.23%  runtime.deferreturn
>  210ms  1.46% 44.30%  260ms  1.81%  bytes.genSplit
>  210ms  1.46% 45.76%  260ms  1.81%  runtime.greyobject
>  210ms  1.46% 47.22% 6080ms 42.28%  runtime.systemstack
>  200ms  1.39% 48.61%  210ms  1.46%  runtime.acquireSudog
>  200ms  1.39% 50.00% 1760ms 12.24%  runtime.scanframeworker
>  190ms  1.32% 51.32%  790ms  5.49%  sync.(*Mutex).Lock
>  180ms  1.25% 52.57%  180ms  1.25%  runtime.gogo
>  180ms  1.25% 53.82%  180ms  1.25%  runtime.memmove
>  170ms  1.18% 55.01%  790ms  5.49%  runtime.chanrecv
>  170ms  1.18% 56.19%  530ms  3.69%  runtime.lock
>  170ms  1.18% 57.37%  170ms  1.18%  runtime.usleep
>  160ms  1.11% 58.48%  220ms  1.53%  runtime.siftdownTimer
>  160ms  1.11% 59.60%  160ms  1.11%  runtime/internal/atomic.Load
>  150ms  1.04% 60.64% 3800ms 26.43%  main.main.func2
>
> Also, I m not totally comfortable with flat Vs cum meanings,
> and those basics are not covered in
> https://blog.golang.org/profiling-go-programs
>
> Can you take a moment to explain those two notions ?

All the top functions in this profile are being run by the garbage collector.

The "flat" numbers are the time used by just that function.  The "cum"
numbers are the time used by that function and all the functions that
it calls.


> One more Q. In this pprof output
>
> (pprof) list Push
> Total: 14.38s
> ROUTINE 
> github.com/mh-cbon/chan/stream.(*Readable).Push in
> /home/mh-cbon/gow/src/github.com/mh-cbon/chan/stream/reader.go
>  0  1.65s (flat, cum) 11.47% of Total
>  .  . 79:func (s *Readable) Push(b interface{}) {
>  .  . 80:  for s.Paused() {
>  .  . 81:<- time.After(time.Millisecond * 1)
>  .  . 82:  }
>  .  . 83:  s.writeMutex.Lock()
>  .   10ms 84:  defer s.writeMutex.Unlock()
>  .  . 85:  for _, o := range s.Pipes {
>  .  1.64s 86:o.Write(b)
>  .  . 87:  }
>  .  . 88:}
>  .  . 89:func (s *Readable) Catch(fn ErrorFunc) Stream {
>  .  . 90:  s.E = append(s.E, fn)
>  .  . 91:  return s
> ROUTINE 
> github.com/mh-cbon/chan/stream.(*Transform).Push in
> /home/mh-cbon/gow/src/github.com/mh-cbon/chan/stream/transform.go
>  240ms  4.13s (flat, cum) 28.72% of Total
>  .  . 72:  } else {
>  .  . 73:s.Push(b)
>  .  . 74:  }
>  .  . 75:}
>  .  . 76:// Push writes data down in the stream.
>   40ms   40ms 77:func (s *Transform) Push(b interface{}) {
>  .  . 78:  // s.pipeMutex.Lock()
>  .  . 79:  // defer s.pipeMutex.Unlock()
>  190ms  190ms 80:  for _, o := range s.Pipes {
>   10ms  3.90s 81:o.Write(b)
>  .  . 82:  }
>  .  . 83:}
>  .  . 84:func (s *Transform) Catch(fn ErrorFunc) Stream
> {
>  .  . 85:  s.E = append(s.E, fn)
>  .  . 86:  return s
>
>
>
> The tow methods are very similar, but Transform.Push shows 190ms for the for
> loop, where Readable.Push does not show anything.
> How to understand this ?

Re: [go-nuts] Pop out first element of a slice.

['Axel Wagner' via golang-nuts]

if we assume, that the number of elements N is roughly stable (so pops and
pushes are roughly balanced):

• If we append and reslice, we need to reallocate every N pops, because
when space runs out, append will allocate 2N elements, so it has space for
N new ones. After N pop/push sequences, it will run out of space and the
capacity is again N, so it will allocate a new array with 2N elements, copy
over N and continue. So every N pops, we need to allocate N elements and
copy N elements, but don't incur any other costs.
• If we shift down, then we ~never need to reallocate, but need to copy N
elements on each pop, so over N pops we need to shift N² elements

So, really, the question is how (N•copy(N)) compares  (alloc(N) + copy(N)).
Which, as it seems to me, heavily depends on N, the memory you have, the GC
pressure you have and your requirements.

The answer, as always with these kinds of question is: Benchmark both on
your specific machine and your specific workload. If there is a clear
winner, pick that one. Otherwise, choose either.
In general, I'd indeed assume that for somewhat filled fifo-queues the
append-approach is faster. But I'd be willing to be surprised.

On Tue, Sep 20, 2016 at 6:03 PM, Ian Davis  wrote:

> On Tue, Sep 20, 2016, at 04:15 PM, Gabriel Adumitrachioaiei wrote:
>
> You might be right, but I just don't realize how. Since capacity will be
> 2x or 1.5x as before, reallocating the slice will not happen often. Or do
> you think that this would still be worse than copying almost all slice
> everytime there is a pop ?
>
>
> I think it depends on your use case. For the sql package I suspect the use
> case is a long running process with only a few members in the slice and a
> roughly level number over time. In that case the cost of copying will be
> small and offset by the savings in not needing to allocate.
>
> However, if your use case is a a large slice that is iterated over via a
> pop operation, with few or no pushes then it makes more sense to simply
> reslice as you were doing in your original version.
>
> Ian
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Pop out first element of a slice.

[Ian Davis]

On Tue, Sep 20, 2016, at 04:15 PM, Gabriel Adumitrachioaiei wrote:
> You might be right, but I just don't realize how. Since capacity will
> be 2x or 1.5x as before, reallocating the slice will not happen often.
> Or do you think that this would still be worse than copying almost all
> slice everytime there is a pop ?

I think it depends on your use case. For the sql package I suspect the
use case is a long running process with only a few members in the slice
and a roughly level number over time. In that case the cost of copying
will be small and offset by the savings in not needing to allocate.

However, if your use case is a a large slice that is iterated over via a
pop operation, with few or no pushes then it makes more sense to simply
reslice as you were doing in your original version.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] go-scp: a scp client library in go

['Paul Borman' via golang-nuts]

I would suggest just putting your package out there and see how it is
received.  At some point, if it becomes the defacto package, moving it
might make sense.

I actually wrote an internal SCP package that works with a ssh multiplexer
built on top of crypto/ssh.  For various reasons, I am not in a position to
OpenSource the package.

It provides a NewSCP that returns an SCP object.  You provide NewSCP with a
configuration structure that specifies optional default timestamps, mode,
if -p should be used, and if Start or Shell should be used to make the
session (some SSH servers do not support exec mode (Start)).

The SCP type only has 3 methods (and one is actually a convenience wrapper):

func (*SCP) *Send*(ctx context.Context, dst string, srcs ...string) error

*Send sends the srcs to dst. If srcs has a length greater than 1, or any of
the srcs is a Directory, dst must reference a directory on the remote host.
Send returns any errors encountered. If an error is returned, it may
contain multiple errors. User Errors(err) to retrieve the list of errors.*

func (*SCP) *Receive*(ctx context.Context, src string, f func(*Incoming)
error) error

*Receive requests the directory or file src from the remote host. For each
directory and file in src, f is called with a populated Incoming structure.
If IsDir is not set then f must either populate W or return an error. In
any event, if f returns an error then that file or directory's transfer is
stopped. If the returned error is wrapped by Fatal{}, the entire SCP
session is shutdown.*

type *Incoming* struct {
Pathstring  // Name of file on remote host
Modeos.FileMode // Mode of the file on the remote host
MTime   time.Time   // Last modified time from the remote host
ATime   time.Time   // Last access time from the remote host
Length  int64   // Length of incoming file
IsDir   bool// Set if receiving a directory
WErrerror   // Error encountered during write, if any
W   io.Writer   // Destination for read data
NoClose bool// Do not close W when finished.
}

*Incoming is where to send information about a file or directory. When an
Incoming is provided to ReceiveFile, W must be set and IsDir must not be
set. If the set W also implements io.Closer, W.Close is called after the
fimal write to W, unless NoClose is set. NoClose is normally only set when
using a single io.Writer for all data (such as os.Stdout).*

The convenience function is:

func (*SCP) *ReceiveFile*(ctx context.Context, in *Incoming, src string)
error

*ReceiveFile requests the remote host send the file named src. ReceiveFile
writes the received contents to in.W. Path, Mode, MTime and ATime are set
prior to the first write to in.W. MTime and ATime may be the zero value for
time.Time if not provided by the remote host.*

*If there is an error writing to in.W, in.WErr is set to the error. in.WErr
is only valid after ReceiveFile returns. If there is an error receiving the
file from the remote host, it is returned.*

You could easily imagine a wrapper that called Receive and provided an
internal function that did whatever needed to be done, or even a stock
function that can be passed to Receive.

I hope this is helpful.

-Paul

On Tue, Sep 20, 2016 at 6:47 AM, Hiroaki Nakamura 
wrote:

> Hi Paul,
> Thanks for your feedback!
>
> 2016-09-20 0:02 GMT+09:00 Paul Borman :
> > Adding an scp package is a nice addition.
>
> I agree.
> Should I send a pull request to https://github.com/golang/crypto?
> If yes, what package? golang.org/x/crypto/scp or
> golang.org/x/crypto/ssh/scp?
>
> > You might want to consider simple names like:
> >
> > Send - Sends from []byte to file on remote host
> > SendDir - Send files in dir to a remote host
> > SendFile - Sends the contents of a file to the remote host
> > Fetch - Fetches the contents of a file on remote host into memory
> > FetchFile - Fetches a file from remote host into file on local host
> > FetchDir - Fetches the files in a directory from the remote host
> >
> > These would translate in code to names like scp.SendFile, which is pretty
> > descriptive all by itself.
>
> Thanks for simple and descriptive function names.
> I renamed functions.
> https://github.com/hnakamur/go-scp/commit/8cd6d9e5ab17187556e5efc3a666d2
> 9b4b561c78
>
> >
> > For the directory copy, it might be better to have a function return the
> > io.Writer to write the file to, rather than force the files into a
> > directory.  This would make it easy to keep the contents in memory,
> change
> > file names, or whatever.
>
> Yes, I agree it might be better not to force the files in a directory.
> However I don't think having a function return the io.Writer will do
> since we need to read a reply for each scp protocol header or body.
>
> I had read the article below and the openssh source code and
> implemented my scp package https://github.com/hnakamur/go-scp.
>
> "How the SCP protocol works (Jan Pechanec's weblog)"
> 

Re: [go-nuts] Randomizing contents of a file

[Bakul Shah]

Not quite what you asked for but rather than use N goroutines, why not slurp in 
the whole file in a list  ([]string) of lines and then output them in random 
order? Something like

for _, i := range deal(len(lines)) {
  fmt.Println(lines[i])
}

Where you'd write deal(n) to return a list of n numbers in range 0..n-1 in a 
randomized order, without repeating a number.

> On Sep 20, 2016, at 3:43 AM, Unknown User  wrote:
> 
> Hi All,
> 
> I am trying to print the contents of  a file randomizing the lines.
> The following code works, but does not exit even after all the lines are read.
> What needs to be done to correct it?
> 
> package main
> 
> import(
> "fmt"
> "bufio"
> "os"
> "math/rand"
> "time"
> )
> 
> func main() {
> filename := os.Args[1]
> if filename == "" {
> panic("No filename")
> }
> fh,err := os.Open(filename)
> if err != nil { 
> panic(err)
> }
> scanner := bufio.NewScanner(fh)
> c := make(chan string)
> for scanner.Scan() {
> line := scanner.Text()
> err := scanner.Err()
> if err != nil { 
> close(c)
> }
> go func(l string){
> time.Sleep(time.Duration(rand.Intn(30)) * time.Millisecond)
> c <-  l
> }(line)
> }
> for  { 
> select { 
> case myline := <- c:
> fmt.Println(myline)
> default:
> break
> }
> }
> }
>  
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] pprof: what are runtime.scanobject/gentraceback/pcvalue/etc

[mhhcbon]

Hi,

on my way to learn to use pprof, i now face some intriguing results with 
lots of runtime.* calls, 

can you help me to understand what those call are and how i should care 
about ?

(pprof) top30
8720ms of 14380ms total (60.64%)
Dropped 118 nodes (cum <= 71.90ms)
Showing top 30 nodes out of 143 (cum >= 3800ms)
  flat  flat%   sum%cum   cum%
 720ms  5.01%  5.01% 1050ms  7.30%  runtime.scanobject
 700ms  4.87%  9.87% 3640ms 25.31%  runtime.gentraceback
 670ms  4.66% 14.53% 1780ms 12.38%  runtime.pcvalue
 490ms  3.41% 17.94%  490ms  3.41%  runtime.readvarint
 420ms  2.92% 20.86%  910ms  6.33%  runtime.step
 420ms  2.92% 23.78%  420ms  2.92%  runtime/internal/atomic.Xchg
 380ms  2.64% 26.43%  380ms  2.64%  runtime.heapBitsForObject
 340ms  2.36% 28.79% 1580ms 10.99%  runtime.mallocgc
 310ms  2.16% 30.95%  360ms  2.50%  runtime.gopark
 270ms  1.88% 32.82%  270ms  1.88%  runtime.heapBitsSetType
 260ms  1.81% 34.63%  510ms  3.55%  runtime.scanblock
 250ms  1.74% 36.37%  250ms  1.74%  runtime/internal/atomic.Cas
 240ms  1.67% 38.04% 4040ms 28.09%  github.com/mh-cbon/chan/stream
.(*Transform).Push
 240ms  1.67% 39.71% 1210ms  8.41%  runtime.schedule
 230ms  1.60% 41.31%  230ms  1.60%  runtime.newdefer
 220ms  1.53% 42.84%  320ms  2.23%  runtime.deferreturn
 210ms  1.46% 44.30%  260ms  1.81%  bytes.genSplit
 210ms  1.46% 45.76%  260ms  1.81%  runtime.greyobject
 210ms  1.46% 47.22% 6080ms 42.28%  runtime.systemstack
 200ms  1.39% 48.61%  210ms  1.46%  runtime.acquireSudog
 200ms  1.39% 50.00% 1760ms 12.24%  runtime.scanframeworker
 190ms  1.32% 51.32%  790ms  5.49%  sync.(*Mutex).Lock
 180ms  1.25% 52.57%  180ms  1.25%  runtime.gogo
 180ms  1.25% 53.82%  180ms  1.25%  runtime.memmove
 170ms  1.18% 55.01%  790ms  5.49%  runtime.chanrecv
 170ms  1.18% 56.19%  530ms  3.69%  runtime.lock
 170ms  1.18% 57.37%  170ms  1.18%  runtime.usleep
 160ms  1.11% 58.48%  220ms  1.53%  runtime.siftdownTimer
 160ms  1.11% 59.60%  160ms  1.11%  runtime/internal/atomic.Load
 150ms  1.04% 60.64% 3800ms 26.43%  main.main.func2

Also, I m not totally comfortable with flat Vs cum meanings, 
and those basics are not covered in 
https://blog.golang.org/profiling-go-programs

Can you take a moment to explain those two notions ?

One more Q. In this pprof output

(pprof) list Push
Total: 14.38s
ROUTINE  github.com/mh-cbon/chan/stream.(*Readable).
Push in /home/mh-cbon/gow/src/github.com/mh-cbon/chan/stream/reader.go
 0  1.65s (flat, cum) 11.47% of Total
 .  . 79:func (s *Readable) Push(b interface{}) {
 .  . 80:  for s.Paused() {
 .  . 81:<- time.After(time.Millisecond * 1)
 .  . 82:  }
 .  . 83:  s.writeMutex.Lock()
 .   10ms 84:  defer s.writeMutex.Unlock()
 .  . 85:  for _, o := range s.Pipes {
 .  1.64s 86:o.Write(b)
 .  . 87:  }
 .  . 88:}
 .  . 89:func (s *Readable) Catch(fn ErrorFunc) Stream {
 .  . 90:  s.E = append(s.E, fn)
 .  . 91:  return s
ROUTINE  github.com/mh-cbon/chan/stream.(*Transform
).Push in /home/mh-cbon/gow/src/github.com/mh-cbon/chan/stream/transform.go
 240ms  4.13s (flat, cum) 28.72% of Total
 .  . 72:  } else {
 .  . 73:s.Push(b)
 .  . 74:  }
 .  . 75:}
 .  . 76:// Push writes data down in the stream.
  40ms   40ms 77:func (s *Transform) Push(b interface{}) {
 .  . 78:  // s.pipeMutex.Lock()
 .  . 79:  // defer s.pipeMutex.Unlock()
 190ms  190ms 80:  for _, o := range s.Pipes {
  10ms  3.90s 81:o.Write(b)
 .  . 82:  }
 .  . 83:}
 .  . 84:func (s *Transform) Catch(fn ErrorFunc) Stream 
{
 .  . 85:  s.E = append(s.E, fn)
 .  . 86:  return s



The tow methods are very similar, but Transform.Push shows 190ms for the 
for loop, where Readable.Push does not show anything.
How to understand this ?


thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Pop out first element of a slice.

[Gabriel Adumitrachioaiei]

You might be right, but I just don't realize how. Since capacity will be 2x 
or 1.5x as before, reallocating the slice will not happen often. Or do you 
think that this would still be worse than copying almost all slice 
everytime there is a pop ?

On Tuesday, September 20, 2016 at 5:59:27 PM UTC+3, Ian Davis wrote:
>
> On Tue, Sep 20, 2016, at 03:54 PM, Gabriel Adumitrachioaiei wrote:
>
> Well, the capacity will be reduced by one. I don't think this makes much 
> difference.
>
>
> It makes a difference for a long running service that repeatedly pushes 
> and pops.
>
> Ian
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Govendor and Docker Build Container issues

[sc28]

I have a application structure as follows in my GoPath:

--/ myproject
|
|-- apps
   |-- app1
 |-- main.go
   |-- app2
 |-- main.go
   |-- appN
 |-- main.go
|-- libs
  |-- myCommonPackage
   |-- gofile1.go
   |-- gofile2.go


If I go into each "app" and execute the following:

   >govendor init
   >govendor add +external +local

I get the external (github) packages and my local (libs/myCommonPackage) 
vendored int he /vendor folder and everything builds fine locally.


Now I'm trying to use docker to build the apps, using the following command:

 docker run --rm -it  -v "$PWD":/usr/src/app -w /usr/src/app golang:1.7 
go get (the container's GOPATH is at /go/app)


The goal here is to allow my DevOps guy to pull the source and build the 
executable inside a docker container (containing GOlang). 

The build inside the 'build' container fails on my local package (all the 
vendored github/golang.org) are found, but it errors on the 
/libs/myCommonPackage saying it cannot be found.


What is a better structure that would allow me to use 'local' packages that 
are vendored, but that could be found by the docker build container?






-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Pop out first element of a slice.

[Ian Davis]

On Tue, Sep 20, 2016, at 03:54 PM, Gabriel Adumitrachioaiei wrote:
> Well, the capacity will be reduced by one. I don't think this makes
> much difference.

It makes a difference for a long running service that repeatedly
pushes and pops.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Pop out first element of a slice.

[Gabriel Adumitrachioaiei]

Well, the capacity will be reduced by one. I don't think this makes much 
difference.

On Tuesday, September 20, 2016 at 5:44:20 PM UTC+3, Ian Davis wrote:
>
> On Tue, Sep 20, 2016, at 03:23 PM, Gabriel Adumitrachioaiei wrote:
>
> I don't understand something when I want to pop out first element of a 
> slice and use it.
> Here is my version:
>
> s := []int{1,2,3}
> first := s[0]
> s = s[1:]
>
>
> Here is a version that I saw in the standard library: 
> https://golang.org/src/database/sql/sql.go#L791
>
> first := s[0]
> copy(s, s[1:])
> s = s[:len(s) - 1]
>
>
> I wonder, why do we need to translate the other elements to the left with 
> a copy ? 
> In the first case, I guess the first element will still be found in the 
> underlying array, and in the second case the last element.
> It's not like for avoiding a memory leak, because neither version 
> allocates a new underlying array.
>
>
> If you append to the slice in the second version then the new element will 
> reuse the space at the end of the backing array. In the first version Go 
> may have to allocate to expand the backing array since there is no spare 
> capacity.
>
> Ian
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: playground URL to fetch code only

[Marvin Renich]

* Ivan Anfilatov  [160920 10:36]:
> https://play.golang.org/p/Vg6f0gSs3l.go
> 
> compile - post request to https://play.golang.org/compile in body request - 
> source 
> response - json

Thanks, Ivan!

...Marvin

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: http ssl "can't identify protocol"

[James Bardin]

It would really help if we had some complete code to reproduce the issue. 

It sounds like the server isn't closing connections, but with only these 
simple handlers I don't see where that could happen. Are there any Hijack 
calls, x/net/ipv4, or network related syscalls anywhere in your code?

Does this only happen with your custom tls.Config, or can you reproduce 
this with the default ListenAndServeTLS?

What is the request rate you're seeing when things start to fail?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: playground URL to fetch code only

[Ivan Anfilatov]

https://play.golang.org/p/Vg6f0gSs3l.go

compile - post request to https://play.golang.org/compile in body request - 
source 
response - json


вторник, 20 сентября 2016 г., 16:09:47 UTC+3 пользователь Marvin Renich 
написал:
>
> Is there a way to take a playground URL, such as 
> https://play.golang.org/p/Vg6f0gSs3l, and modify it (e.g. with a query 
> string) so that you can retrieve just the code as text without the 
> surrounding html?  Something like 
>
> curl 'https://play.golang.org/p/Vg6f0gSs3l?text' 
>
> If not, would such a feature request be likely to get implemented? 
>
> Thanks...Marvin 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Ivy crashes on iOS 10

[avulagaurav]

Yeah me too 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] playground URL to fetch code only

[Marvin Renich]

* Nicolas Martyanoff  [160920 10:07]:
> curl 'https://play.golang.org/p/Vg6f0gSs3l.go', i.e. just appending '.go' will
> do the trick.

Excellent!  Thank you!

...Marvin

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Pop out first element of a slice.

[Gabriel Adumitrachioaiei]

I don't understand something when I want to pop out first element of a 
slice and use it.
Here is my version:
s := []int{1,2,3}
first := s[0]
s = s[1:]

Here is a version that I saw in the standard library: 
https://golang.org/src/database/sql/sql.go#L791
first := s[0]
copy(s, s[1:])
s = s[:len(s) - 1]

I wonder, why do we need to translate the other elements to the left with a 
copy ? 
In the first case, I guess the first element will still be found in the 
underlying array, and in the second case the last element.
It's not like for avoiding a memory leak, because neither version allocates 
a new underlying array.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] go-scp: a scp client library in go

[Hiroaki Nakamura]

Hi Paul,
Thanks for your feedback!

2016-09-20 0:02 GMT+09:00 Paul Borman :
> Adding an scp package is a nice addition.

I agree.
Should I send a pull request to https://github.com/golang/crypto?
If yes, what package? golang.org/x/crypto/scp or golang.org/x/crypto/ssh/scp?

> You might want to consider simple names like:
>
> Send - Sends from []byte to file on remote host
> SendDir - Send files in dir to a remote host
> SendFile - Sends the contents of a file to the remote host
> Fetch - Fetches the contents of a file on remote host into memory
> FetchFile - Fetches a file from remote host into file on local host
> FetchDir - Fetches the files in a directory from the remote host
>
> These would translate in code to names like scp.SendFile, which is pretty
> descriptive all by itself.

Thanks for simple and descriptive function names.
I renamed functions.
https://github.com/hnakamur/go-scp/commit/8cd6d9e5ab17187556e5efc3a666d29b4b561c78

>
> For the directory copy, it might be better to have a function return the
> io.Writer to write the file to, rather than force the files into a
> directory.  This would make it easy to keep the contents in memory, change
> file names, or whatever.

Yes, I agree it might be better not to force the files in a directory.
However I don't think having a function return the io.Writer will do
since we need to read a reply for each scp protocol header or body.

I had read the article below and the openssh source code and
implemented my scp package https://github.com/hnakamur/go-scp.

"How the SCP protocol works (Jan Pechanec's weblog)"
https://blogs.oracle.com/janp/entry/how_the_scp_protocol_works

I built two structs sourceProtocol and sinkProtocol
https://github.com/hnakamur/go-scp/blob/master/protocol.go

I had thought to export these structs or make interfaces for that.
However the implementation of two functions SendDir and FetchDir
which using these structs become somewhat complex.

https://github.com/hnakamur/go-scp/blob/8cd6d9e5ab17187556e5efc3a666d29b4b561c78/source.go#L81-L175
https://github.com/hnakamur/go-scp/blob/8cd6d9e5ab17187556e5efc3a666d29b4b561c78/sink.go#L111-L223

So I was not confident about exporting sourceProtocol and sinkProtocol,
and I did not export them at the time.

If we can define structs, functions or interfaces which are easy to use,
I'm glad to export them.

Do you have an idea about such structs, functions or interfaces?
Thanks!

Hiroaki Nakamura

>
> On Fri, Sep 16, 2016 at 9:41 AM, Hiroaki Nakamura 
> wrote:
>>
>> Hi all,
>>
>> I noticed the golang.org/x/crypto/ssh package exists, but the scp
>> package does not.
>> So I wrote a scp client library in go.
>> https://github.com/hnakamur/go-scp
>>
>> I also wrote a sshd server just usable for testing go-scp.
>> https://github.com/hnakamur/go-sshd
>>
>> Right now, go-scp only exports high level functions which are supposed
>> to be easy to use.
>> https://godoc.org/github.com/hnakamur/go-scp
>>
>> However I wonder if there APIs can be improved. For example,
>> better function names and better arguments.
>>
>> Could you tell me what you think?
>> Thanks!
>>
>> Hiroaki Nakamura
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Ivy crashes on iOS 10

[David Crawshaw]

I'm going to look into pushing a new version with 1.7.

On Mon, Sep 19, 2016 at 10:39 AM, Jan Mercl <0xj...@gmail.com> wrote:
> On Mon, Sep 19, 2016 at 3:58 PM  wrote:
>
>> Anybody with same experiences?
>
> https://github.com/robpike/ivy/issues/29
>
> --
>
> -j
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: memory profiler for benchmark test reports 0s across the board

[Dave Cheney]

Can you share a small runable reproduction which shows the problem ?

On Tuesday, 20 September 2016 23:26:29 UTC+10, Ethan Kennedy wrote:
>
> Dave,
>
> Thank you for the tip. That does at least show me some measurements of 
> allocations. I guess I was hoping to do some more fine-grained analysis, if 
> only as an educational exercise.
>
> So, I tried using the runtime/pprof package to write a heap profile at a 
> specific point in my program and still got some less than informative 
> results. I'm wondering if this means my program is triggering GC at a point 
> where I'm not expecting it, which is preventing capture of a good profile.
>
> On Monday, September 19, 2016 at 5:18:56 PM UTC-5, Dave Cheney wrote:
>>
>> Try b.ReportAllocs() before your benchmark loop. That's the easiest way 
>> to benchmark the allocations per operation.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: memory profiler for benchmark test reports 0s across the board

['Ethan Kennedy' via golang-nuts]

Dave,

Thank you for the tip. That does at least show me some measurements of 
allocations. I guess I was hoping to do some more fine-grained analysis, if 
only as an educational exercise.

So, I tried using the runtime/pprof package to write a heap profile at a 
specific point in my program and still got some less than informative 
results. I'm wondering if this means my program is triggering GC at a point 
where I'm not expecting it, which is preventing capture of a good profile.

On Monday, September 19, 2016 at 5:18:56 PM UTC-5, Dave Cheney wrote:
>
> Try b.ReportAllocs() before your benchmark loop. That's the easiest way to 
> benchmark the allocations per operation.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Camel Enterprise Integration Patterns in golang

[adrian . lopez . gomez]

Hi Antonio,

It seems the project was deleted. Did you continue it elsewhere?

Regards,
Adrian.

On Friday, 6 February 2015 12:29:10 UTC+1, Antonio Linari wrote:
>
> Hi,
> here  you can find a 1st version, very basic, of Gamel (Camel Enterprise 
> Integration Patterns in Golang)
>
> https://github.com/advancedlogic/Gamel
>
> For now,a simple std:in and std:out component works but more components 
> are on the road
> Please join the project
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] playground URL to fetch code only

[Marvin Renich]

Is there a way to take a playground URL, such as
https://play.golang.org/p/Vg6f0gSs3l, and modify it (e.g. with a query
string) so that you can retrieve just the code as text without the
surrounding html?  Something like

curl 'https://play.golang.org/p/Vg6f0gSs3l?text'

If not, would such a feature request be likely to get implemented?

Thanks...Marvin

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Randomizing contents of a file

[Marvin Renich]

* Unknown User  [160920 06:43]:
> Hi All,
> 
> I am trying to print the contents of  a file randomizing the lines.
> The following code works, but does not exit even after all the lines are 
> read.
> What needs to be done to correct it?

Try this 

package main

import(
"fmt"
"bufio"
"os"
"math/rand"
"sync"
"time"
)

func main() {
filename := os.Args[1]
if filename == "" {
panic("No filename")
}
fh,err := os.Open(filename)
if err != nil {
panic(err)
}
scanner := bufio.NewScanner(fh)
c := make(chan string)
var wg sync.WaitGroup
for scanner.Scan() {
line := scanner.Text()
wg.Add(1)
go func(l string){
time.Sleep(time.Duration(rand.Intn(30)) * time.Millisecond)
c <-  l
wg.Done()
}(line)
}
go func() {
wg.Wait()
close(c)
}()
for  {
myline, ok := <-c
if !ok {
return
}
fmt.Println(myline)
}
}

You need a way to determine when all of the spawned goroutines have sent
their line to the channel so you know when to exit the bottom for loop.
A select doesn't help, because the goroutines are sending on the channel
at random intervals, so the default case will cause a busy loop between
sends, not just after all sends have completed.

The WaitGroup keeps track of how many worker goroutines have not yet
finished.  The final goroutine (started below the top for loop) waits
for all the workers to finish, then closes the channel.

The bottom for loop does not use a select statement; it merely reads
from the channel, which will block until the next worker sends its line.
When the final goroutine closes the channel, the channel read will
return "", false, and the bottom for loop will exit.

As a general rule of thumb, use a select only when you have more than
one channel send and/or receive that you want to wait for, and only use
a default case when you have other work that can be done if the channel
operations are all blocked (waiting).  Don't use default in an otherwise
empty for loop around a select.

hth...Marvin

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] http ssl "can't identify protocol"

[Greg Saylor]

Hello,

I've been struggling with someone all day on a very busy server.   And I'm 
at a loss as to what is going on.   

I have a Go server listening on two ports, like so:

tls_config := MustGetTlsConfiguration(*ssl_cert, *ssl_key, 
*ssl_ca)
listener, _ := tls.Listen("tcp", ":"+*ssl_port, tls_config)
ssl_srv := {
ReadTimeout:  5 * time.Second,
WriteTimeout: 10 * time.Second,
Handler:  r_ssl,
}
go ssl_srv.Serve(listener)

and

tls_config := MustGetTlsConfiguration(*ssl_listen_cert, 
*ssl_listen_key, *ssl_listen_ca)
listener, _ := tls.Listen("tcp", ":"+*ssl_listen_port, 
tls_config)
ssl_srv := {
ReadTimeout:  5 * time.Second,
WriteTimeout: 10 * time.Second,
Handler:  l_ssl,
}
go ssl_srv.Serve(listener)

And here's that MustGetTlsConfiguration function:

func MustGetTlsConfiguration(certificateFile string, privateKeyFile string, 
caFile string) *tls.Config { 
config := {} 
mycert, certPool := MustLoadCertificates(certificateFile, 
privateKeyFile, caFile) 
config.Certificates = make([]tls.Certificate, 1) 
config.Certificates[0] = mycert 
 
config.RootCAs = certPool 
config.ClientCAs = certPool 
 
config.CipherSuites = []uint16{tls.TLS_RSA_WITH_AES_128_CBC_SHA, 
tls.TLS_RSA_WITH_AES_256_CBC_SHA, 
tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, 
tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, 
tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, 
tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, 
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, 
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256} 
 
config.MinVersion = tls.VersionTLS12 
 
config.SessionTicketsDisabled = true 
return config 
} 


The handler for these requests which do not read the response body look 
something like:

func Get(w http.ResponseWrite, r *http.request) {
w.WriteHeader(http.StatusNotFound)
w.Write([]byte("some reason"))
}

for responses that do read the request body, its more like:

func Get(w http.ResponseWrite, r *http.request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
   w.WriteHeader(http.StatusNotFound)
return
}
defer r.Body.Close()
w.WriteHeader(http.StatusNotFound)
w.Write([]byte("some reason"))
}


.. I've pruned and simplified this quite a bit, but I've tried to isolate 
it the problem to somewhere in this area...

What essentially happens is that the server starts off fine.   Then as the 
requests start to ramp up, all of a sudden I'll see sockets start to pile 
up that I can see in lsof in a state of "can't identify protocol".  If I 
monitor it very closely I can ocassionally see one or two.  But suddenly 
that number jumps and starts to climb rapidly.   After which point 
connections will go to CLOSE_WAIT status.   I'll end up with about 50 
connections in CLOSE_WAIT and the out of file handles because of "can't 
identify protocol" file handles.  Once its in this frozen state the server 
is non-response to any incoming http request.

I've tried increasing the number of file handles - but it exhibits the same 
behavior just taking longer to get to the frozen state.


I never saw this before upgrading to go 1.5.   I've tried 1.6 and 1.7 - 
same result.

I've also tried adding "defer r.Body.Close()" as the first line of every 
function call.   And I also tried preceding that with "defer 
ioutil.ReadAll(r.Body)".  This didn't seem to make any difference (not that 
I'd really expect it to after looking at the handler code).

I'll try to get a better stack trace, apparently syslog is not getting 
everything from kill -ABRT.  But here's one thing i saw:

Sep 20 08:11:46 ip-10-229-22-47 realtime: goroutine 42 [IO wait]:
Sep 20 08:11:46 ip-10-229-22-47 realtime: 
net.runtime_pollWait(0x7f5ba075c0c8, 0x72, 0x4232c3)
Sep 20 08:11:46 ip-10-229-22-47 realtime: 
#011/usr/local/go/src/runtime/netpoll.go:160 +0x5e
Sep 20 08:11:46 ip-10-229-22-47 realtime: 
net.(*pollDesc).wait(0xc4201eb480, 0x72, 0x41fb78, 0x9399e0)
Sep 20 08:11:46 ip-10-229-22-47 realtime: 
#011/usr/local/go/src/net/fd_poll_runtime.go:73 +0x5b
Sep 20 08:11:46 ip-10-229-22-47 realtime: 
net.(*pollDesc).waitRead(0xc4201eb480, 0xdb27a0, 0xc4200600c8)
Sep 20 08:11:46 ip-10-229-22-47 realtime: 
#011/usr/local/go/src/net/fd_poll_runtime.go:78 +0x42
Sep 20 08:11:46 ip-10-229-22-47 realtime: net.(*netFD).accept(0xc4201eb420, 
0x0, 0xdb1320, 0xc42020de40)
Sep 20 08:11:46 ip-10-229-22-47 realtime: 
#011/usr/local/go/src/net/fd_unix.go:419 +0x2b8
Sep 20 08:11:46 ip-10-229-22-47 realtime: 
net.(*TCPListener).accept(0xc420022708, 

[go-nuts] Randomizing contents of a file

[Dave Cheney]

Break breaks out of the select, not the for. Return would work here, or you 
could use a labled break. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: [ANN] Cgogen - automatic Go bindings generator released

[Maxim Kupriianov]

Hi Markus, nice project! I must agree that the subject-specific bindings
will always be superior over the generic ones. Another good example of that
is https://github.com/therecipe/qt bindings with custom generator as well.

As for LLVM, I'm trying to avoid using it for now, because that's a very
heavy dependency to have. Also that'd require rewriting more than half of
the current code. One day we may join our efforts working on a generic C
code transcriber, but that's another story.

> Maybe we can find some inspiration from each others projects?

I find the "ArrayNameFromLength" function curious, sadly things like that
are almost impossible in a generic context, even with YAML hints.
Take a look onto my helper pipeline (gen_bindings.go), I used that approach
instead of using templates that are pure evil for generating code.

I definitely will study your code deeply because it's interesting indeed to
compare our approaches to the same problems.
Feel free to reach me out :)



On Tue, Sep 20, 2016 at 1:23 PM, Markus Zimmermann 
wrote:

> This looks pretty neat. We did something similar for
> https://github.com/go-clang/ The generator is here
> https://github.com/go-clang/gen and a resulting binding is here
> https://github.com/go-clang/v3.7 Maybe we can find some inspiration from
> each others projects? It would be also interesting to figure out how we
> could merge each efforts?
>
> Cheers,
> Markus
>
> On Tuesday, September 20, 2016 at 10:19:14 AM UTC+2, Maxim Kupriianov
> wrote:
>>
>> Hello everyone,
>>
>> today I'm glad to announce that after 3 months of full-time development
>> back in 2015 and after 1 year of part-time field testing and improvements
>> in 2016,
>> an automatic CGo bindings generator for Golang is finally released to the
>> public. Visit https://cgogen.com
>>
>> Sources: http://github.com/xlab/cgogen
>> Documentation: https://github.com/xlab/cgogen/wiki
>>
>> That is the same generator that brought us Go bindings for Android NDK,
>> Vulkan Graphics API, CMU PocketSphinx, ALAC and Ogg/Vorbis decoders, Pure
>> Data embeddable library, PortAudio and PortMIDI adapters. And bindings for
>> the libpvpx from WebM are on their way.
>>
>> I hope the project will be useful for the community and awaiting for the
>> feedback and issues.
>> Good luck y all!
>>
>> --
>> Max
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/golang-nuts/3I7TzmEirbo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Randomizing contents of a file

[Unknown User]

Hi All,

I am trying to print the contents of  a file randomizing the lines.
The following code works, but does not exit even after all the lines are 
read.
What needs to be done to correct it?

package main

import(
"fmt"
"bufio"
"os"
"math/rand"
"time"
)

func main() {
filename := os.Args[1]
if filename == "" {
panic("No filename")
}
fh,err := os.Open(filename)
if err != nil { 
panic(err)
}
scanner := bufio.NewScanner(fh)
c := make(chan string)
for scanner.Scan() {
line := scanner.Text()
err := scanner.Err()
if err != nil { 
close(c)
}
go func(l string){
time.Sleep(time.Duration(rand.Intn(30)) * time.Millisecond)
c <-  l
}(line)
}
for  { 
select { 
case myline := <- c:
fmt.Println(myline)
default:
break
}
}
}
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [ANN] Cgogen - automatic Go bindings generator released

[Markus Zimmermann]

This looks pretty neat. We did something similar for 
https://github.com/go-clang/ The generator is here 
https://github.com/go-clang/gen and a resulting binding is here 
https://github.com/go-clang/v3.7 Maybe we can find some inspiration from 
each others projects? It would be also interesting to figure out how we 
could merge each efforts?

Cheers,
Markus

On Tuesday, September 20, 2016 at 10:19:14 AM UTC+2, Maxim Kupriianov wrote:
>
> Hello everyone, 
>
> today I'm glad to announce that after 3 months of full-time development 
> back in 2015 and after 1 year of part-time field testing and improvements 
> in 2016,
> an automatic CGo bindings generator for Golang is finally released to the 
> public. Visit https://cgogen.com
>
> Sources: http://github.com/xlab/cgogen
> Documentation: https://github.com/xlab/cgogen/wiki
>
> That is the same generator that brought us Go bindings for Android NDK, 
> Vulkan Graphics API, CMU PocketSphinx, ALAC and Ogg/Vorbis decoders, Pure 
> Data embeddable library, PortAudio and PortMIDI adapters. And bindings for 
> the libpvpx from WebM are on their way.
>
> I hope the project will be useful for the community and awaiting for the 
> feedback and issues.
> Good luck y all!
>
> --
> Max
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] [ANN] Cgogen - automatic Go bindings generator released

[Maxim Kupriianov]

Hello everyone, 

today I'm glad to announce that after 3 months of full-time development 
back in 2015 and after 1 year of part-time field testing and improvements 
in 2016,
an automatic CGo bindings generator for Golang is finally released to the 
public. Visit https://cgogen.com

Sources: http://github.com/xlab/cgogen
Documentation: https://github.com/xlab/cgogen/wiki

That is the same generator that brought us Go bindings for Android NDK, 
Vulkan Graphics API, CMU PocketSphinx, ALAC and Ogg/Vorbis decoders, Pure 
Data embeddable library, PortAudio and PortMIDI adapters. And bindings for 
the libpvpx from WebM are on their way.

I hope the project will be useful for the community and awaiting for the 
feedback and issues.
Good luck y all!

--
Max

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.