Re: [go-nuts] slice of pointer of struct vs slice of struct

[Jan Mercl]

On Fri, Oct 20, 2017 at 7:25 AM Feby Tanzil  wrote:

> Which is better & preferable in Go?

Depends on size of T.

-- 

-j

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] slice of pointer of struct vs slice of struct

[Feby Tanzil]

Hi,

I got vague answers in the internet over this.
Which is better & preferable in Go?

type T struct {
 // some attributes
 ...
}

func a() []T {

}

func b() []*T {

}



Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] repeatable builds

[Nathan Kerr]

Another option for 2 is https://github.com/cloudflare/hellogopher

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Any way to somehow include readable source code into compiled binaries?

[Samuel Lampa]

Hi Seb! :)

On Thursday, October 19, 2017 at 9:00:46 PM UTC+2, Sebastien Binet wrote:
>
> On Thu, Oct 19, 2017 at 8:51 PM, Alex Buchanan  > wrote:
>
>> But, maybe a better solution is to faithfully record the current git (or 
>> other VCS) details you'd need to find the code which resulted in the 
>> binary? That way you have access to the source in its natural state.
>>
>
> yes.
> using go-bindata is a nice, quick'n'dirty solution for the problem at 
> hand, but I fear it's just duplication of work somehow.
> why not just using "dep" that captures the state of the code (ie: git sha) 
> *and* its dependencies.
>
> this sounds more reliable. at least to me.
> and it encourages people to put their code into a repository which, for 
> reproducibility of scientific results, is step-0.
>

Absolutely, and this is what we do for our own use.

What I'm thinking about is how the ability to compile a workflow to a 
binary will make it easy to ship "ready made" workflows for less tech-savvy 
users, who might not be comfortable using git and the go tools, or might do 
unwarranted, undocumented changes if we'd just give them the Go source and 
the Go tools.

Best
// Samuel

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Any way to somehow include readable source code into compiled binaries?

[Samuel Lampa]

We use go-bindata to bundle text files (html, js, css, yaml) into our 
binary. I bet you could use it to bundle Go source.


https://github.com/jteeuwen/go-bindata 





Ah, this is great, Thanks!

On 2017-10-19 20:51, Alex Buchanan wrote:
But, maybe a better solution is to faithfully record the current git 
(or other VCS) details you'd need to find the code which resulted in 
the binary? That way you have access to the source in its natural state.


Yes, this is what we do not, but it turns out, in reality it is often 
too easy that these get disconnected for whatever reason, like that very 
tempting last minute fix, file open in multiple editors at the same time 
(we're on a shared cluster), or similar. Having a "locked" link between 
what is executed and its definition, will make us sleep better at night, 
I think :)


Thanks, and Cheers
// Samuel

--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Any way to somehow include readable source code into compiled binaries?

[Sebastien Binet]

On Thu, Oct 19, 2017 at 8:51 PM, Alex Buchanan 
wrote:

> But, maybe a better solution is to faithfully record the current git (or
> other VCS) details you'd need to find the code which resulted in the
> binary? That way you have access to the source in its natural state.
>

yes.
using go-bindata is a nice, quick'n'dirty solution for the problem at hand,
but I fear it's just duplication of work somehow.
why not just using "dep" that captures the state of the code (ie: git sha)
*and* its dependencies.

this sounds more reliable. at least to me.
and it encourages people to put their code into a repository which, for
reproducibility of scientific results, is step-0.

-s

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Any way to somehow include readable source code into compiled binaries?

[Alex Buchanan]

But, maybe a better solution is to faithfully record the current git (or 
other VCS) details you'd need to find the code which resulted in the 
binary? That way you have access to the source in its natural state.

On Thursday, October 19, 2017 at 11:49:39 AM UTC-7, Alex Buchanan wrote:
>
> We use go-bindata to bundle text files (html, js, css, yaml) into our 
> binary. I bet you could use it to bundle Go source.
>
> https://github.com/jteeuwen/go-bindata
>
> -Alex
>
> On Thursday, October 19, 2017 at 9:46:37 AM UTC-7, Samuel Lampa wrote:
>>
>> Hi Gophers,
>>
>> *Question:* Is there any way to include a readable verison of the source 
>> code of my program into the compiled Go binary, either in plain text, or 
>> the ability extract it somehow, with some go tooling or 3rd party tools?
>>
>> *Background:* The reason for asking is that we are using Go to write 
>> scientific workflows with scipipe . This means we 
>> can compile our scientific workflows into static binaries for easier 
>> deployment and protecting against undocumented code changes that code make 
>> us loose track of the provenance ("complete track record") of the steps ... 
>> and potentially lead to flawed science ("shudder").
>>
>> But with compiled binaries, we instead run into another problem of 
>> provenance: Allowing ourselves and others to inspect that a particular 
>> workflow binary actually does what it says.
>>
>> Thus, having a statically compiled binary with the source code included 
>> in some form, would be the best of two worlds.
>>
>> Cheers
>> // Samuel
>> PhD Student @ pharmb.io
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Panic recovery and mutex lock question

[Tamás Gulácsi]

Quote from the language reference:
"A "defer" statement invokes a function whose execution is deferred to the 
moment the surrounding function returns, either because the surrounding 
function executed a return statement, reached the end of its function body, or 
because the corresponding goroutine is panicking."

So, a deferred function will be called in case of panic, too.
No need to unlock in the panic handler!

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Any way to somehow include readable source code into compiled binaries?

[Alex Buchanan]

We use go-bindata to bundle text files (html, js, css, yaml) into our 
binary. I bet you could use it to bundle Go source.

https://github.com/jteeuwen/go-bindata

-Alex

On Thursday, October 19, 2017 at 9:46:37 AM UTC-7, Samuel Lampa wrote:
>
> Hi Gophers,
>
> *Question:* Is there any way to include a readable verison of the source 
> code of my program into the compiled Go binary, either in plain text, or 
> the ability extract it somehow, with some go tooling or 3rd party tools?
>
> *Background:* The reason for asking is that we are using Go to write 
> scientific workflows with scipipe . This means we can 
> compile our scientific workflows into static binaries for easier deployment 
> and protecting against undocumented code changes that code make us loose 
> track of the provenance ("complete track record") of the steps ... and 
> potentially lead to flawed science ("shudder").
>
> But with compiled binaries, we instead run into another problem of 
> provenance: Allowing ourselves and others to inspect that a particular 
> workflow binary actually does what it says.
>
> Thus, having a statically compiled binary with the source code included in 
> some form, would be the best of two worlds.
>
> Cheers
> // Samuel
> PhD Student @ pharmb.io
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] nil maps

[roger peppe]

Or taking it a bit further: https://play.golang.org/p/g4uF2QjiJQ


On 18 October 2017 at 23:38, Alex Dvoretskiy  wrote:
> Looks fun!: https://play.golang.org/p/8mqzb-1H7f
>
> On Tuesday, October 17, 2017 at 10:39:46 AM UTC-7, Thomas Bushnell, BSG
> wrote:
>>
>> Here's a case that comes up for me:
>>
>> type table map[string]map[string]string
>>
>> func (t table) add(x, y, z string) {
>>   if t[x] == nil {
>> t[x] = make(map[string]string)
>>   }
>>   t[x][y] = z
>> }
>>
>> func (t table) get(x, y string) string {
>>   return t[x][y]
>> }
>>
>> The fact that t[x] can be indexed even if it hasn't been created makes
>> this much simpler.
>>
>> On Tue, Oct 17, 2017 at 12:52 AM Alex Dvoretskiy 
>> wrote:
>>>
>>> Hello, Golang Nuts!
>>>
>>> I have an interesting question about maps. What is the possible usage of
>>> nil maps, which can be declared like "var m map[string]int"? You can't write
>>> to nil map but have an option to create it.
>>>
>>> Perhaps there is no use at all and this is just language specific
>>> feature?
>>>
>>> Thank you.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to golang-nuts...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Blocking in C functions called from go

['Pushkar Pradhan' via golang-nuts]

Konstantin,
Thanks for the detailed explanation. That was helpful.

On Thu, Oct 19, 2017 at 3:51 AM, Konstantin Khomoutov 
wrote:

> On Fri, Oct 13, 2017 at 12:23:42PM -0700, 'Pushkar' via golang-nuts wrote:
>
> > I need to call some C functions from Go. I think I know how to proceed
> > using cgo.
> > However, I was wondering how goroutines and blocking calls in the C
> > functions work together.
> > So in the below example (pseudocode) will the goroutine be suspended when
> > the pthread_lock call is waiting to acquire the lock?
> >
> > I am guessing that the Go engine will detect that the OS thread on which
> > the goroutine(s) are running is blocked so it will suspend them too.
> > Once the thread is runnable, the blocked goroutines will be scheduled to
> > run.
>
> To what Ian said, I would add only a single goroutine runs on a given
> OS thread at any given time, and when a goroutine enters a cgo call, the
> runtime makes sure that goroutine gets locked to the underlying OS
> thread it's running on (as if the goroutine explicitly called
> runtime.LockOSThread()). This makes sure both Go and C sides are
> consistent about the thread their code executes on during the cgo call.
>
> Another thing to keep in mind is that the Go runtime scheduler gives a
> goroutine which blocks in a syscall or a cgo calls about 20 us (as of Go
> 1.9) to complete [1], and if it doesn't, the scheduler parks the
> goroutine and may spawn another OS thread if there are runnable
> goroutines -- to keep up with its promise to have runtime.GOMAXPROCS
> goroutines running.
>
> A corollary to this is that if you make many such blocking cgo calls
> roughly in parallel, this might lead to spawning many OS threads.
> This is not a bad thing in itself, but it's something to keep in mind.
>
> 1. https://github.com/golang/go/issues/21827#issuecomment-329103577
>
>


-- 
pushkar

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] C's readdir equivalent

[Bakul Shah]

On Thu, 19 Oct 2017 09:29:09 -0700 Ian Lance Taylor  wrote:
Ian Lance Taylor writes:
> On Wed, Oct 18, 2017 at 10:54 PM, Bakul Shah  wrote:
> >
> > os.Readdirnames() just returns dir enry names[1], while
> > os.Readdir() lstats all the dir entries.
> >
> > Pretty much every OS's (C lang) readdir() returns a direntry's
> > type without having to stat the underlying file.  More often
> > than not this is all one wants -- not everyone wants to walk a
> > directory tree -- and even there filepath.Walk() lstats all
> > the files unnecessarily.
> 
> (As far as I know Solaris does not yet provide the direntry's type.)

I should've checked.  It is not in the POSIX defn of dirent
either.  I guess most older filesystems won't

> > I have directories with thousands of files and a very smal
> > number of subdirs so the lstat overhead is considerable.
> >
> > Is there a package that does ReadDirents() returning a pair
> > ([]string, []byte) or []struct {type byte; name string}?
> 
> I'm not aware of one.  It would need to be somewhat system-specific as
> different systems lay out the directory entries differently.  It
> should be possible to write such a package building on ReadDirent and
> ParseDirent in golang.org/x/sys/unix.

Ok, I'll put something together. It will return DT_UNKNOWN for
each entry on OSes that don't support this feaure.

Thanks!

Bakul

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Any way to somehow include readable source code into compiled binaries?

[Samuel Lampa]

Hi Gophers,

*Question:* Is there any way to include a readable verison of the source 
code of my program into the compiled Go binary, either in plain text, or 
the ability extract it somehow, with some go tooling or 3rd party tools?

*Background:* The reason for asking is that we are using Go to write 
scientific workflows with scipipe . This means we can 
compile our scientific workflows into static binaries for easier deployment 
and protecting against undocumented code changes that code make us loose 
track of the provenance ("complete track record") of the steps ... and 
potentially lead to flawed science ("shudder").

But with compiled binaries, we instead run into another problem of 
provenance: Allowing ourselves and others to inspect that a particular 
workflow binary actually does what it says.

Thus, having a statically compiled binary with the source code included in 
some form, would be the best of two worlds.

Cheers
// Samuel
PhD Student @ pharmb.io

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] C's readdir equivalent

[Ian Lance Taylor]

On Wed, Oct 18, 2017 at 10:54 PM, Bakul Shah  wrote:
>
> os.Readdirnames() just returns dir enry names[1], while
> os.Readdir() lstats all the dir entries.
>
> Pretty much every OS's (C lang) readdir() returns a direntry's
> type without having to stat the underlying file.  More often
> than not this is all one wants -- not everyone wants to walk a
> directory tree -- and even there filepath.Walk() lstats all
> the files unnecessarily.

(As far as I know Solaris does not yet provide the direntry's type.)


> I have directories with thousands of files and a very smal
> number of subdirs so the lstat overhead is considerable.
>
> Is there a package that does ReadDirents() returning a pair
> ([]string, []byte) or []struct {type byte; name string}?

I'm not aware of one.  It would need to be somewhat system-specific as
different systems lay out the directory entries differently.  It
should be possible to write such a package building on ReadDirent and
ParseDirent in golang.org/x/sys/unix.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: dynamic xml

[Jim Cote]

You'll need to use an UnmarshalXML func for the params:

// Param is a tag w/ any name
type Param struct {
Namestring `xml:"-"`
Typestring `xml:"type,attr"`
MinOccurs   int`xml:"minOccurs,attr"`
MaxOccurs   int`xml:"maxOccurs,attr"`
Description string `xml:"description"`
}


// ParamList type for unmarhsal
type ParamList []Param


// UnmarshalXML for the win
func (pl *ParamList) UnmarshalXML(d *xml.Decoder, s xml.StartElement) error 
{
tk, err := d.Token()
for err == nil {
switch t := tk.(type) {
case xml.StartElement:
param := Param{Name: t.Name.Local}
if err = d.DecodeElement(, ); err != nil {
return err
}
*pl = append(*pl, param)
 }
 if tk, err = d.Token(); err == io.EOF {
 return nil
 }
}
return err
}


// TopLevel is of course top level
type TopLevel struct {
Description string   `xml:"description"`
Rights  Rights   `xml:"rights"`
Method  []Method `xml:"method"`
}


// Method has request
type Method struct {
Namestring   `xml:"name,attr"`
Description string   `xml:"description"`
Rights  []Right  `xml:"rights"`
Request Request  `xml:"request"`
ResponseResponse `xml:"response"`
}


// Right whatever
type Right struct {
Right string `xml:"right"`
}


// Request like Response
type Request struct {
Params ParamList `xml:"params"`
}


// Response like Request
type Response struct {
Params ParamList `xml:"params"`
}


// Rights 
type Rights struct {
PrivLevel string `xml:"privLevel"`
}


On Wednesday, October 18, 2017 at 11:50:45 AM UTC-4, Jeffrey Smith wrote:
>
> I have a lot of XML documents that have sections in that contain tags that 
> can be of any name but will contain type,minOccurs,maxOccurs and have a 
> description inside it. For instance name,description,definition and id 
> below.
>
>
>
> 
> 
>   description number1
>   
> user
>   
>   
> graphCreate API
> 
> 
>   
> 
>   A friendly name to identify the graph
> 
> 
>   Detailed description of the graph
> 
> 
>   Specify the graph definition. i.e how this graph 
> should be built.
> 
>   
> 
> 
>   
>
>   graph identifier.
> 
>   
> 
>   
>  
>
> Whats the best way to parse this. So far I have but can't work out how to 
> dynamically generate the struct to stuff this into. Any suggestions?
>
> type topLevel struct {
> Description string   `xml:"description"`
> Rights  rights   `xml:"rights"`
> Method  []method `xml:"method"`
> }
>
>
> type method struct {
> Namestring  `xml:"name,attr"`
> Description string  `xml:"description"`
> Rights  []right `xml:"rights"`
> }
>
>
> type right struct {
> Right string `xml:"right"`
> }
>
>
> type request struct {
> }
>
>
> type rights struct {
> PrivLevel string `xml:"privLevel"`
> }
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: dynamic xml

[Jim Cote]

You'll need to use a custom UnmarshalXML func to handle this.



// TopLevel is of course top level
type TopLevel struct {
 Description string   `xml:"description"`
 Rights  Rights   `xml:"rights"`
 Method  []Method `xml:"method"`
}


// Method has request
type Method struct {
 Namestring   `xml:"name,attr"`
 Description string   `xml:"description"`
 Rights  []Right  `xml:"rights"`
 Request Request  `xml:"request"`
 ResponseResponse `xml:"response"`
}


// Right whatever
type Right struct {
 Right string `xml:"right"`
}


// Request like Response
type Request struct {
 Params ParamList `xml:"params"`
}


// Response like Request
type Response struct {
 Params ParamList `xml:"params"`
}


// ParamList type for unmarhsal
type ParamList []Param


// UnmarshalXML for the win
func (pl *ParamList) UnmarshalXML(d *xml.Decoder, s xml.StartElement) error 
{
 tk, err := d.Token()
 for err == nil {
 switch t := tk.(type) {
 case xml.StartElement:
 var param = Param{Name: t.Name.Local}
 if err = d.DecodeElement(, ); err != nil {
 return err
 }
 *pl = append(*pl, param)
 }
 tk, err = d.Token()
 }
 return err
}


// Param holds data
type Param struct {
 Namestring `xml:"name"`
 Typestring `xml:"type,attr"`
 MinOccurs   int`xml:"minOccurs,attr"`
 MaxOccurs   int`xml:"maxOccurs,attr"`
 Description string `xml:"description"`
}


// Rights hmm
type Rights struct {
 PrivLevel string `xml:"privLevel"`
}


On Wednesday, October 18, 2017 at 11:50:45 AM UTC-4, Jeffrey Smith wrote:
>
> I have a lot of XML documents that have sections in that contain tags that 
> can be of any name but will contain type,minOccurs,maxOccurs and have a 
> description inside it. For instance name,description,definition and id 
> below.
>
>
>
> 
> 
>   description number1
>   
> user
>   
>   
> graphCreate API
> 
> 
>   
> 
>   A friendly name to identify the graph
> 
> 
>   Detailed description of the graph
> 
> 
>   Specify the graph definition. i.e how this graph 
> should be built.
> 
>   
> 
> 
>   
>
>   graph identifier.
> 
>   
> 
>   
>  
>
> Whats the best way to parse this. So far I have but can't work out how to 
> dynamically generate the struct to stuff this into. Any suggestions?
>
> type topLevel struct {
> Description string   `xml:"description"`
> Rights  rights   `xml:"rights"`
> Method  []method `xml:"method"`
> }
>
>
> type method struct {
> Namestring  `xml:"name,attr"`
> Description string  `xml:"description"`
> Rights  []right `xml:"rights"`
> }
>
>
> type right struct {
> Right string `xml:"right"`
> }
>
>
> type request struct {
> }
>
> Enter code here...
>
>
> type rights struct {
> PrivLevel string `xml:"privLevel"`
> }
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: "html/dom" alternative to html/template for true separation of concerns?

[jens . schoedt]

@Karv Prime 
I was looking for a library for this but i have not been able to find any 
yet. Did you have any luck? I have worked with http://www.thymeleaf.org/ 
for Java extensively and I really enjoyed it. Its a lot easier to work with 
the front-end people with legal html. Dom manipulation gives a lot more 
separation than is possible with simple template/injection in my opinion. 
Maybe we should start on making a golang lib for this... :-)

@Marvin Renich
Not to start a war on which approach is better but when using the dom 
manipulation.

Hello, {{.Name}}, you last logged in on {{.PrevLogin}}.

would be:

All this text will be replaced by the hello var. This 
way the front end developer can test with long/short texts (user names) and 
see how they look

Also if doing localization you would not have static text like that in your 
html template.

Also:
  - is much better IMO. The front-end guy 
can put in whatever html he needs to test the div content and it will all 
get replaced when the dom is parsed.



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Help with viper please.

[John Episcopo]

Hi guys,

I'm having trouble with this too.

My code is as follows:
v := viper.New()

viper.SetConfigName("settings")
viper.SetConfigType("json")
viper.AddConfigPath(".")
v.AutomaticEnv()
err := v.ReadInConfig()
if err != nil {
logrus.Fatal("Fatal error config file: ", err)
}

However, I'm getting the error: Config File "config" Not Found in "[]". 

I've tried it with and without the SetConfigType but the error remains 
consistent. I've confirmed that the config file is in the same directory as 
the main.go file where this is being called too. Another odd thing is that 
the error message says Config File "config" not found, yet the config name 
is being set to "settings" (it had previously been called "config" but I 
changed it a while ago).

Is there anything that you can see that I'm doing wrong? I'm running the 
app using the 'go run main.go' command too.

My config is as follows:
{
"EMAIL_IDENTITY": "*",
"EMAIL_USERNAME": "*",
"EMAIL_PASSWORD": "*",
"EMAIL_HOST": "*",
"EMAIL_PORT": "*",
"EMAIL_RECIPIENT_NAME": "*"
}

Thanks,
John


On Sunday, September 10, 2017 at 1:05:40 AM UTC+1, Tim Uckun wrote:
>
> I have this code
>
> v := viper.New()
> v.SetConfigFile("settings.yaml")
> v.AddConfigPath(".")
> v.AddConfigPath("./settings")
> v.AddConfigPath("./config")
> err := v.ReadInConfig()
> if err != nil {
> panic(fmt.Errorf("Error when reading config: %v\n", err))
> }
>
> if as the instructions say I don't put the file extension in the 
> SetConfigFile it fails and says it doesn't recognize the type "". so I am 
> specifying the full name of the file.
>
> If I put this in the "." directory (same directory as my go file) it works
>
> if I put the file in the config or the settings or config directories it 
> doesn't work and gives me the following error
>
>
> panic: Error when reading config: open settings.yaml: no such file or 
> directory
>
> How do I specify a relative path to viper?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Panic recovery and mutex lock question

[David Renne]

OK I think I have my questions answered, I think the moral is dont intermix 
or add code above your lock if you are worried about any panics calling it 
before, this way you know that you are in a lock right when the function 
hits it and can always defer the unlock until after.   I suppose in some 
really bad code with a panic handler someone could accidentally throw a 
panic deeper down in a function and since that code was above the Lock the 
panic handler wouldnt know whether its open or not.  I just wish sometimes 
we could know whether a lock is open or not on a given sync.RWMutex as a 
public variable.  Oh well... 

I really like some of Juliusz's examples to separate out the locks with 
anonymous functions because it keeps the logic safer from anyone adding 
code or function calls that can be more dangerous for things like panic 
handlers.  

Thanks everyone, I think I am good on this question and my brother solved 
some of our web socket lock issues in our code dealing with websockets.

https://github.com/DanielRenne/GoCore/commit/1a6418c2734f985955571ff6015f8d82736890ae

I still get scared any minute a client websocket or some unhandled thing 
might cause a lock, but only time will tell.  Node.js websockets are easy 
because theres only one thread!

If anyone feels like reviewing master of this project and helping us find 
some potential bad code, we need some golang experts to review any lock 
practices we can improve on problems before they become problems and 
refactor any bad code out.  Sometimes I wish we found this melody library 
which handles all the locking for websockets with 
gorilla package.  https://github.com/olahol/melody  Oh well.  That package 
might not even work for our use case anyway.

I removed some unlocks he put in on the recovery because the second defer 
will always be called regardless of panic right?  There is no way 
golang will skip any deferred on a panic for any reason right?

https://github.com/DanielRenne/GoCore/commit/9685fb3f6fabe612e005238119349144da3bba18

Thanks for listening to my rants everyone!  If anyone explores some of our 
bad code and wants to help us just reply here or email me direct from 
github https://github.com/davidrenne/

On Tuesday, October 17, 2017 at 8:15:23 PM UTC-4, David Renne wrote:
>
> Hi there,
>
> I was wondering if the code ever panics and we have a if recover := 
> recover(); recover != nil {
>
> in a defer func and we are using sync.RWMutexes which do not have a defer 
> to unlock for example.
>
> What kinds of things can happen if the code unexpectedly panics when we 
> are in a Lock() in the code and then something panics before it Unlocks.  
> My defer funcs almost never unlock the syncRW mutexes or any RLock()'s that 
> were called to unlock.
>
> Should maybe if we are in the recover() block should we then call Unlock() 
> or RUnlock on all sync.RWMutex's that the function is recovering from?  How 
> would one know if you are in a lock or not if we made it far enough to open 
> the lock to know which ones to unlock if it truly keeps the lock open?
>
> Since we rarely get panics and we are dealing with some deadlocks, I am 
> asking myself this question and dont really know how to test it easily to 
> verify what really happens.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: perfs of a Go binary VS C++

[mura]

Hi,
Can you publish the C++ test program as well ? I have the ROOT installed 
and would like to explore the issue further.

-mura

On Wednesday, October 18, 2017 at 6:21:17 PM UTC+8, mura wrote:
>
> Could you use `perf record ./test f64s.root` and then `perf report 
> --stdio` for both the Go and C++ programs? (You may need to `echo 0 | sudo 
> tee /proc/sys/kernel/kptr_restrict before perf record, supposing that you 
> can test it on Linux)
>
> I am just interested in the performance comparison between C++ and Go too. 
> :D
>
> -mura
>
> On Tuesday, October 17, 2017 at 6:48:43 PM UTC+8, Sebastien Binet wrote:
>>
>> hi,
>>
>> (thanks for having a look at this)
>>
>> On Tue, Oct 17, 2017 at 6:01 AM, mura  wrote:
>>
>>> Hi,
>>>
>>> A sequential read with `dd if=f64s.root of=/dev/null bs=32` takes around 
>>> 8.5 secs (10.4 secs using read-data.go) on my machine. It would even take 
>>> up to 55 secs when bs = 1.
>>> Also I observe that the MaxRSS of the C++ program is much bigger than 
>>> the Go version, so I suspect the C++ program probably does some kind of 
>>> caching or preloading.
>>>
>>
>> most of that extra RSS is coming from the C++ libraries that are linked 
>> against.
>> that said, it's true the C++ version does both caching and prefetching.
>> I am trying to devise a test case where both are disabled.
>>
>> but, if I were to ioutil.ReadFile("foo.root") and then use that []byte 
>> slice as input (implementing all the io.Reader, io.ReaderAt ... needed 
>> interfaces) I should be able to at least recoup the caching effect, right? 
>> (that's in effect pretty much the same thing than mmap-ing the whole in 
>> memory, isn't?)
>>
>> -s
>>  
>>
>>>
>>> -mura
>>>
>>>
>>> On Monday, October 16, 2017 at 11:53:56 PM UTC+8, Sebastien Binet wrote:

 hi there,

 I have this Go program that reads a binary file format coming from CERN.
 I am trying to get it running almost as fast as its C++ counter-part.

 the profile and code are here:
 - https://cern.ch/binet/go-hep/rootio.cpu.pprof
 - https://cern.ch/binet/go-hep/read-data.go

 on my machine, I get the following timings for the C++ (g++-7.2.0) and 
 the Go-1.9.1 programs:

 === C++/ROOT ===
 real=6.84 user=6.29 sys=0.52 CPU=99% MaxRSS=258384 I/O=0/0
 real=6.99 user=6.42 sys=0.55 CPU=99% MaxRSS=258312 I/O=0/0
 real=6.68 user=6.12 sys=0.55 CPU=99% MaxRSS=257956 I/O=0/0
 real=6.88 user=6.33 sys=0.54 CPU=99% MaxRSS=258964 I/O=0/0
 real=6.87 user=6.30 sys=0.56 CPU=99% MaxRSS=258024 I/O=0/0
 real=6.89 user=6.32 sys=0.56 CPU=99% MaxRSS=260452 I/O=0/0
 real=6.70 user=6.14 sys=0.56 CPU=99% MaxRSS=258368 I/O=0/0
 real=7.02 user=6.47 sys=0.54 CPU=99% MaxRSS=258128 I/O=0/0
 real=7.46 user=6.44 sys=0.56 CPU=93% MaxRSS=257972 I/O=1840/0
 real=7.11 user=6.47 sys=0.55 CPU=98% MaxRSS=258148 I/O=3984/0

 === go-hep/rootio ===
 real=13.06 user=12.43 sys=0.61 CPU=99%  MaxRSS=40864 I/O=0/0
 real=13.05 user=12.46 sys=0.58 CPU=100% MaxRSS=40892 I/O=0/0
 real=13.05 user=12.46 sys=0.59 CPU=100% MaxRSS=40888 I/O=0/0
 real=13.07 user=12.48 sys=0.58 CPU=99%  MaxRSS=40940 I/O=0/0
 real=13.08 user=12.48 sys=0.60 CPU=100% MaxRSS=40924 I/O=0/0
 real=13.09 user=12.46 sys=0.60 CPU=99%  MaxRSS=40976 I/O=0/0
 real=13.01 user=12.41 sys=0.60 CPU=100% MaxRSS=40876 I/O=0/0
 real=13.09 user=12.52 sys=0.58 CPU=100% MaxRSS=40880 I/O=0/0
 real=13.01 user=12.41 sys=0.60 CPU=100% MaxRSS=40880 I/O=0/0
 real=12.99 user=12.41 sys=0.58 CPU=100% MaxRSS=40872 I/O=0/0

 any idea on how to speed that up?

 I have also this rather interesting strace-log for the Go program:
 - https://cern.ch/binet/go-hep/strace-read-go.txt

 ie: there is a clear pattern of "epoll_wait/pselect6" on one thread 
 followed by a bunch of "pread64" on another thread.
 I suspect this is the runtime I/O layer (?) dispatching data to the 
 "worker" thread/goroutine ?
 (I have tried with GOGC=off and it was still there, as well as the same 
 timings for my Go binary.)

 I have put a ~764Mb data file there:
 - https://cern.ch/binet/go-hep/f64s.root

 if anybody wants to have a look.

 I initially thought it was a (lack of) bufio.Reader issue (as the file 
 format forces you to do a bunch of seeks or readat, using bufio.Reader 
 wasn't completely straightforward.)
 but, mmap-ing the whole file in memory didn't help.

 here is the top50 from the CPU profile:

 (pprof) top50   
 Showing nodes accounting for 12.90s, 99.15% of 13.01s total
 Dropped 4 nodes (cum <= 0.07s)
   flat  flat%   sum%cum   cum%
  2.83s 21.75% 21.75%  3.21s 24.67%  
 go-hep.org/x/hep/rootio.(*LeafD).readBasket 
 /home/binet/dev/go/gocode/src/go-hep.org/x/hep/rootio/leaf_gen.go
  1.35s 10.38% 32.13% 12.99s 99.85%  
 

Re: [go-nuts] Re: perfs of a Go binary VS C++

[Sebastien Binet]

On Thu, Oct 19, 2017 at 8:32 AM, Sokolov Yura 
wrote:

> Why io is not buffered?
> https://github.com/go-hep/hep/blob/master/rootio/file.go#L95
>
> Use bufio
>

that was also my first knee-jerk reaction.
but I can't do that:
the reader needs to implement io.Reader, io.ReaderAt and io.Seeker.
bufio.Reader doesn't implement the last two.

that's why I've implemented (in another branch) a little reader that just
call ioutil.ReadFile, works off the raw []byte and the implements the above
3 interfaces.
but I didn't notice any improvements overall.

-s

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: perfs of a Go binary VS C++

[Sokolov Yura]

Why io is not buffered?
https://github.com/go-hep/hep/blob/master/rootio/file.go#L95

Use bufio

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.