Re: [go-nuts] Vendored trace package causes panic because of implicit handlers

[Ian Davis]

On Wed, Jul 6, 2016, at 12:34 PM, Jakob Borg wrote:
> 2016-07-06 9:11 GMT+02:00 Peter Bourgon :
> > You can't actually make this distinction.
> > There should be a single vendor/ dir at the root of the repo.
> > See https://github.com/zellyn/wtf2 for a demonstration.
> > (And https://github.com/peterbourgon/wtf for context.)
> 
> Indeed. Which means that vendoring shouldn't be used in combination
> with "package at top level, cmd/ inside for commands" style repos. If
> the top level package isn't the binary, the repo represents a
> importable package and should not use vendoring.
> 
> The cmd/ directory can be used for small demo programs that don't
> require vendoring, but when it's something more complex it's time to
> split it out into it's own repo and use a better build solution like
> gb.
> 
> I know this is an unpopular opinion, but I really think it's time to
> recognize vendoring for what it is - an ugly hack to work around lack
> of proper package management - and not try to make it work in all the
> situations where it just doesn't...

I think the rules for Go organising public repositories to be compatible
with vendoring need to be as follows:

1) repositories that do not contain a "main" package must not vendor
dependencies

2) repositories that contain a "main" package must place that package at
the root of the repository and may vendor external dependencies in a
vendor directory at the root of the repository.  Any packages included
as source code within this repository should be in an internal directory
at the root of the repository.

I would characterise type 1 repositories as "libraries" and type 2 as
"binaries"

I think these 2 rules would solve all the recently reported issues with
vendoring 

-- Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Vendored trace package causes panic because of implicit handlers

[Peter Bourgon]

On Tue, Jul 5, 2016 at 9:19 PM, Sam Whited  wrote:
> On Tue, Jul 5, 2016 at 1:52 PM, Peter Bourgon  wrote:
>> Since you removed the vendor/ directory from jitsi/jap, everything is
>> working as expected, right?
>
> Yes, but the vendor dir was in the application code, not the library
> (eg. it was at jap/cmd/jap/vendor, not at jap/vendor). The library was
> doing the correct thing and not vendoring.

You can't actually make this distinction.
There should be a single vendor/ dir at the root of the repo.
See https://github.com/zellyn/wtf2 for a demonstration.
(And https://github.com/peterbourgon/wtf for context.)


>
> —Sam
>
>
> --
> Sam Whited
> pub 4096R/54083AE104EA7AD3

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Vendored trace package causes panic because of implicit handlers

[Chad]

Seems to me that not leaking API outside of the package it is being used is 
a must for a vendored package.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Vendored trace package causes panic because of implicit handlers

[Sam Whited]

On Tue, Jul 5, 2016 at 1:52 PM, Peter Bourgon  wrote:
> Since you removed the vendor/ directory from jitsi/jap, everything is
> working as expected, right?

Yes, but the vendor dir was in the application code, not the library
(eg. it was at jap/cmd/jap/vendor, not at jap/vendor). The library was
doing the correct thing and not vendoring.

—Sam


-- 
Sam Whited
pub 4096R/54083AE104EA7AD3

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Vendored trace package causes panic because of implicit handlers

[Peter Bourgon]

Since you removed the vendor/ directory from jitsi/jap, everything is
working as expected, right?

On Tue, Jul 5, 2016 at 8:46 PM, Sam Whited  wrote:
> On Tue, Jul 5, 2016 at 1:37 PM, Peter Bourgon  wrote:
>> As of today, if your binary vendors any other package, you need to
>> ensure that package doesn't have a vendor/ directory anywhere in the
>> source tree. Library maintainers can be good citizens by ensuring
>> their libraries don't vendor their dependencies; this is how the
>> jitsi/jap repo exists today, which is fine. But the responsibility is
>> ultimately on you; you can manually remove the vendor/ directory of
>> your vendored deps, or you can use a tool that does it automatically.
>
> I didn't follow this; the library *doesn't* have any vendor/ dir, nor
> do any of the other things the application has vendored.
>
> —Sam
>
>
>
> --
> Sam Whited
> pub 4096R/54083AE104EA7AD3
> https://blog.samwhited.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Vendored trace package causes panic because of implicit handlers

[Sam Whited]

On Tue, Jul 5, 2016 at 1:37 PM, Peter Bourgon  wrote:
> As of today, if your binary vendors any other package, you need to
> ensure that package doesn't have a vendor/ directory anywhere in the
> source tree. Library maintainers can be good citizens by ensuring
> their libraries don't vendor their dependencies; this is how the
> jitsi/jap repo exists today, which is fine. But the responsibility is
> ultimately on you; you can manually remove the vendor/ directory of
> your vendored deps, or you can use a tool that does it automatically.

I didn't follow this; the library *doesn't* have any vendor/ dir, nor
do any of the other things the application has vendored.

—Sam



-- 
Sam Whited
pub 4096R/54083AE104EA7AD3
https://blog.samwhited.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Vendored trace package causes panic because of implicit handlers

[Sam Whited]

On Tue, Jul 5, 2016 at 12:15 PM, Keith Rarick  wrote:
> You don't have to vendor anything from your own repo, but you do have to
> vendor everything from outside your repo. (To be precise, the distinction is
> between inside and outside the directory containing the vendor directory.)

In this case the repo's structure is:

library/
  library.go which imports "golang.org/x/net/trace"
  cmd/
main.go which imports library (using an absolute import, no
relative path hackery and "golang.org/x/net/trace")
vendor/
  all external libraries (including golang.org/x/net/trace)
required by main.go except "library"

To be specific, it's https://github.com/jitsi/jap (I've removed the
vendor/ dir temporarily though; browse back a commit or two).

> Aside from that, make sure that the dependency in question is being imported
> using the same import path by all the packages that need it. Not doing this
> is probably the most common cause of having multiple copies of a package
> linked in to your binary.

The import path is the same, it's always "golang.org/x/net/trace".

—Sam


-- 
Sam Whited
pub 4096R/54083AE104EA7AD3

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Vendored trace package causes panic because of implicit handlers

[Keith Rarick]

You don't have to vendor anything from your own repo, but you do have to
vendor everything from outside your repo. (To be precise, the distinction
is between inside and outside the directory containing the vendor
directory.)

Aside from that, make sure that the dependency in question is being
imported using the same import path by all the packages that need it. Not
doing this is probably the most common cause of having multiple copies of a
package linked in to your binary.


On Tue, Jul 5, 2016 at 8:29 AM Sam Whited  wrote:

> On Tue, Jul 5, 2016 at 10:17 AM, Konstantin Khomoutov
>  wrote:
> > AFAIK, vendoring explicitly implements an all-or-nothing approach:
> > you're supposed to resolve transitive dependencies yourself, and vendor
> > them all.
>
> That's my understanding too, but it seems foolish to vendor a version
> of a library that's in the same repo (eg. using the "root of the repo
> is the library, and a cmd/ directory has a few commands are thin
> wrappers around the library" project structure that's so common).
> Since keeping up to date with the library that actually implements the
> commands is part of the contract, vendoring it seems like a bad idea.
>
> —Sam
>
>
>
> --
> Sam Whited
> pub 4096R/54083AE104EA7AD3
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Vendored trace package causes panic because of implicit handlers

[Sam Whited]

On Tue, Jul 5, 2016 at 10:17 AM, Konstantin Khomoutov
 wrote:
> AFAIK, vendoring explicitly implements an all-or-nothing approach:
> you're supposed to resolve transitive dependencies yourself, and vendor
> them all.

That's my understanding too, but it seems foolish to vendor a version
of a library that's in the same repo (eg. using the "root of the repo
is the library, and a cmd/ directory has a few commands are thin
wrappers around the library" project structure that's so common).
Since keeping up to date with the library that actually implements the
commands is part of the contract, vendoring it seems like a bad idea.

—Sam



-- 
Sam Whited
pub 4096R/54083AE104EA7AD3

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Vendored trace package causes panic because of implicit handlers

[Konstantin Khomoutov]

On Tue, 5 Jul 2016 10:05:27 -0500
Sam Whited  wrote:

> I'm running into a problem with the trace package; I'm trying to build
> an application that has a vendored version of it, and one of the
> modifications I've made imports a library that also uses the trace
> package.
> 
> As I've complained about on the issue tracker before, the trace
> package registers some handlers implicitly in init() which in my mind
> is extremely bad behavior. However, since I've got a vendored version
> in my application, and one of the libraries I import also imports
> trace (and Go has no way of telling that these are the same package),
> the init() function gets called twice, tries to register the handler
> twice, and causes a panic.
> 
> Is there some way around this that I'm not thinking of (other than not
> vendoring trace, or also vendoring the other library that imports
> trace, which feels poor given that it's actually in the same repo as
> the application)?

AFAIK, vendoring explicitly implements an all-or-nothing approach:
you're supposed to resolve transitive dependencies yourself, and vendor
them all.

Basically that «Go has no way of telling that these are the same
package» is a part of the design of the Go toolchain: the package to
use is defined solely by its import path.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.