subject:"\[PATCH 0\/1\] Go importer"

Re: [PATCH 0/1] Go importer

2018-07-26 Thread Ludovic Courtès

Hello!

Pierre-Antoine Rouby  skribis:

>> From: "Leo Famulari" 
>> On Thu, Jul 19, 2018 at 08:56:03AM +0200, Pierre-Antoine Rouby wrote:
>>> I trying to import 'gitlab-runner' 
>>> (https://gitlab.com/gitlab-org/gitlab-runner)
>>> with tag 'v10.6.0'.
>> 
>> Okay, thanks. I can reproduce the error:
>
> I have fix this issue (patch attached), let me know if that work for you.

Seems to me that this importer is almost ready.  Some comments:

> From 2f97b62beeacc58935a86d08a1635c082d078189 Mon Sep 17 00:00:00 2001
> From: Rouby Pierre-Antoine 
> Date: Wed, 25 Jul 2018 10:34:44 +0200
> Subject: [PATCH] import: Add gopkg importer.
>
> * guix/import/gopkg.scm: New file.
> * guix/scripts/import/gopkg.scm: New file.
> * guix/scripts/import.scm: Add 'gopkg'.
> * Makefile.am: Add 'gopkg' importer in modules list.

Please mention the guix.texi changes as well.

It would be good to have unit tests in tests/go.scm for the Toml parser
and/or for the whole importer, like we do in tests/gem.scm, for
instance.

> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -167,7 +167,7 @@ MODULES = \
>guix/build/rpath.scm   \
>guix/build/cvs.scm \
>guix/build/svn.scm \
> -  guix/build/syscalls.scm   \
> +  guix/build/syscalls.scm\

Please remove whitespace changes.

> +@item gopkg
> +@cindex gopkg
> +@cindex Golang
> +@cindex Go
> +Import metadata from the @uref{https://gopkg.in/, gopkg} package
> +versioning service used by some Go software.

A few words on limitations (if any ;-)), or mentioning that it’s
recursive by default (right?), and an example would be nice.

> +(define (vcs-file? file stat)

With “TODO: Factorize” please.  :-)

> +(define (file->hash-base32 file)
> +  "Return hash of FILE in nix base32 sha256 format.  If FILE is a directory,
> +exclude vcs files."
> +  (let-values (((port get-hash) (open-sha256-port)))
> +(write-file file port #:select? (negate vcs-file?))

Shouldn’t it be #:select? vcs-file? ?

> +(define (git->hash url commit file)
> +  "Clone git repository and return FILE hash in nix base32 sha256 format."
> +  (if (not (file-exists? (string-append file "/.git")))
> +  (git-fetch url commit file #:recursive? #f))
> +  (file->hash-base32 file))
> +
> +(define (git-ref->commit path tag)
> +  "Return commit number coresponding to git TAG.  Return \"XXX\" if tag is 
> not
> +found."
> +  (define (loop port)
> +(let ((line (read-line port)))
> +  (cond
> +   ((eof-object? line)  ; EOF
> +(begin
> +  (close-port port)
> +  "XXX"))
> +   ((string-match tag line) ; Match tag
> +(let ((commit (car (string-split (transform-string line #\tab " ")
> + #\ 
> +  commit))
> +   (else; Else
> +(loop port)
> +
> +  (let ((file (if (file-exists? (string-append path "/.git/packed-refs"))
> +  (string-append path "/.git/packed-refs")
> +  (string-append path "/.git/FETCH_HEAD"
> +(loop (open-input-file file
> +
> +(define* (git-fetch url commit directory
> +#:key (git-command "git") recursive?)
> +  "Fetch COMMIT from URL into DIRECTORY.  COMMIT must be a valid Git commit
> +identifier.  When RECURSIVE? is true, all the sub-modules of URL are fetched,
> +recursively.  Return #t on success, #f otherwise."
> +  (mkdir-p directory)
> +  
> +  (with-directory-excursion directory
> +(invoke git-command "init")
> +(invoke git-command "remote" "add" "origin" url)
> +(if (zero? (system* git-command "fetch" "--depth" "1" "origin" commit))
> +(invoke git-command "checkout" "FETCH_HEAD")
> +(begin
> +  (invoke git-command "fetch" "origin")
> +  (if (not (zero? (system* git-command "checkout" commit)))
> +  (let ((commit-hash (git-ref->commit directory commit)))
> +(invoke git-command "checkout" "master")
> +(if (not (equal? "XXX" commit-hash)) ;HACK else stay on 
> master
> +(zero? (system* git-command "checkout" commit-hash
> +  #t)

I would highly recommend Guile-Git for all this (or (guix git)) since
it’s already a hard dependency, but we can leave that for later.

> +(define (cut-url url)
> +  "Return URL without protocol prefix and git file extension."
> +  (string-replace-substring
> +   (cond
> +((string-match "http://;  url)
> + (string-replace-substring url "http://; ""))
> +((string-match "https://; url)
> + (string-replace-substring url "https://; ""))
> +((string-match "git://"   url)
> + (string-replace-substring url "git://" ""))
> +(else
> + url))
> +   ".git" ""))

Use (uri-path (string->uri url)) to get the “path” part of the URL.  And
then perhaps:

  (if (string-suffix?

Re: [PATCH 0/1] Go importer

2018-07-25 Thread Pierre-Antoine Rouby

Hi leo,

> From: "Leo Famulari" 
> On Thu, Jul 19, 2018 at 08:56:03AM +0200, Pierre-Antoine Rouby wrote:
>> I trying to import 'gitlab-runner' 
>> (https://gitlab.com/gitlab-org/gitlab-runner)
>> with tag 'v10.6.0'.
> 
> Okay, thanks. I can reproduce the error:

I have fix this issue (patch attached), let me know if that work for you.From 2f97b62beeacc58935a86d08a1635c082d078189 Mon Sep 17 00:00:00 2001
From: Rouby Pierre-Antoine 
Date: Wed, 25 Jul 2018 10:34:44 +0200
Subject: [PATCH] import: Add gopkg importer.

* guix/import/gopkg.scm: New file.
* guix/scripts/import/gopkg.scm: New file.
* guix/scripts/import.scm: Add 'gopkg'.
* Makefile.am: Add 'gopkg' importer in modules list.

Co-authored-by: Leo Famulari 
---
 Makefile.am   |  12 +-
 doc/guix.texi |   9 +-
 guix/import/gopkg.scm | 354 ++
 guix/scripts/import.scm   |   2 +-
 guix/scripts/import/gopkg.scm |  99 ++
 5 files changed, 469 insertions(+), 7 deletions(-)
 create mode 100644 guix/import/gopkg.scm
 create mode 100644 guix/scripts/import/gopkg.scm

diff --git a/Makefile.am b/Makefile.am
index 6733f4f89..7d557f0c8 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -167,7 +167,7 @@ MODULES =	\
   guix/build/rpath.scm\
   guix/build/cvs.scm\
   guix/build/svn.scm\
-  guix/build/syscalls.scm   \
+  guix/build/syscalls.scm			\
   guix/build/gremlin.scm			\
   guix/build/emacs-utils.scm			\
   guix/build/java-utils.scm			\
@@ -185,8 +185,9 @@ MODULES =	\
   guix/import/cabal.scm\
   guix/import/cran.scm\
   guix/import/hackage.scm			\
-  guix/import/elpa.scm   			\
-  guix/import/texlive.scm   			\
+  guix/import/elpa.scm\
+  guix/import/texlive.scm			\
+  guix/import/gopkg.scm\
   guix/scripts.scm\
   guix/scripts/download.scm			\
   guix/scripts/perform-download.scm		\
@@ -210,8 +211,9 @@ MODULES =	\
   guix/scripts/import/gnu.scm			\
   guix/scripts/import/nix.scm			\
   guix/scripts/import/hackage.scm		\
-  guix/scripts/import/elpa.scm  		\
-  guix/scripts/import/texlive.scm  		\
+  guix/scripts/import/elpa.scm			\
+  guix/scripts/import/texlive.scm		\
+  guix/scripts/import/gopkg.scm			\
   guix/scripts/environment.scm			\
   guix/scripts/publish.scm			\
   guix/scripts/edit.scm\
diff --git a/doc/guix.texi b/doc/guix.texi
index 84347d156..9b796554c 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -20,7 +20,7 @@ Copyright @copyright{} 2014, 2015, 2016 Alex Kost@*
 Copyright @copyright{} 2015, 2016 Mathieu Lirzin@*
 Copyright @copyright{} 2014 Pierre-Antoine Rault@*
 Copyright @copyright{} 2015 Taylan Ulrich BayÄ±rlÄ±/Kammer@*
-Copyright @copyright{} 2015, 2016, 2017 Leo Famulari@*
+Copyright @copyright{} 2015, 2016, 2017, 2018 Leo Famulari@*
 Copyright @copyright{} 2015, 2016, 2017, 2018 Ricardo Wurmus@*
 Copyright @copyright{} 2016 Ben Woodcroft@*
 Copyright @copyright{} 2016, 2017, 2018 Chris Marusich@*
@@ -6740,6 +6740,13 @@ Import metadata from the crates.io Rust package repository
 @cindex OCaml
 Import metadata from the @uref{https://opam.ocaml.org/, OPAM} package
 repository used by the OCaml community.
+
+@item gopkg
+@cindex gopkg
+@cindex Golang
+@cindex Go
+Import metadata from the @uref{https://gopkg.in/, gopkg} package
+versioning service used by some Go software.
 @end table
 
 The structure of the @command{guix import} code is modular.  It would be
diff --git a/guix/import/gopkg.scm b/guix/import/gopkg.scm
new file mode 100644
index 0..739261882
--- /dev/null
+++ b/guix/import/gopkg.scm
@@ -0,0 +1,354 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright Â© 2018 Pierre-Antoine Rouby 
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see .
+
+(define-module (guix import gopkg)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 regex)
+  #:use-module ((ice-9 rdelim) #:select (read-line))
+  #:use-module (srfi srfi-11)
+  #:use-module (texinfo string-utils) ; transform-string
+  #:use-module (guix hash)
+  #:use-module (guix base32)
+  #:use-module (guix serialization)
+  #:use-module (guix utils)
+  #:use-module (guix build utils)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:export (gopkg->guix-package))
+
+(define (vcs-file? file stat)
+  (case (stat:type stat)
+((directory)
+ (member (basename file)

Re: [PATCH 0/1] Go importer

2018-07-19 Thread Leo Famulari

On Thu, Jul 19, 2018 at 08:56:03AM +0200, Pierre-Antoine Rouby wrote:
> I trying to import 'gitlab-runner' 
> (https://gitlab.com/gitlab-org/gitlab-runner)
> with tag 'v10.6.0'.

Okay, thanks. I can reproduce the error:

--
$ ./pre-inst-env guix import gopkg https://gitlab.com/gitlab-org/gitlab-runner 
v10.6.0  
Initialized empty Git repository in /tmp/guix-directory.silSgB/.git/
   
warning: redirecting to https://gitlab.com/gitlab-org/gitlab-runner.git/
remote: Counting objects: 10694, done.
remote: Compressing objects: 100% (8261/8261), done.
remote: Total 10694 (delta 1974), reused 8461 (delta 1826)
Receiving objects: 100% (10694/10694), 29.12 MiB | 2.57 MiB/s, done.
Resolving deltas: 100% (1974/1974), done.
From https://gitlab.com/gitlab-org/gitlab-runner
 * tag   v10.6.0-> FETCH_HEAD
Note: checking out 'FETCH_HEAD'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b 

HEAD is now at a3543a2 Update v10.6.0 changelog
Backtrace:
  14 (apply-smob/1 #)
In ice-9/boot-9.scm:
705:2 13 (call-with-prompt _ _ #)
In ice-9/eval.scm:
619:8 12 (_ #(#(#)))
In guix/ui.scm:
  1579:12 11 (run-guix-command _ . _)
In guix/scripts/import.scm:
   115:11 10 (guix-import . _)
In guix/scripts/import/gopkg.scm:
85:19  9 (guix-import-gopkg . _)
In guix/utils.scm:
633:8  8 (call-with-temporary-directory #)
In guix/import/gopkg.scm:
   343:22  7 (_ "/tmp/guix-directory.silSgB")
   322:22  6 (gopkg-dep->packages+dependencies _)
   303:22  5 (parse-toml->packages+dependencies _ _ _)
   216:16  4 (create-package->packages+dependencies () () "github.c…" …)
 47:4  3 (file->hash-base32 #)
In guix/serialization.scm:
343:6  2 (dump #)
   275:34  1 (_ _)
In unknown file:
   0 (lstat #)

ERROR: In procedure lstat:
Wrong type (expecting string): #
--

It does work with some other packages. I guess the difference is whether
or not the package has a 'Gopkg.toml' file. For example, this works:

--
$ ./pre-inst-env guix import gopkg https://gopkg.in/redsync.v1  
  
Initialized empty Git repository in /tmp/guix-directory.6rwYux/.git/

remote: Counting objects: 13, done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 13 (delta 0), reused 10 (delta 0), pack-reused 0
Unpacking objects: 100% (13/13), done.
From https://gopkg.in/redsync.v1
 * branchmaster -> FETCH_HEAD
 * [new branch]  master -> origin/master
Note: checking out 'FETCH_HEAD'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b 

HEAD is now at 98dabdf Merge pull request #2 from rayzyar/master
(define-public go-gopkg-in-redsync-v1
  (let ((commit
  "98dabdf1c8574561bf976911c14ff652c47b1ddf")
(revision "0"))
(package
  (name "go-gopkg-in-redsync-v1")
  (version (git-version "0.0.0" revision commit))
  (source
(origin
  (method git-fetch)
  (uri (git-reference
 (url "https://gopkg.in/redsync.v1.git;)
 (commit commit)))
  (file-name (git-file-name name version))
  (sha256
(base32
  "1ir3xj8rz2igw5ciha2nhkvs55w0yf2k1w5xyizy8d0r4xh3x1p5"
  (build-system go-build-system)
  (arguments
'(#:import-path "gopkg.in/redsync.v1"))
  (native-inputs `())
  (home-page "https://gopkg.in/redsync.v1;)
  (synopsis "XXX")
  (description "XXX")
  (license #f
--


signature.asc
Description: PGP signature

Re: [PATCH 0/1] Go importer

2018-07-19 Thread Pierre-Antoine Rouby

> From: "Leo Famulari" 
> What package are you trying to import here?

I trying to import 'gitlab-runner' (https://gitlab.com/gitlab-org/gitlab-runner)
with tag 'v10.6.0'.

Re: [PATCH 0/1] Go importer

2018-07-18 Thread Leo Famulari

On Wed, Jul 18, 2018 at 03:11:36PM +0200, Pierre-Antoine Rouby wrote:
> I thinks the modifications doesn't works on my computer.
> 
> ---
> Backtrace:
>   13 (apply-smob/1 #)
> In ice-9/boot-9.scm:
> 705:2 12 (call-with-prompt _ _ #)
> In ice-9/eval.scm:
> 619:8 11 (_ #(#(#)))
> In guix/ui.scm:
>   1579:12 10 (run-guix-command _ . _)
> In guix/scripts/import.scm:
>115:11  9 (guix-import . _)
> In guix/scripts/import/gopkg.scm:
> 85:19  8 (guix-import-gopkg . _)
> In guix/utils.scm:
> 633:8  7 (call-with-temporary-directory #)
> In unknown file:
>6 (_ # # …)
>5 (_ # # …)
>4 (_ # # …)
> In ice-9/eval.scm:
>298:34  3 (_ #(#(#(#) # …) …))
> 619:8  2 (_ #(#(# #) …))
> In guix/serialization.scm:
>270:25  1 (write-file # …)
> In unknown file:
>0 (lstat #)
> 
> ERROR: In procedure lstat:
> Wrong type (expecting string): # 15>
> ---

What package are you trying to import here?


signature.asc
Description: PGP signature

Re: [PATCH 0/1] Go importer

2018-07-18 Thread Pierre-Antoine Rouby

Hi Leo,

- Original Message -
> From: "Leo Famulari" 

>> * guix/import/gopkg.scm: New file.
>> * guix/scripts/import/gopkg.scm: New file.
>> * guix/scripts/import.scm: Add 'gopkg'.
>> * Makefile.am: Add 'gopkg' importer in modules list.
> 
> I wonder which of the new files needs to be added to Makefile.am? My
> Autotools knowledge is not very strong...

Oups, yes guix/scripts/import/gopkg.scm need to added to Makefile.am, my bad.

> I noticed a couple issues with this code. First, the names of the
> temporary directories are predictable (they use an incrementing
> integer). Second, the temporary files are not deleted after the importer
> runs. I've attached a modified patch that addresses this by using ((guix
> utils) call-with-temporary-directory), which should address these
> problems. [0]

> What do you think of my patch? Does it still work for you?

I thinks the modifications doesn't works on my computer.

---
Backtrace:
  13 (apply-smob/1 #)
In ice-9/boot-9.scm:
705:2 12 (call-with-prompt _ _ #)
In ice-9/eval.scm:
619:8 11 (_ #(#(#)))
In guix/ui.scm:
  1579:12 10 (run-guix-command _ . _)
In guix/scripts/import.scm:
   115:11  9 (guix-import . _)
In guix/scripts/import/gopkg.scm:
85:19  8 (guix-import-gopkg . _)
In guix/utils.scm:
633:8  7 (call-with-temporary-directory #)
In unknown file:
   6 (_ # # …)
   5 (_ # # …)
   4 (_ # # …)
In ice-9/eval.scm:
   298:34  3 (_ #(#(#(#) # …) …))
619:8  2 (_ #(#(# #) …))
In guix/serialization.scm:
   270:25  1 (write-file # …)
In unknown file:
   0 (lstat #)

ERROR: In procedure lstat:
Wrong type (expecting string): #
---

--
Pierre-Antoine Rouby

Re: [PATCH 0/1] Go importer

2018-07-11 Thread Leo Famulari

On Mon, Jun 04, 2018 at 10:18:53AM +0200, Pierre-Antoine Rouby wrote:
> Attached an update of importer.

Thanks! I'm sorry my response is so late.

> From 3e9b4fa12811432fdd7a4d6330f9093dcc72d25a Mon Sep 17 00:00:00 2001
> From: Rouby Pierre-Antoine 
> Date: Thu, 26 Apr 2018 15:05:23 +0200
> Subject: [PATCH] import: Add gopkg importer.
> 
> * guix/import/gopkg.scm: New file.
> * guix/scripts/import/gopkg.scm: New file.
> * guix/scripts/import.scm: Add 'gopkg'.
> * Makefile.am: Add 'gopkg' importer in modules list.

I wonder which of the new files needs to be added to Makefile.am? My
Autotools knowledge is not very strong...

> +(define (get-new-tmp-dir)
> +  "Return new temp directory."
> +  (let ((tmp "/tmp/guix-import-gopkg"))
> +(define (new num)
> +  (let ((new-dir (string-append tmp "-" (number->string num
> +(if (file-exists? new-dir)
> +(new (+ num 1))
> +new-dir)))
> +(if (file-exists? tmp)
> +(new 0)
> +tmp)))
> +
> +(define tmp-dir (get-new-tmp-dir))

I noticed a couple issues with this code. First, the names of the
temporary directories are predictable (they use an incrementing
integer). Second, the temporary files are not deleted after the importer
runs. I've attached a modified patch that addresses this by using ((guix
utils) call-with-temporary-directory), which should address these
problems. [0]

> +(define* (git-fetch url commit directory
> +#:key (git-command "git") recursive?)
> +  "Fetch COMMIT from URL into DIRECTORY.  COMMIT must be a valid Git commit
> +identifier.  When RECURSIVE? is true, all the sub-modules of URL are fetched,
> +recursively.  Return #t on success, #f otherwise."
> +  ;; Disable TLS certificate verification.  The hash of the checkout is known
> +  ;; in advance anyway.
> +  (setenv "GIT_SSL_NO_VERIFY" "true")

My patch turns certificate verification back on. When importing, the
hash of the checkout is not known in advance.

And finally I added some brief documentation to the manual. Maybe there
could be further clean-up and code deduplication with other parts of the
Guix codebase, but I think it's better to have this importer in Guix
now.

What do you think of my patch? Does it still work for you?

[0] Actually, the temp directories will not be cleaned up due to
, but I think this will eventually be fixed.
From 50c12ff1770286fae00dac469cad3af4a9df1070 Mon Sep 17 00:00:00 2001
From: Rouby Pierre-Antoine 
Date: Thu, 26 Apr 2018 15:05:23 +0200
Subject: [PATCH] import: Add gopkg importer.

* guix/import/gopkg.scm: New file.
* guix/scripts/import/gopkg.scm: New file.
* guix/scripts/import.scm: Add 'gopkg'.
* Makefile.am: Add 'gopkg' importer in modules list.

Co-authored-by: Leo Famulari 
---
 Makefile.am   |   1 +
 doc/guix.texi |   9 +-
 guix/import/gopkg.scm | 352 ++
 guix/scripts/import.scm   |   2 +-
 guix/scripts/import/gopkg.scm |  99 ++
 5 files changed, 461 insertions(+), 2 deletions(-)
 create mode 100644 guix/import/gopkg.scm
 create mode 100644 guix/scripts/import/gopkg.scm

diff --git a/Makefile.am b/Makefile.am
index 618d1653e..a93a280b2 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -186,6 +186,7 @@ MODULES =   \
   guix/import/hackage.scm  \
   guix/import/elpa.scm \
   guix/import/texlive.scm  \
+  guix/import/gopkg.scm\
   guix/scripts.scm \
   guix/scripts/download.scm\
   guix/scripts/perform-download.scm\
diff --git a/doc/guix.texi b/doc/guix.texi
index a8e53a530..07100cd25 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -20,7 +20,7 @@ Copyright @copyright{} 2014, 2015, 2016 Alex Kost@*
 Copyright @copyright{} 2015, 2016 Mathieu Lirzin@*
 Copyright @copyright{} 2014 Pierre-Antoine Rault@*
 Copyright @copyright{} 2015 Taylan Ulrich Bayırlı/Kammer@*
-Copyright @copyright{} 2015, 2016, 2017 Leo Famulari@*
+Copyright @copyright{} 2015, 2016, 2017, 2018 Leo Famulari@*
 Copyright @copyright{} 2015, 2016, 2017, 2018 Ricardo Wurmus@*
 Copyright @copyright{} 2016 Ben Woodcroft@*
 Copyright @copyright{} 2016, 2017, 2018 Chris Marusich@*
@@ -6667,6 +6667,13 @@ Import metadata from the crates.io Rust package 
repository
 @cindex OCaml
 Import metadata from the @uref{https://opam.ocaml.org/, OPAM} package
 repository used by the OCaml community.
+
+@item gopkg
+@cindex gopkg
+@cindex Golang
+@cindex Go
+Import metadata from the @uref{https://gopkg.in/, gopkg} package
+versioning service used by some Go software.
 @end table
 
 The structure of the @command{guix import} code is modular.  It would be
diff --git a/guix/import/gopkg.scm b/guix/import/gopkg.scm
new file mode 100644
index 0..c2b72616a
--- /dev/null
+++ b/guix/import/gopkg.scm
@@ -0,0 +1,352 @@
+;;; GNU Guix --- Functional

Re: [PATCH 0/1] Go importer

2018-06-04 Thread Pierre-Antoine Rouby

Hi, Leo

> From: "Leo Famulari" 
> I've noticed several different dependency manifest formats for Go
> software "in the wild". So, this one is specific to Go software using
> the Gopkg tool?

Yes, Gopkg it's specific for Go software, it's not used by all Go 
projects, but lot of big project use it.

Attached an update of importer.

Thanks for comments.

P-AFrom 3e9b4fa12811432fdd7a4d6330f9093dcc72d25a Mon Sep 17 00:00:00 2001
From: Rouby Pierre-Antoine 
Date: Thu, 26 Apr 2018 15:05:23 +0200
Subject: [PATCH] import: Add gopkg importer.

* guix/import/gopkg.scm: New file.
* guix/scripts/import/gopkg.scm: New file.
* guix/scripts/import.scm: Add 'gopkg'.
* Makefile.am: Add 'gopkg' importer in modules list.
---
 Makefile.am   |   1 +
 guix/import/gopkg.scm | 384 ++
 guix/scripts/import.scm   |   2 +-
 guix/scripts/import/gopkg.scm |  99 +
 4 files changed, 485 insertions(+), 1 deletion(-)
 create mode 100644 guix/import/gopkg.scm
 create mode 100644 guix/scripts/import/gopkg.scm

diff --git a/Makefile.am b/Makefile.am
index 9f134c970..e103517fc 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -183,6 +183,7 @@ MODULES =	\
   guix/import/hackage.scm			\
   guix/import/elpa.scm   			\
   guix/import/texlive.scm   			\
+  guix/import/gopkg.scm   			\
   guix/scripts.scm\
   guix/scripts/download.scm			\
   guix/scripts/perform-download.scm		\
diff --git a/guix/import/gopkg.scm b/guix/import/gopkg.scm
new file mode 100644
index 0..200d9ffd3
--- /dev/null
+++ b/guix/import/gopkg.scm
@@ -0,0 +1,384 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright Â© 2018 Pierre-Antoine Rouby 
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see .
+
+(define-module (guix import gopkg)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 regex)
+  #:use-module ((ice-9 rdelim) #:select (read-line))
+  #:use-module (srfi srfi-11)
+  #:use-module (texinfo string-utils) ; transform-string
+  #:use-module (guix hash)
+  #:use-module (guix base32)
+  #:use-module (guix serialization)
+  #:use-module (guix utils)
+  #:use-module (guix build utils)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:export (gopkg->guix-package))
+
+;;
+;; Directory.
+;;
+
+(define (get-new-tmp-dir)
+  "Return new temp directory."
+  (let ((tmp "/tmp/guix-import-gopkg"))
+(define (new num)
+  (let ((new-dir (string-append tmp "-" (number->string num
+(if (file-exists? new-dir)
+(new (+ num 1))
+new-dir)))
+(if (file-exists? tmp)
+(new 0)
+tmp)))
+
+(define tmp-dir (get-new-tmp-dir))
+
+;;
+;; Git.
+;;
+
+(define (vcs-file? file stat)
+  (case (stat:type stat)
+((directory)
+ (member (basename file) '(".bzr" ".git" ".hg" ".svn" "CVS")))
+((regular)
+ ;; Git sub-modules have a '.git' file that is a regular text file.
+ (string=? (basename file) ".git"))
+(else
+ #f)))
+
+(define (file->hash-base32 file)
+  "Return hash of FILE in nix base32 sha256 format.  If FILE is a directory,
+exclude vcs files."
+  (let-values (((port get-hash) (open-sha256-port)))
+(write-file file port #:select? (negate vcs-file?))
+(force-output port)
+(bytevector->nix-base32-string (get-hash
+
+(define (git->hash url commit file)
+  "Clone git repository and return FILE hash in nix base32 sha256 format."
+  (if (not (file-exists? file))
+  (git-fetch url commit file #:recursive? #f))
+  (file->hash-base32 file))
+
+(define (git-ref->commit path tag)
+  "Return commit number coresponding to git TAG.  Return \"XXX\" if tag is not
+found."
+  (define (loop port)
+(let ((line (read-line port)))
+  (cond 
+   ((eof-object? line)  ; EOF
+(begin
+  (close-port port)
+  "XXX"))
+   ((string-match tag line) ; Match tag
+(let ((commit (car (string-split (transform-string line #\tab " ")
+ #\ 
+  commit))
+   (else; Else
+(loop port)
+
+  (let ((file (if (file-exists? (string-append path "/.git/packed-refs"))
+  (string-append path "/.git/packed-refs")
+  (string-append path "/.git/FETCH_HEAD"
+(loop (open-input-file file
+
+(define*

Re: [PATCH 0/1] Go importer

2018-05-02 Thread Leo Famulari

On Thu, Apr 26, 2018 at 06:22:38PM +0200, Rouby Pierre-Antoine wrote:
> This patch is a importer for go packages, it's use git repository and the
> 'Gopkg.toml' file (https://golang.github.io/dep/docs/Gopkg.toml.html).

Neat! I didn't have a chance to review the code yet but I'm happy to see
progress in this area. The examples look promising! I'll review it
properly soon...

I've noticed several different dependency manifest formats for Go
software "in the wild". So, this one is specific to Go software using
the Gopkg tool?

Skimming the patch, I noticed a few places that I think could use some
explanatory comments. For example, the git-checkout procedure. Since a
few days have passed, it might be a good time for you to read it over
and write some commentary on the sections that are beginning to become
confusing, as all code does with time :)

Nitpick...

> (define-public go-github-com-fsouza-go-dockerclient
>   (let ((commit "master") (revision "0"))

Does "master" refer to the branch? If so, we should avoid using it,
since its value will probably change over time. We should stick to
commits and tags. Tags can be changed but this is rarely done.

signature.asc
Description: PGP signature

[PATCH 0/1] Go importer

2018-04-26 Thread Rouby Pierre-Antoine

Hi Guix !

This patch is a importer for go packages, it's use git repository and the
'Gopkg.toml' file (https://golang.github.io/dep/docs/Gopkg.toml.html).

Example with totally random package in Go language ;)
---Start---
$ guix import gopkg \
https://gitlab.com/ContinuousEvolution/continuous-evolution.git \
0.8.1

(define-public go-github-com-fsouza-go-dockerclient
  (let ((commit "master") (revision "0"))
(package
  (name "go-github-com-fsouza-go-dockerclient")
  (version (git-version "0.0.0" revision commit))
  (source
(origin
  (method git-fetch)
  (uri (git-reference
 (url "https://github.com/fsouza/go-dockerclient.git;)
 (commit commit)))
  (file-name (git-file-name name version))
  (sha256
(base32
  "0viwysdg5i9dgdazks9wx0f93180j72x3cq9fx2synaqvqa7ldr4"
  (build-system go-build-system)
  (arguments
'(#:import-path
  "github.com/fsouza/go-dockerclient"))
  (native-inputs `())
  (home-page
"https://github.com/fsouza/go-dockerclient;)
  (synopsis "XXX")
  (description "XXX")
  (license #f

(define-public go-github-com-sirupsen-logrus
  (let ((commit "v1.0.4") (revision "0"))
(package
  (name "go-github-com-sirupsen-logrus")
  (version (git-version "0.0.0" revision commit))
  (source
(origin
  (method git-fetch)
  (uri (git-reference
 (url "https://github.com/sirupsen/logrus.git;)
 (commit commit)))
  (file-name (git-file-name name version))
  (sha256
(base32
  "0s8s8wvshmh7cb2f4fqnibqrpxahbaydyvskn3xsrl7z2a5wwajz"
  (build-system go-build-system)
  (arguments
'(#:import-path "github.com/sirupsen/logrus"))
  (native-inputs `())
  (home-page "https://github.com/sirupsen/logrus;)
  (synopsis "XXX")
  (description "XXX")
  (license #f

(define-public go-github-com-heroku-docker-registry-client
  (let ((commit "master") (revision "0"))
(package
  (name "go-github-com-heroku-docker-registry-client")
  (version (git-version "0.0.0" revision commit))
  (source
(origin
  (method git-fetch)
  (uri (git-reference
 (url "https://github.com/heroku/docker-registry-client.git;)
 (commit commit)))
  (file-name (git-file-name name version))
  (sha256
(base32
  "0db2wdgizhg71hn3f3zvan3knb8yglr0njd3mh1jn70ab7277hll"
  (build-system go-build-system)
  (arguments
'(#:import-path
  "github.com/heroku/docker-registry-client"))
  (native-inputs `())
  (home-page
"https://github.com/heroku/docker-registry-client;)
  (synopsis "XXX")
  (description "XXX")
  (license #f

(define-public go-github-com-blang-semver
  (let ((commit "3.5.1") (revision "0"))
(package
  (name "go-github-com-blang-semver")
  (version (git-version "0.0.0" revision commit))
  (source
(origin
  (method git-fetch)
  (uri (git-reference
 (url "https://github.com/blang/semver.git;)
 (commit commit)))
  (file-name (git-file-name name version))
  (sha256
(base32
  "19pli07y5592g4dyjyj0jq5rn548vc3fz0qg3624vm1j5828p1c2"
  (build-system go-build-system)
  (arguments
'(#:import-path "github.com/blang/semver"))
  (native-inputs `())
  (home-page "https://github.com/blang/semver;)
  (synopsis "XXX")
  (description "XXX")
  (license #f

(define-public go-github-com-pelletier-go-toml
  (let ((commit "1.0.1") (revision "0"))
(package
  (name "go-github-com-pelletier-go-toml")
  (version (git-version "0.0.0" revision commit))
  (source
(origin
  (method git-fetch)
  (uri (git-reference
 (url "https://github.com/pelletier/go-toml.git;)
 (commit commit)))
  (file-name (git-file-name name version))
  (sha256
(base32
  "1n8na0yg90gm0rpifmzrby5r385vvd62cdam3ls7ssy02bjvfw15"
  (build-system go-build-system)
  (arguments
'(#:import-path "github.com/pelletier/go-toml"))
  (native-inputs `())
  (home-page
"https://github.com/pelletier/go-toml;)
  (synopsis "XXX")
  (description "XXX")
  (license #f

(define-public go-gitlab-com-continuousevolution-continuous-evolution
  (let ((commit "0.8.1") (revision "0"))
(package
  (name "go-gitlab-com-continuousevolution-continuous-evolution")
  (version (git-version "0.0.0" revision commit))
  (source
(origin
  (method git-fetch)
  (uri (git-reference
 (url

Re: [PATCH 0/1] Go importer

Re: [PATCH 0/1] Go importer

Re: [PATCH 0/1] Go importer

Re: [PATCH 0/1] Go importer

Re: [PATCH 0/1] Go importer

Re: [PATCH 0/1] Go importer

Re: [PATCH 0/1] Go importer

Re: [PATCH 0/1] Go importer

Re: [PATCH 0/1] Go importer

[PATCH 0/1] Go importer

10 matches

Site Navigation

Mail list logo

Footer information