Re: [Lldb-commits] [lldb] r295897 - Changed builld-llvm.py to use .json files

[Sean Callanan via lldb-commits]

I've recommitted with a bunch of fixes:

SendingINSTALL.txt
Sendingscripts/Xcode/build-llvm.py
Adding scripts/Xcode/repo.py
Adding scripts/Xcode/repos
Adding scripts/Xcode/repos/FALLBACK
Adding scripts/Xcode/repos/svn-trunk.json
Transmitting file data .done
Committing transaction...
Committed revision 295922.

Please let me know if you see any problems.

Sean

> On Feb 22, 2017, at 4:56 PM, Sean Callanan via lldb-commits 
>  wrote:
> 
> I'll roll it back pending some fixes on my end.
> Sorry for the break.
> 
> Sean
> 
>> On Feb 22, 2017, at 4:40 PM, Tim Hammerquist > > wrote:
>> 
>> Hi Sean,
>> 
>> Looks like this is breaking builds in Green Dragon (and locally for me from 
>> master branch). Can you take a look at the failures here?
>> 
>> http://lab.llvm.org:8080/green/view/LLDB/job/lldb_build_test/26076/console 
>> 
>> 
>> Thanks!
>> -Tim
>> 
>> 
>> On Wed, Feb 22, 2017 at 2:57 PM, Sean Callanan via lldb-commits 
>> > wrote:
>> Author: spyffe
>> Date: Wed Feb 22 16:57:59 2017
>> New Revision: 295897
>> 
>> URL: http://llvm.org/viewvc/llvm-project?rev=295897=rev 
>> 
>> Log:
>> Changed builld-llvm.py to use .json files
>> 
>> LLDB has many branches in a variety of repositories.
>> The build-script.py file is subtly different for each set.
>> This is unnecessary and causes merge headaches.
>> 
>> This patch makes build-llvm.py consult a directory full
>> of .json files, each one of which matches a particular
>> branch using a regular expression.
>> 
>> Differential revision: https://reviews.llvm.org/D30275 
>> 
>> 
>> Added:
>> lldb/trunk/scripts/Xcode/repo.py
>> lldb/trunk/scripts/Xcode/repos/
>> lldb/trunk/scripts/Xcode/repos/svn-trunk.json
>> Modified:
>> lldb/trunk/scripts/Xcode/build-llvm.py
>> 
>> Modified: lldb/trunk/scripts/Xcode/build-llvm.py
>> URL: 
>> http://llvm.org/viewvc/llvm-project/lldb/trunk/scripts/Xcode/build-llvm.py?rev=295897=295896=295897=diff
>>  
>> 
>> ==
>> --- lldb/trunk/scripts/Xcode/build-llvm.py (original)
>> +++ lldb/trunk/scripts/Xcode/build-llvm.py Wed Feb 22 16:57:59 2017
>> @@ -6,6 +6,7 @@ import fnmatch
>>  import os
>>  import platform
>>  import re
>> +import repo
>>  import subprocess
>>  import sys
>> 
>> @@ -17,42 +18,38 @@ from lldbbuild import *
>>  def LLVM_HASH_INCLUDES_DIFFS():
>>  return False
>> 
>> -# The use of "x = "..."; return x" here is important because tooling looks 
>> for
>> -# it with regexps.  Only change how this works if you know what you are 
>> doing.
>> -
>> -
>> -def LLVM_REF():
>> -llvm_ref = "master"
>> -return llvm_ref
>> -
>> -
>> -def CLANG_REF():
>> -clang_ref = "master"
>> -return clang_ref
>> -
>>  # For use with Xcode-style builds
>> 
>> +def process_vcs(vcs):
>> +return {
>> +"svn": VCS.svn,
>> +"git": VCS.git
>> +}[vcs]
>> +
>> +def process_root(name):
>> +return {
>> +"llvm": llvm_source_path(),
>> +"clang": clang_source_path(),
>> +"ninja": ninja_source_path()
>> +}[name]
>> +
>> +def process_repo(r):
>> +return {
>> +'name': r["name"],
>> +'vcs': process_vcs(r["vcs"]),
>> +'root': process_root(r["name"]),
>> +'url': r["url"],
>> +'ref': r["ref"]
>> +}
>> 
>>  def XCODE_REPOSITORIES():
>> -return [
>> -{'name': "llvm",
>> - 'vcs': VCS.git,
>> - 'root': llvm_source_path(),
>> - 'url': "http://llvm.org/git/llvm.git 
>> ",
>> - 'ref': LLVM_REF()},
>> -
>> -{'name': "clang",
>> - 'vcs': VCS.git,
>> - 'root': clang_source_path(),
>> - 'url': "http://llvm.org/git/clang.git 
>> ",
>> - 'ref': CLANG_REF()},
>> -
>> -{'name': "ninja",
>> - 'vcs': VCS.git,
>> - 'root': ninja_source_path(),
>> - 'url': "https://github.com/ninja-build/ninja.git 
>> ",
>> - 'ref': "master"}
>> -]
>> +identifier = repo.identifier()
>> +if identifier == None:
>> +sys.exit("Couldn't identify the current branch")
>> +set = repo.find(identifier)
>> +if set == None:
>> +sys.exit("Couldn't find a repository set for the current branch")
>> +return [process_repo(r) for r in set]
>> 
>> 
>>  def get_c_compiler():
>> 
>> Added: lldb/trunk/scripts/Xcode/repo.py
>> URL: 
>> 

Re: [Lldb-commits] [lldb] r295897 - Changed builld-llvm.py to use .json files

[Sean Callanan via lldb-commits]

I'll roll it back pending some fixes on my end.
Sorry for the break.

Sean

> On Feb 22, 2017, at 4:40 PM, Tim Hammerquist  wrote:
> 
> Hi Sean,
> 
> Looks like this is breaking builds in Green Dragon (and locally for me from 
> master branch). Can you take a look at the failures here?
> 
> http://lab.llvm.org:8080/green/view/LLDB/job/lldb_build_test/26076/console 
> 
> 
> Thanks!
> -Tim
> 
> 
> On Wed, Feb 22, 2017 at 2:57 PM, Sean Callanan via lldb-commits 
> > wrote:
> Author: spyffe
> Date: Wed Feb 22 16:57:59 2017
> New Revision: 295897
> 
> URL: http://llvm.org/viewvc/llvm-project?rev=295897=rev 
> 
> Log:
> Changed builld-llvm.py to use .json files
> 
> LLDB has many branches in a variety of repositories.
> The build-script.py file is subtly different for each set.
> This is unnecessary and causes merge headaches.
> 
> This patch makes build-llvm.py consult a directory full
> of .json files, each one of which matches a particular
> branch using a regular expression.
> 
> Differential revision: https://reviews.llvm.org/D30275 
> 
> 
> Added:
> lldb/trunk/scripts/Xcode/repo.py
> lldb/trunk/scripts/Xcode/repos/
> lldb/trunk/scripts/Xcode/repos/svn-trunk.json
> Modified:
> lldb/trunk/scripts/Xcode/build-llvm.py
> 
> Modified: lldb/trunk/scripts/Xcode/build-llvm.py
> URL: 
> http://llvm.org/viewvc/llvm-project/lldb/trunk/scripts/Xcode/build-llvm.py?rev=295897=295896=295897=diff
>  
> 
> ==
> --- lldb/trunk/scripts/Xcode/build-llvm.py (original)
> +++ lldb/trunk/scripts/Xcode/build-llvm.py Wed Feb 22 16:57:59 2017
> @@ -6,6 +6,7 @@ import fnmatch
>  import os
>  import platform
>  import re
> +import repo
>  import subprocess
>  import sys
> 
> @@ -17,42 +18,38 @@ from lldbbuild import *
>  def LLVM_HASH_INCLUDES_DIFFS():
>  return False
> 
> -# The use of "x = "..."; return x" here is important because tooling looks 
> for
> -# it with regexps.  Only change how this works if you know what you are 
> doing.
> -
> -
> -def LLVM_REF():
> -llvm_ref = "master"
> -return llvm_ref
> -
> -
> -def CLANG_REF():
> -clang_ref = "master"
> -return clang_ref
> -
>  # For use with Xcode-style builds
> 
> +def process_vcs(vcs):
> +return {
> +"svn": VCS.svn,
> +"git": VCS.git
> +}[vcs]
> +
> +def process_root(name):
> +return {
> +"llvm": llvm_source_path(),
> +"clang": clang_source_path(),
> +"ninja": ninja_source_path()
> +}[name]
> +
> +def process_repo(r):
> +return {
> +'name': r["name"],
> +'vcs': process_vcs(r["vcs"]),
> +'root': process_root(r["name"]),
> +'url': r["url"],
> +'ref': r["ref"]
> +}
> 
>  def XCODE_REPOSITORIES():
> -return [
> -{'name': "llvm",
> - 'vcs': VCS.git,
> - 'root': llvm_source_path(),
> - 'url': "http://llvm.org/git/llvm.git 
> ",
> - 'ref': LLVM_REF()},
> -
> -{'name': "clang",
> - 'vcs': VCS.git,
> - 'root': clang_source_path(),
> - 'url': "http://llvm.org/git/clang.git 
> ",
> - 'ref': CLANG_REF()},
> -
> -{'name': "ninja",
> - 'vcs': VCS.git,
> - 'root': ninja_source_path(),
> - 'url': "https://github.com/ninja-build/ninja.git 
> ",
> - 'ref': "master"}
> -]
> +identifier = repo.identifier()
> +if identifier == None:
> +sys.exit("Couldn't identify the current branch")
> +set = repo.find(identifier)
> +if set == None:
> +sys.exit("Couldn't find a repository set for the current branch")
> +return [process_repo(r) for r in set]
> 
> 
>  def get_c_compiler():
> 
> Added: lldb/trunk/scripts/Xcode/repo.py
> URL: 
> http://llvm.org/viewvc/llvm-project/lldb/trunk/scripts/Xcode/repo.py?rev=295897=auto
>  
> 
> ==
> --- lldb/trunk/scripts/Xcode/repo.py (added)
> +++ lldb/trunk/scripts/Xcode/repo.py Wed Feb 22 16:57:59 2017
> @@ -0,0 +1,33 @@
> +import json
> +import os
> +import re
> +import subprocess
> +
> +def identifier():
> +   try:
> +   svn_output = subprocess.check_output(["svn", "info", 
> "--show-item", "url"], stderr=subprocess.STDOUT).rstrip()
> +   return svn_output
> +   except:
> +   pass
> +   try:
> +   git_remote_and_branch = 

Re: [Lldb-commits] [lldb] r295897 - Changed builld-llvm.py to use .json files

[Tim Hammerquist via lldb-commits]

Hi Sean,

Looks like this is breaking builds in Green Dragon (and locally for me from
master branch). Can you take a look at the failures here?

http://lab.llvm.org:8080/green/view/LLDB/job/lldb_build_test/26076/console

Thanks!
-Tim


On Wed, Feb 22, 2017 at 2:57 PM, Sean Callanan via lldb-commits <
lldb-commits@lists.llvm.org> wrote:

> Author: spyffe
> Date: Wed Feb 22 16:57:59 2017
> New Revision: 295897
>
> URL: http://llvm.org/viewvc/llvm-project?rev=295897=rev
> Log:
> Changed builld-llvm.py to use .json files
>
> LLDB has many branches in a variety of repositories.
> The build-script.py file is subtly different for each set.
> This is unnecessary and causes merge headaches.
>
> This patch makes build-llvm.py consult a directory full
> of .json files, each one of which matches a particular
> branch using a regular expression.
>
> Differential revision: https://reviews.llvm.org/D30275
>
> Added:
> lldb/trunk/scripts/Xcode/repo.py
> lldb/trunk/scripts/Xcode/repos/
> lldb/trunk/scripts/Xcode/repos/svn-trunk.json
> Modified:
> lldb/trunk/scripts/Xcode/build-llvm.py
>
> Modified: lldb/trunk/scripts/Xcode/build-llvm.py
> URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/scripts/
> Xcode/build-llvm.py?rev=295897=295896=295897=diff
> 
> ==
> --- lldb/trunk/scripts/Xcode/build-llvm.py (original)
> +++ lldb/trunk/scripts/Xcode/build-llvm.py Wed Feb 22 16:57:59 2017
> @@ -6,6 +6,7 @@ import fnmatch
>  import os
>  import platform
>  import re
> +import repo
>  import subprocess
>  import sys
>
> @@ -17,42 +18,38 @@ from lldbbuild import *
>  def LLVM_HASH_INCLUDES_DIFFS():
>  return False
>
> -# The use of "x = "..."; return x" here is important because tooling
> looks for
> -# it with regexps.  Only change how this works if you know what you are
> doing.
> -
> -
> -def LLVM_REF():
> -llvm_ref = "master"
> -return llvm_ref
> -
> -
> -def CLANG_REF():
> -clang_ref = "master"
> -return clang_ref
> -
>  # For use with Xcode-style builds
>
> +def process_vcs(vcs):
> +return {
> +"svn": VCS.svn,
> +"git": VCS.git
> +}[vcs]
> +
> +def process_root(name):
> +return {
> +"llvm": llvm_source_path(),
> +"clang": clang_source_path(),
> +"ninja": ninja_source_path()
> +}[name]
> +
> +def process_repo(r):
> +return {
> +'name': r["name"],
> +'vcs': process_vcs(r["vcs"]),
> +'root': process_root(r["name"]),
> +'url': r["url"],
> +'ref': r["ref"]
> +}
>
>  def XCODE_REPOSITORIES():
> -return [
> -{'name': "llvm",
> - 'vcs': VCS.git,
> - 'root': llvm_source_path(),
> - 'url': "http://llvm.org/git/llvm.git;,
> - 'ref': LLVM_REF()},
> -
> -{'name': "clang",
> - 'vcs': VCS.git,
> - 'root': clang_source_path(),
> - 'url': "http://llvm.org/git/clang.git;,
> - 'ref': CLANG_REF()},
> -
> -{'name': "ninja",
> - 'vcs': VCS.git,
> - 'root': ninja_source_path(),
> - 'url': "https://github.com/ninja-build/ninja.git;,
> - 'ref': "master"}
> -]
> +identifier = repo.identifier()
> +if identifier == None:
> +sys.exit("Couldn't identify the current branch")
> +set = repo.find(identifier)
> +if set == None:
> +sys.exit("Couldn't find a repository set for the current branch")
> +return [process_repo(r) for r in set]
>
>
>  def get_c_compiler():
>
> Added: lldb/trunk/scripts/Xcode/repo.py
> URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/scripts/
> Xcode/repo.py?rev=295897=auto
> 
> ==
> --- lldb/trunk/scripts/Xcode/repo.py (added)
> +++ lldb/trunk/scripts/Xcode/repo.py Wed Feb 22 16:57:59 2017
> @@ -0,0 +1,33 @@
> +import json
> +import os
> +import re
> +import subprocess
> +
> +def identifier():
> +   try:
> +   svn_output = subprocess.check_output(["svn", "info",
> "--show-item", "url"], stderr=subprocess.STDOUT).rstrip()
> +   return svn_output
> +   except:
> +   pass
> +   try:
> +   git_remote_and_branch = subprocess.check_output(["git",
> "rev-parse", "--abbrev-ref", "--symbolic-full-name", "@{u}"]).rstrip()
> +   git_remote = git_remote_and_branch.split("/")[0]
> +   git_branch = "/".join(git_remote_and_
> branch.split("/")[1:])
> +   git_url = subprocess.check_output(["git", "remote",
> "get-url", git_remote]).rstrip()
> +   return git_url + ":" + git_branch
> +   except:
> +   pass
> +   return None
> +
> +def find(identifier):
> +   dir = os.path.dirname(os.path.realpath(__file__))
> +   repos_dir = os.path.join(dir, "repos")
> +   json_regex = re.compile(r"^.*.json$")
> +   override_path = os.path.join(repos_dir, "OVERRIDE.json")
> +