Re: [OE-core] [PATCH 1/1] layerindex.bbclass: Add ability to fetch layers from layer index

2015-01-08 Thread Chong Lu


On 01/07/2015 05:20 PM, Paul Eggleton wrote:

Hi Chong,

On Wednesday 07 January 2015 14:35:42 Chong Lu wrote:

It maybe depends on other layers when one layer is added to BBLAYERS. If
define LAYERDEPENDS variable in this layer, we will get error from bitbake.
But sometimes, we don't have defined. Add a mechanism to fetch layers from
layer index and update bblayers.conf as appropriate. The layer index stores
dependencies of all layers. Query dependency from layer index and fetch
layer repo, then add this layer to BBLAYERS.

Hmm, this is certainly interesting but not quite what I had in mind - I was
thinking rather that bitbake-layers would be extended and it would be an
explicit operation to add a layer from the layer index + all of its
dependencies. We can of course also have something semi-automatic like this
class, but the manual tool ought to come first IMHO (which the class could then
make use of).

Cheers,
Paul


Hi Paul,

Thanks for your reply. I will try to extend bitbake-layers.

Best Regards
Chong





--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/1] layerindex.bbclass: Add ability to fetch layers from layer index

2015-01-07 Thread Paul Eggleton
Hi Chong,

On Wednesday 07 January 2015 14:35:42 Chong Lu wrote:
 It maybe depends on other layers when one layer is added to BBLAYERS. If
 define LAYERDEPENDS variable in this layer, we will get error from bitbake.
 But sometimes, we don't have defined. Add a mechanism to fetch layers from
 layer index and update bblayers.conf as appropriate. The layer index stores
 dependencies of all layers. Query dependency from layer index and fetch
 layer repo, then add this layer to BBLAYERS.

Hmm, this is certainly interesting but not quite what I had in mind - I was 
thinking rather that bitbake-layers would be extended and it would be an 
explicit operation to add a layer from the layer index + all of its 
dependencies. We can of course also have something semi-automatic like this 
class, but the manual tool ought to come first IMHO (which the class could then 
make use of).

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/1] layerindex.bbclass: Add ability to fetch layers from layer index

2015-01-06 Thread Chong Lu
It maybe depends on other layers when one layer is added to BBLAYERS. If define
LAYERDEPENDS variable in this layer, we will get error from bitbake. But
sometimes, we don't have defined. Add a mechanism to fetch layers from layer
index and update bblayers.conf as appropriate. The layer index stores
dependencies of all layers. Query dependency from layer index and fetch layer
repo, then add this layer to BBLAYERS.

[YOCTO #5348]

Signed-off-by: Chong Lu chong...@windriver.com
---
 meta/classes/layerindex.bbclass | 154 
 1 file changed, 154 insertions(+)
 create mode 100644 meta/classes/layerindex.bbclass

diff --git a/meta/classes/layerindex.bbclass b/meta/classes/layerindex.bbclass
new file mode 100644
index 000..21d4d97
--- /dev/null
+++ b/meta/classes/layerindex.bbclass
@@ -0,0 +1,154 @@
+LAYER_FETCH_DIR ??= ${COREBASE}
+
+python layerindex_handler() {
+import bb.event
+if not e.data:
+return
+
+import httplib, urlparse, json
+import os
+
+def get_json_data(apiurl = 
http://layers.openembedded.org/layerindex/api/;):
+proxy_settings = os.environ.get(http_proxy, None)
+conn = None
+_parsedurl = urlparse.urlparse(apiurl)
+path = _parsedurl.path
+query = _parsedurl.query
+def parse_url(url):
+parsedurl = urlparse.urlparse(url)
+try:
+(host, port) = parsedurl.netloc.split(:)
+except ValueError:
+host = parsedurl.netloc
+port = None
+
+if port is None:
+port = 80
+else:
+port = int(port)
+return (host, port)
+
+if proxy_settings is None:
+host, port = parse_url(apiurl)
+conn = httplib.HTTPConnection(host, port)
+conn.request(GET, path + ? + query)
+else:
+host, port = parse_url(proxy_settings)
+conn = httplib.HTTPConnection(host, port)
+conn.request(GET, apiurl)
+
+r = conn.getresponse()
+if r.status != 200:
+bb.fatal(Failed to read  + path + : %d %s % (r.status, 
r.reason))
+return json.loads(r.read())
+
+def get_deps_url(name, layeritems, layerbranches, layerdependencies):
+def layeritems_info(items_name = None, items_id = None):
+litems = {}
+for li in layeritems:
+if li['name'] == items_name:
+litems['id'] = li['id']
+break
+if li['id'] == items_id:
+litems['vcs_url'] = li['vcs_url']
+litems['name'] = li['name']
+break
+return litems
+
+def layerbranches_info(items_id):
+lbranch = {}
+for lb in layerbranches:
+# branch is master.
+if lb['layer'] == items_id and lb['branch'] == 1:
+lbranch['id'] = lb['id']
+lbranch['vcs_subdir'] = lb['vcs_subdir']
+break
+if not lbranch['id']:
+bb.fatal(The id of layerBranches is not found.)
+return lbranch
+
+def layerdependencies_info(lb_id):
+ld_deps = []
+for ld in layerdependencies:
+if ld['layerbranch'] == lb_id and not ld['dependency'] in 
ld_deps:
+ld_deps.append(ld['dependency'])
+if not ld_deps:
+bb.fatal(The dependency of layerDependencies is not found.)
+return ld_deps
+
+itemsid = layeritems_info(items_name = name)['id']
+lbid = layerbranches_info(itemsid)['id']
+layerdict = {}
+for dependency in layerdependencies_info(lbid):
+layername = layeritems_info(items_id = dependency)['name']
+layerurl = layeritems_info(items_id = dependency)['vcs_url']
+layersubdir = layerbranches_info(dependency)['vcs_subdir']
+layerdict[layername] = layerurl, layersubdir
+return layerdict
+
+import sys
+import subprocess
+
+def download_layer(url, subdir):
+fetchdir = e.data.getVar(LAYER_FETCH_DIR, True)
+if not fetchdir:
+bb.fatal(Please set LAYER_FETCH_DIR variable.)
+if not os.path.exists(fetchdir):
+os.makedirs(fetchdir)
+urldir = url.split(/)[-1]
+if urldir.endswith(.git):
+urldir = urldir.split(.)[0]
+repodir = os.path.join(fetchdir, urldir)
+if subdir:
+layerdir = os.path.join(repodir, subdir)
+else:
+layerdir = repodir
+if not os.path.exists(layerdir):
+bb.warn(Fetch '%s' to '%s' % (layer, 
e.data.getVar(LAYER_FETCH_DIR, True)))
+result = subprocess.call('git clone %s %s' % (url, repodir), shell 
= True)
+if result:
+bb.fatal(Failed to download %s % url)
+