Re: [LEDE-DEV] [PATCH v2] ar71xx: Add TP-LINK TL-WR841N v12 support.

2017-03-29 Thread Vittorio G (VittGam)
Hello Piotr,

On 27/03/2017 11:22:05 CEST, Piotr Dymacz wrote:
> Hello Vittorio,
> 
> On 26.03.2017 09:55, Vittorio Gambaletta (VittGam) wrote:
>> This router has the same hardware of TP-LINK TL-WR841N v11 (same
>> FCC ID, same TFTP image name...).
>>
>> The stock firmware web interface does not seem to accept the LEDE
>> factory image, but it can be flashed via the u-boot TFTP recovery
>> by long-pressing the reset button after power on.
> 
> So, I have just made a test (using TL-WR841N v9 "converted" to v12 EU) 
> and factory-eu image works without any problems (Firefox 52).

Ok, perfect, then!

> I'm going to pick up your patch with small change in commit message and 
> subject (please, don't add dot at the end of the subject next time).

Thanks for your suggestion. I knew the exact contrary and in fact I'm
adding dot at end of subject every time... Do you know if there is some
place (that I might have very well missed if it exists...) where this
kind of style rules are written?

Cheers,
Vittorio

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] [PATCH v2] ar8327: Add workarounds for AR8337 switch.

2017-03-29 Thread Vittorio G (VittGam)
Hello,

On 29/03/2017 10:44:36 CEST, John Crispin wrote:
> On 26/03/17 10:08, Vittorio Gambaletta (VittGam) wrote:
>> -.phy_fixup = ar8327_phy_fixup,
>> +//.phy_fixup = ar8337_phy_fixup, // not needed at the moment
>>  .set_mirror_regs = ar8327_set_mirror_regs,
>>  .get_arl_entry = ar8327_get_arl_entry,
>>  .sw_hw_apply = ar8327_sw_hw_apply
> Hi,
> 
> why is this not needed ?

Because QCA in their QSDK only execute the AR8327 fixup function
for the AR8327 [0]. Since the fixup function does black magic with
hidden or reserved register bits (some are not even appearing in
the datasheet...), I think that it's better to leave it out
like they do...

Cheers,
Vittorio

[0] 
https://source.codeaurora.org/quic/qsdk/oss/system/openwrt/tree/target/linux/generic/files/drivers/net/phy/ar8216.c?h=aa%2Fbanana#n1324

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] Release 17.01.0 binary packages have changed and SDK inconsistency

2017-03-29 Thread Matthias Schiffer
On 03/29/2017 08:57 PM, Jo-Philipp Wich wrote:
> Hi Pau,
> 
> I will not reiterate the arguments made elsewhere in this thread but
> merely describe why the version tagged 17.01.0 SDK uses fixed feed sources.
> 
> The main objective here was to make the build reproducible in a way that
> you can use the 17.01.0 SDK to build artifacts resembling the released
> 17.01.0 images. Note that I specifically mean images here and not the
> package feed universe. No effort was made to freeze binary package
> repositories on the server.
> 
> Only a tiny subset of all binary packages is frozen and tied to a
> specific release, this subset consists of a) source packages marked as
> "nonshared" in the source trees and b) of kmod packages.
> 
> All other packages are rebuilt incrementally from the latest HEAD of the
> "lede-17.01" branch in each feed repository.
> 
> The reasoning behind rebuilding non-"nonshared" and non-kmod packages is:
> 
>  - by definition any commit made to a release maintenance branch is
>either a bug fix or important update (e.g. to address security issue)
> 
>  - it is expected that commits made to release maintenance branches
>are backwards, abi and config compatible (e.g. no libfoo 1.x -> 2.x
>bump)
> 

Note that we aren't exactly good at keeping ABI compatiblity, especially in
our own core libraries that don't even have versioned SONAMEs. We broke the
libubus ABI in lede-17.01 literally 5 days after the 17.01.0 release:

https://git.lede-project.org/?p=source.git;a=commit;h=fdc22b616cdedea6d68e36ae464551607c51d2a7

"libubus: do not modify uloop_cancelled" changes the layout of struct
ubus_context, which is directly accessed via static functions in libubus.h
and thus ABI. I'm not sure if the situation has been resolved, as I was
busy at the time this happened. As I understand it, the result is that some
package updates are more or less impossible in lede-17.01 now, as both
shared and non-shared packages depend on libubus? Also, I'm not sure if
dependent packages are rebuilt correctly, as we've never set something like
ABI_VERSION for libubus.

My explanation could be wrong, as I didn't find time to check myself that
these issues exist in the way I described, but I heard that people were
complaining in IRC about crashing services after opkg updates on LEDE 17.01.1.

Matthias



signature.asc
Description: OpenPGP digital signature
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] Release 17.01.0 binary packages have changed and SDK inconsistency

2017-03-29 Thread Pau
On 29/03/17 20:57, Jo-Philipp Wich wrote:
> Hi Pau,
> 
> I will not reiterate the arguments made elsewhere in this thread but
> merely describe why the version tagged 17.01.0 SDK uses fixed feed sources.
> 
> The main objective here was to make the build reproducible in a way that
> you can use the 17.01.0 SDK to build artifacts resembling the released
> 17.01.0 images. Note that I specifically mean images here and not the
> package feed universe. No effort was made to freeze binary package
> repositories on the server.
> 
> Only a tiny subset of all binary packages is frozen and tied to a
> specific release, this subset consists of a) source packages marked as
> "nonshared" in the source trees and b) of kmod packages.
> 
> All other packages are rebuilt incrementally from the latest HEAD of the
> "lede-17.01" branch in each feed repository.
> 
> The reasoning behind rebuilding non-"nonshared" and non-kmod packages is:
> 
>  - by definition any commit made to a release maintenance branch is
>either a bug fix or important update (e.g. to address security issue)
> 
>  - it is expected that commits made to release maintenance branches
>are backwards, abi and config compatible (e.g. no libfoo 1.x -> 2.x
>bump)
> 
>  - it rarely (if ever) makes sense to stick to a known old version of a
>binary package, at least when assuming that a newer version fulfills
>the compatibility requirements raised above
> 
>  - the vast majority of such updated packages is not bundled with the
>image root filesystem so an "opkg install" of a 3rd party package
>will fetch the latest release maintenance branch version of it, not
>some outdated version that happened to be current at the release
> 
> This of course leads to a number of considerations and constraints one
> has to be aware of:
> 
>  - there needs to be a certain "branch discipline" which means that
>commits introducing incompatible ABI changes or other disruptive
>updates are to be avoided
> 
>  - critical updates (such as CVE fixes) within the release maintenance
>branches made to packages which are part of the released images need
>additional care; while updated/fixed binary packages will be
>available for users to install via "opkg upgrade", there still is a
>need to roll a new point release with updated images containing the
>newer versions of such builtin binary packages from the get-go since
>squashfs based root file systems mostly make in-place updates
>impossible due to inherent space constraints.
> 
>  - Without local modifications to the feed manifest, it is indeed not
>possible to use the released SDK as-is to build 100% identical
>packages for a given release at a later point in time
> 
> 
> The decision to release point version SDKs with fixed feed URL pointers
> is not set in stone and can be revised; it was merely done to address
> the expressed need for being able to exactly rebuild a release image as
> found on the download server. It also serves as a reference to what
> components in which version were used to form a distribution release at
> a specific point in time.
> 
> The buildfarm incrementally producing updated binary repositories for
> the current release is not actually utilizing the v17.01.0 SDK but the
> one found at
> https://downloads.lede-project.org/releases/17.01-SNAPSHOT/targets/*/*/lede-sdk-17.01-SNAPSHOT-*.tar.xz
> which always represents the state of the latest release maintenance work.
> 
> 
> To summarize; the 17.01.0 SDK is good for:
> 
>  - building out-of-tree kmods that fit to the kernel found in released
>images
> 
>  - identically rebuilding release images with the same set or a subset
>of the images included in official images
> 
> It is (out of the box) not suitable to:
> 
>  - build the latest packages available for a release as found on the
>binary download server
> 
> 
> In the future it might be possible to have both a copy of the original
> repository state at the time of the release and a kind of "updates"
> repository which is overlaid over the release repository and only
> containing binary packages which have been updated since the release.
> 
> We are, however, not there yet as both buildroot and opkg infrastructure
> is lacking to support such an overlay repository scenario which is why I
> opted for preferring current binary packages over frozen repositories.
> 
> What we can do on a short notice (e.g. for 17.01.1) is retaining a copy
> of the original initial repository but build images to use another
> repository which is continuously getting updated.
> 

Hi Jow.

Thanks for taking the time to clarify. I think I just understood wrong
the concept of release for LEDE. All your points sound reasonable to me.
The only minor thing to comment is that instead of using static commit
hashes into feeds.conf.default IMO would be better to use TAGS to easier
identify where the release is pointing when looking at the feed Git history.


Re: [LEDE-DEV] dnsmasq sometimes fails to start on 17.01.0

2017-03-29 Thread Eric Luehrsen
On 03/29/2017 11:57 AM, Yousong Zhou wrote:
> On 24 March 2017 at 03:17, Gui Iribarren via Lede-dev
>> I continued yeterday further debugging this, i played with this
>> particular number in /e/i/dnsmasq
>> (line 815 in http://pastebin.com/FV09f2jG)
>>
>> procd_add_raw_trigger "interface.*" 2000 /etc/init.d/dnsmasq reload
>
> I cannot yet reproduce the issue, but here are some findings after
> reading code of procd
>
> The 2000ms and 6ms above as argument of procd_add_raw_trigger is a
> delay before running the specified action when the event occurs.  It's
> NOT a timeout to wait for the action to complete before killing it.

I have noticed other quirks with this trigger. It seems to be busy or 
noisy. If you replace "reload" with "restart" for example or if you 
inspect time stamps of /var/etc/dnsmasq.conf, then you will see that the 
trigger issues every few minutes. The hotplug/iface script is much more 
targeted at IFUP with new and unique configuration. The busyness and the 
short fuse (2 seconds) may be falling over itself.

I ran into this setting up the net/unbound package to work like dnsmasq. 
I made the mistake of "restart" instead of "reload". I just though I'd 
use hotplug I put in last fall, but with procd. The delay timer is nice. 
When a power outage cause the router and modem to go through a few 
handshakes on reboot, Unbound would be crashed out in procd. "reload" 
fixed that, but then no hard "restart" trigger is configured. It would 
be nice if (1) procd had a README with a solid paragraph on each 
procd.sh user-end function. (2) there were ways to be better refine the 
interface triggers to only a true interface reconfiguration.

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] [PATCH 2/3][RFC] base-files: add option to rc.common to disable default reload behavior

2017-03-29 Thread Eric Luehrsen
On 03/29/2017 03:17 AM, Alexandru Ardelean wrote:
> On Tue, Mar 28, 2017 at 11:36 PM, Hans Dedecker  wrote:
>> On Mon, Mar 20, 2017 at 3:08 PM, Alexandru Ardelean
>>  wrote:
>>> From: Alexandru Ardelean 
>>>
>>> Traditionally if a reload script fails, it will fallback to restart.
>>>
>>> That seems to be the default behavior in case no reload
>>> handler has been specified, and `reload` will return 1.
>>>
>>> That also has the disadvantage of masking reload errors
>>> from `/etc/init.d/ reload`.
>>>
>>> Still, it's a pretty old behavior, and in most cases
>>> it should be fine.
>>>
>>> For other cases, the `RESTART_ON_RELOAD_ERR=0` can
>>> be specified to override this.
>>>
>>> Not sure about the correctness of this approach,
>>> so this patch is RFC.
>> Discussed the restart-on-reload-fail behaviour with Felix and
>> Matthias; general consensus is to remove the restart-on-reload-fail
>> behaviour in rc.common. Packages do not seem to depend on this
>> restart-on-reload fail behaviour and it's more logical to handle
>> reload failure in the packages itself if extra logic is required.
>
> Thanks for the consideration.
>
> I'll submit another patch series just for this restart-on-reload fail.
>
> I feel this mechanism was maybe intended to act as default reload
> implementation.
> In the sense, that a default reload hook is implemented returning
> non-zero, and if no specific reload is implemented, this logic [
> restart on reload fail ] would work.
>
> Will think about it a bit.
>
> Thanks
> Alex

I would have concerns with the lack of documentation on how to control 
the outcome of procd triggers. Restart on reload behavior may fix things 
without people realizing it. Cross platform contributers may not fully 
appreciate the quirks of procd vs systemd vs ... This is to be expected 
without a user guide.

For example of a gotcha, triggers on interfaces may be noisy. Look at 
how often dnsmasq.conf is rewritten on "interface.*" raw trigger. It 
isn't restarted because the command line doesn't change. The delay timer 
is a nice robustness action over hotplug scripts. Hotplug scripts are 
easier to target. Reload needs to be used with this procd trigger but 
not restart.

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] [PATCH packages] libwebsockets: set PROVIDES for all variants

2017-03-29 Thread Rafał Miłecki
From: Rafał Miłecki 

This allows software using libwebsockets to simply specify a
libwebsockets dependency. Otherwise a specific variant (or set of them)
would need to be specified.

Signed-off-by: Rafał Miłecki 
---
 libs/libwebsockets/Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libs/libwebsockets/Makefile b/libs/libwebsockets/Makefile
index cfc441b5..d8d307a3 100644
--- a/libs/libwebsockets/Makefile
+++ b/libs/libwebsockets/Makefile
@@ -46,6 +46,7 @@ define Package/libwebsockets/Default
DEPENDS:=+zlib
URL:=https://libwebsockets.org
MAINTAINER:=Karl Palsson 
+   PROVIDES:=libwebsockets
 endef
 
 define Package/libwebsockets-openssl
-- 
2.11.0


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] ar7 (TI AR7): update to kernel 4.4, please test

2017-03-29 Thread James Le Cuirot
On 09/18/2016 04:18 PM, Hauke Mehrtens wrote:
> I updated the ar7 target to kernel 4.4 in this branch:
> https://git.lede-project.org/?p=lede/hauke/staging.git;a=shortlog;h=refs/heads/kernel-4.4
> 
> If you have any device supported by this target please test it and
> report back if the ar7 target is still working with kernel 4.4 like
> with the old kernel or if you see new problems. Please report back if it
> worked and if it does not work. I only compile tested this because I do
> not have this hardware.
> 
> Having the target at kennel 4.4 is a requirement to get it into the next
> release.

I have tested this on a Netgear DG834Gv2. Unfortunately I never got a
ping back from it and I don't have serial access. An otherwise
identical configuration against 3.18.43 works.

I did subsequently try to build 4.1 but the config kept screwing up and
I don't have time to figure it out right now.

Regards,
-- 
James Le Cuirot (chewi)
Gentoo Linux Developer


pgpvSmUyjv4T_.pgp
Description: OpenPGP digital signature
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] Release 17.01.0 binary packages have changed and SDK inconsistency

2017-03-29 Thread Alberto Bursi

On 03/29/2017 05:39 PM, James Feeney wrote:
> Hi Alberto
>
>> Also the kernel is handled like a package for the build system, but
>> since most devices expect it outside of root filesystem in various ways,
>> it's added to the firmware image the way the device's bootloader expects it.
> I haven't seen a preferred term for the class of this combined.  LEDE does 
> name each of them individually, based upon the target
> architecture.  It would be nice to have some generally recognized single name
> for this class, "the kernel-fs packages" or some such.
>
> Of course it's a little complicated, since most traditional Linux distros 
> target
> only one or two hardware architectures, and LEDE targets many many different
> target architectures.  But then, that's the whole point of LEDE.
>

I've seen it called "image", as it is basically a disk image ready to be 
flashed on raw flash (for most devices), or simply "firmware" as the end 
result is a complete and bootable firmware if installed correctly.

> So, if I understand, there always exists a set of hashes to verify that, at 
> any
> time in the future, a package can be built which is identical to any package
> built in the past.

The hash is to verify integrity of source package downloaded from 
upstream, afaik there is a hash after compilation, but that is generated 
automatically and listed in the package lists read by opkg only. It's 
not stored permanently.

You can revert to an older version of LEDE git and compile the same 
package with its build system, assuming you still have access to the 
same source package or git repo or whatever that was used before.

> Bastian also mentions "checkout LEDE and all feeds at a specific timestamp" as
> an approach.  This suggest two general tracking mechanisms: 1) tracking by 
> hash,
> and 2) tracking by time-stamp.  Is one of these mechanisms, or are both of 
> these
> mechanisms, used universally in the LEDE package management system?

I think he meant that they are doing like that to make their own 
downstream LEDE-based firmwares. They tag a timestamp in LEDE snapshot 
or whatever, and stick to it until they decide to tag the next.

In any case LEDE is either downloading a source archive from a specific 
release version (and using the hash) or fetching source from a specific 
commit in a versioning system like git or subversion or whatever.

>
> Hmm - it is not entirely clear to me that, if the patch set changes in the
> future, the version number of the package will be distinctly incremented, as
> would be pretty standard for any other Linux distro.  Does each package have a
> minor version number, in the source and binary repositories, to allow
> distinguishing patch set changes, used in each build of every package?
>

There is a secondary version number, yes.

Packages have PKG_VERSION that shows the upstream version and 
PKG_RELEASE that is used to show changes on the LEDE side. If you see a 
package that lists version as 123-1, it's major version is 123 and its 
minor version is 1.

Packages with a timestamp use that instead of major version, and they 
still have a minor version.

for example  2016-09-21-42ad5367-1 where the last "1" is the minor version.

You can see all versions used in the first page of the table of packages 
in the wiki https://lede-project.org/packages/start

Or by browsing a Package.manifest in the package download folders 
https://downloads.lede-project.org/releases/17.01.0/packages/aarch64_armv8-a/base/Packages.manifest

> Perhaps, assuming that these distinct version numbers exist, and that these
> historic time-stamp or hash verified sources are archived somewhere and 
> available.

The hashes and time stamps of source archives or commits are written in 
the package's makefile in the git repository (either main repo or 
community package feeds), you can see full history of changes to that 
file with git.

There is a cache server that stores source archives because it's more 
convenient than having the build bots spamming upstream download servers 
and it is a convenient fallback if the upstream download disappears, it 
should keep stuff for a long while (as long as the release is supported 
for sure, as it is used by build bots).

it is here http://sources.lede-project.org/

Also OpenWRT's source cache server is used (as a fallback maybe) 
https://downloads.openwrt.org/sources/

> It sounds like the LEDE package management system has changed and improved 
> over
> the past months.  It would be great to see your description of the LEDE 
> package
> management system and build system on the wiki!

Ok, will move this to wiki in a few days.

-Alberto
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] Release 17.01.0 binary packages have changed and SDK inconsistency

2017-03-29 Thread Jo-Philipp Wich
Hi Pau,

I will not reiterate the arguments made elsewhere in this thread but
merely describe why the version tagged 17.01.0 SDK uses fixed feed sources.

The main objective here was to make the build reproducible in a way that
you can use the 17.01.0 SDK to build artifacts resembling the released
17.01.0 images. Note that I specifically mean images here and not the
package feed universe. No effort was made to freeze binary package
repositories on the server.

Only a tiny subset of all binary packages is frozen and tied to a
specific release, this subset consists of a) source packages marked as
"nonshared" in the source trees and b) of kmod packages.

All other packages are rebuilt incrementally from the latest HEAD of the
"lede-17.01" branch in each feed repository.

The reasoning behind rebuilding non-"nonshared" and non-kmod packages is:

 - by definition any commit made to a release maintenance branch is
   either a bug fix or important update (e.g. to address security issue)

 - it is expected that commits made to release maintenance branches
   are backwards, abi and config compatible (e.g. no libfoo 1.x -> 2.x
   bump)

 - it rarely (if ever) makes sense to stick to a known old version of a
   binary package, at least when assuming that a newer version fulfills
   the compatibility requirements raised above

 - the vast majority of such updated packages is not bundled with the
   image root filesystem so an "opkg install" of a 3rd party package
   will fetch the latest release maintenance branch version of it, not
   some outdated version that happened to be current at the release

This of course leads to a number of considerations and constraints one
has to be aware of:

 - there needs to be a certain "branch discipline" which means that
   commits introducing incompatible ABI changes or other disruptive
   updates are to be avoided

 - critical updates (such as CVE fixes) within the release maintenance
   branches made to packages which are part of the released images need
   additional care; while updated/fixed binary packages will be
   available for users to install via "opkg upgrade", there still is a
   need to roll a new point release with updated images containing the
   newer versions of such builtin binary packages from the get-go since
   squashfs based root file systems mostly make in-place updates
   impossible due to inherent space constraints.

 - Without local modifications to the feed manifest, it is indeed not
   possible to use the released SDK as-is to build 100% identical
   packages for a given release at a later point in time


The decision to release point version SDKs with fixed feed URL pointers
is not set in stone and can be revised; it was merely done to address
the expressed need for being able to exactly rebuild a release image as
found on the download server. It also serves as a reference to what
components in which version were used to form a distribution release at
a specific point in time.

The buildfarm incrementally producing updated binary repositories for
the current release is not actually utilizing the v17.01.0 SDK but the
one found at
https://downloads.lede-project.org/releases/17.01-SNAPSHOT/targets/*/*/lede-sdk-17.01-SNAPSHOT-*.tar.xz
which always represents the state of the latest release maintenance work.


To summarize; the 17.01.0 SDK is good for:

 - building out-of-tree kmods that fit to the kernel found in released
   images

 - identically rebuilding release images with the same set or a subset
   of the images included in official images

It is (out of the box) not suitable to:

 - build the latest packages available for a release as found on the
   binary download server


In the future it might be possible to have both a copy of the original
repository state at the time of the release and a kind of "updates"
repository which is overlaid over the release repository and only
containing binary packages which have been updated since the release.

We are, however, not there yet as both buildroot and opkg infrastructure
is lacking to support such an overlay repository scenario which is why I
opted for preferring current binary packages over frozen repositories.

What we can do on a short notice (e.g. for 17.01.1) is retaining a copy
of the original initial repository but build images to use another
repository which is continuously getting updated.


Now on to your specific questions below:

> I am using the SDK and IB from LEDE 17.01.0 release (mips_24kc).
> I've been using it successfully the last days until now. I did not
> change anything but I get the error:
> 
> * opkg_install_pkg: Package luci-lib-nixio sha256sum mismatch.
> Either the opkg or the package index are corrupt. Try 'opkg update'.
> 
> Then I went to downloads.lede-project.org [1] and I see that there
> are new compiled packages with date "Tue Mar 28 18:35:15 2017". 
> Luci-lib-nixio is one of them [2] (which now is published on the 
> repository with a new version).
> 
> In the 

[LEDE-DEV] [PATCH packages] libwebsockets: bump to 2.2.1

2017-03-29 Thread Rafał Miłecki
From: Rafał Miłecki 

For changes since 2.0.x see
https://github.com/warmcat/libwebsockets/blob/v2.2-stable/changelog

Signed-off-by: Rafał Miłecki 
---
 libs/libwebsockets/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libs/libwebsockets/Makefile b/libs/libwebsockets/Makefile
index 27455a34..cfc441b5 100644
--- a/libs/libwebsockets/Makefile
+++ b/libs/libwebsockets/Makefile
@@ -8,13 +8,13 @@
 include $(TOPDIR)/rules.mk
 
 PKG_NAME:=libwebsockets
-PKG_VERSION:=2.0.3
+PKG_VERSION:=2.2.1
 PKG_RELEASE:=1
 
 PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
 PKG_SOURCE_SUBDIR:=$(PKG_NAME)-$(PKG_VERSION)
 
PKG_SOURCE_URL:=https://codeload.github.com/warmcat/libwebsockets/tar.gz/v$(PKG_VERSION)?
-PKG_MD5SUM:=a025156d606d90579e65d53ccd062a94
+PKG_MD5SUM:=1f641cde2ab3687db3d553f68fe0f620
 
 
 PKG_SOURCE_VERSION:=v$(PKG_VERSION)
-- 
2.11.0


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] Release 17.01.0 binary packages have changed and SDK inconsistency

2017-03-29 Thread Pau
On 29/03/17 09:16, Alberto Bursi wrote:
> 
> 
> On 03/29/2017 02:45 AM, Daniel Golle wrote:
>> I get that problem, and it can be solved easily by removing the git
>> commit hash from feeds.conf in the SDK and rather use the current
>> snapshot instead.
>> I agree that the lack of a lede-17.01 release branch on the package
>> feeds may cause weird and problematic situations, especially in the
>> future once LEDE HEAD has diverged more from 17.01.x released versions.
>>
> 
> Wat? Package feeds all have a branch for LEDE 17.01.
> 
> https://github.com/openwrt-routing/packages/tree/lede-17.01
> https://github.com/openwrt/telephony/tree/lede-17.01
> https://github.com/openwrt/luci/tree/lede-17.01
> https://github.com/openwrt/packages/tree/lede-17.01
> 
> Why is the SDK not using that and is targeting commits instead?
> 

+1 to that question.

If the idea is to have a "moving release" instead of a "snapshot
release" why the SDK feeds are pointing to static commits and TAGS and
not to branches?

> -Alberto
> 
> ___
> Lede-dev mailing list
> Lede-dev@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/lede-dev
> 

-- 
./p4u



signature.asc
Description: OpenPGP digital signature
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] dnsmasq sometimes fails to start on 17.01.0

2017-03-29 Thread Yousong Zhou
On 24 March 2017 at 03:17, Gui Iribarren via Lede-dev
 wrote:
> The sender domain has a DMARC Reject/Quarantine policy which disallows
> sending mailing list messages using the original "From" header.
>
> To mitigate this problem, the original message has been wrapped
> automatically by the mailing list software.
>
>
> -- Forwarded message --
> From: Gui Iribarren 
> To: Yousong Zhou 
> Cc: lede-dev@lists.infradead.org
> Bcc:
> Date: Thu, 23 Mar 2017 16:10:06 -0300
> Subject: Re: [LEDE-DEV] dnsmasq sometimes fails to start on 17.01.0
> Thanks both for the replies!
>
> I continued yeterday further debugging this, i played with this
> particular number in /e/i/dnsmasq
> (line 815 in http://pastebin.com/FV09f2jG)
>
> procd_add_raw_trigger "interface.*" 2000 /etc/init.d/dnsmasq reload
>
> this number, as the following link suggests
> http://wiki.prplfoundation.org/wiki/Procd_reference#procd_add_raw_trigger.28event.2C_timeout.2C_.5Bscript.5D.29
> is the number of milliseconds that procd will wait after the trigger (in
> this case, anything related to an "interface" AFAIU) before executing
> "dnsmasq reload"
>
> i put 6 (60 seconds) and now the reload only happens once (after ~60
> seconds from the first "/e/i/dnsmasq boot") and dnsmasq starts without
> problem
>
> so it looks to me like a race condition, where two "interface.*" events
> are happening one after the other, triggering two consecutive reloads,
> the first reload doesn't finish its work before the second reload comes,
> and the second reload kills the first reload, and suicides itself for
> some reason.
>
> setting a long raw_trigger timeout works around the problem because the
> "interface.*" events happen all inside the 60 second window, and procd
> runs "/e/i/dnsmasq reload" only once
>

I cannot yet reproduce the issue, but here are some findings after
reading code of procd

The 2000ms and 6ms above as argument of procd_add_raw_trigger is a
delay before running the specified action when the event occurs.  It's
NOT a timeout to wait for the action to complete before killing it.

When the event first occurs, procd will schedule the action to be run
after a "delay".  If the same kind of event happens again

 - during that delay (before the action has been started), the delay
will be reset to its initial value (wait for that amount of time
again).
 - after the delay and the action is still being carried out, the
action will be marked to re-run after the current one completes

I am wondering if FS#660 was caused by the same reason

  https://bugs.lede-project.org/index.php?do=details_id=660

Regards,
yousong

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] Release 17.01.0 binary packages have changed and SDK inconsistency

2017-03-29 Thread James Feeney
Hi Alberto

> Also the kernel is handled like a package for the build system, but
> since most devices expect it outside of root filesystem in various ways,
> it's added to the firmware image the way the device's bootloader expects it.

I haven't seen a preferred term for the class of this combined kernel and base
filesystem.  LEDE does name each of them individually, based upon the target
architecture.  It would be nice to have some generally recognized single name
for this class, "the kernel-fs packages" or some such.

Of course it's a little complicated, since most traditional Linux distros target
only one or two hardware architectures, and LEDE targets many many different
target architectures.  But then, that's the whole point of LEDE.

> You will notice that it states the official download link for the
> sources, a SHA256 hash and a specific version for the package name.
> On some other packages it will state git commit or other way to identify
> the same source to pull down and build.
> ...

So, if I understand, there always exists a set of hashes to verify that, at any
time in the future, a package can be built which is identical to any package
built in the past.

Bastian also mentions "checkout LEDE and all feeds at a specific timestamp" as
an approach.  This suggest two general tracking mechanisms: 1) tracking by hash,
and 2) tracking by time-stamp.  Is one of these mechanisms, or are both of these
mechanisms, used universally in the LEDE package management system?

> In /patches folder of each package's folder you will find any patches
> that will be applied to the source after downloading it, before
> compiling, again the example package:
> ...

Hmm - it is not entirely clear to me that, if the patch set changes in the
future, the version number of the package will be distinctly incremented, as
would be pretty standard for any other Linux distro.  Does each package have a
minor version number, in the source and binary repositories, to allow
distinguishing patch set changes, used in each build of every package?

> Is this enough for having repeatable builds?

Perhaps, assuming that these distinct version numbers exist, and that these
historic time-stamp or hash verified sources are archived somewhere and 
available.

> Yeah, I agree that adding this to the wiki would be good.
> If you confirm that this answers your questions (more or less), I'm 
> adding it to the wiki.

It sounds like the LEDE package management system has changed and improved over
the past months.  It would be great to see your description of the LEDE package
management system and build system on the wiki!

Still, as Daniel has suggested, I may not quite understand Pau's underlying
concern, which started this thread.  Looking back, Pau says:
"The SDK published then is not consistent with the current package binary
repositories", and "... if there are fixes a new revision must be published
instead of modifying the current one (i.e 17.01.1).

Are these concerns simply misunderstandings of the changing LEDE package
management system?  Or, are these concerns addressing actual shortcomings with
version control in the LEDE package management system?  I don't think I've heard
a definite answer.  I was not trying to hijack Pau's thread, but Daniel has
questioned whether my issues and Pau's issues are the same, or not.

Thanks
James

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] OpenSSL: question about package Makefile

2017-03-29 Thread Nino Benedetto
Hi all,
I was looking at the openssl package Makefile and I have a question about that.
The Build/Compile target is formatted in the following way:

define Build/Compile
+$(MAKE) $(PKG_JOBS) -C $(PKG_BUILD_DIR) \
CC="$(TARGET_CC)" \
ASFLAGS="$(TARGET_ASFLAGS) -I$(PKG_BUILD_DIR)/crypto -c" \
AR="$(TARGET_CROSS)ar r" \
RANLIB="$(TARGET_CROSS)ranlib" \
OPENWRT_OPTIMIZATION_FLAGS="$(TARGET_CFLAGS)" \
$(OPENSSL_MAKEFLAGS) \
all
+$(MAKE) $(PKG_JOBS) -C $(PKG_BUILD_DIR) \
CC="$(TARGET_CC)" \
ASFLAGS="$(TARGET_ASFLAGS) -I$(PKG_BUILD_DIR)/crypto -c" \
AR="$(TARGET_CROSS)ar r" \
RANLIB="$(TARGET_CROSS)ranlib" \
OPENWRT_OPTIMIZATION_FLAGS="$(TARGET_CFLAGS)" \
$(OPENSSL_MAKEFLAGS) \
build-shared
# Work around openssl build bug to link libssl.so with libcrypto.so.
-rm $(PKG_BUILD_DIR)/libssl.so.*.*.*
+$(MAKE) $(PKG_JOBS) -C $(PKG_BUILD_DIR) \
CC="$(TARGET_CC)" \
OPENWRT_OPTIMIZATION_FLAGS="$(TARGET_CFLAGS)" \
$(OPENSSL_MAKEFLAGS) \
do_linux-shared
$(MAKE) -C $(PKG_BUILD_DIR) \
CC="$(TARGET_CC)" \
INSTALL_PREFIX="$(PKG_INSTALL_DIR)" \
$(OPENSSL_MAKEFLAGS) \
install
endef

I'm not totally understanding why we need to perform make build-shared
and make do_linux-shared for openssl source code >= than 1.0.1.
If I remove the compile the package without the workarounds, the
library seems to work fine and the shared objects seem to be correct.

define Build/Compile
+$(MAKE) $(PKG_JOBS) -C $(PKG_BUILD_DIR) \
CC="$(TARGET_CC)" \
ASFLAGS="$(TARGET_ASFLAGS) -I$(PKG_BUILD_DIR)/crypto -c" \
AR="$(TARGET_CROSS)ar r" \
RANLIB="$(TARGET_CROSS)ranlib" \
OPENWRT_OPTIMIZATION_FLAGS="$(TARGET_CFLAGS)" \
$(OPENSSL_MAKEFLAGS) \
all
$(MAKE) -C $(PKG_BUILD_DIR) \
CC="$(TARGET_CC)" \
INSTALL_PREFIX="$(PKG_INSTALL_DIR)" \
$(OPENSSL_MAKEFLAGS) \
install
endef

I noticed that this section of the package Makefile didn't change
since 2012 (or maybe before) so I suppose was related to an old
openssl source code.
Is it safe to compile the package using the Compile section I proposed?
Thanks!

Carmine Benedetto

-- 
Website: www.carminebenedetto.net
Skype: neoben86

Keep an open mind, but not so open that your brain falls out. [Richard Feynmann]

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] dnsmasq sometimes fails to start on 17.01.0

2017-03-29 Thread Gui Iribarren via Lede-dev
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.

To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.--- Begin Message ---
On 23/03/17 16:10, Gui Iribarren wrote:
> Thanks both for the replies!
> 
> I continued yeterday further debugging this, i played with this
> particular number in /e/i/dnsmasq
> (line 815 in http://pastebin.com/FV09f2jG)
> 
> procd_add_raw_trigger "interface.*" 2000 /etc/init.d/dnsmasq reload
> 
> this number, as the following link suggests
> http://wiki.prplfoundation.org/wiki/Procd_reference#procd_add_raw_trigger.28event.2C_timeout.2C_.5Bscript.5D.29
> is the number of milliseconds that procd will wait after the trigger (in
> this case, anything related to an "interface" AFAIU) before executing
> "dnsmasq reload"
> 
> i put 6 (60 seconds) and now the reload only happens once (after ~60
> seconds from the first "/e/i/dnsmasq boot") and dnsmasq starts without
> problem
> 
> so it looks to me like a race condition, where two "interface.*" events
> are happening one after the other, triggering two consecutive reloads,
> the first reload doesn't finish its work before the second reload comes,
> and the second reload kills the first reload, and suicides itself for
> some reason.
> 
> setting a long raw_trigger timeout works around the problem because the
> "interface.*" events happen all inside the 60 second window, and procd
> runs "/e/i/dnsmasq reload" only once
> 
> i can imagine that few people came across this since dnsmasq normally
> takes less than 2 seconds to start, and so the first reload is already
> done starting when the second reload is triggered.
> 
> my added "sleeps" help reproduce the issue by artificially making
> dnsmasq take always longer than 2 seconds, but without them the race
> condition is still there, only it doesn't end badly every boot (only
> some boots, and on some hardware)
> 
> is this enough info to point the right direction for someone to look
> into procd? (not me, not enough proficient in C, sorry)

bump?
i'm more than happy to do any more needed tests, given any pointer in
the right direction

> 
> (running two long "/e/i/dnsmasq reload" in parallel manually via
> console, the second one doesn't kill the first one, which makes me think
> that procd is the murderer)
> 
> On 22/03/17 23:29, Yousong Zhou wrote:
>> From the log, there are at least 4 runs of /etc/init.d/dnsmasq
>>
>> This is "/e/c/dnsmasq boot" and returns immediately because the script skips
>> this phase and the dnsmasq service is expected to be brought up by interface
>> trigger events
>>
>>  root@coco:~# logread | grep dns
>>  Wed Mar 22 18:25:56 2017 user.notice root: guidebug dnsmasq init
>>  Wed Mar 22 18:25:56 2017 user.notice root: guidebug dnsmasq boot
>>  Wed Mar 22 18:25:56 2017 user.notice root: guidebug dnsmasq startsrv 1,
>>  Wed Mar 22 18:25:56 2017 user.notice root: guidebug dnsmasq srvtrg
>>
>> This one I do not have much clue about
>>
>>  Wed Mar 22 18:25:57 2017 user.notice root: guidebug dnsmasq init
>>
>> This one was brought up by the interface trigger event.  Your log stopped at
>> "wait 18" without "wait 16" because the service_triggers call in the
>> /e/c/dnsmasq has set the timeout of event handling to be 2000ms.  That's how
>> the logger and sleep call were impeding the process
>>
>>  Wed Mar 22 18:26:06 2017 user.notice root: guidebug dnsmasq init
>>  Wed Mar 22 18:26:06 2017 user.notice root: guidebug dnsmasq reload
>>  Wed Mar 22 18:26:06 2017 user.notice root: guidebug dnsmasq startsrv ,
>>  Wed Mar 22 18:26:06 2017 user.notice root: guidebug dnsmasq realstart 
>> cfg02411c wait 20
>>  Wed Mar 22 18:26:08 2017 user.notice root: guidebug dnsmasq realstart 
>> cfg02411c wait 18
>>
>> This one I also have no clue about.  It didn't even make it to the 
>> "realstart"
>> part.  So checking further what happened in between was causing that should 
>> be
>> helpful
>>
>>  Wed Mar 22 18:26:10 2017 user.notice root: guidebug dnsmasq init
>>  Wed Mar 22 18:26:10 2017 user.notice root: guidebug dnsmasq reload
>>  Wed Mar 22 18:26:10 2017 user.notice root: guidebug dnsmasq startsrv ,
>>
> 

--- End Message ---
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] Release 17.01.0 binary packages have changed and SDK inconsistency

2017-03-29 Thread Daniel Golle
On Wed, Mar 29, 2017 at 05:28:07AM +0200, Pau wrote:
> On 29/03/17 02:45, Daniel Golle wrote:
> > On Tue, Mar 28, 2017 at 10:55:09PM +0200, Pau wrote:
> >> Hi Daniel, thanks for your reply. Find my comments in-line.
> >>
> >> On 28/03/17 22:37, Daniel Golle wrote:
> >>> Hi Pau,
> >>>
> >>> On Tue, Mar 28, 2017 at 09:28:22PM +0200, Pau wrote:
>  Hi.
> 
>  I am using the SDK and IB from LEDE 17.01.0 release (mips_24kc). I've
>  been using it successfully the last days until now. I did not change
>  anything but I get the error:
> 
>  * opkg_install_pkg: Package luci-lib-nixio sha256sum mismatch. Either
>  the opkg or the package index are corrupt. Try 'opkg update'.
> 
>  Then I went to downloads.lede-project.org [1] and I see that there are
>  new compiled packages with date "Tue Mar 28 18:35:15 2017".
>  Luci-lib-nixio is one of them [2] (which now is published on the
>  repository with a new version).
> >>>
> >>> Packages from the feeds and even base-packages (think: openssl) can
> >>> change after a release, just like for other distributions.
> >>> The error message you see more looks like being caused by inconsistent
> >>> Package index not matching the actual binaries. Maybe regenerating
> >>> the index can fix that...?
> >>
> >> I know I can manually fix it. But let me explain my corner case:
> >>
> >> The current SDK is compiling luci-lib-nixio which I want to include to
> >> the firmware generated by the ImageBuilder (I want to include my local
> >> version, not the repository one).
> > 
> > Ok, now I understand better. So the package index of the binary feed
> > is consistent.
> > 
> >>
> >> The source of this package comes from the SDK's feeds.conf.default which
> >> points to commit "a100738163585ae1edc24d832ca9bef1f34beef0".
> >>
> >> As the current LEDE official repository for 17.01.0 has been updated the
> >> luci-lib-nixio package has a newer version. In consequence the
> >> ImageBuilder selects it instead of my local compiled package.
> > 
> > I get that problem, and it can be solved easily by removing the git
> > commit hash from feeds.conf in the SDK and rather use the current
> > snapshot instead.
> > I agree that the lack of a lede-17.01 release branch on the package
> > feeds may cause weird and problematic situations, especially in the
> > future once LEDE HEAD has diverged more from 17.01.x released versions.
> 
> Sure, it is what I did. But I liked the idea of sticking and trusting
> the official feeds.conf.default included in the SDK.

I just realize that a lede-17.01 branch has been created for
openwrt/packages.git, openwrt/luci.git, openwrt/telephony.git and
openwrt-routing/packhages.git, just openwrt-management/packages.git
seems to lack a lede-17.01 release branch.


> 
> >>
> >> So it is an unexpected behavior, and it makes impossible for us to be
> >> sure that our LibreMesh SDK will work, not even when sticking to a LEDE
> >> release.
> > 
> > I fail to see the drama, why would that truely prevent you from doing
> > anything? What is the behaviour you'd expect?
> 
> The drama is that I've been following OpenWRT (and now LEDE) for so many
> years and stability is not (¿yet?) a word that I would use to describe it.
> 
> As you already know in the past we were freezing the OpenWRT buildroot
> so we were 100% sure that we were producing always the same release
> firmware (which was kind of deeply tested by us). Now (in 2017) I'd like
> to trust on LEDE releases and be sure that no new bugs are introduced.
> But two months after (not even) the public announcement of 17.01.0 I see
> new versions entering into the release official packages. We cannot be
> testing all this stuff all the time... it's a long process.

As with every release, those updates (to the respective lede-17.01
branch) should be bug- and security-fixes only and should never break
compatibility of depending packages.
If that's still too much movement, you can bake your own package builds
and use those in the ImageBuilder by editing repositories.conf instead
of using the upstream ones.

> 
> So it is not an actual drama but I am a bit afraid. I'm really happy
> with the evolution of OpenWRT and now LEDE (mainly since AA) but still I
> need to see with my own eyes that a release is truly stable and
> seriously taken.
> 
> >>
> >> In this case luci.lib-nixio is not a critical package and we can use the
> >> vanilla one, but the same might happen with any other package that we
> >> are compiling with different options and we expect the IB to use it
> >> instead of the online one.
> > 
> > Ok, all this is expected behaviour and shouldn't prevent anyone from
> > building their packages using the SDK and then including their binaries
> > in the ImageBuilder.
> > 
> > Simply speaking, if you modify (ie. fork) a package you should either
> > rename it or pull-request your changes upstream.
> 
> Not all changes can be always pull-requested**. Of course we have to
> walk 

Re: [LEDE-DEV] Release 17.01.0 binary packages have changed and SDK inconsistency

2017-03-29 Thread Bastian Bittorf
* James Feeney  [29.03.2017 10:15]:
> Let me say again, I think that this is an important issue that the LEDE 
> project
> needs to address and remedy.  I believe that the ultimate credibility and
> reputation of the LEDE project is at stake.

what we do "internally":

checkout LEDE and all feeds at a specific timestamp.
Maybe thats an option for you too.

bye, bastian

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] [PATCH v2] ar8327: Add workarounds for AR8337 switch.

2017-03-29 Thread John Crispin



On 26/03/17 10:08, Vittorio Gambaletta (VittGam) wrote:

-   .phy_fixup = ar8327_phy_fixup,
+   //.phy_fixup = ar8337_phy_fixup, // not needed at the moment
.set_mirror_regs = ar8327_set_mirror_regs,
.get_arl_entry = ar8327_get_arl_entry,
.sw_hw_apply = ar8327_sw_hw_apply

Hi,

why is this not needed ?

John

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] [PATCH 2/3][RFC] base-files: add option to rc.common to disable default reload behavior

2017-03-29 Thread Alexandru Ardelean
On Tue, Mar 28, 2017 at 11:36 PM, Hans Dedecker  wrote:
> On Mon, Mar 20, 2017 at 3:08 PM, Alexandru Ardelean
>  wrote:
>> From: Alexandru Ardelean 
>>
>> Traditionally if a reload script fails, it will fallback to restart.
>>
>> That seems to be the default behavior in case no reload
>> handler has been specified, and `reload` will return 1.
>>
>> That also has the disadvantage of masking reload errors
>> from `/etc/init.d/ reload`.
>>
>> Still, it's a pretty old behavior, and in most cases
>> it should be fine.
>>
>> For other cases, the `RESTART_ON_RELOAD_ERR=0` can
>> be specified to override this.
>>
>> Not sure about the correctness of this approach,
>> so this patch is RFC.
> Discussed the restart-on-reload-fail behaviour with Felix and
> Matthias; general consensus is to remove the restart-on-reload-fail
> behaviour in rc.common. Packages do not seem to depend on this
> restart-on-reload fail behaviour and it's more logical to handle
> reload failure in the packages itself if extra logic is required.

Thanks for the consideration.

I'll submit another patch series just for this restart-on-reload fail.

I feel this mechanism was maybe intended to act as default reload
implementation.
In the sense, that a default reload hook is implemented returning
non-zero, and if no specific reload is implemented, this logic [
restart on reload fail ] would work.

Will think about it a bit.

Thanks
Alex

>
> Hans
>>
>> Signed-off-by: Alexandru Ardelean 
>> ---
>>  package/base-files/files/etc/rc.common | 8 +++-
>>  1 file changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/package/base-files/files/etc/rc.common 
>> b/package/base-files/files/etc/rc.common
>> index 95cf956..a893b09 100755
>> --- a/package/base-files/files/etc/rc.common
>> +++ b/package/base-files/files/etc/rc.common
>> @@ -139,7 +139,13 @@ ${INIT_TRACE:+set -x}
>> }
>>  }
>>
>> +RESTART_ON_RELOAD_ERR=${RESTART_ON_RELOAD_ERR:-1}
>> +
>>  ALL_COMMANDS="start stop reload restart boot shutdown enable disable 
>> enabled depends ${EXTRA_COMMANDS}"
>>  list_contains ALL_COMMANDS "$action" || action=help
>> -[ "$action" = "reload" ] && action='eval reload "$@" || restart "$@" && :'
>> +[ "$action" = "reload" ] && {
>> +   if [ "$RESTART_ON_RELOAD_ERR" == "1" ] ; then
>> +   action='eval reload "$@" || restart "$@" && :'
>> +   fi
>> +}
>>  $action "$@"
>> --
>> 2.7.4
>>

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] Release 17.01.0 binary packages have changed and SDK inconsistency

2017-03-29 Thread Alberto Bursi


On 03/29/2017 06:08 AM, James Feeney wrote:
> Realize?  No - I'm still learning how this build process works in LEDE.  My
> impression is that you are distinguishing between "packages" and some other 
> type
> of thing, which the lede-project.org site seems to implicitly, and 
> generically,
> refer to as just "firmware" - even though a "package" is also a type of
> "firmware", and this "firmware" is also a type of "package".  I will infer 
> that
> you mean to distinguish between the "kernel and base filesystem" package from
> all other "packages".

LEDE is using the standard term for "package" in linux world, where a 
"package" is an archive containing a program or some scripts and its 
accompanying configuration and information used to integrate it in the 
system, installed by a package manager (opkg in this case), and the 
whole system is made of such packages assembled together around a Linux 
kernel.

Each package is compiled on its own and then when it's all done the 
packages are "installed" in a temporary root that will then be 
compressed to become the squashfs root in the device firmware.

Also the kernel is handled like a package for the build system, but 
since most devices expect it outside of root filesystem in various ways, 
it's added to the firmware image the way the device's bootloader expects it.


This is all pretty standard for any other Linux distro, Ubuntu, Debian, 
OpenSUSE, RHEL, whatever.

> You also seem to be implying that all the various "kernel and base filesystem"
> package builds are now precisely repeatable - even though this fact does not
> seem to be documented anywhere.

If you look at a random package makefile (the file defining settings for 
building a specific package)
https://github.com/lede-project/source/blob/master/package/utils/adb/Makefile

You will notice that it states the official download link for the 
sources, a SHA256 hash and a specific version for the package name.
On some other packages it will state git commit or other way to identify 
the same source to pull down and build.
They generally favor tarballs, archives with sources from official 
releases, but some upstream projects don't always have that.

In /patches folder of each package's folder you will find any patches 
that will be applied to the source after downloading it, before 
compiling, again the example package:
https://github.com/lede-project/source/blob/master/package/utils/adb/patches/001-create_Makefile.patch
There are also other folders for configuration files or for uci 
integration for each package that needs it (the example package does not).

The kernel's makefile is a bit more complex because of reasons, but you 
can see that it still uses the same structure,
pulling down the same kernel version (depending from device) every time.

https://github.com/lede-project/source/blob/master/package/kernel/linux/Makefile

Then all this stuff is compiled by LEDE's own toolchain, which is again 
handled like packages (see /toolchain and /tools) so you will always 
have the same compiler/tools as everyone else as they get downloaded 
from the same sources and at the same version.

When you run a "make" the LEDE build system will use your system's 
existing building infrastructure to compile the LEDE's toolchain first, 
and then use that to compile the packages. This also has the major 
benefit of not requiring the user to set up a cross-compiling toolchain 
that is rather annoying and relatively complex.

Is this enough for having repeatable builds?
I'm more like a long-time user so I don't know what "repeatable builds" 
mean beyond what I read from the internet, it seems this is more or less it.

>
> With respect to the lingo, "package feeds" and "project making the release", I
> have no understanding of the meaning of that statement.  Perhaps you could
> define precisely "package feeds" and "project making the release"?

Just as OpenWRT, there is a main project where the packages are 
maintained directly from core developers, and "package feeds" that are 
source repositories that contain additional packages, maintained by 
community (each package has its own maintainer).
This is LEDE's main repo (on github) https://github.com/lede-project/source

These are official package feeds (currently shared with OpenWRT):
https://github.com/openwrt/luci
https://github.com/openwrt/telephony
https://github.com/openwrt-routing/packages

Being "official feeds", the stuff from there will get compiled and 
offered in official download server, but it's not technically LEDE or 
OpenWRT proper, it's community-maintained.
The "package feed" system is designed to allow easy addition of your own 
custom feeds too in your custom firmware images, but you will have to 
compile them and host them on your own servers, of course.

If you look at the built packages here, 
https://downloads.lede-project.org/releases/17.01.0/packages/x86_64/

You find that they are divided by feed name.
Packages in "base" come from main