Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[Delbar Jos]

Hauke Mehrtens  wrote:
> On 05/27/2016 01:43 AM, Delbar Jos wrote:
> > At Technicolor we have followed with great interest the recent proposals
> to enhance OpenWrt with an open source solution for TR-069 remote
> management. As one of the world's largest vendors of modems and routers
> for carrier applications, making use of OpenWrt in a significant share of our
> install base, we want to support this initiative in a meaningful way.
> Concretely, we are willing to open source Technicolor's in-house TR-069
> solution and thereby contribute to OpenWrt:
> >  * a TR-069 protocol agent,
> >  * a data model mapping framework that we use to bridge the world of
> > OpenWrt, UCI, UBUS ... with the world of TR-069, TR-098, TR-181 ...
> > (and by extension with the world of SNMP, MIB, NETCONF, YANG ...),
> >  * a set of mappings.
> 
> That is really nice to hear. For me personally it looks like the remote
> management with TR-069 and similar protocols is one of the biggest
> extensions commercial vendors add to the user space of OpenWrt to make it
> fit their needs.
> 
> In addition to the TR-* family does this also include support for SNMP, MIB,
> NETCONF, YANG ?

Our proposal does not include protocol agents for SNMP or NETCONF but it does 
include a data model mapping framework that can be used to bridge between these 
protocols' data models MIB and YANG and an OpenWrt environment. If you compare 
this framework with the slides shown by Felix at the prpl weekly then it's an 
implementation of the "configuration backend", or if you compare it with the 
architecture of a NETCONF project like 
https://wiki.openwrt.org/inbox/howto/opencpe then it's an implementation of 
"mand". (We aren't using mand.)

> That's good to hear, would it also be possible that other interested people
> can join such a workshop?

The answer is yes as far as I'm concerned.

Jos
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[Delbar Jos]

Felix Fietkau  wrote:
> > We are conscious of the fact that together with the proposals made by
> > Felix, Luka and Wojtek we are now looking at many "competing" proposals.
> > As a next step, we recommend to organize a workshop, at a practical
> > location and time, where we put everything on the table and define the
> > most appropriate path forward to the benefit of OpenWrt as a whole.
> I think such a workshop would be a great idea. It would be nice to have the
> code available for review some time before that workshop, so we can all take
> a detailed look at the various proposals before we sit down and decide how
> to move forward with this.

I agree that would be helpful. In our case it will take us some weeks before we 
can formally release sources. We will be able to share documentation on 
architecture, API, features, etc. up front to get the discussions going.

Jos
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[Delbar Jos]

David Lang  wrote:
> I wasn't questioning why it's useful to support TR-069. The only part I was
> questioning was the statement that OpenWRT needed work to make it
> support remote management.
> 
> There are already many tools to remotely manage/monitor OpenWRT
> 
> But that's why I'm saying that it seems like most of the work is in the 
> protocol
> interface. If there is already a daemon that does the network protocol
> properly, that should make things very easy. If such a daemon needs to be
> written, that would be the place I would suggest that they focus. There are a
> lot of people who can do the plumbing work to make the daemon do the
> right thing on the system, who are not in a position to work on the network
> protocol side and make sure that it works properly with the management
> software.

Listing some of the things that come to mind:
 * Data models. The language that TR-069 uses to describe a certain feature or 
function is not the same language that OpenWrt uses. The translation is often 
not straightforward.
 * Events/notifications. This boils down to the TR-069 protocol agent notifying 
the ACS when something on the CPE changes. Now imagine that a change occurs to 
a configuration parameter in UCI, like the wireless SSID. How to address this?
 * Commit and apply. When the ACS changes the configuration of the CPE, it also 
expects this configuration to be applied soon after. Updating the configuration 
in UCI is not enough.
 * Transactional behavior. A set of TR-069 configuration changes is either 
applied successfully or not at all. UCI has a commit function but how to check 
if a configuration change in UCI will also be successfully applied by the 
system? Now one can debate what "successful" means but you get the point.
 * Persistent indices. TR-069 expects that a certain object in an array, like 
"the second network interface", retains its index "the second" during the 
lifetime of the CPE (barring a factory reset) even if "the first network 
interface" is deleted. The way that lists work in UCI doesn't make this 
trivial. Using named instances or aliases are possible ways out.

We've attempted to overcome these challenges in our TR-069 solution within the 
confines of OpenWrt. I'm sure that other management systems (local ones like 
LuCi or remote ones) are solving similar problems in different ways. Instead of 
doing all the heavy lifting on the protocol side, I'm hopeful that we can carve 
out a common subset of requirements that apply to management systems in general 
and come to better and more reusable frameworks.

Jos
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[David Lang]

On Sat, 28 May 2016, Hauke Mehrtens wrote:


On 05/27/2016 12:43 PM, David Lang wrote:

On Thu, 26 May 2016, Delbar Jos wrote:


We are conscious of the fact that together with the proposals made by
Felix, Luka and Wojtek we are now looking at many "competing"
proposals. As a next step, we recommend to organize a workshop, at a
practical location and time, where we put everything on the table and
define the most appropriate path forward to the benefit of OpenWrt as
a whole.


nothing wrong with supporting many different remote management daemons.


TR-069 is a complicated remote management system and in order to make
this initiative a success, we must ensure that the complexity is
handled in an elegant way and with respect for OpenWrt's core
architecture. More than on the protocol itself, we believe that we
should focus on the architectural enhancements required to support
remote management in general.


What is it that you think is needed to "support remote management in
general"?

It's worth pointing out that many people are remotely managing OpenWRT
devices, Ansible/Salt/Puppet/Chef/etc are all common tools for the job.

now, those are all tools aimed at managing Linux Servers, not networking
gear, but OpenWRT is a server.

So I'd suggest starting off by creating a daemon that talks  and just stores the stuff it's sent in some simple files so
that it can return the info when queried.

Once you have something that talks the network protocol correctly,
modifying it to change the real files, make uci calls, etc for different
distros is much easier (just write your daemon with the expectation that
the input and output details are going to change, so don't get fancy
with them).

David Lang


The TR-069 family is currently wildly used by ISPs controlling the (DSL)
CPE devices of their customers. There are probably more than 100 million
device controlled by standards from the TR-069 family out there.  When
you get a DSL router from your ISP or buy one in the retail store it is
very likely it supports the standards from the TR-069 family, as a
vendor in this area you basically need support for this to sell your
devices.


I wasn't questioning why it's useful to support TR-069. The only part I was 
questioning was the statement that OpenWRT needed work to make it support remote 
management.


There are already many tools to remotely manage/monitor OpenWRT

But that's why I'm saying that it seems like most of the work is in the protocol 
interface. If there is already a daemon that does the network protocol properly, 
that should make things very easy. If such a daemon needs to be written, that 
would be the place I would suggest that they focus. There are a lot of people 
who can do the plumbing work to make the daemon do the right thing on the 
system, who are not in a position to work on the network protocol side and make 
sure that it works properly with the management software.


David Lang
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[Florian Eckert]

Hello,

nice to hear the there are intentions to make a openwrt/lede solution. We
are now using a freecwmp fork ( EasyCwmp ) to configure our devices.
Freecwmo was mit maintained anymore. We extend the parameter set with the
openwrt parameter which are not in the standard.

Will the Meeting online (irc) for everyone?

Regards

Flo

Am 28.05.2016 1:34 nachm. schrieb "Hauke Mehrtens" :
>
> On 05/27/2016 12:43 PM, David Lang wrote:
> > On Thu, 26 May 2016, Delbar Jos wrote:
> >
> >> We are conscious of the fact that together with the proposals made by
> >> Felix, Luka and Wojtek we are now looking at many "competing"
> >> proposals. As a next step, we recommend to organize a workshop, at a
> >> practical location and time, where we put everything on the table and
> >> define the most appropriate path forward to the benefit of OpenWrt as
> >> a whole.
> >
> > nothing wrong with supporting many different remote management daemons.
> >
> >> TR-069 is a complicated remote management system and in order to make
> >> this initiative a success, we must ensure that the complexity is
> >> handled in an elegant way and with respect for OpenWrt's core
> >> architecture. More than on the protocol itself, we believe that we
> >> should focus on the architectural enhancements required to support
> >> remote management in general.
> >
> > What is it that you think is needed to "support remote management in
> > general"?
> >
> > It's worth pointing out that many people are remotely managing OpenWRT
> > devices, Ansible/Salt/Puppet/Chef/etc are all common tools for the job.
> >
> > now, those are all tools aimed at managing Linux Servers, not networking
> > gear, but OpenWRT is a server.
> >
> > So I'd suggest starting off by creating a daemon that talks  > protocol> and just stores the stuff it's sent in some simple files so
> > that it can return the info when queried.
> >
> > Once you have something that talks the network protocol correctly,
> > modifying it to change the real files, make uci calls, etc for different
> > distros is much easier (just write your daemon with the expectation that
> > the input and output details are going to change, so don't get fancy
> > with them).
> >
> > David Lang
>
> The TR-069 family is currently wildly used by ISPs controlling the (DSL)
> CPE devices of their customers. There are probably more than 100 million
> device controlled by standards from the TR-069 family out there.  When
> you get a DSL router from your ISP or buy one in the retail store it is
> very likely it supports the standards from the TR-069 family, as a
> vendor in this area you basically need support for this to sell your
> devices.
>
> In other technologies you have different protocols to manage your
> devices, like cable often uses something different and EPON and GPON
> even have all their own management standards. Then there are also some
> technology independent standards and so on. It makes sense to build such
> a solution in a way to make it easily to expendable for new protocols.
>
> Hauke
> ___
> openwrt-devel mailing list
> openwrt-devel@lists.openwrt.org
> https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[Hauke Mehrtens]

On 05/27/2016 12:43 PM, David Lang wrote:
> On Thu, 26 May 2016, Delbar Jos wrote:
> 
>> We are conscious of the fact that together with the proposals made by
>> Felix, Luka and Wojtek we are now looking at many "competing"
>> proposals. As a next step, we recommend to organize a workshop, at a
>> practical location and time, where we put everything on the table and
>> define the most appropriate path forward to the benefit of OpenWrt as
>> a whole.
> 
> nothing wrong with supporting many different remote management daemons.
> 
>> TR-069 is a complicated remote management system and in order to make
>> this initiative a success, we must ensure that the complexity is
>> handled in an elegant way and with respect for OpenWrt's core
>> architecture. More than on the protocol itself, we believe that we
>> should focus on the architectural enhancements required to support
>> remote management in general.
> 
> What is it that you think is needed to "support remote management in
> general"?
> 
> It's worth pointing out that many people are remotely managing OpenWRT
> devices, Ansible/Salt/Puppet/Chef/etc are all common tools for the job.
> 
> now, those are all tools aimed at managing Linux Servers, not networking
> gear, but OpenWRT is a server.
> 
> So I'd suggest starting off by creating a daemon that talks  protocol> and just stores the stuff it's sent in some simple files so
> that it can return the info when queried.
> 
> Once you have something that talks the network protocol correctly,
> modifying it to change the real files, make uci calls, etc for different
> distros is much easier (just write your daemon with the expectation that
> the input and output details are going to change, so don't get fancy
> with them).
> 
> David Lang

The TR-069 family is currently wildly used by ISPs controlling the (DSL)
CPE devices of their customers. There are probably more than 100 million
device controlled by standards from the TR-069 family out there.  When
you get a DSL router from your ISP or buy one in the retail store it is
very likely it supports the standards from the TR-069 family, as a
vendor in this area you basically need support for this to sell your
devices.

In other technologies you have different protocols to manage your
devices, like cable often uses something different and EPON and GPON
even have all their own management standards. Then there are also some
technology independent standards and so on. It makes sense to build such
a solution in a way to make it easily to expendable for new protocols.

Hauke
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[Hauke Mehrtens]

On 05/27/2016 01:43 AM, Delbar Jos wrote:
> Hi all,
> 
> At Technicolor we have followed with great interest the recent proposals to 
> enhance OpenWrt with an open source solution for TR-069 remote management. As 
> one of the world's largest vendors of modems and routers for carrier 
> applications, making use of OpenWrt in a significant share of our install 
> base, we want to support this initiative in a meaningful way. Concretely, we 
> are willing to open source Technicolor's in-house TR-069 solution and thereby 
> contribute to OpenWrt:
>  * a TR-069 protocol agent,
>  * a data model mapping framework that we use to bridge the world of OpenWrt, 
> UCI, UBUS ... with the world of TR-069, TR-098, TR-181 ... (and by extension 
> with the world of SNMP, MIB, NETCONF, YANG ...),
>  * a set of mappings.

That is really nice to hear. For me personally it looks like the remote
management with TR-069 and similar protocols is one of the biggest
extensions commercial vendors add to the user space of OpenWrt to make
it fit their needs.

In addition to the TR-* family does this also include support for SNMP,
MIB, NETCONF, YANG ?

> We are conscious of the fact that together with the proposals made by Felix, 
> Luka and Wojtek we are now looking at many "competing" proposals. As a next 
> step, we recommend to organize a workshop, at a practical location and time, 
> where we put everything on the table and define the most appropriate path 
> forward to the benefit of OpenWrt as a whole.

That's good to hear, would it also be possible that other interested
people can join such a workshop?

> TR-069 is a complicated remote management system and in order to make this 
> initiative a success, we must ensure that the complexity is handled in an 
> elegant way and with respect for OpenWrt's core architecture. More than on 
> the protocol itself, we believe that we should focus on the architectural 
> enhancements required to support remote management in general.
> 
> Looking forward to hearing from you.
> 
> Jos Delbar & Dirk Feytons
> Technicolor

Hauke
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[Felix Fietkau]

On 2016-05-27 01:43, Delbar Jos wrote:
> Hi all,
> 
> At Technicolor we have followed with great interest the recent
> proposals to enhance OpenWrt with an open source solution for TR-069
> remote management. As one of the world's largest vendors of modems
> and routers for carrier applications, making use of OpenWrt in a
> significant share of our install base, we want to support this
> initiative in a meaningful way. Concretely, we are willing to open
> source Technicolor's in-house TR-069 solution and thereby contribute
> to OpenWrt:
>  * a TR-069 protocol agent,
>  * a data model mapping framework that we use to bridge the world of
>OpenWrt, UCI, UBUS ... with the world of TR-069, TR-098, TR-181 ...
>(and by extension with the world of SNMP, MIB, NETCONF, YANG ...),
>  * a set of mappings.
Nice!

> We are conscious of the fact that together with the proposals made
> by Felix, Luka and Wojtek we are now looking at many "competing" proposals.
> As a next step, we recommend to organize a workshop, at a practical
> location and time, where we put everything on the table and define the
> most appropriate path forward to the benefit of OpenWrt as a whole.
I think such a workshop would be a great idea. It would be nice to have
the code available for review some time before that workshop, so we can
all take a detailed look at the various proposals before we sit down and
decide how to move forward with this.

> TR-069 is a complicated remote management system and in order to make
> this initiative a success, we must ensure that the complexity is
> handled in an elegant way and with respect for OpenWrt's core
> architecture. More than on the protocol itself, we believe that we
> should focus on the architectural enhancements required to support
> remote management in general.
Sounds good!

- Felix
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[Drasko DRASKOVIC]

Hi Karl,

On Fri, May 27, 2016 at 1:55 PM, Karl Palsson  wrote:
>
> David Lang  wrote:
>> On Thu, 26 May 2016, Delbar Jos wrote:
>>
>> > We are conscious of the fact that together with the proposals made by 
>> > Felix,
>> > Luka and Wojtek we are now looking at many "competing" proposals. As a next
>> > step, we recommend to organize a workshop, at a practical location and 
>> > time,
>> > where we put everything on the table and define the most appropriate path
>> > forward to the benefit of OpenWrt as a whole.
>>
>> nothing wrong with supporting many different remote management
>> daemons.
>>
>> > TR-069 is a complicated remote management system and in order to make this
>> > initiative a success, we must ensure that the complexity is handled in an
>> > elegant way and with respect for OpenWrt's core architecture. More than on 
>> > the
>> > protocol itself, we believe that we should focus on the architectural
>> > enhancements required to support remote management in general.
>>
>> What is it that you think is needed to "support remote
>> management in general"?
>>
>> It's worth pointing out that many people are remotely managing
>> OpenWRT devices, Ansible/Salt/Puppet/Chef/etc are all common
>> tools for the job.
>
> Really?  python, python, ruby, ruby.  None of those are really fun enjoyable 
> tasks on _my_ openwrt/leded devices.
>
>> now, those are all tools aimed at managing Linux Servers, not
>> networking gear, but OpenWRT is a server.
>>
>> So I'd suggest starting off by creating a daemon that talks
>>  and just stores the stuff it's sent in some
>> simple files so that it can return the info when queried.
>
> Did you read the intro to Delbar's mail, describing that they
> already have a complete tr069 project, for managing openwrt
> devices, that they want to open source, and want to collaborate
> on making it more useful for all, and perhaps see if there are
> common pain points that can be resolved by handling things
> differently on the lede/openwrt side, rather than working around
> on the tr069 side?
>
> I think it's exciting and I'd love to hear more about it.
> ansible/salt/puppet/chef have been far too heavy to run, and
> openvpn servers granting remote shell access is far too tedious
> for daily use.

I am very interested to see TR-069 solution. IMHO what is really
useful in it is Amendment 5, NAT traversal based on XMMP. Both AllJoyn
and Iotivity seem to push this approach to managing CPEs - naturally,
it has been proven and widely used. For the reference, here is an
interesting discussion I had with Thiago Macieira from Iotivity:
https://lists.linuxfoundation.org/pipermail/iotivity-dev/2015-October/002867.html

However, I would personally look more at OMA LwM2M - it would be much
lighter and more fun to implement ;). NAT traversal is not so
straightforward, but it would be interesting to investigate it. Then
clients will be ultra-light.

BR,
Drasko
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[Karl Palsson]

David Lang  wrote:
> On Thu, 26 May 2016, Delbar Jos wrote:
> 
> > We are conscious of the fact that together with the proposals made by 
> > Felix, 
> > Luka and Wojtek we are now looking at many "competing" proposals. As a next 
> > step, we recommend to organize a workshop, at a practical location and 
> > time, 
> > where we put everything on the table and define the most appropriate path 
> > forward to the benefit of OpenWrt as a whole.
> 
> nothing wrong with supporting many different remote management
> daemons.
> 
> > TR-069 is a complicated remote management system and in order to make this 
> > initiative a success, we must ensure that the complexity is handled in an 
> > elegant way and with respect for OpenWrt's core architecture. More than on 
> > the 
> > protocol itself, we believe that we should focus on the architectural 
> > enhancements required to support remote management in general.
> 
> What is it that you think is needed to "support remote
> management in general"?
> 
> It's worth pointing out that many people are remotely managing
> OpenWRT devices, Ansible/Salt/Puppet/Chef/etc are all common
> tools for the job.

Really?  python, python, ruby, ruby.  None of those are really fun enjoyable 
tasks on _my_ openwrt/leded devices.  

> now, those are all tools aimed at managing Linux Servers, not
> networking gear, but OpenWRT is a server.
> 
> So I'd suggest starting off by creating a daemon that talks
>  and just stores the stuff it's sent in some
> simple files so that it can return the info when queried.

Did you read the intro to Delbar's mail, describing that they
already have a complete tr069 project, for managing openwrt
devices, that they want to open source, and want to collaborate
on making it more useful for all, and perhaps see if there are
common pain points that can be resolved by handling things
differently on the lede/openwrt side, rather than working around
on the tr069 side?

I think it's exciting and I'd love to hear more about it.
ansible/salt/puppet/chef have been far too heavy to run, and
openvpn servers granting remote shell access is far too tedious
for daily use.

Cheers,
Karl P

signature.asc
Description: OpenPGP Digital Signature
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [LEDE-DEV] TR-069 for OpenWrt

[David Lang]

On Thu, 26 May 2016, Delbar Jos wrote:

We are conscious of the fact that together with the proposals made by Felix, 
Luka and Wojtek we are now looking at many "competing" proposals. As a next 
step, we recommend to organize a workshop, at a practical location and time, 
where we put everything on the table and define the most appropriate path 
forward to the benefit of OpenWrt as a whole.


nothing wrong with supporting many different remote management daemons.

TR-069 is a complicated remote management system and in order to make this 
initiative a success, we must ensure that the complexity is handled in an 
elegant way and with respect for OpenWrt's core architecture. More than on the 
protocol itself, we believe that we should focus on the architectural 
enhancements required to support remote management in general.


What is it that you think is needed to "support remote management in general"?

It's worth pointing out that many people are remotely managing OpenWRT devices, 
Ansible/Salt/Puppet/Chef/etc are all common tools for the job.


now, those are all tools aimed at managing Linux Servers, not networking gear, 
but OpenWRT is a server.


So I'd suggest starting off by creating a daemon that talks  and 
just stores the stuff it's sent in some simple files so that it can return the 
info when queried.


Once you have something that talks the network protocol correctly, modifying it 
to change the real files, make uci calls, etc for different distros is much 
easier (just write your daemon with the expectation that the input and output 
details are going to change, so don't get fancy with them).


David Lang
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel