On 10/19/2018 7:23 PM, Federico Capoano wrote:
On Thursday, October 18, 2018 at 9:36:47 PM UTC+2, Pim Janssen wrote:
On 10/18/2018 6:19 PM, Federico Capoano wrote:
Nice discussion here guys, sorry for my late reply, I was away
and busy at the Google Mentor Summit.
On Sunday, October 14, 2018 at 11:12:18 AM UTC+2, Pim Janssen wrote:
Hi everyone,
I am looking and playing with Openwisp and Netjson. I hope i
can ask u
some questions.
Currently i am trying to automate a managed switch. I came to
the
horrible conclusion it only allows for changes using web-ui.
I made a
very basic webscraper that is able to do some very basic
things like
enabling / disabling network ports or enabling / disabling
Power over
Ethernet (POE)
For managed switch on OpenWRT do we mean the "switch",
"switch_vlan" and "switch_port" sections in /etc/config/network?
If yes, you may have tried to manage those via OpenWISP without
success, is this the case?
If that's the case, it was because those type of configurations
were flagged as "unmanaged" by default in openwisp-config (the
openwrt agent which updates the configuration) and hence were
being ignored by the agent.
We changed this recently (thanks to Oliver):
https://github.com/openwisp/openwisp-config/commit/c8e333089b62c5afe19ccd5268f00fb6d267cfe1
<https://github.com/openwisp/openwisp-config/commit/c8e333089b62c5afe19ccd5268f00fb6d267cfe1>
I have not yet tried to connect it to openwisp. After reading the
project documentation i had the feeling this could be part of
OpenWisp. But before trying to implement it i first wanted to be
sure you all share part of the idea. I did networking for a few
different company's. I used hardware/software from most big
vendors. One of the things that started to annoy me is that for
every serie of devices (even within one brand) They use a
different design for there devices. I think it would be a
advantage if they could be managed from one interface given
someone writes a adapter.
Pim, I don't understand what we are talking about here specifically.
You mentioned being able to manage switches.
OpenWISP at the moment only supports OpenWRT, there are two
experimental configuration backends for Ubiquiti AirOS and Raspbian
but I'm not aware of anyone using those in production yet, more work
needs to be done on that front. So at the moment the only production
ready configuration backend for OpenWISP is OpenWRT. OpenWRT has a
programmable switch feature described here:
- https://openwrt.org/docs/guide-user/network/vlan/switch
- https://openwrt.org/docs/techref/hardware/switch
If this feature works for you, you can use OpenWISP for it.
You can update the configuration in /etc/config/openwisp and
remove the unmanaged sections.
It seems also that somebody is working on improving the switch
support in netjsonconfig (the component of OpenWISP which
generates the native router configurations from NetJSON):
https://github.com/peshev/netjsonconfig/commits/master
<https://github.com/peshev/netjsonconfig/commits/master>
Would such a module be something u see as being in the scope
of netjson?
I mean having some kind of adapter <Openwisp/configuration
frontend> -
<Adapter script> - <Switch WebUI>. The way i see netjson
would be as a
universal configuration format. Having something like
Openwisp produce a
netjson file and having something else (could be the device
itself)
applying it on the device.
Yes, I think it would be useful.
Adding it to the NetJSON spec is a slow process though, but
nobody is holding us to implement it in OpenWISP first.
Infact, that's how I've been doing recently, I'm implementing
things first in OpenWISP and improving the NetJSON spec once
things work well.
I have not looked very deep into it. But is there a document where
the endpoint to the OpenWisp controller is documented so i can
figure out how to get the config from the controller?
I think it's a 4 stage process
1. Write a config into OpenWisp
2. Get the netjson from the OpenWisp controller
3. Convert the netjson into actions that need to be applied on the
device.
4. Apply the actions on the device by any means needed for this
device (in my case a webui scraper)
I'm not sure I understand what you want to do.
Are you trying to say you want to use OpenWISP programmatically via
some sort of API or scripting?
I basically want to make my own client that will get the settings i
configure in OpenWisp and write it to the device.
The internal processes and flows of the software are not documented
yet. The project has been evolving fast and I hadn't had the time to
document these parts that are moving and evolving fast. I think that
can be done once the project reaches maturity and stability, which
could happen once the base modules are all completed and well
integrated with one another.
At the moment the only sensible option is to read the code, the module
you need to look into is: https://github.com/openwisp/django-netjsonconfig
With particular attention
to: https://github.com/openwisp/django-netjsonconfig/blob/master/django_netjsonconfig/controller/generics.py
Ill have a look thanks
Part 4 is implemented. But the other parts i still have to figure
out.
I miss somethings in the netjson standard
- encryption. I think there should be a standard way to
encrypt netjson
files on the wire with for example a shared key. U could
leave that to
the transport but i think it would be better to have a way of
doing it
in a standard.
NetJSON is a format, not a protocol.
If you check other similar JSON based formats like GeoJSON, they
also do not specify a standard way to encrypt it.
Additional RFCs may do that, but is it really needed right now?
There's so many other things to improve in OpenWISP and NetJSON
that I think we can leave this to users and developers, let them
encrypt NetJSON and send it how they want.
At the moment as Edoardo said, encryption of NetJSON is not even
important because the configuration is sent in native format to
routers via HTTPs.
I was/am thinking about sending the NetJSON files over MQTT. If u
do it properly i don't think u need encryption. I saw encryption
somewhere in the netjson docs without telling how. Never mind :)
Yes there's encryption mentioned in the DeviceConfiguration object,
that's for wifi encryption (WPA2 Personal, WPA2 Enterprise, WPS, etc).
- push vs pull:
Now all openwisp device do a configuration pull every x
minutes. I think
that's a waste of resources on the controller side. What it
your vision
on using MQTT/AMQP or another open protocol to push
configurations to
devices. Advantage in this case would also be that u don't
need to pull
config that often or not at all. The configuration can be
applied within
seconds after the are changed on the server no matter where
the device
is located (behind NAT/Firewall).
This does not concern NetJSON but rather OpenWISP (an
implementation of NetJSON).
Correct
First of all let me clarify that openwisp-config (the OpenWRT
agent) does not pull the entire configuration but an md5
checksum, it will download the entire configuration only if the
stored checksum is different from the checksum sent by the
server. Hence the resources used are really minimal.
Lets say a controller handles 1000 devices. U can chose to have a
delay of 5 minutes. But that still means u get ~3 hits per second.
It's not that heavy. But still feels like a waste of resources.
And still u have a delay of 5 minutes before the configuration
gets applied.
Most people will not get to that device count. But if it is
possible to add a lot of different devices on different sites.
Things could add up quickly.
Even if we used another method like MQTT over websockets we would
still have the overhead of all the devices being connected to the
controller all the time and sending hearthbeats and even in that case
it wouldn't be a big deal.
It's not a big deal really, I don't think you should worry about this
micro-optimization right now unless you plan on using a number of
devices that is higher than 10K and in that case we can resort to
caching and horizontal scaling, something we'll have to do anyway even
when we'll introduce the MQTT over websocket method of managing devices.
Remember: machine time is cheap. Programmer time is expensive. It's
better to sacrifice some machine/hardware time now in order to use the
human programmer time we save to reinvest it in completing other
modules that are urgently needed to properly manage a network. If you
talk to most people using and contributing to OpenWISP right now they
will tell you they all pretty much need the same things: monitoring,
alerts, firmware upgrade, a full geographic map, the ability to store
the full configuration of all devices in OpenWISP, freeradius, a guide
that explains how to integrate a captive portal solution, user
management system, users that can register to the wifi service
autonomously and have their phone number verified and so on. Once we
have completed these we'll free resources to work on optimizations and
improvements.
Is not that it's not possible to do all those things right now, it's
possible with some additional work to integrate other existing
software. But in OpenWISP I aim to provide some base, programmable,
hackable modules and/or some documentation that clearly explains how
to do that, even better if some sort of automation is provided to
achieve it with minimum effort.
It will take time to get there, as it has taken time to get where we
are now, but it greatly pays off. I'm more interested in long term
sustainability over short term improvements.
This strategy is already showing very encouraging results, we just
have to keep at insisting and patiently pushing the project forward.
Sorry for the lecture but I believe it's important for me to explain
why things as they are and show to everyone the direction where we are
headed to.
I want to be sure the community is aware of the direction of the
project so we can be aligned and push all in the same direction.
I should set some time aside to copy this kind of information on the
website or in the documentation somewhere indeed.
I understand what u mean. It's part of the reason i am asking.
In the connections branch of openwisp-controller we have an
implementation that allows to push configuration updates via SSH:
https://github.com/openwisp/openwisp-controller/tree/connections
<https://github.com/openwisp/openwisp-controller/tree/connections>
This will require either a VPN or port forwarding. I don't want to
forward ports to the public internet and a VPN could be a option
in some cases.
If you plan to deploy devices on other networks you don't directly
manage, you should plan to have a management VPN anyway because if any
problem happens you will need to SSH into the device to check what's
wrong. You will likely have clients or users complaining something is
not working and ask you to fix it and if you won't have a way to
connect to the troubled device you'll either have to go there
physically or suffer the complaints of your users/clients, eheheh.
OpenWISP can help you to create VPN clients automatically, and the new
version (which we'll release soon) is aware of the management
interface IP address: it shows it in the web UI and it can use that to
talk to the device.
Using other protocols is possible, I've been thinking of using
MQTT over websocket for quite a while now but haven't got around
starting this because the current mechanism is working quite well
and hence this rewrite is not as urgent as other things we are
working on now, but sooner or later we will get there.
This would also allow for a new possibility of having a easy
way to get
data from the device. And also integration in home-automation
systems
would be a lot easier.
Agreed, I think allowing OpenWISP to be used in other contexts
will be a win for the project.
I've seen many network controllers and dashboards being created
from scratch in different contexts: each network hardware vendor
makes its own controller (eg: Ubiquiti, Engenius, Mikrotic) which
only works with their own hardware, a new technology comes out
(eg: SDN, LoRa) and new controllers are created with very similar
feature sets (the ones OpenWISP also has or aims to have). That's
a waste of human resources and time. It would be like having
different groups of people re-implementing a different operating
system kernel (like the linux kernel) from scratch, each kernel
doing only a limited number of things and never achieving any
breakthrough.
If we can help to fix that problem with OpenWISP by creating an
open, generic, modular, reusable, robust and well documented
network controller and dashboard we'll have achieved one of the
goals of the project.
But to get there we will need time, great development effort and
a lot of patience from our users which will have to deal with
bugs, filing bug reports and sending feedback (as you are doing
now, which is very helpful BTW) to us developers so we can keep
improving the project.
A bit more notes:
- Edoardo mentioned SMTP, but I think he meant SNMP (Simple
Network Management Protocol)
- NetJSON is aimed at easing the development and improving
interoperability in networking software, its focus is not only
WISPs, it can be any type of network (eg: ISPs, public wifi,
mesh, LoRa, IoT)
I hope this clarifies things a bit.
It does thanks.
Welcome!
--
You received this message because you are subscribed to the Google
Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
