Hi Ben,
I guess as a first way, using such a discovery server sounds like a good
plan.
In the future I even imagine extending the query options we use so you
could for example provide an IP range and a set of protocols to check
and then the system probes every IP in that range for connections.
But that's future stuff ... now I think that this discovery server is
exactly what we need for OPC-UA.
And I agree, we can't provide user-names and passwords ... however it
would definitely be fun, if we could ;)
in this case I guess we can output {username} and {password} or alike in
the connection string. If we follow a standard format here, tools like
Streampipes can probably process it.
Regarding filtering: I agree. It would be cool to limit which devices
discovery is run on how many vlan hops should be allowed and in case of
active ip scans, we definitely need to limit the ip ranges and port
ranges. But I think we'll concentrate on the immediate tasks of finding
stuff and deal with filtering afterwards.
Chris
On 04.07.21 13:02, Ben Hutcheson wrote:
> Hi,
>
> Just commenting on the OPCUA side of things,
> Your description of discovery matches what is used for OPCUA, it is
> described here.
> https://reference.opcfoundation.org/v104/Core/docs/Part4/5.4.1/
> I would expect that we pass a url for the discovery server and a list of
> endpoints is returned using the format you described.
>
> Also as we use certificates/usernames/passwords that are defined in the
> connection string, we probably can't assume that we are going to be able to
> return a valid connection string for each endpoint as this information
> would also need to be included as parameters.
>
> For browsing, it is similar and is described here,
> https://reference.opcfoundation.org/v104/Core/docs/Part4/5.8.2/
> One concern I have is we probaby should have someway of limiting/filtering
> the fields returned, as having a large number of fields is kind of expected.
>
> Kind Regards
>
> Ben
>
>
>
> On Fri, Jul 2, 2021 at 3:02 PM Otto Fowler <[email protected]> wrote:
>
>> Enip and Profinet Discovery and configuration protocol may ‘switch’ after
>> discovery.
>>
>> Like, you send ENIP Identify on udp, and then connect on tcp to do other
>> commands I believe.
>> Also, you may use DCP to discover things, and then use s7 to talk to them (
>> the DCP identity response is over link layer, but has ip address as a field
>> ).
>>
>>
>>
>> From: Łukasz Dywicki <[email protected]> <[email protected]>
>> Reply: [email protected] <[email protected]> <[email protected]>
>> Date: July 2, 2021 at 13:17:14
>> To: [email protected] <[email protected]> <[email protected]>
>> Subject: Re: [DISCUSS] Discover and Browse API for PLC4J?
>>
>> Hey Chris,
>> Indeed - discovery is about finding interfaces/devices while browsing is
>> already interacting with them. I believe that in most of cases once
>> discovery is done browsing uses same transport as PLC connection itself.
>> At least this would be logical consequence. If someone knows other case
>> then it is right time to speak! :-)
>>
>> The AMS discovery answer returns you AMS net id of a node, so you know
>> IP of PLC and also its logical network identifier. As you pointed
>> usually it is just an <ip>.1.1. I actually use this convention when
>> making discovery caller ams. It is just my-ip + 1.1.
>> What we miss is a ams port (by default there is 851, but there are
>> others and it can vary). The user and password is needed only for route
>> setup, if you have route set up on PLC then this step is not necessary.
>> Anyhow, I think we need to have a look on OPC UA discovery (which is a
>> separate sub-spec) to see if we're set on discovery inputs. Given its
>> verbosity I do expect additional set of requirements coming from that end.
>>
>> For KNX I see indeed a similarity between CANopen (and profinet) how
>> node discovery could work. All these are "bus systems" thus connection
>> is not a point to point. In their case we see multiple participants
>> which are listening to a single communication channel. As soon as we
>> acquire link we can discover and browse through it.
>>
>> A short note. I am not sure if we could automate plain CAN discovery,
>> cause then we have 11 bit identifier which can be anything. Role of
>> these 11 bits is known only if we assume CANopen.
>> In worst case basic CAN could lead to us to discovery of up to 4095 CAN
>> nodes. ;-)
>>
>> Anyhow, I believe what you have sketched is pretty clear.
>>
>> Best,
>> Łukasz
>>
>>
>> On 02.07.2021 18:23, Christofer Dutz wrote:
>>> Hi Lukasz,
>>> br/>> so from what I read I sort of think you also aggree that
>> "Discovery"
>>> should find devices and the result is connection information we can then
>>> pass to the DriverManager to get a connection. And Browse will result
>>> with fields (Or parsed address strings) for a given connection.
>>> br/>> I think in the ADS Case ... we shouldn't do thee setting up of ADS
>> Routes
>>> as it involves sending a servers Admin password in clear text. However I
>>> think we are able to find an ADS device. As with ADS you also have to
>>> provide the source and targer ams net id, it will be challenging to
>>> guess these (Or does the discovery handle that?). However we could
>>> assume the default of "appending ".1.1" to the local IP which seems to
>>> be quite established. If however the discovery allows the listing up of
>>> AMSNetIds a device running on a given IP has, this would be awesome.
>>> br/>> I think KNX and CAN can be considered a bit simmilar regarding
>> what's a
>>> device and what's a ressouce. For KNX we currently consider the KNXNetIP
>>> Gateway the device (and therefore the connection string only has the
>>> coordinates to connect to that) and then the KNX Devices on the KNXBus
>>> are considered Resources which are found by a Browse operation.
>>> When thinking of KNX I would consider the Device resources sort of like
>>> with S7 ... the KNX Device on the KNX Bus is the DataBlock and the
>>> resource on the device is the address relative to that DataBlock. So if
>>> I want to read a parameter of a KNX device, then I add the parmeter
>>> coordinates to the knx address which makes up the field address in PLC4X.
>>> br/>> It looks like it should be possible to bring thhe CAN driver into
>>> supporting the same mechanisms here too.
>>> br/>> Chris <
>>> br/>> On 02.07.21 17:53, &##321;ukasz Dywicki wrote:
>>>> Hey Chris,
>>>> I do agree that browse and discovery is different and it might be either
>>>> expanded or narrowed.
>>>>
>>>> For example openHAB stuff I did based on CANopen driver does following:
>>>> - Discovery of CAN enabled interface
>>>> For linux it is quite straight cause you can use either a "net" part of
>>>> sysfs or even a dbus [1] in order to find interesting network nodes.
>>>> - Discvoery of CANopen nodes
>>>> Depending on strategy this could be a scan with predefined address range
>>>> or simply listen who talks to the bus. For CANopen scan is quite
>>>> straight, cause there is up to 127 nodes. You can submit a read out
>>>> request for well known register address, ie. manufacturer code.
>>>> Listening is also possible cause each node should (in theory) emit a
>>>> heartbeat.
>>>> - Discovery of CANopen fields readable or writable via driver.
>>>> This is scan of node registers within predefined address ranges or
>>>> ranges retrieved from initial node read out.
>>>> I don't have this part (nor didn't plan to work on it anytime soon).
>>>> This is probably a thing which is most interesting in case of data
>>>> acquisition.
>>>>
>>>> Process is following:
>>>> - [discovery] Physical interface [0..N]
>>>> - [discovery] Discovery connection string [1:1] (?)
>>>> - [discovery/browse] Finding accessible nodes [0..N]
>>>> - [browse] Fields for a specific node [0..N]
>>>>
>>>> Discovery of Beckhoff stuff I did works in a following way:
>>>> - Find network interfaces
>>>> - Send broadcast over network interface
>>>> - For each answered node
>>>> - Try setting up a ADS route (requires PLC username and password)
>>>> - Connect
>>>> - Retrieve symbols (I did not implement this)
>>>> Beckhoff uses a UDP for discovery and connection to PLC is over TCP thus
>>>> you can clearly see that there are two different transports used. I
>>>> believe KNX is same.
>>>> For Profinet-DCP what I managed to complete with Adrian is focused only
>>>> on two parts:
>>>> - Finding accessible profinet devices
>>>> Since they do answer with their own datas we can get their MAC which is
>>>> later on used to communicate with them. Initial mapping of DCP ident
>>>> response I did mapped that response to PlcStruct. Mainly because there
>>>> are multiple fields which can come, including device name, DHCP settings
>>>> or its ip address.
>>>>
>>>> Given the nature of node discovery it should by default stick with
>>>> callbacks and not request/response modes. Browse is something we briefly
>>>> spooked before, but it also fails into same category. We never know if
>>>> discovery or browse is finite or not. For that reason API should rather
>>>> assume later than earlier and let driver decide.
>>>> For address range queries we indeed can have a progress indication,
>>>> while for PN-DCP I don't think we should make any assumptions. I am not
>>>> quire sure how big PN network can be, so I'll let others speak.
>>>>
>>>> In my understanding, also based on above openHAB experiences, discovery
>>>> and browse APIs are kind of related, yet have different syntax.
>>>> For Beckhoff definitely we get some parts for transportUrl options such
>>>> as node AMS id which is advertised in identification response.
>>>> For AMS there is also an authentication, however I don't think we're
>>>> able to scrap this. In new versions I think there might be also a device
>>>> certificate signature.
>>>>
>>>> Anyhow, these would be my 0.02€ of input.
>>>>
>>>> Best,
>>>> Łukasz
>>>>
>>>> On 02.07.2021 17:02, Otto Fowler wrote:
>>>>> Some protocols the discovery process returns information about the
>> device
>>>>> (DCP, ENIP) , that might be important as well. How would this be
>> handled /
>>>>> modeled ?
>>>>>
>>>>> The idea here is that plc4x doesn’t store any of this right? You’d have
>> to
>>>>> handle the callback and store the information yourself?
>>>>>
>>>>> From: Christofer Dutz <[email protected]>
>>>>> <[email protected]>
>>>>> Reply: [email protected] <[email protected]> <
>> [email protected]>
>>
>>>>> Date: June 30, 2021 at 13:53:32
>>>>> To: [email protected] <[email protected]> <[email protected]>
>>>>> Subject: [DISCUSS] Discover and Browse API for PLC4J?
>>>>>
>>>>> Hi all,
>>>>>
>>>>> as you all know, I have threatened to bring API additions from plc4go
>>>>> br/>back to plc4j and now the time has come. But I want to invollve you
>> all
>>>>> br/>in it as we all have to live with it ;-) <
>>>>>
>>>>> So in general in go I did it the following way:
>>>>>
>>>>> - I split the topic up into "Discovery" and "Browse"
>>>>>
>>>>> "Browse" I'll focus on as soon ad "Discovery" is defined. In general
>>>>> br/>this should produce a stream of PlcBrowseEvents whichh provide a
>> stream
>>>>> br/>of PlcFField entries that can be used to query information. So this
>> is
>>>>> br/>inside an established connection to a device. It won't find devices
>>>>> br/>(However this term is slightly ambbiguous because in KNX for
>> example we
>>>>> br/>consider a KNX devices GrouupAddress an address, even if in the
>>>>> physical br/>world it actually iis a device)
>>>>>
>>>>> Discovery is the operation of finding out which devices we can connect
>>>>> br/>to. The result is a list of PlcDiscoveryEvents. These contaain:
>>>>> - protocolCode (like s7, modbus, ...)
>>>>> - transportCode (like tcp, udp, serial, ...)
>>>>> - transportUrl (which is the part after the "://" but before the "?"
>>>>> - option (which is a map of key-value pairs making up the part after
>> the
>>>>> br/>""?" .... the part which Lukazs hates so much ;-) )
>>>>>
>>>>> In go I extended the DriverManager to have a discovery function which
>>>>> br/>simply iterates over all drivers it knows and if they supporrt
>>>>> discovery, br/>the functionality is executed. <
>>>>>
>>>>> In the past we usually returned a Future and this became complete as
>>>>> br/>soon as the operation was finished. However Discovery can
>> ssometimes
>>>>> take br/>quite long. Some protocols support a direct ""Please all
>> devices
>>>>> br/>supporting this report to me"" functionality, but some we will need
>> to
>>>>> br/>potentially probe by trying all IP addreesses in a given range.
>>>>> Therefore br/>I think such a future approach is sub-ideal.
>>>>>
>>>>> In go I gave the discover function a callback argument where you set a
>>>>> br/>handler that gets notified directly as soon as a device is found. I
>> am
>>>>> a br/>bit unsure if this should be the way to go. <
>>>>>
>>>>> What I would love to do, would be to have the option to add one
>>>>> (possibly multiple) handlers to a DiscoveryRequest. Whenever something
>>>>> br/>is found, then every handler is called and at the end the resuult
>> is
>>>>> put br/>into a list. As soon as the operation is complete, thee user
>> could
>>>>> then br/>use this list, just like our normal read-resultts and be happy
>>>>> with it, br/>or he could use the callbacks if he wantts to be informed
>>>>> directly.
>>>>>
>>>>> Unfortunately the Futures are so totally all-or-nothing. I would love
>> to
>>>>> br/>return something that knows the progress of the operatiion. In a
>>>>> br/>Broadcast search it could simply be the progress towardd the
>> timeout or
>>>>> br/>when scanning an IP range it could be the percenntage of the
>> addresses
>>>>> br/>that were probed. This could help tools too show a nice progress on
>> the
>>>>> br/>auto-discovery. <
>>>>>
>>>>> What do you think? As I need this for my work on the PROFINET, I'm
>>>>> br/>working on this on the feature/profinet-chris branch. <
>>>>>
>>>>> Chris
>>>>>
>>
>
