Re: Serial port Connection

2020-06-30 Thread Christofer Dutz
Hi Niclas,

Now your're borrowing my reasoning to use PLC4X over OPC UA to argument for 
Serial communication ... I like it ;-)
Yeah, I know that industrial hardware tends to be run a tad longer than a 
typical IPhone ... and I agree ... we need to get serial in shape.
I did notice with the Firmata driver that there will be issues as this is 
actually a serial protocol, which is stateful 
(You need to configure the connection in order to use it)
Here I encountered the limitation of connection-to-port linking for the first 
time for which I still don't really have an answer.

> Not sure what you mean here. ModBus/RTU has a one byte Address and a
>Checksum, and those are dropped in Modbus/TCP and that relies on the
>delivery/checksum of TCP/IP.

I think you initially mentioned, that we have the device-address in the 
connection string, which makes it necessary to connect and disconnect all the 
time.
My proposal was to leave that option in the connection string, but to treat it 
as a "default-device-address" which is used if the actual address doesn't have 
it (Typical TCP case) ... then we would add a second set of address types that 
contain an additional device-address prefix. So you could simply connect to the 
serial version without providing the device-address and then use the extended 
address format for resources and pass in the additional device-address.

That a little clearer?

Chris


Am 30.06.20, 09:12 schrieb "Niclas Hedhman" :

On Tue, Jun 30, 2020 at 2:31 PM Christofer Dutz 
wrote:

> But can we assume that only one protocol is handled over one serial
> interface? So not multiple Modbus devices sharing with multiple S5 devices
> ...
>

So, I have never seen anyone trying multiple protocols on the same port, so
that is a safe assumption. Even if I think there are devices that can
auto-sense Modbus and Bacnet, I doubt anyone will ever want to do both.



> Because in this case I guess we could refactor things a bit.
>

And I totally agree that as little changes as possible should be attempted,
so I won't suggest a rewrite, don't worry ;-)

Perhaps to have an optional device-address on the connection level which
> will act as a default and to extend or provide a second set of addresses,
> that have the device-address within. So you could either use the
> device-address together with the smaller field addresses (Mainly for TCP
> connections) and these would inherit the default device-address or you
> could use the bigger ones where you provide the device-address for every
> field.
>

Not sure what you mean here. ModBus/RTU has a one byte Address and a
Checksum, and those are dropped in Modbus/TCP and that relies on the
delivery/checksum of TCP/IP.


> I wouldn’t want to change the naming however ... you could consider in an
> extended serial modbus scenario (If we changed the serial part the way you
> proposed), that the connection is from the software to the port ...Which
> would be true. And most of the other drivers in PLC4X actually do more 
have
> a Connection-oriented communication form. I guess this is due to the fact
> that we concentrated on the SCADA-Level protocols first and now are
> gradually stepping deeper into the fieldbus area.
> I think only the EIP protocol is also somewhat connection-less, but S7,
> OPC UA, AMS/ADS, ... definitely are.
>

Interesting. I have very little experience "higher up" and perhaps there
are enough reasons to make abstractions different for stateless (or packet)
protocol vs those that are stateful. The compromise now is that stateless
pretends to be stateful, and a cycle of "open->close" takes place, very
much similar to HTTP/1.0 being stateless over the stateful TCP. I am not
100% confident what is the best way forward, so I'll come back to that
later. I think in the short term, I can survive because my current need is
very small (reading 5-10 values from each of 3-5 devices) and it is
hourly-relevant data.
In the process, I will try to formulate ideas for changes...

I know that serial comms are slowly dying out, but the amount of stuff
already out there and lifespans of decades, it will take a while before it
is all gone.

// Niclas



Re: Serial port Connection

2020-06-30 Thread Niclas Hedhman
On Tue, Jun 30, 2020 at 2:31 PM Christofer Dutz 
wrote:

> But can we assume that only one protocol is handled over one serial
> interface? So not multiple Modbus devices sharing with multiple S5 devices
> ...
>

So, I have never seen anyone trying multiple protocols on the same port, so
that is a safe assumption. Even if I think there are devices that can
auto-sense Modbus and Bacnet, I doubt anyone will ever want to do both.



> Because in this case I guess we could refactor things a bit.
>

And I totally agree that as little changes as possible should be attempted,
so I won't suggest a rewrite, don't worry ;-)

Perhaps to have an optional device-address on the connection level which
> will act as a default and to extend or provide a second set of addresses,
> that have the device-address within. So you could either use the
> device-address together with the smaller field addresses (Mainly for TCP
> connections) and these would inherit the default device-address or you
> could use the bigger ones where you provide the device-address for every
> field.
>

Not sure what you mean here. ModBus/RTU has a one byte Address and a
Checksum, and those are dropped in Modbus/TCP and that relies on the
delivery/checksum of TCP/IP.


> I wouldn’t want to change the naming however ... you could consider in an
> extended serial modbus scenario (If we changed the serial part the way you
> proposed), that the connection is from the software to the port ...Which
> would be true. And most of the other drivers in PLC4X actually do more have
> a Connection-oriented communication form. I guess this is due to the fact
> that we concentrated on the SCADA-Level protocols first and now are
> gradually stepping deeper into the fieldbus area.
> I think only the EIP protocol is also somewhat connection-less, but S7,
> OPC UA, AMS/ADS, ... definitely are.
>

Interesting. I have very little experience "higher up" and perhaps there
are enough reasons to make abstractions different for stateless (or packet)
protocol vs those that are stateful. The compromise now is that stateless
pretends to be stateful, and a cycle of "open->close" takes place, very
much similar to HTTP/1.0 being stateless over the stateful TCP. I am not
100% confident what is the best way forward, so I'll come back to that
later. I think in the short term, I can survive because my current need is
very small (reading 5-10 values from each of 3-5 devices) and it is
hourly-relevant data.
In the process, I will try to formulate ideas for changes...

I know that serial comms are slowly dying out, but the amount of stuff
already out there and lifespans of decades, it will take a while before it
is all gone.

// Niclas


Re: Serial port Connection

2020-06-30 Thread Christofer Dutz
Hi all,

and thank you Niclas for your very detailed post.

Indeed I think you made me realize something I didn't realize till now: You can 
connect multiple devices to a serial Interface.
For me, from the first zero-modem-cable I welded to play command and conquer 
against a friend, serial interfaces were only one-to-one connections.
I guess that's why I built things the way I built them. If I had known what you 
now made me know, I agree at least the handling of the serial port I would have 
done differently.

Also I have to admit, that the serial communication option was not handled as a 
first-class citizen by me and probably others here as I at least only have one 
serial-capable PLC but I don't even have a serial to usb-c adapter (Should 
really get one ...) So it was always a rather theoretical construct. Also I 
don't recall people reporting to have used it before.  

So I agree we should address your findings asap.

But can we assume that only one protocol is handled over one serial interface? 
So not multiple Modbus devices sharing with multiple S5 devices ... 
Because in this case I guess we could refactor things a bit. 

Perhaps to have an optional device-address on the connection level which will 
act as a default and to extend or provide a second set of addresses, that have 
the device-address within. So you could either use the device-address together 
with the smaller field addresses (Mainly for TCP connections) and these would 
inherit the default device-address or you could use the bigger ones where you 
provide the device-address for every field.

I wouldn’t want to change the naming however ... you could consider in an 
extended serial modbus scenario (If we changed the serial part the way you 
proposed), that the connection is from the software to the port ...Which would 
be true. And most of the other drivers in PLC4X actually do more have a 
Connection-oriented communication form. I guess this is due to the fact that we 
concentrated on the SCADA-Level protocols first and now are gradually stepping 
deeper into the fieldbus area. 
I think only the EIP protocol is also somewhat connection-less, but S7, OPC UA, 
AMS/ADS, ... definitely are.

I know that in a case like PLC4X where we want to map all sorts of 
communication variants under one API, we will never get it that it's a perfect 
match for all protocols, especially if it's the projects goal to have one 
shared API. 

But again ... thank you for your detailed input ... this is some very important 
information and it even made a lot of things clearer to me which customers in 
the past said and I simply didn't understand ;-)


Chris


Am 30.06.20, 06:12 schrieb "Cesar Garcia" :

Hi Niclas,

You are right on all points. I have not tested the Modbus serial driver,
but I am very interested that it works on Multidrop connections.

Version 0.7.0 is based on Mspec, so a proposal could be:

1. Modify the PLCField to include the UID of the device within the frame
from the address of the Item.

2. The default UID must be the one inserted in the Connection String.

3. Evaluate the queue of requests, I think the standard allows a single
frame depending on the connection (Half / Full Duplex).

It should not be very laborious, I can keep an eye on it.

Here Chris can guide us if it is the way.

Best regards,

El lun., 29 jun. 2020 a las 22:59, Niclas Hedhman ()
escribió:

> Ok, let's dissect this...
>
> 1. There are many devices connected to a serial ModBus port.
>
> 2. ModBus (and every other serial protocol I have dealt with) are not
> "connection" oriented, but packets, most of the time very small. The
> "connection" metaphor in PLC4X for non-TCP comms is actually quite 
poor.[1]
>
> 3. All serial protocols (that I worked with) have at least the destination
> present in the protocol packet itself. When some of these serial protocols
> were "wrapped" for modern TCP/IP communications, an ambiguity surfaced. 
And
> in case of ModBus, the device address was simply defaulted to 1 (although 
I
> think some gateways can handle many, and perhaps some TCP devices will map
> out more than one ModbUs device, but I have no experience in that)
>
>
> So, as things are right now, and if I didn't misunderstood something
> (totally possible), is that one is required to "open->->close" the
> PLC4X connection for each device that I want to scan/read, as I think the
> device address (1 byte in case of ModBus) is part of the connection 
string.
> The biggest system I have ever been involved with was just under 1000 PLCs
> (granted, not modbus) with 30,000 data points readable in total. It feels
> "s wrong" to have "open->send->receive->close" operation for all of
> those, AND that the timing on that particular system was that for optimal
> performance, exactly 1 byte of delay (related to half-duplex 

Re: Serial port Connection

2020-06-29 Thread Cesar Garcia
Hi Niclas,

You are right on all points. I have not tested the Modbus serial driver,
but I am very interested that it works on Multidrop connections.

Version 0.7.0 is based on Mspec, so a proposal could be:

1. Modify the PLCField to include the UID of the device within the frame
from the address of the Item.

2. The default UID must be the one inserted in the Connection String.

3. Evaluate the queue of requests, I think the standard allows a single
frame depending on the connection (Half / Full Duplex).

It should not be very laborious, I can keep an eye on it.

Here Chris can guide us if it is the way.

Best regards,

El lun., 29 jun. 2020 a las 22:59, Niclas Hedhman ()
escribió:

> Ok, let's dissect this...
>
> 1. There are many devices connected to a serial ModBus port.
>
> 2. ModBus (and every other serial protocol I have dealt with) are not
> "connection" oriented, but packets, most of the time very small. The
> "connection" metaphor in PLC4X for non-TCP comms is actually quite poor.[1]
>
> 3. All serial protocols (that I worked with) have at least the destination
> present in the protocol packet itself. When some of these serial protocols
> were "wrapped" for modern TCP/IP communications, an ambiguity surfaced. And
> in case of ModBus, the device address was simply defaulted to 1 (although I
> think some gateways can handle many, and perhaps some TCP devices will map
> out more than one ModbUs device, but I have no experience in that)
>
>
> So, as things are right now, and if I didn't misunderstood something
> (totally possible), is that one is required to "open->->close" the
> PLC4X connection for each device that I want to scan/read, as I think the
> device address (1 byte in case of ModBus) is part of the connection string.
> The biggest system I have ever been involved with was just under 1000 PLCs
> (granted, not modbus) with 30,000 data points readable in total. It feels
> "s wrong" to have "open->send->receive->close" operation for all of
> those, AND that the timing on that particular system was that for optimal
> performance, exactly 1 byte of delay (related to half-duplex modem
> direction switching) between receiving the last byte of previous packet
> before the first byte of the new packet. Without the "Serial Port"
> abstraction, with a queue of requests to send, such timing is not trivial.
>
> What I think has "gone wrong" here is that the "serial port" is an
> interface, pretty much like "eth0" and should be handled "further down" but
> since the operating system has no support for it, it should have been in
> PLC4X's architecture. I also think that the "connection" metaphor is poor,
> and a "packet" metaphor would have been much better throughout PLC4X
> regardless whether it is TCP/IP, UDP or serial. The actual TCP connection
> when needed should be handled under the hood, rather invisibly to the user.
> As I said before, I understand that this is not something that can be
> changed, just like that... Only expressing how I think it should have been
> done.
>
> I will try to work with what we have, and dig into the details on the
> serial port side, as that's where I have plenty of experience and I get the
> impression that it isn't the strength of the community. I am sure you will
> hear a lot from me going forward, as I have decided to put PLC4X into my
> commercial product, replacing the existing j2mod.
>
>
> Niclas
>
> [1] It is quite funny that the serial port is natively a stream, and we map
> packets onto it. TCP/IP is natively packets, which is then made into a
> stream, which we then map packets onto the stream. Ideally, UDP is fit for
> purpose for PLC protocols, but for some unknown reason few are.
>
>
>
>
> On Mon, Jun 29, 2020 at 4:00 PM Christofer Dutz  >
> wrote:
>
> > Hi all,
> >
> > sorry for joining in the party late ... but hopefully not too late.
> >
> > Regarding making the serial port part of the connection string ... I have
> > to admit that right now I couldn't imagine how to not have it in there
> and
> > not over-complicate things for the users. Yes, if you create a connection
> > this will have exclusive handle to the port, but this is actually an
> > important thing to have. In contrast to network devices that can handle
> > multiple connections in parallel ... a serial port is ... well ...
> serial.
> > Havin asynchronous access to one serial port from multiple areas in an
> > application sound like a bad Idea.
> >
> > However you could always use the connection pool ... in this case the
> pool
> > would have one connection open and could share this (serially) with
> > multiple parts of an application. Here a part would request the
> connection,
> > do its work and then give the connection back to the pool where another
> > part of the application could use it. This way I think we have the ideal
> > tradeoff between sharing a connection and not having to be concerned
> about
> > breaking the serial operation of the serial-port communication.
> 

Re: Serial port Connection

2020-06-29 Thread Niclas Hedhman
Ok, let's dissect this...

1. There are many devices connected to a serial ModBus port.

2. ModBus (and every other serial protocol I have dealt with) are not
"connection" oriented, but packets, most of the time very small. The
"connection" metaphor in PLC4X for non-TCP comms is actually quite poor.[1]

3. All serial protocols (that I worked with) have at least the destination
present in the protocol packet itself. When some of these serial protocols
were "wrapped" for modern TCP/IP communications, an ambiguity surfaced. And
in case of ModBus, the device address was simply defaulted to 1 (although I
think some gateways can handle many, and perhaps some TCP devices will map
out more than one ModbUs device, but I have no experience in that)


So, as things are right now, and if I didn't misunderstood something
(totally possible), is that one is required to "open->->close" the
PLC4X connection for each device that I want to scan/read, as I think the
device address (1 byte in case of ModBus) is part of the connection string.
The biggest system I have ever been involved with was just under 1000 PLCs
(granted, not modbus) with 30,000 data points readable in total. It feels
"s wrong" to have "open->send->receive->close" operation for all of
those, AND that the timing on that particular system was that for optimal
performance, exactly 1 byte of delay (related to half-duplex modem
direction switching) between receiving the last byte of previous packet
before the first byte of the new packet. Without the "Serial Port"
abstraction, with a queue of requests to send, such timing is not trivial.

What I think has "gone wrong" here is that the "serial port" is an
interface, pretty much like "eth0" and should be handled "further down" but
since the operating system has no support for it, it should have been in
PLC4X's architecture. I also think that the "connection" metaphor is poor,
and a "packet" metaphor would have been much better throughout PLC4X
regardless whether it is TCP/IP, UDP or serial. The actual TCP connection
when needed should be handled under the hood, rather invisibly to the user.
As I said before, I understand that this is not something that can be
changed, just like that... Only expressing how I think it should have been
done.

I will try to work with what we have, and dig into the details on the
serial port side, as that's where I have plenty of experience and I get the
impression that it isn't the strength of the community. I am sure you will
hear a lot from me going forward, as I have decided to put PLC4X into my
commercial product, replacing the existing j2mod.


Niclas

[1] It is quite funny that the serial port is natively a stream, and we map
packets onto it. TCP/IP is natively packets, which is then made into a
stream, which we then map packets onto the stream. Ideally, UDP is fit for
purpose for PLC protocols, but for some unknown reason few are.




On Mon, Jun 29, 2020 at 4:00 PM Christofer Dutz 
wrote:

> Hi all,
>
> sorry for joining in the party late ... but hopefully not too late.
>
> Regarding making the serial port part of the connection string ... I have
> to admit that right now I couldn't imagine how to not have it in there and
> not over-complicate things for the users. Yes, if you create a connection
> this will have exclusive handle to the port, but this is actually an
> important thing to have. In contrast to network devices that can handle
> multiple connections in parallel ... a serial port is ... well ... serial.
> Havin asynchronous access to one serial port from multiple areas in an
> application sound like a bad Idea.
>
> However you could always use the connection pool ... in this case the pool
> would have one connection open and could share this (serially) with
> multiple parts of an application. Here a part would request the connection,
> do its work and then give the connection back to the pool where another
> part of the application could use it. This way I think we have the ideal
> tradeoff between sharing a connection and not having to be concerned about
> breaking the serial operation of the serial-port communication.
>
> And yes: We could use some documentation on the connection pool ... and
> the scraper ;-/
>
> Perhaps I'll whip up a little documentation on the connection-pool as I
> just recently had to find out how to use it myself ;-)
>
> Chris
>
>
> Am 25.06.20, 11:41 schrieb "Julian Feinauer" <
> j.feina...@pragmaticminds.de>:
>
> Hey,
>
> yes, its in the module scraper or plc-scraper.
> Its easily configurable by yaml, json or programmatically.
>
> Julian
>
> Am 25.06.20, 10:30 schrieb "Niclas Hedhman" :
>
> Thanks
>
> Another question; is there any "loop" system that just
> continuously reads
> until I tell it to stop?
>
> On Thu, Jun 25, 2020, 15:16 Julian Feinauer <
> j.feina...@pragmaticminds.de>
> wrote:
>
> > Hi Nic,
> >
> > I have to dig in a bit later 

Re: Serial port Connection

2020-06-29 Thread Christofer Dutz
Hi all,

sorry for joining in the party late ... but hopefully not too late.

Regarding making the serial port part of the connection string ... I have to 
admit that right now I couldn't imagine how to not have it in there and not 
over-complicate things for the users. Yes, if you create a connection this will 
have exclusive handle to the port, but this is actually an important thing to 
have. In contrast to network devices that can handle multiple connections in 
parallel ... a serial port is ... well ... serial. Havin asynchronous access to 
one serial port from multiple areas in an application sound like a bad Idea. 

However you could always use the connection pool ... in this case the pool 
would have one connection open and could share this (serially) with multiple 
parts of an application. Here a part would request the connection, do its work 
and then give the connection back to the pool where another part of the 
application could use it. This way I think we have the ideal tradeoff between 
sharing a connection and not having to be concerned about breaking the serial 
operation of the serial-port communication.

And yes: We could use some documentation on the connection pool ... and the 
scraper ;-/

Perhaps I'll whip up a little documentation on the connection-pool as I just 
recently had to find out how to use it myself ;-)

Chris


Am 25.06.20, 11:41 schrieb "Julian Feinauer" :

Hey,

yes, its in the module scraper or plc-scraper.
Its easily configurable by yaml, json or programmatically.

Julian

Am 25.06.20, 10:30 schrieb "Niclas Hedhman" :

Thanks

Another question; is there any "loop" system that just continuously 
reads
until I tell it to stop?

On Thu, Jun 25, 2020, 15:16 Julian Feinauer 

wrote:

> Hi Nic,
>
> I have to dig in a bit later to give you good answers but AFAIR the
> connection can be left open and configuration should be as you wrote 
it.
> I also did some tests with it but that was some months ago...
>
> Hope to have some more information later.
>
> Julian
>
> Am 25.06.20, 05:58 schrieb "Niclas Hedhman" :
>
> Hi,
> IIUIC, the connection string for Serial Modbus would be something 
like;
>
>  modbus:serial:/dev/ttyS0?unit-identifier=7
>
> assuming I want to communicate with device with address 7.
>
> That would mean that I have one connection per device I talk to 
over a
> single serial port. Is that correct? (Personally, I would not 
made the
> device part of the connection abstraction, but realize that it is 
a bit
> late for that)
>
> Are there any issues here, regarding which connection has access 
to the
> port, or do I need to open and close the connection on each set of
> messages?
>
> Couldn't find info about this in the documentation.
>
>
> The serial port page is missing information on the configuration 
(data,
> parity, stop,...). The source code doesn't have the
> @ConfigurationParameter
> annotations, so will they still work or do I need to add that
> programmatically somehow?
>
>
> Cheers
> Niclas
>
>




Re: Serial port Connection

2020-06-25 Thread Julian Feinauer
Hey,

yes, its in the module scraper or plc-scraper.
Its easily configurable by yaml, json or programmatically.

Julian

Am 25.06.20, 10:30 schrieb "Niclas Hedhman" :

Thanks

Another question; is there any "loop" system that just continuously reads
until I tell it to stop?

On Thu, Jun 25, 2020, 15:16 Julian Feinauer 
wrote:

> Hi Nic,
>
> I have to dig in a bit later to give you good answers but AFAIR the
> connection can be left open and configuration should be as you wrote it.
> I also did some tests with it but that was some months ago...
>
> Hope to have some more information later.
>
> Julian
>
> Am 25.06.20, 05:58 schrieb "Niclas Hedhman" :
>
> Hi,
> IIUIC, the connection string for Serial Modbus would be something 
like;
>
>  modbus:serial:/dev/ttyS0?unit-identifier=7
>
> assuming I want to communicate with device with address 7.
>
> That would mean that I have one connection per device I talk to over a
> single serial port. Is that correct? (Personally, I would not made the
> device part of the connection abstraction, but realize that it is a 
bit
> late for that)
>
> Are there any issues here, regarding which connection has access to 
the
> port, or do I need to open and close the connection on each set of
> messages?
>
> Couldn't find info about this in the documentation.
>
>
> The serial port page is missing information on the configuration 
(data,
> parity, stop,...). The source code doesn't have the
> @ConfigurationParameter
> annotations, so will they still work or do I need to add that
> programmatically somehow?
>
>
> Cheers
> Niclas
>
>



Re: Serial port Connection

2020-06-25 Thread Niclas Hedhman
Thanks

Another question; is there any "loop" system that just continuously reads
until I tell it to stop?

On Thu, Jun 25, 2020, 15:16 Julian Feinauer 
wrote:

> Hi Nic,
>
> I have to dig in a bit later to give you good answers but AFAIR the
> connection can be left open and configuration should be as you wrote it.
> I also did some tests with it but that was some months ago...
>
> Hope to have some more information later.
>
> Julian
>
> Am 25.06.20, 05:58 schrieb "Niclas Hedhman" :
>
> Hi,
> IIUIC, the connection string for Serial Modbus would be something like;
>
>  modbus:serial:/dev/ttyS0?unit-identifier=7
>
> assuming I want to communicate with device with address 7.
>
> That would mean that I have one connection per device I talk to over a
> single serial port. Is that correct? (Personally, I would not made the
> device part of the connection abstraction, but realize that it is a bit
> late for that)
>
> Are there any issues here, regarding which connection has access to the
> port, or do I need to open and close the connection on each set of
> messages?
>
> Couldn't find info about this in the documentation.
>
>
> The serial port page is missing information on the configuration (data,
> parity, stop,...). The source code doesn't have the
> @ConfigurationParameter
> annotations, so will they still work or do I need to add that
> programmatically somehow?
>
>
> Cheers
> Niclas
>
>


Re: Serial port Connection

2020-06-25 Thread Julian Feinauer
Hi Nic,

I have to dig in a bit later to give you good answers but AFAIR the connection 
can be left open and configuration should be as you wrote it.
I also did some tests with it but that was some months ago...

Hope to have some more information later.

Julian

Am 25.06.20, 05:58 schrieb "Niclas Hedhman" :

Hi,
IIUIC, the connection string for Serial Modbus would be something like;

 modbus:serial:/dev/ttyS0?unit-identifier=7

assuming I want to communicate with device with address 7.

That would mean that I have one connection per device I talk to over a
single serial port. Is that correct? (Personally, I would not made the
device part of the connection abstraction, but realize that it is a bit
late for that)

Are there any issues here, regarding which connection has access to the
port, or do I need to open and close the connection on each set of messages?

Couldn't find info about this in the documentation.


The serial port page is missing information on the configuration (data,
parity, stop,...). The source code doesn't have the @ConfigurationParameter
annotations, so will they still work or do I need to add that
programmatically somehow?


Cheers
Niclas