Hi!
Enus Linden wrote:
moving this to the list. (I took the liberty of pasting this mail's
predecessor below as well)
Thanks, Enus :)
And good to see that we seem to be reaching some consensus.
Following are my attempts to answer the questions raised:
[...]
Similarly, the server can make any packet ackable (by setting the send
flag with the ack flag), and so we must ack it, otherwise the server
gets angry. We don't know which packet we will need to ack, we have to
determine that when we receive one.
Ok, so that seems to be part of what a connection needs to handle.
It would follow the pattern seen in e.g. smtplib, urllib2 (where the
Request is the message). And most network modules actually have a
connection object, such as ftplib, nntplib, gopherlib etc. Not all
have message classes though because if it's just a file you send, then
there is no need to encapsulate this in a separate object.
Here also is another example of message objects in email:
http://docs.python.org/lib/message-objects.html
I'm starting to like the idea of a Message. Maybe this Message could be
only the payload of the message, with a Packet class (I think you have
suggested this) having the other necessary fields. Then, a serializer
can serialize the packet and a net framework can send it on a connection.
Right, if you do ISerialization(packet) you will get the packet
serializer which in turn can do a ISerialization(msg) to get the message
serializer. It would then add everything togethet to one binary blob.
Also be aware that connections will change during the lifetime of the
client. You don't have a single udp connection. You communication with
neighboring sims, you may switch regions, etc. This causes you to create
a new connection to send on.
On a higher level I was thinking about having different Region objects
which all have their UDP and HTTP/Cap connection.
But also remember that for udp messages you don't need a connection, you
can simply send the message to any given host. So, it may be extra to
have a connection class doing such a thing because you can use a single
connection to send and receive on. The target we are sending to changes,
but we don't need to change the sockets or anything.
To me that would mean I need to know that it's a socket connection. On a
higher level though I might not want or need to know. It's more a
logical connection to that region, whatever the implementation.
*Current Design: *
This is taken from
http://wiki.secondlife.com/wiki/Pyogp/Documentation/Specification/pyogp.lib.base
messenger = MessageSystem()
host = Host('sim_ip', 'sim_port') #note: these aren't true values, of
course
messenger.new_message("UseCircuitCode")
messenger.next_block("CircuitCode")
messenger.add_data('Code', circuit_code, \
MsgType.MVT_U32)
messenger.add_data('SessionID', \
uuid.UUID(session_id), \
MsgType.MVT_LLUUID)
messenger.add_data('ID', \
uuid.UUID(agent_id), \
MsgType.MVT_LLUUID)
messenger.send_message(host)
_Explanation:_
The thing to know about the current design is that it is encapsulated
into a MessageSystem. Everything from building, reading, sending, and
receiving messages all occurs in the Message System (though each of
the sets of functionality are performed by other objects that the
system HAS).
I think this is a quite good explanation where we differ. As said
before this feels very uncommon in the python world to me.
One concern I also have is all the sort of global state in these long
living objects. It doesn't need to but might lead to problems with
threading or coroutines. I would try to keep locking zones as small as
possible. I also might think of this scenario with coroutines:
- You create and send a message in coroutine A
- Sending blocks for whatever reasons
- Coroutine B gets activated, creates a message and sends it. Maybe
with the same message system. It also blocks on sending.
- current_msg is now message of B and this is what A sends.
So this would mean that you need to separate message system per
thread. This also means though that it's only one host you connect to
per message system and thus the host could be in the constructor as
it's quite fixed then.
Yea, I do agree that it is confusing having the message remembered in
state by the system, builders, and readers. I'm starting to like the
idea of outputting a Message that the user adds data to and sends. Maybe
the Message System could remain as the connection you send through and
receive on, which automatically serializes sending packets and
deserializes receiving packets, keep track of all acks and such.
Seems like it's basically doing the work already so I think it makes
sense. Not sure what it should be called but we can discuss this later.
But from a logical view on it it seems mostly in there although I would
move some stuff to the Packet class and it's serializer (what you also
suggested I think).
In Python you don't care about this. If there is a 1 you mean 1 and
you don't care how it's sent over the wire on a lower level of the
system. Yes, you might run into a problem if you don't know the type
but in my experience this rarely actually leads into problems. Having
no type also makes coding faster as you have to type less and you
don't have to consult the documentation.
So let's get rid of the type-checking, I'm fine with that. It IS just
extra junk I don't feel like typing anyway :)
Cool :-)
There might then also be a MessageDispatcher which does the same so it
knows over which channel to send this message (I guess for XML
messages it's simply the cap we have and we do cap.POST(data).
Right, so I'm thinking the Message System could do all this. Maybe the
Message System could be the factory and dispatcher, with all messaging
being sent and received going through it (but BUILDING messages not
going through this).
In my branch I made the factory a utility as well but that's just a
proposal. If we see it as MessageSystem and not as connection I guess it
can also implement the message creation. If we see it as a connection
IMHO it's strange to say that a connection produces messages. I would
more think that a connection really just cares about serializing and
deserializing and keeping track of state (acks) but not about how to
create new messages.. They either get created by the system who wants to
send it via that connection or they get somehow created on the other
side of the network and it just converts them back from binary from to
Python structures.
Maybe not too important though.
[...]
message (up to 500 I believe) will have its own unique class that
will initialize the data attributes.
We would start with the ones we actually use in the library. If
somebody needs to use an additional one he can still use the more low
level version (Message('name', Block(...), Block(...)) ).
We also have to look at every message in the protocol spec anyway and
define it there in detail. When we do this we can go along and define
them in code as well. I am also willing to do that.
A pro here would be that you can put default values in the class so
that you don't have to specify all parameters.
When receiving a message you would have the possibility to attach an
event handler directly to that class using ZCA.
Another pro is that the user of this level doesn't have to know about
blocks and the sequence of these. She only needs to know about the
actual data to be passed in.
Well, we can do this with ZCA without deriving a class for each message.
We can have them all implement an interface and register them with a
certain name. This way we don't have to write each individual class, but
can have a generic Message which can handle them all, with handlers. The
default data can be added in by the Message Factory (which looks into
the template and fills in the message with default values).
I guess this is the problem then. If we have a single Message which
builds itself (add_block methods), we cannot write a Message class which
tests that the data being added is correct and expected. Unless the
message itself is a UDP message derivation and can look at the message
template itself and do the checking.
I would propose that for now we don't think about such high level
classes. You also raised the issue yesterday of how those are
deserialized and this probably needs some thinking.
If we have a structure like Message(Block('name', param1=10, param2=20))
it's maybe readable enough for now.
We might discuss which layers to add later then and such classes should
probably also be part of a separate package.
I might write another mail with some proposals on how to maybe get a
release out and what we should do before that (like cleaning up what we
have, doing logging, exceptions etc.)
Some Connection class which seems to me similar to your message system
and circuits.
BTW, what actually is a circuit? Is it a connection to a region? Or
can you have many circuits to one region? This part of the protocol is
not that clear to me right now. We probably should write it down if it
isn't somewhere (but it should be part of the spec at some point anyway).
A circuit is a UDP connection. So, it is a UNIQUE connection to ip
address and port combination. Can only have 1 circuit for each ip and
port combination.
Sounds also like some connection then. I wonder then if MessageSystem
and Circuit could be merged somehow and the circuit bit is just an
implementation detail of the Connection class. But I am not into that
code so that might be wrong.
Thanks for your work, I'm starting to see where we can improve things.
I'll start writing down my new proposal and see if we can get something
working.
Thanks for writing down that proposal! I will also be looking at your
branch once you think it's lookworthy ;-) And we should keep the
discussison going I think to get the best result we can.
PS: I don't like the idea of ZCAifying things like the dictionary just
so that we can register them with ZCA as a global utility. It is an
extra abstraction that is confusing and the reasoning not clearly seen.
Something else we can do?
Well, this seemed to me as one of the main examples of using utilities
because:
- they are singletons (and thus global)
- zope uses similar utilities a lot (like MessageCatalog/Translation)
- they only get instantiated once and the reading is only done once
- you don't need to pass them around
- you can still change them and override the utility if you think you
need to.
Another approach would be to use a module level variable which holds the
instantiated dictionary. You could import it with
import message_template
tmpldict = message_template.tmpldict
and you could also still override it in your own code by "patching" the
module.
But I would prefer to only use one variant because otherwise it really
gets confusing and you never know which variant is used now (and the
deserializer is also a utility btw. for the same reason of being simply
a global service).
Maybe it needs to be documented more? (I guess the answer to such
question is never No ;-) ).
Speaking of docs, I will try to put some more stuff on the wiki this
weekend, like explaining how to use grok, how to use exceptions, the
logger etc.
cheers,
Christian
--
Christian Scholz Homepage: http://comlounge.net
COM.lounge blog: http://mrtopf.de/blog
Luetticher Strasse 10 Skype: HerrTopf
52064 Aachen Video Blog: http://comlounge.tv
Tel: +49 241 400 730 0 E-Mail [EMAIL PROTECTED]
Fax: +49 241 979 00 850 IRC: MrTopf, Tao_T
neue Show: TOPFtäglich (http://mrtopf.de/blog/category/topf-taglich/)
_______________________________________________
Click here to unsubscribe or manage your list subscription:
https://lists.secondlife.com/cgi-bin/mailman/listinfo/pyogp
