Hi Adriano,
On Fri, Jun 10, 2011 at 11:03 PM, Adriano Monteiro Marques <
[email protected]> wrote:
> Hi,
>
> On Jun 10, 2011, at 11:34 AM, Zhongjie Wang wrote:
>
> Hi Guys,
>
> On Fri, Jun 10, 2011 at 10:02 PM, Zubair Nabi <[email protected]>wrote:
>
>> Hi Guys,
>>
>>
>> If the token is sent with the message in plain text, then someone may
>>>> perform a wire-tapping attack.
>>>> And if the two party in communication has already had a handshake
>>>> progress and figured out a symmetric key
>>>> for current session, then we don't need the token i guess, the symmetric
>>>> key itself is enough for the encryption and authentication purpose.
>>>> It's OK to keep agentID as you said.
>>>> Hm... Zubair helped me to set that there, let's just confirm with him
>>>> that we don't need it there. Your consideration is fair, but I'm not sure
>>>> this is the only purpose the token is there.
>>>
>>> The token is also supposed to reflect the state of the agent binary. For
>> example, if the binary is updated, the symmetric key would remain the same
>> but the token would change.
>> And we aren't going to send messages in plain text. We're going to encrypt
>> each message.
>>
>> Yes, in the Event message we don't need to send back the unnecessary
>>> fields.
>>> But since they (redirectLink, htmlResponse, htmlMedia) are "optional" in
>>> the Detail structure,
>>> they can be omitted while sending.
>>> Because when we send back Detail in Event message, we need to clone it
>>> and only send the necessary fields.
>>> Ok, sounds good. Can you get them fixed, and let the others know about
>>> your fix?
>>
>> +1
>>
>> So that's what I propose at the beginning.
>>> All messages will be instantiate as a common message type.
>>> It has a type field and some option fields.
>>> The receiver will instantiate it as a common ICMMessage for example,
>>> then read the type field, and choose which optional field to read.
>>> The optional fields which doesn't exists in a message don't take any
>>> spaces.
>>> e.g.
>>> message ICMMessage {
>>> required int64 agentID = 1;
>>> required int32 type = 2;
>>> optional AssignTask assignTask = 100;
>>> optional UpgradeToSuper upgradeToSuper = 101;
>>> optional VersionUpdate versionUpdate = 102;
>>> // extensions 100 to max;
>>> }
>>> Did you try this approach? Not sure it will work with protocol buffers.
>>
>>
>> Luis and I discussed this a while back and we came to the conclusion that
>> this won't be work. In case of p2p communication, we will have to send an
>> additional message.
>> For example our send would look something like send(string type, string
>> message)
>> At the receiving end, this "type" would tell us which type of Protobuf
>> message this is.
>>
>
> Hi Zubair, I'm using the way you said now. But I think what I mentioned
> also works. I just tested that. ;)
> Here's the .proto file and test.py I used.
>
>
> Oh, now I understood your approach. We would have one type of message that
> would encapsulate all others, right?
> The problem with this approach is that the message gets to big and slower
> to process. Though it works, protobuf is got to parse the whole thing, and
> it will place several markers in the message to indicate that the field is
> empty. This can escalate to a few hundreds easily as we extend our app.
>
>
I was worrying about the message size. But after tested, I found even I
added more optional fields to that message. The size didn't grow, as I have
printed out. (I added ExtendedMessageType up to 10).
I think it's because the protobuf message use *unique tag number* to
indicate the field.
If an optional field doesn't exist, it will not take any space in the
message.
> test.proto:
>
> // this is a sample message sent from the aggregator to the agent
> message ICMMessage {
> required int32 type = 1;
>
> optional AssignTask assignTask = 100;
> optional UpgradeToSuper upgradeToSuper = 101;
> optional ExtendedMessageType1 em1 = 102;
> optional ExtendedMessageType2 em2 = 103;
> optional ExtendedMessageType3 em3 = 104;
> // extensions 100 to max;
> }
>
> message AssignTask {
> required int32 foo = 1;
> required int32 bar = 2;
> }
>
> message UpgradeToSuper {
> required int32 foo = 1;
> required int32 bar = 2;
> }
>
> message ExtendedMessageType1 {
> required int32 foo = 1;
> required int32 bar = 2;
> }
>
> message ExtendedMessageType2 {
> required int32 foo = 1;
> required int32 bar = 2;
> }
>
> message ExtendedMessageType3 {
> required int32 foo = 1;
> required int32 bar = 2;
> }
>
>
> test.py:
>
> import test_pb2
>
> MSG_ASSIGN_TASK = 1
> MSG_UPGRADE_TO_SUPER = 2
> MessageTypes = {1:'AssignTask', 2:'UpgradeToSuper'}
>
> msg = test_pb2.ICMMessage()
> msg.type = MSG_ASSIGN_TASK
> msg.assignTask.foo = 11
> msg.assignTask.bar = 22
> msg_send = msg.SerializeToString()
>
> print(msg)
> print(msg_send)
> print("Length: %d" % len(msg_send))
>
> msg_recv = msg_send
>
> msg1 = test_pb2.ICMMessage()
> msg1.ParseFromString(msg_recv)
> print("Got a message. Type: %s" % MessageTypes[msg1.type])
> print("Detail:")
> print(msg1.assignTask)
>
> The result:
>
> <image.png>
>
>
>
>>
>> On Fri, Jun 10, 2011 at 6:49 PM, Adriano Monteiro Marques <
>> [email protected]> wrote:
>>
>>> Hi Alan,
>>>
>>> On Jun 10, 2011, at 12:53 AM, Zhongjie Wang wrote:
>>>
>>> Hi Adriano,
>>> Thanks for the timely response. :)
>>>
>>> On Thu, Jun 9, 2011 at 10:44 PM, Adriano Monteiro Marques <
>>> [email protected]> wrote:
>>>
>>>> Hi Alan,
>>>>
>>>> On Jun 9, 2011, at 3:01 AM, Zhongjie Wang wrote:
>>>>
>>>> Hi all,
>>>>
>>>> About the message format, I have some comments.
>>>> 1. Seems there're duplicate fields in SendWebsiteReport and
>>>> SendServiceReport:
>>>>
>>>> 162<http://dev.umitproject.org/projects/icm/repository/revisions/master/entry/proto/messages.proto#L162>
>>>>
>>>> // send_report
>>>>
>>>> 163<http://dev.umitproject.org/projects/icm/repository/revisions/master/entry/proto/messages.proto#L163>
>>>>
>>>> message SendWebsiteReport {
>>>>
>>>> 164<http://dev.umitproject.org/projects/icm/repository/revisions/master/entry/proto/messages.proto#L164>
>>>>
>>>> required RequestHeader header = 1;
>>>>
>>>> 165<http://dev.umitproject.org/projects/icm/repository/revisions/master/entry/proto/messages.proto#L165>
>>>>
>>>> required WebsiteReport report = 2;
>>>>
>>>> 166<http://dev.umitproject.org/projects/icm/repository/revisions/master/entry/proto/messages.proto#L166>
>>>>
>>>> }
>>>>
>>>> In the RequestHeader structure to the aggregator, there're are
>>>> token and agentID. And in WebsiteReport.ICMReport, there're also agentID.
>>>> Though I didn't know the difference between token and agentID. I suggest we
>>>> eliminate the agentID field in ICMReport.
>>>>
>>>>
>>>> Token is our to confirm to the aggregator that we are legitimate. On the
>>>> duplicated agentID, it is fine: If a node is passing a website report from
>>>> another node, then the ICMReport will show the agentID for the origin node,
>>>> and the RequestHeader will provide the passing agent's id.
>>>>
>>>>
>>> If the token is sent with the message in plain text, then someone may
>>> perform a wire-tapping attack.
>>> And if the two party in communication has already had a handshake
>>> progress and figured out a symmetric key
>>> for current session, then we don't need the token i guess, the symmetric
>>> key itself is enough for the encryption and authentication purpose.
>>> It's OK to keep agentID as you said.
>>>
>>>
>>> Hm... Zubair helped me to set that there, let's just confirm with him
>>> that we don't need it there. Your consideration is fair, but I'm not sure
>>> this is the only purpose the token is there.
>>>
>>>
>>>
>>>> 2. Besides that, the optional fields: redirectLink,
>>>> htmlResponse, htmlMedia, seems to be better in the WebsiteReportDetail
>>>> structure, because it's more test specific. I suppose the Report messages
>>>> should have only two parts, the header and the detail.
>>>>
>>>>
>>>> Check the Event message... when we send back a WebsiteReportDetail or
>>>> ServiceReportDetail we don't send out those details as they're not really
>>>> necessary.
>>>>
>>>>
>>> Yes, in the Event message we don't need to send back the unnecessary
>>> fields.
>>> But since they (redirectLink, htmlResponse, htmlMedia) are "optional" in
>>> the Detail structure,
>>> they can be omitted while sending.
>>> Because when we send back Detail in Event message, we need to clone it
>>> and only send the necessary fields.
>>>
>>>
>>> Ok, sounds good. Can you get them fixed, and let the others know about
>>> your fix?
>>>
>>>
>>>
>>>> 3. Since we don't have a field in the message to identify the
>>>> message type, I don't know how we can create a message instance after
>>>> receiving a bunch of raw bytes. Currently I send the message type string
>>>> before send the message itself. But I think the better solution is tag it
>>>> in
>>>> the message structure.
>>>>
>>>>
>>>> Even if we had a field in the message, how would we retrieve it to know
>>>> the type prior to instantiating it? What we're doing on the Aggregator side
>>>> is to presume the sort of message a given API is supposed to send/receive.
>>>> Perhaps we could do the same with the desktop agent, what do you think?
>>>>
>>>>
>>> So that's what I propose at the beginning.
>>> All messages will be instantiate as a common message type.
>>> It has a type field and some option fields.
>>> The receiver will instantiate it as a common ICMMessage for example,
>>> then read the type field, and choose which optional field to read.
>>> The optional fields which doesn't exists in a message don't take any
>>> spaces.
>>> e.g.
>>> message ICMMessage {
>>> required int64 agentID = 1;
>>> required int32 type = 2;
>>>
>>> optional AssignTask assignTask = 100;
>>> optional UpgradeToSuper upgradeToSuper = 101;
>>> optional VersionUpdate versionUpdate = 102;
>>> // extensions 100 to max;
>>> }
>>>
>>>
>>> Did you try this approach? Not sure it will work with protocol buffers.
>>>
>>>
>>>
>>>> 4. Another slightly modification, I suggest that we use float
>>>> type for responseTime, the unit is seconds, for accuracy.
>>>>
>>>>
>>>> We're using miliseconds there as mentioned in a previous email, that's
>>>> the reason why we use integer. It saves on message size and keeps accuracy.
>>>>
>>>> Well noticed Alan! Keep pushing! Let me know if there still is anything
>>>> you feel like it is wrong or that is stopping you from progressing.
>>>>
>>>>
>>> Thanks, Adriano! I'll keep on thinking and make it perfect. :)
>>>
>>>
>>>>
>>>> Cheers!
>>>>
>>>>
>>>> Best Regards.
>>>> Alan
>>>>
>>>> On Sat, Jun 4, 2011 at 5:34 PM, Adriano Monteiro Marques <
>>>> [email protected]> wrote:
>>>>
>>>>> Right, mainly for nodes who stays connected for long periods of time.
>>>>> Also, if we want a test to be run ASAP, we need to send this info
>>>>> everytime
>>>>> so the node knows of a brand new test version, updates their test set and
>>>>> run them ASAP. On the software version, if we figure any security issue,
>>>>> the
>>>>> nodes will also update ASAP.
>>>>>
>>>>>
>>>>> On Jun 4, 2011, at 6:22 AM, Zubair Nabi wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I think the idea was to keep the binary and test versions as fresh as
>>>>> possible. This ResponseHeader would make sure of that.
>>>>>
>>>>> On Sat, Jun 4, 2011 at 4:04 AM, Diogo Pinheiro <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> I have another question. Why the ResponseHeader has the
>>>>>> currentVersionNo and currentTestsVersion ? I think we shouldn't send that
>>>>>> information all the time. Just when the agent connect, and in a rpc call.
>>>>>>
>>>>>> What do you think ?
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best,
>>>>> __
>>>>> Zubair
>>>>>
>>>>>
>>>>> ---
>>>>> Adriano Monteiro Marques
>>>>>
>>>>> http://www.thoughtspad.com
>>>>> http://www.umitproject.org
>>>>> http://blog.umitproject.org
>>>>> http://www.pythonbenelux.org
>>>>>
>>>>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Simplify data backup and recovery for your virtual environment with
>>>>> vRanger.
>>>>> Installation's a snap, and flexible recovery options mean your data is
>>>>> safe,
>>>>> secure and there when you need it. Discover what all the cheering's
>>>>> about.
>>>>> Get your free trial download today.
>>>>> http://p.sf.net/sfu/quest-dev2dev2
>>>>> _______________________________________________
>>>>> Umit-devel mailing list
>>>>> [email protected]
>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Zhongjie Wang
>>>> Master Candidate
>>>> Computer System Architecture
>>>> Peking University, China
>>>>
>>>>
>>>> ---
>>>> Adriano Monteiro Marques
>>>>
>>>> http://www.thoughtspad.com
>>>> http://www.umitproject.org
>>>> http://blog.umitproject.org
>>>> http://www.pythonbenelux.org
>>>>
>>>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>>>
>>>>
>>>
>>>
>>> --
>>> Zhongjie Wang
>>> Master Candidate
>>> Computer System Architecture
>>> Peking University, China
>>>
>>>
>>> ---
>>> Adriano Monteiro Marques
>>>
>>> http://www.thoughtspad.com
>>> http://www.umitproject.org
>>> http://blog.umitproject.org
>>> http://www.pythonbenelux.org
>>>
>>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>>
>>>
>>
>>
>> --
>> Best,
>> __
>> Zubair
>>
>
>
>
> --
> Zhongjie Wang
> Master Candidate
> Computer System Architecture
> Peking University, China
>
>
> ---
> Adriano Monteiro Marques
>
> http://www.thoughtspad.com
> http://www.umitproject.org
> http://blog.umitproject.org
> http://www.pythonbenelux.org
>
> "Don't stay in bed, unless you can make money in bed." - George Burns
>
>
--
Zhongjie Wang
Master Candidate
Computer System Architecture
Peking University, China
------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Umit-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/umit-devel
