Hey, it's me. :-)
On Fri, May 27, 2011 at 2:57 AM, Adriano Monteiro Marques <
[email protected]> wrote:
> Hi Zubair,
>
>
> On May 26, 2011, at 3:46 AM, Zhongjie Wang wrote:
>
> Hi guys. :)
> I've made some changes to the proto file, and now it's more
> extendable.
> You may view the proto file here:
>
> http://dev.umitproject.org/projects/desktop-agent/repository/revisions/master/entry/report.proto
>
> The agent will always send a ICMReport message each time.
> And it may contain a bunch of test results, website results, service
> results, etc.
> Becuase the agent will test many websites and services in a period of
> time,
> and all the results will be sent by one report message.
>
> Here's a example of ICMReport message:
>
> reportID: 89734
> senderID: 10000
> timestamp: 1306391787312
> testID: 100
> passedNode: "219.223.222.99:6060"
> passedNode: "219.223.192.198:6060"
> websiteReport {
> websiteURL: "http://www.baidu.com";
> statusCode: 200
> }
> websiteReport {
> websiteURL: "https://www.alipay.com";
> statusCode: 200
> }
>
>
> Where will the website content data come together in this message?
>
That's a problem indeed. Because in the specification of protobuf, it is
stated:
*"Protocol Buffers are not designed to handle large messages. As a general
rule of thumb, if you are dealing in messages larger than a megabyte each,
it may be time to consider an alternate strategy."*
So we may zip the website content into a tarball and send it to a specific
webservice. Is it feasible?
>
>
> It's also very easy to add new test types.
> Do you guys think this is feasible?
>
>
> On Thu, May 26, 2011 at 12:28 AM, Adriano Monteiro Marques <
> [email protected]> wrote:
>
>>
>> On May 25, 2011, at 1:01 PM, Zhongjie Wang wrote:
>>
>> Hi Adriano,
>>
>> On Wed, May 25, 2011 at 11:54 PM, Adriano Monteiro Marques <
>> [email protected]> wrote:
>>
>>> Hi Alan,
>>>
>>> On May 25, 2011, at 12:25 PM, Zhongjie Wang wrote:
>>>
>>> Hi, I simply write it this way. :)
>>> Help me to check if there's anything wrong. I'm new to protobuf.
>>>
>>> message ReportHeader {
>>> required int64 reportID = 1;
>>> required int64 senderID = 2;
>>> required int64 timestamp = 3;
>>> required int32 testID = 4;
>>> repeated bytes passedNode = 5; // the peer in the way of routing to the
>>> aggregator, this field is repeated
>>> }
>>>
>>> message WebsiteReport {
>>> required ReportHeader header = 1;
>>> required string websiteURL = 2;
>>> required int32 statusCode = 3;
>>> optional double responseTime = 4;
>>> optional double bandwidth = 5;
>>> extensions 10 to 99; // can add more fields later
>>> }
>>>
>>> message ServiceReport {
>>> required ReportHeader header = 1;
>>> required string serviceName = 2;
>>> required int32 statusCode = 3;
>>> optional double responseTime = 4;
>>> optional double bandwidth = 5;
>>> extensions 10 to 99; // can add more fields later
>>> }
>>>
>>>
>>> Ok, gotcha.But we'e still missing the response content. This is important
>>> for us to do two things (I have discussed this previously with Diogo): get
>>> statistical data from the response (this is helpful for finding false
>>> results) and also to show people what people in that region is seeing while
>>> trying to access that url. This view will actually add another set of fields
>>> or perhaps change that one to a blob so it can hold all media and images
>>> that would allow us for showing the website properly from inside the
>>> aggregator.
>>>
>>
>> I understand. Shall we add the whole webpage html source as a string field
>> named "page_content" in the WebsiteReport.
>> If we download all the images and media in the page, then it's too large.
>> and it will be an heavy load for the aggregator. :)
>>
>>
>> There are no limits in the cloud, dude :-) Let's add all the media files
>> to provide our users with a better experience and let the aggregator handle
>> it.
>>
>>
>>>
>>> On Wed, May 25, 2011 at 9:26 PM, Adriano Monteiro Marques <
>>> [email protected]> wrote:
>>>
>>>> Hi Alan,
>>>>
>>>> Would you provide us with a sample based on Zubair's defs?
>>>>
>>>> Meanwhile, what do you guys think of Alan's approach? I think it is more
>>>> extensible, though I'm unsure on how he intend to implement that using
>>>> protobuf. Zubair, What's your vision on extending the message based on the
>>>> defs you wrote?
>>>>
>>>> On May 25, 2011, at 10:22 AM, Zhongjie Wang wrote:
>>>>
>>>> Hi Adriano,
>>>> I haven't done that yet, It's just a bit different with the one
>>>> Zubair has made. :)
>>>>
>>>> On Wed, May 25, 2011 at 9:12 PM, Adriano Monteiro Marques <
>>>> [email protected]> wrote:
>>>>
>>>>> Hi Alan,
>>>>>
>>>>> Do you have a protobuf message already implemented the way you just
>>>>> described?
>>>>>
>>>>> On May 25, 2011, at 10:00 AM, Zhongjie Wang wrote:
>>>>>
>>>>> Hi, folks. For the first report scheme, I was planning to describe
>>>>> a hierarchical format as the network protocols, which has header and
>>>>> payload.
>>>>> The header consists of Report ID (generated by hash), Sender ID (agent
>>>>> token), Timestamp (UTC), Test ID (unique for each kind of test).
>>>>> The payload is the Test Result and Routing Information. For each kind
>>>>> of tests, the format of result if different. e.g. Website Test has the
>>>>> target url, status info, response time, and some meta data. Service Test
>>>>> has
>>>>> the service name (or port num), status info, response time.
>>>>>
>>>>>
>>>>> 2011/5/25 Zubair Nabi <[email protected]>
>>>>>
>>>>>> Currently, one the specification, there are two different message
>>>>>> formats. I chose the second one.
>>>>>>
>>>>>> Alan earlier mentioned some extra fields. Alan, can you please repeat
>>>>>> it so that we can address it right now and close this issue.
>>>>>>
>>>>>> 2011/5/25 Luís A. Bastião Silva <[email protected]>
>>>>>>
>>>>>>
>>>>>>> http://dev.umitproject.org/projects/icm-mobile/repository/revisions/master/entry/ICM-Mobile/src/org/umit/icm/mobile/proto/Messages.proto
>>>>>>>
>>>>>>> It's the link
>>>>>>>
>>>>>>> On Wed, May 25, 2011 at 12:31 AM, Adriano Monteiro Marques <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>> Hi folks,
>>>>>>>>
>>>>>>>> Sorry, I was getting my umit-devel messages sent somewhere else and
>>>>>>>> wasn't receiving any of the messages in this mailing list. Fixed the
>>>>>>>> issue
>>>>>>>> now. Thanks Luis for letting me know about the discussion here.
>>>>>>>>
>>>>>>>> So the mentioned fields are necessary in order for us to gather
>>>>>>>> statistical data. They're fine just like that.
>>>>>>>>
>>>>>>>>
>>>>>>>> Cheers!
>>>>>>>>
>>>>>>>> On May 16, 2011, at 10:57 PM, Zhongjie Wang wrote:
>>>>>>>>
>>>>>>>> Hi Zubair!
>>>>>>>> Thanks for your modeling.
>>>>>>>> I think we need extra report ID and sender ID fields.
>>>>>>>> Beside that, I'm not sure if we need these two fields:
>>>>>>>>
>>>>>>>>
>>>>>>>> 29<http://dev.umitproject.org/projects/icm-mobile/repository/revisions/master/entry/src/org/umit/icm/mobile/proto/Messages.proto#L29>
>>>>>>>>
>>>>>>>> required string responseContent = 4;
>>>>>>>>
>>>>>>>> 30<http://dev.umitproject.org/projects/icm-mobile/repository/revisions/master/entry/src/org/umit/icm/mobile/proto/Messages.proto#L30>
>>>>>>>>
>>>>>>>> required bool matchPattern = 5;
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Alan
>>>>>>>>
>>>>>>>> On Mon, May 16, 2011 at 10:23 PM, Zubair Nabi <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> Hey Everyone,
>>>>>>>>>
>>>>>>>>> Apropos the Test Report Format from the ICM specification. I
>>>>>>>>> implemented the 4 formats in Google Protobuf and wanted to discuss
>>>>>>>>> them with
>>>>>>>>> everyone before proceeding because this format should be uniform
>>>>>>>>> across all
>>>>>>>>> entities in ICM.
>>>>>>>>>
>>>>>>>>> Here's a link to the .proto in the code:
>>>>>>>>> http://dev.umitproject.org/projects/icm-mobile/repository/revisions/master/entry/src/org/umit/icm/mobile/proto/Messages.proto
>>>>>>>>> Also, refer to the "Another Test Report Format" in the
>>>>>>>>> specification for the format discussion.
>>>>>>>>>
>>>>>>>>> <http://dev.umitproject.org/projects/icm-mobile/repository/revisions/master/entry/src/org/umit/icm/mobile/proto/Messages.proto>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best,
>>>>>>>>> __
>>>>>>>>> Zubair
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>> Achieve unprecedented app performance and reliability
>>>>>>>>> What every C/C++ and Fortran developer should know.
>>>>>>>>> Learn how Intel has extended the reach of its next-generation tools
>>>>>>>>> to help boost performance applications - inlcuding clusters.
>>>>>>>>> http://p.sf.net/sfu/intel-dev2devmay
>>>>>>>>> _______________________________________________
>>>>>>>>> Umit-devel mailing list
>>>>>>>>> [email protected]
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Zhongjie Wang
>>>>>>>> Master Candidate
>>>>>>>> Computer System Architecture
>>>>>>>> Peking University, China
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Achieve unprecedented app performance and reliability
>>>>>>>> What every C/C++ and Fortran developer should know.
>>>>>>>> Learn how Intel has extended the reach of its next-generation tools
>>>>>>>> to help boost performance applications - inlcuding clusters.
>>>>>>>>
>>>>>>>> http://p.sf.net/sfu/intel-dev2devmay_______________________________________________
>>>>>>>> Umit-devel mailing list
>>>>>>>> [email protected]
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>>>>
>>>>>>>>
>>>>>>>> ---
>>>>>>>> Adriano Monteiro Marques
>>>>>>>>
>>>>>>>> http://www.thoughtspad.com
>>>>>>>> http://www.umitproject.org
>>>>>>>> http://blog.umitproject.org
>>>>>>>> http://www.pythonbenelux.org
>>>>>>>>
>>>>>>>> "Don't stay in bed, unless you can make money in bed." - George
>>>>>>>> Burns
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> vRanger cuts backup time in half-while increasing security.
>>>>>>>> With the market-leading solution for virtual backup and recovery,
>>>>>>>> you get blazing-fast, flexible, and affordable data protection.
>>>>>>>> Download your free trial now.
>>>>>>>> http://p.sf.net/sfu/quest-d2dcopy1
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Umit-devel mailing list
>>>>>>>> [email protected]
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Luís A. Bastião Silva
>>>>>>> Umit Project Developer
>>>>>>> Skype: koplabs
>>>>>>> http://www.umitproject.org
>>>>>>> http://www.bastiao.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> vRanger cuts backup time in half-while increasing security.
>>>>>>> With the market-leading solution for virtual backup and recovery,
>>>>>>> you get blazing-fast, flexible, and affordable data protection.
>>>>>>> Download your free trial now.
>>>>>>> http://p.sf.net/sfu/quest-d2dcopy1
>>>>>>> _______________________________________________
>>>>>>> Umit-devel mailing list
>>>>>>> [email protected]
>>>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best,
>>>>>> __
>>>>>> Zubair
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> vRanger cuts backup time in half-while increasing security.
>>>>>> With the market-leading solution for virtual backup and recovery,
>>>>>> you get blazing-fast, flexible, and affordable data protection.
>>>>>> Download your free trial now.
>>>>>> http://p.sf.net/sfu/quest-d2dcopy1
>>>>>> _______________________________________________
>>>>>> Umit-devel mailing list
>>>>>> [email protected]
>>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Zhongjie Wang
>>>>> Master Candidate
>>>>> Computer System Architecture
>>>>> Peking University, China
>>>>> ------------------------------------------------------------------------------
>>>>> vRanger cuts backup time in half-while increasing security.
>>>>> With the market-leading solution for virtual backup and recovery,
>>>>> you get blazing-fast, flexible, and affordable data protection.
>>>>> Download your free trial now.
>>>>>
>>>>> http://p.sf.net/sfu/quest-d2dcopy1_______________________________________________
>>>>> Umit-devel mailing list
>>>>> [email protected]
>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>
>>>>>
>>>>> ---
>>>>> Adriano Monteiro Marques
>>>>>
>>>>> http://www.thoughtspad.com
>>>>> http://www.umitproject.org
>>>>> http://blog.umitproject.org
>>>>> http://www.pythonbenelux.org
>>>>>
>>>>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Zhongjie Wang
>>>> Master Candidate
>>>> Computer System Architecture
>>>> Peking University, China
>>>>
>>>>
>>>> ---
>>>> Adriano Monteiro Marques
>>>>
>>>> http://www.thoughtspad.com
>>>> http://www.umitproject.org
>>>> http://blog.umitproject.org
>>>> http://www.pythonbenelux.org
>>>>
>>>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>>>
>>>>
>>>
>>>
>>> --
>>> Zhongjie Wang
>>> Master Candidate
>>> Computer System Architecture
>>> Peking University, China
>>>
>>>
>>> ---
>>> Adriano Monteiro Marques
>>>
>>> http://www.thoughtspad.com
>>> http://www.umitproject.org
>>> http://blog.umitproject.org
>>> http://www.pythonbenelux.org
>>>
>>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>>
>>>
>>
>>
>> --
>> Zhongjie Wang
>> Master Candidate
>> Computer System Architecture
>> Peking University, China
>>
>>
>> ---
>> Adriano Monteiro Marques
>>
>> http://www.thoughtspad.com
>> http://www.umitproject.org
>> http://blog.umitproject.org
>> http://www.pythonbenelux.org
>>
>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>
>>
>
>
> --
> Zhongjie Wang
> Master Candidate
> Computer System Architecture
> Peking University, China
>
>
> ---
> Adriano Monteiro Marques
>
> http://www.thoughtspad.com
> http://www.umitproject.org
> http://blog.umitproject.org
> http://www.pythonbenelux.org
>
> "Don't stay in bed, unless you can make money in bed." - George Burns
>
>
--
Zhongjie Wang
Master Candidate
Computer System Architecture
Peking University, China
------------------------------------------------------------------------------
vRanger cuts backup time in half-while increasing security.
With the market-leading solution for virtual backup and recovery,
you get blazing-fast, flexible, and affordable data protection.
Download your free trial now.
http://p.sf.net/sfu/quest-d2dcopy1
_______________________________________________
Umit-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/umit-devel
