Hi Kay, It is good see your proposal evolving. I just read your proposal and want to share some ideas.
On Wed, Mar 30, 2011 at 4:54 AM, Kay <[email protected]> wrote: > Hi, all > A have prepared a draft proposal about the project I am going to do. I write > them on the Google doc. > The project details : > https://docs.google.com/document/d/1r1plWP8B5FcVD5wolVTX7wQaNIMsTTOzO7GVHCvolws/edit?hl=zh_CN&authkey=CImv4YYN > The full proposal based on the template: > https://docs.google.com/document/d/1pjfcenqN74dZZGN1ZSDO2LYwjPAiFhAPFS69UzWThsc/edit?hl=zh_CN&authkey=CIjK59ML > Thanks for your review and suggestions. > --Kay First, you define that your solution will be a extensible framework. That is really good! So, actually, how this extension will be possible, using an script language (in python itself)? Or, you plan something more restrict like grammar rules and regular expression applied on the format message of network protocols? Regarding the port-scan detection there is this list of some introductory references: - http://www.phrack.org/issues.html?issue=53&id=13#article - http://nms.csail.mit.edu/papers/portscan-oakland04.pdf - http://cseweb.ucsd.edu/~clbailey/PortScans.pdf Reading them, and the references on it, will probably open your eyes in the finding of an good algorithm. > On Mon, Mar 28, 2011 at 8:11 PM, [email protected] <[email protected]> > wrote: >> >> Hi, >> >> Nice see your dedication. >> >> On Sun, Mar 27, 2011 at 11:26 PM, Kay <[email protected]> wrote: >> > There are so many mathematics in João's slide which I am not quite good >> > at >> > =.= . You mean there will be no readymade information in UMIT as the >> > input >> > of IDS, so I need to implement the algorithm all by myself. >> >> UMPA is, among other things, a sniffer, so is not a big deal write >> something as a input of your program/algorithm. >> >> The approach described in the slides is something more robust than >> simple window approach. There is long term time factor also. Consider >> a port-scan which I can configure to send one packet after an interval >> T, and the IDS tool has a time window W < T. Will be a nice thought >> find out the implications. :) >> >> By the way, the approach you are building should be on feature, that >> can be improved over time. >> >> > I think it will be a waste of time and space to store the access time of >> > each port (65535). An IDS can estimate whether ports are accessed in a >> > relatively small time interval and do not need the accurate access time. >> > By reading the papers, I found that the bitmap method is a general >> > approach >> > in network monitoring. >> > The port scan, there can be a port bitmap. This will only take a >> > 8KB (65535bit / 8 = 8KB) memory. When a port is accessed in a received >> > packet, its corresponding bit is set to 1. In a time interval(take 10s >> > for >> > an example), we count the bits which are set to 1. If the number is >> > larger >> > than a certain value(such as 1000?). We can report this as a port-scan >> > event, because no regular traffic will access so many ports in such a >> > small >> > time slot. After the time slot, the bitmap is set to zero again. >> > The Sync-attack. I think there is an excellent paper in counting number >> > of >> > active flows with bitmap and can be used in this detection. >> > >> > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.105.7004&rep=rep1&type=pdf >> > By estimating the number of active flows in a time interval, we can >> > report >> > it as a sync-attack when the number become particularly large. >> > Any suggestions? Thanks. >> >> My suggestion is to start write your proposal. An web page or a shared >> google document. So we can discuss with a common text. >> >> > --Kay >> > >> > >> > On Mon, Mar 28, 2011 at 6:21 AM, Luis A. Bastiao Silva >> > <[email protected]> >> > wrote: >> >> >> >> Hi Kay, >> >> >> >> On Sun, Mar 27, 2011 at 2:57 PM, Kay <[email protected]> wrote: >> >>> >> >>> Hi, >> >>> I have been reading papers about port scan detection and sync-attack >> >>> detection these days. I am trying to find an algorithm which can >> >>> report >> >>> attack ASAP but not aimed at detecting the scanner/attacker. Is this >> >>> the >> >>> right way? >> >> >> >> Yes, of course. It is a good way to start. >> >> >> >>> >> >>> I hope to design a new algorithm which servers the above purpose. >> >>> Maybe >> >>> we can write a paper on this after the project =) >> >> >> >> I'm impressive that you are looking for scientific paper. It will be >> >> awesome of course. >> >> >> >>> >> >>> Since I am not quite familiar with the UMIT, is there some readymade >> >>> information in implementing the IDS? such as the port's time access >> >>> mentioned by João, or I will get these myself. >> >> >> >> Well, Umit Project does not have this kind of documentation, yet. >> >> However >> >> João in his presentations use neural networks to "learn" about this. >> >> Did you >> >> get this in his slides? :) >> >> I think he fit algorithm in real-time. João, is it right? >> >> >> >>> >> >>> Thanks. >> >>> --Kay >> >>> >> >>> >> >>> On Fri, Mar 25, 2011 at 10:26 PM, Luis A. Bastiao Silva >> >>> <[email protected]> wrote: >> >>>> >> >>>> I was open the spectrum of the idea. You refer some experience with >> >>>> NIDS >> >>>> and João give some clues how you can do that. >> >>>> >> >>>> On Fri, Mar 25, 2011 at 1:34 PM, Kay <[email protected]> wrote: >> >>>>> >> >>>>> I am not quite clear about what functions a personal IDS should >> >>>>> have. >> >>>>> What about the followings: >> >>>>> 1) Port-scan detection(Includes lots of types, UDP/SYN/FIN,etc.), >> >>>>> 2) Syn-attack detection (can be implemented with a bitmap method I >> >>>>> have >> >>>>> read in a paper) >> >>>>> 3) Ping flood detection >> >>>>> 4) What the attacker is looking for, I think I need time to study >> >>>>> the >> >>>>> regular scan methods and the OS fingerprints so that I can evaluate >> >>>>> the >> >>>>> workload >> >>>>> >> >>>>> Frankly speaking, I am a bit confused about what to do in the >> >>>>> project. >> >>>>> Can you give some suggestions about it? >> >>>> >> >>>> There is a possibility to present a proposal with a IDS for Umit >> >>>> Project, indeed. The topics that you point out are a good start. And >> >>>> you can >> >>>> check material that João has shared with you. :) >> >>>> >> >>>>> >> >>>>> Thanks. >> >>>>> --Kay >> >>>>> >> >>>>> >> >>>>> On Fri, Mar 25, 2011 at 10:06 AM, Luis A. Bastiao Silva >> >>>>> <[email protected]> wrote: >> >>>>>> >> >>>>>> Hi, >> >>>>>> >> >>>>>> On Fri, Mar 25, 2011 at 1:36 AM, Kay <[email protected]> wrote: >> >>>>>>> >> >>>>>>> Thanks for your suggestion. >> >>>>>>> About the port-scan, how about this way: Pick a few sets of ports >> >>>>>>> randomly at a certain interval(for instance, 30s), calculate their >> >>>>>>> access >> >>>>>>> time difference. If most of the results are less than a certain >> >>>>>>> value(3s), >> >>>>>>> and the access time are all within the latest interval >> >>>>>>> (current_time-interval ~ current_time). We can report this as an >> >>>>>>> event of >> >>>>>>> port-scan which happens in the last interval. >> >>>>>> >> >>>>>> It's a basic approach. Certainly, you're on the right way. >> >>>>>> Nevertheless, there are several papers discussing the subject. I'm >> >>>>>> going to >> >>>>>> point out one of them: >> >>>>>> http://www.aloul.net/Papers/faloul_iwcmc08.pdf >> >>>>>> @ignotus21 (João): Do you have any own theory for such feature? >> >>>>>> >> >>>>>>> >> >>>>>>> I have took a look at the UMPA, it's really a good work =) I think >> >>>>>>> you mean that I can use it to sniff packets and analysis the >> >>>>>>> captured >> >>>>>>> packets to detect intrusion. >> >>>>>> >> >>>>>> Yes, also you can use Audit Framework. There are several passive >> >>>>>> audits. So IDS should be a new one. Take a look: >> >>>>>> http://trac.umitproject.org/wiki/AuditFramework >> >>>>>> and >> >>>>>> http://trac.umitproject.org/browser/packet-manipulator/trunk/audits >> >>>>>> <- >> >>>>>> Passive + Active >> >>>>>> >> >>>>>> >> >>>>>>> >> >>>>>>> I am not quite familiar with statistical analysis. What I have >> >>>>>>> been >> >>>>>>> focused on is the multi-core architecture and how to accelerate >> >>>>>>> network >> >>>>>>> processing on it. I'd like to know exactly what functions should >> >>>>>>> have in a >> >>>>>>> personal NIDS so that I can evaluate if I have the ability to work >> >>>>>>> on this >> >>>>>>> project. Port-scan detection, DDoS detection, or something else? >> >>>>>> >> >>>>>> Indeed, it is a good idea. >> >>>>>> Port-scan detectiong and DDoS has a huge spectrum. For instance, >> >>>>>> detect malware on networks, software that polls servers, etc. >> >>>>>> It will be nice also to know what attacker is looking for: >> >>>>>> Services/Services Information/OS Fingerprints. >> >>>>>> >> >>>>>>> >> >>>>>>> Best regards, >> >>>>>>> --Kay >> >>>>>>> >> >>>>>>> >> >>>>>>> On Thu, Mar 24, 2011 at 7:49 PM, [email protected] >> >>>>>>> <[email protected]> wrote: >> >>>>>>>> >> >>>>>>>> Dear Kay, >> >>>>>>>> >> >>>>>>>> When I was reading your e-mail I have some ideas that I wish to >> >>>>>>>> share >> >>>>>>>> with you... >> >>>>>>>> >> >>>>>>>> On Thu, Mar 24, 2011 at 6:45 AM, Luis A. Bastiao Silva >> >>>>>>>> <[email protected]> wrote: >> >>>>>>>> > Hello Kay, >> >>>>>>>> > On Thu, Mar 24, 2011 at 7:08 AM, Kay <[email protected]> wrote: >> >>>>>>>> >> >> >>>>>>>> >> Hi, all >> >>>>>>>> >> I am a master student of computer science in University of >> >>>>>>>> >> Science and >> >>>>>>>> >> Technology of China and want to participate in GSoC 2011. The >> >>>>>>>> >> focus of my >> >>>>>>>> >> lab program lies in building parallel NIDS on multi-core >> >>>>>>>> >> platforms, and >> >>>>>>>> >> based on the lab experimens I built a high-performance >> >>>>>>>> >> parallel >> >>>>>>>> >> HTTP parser >> >>>>>>>> >> which can achieve at least 5Gbps line rate in a harsh >> >>>>>>>> >> environment. >> >>>>>>>> > >> >>>>>>>> > Thanks for introduce yourself. It should be a cool research >> >>>>>>>> > area, >> >>>>>>>> > for sure! >> >>>>>>>> >> >>>>>>>> It sounds someone is able to write a possible new Umit >> >>>>>>>> application... >> >>>>>>>> What you guys think about a personal NIDS (using UMPA)? >> >>>>>>>> >> >>>>>>>> >> The HTTP parser I built is aimed at measuring network >> >>>>>>>> >> latencies(match the >> >>>>>>>> >> request and response to get the time difference). I am >> >>>>>>>> >> experienced with C >> >>>>>>>> >> and specialized in network domain knowledge. Frankly speaking, >> >>>>>>>> >> I >> >>>>>>>> >> know Python >> >>>>>>>> >> a little and only wrote a few small programs with it. But I >> >>>>>>>> >> think >> >>>>>>>> >> I can >> >>>>>>>> >> learn it quickly and use it in the development. >> >>>>>>>> >> >>>>>>>> It seems you are friend of statistical analysis. So, let me point >> >>>>>>>> out one idea: >> >>>>>>>> - It is possible to that my machine is being attacked by a >> >>>>>>>> port-scan? >> >>>>>>>> - Even if the only information I have is the port's time >> >>>>>>>> access? >> >>>>>>>> >> >>>>>>>> > Indeed. If you already know C, enhance Python will not be an >> >>>>>>>> > issue. >> >>>>>>>> > >> >>>>>>>> >> >> >>>>>>>> >> So I want to do some work in the network domain and found the >> >>>>>>>> >> "5. Packet >> >>>>>>>> >> Tracker Platform" suitable for me. The "Jitter based" and >> >>>>>>>> >> "Dipacket Inspection: inspect packet contents (e.g. HTTP >> >>>>>>>> >> contents)" is >> >>>>>>>> >> related to my previous project. >> >>>>>>>> > >> >>>>>>>> > Sure. This idea is over network-domain, mainly focuses into >> >>>>>>>> > packet analyses. >> >>>>>>>> > >> >>>>>>>> >> >> >>>>>>>> >> However, I found this idea is not that specific. Maybe because >> >>>>>>>> >> my >> >>>>>>>> >> lack of >> >>>>>>>> >> domain knowledge or poor in English, I don't quite understand >> >>>>>>>> >> the >> >>>>>>>> >> "Detect >> >>>>>>>> >> packets with debit (e.g. more/less than 100Kb/s)" >> >>>>>>>> >> >> >>>>>>>> >> Can someone give me detailed information about this idea and >> >>>>>>>> >> where I >> >>>>>>>> >> should begin with to learn something or make some >> >>>>>>>> >> contributions >> >>>>>>>> >> now? >> >>>>>>>> > >> >>>>>>>> > Yes, of course. >> >>>>>>>> > >> >>>>>>>> > Read http://trac.umitproject.org/wiki/PacketManipulator >> >>>>>>>> > Checkout source of PacketManipulator >> >>>>>>>> > >> >>>>>>>> > svn co >> >>>>>>>> > >> >>>>>>>> > http://svn.umitproject.org/svnroot/umit/packet-manipulator/trunk >> >>>>>>>> > PacketManipulator >> >>>>>>>> > >> >>>>>>>> > Read http://trac.umitproject.org/wiki/AuditFramework and >> >>>>>>>> > related >> >>>>>>>> > links >> >>>>>>>> > >> >>>>>>>> > In this idea, it expected to has a real-time statistic depend >> >>>>>>>> > on >> >>>>>>>> > the amount >> >>>>>>>> > of sniffed packets. >> >>>>>>>> > >> >>>>>>>> > Packets >> >>>>>>>> > Multicast/Broadcast packets >> >>>>>>>> > IPv4/IPv6 >> >>>>>>>> > Bytes >> >>>>>>>> > Fragments >> >>>>>>>> > Detect retransmissions/error packets >> >>>>>>>> > Count of packets by protocol >> >>>>>>>> > etc. >> >>>>>>>> > >> >>>>>>>> > Such information should presented in the GUI of >> >>>>>>>> > PacketManipulator >> >>>>>>>> > (for >> >>>>>>>> > instance, expand Host Table into Packet Manipulator GUI). >> >>>>>>>> > Also, the end-user should be able to configure an alarm/event, >> >>>>>>>> > e.g. when >> >>>>>>>> > detect a specific packet from/to a destination. Such details, >> >>>>>>>> > should be >> >>>>>>>> > exploit into the proposal. More tips: >> >>>>>>>> > >> >>>>>>>> > Define a threshold of utilization >> >>>>>>>> > Define latency threshold >> >>>>>>>> > >> >>>>>>>> > Finally, to present a GSoC proposal take a look: >> >>>>>>>> > >> >>>>>>>> > http://www.umitproject.org/?active=gsoc&mode=ideas&lang=en >> >>>>>>>> > http://www.umitproject.org/?active=gsoc&mode=tips&lang=en >> >>>>>>>> > >> >>>>>>>> > http://www.google-melange.com/gsoc/org/show/google/gsoc2011/umit >> >>>>>>>> > >> >>>>>>>> > I'm look forward to discussing more details about this >> >>>>>>>> > proposal. >> >>>>>>>> > If you have >> >>>>>>>> > any doubts, do not hesitate to contact us for further details. >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> >> Thanks a lot! >> >>>>>>>> >> --Kay >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> ------------------------------------------------------------------------------ >> >>>>>>>> >> Enable your software for Intel(R) Active Management Technology >> >>>>>>>> >> to >> >>>>>>>> >> meet the >> >>>>>>>> >> growing manageability and security demands of your customers. >> >>>>>>>> >> Businesses >> >>>>>>>> >> are taking advantage of Intel(R) vPro (TM) technology - will >> >>>>>>>> >> your >> >>>>>>>> >> software >> >>>>>>>> >> be a part of the solution? Download the Intel(R) Manageability >> >>>>>>>> >> Checker >> >>>>>>>> >> today! http://p.sf.net/sfu/intel-dev2devmar >> >>>>>>>> >> _______________________________________________ >> >>>>>>>> >> Umit-devel mailing list >> >>>>>>>> >> [email protected] >> >>>>>>>> >> https://lists.sourceforge.net/lists/listinfo/umit-devel >> >>>>>>>> >> >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> > Best Regards, >> >>>>>>>> > -- >> >>>>>>>> > Luís A. Bastião Silva >> >>>>>>>> > Skype: koplabs >> >>>>>>>> > http://www.bastiao.org >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> > ------------------------------------------------------------------------------ >> >>>>>>>> > Enable your software for Intel(R) Active Management Technology >> >>>>>>>> > to >> >>>>>>>> > meet the >> >>>>>>>> > growing manageability and security demands of your customers. >> >>>>>>>> > Businesses >> >>>>>>>> > are taking advantage of Intel(R) vPro (TM) technology - will >> >>>>>>>> > your >> >>>>>>>> > software >> >>>>>>>> > be a part of the solution? Download the Intel(R) Manageability >> >>>>>>>> > Checker >> >>>>>>>> > today! http://p.sf.net/sfu/intel-dev2devmar >> >>>>>>>> > _______________________________________________ >> >>>>>>>> > Umit-devel mailing list >> >>>>>>>> > [email protected] >> >>>>>>>> > https://lists.sourceforge.net/lists/listinfo/umit-devel >> >>>>>>>> >> >>>>>>>> -- >> >>>>>>>> Att, João Medeiros >> >>>>>>> >> >>>>>> >> >>>>>> if you have any doubts, let us know. I'm look forward to know more >> >>>>>> details about your proposal >> >>>>>> >> >>>>>> Best Regards, >> >>>>>> -- >> >>>>>> Luís A. Bastião Silva >> >>>>>> Skype: koplabs >> >>>>>> http://www.bastiao.org >> >>>>> >> >>>> >> >>>> >> >>>> Keep in touch. >> >>>> >> >>>> Best Regards, >> >>>> -- >> >>>> Luís A. Bastião Silva >> >>>> Skype: koplabs >> >>>> http://www.bastiao.org >> >>> >> >> >> >> >> >> Best Regards, >> >> -- >> >> Luís A. Bastião Silva >> >> Skype: koplabs >> >> http://www.bastiao.org >> > >> > >> >> >> >> -- >> Att, João Medeiros > > -- Att, João Medeiros ------------------------------------------------------------------------------ Create and publish websites with WebMatrix Use the most popular FREE web apps or write code yourself; WebMatrix provides all the features you need to develop and publish your website. http://p.sf.net/sfu/ms-webmatrix-sf _______________________________________________ Umit-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/umit-devel
