Hi, Nice see your dedication.
On Sun, Mar 27, 2011 at 11:26 PM, Kay <[email protected]> wrote: > There are so many mathematics in João's slide which I am not quite good at > =.= . You mean there will be no readymade information in UMIT as the input > of IDS, so I need to implement the algorithm all by myself. UMPA is, among other things, a sniffer, so is not a big deal write something as a input of your program/algorithm. The approach described in the slides is something more robust than simple window approach. There is long term time factor also. Consider a port-scan which I can configure to send one packet after an interval T, and the IDS tool has a time window W < T. Will be a nice thought find out the implications. :) By the way, the approach you are building should be on feature, that can be improved over time. > I think it will be a waste of time and space to store the access time of > each port (65535). An IDS can estimate whether ports are accessed in a > relatively small time interval and do not need the accurate access time. > By reading the papers, I found that the bitmap method is a general approach > in network monitoring. > The port scan, there can be a port bitmap. This will only take a > 8KB (65535bit / 8 = 8KB) memory. When a port is accessed in a received > packet, its corresponding bit is set to 1. In a time interval(take 10s for > an example), we count the bits which are set to 1. If the number is larger > than a certain value(such as 1000?). We can report this as a port-scan > event, because no regular traffic will access so many ports in such a small > time slot. After the time slot, the bitmap is set to zero again. > The Sync-attack. I think there is an excellent paper in counting number of > active flows with bitmap and can be used in this detection. > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.105.7004&rep=rep1&type=pdf > By estimating the number of active flows in a time interval, we can report > it as a sync-attack when the number become particularly large. > Any suggestions? Thanks. My suggestion is to start write your proposal. An web page or a shared google document. So we can discuss with a common text. > --Kay > > > On Mon, Mar 28, 2011 at 6:21 AM, Luis A. Bastiao Silva <[email protected]> > wrote: >> >> Hi Kay, >> >> On Sun, Mar 27, 2011 at 2:57 PM, Kay <[email protected]> wrote: >>> >>> Hi, >>> I have been reading papers about port scan detection and sync-attack >>> detection these days. I am trying to find an algorithm which can report >>> attack ASAP but not aimed at detecting the scanner/attacker. Is this the >>> right way? >> >> Yes, of course. It is a good way to start. >> >>> >>> I hope to design a new algorithm which servers the above purpose. Maybe >>> we can write a paper on this after the project =) >> >> I'm impressive that you are looking for scientific paper. It will be >> awesome of course. >> >>> >>> Since I am not quite familiar with the UMIT, is there some readymade >>> information in implementing the IDS? such as the port's time access >>> mentioned by João, or I will get these myself. >> >> Well, Umit Project does not have this kind of documentation, yet. However >> João in his presentations use neural networks to "learn" about this. Did you >> get this in his slides? :) >> I think he fit algorithm in real-time. João, is it right? >> >>> >>> Thanks. >>> --Kay >>> >>> >>> On Fri, Mar 25, 2011 at 10:26 PM, Luis A. Bastiao Silva >>> <[email protected]> wrote: >>>> >>>> I was open the spectrum of the idea. You refer some experience with NIDS >>>> and João give some clues how you can do that. >>>> >>>> On Fri, Mar 25, 2011 at 1:34 PM, Kay <[email protected]> wrote: >>>>> >>>>> I am not quite clear about what functions a personal IDS should have. >>>>> What about the followings: >>>>> 1) Port-scan detection(Includes lots of types, UDP/SYN/FIN,etc.), >>>>> 2) Syn-attack detection (can be implemented with a bitmap method I have >>>>> read in a paper) >>>>> 3) Ping flood detection >>>>> 4) What the attacker is looking for, I think I need time to study the >>>>> regular scan methods and the OS fingerprints so that I can evaluate the >>>>> workload >>>>> >>>>> Frankly speaking, I am a bit confused about what to do in the project. >>>>> Can you give some suggestions about it? >>>> >>>> There is a possibility to present a proposal with a IDS for Umit >>>> Project, indeed. The topics that you point out are a good start. And you >>>> can >>>> check material that João has shared with you. :) >>>> >>>>> >>>>> Thanks. >>>>> --Kay >>>>> >>>>> >>>>> On Fri, Mar 25, 2011 at 10:06 AM, Luis A. Bastiao Silva >>>>> <[email protected]> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> On Fri, Mar 25, 2011 at 1:36 AM, Kay <[email protected]> wrote: >>>>>>> >>>>>>> Thanks for your suggestion. >>>>>>> About the port-scan, how about this way: Pick a few sets of ports >>>>>>> randomly at a certain interval(for instance, 30s), calculate their >>>>>>> access >>>>>>> time difference. If most of the results are less than a certain >>>>>>> value(3s), >>>>>>> and the access time are all within the latest interval >>>>>>> (current_time-interval ~ current_time). We can report this as an event >>>>>>> of >>>>>>> port-scan which happens in the last interval. >>>>>> >>>>>> It's a basic approach. Certainly, you're on the right way. >>>>>> Nevertheless, there are several papers discussing the subject. I'm going >>>>>> to >>>>>> point out one of them: >>>>>> http://www.aloul.net/Papers/faloul_iwcmc08.pdf >>>>>> @ignotus21 (João): Do you have any own theory for such feature? >>>>>> >>>>>>> >>>>>>> I have took a look at the UMPA, it's really a good work =) I think >>>>>>> you mean that I can use it to sniff packets and analysis the captured >>>>>>> packets to detect intrusion. >>>>>> >>>>>> Yes, also you can use Audit Framework. There are several passive >>>>>> audits. So IDS should be a new one. Take a look: >>>>>> http://trac.umitproject.org/wiki/AuditFramework >>>>>> and >>>>>> http://trac.umitproject.org/browser/packet-manipulator/trunk/audits <- >>>>>> Passive + Active >>>>>> >>>>>> >>>>>>> >>>>>>> I am not quite familiar with statistical analysis. What I have been >>>>>>> focused on is the multi-core architecture and how to accelerate network >>>>>>> processing on it. I'd like to know exactly what functions should have >>>>>>> in a >>>>>>> personal NIDS so that I can evaluate if I have the ability to work on >>>>>>> this >>>>>>> project. Port-scan detection, DDoS detection, or something else? >>>>>> >>>>>> Indeed, it is a good idea. >>>>>> Port-scan detectiong and DDoS has a huge spectrum. For instance, >>>>>> detect malware on networks, software that polls servers, etc. >>>>>> It will be nice also to know what attacker is looking for: >>>>>> Services/Services Information/OS Fingerprints. >>>>>> >>>>>>> >>>>>>> Best regards, >>>>>>> --Kay >>>>>>> >>>>>>> >>>>>>> On Thu, Mar 24, 2011 at 7:49 PM, [email protected] >>>>>>> <[email protected]> wrote: >>>>>>>> >>>>>>>> Dear Kay, >>>>>>>> >>>>>>>> When I was reading your e-mail I have some ideas that I wish to >>>>>>>> share >>>>>>>> with you... >>>>>>>> >>>>>>>> On Thu, Mar 24, 2011 at 6:45 AM, Luis A. Bastiao Silva >>>>>>>> <[email protected]> wrote: >>>>>>>> > Hello Kay, >>>>>>>> > On Thu, Mar 24, 2011 at 7:08 AM, Kay <[email protected]> wrote: >>>>>>>> >> >>>>>>>> >> Hi, all >>>>>>>> >> I am a master student of computer science in University of >>>>>>>> >> Science and >>>>>>>> >> Technology of China and want to participate in GSoC 2011. The >>>>>>>> >> focus of my >>>>>>>> >> lab program lies in building parallel NIDS on multi-core >>>>>>>> >> platforms, and >>>>>>>> >> based on the lab experimens I built a high-performance parallel >>>>>>>> >> HTTP parser >>>>>>>> >> which can achieve at least 5Gbps line rate in a harsh >>>>>>>> >> environment. >>>>>>>> > >>>>>>>> > Thanks for introduce yourself. It should be a cool research area, >>>>>>>> > for sure! >>>>>>>> >>>>>>>> It sounds someone is able to write a possible new Umit >>>>>>>> application... >>>>>>>> What you guys think about a personal NIDS (using UMPA)? >>>>>>>> >>>>>>>> >> The HTTP parser I built is aimed at measuring network >>>>>>>> >> latencies(match the >>>>>>>> >> request and response to get the time difference). I am >>>>>>>> >> experienced with C >>>>>>>> >> and specialized in network domain knowledge. Frankly speaking, I >>>>>>>> >> know Python >>>>>>>> >> a little and only wrote a few small programs with it. But I think >>>>>>>> >> I can >>>>>>>> >> learn it quickly and use it in the development. >>>>>>>> >>>>>>>> It seems you are friend of statistical analysis. So, let me point >>>>>>>> out one idea: >>>>>>>> - It is possible to that my machine is being attacked by a >>>>>>>> port-scan? >>>>>>>> - Even if the only information I have is the port's time access? >>>>>>>> >>>>>>>> > Indeed. If you already know C, enhance Python will not be an >>>>>>>> > issue. >>>>>>>> > >>>>>>>> >> >>>>>>>> >> So I want to do some work in the network domain and found the >>>>>>>> >> "5. Packet >>>>>>>> >> Tracker Platform" suitable for me. The "Jitter based" and >>>>>>>> >> "Dipacket Inspection: inspect packet contents (e.g. HTTP >>>>>>>> >> contents)" is >>>>>>>> >> related to my previous project. >>>>>>>> > >>>>>>>> > Sure. This idea is over network-domain, mainly focuses into >>>>>>>> > packet analyses. >>>>>>>> > >>>>>>>> >> >>>>>>>> >> However, I found this idea is not that specific. Maybe because my >>>>>>>> >> lack of >>>>>>>> >> domain knowledge or poor in English, I don't quite understand the >>>>>>>> >> "Detect >>>>>>>> >> packets with debit (e.g. more/less than 100Kb/s)" >>>>>>>> >> >>>>>>>> >> Can someone give me detailed information about this idea and >>>>>>>> >> where I >>>>>>>> >> should begin with to learn something or make some contributions >>>>>>>> >> now? >>>>>>>> > >>>>>>>> > Yes, of course. >>>>>>>> > >>>>>>>> > Read http://trac.umitproject.org/wiki/PacketManipulator >>>>>>>> > Checkout source of PacketManipulator >>>>>>>> > >>>>>>>> > svn co >>>>>>>> > http://svn.umitproject.org/svnroot/umit/packet-manipulator/trunk >>>>>>>> > PacketManipulator >>>>>>>> > >>>>>>>> > Read http://trac.umitproject.org/wiki/AuditFramework and related >>>>>>>> > links >>>>>>>> > >>>>>>>> > In this idea, it expected to has a real-time statistic depend on >>>>>>>> > the amount >>>>>>>> > of sniffed packets. >>>>>>>> > >>>>>>>> > Packets >>>>>>>> > Multicast/Broadcast packets >>>>>>>> > IPv4/IPv6 >>>>>>>> > Bytes >>>>>>>> > Fragments >>>>>>>> > Detect retransmissions/error packets >>>>>>>> > Count of packets by protocol >>>>>>>> > etc. >>>>>>>> > >>>>>>>> > Such information should presented in the GUI of PacketManipulator >>>>>>>> > (for >>>>>>>> > instance, expand Host Table into Packet Manipulator GUI). >>>>>>>> > Also, the end-user should be able to configure an alarm/event, >>>>>>>> > e.g. when >>>>>>>> > detect a specific packet from/to a destination. Such details, >>>>>>>> > should be >>>>>>>> > exploit into the proposal. More tips: >>>>>>>> > >>>>>>>> > Define a threshold of utilization >>>>>>>> > Define latency threshold >>>>>>>> > >>>>>>>> > Finally, to present a GSoC proposal take a look: >>>>>>>> > >>>>>>>> > http://www.umitproject.org/?active=gsoc&mode=ideas&lang=en >>>>>>>> > http://www.umitproject.org/?active=gsoc&mode=tips&lang=en >>>>>>>> > http://www.google-melange.com/gsoc/org/show/google/gsoc2011/umit >>>>>>>> > >>>>>>>> > I'm look forward to discussing more details about this proposal. >>>>>>>> > If you have >>>>>>>> > any doubts, do not hesitate to contact us for further details. >>>>>>>> > >>>>>>>> > >>>>>>>> >> Thanks a lot! >>>>>>>> >> --Kay >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> ------------------------------------------------------------------------------ >>>>>>>> >> Enable your software for Intel(R) Active Management Technology to >>>>>>>> >> meet the >>>>>>>> >> growing manageability and security demands of your customers. >>>>>>>> >> Businesses >>>>>>>> >> are taking advantage of Intel(R) vPro (TM) technology - will your >>>>>>>> >> software >>>>>>>> >> be a part of the solution? Download the Intel(R) Manageability >>>>>>>> >> Checker >>>>>>>> >> today! http://p.sf.net/sfu/intel-dev2devmar >>>>>>>> >> _______________________________________________ >>>>>>>> >> Umit-devel mailing list >>>>>>>> >> [email protected] >>>>>>>> >> https://lists.sourceforge.net/lists/listinfo/umit-devel >>>>>>>> >> >>>>>>>> > >>>>>>>> > >>>>>>>> > Best Regards, >>>>>>>> > -- >>>>>>>> > Luís A. Bastião Silva >>>>>>>> > Skype: koplabs >>>>>>>> > http://www.bastiao.org >>>>>>>> > >>>>>>>> > >>>>>>>> > ------------------------------------------------------------------------------ >>>>>>>> > Enable your software for Intel(R) Active Management Technology to >>>>>>>> > meet the >>>>>>>> > growing manageability and security demands of your customers. >>>>>>>> > Businesses >>>>>>>> > are taking advantage of Intel(R) vPro (TM) technology - will your >>>>>>>> > software >>>>>>>> > be a part of the solution? Download the Intel(R) Manageability >>>>>>>> > Checker >>>>>>>> > today! http://p.sf.net/sfu/intel-dev2devmar >>>>>>>> > _______________________________________________ >>>>>>>> > Umit-devel mailing list >>>>>>>> > [email protected] >>>>>>>> > https://lists.sourceforge.net/lists/listinfo/umit-devel >>>>>>>> >>>>>>>> -- >>>>>>>> Att, João Medeiros >>>>>>> >>>>>> >>>>>> if you have any doubts, let us know. I'm look forward to know more >>>>>> details about your proposal >>>>>> >>>>>> Best Regards, >>>>>> -- >>>>>> Luís A. Bastião Silva >>>>>> Skype: koplabs >>>>>> http://www.bastiao.org >>>>> >>>> >>>> >>>> Keep in touch. >>>> >>>> Best Regards, >>>> -- >>>> Luís A. Bastião Silva >>>> Skype: koplabs >>>> http://www.bastiao.org >>> >> >> >> Best Regards, >> -- >> Luís A. Bastião Silva >> Skype: koplabs >> http://www.bastiao.org > > -- Att, João Medeiros ------------------------------------------------------------------------------ Enable your software for Intel(R) Active Management Technology to meet the growing manageability and security demands of your customers. Businesses are taking advantage of Intel(R) vPro (TM) technology - will your software be a part of the solution? Download the Intel(R) Manageability Checker today! http://p.sf.net/sfu/intel-dev2devmar _______________________________________________ Umit-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/umit-devel
