Re: FreeBSD GSOC proposal in 2014
Any comments for this thread? There is only three days left for application. If the proposed idea should not be considered in this year, it should be removed from the GSoC idea list and I will submit a different proposal about the CPU hot plug problem in the FreeBSD kernel. Thanks, Yan 2014-03-18 15:39 GMT-04:00 yan cui ccuiy...@gmail.com: Really? Maybe I can download his code from previous GSoC. Actually, before applying for this idea, I did not scan the projects in previous years and just pick up one which I like. Are there any possibilities to improve on this part (or this idea should not be considered any more)? Yan 2014-03-18 14:26 GMT-04:00 John Baldwin j...@freebsd.org: On Friday, March 14, 2014 3:02:18 am Wojciech A. Koszek wrote: On Thu, Mar 13, 2014 at 09:56:35PM -0400, yan cui wrote: Hi all, I write this mail to make my question clear. I know witness can be used to detect wrong lock order in the kernel. However, can it be used to do lock profiling (what I mean is to report the information such as which locks are most contended and print some related statistics such as calling graph, etc)? In other words, is it enough to finish the task by porting witness to the pthread library? Yan, To my knowledge WITNESS is the only tool for lock order verification. For lock profiling in the FreeBSD kernel there's a KTR subsystem. KTR mechanism is basically like syslog() in the user-space, but for the kernel. KTR subsystem will receive messages from KTR API that is placed in the FreeBSD kernel. Messages get stored on the list of some sort. List can be exported to a file. File you can later analyze. Jeff wrote a Python app which can be used for pre-processing the KTR logs from scheduler and protting them visually. Link: http://svnweb.freebsd.org/base/head/tools/sched/schedgraph.py Instead of porting witness to pthreads, maybe we could evaluate expanding WITNESS to cover kern_umtx? This could prove to be more universal. Wojciech There is a dedicated lock profiler (LOCK_PROFILING) in the kernel. A previous GSoC student from an earlier year has already re-implemented both LOCK_PROFILING and WITNESS for pthreads. -- John Baldwin -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD GSOC proposal in 2014
On Wed, Mar 19, 2014 at 01:08:58PM -0400, yan cui wrote: Any comments for this thread? There is only three days left for application. If the proposed idea should not be considered in this year, it should be removed from the GSoC idea list and I will submit a different proposal about the CPU hot plug problem in the FreeBSD kernel. Yan, Please submit all possible ideas you have. We'll carefully look into them and judge which one is the most interesting from the FreeBSD point of view. If you were to stay with locking work, given that the work on pthreads was finished, we'd have to look into whether there's anything else to do in this domain. It still can be valuable to perform some improvements, but somebody else with expertise would have to judge. Thanks, Wojciech 2014-03-18 15:39 GMT-04:00 yan cui ccuiy...@gmail.com: Really? Maybe I can download his code from previous GSoC. Actually, before applying for this idea, I did not scan the projects in previous years and just pick up one which I like. Are there any possibilities to improve on this part (or this idea should not be considered any more)? Yan 2014-03-18 14:26 GMT-04:00 John Baldwin j...@freebsd.org: On Friday, March 14, 2014 3:02:18 am Wojciech A. Koszek wrote: On Thu, Mar 13, 2014 at 09:56:35PM -0400, yan cui wrote: Hi all, I write this mail to make my question clear. I know witness can be used to detect wrong lock order in the kernel. However, can it be used to do lock profiling (what I mean is to report the information such as which locks are most contended and print some related statistics such as calling graph, etc)? In other words, is it enough to finish the task by porting witness to the pthread library? Yan, To my knowledge WITNESS is the only tool for lock order verification. For lock profiling in the FreeBSD kernel there's a KTR subsystem. KTR mechanism is basically like syslog() in the user-space, but for the kernel. KTR subsystem will receive messages from KTR API that is placed in the FreeBSD kernel. Messages get stored on the list of some sort. List can be exported to a file. File you can later analyze. Jeff wrote a Python app which can be used for pre-processing the KTR logs from scheduler and protting them visually. Link: http://svnweb.freebsd.org/base/head/tools/sched/schedgraph.py Instead of porting witness to pthreads, maybe we could evaluate expanding WITNESS to cover kern_umtx? This could prove to be more universal. Wojciech There is a dedicated lock profiler (LOCK_PROFILING) in the kernel. A previous GSoC student from an earlier year has already re-implemented both LOCK_PROFILING and WITNESS for pthreads. -- John Baldwin -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. -- Wojciech A. Koszek wkos...@freebsd.czest.pl http://FreeBSD.czest.pl/~wkoszek/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD GSOC proposal in 2014
On Friday, March 14, 2014 3:02:18 am Wojciech A. Koszek wrote: On Thu, Mar 13, 2014 at 09:56:35PM -0400, yan cui wrote: Hi all, I write this mail to make my question clear. I know witness can be used to detect wrong lock order in the kernel. However, can it be used to do lock profiling (what I mean is to report the information such as which locks are most contended and print some related statistics such as calling graph, etc)? In other words, is it enough to finish the task by porting witness to the pthread library? Yan, To my knowledge WITNESS is the only tool for lock order verification. For lock profiling in the FreeBSD kernel there's a KTR subsystem. KTR mechanism is basically like syslog() in the user-space, but for the kernel. KTR subsystem will receive messages from KTR API that is placed in the FreeBSD kernel. Messages get stored on the list of some sort. List can be exported to a file. File you can later analyze. Jeff wrote a Python app which can be used for pre-processing the KTR logs from scheduler and protting them visually. Link: http://svnweb.freebsd.org/base/head/tools/sched/schedgraph.py Instead of porting witness to pthreads, maybe we could evaluate expanding WITNESS to cover kern_umtx? This could prove to be more universal. Wojciech There is a dedicated lock profiler (LOCK_PROFILING) in the kernel. A previous GSoC student from an earlier year has already re-implemented both LOCK_PROFILING and WITNESS for pthreads. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD GSOC proposal in 2014
Really? Maybe I can download his code from previous GSoC. Actually, before applying for this idea, I did not scan the projects in previous years and just pick up one which I like. Are there any possibilities to improve on this part (or this idea should not be considered any more)? Yan 2014-03-18 14:26 GMT-04:00 John Baldwin j...@freebsd.org: On Friday, March 14, 2014 3:02:18 am Wojciech A. Koszek wrote: On Thu, Mar 13, 2014 at 09:56:35PM -0400, yan cui wrote: Hi all, I write this mail to make my question clear. I know witness can be used to detect wrong lock order in the kernel. However, can it be used to do lock profiling (what I mean is to report the information such as which locks are most contended and print some related statistics such as calling graph, etc)? In other words, is it enough to finish the task by porting witness to the pthread library? Yan, To my knowledge WITNESS is the only tool for lock order verification. For lock profiling in the FreeBSD kernel there's a KTR subsystem. KTR mechanism is basically like syslog() in the user-space, but for the kernel. KTR subsystem will receive messages from KTR API that is placed in the FreeBSD kernel. Messages get stored on the list of some sort. List can be exported to a file. File you can later analyze. Jeff wrote a Python app which can be used for pre-processing the KTR logs from scheduler and protting them visually. Link: http://svnweb.freebsd.org/base/head/tools/sched/schedgraph.py Instead of porting witness to pthreads, maybe we could evaluate expanding WITNESS to cover kern_umtx? This could prove to be more universal. Wojciech There is a dedicated lock profiler (LOCK_PROFILING) in the kernel. A previous GSoC student from an earlier year has already re-implemented both LOCK_PROFILING and WITNESS for pthreads. -- John Baldwin -- Think big; Dream impossible; Make it happen. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD GSOC proposal in 2014
On Thu, Mar 13, 2014 at 09:56:35PM -0400, yan cui wrote: Hi all, I write this mail to make my question clear. I know witness can be used to detect wrong lock order in the kernel. However, can it be used to do lock profiling (what I mean is to report the information such as which locks are most contended and print some related statistics such as calling graph, etc)? In other words, is it enough to finish the task by porting witness to the pthread library? Yan, To my knowledge WITNESS is the only tool for lock order verification. For lock profiling in the FreeBSD kernel there's a KTR subsystem. KTR mechanism is basically like syslog() in the user-space, but for the kernel. KTR subsystem will receive messages from KTR API that is placed in the FreeBSD kernel. Messages get stored on the list of some sort. List can be exported to a file. File you can later analyze. Jeff wrote a Python app which can be used for pre-processing the KTR logs from scheduler and protting them visually. Link: http://svnweb.freebsd.org/base/head/tools/sched/schedgraph.py Instead of porting witness to pthreads, maybe we could evaluate expanding WITNESS to cover kern_umtx? This could prove to be more universal. Wojciech 2014-03-13 19:19 GMT-04:00 yan cui ccuiy...@gmail.com: Hi all, I have downloaded the newest FreeBSD-release kernel and scanned some codes. Wonder to know whether the lock order verification and lock profiling tool mentioned in the GSoC idea list is witness? Are there any other tools that needs to look at in the FreeBSD kernel? Thanks, Yan 2014-03-09 15:46 GMT-04:00 yan cui ccuiy...@gmail.com: Hi All, I am a student in Columbia University (Yan Cui), and want to join the FreeBSD GSOC 2014. After scanned the idea list posted online, I think I am interested in the idea titled user space pthread mutex lock contention profiling and lock order verification tools. I have several year experiences in kernel and user locking and believe I can complete the task in time. Currently, I wonder to know, before submitting an application on GSOC home page, do I need to submit some documents in the community (to review?) Best Wishes! Yan -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. ___ soc-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/soc-status To unsubscribe, send any mail to soc-status-unsubscr...@freebsd.org -- Wojciech A. Koszek wkos...@freebsd.czest.pl http://FreeBSD.czest.pl/~wkoszek/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD GSOC proposal in 2014
Thanks for the reply! I will get more information about KTR subsystem. 2014-03-14 3:02 GMT-04:00 Wojciech A. Koszek wkos...@freebsd.org: On Thu, Mar 13, 2014 at 09:56:35PM -0400, yan cui wrote: Hi all, I write this mail to make my question clear. I know witness can be used to detect wrong lock order in the kernel. However, can it be used to do lock profiling (what I mean is to report the information such as which locks are most contended and print some related statistics such as calling graph, etc)? In other words, is it enough to finish the task by porting witness to the pthread library? Yan, To my knowledge WITNESS is the only tool for lock order verification. For lock profiling in the FreeBSD kernel there's a KTR subsystem. KTR mechanism is basically like syslog() in the user-space, but for the kernel. KTR subsystem will receive messages from KTR API that is placed in the FreeBSD kernel. Messages get stored on the list of some sort. List can be exported to a file. File you can later analyze. Jeff wrote a Python app which can be used for pre-processing the KTR logs from scheduler and protting them visually. Link: http://svnweb.freebsd.org/base/head/tools/sched/schedgraph.py Instead of porting witness to pthreads, maybe we could evaluate expanding WITNESS to cover kern_umtx? This could prove to be more universal. Wojciech 2014-03-13 19:19 GMT-04:00 yan cui ccuiy...@gmail.com: Hi all, I have downloaded the newest FreeBSD-release kernel and scanned some codes. Wonder to know whether the lock order verification and lock profiling tool mentioned in the GSoC idea list is witness? Are there any other tools that needs to look at in the FreeBSD kernel? Thanks, Yan 2014-03-09 15:46 GMT-04:00 yan cui ccuiy...@gmail.com: Hi All, I am a student in Columbia University (Yan Cui), and want to join the FreeBSD GSOC 2014. After scanned the idea list posted online, I think I am interested in the idea titled user space pthread mutex lock contention profiling and lock order verification tools. I have several year experiences in kernel and user locking and believe I can complete the task in time. Currently, I wonder to know, before submitting an application on GSOC home page, do I need to submit some documents in the community (to review?) Best Wishes! Yan -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. ___ soc-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/soc-status To unsubscribe, send any mail to soc-status-unsubscr...@freebsd.org -- Wojciech A. Koszek wkos...@freebsd.czest.pl http://FreeBSD.czest.pl/~wkoszek/ -- Think big; Dream impossible; Make it happen. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD GSOC proposal in 2014
On Fri, Mar 14, 2014 at 01:07:48PM -0400, yan cui wrote: Thanks for the reply! I will get more information about KTR subsystem. You can also look here: http://people.freebsd.org/~kris/scaling/Help_my_system_is_slow.pdf Wojciech 2014-03-14 3:02 GMT-04:00 Wojciech A. Koszek wkos...@freebsd.org: On Thu, Mar 13, 2014 at 09:56:35PM -0400, yan cui wrote: Hi all, I write this mail to make my question clear. I know witness can be used to detect wrong lock order in the kernel. However, can it be used to do lock profiling (what I mean is to report the information such as which locks are most contended and print some related statistics such as calling graph, etc)? In other words, is it enough to finish the task by porting witness to the pthread library? Yan, To my knowledge WITNESS is the only tool for lock order verification. For lock profiling in the FreeBSD kernel there's a KTR subsystem. KTR mechanism is basically like syslog() in the user-space, but for the kernel. KTR subsystem will receive messages from KTR API that is placed in the FreeBSD kernel. Messages get stored on the list of some sort. List can be exported to a file. File you can later analyze. Jeff wrote a Python app which can be used for pre-processing the KTR logs from scheduler and protting them visually. Link: http://svnweb.freebsd.org/base/head/tools/sched/schedgraph.py Instead of porting witness to pthreads, maybe we could evaluate expanding WITNESS to cover kern_umtx? This could prove to be more universal. Wojciech 2014-03-13 19:19 GMT-04:00 yan cui ccuiy...@gmail.com: Hi all, I have downloaded the newest FreeBSD-release kernel and scanned some codes. Wonder to know whether the lock order verification and lock profiling tool mentioned in the GSoC idea list is witness? Are there any other tools that needs to look at in the FreeBSD kernel? Thanks, Yan 2014-03-09 15:46 GMT-04:00 yan cui ccuiy...@gmail.com: Hi All, I am a student in Columbia University (Yan Cui), and want to join the FreeBSD GSOC 2014. After scanned the idea list posted online, I think I am interested in the idea titled user space pthread mutex lock contention profiling and lock order verification tools. I have several year experiences in kernel and user locking and believe I can complete the task in time. Currently, I wonder to know, before submitting an application on GSOC home page, do I need to submit some documents in the community (to review?) Best Wishes! Yan -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. ___ soc-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/soc-status To unsubscribe, send any mail to soc-status-unsubscr...@freebsd.org -- Wojciech A. Koszek wkos...@freebsd.czest.pl http://FreeBSD.czest.pl/~wkoszek/ -- Think big; Dream impossible; Make it happen. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org -- Wojciech A. Koszek wkos...@freebsd.czest.pl http://FreeBSD.czest.pl/~wkoszek/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD GSOC proposal in 2014
Hi all, I have downloaded the newest FreeBSD-release kernel and scanned some codes. Wonder to know whether the lock order verification and lock profiling tool mentioned in the GSoC idea list is witness? Are there any other tools that needs to look at in the FreeBSD kernel? Thanks, Yan 2014-03-09 15:46 GMT-04:00 yan cui ccuiy...@gmail.com: Hi All, I am a student in Columbia University (Yan Cui), and want to join the FreeBSD GSOC 2014. After scanned the idea list posted online, I think I am interested in the idea titled user space pthread mutex lock contention profiling and lock order verification tools. I have several year experiences in kernel and user locking and believe I can complete the task in time. Currently, I wonder to know, before submitting an application on GSOC home page, do I need to submit some documents in the community (to review?) Best Wishes! Yan -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD GSOC proposal in 2014
Hi all, I write this mail to make my question clear. I know witness can be used to detect wrong lock order in the kernel. However, can it be used to do lock profiling (what I mean is to report the information such as which locks are most contended and print some related statistics such as calling graph, etc)? In other words, is it enough to finish the task by porting witness to the pthread library? Thanks, Yan 2014-03-13 19:19 GMT-04:00 yan cui ccuiy...@gmail.com: Hi all, I have downloaded the newest FreeBSD-release kernel and scanned some codes. Wonder to know whether the lock order verification and lock profiling tool mentioned in the GSoC idea list is witness? Are there any other tools that needs to look at in the FreeBSD kernel? Thanks, Yan 2014-03-09 15:46 GMT-04:00 yan cui ccuiy...@gmail.com: Hi All, I am a student in Columbia University (Yan Cui), and want to join the FreeBSD GSOC 2014. After scanned the idea list posted online, I think I am interested in the idea titled user space pthread mutex lock contention profiling and lock order verification tools. I have several year experiences in kernel and user locking and believe I can complete the task in time. Currently, I wonder to know, before submitting an application on GSOC home page, do I need to submit some documents in the community (to review?) Best Wishes! Yan -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. -- Think big; Dream impossible; Make it happen. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD GSOC proposal in 2014
On Sun, 9 Mar 2014 15:46:10 -0400 yan cui ccuiy...@gmail.com wrote: Hi All, I am a student in Columbia University (Yan Cui), and want to join the FreeBSD GSOC 2014. After scanned the idea list posted online, I think I am interested in the idea titled user space pthread mutex lock contention profiling and lock order verification tools. I have several year experiences in kernel and user locking and believe I can complete the task in time. Currently, I wonder to know, before submitting an application on GSOC home page, do I need to submit some documents in the community (to review?) There is no requirement to submit something to the community. The review will be done after your submission. There is the possibility to improve your application, either based upon feedback from the reviewers, or even on your own if you notice that your forgot something or want to add something. Participating in the community before the GSoC would tell something about your interest (the above message does already tell something) to participate and may also show something about your knowledge level. Bye, Alexander. -- http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
FreeBSD GSOC proposal in 2014
Hi All, I am a student in Columbia University (Yan Cui), and want to join the FreeBSD GSOC 2014. After scanned the idea list posted online, I think I am interested in the idea titled user space pthread mutex lock contention profiling and lock order verification tools. I have several year experiences in kernel and user locking and believe I can complete the task in time. Currently, I wonder to know, before submitting an application on GSOC home page, do I need to submit some documents in the community (to review?) Best Wishes! Yan -- Think big; Dream impossible; Make it happen. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
