Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
On Mon, Sep 02, 2013 at 12:11:35PM +0300, Adrian Hunter wrote: > On 02/09/13 11:37, Jiri Olsa wrote: > > On Mon, Sep 02, 2013 at 10:52:46AM +0300, Adrian Hunter wrote: > >> On 01/09/13 13:36, Jiri Olsa wrote: > >>> split event data into multiple files based on the file > >>> size or time delta specified as an argument to the option. > >>> > >>> Adding multi file '-M' option for record command to store > >>> output perf.data into multiple files based on the size > >>> threshold. > >>> > >>> The threshold is specified either as size (B/K/M/G) or time > >>> (s/m/h) by appending the size with appropriate unit, like: > >>> -M 5M for 5 megabytes threshold > >>> -M 1h for 1 hour threshold > >>> The generated name for each data file is appended with sequential > >>> number (prepended by 5 zeros).For default output file name it > >>> will be: > >>> perf.data-0 > >>> perf.data-1 > >>> perf.data-2 > >>> ... > >>> > >>> Also watermark/wakeup_watermark is set accordingly to get > >>> wake ups more often so we could get close enough to the > >>> file size promise. > >>> > >>> Example: > >>> $ perf record -M 5M yes > /dev/null > >>> ^C[ perf record: Woken up 228 times to write data ] > >>> [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 > >>> samples) ] > >>> yes: Interrupt > >>> $ ls -l perf.data-0* > >>> -rw--- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-0 > >>> -rw--- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-1 > >>> -rw--- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-2 > >>> -rw--- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-3 > >>> -rw--- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-4 > >> > >> Did you consider doing this with "perf inject" instead of "perf record"? > >> That way you could get the split exactly right, and you could split the > >> same > >> file different ways. > > > > right, this could be added into inject command as well > > I was really thinking "instead" not "as well" e.g. > > perf record yes | perf inject -M 5M > yep, got it.. this gives you more fine grained size resolution and lets you split already stored data files, nice But while I think this is nice feature to have, two things keeps me thinking we should leave the support in record command as well: - the extra inject command overhead, that needs to parse the whole stream, which might not be wanted in some cases - AFAIU the vast majority of perf users do just record/report and would be restrained from this feature ;-) jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
On 02/09/13 11:37, Jiri Olsa wrote: > On Mon, Sep 02, 2013 at 10:52:46AM +0300, Adrian Hunter wrote: >> On 01/09/13 13:36, Jiri Olsa wrote: >>> split event data into multiple files based on the file >>> size or time delta specified as an argument to the option. >>> >>> Adding multi file '-M' option for record command to store >>> output perf.data into multiple files based on the size >>> threshold. >>> >>> The threshold is specified either as size (B/K/M/G) or time >>> (s/m/h) by appending the size with appropriate unit, like: >>> -M 5M for 5 megabytes threshold >>> -M 1h for 1 hour threshold >>> The generated name for each data file is appended with sequential >>> number (prepended by 5 zeros).For default output file name it >>> will be: >>> perf.data-0 >>> perf.data-1 >>> perf.data-2 >>> ... >>> >>> Also watermark/wakeup_watermark is set accordingly to get >>> wake ups more often so we could get close enough to the >>> file size promise. >>> >>> Example: >>> $ perf record -M 5M yes > /dev/null >>> ^C[ perf record: Woken up 228 times to write data ] >>> [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 >>> samples) ] >>> yes: Interrupt >>> $ ls -l perf.data-0* >>> -rw--- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-0 >>> -rw--- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-1 >>> -rw--- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-2 >>> -rw--- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-3 >>> -rw--- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-4 >> >> Did you consider doing this with "perf inject" instead of "perf record"? >> That way you could get the split exactly right, and you could split the same >> file different ways. > > right, this could be added into inject command as well I was really thinking "instead" not "as well" e.g. perf record yes | perf inject -M 5M -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
On Mon, Sep 02, 2013 at 10:52:46AM +0300, Adrian Hunter wrote: > On 01/09/13 13:36, Jiri Olsa wrote: > > split event data into multiple files based on the file > > size or time delta specified as an argument to the option. > > > > Adding multi file '-M' option for record command to store > > output perf.data into multiple files based on the size > > threshold. > > > > The threshold is specified either as size (B/K/M/G) or time > > (s/m/h) by appending the size with appropriate unit, like: > > -M 5M for 5 megabytes threshold > > -M 1h for 1 hour threshold > > The generated name for each data file is appended with sequential > > number (prepended by 5 zeros).For default output file name it > > will be: > > perf.data-0 > > perf.data-1 > > perf.data-2 > > ... > > > > Also watermark/wakeup_watermark is set accordingly to get > > wake ups more often so we could get close enough to the > > file size promise. > > > > Example: > > $ perf record -M 5M yes > /dev/null > > ^C[ perf record: Woken up 228 times to write data ] > > [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 > > samples) ] > > yes: Interrupt > > $ ls -l perf.data-0* > > -rw--- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-0 > > -rw--- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-1 > > -rw--- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-2 > > -rw--- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-3 > > -rw--- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-4 > > Did you consider doing this with "perf inject" instead of "perf record"? > That way you could get the split exactly right, and you could split the same > file different ways. right, this could be added into inject command as well jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
On 01/09/13 13:36, Jiri Olsa wrote: > split event data into multiple files based on the file > size or time delta specified as an argument to the option. > > Adding multi file '-M' option for record command to store > output perf.data into multiple files based on the size > threshold. > > The threshold is specified either as size (B/K/M/G) or time > (s/m/h) by appending the size with appropriate unit, like: > -M 5M for 5 megabytes threshold > -M 1h for 1 hour threshold > The generated name for each data file is appended with sequential > number (prepended by 5 zeros).For default output file name it > will be: > perf.data-0 > perf.data-1 > perf.data-2 > ... > > Also watermark/wakeup_watermark is set accordingly to get > wake ups more often so we could get close enough to the > file size promise. > > Example: > $ perf record -M 5M yes > /dev/null > ^C[ perf record: Woken up 228 times to write data ] > [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 > samples) ] > yes: Interrupt > $ ls -l perf.data-0* > -rw--- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-0 > -rw--- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-1 > -rw--- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-2 > -rw--- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-3 > -rw--- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-4 Did you consider doing this with "perf inject" instead of "perf record"? That way you could get the split exactly right, and you could split the same file different ways. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
On 01/09/13 13:36, Jiri Olsa wrote: split event data into multiple files based on the file size or time delta specified as an argument to the option. Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold. The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like: -M 5M for 5 megabytes threshold -M 1h for 1 hour threshold The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be: perf.data-0 perf.data-1 perf.data-2 ... Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise. Example: $ perf record -M 5M yes /dev/null ^C[ perf record: Woken up 228 times to write data ] [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ] yes: Interrupt $ ls -l perf.data-0* -rw--- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-0 -rw--- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-1 -rw--- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-2 -rw--- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-3 -rw--- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-4 Did you consider doing this with perf inject instead of perf record? That way you could get the split exactly right, and you could split the same file different ways. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
On Mon, Sep 02, 2013 at 10:52:46AM +0300, Adrian Hunter wrote: On 01/09/13 13:36, Jiri Olsa wrote: split event data into multiple files based on the file size or time delta specified as an argument to the option. Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold. The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like: -M 5M for 5 megabytes threshold -M 1h for 1 hour threshold The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be: perf.data-0 perf.data-1 perf.data-2 ... Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise. Example: $ perf record -M 5M yes /dev/null ^C[ perf record: Woken up 228 times to write data ] [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ] yes: Interrupt $ ls -l perf.data-0* -rw--- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-0 -rw--- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-1 -rw--- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-2 -rw--- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-3 -rw--- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-4 Did you consider doing this with perf inject instead of perf record? That way you could get the split exactly right, and you could split the same file different ways. right, this could be added into inject command as well jirka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
On 02/09/13 11:37, Jiri Olsa wrote: On Mon, Sep 02, 2013 at 10:52:46AM +0300, Adrian Hunter wrote: On 01/09/13 13:36, Jiri Olsa wrote: split event data into multiple files based on the file size or time delta specified as an argument to the option. Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold. The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like: -M 5M for 5 megabytes threshold -M 1h for 1 hour threshold The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be: perf.data-0 perf.data-1 perf.data-2 ... Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise. Example: $ perf record -M 5M yes /dev/null ^C[ perf record: Woken up 228 times to write data ] [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ] yes: Interrupt $ ls -l perf.data-0* -rw--- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-0 -rw--- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-1 -rw--- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-2 -rw--- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-3 -rw--- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-4 Did you consider doing this with perf inject instead of perf record? That way you could get the split exactly right, and you could split the same file different ways. right, this could be added into inject command as well I was really thinking instead not as well e.g. perf record yes | perf inject -M 5M -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
On Mon, Sep 02, 2013 at 12:11:35PM +0300, Adrian Hunter wrote: On 02/09/13 11:37, Jiri Olsa wrote: On Mon, Sep 02, 2013 at 10:52:46AM +0300, Adrian Hunter wrote: On 01/09/13 13:36, Jiri Olsa wrote: split event data into multiple files based on the file size or time delta specified as an argument to the option. Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold. The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like: -M 5M for 5 megabytes threshold -M 1h for 1 hour threshold The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be: perf.data-0 perf.data-1 perf.data-2 ... Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise. Example: $ perf record -M 5M yes /dev/null ^C[ perf record: Woken up 228 times to write data ] [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ] yes: Interrupt $ ls -l perf.data-0* -rw--- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-0 -rw--- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-1 -rw--- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-2 -rw--- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-3 -rw--- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-4 Did you consider doing this with perf inject instead of perf record? That way you could get the split exactly right, and you could split the same file different ways. right, this could be added into inject command as well I was really thinking instead not as well e.g. perf record yes | perf inject -M 5M yep, got it.. this gives you more fine grained size resolution and lets you split already stored data files, nice But while I think this is nice feature to have, two things keeps me thinking we should leave the support in record command as well: - the extra inject command overhead, that needs to parse the whole stream, which might not be wanted in some cases - AFAIU the vast majority of perf users do just record/report and would be restrained from this feature ;-) jirka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
