On Tue, Sep 15, 2020 at 1:47 AM Liyichao <liyic...@huawei.com> wrote:
> I believe we make many small writes to disk (i.e., each object is a > separate write). > > > > -----yes, this is a point we have modified it and to be tested. > > > > It may be possible to instead use an mmap file and flush it to disk at the > time of the checkpoint. > > -----from the GEM5 source code,the physical memory file is opened and > written using the gzopen/gzwrite of the zlib library. The gzopen opens a > file path, or the gzdopen file opens an opened file handle. However, mmap > returns the start address of an opened file handle, it is not a type of > args in gzopen/gzdopen ,so we can't compress and write files to reduce the > file size. > Another option is to use a less complex compression scheme. In fact, gzip shouldn't be very effective with memory data anyway. I don't know off the top of my head, but using a simple zero-based compression (compress 0s but nothing else) could be much faster and result in files that are about the same size. Cheers, Jason > > > > > > > > ------------------------------ > > 李翼超（Charlie） > > > > 华为技术有限公司 Huawei Technologies Co., Ltd. > > [image: Company_logo] > > 部门：计算系统与组件开发部 [云与计算BG] > > 手 机：15858232899 > 电子邮件：liyic...@huawei.com > > 地址：中国(China)-杭州(Hangzhou)-滨江区江淑路360号华为杭州研发中心Z4# [3-A06] > ------------------------------ > > 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 > 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 > 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ > This e-mail and its attachments contain confidential information from > HUAWEI, which > is intended only for the person or entity whose address is listed above. > Any use of the > information contained herein in any way (including, but not limited to, > total or partial > disclosure, reproduction, or dissemination) by persons other than the > intended > recipient(s) is prohibited. If you receive this e-mail in error, please > notify the sender by > phone or email immediately and delete it! > > > > *发件人:* Jason Lowe-Power [mailto:ja...@lowepower.com] > *发送时间:* 2020年9月14日 23:15 > *收件人:* gem5 users mailing list <gem5-users@gem5.org> > *抄送:* Liyichao <liyic...@huawei.com> > *主题:* Re: [gem5-users] Are there any ideas to accelerate the speed of > m5.checkpoint on disks? > > > > This isn't a problem we've seen before, but it's not particularly > surprising. I believe we make many small writes to disk (i.e., each object > is a separate write). I'm not sure how to fix this, but it might be a place > to start. > > > > Another place to look is that most of the data written is the contents of > main memory. It may be possible to instead use an mmap file and flush it to > disk at the time of the checkpoint. Again, just a place to look :). > > > > We'd be happy to accept any improvements to the checkpointing system! > > > > Cheers, > > Jason > > > > On Mon, Sep 14, 2020 at 12:53 AM Liyichao via gem5-users < > gem5-users@gem5.org> wrote: > > Hi All: > > Are there any ideas to accelerate the speed of m5.checkpoint on > disks? In my NVME ssd , the speed of taking checkpoint is only about 145K/s. > > > > > ------------------------------ > > 李翼超（Charlie） > > > > 华为技术有限公司 Huawei Technologies Co., Ltd. > > [image: Company_logo] > > 部门：计算系统与组件开发部 [云与计算BG] > > 手 机：15858232899 > 电子邮件：liyic...@huawei.com > > 地址：中国(China)-杭州(Hangzhou)-滨江区江淑路360号华为杭州研发中心Z4# [3-A06] > ------------------------------ > > 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 > 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 > 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ > This e-mail and its attachments contain confidential information from > HUAWEI, which > is intended only for the person or entity whose address is listed above. > Any use of the > information contained herein in any way (including, but not limited to, > total or partial > disclosure, reproduction, or dissemination) by persons other than the > intended > recipient(s) is prohibited. If you receive this e-mail in error, please > notify the sender by > phone or email immediately and delete it! > > > > _______________________________________________ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s > >
_______________________________________________ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
