On Thu, Mar 30, 2017 at 12:22 PM, Alisa Matsak <[email protected]> wrote: > The system supports automatic log rotation with per-VM quotas to prevent > DDoS attacks.
Not DDoS. Just DoS. Also, some careful thought needs to be put into log rotation. It is desirable to prevent the case where a VM gets attacked in some manner which produces logs showing what happened and how, and then proceeds to fill up the logs with garbage. It is not desirable for rotation to be implemented in such a manner that the attacker can reliably cause the logs detailing their attack to be discarded. There is of course the fundamental problem of logging generating data and storage being finite, so something must be discarded. So in some way the above may be unavoidable (keep-oldest policy results in attackers filling logs before attacking, keep-newest policy results in attackers filling logs after attacking). Some user interaction here may perhaps be the best option. This requires more thought. Regardless, there is still plenty to do without getting hung up on this issue. > If time permits, some GUI can be implemented in the LogVM designed specially > for viewing a list of all collected logs and easily opening them with a > suitable program in a DispVM or otherwise managing them. I think there should be sufficient time. Just sending & receiving logs is really not a multi-month task, and the GUI itself can be really quite simple. > This part includes a daemon (named log-exporter) that retrieves logs in real > time ... It can be written as a bash script or a C program. The majority of the Qubes code base is in Python. We prefer Python over C for safety reasons, and Python over bash for portability reasons. I would recommend Python be used for this as well. How familiar are you with Python? > The part on the LogVM side. > > [...] > > A program (named log-collector) that receives logs sent via the vchan/qrexec > ... Can be implemented as a bash script. Technically it certainly could, but Python is preferred. See note above, and https://www.qubes-os.org/doc/coding-style/#bash-specific-guidelines > A daemon (named log-compressor) that tracks the size of those directories > and is responsible for intelligent and secure log rotation. This tool > compresses medium-aged files to .zip, .gz, etc, deletes not the old ones and > doesn't touch recent ones. The daemon is started during the process of the > LogVM boot-up and works till it's shut down. Can be implemented as a C > program (because there is a very useful C library called libzip which is so > suitable for this daemon’s implementation). I'd recommend against zip as it is effectively a 2nd-class citizen on unix systems. gzip (/zlib) and xzip (/lzma) are much more common, especially for logs. Python has libraries for everything too. > # Timeline Your proposed timeline looks somewhat sparse to be honest. I encourage you to be more ambitious ;) > June - working with the Linux AppVM side: > May 30th - June 13th (two weeks) > Reading more about daemon programs in Linux to upgrade my knowledge. Daemons are not really any different from regular programs. Traditionally you'd fork twice & kill the middle process (known as backgrounding the process), but these days it's more common to not bother with that and let systemd handle all process management for you. > Determining fields of log entries which reasonably should be collected and > the exact way of getting them from log-collecting system. Sounds good. > Working with my knowledge in the data parsing. Not sure what this means. > June 14th - June 27th (two weeks) > Applying the new knowledge and writing the log-exporter daemon for the Linux > AppVM side. > July - working with the LogVM side: > June 28th - July 8th (one and a half weeks) > Determining requirements to the LogVM as to a system and solving related > problems. I think the way we'd want to implement the LogVM itself (as opposed to the services inside it) would be to describe it via Salt. [1][2][3] [1]: https://www.qubes-os.org/news/2015/12/14/mgmt-stack/ [2]: https://www.qubes-os.org/doc/salt/ [3]: https://docs.saltstack.com/en/latest/topics/tutorials/walkthrough.html Salt has a non-negligible learning curve though, so if you'd prefer to just work on the actual log handling and let someone else (most likely me) integrate it and do automatic creation of an actual LogVM by the installer, etc. I think that'd be fine. > Upgrading my knowledge in bash scripting and writing the log-collector > program in the right way. > July 10th - July 31th (three weeks) > Writing the log-compressor daemon for the LogVM side. I think you over-estimate the complexity & difficulty here. I'm not sure exactly why we need an independent daemon for compressing logs as opposed to just compressing them as they are received. We should also be careful to avoid reinventing the wheel [4]. [4]: http://www.linuxcommand.org/man_pages/logrotate8.html > August - working with a documentation, unforeseen circumstances and final > evaluations: > > August 1th - August 20th (two weeks) > Documentating the written project and dealing with unforeseen circumstances. Er... 1st to 20th is closer to 3 weeks. That's 3 weeks to write documentation? I think even two weeks is way more time than necessary. Dealing with unforeseen circumstances is implicitly assumed. I think you should aim to do more in this period (GUI perhaps?). If prior work turns out to be more difficult and things get pushed back, so be it. As long as you are making reasonable progress and maintain good communication, then you should pass the evaluations. Your plan as-is looks like you won't have much to do during the last month. > August 21st - August 29th (one week) > Final code submission and final evaluations. > I plan to test the written components separately and the system in general > all the time during the work, that’s why I don’t allocate any special time > for testing. Does this mean you intend to write unit tests, etc. as you go? Or just manual testing? > Frankly speaking, I have exams at my University until the end of June. I > hope to pass most of them ahead of time, so I think I'll have enough free > time to work on the project. But keeping this in mind, I find it more > reasonable not to plan any time-consuming tasks for June. So, a timeline > would be similar to something like this: All of June potentially gone sounds like it would make it difficult to make sufficient progress by the first midterm evaluation. You need to have a clear plan for how you will make it work. To quote the GSoC FAQ: [5] How much time does GSoC participation take? You are expected to spend around 30+ hours a week working on your project during the 3 month coding period. If you already have an internship, another summer job, or plan to be gone on vacation for more than a week during that time, GSoC is not the right program for you this year. and the GSoC mentor manual: [6] # Warning Signs Not enough hours in the day: If your student has a full-time job or is attempting to defend a graduate thesis during the summer, that is probably going to not work. Even though your student thinks they will have enough extra time, don't believe them. They won't. If your student has every single minute of every day completely booked, any unexpected event, such as getting sick or a family emergency, derails this plan beyond repair. If your student cannot commit to a specified time schedule, this is an immediate red flag that they need serious help with time management. I'm not bringing this up to discourage you or to say it can't be done, but to be honest it is somewhat concerning. Some more information about how you plan to handle both would be most welcome. [5]: https://developers.google.com/open-source/gsoc/faq#how_much_time_does_gsoc_participation_take [6]: http://write.flossmanuals.net/gsoc-mentoring/warning-signs/ Regards, Jean-Philippe -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/CABQWM_D3KwTeNa3Y9O1JL5c8osxrz%2Bw_JqGkyobeRfvk%2B0YmMA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
