#25227: Avoid storing all Tor nodes in RAM ------------------------------+-------------------------------- Reporter: teor | Owner: (none) Type: enhancement | Status: new Priority: Medium | Milestone: Tor: 0.3.4.x-final Component: Core Tor/Tor | Version: Severity: Normal | Keywords: tor-client Actual Points: | Parent ID: Points: 5 | Reviewer: Sponsor: Sponsor8-can | ------------------------------+-------------------------------- {{{ <teor4> ahf: I've been thinking about reducing Tor's RAM usage by sampling from each consensus, rather than keeping it all in RAM <teor4> there are ways to stream the consensus, pick a weighted sample, and just keep those nodes <teor4> https://en.wikipedia.org/wiki/Reservoir_sampling <teor4> it would really help the iOS VPN, and other embedded impls <teor4> The catch is that if you run out of sampled nodes, you need to re- parse the consensus <+ahf> teor4: i had been thinking a bit in the back of my head if we could refactor the code that is used for accessing the consensus to be a bit more pluggable to experiment with different ways of backing them <+ahf> teor4: on ios i think we can mmap() files without them adding to the memory limit of the application too <teor4> that would be helpful, because we could only keep the selected node queues in RAM <teor4> if we got the queue size right, we could regenerate them once per consensus <+ahf> do you want to create a ticket with this? maybe add sponsor8-can to the keywords }}}
We could be smart about node selection, and just store some weights for each node. And then when we use too much RAM for nodes, we could evict older nodes. -- Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25227> Tor Bug Tracker & Wiki <https://trac.torproject.org/> The Tor Project: anonymity online
_______________________________________________ tor-bugs mailing list tor-bugs@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs