On 05/06/2011 02:14 PM, Shane Hathaway wrote:
> However, there is a different class of problems that prefetching could
> help solve. Let's say you have pages with a lot of little pieces on it,
> such as a comment page with a profile image for every comment. It would
> be useful to tell ZODB to lo
On 05/06/2011 02:38 PM, Jim Fulton wrote:
> If there is memory pressure and you take away ram for a ram disk, then you're
> going to start swapping, which will give you other problems.
In my experience, Linux moves pieces of the ZEO cache out of RAM long
before it starts swapping much.
> I tried
On Fri, May 6, 2011 at 4:21 PM, Shane Hathaway wrote:
> On 05/06/2011 02:14 PM, Jim Fulton wrote:
>>>
>>> It sounds like you primarily need a bigger and faster cache. If you
>>> want to make minimal changes to your setup, try increasing the size of
>>> your ZEO cache and store the ZEO cache on ei
On Fri, May 6, 2011 at 10:14 PM, Shane Hathaway wrote:
> From my experience, most people who want ZODB to be faster want Zope
> catalogs in particular to be faster. I don't think prefetching can make
> catalogs much faster, though.
I've spent a lot of time lately on making ZCatalog faster. The
On 05/06/2011 02:14 PM, Jim Fulton wrote:
>> It sounds like you primarily need a bigger and faster cache. If you
>> want to make minimal changes to your setup, try increasing the size of
>> your ZEO cache and store the ZEO cache on either a RAM disk (try mount
>> -t tmpfs none /some/path) or a sol
On Fri, May 6, 2011 at 4:07 PM, Shane Hathaway wrote:
> On 05/06/2011 06:22 AM, Pedro Ferreira wrote:
>> But isn't RelStorage supposed be slower than FileStorage/ZEO?
>
> No, every measurement I've tried suggests RelStorage (with PostgreSQL or
> MySQL) is faster than ZEO on the same hardware. ZEO
On 05/06/2011 10:18 AM, Paul Winkler wrote:
> On Fri, May 06, 2011 at 05:19:25PM +0200, Matthias wrote:
>> It would be cool if you could give a hint to ZEO somehow to prefetch a
>> certain set of objects along with their subobjects and then return
>> everything in one batch. This way you avoid all
On 05/06/2011 06:22 AM, Pedro Ferreira wrote:
> But isn't RelStorage supposed be slower than FileStorage/ZEO?
No, every measurement I've tried suggests RelStorage (with PostgreSQL or
MySQL) is faster than ZEO on the same hardware. ZEO has certainly
gotten faster lately, but RelStorage still see
> It would be cool if you could give a hint to ZEO somehow to prefetch a
> certain set of objects along with their subobjects and then return
> everything in one batch. This way you avoid all the round-trips when you
> discover you want to retrieve a suboject.
+1
But I guess that could be tricky
Hello,
> I don't think you've mentioned what the latency is like between your
> ZEO server and clients. What's the ping time look like?
~0.115 ms
> As an example, let's say your average object size is only 1k, but due
> to network latency it takes 5ms per fetch. Let's say your slow page
> loa
On Fri, May 06, 2011 at 05:19:25PM +0200, Matthias wrote:
> It would be cool if you could give a hint to ZEO somehow to prefetch a
> certain set of objects along with their subobjects and then return
> everything in one batch. This way you avoid all the round-trips when you
> discover you wan
Am 06.05.2011, 17:12 Uhr, schrieb Paul Winkler :
> On Fri, May 06, 2011 at 12:19:39PM +0200, Pedro Ferreira wrote:
>> A quick check with nethogs shows values of network usage oscillating
>> between 100 and 200 KB/s. But I guess that if I were loading an
>> excessive amount of data, this value woul
On Fri, May 06, 2011 at 12:19:39PM +0200, Pedro Ferreira wrote:
> A quick check with nethogs shows values of network usage oscillating
> between 100 and 200 KB/s. But I guess that if I were loading an
> excessive amount of data, this value would be higher, no?
So throughput isn't the problem.
I d
On Fri, May 6, 2011 at 6:19 AM, Pedro Ferreira
wrote:
> Hello.
>
>> Is your storage server CPU bound?
>
> load average: 1.47, 1.34, 1.20
>
> mpstat:
> """
> 11:40:13 AM CPU %user %nice %sys %iowait %irq %soft %steal
> %idle intr/s
> 11:40:13 AM all 5.82 0.00 0.69 1.11
On Fri, May 6, 2011 at 2:22 PM, Pedro Ferreira
wrote:
> That's hard to do for a project that is already 8 or 9 years old, as you
> can see in the attached file, we've got have many cases that fall
> outside your limits. I've noticed, for instance, that pages that involve
> the loading of >200 MaKa
Hello,
> Query fewer objects from the database. Make sure you don't store lots
> of tiny persistent objects in the database, I'd aim for storing data
> in chunks of 8-32kb or use blobs for larger objects. Remember that
> ZODB is a key/value storage for the most part. Model your data
> accordingly.
Hello.
> Is your storage server CPU bound?
load average: 1.47, 1.34, 1.20
mpstat:
"""
11:40:13 AM CPU %user %nice%sys %iowait%irq %soft %steal
%idleintr/s
11:40:13 AM all5.820.000.691.110.010.120.00
92.25548.52
"""
I guess it's not very
On Thu, May 5, 2011 at 4:43 AM, Pedro Ferreira
wrote:
...
> Since we are talking about speed, does anyone have any tips on making
> ZODB (in general) faster?
It's hard to give general advice. The basic thing to be aware of is
that lods from ZEO cache have some cost, but are fairly cheap
especi
On Thu, May 5, 2011 at 10:43 AM, Pedro Ferreira
wrote:
> Since we are talking about speed, does anyone have any tips on making
> ZODB (in general) faster?
Query fewer objects from the database. Make sure you don't store lots
of tiny persistent objects in the database, I'd aim for storing data
in
19 matches
Mail list logo
