Re: [ZODB-Dev] grab zodb dict keys by search term
And that is exactly what I was looking for! Thanks! Tamer On 11.12.2013 14:27, Vincent Pelletier wrote: On Wed, 11 Dec 2013 12:27:19 +0200, Tamer Higazi wrote : If i would have 100.000 - 500.000 entries, where 150 would match, I don't want to go over those completely. So, if there is a pythonic way to get it solved on a performant way, or a ZODB way as well that would be wondefull. BTrees have a very nice API, which allow scanning only a subset of keys by providing min and max boundaries: for key, value in some_btree.items( min='aa', max='b', excludemax=True): # Do stuff should do what you want. http://pythonhosted.org/BTrees/ ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] grab zodb dict keys by search term
Hi Darryl, Thank you very much for your quick response. Perhaps it's my fault, the way I stored the data. The point is, that I am looking the fastest and performant way to grab the data from a big pool. I don't want to iterate the whole thing over and over again to get the matched entries. If i would have 100.000 - 500.000 entries, where 150 would match, I don't want to go over those completely. So, if there is a pythonic way to get it solved on a performant way, or a ZODB way as well that would be wondefull. Thanks a lot Tamer On 11.12.2013 03:56, Darryl Dixon - Winterhouse Consulting wrote: Hi Tamer, It depends on what you're trying to achieve. BTrees should not be mutated whilst iterating over them, so depending on how you have structured your data, you might do something like: for key in [x for x in BTree.keys() if x.startswith('aa')]: # Do stuff Or perhaps something else like: for key in list(BTree.keys()): if key['subkey'] == 5: yield BTree[key] ...Or some other pattern depending on your use case. The key here is that the ZODB really allows you to "cut out the middle man" where previously you might store data in an RDBMS and then do processing/transformation of that data in Python after extracting it with SQL (and then re-storing it again afterwards), instead now you can just directly process the data as if it was in Python objects waiting the entire time. Hope this helps. regards, Darryl Dixon Winterhouse Consulting Ltd Hi people! I am working a lot with IOB and OOB Trees as well with PersistentDict and PersistentList to store my Data in ZODB. I want to ask if there is a way to grab data from a dataset by search time. I know only the SQL way: select lower(COL1) from table1 where COL1 LIKE 'aa%' so usually I want to get the entryies only that starts with 'aa' do I have to iterate the entire dict with an interator or are there builtin functions that could accomplish those tasks for me ?! I would kindly thank you Tamer ___ For more information about ZODB, seehttp://zodb.org/ ZODB-Dev mailing list -ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] grab zodb dict keys by search term
Hi people! I am working a lot with IOB and OOB Trees as well with PersistentDict and PersistentList to store my Data in ZODB. I want to ask if there is a way to grab data from a dataset by search time. I know only the SQL way: select lower(COL1) from table1 where COL1 LIKE 'aa%' so usually I want to get the entryies only that starts with 'aa' do I have to iterate the entire dict with an interator or are there builtin functions that could accomplish those tasks for me ?! I would kindly thank you Tamer ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] modifying tree under root and save ?!
Works out of the box like sugar! Thank you! Tamer On 09.12.2013 19:21, Claudiu wrote: Use persistent.dict.PersistentDict and persistent.list.PersistentList instead of dict and list. These have hooks to dirty the object whenever anything changes inside them. ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] modifying tree under root and save ?!
Hi people! Inside the ZODB Tree I have a dictionary! root = { 'person: { 'amanda': [10,20,30] } } if I run: root['person']['amanda'].append(1000) and run: transaction.commit() close the db, start a new python shell nothing has changed so far. Why is that ?! Is there a way, to change the ZODB keys subcontent without to copy the entire entry in the memory ?! For a short reply I would thank you. Tamer ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB on ironPython?!
pypy as ironpython have internal JIT compilers. However, I will see later the results and post them here as pure ZODB would be ready. of course ironpython gener .NET intermadiate code (bytecode), but I don't care. We will see how and where it leads.. Tamer Am 26.10.2011 23:52, schrieb Leonardo Santagada: > On Wed, Oct 26, 2011 at 4:28 PM, Jim Fulton wrote: If this is a customer project, I'd recommend dropping either iron python or ZODB at this stage. >>> Why that?! As I know, ironpython creates with the JIT compiler bytecode >>> in realtime. >> >> Because this is going to be pretty new code and you'll be a trail blazer. > > AFAIK iron python is not considerably faster than cpython, so I think > you will end up with exactly the same performance problems of cpython > using a pure python ZODB on ironpython. > >>> what about pypy ?! >> >> It will be interesting to see how much of the performance pypy >> can gain back. > > As has been shown with the json module pypy can be really fast with > pure python code, in the next version of pypy the pure python json > module is faster than the c extension optimizer _json on cpython. So I > really think ZODB can be faster on pypy, the problem being how much > work would it be? > > But looking at a future of a pure python ZODB faster than the one we > have today (that is quite fast) would be awesome. Not so much for the > speed but to be able to implement everything in the language we all > love :) > > Good luck with your work. ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB on ironPython?!
dear Jim, Thank your for your quick answer. Am 26.10.2011 19:34, schrieb Jim Fulton: > On Wed, Oct 26, 2011 at 12:59 PM, Tamer Higazi wrote: >> Hey Jim! >> Hey just cool down. > > /me cools down > >> I love the ZODB database, and this toy is just >> beautiful. I iam willing to offer myself to support you porting the >> database in pure python, if you need volunteers. > > Thanks. I'll keep that in mind. > >> I really didn't mean, to come anybody stupid. > > I hope you don't mind me playfully amplifying your !s. :) > >> I am just thinking porting >> a customer project from python to iron-python where ZODB plays a role. > > If this is a customer project, I'd recommend dropping either > iron python or ZODB at this stage. Why that?! As I know, ironpython creates with the JIT compiler bytecode in realtime. > >> Is there much work to do?! > > Not sure. I suspect not. Tres created a Python implementation of > persistence and timestamp. I've got a BTrees patch to review. > > Keep in mind that these will run a lot slower than what you're used to. > If a performance is an issue, you may find yourself wanting to write > c# versions of these. > what about pypy ?! > Jim > Tamer ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB on ironPython?!
Hey Jim! Hey just cool down. I love the ZODB database, and this toy is just beautiful. I iam willing to offer myself to support you porting the database in pure python, if you need volunteers. I really didn't mean, to come anybody stupid. I am just thinking porting a customer project from python to iron-python where ZODB plays a role. Is there much work to do?! Tamer Am 20.10.2011 21:03, schrieb Jim Fulton: > On Thu, Oct 20, 2011 at 2:34 PM, Tamer Higazi wrote: >> You told amount of work had been done?! > > Yes!!! > >> In what stage are they now?! > > They aren't done yet!!! > >> Where can I get the latest informations according these cases? > > Here. You just saw it. :) > > Jim > ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB on ironPython?!
You told amount of work had been done?! In what stage are they now?! Where can I get the latest informations according these cases? thanks in advance Tamer Am 20.10.2011 19:59, schrieb Alan Runyan: > On Thu, Oct 20, 2011 at 12:39 PM, Tamer Higazi wrote: >> Hi! >> I am interisted getting ZODB to run on ironpython. Is that at the moment >> possible?! >> >> As I red, all python modules could be loaded on .NET without any >> problems as long they wouldn't need any compiled modules. Is that at the >> moment possible running ZODB on .NET ironpython?! >> > > Not until a pure Python implementation of persistent and BTree's exist. > There has been substantial amount of work done but it more is required. > > runyaga ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ZODB on ironPython?!
Hi! I am interisted getting ZODB to run on ironpython. Is that at the moment possible?! As I red, all python modules could be loaded on .NET without any problems as long they wouldn't need any compiled modules. Is that at the moment possible running ZODB on .NET ironpython?! thanks Tamer ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev