On Wed, Jun 6, 2012 at 11:18 AM, Jon Wilson <j...@fnal.gov> wrote:
> Hi Anthony,
>
> Well I think the issue at hand is that you are trying to support two
> disparate cases with one expression: sparse and dense selection. We have
> tools for dealing with these cases individually and performing out-of-core
> calculations. And if you know a priori which case you are going to fall
> into, you can do the right thing. So without doing anything special, I
> think medium-fast is probably the best and easiest thing that you can
> expect right now. (Though I would be delighted to be proved wrong on this
> point.)
>
> True enough. Sometimes I can't know anything a priori about the density
> of the selection, of course. And I'm happy to worry about the internals,
> but some of my colleagues, not so much ;)
>
I fully understand your position!
> but I think the ideal would be to have a .where type query operator that
>> returns Column objects or a Table object, with a "view" imposed in either
>> case.
>>
>
> We are very open to pull requests if you come up with an implementation
> of this that you like more ;).
>
> Very fair. We'll see if I can get to it. Is there any sort of
> guide-to-the-source to help me get started when and if that happens? I
> guess just the reference guide?
>
Not really as such. There is this list, and then there is
pytables-...@googlegroups.com for development specific questions and issues.
>
> I'll have a lot to learn before I can contribute usefully, I'm sure. I
> don't know enough about the implementation details yet to know: would a
> selection make the out-of-core performance gains from chunking and other
> things moot because you'd have to skip around too much?
>
Basically, I would say that if you were to write a function that does what
you described, we could take care of / help with the PyTables integration
issues. You could start by looking at the code for Expr [1] and seeing if
you could track down the one field issue. Or maybe there is a way to add
this functionality easily... Don't hesitate to ask if you run into
problems.
Be Well
Anthony
> Regards,
> Jon
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
