Anthony,
I created an issue with more info
I am not sure if this is a bug, or just a way both ne/pytables treat strings
that need to touch an encoded value;
I found workaround by specifying the condvars to readWhere. Any more thoughts
on this?
thanks Jeff
https://github.com/PyTables/PyTables/issues/265
I can be reached on my cell (917)971-6387
________________________________
From: Anthony Scopatz <scop...@gmail.com>
To: Jeff Reback <j...@reback.net>
Cc: Discussion list for PyTables <pytables-users@lists.sourceforge.net>
Sent: Tuesday, June 4, 2013 6:39 PM
Subject: Re: [Pytables-users] pytable 30 - encoding
Hi Jeff,
Hmmm, Could you try doing the same thing on just an in-memory numpy array using
numexpr. If this succeeds it tells us that the problem is in PyTables, not
numexpr.
Be Well
Anthony
On Tue, Jun 4, 2013 at 11:35 AM, Jeff Reback <jreb...@yahoo.com> wrote:
Anthony,
>
>I am using numexpr 2.1 (latest)
>
>this is puzzling; doesn't matter what I pass (bytes or str) , same result?
>
>(column == 'str-2')
>> /mnt/code/arb/test/pytables-3.py(38)<module>()
>-> result = handle.root.test.table.readWhere(selector)
>(Pdb) handle.root.test.table.readWhere(selector)
>*** TypeError: string argument without an encoding
>(Pdb) handle.root.test.table.readWhere(selector.encode(encoding))
>*** TypeError: string argument without an encoding
>(Pdb)
>
>
> From: Anthony Scopatz <scop...@gmail.com>
>To: Jeff Reback <j...@reback.net>; Discussion list for PyTables
><pytables-users@lists.sourceforge.net>
>Sent: Tuesday, June 4, 2013 12:25 PM
>Subject: Re: [Pytables-users] pytable 30 - encoding
>
>
>
>Hi Jeff,
>
>
>Have you also updated numexpr to the most recent version? The error is coming
>from numexpr not compiling the expression correctly. Also, you might try
>making selector a str, rather than bytes:
>
>
>selector = "(column == 'str-2')"
>
>
>
>rather than
>
>
>selector = "(column == 'str-2')".encode(encoding)
>
>
>
>Be Well
>Anthony
>
>
>
>On Tue, Jun 4, 2013 at 8:51 AM, Jeff Reback <jreb...@yahoo.com> wrote:
>
>anthony,where am I going wrong here?
>>#!/usr/local/bin/python3
>>import tables
>>import numpy as np
>>import datetime, time
>>encoding = 'UTF-8'
>>test_file = 'test_select.h5'
>>handle = tables.openFile(test_file, "w")
>>node = handle.createGroup(handle.root, 'test')
>>table = handle.createTable(node, 'table', dict(
>>index = tables.Int64Col(),
>> column = tables.StringCol(25),
>>values = tables.FloatCol(shape=(3)),
>>))
>>
>># add data
>>r = table.row
>>for i in range(10):
>>r['index'] = i
>>r['column'] = ("str-%d" % (i % 5)).encode(encoding)
>>r['values'] = np.arange(3)
>>r.append()
>>table.flush()
>>handle.close()
>># read
>>handle =
tables.openFile(test_file,"r")
>>result = handle.root.test.table.read()
>>print("table data\n")
>>print(result)
>># where
>>print("\nselector\n")
>>selector = "(column == 'str-2')".encode(encoding)
>>print(selector)
>>result = handle.root.test.table.readWhere(selector)
>>print(result)
>>
>>and the following out:
>>
>>[sheep-jreback-/code/arb/test] python3 pytables-3.py
>>table data
>>[(b'str-0', 0, [0.0, 1.0, 2.0]) (b'str-1', 1, [0.0, 1.0, 2.0])
>>(b'str-2', 2, [0.0, 1.0, 2.0]) (b'str-3', 3, [0.0, 1.0, 2.0])
>>(b'str-4', 4, [0.0, 1.0, 2.0]) (b'str-0', 5, [0.0, 1.0, 2.0])
>>(b'str-1', 6, [0.0, 1.0, 2.0]) (b'str-2', 7, [0.0, 1.0, 2.0])
>>(b'str-3', 8, [0.0, 1.0, 2.0]) (b'str-4', 9, [0.0, 1.0, 2.0])]
>>selector
>>b"(column == 'str-2')"
>>Traceback (most recent call last):
>>File "pytables-3.py", line 37, in <module>
>>result =
handle.root.test.table.readWhere(selector)
>>File
>>"/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/_past.py",
>> line 35, in oldfunc
>>return obj(*args, **kwargs)
>>File
>>"/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/table.py",
>> line 1522, in read_where
>>self._where(condition, condvars, start, stop, step)]
>>File
>>"/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/table.py",
>> line 1484, in _where
>>compiled = self._compile_condition(condition, condvars)
>>File
>>"/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/table.py",
>> line 1358, in _compile_condition
>>compiled = compile_condition(condition, typemap, indexedcols)
>>File
>>"/usr/local/lib/python3.3/site-packages/tables-3.0.0-py3.3-linux-x86_64.egg/tables/conditions.py",
>> line 419, in compile_condition
>>func = NumExpr(expr, signature)
>>File
>>"/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py",
>> line 559, in NumExpr
>>precompile(ex, signature, context)
>>File
>>"/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py",
>> line 511, in precompile
>>constants_order, constants = getConstants(ast)
>>File
>>"/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py",
>> line 294, in getConstants
>>for a in constants_order]
>>File
>>"/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py",
>> line 294, in <listcomp>
>>for a in constants_order]
>>File
>>"/usr/local/lib/python3.3/site-packages/numexpr-2.1-py3.3-linux-x86_64.egg/numexpr/necompiler.py",
>> line 284, in convertConstantToKind
>>return kind_to_type[kind](x)
>>TypeError: string argument without an encoding
>>Closing remaining open files:
test_select.h5... done
>>
>>------------------------------------------------------------------------------
>>How ServiceNow helps IT people transform IT departments:
>>1. A cloud service to automate IT design, transition and operations
>>2. Dashboards that offer high-level views of enterprise services
>>3. A single system of record for all IT processes
>>http://p.sf.net/sfu/servicenow-d2d-j
>>_______________________________________________
>>Pytables-users mailing list
>>Pytables-users@lists.sourceforge.net
>>https://lists.sourceforge.net/lists/listinfo/pytables-users
>>
>>
>
>
> ------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. A cloud service to automate IT design, transition and operations
2. Dashboards that offer high-level views of enterprise services
3. A single system of record for all IT processes
http://p.sf.net/sfu/servicenow-d2d-j
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
