RE: Script profiling details

2022-01-11 Thread Joseph L. Casale
> You might try `py-spy`. That worked well, I started trying to get more data from the profile output with the stats module but didn't quite get there. Thank you everyone, jlc -- https://mail.python.org/mailman/listinfo/python-list

Script profiling details

2022-01-10 Thread Joseph L. Casale
I am trying to track down a slow script startup time. I have executed the script using `python -m cProfile -o profile /path/script.py` and read through the results, but the largest culprit only shows various built-ins. I expected this given the implementation, but I was hoping to get some finer

RE: How to implement logging for an imported module?

2021-03-07 Thread Joseph L. Casale
> I couldn't find any information on how to implement logging in a library that > doesn't know the name of the application that uses it. How is that done? Hello, That's not how it works, it is the opposite. You need to know the name of its logger, and since you imported it, you do. Logging is

Overriding property types on instances from a library

2021-02-20 Thread Joseph L. Casale
I have some code that makes use of the typing module. This code creates several instances of objects it creates from a library that has some issues. For example, I have multiple list comps that iterate properties of those instance and the type checker fails with: Expected type

asyncio cancellation pattern

2020-12-28 Thread Joseph L. Casale
I've started writing some asyncio code in lieu of using threads and managing concurrency and primitives manually. Having spent a lot of time using c#'s async implementation, I am struggling to see an elegant pattern for implementing cancellation. With the necessity for the loop (that of which I

RE: pexpect with kadmin

2020-12-23 Thread Joseph L. Casale
> If you have windows 10 can you use Windows Subsystem for Linux (WSL) > to install one of the Linux distros and use that? Interesting idea, sadly I am too far past the deadline on this to go through the red tape needed to get that in place. Thanks, jlc --

pexpect with kadmin

2020-12-22 Thread Joseph L. Casale
Anyone ever used pexpect with tooling like kadmin and have insight into how to manage interacting with it? After setting up debug logging, I was able to adjust the expect usage to get the input and output logs to at least appear correct when setting a password for a principal, however even with a

RE: setuptools issue

2020-12-17 Thread Joseph L. Casale
> Installed on this Slackware-14.2/x86_64 workstation with python-3.9.1 are: > python-setuptools-22.0.5-x86_64-1 I just ran into this recently, I don't recall the actual source but it was the version of setuptools having been so old. Your version is from Jun 3, 2016... Update it, that was what

RE: Python Client Rest API Invocation - POST with empty body - Invalid character found in method name [{}POST]. HTTP method names must be tokens

2020-11-20 Thread Joseph L. Casale
> Invalid character found in method name [{}POST]. HTTP method names must be > tokens. /snip > I could see in from wireshark dumps it looked like - {}POST > HTTP/1.1 The error message and your own debugging indicate the error. Your method *name* is {}POST, you have somehow included two

Constructing mime image attachment

2020-05-28 Thread Joseph L. Casale
I have some json encoded input for nodemailer (https://nodemailer.com/message/embedded-images) where the path key is a string value which contains the base64 encoded data such as: { html: 'Embedded image: ', attachments: [{ filename: 'image.png', path:

policy based variable composition

2020-04-03 Thread Joseph L. Casale
I am looking to replace a home built solution which allows a program to derive a series of variable values through configuration or policy. The existing facility allows dependences so one of the requested variables can depend on another, they are ordered and computed. It also allows callbacks so

RE: Class initialization with multiple inheritance

2019-07-20 Thread Joseph L. Casale
-Original Message- From: Barry Scott Sent: Tuesday, July 16, 2019 11:53 AM To: Joseph L. Casale Cc: python-list@python.org Subject: Re: Class initialization with multiple inheritance > And here is the MRO for LeftAndRight. > > >>> import m > LeftAndRight.__ini

Class initialization with multiple inheritance

2019-07-15 Thread Joseph L. Casale
I am trying to find explicit documentation on the initialization logic for a Base class when multiple exist. For the example in the documentation at https://docs.python.org/3/tutorial/classes.html#multiple-inheritance, if Base1 and Base2 both themselves inherited from the same base class, only

Type hinting

2019-04-08 Thread Joseph L. Casale
Hi, Is it possible to associate combinations of types for a given signature, for example: T = TypeVar('T', Foo, Bar, Baz) S = TypeVar('S', FooState, BarState, BazState) closure = 'populated dynamically' def foo(factory: Callable[[List[T], str], None], state: S) -> List[T]: results = []

RE: asyncio Question

2019-03-15 Thread Joseph L. Casale
> -Original Message- > From: Python-list bounces+jcasale=activenetwerx@python.org> On Behalf Of Simon > Connah > Sent: Thursday, March 14, 2019 3:03 AM > To: Python > Subject: asyncio Question > > Hi, > > Hopefully this isn't a stupid question. For the record I am using Python >

RE: Serializing complex objects

2018-09-21 Thread Joseph L. Casale
-Original Message- From: Python-list On Behalf Of Rhodri James Sent: Friday, September 21, 2018 11:39 AM To: python-list@python.org Subject: Re: Serializing complex objects > Depending on what exactly your situation is, you may be able to use the > pickle module (in the standard library)

Serializing complex objects

2018-09-21 Thread Joseph L. Casale
I need to serialize a deep graph only for the purposes of visualizing it to observe primitive data types on properties throughout the hierarchy. In my scenario, I cannot attach a debugger to the process which would be most useful. Using json is not the easiest as I need to chase endless custom

RE: lxml namespace as an attribute

2018-08-15 Thread Joseph L. Casale
-Original Message- From: Python-list On Behalf Of Skip Montanaro Sent: Wednesday, August 15, 2018 3:26 PM To: Python Subject: lxml namespace as an attribute > Much of XML makes no sense to me. Namespaces are one thing. If I'm > parsing a document where namespaces are defined at the top

[issue34006] Windows HTML Help (chm) has fixed line length

2018-07-02 Thread Joseph L. Casale
Change by Joseph L. Casale : -- nosy: -jcasale ___ Python tracker <https://bugs.python.org/issue34006> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34006] Windows HTML Help (chm) has fixed line length

2018-06-30 Thread Joseph L. Casale
Change by Joseph L. Casale : -- nosy: +jcasale ___ Python tracker <https://bugs.python.org/issue34006> ___ ___ Python-bugs-list mailing list Unsubscribe:

RE: Issue with python365.chm on window 7

2018-04-24 Thread Joseph L. Casale
From: Brian Gibbemeyer <bgibb...@us.ibm.com> Sent: Tuesday, April 24, 2018 12:36 PM To: Joseph L. Casale <jcas...@activenetwerx.com> Cc: python-list@python.org Subject: RE: Issue with python365.chm on window 7 > I right clicked on the file, no option to unblock. Sorry, ch

RE: Issue with python365.chm on window 7

2018-04-24 Thread Joseph L. Casale
-Original Message- From: Python-list On Behalf Of Brian Gibbemeyer Sent: Tuesday, April 24, 2018 11:01 AM To: Ethan Furman Cc: python-list@python.org Subject: Re: Issue with python365.chm on window 7 > The file at >

Re: Instance variables question

2018-04-16 Thread Joseph L. Casale
From: Python-list on behalf of Irv Kalb Sent: Monday, April 16, 2018 10:03 AM To: python-list@python.org Subject: Instance variables question   > class PartyAnimal(): >     x = 0 > >     def party(self): >    

RE: Python regex pattern from array of hex chars

2018-04-13 Thread Joseph L. Casale
-Original Message- From: Python-list On Behalf Of MRAB Sent: Friday, April 13, 2018 12:05 PM To: python-list@python.org Subject: Re: Python regex pattern from array of hex chars > Use re.escape: > > regex = re.compile('[^{}]+'.format(re.escape(''.join(c for c in > character_class

Python regex pattern from array of hex chars

2018-04-13 Thread Joseph L. Casale
I have an array of hex chars which designate required characters. and one happens to be \x5C or "\". What foo is required to build the pattern to exclude all but: regex = re.compile('[^{}]+'.format(''.join(c for c in character_class))) I would use that in a re.sub to collapse and replace all but

RE: issues when buidling python3.* on centos 7

2018-03-26 Thread Joseph L. Casale
-Original Message- From: Python-list On Behalf Of joseph pareti Sent: Sunday, March 25, 2018 10:15 AM To: python-list@python.org Subject: issues when buidling python3.* on centos 7 > The following may give a clue because of inconsistent python versions: > > [joepareti54@xxx ~]$ python

Re: Context manager on method call from class

2018-03-15 Thread Joseph L. Casale
From: Python-list on behalf of Rob Gaddi Sent: Thursday, March 15, 2018 12:47 PM To: python-list@python.org Subject: Re: Context manager on method call from class   > from contextlib import

Context manager on method call from class

2018-03-15 Thread Joseph L. Casale
I have a class which implements a context manager, its __init__ has a signature and the __enter__ returns an instance of the class. Along with several methods which implement functionality on the instance, I have one method which itself must open a context manager against a call on an instance

RE: Python DB API - commit() v. execute("commit transaction")?

2017-05-30 Thread Joseph L. Casale
-Original Message- From: Python-list [mailto:python-list- bounces+jcasale=activenetwerx@python.org] On Behalf Of Ian Kelly Sent: Tuesday, May 30, 2017 8:12 AM To: Python Subject: Re: Python DB API - commit() v. execute("commit transaction")? > There's no

RE: Regular Expressions, Speed, Python, and NFA

2017-04-14 Thread Joseph L. Casale
-Original Message- From: Python-list [mailto:python-list- bounces+jcasale=activenetwerx@python.org] On Behalf Of Malik Rumi Sent: Friday, April 14, 2017 9:12 AM To: python-list@python.org Subject: Regular Expressions, Speed, Python, and NFA > I am running some tests using the site

RE: What are your opinions on .NET Core vs Python?

2017-01-30 Thread Joseph L. Casale
> C# hardly seems any better than Java to me as far as a language goes. Which sounds pretty good to me, they are both high performance, mature and rich languages. > Being forced into working with classes even when they are not > appropriate is jarring. And 100% irrelevant, it doesn't prevent

RE: What are your opinions on .NET Core vs Python?

2017-01-30 Thread Joseph L. Casale
> Python still has my heart, but .NET Core tempts me. One great thing of > coding in C# would be no GIL. Seriously, check out the benchmarks at https://github.com/aspnet/benchmarks. I think aside from the obvious, you'll find the Razor engine and the overall library to be a pleasure to work

RE: What are your opinions on .NET Core vs Python?

2017-01-30 Thread Joseph L. Casale
> What do you mean by "both platforms"? Python scripts already run on > three major operating systems (Win/Lin/Mac) and a good number of > less-popular OSes; a well-written Python script will run in four major > Pythons (CPython, PyPy, Jython, IronPython) and a number of others; > and all manner

RE: What are your opinions on .NET Core vs Python?

2017-01-29 Thread Joseph L. Casale
> What .NET APIs are anticipated to be released that aren't on the > official CLI list now: > https://en.wikipedia.org/wiki/List_of_CLI_languages#Current_Languages, > and/or, are .NET supported languages expected to expand beyond the CLI > list? I think this (and the last point) misinterprets the

RE: What are your opinions on .NET Core vs Python?

2017-01-29 Thread Joseph L. Casale
> .NET is a library that can be used from many languages, including Python. No. .NET Core (what the OP asked about which is not .NET) is a cross-platform framework. Obviously Python and .NET differ in runtime semantics with respect to the original source code, however they are now roughly

RE: Sockets: IPPROTO_IP not supported

2017-01-16 Thread Joseph L. Casale
> Trying to sniff Ethernet packets, I do this: > >s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP) > > but it results in this: > > $ sudo python3 sniff_survey.py > Traceback (most recent call last): > File "sniff_survey.py", line 118, in > s =

RE: multiprocessing.Process call blocks other processes from running

2017-01-14 Thread Joseph L. Casale
> while True: >for client in clients: > stats = ThreadStats() > stats.start() > p = Process(target=getWhispererLogsDirSize, args=(client,queue,)) > jobs.append(p) > p.start() > p.join() You start one client then join before starting the next... Start them all

RE: Can not run the Python software

2017-01-13 Thread Joseph L. Casale
> And this is coming up a lot. This is something that should already be > on all supported versions of Windows if Windows updates are done, right? No, it's not an update. You install the runtime *if* you need it. > but maybe it's time that the > Python installer bundles the redistributable

RE: Can not run the Python software

2017-01-13 Thread Joseph L. Casale
> Just downloaded Python 3.6.0 2016-12-23 and PyCharm. Tried to run the "Hello > World" program and got the following message: > "Process finished with exit code 1073741515 (0xC135)" > I am using Windows 8.1 on an HP ENVY Touchsmart Notebook (64-bit OS, > x64-based processor). If you track

RE: reactiveX vs Async

2017-01-11 Thread Joseph L. Casale
> So you are saying that nuget-ing .Net core would be a workable pre-requisite > for > Rx on mono? Microsoft open sourced .net a while ago. With that came the movement to bring .net to other platforms. https://en.wikipedia.org/wiki/.NET_Framework#.NET_Core As its currently being heavily

RE: reactiveX vs Async

2017-01-11 Thread Joseph L. Casale
> One more question: Do you know if (and how much) of these things would work > in Linux/C# (ie mono)? Mono, I forgot what that is when .net core debuted:) Looks like the .net Rx guys have a port, https://github.com/Reactive-Extensions/Rx.NET/issues/148 A package for .net core is up on nuget.

RE: reactiveX vs Async

2017-01-11 Thread Joseph L. Casale
> Thanks Joseph > Trouble is there is stew of technologies/languages… > (meta)-stewed with more abstract concepts, eg push vs pull, > Enumerable-Observable > duality, continuous vs discrete time > The last causing its own share of confusion with “functional reactive > programming” (FRP) meaning

RE: reactiveX vs Async

2017-01-11 Thread Joseph L. Casale
>> There is the recent flurry around the new async additions to python > > I meant to add: “… which I dont pretend to understand…” Try these links on for size: https://msdn.microsoft.com/en-us/library/hh242982(v=vs.103).aspx which links to

RE: reactiveX vs Async

2017-01-11 Thread Joseph L. Casale
> Try these links on for size: > > https://msdn.microsoft.com/en-us/library/hh242982(v=vs.103).aspx which links > to > https://msdn.microsoft.com/en-us/library/hh242983(v=vs.103).aspx near the end. These two SO threads have a variation of pretty good explanations:

thread local storage

2016-10-26 Thread Joseph L. Casale
Looks like the shipped implementation doesn't give access to all the discrete copies of tls for us in a case where a context manager needs to perform any cleanup. Does something exists where I can leverage this feature or do I need to roll my own? Thanks, jlc --

RE: MySQL connector issue

2016-10-23 Thread Joseph L. Casale
> Interesting. Generally, I allocate cursors exactly at the same time as I open > transactions; > not sure if this works with the mysql connector, but with psycopg2 > (PostgreSQL), my code looks like this: > > with conn, conn.cursor() as cur: > cur.execute(...) > ... = cur.fetchall() >

RE: MySQL connector issue

2016-10-23 Thread Joseph L. Casale
> Perhaps you simplified too much, but changes between the select and the > update could be lost. I think you need at least three states: > > 1 mark rows where baz is null (by setting baz to some value other than NULL > or 42, 24, say: set baz = 24 where baz is NULL) > 2 show marked rows (select

MySQL connector issue

2016-10-23 Thread Joseph L. Casale
I have some code that I am testing on Windows without c extensions which runs on a RHEL server with c extensions. In a simplified test case as follows: connection = mysql.connector.connect(...) cursor = connection.cursor(cursor_class=MySQLCursorDict) while True: cursor.execute('SELECT foo,biz

[issue14965] super() and property inheritance behavior

2016-09-11 Thread Joseph L. Casale
Changes by Joseph L. Casale <jcas...@activenetwerx.com>: -- nosy: +jcasale ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue14965> ___

RE: SOAP and Zeep

2016-07-29 Thread Joseph L. Casale
> Or any other libraries that can be recommended? I'd recommend Spyne, code and docs are good, but more importantly the lead dev is responsive and very helpful. Can't speak highly enough about him... http://spyne.io/ hth, jlc -- https://mail.python.org/mailman/listinfo/python-list

RE: argparse and subparsers

2016-06-27 Thread Joseph L. Casale
> Not sure if this fits the bill, or makes sense here, but I came cross > "docopt" which touts itself as a "Command-line interface description > language". I used it in a project and it seems to be pretty easy to use > as well as elegant. It stores the arguments & values as a dictionary, > keyed

argparse and subparsers

2016-06-26 Thread Joseph L. Casale
I have some code where sys.argv is sliced up and manually fed to discrete argparse instances each with a single subparser. The reason the discrete parsers all having a single subparser was to make handling the input simpler, the first arg in the slice could be left in. This has become

RE: Ctypes c_void_p overflow

2016-05-05 Thread Joseph L. Casale
> I generally avoid c_void_p because its lenient from_param method > (called to convert arguments) doesn't provide much type safety. If a > bug causes an incorrect argument to be passed, I prefer getting an > immediate ctypes.ArgumentError rather than a segfault or data > corruption. For example,

Ctypes c_void_p overflow

2016-05-05 Thread Joseph L. Casale
I have CDLL function I use to get a pointer, several other functions happily accept this pointer which is really a long when passed to ctypes.c_void_p. However, only one with same type def in the prototype overflows. Docs suggest c_void_p takes an int but that is not what the first call

RE: Static caching property

2016-03-22 Thread Joseph L. Casale
> If a method call on any instance defines the return value for > all instances, then this method likely should be a class method -- > and use a class attribute to store the result -- something like this: > > class C(object): > > _cache = {} > > @classmethod > def f(cls, ...): >

Re: Static caching property

2016-03-21 Thread Joseph L. Casale
> One solution is to use descriptor protocol on the class, which means > using a metaclass. I'm not sure it's the best option, but it is an > option. I will look at that, I wonder if however I am not over complicating it: class Foo: _bar = None @property def expensive(self):

Re: Static caching property

2016-03-21 Thread Joseph L. Casale
> I think Joseph is using "static" in the Java sense of being associated with > the class rather than an instance. (In Java, members of classes must be > known at compile-time.) Yup, so a single value on the class itself, not instance specific. > But what you can do is have the property refer to

Static caching property

2016-03-21 Thread Joseph L. Casale
With non static properties, you can use a decorator that overwrites the method on the instance with an attribute containing the methods return effectively caching it. What technique for a static property can be used to accomplish what the descriptor protocol does? I need to cache the results of

Re: Passing data across callbacks in ThreadPoolExecutor

2016-02-19 Thread Joseph L. Casale
> It's still not clear to me specifically what you're trying to do. It > would really help if you would describe the problem in more detail. > Here's what I think you're trying to do: > > 1) Submit a task to a ThreadPoolExecutor and get back a future. > > 2) When the task is complete, submit

Re: Passing data across callbacks in ThreadPoolExecutor

2016-02-18 Thread Joseph L. Casale
On Thur, Feb 17, 2016 at 9:24 AM, Ian Kelly wrote: >> What is the pattern for chaining execution of tasks with ThreadPoolExecutor? >> Callbacks is not an adequate facility as each task I have will generate new >> output. > > Can you specify in more detail what your use

Passing data across callbacks in ThreadPoolExecutor

2016-02-16 Thread Joseph L. Casale
What is the pattern for chaining execution of tasks with ThreadPoolExecutor? Callbacks is not an adequate facility as each task I have will generate new output. Thanks, jlc -- https://mail.python.org/mailman/listinfo/python-list

Re: Subclassing tuple and introspection

2015-12-02 Thread Joseph L. Casale
> If you're not already familiar with collections.namedtuple, have a > look at it, as it sounds like just naming the fields may be all that > you need. You can also subclass it further to add methods if desired. Yeah, all the types in these collections are named tuples... The collection itself

Subclassing tuple and introspection

2015-12-02 Thread Joseph L. Casale
I need to return a collection of various types, since python doesn't have the terse facility of extension methods like C#, subclassing tuple and adding a method seems like a terse way to accommodate this. However, if the method returns one element of the collection, how can one enable

Enum class

2015-10-15 Thread Joseph L. Casale
Is it possible to override __contains__ from the meta class in the derived class with the Enum type? Thanks, jlc -- https://mail.python.org/mailman/listinfo/python-list

Re: Enum class

2015-10-15 Thread Joseph L. Casale
import enum class M(enum.EnumMeta): >... def __contains__(self, value): >... print("checking", value) >... return super().__contains__(value) >... class Colors(enum.Enum, metaclass=M): >... red = 1 >... green = 2 >... blue = 3 >... Colors.red in

RE: Enum class

2015-10-15 Thread Joseph L. Casale
> Like that? > > >>> class M2(enum.EnumMeta): >... def __contains__(self, value): >... print(value, "? never", sep="") >... return False >... > >>> Colors.__class__ > > >>> Colors.red in Colors > checking Colors.red > True > >>> Colors.__class__ = M2 > >>> Colors.red in

Modifying signature of ctor in class

2015-09-24 Thread Joseph L. Casale
I have a class factory where I dynamically add a constructor to the class output. The method is a closure and works just fine, however to accommodate the varied input its signature is (*args, **kwargs). While I modify the doc strings, the ctor sig is not optimal. Without building this a string

Re: Modifying signature of ctor in class

2015-09-24 Thread Joseph L. Casale
> You can use the FunctionType class found in the types module (and its > friends) to create functions on the run. > And you can use the 'inspect' module to inspect existing functions, if you > need to base the new function on an existing one. Hi Gal, Seems the types module docs do not even

Re: Modifying signature of ctor in class

2015-09-24 Thread Joseph L. Casale
> I don't think you can easily change the function's own definition, > other than by using eval (or equivalent shenanigans, like crafting > your own bytecode); but as of Python 3.something, the help() function > looks for a __wrapped__ attribute and will take function args from > that instead of

Re: Modifying signature of ctor in class

2015-09-24 Thread Joseph L. Casale
> py> from inspect import Signature, Parameter > py> def foo(*args, **kwargs): pass > ... > py> foo.__signature__ = Signature([Parameter('x', > Parameter.POSITIONAL_OR_KEYWORD), Parameter('y', > Parameter.KEYWORD_ONLY)]) > py> help(foo) > Help on function foo in module __main__: > > foo(x, *, y)

Re: Writing a module to abstract a REST api

2015-09-18 Thread Joseph L. Casale
Hi Sven, >> The problem now comes from the following: >> >> # foo now contains a handle to the remote api. >> foo = InstanceOfApiWrapper() > > Is it necessary to have an instance of that API? Just curiosity here. Not always but often as the pattern here might rely on a handle to a c types based

Re: Writing a module to abstract a REST api

2015-09-18 Thread Joseph L. Casale
> Well, I would be interested in seeing such a module as well. > > Most modules and frameworks, I know, providing REST and interacting with > REST are more like traditional SOAP-like web services. You got your > functions which have a 1-to-1 correspondence with some resource URLs and > that's it.

Re: Writing a module to abstract a REST api

2015-09-17 Thread Joseph L. Casale
> There is https://pypi.python.org/pypi/librabbitmq ? Hi Jon, That is the AMQP client that utilizes the c extensions, I am writing a module to interact with a plugin that exposes a REST API. So I am not really after anything AMQP specific, just a pointer to a project that abstracts anything with

Writing a module to abstract a REST api

2015-09-17 Thread Joseph L. Casale
I need to write a module to abstract the RabbitMQ HTTP REST api. Before I do this, I would like to see how other projects have done similar in the hopes I make something consistent and generic etc. Does anyone regularly work with a library that abstracts a REST API and can recommend it for

Re: Writing a module to abstract a REST api

2015-09-17 Thread Joseph L. Casale
> If I understand you: > http://www.python-requests.org/en/latest/ > > is an example of what you are looking for? > > It's great. > > Also check out > http://cramer.io/2014/05/20/mocking-requests-with-responses/ > > if you need to mock requests. Hi Laura, The twitter samples Jon sent were

Attaching to python process in alternate windows session

2015-08-26 Thread Joseph L. Casale
A while ago I stumbled on a workaround for this, but forgot. Anyone know the trick to accomplish this? Thanks, jlc -- https://mail.python.org/mailman/listinfo/python-list

RE: return types from library api wrappers

2015-08-17 Thread Joseph L. Casale
Current practice is a NamedTuple for python code or the C equivalent. I forget the C name, but I believe it is used by os.stat Hi Terry, Ok, that is what I will go with. Thanks for the confirmation, jlc -- https://mail.python.org/mailman/listinfo/python-list

return types from library api wrappers

2015-08-16 Thread Joseph L. Casale
What's the accepted practice for return types from a c based API Python wrapper? I have many methods which return generators which yield potentially many fields per iteration. In lower level languages we would yield a struct with readonly fields. The existing implementation returns a dict which I

Module load times

2015-08-13 Thread Joseph L. Casale
I have an auto generated module that provides functions exported from a c dll. Its rather large and we are considering some dynamic code generation and caching, however before I embark on that I want to test import times. As the module is all auto generated through XSL, things like __all__ are

Re: Module load times

2015-08-13 Thread Joseph L. Casale
Hi Stefan, How is the DLL binding implemented? Using ctypes? Or something else? It is through ctypes. Obviously, instantiating a large ctypes wrapper will take some time. A binary module would certainly be quicker here, both in terms of import time and execution time. Since you're

RE: Module load times

2015-08-13 Thread Joseph L. Casale
Importing is not the same as instantiation. When you import a module, the code is only read from disk and instantiated the first time. Then it is cached. Subsequent imports in the same Python session use the cached version. I do mean imported, in the original design there were many ctype

Callbacks with concurrent.futures

2015-03-12 Thread Joseph L. Casale
I have a ProcessPoolExecutor for which I am attaching multiple callbacks. As this must be process based and not thread based, I don't have the luxury communication between threads. Without a queue, does something inherent exist in concurrent futures that allows me to accumulate some data from the

Re: Callbacks with concurrent.futures

2015-03-12 Thread Joseph L. Casale
ProcessPoolExecutor is built on the multiprocessing module, so I expect you should be able to use multiprocessing.Queue or multiprocessing.Pipe in place of threading.Queue. Hi Ian, Yeah I am using a Manager.Queue as the method polling the queue is itself in a process. I just wondered if there

Hashed lookups for tabular data

2015-01-19 Thread Joseph L. Casale
I have some tabular data for example 3 tuples that I need to build a container for where lookups into any one of the three fields are O(1). Does something in the base library exist, or if not is there an efficient implementation of such a container that has been implemented before I give it a go?

Re: Hashed lookups for tabular data

2015-01-19 Thread Joseph L. Casale
So presumably your data's small enough to fit into memory, right? If it isn't, going back to the database every time would be the best option. But if it is, can you simply keep three dictionaries in sync? Hi Chris, Yeah the data can fit in memory and hence the desire to avoid a trip here.

Re: Hashed lookups for tabular data

2015-01-19 Thread Joseph L. Casale
Why not take a look at pandas as see if there's anything there you could use? Excellent docs here http://pandas.pydata.org/pandas-docs/stable/ and the mailing list is available at gmane.comp.python.pydata amongst other places. Mark, Actually it was the first thing that came to mind. I did

Re: Hashed lookups for tabular data

2015-01-19 Thread Joseph L. Casale
If you want an sql-like interface, you can simply create an in-memory sqlite3 database. import sqlite3 db = sqlite3.Connection(':memory:') You can create indexes as you need, and query using SQL. Later, if you find the data getting too big to fit in memory, you can switch to using an

Re: Hashed lookups for tabular data

2015-01-19 Thread Joseph L. Casale
The IDs of the objects prove that they're actually all the same object. The memory requirement for this is just what the dictionaries themselves require; their keys and values are all shared with other usage. Chris, I would have never imagined that, much appreciated for that! jlc --

Argparse defaults

2015-01-04 Thread Joseph L. Casale
Does a facility exist to add an argument with a default being a function that leverages the final parsed Namespace? For example: group = parser.add_argument_group(' some_group ') group.add_argument( '--some_group', nargs='*', type=str )

ORM opinion

2014-12-04 Thread Joseph L. Casale
Begrudgingly, I need to migrate away from SQLAlchemy onto a package that has fast imports and very fast model build times. I have a less than ideal application that uses Python as a plugin interpreter which is not performant in this use case where its being invoked freshly several times per

RE: ORM opinion

2014-12-04 Thread Joseph L. Casale
First recommendation: Less layers. Instead of SQLAlchemy, just import sqlite3 and use it directly. You should be able to switch out import sqlite as db for import psycopg2 as db or any other Python DB API module, and still have most/all of the benefit of the extra layer, without any extra

RE: ORM opinion

2014-12-04 Thread Joseph L. Casale
Anything listed here http://www.pythoncentral.io/sqlalchemy-vs-orms/ you've not heard about? I found peewee easy to use although I've clearly no idea if it suits your needs. There's only one way to find out :) Hi Mark, I found that article before posting and some of the guys here have

RE: Using map()

2014-11-16 Thread Joseph L. Casale
I checked my modules with pylint and saw the following warning: W: 25,29: Used builtin function 'map' (bad-builtin) Why is the use of map() discouraged? It' such a useful thing. The warning manifests from the opinion that a comprehension is more suitable. You can disable the warning or you

Processing xml for output with cElementTree

2014-10-17 Thread Joseph L. Casale
I am unfortunately unable to use lxml for a project and must resort to base only libraries to create several nested elements located directly under a root element. The caveat is the incremental writing and flushing of the nested elements as they are created. So assuming the structure is

RE: GIL detector

2014-08-17 Thread Joseph L. Casale
I don't have to care about threading issues all the time and can otherwise freely choose the right model of parallelism that suits my current use case when the need arises (and threads are rarely the right model). I'm sure that's not just me. The sound bite of a loyal Python coder:) If it

RE: Using pyVmomi

2014-07-24 Thread Joseph L. Casale
You could: - have a single point of entry that can check and, if necessary, revalidate - create a helper that checks and, if necessary, revalidate, which is then called where ever needed - create a decorator that does the above for each function that needs it Hi Ethan,

Using pyVmomi

2014-07-23 Thread Joseph L. Casale
I am doing some scripting with pyVmomi under 2.6.8 so the code may run directly on a vmware esxi server. As the code is long running, it surpasses the authentication timeout. For anyone familiar with this code and/or this style of programming, does anyone have a recommendation for an elegant

RE: compiled cx_freeze

2014-05-25 Thread Joseph L. Casale
Anyone knows where to get a compiled cx_freeze that has already has this patch? http://www.lfd.uci.edu/~gohlke/pythonlibs/#cx_freeze -- https://mail.python.org/mailman/listinfo/python-list

RE: compiled cx_freeze

2014-05-25 Thread Joseph L. Casale
Unfortunately, this is buggy too. Here is a test output from a compiled console exe created with the above version of cx freeze: Let Christoph know, he is very responsive and extremely helpful. -- https://mail.python.org/mailman/listinfo/python-list

RE: Exception problem with module

2014-05-19 Thread Joseph L. Casale
Well I am not sure what advantage this has for the user, not my code as I don't advocate the import to begin with it, its fine spelled as it was from where it was... The advantage for the user is: /snip Hey Steven, Sorry for the late reply (travelling). My comment wasn't clear, I was

  1   2   >