Re: How much sanity checking is required for function inputs?
On Thu, Apr 21, 2016, at 08:33 PM, Christopher Reimer wrote:
> On 4/21/2016 7:20 PM, Stephen Hansen wrote:
> > I... that... what... I'd forget that link and pretend you never went
> > there. Its not helpful.
>
> I found it on the Internet, so it must be true -- and Pythonic at that!
My advice is to not look at that site further. I can't count the number
of things that are just... not useful or helpful.
Directly translating the Gang of Four Design Pattern book to Python
doesn't generally result in useful ideas, except in certain abstractions
like the visitor pattern when you're designing big systems.
>
> > What's the contents of this big dictionary that has everything in it
> > for some reason?
>
> Keep in mind that I'm coming from a Java background (not by choice) with
> a smattering of C programming. I initially had ALL THESE CONSTANTS in
> different parts of my code and couldn't easily use them across different
> modules. I didn't like all these constants, created a singleton class
> that uses ConfigParser, dumped everything into a .ini file, and accessed
> from in each module. As I learn to write Pythonic code, the constants
> may go away. Switching from my original code to the factory method
> eliminated a half-dozen constants.
I'm not criticizing, I'm *asking* in the hopes you'll explain what
you're doing better, so I can understand and give advice that's more
relevant: I don't understand what it is you're doing with the VARS or
const dictionary. Even with your explanation here, I don't get it.
Why do you need to read VARS["COLOR_BLACK"] or the new
const["COLOR_BLACK"} from a configuration file? How is this stuff
configurable?
Why not, 'color in ("black", "white")'?
The only reason I can think of why you'd want to read what is "black" or
"white" out of a configuration file is internationalization, and if
that's your aim, there's a couple really good projects I can recommend
that solve that issue better.
I can't think of any reason why you'd want a structure like:
struct = {}
struct["COLOR_BLACK"] = "black"
struct["COLOR_WHITE"] = "white"
"black" is a perfectly good way to express "black", you have this
indirection/abstraction for a reason I don't understand. Granted, maybe
this is a heavy javaism, and I'm utterly ignorant of Java, but my C/C++
mind can't think of a reason for this either.
That said, if you're wanting to share constants across different parts
of your code, use a module.
Create a module, say, things.py, that itself doesn't import other
things, and it says:
black = "Black"
white = "White"
(Bear in mind I still don't know why you want or need this indirection)
Then you "import things" and refer to those constants as things.black,
things.white.
You can then "if color in (things.black, things.white)". Maybe even "if
color in things.colors" if in your things file you have things = (black,
white).
Modules are great for sharing information. The admonition of GLOBALS R
BAD is not as strong in Python as you find in some other languages.
(Don't take that to mean GLOBALS R GUD, granted. Passing state is best
when it makes sense).
--S
m e @ i x o k a i . i o
--
https://mail.python.org/mailman/listinfo/python-list
Re: Error 0*80070570
On Thu, Apr 21, 2016 at 11:15 PM, Allan Leo wrote: > On Apr 21, 2016 9:51 PM, "eryk sun" wrote: >> On Thu, Apr 21, 2016 at 4:06 AM, Allan Leo wrote: >> > When running the setup for your 3.5.1(32-bit version), the setup >> > experiences error 0*80070570 and tells me to check the log file. What >> > could be the problem and whats the solution. >> >> ERROR_FILE_CORRUPT (0x0570) seems to occur frequently with the new >> installer. Are you using the web installer or the offline installer, >> i.e. what's the name of the file you downloaded? What version of >> Windows do you have, i.e. what does running "ver" print in the command >> prompt? Or, in more detail, what does the following print? >> >> wmic os get buildnumber,caption,csdversion,osarchitecture /value > > I'm using the offline installer, windows 7 pro, Java-8u92-windows-i586.exe. The 32-bit offline installer is named "python-3.5.1.exe". Delete the copy you downloaded, clear your browser cache, and try again. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 04/21/2016 08:33 PM, Christopher Reimer wrote: On 4/21/2016 7:20 PM, Stephen Hansen wrote: Keep in mind that I'm coming from a Java background (not by choice) with a smattering of C programming. A refugee! Water! Food! import this!! :) Oh! and Enum!!! ;) -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/21/2016 7:20 PM, Stephen Hansen wrote: Whyyy are you using getattr? Something wrong with PieceFactory.factory(color, piece, position)? (Or, better yet, yield piece_factory(color, piece, position) where piece_factory is just a function) Because the example I found used it, I implemented it, and it worked in my code. Based on feedback that I gotten from this list, I'll go back and clean it up. I... that... what... I'd forget that link and pretend you never went there. Its not helpful. I found it on the Internet, so it must be true -- and Pythonic at that! What's the contents of this big dictionary that has everything in it for some reason? Keep in mind that I'm coming from a Java background (not by choice) with a smattering of C programming. I initially had ALL THESE CONSTANTS in different parts of my code and couldn't easily use them across different modules. I didn't like all these constants, created a singleton class that uses ConfigParser, dumped everything into a .ini file, and accessed from in each module. As I learn to write Pythonic code, the constants may go away. Switching from my original code to the factory method eliminated a half-dozen constants. Thank you, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/21/2016 7:10 PM, Ethan Furman wrote: I do plan to incorporate a sanity test in each Piece class to validate the initial position value. Pawns have 16 specific positions. Bishop, Knight and Rook each have four specific positions. King and Queen each have two specific positions. An invalid value will raise an exception. This will make it so you cannot use your PieceFactory for custom setups. The sanity check won't be in the PieceFactory, but in the Piece class as an interface and each Piece subclass will implement the correct positions for comparison. Thank you, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/21/2016 6:54 PM, Tim Chase wrote:
I'd simplify this code to something like
class PieceFactory(object):
@staticmethod
def factory(color, piece, position):
try:
return {
'Bishop': Bishop,
'King': King,
'Knight': Knight,
'Pawn': Pawn,
'Queen': Queen,
'Rook': Rook,
}[piece](color, position)
except KeyError:
raise PieceException(...)
I like this better. I'll probably remove the try/except and let the
KeyError filter up.
Thanks,
Chris R.
--
https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On Fri, 22 Apr 2016 11:34 am, Christopher Reimer wrote:
> Greetings,
>
> Thanks to everyone for providing feedback. Here's my revised code to
> generate a set of chess pieces.
> class PieceFactory(object):
>
> def factory(color, piece, position):
> if piece == 'Bishop':
> return Bishop(color, position)
> if piece == 'King':
> return King(color, position)
> if piece == 'Knight':
> return Knight(color, position)
> if piece == 'Pawn':
> return Pawn(color, position)
> if piece == 'Queen':
> return Queen(color, position)
> if piece == 'Rook':
> return Rook(color, position)
>
> raise PieceException('No valid Piece object for factory,
> got {}'
> ' instead'.format(piece))
>
> factory = staticmethod(factory)
Eww :-)
Creating an entire class with no state just to hold one method is an abuse
of classes. If your class doesn't include both state (data) and behaviour
(methods), it probably shouldn't be a class.
class King: ...
class Queeen: ...
# etc.
PIECES = dict((piece.__name__, piece) for piece in
[King, Queen, Bishop, Knight, Rook, Pawn])
def make_piece(color, name, position):
name = name.title() # Accept 'king', 'KING', 'King' etc.
P = PIECES.get(name, None)
if P is None:
raise PieceException('unknown name %r' % name)
return P(color, position)
> def generate_set(color, pieces, positions):
> for piece, position in zip(pieces, positions):
> yield getattr(PieceFactory, 'factory')(color, piece, position)
def generate_pieces(color, names, positions):
for name, position in zip(names, positions):
yield make_piece(color, name, position)
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 2016-04-21 18:34, Christopher Reimer wrote:
> class PieceFactory(object):
>
> def factory(color, piece, position):
> if piece == 'Bishop':
> return Bishop(color, position)
> if piece == 'King':
> return King(color, position)
> if piece == 'Knight':
> return Knight(color, position)
> if piece == 'Pawn':
> return Pawn(color, position)
> if piece == 'Queen':
> return Queen(color, position)
> if piece == 'Rook':
> return Rook(color, position)
>
> raise PieceException('No valid Piece object for
> factory, got {}'
> ' instead'.format(piece))
>
> factory = staticmethod(factory)
I'd simplify this code to something like
class PieceFactory(object):
@staticmethod
def factory(color, piece, position):
try:
return {
'Bishop': Bishop,
'King': King,
'Knight': Knight,
'Pawn': Pawn,
'Queen': Queen,
'Rook': Rook,
}[piece](color, position)
except KeyError:
raise PieceException(...)
which removes some of the redundancy. I might even be tempted to
simply ignore the exception and let the KeyError percolate up the
call-stack if someone calls it with an unknown piece/key, rather
than converting it into a PieceException.
-tkc
--
https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On Thu, Apr 21, 2016, at 06:34 PM, Christopher Reimer wrote:
> class PieceFactory(object):
>
> def factory(color, piece, position):
> if piece == 'Bishop':
> return Bishop(color, position)
> if piece == 'King':
> return King(color, position)
> if piece == 'Knight':
> return Knight(color, position)
> if piece == 'Pawn':
> return Pawn(color, position)
> if piece == 'Queen':
> return Queen(color, position)
> if piece == 'Rook':
> return Rook(color, position)
This whole section is begging for a dictionary. Like...
_PIECE_TYPES= {"Bishop": Bishop, "King": King, ...]
class PieceFactory(object):
def factory(color, piece, position):
klass = __PIECE_TYPES.get(piece)
if klass is None:
raise PieceException("...")
return klass(color, position)
Or something like that.
That said, I'm not sure why its not just a function that does the same
thing. Why is it in a class that only does one thing? You never even
instantiate it.
> def generate_set(color, pieces, positions):
> for piece, position in zip(pieces, positions):
> yield getattr(PieceFactory, 'factory')(color, piece, position)
Whyyy are you using getattr? Something wrong with
PieceFactory.factory(color, piece, position)? (Or, better yet, yield
piece_factory(color, piece, position) where piece_factory is just a
function)
> I got the factory method from here:
> http://python-3-patterns-idioms-test.readthedocs.org/en/latest/Factory.html
I... that... what... I'd forget that link and pretend you never went
there. Its not helpful.
> Finally, VARS['VARIABLE_NAME'] got change to const['variable_name'].
> Should smell better.
I don't know that this changes anything about the small at all. What's
the contents of this big dictionary that has everything in it for some
reason?
That said, dear god, 'piece' doesn't look like an english word to me
anymore. I've never suffered semantic satiation from text before.
--Stephen
m e @ i x o k a i . i o
--
https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 04/21/2016 06:34 PM, Christopher Reimer wrote: class PieceFactory(object): > [...] Better. I do plan to incorporate a sanity test in each Piece class to validate the initial position value. Pawns have 16 specific positions. Bishop, Knight and Rook each have four specific positions. King and Queen each have two specific positions. An invalid value will raise an exception. This will make it so you cannot use your PieceFactory for custom setups. -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: A pickle problem!
Às 22:43 de 21-04-2016, Paulo da Silva escreveu:
> Hi.
>
> Why in this code fragment self.__name is not kept between pickle
> dumps/loads? How to fix it?
>
> Thanks.
>
> import pickle
> import pandas as pd
> import numpy as np
>
> class C(pd.DataFrame):
> def __init__(self,name,*a,**b):
> super(C,self).__init__(*a,**b)
> self.__name=name
>
> def GetName(self):
> return self.__name
>
# Adding this works but looks tricky!
def __getstate__(self):
dfstate=super(C,self).__getstate__()
cstate=(dfstate,self.__name)
return cstate
def __setstate__(self,cstate):
super(C,self).__setstate__(cstate[0])
self.__name=cstate[1]
>
> dates = pd.date_range('20130101', periods=6)
> c = C("FOO",np.random.randn(6,4), index=dates, columns=list('ABCD'))
>
> cd=pickle.dumps(c,pickle.HIGHEST_PROTOCOL)
>
> d=pickle.loads(cd)
>
> d.GetName()
>
> # AttributeError: 'C' object has no attribute '_C__name'
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
Greetings,
Thanks to everyone for providing feedback. Here's my revised code to
generate a set of chess pieces.
class PieceFactory(object):
def factory(color, piece, position):
if piece == 'Bishop':
return Bishop(color, position)
if piece == 'King':
return King(color, position)
if piece == 'Knight':
return Knight(color, position)
if piece == 'Pawn':
return Pawn(color, position)
if piece == 'Queen':
return Queen(color, position)
if piece == 'Rook':
return Rook(color, position)
raise PieceException('No valid Piece object for factory,
got {}'
' instead'.format(piece))
factory = staticmethod(factory)
def generate_set(color, pieces, positions):
for piece, position in zip(pieces, positions):
yield getattr(PieceFactory, 'factory')(color, piece, position)
The input values for 'pieces' and 'positions' are 16-item lists zipped
together to produce a piece name and a position coordinate for the
factory method. With slight modifications to the code, the factory
method could also return checker pieces.
I got the factory method from here:
http://python-3-patterns-idioms-test.readthedocs.org/en/latest/Factory.html
I do plan to incorporate a sanity test in each Piece class to validate
the initial position value. Pawns have 16 specific positions. Bishop,
Knight and Rook each have four specific positions. King and Queen each
have two specific positions. An invalid value will raise an exception.
Finally, VARS['VARIABLE_NAME'] got change to const['variable_name'].
Should smell better.
Thanks,
Chris R
--
https://mail.python.org/mailman/listinfo/python-list
A pickle problem!
Hi.
Why in this code fragment self.__name is not kept between pickle
dumps/loads? How to fix it?
Thanks.
import pickle
import pandas as pd
import numpy as np
class C(pd.DataFrame):
def __init__(self,name,*a,**b):
super(C,self).__init__(*a,**b)
self.__name=name
def GetName(self):
return self.__name
dates = pd.date_range('20130101', periods=6)
c = C("FOO",np.random.randn(6,4), index=dates, columns=list('ABCD'))
cd=pickle.dumps(c,pickle.HIGHEST_PROTOCOL)
d=pickle.loads(cd)
d.GetName()
# AttributeError: 'C' object has no attribute '_C__name'
--
https://mail.python.org/mailman/listinfo/python-list
Re: Error 0*80070570
On Thu, Apr 21, 2016 at 4:06 AM, Allan Leo wrote: > When running the setup for your 3.5.1(32-bit version), the setup > experiences error 0*80070570 and tells me to check the log file. What could > be the problem and whats the solution. ERROR_FILE_CORRUPT (0x0570) seems to occur frequently with the new installer. Are you using the web installer or the offline installer, i.e. what's the name of the file you downloaded? What version of Windows do you have, i.e. what does running "ver" print in the command prompt? Or, in more detail, what does the following print? wmic os get buildnumber,caption,csdversion,osarchitecture /value -- https://mail.python.org/mailman/listinfo/python-list
Re: Re: Error 0*80070570
On Thursday, April 21, 2016 at 10:47:04 AM UTC-7, Allan Leo wrote: > I need help with this setup error. > -- Forwarded message -- > From: "Allan Leo" > Date: Apr 21, 2016 10:06 AM > Subject: Re: Error 0*80070570 > To: > Cc: > > When running the setup for your 3.5.1(32-bit version), the setup > experiences error 0*80070570 and tells me to check the log file. What could > be the problem and whats the solution. > On Apr 21, 2016 7:05 AM, "Allan Leo" wrote: > > > When running the setup for your 3.5.1(32-bit version) the setup > > experiences error 0*80070570 and tells me to checkout the log file. What > > could be the problem and whats the resolution. > > Justin already gave you something to try: read the log file and search Google for the error code. How about you try those before asking a FOURTH time? I'll even give you a handy link so you don't have to search yourself: http://lmgtfy.com/?q=0x80070570+error+when+installing+python -- https://mail.python.org/mailman/listinfo/python-list
Re: Different names for Unicode codepoint
> From: Lele Gaifax
> Date: Thu, 21 Apr 2016 21:04:32 +0200
> Cc: python-list@python.org
>
> is there a particular reason for the slightly different names that Emacs
> (version 25.0.92) and Python (version 3.6.0a0) give to a single Unicode
> entity?
They don't.
> Just to mention one codepoint, ⋖ is called "LESS THAN WITH DOT" accordingly to
> Emacs' C-x 8 RET TAB menu, while in Python:
>
> >>> import unicodedata
> >>> unicodedata.name('⋖')
> 'LESS-THAN WITH DOT'
> >>> print("\N{LESS THAN WITH DOT}")
> File "", line 1
> SyntaxError: (unicode error) ...: unknown Unicode character name
Emacs shows both the "Name" and the "Old Name" properties of
characters as completion candidates, while Python evidently supports
only "Name". If you type "C-x 8 RET LESS TAB", then you will see
among the completion candidates both "LESS THAN WITH DOT" and
"LESS-THAN WITH DOT". The former is the "old name" of this character,
according to the Unicode Character Database (which is where Emacs
obtains the names and other properties of characters).
--
https://mail.python.org/mailman/listinfo/python-list
python regex dna processing
>From time to time there are DNA related question posted here. I came upon this in the hopes it may be useful to those who do that kind of software http://benchling.engineering/dna-regex-search/ -- Joel Goldstick http://joelgoldstick.com/blog http://cc-baseballstats.info/stats/birthdays -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Re: Error 0*80070570
On Thu, Apr 21, 2016, at 10:46 AM, Allan Leo wrote: > I need help with this setup error. > -- Forwarded message -- > From: "Allan Leo" > Date: Apr 21, 2016 10:06 AM > Subject: Re: Error 0*80070570 > To: > Cc: > > When running the setup for your 3.5.1(32-bit version), the setup > experiences error 0*80070570 and tells me to check the log file. What > could > be the problem and whats the solution. > On Apr 21, 2016 7:05 AM, "Allan Leo" wrote: > > > When running the setup for your 3.5.1(32-bit version) the setup > > experiences error 0*80070570 and tells me to checkout the log file. What > > could be the problem and whats the resolution. What version of windows? -- https://mail.python.org/mailman/listinfo/python-list
Re: How are you supposed to define subclasses in C?
Random832 schrieb am 21.04.2016 um 18:35: > I was trying to write a proof of concept on including descriptors (e.g. > a "sys.recursionlimit" instead of set/get methods) in the sys module, > and couldn't figure out how to "properly" define a type using > PyType_FromSpecWithBases. Everything I tried just segfaulted. I ended up > just calling PyObject_CallFunctionObjArgs((PyObject *)&PyType_Type, ...) > but I assume there's a better way to do it. I couldn't find any examples > or tutorial. I suppose you might find Cython useful: http://cython.org/ http://docs.cython.org/ In a nutshell, it lets you write beautiful Python code instead of ugly C code full of leaks and crashes, and then takes care of making it faster than the usual C-API calls you'd write by hand. It has some extended syntax for extension types and other C-ish things to make their usage explicit. Stefan -- https://mail.python.org/mailman/listinfo/python-list
Fwd: Re: Error 0*80070570
I need help with this setup error. -- Forwarded message -- From: "Allan Leo" Date: Apr 21, 2016 10:06 AM Subject: Re: Error 0*80070570 To: Cc: When running the setup for your 3.5.1(32-bit version), the setup experiences error 0*80070570 and tells me to check the log file. What could be the problem and whats the solution. On Apr 21, 2016 7:05 AM, "Allan Leo" wrote: > When running the setup for your 3.5.1(32-bit version) the setup > experiences error 0*80070570 and tells me to checkout the log file. What > could be the problem and whats the resolution. > -- https://mail.python.org/mailman/listinfo/python-list
Re: How are you supposed to define subclasses in C?
On Thu, Apr 21, 2016 at 11:35 AM, Random832 wrote: > I was trying to write a proof of concept on including descriptors (e.g. > a "sys.recursionlimit" instead of set/get methods) in the sys module, > and couldn't figure out how to "properly" define a type using > PyType_FromSpecWithBases. Everything I tried just segfaulted. I ended up > just calling PyObject_CallFunctionObjArgs((PyObject *)&PyType_Type, ...) > but I assume there's a better way to do it. I couldn't find any examples > or tutorial. Have a look at https://hg.python.org/cpython/file/default/Modules/xxsubtype.c -- Zach -- https://mail.python.org/mailman/listinfo/python-list
How are you supposed to define subclasses in C?
I was trying to write a proof of concept on including descriptors (e.g. a "sys.recursionlimit" instead of set/get methods) in the sys module, and couldn't figure out how to "properly" define a type using PyType_FromSpecWithBases. Everything I tried just segfaulted. I ended up just calling PyObject_CallFunctionObjArgs((PyObject *)&PyType_Type, ...) but I assume there's a better way to do it. I couldn't find any examples or tutorial. -- https://mail.python.org/mailman/listinfo/python-list
Re: PEP proposal: sequence expansion support for yield statement: yield *
On Fri, Apr 22, 2016 at 1:19 AM, justin walters wrote: > I agree with the others that the new syntax is not needed. > > I would also like to point out that I believe any new added syntax or > functionality should avoid the use of '*' and '**' as both of these > characters are already used for many things such as optional arguments and > mathematical operators. Adding more uses for said characters only decreases > the readability of the code. The single asterisk is used for packing and unpacking, though. See PEP 448 [1] for some of the ways this can be used (in the context of "the ways this can NOW be used in Python 3.5"). So the proposed "yield *sequence" does make good sense; however, "yield from" has, as Steven pointed out, *far* more detailed semantics, as it basically allows generators to be refactored, with all their semantics (yield, send, throw, return). ChrisA [1] https://www.python.org/dev/peps/pep-0448/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Error 0*80070570
On Thu, Apr 21, 2016 at 2:06 AM, Allan Leo wrote: > When running the setup for your 3.5.1(32-bit version), the setup > experiences error 0*80070570 and tells me to check the log file. What could > be the problem and whats the solution. > On Apr 21, 2016 7:05 AM, "Allan Leo" wrote: > > > When running the setup for your 3.5.1(32-bit version) the setup > > experiences error 0*80070570 and tells me to checkout the log file. What > > could be the problem and whats the resolution. > > > -- > https://mail.python.org/mailman/listinfo/python-list > Have you tried checking the log file or googling the error code? -- https://mail.python.org/mailman/listinfo/python-list
Re: PEP proposal: sequence expansion support for yield statement: yield *
I agree with the others that the new syntax is not needed. I would also like to point out that I believe any new added syntax or functionality should avoid the use of '*' and '**' as both of these characters are already used for many things such as optional arguments and mathematical operators. Adding more uses for said characters only decreases the readability of the code. -- https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
On Fri, Apr 22, 2016 at 12:30 AM, Oscar Benjamin wrote: > On 21 April 2016 at 15:12, Chris Angelico wrote: >> On Fri, Apr 22, 2016 at 12:01 AM, Oscar Benjamin >> wrote: >>> In the recursive stack overflow case what you'll usually have is >>> >>> 1) A few frames leading up to the start of recursion >>> 2) A long repetitive sequence of frames >>> 3) A few frames at the end showing how the exception was ultimately >>> triggered. >>> >>> You just need to find the cycle that makes that big long sequence. >> >> If the stack got overflowed, there won't usually be a part 3, as part >> 2 is the bit that hits sys.recursionlimit (unless increasing the >> recursion limit by a finite number would solve the problem). For other >> exceptions, yes, this is what you'd see. > > If you have: > > def f(x): > return g(x+1) > > def g(x): > x = h(x) # <-- stack can overflow inside here > return f(x+1) > > # etc. > > So you have a long sequence that goes f, g, f, g but at the end the > stack can overflow while (not recursively) calling h leaving a small > non-cyclic part at the end. Right, good point. Forgot about that. So this situation can be triggered by anywhere up to lines up. Not as helpful an optimization now. Oh well. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
On 21 April 2016 at 15:12, Chris Angelico wrote: > On Fri, Apr 22, 2016 at 12:01 AM, Oscar Benjamin > wrote: >> In the recursive stack overflow case what you'll usually have is >> >> 1) A few frames leading up to the start of recursion >> 2) A long repetitive sequence of frames >> 3) A few frames at the end showing how the exception was ultimately >> triggered. >> >> You just need to find the cycle that makes that big long sequence. > > If the stack got overflowed, there won't usually be a part 3, as part > 2 is the bit that hits sys.recursionlimit (unless increasing the > recursion limit by a finite number would solve the problem). For other > exceptions, yes, this is what you'd see. If you have: def f(x): return g(x+1) def g(x): x = h(x) # <-- stack can overflow inside here return f(x+1) # etc. So you have a long sequence that goes f, g, f, g but at the end the stack can overflow while (not recursively) calling h leaving a small non-cyclic part at the end. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
On Fri, Apr 22, 2016 at 12:01 AM, Oscar Benjamin wrote: > In the recursive stack overflow case what you'll usually have is > > 1) A few frames leading up to the start of recursion > 2) A long repetitive sequence of frames > 3) A few frames at the end showing how the exception was ultimately triggered. > > You just need to find the cycle that makes that big long sequence. If the stack got overflowed, there won't usually be a part 3, as part 2 is the bit that hits sys.recursionlimit (unless increasing the recursion limit by a finite number would solve the problem). For other exceptions, yes, this is what you'd see. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
On 21 April 2016 at 13:15, Steven D'Aprano wrote:
> On Thu, 21 Apr 2016 06:53 pm, Oscar Benjamin wrote:
>
>> On 21 April 2016 at 04:07, Steven D'Aprano wrote:
>>> I want to group repeated items in a sequence. For example, I can group
>>> repeated sequences of a single item at a time using groupby:
>>>
>>>
>>> from itertools import groupby
>>> for key, group in groupby("BBCDDEEE"):
>>> group = list(group)
>>> print(key, "count =", len(group))
>>>
>>>
>>> outputs:
>>>
>>> A count = 4
>>> B count = 2
>>> C count = 1
>>> D count = 2
>>> E count = 3
>>> F count = 4
>>>
>>>
>>> Now I want to group subsequences. For example, I have:
>>>
>>> "ABCABCABCDEABCDEFABCABCABCB"
>>>
>>> and I want to group it into repeating subsequences. I can see two ways to
>>> group it:
>>>
>>> ABC ABC ABCDE ABCDE F ABC ABC ABC B
>>
>> There are some algorithms (helpfully shown in Python) here:
>>
>> https://en.wikipedia.org/wiki/Cycle_detection
>
> It's not necessarily a cycle though. Consider a sequence of function calls:
>
> def f(x):
> return g(x)
>
> def g(x):
> if x < 7:
> return h(x)
> elif x < 50:
> return g(x//2)
> else:
> return x+f(x-1)
>
> def h(x):
> raise ValueError # oops, a bug
>
>
> and a function call f(54). That will result in the chain of calls:
>
> f(54) -> g(54) -> f(53) -> g(53) -> f(52) -> g(52) -> f(51) -> g(51) ->
> f(50) -> g(50) -> g(25) -> g(12) -> g(6) -> h(6) raises
>
> So you have that almost-cycle f calls g calls f, but it isn't periodic
> because you break out of it. I'd still like to detect the repeated f->g
> calls.
It doesn't matter that you break out of it. It's periodic for some
part and the algorithms listed on that page can find the cycle. In the
recursive stack overflow case what you'll usually have is
1) A few frames leading up to the start of recursion
2) A long repetitive sequence of frames
3) A few frames at the end showing how the exception was ultimately triggered.
You just need to find the cycle that makes that big long sequence.
--
Oscar
--
https://mail.python.org/mailman/listinfo/python-list
Re: delete from pattern to pattern if it contains match
harirammano...@gmail.com writes:
> On Monday, April 18, 2016 at 12:38:03 PM UTC+5:30,
> hariram...@gmail.com wrote:
>> HI All,
>>
>> can you help me out in doing below.
>>
>> file:
>>
>> guava
>> fruit
>>
>>
>> mango
>> fruit
>>
>>
>> orange
>> fruit
>>
>>
>> need to delete from start to end if it contains mango in a file...
>>
>> output should be:
>>
>>
>> guava
>> fruit
>>
>>
>> orange
>> fruit
>>
>>
>> Thank you
>
> any one can guide me ? why xml tree parsing is not working if i have
> root.tag and root.attrib as mentioned in earlier post...
Assuming the real consists of lines between a start marker and end
marker, a winning plan is to collect a group of lines, deal with it, and
move on.
The following code implements something close to the plan. You need to
adapt it a bit to have your own source of lines and to restore the end
marker in the output and to account for your real use case and for
differences in taste and judgment. - The plan is as described above, but
there are many ways to implement it.
from io import StringIO
text = '''\
guava
fruit
mango
fruit
orange
fruit
'''
def records(source):
current = []
for line in source:
if line.startswith(''):
yield current
current = []
else:
current.append(line)
def hasmango(record):
return any('mango' in it for it in record)
for record in records(StringIO(text)):
hasmango(record) or print(*record)
--
https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
On Thu, 21 Apr 2016 06:53 pm, Oscar Benjamin wrote:
> On 21 April 2016 at 04:07, Steven D'Aprano wrote:
>> I want to group repeated items in a sequence. For example, I can group
>> repeated sequences of a single item at a time using groupby:
>>
>>
>> from itertools import groupby
>> for key, group in groupby("BBCDDEEE"):
>> group = list(group)
>> print(key, "count =", len(group))
>>
>>
>> outputs:
>>
>> A count = 4
>> B count = 2
>> C count = 1
>> D count = 2
>> E count = 3
>> F count = 4
>>
>>
>> Now I want to group subsequences. For example, I have:
>>
>> "ABCABCABCDEABCDEFABCABCABCB"
>>
>> and I want to group it into repeating subsequences. I can see two ways to
>> group it:
>>
>> ABC ABC ABCDE ABCDE F ABC ABC ABC B
>
> There are some algorithms (helpfully shown in Python) here:
>
> https://en.wikipedia.org/wiki/Cycle_detection
It's not necessarily a cycle though. Consider a sequence of function calls:
def f(x):
return g(x)
def g(x):
if x < 7:
return h(x)
elif x < 50:
return g(x//2)
else:
return x+f(x-1)
def h(x):
raise ValueError # oops, a bug
and a function call f(54). That will result in the chain of calls:
f(54) -> g(54) -> f(53) -> g(53) -> f(52) -> g(52) -> f(51) -> g(51) ->
f(50) -> g(50) -> g(25) -> g(12) -> g(6) -> h(6) raises
So you have that almost-cycle f calls g calls f, but it isn't periodic
because you break out of it. I'd still like to detect the repeated f->g
calls.
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
On Thu, 21 Apr 2016 18:05:40 +1000, Steven D'Aprano wrote: > The specific problem I am trying to solve is that I have a sequence of > strings (in this case, error messages from a Python traceback) and I'm > looking for repeated groups that may indicate mutually recursive calls. E.g. > suppose I have a function f which calls g, and g calls h, and h calls f > again, and there's an exception, you will see a traceback in part: This is a specific case of finding cycles in a directed graph. But treating it as such probably isn't useful here, because you're interested in a specific traversal of that graph rather than the graph itself. One way to approach it is: sofar = [] for line in traceback: if line in sofar: j = sofar.index(line) if sofar[:j] == sofar[j:j*2]: # found repeat sofar = [line] + sofar Note that sofar needs to be in reverse order, because list doesn't have .rindex() or .rfind(). Detecting nested cycles is somewhat harder because given e.g. ababxabababababxababab you'd want the five repeats of ab in the middle to be treated as two repeats of ab followed by three repeats of ab, but there's no way to spot that until later. -- https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
On 21.04.16 06:07, Steven D'Aprano wrote: Now I want to group subsequences. For example, I have: "ABCABCABCDEABCDEFABCABCABCB" and I want to group it into repeating subsequences. [...] How can I do this? Does this problem have a standard name and/or solution? This is a part of lossless data compression algorithms. See for example LZ, LZW. -- https://mail.python.org/mailman/listinfo/python-list
Re: delete from pattern to pattern if it contains match
harirammano...@gmail.com wrote: > On Monday, April 18, 2016 at 12:38:03 PM UTC+5:30, hariram...@gmail.com > wrote: >> HI All, >> >> can you help me out in doing below. >> >> file: >> >> guava >> fruit >> >> >> mango >> fruit >> >> >> orange >> fruit >> Is that literally what you have in the file? > any one can guide me ? why xml tree parsing is not working if i have > root.tag and root.attrib as mentioned in earlier post... The data above is not valid xml. Instead of ... you need ... i. e. the end tag must be the same as the start tag, but with a leading "/". -- https://mail.python.org/mailman/listinfo/python-list
Re: delete from pattern to pattern if it contains match
On Monday, April 18, 2016 at 12:38:03 PM UTC+5:30, hariram...@gmail.com wrote: > HI All, > > can you help me out in doing below. > > file: > > guava > fruit > > > mango > fruit > > > orange > fruit > > > need to delete from start to end if it contains mango in a file... > > output should be: > > > guava > fruit > > > orange > fruit > > > Thank you any one can guide me ? why xml tree parsing is not working if i have root.tag and root.attrib as mentioned in earlier post... -- https://mail.python.org/mailman/listinfo/python-list
Re: Error 0*80070570
When running the setup for your 3.5.1(32-bit version), the setup experiences error 0*80070570 and tells me to check the log file. What could be the problem and whats the solution. On Apr 21, 2016 7:05 AM, "Allan Leo" wrote: > When running the setup for your 3.5.1(32-bit version) the setup > experiences error 0*80070570 and tells me to checkout the log file. What > could be the problem and whats the resolution. > -- https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
On 21 April 2016 at 04:07, Steven D'Aprano wrote:
> I want to group repeated items in a sequence. For example, I can group
> repeated sequences of a single item at a time using groupby:
>
>
> from itertools import groupby
> for key, group in groupby("BBCDDEEE"):
> group = list(group)
> print(key, "count =", len(group))
>
>
> outputs:
>
> A count = 4
> B count = 2
> C count = 1
> D count = 2
> E count = 3
> F count = 4
>
>
> Now I want to group subsequences. For example, I have:
>
> "ABCABCABCDEABCDEFABCABCABCB"
>
> and I want to group it into repeating subsequences. I can see two ways to
> group it:
>
> ABC ABC ABCDE ABCDE F ABC ABC ABC B
There are some algorithms (helpfully shown in Python) here:
https://en.wikipedia.org/wiki/Cycle_detection
Note that those are for a sequence made as x[n+1] = f(x[n]) for some
function f. In your case that's just the function that gets the next
frame up/down in the call stack.
--
Oscar
--
Oscar
--
https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
On Thursday 21 April 2016 16:35, Michael Selik wrote: > On Wed, Apr 20, 2016 at 11:11 PM Steven D'Aprano > wrote: > >> I want to group [repeated] subsequences. For example, I have: >> "ABCABCABCDEABCDEFABCABCABCB" >> and I want to group it into repeating subsequences. I can see two >> ways... How can I do this? Does this problem have a standard name and/or >> solution? >> > > I'm not aware of a standard name. This sounds like an unsupervised > learning problem. There's no objectively correct answer unless you add > more specificity to the problem statement. > > Regexes may sound tempting at first, Ah, I may have mislead you all. I cannot use regexes, since the *actual* sequences I'm working on are sequences (lists, tuples, etc) or iterators of arbitrary items. The items themselves may be strings, but the sequence itself is definitely not a string. I just showed a string for convenience. Sorry about that. So no regexes. > but because a repeating subsequence > may have nested repeating subsequences and this can go on infinitely, I > think we at least need a push-down automata. Fortunately, for my *immediate* problem, I would be good with some fairly arbitrary restrictions on subsequence detection. > I checked out some links for clustering algorithms that work on series > subsequences and I found some fun results. > > Clustering is meaningless! > http://www.cs.ucr.edu/~eamonn/meaningless.pdf > > I think you're in "no free lunch" territory. "Clustering of subsequence > time series remains an open issue in time series clustering" > http://www.hindawi.com/journals/tswj/2014/312521/ > > Any more detail on the problem to add constraints? The specific problem I am trying to solve is that I have a sequence of strings (in this case, error messages from a Python traceback) and I'm looking for repeated groups that may indicate mutually recursive calls. E.g. suppose I have a function f which calls g, and g calls h, and h calls f again, and there's an exception, you will see a traceback in part: File "", line 2, in f File "", line 5, in g File "", line 9, in h File "", line 2, in f File "", line 5, in g File "", line 9, in h File "", line 2, in f File "", line 5, in g File "", line 9, in h File "", line 7, in f File "", line 5, in g File "", line 9, in h etc. Note that I only care about lines which are identical, e.g. if the line numbers differ (as in the last call to f), they will be treated as distinct items. So I'd like to group the above as: File "", line 2, in f File "", line 5, in g File "", line 9, in h *** above 3 calls repeated 3 times *** File "", line 7, in f File "", line 5, in g File "", line 9, in h -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
Steven D'Aprano writes: > I want to group repeated items in a sequence. For example, I can group > repeated sequences of a single item at a time using groupby: [...] > Now I want to group subsequences. For example, I have: > > "ABCABCABCDEABCDEFABCABCABCB" > > and I want to group it into repeating subsequences. I can see two ways to > group it: > > ABC ABC ABCDE ABCDE F ABC ABC ABC B [...] > or: > > ABC ABC ABC D E A B C D E F ABC ABC ABC B [...] > How can I do this? Does this problem have a standard name and/or solution? Looks like a tough one. I don't think it has a name (I'm not even sure to be able to formally define it). Lets say it is an instance of "longest repeating substring" (https://en.wikipedia.org/wiki/Longest_repeated_substring_problem -- which really does not say much). Anyway, it looks like a job for a suffix trees. Depending on what you are after, you may also be interested in the sequitur algorithm (http://www.sequitur.info/). -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Detecting repeated subsequences of identical items
On Thu, Apr 21, 2016 at 2:55 AM Vlastimil Brom wrote: > 2016-04-21 5:07 GMT+02:00 Steven D'Aprano : > > I want to group subsequences. > > "ABCABCABCDEABCDEFABCABCABCB" > > ABC ABC ABCDE ABCDE F ABC ABC ABC B > > or: > > ABC ABC ABC D E A B C D E F ABC ABC ABC B > > if I am not missing something, the latter form of grouping might be > achieved with the following regex: [snip] > The former one seems to be more tricky... > Right. If the problem is constrained to say that repeated subsequences can have no nested repeated subsequences, it's much easier to solve. If you had "ABCABCABCABC" should that result in ABC ABC ABC ABC, with 4 repetitions or ABCABC ABCABC with 2 repetitions? In this example, one might say the higher count is obviously better, but I think it depends on the context. Maybe the user is looking for the biggest patterns rather than the biggest counts. -- https://mail.python.org/mailman/listinfo/python-list
