Re: [Tutor] What is the best way for a program suite to know where it is installed?

2018-10-23 Thread Peter Otten
eryk sun wrote:

> On 10/22/18, boB Stepp  wrote:
>>
>> Importing the various program modules/module contents is
>> no issue.  Where I believe I need to know the paths to things are to
>> get to data folders, config files, and occasionally utility programs
>> that I have written that are on my hard drive, but not copied to my
>> current program suite.
> 
> It's common to use __file__ for a portable application, except you
> should use sys.executable if it's a frozen executable, i.e. if
> sys.frozen exists.
> 
> For an installed application, limit this approach to the application's
> immutable resources. Don't use the installation directory to store
> modifiable data. You may not have write access (e.g. a system
> installation and the user lacks root/admin access, or it's an
> immutable directory or read-only disk mount).

I've not tried it, but for per-app read-only data importlib.resources
appears to be the right tool.

https://docs.python.org/dev/library/importlib.html#module-importlib.resources


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

2018-10-23 Thread Cameron Simpson

On 23Oct2018 11:24, Peter Otten <__pete...@web.de> wrote:

Cameron Simpson wrote:
The doco for mktemp (do not use! use mkstemp or the 
NamedTemporaryFile

classes instead!) explicitly mentions using delete=False.


Well, "permanent temporary file" does sound odd.

By the way, NamedTemporaryFile returns a proxy instead of the file itself.
In some rare cases that could be a problem.

Would mktemp() really be dangerous if you used it like this,

def new_game(directory):
   for _retry in range(3):
   filename = mktemp("game_", ".json", dir=directory)
   try:
  return open(filename, "x")
   except FileExistsError:
  pass
   raise FileExistsError

with the "x" mode?


In terms of a race, maybe not. But in terms of security? Probably.

Consider: the issue with mktemp is that it can be switched out before 
use. So:


Alice: mktemp() -> filename

Mallory: guess filename, put a symlink there pointing at a file which 
doesn't exist, but which has an effect if it does. For example, in 
ancient windows, an autorun.ini file. Or cooler, on UNIX, a file in 
/etc/cron.d.


Alice: write to filename, not accidentally _creating_ the target of the 
symlink, now writing a file somewhere unwanted.


Now, the examples above pretend that Alice has root privileges so that 
Mallory affects a root run system. But for Alice, it is just as bad if 
Mallory just subverts her personal account via some other pathname.


Also, there's the issue of privacy: open(), in your example, will use 
the default umask, which may be more open than one wants. And so on...


Cheers,
Cameron Simpson 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

2018-10-23 Thread Peter Otten
Cameron Simpson wrote:

> On 21Oct2018 10:55, Peter Otten <__pete...@web.de> wrote:
>>boB Stepp wrote:
>>> So I am now wondering if using
>>> tempfile.NamedTemporaryFile(delete=False) would solve this problem
>>> nicely?  As I am not very familiar with this library, are there any
>>> unforeseen issues I should be made aware of?  Would this work equally
>>> well on all operating systems?
>>
>>I think this is cool thinking outside of the box.
>>
>>I would not have "dared" this, but now you suggest it I cannot see
>>anything wrong with your approach.
> 
> The doco for mktemp (do not use! use mkstemp or the NamedTemporaryFile
> classes instead!) explicitly mentions using delete=False.

Well, "permanent temporary file" does sound odd.

By the way, NamedTemporaryFile returns a proxy instead of the file itself. 
In some rare cases that could be a problem.

Would mktemp() really be dangerous if you used it like this,

def new_game(directory):
for _retry in range(3):
filename = mktemp("game_", ".json", dir=directory)
try:
   return open(filename, "x")
except FileExistsError:
   pass
raise FileExistsError


with the "x" mode?




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

2018-10-23 Thread Avi Gross
Since this topic is not focused on efficiency, why exactly does it matter if
your function should check if a file exists and then avoid a collision?

What you are describing sounds a bit like a hash function. In many cases,
when you perform the computations you may find the slot you hash to is
already occupied. A common solution is to check and if needed make some
adjustment like adding 1 or using a second hash function or changing the
thing being hashed in a deterministic way like changing "key" to "keykey"
and even "keykeykey" before hashing till you get an empty slot. Note in
Python, you can simply use a dictionary and have no idea how the hash is
done.

Since you are describing a need for a permanent file, many suggested ways of
making a unique file name can repeat and thus not be unique. The process ID
wraps around and maybe restarts every time the machine boots, I suspect.
Dates repeat unless they include the year as in 2018101903 to the
nearest second. If you are concerned about two files being created in the
same second, there is a trivial solution. Before or after creating the file,
sleep for a time period like a second so any "second" saved file is
guaranteed to come at least a second later, in the above scheme. 

I heard a concern about what happens if you just use sequence numbers as in
file3 then file4 if the user later deletes some. The concern was
subtle about what happens if a file is later deleted and your algorithm
later will search for the first available empty slot. To each their own but
it strikes me as fairly easy in Python to not check EVERY filename but skip
past the last one.

You have functions that return a list of filenames in a directory as a list
of strings. In my example, you could take the list and replace "file" with
nothing to get [ "1", "2", "4", ... ]

Then in memory, efficiently, without opening a single file, you can sort the
strings, pop off the last one, convert it to an int, add one, make a new
file name like "file00111" that is guaranteed not to exist unless some other
program sneaks it in, and continue.

There are lots of ideas you can try. This would only need to happen perhaps
once per game and you can encapsulate the logic in your own function so all
your code will say is something like:
save_game_info(...)

Of course if someone else had already come up with similar functionality,
use it. 

Oddly, you may not be aware that your method indicates thinking too much
inside the box. Instead of saving files on your own, consider creating a
data structure and letting known modules save it in the file system and
retrieve it later. There are packages like pickle and shelve.

https://docs.python.org/3/library/pickle.html
https://docs.python.org/3/library/shelve.html

So if you load an existing object, such as a dictionary or list before a
game starts, and after each game extend the object and then save it to disk
this way, then you don't need to deal with details as long as the program
knows where it is stored. The person running the program may not trivially
be able to access the data and is not likely to delete easily. The actual
file names may depend on what the package does and you may find other such
packages better fit your needs and let you easily save all kinds of info
like scores and time elapsed and number of moves as you are saving entire
data structures you create and control.

Just some thoughts.

Avi

-Original Message-
From: Tutor  On Behalf Of boB
Stepp
Sent: Monday, October 22, 2018 8:31 PM
To: tutor 
Subject: Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be
used to create *permanent* uniquely named files?

On Mon, Oct 22, 2018 at 11:57 AM Mats Wichmann  wrote:
>
> On 10/22/18 8:24 AM, boB Stepp wrote:
> > Forwarding to the Tutor list.  Herr Maier offers a good idea that 
> > would take away much of a remaining issue -- the name "Temporary".  
> > I need to look into the functools library to see what "partial" does.
>
>
> if you really don't care what the file is called because you will 
> maintain a map which leads you to the filename, you might as well use 
> a uuid.

Wouldn't I have to write a check to ensure such a named file (However
unlikely that would be.) did not exist before creating it?  And if yes,
would not that get into the same potential security issue that cause
tempfile.mktemp() to be deprecated?


--
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor