Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?
Since this topic is not focused on efficiency, why exactly does it matter if your function should check if a file exists and then avoid a collision? What you are describing sounds a bit like a hash function. In many cases, when you perform the computations you may find the slot you hash to is already occupied. A common solution is to check and if needed make some adjustment like adding 1 or using a second hash function or changing the thing being hashed in a deterministic way like changing "key" to "keykey" and even "keykeykey" before hashing till you get an empty slot. Note in Python, you can simply use a dictionary and have no idea how the hash is done. Since you are describing a need for a permanent file, many suggested ways of making a unique file name can repeat and thus not be unique. The process ID wraps around and maybe restarts every time the machine boots, I suspect. Dates repeat unless they include the year as in 2018101903 to the nearest second. If you are concerned about two files being created in the same second, there is a trivial solution. Before or after creating the file, sleep for a time period like a second so any "second" saved file is guaranteed to come at least a second later, in the above scheme. I heard a concern about what happens if you just use sequence numbers as in file3 then file4 if the user later deletes some. The concern was subtle about what happens if a file is later deleted and your algorithm later will search for the first available empty slot. To each their own but it strikes me as fairly easy in Python to not check EVERY filename but skip past the last one. You have functions that return a list of filenames in a directory as a list of strings. In my example, you could take the list and replace "file" with nothing to get [ "1", "2", "4", ... ] Then in memory, efficiently, without opening a single file, you can sort the strings, pop off the last one, convert it to an int, add one, make a new file name like "file00111" that is guaranteed not to exist unless some other program sneaks it in, and continue. There are lots of ideas you can try. This would only need to happen perhaps once per game and you can encapsulate the logic in your own function so all your code will say is something like: save_game_info(...) Of course if someone else had already come up with similar functionality, use it. Oddly, you may not be aware that your method indicates thinking too much inside the box. Instead of saving files on your own, consider creating a data structure and letting known modules save it in the file system and retrieve it later. There are packages like pickle and shelve. https://docs.python.org/3/library/pickle.html https://docs.python.org/3/library/shelve.html So if you load an existing object, such as a dictionary or list before a game starts, and after each game extend the object and then save it to disk this way, then you don't need to deal with details as long as the program knows where it is stored. The person running the program may not trivially be able to access the data and is not likely to delete easily. The actual file names may depend on what the package does and you may find other such packages better fit your needs and let you easily save all kinds of info like scores and time elapsed and number of moves as you are saving entire data structures you create and control. Just some thoughts. Avi -Original Message- From: Tutor On Behalf Of boB Stepp Sent: Monday, October 22, 2018 8:31 PM To: tutor Subject: Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files? On Mon, Oct 22, 2018 at 11:57 AM Mats Wichmann wrote: > > On 10/22/18 8:24 AM, boB Stepp wrote: > > Forwarding to the Tutor list. Herr Maier offers a good idea that > > would take away much of a remaining issue -- the name "Temporary". > > I need to look into the functools library to see what "partial" does. > > > if you really don't care what the file is called because you will > maintain a map which leads you to the filename, you might as well use > a uuid. Wouldn't I have to write a check to ensure such a named file (However unlikely that would be.) did not exist before creating it? And if yes, would not that get into the same potential security issue that cause tempfile.mktemp() to be deprecated? -- boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?
> On Oct 22, 2018, at 8:30 PM, boB Stepp wrote: > > On Mon, Oct 22, 2018 at 11:57 AM Mats Wichmann wrote: >> >> On 10/22/18 8:24 AM, boB Stepp wrote: >>> Forwarding to the Tutor list. Herr Maier offers a good idea that >>> would take away much of a remaining issue -- the name "Temporary". I >>> need to look into the functools library to see what "partial" does. >> >> >> if you really don't care what the file is called because you will >> maintain a map which leads you to the filename, you might as well use a >> uuid. > > Wouldn't I have to write a check to ensure such a named file (However > unlikely that would be.) did not exist before creating it? And if > yes, would not that get into the same potential security issue that > cause tempfile.mktemp() to be deprecated? > The whole point of UUIDs is to make the probability of a UUID collision so infinitesimally small as to make that hypothetical collision simply not worth worrying about, even when they are created on different systems. Since a big chunk of a UUID is a high precision time stamp, any UUIDs created on a single system are pretty much guaranteed to be unique. Bill > -- > boB > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?
On Mon, Oct 22, 2018 at 11:57 AM Mats Wichmann wrote: > > On 10/22/18 8:24 AM, boB Stepp wrote: > > Forwarding to the Tutor list. Herr Maier offers a good idea that > > would take away much of a remaining issue -- the name "Temporary". I > > need to look into the functools library to see what "partial" does. > > > if you really don't care what the file is called because you will > maintain a map which leads you to the filename, you might as well use a > uuid. Wouldn't I have to write a check to ensure such a named file (However unlikely that would be.) did not exist before creating it? And if yes, would not that get into the same potential security issue that cause tempfile.mktemp() to be deprecated? -- boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?
On 10/22/18 8:24 AM, boB Stepp wrote: > Forwarding to the Tutor list. Herr Maier offers a good idea that > would take away much of a remaining issue -- the name "Temporary". I > need to look into the functools library to see what "partial" does. if you really don't care what the file is called because you will maintain a map which leads you to the filename, you might as well use a uuid. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?
Forwarding to the Tutor list. Herr Maier offers a good idea that would take away much of a remaining issue -- the name "Temporary". I need to look into the functools library to see what "partial" does. -- Forwarded message - From: Wolfgang Maier Date: Mon, Oct 22, 2018 at 5:25 AM Subject: Re: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files? To: boB Stepp On 21.10.18 08:13, boB Stepp wrote: > Use case: I want to allow a user of my Solitaire Scorekeeper program > to be able to give any name he wants to each game of solitaire he > wishes to record. My thought for permanent storage of each game's > parameters is to use a dictionary to map the user-chosen game names to > a unique json filename. This way I hope no shenanigans can occur with > weird characters/non-printing characters. > > My initial thought was to just have a sequence of game names with > incrementing numerical suffixes: game_0, game_1, ... , game_n. But > this would require the program to keep track of what the next > available numerical suffix is. Additionally, if a user chooses to > delete a game, then there would be a gap in the numerical sequence of > the game filenames. I find such a gap aesthetically displeasing and > would probably go to additional efforts to reuse such deleted > filenames, so there would be no such "gaps". > > So I am now wondering if using > tempfile.NamedTemporaryFile(delete=False) would solve this problem > nicely? As I am not very familiar with this library, are there any > unforeseen issues I should be made aware of? Would this work equally > well on all operating systems? > > TIA! > This sounds like a good though surprising use case. The only odd thing about this is the misleading name then of the function, plus there is the (vague) possibility that the stdlib module might evolve and no longer support this undocumented (given that the first sentence in the module description reads: "This module creates temporary files and directories.") use case. I would probably address these issues and the handling of the dir parameter via a partial function like this: from functools import partial SavedGameFile = partial( tempfile.NamedTemporaryFile, dir=my_saved_games_dir ) # if you like also set the suffix here SavedGameFile.__doc__ = """\ Create and return a saved game file, ... """ This way you have a good name for the function, and you can redefine the function, if you ever want/need to move away from NamedTemporaryFile, without having to rewrite every function call. Wolfgang -- boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
