Re: ThePath - Convenient lib to deal with paths and files. (Alpha version)

2023-01-19 Thread Dmytro Katyukha via Digitalmars-d-announce

On Tuesday, 17 January 2023 at 23:12:26 UTC, H. S. Teoh wrote:
On Sun, Jan 15, 2023 at 01:53:51PM +, Dmytro Katyukha via 
Digitalmars-d-announce wrote: [...]

[...]


Yes it would be nice.  But there may be security implications.  
For Posix, I see you use mkdtemp, which is secured by the OS / 
libc implementor.  But for non-Posix, you used std.random; this 
is insecure because std.random is not intended for 
cryptographic applications, and anything not designed for 
crytographic security is vulnerable to exploits.  Also, you 
need to be careful with the default permissions with the temp 
directory is created; leaving it up to whatever's set in the 
user's environment is generally unwise.


[...]


Hi,

Thank you for your feedback)


Re: ThePath - Convenient lib to deal with paths and files. (Alpha version)

2023-01-17 Thread H. S. Teoh via Digitalmars-d-announce
On Sun, Jan 15, 2023 at 01:53:51PM +, Dmytro Katyukha via 
Digitalmars-d-announce wrote:
[...]
> Also, this lib contains function
> [createTempDirectory](https://github.com/katyukha/thepath/blob/master/source/thepath/utils.d),
> that, i think, would be nice to have it in Phobos.

Yes it would be nice.  But there may be security implications.  For
Posix, I see you use mkdtemp, which is secured by the OS / libc
implementor.  But for non-Posix, you used std.random; this is insecure
because std.random is not intended for cryptographic applications, and
anything not designed for crytographic security is vulnerable to
exploits.  Also, you need to be careful with the default permissions
with the temp directory is created; leaving it up to whatever's set in
the user's environment is generally unwise.


> So, the questions are:
> - Do it have sense to convert `Path` to a class? Or keep it as struct?

Struct.  In general, idiomatic D code prefers structs over classes. If
you're not using inheritance and runtime polymorphism, there's no need
to use classes.


> - Do it have sense to convert `Path` to template struct to make it
>   possible to work with other types of strings (except `string` type)?

IMO, this only introduces needless complexity.  For example std.regex
templatizes over char/wchar/dchar, but I've basically never needed to
use anything except the char instantiation.  This needless template
parametrization only adds to std.regex's slow compile times; in
retrospect it was IMO a mistake.  Regular D code should just use strings
(UTF-8) for everything, and convert to wstring at the OS boundary if
you're on Windows and need something to be in UTF-16.  And dstring is
essentially useless; I've not heard of anyone needing to use dstring for
the 10 or so years I've been using D.

Just use string, that's good enough.


> - What are the requirements to place 
> [createTempDirectory](https://github.com/katyukha/thepath/blob/master/source/thepath/utils.d#L11)
>   function in Phobos?

Use Phobos coding style, bring it up to Phobos coding standards.


> - What else could be changed to make it better?
[...]

Probably should always use the libc or OS function for creating a temp
directory; it's generally bad idea to roll your own when it comes to
creating temporary files or directories where there can be serious
security implications. Other than insecure random name generation,
there's also timing issues to be considered, i.e., if an attacker could
predict the name, he could preemptively create the directory with the
wrong permissions between your call to std.file.exists and
std.file.mkdir, and exploit those permissions to manipulate the
behaviour of your program later.  You need to leverage OS APIs to
guarantee the atomicity of checking for existence and creating the
directory.


T

-- 
Ignorance is bliss... until you suffer the consequences!


ThePath - Convenient lib to deal with paths and files. (Alpha version)

2023-01-15 Thread Dmytro Katyukha via Digitalmars-d-announce

Hi,

I would like to anounce the new 
[ThePath](https://code.dlang.org/packages/thepath) published to 
https://code.dlang.org/.


The basic functionality of 
[ThePath](https://code.dlang.org/packages/thepath) library seems 
to be completed, and before stabilizing it, i would like to ask 
community to review this lib and suggest what could be added / 
changed to make it usable for your usecases. Also, currently, 
this lib is tested only on linux, thus, may be some of you would 
be interested in making it cross-platform (help with testing and 
adding unittests for other platforms (Windows, MacOs, etc)). It 
should work on other platforms, but possibly with restricted 
functionality.


Basically, this lib provides single struct `Path`, that have to 
be used to deal with file system paths and files in 
object-oriented way, making code more readable.


Also, this lib contains function 
[createTempDirectory](https://github.com/katyukha/thepath/blob/master/source/thepath/utils.d), that, i think, would be nice to have it in Phobos.


So, the questions are:
- Do it have sense to convert `Path` to a class? Or keep it as 
struct?
- Do it have sense to convert `Path` to template struct to make 
it possible to work with other types of strings (except `string` 
type)?
- What are the requirements to place 
[createTempDirectory](https://github.com/katyukha/thepath/blob/master/source/thepath/utils.d#L11) function in Phobos?

- What else could be changed to make it better?

Short list of features:
- automatic expansion of `~` when needed (before passing path to 
std.file or std.stdio funcs)

- single method to copy path (file or directory) to dest path
- single method to remove path (file or directory)
- simple method to `walk` through the path
- `foreach(p; Path.current.walk) writeln(p.toString);`
- `foreach(p; Path("/tmp").walk) writeln(p.toString);`
- simple construction of paths from parts:
- `Path("a", "b", "c")`
- `Path("a").join("b", "c")`
- simple deconstruction of paths
- `Path("a/b/c/d").segments == ["a", "b", "c", "d"]`
- `Path("a", "b", "c", "d").segments == ["a", "b", "c", "d"]`
- overriden comparison operators for paths.
- `Path("a", "b") == Path("a", "b")`
- `Path("a", "b") != Path("a", "c")`
- `Path("a", "b") < Path("a", "c")`
- `hasAttributes` / `getAttributes` / `setAttributes` methods to 
work with file attrs

- file operations as methods:
- `Path("my-path").writeFile("Hello world")`
- `Path("my-path").readFile()`
- support search by glob-pattern
- `foreach(path; Path.current.glob("*.py")) 
writeln(p.toString);`



Short example is described below. More examples available in 
unittests and documentation.



```d
import thepath;


Path app_dir = Path("~/.local/my-app");
Path catalog_dir = app_dir.join("catalog");


void init() {
// Note, that automatic '~' expansion will be done before 
checking the

// existense of directory
if (!app_dir.exists) {
app_dir.mkdir(true);  // create recursive
}
if (!catalog_dir.exists) {
catalog_dir.mkdir(true);
}
}

void list_dir() {
// Easily print content of the catalog directory
foreach(Path p; catalog_dir.walkBreadth) {
writeln(p.toAbsolute().toString());
}
}

// Print all python files in current directory
void find_python_files() {
foreach(path; Path.current.glob("*.py", SpanMode.breadth))
// Print paths relative to current directory
writeln(p.relativeTo(Path.current).toString);
}

Path findConfig() {
// Search for "my-project.conf" in current directories and in
// its parent directories
auto config = Path.current.searchFileUp("my-project.conf");
enforce(!config.isNull);
return config.get;
}
```