Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Gary Gregory]

You're approach is fine. In this case it is better to talk it out here IMO,
before writing code in a PR. I'm not sure it is possible to change the
existing File-based API and have behavior that kinda matches across all
Commons IO versions in a way that will make everyone happy. The good thing
is that you have a way forward that works.

Gary

On Sat, Aug 20, 2022, 16:45 Tres Finocchiaro 
wrote:

> >
> > What about using an empty array as input to PathUtils.copyDirectory(Path,
> > Path, CopyOption...) ?
>
>
> Yes, as long as StandardCopyOptions.*REPLACE_EXISTING* is provided, which
> seems to mimic the former:
>
> PathUtils.copyDirectory(folder.toPath(), dest.toPath(),
> > StandardCopyOption.REPLACE_EXISTING);
>
>
> This new PathUtils API does work as expected.
>
> In my case, I'm already using Path (not File) internally in most places, so
> this could be argued as a slight improvement, however the lack of a
> FileUtils.copyFile(...) equivalent still means we need to -- for now --
> keep around the lingering "false" in places that we copy single files while
> changing their names.
>
> > You can always provide a PR on GitHub to see what a fix would look like.
>
>
> Do you recommend this direct-proposal approach versus submitting a bug
> report and getting buy-in first?
>
> In its current form copying Windows files is (in my opinion) completely
> broken, so I would suspect the the only instances that would complain are:
>
>- Edge-cases with Unix (I would need to test the impact, similar to how
>I used "icacls" on Windows)
>- Windows environments that were accidentally benefiting from this bug,
>which is really people who really wanted COPY_ATTRIBUTES but never
> provided
>this as an option.
>
> The former (to me) is more concerning.
>
> -Tres
>
> - tres.finocchi...@gmail.com
>
>
> On Sat, Aug 20, 2022 at 4:13 PM Gary Gregory 
> wrote:
>
> > Tres,
> >
> > Yeah, I/we got caught here in the desire to provide backward
> compatibility
> > and update the implementation to NIO. You might want to also try the
> > 2.12.0-SNAPSHOT from
> >
> >
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.12.0-SNAPSHOT/
> >
> > What about using an empty array as input to PathUtils.copyDirectory(Path,
> > Path, CopyOption...) ?
> >
> > Gary
> >
> > On Sat, Aug 20, 2022 at 3:17 PM Tres Finocchiaro <
> > tres.finocchi...@gmail.com>
> > wrote:
> >
> > > Gary,
> > >
> > > Thanks again for your reply however COPY_ATTRIBUTES is the source of
> the
> > > problem, it's being added and wiping out the permissions.
> > >
> > > I've linked my suspicion as to why in my previous email.
> > >
> > > For now, I will toggle-off the preserveFileDate, which seems to be the
> > only
> > > workaround for this problem.
> > >
> > > Best regards.
> > >
> > >
> > > - tres.finocchi...@gmail.com
> > >
> > > >
> > > >
> > >
> >
>

Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Tres Finocchiaro]

>
> What about using an empty array as input to PathUtils.copyDirectory(Path,
> Path, CopyOption...) ?


Yes, as long as StandardCopyOptions.*REPLACE_EXISTING* is provided, which
seems to mimic the former:

PathUtils.copyDirectory(folder.toPath(), dest.toPath(),
> StandardCopyOption.REPLACE_EXISTING);


This new PathUtils API does work as expected.

In my case, I'm already using Path (not File) internally in most places, so
this could be argued as a slight improvement, however the lack of a
FileUtils.copyFile(...) equivalent still means we need to -- for now --
keep around the lingering "false" in places that we copy single files while
changing their names.

> You can always provide a PR on GitHub to see what a fix would look like.


Do you recommend this direct-proposal approach versus submitting a bug
report and getting buy-in first?

In its current form copying Windows files is (in my opinion) completely
broken, so I would suspect the the only instances that would complain are:

   - Edge-cases with Unix (I would need to test the impact, similar to how
   I used "icacls" on Windows)
   - Windows environments that were accidentally benefiting from this bug,
   which is really people who really wanted COPY_ATTRIBUTES but never provided
   this as an option.

The former (to me) is more concerning.

-Tres

- tres.finocchi...@gmail.com


On Sat, Aug 20, 2022 at 4:13 PM Gary Gregory  wrote:

> Tres,
>
> Yeah, I/we got caught here in the desire to provide backward compatibility
> and update the implementation to NIO. You might want to also try the
> 2.12.0-SNAPSHOT from
>
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.12.0-SNAPSHOT/
>
> What about using an empty array as input to PathUtils.copyDirectory(Path,
> Path, CopyOption...) ?
>
> Gary
>
> On Sat, Aug 20, 2022 at 3:17 PM Tres Finocchiaro <
> tres.finocchi...@gmail.com>
> wrote:
>
> > Gary,
> >
> > Thanks again for your reply however COPY_ATTRIBUTES is the source of the
> > problem, it's being added and wiping out the permissions.
> >
> > I've linked my suspicion as to why in my previous email.
> >
> > For now, I will toggle-off the preserveFileDate, which seems to be the
> only
> > workaround for this problem.
> >
> > Best regards.
> >
> >
> > - tres.finocchi...@gmail.com
> >
> > >
> > >
> >
>

Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Gary Gregory]

You can always provide a PR on GitHub to see what a fix would look like.
The hard parts are maintaining compatibility with the current version as
opposed to an antique version and updating the tests. Going back to the 2.4
behavior will obviously cause people expecting the 2.11.0 behavior to
complain. Sometimes the only way to address this is (1) a new API or (2)
update the client code to call other APIs with different input.

Gary

On Sat, Aug 20, 2022 at 3:25 PM Tres Finocchiaro 
wrote:

> I've answered my own question on stackoverflow.
>
> I'm curious how the community feels about filing this as a bug.  It's
> pretty big regression for those upgrading from 2.4 with Windows target
> deployments.
>
>
> - tres.finocchi...@gmail.com
>
>
> On Sat, Aug 20, 2022 at 3:16 PM Tres Finocchiaro <
> tres.finocchi...@gmail.com>
> wrote:
>
> > Gary,
> >
> > Thanks again for your reply however COPY_ATTRIBUTES is the source of the
> > problem, it's being added and wiping out the permissions.
> >
> > I've linked my suspicion as to why in my previous email.
> >
> > For now, I will toggle-off the preserveFileDate, which seems to be the
> > only workaround for this problem.
> >
> > Best regards.
> >
> >
> > - tres.finocchi...@gmail.com
> >
> >>
> >>
>

Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Gary Gregory]

Tres,

Yeah, I/we got caught here in the desire to provide backward compatibility
and update the implementation to NIO. You might want to also try the
2.12.0-SNAPSHOT from
https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.12.0-SNAPSHOT/

What about using an empty array as input to PathUtils.copyDirectory(Path,
Path, CopyOption...) ?

Gary

On Sat, Aug 20, 2022 at 3:17 PM Tres Finocchiaro 
wrote:

> Gary,
>
> Thanks again for your reply however COPY_ATTRIBUTES is the source of the
> problem, it's being added and wiping out the permissions.
>
> I've linked my suspicion as to why in my previous email.
>
> For now, I will toggle-off the preserveFileDate, which seems to be the only
> workaround for this problem.
>
> Best regards.
>
>
> - tres.finocchi...@gmail.com
>
> >
> >
>

Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Tres Finocchiaro]

I've answered my own question on stackoverflow.

I'm curious how the community feels about filing this as a bug.  It's
pretty big regression for those upgrading from 2.4 with Windows target
deployments.


- tres.finocchi...@gmail.com


On Sat, Aug 20, 2022 at 3:16 PM Tres Finocchiaro 
wrote:

> Gary,
>
> Thanks again for your reply however COPY_ATTRIBUTES is the source of the
> problem, it's being added and wiping out the permissions.
>
> I've linked my suspicion as to why in my previous email.
>
> For now, I will toggle-off the preserveFileDate, which seems to be the
> only workaround for this problem.
>
> Best regards.
>
>
> - tres.finocchi...@gmail.com
>
>>
>>

Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Tres Finocchiaro]

Gary,

Thanks again for your reply however COPY_ATTRIBUTES is the source of the
problem, it's being added and wiping out the permissions.

I've linked my suspicion as to why in my previous email.

For now, I will toggle-off the preserveFileDate, which seems to be the only
workaround for this problem.

Best regards.


- tres.finocchi...@gmail.com

>
>

Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Gary Gregory]

Hello Tres,

I think what you want is to call FileUtils.copyDirectory() or
PathUtils.copyDirectory()
with java.nio.file.StandardCopyOption.COPY_ATTRIBUTES.

Gary

On Sat, Aug 20, 2022 at 2:51 PM Tres Finocchiaro 
wrote:

> >
> > Use *org.apache.commons.io.FileUtils.copyFile(File, File, boolean,
> > CopyOption...)* to exercise full control over what you want to do.
> >
>
> Thanks for the reply.
>
> From my understanding, CopyOption supports one or more of three
> options: REPLACE_EXISTING, COPY_ATTRIBUTES, NOFOLLOW_LINKS, none of which
> help with the aforementioned question.  Do you agree?
>
> The behavior of Windows is to preserve the destination permissions, per
> Microsoft:
>
> "By default, an object inherits permissions from its parent object, either
> > at the time of creation or when it is copied or moved to its parent
> folder.
> > The only exception to this rule occurs when you move an object to a
> > different folder on the same volume. In this case, the original
> permissions
> > are retained."
>
>
> So, the default copy behavior is not useful on Windows except
> edge-cases where people are looking to preserve permissions, which is
> non-standard and should not be default.
>
> Under the covers it ends up calling *java.nio.file.Files.copy(Path, Path,
> > CopyOption...)*.
>
>
> I'm happy to try a workaround that uses CopyOption to remedy this, but I'm
> not really sure where to start since the above options don't look like
> they'll help.
>

Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Tres Finocchiaro]

>
> Under the covers it ends up calling *java.nio.file.Files.copy(Path, Path,
> CopyOption...)*.


Sure, but from what I'm reading, it's adding CopyOptions that no one asked
for and is not in alignment with how NIO works, so this statement is a bit
misleading.

I think this is the problematic line:
https://github.com/apache/commons-io/blob/f22a4227401855ecbfdf8184bbe37275c3aeb5c3/src/main/java/org/apache/commons/io/FileUtils.java#L701

>From what I'm reading, toggling off "preserveFileDate" will
inadvertently and unobviously fix this, but at this point.  This fix seems
like undesired, undocumented and prone-to-break-later behavior, at least
for Windows environments.

I'm curious what reasoning was used to associate a file's modified date
with the rest of its attributes.  Perhaps this was a backward-compat
shortcut to maintain the modified date using the new NIO API?  Regardless,
this introduces permissions changes which can and will break standard file
copy operations for environments which expect inherited permissions, e.g.
Windows.

Ideally, the file modified date could stay without clobbering the default
NIO behavior.

For now a viable workaround is:

FileUtils.copyDirectory(folder, dest, false /* <--- change to FALSE */);


... hjowever this does have the disadvantage of losing timestamp
information, which is still a regression, albeit smaller.

Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Tres Finocchiaro]

>
> Use *org.apache.commons.io.FileUtils.copyFile(File, File, boolean,
> CopyOption...)* to exercise full control over what you want to do.
>

Thanks for the reply.

>From my understanding, CopyOption supports one or more of three
options: REPLACE_EXISTING, COPY_ATTRIBUTES, NOFOLLOW_LINKS, none of which
help with the aforementioned question.  Do you agree?

The behavior of Windows is to preserve the destination permissions, per
Microsoft:

"By default, an object inherits permissions from its parent object, either
> at the time of creation or when it is copied or moved to its parent folder.
> The only exception to this rule occurs when you move an object to a
> different folder on the same volume. In this case, the original permissions
> are retained."


So, the default copy behavior is not useful on Windows except
edge-cases where people are looking to preserve permissions, which is
non-standard and should not be default.

Under the covers it ends up calling *java.nio.file.Files.copy(Path, Path,
> CopyOption...)*.


I'm happy to try a workaround that uses CopyOption to remedy this, but I'm
not really sure where to start since the above options don't look like
they'll help.

Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Gary Gregory]

Under the covers it ends up calling *java.nio.file.Files.copy(Path, Path,
CopyOption...)*.

On Sat, Aug 20, 2022 at 2:30 PM Gary Gregory  wrote:

> Hello Tres,
>
> Use *org.apache.commons.io.FileUtils.copyFile(File, File, boolean,
> CopyOption...)* to exercise full control over what you want to do.
>
> Gary
>
>
> On Sat, Aug 20, 2022 at 12:30 PM Tres Finocchiaro <
> tres.finocchi...@gmail.com> wrote:
>
>> Hi,
>>
>> I'm updating some dependent libraries and hit a snag with commons-io.  I
>> was hoping someone here could help.
>>
>> I've noticed a major change in behavior with FileUtils.copyDirectory()
>> between commons-io@2.4 and commons-io@2.11 that affects the permission of
>> files being copied.
>>
>> I've posted the technical details here, with steps to reproduce:
>> https://stackoverflow.com/questions/73428286
>>
>> In short:
>>
>>1. The default 2.4 behavior was to inherit permissions of the target
>>2. The default 2.11 behavior seems to not inherit permissions of the
>>target
>>
>> Any insight is greatly appreciated.
>>
>> - tres.finocchi...@gmail.com
>>
>

Re: [io] Regression in FileUtils.copyDirectory() for Windows?

[Gary Gregory]

Hello Tres,

Use *org.apache.commons.io.FileUtils.copyFile(File, File, boolean,
CopyOption...)* to exercise full control over what you want to do.

Gary


On Sat, Aug 20, 2022 at 12:30 PM Tres Finocchiaro <
tres.finocchi...@gmail.com> wrote:

> Hi,
>
> I'm updating some dependent libraries and hit a snag with commons-io.  I
> was hoping someone here could help.
>
> I've noticed a major change in behavior with FileUtils.copyDirectory()
> between commons-io@2.4 and commons-io@2.11 that affects the permission of
> files being copied.
>
> I've posted the technical details here, with steps to reproduce:
> https://stackoverflow.com/questions/73428286
>
> In short:
>
>1. The default 2.4 behavior was to inherit permissions of the target
>2. The default 2.11 behavior seems to not inherit permissions of the
>target
>
> Any insight is greatly appreciated.
>
> - tres.finocchi...@gmail.com
>

Re: [daemon] Procrun and -javaagent

[Martin Gainty]

Apparently you can but only as a jenkinsWindowService.exe
How to install Windows agents as a service? 
(cloudbees.com)
How to install Windows agents as a service? - 
CloudBees
You can use ssh to connect the agent by following Configuring Windows agents 
Note: This solution does not actually use a Windows service, as the agent 
launch is handeled by your controller automatically.
docs.cloudbees.com
Follow these instructions faithfully to get Windows Service WinSW-64x.exe (now 
renamed as jenkinsWindowService.exe) as a service to run your agents main jar 
working for you

Un saludo

From: Fernando O. 
Sent: Friday, August 19, 2022 8:46 AM
To: Commons Users List 
Subject: [daemon] Procrun and -javaagent

Hi all,
Disclaimer:  I'm pretty new to windows, I worked most of my life on linux

I'm trying to add a java agent to a windows service, that was installed 
with procrun. In the doc it seems like there's no option for adding an agent.
I tried adding it to options (the doc says that accepts -D and -X) but just in 
case I tried with -javaagent as in command line but it's failing. I can't find 
a way to get proper logs on what's going on either. Does anyone know if this is 
doable?
Or how to get trace/debug lvl logs from the jvm?

I'ḿ running it in jvm mode.


[image.png]

[io] Regression in FileUtils.copyDirectory() for Windows?

[Tres Finocchiaro]

Hi,

I'm updating some dependent libraries and hit a snag with commons-io.  I
was hoping someone here could help.

I've noticed a major change in behavior with FileUtils.copyDirectory()
between commons-io@2.4 and commons-io@2.11 that affects the permission of
files being copied.

I've posted the technical details here, with steps to reproduce:
https://stackoverflow.com/questions/73428286

In short:

   1. The default 2.4 behavior was to inherit permissions of the target
   2. The default 2.11 behavior seems to not inherit permissions of the
   target

Any insight is greatly appreciated.

- tres.finocchi...@gmail.com