Re: [ros-dev] Pale Moon drops ReactOS support

[David Quintana (gigaherz)]

People need to stop worrying about "jumping in" on this conversation --
it's being discussed on the public mailing list because the opinion of the
community DOES MATTER. so feel free to provide any and all input you think
is relevant.

On 19 May 2016 at 00:28, Dimitrij Klingbeil  wrote:

> Hi all
>
> Sorry for jumping in since I'm not a ReactOS developer (though I've been
> following the progress regularly and looking for any ReactOS news for some
> years already).
>
> From what I think, making some special allowance for the behaviour of
> applications checking APIs whether they exist is certainly a good idea. But
> I don't think that shimming each and every DLL that may have differences is
> the way to go toward this end.
>
> There's an old saying in Linux circles: "Make the common case fast..." -
> but from the maintenance point of view, particularly for a smaller project,
> I'd like to extend it to a slightly different interpretation: Make the
> common case less error-prone!
>
> There are 2 main observations that are the basis for my reasoning. They
> are hardly "scientific", so correct me if you feel that they are wrong.
>
> 1. Most APIs of any one DLL, even if different vintage, can peacefully
> coexist in that same DLL at the same time - exceptions, if any, are rare.
> 2. The most common user-mode check for an API is through GetProcAddress()
>
> The reasoning for 1. is that newer versions usualls only add APIs, but
> very rarely remove them. Deprecated APIs are mostly retained. If the DLL is
> made aware of what the process expects, it could still fine-tune special
> cases where they apply on an as-needed basis. As for 2. this is the
> documented way (rather than calling NTDLL APIs directly, which very few
> applications do, or parsing the export sections of system DLLs, which
> should be extremely rare for a "normal" program.
>
> There is another observation (at least I guess that's the case, my
> knowledge of the Win32 API basically ends with XP):
>
> 3. There seems to be no well-known defined way (yet) to assign to a
> process the information, which API version it expects to see and use.
>
> From my point of view, Microsoft made a huge mistake here, one that
> ReactOS should rather avoid than copy. Having no standardized ability for
> the modules to know (to "publish" the information), on a per-process basis,
> what behaviour is expected by the application, leads to a lot of DLL
> duplication which is a maintenance nightmare. As evidenced by the
> tremendous size increase of the WinSxS directory in each Windows version in
> recent years, even Microsoft seems to have a struggle with it - despite the
> size and resources of that company.
>
> Therefore I would like to propose a ReactOS specific addition in order to
> keep this potential nightmare under control:
>
> Introduce the concept of a process specific ReactOS API version numbering.
>
> In a central place (preferably inside NTDLL, I'll explain later why), in
> each process, there should be located a (new) structure that stores the
> process API version. It should be read-only to the process, accessible only
> through a special (new) API.
>
> Maybe one like this example:
>
> typedef struct {
>DWORD  cbSize;  /* of this structure */
>/* all the data that GetVersionEx may need  */
>OSVERSIONINFOEXOSVersionInfo;
>/* possibly some space for future version-related things */
>BYTE   bReserved[64];
> } NT_API_VERSION_PROCESS;
>
> void NTAPI GetNtApiVersionProcess(NT_API_VERSION_PROCESS *lpVersion);
>
> Although this is just an example, feel free to use something more fitting.
>
> It should be populated early at the process creation, normally with the
> default ReactOS API version (now NT 5.2), but in special cases either a
> compatibility SDB shim or the process loader may write something else here
> (an application-specific dataset).
>
> Particularly, if the loader finds (from the PE header where possible) that
> the application was linked for a higher API version, it should initialize
> the structure to the nearest compatible OS version that ReactOS is (in the
> future hopefully) able to support.
>
> Because NTDLL is (hopefully, my knowledge in this regard may be outdated)
> the first DLL in a process to be loaded, this would make the version info
> easily accessible early during process startup, particularly during the
> DLL_PROCESS_ATTACH phase of all the other DLLs that this process will load
> during its lifetime.
>
> Most DLLs won't need this info, but those that will (Kernel32, User32,
> GDI32, AdvApi32 most likely) can initialize whatever special behaviour they
> need to initialize based on this data. The data is read-only and static for
> the process lifetime, so early initialization should be OK.
>
> A special case: GetProcAddress should respect the per-process API version
> info in order to present the caller with a consistent view of APIs for the
> core DLLs (it should 

Re: [ros-dev] Pale Moon drops ReactOS support

[Dimitrij Klingbeil]

Hi all

Sorry for jumping in since I'm not a ReactOS developer (though I've been 
following the progress regularly and looking for any ReactOS news for some 
years already).


From what I think, making some special allowance for the behaviour of 
applications checking APIs whether they exist is certainly a good idea. But 
I don't think that shimming each and every DLL that may have differences is 
the way to go toward this end.


There's an old saying in Linux circles: "Make the common case fast..." - but 
from the maintenance point of view, particularly for a smaller project, I'd 
like to extend it to a slightly different interpretation: Make the common 
case less error-prone!


There are 2 main observations that are the basis for my reasoning. They are 
hardly "scientific", so correct me if you feel that they are wrong.


1. Most APIs of any one DLL, even if different vintage, can peacefully 
coexist in that same DLL at the same time - exceptions, if any, are rare.

2. The most common user-mode check for an API is through GetProcAddress()

The reasoning for 1. is that newer versions usualls only add APIs, but very 
rarely remove them. Deprecated APIs are mostly retained. If the DLL is made 
aware of what the process expects, it could still fine-tune special cases 
where they apply on an as-needed basis. As for 2. this is the documented way 
(rather than calling NTDLL APIs directly, which very few applications do, or 
parsing the export sections of system DLLs, which should be extremely rare 
for a "normal" program.


There is another observation (at least I guess that's the case, my knowledge 
of the Win32 API basically ends with XP):


3. There seems to be no well-known defined way (yet) to assign to a process 
the information, which API version it expects to see and use.


From my point of view, Microsoft made a huge mistake here, one that ReactOS 
should rather avoid than copy. Having no standardized ability for the 
modules to know (to "publish" the information), on a per-process basis, what 
behaviour is expected by the application, leads to a lot of DLL duplication 
which is a maintenance nightmare. As evidenced by the tremendous size 
increase of the WinSxS directory in each Windows version in recent years, 
even Microsoft seems to have a struggle with it - despite the size and 
resources of that company.


Therefore I would like to propose a ReactOS specific addition in order to 
keep this potential nightmare under control:


Introduce the concept of a process specific ReactOS API version numbering.

In a central place (preferably inside NTDLL, I'll explain later why), in 
each process, there should be located a (new) structure that stores the 
process API version. It should be read-only to the process, accessible only 
through a special (new) API.


Maybe one like this example:

typedef struct {
   DWORD  cbSize;  /* of this structure */
   /* all the data that GetVersionEx may need  */
   OSVERSIONINFOEXOSVersionInfo;
   /* possibly some space for future version-related things */
   BYTE   bReserved[64];
} NT_API_VERSION_PROCESS;

void NTAPI GetNtApiVersionProcess(NT_API_VERSION_PROCESS *lpVersion);

Although this is just an example, feel free to use something more fitting.

It should be populated early at the process creation, normally with the 
default ReactOS API version (now NT 5.2), but in special cases either a 
compatibility SDB shim or the process loader may write something else here 
(an application-specific dataset).


Particularly, if the loader finds (from the PE header where possible) that 
the application was linked for a higher API version, it should initialize 
the structure to the nearest compatible OS version that ReactOS is (in the 
future hopefully) able to support.


Because NTDLL is (hopefully, my knowledge in this regard may be outdated) 
the first DLL in a process to be loaded, this would make the version info 
easily accessible early during process startup, particularly during the 
DLL_PROCESS_ATTACH phase of all the other DLLs that this process will load 
during its lifetime.


Most DLLs won't need this info, but those that will (Kernel32, User32, 
GDI32, AdvApi32 most likely) can initialize whatever special behaviour they 
need to initialize based on this data. The data is read-only and static for 
the process lifetime, so early initialization should be OK.


A special case: GetProcAddress should respect the per-process API version 
info in order to present the caller with a consistent view of APIs for the 
core DLLs (it should know certain core DLLs and filter APIs and return NULL 
for those that are not supposed to exist form the point of view of the 
current process). The dynamic linker / process loader should also do 
accordingly (preferably by sharing the code, but treating them as unresolved 
externals). This way the early-load core DLLs (Kernel32, User32, GDI32, 
AdvAPI32, plus NTDLL itself) can be kept in a single version and 

Re: [ros-dev] Regarding r71352 and SubGit

[Pierre Schweitzer]

Complementary to the work already done by Colin, Fisheye backups have
been restored so that SVN cache is consistent with Colin's rollback.
Same goes to git. Todays commits will be replayed on next commit for
Git. Fisheye is already up to date.

Never ever this thing on our infra. Be glad we have backups.

Le 18/05/2016 18:51, Colin Finck a écrit :
> Hello all,
> 
> As most of you know, I have been busy setting up a two-way Git mirror of
> our ReactOS repository using SubGit.
> Testing was done in the recent weeks by multiple people using a sandbox
> repository and the results looked pretty well. Therefore, SubGit was
> considered ready for prime time and I started a first import of our
> ReactOS repository.
> 
> The only problem with this: Even without mirroring branches, the Git
> repository became a 5.5GB monster (from the 750MB of our current git-svn
> mirror). This size was totally not clonable as shown in testing.
> The proposed solution: Git offers a garbage cleaner using "git gc
> --aggressive". And hell yeah, it reduced the size to a nice 500MB repo.
> 
> While that optimization was running and SubGit was not syncing, two
> revisions were committed: r71350 and r71351.
> Without expecting any problems, I resumed SubGit mirroring and then
> things started to get weird: Apparently, SubGit totally lost traction of
> the SVN repository and instead of syncing the two new revisions, it
> committed one in r71352 that _replaces_ trunk by r71349.
> This is one of the worst commits possible, as it forces SVN clients to
> redownload everything when updating their working copies.
> 
> Even more, it shows me that a clear separation of permissions for the
> SVN Server and SubGit isn't sufficient. SubGit can and actually did harm
> to our repository this way. This is exactly what I was warned of and
> what now happened...
> I'm sure everybody can understand that I cannot continue with the SubGit
> installation under these circumstances. We cannot put our repository at
> risk, especially now that we're in the mid of GSoC.
> This way, SubGit has definitely shown that it's not ready for our
> repository.
> I deeply apologize for this trouble and especially to the critics who
> remained right. And of course to Hermes, whose name SubGit abused for
> making the guilty commit.
> 
> Right now, I'm restoring the repository to r71351 using backups and
> "svnadmin dump". This effectively erases r71352 from SVN history.
> As r71352 was only online for 10 minutes, I believe it's better to get
> rid of that commit in SVN history rather than having everyone suffer
> from it.
> 
> Again, I deeply apologize!
> 
> 
> With best regards,
> 
> Colin
> 
> ___
> Ros-dev mailing list
> Ros-dev@reactos.org
> http://www.reactos.org/mailman/listinfo/ros-dev
> 


-- 
Pierre Schweitzer 
System & Network Administrator
Senior Kernel Developer
ReactOS Deutschland e.V.



smime.p7s
Description: Signature cryptographique S/MIME
___
Ros-dev mailing list
Ros-dev@reactos.org
http://www.reactos.org/mailman/listinfo/ros-dev

Re: [ros-dev] Pale Moon drops ReactOS support

[Timo Kreuzer]

Looks like I have to chime in here.

We have been discussing this before and I wonder that Alex is not 
heavily opposing the idea of randomly adding new exports to our user 
mode DLLs. It is a well known fact that applications check for existance 
of exports to decide how to behave, so ... not going to go over this again.


To addess this issue I suggested to implement a compatibility layer. And 
there was a detailed discussion about that on this mailing list. 
https://www.reactos.org/pipermail/ros-dev/2015-March/017216.html


Please take your time to read the whole thread again so we can avoid 
wasting time, talking about the same things again.


Timo

...

Did you read it? Ok, then we can continue ;-)

Some remaining questions:
1. Does our SxS / ActiveCtx work properly, and does it work with 
registry/appcompat db? Otherwise what do we need to make it work? Aleksey?
2. How do we organize the DLLs? I would suggest to unite some core DLLs 
into "ros-kernelbase.dll" or something and gdi32/user32 into 
ros-win32.dll and export all functions from those, using forwarder DLLs 
through SxS. At the same time we should reorganize the code, organizing 
it the way that MS API sets do it. And also move the stuff out of the 
"dlls" folder into "win32core" or something.
3. How do we handle ntdll? Can we use SxS for ntdll as well? Obviously 
we cannot load the original ntdll with it, but we can probably load 
versioned wrapper ntdlls and resolve imports to those.


Timo

Am 18.05.2016 um 17:01 schrieb Ged Murphy:

Okay, considering no other devs are chiming in on this, and Alex and I had a 
similar view, I'd like to propose that we move towards a more hybrid system and 
remove the rule that APIs that weren’t in 2k3 shouldn't be added.

Proposal:
   1) Add all user mode APIs directly into the code base, and do away with the 
'kernel32_vista' libraries we have.
   2) Start to add kernel APIs which aren’t reliant on a particular feature. 
Rtl APIs are the  obvious one as Alex highlighted, missing syscalls or enums 
for 'get/set info' APIs are another obvious one.
   3) Features from later versions of NT can be added (e.g. IO cancelation, 
UEFI, ASLR, etc). The supporting APIs should only be added when the underlying 
support is in place. (i.e. don't stub NT6 APIs)
   4) Continue to report as NT5.2/win2k3 for now. We can consider bumping 
usermode to report something newer at a later stage.
   5) In the short term add a simple compatibility mode which can provide a 
different OS version for whitelisted processes. The front end could be similar 
Win7's ' compatibility tab in the file properties, however it wouldn't use 
shimeng.dll, apphelp.dll, etc, it would just modify the versioning APIs.

IMO the above is a quick win to get newer apps running. Apps that check for OS 
version by looking for APIs should now be happy due to #1. Apps which call 
VerifyVersionInfo / GetVersion / etc can be tricked into thinking they're 
running on a later OS due to #5

David mentioned a compatibility shim would open the doors to NT6, but I don't 
think we should be using it for this purpose. A compatibility shim should 
behave more like the Windows one, that is it should provide 'hacks' to a more 
modern codebase to get older apps to work (e.g. 9x APIs, missing flags, bugs 
some apps rely on, etc).
I do think the compatibility shim is worthwhile technology to add, but I think 
we should use it as intended instead of trying to make it look into the future 
as well as the past. (btw, didn't Alex write a tool for dumping the shim db in 
Windows which shows all the 'hacks' MS apply??)

Comments please.

Ged.


-Original Message-
From: Ros-dev [mailto:ros-dev-boun...@reactos.org] On Behalf Of Alex Ionescu
Sent: 17 May 2016 19:05
To: ReactOS Development List 
Subject: Re: [ros-dev] Pale Moon drops ReactOS support

The project doesn't have to be hard-coded to NT5. For example, I am building a 
UEFI loader/bootmgr based on Windows 10, because 2003 doesn't boot on UEFI 
systems.

That being said, I don't see any good reason for us not to still mainly focus 
on 2003 for the kernel. The kernel is NOT what's preventing apps from working, 
or hardware from working. What's preventing that from working is:

1) Lacking user-mode APIs, and in some cases Rtl APIs (sure, implement Win 10 
ones!)
2) Lacking hardware support for things like UEFI (I'm working on it), AHCI (we 
have a student working on it), USB 3 (someone can implement this...but USB 2 
barely works), etc..etc..etc..

Find me a single device driver that *only* works on NT 6... Server
2003 is still a support MS OS, so by definition there's still drivers for it.
Best regards,
Alex Ionescu


On Tue, May 17, 2016 at 4:44 PM, Javier Agustìn Fernàndez Arroyo 
 wrote:

"companies and such support NT5?"

where "support NT5", i meant "drop support for NT5", sorry

On Tue, May 17, 2016 at 4:43 PM, Javier Agustìn Fernàndez Arroyo
 wrote:

And 

Re: [ros-dev] Pale Moon drops ReactOS support

[Alex Ionescu]

I don't believe I said 'let's add random exports to our DLLs'. In
fact, in your old thread, I was totally FOR your idea, I even wrote
that the only sane way of doing it is with the app compat work.

On Wed, May 18, 2016 at 9:27 PM, Timo Kreuzer  wrote:
> We have been discussing this before and I wonder that Alex is not heavily
> opposing the idea of randomly adding new exports to our user mode DLLs. It
> is a well known fact that applications check for existance of exports to
> decide how to behave, so ... not going to go over this again.
>
> To addess this issue I suggested to implement a compatibility layer. And
> there was a detailed discussion about that on this mailing list.
> https://www.reactos.org/pipermail/ros-dev/2015-March/017216.html
>
> Please take your time to read the whole thread again so we can avoid wasting
> time, talking about the same things again.
>
> Timo



Best regards,
Alex Ionescu

___
Ros-dev mailing list
Ros-dev@reactos.org
http://www.reactos.org/mailman/listinfo/ros-dev

Re: [ros-dev] Pale Moon drops ReactOS support

[Riccardo Paolo Bestetti]

Ah, good to know! Looks like I'm even more right :-).

It was about time.

Regards,
--- /Riccardo Paolo Bestetti/


Il 17/05/2016 22:57, Neal Gompa ha scritto:

On Tue, May 17, 2016 at 2:58 PM, Riccardo Paolo Bestetti
 wrote:

Windows Server 2003 is definitely supported by Microsoft (which doesn't mean
it is supported by others, and it mostly isn't), but it is almost dead, and
it would only do harm to deny that.

I had to jump in on this, because this is becoming a repeated fact
that isn't true. Microsoft killed Windows Server 2003 support last
July[1]. It is most certainly dead to Microsoft. Please stop
perpetuating the idea that Microsoft supports a variant of a
15-year-old (!!!) operating system.

[1]: https://www.microsoft.com/en-us/server-cloud/products/windows-server-2003/

___
Ros-dev mailing list
Ros-dev@reactos.org
http://www.reactos.org/mailman/listinfo/ros-dev