subject:"Re\: 1.6 release timetable"

* Yann Ylavic:

>> It's been a bit of a struggle to get this right.
>
> I think the confusion comes from the term "directory stream", which
> people (at least me :p ) may read as underlying directory (i.e.
> filesystem's), though it's the term used to talk about the DIR* in the
> whole man page...

> I now realize that there is no real thread-safety issue, it indeed
> shouldn't happen to call readdir() concurrently on the same DIR, IMHO
> the doc should say that it's simply undefined behaviour (actually I
> thought it was the case with readdir_r too, so the readdir_r part is
> the misleading one I think).

Thanks for this feedback.

>> I think the Solaris manual also does not mark readdir as thread-safe,
>> implicitly suggesting to use readdir_r in multi-threaded programs.
>> But this suggestion isn't helpful on Solaris, either.
>
> So Solaris' readdir() does not return static data either.

Right.

> Any idea of which system keeps doing this?

No one who monitors the Austin Groups tracker for POSIX issues does
it, otherwise they would have spoken out against the readdir_r
deprecation.

Re: 1.6 release timetable

On Sat, Mar 25, 2017 at 1:34 PM, Florian Weimer  wrote:
> * Yann Ylavic:
>> "In the current POSIX.1 specification (POSIX.1-2008),
>> readdir() is not required to be thread-safe.  However, in modern
>> implementations (including the glibc implementation), concurrent calls
>> to readdir() that specify different directory streams are thread-safe.
>> In cases where multiple threads must read from the same directory
>> stream, using readdir() with external synchronization is still
>> preferable to the use of the  deprecated readdir_r(3) function.  It is
>> expected that a future version of POSIX.1 will require that readdir()
>> be thread-safe when concurrently employed on different directory
>> streams."
>
> It's been a bit of a struggle to get this right.

I think the confusion comes from the term "directory stream", which
people (at least me :p ) may read as underlying directory (i.e.
filesystem's), though it's the term used to talk about the DIR* in the
whole man page...

> People are really
> concerned about the case where multiple threads read from the same
> directory stream.  But how often does that happen in practice?

I now realize that there is no real thread-safety issue, it indeed
shouldn't happen to call readdir() concurrently on the same DIR, IMHO
the doc should say that it's simply undefined behaviour (actually I
thought it was the case with readdir_r too, so the readdir_r part is
the misleading one I think).

>>
>> Are you thinking of the above editorial?
>
> I meant this part:
>
>   Preliminary: | MT-Unsafe race:dirstream | AS-Unsafe lock | AC-Unsafe
> lock | See POSIX Safety Concepts.
>
> 
>
> “MT-Unsafe race:dirstream” doesn't make much sense because we don't
> have this as a category for memcpy because for some reason, it is
> “obvious” for memcpy that it's only thread-safe if it is called for
> completely separate arguments.

OK, got it now, thanks!

>
> I think the Solaris manual also does not mark readdir as thread-safe,
> implicitly suggesting to use readdir_r in multi-threaded programs.
> But this suggestion isn't helpful on Solaris, either.

So Solaris' readdir() does not return static data either.
Any idea of which system keeps doing this?

Re: 1.6 release timetable

* Yann Ylavic:

> [Resend to the whole list, sorry Florian for private message]
>
> On Sat, Mar 25, 2017 at 1:20 PM, Yann Ylavic  wrote:
>>
>> Right, modern readdir()s seem to be thread-safe but with regard to
>> different directories only, at least Linux' man page states:
>> "In the current POSIX.1 specification (POSIX.1-2008),
>> readdir() is not required to be thread-safe.  However, in modern
>> implementations (including the glibc implementation), concurrent calls
>> to readdir() that specify different directory streams are thread-safe.
>> In cases where multiple threads must read from the same directory
>> stream, using readdir() with external synchronization is still
>> preferable to the use of the  deprecated readdir_r(3) function.  It is
>> expected that a future version of POSIX.1 will require that readdir()
>> be thread-safe when concurrently employed on different directory
>> streams."
>
> Maybe we could simply forward this requirement (external
> synchronization) in the definition/usage of apr_dir_read()...

That's reasonable, but it's counterproductive to document that.  Most
programmers will assume that they have to perform special locking for
this function, which isn't true.  There really isn't anything special
about readdir and apr_dir_read.  It's mostly an accident that readdir
was labeled as thread-unsafe, which led to this mess, so please do not
repeat this for apr_dir_read.

Re: 1.6 release timetable

[Resend to the whole list, sorry Florian for private message]

On Sat, Mar 25, 2017 at 1:20 PM, Yann Ylavic  wrote:
>
> Right, modern readdir()s seem to be thread-safe but with regard to
> different directories only, at least Linux' man page states:
> "In the current POSIX.1 specification (POSIX.1-2008),
> readdir() is not required to be thread-safe.  However, in modern
> implementations (including the glibc implementation), concurrent calls
> to readdir() that specify different directory streams are thread-safe.
> In cases where multiple threads must read from the same directory
> stream, using readdir() with external synchronization is still
> preferable to the use of the  deprecated readdir_r(3) function.  It is
> expected that a future version of POSIX.1 will require that readdir()
> be thread-safe when concurrently employed on different directory
> streams."

Maybe we could simply forward this requirement (external
synchronization) in the definition/usage of apr_dir_read()...

Re: 1.6 release timetable

* Yann Ylavic:

>> readdir is thread-safe.  There used to be this idea that fdopendir
>> could be implemented like this:
>>
>> DIR *
>> fdopendir (int fd)
>> {
>>   return (DIR *) fd;
>> }
>>
>> And readdir would use a static buffer for the directory entry (like
>> gethostbyname) instead of something which is allocated as part of the
>> opaque the DIR * object (similar to a FILE *).  Such systems may
>> indeed have existed at one point, but I doubt that you can compile APR
>> on them.  It's unlikely that these systems will support readdir_r
>> because it is much to recent an interface.
>
> Right, modern readdir()s seem to be thread-safe but with regard to
> different directories only, at least Linux' man page states:
> "In the current POSIX.1 specification (POSIX.1-2008),
> readdir() is not required to be thread-safe.  However, in modern
> implementations (including the glibc implementation), concurrent calls
> to readdir() that specify different directory streams are thread-safe.
> In cases where multiple threads must read from the same directory
> stream, using readdir() with external synchronization is still
> preferable to the use of the  deprecated readdir_r(3) function.  It is
> expected that a future version of POSIX.1 will require that readdir()
> be thread-safe when concurrently employed on different directory
> streams."

It's been a bit of a struggle to get this right.  People are really
concerned about the case where multiple threads read from the same
directory stream.  But how often does that happen in practice?

>> For systems which use a buffer embedded in the DIR * object (and I'm
>> not aware of any which don't), readdir is as thread-safe as memcpy,
>> although some manual pages claim it is not.  This is very likely an
>> editorial mistake because thread safety guarantees for functions which
>> operate on separate objects are still an evolving concept.
>
> Are you thinking of the above editorial?

I meant this part:

  Preliminary: | MT-Unsafe race:dirstream | AS-Unsafe lock | AC-Unsafe
lock | See POSIX Safety Concepts.



“MT-Unsafe race:dirstream” doesn't make much sense because we don't
have this as a category for memcpy because for some reason, it is
“obvious” for memcpy that it's only thread-safe if it is called for
completely separate arguments.

I think the Solaris manual also does not mark readdir as thread-safe,
implicitly suggesting to use readdir_r in multi-threaded programs.
But this suggestion isn't helpful on Solaris, either.

>> Just stop using readdir_r.  I know that many people are invested in
>> that interface for various reasons, but sometimes, you just have to
>> delete pointless code and get on with it.
>
> I'm not sure we can use readdir() blindy though, what if multiple
> threads use it on the same dir?

Then the current implementation is already broken because apr_dir_read
does not perform any locking: The call to readdir_r can write to the
d_name buffer, and the reads of the d_name in apr_dir_read constitute
a data race.

If you mean “the same directory on the file system”: What counts is
the same DIR * object.  If two objects iterate through the same
directory, this does not matter because each DIR * object is required
to keep its own iteration position.

Re: 1.6 release timetable

On Sat, Mar 25, 2017 at 12:53 PM, Florian Weimer  wrote:
> * Yann Ylavic:
>
>> On Fri, Mar 24, 2017 at 7:35 PM, William A Rowe Jr
>>  wrote:
>>>
>>> I haven't built 1.x branch against openssl 110 yet,
>>
>> I did with 1.6.x (on latest Debian's libbsl-1.1) with no issue.
>>
>>> so here's some Unix (latest Fedora) feedback;
>>>
>>> ../../apr-1.6/file_io/unix/dir.c: In function ‘apr_dir_read’:
>>> ../../apr-1.6/file_io/unix/dir.c:162:5: warning: ‘readdir_r’ is
>>> deprecated [-Wdeprecated-declarations]
>>>  ret = readdir_r(thedir->dirstruct, thedir->entry, );
>>>  ^~~
>>> In file included from
>>> /home/wrowe/dev/apr-1.6/include/arch/unix/apr_arch_file_io.h:50:0,
>>>  from ../../apr-1.6/file_io/unix/dir.c:17:
>>> /usr/include/dirent.h:183:12: note: declared here
>>>  extern int readdir_r (DIR *__restrict __dirp,
>>> ^
>>
>> Same here, but this is really a weird situation where readdir_r() is
>> deprecated and "googling" advises are either to use readdir() with a
>> lock (because it's not thread-safe...),
>
> readdir is thread-safe.  There used to be this idea that fdopendir
> could be implemented like this:
>
> DIR *
> fdopendir (int fd)
> {
>   return (DIR *) fd;
> }
>
> And readdir would use a static buffer for the directory entry (like
> gethostbyname) instead of something which is allocated as part of the
> opaque the DIR * object (similar to a FILE *).  Such systems may
> indeed have existed at one point, but I doubt that you can compile APR
> on them.  It's unlikely that these systems will support readdir_r
> because it is much to recent an interface.

Right, modern readdir()s seem to be thread-safe but with regard to
different directories only, at least Linux' man page states:
"In the current POSIX.1 specification (POSIX.1-2008),
readdir() is not required to be thread-safe.  However, in modern
implementations (including the glibc implementation), concurrent calls
to readdir() that specify different directory streams are thread-safe.
In cases where multiple threads must read from the same directory
stream, using readdir() with external synchronization is still
preferable to the use of the  deprecated readdir_r(3) function.  It is
expected that a future version of POSIX.1 will require that readdir()
be thread-safe when concurrently employed on different directory
streams."

>
> For systems which use a buffer embedded in the DIR * object (and I'm
> not aware of any which don't), readdir is as thread-safe as memcpy,
> although some manual pages claim it is not.  This is very likely an
> editorial mistake because thread safety guarantees for functions which
> operate on separate objects are still an evolving concept.

Are you thinking of the above editorial?

>
>> or to still use readdir_r()
>> but malloc the passed in entry (with an overly complicated/hardly
>> portable thing to compute its size) so to avoid the defect (albeit not
>> the warning!) because of which readdir_r() is deprecated.
>
> It doesn't avoid all defects.  On most systems, computing the correct
> size of the readdir_r buffer is close to impossible (because of races
> which are very difficult to eliminate, or lack of correct name length
> information from the operating system), and readdir_r cannot check
> that the buffer is large enough because it is not passed the
> allocation size.

Agreed, if there were a solution with readdir_r() I guess the libc
would have implemented it...

>
> There is at least one implementation of readdir_r which will return
> ENAMETOOLONG if names are longer than NAME_MAX characters, while
> readdir will return such names, so using readdir_r will result in
> non-listable directories.
>
> Just stop using readdir_r.  I know that many people are invested in
> that interface for various reasons, but sometimes, you just have to
> delete pointless code and get on with it.

I'm not sure we can use readdir() blindy though, what if multiple
threads use it on the same dir?

Re: 1.6 release timetable

* Yann Ylavic:

> On Fri, Mar 24, 2017 at 7:35 PM, William A Rowe Jr
>  wrote:
>>
>> I haven't built 1.x branch against openssl 110 yet,
>
> I did with 1.6.x (on latest Debian's libbsl-1.1) with no issue.
>
>> so here's some Unix (latest Fedora) feedback;
>>
>> ../../apr-1.6/file_io/unix/dir.c: In function ‘apr_dir_read’:
>> ../../apr-1.6/file_io/unix/dir.c:162:5: warning: ‘readdir_r’ is
>> deprecated [-Wdeprecated-declarations]
>>  ret = readdir_r(thedir->dirstruct, thedir->entry, );
>>  ^~~
>> In file included from
>> /home/wrowe/dev/apr-1.6/include/arch/unix/apr_arch_file_io.h:50:0,
>>  from ../../apr-1.6/file_io/unix/dir.c:17:
>> /usr/include/dirent.h:183:12: note: declared here
>>  extern int readdir_r (DIR *__restrict __dirp,
>> ^
>
> Same here, but this is really a weird situation where readdir_r() is
> deprecated and "googling" advises are either to use readdir() with a
> lock (because it's not thread-safe...),

readdir is thread-safe.  There used to be this idea that fdopendir
could be implemented like this:

DIR *
fdopendir (int fd)
{
  return (DIR *) fd;
}

And readdir would use a static buffer for the directory entry (like
gethostbyname) instead of something which is allocated as part of the
opaque the DIR * object (similar to a FILE *).  Such systems may
indeed have existed at one point, but I doubt that you can compile APR
on them.  It's unlikely that these systems will support readdir_r
because it is much to recent an interface.

For systems which use a buffer embedded in the DIR * object (and I'm
not aware of any which don't), readdir is as thread-safe as memcpy,
although some manual pages claim it is not.  This is very likely an
editorial mistake because thread safety guarantees for functions which
operate on separate objects are still an evolving concept.

> or to still use readdir_r()
> but malloc the passed in entry (with an overly complicated/hardly
> portable thing to compute its size) so to avoid the defect (albeit not
> the warning!) because of which readdir_r() is deprecated.

It doesn't avoid all defects.  On most systems, computing the correct
size of the readdir_r buffer is close to impossible (because of races
which are very difficult to eliminate, or lack of correct name length
information from the operating system), and readdir_r cannot check
that the buffer is large enough because it is not passed the
allocation size.

There is at least one implementation of readdir_r which will return
ENAMETOOLONG if names are longer than NAME_MAX characters, while
readdir will return such names, so using readdir_r will result in
non-listable directories.

Just stop using readdir_r.  I know that many people are invested in
that interface for various reasons, but sometimes, you just have to
delete pointless code and get on with it.

Re: 1.6 release timetable

On Fri, Mar 24, 2017 at 7:35 PM, William A Rowe Jr  wrote:
>
> I haven't built 1.x branch against openssl 110 yet,

I did with 1.6.x (on latest Debian's libbsl-1.1) with no issue.

> so here's some Unix (latest Fedora) feedback;
>
> ../../apr-1.6/file_io/unix/dir.c: In function ‘apr_dir_read’:
> ../../apr-1.6/file_io/unix/dir.c:162:5: warning: ‘readdir_r’ is
> deprecated [-Wdeprecated-declarations]
>  ret = readdir_r(thedir->dirstruct, thedir->entry, );
>  ^~~
> In file included from
> /home/wrowe/dev/apr-1.6/include/arch/unix/apr_arch_file_io.h:50:0,
>  from ../../apr-1.6/file_io/unix/dir.c:17:
> /usr/include/dirent.h:183:12: note: declared here
>  extern int readdir_r (DIR *__restrict __dirp,
> ^

Same here, but this is really a weird situation where readdir_r() is
deprecated and "googling" advises are either to use readdir() with a
lock (because it's not thread-safe...), or to still use readdir_r()
but malloc the passed in entry (with an overly complicated/hardly
portable thing to compute its size) so to avoid the defect (albeit not
the warning!) because of which readdir_r() is deprecated.

No good solution is short (AIUI)...

Re: 1.6 release timetable

2017-03-24 Thread William A Rowe Jr

On Thu, Mar 23, 2017 at 1:01 PM, Nick Kew  wrote:
> On Thu, 2017-03-23 at 10:13 -0700, Gregg Smith wrote:
>
>> My retro build changes and Jan's revised patch have been commited.
>> Windows should be good to go.
>
> Brilliant, thanks both.

+1

> I'll take the activity on this issue as an indication that
> Windows was already *otherwise* good.  Unless we hear to
> the contrary!

I'll have more before month end, but no availability in the next few days
to review Windows. I haven't built 1.x branch against openssl 110 yet,
so here's some Unix (latest Fedora) feedback;

../../apr-1.6/file_io/unix/dir.c: In function ‘apr_dir_read’:
../../apr-1.6/file_io/unix/dir.c:162:5: warning: ‘readdir_r’ is
deprecated [-Wdeprecated-declarations]
 ret = readdir_r(thedir->dirstruct, thedir->entry, );
 ^~~
In file included from
/home/wrowe/dev/apr-1.6/include/arch/unix/apr_arch_file_io.h:50:0,
 from ../../apr-1.6/file_io/unix/dir.c:17:
/usr/include/dirent.h:183:12: note: declared here
 extern int readdir_r (DIR *__restrict __dirp,
^

../../apr-util-1.6/dbm/apr_dbm_ndbm.c: In function ‘vt_ndbm_open’:
../../apr-util-1.6/dbm/apr_dbm_ndbm.c:100:25: warning: passing
argument 1 of ‘dbm_open’ discards ‘const’ qualifier from pointer
target type [-Wdiscarded-qualifiers]
 file = dbm_open(pathname, dbmode, apr_posix_perms2mode(perm));
 ^~~~
In file included from ../../apr-util-1.6/dbm/apr_dbm_ndbm.c:29:0:
/usr/include/ndbm.h:55:14: note: expected ‘char *’ but argument is of
type ‘const char *’
 extern DBM  *dbm_open (char *file, int flags, int mode);
  ^~~~

And normal gummed up private vs public autogunk side-effects;

In file included from /usr/include/mysql/my_config.h:14:0,
 from /usr/include/mysql/my_global.h:75,
 from ../../apr-util-1.6/dbd/apr_dbd_mysql.c:38:
/usr/include/mysql/my_config-x86_64.h:649:0: warning: "PACKAGE_NAME" redefined
 #define PACKAGE_NAME "MySQL Server"

In file included from ../../apr-util-1.6/dbd/apr_dbd_mysql.c:22:0:
/home/wrowe/dev/build/aprutil16-ossl110/include/private/apu_config.h:175:0:
note: this is the location of the previous definition
 #define PACKAGE_NAME ""

In file included from /usr/include/mysql/my_config.h:14:0,
 from /usr/include/mysql/my_global.h:75,
 from ../../apr-util-1.6/dbd/apr_dbd_mysql.c:38:
/usr/include/mysql/my_config-x86_64.h:650:0: warning: "PACKAGE_STRING" redefined
 #define PACKAGE_STRING "MySQL Server 10.1.21"

In file included from ../../apr-util-1.6/dbd/apr_dbd_mysql.c:22:0:
/home/wrowe/dev/build/aprutil16-ossl110/include/private/apu_config.h:178:0:
note: this is the location of the previous definition
 #define PACKAGE_STRING ""

In file included from /usr/include/mysql/my_config.h:14:0,
 from /usr/include/mysql/my_global.h:75,
 from ../../apr-util-1.6/dbd/apr_dbd_mysql.c:38:
/usr/include/mysql/my_config-x86_64.h:651:0: warning:
"PACKAGE_TARNAME" redefined
 #define PACKAGE_TARNAME "mysql"

In file included from ../../apr-util-1.6/dbd/apr_dbd_mysql.c:22:0:
/home/wrowe/dev/build/aprutil16-ossl110/include/private/apu_config.h:181:0:
note: this is the location of the previous definition
 #define PACKAGE_TARNAME ""

In file included from /usr/include/mysql/my_config.h:14:0,
 from /usr/include/mysql/my_global.h:75,
 from ../../apr-util-1.6/dbd/apr_dbd_mysql.c:38:
/usr/include/mysql/my_config-x86_64.h:652:0: warning:
"PACKAGE_VERSION" redefined
 #define PACKAGE_VERSION "10.1.21"

In file included from ../../apr-util-1.6/dbd/apr_dbd_mysql.c:22:0:
/home/wrowe/dev/build/aprutil16-ossl110/include/private/apu_config.h:187:0:
note: this is the location of the previous definition
 #define PACKAGE_VERSION ""

Re: 1.6 release timetable

2017-03-23 Thread Nick Kew

On Thu, 2017-03-23 at 10:13 -0700, Gregg Smith wrote:

> My retro build changes and Jan's revised patch have been commited.
> Windows should be good to go.

Brilliant, thanks both.

I'll take the activity on this issue as an indication that
Windows was already *otherwise* good.  Unless we hear to
the contrary!

-- 
Nick Kew

Re: 1.6 release timetable

2017-03-23 Thread Gregg Smith



On 3/22/2017 8:30 AM, Nick Kew wrote:


Gregg, any thoughts on a timeframe for commit?  Obviously
nice-to-have,
though I'm also thinking if we have to release without a fix, we
can
at least use Jan's latest post in a release note.

--
Nick Kew



My retro build changes and Jan's revised patch have been commited.
Windows should be good to go.

Re: 1.6 release timetable - diffCMakeLists-jan.patch (1/1)

2017-03-23 Thread Jan Ehrhardt


begin 644 diffCMakeLists-jan.patch
M26YD97@Z($--86ME3=',N='AT#0H]/3T]/3T]/3T]/3T]/3T]/3T]/3T]
M/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]
M#0HM+2T@0TUA:V5,:7-T'0)*')E=FES:6]N(#$W.#2D-"D!`("TR,S@L-R`K,C,X
M+#<@0$`-"B!3150H:6YS=&%L;%]T87)G971S("1[:6YS=&%L;%]T87)G971S
M?2!L:6)A<')U=+3$I"B!3150H:6YS=&%L;%]B:6Y?<&1B("1[:6YS=&%L
M;%]B:6Y?<&1B?2`D>U!23TI%0U1?0DE.05)97T1)4GTO;!R=71I;"TQ
M+G!D8BD*(%1!4D=%5%],24Y+7TQ)0E)!4DE%4RAL:6)A<')U=+3$@)'M!
M4%)?3$E"4D%22453?2`D>UA-3$Q)0E],24)205))15-]*0HM4T547U1!4D=%
M5%]04D]015)42453*%P7!T;U]O
M<&5N7!T;U]O<&5N

Re: 1.6 release timetable - diffCMakeLists-jan.patch (0/1)

2017-03-23 Thread Jan Ehrhardt

Gregg Smith in gmane.comp.apache.apr.devel (Wed, 22 Mar 2017 10:20:54
-0700):
>I got a patch for cmake from Jan (attached) which ended up off list due 
>to the way "reply" works on this list.

The patch added -DAPR_DECLARE_EXPORT=1 -DAPU_DECLARE_EXPORT=1 twice to
apr_crypto_openssl. New patch attached.
-- 
Jan

Re: 1.6 release timetable

2017-03-22 Thread Gregg Smith


On 3/22/2017 8:30 AM, Nick Kew wrote:

[sorry, reposting by cut from this morning when mail was broken]
On Sun, 2017-03-19 at 20:44 -0700, Gregg Smith wrote:
On Sun, 2017-03-19 at 20:44 -0700, Gregg Smith wrote:

> I've been hacking on the .mak and .win files.
> Had to add a -ossl1 to apr/build/cvtdsp.pl for the dsp file,
no other
> way around it that I have been able to figure out.
>
> I have apu building at the command line, with either openssl
1.0.2 or
> 1.1.0. IIRC 1.0.2 is a LTS version and is going to be around
for awhile
> still.
>
> Have comitted nothing at this point as I want to test a IDE
build. Not
> that we have ever been able to build the crypto modules from
the IDE
> without modifying the build manually/scripted.

Gregg, any thoughts on a timeframe for commit?  Obviously
nice-to-have,
though I'm also thinking if we have to release without a fix, we
can
at least use Jan's latest post in a release note.

--
Nick Kew



I'll try to get the retro build stuff committed before bed tonight, 
tomorrow latest.


I got a patch for cmake from Jan (attached) which ended up off list due 
to the way "reply" works on this list.



Gregg
Index: CMakeLists.txt
===
--- CMakeLists.txt  (revision 1787649)
+++ CMakeLists.txt  (working copy)
@@ -238,7 +238,7 @@
 SET(install_targets ${install_targets} libaprutil-1)
 SET(install_bin_pdb ${install_bin_pdb} ${PROJECT_BINARY_DIR}/libaprutil-1.pdb)
 TARGET_LINK_LIBRARIES(libaprutil-1 ${APR_LIBRARIES} ${XMLLIB_LIBRARIES})
-SET_TARGET_PROPERTIES(libaprutil-1 PROPERTIES COMPILE_DEFINITIONS 
"APU_DECLARE_EXPORT;XML_STATIC;WINNT")
+SET_TARGET_PROPERTIES(libaprutil-1 PROPERTIES COMPILE_DEFINITIONS 
"APU_DECLARE_EXPORT;APR_DECLARE_EXPORT;XML_STATIC;WINNT")
 
 ADD_LIBRARY(aprutil-1 STATIC ${APR_SOURCES} ${APR_PUBLIC_HEADERS_GENERATED})
 SET(install_targets ${install_targets} aprutil-1)
@@ -254,7 +254,7 @@
   SET(install_bin_pdb ${install_bin_pdb} 
${PROJECT_BINARY_DIR}/apr_crypto_openssl-1.pdb)
   SET_TARGET_PROPERTIES(apr_crypto_openssl-1 PROPERTIES INCLUDE_DIRECTORIES 
"${APR_INCLUDE_DIRECTORIES};${OPENSSL_INCLUDE_DIR}")
   SET_TARGET_PROPERTIES(apr_crypto_openssl-1 PROPERTIES COMPILE_DEFINITIONS 
"WINNT")
-  SET_TARGET_PROPERTIES(apr_crypto_openssl-1 PROPERTIES COMPILE_FLAGS 
"-DDLL_NAME=apr_crypto_openssl")
+  SET_TARGET_PROPERTIES(apr_crypto_openssl-1 PROPERTIES COMPILE_FLAGS 
"-DAPR_DECLARE_EXPORT=1 -DAPU_DECLARE_EXPORT=1 -DAPR_DECLARE_EXPORT=1 
-DAPU_DECLARE_EXPORT=1 -DDLL_NAME=apr_crypto_openssl")
   TARGET_LINK_LIBRARIES(apr_crypto_openssl-1 libaprutil-1 ${APR_LIBRARIES} 
${OPENSSL_LIBRARIES})
 ENDIF()
 
@@ -265,8 +265,8 @@
   SET(dbd_drivers ${dbd_drivers} odbc)
   TARGET_LINK_LIBRARIES(apr_dbd_odbc-1 libaprutil-1 ${APR_LIBRARIES} odbc32 
odbccp32)
   SET_PROPERTY(TARGET apr_dbd_odbc-1 APPEND PROPERTY LINK_FLAGS 
/export:apr_dbd_odbc_driver)
-  SET_TARGET_PROPERTIES(apr_dbd_odbc-1 PROPERTIES COMPILE_DEFINITIONS 
"APU_HAVE_ODBC;HAVE_SQL_H;APU_DECLARE_EXPORT;APU_DSO_MODULE_BUILD;WINNT")
-  SET_TARGET_PROPERTIES(apr_dbd_odbc-1 PROPERTIES COMPILE_FLAGS 
"-DDLL_NAME=apr_dbd_odbc")
+  SET_TARGET_PROPERTIES(apr_dbd_odbc-1 PROPERTIES COMPILE_DEFINITIONS 
"APU_HAVE_ODBC;HAVE_SQL_H;APU_DECLARE_EXPORT;APR_DECLARE_EXPORT;APU_DSO_MODULE_BUILD;WINNT")
+  SET_TARGET_PROPERTIES(apr_dbd_odbc-1 PROPERTIES COMPILE_FLAGS 
"-DAPR_DECLARE_EXPORT=1 -DAPU_DECLARE_EXPORT=1 -DDLL_NAME=apr_dbd_odbc")
 ENDIF()
 
 IF(APR_HAS_LDAP)
@@ -276,7 +276,7 @@
   SET(install_bin_pdb ${install_bin_pdb} ${PROJECT_BINARY_DIR}/apr_ldap-1.pdb)
   TARGET_LINK_LIBRARIES(apr_ldap-1 libaprutil-1 ${APR_LIBRARIES} 
${LDAP_LIBRARIES})
   SET_TARGET_PROPERTIES(apr_ldap-1 PROPERTIES COMPILE_DEFINITIONS "WINNT")
-  SET_TARGET_PROPERTIES(apr_ldap-1 PROPERTIES COMPILE_FLAGS 
"-DDLL_NAME=apr_ldap")
+  SET_TARGET_PROPERTIES(apr_ldap-1 PROPERTIES COMPILE_FLAGS 
"-DAPR_DECLARE_EXPORT=1 -DAPU_DECLARE_EXPORT=1 -DDLL_NAME=apr_ldap")
   SET(apr_ldap_libraries apr_ldap-1)
 ELSE()
   SET(apr_ldap_libraries)

Re: 1.6 release timetable

2017-03-22 Thread Nick Kew

[sorry, reposting by cut from this morning when mail was broken]
On Sun, 2017-03-19 at 20:44 -0700, Gregg Smith wrote:
On Sun, 2017-03-19 at 20:44 -0700, Gregg Smith wrote:

> I've been hacking on the .mak and .win files.
> Had to add a -ossl1 to apr/build/cvtdsp.pl for the dsp file,
no other 
> way around it that I have been able to figure out.
> 
> I have apu building at the command line, with either openssl
1.0.2 or 
> 1.1.0. IIRC 1.0.2 is a LTS version and is going to be around
for awhile 
> still.
> 
> Have comitted nothing at this point as I want to test a IDE
build. Not 
> that we have ever been able to build the crypto modules from
the IDE 
> without modifying the build manually/scripted.

Gregg, any thoughts on a timeframe for commit?  Obviously
nice-to-have,
though I'm also thinking if we have to release without a fix, we
can
at least use Jan's latest post in a release note.

-- 
Nick Kew

Re: 1.6 release timetable

2017-03-22 Thread Nick Kew

[sorry, reposting by cut from this morning when mail was broken]
On Mon, 2017-03-20 at 03:20 +0100, Jan Ehrhardt wrote:

> Jan Ehrhardt in gmane.comp.apache.apr.devel (Sun, 19 Mar 2017
20:21:47 +0100):
> >>So, the answer to the question "does it work for you?" seems
to be No.

I guess that means the platform doesn't benefit from a toolchain
anything like the *X configuration scripts that work flexibly
with whatever is available?

(Sorry, it's about 15 years since I touched any Windows
platform,
beyond trivial rescue when someone insists I'm the computer
man).

> 
> CMake almost works. I had the following dirs

> Steps for compiling the VC14 x64 libs and dll's:

Thanks for the very clear description.  I guess that will make a
release note if we have to release without anything better.

> I did not run any tests, but it is a start.

crypto appears to be a gap in the tests, anyway!

-- 
Nick Kew

Re: 1.6 release timetable

2017-03-19 Thread Gregg Smith

Arg, this list bites me again. Sorry for the double Jan.

On 3/19/2017 11:44 AM, Jan Ehrhardt wrote:

Nick Kew in gmane.comp.apache.apr.devel (Sat, 18 Mar 2017 07:57:37
+):

On Sat, 2017-03-18 at 08:28 +0100, Jan Ehrhardt wrote:

Will this 1.6 release contain support of OpenSSL 1.1.x or is that part
of a future release?

Good question. I see no bugzilla entry for it. A quick search
on this list reveals just two messages, dated 29/3/16.
Are you asking whether the changes Rainer refers to in
http://mail-archives.apache.org/mod_mbox/apr-dev/201603.mbox/%3C56FA52EC.8050301%40kippdata.de%3E
are in it? If so, your own feedback to Rainer's "does it
work for you?" question would be great?

The 2 Windows related files in apr/apr-util/crypto still use the pre-1.1
OpenSSL lib files, libeay32.lib and ssleay32.lib:
http://svn.apache.org/viewvc/apr/apr-util/branches/1.6.x/crypto/apr_crypto_openssl.dsp?view=markup#l121
http://svn.apache.org/viewvc/apr/apr-util/branches/1.6.x/crypto/apr_crypto_openssl.mak?view=markup#l108

So, the answer to the question "does it work for you?" seems to be No.

I've been hacking on the .mak and .win files.
Had to add a -ossl1 to apr/build/cvtdsp.pl for the dsp file, no other
way around it that I have been able to figure out.

I have apu building at the command line, with either openssl 1.0.2 or
1.1.0. IIRC 1.0.2 is a LTS version and is going to be around for awhile
still.

Have comitted nothing at this point as I want to test a IDE build. Not
that we have ever been able to build the crypto modules from the IDE
without modifying the build manually/scripted.

Re: 1.6 release timetable

Jan Ehrhardt in gmane.comp.apache.apr.devel (Mon, 20 Mar 2017 03:20:36 +0100):
>. compile apr-util, with the CMake args on 1 line:
snip
>call msbuild APR.sln /p:Configuration=RelWithDebInfo /p:Platform=x64
Copy-paste eror. For apr-util this has to be

call msbuild APR-Util.sln /p:Configuration=RelWithDebInfo /p:Platform=x64
-- 
Jan

Re: 1.6 release timetable

Jan Ehrhardt in gmane.comp.apache.apr.devel (Sun, 19 Mar 2017 20:21:47 +0100):
>>So, the answer to the question "does it work for you?" seems to be No.
>
>Or Yes, if CMake works. I will try that later.

CMake almost works. I had the following dirs
- apr
  - apr
  - apr-util
- xml
  - openssl

For apr and apr-util I used the 1.6.x branches:
http://svn.apache.org/viewvc/apr/apr/branches/1.6.x/
http://svn.apache.org/viewvc/apr/apr-util/branches/1.6.x/

Steps for compiling the VC14 x64 libs and dll's:
. compile OpenSSL (version 1.1.0e).
. copy the XML headers to \apr\apr-util\include
. install \apr\apr\include\apr.h manually (cf apr 1.4/OpenSSL 1.0.2)
. compile apr, with the CMake args on 1 line:

set INCLUDE=\apr\openssl\include;%INCLUDE%
set LIB=\apr\openssl\;%LIB%
set PATH=\apr\openssl\apps;%PATH%
cd \apr\apr
if exist \apr\apr\vc14.x64 rd \apr\apr\vc14.x64
md \apr\apr\vc14.x64
cd \apr\apr\vc14.x64
cmake -G "Visual Studio 14 2015 Win64" -D CMAKE_BUILD_TYPE=RelWithDebInfo \
-D APR_BUILD_TESTAPR=1 -D CMAKE_INSTALL_PREFIX=\apr\install \
-D INSTALL_PDB=OFF .. 
pause
call msbuild APR.sln /p:Configuration=RelWithDebInfo /p:Platform=x64
call msbuild INSTALL.vcxproj /p:Configuration=RelWithDebInfo /p:Platform=x64

. compile apr-util, with the CMake args on 1 line:

set INCLUDE=\apr\openssl\include;%INCLUDE%
set LIB=\apr\openssl\;%LIB%
set PATH=\apr\openssl\apps;%PATH%
cd \apr\apr-util
if exist \apr\apr-util\vc14.x64 rd \apr\apr-util\vc14.x64
md \apr\apr-util\vc14.x64
cd \apr\apr-util\vc14.x64
cmake -G "Visual Studio 14 2015 Win64" \
-D OPENSSL_ROOT_DIR=\apr\openssl -D 
OPENSSL_INCLUDE_DIR=\apr\openssl\include \
-D APR_INCLUDE_DIR=\apr\apr\include -D 
APR_LIBRARIES=\apr\apr\vc14.x64\RelWithDebInfo\libapr-1.lib \
-D APU_HAVE_CRYPTO=1 -D APR_BUILD_TESTAPR=1 -D 
CMAKE_INSTALL_PREFIX=\apr\install -D INSTALL_PDB=OFF ..
pause
perl -pi.bak -e 
"s/;APU_DECLARE_EXPORT;/;APR_DECLARE_EXPORT;APU_DECLARE_EXPORT;/gi" 
libaprutil-1.vcxproj
for %%f in (*.vcxproj) do perl -pi.bak -e 
"s/;DLL_NAME=/;APR_DECLARE_EXPORT;APU_DECLARE_EXPORT;DLL_NAME=/gi" %%f
call msbuild APR.sln /p:Configuration=RelWithDebInfo /p:Platform=x64
call msbuild INSTALL.vcxproj /p:Configuration=RelWithDebInfo /p:Platform=x64

The two perl replacements are need to avoid errors that dllimport
is not allowed. They can be patched into the apr-util CMakeLists.txt

Result (zipped from \apr\install):
https://phpdev.toolsforresearch.com/apr-1.6.x.zip

I did not run any tests, but it is a start.
-- 
Jan

Re: 1.6 release timetable

Jan Ehrhardt in gmane.comp.apache.apr.devel (Sun, 19 Mar 2017 19:44:01
+0100):
>The 2 Windows related files in apr/apr-util/crypto still use the pre-1.1
>OpenSSL lib files, libeay32.lib and ssleay32.lib:
>http://svn.apache.org/viewvc/apr/apr-util/branches/1.6.x/crypto/apr_crypto_openssl.dsp?view=markup#l121
>http://svn.apache.org/viewvc/apr/apr-util/branches/1.6.x/crypto/apr_crypto_openssl.mak?view=markup#l108
>
>So, the answer to the question "does it work for you?" seems to be No.

Or Yes, if CMake works. I will try that later.
-- 
Jan

Re: 1.6 release timetable