Move embedded 3rd party libraries in the source tree

[jmansion]

I'd like to propose moving the embedded zlib, regex, and expat libraries within the source tree, so that zlib.h is NOT in Foundation/include/Poco.

Its not so important where it is, though I would recommend it should be something like:
Foundation/include/Poco/contrib/zlib/zlib.h
or
Foundation/include/Poco/contrib/zlib/Poco/zlib.h

The latter is ugly, but more likely too work with application code with just an include path change rather than code changes.

My reasoning is that at present there is no way that I can get Poco to use the pre-installed zlib (same for expat, regex).

The problem is that if a different include in Foundation/include/Poco wants to include zlib.h, then it will typically start searching in that directory and find the embedded one rather than the system header. And it it looks for Poco/zlib.h, then it will always find the embedded one too.

I'd like to be able to adjust the build so that the embedded libraries are not used and I can use more up-to-date versions from upstream without having to assume that Poco releases will import all the upstream fixes and enhancements, which so far as I can tell they don't as a matter of course.

Am I the only one that wants this?

[alex]

The idea is valid from the standpoint of using the most recent version of 3rd party code. However, there are few downnsides:

1) The configuration of libs included in Poco may not correspond to the one installed on the target system, which can lead to some headaches.

2) Versions of external libraries other than the ones officially included and tested with Poco may break builds and/or tests.

3) The above may cause unnecessary support requests if people are not able to build or things won't run properly.

I would personally have no problem with relocation of headers in order to make it easier to choose usage of pre-installed libraries for folks who know what they're doing., provided that by default, built-in headers and code are used.

[jmansion]

> The idea is valid from the standpoint of using the most recent version of 3rd party code. However, there are few downnsides:

I entirely agree with those points of course. But I think the benefits are there. Note that at present there are further issues relating to the global symbols in the embedded copies of the libraries, which will tend to clash with any attempt to use the upstream library.

If you must include a copy, then at least do so in such a way that if I access zlib through the Poco interface then I get Poco's embedded copy - but do leave me free to link and use the upstream library (or the one on my OS distribution).

I also feel that where we have ripoff inclusion of code (and I'm looking at adler32.c as an example) there should be indication in the file concerned:
- what version of the upstream was included originally
- what local changes have been made

Some of these files are really pretty ancient. I'm guessing that the PCRE is probably based on the pcre5.x release, but version 7 was released last November. But shouldn't have to guess, should I?

James

[alex]

> If you must include a copy, then at least do so in such a way that if I access zlib through the Poco interface then I get Poco's embedded copy - but do leave me free to link and use the upstream library (or the one on my OS distribution).
>
> I also feel that where we have ripoff inclusion of code (and I'm looking at adler32.c as an example) there should be indication in the file concerned:
> - what version of the upstream was included originally
> - what local changes have been made
>
> Some of these files are really pretty ancient. I'm guessing that the PCRE is probably based on the pcre5.x release, but version 7 was released last November. But shouldn't have to guess, should I?

James,

I think you are raising a legitimate point here. I would not mind it, but I did not feel an itch for this kind of stuff - clearly you have. I vote yes for some coherent strategy there along the lines you have described. I think that the most important issue that will pop up is the labor and maintenance - all the major contributors are pretty loaded with other stuff, so this has rather low priority (if it has any priority at all). If you'd be willing to put in some effort there, check with Guenter and see what he says.

Alex

[jmansion]

>If you'd be willing to put in some effort there, check with Guenter and see what he says.

I could certainly provide something. The trouble with this sort of move is that it looks like a mass delete and add when that's not really the story.

How about a script to run the appropriate SVN commands?

James

[guenter]

Can we wait with that move until 1.3 is out? We're at this time already fully occupied with other stuff (including the 1.3 release), and such a change would also mean that we'd have to change some of our internal scripts that deal with the POCO source tree.

[jmansion]

> Can we wait with that move until 1.3 is out? We're at this time already fully occupied with other stuff (including the 1.3 release), and such a change would also mean that we'd have to change some of our internal scripts that deal with the POCO source tree.

What form do you want for this? I have a fairly strong preference to do things on Win32 (because I'm lazy and haven't changed from file: svn to server based yet). It'll take a bit more to get the svn access from my *NIXVMs.

Having said that the script needn't be very smart and will be easy to port from Win32 to *NIX.

Is there a consensus for a target directory structure?

[guenter]

Having thought about this issue for some time now, I'd like to propose the following strategy:

- we leave all third-party source files used by POCO where they are now. This will keep the changes to the build system to an absolute minimum.
- since third party header files are included in only very few places, we change those #includes in the following way (in the following example, for zlib):

Code: Select all


...
#ifdef POCO_USE_EXTERNAL_ZLIB
#include
#else
#include "Poco/poco_zlib.h"
#endif
...


This implies that we have to rename the zlib.h in Foundation/include/Poco to poco_zlib.h.

In the Makefiles, we do the following:

Code: Select all


...
objects = ArchiveStrategy ASCIIEncoding ...

ifdef POCO_USE_EXTERNAL_ZLIB
objects += adler32 compress ...
endif
...


I can then add new options to the configure script that automatically defines POCO_USE_EXTERNAL_ZLIB for gmake and as a preprocessor macro.

Similarly, one could add a new configuration to the Visual Studio projects that exclude the zlib (etc.) sources from compilation, if required (although, I think most Windows users wouldn't want to do this anyway).

The only problem I see is with PCRE, since in POCO 1.3, we use some of the Unicode support in PCRE that's not part of the public interface - so it's probably hard to remove PCRE from POCO for now.

[jmansion]

> Having thought about this issue for some time now, I'd like to propose the following strategy:
>
> - we leave all third-party source files used by POCO where they are now. This will keep the changes to the build system to an absolute minimum.
> - since third party header files are included in only very few places, we change those #includes in the following way (in the following example, for zlib):

OK, that's an equivalent way to do it.

>

Code: Select all


> ...
> objects = ArchiveStrategy ASCIIEncoding ...
>
> ifdef POCO_USE_EXTERNAL_ZLIB
> objects += adler32 compress ...
> endif
> ...
>

I think you mean ifndef here, but that's a small point.

> The only problem I see is with PCRE, since in POCO 1.3, we use some of the Unicode support in PCRE that's not part of the public interface - so it's probably hard to remove PCRE from POCO for now.

Fair enough, but perhaps the module(s) in question could then still be included but provide decorated names that don't clash.

Why isn't this same policy applied to OpenSSL by the way?