Poco on Android

[GoG]

Hi everybody,
I'd like to share with the community, my experience of porting Poco on the Android platform.
No, it's not a joke, since Google release its NDK, you can build C/C++ libs to be use with your Java app.
More precisely, it's a custom version of the NDK that allow you to build C++ code with exceptions, rtti and stl support.
You can find the version and more details at Dmitry Moskalchuk website.

So, as you may know, the Android platform is based on a Linux kernel (2.6.x) and Google version of the standard C libraries (Bionic).
This re-coded version is quite light compared to the standard glibc because of some removal. So, here come problems.
I already succeed to build Poco foundation as a static library with the NDK toolchain but I had to remove a few things :
('--' means removed sources)

* File_Unix::setSizeImpl() : temporary turn empty due to missing 'truncate()' function in 'unistd.h'
* -- NamedEvent, -- NamedMutex : missing 'sys/sem.h'
* -- RWLock : missing typdefs in 'sys/types.h' (like 'pthread_rwlock_t')
* No FPEnvironment (there's no floating point env on default ARM architectures)
* No SharedMemory : missing 'shm_open' and 'shm_unlink' in 'sys/mman.h)
* -- TextConverter, -- TextEncoding : missing RWLock
* -- TextIterator, ASCIIEncoding : missing TextConverter and TextEncoding
* -- Latin1Encoding, -- Latin9Encoding, -- StreamConverter : missing TextEncoding
* -- UnicodeConverter, -- UTF8Encoding, -- UTF16Encoding, -- UTF8String : missing TextConverter
* -- Glob : missing TextIterator
* Thread_POSIX::setStackSizeImpl : set size to 0 (as if PTHREAD_STACK_MIN not defined) because of missing PAGE_SIZE definition

I built Poco Foundation with flags '__arm' (default is little_endian) as NDK toolchain only build libs for ARM targets and as Android
phones are (almost) all using such architecture. I also set 'POCO_NO_FPENVIRONMENT', 'POCO_NO_SHAREDMEMORY' and define
'POCO_OS_FAMILY_UNIX'.
After all, I'd like to have some feedback on this. Does somebody can give me some piece of advice to improve this results ?
I'd like to hear about the Posix thread typedefs too. Even if Google remove a lot of original functionnalities, I hope I can
build RWLock and dependencies, by emulating (or something), theses (maybe like in this old Android project).
Thanks for your help.

[guenter]

Great news!

It looks like fixing RWLock is the most important thing. Apparently Bionic does not support pthread_rwlock_t. The easiest fix would be to add a new config #define POCO_NO_PTHREAD_RWLOCK. If this is defined, simply implement Poxo::RWLock using an ordinary pthread_mutex_t. This should not cause any issues except for performance. But since I don't expect Android apps to be heavily threaded, this should not be a problem.
For NamedEvent and NamedMutex: Based on a quick Google search, Android seems to support sem_open() (<semaphore.h>). NamedMutex_UNIX and NamedEvent_UNIX contains conditional code that either uses <sys/sem.h> (semget(), etc.) or <semaphore.h> (sem_open(), etc.). Simply adding Android to the check should fix that (e.g., Mac OS X and SunOS use <semaphore.h>). If not, add a POCO_NO_SEMAPHORES config macro and handle like POCO_NO_FPENVIRONMENT, etc.
Once you have Foundation working, porting the rest of the libraries should not be hard. XML is trivial (no OS specific stuff in there), Util should not be that hard, too (there's some Unix daemon stuff in there that may cause issues), and Net may have minor issues, depending on how they have implemented sockets in Bionic.

Anyway, when you think your port is ready, send me the patches and I will integrate it into the next release.

[GoG]

Hi,
and thanks for the help Guenter ! I applied your recommendations and it works just fine.
By adding an 'POCO_NO_PTHREAD_RWLOCK' flag, I was able to replace 'pthread_rwlock_t' by 'pthread_mutex_t' when needed.
I also add an '#if defined' condition for Android in NamedEvent and NamedMutex, in order to include <semaphore.h> instead of <sys/sem.h>.
When linking, I faced another issue about undefined 'timezone' in TimeZone_UNIX. So I had to add Android as a target with "no timezone global var" (like Apple and FreeBSD).
Finally, I still have two questions about the Foundation build :
- in 'File_unix::setSizeImpl()', can I replace unsupported 'truncate()' function by supported 'ftruncate' one ? How ?
- in 'Thread_POSIX::setStackSizeImpl', is this the best way to fix missing PAGE_SIZE macro :

Code: Select all

#if not defined PTHREAD_STACK_MIN || defined (__ANDROID)
   _pData->stackSize = 0;
#else
   if (size != 0)
   {
#if defined(__APPLE__)
      // we must round up to a multiple of the memory page size
      const int PAGE_SIZE = 4096;
      size = ((size + PAGE_SIZE - 1)/PAGE_SIZE)*PAGE_SIZE;
#endif
      if (size < PTHREAD_STACK_MIN)
         size = PTHREAD_STACK_MIN;
   }
   _pData->stackSize = size;
#endif


By the way, I also succeed to build PocoXml, just adding '-DHAVE_EXPAT_CONFIG_H' flag.
Thanks.

PS: I'll send you patches as soon as I can...

[GoG]

Hi,
I just remember that I forgot to post my changes to make Poco build on Android.
In fact, I waited for more informations about my lastest issues :

- in 'File_unix::setSizeImpl()', can I replace unsupported 'truncate()' function by supported 'ftruncate' one ? How ?
- in 'Thread_POSIX::setStackSizeImpl', is there a good way to fix missing PAGE_SIZE macro ?

For many weeks now, I use Poco in some of my personal Android apps and it works fine.
I'll be glad to share it with the community as soon as I can fix theses 2 minor issues
(I'm not very happy with the way I go through it).

BTW, a new Android NDK is out (r4). A custom version with full C++ support is already available :
http://www.crystax.net/android/ndk-r4.php
I successfully build Poco with this version too.

Thanks again for the help !

GoG

[guenter]

Hi,

regarding truncate()/ftruncate(): you can probably do something like:

Code: Select all

int fd = open(path.c_str(), O_WRONLY, 0);
int rc = ftruncate(fd, size);
close(fd);
if (rc) handleLastErrorImpl(_path);

Regarding PAGE_SIZE: This should only be needed on Mac OS X - it's used if __APPLE__ is defined during compilation, which usually means you're compiling for Mac OS X or i(Phone)OS. I guess you actually mean PTHREAD_STACK_MIN. Not sure how to handle this.

Looking forward to your patches!

[TcT]

I'm also currently trying to port Poco for the mentioned Android NDK R4.
I have made a set of Android.mk files for Foundation, Net and XML.

I attached a patch file for the 1.3.6p2 branch.
This is currently messy but it compiles.

Simply apply the patch and put the resulting folder into your apps JNI folder this will it will be picked up by the ndk-build command.

Your applications Android.mk should look something like this:

Code: Select all

MY_ROOT_PATH := $(call my-dir)
LOCAL_PATH := $(call my-dir)
POCO_ROOT_PATH := $(LOCAL_PATH)/poco-1.3.6p2

include $(call all-subdir-makefiles)

LOCAL_PATH := $(MY_ROOT_PATH)

include $(CLEAR_VARS)

LOCAL_C_INCLUDES := $(POCO_ROOT_PATH)/Foundation/include $(POCO_ROOT_PATH)/XML/include $(POCO_ROOT_PATH)/Net/include

LOCAL_CFLAGS := -Dlinux -DARM

LOCAL_LDLIBS := -lz

LOCAL_MODULE    := hello-jni
LOCAL_SRC_FILES := hello-jni.cpp

LOCAL_STATIC_LIBRARIES := PocoXML PocoFoundation PocoNet

include $(BUILD_SHARED_LIBRARY)


Be advised that all this is very early work and is just meant as a starting point. I'll post a patch when I'm further in using the code. And would welcome any comments and tips on this.

I removed the following foundation Classes: NamedEvent NamedMutex SharedMemory SharedLibrary
I have only commented out code for #ifdef ANDROID in RWLock to make it and dependent modules compile (obviously needs change).
I implemented truncate with ftruncate.

Currently this builds static libraries.
You can compile Poco on Android as shared modules by replacing $(BUILD_STATIC_LIBRARY) with $(BUILD_SHARED_LIBRARY) and defining a few dependencies.

[guenter]

Thank you for your work. May I suggest you and GoG join forces and work together on the Android port, so that we end up with a single set of patches. This will make it much easier to integrate the changes into our codebase.

[GoG]

Hi,
and thanks for the help.
I'm not sure I will have time to make a proper patch of my modifs
since I didn't find the time yet to make the final arrangements.
But every change I made are explained in my previous posts.
So if you need more details, let me know. I'll be glad to help you
if I can.
Keep you aware of any progress still.

GoG

[laoyao]

Hi,

regarding truncate()/ftruncate(): you can probably do something like:

Code: Select all

int fd = open(path.c_str(), O_WRONLY, 0);
int rc = ftruncate(fd, size);
close(fd);
if (rc) handleLastErrorImpl(_path);

Regarding PAGE_SIZE: This should only be needed on Mac OS X - it's used if __APPLE__ is defined during compilation, which usually means you're compiling for Mac OS X or i(Phone)OS. I guess you actually mean PTHREAD_STACK_MIN. Not sure how to handle this.

Looking forward to your patches!

hi,guenter

I am using poco in my android project, could you give me some advise or instructions about the poco compiling of android version?

Thanks a lot!

[GoG]

Hi everyone,
and sorry for my previous post I had to delete (my bad).

Finally, with Poco 1.4.0 and Android NDK r5 releases, I thought it was a good
time to make things right, as STL, exceptions and RTTI are now supported.
So I managed to build Foundation and XML using Google toolchain for Android platforms.

As NDK use its own makefiles, I wrote a 'Android.mk' file for each target.
Modified Poco files and android mk files can be found here :
http://guillaume.aujay.free.fr/android/ ... _1.4.0.zip

Default config is static build, and, as NDK only build static libs that
are needed by shared libs, you will have to create a JNI project that use
Poco static libs in order to test them (or turn them into shared libs).

I had to change few lines in 1.4.0 source code and add some compile flags
to make Poco "Android compliant".
For exemple, Android does not support some semaphore Posix features (eg. semget, ...).
As far as I understood, Android platform is designed to be very flexible on process killing.
Or app wild kill can lead to dead semaphores in Linux kernel, so Google team decided
to not support this features in NDK.
Another change is about "pthread_rw_lock" that Android does not support. I used new flag
to replace them with classical "pthread_mutex_lock".

I hope this can be useful to some, and one day, added to Poco official release.

Thanks.

GoG