Building dynamic libraries with undefined references.

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Building dynamic libraries with undefined references.

Raj Shekar

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Building dynamic libraries with undefined references.

J Decker


On Thu, Aug 1, 2019 at 5:15 AM Raj Shekar <[hidden email]> wrote:

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.
If you're using dlopen to load the library, then you should use dlsym to get the function from that library and just declare a `void (*test_func)(...)` which is a function pointer to set from the dlsym then just call test_func();
then it doesn't need to exist ever.
Otherwise you'll need a dummy library of some name, but even then, when it links to the test_func in THAT library, it would refer to that library's name, and not nesscarily be obscured by loading other libraries also containing that symbol.
 

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Building dynamic libraries with undefined references.

Raj Shekar
Thanks for your time J Decker,  Instead of using the function pointers won't it be possible to link the library with the functions which are exposed by already loaded libraries. I would like to give a try by disabling the --no-undefined flag from the linker options. 

With verbose logging of cmake, how can we disable the - - no-undefined flag from the linker flags, it seems to get added by default. 

LANGUAGE_COMPILE_FLAGS = -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++ -std=c++11 -frtti -fexceptions -O0 -fno-limit-debug-info
LINK_FLAGS = -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-allow-shlib-undefined -Wl,-undefined


Thanks, 
Raj. 

On Thu, Aug 1, 2019, 21:14 J Decker <[hidden email]> wrote:


On Thu, Aug 1, 2019 at 5:15 AM Raj Shekar <[hidden email]> wrote:

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.
If you're using dlopen to load the library, then you should use dlsym to get the function from that library and just declare a `void (*test_func)(...)` which is a function pointer to set from the dlsym then just call test_func();
then it doesn't need to exist ever.
Otherwise you'll need a dummy library of some name, but even then, when it links to the test_func in THAT library, it would refer to that library's name, and not nesscarily be obscured by loading other libraries also containing that symbol.
 

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Building dynamic libraries with undefined references.

J Decker


On Fri, Aug 2, 2019 at 2:37 AM Raj Shekar <[hidden email]> wrote:
Thanks for your time J Decker,  Instead of using the function pointers won't it be possible to link the library with the functions which are exposed by already loaded libraries. I would like to give a try by disabling the --no-undefined flag from the linker options. 

With verbose logging of cmake, how can we disable the - - no-undefined flag from the linker flags, it seems to get added by default. 

LANGUAGE_COMPILE_FLAGS = -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++ -std=c++11 -frtti -fexceptions -O0 -fno-limit-debug-info
LINK_FLAGS = -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-allow-shlib-undefined -Wl,-undefined

Relocations are rarely done after a library is loaded.
If the library to use the undefined function doesn't previously resolve the function, it won'tload.  If it does, then loading a new one won't replace the old relocation.
A function pointer for most purposes is the same as the name of a function... inporting a function or function pointer in Java looks the same as any other function...


Thanks, 
Raj. 

On Thu, Aug 1, 2019, 21:14 J Decker <[hidden email]> wrote:


On Thu, Aug 1, 2019 at 5:15 AM Raj Shekar <[hidden email]> wrote:

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.
If you're using dlopen to load the library, then you should use dlsym to get the function from that library and just declare a `void (*test_func)(...)` which is a function pointer to set from the dlsym then just call test_func();
then it doesn't need to exist ever.
Otherwise you'll need a dummy library of some name, but even then, when it links to the test_func in THAT library, it would refer to that library's name, and not nesscarily be obscured by loading other libraries also containing that symbol.
 

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Building dynamic libraries with undefined references.

Raj Shekar

 Thanks for your time and inputs J Decker, and due to some unavoidable circumstances I couldn't reply earlier. Now, I'm using the dlopen to load the libraries at run time and link the function pointers using dlsym (instead of relying on undefined references). Got to know that the best way to load the libraries is to use System.loadLibrary("lib.so") calls from Java side. But I want to explicitly do it from C side using dlopen( ) calls only. So, as of now the libraries are placed at '/data/data/<package>/libs' and I'm using this path, but the problem is that the path is undocumented. As per the answer in (https://stackoverflow.com/questions/15719149/how-to-integrate-native-runtime-library-with-dlopen-on-ndk), the best way to obtain the path to the libraries is to use '/proc' and 'derive from that the path to other libs' - I couldn't find more details on how to use '/proc' to obtain the path to the libraries. Can anyone please point me to a tutorial or an example from which I can refer.

  

   Another question: If we have an update to a dynamic library, during my program start-up, can I copy the updated library from another folder to my project libraries folder ('/data/data/<package>/libs' or the one obtained with the help of '/proc'). We would like to do this to avoid giving an updated APK when ever we would have an update to one of our dynamic libraries ? Is it recommended not to use such methods or is it valid on Android platform.. 

 

Thanks,

Raj.


On Fri, Aug 2, 2019, 17:42 J Decker <[hidden email]> wrote:


On Fri, Aug 2, 2019 at 2:37 AM Raj Shekar <[hidden email]> wrote:
Thanks for your time J Decker,  Instead of using the function pointers won't it be possible to link the library with the functions which are exposed by already loaded libraries. I would like to give a try by disabling the --no-undefined flag from the linker options. 

With verbose logging of cmake, how can we disable the - - no-undefined flag from the linker flags, it seems to get added by default. 

LANGUAGE_COMPILE_FLAGS = -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++ -std=c++11 -frtti -fexceptions -O0 -fno-limit-debug-info
LINK_FLAGS = -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-allow-shlib-undefined -Wl,-undefined

Relocations are rarely done after a library is loaded.
If the library to use the undefined function doesn't previously resolve the function, it won'tload.  If it does, then loading a new one won't replace the old relocation.
A function pointer for most purposes is the same as the name of a function... inporting a function or function pointer in Java looks the same as any other function...


Thanks, 
Raj. 

On Thu, Aug 1, 2019, 21:14 J Decker <[hidden email]> wrote:


On Thu, Aug 1, 2019 at 5:15 AM Raj Shekar <[hidden email]> wrote:

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.
If you're using dlopen to load the library, then you should use dlsym to get the function from that library and just declare a `void (*test_func)(...)` which is a function pointer to set from the dlsym then just call test_func();
then it doesn't need to exist ever.
Otherwise you'll need a dummy library of some name, but even then, when it links to the test_func in THAT library, it would refer to that library's name, and not nesscarily be obscured by loading other libraries also containing that symbol.
 

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7NjnSa%3DrU04mBqK_%3D9xCZfrWj0gheg1EtOv0mPbNL-S2Q%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Building dynamic libraries with undefined references.

Dan Albert
That path isn't documented because it's not guaranteed. On newer versions of the OS your app is installed to a semi-random location. Anything hard coded will be wrong.

But that doesn't matter. You don't need to give dlopen the full path. Just load the library by name (the file name without the path) and it will be found.

On Tue, Aug 13, 2019, 23:51 Raj Shekar <[hidden email]> wrote:

 Thanks for your time and inputs J Decker, and due to some unavoidable circumstances I couldn't reply earlier. Now, I'm using the dlopen to load the libraries at run time and link the function pointers using dlsym (instead of relying on undefined references). Got to know that the best way to load the libraries is to use System.loadLibrary("lib.so") calls from Java side. But I want to explicitly do it from C side using dlopen( ) calls only. So, as of now the libraries are placed at '/data/data/<package>/libs' and I'm using this path, but the problem is that the path is undocumented. As per the answer in (https://stackoverflow.com/questions/15719149/how-to-integrate-native-runtime-library-with-dlopen-on-ndk), the best way to obtain the path to the libraries is to use '/proc' and 'derive from that the path to other libs' - I couldn't find more details on how to use '/proc' to obtain the path to the libraries. Can anyone please point me to a tutorial or an example from which I can refer.

  

   Another question: If we have an update to a dynamic library, during my program start-up, can I copy the updated library from another folder to my project libraries folder ('/data/data/<package>/libs' or the one obtained with the help of '/proc'). We would like to do this to avoid giving an updated APK when ever we would have an update to one of our dynamic libraries ? Is it recommended not to use such methods or is it valid on Android platform.. 

 

Thanks,

Raj.


On Fri, Aug 2, 2019, 17:42 J Decker <[hidden email]> wrote:


On Fri, Aug 2, 2019 at 2:37 AM Raj Shekar <[hidden email]> wrote:
Thanks for your time J Decker,  Instead of using the function pointers won't it be possible to link the library with the functions which are exposed by already loaded libraries. I would like to give a try by disabling the --no-undefined flag from the linker options. 

With verbose logging of cmake, how can we disable the - - no-undefined flag from the linker flags, it seems to get added by default. 

LANGUAGE_COMPILE_FLAGS = -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++ -std=c++11 -frtti -fexceptions -O0 -fno-limit-debug-info
LINK_FLAGS = -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-allow-shlib-undefined -Wl,-undefined

Relocations are rarely done after a library is loaded.
If the library to use the undefined function doesn't previously resolve the function, it won'tload.  If it does, then loading a new one won't replace the old relocation.
A function pointer for most purposes is the same as the name of a function... inporting a function or function pointer in Java looks the same as any other function...


Thanks, 
Raj. 

On Thu, Aug 1, 2019, 21:14 J Decker <[hidden email]> wrote:


On Thu, Aug 1, 2019 at 5:15 AM Raj Shekar <[hidden email]> wrote:

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.
If you're using dlopen to load the library, then you should use dlsym to get the function from that library and just declare a `void (*test_func)(...)` which is a function pointer to set from the dlsym then just call test_func();
then it doesn't need to exist ever.
Otherwise you'll need a dummy library of some name, but even then, when it links to the test_func in THAT library, it would refer to that library's name, and not nesscarily be obscured by loading other libraries also containing that symbol.
 

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7NjnSa%3DrU04mBqK_%3D9xCZfrWj0gheg1EtOv0mPbNL-S2Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFVaGhvrdAvFiGhYrvjoM%2BFa1Vyn6%3DNvZq8EgkPzahpbGvpQvQ%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Building dynamic libraries with undefined references.

Alex Cohn
In reply to this post by Raj Shekar
The /data/data/<your.package>/lib path is not only undocumented, it's not valid anymore. The path to installed native libs depend on the version of Android, and the most recent versions randomize this path for further protection. The supported way to find location of the native libraries is through Java API ApplicationInfo.nativeLibDir, but even this has some caveats, see htps://stackoverflow.com/a/56551499/192373.

BR,
Alex Cohn

On Wednesday, August 14, 2019 at 9:51:35 AM UTC+3, Raj wrote:

 Thanks for your time and inputs J Decker, and due to some unavoidable circumstances I couldn't reply earlier. Now, I'm using the dlopen to load the libraries at run time and link the function pointers using dlsym (instead of relying on undefined references). Got to know that the best way to load the libraries is to use System.loadLibrary("lib.so") calls from Java side. But I want to explicitly do it from C side using dlopen( ) calls only. So, as of now the libraries are placed at '/data/data/<package>/libs' and I'm using this path, but the problem is that the path is undocumented. As per the answer in (<a href="https://www.google.com/url?q=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F15719149%2Fhow-to-integrate-native-runtime-library-with-dlopen-on-ndk&amp;sa=D&amp;sntz=1&amp;usg=AFQjCNG48wlZwO81AO3gLbfFEzylpfPWRQ" style="color:rgb(66,133,244)" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fstackoverflow.com%2Fquestions%2F15719149%2Fhow-to-integrate-native-runtime-library-with-dlopen-on-ndk\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG48wlZwO81AO3gLbfFEzylpfPWRQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fstackoverflow.com%2Fquestions%2F15719149%2Fhow-to-integrate-native-runtime-library-with-dlopen-on-ndk\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG48wlZwO81AO3gLbfFEzylpfPWRQ&#39;;return true;">https://stackoverflow.com/questions/15719149/how-to-integrate-native-runtime-library-with-dlopen-on-ndk), the best way to obtain the path to the libraries is to use '/proc' and 'derive from that the path to other libs' - I couldn't find more details on how to use '/proc' to obtain the path to the libraries. Can anyone please point me to a tutorial or an example from which I can refer.

  

   Another question: If we have an update to a dynamic library, during my program start-up, can I copy the updated library from another folder to my project libraries folder ('/data/data/<package>/libs' or the one obtained with the help of '/proc'). We would like to do this to avoid giving an updated APK when ever we would have an update to one of our dynamic libraries ? Is it recommended not to use such methods or is it valid on Android platform.. 

 

Thanks,

Raj.


On Fri, Aug 2, 2019, 17:42 J Decker <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="K9ZQLghXAgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">d3c...@...> wrote:


On Fri, Aug 2, 2019 at 2:37 AM Raj Shekar <<a href="javascript:" rel="nofollow" target="_blank" gdf-obfuscated-mailto="K9ZQLghXAgAJ" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rajash...@...> wrote:
Thanks for your time J Decker,  Instead of using the function pointers won't it be possible to link the library with the functions which are exposed by already loaded libraries. I would like to give a try by disabling the --no-undefined flag from the linker options. 

With verbose logging of cmake, how can we disable the - - no-undefined flag from the linker flags, it seems to get added by default. 

LANGUAGE_COMPILE_FLAGS = -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++ -std=c++11 -frtti -fexceptions -O0 -fno-limit-debug-info
LINK_FLAGS = -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-allow-shlib-undefined -Wl,-undefined

Relocations are rarely done after a library is loaded.
If the library to use the undefined function doesn't previously resolve the function, it won'tload.  If it does, then loading a new one won't replace the old relocation.
A function pointer for most purposes is the same as the name of a function... inporting a function or function pointer in Java looks the same as any other function...


Thanks, 
Raj. 

On Thu, Aug 1, 2019, 21:14 J Decker <<a href="javascript:" rel="nofollow" target="_blank" gdf-obfuscated-mailto="K9ZQLghXAgAJ" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">d3c...@...> wrote:


On Thu, Aug 1, 2019 at 5:15 AM Raj Shekar <<a href="javascript:" rel="nofollow" target="_blank" gdf-obfuscated-mailto="K9ZQLghXAgAJ" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rajash...@...> wrote:

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.
If you're using dlopen to load the library, then you should use dlsym to get the function from that library and just declare a `void (*test_func)(...)` which is a function pointer to set from the dlsym then just call test_func();
then it doesn't need to exist ever.
Otherwise you'll need a dummy library of some name, but even then, when it links to the test_func in THAT library, it would refer to that library's name, and not nesscarily be obscured by loading other libraries also containing that symbol.
 

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" rel="nofollow" target="_blank" gdf-obfuscated-mailto="K9ZQLghXAgAJ" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">andro...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" rel="nofollow" target="_blank" gdf-obfuscated-mailto="K9ZQLghXAgAJ" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">andro...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" rel="nofollow" target="_blank" gdf-obfuscated-mailto="K9ZQLghXAgAJ" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">andro...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" rel="nofollow" target="_blank" gdf-obfuscated-mailto="K9ZQLghXAgAJ" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">andro...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/4ee4455f-7ce3-45a9-bf3c-5b49cac03e71%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Building dynamic libraries with undefined references.

J Decker
Well, I assumed that one library would be with all the other libraries so I have this bit of code


that reads /proc/self/maps and finds its own name (defined as part of the compile options TARGET_NAME as a string), and gets the path from that.  Works on any system that puts libraries in some place. 

On Wed, Aug 14, 2019 at 1:17 AM Alex Cohn <[hidden email]> wrote:
The /data/data/<your.package>/lib path is not only undocumented, it's not valid anymore. The path to installed native libs depend on the version of Android, and the most recent versions randomize this path for further protection. The supported way to find location of the native libraries is through Java API ApplicationInfo.nativeLibDir, but even this has some caveats, see htps://stackoverflow.com/a/56551499/192373.

BR,
Alex Cohn

On Wednesday, August 14, 2019 at 9:51:35 AM UTC+3, Raj wrote:

 Thanks for your time and inputs J Decker, and due to some unavoidable circumstances I couldn't reply earlier. Now, I'm using the dlopen to load the libraries at run time and link the function pointers using dlsym (instead of relying on undefined references). Got to know that the best way to load the libraries is to use System.loadLibrary("lib.so") calls from Java side. But I want to explicitly do it from C side using dlopen( ) calls only. So, as of now the libraries are placed at '/data/data/<package>/libs' and I'm using this path, but the problem is that the path is undocumented. As per the answer in (https://stackoverflow.com/questions/15719149/how-to-integrate-native-runtime-library-with-dlopen-on-ndk), the best way to obtain the path to the libraries is to use '/proc' and 'derive from that the path to other libs' - I couldn't find more details on how to use '/proc' to obtain the path to the libraries. Can anyone please point me to a tutorial or an example from which I can refer.

  

   Another question: If we have an update to a dynamic library, during my program start-up, can I copy the updated library from another folder to my project libraries folder ('/data/data/<package>/libs' or the one obtained with the help of '/proc'). We would like to do this to avoid giving an updated APK when ever we would have an update to one of our dynamic libraries ? Is it recommended not to use such methods or is it valid on Android platform.. 

 

Thanks,

Raj.


On Fri, Aug 2, 2019, 17:42 J Decker <[hidden email]> wrote:


On Fri, Aug 2, 2019 at 2:37 AM Raj Shekar <[hidden email]> wrote:
Thanks for your time J Decker,  Instead of using the function pointers won't it be possible to link the library with the functions which are exposed by already loaded libraries. I would like to give a try by disabling the --no-undefined flag from the linker options. 

With verbose logging of cmake, how can we disable the - - no-undefined flag from the linker flags, it seems to get added by default. 

LANGUAGE_COMPILE_FLAGS = -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++ -std=c++11 -frtti -fexceptions -O0 -fno-limit-debug-info
LINK_FLAGS = -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-allow-shlib-undefined -Wl,-undefined

Relocations are rarely done after a library is loaded.
If the library to use the undefined function doesn't previously resolve the function, it won'tload.  If it does, then loading a new one won't replace the old relocation.
A function pointer for most purposes is the same as the name of a function... inporting a function or function pointer in Java looks the same as any other function...


Thanks, 
Raj. 

On Thu, Aug 1, 2019, 21:14 J Decker <[hidden email]> wrote:


On Thu, Aug 1, 2019 at 5:15 AM Raj Shekar <[hidden email]> wrote:

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.
If you're using dlopen to load the library, then you should use dlsym to get the function from that library and just declare a `void (*test_func)(...)` which is a function pointer to set from the dlsym then just call test_func();
then it doesn't need to exist ever.
Otherwise you'll need a dummy library of some name, but even then, when it links to the test_func in THAT library, it would refer to that library's name, and not nesscarily be obscured by loading other libraries also containing that symbol.
 

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/4ee4455f-7ce3-45a9-bf3c-5b49cac03e71%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqXqfx0iJPo7eQm-ccy3JD9MHPv6FPTBQLVtrAWuBsi--A%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Building dynamic libraries with undefined references.

J Decker


On Wed, Aug 14, 2019 at 3:36 AM J Decker <[hidden email]> wrote:
Well, I assumed that one library would be with all the other libraries so I have this bit of code


Err I guess it looks for the address of the routine itself instead of by name... the range that has the code is probably this library :) 

that reads /proc/self/maps and finds its own name (defined as part of the compile options TARGET_NAME as a string), and gets the path from that.  Works on any system that puts libraries in some place. 

On Wed, Aug 14, 2019 at 1:17 AM Alex Cohn <[hidden email]> wrote:
The /data/data/<your.package>/lib path is not only undocumented, it's not valid anymore. The path to installed native libs depend on the version of Android, and the most recent versions randomize this path for further protection. The supported way to find location of the native libraries is through Java API ApplicationInfo.nativeLibDir, but even this has some caveats, see htps://stackoverflow.com/a/56551499/192373.

BR,
Alex Cohn

On Wednesday, August 14, 2019 at 9:51:35 AM UTC+3, Raj wrote:

 Thanks for your time and inputs J Decker, and due to some unavoidable circumstances I couldn't reply earlier. Now, I'm using the dlopen to load the libraries at run time and link the function pointers using dlsym (instead of relying on undefined references). Got to know that the best way to load the libraries is to use System.loadLibrary("lib.so") calls from Java side. But I want to explicitly do it from C side using dlopen( ) calls only. So, as of now the libraries are placed at '/data/data/<package>/libs' and I'm using this path, but the problem is that the path is undocumented. As per the answer in (https://stackoverflow.com/questions/15719149/how-to-integrate-native-runtime-library-with-dlopen-on-ndk), the best way to obtain the path to the libraries is to use '/proc' and 'derive from that the path to other libs' - I couldn't find more details on how to use '/proc' to obtain the path to the libraries. Can anyone please point me to a tutorial or an example from which I can refer.

  

   Another question: If we have an update to a dynamic library, during my program start-up, can I copy the updated library from another folder to my project libraries folder ('/data/data/<package>/libs' or the one obtained with the help of '/proc'). We would like to do this to avoid giving an updated APK when ever we would have an update to one of our dynamic libraries ? Is it recommended not to use such methods or is it valid on Android platform.. 

 

Thanks,

Raj.


On Fri, Aug 2, 2019, 17:42 J Decker <[hidden email]> wrote:


On Fri, Aug 2, 2019 at 2:37 AM Raj Shekar <[hidden email]> wrote:
Thanks for your time J Decker,  Instead of using the function pointers won't it be possible to link the library with the functions which are exposed by already loaded libraries. I would like to give a try by disabling the --no-undefined flag from the linker options. 

With verbose logging of cmake, how can we disable the - - no-undefined flag from the linker flags, it seems to get added by default. 

LANGUAGE_COMPILE_FLAGS = -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++ -std=c++11 -frtti -fexceptions -O0 -fno-limit-debug-info
LINK_FLAGS = -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-allow-shlib-undefined -Wl,-undefined

Relocations are rarely done after a library is loaded.
If the library to use the undefined function doesn't previously resolve the function, it won'tload.  If it does, then loading a new one won't replace the old relocation.
A function pointer for most purposes is the same as the name of a function... inporting a function or function pointer in Java looks the same as any other function...


Thanks, 
Raj. 

On Thu, Aug 1, 2019, 21:14 J Decker <[hidden email]> wrote:


On Thu, Aug 1, 2019 at 5:15 AM Raj Shekar <[hidden email]> wrote:

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.
If you're using dlopen to load the library, then you should use dlsym to get the function from that library and just declare a `void (*test_func)(...)` which is a function pointer to set from the dlsym then just call test_func();
then it doesn't need to exist ever.
Otherwise you'll need a dummy library of some name, but even then, when it links to the test_func in THAT library, it would refer to that library's name, and not nesscarily be obscured by loading other libraries also containing that symbol.
 

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/4ee4455f-7ce3-45a9-bf3c-5b49cac03e71%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgrBoiFQC4MX%3D3typNQwwMof4vcT7een6YsCkA%3DcBt0A%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Building dynamic libraries with undefined references.

Alex Cohn
In reply to this post by J Decker
As Dan pointed out, on Android you should not worry about the paths to native libs. They can actually not exist on the file system (see https://developer.android.com/topic/performance/reduce-apk-size#extract-false). It's enough to call dlopen for the well-known name of the library. Furthermore, because the only legitimate way to deliver these libraries is to pack them inside your APK or bundle (possibly, one of the secondary APKs), in the prescribed path (/lib/{ABI}), there is no way you can use the path to do some logic (e.g. you cannot have a/libtest.so and b/libtest.so in your app),

Regards,
Alex

On Wednesday, August 14, 2019 at 1:36:33 PM UTC+3, J Decker wrote:
Well, I assumed that one library would be with all the other libraries so I have this bit of code

<a href="https://github.com/d3x0r/SACK/blob/master/src/systemlib/system.c#L510-L561" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fd3x0r%2FSACK%2Fblob%2Fmaster%2Fsrc%2Fsystemlib%2Fsystem.c%23L510-L561\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGQt9gN_W_jFLrrDWrTJj5htH8xRw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fd3x0r%2FSACK%2Fblob%2Fmaster%2Fsrc%2Fsystemlib%2Fsystem.c%23L510-L561\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGQt9gN_W_jFLrrDWrTJj5htH8xRw&#39;;return true;">https://github.com/d3x0r/SACK/blob/master/src/systemlib/system.c#L510-L561 

that reads /proc/self/maps and finds its own name (defined as part of the compile options TARGET_NAME as a string), and gets the path from that.  Works on any system that puts libraries in some place. 

On Wed, Aug 14, 2019 at 1:17 AM Alex Cohn <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="8et5105jAgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">sash...@...> wrote:
The /data/data/<your.package>/lib path is not only undocumented, it's not valid anymore. The path to installed native libs depend on the version of Android, and the most recent versions randomize this path for further protection. The supported way to find location of the native libraries is through Java API ApplicationInfo.nativeLibDir, but even this has some caveats, see htps://<a href="http://stackoverflow.com/a/56551499/192373" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fa%2F56551499%2F192373\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGsN77VJKeTQE9adqceVKJw0j59wA&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fa%2F56551499%2F192373\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGsN77VJKeTQE9adqceVKJw0j59wA&#39;;return true;">stackoverflow.com/a/56551499/192373.

BR,
Alex Cohn

On Wednesday, August 14, 2019 at 9:51:35 AM UTC+3, Raj wrote:

 Thanks for your time and inputs J Decker, and due to some unavoidable circumstances I couldn't reply earlier. Now, I'm using the dlopen to load the libraries at run time and link the function pointers using dlsym (instead of relying on undefined references). Got to know that the best way to load the libraries is to use System.loadLibrary("lib.so") calls from Java side. But I want to explicitly do it from C side using dlopen( ) calls only. So, as of now the libraries are placed at '/data/data/<package>/libs' and I'm using this path, but the problem is that the path is undocumented. As per the answer in (<a href="https://www.google.com/url?q=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F15719149%2Fhow-to-integrate-native-runtime-library-with-dlopen-on-ndk&amp;sa=D&amp;sntz=1&amp;usg=AFQjCNG48wlZwO81AO3gLbfFEzylpfPWRQ" style="color:rgb(66,133,244)" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fstackoverflow.com%2Fquestions%2F15719149%2Fhow-to-integrate-native-runtime-library-with-dlopen-on-ndk\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG48wlZwO81AO3gLbfFEzylpfPWRQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fstackoverflow.com%2Fquestions%2F15719149%2Fhow-to-integrate-native-runtime-library-with-dlopen-on-ndk\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG48wlZwO81AO3gLbfFEzylpfPWRQ&#39;;return true;">https://stackoverflow.com/questions/15719149/how-to-integrate-native-runtime-library-with-dlopen-on-ndk), the best way to obtain the path to the libraries is to use '/proc' and 'derive from that the path to other libs' - I couldn't find more details on how to use '/proc' to obtain the path to the libraries. Can anyone please point me to a tutorial or an example from which I can refer.

  

   Another question: If we have an update to a dynamic library, during my program start-up, can I copy the updated library from another folder to my project libraries folder ('/data/data/<package>/libs' or the one obtained with the help of '/proc'). We would like to do this to avoid giving an updated APK when ever we would have an update to one of our dynamic libraries ? Is it recommended not to use such methods or is it valid on Android platform.. 

 

Thanks,

Raj.


On Fri, Aug 2, 2019, 17:42 J Decker <[hidden email]> wrote:


On Fri, Aug 2, 2019 at 2:37 AM Raj Shekar <[hidden email]> wrote:
Thanks for your time J Decker,  Instead of using the function pointers won't it be possible to link the library with the functions which are exposed by already loaded libraries. I would like to give a try by disabling the --no-undefined flag from the linker options. 

With verbose logging of cmake, how can we disable the - - no-undefined flag from the linker flags, it seems to get added by default. 

LANGUAGE_COMPILE_FLAGS = -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++ -std=c++11 -frtti -fexceptions -O0 -fno-limit-debug-info
LINK_FLAGS = -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-allow-shlib-undefined -Wl,-undefined

Relocations are rarely done after a library is loaded.
If the library to use the undefined function doesn't previously resolve the function, it won'tload.  If it does, then loading a new one won't replace the old relocation.
A function pointer for most purposes is the same as the name of a function... inporting a function or function pointer in Java looks the same as any other function...


Thanks, 
Raj. 

On Thu, Aug 1, 2019, 21:14 J Decker <[hidden email]> wrote:


On Thu, Aug 1, 2019 at 5:15 AM Raj Shekar <[hidden email]> wrote:

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.
If you're using dlopen to load the library, then you should use dlsym to get the function from that library and just declare a `void (*test_func)(...)` which is a function pointer to set from the dlsym then just call test_func();
then it doesn't need to exist ever.
Otherwise you'll need a dummy library of some name, but even then, when it links to the test_func in THAT library, it would refer to that library's name, and not nesscarily be obscured by loading other libraries also containing that symbol.
 

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-ndk/CAA2GJqWDOQny-X79p5Uisrxzh3ODBWiwt%2B4Vf7N6t11ZNBvjog%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-ndk/CAOpHf7NTbPmo--ngdArUc5TSfSv-PZRZdZcct_F1xBQDsOyG0Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-ndk/CAA2GJqWgiOJMs9rwGjb6Dycin%2Byj49EFRnJEuQYqmgUbcttGBg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="8et5105jAgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">andro...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-ndk/4ee4455f-7ce3-45a9-bf3c-5b49cac03e71%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/4ee4455f-7ce3-45a9-bf3c-5b49cac03e71%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-ndk/4ee4455f-7ce3-45a9-bf3c-5b49cac03e71%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-ndk/4ee4455f-7ce3-45a9-bf3c-5b49cac03e71%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/8fc48a05-d272-4174-b700-cfafed027a6b%40googlegroups.com.