Understanding of Android Build system

Hello Guys !!! Hope you all doing well. Today I am going to discuss AOSP build system.

Android Build system is a part of the AOSP source code. How to download and build AOSP you can follow my previous blog link.

• Android Build system used to compile the Android system, Android SDK and related package and documents.
• Android Build system is mainly contain Make files, Shell scripts and Python scripts. The most important one is Make files.
• From Android 6 (M, API 23) AOSP master platform builds with ninja instead ofGNU make and in Android 7 (N, API 24, 25) , ninja and kati become default build system.
• The build/core makefiles and Android.mk files are read by kati which will interpret them and write out a$OUT_DIR/build_$(TARGET_file, and then the execution of the build rules will be handled by ninja.

Building with kati and ninja provides some immediate benefits:

1. Faster start-up time for new builds. Running kati and then ninja results in builds starting about 25% faster than GNU make.
2. Much faster incremental builds. kati caches the mtimes of files and directories it read, what environment variables it used, etc., and only re-reads all the makefiles if something has changed. This includes globs like $(call all-files-under). Incremental builds with a warm disk cache will start within 5 seconds.
3. Better build command output. Ninja buffers the output of each command and prints it without mixing output from multiple commands, meaning error messages will be printed as a group after a build failure, and not mixed with other concurrent build output. Ninja also prints a configurable status line that defaults to printing the percentage of build commands that have been run, the number that have been run, and the number that need to run, giving a reasonable approximation of the progress through the build.
4. Incremental builds should be more reliable. Ninja considers outputs whose command line has changed to be dirty, which will catch changes to global cflags, etc. and rebuild anything that has changed.
5. The use of ninja should be transparent, it is wrapped by the top level Makefile, so all make-based commands should continue to work.

For more detail you can follow following link

https://android.googlesource.

https://github.com/google/

https://android.googlesource.

https://ninja-build.org/

https://android-review.

https://groups.google.com/

https://www.xda-developers.

The build system uses some pre-set environment variables and a series of 'make' files in order to build an Android system and prepare it for deployment to a platform.

Step by step detail of AOSP build System

1. Types of tools used by AOSP build system

• During the build we use 'make' to control the build steps themselves.
• A host toolchain (compiler, linker and other tools) and libraries will be used to build programs and tools that will run on the host.
• A different toolchain is used to compile the C and C++ code that will wind up on the target (which is an embedded board, device or the emulator).
• The kernel is compiled as a standalone binary (it does not use a program loader or link to any outside libraries).
• Native programs (e.g. init or toolbox), daemons or libraries will link against bionic or other system libraries.
• A Java compiler and a bunch of java-related tools to build most of the application framework, system services and Android applications themselves.
• Finally, tools are used to package the applications and resource files, and to create the filesystem images that can be installed on a device or used with the Emulator.

2. Setup "what to build"

• In order to decide what to build, and how to build it, the build system requires that some variables be set.
• Different products, with different packages and options can be built from the same source tree.
• The variables to control this can be set via a file with declarations of 'make' variables, or can be specified in the environment.
• A device vendor can create definition files that describe what is to be included on a particular board or for a particular product. The definition file is called: buildspec.mk, and it is located in the top-level source directory.
• Another method of specifying options is to set environment variables. To set up your build environment, you need to load the variables and function in build/envsetup.sh. your can set shell environment, like this

$ . build/envsetup.sh

or

$ source build/envsetup.sh

You can type 'help' (or 'hmm') at this point to see some utility functions that are available to make it easier to work with the source.

3. Building the system

Once We have things set up, you actually build the system with the 'make' command. Before issuing the make command we have to put lunch command in our terminal.

• lunch command provide a list of available build types.
• It will show both non-official or non-standard builds for your device
• It includes special debug versions and also allows you to build for Android Emulator

After lunch command, We have to use 'make' command for final AOSP building.

• $ make -j4 // Based on your CPU core and RAM you can give -j8, -j16 etc
• $ make -j4 showcommands // It will show build process used by AOSP

4. Make targets

List of different make targets we can use to build different parts of the system

• make sdk- build the tools that are part of an SDK (adb, fastboot, etc.)
• make snod- build the system image from the current software binaries
• make services
• make runtime
• make all- make everything, whether it is included in the product definition or not
• make clean- remove all built files (prepare for a new build). Same as rm -rf out/<configuration>/
• make modules- shows a list of submodules that can be built (List of all LOCAL_MODULE definitions)
• make update-api- To update the framework and native libs during AOSP build

5. Helper macros and functions

There are some helper macros and functions that are installed when we set envsetup.sh. Following are most used one :-

• m- execute 'make' from the top of the tree (even if your current directory is somewhere else)
• mm- Builds all of the modules in the current directory, but not their dependencies
• mmm <dir1>- Builds all of the modules in the supplied directories, but not their dependencies
• mma- Builds all of the modules in the current directory, and their dependencies.
• mmma <dir>- Builds all of the modules in the supplied directories, and their dependencies
• mm -B : Rebuild the module even it hasn’t any changes.
• cgrep <pattern>- grep on all local C/C++ files
• jgrep <pattern>- grep on all local Java files
• resgrep <pattern>- grep on all local res/*.xml files
• godir <filename>- go to the directory containing a file

6. Syncing the changes directly

If you make changes in some file and you want to update this changes in your connected device or emulator you can do it easily. Example:-

we can do our change on the PhoneStatusBar.java file and run the following commands to update the device:

• $ mmm frameworks/base/packages/
• $ adb sync // Sync the updated module to the device
• $ adb shell stop // Stopping the shell
• $ adb shell start // Starting the shell again

And all that will take less than 30 seconds!

Hope Guys!!! You enjoyed this articles.

Describing Content of AOSP Folder

Hello Guys!!! Hope You all doing well.

Toady I am going to describe what all file and folder contained by AOSP source code folder and its uses.  In my Previous Post, I had described "How to download Android AOSP Code". Click here for that post.
 So after successfully download of Android AOSP source code for any version(L,M or N), you find so many subfolders and file inside AOSP source Code Folder.   Today I am going to put some light on it.

 Now I am starting folder name in alphabetical order .

1. abi - This folder contains a subfolder called cpp which actually contains many C++ files linked to many places.

2. art - it is the folder that deals with the compilation of the latest android ART run time.

3. bionic - Bionic is mainly a port of the BSD C library to our Linux kernel with the following additions/changes:

• No support for locales.
• No support for wide chars (i.e. multi-byte characters).
• its own smallish implementation of pthreads based on Linux futexes.
• Support for x86, ARM and ARM thumb CPU instruction sets and kernel interfaces.

4. bootable - Boot and startup related code. Some of it is legacy, the fastboot protocol info could be interesting since it is implemented by boot loaders in a number of devices such as the Nexus ones.

5. build - The main entry point of the build system resides here - envsetup.sh. 
First step before you do anything to build Android is to use the command
source build/envsetup.sh
This script will check few things to make sure all the needed application available in the local machine. It also setup the devices that can be built, which is extracted from the directory device.

6. cts - the compatibility tests. The test suite to ensure that a build complies with the Android specification. How to use it or download it. Please follow my this Blog Link

7. dalvik - This is the folder responsible for the compilation of the Dalvik runtime for the Android devices.
Have a look at the difference between the two(art and dalvik) folders and you'll have an idea of how things work in that case.

8. Development - This directory contains application that are not part of the deployed app in the OS. There are some useful application such as widget builder, etc

9. Device - It contains the device specific configurations for many devices. It contains following   subfolder  'common',  'generic', 'Google' and 'Sample'.
 common - This directory contains gps information and also a script that allows you to extract proprietary binary files from your phone to be part of the build process.(You can try to have a look at your device's device tree and then the cm.mk file, where you could find relations of these files. In my case, it shows like this:

# Include GSM stuff

$(call inherit-product, vendor/cm/config/gsm.mk)

-and-

# Inherit some common cyanogenmod stuff.

$(call inherit-product, device/common/gps/gps_eu_supl.mk)

generic - This directory contains the generic device configuration that is called ‘goldfish’. This is the device classification used when building for the emulator.

Google - This directory contains the Android Accessories Kit code. It contains a demokit Android app that allows you to control the ADK board.

sample - This directory contains a full example of writing your own Android platform shared library, without changing the Android framework. It also shows how to write JNI code for incorporating native code into the library, and a client application that uses the library. This example is ONLY for people working with the open source platform to create a system image that will be delivered on a device which will include a custom library as shown here. It can not be used to create a third party shared library, which is not currently supported in Android.

10. external - This directory contains source code for all external open source projects such as SQLite, Freetype, webkit and webview.

11. frameworks - One of the most important directories. it contains the sources for the framework. Here you will find the implementation of key services such as the System Server with the Package and Activity managers.
A lot of the mapping between the java application APIs and the native libraries is also done here.

12. hardware - Hardware related source code such as the Android hardware abstraction layer (HAL) specification and implementation. This folder also contains the reference Radio Interface Layer(RIL - To communicate with the modem side) implementation.

13. Kernel - It's not a default folder in the source code, but it's a part of device configuration set-up. It contains the kernel source of your device.

14. libcore - I'll explain this one with the important folders inside this, since every folder performs a different function.
dalvik - DalvikVM runtime for Android
dom - Java test classes for DOM
expectations - Contains information about the test cases
include - Some C/C++ include files that used for Array and String handling
json - JSON based Java implementation
luni - Contains test source code for loading .jar and .dex files
support - Contains support class file for testing Dalvik
xml - XML pull and push implementation

15. ndk - Contains build scripts and helper files for building the NDK

16. out - The build output will be placed here after you run make.
The folder structure is out/target/product/. In the default build for the emulator the output will be placed in out/target/product/generic. This is where you will find the images used by the emulator to start (or to be downloaded and flashed to a device if you are building for a hardware target).

17. packages - Standard Android application that are available as part of the AOSP - Camera, SMS, Dialer, Launcher, etc

18. prebuilt - Contains files that are distributed in binary form for convenience. Examples include the cross compilations toolchains for different development machines.

19. sdk - This directory contains lots of apps that are not part of operating system. There are quite useful apps that developers can leverage on and can be enhanced further as part of the operating system.

20. system - Source code files for the core Android system. That is the minimal Linux system that is started before the Dalvik VM and any java based services are enabled. This includes the source code for the init process and the default init.rc script that provide the dynamic configuration of the platform.

21. tools - Some external important tools that help in compiling. Not sure.

22. vendor - This directory contains vendors specific libraries. Most of the proprietary binary libraries from non-open source projects are stored here when building AOSP.

Apart from above mention folder , there is  a hidden .repo directory that contains the source for the repo utility. It also contain a manifest specifying what git repositories you want to track for this Android source project. If you have your own additions you could automatically track them by adding a local manifest here.

For modifications of the platform framework there are some instructions available in the device/sample folder of the source code tree. That will show you how to add APIs to Android without having to modify the core framework.

Hope it will help you understanding the Android AOSP Source code subfolder and its uses. In my next Blog I will deal "Android AOSP Build Process and its command"

Happy Coding!!!

Thanks
Saurabh
Note:- Please proved your comment and suggestion for my post. It will energize me.

Add .so file in Android AOSP source code

Hello Guys !!! Hope You are doing well.
Today I am going to discuss How to add .so file in Android AOSP source code external folder.
You can add your prebuilt library in Android AOSP source code and it be a part of your AOSP System Image. I am describing step by step procedure for it.

Step 1 Create a folder ( let say myLibs) inside external folder of AOSP source code.

external folder of AOSP source code refers to external open source libraries.
That means libraries that the Android platform depend upon but that are not primarily developed and maintained by the Android open source project.

examples are webkit for the browser, FreeType for fonts, SqlLite for databases and so on. As more features are added to Android, more of these libraries are included in external.

Step 2 Create a Android.mk file

Create a Android.mk file inside your folder(let say myLibs) and copy your .so file in it.
You can use following content for your android.mk file

# Prebuilt Lib
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := libMyabc # your lib name
LOCAL_SRC_FILES := libMyabc.so
# your lib .so file name
include $(BUILD_SHARED_LIBRARY)

Step 3 Add your library in Framework

In final step you have to add your library in Android AOSP framework makefile so that it will recognise and build as a part of System image.
You find Framework Android.mk file on following location
/android_aosp_sourcecode_download_folder/frameworks/base/core/jni/
Open Android.mk file and add your library in following section
LOCAL_SHARED_LIBRARIES := \
You can put your library name in that section example libMyabc \
That's it... now make it (make -j4) and you find your added so file in following folder
/android_aosp_sourcecode_download_folder/out/target/product/generic/obj/lib
with file name like :- libMyabc.so and libMyabc.so.toc
and you also found it in system/lib folder
/android_aosp_sourcecode_download_folder/out/target/product/system/lib

Thanks
Saurabh
Happy Coding...