Journal html training file in progress of converting to ollama format:
Summary in html for IPA development:
IOS 13.2
Recommended development settings: MacOS 10.13 High Sierra, iPhone 6s
unCover 8.2 ipa
Ios 15.8
Recommended development settings: MacOS 15.3 Sequoia, iPhone 6s, Iphone 6sp
Dopamine.ipa, AFNET wifi-settings, trollstore
Server address for downloading .ipa files
www.remakestar.net/ipafiles/Dopamine.ipa including a git source of dopamine-mas.tar
Also included a html for grapheneOS build environment for pixel phone:
Notes for building Graphene OS:
This is a guide on building, modifying and contributing to GrapheneOS as a developer.
Smartphone and tablet targets:
These are all fully supported production-ready targets supporting all the baseline security features and receiving full monthly security updates covering all firmware, kernel drivers, driver libraries / services and other device-specific code. A fully signed user build for these devices is a proper GrapheneOS release. Newer generation devices have stronger hardware / firmware security and hardware-based OS security features and are better development devices for that reason. It's not possible to work on everything via past generation devices. The best development devices are the Pixel 6 and 7 series.
SDK emulator target:
These emulator targets don't receive full monthly security updates, don't provide all of the baseline security features and are intended for development usage.
We recommend using the sdk_phone64_x86_64 target in either the userdebug or eng variant for most development work.
Providing proper support for a device or generic device family requires providing an up-to-date kernel and device support code including driver libraries, firmware and device SELinux policy extensions. Other than some special cases like the emulator, the generic targets rely on the device support code present on the device. Shipping all of this is necessary for full security updates and is tied to enabling verified boot / attestation. Device-specific driver changes are required to support GrapheneOS features such as the hardware-level USB-C port control. There's also a lot of device-specific work to work around or resolve memory corruption bugs uncovered by our features. Most devices are also missing a lot of standard hardware-based security features we use such as hardware memory tagging so those features will inherently not be possible to port to a device not capable of it.
Arch Linux, Debian bookworm, Ubuntu 24.10 and Ubuntu 24.04 LTS are the officially supported operating systems for building GrapheneOS.
Set up Debian GNU/Linux 12
(bookworm)
To build GrapheneOS, install the required packages:
apt install repo yarnpkg zip rsync
Your PATH may not contain directories like /sbin, and many system
administration commands will fail. The adevtool from GrapheneOS
requires the debugfs binary in /sbin. The simplest workaround is to add these directories to your PATH environment
variable (and apply the change to your current shell):
echo 'export PATH=$PATH:/sbin:/usr/sbin:/usr/local/sbin' >> ~/.bashrcsource ~/.bashrc
Dependencies for fetching and verifying the sources:
Baseline build dependencies:
You can either obtain repo as a distribution
package or the self-updating standalone version from the Android Open Source
Project. The self-updating variant avoids dealing with out-of-date distribution
packages and depends on GPG to verify updates.
The Android Open Source Project build system is designed to provide reliable and reproducible builds. To accomplish this, it provides a prebuilt toolchain and other utilities fulfilling most of the build dependency requirements itself. These prebuilt tools have reproducible builds themselves. It runs the build process within a loose sandbox to avoid accidental dependencies on the host system. The process of moving to a fully self-contained build process with minimal external dependencies is gradual and there are still dependencies that need to be installed on the host system.
Additional Android Open Source Project build dependencies not provided by the source tree:
Additional dependencies for extracting vendor files with adevtool:
Additional Vanadium (Chromium) build dependencies not provided by the source tree:
Additional dependencies for signing factory images releases:
Since this is syncing the sources for the entire operating system and application layer, it will use a lot of bandwidth and storage space.
You likely want to use the most recent stable tag, not the development branch, even for developing a feature. It's easier to port between stable tags that are known to work properly than dealing with a moving target.
The 15 branch is the main development branch of GrapheneOS based on the
latest release of Android 15. It will follow along with the latest monthly and
quarterly releases of Android 15. The 15 branch of GrapheneOS
should be used for generic builds including the emulator, porting to other
devices and for non-legacy officially supported devices.
The 14 branch is used for the end-of-life Pixel 5a.
The 14-redfin branch is used for the end-of-life Pixel 4a (5G) and Pixel 5.
The 13 branch is used for the end-of-life Pixel 4a.
The 13-coral branch is used for the end-of-life Pixel 4 and Pixel 4 XL.
To sync the 15 branch:
mkdir grapheneos-15cd grapheneos-15repo init -u https://github.com/GrapheneOS/platform_manifest.git -b 15repo sync -j8
If your network is unreliable and repo sync fails, you can
run the repo sync command again to continue from where it was interrupted. It handles
connection failures robustly and you shouldn't start over from scratch.
Pick a specific release for a device from the releases page and download the source tree. Note that some devices use different Android Open Source Project branches so they can end up with different tags. Make sure to use the correct tag for a device. For devices without official support, use the latest tag marked as being appropriate for generic / other devices in the release notes.
mkdir grapheneos-TAG_NAMEcd grapheneos-TAG_NAMErepo init -u https://github.com/GrapheneOS/platform_manifest.git -b refs/tags/TAG_NAME
Obtain SSH public key for verifying tags:
curl https://grapheneos.org/allowed_signers > ~/.ssh/grapheneos_allowed_signers
Verify the manifest:
cd .repo/manifestsgit config gpg.ssh.allowedSignersFile ~/.ssh/grapheneos_allowed_signersgit verify-tag $(git describe)cd ../..
Complete the source tree download:
repo sync -j8
The manifest for the latest stable release refers to the revisions in other repositories via commit hashes rather than tag names. This avoids the need to use a script to verify tag signatures across all the repositories, since they simply point to the same commits with the same hashes.
Note that the repo command itself takes care of updating itself and uses GPG to verify by default.
To update the source tree, run the repo init command again to
select the branch or tag and then run repo sync -j8 again. You
may need to add --force-sync if a repository switched from one source to another, such as when
GrapheneOS forks an additional Android Open Source Project repository. You
don't need to start over to switch between different branches or tags. You may
need to run repo init again to continue down the same branch since GrapheneOS only provides
a stable history via tags.
This section is specific to Pixel devices. The emulator and generic targets don't require extra vendor files.
Many of these components are already open source, but not everything is set up to be built by the Android Open Source Project build system. Switching to building these components from source will be an incremental effort. In many cases, the vendor files simply need to be ignored and AOSP will already provide them instead. Firmware cannot generally be built from source even when sources are available, other than to verify that the official builds match the sources, since it has signature verification (which is an important part of the verified boot and attestation security model).
The below commands need to only be run once to initially
create a working environment. (Note: On Debian, yarn is reserved for a
binary in cmdtest; use yarnpkg instead.)
yarn install --cwd vendor/adevtool/source build/envsetup.shlunch sdk_phone64_x86_64-cur-userm aapt2
Download, extract and prepare the vendor files:
adevtool generate-all -d PIXEL_CODENAME
Replace PIXEL_CODENAME with Pixel device codename, which is the same as build target name.
The build has to be done from bash or zsh as envsetup.sh is not compatible with other shells.
Set up the build environment:
source build/envsetup.sh
Select the desired build target (husky is the Pixel 8 Pro):
lunch husky-cur-user
For past stable tags and legacy branches, don't pass the Android release:
lunch husky-user
For a development build, you may want to replace user with userdebug in order
to have better debugging support. Production builds should be user builds as
they are significantly more secure and don't make additional performance
sacrifices to improve debugging.
Set OFFICIAL_BUILD=true to include the Updater app. You must change the URL
in packages/apps/Updater/res/values/config.xml to your own update server URL. Using the official update server with a
build signed with different keys will not work and will essentially perform a
denial of service attack on our update service. If you try to use the official
URL, the app will download an official update and will detect it as corrupted
or tampered. It will delete the update and try to download it over and over
again since it will never be signed with your key.
export OFFICIAL_BUILD=true
To reproduce a past build, you need to export BUILD_DATETIME and
BUILD_NUMBER to the values set for the past build. These can be obtained from out/build_date.txt
and out/build_number.txt in a build output directory and the ro.build.date.utc and ro.build.version.incremental properties which are also included in the over-the-air zip metadata
rather than just the OS itself.
The signing process for release builds is done after completing builds and replaces the dm-verity trees, apk signatures, etc. and can only be reproduced with access to the same private keys. If you want to compare to production builds signed with different keys you need to stick to comparing everything other than the signatures.
Additionally, set OFFICIAL_BUILD=true per the instructions
above to reproduce the official builds. Note that if you do not change the URL
to your own domain, you must disable the Updater app before
connecting the device to the internet, or you will be performing a denial of
service attack on our official update server.
Incremental builds (i.e. starting from the old build) usually work for development and are the normal way to develop changes. However, there are cases where changes are not properly picked up by the build system. For production builds, you should remove the remnants of any past builds before starting, particularly if there were non-trivial changes:
rm -r out
Next, start the build process with the m command:
m target-files-package
For the Pixel 6, Pixel 6 Pro and Pixel 6a you currently need m vendorbootimage target-files-package instead of target-files-package.
For the Pixel 7, Pixel 7 Pro, Pixel 7a, Pixel Tablet, Pixel
Fold, Pixel 8, Pixel 8 Pro, Pixel 8a, Pixel 9, Pixel 9 Pro, Pixel 9 Pro XL and
Pixel 9 Pro Fold you currently need m vendorbootimage vendorkernelbootimage
target-files-package instead of target-files-package.
The -j parameter can be passed to m to set a specific number
of jobs such as -j4 to use 4 jobs. By default, the build system sets the number of jobs to
NumCPU() + 2 where NumCPU() is the number of available logical CPUs.
For an emulator build, always use the development build approach below.
The normal production build process involves building a target files package to be resigned with secure release keys and then converted into factory images and/or an update zip via the sections below. If you have a dedicated development device with no security requirements, you can save time by using the default build target rather than target-files-package. Leave the bootloader unlocked and flashing the raw images that are signed with the default public test keys.
To build the default build target:
m
Technically, you could generate test key signed update packages. However, there's no point of sideloading update packages when the bootloader is unlocked and there's no value in a locked bootloader without signing the build using release keys, since verified boot will be meaningless and the keys used to verify sideloaded updates are also public. The only reason to use update packages or a locked bootloader without signing the build with release keys would be testing that functionality and it makes a lot more sense to test it with proper signing keys rather than the default public test keys.
Keys need to be generated for resigning completed builds from the publicly available test keys. The keys must then be reused for subsequent builds and cannot be changed without flashing the generated factory images again which will perform a factory reset. Note that the keys are used for a lot more than simply verifying updates and verified boot.
The sample certificate subject (CN=GrapheneOS) should be
replaced with your own information.
You should set a passphrase for the signing keys to keep them
at rest until you need to sign a release with them. The GrapheneOS scripts (make_key and encrypt-keys)
encrypt the signing keys using scrypt for key derivation and AES256 as the
cipher. If you use swap, make sure it's encrypted, ideally with an ephemeral
key rather a persistent key to support hibernation. Even with an ephemeral key,
swap will reduce the security gained from encrypting the keys since it breaks
the guarantee that they become at rest as soon as the signing process is
finished. Consider disabling swap, at least during the signing process.
The encryption passphrase for all the keys generated for a device needs to match for compatibility with the GrapheneOS scripts.
To generate keys for raven (you should use unique keys per device variant):
mkdir -p keys/ravencd keys/ravenCN=GrapheneOS../../development/tools/make_key releasekey "/CN=$CN/"../../development/tools/make_key platform "/CN=$CN/"../../development/tools/make_key shared "/CN=$CN/"../../development/tools/make_key media "/CN=$CN/"../../development/tools/make_key networkstack "/CN=$CN/"../../development/tools/make_key sdk_sandbox "/CN=$CN/"../../development/tools/make_key bluetooth "/CN=$CN/"openssl genrsa 4096 | openssl pkcs8 -topk8 -scrypt -out avb.pem../../external/avb/avbtool.py extract_public_key --key avb.pem --output avb_pkmd.bincd ../..
The avb_pkmd.bin file isn't needed for generating a signed release but rather to set
the public key used by the device to enforce verified boot.
Generate an OpenSSH key for signing factory images:
ssh-keygen -t ed25519 -f keys/raven/id_ed25519
The passphrase for the SSH key should also match the other keys, although we don't currently handle it automatically in the encrypt/decrypt scripts.
You can (re-)encrypt your signing keys using the encrypt-keys
script, which will prompt for the old passphrase (if any) and new passphrase:
script/encrypt-keys keys/raven
The script/decrypt-keys script can be used to remove encryption, which is not recommended. The
script exists primarily for internal usage to decrypt the keys in tmpfs to
perform signing.
GrapheneOS currently doesn't use out-of-band updates to APEX components, so these are all signed with the OS releasekey and verified boot key to avoid needing many extra pairs of keys. Each APEX needs an APK signing key and verified boot signing key. If you want to ship out-of-band updates to APEX components, you'll need to deal with this and you should make a separate pair of keys for each one.
Consult the upstream documentation on generating these keys. It will likely be covered here in the future, especially if non-flattened APEX components become unavoidable.
Build and package up the tools needed to generate over-the-air update packages:
m otatools-package
Copy build artifacts to releases directory:
script/finalize.sh
Generate a signed release build with the generate-release.sh script:
script/generate-release.sh raven BUILD_NUMBER
The factory images and update package will be in releases/BUILD_NUMBER/release-raven-BUILD_NUMBER. The update zip performs a full OS installation so it can be used to
update from any previous version. More efficient incremental updates are used
for official over-the-air GrapheneOS updates and can be generated by keeping
around past signed target_files zips and generating incremental updates from those to the most recent
signed target_files zip.
See the install page for information on how to use the factory images. See the usage guide section on sideloading updates for information on how to use the update packages.
Running script/generate-release.sh also generates channel metadata for the update server. If you
configured the Updater client URL and set the build to include it (see the
information on OFFICIAL_BUILD above), you can push signed over-the-air updates via the update
system. Simply upload the update package to the update server along with the
channel metadata for the release channel, and it will be pushed out to the
update client. The DEVICE-beta and DEVICE-stable metadata
provide the Beta and Stable release channels used by the update client. The DEVICE-testing metadata
provides an internal testing channel for the OS developers, which can be
temporarily enabled using adb
shell setprop sys.update.channel testing. The name
is arbitrary and you can also use any other name for internal testing channels.
For GrapheneOS itself, the testing channel is used to push out updates to developer devices, followed by a sample future release to test that the release which is about to be pushed out to the Beta channel is able to update to a future release. Once it's tested internally, the release is pushed out to the Beta channel, and finally to the Stable channel after public testing. A similar approach is recommended for derivatives of GrapheneOS.
Incremental updates shipping only the changes between two
versions can be generated as a much more efficient way of shipping updates than
a full update package containing the entire operating system. The GrapheneOS
Updater app will automatically use a delta update if one exists for going
directly from the currently installed version to the latest release. In order
to generate a delta update, the original signed target files package for both
the source version and target version are needed. The script/generate-delta.sh script provides a wrapper script for generating delta updates by
passing the device, source version build number and target version build
number. For example:
script/generate-delta.sh raven 2021102503 2021102613
The script assumes that the releases are organized in the following directory structure:
releases___ 2022050700_ ___ release-raven-2022050700___ 2022050800 ___ release-raven-2022050800
Incremental updates are uploaded alongside the update packages and update metadata on the static web server used as an update server. The update client will automatically check for an incremental update and use it if available. No additional metadata is needed to make incremental updates work.
Like the Android Open Source Project, GrapheneOS contains some code that's built separately and then bundled into the source tree as binaries. This section will be gradually expanded to cover building all of it.
The Linux kernel is built separately using an AOSP kernel build system wrapping the upstream Linux kernel build system. Since it's a separate build system from the rest of the OS, the kernels are built separately. Many of the repositories are shared with the rest of the OS including the repositories providing the prebuilt toolchain for reproducible builds not depending on the tools from the host OS.
Additional Linux kernel build dependencies for 4th and 5th generation Pixels not provided by the source tree:
The default kernel version for the emulator is 6.6. You can
change the kernel version used for the x86_64 emulator target supported by
GrapheneOS in device/generic/goldfish/board/kernel/x86_64.mk.
To sync the 6.6 kernel sources:
mkdir -p android/kernel/6.6cd android/kernel/6.6repo init -u https://github.com/GrapheneOS/kernel_manifest-6.6.git -b 15repo sync -j8
To build the 6.6 kernel image and modules for the emulator:
ARCH=x86_64 common/build_virt.sh
Replace the prebuilts in the OS source tree:
ANDROID_BUILD_TOP=~/android/grapheneos-15 ARCH=x86_64 common/update_virt_prebuilts.sh
To sync the 6.1 kernel sources:
mkdir -p android/kernel/6.1cd android/kernel/6.1repo init -u https://github.com/GrapheneOS/kernel_manifest-6.1.git -b 15repo sync -j8
To build the 6.1 kernel image and modules for the emulator:
ARCH=x86_64 common/build_virt.sh
Replace the prebuilts in the OS source tree:
ANDROID_BUILD_TOP=~/android/grapheneos-15 ARCH=x86_64 common/update_virt_prebuilts.sh
To sync the kernel sources:
mkdir -p android/kernel/6.6cd android/kernel/6.6repo init -u https://github.com/GrapheneOS/kernel_manifest-6.6.git -b 15repo sync -j8
To build the arm64 kernel image for microdroid:
tools/bazel run //common:kernel_aarch64_microdroid_dist --config=stamp --lto=full
Replace the prebuilt kernel image in the OS source tree at packages/modules/Virtualization/microdroid/kernel/android15-6.6/arm64/kernel-6.6 with your image out/kernel_aarch64_microdroid/dist/Image.
To build the x86_64 kernel image for microdroid:
tools/bazel run //common:kernel_x86_64_microdroid_dist --config=stamp --lto=full
Replace the prebuilt kernel image in the OS source tree at packages/modules/Virtualization/microdroid/kernel/android15-6.6/x86_64/kernel-6.6 with your image out/kernel_x86_64_microdroid/dist/bzImage.
To sync the kernel sources:
mkdir -p android/kernel/pixelcd android/kernel/pixelrepo init -u https://github.com/GrapheneOS/kernel_manifest-pixel.git -b 15repo sync -j8
Select the correct kernel codename for the device model:
To build the kernel and modules:
./build_codename.sh --config=no_download_gki --config=no_download_gki_fips140 --lto=full
Replace the files in the OS source tree at device/google/codename-kernels/6.1/grapheneos/ with your build in out/codename/dist/.
To sync the kernel sources:
mkdir -p android/kernel/zumacd android/kernel/zumarepo init -u https://github.com/GrapheneOS/kernel_manifest-zuma.git -b 15repo sync -j8
Select the correct kernel codename for the device model:
To build the kernel and modules:
./build_codename.sh --config=no_download_gki --lto=full
Replace the files in the OS source tree at device/google/codename-kernels/5.15/grapheneos/ with your build in out/codename/dist/.
To sync the kernel sources:
mkdir -p android/kernel/gscd android/kernel/gsrepo init -u https://github.com/GrapheneOS/kernel_manifest-gs.git -b 15repo sync -j8
Select the correct kernel codename for the device model:
To build the kernel and modules:
BUILD_AOSP_KERNEL=1 LTO=full ./build_codename.sh
Replace the files in the OS source tree at device/google/codename-kernels/5.10/grapheneos/ with your build in out/mixed/dist/.
To sync the kernel sources:
mkdir -p android/kernel/redbullcd android/kernel/redbullrepo init -u https://github.com/GrapheneOS/kernel_manifest-redbull.git -b 15repo sync -j8
To build the redbull kernel for the Pixel 4a (5G), Pixel 5 and Pixel 5a kernel:
BUILD_CONFIG=private/msm-google/build.config.redbull.vintf build/build.sh
Replace the files in the OS source tree at device/google/redbull-kernel/ (userdebug/eng) and device/google/redbull-kernel/vintf/ (user) with your build in out/android-msm-pixel-4.19/dist/.
To sync the kernel sources:
mkdir -p android/kernel/coralcd android/kernel/coralrepo init -u https://github.com/GrapheneOS/kernel_manifest-coral.git -b 13repo sync -j8
To build the coral kernel for the Pixel 4 and Pixel 4 XL:
KBUILD_BUILD_VERSION=1 KBUILD_BUILD_USER=build-user KBUILD_BUILD_HOST=build-host KBUILD_BUILD_TIMESTAMP="Thu 01 Jan 1970 12:00:00 AM UTC" BUILD_CONFIG=private/msm-google/build.config.floral build/build.sh
To build the sunfish kernel for the Pixel 4a:
KBUILD_BUILD_VERSION=1 KBUILD_BUILD_USER=build-user KBUILD_BUILD_HOST=build-host KBUILD_BUILD_TIMESTAMP="Thu 01 Jan 1970 12:00:00 AM UTC" BUILD_CONFIG=private/msm-google/build.config.sunfish build/build.sh
Replace the files in the OS source tree at device/google/coral-kernel/ or device/google/sunfish-kernel/ with your build in out/android-msm-pixel-4.14/dist/.
Vanadium is a hardened fork of Chromium developed by GrapheneOS and used to provide the WebView and optionally the standalone browser app. It tracks the Chromium release cycles along with having additional updates for downstream changes to the privacy and security hardening patches, so it's updated at a different schedule than the monthly Android releases.
The browser and the WebView are independent applications built from the Chromium source tree. The GrapheneOS browser build is located at external/vanadium and the WebView is at external/chromium-webview.
See Chromium's Android build instructions for details on obtaining the prerequisites.
git clone https://github.com/GrapheneOS/Vanadium.gitcd Vanadiumgit checkout CORRECT_BRANCH_OR_TAG
Generate a signing key for Vanadium if this is the initial build:
keytool -genkey -v -keystore vanadium.keystore -storetype pkcs12 -alias vanadium -keyalg RSA -keysize 4096 -sigalg SHA512withRSA -validity 10000 -dname "cn=GrapheneOS"
You will be prompted to enter a password which will be
requested by the generate-release.sh script for signing releases. You should back up the generated keystore
with your other keys.
Fetch the Chromium sources:
fetch --nohooks android
Sync to the latest stable release for Android (replace VERSION with the correct value):
cd srcgit fetch --tagsgit checkout VERSION
Apply the GrapheneOS patches on top of the tagged release:
git am --whitespace=nowarn --keep-non-patch ../patches/*.patch
Sync submodules and obtain build dependencies:
gclient sync -D --with_branch_heads --with_tags --jobs 32
Then, configure the build in the src directory:
gn args out/Default
Copy the GrapheneOS configuration from ../args.gn and
save/exit the editor. Modify target_cpu as needed if the target is not arm64. For x86_64, the correct value
for target_cpu is x64, but note that the Android source tree refers to it as x86_64.
You need to set trichrome_certdigest to the correct value
for your generated signing key. You can obtain this with the following command:
keytool -export-cert -alias vanadium -keystore vanadium.keystore | sha256sum
Build the components:
chrt -b 0 autoninja -C out/Default/ trichrome_webview_64_32_apk trichrome_chrome_64_32_apk trichrome_library_64_32_apk vanadium_config_apk
Sign the apks:
../generate-release.sh out
The apks need to be copied from out/Default/apks/release/*.apk into the Android source tree at external/vanadium/prebuilt/arm64/ with arm64 substituted with the correct value for other architectures
(arm, x86, x86_64).
WebView provider apps need to be whitelisted in frameworks/base/core/res/res/xml/config_webview_packages.xml. By default, only the Vanadium WebView is whitelisted.
The official releases of our App Store, Auditor, Camera and
PdfViewer apps are bundled as apks into external/ repositories. The no-code
AppCompatConfig and GmsCompatConfig apps are done the same way. These are built
and signed with the standard gradle Android plugin build system.
The TalkBack screen reader is built from our talkback fork repository and included as a prebuilt signed with the OS releasekey. It may be presigned in the future similar to our standalone apps if we decide to do out-of-band releases.
GrapheneOS uses a static web server as the update server. The
release signing script generates the necessary metadata alongside the release
files. You simply need to host these files at the URL configured in packages/apps/Updater/res/values/config.xml. See above for details on including the Updater app in a release.
These are the relevant files:
DEVICE-ota_update-BUILD_NUMBER.zip
DEVICE-factory-BUILD_NUMBER.zip
DEVICE-factory-BUILD_NUMBER.zip.sig
DEVICE-testing
DEVICE-beta
DEVICE-stable
Generally, you should start by uploading the ota_update, factory images and testing channel metadata.
The testing release channel is an example of an internal release channel not
configurable via the update client GUI. Internal release channels can have arbitrary
names. You can override the release channel configured in the update client via
ADB with the following command:
adb shell setprop sys.update.channel CHANNEL_NAME
Replace CHANNEL_NAME with the
name of the release channel, such as testing.
After pushing out and testing the new release via the internal release channel, it's recommended to build a sample future release and push that out as another update via an internal testing channel. This is important to test that the changes in your latest release have not broken the future upgrade path.
Finally, once the release has gone through internal testing, upload the metadata for the beta channel. Once the release has gone through beta testing, upload the metadata for the stable channel.
Delta update packages should simply be uploaded alongside the rest of the releases. The update client will check for the presence of a delta update from the current version on the device to the newer release in the selected release channel. There is no additional metadata to include alongside the delta update package.
Manifests for stable releases are generated with repo manifest -r
after tagging the release across all the repositories in a temporary branch and
syncing to it. This provides a manifest referencing the commits by hashes
instead of just tags to lock in the revisions. This makes verification of the
releases simpler, since only the manifest tag needs to be verified rather than
tags for each repository. This also means the whole release can be verified
using the GrapheneOS signing key despite referencing many upstream repositories
that are not forked by the GrapheneOS project.
It can be useful to set up a standalone installation of the SDK separate from the Android Open Source Project tree. This is how the prebuilt apps are built, rather than using the older branch of the SDK in the OS source tree.
Android Studio can also be set up to use an existing SDK and will recognize it and use it automatically if Android Studio is installed with an SDK installation already available and set up in the environment. You'll also likely want a working command-line SDK environment even if you do heavily use Android Studio.
Using the official releases of the SDK is recommended for
simplicity, although with a lot of effort you can build everything yourself.
Distribution packages are generally quite out-of-date and should be avoided. To
set up a minimal SDK installation at ~/android/sdk without
Android Studio:
mkdir -p ~/android/sdk/bootstrapcd ~/android/sdk/bootstrapcurl -O https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zipecho '2d2d50857e4eb553af5a6dc3ad507a17adf43d115264b1afc116f95c92e5e258 commandlinetools-linux-11076708_latest.zip' | sha256sum -cunzip commandlinetools-linux-11076708_latest.zipcmdline-tools/bin/sdkmanager 'cmdline-tools;latest' --sdk_root=$HOME/android/sdkcd ..rm -r bootstrap
Set ANDROID_HOME to point at the SDK installation in your current shell and shell
profile configuration. You also need to add the cmdline-tools binaries to
your PATH. For example:
export ANDROID_HOME="$HOME/android/sdk"export PATH="$HOME/android/sdk/cmdline-tools/latest/bin:$PATH"
Install platform-tools for tools like adb and fastboot:
sdkmanager platform-tools
Add the platform-tools executables to your PATH:
export PATH="$HOME/android/sdk/platform-tools:$PATH"
For running the Compatibility Test Suite you'll also need the build-tools for aapt:
sdkmanager 'build-tools;35.0.0'
Add the build-tools executables to your PATH:
export PATH="$HOME/android/sdk/build-tools/35.0.0:$PATH"
For working with native code, you need the NDK:
sdkmanager 'ndk;27.2.12479018'
Add the ndk-bundle executables to your PATH:
export PATH="$HOME/android/sdk/ndk/27.2.12479018:$PATH"
You should update the sdk before use from this point onwards:
sdkmanager --update
You can install Android Studio alongside the standalone SDK
and it will detect it via the ANDROID_HOME environment variable rather than installing another copy of it. For
example:
cd ~/androidcurl -O https://dl.google.com/dl/android/studio/ide-zips/2024.2.2.13/android-studio-2024.2.2.13-linux.tar.gzecho 'b7fe1ed4a7959bdaca7a8fd57461dbbf9a205eb23cc218ed828ed88e8b998cb5 android-studio-2024.2.2.13-linux.tar.gz' | sha256sum -ctar xvf android-studio-2024.2.2.13-linux.tar.gzrm android-studio-2024.2.2.13-linux.tar.gzmv android-studio studio
Add the Android Studio executables to your PATH:
export PATH="$PATH:$HOME/android/studio/bin"
You can start it with studio.sh.
Obtain the latest Gradle version and binary-only zip checksum from the Gradle releases page.
Use the Gradle wrapper script to update Gradle to the latest version. You should run this twice rather than only once to upgrade the Gradle wrapper with the new Gradle version. For example, with Gradle 8.10.2:
./gradlew wrapper --gradle-version=8.10.2 --gradle-distribution-sha256-sum=31c55713e40233a8303827ceb42ca48a47267a0ad4bab9177123121e71524c26./gradlew wrapper --gradle-version=8.10.2 --gradle-distribution-sha256-sum=31c55713e40233a8303827ceb42ca48a47267a0ad4bab9177123121e71524c26
This section will be expanded to cover various test suites and testing procedures rather than only the current very minimal coverage of the Compatibility Test Suite (CTS).
To test a build for the emulator, run emulator within
the build environment. The emulator will use CPU hardware acceleration via KVM
along with optional graphics acceleration via the host GPU if these are
available. The emulator does not work on Wayland due to the AOSP provided Qt
build missing the Wayland platform plugin.
Testing with the Compatibility Test Suite (CTS) can be done by either building the test suite from source or using the official releases.
Official releases of the CTS can be downloaded from the Compatibility Suite Downloads page. You should download the CTS for the relevant release (Android 12) and architecture (ARM). There's a separate zip for the main CTS, the manual portion (CTS Verifier) and the CTS for Instant Apps. The latest release of the CTS Media Files also needs to be downloaded from that section.
mkdir -p ~/android/cts/{arm,x86}cd ~/android/cts/armcurl -O https://dl.google.com/dl/android/cts/android-cts-12_r1-linux_x86-arm.zipunzip android-cts-12_r1-linux_x86-arm.ziprm android-cts-12_r1-linux_x86-arm.zipcurl -O https://dl.google.com/dl/android/cts/android-cts-verifier-12_r1-linux_x86-arm.zipunzip android-cts-verifier-12_r1-linux_x86-arm.ziprm android-cts-verifier-12_r1-linux_x86-arm.zipcd ~/android/cts/x86curl -O https://dl.google.com/dl/android/cts/android-cts-verifier-12_r1-linux_x86-x86.zipunzip android-cts-verifier-12_r1-linux_x86-x86.ziprm android-cts-verifier-12_r1-linux_x86-x86.zipcurl -O https://dl.google.com/dl/android/cts/android-cts-12_r1-linux_x86-x86.zipunzip android-cts-12_r1-linux_x86-x86.ziprm android-cts-12_r1-linux_x86-x86.zipcd ~/android/ctscurl -O https://dl.google.com/dl/android/cts/android-cts-media-1.5.zipunzip android-cts-media-1.5.ziprm android-cts-media-1.5.zip
You'll need a device attached to your computer with ADB enabled along with the Android SDK installed. The build-tools and platform-tools packages need to be installed and the binaries need to be added to your PATH. See the standalone SDK installation instructions above.
Copy media onto the device:
cd android-cts-media-1.5./copy_images.sh./copy_media.sh
You also need to do some basic setup for the device. It's possible for changes from a baseline install to cause interference, so it can be a good idea to factory reset the device if assorted changes have been made. The device needs to be running a user build for the security model to be fully intact in order to pass all the security tests. A userdebug build is expected to fail some of the tests. GrapheneOS also makes various changes intentionally deviating from the requirements expected by the CTS, so there will always be some expected failures. A few of the tests are also known to be quite flaky or broken even with the stock OS and/or AOSP. These will be documented here at some point.
Run the test harness:
./android-cts/tools/cts-tradefed
Note that _JAVA_OPTIONS being set will break the version detection.
To obtain a list of CTS modules:
list modules
To run a specific module and avoid wasting time capturing device information:
run cts --skip-device-info --module CTS_MODULE_NAME
To speed up initialization after running some initial tests:
run cts --skip-device-info --skip-preconditions --module CTS_MODULE_NAME
It's possible to run the whole standard CTS plan with a single command, but running specific modules is recommended, especially if you don't have everything set up for the entire test suite.
The Android Open Source Project has branches and/or tags for the releases of many different components. There are tags and/or branches for the OS, device kernels, mainline components (APEX), the NDK, Android Studio, the platform-tools distribution packages, the CTS, androidx components, etc. You should obtain the sources via manifests using the repo tool, either using the manifest for a tag / branch in platform/manifest.git or a manifest provided elsewhere. Different projects use different subsets of the repositories. Many of the repositories only exist as an archive for older releases and aren't referenced in current manifests.
Some components don't have the infrastructure set up to
generate and push their own branches and tags to AOSP. In other cases, it's
simply not obvious to an outsider which one should be used. As long as the
component is built on the standard Android project CI infrastructure, it's
possible to obtain the manifests to build it based on the build number, which
is generally incorporated into the build. For example, even without a
platform-tools tag, you can obtain the build number from adb version or fastboot version.
Their version output uses the format VERSION-BUILD_NUMBER such as 30.0.3-6597393 for the version 30.0.3 where the official release had the build number 6597393. You can
obtain the manifest properties with the appropriate repository revisions from
ci.android.com with a URL like this: https://ci.android.com/builds/submitted/6597393/sdk/latest/view/repo.prop
The platform-tools tags exist because the GrapheneOS project requested them. The same could be done for other projects, but it's not strictly necessarily as long as it's possible to obtain the build number to request the information from the Android project CI server.
As another kind of example, prebuilts/clang, prebuilts/build-tools, etc. have a manifest file committed alongside the prebuilts. Other
AOSP toolchain prebuilts reference a build number.
The following programming languages are acceptable for completely new GrapheneOS projects:
no_std for low-level
code used in a hypervisor, kernel, daemon, system library, etc. Keep in
mind that low-level code is to be avoided whenever a higher-level language
is better suited to the job. In general, the project aims to avoid
creating more low-level code manually dealing with memory ownership and
lifetimes in the first place.Much of the work is done on existing projects, and the existing languages should be used unless there are already clear stable API boundaries where a different language could be used without causing a substantial maintenance burden. The following languages are typical from most to least common: Java, C++, C, JavaScript, arm64 assembly, POSIX shell, Bash.
For existing projects, use the official upstream code style. Avoid using legacy conventions that they're moving away from themselves. Follow the code style they use for new additions. Some projects have different code styles for different directories or files depending on their sources, in which case respect the per-file style.
For new projects, follow the official code style for the
language. Treat the standard library APIs as defining the naming style for
usage of the language, i.e. C uses variable_or_function_name, type_name, MACRO_NAME while
JavaScript uses variable_or_function_name, ClassName and CONSTANT_NAME. For Python, follow PEP8 and the same goes for other languages with
official styles whether defined in a document or by the default mode for the
official formatting tool like rustfmt.
For cases where there isn't an official or prevailing code
style for other things, avoid tabs, use 4-space indents, function_name, variable_name, TypeName and CONSTANT_NAME.
Prefer single-line comment syntax other than rare cases where it makes sense to
add a tiny comment within a line of code. In languages with the optional braces
misfeature (C, C++, Java), always use them. Open braces on the same line as
function definitions / statements. Wrap lines at 100 columns except in rare
cases where it would be far uglier to wrap the line.
For JavaScript, all code should be contained within ES6
modules. This means every script element should use type="module".
Modules provide proper namespacing with explicit imports and exports. Modules
automatically use strict mode, so "use strict"; is no longer
needed. By default, modules are also deferred until after the DOM is ready,
i.e. they have an implicit defer attribute. This should be relied upon rather than unnecessarily
listening for an event to determine if the DOM is ready for use. It can make
sense to use async to run the code earlier if the JavaScript is essential to the content
and benefits from being able to start tasks before the DOM is ready, such as
retrieving important content or checking if there's a login session. Always end
lines with semicolons (since automatic insertion is poorly designed) and always
use const to declare variables, unless they are reassigned in which case they
should be declared with let but never use var as it is effectively broken. Try to prefer loops with for..of.
JavaScript must pass verification with eslint using the following
eslint.config.js configuration:
import js from "@eslint/js";import globals from "globals";import stylistic from "@stylistic/eslint-plugin"; export default [ js.configs.recommended, { plugins: { "@stylistic": stylistic }, languageOptions: { globals: { ...globals.browser }, ecmaVersion: 2022, sourceType: "module" }, rules: { "no-var": ["error"], "@stylistic/indent": ["error", 4], "@stylistic/brace-style": ["error", "1tbs"], "@stylistic/linebreak-style": ["error", "unix"], "@stylistic/quotes": ["error", "double"], "@stylistic/semi": ["error", "always"], } }];
Cookies are only used for login sessions. Every cookie must
have the __Host prefix to guarantee that it has the Secure attribute and Path=/. The HttpOnly and SameSite=Strict
flags should also always be included. These kinds of cookies can provide secure
login sessions in browsers with fully working SameSite=Strict support.
CSRF mitigation should be implemented via enforcing the presence of Sec-Fetch-Site with the value same-origin. Services using only POST can also do this via the more backwards compatible Origin header.
For web content, use dashes as user-facing word separators rather than underscores. Page titles should follow the scheme "Page | Directory | Higher-level directory | Site" for usability with a traditional title as the Open Graph title.
HTML must pass verification with validatornu and xmllint. Ensuring
that it parses as XML with xmllint catches many common mistakes and typos that are missed by HTML
validation due to the ridiculously permissive nature of HTML. This enforces
closing every tag, using proper escaping and so on. XHTML does not really exist
anymore and we simply use XML parsing as an enforced coding standard and lint
pass. It can also be useful to make it compatible with XML-based tooling.
Avoid designing around class inheritance unless it's a rare case where it's an extremely good fit or the language sucks (Java) and it's the least bad approach, but still try to avoid it.
Use concise but self-explanatory variable names. Prefer
communicating information via naming rather than using comments whenever
possible. Don't name variables i, j, k, etc. like C
programmers. It's okay to use things like x and y for parameters
if the function is genuinely that generic and operates on arbitrary values. In
general, try to scope variables into the most limited scope (in C or C++, be
careful about this when references are taken).
Write code that's clean and self-explanatory. Use comments to explain or justify non-obvious things, but try to avoid needing them in the first place. In most cases, they should just be communicating non-local information such as explaining why an invariant is true based on the code elsewhere (consider a runtime check to make sure it's true, or an assertion if performance would be an issue). Docstrings at the top of top-level functions, modules, etc. are a different story and shouldn't be avoided.
Make extensive usage of well designed standard library modules. For apps, treat Jetpack (androidx) as part of the standard library and make good use of it. For Java, Guava can also be treated as part of the standard library.
Libraries outside of the standard library should be used very cautiously. They should be well maintained, stable, well tested and widely used. Libraries implemented with memory unsafe languages should generally be avoided (one exception: SQLite).
Generally, frameworks and libraries existing solely to provide different paradigms and coding patterns are to be avoided. They increase barrier to entry for developers, generally only increase complexity unless used at very large scales (and may not even make things simpler in those cases) and come and go as fads. This is only okay when it's part of the standard libraries or libraries that are considered standard (androidx, Guava) by GrapheneOS and should still be approached cautiously. Only use it if it truly makes the correct approach simpler. Ignore fads and figure out if it actually makes sense to use, otherwise just stick to the old fashioned way if the fancy alternatives aren't genuinely better.