Reverse-Engineering the Compose Compiler Plugin: Intercepting the Frontend
How Jetpack Compose compiler plugin bends the rules of the Kotlin compiler
Have you ever wondered why error messaging appears in your IDE when writing Composables without the Jetpack Compose dependency?
And why a @Composable
annotated function is turned into a Composable
type when you import the necessary Compose dependencies?
This is due to Jetpack Compose compiler plugins, which send diagnostics to the IDE for user feedback. Today, we’ll focus on how these diagnostics are retrieved by the Kotlin compiler, as compiler plugins can intercept code compilation and runtime behavior.
What better way to learn about the Kotlin compiler than doing a little reverse engineering? We’ll need two different “maps”: a map of the Kotlin compiler and a map of compiler plugin architecture.
Note: This may mix up parts of the compiler phases with the Analysis API, which is related but not exactly the same as the compiler pipeline. I’m still working through this, especially with the resolution phase. However, compiler testing offers a good look into how it works, even if the Analysis API concepts are more advanced. Find the Analysis API documentation here.
Further insights on Analysis API or corrections would be greatly welcomed— DM me on the KotlinLang Slack! I expect further explanations and clarifications in the future as my understanding solidifies.
Dec. 19, 2024 Note: Roles performed at different phases for the Compose compiler plugin are subject to changes. The functionality of the compiler may also change over time.
The Kotlin compiler
Discussions on compiler plugins are best understood within the context of the compiler itself. It’s dangerous to go alone! It may be helpful to open these resources on the side while navigating this article.
It’s helpful to know the general overall architecture of the Kotlin compiler. For this article, we will deep-dive into the compiler frontend.
The Kotlin compiler frontend has 2 current versions: a newer and older frontend. We will work with the newer frontend, referenced as the FIR frontend (or k2 compiler). Based on my notes, this is my latest rendering of the k2 compiler:
In terms of oversimplification: the compiler scans the code repeatedly to create new data formats and continues to augment itself to create a more complete picture.
In a grosser oversimplification- the compiler repeatedly completes these two actions:
- compiles: compiles data into a new format
- lowers: Sometimes simplifies and optimizes an existing data format. Sometimes, data is added to the existing structure to calculate compiler analysis.
Now that we have a basic, general idea and map of the Kotlin compiler, let’s look at how compiler plugins fit into a compiler. Then, we can lift the hood under the Compose compiler plugin to see what we’re working with.
Compiler Plugin Architecture
One of the most straightforward explanations of compiler plugin architecture is best summarized by Kevin Most in his 2018 talk on Kotlin compiler plugins:
But if you don’t have time to watch a 45-minute talk, I wrote down a quick summary here to help us create a “mental map” of how to navigate our compiler plugin:
Plugin
- Gradle API (totally unrelated to Kotlin)
- Provides an entry point from build.gradle script
- Allows configuration via Gradle extensions
Subplugin
- Interface between Gradle and Kotlin API
- Read Gradle extension options
- Write out Kotlin SubpluginOptions
- Define the compiler plugin's ID (internal unique key)
- Define the Kotlin plugin's Maven coordinates so the compiler can download it
CommandLineProcessor
- Reads kotlinc -Xplugin args
- Subplugin options actually get passed through this pipeline
- Write CompilerConfigurationKeys
ComponentRegistrar
- Read CompilerConfigurationKeys
Extensions
- Hooks in to phases of the compiler to intercept default behavior i.e.
- ExpressionCodegenExpression
- ClassBuilderInterceptorExtension
- StorageComponentContainerContribtor
- IrGenerationExtension
Using these “maps” of a compiler plugin architecture and the Kotlin compiler, we are ready to venture into the Compose compiler plugin code.
Lifting the hood the Compose compiler plugin
The Compose compiler plugin has moved from Androidx over to the Kotlin repository. If we open up the file system and drill down to the source code, we can see the following files and folders:
The quickest way to figure out where we should break down these parts into frontend/backend responsibilities.
/analysis
— Despite the confusing naming, this kind of analysis is reserved for the backend, since analysis is done on IR data. Classes stored in this folder appear to determine stability for Compose performance. Documentation of this feature is found here./inference
— Used for both frontend and backend, related to Applier inferencing. Remember, an Applier in Compose applies tree-based operations that emit during a Composition. ApplierInference can be used with frontend AST or backend IR. This package also contains Schemes and Bindings/CallBindings, which help to create rules around how to bind tokens together./k1
— classes intended to intercept the original K1 frontend compiler/k2
— classes intended to intercept the new K2 (FIR) frontend compiler/lower
— backend portion to lower IR data, not covered in this article
We also have a list of classes located at the top level:
BuildMetrics.kt
— Metrics and logging for Compose plugin reportsComposeFqNames.kt
— Fully qualified names for Composable classes along with some extension IR functionsComposeIrGenerationExtension.kt
— Intercepts IR generation in the backend for Compose-specific behaviors.ComposeNames.kt
— A dictionary of names for checking IR values in the backend.ComposePlugin.kt
— the entry point for the Compose compiler plugin. Check your compose compiler architecture map with this, you may be able to line a few things up here.WeakBindingTrace.kt
— According to class documentation: “This class is meant to have the shape of a BindingTrace object that could exist and flow through the Psi2Ir -> Ir phase, but doesn’t currently exist”.PSI → IR
transformation implies this class relates to K1 compiler functions.
What does the Compose compiler plugin intercept?
Like other compiler plugins, the Jetpack Compose plugin can and does intercept compilation behavior both frontend and backend of the Kotlin compiler(s).
If we look at the main entry class ComposePlugin.kt
, and scroll down to the registered extension functions where the ComposePluginRegistrar
is located, we see the following
override fun ExtensionStorage.registerExtensions(configuration: CompilerConfiguration) {
if (checkCompilerConfiguration(configuration)) {
val usesK2 = configuration.languageVersionSettings.languageVersion.usesK2
val descriptorSerializerContext =
if (usesK2) null
else ComposeDescriptorSerializerContext()
registerCommonExtensions(descriptorSerializerContext) <---
IrGenerationExtension.registerExtension(
createComposeIrExtension(
configuration,
descriptorSerializerContext
)
)
if (!usesK2) {
registerNativeExtensions(descriptorSerializerContext!!)
}
}
}
Remember, we’re interested in looking at the frontend. IR implies the backend, so if we scroll down to registerCommonExtensions(...)
, we see the following code:
fun ExtensionStorage.registerCommonExtensions(
composeDescriptorSerializerContext: ComposeDescriptorSerializerContext? = null,
) {
StorageComponentContainerContributor.registerExtension(
ComposableCallChecker()
)
StorageComponentContainerContributor.registerExtension(
ComposableDeclarationChecker()
)
StorageComponentContainerContributor.registerExtension(
ComposableTargetChecker()
)
DiagnosticSuppressor.registerExtension(ComposeDiagnosticSuppressor())
@Suppress("OPT_IN_USAGE_ERROR")
TypeResolutionInterceptor.registerExtension(
ComposeTypeResolutionInterceptorExtension()
)
DescriptorSerializerPlugin.registerExtension(
ClassStabilityFieldSerializationPlugin(
composeDescriptorSerializerContext?.classStabilityInferredCollection
)
)
FirExtensionRegistrarAdapter.registerExtension(ComposeFirExtensionRegistrar())
}
The following classes have been registered extensions as common extensions in the frontend. Checking what these classes extend can give us some hints on where in the compiling process these steps may reside:
ComposableCallChecker.kt
— extends FirExpressionChecker in packageorg.jetbrains.kotlin.fir.analysis.checkers.expression
ComposableDeclarationChecker.kt
— extends DeclarationChecker in packageorg.jetbrains.kotlin.resolve.checkers
ComposableTargetChecker.kt
— extends CallChecker in packageorg.jetbrains.kotlin.resolve.calls.checkers
ComposableDiagnosticSupressor.kt
—ComposeFirExtensions.kt
— Intended for K2 serialization formats needed to turn a KComposable type for FIR data format type.
Conveniently, we can spot these particular classes in both the k1 and k2 packages. I’m assuming we still use some of the registered extensions found in k1 for k2.
Now that we have reviewed the contents of the Compose compiler plugin itself, we can use the compiler plugin to figure out what intercepts where when running through portions of the Compiler phases.
Remember, compiler plugins can hook into any part of the Kotlin compiler. If we run code through the compiler which configures the Gradle plugin, we can simultaneously see:
- How the code moves through the compiler as it compiles and lowers the data
- How the Compose compiler plugin intercepts and extends certain phases of the Kotlin compiler to modify compiling behavior
Let us revisit our original question. Suppose we write the following code in our IDE, and we have configured the Compose dependency into our project:
@Composable fun HelloWorld() { Text("Hello!") }
Let’s trace this code mentally through our map of the Kotlin compiler. I did this through Kotlin compiler FIR testing. We leave debugging points in the intercepting compiler plugin classes and trace back in the call stack to where the original phases lie in the compiler. A forensics expedition, if you will.
Parsing Phase
First, your source code is broken down into lexical tokens and turned into PSI trees after.
During Lexer Analysis, the Compose plugin adds to the definition of tokens telling the compiler to “look out for these additional keywords” like @Composable
. These tokens are broken down and then assembled in PSI trees.
PSI tells us what code has been written by the end user, and whether the syntax is grammatically correct: but it cannot tell whether the code compiles, what tokens are types, what tokens are arguments, etc.
Codegen PSI → FIR
In the K2 compiler, code generation takes the PSI trees to create RawFIR. FIR stands for Frontend Intermediate Representation: the initial version created is the raw version.
The PSITree created from
@Composable fun HelloWorld() { Text("Hello!") }
now becomes a different data format:
RAW_FIR:
FILE: [ResolvedTo(RAW_FIR)] typeParameterOfClass2.kt
@Composable[ResolvedTo(RAW_FIR)]() public? final? [ResolvedTo(RAW_FIR)] fun HelloWorld(): R|kotlin/Unit| { LAZY_BLOCK }
Remember how I mentioned that the compiler will scan itself repeatedly to keep building on itself? The same is also true for RawFIR in order for it to become FIR.
When the RawFIR runs through the resolution phase, the analysis it creates augments the existing structure with semantic information to create a more complete picture.
Resolution Phase
In resolution, the heaviest work lies in creating relationships for these FIR elements: static analysis, type verification, diagnostics, IDE warning messages, etc. The Compose plugin performs some of its heavier function interceptions here, particularly in diagnostics (checkers).
The first phase in resolution is semantic analysis on the RawFIR to generate additional relational data. RawFIR is augmented with this information, and then analysis is conducted for checks for verification and type-checking and more.
ComposableTypeResolutionInterceptorExtension
intercepts type resolution to infer Composable types by examining PSI and their descriptors and adding on to the descriptors. If lambda is marked as a @Composable
, then the inferred type for the literal function should become a Kotlin @Composable
type.
import androidx.compose.runtime.Composable
val lambda: @Composable() -> Unit? = { }
@Composable
fun HelloWorld(content: @Composable () -> Unit) {
content
}
This phase also generates descriptors, containing useful information like type resolution, modifiers, and control flow. Descriptors can be used for more complicated evaluations that require more information about their relationships (other files, imports, declarations, diagnostics, etc. later on while checking compiler findings). With the augmented information created during resolution, previously generated RawFIR:
RAW_FIR:
FILE: [ResolvedTo(RAW_FIR)] typeParameterOfClass2.kt
@Composable[ResolvedTo(RAW_FIR)]() public? final? [ResolvedTo(RAW_FIR)] fun HelloWorld(): R|kotlin/Unit| { LAZY_BLOCK }
Now becomes:
FUN name:HelloWorld visibility:public modality:FINAL <> ($composer:androidx.compose.runtime.Composer?, $changed:kotlin.Int) returnType:kotlin.Unit
annotations:
Composable
ComposableTarget(applier = "androidx.compose.ui.UiComposable")
VALUE_PARAMETER name:$composer index:0 type:androidx.compose.runtime.Composer? [assignable]
VALUE_PARAMETER name:$changed index:1 type:kotlin.Int
BLOCK_BODY
BLOCK type=kotlin.Unit origin=null
SET_VAR '$composer: androidx.compose.runtime.Composer? [assignable] declared in home.HelloWorld' type=kotlin.Unit origin=null
CALL 'public abstract fun startRestartGroup (key: kotlin.Int): androidx.compose.runtime.Composer declared in androidx.compose.runtime.Composer' type=androidx.compose.runtime.Composer origin=null
$this: GET_VAR '$composer: androidx.compose.runtime.Composer? [assignable] declared in home.HelloWorld' type=androidx.compose.runtime.Composer? origin=null
key: CONST Int type=kotlin.Int value=-4391552
CALL 'public final fun sourceInformation (composer: androidx.compose.runtime.Composer, sourceInformation: kotlin.String): kotlin.Unit declared in androidx.compose.runtime' type=kotlin.Unit origin=null
composer: GET_VAR '$composer: androidx.compose.runtime.Composer? [assignable] declared in home.HelloWorld' type=androidx.compose.runtime.Composer? origin=null
sourceInformation: CONST String type=kotlin.String value="C(HelloWorld)105@511L19:main.kt#1wrmn"
WHEN type=kotlin.Unit origin=IF
BRANCH
if: WHEN type=kotlin.Boolean origin=OROR
BRANCH
if: CALL 'public final fun not (): kotlin.Boolean [operator] declared in kotlin.Boolean' type=kotlin.Boolean origin=null
$this: CALL 'public final fun EQEQ (arg0: kotlin.Any?, arg1: kotlin.Any?): kotlin.Boolean declared in kotlin.internal.ir' type=kotlin.Boolean origin=null
arg0: GET_VAR '$changed: kotlin.Int declared in home.HelloWorld' type=kotlin.Int origin=null
arg1: CONST Int type=kotlin.Int value=0
then: CONST Boolean type=kotlin.Boolean value=true
BRANCH
if: CONST Boolean type=kotlin.Boolean value=true
then: CALL 'public final fun not (): kotlin.Boolean [operator] declared in kotlin.Boolean' type=kotlin.Boolean origin=null
$this: CALL 'public abstract fun <get-skipping> (): kotlin.Boolean declared in androidx.compose.runtime.Composer' type=kotlin.Boolean origin=null
$this: GET_VAR '$composer: androidx.compose.runtime.Composer? [assignable] declared in home.HelloWorld' type=androidx.compose.runtime.Composer? origin=null
then: BLOCK type=kotlin.Unit origin=null
WHEN type=kotlin.Unit origin=IF
BRANCH
if: CALL 'public final fun isTraceInProgress (): kotlin.Boolean declared in androidx.compose.runtime' type=kotlin.Boolean origin=null
then: CALL 'public final fun traceEventStart (key: kotlin.Int, dirty1: kotlin.Int, dirty2: kotlin.Int, info: kotlin.String): kotlin.Unit declared in androidx.compose.runtime' type=kotlin.Unit origin=null
key: CONST Int type=kotlin.Int value=-4391552
dirty1: GET_VAR '$changed: kotlin.Int declared in home.HelloWorld' type=kotlin.Int origin=null
dirty2: CONST Int type=kotlin.Int value=-1
info: CONST String type=kotlin.String value="home.HelloWorld (main.kt:104)"
CALL 'public final fun BasicText (text: kotlin.String, modifier: androidx.compose.ui.Modifier?, style: androidx.compose.ui.text.TextStyle?, onTextLayout: kotlin.Function1<androidx.compose.ui.text.TextLayoutResult, kotlin.Unit>?, overflow: androidx.compose.ui.text.style.TextOverflow, softWrap: kotlin.Boolean, maxLines: kotlin.Int, minLines: kotlin.Int, color: androidx.compose.ui.graphics.ColorProducer?, $composer: androidx.compose.runtime.Composer?, $changed: kotlin.Int, $default: kotlin.Int): kotlin.Unit declared in androidx.compose.foundation.text' type=kotlin.Unit origin=null
text: CONST String type=kotlin.String value="Hello!"
modifier: COMPOSITE type=kotlin.Nothing? origin=DEFAULT_VALUE
CONST Null type=kotlin.Nothing? value=null
style: COMPOSITE type=kotlin.Nothing? origin=DEFAULT_VALUE
CONST Null type=kotlin.Nothing? value=null
onTextLayout: COMPOSITE type=kotlin.Nothing? origin=DEFAULT_VALUE
CONST Null type=kotlin.Nothing? value=null
overflow: COMPOSITE type=androidx.compose.ui.text.style.TextOverflow origin=DEFAULT_VALUE
CALL 'public final fun <unsafe-coerce> <T, R> (v: T of kotlin.jvm.internal.<unsafe-coerce>): R of kotlin.jvm.internal.<unsafe-coerce> declared in kotlin.jvm.internal' type=androidx.compose.ui.text.style.TextOverflow origin=null
<T>: kotlin.Int
<R>: androidx.compose.ui.text.style.TextOverflow
v: CONST Int type=kotlin.Int value=0
softWrap: COMPOSITE type=kotlin.Boolean origin=DEFAULT_VALUE
CONST Boolean type=kotlin.Boolean value=false
maxLines: COMPOSITE type=kotlin.Int origin=DEFAULT_VALUE
CONST Int type=kotlin.Int value=0
minLines: COMPOSITE type=kotlin.Int origin=DEFAULT_VALUE
CONST Int type=kotlin.Int value=0
color: COMPOSITE type=kotlin.Nothing? origin=DEFAULT_VALUE
CONST Null type=kotlin.Nothing? value=null
$composer: GET_VAR '$composer: androidx.compose.runtime.Composer? [assignable] declared in home.HelloWorld' type=androidx.compose.runtime.Composer? origin=null
$changed: CONST Int type=kotlin.Int value=6
$default: CONST Int type=kotlin.Int value=510
WHEN type=kotlin.Unit origin=IF
BRANCH
if: CALL 'public final fun isTraceInProgress (): kotlin.Boolean declared in androidx.compose.runtime' type=kotlin.Boolean origin=null
then: CALL 'public final fun traceEventEnd (): kotlin.Unit declared in androidx.compose.runtime' type=kotlin.Unit origin=null
BRANCH
if: CONST Boolean type=kotlin.Boolean value=true
then: CALL 'public abstract fun skipToGroupEnd (): kotlin.Unit declared in androidx.compose.runtime.Composer' type=kotlin.Unit origin=null
$this: GET_VAR '$composer: androidx.compose.runtime.Composer? [assignable] declared in home.HelloWorld' type=androidx.compose.runtime.Composer? origin=null
BLOCK type=kotlin.Unit origin=null
BLOCK type=kotlin.Unit origin=SAFE_CALL
VAR IR_TEMPORARY_VARIABLE name:tmp_0 type:androidx.compose.runtime.ScopeUpdateScope? [val]
CALL 'public abstract fun endRestartGroup (): androidx.compose.runtime.ScopeUpdateScope? declared in androidx.compose.runtime.Composer' type=androidx.compose.runtime.ScopeUpdateScope? origin=null
$this: GET_VAR '$composer: androidx.compose.runtime.Composer? [assignable] declared in home.HelloWorld' type=androidx.compose.runtime.Composer? origin=null
WHEN type=kotlin.Unit origin=IF
BRANCH
if: CALL 'public final fun EQEQ (arg0: kotlin.Any?, arg1: kotlin.Any?): kotlin.Boolean declared in kotlin.internal.ir' type=kotlin.Boolean origin=null
arg0: GET_VAR 'val tmp_0: androidx.compose.runtime.ScopeUpdateScope? [val] declared in home.HelloWorld' type=androidx.compose.runtime.ScopeUpdateScope? origin=null
arg1: CONST Null type=kotlin.Any? value=null
then: CONST Null type=kotlin.Any? value=null
BRANCH
if: CONST Boolean type=kotlin.Boolean value=true
then: CALL 'public abstract fun updateScope (block: kotlin.Function2<androidx.compose.runtime.Composer, kotlin.Int, kotlin.Unit>): kotlin.Unit declared in androidx.compose.runtime.ScopeUpdateScope' type=kotlin.Unit origin=null
$this: GET_VAR 'val tmp_0: androidx.compose.runtime.ScopeUpdateScope? [val] declared in home.HelloWorld' type=androidx.compose.runtime.ScopeUpdateScope? origin=null
block: FUN_EXPR type=kotlin.Function2<androidx.compose.runtime.Composer?, kotlin.Int, kotlin.Unit> origin=LAMBDA
FUN LOCAL_FUNCTION_FOR_LAMBDA name:<anonymous> visibility:local modality:FINAL <> ($composer:androidx.compose.runtime.Composer?, $force:kotlin.Int) returnType:kotlin.Unit
VALUE_PARAMETER name:$composer index:0 type:androidx.compose.runtime.Composer?
VALUE_PARAMETER name:$force index:1 type:kotlin.Int
BLOCK_BODY
RETURN type=kotlin.Nothing from='local final fun <anonymous> ($composer: androidx.compose.runtime.Composer?, $force: kotlin.Int): kotlin.Unit declared in home.HelloWorld'
CALL 'public final fun HelloWorld ($composer: androidx.compose.runtime.Composer?, $changed: kotlin.Int): kotlin.Unit declared in home' type=kotlin.Unit origin=null
$composer: GET_VAR '$composer: androidx.compose.runtime.Composer? declared in home.HelloWorld.<anonymous>' type=androidx.compose.runtime.Composer? origin=null
$changed: CALL 'internal final fun updateChangedFlags (flags: kotlin.Int): kotlin.Int declared in androidx.compose.runtime' type=kotlin.Int origin=null
flags: CALL 'public final fun or (other: kotlin.Int): kotlin.Int [infix] declared in kotlin.Int' type=kotlin.Int origin=null
$this: GET_VAR '$changed: kotlin.Int declared in home.HelloWorld' type=kotlin.Int origin=null
other: CONST Int type=kotlin.Int value=1
We now have enough information to infer diagnostic compiler findings. This is where checkers come in. In the Compose compiler plugin, ComposableCallChecker
and ComposableDeclarationChecker
will be added to the “correct syntax definitions” as defined by FIR elements.
ComposableCallChecker
sets the rules for what functions can legally call other Composable
functions. It analyzes the augmented FIR for lambda expression types to validate if a certain call fits the constraints of the contracts defined in Compose. This call checker extension can validate that Composable
types may only be called from other Composable
types.
The resolution phase determines control flow and type refinement, type inferencing, and so on. So for example, resolving can tell us which of the following compiles (in human-readable format):
import androidx.compose.material.Text
import androidx.compose.runtime.Composable// compiles
@Composable fun HelloWorld() { Text("Hello!") }// compiles
val lambdaInvo1 = @Composable { HelloWorld() }// compiles
val lambdaInvo2: @Composable (()->Unit) = { HelloWorld() }
The compiler plugin forces the Kotlin compiler to now ask “Is the lambda invocation marked Composable?”. If not, then the contract defined by the Compose compiler plugin has been violated, and the resolution phase calculates error messages for diagnostics to show.
ComposableDeclarationChecker
takes a similar approach looking at the PSI and its properties to determine proper rules for Compose around top-level declarations, class members, and local declarations within (functions, values, etc). For example, Compose wants to ensure that main
functions are not accidentally marked as a Composable type!
import androidx.compose.runtime.Composable@Composable
fun main(args: Array<String>) { // does not compile!
print(args)
}
After RawFIR runs through the Resolution phase, the FIR is transformed to backend IR, which becomes input for the backend.
Diagnostics after Resolution sent to the IDE
After the Resolution phase, FIR can be sent to the IDE plugin to give you immediate feedback in the IDE, and/or the backend of the Kotlin compiler. IDE plugins are out of the scope for today, as those are different from compiler plugins.
The resulting FIR sends information to the corresponding IDE plugin so that we can see the error in real-time— but the most important thing to understand is that your IDE reacting to the code you write in real time is repetitively running the analysis/resolution to be able to produce these results.
Additionally, it’s worth notingComposeDiagnosticSupressor
adds to the list of diagnostic warnings/errors to bypass certain language restrictions that would otherwise cause compilation to fail in Kotlin: namely, being able to annotate @Composable for higher-order functions to call another composable:
@Composable
fun HelloWorld(content: @Composable (()->Unit)) { // compiles
content
}
Dense article, thanks for sticking with me through it! If you liked this article, let me know. With enough traction, I’ll add a blurb on running compiler FIR tests.
Watch talks directly on this topic:
Droidcon 2022 SF — Hitchiker’s Guide to Compose: Compilers, Composers, and Compiler Plugins: https://www.droidcon.com/2022/06/28/ha-hitchhikers-guide-to-compose-compiler-composers-compiler-plugins-and-snapshots/
Additional resources
- Access additional compiler scratch notes on Patreon
- Jetpack Compose source code — https://github.com/androidx/androidx/tree/androidx-main/compose
- Frontend testing for Compose
- Jetpack Compose Internals by Jorge Castillo — https://jorgecastillo.dev/book/