Go에 이어서 Java로 Dag 작성을..?

choo121600 · 4월 28, 2026, 10:19오전

최근 PR들을 둘러보다가..
Java SDK 작업이 올라와서 봤는데 Go로 Dag를 작성할 수 있게하는 Go SDK 이후로 Java가 진행되는 것 같네요.

github.com/apache/airflow

Add Java SDK (#65956)

main ← astronomer:feature/java-sdk

열림 02:30PM - 27 Apr 26 UTC

jason810496

+7022 -0

# Add Java SDK > **Merge order:** This is PR **1 of 3** — merge this first. …> 1. https://github.com/apache/airflow/pull/65956 (this PR) > 2. https://github.com/apache/airflow/pull/65958 > 3. https://github.com/apache/airflow/pull/65959 > > - **Try it out:** A combined PoC branch with all changes cherry-picked is available at https://github.com/apache/airflow/pull/65960 for reviewers who want to test the full integration end-to-end. ## Why To enable DAGs and tasks to be written entirely in Java (or mixed with Python via `@task.stub`), Airflow needs a Java SDK that speaks the Task SDK protocol — msgpack-framed communication over TCP for DAG parsing and task execution. ## How The Java SDK is a standalone Gradle project (`java-sdk/`) that implements the Airflow Task SDK protocol in Kotlin/Java. It communicates with the Airflow task runner through the same TCP comm/logs channels that the Python SDK uses, producing byte-compatible serialized DAG JSON. ### Architecture The SDK follows a client-server model where the JVM process connects to the coordinator's TCP servers: ``` Airflow Task Runner (Python) Java SDK Subprocess ──────────────────────────── ──────────────────── TCP comm server (127.0.0.1:random) ◄──── CoordinatorComm (msgpack framing) TCP logs server (127.0.0.1:random) ◄──── Logger (structured JSON) ``` - **DAG Parsing**: On `DagFileParseRequest`, the SDK calls `DagBundle.getDags()`, serializes to DagSerialization v3 format, and returns a `DagFileParsingResult`. - **Task Execution**: On `StartupDetails`, the SDK looks up the task, executes `Task.execute(Client)`, and sends `SucceedTask` / `TaskState`. During execution, the task can call `client.getConnection()`, `client.getVariable()`, `client.getXCom()`, `client.setXCom()` — each is a synchronous request/response over the comm channel. ### Serialization Compatibility Both SDKs share test cases in `test_dags.yaml`. A field-by-field comparison (`compare.py`) ensures the Java SDK produces structurally equivalent output to Python's `DagSerialization.serialize_dag()`. ## What - Add `java-sdk/` directory with the full SDK implementation: - **Public API**: `Task` interface, `Client`, `Dag`, `DagBundle`, `Bundle`, `Connection` - **Execution internals** (Kotlin): `CoordinatorComm` (msgpack TCP framing), `Serde` (DagSerialization v3 serializer), `TaskSdkFrames` (protocol message encoding/decoding), `TaskRunner`, `DagParser`, `Server`, `Supervisor`, `Logger` - **Build-time tooling**: `BundleInspector` (generates `airflow-metadata.yaml`), `BundleScanner` (discovers bundles from JAR manifests) - **Configuration**: `Config` (reads `airflow.cfg` for Java bundles folder, execution API URL, etc.) - Add `java-sdk/example/` with a complete `JavaExample` bundle demonstrating pure Java DAG authoring and `@task.stub` interop - Add `java-sdk/dags/stub_dag.py` demonstrating mixed Python + Java DAGs via `@task.stub(sdk="java")` - Add cross-language serialization validation infrastructure (`test_dags.yaml`, `serialize_python.py`, `compare.py`) - Add unit tests for `BundleScanner`, `CoordinatorComm`, `Serde`, `TaskRunner`, `DagParser`, `Config` --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes, with help of Claude Code Ops 4.6 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

https://lists.apache.org/thread/fxxg0tgxd774vd9sl3w61mx69mmlk1gt

메일링 리스트를 보면 이런 논의를 하고 있고요.
저는 java를 사용하지 않아왔어서 그닥 큰 관심은 없습니다만…
이왕 시작되는거 잘됐음 좋겠네요