Spark and Iceberg from your IDE
Spark 4.0.1 over Spark Connect with Iceberg 1.10.1 tables on a REST catalog. Own host and Knox 3.0.0, with VS Code, Kiro, Cursor, Claude Desktop and the MCP Python SDK. The complete developer path.
TUTORIALS
Every tutorial puts Aegis between an IDE agent and one piece of real Apache software on a single machine. Start with the Hello World guide for the shape of the setup, then pick a deep-dive walkthrough or the project you already operate. New reviewers should read the security guide first.
DEEP DIVES
These carry the full developer path: a tiny dataset or workload, an IDE agent, a governed change, and the same run through both front doors where noted.
Spark 4.0.1 over Spark Connect with Iceberg 1.10.1 tables on a REST catalog. Own host and Knox 3.0.0, with VS Code, Kiro, Cursor, Claude Desktop and the MCP Python SDK. The complete developer path.
Spark 4.0.1 monitoring REST API and Spark Connect through a scripted client. Run Spark jobs and SQL through an agent.
The Iceberg 1.10.1 RESTCatalogServer with resource scopes, approvals and destructive-operation receipts over namespaces, properties and drops.
A Kafka 3.9.1 KRaft broker and a Connect worker. Govern topic administration and the connector lifecycle.
Flink 1.20.1 jobs, jars, savepoints, stopping and cancellation, each as a governed action.
Spark, Kafka, Iceberg and Flink as one independent server each, with a signed server catalog and incident containment end to end. Repository walkthrough.
Evidence boundary: tutorials describe retained historical runs. Read the validation report and claims matrix before treating any integration as release-candidate evidence.
ONE PER PROJECT
These are built by tools/build-project-tutorials.py from each project's retained run. All have historical own-host and Knox records, and every kept audit journal verifies. Each tutorial gives the engine command line, both front doors step by step, VS Code, Kiro and Cursor files, prompts mapped to tools with what Aegis answered, and the full request table of that run.
| Project | Version exercised | Tutorial |
|---|---|---|
| Apache ActiveMQ | 6.3.2 (Jolokia) | Open tutorial |
| Apache Airflow | 2.9.3 | Open tutorial |
| Apache BookKeeper | 4.18.1 | Open tutorial |
| Apache Burr (incubating) | 0.42.0 | Open tutorial |
| Apache Calcite over Avatica | 1.42.0 / 1.28.0 | Open tutorial |
| Apache CouchDB | 3.4.2 | Open tutorial |
| Apache Doris | 4.1.3 | Open tutorial |
| Apache Druid | 28.0.1 | Open tutorial |
| Existing MCP server | Python SDK and filesystem server | Open tutorial |
| Apache Flink | 1.20.1 | Open tutorial |
| Apache Flume | 1.11.0 | Open tutorial |
| Apache Hadoop HDFS | 3.2.1 | Open tutorial |
| Apache HBase | REST | Open tutorial |
| Apache Hive | 4.0.1 | Open tutorial |
| Apache Iceberg REST catalog | 1.10.1 | Open tutorial |
| Apache Ignite | 2.18.0 | Open tutorial |
| Apache Kafka and Kafka Connect | 3.9.1 | Open tutorial |
| Apache NiFi | 1.28.1 | Open tutorial |
| Apache Pinot | 1.2.0 | Open tutorial |
| Apache Pulsar | 4.0.13 | Open tutorial |
| Apache Ranger | 2.9.0 | Open tutorial |
| Apache Solr | 9.10.1 | Open tutorial |
| Apache Spark | 4.0.1 | Open tutorial |
| Apache Spark Connect | 4.0.1 | Open tutorial |
| Apache Storm | 2.6.2 | Open tutorial |
| Apache Superset | 4.0.2 | Open tutorial |
| Apache ZooKeeper | 3.9.3 | Open tutorial |
The IDE settings files are checked by DocumentationAssetsTest. The transport was exercised with the official MCP Python SDK in the Spark and Iceberg tutorial; the IDEs themselves were not driven by a test.
NOT YET
Atlas, Cassandra, Impala and Paimon each have a configuration profile that passes the contract test, which proves route rendering and governance order, not that the product answers. None has a real run on record, so none has a tutorial yet.
The status and blocker of each are in the adopter real-engine status and the repository adopter catalog.
REFERENCE MAP