Technical Appendix
Technical Appendix
Composable Analytics Platform · Technical Appendix
The component inventory, selection criteria, deployment models, and migration method behind the Composable Analytics Platform — written to be checked. Every component links to its own site, documentation, and source repository.
01 · Architecture
The platform is functioning software VisionWrights licenses and operates: established open-source components assembled into one system, with a small set of custom surfaces delivered as source code. It rebuilds a client's full BI environment — pipelines, semantic definitions, dashboards, and product-embedded analytics — at production scale, with every migrated number reconciled against the incumbent platform's output. The reconciliation checks run as tests inside the pipeline itself, on every build (§5).
Three properties drive every design choice, and each is verifiable in a technical review:
Each pipeline step is plain SQL in a dbt model, in a git repository the client's team can open. The operations portal shows the SQL, inputs, outputs, tests, and last-run status of every step — the same view an operator uses in production.
Migrated dashboards are reconciled against the incumbent platform's output using dbt's built-in test framework, applied as best practice from VisionWrights' implementation experience. Each reconciliation is a test that re-runs on every rebuild — a change that breaks reconciliation blocks promotion before it reaches a user.
The BI tools are interchangeable because they all read the same semantic layer; the semantic layer and pipeline are standard open-source components that run wherever the client chooses (§3). The same governed layer serves AI applications — the natural-language builder composes queries from named measures only.
02 · Components & selection
Nine checks gate inclusion; each is verifiable by an outside engineer, and each has a named failure. The criteria gate inclusion without uniquely determining the stack — other components also pass (Apache Airflow, for example, passes the license and governance checks); among passing candidates, selection is an engineering decision.
| Criterion | The rule | What would disqualify a component |
|---|---|---|
| 1 · Readable open license | An OSI-approved license, verifiable in the public repository | Source-available or proprietary licenses |
| 2 · Continuity beyond the vendor | Foundation governance, or a license that lets the community continue the project if the vendor changes course | A license the vendor can close later. This has happened: Terraform (2023), Redis (2024), and Elastic (2021) all relicensed, and the projects survived only because their earlier licenses allowed community forks (OpenTofu, Valkey, OpenSearch) |
| 3 · Named production evidence | Named companies with first-party accounts of running it at scale (see the usage table below) | Marketing logo walls and self-registered user lists with no first-party account behind them |
| 4 · Runs on client-owned infrastructure | Fully self-hostable from public artifacts | Cloud-only products — Domo (per its own platform pages) and Looker core ("hosted by Google… not available for customer-hosted environments" — Google's docs) |
| 5 · Engine & warehouse portability | Adapters or connectors for multiple engines, per the official docs | A hard coupling to a single warehouse |
| 6 · Replaceable through standard interfaces | Speaks SQL, the Postgres wire protocol, open table formats, and git-versioned text | Content locked in formats only the vendor's tools can read (LookML models, Tableau extracts) |
| 7 · Server-enforced multi-tenant security | Tokenized embedding with row-level restriction enforced on the server | Filtering that happens only in the browser, or static embeds with no row security |
| 8 · Operable with standard tooling | Runs in containers with documented upgrade paths | Systems operable only as a vendor appliance, outside standard tooling |
| 9 · Staffable from public knowledge | Deep public documentation and a large practitioner base (industry surveys, first-party community data) | Skills learnable only inside one vendor's ecosystem |
Each row states the component's role, origin, license, and continuity path — the layer-by-layer answer to how the platform behaves over a long horizon. The custom components (widget framework, operations portal, natural-language builder) are VisionWrights' proprietary platform product — built to serve many clients and licensed per client. The embeddable widget is delivered as source into the client's repository; the operations portal and natural-language builder are operated as part of the platform, and their architecture and behavior are walked through with the client's engineers on request. Each is a thin layer over documented APIs.
| Component | Role | Origin & maturity | License | Continuity path |
|---|---|---|---|---|
 OSSdbt Coregetdbt.com·Docs·GitHub·Redshift adapter·Tests | All transformations (SQL models + tests) | 2016; the de-facto industry standard; tens of thousands of production deployments. dbt Labs merged with Fivetran in June 2026; dbt Core remains Apache-2.0, and the new Fusion runtime was released under the same license | Apache 2.0 | The client's SQL, in the client's repo; runs unchanged with native support for Redshift, Snowflake, BigQuery, and Postgres |
 OSSDagsterdagster.io·Docs·GitHub·AWS deployment | Orchestration, run history, alerting | 2018; widely deployed; commercial company behind it | Apache 2.0 | Keeps running as-is, or swaps for Airflow/MWAA — dbt is orchestrator-agnostic |
 OSSPostgreSQLpostgresql.org·License | Warehouse in the current build | 1996; the longest-lived component in the stack | PostgreSQL License | Community-governed since 1996 — or replaced by the client's existing warehouse (§3) |
| OSSApache Parquet / Icebergiceberg.apache.org·Docs·GitHub | Open-format storage path — an alternative to (or stage alongside) the client's warehouse | Iceberg created at Netflix (2017), donated to the Apache Software Foundation; Parquet is the standard open columnar format | Apache 2.0 | Foundation-governed; data stays in open files any engine can read |
 OSSCubecube.dev·Docs·GitHub·Row-level security·Data modeling | Semantic layer: governed measure definitions, row-level security, caching, AI-ready access | 2019; commercial company behind it; large OSS deployment base | Apache 2.0 (core) | Definitions are YAML/SQL in the client's repo; measures portable to dbt metrics or another semantic layer |
 OSSMetabasemetabase.com·GitHub·Licensing (AGPL note)·Embedding | BI option A — analyst-facing dashboards | 2015; very large deployment base | AGPL v3 (OSS edition) — see the licensing note | Keeps running; dashboards exportable |
 OSSApache Supersetsuperset.apache.org·Docs·GitHub·Embedded dashboards | BI option B — analyst-facing + embeddable (Apache-licensed) | 2015; Apache Software Foundation project | Apache 2.0 | Foundation-governed; keeps running |
 OSSLightdashlightdash.com·Docs·GitHub | BI option C — dbt-native exploration | 2021; younger than options A/B; growing base | MIT | Keeps running; a narrow, swappable role by design |
 OSSdltdlthub.com·Docs·GitHub·Redshift destination | Direct source ingestion (extract-load from client systems) | 2022; young, with a deliberately focused scope (extract-load) | Apache 2.0 | Replaceable by Glue, Fivetran, or equivalents — loading is the most commoditized layer |
| CustomEmbeddable widgetSource code delivered to the client — repository access for review on request | Product-embedded dashboards, RLS-scoped; built to be drop-in compatible with existing embed integration patterns | Built by VisionWrights, 2026. Small, plain React/JS codebase over documented APIs. | VisionWrights proprietary; source delivered under license for the client's use, modification, and continued operation | The source lives in the client's repo; it reads standard semantic-layer APIs; sized for an in-house engineer to maintain |
| CustomOperations portalVisionWrights platform product — architecture and behavior walked through with the client's engineers on request | Lineage, per-step SQL, tests, freshness (operator + stakeholder visibility) | Built by VisionWrights, 2026. Renders live pipeline metadata (dbt manifest, Dagster API) — holds no data or logic of its own. | VisionWrights proprietary, operated as part of the platform | A convenience layer over the pipeline's own metadata; the same facts are visible in Dagster and dbt docs directly |
| CustomNatural-language builderVisionWrights platform product — architecture and behavior walked through with the client's engineers on request | Ad-hoc charts from plain English, restricted to governed measures | Built by VisionWrights, 2026. Composes queries only from the semantic layer's named measures; shows the generated queries; when asked for data that has not been modeled and approved, it says so and escalates rather than improvising a number. | VisionWrights proprietary, operated as part of the platform | An optional layer; removing it removes no data capability |
Each selection below is one that larger engineering organizations reached independently; every source is first-party (the company's own case study, engineering blog, or the project's records) and was link-verified in July 2026.
| Component | Run at scale by | Source |
|---|---|---|
| dbt Core | Siemens (enterprise data mesh), JetBlue, Nasdaq, HubSpot | dbt Labs case studies |
| Dagster | US Foods — 99.996% uptime behind a $24B operation | Dagster case study |
| Cube | Alcon and Brex — both building AI analysts grounded in the semantic layer, the same pattern as this platform's conversational applications | Cube case studies |
| Apache Superset | Created at Airbnb; adopted by Dropbox after a published evaluation | Dropbox engineering blog |
| Metabase | 80,000+ organizations (first-party install-base figure, stated as such); the named evidence is N26, Europe's first mobile bank | Metabase case study |
| Apache Iceberg / Parquet | Created at Netflix; Adobe migrated 1PB+ onto it; LinkedIn runs it via OpenHouse | Adobe Tech Blog |
| PostgreSQL | The most-used (55.6%) and most-admired (66%) database among professional developers | Stack Overflow Developer Survey 2025 |
| dlt | 3,000+ companies in production, including PostHog and Flatiron Health — install-base evidence, stated as such | dltHub |
The architecture is as vetted as the parts: the layered ingest → transform → warehouse → semantic-layer pattern is the blueprint documented in a16z's Emerging Architectures for Modern Data Infrastructure; the open-format storage path rests on the peer-reviewed Lakehouse paper (CIDR 2021); and GitLab operates a comparable dbt-based stack entirely in public in its data-team handbook. A full source list, with source-strength grading, is available on request.
03 · Deployment models
The platform subscription covers the assembled software, its operation, upgrades, and support in every model; what varies is where the infrastructure runs and who holds the data.
| A · VisionWrights-hosted | B · Hybrid | C · In the client's cloud | |
|---|---|---|---|
| Where it runs | Isolated single-tenant environment operated by VisionWrights (evaluations typically start here) | Pipeline + warehouse in the client's AWS; VisionWrights operates tooling and deploys into it | Everything in the client's cloud account: dbt against the client's warehouse (e.g. Redshift), Dagster on ECS/MWAA (or the client's orchestrator), BI tools + widgets on the client's infrastructure |
| Who holds the data | VisionWrights environment (contractually bounded) | The client | The client |
| Procurement shape | Single subscription, uptime commitments baked in | Split: client infrastructure + VisionWrights subscription | Client infrastructure; VisionWrights subscription covers build, operation, and support |
| Fits | Fastest start; evaluation and early phases | Transition state | Teams that want full infrastructure ownership |
These are phases as much as options: A is how a team evaluates quickly, C is where the stack was designed to land. dbt's warehouse adapters are first-class, and any prior investment in the client's own warehouse is the destination of this design — the semantic layer and BI tools sit on top of the warehouse the client already owns. Moving A→C is configuration and deployment work; every component in §2 runs identically wherever it is deployed. VisionWrights operates the platform as code in every model: deployment, configuration, and monitoring are programmatic, so the same automation that runs model A runs identically inside the client's account — under a scoped deployment role the client grants and can audit. Hybrid and client-cloud engagements include that role as part of environment setup.
04 · Ownership, layer by layer
The table shows, layer by layer, what lives in the client's own repository and infrastructure on this platform — next to how the same layer behaves in a typical proprietary BI suite.
| Layer | Typical proprietary BI platform | In this platform |
|---|---|---|
| Transformation logic | GUI-defined tiles/flows, exportable only as screenshots | Plain SQL in dbt, in the client's git repo; testable, diffable, reviewable — and dbt runs natively against the client's own warehouse, inside the client's cloud account |
| Warehouse | The vendor's | The client's (model C), or Postgres operated by VisionWrights (models A/B) |
| Measure definitions | Implicit in cards/calculated fields, re-implemented per dashboard | Named, versioned, documented in the semantic layer; one definition serves every tool, with row-level security enforced at the API |
| Dashboards | The vendor's, tied to the vendor's platform | Three interchangeable OSS tools + the VisionWrights widget, source delivered under license |
| Orchestration | The vendor's scheduler | Dagster (OSS) — swappable for MWAA/Airflow |
Because every transformation lands as portable SQL against a warehouse the client owns, the work done on this platform stays valid under any future infrastructure decision — the dbt models, tests, and measure definitions travel with the client.
05 · Migration method
Production surfaces the business relies on are the operating constraint in every migration, and the method is designed around them: five gates, each precise about what is being decided and what is at stake at that stage.
| Gate | What happens | What is at stake at this stage |
|---|---|---|
| 1 · Parallel build | Pipelines rebuilt from the current platform's artifacts as the reference; the incumbent stays untouched and authoritative; users notice nothing | Budget only — everything keeps running exactly as today |
| 2 · Reconciliation, continuously | Each migrated dashboard proven against the incumbent's numbers — row counts, sums, distinct counts, to the unit; the proof is a standard dbt test that re-runs on every build | Failures block promotion; a number that drifts is caught on the next build |
| 3 · Internal trial | New dashboards and embeds mounted on an internal page; the client's teams use them alongside the incumbent and file feedback | Internal feedback only; production surfaces are untouched |
| 4 · Per-surface signoff | Each production surface cuts over only when the client's team signs off on that surface; surfaces move independently | Bounded per surface; rollback is the incumbent embed that never stopped working |
| 5 · Incumbent retirement | Last — after all surfaces and direct-access users are migrated and stable | The savings start when the client says so, in whole or in part |
The speed in this method comes from tooling; the safety comes from the gates above. VisionWrights built a translation engine that parses the machine-readable flow definitions behind GUI-defined ETL tools and emits idiomatic, layered dbt models with their tests — plus a growing registry of flow-pattern translations that improves with every pipeline migrated. Engineer review and the reconciliation harness gate every generated model before it counts. The effect: pipeline rebuilds that take days per dashboard by hand land in hours, at reconciliation-proven fidelity. The generator covers deterministic patterns; genuinely novel shapes are built by hand and added to the registry, so the tooling covers more of each successive pipeline.
Deterministic logic in the incumbent (ETL tiles, dataset views, calculated fields) is extracted and re-expressed as dbt SQL, then verified against the incumbent's own output — treating it as the oracle, so correctness is demonstrated rather than asserted. Snapshot and recursive-accumulation patterns are re-implemented as incremental models or change-log views, validated both ways: the incremental build and a from-scratch rebuild must produce identical results before the model is accepted. The method reconciles every migrated pipeline to the unit — row counts, sums, and distinct counts, including revenue matched to the cent — with every check left running as a permanent test in the client's repository.
06 · Operating & support models
| Dimension | Vendor-operated | Co-owned (typical target) | Client-owned |
|---|---|---|---|
| Pipeline changes & new metrics | VisionWrights builds on request | VisionWrights builds; the client's analysts review PRs and increasingly author dbt models (they are SQL) | The client's team; VisionWrights on retainer for escalation |
| Dashboard iteration | VisionWrights | The client's analysts, directly in the BI tool — the point of the decoupled design | The client's team |
| Ops visibility | Identical in all models: analysts, operators, and engineering get the portal + Dagster run histories, data previews, per-step SQL, and test results — the same instrumentation shown in evaluations | ||
| Cost structure | Platform subscription (the assembled software, its operation, upgrades, and support) + build hours | Client infrastructure at cost + platform subscription | Client infrastructure at cost + optional retainer |
The design supports vendor support and an eventual in-house hire at the same time: because the artifacts are standard (dbt, SQL, git), a future in-house data engineer inherits a documented, tested codebase rather than a vendor relationship — and the co-owned model above is explicitly a glide path toward that hire.