ATLAS
Docs Start building

Web intelligence, compiled

Turn the web into
deterministic intelligence.

Traditional scraping returns page-shaped noise. Prompt-based extraction asks a model to infer structure each time. Atlas applies versioned semantic transformations and emits artifacts your software can inspect.

Atlas is infrastructure for systems built on web data. Compilation is the engine beneath discovery, crawling, and batch processing—one deterministic semantic foundation from a single URL to an entire domain.

Start building Inspect the pipeline
  • Compile
  • Map
  • Crawl
  • Batch

Artifact emitted

Source

Public HTML

example.com/docs 
<nav data-kind="presentation">
  Product · Pricing · Sign in
</nav>
<main data-kind="content">
  <h1>Reliable web context</h1>
  <p>Structure software can use.</p>
</main>
<aside>Related · Subscribe</aside>
content
retained
presentation
excluded
Transform

Compiler 1.0

5 stages

Fetch Public target retrieved within explicit bounds.

Parse Reading order reconstructed as semantic nodes.

Analyze Excluded navigation and page chrome identified.

Optimize Normalized hierarchy and repeated noise reduced.

Emit Source provenance retained in the typed artifact.

Artifacts

Two stable forms

ir@1.0
Markdown text/markdown 
# Reliable web context

Structure software can use.
Atlas IR application/json 
{
  "version": "1.0",
  "metadata": {
    "title": "Reliable web context",
    "canonicalUrl": "https://example.com/docs"
  },
  "contentGraph": {
    "readingOrder": ["n_01"],
    "nodes": {
      "n_01": {
        "id": "n_01",
        "type": "heading",
        "parent": "root",
        "children": [],
        "content": "Reliable web context",
        "level": 1,
        "provenance": { "tag": "h1", "selector": "h1" }
      }
    }
  }
}
captured source compiler 1.0 stable semantic artifact

Why deterministic?

Reliability starts before the output.

Developer infrastructure should make its transformations and execution behavior legible. Determinism gives web data a foundation software can reason about.

01 Reproducible

Captured source plus the same compiler version yields a stable semantic artifact.

source + compiler version
02 Inspectable

Typed nodes retain structure, reading order, and provenance back to the source.

nodes + reading order + provenance
03 Versionable

IR and API contracts evolve explicitly instead of changing behind an opaque prompt.

IR version + OpenAPI contract
04 Reliable

Idempotent requests and named job states make execution behavior operable.

request identity + canonical state

The semantic engine

A pipeline you can reason about.

Atlas does not ask a model what a page means. It executes named stages with bounded inputs, ordered transformations, and typed outputs.

  1. 01
    retrieve()

    Retrieve within explicit bounds

    Public targets move through a DNS-aware egress boundary with redirect, time, byte, and content-type limits.

  2. 02
    parse()

    Reconstruct the source

    Framework detection and targeted normalization run before the page becomes a semantic document.

  3. 03
    analyze()

    Understand structure

    Density, repetition, tables, figures, code, metadata, and hierarchy become explicit signals.

  4. 04
    optimize()

    Reduce presentational noise

    Dependency-ordered passes remove boilerplate, resolve links, normalize headings, and preserve meaning.

  5. 05
    emit()

    Emit complementary artifacts

    One semantic document becomes clean Markdown for context and typed Atlas IR for software.

captured source compiler version stable semantic artifact

The platform

One engine. Four operating scopes.

Start with one page, understand a domain, or run bounded work across a known target set. Every scope shares the same semantic model and execution vocabulary.

01

Compile

POST /v1/compile

One public URL

normalize compile emit

Transform a page into complementary human-readable and machine-operable artifacts.

Markdown, typed IR, metadata

02

Map

POST /v1/map

One domain

discover deduplicate classify

Discover useful pages across robots, sitemaps, and HTML before committing crawl budget.

Ranked URL graph

03

Crawl

POST /v1/crawls

Domain + explicit bounds

discover prioritize compile reconcile

Turn a discovered site graph into prioritized, bounded compilation work.

Canonical job + retained artifacts

04

Batch

POST /v1/batches

Explicit URL set

deduplicate prioritize compile reconcile

Process a known set through the same contracts, artifacts, and job lifecycle.

Canonical job + retained artifacts

Different reach. The same semantic model, execution contract, and artifact vocabulary.

Build with Atlas

A stable web layer for any software system.

Atlas supplies the structured artifacts and operating primitives. Your product decides what to build with them.

01

Search and indexing

Index meaning, not page chrome.

Typed nodes · canonical URLs · normalized hierarchy
02

Monitoring and change detection

Compare stable artifacts on your schedule.

Document hashes · versioned IR · retained metadata
03

ETL and data pipelines

Move web content through a predictable schema.

Structured metadata · batch jobs · explicit errors
04

Knowledge bases and RAG

Build context with less presentation noise.

Clean Markdown · semantic structure · provenance
05

Agents and automation

Give software ranked targets and inspectable inputs.

Discovery graph · bounded crawl · typed artifacts

These are systems developers build with Atlas outputs—not separate black-box endpoints.

Developer interfaces

One contract, wherever you build.

Use the public API directly, work through generated clients, inspect operations in the Dashboard and Playground, or react to canonical lifecycle events.

Available interfaces

REST + OpenAPI

OpenAPI 3.1

A workspace-scoped HTTP API described by one reviewed, canonical contract.

JavaScript

Generated client

A generated typed client with transport metadata, errors, and job helpers.

Python

Generated client

Contract-conformant async workflows for application and data infrastructure.

Go

Generated client

Typed operations for services that keep web intelligence close to the backend.

Dashboard

Browser control plane

The control plane for inspecting jobs, usage, webhooks, and workspace resources.

Playground

Interactive workbench

Run Compile, Map, Crawl, and Batch while inspecting rendered output and typed IR.

Webhooks

Event delivery

Workspace-owned endpoints, signed delivery, retries, and observable history.

On the interface roadmap

CLI Planned

Bring repeatable Atlas workflows into local development and CI.

MCP Planned

Expose Atlas operations to compatible agent and tool environments.

JavaScript SDK compile.ts 
import { randomUUID } from 'node:crypto';
import { AtlasClient } from '@atlascompiler/sdk';

const atlas = new AtlasClient({
  workspaceId: process.env.ATLAS_WORKSPACE_ID!,
  apiKey: process.env.ATLAS_API_KEY!,
});

const result = await atlas.compile(
  {
    projectId: process.env.ATLAS_PROJECT_ID!,
    url: 'https://example.com/docs',
  },
  randomUUID(), // idempotency key
);

console.log(result.value);
Workspace scoped · idempotent by request

Inspect the system

Trust is something infrastructure can show.

Atlas exposes the contracts, state, boundaries, and engineering evidence developers need to evaluate the system without relying on marketing proxies.

01

Contracts

Public behavior has a shape you can inspect before integration.

Inspect the API contract Inspect error contracts
  • Typed Atlas IR with reading order and provenance
  • Canonical OpenAPI 3.1 contract
  • Generated JavaScript, Python, and Go clients
  • RFC 9457 problems with stable machine codes
02

Execution

Long-running work remains explicit from admission through retained artifacts.

Read the job lifecycle
  • Idempotency keys for execution and mutation requests
  • Canonical status, result, and error resources
  • Independent opaque cursors with bounded pages
  • Artifact counts, checksums, and manifest reconciliation
03

Security

The public control plane and private execution plane have separate responsibilities.

Review how Atlas works
  • Workspace-scoped authorization at the public API
  • Private Engine and egress service boundaries
  • DNS-aware public-address validation and redirect checks
  • Capability gates fail closed when infrastructure is absent
04

Engineering

Reliability claims are backed by repeatable repository and release evidence.

Explore the source evidence
  • Fixture and snapshot benchmarks used as regression evidence
  • Cloudflare runtime contract tests
  • Protected infrastructure and forward-only migrations
  • Release, backup, restore, load, soak, and canary workflows

Evidence is labelled by what it proves. Repository implementation, environment activation, and measured production behavior remain distinct claims.

The long-term layer

The web becomes dependable infrastructure when software can inspect, version, and operate on its meaning.

Atlas is building that layer: transforming public web content into structured, deterministic intelligence that search systems, data pipelines, automation, agents, and products can reliably build upon.

Start building Read the docs