Back to What's New
launch Cloud APIDashboardMCP ServerTypeScript SDKPython SDKGo SDK

Introducing Atlas

A more structured way to work with the web. Today, we're opening Atlas to developers.

The web was built for humans. AI agents and software need a compiler.

Today, we’re opening Atlas to developers.

Atlas is a deterministic semantic compiler and web scraper built for AI and autonomous agents. It transforms bloated, unstructured HTML, single-page applications (SPAs), and complex documentation into clean, machine-readable structured intelligence that LLMs, AI agents, and data pipelines can reliably execute on.

Rather than treating web extraction as a prompt-engineering problem or relying on an LLM to probabilistically guess what a page means, Atlas takes a compiler-first approach: a deterministic pipeline designed to translate unstructured web documents into clean, structured representations.


Why Web Extraction Is Broken

Modern applications are increasingly dependent on live web data, yet the tooling to extract it has remained remarkably fragile:

  1. The LLM Token Tax: Raw web pages are filled with megabytes of layout boilerplate, tracking telemetry scripts, presentational CSS, and navigation menus. Feeding raw HTML directly into LLM context windows wastes context budgets, drives up inference costs, and dilutes model attention, leading to reasoning hallucinations.
  2. Fragile DOM Selectors: Traditional scrapers rely on hardcoded CSS selectors or XPath queries that break the moment an upstream site modifies its layout, class names, or component tree.
  3. Sluggish Headless Browsers: Running heavy containerized headless browsers consumes hundreds of megabytes of RAM per page, introduces excessive latency, and frequently crashes under production workloads.
  4. No Intermediate Representation (IR): Downstream systems need predictable schemas, tabular precision, and clean semantics—not unvalidated string blobs mixed with layout trivia and navigation chrome.

We believe web intelligence infrastructure should be more than a collection of extraction prompts. It should be predictable, programmable, and built to integrate directly into the software developers are already building.


A Compiler-First Architecture

Atlas is built around a simple premise: web extraction should be treated as a compilation problem, not an ad-hoc scraping or prompting exercise.


5 Core Operations

Atlas provides a comprehensive API surface engineered for both interactive developer tools and high-throughput automated agent loops:


Developer Platform & Agent Ecosystem

Atlas is built to integrate seamlessly with the developer tools and AI assistants you use every day:


Start Building in 30 Seconds

You can make your first request to the Atlas API right now with cURL:

curl -X POST https://api.atlas-compiler.com/v1/scrape \
  -H "Authorization: Bearer atlas_your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com"
  }'

The response delivers clean, structured Markdown, rich metadata, and outbound document links without navigation chrome, footers, or tracking beacons:

{
  "data": {
    "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.\n\n[More information...](https://www.iana.org/domains/example)",
    "links": [
      "https://www.iana.org/domains/example"
    ]
  },
  "metadata": {
    "title": "Example Domain",
    "canonicalUrl": "https://example.com/",
    "author": null,
    "description": "Illustrative example domain managed by IANA.",
    "publishedDate": null,
    "extractionConfidence": 0.98,
    "language": "en"
  },
  "usage": {
    "credits": 1
  }
}

Or using the TypeScript SDK:

import { AtlasClient } from '@atlascompiler/sdk';

const client = new AtlasClient({
  apiKey: process.env.ATLAS_API_KEY!,
});

const response = await client.scrape({
  url: 'https://example.com',
});

if (response.status === 200) {
  console.log('Title:', response.value.metadata.title);
  console.log('Markdown:\n', response.value.data.markdown);
  console.log('Credits Used:', response.value.usage.credits);
}

What’s Next

Today’s v1.0.0 public launch establishes the foundation for deterministic web compilation. In upcoming releases, we’re expanding the developer platform and agent ecosystem with:

The web is full of information. We’re building better infrastructure to make it usable.

Welcome to Atlas.

Back to What's New View on GitHub ↗