How to rebuild a website from the Wayback Machine

Rebuilding from the Internet Archive is not a restore in the backup sense. There is no database, no admin panel and no media library: there are HTTP responses the crawler happened to request. The work splits into five steps, and most failures happen in the first and the last.

1. Check coverage first

Open web.archive.org and enter the domain. The calendar shows capture dates. Two things matter: the earliest and latest snapshot, and whether captures are spread over the whole site or concentrated on the home page.

A site with a handful of home-page snapshots is not recoverable in any meaningful sense — you will get a landing page, nothing else. A site with hundreds of URLs across categories usually rebuilds well. Say this out loud before starting: it is better to disappoint at minute five than at hour six.

2. Export the address list

The CDX index returns every captured URL as structured text:

https://web.archive.org/cdx/search/cdx?url=example.com&matchType=domain&output=json&collapse=urlkey&fl=timestamp,original,mimetype,statuscode

• matchType=domain — include subdomains;
• collapse=urlkey — one row per address instead of every capture;
• fl — the fields you need: timestamp, original URL, MIME type, status;
• filter=statuscode:200 — drop redirects and error pages.

Save the result as CSV. From here on, everything is driven by that file: how many pages exist, which types, and how long the download will take.

3. Download in id_ mode

Each archived file is addressed by timestamp plus the original URL. Without a modifier the archive returns a rewritten page whose links point back to web.archive.org:

https://web.archive.org/web/20240101120000id_/https://example.com/about/

The id_ modifier returns the raw stored response. That is what you assemble: links stay relative to the original domain, and the copy can be made to work offline later.

Practical constraints you will meet within the first hundred requests:

• Rate limits. Codes 429 and 503 mean slow down, not fail. Backoff and retries are mandatory, not optional.
• Partial captures. Some responses are 200 with an error page inside. Check content length and signatures instead of trusting the status.
• Resume. Record what is already downloaded, otherwise a restart repeats hours of work.
• Concurrency. A fixed high number of parallel requests gets you throttled; adapt to the response codes.

4. Assemble the local copy

Downloaded files are a pile of responses. Three transformations turn them into a site:

1. Paths. Map each URL to a file path. Query strings have to be hashed or dropped deliberately, and directory URLs become index.html.
2. Links. Rewrite absolute URLs to relative paths so the copy resolves locally. Handle percent-encoding and IDN domains explicitly.
3. Encodings. Older pages are often windows-1251 or iso-8859-1; without conversion to UTF-8 the text turns into mojibake after the rewrite.

Missing assets are expected. A stylesheet the crawler never fetched will 404 locally; list it rather than silently replacing it with an empty file — you will want that list when deciding what to rebuild by hand.

5. Verify before publishing

• Serve the copy locally (python -m http.server is enough) and open it.
• Disconnect the network and reload: any request to web.archive.org means the copy is still archive-bound.
• Walk the main navigation and the five most important pages; check images, fonts and scripts.
• Grep the output for web.archive.org — leftovers hide in inline styles and old scripts.
• Check that relative links survive directory depth: ../../ mistakes only show up on inner pages.

Only then publish. If the site goes back to the same URLs, search traffic returns fastest; if the structure changes, add 301 redirects from the old paths and tell search engines to re-crawl.

What will not come back

A rebuilt site is a static snapshot. That is enough to keep content online, keep search traffic and give customers a working reference — it is not a working online store.

Checklist

• ☐ Coverage checked: date range and URL count
• ☐ CDX list exported and deduplicated
• ☐ Downloads taken in id_ mode
• ☐ Retries and backoff on 429/503
• ☐ Paths and links rewritten to relative
• ☐ Encodings converted to UTF-8
• ☐ Missing assets listed, not hidden
• ☐ Copy opens with the network off
• ☐ No web.archive.org references left
• ☐ Redirects and re-crawl arranged

FAQ

Can a whole site be rebuilt from the Wayback Machine?

Only the part the crawler captured. Pages, styles, scripts and some images are usually there; uploads, files behind a login and anything generated by a database are not. The CDX index lists exactly what exists.

How long does it take?

A few hundred pages take about an hour with assembly and link checking. Large sites are limited by the archive's rate limits, not by your connection — expect hours.

Will the rebuilt copy work offline?

Yes, once links point to relative paths and resources are downloaded. If the copy still requests web.archive.org, it was taken without id_ and needs re-downloading.

Do forms, search and checkout work after the rebuild?

No. The archive stores rendered pages, not the application. Anything requiring a server and a database has to be rebuilt separately.

Is it legal to rebuild a site this way?

Rebuilding your own site is fine. Republishing someone else's content is not: the content stays under copyright, and the Internet Archive has its own terms of use.

Related guides

• Hosting lost your data: the first hours — when there is nothing left to assemble from
• Почему картинки не восстанавливаются — why media is usually missing (in Russian)
• CDX API: как получить все страницы — the export in detail (in Russian)

Based on the public capabilities of the Internet Archive and hands-on recovery work. Also available in Russian. Updated 10 November 2026.