wail by N0taN3rd

One-Click User Instigated Preservation

updated at May 10, 2024, 3:37 a.m.

13 +0

120 +0

9 +0

html2warc by steffenfritz

simple script to convert web resources to a single warc file

updated at May 8, 2024, 5:21 a.m.

4 +0

15 +0

2 +0

solrwayback by netarchivesuite

A search interface and wayback machine for the UKWA Solr based warc-indexer framework.

updated at May 7, 2024, 6:08 a.m.

24 +0

95 +0

18 +0

py-wasapi-client by unt-libraries

A client for the Archive-It And Webrecorder WASAPI Data Transfer API

updated at May 7, 2024, 3:08 a.m.

5 +0

14 +0

4 +0

ArchiveSpark by helgeho

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

updated at May 5, 2024, 4:14 a.m.

14 +0

141 +0

19 +0

outbackcdx by nla

Web archive index server based on RocksDB

updated at May 4, 2024, 5:05 a.m.

23 +0

29 +0

20 +0

brozzler by internetarchive

brozzler - distributed browser-based web crawler

updated at May 4, 2024, 4:59 a.m.

36 +0

630 +0

93 +0

chronicler by CGamesPlay

Offline-first web browser

updated at May 1, 2024, 4:40 p.m.

6 +0

83 +0

5 +0

cairn by wabarc

NPM package and CLI tool for saving web page as single HTML file

updated at May 1, 2024, 4:40 p.m.

4 +0

37 +0

2 +0

aut by archivesunleashed

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

updated at May 1, 2024, 4:39 p.m.

15 +0

133 +0

33 +0

cc-notebooks by commoncrawl

Various Jupyter notebooks about Common Crawl data

updated at May 1, 2024, 4:06 p.m.

16 +0

40 +0

8 +0

node-warc by N0taN3rd

Parse And Create Web ARChive (WARC) files with node.js

updated at May 1, 2024, 4:04 p.m.

9 +0

92 +0

20 +0

WarcDB by Florents-Tselai

WarcDB: Web crawl data as SQLite databases.

updated at May 1, 2024, 4:03 p.m.

10 +0

384 +0

11 +0

awesome-memento by machawk1

A list of things related to software, literature, and other content for 🕣 Memento

updated at April 27, 2024, 8:55 a.m.

8 +0

77 +0

8 +0

warc2html by iipc

Converts WARC files to static HTML

updated at April 26, 2024, 4:02 p.m.

10 +0

38 +0

3 +0

arch by internetarchive

Web application for distributed compute analysis of Archive-It web archive collections.

updated at April 24, 2024, 8:10 p.m.

19 +0

13 +0

4 +0

Squidwarc by N0taN3rd

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

updated at April 23, 2024, 1:39 a.m.

10 +0

164 +0

26 +0

warcworker by peterk

A dockerized, queued high fidelity web archiver based on Squidwarc

updated at April 23, 2024, 1:39 a.m.

6 +0

53 +0

9 +0

warctools by internetarchive

Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)

updated at April 11, 2024, 9:06 a.m.

36 +0

141 +0

25 +0

Mink by machawk1

Chrome extension that uses Memento to indicate that a page a user is viewing on the live web has an archived copy and to give the user access to the copy

updated at April 10, 2024, 6:35 p.m.

6 +0

45 +0

3 +0