warcworker by peterk
A dockerized, queued high fidelity web archiver based on Squidwarc
created at July 21, 2018, 8:31 a.m.
webarchive-indexing by ikreymer
Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.
created at March 9, 2015, 8:32 p.m.
One-Click User Instigated Preservationwarclight by archivesunleashed
A Rails engine supporting the discovery of web archives.
created at Aug. 3, 2017, 5:45 p.m.
WarcDB by Florents-Tselai
WarcDB: Web crawl data as SQLite databases.
created at May 29, 2022, 11:09 a.m.
fbarc by justinlittman
A commandline tool and Python library for archiving data from Facebook using the Graph API.
created at Feb. 14, 2017, 11:45 p.m.
zotero-memento by leonkt
Zotero extension that combats link rot by archiving webpages and journal articles.
created at Aug. 29, 2019, 5:51 p.m.
gogetcrawl by karust
Extract web archive data using Wayback Machine and Common Crawl
created at June 14, 2019, 7:02 p.m.
ArchiveTools by recrm
A collection of tools for archiving and analysing the internet.
created at Jan. 14, 2015, 6:53 p.m.
freeze-dry by WebMemex
Snapshots a web page to get it as a static, self-contained HTML document.
created at July 13, 2017, 11:31 p.m.
solrwayback by netarchivesuite
A search interface and wayback machine for the UKWA Solr based warc-indexer framework.
created at Feb. 8, 2017, 9:33 a.m.
ArchiveSpark by helgeho
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
created at Aug. 6, 2015, 7:42 p.m.
webarchive-discovery by ukwa
WARC and ARC indexing and discovery tools.
created at Dec. 20, 2012, 12:17 p.m.
warctools by internetarchive
Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)
created at March 22, 2013, 8:52 p.m.