Various examples of notebooks for working with web archives with the Archives Unleashed Toolkit, and derivatives generated by the Archives Unleashed Toolkit.
created at Nov. 6, 2019, 3:09 a.m.
Web application for distributed compute analysis of Archive-It web archive collections.
created at April 28, 2022, 3:18 p.m.
Java application to download WARCs from WASAPI
created at April 28, 2017, 9:15 p.m.
A client for the Archive-It And Webrecorder WASAPI Data Transfer API
created at Aug. 10, 2017, 5:25 p.m.
A Splitable Hadoop InputFormat for Concatenated GZIP Files and *.(w)arc.gz
created at Aug. 8, 2016, 1:36 p.m.
Convert HTTP Archive (HAR) -> Web Archive (WARC) format
created at March 16, 2017, 12:14 a.m.
simple script to convert web resources to a single warc file
created at Dec. 30, 2015, 2:29 p.m.
An open-source toolkit for analyzing line-oriented JSON Twitter archives with Apache Spark.
created at Nov. 29, 2019, 2:52 p.m.
golang readers for ARC and WARC webarchive formats
created at Sept. 21, 2015, 6:38 a.m.
Create Robust Links from within Zotero
created at June 28, 2021, 9:38 p.m.
Internet Archive's Sparkling Data Processing Library
created at April 28, 2022, 2:28 p.m.
Partition (W)ARC Files by MIME Type and Year
created at Feb. 13, 2017, 3:45 p.m.