Awesome List Updates on Mar 26, 2020

8 awesome lists updated today.

🏠 Home · 🔍 Search · 🔥 Feed · 📮 Subscribe · ❤️ Sponsor

1. Awesome Web Archiving

Tools & Software / Acquisition

• archivenow (⭐391) - A Python library to push web resources into on-demand web archives. (Stable)

• Chronicler (⭐82) - Web browser with record and replay functionality. (In Development)

• Crawl - A simple web crawler in Golang. (Stable)

• crocoite (⭐42) - Crawl websites using headless Google Chrome/Chromium and save resources, static DOM snapshot and page screenshots to WARC files. (In Development)

• F(b)arc (⭐78) - A commandline tool and Python library for archiving data from Facebook using the Graph API. (Stable)

• freeze-dry (⭐265) - JavaScript library to turn page into static, self-contained HTML document; useful for browser extensions. (In Development)

• grab-site (⭐1.3k) - The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns. (Stable)

• html2warc (⭐15) - A simple script to convert offline data into a single WARC file. (Stable)

• HTTrack - An open source website copying utility. (Stable)

• monolith (⭐9.9k) - CLI tool to save a web page as a single HTML file. (Stable)

• SingleFile (⭐14k) - Browser extension for Firefox/Chrome and CLI tool to save a faithful copy of a complete page as a single HTML file. (Stable)

• SiteStory - A transactional archive that selectively captures and stores transactions that take place between a web client (browser) and a web server. (Stable)

• Social Feed Manager - Open source software that enables users to create social media collections from Twitter, Tumblr, Flickr, and Sina Weibo public APIs. (Stable)

• Squidwarc (⭐164) - An open source, high-fidelity, page interacting archival crawler that uses Chrome or Chrome Headless directly. (In Development)

• StormCrawler - A collection of resources for building low-latency, scalable web crawlers on Apache Storm. (Stable)

• twarc (⭐1.4k) - A command line tool and Python library for archiving Twitter JSON data. (Stable)

• WARCreate - A Google Chrome extension for archiving an individual webpage or website to a WARC file. (Stable)

• Warcworker (⭐53) - An open source, dockerized, queued, high fidelity web archiver based on Squidwarc with a simple web GUI. (Stable)

• Web2Warc (⭐24) - An easy-to-use and highly customizable crawler that enables anyone to create their own little Web archives (WARC/CDX). (Stable)

• Wget - An open source file retrieval utility that of version 1.14 supports writing warcs. (Stable)

• Wget-lua (⭐22) - Wget with Lua extension. (Stable)

• Wpull (⭐532) - A Wget-compatible (or remake/clone/replacement/alternative) web downloader and crawler. (Stable)

Tools & Software / Search & Discovery

• Tempas v1 - Temporal web archive search based on Delicious tags. (Stable)

• Tempas v2 - Temporal web archive search based on links and anchor texts extracted from the German web from 1996 to 2013 (results are not limited to German pages, e.g., Obama@2005-2009 in Tempas). (Stable)

Tools & Software / Utilities

• MementoMap (⭐9) - A Tool to Summarize Web Archive Holdings (Python). (In Development)

• MemGator (⭐51) - A Memento Aggregator CLI and Server (Golang). (Stable)

• node-cdxj (⭐0) - CDXJ (⭐15) file parser (Node.js). (Stable)

• OutbackCDX (⭐28) - RocksDB-based capture index (CDX) server supporting incremental updates and compression. Can be used as backend for OpenWayback, PyWb and Heritrix (⭐9). (Stable)

• py-wasapi-client (⭐12) - Command line application to download crawls from WASAPI (Python). (Stable)

• tikalinkextract (⭐8) - Extract hyperlinks as a seed for web archiving from folders of document types that can be parsed by Apache Tika (Golang, Apache Tika Server). (In Development)

• wasapi-downloader (⭐6) - Java command line application to download crawls from WASAPI. (Stable)

• WarcPartitioner (⭐1) - Partition (W)ARC Files by MIME Type and Year. (Stable)

• wikiteam (⭐687) - Tools for downloading and preserving wikis. (Stable)

Tools & Software / WARC I/O Libraries

• HadoopConcatGz (⭐9) - A Splitable Hadoop InputFormat for Concatenated GZIP Files (and *.warc.gz). (Stable)

• node-warc (⭐91) - Parse WARC files or create WARC files using either Electron or chrome-remote-interface (⭐4.2k) (Node.js). (Stable)

• Warcat (⭐135) - Tool and library for handling Web ARChive (WARC) files (Python). (Stable)

Tools & Software / Analysis

• ArchiveSpark (⭐140) - An Apache Spark framework (not only) for Web Archives that enables easy data processing, extraction as well as derivation. (Stable)

• Archives Unleashed Notebooks (⭐21) - Notebooks for working with web archives with the Archives Unleashed Toolkit, and derivatives generated by the Archives Unleashed Toolkit. (Stable)

• Archives Unleashed Toolkit (⭐131) - Archives Unleashed Toolkit (AUT) is an open-source platform for analyzing web archives with Apache Spark. (Stable)

• Tweet Archvies Unleashed Toolkit (⭐9) - An open-source toolkit for analyzing line-oriented JSON Twitter archives with Apache Spark. (In Development)

2. Awesome Cybersecurity Blueteam

Host-based tools / Tarpits

• Crowd Inspect - Free tool for Windows systems aimed to alert you to the presence of malware that may be communicating over the network.

Security monitoring / Endpoint Detection and Response (EDR)

• Wazuh - Open source, multiplatform agent-based security monitoring based on a fork of OSSEC HIDS.

Security monitoring / Network Security Monitoring (NSM)

• ChopShop (⭐487) - Framework to aid analysts in the creation and execution of pynids-based decoders and detectors of APT tradecraft.

• Maltrail (⭐5.9k) - Malicious network traffic detection system.

• OwlH - Helps manage network IDS at scale by visualizing Suricata, Zeek, and Moloch life cycles.

• Snort - Widely-deployed, Free Software IPS capable of real-time packet analysis, traffic logging, and custom rule-based triggers.

• SpoofSpotter (⭐48) - Catch spoofed NetBIOS Name Service (NBNS) responses and alert to an email or log file.

• Suricata - Free, cross-platform, IDS/IPS with on- and off-line analysis modes and deep packet inspection capabilities that is also scriptable with Lua.

• Wireshark - Free and open-source packet analyzer useful for network troubleshooting or forensic netflow analysis.

• netsniff-ng - Free and fast GNU/Linux networking toolkit with numerous utilities such as a connection tracking tool (flowtop), traffic generator (trafgen), and autonomous system (AS) trace route utility (astraceroute).

Security monitoring / Threat hunting

• CimSweep (⭐636) - Suite of CIM/WMI-based tools enabling remote incident response and hunting operations across all versions of Windows.

• DeepBlueCLI (⭐2.1k) - PowerShell module for hunt teaming via Windows Event logs.

• GRR Rapid Response (⭐4.7k) - Incident response framework focused on remote live forensics consisting of a Python agent installed on assets and Python-based server infrastructure enabling analysts to quickly triage attacks and perform analysis remotely.

• Hunting ELK (HELK) (⭐3.7k) - All-in-one Free Software threat hunting stack based on Elasticsearch, Logstash, Kafka, and Kibana with various built-in integrations for analytics including Jupyter Notebook.

• MozDef (⭐2.2k) - Automate the security incident handling process and facilitate the real-time activities of incident handlers.

• PSHunt (⭐276) - PowerShell module designed to scan remote endpoints for indicators of compromise or survey them for more comprehensive information related to state of those systems.

• PSRecon (⭐474) - PSHunt-like tool for analyzing remote Windows systems that also produces a self-contained HTML report of its findings.

• PowerForensics (⭐1.4k) - All in one PowerShell-based platform to perform live hard disk forensic analysis.

• Redline - Freeware endpoint auditing and analysis tool that provides host-based investigative capabilities, offered by FireEye, Inc.

• rastrea2r (⭐235) - Multi-platform tool for triaging suspected IOCs on many endpoints simultaneously and that integrates with antivirus consoles.

3. Awesome Actions

Utility / Docker Container Actions

• Has Changed Path (⭐180) - Conditionally run actions based on changed paths.

4. Awesome Game Remakes

Platformer

• Omnispeak (⭐289) - Reimplementation of Commander Keen in Goodbye Galaxy.

5. Awesome Flutter

Experimental / Redux / ELM / Dependency Injection

• styled_widget (⭐1.2k) - Simplifying your widget tree structure by defining widgets using methods by Rein Gundersen Bentdal.

6. Awesome Decentralized

Applications

• Berty (⭐7.4k): secure peer-to-peer messaging app that works with or without internet access, cellular data or trust in the network.

7. Awesome Sitecore

Commerce

• Reference Storefront (⭐25) - Sitecore Reference Storefront.

Demo

• Sitecore Habitat (⭐411) - Habitat is an example Sitecore solution built on the Helix architecture principles. No longer being updated or maintained, and is being replaced with Sitecore Helix Examples.

8. Awesome Ios

Menu

• SwiftyMenu (⭐572) - A Simple and Elegant DropDown menu for iOS 🔥💥

• Prev: Mar 27, 2020
• Next: Mar 25, 2020