Hakrawler
beginner
Fast golang web crawler for gathering URLs and JavaScript file locations. This is basically a simple implementation of the awesome Gocolly library.
Back to Tools
crawler

Command

cat urls.txt | hakrawler -timeout 5

Platforms

linux
mac
windows