4 lines
85 B
Plaintext
4 lines
85 B
Plaintext
A simple library to obey robots.txt files
|
|
|
|
WWW: https://github.com/chriskite/robotex
|