User-agent: *
Disallow: *.pdf
Disallow: *.txt

User-agent: ia_archiver
Disallow: /