Producer references and contact data · C0436
Do you respect robots.txt?
Open decision or unverified
Short answer
We do not have a documented statement on this, so we make no claim beyond the rule that blocked pages are skipped.
This is an open decision or an unverified fact. We show it rather than guess.
Checked September 30, 2026. Product version 0.1.3. Figures are dated where they appear. See status. This is not legal, security or purchasing advice.
Related answers
- How fast do you request pages from sources? Open
- Do you identify yourself when you collect? Open
- Are licences recorded per source? Built
- What happens when a source blocks you? Built
See it in context: Collection discipline · Search the help hub · Ask a question