Wiramu crawler

This crawler is operated by Wiramu Ltd (company number NI718907), United Kingdom. If it is causing you a problem, email [email protected] and we will reply within 2 working days.

What it collects

Only data that the publishers below make publicly available, through their documented interfaces. We never circumvent authentication, rate limits, access controls or CAPTCHAs, and we never access anything behind a login. If a request is blocked, we stop and record it rather than working around it.

Publishers accessed

• Birmingham City Council: cityobservatory.birmingham.gov.uk
• Companies House: download.companieshouse.gov.uk, stream.companieshouse.gov.uk
• Food Standards Agency: ratings.food.gov.uk
• London Borough of Camden: opendata.camden.gov.uk

How it behaves

• Requests are serial, at no more than 1 request per second per host.
• robots.txt is fetched and honoured. If it disallows a path, we do not fetch it.
• Retry-After is honoured, and errors back off exponentially.
• Conditional requests are used so unchanged content is not re-downloaded.
• There is no deadline on any of this work, so we would rather be slow than a nuisance.

Identifying us in your logs

Every request we make carries this User-Agent, and no other:

Wiramu-Capture/1.0 (+https://data.wiramu.co.uk/crawler; [email protected])

Asking us to stop or slow down

Email [email protected] and say which host or paths to exclude, or what rate you would prefer. We action it on receipt. You do not need to give a reason, and we will not ask for one. A Crawl-delay in your robots.txt works too, and we will use your number rather than ours.

Personal data

Some registers we collect contain personal data. How we handle it is set out in our privacy notice.