Some publishers don’t allow to crawl their website and articles and set a restriction in the domains robots.txt – file. If a publisher has declined to crawl his website we’re not allowed to gather any contextual information from this specific domain/URL.
Articles in this section
- What is the difference between API v2.x and API v3.x?
- My API-Key is not working, what’s wrong?
- Is the API not working? I can’t connect to the API and get permanently a “Bad Gateway” error message.
- What kind of URL should I send to the API?
- Why there is from time to time no result at all?
- What is the meaning of sentiment result “Could not get sentiment.”?
- Why is the category or IAB result “retry”, “null” or “no channel returned” in the JSON response?
- Why the first request of a URL shows a response “No data yet” in the JSON response body?
- Sometimes there is a weird looking keyword in the results which have nothing to do with the contextual meaning.
- How can I provide feedback when the JSON response is looking weird. The result does not fit my expectation.