Start your crawl

For pagination and targeted URL crawling, you can trigger a crawl by either setting your API frequency to scrape on a schedule or to scrape on demand when you trigger the crawl from the 'API setup' of your API detail page or via the kimono API

To start your crawl, click:

Kimono will then extract data from each page in the crawl - your API's progress through the crawl will be displayed in the progress bar with the rest of your crawl stats.

Depending on how many pages you are crawling, it might take some time. Feel free navigate away from this page during the crawl. It will continue to run on our servers until it finishes the job. If you call your API mid-crawl, you will retrieve partial results from the crawl to date.

Interim results while crawling

Because crawling high numbers of pages can take significant time to complete, we allow you to retrieve interim results by calling the API end point at any time during the crawl. When you do this, kimono simply returns the results that have been fetched.

This API call behaves as a scheduled API call.

The results that show on your API detail page, are the cached results from the last crawl, not the interim results

Powered by Zendesk