Tutorial: Change the source URL via your REST API

So you’ve made an awesome new Kimono API, and it works great for the site you set it up on. Sweet! But why stop there? We can use the Kimono API to programmatically manipulate our own Kimono APIs, giving our programs more control over our APIs, thus letting us do even more awesome stuff. Sweet!

In this post, we’ll talk about how you can use python to dynamically update the source URL for a kimono API. So if you start by crawling myawesomewebsite.com/1, we can make an api call and have kimono crawl myawesomewebsite.com/2.

Cool, let’s hop in. The Kimono API makes this really easy. For this post, we’ll be using python 2, but you can apply the concepts to basically any language. 

Prerequisites

For this tutorial, we’re going to need just a few things:

  • Python 2.6.x/python 2.7.x. If you don’t know what you have installed, go to your terminal and just type python, it should tell you.
  • The requests module for python. If you’ve got pip, it’s a simple “pip install requests” to get it set up.
  • A Kimono Account
  • A Kimono API, built for a single webstie (eg mywebsite.com/1), and at least one more page from the same site (eg mywebsite.com/2)

 

Got it? Let’s get into it!

 

Here’s the code:

 

import requests



#enter your info here

API_ID = ""

 

API_KEY = ""

 

API_TOKEN = ""

 

#add whatever the new url is

targeturl = "mywebsite.com/2"

 

#change payload based on what you need to deliver

payload = {

'apikey': API_KEY,

'targeturl': targeturl,

}

 

headers = {"authorization" : "Bearer %s" % API_TOKEN}

 

#you can edit the end of this url to do different things as specced in the api docs

#you can also change the p

#https://www.kimonolabs.com/apidocs#SetTargetUrl

 

api_url = "https://www.kimonolabs.com/kimonoapis/%s/update" % API_ID

 

r = requests.post(api_url, headers = headers, data = payload)

 

#and probs do something else with the response....

print(r.text)



Let’s break it down:

 

Import Requests - This just pulls in our requests module which makes it a little prettier to make http requests in python.

 

API_ID = ""

 

API_KEY = ""

 

API_TOKEN = ""

 

You’ll want to add the api info for your specific API here. You can find all that good stuff on your API detail page (kimonolabs.com/apis/[my 8 digit API key]). Once you’ve done that, we use these variables to tell the Kimono API which API we want to work on, and authenticate that we’ve got the permission to do so.

 

targeturl = "mywebsite.com/2"

This is the url you want to change your url to. Remember we want to target another page on the same site, so that our individual Kimono API will be able to use the same logic to pull in all that juicy data. Yum!

 

payload = {

'apikey': API_KEY,

'targeturl': targeturl,

}

Here we’re creating a dictionary of the info we want to deliver to the Kimono API. So we’re sending two things. The first is our api key -- we need this to authenticate that we have permission. Notice we’re pulling that from our API_KEY variable earlier.

 

The second is our target url - we’re sending this so Kimono knows which url to change our crawl to. If we were, say, not changing the url, but instead telling Kimono to start a new crawl, we wouldn’t need to include target url.

 

headers = {"authorization" : "Bearer %s" % API_TOKEN}

We need to include these with our request for authentication. Security yo!



api_url = "https://www.kimonolabs.com/kimonoapis/%s/update" % API_ID

Ok, here we’re creating a url to send our request to.  A few things to note here -

  • Calls to the Kimono API (as opposed to our individual api) go to https://www.kimonolabs.com/kimonoapis/API_ID. This is how Kimono knows that we want to perform meta operations
  • We use %s to fill in our API_ID from earlier, so Kimono knows which of our APIs to operate on
  • We end the url with /update. This can be used to do a few things besides changing the target url, for instance: setting crawl frequency, starting a crawl, or setting crawl urls. There are also GET methods where we can learn about the status of our apis. Check it all out at: kimonolabs.com/apidocs.

 

r = requests.post(api_url, headers = headers, data = payload)

Ah, here it is. The workhorse of our whole script. Let’s break it down.

  1. We set a variable, r equal to the following:
  2. first we open up the requests module, and then call its post method with “requests.post”
  3. We invoke request.post as a function, passing it the parameters inside the parenthesis, in this case (api_url, headers=headers, data=payload).
    1. api_url is the url to make the request to. So we’ve created that in the line above, and now we’re just referencing it.
    2. headers=headers. headers is an optional argument that we get from requests. In this case, we need it to handle our authentication (yay security!).
    3. data=payload. Data is telling requests that we’re sending it data, and that data is what’s contained in our payload dictionary from earlier.
  4. We wrap it all in parenthesis and make our request.

 

print(r.text)

Sweet Success! We print our response by calling r.text. Of course, you probably want to wrap the response in JSON to do something more substantial than just printing it. Easy! We can turn our response object into json with a simple r.json().


That’s it! There’s a ton that you can do with the Kimono API!

Powered by Zendesk