Kategoriarkiv: REST

Using CAML with SharePoint REST API

Do you prefer REST over CSOM as I do? I’ll skip the whys. Andrew Connell put it already in wrtiting so nicely. Well, if you do prefer REST, then you must have discovered some shortcomings of REST, or its incompleteness compared to CSOM. I think of:

  1. Inability to filter items based on multivalued taxonomy fields
  2. Inability to filter items based on user fields where user is added through a group, rather than directly, e.g. AssignedTo=[Me] combined with a SharePoint group.

In such situations I was forced to use CSOM. Until yesterday. Yesterday I learned that we can actually use CAML queries in REST requests.


This enables using REST in all situations. The REST API is still developed and many features are added. Maybe a particular operation that needs a CAML query today, can be supported in the core REST API and can be easily refactored then.

But until then, we can use CAML queries in REST requests. Here are the important things about it:

  • A REST request with a CAML query is always a POST request
  • A REST request with a CAML query has always to have X-RequestDigest http header (actually because it is a POST request)
  • A REST request with a CAML query should always have the attached CAML query in the request body (and not in a query string). We don’t want to mess with long urls, do we?
  • A REST request with a CAML query must have the http header “Content-Type: application/json;odata=verbose” unless you use xml in the request body.
Needed HTTP Headers in REST requests

HTTP headers you have to provide in REST requests with CAML queries

You can use jQuery or SP.RequestExecutor to make an ajax call. The REST endpoint is:

The request body (if you use json, and I bet, you do) is in this format:

Here is the boilerplate for a REST request with a CAML Query:

This function is just an example. It has no error handling, and it takes for granted that your list is in the root site for on your (sub-)domain (“/”). So take it as an example only.

Here is how the function can be invoked

Basic REST request to SharePoint using Postman

I wanted to share this tutorial on how to consume SharePoint’s REST service using the HTTP client Postman. I have to do this on a daily basis but keep forgetting the details and have to Google it, but Google is not that helpful and I get results that are unnecessarily complex.

I’m using a development environment and am logged in as a user with permission to read the data I’m requesting, which is a requirement. It takes to steps, first we have to prove that Postman is authenticated and then we can request the data.

Getting authenticated

Type in

into the input field and make sure it’s a POST-request. Now we need to add a header, to tell SharePoint what kind of data we want back, so click on the ”Headers” button at the top right corner. At the left side on the new lines that appear, enter Accept. At the right side enter application/json; odata=verbose. It should now look something like this:


Then click send and you get the results back. Copy the FormDigestValue, which is a really long string ending with a date and time. Mine looked like this:

Getting data

Change the URL to, for example,

and it should still be a POST-request. The accept-header should look the same, but now you need to add the FormDigestValue as another header. To the left, enter X-RequestDigest and to the right enter the value you copied before. Then all you have to do is click send, and the data appears. The finished result:


Using Python to request data from SharePoint via REST

Recently Bool participated in a bidding for a public sector procurement contract. Among the endless number of questions to answer were some which covered whether our system could interact with outside applications. And that got me thinking: could I use python to access data from a SharePoint site? Turns out I could, and here’s how:

This is just a basic example, and not a polished piece of code ready for production. For one thing, the password and username is written in plain text, not something you usually want. But it is a fun and agile way you could interact with the gigantic application that is SharePoint.

Setup the environment


I’m using a virtual machine with SharePoint 2013 installed. You might need to enable Basic authentication on your webapplication. Open the Internet Information Services Manager and find your webapplication under ’sites’ to the left. Dubbleclick on the Authentication, click on Basic Authentication and enable.


Enable Basic auth

Python and the Requests library

First you need python, if you don’t already have it. I use Chocolatey package manager to install it, but you can find more instruction on the python page. Run the following command in your command prompt. If it complains that it can’t find python when you run the command, try restarting cmd. Sometimes it needs to be restarted in order to add the installed programs to path.


That installs python. Then I need pip, the python package manager. Again, I use Chocolatey (everyone with a machine running windows should get it, it’s awesome!), but alternatively you could use the pip install instructions from their page.


After that we need the library Requests. It’s an easy and humane way to make HTTP-requests in python. I love the library, but they need to work a bit on their web page. I hade some difficulties finding the API documentation.


And now the setup is complete and we are good to go!

Creating the python file

Create a file and name it app.py. At the top you import the requests library and HTTBBasicAuth (assuming you are using HTTP authentication), like this:

Then we start by simply requesting a web page and to do that we need to specify the HTTP-method, url and authentication method. We are going to use a GET-request, since we only want to look at a page. My root site is called http://example and we are going to use basic authentication. The request looks like this:

Just exchange someUser and somePassword with your own credentials. And that is actually it. It is that simple. Just put a print at the end to see what you got back.

This prints the status code, and you hope for 200. If you add

print r.content

you get the entire page’s HTML. The complete file looks like this:

Run it

Open the command prompt to where you saved your file and type

Hopefully it will now print the number 200 and a bunch of HTML.

Using JSON and SharePoint REST API

But the code displayed above isn’t very useful, unless you are building a new web browser using a python middle man. What we want is solid data to work with, and for that we want JSON, and for that we need to use SharePoint’s REST API. You can locate the REST service on the URL /_api/web on any site. In my case this means http://example/_api/web. You also need to specify the type of data you want back from the API. This is done through the accept header, and SharePoint has a very specific string it required to respond with JSON.

Place that in front of your request, and update the request to look like this:

Now it calls the API URL and declares that the returned information should be in JSON format.

This prints the entire JSON object. Now it’s just a matter of sorting through the data and finding what you want.

For example, that gives you the Custom Master Page!

My finished code looks like this:

To summarize: it’s easy to request data from SharePoint using any platform and any language. This might not be the safest way, but it’s a fun way to try it out. If you’re building a proper application, you probably want to look into OAuth, which is supported.

Apps can only call the OOB CSOM and REST endpoints

As a SharePoint architect or a SharePoint developer, you must have been thinking about the benefits/limitations of SharePoint apps a lot. I want to point out one of them today, which is very important: using custom webservices deployed to SharePoint inside apps. That is impossible and it is designed to be so due to the security architecture in the sharepoint app framework.

I have read much about SharePoint apps (books, whitepapers, blog posts) and stumbled over these two contradictive statements:

[…] app authenticatton is supported only for scenarios in which an app is calling to the SharePoint host environment by using client-side object model (CSOM) or the REST API. SharePoint 2013 does not support app authentication in any other endpoints beyond these. This means it is not possible to develop and deploy a set of custom web service entry points that support app authentication.

Microsoft SharePoint 2013 App Development, Microsoft Press, page 88.

This is really important to know, because it is exactly the opposite to what it is said in a Microsoft WhitePaper, where we are requested to “some outside-of-the-box thinking”:

The main reason why you would still use full-trust solutions is that a feature or API you want to use is not yet available through the REST endpoints or CSOM APIs. However, this doesn’t mean you can’t still use the server object model from an app that is on-premises. It just requires some outside-of-the-box thinking. Instead of writing a full-trust solution that completely covers the entire business scenario you are trying to satisfy, write a full-trust solution that exposes the functionality you are looking for as a REST endpoint and write an app that can use that endpoint. This will allow your solution to scale while reducing the overall exposure of full-trust code to the SharePoint environment.

Deciding between apps for SharePoint and SharePoint solutions, Microsoft. Keenan Newton.

Just to clearify, the first statement is correct. The second is unfortunately not correct. To be sure I created a web service which I deployed as a farm solution. And when I tried to invoke this webservice from my app I got 403 error and this message

The endpoint cannot be added within the manifest neither:


While developing SharePoint apps we are restricted to the OOB CSOM and REST endpoint only.

OOB here means out-of-the-box, not outside-of-the-box.