Jaap Vossers' SharePoint Blog

Just another WordPress.com site

Archive for the ‘SharePoint’ Category

SharePoint Asynchronous Event Receivers and the powershell.exe process

with 5 comments

We recently had a very strange problem with one of our custom Event Receivers in our production SharePoint environment. The Event Receiver implemented the Asynchronous ItemUpdated event. It was used to generate and set a value on one of our fields on the document being updated. The code in the Event Receiver appeared to work most of the times, but would fail at seemingly random occasions and would leave the updated document without the generated field set.

We were struggling to isolate the combination of factors that made it fail. The weirdest thing was that there were no errors to be found in the ULS logs or the Event Log. We added lots of logging and try/catch blocks, but for some reason when the Event Receiver failed it would never enter the catch block, so there was no exception to log.

One key point that helped us with the troubleshooting was that we had noticed that the Event Receiver ALWAYS worked when the document was being updated through the SharePoint web UI. We also had a PowerShell script which was used for bulk updating of documents. This script was scheduled to run at regular intervals using Windows Task Scheduler. It appeared that the issue only occurred when the Updated event was triggered via this scheduled PowerShell script, but even then it still seemed intermittent as it would often work just fine.

We were unable to reproduce the issue at all when calling the ps1 file directly from the PowerShell console. So what was different when the script was run from the Task Scheduler vs directly from the PowerShell console? Well, the Task Scheduler actually calls a BATCH script which in turn invokes the PowerShell script which fires up a new PowerShell process. This process dies when it finishes execution of the ps1 file!

Remember, our Event Receiver is an Asynchronous one, so it would not block the execution of the PowerShell script. The Event Receiver is actually executed on a thread inside the PowerShell process since the ps1 script triggered the Updated event. So, when the PowerShell.exe process dies, it does not seem to wait for any background threads to complete, which in our case causes our Event Receiver to suffer from a sudden death. I was a bit surprised to see this to be honest!

Anyway, I guess one of the reasons why in our case the problem seemed to be appearing randomly is that only the last document in a batch would be affected, which sometimes meant 1 in a couple thousand documents. Only recently users had started feeding the script with “batches” consisting of just one document, which is what highlighted the problem to us and lead to this investigation. We were wondering what had changed recently (we had not touched this part of the code for a while!), since it was all working fine before (we thought), but in reality the bug had always been there but it had never occurred to us!

So everyone please beware when invoking PowerShell scripts from BATCH scripts when you have Asynchronous Event Receivers in your SharePoint environment!

What did we do to work around our problem? We just put a little Sleep at the end of our PowerShell script… 🙂

Written by jvossers

February 3, 2014 at 10:10 pm

Using jQuery to submit forms on remote SharePoint admin pages

with one comment

Imagine you need to develop a piece of functionality to extend SharePoint, but the available APIs do not directly allow you to do this for any of the following reasons:

  1. You are not allowed to deploy any server side code
  2. The server side code you can deploy has limited access to the server object model (e.g. Sandbox Solutions in SharePoint 2010).
  3. The API you need to access is private or internal

Consider the following scenario.

For our SharePoint Online site, we want to implement a Web Part that allows us to save the current site as a WSP in the solution gallery. As it’s SharePoint Online, we can’t deploy Farm Solutions, so we will have to deploy it as a Sandbox Solution. Unfortunately, we have limited access to the Object Model on the server, and there is nothing available in the Client Object Model which we can use to save the current site as a template.

Now, if there is an existing administration page that does what we want to do (in our case /_layouts/savetmpl.aspx does what we want to do), then technically all we need to do is a submit an HTTP POST request to that page with the right HTTP headers and form parameters and the server will happily process the request, as it has no way of telling whether the request was triggered by a user submitting the form, or by something else.

Welcome to the world of stateless protocols.

So we need to find out what the request should look like so that we can use jQuery to build it and issue it as an AJAX request from our own code.

image

Let’s open up Fiddler. With Fiddler open, in the browser we press the button that submits the form on /_layouts/savetmpl.aspx,. After that, we can inspect the form parameters of the HTTP POST request that the browser then makes. We are not really interested in the HTTP headers as the browser will take care of passing any headers related to the current browser session (including the authorization header) to the server when we issue an AJAX request.

image

We have three main categories of form parameters.

The first category is made up of parameters that directly map to user input fields.

  • ctl00$PlaceHolderMain$ctl00$ctl00$TxtSaveAsTemplateName
  • ctl00$PlaceHolderMain$ctl01$ctl00$TxtSaveAsTemplateTitle
  • ctl00$PlaceHolderMain$ctl01$ctl01$TxtSaveAsTemplateDescription
  • ctl00$PlaceHolderMain$ctl03$CbSaveData

The second category consists of a set of hidden fields which are used by ASP.NET & SharePoint to do its postback magic, including validation of the form post.

  • __REQUESTDIGEST
  • __VIEWSTATE
  • __EVENTVALIDATION

The third category is “the rest”. This is stuff we are not really interested in, but we still need to send it to the server.

  • MSOWebPartPage_PostbackSource
  • MSOTlPn_SelectedWpId
  • MSOTlPn_View
  • MSOTlPn_ShowSettings
  • MSOGallery_SelectedLibrary
  • MSOGallery_FilterString
  • MSOTlPn_Button
  • MSOSPWebPartManager_DisplayModeName
  • MSOSPWebPartManager_ExitingDesignMode
  • MSOWebPartPage_Shared
  • MSOLayout_LayoutChanges
  • MSOLayout_InDesignMode
  • MSOSPWebPartManager_OldDisplayModeName
  • MSOSPWebPartManager_StartWebPartEditingName
  • MSOSPWebPartManager_EndWebPartEditing
  • _maintainWorkspaceScrollPosition
  • __spText1
  • __spText2

It’s easy to determine what we want to submit as values for the first category. We either capture these values in our custom UI, or we have some logic in our code that determines these values for us.

The second category is a bit more difficult. Essentially, the server is expecting us to post back these values, which were provided by the server and rendered on the page as hidden input fields at the time of requesting the page which has the form on it. This means we need to make an initial GET request using jQuery so we can extract the values from the form, before we can submit them in our post.

The third category is easy, as we can copy the values from the request we captured with fiddler.

The script

Following the instruction above, we can write a bit of javascript like this to allow us to submit forms on “remote” pages.

function CreateWSP(callback) {

    var sitePrefix = "/";

    if (_spPageContextInfo.siteServerRelativeUrl != "/") {
        sitePrefix = _spPageContextInfo.siteServerRelativeUrl + "/";
    }

    var url = sitePrefix + "_layouts/savetmpl.aspx";

    $.get(url, function (data, textStatus, XMLHttpRequest) {

        var ctx = $(data);

        var rd = ctx.find("[name='__REQUESTDIGEST']").val();
        var vs = ctx.find("[name='__VIEWSTATE']").val();
        var ev = ctx.find("[name='__EVENTVALIDATION']").val();

        var postParams = {
            "MSOWebPartPage_PostbackSource": "",
            "MSOTlPn_SelectedWpId": "",
            "MSOTlPn_View": "0",
            "MSOTlPn_ShowSettings": "False",
            "MSOGallery_SelectedLibrary": "",
            "MSOGallery_FilterString": "",
            "MSOTlPn_Button": "none",
            "MSOSPWebPartManager_DisplayModeName": "Browse",
            "MSOSPWebPartManager_ExitingDesignMode": "false",
            "__EVENTTARGET": "ctl00$PlaceHolderMain$ctl02$RptControls$BtnSaveAsTemplate",
            "__EVENTARGUMENT": "",
            "MSOWebPartPage_Shared": "",
            "MSOLayout_LayoutChanges": "",
            "MSOLayout_InDesignMode": "",
            "MSOSPWebPartManager_OldDisplayModeName": "Browse",
            "MSOSPWebPartManager_StartWebPartEditingName": "false",
            "MSOSPWebPartManager_EndWebPartEditing": "false",
            "_maintainWorkspaceScrollPosition": "0",
            "__REQUESTDIGEST": rd,
            "__VIEWSTATE": vs,
            "__SCROLLPOSITIONX": "0",
            "__SCROLLPOSITIONY": "0",
            "__EVENTVALIDATION": ev,
            "ctl00$PlaceHolderMain$ctl00$ctl00$TxtSaveAsTemplateName": "VossersTeamSite.wsp",
            "ctl00$PlaceHolderMain$ctl01$ctl00$TxtSaveAsTemplateTitle": "Vossers Team Site",
            "ctl00$PlaceHolderMain$ctl01$ctl01$TxtSaveAsTemplateDescription": "Vossers Team Site Template",
            "ctl00$PlaceHolderMain$ctl03$CbSaveData": "on",
            "__spText1": "",
            "__spText2": ""
        };

        var options = {
            url: url,
            type: "POST",
            data: postParams,
            success: function (data, textStatus, XMLHttpRequest) {

                callback();

            }
        };

        $.ajax(options);

    }, "html");
}

Things to consider

When the page you are posting to is modified, for example by a SharePoint update, then it’s possible that your script breaks due to changes in form parameter names. This makes this technique a bit fragile. For this reason I recommend that you only consider using this technique once you have confirmed that it’s not possible to achieve what you want by using public APIs.

Written by jvossers

February 4, 2012 at 3:22 pm

SharePoint InlineSiteSettings 2010 – improved productivity for Administrators and Developers

leave a comment »

After having released SharePoint InlineSiteSettings for SharePoint 2007 a while ago, and having used a little desktop application called Launchy which is used to start desktop applications using just a few keystrokes, I decided to build an enhanced version of InlineSiteSettings, built for SharePoint 2010 with features similar to Launchy’s.

The end result is SharePoint InlineSiteSettings 2010, which can be downloaded from CodePlex at http://sitesettings2010.codeplex.com/

inlinesitesettings2010

The purpose of the solution is to improve productivity for SharePoint 2010 users who regularly access the Site Settings page, i.e. SharePoint Administrators and SharePoint Developers. It allows them to access the Site Settings in a dialog by pressing Ctrl+s, so no need to move your mouse to Site Actions, click it, click Site Settings, and wait for the full page to load.

As we all know, once the Site Settings page has been loaded, it can actually take a few seconds to spot the link you are looking for (as the links are not listed in alphabetical order), so what’s new in this version of SharePoint InlineSiteSettings is that users can start typing the title of the link they whish to navigate to, and with real-time filtering functionality, all links that do not match your filter will disappear from view. In addition to that, as soon as exactly one link is left that matches your filter, it will automatically redirect you to that page, as can be seen in the demo screencast below. As a result, navigating between administrative pages in SharePoint 2010 will be less painful.

SharePoint InlineSiteSettings 2010 is packaged as a Sandbox Solution, and does not depend on any server side code. The good thing about this is that it works on SharePoint Online (Office365).

Download SharePoint InlineSiteSettings 2010 from CodePlex

Written by jvossers

May 8, 2011 at 9:59 pm

Bypass caching with jQuery AJAX GET requests

with one comment

As I seem to use this trick quite often and I keep forgetting the exact details on how to implement it, I thought it would be good to document this.

Using jQuery, I often make ansynchronous GET requests to a custom ASHX handler in SharePoint’s _layouts folder which returns some data that I want to display. This data is always dynamic, but sometimes the browser tries to cache the results from the previous request, so you might not get the response you expected.

To avoid this, simply make the url for eacht request unique by adding a timestamp to it in javascript.

var url = '/_layouts/MyProject/MyHandler.ashx?unique=' + new Date().getTime();

Written by jvossers

January 11, 2011 at 10:41 am

Released: SharePoint WebPartSlices

with one comment

I have recently released my sixth CodePlex project (list of all my CodePlex projects), titled SharePoint WebPartSlices.

To all the jQuery fans – Sorry, no jQuery this time! SharePoint WebPartSlices is a server-side solution that allows users to transform all Web Parts on a page into IE8 Web Slices simply by adding the webslices=1 querystring parameter to the url of the page containing the web parts to be transformed, basically allowing you to add web parts to your IE8 Favorites Bar.

webpartslices1

VoilĂ  – No need to navigate to the original page to see the updated Web Part.

webpartslices2

Installation consists of a WSP deployment + Feature activation only.

SharePoint WebPartSlices on CodePlex

Written by jvossers

February 4, 2010 at 12:06 pm

jQuery Developer Frameworks for SharePoint

with 2 comments

If you are into developing SharePoint solutions using jQuery, you should really have a look at the following GREAT CodePlex projects. Both projects have been developed by real experts in the SharePoint & jQuery field 🙂

jPoint – Developed by a team lead by Will Lawrence (@willhlaw)

jPoint is an open source javascript API and script deployment framework for Sharepoint. It leverages the speed and agility of jQuery to manipulate Sharepoint forms and pages, communicate to Sharepoint webservices, and to build jPart plugins.

jQuery Library for SharePoint Web Services – Developed by Marc D Anderson (@sympmarc)

This is a jQuery library which abstracts SharePoint’s Web Services and makes them easier to use. It also includes functions which use the various Web Service operations to provide more useful (and cool) capabilities. It works entirely client side and requires no server install.

Written by jvossers

December 16, 2009 at 12:06 pm

Released: SharePoint LiveListData

with 20 comments

Last week I published my fourth CodePlex project, called SharePoint LiveListData. It is an “assembly-free”  solution which means that it contains no server side code. It is implemented as a jQuery plugin.

So what does it do? It uses AJAX to automatically reload any list-based Web Parts as soon as a change in the underlying list data is detected. You can also have your custom web parts refreshed, as long as they have a web part property that contains the ID of the list they depend on!

I have put online a screencast that demonstrates the the user experience.

Want to know how it works? Allow me to describe using some pseudo code..

  • OnDocumentLoaded
    • Make a single call to WebParts.asmx web service using AJAX to return an xml document with all web parts on the current page
    • Parse returned xml and find all web part nodes that contain a ListID element (which holds the value for the ListID web part property)
    • Create and populate a javascript object that contains a list of all ListIDs that we found in the returned xml, plus a list of all WebPartIDs per ListID, also retrieved from the xml (more or less creating a hashtable with the key being the ListID and the value being an array or WebPartIDs).
    • LOOP with interval = $configured_interval
      • Make a single call to Lists.asmx web service using AJAX to retrieve “LastDeleted” and “Modified” properties for each ListID in the hashtable that was created earlier.
      • For each ListID in hashtable
        • compare values for LastDeleted and Modified with those values retrieved as part of the previous loop iteration for this ListID (unless it’s the first iteration).
        • If a change in one of the two property values has been detected (indicating that someone inserted, updated or deleted an item) mark this ListID as “UpdatePending”, storing it on our javascript object that holds our ListIDs and WebPartIDs.
      • If any of the ListIDs are marked as UpdatePending
        • Make an AJAX request to page that is currently loaded in the browser (allowing us to get a fresh copy of the current page containing the new list data)
        • For each ListID marked as UpdatePending
          • For each WebPartID associated with ListID in our javascript hashtable object thingy
            • Replace div with matching WebPartID attribute in current document with “same” div in AJAX response (which contains the new list data).

This is basically how it works in a nutshell.

For examples on usage, check out the downloads tab on the SharePoint LiveListData CodePlex site, where you can also download the script!

Written by jvossers

November 3, 2009 at 11:40 pm

A developer’s thoughts on the Microsoft SharePoint Conference 2009 in Las Vegas

with 5 comments

After having attended the Microsoft SharePoint Conference in Las Vegas last week, I decided to write a blog post in which I will summarise my thoughts on SPC09 and SharePoint 2010 from a Developer point of view.

The majority of the sessions I attended were developer-oriented sessions.  There are so many new promising features that will greatly increase developer productivity.  I seriously can’t wait to start my first SharePoint 2010 project!

Let’s just walk through some of the new features that I think are most promising.

Client Object Model

The Client Object Model is a richer alternative to the already existing SharePoint web services. Before SharePoint 2010, anyone writing code that would run somewhere else then on the actual SharePoint server could not use the Server Object model but instead had to rely on using the SharePoint web services.  If you knew how to use the Server Object model, that didn’t necessarily mean you knew how to deal with the web services, since they are so different!

SharePoint 2010 Introduces the Client Object Model which comes in three flavours: Javascript, .NET CLR and Silverlight. It basically means that your code can call into the Client Object Model, which exposes a programming interface that is very similar to the Server Object model programming interface. The idea is that you can apply your knowledge of the Server Object Model to the Client Object Model.

I am particularly looking forward to using the Javascript Client Object Model, combined with jQuery. Any half-decent developer will now be able to build rich, AJAX-enabled UI’s in SharePoint 2010 without too much effort.

REST API

The REST API is another feature that will simplify the process of consuming data from client applications by exposing a very simple HTTP based interface. It basically allows you to fetch or update/insert/delete data in SharePoint. You can specify what data you want to work with by constructing a url that SharePoint understands. The url will include parameters for the web, the list and possibly a filter that SharePoint will use to scope the operation to. If you are simply performing a fetch of the data, the server will return XML or JSON (depending on what you asked for by adding the right HTTP header to your request) for you to consume in your client application.

Again something that will be very useful in a javascript client, with or without jQuery, as you can perform AJAX requests in the background to fetch or modify data. Fetched data returned as JSON can easily be converted into javascript objects for further consumption in your javascript code.

For .NET clients there is a thing called ADO.NET Data Services which allows you to create DataSources that point to a REST url, which can then be used as the DataSource for databound controls. One of the sessions at SPC09 demonstrated how easy it is to set up a grid with insert/update/delete support, simply by binding it to such a datasource. Amazing!

SPMetal + LINQ

Cool! Now you won’t have to write those pesky CAML queries anymore AND you won’t have to access SPListItem field values through an indexer that returns an object that isn’t strongly typed – which in turn means that you don’t have to perform any casting anymore. Querying is now done via LINQ, which I am sure you have heard of before. You can actually perform inner joins in LINQ allowing you to use only one query to return all Employee items where Employee.Company.CompanyName == “Microsoft”. In SharePoint 2007 you need to perform two separate queries to achieve this. I am curious how this works under the hood. My understanding is that your LINQ query is translated to a CAML query which is then executed. If that’s true then presumably they have extended the CAML query schema to support JOINs. Does anyone have an answer to this?

SPMetal is the commandline tool that generates the classes to support LINQ Querying and strongly typed access to properties of an SPListItem.

Business Connectivity Services

This is the replacement for Business Data Catalog. It allows you to write data access code which can be used to connect alternative data source to SharePoint lists. SharePoint users won’t even notice that the “list data” they are working with is in fact stored somewhere else.

I really hope my code can access this external data as if it was contained in a standard SharePoint list, for example by using the SPList and SPListItem classes or performing LINQ queries. Does anyone have an answer to this?

Developer Dashboard

The developer dashboard helps developers with identifying poorly performing code by having diagnostic information related to your current request printed on the page. It is similar to the ASP.NET trace information, but more SharePoint-specific. It prints processing time for Web Parts (allowing you to instantly spot which one is causing the page to be so slow) and even processing time for the underlying Stored Procedures that are executed by SharePoint!

PowerShell

PowerShell is the new STSADM for SharePoint. Microsoft has invested heavily in PowerShell support for SharePoint by providing hundreds new SharePoint-specific Cmdlets! Looks very promising. I have this PowerShell book somewhere – I think I will pick it up again soon!

Visual Studio integration

Visual Studio 2010 integration for SharePoint 2010 development has strongly improved. I must say that for some reason I had assumed Microsoft would provide this level of integration when MOSS 2007 came out a few years ago (wishful thinking), but unfortunately we’ve had to depend on third party tools like WSPBuilder to produce our deployment artifacts. I guess it was necessary for us to go through this pain in order for us to be able to appreciate what’s coming our way now!

Custom Service Applications

There is no such thing as an SSP in SharePoint 2010. Instead there is a collection of Services Applications, to which you can add your own custom Service Application. The main focus lies on scalability – it supports load balancing and you can decide which servers in the farm will run your custom Service Application.

It is very important to plan properly and decide whether a custom Service Application is the right solution to your problem. Developing custom Service Applications might well be the most complex subject within SharePoint Development.

Written by jvossers

October 30, 2009 at 1:09 pm

New version of SharePoint InlineSiteSettings released (javascript only)

with one comment

SharePoint InlineSiteSettings is a tool which I originally released several months ago. It provides a quick way to access your Site Settings in SharePoint; it allows you to pop up the Site Settings of the current site – triggered by a keyboard shortcut – without navigating away from the current page. InlineSiteSettings uses jQuery to power its AJAX functionality.

After I was hinted by Jeroen Ritmeijer from Muhimbi that a javascript only version of SharePoint InlineSiteSettings (the previous version was implemented as a serverside WebControl spitting out javascript) would work very well with their free SharePoint Infuser product, I started my investigation and ended up implementing a new version of SharePoint InlineSiteSettings! I am quite pleased that it is now a very compact solution compared to it’s previous release.

You COULD use the new version SharePoint InlineSiteSettings without Muhimbi Infuser by pasting the script into a Content Editor Web Part on each page where you want to enable the SharePoint InlineSiteSettings functionality, BUT to be honest, this doesn’t really make sense and it would be very time consuming. What we really want is a solution that allows us to inject the script on every page and configure it only once. This is where SharePoint Infuser comes into play.

Please try it out and let me know your thoughts!

Download SharePoint InlineSiteSettings 1.2 from CodePlex

Written by jvossers

September 20, 2009 at 8:42 pm

Reusable Content & HTML Snippets

with 3 comments

So your client wants to be able to insert more than just formatted text and images in his Publishing HTML field control? So he wants to insert those fancy looking boxes that your graphics designer made? Ah, but they are variable height and require several background images so we are talking about inserting a bit of “complex” html. How can we provide a user friendly interface with as little customisation as possible? I personally like the out-of-the-box Reusable Content functionality provided by MOSS 2007. I will demonstrate how it can be used to solve our problem.

Let’s take the following box for example:

rc-start

Because of the vertical scaling requirements, the box needs to be split into three logical sections: header, content, footer. The html for the box shown is as follows:

<div class="box">
    <div class="box-header">This is the header</div>
    <div class="box-content">Lorem ipsum ...</div>
    <div class="box-footer">&nbsp;</div>
</div>

The styling of the box, including the background images is done via CSS. The header and footer are fixed height, whereas the content area is variable height and has a background image that repeats in the vertical direction.

Back to SharePoint – when you create a site collection using the Publishing Site Site Definition, you should have a list called Reusable Content in your root web. You can navigate to it using Site Content and Structure. We will create an item in this list that will be a “template” for our box.

Insert a new item into the Reusable Content list. Ensure you have Automatic Update turned OFF. Use the Edit HTML Source on the html editor to paste in the following html:

<div class="box">
    <div class="box-header">[HEADER] </div>
    <div class="box-content">[CONTENT]<div>
    <div class="box-footer">&nbsp;</div>
</div>

The rest of the fields are for you to fill in. Save the item and then approve it (the Reusable Content list has content approval turned on by default).

Now that we have our box html template in place, let’s go to our page that needs to have an instance of the box appear on it. Go to edit mode and locate your Publishing Html Field Control. Click the toolbar button Open a new window to Insert Reusable Content.

rc-insertfromeditor

Select the item that we created earlier and press OK. If everything went well, an instance of the box with text placeholders has been inserted into your Publishing HTML Field control.

rc-inserted1

All there is left to do is to place the cursor in the text placeholders and replace it with the actual content.

rc-inserted2

On a final note, be aware of the risk involved when content editors play around with div elements. I have seen situations where content editors got “creative” and partially deleted boxes, leaving orphaned div opening tags and turning the layout of your page into scrambled eggs.

Written by jvossers

April 19, 2009 at 11:10 pm