Jaap Vossers' SharePoint Blog

Just another WordPress.com site

Archive for February 2014

SharePoint Asynchronous Event Receivers and the powershell.exe process

with 5 comments

We recently had a very strange problem with one of our custom Event Receivers in our production SharePoint environment. The Event Receiver implemented the Asynchronous ItemUpdated event. It was used to generate and set a value on one of our fields on the document being updated. The code in the Event Receiver appeared to work most of the times, but would fail at seemingly random occasions and would leave the updated document without the generated field set.

We were struggling to isolate the combination of factors that made it fail. The weirdest thing was that there were no errors to be found in the ULS logs or the Event Log. We added lots of logging and try/catch blocks, but for some reason when the Event Receiver failed it would never enter the catch block, so there was no exception to log.

One key point that helped us with the troubleshooting was that we had noticed that the Event Receiver ALWAYS worked when the document was being updated through the SharePoint web UI. We also had a PowerShell script which was used for bulk updating of documents. This script was scheduled to run at regular intervals using Windows Task Scheduler. It appeared that the issue only occurred when the Updated event was triggered via this scheduled PowerShell script, but even then it still seemed intermittent as it would often work just fine.

We were unable to reproduce the issue at all when calling the ps1 file directly from the PowerShell console. So what was different when the script was run from the Task Scheduler vs directly from the PowerShell console? Well, the Task Scheduler actually calls a BATCH script which in turn invokes the PowerShell script which fires up a new PowerShell process. This process dies when it finishes execution of the ps1 file!

Remember, our Event Receiver is an Asynchronous one, so it would not block the execution of the PowerShell script. The Event Receiver is actually executed on a thread inside the PowerShell process since the ps1 script triggered the Updated event. So, when the PowerShell.exe process dies, it does not seem to wait for any background threads to complete, which in our case causes our Event Receiver to suffer from a sudden death. I was a bit surprised to see this to be honest!

Anyway, I guess one of the reasons why in our case the problem seemed to be appearing randomly is that only the last document in a batch would be affected, which sometimes meant 1 in a couple thousand documents. Only recently users had started feeding the script with “batches” consisting of just one document, which is what highlighted the problem to us and lead to this investigation. We were wondering what had changed recently (we had not touched this part of the code for a while!), since it was all working fine before (we thought), but in reality the bug had always been there but it had never occurred to us!

So everyone please beware when invoking PowerShell scripts from BATCH scripts when you have Asynchronous Event Receivers in your SharePoint environment!

What did we do to work around our problem? We just put a little Sleep at the end of our PowerShell script… 🙂

Written by jvossers

February 3, 2014 at 10:10 pm