Now this is using a list with over 20,000 entries in it and its growing all the time.  An associated list that references this list as a lookup has over 144,000 entries in it.

The event firing problem is becoming more frequent now as more users start using the system, there are at least 20 occurances in that 20,000 items, so it has to be fixed.

Managed to track down an error in the log

System.Data.SqlClient.SqlException: Transaction (Process ID 61) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

It’s stack trace was similar too this (I cut it down a bit and changed some names, my naming is not this bad)

at Microsoft.SharePoint.SPListItemCollection.EnsureListItemsData()
   at Microsoft.SharePoint.SPListItemCollection.get_Item(Guid uniqueId)
   at BinaryJam.Core.Code.TicketListReceiver.ItemReceiver.ItemAddedActual(Guid SiteId, Guid ListItemId, String UserDisplayName)
   at BinaryJam.Core.Code.TicketListReceiver.ItemReceiver.<>c__DisplayClass11.<ItemAdded>b__e()
   at Microsoft.SharePoint.SPSecurity.CodeToRunElevatedWrapper(Object state)
   at Microsoft.SharePoint.SPSecurity.<>c__DisplayClass4.<RunWithElevatedPrivileges>b__2()
   at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode)
   at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(WaitCallback secureCode, Object param)
   at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated secureCode)
   at BinaryJam.Core.Code.ItemListReceiver.ItemReceiver.ItemAdded(SPItemEventProperties properties)

So as you can see, it explodes on the GetItem.

This was my stupid mistake, a mistake I bet many of you are making every day.

SPListItem item = List.Items[listItemId];

This I believe is the primary cause of the problem as this statement loads the complete List before returning the specific item I needed and as this list grows, remember 20,000 items already, its going to get worse.  Now for most of you this won’t be a problem as you will be just getting the ListItem from the parameters, in my case I have to run as elevated as Im doing lots of stuff with Security Groups, that the entering user may not be in, hence my needing to re-get the List Item in the right security context.

So if your having a problem like this remember, Always use the

List.GetItemByID(id); //Or
List.GetItemByUniqueId(guid);

This may or may not solve my problem I believe it will, if it doesn’t it’s still going to improve the performance of my application.  If your wondering, this is the only time I use this and don’t know why I did, the rest of the App uses getby id or SPQueries.

A Good Article on Performance I did actually read, hence the app performs pretty well, except for this one bug, which is now hopefully fixed.

A nice man at microsoft diagnosed the problem and came up with the code you should run to ensure your not messing up the Web.

Here is a posting of the proposed solution to the bug we found (I don’t care if they don’t think its a bug, not sure if they do or don’t but I do!)

http://blogs.msdn.com/joerg_sinemus/archive/2009/11/24/overwriting-spwebprovisioningprovider-provision.aspx

What we did not have was a record of who actually approved the sites.  Whilst this info is in the workflow history after a while that information gets purged, leaving us with Sites that were approved but by whom we know not.

So this is a simple case of changing the workflow and recording all of the approvers in the SPWeb properties, except that the approvers as we can tell happen to be Sharepoint Groups or AD groups, neither of which contain a history of membership, if AD does none I know how to access and certainly not easy for the customer and in a large membership we still would not know who actually did the deed.

I dug and dug for this and eventually found the User Login that actually did the change and here is how.

In your workflow you need a “Workflow Action” of EventName “OnTaskChanged”, this object in the code behind, when Invoked contains AfterProperties,

string user = OnApprovalTaskChanged.AfterProperties.ExtendedProperties[new Guid("d31655d1-1d5b-4511-95a1-7a09e9b75bf2")].ToString();

The above statement gets the User Login frmo the ExtendedProperties Collection.  It Stores the User under the key of that Guid in a User type field that refers to the Editor.

A list of fields can be found here .

So that’s it, took a lot of debugging and poking about but next time your looking for the actual approving user, take a look here.

With a variety of developers before me different styles of editing the foreign language files emerged.

I think that much better tool support for managing resx’s is required,  and I recently found one that is a great help to us with a vareity of resx’s in various states of translation and existence.

http://resxtranslator.codeplex.com/

This resx tool allows you to pick a file and it displays the nuetral and foreign side by side along with comments and allows you to edit them all at once.  Corking piece of work, some polish would go amiss but Im hard pushed to say where as it does exactly what I need it to do and I can pass it to a non-techie developer.  It still down to me to edit the comments first to identify what needs translating and what definately needs leaving alone.

If you doing multi-language work go and check this tool out it might just help.

I thought this time I would just point out some of the very simple statements in PowerShell that differ from your usual C# so that you can quickly make a transition from one to another.

One of the first things any programmer might need to do in a script is conditions so here is a PowerShell condition and how it differs to c#

if ($i -eq 10)
{
    write "It's ten you know"
}

The comparison operator for equals is not "=" or "==" but a scripting style operator of -eq. So this is the first thing you’re going to have to remember.  Other operators are

-gt greater than,  -lt less than,  -ge greater than or equal to,  -ne not equal

There are more I found a simple table of them here.

When dealing with boolean responses its important to know that they are represented by $true and $false so a condition to test this would look like

if ($kidding -eq $false)

{

    write "How strange"

}

Now being good little programmers you’re bound to want to structure your scripts, so functions!  PowerShell has functions, but remember this is not a compiled language but a scripting language, this means you have to declare your functions before you can call them, this is not VBScript where you could have your main at the top and functions below, your execution code starts at the bottom after you declared your functions. 

So declaring a function can be as simple as (there are other ways but I’m sticking with this one for now)

function GoGetMyList($name)
{
    $list=$lists["MyList"]
    return $list
}

The function has a parameter of $name and returns $list.  Now the observant of you will have noticed that I created my $list on the fly giving it a scope of function level, but I access $lists whose scope is outside of the function, ok bad coding perhaps, but it’s to prove a point.

Variable scopes.  There are I believe three scopes  Global, Script, Stack (function) level.

Global scope is used in the PowerShell environment mostly you can set something of Global level in a script and it will be available on completion of the script in the scripting "Shell/Command Prompt" environment, if you are chaining scripts together this could be useful.

Script scope.  This is like a global or module level variable to your script code, once declared it’s accessible inside and outside all your functions

Stack/Function scope.   Declare a variable inside a function it’s scope and life-span are inside that function.

The weird thing is how you declare and reference these variables.  So here is how to define 3 scopes within a function

function GetBaseObjects($SiteUrl)
{

    $script:site = New-Object Microsoft.SharePoint.SPSite $SiteUrl
    $script:web = $site.RootWeb
    $script:lists=$web.Lists
    $script:list=$lists["MyList"]
    $item=$list.GetItemById(1)
    $Global:DispName=$item.DisplayName
    return $item
}

Now this is a thoroughly useless function but it demonstrates the differing levels.

$script:site = New-Object Microsoft.SharePoint.SPSite $SiteUrl

The $scipt:site  declares the variable at a script level accessible from another function or the script "main body", there is no need to pre-declare these variables because at EVERY assignment you make it sets the scope, this is a valid set of statements

function dave()
{
    $script:tellme="Script"
    write $tellme
    $tellme="function"
    write $tellme
}
dave
write $tellme

This will display "Script", "function" and "Script".  This is because PowerShell with search each scope for a variable and if not there it looks up the inherited variable scope tree, for want of better words.  The first time referenced we have a script level but the second time we have a function level and the third time the function itself is out of scope along with its variables so we access the script level one again.

So remember the order of your functions when setting scope level variables and ALWAYS when setting a variable to define the scope.

That’s all for this post, I’ll try and do another one encompassing error handling and arguments in a complete script that changes SharePoint settings.

First there are a bunch of events that can be raised, I won’t list all but here are some of the good ones

ItemAdding /  ItemAdded
ItemUpdating / ItemUpdated

There are also specific ways of dealing with the data in each of these events, those that are “ing” events then the changes have not been applied yet so you get the change values from the AfterProperties passed in, those ending in “ed” you can grab the List Item directly as changes have already happened.

My experiences have been problematic so here is a few points that may help you when using these events.

Attachments

If you want to get hold of the attachments being added to a List, you can’t. (This may differ for doc libraries I have not inspected this).  Attachments are not on the list item as it does not exist yet and they are not in the Properties either.  Anything you want to do regarding attachments you have to do in “ed” events.

ID’s

When in the “Adding” event the one thing you have not got is the List.ID value.  If you were hoping to create entries in another list that has a lookup field of this list, your out of luck. There is no ID available to you, neither is there an @@IDENTITY equivalent.

Synchronous/Asynchronous Calls

The different types of events run in different contexts. In an Adding event this runs in the context of the Add from the page itself, after hitting Add in NewForm.aspx it will execute this Adding event code under the w3wp.exe process.  This is very handy when debugging as you can attach to the w3wp process and just debug.

**I have found that Added/Updated events are not called like this they are executed, I believe, I can’t be sure, by OWSTIMER, but regardless of the process, they are not executed immediately after the Adding events.  This can be witnessed by debugging the process and placing a breakpoint on the first line of Added code, After you hit “Add” the page refreshes and shows the list, however your breakpoint code is still waiting so this is on another thread of execution to your Adding code.

**Absolute nonsense.  The Added events also fire on the w3wp process but they do fire on a separate thread.  I have had problems in the past where the Added/Updated events do not stop on my breakpoints but it is not because they happen in the timer service, it’s some other reason.  I tried it just and it breaks on both adding and added.

This sounds like no big deal, in my example I needed to format the Title field to be the ID + “Mask”.  I did not have access to the ID until Added event.  After adding the list item the list view would re-appear with a blank title.  This was unacceptable to the client so I had to populate the field with some text “to be assigned”.  Later the Adding event kicks in has access to the ID and sets the Title to the Masked value I need.  The users on refresh of the list got what they wanted.

I know its a fudge, but you can’t do it in the event you cant use calculated fields or any of those tools this is what I had, but it’s worth knowing your “ed” events happen on a separate thread.

Enable/DisableFiring

In the “ing” events above, your receiver may need to alter the data, However you will be altering the data on the List.  If in the Adding event you set a field and update the list item then it will call the Updating/Updated events.  This is not normally what you want to happen as you might get into an endless update loop.

For this very purpose the SPItemEventReceiver Class you inherited can turn off this event firing. So in your event handler you can execute

this.DisableEventFiring();

This will stop all further event firing, now here’s the rub AT LIST LEVEL, NOT LIST ITEM. So if you have a high usage list and add two at the same time, you may find you turn off eventing in one add to prevent the other list item executing the event receiver at all.  Now you say that will never happen but in my previous example the “ed” event was not firing and I was left with “to be assigned” items that never sent notifications, modified their item level security etc because of this.

this.DisableEventFiring();
listItem.Update();
this.EnableEventFiring();

Really BIG HINT, always turn it back on again immediately.

Exception Handling

If you have code that MUST execute, like setting security, and other code that must also execute but if it broke you can live with it then organise the order of execution and consider multiple update() calls.  This is not a transaction, you have no rollback if your event fails, you just have a fail. This is also not exactly efficient either.

Consider the example where you set item level security based on a fields value, then emailing the customer or whoever.

Normally you might

SPSecurity.RunWithElevatedPrivileges(delegate() { ItemAddedElevated(properties.ListItem); });

private void ItemAddedElevated(SPListItem ListItem){

    this.DisableEventFiring();    SendEmails(ListItem);

    SetSecurity(ListItem);    ListItem["Sent"] = "Yes";    this.EnableEventFiring();

}

Obviously Bad Code. No exception handling, email servers go wrong, the SMTP format may not have been validated causing exceptions in the Sendmail bit, this will error, write to the log and never be heard of again, meanwhile you have bad security in your application. Also look at all that code being executed with events turned off, and what happens if an exception occurs before event firing is turned on, I don’t know, might not be good though.

Rewrite this code

SPSecurity.RunWithElevatedPrivileges(delegate() { ItemAddedElevated(properties.ListItem); });

private void ItemAddedElevated(SPListItem ListItem){

    SetSecurity(ListItem);
    ListItem["Sent"] = "Yes";
    this.DisableEventFiring();
    ListItem.Update();    

    this.EnableEventFiring();

    SendEmails(ListItem)

}

This slight change is less likely to go wrong but it’s still not brilliant is it, a bunch of try catch finally’s would be good here, but I’ll let you figure that out.

That’s all I can think of for now.

But when things go wrong, often the only way to quickly get in a fix something in a structured, repeatable and documented manner is PowerShell.  Hell and its fun too.

So as a Sharepointy person what do you need to learn to get you quickly started.

First go and install PowerShell, this is version 1 which is what I am using, version 2 will be great but is not in production yet, so try getting that on your live servers, if they let you, sack them – pre-release code on production servers pah!.

Next go and download Quest’s PowerGUI.  This comes with the fantastic script editor, that allows stepping through code and debugging, i tend to use this for all my PowerShell development, I’m used to Visual Studio so being confronted with a command shell environment can be a little daunting.

Now if you go looking on the web at powershell there are some great resources and experts, I am not one of them I barely know PowerShell at all, I just rely on some simple things to get it working with SharePoint.

The biggest thing about PowerShell is you can just create .NET objects in it.  Yep just go ahead and create an SPSite object and away you go.

So lets dive into an example straight away

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")

$site = New-Object Microsoft.SharePoint.SPSite "http://localhost/products/MyWSSApp"
$web = $site.RootWeb
$lists=$web.Lists

$list=$lists["MyList"]

$item=$list.GetItemById(1)

$item["Title"]="Jeff"
$item.Update()
$item=$null $list=$null
$lists=$null
$web=$null
$site.Dispose()
$site=$null

This code will be executed on a machine with WSS installed on it, you can’t remote it from a box without PS and WSS installed. It’s LOCAL.

The first thing we do is load the assembly, without which PS knows nothing of the objects. 

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")

the next thing I’m doing is to create a reference to my site collection

$site = New-Object Microsoft.SharePoint.SPSite "http://localhost/products/MyWSSApp"

The New-Object statement is PowerShell’s way of creating a .NET object, the C# equivalent would be

SPSite site = new SPSite(http://localhost/products/MyWSSApp)

There is no need to declare the type in PowerShell, although it is possible to cast types, which you will need later on in your Scripting, but not today.

All variables in PowerShell are prefixed with a $ sign.

In the next sequence of statement you can see I easily manipulate the .NET Sharepoint object model from within PowerShell.  I can grab the RootWeb and assign it to a new PowerSHell variable and likewise with the list and a list Item.  Then a simple data change and list update.  All things you will be used to doing in C# can be done easily ni PowerShell to.

$web = $site.RootWeb

$lists=$web.Lists

$list=$lists["MyList"]

$item=$list.GetItemById(1)

$item["Title"]="Jeff" $item.Update()

Notice that I still deliberately DISPOSE of the site, this is still important to do even in powerShell.

Now I’m not sure if you can do security in the ONET but for my requirements I have $Resources that need concatenating together to form the group names (don’t ask).  Which it can’t do so it has to be done as a site provisioning thing.

What I noticed though is in my list, "of parent SPWEb of Parent SPWeb(RootWeb)" things where not as they were supposed to be.

The security on the list was completely wrong, as in it stated it had Unique permissions, when actually it does not, it will have when I’m finished with it, but at Site Provisioning time it should not.  Also all the groups were relating to the Site Collection level not the custom groups of the sub site it should have inherited.

It’s as if none of the list processing for fixing up the security on the list has happened, as it had done previously on SPWeb objects.

After Large amounts of digging I found that at time of Site Provisioning the List.Update() method has not been called. Calling this on the list before I do my processing, resolves all security issues.

So if having trouble provisioning list security remember the lists still in flux till you call the Update, whereas the SPWeb objects are not.  Weird, but true.

The SiteDef had some SiteProvisioning code, it read an XML file to do additional processes as part of the site provisioning, rather handy in some cases.

But I did notice a few oddities.

In SiteProv he was creating lists.  Also He was copying files from the file system into Lists.  Some users just don’t have access to this part of the file system and this was exploding.

Two things about this technique,

• 1st. You can Provision Lists in the ONET.
• 2nd You can copy files into lists in the ONET.

The first item he should have known, so WHY do it as part of site provisioning event ?  I don’t know. Perhaps something to do with sequence, perhaps stupidity .

Second I can forgive, someone not trained in SharePoint might not know exactly what the <Module> Tag is really for.

Think of the <Module> tag in Features and SiteDefs as a kind of File copy as well as configuring what webparts go where on a page, it can copy files from the file system into a DocLibrary, but most importantly it has the correct rights to do it.

Some extra info on ONET’s and Modules.

From all the examples you have seen in existing SiteDefs the most likely arrangement would be something like this

 <Project>
    <Configurations>
        <Configuration ID="0" Name="Default">
            <Lists>
               <List FeatureId="00BFEA71-E717-4E80-AA17-D0C71B360101" Type="101" Title="MyPlaceForFiles" Url="MyPlaceForFiles"/>
            </Lists>
            <Modules>
                <Module Name="Default" />
            </Modules>
        </Configuration>
    </Configurations>       
    <Modules>
        <Module Name="Default" Url="" Path="">
            <File Url="default.aspx" NavBarHome="True">
                <AllUsersWebPart WebPartZoneID="LeftZone1" WebPartOrder="0">
                    <![CDATA[.....WebPart Stuff.......]]>
                </AllUsersWebPart–>
            </File>
        </Module>
    </Modules>
</Project>

So in the example I mentioned where I want to copy a file from my SiteDef into a List would go something 

<Project>
    <Configurations>
        <Configuration ID="0" Name="Default">
            <Lists>
               <List FeatureId="00BFEA71-E717-4E80-AA17-D0C71B360101" Type="101" Title="MyListForFiles" Url="MyPlaceForFiles"/>
            </Lists>
            <Modules>
                <Module Name="Default" />
                <Module Name="WhatASecondModule" />
            </Modules>
        </Configuration>
    </Configurations>       
    <Modules>
        <Module Name="Default" Url="" Path="">
            <File Url="default.aspx" NavBarHome="True">
                <AllUsersWebPart WebPartZoneID="LeftZone1" WebPartOrder="0">
                    <![CDATA[.....WebPart Stuff.......]]>
                </AllUsersWebPart–>
            </File>
        </Module>
        <Module Name="WhatASecondModule" Url="MyListForFiles" Path="FolderItCameFromOnDiskInSiteDef" RootWebOnly="FALSE">
            <File Url="SourceFileIWantToChangeOnFlyInDocLib.js" IgnoreIfAlreadyExists="TRUE" Type="GhostableInLibrary"/>
        </Module>
    </Modules>
</Project>

The thing to notice here is that there is a Second Module entry in the Config Section, this module has a different function to the standard one so needs a second declaration.  The rest of the syntax in the module is straight from the documentation, I have a folder to copy to (URL) in this case its a DocumentLibrary, I have a Path that the file on the disk lives, in my case I have a sub folder in the SiteDef to store this file so I put that as its Path, if its in the root then leave it blank, Then in the <FILE> tag I define what files to copy etc.

I’ve decided to call the WSP solution packages for WSS, Wasps.

It really appeals to me that I can send our support department a bunch of wasps.