Sitecore Fetch Squad

The following tips for working with Sitecore shared source are based on my recent experience contributing the FieldValueComparer (based on to the comparer in this post) to the Sitecore Shared Source project.

• Review this introduction to contributing shared source to obtain access to Subversion or to request a new project room.
• Each project room is basically a Wiki. The only way back to the list of project rooms seems to be http://trac.sitecore.net/Index, so most project rooms include this link.
• Review this guide to contributing shared source.
• Copy files into the Trunk directory and check them in.
• I don’t like learning a new markup, but when editing the Wiki:
  • It seems to format linefeeds as HTML paragraphs or line breaks.
  • Use equals signs for major headings (the spacing seems to be important): 

    = heading =
    

  • Use more equals signs for minor headings:

    == heading ==
    

  • Preceed text with an exclamation mark to avoid the Wiki automatically creating a link:

    = Welcome to the !ProjectRoomName =
    

  • Wrap URLs with quare brackets, optionally followed by a space and the text of the link:

    [http://sitecore.net Sitecore Marketing Site]
    

  • Wrap code with three curly braces to preformat it for easy copy and paste:

    {{{
    // code
    }}}
    

  • You can create entries in numbered lists by entering a space character, a number, and a period on a blank line:

     1. First.
     2. Second.
    

• Please report issues and feature requests for shared source components by posting on the Shared Source Modules forum on the Sitecore Developer Network.

I will try to maintain this list and the FieldValueComparer project room as a template for cutting and pasting when creating new projects.

Applies to 32 bit Windows

As many a developer has experienced memory is not an unlimited resource when using In-process memory like the ASP.Net cache. Ruuning out of it shows symptoms like the process recycling or OutOfmemory and MemoryLimit Exceded Exceptions here and there. Having a basic understanding of what is going on will help you a lot resolving problems like this.

Fact: In a standard setup your worker process always have 2GB Virtual memory available (no matter if you have 1, 2 or 4GB physical memory in the machine). If you use 1.5GB memory in your application but your machine only has 1GB physical memory, some of it is just swapped to disk. Thats why it is virtual memory. This also means that if you are running out of (virtual) memory it doesn’t help to add more physical memory.

So why am I limited to 2GB virtual memory?: Well, a 32bit operating system (OS) is only able to address 4GB memory with these 32 bits. In a standard setup 2 of these are reserved for the OS, and the remaining 2 reserved for applications like ASP.Net.

A common error in memory heavy applications is the OutOfMemory Exception (OOM). It happens when the framework is unable to allocate enough virtual memory to run a piece of code. This is a bad thing because they can be thrown almost anywhere in your code, making it hard to predict the outcome. What happens is that the framework lost a battle against custom code about access to the limited 2GB memory.

Microsoft solved this in IIS 5.0 by adding the setting memoryLimit to the machine.config file. This setting tells how many of the 2GB the workerprocess (custom code) should be allowed to use. The rest of the 2GB is then free for the Framework to use for executing code. In IIS 6.0 the Maximum used memory (in megabytes) property on the application pool is used for the same purpose. Where the setting is quite straightforward in IIS 6.0, this is not the case in IIS 5.0 where it is a percentage of physical memory. Say you have a machine with 2GB memory and the default setting is 60 (%) equalling 1.2GB. If you now add an extra 2GB for a total of 4GB, the 60% now equals 2.4GB out of 2 possible, which of course does not make sense. So make sure to review this setting in the beginning as well as at every change of physical memory.

So how many megabytes should be allowed for the workerprocess?: Microsoft recommends to “set the memory limit to the smaller of 60% of physical RAM or 800 MB“. Here is the explanation why. On almost anything but a 1GB machine 800MB will always be the smaller of the two numbers. So 800MB is the recommended setting for 2GB+ machines. Not really a lot of available cache memory if you have a machine with 4GB physical memory and 2GB virtual.

Microsoft argues: “There are a couple things to consider: First, the likelihood of experiencing an OutOfMemoryException begins to increase dramatically when “Process\Virtual Bytes” is within 600 MB of the virtual address space limit (generally 2 GB), and secondly, tests have shown that “Process\Virtual Bytes” is often larger than “Process\Private Bytes” by no more than 600 MB“

If you search and read long on the net, you will eventually find that when you set the memoryLimit setting, the value it is compared against is the performance counter “Process\Private Bytes“, which you force to stay below 800MB. The logic is: 800MB plus 600 MB (that Virtual Bytes may be larger than Private Bytes) + 600MB (which is needed as buffer to avoid OOM’s) = 2000MB = 2GB which is the available amount of virtual memory. So if you set memoryLimit higher, you start risking OOM’s. Thats the logic behind.

So what will happen if you use more ASP.Net cache than is allowed through the memoryLimit? The system will throw a “memoryLimit Exceeded” exception and recycle the worker process to release memory. If you want to avoid this, you should make sure that Private Bytes never exceed the memoryLimit.

The last topic I will cover here is the 3GB switch which is available on some versions of Windows, but not all (none mentioned by purpose). This setting  makes only 1 GB available for the OS, and therefore leaves 3GB for applications like the worker process. Through the same maths as we used to determine the 800MB limit above, we can calculate that this setting allows a memory limut of 1800MB, as this still leaves 1200 MB for the rest. Below I have inserted a picture visualizing this:

Links: http://msdn2.microsoft.com/en-us/library/ms972959.aspx#monitor_perf_topic12

http://msdn2.microsoft.com/en-us/library/ms998583.aspx#scalenetchapt17_topic14

childrenRepeater.DataSource = Context.Database.SelectItems(“/sitecore/content/home/*[@@templatename='DisplayBox']“);

//C# 1.0 way, using a standard foreach loop

ArrayList list = new ArrayList();

foreach (Item item in Sitecore.Context.Item.Children) {
if (item.TemplateName == “DisplayBox”)
list.Add(item);
}
childrenRepeater.DataSource = list;

//C# 2.0 way, using the iterator pattern (yield statement),and anonomouse delegate

childrenrepeater.DataSource = delegate {

foreach (Item child in Sitecore.Context.Item.Children) {
if (child.TemplateName == “DisplayBox”)
yield return child;
}
};

//C# 3.0 way using the query syntax
childrenRepeater.DataSource = from Item child in Sitecore.Context.Item.Children
where child.TemplateName ==“DisplayBox”

select child;

childrenRepeater.DataBind();