Monday, September 6, 2010

Sitecore Fetch Squad

Automated crawler fetching websites and blogs from Sitecore content

Archive for the ‘Sitecore Blogs’ Category

Integrated Pipeline Update

Posted by admin On March - 9 - 2009
Bit busy, so just a very quick update.

The Integrated Session Fix I wrote about a couple of weeks back has now passed Sitecore QA and is now made available on SDN. From personal experience I can concur, I’ve been running it myself on the websites I am working on and not had any problems (other than those already known and mentioned on SDN, specifically that IIS would recycle my pool often if not moving the Index folder out of the Web Root).

I fully realise that me running it on a handful of non-live sites doesn’t qualify as QA however, so I’m glad to now see this somewhat officially endorsed :-)

The original post has also been updated.

Into the core

Sitecore Continuous Integration Continued

Posted by admin On March - 5 - 2009

Here’s an update on the post I wrote a year ago on our Continuous Integration project here at Pentia.
To put it short: it’s turned out to be one of the most successful internal projects in our company. In the last year we’ve added 20+ new projects into the system, and we are in the process of moving our old projects from SourceSafe to SVN, and creating build files for them.
Our setup is not a conventional build setup with specifically designed build scripts for a major project, it’s more of a build framework for the many Sitecore project passing through Pentia. We’ve designed our setup around our naming and location best practises, so in order to add a project to the build system, we only need to create a single file with about 10 lines of XML specifically for the project.

Our entire setup consist of:

1: nAnt build framework
This is a set of approx. 30 nAnt scripts which is shared across all our projects. The scripts handles everything from SVN, MSBuild, FxCop, deployment, zipped releases, automatic versioning and much more. The files are versioned so that changes and new features will not influence the older projects and break the builds. In the last year we have released 5 versions of the nAnt scripts.

2: Subversion version control
In my approx. 15 years of programming I’ve only worked with Visual SourceSafe as version control (and as those of you who knows the product knows, is hasn’t changed for 15 years either :-) , so the switch to SVN has been a daunting task for me. After a year, my only recommendation for those of you who consider it is to go for it! Subversion is easy to install and maintain (I recommend VisualSVN Server), easy to use (TortoiseSVN) and the integration into Visual Studio (VisualSVN) is perfect. Naturally the integration with nAnt is nice too as. E.g. it made it possible for us to automatically version our projects when releasing, by extracting the latest revision from SVN.

3: CruiseControl.NET
The hardest part to integrate into the company has been the CruiseControl.NET server. It’s not difficult to explain its purpose to the developers (to continuously check the projects on commits in SVN and build nightly releases), but the benefits of the other parts of the setup, for the individual developer has just been so much more evident. But still, its running, checking our builds and keeping our internal testing environment updated, by releasing each night.

4: Module library
The module library is a really neat feature of our build system. In short, it allows us to only maintain our own codebase in a project in version control and still allows developers to get quickly up and running, not making it necessary to install Sitecore version and thirdparty tools on their local machines. In the last year we’ve added 12 versions of Sitecore, 16 Sitecore modules (with 3+ versions each), 10+ third party modules like PDF, FTP and unit testing, and much more. The library allows us not only to add our old projects to the build system, but the cool thing is that when e.g. a new update to Sitecore is released, all our projects can be upgraded by changing a single line in the project build files.

5: Configuration merging
Both Sitecore and Microsoft has acknowledged the problem of mainting large configuration files. And the both have features which allows config changes to be merged from external files. The only thing they do not make easy, is to have different configurations for the different environments. For this we’ve developed a nAnt extension which allows the developers to create configuration merge files (configmerge files) which can be varied according to the environment we are building to. This has made a huge difference as opposed to juggling different configuration files for production/test/training/development environments for frontend and backend servers (6-7 web.configs!).

6: GUI tools.
We have lazy developers in Pentia :-) Joke aside, we quickly acknowledged that in order to successfully get the system to the developers, we had to provide them with better tools than the nAnt command-line and CruiseControl xml files. So we created a nice GUI for our nAnt scripts and for editing our CruiseControl server.


As you can hear, I am still pretty excited by the accomplishments we’ve made in the last year - and we have a lot of cool features waiting. Our next release will incorporate generation of SQL scripts for mounting Sitecore databases and setting up database access and automatic IIS setup. We also want to automate our releases even more, by generating batch files for folder security and automated FTP upload to the production servers.

I’ll still be happy to hear from anyone with experience or ideas - or you can give me or Pentia a shout if you want to hear more about our setup.

Molten Core

Top 10 Must-Read Books for Personal Success

Posted by admin On March - 5 - 2009

I love reading books. Usually that means one, sometimes two or three books per week. In the last four years I extended my personal library to several self-development and management books and I have to say, I’m glad that I read most of them. Below you can find my personal top ten list in those two categories:

Getting Things Done
The 7 Habits of Highly Effective People
Brain Rules
Slowing Down to the Speed of Life
How To Be Brilliant
Behind Closed Doors
Good to Great
Speed Reading for Professionals
First Things First
Mind Performance Hacks

Additionally I own several other books that I haven’t read yet but that might make it to this list, maybe even replacing one or two of the ones listed above. Here are those I’m eager to read:

Think And Grow Rich
How To Win Friends And Influence Others
Your Money or Your Life
The Power Of Less

For a complete list of all my books, please visit my website at www.juliusganns.com.

Julius Ganns . netzkern

Developing with today’s platforms, a developer can utilize great tools and libraries to increase development speed and software quality. Two very common in the .NET space are WCF and LINQ to SQL, besides various other O/RM tools like NHibernate and the ADO.NET Entity Framework.

Unfortunately, using a messaging framework like WCF that has to perform quite a lot serialization work in combination with a tool to automatically retrieve data from data sources and create deeply nested object graphs like LINQ to SQL creates some pretty hard challenges. The reason for this is that WCF has to transform the graph into a “flat” message representation. If the graph contains cyclic references, WCF has two options to avoid running in circles (and kill your App) when it encounters an object it already has processed:

  1. Abort the whole serialization. This is the default behavior and it is very reasonable to throw an exception here.
  2. “Mark” that part of the message as “reference” to an earlier serialized part. This behavior can be introduced using your own WCF components (see the links at the bottom).

For service developers that want to use object-relational mapping tools to automate database queries, neither of one is really sufficient. Although the latter part may be an option in a strict .NET to .NET scenario, it does not implement any standard (because there is none) and thereby makes it nearly impossible for other platforms to use the serialized message with reasonable effort.

Even worse, in the Web 2.0 area more and more WCF services create JSON and XML output for client-side components like ASP.NET AJAX, JQuery, Silverlight and Flex. Therefore it is essential to provide client-side developers with all necessary information in a strctured way without the need to blow up the server side development.

Taking a look at the .NET framework, Microsoft currently does not offer any solution for this in its primary frameworks WCF and LINQ.

After re-designing and overworking our Reference Architecture over the last week, we finally managed to develop an approach here at netzkern to integrate both technologies with each other in a pragmatic and pretty “clean” way. We created a utility called the DataTransferObjectManager that is able to dynamically fill DTOs with the values and copies of all requested references. The trick here is that it does only selectively chooses the right references by letting the client-side developer provide a configuration parameter that indicates which properties he really needs to do his work. All other references are ignored and not returned to the client, they are usually not even loaded from the database.

Let’s have a look. First, we create two entity classes as part of our model. They don’t need to be public as they are never used outside of the core assembly.

namespace App.Core
{
    class House
    {
        public string Name { get; set; }
        public Person Owner { get; set; }
    }

    class Person
    {
        public string Name { get; set; }
        public House Residence { get; set; }
    }
}

Let’s pretend that instances of these classes are being tracked by an O/RM manager component that automatically detects changes and performs updates. Now, let’s create the corresponding DTO’s:

namespace App.DTO
{
    public class HouseDTO
    {
        public string Name { get; set; }
        public PersonDTO Owner { get; set; }
    }

    public class PersonDTO
    {
        public string Name { get; set; }
        public HouseDTO Residence { get; set; }
    }
}

These classes are basically message definitions. In WCF you would use these as DataContracts and annotate them with the appropriate attributes. Instances of these classes are somewhat “dumb”, they only contain data.

Now let’s consider, we have the following object graph:

House h1 = new House();
h1.Name = “House No. 1”;
Person p1 = new Person();
p1.Name = “Person A”;
h1.Owner = p1;
p1.Residence = h1;

This creates a graph with just two objects and a cyclic reference. When you try to serialize “h1” using WCF’s default DataContractSerializer, you get the above mentioned exception. Now let’s use the netzkern.DataTransferObjectManager (or nk.DTOM for short) and see what it can do for us:

// Create an empty base DTO.
HouseDTO h2 = new HouseDTO();

// Call the nk.DTOM and copy the contents of h1 to h2.
DataTransferObjectManager.CreateDataTransferObject(h1, h2);

What we get from this is the following “filled” DTO message object:
h2.Name == “House No. 1”;
h2.Owner == null;

So far we created a flat copy of “h1” with all references excluded. Now let’s include a copy of “h1.Owner” as part of “h2”:

// Create an empty base DTO.
HouseDTO h2 = new HouseDTO();

// Tell the nk.DTOM to include the following properties.
PropertyPathCollection includePaths = new PropertyPathCollection();
includePaths.Add(“Owner”);

// Call the nk.DTOM and copy the contents of h1 to h2.
DataTransferObjectManager.CreateDataTransferObject(h1, h2);

Our result now includes a copy of “p1” that has been mapped to a PersonDTO type:
h2.Name == “House No. 1”;
h2.Owner == [PersonDTO];
h2.Owner.Name == “Person A”;
h2.Owner.Residence == null;

We can now even include “deeper” properties, that are being copied equally by the nk.DTOM:

// Create an empty base DTO.
HouseDTO h2 = new HouseDTO();

// Tell the nk.DTOM to include the following properties.
PropertyPathCollection includePaths = new PropertyPathCollection();
includePaths.Add(“Owner.Residence”);

// Call the nk.DTOM and copy the contents of h1 to h2.
DataTransferObjectManager.CreateDataTransferObject(h1, h2);

That creates the following result:
h2.Name == “House No. 1”;
h2.Owner == [PersonDTO];
h2.Owner.Name == “Person A”;
h2.Owner.Residence == [HouseDTO];
h2.Owner.Residence.Name = “House No. 1”;
h2.Owner.Residence.Owner = null;

So, how do we use this in WCF? After you have successfully loaded entities in your service operation using an O/RM, you just call the nk.DTOM and return the DataTransferObject as result. The trick is to let the client(!) choose which parts of your internal object graph are necessary to do its work. A simple service call might look like this:

IHouseService service = new HouseServiceClient();
HouseRequest req = new HouseRequest();
req.HouseID = houseID;
req.IncludeProperties = “Owner.Residence”;
HouseResponse resp = service.GetHouse(req);

That’s basically it. By using the netzkern.DataTransferObjectManager, you can define your data transfer objects independently of your internal entity objects. You can even omit certain properties and nk.DTOM will ignore them. We will now test this component internally and make it available on CodePlex as soon as it reaches BETA status. We also would love to hear your thoughts, just send a mail to blog@juliusganns.com.

Julius Ganns . netzkern

Redirect Action Tutorial

Posted by admin On March - 5 - 2009

Introduction

In my opinion the WebForms for Marketers module is one of the nicest Sitecore modules out there. It’s beautifully designed and serves its purpose well. It comes bundled with a number of actions (which are executed after you submit), but one action is missing - an action which redirects the user to another page after the form has been submitted.

I was interested in how it works to set up a new action and thus I set out to create this action myself. Apart from just letting you download the package (here) I will explain what I did, step by step, because over and over again I find that certain XAML features are not documented and maybe this post will help some of you.

Let’s have a look at what we need to create this action:

  1. An item for the action itself and a corresponding class
  2. An editor window that allows the user where to redirect to

Obviously we also need to have the Web Forms for Marketers module installed and we need a valid license for it.

Creating the action and class

To create a new action, we need to create a .NET class in our solution, plus an item in the content tree under /sitecore/system/Modules/Web Forms for Marketers/Settings/Actions. The item is easily created and doesn’t need a lot of configuration for now, apart from the class and assembly names (these depend on your solutions setup).

Our class could look like this:

public class RedirectAction : ISubmit
{
  public void Submit(ID formid, Sitecore.Form.Core.Client.Data.Submit.AdaptedResultList fields)
  {
    Sitecore.Data.Database curDB = Sitecore.Context.Database;
    Sitecore.Data.Items.Item redItem = curDB.GetItem("sitecore/content/home");
    Sitecore.Context.Item = redItem;
  }
}

The class is implementing the ISubmit interface and overrides the Submit method. All this class does at the moment is redirect the user to /sitecore/content/home. Simple, but for now this is exactly what we want.

That’s it. Once the solution is compiled, the action can now be added to any form and after the form has been filled out, the user will be redirected to /sitecore/content/home.

Extending the solution with parameters

Currently the redirection path is hardcoded and we want to give our users the ability to choose a path for themselves (and a different one for each form).

To do this we need to implement an XML control which serves as the property window for our action. The property window is called when the user presses "Edit" in the action editor.

redirect_edit

To enable the "Edit" button, we first need to tell Sitecore what to do when the button is pressed. We do this by filling out the "Editor" field on the redirect action item itself, pointing the editor to control:pickItem (see below). This will automatically call and display the Sitecore XML Control pickItem (it is important that the XML node of the control is called pickItem, e.g. <control …><pickItem>…</pickItem></control>, but more about this in a second).

redirect_item

Now we have to create the pickItem control itself. I placed the pickItem.xml file into C:\inetpub\wwwroot\<yoursite>\WebSite\sitecore\shell\Applications\Modules\Web Forms for Marketers\Dialogs\Action Editor and the codebeside class myItemPickDialog.cs into C:\inetpub\wwwroot\<yoursite>\WebSite\starterkit. The locations don’t really matter though.

pickItem.xml:

<?xml version="1.0" encoding="utf-8" ?>
<control xmlns:def="Definition" xmlns="http://schemas.sitecore.net/Visual-Studio-Intellisense">
  <pickItem>
    <Stylesheet runat="server" Src="Content Manager.css" DeviceDependant="true"/>
      <FormDialog Icon="Applications/32×32/exchange.png" Header="Pick Item" Text="Select the item that you want to redirect to." OKButton="OK">
      <DataContext ID="TreePickerDataContext" />
      <CodeBeside Type="Sitecore.Starterkit.starterkit.MyItemPickDialog, Sitecore.Starterkit"/>
      <TreePicker ID="treepicker1" />
    </FormDialog>
  </pickItem>
</control>

This is our pickItem control. As we can see, it’s a standard XML Control Form Dialog with just one extra control - TreePicker. A TreePicker control is a dropdown which displays a tree once the arrow in the right is clicked. In addition we need to add a DataContext control and a reference to our codebeside file (full class name, assembly).

myItemPickDialog.cs:

public class MyItemPickDialog : DialogForm
{
  protected TreePicker treepicker1;
  protected DataContext TreePickerDataContext;
  protected override void OnLoad(EventArgs args)
  {
    // If this is not a postback…
    if (!Context.ClientPage.IsEvent)
    {
      // initialise DataContext
      TreePickerDataContext.DataViewName = "Master";
      // set root node for DataContext - only content items in this case
      Database mdb = Sitecore.Context.ContentDatabase;
      TreePickerDataContext.Root = mdb.GetItem("/sitecore/content").ID.ToString();
      treepicker1.DataContext = TreePickerDataContext.ID;
    }
    // without calling this, the OK and Cancel buttons will NOT work
    base.OnLoad(args);
  }

  protected override void OnOK(object sender, EventArgs args)
  {
    // add the current value of the treepicker to the parameters collection and close dialog
    NameValueCollection nvParams = new NameValueCollection();
    nvParams.Add("redirectItem", treepicker1.Value);
    string result = ParametersUtil.NameValueCollectionToXml(nvParams);
    SheerResponse.SetDialogValue(result);
    base.OnOK(sender, args);
  }

  protected override void OnCancel(object sender, EventArgs args)
  {
    base.OnCancel(sender, args);
  }
}

This class initialises the FormDialog by first registering all the controls and then (in the OnLoad method) sets the DataContext for the TreePicker control. At the end it calls the OnLoad method of its parent class, initialising the whole form. This is necessary, otherwise the Ok and Cancel buttons will not work (yes, I found out the hard way :) ).

The only really interesting method here is OnOk. This method is called, when the OK button on the form is clicked and stores the TreePicker selection (the item to which we want to redirect) into a NameValueCollection with they name "redirectItem", which is later accessible by the form action itself.

redirect_control

The result is the beautiful form shown above, all rendered using Sitecore’s Sheer UI.

Now that we have the information on where to redirect to, we need to rewrite our RedirectAction class to actually not redirect to /sitecore/content/home anymore, but to whatever item the user has picked.

public class RedirectAction : ISubmit
{

  // Sitecore will automatically initialise this string for us
  public string redirectItem { get; set; }

  public void Submit(ID formid, Sitecore.Form.Core.Client.Data.Submit.AdaptedResultList fields)
  {
    Sitecore.Data.Database curDB = Sitecore.Context.Database;
    Sitecore.Data.ID redirectItemID = Sitecore.Data.ID.Parse(redirectItem);
    Sitecore.Data.Items.Item redItem = curDB.GetItem(redirectItemID);
    Sitecore.Context.Item = redItem;
  }
}

This is very basic - if you know that Sitecore automatically initialises the redirectItem string. We specified the name earlier in the myItemPickDialog’s OnOK method, remember? So once you know this, it’s a piece of cake.

Conclusion

The conclusion is, as always with Sitecore - everything is possible. You can obviously write any kind of form action you like, from a redirect action over an SMS action to a workflow trigger action. As long as you can do it in .NET, you can do it in Sitecore.

I want to thank Igor Zhukovsky, who initially developed the Web Forms module and was very helpful during my little coding exercise. I also want to thank Yan Sklyarenko for facilitating the contact with Igor.

Links of Interest

Sitecore Developer Network - My First Sheer/UI App

Sitecore Developer Network - Publishing Queue Viewer (great advanced Sheer UI/XAML tutorial)

Philipp Heltewig

The http://trac.sitecore.net/LinkProvider/ Sitecore Shared Source project provides many of the features described in this post.

John West Blogs about Sitecore

Here are the Sitecore Shared Source projects to which I have recently contributed:

Here are some suggestions for Sitecore shared source contributions:

  • Use C#.
  • Use Resharper, Stylecop, Agent Smith, and Agent Johnson as described in this post.
  • Resolve all code validtion issues before checking in any file.
  • Keep line length below 80 characters when possible.
  • Include the URL of the shared source project in each source code file.
  • To make your code easier for other developers to read, avoid ternary expressions.
  • To make your code easier for other developers to read, fully qualify classes, at least on the first use of each class in each method.
  • To make your code easier for other developers to read, avoid use of the var keyword.
  • Use protected instead of private where possible.
  • Use namespaces within the Sitecore.Sharedsource namespace that correspond to system and Sitecore namespaces, such as Sitecore.Sharedsource.Web.UI.WebControls.
  • Create directories as required to check in files from directories within the Subversion Trunk folder that correspond to the namespace, such as /Web/UI/WebControls for files in the Sitecore.Sharedsource.Web.UI.WebControls namespace.
  • Place extension methods in classes named after the method plus the word "Extension" in folders named after the original class. For example, to extend the Sitecore.Sites.SiteContext class with a method named GetSomething(), create /Sites/SiteContext/GetSomethingExtension.cs containing the class Sitecore.Sharedsource.Sites.SiteContext.GetSomethingExtension containing the GetSomething() method. Use namespaces such as Sitecore.Sharedsource.SystemString for extensions to system classes. For example, to extend System.String with a method named GetSomething, create /SystemString/GetSomething.cs containing the class Sitecore.Sharedsource.SystemString.GetSomethingExtension containing the GetSomething() method.

Also, I learned that to nest lists in the wiki, you indent with spacing. So:

 1. Item One.
  * Item One Bullet One.

This post has some other Wiki formatting and shared source contribution suggestions.

John West Blogs about Sitecore

Two for the bookmarks

Posted by admin On February - 24 - 2009
Some of this information has been available for a while, it just hasn’t been easy to locate. It actually still isn’t ;-) But at least these two links look like they might be more or less permanent.

1) Sitecore 6 Release Notes

Gives you all information about known current issues. A big help, which should help you avoid getting too stuck trying to solve a particular issue - only to find that it is already known, and in some cases even comes with a workaround.

2) Sitecore 6 Change Log

Detailed information about the individual updates, so you can now be confident when choosing whether to go with the Recommended Release or the Latest Release.

Into the core

Sitecore 6 Service Release 1 is Out

Posted by admin On February - 24 - 2009

Sitecore
6 Service Release One
is officially out by the name of Sitecore 6.0.1 rev.090212.

A
massive list of fixed issues
, props to our documentation team for providing nicely
formatted change log.

I’ve done my share of work on it, so I’m just as excited to see it being released. Yay!

This is not a recommended release yet, as we require builds to be used in production
for some time to ensure stability first. So make sure to read this notice: It
is appropriate for use if it contains fixes for issues that you encounter with the
recommended release.

Alexey Rusakov on Sitecore development

Sitecore MemoryMonitorHook Threshold

Posted by admin On February - 23 - 2009

I spent February 18 through 20 with a customer experiencing performance issues with a Sitecore 5.3.1 solution. We identified a configuration problem within the first few hours, but I stayed on the project for a little while to monitor the solution.

As most Sitecore developers are aware, Sitecore is thoroughly optimized for performance, relying heavily on caching. Caching consumes memory, and caching requirements for a solution with significant data or load can exceed available memory. Out of memory conditions can cause exceptions and can cause ASP.NET to crash or restart, resulting in errors, delays, and outages.

Sitecore provides a memory monitor hook to flush caches when memory exceeds a given threshold. You configure the memory monitor hook using the /configuration/sitecore/hooks section of web.config:

<hooks>
  <hook type="Sitecore.Diagnostics.HealthMonitorHook, Sitecore.Kernel" />
  <hook type="Sitecore.Diagnostics.MemoryMonitorHook, Sitecore.Kernel">
    <param desc="Threshold">500MB</param>
    <param desc="Check interval">00:00:05</param>
    <param desc="Minimum time between log entries">00:01:00</param>
    <ClearCaches>true</ClearCaches>
    <GarbageCollect>true</GarbageCollect>
    <AdjustLoadFactor>true</AdjustLoadFactor>
  </hook>
</hooks>

The first parameter (the first <param> element) to the constructor for the Sitecore.Diagnostics.MemoryMonitorHook class specifies the threshold for the memory monitor hook, which is 500MB by default in Sitecore 5.3.1. Sitecore frequently checks the amount of memory consumed by ASP.NET and if it exceeds this limit, then Sitecore will flush caches. The following message repeated frequently in the logs should have made this pretty obvious:

WARN  Memory usage exceeded the MemoryMonitor threshold. All caches have been cleared and a forced GC has been induced.
WARN  Memory used before/after GC: 554,590,208 / 554,590,208

If you see a message like this, your cache may be too small. If you see it constantly, your cache may be way too small, especially if the before/after values are the same.

The problem for this customer was that the amount of memory consumed by ASP.NET was always greater than 500MB. Therefore, Sitecore was flushing caches constantly. It’s somewhat amazing that this didn’t completely overwhelm the content delivery and the database servers.

The solution is quite simple: either set the memory monitor threshold to an appropriate value, or set the ClearCaches property to false. In this case, we set the size of the cache to 1792MB and afterwards were unable to reproduce the issue.

Note that in Sitecore 6, the default value for the memory monitor threshold is higher (800MB), but it shouldn’t matter anyway because cache clearing is disabled by default:

<hooks>
  <hook type="Sitecore.Diagnostics.HealthMonitorHook, Sitecore.Kernel" />
  <hook type="Sitecore.Diagnostics.MemoryMonitorHook, Sitecore.Kernel">
    <param desc="Threshold">800MB</param>
    <param desc="Check interval">00:00:05</param>
    <param desc="Minimum time between log entries">00:01:00</param>
    <ClearCaches>false</ClearCaches>
    <GarbageCollect>false</GarbageCollect>
    <AdjustLoadFactor>false</AdjustLoadFactor>
  </hook>
</hooks>

I would like to thank Alexey Romaniuha of the Sitecore Support team for his identification of the issue.

John West Blogs about Sitecore