Friday, July 30, 2010

Sitecore Fetch Squad

Automated crawler fetching websites and blogs from Sitecore content

(Version 1.0 - January 19th, 2009)

In this series, I’m going to compare the functionality of SharePoint and Sitecore from a developer perspective in each area and both candidates can collect “points” from 0 (has nothing to contribute) to 10 (great integration and functionality).

This series will cover the following areas:
- Introduction (Part 1)
- Installation and General Architecture (Part 2)
- Configuration (Part 3)
- Customization and Designer Experience (Part 4)
- Integration with Developer Tools (Part 5)
- Feature Deployment (Part 6)
- API und Data Model (Part 7)
- Integration with other systems (Part 8)
- Conclusion (Part 9)

Integration with Developer Tools

In the last part of this series I examined features and tools both framework offer in terms of customization and web designer support. This part of the series is diving deeper into the world of web development with SharePoint and Sitecore.

One of the main reason for an organization to invest in enterprise-class Content Management Systems is to build extended functionality on top of those frameworks. At netzkern we have several customers that made significant investments into their IT infrastructure to enable enterprise application and business process integration, especially using open standards like XML (SOAP, REST, POX). Because of that, extensibility and integration are critical parts of Content Management Systems today and both SharePoint and Sitecore have various integration features and extension points. The question is, however: How easy is it to use those features and interfaces as developer to plug in my own code? Do I need to change the way I’m used to work? And how does the system support me?

As SharePoint comes as part of Microsoft’s server family, the tool to develop SharePoint functionality is of course Visual Studio. So, how do we get started? Well, as developer with some experience in the Microsoft space, you might expect the following procedure: Install SharePoint, install Visual Studio, download “the SDK” (in this case the so called “Windows SharePoint Services 3.0 Tools: Visual Studio 2005 Extensions”, formerly known as “Visual Studio extension for Windows SharePoint Services” -or- “Windows SharePoint Services 3.0 Tools: Visual Studio 2008 Extensions, Version 1.2″), take a short look at some API samples, write three lines of code, hit “F5″ and see your first shiny “Hello World”. Unfortunately, that’s not the way it works with SharePoint.

Developing custom SharePoint solutions is surprisingly complex and terrifyingly underwhelming - at least if you’re used to the .NET world. First of all, to install either MOSS2007 or WSS 3.0, you need a Windows Server operating system. It is not possible to install one of them on WinXP or Vista. As developer you usually choose one of the following solutions: Install Windows Server on your workstation, use a virtual machine (VMware, Virtual PC, etc.) with Windows Server or use a central Windows Server with Remote Desktop for development. Although you can use Visual Studio on your WinXP for local development, you need to deploy your code on a server machine for execution and remote debugging (which, by the way, is not that straightforward in most scenarios). Annoying? Absolutely… but it even gets worse.
The by far most disappointing limitation in SharePoint is the fact, that even if you have a Windows Server with VisualStudio and all necessary extensions installed, you cannot just create a new “SharePoint Project”, write a few statements in your IDE, hit F5 and see the results - it’s a long and difficult path to get to that point. But let’s start at the beginning…
As I briefly described earlier, the way to add functionality to SharePoint is by developing so called “Features”. Features can be deployed manually by copying all necessary files into the “12hive” folder and executing a few commands on the command line -or- by using a so called Solution package (a “WSP”). A Solution is basically a .CAB-File (Microsoft’s ZIP format) with a specialized internal file structure, additional configuration files and a .WSP extension. The problem is that there is no built-in functionality to create a WSP file using Visual Studio, neither in the 2005 nor in the 2008 version (this will hopefully change in VS2010 as we will see). We will take a look at how to create such a solution file later in this article, but let’s for the moment focus on how to deploy such a package in a SharePoint environment.
After you have created your Solution file, the next step is to install it using the command line utility stsadm.exe. There is currently no way to install a SharePoint Solution using the SharePoint Central Administration website or Visual Studio, so using stsadm.exe for that task (and some others) is mandatory. That means, you need to have access to the command line on your Windows Server installation and all necessary permissions, which in most cases means those of a server administrator. If the installation is successful you need to open your SharePoint Central Administration website and deploy your solution to all servers in your farm using a certain admin option. At this point you might ask yourself: What farm, for heaven’s sake? Well, your local SharePoint development sever is a farm in and of itself, so deploying your few lines of code for debugging purposes locally is basically the same thing an administrator would do in a huge SharePoint installation with several servers (there is another, slightly shorter way, but it has some additional drawbacks and it is not recommended by Microsoft).
Now that your Solution is deployed on every server in your farm, you need to activate the included Features by executing the stsadm.exe command line utility or by choosing the right options on a particular admin page within your SharePoint installation. Depending on the scope of the Feature (which can be Farm, Web Application, Site Collection or Site), you have to activate it using the Operations Tab in the Central Administration, the Application Management Tab in the Central Administration, the Site Setting of a Top-Level Site (be careful not to confuse the Site Collection Feature setting with the Site Feature setting) or the Settings of all individual Sites. Okay, pretty exhausting, but we’re not finished yet. Remember, that I said “After you have created your Solution file…”? Well, let’s see what goes into a Solution file and how to build it.
First of all you need to create a separate DDF configuration file for the MakeCAB.exe command that is being used to create the CAB archive (which will be renamed to .WSP). I will not go into detail here as nearly every SharePoint development books covers this process in discrete chapters (which is necessary because you can virtually not do anything without knowing how to perform that task). Then you have to create a file named Manifest.xml. This file is called the Solution Manifest and contains information about all contents of the WSP and directives about how to deploy the contained Features to a SharePoint farm. A Solution file can contain several Features in separate folders and each of those Features needs to have a Feature Definition file (named Feature.xml) and an Elements file (usually named Elements.xml). Those three files have at least a couple of XML code lines each but they are usually much larger and contain dozens of statements.
Now it’s time to write your code and create additional entries in the appropriate XML configuration files (Manifest.xml, Feature.xml, Elements.xml), which you can by used to create SharePoint artifacts during the installation (Lists, Libraries, etc.), react on certain events (like Feature activation) and register resources and assemblies in the GAC. When you’re done, you create the WSP file using the MakeCAB.exe command with the prepared DDF file and install the Solution into the server farm using the stsadm.exe command utility as described above. (I strongly encourage you to automate this tasks using MSBuild, NAnt, Automise or another build utility.)
Now its time to deploy and activate your feature as described above. At this point you should make sure that the Solution Framework has “whitelisted” your code in the appropriate configuration file. This security feature makes sure that no “bad code” is executed in the system, but SharePoint sometimes fails to handle that situation correctly for you, so to be sure you better check.
Finally it’s time to test your code. Got an error? Oh sorry, please start from the beginning (or at least, repeat most steps of the whole process). If you want to debug your code (which is most likely the case), you have to attach your Visual Studio to the IIS Worker Process (w3wp.exe), presuming that you have all necessary permissions. (I hereby strongly suggest that you develop SharePoint customization using an administrator account.)
So, why on earth is it so lengthy to install and run a very simple example in SharePoint? Seriously, I have no idea… As Software Architect with experience in several enterprise projects, I can understand the intention behind the package mechanism, the multi-step farm-wide deployment model and the feature activation process when running SharePoint in a large server farm, but it is a mystery to me why it has to be that complex and confusing during development and in simple scenarios.
There are, however, a couple of advantages using a complex deployment system like the SharePoint Solution Framework. One of which is the possibility to “rollback” the installation of a package completely, but we will cover this in the next part of our series.
To get more details about how to develop custom solutions for MOSS 2007 and WSS 3.0, I recommend reading the following articles on MSDN (part 01, part 02) and to take a look at the WROX SharePoint Box, which I would have chosen for myself if I hadn’t ordered every book right after the initial release separately (Damn… ;-)!).
A couple of weeks ago, Microsoft released a statement to the press that they are going to enhance the support for SharePoint development in Visual Studio (VS2010) significantly in the next release, which is expected to ship in the end of 2009 (hopefully, click here to read more). I for myself can absolutely not understand why Microsoft would wait for more than six years to announce full support for their primary office collaboration framework in their primary integrated development environment VisualStudio, especially as SharePoint is even set to replace functionality of Exchange and Access / Excel / SQL Server. They put so much effort in developing and selling SharePoint as the number one web platform and today there is only half-baked support for customizing the system.

Advantages

  1. SharePoint provides a Visual Studio extensions packages that helps the user by adding project and item templates, references and some documentation to Visual Studio. However, it does not help to ease the burden of deployment which remains overly complex.
  2. SharePoint provides a full-features deployment system called the Solution Framework that developers can use to deploy their code within a SharePoint farm. We will cover that in more detail in the next part of the series. Unfortunately you cannot really bypass the whole deployment system during development (although you can bypass the Solution part and install Features without a WSP package “by hand” by manually copying your files to the “12hive” and running some additional stsadm.exe commands).

Drawbacks

  1. SharePoint is not integrated with Visual Studio at all. It is not possible to hit “Run with Debug” (F5) and see your code executing. Instead, you need to describe your code using various XML files, pack it into an archive with specialized tools (MakeCAB.exe + a DDF file), move the resulting WSP package to the runtime environment, install it using the stsadm.exe command line utility, distribute it within the farm using the Central Administration web application, activate the appropriate Features in different locations within SharePoint (depending of the scope of your Features) and attach your Visual Studio manually to the executing process (which may execute on another system).
  2. It is not possible to use Visual Studio for all development tasks, so you regularly need to switch to SharePoint Designer (which can be used to access SharePoint directly, which in turns makes trouble with Source Code Management), the command line and the SharePoint browser interface.
  3. SharePoint can not be developed completely on a usual WinXP or Vista workstation because it is at least necessary to use Windows Server with an installed SharePoint as runtime environment. To have a reasonable development lifecycle, using Windows Server as development platform (be it native, in a virtual environment or on a central server) is strongly recommended.
  4. Even very simple tasks like deploying a custom web part (which is in this special case possible without the whole Solution mechanism by deploying it “locally” to the IIS site into the physical /bin directory) is not straightforward, as you need to create at least one additional deployment file (for registering the WebPart within the web application) and you need to “mark” your code as trusted using the SafeControl directive in the application’s web.config file, sometimes in combination with additional directives to manage permissions.
  5. The packaging mechanism is not able to handle all update and rollback tasks correctly, which is really odd as it has been designed for exactly that purpose.

As you know, Sitecore is based on .NET, and so the tool to develop extended Sitecore functionality is of course Visual Studio as well. So, let’s see how to get started here? Again, as developer your expectation might be the following: Install Sitecore and VS2005 (or VS2008), download “the SDK”, take a short look at some API sample, write three lines of code, hit “F5″ and see your first shiny “Hello World”. And guess what? That’s exactly the way it works.

From the perspective of .NET, Sitecore is just a well integrated ASP.NET web application with advanced functionality and you can even run it using the WebDev.exe development server that is coming with Visual Studio, regardless whether you’re running WinXP, Vista or Windows Server 2003/2008. You can also use the local IIS coming with those operating systems (yes, even IIS 5.1 in WinXP), it is totally up to you.
This means you get all the things you’re familiar with: Debugging .NET code (and even XSLT code) without any additional tasks, deploying additional assemblies to the /bin folder, using source code management tools and experience no limitations while doing your job. The integration of Sitecore and ASP.NET is really seamless. Another great thing is that you’re not limited to Visual Studio, but you can even use Microsoft Expression Web to run Sitecore and choose every external editor (e.g. for editing your XSLT files) to change Sitecore solution and resource files.

In contrast to SharePoint, Sitecore provides you with a couple of additional developer tools within the system that can dramatically improve the quality of your results and your personal productivity. By using the integrated debugger tool to debug your web pages, you get a lot of useful information about your code, your renderings and your design, right where you need it. You can also use the Sitecore Developer Center to edit your XSLT code, your layouts and even your .NET code, so you don’t need to use Visual Studio for very simple task like a short bugfix or minor update. Sitecore’s Package mechanism makes it ease for you to deploy the code you created in your local or central development environment to integration, test and production environments (more about that in the next part of the series) and you can use the integrated log4net to trace the flow of your application without configuring and maintaining a separate logging framework or the Windows EventLog. While creating a reusable content type usually involves a lot of XML hacking, deploying, installing, activating and configuring within SharePoint, using the Template Manager in Sitecore to create your own information structures is a breeze.

Advantages

  1. Sitecore is 100% .NET, not only at the code-level, but also on at the development and the deployment-level. There are no surprises whatsoever when starting to work with the framework.
  2. Developing for Sitecore is exactly like developing with any other .NET library, as it is fully integrated with Visual Studio and the WebDev server. You can even use the free editions of Visual Studio or Expression Web - or any other tool you prefer as long as it runs on the Windows platform.
  3. Sitecore offers additional developer tools as part of the distribution and in form of additional tools through the Sitecore Developer Network.
  4. Visual Studio is your one-stop place for developing Sitecore features, there is no need to switch to any other application, except you want to.
  5. Sitecore can be used with any Source Code Management tool without any problems.
  6. There is nothing more to say here: It just works.

Drawbacks

  1. For the time being it is not possible to rollback Sitecore package installations. This would be a very, very nice feature for the future, although I’m aware of the complexity of that request.

So let’s draw our conclusion here. Regrettably, developing custom code for SharePoint is neither fun nor productive. It is - sorry! - a pain in the ass. There is no real support in VS2005 or VS2008 and we all have to wait until the release of VS2010 to get a more optimized development lifecycle (and hopefully the “F5″ experience) when developing extensions to SharePoint.
Developing Sitecore on the other hand is a breeze for every .NET developer. Of course, no framework is ever complete nor perfect, but Sitecore has managed to create a great developer experience and a pretty great framework.

Sitecore: +10 points = 34 points total
SharePoint: +3 points = 20 points total

See you soon for part six…

Julius Ganns . netzkern

Comments are closed.