I am spending my time these days walking the halls of Berlin Messe at the Microsoft TechEd 2009. One of the interesting upcoming features of .NET 4 is the introduction of dynamic language support in the .NET Language Runtime. This does not only mean that you can write your .NET code in dynamic languages such as Python (IronPython) and Ruby (IronRuby), but also that you can use some of the dynamic features from C#. With the introduction of the dynamic type keyword, method calls and member access can be evaluated at runtime instead of at compile time (previously known as late-binding). This does not mean that C# is no longer a type safe language, it merely means that the language now provides a more “syntax correct” way of accessing those dynamic types of data, we previously used to access via strings, invokation, reflection etc. Examples of dynamic data models includes COM, the HTML DOM and XML.

Another one of the dynamic types of data we all works with each day, is Sitecore. The structure, i.e. templates and fields, we are coding up against changes constantly, and Sitecore provides a string based way to access specific items and fields and values in fields. Some of these dynamic types are exposed via classes in the Sitecore API, e.g. the fields type classes in the Sitecore.Data.Fields namespace. In Pentia we have a concept we call Sitecore wrapper classes, i.e. classes which we custom write to provide a proper interface to items of a given template. Writing these classes are tedious and potentially error-prone, which is why we are working on code generator for generating these classes (similar to e.g. LINQ to SQL). But with dynamics in .NET 4, we have a way of eliminating these classes completely, while still maintaining a proper syntax – and hopefully Sitecore will implement this to our common advantage. Here is an example of how code from the current version could look in a future version of Sitecore:

//BEFORE: .NET 3.5 Framework, Sitecore 6
public string GetContextItemDocumentImage() {
  Item item = Sitecore.Context.Item;
  ImageField field = ((ImageField)item.Fields["DocumentImage"]);
  return field.Src;
}

//AFTER .NET 4 Framework, Sitecore 7?
public string GetContextItemDocumentImage()
{
  dynamic item = Sitecore.Context.Item;
  return item.DocumentImage.Src;
}

Note that although the syntax differs, the actual functionality executed in both example would be the same. So, what does it require from Sitecore to do this? The secret lies in the System.Dynamics. IDynamicMetaObjectProvider interface or more easily, the DynamicObject class. This interface and class allows the derived class to provide its own resolving to e.g. members access and method calls via a number of virtual methods. The methods are Try based, which means that you should return false if the functionality is not supported, or e.g. a method with an invalid method is called. Some of the most important virtual methods which can be implemented via DynamicObject are:

An example of the above Sitecore 7 (:-)) implementation could therefore be:

namespace Sitecore.Data.Items
{
  public class Item : DynamicObject //...and other interfaces
  {
    //...
    public override bool TryGetMember(GetMemberBinder binder, out object result)
    {
      dynamic field = Fields[binder.Name];
      if (field != null) {
        result = field;
        return true;
      }
     	return base.TryGetMember(binder, out result);
    }
    //...
  }
}

Let this be an inspiration to Sitecore for the next version – I for one truly hope we see something like this in the product.

A lot of sites which have a large amount of visitors use a dedicated image server.  There are several reasons for this, but mostly it is because that most browsers only allow a few simultaneous connections to the same domain, which stalls the download of a page that has more than two resources. For instance imagine you have a page consisting of multiple images, stylesheets and javascript files. As the clients only allows a few simultaneous downloads, the browser won’t download all the resources at the same time, but wait for each download to complete and first then start the next download.

The reason of this behavior is that the HTTP standard says that only two simultaneous connections between a client and a server should be allowed. This was specified to save servers from heavy IO load, when files where requested. Browsers like IE7 and Firefox 2 lives up to this standard and only allow 2 connections, while new browsers like IE8 and Firefox 3 allows 6 connections.
To ensure that the HTML can be fully loaded independently and without waiting for other downloads and to remove the load from the primary web server; a dedicated image server can be used. This server holds all files and uses a different domain (for instance images.mydomain.com). All files used on the page can then be fully referenced and thereby be downloaded from a different domain. Eg <img src=”http://images.mydomain.com/image1.jpg alt=”image1”/>. As the images are now on a different domain, these can be requested independently of the server providing the HTML and more simultaneous connections can be opened.

But how is this possible in Sitecore where the media library controls all the images?

One of the easiest ways is to have another publishing target for the media. In that way you will have two frontend servers: One serving the normal requests and one serving all media items. You can easily set up another publishing target and for instance use the staging module to clear the cache. Read more on SDN if you want to know how to set it up.

The remaining problem is: How do we ensure that all media items gets prefixed with another domain? Unfortunately there isn’t a simple setting for this (it will probably come in a future a release of Sitecore – I hope ). However there is a simple solution to the problem as links are expanded by the LinkProvider, which can be overridden. More precisely the links are expanded in the LinkProvider.ExpandDynamicLink method, so we need to override this replacing the media path:

namespace TestApplication
{
  public class CustomLinkProvider : LinkProvider
  {
    public const string MEDIA_PATH = “~/media/”;

    public override string ExpandDynamicLinks(string text, bool resolveSites)
    {
      string baseExpands = base.ExpandDynamicLinks(text, resolveSites);
      return baseExpands.Replace(MEDIA_PATH, “http://images.pentia.dk/” + MEDIA_PATH);
    }
  }
}

Here I override the ExpandDynamicLinks in the inherited class CustomLinkProvider. I replace all media paths (identified by the prefix ~/media/) and replace it with the full path to the image server. The remaining thing to do is to replace the LinkProvider type in the web.config:

<linkManager defaultProvider=”sitecore”>
  <providers>
    <clear />
    <add name=”sitecore” type=”TestApplication.CustomLinkProvider, TestApplication” addAspxExtension=”true” alwaysIncludeServerUrl=”false” encodeNames=”true” languageEmbedding=”asNeeded” languageLocation=”filePath” shortenUrls=”true” useDisplayName=”false” />
  </providers>
</linkManager>

Here I just point the LinkProvider type to my own implementation.

Now that this is set up, all requests for media items will be handled by the image server. This does not just allow faster load times for the client, but will put less pressure on the main server. Further you can tune the media server to handling media items by increasing the media cache, setting the prefetch cache etc.

Enjoy your new Sitecore media server

I have earlier questioned Sitecores way of releasing new versions. You can read the full discussion here: http://sdn.sitecore.net/SDN5/Forum/ShowPost.aspx?PostID=17750. The whole notion of Sitecore having “recommended” and “not recommended” releases are to me rather strange. Especially considering the frequency of when new versions gets to the “recommended” level. For instance it is at the moment not possible to use a recommended version of Sitecore which supports Sitecore OMS!

In Pentia we finally got to that point, where we just decided to ignore the recommended label and upgrade and use the latest released version, as it seems rather arbitrary whether a version is recommended or not. This means that I wanted to download the latest version of Sitecore… But guess what. That is not possible. You have to download the release 090630; install it and then afterward download and install two updates.

So I couldn’t help but wonder… Why can’t I download a package containing the latest release of Sitecore?

Me and my colleague Brian Pedersen have been spending time in London (thx to Google London and Dave Gallerizzo of Fig Leaf) this week looking into the Google Search Appliance (GSA) – which is a really cool search product which I hope we will implement in a lot of future solutions. By the way, Pentia is now a Google Enterprise Partner

The GSA though, is a double edged knife – on one hand, it provides really good value out of the box (best of class in search relevance), and has a lot of nice features and configuration possibilities. On the other hand, the configuration interface on the GSA sucks! yes, it sucks! I would never expect any end-users to understand what happens in the UI, and in worst case, they will mess up the nice out-of-the-box functionality. Said in other words: the summer intern who wrote the config UI should be definately find work in a non-computer related sector.
This said, the GSA has some really cool API’s, which will allow you to provide nice functionality for your users:

Admin API
If you consider integrating a GSA in a Sitecore (or any CMS) solution, you should consider integrating some of the configuration directly in the CMS backend. Imagine users being able to tweak the biasing of content, or adding content to the “recommended” box in the top of the results directly in Sitecore. This is all available through the Admin API – which is nicely wrapped in .NET assemblies.
You can find the Admin API here.

Feeds API
The Feeds API allows you to insert content directly into the crawler queue. There are three options: web feeds, which simply adds URLs you want the GSA to crawl, url-and-metadata feeds, which allows you to add additional metadata to URLs being crawled (think adding metadata to PDF files), and content feeds which actually insert the html or binary data directly into the GSA.
You can find the feeds API here.

Policy ACL API
One of the strong sides of the GSA is the possibility to crawl and serve secure content, e.g. on your extranet or intranet. The problem though, is how to determine which URLs the individual users in your custom security solution has access to. This is where the Policy ACL (access control list) API comes handy. In short, it allows you to insert access lists in the GSA and map them to URLs – which again allows you to have fast secure queries – simple and wrapped in .NET.
Find the Policy ACL API here.

There are tons of other APIs and documentation on the Google Search Appliance on code.google.com.

I’ve always been a big fan of keeping things that belongs together, together. This might be an obvious fact, but I sometimes find that in Sitecore, it’s not so easy to follow. If you read my post on agile Sitecore design, you’ll get my drift. But this post is on something much more practical, i.e. composing your layouts in Sitecore, and how something I choose to call composite layouts can help you make your practice better.

When it comes to putting your pages together in Sitecore, you have two choices:

You can choose to build up your entire page composition on your templates in Sitecore, i.e. choose a layout and putting all sublayouts, renderings etc. on a template (which is naturally the __standard values of the template). This method is in my book the most correct, as it allows you, your fellow developers and the administrators to see the complete structure of a page, without leaving Sitecore. Unfortunately Sitecore does not deliver any layout inheritance etc., so this way requires a lot of hard work and mouse clicking (“try adding a rendering to all pagetypes late in the process – phew”) and is potential place for mistakes. Also, the page designer and Webforms module copies the layout from the template, locally to the item – and trust me, you do not want your layout with 20 renderings spread over your 15 pagetypes and 356 local items (“try adding a rendering to all pagetypes late in the process – double-phew”).

So, unless you are committed to keeping your page composition in one place, and choose not to use the page designer and webforms, you are left with the second option – putting page composition in sublayouts. This is where you put renderings and other sublayouts directly in the .ascx file, either through Visual Studio or the Sitecore layout designer. The big problem I have with this option, is that you separate page composition into two places – Sitecore and the ascx files. This forces you to use two different systems to setup caching etc. and makes it difficult to overview the complete page composition.

Enter Composite Layouts:

Composite Layouts is a new type of rendering I developed for a large project, which bridges the gap between the two options. It utilizes the fact that even rendering items in Sitecore have a Layout field, and it merges a layout set directly on a composite layout item into the layout set on a template. I.e. a composite layout is pretty much like a sublayout, in that it allows you to group renderings and place them on a page as a unit. The exception is that all configuration is done in Sitecore and the renderings on a composite layout can be placed in all placeholders – even placeholders outside the composite layout.

Our Form pagetype is a standard document page (with the usual header, topmenu, title, leftmenu, footer, spots etc.) which renders a selected form (which is incidentally created using Webform for Marketeers) in the content area:

The DocumentPage rendering is a composite layout which is defined with the following layout:

Note that the layout has a NullLayout selected – this is merely because the Sitecore UI forces us to select a layout before saving the layout on the item. The NullLayout is naturally never rendered.

Just to make the point; in turn, here’s how what the HeaderBlocks composite layout contains:

As you can see, composite layouts uses standard Sitecore UI with no addition, and all I’ve had to do at this point is to create a Composite Layout template and add it to the insert options of the Sublayout Folder template. The template has no fields whatsoever.

So how do you get it to merge the layouts? First you create a class – let’s call it InsertCompositeLayoutRenderings – derive it from Sitecore.Pipelines.RenderLayout.RenderLayoutProcessor and insert it into the renderLayout pipeline in the web.config. The entry needs to be right below the InsertRenderings entry, as this unfolds all the renderings in the current item and fills the Sitecore.Context.Page.Renderings list.

Second, you implement the Process() method of your InsertCompositeLayoutRenderings class, run through the Sitecore.Context.Page.Renderings and detects any composite layouts in the list, parse the layout field in the composite layout item and call Sitecore.Context.Page.AddRendering for each rendering found. If you want recursive support, you merely give the renderings on each composite the same treatment.

Granted, the design has some flaws – e.g. all renderings are added to the end of the layout (theres no InsertRendering method), so you have to be careful about the placeholders you use and order of the composite layouts – but I still consider the solution to be much better than the two options first mentioned.

We could all hope Sitecore reads this post and implements something similar in a future version, but until then if anyone is interested, I’ll consider posting the code on the Sitecore shared source network.

My cooworker Jimmi Lyhne Andersen and I are glad to announce that we have finally finished a project we have been working on for quite a while now; a complete learning site for Sitecore. Check it out at http://learnsitecore.cmsuniverse.net/

So what is it all about? Well we have been working with Sitecore for quite a while now, and we found that documentation was somewhat limited. Documentation revolves around SDN which is great as a reference (especially if the search functionality worked probably) and blog post by enthusiastic Sitecore developers. This leaves quite a gap, as there are no in depth article base and nowhere to go, if you want to start learning about Sitecore. Further there is only limited documentation for editors, server administrators and architects.

Therefore our vision was to create a site, which matches these target groups and goes in depth with the topics. Our mission is not to compete with SDN, as it is quite resourceful for snippets and references. Our mission is neither to compete with the blogs, as these are quite resourceful when you want biased opinions on Sitecore or new and interesting tweaks.

So in short learnsitecore.com is the place to go if you want to learn Sitecore or read in depth articles on different technologies which Sitecore uses. Right now there are a limited number of articles, but we hope to get some contributors and of cause write a lot of articles our self.

We hope you’ll find it resourceful and that you might be motivated to write an article or two.

Following up on my previous post about content and presentation separation , I realized that separating presentation and content is extremely important, if you want to utilize some of the new features in Sitecore such as the page editor and the new rule engine . These features let the end user meddle with the presentation which gives quite a few nice features. For instance an editor can add a rendering to a given page in a rich and intuitive GUI and set up rules for a visualization of a certain rendering.

The problem, as earlier discussed, is what to do with “presentation content”. With presentation content I mean settings or similar. For instance take a rendering where you want to present a list of news, which are all descendants of a specified root item. This rendering needs the following settings: How many news items to show, a root item, a read more link etc. Normally I would put these properties on a template as fields. However this would bind the presentation to content and the sublayout or rendering would only work on an item, which is derived from that template. As the end user can now meddle with the renderings and add or remove them at will, he will have to know the dependencies to different templates, which is practically impossible.

So what is the solution to this problem? Well as fare as I can figure out, you need to use the datasource and parameter properties on the rendering instead. However this introduces some issues when it comes to usability, as it is now the end user who needs to set these properties. Sitecore has a really rich and user friendly interface for altering content, but for some reason the UI for setting properties on a rendering/sublayout is rather hopeless. There is only one field type – a simple text field. So when you need to set an external or internal link you need to enter it as simple text. No lookups, nothing. Further there is no way, to my knowledge, to create validators for these fields, so you aren’t able to help the editor.

So for this to work Sitecore needs to implement a better UI or use the content editor to allow editors to configure renderings. I sort of figure it would be quite nifty, if there was created an item, whenever you add a rendering. The item should be based on a specific template for the rendering type, which would mean you could create different templates for different types of renderings and thereby control which parameters would be needed. This should be completely transparent for the editor, which wouldn’t notice that an actual item was created, except he would get the user friendly interface of the content editor.

This would give better usability and thereby allow a better separation of content and presentation (and the content needed by the presentation).

The end user issue doesn’t stand alone. Recently I implemented a sublayout, which should read some parameters. If I have done this in a best practice way, I must admit, that the API for reading parameters is completely hopeless. Check it out on SDN. What? Is this really the easiest way? Further the parameters data type is a concatenation of strings much like query parameters. This is quite frustrating as you have to parse the string or use the .net UrlString class to extract a specific parameter.

Adding all this together and adding different validation of the parameters made my class turn into one long unreadable mess.

The conclusion? If you want to separate content and presentation you lower the capabilities and usability for the editor and make your code one big mess. Either that or I am doing something completely wrong.

If Sitecore really wants the editors to meddle with presentation, they should really improve the UI and API for sublayout parameters and similar.

It been a while since my last blog post – my second child Annika was born March 16^th and since then I’ve been forced to realize that two children takes up a lot of blog-writing time.

A year ago I wrote a post titled Type before function, which pointed out the very type-centric design of Sitecore. Since then, we have in Pentia tried to develop a design practice which focuses more on isolating logically grouped functionality in our design and thinking outside the type-centric nature of Sitecore. For this purpose, we have “invented” the term component in our design. The purpose of components is to separate the functionality from the type, allowing the developer to focus on one purpose and its interfaces.

The practise has been quite successful – our large-scale solutions have become increasingly easy to maintain and extend. As a side effect of the process, we now have a high degree of reuse – not just knowledge, but actual code – between even seemingly un-similar projects. Not very surprisingly, it turns out that developers are more inclined to reuse code, which is easily overviewed. The design practice is so successful, that we have passed it on to external projects, through our Professional Services.

Here are a few tips on getting you started:

Consistent placement and naming

Not all items belonging to a component can be placed together. Naming and placement makes it easier to identify component items – in a type-centric system.

Keep your files together – we never use the standard Sitecore folders, e.g. /xsl and /layouts. All files in a component are placed below a folder named /Components/[ComponentName]. This makes it easy to identify which files belong together without e.g. garbling the filename with unnecessary prefixes. And if you think about it, it’s not hard to identify a file as an XSLT – so why put it in the/xsl folder? Keeping component items together also goes for templates, renderings and layouts in Sitecore. For the purpose of Sitecore best practice, we still place templates under /sitecore/templates (although I suppose it’s not strictly necessary), and layout items under /sitecore/layouts. But we always use subfolders which are named consistently, e.g. /sitecore/templates/Components/[ComponentName].

Component projects

Each component has a separate Visual Studio project, and therefore a separate assembly. Aside from grouping files even further, this also helps to identify dependencies between components. Use build events on the project to move files from component folders to website folders if needed.

Interface templates

Think of templates in Sitecore as interfaces – never basetypes – which provide your pagetypes or data items with certain functionality. Always refer to these interfaces in the code within the components.

Example: The component Navigation (menu, breadcrumb and sitemap) defines the template Navigable with the fields Menu Title and Show In Menu. This template is assigned to the pagetypes which can be shown in the menu. The renderings Menu.xslt and Breadcrumb.xslt only references the Navigable template by using the XslHelper.IsItemOfType() method (or your own more optimized version).

Common components

There are a number of components which is always in a project, and which brings the functionality of the other components together.

The Design component contains all the graphic design for the project. Do not be tempted to place stylesheets and graphics within each component – the design for a single component is always related to other components, and therefore you will be forced to create unnecessary dependencies.

The PageTypes component brings together the interface templates and renderings and sublayouts in the other components on the actual page type template. Only page type templates are instantiated in the content tree. If you define the page type template News in the News component and not in the PageTypes component, you will be forced to create dependencies to other components e.g. Navigation.

Dependencies and Inversion of control

Keep your dependencies amongst components as few and as obvious as possible. It should not be necessary for a developer to know more than the component he is working on – too many dependencies makes the components too complex and less flexible. The common components is one way to reduce dependencies, another is the use of the inversion of control pattern and possibly an IoC container. Suppose the News component uses the Indexing component for providing data quickly. This could naturally be done by calling the indexing component directly through the News code, thereby creating a dependency. Alternately the News component could provide an appropriate interface which defines its necessary functionality. This interface could be implemented by the Indexing component and instantiated though Reflection or an IoC container like Unity or Spring, thereby inverting the dependency and making the News component more lean.

I don’t know about you, but I actually haven’t been giving command templates any attention since they were introduced in Sitecore 6. This is really a shame, as it is pretty powerful and can help editors a lot.

For instance take the often used structure for news, where you have a news folder. To avoid too many items in a flat structure you have folders for the current year and the current month. This could back in the old days be used by letting the editors them self create the folders they needed or you could have implemented a lot of event handlers or similar. Now you can just use a command template and let the command create the folders from the current DateTime.

To create command templates follow the guide in the data definition cookbook . Then implement the command to create the folders like this:

public class MyCustomCommand : Command
{
  public override void Execute(CommandContext context)
  {
  if (context.Items.Length == 1)
  {
  Item item = context.Items[0];
  NameValueCollection parameters = new NameValueCollection();
  parameters["id"] = item.ID.ToString();
  parameters["language"] = item.Language.ToString();
  parameters["database"] = item.Database.Name;
  Sitecore.Context.ClientPage.Start(this, "Run", parameters);
  }
}

protected void Run(Sitecore.Web.UI.Sheer.ClientPipelineArgs args)
{
  if (!args.IsPostBack)
  {
    //Means we are in the initial step, we want to ask for the name of the news
    Sitecore.Context.ClientPage.ClientResponse.Input("Enter the name of the news item:", Sitecore.Globalization.Translate.Text("New NewsItem"), Sitecore.Configuration.Settings.ItemNameValidation, "'$Input' is not a valid name.", 100);
    args.WaitForPostBack();
  }
  else
  {
    //Now we got a postback, which means we got a response
    if (!String.IsNullOrEmpty(args.Result) && args.Result != "undefined")
    {
      Database db = Sitecore.Configuration.Factory.GetDatabase(args.Parameters["database"]);
      Item parent = db.GetItem(args.Parameters["id"], Language.Parse(args.Parameters["language"]));
      //Create the folder structure if it doesn't exist
      string year = DateTime.Now.Year.ToString();
      string month = DateTime.Now.Month.ToString();
      TemplateItem folderTemplate = db.Templates[Sitecore.TemplateIDs.Folder];
      Item yearFolder = parent.Children[year];
      if (yearFolder == null)
        yearFolder = parent.Add(year, folderTemplate);
      Item monthFolder = yearFolder.Children[month];
      if (monthFolder == null)
        monthFolder = yearFolder.Add(month, folderTemplate);
      //Create the news Item
      TemplateItem template = Sitecore.Context.ContentDatabase.Templates[new ID("{B2612CF6-16D6-426D-9E74-EE3A4E3989B2}")];
      Item item = monthFolder.Add(args.Result, template);
      //Load the Item in the content editor
      Sitecore.Context.ClientPage.SendMessage(this, "item:load(id=" + item.ID.ToString() + ")");
    }
  }
 }
}

There you go – auto creation of folders. It’s that simple. I can think of really many usages of this – also more complex situations. Too bad the sheer UI documentation is so limited, as it makes it difficult to implement complex wizards etc.

In Pentia we have created a new service – Pentia Professional Services, where we offer our deep knowledge of and experiences with Sitecore. The purpose of this service is a bit like Sitecore Professional Services. We offer to help clients and partners with architecture, codereviews, performance testing and debugging Sitecore solutions. To differentiate us from Sitecore Professional Services we focus on the custom solution, where Sitecore focuses on the product.

A couple of weeks ago I was assigned to a task, where a client experienced that the Sitecore backend were performing really bad. Issues like these have always been harder to debug then frontend performance issues, as there are very few indicators to which process, is making the solution stall. Is it a scheduled task? Is it a custom pipeline? Is it an event hookup? Etc. etc.

Well on this particular task I found a rather new entry type in the log. Sitecore has implemented a new counter – the LongRunningOperationWatcher. This class is used to log entries like this:

Long running operation: Running Validation Rules

This is a great indicator and makes it easier to debug the backend. The class is called with a threshold, a message and some custom parameters, which are outputted in the log entry if the threshold timer is exceeded when the object is disposed. It is used in different methods throughout the kernel – for instance when validating different validation rules and in different pipelines. I can only hope that Sitecore will extend it to more operations and pipelines. Further it is great to use yourself, when creating a custom scheduled task, which might run wild.

After analyzing the log file for the client, I found that the “Long operation: Running validation rules” entry came up constantly, which told me, that either they had some custom validations, which were quite heavy on performance or that the server couldn’t bare the load from the standard validations.