7 Dec 2016

SharePoint Development - MOSS 2007

This post is the second in a series looking at how SharePoint development has changed over the years, and how it is likely to change in the future. In this post, we will look the SharePoint 2007 release.

Series Outline:

 

MOSS 2007

The 2007 release (actually released in 2006) attempted to build on SharePoint 2003, and to integrate it more closely with the Microsoft Office suite. This version fixed many of the shortcomings of the 2003 product but also vastly expanded the platform’s core functionality. A major focus was placed on improving document and records management, alongside web content and portals. This version had a fully functional and integrated content management system, a result of the feedback on Content Management in the 2003 release. This gave rise to the infamous pie diagram for the SharePoint 2007 release:

Image showing pie chart of the MOSS 2007 Features

End users were given the option to navigate the platform to create team sites and manage workflows. As a result, SharePoint moved to become a more collaborative platform, and started to resemble the product we know today.

As in 2003, the product came in 2 different versions:

  • Windows SharePoint Services v3 (WSS 3.0), a basic free version for Window Server.
  • Microsoft Office SharePoint Services (MOSS 2007), the premium version built on top of WSS, with the new MOSS designation indicating the greater level of integration with the Office suite. This version was bundled with the Office 2007 suite and added enterprise search and people search, document repositories, additional Web parts, workflow, and content syndication. MOSS targeted organizations storing more than 500,000 documents (large enterprise customers).

The new features included with MOSS 2007 were:

  • Business Data Catalog
  • InfoPath Form Services
  • Excel Services
  • Content types
  • Use of SharePoint Designer to develop portals (replacing FrontPage)
  • Rights Management
  • Shared Service providers
  • Updated security model
  • Improved administration (now create and configure IIS web sites via Central Admin)
  • Improved authoring and site management experience
  • Updated MySite UI
  • Introduced minor versioning and document check out/in in document libraries for editing

 

Developing for SharePoint 2007

WSS 3.0 was built on top of ASP.NET 2.0 and IIS 6.0. Unlike WSS 2.0, requests to WSS 3.0 were routed through the ASP.NET runtime before reaching the WSS layer. This allowed the use of AJAX with SharePoint web parts and pages in WSS 3.0, as well as wrapping custom ASP.NET user controls (.ascx files) in web parts for deployment. Additionally, it allowed the use of  ASP.NET authentication (Forms Based Authentication). As portal sites no longer had to be created at the root of the IIS web site, hundreds of portal sites could be hosted inside a single IIS web site (in current SharePoint terminology, at the web application level).

SharePoint 2007 Stack Architecture (taken from http://dotnetslackers.com/articles/net/MOSS-2007-Architecture-and-Requests.aspx)

The stacked architecture was meant to encourage existing ASP.NET developers to start developing for SharePoint, as it meant that their existing skills and tools could be re-used for SharePoint development. However, developers new to the SharePoint platform still struggled to cope with SharePoint’s particular development methodology (such as features) and gotchas.

As a result of these fundamental architectural changes in the SharePoint architecture, upgrading from 2003 to 2007 was a painful  and laborious process. Additionally, updating custom site definitions (.stp files) when upgrading to 2007 was to be avoided - something that would be repeated in the next release as well. The use of custom site templates for application development, although available since the 2003 release, was heavily promoted in MOSS 2007 by Microsoft. This gave developers a means of creating custom templates for creating new sites in SharePoint with specific site or web features automatically activated. This led to a lot of messing around with GUIDs for features in onet.xml files, and using the Visual Studio Extensions for Windows SharePoint Services (VSeWSS) to package and deploy the custom templates.  In general, you could add custom site definitions easily enough, but it was very difficult to edit or delete an existing site definition.

Additionally, in MOSS 2007, an existing site could be saved as a site template (.stp file) for new sites. It is a package containing a set of differences and changes from a base site definition. It wasn’t as performant as the site definitions, as it was stored in the content database, while the site definition files were cached on IIS on the web front end servers. Site templates also weren’t as portable as site definitions, as to move them between farms, the farms had to be at the same version and patch level. This was because the site templates were dependent on the original base site definitions they were created from.

In WSS 3.0, features and farm (full trust) solutions were introduced. A feature is simply an XML definition for a component deployed to the SharePoint farm, with a specified scope (Farm, Web Application, Site and Web). Features were introduced in WSS 3.0 to address the failings in WSS 2.0 with regard to defining functionality used across multiple sites. Feature Receivers and Feature Stapling were also introduced, allowing modification of features on activation, and allowing features to be automatically enabled on specific site templates. Features were deployed in the new Windows SharePoint Services Solution Package (WSP) files that replaced the .cab file format used previously, and were deployed either using the SharePoint UI, or more commonly using the STSADM.exe tool (and a plethora of .bat files).  Again, I seem to recall a lot of messing around with DDF files to package and deploy web parts correctly.

In addition to all this, developers could now use:

  • Content Types: metadata used to describe the attributes and actions applied to a particular category of information, and used  to develop a logical information architecture. Content types are in practice the schema definition of a site or list, and allow us to encapsulate a data schema and make it independent of a location on a SharePoint site.
  • Integrated workflow and out of the box workflow templates
  • InfoPath Forms Services and Excel Services to integrate with existing business processes
  • Business Data Catalog
  • Integration with SQL Server Reporting Services

With the release of MOSS 2007, there was major growth in the use of SharePoint, particularly across the enterprise. In 2008, Microsoft reported more than 100 million SharePoint users across over 17,000 client organisations, and total SharePoint sales of more than $1 billion. The product and associated tools had matured enough to be a realistic platform for internet and intranet usage in large organisations. As a developer, it was better than the 2003 release to work with (a very low bar), but it was still an extremely frustrating product. The tooling in Visual Studio 2007, whilst improved since Visual Studio 2003, was still clunky and slow. Developers instead looked at third party tools to help with packaging and deploying solutions, such as:

There are probably many other tools that I can’t recall – feel free to call out any in the comments below.

SharePoint Online (BPOS)

In October 2007, the Business Productivity Online Suite (BPOS) was released. This was Microsoft's first attempt at establishing a SaaS offering. BPOS included SharePoint Online, Exchange Online, Office Communications Online and Live Meeting. The SharePoint Online offering was based on the SharePoint 2007 release. Depending on the subscription chosen, this would be either a Dedicated (single tenant) or a Standard (multi-tenant) instance.

While the BPOS offering was successful, Microsoft encountered a number of issues running high performance multi-tenanted instances of SharePoint Online using MOSS 2007, and this experience fed directly into the development of SharePoint 2010.

Next week, I’ll look at the release of SharePoint 2010 and how the developer story changed significantly.

29 Nov 2016

SharePoint Development - Origins

This post is the first in a series looking at how SharePoint development has changed over the years, and how it is likely to change in the future. In this post, we will look back at SharePoint's origins, up to and including the release of SharePoint 2003.

Series Outline:

 

SharePoint's Origins

SharePoint evolved from two separate Microsoft projects, named "Office Server" and "Tahoe", that were developed during the development of Office XP. It was also influenced by the Platinum project that introduced the concept of digital dashboards, and the TeamPages project that allowed users to create and edit simple web-based lists (the origin of the CAML markup language). These projects were all aimed at accessing and sharing information across organisations. The initial development on these products started back in March 1998, prior to the release of the .NET framework (the initial beta versions of the .NET framework were released in late 2000).

SharePoint 2001

The initial SharePoint release was actually two separate products, the Microsoft SharePoint Portal Server (SPS, released 2001) and the Microsoft SharePoint Team Services (STS, released in 2002). The SPS product was based on "Tahoe", offering top-down portals, search and document management. Tahoe was a collection of different technologies from various companies acquired by Microsoft that was built on top of the Exchange data store. STS was based on "Office Server", and was a bottom up team collaboration product.

Thankfully, I have never had to work with this early version of SharePoint, which was very basic.

SharePoint 2003

In 2003, SharePoint as we would recognise it today was released. It combined STS and SPS together to offer collaboration, search, content management and portal capabilities in a single product. It had an improved user interface, better personalization, and a collaboration store.

A lot of the criticism aimed at the initial SharePoint 2001 release was around the under-powered web store that limited the functionality of the product and a digital dashboard that was outside Microsoft’s core development platform, limiting support options for users. As a result, significant work was put into making SharePoint 2003 more reliable and scalable, and to improve support options by making use of the same developer tools as other Microsoft products.

The product came in 2 different versions:

  • Windows SharePoint Services v2 (WSS 2.0), a basic free version that was included with Windows Server 2003.
  • SharePoint Portal services 2003 (SPS 2003), a premium version was built on top of WSS that included additional functionality for document management and search.

The main features included:

  • Alerts
  • Site templates
  • Calendars
  • Surveys
  • Document Libraries
  • Lists
  • MySite
  • Site Directory
  • User Profile
  • Improved search and indexing (SPS)
  • Improved user interface and personalization options
  • Taxonomy
  • Document collaboration and versioning features
  • Single Sign-on
  • Audiences
  • Web Parts (.cab)

 

Developing for SharePoint 2003

The major developer features in SharePoint 2003 were the introduction of server-side API, and the replacement of the Exchange data store with SQL Server. The server-side API made use of the new integration with the .NET framework, and allowed the development of custom code solutions, including web parts. In addition to the default web parts (Content Editor, Image, Form and Contacts), developers could develop custom web parts using Visual Studio 2003 (using the .cab file format).

Note that in SharePoint 2003, and all subsequent versions of SharePoint, COM components (unmanaged code) are used for core features, with a (smaller) managed layer wrapped around it for the .NET API. This is the cause of the dispose issues that affects fundamental SharePoint objects such as SPSite and SPWeb (here and also here).

I did have to work with SharePoint 2003, and it was a pretty horrible experience. Even in it's second release, it was a very immature product, and the developer tooling (based around Visual Studio) was lacking.

In the next post, I’ll look at at the release of MOSS 2007.

SharePoint Development - Past, Present and Future

With the recent release of SharePoint 2016, and with details of the latest development framework (the SharePoint Framework) for SharePoint becoming available, I thought it would be useful to produce a series of blog posts looking at SharePoint development over the years, and how it is likely to change in the future, based on trends from past releases and the current focus on cloud based solutions from Microsoft. The series will consist of 6 articles, published weekly:

This series will be focused primarily on on-premise SharePoint development. While Office 365/SharePoint Online are growing rapidly and are influencing the development of the SharePoint platform greatly, the majority of large organisations are still using on-premise, and are (rightly) wary of having all their data in a third party data centre, even if that third party is Microsoft. I'll point out trends in Office 365 that I think will affect future SharePoint development, but it isn't the main focus for these articles.

I'll be drawing heavily on my own experiences of developing for SharePoint. I first started developed solutions for SharePoint back in 2006, when as a recent graduate I was part of a large team building integration products with SharePoint 2003 (a hideous project). I then spent over 3 years working full time on SharePoint solutions (MOSS 2007 and SharePoint 2010), at which point I took a break to spend several years lost in rewriting large .NET legacy applications. Last year, I started working with SharePoint once more. In this new role, I am currently part of a small team working on a phased SharePoint migration (2010 to 2013) that will be completed in the new year (January 2017). Over all this time, I've worked on a wide variety of SharePoint solutions (intranets, various organisational platforms, eCommerce sites, HR applications and currently a Virtual Learning Environment).

So, lets start by looking at the origins of the SharePoint platform.

5 Oct 2016

My Son

My son, Matthew Oiyan Parkhill, was born on Monday 26th September at 04:18, at the Royal Jubilee Maternity Hospital, Belfast. He weighed in at 5 lbs 9 oz. Mum and baby are doing well.

Mei and I would like to pass on our thanks to everyone at the Royal Jubilee Maternity Hospital for all their help during Mei’s & Matthew’s stay, especially the midwives Patricia, Deirdre and Heidi. We won’t forget their many kindnesses to our little family.

Matthew comes from the Hebrew name Matityahu, meaning ‘Gift of God’, and Oiyan comes from the Cantonese for ‘Love and Grace’.

21 Aug 2016

Forgotten Your Restriction PIN for IPhone?

Recently, I forgot the restrictions PIN on my iPhone (fat thumbs when initially setting it). I eventually recovered it by using the very useful pinfinder utility. This console program, written in Go, was kindly developed and open sourced by Gareth Watts. His utility recovers the restrictions (parental) passcode on iPhones, iPads and iPod touches from an (unencrypted) iTunes backup.

I’m blogging about this to help anyone with the same problem, and also to recommend pinfinder, so people use it rather than one of the rather dubious other commercial products that are available.

Download the latest release of pinfinder from here:

https://github.com/gwatts/pinfinder/releases

6 Jul 2016

What I’ve Been Reading…

I’ve always loved reading. I can remember (aged 6-7?) looking forward to every Thursday afternoon, when my sister and I would be taken to a local newsagent for a comic. My first reading material consisted of The Beezer, The Dandy and The Beano.

Comics

At the age of 9, my mum left me for an hour in the local library. By the time she returned, I was hooked on books. I can still remember the first library book I took out on my new library card - Gorilla Adventure by Willard Price. I was enthralled by this story of 2 teenagers who travelled the world capturing wild animals. I ended up reading most of the titles in the excellent Adventure book series (I seem to recall it being called ‘The Adventures of Hal & Roger’ – a UK printing possibly?).

With the recent migration of my data from Shelfari, I wondered – what have I actually been reading?  After importing the data into a local database, I decided to take a look.

Since November 2011 (when I signed up to Shelfari), I’ve read 332 books. 29 of these were second readings, so this brings the total down to 303 unique books. So, how many books do I read per year?

So I vary from 56 to 91 books per year (over the 4 full years of data, 2012 to 2015), averaging at around 74 books. If we look at the type of books (fiction/non-fiction), does this help explain the variability in the number of books read? I would expect the number of books to decrease if I’m reading more non-fiction in a given year. My impression is that it takes me longer to complete a non-fiction book, and I’m also less inclined to start a new book after I finish.

And looking at non-fiction books as a percentage of the total number of books read each year:

There doesn't appear to be a clear relationship between the amount of non-fiction I read and the total number of books. What other trends can we see in my reading? Looking at my fiction reading:

Author Number of Highly Rated Books
Patrick O'Brian 18
J.K. Rowling 8
George R. R. Martin 6
Paul McAuley 6
Stephen Baxter 6
James Clavell 5
Robert Harris 5
Stephen King 5
William Gibson 5
Neil Gaiman 4

Over the past few years, I’ve enjoyed reading Patrick O’Brian’s amazing Aubrey–Maturin series (think Jane Austin on the high seas), the Harry Potter books, A Song of Ice and Fire (the book series on which the TV series Game of Thrones is based on), as well as several science fiction series from the talented UK writers Paul Mcauley and Stephen Baxter. The Asian Saga from James Clavell is a series I return to time and again.

As you can probably tell from the list above, I like reading complex novels, preferably in a series, with intricate plots and multi-facetted characters. One author who is not featured above is Frank Herbert – I still consider his Dune series the best writing I have ever read. I plan to read it again soon. I especially enjoy science fiction (Macauley, Baxter, Gibson - Neal Stephenson just fell outside the top 10) and Fantasy (Martin, Gaiman and Rowling).

In my non-fiction, I’m tend to read more widely – the topic interests me more than the author. There are a few exceptions:

Author Number of Highly Rated Books
Fergal Keane 2
Jared Diamond 2
John Briffa 2
Max Hastings 2
Michael Lewis 2

I’ve particularly enjoy reading Fergal Keane’s books, in particular his epic story Road of Bones, about the battle of Kohima in 1944. I’ve also recently enjoyed Jared Diamond's books for their unique mix of anthropology, ecology, geography and evolutionary biology

What about books I didn’t like? Here is a list of the worst from the past 4 years:

Title Author My Review
Antifragile    Nassim Nicholas Taleb ‘Vainglorious and pompous nonsense. Read about 15 pages and gave up, not on the ideas being conveyed, but on the arrogance of the author.’
American Sniper Chris Kyle, Scott McEwen, Jim DeFelice Very poor, jingoistic fare.  Tells a completely different, and much grubbier, story than the film of the same name.  I found this book very depressing reading indeed.
The Bone Clocks David Mitchell Very disappointing. After enjoying the wonderful Cloud Atlas so much, very let down by this novel. It read like a terrible 1950's pulp Sci-Fi novel. Yet there was the odd glimpse of some great writing.  Such a pity.
Slow-Tech Andrew Price Rubbish.  The author has a very poor understanding of sound engineering principles.  The book reads more as a left wing diatribe, as opposed to a serious discussion of how to improve design choices.
A Discovery of Witches Deborah E. Harkness A truly dreadful book.  I threw the book away after approx. 30 pages. Dire stuff.

I regret putting David Mitchell in the above list – he is a fantastic author (Cloud Atlas, Black Swan Green), but Bone Clocks was a great let-down.

At some point, I hope to spend some time analysing the hashtags I associated with each book in my reading list. I’ll update this post with the results. Until then, happy reading!

21 Jun 2016

Shelfari

Recently, the Shelfari book cataloguing service was shut down by Amazon. Officially, the site was merged with GoodReads, another Amazon purchase, but effectively it was a shutdown. I've used Shelfari to track my reading habits and to maintain a list of 'must read books' for over 4 years, and I was a bit put out by the announcement, but not particularly surprised. The Shelfari site hasn't been upgraded in years, and was slow, unresponsive (both in terms of performance and mobile usage) and prone to outages.

I looked at alternative sites that I could migrate my data to (GoodReads, LibraryThing, Open Library), and decided that none of them was what I was after. Given that I only had 2 lists (books I've read, and books I want to read), a simple spreadsheet would do. So I migrated my data out from Shelfari using the data export functionality, and looked at the contents of the export file.

A cursory examination of the file (unusually, a .tsv file rather than the more popular .csv format) showed:
  1. The reading list was missing key data. Primarily, if a book was recorded twice (when I had re-read it), that book's details was completely missing from the list.
  2. The file format was invalid - it inconsistently quoted string fields (i.e. empty/null values didn't have quotes, while other values in the same column were quoted). This meant it was extremely difficult to import the data into a database/Excel to manipulate. 
In order to import my Shelfari data into an Excel file, I had to write a small command line application to carry out the conversion.  While it was an interesting exercise (I came across the excellent EPPlus and CsvHelper libraries for manipulating Excel and .csv files respectively), it took up time I could have used on other projects. Also, how is a non-programmer meant to deal with invalid data?  It is pretty poor for Amazon to ignore a site like Shelfari for years and then on shutting it down, to hamper users from accessing their own data due to an incompently coded and untested data export functionality. It also stands in stark contrast to the excellent work from both Google and Twitter in allowing users to export their data in a usable format.

Thankfully, I had taken screenshots of my book history and was able to recreate the missing book data. After managing to get all of my data into an Excel spreadsheet, I've now generated an updated list of the books I've read since joining Shelfari back in November 2011. You can view here: