My take on a software development success philosophy

Below I have compiled a list of  6 personal (overall) viewpoints about software development philosophy that I find particularly important when successfully running software projects. Some of them might be obvious (?), but then maybe not as I have observed that many projects don’t really adhere to them.  In fact, in many projects the opposite is done intentionally or not thus resulting in significant delays, cost overruns, unsatisfied users/customers and missed opportunities [reader: What is your take?] :

• Quality pays – Time and resources spent on quality gives positive return on investment on development time, development costs, team and customer satisfaction etc . The earlier in development process resources are spent on quality related tasks, the better ROI. For example, is well known that finding bugs in requirements or design can be 100-1000 times less costly then in a shipped solution. Unfortunately, many projects are not managed with this fact in mind and many developers don’t have a good enough grasp of quality issues.
• Careful risk-management is needed. – Too many projects are delayed, are over budget or completely fail to deliver. Managing the risks that can cause projects to fail is therefore essential. In particular, I find it important to limit scope of the solution/version worked on (not trying to do too many – possibly half baked – things at the same time), involve users from the beginning, consistent focus on writing well-crafted maintainable software (avoiding technical debt) and to address technical risks early on (like for instance scalability, performance or integration-issues).
• People, development process, technology and clear goals are all important for success. Even great developers fail at projects if process is wrong, technology has major problems or goals are unclear. In particular, the value of a good R&D process is often underestimated. For larger projects, average developers all following a good software process will perform far better then great developers following a bad process.
• Long-term priorities must be balanced with short time priorities. Quick workarounds, hacks and half-baked solutions may help to accomplish some business goals on the short term. However, unless addressed and reworked afterwards they risk permanently lowering the quality of the solution and imposing a technical debt that will effectively tax all future development. After many such short-term solutions, it is likely that development productivity will decrease substantially to the point where new features or bug fixes that should take hours will consistently take days or longer.
• Use modern efficient technology as long as they are not bleeding cutting-edge. Be open to the use of technologies before they are mainstream if (and only if) they offer substantial benefits, are actively maintained, have a significant growing community, reasonable tool support and have books and training/support organisations behind them, Note that besides obvious technical benefits and savings, use of non/pre-mainstream technologies also helps attract the very best developers.
• Investment in automation is key for efficient (agile) software development. Partly to save costs in the long run, partly to be more agile and responsive. In particular automate the build pipeline, deployment tasks and (most) technical tests used for regression. Beware that some automation efforts like for example GUI test automation requires special skills and tools to do right (high risk of failure or low ROI if done wrong).

Breaking encapsulation with C# 2.0 partial classes (moved posting)

For good or bad partial classes in C# 2.0 allows breaking of encapsulation as this example will show.

In a consulting job I recently ran into an interesting case involving a webservice with several different service methods f1, f2, fn (sample names, not actual names) all taking the same string argument and all returning a string. The user would select an operation name after which my code had to call the named operation on a web service using a standard parameter. Trivial really, if one would do accept bad code like this below, but I don’t:

 String operationName = …
 String arg = …
 Webserviceproxy webserviceproxy = …
 // Warning: Badly coupled code begins here (need to update each time we  add/rename/delete operations).
 switch (operationName) {
 case “f1″: return webserviceproxy.f1(arg); break;
 case “f2″: return webserviceproxy.f2(arg); break;
 }

What is really needed is a method to invoke a webservice method by name, while still using the generated .NET proxy to do the hard soap/http stuff (no time to reinvent a better wheel here). Reflection is one way to do this, but let’s try another type-safe method for this posting, because the technique shown here is quite powerful for all sorts of related problems:

Let’s look at an extract of the generated proxy:

public partial class Webserviceproxy :  System.Web.Services.Protocols.SoapHttpClientProtocol
 {
  …
  public string f1(string arg) {
   object[] results = this.Invoke(”f1″, new object[] {arg});
   return ((string)(results[0]));
  }
  …
 }

and at the inherited SoapHttpClientProtocol:

 public  class SoapHttpClientProtocol : HttpWebClientProtocol {
  …
  protected object[]  Invoke(string methodName, object[] parameters) {
  …
  }
  …

  }

It seems the method “invoke” would fulfil our needs if it was only public (which it isn’t). So what do we do? Certainly we do not want to modify the generated file (and lose our changes each time it is regenerated).

The good news is that Generatedwebserviceproxy is a partial class so we can extend it with the following code. We will place the code in a file safely outside the generated proxy class file:

public partial class Webserviceproxy :  System.Web.Services.Protocols.SoapHttpClientProtocol
 {
  ///
  /// Dynamic operation that allows us to call an operation by name.
  ///
  public string InvokeAny(String operationName, string arg)
  {
   object[] results = this.Invoke(operationName, new object[] {arg});
   return ((string)(results[0]));
  }
 }

The compiler will merge the two class definitions effectively adding a new public InvokeAny method to the generated class. And now we can call our web service calls dynamicly from using InvokeAny:

 String operationName = …
 String arg = …
 Webserviceproxy webserviceproxy = …
 return webserviceproxy.InvokeAny(operationName, arg);

Clearly, easy to do and with better overall code than a “switch” – even though it is not without drawbacks as it breaks encapsulation of the generated proxy.

Post scriptum:
Used the same partial classes trick today to add a common custom interface to two differently generated proxies. I now officially miss this feature in Java (yes, AspectJ can do the thing but it is not a official part of the language).

Recent update and notice:

This post an almost identical copy from my old blog at “http://www.mortench.net/blog” which I will shortly retire for good.

C# 4.0 specs mentions Ruby in line 4

We are currently seeing Microsoft being more and more aware of Ruby and Ruby on Rails world. They’ve launched there own MVC framework and they are working on IronRuby.

Now a whitepaper on the future specs of C# has seen the light of day and I’m proud to say that Ruby is mentioned in line 4 (if you don’t count the introduction). They are implementing more dynamic structures in the language.

The following is a quote from the whitepaper. They are talking about “dynamic” objects:

“Some examples include
a) objects from dynamic programming languages, such as Python and Ruby”

I think this is a funny development of a compiled language. Ruby has excisted for 15 years but suddenly Microsoft finds it the source of great inspiration?!

When that is said the feature I’m currently looking most forward to in C# 4.0 is co- and contravariance. Now we can use generics for more than simple collections 🙂

If you want to read the Microsoft whitepaper, you can get it here

Ruby on Rails’ footprint on Microsoft

This post could also be called how David (DHH) inspired Goliath (MS).

Lately Microsoft has release a number of products and new technologies on the .NET platform. Of course this could just be seen as the way of .NET framework. I mean.. 3.0 and 3.5 are just what comes after 2.0, right?!

Anyway… We’ve had this WebForms framework for doing websites for years now, and MS never really strafed from this strategy even though the Model View Controller pattern has had alot of followers and proven itself on the Java platform. Why should we have an MVC stack when working with .NET?!? Well.. I for one find it really easy to understand. No more advanced event cycles and s… viewstate. No more postback and advanced control design… Well that’s just me. When that is said, I’ve worked almost exclusivly with WebForms since the .NET 1.0 beta was out… Of course there has been some alternatives. In the last post I linked to the MonoRails project. On the frontpage it even says that it is inspired by Ruby on Rails (just see the .rails file extension for the httphandler in all their examples).

As I wrote in the last post, Microsoft are now launching their own MVC stack… Hmmm… Why the sudden change? As with everything Microsoft goes into, they pretty much market it as something completely new and innovative. Just look at the MVC frenzy going on over at: ScottGu and Haacked.com (ok.. also alot of great examples… ).

So is this another MS MVC post… Well not really, I just wanted to sum up some recent stuff out of MS and then leave it up to you to decide if they are not (heavily) inspired by RoR:

MVC

As already discussed, the heart of RoR and a sudden shift from MS.

REST

While not invented by DHH, he spoke (and) … and the world listened. This completely suprised MS. “Does people actually need this silly rest thingy?!”. Well apparently MS is now REST’ing a but with the Astoria Project.

NRuby?

We currently have JRuby and MS recently launched their first dynamic language (IronPython) on the .NET platform. Furthermore there are job ads like this. There is also this open source project however.

O/R Mapping

While RoR uses Martin Fowler’s Active Record pattern for its o/r mapper, Microsoft finally has something they almost can call an o/r mapper. While MS has brought the dynamic expression i c# to the table with the LINQ technology, LINQ for Entities (and SQL) is really too little too late.. I’ve been using NHibernate for years now and with the LINQ for NHibernate I might just stay there :). And also remember that this is not the first o/r mapper MS tries to bring to the market. Who doesn’t remember ObjectSpaces.

While I don’t claim that MS has picked up on o/r mappers after RoR, they certainly comes in handy with their brand new lightweight MVC package.

So…

What do you think?! Is MS inspired by RoR?! I believe they are at least keeping a very big eye on it (Sauron style) but that’s just my to cents?

Asp.net MVC framework

And so it happened… Microsoft acknowledged that not all developers like the whole event driven/viewstate model.

Other web frameworks like the MVC model has been used for years on other platforms, alternatives like MonoRails have existed on the .net platform with a small loyal crowd of developers.

But now…. (my guess is because the whole buzz about Ruby on Rails and its MVC model) Microsoft launches their own MVC framework:

http://codebetter.com/blogs/jeffr…mvc-framework-at-alt-net-conf.aspx

At last… I can’t wait to try it out and to see how adoption goes.

Don’t rename the title field in sharepoint lists and doing caml lookup by id

I guess alot of you that if you rename the Title field in a SharePoint list, the internal name is still “Title” (only the viewable name is actually changed). I had such a scenario where another developer had renamed the Title field to E-mail because there was really no need for having a title field in the list.

By itself this posed no problem, but when you work intensive with CAML problems quickly add up. When writing CAML queries in SharePoint you query on the internal name of the field. This didn’t pose as a problem for me, I just made a comment in the code so my fellow developers could see why I’d written “Title” in my query when the “E-mail” was what I needed. After I then ran my code nothing happend. SharePoint just returns “0 found posts” if anything is wrong with the CAML query making it hard to really see where the error is.

Somewhat confused by this, I downloaded the Stramit SharePoint Caml Viewer. Great application. When looking at the data for the entire list, I could see another field called “LinkTitle”. When I used this in my query it magically worked.

While I found a solution, I was really too far out in the internal fields of SharePoint, so I deleted the list and created a new one WITH THE TITLE FIELD :). I’m not sure if this oddity was caused by all the renaming but the lesson learned (at least for me) was: Don’t mess with the title field…

If you’re still reading, a quick tip for working with lookup fields in CAML. A normal caml query with lookup looks like:

<Where><Eq><FieldRef Name=”Category” /><Value Type=”Lookup”>My category</Value></Eq></Where>

Nothing fancy here, but something that I’ve only found on a few blogs out there is how to do the lookup with the item id. Here’s how:

<Where><Eq><FieldRef Name=”Category” LookupId=”TRUE” /><Value Type=”Lookup”>234</Value></Eq></Where>

Happy SharePointing 🙂

VM-ware Fusion and remote desktop fun

While I’ve been using apple computers for years now, I’m new to virtualization.

Because we develop web applications using Ruby on Rails, we decided to work with apple MacBooks. This is really nice because we then look cool while writing cool code, but a challenge when we do consultancy work on other platforms. Currently I’m doing Sharepoint (MOSS 2007) development for a customer and for this MacOS just won’t cut it. For this we’ve bought VMware Fusion. I cannot praise this magnificent piece of software enough. It just works (and I’m still running with the beta version).

So a couple of days ago I had to connect via remote desktop to my MOSS development environment and change my default password. To change my password on the Windows 2003 server I had to do a “ctrl-alt-delete“, and then it hit me… How the heck do I do this with my MacBook keyboard?! I’ve started a Windows XP in VMware Fusion and connected to a (virtual) Windows 2003 Server through Remote Desktop (talk about virtualization :)). I search the net but could not really find anything about this cocktail, so I wanted to share my findings on this tricky key combination (or at least store it somewhere for when I need it again in six months):

fn-M + numeric enter (the funny up arrow next to the right apple key) + <– (the top right delete key)

I used like an hour to figure this one out 🙂

Shooting yourself in the leg with a bazooka

The colorful title reflects that this posting is about common mistakes done by good but relatively inexperienced software developers. Big mistakes that a developer actually need quite some skills to make. Mistakes that we unfortunately see all too often and we would rather not see much of again (hopefully this blog entry can aid a bit towards that goal):

1. Overly complicated design – Instead of a simple design for a simple project some developers insist on using a impracticable mix of all the newest, fanciest abstractions, design patterns, techniques and advanced language constructs that can possibly be combined in one software solution. All design elements have benefits and drawbacks. Great (experienced) developers know when a particular benefit of an abstraction, pattern, technique or feature outweighs the drawbacks. There is no silver bullet. No design element works well in all situations (just as no rules that you learned in “programming school” are in fact absolute). Developers that get way too eager ends up with one big unmaintainable design mess with the combined drawbacks of all decisions but few if any real benefits remaining…. Remember, a simple design is a beautiful design!
2. Writing too much code – Writing lengthily code with a high maintenance cost by hand when writing such code can be avoided. Much code can be avoided by choosing a more suitable design/architecture, using standard framework/library features (xml serialization is a common example), using techniques such as dynamic reflection or specialized code generation and aspect oriented tools (be careful though).
3. Reinventing the wheel – Some developers think they can write their own code much faster than learning to understand the underlying framework and available libraries. This might in some cases even be true, but when considering overall quality and maintainability (which developers seldom do) reinventing the wheel is almost always a bad idea.
4. Incorrect use of advanced concepts such as multithreading – Some developers use multithreading without the discipline and deep understanding that writing correct, safe multi-threaded code requires. Multithreading can improve the user experience immensely. It is also the answer to scalability nowadays. However, before even considering to use multithreading in your design, make sure to know the theory and features in your environment well. A vague recollection of mutexes and semaphores from school is not good enough. You should also realize that by deciding to use multithreading, you generally need to upgrade on testing, documentation, reviews and quality insurance.