As I was coding away on the train today, for a few minutes my mind blanked while watching the landscapes Virgin offer to us regular travelers. The did an amazing job at decorating the length of the track, to the point where you'd nearly get fooled into thinking there are actually villages, cows and other things between London and Manchester. As it happens sometimes, I started thinking about my early days as a fresh and learning developer, many many years ago.

Against the normal academic training some IT people receive, I evolved as a developer from the top down. I didn't do University and got no degree in IT. I actually got as far as studying law. I learnt everything by starting to write HTML web sites back in 95, then started JavaScript in 96 and VB in 97, and discovered JScript.net and C# when I got through a friend an early preview of what was to become .net (NGWS anyone?). And I'll admit that while I understand how x86 works and how pointers work, I am absolutely unable to write a single like of C++ or any other esoteric close to the metal languages (I hope esoteric here is not taken as a challenge of C++ usefulness, as I'm sure the gurus amongst you will have understood). And it's only after all these years of coding that I start understanding calling conventions, Win32, etc.

It all got me thinking about how you understand a language and a platform. Until you reach the thorough knowledge that only years of experience will bring you, you often ignore the why question, preferring the answer to the how: you care about how you achieve a result, with the acceptance that you do not understand why it needs to be done in a specific way.

For example, in JavaScript, you'd add a method to an object type by adding it to its prototype property. You know that instance methods and type methods are two different beasts intuitively, because var $cube = new Cube(); $cube.getValue = function () { return 2; } will not work for another Cube you'd create later on. This empirical knowledge is surface magic for however long you'll take to dig deeper, learn and discover what a prototype is and how it relates to objects. At this stage, if things go wrong you'll probably have to learn more things. And it's an open ended confusing thing because both $cube.getValue and $cube.prototype.getValue can exist.

Another example comes from the WPF world.. Whenever you create a custom library of Controls for WPF, you learn about themes and how styles and templates get applied to your controls. You know that the themes being selected for a control are the end of the line customizations you provide, and you know you have to declare an attribute at the assembly level, and we're still very much in the domain of empirical knowledge. But you also know that the fallback when you don't provide themes is a file in the themes folder called general.xaml. You know the how but the why has no answer. It is a decision the framework designers took, and you have no way of changing it or influencing it in any way. This by design rule is crystal magic. The behavior cannot be changed, and once you learn how to do it, the why question is irrelevant. And if it doesn't work, there's no additional knowledge cost, because you don't need to learn anything else.

My point in all that is that these two magical elements have conflicting developer audiences. More Junior developers (and some senior developers I've worked with in the past), let's call them Mortimer, are happy with crystal magic, because it massively reduces the surface of the API you need to learn. More senior developers that have a will to understand (anyone that uses reflector as his main documentation tool would fall in this category), let's call them Einstein, tolerate surface magic because they can crack the surface, dig deeper and know all the intimate details of how the surface is achieved. They don't necessarily like crystal magic because it just works and they have no control over it, but as it's a closed ending, there's nothing that can be done about it, so they move on.

Everything would be for the best if surface magic worked for Mortimer. APIs like WCF provide a very simple surface magic, with only ServiceContract, OperationContract and DataContract concepts to learn. With the tooling support for WCF services in visual studio, the surface is small enough for most developers to start being productive very quickly. The reality strikes as soon as something unexpected happens: passing too big of an object graph, or worse, a cyclic reference within the DataContracts. Suddenly, Mortimer faces understanding notions such as bindings, behaviors, configuration files, and many other concepts that multiplies exponentially the API surface they need to understand before they can fix a problem. You just designed an API that targets Mortimer when everything works, but that requires Einstein to fix if anything goes wrong.  That said, tooling support is appropriate for being productive, which should ensure WCF adoption, so we're pretty close to the sweet point.

WPF on the other hand has a bit of both. Xaml is easy enough for Mortimer: it's understandable and can be fixed by Mortimer, because most of the bindings between DependencyObject and Xaml follows a crystal magic design. The property store will map Xaml and DependencyProperties based on the name, and nothing you want or can do will change this. You also very much pay as you go when you learn Xaml, because you can start with simple object graphs, and discover attached properties at a later time. You can use them without understanding them, and often you'll be able to fix issues by learning a bit about the Binding syntax. On the other hand, it's a very large platform and requires much more time than Mortimer would spend learning a new technology. Areas like Bindings are complex when they don't work, outputting errors to the debug window is confusing when you don't know where to look, the coexistence of logical and visual trees is difficult to grasp for winforms developers, etc.  Tooling support should help, but from what I see in beta 2, I don't believe the sweet spot will be hit with this version of Visual Studio.

And finally PowerShell is a deliciously efficient crystal magic solution. Things work as expected, pipes and formats work by magic, using a defined behavior that will not change. The framework is static enough that people are confident that the knowledge they learn using the script engine will be valid whichever Commandlet they call. The built-in variables work automatically. $_ will always have the same meaning, $$ too.

What Microsoft doesn't have anymore is a completely crystal magic solution for development. ASP was very much crystal magic, ColdFusion was too, back in the CFML days. Early days VB had few extensibility points which let developers grasp the platform more easily. PHP still has a bit of that feel, just like many other frameworks that limit choice and extensibility to increase the pace at which their developers become experts. When most of your developers know 80%+ of your API, you know you're on the right track. Anything under that and you face a problem: You're segmenting your market between Mortimer and Einstein. Your Mortimer has to get more knowledgeable to survive your technology update, your Einstein will just keep doing his expert job. The low end of the Mortimer market is gone to join the ranks of other easier and simpler technologies.

The sweet spot is not to try and provide one API for everyone, but to provide one crystal magic API for the quick and easy, and a completely different solution for Einstein and upgraded Mortimer.

Getting dynamically typed languages with the DLR is going the right direction. But now, this is a call for action to you Microsoft: Provide us with simple tools with a lot of magic to gain back the people gone to LAMP. Give us a forms design tool that maps any control on a form if the form is named after a table and a control is named after a field in that table. Give us a web tool that simply easy work with no mvc, no presenter, and a simple script language to inject a bit of server side code where it makes sense.

And remember that a crystal magic API is one that you *do not extend*. Not everyone needs a framework.

P.S.: This is just my opinion on why we're loosing so many people to php and other simpler technologies. I have a lot of designer friends that just plain refuse to touch any Microsoft technologies, because PHP is so simple. I'm probably not the target audience for any of this. Any resemblance with current or previous coworkers is purely intentional. No guarantees implied or granted. Do not use in a nuclear facility. Etc.

One of the piece of code I disseminate on each project I work on is a lightweight factory builder. The idea is that you declare which interfaces are implemented by which types, or constructed by which builder. It's perfect for replacing all the types you abstract when unit testing, and put the responsibility of the dependency injection on the configuration file.

Here's a small example of how one would declare an interface as having a concrete implementation.

<factories>
  <factory interface-type="CaffeineIT.Blog.IConfigProvider, CaffeineIT.Blog.Demo"
           concrete-type="CaffeineIT.Blog.SettingsConfigProvider, CaffeineIT.Blog.Demo" />
</factories>

Very straightforward. From your code you initialize the type like so.

var myConfig = FactoryActivator.CreateInstance<IConfigProvider>();

The problem when unit testing my configuration sections is that they're read-only by default. That means I need to do some black magic swapping config files around when running my tests.

Well, it's not very well documented, but with the help of reflector, you discover the secret so many have kept to themselves. To make your ConfigurationElement read-write, override the SetReadOnly method and do nothing.

protected override void SetReadOnly()
{
    //do nothing!
}

I don't know about you but I feel like I'm cheating.

In the View-ViewModel-DataModel pattern that has been documented a fair bit for WPF, it is often the case that you want to have a testable class that wraps your data to expose an object model that will make binding from Xaml code easier. However, when writing prototyping or simply answering messages on the Msdn forums, as well as for the code examples on my blog, it's often the case that you want to have a sort-of ViewModel that you want to quickly hack together.

The solution is to simply leverage the code-behind of your window! Let's create a simple property on our window code-behind.

And now in the Xaml code you just need to bind the DataContext property to the class itself.

Voila! You can now bind straight to the properties defined on your Window class! But as the complexity of your project increases, you'll want to unit test the ViewModel, and for this you'll have to separate the code in a separate class. The way I choose to do it is to embed the object directly inside the Xaml code. With this approach, you'll have the a ViewModel class. Here I implement it by inheriting from DependencyObject to keep the DP we defined earlier, as it provides support for validate and coerce callbacks, but this is absolutely not necessary. You're more than welcome to implement your property through accessors and INotifyPropertyChanged.

And now I can declare this object in my Xaml.

I find the code very tidy. Through this mechanism, you can also bind properties from your view to the ViewModel. Imagine: View updates ViewModel updates View... You can abstract all the logic of your view in your ViewModel in an oh-so-clean way it makes me salivate every time. You can then leverage commands to encapsulate actions that modifies the ViewModel which update the View. No more tier leakage!