The Sherpa Project

I was recently thinking about why I love ReSharper so much, and the biggest reason came down to achieving flow. It’s amazing the productivity and the enjoyment that come out of being put in an energetic and focused mindset.

The components of flow (from WikiPedia / Mihaly Csikszentmihalyi) are:

• We are up to the activity.
• We are able to concentrate on the activity.
• The activity has clear goals.
• The activity has direct feedback.
• We feel that we control the activity.
• Our worries and concerns disappear.
• Our subjective experience of time is altered.

Although ReSharper doesn’t provide all of this directly, it’s the glue that tends brings my IDE experience together.

If you haven’t seen it yet, be sure to checkout CodeKeep. The VS.Net add-in is great and makes the whole process extremely smooth. Great job Dave!

Anyone that knows me knows that I’m a huge fan of TDD. Not just TDD for testing - but TDD for architecture (and many other things too).

I’m currently working on a project where I generate a lot of pdf documents. I had a couple of big-picture ideas for how to write the code, but there were a few things that I knew would emerge and become clearer as I starting writing the code. When we finally picked a pdf-library for the document generation I quickly found out that the API was horrendous. There were areas that this significantly complicated the “ideal” architecture for the application (at least the way I had originally envisioned it).

I didn’t try to do much BDUF (intentionally), so I figured I would just let the code and the circumstances tell me what to do and this pretty much how it went…

1. I wrote the implementation for the drawing (the api I’m using is based on a nasty coordinate system used for generating pdf files) of the first document.
2. I wrote the implementation for the drawing of the second document, and the third document, and the fourth document.
3. I noticed that I was duplicating a *lot* of code (it was nasty code too).
4. Since I had tests around everything, I figured I could easily refactor and remove the duplication. Unfortunately, I couldn’t figure out how to refactor the duplication out to something clean. At this point, I got a little worried. I typically like to remove duplication as soon as I notice it. I had a lot of duplicated code in three different places and to say the least — I hated knowing that it was duplicated. I wasn’t sure of what to do so I kept duplicating code. The code that was being duplicated wasn’t one-for-one, but it was close enough that it was obvious that at some point it should by consolidated and reused.
5. I wrote the implementation for the fourth document.
6. I started to see some common patterns where I could start to pull out the duplication.
7. I wrote the fifth implementation.
8. I had a good weekend.
9. I started refactoring and was finally happy (thank god for tests).

I usually follow the red-green-refactor cycle but in this case I just wasn’t sure how to refactor. After lots of duplication and more experience I learned more about the API that I was using and this allowed me to refactor back into code that *felt* right. I still have some duplication, but I’m sure as I write a few more implementations that the duplicated portions will work themselves out like the other duplications did.It’s interesting, because I would usually have recommended to remove the duplication at the first (or second) sight of the repetition.

I’ve done three projects within the past couple of years where I have built the same server components for asp.net-based applications. It seems like everyone is using custom phone controls, date controls, calendar controls, credit card controls, zip code controls, etc. It’s always the same thing over and over.

Every time that I’ve done this, I’ve always felt that it wasn’t worth the effort. The requirements always change for the interaction of these controls and when you look at the final price of the man-hours to build these things it ends up being ridiculous. When you look at it on the surface, these are simple things. For some reason, it just always seems to take forever to get these things built and tested.

Here’s my advice… don’t build them if you can avoid it. Take a look at the controls that are for sale and evaluate them first. These are common problems with common solutions and there is no reason to re-invent the wheel over and over. If anyone has any control recommendations, feel free to leave them in the comments. I’ve never seen a really good control comparison for what is out there, but if you know of one then I’d love to see it.

If so, you can start with delivering less.

According to the ChAOS report by the Standish Group…

• 42% of implemented functionality is never used
• 18% of implemented functionality is rarely used

Which basically says that 60% of what we do is waste.

Eliminate that 60% (or at least the 42%).

If you aren’t sure where to start, I would recommend taking a look at Lean Development and the Predictability Paradox.

I’ve been involved with building mutliple app-dev frameworks throughout my career and it never fails that the frameworks end up being littered with individual application-specific code or hacks for single applications that use the framework (this seems to be most prevalent in UI frameworks). After this happens multiple times, it seems like there is really no need for the framework at all because it moves from the lofty idea of being an enabler for individual applications - to being a “this is an emergency and nobody else has time to do it

It never ceases to amaze me how complicated we tend to make things. It’s mostly learned behavior or a desire to implement the latest and greatest pattern that we’ve read about. I think that over-engineering is really one of the biggest problems that our industry faces. It’s prominent with developers, but it’s also common throughout all ranks and positions in our industry.

Unfortunately, this adds up to massive amounts of waste. We write a ton of features that never get used, we come up with really cool implementations that are really hard to maintain, and we spend a lot of time worrying about how to increase the speed of our app by a few milliseconds when it’s likely that the performance is sufficient. We think we have to concurrently handle China when our app will probably never have to concurrently handle the number of people that live on our block. The entire time that we are doing this, we are likely wasting the time and money of our clients or our organizations.

The next time you’re at work take a few minutes and notice the interaction and thoughts of the development, management, and business teams. Is everyone focused on delivering value as soon as possible to their clients? Do the daily discussions revolve more around features or more around layers? Does everyone believe and think about the intended value of the system that is being worked on, or are they just focused on writing a good data access layer or a scalable middle tier?

Imagine what would happen if you did the simplest thing that could possibly work. Imagine that you implemented your application in vertical slices of business needs versus horizontal slices of technical needs. Imagine if you allowed the architecture to evolve with the code and you let the system emerge rather than spent the time upfront to diagram out every possible scenario. It all sounds so simple – but as industry we have a lot of unlearning to do.

I was working with Drew today and preparing for our .Net Users Group presentation on leveraging open source development tools with .Net. For this presentation, we have consistently met at Panera because of the free wifi access. When we arrived this evening we were unable to connect to the wifi. At first, this seemed like a pretty big problem, but it ended up being a perfect scenario for how mock objects can provide a significant advantage..

My pieces of the presentations are centered around NUnit and NMock. In the example code that we put together for the presentation, the data layer is comprised of an external (XMethods) web service. Without wifi, we were obviously unable to connect to the web service. The wonderful part about all of this was … it didn’t matter.

Since the tests in the business layer utilized the functionality of NMock, the tests all ran as expected. The only exception to this was the suite of integration tests, which confirmed that the service wasn’t available and that the integration tests were doing their job correctly.

So, even without the core service that we needed to talk to, Drew was able to setup NAnt scripts and get Cruise working with full unit test integration. We would have been able to do a portion of this work anyway, but we wouldn’t have been able to do any refactoring of the tests or code modifications with full confidence that the code was still functioning as we expected (the test would have all failed instead of passing).

There are a variety of scenarios where mock objects can be a wonderful addition to your toolset and development process. This scenario was a prime example. We didn’t control the internet connection at Panera, but it wasn’t able to stop us from moving forward and preparing the presentation or performing the integration of the code and various toolsets.

A couple of other places where mocks (implemented correctly) can save the day include:

• When your unit tests take a long time to run (the faster the feedback the better)
• When systems are out of your control (that big mainframe app that goes down all the time)
• When systems are not always available (when that big mainframe has to be rebooted every night and takes a couple of hours to get back up successfully)
• When systems return dynamic results (such as stock prices, current temperature, the current election results, etc.)
• When a system doesn’t exist yet (parallel development, etc.)

If you are interested in learning more about how to implement mock objects in .Net, take a look at this MSDN article. The article provides all you need to know to get started with NMock up and running.

The MSDN Software Factories page is up and running (and worth keeping an eye on).

The Software Factories web site defines a software factory as: A Software Factory is a software product line that configures extensible development tools like Visual Studio Team System with packaged content like DSLs, patterns, frameworks and guidance, based on recipes for building specific kinds of applications.

I’ve had the opportunity to participate in the creation of this type of system for an order management product (a couple of years ago) and to say the least, it was a very interesting experiment. As a result of the creation of the system, the business was able to realize substantial value when new customers were enlisted for the product suite. The software factory approach worked very well because the company was an application service provider based around B2B procurement solutions. Although this is a big space, the majority of companies are using similar workflows and have (roughly) similar objectives and pain points in their current solutions. We were able to take the time to implement an end-to-end solution (previously done in a one-off manner) from approximately 6-7 months to 2-3 months (for full life cycle). The development time was drastically reduced.

We were able to automate the majority of the construction of an implmentation for a new customer and get them up-an-running very quickly. The presentation, business logic, and workflow layers each were split into their own domain-specific languages and the data layer could be reused across all of the customer implementations. The data layer was able to stay common because we mapped all of the customer-specific document formats (think edi, flat-files, etc.) to an internal standard. The presentation, business, and workflow layers were usually able to be at least partially generated and then hand-modified for customization. All of our domain-specific languages were written in xml (similar to NAnt) and although we didn’t have any graphical tools, we were able to use XSD and XML Spy to provide autocomplete functionality and syntax highlighting.

Overall, I really liked the software factory approach. It was a good fit in this case because of the repetitive work that was done by the ASP to implement solutions for their customers. It’s nice to see some movement in this space, and I think we’ll likely see a lot more moving forward.