The Joel test

Yesterday I got asked my thoughts on the “Joel test”, as a good friend of mine got the bad news that his development team is scoring just a 7 on the Joel test. He wanted to know what it is and “Is that score a cause for concern?”

This post is going to be a little tech focussed as I am sure you are guessing, but if you are a CEO/CFO and want to know what’s going on (what you’re spending your money on in an IT development team), then you will find this post of value.

Now, I’m not a strong believer in trying to measure your development team success or their strengths by any means other than, are you happy with the product being delivered? All too often we get caught up in some form of metric for measuring just how good something is, and while we are doing that and maybe getting great “scores” we seem to lose sight that the product being delivered is actually poor. However, that being said, I think the Joel test is a good indication of your software development environment, and if that is in good working order you are at least giving them the best chances to succeed.

The Joel test is dead simple, and though I’ve read lots of opinion on it not working for Agile, I simply have to say – use a bit of common sense and apply it in the correct fashion to your preferred development methodology. I am a strong believer in agile and SCRUM, we operate that here religiously, and I would say our Joel score is at 11. Not the perfect 12, simply because I don’t always fix bugs before continuing on with new development work, I personally prefer to address bugs towards the end of a development cycle.

So here we go, the Joel test:

Do you use source control? You must be saying YES to this, simple as that. Good source control will also provide you with build services for continuous builds, see a later question.

Can you make a build in one step? Should be a YES. Builds or build scripts or continuous build services ensure your code is at least always able to build and run. When a build is broken, you have to fix that before anything else, and what’s great about a continuous build is you find these problems out sooner rather than later.

Can you make daily builds? See above I would say

Do you have a bug database? You MUST have something like this otherwise you have no hope tracking issues and fixing them. You don’t even need to be that sophisticated, though I like my UAT testers to push bug issues into the same control we use for specifying out storyboards (SCRUM).

Do you fix bugs before writing new code? This is the one I let slide. I make sure everyone is aware of them, and if they are in an area of the system that will be worked on then YES, let’s do that. However, often bugs are not in the same areas, and in such cases I prefer to keep the development velocity up and come back to those bugs at a specified date and time (typically the start of the following development SCRUM).

Do you have an up-to-date schedule? Now some will say NO to this as they use XP or something. Personally XP is hit and miss. SCRUM lets me specify out what storyboards we need to work on, and then we work on them. We don’t have an old fashion specification as such, nor an old fashion schedule, rather we have lightweight roadmaps and storyboards, because that is what SCRUM needs. So I still answer YES to this question, though we use SCRUM.

Do you have a spec? You need to have some spec, so if you answer no to this, then your development efforts will fail. SCRUM provides developers with a spec in terms of the storyboards they follow with the identified tasks. Without them, you have no hope.

Do programmers have quiet working conditions? This should be a yes, even if you are using XP. Collaboration is always ok, but the conditions will on the whole be conducive to concentration.

Do you use the best tools money can buy? We do, but I don’t think this is the end of the world if you write no to this. I personally like to push the team forward as the best tools typically help productivity.

Do you have testers? I hope you answer YES to this.

Do new candidates write code during their interview? This is harsh, but I insist on this, and what’s worse I insist it be done with just a pen and paper. I’m not looking for syntax, rather a good understanding of OOP and problem solving.

Do you do hallway usability testing? Not sure many people do this, but I do like it. I especially like expanding this out to focus groups if and when you get the chance. If you don’t have resources for this, hallway testing can be easily completed, just get some friends and family involved J

Anyway, that’s my take on the Joel test, don’t get too hung up on your score, but like Joel states, a score lower than 10 indicates serious development problems…I would probably say lower than 9 is big trouble…

The Hacker mentality in development

There is a lot to be said for hackers, many of them are, well quite simply, brilliant with computers and poses some great knowledge. I’m not approving of what they do, as I think those actions are disgraceful and show a blatant disregard for the law, privacy and peoples livelihoods. However, many companies build their software with a hacker mentality, which is “get on and do it, make it work” kind of thinking….Lots of doing, little planning….I can’t think of anything positive to be said about having a “hacker mentality” when developing a piece of software or a platform. Such a mentality is ok for building proof of concept parts of software, you simple cobble together a rough solution that shows the concept and you use it to overcome / address some technical difficulties you may see in the future. However, take that mentality and approach into the actual project and it’s only a matter of time before something comes back to bite you…

 

Facebook

Facebook is the biggest and most obvious example I can think of. The platform and social network is brilliant in many ways. However, spend much time trying to use it for business, for managing ads, campaigns etc and you get the feeling that everything really is “cobbled” together in true hacker mentality fashion. Things just don’t seem to play well or consistently and you seem hamstrung by so many restrictions which have been basically caused by not enough forward planning, rather developers just doing…You really do feel like the platform was built for its one role, and that everything else is almost stand alone trying to be bolted to the UI.

In recent weeks I have watched a digital marketing agency wrestle with Facebook in order to get it to behave how it really should. It’s not the agencies fault, far from it, once they managed to get someone from Facebook to talk too; it became apparent that most of the staff at Facebook are aware of the amount of bugs in the system and the restrictions that are in play. For me, the whole experience is at best “proof of concept” and essentially shows that there has been a real lack of forward planning. Facebook is a company though that was founded largely on “doing” rather than planning…

 

Don’t get me wrong, there are some impressive concepts with Facebook and many parts of it are implemented very well, but you cannot get away from the fact that anything outside its original core feels like its been added on as very much an afterthought and that its very “prototype” in terms of feel and implementation.

 

Technical Architecture

I’m not going to bang on about the right way to do things, rather just to say that any development project really needs certain key roles, and the biggest two are ETA (Enterprise Technical Architect) and TA (Technical Architect). These roles don’t fit in with the hacker mentality at all. If you have two people who fill these roles, and do their jobs well, you will find that as a project grows and new requirements come on board, that they can be added to the system in quite an efficient way. Tested separately and rolled out smoothly. The new additions become seamlessly part of the software and things don’t feel cobbled together and they don’t feel like prototype or beta software. Why? Well because these people plan for the unexpected, they see the bigger picture and look at what potentially one day may also be required from any given part of software. If you plan your system like that, when new requirements do come along, you can build them into the overall solution without causing any “jarring” effects within the software or for the user.

Having people who ensure you plan all components for the future keeps longer term costs down, and delivers software that can truly grow with your business. There is nothing worse than having to go back and re-engineer code because it was implemented with only that one function / requirement in mind…

 

Adaptive, SCRUM, RAD…

There will always be a better way to run your IT projects. New methods for ensuring you get code developed quickly, built to specification and future proofed. However, without having people who think ahead at an enterprise level (regarding technology, architecture etc) or people actually ensuring developers do the same, then no matter your methodology, changes further down the line won’t be easy, rather they will be quite troublesome, slow to implement and often result in a cobbled together field, ala Facebook ads management…

You simply can’t beat people who forward think…

QA and Testing…Use a model office or test live….

This past couple of weeks has seen me overseeing some of our components as they go through QA and testing phases, be it early phases of a bunch of our products. I have also been keeping tabs on some of the testing of solutions we are providing for clients, and to be honest, I do feel we have a great approach to QA and testing. We always ensure components are tested in full in house, and then go through a number of testing iterations on either a model office environment or the actual live environment…

However, in the past month I have also seen a number of “testing phases” by organisations that simply are not realistic, and to be frank, mean you will need to repeat all that testing and work again…So what is the issue. Well, typically it is testing in environments that don’t quite match those that the final system will be implemented on…Now to me, that is mad and asking for trouble, and yet there are endless organisations out there that do this….

Making the testing phase easier…

The primary reason for cutting corners or not testing on exact replicas of live environments is admin and cost. To set up too exact environments will take time and effort, and for some organisations they just don’t want to invest either in this area of development…

Because of this, there are a number of options project managers take, some are simply to use whatever environment they have that is close to the live one for testing. Others are to utilise automated testing tools too much (to save on man power and to shorten the testing period) or finally, others use cloud computing to cut down on the admin and set costs, while maximising the potential users / tester access to the solution.

I have no real problem with any of these, however I would say, if you are in this situation, utilise what will be the live environment for your testing, if possible, and simply clean it down when you ready to go live….Whatever you do, don’t though rely on just that testing if there is any difference, no matter how small between to the two environments…Only today I have seen that an updated component on the server (different to that of the testing environment) caused what potentially could have been a big issue. However, luckily, the client in question took our advice and was testing the solution from scratch on the live environment…

Using the Cloud to test…

I have read that people are now looking to use cloud computing to carry out their QA and testing. Again this is fine, if your application and software is to be running in a cloud environment and on the servers provided by the same cloud provider. If not, it is still fine, as long as you then test in full again, on the “live environment”…

With cloud testing you have to remember, that you may have a complete change of underlying server components without actually being aware of it. You may also find that your cloud provider uses other cloud providers for certain components, making it even harder to be sure just what components are where (and what version). So this testing requires strict control and good relations with your cloud provider.

Unfortunately I have now seen software being tested in this way that is to be implemented in house, not up on the cloud. For me this is a massive no no….Why are they doing it? The answer is usually a simple one; it keeps the cost of testing environments down and ensures a wider range of users can gain quick access to the system to perform QA and user acceptance testing. Especially since what will be the live environment isn’t available (for whatever reason).

The basics of this are ok, however, when you get into the live environment you need to repeat all those tests, otherwise you leave yourself open to a raft of issues that will raise their ugly heads, not necessarily on go live day, but at any point…You see, any small difference can potentially have a massive impact, and testing and QA is all about minimising the risk of the system going wrong. If you don’t test on the correct platforms then that risk can only be increased, no matter how much testing you do…

Model office

When I first started in IT, I was told by a wise old man (he actually was old, and looking back, very wise, oh, and short tempered), “when testing, for God sake ensure everything is the same as it will be in live…..you can never test too much son….never…” He was a strong supporter of using a Model Office environment, and by model office environment, he meant an exact replica of the live environment…Right down to hardware used….He also believed that environment needed to be there long after go live was complete, and that any components that were to make their way to live, (even if OS related) should be tested on the model office in full first….Now I thought this goes without saying, however my first exposure to a project that required a lot of testing was for a very large bank. It soon became apparent that their testing environment wasn’t an exact replica of what would eventually be the live environment. There were subtle differences in hardware (which it turned out did come back to bite us), there were even some instances of components on the testing environment that were and would be newer than those on the live…

What has to be understood is, testing is as important as design and development phases. It cannot be scrimped on, and your testing environment should be there for the life time of your solution. If this isn’t possible, you need to come up with a way of making this possible, if not, your QA and testing will always leave you at greater risk come the day the system goes live….