danux.co.uk :: Homepage of Daniel Davies

In a typical X was chatting to Y situation, with X being Adam from work, and Y being myself, I began thinking. We've been working together on and off at work for the past few weeks on a Django site. This is my third site in Django and feeling quite competent in the basics I've started biting off bigger tasks and thinking beyond what the framework provides. This had lead to some pretty decent code being written as we adapt Django, in particular the admin side of things, to the needs of our company.

So as Y was gloating to X about the latest addition, some fancy javascript that allows images to inserted into a tinyMCE area from the Django admin, it occurred to me that at best maybe 5 people will ever see or use this code I'd written. Of them only 2 people, X and Y, would have any understanding of its inner workings, its beauty and its elegance. Thus, over 50% of the users will simply click a button, see their image or file appear and continue doing whatever task it was they set out to do.

I've always held technical excellence as perhaps the highest metric for good software but as I gain more experience in the commercial world of development its importance is diminishing. In fact, let me rephrase that; I've begun to discover new metrics, and their relative importance has been rising at the expense of technical excellence. I realised that the real beauty of the code I'd written was the very fact my end user will never notice it. As I said, the button will be clicked, and on they go.

I remember a few years ago watching an interview with someone from the Lord of The Rings films who had been instrumental in the creation of Golem. He remarked that the thing that really set the film apart from the average High Fantasy adventure was how little people noticed the special effects. The fact that Lord of The Rings had made industry break throughs with its computer animation advancements was ratified by no one even noticing. The character blended seamlessly in to the story without the viewer ever needing to have their attention drawn to it.

I guess this is similar to my feelings now. The whole is greater than the sum of its parts comes to mind, and its something I think has taken me a long time to really understand. Clever widgets may impress, but what really matters is their ability to blend seamlessly into one coherent whole.

So where do I stand now? Well, technical excellence should be a given. Bad code simply shouldn't be written to start with. But what I now strive towards is this new kind of beauty I have discovered, a completely functional system that does exactly what the client needs. A simple, elegant whole.

Phew, at long last I've got some real time to update this site. I've been extremely busy over the last month, the main reason for that being a new site I have just this minute finished launching - Amarus! The first post on there explains my reasons for why I've started (or relaunched) Amarus so I won't repeat my self here.

Amarus uses exactly the same software as this blog, I literally just had to reskin it and it was ready for launch. But... as I started going through the code I naturally started to notice the little 'code-smells' that live in all systems built in a rush. I took a bit of time to refactor and improve the code and will be able to merge my improvements into this site in the next week or so.

Yesterday I received a call from my landlord at roughly 4:34pm informing me my room had been broken into and that the door had been 'done in'. Thankfully nothing was taken, despite my macbook being sat on the bed and my desktop fully setup. Semi-conscious that a return visit could be due I've taken basic security into my own hands by installing a do-it-yourself security kit. A blog post will be due in the next few days once I've finished tweeking the system but currently I have a few precautions in place.

When movement is detected around the door the camera begins to take burst shots. These shots are then sent straight up to Apollo, my VM for safe keeping online. I can access the photos from work too. The shots are also kept on a USB pen which has been bolted firmly under the desk in case the Internet connection fails. I've got a few other tricks planned as further fall backs should the two mentioned fail.

The pictures aren't too bad, no worse than the quality of CCTV you see on Crimewatch, here's a couple of snaps it took while I was testing it all:

About a year ago I ditched my web hosting company in search of more freedom. I wanted to control my own email, I wanted to configure Apache exactly how I wanted, be able to modify the php.ini when ever I wanted. I had limited options:

1. Get a dedicated server,
2. host from home,
3. continue paying a web host

One night I was talking about this dilemma with a friend who told me that he had a 'virtual server'. He explained to me how its designed basically for web geeks and small companies who want their own setups, but cannot afford for a server, taking up an entire unit in a rack. The concept is simple, one physical machine running 64gb+ of ram, vast amounts of RAIDed storage and several quad-core processors hosts many 'virtual machines'. Each virtual machine essentially runs it own kernel, linux or BSD being the typical flavours, that sits on top of the host machine. This provides users with their own OS environment to do with it what they want. I of course chose Debian GNU/Linux.

But now I was on my own. If I broke something I'd be charged per hour for tech time, what happens if I crash the machine and all my mail gets bounced and lost? What if someone hacks me? There was a lot that could go wrong, but thankfully I had  a number of freebies and useful services included with my vm.

• A backup MX server that will hold and forward you any mail which can't reach your machine,
  
• three DNS servers to host your domains, or act as slaves to your own DNS server,
• internal repositories so you can get your packages extremely quickly and without paying for the bandwidth,
• SSH access to the host machine so when everything does go wrong you still have a life line to your machine. This has been a life saver one more than one occasion.

Everything turned out to be a lot easier than I could have anticipated. The freedom I now had to run my own BIND server, my own Postfix, Apache was just how I wanted it, all the PHP modules I wanted installed. This is what I had been looking for and I've recommended virtual machines to many people like myself, web developers and Linux geeks that want a remote machine but are scared off by the committment to a dedicated server. A year on and I still recommend them to anyone who reads this.

The price increase from a semi-decent web host to a VM is minimal, and I guarantee in no time you'll be using the vm for more than just web hosting. I now run Irssi/screen for IRC, multiple SVN repositories and host a number of friend's sites. I've taken advantage of the compile farm a few times too. When I think back now I simply couldn't imagine using a web hosting company again.

Here's some links for those interested:

• Bytemark are the company I have my virtual machine with, I can't recommend them enough.
• Xen is the brains behind behind my vm and its free (GPL).
• Wikipedia has a really nice comparison of virtual machines, you'll find more here than I could possibly tell you.
  

DigiKev made a good post over on his blog about web designers and graphic designers being different disciplines, and the problems produced by this. He however came to the web through graphics, I came through programming, so I guess our responses to the issue are bound to differ slightly. I agree, they are two disciplines that have collided in a great big muddle that is the web, but lets also throw us humble programmers into the mix too, and I'll take up our corner. To save confusion I'm going to lump all crayon wielding monkeys into one group; the designers. Mostly because I'm a code monkey and all I really know about design is not to put said crayons in to my mouth.

Firstly, the MVC pattern. Its a great idea and it works really well, but it has lead many people to believe that we should keep our coders away from the HTML. In reality the HTML view is only a small part of the systems, and I fail to see why we don't want our developers writing HTML... after all its the bread and butter of web development, and where many of the modern day programmers actually started out. Coders should really be better tuned to produce HTML than a designer, so why are we burdening designers with the task? Surely all they need is a quick 101 on what's possible, a quick word with their PM if they're ever unsure of anything and they should be producing good designs in no time which a skilled programmer can markup in no time.

This leads me on to the second reason, WYSIWYG editors such as Dreamweaver. These were really popular before web standards become the norm because they were packaged up in graphic suites along side the likes of Publisher and Flash. They allowed designers to produce HTML quickly and easily with no real knowledge of it, its standards or its affects in different browsers. The issue of design feasibility is non-existant when throwing everything into a table is the industry norm. Why waste your skilled programmers on boring WYSIWYG tasks which are packaged and sold as designer tools? Of course these editors have been ridiculed into the obsolete at the top of the industry now, web standards are so vital these days and our new memes are all about quality HTML which allows SEO, good navigation, AJAX, DOM, etc... Javascript is finally at a point where it can be used with relative ease cross browser.

This makes it difficult for our designers, who we've come to believe should be producing the HTML, and theoretically can because the MVC is so wonderful, actually can't. The changes in web development have come about because of the programmers with a deep background in standards, RFCs and knowing how to scale systems pushing for the standards. Producing good HTML is really quite difficult now, I know a lot of people who work in web development who don't understand the importance of the label tag or title property - if the W3C validator doesn't pick you up on it why should you care?. We shouldn't be expecting our designers to know how navigation structures get coded, or which properties of CSS actually make rollover effects possible, they should just be designing. But if programmers are too busy programming, and fixing all the HTML thats not going to work after all, then who is it that can stop clients getting shown a design that's impossible to markup?

The third reason for our problem I believe to be the project managers. Every computer science graduate leaves University with a brain full of case studies on London Ambulances and planes crashing in Holland due to bad software causing a nose dive below sea level and poor project management failing to spot things which we as developers blog and complain about daily. The concepts of XP, $meme-of-the-month driven development and waterfall models are drilled into us, yet still the industry is full of PMs who fail to utilise these very software specific skills from their development team. I'm not going to argue that all PMs should be programmers, ideally they would, but that's about as feasible as making programmers graphic designers. If PMs just learn to trust developers more, and manage the channel well between them and design I think we eliminate many of the problems.

So where does that leave us developers... designers who can't cope with MVC because they still think Dreamweaver is a good idea, and people running our projects who don't have any knowledge of actual web development? Thankfully its not all doom and gloom.

Most of the 'older' and more 'seasoned' web developers I know were brought up without the web. They were software programmers who moved out onto the web. Original designers were in the media or digital design creating things that didn't need to be stuck together with HTML. There is a new type of developer entering the industry and I am fortunate enough to be part of this generation of programmers who have their entire background in web applications; right from the age of 13 I was writing HTML, before I even knew what a compiler was. I branched from HTML into programming, producing software for the web... not a desktop computer, not a server; a website. I became a web developer with no pre-conceptions on development, and everything I learnt about software development at university I adapted to the web.

There are also many pure 'web designers' now who can produce good HTML, or at least good designs, because they got into design through writing websites, building their own homepage, skinning their clan's forums, their blog or myspace page, etc. Everything they learnt on formal design courses they applied to the web, because that was their access point and root in their chosen profession. Sure they may struggle with scaling all the frontend java, html or css for a massive commercial site, its become a programming discipline like I said, but they will understand enough to produce good web designs ready for their progammers to come in and bolt on the javascript and interactive bits and then let the developer slot them into the MVC.

The old hat's have left us with a real muddle to sort, but they've made many mistakes we're able to learn from. We needed them to come before us so that we could see what not to do just as much as we need to see what we should do. I think this can only be a good thing for the web and as this generation of thorough bred web developers enter the industry its suggesting a very exciting future.