Once upon a time I had a very bad scare. One of my responsibilities was to maintain a vendor created application that handles the flow of commodity trades on an exchange into our risk management system. That particular aspect of my job was not my favorite part but someone had to do it, right?
This vendor application was primarily a configuration driven monstrosity. Messages are in XML and the conversion from the format that the exchange provided us with to the schema that our back end systems required was primarily done using a tag mapping approach. For example, if the exchange provides us with an XML messages and the counterparty in the deal was represented as an ID, say 12345, the application then maps that number to a mnemonic that is used by our system, say BGKG. Simple, right?
Yes, it is very simple. It also doesn't scale for a damn. There were around 500-ish of these mappings in production. And that's just the tag mapping portion. That doesn't take into account any business logic driven mappings, XSL files that mangle the XML, etc. This quickly becomes a pain point for maintenance, especially when there are potentially multiple different places for logic to hide on any given mapping. The logic could be in the XSL outside the application, the message logic built into the application, there are templates for each message route that can contain the logic, and several more places. On top of this there was nothing preventing the spread of logic affecting the same element(s) from being shared among ALL of these locations!
Now that you have more background than you care about, I will actually get to the scary part. When I first started working on this, the traders that actually execute the trades on the exchange have an ID assigned by the exchange. The means of mapping this to our system ID was to have a list of all the traders exchange ID's in an XSL file and test for a match, replacing the exchange ID with the local system ID. This is ugly for several reasons. First, it's long, tedious and error prone to maintain this list. I don't know when the last time it was updated before I saw it but I know that they never removed anyone from that list, ever. Even if someone left the company, they stayed there. Second, this means that the Risk Management group (the business unit owner of the application) was dependent on IT development (i.e., me) to make a code change, promote it from the development environment through the QA environment to the production environment every time they wanted to add a trader to the system. This can become painful for a company that grows through acquisition on a regular basis, by the way. I could go on but you get the point and I'm sure you can come up with your own reasons I haven't thought of as to why this is a bad way to handle it.
I wanted to empower the business users of this application to be able to manage their own traders in the system. They do it for all the other aspects and this is supposed to be configurable by the business users anyway. Configuration changes don't have to go through the same channels as development and I don't want a call while on vacation to run something like this through development to production because they can't do it themselves.
I started by creating a simple DB schema that holds look up tables exchange IDs to our system IDs (and a few other things). Nothing complicated but if I put that kind of mapping into a database, I can then stand a web front end up on it and make that available to the users. Tada!! Now they can configure to their hearts content without bothering me.
Now, life being what it is, while I converted the mappings to use the database and had promoted it to production several months earlier, I had not set up a web front end yet.
We were testing changes in QA that would allow us to trade financial instruments that we had never traded before. Obviously we needed to test and make sure that the new trade types worked as expected but also to do some regression testing and make sure that these changes hadn't broken existing trades. We were getting some odd results from the new trades in QA so I went investigating. After discounting all the obvious answers, I went back to basics and looked again at them. It turns out that the configuration of the data source that was being used for the table look ups was not providing the right answer because it was pointed at the wrong database. It was still looking at the development environment.
I had a sudden cold chill and after the shivers stopped I dared to look at the configuration in production and saw that yes, indeed, production was pointed to the development environment. I thought I might have a heart attack. Now, kids, this is emphatically NOT anything you ever want to have happen to you. Bear in mind that this industry was heavily regulated by SOX. Any idea what would happen if a SOX auditor got wind of something like that? Phrases like, 'career changing learning experience', spring immediately to mind.
Now, the good news is that all is well that ends well. Everything was fine, we got it all changed without a hiccup and life went on. The part that scares me is what might have happened. I mean, this is development, man. If had taken it into my head at any point during those intervening months to blow away the entire database, much less wipe a table (both of which I can, will and are perfectly valid things to do in development) what kind of damage would have been done? Fortunately, not a lot because recovery would have been simple (thank GOD) but how long would it have taken for us to figure out what happened? It took me a couple of days of poking around to finally decide I should go back to basics and check the data source configuration at the application server level.
My point here is, that mistake made me want to crap my pants. It was amateur hour kind of stuff that I should never have let happen. Even if I wasn't the one actually handling the deployment, I sit over the shoulder of the SA and DBA's for almost every production deployment I make and I should have seen it, thought about it or something! But despite all that that, it was good for me in the end. Like most everyone who has ever done something as boneheaded as that, you can be sure I will be a damned sight more careful with my deployment instructions and double-checking both my and the deployment engineers work. It also reminded me that while things like vendor applications are a fact of life sometimes and you don't have an actual build process for them, there is nothing preventing you from creating a deployment build for it. I might not have to compile code but I can damned sure set up the build server to check out a tokenized version of the any configuration files that have changed and have those tokens replaced with the proper environment settings for whatever environment is being promoted to. Also, you can set up such a build so that it breaks on deployment if you forgot to put in the environments DB password, for example. You really don't store password in your source code repository right? Right??
The only real defenses against mistakes like these are to be disciplined and diligent in your pursuit of the perfect build process to automate these tedious things and/or to document them. I suck at documentation but it would have saved much time and headache if I didn't. Obviously, I was not diligent or disciplined enough in my build process, mostly because until that event happened, I really didn't think of it as a build process. My only defense is that working with vendor products is not something that I have had a lot of experience at. I mean, create and deploy a JBoss server with whatever configuration files it needs but that was always in the context of writing actual code. This puts my brain in a completely different mindset than I was in when I was thinking, how can I configure this vendor application in a more useful fashion.
Ah well, live and learn.
Friday, December 7, 2012
Monday, December 3, 2012
Resurrection of a Classic
I had originally intended to limit my posts to just development related topics but I have found that I don't always feel I have something remotely interesting to say about that even once a week so I am taking a moment to post about something else, namely, my current favorite game.
I loved the original X-COM: UFO Defense published by MicroProse in the mid-nineties. I was in the military at the time and living in the barracks. There were only a couple of guys that had a PC and the games always turned into a group effort within our circle of friends. Four or five of us would gather around the PC, drink beer and at each turn of the game we would swap drivers and someone else would deploy the squad while the others either cheered them on, badger them at what we thought was a poor move or drop our jaws when the completely unexpected would happen.
This game had a LOT going for it. It was detailed, complex, and especially at the later stages of the game, quite time consuming to control each of the 12 team members you could send on a given mission. We played through it any number of times as well as the follow up games that were published. We all gained immense enjoyment out of it and I remembered it fondly for many years.
You can imagine my reaction when I heard that X-COM was getting a reboot over fifteen years later and not only was it available for the PC but for consoles as well. I was overjoyed at the idea that my beloved game was going to be resurrected using technologies that were hard to imagine at the time but I was also extremely worried. I had no idea if the game that I remembered so well would be recognizable to me any longer.
I am happy to say that the fear was misplaced and the game that I remembered, while not exactly what it was, was still, at heart, the same game I fell in love with. They managed to maintain the same feel that I had in the first game while at the same time streamlining the controls, (and though I hate to admit it) the tedium of having to take care of so many details for so many characters on a team.
The transition was very cleverly accomplished. You still have resources, research, engineering, facilities and soldiers that need to be developed and managed, just like the original but the menu structures and controls for handling soldiers in mission have been streamlined and well thought out.
Maybe it's just my imagination but the amount of items that can be developed and created seems to be more limited than the original. For example, I was surprised to discover that there was only one aircraft, a fighter, that could be developed in the game using alien technologies. The original had a number of different aircraft both fighters and troop transports, that could be developed and built. It was disappointed to know that I couldn't develop a superior troop transport and that I would forever be limited to a maximum of six troops on any given mission.
The limitation on the number of troops concerned me because at the later stages, in the original at least, I don't think you could have reasonably completed the game. I was afraid that the game was going to abruptly become either too soft or too difficult in the later stages.
Thankfully, I needn't have worried. In the reboot, the game designers have overcome this lack of firepower in the field by added some features that both simplified the development of soldiers and balanced the gameplay between the large numbers of enemy troops and the limited squad size of your team.
In the original, if you wanted to outfit any given soldier with the proper gear, you had to view that soldiers complete stat and skill list, from the demolition (for rockets) or throwing skills (for grenades) to the stamina stat (how many things the soldier could do on a single turn) to the strength of a given soldier (how much gear can they carry before they just can't move).
The reboot introduces a class system for each solider. After the first promotion they are given a class of heavy, assault, support or sniper. Each class has it's own skill tree and each skill tree has two separate but complimentary branches that you can choose between as they advance. This both simplifies soldier development (I don't have to worry about developing a soldiers strength to carry a larger weapon) and load out (only the heavy can carry a rocket launcher). At the same time, the skill trees give each class distinct advantages that allow each soldier to easily overcome one or two aliens if played to the strengths of its class.
The engineering foundry also contributes to this, allowing engineering research that will provide across the board improvements to equipment or the development of an entirely new branch of technologies.
The fifteen plus years of hardware improvements in consoles (I play on my PS3) are pretty self-explanatory. Obviously the graphics are better but also the way the cinematics are worked into a soldiers (or aliens) actions, as well as the cut scenes are extremely well done and lend flavor to the game without getting in the way.
All in all, I would say that X-COM: Enemy Unknown is a huge win for Firaxis and 2K Games and, thankfully, me.
I loved the original X-COM: UFO Defense published by MicroProse in the mid-nineties. I was in the military at the time and living in the barracks. There were only a couple of guys that had a PC and the games always turned into a group effort within our circle of friends. Four or five of us would gather around the PC, drink beer and at each turn of the game we would swap drivers and someone else would deploy the squad while the others either cheered them on, badger them at what we thought was a poor move or drop our jaws when the completely unexpected would happen.
This game had a LOT going for it. It was detailed, complex, and especially at the later stages of the game, quite time consuming to control each of the 12 team members you could send on a given mission. We played through it any number of times as well as the follow up games that were published. We all gained immense enjoyment out of it and I remembered it fondly for many years.
You can imagine my reaction when I heard that X-COM was getting a reboot over fifteen years later and not only was it available for the PC but for consoles as well. I was overjoyed at the idea that my beloved game was going to be resurrected using technologies that were hard to imagine at the time but I was also extremely worried. I had no idea if the game that I remembered so well would be recognizable to me any longer.
I am happy to say that the fear was misplaced and the game that I remembered, while not exactly what it was, was still, at heart, the same game I fell in love with. They managed to maintain the same feel that I had in the first game while at the same time streamlining the controls, (and though I hate to admit it) the tedium of having to take care of so many details for so many characters on a team.
The transition was very cleverly accomplished. You still have resources, research, engineering, facilities and soldiers that need to be developed and managed, just like the original but the menu structures and controls for handling soldiers in mission have been streamlined and well thought out.
Maybe it's just my imagination but the amount of items that can be developed and created seems to be more limited than the original. For example, I was surprised to discover that there was only one aircraft, a fighter, that could be developed in the game using alien technologies. The original had a number of different aircraft both fighters and troop transports, that could be developed and built. It was disappointed to know that I couldn't develop a superior troop transport and that I would forever be limited to a maximum of six troops on any given mission.
The limitation on the number of troops concerned me because at the later stages, in the original at least, I don't think you could have reasonably completed the game. I was afraid that the game was going to abruptly become either too soft or too difficult in the later stages.
Thankfully, I needn't have worried. In the reboot, the game designers have overcome this lack of firepower in the field by added some features that both simplified the development of soldiers and balanced the gameplay between the large numbers of enemy troops and the limited squad size of your team.
In the original, if you wanted to outfit any given soldier with the proper gear, you had to view that soldiers complete stat and skill list, from the demolition (for rockets) or throwing skills (for grenades) to the stamina stat (how many things the soldier could do on a single turn) to the strength of a given soldier (how much gear can they carry before they just can't move).
The reboot introduces a class system for each solider. After the first promotion they are given a class of heavy, assault, support or sniper. Each class has it's own skill tree and each skill tree has two separate but complimentary branches that you can choose between as they advance. This both simplifies soldier development (I don't have to worry about developing a soldiers strength to carry a larger weapon) and load out (only the heavy can carry a rocket launcher). At the same time, the skill trees give each class distinct advantages that allow each soldier to easily overcome one or two aliens if played to the strengths of its class.
The engineering foundry also contributes to this, allowing engineering research that will provide across the board improvements to equipment or the development of an entirely new branch of technologies.
The fifteen plus years of hardware improvements in consoles (I play on my PS3) are pretty self-explanatory. Obviously the graphics are better but also the way the cinematics are worked into a soldiers (or aliens) actions, as well as the cut scenes are extremely well done and lend flavor to the game without getting in the way.
All in all, I would say that X-COM: Enemy Unknown is a huge win for Firaxis and 2K Games and, thankfully, me.
Wednesday, November 21, 2012
Coursera classes
When I mentioned in my first post that I was doing this as part of a regime to get myself out of the path of steady decline that I feel I've been in professionally for a few years now, I mentioned Coursera. Now, I haven't had a lot to talk about recently but I need to keep working on my skills as a writer, feeble though they might be so I thought I would put in a brief plug for Coursera.
Like many people in our industry, I just sort of stumbled into programming as a career because I found it to be fun and interesting. As such I have no formal schooling at all. I don't have a CS, IS, or.....well, any degree at all, actually, not even an associates. I have found this to be a mixed blessing over the years. I walked into as an information sponge. I would learn anything and everything that caught my attention, and there was always at least fifty-somethings that were catching my attention all the time. I learned a little bit about everything and occasionally a moderate amount about a few things. I had no preconceptions when I approached any type of technology task and I constantly labored under the foolish notion that there was absolutely nothing that I couldn't make a machine do because, after all, I'd been bending machines to my will for years and never found a real-world problem that I couldn't crack.
Some of the code I have written I wouldn't wish on my worst enemy. The code always got the job done but, especially in the early years, it had a tendency to turn into a nightmare after while. Oh, what a fool that mortal was. Which brings me to the downside; I had no idea what I was doing. This meant that all the lessons that people would normally learn from an experienced developer or all the brilliant minds in our industry that publish scads of good advice about how to code well, escaped me.
I would not trade the way I was exposed and grew up around computers but I can't help but wonder how I would be coding today if I had somehow had a more structured approach. Because I had no degree or even real world experience on my first professional coding job, most of my personal time was spent devouring technical books for basic information that these days someone would normally turn to Google for an answer about. Whats the proper syntax for a join if I'm using Oracle, for example.
Missing out on formal schooling also meant, of course, that all the standard things you would learn in school, such as recursion, standard sorting approaches like quicksorts, binary sorts, blah, blah, blah, simply weren't in my vocabulary unless they had somehow been brought to my attention by sheer need or blind luck.
Now, I would like to believe that in my ideal universe, that if I had it to do over again the best choice I could make would be to learn exactly as I did before except I'd have a professional mentor that could have made the resources available to me to learn these more formal concepts. I don't care if that means he simply coded and explained to me in a pair situation, afterwards making me do the coding on the versions that would ship in production or simply said, here are the classes you need to take, go find these college courses.
Well, those days are long behind me and there are no do-overs but there is still something I can do now. And this is where the Coursera plug happens. After I finished the Scala class, which was fantastic by the way, I went back a week or so later and decided that it never hurts to continue learning the fundamentals. I went back and signed up for another seven classes in the upcoming year. Now, these classes, for the most part, are not what would interest most formally educated people and frankly, I will drop one or more courses if I don't feel I'm getting what I need out of them. I'm talking beginner classes here. Things like, Programming Languages and Algorithms Part 1. I wish I had access to these kinds of MOOC's running around when I was a kid. I would love to have taken a bunch of these courses (for FREE) and learned all the things that the kids in college were learning but spending only as much time as I needed in study and able to ask questions of everyone taking the course with me!
I sincerely think that everyone should take a look at what Coursera has to offer. You might be surprised at what you might learn. And if you have no idea what the acronym MOOC stands for I refer you to the following link with an interesting article by Clay Shirky from which I myself learned it (after having followed a tweet from Martin Fowler to the site). Napster, Udacity, and the Academy
Like many people in our industry, I just sort of stumbled into programming as a career because I found it to be fun and interesting. As such I have no formal schooling at all. I don't have a CS, IS, or.....well, any degree at all, actually, not even an associates. I have found this to be a mixed blessing over the years. I walked into as an information sponge. I would learn anything and everything that caught my attention, and there was always at least fifty-somethings that were catching my attention all the time. I learned a little bit about everything and occasionally a moderate amount about a few things. I had no preconceptions when I approached any type of technology task and I constantly labored under the foolish notion that there was absolutely nothing that I couldn't make a machine do because, after all, I'd been bending machines to my will for years and never found a real-world problem that I couldn't crack.
Some of the code I have written I wouldn't wish on my worst enemy. The code always got the job done but, especially in the early years, it had a tendency to turn into a nightmare after while. Oh, what a fool that mortal was. Which brings me to the downside; I had no idea what I was doing. This meant that all the lessons that people would normally learn from an experienced developer or all the brilliant minds in our industry that publish scads of good advice about how to code well, escaped me.
I would not trade the way I was exposed and grew up around computers but I can't help but wonder how I would be coding today if I had somehow had a more structured approach. Because I had no degree or even real world experience on my first professional coding job, most of my personal time was spent devouring technical books for basic information that these days someone would normally turn to Google for an answer about. Whats the proper syntax for a join if I'm using Oracle, for example.
Missing out on formal schooling also meant, of course, that all the standard things you would learn in school, such as recursion, standard sorting approaches like quicksorts, binary sorts, blah, blah, blah, simply weren't in my vocabulary unless they had somehow been brought to my attention by sheer need or blind luck.
Now, I would like to believe that in my ideal universe, that if I had it to do over again the best choice I could make would be to learn exactly as I did before except I'd have a professional mentor that could have made the resources available to me to learn these more formal concepts. I don't care if that means he simply coded and explained to me in a pair situation, afterwards making me do the coding on the versions that would ship in production or simply said, here are the classes you need to take, go find these college courses.
Well, those days are long behind me and there are no do-overs but there is still something I can do now. And this is where the Coursera plug happens. After I finished the Scala class, which was fantastic by the way, I went back a week or so later and decided that it never hurts to continue learning the fundamentals. I went back and signed up for another seven classes in the upcoming year. Now, these classes, for the most part, are not what would interest most formally educated people and frankly, I will drop one or more courses if I don't feel I'm getting what I need out of them. I'm talking beginner classes here. Things like, Programming Languages and Algorithms Part 1. I wish I had access to these kinds of MOOC's running around when I was a kid. I would love to have taken a bunch of these courses (for FREE) and learned all the things that the kids in college were learning but spending only as much time as I needed in study and able to ask questions of everyone taking the course with me!
I sincerely think that everyone should take a look at what Coursera has to offer. You might be surprised at what you might learn. And if you have no idea what the acronym MOOC stands for I refer you to the following link with an interesting article by Clay Shirky from which I myself learned it (after having followed a tweet from Martin Fowler to the site). Napster, Udacity, and the Academy
Friday, November 16, 2012
Boring Stuff Matters. A Lot.
Developers in general, but especially developers that are working in relative isolation, don't like to do the boring things. Let me clarify a couple of things about that statement.
By isolation, I don't mean that you are locked in a box like a shut-in and never see or communicate with anyone else, I mean you're involved in a project where you are the only, or one of the only, IT participants.
By boring, I don't mean that the code is unexciting, old or simplistic. I mean the stuff that is inherently related to the code you're working on but that a lot of developers still don't really consider to be code related; for example, user acceptance tests, test plans, user documentation, returning that loaner laptop you requested because you had a consultant on-site for specialized assistance, all the things that you have to do that you you keep putting off or just plain forgetting about because secretly you give a mental groan and say to yourself, Crap, do I really have to do that? Can it at least wait?
Now, your experiences and opinions may be different, obviously. You may live to do these things because you love them or find great satisfaction and make it a point of pride to do them. If so, more power to you. You have a firm gut-level grasp of something that I think most of us don't have and I truly, honestly, want to be like you when I grow up.
Things like I describe above have one thing in common. To some extent or other they seem to live in a murky, no-mans land of it's an administrative task but not really. I'm sure you can think of a lot more examples than I gave off the top of your head. You might throw in dealing with a ticketing systems for things like change control, or whatever. The point is, these tasks are almost universally despised and the majority of people would foist them off on someone else if they had a chance.
They also have one other thing in common. Developers that don't appreciate their value and dread doing them do so because at some level they disdain them as pointless, or at the very least low value, tasks. They are things that take time away from your primary job duty of creating awesome code that everyone ooh and ahhs over. I mean, seriously, who actually wants to write a test plan, or put together a set of UAT's because the customer, whoever that may be, doesn't have a clue how to do it. They may honestly think that it really isn't their job to write UATs or worse yet, thinks its beneath them to do it, they are the customer or they're too busy actually making the company money instead of just being a necessary evil and sucking money away from the company.
However, I submit to you that these things are not annoying, ancillary tasks that any monkey could do and aren't really part of development. They are not administrative. They aren't even a necessary evil to be tolerated and grudgingly taken care of, like smelly laundry or the cat's litter box. They are just as much a part of development as actually writing code.
My personal feelings are that these types of tasks, by and large, tend to slip through the cracks in inverse proportion to the size of the team working on a project. The larger the team, the less often they tend to get ignored or slip through the cracks but the smaller the team, the more often they get ignored, put off or just not brought up at all.
I think there are several contributing factors to this;
Now, looking at the first reason, I might be lucky if you didn't suggest naming the Blog Captain Obvious Q&A Time. I mean, no shit, the fewer qualified people to reasonably handle the same number of tasks increases the overall workload. But it's also so obvious that if I left it out, people would be letting me know what an idiot I was for not mentioning it. Also, given the nature of the tasks we are talking about, it's a reasonable answer. In the examples above, why couldn't some of the totally IT clueless people take care of at least some of them? I mean, do you really have to be IT Savvy to return a freaking laptop??
Of course not, but that's where the reasonable qualification comes in. When that consultant comes in to do his specialized task, who is he going to come to in order to get set up with a network login, laptop or whatever else he needs to do his job for the next week or two? Even if he's working at the elbow of the end user he will probably not be asking them to set him up and even if he did, what are the odds that the end user will take care of it himself rather than sending him to the only IT related team member?
Frankly, you probably want to take care of it yourself so that you know for fact he is ready to go, has all the proper access rights and whatever else he needs. Why, you ask? Because when he (or she) leaves, most organizations have much greater odds of failing to properly shutdown/delete everything that was opened / created for the employee coming in. The end user will almost certainly simply turn in the laptop and think it's over. You will probably be inclined to double-check that all of the ancillary tasks are taken care of; accounts suspended or removed, software licenses that may have been temporarily issued are return to the pool, etc. Especially since you were probably the one who had to request all that stuff in the first place! And lastly on this reason, remember the saying, 'If you pack it in, you pack it out'. You probably set it up in the first place and are probably in the best position to make sure that it's torn down properly. If nothing else, it's just good manners.
The second reason is again pretty obvious. The smaller a team the less need there is to have a formal process, especially if everyone is local. Communication is a snap, just yell over the cube, have an informal stand up while sharing a burger at lunch, throw a paper airplane into the cube with a note on it or put a sticky note on their forehead while they pretend to work from their corner cube. These techniques tend to become less effective the larger and more physically dispersed the team becomes. This is known fact. Also known is that humans are lazy creatures. I am not a representative example, I am far lazier than most humans, even developers, a poster child for the deadly sin of sloth, to be honest. However, most humans will give in to their level of laziness if given a chance. The smaller the group, the more likely it is that one member slacking off will affect the rest of the team. One of the things that makes pair programming and good code reviews effective is that you have lots of eyes on your code. The more eyes the more likely the code is to be good, or at least be improved significantly from where it started. Same thing goes for the little things. If I know that no one is going to look at my test plan until we are ready to run UATs on the project, the more likely I am to treat a test plan as some sort of third class citizen. This usually means it gets ignored as long as it possibly can and then crammed one together at the last minute, hoping that some crucial test didn't get missed.
The third one is all me. I have no evidence or numbers but I have been involved in a lot of small development teams. Quite frequently I was the only team member that was actually in the IT department, so obviously, this one is extremely biased. However, it does seem to me that the smaller the team, the more they get jerked around. Let's face it. We all know that typically there is strength in numbers, even for developers. This is why a lot of herbivores herd, right? Larger teams look more intimidating to end users than small ones. These predators will hesitate to bring a last minute change or tell the development team that they should be the ones writing the UAT cases if they have to face a dozen developers to do it. In most companies, politically speaking certainly, smaller teams almost certainly have no clout. Saying no becomes significantly more difficult. This is not the case in the ideal world, obviously, but we don't live in the ideal world, we live in the real world and the real world just isn't that happy. There are exceptions, absolutely, but they are exceptions, not the rule.
OK, so now that I've said all that stuff, what's my point? My point is that all these annoying, trivial, and small (or not so small, sometimes) issues that you normally don't think about become even more important in small teams. Yes, it sucks, yes, you may have to force yourself to be more disciplined than you have before and sacrifice always hurts in some way. So why do it? Let's look at what you gain.
First, it's just professional and I'm trying to become better at my profession. If you want to be good at your craft, you should be good, or at least improving, in as many aspects as possible, not just the parts you like. That's what makes it a craft. If it's just the parts you like, you're a hobbyist, even if you're getting paid for it. Don't take my word for it, go read some books from some of the actual smart people in the industry that give advice on how to become a better developer, like Martin Fowler, Uncle Bob Martin, Kent Beck, Erich Gamma, Donald Knuth, and gobs of others. They focus on a lot more than just writing code.
Second, people notice. Maybe not all the time and maybe not at first but your end users truly just don't give a damn how cool your code is (we all know this) because they can't relate to it easily. If they could, they would probably be in IT instead of running a sales team or travel agents or a risk management group or whatever they do. BUT, they can and eventually will relate to the kinds of things I've been talking about. I mean, I can go crazy fancy with putting together a UAT or use case or test plan or documentation but at the end of the day, I can also be as lazy as possible while doing those things. I don't need some huge, comprehensive document that could be used as a doorstop. I could do all those things with a simple spreadsheet. And that, my friends, they will understand. They can relate to that since they almost certainly use spreadsheets on a daily basis. If you consistently deliver things like this when they are needed, they will figure out that while you do some mystical crap they don't get, you're still one of the crew, a professional, even if you do act or look a little different.
This has so many beneficial side effects that I doubt I could think of them all. Anytime you have a chance to raise the awareness that developers are professionals too, take it! It's a huge win for everyone, not just you, or even your IT department in general but for that business unit. Because it opens that business unit up to the possibility of closer and better collaboration. And trust me, if you don't immediately see the benefits of that, let me know where you work because it must be a fantastic place!
Also, as an aside, you're also building up the political clout with that client to be able to say no and actually have them listen. If they respect you as a professional, they will start to listen to your opinion and why you're telling them no.
If you've made it this far, I would like to encourage you to post questions, contradictions, ideas for questions or whatever.
By isolation, I don't mean that you are locked in a box like a shut-in and never see or communicate with anyone else, I mean you're involved in a project where you are the only, or one of the only, IT participants.
By boring, I don't mean that the code is unexciting, old or simplistic. I mean the stuff that is inherently related to the code you're working on but that a lot of developers still don't really consider to be code related; for example, user acceptance tests, test plans, user documentation, returning that loaner laptop you requested because you had a consultant on-site for specialized assistance, all the things that you have to do that you you keep putting off or just plain forgetting about because secretly you give a mental groan and say to yourself, Crap, do I really have to do that? Can it at least wait?
Now, your experiences and opinions may be different, obviously. You may live to do these things because you love them or find great satisfaction and make it a point of pride to do them. If so, more power to you. You have a firm gut-level grasp of something that I think most of us don't have and I truly, honestly, want to be like you when I grow up.
Things like I describe above have one thing in common. To some extent or other they seem to live in a murky, no-mans land of it's an administrative task but not really. I'm sure you can think of a lot more examples than I gave off the top of your head. You might throw in dealing with a ticketing systems for things like change control, or whatever. The point is, these tasks are almost universally despised and the majority of people would foist them off on someone else if they had a chance.
They also have one other thing in common. Developers that don't appreciate their value and dread doing them do so because at some level they disdain them as pointless, or at the very least low value, tasks. They are things that take time away from your primary job duty of creating awesome code that everyone ooh and ahhs over. I mean, seriously, who actually wants to write a test plan, or put together a set of UAT's because the customer, whoever that may be, doesn't have a clue how to do it. They may honestly think that it really isn't their job to write UATs or worse yet, thinks its beneath them to do it, they are the customer or they're too busy actually making the company money instead of just being a necessary evil and sucking money away from the company.
However, I submit to you that these things are not annoying, ancillary tasks that any monkey could do and aren't really part of development. They are not administrative. They aren't even a necessary evil to be tolerated and grudgingly taken care of, like smelly laundry or the cat's litter box. They are just as much a part of development as actually writing code.
My personal feelings are that these types of tasks, by and large, tend to slip through the cracks in inverse proportion to the size of the team working on a project. The larger the team, the less often they tend to get ignored or slip through the cracks but the smaller the team, the more often they get ignored, put off or just not brought up at all.
I think there are several contributing factors to this;
- The fewer the team members who are capable of reasonably doing these tasks, the higher the workload on the team members who are
- The smaller the team, the less disciplined the team process is
- The smaller the team, the easier they are to jerk around
Now, looking at the first reason, I might be lucky if you didn't suggest naming the Blog Captain Obvious Q&A Time. I mean, no shit, the fewer qualified people to reasonably handle the same number of tasks increases the overall workload. But it's also so obvious that if I left it out, people would be letting me know what an idiot I was for not mentioning it. Also, given the nature of the tasks we are talking about, it's a reasonable answer. In the examples above, why couldn't some of the totally IT clueless people take care of at least some of them? I mean, do you really have to be IT Savvy to return a freaking laptop??
Of course not, but that's where the reasonable qualification comes in. When that consultant comes in to do his specialized task, who is he going to come to in order to get set up with a network login, laptop or whatever else he needs to do his job for the next week or two? Even if he's working at the elbow of the end user he will probably not be asking them to set him up and even if he did, what are the odds that the end user will take care of it himself rather than sending him to the only IT related team member?
Frankly, you probably want to take care of it yourself so that you know for fact he is ready to go, has all the proper access rights and whatever else he needs. Why, you ask? Because when he (or she) leaves, most organizations have much greater odds of failing to properly shutdown/delete everything that was opened / created for the employee coming in. The end user will almost certainly simply turn in the laptop and think it's over. You will probably be inclined to double-check that all of the ancillary tasks are taken care of; accounts suspended or removed, software licenses that may have been temporarily issued are return to the pool, etc. Especially since you were probably the one who had to request all that stuff in the first place! And lastly on this reason, remember the saying, 'If you pack it in, you pack it out'. You probably set it up in the first place and are probably in the best position to make sure that it's torn down properly. If nothing else, it's just good manners.
The second reason is again pretty obvious. The smaller a team the less need there is to have a formal process, especially if everyone is local. Communication is a snap, just yell over the cube, have an informal stand up while sharing a burger at lunch, throw a paper airplane into the cube with a note on it or put a sticky note on their forehead while they pretend to work from their corner cube. These techniques tend to become less effective the larger and more physically dispersed the team becomes. This is known fact. Also known is that humans are lazy creatures. I am not a representative example, I am far lazier than most humans, even developers, a poster child for the deadly sin of sloth, to be honest. However, most humans will give in to their level of laziness if given a chance. The smaller the group, the more likely it is that one member slacking off will affect the rest of the team. One of the things that makes pair programming and good code reviews effective is that you have lots of eyes on your code. The more eyes the more likely the code is to be good, or at least be improved significantly from where it started. Same thing goes for the little things. If I know that no one is going to look at my test plan until we are ready to run UATs on the project, the more likely I am to treat a test plan as some sort of third class citizen. This usually means it gets ignored as long as it possibly can and then crammed one together at the last minute, hoping that some crucial test didn't get missed.
The third one is all me. I have no evidence or numbers but I have been involved in a lot of small development teams. Quite frequently I was the only team member that was actually in the IT department, so obviously, this one is extremely biased. However, it does seem to me that the smaller the team, the more they get jerked around. Let's face it. We all know that typically there is strength in numbers, even for developers. This is why a lot of herbivores herd, right? Larger teams look more intimidating to end users than small ones. These predators will hesitate to bring a last minute change or tell the development team that they should be the ones writing the UAT cases if they have to face a dozen developers to do it. In most companies, politically speaking certainly, smaller teams almost certainly have no clout. Saying no becomes significantly more difficult. This is not the case in the ideal world, obviously, but we don't live in the ideal world, we live in the real world and the real world just isn't that happy. There are exceptions, absolutely, but they are exceptions, not the rule.
OK, so now that I've said all that stuff, what's my point? My point is that all these annoying, trivial, and small (or not so small, sometimes) issues that you normally don't think about become even more important in small teams. Yes, it sucks, yes, you may have to force yourself to be more disciplined than you have before and sacrifice always hurts in some way. So why do it? Let's look at what you gain.
First, it's just professional and I'm trying to become better at my profession. If you want to be good at your craft, you should be good, or at least improving, in as many aspects as possible, not just the parts you like. That's what makes it a craft. If it's just the parts you like, you're a hobbyist, even if you're getting paid for it. Don't take my word for it, go read some books from some of the actual smart people in the industry that give advice on how to become a better developer, like Martin Fowler, Uncle Bob Martin, Kent Beck, Erich Gamma, Donald Knuth, and gobs of others. They focus on a lot more than just writing code.
Second, people notice. Maybe not all the time and maybe not at first but your end users truly just don't give a damn how cool your code is (we all know this) because they can't relate to it easily. If they could, they would probably be in IT instead of running a sales team or travel agents or a risk management group or whatever they do. BUT, they can and eventually will relate to the kinds of things I've been talking about. I mean, I can go crazy fancy with putting together a UAT or use case or test plan or documentation but at the end of the day, I can also be as lazy as possible while doing those things. I don't need some huge, comprehensive document that could be used as a doorstop. I could do all those things with a simple spreadsheet. And that, my friends, they will understand. They can relate to that since they almost certainly use spreadsheets on a daily basis. If you consistently deliver things like this when they are needed, they will figure out that while you do some mystical crap they don't get, you're still one of the crew, a professional, even if you do act or look a little different.
This has so many beneficial side effects that I doubt I could think of them all. Anytime you have a chance to raise the awareness that developers are professionals too, take it! It's a huge win for everyone, not just you, or even your IT department in general but for that business unit. Because it opens that business unit up to the possibility of closer and better collaboration. And trust me, if you don't immediately see the benefits of that, let me know where you work because it must be a fantastic place!
Also, as an aside, you're also building up the political clout with that client to be able to say no and actually have them listen. If they respect you as a professional, they will start to listen to your opinion and why you're telling them no.
If you've made it this far, I would like to encourage you to post questions, contradictions, ideas for questions or whatever.
Wednesday, November 14, 2012
Full Throttle Passion
So this is how it all starts. Over the past handful of years I have felt like my career has been in a slump. Sure, I have jobs, I write code, I have done all the things that you do when you have a professional career as a software developer. The problem was, my heart just didn't seem to be in it.
No matter how much I told myself, "Get excited, this is a fun job! You're working with cool new frameworks and toys!", the shine wore off faster and faster. I eventually switched to the opposite side of the coin, telling myself, "Hey, look, it's not that bad. Just do your job and you can go home. This isn't your life, it's just a job". It took even less time for that to start failing as a motivation technique.
I have recently discovered a bit of information that I never realized, not about people in general, developers as a group or certainly not myself. Uncle Bob, Martin Fowler, Jeff Atwood, everyone that has ever written about software development and craftsmanship from the viewpoint of a developer looking past the code, are right.
Developers, by and large, cannot thrive in a vacuum (along with most other forms of complex life). We, or at least I, honestly cannot grow as a developer without at least some minimal interaction with other humans who are truly interested in code. It can be done for a while, sometimes a very long while, but eventually, I think it gets to you, you just flame out. I can't formulate it into words at the moment, but that's how I feel.
Clearly, none of what I just said is new information. Every developer who has ever read a book on development, or hell, even taken a computer related class and studied with other people, is familiar with this concept. I mean, this really is a, "No shit, Sherlock", kind of moment. Of course, I knew this concept. I have read a lot of those books, the blogs, hell, even knowing the names of the authors you almost have to have heard these thoughts aired out before. But here's the thing; It never made it to my gut.
I have been working as a professional developer for almost 16 years. And by professional, I mean, someone was foolish enough to pay me for it, although by that definition I started developing almost 28 years ago when I was a junior in high school and the only travel agency in town asked me to do some report creation for them once a month. But the point is, people have been paying me to write code for a while now.
I have had significant ups and downs in my career. I have worked for multiple startups (most of which failed, but those were not my fault, I swear), large ISPs manufacturing companies, and game companies, I have worked in positions that were not, in my opinion, software development (no matter what the title or job description said) but were related; QA roles and BA roles. There is nothing wrong with these roles, they can be indispensable, but in neither of those was my primary duty to actually sit and create code. I have had moments when I felt that I really didn't belong in the industry writing code, that every single developer out there, had more skill and plain raw talent than I did. There were times where I did truly shameful things that damaged both myself and my career.
I have also, on a few occasions, been the hero, saving the day with an idea that no one had brought to the table or had heard of before.
OK, I know, that was a bit of a ramble. Those previous few paragraphs were all written to say this; I have been doing this for a while now, have suffered my ups and downs, crises of faith, but it hasn't been until very recently that my gut simply has accepted that My most successful and satisfying moments have been the times during which I spent significant time around fellow coders who were also passionate about code.
Now, that last part is worth reading again.I didn't qualify that with work, hobby, personally, or anything else. It didn't matter if I was in junior high learning to program a TRS-80 after school at the office of a friends parent, trying to program wire framed perspective graphics on a Commodore 64 in high school with a mentor that was trying to teach me how vector math could be used to draw shapes in perspective or creating one of the first scaled services allowing a customer to make changes to their ISP package without having to call in to customer support (things like adding email addresses, provisioning websites, etc.) I had people that I spent time with that were passionate about code. We all have things we are passionate about aside from code but they all had that passion for code in common.
Now to swing this back around to my opening statement that, 'my heart just wasn't in it'. How did I not know this? I just didn't. I lost track. I had moved on, changing jobs, changing towns, and lets face, I'm really not a people person anyway. Pick any excuse you want but the bottom line is I lost touch with any kind of community of developers that were passionate about code and lost my passion in the process.
The good news is, that can change. And I WANT that to change. I miss that old me and I want to get it back. Not to mention, when I speak to peers that I used to mentor and realize how much further ahead of me they are it provides a stark and unforgiving mirror of how much I have missed over the years. OK, so that sounds like a childish and selfish thought. OK, I can understand that. I'm also OK with that. As long as it motivates me, and it does, without making me bitter or angry, which (it mostly) doesn't, I can live with that answer.
To catch up, I have to go full throttle. I don't have time to waste taking 10 years to get to where I should be, where I want to be. That means, taking into account all the advice that I've read, agreed with and nodded sagely at but for some reason, just didn't apply it to me.
So, step one, make a decision, check. Step two, act on it in as many ways possible as often as possible. This will take discipline and I will be forced to change years of habit but that's the goal. Communication is key in learning and certainly in learning at an accelerated rate. Hence, taking the advice of Jeff Atwood, this blog. It doesn't matter that it sucks and no one will read it, or even worse that they read it and blow me off as some sort of self-help slob. It matters that I do it. I've recently finished an online class at Coursera presented by Martin Odersky on functional programming principles using Scala. The course was fantastic. I signed up for 6 more courses a couple of days ago it got me so excited and I recommend that you run, not walk, to check it out for yourself.
It also forced me to interact with other students, even if it was just through the class forum and participating in the Q&A session on assignments. I have started reading more and more blogs. I've been using StackOverflow since it was created 5 years ago but never participated. Now I've created an account and will start giving back as best I can. I have a ton of ideas that I have planned or have already begun to implement. And this entire turn around was inspired by one, single, thing. Other people passionate about code. Same room, like or don't like them as a person, as long as you participate in the conversation about code with someone else that is passionate about it.
This isn't about my career or my job, it's about me enjoying my life again and not being trapped in a rut feeling like a gerbil on a wheel. So, please, share your thoughts, your ideas, whatever it is you're passionate about, with me. I am looking for good feedback to help me grow personally as well as professionally and if I fail, well, shit happens. On the plus side, if I'm going full throttle at least the impact will be sudden, swift and painless!
So let's hear it for Full Throttle Passion! Damn the torpedo's! Safety is NOT guaranteed.
No matter how much I told myself, "Get excited, this is a fun job! You're working with cool new frameworks and toys!", the shine wore off faster and faster. I eventually switched to the opposite side of the coin, telling myself, "Hey, look, it's not that bad. Just do your job and you can go home. This isn't your life, it's just a job". It took even less time for that to start failing as a motivation technique.
I have recently discovered a bit of information that I never realized, not about people in general, developers as a group or certainly not myself. Uncle Bob, Martin Fowler, Jeff Atwood, everyone that has ever written about software development and craftsmanship from the viewpoint of a developer looking past the code, are right.
Developers, by and large, cannot thrive in a vacuum (along with most other forms of complex life). We, or at least I, honestly cannot grow as a developer without at least some minimal interaction with other humans who are truly interested in code. It can be done for a while, sometimes a very long while, but eventually, I think it gets to you, you just flame out. I can't formulate it into words at the moment, but that's how I feel.
Clearly, none of what I just said is new information. Every developer who has ever read a book on development, or hell, even taken a computer related class and studied with other people, is familiar with this concept. I mean, this really is a, "No shit, Sherlock", kind of moment. Of course, I knew this concept. I have read a lot of those books, the blogs, hell, even knowing the names of the authors you almost have to have heard these thoughts aired out before. But here's the thing; It never made it to my gut.
I have been working as a professional developer for almost 16 years. And by professional, I mean, someone was foolish enough to pay me for it, although by that definition I started developing almost 28 years ago when I was a junior in high school and the only travel agency in town asked me to do some report creation for them once a month. But the point is, people have been paying me to write code for a while now.
I have had significant ups and downs in my career. I have worked for multiple startups (most of which failed, but those were not my fault, I swear), large ISPs manufacturing companies, and game companies, I have worked in positions that were not, in my opinion, software development (no matter what the title or job description said) but were related; QA roles and BA roles. There is nothing wrong with these roles, they can be indispensable, but in neither of those was my primary duty to actually sit and create code. I have had moments when I felt that I really didn't belong in the industry writing code, that every single developer out there, had more skill and plain raw talent than I did. There were times where I did truly shameful things that damaged both myself and my career.
I have also, on a few occasions, been the hero, saving the day with an idea that no one had brought to the table or had heard of before.
OK, I know, that was a bit of a ramble. Those previous few paragraphs were all written to say this; I have been doing this for a while now, have suffered my ups and downs, crises of faith, but it hasn't been until very recently that my gut simply has accepted that My most successful and satisfying moments have been the times during which I spent significant time around fellow coders who were also passionate about code.
Now, that last part is worth reading again.I didn't qualify that with work, hobby, personally, or anything else. It didn't matter if I was in junior high learning to program a TRS-80 after school at the office of a friends parent, trying to program wire framed perspective graphics on a Commodore 64 in high school with a mentor that was trying to teach me how vector math could be used to draw shapes in perspective or creating one of the first scaled services allowing a customer to make changes to their ISP package without having to call in to customer support (things like adding email addresses, provisioning websites, etc.) I had people that I spent time with that were passionate about code. We all have things we are passionate about aside from code but they all had that passion for code in common.
Now to swing this back around to my opening statement that, 'my heart just wasn't in it'. How did I not know this? I just didn't. I lost track. I had moved on, changing jobs, changing towns, and lets face, I'm really not a people person anyway. Pick any excuse you want but the bottom line is I lost touch with any kind of community of developers that were passionate about code and lost my passion in the process.
The good news is, that can change. And I WANT that to change. I miss that old me and I want to get it back. Not to mention, when I speak to peers that I used to mentor and realize how much further ahead of me they are it provides a stark and unforgiving mirror of how much I have missed over the years. OK, so that sounds like a childish and selfish thought. OK, I can understand that. I'm also OK with that. As long as it motivates me, and it does, without making me bitter or angry, which (it mostly) doesn't, I can live with that answer.
To catch up, I have to go full throttle. I don't have time to waste taking 10 years to get to where I should be, where I want to be. That means, taking into account all the advice that I've read, agreed with and nodded sagely at but for some reason, just didn't apply it to me.
So, step one, make a decision, check. Step two, act on it in as many ways possible as often as possible. This will take discipline and I will be forced to change years of habit but that's the goal. Communication is key in learning and certainly in learning at an accelerated rate. Hence, taking the advice of Jeff Atwood, this blog. It doesn't matter that it sucks and no one will read it, or even worse that they read it and blow me off as some sort of self-help slob. It matters that I do it. I've recently finished an online class at Coursera presented by Martin Odersky on functional programming principles using Scala. The course was fantastic. I signed up for 6 more courses a couple of days ago it got me so excited and I recommend that you run, not walk, to check it out for yourself.
It also forced me to interact with other students, even if it was just through the class forum and participating in the Q&A session on assignments. I have started reading more and more blogs. I've been using StackOverflow since it was created 5 years ago but never participated. Now I've created an account and will start giving back as best I can. I have a ton of ideas that I have planned or have already begun to implement. And this entire turn around was inspired by one, single, thing. Other people passionate about code. Same room, like or don't like them as a person, as long as you participate in the conversation about code with someone else that is passionate about it.
This isn't about my career or my job, it's about me enjoying my life again and not being trapped in a rut feeling like a gerbil on a wheel. So, please, share your thoughts, your ideas, whatever it is you're passionate about, with me. I am looking for good feedback to help me grow personally as well as professionally and if I fail, well, shit happens. On the plus side, if I'm going full throttle at least the impact will be sudden, swift and painless!
So let's hear it for Full Throttle Passion! Damn the torpedo's! Safety is NOT guaranteed.
Subscribe to:
Posts (Atom)