QA Help Wanted
MySQL Sun Oracle needs QA help. How do I know this? Because I’ve used their products.
We use MySQL 5.1.x “Enterprise” on several servers at work. We have several production servers that are replicated to slave servers as hot spares, reporting servers, and to a disaster recovery server. As part of our setup one of our backup servers needs to replicate from two different servers and databases to two different local databases. So, I need to run two instances of MySQL. There are three basic ways to accomplish this: run two instances of MySQL directing each to a different configuration file with appropriate port, directory, pid file settings; use mysqladmin to manage multiple instances; use mysqld_multi to start multiple instances. Mysqladmin has been deprecated and will be removed in 5.4, so I’m not investing energy in that solution. Option one will work, it just means more files to manage. Mysqld_multi allows using a single configuration file with definitions for each server instance. It’s straightforward and pretty easy to set up.
Mysqld_multi is a Perl script that parses a single config file containing multiple server instance definitions. There is just one problem. It doesn’t work. It throws the following error:
Unmatched right curly bracket at /usr/bin/mysqld_multi line 171, at end of line syntax error at /usr/bin/mysqld_multi line 171, near "}" Execution of /usr/bin/mysqld_multi aborted due to compilation errors.
As a programmer, and sysadmin, I expect that an error like that must be the result of some combination of version incompatabilities or poor configuration. Since I had just upgraded MySQL to the latest version this made sense. I updated all the packages on the server, rebooted, tried again.
Unmatched right curly bracket at /usr/bin/mysqld_multi line 171, at end of line syntax error at /usr/bin/mysqld_multi line 171, near "}" Execution of /usr/bin/mysqld_multi aborted due to compilation errors.
After way too much effort I pulled the script into a language aware editor and found that the script did, in fact, have an unmatched right curly bracket. I removed it. I ran it again, thinking that this couldn’t possibly be the problem. A large technology company like Sun Oracle wouldn’t let a product out the door with this kind of fundimental bug. They must have run this script at some time and found this error. No, they didn’t. Once the errant bracket was removed, the script ran. Perfectly.
Coincidentally, I just received our quote from Sun Oracle for “enterprise” support for MySQL. Seems kinda high.
Linux Loses My Desktop
Okay, I give up.
As i’ve written before, I’ve run Linux (SUSE) on my personal desktop and home servers for several years prior to moving to the Mac. I still run Linux for my home server. The only Windows computer I have running at home is my kid’s computer. They have games that won’t run elsewhere.
I installed Ubuntu 8.10 on my office desktop in a dual boot configuration. It handled my dual monitors, quad core CPU, 4Gb of RAM without a hitch. But what it couldn’t handle was working in my office. Now, I had no problem administering our dozen or so Linux servers, reading/writing Word, Excel, Powerpoint documents. I had no problems dealing with our various Java applications, or accessing data on our office network. But, two things have made a permanent conversion difficult, at best. Number one is Exchange/Outlook. I wrote about this earlier. Interacting with Exchange via Linux is painful if you use Exchange calendaring, contacts, or anything other than mail. There is no good way to interact with Exchange calendaring in Linux. Number two is somewhat surprising. As part of a server upgrade project we have adopted Citrix XenServer to handle server virtualization. This is based on the Linux-based Xen project. You would expect it to be very cross platform. You would be wrong. The management tools are Windows only. So, it uses a Windows only management console to manage Linux based hypervisors with Linux guest virtual machines.
Interestingly, both of these observations have been commented on in the last few days:
Where are the Enterprise Management Tools for Linux on the Desktop? by David Lane
What would you exchange Exchange for? by Doc Searls
So, I’m not alone.
I think as desktop virtualization matures, it will be easier to run the desktop of your choice. I use VMWare Fusion on my Mac to run Linux, Windows, and even Haiku. The downside for Linux is there will be less of an incentive to develop the replacements for Outlook and the enterprise management tools. There will also be fewer requests for those applications from corporate users. Why wait for the open source equivalent when I can run the Windows version in a virtual machine.
In the meantime, it’s back to XP. I’ve got work to do.
Exchange is the Key to the Enterprise
As an IT manager, I have to make various decisions that affect the operation of my employer. One decision I was able to make was what OS I would run on my desktop. We run a number of Linux servers, most of our desktops (except for the graphic artist’s Mac) are running Windows XP. Personally, I have a MacBook Pro, and I run various Linux distributions via VMWare Fusion. The workstation I inherited is quite nice, 4GB RAM, 250GB RAID 1 15K SAS, Quad core 2.4Ghz processor, dual LCD monitors. Way too much PC for a manager. We, like many (most?) enterprises rely on Microsoft Exchange and Outlook for our email and calendaring.
I am not a fan of Windows. For the last several years, the only time I used Windows was at work. Until now, I didn’t have the authority to choose otherwise. I decided about a month ago to switch my OS to Linux. Of course that means choosing a distribution. I used SUSE for several years, and was happy with it. So, I decided to load openSUSE 11.0 on my desktop. It was a painless install. openSUSE was considerate enough to make room for itself and leave room for Windows and set up dual boot. Very nice.
Time to boot up. Ok, I have two identical desktops running mirrored on both monitors. I find the desktop/monitor settings and set it accordingly. And it doesn’t work. No matter what I tried, I couldn’t get openSUSE to correctly span my desktop across both monitors. It insisted on mirroring on both. That sucks. I got tired of screwing with openSUSE, and fired up the Ubuntu 8.10 install. I did have to intervene so it used the existing partition setup, but that was not a big deal. The install proceeded normally. I booted up and was met with a desktop that correctly spanned both monitors without intervention from me. Perfect. Ubuntu wins.
I have used OpenOffice (and NeoOffice) for several years. I have never had a situation where it couldn’t do what I needed, and interoperate with MS Office. But, I need to interoperate with Exchange. So, I installed Evolution and set it up to work with our Exchange hosting provider.
Evolution seemed like a good solution. It has many of Outlooks features, it sets up quickly. But, it stinks. It’s slow syncing mail. It would frequently freeze when trying switch to the calendar while mail was syncing. And the calendaring synchronization is terrible. It’s slow, it’s unreliable, and it only works with Outlook Web Access.
So, I loaded Thunderbird for email (using IMAP) and I update my calendar by hand.
Every year for the past 6 or 7 has been declared the year of the Linux desktop. The year it takes over the office. It’s not going to happen. Not until the Linux community recognizes and addresses the real obstacles to widespread adoption, MS Office and Exchange. OpenOffice addresses the main components of MS Offices (except Access, but there are substitutes for it). And OO works well. But, Exchange is entrenched in most enterprises, and it won’t be dislodged easily. Sure, there are alternatives for Exchange, IMAP/POP3 servers, iCal servers, etc. There is NO good way to inter-operate with an existing Exchange environment. And without one, Linux cannot make significant inroads in existing enterprises.
In the meantime, I am loading Windows XP as a VirtualBox VM so I can run Outlook.
The Myth of the Collaborative Cube Farm, Redux
Not everyone agrees with my thesis. But, it is still an active topic of conversation. It’s clear that most companies give the subject little thought. Here are a few links to related articles around the internets:
Out of the box: Valley companies dump cubicles for open office spaces
Geek to Live: Firewall your attention at the office
Why Proximity Kills Productivity
Collaboration Cancels the Cubicle Culture
Us vs. Them
I am a software developer (or programmer/analyst, if you wish). Over my 20 year career i’ve flirted with management positions a number of times. I’ve been a project lead developer (as I am now), i’ve been a team lead with several developers working for me, i’ve even been an IT manager with a budget and employees (in an IT department of 5). But, mostly, i’ve been a developer, probably 16 of the 20 years. Writing software is the reason i’m in the industry.
I am always wary of efforts to seduce me to the dark side (management). I find that I don’t think like most of the managers i’ve ever had. I laugh at jokes that make fun of managers or the corporate bureaucracy. And I find that I have a strong bias toward the developer side of the argument, no matter the subject. No matter what the management edict, I question and push back before submitting willingly. I question the reasoning of management decisions, or the source of management facts and information. It’s just the way I’m built.
This leads me to the inspiration of this post: The Gartner Group (The Big “G”, or GG). I work for an organization with about 900 employees in the corporate HQ. The are about 130 folks in IT.
My coworker and friend (see his posts about the Pacific Northwest Software Symposium, where we ate some excellent Pad Thai, and talked about Microsoft, Open Source, and Robert Scoble). Have been discussing the role of Gartner Group in our organization. When we have discussed this with a mutual friend and member of upper IT management, there is clearly a wide divide between us (the developers) and them (IT management).
Our first reaction when there is a mention of Gartner Group is pretty predictable. We roll our eyes, and let out an exsaparated “Puh-leeeze!”. My actual contact with Gartner Group has been limited, a teleconference or two, and a few too many white papers, and far too many web stories with Gartner Group quotes to back up one claim or another.
In our organization, Gartner Group is the first stop before any decision making takes place. Management wields Gartner Group information like a bludgeon, justifying any inane decision with a white paper, or analyst quote. The developers are instantly on the attack when the Gartner Group club is pulled out.
It’s all about trust. Management trusts the large, well funded, analyst organization that provides wishy-washy and vague guidance that can be spun to fit any organization. Developers find such organizations without credibility, because they are populated by people who aren’t fighting in the trenches with them. They are in nice window offices, wearing suits, and fiddling with Blackberries, just like our managers. They are one of them. That’s why we trust Slashdot, Digg, Joel On Software, and that Java developer we met at a conference who has a blog. They are us.
Self Inflicted Pain, Part II
In my previous missive I examined how we, the developers, had created our own implements of torture to apply to ourselves. I just received the final draft of the documentation standards from our internal team. It covers most of the expected things, why we need docs, why we need well written docs, where to store the docs, and most importantly, the list of required docs.
The required docs list is a matrix with the doc list on the left and across the top the types of development products (web service, portlet, jar file, etc). At the intersection of doc and product is a single letter code indicating whether the doc is required, optional, or occasionally required. As I expected there were no optional docs and only one that had occasional applicability.
As I read further the document went on to discuss the processes surrounding the production of the required development documents.
At the end, the last paragraph concedes that with time pressures on the development staff being what they are, we won’t actually have time to write all of the docs on the required list. So, it specifies a short “really” required list. That excludes the one document that is actually required to get an application deployed to QA or production.
It only took 5 months to write this 6 page document.
Self Inflicted Pain
Over the past few months, the development team I belong to has been working on a variety of team objectives whose goal is to improve our ability to write better software. It’s hard to disagree with such a thing, in theory. The objectives revolve around unit testing (my group), automated build/continuous integration, programming standards/code review, documentation, and system deployment process (me again). The unit testing group I am part of has so far produced a four page ‘best practices’ document related to unit testing and using JUnit. We plan to next hold two classes on JUnit and writing testable classes. We are trying to keep it lightweight and focused on the goal, improved unit testing. The only tool involved is JUnit, which we are already using. K.I.S.S. The automated build team is sticking with basic, well proven tools and techniques (Ant and CruiseControl) to build the beginnings of a continuous integration process. The coding standards/review team started off okay. They based the standards on the Sun Java standards that have been around for several years. Then, they ran off the road and into the weeds. They included PMD as part of the coding standards/review process, good choice – no complaint. Then they hit upon a code review tool that they demoed in our team meeting. This ill conceived mess of a program/process was something only a disengaged manager would love. It is cumbersome, tedious, and intrusive. It’s everything a software development tool shouldn’t be.
The same story can be told of the documentation team and the deployment team. These were all good ideas, but in the majority (3 of 5) of the teams, the original idea was lost and committee thinking took over. Gartner Group teleconferences were held, software vendors questioned, parent corporation white papers considered. The end results are more red tape, less time to produce good code (or any code), and happier corporate management. Who is to blame? Ourselves, the developers. The lure of more tools, the ‘perfect’ process, and the desire to make a ‘big’ impact all played a part. I’ve heard the phrase ‘Agile development’ thrown about the office many times (not to mention SOA, and other buzzspeak), but we as a team, and company keep heading the opposite direction.
It seems that simplicity has no traction in the big corporate IT world.
The Iron Triangle Still Rules
I’m in a session titled ‘Real World Agile’. It’s an introduction to the how’s and why’s of Agile development. One of the attendees wanted to know how Agile would help in his environment where they had a fixed required list of features and a hard deadline. Failure could torpedo the company. It was clear to me that he (like many) either hadn’t heard of, or wasn’t aware of the ‘Iron Triangle’ principle. This principle basically says that you can only control two of the these three items in a project: cost, scope, resources (people). If you have a fixed cost budget, and limited resources, the scope will have to change to meet the budget with your fixed resources. The resource element of the triangle can be people, hardware, software, etc. Scope can be a feature list, requirements, or an acceptable level of quality.
Software methodologies exist to manage the constraints of the Iron Triangle. Agile development is the latest in a long history of methodologies (and Agile is really an umbrella term that covers a number of methodologies) that hope to make it easier to manage to successfully complete projects.
Remember: You can’t have it all.
No Fluff, Just Stuff, Seattle Style
I’m sitting in the first session of day 2 of the Pacific Northwest Software Symposium. It’s part of the No Fluff, Just Stuff series of software development conferences. Yesterday I attended sessions on the Spring framework and Agile tools. Today will Agile development and Hibernate sessions. So far, the sessions have been very good. Lot’s of good info, with NO marketing crap. The speakers are generally authors and practiioners, not company representatives.
Arriving in Seattle proved the stereotype: Cool, gray, and drizzly. Natives have tried to convince Kelly and I that the weather has been really nice lately, sunny and warm. We suspect they work for the chamber of commerce. The hotel is very nice. It’s in the middle of an outside shopping mall area, just outside old downtown Redmond. We found a Thai resturant and had some very good Pad Thai to help wake me up before the first sessions. People have been a little surprised to find out that we flew from Kansas City to Seattle for this event. There have been other events that are part of the NFJS series as close as Omaha and Des Moines. I won’t try to explain why we decided we would rather go to Seattle.
Of course, my cell phone rang half way through lunch yesterday with a call from work about a project that is in testing. So, I had to fire up my laptop during one session to change some code and email the resulting EAR file back to the office for deployment.
I’m a little torn at the conference. Next door to the software symposium is the BetterPhoto seminar. I have already spent some time checking out the books for sale out front. I think I would rather spend a weekend at a photography seminar, but I am enjoying the software seminar so far.
Sitting in these sessions makes you very self conscious about your appearance. Just how well do I blend in with this collection of the geekiest people you will generally ever see outside of a Star Trek convention? Do I really look like these people? Unfortunately, the answer is probably yes. And once someone overhears your conversations during this event, your geekiness is confirmed. I guess that’s the curse of being a software development.
Simplicity is the ultimate sophistication. –Leonardo da Vinci
I’ve been thinking about application architecture and design since I arrived at my present employer. Along with development team dynamics (a topic for another time), architecture and design have reared thier ugly heads a number of times. Being new to this team and corporate environment I have looked at apps others have built, and have spent considerable time, debugging, modifying, and otherwise maintaining several examples. The experience level here varies from newbies with a year or less of Java experience to those with a considerable number of years of experience. There is a significant level of confidence on the part of the senior staff of the correctness of the design and architecture of several of the applications I have worked seen. At first, I questioned my own knowledge and experience (I had not been a 100% fulltime Java developer prior to this) when confronted by these apps. I blamed my confusion and lack of understanding on my inexperience. However, as I talked to others and had more contact with some of these applications and the authors and architects, I came to realize that the problem wasn’t entirely my own.
Increasingly, people seem to misinterpret complexity as sophistication, which is baffling—the incomprehensible should cause suspicion rather than admiration. Possibly this trend results from a mistaken belief that using a somewhat mysterious device confers an aura of power on the user –Niklaus Wirth
I was told of the power of this application server framework, and that open source framework, and the in-house sub-framework built upon it. The ‘benefits’ hit all of the buzzwords and phrases: loose coupling, service oriented, flexibility, testability, etc.
Unfortunately, what I saw was complexity that addressed no real problem domain, or an over-engineered solution to a simple problem.
In an article by David Hayden he observed:
“In the past year, however, I have started to see various classes and interfaces being added to my code to support looser coupling. I remember the effort of creating the classes, but have not realized any benefit from their existence because their creation wasn’t based on actual requirements, but my desire to use the ever evolving best practices and patterns to make my software ‘more maintainable and flexible.’ “
In another article he goes on to define ‘low coupling’ and the downside of it:
“If a part of your application is not really unstable, don’t treat it as such. How many times have you heard a person boast about having a data access layer that is loosely coupled to the rest of the application, but which there is an unlikely chance that a different database / data access layer will be used by the application?
Low coupling for the sake of lowering coupling can (but not necessarily will) cause unnecessary complexity in your application and perhaps cause more harm than good. Pick your battles wisely. “
The trend here is to apply low or loose coupling everywhere, because it’s a ‘best practice’, not because it solves a problem we actually need to solve. This trend has also led to the proliferation of ‘factory‘ classes. In the name of loose coupling, build a factory class to return an instance of an implementation of an interface. These have almost universally been one-offs with no return on the development investment. In most cases, instantiating a concete class directly would be faster, simpler, easier to understand and maintain, and most likely never change enough to require the loosely coupled version.
This article by Zed A. Shaw rings very true for me:
“This is the actions of an expert. They love complexity because the art is still new to them, something which should be explored. A list is not just a container, it’s a linked list, or red-black tree, or doubly linked list. To me, it’s just a container. I realize now that they’re missing my need for simple beautiful things. They don’t love quiet elegance, and would rather shout their superiority from the top of the mountain. Meanwhile, I’m just a lazy old man who wants to get his job done and write something without any wasted energy. I want to climb the mountain with the least amount of effort and their shouting is causing an avalanche of bad code. “
And here are some quotes that apply as well:
A charlatan makes obscure what is clear; a thinker makes clear what is obscure. –Hugh Kingsmill
Unformed people delight in the gaudy and in novelty. Cooked people delight in the ordinary. –Erik Naggum
Simplicity is prerequisite for reliability –Edsger W.Dijkstra
What have I learned? I’ve learned that my desire for simplicity in design and implementation is something to nurture, not second guess.
