Archive for the ‘General’ category

Cleverness is Good

August 21, 2008

I stumbled across a nice essay on why Cleverness is bad and it bothered me.

I mean, it’s a good rule of thumb and all, and Uncle Bob’s advice is dead on in the situation he describes. Like this:

Trick #9 was the use of the and keyword to couple statements together to make “one liners”. It was billed as a trick that [cough] “more confident” Ruby programmers use.

So I got the pickaxe book out and looked up the and keyword. I showed Justin where it said that the second clause won’t be executed if the first clause is false (or nil!) So if handle_batch ever returned nil, then display_batch would never be called.

Yep. Bad. And in his story the code works at first and then goes bad later. It’s pitfalls of cleverness 101.

But suppose we truly carried this out to the extreme. What if we truly banned cleverness outright, retroactively, 50 years ago. What would be have saved the world from?

The basic diff algorithm is incredibly clever. Without that I’d be sunk. If we weren’t allowed to be clever at all then diffing two files might take minutes instead of under a second. The code that implemented the algorithm would be simple and maintainable, and almost too slow to be useful.

Ok. I totally cheated. That’s not a bad kind of cleverness. That’s important fundamental algorithm stuff.

I could come up with more examples. Atomic database commits we use hundreds of times every day, really fast web searches, clever clever clever. And it’s all good.

We need to precise about where we draw the line.

Could it be that only truly smart people, like Google engineers or Bell Labs people are allowed to be clever? I don’t buy it. How would you know when you were smart enough? You can’t know.

Could it be that when you’re practicing math it’s ok to be clever but not when coding? There’s some Haskell programmers out there who might agree. But there are enough Django and Rails users out there making clear every day that there’s plenty of good cleverness outside of the maths.

Here’s where we should draw the line. And it’s so simple that it’s not brilliant. It gets boring here,

Cleverness comes with a cost. Cost is the thing. It so boring I can barely write this.

I give up something when I come up with some clever way of making a system plan it’s own actions instead of making the user write long brittle configuration files. My planning algorithm could fail. I have to invest more in testing it up front to try and reassure myself and my users that it will actually work in the field.

But every once in awhile cleverness delivers big value. Maybe it gets things done faster by orders of magnitude. Maybe it saves humans from raking through minutia. When the value exceeds the cost, cleverness wins.

And Uncle Bob was dead on in that situation. The cost of (ab)using “and” in ruby code is that it sits as a landmine. Someday your code changes and you might not have tests to catch the failure. The benefit is almost nil.

But sometimes there’s real gold waiting outside of the usual box. Users tend to like the gold.

We won’t call you. We won’t email you. We won’t bother you. Period.

October 2, 2007

Anyone ever been duped while trying to buy insurance or refinance your home on the Internet? One day you were quietly minding your business, curious, then for a couple of weeks you’re getting calls multiple times per day. The commercial said I’d “win” but I’m just getting a bunch of calls and they’re all offering the same lousy rates.

What happens is sales people are new or having a lag in sales, so they buy “leads” from these web sites. They might have paid $50 for your name and phone number. You, dear friend, are now a “warm lead,” which overrides your status on the do not call list, and now you’re fair game for awhile. You’re fair game and they’re cash out of pocket to find out about you. Expect some calls, maybe some hard sell.

I want to do something different. I want to take this whole dirty game and turn it on its head. I’ll explain below. For those in a hurry, I’m building this now. The site is called “House Chat Monkeys.” You can search for Indianapolis Real Estate now. Once I’ve built this site up and get regular traffic I’m moving on to building the chat. After that it’s on to a bigger town and then a rapid expansion into other markets.

So follow my thinking out loud here. What if I put up a real estate search engine of my own. Mine will be cool and all, but what if on my search engine my super charismatic talented agent wife could talk with you directly?

Ok, that sounds less cool. You’re getting hassled. That’s not going to work.

See because half the problem is that when you ask a question and give your email address out, someone is going to follow up. So you’re going to have to go to all kinds of trouble telling them you were just curious. No, you are not thinking about moving right now. No, you would not like a free report on the current value of your home. No, get lost. Better to keep your question to yourself, no?

So what if you can ask your question, get an answer from a licensed agent, but they can’t hassle you. They can answer your question, and for as long as you like they can help you search, answer your questions, but they can’t know who you are or your email address or your phone number. Now we’re getting into some cool uncharted territory.

What if my search engine had two paths. Path 1, search for homes, ask questions, get answers, meet real estate agents, they can’t bother you. Path 2, describe the home you’d like, a real estate agent starts searching for you and showing you homes. Agent not getting it? Try a new one. They’ll never know who you are.

Our motto would be: “We won’t call you. We won’t email you. We won’t bother you. Period.” Too many words. I need a better motto.

Are you a real estate agent? How about his motto: “We get paid when you get paid.”

I’m interested in technical help and capital. (317) 753-8526

Maybe later we can sell some cars.

Why No One Likes Your Bug Tracker

September 1, 2006

This is how not to build communities, how not to get participation. I tried to report a bug in inkscape and was confronted with this:

Sourceforge Bug Submission Hurdle
Sourceforge Bug Submission Hurdle
I can’t think of a new password I’m willing to hand to sourceforge that I can remember.

Fortunately I found I could submit it anonymously. Fortunately I discovered this before I gave up.

14 Things You Can Say In A Fantasy Draft That You Can’t Say In a Real Draft

July 19, 2006

It’s NFL fantasy draft season, and I’m participating for the first time. I brought fresh ears to my league’s draft party, and a few things struck me as humorous, so much so that I kept a list.

In a real pro sports draft there are national TV cameras, ESPN commentators analyzing everything into the ground to fill time, and announcements of picks with great fanfare. There must be an army of folks surrying around behind the scenes finalizing decisions.

At a fantasy draft there are a bunch of folks at a table, some attacking the problem at hand with as much gravity as the executives and scouts at the real draft. They have notebooks, or notebook computers filled with research.

Then there are a few who just came for the food. Armed only with a printed out internet draft guide, thank you Fox Sports, they endure seventeen rounds of picks, bored out of their skulls by the ninth.

Sometimes what both kinds of folks say lacks the research depth of what is said at the real draft. Or maybe at the fantasy draft we can relax in the just a game atmosphere. (Yeah right…)

Whatever the reason, I enjoyed picturing an executive coming to a podium and announcing:

  • “Did somebody already pick Larry Johnson?”
  • “How many wide receivers do I need?”
  • “The Patriots select Larry Johnson.” “Dude, he’s taken.”
  • “What team am I?”
  • Responding to a press attack later: “We picked Larry for his bye week.”
  • “Can I phone a friend?”
  • “The Patriots select Larry Johnson.” “Dude, did you know he’s injured?” “Oh, wait…”
  • “Dude, you can’t take it back after you let go of the piece.”
  • “Computer, remove two bad players.”
  • “Just give me a player, doesn’t matter who.”
  • “The Patriots select Larry Johnson.” “Dude, you already have him.”
  • “Just pick already!”
  • This really happened to us after our draft: League commisioner: “We’ll need everyone to email in their picks. My computer lost power before I saved.”

Our draft was all men, and later, the wives of the veteran managers, tired of being fantasy football widows formed their own league. They had a few announcements of their own:

  • “What’s a tight end?” “I think it means he has a nice touche.”‘

And lastly, here are my picks:

  • 01.02 Larry Johnson
  • 02.11 Marvin Harrison (sentiment driven)
  • 03.02 Larry Fitzgerald
  • 04.11 Tatum Bell
  • 05.02 Daunte Culpepper
  • 06.11 Terry Glenn
  • 07.02 Chris Brown
  • 08.11 L.J. Smith
  • 09.02 Indianapolis Defense (sentimental influence)
  • 10.11 Roddy White
  • 11.02 Neil Rackers
  • 12.11 Brett Favre
  • 13.02 Ben Troupe
  • 14.11 Marty Booker
  • 15.02 Chat Morton (because he was the last name in the draft guide)
  • 16.11 Ryan Longwell
  • 17.02 Santonio Holmes

Subversion Update Command Considered Harmful

June 19, 2006

This goes for CVS too.

Version control is the tool most programming workgroups won’t do without. It provides a kind of de facto backup mechanism, and lets us look at history to occasionally wonder about a particular change or line of code.

If multiple people are going to work on the same set of files, there will occasionally be two people wanting to work on the same thing. CVS pioneered or at least popularized the update comand as it works today. It was hailed as a major breakthrough for programming workgroups. Before CVS, if someone else was working on a file, you couldn’t touch it until they were done and had “unlocked” it, maybe days later. At that point you could pull their version of the file and make your change.

With CVS or SVN update you can mostly forget about the problem, as CVS will seamlessly merge others’ changes into your work before you commit yours, informing you only when it detects a problem, and leaving you to resolve it. These conflicts turn out to be quite rare in practice. Once you convince yourself that it works, it saves a lot of hassle.

I’d like to posit that both the pessimistic “locking” model and the optimistic “CVS update” model are badly broken. CVS made badly broken much more convenient, but it only accelerated a serious problem.

If I could change one thing about how I initially ran the pdk project I would have posted all changes to the list in patch form.

Early in pdk’s history I needed a small feature in git. I needed to expand on their already existing http support to include https with self signed certificates and http basic auth. The feature turned out to be small enough that I could add it myself. I didn’t want to maintain it so I contributed it upstream, and it was accepted. The feature is still in git today.

To get the patch accepted took some work and learning about the Linux development process. I fell in love with the culture, and in the process, the tool they built to support the culture.

Acceptance of the patch started with me posting it to the development mailing list.

Writing code for publication is different from writing code to get another feature done. The temptation to take a shortcut is virtually eliminated. If I do paper over an important case I’m going to have to point it out in a comment and/or in the email describing the patch. My patch and my description of it are going to end up in the inbox of hundreds, maybe thousands of people. That makes me want to get it right.

So while my patch was small, I was careful to make sure it was correct and the change was described clearly.

A few folks, including Junio Hamano, were interested in making git work over “dumb http.” Linus Torvalds, original author and maintainer of git at the time, didn’t give a rip about dumb http, preferring smart protocols. Despite his apathy over the feature, Linus trusted the judgement of Junio and took patches which improved the dumb http support over time. Ultimately Junio Hamano put my patch in with a series of his own http related patches and mine went into the official Linus tree as part of Junio’s set.

Today, Junio, not Linus, is the official maintainer of git. Linus still participates as a regular developer/user in the project. Even today Junio posts most of his work, maybe all of it, as patches and sets of patches to the mailing list before incorporating them into a release. Because he is careful about this, everyone’s work tends to be reviewed on list also. This isn’t so much a hard rule as it is a principle. Why would I want to disturb a complex system without help? Could someone look at this? Have I missed anything obvious?

Git is simply a flexible tool that can, among other things, support the Linux development model better than any.

The Linux development model stands in stark contrast to the more common CVS inspired development practice in closed and open source projects.

CVS and SVN update command encourages, nay insists, that developers skip the review step and trust one another blindly. How does this work in the normal project storing their code in SVN? Multiple times in a development cycle you incorporate into your code patches which have only ever been seen by their author. Could we imagine a more dangerous act? The times when you run cvs update are never convenient points for reviewing the incoming horde of changes. If you were to find a bug, you can’t reject the incoming change. You can add a review process to your dev group, but when the tool assumes your neighbor’s changes are good, your review is always second priority. The evidence of years of my own experience is that the implied trust truly is blind most of the time, this causes compounding trouble and we pay a heavy price.

You can’t even use SVN branches to keep separate lines of active development because you get zero help doing repeated merges.

The Linux culture promotes patch acceptance between developers as an explicit act. It implies a slight distrust and review follows naturally. Making a merge between developers is also always an explicit and transparent act. The only time there is a mass update (see git-rebase) is when someone at the tip of the project’s food chain (Linus, Junio) makes a particular line of _reviewed_ history the new official branch. Again I emphasize, these mass rebases, similar to the frequent cvs updates in normal land, are composed of mostly reviewed patches, so the whole thing is a lot safer. Furthermore, these rebases are never a barrier to commit a changeset.

When the history books are written, it will be known that Greg Hudson was dead wrong about distributed version control. His assertion that single integrators are a choke point throttling throughput has been thoroughly debunked by the last few years of Linux kernel development. Granted, some of the problems he pointed out did exist during the 2.4/2.5 Linux Kernel release series, but those problems cannot be attributed to the causes he proposes.

When the history books are written, it will be known that the Linux Kernel is, as of 2006, the largest agile project on the planet, having solved the “agile over TCP/IP” problem in a novel way. Git, and the torrent of emailed patches, are large part of what makes it possible.

Spammer Blows It

February 23, 2006

I’ve never seen a spammer fall so flat before. Check out this spam:

From: Theresa Polk
To: [email omitted]
Subject: Cecelia Kerr
Date: Thu, 23 Feb 2006 09:44:58 -0600 (10:44 EST)



Theresa Polk

How’s that for misconfiguring your software?

For the uninitiated, the %WHATEVER words should have been replaced with something useful, but they blew it. Reading this one was wierdly satisfying. Apparently spam which cannot possibly accomplish anything can escape my local spam filters. That’s wierdly comforting.

Rant: Python Plugin Systems

February 20, 2006

A plugin system should allow plugin authors to install their plugins using distutils.

The One True Way(tm) for writing a plugin system is this:

Have a configuration file. In the configuration file name python modules. The system exposing the plugin interface then dynamicly imports the named modules. The modules then have the opportunity to plug themselves into provided hooks.

There is no other way.

If you are scanning for python files in a directory, you are doing it wrong. Why? Plugin authors have to write their own system installation code. Distutils won’t do it.

If you are scanning for modules in a package directory, you are even more wrong. Why? See previous.

If you disagree, you are wrongest. Why? Because. You are wrong.