Cleverness is Good

I stumbled across a nice essay on why Cleverness is bad and it bothered me.

I mean, it’s a good rule of thumb and all, and Uncle Bob’s advice is dead on in the situation he describes. Like this:

Trick #9 was the use of the and keyword to couple statements together to make “one liners”. It was billed as a trick that [cough] “more confident” Ruby programmers use.

So I got the pickaxe book out and looked up the and keyword. I showed Justin where it said that the second clause won’t be executed if the first clause is false (or nil!) So if handle_batch ever returned nil, then display_batch would never be called.

Yep. Bad. And in his story the code works at first and then goes bad later. It’s pitfalls of cleverness 101.

But suppose we truly carried this out to the extreme. What if we truly banned cleverness outright, retroactively, 50 years ago. What would be have saved the world from?

The basic diff algorithm is incredibly clever. Without that I’d be sunk. If we weren’t allowed to be clever at all then diffing two files might take minutes instead of under a second. The code that implemented the algorithm would be simple and maintainable, and almost too slow to be useful.

Ok. I totally cheated. That’s not a bad kind of cleverness. That’s important fundamental algorithm stuff.

I could come up with more examples. Atomic database commits we use hundreds of times every day, really fast web searches, clever clever clever. And it’s all good.

We need to precise about where we draw the line.

Could it be that only truly smart people, like Google engineers or Bell Labs people are allowed to be clever? I don’t buy it. How would you know when you were smart enough? You can’t know.

Could it be that when you’re practicing math it’s ok to be clever but not when coding? There’s some Haskell programmers out there who might agree. But there are enough Django and Rails users out there making clear every day that there’s plenty of good cleverness outside of the maths.

Here’s where we should draw the line. And it’s so simple that it’s not brilliant. It gets boring here,

Cleverness comes with a cost. Cost is the thing. It so boring I can barely write this.

I give up something when I come up with some clever way of making a system plan it’s own actions instead of making the user write long brittle configuration files. My planning algorithm could fail. I have to invest more in testing it up front to try and reassure myself and my users that it will actually work in the field.

But every once in awhile cleverness delivers big value. Maybe it gets things done faster by orders of magnitude. Maybe it saves humans from raking through minutia. When the value exceeds the cost, cleverness wins.

And Uncle Bob was dead on in that situation. The cost of (ab)using “and” in ruby code is that it sits as a landmine. Someday your code changes and you might not have tests to catch the failure. The benefit is almost nil.

But sometimes there’s real gold waiting outside of the usual box. Users tend to like the gold.

We won’t call you. We won’t email you. We won’t bother you. Period.

Anyone ever been duped while trying to buy insurance or refinance your home on the Internet? One day you were quietly minding your business, curious, then for a couple of weeks you’re getting calls multiple times per day. The commercial said I’d “win” but I’m just getting a bunch of calls and they’re all offering the same lousy rates.

What happens is sales people are new or having a lag in sales, so they buy “leads” from these web sites. They might have paid $50 for your name and phone number. You, dear friend, are now a “warm lead,” which overrides your status on the do not call list, and now you’re fair game for awhile. You’re fair game and they’re cash out of pocket to find out about you. Expect some calls, maybe some hard sell.

I want to do something different. I want to take this whole dirty game and turn it on its head. I’ll explain below. For those in a hurry, I’m building this now. The site is called “House Chat Monkeys.” You can search for Indianapolis Real Estate now. Once I’ve built this site up and get regular traffic I’m moving on to building the chat. After that it’s on to a bigger town and then a rapid expansion into other markets.

So follow my thinking out loud here. What if I put up a real estate search engine of my own. Mine will be cool and all, but what if on my search engine my super charismatic talented agent wife could talk with you directly?

Ok, that sounds less cool. You’re getting hassled. That’s not going to work.

See because half the problem is that when you ask a question and give your email address out, someone is going to follow up. So you’re going to have to go to all kinds of trouble telling them you were just curious. No, you are not thinking about moving right now. No, you would not like a free report on the current value of your home. No, get lost. Better to keep your question to yourself, no?

So what if you can ask your question, get an answer from a licensed agent, but they can’t hassle you. They can answer your question, and for as long as you like they can help you search, answer your questions, but they can’t know who you are or your email address or your phone number. Now we’re getting into some cool uncharted territory.

What if my search engine had two paths. Path 1, search for homes, ask questions, get answers, meet real estate agents, they can’t bother you. Path 2, describe the home you’d like, a real estate agent starts searching for you and showing you homes. Agent not getting it? Try a new one. They’ll never know who you are.

Our motto would be: “We won’t call you. We won’t email you. We won’t bother you. Period.” Too many words. I need a better motto.

Are you a real estate agent? How about his motto: “We get paid when you get paid.”

I’m interested in technical help and capital. (317) 753-8526

Maybe later we can sell some cars.

Why No One Likes Your Bug Tracker

This is how not to build communities, how not to get participation. I tried to report a bug in inkscape and was confronted with this:

I can’t think of a new password I’m willing to hand to sourceforge that I can remember.

Fortunately I found I could submit it anonymously. Fortunately I discovered this before I gave up.

14 Things You Can Say In A Fantasy Draft That You Can’t Say In a Real Draft

It’s NFL fantasy draft season, and I’m participating for the first time. I brought fresh ears to my league’s draft party, and a few things struck me as humorous, so much so that I kept a list.

In a real pro sports draft there are national TV cameras, ESPN commentators analyzing everything into the ground to fill time, and announcements of picks with great fanfare. There must be an army of folks surrying around behind the scenes finalizing decisions.

At a fantasy draft there are a bunch of folks at a table, some attacking the problem at hand with as much gravity as the executives and scouts at the real draft. They have notebooks, or notebook computers filled with research.

Then there are a few who just came for the food. Armed only with a printed out internet draft guide, thank you Fox Sports, they endure seventeen rounds of picks, bored out of their skulls by the ninth.

Sometimes what both kinds of folks say lacks the research depth of what is said at the real draft. Or maybe at the fantasy draft we can relax in the just a game atmosphere. (Yeah right…)

Whatever the reason, I enjoyed picturing an executive coming to a podium and announcing:

• “Did somebody already pick Larry Johnson?”
• “How many wide receivers do I need?”
• “The Patriots select Larry Johnson.” “Dude, he’s taken.”
• “What team am I?”
• Responding to a press attack later: “We picked Larry for his bye week.”
• “Can I phone a friend?”
• “The Patriots select Larry Johnson.” “Dude, did you know he’s injured?” “Oh, wait…”
• “Dude, you can’t take it back after you let go of the piece.”
• “Computer, remove two bad players.”
• “Just give me a player, doesn’t matter who.”
• “The Patriots select Larry Johnson.” “Dude, you already have him.”
• “Just pick already!”
• This really happened to us after our draft: League commisioner: “We’ll need everyone to email in their picks. My computer lost power before I saved.”

Our draft was all men, and later, the wives of the veteran managers, tired of being fantasy football widows formed their own league. They had a few announcements of their own:

• “What’s a tight end?” “I think it means he has a nice touche.”‘

And lastly, here are my picks:

• 01.02 Larry Johnson
• 02.11 Marvin Harrison (sentiment driven)
• 03.02 Larry Fitzgerald
• 04.11 Tatum Bell
• 05.02 Daunte Culpepper
• 06.11 Terry Glenn
• 07.02 Chris Brown
• 08.11 L.J. Smith
• 09.02 Indianapolis Defense (sentimental influence)
• 10.11 Roddy White
• 11.02 Neil Rackers
• 12.11 Brett Favre
• 13.02 Ben Troupe
• 14.11 Marty Booker
• 15.02 Chat Morton (because he was the last name in the draft guide)
• 16.11 Ryan Longwell
• 17.02 Santonio Holmes

Subversion Update Command Considered Harmful

This goes for CVS too.

Version control is the tool most programming workgroups won’t do without. It provides a kind of de facto backup mechanism, and lets us look at history to occasionally wonder about a particular change or line of code.

If multiple people are going to work on the same set of files, there will occasionally be two people wanting to work on the same thing. CVS pioneered or at least popularized the update comand as it works today. It was hailed as a major breakthrough for programming workgroups. Before CVS, if someone else was working on a file, you couldn’t touch it until they were done and had “unlocked” it, maybe days later. At that point you could pull their version of the file and make your change.

With CVS or SVN update you can mostly forget about the problem, as CVS will seamlessly merge others’ changes into your work before you commit yours, informing you only when it detects a problem, and leaving you to resolve it. These conflicts turn out to be quite rare in practice. Once you convince yourself that it works, it saves a lot of hassle.

I’d like to posit that both the pessimistic “locking” model and the optimistic “CVS update” model are badly broken. CVS made badly broken much more convenient, but it only accelerated a serious problem.

If I could change one thing about how I initially ran the pdk project I would have posted all changes to the list in patch form.

Early in pdk’s history I needed a small feature in git. I needed to expand on their already existing http support to include https with self signed certificates and http basic auth. The feature turned out to be small enough that I could add it myself. I didn’t want to maintain it so I contributed it upstream, and it was accepted. The feature is still in git today.

To get the patch accepted took some work and learning about the Linux development process. I fell in love with the culture, and in the process, the tool they built to support the culture.

Acceptance of the patch started with me posting it to the development mailing list.

Writing code for publication is different from writing code to get another feature done. The temptation to take a shortcut is virtually eliminated. If I do paper over an important case I’m going to have to point it out in a comment and/or in the email describing the patch. My patch and my description of it are going to end up in the inbox of hundreds, maybe thousands of people. That makes me want to get it right.

So while my patch was small, I was careful to make sure it was correct and the change was described clearly.

A few folks, including Junio Hamano, were interested in making git work over “dumb http.” Linus Torvalds, original author and maintainer of git at the time, didn’t give a rip about dumb http, preferring smart protocols. Despite his apathy over the feature, Linus trusted the judgement of Junio and took patches which improved the dumb http support over time. Ultimately Junio Hamano put my patch in with a series of his own http related patches and mine went into the official Linus tree as part of Junio’s set.

Today, Junio, not Linus, is the official maintainer of git. Linus still participates as a regular developer/user in the project. Even today Junio posts most of his work, maybe all of it, as patches and sets of patches to the mailing list before incorporating them into a release. Because he is careful about this, everyone’s work tends to be reviewed on list also. This isn’t so much a hard rule as it is a principle. Why would I want to disturb a complex system without help? Could someone look at this? Have I missed anything obvious?

Git is simply a flexible tool that can, among other things, support the Linux development model better than any.

The Linux development model stands in stark contrast to the more common CVS inspired development practice in closed and open source projects.

CVS and SVN update command encourages, nay insists, that developers skip the review step and trust one another blindly. How does this work in the normal project storing their code in SVN? Multiple times in a development cycle you incorporate into your code patches which have only ever been seen by their author. Could we imagine a more dangerous act? The times when you run cvs update are never convenient points for reviewing the incoming horde of changes. If you were to find a bug, you can’t reject the incoming change. You can add a review process to your dev group, but when the tool assumes your neighbor’s changes are good, your review is always second priority. The evidence of years of my own experience is that the implied trust truly is blind most of the time, this causes compounding trouble and we pay a heavy price.

You can’t even use SVN branches to keep separate lines of active development because you get zero help doing repeated merges.

The Linux culture promotes patch acceptance between developers as an explicit act. It implies a slight distrust and review follows naturally. Making a merge between developers is also always an explicit and transparent act. The only time there is a mass update (see git-rebase) is when someone at the tip of the project’s food chain (Linus, Junio) makes a particular line of _reviewed_ history the new official branch. Again I emphasize, these mass rebases, similar to the frequent cvs updates in normal land, are composed of mostly reviewed patches, so the whole thing is a lot safer. Furthermore, these rebases are never a barrier to commit a changeset.

When the history books are written, it will be known that Greg Hudson was dead wrong about distributed version control. His assertion that single integrators are a choke point throttling throughput has been thoroughly debunked by the last few years of Linux kernel development. Granted, some of the problems he pointed out did exist during the 2.4/2.5 Linux Kernel release series, but those problems cannot be attributed to the causes he proposes.

When the history books are written, it will be known that the Linux Kernel is, as of 2006, the largest agile project on the planet, having solved the “agile over TCP/IP” problem in a novel way. Git, and the torrent of emailed patches, are large part of what makes it possible.

Spammer Blows It

I’ve never seen a spammer fall so flat before. Check out this spam:

From: Theresa Polk
To: [email omitted]
Subject: Cecelia Kerr
Date: Thu, 23 Feb 2006 09:44:58 -0600 (10:44 EST)

Myrna,

%CUSTOM_LINK

Theresa Polk

How’s that for misconfiguring your software?

For the uninitiated, the %WHATEVER words should have been replaced with something useful, but they blew it. Reading this one was wierdly satisfying. Apparently spam which cannot possibly accomplish anything can escape my local spam filters. That’s wierdly comforting.

Rant: Python Plugin Systems

A plugin system should allow plugin authors to install their plugins using distutils.

The One True Way(tm) for writing a plugin system is this:

Have a configuration file. In the configuration file name python modules. The system exposing the plugin interface then dynamicly imports the named modules. The modules then have the opportunity to plug themselves into provided hooks.

There is no other way.

If you are scanning for python files in a directory, you are doing it wrong. Why? Plugin authors have to write their own system installation code. Distutils won’t do it.

If you are scanning for modules in a package directory, you are even more wrong. Why? See previous.

If you disagree, you are wrongest. Why? Because. You are wrong.

/rant.

Re: Listen to Women

Just want to say thanks to everyone for reading HOW Listen to Women. I’ve enjoyed reading the comments, for and against.

Perhaps sometime soon I’ll get time to address Lauren’s comment. It’s highly critical and I think she deserves an explanation. Fortunately I think I have one.

My favorite comment was from Lars:

The terminology here confuses me. We have this archetypical conversation:

A: I have problem X.

With the two possible answers

B1: You could solve that by Y.
B2: That must really make you feel Z.

I get that the message is that B2 is often the more appropriate answer, bizarre as it may seem to me. But I don’t understand why the B2 speaker is thought of as listening, while the B1 speaker is not. To device a solution to X, you clearly have to have heard the problem description. The response of someone who was REALLY not listening would perhaps be

B3: So what’s on TV?

I’m guessing this is s different usage of “listening” than I’m used to?

Again, thanks a million to all of you. It was an honor to host the discussion.

HOWTO Listen to Women: The One Thing Every Mangeek Should Know

Ok, first a little expectations management. I’m not going to talk about “breaking the ice” with girls in bars. I’m not going to tell you how to have a woman in every port, be a ladies’ man, or any other such dork.

I am going to tell you how to respond to a particular kind of situation, one that is particualarly important to handle well. This seems to be a deep dark secret that few men know. Even now that I know it have trouble remembering to use it.

One more thing before I get started: I don’t mean to generalize all women. Everybody’s different. I’ve observed over time that a lot of women verbally approach problems in a similar way and a men tend to fumble it repeatedly. This is just some non-obvious advice I learned from a Christian preacher. I’ve rarely heard it explained this way, and it’s a remarkably simple way for a man to get the right words out of his mouth in a sticky situation. ‘Nuff said.

Let’s begin with an illustration. What would you say?

She: “My landlord is a jerk. When I ask him to fix things, he’s always late and grouchy about it. He always brings up the time that I was two weeks late paying…”

You (reasonable but stupid): “Yeah, landlords are all that way. You just have to put up with it.”

Do not give advice.

You (stupider): “You should get a new apartment.”

Do not solve the problem.

You (stupidest): “Why didn’t you pay him?”

Do not criticize.

This is where the secret begins. Most men would have given one of the stupid answers. They seem perfectly reasonable from their point of view.

When a woman lets you in on a problem this way, the words she uses do not mean what you think they mean. Often a woman will frame a question or statement in a way that clearly begs advice or a solution. That’s just how you heard it. Remember this, she’s not stupid. She knows the answer already. There is something about the wiring in women that requires a kind of periodic verbal maintenance. She sounds like she wants advice, but really she’s going into this maintenance mode. Learn to follow the protocol.

Discipline grasshopper. When she presents you with the problem, close your mouth and think. Think about how she felt in the situation: angry that the landlord was late, annoyed that he’s lazy in fulfilling his agreement, and embarrassed that he keeps bringing up a past failure… Don’t let your curiosity about how she might have been out of work for two weeks distract you. I’d start with emotion of anger. Take a moment and empathize with her anger.

You: “Wow, that must have made you angry.”

Watch her face as you say this. Often there’s a look that will cross her face, kind of like shock and relief. You may be one of the first men in the world to actually listen to her.

She: “Yeah, I need to get a new apartment.”

Often she will solve her own problem. Really, there are three ways this can go. If she is just venting, she will solve the problem herself. If you guessed at the emotion wrong, she’ll correct you. You are still ok. You were listening.

The third option is that she will let you into a deeper friendship. The telltale sign is that she will immediately follow up this problem with another. The next one tends to be more personal.

I used to do this with a lot of girls. I don’t think it was a hot idea. Choose carefully.

Unless you are a therapist, don’t do this with married women. If her problem involves her mob-boss boyfriend, don’t do this. If you are married, realize that this seems to really poke a deep emotional nerve in women, and poking deep emotional nerves is something you should only be doing with your wife.

pdk 0.0.20

At work I just put out pdk 0.0.20. While only one new major feature is in place, it brings out a significant piece of pdk’s original vision.

The ability to publish your workspace history over anonymous http means that it is really easy for another person to pick up where you left off and make a few changes. They can either pull your work into their workspace as an add-on, or make some changes to your work and publish them back to you. Installing pdk on your web server will make this easy and safe, but it isn’t absolutely required. A simple rsync-to-public-server will get the job done with some minor race conditions.

We’ll have to rework some tutorials in the coming weeks. We can make them simpler now. Basically now you can get started customizing a distro in a couple of easy steps. See the mailing list for details, but it boils down to:

1. Pull Ian’s (or somebody else’s) workspace.
2. Add your stuff.

The only major headaches left in the distro development process are making installable iso images and package maintenance.

Jeff has been putting a lot of work into pickaxe, and now he is working on integrating it directly into pdk. This will make the new user experience really easy all the way out to making new installable iso images.

I’ve got package build-farming across multiple architectures and some really jaw-dropping maintenance reporting stuff on my mental todo list.

I’m getting excited about all this progress. One user has already proven that the pull functionality works in the field.

Very cool.