It's quite rare for me to be so excited about a new book that I'll write about it even before I've read it. But I just got a copy of Scott Rosenberg's new Dreaming in Code, and I can't wait to read it.
Scott Rosenberg is a founder and editor of Salon. The reason I'm so excited about this book is this: Of all the journalists covering technology and especially software development I think Scott understands it the best. He's an excellent writer, too. Two years ago, Scott interviewed me for an article in Salon called The Shlemiel way of software. A lot of other people have interviewed me, but this was the only time I ever felt that the interviewer did a great job of capturing exactly what I think.
The other reason I can't wait to read this book is that it's about the development of Chandler. I've been watching Chandler closely ever since it was probably doomed by being breathlessly hailed by Wired Magazine as "The Outlook Killer" in January, 2003. Well, it's January 2007 now, Outlook isn't dead, and Chandler is up to 0.7alpha4. Being hyped by Wired Magazine is almost certainly a bad sign. I've been wondering what the heck went so wrong on the Chandler team, or perhaps it's just a matter of good software taking ten years? Usually the first useful version takes less than that, though.
Anyway, only three days after announcing my new policy of "no gifts," I'm stuck with this copy of the book I got in the mail for free. Should I send it back and buy my own copy? The New York Times policy on Journalism Ethics says, "Staff members may keep for their own collections—but may not sell or copy—books, recordings, tapes, compact discs and computer programs sent to them for review. Such submissions are considered press releases."
Eyes work using a page fault mechanism. They’re so good at it that you don’t even notice.
You can only see at a high-resolution in a fairly small area, and even that has a big fat blind spot right exactly in the middle, but you still walk around thinking you have a ultra-high resolution panoramic view of everything. Why? Because your eyes move really fast, and, under ordinary circumstances, they are happy to jump instantly to wherever you need them to jump to. And your mind provides this really complete abstraction, providing you with the illusion of complete vision when all you really have is a very small area of high res vision, a large area of extremely low-res vision, and the ability to page-fault-in anything you want to see—so quickly that you walk around all day thinking you have the whole picture projected internally in a little theatre in your brain.
This is really, really useful, and lots of other things work this way, too. Your ears are good at tuning in important parts of conversations. Your fingers reach around and touch anything they need to, whether it’s a fine merino wool sweater or the inside of your nose, giving you a full picture of what everything feels like. When you dream, your mind asks all kinds of questions that it’s used to asking the senses (what’s that? Look over there!) but your senses are temporarily turned off (you are, after all, asleep), so they get back sort-of random answers, which you combine into a funny story in your brain called a dream. And then when you try to recount the dream to your boyfriend in the morning, even though it seemed totally, completely realistic, you suddenly realize that you don’t know what happened, actually, so you have to make shit up. If you had stayed asleep for another minute or two your brain would have asked your senses what kind of mammal was swimming with you in the rose bush, and gotten back some retarded, random answer (a platypus!), but you woke up, so until you tried to tell the story, you didn’t even realize that you needed to know what was in the rose bushes with you to make the story coherent to your partner. Which it never is. So please don’t tell me about your dreams.
One of the unfortunate side effects is that your mind gets into a bad habit of overestimating how clearly it understands things. It always thinks it has The Big Picture even when it doesn’t.
This is a particularly dangerous trap when it comes to software development. You get some big picture idea in your head for what you want to do, and it all seems so crystal clear that it doesn’t even seem like you need to design anything. You can just dive in and start implementing your vision.
Say, for example, that your vision is to rebuild an old DOS personal information manager, which was really really great but totally unappreciated. It seems easy. Everything about how the whole thing works seems so obvious, you don’t even try to design the thing… you just hire a bunch of programmers and start banging out code.
Now you’ve made two mistakes.
Number one, you fell for that old overconfidence trick of your mind. “Oh, yeah, we totally know how to do this! It’s all totally clear to us. No need to spec it out. Just write the code.”
Number two, you hired programmers before you designed the thing. Because the only thing harder than trying to design software is trying to design software as a team.
I can’t tell you how many times I’ve been in a meeting with even one or two other programmers, trying to figure out how something should work, and we’re just not getting anywhere. So I go off in my office and take out a piece of paper and figure it out. The very act of interacting with a second person was keeping me from concentrating enough to design the dang feature.
What kills me is the teams who get into the bad habit of holding meetings every time they need to figure out how something is going to work. Did you ever try to write poetry in a committee meeting? It’s like a bunch of fat construction guys trying to write an opera while sitting on the couch watching Baywatch. The more fat construction guys you add to the couch, the less likely you are to get opera out of it.
At least turn off the TV!
Now, it would be shockingly presumptuous of me to try to guess what happened on the Chandler team, and why it’s taken them millions of dollars and several years to get to where they are now, which is, they have a pretty buggy and incomplete calendar application that’s not very impressive compared to the 58 me-too Web 2.0 calendars that came out last year, each of which was developed by two college kids in their spare time, one of whom really mostly just drew mascots.
Chandler doesn’t even have a mascot!
Like I say, I can’t presume to know what went wrong. Maybe nothing. Maybe they feel like they’re right on track. Scott Rosenberg’s excellent new book, which was supposed to be a Soul of a New Machine for the hottest open source startup of the decade, ends up, in frustration, with Scott cutting the story short because Chandler 1.0 was just not going to happen any time soon (and presumably Rosenberg couldn’t run the risk that we wouldn’t be using books at all by the time it shipped, opting instead to absorb knowledge by taking a pill).
Still, it’s a great look at one particular type of software project: the kind that ends up spinning and spinning its wheels without really going anywhere because the vision was too grand and the details were a little short. Near as I can tell, Chandler’s original vision was pretty much just to be “revolutionary.” Well, I don’t know about you, but I can’t code “revolutionary.” I need more details to write code. Whenever the spec describes the product in terms of adjectives (“it will be extremely cool”) rather than specifics (“it will have brushed-aluminum title bars and all the icons will be reflected a little bit, as if placed on a grand piano”) you know you’re in trouble.
The only concrete design ideas, as far as I could tell from Rosenberg’s book, were “peer-to-peer,” “no silos,” and “natural language date interpretation.” This may be the a limitation of the book, but the initial design sure seemed to be extremely vague.
“Peer-to-peer” was the raison-d’être of Chandler… why should you have to buy Microsoft Exchange Server to coordinate schedules? It turned out that peer-to-peer synchronization was too hard, or something, and this feature was cut. Now there’s a server called Cosmo.
“No Silos” was supposed to mean that instead of having your email in one silo, and your calendar in another silo, and your reminder notes in a third, there would just be a single unified silo holding everything.
As soon as you start asking questions about “No Silos,” you realize it’s not going to work. Do you put your email on the calendar? Where? On the day when it arrived? So now I have 200 Viagra ads on Friday obscuring the one really important shareholder meeting?
Eventually “No Silos” got designed into this idea of stamps, so, for example, you could “stamp” any document or note or calendar item with an email stamp and suddenly that item could be mailed to anyone. Guess what? That feature has been in Microsoft Office for the last decade or so. They finally took it out in Office 2007 because nobody cared. There are too many easy ways to email people things.
Indeed, I think the idea of “No Silos” is most appealing to architecture astronauts, the people who look at subclasses and see abstract base classes, and who love to move functionality from the subclass into the base class for no good reason other than architectural aesthetics. This is usually a terrible user interface design technique. The way you make users understand your program model is with metaphors. When you make things look, feel, and most importantly, behave like things in the real world, users are more likely to figure out how to use the program, and the app will be easier to use. When you try to combine two very dramatically different real-world items (email and appointments) into the same kind of thing in the user interface, usability suffers because there’s no longer a real-world metaphor that applies.
The other cool thing that Kapor kept telling everyone who would listen is that Agenda would let you type things like “Next Tuesday” and magically you’d get an appointment for next Tuesday. This is slicker than heck, but every half-decent calendar program for the last decade has done this. Not revolutionary.
The Chandler team also overestimated how much help they would get from volunteers. Open source doesn’t quite work like that. It’s really good at implementing copycat features, because there’s a spec to work from: the implementation you’re copying. It’s really good at Itch Scratching features. I need a command line argument for EBCDIC, so I’ll add it and send in the code. But when you have an app that doesn’t do anything yet, nobody finds it itchy. They’re not using it. So you don’t get volunteers. Almost everyone on the Chandler dev team got paid.
Again I must forcefully apologize to the Chandler team if Rosenberg missed the point somehow, or if he gave a completely incorrect impression of what was really holding up progress, and my bias—to blame these kinds of debacles on a failure to design—is showing.
All that said, one good thing did come out of the project: a fascinating book in the style of Soul of a New Machine or Showstopper about a software development project that failed to converge. Highly recommended.
I fixed a bunch of little bugs in FogBugz Add-in for Visual Studio. The new version, 2.1, is now available.
It now remembers its position when you exit Visual Studio, and works better with SSL and authenticated FogBugz installations. A complete list of bug fixes is here.
Thanks to Olivier, James, Matt, Jason, Alan, and Miguel for detailed bug reports which helped me track down and fix all these problems.
The Unofficial Apple Weblog: “I'm not the only person who knows the joys of trying to talk a frustrated parent through some troubleshooting technique.”
Infinite Loop: “Apple Remote Desktop costs mucho bucks, and these options usually require at least a little bit of configuration—configuration that, unless you set it up yourself, can be a challenge for computer-frustrated family members. Copilot, like similar other ‘remote tech support’ systems out there, does not require any sort of configuration on the part of the client, making it easy for you to have someone install it and you accessing their computer in no time.”
Daniel Jalkut: “Up to now Fog Creek was ‘the best software company that doesn’t make Mac products.’ Now I guess I’ll have to drop that qualification.”
Hoorah! Fog Creek Copilot 2.0 is now online, with three, no wait, five, no, three new features.
Well, I guess it depends how you count. In a moment I’ll count ‘em. In the meantime, a little background.
Fog Creek Copilot is a remote tech support service that lets one person control another computer remotely, much like VNC or RDC, with the advantage that it requires zero configuration, works through firewalls, and installs nothing.
Two summers ago, we had four interns here who built it, all by themselves, over the course of their 12 week internship. The only thing we gave them was a spec, some desks, and computers. They put together the web site, the documentation, all the code for five major components, came up with the marketing plan, did usability testing, and demoed to the public at a trade show. They kept a blog, which you can still read, and someone even made a documentary movie made about their summer.
(Sidebar: One of the reasons they were able to accomplish so much in one summer is that they used open source software as a starting point. Of course, everything they did, with the exception of our proprietary back end server code, is available under the GPL license.)
OK. New features!
1. Support for Mac! OMG! All versions of Mac OS X from 10.2 on are now supported. I’m fairly confident that our Mac remote desktop implementation is second to none.
Oh, wait. Interruption. You may be wondering, if the interns did the whole thing over a summer, who wrote all this new code?
The answer is that two of the interns accepted full-time jobs at Fog Creek, Tyler and Ben. Tyler extended his summer until December, and then headed off for a leave of absence to finish his Masters degree at Stanford. Ben graduated from Duke last summer and has been cranking away on 2.0 since then. Ben, by the way, is the only person I know who writes code in C, C++, C#, and Objective C all in the same day, while writing a book about Smalltalk at night. We also had a Mac programming guru, Daniel Jalkut, get us started with the Mac port.
OK, next new big feature:
2. Direct Connect! We’ve always done everything we can to make sure that Fog Creek Copilot can connect in any networking situation, no matter what firewalls or NATs are in place. To make this happen, both parties make outbound connections to our server, which relays traffic on their behalf. Well, in many cases, this isn’t necessary. So version 2.0 does something rather clever: it sets up the initial connection through our servers, so you get connected right away with 100% reliability. But then once you’re all connected, it quietly, in the background, looks for a way to make a direct connection. If it can’t, no big deal: you just keep relaying through our server. If you can make a direct peer-to-peer connection, it silently shifts your data onto the direct connection. You won’t notice anything except, probably, much faster communication.
3. File transfer! An easy-to-use file upload and download feature makes this the PERFECT application for installing Firefox on all your relatives’ computers. It’s especially handy for tech support scenarios. Imagine this: your new software works great everywhere but this one guy has a wacky system that makes your software keep crashing. So you use Fog Creek Copilot to take over his system, and then you use the file transfer feature to copy new builds to his computer as you try to fix the problem.
4. Does this count as a feature? We lowered the price for day passes–24 hours of usage—from $10 to no, not $9, not $8, not $7, would you believe it’s only FIVE DOLLARS? That’s right, unlimited usage for 24 hours for five lonely bucks.
I guess I should explain the reasoning behind that. First of all, the direct connect feature (#2) should reduce our bandwidth bills in many situations, so we can pass that savings on.
Second, we don’t want anyone to have an excuse not to use Fog Creek Copilot. To avoid paying $10, you might actually be crazy enough to try to just talk your mom into uninstalling Norton Utilities, punching the appropriate holes in the Windows firewall, and setting up appropriate port-forwarding rules on her broadband router… but for $5, why go through the trouble? Or you might be willing to set up your own server outside the firewall, with VNC running as a listener, and walk your customers through setting up VNC and connecting back to you, but again, why bother for five bucks?
We think that’s a negligible price to pay to know that all you need to tell your mom, or your customer, is “Go to copilot.com, type in this number, and download and run the program you find there.” And to know that it will Just Work.
We’re betting that the lower price will lead to more users, which will lead to more corporate subscriptions, which will lead to higher total revenues.
So. Congratulations to Tyler and Ben for a fantastic upgrade!
She posted a couple of chapters online, including the interview with me, which is probably the most complete story of the early days of Fog Creek Software in print.
“There's a bunch of people out there doing certain types of things and they seem to be pretty incompetent, but they're getting huge valuations. Surely if I did those same things, knowing that I am less incompetent—merely semi-incompetent as opposed to extremely incompetent—I should be able to achieve at least their level of success.”
I haven't read the book yet... still waiting for my copy to arrive from Amazon, which appears to be beyond semi-incompetent with regards to what "overnight shipping" might mean when you order something on Thursday. Check out the list of interviews, though: virtually everyone starting a high tech company will be all over this book like senators on cake.
1113 posts over 15 years. Everything I’ve ever published is right here.
There’s a software company in New York City dedicated to doing things the right way and proving that it can be done profitably and successfully.