The Cathedral and the Bazaar (2)
Popclient becomes Fetchmail
The real turning point in the project was when Harry Hochheiser sent me his scratch code for forwarding mail to the client machine's SMTP port. I realized almost immediately that a reliable implementation of this feature would make all the other mail delivery modes next to obsolete.
For many weeks I had been tweaking fetchmail rather incrementally while feeling like the interface design was serviceable but grubby—inelegant and with too many exiguous options hanging out all over. The options to dump fetched mail to a mailbox file or standard output particularly bothered me, but I couldn't figure out why.
Popclient 变成了 Fetchmail
这个项目的真正转折点是哈利·浩赫海斯(Harry Hochheiser)把他的转发邮件到客户机SMTP端口的草稿编码发给了我。我几乎马上意识到这个功能要是可靠地实现出来,会让其它所有的邮件递送模式几近成为往事。
许多个星期以来,我更像是在一点点地摆弄fetchmail;一边感觉到界面设计还可以用,但是不够干净漂亮——太多不足道的选项,比比皆是。把收取的邮件扔在一个邮件箱里或转到标准输出里的选项尤其让我心烦,但我想不出为什么。
(If you don't care about the technicalia of Internet mail, the next two paragraphs can be safely skipped.)
What I saw when I thought about SMTP forwarding was that popclient had been trying to do too many things. It had been designed to be both a mail transport agent (MTA) and a local delivery agent (MDA). With SMTP forwarding, it could get out of the MDA business and be a pure MTA, handing off mail to other programs for local delivery just as sendmail does.
Why mess with all the complexity of configuring a mail delivery agent or setting up lock-and-append on a mailbox when port 25 is almost guaranteed to be there on any platform with TCP/IP support in the first place? Especially when this means retrieved mail is guaranteed to look like normal sender-initiated SMTP mail, which is really what we want anyway.
(Back to a higher level....)
Even if you didn't follow the preceding technical jargon, there are several important lessons here. First, this SMTP-forwarding concept was the biggest single payoff I got from consciously trying to emulate Linus's methods. A user gave me this terrific idea—all I had to do was understand the implications.
11. The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better.
Interestingly enough, you will quickly find that if you are completely and self-deprecatingly truthful about how much you owe other people, the world at large will treat you as though you did every bit of the invention yourself and are just being becomingly modest about your innate genius. We can all see how well this worked for Linus!
(如果你对互联网邮件的技术细节不感兴趣,可以安全跳过下面的两个段落。)
当我考虑SMTP转发的时候,我看到的是popclient在试图做太多的事情。它被设计成了既是一个邮件传输工具(MTA),又是一个本地投递工 具(MDA)。有了SMTP转发功能,它就可以摆脱MDA的负荷,专心只作MTA,把邮件的本地投递留给sendmail之类的程序来做。
当几乎每一个有TCP/IP支持的平台上都预留了25号端口的时候,为什么还要去折腾MDA的复杂配置或邮箱的“锁定-添加”呢?尤其是这意味着收到的邮件看起来几乎保证和正常的人工发送的SMTP邮件一样——正中我们下怀。
(返回到高一层次上……)
即使你没有去读上面的技术术语,这里有几条重要的经验。首先,这个SMTP转发的点子是我有意模仿林纳斯的方法的最大的收获。一个用户给了我这个巨棒的点子——我需要做的仅仅是理解它的含义。
11)仅次于拥有好的主意的是认识到来自于用户的好主意。有时候后者更好一些。
有意思的是,如果你完全坦诚和谦虚地承认你欠了别人多少,你很快就会发现外面的世界会把你放在这样一个地位上——好像你自己做了发明的每一部分、只不过对你生来的天才一味谦虚而已。我们都可以看到林纳斯如此受益了多少!
(When I gave my talk at the first Perl Conference in August 1997, hacker extraordinaire Larry Wall was in the front row. As I got to the last line above he called out, religious-revival style, ``Tell it, tell it, brother!''. The whole audience laughed, because they knew this had worked for the inventor of Perl, too.)
After a very few weeks of running the project in the same spirit, I began to get similar praise not just from my users but from other people to whom the word leaked out. I stashed away some of that email; I'll look at it again sometime if I ever start wondering whether my life has been worthwhile :-).
But there are two more fundamental, non-political lessons here that are general to all kinds of design.
12. Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong.
I had been trying to solve the wrong problem by continuing to develop popclient as a combined MTA/MDA with all kinds of funky local delivery modes. Fetchmail's design needed to be rethought from the ground up as a pure MTA, a part of the normal SMTP-speaking Internet mail path.
When you hit a wall in development—when you find yourself hard put to think past the next patch—it's often time to ask not whether you've got the right answer, but whether you're asking the right question. Perhaps the problem needs to be reframed.
Well, I had reframed my problem. Clearly, the right thing to do was (1) hack SMTP forwarding support into the generic driver, (2) make it the default mode, and (3) eventually throw out all the other delivery modes, especially the deliver-to-file and deliver-to-standard-output options.
(当我在1997年的第一次Perl大会上发言的时候,黑客大亨Larry Wall正坐在前排上。当我讲到上面的那句话的时候,他喊了起来,宗教复活式的口吻,“说出来,说出来,哥们!”全场都笑了,因为他们知道这一点对Perl的发明者也不例外。)
在我发扬这种精神把项目运行了几个星期以后,我开始得到类似的赞扬——不仅来自我的用户们,而且来自于其他有所耳闻的人们。我把一些邮件收藏了起来;要是什么时候我开始疑惑我生命的意义的时候,我就拿出来再看看:-)。
但是有两条基本的、非政治性的经验对各种设计都适用。
12)最有突破和创新的方案常常来自于意识到你把问题的模型弄错了。
当我继续把popclient开发成一个带有七七八八的本地递送模式的MTA/MDA时,我就是试图在解决错误的问题。Fetchmail的设计应该作为一个纯粹的MTA——正常的SMTP互联网邮件传输路径的一部分——来重起炉灶。
当你在开发中碰到死胡同时——当你绞尽脑汁要超越下一个补丁的时候——一般来讲你该问的不是你的答案对不对,而是你的问题对不对。或许你的问题需要重新定义。
嗯……我重新定义了我的问题。显然,正确的路子是(1)把SMTP转发支持加入到通用驱动里去,(2)把它设置为默认模式,(3)最终把其它的传递模式都去掉,尤其是传递到文件和标准输出的模式。
I hesitated over step 3 for some time, fearing to upset long-time popclient users dependent on the alternate delivery mechanisms. In theory, they could immediately switch to .forward files or their non-sendmail equivalents to get the same effects. In practice the transition might have been messy.
But when I did it, the benefits proved huge. The cruftiest parts of the driver code vanished. Configuration got radically simpler—no more grovelling around for the system MDA and user's mailbox, no more worries about whether the underlying OS supports file locking.
Also, the only way to lose mail vanished. If you specified delivery to a file and the disk got full, your mail got lost. This can't happen with SMTP forwarding because your SMTP listener won't return OK unless the message can be delivered or at least spooled for later delivery.
Also, performance improved (though not so you'd notice it in a single run). Another not insignificant benefit of this change was that the manual page got a lot simpler.
Later, I had to bring delivery via a user-specified local MDA back in order to allow handling of some obscure situations involving dynamic SLIP. But I found a much simpler way to do it.
The moral? Don't hesitate to throw away superannuated features when you can do it without loss of effectiveness. Antoine de Saint-Exupéry (who was an aviator and aircraft designer when he wasn't authoring classic children's books) said:
13. ``Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away.''
这第三步让我犹豫了一段时间,担心会影响那些依赖于另类模式的popclient的老用户。理论上,他们可以马上用.forward文件或sendmail的替代程序来获得相同的效果。在实践中,这种转换可能会让人头大。
但是一旦我这样做了,好处非常明显。驱动代码中毛病最多的地方消失了。配置容易了太多——再也不需要围着系统MDA和用户信箱打转,再也不需要担心背后的操作系统是否支持文件锁定。
而且,唯一可能丢失邮件的途径不见了。如果你指定递送到文件而磁盘满了的话,你的邮件就丢了。这在SMTP转发中不会发生,因为除非邮件成功传送或至少缓存了,SMTP的聆听端不会给以确认。
而且,性能提高了(尽管不是你单独运行一次就能感觉到的)。改变后另外一个不是非同小可的好处是说明手册简化了许多。
后来,我为了对付一些涉及到动态SLIP(Serial Line Internet Protocol,串行线互联网协议)的晦涩情形,曾经不得以把用户指定的本地MDA递送功能加回来。但是我找到了一个简单的多的办法来做它。
说明了什么道理?在不损失效率的情况下,不要犹豫把过了气的功能扔掉。埃克絮佩利*(他不在写作经典儿童图书的时候是个飞行员和飞机设计师)曾说过:
13)“设计达到完美的时候,不是增加得不能再增加了、而是减少得不能再减少了”。
*译注:Antoine de Saint-Exupéry,《小王子》的作者
When your code is getting both better and simpler, that is when you know it's right. And in the process, the fetchmail design acquired an identity of its own, different from the ancestral popclient.
It was time for the name change. The new design looked much more like a dual of sendmail than the old popclient had; both are MTAs, but where sendmail pushes then delivers, the new popclient pulls then delivers. So, two months off the blocks, I renamed it fetchmail.
There is a more general lesson in this story about how SMTP delivery came to fetchmail. It is not only debugging that is parallelizable; development and (to a perhaps surprising extent) exploration of design space is, too. When your development mode is rapidly iterative, development and enhancement may become special cases of debugging—fixing `bugs of omission' in the original capabilities or concept of the software.
Even at a higher level of design, it can be very valuable to have lots of co-developers random-walking through the design space near your product. Consider the way a puddle of water finds a drain, or better yet how ants find food: exploration essentially by diffusion, followed by exploitation mediated by a scalable communication mechanism. This works very well; as with Harry Hochheiser and me, one of your outriders may well find a huge win nearby that you were just a little too close-focused to see.
当你的代码变得既优良又简单的时候,这时你知道它上了正轨了。在这个过程中,fetchmail的设计获得了它自己的特色,脱离了上一代的popclient。
到了该换名字的时候了。新的设计和老的popclient相比,更像是一个sendmail的对手;二者都是MTA,但sendmail是发出去再投递,新的popclient是接过来再投递。所以在动工两个月后,我把它重命名为fetchmail。
在这个SMTP转发功能如何进入fetchmail的故事里,有一个更普遍的经验。那是不仅调试是可并行的;开发和搜索设计空间(在可能令人吃惊的程度上)也是。当你的开发模式在快速循环中,开发和改进有可能成为调试的特例——修正软件模型的原始设计中的“不足的问题”。
即使在高一层次的设计上,有许多共同开发者在你的产品的设计空间附近随机行走可以是很有价值的。想象一下一摊水怎样发现下水口的,或者更恰当一点,蚂蚁怎 样发现食物的:本质上以分散来搜索,然后以一个可扩展的通讯机制来利用。这一点很管用;就像浩赫海斯和我一样,你们随行中的一个很可能发现一个宝藏——你 只不过太专注了一点而看不到。
Fetchmail Grows Up
There I was with a neat and innovative design, code that I knew worked well because I used it every day, and a burgeoning beta list. It gradually dawned on me that I was no longer engaged in a trivial personal hack that might happen to be useful to few other people. I had my hands on a program that every hacker with a Unix box and a SLIP/PPP mail connection really needs.
With the SMTP forwarding feature, it pulled far enough in front of the competition to potentially become a ``category killer'', one of those classic programs that fills its niche so competently that the alternatives are not just discarded but almost forgotten.
I think you can't really aim or plan for a result like this. You have to get pulled into it by design ideas so powerful that afterward the results just seem inevitable, natural, even foreordained. The only way to try for ideas like that is by having lots of ideas—or by having the engineering judgment to take other peoples' good ideas beyond where the originators thought they could go.
Andy Tanenbaum had the original idea to build a simple native Unix for IBM PCs, for use as a teaching tool (he called it Minix). Linus Torvalds pushed the Minix concept further than Andrew probably thought it could go—and it grew into something wonderful. In the same way (though on a smaller scale), I took some ideas by Carl Harris and Harry Hochheiser and pushed them hard. Neither of us was `original' in the romantic way people think is genius. But then, most science and engineering and software development isn't done by original genius, hacker mythology to the contrary.
Fetchmail长大了
现在我有了一个整洁新颖的设计;我知道代码工作良好因为我天天都在使用;beta测试名单繁荣热闹。我慢慢明白了我不再是在作一个可能对几个人有用的琐碎的个人编程。我在主持一个所有拥有Unix机器和SLIP/PPP邮件接口的用户都需要的程序。
带有SMTP转发的功能的fetchmail在竞争对手面前表现强劲,潜质上可能成为一个“类型杀手”——那种在它的功能类型里如此称职以至于对手们不仅被放弃了而且几乎被遗忘了。
我觉得这种结果是有点可遇而不可求的。你必须要有强大的设计构想,能把你整个吸引进去,而产出的结果就像是不可避免的、天然的、甚至命中注定的。追求这种构想的唯一方法就是要有很多想法——或者有工程眼光能够把其他人的好主意推进到他们都想不到的地步。
安迪·塔内保(Andy Tanenbaum)有了一个原始主意,作一个针对IBM兼容机的简单Unix,作为教学工具来使用(他称之为Minix)。林纳斯·托瓦兹把Minix的概 念推进到了安迪可能想都想不到的地步——演变成了一个奇妙的东西。与此类似(然而在小一些的规模上),我从卡尔·哈里斯和哈利·浩赫海斯那里借来主意并努力 推进它们。我们都没有人们对天才的浪漫想象中的那种“原创性”。但是说回来,多数科学、技术和软件开发都不是由原创的天才完成的,而是相反,来自于黑客一 派。
The results were pretty heady stuff all the same—in fact, just the kind of success every hacker lives for! And they meant I would have to set my standards even higher. To make fetchmail as good as I now saw it could be, I'd have to write not just for my own needs, but also include and support features necessary to others but outside my orbit. And do that while keeping the program simple and robust.
The first and overwhelmingly most important feature I wrote after realizing this was multidrop support—the ability to fetch mail from mailboxes that had accumulated all mail for a group of users, and then route each piece of mail to its individual recipients.
I decided to add the multidrop support partly because some users were clamoring for it, but mostly because I thought it would shake bugs out of the single-drop code by forcing me to deal with addressing in full generality. And so it proved. Getting RFC 822 address parsing right took me a remarkably long time, not because any individual piece of it is hard but because it involved a pile of interdependent and fussy details.
But multidrop addressing turned out to be an excellent design decision as well. Here's how I knew:
14. Any tool should be useful in the expected way, but a truly great tool lends itself to uses you never expected.
The unexpected use for multidrop fetchmail is to run mailing lists with the list kept, and alias expansion done, on the client side of the Internet connection. This means someone running a personal machine through an ISP account can manage a mailing list without continuing access to the ISP's alias files.
Another important change demanded by my beta-testers was support for 8-bit MIME (Multipurpose Internet Mail Extensions) operation. This was pretty easy to do, because I had been careful
结果都一致是那种很眩目的东西——事实上,正是每一个黑客毕生追求的那种成功!而且这意味着我将不得不把我的标准设得更高。为了让fetchmail达到 我这时预期的水平,我必须不仅为自己的需要编程,而且要包括和支持他人必需的然而在我的轨道之外的功能。这样做的同时要保持程序简单结实。
意识到这点之后,我所写的第一个也是绝对最重要的一个功能是集体收发——从一群用户的集体信箱里把累积的所有邮件取来,然后把每一封分发给单独的收信人。
我决定添加集体收发功能部分上是因为用户们吵着要,然而主要是因为我觉得它会迫使我在完全普遍的条件下处理地址解析问题,从而甩掉单信发送模式中的臭虫。 结果如我所愿。我花了奇长的时间才把RFC 822的地址解析搞定,不是因为它的哪一部分很难,而是因为它涉及了一堆相互关联的烦人的细节。
然而集体收发也成为了一项优秀的设计决定。我是这样知道的:
14)任何一个工具都应该达到预期的用处,但是一个真正棒的工具会带来你从来预期不到的用处。
fetchmail中集体收发的不曾预期的功能是邮件列表可以在网络连接的客户端保存列表和别名扩展。这样,使用个人机通过ISP上网的一个人不必持续访问ISP方的别名扩展文件就可以管理一个邮件列表。
我的beta测试员们要求的另一个重要变化是支持8位的MIME操作。这个很容易,因为我已经很小心的保持了8
to keep the code 8-bit clean (that is, to not press the 8th bit, unused in the ASCII character set, into service to carry information within the program). Not because I anticipated the demand for this feature, but rather in obedience to another rule:
15. When writing gateway software of any kind, take pains to disturb the data stream as little as possible—and never throw away information unless the recipient forces you to!
Had I not obeyed this rule, 8-bit MIME support would have been difficult and buggy. As it was, all I had to do is read the MIME standard (RFC 1652) and add a trivial bit of header-generation logic.
Some European users bugged me into adding an option to limit the number of messages retrieved per session (so they can control costs from their expensive phone networks). I resisted this for a long time, and I'm still not entirely happy about it. But if you're writing for the world, you have to listen to your customers—this doesn't change just because they're not paying you in money.
A Few More Lessons from Fetchmail
Before we go back to general software-engineering issues, there are a couple more specific lessons from the fetchmail experience to ponder. Nontechnical readers can safely skip this section.
The rc (control) file syntax includes optional `noise' keywords that are entirely ignored by the parser. The English-like syntax they allow is considerably more readable than the traditional terse keyword-value pairs you get when you strip them all out.
位代码的洁净(就是,没有强迫ASCII字符集中没有使用的第8位比特去携带程序中的信息)。不是因为我预料到了这个功能要求,而是遵循了另一个规则:
15)在写任何关口软件的时候,花点功夫尽可能不要干扰数据流——除非用户强迫你,永远不要扔掉任何信息!
要是我没有遵守这个规则,8位MIME支持会很困难且毛病不断。事实上,我所需要做的仅仅是读一下MIME标准(RFC 1652),添加一条小小的文件头生成规则。
在一些欧洲的用户的要求下,我添加了一个选项来限制每次连接能下载的邮件数目(这样他们可以控制他们昂贵的电话费)。我对这件事抵制了很长一段时间,直到现在也不是完全满意。但是如果你给外边的世界写程序,你不得不聆听你的顾客——就算他们不付你钱也是这个道理。
Fetchmail带来的其它几条经验
在我们回到普遍的软件工程问题之前,fetchmail的经历中还有几条经验值得细想。非技术性的读者可以安全地跳开这一节。
rc控制文件的语法中包括了一些完全不解析的、可选的“噪音”关键词。它们所允许的类似英语的语法比起你把它们全部梳理掉之后所剩下的传统的简洁的“关键词-对应值”要可读得多。
These started out as a late-night experiment when I noticed how much the rc file declarations were beginning to resemble an imperative minilanguage. (This is also why I changed the original popclient ``server'' keyword to ``poll'').
It seemed to me that trying to make that imperative minilanguage more like English might make it easier to use. Now, although I'm a convinced partisan of the ``make it a language'' school of design as exemplified by Emacs and HTML and many database engines, I am not normally a big fan of ``English-like'' syntaxes.
Traditionally programmers have tended to favor control syntaxes that are very precise and compact and have no redundancy at all. This is a cultural legacy from when computing resources were expensive, so parsing stages had to be as cheap and simple as possible. English, with about 50% redundancy, looked like a very inappropriate model then.
This is not my reason for normally avoiding English-like syntaxes; I mention it here only to demolish it. With cheap cycles and core, terseness should not be an end in itself. Nowadays it's more important for a language to be convenient for humans than to be cheap for the computer.
There remain, however, good reasons to be wary. One is the complexity cost of the parsing stage—you don't want to raise that to the point where it's a significant source of bugs and user confusion in itself. Another is that trying to make a language syntax English-like often demands that the ``English'' it speaks be bent seriously out of shape, so much so that the superficial resemblance to natural language is as confusing as a traditional syntax would have been. (You see this bad effect in a lot of so-called ``fourth generation'' and commercial database-query
这些开始于一个深夜实验——当我注意到rc文件的定义开始看起来多么像一个微型的指令语言。(这也是我为什么把popclient原有的“server”关键词换成了“poll”。)
在我来看,努力把这个微型指令语言做得更像英语可能会使它更容易使用。尽管我现在是一个“把它做成一门语言”设计流派——就像Emacs和HTML和许多数据库引擎展示的那样——的信徒,我一般不是特别热衷于“类似英语的”语法。
传统上,程序员们倾向于选用简洁紧凑、完全没有冗余的控制语法。这是计算资源昂贵的时期的文化遗留,那时解析过程不得不尽可能的廉价和简单。那时,大概有50%冗余的英语看起来像是一个非常不合适的模型。
这不是我一般避免英语式语法的原因;我在这儿提起它正是为了打破这个看法。有了便宜的循环和核心,简洁不应该为了简洁而简洁。现在一门语言对于人的方便比对于计算机的廉价更重要。
然而我们还有需要小心的原因。其中之一是解析过程的复杂性成本——你不想把它提高到富产臭虫和困惑用户的程度。另外,试图把一门语言的语法做得像英 语经常迫使它所使用的“英语”严重扭曲变形,以至于对自然语言的表面模仿变得像传统语法一样令人困惑。(你可以在许多所谓的“第四代”和商业数据库查询语 言中看到这个坏效果。)
languages.)
The fetchmail control syntax seems to avoid these problems because the language domain is extremely restricted. It's nowhere near a general-purpose language; the things it says simply are not very complicated, so there's little potential for confusion in moving mentally between a tiny subset of English and the actual control language. I think there may be a broader lesson here:
16. When your language is nowhere near Turing-complete, syntactic sugar can be your friend.
Another lesson is about security by obscurity. Some fetchmail users asked me to change the software to store passwords encrypted in the rc file, so snoopers wouldn't be able to casually see them.
I didn't do it, because this doesn't actually add protection. Anyone who's acquired permissions to read your rc file will be able to run fetchmail as you anyway—and if it's your password they're after, they'd be able to rip the necessary decoder out of the fetchmail code itself to get it.
All .fetchmailrc password encryption would have done is give a false sense of security to people who don't think very hard. The general rule here is:
17. A security system is only as secure as its secret. Beware of pseudo-secrets.
fetchmail的控制语法似乎避免了这些问题,因为它的语言空间极端有限。它和一个普通用途的语言更本接不上边儿;它所说的事情压根儿不复杂,所以在英语的一个微小子集里和实际上的控制语言之间进行脑力转换而发生迷惑的可能性很小。我觉得这里可能有一个更普适的经验:
16)当你的语言离图灵穷尽还差得远的时候,,给语法加点风味可以有帮助。
另一条经验是关于隐藏带来的安全性。一些fetchmail的用户要求我改一下软件来把加密的密码储存在rc控制文件里,这样入侵者就不会在无意中看到它们。
我没有照办,因为这实际上并不会添加保护。不管怎样,任何一个取得了权限来读你的rc文件的人都可以像你一样来运行fetchmail——如果他们真的来找你的密码,他们可以从fetchmail代码本身中剥离出必要的解码器来得手。
所有.fetchmail密码加密能做的是给那些不怎么用心思考的人一种虚假的安全感。这里的一般规则是:
17)一个安全系统的安全性取决于它保守的秘密的安全性。小心伪秘密。
Necessary Preconditions for the Bazaar Style
Early reviewers and test audiences for this essay consistently raised questions about the preconditions for successful bazaar-style development, including both the qualifications of the project leader and the state of code at the time one goes public and starts to try to build a co-developer community.
It's fairly clear that one cannot code from the ground up in bazaar style [IN]. One can test, debug and improve in bazaar style, but it would be very hard to originate a project in bazaar mode. Linus didn't try it. I didn't either. Your nascent developer community needs to have something runnable and testable to play with.
When you start community-building, what you need to be able to present is a plausible promise. Your program doesn't have to work particularly well. It can be crude, buggy, incomplete, and poorly documented. What it must not fail to do is (a) run, and (b) convince potential co-developers that it can be evolved into something really neat in the foreseeable future.
Linux and fetchmail both went public with strong, attractive basic designs. Many people thinking about the bazaar model as I have presented it have correctly considered this critical, then jumped from that to the conclusion that a high degree of design intuition and cleverness in the project leader is indispensable.
But Linus got his design from Unix. I got mine initially from the ancestral popclient (though it would later change a great deal, much more proportionately speaking than has Linux). So does the leader/coordinator for a bazaar-style effort really have to have exceptional design talent, or can he get by through leveraging the design talent of others?
市集风格的必要前提
这篇文章的早期审阅者和试验听众们持续就成功的市集风格开发的前提提出问题,包括项目领导人的素质和他开放项目和开始搭建合作者社区时的程序代码状态。
很显然的,在市集风格里你不能从零开始编程。你可以在市集风格里检测、调试和提高,但是以市集模式孕育一个项目会是很困难的。林纳斯没有这样试过。我也没有。你的新生的开发者社区需要能运行和测试的东西来展示身手。
当你开始社区建设的时候,你需要能够呈现一个可行的前景。你的程序不一定要工作的非常好。它可以是粗糙的、问题多多的、不完整的、缺少文档记录的。它一定不能失败的是(1)能运行,(2)说服潜在的合作者它可以在可预见的将来进化成真正漂亮的东西。
Linux和fetchmail开放的时候都带有强劲、吸引人的基本设计。许多像我的描述里那样来思考市集模式的人正确地认为这一点很关键;于是进而断定在项目领导人身上,高度的设计灵感和聪明不可或缺。
但是林纳斯的设计来自于Unix。我的最初来自于先前的popclient(尽管它后来变化很大,按比例来说比Linux的大的多)。那么一个市集风格的项目的领导人/主持人真的一定要有杰出的设计天才,还是他可以四两拨千斤、通过带动他人的设计才能来述职?
I think it is not critical that the coordinator be able to originate designs of exceptional brilliance, but it is absolutely critical that the coordinator be able to recognize good design ideas from others.
Both the Linux and fetchmail projects show evidence of this. Linus, while not (as previously discussed) a spectacularly original designer, has displayed a powerful knack for recognizing good design and integrating it into the Linux kernel. And I have already described how the single most powerful design idea in fetchmail (SMTP forwarding) came from somebody else.
Early audiences of this essay complimented me by suggesting that I am prone to undervalue design originality in bazaar projects because I have a lot of it myself, and therefore take it for granted. There may be some truth to this; design (as opposed to coding or debugging) is certainly my strongest skill.
But the problem with being clever and original in software design is that it gets to be a habit—you start reflexively making things cute and complicated when you should be keeping them robust and simple. I have had projects crash on me because I made this mistake, but I managed to avoid this with fetchmail.
So I believe the fetchmail project succeeded partly because I restrained my tendency to be clever; this argues (at least) against design originality being essential for successful bazaar projects. And consider Linux. Suppose Linus Torvalds had been trying to pull off fundamental innovations in operating system design during the development; does it seem at all likely that the resulting kernel would be as stable and successful as what we have?
A certain base level of design and coding skill is required, of course, but I expect almost anybody seriously thinking of launching a bazaar effort will already be above that minimum. The
我认为主持的人能否想出杰出灿烂的设计不是很关键,但绝对关键的是,主持的人能够慧眼识别出他人的优秀设计想法。
Linux和fetchmail的项目都显示了这方面的证据。林纳斯,(像前面讨论过的)尽管不是一个特别有原创性的设计者,却展现了识别优秀设计并把它 集成到Linux内核里强大才能。我也已经描述了在fetchmail里的最有力的一个设计思想(SMTP转发)怎样来自于另一个人。
这篇文章的早期听众捧我的场说我容易低估市集项目里的设计原创性的价值,因为我自己有很多,因而就想当然的习惯了。这话大概有一点点的真实性在里面;设计(而不是编码或调试)确实是我的强项。
但是在软件设计里表现聪明和创造力的问题在于它形成一种坏习惯——当你应该保持事情稳固和简单的时候,你开始放任地把它们搞的好玩和复杂。我曾经因为犯了这个错误把项目搞砸过,但是我在fetchmail里做到了避开这个错误。
所以我相信fetchmail项目的成功部分上是因为我克制住了自作聪明的习惯;这(至少)反驳了设计的原创性是成功的市集项目的关键。而且想一下 Linux。假设林纳斯·托瓦兹在开发中试图整出操作系统设计的根本性创新;作出来的内核压根儿可能会像我们现在的这么稳定和成功吗?
当然一定的设计和编码技能的基本水准还是必要的,但是我预期几乎每个认真考虑发起一个市集型项目的人已经超
open-source community's internal market in reputation exerts subtle pressure on people not to launch development efforts they're not competent to follow through on. So far this seems to have worked pretty well.
There is another kind of skill not normally associated with software development which I think is as important as design cleverness to bazaar projects—and it may be more important. A bazaar project coordinator or leader must have good people and communications skills.
This should be obvious. In order to build a development community, you need to attract people, interest them in what you're doing, and keep them happy about the amount of work they're doing. Technical sizzle will go a long way towards accomplishing this, but it's far from the whole story. The personality you project matters, too.
It is not a coincidence that Linus is a nice guy who makes people like him and want to help him. It's not a coincidence that I'm an energetic extrovert who enjoys working a crowd and has some of the delivery and instincts of a stand-up comic. To make the bazaar model work, it helps enormously if you have at least a little skill at charming people.
出了这个基本要求。开源社区内部的声望机制给人们一种微妙的压力:[如果你没有几把刷子,]不要发起自己不能胜任的开发项目。迄今为止,这一点似乎工作得很有效。
另外有一种和软件开发一般无关的技能,我认为对于市集型项目来讲,和设计才能一样的重要——甚至可能更重要。一个市集项目的主持人或领导者必须有良好的人际、交流技能。
这应该是显而易见的。要建设一个开发社区,你需要吸引人群,让他们对你做的事情感兴趣,并且让他们对自个儿的工作量舒心。要做到这一点,巧妙的手段会起很大的作用,但远远不是故事的全部。你所展现的人格也很重要。
林纳斯是一个平易的人,让人们喜欢他、想帮助他——这不是巧合。我是个活泼外向的人,喜欢和人群打交道,有着一些现场喜剧演员的直觉和本事——这不是巧合。要使市集模式运行起来,你至少有一点点让人们喜欢你的本领是非常重要的。
The Social Context of Open-Source Software
It is truly written: the best hacks start out as personal solutions to the author's everyday problems, and spread because the problem turns out to be typical for a large class of users. This takes us back to the matter of rule 1, restated in a perhaps more useful way:
18. To solve an interesting problem, start by finding a problem that is interesting to you.
So it was with Carl Harris and the ancestral popclient, and so with me and fetchmail. But this has been understood for a long time. The interesting point, the point that the histories of Linux and fetchmail seem to demand we focus on, is the next stage—the evolution of software in the presence of a large and active community of users and co-developers.
In The Mythical Man-Month, Fred Brooks observed that programmer time is not fungible; adding developers to a late software project makes it later. As we've seen previously, he argued that the complexity and communication costs of a project rise with the square of the number of developers, while work done only rises linearly. Brooks's Law has been widely regarded as a truism. But we've examined in this essay an number of ways in which the process of open-source development falsifies the assumptionms behind it—and, empirically, if Brooks's Law were the whole picture Linux would be impossible.
Gerald Weinberg's classic The Psychology of Computer Programming supplied what, in hindsight, we can see as a vital correction to Brooks. In his discussion of ``egoless programming'', Weinberg observed that in shops where developers are not territorial about their code, and encourage other people to look for
开源软件的社会语境
这句话写到了实处:最好的程序开始于作者日常问题的个人解决方案,因为一大批人正好都有这个问题而流行。这把我们带回了第一条经验的内容,用一种或许更有用的方式来表达是:
18)要解决一个有意思的问题,首先找到一个你觉得有意思的问题。
卡尔·哈里斯和先前的popclient是这样的,我和fetchmail也是这样的。但是这点大家已经明白很久了。有意义的一点是,Linux和fetchmail的历史看来要求我们关心的一点是,下一个阶段————在用户和合作者形成了庞大活跃的社区时的软件的进化。
在《人月神话》中,弗里德·布洛克表述了程序员的时间是不能线性叠加的;添加开发人员的做法只能使得已经延期的软件项目更为延期。像我们前面提到 的,他论述了项目的复杂程度和通讯成本按开发人员数目的平方增加,而业绩仅以直线增加。布洛克法则被广泛的认同为真理。但在这篇文章里,我们已经探讨了开 源的开发过程破除它背后的预设的几种途径——而且事实证明,如果布洛克法则统领一切,Linux就不可能发生。
以事后之明看来,杰拉德·委恩伯格的经典《计算机编程的心理学》提供了一个对布洛克的关键修正。在他对“无私编程”的讨论里,委恩伯格注意到一些地 方的开发人员不对自己的源码画地为牢,而是鼓励他人在其中寻找错误和改
bugs and potential improvements in it, improvement happens dramatically faster than elsewhere. (Recently, Kent Beck's `extreme programming' technique of deploying coders in pairs looking over one anothers' shoulders might be seen as an attempt to force this effect.)
Weinberg's choice of terminology has perhaps prevented his analysis from gaining the acceptance it deserved—one has to smile at the thought of describing Internet hackers as ``egoless''. But I think his argument looks more compelling today than ever.
The bazaar method, by harnessing the full power of the ``egoless programming'' effect, strongly mitigates the effect of Brooks's Law. The principle behind Brooks's Law is not repealed, but given a large developer population and cheap communications its effects can be swamped by competing nonlinearities that are not otherwise visible. This resembles the relationship between Newtonian and Einsteinian physics—the older system is still valid at low energies, but if you push mass and velocity high enough you get surprises like nuclear explosions or Linux.
The history of Unix should have prepared us for what we're learning from Linux (and what I've verified experimentally on a smaller scale by deliberately copying Linus's methods [EGCS]). That is, while coding remains an essentially solitary activity, the really great hacks come from harnessing the attention and brainpower of entire communities. The developer who uses only his or her own brain in a closed project is going to fall behind the developer who knows how to create an open, evolutionary context in which feedback exploring the design space, code contributions, bug-spotting, and other improvements come from from hundreds (perhaps thousands) of people.
进的余地——在这些地方,改进进展得比别处明显快很多。(最近,肯特·贝克的“极度编程”技术——把编程者配对让他们互相关照——或许可以看作是强制这一效果的尝试。)
委恩伯格在用词上的选择可能阻碍了他的分析获得应有的认可——把网络黑客形容成不讲个人英雄主义的一群,光是这样的念头就足以令人莞尔。但是我认为他的争论在今天看起来比以往任何时间更让人信服。
市集模式,通过借助“无私编程”效果的极致动力,强烈抵制了布洛克法则的效果。布洛克法则背后的原理没有被推翻,但是有了一个庞大的开发者群体和廉 价的通讯,它的效果可以被否则不可见的对立的非线性因素所淹没。这就像牛顿式的和爱因斯坦式的物理之间的关系——旧的系统在低能量下仍然有效,但当你把质 量和速度变得足够大的时候,你就得到了如同核爆炸或Linux那样的惊奇。
Unix的历史应该使得我们对研究Linux的结果(和我在小规模上有意拷贝林纳斯的方法所实验确认的结果)有了心理准备。这是说,虽 然编程基本上仍是一种个人封闭的活动,真正高超的程序来自于借助整个社区的注意力和脑力。一个在封闭的项目中只使用自己脑力的开发者,将会输给一个知道怎 样创造一个开放的、进化式的环境——从中吸收成千或上万人的探索设计空间的反馈、编码贡献、臭虫检测和其它的改进——的开发者。
But the traditional Unix world was prevented from pushing this approach to the ultimate by several factors. One was the legal contraints of various licenses, trade secrets, and commercial interests. Another (in hindsight) was that the Internet wasn't yet good enough.
Before cheap Internet, there were some geographically compact communities where the culture encouraged Weinberg's ``egoless'' programming, and a developer could easily attract a lot of skilled kibitzers and co-developers. Bell Labs, the MIT AI and LCS labs, UC Berkeley—these became the home of innovations that are legendary and still potent.
Linux was the first project for which a conscious and successful effort to use the entire world as its talent pool was made. I don't think it's a coincidence that the gestation period of Linux coincided with the birth of the World Wide Web, and that Linux left its infancy during the same period in 1993–1994 that saw the takeoff of the ISP industry and the explosion of mainstream interest in the Internet. Linus was the first person who learned how to play by the new rules that pervasive Internet access made possible.
While cheap Internet was a necessary condition for the Linux model to evolve, I think it was not by itself a sufficient condition. Another vital factor was the development of a leadership style and set of cooperative customs that could allow developers to attract co-developers and get maximum leverage out of the medium.
But what is this leadership style and what are these customs? They cannot be based on power relationships—and even if they could be, leadership by coercion would not produce the results we see. Weinberg quotes the autobiography of the 19th-century Russian anarchist Pyotr Alexeyvich Kropotkin's Memoirs of a
但是有几个因素阻止了传统的Unix世界把这个方法发挥到极致。一个是各种执照许可、贸易秘密和商业利益的法律限制。另一个(回头来看)是互联网还不够好。
在便宜的互联网之前有过一些地域性的紧密团体,在文化上鼓励委恩伯格的“无私编程”、一个开发者可以容易地吸引到一批有水平的“军师”和合作者。贝尔实验室、麻省理工的人工智能和计算机实验室、伯克利加州大学——这些成为了传奇性的和依然强劲的发明的家园。
Linux是第一个作了有意识的、成功的努力来把全世界当作智囊库使用的项目。我不认为Linux的孕育期与互联网的诞生重叠是一个巧合, Linux在1993-1994之间网络服务业起步和对互联网的主流兴趣的爆发的同期脱离了它的婴儿时代也不是巧合。林纳斯是第一个学会按照普及的互联网 连接所促成的新规则来运作的人。
虽然便宜的互联网是Linux模式进化出来的必要条件,我觉得它自己不是充分条件。另一个关键的因素是一种领导风格和一套合作制度的建立——使得开发者可以吸引合作者、在这个媒介中获取最大程度的收益。
但什么是这种领导风格、什么是这些制度呢?它们不可能是基于权力关系的——就算是可能,强制性的领导不会产生我们所看到的成果。在这个题目上,委恩 伯格很恰当的引用了19世纪俄罗斯无政府主义者Pyotr Alexeyvich
Revolutionist to good effect on this subject:
Having been brought up in a serf-owner's family, I entered active life, like all young men of my time, with a great deal of confidence in the necessity of commanding, ordering, scolding, punishing and the like. But when, at an early stage, I had to manage serious enterprises and to deal with [free] men, and when each mistake would lead at once to heavy consequences, I began to appreciate the difference between acting on the principle of command and discipline and acting on the principle of common understanding. The former works admirably in a military parade, but it is worth nothing where real life is concerned, and the aim can be achieved only through the severe effort of many converging wills.
The ``severe effort of many converging wills'' is precisely what a project like Linux requires—and the ``principle of command'' is effectively impossible to apply among volunteers in the anarchist's paradise we call the Internet. To operate and compete effectively, hackers who want to lead collaborative projects have to learn how to recruit and energize effective communities of interest in the mode vaguely suggested by Kropotkin's ``principle of understanding''. They must learn to use Linus's Law.[SP]
Earlier I referred to the ``Delphi effect'' as a possible explanation for Linus's Law. But more powerful analogies to adaptive systems in biology and economics also irresistably suggest themselves. The Linux world behaves in many respects like a free market or an ecology, a collection of selfish agents attempting to maximize utility which in the process produces a self-correcting spontaneous order more elaborate and efficient than any amount of central planning could have achieved. Here, then, is the place to seek the ``principle of understanding''.
Kropotkin的自传《一个革命者的回忆录》:
成长于一个农奴主的家庭,我进入社会后,像我那个时候所有的年轻人一样,很是相信领导、命令、训斥、惩罚等等的必要性。但是在早期我不得不管理重要 的事业和对付[自由的]人们的时候,在每个错误都会立刻导致严重后果的时候,我开始领悟到按指令和纪律的原则行事与按共同理解的原则行事之间的区别。前者 在阅兵式中运行得令人崇敬,然而就真实的生活而言,它却一文不值;而且目标只有通过许多共同意志的竭诚努力才能实现。
这个“许多共同意志的竭诚努力”正是Linux这种项目所要求的;在这个我们叫作互联网的无政府主义者的天堂里,对志愿者们行使“指令的原则”事实 上是不可能的。要有效的运作和竞争,想要领导协作性项目的黑客们不得不学会怎样按照Kropotkin的“共同理解的原则”里模糊地提出的模式、招募和激 励活动中的相关社区。他们必须学会使用林纳斯法则。
我早先引用了“神庙效应”作为林纳斯法则的一个可行的解释。但是与生物和经济中的可适应系统的更有力的类比不可避免地自我呈现出来。Linux世界 在很多方面都表现的类似自由市场或生态系统:由自私的成员组成的一个试图把功效最大化的集合,在这个过程中形成了一个自我纠正的自发秩序——它比不管多少 中央计划有可能达到的成就都更广泛和有效。那么,这里就是寻找“共同理解的原则”的地方。
The ``utility function'' Linux hackers are maximizing is not classically economic, but is the intangible of their own ego satisfaction and reputation among other hackers. (One may call their motivation ``altruistic'', but this ignores the fact that altruism is itself a form of ego satisfaction for the altruist). Voluntary cultures that work this way are not actually uncommon; one other in which I have long participated is science fiction fandom, which unlike hackerdom has long explicitly recognized ``egoboo'' (ego-boosting, or the enhancement of one's reputation among other fans) as the basic drive behind volunteer activity.
Linus, by successfully positioning himself as the gatekeeper of a project in which the development is mostly done by others, and nurturing interest in the project until it became self-sustaining, has shown an acute grasp of Kropotkin's ``principle of shared understanding''. This quasi-economic view of the Linux world enables us to see how that understanding is applied.
We may view Linus's method as a way to create an efficient market in ``egoboo''—to connect the selfishness of individual hackers as firmly as possible to difficult ends that can only be achieved by sustained cooperation. With the fetchmail project I have shown (albeit on a smaller scale) that his methods can be duplicated with good results. Perhaps I have even done it a bit more consciously and systematically than he.
Many people (especially those who politically distrust free markets) would expect a culture of self-directed egoists to be fragmented, territorial, wasteful, secretive, and hostile. But this expectation is clearly falsified by (to give just one example) the stunning variety, quality, and depth of Linux documentation. It is a hallowed given that programmers hate documenting; how is it, then, that Linux hackers generate so much documentation? Evidently Linux's free market in egoboo works better to produce
Linux黑客们最大化的这个“功效方程”不是经典经济学上的,而是他们自我的满足和在其他黑客中的声望这些摸不到的东西。(有人或许可以把他们的 动机叫作“利他性的”,但这忽略了利他主义自身对利他主义者就是一种自我满足的形式这一事实。)这样运作的自愿者文化其实不是不寻常的;我已经长期参与的 另一个就是科幻粉丝族。不像黑客族,他们早就明白的认识到了“egoboo”(ego-boosting,或在其他粉丝中增强个人的声望)是志愿者活动背 后的基本动力。
林纳斯,通过成功的把自己定位为一个主要由其他人来开发的项目的看门人、培养对这个项目的兴趣直到它可以自我维持,表现了对Kropotkin“共同理解的原则”的敏锐把握。这个对Linux世界的类经济学视角使得我们能够看到这种理解是如何应用的。
我们可以把林纳斯的方法看作创造有效的“自我彰显”的市场的一条路子——把单个黑客的自我体现尽可能紧密的连接在困难的、只有通过持续合作才能达到 的目标上。通过fetchmail项目,我展现了(尽管在小规模上)他的方法可以复制、产生出好的结果。或许我甚至做得比他更有意识和更系统一点。
很多人(尤其是在政治上不相信自由市场的那些)会以为自我驱动的个人英雄主义者们形成的文化会是支离破碎、占山为王、低效浪费、秘密诡异和充满敌意的。但 是这个设想显然被(只举一个例子)Linux文档的惊人的广度、质量和深度所证伪。程序员痛恨写文档是众所周知的;那么,Linux黑客们是怎样生产出这 么多文档的呢?显然林纳斯的
virtuous, other-directed behavior than the massively-funded documentation shops of commercial software producers.
Both the fetchmail and Linux kernel projects show that by properly rewarding the egos of many other hackers, a strong developer/coordinator can use the Internet to capture the benefits of having lots of co-developers without having a project collapse into a chaotic mess. So to Brooks's Law I counter-propose the following:
19: Provided the development coordinator has a communications medium at least as good as the Internet, and knows how to lead without coercion, many heads are inevitably better than one.
I think the future of open-source software will increasingly belong to people who know how to play Linus's game, people who leave behind the cathedral and embrace the bazaar. This is not to say that individual vision and brilliance will no longer matter; rather, I think that the cutting edge of open-source software will belong to people who start from individual vision and brilliance, then amplify it through the effective construction of voluntary communities of interest.
Perhaps this is not only the future of open-source software. No closed-source developer can match the pool of talent the Linux community can bring to bear on a problem. Very few could afford even to hire the more than 200 (1999: 600, 2000: 800) people who have contributed to fetchmail!
Perhaps in the end the open-source culture will triumph not because cooperation is morally right or software ``hoarding'' is morally wrong (assuming you believe the latter, which neither Linus nor I do), but simply because the closed-source world cannot win an evolutionary arms race with open-source
自我彰显的自由市场在产出优良的、协同的行为上优于商业软件厂商的大笔资金驱动的文档工厂。
fetchmail项目和Linux内核项目都表明,通过适当的表彰众多其他黑客的egos,一个有力的开发者/协调者可以使用互联网来收获拥有许多合作者的好处,而不至于让项目陷入嘈杂的混乱。所以针对布洛克法则,我反过来建议下面的一条:
19)如果开发的协调者有一个至少和互联网一样好的通讯媒介,而且懂得如何不通过强迫来领导,多个头脑不可避免地优于单个头脑。
我认为开源软件的未来会更多的属于那些懂得如何运行林纳斯的游戏的人们,告别大教堂来拥抱市集的人们。这不是说个人的远见和才华不再重要;而是,就我看来开源软件的前沿会属于那些开始于个人的远见和才华、然后通过有效的建设相关志愿者社区来扩展放大的人们。
或许这不仅是开源软件的未来。要对付一个问题,没有闭源的开发者可以比得上Linux社区所能驱动的才能之众。极少有人甚至能雇得起那些对fetchmail作出了贡献的200(1999: 600, 2000: 800)多人!
开源文化会最终胜利,或许不是因为合作在道德上正确或软件“劳役”在道德上错误(假设你相信后者,林纳斯和我都不),而只是因为开源社区可以在一个问题上投入多几个数量级的技术工时、闭源世界无法赢得这场进化式的军备竞争。
communities that can put orders of magnitude more skilled time into a problem.
On Management and the Maginot Line
The original Cathedral and Bazaar paper of 1997 ended with the vision above—that of happy networked hordes of programmer/anarchists outcompeting and overwhelming the hierarchical world of conventional closed software.
A good many skeptics weren't convinced, however; and the questions they raise deserve a fair engagement. Most of the objections to the bazaar argument come down to the claim that its proponents have underestimated the productivity-multiplying effect of conventional management.
Traditionally-minded software-development managers often object that the casualness with which project groups form and change and dissolve in the open-source world negates a significant part of the apparent advantage of numbers that the open-source community has over any single closed-source developer. They would observe that in software development it is really sustained effort over time and the degree to which customers can expect continuing investment in the product that matters, not just how many people have thrown a bone in the pot and left it to simmer.
There is something to this argument, to be sure; in fact, I have developed the idea that expected future service value is the key to the economics of software production in the essay The Magic Cauldron.
关于管理和马其诺防线
原始的1997年的《大教堂和市集》论文以以上的预见结束——程序员/无政府主义者的幸福网络群体胜出并压倒了传统闭源软件的阶层化世界。
然而,很多怀疑者并不信服;他们提出的问题也值得一场像样的论战。多数对市集模式的反对归结到一个观点:它的鼓吹者们低估了传统管理促进生产率的效果。
老脑筋的软件开发经理经常指责开源世界里项目群体形成-转变-消亡的随意性大大抵消了开源社区对单个闭源开发者在数目上的显然优越性。他们会指出在软件开 发里,真正重要的是长时间里不懈的努力和多大程度上顾客可以预期对产品的持续投资,而不仅仅是多少人往锅里扔一块骨头让它炖着。
没有疑问,这条争论有料;实际上,我在“The Magic Cauldron”一文中就已经展示了预期的未来服务价值是软件产业经济的关键的观点。
But this argument also has a major hidden problem; its implicit assumption that open-source development cannot deliver such sustained effort. In fact, there have been open-source projects that maintained a coherent direction and an effective maintainer community over quite long periods of time without the kinds of incentive structures or institutional controls that conventional management finds essential. The development of the GNU Emacs editor is an extreme and instructive example; it has absorbed the efforts of hundreds of contributors over 15 years into a unified architectural vision, despite high turnover and the fact that only one person (its author) has been continuously active during all that time. No closed-source editor has ever matched this longevity record.
This suggests a reason for questioning the advantages of conventionally-managed software development that is independent of the rest of the arguments over cathedral vs. bazaar mode. If it's possible for GNU Emacs to express a consistent architectural vision over 15 years, or for an operating system like Linux to do the same over 8 years of rapidly changing hardware and platform technology; and if (as is indeed the case) there have been many well-architected open-source projects of more than 5 years duration -- then we are entitled to wonder what, if anything, the tremendous overhead of conventionally-managed development is actually buying us.
Whatever it is certainly doesn't include reliable execution by deadline, or on budget, or to all features of the specification; it's a rare `managed' project that meets even one of these goals, let alone all three. It also does not appear to be ability to adapt to changes in technology and economic context during the project lifetime, either; the open-source community has proven far more effective on that score (as one can readily verify, for example, by
但是这条争论也有一个主要的潜在问题:它的暗中假设是开源开发不能形成持续的努力。事实上,有的开源项目在很长的时间段里保持了一致的方向和有效的维护团 体,而不需要传统管理上觉得关键的那些奖掖性的结构或制度性的控制。GNU Emacs编辑器的开发是一个极端的、说明问题的例子:它在超出15年的时间里吸取了成百上千贡献者的劳动、形成了一个统一的结构设计,尽管人事变换频 繁、一贯持续下来的人只有一个(它的作者)。没有闭源的编辑器或曾比得上这个长寿记录。
这建议了一个质疑传统管理模式下软件开发的优势的理由,与其它关于大教堂和市集模式的争议不相干的理由。如果GNU Emacs可以在15年里表述一个一致的框架设计,或者一个像Linux的操作系统在8年多里快速变化的硬件和平台技术中作到了同一点;如果(事实的确如 此)存在许多设计优良的开源项目超出了5年的历史——那么我们有权发问传统管理开发的庞大开销给我们买来了什么——如果有的话。
不管是什么,它显然不包括按期限、或预算、或所有指定功能的可靠执行;能满足这些目标中仅仅一条就是一个罕见的“管理好”的项目了,更不用说全部三 条了。它看来也不包括在项目进行过程中适应技术和经济环境变化的能力;在这上面,开源社区证明了远远更为有效(来简单确认这
comparing the 30-year history of the Internet with the short half-lives of proprietary networking technologies—or the cost of the 16-bit to 32-bit transition in Microsoft Windows with the nearly effortless upward migration of Linux during the same period, not only along the Intel line of development but to more than a dozen other hardware platforms, including the 64-bit Alpha as well).
One thing many people think the traditional mode buys you is somebody to hold legally liable and potentially recover compensation from if the project goes wrong. But this is an illusion; most software licenses are written to disclaim even warranty of merchantability, let alone performance—and cases of successful recovery for software nonperformance are vanishingly rare. Even if they were common, feeling comforted by having somebody to sue would be missing the point. You didn't want to be in a lawsuit; you wanted working software.
So what is all that management overhead buying?
In order to understand that, we need to understand what software development managers believe they do. A woman I know who seems to be very good at this job says software project management has five functions:
To define goals and keep everybody pointed in the same direction
To monitor and make sure crucial details don't get skipped
To motivate people to do boring but necessary drudgework
To organize the deployment of people for best productivity
To marshal resources needed to sustain the project
点,比方说,可以比较互联网 30年的历史和私有网络技术短短的半衰期;或者比较微软视窗从16位转换为32位的成本和Linux同一时期几乎毫不费力的升级——不仅是围绕英特尔系列 的开发,而且也扩展到包括64位Alpha芯片的十多个其他硬件平台上)。
很多人觉得从传统模式中购买到的一件东西是有人负法律责任、如果事情出了问题可能找赔偿。但这是一个幻觉;大多数的软件协议是写了来免除甚至商品化 的保证,更不用说性能了——而且从软件性能问题上成功索赔的案例少得近于虚幻。即使这类事很平常,因为有人来打官司而觉得心安是搞错了要点。您不想打官 司;您要的是能工作的软件。
那么这些管理开销都买了什么呢?
要理解这个问题,我们需要理解管理软件开发的经理们是怎么想的。我认识的一个似乎很优秀的女经理说软件项目管理有五项功能:
明确目标并保持大家向同一个方向努力
监测并确保关键的细节不被漏掉
动员人们做枯燥但是必要的苦力活儿
组织人员分配来达到最佳生产力
监护项目持续所需要的资源
Apparently worthy goals, all of these; but under the open-source model, and in its surrounding social context, they can begin to seem strangely irrelevant. We'll take them in reverse order.
My friend reports that a lot of resource marshalling is basically defensive; once you have your people and machines and office space, you have to defend them from peer managers competing for the same resources, and from higher-ups trying to allocate the most efficient use of a limited pool.
But open-source developers are volunteers, self-selected for both interest and ability to contribute to the projects they work on (and this remains generally true even when they are being paid a salary to hack open source.) The volunteer ethos tends to take care of the `attack' side of resource-marshalling automatically; people bring their own resources to the table. And there is little or no need for a manager to `play defense' in the conventional sense.
Anyway, in a world of cheap PCs and fast Internet links, we find pretty consistently that the only really limiting resource is skilled attention. Open-source projects, when they founder, essentially never do so for want of machines or links or office space; they die only when the developers themselves lose interest.
That being the case, it's doubly important that open-source hackers organize themselves for maximum productivity by self-selection—and the social milieu selects ruthlessly for competence. My friend, familiar with both the open-source world and large closed projects, believes that open source has been successful partly because its culture only accepts the most talented 5% or so of the programming population. She spends most of her time organizing the deployment of the other 95%, and has thus observed first-hand the well-known variance of a factor of one hundred in productivity between the most able programmers and
显然是有价值的目标,所有这些都是;但是在开源模式下,并且在它周围的社会语境中,这些会奇怪地开始显得不相干。我们来按逆序讨论这几条。
我的朋友报告说许多资源监护基本是防卫性的;一旦你有了你的人员机器和办公空间,你不得不防卫其他经理竞争相同的资源,和防卫上级调用有限资源中最高效的部分。
但是开源开发者是志愿的、在兴趣和对所参与项目的贡献能力上自我挑选的(甚至在他们领了薪水为开源项目编码的情况下这条也一般适用)。志愿者的特点会自动解决资源监护的“攻击方”;人们把自己的资源带到桌面上来。而且对管理者来说,没有或几乎没有必要来作传统意义上的“防卫”。
不管怎样,在一个便宜电脑和快速互联网连接的世界里,我们很一致的发现真正唯一的稀缺资源是有技术的努力。开源项目本质上从不会为了争夺机器或网络或办公空间而成立;它们只在开发者自己失掉兴趣的时候消亡。
在这点之外,加倍重要的是开源黑客们通过自我选择组织达到最大生产率——社会环境也无情的对能力作选择。我的朋友,对开源世界和大型闭源项目都熟 悉,认为开源的成功部分归功于它的文化只接受编程人员中最有才华的5%左右。她花费了她大多的时间来组织部署其他的95%,于是第一手观察到了那著名的、 最优秀的和仅仅及格的程序员之间一百倍的效率差别。
the merely competent.
The size of that variance has always raised an awkward question: would individual projects, and the field as a whole, be better off without more than 50% of the least able in it? Thoughtful managers have understood for a long time that if conventional software management's only function were to convert the least able from a net loss to a marginal win, the game might not be worth the candle.
The success of the open-source community sharpens this question considerably, by providing hard evidence that it is often cheaper and more effective to recruit self-selected volunteers from the Internet than it is to manage buildings full of people who would rather be doing something else.
Which brings us neatly to the question of motivation. An equivalent and often-heard way to state my friend's point is that traditional development management is a necessary compensation for poorly motivated programmers who would not otherwise turn out good work.
This answer usually travels with a claim that the open-source community can only be relied on only to do work that is `sexy' or technically sweet; anything else will be left undone (or done only poorly) unless it's churned out by money-motivated cubicle peons with managers cracking whips over them. I address the psychological and social reasons for being skeptical of this claim in Homesteading the Noosphere. For present purposes, however, I think it's more interesting to point out the implications of accepting it as true.
If the conventional, closed-source, heavily-managed style of software development is really defended only by a sort of Maginot Line of problems conducive to boredom, then it's going to remain
这个差别的巨大程度总是引发一个尴尬的问题:不管单个的项目还是整个行业,甩脱了那50%以上最差的是不是会更好一些?肯动脑的管理者很久以来就懂得,如果传统软件管理的唯一功能是把最差的一帮人从净损失转为微盈利,这事儿恐怕就不值得折腾。
开源社区的成功,通过提供硬性证据显示从互联网上招募自我选择的志愿者经常要比管理整栋楼的“身在曹营心在汉”的人们要便宜和有效的多,在相当程度上尖锐化了这个问题。
这恰好把我们带到驱动力的问题。一个同等的、常见的方式来提出我朋友的观点是:传统开发管理是对缺乏动力的程序员的必要补充,不然他们做不好工作。
这个回答一般携带着一个说法:只能指望开源社区做那种“眩目”的或技术上好玩的工作;任何其它的会被拉下(或敷衍其事)——除非金钱驱动的坐隔间的人在经 理们的鞭策下把它搅出来。我在“Homesteading the Noosphere”里解释对这个说法怀疑的心理学和社会学原因。然而就当前的目的,我觉得指出如果假定它是正确而衍生出的推论更有意思一些。
如果传统的、闭源的、冗肿管理下的软件开发真的只是在枯燥引发的问题造成的一条马其诺防线的守卫下,那么它
viable in each individual application area for only so long as nobody finds those problems really interesting and nobody else finds any way to route around them. Because the moment there is open-source competition for a `boring' piece of software, customers are going to know that it was finally tackled by someone who chose that problem to solve because of a fascination with the problem itself—which, in software as in other kinds of creative work, is a far more effective motivator than money alone.
Having a conventional management structure solely in order to motivate, then, is probably good tactics but bad strategy; a short-term win, but in the longer term a surer loss.
So far, conventional development management looks like a bad bet now against open source on two points (resource marshalling, organization), and like it's living on borrowed time with respect to a third (motivation). And the poor beleaguered conventional manager is not going to get any succour from the monitoring issue; the strongest argument the open-source community has is that decentralized peer review trumps all the conventional methods for trying to ensure that details don't get slipped.
Can we save defining goals as a justification for the overhead of conventional software project management? Perhaps; but to do so, we'll need good reason to believe that management committees and corporate roadmaps are more successful at defining worthy and widely shared goals than the project leaders and tribal elders who fill the analogous role in the open-source world.
That is on the face of it a pretty hard case to make. And it's not so much the open-source side of the balance (the longevity of Emacs, or Linus Torvalds's ability to mobilize hordes of developers with talk of ``world domination'') that makes it tough.
在每一个应用领域里的寿命就只有指望没有人发现 这些问题真正有意思、而且没有他人发现绕道而行的方法。因为一旦一种“枯燥”的软件出现了开源的竞争者,顾客们会知道终于有人因为关注这个问题本身而选择 来解决它了——这一点,对软件业像对其它任何的创造性工作一样,是比单独的金钱远为有效的驱动力。
仅仅为了驱动力来要一个传统的管理结构,或许就是一个好的计谋、坏的策略,短期的利益、长期的必然亏损。
说到这里,传统式开发管理和开源相比,现在在两点(资源监护,组织)上看起来不是明智之选,而且在第三点(驱动力)上朝不保夕。可怜的受困的传统项 目经理也不会在监测一项上得到任何帮助;开源社区的最强大的一个证据就是非集中化了的同行评议在试图保证细节不被忽略上胜出了所有的传统方法。
我们可以把目标的定义留下来作为接受传统软件项目管理的成本的理由吗?可能吧;但是这样,我们需要好的理由来相信管理委员会们和公司路线图比开源世界里的扮演类似角色的项目领导和部落长者们在定义有价值的、广泛共享的目标上更为成功。
就面上来看,这很难讲得通的。困难没有多少是来自于对峙的开源一方(Emacs的长寿,或林纳斯·托瓦兹以“统领世界”的说辞动员大群的开发者的能力)。而是来自于传
Rather, it's the demonstrated awfulness of conventional mechanisms for defining the goals of software projects.
One of the best-known folk theorems of software engineering is that 60% to 75% of conventional software projects either are never completed or are rejected by their intended users. If that range is anywhere near true (and I've never met a manager of any experience who disputes it) then more projects than not are being aimed at goals that are either (a) not realistically attainable, or (b) just plain wrong.
This, more than any other problem, is the reason that in today's software engineering world the very phrase ``management committee'' is likely to send chills down the hearer's spine—even (or perhaps especially) if the hearer is a manager. The days when only programmers griped about this pattern are long past; Dilbert cartoons hang over executives' desks now.
Our reply, then, to the traditional software development manager, is simple—if the open-source community has really underestimated the value of conventional management, why do so many of you display contempt for your own process?
Once again the example of the open-source community sharpens this question considerably—because we have fun doing what we do. Our creative play has been racking up technical, market-share, and mind-share successes at an astounding rate. We're proving not only that we can do better software, but that joy is an asset.
Two and a half years after the first version of this essay, the most radical thought I can offer to close with is no longer a vision of an open-source–dominated software world; that, after all, looks plausible to a lot of sober people in suits these days.
统机制在定义软件项目目标上表现出来的尴尬。
软件工程里一个最出名的大众定理是60%到75%的传统软件项目要么从没完成过,要么被目标用户否决了。要是这个范围真的和事实沾边(我从没有遇到过一个有经验的管理者否定过这点),那么占多数的项目都瞄向了(a)现实里达不到的或(b)错得离谱的目标。
在今天软件工程的世界里,“管理委员会”这个词会让听者背上直冒冷气——甚至(或者尤其)当听者是管理者的时候;上述这一点比其它任何的问题都是更主要的原因。那些只有程序员们咒骂这一现象的日子早已过去了;迪尔伯特的卡通*如今挂上了管理层的案头。
我们对传统软件开发经理的回答,那么就很简单——如果开源社区真的低估了传统管理的价值,为什么你们这么多人表现了对你们自己工作的轻蔑?
开源社区的例子再次把这个问题尖锐化了——因为我们做事乐在其中。我们创新的游戏已在技术、市场占有和观念的成功上以惊人的速率得分晋级。我们在证明不仅我们可以开发更好的软件,而且欢乐是一种宝贵财产。
在这篇文章第一版的两年半后,我能用来结尾的最激进的观点不再是一个开源统领的软件世界;那毕竟,在今天很多穿西服套装的人看来也是有可能的了。
*[译注]迪尔伯特是美国著名卡通系列的人物;该系列的主题是技术人员对管理层的揶揄。
Rather, I want to suggest what may be a wider lesson about software, (and probably about every kind of creative or professional work). Human beings generally take pleasure in a task when it falls in a sort of optimal-challenge zone; not so easy as to be boring, not too hard to achieve. A happy programmer is one who is neither underutilized nor weighed down with ill-formulated goals and stressful process friction. Enjoyment predicts efficiency.
Relating to your own work process with fear and loathing (even in the displaced, ironic way suggested by hanging up Dilbert cartoons) should therefore be regarded in itself as a sign that the process has failed. Joy, humor, and playfulness are indeed assets; it was not mainly for the alliteration that I wrote of "happy hordes" above, and it is no mere joke that the Linux mascot is a cuddly, neotenous penguin.
It may well turn out that one of the most important effects of open source's success will be to teach us that play is the most economically efficient mode of creative work..
相反,我想提出一个或许更广泛的、对软件业的教训,(或许也是对于任何一种创造性的或专业性的工作)。人们一般在一项任务处于一种适当难度范围的时 候享有乐趣;不要太简单了至于无聊,不要太难了不好实现。一个快乐程序员是一个既没有被浪费也没有被错误制定的目标和烦人过程摩擦所压倒的人。乐趣通往效 率。
以畏惧和厌恶来谈论你自己的工作过程(即使通过悬挂迪尔伯特卡通这种改头换面的讽刺性方式)因此本身应该被看作一个过程失败了的信号。欢乐、幽默,和趣味 是真正的财富;我在上面写的关于“快乐的一群”主要不是为了押韵,Linux的吉祥物是一个可亲的、稚气犹存的企鹅也不仅仅是玩笑。
结果很可能是,开源的成功带来的一个最重要的影响会是教育我们乐趣是创造性工作的经济上最有效的模式。
最新评论
2 周 2 天 前
7 周 1 天 前
10 周 3 天 前
20 周 1 天 前
20 周 3 天 前
21 周 5 天 前
24 周 6 天 前
32 周 5 天 前
33 周 3 天 前
33 周 5 天 前