From dillon@flea.best.net Sat Feb 15 16:12:43 1997 Return-Path: Received: from flea.best.net (root@flea.best.net [206.184.139.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA27176 for ; Sat, 15 Feb 1997 16:12:40 -0800 (PST) Received: (from dillon@localhost) by flea.best.net (8.8.5/8.8.3) id QAA13524; Sat, 15 Feb 1997 16:11:01 -0800 (PST) Date: Sat, 15 Feb 1997 16:11:01 -0800 (PST) From: Matt Dillon Message-Id: <199702160011.QAA13524@flea.best.net> To: auditors@freebsd.org Subject: WWW page up to organize auditor/reviewer assignments http://www.best.net:81/freebsd/ Enter in your key, select a password, and click on create. Then backup and click on the login button. For example, my key, as listed on Jordan's page, is 'md'. You can break out the ports/* and src/* trees to whatever depth you like, assign yourself as an auditor or reviewer, set the current status of your auditing, zoom in on a particular subtree, bookmark anything... and so on... Experiment a bit and tell me what you think. -Matt Matthew Dillon Engineering, BEST Internet Communications, Inc. [always include a portion of the original email in any response!] From dillon@flea.best.net Sat Feb 15 16:31:14 1997 Return-Path: Received: from flea.best.net (root@flea.best.net [206.184.139.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA28546 for ; Sat, 15 Feb 1997 16:31:13 -0800 (PST) Received: (from dillon@localhost) by flea.best.net (8.8.5/8.8.3) id QAA14101; Sat, 15 Feb 1997 16:29:31 -0800 (PST) Date: Sat, 15 Feb 1997 16:29:31 -0800 (PST) From: Matt Dillon Message-Id: <199702160029.QAA14101@flea.best.net> To: "Jordan K. Hubbard" Cc: Sergei Chechetkin , auditors@freebsd.org Subject: Re: Sign-up sheet generator :> Hello Jordan, I have written on perl early version of script that I :> think does it. I assumed that "audit-list", and "review-list" are comma :> separated lists of modules. It does not generate aliases list yet. :> But if it do job with tables right it will be easy to generate aliases. : :Eek, it looks like 3 people are working on this now! :-) : :However, I can also say that all 3 (including yourself) ran afoul of :an ambiguity in my wording and expected the email address to be a single :field, when in fact it's a quoted string with spaces in it so that you :can also represent the full name of the volunteer (as the :http://www.freebsd.org/auditors.html page currently does). :... Ahhh.. but mine is the best (!) It should be fully operational now. flea.best.net:81/freebsd/ -Matt From dillon@flea.best.net Sat Feb 15 16:32:48 1997 Return-Path: Received: from flea.best.net (root@flea.best.net [206.184.139.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA28852 for ; Sat, 15 Feb 1997 16:32:45 -0800 (PST) Received: (from dillon@localhost) by flea.best.net (8.8.5/8.8.3) id QAA14201; Sat, 15 Feb 1997 16:32:05 -0800 (PST) Date: Sat, 15 Feb 1997 16:32:05 -0800 (PST) From: Matt Dillon Message-Id: <199702160032.QAA14201@flea.best.net> To: auditors@freebsd.org Subject: Re: WWW page up to organize auditor/reviewer assignments Oops!! Sorry guys. Try: http://flea.best.net:81/freebsd/ Matthew Dillon Engineering, BEST Internet Communications, Inc. [always include a portion of the original email in any response!] From jkh@time.cdrom.com Sat Feb 15 16:38:48 1997 Return-Path: Received: from time.cdrom.com (time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA29191 for ; Sat, 15 Feb 1997 16:38:47 -0800 (PST) Received: from time.cdrom.com (localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id QAA01687; Sat, 15 Feb 1997 16:34:26 -0800 (PST) To: Matt Dillon cc: Sergei Chechetkin , auditors@freebsd.org Subject: Re: Sign-up sheet generator In-reply-to: Your message of "Sat, 15 Feb 1997 16:29:31 PST." <199702160029.QAA14101@flea.best.net> Date: Sat, 15 Feb 1997 16:34:20 -0800 Message-ID: <1669.856053260@time.cdrom.com> From: "Jordan K. Hubbard" > Ahhh.. but mine is the best (!) It should be fully operational now. > > flea.best.net:81/freebsd/ OK, this URL works fine. Um. It looks promising but still somewhat strange and difficult to operate at this early stage. I assume that more driving instructions and slightly more spacing between items is planned? The push/audit/review/clear labels are rather close together and there's no clear indication as to what it is they do, exactly. :-) Jordan From tenser@spitfire.ecsel.psu.edu Sat Feb 15 22:57:57 1997 Return-Path: Received: from spitfire.ecsel.psu.edu (spitfire.ecsel.psu.edu [146.186.218.51]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id WAA16456 for ; Sat, 15 Feb 1997 22:57:56 -0800 (PST) Received: (qmail 23883 invoked by uid 1000); 16 Feb 1997 06:57:10 -0000 Message-ID: <19970216065710.23881.qmail@spitfire.ecsel.psu.edu> To: "Jordan K. Hubbard" cc: auditors@freebsd.org Subject: Re: Sign-up sheet generator In-reply-to: Your message of "Sat, 15 Feb 1997 15:30:25 PST." <12446.856049425@time.cdrom.com> Date: Sun, 16 Feb 1997 01:57:09 -0500 From: Dan Cross > dc "Dan Cross " none none ^^^^ Ack! Nononono. :-) Mark Murray and I are splitting secure between us. He's going to handle the international stuff, and I'm going to do the domestic stuff, and we'll compare notes. It's not a big deal, just don't sell me short THIS early in the game. :-) :-) :-) - Dan C. From jkh@time.cdrom.com Sat Feb 15 23:15:54 1997 Return-Path: Received: from time.cdrom.com (time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id XAA17402 for ; Sat, 15 Feb 1997 23:15:54 -0800 (PST) Received: from time.cdrom.com (localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id XAA24637; Sat, 15 Feb 1997 23:15:43 -0800 (PST) To: Dan Cross cc: auditors@freebsd.org Subject: Re: Sign-up sheet generator In-reply-to: Your message of "Sun, 16 Feb 1997 01:57:09 EST." <19970216065710.23881.qmail@spitfire.ecsel.psu.edu> Date: Sat, 15 Feb 1997 23:15:43 -0800 Message-ID: <24633.856077343@time.cdrom.com> From: "Jordan K. Hubbard" > Ack! Nononono. :-) Mark Murray and I are splitting secure between us. > He's going to handle the international stuff, and I'm going to do the > domestic stuff, and we'll compare notes. It's not a big deal, just don't > sell me short THIS early in the game. :-) :-) :-) Sorry, I didn't mean to sell you short at all - we just must have slipped an email or so along the way since I didn't have any record of you as signed up for anything. :-) Changed. Jordan From tenser@spitfire.ecsel.psu.edu Sat Feb 15 23:25:51 1997 Return-Path: Received: from spitfire.ecsel.psu.edu (qmailr@spitfire.ecsel.psu.edu [146.186.218.51]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id XAA17896 for ; Sat, 15 Feb 1997 23:25:43 -0800 (PST) Received: (qmail 24465 invoked by uid 1000); 16 Feb 1997 07:25:41 -0000 Message-ID: <19970216072541.24464.qmail@spitfire.ecsel.psu.edu> To: "Jordan K. Hubbard" cc: auditors@freebsd.org Subject: Re: Sign-up sheet generator In-reply-to: Your message of "Sat, 15 Feb 1997 23:15:43 PST." <24633.856077343@time.cdrom.com> Date: Sun, 16 Feb 1997 02:25:41 -0500 From: Dan Cross > Sorry, I didn't mean to sell you short at all - we just must have > slipped an email or so along the way since I didn't have any record > of you as signed up for anything. :-) Haha, that's cool. I just didn't want anyone to think that I was THAT much of a slacker... :-) > Changed. Cool, thanks. - Dan C. From dillon@flea.best.net Sun Feb 16 01:49:00 1997 Return-Path: Received: from flea.best.net (root@flea.best.net [206.184.139.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id BAA21397 for ; Sun, 16 Feb 1997 01:49:00 -0800 (PST) Received: (from dillon@localhost) by flea.best.net (8.8.5/8.8.3) id BAA01053; Sun, 16 Feb 1997 01:46:43 -0800 (PST) Date: Sun, 16 Feb 1997 01:46:43 -0800 (PST) From: Matt Dillon Message-Id: <199702160946.BAA01053@flea.best.net> To: auditors@freebsd.org Subject: Re: WWW page up to organize auditor/reviewer assignments : Oops!! : : Sorry guys. Try: : : http://flea.best.net:81/freebsd/ : Sigh. Well that was fun... flea crashed with a page fault and then had a cascade failure due to two hard sector errors on /usr/home, plus fsck kept crashing it due to the quantum firmware bug. That was a nice little 8 hour exercise. NOT! So, my fancy smanshy auditor/reviewer assigner is off for good.. I can't risk crashing flea like that again. I'm going to have to track down that friggin mmap() bug once and for all. Sigh. -Matt Matthew Dillon Engineering, BEST Internet Communications, Inc. [always include a portion of the original email in any response!] From jkh@time.cdrom.com Sun Feb 16 02:03:06 1997 Return-Path: Received: from time.cdrom.com (time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id CAA21891 for ; Sun, 16 Feb 1997 02:03:00 -0800 (PST) Received: from time.cdrom.com (localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id CAA27165; Sun, 16 Feb 1997 02:02:12 -0800 (PST) To: Matt Dillon cc: auditors@freebsd.org Subject: Re: WWW page up to organize auditor/reviewer assignments In-reply-to: Your message of "Sun, 16 Feb 1997 01:46:43 PST." <199702160946.BAA01053@flea.best.net> Date: Sun, 16 Feb 1997 02:02:11 -0800 Message-ID: <27140.856087331@time.cdrom.com> From: "Jordan K. Hubbard" > So, my fancy smanshy auditor/reviewer assigner is off for > good.. I can't risk crashing flea like that again. I'm going > to have to track down that friggin mmap() bug once and for all. > > Sigh. Sorry to hear that. It certainly looked interesting. :) In any case, to get back then to the existing non-mmap() using auditors.sgml (:-), I think it'd be functional enough if just two features were added: 1. In the sign-up sheet, all categories and keys are links to the corresponding email addresses for easy access. 2. Clicking "sign up for something" should go to a more mundane CGI script and fill-in-the-blanks form which eventually results in an email message to me being sent in "auditor record" format for inclusion if everything looks legit. Of course, I still need some way of generating the aliases on freefall. If I weren't generating them by hand, I'd have my little script spit out something like this: auditor-ab: aaronb@j51.com auditor-ac: adrian@psinet.net.au ... auditors: auditor-ab, auditor-ac, ... audit-bin: auditor-ab audit-games: auditor-ac I'm still keeping the auditors datafile faithfully up to date, in hopes... :-) Jordan From csl@whale.sunbay.crimea.ua Sun Feb 16 05:28:33 1997 Return-Path: Received: from whale.sunbay.crimea.ua (sunbay-10BASE-T.cris.net [194.93.176.88]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id FAA04378 for ; Sun, 16 Feb 1997 05:27:55 -0800 (PST) Received: (from csl@localhost) by whale.sunbay.crimea.ua (8.8.3/1.6) id QAA04249; Sun, 16 Feb 1997 16:23:49 +0300 (MSK) From: Sergei Chechetkin Message-Id: <199702161323.QAA04249@whale.sunbay.crimea.ua> Subject: Re: Sign-up sheet generator To: dillon@best.net (Matt Dillon) Date: Sun, 16 Feb 1997 16:23:49 +0300 (MSK) Cc: jkh@time.cdrom.com, auditors@freebsd.org In-Reply-To: <199702160029.QAA14101@flea.best.net> from "Matt Dillon" at "Feb 15, 97 04:29:31 pm" Return-Receipt-To: csl@sunbay.crimea.ua X-Mailer: ELM [version 2.4ME+ PL22 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit According to Matt Dillon > :However, I can also say that all 3 (including yourself) ran afoul of > :an ambiguity in my wording and expected the email address to be a single > :field, when in fact it's a quoted string with spaces in it so that you > :can also represent the full name of the volunteer (as the > :http://www.freebsd.org/auditors.html page currently does). > :... > > Ahhh.. but mine is the best (!) It should be fully operational now. Agree. Looks fine. Since we don't need two different branches of the same project I would offer any help in your one. Did somebody done /etc/aliases ? Sergei Chechetkin From taob@risc.org Sun Feb 16 10:10:46 1997 Return-Path: Received: from alpha.risc.org (taob@trt-on9-34.netcom.ca [207.181.83.98]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA16651 for ; Sun, 16 Feb 1997 10:10:39 -0800 (PST) Received: from localhost (taob@localhost) by alpha.risc.org (8.8.4/8.8.4) with SMTP id NAA26102 for ; Sun, 16 Feb 1997 13:08:40 -0500 (EST) Date: Sun, 16 Feb 1997 13:08:34 -0500 (EST) From: Brian Tao To: auditors@freebsd.org Subject: Register pointers in gcc? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII I was looking at the disklabel(8) source today and found this in editit(): if (pid == 0) { register char *ed; sigsetmask(omask); setgid(getgid()); setuid(getuid()); if ((ed = getenv("EDITOR")) == (char *)0) ed = DEFEDITOR; execlp(ed, ed, tmpfil, 0); perror(ed); exit(1); } How come I can stuff anything into EDITOR and not cause disklabel to segfault? I've setenv EDITOR to 16K strings and it just complains that the argument list is too long, or that the editor does not exist. Where is it storing the contents of getenv("EDITOR")? -- Brian Tao (BT300, taob@risc.org) "Though this be madness, yet there is method in't" From pst@shockwave.com Sun Feb 16 10:35:28 1997 Return-Path: Received: from precipice.shockwave.com (ppp-206-170-5-89.rdcy01.pacbell.net [206.170.5.89]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA18402 for ; Sun, 16 Feb 1997 10:35:27 -0800 (PST) Received: from shockwave.com (localhost.shockwave.com [127.0.0.1]) by precipice.shockwave.com (8.8.4/8.7.3) with ESMTP id KAA01438; Sun, 16 Feb 1997 10:35:15 -0800 (PST) Message-Id: <199702161835.KAA01438@precipice.shockwave.com> To: Brian Tao cc: auditors@freebsd.org Subject: Re: Register pointers in gcc? In-reply-to: Your message of "Sun, 16 Feb 1997 13:08:34 EST." Date: Sun, 16 Feb 1997 10:35:15 -0800 From: Paul Traina From: Brian Tao Subject: Register pointers in gcc? I was looking at the disklabel(8) source today and found this in editit(): if (pid == 0) { register char *ed; sigsetmask(omask); setgid(getgid()); setuid(getuid()); if ((ed = getenv("EDITOR")) == (char *)0) ed = DEFEDITOR; execlp(ed, ed, tmpfil, 0); perror(ed); exit(1); } How come I can stuff anything into EDITOR and not cause disklabel to segfault? I've setenv EDITOR to 16K strings and it just complains that the argument list is too long, or that the editor does not exist. Where is it storing the contents of getenv("EDITOR")? It doesn't. The contents were already in the environment variable block as initialized by the shell. In this case, we're never keeping a separate copy, so there is no overflow bug here. From imp@village.org Sun Feb 16 10:38:59 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id KAA18880 for ; Sun, 16 Feb 1997 10:38:57 -0800 (PST) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 0.56 #1) id E0vwBU9-0005tj-00; Sun, 16 Feb 1997 11:38:49 -0700 To: Brian Tao Subject: Re: Register pointers in gcc? Cc: auditors@freebsd.org In-reply-to: Your message of "Sun, 16 Feb 1997 13:08:34 EST." References: Date: Sun, 16 Feb 1997 11:38:48 -0700 From: Warner Losh Message-Id: In message Brian Tao writes: : I was looking at the disklabel(8) source today and found this in : editit(): : if ((ed = getenv("EDITOR")) == (char *)0) : ed = DEFEDITOR; : How come I can stuff anything into EDITOR and not cause disklabel : to segfault? I've setenv EDITOR to 16K strings and it just complains : that the argument list is too long, or that the editor does not exist. : Where is it storing the contents of getenv("EDITOR")? getenv is stored in the environment space of the process, so it grows and shrinks dynamically. Since there are no strcpy's here, it works. I'm a little surprised that the kernel doesn't barf on the string being >> MAXPATHLEN, but that's a whole other set of bugs that I don't think have been well looked for, even in OpenBSD: Passing args to the kernel that are too big. I think that this turns into a NDINIT call in the kernel, but haven't traced it past that point yet. BTW, OpenBSD integrated some changes, appended below with change info, to disklabel that are likely desirable. There were some problems with signals that you might want to look into. Warner revision 1.27 date: 1997/02/16 07:42:52; author: deraadt; state: Exp; lines: +12 -14 when spawning editor child, use signal() instead of sigprocmask(SIG_BLOCK... this appears to prevent the intermediate shell from playing with the signals such that it gets a tty signal inside an editor such as emacs. this was very annoying Index: disklabel.c =================================================================== RCS file: /home/imp/OpenBSD/CVS/src/sbin/disklabel/disklabel.c,v retrieving revision 1.26 retrieving revision 1.27 diff -u -r1.26 -r1.27 --- disklabel.c 1996/12/13 16:58:25 1.26 +++ disklabel.c 1997/02/16 07:42:52 1.27 @@ -1,4 +1,4 @@ -/* $OpenBSD: disklabel.c,v 1.25 1996/12/07 10:09:24 deraadt Exp $ */ +/* $OpenBSD: disklabel.c,v 1.26 1996/12/13 16:58:25 millert Exp $ */ /* $NetBSD: disklabel.c,v 1.30 1996/03/14 19:49:24 ghudson Exp $ */ /* @@ -44,7 +44,7 @@ #endif /* not lint */ #ifndef lint -static char rcsid[] = "$OpenBSD: disklabel.c,v 1.25 1996/12/07 10:09:24 deraadt Exp $"; +static char rcsid[] = "$OpenBSD: disklabel.c,v 1.26 1996/12/13 16:58:25 millert Exp $"; #endif /* not lint */ #include @@ -960,7 +960,6 @@ int pid, xpid; int stat; extern char *getenv(); - sigset_t sigset, osigset; char *argp[] = {"sh", "-c", NULL, NULL}; char *ed, *p; @@ -974,24 +973,20 @@ sprintf(p, "%s %s", ed, tmpfil); argp[2] = p; - sigemptyset(&sigset); - sigaddset(&sigset, SIGINT); - sigaddset(&sigset, SIGQUIT); - sigaddset(&sigset, SIGHUP); - sigprocmask(SIG_BLOCK, &sigset, &osigset); + /* Turn off signals. */ + (void)signal(SIGHUP, SIG_IGN); + (void)signal(SIGINT, SIG_IGN); + (void)signal(SIGQUIT, SIG_IGN); while ((pid = fork()) < 0) { if (errno != EAGAIN) { - sigprocmask(SIG_SETMASK, &osigset, (sigset_t *)0); warn("fork"); free(p); - return (0); + stat = 1; + goto bail; } sleep(1); } if (pid == 0) { - sigprocmask(SIG_SETMASK, &osigset, (sigset_t *)0); - setgid(getgid()); - setuid(getuid()); execv(_PATH_BSHELL, argp); _exit(127); } @@ -1003,7 +998,10 @@ else if (WIFEXITED(stat)) break; } - sigprocmask(SIG_SETMASK, &osigset, (sigset_t *)0); +bail: + (void)signal(SIGHUP, SIG_DFL); + (void)signal(SIGINT, SIG_DFL); + (void)signal(SIGQUIT, SIG_DFL); return (!stat); } From taob@risc.org Sun Feb 16 10:45:23 1997 Return-Path: Received: from alpha.risc.org (taob@trt-on9-34.netcom.ca [207.181.83.98]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA19112 for ; Sun, 16 Feb 1997 10:45:16 -0800 (PST) Received: from localhost (taob@localhost) by alpha.risc.org (8.8.4/8.8.4) with SMTP id NAA26128; Sun, 16 Feb 1997 13:45:01 -0500 (EST) Date: Sun, 16 Feb 1997 13:45:00 -0500 (EST) From: Brian Tao To: Paul Traina cc: auditors@freebsd.org Subject: Re: Register pointers in gcc? In-Reply-To: <199702161835.KAA01438@precipice.shockwave.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Sun, 16 Feb 1997, Paul Traina wrote: > > It doesn't. The contents were already in the environment variable > block as initialized by the shell. In this case, we're never > keeping a separate copy, so there is no overflow bug here. *sigh*... sorry, stupid question. :( I was investigating environment variable overflows and looked for problems in the wrong place. -- Brian Tao (BT300, taob@risc.org) "Though this be madness, yet there is method in't" From eivind@dimaga.com Sun Feb 16 11:10:25 1997 Return-Path: Received: from nic.follonett.no (nic.follonett.no [194.198.43.10]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id LAA20465 for ; Sun, 16 Feb 1997 11:10:24 -0800 (PST) Received: (from uucp@localhost) by nic.follonett.no (8.8.5/8.8.3) with UUCP id UAA09130; Sun, 16 Feb 1997 20:08:34 +0100 (MET) Received: from oo7 (oo7.dimaga.com [192.0.0.65]) by dimaga.com (8.7.5/8.7.2) with SMTP id TAA10058; Sun, 16 Feb 1997 19:57:58 +0100 (MET) Message-Id: <3.0.32.19970216195758.00a54c30@dimaga.com> X-Sender: eivind@dimaga.com X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Sun, 16 Feb 1997 19:57:59 +0100 To: Aaron Bornstein From: Eivind Eklund Subject: Re: strncpy() and NULL termination Cc: auditors@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 04:12 PM 2/15/97 -0500, Aaron Bornstein wrote: > > As I've been going through code, I've noticed a lot of >strncpy()'s that don't properly NULL-terminate the strings. Seeing as >this is a security audit, I have a feeling that qiute a few new >strncpy()'s will be introduced into the tree, and I'd just like to remind >everyone that strncpy does NOT guarantee NULL termination. > >Bad: > strncpy(buf, somestring, BUFSIZ); > >Good: > strncpy(buf, somestring, BUFSIZ); > buf[BUFSIZ-1] = '\0'; Better: (IMHO - as this is more robust to code changes) strncpy(buf, somestring, sizeof(buf)); buf[sizeof(buf)-1] = '\0'; While we're at it, I'd like to remind everybody of strdup() - and suggest it might be an idea to introduce smprintf(), an sprintf to an automatically malloced buffer. These functions together make wirting correct and robust programs as easy as writing brittle programs without them. Eivind Eklund perhaps@yes.no http://maybe.yes.no/perhaps/ eivind@freebsd.org From eivind@dimaga.com Sun Feb 16 11:48:50 1997 Return-Path: Received: from nic.follonett.no (nic.follonett.no [194.198.43.10]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id LAA22372 for ; Sun, 16 Feb 1997 11:48:49 -0800 (PST) Received: (from uucp@localhost) by nic.follonett.no (8.8.5/8.8.3) with UUCP id UAA09554; Sun, 16 Feb 1997 20:47:15 +0100 (MET) Received: from oo7 (oo7.dimaga.com [192.0.0.65]) by dimaga.com (8.7.5/8.7.2) with SMTP id UAA10726; Sun, 16 Feb 1997 20:47:14 +0100 (MET) Message-Id: <3.0.32.19970216204713.00a00560@dimaga.com> X-Sender: eivind@dimaga.com X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Sun, 16 Feb 1997 20:47:14 +0100 To: Charles Henrich From: Eivind Eklund Subject: Re: strncpy() and NULL termination Cc: aaronb@j51.com, auditors@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 02:20 PM 2/16/97 -0500, Charles Henrich wrote: >> Better: (IMHO - as this is more robust to code changes) >> strncpy(buf, somestring, sizeof(buf)); >> buf[sizeof(buf)-1] = '\0'; > >And when buf is a pointer? :) Then you don't do that. That's hopefully fairly obvious. I'll admit it isn't quite that safe for that kind of code changes; OTOH, it usually prevent people from doing the obvious typos "oh, was that size for _that_ buffer? I didn't notice..." which come fairly often. Eivind Eklund perhaps@yes.no http://maybe.yes.no/perhaps/ eivind@freebsd.org From phk@critter.dk.tfs.com Sun Feb 16 11:52:18 1997 Return-Path: Received: from bofh.cybercity.dk (bofh.cybercity.dk [195.8.128.254]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id LAA22601 for ; Sun, 16 Feb 1997 11:52:16 -0800 (PST) Received: from critter.dk.tfs.com (phk.cybercity.dk [195.8.133.247]) by bofh.cybercity.dk (8.8.3/8.7.3) with ESMTP id UAA05669; Sun, 16 Feb 1997 20:54:45 +0100 (MET) Received: from critter.dk.tfs.com (localhost [127.0.0.1]) by critter.dk.tfs.com (8.8.2/8.8.2) with ESMTP id UAA16972; Sun, 16 Feb 1997 20:54:20 +0100 (MET) To: Eivind Eklund cc: Aaron Bornstein , auditors@freebsd.org Subject: Re: strncpy() and NULL termination In-reply-to: Your message of "Sun, 16 Feb 1997 19:57:59 +0100." <3.0.32.19970216195758.00a54c30@dimaga.com> Date: Sun, 16 Feb 1997 20:54:19 +0100 Message-ID: <16970.856122859@critter.dk.tfs.com> From: Poul-Henning Kamp In message <3.0.32.19970216195758.00a54c30@dimaga.com>, Eivind Eklund writes: >At 04:12 PM 2/15/97 -0500, Aaron Bornstein wrote: >> >> As I've been going through code, I've noticed a lot of >>strncpy()'s that don't properly NULL-terminate the strings. Seeing as >>this is a security audit, I have a feeling that qiute a few new >>strncpy()'s will be introduced into the tree, and I'd just like to remind >>everyone that strncpy does NOT guarantee NULL termination. >> >>Bad: >> strncpy(buf, somestring, BUFSIZ); >> >>Good: >> strncpy(buf, somestring, BUFSIZ); >> buf[BUFSIZ-1] = '\0'; > >Better: (IMHO - as this is more robust to code changes) > strncpy(buf, somestring, sizeof(buf)); > buf[sizeof(buf)-1] = '\0'; Excuse me for meddling. I would think that the return val from strncpy should be checked and if truncation happens the program should exit. right ? -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@tfs.com TRW Financial Systems, Inc. Power and ignorance is a disgusting cocktail. From imp@village.org Sun Feb 16 12:09:03 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id MAA23580 for ; Sun, 16 Feb 1997 12:09:00 -0800 (PST) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 0.56 #1) id E0vwCtK-00062U-00; Sun, 16 Feb 1997 13:08:54 -0700 To: Aaron Bornstein Subject: Re: strncpy() and NULL termination Cc: auditors@freebsd.org In-reply-to: Your message of "Sat, 15 Feb 1997 16:12:58 EST." References: Date: Sun, 16 Feb 1997 13:08:54 -0700 From: Warner Losh Message-Id: In message Aaron Bornstein writes: : As I've been going through code, I've noticed a lot of : strncpy()'s that don't properly NULL-terminate the strings. Seeing as : this is a security audit, I have a feeling that qiute a few new : strncpy()'s will be introduced into the tree, and I'd just like to remind : everyone that strncpy does NOT guarantee NULL termination. : : Bad: : strncpy(buf, somestring, BUFSIZ); : : Good: : strncpy(buf, somestring, BUFSIZ); : buf[BUFSIZ-1] = '\0'; : : Not having NULL-terminated strings is usually a Bad Thing. While : it's never a Good Idea to assume a string is NULL terminated, we can't : assume (especially those of us doing libc reviews) that everyone using : our code will be conscious enough to recognize this. I agree that NUL (note one L) termination is a desirable thing. Many times it would be better to check to see if the last character of the string is a NUL before continuing, and exiting (in a program) or returning an error condition if possible (in a library). I disagree with the silent truncation path that OpenBSD has taken on many of its fixes. Makes it too easy to play ../../../x/x/x/../../../etc/passwd/other/stuff type games (where /other/stuff gets silently truncated). I generally use sizeof(buf) because its size can change from BUFSIZ to BUFSIZ*2 from time to time. I then make double sure that buf is an array rather than a pointer. Warner From pst@shockwave.com Sun Feb 16 12:22:52 1997 Return-Path: Received: from precipice.shockwave.com (ppp-206-170-5-89.rdcy01.pacbell.net [206.170.5.89]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id MAA24471 for ; Sun, 16 Feb 1997 12:22:51 -0800 (PST) Received: from shockwave.com (localhost.shockwave.com [127.0.0.1]) by precipice.shockwave.com (8.8.4/8.7.3) with ESMTP id MAA02115; Sun, 16 Feb 1997 12:22:20 -0800 (PST) Message-Id: <199702162022.MAA02115@precipice.shockwave.com> To: Eivind Eklund cc: Aaron Bornstein , auditors@freebsd.org Subject: Re: strncpy() and NULL termination In-reply-to: Your message of "Sun, 16 Feb 1997 19:57:59 +0100." <3.0.32.19970216195758.00a54c30@dimaga.com> Date: Sun, 16 Feb 1997 12:22:20 -0800 From: Paul Traina Better: (IMHO - as this is more robust to code changes) strncpy(buf, somestring, sizeof(buf)); buf[sizeof(buf)-1] = '\0'; WARNING: Don't do this blindly! If buf is a pointer, you will store the first 4 characters in buf. Trust me, this has happened to me. I like sizeof, USE SIZEOF, but don't be stupid about it. I've been reluctant to introduce smprintf and sstrncpy() because they make portability to other POSIX os's painful. For now, let's not add these new features (I really want sstrncpy(), but it's not POSIX). From marcs@znep.com Sun Feb 16 12:53:13 1997 Return-Path: Received: from scanner.worldgate.com (scanner.worldgate.com [198.161.84.3]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id MAA26340 for ; Sun, 16 Feb 1997 12:53:05 -0800 (PST) Received: from znep.com (uucp@localhost) by scanner.worldgate.com (8.8.5/8.7.3) with UUCP id NAA11755; Sun, 16 Feb 1997 13:52:47 -0700 (MST) Received: from localhost (marcs@localhost) by alive.znep.com (8.7.5/8.7.3) with SMTP id NAA03880; Sun, 16 Feb 1997 13:45:29 -0700 (MST) Date: Sun, 16 Feb 1997 13:45:28 -0700 (MST) From: Marc Slemko To: Eivind Eklund cc: auditors@freebsd.org Subject: Re: strncpy() and NULL termination In-Reply-To: <3.0.32.19970216195758.00a54c30@dimaga.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Sun, 16 Feb 1997, Eivind Eklund wrote: > Better: (IMHO - as this is more robust to code changes) > strncpy(buf, somestring, sizeof(buf)); > buf[sizeof(buf)-1] = '\0'; Agreed, just be _sure_ buf isn't a pointer. > > While we're at it, I'd like to remind everybody of strdup() - and suggest > it might be an idea to introduce smprintf(), an sprintf to an automatically > malloced buffer. These functions together make wirting correct and robust > programs as easy as writing brittle programs without them. Just be careful modiying code where there are existing limits to introduce strings of possibly unlimited length without being very sure you traced the dataflow down to the very bottom of the program. A good number of potential overflows are avoided right now by some arbitrary limitation higher up in the code; remove that without fixing everything else and you have problems. Note that some of these dependencies may be in other programs that are executed. From eivind@dimaga.com Sun Feb 16 13:25:52 1997 Return-Path: Received: from nic.follonett.no (nic.follonett.no [194.198.43.10]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id NAA28324 for ; Sun, 16 Feb 1997 13:25:44 -0800 (PST) Received: (from uucp@localhost) by nic.follonett.no (8.8.5/8.8.3) with UUCP id WAA10532; Sun, 16 Feb 1997 22:23:37 +0100 (MET) Received: from oo7 (oo7.dimaga.com [192.0.0.65]) by dimaga.com (8.7.5/8.7.2) with SMTP id WAA11659; Sun, 16 Feb 1997 22:09:29 +0100 (MET) Message-Id: <3.0.32.19970216220929.00cbacb0@dimaga.com> X-Sender: eivind@dimaga.com X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Sun, 16 Feb 1997 22:09:30 +0100 To: Poul-Henning Kamp From: Eivind Eklund Subject: Re: strncpy() and NULL termination Cc: Aaron Bornstein , auditors@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 08:54 PM 2/16/97 +0100, Poul-Henning Kamp wrote: >In message <3.0.32.19970216195758.00a54c30@dimaga.com>, Eivind Eklund writes: >>Better: (IMHO - as this is more robust to code changes) >> strncpy(buf, somestring, sizeof(buf)); >> buf[sizeof(buf)-1] = '\0'; > >Excuse me for meddling. I would think that the return val from >strncpy should be checked and if truncation happens the program >should exit. > >right ? Absolutely right. My fault - I've read too many of the 'normal style' fixes. Eivind Eklund perhaps@yes.no http://maybe.yes.no/perhaps/ eivind@freebsd.org From dillon@flea.best.net Sun Feb 16 15:17:05 1997 Return-Path: Received: from flea.best.net (root@flea.best.net [206.184.139.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id PAA04335 for ; Sun, 16 Feb 1997 15:17:04 -0800 (PST) Received: (from dillon@localhost) by flea.best.net (8.8.5/8.8.3) id PAA12008; Sun, 16 Feb 1997 15:15:56 -0800 (PST) Date: Sun, 16 Feb 1997 15:15:56 -0800 (PST) From: Matt Dillon Message-Id: <199702162315.PAA12008@flea.best.net> To: auditors@freebsd.org Subject: bsdaudit reservation page back up ok, I *think* I've got it so it will not crash flea :-) If it does, then at least I'll have a crash dump this time. http://flea.best.net:81/freebsd/ The entire src/ and ports/ tree is listed. -Matt Matthew Dillon Engineering, BEST Internet Communications, Inc. [always include a portion of the original email in any response!] From jkh@time.cdrom.com Sun Feb 16 15:28:04 1997 Return-Path: Received: from time.cdrom.com (time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id PAA04848 for ; Sun, 16 Feb 1997 15:28:03 -0800 (PST) Received: from time.cdrom.com (localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id PAA15820; Sun, 16 Feb 1997 15:27:56 -0800 (PST) To: Matt Dillon cc: auditors@freebsd.org Subject: Re: bsdaudit reservation page back up In-reply-to: Your message of "Sun, 16 Feb 1997 15:15:56 PST." <199702162315.PAA12008@flea.best.net> Date: Sun, 16 Feb 1997 15:27:56 -0800 Message-ID: <15816.856135676@time.cdrom.com> From: "Jordan K. Hubbard" > ok, I *think* I've got it so it will not crash flea :-) If it does, > then at least I'll have a crash dump this time. > > http://flea.best.net:81/freebsd/ > > The entire src/ and ports/ tree is listed. OK, looking better. I'm still not all that sure what "push" does though. Anybody here want to donate some nice looking buttons to Matt so I also don't go blind trying to click on all that tiny text? :-) Jordan From dillon@flea.best.net Sun Feb 16 16:23:13 1997 Return-Path: Received: from flea.best.net (root@flea.best.net [206.184.139.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA09043 for ; Sun, 16 Feb 1997 16:23:13 -0800 (PST) Received: (from dillon@localhost) by flea.best.net (8.8.5/8.8.3) id QAA13507; Sun, 16 Feb 1997 16:22:33 -0800 (PST) Date: Sun, 16 Feb 1997 16:22:33 -0800 (PST) From: Matt Dillon Message-Id: <199702170022.QAA13507@flea.best.net> To: "Jordan K. Hubbard" Cc: auditors@freebsd.org Subject: Re: bsdaudit reservation page back up :> ok, I *think* I've got it so it will not crash flea :-) If it does, :> then at least I'll have a crash dump this time. :> :> http://flea.best.net:81/freebsd/ :> :> The entire src/ and ports/ tree is listed. : :OK, looking better. I'm still not all that sure what "push" does :though. : :Anybody here want to donate some nice looking buttons to Matt so :I also don't go blind trying to click on all that tiny text? :-) : : Jordan There are two ways to expand a subhierarchy... you can either click on the name of the directory (i.e. 'src', or 'cat' under 'src'), or you can push into the directory, making it root. Clicking on a directory name expands it, but your current 'root' remains the same... i.e. if you have opened up bin, you can click on 'cat' and 'chmod' and see both of those expanded all on one screen. But if you 'push' into cat, then your root becomes cat and you only see cat on your screen. 'push' is very useful in that it reduces the amount of material sent over the wire, making the system useful for people running over tacked up async modems. You can also bookmark pushes. The database remembers any expansions you do between sessions, which is also cool. I am working on personal email hotlinks.. I also want to mirror the current -current tree onto flea and have a 'read' hotlink so you can bring up (or download) individual source files, and I also want to have hotlinks to email auditors or the specific auditor sub-set alias. -Matt Matthew Dillon Engineering, BEST Internet Communications, Inc. [always include a portion of the original email in any response!] From tenser@spitfire.ecsel.psu.edu Sun Feb 16 16:28:29 1997 Return-Path: Received: from spitfire.ecsel.psu.edu (spitfire.ecsel.psu.edu [146.186.218.51]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA09250 for ; Sun, 16 Feb 1997 16:28:16 -0800 (PST) Received: (qmail 27340 invoked by uid 1000); 17 Feb 1997 00:27:33 -0000 Message-ID: <19970217002733.27339.qmail@spitfire.ecsel.psu.edu> To: Poul-Henning Kamp cc: Aaron Bornstein , auditors@freebsd.org Subject: Re: strncpy() and NULL termination In-reply-to: Your message of "Sun, 16 Feb 1997 20:54:19 +0100." <16970.856122859@critter.dk.tfs.com> Date: Sun, 16 Feb 1997 19:27:33 -0500 From: Dan Cross > Excuse me for meddling. I would think that the return val from > strncpy should be checked and if truncation happens the program > should exit. > > right ? I agree, but strncpy(3) doesn't return a value which is useful for this purpose. :-) - Dan C. (strncpy(3) only returns NULL or a valid pointer, but there is no indication of truncation at all, unless you check to see if buf[len - 1] != '\0'.) From marcs@znep.com Sun Feb 16 18:29:33 1997 Return-Path: Received: from scanner.worldgate.com (scanner.worldgate.com [198.161.84.3]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id SAA15007 for ; Sun, 16 Feb 1997 18:29:31 -0800 (PST) Received: from znep.com (uucp@localhost) by scanner.worldgate.com (8.8.5/8.7.3) with UUCP id TAA25659; Sun, 16 Feb 1997 19:29:15 -0700 (MST) Received: from localhost (marcs@localhost) by alive.znep.com (8.7.5/8.7.3) with SMTP id TAA06090; Sun, 16 Feb 1997 19:24:20 -0700 (MST) Date: Sun, 16 Feb 1997 19:24:19 -0700 (MST) From: Marc Slemko To: Dan Cross cc: Poul-Henning Kamp , auditors@freebsd.org Subject: Re: strncpy() and NULL termination In-Reply-To: <19970217002733.27339.qmail@spitfire.ecsel.psu.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Sun, 16 Feb 1997, Dan Cross wrote: > > Excuse me for meddling. I would think that the return val from > > strncpy should be checked and if truncation happens the program > > should exit. > > > > right ? > > I agree, but strncpy(3) doesn't return a value which is useful for > this purpose. :-) > > - Dan C. > > (strncpy(3) only returns NULL or a valid pointer, but there is > no indication of truncation at all, unless you check to see if > buf[len - 1] != '\0'.) If you do that then you need to set buf[len-1] = '\0' before the strncpy. So, anyone up for doing a standard routine for logging (syslog?) when a program aborts for this reason? No, you don't want to blindly do it everywhere but it could be useful in some places if done correctly. From tenser@spitfire.ecsel.psu.edu Sun Feb 16 18:51:56 1997 Return-Path: Received: from spitfire.ecsel.psu.edu (qmailr@spitfire.ecsel.psu.edu [146.186.218.51]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id SAA16129 for ; Sun, 16 Feb 1997 18:51:12 -0800 (PST) Received: (qmail 27927 invoked by uid 1000); 17 Feb 1997 02:51:04 -0000 Message-ID: <19970217025104.27926.qmail@spitfire.ecsel.psu.edu> To: Marc Slemko cc: Poul-Henning Kamp , auditors@freebsd.org Subject: Re: strncpy() and NULL termination In-reply-to: Your message of "Sun, 16 Feb 1997 19:24:19 MST." Date: Sun, 16 Feb 1997 21:51:04 -0500 From: Dan Cross > > (strncpy(3) only returns NULL or a valid pointer, but there is > > no indication of truncation at all, unless you check to see if > > buf[len - 1] != '\0'.) > > If you do that then you need to set buf[len-1] = '\0' before the strncpy. Eh? Why would you need to do that? strncpy(3) does nul padding... ;-) > So, anyone up for doing a standard routine for logging (syslog?) when a > program aborts for this reason? No, you don't want to blindly do it > everywhere but it could be useful in some places if done correctly. That's not a bad idea. btw, would folks find something like this useful? char * sstrtncpy(dst, src, len) char *dst; const char *src; size_t len; { register size_t last; if (len != 0) { (void)strncpy(dst, src, len); last = len - 1; if (dst[last] != '\0') { dst[last] = '\0'; return(NULL); } } return(dst); } The idea here is to check for truncation by looking at the last character in dst to see if it is non-nul. If that's true, then the string was trun- cated, so nul-terminate it and use the return as NULL to indicate this. If not, then the string was NOT truncated. The point is that the return value can now be used as an indication of truncation, and action can be taken acc- ordingly. The register variable for ``last'' is an attempt at an optimiz- ation, but wether or not this really GETS you anything is debatable. Of course, all the usual caveats regarding standards compliance (or rather, lack thereof) apply. - Dan C. From root@implode.root.com Sun Feb 16 20:04:18 1997 Return-Path: Received: from root.com (implode.root.com [198.145.90.17]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id UAA19787 for ; Sun, 16 Feb 1997 20:04:17 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by root.com (8.7.6/8.6.5) with SMTP id UAA04242; Sun, 16 Feb 1997 20:04:50 -0800 (PST) Message-Id: <199702170404.UAA04242@root.com> X-Authentication-Warning: implode.root.com: Host localhost [127.0.0.1] didn't use HELO protocol To: Warner Losh cc: Brian Tao , auditors@freebsd.org Subject: Re: Register pointers in gcc? In-reply-to: Your message of "Sun, 16 Feb 1997 11:38:48 MST." From: David Greenman Reply-To: dg@root.com Date: Sun, 16 Feb 1997 20:04:50 -0800 Sender: root@implode.root.com >I'm a little surprised that the kernel doesn't barf on the string >being >> MAXPATHLEN, but that's a whole other set of bugs that I don't It would be a bug if the kernel didn't allow args being > MAXPATHLEN. The only restriction is that the total length of all of the arguments plus environment must be less than ARG_MAX, which is currently 64K bytes. -DG David Greenman Core-team/Principal Architect, The FreeBSD Project From peter@spinner.DIALix.COM Sun Feb 16 21:24:15 1997 Return-Path: Received: from spinner.DIALix.COM (spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id VAA24869 for ; Sun, 16 Feb 1997 21:24:09 -0800 (PST) Received: from spinner.DIALix.COM (localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id NAA29874; Mon, 17 Feb 1997 13:23:18 +0800 (WST) Message-Id: <199702170523.NAA29874@spinner.DIALix.COM> X-Mailer: exmh version 2.0gamma 1/27/96 To: Paul Traina cc: Eivind Eklund , Aaron Bornstein , auditors@freebsd.org Subject: Re: strncpy() and NULL termination In-reply-to: Your message of "Sun, 16 Feb 1997 12:22:20 PST." <199702162022.MAA02115@precipice.shockwave.com> Date: Mon, 17 Feb 1997 13:23:17 +0800 From: Peter Wemm Paul Traina wrote: > I've been reluctant to introduce smprintf and sstrncpy() because they > make portability to other POSIX os's painful. For now, let's not add > these new features (I really want sstrncpy(), but it's not POSIX). Just as a BTW, smprintf() was commented out of the stdio man pages on 4.4Lite. It looks like it was intended to be implemented but ran out of time. Don't forget the pseudo-standard asprintf() and vasprintf(): char *buf; len = asprintf(&buf, "foo %s bar %d, x, y); The buffer is malloc()ed "bug enough" and returned via &buf. It must be manually free()d when finished. This was first implemented in gnu libc I think, and is in all linux libc's. If memory serves correctly, smprintf() was pretty much like asprintf(), except that it wasn't clear how it was supposed to work. From printf.3: .\" .Ft int .\" .Fn smprintf "const char *format" ... .\" .Ft int .\" .Fn vsmprintf "const char *format" "va_list ap" [..] .\" .I smprintf .\" and .\" .I vsmprintf .\" dynamically allocate a new string with .\" .IR malloc . [..] .\" Except for .\" .I smprintf .\" and .\" .IR vsmprintf , .\" all of these functions return .\" the number of characters printed [..] .\" .I Smprintf .\" and .\" .I vsmprintf .\" return a pointer to a string of an appropriate length; .\" this pointer should be passed to .\" .I free .\" to release the associated storage .\" when it is no longer needed. .\" If sufficient space is not avaliable, .\" .I smprintf .\" and .\" .I vsmprintf .\" will return .\" .SM .\" .BR The commented out description is internally inconsistant. It's listed as returning an int, but later described as returning a malloc'ed pointer. (This was killed in rev 1.4 of printf.3) Cheers, -Peter From henrich@crh.cl.msu.edu Sun Feb 16 23:09:04 1997 Return-Path: Received: from crh.cl.msu.edu (crh.cl.msu.edu [35.8.1.24]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id XAA28465 for ; Sun, 16 Feb 1997 23:09:02 -0800 (PST) Received: (from henrich@localhost) by crh.cl.msu.edu (8.8.5/8.8.4) id OAA02040; Sun, 16 Feb 1997 14:20:46 -0500 (EST) From: Charles Henrich Message-Id: <199702161920.OAA02040@crh.cl.msu.edu> Subject: Re: strncpy() and NULL termination To: eivind@dimaga.com (Eivind Eklund) Date: Sun, 16 Feb 1997 14:20:46 -0500 (EST) Cc: aaronb@j51.com, auditors@freebsd.org In-Reply-To: <3.0.32.19970216195758.00a54c30@dimaga.com> from Eivind Eklund at "Feb 16, 97 07:57:59 pm" X-Mailer: ELM [version 2.4ME+ PL22 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit > Better: (IMHO - as this is more robust to code changes) > strncpy(buf, somestring, sizeof(buf)); > buf[sizeof(buf)-1] = '\0'; And when buf is a pointer? :) -Crh Charles Henrich Michigan State University henrich@msu.edu http://pilot.msu.edu/~henrich From mollers.pad@sni.de Sun Feb 16 23:42:34 1997 Return-Path: Received: from nixpbe.pdb.sni.de (mail.sni.de [192.109.2.33]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id XAA29426 for ; Sun, 16 Feb 1997 23:42:33 -0800 (PST) Received: (from nerv@localhost) by nixpbe.pdb.sni.de (8.6.12/8.6.12) id IAA08566 for auditors@freebsd.org; Mon, 17 Feb 1997 08:41:02 +0100 Message-Id: <199702170741.IAA08566@nixpbe.pdb.sni.de> Subject: W: Assignment To: auditors@freebsd.org Date: Mon, 17 Feb 97 8:42:25 MET From: Josef Moellers X-Mailer: xmail 2.4 (based on ELM 2.2 PL16) Hi, I'd like to join the [/usr]/[s]bin people. How do I get to the sources? All I have at home is a rather old 2.1 system (the January '96 WC CD). Happy searching, Josef -- Josef Moellers mollers.pad@sni.de PS Dieser Artikel enthaelt einzig und allein meine persoenlichen Ansichten! PS This article contains my own, personal opinion only! From dillon@flea.best.net Sun Feb 16 23:59:09 1997 Return-Path: Received: from flea.best.net (root@flea.best.net [206.184.139.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id XAA00346 for ; Sun, 16 Feb 1997 23:59:09 -0800 (PST) Received: (from dillon@localhost) by flea.best.net (8.8.5/8.8.3) id XAA19161; Sun, 16 Feb 1997 23:57:20 -0800 (PST) Date: Sun, 16 Feb 1997 23:57:20 -0800 (PST) From: Matt Dillon Message-Id: <199702170757.XAA19161@flea.best.net> To: Dan Cross Cc: Marc Slemko , auditors@freebsd.org Subject: Re: strncpy() and NULL termination I don't think we should get fancy... this pass should be strictly hole-closing, not clib expansion. If we get sidetracked, we are going to wind up modifying every program in the source tree. The phrase 'if it aint broke, don't fix it!' comes to mind :-) It will be easier to verify fixes if they are self contained and do not rely on new library functions. And it will probably be easier on the rest of the community as well... we need to maintain backwards compatibility with the shared libraries at the very least. -Matt :> > (strncpy(3) only returns NULL or a valid pointer, but there is :> > no indication of truncation at all, unless you check to see if :> > buf[len - 1] != '\0'.) :> :> If you do that then you need to set buf[len-1] = '\0' before the strncpy. : :Eh? Why would you need to do that? strncpy(3) does nul padding... ;-) : :> So, anyone up for doing a standard routine for logging (syslog?) when a :> program aborts for this reason? No, you don't want to blindly do it :> everywhere but it could be useful in some places if done correctly. : :That's not a bad idea. btw, would folks find something like this useful? : :char * :sstrtncpy(dst, src, len) : char *dst; : const char *src; : size_t len; :{ : register size_t last; : : if (len != 0) : { :... Matthew Dillon Engineering, BEST Internet Communications, Inc. [always include a portion of the original email in any response!] From robmel@nadt.org.uk Mon Feb 17 03:56:11 1997 Return-Path: Received: from sys3.cambridge.uk.psi.net (sys3.cambridge.uk.psi.net [154.32.106.10]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id DAA10567 for ; Mon, 17 Feb 1997 03:56:10 -0800 (PST) Received: from sys4.cambridge.uk.psi.net (uucp1.mail.uk.psi.net [154.32.105.26]) by sys3.cambridge.uk.psi.net (8.8.4/) with ESMTP id LAA23586 for ; Mon, 17 Feb 1997 11:56:06 GMT Received: from nadt.org.uk by sys4.cambridge.uk.psi.net (8.7.5/SMI-5.5-UKPSINet) id LAA26689; Mon, 17 Feb 1997 11:46:25 GMT Received: from infodev.nadt.org.uk (infodev.nadt.org.uk [194.155.224.205]) by charlie.nadt.org.uk (8.6.12/8.6.12) with SMTP id LAA00288; Mon, 17 Feb 1997 11:31:17 GMT Posted-Date: Mon, 17 Feb 1997 11:31:17 GMT X-Website: http://www.innotts.co.uk/~nadt Message-Id: <1.5.4.32.19970217113146.006dcd18@wrcmail> X-Sender: robmel@wrcmail X-Mailer: Windows Eudora Light Version 1.5.4 (32) Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Mon, 17 Feb 1997 11:31:46 +0000 To: Dan Cross From: Robin Melville Subject: Re: strncpy() and NULL termination Cc: auditors@freebsd.org At 19:27 16/02/97 -0500, you wrote: >(strncpy(3) only returns NULL or a valid pointer, but there is >no indication of truncation at all, unless you check to see if >buf[len - 1] != '\0'.) No good if buf wasn't zeroised... the '\0' might be further back. -------------------------------------------------------- Robin Melville, Addiction & Forensic Information Service Nottingham Alcohol & Drug Team (Extn. 49178) Vox: +44 (0)115 952 9478 Fax: +44 (0)115 952 9421 Email: robmel@nadt.org.uk WWW: http://www.innotts.co.uk/nadt/ --------------------------------------------------------- From espel@llaic.univ-bpclermont.fr Mon Feb 17 05:14:47 1997 Return-Path: Received: from llaic.univ-bpclermont.fr (llaic.univ-bpclermont.fr [192.54.142.163]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id FAA14212 for ; Mon, 17 Feb 1997 05:14:45 -0800 (PST) Message-Id: <199702171314.FAA14212@freefall.freebsd.org> Received: by llaic.univ-bpclermont.fr (1.38.193.4/16.2) id AA22651; Mon, 17 Feb 1997 14:14:02 +0100 From: Roger Espel Llima Subject: Re: strncpy() and NULL termination To: auditors@freebsd.org Date: Mon, 17 Feb 1997 14:14:01 +0100 (MET) In-Reply-To: from "Aaron Bornstein" at Feb 15, 97 04:12:58 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit > As I've been going through code, I've noticed a lot of > strncpy()'s that don't properly NULL-terminate the strings. Seeing as > this is a security audit, I have a feeling that qiute a few new > strncpy()'s will be introduced into the tree, and I'd just like to remind > everyone that strncpy does NOT guarantee NULL termination. > > Bad: > strncpy(buf, somestring, BUFSIZ); > > Good: > strncpy(buf, somestring, BUFSIZ); > buf[BUFSIZ-1] = '\0'; > > > Not having NULL-terminated strings is usually a Bad Thing. While > it's never a Good Idea to assume a string is NULL terminated, we can't > assume (especially those of us doing libc reviews) that everyone using > our code will be conscious enough to recognize this. Yep. One neat way to deal with this, that is used all over the IRC server sources, is with this define: #define strncpyzt(x, y, N) do{(void)strncpy(x,y,N);x[N-1]='\0';}while(0) Roger -- e-mail: roger.espel.llima@ens.fr WWW page & PGP key: http://www.eleves.ens.fr:8080/home/espel/index.html From adrian@cougar.aceonline.com.au Mon Feb 17 07:02:04 1997 Return-Path: Received: from cougar.aceonline.com.au (adrian@cougar.aceonline.com.au [203.103.81.36]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id HAA19700 for ; Mon, 17 Feb 1997 07:01:59 -0800 (PST) Received: from localhost (adrian@localhost) by cougar.aceonline.com.au (8.8.4/8.7) with SMTP id XAA30655; Mon, 17 Feb 1997 23:01:20 +0800 Date: Mon, 17 Feb 1997 23:01:20 +0800 (WST) From: Adrian Chadd To: Roger Espel Llima cc: auditors@freebsd.org Subject: Re: strncpy() and NULL termination In-Reply-To: <199702171314.FAA14212@freefall.freebsd.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Another thing I've noticed is some bits of source pass argv[] values directly, eg this evil bit from domainname.c : if (*argv) { if (setdomainname(*argv, strlen(*argv)) err(1, "setdomainname"); } else { ... Passing argv like this is evil, and a check SHOULD be made before passing something like this to a function. FYI the library call does the argument length checking, but if it was calling something that didn't do range checking.. maybe suid/sgid? Cya. -- Adrian Chadd | Windows 95 - the XT emulator for | your 486 and above! | Being superstitious is bad luck. From tenser@spitfire.ecsel.psu.edu Mon Feb 17 09:47:57 1997 Return-Path: Received: from spitfire.ecsel.psu.edu (qmailr@spitfire.ecsel.psu.edu [146.186.218.51]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id JAA00528 for ; Mon, 17 Feb 1997 09:47:55 -0800 (PST) Received: (qmail 1036 invoked by uid 1000); 17 Feb 1997 17:47:45 -0000 Message-ID: <19970217174745.1035.qmail@spitfire.ecsel.psu.edu> To: Robin Melville cc: auditors@freebsd.org Subject: Re: strncpy() and NULL termination In-reply-to: Your message of "Mon, 17 Feb 1997 11:31:46 GMT." <1.5.4.32.19970217113146.006dcd18@wrcmail> Date: Mon, 17 Feb 1997 12:47:45 -0500 From: Dan Cross > >(strncpy(3) only returns NULL or a valid pointer, but there is > >no indication of truncation at all, unless you check to see if > >buf[len - 1] != '\0'.) > > No good if buf wasn't zeroised... the '\0' might be further back. Okay, just to set the record straight; strncpy(3) does nul padding from i to len - 1 where i is the current position in the dst array in the event that len < strlen(src), so checking to see if dst[len - 1] == '\0' is absolutely fine, without pre-initialization. This is in the man page, and can easily be verfied by looking at the source: char * strncpy(dst, src, n) char *dst; const char *src; register size_t n; { if (n != 0) { register char *d = dst; register const char *s = src; do { if ((*d++ = *s++) == 0) { /* NUL pad the remaining n-1 bytes */ while (--n != 0) *d++ = 0; break; } } while (--n != 0); } return (dst); } Note the while loop inside the do {} while(). Thanks for being on your toes, though. :-) That's what this is all about, right? - Dan C. From albert@gamp.hacom.nl Mon Feb 17 12:21:56 1997 Return-Path: Received: from box.hacom.nl (root@box.hacom.nl [193.67.233.8]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id MAA12024 for ; Mon, 17 Feb 1997 12:21:52 -0800 (PST) Received: (from uucp@localhost) by box.hacom.nl (8.8.5/8.8.5) with UUCP id VAA08865 for auditors@freebsd.org; Mon, 17 Feb 1997 21:20:17 +0100 Received: (from albert@localhost) by beowulf.gamp.hacom.nl (8.6.12/10.2) id VAA25697; Mon, 17 Feb 1997 21:18:34 +0100 Date: Mon, 17 Feb 1997 21:18:34 +0100 Message-Id: <199702172018.VAA25697@beowulf.gamp.hacom.nl> From: Albert Mietus To: auditors@freebsd.org Subject: Source, I need source ..... (and coordination) Hai all, I'm just a "simple" FreeBSD user, with only a slow (28k8) dailup modem and the 2.1 CDrom. To do some auditoring/reviewing I need the latest, source .. I there any "smart" way to retreve them, or can sombody create that. I assume I'm not the only one ... BTW. I also need some more coordination: Wat do you call auditoring and wat is reviewing. I know the term, but I also know they are have more explanation then there are programmers on earth:-) Last, I think, it should be handy to know which parts do "still" need auditoring/reviewing, and which should be done first. If I don't get "good" answers to these question I'm afraid I can't offer much help ... P.S. Can the list owner, somebody, change mine mail address that is used on this list to "gam@gamp.hacom.nl" (instead of albert@gamp.hacom.nl) "gam", that's me also, but I use "albert" only direct/personal mail and "gam" for mail-lists. PLEASE and THANKS. ---GAM "This should be a jolly quote" From giles@nemeton.com.au Mon Feb 17 13:23:58 1997 Return-Path: Received: from perki0.connect.com.au (perki0.connect.com.au [192.189.54.85]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id NAA16027 for ; Mon, 17 Feb 1997 13:23:54 -0800 (PST) Received: from nemeton.UUCP (Unemeton@localhost) by perki0.connect.com.au with UUCP id IAA04243 (8.7.6h/IDA-1.6); Tue, 18 Feb 1997 08:23:45 +1100 (EST) X-Authentication-Warning: perki0.connect.com.au: Unemeton set sender to giles@nemeton.com.au using -f Received: from localhost.nemeton.com.au (localhost.nemeton.com.au [127.0.0.1]) by nemeton.com.au (8.8.5/8.8.5) with SMTP id HAA29046; Tue, 18 Feb 1997 07:39:16 +1100 (EST) Message-Id: <199702172039.HAA29046@nemeton.com.au> To: Albert Mietus cc: auditors@freebsd.org Subject: Re: Source, I need source ..... (and coordination) In-reply-to: <199702172018.VAA25697@beowulf.gamp.hacom.nl> Date: Tue, 18 Feb 1997 07:39:16 +1100 From: Giles Lean On Mon, 17 Feb 1997 21:18:34 +0100 Albert Mietus wrote: > I'm just a "simple" FreeBSD user, with only a slow (28k8) dailup modem > and the 2.1 CDrom. To do some auditoring/reviewing I need the latest, > source .. I there any "smart" way to retreve them, or can sombody > create that. I assume I'm not the only one ... Read the handbook for CTM and CVSup, and choose one. It is quite feasible to obtain -current sources via modem, or even the full CVS tree. > Wat do you call auditoring and wat is reviewing 'auditing' is the first pass by someone looking for security problems. 'reviewing' is a review of patches generated. Reviewers should have commit access to the CVS tree. > Last, I think, it should be handy to know which parts do "still" need > auditoring/reviewing, and which should be done first. Jordan has posted a few things ... this is still settling down. Look at http://www.freebsd.org/auditors.html for a start. Giles From imp@village.org Mon Feb 17 13:44:36 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id NAA17556 for ; Mon, 17 Feb 1997 13:44:34 -0800 (PST) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 0.56 #1) id E0vwar9-0000BX-00; Mon, 17 Feb 1997 14:44:15 -0700 To: dg@root.com Subject: Re: Register pointers in gcc? Cc: Brian Tao , auditors@freebsd.org In-reply-to: Your message of "Sun, 16 Feb 1997 20:04:50 PST." <199702170404.UAA04242@root.com> References: <199702170404.UAA04242@root.com> Date: Mon, 17 Feb 1997 14:44:15 -0700 From: Warner Losh Message-Id: In message <199702170404.UAA04242@root.com> David Greenman writes: : >I'm a little surprised that the kernel doesn't barf on the string : >being >> MAXPATHLEN, but that's a whole other set of bugs that I don't : : It would be a bug if the kernel didn't allow args being > MAXPATHLEN. : The only restriction is that the total length of all of the arguments plus : environment must be less than ARG_MAX, which is currently 64K bytes. Even for the path of the executable to execute? Warner From jkh@time.cdrom.com Mon Feb 17 16:48:41 1997 Return-Path: Received: from time.cdrom.com (time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA29427 for ; Mon, 17 Feb 1997 16:48:35 -0800 (PST) Received: from time.cdrom.com (localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id QAA28395; Mon, 17 Feb 1997 16:47:57 -0800 (PST) To: Josef Moellers cc: auditors@freebsd.org Subject: Re: W: Assignment In-reply-to: Your message of "Mon, 17 Feb 1997 08:42:25 +0700." <199702170741.IAA08566@nixpbe.pdb.sni.de> Date: Mon, 17 Feb 1997 16:47:57 -0800 Message-ID: <28391.856226877@time.cdrom.com> From: "Jordan K. Hubbard" > Hi, > > I'd like to join the [/usr]/[s]bin people. Done! > How do I get to the sources? All I have at home is a rather old 2.1 > system (the January '96 WC CD). Well, if you've got FTP and are only working on small pieces at a time, I can suggest the source tree at: ftp://ftp.freebsd.org/pub/FreeBSD/FreeBSD-current/src Most people seem to forget that resource exists. :-) Jordan From jkh@time.cdrom.com Mon Feb 17 16:51:26 1997 Return-Path: Received: from time.cdrom.com (time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA29615 for ; Mon, 17 Feb 1997 16:51:15 -0800 (PST) Received: from time.cdrom.com (localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id QAA28445 for ; Mon, 17 Feb 1997 16:51:12 -0800 (PST) Prev-Resent: Mon, 17 Feb 1997 16:51:11 -0800 Prev-Resent: "auditors@freebsd.org " Received: from mirriwinni.cse.rmit.edu.au (pm.cse.rmit.EDU.AU [131.170.118.50]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id FAA01569 for ; Mon, 17 Feb 1997 05:08:14 -0800 (PST) Received: (from phillip@localhost) by mirriwinni.cse.rmit.edu.au (8.8.3/8.6.12) id AAA26318; Tue, 18 Feb 1997 00:08:10 +1100 (EST) Date: Tue, 18 Feb 1997 00:08:10 +1100 (EST) Message-Id: <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> From: Phillip Musumeci To: "Jordan K. Hubbard" In-reply-to: "Jordan K. Hubbard"'s message of 11 Feb 1997 22:39:19 -0600 Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org References: <5drhhn$4fd@bonkers.taronga.com> Resent-To: auditors@freebsd.org Resent-Date: Mon, 17 Feb 1997 16:51:11 -0800 Resent-Message-ID: <28442.856227071@time.cdrom.com> Resent-From: "Jordan K. Hubbard" Hi, Jordan> ... to begin a much more serious and comprehensive security Jordan> audit, we will take advantage of this opportunity to see that Jordan> all code in the FreeBSD source tree, old and new alike, is Jordan> reviewed line by line for buffer overflows, unguarded copies, Jordan> back doors, whatever. I think this source code checking is a good idea. Another thing that might be practical, or maybe not (I'm not sure), is run-time checking for buffer overflows. There exists a set of memory allocation routines that I have seen other people use in order to locate memory leaks and find illegal memory accesses via pointers (e.g. writing past the end of an allocated piece of memory storage --- search for string "writing past"). It has been ported to FreeBSD, and here is the README file. Maybe this alternative memory allocation library could be used whenever someone wanted to "test drive" a utility and look for memory leaks. The default make could use the normal memory allocation. As I said, this is just an idea and I'm not sure how practical it would be to use it for FreeBSD testing. I thought I recognised the name of the guy who did the FreeBSD though, so maybe he is closely associated with the FreeBSD team and has already suggested this. regards, phillip p.s. I became aware of this set of memory management tools because it is used in the public domain RLaB matrix maths package. --------------------------------------------------------------------------- Copyright 1988, 1989 Hans-J. Boehm, Alan J. Demers Copyright (c) 1991-1995 by Xerox Corporation. All rights reserved. THIS MATERIAL IS PROVIDED AS IS, WITH ABSOLUTELY NO WARRANTY EXPRESSED OR IMPLIED. ANY USE IS AT YOUR OWN RISK. Permission is hereby granted to use or copy this program for any purpose, provided the above notices are retained on all copies. Permission to modify the code and to distribute modified code is granted, provided the above notices are retained, and a notice that the code was modified is included with the above copyright notice. This is version 4.10 of a conservative garbage collector for C and C++. HISTORY - Early versions of this collector were developed as a part of research projects supported in part by the National Science Foundation and the Defense Advance Research Projects Agency. Much of the code was rewritten by Hans-J. Boehm at Xerox PARC. The SPARC specific code was contributed by Mark Weiser (weiser@parc.xerox.com). The Encore Multimax modifications were supplied by Kevin Kenny (kenny@m.cs.uiuc.edu). The adaptation to the RT is largely due to Vernon Lee (scorpion@rice.edu), on machines made available by IBM. Much of the HP specific code and a number of good suggestions for improving the generic code are due to Walter Underwood (wunder@hp-ses.sde.hp.com). Robert Brazile (brazile@diamond.bbn.com) originally supplied the ULTRIX code. Al Dosser (dosser@src.dec.com) and Regis Cridlig (Regis.Cridlig@cl.cam.ac.uk) subsequently provided updates and information on variation between ULTRIX systems. Parag Patel (parag@netcom.com) supplied the A/UX code. Jesper Peterson(jep@mtiame.mtia.oz.au) and Michel Schinz supplied the Amiga port. Thomas Funke (thf@zelator.in-berlin.de(?)) and Brian D.Carlstrom (bdc@clark.lcs.mit.edu) supplied the NeXT ports. Douglas Steel (doug@wg.icl.co.uk) provided ICL DRS6000 code. Bill Janssen (janssen@parc.xerox.com) supplied the SunOS dynamic loader specific code. Manuel Serrano (serrano@cornas.inria.fr) supplied linux and Sony News specific code. Al Dosser provided Alpha/OSF/1 code. He and Dave Detlefs(detlefs@src.dec.com) also provided several generic bug fixes. Alistair G. Crooks(agc@uts.amdahl.com) supplied the NetBSD and 386BSD ports. Jeffrey Hsu (hsu@soda.berkeley.edu) provided the FreeBSD port. Brent Benson (brent@jade.ssd.csd.harris.com) ported the collector to a Motorola 88K processor running CX/UX (Harris NightHawk). Ari Huttunen (Ari.Huttunen@hut.fi) generalized the OS/2 port to nonIBM development environments (a nontrivial task). Patrick Beard (beard@cs.ucdavis.edu) provided the initial MacOS port. David Chase, then at Olivetti Research, suggested several improvements. Scott Schwartz (schwartz@groucho.cse.psu.edu) supplied some of the code to save and print call stacks for leak detection on a SPARC. Jesse Hull and John Ellis supplied the C++ interface code. Zhong Shao performed much of the experimentation that led to the current typed allocation facility. (His dynamic type inference code hasn't made it into the released version of the collector, yet.) (Blame for misinstallation of these modifications goes to the first author, however.) This is intended to be a general purpose, garbage collecting storage allocator. The algorithms used are described in: Boehm, H., and M. Weiser, "Garbage Collection in an Uncooperative Environment", Software Practice & Experience, September 1988, pp. 807-820. Boehm, H., A. Demers, and S. Shenker, "Mostly Parallel Garbage Collection", Proceedings of the ACM SIGPLAN '91 Conference on Programming Language Design and Implementation, SIGPLAN Notices 26, 6 (June 1991), pp. 157-164. Boehm, H., "Space Efficient Conservative Garbage Collection", Proceedings of the ACM SIGPLAN '91 Conference on Programming Language Design and Implementation, SIGPLAN Notices 28, 6 (June 1993), pp. 197-206. Possible interactions between the collector and optimizing compilers are discussed in Boehm, H., and D. Chase, "A Proposal for GC-safe C Compilation", The Journal of C Language Translation 4, 2 (December 1992). (Also available from parcftp.xerox.com:pub/gc, among other places.) Unlike the collector described in the second reference, this collector operates either with the mutator stopped during the entire collection (default) or incrementally during allocations. (The latter is supported on only a few machines.) It does not rely on threads, but is intended to be thread-safe. Some of the ideas underlying the collector have previously been explored by others. (Doug McIlroy wrote a vaguely similar collector that is part of version 8 UNIX (tm).) However none of this work appears to have been widely disseminated. Rudimentary tools for use of the collector as a leak detector are included, as is a fairly sophisticated string package "cord" that makes use of the collector. (See cord/README.) GENERAL DESCRIPTION This is a garbage collecting storage allocator that is intended to be used as a plug-in replacement for C's malloc. Since the collector does not require pointers to be tagged, it does not attempt to ensure that all inaccessible storage is reclaimed. However, in our experience, it is typically more successful at reclaiming unused memory than most C programs using explicit deallocation. Unlike manually introduced leaks, the amount of unreclaimed memory typically stays bounded. In the following, an "object" is defined to be a region of memory allocated by the routines described below. Any objects not intended to be collected must be pointed to either from other such accessible objects, or from the registers, stack, data, or statically allocated bss segments. Pointers from the stack or registers may point to anywhere inside an object. However, it is usually assumed that all pointers originating in the heap point to the beginning of an object. (This does not disallow interior pointers; it simply requires that there must be a pointer to the beginning of every accessible object, in addition to any interior pointers.) There are two facilities for altering this behavior. The macro ALL_INTERIOR_POINTERS may be defined in gc_private.h to cause any pointer into an object (or one past the end) to retain the object. A routine GC_register_displacement is provided to allow for more controlled interior pointer use in the heap. Defining ALL_INTERIOR_POINTERS is somewhat dangerous, in that it can result in unnecessary memory retention. However this is much less of a problem than with older collector versions. The routine GC_register_displacement is described in gc.h. Note that pointers inside memory allocated by the standard "malloc" are not seen by the garbage collector. Thus objects pointed to only from such a region may be prematurely deallocated. It is thus suggested that the standard "malloc" be used only for memory regions, such as I/O buffers, that are guaranteed not to contain pointers. Pointers in C language automatic, static, or register variables, are correctly recognized. (Note that GC_malloc_uncollectable has semantics similar to standard malloc, but allocates objects that are traced by the collector.) The collector does not generally know how to find pointers in data areas that are associated with dynamic libraries. This is easy to remedy IF you know how to find those data areas on your operating system (see GC_add_roots). Code for doing this under SunOS and IRIX 5.X is included (see dynamic_load.c). Note that the garbage collector does not need to be informed of shared read-only data. However if the shared library mechanism can introduce discontiguous data areas that may contain pointers, then the collector does need to be informed. Signal processing for most signals may be deferred during collection, and during uninterruptible parts of the allocation process. Unlike standard ANSI C mallocs, it can be safe to invoke malloc from a signal handler while another malloc is in progress, provided the original malloc is not restarted. (Empirically, many UNIX applications already assume this.) To obtain this level of signal safety, remove the definition of -DNO_SIGNALS in Makefile. The allocator/collector can also be configured for thread-safe operation. (Full signal safety can also be achieved, but only at the cost of two system calls per malloc, which is usually unacceptable.) INSTALLATION AND PORTABILITY As distributed, the macro SILENT is defined in Makefile. In the event of problems, this can be removed to obtain a moderate amount of descriptive output for each collection. (The given statistics exhibit a few peculiarities. Things don't appear to add up for a variety of reasons, most notably fragmentation losses. These are probably much more significant for the contrived program "test.c" than for your application.) Note that typing "make test" will automatically build the collector and then run setjmp_test and gctest. Setjmp_test will give you information about configuring the collector, which is useful primarily if you have a machine that's not already supported. Gctest is a somewhat superficial test of collector functionality. Failure is indicated by a core dump or a message to the effect that the collector is broken. Gctest takes about 35 seconds to run on a SPARCstation 2. On a slower machine, expect it to take a while. It may use up to 8 MB of memory. (The multi-threaded version will use more.) "Make test" will also, as its last step, attempt to build and test the "cord" string library. This will fail without an ANSI C compiler. The Makefile will generate a library gc.a which you should link against. Typing "make cords" will add the cord library to gc.a. Note that this requires an ANSI C compiler. It is suggested that if you need to replace a piece of the collector (e.g. GC_mark_rts.c) you simply list your version ahead of gc.a on the ld command line, rather than replacing the one in gc.a. (This will generate numerous warnings under some versions of AIX, but it still works.) All include files that need to be used by clients will be put in the include subdirectory. (Normally this is just gc.h. "Make cords" adds "cord.h" and "ec.h".) The collector currently is designed to run essentially unmodified on the following machines (most of the operating systems mentioned are trademarks of their respective holders): Sun 3 Sun 4 under SunOS 4.X or Solaris2.X (with or without threads) Vax under 4.3BSD, Ultrix Intel 386 or 486 under most operating systems, but not MSDOS. (Win32S is somewhat supported, so it is possible to build applications for Windows 3.1. There exists a port to DOS + 32 bit extender for at least one 32 bit extender. However, I don't have source for this.) Sequent Symmetry (single threaded) Encore Multimax (single threaded) MIPS M/120 (and presumably M/2000) (RISC/os 4.0 with BSD libraries) IBM PC/RT (Berkeley UNIX) IBM RS/6000 HP9000/300 HP9000/700 DECstations under Ultrix DEC Alpha running OSF/1 SGI workstations under IRIX 4 & 5 Sony News Apple Macintosh under A/UX or MacOS Commodore Amiga (see README.amiga) NeXT machines In a few cases (Amiga, OS/2, Win32, MacOS) a separate makefile or equivalent is supplied. Dynamic libraries are completely supported only under SunOS (and even that support is not functional on the last Sun 3 release), IRIX 5, Win32 (not Win32S) and OSF/1 on DEC AXP machines. On other machines we recommend that you do one of the following: 1) Add dynamic library support (and send us the code). 2) Use static versions of the libraries. 3) Arrange for dynamic libraries to use the standard malloc. This is still dangerous if the library stores a pointer to a garbage collected object. But nearly all standard interfaces prohibit this, because they deal correctly with pointers to stack allocated objects. (Strtok is an exception. Don't use it.) In all cases we assume that pointer alignment is consistent with that enforced by the standard C compilers. If you use a nonstandard compiler you may have to adjust the alignment parameters defined in gc_priv.h. A port to a machine that is not byte addressed, or does not use 32 bit or 64 bit addresses will require a major effort. A port to MSDOS is hard, unless you are willing to assume an 80386 or better, and that only flat 32 bit pointers will ever need to be seen by the collector. For machines not already mentioned, or for nonstandard compilers, the following are likely to require change: 1. The parameters in config.h. The parameters that will usually require adjustment are STACKBOTTOM, ALIGNMENT and DATASTART. Setjmp_test prints its guesses of the first two. DATASTART should be an expression for computing the address of the beginning of the data segment. This can often be &etext. But some memory management units require that there be some unmapped space between the text and the data segment. Thus it may be more complicated. On UNIX systems, this is rarely documented. But the adb "$m" command may be helpful. (Note that DATASTART will usually be a function of &etext. Thus a single experiment is usually insufficient.) STACKBOTTOM is used to initialize GC_stackbottom, which should be a sufficient approximation to the coldest stack address. On some machines, it is difficult to obtain such a value that is valid across a variety of MMUs, OS releases, etc. A number of alternatives exist for using the collector in spite of this. See the discussion in config.h immediately preceding the various definitions of STACKBOTTOM. 2. mach_dep.c. The most important routine here is one to mark from registers. The distributed file includes a generic hack (based on setjmp) that happens to work on many machines, and may work on yours. Try compiling and running setjmp_t.c to see whether it has a chance of working. (This is not correct C, so don't blame your compiler if it doesn't work. Based on limited experience, register window machines are likely to cause trouble. If your version of setjmp claims that all accessible variables, including registers, have the value they had at the time of the longjmp, it also will not work. Vanilla 4.2 BSD on Vaxen makes such a claim. SunOS does not.) If your compiler does not allow in-line assembly code, or if you prefer not to use such a facility, mach_dep.c may be replaced by a .s file (as we did for the MIPS machine and the PC/RT). At this point enough architectures are supported by mach_dep.c that you will rarely need to do more than adjust for assembler syntax. 3. os_dep.c (and gc_priv.h). Several kinds of operating system dependent routines reside here. Many are optional. Several are invoked only through corresponding macros in gc_priv.h, which may also be redefined as appropriate. The routine GC_register_data_segments is crucial. It registers static data areas that must be traversed by the collector. (User calls to GC_add_roots may sometimes be used for similar effect.) Routines to obtain memory from the OS also reside here. Alternatively this can be done entirely by the macro GET_MEM defined in gc_priv.h. Routines to disable and reenable signals also reside here if they are need by the macros DISABLE_SIGNALS and ENABLE_SIGNALS defined in gc_priv.h. In a multithreaded environment, the macros LOCK and UNLOCK in gc_priv.h will need to be suitably redefined. The incremental collector requires page dirty information, which is acquired through routines defined in os_dep.c. Unless directed otherwise by config.h, these are implemented as stubs that simply treat all pages as dirty. (This of course makes the incremental collector much less useful.) 4. dyn_load.c This provides a routine that allows the collector to scan data segments associated with dynamic libraries. Often it is not necessary to provide this routine unless user-written dynamic libraries are used. For a different version of UN*X or different machines using the Motorola 68000, Vax, SPARC, 80386, NS 32000, PC/RT, or MIPS architecture, it should frequently suffice to change definitions in config.h. THE C INTERFACE TO THE ALLOCATOR The following routines are intended to be directly called by the user. Note that usually only GC_malloc is necessary. GC_clear_roots and GC_add_roots calls may be required if the collector has to trace from nonstandard places (e.g. from dynamic library data areas on a machine on which the collector doesn't already understand them.) On some machines, it may be desirable to set GC_stacktop to a good approximation of the stack base. (This enhances code portability on HP PA machines, since there is no good way for the collector to compute this value.) Client code may include "gc.h", which defines all of the following, plus many others. 1) GC_malloc(nbytes) - allocate an object of size nbytes. Unlike malloc, the object is cleared before being returned to the user. Gc_malloc will invoke the garbage collector when it determines this to be appropriate. GC_malloc may return 0 if it is unable to acquire sufficient space from the operating system. This is the most probable consequence of running out of space. Other possible consequences are that a function call will fail due to lack of stack space, or that the collector will fail in other ways because it cannot maintain its internal data structures, or that a crucial system process will fail and take down the machine. Most of these possibilities are independent of the malloc implementation. 2) GC_malloc_atomic(nbytes) - allocate an object of size nbytes that is guaranteed not to contain any pointers. The returned object is not guaranteed to be cleared. (Can always be replaced by GC_malloc, but results in faster collection times. The collector will probably run faster if large character arrays, etc. are allocated with GC_malloc_atomic than if they are statically allocated.) 3) GC_realloc(object, new_size) - change the size of object to be new_size. Returns a pointer to the new object, which may, or may not, be the same as the pointer to the old object. The new object is taken to be atomic iff the old one was. If the new object is composite and larger than the original object, then the newly added bytes are cleared (we hope). This is very likely to allocate a new object, unless MERGE_SIZES is defined in gc_priv.h. Even then, it is likely to recycle the old object only if the object is grown in small additive increments (which, we claim, is generally bad coding practice.) 4) GC_free(object) - explicitly deallocate an object returned by GC_malloc or GC_malloc_atomic. Not necessary, but can be used to minimize collections if performance is critical. Probably a performance loss for very small objects (<= 8 bytes). 5) GC_expand_hp(bytes) - Explicitly increase the heap size. (This is normally done automatically if a garbage collection failed to GC_reclaim enough memory. Explicit calls to GC_expand_hp may prevent unnecessarily frequent collections at program startup.) 6) GC_malloc_ignore_off_page(bytes) - identical to GC_malloc, but the client promises to keep a pointer to the somewhere within the first 256 bytes of the object while it is live. (This pointer should nortmally be declared volatile to prevent interference from compiler optimizations.) This is the recommended way to allocate anything that is likely to be larger than 100Kbytes or so. (GC_malloc may result in failure to reclaim such objects.) 7) GC_set_warn_proc(proc) - Can be used to redirect warnings from the collector. Such warnings should be rare, and should not be ignored during code development. 8) GC_enable_incremental() - Enables generational and incremental collection. Useful for large heaps on machines that provide access to page dirty information. Some dirty bit implementations may interfere with debugging (by catching address faults) and place restrictions on heap arguments to system calls (since write faults inside a system call may not be handled well). 9) Several routines to allow for registration of finalization code. User supplied finalization code may be invoked when an object becomes unreachable. To call (*f)(obj, x) when obj becomes inaccessible, use GC_register_finalizer(obj, f, x, 0, 0); For more sophisticated uses, and for finalization ordering issues, see gc.h. The global variable GC_free_space_divisor may be adjusted up from its default value of 4 to use less space and more collection time, or down for the opposite effect. Setting it to 1 or 0 will effectively disable collections and cause all allocations to simply grow the heap. The variable GC_non_gc_bytes, which is normally 0, may be changed to reflect the amount of memory allocated by the above routines that should not be considered as a candidate for collection. Careless use may, of course, result in excessive memory consumption. Some additional tuning is possible through the parameters defined near the top of gc_priv.h. If only GC_malloc is intended to be used, it might be appropriate to define: #define malloc(n) GC_malloc(n) #define calloc(m,n) GC_malloc((m)*(n)) For small pieces of VERY allocation intensive code, gc_inl.h includes some allocation macros that may be used in place of GC_malloc and friends. All externally visible names in the garbage collector start with "GC_". To avoid name conflicts, client code should avoid this prefix, except when accessing garbage collector routines or variables. There are provisions for allocation with explicit type information. This is rarely necessary. Details can be found in gc_typed.h. THE C++ INTERFACE TO THE ALLOCATOR: The Ellis-Hull C++ interface to the collector is included in the collector distribution. If you intend to use this, type "make c++" after the initial build of the collector is complete. See gc_cpp.h for the definition of the interface. This interface tries to approximate the Ellis-Detlefs C++ garbage collection proposal without compiler changes. Cautions: 1. Arrays allocated without new placement syntax are allocated as uncollectable objects. They are traced by the collector, but will not be reclaimed. 2. Failure to use "make c++" in combination with (1) will result in arrays allocated using the default new operator. This is likely to result in disaster without linker warnings. 3. If your compiler supports an overloaded new[] operator, then gc_cpp.cc and gc_cpp.h should be suitably modified. 4. Many current C++ compilers have deficiencies that break some of the functionality. See the comments in gc_cpp.h for suggested workarounds. USE AS LEAK DETECTOR: The collector may be used to track down leaks in C programs that are intended to run with malloc/free (e.g. code with extreme real-time or portability constraints). To do so define FIND_LEAK in Makefile This will cause the collector to invoke the report_leak routine defined near the top of reclaim.c whenever an inaccessible object is found that has not been explicitly freed. Productive use of this facility normally involves redefining report_leak to do something more intelligent. This typically requires annotating objects with additional information (e.g. creation time stack trace) that identifies their origin. Such code is typically not very portable, and is not included here, except on SPARC machines. If all objects are allocated with GC_DEBUG_MALLOC (see next section), then the default version of report_leak will report the source file and line number at which the leaked object was allocated. This may sometimes be sufficient. (On SPARC/SUNOS4 machines, it will also report a cryptic stack trace. This can often be turned into a sympolic stack trace by invoking program "foo" with "callprocs foo". Callprocs is a short shell script that invokes adb to expand program counter values to symbolic addresses. It was largely supplied by Scott Schwartz.) Note that the debugging facilities described in the next section can sometimes be slightly LESS effective in leak finding mode, since in leak finding mode, GC_debug_free actually results in reuse of the object. (Otherwise the object is simply marked invalid.) Also note that the test program is not designed to run meaningfully in FIND_LEAK mode. Use "make gc.a" to build the collector. DEBUGGING FACILITIES: The routines GC_debug_malloc, GC_debug_malloc_atomic, GC_debug_realloc, and GC_debug_free provide an alternate interface to the collector, which provides some help with memory overwrite errors, and the like. Objects allocated in this way are annotated with additional information. Some of this information is checked during garbage collections, and detected inconsistencies are reported to stderr. Simple cases of writing past the end of an allocated object should be caught if the object is explicitly deallocated, or if the collector is invoked while the object is live. The first deallocation of an object will clear the debugging info associated with an object, so accidentally repeated calls to GC_debug_free will report the deallocation of an object without debugging information. Out of memory errors will be reported to stderr, in addition to returning NIL. GC_debug_malloc checking during garbage collection is enabled with the first call to GC_debug_malloc. This will result in some slowdown during collections. If frequent heap checks are desired, this can be achieved by explicitly invoking GC_gcollect, e.g. from the debugger. GC_debug_malloc allocated objects should not be passed to GC_realloc or GC_free, and conversely. It is however acceptable to allocate only some objects with GC_debug_malloc, and to use GC_malloc for other objects, provided the two pools are kept distinct. In this case, there is a very low probablility that GC_malloc allocated objects may be misidentified as having been overwritten. This should happen with probability at most one in 2**32. This probability is zero if GC_debug_malloc is never called. GC_debug_malloc, GC_malloc_atomic, and GC_debug_realloc take two additional trailing arguments, a string and an integer. These are not interpreted by the allocator. They are stored in the object (the string is not copied). If an error involving the object is detected, they are printed. The macros GC_MALLOC, GC_MALLOC_ATOMIC, GC_REALLOC, GC_FREE, and GC_REGISTER_FINALIZER are also provided. These require the same arguments as the corresponding (nondebugging) routines. If gc.h is included with GC_DEBUG defined, they call the debugging versions of these functions, passing the current file name and line number as the two extra arguments, where appropriate. If gc.h is included without GC_DEBUG defined, then all these macros will instead be defined to their nondebugging equivalents. (GC_REGISTER_FINALIZER is necessary, since pointers to objects with debugging information are really pointers to a displacement of 16 bytes form the object beginning, and some translation is necessary when finalization routines are invoked. For details, about what's stored in the header, see the definition of the type oh in debug_malloc.c) INCREMENTAL/GENERATIONAL COLLECTION: The collector normally interrupts client code for the duration of a garbage collection mark phase. This may be unacceptable if interactive response is needed for programs with large heaps. The collector can also run in a "generational" mode, in which it usually attempts to collect only objects allocated since the last garbage collection. Furthermore, in this mode, garbage collections run mostly incrementally, with a small amount of work performed in response to each of a large number of GC_malloc requests. This mode is enabled by a call to GC_enable_incremental(). Incremental and generational collection is effective in reducing pause times only if the collector has some way to tell which objects or pages have been recently modified. The collector uses two sources of information: 1. Information provided by the VM system. This may be provided in one of several forms. Under Solaris 2.X (and potentially under other similar systems) information on dirty pages can be read from the /proc file system. Under other systems (currently SunOS4.X) it is possible to write-protect the heap, and catch the resulting faults. On these systems we require that system calls writing to the heap (other than read) be handled specially by client code. See os_dep.c for details. 2. Information supplied by the programmer. We define "stubborn" objects to be objects that are rarely changed. Such an object can be allocated (and enabled for writing) with GC_malloc_stubborn. Once it has been initialized, the collector should be informed with a call to GC_end_stubborn_change. Subsequent writes that store pointers into the object must be preceded by a call to GC_change_stubborn. This mechanism performs best for objects that are written only for initialization, and such that only one stubborn object is writable at once. It is typically not worth using for short-lived objects. Stubborn objects are treated less efficiently than pointerfree (atomic) objects. A rough rule of thumb is that, in the absence of VM information, garbage collection pauses are proportional to the amount of pointerful storage plus the amount of modified "stubborn" storage that is reachable during the collection. Initial allocation of stubborn objects takes longer than allocation of other objects, since other data structures need to be maintained. We recommend against random use of stubborn objects in client code, since bugs caused by inappropriate writes to stubborn objects are likely to be very infrequently observed and hard to trace. However, their use may be appropriate in a few carefully written library routines that do not make the objects themselves available for writing by client code. BUGS: Any memory that does not have a recognizable pointer to it will be reclaimed. Exclusive-or'ing forward and backward links in a list doesn't cut it. Some C optimizers may lose the last undisguised pointer to a memory object as a consequence of clever optimizations. This has almost never been observed in practice. Send mail to boehm@parc.xerox.com for suggestions on how to fix your compiler. This is not a real-time collector. In the standard configuration, percentage of time required for collection should be constant across heap sizes. But collection pauses will increase for larger heaps. (On SPARCstation 2s collection times will be on the order of 300 msecs per MB of accessible memory that needs to be scanned. Your mileage may vary.) The incremental/generational collection facility helps, but is portable only if "stubborn" allocation is used. Please address bug reports to boehm@parc.xerox.com. If you are contemplating a major addition, you might also send mail to ask whether it's already been done (or whether we tried and discarded it). RECENT VERSIONS: Version 1.3 and immediately preceding versions contained spurious assembly language assignments to TMP_SP. Only the assignment in the PC/RT code is necessary. On other machines, with certain compiler options, the assignments can lead to an unsaved register being overwritten. Known to cause problems under SunOS 3.5 WITHOUT the -O option. (With -O the compiler recognizes it as dead code. It probably shouldn't, but that's another story.) Version 1.4 and earlier versions used compile time determined values for the stack base. This no longer works on Sun 3s, since Sun 3/80s use a different stack base. We now use a straightforward heuristic on all machines on which it is known to work (incl. Sun 3s) and compile-time determined values for the rest. There should really be library calls to determine such values. Version 1.5 and earlier did not ensure 8 byte alignment for objects allocated on a sparc based machine. Version 1.8 added ULTRIX support in gc_private.h. Version 1.9 fixed a major bug in gc_realloc. Version 2.0 introduced a consistent naming convention for collector routines and added support for registering dynamic library data segments in the standard mark_roots.c. Most of the data structures were revamped. The treatment of interior pointers was completely changed. Finalization was added. Support for locking was added. Object kinds were added. We added a black listing facility to avoid allocating at addresses known to occur as integers somewhere in the address space. Much of this was accomplished by adapting ideas and code from the PCR collector. The test program was changed and expanded. Version 2.1 was the first stable version since 1.9, and added support for PPCR. Version 2.2 added debugging allocation, and fixed various bugs. Among them: - GC_realloc could fail to extend the size of the object for certain large object sizes. - A blatant subscript range error in GC_printf, which unfortunately wasn't exercised on machines with sufficient stack alignment constraints. - GC_register_displacement did the wrong thing if it was called after any allocation had taken place. - The leak finding code would eventually break after 2048 byte byte objects leaked. - interface.c didn't compile. - The heap size remained much too small for large stacks. - The stack clearing code behaved badly for large stacks, and perhaps on HP/PA machines. Version 2.3 added ALL_INTERIOR_POINTERS and fixed the following bugs: - Missing declaration of etext in the A/UX version. - Some PCR root-finding problems. - Blacklisting was not 100% effective, because the plausible future heap bounds were being miscalculated. - GC_realloc didn't handle out-of-memory correctly. - GC_base could return a nonzero value for addresses inside free blocks. - test.c wasn't really thread safe, and could erroneously report failure in a multithreaded environment. (The locking primitives need to be replaced for other threads packages.) - GC_CONS was thoroughly broken. - On a SPARC with dynamic linking, signals stayed diabled while the client code was running. (Thanks to Manuel Serrano at INRIA for reporting the last two.) Version 2.4 added GC_free_space_divisor as a tuning knob, added support for OS/2 and linux, and fixed the following bugs: - On machines with unaligned pointers (e.g. Sun 3), every 128th word could fail to be considered for marking. - Dynamic_load.c erroneously added 4 bytes to the length of the data and bss sections of the dynamic library. This could result in a bad memory reference if the actual length was a multiple of a page. (Observed on Sun 3. Can probably also happen on a Sun 4.) (Thanks to Robert Brazile for pointing out that the Sun 3 version was broken. Dynamic library handling is still broken on Sun 3s under 4.1.1U1, but apparently not 4.1.1. If you have such a machine, use -Bstatic.) Version 2.5 fixed the following bugs: - Removed an explicit call to exit(1) - Fixed calls to GC_printf and GC_err_printf, so the correct number of arguments are always supplied. The OS/2 C compiler gets confused if the number of actuals and the number of formals differ. (ANSI C doesn't require this to work. The ANSI sanctioned way of doing things causes too many compatibility problems.) Version 3.0 added generational/incremental collection and stubborn objects. Version 3.1 added the following features: - A workaround for a SunOS 4.X SPARC C compiler misfeature that caused problems when the collector was turned into a dynamic library. - A fix for a bug in GC_base that could result in a memory fault. - A fix for a performance bug (and several other misfeatures) pointed out by Dave Detlefs and Al Dosser. - Use of dirty bit information for static data under Solaris 2.X. - DEC Alpha/OSF1 support (thanks to Al Dosser). - Incremental collection on more platforms. - A more refined heap expansion policy. Less space usage by default. - Various minor enhancements to reduce space usage, and to reduce the amount of memory scanned by the collector. - Uncollectable allocation without per object overhead. - More conscientious handling of out-of-memory conditions. - Fixed a bug in debugging stubborn allocation. - Fixed a bug that resulted in occasional erroneous reporting of smashed objects with debugging allocation. - Fixed bogus leak reports of size 4096 blocks with FIND_LEAK. Version 3.2 fixed a serious and not entirely repeatable bug in the incremental collector. It appeared only when dirty bit info on the roots was available, which is normally only under Solaris. It also added GC_general_register_disappearing_link, and some testing code. Interface.c disappeared. Version 3.3 fixes several bugs and adds new ports: - PCR-specific bugs. - Missing locking in GC_free, redundant FASTUNLOCK in GC_malloc_stubborn, and 2 bugs in GC_unregister_disappearing_link. All of the above were pointed out by Neil Sharman (neil@cs.mu.oz.au). - Common symbols allocated by the SunOS4.X dynamic loader were not included in the root set. - Bug in GC_finalize (reported by Brian Beuning and Al Dosser) - Merged Amiga port from Jesper Peterson (untested) - Merged NeXT port from Thomas Funke (significantly modified and untested) Version 3.4: - Fixed a performance bug in GC_realloc. - Updated the amiga port. - Added NetBSD and 386BSD ports. - Added cord library. - Added trivial performance enhancement for ALL_INTERIOR_POINTERS. (Don't scan last word.) Version 3.5 - Minor collections now mark from roots only once, if that doesn't cause an excessive pause. - The stack clearing heuristic was refined to prevent anomalies with very heavily recursive programs and sparse stacks. - Fixed a bug that prevented mark stack growth in some cases. GC_objects_are_marked should be set to TRUE after a call to GC_push_roots and as part of GC_push_marked, since both can now set mark bits. I think this is only a performance bug, but I wouldn't bet on it. It's certainly very hard to argue that the old version was correct. - Fixed an incremental collection bug that prevented it from working at all when HBLKSIZE != getpagesize() - Changed dynamic_loading.c to include gc_priv.h before testing DYNAMIC_LOADING. SunOS dynamic library scanning must have been broken in 3.4. - Object size rounding now adapts to program behavior. - Added a workaround (provided by Manuel Serrano and colleagues) to a long-standing SunOS 4.X (and 3.X?) ld bug that I had incorrectly assumed to have been squished. The collector was broken if the text segment size was within 32 bytes of a multiple of 8K bytes, and if the beginning of the data segment contained interesting roots. The workaround assumes a demand-loadable executable. The original may have have "worked" in some other cases. - Added dynamic library support under IRIX5. - Added support for EMX under OS/2 (thanks to Ari Huttunen). Version 3.6: - fixed a bug in the mark stack growth code that was introduced in 3.4. - fixed Makefile to work around DEC AXP compiler tail recursion bug. Version 3.7: - Added a workaround for an HP/UX compiler bug. - Fixed another stack clearing performance bug. Reworked that code once more. Version 4.0: - Added support for Solaris threads (which was possible only by reimplementing some fraction of Solaris threads, since Sun doesn't currently make the thread debugging interface available). - Added non-threads win32 and win32S support. - (Grudgingly, with suitable muttering of obscenities) renamed files so that the collector distribution could live on a FAT file system. Files that are guaranteed to be useless on a PC still have long names. Gc_inline.h and gc_private.h still exist, but now just include gc_inl.h and gc_priv.h. - Fixed a really obscure bug in finalization that could cause undetected mark stack overflows. (I would be surprised if any real code ever tickled this one.) - Changed finalization code to dynamically resize the hash tables it maintains. (This probably does not matter for well- -written code. It no doubt does for C++ code that overuses destructors.) - Added typed allocation primitives. Rewrote the marker to accommodate them with more reasonable efficiency. This change should also speed up marking for GC_malloc allocated objects a little. See gc_typed.h for new primitives. - Improved debugging facilities slightly. Allocation time stack traces are now kept by default on SPARC/SUNOS4. (Thanks to Scott Schwartz.) - Added better support for small heap applications. - Significantly extended cord package. Fixed a bug in the implementation of lazily read files. Printf and friends now have cord variants. Cord traversals are a bit faster. - Made ALL_INTERIOR_POINTERS recognition the default. - Fixed de so that it can run in constant space, independent of file size. Added simple string searching to cords and de. - Added the Hull-Ellis C++ interface. - Added dynamic library support for OSF/1. (Thanks to Al Dosser and Tim Bingham at DEC.) - Changed argument to GC_expand_hp to be expressed in units of bytes instead of heap blocks. (Necessary since the heap block size now varies depending on configuration. The old version was never very clean.) - Added GC_get_heap_size(). The previous "equivalent" was broken. - Restructured the Makefile a bit. Since version 4.0: - Changed finalization implementation to guarantee that finalization procedures are called outside of the allocation lock, making direct use of the interface a little less dangerous. MAY BREAK EXISTING CLIENTS that assume finalizers are protected by a lock. Since there seem to be few multithreaded clients that use finalization, this is hopefully not much of a problem. - Fixed a gross bug in CORD_prev. - Fixed a bug in blacklst.c that could result in unbounded heap growth during startup on machines that do not clear memory obtained from the OS (e.g. win32S). - Ported de editor to win32/win32S. (This is now the only version with a mouse-sensitive UI.) - Added GC_malloc_ignore_off_page to allocate large arrays in the presence of ALL_INTERIOR_POINTERS. - Changed GC_call_with_alloc_lock to not disable signals in the single-threaded case. - Reduced retry count in GC_collect_or_expand for garbage collecting when out of memory. - Made uncollectable allocations bypass black-listing, as they should. - Fixed a bug in typed_test in test.c that could cause (legitimate) GC crashes. - Fixed some potential synchronization problems in finalize.c - Fixed a real locking problem in typd_mlc.c. - Worked around an AIX 3.2 compiler feature that results in out of bounds memory references. - Partially worked around an IRIX5.2 beta problem (which may or may not persist to the final release). - Fixed a bug in the heap integrity checking code that could result in explicitly deallocated objects being identified as smashed. Fixed a bug in the dbg_mlc stack saving code that caused old argument pointers to be considered live. - Fixed a bug in CORD_ncmp (and hence CORD_str). - Repaired the OS2 port, which had suffered from bit rot in 4.0. Worked around what appears to be CSet/2 V1.0 optimizer bug. - Fixed a Makefile bug for target "c++". Since version 4.1: - Multiple bug fixes/workarounds in the Solaris threads version. (It occasionally failed to locate some register contents for marking. It also turns out that thr_suspend and friends are unreliable in Solaris 2.3. Dirty bit reads appear to be unreliable under some weird circumstances. My stack marking code contained a serious performance bug. The new code is extremely defensive, and has not failed in several cpu hours of testing. But no guarantees ...) - Added MacOS support (thanks to Patrick Beard.) - Fixed several syntactic bugs in gc_c++.h and friends. (These didn't bother g++, but did bother most other compilers.) Fixed gc_c++.h finalization interface. (It didn't.) - 64 bit alignment for allocated objects was not guaranteed in a few cases in which it should have been. - Added GC_malloc_atomic_ignore_off_page. - Added GC_collect_a_little. - Added some prototypes to gc.h. - Some other minor bug fixes (notably in Makefile). - Fixed OS/2 / EMX port (thanks to Ari Huttunen). - Fixed AmigaDOS port. (thanks to Michel Schinz). - Fixed the DATASTART definition under Solaris. There was a 1 in 16K chance of the collector missing the first 64K of static data (and thus crashing). - Fixed some blatant anachronisms in the README file. - Fixed PCR-Makefile for upcoming PPCR release. Since version 4.2: - Fixed SPARC alignment problem with GC_DEBUG. - Fixed Solaris threads /proc workaround. The real problem was an interaction with mprotect. - Incorporated fix from Patrick Beard for gc_c++.h (now gc_cpp.h). - Slightly improved allocator space utilization by fixing the GC_size_map mechanism. - Integrated some Sony News and MIPS RISCos 4.51 patches. (Thanks to Nobuyuki Hikichi of Software Research Associates, Inc. Japan) - Fixed HP_PA alignment problem. (Thanks to xjam@cork.cs.berkeley.edu.) - Added GC_same_obj and friends. Changed GC_base to return 0 for pointers past the end of large objects. Improved GC_base performance with ALL_INTERIOR_POINTERS on machines with a slow integer mod operation. Added GC_PTR_ADD, GC_PTR_STORE, etc. to prepare for preprocessor. - changed the default on most UNIX machines to be that signals are not disabled during critical GC operations. This is still ANSI-conforming, though somewhat dangerous in the presence of signal handlers. But the performance cost of the alternative is sometimes problematic. Can be changed back with a minor Makefile edit. - renamed IS_STRING in gc.h, to CORD_IS_STRING, thus following my own naming convention. Added the function CORD_to_const_char_star. - Fixed a gross bug in GC_finalize. Symptom: occasional address faults in that function. (Thanks to Anselm Baird-Smith (Anselm.BairdSmith@inria.fr) - Added port to ICL DRS6000 running DRS/NX. Restructured things a bit to factor out common code, and remove obsolete code. Collector should now run under SUNOS5 with either mprotect or /proc dirty bits. (Thanks to Douglas Steel (doug@wg.icl.co.uk)). - More bug fixes and workarounds for Solaris 2.X. (These were mostly related to putting the collector in a dynamic library, which didn't really work before. Also SOLARIS_THREADS didn't interact well with dl_open.) Thanks to btlewis@eng.sun.com. - Fixed a serious performance bug on the DEC Alpha. The text segment was getting registered as part of the root set. (Amazingly, the result was still fast enough that the bug was not conspicuous.) The fix works on OSF/1, version 1.3. Hopefully it also works on other versions of OSF/1 ... - Fixed a bug in GC_clear_roots. - Fixed a bug in GC_generic_malloc_words_small that broke gc_inl.h. (Reported by Antoine de Maricourt. I broke it in trying to tweak the Mac port.) - Fixed some problems with cord/de under Linux. - Fixed some cord problems, notably with CORD_riter4. - Added DG/UX port. Thanks to Ben A. Mesander (ben@piglet.cr.usgs.gov) - Added finalization registration routines with weaker ordering constraints. (This is necessary for C++ finalization with multiple inheritance, since the compiler often adds self-cycles.) - Filled the holes in the SCO port. (Thanks to Michael Arnoldus .) - John Ellis' additions to the C++ support: From John: * I completely rewrote the documentation in the interface gc_c++.h (later renamed gc_cpp.h). I've tried to make it both clearer and more precise. * The definition of accessibility now ignores pointers from an finalizable object (an object with a clean-up function) to itself. This allows objects with virtual base classes to be finalizable by the collector. Compilers typically implement virtual base classes using pointers from an object to itself, which under the old definition of accessibility prevented objects with virtual base classes from ever being collected or finalized. * gc_cleanup now includes gc as a virtual base. This was enabled by the change in the definition of accessibility. * I added support for operator new[]. Since most (all?) compilers don't yet support operator new[], it is conditionalized on -DOPERATOR_NEW_ARRAY. The code is untested, but its trivial and looks correct. * The test program test_gc_c++ (later renamed test_cpp.cc) tries to test for the C++-specific functionality not tested by the other programs. - Added include to misc.c. (Needed for ppcr.) - Added PowerMac port. (Thanks to Patrick Beard again.) - Fixed "srcdir"-related Makefile problems. Changed things so that all externally visible include files always appear in the include subdirectory of the source. Made gc.h directly includable from C++ code. (These were at Per Bothner's suggestion.) - Changed Intel code to also mark from ebp (Kevin Warne's suggestion). - Renamed C++ related files so they could live in a FAT file system. (Charles Fiterman's suggestion.) - Changed Windows NT Makefile to include C++ support in gc.lib. Added C++ test as Makefile target. Since version 4.3: - ASM_CLEAR_CODE was erroneously defined for HP PA machines, resulting in a compile error. - Fixed OS/2 Makefile to create a library. (Thanks to Mark Boulter (mboulter@vnet.ibm.com)). - Gc_cleanup objects didn't work if they were created on the stack. Fixed. - One copy of Gc_cpp.h in the distribution was out of synch, and failed to document some known compiler problems with explicit destructor invocation. Partially fixed. There are probably other compilers on which gc_cleanup is miscompiled. - Fixed Makefile to pass C compiler flags to C++ compiler. - Added Mac fixes. - Fixed os_dep.c to work around what appears to be a new and different VirtualQuery bug under newer versions of win32S. - GC_non_gc_bytes was not correctly maintained by GC_free. Fixed. Thanks to James Clark (jjc@jclark.com). - Added GC_set_max_heap_size. - Changed allocation code to ignore blacklisting if it is preventing use of a very large block of memory. This has the advantage that naive code allocating very large objects is much more likely to work. The downside is you might no longer find out that such code should really use GC_malloc_ignore_off_page. - Changed GC_printf under win32 to close and reopen the file between calls. FAT file systems otherwise make the log file useless for debugging. - Added GC_try_to_collect and GC_get_bytes_since_gc. These allow starting an abortable collection during idle times. This facility does not require special OS support. (Thanks to Michael Spertus of Geodesic Systems for suggesting this. It was actually an easy addition. Kumar Srikantan previously added a similar facility to a now ancient version of the collector. At the time this was much harder, and the result was less convincing.) - Added some support for the Borland development environment. (Thanks to John Ellis and Michael Spertus.) - Removed a misfeature from checksums.c that caused unexpected heap growth. (Thanks to Scott Schwartz.) - Changed finalize.c to call WARN if it encounters a finalization cycle. WARN is defined in gc_priv.h to write a message, usually to stdout. In many environments, this may be inappropriate. - Renamed NO_PARAMS in gc.h to GC_NO_PARAMS, thus adhering to my own naming convention. - Added GC_set_warn_proc to intercept warnings. - Fixed Amiga port. (Thanks to Michel Schinz (schinz@alphanet.ch).) - Fixed a bug in mark.c that could result in an access to unmapped memory from GC_mark_from_mark_stack on machines with unaligned pointers. - Fixed a win32 specific performance bug that could result in scanning of objects allocated with the system malloc. - Added REDIRECT_MALLOC. Since version 4.4: - Fixed many minor and one major README bugs. (Thanks to Franklin Chen (chen@adi.com) for pointing out many of them.) - Fixed ALPHA/OSF/1 dynamic library support. (Thanks to Jonathan Bachrach (jonathan@harlequin.com)). - Added incremental GC support (MPROTECT_VDB) for Linux (with some help from Bruno Haible). - Altered SPARC recognition tests in gc.h and config.h (mostly as suggested by Fergus Henderson). - Added basic incremental GC support for win32, as implemented by Windows NT and Windows 95. GC_enable_incremental is a noop under win32s, which doesn't implement enough of the VM interface. - Added -DLARGE_CONFIG. - Fixed GC_..._ignore_off_page to also function without -DALL_INTERIOR_POINTERS. - (Hopefully) fixed RS/6000 port. (Only the test was broken.) - Fixed a performance bug in the nonincremental collector running on machines supporting incremental collection with MPROTECT_VDB (e.g. SunOS 4, DEC AXP). This turned into a correctness bug under win32s with win32 incremental collection. (Not all memory protection was disabled.) - Fixed some ppcr related bit rot. - Caused dynamic libraries to be unregistered before reregistering. The old way turned out to be a performance bug on some machines. - GC_root_size was not properly maintained under MSWIN32. - Added -DNO_DEBUGGING and GC_dump. - Fixed a couple of bugs arising with SOLARIS_THREADS + REDIRECT_MALLOC. - Added NetBSD/M68K port. (Thanks to Peter Seebach .) - Fixed a serious realloc bug. For certain object sizes, the collector wouldn't scan the expanded part of the object. (Thanks to Clay Spence (cds@peanut.sarnoff.com) for noticing the problem, and helping me to track it down.) Since version 4.5: - Added Linux ELF support. (Thanks to Arrigo Triulzi .) - GC_base crashed if it was called before any other GC_ routines. This could happen if a gc_cleanup object was allocated outside the heap before any heap allocation. - The heap expansion heuristic was not stable if all objects had finalization enabled. Fixed finalize.c to count memory in finalization queue and avoid explicit deallocation. Changed alloc.c to also consider this count. (This is still not recommended. It's expensive if nothing else.) Thanks to John Ellis for pointing this out. - GC_malloc_uncollectable(0) was broken. Thanks to Phong Vo for pointing this out. - The collector didn't compile under Linux 1.3.X. (Thanks to Fred Gilham for pointing this out.) The current workaround is ugly, but expected to be temporary. - Fixed a formatting problem for SPARC stack traces. - Fixed some '=='s in os_dep.c that should have been assignments. Fortunately these were in code that should never be executed anyway. (Thanks to Fergus Henderson.) - Fixed the heap block allocator to only drop blacklisted blocks in small chunks. Made BL_LIMIT self adjusting. (Both of these were in response to heap growth observed by Paul Graham.) - Fixed the Metrowerks/68K Mac code to also mark from a6. (Thanks to Patrick Beard.) - Significantly updated README.debugging. - Fixed some problems with longjmps out of signal handlers, especially under Solaris. Added a workaround for the fact that siglongjmp doesn't appear to do the right thing with -lthread under Solaris. - Added MSDOS/djgpp port. (Thanks to Mitch Harris (maharri@uiuc.edu).) - Added "make reserved_namespace" and "make user_namespace". The first renames ALL "GC_xxx" identifiers as "_GC_xxx". The second is the inverse transformation. Note that doing this is guaranteed to break all clients written for the other names. - descriptor field for kind NORMAL in GC_obj_kinds with ADD_BYTE_AT_END defined should be -ALIGNMENT not WORDS_TO_BYTES(-1). This is a serious bug on machines with pointer alignment of less than a word. - GC_ignore_self_finalize_mark_proc didn't handle pointers to very near the end of the object correctly. Caused failures of the C++ test on a DEC Alpha with g++. - gc_inl.h still had problems. Partially fixed. Added warnings at the beginning to hopefully specify the remaining dangers. - Added DATAEND definition to config.h. - Fixed some of the .h file organization. Fixed "make floppy". Since version 4.6: - Fixed some compilation problems with -DCHECKSUMS (thanks to Ian Searle) - Updated some Mac specific files to synchronize with Patrick Beard. - Fixed a serious bug for machines with non-word-aligned pointers. (Thanks to Patrick Beard for pointing out the problem. The collector should fail almost any conceivable test immediately on such machines.) Since version 4.7: - Changed a "comment" in a MacOS specific part of mach-dep.c that caused gcc to fail on other platforms. Since version 4.8 - More README.debugging fixes. - Objects ready for finalization, but not finalized in the same GC cycle, could be prematurely collected. This occasionally happened in test_cpp. - Too little memory was obtained from the system for very large objects. That could cause a heap explosion if these objects were not contiguous (e.g. under PCR), and too much of them was blacklisted. - Due to an improper initialization, the collector was too hesitant to allocate blacklisted objects immediately after system startup. - Moved GC_arrays from the data into the bss segment by not explicitly initializing it to zero. This significantly reduces the size of executables, and probably avoids some disk accesses on program startup. It's conceivable that it might break a port that I didn't test. - Fixed EMX_MAKEFILE to reflect the gc_c++.h to gc_cpp.h renaming which occurred a while ago. Since 4.9: - Fixed a typo around a call to GC_collect_or_expand in alloc.c. It broke handling of out of memory. (Thanks to Patrick Beard for noticing.) --------------------------------------------------------------------------- From adrian@cougar.aceonline.com.au Mon Feb 17 17:12:33 1997 Return-Path: Received: from cougar.aceonline.com.au (adrian@cougar.aceonline.com.au [203.103.81.36]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id RAA01281 for ; Mon, 17 Feb 1997 17:12:31 -0800 (PST) Received: from localhost (adrian@localhost) by cougar.aceonline.com.au (8.8.4/8.7) with SMTP id JAA03751; Tue, 18 Feb 1997 09:13:29 +0800 Date: Tue, 18 Feb 1997 09:13:29 +0800 (WST) From: Adrian Chadd To: Phillip Musumeci cc: "Jordan K. Hubbard" , auditors@freebsd.org Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org In-Reply-To: <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Just wondering, but I heard a while back that there is a copy of a range-checking GCC out there. Anyone confirm / deny this? Cya. -- Adrian Chadd | Windows 95 - the XT emulator for | your 486 and above! | Being superstitious is bad luck. From obrien@dragon.nuxi.com Mon Feb 17 17:42:39 1997 Return-Path: Received: from relay.nuxi.com (nuxi.ucdavis.edu [128.120.37.176]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id RAA02824 for ; Mon, 17 Feb 1997 17:42:36 -0800 (PST) Received: from dragon.nuxi.com (reqd-004.ucdavis.edu [128.120.251.124]) by relay.nuxi.com (8.8.4/8.6.12) with ESMTP id RAA07594 for ; Mon, 17 Feb 1997 17:42:48 -0800 (PST) Received: (from obrien@localhost) by dragon.nuxi.com (8.8.5/8.7.3) id BAA25438; Tue, 18 Feb 1997 01:42:41 GMT Message-ID: <19970217174240.SU36434@dragon.nuxi.com> Date: Mon, 17 Feb 1997 17:42:40 -0800 From: obrien@NUXI.com (David O'Brien) To: auditors@freebsd.org Subject: Re: W: Assignment References: <199702170741.IAA08566@nixpbe.pdb.sni.de> <28391.856226877@time.cdrom.com> X-Mailer: Mutt 0.60_p2-3,5,8-9 Mime-Version: 1.0 X-Disclaimer: Mutt Bites! Organization: The NUXI *BSD group X-PGP-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Keyid: 34F9F9D5 In-Reply-To: <28391.856226877@time.cdrom.com>; from Jordan K. Hubbard on Feb 17, 1997 16:47:57 -0800 Jordan K. Hubbard writes: > > Well, if you've got FTP and are only working on small pieces at a time, > I can suggest the source tree at: > > ftp://ftp.freebsd.org/pub/FreeBSD/FreeBSD-current/src > > Most people seem to forget that resource exists. :-) There's also the RCS files, if you want to use nice tools: (and send rcsdiffs) ftp://ftp.freebsd.org/pub/FreeBSD/FreeBSD-CVS/src -- -- David (obrien@NUXI.com -or- obrien@FreeBSD.org) From obrien@dragon.nuxi.com Mon Feb 17 17:47:13 1997 Return-Path: Received: from relay.nuxi.com (nuxi.ucdavis.edu [128.120.37.176]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id RAA03034 for ; Mon, 17 Feb 1997 17:47:12 -0800 (PST) Received: from dragon.nuxi.com (reqd-004.ucdavis.edu [128.120.251.124]) by relay.nuxi.com (8.8.4/8.6.12) with ESMTP id RAA07607; Mon, 17 Feb 1997 17:47:20 -0800 (PST) Received: (from obrien@localhost) by dragon.nuxi.com (8.8.5/8.7.3) id BAA25467; Tue, 18 Feb 1997 01:47:10 GMT Message-ID: <19970217174708.WW21582@dragon.nuxi.com> Date: Mon, 17 Feb 1997 17:47:08 -0800 From: obrien@NUXI.com (David O'Brien) To: adrian@cougar.aceonline.com.au (Adrian Chadd) Cc: auditors@freebsd.org Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org References: <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> X-Mailer: Mutt 0.60_p2-3,5,8-9 Mime-Version: 1.0 X-Disclaimer: Mutt Bites! Organization: The NUXI *BSD group X-PGP-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Keyid: 34F9F9D5 In-Reply-To: ; from Adrian Chadd on Feb 18, 1997 09:13:29 +0800 Adrian Chadd writes: > Just wondering, but I heard a while back that there is a copy of a > range-checking GCC out there. Anyone confirm / deny this? Confirm. The patches are to GCC 2.7.2 and are 689174 bytes. It built find on Solaris 2.5 (did it yesterday). I have not tried applying the patches to 2.7.2.2 to see if they will apply cleanly. Thanks to Eivind Eklund for providing the URL http://www-dse.doc.ic.ac.uk/~rj3/bounds-checking.html -- -- David (obrien@NUXI.com -or- obrien@FreeBSD.org) From imp@village.org Mon Feb 17 18:43:43 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id SAA05613 for ; Mon, 17 Feb 1997 18:43:41 -0800 (PST) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 0.56 #1) id E0vwfWd-0000Ap-00; Mon, 17 Feb 1997 19:43:23 -0700 To: obrien@NUXI.com (David O'Brien) Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org Cc: adrian@cougar.aceonline.com.au (Adrian Chadd), auditors@freebsd.org In-reply-to: Your message of "Mon, 17 Feb 1997 17:47:08 PST." <19970217174708.WW21582@dragon.nuxi.com> References: <19970217174708.WW21582@dragon.nuxi.com> <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> Date: Mon, 17 Feb 1997 19:43:23 -0700 From: Warner Losh Message-Id: In message <19970217174708.WW21582@dragon.nuxi.com> David O'Brien writes: : Adrian Chadd writes: : > Just wondering, but I heard a while back that there is a copy of a : > range-checking GCC out there. Anyone confirm / deny this? : : Confirm. The patches are to GCC 2.7.2 and are 689174 bytes. It built : find on Solaris 2.5 (did it yesterday). I have not tried applying the : patches to 2.7.2.2 to see if they will apply cleanly. : : Thanks to Eivind Eklund for providing the URL : http://www-dse.doc.ic.ac.uk/~rj3/bounds-checking.html While they are useful for other things, they are nearly useless in security things. The just check what happened on a given run, not what could happen on some sick and perverted edge case. You'd run it all day long, and since all file paths are << MAXPATHLEN, it won't even tell you where you put file names in buffers only 256 bytes long. Warner From imp@village.org Mon Feb 17 19:05:03 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id TAA07385 for ; Mon, 17 Feb 1997 19:05:02 -0800 (PST) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 0.56 #1) id E0vwfrD-0000Bs-00; Mon, 17 Feb 1997 20:04:39 -0700 Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org Cc: obrien@NUXI.com (David O'Brien), adrian@cougar.aceonline.com.au (Adrian Chadd), auditors@freebsd.org In-reply-to: Your message of "Mon, 17 Feb 1997 19:43:23 MST." References: <19970217174708.WW21582@dragon.nuxi.com> <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> Date: Mon, 17 Feb 1997 20:04:39 -0700 From: Warner Losh Message-Id: In message Warner Losh writes: : You'd run it all day long, and since all file paths are << MAXPATHLEN, : it won't even tell you where you put file names in buffers only 256 : bytes long. Ugg. I hate following up my own posts. Most file paths on most systems are << MAXPATHLEN. We had several problems in OI that only cropped up in purify when we tried a file path that was 527 characters or longer... Not exactly a common case. Warner From imp@village.org Mon Feb 17 19:17:42 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id TAA08359 for ; Mon, 17 Feb 1997 19:17:40 -0800 (PST) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 0.56 #1) id E0vwg3Z-0000D7-00; Mon, 17 Feb 1997 20:17:25 -0700 To: Phillip Musumeci Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org Cc: auditors@freebsd.org In-reply-to: Your message of "Tue, 18 Feb 1997 00:08:10 +1100." <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> References: <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> <5drhhn$4fd@bonkers.taronga.com> Date: Mon, 17 Feb 1997 20:17:25 -0700 From: Warner Losh Message-Id: In message <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> Phillip Musumeci writes: : Another thing that might be practical, or maybe not (I'm not sure), is : run-time checking for buffer overflows. There exists a set of memory : allocation routines that I have seen other people use in order to locate : memory leaks and find illegal memory accesses via pointers (e.g. writing : past the end of an allocated piece of memory storage --- search for string : "writing past"). I have seen tools that will do runtime closures on the data. Could this overflow, given where the data has been. Generally, one had to turn off these tests because they were too verbose for normal code. Which is to say generally normal code has lots of problems... I've not seen any of these compiler extensions recently. I wonder what happened to them. Guardian I think was the name of the product. A cooler idea than purify, but not as effective of telling you when you corrupt the heap, stack or other parts of memory. What is needed is some static flow analysis tool to point us at the hotspots that might overflow, and would ignore those cases that can't overflow (eg char buf[1024]; int i = rand(); sprintf( buf, "%d", i);)[*] However, that will do nothing for the race conditions, the bad uses of mktemp, et al, the sloppy use of seteuid(), badly written setuid progarms, etc. Warner [*] Before you object to newer, larger word machines, please consider that it would take ints with 1024 * ln(10)/ln(2) bits (about 3402) before you could even have a single character overflow. From root@implode.root.com Mon Feb 17 19:51:12 1997 Return-Path: Received: from root.com (implode.root.com [198.145.90.17]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id TAA11086 for ; Mon, 17 Feb 1997 19:51:09 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by root.com (8.8.5/8.6.5) with SMTP id TAA03484; Mon, 17 Feb 1997 19:52:06 -0800 (PST) Message-Id: <199702180352.TAA03484@root.com> X-Authentication-Warning: implode.root.com: localhost [127.0.0.1] didn't use HELO protocol To: Warner Losh cc: Brian Tao , auditors@freebsd.org Subject: Re: Register pointers in gcc? In-reply-to: Your message of "Mon, 17 Feb 1997 14:44:15 MST." From: David Greenman Reply-To: dg@root.com Date: Mon, 17 Feb 1997 19:52:06 -0800 Sender: root@implode.root.com >In message <199702170404.UAA04242@root.com> David Greenman writes: >: >I'm a little surprised that the kernel doesn't barf on the string >: >being >> MAXPATHLEN, but that's a whole other set of bugs that I don't >: >: It would be a bug if the kernel didn't allow args being > MAXPATHLEN. >: The only restriction is that the total length of all of the arguments plus >: environment must be less than ARG_MAX, which is currently 64K bytes. > >Even for the path of the executable to execute? Um, well, yeah, I suppose that would be reasonable. It's not checked right now because there is no special treatment of argv[0] when copying out the args. The path is of course checked (as a function of the namei() implied open) when the binary is actually mapped... -DG David Greenman Core-team/Principal Architect, The FreeBSD Project From ppb@baloo.tcp.co.uk Mon Feb 17 23:10:10 1997 Return-Path: Received: from zeus.tcp.net.uk (zeus.tcp.net.uk [195.80.0.33]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id WAA05788 for ; Mon, 17 Feb 1997 22:29:20 -0800 (PST) Received: from ka.home.hft.co.uk (root@al047.du.pipex.com [193.130.251.47]) by zeus.tcp.net.uk (8.7.6/8.7) with ESMTP id GAA16347 for ; Tue, 18 Feb 1997 06:28:38 GMT Received: (from root@localhost) by ka.home.hft.co.uk (8.7.4/8.6.9) id GAA17285; Tue, 18 Feb 1997 06:28:34 GMT Received: from ceres.brunel.ac.uk (pp@ceres.brunel.ac.uk [134.83.176.3]) by zeus.tcp.net.uk (8.7.6/8.7) with SMTP id AAA04461 for ; Tue, 18 Feb 1997 00:53:45 GMT Received: from freefall.freebsd.org by ceres.brunel.ac.uk with SMTP (PP); Tue, 18 Feb 1997 00:53:09 +0000 Received: from time.cdrom.com (time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA29615 for ; Mon, 17 Feb 1997 16:51:15 -0800 (PST) Received: from time.cdrom.com (localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id QAA28445 for ; Mon, 17 Feb 1997 16:51:12 -0800 (PST) Prev-Resent: Mon, 17 Feb 1997 16:51:11 -0800 Prev-Resent: "auditors@freebsd.org " Received: from mirriwinni.cse.rmit.edu.au (pm.cse.rmit.EDU.AU [131.170.118.50]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id FAA01569 for ; Mon, 17 Feb 1997 05:08:14 -0800 (PST) Received: (from phillip@localhost) by mirriwinni.cse.rmit.edu.au (8.8.3/8.6.12) id AAA26318; Tue, 18 Feb 1997 00:08:10 +1100 (EST) Date: Tue, 18 Feb 1997 00:08:10 +1100 (EST) Message-Id: <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> From: Phillip Musumeci X-Orig-To: "Jordan K. Hubbard" In-reply-to: "Jordan K. Hubbard"'s message of 11 Feb 1997 22:39:19 -0600 Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org References: <5drhhn$4fd@bonkers.taronga.com> Resent-To: auditors@FreeBSD.ORG Resent-Date: Mon, 17 Feb 1997 16:51:11 -0800 Resent-Message-ID: <28442.856227071@time.cdrom.com> Resent-From: "Jordan K. Hubbard" X-Processed-By: smtp-pop3 2.0 at TCP Ltd X-MAIL-From: jkh@time.cdrom.com To: ppb@baloo.tcp.co.uk X-UIDL: 0fb962433a954ed777c2f802c02d9faf Hi, Jordan> ... to begin a much more serious and comprehensive security Jordan> audit, we will take advantage of this opportunity to see that Jordan> all code in the FreeBSD source tree, old and new alike, is Jordan> reviewed line by line for buffer overflows, unguarded copies, Jordan> back doors, whatever. I think this source code checking is a good idea. Another thing that might be practical, or maybe not (I'm not sure), is run-time checking for buffer overflows. There exists a set of memory allocation routines that I have seen other people use in order to locate memory leaks and find illegal memory accesses via pointers (e.g. writing past the end of an allocated piece of memory storage --- search for string "writing past"). It has been ported to FreeBSD, and here is the README file. Maybe this alternative memory allocation library could be used whenever someone wanted to "test drive" a utility and look for memory leaks. The default make could use the normal memory allocation. As I said, this is just an idea and I'm not sure how practical it would be to use it for FreeBSD testing. I thought I recognised the name of the guy who did the FreeBSD though, so maybe he is closely associated with the FreeBSD team and has already suggested this. regards, phillip p.s. I became aware of this set of memory management tools because it is used in the public domain RLaB matrix maths package. --------------------------------------------------------------------------- Copyright 1988, 1989 Hans-J. Boehm, Alan J. Demers Copyright (c) 1991-1995 by Xerox Corporation. All rights reserved. THIS MATERIAL IS PROVIDED AS IS, WITH ABSOLUTELY NO WARRANTY EXPRESSED OR IMPLIED. ANY USE IS AT YOUR OWN RISK. Permission is hereby granted to use or copy this program for any purpose, provided the above notices are retained on all copies. Permission to modify the code and to distribute modified code is granted, provided the above notices are retained, and a notice that the code was modified is included with the above copyright notice. This is version 4.10 of a conservative garbage collector for C and C++. HISTORY - Early versions of this collector were developed as a part of research projects supported in part by the National Science Foundation and the Defense Advance Research Projects Agency. Much of the code was rewritten by Hans-J. Boehm at Xerox PARC. The SPARC specific code was contributed by Mark Weiser (weiser@parc.xerox.com). The Encore Multimax modifications were supplied by Kevin Kenny (kenny@m.cs.uiuc.edu). The adaptation to the RT is largely due to Vernon Lee (scorpion@rice.edu), on machines made available by IBM. Much of the HP specific code and a number of good suggestions for improving the generic code are due to Walter Underwood (wunder@hp-ses.sde.hp.com). Robert Brazile (brazile@diamond.bbn.com) originally supplied the ULTRIX code. Al Dosser (dosser@src.dec.com) and Regis Cridlig (Regis.Cridlig@cl.cam.ac.uk) subsequently provided updates and information on variation between ULTRIX systems. Parag Patel (parag@netcom.com) supplied the A/UX code. Jesper Peterson(jep@mtiame.mtia.oz.au) and Michel Schinz supplied the Amiga port. Thomas Funke (thf@zelator.in-berlin.de(?)) and Brian D.Carlstrom (bdc@clark.lcs.mit.edu) supplied the NeXT ports. Douglas Steel (doug@wg.icl.co.uk) provided ICL DRS6000 code. Bill Janssen (janssen@parc.xerox.com) supplied the SunOS dynamic loader specific code. Manuel Serrano (serrano@cornas.inria.fr) supplied linux and Sony News specific code. Al Dosser provided Alpha/OSF/1 code. He and Dave Detlefs(detlefs@src.dec.com) also provided several generic bug fixes. Alistair G. Crooks(agc@uts.amdahl.com) supplied the NetBSD and 386BSD ports. Jeffrey Hsu (hsu@soda.berkeley.edu) provided the FreeBSD port. Brent Benson (brent@jade.ssd.csd.harris.com) ported the collector to a Motorola 88K processor running CX/UX (Harris NightHawk). Ari Huttunen (Ari.Huttunen@hut.fi) generalized the OS/2 port to nonIBM development environments (a nontrivial task). Patrick Beard (beard@cs.ucdavis.edu) provided the initial MacOS port. David Chase, then at Olivetti Research, suggested several improvements. Scott Schwartz (schwartz@groucho.cse.psu.edu) supplied some of the code to save and print call stacks for leak detection on a SPARC. Jesse Hull and John Ellis supplied the C++ interface code. Zhong Shao performed much of the experimentation that led to the current typed allocation facility. (His dynamic type inference code hasn't made it into the released version of the collector, yet.) (Blame for misinstallation of these modifications goes to the first author, however.) This is intended to be a general purpose, garbage collecting storage allocator. The algorithms used are described in: Boehm, H., and M. Weiser, "Garbage Collection in an Uncooperative Environment", Software Practice & Experience, September 1988, pp. 807-820. Boehm, H., A. Demers, and S. Shenker, "Mostly Parallel Garbage Collection", Proceedings of the ACM SIGPLAN '91 Conference on Programming Language Design and Implementation, SIGPLAN Notices 26, 6 (June 1991), pp. 157-164. Boehm, H., "Space Efficient Conservative Garbage Collection", Proceedings of the ACM SIGPLAN '91 Conference on Programming Language Design and Implementation, SIGPLAN Notices 28, 6 (June 1993), pp. 197-206. Possible interactions between the collector and optimizing compilers are discussed in Boehm, H., and D. Chase, "A Proposal for GC-safe C Compilation", The Journal of C Language Translation 4, 2 (December 1992). (Also available from parcftp.xerox.com:pub/gc, among other places.) Unlike the collector described in the second reference, this collector operates either with the mutator stopped during the entire collection (default) or incrementally during allocations. (The latter is supported on only a few machines.) It does not rely on threads, but is intended to be thread-safe. Some of the ideas underlying the collector have previously been explored by others. (Doug McIlroy wrote a vaguely similar collector that is part of version 8 UNIX (tm).) However none of this work appears to have been widely disseminated. Rudimentary tools for use of the collector as a leak detector are included, as is a fairly sophisticated string package "cord" that makes use of the collector. (See cord/README.) GENERAL DESCRIPTION This is a garbage collecting storage allocator that is intended to be used as a plug-in replacement for C's malloc. Since the collector does not require pointers to be tagged, it does not attempt to ensure that all inaccessible storage is reclaimed. However, in our experience, it is typically more successful at reclaiming unused memory than most C programs using explicit deallocation. Unlike manually introduced leaks, the amount of unreclaimed memory typically stays bounded. In the following, an "object" is defined to be a region of memory allocated by the routines described below. Any objects not intended to be collected must be pointed to either from other such accessible objects, or from the registers, stack, data, or statically allocated bss segments. Pointers from the stack or registers may point to anywhere inside an object. However, it is usually assumed that all pointers originating in the heap point to the beginning of an object. (This does not disallow interior pointers; it simply requires that there must be a pointer to the beginning of every accessible object, in addition to any interior pointers.) There are two facilities for altering this behavior. The macro ALL_INTERIOR_POINTERS may be defined in gc_private.h to cause any pointer into an object (or one past the end) to retain the object. A routine GC_register_displacement is provided to allow for more controlled interior pointer use in the heap. Defining ALL_INTERIOR_POINTERS is somewhat dangerous, in that it can result in unnecessary memory retention. However this is much less of a problem than with older collector versions. The routine GC_register_displacement is described in gc.h. Note that pointers inside memory allocated by the standard "malloc" are not seen by the garbage collector. Thus objects pointed to only from such a region may be prematurely deallocated. It is thus suggested that the standard "malloc" be used only for memory regions, such as I/O buffers, that are guaranteed not to contain pointers. Pointers in C language automatic, static, or register variables, are correctly recognized. (Note that GC_malloc_uncollectable has semantics similar to standard malloc, but allocates objects that are traced by the collector.) The collector does not generally know how to find pointers in data areas that are associated with dynamic libraries. This is easy to remedy IF you know how to find those data areas on your operating system (see GC_add_roots). Code for doing this under SunOS and IRIX 5.X is included (see dynamic_load.c). Note that the garbage collector does not need to be informed of shared read-only data. However if the shared library mechanism can introduce discontiguous data areas that may contain pointers, then the collector does need to be informed. Signal processing for most signals may be deferred during collection, and during uninterruptible parts of the allocation process. Unlike standard ANSI C mallocs, it can be safe to invoke malloc from a signal handler while another malloc is in progress, provided the original malloc is not restarted. (Empirically, many UNIX applications already assume this.) To obtain this level of signal safety, remove the definition of -DNO_SIGNALS in Makefile. The allocator/collector can also be configured for thread-safe operation. (Full signal safety can also be achieved, but only at the cost of two system calls per malloc, which is usually unacceptable.) INSTALLATION AND PORTABILITY As distributed, the macro SILENT is defined in Makefile. In the event of problems, this can be removed to obtain a moderate amount of descriptive output for each collection. (The given statistics exhibit a few peculiarities. Things don't appear to add up for a variety of reasons, most notably fragmentation losses. These are probably much more significant for the contrived program "test.c" than for your application.) Note that typing "make test" will automatically build the collector and then run setjmp_test and gctest. Setjmp_test will give you information about configuring the collector, which is useful primarily if you have a machine that's not already supported. Gctest is a somewhat superficial test of collector functionality. Failure is indicated by a core dump or a message to the effect that the collector is broken. Gctest takes about 35 seconds to run on a SPARCstation 2. On a slower machine, expect it to take a while. It may use up to 8 MB of memory. (The multi-threaded version will use more.) "Make test" will also, as its last step, attempt to build and test the "cord" string library. This will fail without an ANSI C compiler. The Makefile will generate a library gc.a which you should link against. Typing "make cords" will add the cord library to gc.a. Note that this requires an ANSI C compiler. It is suggested that if you need to replace a piece of the collector (e.g. GC_mark_rts.c) you simply list your version ahead of gc.a on the ld command line, rather than replacing the one in gc.a. (This will generate numerous warnings under some versions of AIX, but it still works.) All include files that need to be used by clients will be put in the include subdirectory. (Normally this is just gc.h. "Make cords" adds "cord.h" and "ec.h".) The collector currently is designed to run essentially unmodified on the following machines (most of the operating systems mentioned are trademarks of their respective holders): Sun 3 Sun 4 under SunOS 4.X or Solaris2.X (with or without threads) Vax under 4.3BSD, Ultrix Intel 386 or 486 under most operating systems, but not MSDOS. (Win32S is somewhat supported, so it is possible to build applications for Windows 3.1. There exists a port to DOS + 32 bit extender for at least one 32 bit extender. However, I don't have source for this.) Sequent Symmetry (single threaded) Encore Multimax (single threaded) MIPS M/120 (and presumably M/2000) (RISC/os 4.0 with BSD libraries) IBM PC/RT (Berkeley UNIX) IBM RS/6000 HP9000/300 HP9000/700 DECstations under Ultrix DEC Alpha running OSF/1 SGI workstations under IRIX 4 & 5 Sony News Apple Macintosh under A/UX or MacOS Commodore Amiga (see README.amiga) NeXT machines In a few cases (Amiga, OS/2, Win32, MacOS) a separate makefile or equivalent is supplied. Dynamic libraries are completely supported only under SunOS (and even that support is not functional on the last Sun 3 release), IRIX 5, Win32 (not Win32S) and OSF/1 on DEC AXP machines. On other machines we recommend that you do one of the following: 1) Add dynamic library support (and send us the code). 2) Use static versions of the libraries. 3) Arrange for dynamic libraries to use the standard malloc. This is still dangerous if the library stores a pointer to a garbage collected object. But nearly all standard interfaces prohibit this, because they deal correctly with pointers to stack allocated objects. (Strtok is an exception. Don't use it.) In all cases we assume that pointer alignment is consistent with that enforced by the standard C compilers. If you use a nonstandard compiler you may have to adjust the alignment parameters defined in gc_priv.h. A port to a machine that is not byte addressed, or does not use 32 bit or 64 bit addresses will require a major effort. A port to MSDOS is hard, unless you are willing to assume an 80386 or better, and that only flat 32 bit pointers will ever need to be seen by the collector. For machines not already mentioned, or for nonstandard compilers, the following are likely to require change: 1. The parameters in config.h. The parameters that will usually require adjustment are STACKBOTTOM, ALIGNMENT and DATASTART. Setjmp_test prints its guesses of the first two. DATASTART should be an expression for computing the address of the beginning of the data segment. This can often be &etext. But some memory management units require that there be some unmapped space between the text and the data segment. Thus it may be more complicated. On UNIX systems, this is rarely documented. But the adb "$m" command may be helpful. (Note that DATASTART will usually be a function of &etext. Thus a single experiment is usually insufficient.) STACKBOTTOM is used to initialize GC_stackbottom, which should be a sufficient approximation to the coldest stack address. On some machines, it is difficult to obtain such a value that is valid across a variety of MMUs, OS releases, etc. A number of alternatives exist for using the collector in spite of this. See the discussion in config.h immediately preceding the various definitions of STACKBOTTOM. 2. mach_dep.c. The most important routine here is one to mark from registers. The distributed file includes a generic hack (based on setjmp) that happens to work on many machines, and may work on yours. Try compiling and running setjmp_t.c to see whether it has a chance of working. (This is not correct C, so don't blame your compiler if it doesn't work. Based on limited experience, register window machines are likely to cause trouble. If your version of setjmp claims that all accessible variables, including registers, have the value they had at the time of the longjmp, it also will not work. Vanilla 4.2 BSD on Vaxen makes such a claim. SunOS does not.) If your compiler does not allow in-line assembly code, or if you prefer not to use such a facility, mach_dep.c may be replaced by a .s file (as we did for the MIPS machine and the PC/RT). At this point enough architectures are supported by mach_dep.c that you will rarely need to do more than adjust for assembler syntax. 3. os_dep.c (and gc_priv.h). Several kinds of operating system dependent routines reside here. Many are optional. Several are invoked only through corresponding macros in gc_priv.h, which may also be redefined as appropriate. The routine GC_register_data_segments is crucial. It registers static data areas that must be traversed by the collector. (User calls to GC_add_roots may sometimes be used for similar effect.) Routines to obtain memory from the OS also reside here. Alternatively this can be done entirely by the macro GET_MEM defined in gc_priv.h. Routines to disable and reenable signals also reside here if they are need by the macros DISABLE_SIGNALS and ENABLE_SIGNALS defined in gc_priv.h. In a multithreaded environment, the macros LOCK and UNLOCK in gc_priv.h will need to be suitably redefined. The incremental collector requires page dirty information, which is acquired through routines defined in os_dep.c. Unless directed otherwise by config.h, these are implemented as stubs that simply treat all pages as dirty. (This of course makes the incremental collector much less useful.) 4. dyn_load.c This provides a routine that allows the collector to scan data segments associated with dynamic libraries. Often it is not necessary to provide this routine unless user-written dynamic libraries are used. For a different version of UN*X or different machines using the Motorola 68000, Vax, SPARC, 80386, NS 32000, PC/RT, or MIPS architecture, it should frequently suffice to change definitions in config.h. THE C INTERFACE TO THE ALLOCATOR The following routines are intended to be directly called by the user. Note that usually only GC_malloc is necessary. GC_clear_roots and GC_add_roots calls may be required if the collector has to trace from nonstandard places (e.g. from dynamic library data areas on a machine on which the collector doesn't already understand them.) On some machines, it may be desirable to set GC_stacktop to a good approximation of the stack base. (This enhances code portability on HP PA machines, since there is no good way for the collector to compute this value.) Client code may include "gc.h", which defines all of the following, plus many others. 1) GC_malloc(nbytes) - allocate an object of size nbytes. Unlike malloc, the object is cleared before being returned to the user. Gc_malloc will invoke the garbage collector when it determines this to be appropriate. GC_malloc may return 0 if it is unable to acquire sufficient space from the operating system. This is the most probable consequence of running out of space. Other possible consequences are that a function call will fail due to lack of stack space, or that the collector will fail in other ways because it cannot maintain its internal data structures, or that a crucial system process will fail and take down the machine. Most of these possibilities are independent of the malloc implementation. 2) GC_malloc_atomic(nbytes) - allocate an object of size nbytes that is guaranteed not to contain any pointers. The returned object is not guaranteed to be cleared. (Can always be replaced by GC_malloc, but results in faster collection times. The collector will probably run faster if large character arrays, etc. are allocated with GC_malloc_atomic than if they are statically allocated.) 3) GC_realloc(object, new_size) - change the size of object to be new_size. Returns a pointer to the new object, which may, or may not, be the same as the pointer to the old object. The new object is taken to be atomic iff the old one was. If the new object is composite and larger than the original object, then the newly added bytes are cleared (we hope). This is very likely to allocate a new object, unless MERGE_SIZES is defined in gc_priv.h. Even then, it is likely to recycle the old object only if the object is grown in small additive increments (which, we claim, is generally bad coding practice.) 4) GC_free(object) - explicitly deallocate an object returned by GC_malloc or GC_malloc_atomic. Not necessary, but can be used to minimize collections if performance is critical. Probably a performance loss for very small objects (<= 8 bytes). 5) GC_expand_hp(bytes) - Explicitly increase the heap size. (This is normally done automatically if a garbage collection failed to GC_reclaim enough memory. Explicit calls to GC_expand_hp may prevent unnecessarily frequent collections at program startup.) 6) GC_malloc_ignore_off_page(bytes) - identical to GC_malloc, but the client promises to keep a pointer to the somewhere within the first 256 bytes of the object while it is live. (This pointer should nortmally be declared volatile to prevent interference from compiler optimizations.) This is the recommended way to allocate anything that is likely to be larger than 100Kbytes or so. (GC_malloc may result in failure to reclaim such objects.) 7) GC_set_warn_proc(proc) - Can be used to redirect warnings from the collector. Such warnings should be rare, and should not be ignored during code development. 8) GC_enable_incremental() - Enables generational and incremental collection. Useful for large heaps on machines that provide access to page dirty information. Some dirty bit implementations may interfere with debugging (by catching address faults) and place restrictions on heap arguments to system calls (since write faults inside a system call may not be handled well). 9) Several routines to allow for registration of finalization code. User supplied finalization code may be invoked when an object becomes unreachable. To call (*f)(obj, x) when obj becomes inaccessible, use GC_register_finalizer(obj, f, x, 0, 0); For more sophisticated uses, and for finalization ordering issues, see gc.h. The global variable GC_free_space_divisor may be adjusted up from its default value of 4 to use less space and more collection time, or down for the opposite effect. Setting it to 1 or 0 will effectively disable collections and cause all allocations to simply grow the heap. The variable GC_non_gc_bytes, which is normally 0, may be changed to reflect the amount of memory allocated by the above routines that should not be considered as a candidate for collection. Careless use may, of course, result in excessive memory consumption. Some additional tuning is possible through the parameters defined near the top of gc_priv.h. If only GC_malloc is intended to be used, it might be appropriate to define: #define malloc(n) GC_malloc(n) #define calloc(m,n) GC_malloc((m)*(n)) For small pieces of VERY allocation intensive code, gc_inl.h includes some allocation macros that may be used in place of GC_malloc and friends. All externally visible names in the garbage collector start with "GC_". To avoid name conflicts, client code should avoid this prefix, except when accessing garbage collector routines or variables. There are provisions for allocation with explicit type information. This is rarely necessary. Details can be found in gc_typed.h. THE C++ INTERFACE TO THE ALLOCATOR: The Ellis-Hull C++ interface to the collector is included in the collector distribution. If you intend to use this, type "make c++" after the initial build of the collector is complete. See gc_cpp.h for the definition of the interface. This interface tries to approximate the Ellis-Detlefs C++ garbage collection proposal without compiler changes. Cautions: 1. Arrays allocated without new placement syntax are allocated as uncollectable objects. They are traced by the collector, but will not be reclaimed. 2. Failure to use "make c++" in combination with (1) will result in arrays allocated using the default new operator. This is likely to result in disaster without linker warnings. 3. If your compiler supports an overloaded new[] operator, then gc_cpp.cc and gc_cpp.h should be suitably modified. 4. Many current C++ compilers have deficiencies that break some of the functionality. See the comments in gc_cpp.h for suggested workarounds. USE AS LEAK DETECTOR: The collector may be used to track down leaks in C programs that are intended to run with malloc/free (e.g. code with extreme real-time or portability constraints). To do so define FIND_LEAK in Makefile This will cause the collector to invoke the report_leak routine defined near the top of reclaim.c whenever an inaccessible object is found that has not been explicitly freed. Productive use of this facility normally involves redefining report_leak to do something more intelligent. This typically requires annotating objects with additional information (e.g. creation time stack trace) that identifies their origin. Such code is typically not very portable, and is not included here, except on SPARC machines. If all objects are allocated with GC_DEBUG_MALLOC (see next section), then the default version of report_leak will report the source file and line number at which the leaked object was allocated. This may sometimes be sufficient. (On SPARC/SUNOS4 machines, it will also report a cryptic stack trace. This can often be turned into a sympolic stack trace by invoking program "foo" with "callprocs foo". Callprocs is a short shell script that invokes adb to expand program counter values to symbolic addresses. It was largely supplied by Scott Schwartz.) Note that the debugging facilities described in the next section can sometimes be slightly LESS effective in leak finding mode, since in leak finding mode, GC_debug_free actually results in reuse of the object. (Otherwise the object is simply marked invalid.) Also note that the test program is not designed to run meaningfully in FIND_LEAK mode. Use "make gc.a" to build the collector. DEBUGGING FACILITIES: The routines GC_debug_malloc, GC_debug_malloc_atomic, GC_debug_realloc, and GC_debug_free provide an alternate interface to the collector, which provides some help with memory overwrite errors, and the like. Objects allocated in this way are annotated with additional information. Some of this information is checked during garbage collections, and detected inconsistencies are reported to stderr. Simple cases of writing past the end of an allocated object should be caught if the object is explicitly deallocated, or if the collector is invoked while the object is live. The first deallocation of an object will clear the debugging info associated with an object, so accidentally repeated calls to GC_debug_free will report the deallocation of an object without debugging information. Out of memory errors will be reported to stderr, in addition to returning NIL. GC_debug_malloc checking during garbage collection is enabled with the first call to GC_debug_malloc. This will result in some slowdown during collections. If frequent heap checks are desired, this can be achieved by explicitly invoking GC_gcollect, e.g. from the debugger. GC_debug_malloc allocated objects should not be passed to GC_realloc or GC_free, and conversely. It is however acceptable to allocate only some objects with GC_debug_malloc, and to use GC_malloc for other objects, provided the two pools are kept distinct. In this case, there is a very low probablility that GC_malloc allocated objects may be misidentified as having been overwritten. This should happen with probability at most one in 2**32. This probability is zero if GC_debug_malloc is never called. GC_debug_malloc, GC_malloc_atomic, and GC_debug_realloc take two additional trailing arguments, a string and an integer. These are not interpreted by the allocator. They are stored in the object (the string is not copied). If an error involving the object is detected, they are printed. The macros GC_MALLOC, GC_MALLOC_ATOMIC, GC_REALLOC, GC_FREE, and GC_REGISTER_FINALIZER are also provided. These require the same arguments as the corresponding (nondebugging) routines. If gc.h is included with GC_DEBUG defined, they call the debugging versions of these functions, passing the current file name and line number as the two extra arguments, where appropriate. If gc.h is included without GC_DEBUG defined, then all these macros will instead be defined to their nondebugging equivalents. (GC_REGISTER_FINALIZER is necessary, since pointers to objects with debugging information are really pointers to a displacement of 16 bytes form the object beginning, and some translation is necessary when finalization routines are invoked. For details, about what's stored in the header, see the definition of the type oh in debug_malloc.c) INCREMENTAL/GENERATIONAL COLLECTION: The collector normally interrupts client code for the duration of a garbage collection mark phase. This may be unacceptable if interactive response is needed for programs with large heaps. The collector can also run in a "generational" mode, in which it usually attempts to collect only objects allocated since the last garbage collection. Furthermore, in this mode, garbage collections run mostly incrementally, with a small amount of work performed in response to each of a large number of GC_malloc requests. This mode is enabled by a call to GC_enable_incremental(). Incremental and generational collection is effective in reducing pause times only if the collector has some way to tell which objects or pages have been recently modified. The collector uses two sources of information: 1. Information provided by the VM system. This may be provided in one of several forms. Under Solaris 2.X (and potentially under other similar systems) information on dirty pages can be read from the /proc file system. Under other systems (currently SunOS4.X) it is possible to write-protect the heap, and catch the resulting faults. On these systems we require that system calls writing to the heap (other than read) be handled specially by client code. See os_dep.c for details. 2. Information supplied by the programmer. We define "stubborn" objects to be objects that are rarely changed. Such an object can be allocated (and enabled for writing) with GC_malloc_stubborn. Once it has been initialized, the collector should be informed with a call to GC_end_stubborn_change. Subsequent writes that store pointers into the object must be preceded by a call to GC_change_stubborn. This mechanism performs best for objects that are written only for initialization, and such that only one stubborn object is writable at once. It is typically not worth using for short-lived objects. Stubborn objects are treated less efficiently than pointerfree (atomic) objects. A rough rule of thumb is that, in the absence of VM information, garbage collection pauses are proportional to the amount of pointerful storage plus the amount of modified "stubborn" storage that is reachable during the collection. Initial allocation of stubborn objects takes longer than allocation of other objects, since other data structures need to be maintained. We recommend against random use of stubborn objects in client code, since bugs caused by inappropriate writes to stubborn objects are likely to be very infrequently observed and hard to trace. However, their use may be appropriate in a few carefully written library routines that do not make the objects themselves available for writing by client code. BUGS: Any memory that does not have a recognizable pointer to it will be reclaimed. Exclusive-or'ing forward and backward links in a list doesn't cut it. Some C optimizers may lose the last undisguised pointer to a memory object as a consequence of clever optimizations. This has almost never been observed in practice. Send mail to boehm@parc.xerox.com for suggestions on how to fix your compiler. This is not a real-time collector. In the standard configuration, percentage of time required for collection should be constant across heap sizes. But collection pauses will increase for larger heaps. (On SPARCstation 2s collection times will be on the order of 300 msecs per MB of accessible memory that needs to be scanned. Your mileage may vary.) The incremental/generational collection facility helps, but is portable only if "stubborn" allocation is used. Please address bug reports to boehm@parc.xerox.com. If you are contemplating a major addition, you might also send mail to ask whether it's already been done (or whether we tried and discarded it). RECENT VERSIONS: Version 1.3 and immediately preceding versions contained spurious assembly language assignments to TMP_SP. Only the assignment in the PC/RT code is necessary. On other machines, with certain compiler options, the assignments can lead to an unsaved register being overwritten. Known to cause problems under SunOS 3.5 WITHOUT the -O option. (With -O the compiler recognizes it as dead code. It probably shouldn't, but that's another story.) Version 1.4 and earlier versions used compile time determined values for the stack base. This no longer works on Sun 3s, since Sun 3/80s use a different stack base. We now use a straightforward heuristic on all machines on which it is known to work (incl. Sun 3s) and compile-time determined values for the rest. There should really be library calls to determine such values. Version 1.5 and earlier did not ensure 8 byte alignment for objects allocated on a sparc based machine. Version 1.8 added ULTRIX support in gc_private.h. Version 1.9 fixed a major bug in gc_realloc. Version 2.0 introduced a consistent naming convention for collector routines and added support for registering dynamic library data segments in the standard mark_roots.c. Most of the data structures were revamped. The treatment of interior pointers was completely changed. Finalization was added. Support for locking was added. Object kinds were added. We added a black listing facility to avoid allocating at addresses known to occur as integers somewhere in the address space. Much of this was accomplished by adapting ideas and code from the PCR collector. The test program was changed and expanded. Version 2.1 was the first stable version since 1.9, and added support for PPCR. Version 2.2 added debugging allocation, and fixed various bugs. Among them: - GC_realloc could fail to extend the size of the object for certain large object sizes. - A blatant subscript range error in GC_printf, which unfortunately wasn't exercised on machines with sufficient stack alignment constraints. - GC_register_displacement did the wrong thing if it was called after any allocation had taken place. - The leak finding code would eventually break after 2048 byte byte objects leaked. - interface.c didn't compile. - The heap size remained much too small for large stacks. - The stack clearing code behaved badly for large stacks, and perhaps on HP/PA machines. Version 2.3 added ALL_INTERIOR_POINTERS and fixed the following bugs: - Missing declaration of etext in the A/UX version. - Some PCR root-finding problems. - Blacklisting was not 100% effective, because the plausible future heap bounds were being miscalculated. - GC_realloc didn't handle out-of-memory correctly. - GC_base could return a nonzero value for addresses inside free blocks. - test.c wasn't really thread safe, and could erroneously report failure in a multithreaded environment. (The locking primitives need to be replaced for other threads packages.) - GC_CONS was thoroughly broken. - On a SPARC with dynamic linking, signals stayed diabled while the client code was running. (Thanks to Manuel Serrano at INRIA for reporting the last two.) Version 2.4 added GC_free_space_divisor as a tuning knob, added support for OS/2 and linux, and fixed the following bugs: - On machines with unaligned pointers (e.g. Sun 3), every 128th word could fail to be considered for marking. - Dynamic_load.c erroneously added 4 bytes to the length of the data and bss sections of the dynamic library. This could result in a bad memory reference if the actual length was a multiple of a page. (Observed on Sun 3. Can probably also happen on a Sun 4.) (Thanks to Robert Brazile for pointing out that the Sun 3 version was broken. Dynamic library handling is still broken on Sun 3s under 4.1.1U1, but apparently not 4.1.1. If you have such a machine, use -Bstatic.) Version 2.5 fixed the following bugs: - Removed an explicit call to exit(1) - Fixed calls to GC_printf and GC_err_printf, so the correct number of arguments are always supplied. The OS/2 C compiler gets confused if the number of actuals and the number of formals differ. (ANSI C doesn't require this to work. The ANSI sanctioned way of doing things causes too many compatibility problems.) Version 3.0 added generational/incremental collection and stubborn objects. Version 3.1 added the following features: - A workaround for a SunOS 4.X SPARC C compiler misfeature that caused problems when the collector was turned into a dynamic library. - A fix for a bug in GC_base that could result in a memory fault. - A fix for a performance bug (and several other misfeatures) pointed out by Dave Detlefs and Al Dosser. - Use of dirty bit information for static data under Solaris 2.X. - DEC Alpha/OSF1 support (thanks to Al Dosser). - Incremental collection on more platforms. - A more refined heap expansion policy. Less space usage by default. - Various minor enhancements to reduce space usage, and to reduce the amount of memory scanned by the collector. - Uncollectable allocation without per object overhead. - More conscientious handling of out-of-memory conditions. - Fixed a bug in debugging stubborn allocation. - Fixed a bug that resulted in occasional erroneous reporting of smashed objects with debugging allocation. - Fixed bogus leak reports of size 4096 blocks with FIND_LEAK. Version 3.2 fixed a serious and not entirely repeatable bug in the incremental collector. It appeared only when dirty bit info on the roots was available, which is normally only under Solaris. It also added GC_general_register_disappearing_link, and some testing code. Interface.c disappeared. Version 3.3 fixes several bugs and adds new ports: - PCR-specific bugs. - Missing locking in GC_free, redundant FASTUNLOCK in GC_malloc_stubborn, and 2 bugs in GC_unregister_disappearing_link. All of the above were pointed out by Neil Sharman (neil@cs.mu.oz.au). - Common symbols allocated by the SunOS4.X dynamic loader were not included in the root set. - Bug in GC_finalize (reported by Brian Beuning and Al Dosser) - Merged Amiga port from Jesper Peterson (untested) - Merged NeXT port from Thomas Funke (significantly modified and untested) Version 3.4: - Fixed a performance bug in GC_realloc. - Updated the amiga port. - Added NetBSD and 386BSD ports. - Added cord library. - Added trivial performance enhancement for ALL_INTERIOR_POINTERS. (Don't scan last word.) Version 3.5 - Minor collections now mark from roots only once, if that doesn't cause an excessive pause. - The stack clearing heuristic was refined to prevent anomalies with very heavily recursive programs and sparse stacks. - Fixed a bug that prevented mark stack growth in some cases. GC_objects_are_marked should be set to TRUE after a call to GC_push_roots and as part of GC_push_marked, since both can now set mark bits. I think this is only a performance bug, but I wouldn't bet on it. It's certainly very hard to argue that the old version was correct. - Fixed an incremental collection bug that prevented it from working at all when HBLKSIZE != getpagesize() - Changed dynamic_loading.c to include gc_priv.h before testing DYNAMIC_LOADING. SunOS dynamic library scanning must have been broken in 3.4. - Object size rounding now adapts to program behavior. - Added a workaround (provided by Manuel Serrano and colleagues) to a long-standing SunOS 4.X (and 3.X?) ld bug that I had incorrectly assumed to have been squished. The collector was broken if the text segment size was within 32 bytes of a multiple of 8K bytes, and if the beginning of the data segment contained interesting roots. The workaround assumes a demand-loadable executable. The original may have have "worked" in some other cases. - Added dynamic library support under IRIX5. - Added support for EMX under OS/2 (thanks to Ari Huttunen). Version 3.6: - fixed a bug in the mark stack growth code that was introduced in 3.4. - fixed Makefile to work around DEC AXP compiler tail recursion bug. Version 3.7: - Added a workaround for an HP/UX compiler bug. - Fixed another stack clearing performance bug. Reworked that code once more. Version 4.0: - Added support for Solaris threads (which was possible only by reimplementing some fraction of Solaris threads, since Sun doesn't currently make the thread debugging interface available). - Added non-threads win32 and win32S support. - (Grudgingly, with suitable muttering of obscenities) renamed files so that the collector distribution could live on a FAT file system. Files that are guaranteed to be useless on a PC still have long names. Gc_inline.h and gc_private.h still exist, but now just include gc_inl.h and gc_priv.h. - Fixed a really obscure bug in finalization that could cause undetected mark stack overflows. (I would be surprised if any real code ever tickled this one.) - Changed finalization code to dynamically resize the hash tables it maintains. (This probably does not matter for well- -written code. It no doubt does for C++ code that overuses destructors.) - Added typed allocation primitives. Rewrote the marker to accommodate them with more reasonable efficiency. This change should also speed up marking for GC_malloc allocated objects a little. See gc_typed.h for new primitives. - Improved debugging facilities slightly. Allocation time stack traces are now kept by default on SPARC/SUNOS4. (Thanks to Scott Schwartz.) - Added better support for small heap applications. - Significantly extended cord package. Fixed a bug in the implementation of lazily read files. Printf and friends now have cord variants. Cord traversals are a bit faster. - Made ALL_INTERIOR_POINTERS recognition the default. - Fixed de so that it can run in constant space, independent of file size. Added simple string searching to cords and de. - Added the Hull-Ellis C++ interface. - Added dynamic library support for OSF/1. (Thanks to Al Dosser and Tim Bingham at DEC.) - Changed argument to GC_expand_hp to be expressed in units of bytes instead of heap blocks. (Necessary since the heap block size now varies depending on configuration. The old version was never very clean.) - Added GC_get_heap_size(). The previous "equivalent" was broken. - Restructured the Makefile a bit. Since version 4.0: - Changed finalization implementation to guarantee that finalization procedures are called outside of the allocation lock, making direct use of the interface a little less dangerous. MAY BREAK EXISTING CLIENTS that assume finalizers are protected by a lock. Since there seem to be few multithreaded clients that use finalization, this is hopefully not much of a problem. - Fixed a gross bug in CORD_prev. - Fixed a bug in blacklst.c that could result in unbounded heap growth during startup on machines that do not clear memory obtained from the OS (e.g. win32S). - Ported de editor to win32/win32S. (This is now the only version with a mouse-sensitive UI.) - Added GC_malloc_ignore_off_page to allocate large arrays in the presence of ALL_INTERIOR_POINTERS. - Changed GC_call_with_alloc_lock to not disable signals in the single-threaded case. - Reduced retry count in GC_collect_or_expand for garbage collecting when out of memory. - Made uncollectable allocations bypass black-listing, as they should. - Fixed a bug in typed_test in test.c that could cause (legitimate) GC crashes. - Fixed some potential synchronization problems in finalize.c - Fixed a real locking problem in typd_mlc.c. - Worked around an AIX 3.2 compiler feature that results in out of bounds memory references. - Partially worked around an IRIX5.2 beta problem (which may or may not persist to the final release). - Fixed a bug in the heap integrity checking code that could result in explicitly deallocated objects being identified as smashed. Fixed a bug in the dbg_mlc stack saving code that caused old argument pointers to be considered live. - Fixed a bug in CORD_ncmp (and hence CORD_str). - Repaired the OS2 port, which had suffered from bit rot in 4.0. Worked around what appears to be CSet/2 V1.0 optimizer bug. - Fixed a Makefile bug for target "c++". Since version 4.1: - Multiple bug fixes/workarounds in the Solaris threads version. (It occasionally failed to locate some register contents for marking. It also turns out that thr_suspend and friends are unreliable in Solaris 2.3. Dirty bit reads appear to be unreliable under some weird circumstances. My stack marking code contained a serious performance bug. The new code is extremely defensive, and has not failed in several cpu hours of testing. But no guarantees ...) - Added MacOS support (thanks to Patrick Beard.) - Fixed several syntactic bugs in gc_c++.h and friends. (These didn't bother g++, but did bother most other compilers.) Fixed gc_c++.h finalization interface. (It didn't.) - 64 bit alignment for allocated objects was not guaranteed in a few cases in which it should have been. - Added GC_malloc_atomic_ignore_off_page. - Added GC_collect_a_little. - Added some prototypes to gc.h. - Some other minor bug fixes (notably in Makefile). - Fixed OS/2 / EMX port (thanks to Ari Huttunen). - Fixed AmigaDOS port. (thanks to Michel Schinz). - Fixed the DATASTART definition under Solaris. There was a 1 in 16K chance of the collector missing the first 64K of static data (and thus crashing). - Fixed some blatant anachronisms in the README file. - Fixed PCR-Makefile for upcoming PPCR release. Since version 4.2: - Fixed SPARC alignment problem with GC_DEBUG. - Fixed Solaris threads /proc workaround. The real problem was an interaction with mprotect. - Incorporated fix from Patrick Beard for gc_c++.h (now gc_cpp.h). - Slightly improved allocator space utilization by fixing the GC_size_map mechanism. - Integrated some Sony News and MIPS RISCos 4.51 patches. (Thanks to Nobuyuki Hikichi of Software Research Associates, Inc. Japan) - Fixed HP_PA alignment problem. (Thanks to xjam@cork.cs.berkeley.edu.) - Added GC_same_obj and friends. Changed GC_base to return 0 for pointers past the end of large objects. Improved GC_base performance with ALL_INTERIOR_POINTERS on machines with a slow integer mod operation. Added GC_PTR_ADD, GC_PTR_STORE, etc. to prepare for preprocessor. - changed the default on most UNIX machines to be that signals are not disabled during critical GC operations. This is still ANSI-conforming, though somewhat dangerous in the presence of signal handlers. But the performance cost of the alternative is sometimes problematic. Can be changed back with a minor Makefile edit. - renamed IS_STRING in gc.h, to CORD_IS_STRING, thus following my own naming convention. Added the function CORD_to_const_char_star. - Fixed a gross bug in GC_finalize. Symptom: occasional address faults in that function. (Thanks to Anselm Baird-Smith (Anselm.BairdSmith@inria.fr) - Added port to ICL DRS6000 running DRS/NX. Restructured things a bit to factor out common code, and remove obsolete code. Collector should now run under SUNOS5 with either mprotect or /proc dirty bits. (Thanks to Douglas Steel (doug@wg.icl.co.uk)). - More bug fixes and workarounds for Solaris 2.X. (These were mostly related to putting the collector in a dynamic library, which didn't really work before. Also SOLARIS_THREADS didn't interact well with dl_open.) Thanks to btlewis@eng.sun.com. - Fixed a serious performance bug on the DEC Alpha. The text segment was getting registered as part of the root set. (Amazingly, the result was still fast enough that the bug was not conspicuous.) The fix works on OSF/1, version 1.3. Hopefully it also works on other versions of OSF/1 ... - Fixed a bug in GC_clear_roots. - Fixed a bug in GC_generic_malloc_words_small that broke gc_inl.h. (Reported by Antoine de Maricourt. I broke it in trying to tweak the Mac port.) - Fixed some problems with cord/de under Linux. - Fixed some cord problems, notably with CORD_riter4. - Added DG/UX port. Thanks to Ben A. Mesander (ben@piglet.cr.usgs.gov) - Added finalization registration routines with weaker ordering constraints. (This is necessary for C++ finalization with multiple inheritance, since the compiler often adds self-cycles.) - Filled the holes in the SCO port. (Thanks to Michael Arnoldus .) - John Ellis' additions to the C++ support: From John: * I completely rewrote the documentation in the interface gc_c++.h (later renamed gc_cpp.h). I've tried to make it both clearer and more precise. * The definition of accessibility now ignores pointers from an finalizable object (an object with a clean-up function) to itself. This allows objects with virtual base classes to be finalizable by the collector. Compilers typically implement virtual base classes using pointers from an object to itself, which under the old definition of accessibility prevented objects with virtual base classes from ever being collected or finalized. * gc_cleanup now includes gc as a virtual base. This was enabled by the change in the definition of accessibility. * I added support for operator new[]. Since most (all?) compilers don't yet support operator new[], it is conditionalized on -DOPERATOR_NEW_ARRAY. The code is untested, but its trivial and looks correct. * The test program test_gc_c++ (later renamed test_cpp.cc) tries to test for the C++-specific functionality not tested by the other programs. - Added include to misc.c. (Needed for ppcr.) - Added PowerMac port. (Thanks to Patrick Beard again.) - Fixed "srcdir"-related Makefile problems. Changed things so that all externally visible include files always appear in the include subdirectory of the source. Made gc.h directly includable from C++ code. (These were at Per Bothner's suggestion.) - Changed Intel code to also mark from ebp (Kevin Warne's suggestion). - Renamed C++ related files so they could live in a FAT file system. (Charles Fiterman's suggestion.) - Changed Windows NT Makefile to include C++ support in gc.lib. Added C++ test as Makefile target. Since version 4.3: - ASM_CLEAR_CODE was erroneously defined for HP PA machines, resulting in a compile error. - Fixed OS/2 Makefile to create a library. (Thanks to Mark Boulter (mboulter@vnet.ibm.com)). - Gc_cleanup objects didn't work if they were created on the stack. Fixed. - One copy of Gc_cpp.h in the distribution was out of synch, and failed to document some known compiler problems with explicit destructor invocation. Partially fixed. There are probably other compilers on which gc_cleanup is miscompiled. - Fixed Makefile to pass C compiler flags to C++ compiler. - Added Mac fixes. - Fixed os_dep.c to work around what appears to be a new and different VirtualQuery bug under newer versions of win32S. - GC_non_gc_bytes was not correctly maintained by GC_free. Fixed. Thanks to James Clark (jjc@jclark.com). - Added GC_set_max_heap_size. - Changed allocation code to ignore blacklisting if it is preventing use of a very large block of memory. This has the advantage that naive code allocating very large objects is much more likely to work. The downside is you might no longer find out that such code should really use GC_malloc_ignore_off_page. - Changed GC_printf under win32 to close and reopen the file between calls. FAT file systems otherwise make the log file useless for debugging. - Added GC_try_to_collect and GC_get_bytes_since_gc. These allow starting an abortable collection during idle times. This facility does not require special OS support. (Thanks to Michael Spertus of Geodesic Systems for suggesting this. It was actually an easy addition. Kumar Srikantan previously added a similar facility to a now ancient version of the collector. At the time this was much harder, and the result was less convincing.) - Added some support for the Borland development environment. (Thanks to John Ellis and Michael Spertus.) - Removed a misfeature from checksums.c that caused unexpected heap growth. (Thanks to Scott Schwartz.) - Changed finalize.c to call WARN if it encounters a finalization cycle. WARN is defined in gc_priv.h to write a message, usually to stdout. In many environments, this may be inappropriate. - Renamed NO_PARAMS in gc.h to GC_NO_PARAMS, thus adhering to my own naming convention. - Added GC_set_warn_proc to intercept warnings. - Fixed Amiga port. (Thanks to Michel Schinz (schinz@alphanet.ch).) - Fixed a bug in mark.c that could result in an access to unmapped memory from GC_mark_from_mark_stack on machines with unaligned pointers. - Fixed a win32 specific performance bug that could result in scanning of objects allocated with the system malloc. - Added REDIRECT_MALLOC. Since version 4.4: - Fixed many minor and one major README bugs. (Thanks to Franklin Chen (chen@adi.com) for pointing out many of them.) - Fixed ALPHA/OSF/1 dynamic library support. (Thanks to Jonathan Bachrach (jonathan@harlequin.com)). - Added incremental GC support (MPROTECT_VDB) for Linux (with some help from Bruno Haible). - Altered SPARC recognition tests in gc.h and config.h (mostly as suggested by Fergus Henderson). - Added basic incremental GC support for win32, as implemented by Windows NT and Windows 95. GC_enable_incremental is a noop under win32s, which doesn't implement enough of the VM interface. - Added -DLARGE_CONFIG. - Fixed GC_..._ignore_off_page to also function without -DALL_INTERIOR_POINTERS. - (Hopefully) fixed RS/6000 port. (Only the test was broken.) - Fixed a performance bug in the nonincremental collector running on machines supporting incremental collection with MPROTECT_VDB (e.g. SunOS 4, DEC AXP). This turned into a correctness bug under win32s with win32 incremental collection. (Not all memory protection was disabled.) - Fixed some ppcr related bit rot. - Caused dynamic libraries to be unregistered before reregistering. The old way turned out to be a performance bug on some machines. - GC_root_size was not properly maintained under MSWIN32. - Added -DNO_DEBUGGING and GC_dump. - Fixed a couple of bugs arising with SOLARIS_THREADS + REDIRECT_MALLOC. - Added NetBSD/M68K port. (Thanks to Peter Seebach .) - Fixed a serious realloc bug. For certain object sizes, the collector wouldn't scan the expanded part of the object. (Thanks to Clay Spence (cds@peanut.sarnoff.com) for noticing the problem, and helping me to track it down.) Since version 4.5: - Added Linux ELF support. (Thanks to Arrigo Triulzi .) - GC_base crashed if it was called before any other GC_ routines. This could happen if a gc_cleanup object was allocated outside the heap before any heap allocation. - The heap expansion heuristic was not stable if all objects had finalization enabled. Fixed finalize.c to count memory in finalization queue and avoid explicit deallocation. Changed alloc.c to also consider this count. (This is still not recommended. It's expensive if nothing else.) Thanks to John Ellis for pointing this out. - GC_malloc_uncollectable(0) was broken. Thanks to Phong Vo for pointing this out. - The collector didn't compile under Linux 1.3.X. (Thanks to Fred Gilham for pointing this out.) The current workaround is ugly, but expected to be temporary. - Fixed a formatting problem for SPARC stack traces. - Fixed some '=='s in os_dep.c that should have been assignments. Fortunately these were in code that should never be executed anyway. (Thanks to Fergus Henderson.) - Fixed the heap block allocator to only drop blacklisted blocks in small chunks. Made BL_LIMIT self adjusting. (Both of these were in response to heap growth observed by Paul Graham.) - Fixed the Metrowerks/68K Mac code to also mark from a6. (Thanks to Patrick Beard.) - Significantly updated README.debugging. - Fixed some problems with longjmps out of signal handlers, especially under Solaris. Added a workaround for the fact that siglongjmp doesn't appear to do the right thing with -lthread under Solaris. - Added MSDOS/djgpp port. (Thanks to Mitch Harris (maharri@uiuc.edu).) - Added "make reserved_namespace" and "make user_namespace". The first renames ALL "GC_xxx" identifiers as "_GC_xxx". The second is the inverse transformation. Note that doing this is guaranteed to break all clients written for the other names. - descriptor field for kind NORMAL in GC_obj_kinds with ADD_BYTE_AT_END defined should be -ALIGNMENT not WORDS_TO_BYTES(-1). This is a serious bug on machines with pointer alignment of less than a word. - GC_ignore_self_finalize_mark_proc didn't handle pointers to very near the end of the object correctly. Caused failures of the C++ test on a DEC Alpha with g++. - gc_inl.h still had problems. Partially fixed. Added warnings at the beginning to hopefully specify the remaining dangers. - Added DATAEND definition to config.h. - Fixed some of the .h file organization. Fixed "make floppy". Since version 4.6: - Fixed some compilation problems with -DCHECKSUMS (thanks to Ian Searle) - Updated some Mac specific files to synchronize with Patrick Beard. - Fixed a serious bug for machines with non-word-aligned pointers. (Thanks to Patrick Beard for pointing out the problem. The collector should fail almost any conceivable test immediately on such machines.) Since version 4.7: - Changed a "comment" in a MacOS specific part of mach-dep.c that caused gcc to fail on other platforms. Since version 4.8 - More README.debugging fixes. - Objects ready for finalization, but not finalized in the same GC cycle, could be prematurely collected. This occasionally happened in test_cpp. - Too little memory was obtained from the system for very large objects. That could cause a heap explosion if these objects were not contiguous (e.g. under PCR), and too much of them was blacklisted. - Due to an improper initialization, the collector was too hesitant to allocate blacklisted objects immediately after system startup. - Moved GC_arrays from the data into the bss segment by not explicitly initializing it to zero. This significantly reduces the size of executables, and probably avoids some disk accesses on program startup. It's conceivable that it might break a port that I didn't test. - Fixed EMX_MAKEFILE to reflect the gc_c++.h to gc_cpp.h renaming which occurred a while ago. Since 4.9: - Fixed a typo around a call to GC_collect_or_expand in alloc.c. It broke handling of out of memory. (Thanks to Patrick Beard for noticing.) --------------------------------------------------------------------------- From phillip@pm.cse.rmit.edu.au Tue Feb 18 05:18:27 1997 Return-Path: Received: from mirriwinni.cse.rmit.edu.au (pm.cse.rmit.EDU.AU [131.170.118.50]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id FAA03379 for ; Tue, 18 Feb 1997 05:18:21 -0800 (PST) Received: (from phillip@localhost) by mirriwinni.cse.rmit.edu.au (8.8.3/8.6.12) id AAA02003; Wed, 19 Feb 1997 00:18:07 +1100 (EST) Date: Wed, 19 Feb 1997 00:18:07 +1100 (EST) Message-Id: <199702181318.AAA02003@mirriwinni.cse.rmit.edu.au> From: Phillip Musumeci To: adrian@cougar.aceonline.com.au CC: auditors@freebsd.org In-reply-to: (message from Adrian Chadd on Tue, 18 Feb 1997 09:13:29 +0800 (WST)) Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org >>>>> "Adrian" == Adrian Chadd writes: Adrian> Just wondering, but I heard a while back that there is a copy Adrian> of a range-checking GCC out there. Anyone confirm / deny this? Yes there is! The developer's URL is http://www-ala.doc.ic.ac.uk/~phjk/BoundsChecking.html and I've enclosed the text from this (access the real thing for pointers to their code). The last announcement that I kept a copy of was: > To: gnu-gcc-announce@uunet.uu.net > From: rwmj@doc.ic.ac.uk (Richard Jones) > Newsgroups: ic.general,gnu.gcc.announce,comp.lang.c > Subject: Bounds checking GCC v 1.02 released > Date: 18 Jan 1996 18:31:45 -0000 > Organization: Dept. of Computing, Imperial College, University of London, UK. > Distribution: world > Followup-To: poster > > *** GCC 2.7.2 with Bounds Checking: Version 1.02 released *** Being gcc-2.7.2 means we can possibly use it in FreeBSD 2.2 and beyond. However, I recall (?) that the gc tool doesn't use GPL. phillip --------------------------------------------------------------------------- Bounds Checking for C Richard Jones and Paul Kelly, Imperial College, July 1995 We're very excited about this: we can check every time a program uses a pointer or array and ensure that only valid references are allowed. This isn't new: what's new is that checked code can interwork with unchecked modules, libraries and system calls. We're still working on some rough edges and on improving the performance. This is a short overview; for a full report (and the code), see here. C is unusual among programming languages in providing the programmer with the full power of pointers. Languages in the Pascal/Algol family have arrays and pointers, with the restriction that arithmetic on pointers is disallowed. Languages like BCPL allow arbitrary operations on pointers, but lack types and so require clumsy scaling by object sizes. An advantage of the Pascal/Algol approach is that array references can be checked at run-time fairly efficiently, in fact so efficiently that there is a good case for bounds-checking in production code. Bounds checking is easy for arrays because the array subscript syntax specifies both the address calculation and the array within which the resulting pointer should point. With pointers in C, a pointer can be used in a context divorced from the name of the storage region for which it is valid. Approaches to bounds checking One response to this analysis is to discard C, since this lack of efficient checkability is responsible for many software failures. A second approach is to extend the language to make checking easier. There are various proposals for doing this, and it is an opportunity to add other features such as assertion checking. A third more-or-less workable scheme is to modify the representation of pointers to include three items: the pointer itself, and the lower and upper bounds of the object to which it is supposed to point. Experience with this has shown the benefits of bounds checking (e.g. see the bcc and rtcc compilers cited below), but there are difficulties: * Although some optimisation is possible, execution time of the resulting code increases by a large factor (ten or more, apparently). Even if the checking code can be optimised away, there remains the cost of passing triples for every pointer - which essentially prevents their being allocated to registers. * Because the representation of pointers has been changed, checked code is incompatible with normal code. This means that special versions of all libraries and system calls must be provided, and all the constituent modules of a program must be run with checking on. This adds to the performance problem. Some automatic support for interfacing checked code with normal code can be given, but this only works for straightforward cases. GUI code with call-backs, for example, is tricky. * Code which interfaces to hardware (e.g. a DMA controller) requires special attention since the hardware must be presented with standard addresses. How we solved the problem Our technique provides full checking without changing the representation of pointers. We therefore avoid most of the problems noted above. Some efficiency problems remain, but bounds checking need not be used in all of the files which make up a program, so trusted, performance-critical code can run at full speed. The key idea is this: * Every pointer expression derives a new pointer from a unique original pointer. For example, in "p+2*k+1" we derive a new pointer from "p". By contrast, in "p+q" or "p-q", we derive an integer from two pointers. The integer is nonsense as a pointer. We call this unique original pointer the expression's "base" pointer. * Every pointer value is valid for just one allocated storage region. An allocated storage region may be a global, static, automatic or heap-allocated variable, structure or array. * We can check whether a pointer arithmetic expression is valid by finding its base pointer's storage region, then checking that the expression's result points into the same storage region. * If the base pointer appears not to refer to any valid region, then it must refer to a region originating in unchecked code. In this case we cannot check the result of the expression. * If the base pointer's storage region is an array, say A[100], then (according to the ANSI standard) it is valid to calculate the address of the element after the last one valid (in this example, the address of A[100]). This is so that a pointer can be incremented and then tested for the loop exit condition. To prevent false alarms, we pad the storage layout of arrays to that A[100] is a valid pointer (we still check it when it is used). Implementation We made some small modifications to the C front-end of gcc, the Gnu C compiler, to add code to check pointer arithmetic and use, and to maintain a table of known allocated storage regions. We went to some trouble to ensure that gcc's optimiser could handle the added code, and employed modest inlining for efficiency. The table of known allocated storage regions has to handle insertions, deletions and range lookups extremely fast, but since programs display a high degree of locality the access pattern is highly skewed. For these reasons a splay tree was used, in which objects are migrated to the root when accessed. Performance * nfib (dumb doubly-recursive Fibonacci): no slowdown. + Execution time: same. + Compile-time: slowdown of 3 (very small) + Executable size: much larger due to inclusion of library. * Matrix multiply (ikj, using array subscripting): + Execution time: slowdown of around 30 compared to unoptimised. + Compile-time: slowdown of around 2. + Executable size: roughly the same. Availability The software is distributed free under GNU copyleft, in the form of a patch to the gcc 2.7.0 source distribution ( here). --------------------------------------------------------------------------- From phillip@pm.cse.rmit.edu.au Tue Feb 18 06:19:36 1997 Return-Path: Received: from mirriwinni.cse.rmit.edu.au (pm.cse.rmit.EDU.AU [131.170.118.50]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id GAA06008 for ; Tue, 18 Feb 1997 06:19:34 -0800 (PST) Received: (from phillip@localhost) by mirriwinni.cse.rmit.edu.au (8.8.3/8.6.12) id BAA02310; Wed, 19 Feb 1997 01:19:23 +1100 (EST) Date: Wed, 19 Feb 1997 01:19:23 +1100 (EST) Message-Id: <199702181419.BAA02310@mirriwinni.cse.rmit.edu.au> From: Phillip Musumeci To: imp@village.org CC: auditors@freebsd.org In-reply-to: (message from Warner Losh on Mon, 17 Feb 1997 20:17:25 -0700) Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org >>>>> "Warner" == Warner Losh writes: Warner> I've not seen any of these compiler extensions recently. I Warner> wonder what happened to them. Someone else here mentioned gcc_with_bounds checking --- that might catch simple things like string variables that overflow. The reason that my memory tweaked on run-time memory access checking was due to a hint in a recent announcement that use might have been made of a shell environment string overflow to break into a system. If some memory allocation tool had detected this kind of event, maybe something could have been done. Warner> ......... will do nothing for the race conditions, the bad Warner> uses of mktemp, et al, the sloppy use of seteuid(), badly Warner> written setuid programs, etc. Yes, you need intelligence for these. phillip From adrian@cougar.aceonline.com.au Tue Feb 18 06:54:49 1997 Return-Path: Received: from cougar.aceonline.com.au (adrian@cougar.aceonline.com.au [203.103.81.36]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id GAA07862 for ; Tue, 18 Feb 1997 06:54:43 -0800 (PST) Received: from localhost (adrian@localhost) by cougar.aceonline.com.au (8.8.4/8.7) with SMTP id WAA14248; Tue, 18 Feb 1997 22:54:24 +0800 Date: Tue, 18 Feb 1997 22:54:24 +0800 (WST) From: Adrian Chadd To: Phillip Musumeci cc: imp@village.org, auditors@freebsd.org Subject: Re : Bounds-checking gcc .. In-Reply-To: <199702181419.BAA02310@mirriwinni.cse.rmit.edu.au> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > Warner> I've not seen any of these compiler extensions recently. I > Warner> wonder what happened to them. > > Someone else here mentioned gcc_with_bounds checking --- that might catch > simple things like string variables that overflow. > > The reason that my memory tweaked on run-time memory access checking was > due to a hint in a recent announcement that use might have been made of a > shell environment string overflow to break into a system. If some memory > allocation tool had detected this kind of event, maybe something could have > been done. > Yes. The reason I asked to begin with was that there have been a large surge of buffer overflow conditions recently. > Warner> ......... will do nothing for the race conditions, the bad > Warner> uses of mktemp, et al, the sloppy use of seteuid(), badly > Warner> written setuid programs, etc. > > Yes, you need intelligence for these. > phillip > Agreed. Well, I'm going to try it out. I don't really like the idea of letting a compiler handle all the screwy code fixes, since if we get TOO relaxed and there is a hole in the compiler.. but its worth a shot at, and maybe if its any good, compiling the 'final' binaries with it as an extra precaution. my 2c worth. Adrian. From phillip@pm.cse.rmit.edu.au Tue Feb 18 07:10:38 1997 Return-Path: Received: from mirriwinni.cse.rmit.edu.au (pm.cse.rmit.EDU.AU [131.170.118.50]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id HAA08839 for ; Tue, 18 Feb 1997 07:10:35 -0800 (PST) Received: (from phillip@localhost) by mirriwinni.cse.rmit.edu.au (8.8.3/8.6.12) id CAA02867; Wed, 19 Feb 1997 02:10:30 +1100 (EST) Date: Wed, 19 Feb 1997 02:10:30 +1100 (EST) Message-Id: <199702181510.CAA02867@mirriwinni.cse.rmit.edu.au> From: Phillip Musumeci To: adrian@cougar.aceonline.com.au CC: imp@village.org, auditors@freebsd.org In-reply-to: (message from Adrian Chadd on Tue, 18 Feb 1997 22:54:24 +0800 (WST)) Subject: Re: Re : Bounds-checking gcc .. >>>>> "Adrian" == Adrian Chadd writes: Warner> ......... will do nothing for the race conditions, the bad uses Warner> of mktemp, et al, the sloppy use of seteuid(), badly written Warner> setuid programs, etc. A few years back, sunos had a bit of a scare with sendmail launching shell scripts [or other programs (?)] that could inherit inappropriate environment variables. A temporary fix that was circulated at the time was a very simple piece of code that cleaned out the environment before running the desired task via a call to execl() or one of its friends. This wrapper was simple and therefore easy to trust. If FreeBSD has a need for one task to call another task with a safe environment (no undesired LD_LIBRARY_PATHs etc.), maybe we could also have a single piece of well-read trusted source code that could be maintained for use as a wrapper where appropriate. Sorry if this is getting off the original thread. phillip From root@ka.home.hft.co.uk Tue Feb 18 10:06:11 1997 Return-Path: Received: from zeus.tcp.net.uk (zeus.tcp.net.uk [195.80.0.33]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA18967 for ; Tue, 18 Feb 1997 10:05:42 -0800 (PST) Received: from ka.home.hft.co.uk (root@ao093.du.pipex.com [193.130.254.93]) by zeus.tcp.net.uk (8.7.6/8.7) with ESMTP id SAA15334 for ; Tue, 18 Feb 1997 18:05:01 GMT Received: (from root@localhost) by ka.home.hft.co.uk (8.7.4/8.6.9) id SAA17580; Tue, 18 Feb 1997 18:04:55 GMT Received: from etamin.brunel.ac.uk (pp@etamin.brunel.ac.uk [134.83.128.61]) by zeus.tcp.net.uk (8.7.6/8.7) with SMTP id GAA16574 for ; Tue, 18 Feb 1997 06:32:38 GMT Received: from freefall.freebsd.org by etamin.brunel.ac.uk with SMTP (PP); Tue, 18 Feb 1997 06:32:00 +0000 Received: from zeus.tcp.net.uk (zeus.tcp.net.uk [195.80.0.33]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id WAA05788 for ; Mon, 17 Feb 1997 22:29:20 -0800 (PST) Received: from ka.home.hft.co.uk (root@al047.du.pipex.com [193.130.251.47]) by zeus.tcp.net.uk (8.7.6/8.7) with ESMTP id GAA16347 for ; Tue, 18 Feb 1997 06:28:38 GMT Received: (from root@localhost) by ka.home.hft.co.uk (8.7.4/8.6.9) id GAA17285; Tue, 18 Feb 1997 06:28:34 GMT Received: from ceres.brunel.ac.uk (pp@ceres.brunel.ac.uk [134.83.176.3]) by zeus.tcp.net.uk (8.7.6/8.7) with SMTP id AAA04461 for ; Tue, 18 Feb 1997 00:53:45 GMT Received: from freefall.freebsd.org by ceres.brunel.ac.uk with SMTP (PP); Tue, 18 Feb 1997 00:53:09 +0000 Received: from time.cdrom.com (time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA29615 for ; Mon, 17 Feb 1997 16:51:15 -0800 (PST) Received: from time.cdrom.com (localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id QAA28445 for ; Mon, 17 Feb 1997 16:51:12 -0800 (PST) Prev-Resent: Mon, 17 Feb 1997 16:51:11 -0800 Prev-Resent: "auditors@freebsd.org " Received: from mirriwinni.cse.rmit.edu.au (pm.cse.rmit.EDU.AU [131.170.118.50]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id FAA01569 for ; Mon, 17 Feb 1997 05:08:14 -0800 (PST) Received: (from phillip@localhost) by mirriwinni.cse.rmit.edu.au (8.8.3/8.6.12) id AAA26318; Tue, 18 Feb 1997 00:08:10 +1100 (EST) Date: Tue, 18 Feb 1997 00:08:10 +1100 (EST) Message-Id: <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> From: Phillip Musumeci X-Orig-To: "Jordan K. Hubbard" In-reply-to: "Jordan K. Hubbard"'s message of 11 Feb 1997 22:39:19 -0600 Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org References: <5drhhn$4fd@bonkers.taronga.com> Resent-To: auditors@freebsd.org Resent-Date: Mon, 17 Feb 1997 16:51:11 -0800 Resent-Message-ID: <28442.856227071@time.cdrom.com> Resent-From: "Jordan K. Hubbard" X-Processed-By: smtp-pop3 2.0 at TCP Ltd X-MAIL-From: jkh@time.cdrom.com X-Orig-To: ppb@baloo.tcp.co.uk X-Processed-By: smtp-pop3 2.0 at TCP Ltd X-MAIL-From: ppb@baloo.tcp.co.uk To: ppb@baloo.tcp.co.uk X-UIDL: 0fb962433a954ed777c2f802c02d9faf Hi, Jordan> ... to begin a much more serious and comprehensive security Jordan> audit, we will take advantage of this opportunity to see that Jordan> all code in the FreeBSD source tree, old and new alike, is Jordan> reviewed line by line for buffer overflows, unguarded copies, Jordan> back doors, whatever. I think this source code checking is a good idea. Another thing that might be practical, or maybe not (I'm not sure), is run-time checking for buffer overflows. There exists a set of memory allocation routines that I have seen other people use in order to locate memory leaks and find illegal memory accesses via pointers (e.g. writing past the end of an allocated piece of memory storage --- search for string "writing past"). It has been ported to FreeBSD, and here is the README file. Maybe this alternative memory allocation library could be used whenever someone wanted to "test drive" a utility and look for memory leaks. The default make could use the normal memory allocation. As I said, this is just an idea and I'm not sure how practical it would be to use it for FreeBSD testing. I thought I recognised the name of the guy who did the FreeBSD though, so maybe he is closely associated with the FreeBSD team and has already suggested this. regards, phillip p.s. I became aware of this set of memory management tools because it is used in the public domain RLaB matrix maths package. --------------------------------------------------------------------------- Copyright 1988, 1989 Hans-J. Boehm, Alan J. Demers Copyright (c) 1991-1995 by Xerox Corporation. All rights reserved. THIS MATERIAL IS PROVIDED AS IS, WITH ABSOLUTELY NO WARRANTY EXPRESSED OR IMPLIED. ANY USE IS AT YOUR OWN RISK. Permission is hereby granted to use or copy this program for any purpose, provided the above notices are retained on all copies. Permission to modify the code and to distribute modified code is granted, provided the above notices are retained, and a notice that the code was modified is included with the above copyright notice. This is version 4.10 of a conservative garbage collector for C and C++. HISTORY - Early versions of this collector were developed as a part of research projects supported in part by the National Science Foundation and the Defense Advance Research Projects Agency. Much of the code was rewritten by Hans-J. Boehm at Xerox PARC. The SPARC specific code was contributed by Mark Weiser (weiser@parc.xerox.com). The Encore Multimax modifications were supplied by Kevin Kenny (kenny@m.cs.uiuc.edu). The adaptation to the RT is largely due to Vernon Lee (scorpion@rice.edu), on machines made available by IBM. Much of the HP specific code and a number of good suggestions for improving the generic code are due to Walter Underwood (wunder@hp-ses.sde.hp.com). Robert Brazile (brazile@diamond.bbn.com) originally supplied the ULTRIX code. Al Dosser (dosser@src.dec.com) and Regis Cridlig (Regis.Cridlig@cl.cam.ac.uk) subsequently provided updates and information on variation between ULTRIX systems. Parag Patel (parag@netcom.com) supplied the A/UX code. Jesper Peterson(jep@mtiame.mtia.oz.au) and Michel Schinz supplied the Amiga port. Thomas Funke (thf@zelator.in-berlin.de(?)) and Brian D.Carlstrom (bdc@clark.lcs.mit.edu) supplied the NeXT ports. Douglas Steel (doug@wg.icl.co.uk) provided ICL DRS6000 code. Bill Janssen (janssen@parc.xerox.com) supplied the SunOS dynamic loader specific code. Manuel Serrano (serrano@cornas.inria.fr) supplied linux and Sony News specific code. Al Dosser provided Alpha/OSF/1 code. He and Dave Detlefs(detlefs@src.dec.com) also provided several generic bug fixes. Alistair G. Crooks(agc@uts.amdahl.com) supplied the NetBSD and 386BSD ports. Jeffrey Hsu (hsu@soda.berkeley.edu) provided the FreeBSD port. Brent Benson (brent@jade.ssd.csd.harris.com) ported the collector to a Motorola 88K processor running CX/UX (Harris NightHawk). Ari Huttunen (Ari.Huttunen@hut.fi) generalized the OS/2 port to nonIBM development environments (a nontrivial task). Patrick Beard (beard@cs.ucdavis.edu) provided the initial MacOS port. David Chase, then at Olivetti Research, suggested several improvements. Scott Schwartz (schwartz@groucho.cse.psu.edu) supplied some of the code to save and print call stacks for leak detection on a SPARC. Jesse Hull and John Ellis supplied the C++ interface code. Zhong Shao performed much of the experimentation that led to the current typed allocation facility. (His dynamic type inference code hasn't made it into the released version of the collector, yet.) (Blame for misinstallation of these modifications goes to the first author, however.) This is intended to be a general purpose, garbage collecting storage allocator. The algorithms used are described in: Boehm, H., and M. Weiser, "Garbage Collection in an Uncooperative Environment", Software Practice & Experience, September 1988, pp. 807-820. Boehm, H., A. Demers, and S. Shenker, "Mostly Parallel Garbage Collection", Proceedings of the ACM SIGPLAN '91 Conference on Programming Language Design and Implementation, SIGPLAN Notices 26, 6 (June 1991), pp. 157-164. Boehm, H., "Space Efficient Conservative Garbage Collection", Proceedings of the ACM SIGPLAN '91 Conference on Programming Language Design and Implementation, SIGPLAN Notices 28, 6 (June 1993), pp. 197-206. Possible interactions between the collector and optimizing compilers are discussed in Boehm, H., and D. Chase, "A Proposal for GC-safe C Compilation", The Journal of C Language Translation 4, 2 (December 1992). (Also available from parcftp.xerox.com:pub/gc, among other places.) Unlike the collector described in the second reference, this collector operates either with the mutator stopped during the entire collection (default) or incrementally during allocations. (The latter is supported on only a few machines.) It does not rely on threads, but is intended to be thread-safe. Some of the ideas underlying the collector have previously been explored by others. (Doug McIlroy wrote a vaguely similar collector that is part of version 8 UNIX (tm).) However none of this work appears to have been widely disseminated. Rudimentary tools for use of the collector as a leak detector are included, as is a fairly sophisticated string package "cord" that makes use of the collector. (See cord/README.) GENERAL DESCRIPTION This is a garbage collecting storage allocator that is intended to be used as a plug-in replacement for C's malloc. Since the collector does not require pointers to be tagged, it does not attempt to ensure that all inaccessible storage is reclaimed. However, in our experience, it is typically more successful at reclaiming unused memory than most C programs using explicit deallocation. Unlike manually introduced leaks, the amount of unreclaimed memory typically stays bounded. In the following, an "object" is defined to be a region of memory allocated by the routines described below. Any objects not intended to be collected must be pointed to either from other such accessible objects, or from the registers, stack, data, or statically allocated bss segments. Pointers from the stack or registers may point to anywhere inside an object. However, it is usually assumed that all pointers originating in the heap point to the beginning of an object. (This does not disallow interior pointers; it simply requires that there must be a pointer to the beginning of every accessible object, in addition to any interior pointers.) There are two facilities for altering this behavior. The macro ALL_INTERIOR_POINTERS may be defined in gc_private.h to cause any pointer into an object (or one past the end) to retain the object. A routine GC_register_displacement is provided to allow for more controlled interior pointer use in the heap. Defining ALL_INTERIOR_POINTERS is somewhat dangerous, in that it can result in unnecessary memory retention. However this is much less of a problem than with older collector versions. The routine GC_register_displacement is described in gc.h. Note that pointers inside memory allocated by the standard "malloc" are not seen by the garbage collector. Thus objects pointed to only from such a region may be prematurely deallocated. It is thus suggested that the standard "malloc" be used only for memory regions, such as I/O buffers, that are guaranteed not to contain pointers. Pointers in C language automatic, static, or register variables, are correctly recognized. (Note that GC_malloc_uncollectable has semantics similar to standard malloc, but allocates objects that are traced by the collector.) The collector does not generally know how to find pointers in data areas that are associated with dynamic libraries. This is easy to remedy IF you know how to find those data areas on your operating system (see GC_add_roots). Code for doing this under SunOS and IRIX 5.X is included (see dynamic_load.c). Note that the garbage collector does not need to be informed of shared read-only data. However if the shared library mechanism can introduce discontiguous data areas that may contain pointers, then the collector does need to be informed. Signal processing for most signals may be deferred during collection, and during uninterruptible parts of the allocation process. Unlike standard ANSI C mallocs, it can be safe to invoke malloc from a signal handler while another malloc is in progress, provided the original malloc is not restarted. (Empirically, many UNIX applications already assume this.) To obtain this level of signal safety, remove the definition of -DNO_SIGNALS in Makefile. The allocator/collector can also be configured for thread-safe operation. (Full signal safety can also be achieved, but only at the cost of two system calls per malloc, which is usually unacceptable.) INSTALLATION AND PORTABILITY As distributed, the macro SILENT is defined in Makefile. In the event of problems, this can be removed to obtain a moderate amount of descriptive output for each collection. (The given statistics exhibit a few peculiarities. Things don't appear to add up for a variety of reasons, most notably fragmentation losses. These are probably much more significant for the contrived program "test.c" than for your application.) Note that typing "make test" will automatically build the collector and then run setjmp_test and gctest. Setjmp_test will give you information about configuring the collector, which is useful primarily if you have a machine that's not already supported. Gctest is a somewhat superficial test of collector functionality. Failure is indicated by a core dump or a message to the effect that the collector is broken. Gctest takes about 35 seconds to run on a SPARCstation 2. On a slower machine, expect it to take a while. It may use up to 8 MB of memory. (The multi-threaded version will use more.) "Make test" will also, as its last step, attempt to build and test the "cord" string library. This will fail without an ANSI C compiler. The Makefile will generate a library gc.a which you should link against. Typing "make cords" will add the cord library to gc.a. Note that this requires an ANSI C compiler. It is suggested that if you need to replace a piece of the collector (e.g. GC_mark_rts.c) you simply list your version ahead of gc.a on the ld command line, rather than replacing the one in gc.a. (This will generate numerous warnings under some versions of AIX, but it still works.) All include files that need to be used by clients will be put in the include subdirectory. (Normally this is just gc.h. "Make cords" adds "cord.h" and "ec.h".) The collector currently is designed to run essentially unmodified on the following machines (most of the operating systems mentioned are trademarks of their respective holders): Sun 3 Sun 4 under SunOS 4.X or Solaris2.X (with or without threads) Vax under 4.3BSD, Ultrix Intel 386 or 486 under most operating systems, but not MSDOS. (Win32S is somewhat supported, so it is possible to build applications for Windows 3.1. There exists a port to DOS + 32 bit extender for at least one 32 bit extender. However, I don't have source for this.) Sequent Symmetry (single threaded) Encore Multimax (single threaded) MIPS M/120 (and presumably M/2000) (RISC/os 4.0 with BSD libraries) IBM PC/RT (Berkeley UNIX) IBM RS/6000 HP9000/300 HP9000/700 DECstations under Ultrix DEC Alpha running OSF/1 SGI workstations under IRIX 4 & 5 Sony News Apple Macintosh under A/UX or MacOS Commodore Amiga (see README.amiga) NeXT machines In a few cases (Amiga, OS/2, Win32, MacOS) a separate makefile or equivalent is supplied. Dynamic libraries are completely supported only under SunOS (and even that support is not functional on the last Sun 3 release), IRIX 5, Win32 (not Win32S) and OSF/1 on DEC AXP machines. On other machines we recommend that you do one of the following: 1) Add dynamic library support (and send us the code). 2) Use static versions of the libraries. 3) Arrange for dynamic libraries to use the standard malloc. This is still dangerous if the library stores a pointer to a garbage collected object. But nearly all standard interfaces prohibit this, because they deal correctly with pointers to stack allocated objects. (Strtok is an exception. Don't use it.) In all cases we assume that pointer alignment is consistent with that enforced by the standard C compilers. If you use a nonstandard compiler you may have to adjust the alignment parameters defined in gc_priv.h. A port to a machine that is not byte addressed, or does not use 32 bit or 64 bit addresses will require a major effort. A port to MSDOS is hard, unless you are willing to assume an 80386 or better, and that only flat 32 bit pointers will ever need to be seen by the collector. For machines not already mentioned, or for nonstandard compilers, the following are likely to require change: 1. The parameters in config.h. The parameters that will usually require adjustment are STACKBOTTOM, ALIGNMENT and DATASTART. Setjmp_test prints its guesses of the first two. DATASTART should be an expression for computing the address of the beginning of the data segment. This can often be &etext. But some memory management units require that there be some unmapped space between the text and the data segment. Thus it may be more complicated. On UNIX systems, this is rarely documented. But the adb "$m" command may be helpful. (Note that DATASTART will usually be a function of &etext. Thus a single experiment is usually insufficient.) STACKBOTTOM is used to initialize GC_stackbottom, which should be a sufficient approximation to the coldest stack address. On some machines, it is difficult to obtain such a value that is valid across a variety of MMUs, OS releases, etc. A number of alternatives exist for using the collector in spite of this. See the discussion in config.h immediately preceding the various definitions of STACKBOTTOM. 2. mach_dep.c. The most important routine here is one to mark from registers. The distributed file includes a generic hack (based on setjmp) that happens to work on many machines, and may work on yours. Try compiling and running setjmp_t.c to see whether it has a chance of working. (This is not correct C, so don't blame your compiler if it doesn't work. Based on limited experience, register window machines are likely to cause trouble. If your version of setjmp claims that all accessible variables, including registers, have the value they had at the time of the longjmp, it also will not work. Vanilla 4.2 BSD on Vaxen makes such a claim. SunOS does not.) If your compiler does not allow in-line assembly code, or if you prefer not to use such a facility, mach_dep.c may be replaced by a .s file (as we did for the MIPS machine and the PC/RT). At this point enough architectures are supported by mach_dep.c that you will rarely need to do more than adjust for assembler syntax. 3. os_dep.c (and gc_priv.h). Several kinds of operating system dependent routines reside here. Many are optional. Several are invoked only through corresponding macros in gc_priv.h, which may also be redefined as appropriate. The routine GC_register_data_segments is crucial. It registers static data areas that must be traversed by the collector. (User calls to GC_add_roots may sometimes be used for similar effect.) Routines to obtain memory from the OS also reside here. Alternatively this can be done entirely by the macro GET_MEM defined in gc_priv.h. Routines to disable and reenable signals also reside here if they are need by the macros DISABLE_SIGNALS and ENABLE_SIGNALS defined in gc_priv.h. In a multithreaded environment, the macros LOCK and UNLOCK in gc_priv.h will need to be suitably redefined. The incremental collector requires page dirty information, which is acquired through routines defined in os_dep.c. Unless directed otherwise by config.h, these are implemented as stubs that simply treat all pages as dirty. (This of course makes the incremental collector much less useful.) 4. dyn_load.c This provides a routine that allows the collector to scan data segments associated with dynamic libraries. Often it is not necessary to provide this routine unless user-written dynamic libraries are used. For a different version of UN*X or different machines using the Motorola 68000, Vax, SPARC, 80386, NS 32000, PC/RT, or MIPS architecture, it should frequently suffice to change definitions in config.h. THE C INTERFACE TO THE ALLOCATOR The following routines are intended to be directly called by the user. Note that usually only GC_malloc is necessary. GC_clear_roots and GC_add_roots calls may be required if the collector has to trace from nonstandard places (e.g. from dynamic library data areas on a machine on which the collector doesn't already understand them.) On some machines, it may be desirable to set GC_stacktop to a good approximation of the stack base. (This enhances code portability on HP PA machines, since there is no good way for the collector to compute this value.) Client code may include "gc.h", which defines all of the following, plus many others. 1) GC_malloc(nbytes) - allocate an object of size nbytes. Unlike malloc, the object is cleared before being returned to the user. Gc_malloc will invoke the garbage collector when it determines this to be appropriate. GC_malloc may return 0 if it is unable to acquire sufficient space from the operating system. This is the most probable consequence of running out of space. Other possible consequences are that a function call will fail due to lack of stack space, or that the collector will fail in other ways because it cannot maintain its internal data structures, or that a crucial system process will fail and take down the machine. Most of these possibilities are independent of the malloc implementation. 2) GC_malloc_atomic(nbytes) - allocate an object of size nbytes that is guaranteed not to contain any pointers. The returned object is not guaranteed to be cleared. (Can always be replaced by GC_malloc, but results in faster collection times. The collector will probably run faster if large character arrays, etc. are allocated with GC_malloc_atomic than if they are statically allocated.) 3) GC_realloc(object, new_size) - change the size of object to be new_size. Returns a pointer to the new object, which may, or may not, be the same as the pointer to the old object. The new object is taken to be atomic iff the old one was. If the new object is composite and larger than the original object, then the newly added bytes are cleared (we hope). This is very likely to allocate a new object, unless MERGE_SIZES is defined in gc_priv.h. Even then, it is likely to recycle the old object only if the object is grown in small additive increments (which, we claim, is generally bad coding practice.) 4) GC_free(object) - explicitly deallocate an object returned by GC_malloc or GC_malloc_atomic. Not necessary, but can be used to minimize collections if performance is critical. Probably a performance loss for very small objects (<= 8 bytes). 5) GC_expand_hp(bytes) - Explicitly increase the heap size. (This is normally done automatically if a garbage collection failed to GC_reclaim enough memory. Explicit calls to GC_expand_hp may prevent unnecessarily frequent collections at program startup.) 6) GC_malloc_ignore_off_page(bytes) - identical to GC_malloc, but the client promises to keep a pointer to the somewhere within the first 256 bytes of the object while it is live. (This pointer should nortmally be declared volatile to prevent interference from compiler optimizations.) This is the recommended way to allocate anything that is likely to be larger than 100Kbytes or so. (GC_malloc may result in failure to reclaim such objects.) 7) GC_set_warn_proc(proc) - Can be used to redirect warnings from the collector. Such warnings should be rare, and should not be ignored during code development. 8) GC_enable_incremental() - Enables generational and incremental collection. Useful for large heaps on machines that provide access to page dirty information. Some dirty bit implementations may interfere with debugging (by catching address faults) and place restrictions on heap arguments to system calls (since write faults inside a system call may not be handled well). 9) Several routines to allow for registration of finalization code. User supplied finalization code may be invoked when an object becomes unreachable. To call (*f)(obj, x) when obj becomes inaccessible, use GC_register_finalizer(obj, f, x, 0, 0); For more sophisticated uses, and for finalization ordering issues, see gc.h. The global variable GC_free_space_divisor may be adjusted up from its default value of 4 to use less space and more collection time, or down for the opposite effect. Setting it to 1 or 0 will effectively disable collections and cause all allocations to simply grow the heap. The variable GC_non_gc_bytes, which is normally 0, may be changed to reflect the amount of memory allocated by the above routines that should not be considered as a candidate for collection. Careless use may, of course, result in excessive memory consumption. Some additional tuning is possible through the parameters defined near the top of gc_priv.h. If only GC_malloc is intended to be used, it might be appropriate to define: #define malloc(n) GC_malloc(n) #define calloc(m,n) GC_malloc((m)*(n)) For small pieces of VERY allocation intensive code, gc_inl.h includes some allocation macros that may be used in place of GC_malloc and friends. All externally visible names in the garbage collector start with "GC_". To avoid name conflicts, client code should avoid this prefix, except when accessing garbage collector routines or variables. There are provisions for allocation with explicit type information. This is rarely necessary. Details can be found in gc_typed.h. THE C++ INTERFACE TO THE ALLOCATOR: The Ellis-Hull C++ interface to the collector is included in the collector distribution. If you intend to use this, type "make c++" after the initial build of the collector is complete. See gc_cpp.h for the definition of the interface. This interface tries to approximate the Ellis-Detlefs C++ garbage collection proposal without compiler changes. Cautions: 1. Arrays allocated without new placement syntax are allocated as uncollectable objects. They are traced by the collector, but will not be reclaimed. 2. Failure to use "make c++" in combination with (1) will result in arrays allocated using the default new operator. This is likely to result in disaster without linker warnings. 3. If your compiler supports an overloaded new[] operator, then gc_cpp.cc and gc_cpp.h should be suitably modified. 4. Many current C++ compilers have deficiencies that break some of the functionality. See the comments in gc_cpp.h for suggested workarounds. USE AS LEAK DETECTOR: The collector may be used to track down leaks in C programs that are intended to run with malloc/free (e.g. code with extreme real-time or portability constraints). To do so define FIND_LEAK in Makefile This will cause the collector to invoke the report_leak routine defined near the top of reclaim.c whenever an inaccessible object is found that has not been explicitly freed. Productive use of this facility normally involves redefining report_leak to do something more intelligent. This typically requires annotating objects with additional information (e.g. creation time stack trace) that identifies their origin. Such code is typically not very portable, and is not included here, except on SPARC machines. If all objects are allocated with GC_DEBUG_MALLOC (see next section), then the default version of report_leak will report the source file and line number at which the leaked object was allocated. This may sometimes be sufficient. (On SPARC/SUNOS4 machines, it will also report a cryptic stack trace. This can often be turned into a sympolic stack trace by invoking program "foo" with "callprocs foo". Callprocs is a short shell script that invokes adb to expand program counter values to symbolic addresses. It was largely supplied by Scott Schwartz.) Note that the debugging facilities described in the next section can sometimes be slightly LESS effective in leak finding mode, since in leak finding mode, GC_debug_free actually results in reuse of the object. (Otherwise the object is simply marked invalid.) Also note that the test program is not designed to run meaningfully in FIND_LEAK mode. Use "make gc.a" to build the collector. DEBUGGING FACILITIES: The routines GC_debug_malloc, GC_debug_malloc_atomic, GC_debug_realloc, and GC_debug_free provide an alternate interface to the collector, which provides some help with memory overwrite errors, and the like. Objects allocated in this way are annotated with additional information. Some of this information is checked during garbage collections, and detected inconsistencies are reported to stderr. Simple cases of writing past the end of an allocated object should be caught if the object is explicitly deallocated, or if the collector is invoked while the object is live. The first deallocation of an object will clear the debugging info associated with an object, so accidentally repeated calls to GC_debug_free will report the deallocation of an object without debugging information. Out of memory errors will be reported to stderr, in addition to returning NIL. GC_debug_malloc checking during garbage collection is enabled with the first call to GC_debug_malloc. This will result in some slowdown during collections. If frequent heap checks are desired, this can be achieved by explicitly invoking GC_gcollect, e.g. from the debugger. GC_debug_malloc allocated objects should not be passed to GC_realloc or GC_free, and conversely. It is however acceptable to allocate only some objects with GC_debug_malloc, and to use GC_malloc for other objects, provided the two pools are kept distinct. In this case, there is a very low probablility that GC_malloc allocated objects may be misidentified as having been overwritten. This should happen with probability at most one in 2**32. This probability is zero if GC_debug_malloc is never called. GC_debug_malloc, GC_malloc_atomic, and GC_debug_realloc take two additional trailing arguments, a string and an integer. These are not interpreted by the allocator. They are stored in the object (the string is not copied). If an error involving the object is detected, they are printed. The macros GC_MALLOC, GC_MALLOC_ATOMIC, GC_REALLOC, GC_FREE, and GC_REGISTER_FINALIZER are also provided. These require the same arguments as the corresponding (nondebugging) routines. If gc.h is included with GC_DEBUG defined, they call the debugging versions of these functions, passing the current file name and line number as the two extra arguments, where appropriate. If gc.h is included without GC_DEBUG defined, then all these macros will instead be defined to their nondebugging equivalents. (GC_REGISTER_FINALIZER is necessary, since pointers to objects with debugging information are really pointers to a displacement of 16 bytes form the object beginning, and some translation is necessary when finalization routines are invoked. For details, about what's stored in the header, see the definition of the type oh in debug_malloc.c) INCREMENTAL/GENERATIONAL COLLECTION: The collector normally interrupts client code for the duration of a garbage collection mark phase. This may be unacceptable if interactive response is needed for programs with large heaps. The collector can also run in a "generational" mode, in which it usually attempts to collect only objects allocated since the last garbage collection. Furthermore, in this mode, garbage collections run mostly incrementally, with a small amount of work performed in response to each of a large number of GC_malloc requests. This mode is enabled by a call to GC_enable_incremental(). Incremental and generational collection is effective in reducing pause times only if the collector has some way to tell which objects or pages have been recently modified. The collector uses two sources of information: 1. Information provided by the VM system. This may be provided in one of several forms. Under Solaris 2.X (and potentially under other similar systems) information on dirty pages can be read from the /proc file system. Under other systems (currently SunOS4.X) it is possible to write-protect the heap, and catch the resulting faults. On these systems we require that system calls writing to the heap (other than read) be handled specially by client code. See os_dep.c for details. 2. Information supplied by the programmer. We define "stubborn" objects to be objects that are rarely changed. Such an object can be allocated (and enabled for writing) with GC_malloc_stubborn. Once it has been initialized, the collector should be informed with a call to GC_end_stubborn_change. Subsequent writes that store pointers into the object must be preceded by a call to GC_change_stubborn. This mechanism performs best for objects that are written only for initialization, and such that only one stubborn object is writable at once. It is typically not worth using for short-lived objects. Stubborn objects are treated less efficiently than pointerfree (atomic) objects. A rough rule of thumb is that, in the absence of VM information, garbage collection pauses are proportional to the amount of pointerful storage plus the amount of modified "stubborn" storage that is reachable during the collection. Initial allocation of stubborn objects takes longer than allocation of other objects, since other data structures need to be maintained. We recommend against random use of stubborn objects in client code, since bugs caused by inappropriate writes to stubborn objects are likely to be very infrequently observed and hard to trace. However, their use may be appropriate in a few carefully written library routines that do not make the objects themselves available for writing by client code. BUGS: Any memory that does not have a recognizable pointer to it will be reclaimed. Exclusive-or'ing forward and backward links in a list doesn't cut it. Some C optimizers may lose the last undisguised pointer to a memory object as a consequence of clever optimizations. This has almost never been observed in practice. Send mail to boehm@parc.xerox.com for suggestions on how to fix your compiler. This is not a real-time collector. In the standard configuration, percentage of time required for collection should be constant across heap sizes. But collection pauses will increase for larger heaps. (On SPARCstation 2s collection times will be on the order of 300 msecs per MB of accessible memory that needs to be scanned. Your mileage may vary.) The incremental/generational collection facility helps, but is portable only if "stubborn" allocation is used. Please address bug reports to boehm@parc.xerox.com. If you are contemplating a major addition, you might also send mail to ask whether it's already been done (or whether we tried and discarded it). RECENT VERSIONS: Version 1.3 and immediately preceding versions contained spurious assembly language assignments to TMP_SP. Only the assignment in the PC/RT code is necessary. On other machines, with certain compiler options, the assignments can lead to an unsaved register being overwritten. Known to cause problems under SunOS 3.5 WITHOUT the -O option. (With -O the compiler recognizes it as dead code. It probably shouldn't, but that's another story.) Version 1.4 and earlier versions used compile time determined values for the stack base. This no longer works on Sun 3s, since Sun 3/80s use a different stack base. We now use a straightforward heuristic on all machines on which it is known to work (incl. Sun 3s) and compile-time determined values for the rest. There should really be library calls to determine such values. Version 1.5 and earlier did not ensure 8 byte alignment for objects allocated on a sparc based machine. Version 1.8 added ULTRIX support in gc_private.h. Version 1.9 fixed a major bug in gc_realloc. Version 2.0 introduced a consistent naming convention for collector routines and added support for registering dynamic library data segments in the standard mark_roots.c. Most of the data structures were revamped. The treatment of interior pointers was completely changed. Finalization was added. Support for locking was added. Object kinds were added. We added a black listing facility to avoid allocating at addresses known to occur as integers somewhere in the address space. Much of this was accomplished by adapting ideas and code from the PCR collector. The test program was changed and expanded. Version 2.1 was the first stable version since 1.9, and added support for PPCR. Version 2.2 added debugging allocation, and fixed various bugs. Among them: - GC_realloc could fail to extend the size of the object for certain large object sizes. - A blatant subscript range error in GC_printf, which unfortunately wasn't exercised on machines with sufficient stack alignment constraints. - GC_register_displacement did the wrong thing if it was called after any allocation had taken place. - The leak finding code would eventually break after 2048 byte byte objects leaked. - interface.c didn't compile. - The heap size remained much too small for large stacks. - The stack clearing code behaved badly for large stacks, and perhaps on HP/PA machines. Version 2.3 added ALL_INTERIOR_POINTERS and fixed the following bugs: - Missing declaration of etext in the A/UX version. - Some PCR root-finding problems. - Blacklisting was not 100% effective, because the plausible future heap bounds were being miscalculated. - GC_realloc didn't handle out-of-memory correctly. - GC_base could return a nonzero value for addresses inside free blocks. - test.c wasn't really thread safe, and could erroneously report failure in a multithreaded environment. (The locking primitives need to be replaced for other threads packages.) - GC_CONS was thoroughly broken. - On a SPARC with dynamic linking, signals stayed diabled while the client code was running. (Thanks to Manuel Serrano at INRIA for reporting the last two.) Version 2.4 added GC_free_space_divisor as a tuning knob, added support for OS/2 and linux, and fixed the following bugs: - On machines with unaligned pointers (e.g. Sun 3), every 128th word could fail to be considered for marking. - Dynamic_load.c erroneously added 4 bytes to the length of the data and bss sections of the dynamic library. This could result in a bad memory reference if the actual length was a multiple of a page. (Observed on Sun 3. Can probably also happen on a Sun 4.) (Thanks to Robert Brazile for pointing out that the Sun 3 version was broken. Dynamic library handling is still broken on Sun 3s under 4.1.1U1, but apparently not 4.1.1. If you have such a machine, use -Bstatic.) Version 2.5 fixed the following bugs: - Removed an explicit call to exit(1) - Fixed calls to GC_printf and GC_err_printf, so the correct number of arguments are always supplied. The OS/2 C compiler gets confused if the number of actuals and the number of formals differ. (ANSI C doesn't require this to work. The ANSI sanctioned way of doing things causes too many compatibility problems.) Version 3.0 added generational/incremental collection and stubborn objects. Version 3.1 added the following features: - A workaround for a SunOS 4.X SPARC C compiler misfeature that caused problems when the collector was turned into a dynamic library. - A fix for a bug in GC_base that could result in a memory fault. - A fix for a performance bug (and several other misfeatures) pointed out by Dave Detlefs and Al Dosser. - Use of dirty bit information for static data under Solaris 2.X. - DEC Alpha/OSF1 support (thanks to Al Dosser). - Incremental collection on more platforms. - A more refined heap expansion policy. Less space usage by default. - Various minor enhancements to reduce space usage, and to reduce the amount of memory scanned by the collector. - Uncollectable allocation without per object overhead. - More conscientious handling of out-of-memory conditions. - Fixed a bug in debugging stubborn allocation. - Fixed a bug that resulted in occasional erroneous reporting of smashed objects with debugging allocation. - Fixed bogus leak reports of size 4096 blocks with FIND_LEAK. Version 3.2 fixed a serious and not entirely repeatable bug in the incremental collector. It appeared only when dirty bit info on the roots was available, which is normally only under Solaris. It also added GC_general_register_disappearing_link, and some testing code. Interface.c disappeared. Version 3.3 fixes several bugs and adds new ports: - PCR-specific bugs. - Missing locking in GC_free, redundant FASTUNLOCK in GC_malloc_stubborn, and 2 bugs in GC_unregister_disappearing_link. All of the above were pointed out by Neil Sharman (neil@cs.mu.oz.au). - Common symbols allocated by the SunOS4.X dynamic loader were not included in the root set. - Bug in GC_finalize (reported by Brian Beuning and Al Dosser) - Merged Amiga port from Jesper Peterson (untested) - Merged NeXT port from Thomas Funke (significantly modified and untested) Version 3.4: - Fixed a performance bug in GC_realloc. - Updated the amiga port. - Added NetBSD and 386BSD ports. - Added cord library. - Added trivial performance enhancement for ALL_INTERIOR_POINTERS. (Don't scan last word.) Version 3.5 - Minor collections now mark from roots only once, if that doesn't cause an excessive pause. - The stack clearing heuristic was refined to prevent anomalies with very heavily recursive programs and sparse stacks. - Fixed a bug that prevented mark stack growth in some cases. GC_objects_are_marked should be set to TRUE after a call to GC_push_roots and as part of GC_push_marked, since both can now set mark bits. I think this is only a performance bug, but I wouldn't bet on it. It's certainly very hard to argue that the old version was correct. - Fixed an incremental collection bug that prevented it from working at all when HBLKSIZE != getpagesize() - Changed dynamic_loading.c to include gc_priv.h before testing DYNAMIC_LOADING. SunOS dynamic library scanning must have been broken in 3.4. - Object size rounding now adapts to program behavior. - Added a workaround (provided by Manuel Serrano and colleagues) to a long-standing SunOS 4.X (and 3.X?) ld bug that I had incorrectly assumed to have been squished. The collector was broken if the text segment size was within 32 bytes of a multiple of 8K bytes, and if the beginning of the data segment contained interesting roots. The workaround assumes a demand-loadable executable. The original may have have "worked" in some other cases. - Added dynamic library support under IRIX5. - Added support for EMX under OS/2 (thanks to Ari Huttunen). Version 3.6: - fixed a bug in the mark stack growth code that was introduced in 3.4. - fixed Makefile to work around DEC AXP compiler tail recursion bug. Version 3.7: - Added a workaround for an HP/UX compiler bug. - Fixed another stack clearing performance bug. Reworked that code once more. Version 4.0: - Added support for Solaris threads (which was possible only by reimplementing some fraction of Solaris threads, since Sun doesn't currently make the thread debugging interface available). - Added non-threads win32 and win32S support. - (Grudgingly, with suitable muttering of obscenities) renamed files so that the collector distribution could live on a FAT file system. Files that are guaranteed to be useless on a PC still have long names. Gc_inline.h and gc_private.h still exist, but now just include gc_inl.h and gc_priv.h. - Fixed a really obscure bug in finalization that could cause undetected mark stack overflows. (I would be surprised if any real code ever tickled this one.) - Changed finalization code to dynamically resize the hash tables it maintains. (This probably does not matter for well- -written code. It no doubt does for C++ code that overuses destructors.) - Added typed allocation primitives. Rewrote the marker to accommodate them with more reasonable efficiency. This change should also speed up marking for GC_malloc allocated objects a little. See gc_typed.h for new primitives. - Improved debugging facilities slightly. Allocation time stack traces are now kept by default on SPARC/SUNOS4. (Thanks to Scott Schwartz.) - Added better support for small heap applications. - Significantly extended cord package. Fixed a bug in the implementation of lazily read files. Printf and friends now have cord variants. Cord traversals are a bit faster. - Made ALL_INTERIOR_POINTERS recognition the default. - Fixed de so that it can run in constant space, independent of file size. Added simple string searching to cords and de. - Added the Hull-Ellis C++ interface. - Added dynamic library support for OSF/1. (Thanks to Al Dosser and Tim Bingham at DEC.) - Changed argument to GC_expand_hp to be expressed in units of bytes instead of heap blocks. (Necessary since the heap block size now varies depending on configuration. The old version was never very clean.) - Added GC_get_heap_size(). The previous "equivalent" was broken. - Restructured the Makefile a bit. Since version 4.0: - Changed finalization implementation to guarantee that finalization procedures are called outside of the allocation lock, making direct use of the interface a little less dangerous. MAY BREAK EXISTING CLIENTS that assume finalizers are protected by a lock. Since there seem to be few multithreaded clients that use finalization, this is hopefully not much of a problem. - Fixed a gross bug in CORD_prev. - Fixed a bug in blacklst.c that could result in unbounded heap growth during startup on machines that do not clear memory obtained from the OS (e.g. win32S). - Ported de editor to win32/win32S. (This is now the only version with a mouse-sensitive UI.) - Added GC_malloc_ignore_off_page to allocate large arrays in the presence of ALL_INTERIOR_POINTERS. - Changed GC_call_with_alloc_lock to not disable signals in the single-threaded case. - Reduced retry count in GC_collect_or_expand for garbage collecting when out of memory. - Made uncollectable allocations bypass black-listing, as they should. - Fixed a bug in typed_test in test.c that could cause (legitimate) GC crashes. - Fixed some potential synchronization problems in finalize.c - Fixed a real locking problem in typd_mlc.c. - Worked around an AIX 3.2 compiler feature that results in out of bounds memory references. - Partially worked around an IRIX5.2 beta problem (which may or may not persist to the final release). - Fixed a bug in the heap integrity checking code that could result in explicitly deallocated objects being identified as smashed. Fixed a bug in the dbg_mlc stack saving code that caused old argument pointers to be considered live. - Fixed a bug in CORD_ncmp (and hence CORD_str). - Repaired the OS2 port, which had suffered from bit rot in 4.0. Worked around what appears to be CSet/2 V1.0 optimizer bug. - Fixed a Makefile bug for target "c++". Since version 4.1: - Multiple bug fixes/workarounds in the Solaris threads version. (It occasionally failed to locate some register contents for marking. It also turns out that thr_suspend and friends are unreliable in Solaris 2.3. Dirty bit reads appear to be unreliable under some weird circumstances. My stack marking code contained a serious performance bug. The new code is extremely defensive, and has not failed in several cpu hours of testing. But no guarantees ...) - Added MacOS support (thanks to Patrick Beard.) - Fixed several syntactic bugs in gc_c++.h and friends. (These didn't bother g++, but did bother most other compilers.) Fixed gc_c++.h finalization interface. (It didn't.) - 64 bit alignment for allocated objects was not guaranteed in a few cases in which it should have been. - Added GC_malloc_atomic_ignore_off_page. - Added GC_collect_a_little. - Added some prototypes to gc.h. - Some other minor bug fixes (notably in Makefile). - Fixed OS/2 / EMX port (thanks to Ari Huttunen). - Fixed AmigaDOS port. (thanks to Michel Schinz). - Fixed the DATASTART definition under Solaris. There was a 1 in 16K chance of the collector missing the first 64K of static data (and thus crashing). - Fixed some blatant anachronisms in the README file. - Fixed PCR-Makefile for upcoming PPCR release. Since version 4.2: - Fixed SPARC alignment problem with GC_DEBUG. - Fixed Solaris threads /proc workaround. The real problem was an interaction with mprotect. - Incorporated fix from Patrick Beard for gc_c++.h (now gc_cpp.h). - Slightly improved allocator space utilization by fixing the GC_size_map mechanism. - Integrated some Sony News and MIPS RISCos 4.51 patches. (Thanks to Nobuyuki Hikichi of Software Research Associates, Inc. Japan) - Fixed HP_PA alignment problem. (Thanks to xjam@cork.cs.berkeley.edu.) - Added GC_same_obj and friends. Changed GC_base to return 0 for pointers past the end of large objects. Improved GC_base performance with ALL_INTERIOR_POINTERS on machines with a slow integer mod operation. Added GC_PTR_ADD, GC_PTR_STORE, etc. to prepare for preprocessor. - changed the default on most UNIX machines to be that signals are not disabled during critical GC operations. This is still ANSI-conforming, though somewhat dangerous in the presence of signal handlers. But the performance cost of the alternative is sometimes problematic. Can be changed back with a minor Makefile edit. - renamed IS_STRING in gc.h, to CORD_IS_STRING, thus following my own naming convention. Added the function CORD_to_const_char_star. - Fixed a gross bug in GC_finalize. Symptom: occasional address faults in that function. (Thanks to Anselm Baird-Smith (Anselm.BairdSmith@inria.fr) - Added port to ICL DRS6000 running DRS/NX. Restructured things a bit to factor out common code, and remove obsolete code. Collector should now run under SUNOS5 with either mprotect or /proc dirty bits. (Thanks to Douglas Steel (doug@wg.icl.co.uk)). - More bug fixes and workarounds for Solaris 2.X. (These were mostly related to putting the collector in a dynamic library, which didn't really work before. Also SOLARIS_THREADS didn't interact well with dl_open.) Thanks to btlewis@eng.sun.com. - Fixed a serious performance bug on the DEC Alpha. The text segment was getting registered as part of the root set. (Amazingly, the result was still fast enough that the bug was not conspicuous.) The fix works on OSF/1, version 1.3. Hopefully it also works on other versions of OSF/1 ... - Fixed a bug in GC_clear_roots. - Fixed a bug in GC_generic_malloc_words_small that broke gc_inl.h. (Reported by Antoine de Maricourt. I broke it in trying to tweak the Mac port.) - Fixed some problems with cord/de under Linux. - Fixed some cord problems, notably with CORD_riter4. - Added DG/UX port. Thanks to Ben A. Mesander (ben@piglet.cr.usgs.gov) - Added finalization registration routines with weaker ordering constraints. (This is necessary for C++ finalization with multiple inheritance, since the compiler often adds self-cycles.) - Filled the holes in the SCO port. (Thanks to Michael Arnoldus .) - John Ellis' additions to the C++ support: From John: * I completely rewrote the documentation in the interface gc_c++.h (later renamed gc_cpp.h). I've tried to make it both clearer and more precise. * The definition of accessibility now ignores pointers from an finalizable object (an object with a clean-up function) to itself. This allows objects with virtual base classes to be finalizable by the collector. Compilers typically implement virtual base classes using pointers from an object to itself, which under the old definition of accessibility prevented objects with virtual base classes from ever being collected or finalized. * gc_cleanup now includes gc as a virtual base. This was enabled by the change in the definition of accessibility. * I added support for operator new[]. Since most (all?) compilers don't yet support operator new[], it is conditionalized on -DOPERATOR_NEW_ARRAY. The code is untested, but its trivial and looks correct. * The test program test_gc_c++ (later renamed test_cpp.cc) tries to test for the C++-specific functionality not tested by the other programs. - Added include to misc.c. (Needed for ppcr.) - Added PowerMac port. (Thanks to Patrick Beard again.) - Fixed "srcdir"-related Makefile problems. Changed things so that all externally visible include files always appear in the include subdirectory of the source. Made gc.h directly includable from C++ code. (These were at Per Bothner's suggestion.) - Changed Intel code to also mark from ebp (Kevin Warne's suggestion). - Renamed C++ related files so they could live in a FAT file system. (Charles Fiterman's suggestion.) - Changed Windows NT Makefile to include C++ support in gc.lib. Added C++ test as Makefile target. Since version 4.3: - ASM_CLEAR_CODE was erroneously defined for HP PA machines, resulting in a compile error. - Fixed OS/2 Makefile to create a library. (Thanks to Mark Boulter (mboulter@vnet.ibm.com)). - Gc_cleanup objects didn't work if they were created on the stack. Fixed. - One copy of Gc_cpp.h in the distribution was out of synch, and failed to document some known compiler problems with explicit destructor invocation. Partially fixed. There are probably other compilers on which gc_cleanup is miscompiled. - Fixed Makefile to pass C compiler flags to C++ compiler. - Added Mac fixes. - Fixed os_dep.c to work around what appears to be a new and different VirtualQuery bug under newer versions of win32S. - GC_non_gc_bytes was not correctly maintained by GC_free. Fixed. Thanks to James Clark (jjc@jclark.com). - Added GC_set_max_heap_size. - Changed allocation code to ignore blacklisting if it is preventing use of a very large block of memory. This has the advantage that naive code allocating very large objects is much more likely to work. The downside is you might no longer find out that such code should really use GC_malloc_ignore_off_page. - Changed GC_printf under win32 to close and reopen the file between calls. FAT file systems otherwise make the log file useless for debugging. - Added GC_try_to_collect and GC_get_bytes_since_gc. These allow starting an abortable collection during idle times. This facility does not require special OS support. (Thanks to Michael Spertus of Geodesic Systems for suggesting this. It was actually an easy addition. Kumar Srikantan previously added a similar facility to a now ancient version of the collector. At the time this was much harder, and the result was less convincing.) - Added some support for the Borland development environment. (Thanks to John Ellis and Michael Spertus.) - Removed a misfeature from checksums.c that caused unexpected heap growth. (Thanks to Scott Schwartz.) - Changed finalize.c to call WARN if it encounters a finalization cycle. WARN is defined in gc_priv.h to write a message, usually to stdout. In many environments, this may be inappropriate. - Renamed NO_PARAMS in gc.h to GC_NO_PARAMS, thus adhering to my own naming convention. - Added GC_set_warn_proc to intercept warnings. - Fixed Amiga port. (Thanks to Michel Schinz (schinz@alphanet.ch).) - Fixed a bug in mark.c that could result in an access to unmapped memory from GC_mark_from_mark_stack on machines with unaligned pointers. - Fixed a win32 specific performance bug that could result in scanning of objects allocated with the system malloc. - Added REDIRECT_MALLOC. Since version 4.4: - Fixed many minor and one major README bugs. (Thanks to Franklin Chen (chen@adi.com) for pointing out many of them.) - Fixed ALPHA/OSF/1 dynamic library support. (Thanks to Jonathan Bachrach (jonathan@harlequin.com)). - Added incremental GC support (MPROTECT_VDB) for Linux (with some help from Bruno Haible). - Altered SPARC recognition tests in gc.h and config.h (mostly as suggested by Fergus Henderson). - Added basic incremental GC support for win32, as implemented by Windows NT and Windows 95. GC_enable_incremental is a noop under win32s, which doesn't implement enough of the VM interface. - Added -DLARGE_CONFIG. - Fixed GC_..._ignore_off_page to also function without -DALL_INTERIOR_POINTERS. - (Hopefully) fixed RS/6000 port. (Only the test was broken.) - Fixed a performance bug in the nonincremental collector running on machines supporting incremental collection with MPROTECT_VDB (e.g. SunOS 4, DEC AXP). This turned into a correctness bug under win32s with win32 incremental collection. (Not all memory protection was disabled.) - Fixed some ppcr related bit rot. - Caused dynamic libraries to be unregistered before reregistering. The old way turned out to be a performance bug on some machines. - GC_root_size was not properly maintained under MSWIN32. - Added -DNO_DEBUGGING and GC_dump. - Fixed a couple of bugs arising with SOLARIS_THREADS + REDIRECT_MALLOC. - Added NetBSD/M68K port. (Thanks to Peter Seebach .) - Fixed a serious realloc bug. For certain object sizes, the collector wouldn't scan the expanded part of the object. (Thanks to Clay Spence (cds@peanut.sarnoff.com) for noticing the problem, and helping me to track it down.) Since version 4.5: - Added Linux ELF support. (Thanks to Arrigo Triulzi .) - GC_base crashed if it was called before any other GC_ routines. This could happen if a gc_cleanup object was allocated outside the heap before any heap allocation. - The heap expansion heuristic was not stable if all objects had finalization enabled. Fixed finalize.c to count memory in finalization queue and avoid explicit deallocation. Changed alloc.c to also consider this count. (This is still not recommended. It's expensive if nothing else.) Thanks to John Ellis for pointing this out. - GC_malloc_uncollectable(0) was broken. Thanks to Phong Vo for pointing this out. - The collector didn't compile under Linux 1.3.X. (Thanks to Fred Gilham for pointing this out.) The current workaround is ugly, but expected to be temporary. - Fixed a formatting problem for SPARC stack traces. - Fixed some '=='s in os_dep.c that should have been assignments. Fortunately these were in code that should never be executed anyway. (Thanks to Fergus Henderson.) - Fixed the heap block allocator to only drop blacklisted blocks in small chunks. Made BL_LIMIT self adjusting. (Both of these were in response to heap growth observed by Paul Graham.) - Fixed the Metrowerks/68K Mac code to also mark from a6. (Thanks to Patrick Beard.) - Significantly updated README.debugging. - Fixed some problems with longjmps out of signal handlers, especially under Solaris. Added a workaround for the fact that siglongjmp doesn't appear to do the right thing with -lthread under Solaris. - Added MSDOS/djgpp port. (Thanks to Mitch Harris (maharri@uiuc.edu).) - Added "make reserved_namespace" and "make user_namespace". The first renames ALL "GC_xxx" identifiers as "_GC_xxx". The second is the inverse transformation. Note that doing this is guaranteed to break all clients written for the other names. - descriptor field for kind NORMAL in GC_obj_kinds with ADD_BYTE_AT_END defined should be -ALIGNMENT not WORDS_TO_BYTES(-1). This is a serious bug on machines with pointer alignment of less than a word. - GC_ignore_self_finalize_mark_proc didn't handle pointers to very near the end of the object correctly. Caused failures of the C++ test on a DEC Alpha with g++. - gc_inl.h still had problems. Partially fixed. Added warnings at the beginning to hopefully specify the remaining dangers. - Added DATAEND definition to config.h. - Fixed some of the .h file organization. Fixed "make floppy". Since version 4.6: - Fixed some compilation problems with -DCHECKSUMS (thanks to Ian Searle) - Updated some Mac specific files to synchronize with Patrick Beard. - Fixed a serious bug for machines with non-word-aligned pointers. (Thanks to Patrick Beard for pointing out the problem. The collector should fail almost any conceivable test immediately on such machines.) Since version 4.7: - Changed a "comment" in a MacOS specific part of mach-dep.c that caused gcc to fail on other platforms. Since version 4.8 - More README.debugging fixes. - Objects ready for finalization, but not finalized in the same GC cycle, could be prematurely collected. This occasionally happened in test_cpp. - Too little memory was obtained from the system for very large objects. That could cause a heap explosion if these objects were not contiguous (e.g. under PCR), and too much of them was blacklisted. - Due to an improper initialization, the collector was too hesitant to allocate blacklisted objects immediately after system startup. - Moved GC_arrays from the data into the bss segment by not explicitly initializing it to zero. This significantly reduces the size of executables, and probably avoids some disk accesses on program startup. It's conceivable that it might break a port that I didn't test. - Fixed EMX_MAKEFILE to reflect the gc_c++.h to gc_cpp.h renaming which occurred a while ago. Since 4.9: - Fixed a typo around a call to GC_collect_or_expand in alloc.c. It broke handling of out of memory. (Thanks to Patrick Beard for noticing.) --------------------------------------------------------------------------- From dillon@flea.best.net Tue Feb 18 10:31:41 1997 Return-Path: Received: from flea.best.net (root@flea.best.net [206.184.139.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA20331 for ; Tue, 18 Feb 1997 10:31:39 -0800 (PST) Received: (from dillon@localhost) by flea.best.net (8.8.5/8.8.3) id KAA24561; Tue, 18 Feb 1997 10:30:34 -0800 (PST) Date: Tue, 18 Feb 1997 10:30:34 -0800 (PST) From: Matt Dillon Message-Id: <199702181830.KAA24561@flea.best.net> To: Phillip Musumeci Cc: adrian@cougar.aceonline.com.au, imp@village.org, auditors@freebsd.org Subject: Re: Re : Bounds-checking gcc .. :>>>>> "Adrian" == Adrian Chadd writes: : : Warner> ......... will do nothing for the race conditions, the bad uses : Warner> of mktemp, et al, the sloppy use of seteuid(), badly written : Warner> setuid programs, etc. : :A few years back, sunos had a bit of a scare with sendmail launching shell :scripts [or other programs (?)] that could inherit inappropriate :environment variables. A temporary fix that was circulated at the time was :a very simple piece of code that cleaned out the environment before running :the desired task via a call to execl() or one of its friends. This wrapper :was simple and therefore easy to trust. : :If FreeBSD has a need for one task to call another task with a safe :environment (no undesired LD_LIBRARY_PATHs etc.), maybe we could also have :a single piece of well-read trusted source code that could be maintained :for use as a wrapper where appropriate. : :Sorry if this is getting off the original thread. : :phillip Ooohh... now here's an idea. One could require that suid root binaries make a system call to 'enable' the suid operation. This system call would be embedded in a securitize() subroutine which would go through, clear up the environment, fix the resource limits, and enable suid oeration. If the binary does not make the system call, it doesn't get suid privilages even if chmod'd 4xxx. The program could pass a list of environment variables to the system call to 'pass through'. i.e. the suid bit would have to be set AND the program would have to make the system call. -Matt Matthew Dillon Engineering, BEST Internet Communications, Inc. [always include a portion of the original email in any response!] From imp@village.org Tue Feb 18 10:38:50 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id KAA20683 for ; Tue, 18 Feb 1997 10:38:48 -0800 (PST) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 0.56 #1) id E0vwuQi-0004bp-00; Tue, 18 Feb 1997 11:38:16 -0700 To: Matt Dillon Subject: Re: Re : Bounds-checking gcc .. Cc: Phillip Musumeci , adrian@cougar.aceonline.com.au, auditors@freebsd.org In-reply-to: Your message of "Tue, 18 Feb 1997 10:30:34 PST." <199702181830.KAA24561@flea.best.net> References: <199702181830.KAA24561@flea.best.net> Date: Tue, 18 Feb 1997 11:38:16 -0700 From: Warner Losh Message-Id: In message <199702181830.KAA24561@flea.best.net> Matt Dillon writes: : :>>>>> "Adrian" == Adrian Chadd writes: : Ooohh... now here's an idea. One could require that suid root binaries : make a system call to 'enable' the suid operation. This system call would : be embedded in a securitize() subroutine which would go through, clear : up the environment, fix the resource limits, and enable suid oeration. : If the binary does not make the system call, it doesn't get suid : privilages even if chmod'd 4xxx. The program could pass a list of : environment variables to the system call to 'pass through'. : : i.e. the suid bit would have to be set AND the program would have to : make the system call. I'm not sure what this would buy you, other than a lot of grief. You'd have to start the setuid program out w/o euid changed from the current uid. You'd also not gain any secuiryt from buffer overflows (since the buffer overflow code could call sanitize itself). It would also still not keep you safe from LD_* stuff since that is all done before main gets called. You'd have to add yet another field to the proc structure to keep track of this. Also, how would the kernel know what is to be trusted and not trusted in the env? It sounds like an aweful lot of work for very little real gain. Warner From dillon@flea.best.net Tue Feb 18 10:57:47 1997 Return-Path: Received: from flea.best.net (root@flea.best.net [206.184.139.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA21638 for ; Tue, 18 Feb 1997 10:57:42 -0800 (PST) Received: (from dillon@localhost) by flea.best.net (8.8.5/8.8.3) id KAA25438; Tue, 18 Feb 1997 10:56:17 -0800 (PST) Date: Tue, 18 Feb 1997 10:56:17 -0800 (PST) From: Matt Dillon Message-Id: <199702181856.KAA25438@flea.best.net> To: Warner Losh Cc: Phillip Musumeci , adrian@cougar.aceonline.com.au, auditors@freebsd.org Subject: Re: Re : Bounds-checking gcc .. :In message <199702181830.KAA24561@flea.best.net> Matt Dillon writes: :: :>>>>> "Adrian" == Adrian Chadd writes: :: Ooohh... now here's an idea. One could require that suid root binaries :: make a system call to 'enable' the suid operation. This system call would :: be embedded in a securitize() subroutine which would go through, clear :: up the environment, fix the resource limits, and enable suid oeration. :: If the binary does not make the system call, it doesn't get suid :: privilages even if chmod'd 4xxx. The program could pass a list of :: environment variables to the system call to 'pass through'. :: :: i.e. the suid bit would have to be set AND the program would have to :: make the system call. : :I'm not sure what this would buy you, other than a lot of grief. :You'd have to start the setuid program out w/o euid changed from the :current uid. You'd also not gain any secuiryt from buffer overflows :(since the buffer overflow code could call sanitize itself). It would :also still not keep you safe from LD_* stuff since that is all done :before main gets called. You'd have to add yet another field to the :proc structure to keep track of this. Also, how would the kernel know :what is to be trusted and not trusted in the env? : :It sounds like an aweful lot of work for very little real gain. : :Warner There is a quite a bit of real gain.. It takes work to secure an suid program? Of course it does! You have to do these things anyway if you want an suid program to be truely secure, requiring it only formalizes it. i.e. it means that programmers who would otherwise have no understanding of what 'suid' means would be required to actually *learn* something before writing an suid program. It's better to take an approach similar to the firewall approach... if firewalls are turned on, the *default* operation is to disallow all packet traffic, period. The default operation is to err on the side securedness. The same approach ought to be taken for suid programs. The default should be 'secure', requiring the program to explicitly state what it wants, not 'insecure', where any joe programmer can say 'gee, I'll just make this program suid and not worry about the consequences'. The logistical problems are minor. -- There are a number of ways we can do this... for example, the startup code could, by default, clear the environment if the program was run suid, re-setting it with minimal values and a basic PATH. Fuck the LD_* stuff... those environment variables should never have existed in the first place. If anything, the library load path should be an inherited system resources of some sort. It could store the original envp in an accessible location allowing the main program to 'restore' those environment variable that it explicitly needs. The startup code could also check the resource set if the program is run suid and exit if the resources are not considered 'reasonable'. This would not require a system call to 'enable' suid operation, but it would require suid programs to explicitly specify which (if any) third party environment variables they want to pass through. -Matt Matthew Dillon Engineering, BEST Internet Communications, Inc. [always include a portion of the original email in any response!] From imp@village.org Tue Feb 18 13:40:31 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id NAA03266 for ; Tue, 18 Feb 1997 13:40:27 -0800 (PST) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 0.56 #1) id E0vwxFk-0004oI-00; Tue, 18 Feb 1997 14:39:08 -0700 To: Matt Dillon Subject: Re: Re : Bounds-checking gcc .. Cc: Phillip Musumeci , adrian@cougar.aceonline.com.au, auditors@freebsd.org In-reply-to: Your message of "Tue, 18 Feb 1997 10:56:17 PST." <199702181856.KAA25438@flea.best.net> References: <199702181856.KAA25438@flea.best.net> Date: Tue, 18 Feb 1997 14:39:08 -0700 From: Warner Losh Message-Id: In message <199702181856.KAA25438@flea.best.net> Matt Dillon writes: : There is a quite a bit of real gain.. It takes work to secure : an suid program? Of course it does! You have to do these things : anyway if you want an suid program to be truely secure, requiring : it only formalizes it. i.e. it means that programmers who would : otherwise have no understanding of what 'suid' means would be : required to actually *learn* something before writing an suid : program. That doesn't necessarily follow. Just because they add additional code doesn't mean they'll not do stupid things. This cuts off only one avenue of explotation (the environment variables). : The same approach ought to be taken for suid programs. : The default should be 'secure', requiring the program to explicitly : state what it wants, not 'insecure', where any joe programmer can : say 'gee, I'll just make this program suid and not worry about the : consequences'. The logistical problems are minor. I agree that this could be desirable. However, I'm worried about making sure that we really do have better security afterwards. I worry about mucking too much with the kernel to make this happen, since the euid, et al, semantics are hard enough to understand and verify w/o the added hair of "you gotta enable first". : There are a number of ways we can do this... for example, the : startup code could, by default, clear the environment if the : program was run suid, re-setting it with minimal values and : a basic PATH. Fuck the LD_* stuff... those environment variables : should never have existed in the first place. If anything, the library : load path should be an inherited system resources of some sort. It : could store the original envp in an accessible location allowing the main : program to 'restore' those environment variable that it explicitly needs. : The startup code could also check the resource set if the program : is run suid and exit if the resources are not considered : 'reasonable'. That's a very interesting idea. However, I worry about the cost associated with this sanitation for every program on the system. That is, I worry about the added codebloat to crt0.o which sounds like where you'd place this functionality. If you can come up with (or someone can) a good way to firewall setuid programs that doesn't cost those programs that aren't setuid much (in terms of size or speed), then you may have a winner. : This would not require a system call to 'enable' suid operation, : but it would require suid programs to explicitly specify which : (if any) third party environment variables they want to pass through. True. You have effectively only cut off one form of attack. You still have the problems of what to do about long names on the command line, what to do about long lines the program reads, or DoS-type attacks based on the tmp files that a program is using (some of which happen even in the non-setuid case). This would be a fairly bogus firewall, since it would be really good about some things, but now right promiscuous about others. If you can come up with a good, clear design for this firewall code, I'd be happy to review it to tell you what your vulnerabilities will be. I'd be willing to work with you on designing something that will help protect against many of the common attacks by limiting what information can flow to a setuid program by default. This may be a good idea, but there are a lot of problems with it as you've expressed it. Heck, you could even have the program chroot to /setuid/pid which is a very minimal file system to isolate the damage even more. However, the more you isolate a program, the less useful it will be. Warner From dillon@flea.best.net Tue Feb 18 13:48:27 1997 Return-Path: Received: from flea.best.net (root@flea.best.net [206.184.139.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id NAA03593 for ; Tue, 18 Feb 1997 13:48:24 -0800 (PST) Received: (from dillon@localhost) by flea.best.net (8.8.5/8.8.3) id NAA01438; Tue, 18 Feb 1997 13:47:06 -0800 (PST) Date: Tue, 18 Feb 1997 13:47:06 -0800 (PST) From: Matt Dillon Message-Id: <199702182147.NAA01438@flea.best.net> To: Warner Losh Cc: Phillip Musumeci , adrian@cougar.aceonline.com.au, auditors@freebsd.org Subject: Re: Re : Bounds-checking gcc .. :If you can come up with a good, clear design for this firewall code, :I'd be happy to review it to tell you what your vulnerabilities will :be. I'd be willing to work with you on designing something that will :help protect against many of the common attacks by limiting what :information can flow to a setuid program by default. This may be a :good idea, but there are a lot of problems with it as you've expressed :it. : :Heck, you could even have the program chroot to /setuid/pid which is a :very minimal file system to isolate the damage even more. However, :the more you isolate a program, the less useful it will be. : :Warner Well.. no duh! But if you insist on a perfect solution, the likely outcome will be NO solution at all. Complete solutions are possible only if you do something, say, like redesign the whole damn operating system from scratch. Having a partial solution is better then nothing, especially if it's scaleable (i.e. you start tackling the various system/library resources one at a time). This is getting off topic.. I am not suggesting that we do any of this now, but if you want to keep the system 'clean' in future releases, a 'default more secure' rather then a 'default less secure' paradigm is an absolute necessity. We will never be able to guarentee the system against bozo programmers, but we can do a whole lot better then what we have now. -Matt Matthew Dillon Engineering, BEST Internet Communications, Inc. [always include a portion of the original email in any response!] From adrian@cougar.aceonline.com.au Tue Feb 18 15:28:03 1997 Return-Path: Received: from cougar.aceonline.com.au (adrian@cougar.aceonline.com.au [203.103.81.36]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id PAA08867 for ; Tue, 18 Feb 1997 15:28:00 -0800 (PST) Received: from localhost (adrian@localhost) by cougar.aceonline.com.au (8.8.4/8.7) with SMTP id HAA14015 for ; Wed, 19 Feb 1997 07:27:47 +0800 Date: Wed, 19 Feb 1997 07:27:47 +0800 (WST) From: Adrian Chadd To: auditors@freebsd.org Subject: Re: Re : Bounds-checking gcc .. In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Tue, 18 Feb 1997, Warner Losh wrote: > In message <199702181856.KAA25438@flea.best.net> Matt Dillon writes: > : There is a quite a bit of real gain.. It takes work to secure > : an suid program? Of course it does! You have to do these things > : anyway if you want an suid program to be truely secure, requiring > : it only formalizes it. i.e. it means that programmers who would > : otherwise have no understanding of what 'suid' means would be > : required to actually *learn* something before writing an suid > : program. > > That doesn't necessarily follow. Just because they add additional > code doesn't mean they'll not do stupid things. This cuts off only > one avenue of explotation (the environment variables). > Thats one whole avenue that when it comes down to it, means we have to think less of it from now on. Again, sure we can *CODE* thinking about the problems, to make our code nice and secure to start with, but at the end of the day we have some new layer of security to guard against anything we might miss. Cya. Adrian. From root@ka.home.hft.co.uk Tue Feb 18 22:05:48 1997 Return-Path: Received: from zeus.tcp.net.uk (zeus.tcp.net.uk [195.80.0.33]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id WAA05428 for ; Tue, 18 Feb 1997 22:05:36 -0800 (PST) Received: from ka.home.hft.co.uk (root@ai091.du.pipex.com [193.130.248.91]) by zeus.tcp.net.uk (8.7.6/8.7) with ESMTP id GAA19615 for ; Wed, 19 Feb 1997 06:04:52 GMT Received: (from root@localhost) by ka.home.hft.co.uk (8.7.4/8.6.9) id GAA20136; Wed, 19 Feb 1997 06:04:47 GMT Received: from etamin.brunel.ac.uk (pp@etamin.brunel.ac.uk [134.83.128.61]) by zeus.tcp.net.uk (8.7.6/8.7) with SMTP id WAA28024 for ; Tue, 18 Feb 1997 22:20:32 GMT Received: from freefall.freebsd.org by etamin.brunel.ac.uk with SMTP (PP); Tue, 18 Feb 1997 22:18:45 +0000 Received: from zeus.tcp.net.uk (zeus.tcp.net.uk [195.80.0.33]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA18967 for ; Tue, 18 Feb 1997 10:05:42 -0800 (PST) Received: from ka.home.hft.co.uk (root@ao093.du.pipex.com [193.130.254.93]) by zeus.tcp.net.uk (8.7.6/8.7) with ESMTP id SAA15334 for ; Tue, 18 Feb 1997 18:05:01 GMT Received: (from root@localhost) by ka.home.hft.co.uk (8.7.4/8.6.9) id SAA17580; Tue, 18 Feb 1997 18:04:55 GMT Received: from etamin.brunel.ac.uk (pp@etamin.brunel.ac.uk [134.83.128.61]) by zeus.tcp.net.uk (8.7.6/8.7) with SMTP id GAA16574 for ; Tue, 18 Feb 1997 06:32:38 GMT Received: from freefall.freebsd.org by etamin.brunel.ac.uk with SMTP (PP); Tue, 18 Feb 1997 06:32:00 +0000 Received: from zeus.tcp.net.uk (zeus.tcp.net.uk [195.80.0.33]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id WAA05788 for ; Mon, 17 Feb 1997 22:29:20 -0800 (PST) Received: from ka.home.hft.co.uk (root@al047.du.pipex.com [193.130.251.47]) by zeus.tcp.net.uk (8.7.6/8.7) with ESMTP id GAA16347 for ; Tue, 18 Feb 1997 06:28:38 GMT Received: (from root@localhost) by ka.home.hft.co.uk (8.7.4/8.6.9) id GAA17285; Tue, 18 Feb 1997 06:28:34 GMT Received: from ceres.brunel.ac.uk (pp@ceres.brunel.ac.uk [134.83.176.3]) by zeus.tcp.net.uk (8.7.6/8.7) with SMTP id AAA04461 for ; Tue, 18 Feb 1997 00:53:45 GMT Received: from freefall.freebsd.org by ceres.brunel.ac.uk with SMTP (PP); Tue, 18 Feb 1997 00:53:09 +0000 Received: from time.cdrom.com (time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA29615 for ; Mon, 17 Feb 1997 16:51:15 -0800 (PST) Received: from time.cdrom.com (localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id QAA28445 for ; Mon, 17 Feb 1997 16:51:12 -0800 (PST) Prev-Resent: Mon, 17 Feb 1997 16:51:11 -0800 Prev-Resent: "auditors@freebsd.org " Received: from mirriwinni.cse.rmit.edu.au (pm.cse.rmit.EDU.AU [131.170.118.50]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id FAA01569 for ; Mon, 17 Feb 1997 05:08:14 -0800 (PST) Received: (from phillip@localhost) by mirriwinni.cse.rmit.edu.au (8.8.3/8.6.12) id AAA26318; Tue, 18 Feb 1997 00:08:10 +1100 (EST) Date: Tue, 18 Feb 1997 00:08:10 +1100 (EST) Message-Id: <199702171308.AAA26318@mirriwinni.cse.rmit.edu.au> From: Phillip Musumeci X-Orig-To: "Jordan K. Hubbard" In-reply-to: "Jordan K. Hubbard"'s message of 11 Feb 1997 22:39:19 -0600 Subject: Re: Security Advisory - Recent compromise of freefall.freebsd.org References: <5drhhn$4fd@bonkers.taronga.com> Resent-To: auditors@freebsd.org Resent-Date: Mon, 17 Feb 1997 16:51:11 -0800 Resent-Message-ID: <28442.856227071@time.cdrom.com> Resent-From: "Jordan K. Hubbard" X-Processed-By: smtp-pop3 2.0 at TCP Ltd X-MAIL-From: jkh@time.cdrom.com X-Orig-To: ppb@baloo.tcp.co.uk X-Processed-By: smtp-pop3 2.0 at TCP Ltd X-MAIL-From: ppb@baloo.tcp.co.uk X-Orig-To: ppb@baloo.tcp.co.uk X-Processed-By: smtp-pop3 2.0 at TCP Ltd X-MAIL-From: root@ka.home.hft.co.uk To: ppb@baloo.tcp.co.uk X-UIDL: 0fb962433a954ed777c2f802c02d9faf Hi, Jordan> ... to begin a much more serious and comprehensive security Jordan> audit, we will take advantage of this opportunity to see that Jordan> all code in the FreeBSD source tree, old and new alike, is Jordan> reviewed line by line for buffer overflows, unguarded copies, Jordan> back doors, whatever. I think this source code checking is a good idea. Another thing that might be practical, or maybe not (I'm not sure), is run-time checking for buffer overflows. There exists a set of memory allocation routines that I have seen other people use in order to locate memory leaks and find illegal memory accesses via pointers (e.g. writing past the end of an allocated piece of memory storage --- search for string "writing past"). It has been ported to FreeBSD, and here is the README file. Maybe this alternative memory allocation library could be used whenever someone wanted to "test drive" a utility and look for memory leaks. The default make could use the normal memory allocation. As I said, this is just an idea and I'm not sure how practical it would be to use it for FreeBSD testing. I thought I recognised the name of the guy who did the FreeBSD though, so maybe he is closely associated with the FreeBSD team and has already suggested this. regards, phillip p.s. I became aware of this set of memory management tools because it is used in the public domain RLaB matrix maths package. --------------------------------------------------------------------------- Copyright 1988, 1989 Hans-J. Boehm, Alan J. Demers Copyright (c) 1991-1995 by Xerox Corporation. All rights reserved. THIS MATERIAL IS PROVIDED AS IS, WITH ABSOLUTELY NO WARRANTY EXPRESSED OR IMPLIED. ANY USE IS AT YOUR OWN RISK. Permission is hereby granted to use or copy this program for any purpose, provided the above notices are retained on all copies. Permission to modify the code and to distribute modified code is granted, provided the above notices are retained, and a notice that the code was modified is included with the above copyright notice. This is version 4.10 of a conservative garbage collector for C and C++. HISTORY - Early versions of this collector were developed as a part of research projects supported in part by the National Science Foundation and the Defense Advance Research Projects Agency. Much of the code was rewritten by Hans-J. Boehm at Xerox PARC. The SPARC specific code was contributed by Mark Weiser (weiser@parc.xerox.com). The Encore Multimax modifications were supplied by Kevin Kenny (kenny@m.cs.uiuc.edu). The adaptation to the RT is largely due to Vernon Lee (scorpion@rice.edu), on machines made available by IBM. Much of the HP specific code and a number of good suggestions for improving the generic code are due to Walter Underwood (wunder@hp-ses.sde.hp.com). Robert Brazile (brazile@diamond.bbn.com) originally supplied the ULTRIX code. Al Dosser (dosser@src.dec.com) and Regis Cridlig (Regis.Cridlig@cl.cam.ac.uk) subsequently provided updates and information on variation between ULTRIX systems. Parag Patel (parag@netcom.com) supplied the A/UX code. Jesper Peterson(jep@mtiame.mtia.oz.au) and Michel Schinz supplied the Amiga port. Thomas Funke (thf@zelator.in-berlin.de(?)) and Brian D.Carlstrom (bdc@clark.lcs.mit.edu) supplied the NeXT ports. Douglas Steel (doug@wg.icl.co.uk) provided ICL DRS6000 code. Bill Janssen (janssen@parc.xerox.com) supplied the SunOS dynamic loader specific code. Manuel Serrano (serrano@cornas.inria.fr) supplied linux and Sony News specific code. Al Dosser provided Alpha/OSF/1 code. He and Dave Detlefs(detlefs@src.dec.com) also provided several generic bug fixes. Alistair G. Crooks(agc@uts.amdahl.com) supplied the NetBSD and 386BSD ports. Jeffrey Hsu (hsu@soda.berkeley.edu) provided the FreeBSD port. Brent Benson (brent@jade.ssd.csd.harris.com) ported the collector to a Motorola 88K processor running CX/UX (Harris NightHawk). Ari Huttunen (Ari.Huttunen@hut.fi) generalized the OS/2 port to nonIBM development environments (a nontrivial task). Patrick Beard (beard@cs.ucdavis.edu) provided the initial MacOS port. David Chase, then at Olivetti Research, suggested several improvements. Scott Schwartz (schwartz@groucho.cse.psu.edu) supplied some of the code to save and print call stacks for leak detection on a SPARC. Jesse Hull and John Ellis supplied the C++ interface code. Zhong Shao performed much of the experimentation that led to the current typed allocation facility. (His dynamic type inference code hasn't made it into the released version of the collector, yet.) (Blame for misinstallation of these modifications goes to the first author, however.) This is intended to be a general purpose, garbage collecting storage allocator. The algorithms used are described in: Boehm, H., and M. Weiser, "Garbage Collection in an Uncooperative Environment", Software Practice & Experience, September 1988, pp. 807-820. Boehm, H., A. Demers, and S. Shenker, "Mostly Parallel Garbage Collection", Proceedings of the ACM SIGPLAN '91 Conference on Programming Language Design and Implementation, SIGPLAN Notices 26, 6 (June 1991), pp. 157-164. Boehm, H., "Space Efficient Conservative Garbage Collection", Proceedings of the ACM SIGPLAN '91 Conference on Programming Language Design and Implementation, SIGPLAN Notices 28, 6 (June 1993), pp. 197-206. Possible interactions between the collector and optimizing compilers are discussed in Boehm, H., and D. Chase, "A Proposal for GC-safe C Compilation", The Journal of C Language Translation 4, 2 (December 1992). (Also available from parcftp.xerox.com:pub/gc, among other places.) Unlike the collector described in the second reference, this collector operates either with the mutator stopped during the entire collection (default) or incrementally during allocations. (The latter is supported on only a few machines.) It does not rely on threads, but is intended to be thread-safe. Some of the ideas underlying the collector have previously been explored by others. (Doug McIlroy wrote a vaguely similar collector that is part of version 8 UNIX (tm).) However none of this work appears to have been widely disseminated. Rudimentary tools for use of the collector as a leak detector are included, as is a fairly sophisticated string package "cord" that makes use of the collector. (See cord/README.) GENERAL DESCRIPTION This is a garbage collecting storage allocator that is intended to be used as a plug-in replacement for C's malloc. Since the collector does not require pointers to be tagged, it does not attempt to ensure that all inaccessible storage is reclaimed. However, in our experience, it is typically more successful at reclaiming unused memory than most C programs using explicit deallocation. Unlike manually introduced leaks, the amount of unreclaimed memory typically stays bounded. In the following, an "object" is defined to be a region of memory allocated by the routines described below. Any objects not intended to be collected must be pointed to either from other such accessible objects, or from the registers, stack, data, or statically allocated bss segments. Pointers from the stack or registers may point to anywhere inside an object. However, it is usually assumed that all pointers originating in the heap point to the beginning of an object. (This does not disallow interior pointers; it simply requires that there must be a pointer to the beginning of every accessible object, in addition to any interior pointers.) There are two facilities for altering this behavior. The macro ALL_INTERIOR_POINTERS may be defined in gc_private.h to cause any pointer into an object (or one past the end) to retain the object. A routine GC_register_displacement is provided to allow for more controlled interior pointer use in the heap. Defining ALL_INTERIOR_POINTERS is somewhat dangerous, in that it can result in unnecessary memory retention. However this is much less of a problem than with older collector versions. The routine GC_register_displacement is described in gc.h. Note that pointers inside memory allocated by the standard "malloc" are not seen by the garbage collector. Thus objects pointed to only from such a region may be prematurely deallocated. It is thus suggested that the standard "malloc" be used only for memory regions, such as I/O buffers, that are guaranteed not to contain pointers. Pointers in C language automatic, static, or register variables, are correctly recognized. (Note that GC_malloc_uncollectable has semantics similar to standard malloc, but allocates objects that are traced by the collector.) The collector does not generally know how to find pointers in data areas that are associated with dynamic libraries. This is easy to remedy IF you know how to find those data areas on your operating system (see GC_add_roots). Code for doing this under SunOS and IRIX 5.X is included (see dynamic_load.c). Note that the garbage collector does not need to be informed of shared read-only data. However if the shared library mechanism can introduce discontiguous data areas that may contain pointers, then the collector does need to be informed. Signal processing for most signals may be deferred during collection, and during uninterruptible parts of the allocation process. Unlike standard ANSI C mallocs, it can be safe to invoke malloc from a signal handler while another malloc is in progress, provided the original malloc is not restarted. (Empirically, many UNIX applications already assume this.) To obtain this level of signal safety, remove the definition of -DNO_SIGNALS in Makefile. The allocator/collector can also be configured for thread-safe operation. (Full signal safety can also be achieved, but only at the cost of two system calls per malloc, which is usually unacceptable.) INSTALLATION AND PORTABILITY As distributed, the macro SILENT is defined in Makefile. In the event of problems, this can be removed to obtain a moderate amount of descriptive output for each collection. (The given statistics exhibit a few peculiarities. Things don't appear to add up for a variety of reasons, most notably fragmentation losses. These are probably much more significant for the contrived program "test.c" than for your application.) Note that typing "make test" will automatically build the collector and then run setjmp_test and gctest. Setjmp_test will give you information about configuring the collector, which is useful primarily if you have a machine that's not already supported. Gctest is a somewhat superficial test of collector functionality. Failure is indicated by a core dump or a message to the effect that the collector is broken. Gctest takes about 35 seconds to run on a SPARCstation 2. On a slower machine, expect it to take a while. It may use up to 8 MB of memory. (The multi-threaded version will use more.) "Make test" will also, as its last step, attempt to build and test the "cord" string library. This will fail without an ANSI C compiler. The Makefile will generate a library gc.a which you should link against. Typing "make cords" will add the cord library to gc.a. Note that this requires an ANSI C compiler. It is suggested that if you need to replace a piece of the collector (e.g. GC_mark_rts.c) you simply list your version ahead of gc.a on the ld command line, rather than replacing the one in gc.a. (This will generate numerous warnings under some versions of AIX, but it still works.) All include files that need to be used by clients will be put in the include subdirectory. (Normally this is just gc.h. "Make cords" adds "cord.h" and "ec.h".) The collector currently is designed to run essentially unmodified on the following machines (most of the operating systems mentioned are trademarks of their respective holders): Sun 3 Sun 4 under SunOS 4.X or Solaris2.X (with or without threads) Vax under 4.3BSD, Ultrix Intel 386 or 486 under most operating systems, but not MSDOS. (Win32S is somewhat supported, so it is possible to build applications for Windows 3.1. There exists a port to DOS + 32 bit extender for at least one 32 bit extender. However, I don't have source for this.) Sequent Symmetry (single threaded) Encore Multimax (single threaded) MIPS M/120 (and presumably M/2000) (RISC/os 4.0 with BSD libraries) IBM PC/RT (Berkeley UNIX) IBM RS/6000 HP9000/300 HP9000/700 DECstations under Ultrix DEC Alpha running OSF/1 SGI workstations under IRIX 4 & 5 Sony News Apple Macintosh under A/UX or MacOS Commodore Amiga (see README.amiga) NeXT machines In a few cases (Amiga, OS/2, Win32, MacOS) a separate makefile or equivalent is supplied. Dynamic libraries are completely supported only under SunOS (and even that support is not functional on the last Sun 3 release), IRIX 5, Win32 (not Win32S) and OSF/1 on DEC AXP machines. On other machines we recommend that you do one of the following: 1) Add dynamic library support (and send us the code). 2) Use static versions of the libraries. 3) Arrange for dynamic libraries to use the standard malloc. This is still dangerous if the library stores a pointer to a garbage collected object. But nearly all standard interfaces prohibit this, because they deal correctly with pointers to stack allocated objects. (Strtok is an exception. Don't use it.) In all cases we assume that pointer alignment is consistent with that enforced by the standard C compilers. If you use a nonstandard compiler you may have to adjust the alignment parameters defined in gc_priv.h. A port to a machine that is not byte addressed, or does not use 32 bit or 64 bit addresses will require a major effort. A port to MSDOS is hard, unless you are willing to assume an 80386 or better, and that only flat 32 bit pointers will ever need to be seen by the collector. For machines not already mentioned, or for nonstandard compilers, the following are likely to require change: 1. The parameters in config.h. The parameters that will usually require adjustment are STACKBOTTOM, ALIGNMENT and DATASTART. Setjmp_test prints its guesses of the first two. DATASTART should be an expression for computing the address of the beginning of the data segment. This can often be &etext. But some memory management units require that there be some unmapped space between the text and the data segment. Thus it may be more complicated. On UNIX systems, this is rarely documented. But the adb "$m" command may be helpful. (Note that DATASTART will usually be a function of &etext. Thus a single experiment is usually insufficient.) STACKBOTTOM is used to initialize GC_stackbottom, which should be a sufficient approximation to the coldest stack address. On some machines, it is difficult to obtain such a value that is valid across a variety of MMUs, OS releases, etc. A number of alternatives exist for using the collector in spite of this. See the discussion in config.h immediately preceding the various definitions of STACKBOTTOM. 2. mach_dep.c. The most important routine here is one to mark from registers. The distributed file includes a generic hack (based on setjmp) that happens to work on many machines, and may work on yours. Try compiling and running setjmp_t.c to see whether it has a chance of working. (This is not correct C, so don't blame your compiler if it doesn't work. Based on limited experience, register window machines are likely to cause trouble. If your version of setjmp claims that all accessible variables, including registers, have the value they had at the time of the longjmp, it also will not work. Vanilla 4.2 BSD on Vaxen makes such a claim. SunOS does not.) If your compiler does not allow in-line assembly code, or if you prefer not to use such a facility, mach_dep.c may be replaced by a .s file (as we did for the MIPS machine and the PC/RT). At this point enough architectures are supported by mach_dep.c that you will rarely need to do more than adjust for assembler syntax. 3. os_dep.c (and gc_priv.h). Several kinds of operating system dependent routines reside here. Many are optional. Several are invoked only through corresponding macros in gc_priv.h, which may also be redefined as appropriate. The routine GC_register_data_segments is crucial. It registers static data areas that must be traversed by the collector. (User calls to GC_add_roots may sometimes be used for similar effect.) Routines to obtain memory from the OS also reside here. Alternatively this can be done entirely by the macro GET_MEM defined in gc_priv.h. Routines to disable and reenable signals also reside here if they are need by the macros DISABLE_SIGNALS and ENABLE_SIGNALS defined in gc_priv.h. In a multithreaded environment, the macros LOCK and UNLOCK in gc_priv.h will need to be suitably redefined. The incremental collector requires page dirty information, which is acquired through routines defined in os_dep.c. Unless directed otherwise by config.h, these are implemented as stubs that simply treat all pages as dirty. (This of course makes the incremental collector much less useful.) 4. dyn_load.c This provides a routine that allows the collector to scan data segments associated with dynamic libraries. Often it is not necessary to provide this routine unless user-written dynamic libraries are used. For a different version of UN*X or different machines using the Motorola 68000, Vax, SPARC, 80386, NS 32000, PC/RT, or MIPS architecture, it should frequently suffice to change definitions in config.h. THE C INTERFACE TO THE ALLOCATOR The following routines are intended to be directly called by the user. Note that usually only GC_malloc is necessary. GC_clear_roots and GC_add_roots calls may be required if the collector has to trace from nonstandard places (e.g. from dynamic library data areas on a machine on which the collector doesn't already understand them.) On some machines, it may be desirable to set GC_stacktop to a good approximation of the stack base. (This enhances code portability on HP PA machines, since there is no good way for the collector to compute this value.) Client code may include "gc.h", which defines all of the following, plus many others. 1) GC_malloc(nbytes) - allocate an object of size nbytes. Unlike malloc, the object is cleared before being returned to the user. Gc_malloc will invoke the garbage collector when it determines this to be appropriate. GC_malloc may return 0 if it is unable to acquire sufficient space from the operating system. This is the most probable consequence of running out of space. Other possible consequences are that a function call will fail due to lack of stack space, or that the collector will fail in other ways because it cannot maintain its internal data structures, or that a crucial system process will fail and take down the machine. Most of these possibilities are independent of the malloc implementation. 2) GC_malloc_atomic(nbytes) - allocate an object of size nbytes that is guaranteed not to contain any pointers. The returned object is not guaranteed to be cleared. (Can always be replaced by GC_malloc, but results in faster collection times. The collector will probably run faster if large character arrays, etc. are allocated with GC_malloc_atomic than if they are statically allocated.) 3) GC_realloc(object, new_size) - change the size of object to be new_size. Returns a pointer to the new object, which may, or may not, be the same as the pointer to the old object. The new object is taken to be atomic iff the old one was. If the new object is composite and larger than the original object, then the newly added bytes are cleared (we hope). This is very likely to allocate a new object, unless MERGE_SIZES is defined in gc_priv.h. Even then, it is likely to recycle the old object only if the object is grown in small additive increments (which, we claim, is generally bad coding practice.) 4) GC_free(object) - explicitly deallocate an object returned by GC_malloc or GC_malloc_atomic. Not necessary, but can be used to minimize collections if performance is critical. Probably a performance loss for very small objects (<= 8 bytes). 5) GC_expand_hp(bytes) - Explicitly increase the heap size. (This is normally done automatically if a garbage collection failed to GC_reclaim enough memory. Explicit calls to GC_expand_hp may prevent unnecessarily frequent collections at program startup.) 6) GC_malloc_ignore_off_page(bytes) - identical to GC_malloc, but the client promises to keep a pointer to the somewhere within the first 256 bytes of the object while it is live. (This pointer should nortmally be declared volatile to prevent interference from compiler optimizations.) This is the recommended way to allocate anything that is likely to be larger than 100Kbytes or so. (GC_malloc may result in failure to reclaim such objects.) 7) GC_set_warn_proc(proc) - Can be used to redirect warnings from the collector. Such warnings should be rare, and should not be ignored during code development. 8) GC_enable_incremental() - Enables generational and incremental collection. Useful for large heaps on machines that provide access to page dirty information. Some dirty bit implementations may interfere with debugging (by catching address faults) and place restrictions on heap arguments to system calls (since write faults inside a system call may not be handled well). 9) Several routines to allow for registration of finalization code. User supplied finalization code may be invoked when an object becomes unreachable. To call (*f)(obj, x) when obj becomes inaccessible, use GC_register_finalizer(obj, f, x, 0, 0); For more sophisticated uses, and for finalization ordering issues, see gc.h. The global variable GC_free_space_divisor may be adjusted up from its default value of 4 to use less space and more collection time, or down for the opposite effect. Setting it to 1 or 0 will effectively disable collections and cause all allocations to simply grow the heap. The variable GC_non_gc_bytes, which is normally 0, may be changed to reflect the amount of memory allocated by the above routines that should not be considered as a candidate for collection. Careless use may, of course, result in excessive memory consumption. Some additional tuning is possible through the parameters defined near the top of gc_priv.h. If only GC_malloc is intended to be used, it might be appropriate to define: #define malloc(n) GC_malloc(n) #define calloc(m,n) GC_malloc((m)*(n)) For small pieces of VERY allocation intensive code, gc_inl.h includes some allocation macros that may be used in place of GC_malloc and friends. All externally visible names in the garbage collector start with "GC_". To avoid name conflicts, client code should avoid this prefix, except when accessing garbage collector routines or variables. There are provisions for allocation with explicit type information. This is rarely necessary. Details can be found in gc_typed.h. THE C++ INTERFACE TO THE ALLOCATOR: The Ellis-Hull C++ interface to the collector is included in the collector distribution. If you intend to use this, type "make c++" after the initial build of the collector is complete. See gc_cpp.h for the definition of the interface. This interface tries to approximate the Ellis-Detlefs C++ garbage collection proposal without compiler changes. Cautions: 1. Arrays allocated without new placement syntax are allocated as uncollectable objects. They are traced by the collector, but will not be reclaimed. 2. Failure to use "make c++" in combination with (1) will result in arrays allocated using the default new operator. This is likely to result in disaster without linker warnings. 3. If your compiler supports an overloaded new[] operator, then gc_cpp.cc and gc_cpp.h should be suitably modified. 4. Many current C++ compilers have deficiencies that break some of the functionality. See the comments in gc_cpp.h for suggested workarounds. USE AS LEAK DETECTOR: The collector may be used to track down leaks in C programs that are intended to run with malloc/free (e.g. code with extreme real-time or portability constraints). To do so define FIND_LEAK in Makefile This will cause the collector to invoke the report_leak routine defined near the top of reclaim.c whenever an inaccessible object is found that has not been explicitly freed. Productive use of this facility normally involves redefining report_leak to do something more intelligent. This typically requires annotating objects with additional information (e.g. creation time stack trace) that identifies their origin. Such code is typically not very portable, and is not included here, except on SPARC machines. If all objects are allocated with GC_DEBUG_MALLOC (see next section), then the default version of report_leak will report the source file and line number at which the leaked object was allocated. This may sometimes be sufficient. (On SPARC/SUNOS4 machines, it will also report a cryptic stack trace. This can often be turned into a sympolic stack trace by invoking program "foo" with "callprocs foo". Callprocs is a short shell script that invokes adb to expand program counter values to symbolic addresses. It was largely supplied by Scott Schwartz.) Note that the debugging facilities described in the next section can sometimes be slightly LESS effective in leak finding mode, since in leak finding mode, GC_debug_free actually results in reuse of the object. (Otherwise the object is simply marked invalid.) Also note that the test program is not designed to run meaningfully in FIND_LEAK mode. Use "make gc.a" to build the collector. DEBUGGING FACILITIES: The routines GC_debug_malloc, GC_debug_malloc_atomic, GC_debug_realloc, and GC_debug_free provide an alternate interface to the collector, which provides some help with memory overwrite errors, and the like. Objects allocated in this way are annotated with additional information. Some of this information is checked during garbage collections, and detected inconsistencies are reported to stderr. Simple cases of writing past the end of an allocated object should be caught if the object is explicitly deallocated, or if the collector is invoked while the object is live. The first deallocation of an object will clear the debugging info associated with an object, so accidentally repeated calls to GC_debug_free will report the deallocation of an object without debugging information. Out of memory errors will be reported to stderr, in addition to returning NIL. GC_debug_malloc checking during garbage collection is enabled with the first call to GC_debug_malloc. This will result in some slowdown during collections. If frequent heap checks are desired, this can be achieved by explicitly invoking GC_gcollect, e.g. from the debugger. GC_debug_malloc allocated objects should not be passed to GC_realloc or GC_free, and conversely. It is however acceptable to allocate only some objects with GC_debug_malloc, and to use GC_malloc for other objects, provided the two pools are kept distinct. In this case, there is a very low probablility that GC_malloc allocated objects may be misidentified as having been overwritten. This should happen with probability at most one in 2**32. This probability is zero if GC_debug_malloc is never called. GC_debug_malloc, GC_malloc_atomic, and GC_debug_realloc take two additional trailing arguments, a string and an integer. These are not interpreted by the allocator. They are stored in the object (the string is not copied). If an error involving the object is detected, they are printed. The macros GC_MALLOC, GC_MALLOC_ATOMIC, GC_REALLOC, GC_FREE, and GC_REGISTER_FINALIZER are also provided. These require the same arguments as the corresponding (nondebugging) routines. If gc.h is included with GC_DEBUG defined, they call the debugging versions of these functions, passing the current file name and line number as the two extra arguments, where appropriate. If gc.h is included without GC_DEBUG defined, then all these macros will instead be defined to their nondebugging equivalents. (GC_REGISTER_FINALIZER is necessary, since pointers to objects with debugging information are really pointers to a displacement of 16 bytes form the object beginning, and some translation is necessary when finalization routines are invoked. For details, about what's stored in the header, see the definition of the type oh in debug_malloc.c) INCREMENTAL/GENERATIONAL COLLECTION: The collector normally interrupts client code for the duration of a garbage collection mark phase. This may be unacceptable if interactive response is needed for programs with large heaps. The collector can also run in a "generational" mode, in which it usually attempts to collect only objects allocated since the last garbage collection. Furthermore, in this mode, garbage collections run mostly incrementally, with a small amount of work performed in response to each of a large number of GC_malloc requests. This mode is enabled by a call to GC_enable_incremental(). Incremental and generational collection is effective in reducing pause times only if the collector has some way to tell which objects or pages have been recently modified. The collector uses two sources of information: 1. Information provided by the VM system. This may be provided in one of several forms. Under Solaris 2.X (and potentially under other similar systems) information on dirty pages can be read from the /proc file system. Under other systems (currently SunOS4.X) it is possible to write-protect the heap, and catch the resulting faults. On these systems we require that system calls writing to the heap (other than read) be handled specially by client code. See os_dep.c for details. 2. Information supplied by the programmer. We define "stubborn" objects to be objects that are rarely changed. Such an object can be allocated (and enabled for writing) with GC_malloc_stubborn. Once it has been initialized, the collector should be informed with a call to GC_end_stubborn_change. Subsequent writes that store pointers into the object must be preceded by a call to GC_change_stubborn. This mechanism performs best for objects that are written only for initialization, and such that only one stubborn object is writable at once. It is typically not worth using for short-lived objects. Stubborn objects are treated less efficiently than pointerfree (atomic) objects. A rough rule of thumb is that, in the absence of VM information, garbage collection pauses are proportional to the amount of pointerful storage plus the amount of modified "stubborn" storage that is reachable during the collection. Initial allocation of stubborn objects takes longer than allocation of other objects, since other data structures need to be maintained. We recommend against random use of stubborn objects in client code, since bugs caused by inappropriate writes to stubborn objects are likely to be very infrequently observed and hard to trace. However, their use may be appropriate in a few carefully written library routines that do not make the objects themselves available for writing by client code. BUGS: Any memory that does not have a recognizable pointer to it will be reclaimed. Exclusive-or'ing forward and backward links in a list doesn't cut it. Some C optimizers may lose the last undisguised pointer to a memory object as a consequence of clever optimizations. This has almost never been observed in practice. Send mail to boehm@parc.xerox.com for suggestions on how to fix your compiler. This is not a real-time collector. In the standard configuration, percentage of time required for collection should be constant across heap sizes. But collection pauses will increase for larger heaps. (On SPARCstation 2s collection times will be on the order of 300 msecs per MB of accessible memory that needs to be scanned. Your mileage may vary.) The incremental/generational collection facility helps, but is portable only if "stubborn" allocation is used. Please address bug reports to boehm@parc.xerox.com. If you are contemplating a major addition, you might also send mail to ask whether it's already been done (or whether we tried and discarded it). RECENT VERSIONS: Version 1.3 and immediately preceding versions contained spurious assembly language assignments to TMP_SP. Only the assignment in the PC/RT code is necessary. On other machines, with certain compiler options, the assignments can lead to an unsaved register being overwritten. Known to cause problems under SunOS 3.5 WITHOUT the -O option. (With -O the compiler recognizes it as dead code. It probably shouldn't, but that's another story.) Version 1.4 and earlier versions used compile time determined values for the stack base. This no longer works on Sun 3s, since Sun 3/80s use a different stack base. We now use a straightforward heuristic on all machines on which it is known to work (incl. Sun 3s) and compile-time determined values for the rest. There should really be library calls to determine such values. Version 1.5 and earlier did not ensure 8 byte alignment for objects allocated on a sparc based machine. Version 1.8 added ULTRIX support in gc_private.h. Version 1.9 fixed a major bug in gc_realloc. Version 2.0 introduced a consistent naming convention for collector routines and added support for registering dynamic library data segments in the standard mark_roots.c. Most of the data structures were revamped. The treatment of interior pointers was completely changed. Finalization was added. Support for locking was added. Object kinds were added. We added a black listing facility to avoid allocating at addresses known to occur as integers somewhere in the address space. Much of this was accomplished by adapting ideas and code from the PCR collector. The test program was changed and expanded. Version 2.1 was the first stable version since 1.9, and added support for PPCR. Version 2.2 added debugging allocation, and fixed various bugs. Among them: - GC_realloc could fail to extend the size of the object for certain large object sizes. - A blatant subscript range error in GC_printf, which unfortunately wasn't exercised on machines with sufficient stack alignment constraints. - GC_register_displacement did the wrong thing if it was called after any allocation had taken place. - The leak finding code would eventually break after 2048 byte byte objects leaked. - interface.c didn't compile. - The heap size remained much too small for large stacks. - The stack clearing code behaved badly for large stacks, and perhaps on HP/PA machines. Version 2.3 added ALL_INTERIOR_POINTERS and fixed the following bugs: - Missing declaration of etext in the A/UX version. - Some PCR root-finding problems. - Blacklisting was not 100% effective, because the plausible future heap bounds were being miscalculated. - GC_realloc didn't handle out-of-memory correctly. - GC_base could return a nonzero value for addresses inside free blocks. - test.c wasn't really thread safe, and could erroneously report failure in a multithreaded environment. (The locking primitives need to be replaced for other threads packages.) - GC_CONS was thoroughly broken. - On a SPARC with dynamic linking, signals stayed diabled while the client code was running. (Thanks to Manuel Serrano at INRIA for reporting the last two.) Version 2.4 added GC_free_space_divisor as a tuning knob, added support for OS/2 and linux, and fixed the following bugs: - On machines with unaligned pointers (e.g. Sun 3), every 128th word could fail to be considered for marking. - Dynamic_load.c erroneously added 4 bytes to the length of the data and bss sections of the dynamic library. This could result in a bad memory reference if the actual length was a multiple of a page. (Observed on Sun 3. Can probably also happen on a Sun 4.) (Thanks to Robert Brazile for pointing out that the Sun 3 version was broken. Dynamic library handling is still broken on Sun 3s under 4.1.1U1, but apparently not 4.1.1. If you have such a machine, use -Bstatic.) Version 2.5 fixed the following bugs: - Removed an explicit call to exit(1) - Fixed calls to GC_printf and GC_err_printf, so the correct number of arguments are always supplied. The OS/2 C compiler gets confused if the number of actuals and the number of formals differ. (ANSI C doesn't require this to work. The ANSI sanctioned way of doing things causes too many compatibility problems.) Version 3.0 added generational/incremental collection and stubborn objects. Version 3.1 added the following features: - A workaround for a SunOS 4.X SPARC C compiler misfeature that caused problems when the collector was turned into a dynamic library. - A fix for a bug in GC_base that could result in a memory fault. - A fix for a performance bug (and several other misfeatures) pointed out by Dave Detlefs and Al Dosser. - Use of dirty bit information for static data under Solaris 2.X. - DEC Alpha/OSF1 support (thanks to Al Dosser). - Incremental collection on more platforms. - A more refined heap expansion policy. Less space usage by default. - Various minor enhancements to reduce space usage, and to reduce the amount of memory scanned by the collector. - Uncollectable allocation without per object overhead. - More conscientious handling of out-of-memory conditions. - Fixed a bug in debugging stubborn allocation. - Fixed a bug that resulted in occasional erroneous reporting of smashed objects with debugging allocation. - Fixed bogus leak reports of size 4096 blocks with FIND_LEAK. Version 3.2 fixed a serious and not entirely repeatable bug in the incremental collector. It appeared only when dirty bit info on the roots was available, which is normally only under Solaris. It also added GC_general_register_disappearing_link, and some testing code. Interface.c disappeared. Version 3.3 fixes several bugs and adds new ports: - PCR-specific bugs. - Missing locking in GC_free, redundant FASTUNLOCK in GC_malloc_stubborn, and 2 bugs in GC_unregister_disappearing_link. All of the above were pointed out by Neil Sharman (neil@cs.mu.oz.au). - Common symbols allocated by the SunOS4.X dynamic loader were not included in the root set. - Bug in GC_finalize (reported by Brian Beuning and Al Dosser) - Merged Amiga port from Jesper Peterson (untested) - Merged NeXT port from Thomas Funke (significantly modified and untested) Version 3.4: - Fixed a performance bug in GC_realloc. - Updated the amiga port. - Added NetBSD and 386BSD ports. - Added cord library. - Added trivial performance enhancement for ALL_INTERIOR_POINTERS. (Don't scan last word.) Version 3.5 - Minor collections now mark from roots only once, if that doesn't cause an excessive pause. - The stack clearing heuristic was refined to prevent anomalies with very heavily recursive programs and sparse stacks. - Fixed a bug that prevented mark stack growth in some cases. GC_objects_are_marked should be set to TRUE after a call to GC_push_roots and as part of GC_push_marked, since both can now set mark bits. I think this is only a performance bug, but I wouldn't bet on it. It's certainly very hard to argue that the old version was correct. - Fixed an incremental collection bug that prevented it from working at all when HBLKSIZE != getpagesize() - Changed dynamic_loading.c to include gc_priv.h before testing DYNAMIC_LOADING. SunOS dynamic library scanning must have been broken in 3.4. - Object size rounding now adapts to program behavior. - Added a workaround (provided by Manuel Serrano and colleagues) to a long-standing SunOS 4.X (and 3.X?) ld bug that I had incorrectly assumed to have been squished. The collector was broken if the text segment size was within 32 bytes of a multiple of 8K bytes, and if the beginning of the data segment contained interesting roots. The workaround assumes a demand-loadable executable. The original may have have "worked" in some other cases. - Added dynamic library support under IRIX5. - Added support for EMX under OS/2 (thanks to Ari Huttunen). Version 3.6: - fixed a bug in the mark stack growth code that was introduced in 3.4. - fixed Makefile to work around DEC AXP compiler tail recursion bug. Version 3.7: - Added a workaround for an HP/UX compiler bug. - Fixed another stack clearing performance bug. Reworked that code once more. Version 4.0: - Added support for Solaris threads (which was possible only by reimplementing some fraction of Solaris threads, since Sun doesn't currently make the thread debugging interface available). - Added non-threads win32 and win32S support. - (Grudgingly, with suitable muttering of obscenities) renamed files so that the collector distribution could live on a FAT file system. Files that are guaranteed to be useless on a PC still have long names. Gc_inline.h and gc_private.h still exist, but now just include gc_inl.h and gc_priv.h. - Fixed a really obscure bug in finalization that could cause undetected mark stack overflows. (I would be surprised if any real code ever tickled this one.) - Changed finalization code to dynamically resize the hash tables it maintains. (This probably does not matter for well- -written code. It no doubt does for C++ code that overuses destructors.) - Added typed allocation primitives. Rewrote the marker to accommodate them with more reasonable efficiency. This change should also speed up marking for GC_malloc allocated objects a little. See gc_typed.h for new primitives. - Improved debugging facilities slightly. Allocation time stack traces are now kept by default on SPARC/SUNOS4. (Thanks to Scott Schwartz.) - Added better support for small heap applications. - Significantly extended cord package. Fixed a bug in the implementation of lazily read files. Printf and friends now have cord variants. Cord traversals are a bit faster. - Made ALL_INTERIOR_POINTERS recognition the default. - Fixed de so that it can run in constant space, independent of file size. Added simple string searching to cords and de. - Added the Hull-Ellis C++ interface. - Added dynamic library support for OSF/1. (Thanks to Al Dosser and Tim Bingham at DEC.) - Changed argument to GC_expand_hp to be expressed in units of bytes instead of heap blocks. (Necessary since the heap block size now varies depending on configuration. The old version was never very clean.) - Added GC_get_heap_size(). The previous "equivalent" was broken. - Restructured the Makefile a bit. Since version 4.0: - Changed finalization implementation to guarantee that finalization procedures are called outside of the allocation lock, making direct use of the interface a little less dangerous. MAY BREAK EXISTING CLIENTS that assume finalizers are protected by a lock. Since there seem to be few multithreaded clients that use finalization, this is hopefully not much of a problem. - Fixed a gross bug in CORD_prev. - Fixed a bug in blacklst.c that could result in unbounded heap growth during startup on machines that do not clear memory obtained from the OS (e.g. win32S). - Ported de editor to win32/win32S. (This is now the only version with a mouse-sensitive UI.) - Added GC_malloc_ignore_off_page to allocate large arrays in the presence of ALL_INTERIOR_POINTERS. - Changed GC_call_with_alloc_lock to not disable signals in the single-threaded case. - Reduced retry count in GC_collect_or_expand for garbage collecting when out of memory. - Made uncollectable allocations bypass black-listing, as they should. - Fixed a bug in typed_test in test.c that could cause (legitimate) GC crashes. - Fixed some potential synchronization problems in finalize.c - Fixed a real locking problem in typd_mlc.c. - Worked around an AIX 3.2 compiler feature that results in out of bounds memory references. - Partially worked around an IRIX5.2 beta problem (which may or may not persist to the final release). - Fixed a bug in the heap integrity checking code that could result in explicitly deallocated objects being identified as smashed. Fixed a bug in the dbg_mlc stack saving code that caused old argument pointers to be considered live. - Fixed a bug in CORD_ncmp (and hence CORD_str). - Repaired the OS2 port, which had suffered from bit rot in 4.0. Worked around what appears to be CSet/2 V1.0 optimizer bug. - Fixed a Makefile bug for target "c++". Since version 4.1: - Multiple bug fixes/workarounds in the Solaris threads version. (It occasionally failed to locate some register contents for marking. It also turns out that thr_suspend and friends are unreliable in Solaris 2.3. Dirty bit reads appear to be unreliable under some weird circumstances. My stack marking code contained a serious performance bug. The new code is extremely defensive, and has not failed in several cpu hours of testing. But no guarantees ...) - Added MacOS support (thanks to Patrick Beard.) - Fixed several syntactic bugs in gc_c++.h and friends. (These didn't bother g++, but did bother most other compilers.) Fixed gc_c++.h finalization interface. (It didn't.) - 64 bit alignment for allocated objects was not guaranteed in a few cases in which it should have been. - Added GC_malloc_atomic_ignore_off_page. - Added GC_collect_a_little. - Added some prototypes to gc.h. - Some other minor bug fixes (notably in Makefile). - Fixed OS/2 / EMX port (thanks to Ari Huttunen). - Fixed AmigaDOS port. (thanks to Michel Schinz). - Fixed the DATASTART definition under Solaris. There was a 1 in 16K chance of the collector missing the first 64K of static data (and thus crashing). - Fixed some blatant anachronisms in the README file. - Fixed PCR-Makefile for upcoming PPCR release. Since version 4.2: - Fixed SPARC alignment problem with GC_DEBUG. - Fixed Solaris threads /proc workaround. The real problem was an interaction with mprotect. - Incorporated fix from Patrick Beard for gc_c++.h (now gc_cpp.h). - Slightly improved allocator space utilization by fixing the GC_size_map mechanism. - Integrated some Sony News and MIPS RISCos 4.51 patches. (Thanks to Nobuyuki Hikichi of Software Research Associates, Inc. Japan) - Fixed HP_PA alignment problem. (Thanks to xjam@cork.cs.berkeley.edu.) - Added GC_same_obj and friends. Changed GC_base to return 0 for pointers past the end of large objects. Improved GC_base performance with ALL_INTERIOR_POINTERS on machines with a slow integer mod operation. Added GC_PTR_ADD, GC_PTR_STORE, etc. to prepare for preprocessor. - changed the default on most UNIX machines to be that signals are not disabled during critical GC operations. This is still ANSI-conforming, though somewhat dangerous in the presence of signal handlers. But the performance cost of the alternative is sometimes problematic. Can be changed back with a minor Makefile edit. - renamed IS_STRING in gc.h, to CORD_IS_STRING, thus following my own naming convention. Added the function CORD_to_const_char_star. - Fixed a gross bug in GC_finalize. Symptom: occasional address faults in that function. (Thanks to Anselm Baird-Smith (Anselm.BairdSmith@inria.fr) - Added port to ICL DRS6000 running DRS/NX. Restructured things a bit to factor out common code, and remove obsolete code. Collector should now run under SUNOS5 with either mprotect or /proc dirty bits. (Thanks to Douglas Steel (doug@wg.icl.co.uk)). - More bug fixes and workarounds for Solaris 2.X. (These were mostly related to putting the collector in a dynamic library, which didn't really work before. Also SOLARIS_THREADS didn't interact well with dl_open.) Thanks to btlewis@eng.sun.com. - Fixed a serious performance bug on the DEC Alpha. The text segment was getting registered as part of the root set. (Amazingly, the result was still fast enough that the bug was not conspicuous.) The fix works on OSF/1, version 1.3. Hopefully it also works on other versions of OSF/1 ... - Fixed a bug in GC_clear_roots. - Fixed a bug in GC_generic_malloc_words_small that broke gc_inl.h. (Reported by Antoine de Maricourt. I broke it in trying to tweak the Mac port.) - Fixed some problems with cord/de under Linux. - Fixed some cord problems, notably with CORD_riter4. - Added DG/UX port. Thanks to Ben A. Mesander (ben@piglet.cr.usgs.gov) - Added finalization registration routines with weaker ordering constraints. (This is necessary for C++ finalization with multiple inheritance, since the compiler often adds self-cycles.) - Filled the holes in the SCO port. (Thanks to Michael Arnoldus .) - John Ellis' additions to the C++ support: From John: * I completely rewrote the documentation in the interface gc_c++.h (later renamed gc_cpp.h). I've tried to make it both clearer and more precise. * The definition of accessibility now ignores pointers from an finalizable object (an object with a clean-up function) to itself. This allows objects with virtual base classes to be finalizable by the collector. Compilers typically implement virtual base classes using pointers from an object to itself, which under the old definition of accessibility prevented objects with virtual base classes from ever being collected or finalized. * gc_cleanup now includes gc as a virtual base. This was enabled by the change in the definition of accessibility. * I added support for operator new[]. Since most (all?) compilers don't yet support operator new[], it is conditionalized on -DOPERATOR_NEW_ARRAY. The code is untested, but its trivial and looks correct. * The test program test_gc_c++ (later renamed test_cpp.cc) tries to test for the C++-specific functionality not tested by the other programs. - Added include to misc.c. (Needed for ppcr.) - Added PowerMac port. (Thanks to Patrick Beard again.) - Fixed "srcdir"-related Makefile problems. Changed things so that all externally visible include files always appear in the include subdirectory of the source. Made gc.h directly includable from C++ code. (These were at Per Bothner's suggestion.) - Changed Intel code to also mark from ebp (Kevin Warne's suggestion). - Renamed C++ related files so they could live in a FAT file system. (Charles Fiterman's suggestion.) - Changed Windows NT Makefile to include C++ support in gc.lib. Added C++ test as Makefile target. Since version 4.3: - ASM_CLEAR_CODE was erroneously defined for HP PA machines, resulting in a compile error. - Fixed OS/2 Makefile to create a library. (Thanks to Mark Boulter (mboulter@vnet.ibm.com)). - Gc_cleanup objects didn't work if they were created on the stack. Fixed. - One copy of Gc_cpp.h in the distribution was out of synch, and failed to document some known compiler problems with explicit destructor invocation. Partially fixed. There are probably other compilers on which gc_cleanup is miscompiled. - Fixed Makefile to pass C compiler flags to C++ compiler. - Added Mac fixes. - Fixed os_dep.c to work around what appears to be a new and different VirtualQuery bug under newer versions of win32S. - GC_non_gc_bytes was not correctly maintained by GC_free. Fixed. Thanks to James Clark (jjc@jclark.com). - Added GC_set_max_heap_size. - Changed allocation code to ignore blacklisting if it is preventing use of a very large block of memory. This has the advantage that naive code allocating very large objects is much more likely to work. The downside is you might no longer find out that such code should really use GC_malloc_ignore_off_page. - Changed GC_printf under win32 to close and reopen the file between calls. FAT file systems otherwise make the log file useless for debugging. - Added GC_try_to_collect and GC_get_bytes_since_gc. These allow starting an abortable collection during idle times. This facility does not require special OS support. (Thanks to Michael Spertus of Geodesic Systems for suggesting this. It was actually an easy addition. Kumar Srikantan previously added a similar facility to a now ancient version of the collector. At the time this was much harder, and the result was less convincing.) - Added some support for the Borland development environment. (Thanks to John Ellis and Michael Spertus.) - Removed a misfeature from checksums.c that caused unexpected heap growth. (Thanks to Scott Schwartz.) - Changed finalize.c to call WARN if it encounters a finalization cycle. WARN is defined in gc_priv.h to write a message, usually to stdout. In many environments, this may be inappropriate. - Renamed NO_PARAMS in gc.h to GC_NO_PARAMS, thus adhering to my own naming convention. - Added GC_set_warn_proc to intercept warnings. - Fixed Amiga port. (Thanks to Michel Schinz (schinz@alphanet.ch).) - Fixed a bug in mark.c that could result in an access to unmapped memory from GC_mark_from_mark_stack on machines with unaligned pointers. - Fixed a win32 specific performance bug that could result in scanning of objects allocated with the system malloc. - Added REDIRECT_MALLOC. Since version 4.4: - Fixed many minor and one major README bugs. (Thanks to Franklin Chen (chen@adi.com) for pointing out many of them.) - Fixed ALPHA/OSF/1 dynamic library support. (Thanks to Jonathan Bachrach (jonathan@harlequin.com)). - Added incremental GC support (MPROTECT_VDB) for Linux (with some help from Bruno Haible). - Altered SPARC recognition tests in gc.h and config.h (mostly as suggested by Fergus Henderson). - Added basic incremental GC support for win32, as implemented by Windows NT and Windows 95. GC_enable_incremental is a noop under win32s, which doesn't implement enough of the VM interface. - Added -DLARGE_CONFIG. - Fixed GC_..._ignore_off_page to also function without -DALL_INTERIOR_POINTERS. - (Hopefully) fixed RS/6000 port. (Only the test was broken.) - Fixed a performance bug in the nonincremental collector running on machines supporting incremental collection with MPROTECT_VDB (e.g. SunOS 4, DEC AXP). This turned into a correctness bug under win32s with win32 incremental collection. (Not all memory protection was disabled.) - Fixed some ppcr related bit rot. - Caused dynamic libraries to be unregistered before reregistering. The old way turned out to be a performance bug on some machines. - GC_root_size was not properly maintained under MSWIN32. - Added -DNO_DEBUGGING and GC_dump. - Fixed a couple of bugs arising with SOLARIS_THREADS + REDIRECT_MALLOC. - Added NetBSD/M68K port. (Thanks to Peter Seebach .) - Fixed a serious realloc bug. For certain object sizes, the collector wouldn't scan the expanded part of the object. (Thanks to Clay Spence (cds@peanut.sarnoff.com) for noticing the problem, and helping me to track it down.) Since version 4.5: - Added Linux ELF support. (Thanks to Arrigo Triulzi .) - GC_base crashed if it was called before any other GC_ routines. This could happen if a gc_cleanup object was allocated outside the heap before any heap allocation. - The heap expansion heuristic was not stable if all objects had finalization enabled. Fixed finalize.c to count memory in finalization queue and avoid explicit deallocation. Changed alloc.c to also consider this count. (This is still not recommended. It's expensive if nothing else.) Thanks to John Ellis for pointing this out. - GC_malloc_uncollectable(0) was broken. Thanks to Phong Vo for pointing this out. - The collector didn't compile under Linux 1.3.X. (Thanks to Fred Gilham for pointing this out.) The current workaround is ugly, but expected to be temporary. - Fixed a formatting problem for SPARC stack traces. - Fixed some '=='s in os_dep.c that should have been assignments. Fortunately these were in code that should never be executed anyway. (Thanks to Fergus Henderson.) - Fixed the heap block allocator to only drop blacklisted blocks in small chunks. Made BL_LIMIT self adjusting. (Both of these were in response to heap growth observed by Paul Graham.) - Fixed the Metrowerks/68K Mac code to also mark from a6. (Thanks to Patrick Beard.) - Significantly updated README.debugging. - Fixed some problems with longjmps out of signal handlers, especially under Solaris. Added a workaround for the fact that siglongjmp doesn't appear to do the right thing with -lthread under Solaris. - Added MSDOS/djgpp port. (Thanks to Mitch Harris (maharri@uiuc.edu).) - Added "make reserved_namespace" and "make user_namespace". The first renames ALL "GC_xxx" identifiers as "_GC_xxx". The second is the inverse transformation. Note that doing this is guaranteed to break all clients written for the other names. - descriptor field for kind NORMAL in GC_obj_kinds with ADD_BYTE_AT_END defined should be -ALIGNMENT not WORDS_TO_BYTES(-1). This is a serious bug on machines with pointer alignment of less than a word. - GC_ignore_self_finalize_mark_proc didn't handle pointers to very near the end of the object correctly. Caused failures of the C++ test on a DEC Alpha with g++. - gc_inl.h still had problems. Partially fixed. Added warnings at the beginning to hopefully specify the remaining dangers. - Added DATAEND definition to config.h. - Fixed some of the .h file organization. Fixed "make floppy". Since version 4.6: - Fixed some compilation problems with -DCHECKSUMS (thanks to Ian Searle) - Updated some Mac specific files to synchronize with Patrick Beard. - Fixed a serious bug for machines with non-word-aligned pointers. (Thanks to Patrick Beard for pointing out the problem. The collector should fail almost any conceivable test immediately on such machines.) Since version 4.7: - Changed a "comment" in a MacOS specific part of mach-dep.c that caused gcc to fail on other platforms. Since version 4.8 - More README.debugging fixes. - Objects ready for finalization, but not finalized in the same GC cycle, could be prematurely collected. This occasionally happened in test_cpp. - Too little memory was obtained from the system for very large objects. That could cause a heap explosion if these objects were not contiguous (e.g. under PCR), and too much of them was blacklisted. - Due to an improper initialization, the collector was too hesitant to allocate blacklisted objects immediately after system startup. - Moved GC_arrays from the data into the bss segment by not explicitly initializing it to zero. This significantly reduces the size of executables, and probably avoids some disk accesses on program startup. It's conceivable that it might break a port that I didn't test. - Fixed EMX_MAKEFILE to reflect the gc_c++.h to gc_cpp.h renaming which occurred a while ago. Since 4.9: - Fixed a typo around a call to GC_collect_or_expand in alloc.c. It broke handling of out of memory. (Thanks to Patrick Beard for noticing.) --------------------------------------------------------------------------- From tenser@spitfire.ecsel.psu.edu Tue Feb 18 23:54:37 1997 Return-Path: Received: from spitfire.ecsel.psu.edu (qmailr@spitfire.ecsel.psu.edu [146.186.218.51]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id XAA10866 for ; Tue, 18 Feb 1997 23:54:36 -0800 (PST) Received: (qmail 15034 invoked by uid 1000); 19 Feb 1997 07:54:31 -0000 Message-ID: <19970219075431.15033.qmail@spitfire.ecsel.psu.edu> To: Matt Dillon cc: Phillip Musumeci , adrian@cougar.aceonline.com.au, auditors@freebsd.org Subject: Re: Re : Bounds-checking gcc .. In-reply-to: Your message of "Tue, 18 Feb 1997 13:47:06 PST." <199702182147.NAA01438@flea.best.net> Date: Wed, 19 Feb 1997 02:54:30 -0500 From: Dan Cross > Well.. no duh! But if you insist on a perfect solution, the likely > outcome will be NO solution at all. Complete solutions are possible > only if you do something, say, like redesign the whole damn operating > system from scratch. Having a partial solution is better then nothing, > especially if it's scaleable (i.e. you start tackling the various > system/library resources one at a time). Oh, I disagree. A partial solution is *worse* than a complete or no solution approach. One of the most often cited and most valid complaints against firewalls is that they buy you a false sense of security. Security should be end-to-end, if it's not, then you're doing something wrong. What's the point of all of this, anyway? To simplify the security aspects of setuid programs? That would be much better accomplished by provided a standard set of routines in a library (libsetuid, anyone?) for doing things like scrubbing the environment, safely reading in, copying, and token-izing strings, etc, and then making it a convention to call those routines in setuid programs. As for redesigning the operating system, well, isn't that what the AT&T guys did with Plan 9 and Brazil? ``Not only is UNIX dead, but it's starting to smell bad...'' <-- Rob Pike. > This is getting off topic.. I am not suggesting that we do any of this > now, but if you want to keep the system 'clean' in future releases, > a 'default more secure' rather then a 'default less secure' paradigm > is an absolute necessity. We will never be able to guarentee the > system against bozo programmers, but we can do a whole lot better > then what we have now. For this one, I suggest we take the ``no solution'' approach, and then work towards a standard set of library routines which are useful for handling common tasks inside setuid programs. Oh, btw. Two or 20 messages back, someone brought up the issue of adding ``non-standard'' functions to the libraries. I wanted to address that then, but I got really swamped here (which is why the secure audit has fallen behind my own personal schedule. Mark, have you gotten anything cool yet with the international stuff? :-) That individual (I'm sorry, I can't remember who it was, and I want to get out of here and go home, so I'm not going to look. My apologies) brought up some really good points. But I think that it's important to remember that the functions which appear in the standard *now* do so because they were in common use somewhere before the standard came to be. My point? Well, if we add some of these needed-but-not- there-and-not-standard functions NOW, they stand a pretty good chance of making it into the standard later. If we're doing things correctly (which is my big gripe when people declare main as void main(void). Argh. It's incorrect.), then we have a very good basis for adding new functionality to the system, that will more than likely be incorp- orated into the various standards later. What do folks think about this? I'm really interested, especially on this point. - Dan C. From tenser@spitfire.ecsel.psu.edu Wed Feb 19 00:12:16 1997 Return-Path: Received: from spitfire.ecsel.psu.edu (qmailr@spitfire.ecsel.psu.edu [146.186.218.51]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id AAA11736 for ; Wed, 19 Feb 1997 00:12:15 -0800 (PST) Received: (qmail 15034 invoked by uid 1000); 19 Feb 1997 07:54:31 -0000 Message-ID: <19970219075431.15033.qmail@spitfire.ecsel.psu.edu> To: Matt Dillon cc: Phillip Musumeci , adrian@cougar.aceonline.com.au, auditors@freebsd.org Subject: Re: Re : Bounds-checking gcc .. In-reply-to: Your message of "Tue, 18 Feb 1997 13:47:06 PST." <199702182147.NAA01438@flea.best.net> Date: Wed, 19 Feb 1997 02:54:30 -0500 From: Dan Cross > Well.. no duh! But if you insist on a perfect solution, the likely > outcome will be NO solution at all. Complete solutions are possible > only if you do something, say, like redesign the whole damn operating > system from scratch. Having a partial solution is better then nothing, > especially if it's scaleable (i.e. you start tackling the various > system/library resources one at a time). Oh, I disagree. A partial solution is *worse* than a complete or no solution approach. One of the most often cited and most valid complaints against firewalls is that they buy you a false sense of security. Security should be end-to-end, if it's not, then you're doing something wrong. What's the point of all of this, anyway? To simplify the security aspects of setuid programs? That would be much better accomplished by provided a standard set of routines in a library (libsetuid, anyone?) for doing things like scrubbing the environment, safely reading in, copying, and token-izing strings, etc, and then making it a convention to call those routines in setuid programs. As for redesigning the operating system, well, isn't that what the AT&T guys did with Plan 9 and Brazil? ``Not only is UNIX dead, but it's starting to smell bad...'' <-- Rob Pike. > This is getting off topic.. I am not suggesting that we do any of this > now, but if you want to keep the system 'clean' in future releases, > a 'default more secure' rather then a 'default less secure' paradigm > is an absolute necessity. We will never be able to guarentee the > system against bozo programmers, but we can do a whole lot better > then what we have now. For this one, I suggest we take the ``no solution'' approach, and then work towards a standard set of library routines which are useful for handling common tasks inside setuid programs. Oh, btw. Two or 20 messages back, someone brought up the issue of adding ``non-standard'' functions to the libraries. I wanted to address that then, but I got really swamped here (which is why the secure audit has fallen behind my own personal schedule. Mark, have you gotten anything cool yet with the international stuff? :-) That individual (I'm sorry, I can't remember who it was, and I want to get out of here and go home, so I'm not going to look. My apologies) brought up some really good points. But I think that it's important to remember that the functions which appear in the standard *now* do so because they were in common use somewhere before the standard came to be. My point? Well, if we add some of these needed-but-not- there-and-not-standard functions NOW, they stand a pretty good chance of making it into the standard later. If we're doing things correctly (which is my big gripe when people declare main as void main(void). Argh. It's incorrect.), then we have a very good basis for adding new functionality to the system, that will more than likely be incorp- orated into the various standards later. What do folks think about this? I'm really interested, especially on this point. - Dan C. From giles@nemeton.com.au Wed Feb 19 02:14:18 1997 Return-Path: Received: from perki0.connect.com.au (perki0.connect.com.au [192.189.54.85]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id CAA16231 for ; Wed, 19 Feb 1997 02:14:17 -0800 (PST) Received: from nemeton.UUCP (Unemeton@localhost) by perki0.connect.com.au with UUCP id VAA13281 (8.7.6h/IDA-1.6); Wed, 19 Feb 1997 21:14:08 +1100 (EST) X-Authentication-Warning: perki0.connect.com.au: Unemeton set sender to giles@nemeton.com.au using -f Received: from localhost.nemeton.com.au (localhost.nemeton.com.au [127.0.0.1]) by nemeton.com.au (8.8.5/8.8.5) with SMTP id VAA19165; Wed, 19 Feb 1997 21:13:19 +1100 (EST) Message-Id: <199702191013.VAA19165@nemeton.com.au> To: tqbf@enteract.com cc: auditors@freebsd.org Subject: Re: Security problem in FreeBSD /sbin/init In-reply-to: <199702190134.TAA12057@enteract.com> Date: Wed, 19 Feb 1997 21:13:18 +1100 From: Giles Lean On Tue, 18 Feb 1997 19:34:11 -0600 (CST) "Thomas H. Ptacek" wrote: > This problem will probably be picked up by the sweeping audit of your code > base, but I figured I'd alert you to it anyways. All help appreciated! One less to find the hard way. (Jordan -- please mark me down for 'init', since I've hacked on it before and know the structure.) Thanks, Giles From xaa@alterego.stack.nl Wed Feb 19 05:03:00 1997 Return-Path: Received: from alterego.stack.nl (xaa@alterego.stack.nl [131.155.141.160]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id FAA21399 for ; Wed, 19 Feb 1997 05:02:04 -0800 (PST) Received: (from xaa@localhost) by alterego.stack.nl (8.8.5/8.8.5) id OAA05126; Wed, 19 Feb 1997 14:01:29 +0100 (MET) Message-ID: Date: Wed, 19 Feb 1997 14:01:28 +0100 From: xaa@stack.nl (Mark Huizer) To: auditors@freebsd.org Subject: Termcap use... X-Mailer: Mutt 0.58.1 Mime-Version: 1.0 Hmm... What is the neat way to handle a termcap lookup?? the man page says to reserve a buffer of 1024 places and use tgetent, ok, perhaps a const in termcap would be nicer? (MAXTERMCAPENTRY or something?)... and then... some programs seem to put it in an array with tgetstr... any clue how big that buffer should be? 2048 since any code can result in 2 char string? Anything else??? Mark ------------------------------------------------------------------------- - Mark Huizer - xaa@stack.nl - rcbamh@urc.tue.nl - ------------------------------------------------------------------------- - My one regret in life is that I am not someone else (W. Allen) - ------------------------------------------------------------------------- From imp@village.org Wed Feb 19 07:10:08 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id HAA27045 for ; Wed, 19 Feb 1997 07:10:06 -0800 (PST) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 0.56 #1) id E0vxDeg-0005yW-00; Wed, 19 Feb 1997 08:09:58 -0700 To: xaa@stack.nl (Mark Huizer) Subject: Re: Termcap use... Cc: auditors@freebsd.org In-reply-to: Your message of "Wed, 19 Feb 1997 14:01:28 +0100." References: Date: Wed, 19 Feb 1997 08:09:57 -0700 From: Warner Losh Message-Id: In message Mark Huizer writes: : What is the neat way to handle a termcap lookup?? : : the man page says to reserve a buffer of 1024 places and use tgetent, : ok, perhaps a const in termcap would be nicer? (MAXTERMCAPENTRY or : something?)... : : and then... some programs seem to put it in an array with tgetstr... any : clue how big that buffer should be? 2048 since any code can result in 2 : char string? Anything else??? Hmmm. Not sure. Someone recently told me that telnetd might have a hole in it in this area. Something about "What happens when the TERM (or was that TERMCAP) env variable is > 1024 characters?". Might be a good thing for someone to thread through. Warner From eivind@dimaga.com Wed Feb 19 13:52:31 1997 Return-Path: Received: from nic.follonett.no (nic.follonett.no [194.198.43.10]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id NAA21358 for ; Wed, 19 Feb 1997 13:52:28 -0800 (PST) Received: (from uucp@localhost) by nic.follonett.no (8.8.5/8.8.3) with UUCP id WAA03520; Wed, 19 Feb 1997 22:49:45 +0100 (MET) Received: from oo7 (oo7.dimaga.com [192.0.0.65]) by dimaga.com (8.8.5/8.7.2) with SMTP id WAA15282; Wed, 19 Feb 1997 22:15:04 +0100 (MET) Message-Id: <3.0.32.19970219221503.00bd7860@dimaga.com> X-Sender: eivind@dimaga.com X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Wed, 19 Feb 1997 22:15:06 +0100 To: Warner Losh From: Eivind Eklund Subject: Re: Re : Bounds-checking gcc .. Cc: Matt Dillon , Phillip Musumeci , adrian@cougar.aceonline.com.au, auditors@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 02:39 PM 2/18/97 -0700, Warner Losh wrote: >: The startup code could also check the resource set if the program >: is run suid and exit if the resources are not considered >: 'reasonable'. > >That's a very interesting idea. However, I worry about the cost >associated with this sanitation for every program on the system. That >is, I worry about the added codebloat to crt0.o which sounds like >where you'd place this functionality. If you can come up with (or >someone can) a good way to firewall setuid programs that doesn't cost >those programs that aren't setuid much (in terms of size or speed), >then you may have a winner. I've been thinking along the lines of having a different, especially secure version of libc.so for setuid programs. This should a LITTLE to the startup/dynamic library code. An easy way to do this could be to have the linker look first for libxxx.suid.N.N instead of libxxx.so.N.N for any setuid program; this allow an alternate version of any library we care to replace. The generation of alternate libraries can be handled by #ifdef's or similar. Eivind Eklund perhaps@yes.no http://maybe.yes.no/perhaps/ eivind@freebsd.org From dzerkel@phofarm.com Wed Feb 19 19:04:03 1997 Return-Path: Received: from feephi.phofarm.com ([206.21.77.129]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id TAA09606 for ; Wed, 19 Feb 1997 19:03:58 -0800 (PST) Received: from feephi.phofarm.com (feephi.phofarm.com [206.21.77.130]) by feephi.phofarm.com (8.8.5/8.7.3) with SMTP id WAA00544 for ; Wed, 19 Feb 1997 22:01:36 -0500 (EST) Sender: dzerkel@feephi.phofarm.com Message-ID: <330BBE8F.41C67EA6@phofarm.com> Date: Wed, 19 Feb 1997 22:01:35 -0500 From: "Danny J. Zerkel" Organization: Photon Farmers X-Mailer: Mozilla 3.01 (X11; I; FreeBSD 3.0-CURRENT i386) MIME-Version: 1.0 To: auditors@freebsd.org Subject: Any suggestions? Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit I've got current src. Any suggestions for things that need looking at? Maybe something in lib? Thanks, Danny J. Zerkel Photon Farmers dzerkel@phofarm.com. From jkh@time.cdrom.com Wed Feb 19 23:09:26 1997 Return-Path: Received: from time.cdrom.com (root@time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id XAA24056 for ; Wed, 19 Feb 1997 23:09:19 -0800 (PST) Received: from time.cdrom.com (jkh@localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id XAA11013; Wed, 19 Feb 1997 23:06:42 -0800 (PST) To: "Danny J. Zerkel" cc: auditors@freebsd.org Subject: Re: Any suggestions? In-reply-to: Your message of "Wed, 19 Feb 1997 22:01:35 EST." <330BBE8F.41C67EA6@phofarm.com> Date: Wed, 19 Feb 1997 23:06:42 -0800 Message-ID: <11009.856422402@time.cdrom.com> From: "Jordan K. Hubbard" > I've got current src. Any suggestions for things that need looking at? > Maybe something in lib? I don't think anyone is currently looking at libutil or libm, and libc can always use a pass! :-) I'm sorry I haven't been coordinating very aggressively here, folks, but 2.1.7 sort of side-lined me for 3 days there. I think that's done and we're now ready to start in earnest. :-) It would help if each category had a "captain" though since even if I were a 24 hour manager here, I'd still be confused about who's doing what since there are too many people and categories. Guido seems to be a likely captain for most categories, given that he's signed up to review them all, until more can be found, I think. :-) Jordan From adrian@cougar.aceonline.com.au Thu Feb 20 02:11:49 1997 Return-Path: Received: from cougar.aceonline.com.au (adrian@cougar.aceonline.com.au [203.103.81.36]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id CAA03653; Thu, 20 Feb 1997 02:11:47 -0800 (PST) Received: from localhost (adrian@localhost) by cougar.aceonline.com.au (8.8.4/8.7) with SMTP id SAA16485; Thu, 20 Feb 1997 18:11:58 +0800 Date: Thu, 20 Feb 1997 18:11:57 +0800 (WST) From: Adrian Chadd To: audit-telnetd@freebsd.org cc: auditors@freebsd.org Subject: Patch for utility.c in telnetd.. Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Here is a patch for a couple of blantantly obvious (to me) possible buffer overflow problems. From what I could see from the source, putstr() isn't used for anything bigger than BUFSIZ, so thats my assumption. If anyone thinks I should, I can change it to accept a maxlength parameter. Try it out, tell me if ity works or not, and if its any use at all. Cya. -- Adrian Chadd | Windows 95 - the XT emulator for | your 486 and above! | Being superstitious is bad luck. diff - utility.c 396a397 > int index = 0; 398c399,403 < while (*s) --- > while (*s) { > index++; > if (index > BUFSIZ) { > err(1,"putstr(): overflow in string length.\n"); > } 399a405 > } 427c433,435 < char db[100]; --- > char db[BUFSIZ]; > int index; > 441a450 > index = 0; 444,445c453,463 < putchr(*cp++); < continue; --- > if (index > BUFSIZ) { > err(1, "putf() : overflow in while (*cp)\n"); > } else { > putchr(*cp++); > index++; > continue; > } > } > index++; > if (index > BUFSIZ) { > err(1, "putf() : overflow in switch (*+cpp)\n"); From adrian@cougar.aceonline.com.au Thu Feb 20 02:40:50 1997 Return-Path: Received: from cougar.aceonline.com.au (adrian@cougar.aceonline.com.au [203.103.81.36]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id CAA05053; Thu, 20 Feb 1997 02:40:47 -0800 (PST) Received: from localhost (adrian@localhost) by cougar.aceonline.com.au (8.8.4/8.7) with SMTP id SAA18397; Thu, 20 Feb 1997 18:40:58 +0800 Date: Thu, 20 Feb 1997 18:40:58 +0800 (WST) From: Adrian Chadd To: audit-telnetd@freebsd.org cc: auditors@freebsd.org Subject: Re : util.c patch whoops.. Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sorry guys - here is the patch again using diff -c. -- Adrian Chadd | Windows 95 - the XT emulator for | your 486 and above! | Being superstitious is bad luck. *** utility.c.orig Thu Jan 4 04:33:26 1996 --- utility.c Thu Jan 4 03:58:40 1996 *************** *** 394,402 **** putstr(s) register char *s; { ! while (*s) putchr(*s++); } void --- 394,408 ---- putstr(s) register char *s; { + int index = 0; ! while (*s) { ! index++; ! if (index > BUFSIZ) { ! err(1,"putstr(): overflow in string length.\n"); ! } putchr(*s++); + } } void *************** *** 424,430 **** { char *slash; time_t t; ! char db[100]; #ifdef STREAMSPTY extern char *index(); #else --- 430,438 ---- { char *slash; time_t t; ! char db[BUFSIZ]; ! int index; ! #ifdef STREAMSPTY extern char *index(); #else *************** *** 439,448 **** putlocation = where; while (*cp) { if (*cp != '%') { ! putchr(*cp++); ! continue; } switch (*++cp) { --- 447,466 ---- putlocation = where; + index = 0; while (*cp) { if (*cp != '%') { ! if (index > BUFSIZ) { ! err(1, "putf() : overflow in while (*cp)\n"); ! } else { ! putchr(*cp++); ! index++; ! continue; ! } ! } ! index++; ! if (index > BUFSIZ) { ! err(1, "putf() : overflow in switch (*+cpp)\n"); } switch (*++cp) { From davidn@labs.usn.blaze.net.au Thu Feb 20 07:09:00 1997 Return-Path: Received: from labs.usn.blaze.net.au (labs.usn.blaze.net.au [203.17.53.30]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id HAA16362 for ; Thu, 20 Feb 1997 07:08:55 -0800 (PST) Received: (from davidn@localhost) by labs.usn.blaze.net.au (8.8.5/8.8.5) id CAA29696; Fri, 21 Feb 1997 02:08:42 +1100 (EST) Message-ID: <19970221020839.51222@usn.blaze.net.au> Date: Fri, 21 Feb 1997 02:08:39 +1100 From: David Nugent To: Warner Losh Cc: Mark Huizer , auditors@freebsd.org Subject: Re: Termcap use... References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.61 In-Reply-To: ; from Warner Losh on Feb 02, 1997 at 08:09:57AM On Feb 02, 1997 at 08:09:57AM, Warner Losh wrote: > In message Mark Huizer writes: > : What is the neat way to handle a termcap lookup?? > : > : the man page says to reserve a buffer of 1024 places and use tgetent, > : ok, perhaps a const in termcap would be nicer? (MAXTERMCAPENTRY or > : something?)... > : > : and then... some programs seem to put it in an array with tgetstr... any > : clue how big that buffer should be? 2048 since any code can result in 2 > : char string? Anything else??? > > Hmmm. Not sure. Someone recently told me that telnetd might have a > hole in it in this area. Something about "What happens when the TERM > (or was that TERMCAP) env variable is > 1024 characters?". Might be a > good thing for someone to thread through. I've looked through it fairly briefly, if only to become more familiar with getcap(3), on which termcap is based. getcap() does internal allocation of its own buffers, so it isn't subject to buffer overflows itself (in principle - I haven't actually verified that, which I expect will be done during this audit). Termcap uses getcap() (since it is a more generic form), but does some things for backwards compatibility such as truncating capability tags to two characters (ugh) and prevents any more than TBUFSIZ (#defined as 1024 at the top of lib/libtermcap/termcap.c) from being copied into user space. So, assuming that everyone uses 1024 byte buffers - which is common and a documented limit besides - it is a non-problem. A termcap capability record exceeding that size can't be returned. Of course, this very much limits the usability of termcap for very complex emulations, but that's the price of compatility. getcap(3) doesn't look too bad an interface in comparison and does not suffer the same problen, but of course it is BSD 4.? specific. getcap(3) can be used to access those same records of just about any size. Regards, David Nugent - Unique Computing Pty Ltd - Melbourne, Australia Voice +61-3-9791-9547 Data/BBS +61-3-9792-3507 3:632/348@fidonet davidn@freebsd.org davidn@blaze.net.au http://www.blaze.net.au/~davidn/ From albert@gamp.hacom.nl Thu Feb 27 09:34:57 1997 Return-Path: Received: from box.hacom.nl (root@box.hacom.nl [193.67.233.8]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id JAA08885 for ; Thu, 27 Feb 1997 09:34:56 -0800 (PST) Received: (from uucp@localhost) by box.hacom.nl (8.8.5/8.8.5) with UUCP id SAA17632; Thu, 27 Feb 1997 18:31:28 +0100 Received: (from albert@localhost) by beowulf.gamp.hacom.nl (8.6.12/10.2) id TAA27973; Wed, 26 Feb 1997 19:33:32 +0100 Date: Wed, 26 Feb 1997 19:33:32 +0100 Message-Id: <199702261833.TAA27973@beowulf.gamp.hacom.nl> From: Albert Mietus To: Brian Tao CC: marcs@znep.com, imp@village.org, auditors@FreeBSD.ORG In-reply-to: Brian Tao's message of Wed, 26 Feb 1997 01:17:56 -0500 (EST) Subject: Re: /etc/passwd, etc ownership References: > > How many cases are there where someone can mess with the ownership > > of /etc/passwd but not anything else? > > I've personally seen a couple of instances where a careless > programmer did not set his umask before manipulating password files, > and the result were mode 666 files. In one instance, the utility had > been running for several weeks before someone noticed the problem. True, this a security hole. BUT this example can NEVER (in my opinion) be a argument to "add code to prevent this". I'm sure that when a OS is designed that covers this problem, there is some even more stupid user... Yes, a cronjob (ala daily->security) that gives warnings works. But real restrictions will become a handycap, and so workarounds will be used! To summarize: yes we need a secure OS, yes we can use tools that verify this, NO we don't want restriction that make users (including/especially root) "careless". ---GAM "This should be a jolly quote" From henrich@crh.cl.msu.edu Fri Feb 28 15:11:00 1997 Return-Path: Received: from crh.cl.msu.edu (crh.cl.msu.edu [35.8.1.24]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id PAA24284 for ; Fri, 28 Feb 1997 15:10:58 -0800 (PST) Received: (from henrich@localhost) by crh.cl.msu.edu (8.8.5/8.8.4) id SAA06806 for auditors@freebsd.org; Fri, 28 Feb 1997 18:10:26 -0500 (EST) From: Charles Henrich Message-Id: <199702282310.SAA06806@crh.cl.msu.edu> Subject: Code updating.. To: auditors@freebsd.org Date: Fri, 28 Feb 1997 18:10:26 -0500 (EST) X-Mailer: ELM [version 2.4ME+ PL22 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Does everyone agree that we should fight the screaming urges to clean up massivly obtuse code during this sweep and only touch if security problems are had? -Crh Charles Henrich Michigan State University henrich@msu.edu http://pilot.msu.edu/~henrich From guido@gvr.win.tue.nl Sat Mar 1 05:40:35 1997 Return-Path: Received: from gvr.win.tue.nl (root@gvr.win.tue.nl [131.155.210.19]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id FAA05361 for ; Sat, 1 Mar 1997 05:40:32 -0800 (PST) Received: (from guido@localhost) by gvr.win.tue.nl (8.8.5/8.8.2) id OAA19220; Sat, 1 Mar 1997 14:40:15 +0100 (MET) From: Guido van Rooij Message-Id: <199703011340.OAA19220@gvr.win.tue.nl> Subject: Re: /etc/passwd, etc ownership In-Reply-To: from Marc Slemko at "Feb 26, 97 12:25:33 pm" To: marcs@znep.com (Marc Slemko) Date: Sat, 1 Mar 1997 14:40:15 +0100 (MET) Cc: phk@critter.dk.tfs.com, auditors@freebsd.org X-Mailer: ELM [version 2.4ME+ PL28 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit > > I'm thinking that a solution of simply adding a sysctl variable that > enables or disables chrooting from within a chrooted directory would give > the most bang for the buck. Should simply involve checking if fd_rdir is > NULL and, if not, refusing to chroot; 1 line change, plus the overhead to > make it configurable. You do have to then prevent people from changing > that sysctl variable in a chrooted environment... > > This change alone does not make it secure for someone to have root inside > a chrooted environment, but it is a first ste at making it a little bit > harder to break out. > This does make sense to me. -Guido From jkh@time.cdrom.com Mon Mar 10 10:10:23 1997 Return-Path: Received: from time.cdrom.com (root@time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA10450 for ; Mon, 10 Mar 1997 10:10:22 -0800 (PST) Received: from time.cdrom.com (jkh@localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id KAA16463 for ; Mon, 10 Mar 1997 10:10:35 -0800 (PST) To: auditors@freebsd.org Subject: William McVey: Volunteer for the Great Code Sweep Date: Mon, 10 Mar 1997 10:10:35 -0800 Message-ID: <16460.858017435@time.cdrom.com> From: "Jordan K. Hubbard" Is this effort even still alive? Here's another volunteer. :-) Sigh. I wish I wasn't so busy looking after the release or I'd try to get things going here again. Anyone here want to volunteer for the "Chief Auditor" position? I think it would really help since 99.9% of the people on this list seem to merely be waiting for someone to tell them what to do and then deal with whatever they churn out. :-) Jordan ------- Forwarded Message To: jkh@freebsd.org Subject: Volunteer for the Great Code Sweep Organization: Federal Express Data Protection Distributed Projects Date: Mon, 10 Mar 1997 11:33:39 -0600 From: William McVey I'd like to volunteer to take part in the code sweep. Please let me know what I need to do to get involved. -- William McVey Sr Data Protection Analyst Federal Express ------- End of Forwarded Message From mark@grackle.grondar.za Mon Mar 10 10:53:26 1997 Return-Path: Received: from grackle.grondar.za (M9F7j9vEHtlPPWyIrgPOQxkiml0wYUY9@grackle.grondar.za [196.7.18.131]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA13204 for ; Mon, 10 Mar 1997 10:53:24 -0800 (PST) Received: from grackle.grondar.za (bctRRlQvlHlw0MMDlQ/bTlj458R5JlDk@localhost [127.0.0.1]) by grackle.grondar.za (8.8.5/8.8.4) with ESMTP id UAA24090; Mon, 10 Mar 1997 20:53:04 +0200 (SAT) Message-Id: <199703101853.UAA24090@grackle.grondar.za> X-Mailer: exmh version 2.0gamma 1/27/96 To: "Jordan K. Hubbard" cc: auditors@freebsd.org Subject: Re: William McVey: Volunteer for the Great Code Sweep Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 10 Mar 1997 20:52:56 +0200 From: Mark Murray Dunno how the report-back works. Consider the secure/ area audited, and the eBones area about to be replaced. (I am waiting for the lite2 brouhaha to die down so I can make world again before I do any committing there... "Jordan K. Hubbard" wrote: > Is this effort even still alive? Here's another volunteer. :-) > > Sigh. I wish I wasn't so busy looking after the release or I'd try to > get things going here again. Anyone here want to volunteer for the > "Chief Auditor" position? I think it would really help since 99.9% of > the people on this list seem to merely be waiting for someone to tell > them what to do and then deal with whatever they churn out. :-) > > Jordan > ------- Forwarded Message > > To: jkh@freebsd.org > Subject: Volunteer for the Great Code Sweep > Organization: Federal Express Data Protection Distributed Projects > Date: Mon, 10 Mar 1997 11:33:39 -0600 > From: William McVey > > I'd like to volunteer to take part in the code sweep. Please let me know > what I need to do to get involved. > > -- William McVey > Sr Data Protection Analyst > Federal Express > > ------- End of Forwarded Message > -- Mark Murray PGP key fingerprint = 80 36 6E 40 83 D6 8A 36 This .sig is umop ap!sdn. BC 06 EA 0E 7A F2 CE CE From cschuber@uumail.gov.bc.ca Mon Mar 10 10:58:10 1997 Return-Path: Received: from passer.osg.gov.bc.ca (passer.osg.gov.bc.ca [142.32.110.29]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA13859 for ; Mon, 10 Mar 1997 10:57:48 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by passer.osg.gov.bc.ca (8.8.5/8.6.10) with SMTP id KAA16618 for auditors@freebsd.org; Mon, 10 Mar 1997 10:57:35 -0800 (PST) From: Cy Schubert - ITSD Open Systems Group Message-Id: <199703101857.KAA16618@passer.osg.gov.bc.ca> X-Authentication-Warning: passer.osg.gov.bc.ca: localhost [127.0.0.1] didn't use HELO protocol Reply-to: cschuber@uumail.gov.bc.ca X-Mailer: MH X-Sender: cschuber To: auditors@freebsd.org Subject: Code Scan Project Date: Mon, 10 Mar 97 10:57:34 -0800 X-Mts: smtp I would like to help out with the Great Code Scan Project. I've volunteered but haven't heard anything yet. Can I help out? Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 UNIX Support OV/VM: BCSC02(CSCHUBER) ITSD BITNET: CSCHUBER@BCSC02.BITNET Government of BC Internet: cschuber@uumail.gov.bc.ca cschuber@bcsc02.gov.bc.ca "Quit spooling around, JES do it." ------- Forwarded Message Return-Path: jkh@time.cdrom.com Delivery-Date: Mon, 10 Mar 97 10:51:20 -0800 Return-Path: jkh@time.cdrom.com Received: from orca.gov.bc.ca (orca.gov.bc.ca [142.32.102.25]) by passer.osg.gov.bc.ca (8.8.5/8.6.10) with SMTP id KAA13876 for ; Mon, 10 Mar 1997 10:51:19 -0800 (PST) Received: from time.cdrom.com by orca.gov.bc.ca (5.4R3.10/200.1.1.4) id AA21329; Mon, 10 Mar 1997 10:51:14 -0800 Received: from time.cdrom.com (jkh@localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id KAA16658 for ; Mon, 10 Mar 1997 10:50:56 -0800 (PST) To: cschuber@uumail.gov.bc.ca Subject: Re: 2.1.7-RELEASE CDROM now shipping from Walnut Creek CDROM. In-Reply-To: Your message of "Mon, 10 Mar 1997 10:21:32 PST." <199703101821.KAA16933@passer.osg.gov.bc.ca> Date: Mon, 10 Mar 1997 10:50:56 -0800 Message-Id: <16655.858019856@time.cdrom.com> From: "Jordan K. Hubbard" > Jordan, has the Core Team been able to ascertain whether any trojans have > been installed in the source tree? None have been found so far, that's all I (or anyone) can say. > Though I haven't heard anything yet, I would still like to help out with > the source code scan project. (I don't think I need a free CDROM as I have > not contributed anything yet and am already a FreeBSD subscriber. $24 is > not much to pay for such an excellent product). Well, why not ping auditors@freebsd.org and see what they're up to over there? :-) Jordan ------- End of Forwarded Message From henrich@crh.cl.msu.edu Mon Mar 10 14:01:08 1997 Return-Path: Received: from crh.cl.msu.edu (crh.cl.msu.edu [35.8.1.24]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id OAA29953 for ; Mon, 10 Mar 1997 14:00:57 -0800 (PST) Received: (from henrich@localhost) by crh.cl.msu.edu (8.8.5/8.8.4) id RAA01637; Mon, 10 Mar 1997 17:00:15 -0500 (EST) From: Charles Henrich Message-Id: <199703102200.RAA01637@crh.cl.msu.edu> Subject: Re: William McVey: Volunteer for the Great Code Sweep To: jkh@time.cdrom.com (Jordan K. Hubbard) Date: Mon, 10 Mar 1997 17:00:15 -0500 (EST) Cc: auditors@freebsd.org In-Reply-To: <16460.858017435@time.cdrom.com> from "Jordan K. Hubbard" at "Mar 10, 97 10:10:35 am" X-Mailer: ELM [version 2.4ME+ PL22 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit > Is this effort even still alive? Here's another volunteer. :-) Well I submitted my changes and got blasted for them, so I've been holding back doing anymore, and then we stumbled across this whole RSA contest thing, so I imagine at some point I'll be getting back into it :) Its just hard to get up the courage to go through another submission round :) -Crh Charles Henrich Michigan State University henrich@msu.edu http://pilot.msu.edu/~henrich From tenser@spitfire.ecsel.psu.edu Wed Mar 12 16:38:18 1997 Return-Path: Received: from spitfire.ecsel.psu.edu (qmailr@spitfire.ecsel.psu.edu [146.186.218.51]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA22290 for ; Wed, 12 Mar 1997 16:35:40 -0800 (PST) Received: (qmail 24171 invoked by uid 1000); 13 Mar 1997 00:32:50 -0000 Message-ID: <19970313003250.24170.qmail@spitfire.ecsel.psu.edu> To: "Jordan K. Hubbard" cc: auditors@freebsd.org Subject: Re: William McVey: Volunteer for the Great Code Sweep In-reply-to: Your message of "Mon, 10 Mar 1997 10:10:35 PST." <16460.858017435@time.cdrom.com> Date: Wed, 12 Mar 1997 19:32:47 -0500 From: Dan Cross > Is this effort even still alive? Here's another volunteer. :-) It's still alive, yes, though it's slowed to a crawl. Mark Murray and I both agree that ``secure'' deserves a clean bill of health. Oddly enough, we found that no changes had to be made in terms of security... It was kind of cool. :-) > Sigh. I wish I wasn't so busy looking after the release or I'd try to > get things going here again. Anyone here want to volunteer for the > "Chief Auditor" position? I think it would really help since 99.9% of > the people on this list seem to merely be waiting for someone to tell > them what to do and then deal with whatever they churn out. :-) Unfortunately, we're all in a turbulent state right now. The lite2 merge broke -current, making the task of auditing more difficult in many ways. I'm almost for splitting off the auditing into two waves, one for the 2.2 and 2.1 branches (which are very similar) and a delayed effort for -current to start after the lite2 merge is complete for the userland utilities. If not, we might find a lot of potential bugs that were ``fixed'' in the current code sweep being re-introduced in lite2. Unfortuantely, this leads to an unavoidable duplication of effort two or three months down the road, but it might be worth it. Does anyone have any other comments or thoughts on this matter? Personally, I think that this project is of sufficient importance that it's imperitive that the auditing NOT become a failure due to lack of interest, and I'll personally volunteer as much time as I can spare to see it through. :-) A ``chief auditor'' could definately help things out, but I think a more leak-proof structure for utilizing what brain and finger power is out there now could help. Basically, we need an auditor head for each code section, then two or three auditors to overlap each other and look for bugs. This isn't so that we have people second-guessing each other, but because I firmly believe that no one is perfect, and some bugs are going to get missed by each auditor, but maybe caught by others. Once the auditors agree on the state of a given section, their proposed changes are handed to the reviewers to search for errors and add input. Maybe this is what is supposed to be happening now, but I think that we're lacking in the overlap area, and that's not providing enough checks and balances for each piece of code. Having an auditor-czar to assign sections to people who don't really care is a good thing, but it would also be helpful to assign people who are hip on working on a particular piece of code to that piece of code, in order to maximize that person's interest in what they are doing. We've got a pretty good sized pool of eager people out there, but as we all know, eagerness is killed by managerial stagnation. :-) The question right now is wether or not this moment in time is a good one for a code audit: I personally say, ``no, wait two months and revisit it. There's too much happening right now.'' but that might sour a lot of folks on helping out. So what do ya'll think? I'm interested in other people's thoughts on this more than my own. :-) God help me, I'm sounding like a manager. :-) - Dan C. From gryphon@healer.com Wed Mar 12 23:40:14 1997 Return-Path: Received: from thumper.tmisnet.com (root@thumper.tmisnet.com [204.212.149.7]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id XAA11832 for ; Wed, 12 Mar 1997 23:40:11 -0800 (PST) Received: from token.healer.com (slip4.tmisnet.com [204.212.149.27]) by thumper.tmisnet.com (8.7.4/8.7.3) with SMTP id XAA23465; Wed, 12 Mar 1997 23:38:52 -0800 (PST) Message-ID: <3327AE9B.95D@healer.com> Date: Wed, 12 Mar 1997 23:36:59 -0800 From: Coranth Gryphon Reply-To: gryphon@healer.com X-Mailer: Mozilla 3.01 (Win95; I) MIME-Version: 1.0 To: Dan Cross CC: "Jordan K. Hubbard" , auditors@freebsd.org Subject: Re: Volunteer for the Great Code Sweep References: <19970313003250.24170.qmail@spitfire.ecsel.psu.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Dan Cross wrote: > Unfortunately, we're all in a turbulent state right now. The lite2 merge > broke -current, making the task of auditing more difficult in many ways. >... > code sweep being re-introduced in lite2. Unfortuantely, this leads to an > unavoidable duplication of effort two or three months down the road, but >... > right now is whether or not this moment in time is a good one for a code > audit: I personally say, ``no, wait two months and revisit it. There's > too much happening right now.'' but that might sour a lot of folks on > Personal opinion: Keep the effort going in low-burner mode for those two months, but step up the heat when things are more stable. > now could help. Basically, we need an auditor head for each code section, > then two or three auditors to overlap each other and look for bugs. This > isn't so that we have people second-guessing each other, but because I > firmly believe that no one is perfect, and some bugs are going to get > missed by each auditor, but maybe caught by others. Once the auditors > agree on the state of a given section, their proposed changes are handed > to the reviewers to search for errors and add input. Maybe this is what I like this method. It also makes sure that if someone says "I'll take such-and-such" and sits on it, then someone else will still be looking at the code. If we have enough volunteers, I'd suggest everything get gone over by at least two (or more) people. Also, with a seperate mailling list for each major chunk, those who are involved can discuss issues among themselves which one person is unsure of, so that you don't get three different solutions to the same bug. -coranth ------------------------------------------+---------------------------- Coranth Gryphon | Voice Mail: 888-341-3684 http://www.healer.com | #include ------------------------------------------+---------------------------- Life isn't about getting somewhere, it's about going... From jkh@time.cdrom.com Thu Mar 13 08:13:04 1997 Return-Path: Received: from time.cdrom.com (root@time.cdrom.com [204.216.27.226]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id IAA04883 for ; Thu, 13 Mar 1997 08:13:03 -0800 (PST) Received: from time.cdrom.com (jkh@localhost [127.0.0.1]) by time.cdrom.com (8.8.5/8.6.9) with ESMTP id IAA18168; Thu, 13 Mar 1997 08:12:26 -0800 (PST) To: Dan Cross cc: auditors@freebsd.org Subject: Re: William McVey: Volunteer for the Great Code Sweep In-reply-to: Your message of "Wed, 12 Mar 1997 19:32:47 EST." <19970313003250.24170.qmail@spitfire.ecsel.psu.edu> Date: Thu, 13 Mar 1997 08:12:26 -0800 Message-ID: <18164.858269546@time.cdrom.com> From: "Jordan K. Hubbard" > to start after the lite2 merge is complete for the userland utilities. If > not, we might find a lot of potential bugs that were ``fixed'' in the current > code sweep being re-introduced in lite2. Unfortuantely, this leads to an > unavoidable duplication of effort two or three months down the road, but > it might be worth it. Does anyone have any other comments or thoughts > on this matter? Personally, I think that this project is of sufficient I see that it's definitely a problem, though I don't think you'll have to wait quite so long as all that to see -current fixed enough to run again. Peter Wemm is apparently running a completely -current system again, kernel and all, so at least the core compilation issues are taken care of at this point. After he's finished merging the last of userland (which should be pretty soon, I'd expect), the code should be more than fit for auditing. I don't see the # of changes occuring to the L2 stuff at post-merge time being significant enough to make it unauditable. > importance that it's imperitive that the auditing NOT become a failure > due to lack of interest, and I'll personally volunteer as much time as I > can spare to see it through. :-) Good! :) > now could help. Basically, we need an auditor head for each code section, > then two or three auditors to overlap each other and look for bugs. This Yes, I think so too. A chief auditor can only keep things more or less together, while having each subtree have its own boss at least assures that someone will be in charge of a much more manageable group of people and hopefully making sure that all the right things happen in their section. > We've got a pretty good sized pool of eager people out there, but as we > all know, eagerness is killed by managerial stagnation. :-) The question > right now is wether or not this moment in time is a good one for a code > audit: I personally say, ``no, wait two months and revisit it. There's > too much happening right now.'' but that might sour a lot of folks on > helping out. So what do ya'll think? I'm interested in other people's > thoughts on this more than my own. :-) > > God help me, I'm sounding like a manager. :-) Yes, it's interesting how quickly you took to the role. I'd say you must have a *natural talent* for it and you should jump to volunteer for any and all future management tasks so that you can properly develop and refine this latent talent of yours. :-) :-) Jordan From obrien@dragon.nuxi.com Thu Mar 13 10:13:54 1997 Return-Path: Received: from relay.nuxi.com (nuxi.ucdavis.edu [128.120.175.23]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA12123 for ; Thu, 13 Mar 1997 10:13:53 -0800 (PST) Received: from dragon.nuxi.com (reqd-009.ucdavis.edu [128.120.251.129]) by relay.nuxi.com (8.8.5/8.6.12) with ESMTP id KAA00859 for ; Thu, 13 Mar 1997 10:19:01 GMT Received: (from obrien@localhost) by dragon.nuxi.com (8.8.5/8.7.3) id SAA13176; Thu, 13 Mar 1997 18:13:42 GMT Message-ID: <19970313101342.04331@dragon.nuxi.com> Date: Thu, 13 Mar 1997 10:13:42 -0800 From: "David O'Brien" To: auditors@freebsd.org Subject: Re: William McVey: Volunteer for the Great Code Sweep References: <16460.858017435@time.cdrom.com> <19970313003250.24170.qmail@spitfire.ecsel.psu.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.65_p2,4-7,10-11,15,18,21-22 Reply-To: obrien@NUXI.com X-Warning: Mutt Bites! Organization: The NUXI *BSD group X-PGP-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Keyid: 34F9F9D5 On Wed, Mar 12, 1997 at 07:32:47PM -0500, Dan Cross wrote: > The question right now is wether or not this moment in time is a good > one for a code audit: I personally say, ``no, wait two months and > revisit it. There's too much happening right now.'' Agreed. With -current being so broken and the release about to be reality, it is not a good time. Wait two months would be cool. Then we will be closer to being able to make sure any changes actually compile and don't break anything. > but that might sour a lot of folks on helping out. So what do ya'll > think? BUT, we aren't preventing those that have the time now and the desire to work on the audit from doing so. :-) > God help me, I'm sounding like a manager. :-) Better you than me. hahahaha -- -- David (obrien@NUXI.com -or- obrien@FreeBSD.org) From tenser@spitfire.ecsel.psu.edu Thu Mar 13 19:59:09 1997 Return-Path: Received: from spitfire.ecsel.psu.edu (qmailr@spitfire.ecsel.psu.edu [146.186.218.51]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id TAA13336 for ; Thu, 13 Mar 1997 19:58:38 -0800 (PST) Received: (qmail 12559 invoked by uid 1000); 14 Mar 1997 03:58:26 -0000 Message-ID: <19970314035825.12558.qmail@spitfire.ecsel.psu.edu> To: "Jordan K. Hubbard" cc: auditors@freebsd.org Subject: Re: William McVey: Volunteer for the Great Code Sweep In-reply-to: Your message of "Thu, 13 Mar 1997 08:12:26 PST." <18164.858269546@time.cdrom.com> Date: Thu, 13 Mar 1997 22:58:25 -0500 From: Dan Cross > I see that it's definitely a problem, though I don't think you'll have > to wait quite so long as all that to see -current fixed enough to run > again. > > Peter Wemm is apparently running a completely -current system again, > kernel and all, so at least the core compilation issues are taken care > of at this point. After he's finished merging the last of userland > (which should be pretty soon, I'd expect), the code should be more > than fit for auditing. I don't see the # of changes occuring to the > L2 stuff at post-merge time being significant enough to make it > unauditable. Okay, I guess I over-estimated how long the merge would take. I too am running a completely -current system as of this morning (with a CVSup from cvsup2 as of yesterday). I was quite happy to see make world run to completion. :-) But I still maintain that we should wait until after the release of 2.2 and until the lite2 merge is completed before we continue full force with the audit. I think we can count on 2.2.5-RELEASE or whatever it's going to be being the most secure version of FreeBSD ever, though. :-) > Yes, I think so too. A chief auditor can only keep things more or > less together, while having each subtree have its own boss at least > assures that someone will be in charge of a much more manageable group > of people and hopefully making sure that all the right things happen > in their section. Agreed. > > God help me, I'm sounding like a manager. :-) > > Yes, it's interesting how quickly you took to the role. I'd say you > must have a *natural talent* for it and you should jump to volunteer > for any and all future management tasks so that you can properly > develop and refine this latent talent of yours. :-) :-) Hehehehehe... Since I know that you're joking, I'm not as scared by this as I should be. :-) - Dan C. From imp@village.org Sun Mar 16 01:25:59 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id BAA17470 for ; Sun, 16 Mar 1997 01:25:54 -0800 (PST) Received: (from imp@localhost) by rover.village.org (8.8.5/8.6.6) id CAA00693 for auditors@freebsd.org; Sun, 16 Mar 1997 02:25:37 -0700 (MST) Date: Sun, 16 Mar 1997 02:25:37 -0700 (MST) From: Warner Losh Message-Id: <199703160925.CAA00693@rover.village.org> To: auditors@freebsd.org Subject: FYI I'm currently coming up with a bunch of diffs so that mktemp, tempnam and tmpnam are converted to mkstemp. this was inspired by OpenBSD. I mention it now because NetBSD seems to be doing the same thing. Anybody wanna review the deltas? Warner From imp@village.org Mon Apr 28 21:33:05 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (8.8.5/8.8.5) with SMTP id VAA23280 for ; Mon, 28 Apr 1997 21:33:01 -0700 (PDT) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 1.60 #1) id 0wM4b5-00014j-00; Mon, 28 Apr 1997 22:32:59 -0600 To: auditors@freebsd.org Subject: Hello? Date: Mon, 28 Apr 1997 22:32:59 -0600 From: Warner Losh Message-Id: Anybody home? Hello? Warner From taob@nbc.netcom.ca Mon Apr 28 21:56:29 1997 Return-Path: Received: from tor-adm1.nbc.netcom.ca (taob@tor-adm1.nbc.netcom.ca [207.181.89.5]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id VAA24857 for ; Mon, 28 Apr 1997 21:56:29 -0700 (PDT) Received: from localhost (taob@localhost) by tor-adm1.nbc.netcom.ca (8.8.5/8.8.5) with SMTP id AAA00731; Tue, 29 Apr 1997 00:55:21 -0400 (EDT) Date: Tue, 29 Apr 1997 00:55:21 -0400 (EDT) From: Brian Tao To: Warner Losh cc: auditors@freebsd.org Subject: Re: Hello? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Mon, 28 Apr 1997, Warner Losh wrote: > > Anybody home? > > Hello? Ummmm... I guess we're still all here... -- Brian Tao (BT300, taob@netcom.ca) "Though this be madness, yet there is method in't" From xaa@xaa.stack.nl Tue Apr 29 02:22:52 1997 Return-Path: Received: from terra.stack.nl (terra.stack.nl [131.155.140.128]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id CAA07157 for ; Tue, 29 Apr 1997 02:22:49 -0700 (PDT) Received: from xaa.stack.nl (uucp@localhost) by terra.stack.nl (8.8.5) with UUCP id LAA14518; Tue, 29 Apr 1997 11:22:19 +0200 (MET DST) Received: (from xaa@localhost) by xaa.stack.nl (8.8.5/8.8.2) id LAA01549; Tue, 29 Apr 1997 11:18:40 +0200 (MET DST) Message-ID: <19970429111840.63093@xaa.stack.nl> Date: Tue, 29 Apr 1997 11:18:40 +0200 From: Mark Huizer To: Warner Losh Cc: auditors@freebsd.org Subject: Re: Hello? References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.69 In-Reply-To: ; from "Warner Losh" on Mon, Apr 28, 1997 at 10:32:59PM -0600 Reply-To: xaa@stack.nl (Mark Huizer) The wise Warner Losh produced the following lines: > > Anybody home? > > Hello? > Yep me :-) Is it time to wake up and get back to work already? Mark ------------------------------------------------------------------------- - Mark Huizer - xaa@stack.nl - rcbamh@urc.tue.nl - ------------------------------------------------------------------------- - If you think the problem is bad now, just wait till we solve it. - ------------------------------------------------------------------------- From adrian@wisdom.psinet.net.au Tue Apr 29 02:57:29 1997 Return-Path: Received: from wisdom.psinet.net.au (adrian@wisdom.psinet.net.au [203.62.152.34]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id CAA08247 for ; Tue, 29 Apr 1997 02:57:28 -0700 (PDT) Received: (from adrian@localhost) by wisdom.psinet.net.au (8.8.5/8.7) id RAA13176; Tue, 29 Apr 1997 17:55:15 +0800 Date: Tue, 29 Apr 1997 17:55:14 +0800 (WST) From: Adrian Chadd To: Brian Tao cc: Warner Losh , auditors@freebsd.org Subject: Re: Hello? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII I'm alive too. :) On Tue, 29 Apr 1997, Brian Tao wrote: > On Mon, 28 Apr 1997, Warner Losh wrote: > > > > Anybody home? > > > > Hello? > > Ummmm... I guess we're still all here... > -- > Brian Tao (BT300, taob@netcom.ca) > "Though this be madness, yet there is method in't" > From rajivd@sprynet.com Tue Apr 29 03:26:42 1997 Return-Path: Received: from sprynet.com (root@dd32-178.compuserve.com [199.174.149.178]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id DAA09297 for ; Tue, 29 Apr 1997 03:26:40 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by sprynet.com (8.8.4/8.8.4) with ESMTP id GAA16861; Tue, 29 Apr 1997 06:22:10 -0400 Message-Id: <199704291022.GAA16861@sprynet.com> X-Mailer: exmh version 1.6.9 05/05/96 To: Adrian Chadd cc: Brian Tao , Warner Losh , auditors@freebsd.org Subject: Re: Hello? In-reply-to: Your message of "Tue, 29 Apr 1997 17:55:14 +0800." Reply-To: Rajiv Dighe Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 29 Apr 1997 06:22:10 -0400 From: Rajiv Dighe That makes 5 of us :-) Rajiv From eivind@bitbox.follo.net Tue Apr 29 05:35:34 1997 Return-Path: Received: from bitbox.follo.net (bitbox.follo.net [194.198.43.36]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id FAA14121 for ; Tue, 29 Apr 1997 05:35:13 -0700 (PDT) Received: (from eivind@localhost) by bitbox.follo.net (8.7.6/8.7.3) id OAA18813; Tue, 29 Apr 1997 14:34:08 +0200 (MET DST) Date: Tue, 29 Apr 1997 14:34:08 +0200 (MET DST) Message-Id: <199704291234.OAA18813@bitbox.follo.net> From: Eivind Eklund To: Warner Losh CC: auditors@freebsd.org In-reply-to: Warner Losh's message of Mon, 28 Apr 1997 22:32:59 -0600 Subject: Re: Hello? References: >Anybody home? > >Hello? I was just thinking about sending a similar message yesterday. Are everybody up to date? Any patches that should be brought into the tree? I know, I know, I have audits and patches I should have finished - but I'm trying to fill in two full-time jobs here, so I'm lacking time. I'll at least make sure patches other people create make it into the tree reasonably quick. Eivind. From xaa@alterego.stack.nl Tue Apr 29 05:41:58 1997 Return-Path: Received: from alterego.stack.nl (alterego.stack.nl [131.155.141.160]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id FAA14481 for ; Tue, 29 Apr 1997 05:41:54 -0700 (PDT) Received: (from xaa@localhost) by alterego.stack.nl (8.8.5/8.8.5) id OAA08695; Tue, 29 Apr 1997 14:42:49 +0200 (MET DST) Message-ID: <19970429144248.60760@alterego.stack.nl> Date: Tue, 29 Apr 1997 14:42:48 +0200 From: Mark Huizer To: auditors@freebsd.org Subject: Re: Hello? References: <199704291234.OAA18813@bitbox.follo.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.69 In-Reply-To: <199704291234.OAA18813@bitbox.follo.net>; from "Eivind Eklund" on Tue, Apr 29, 1997 at 02:34:08PM +0200 > I was just thinking about sending a similar message yesterday. Are > everybody up to date? Any patches that should be brought into the > tree? > > I know, I know, I have audits and patches I should have finished - but > I'm trying to fill in two full-time jobs here, so I'm lacking time. > I'll at least make sure patches other people create make it into the > tree reasonably quick. I'll have a look one of these days, probably when doing my helpdesk work at the university. Problem is that I finally decided to reallty finish that study of mine, so time is very sparse Mark ------------------------------------------------------------------------- - Mark Huizer - xaa@stack.nl - rcbamh@urc.tue.nl - ------------------------------------------------------------------------- - Whenever people agree with me, I always feel I must be wrong - - ( Oscar Wilde ) - ------------------------------------------------------------------------- From imp@village.org Tue Apr 29 08:10:26 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (8.8.5/8.8.5) with SMTP id IAA20326 for ; Tue, 29 Apr 1997 08:10:24 -0700 (PDT) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 1.60 #1) id 0wMEXu-0001cP-00; Tue, 29 Apr 1997 09:10:22 -0600 Subject: Re: Hello? To: auditors@freebsd.org In-reply-to: Your message of "Tue, 29 Apr 1997 14:34:08 +0200." <199704291234.OAA18813@bitbox.follo.net> References: <199704291234.OAA18813@bitbox.follo.net> Date: Tue, 29 Apr 1997 09:10:22 -0600 From: Warner Losh Message-Id: In message <199704291234.OAA18813@bitbox.follo.net> Eivind Eklund writes: : I was just thinking about sending a similar message yesterday. Are : everybody up to date? Any patches that should be brought into the : tree? I have been bringing in many of the OpenBSD changes into my own tree here and building it. I have also been bringing in some of the security PRs from Julian A and cross checking against OpenBSD. I'm half tempted to just commit the goofy things and let the world test them for me. :-) Warner From pst@shockwave.com Tue Apr 29 08:26:17 1997 Return-Path: Received: from precipice.shockwave.com (ppp-206-170-6-47.rdcy01.pacbell.net [206.170.6.47]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id IAA20993 for ; Tue, 29 Apr 1997 08:26:15 -0700 (PDT) Received: from shockwave.com (localhost.shockwave.com [127.0.0.1]) by precipice.shockwave.com (8.8.5/8.7.3) with ESMTP id IAA14359; Tue, 29 Apr 1997 08:25:27 -0700 (PDT) Message-Id: <199704291525.IAA14359@precipice.shockwave.com> To: Warner Losh cc: auditors@freebsd.org Subject: Re: Hello? In-reply-to: Your message of "Tue, 29 Apr 1997 09:10:22 MDT." Date: Tue, 29 Apr 1997 08:25:27 -0700 From: Paul Traina Why not post a cvs diff and a digest first so at least some folks can try to do a quickie code review? From: Warner Losh Subject: Re: Hello? In message <199704291234.OAA18813@bitbox.follo.net> Eivind Eklund writes: : I was just thinking about sending a similar message yesterday. Are : everybody up to date? Any patches that should be brought into the : tree? I have been bringing in many of the OpenBSD changes into my own tree here and building it. I have also been bringing in some of the security PRs from Julian A and cross checking against OpenBSD. I'm half tempted to just commit the goofy things and let the world test them for me. :-) Warner From imp@village.org Tue Apr 29 08:33:47 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (8.8.5/8.8.5) with SMTP id IAA21278 for ; Tue, 29 Apr 1997 08:33:46 -0700 (PDT) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 1.60 #1) id 0wMEtx-0001dp-00; Tue, 29 Apr 1997 09:33:09 -0600 To: Paul Traina Subject: Re: Hello? Cc: auditors@freebsd.org In-reply-to: Your message of "Tue, 29 Apr 1997 08:25:27 PDT." <199704291525.IAA14359@precipice.shockwave.com> References: <199704291525.IAA14359@precipice.shockwave.com> Date: Tue, 29 Apr 1997 09:33:08 -0600 From: Warner Losh Message-Id: In message <199704291525.IAA14359@precipice.shockwave.com> Paul Traina writes: : Why not post a cvs diff and a digest first so at least some folks can try : to do a quickie code review? Hmmm, that's not too bad. I think I'll do that. Warner From tenser@spitfire.ecsel.psu.edu Tue Apr 29 09:45:11 1997 Return-Path: Received: from spitfire.ecsel.psu.edu (qmailr@spitfire.ecsel.psu.edu [146.186.218.51]) by hub.freebsd.org (8.8.5/8.8.5) with SMTP id JAA25188 for ; Tue, 29 Apr 1997 09:45:10 -0700 (PDT) Received: (qmail 28985 invoked by uid 1000); 29 Apr 1997 16:45:07 -0000 Message-ID: <19970429164507.28984.qmail@spitfire.ecsel.psu.edu> To: Eivind Eklund cc: auditors@freebsd.org Subject: Re: Hello? In-reply-to: Your message of "Tue, 29 Apr 1997 14:34:08 +0200." <199704291234.OAA18813@bitbox.follo.net> Date: Tue, 29 Apr 1997 12:45:07 -0400 From: Dan Cross > I'm trying to fill in two full-time jobs here, so I'm lacking time. I might be alive again after this semester is over (two weeks). Yee haw!! - Dan C. From imp@village.org Tue Apr 29 15:51:37 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (8.8.5/8.8.5) with SMTP id PAA15817 for ; Tue, 29 Apr 1997 15:51:30 -0700 (PDT) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 1.60 #1) id 0wMLk8-0002Dz-00; Tue, 29 Apr 1997 16:51:28 -0600 To: auditors@freebsd.org Subject: First hunk of diffs Date: Tue, 29 Apr 1997 16:51:28 -0600 From: Warner Losh Message-Id: I've been running with these diffs now for some time. These seem sane. Comments? Warner Here's a digest of what I'm doing. libc/db/btree/bt_open.c >From OpenBSD: revision 1.4 date: 1996/08/26 00:17:14; author: deraadt; state: Exp; lines: +4 -3 use issetugid() to protect against bad getenv ---------------------------------------- libc/gen/glob.c >From OpenBSD: Working file: glob.c revision 1.3 date: 1996/09/11 19:22:46; author: deraadt; state: Exp; lines: +2 -2 protect $HOME expansion; from das33@cornell.edu libc/net/res_comp.c >From OpenBSD by Theo de Raadt: Limit the size of the buffer to MAXDNAME. ---------------------------------------- libc/net/res_init.c >From OpenBSD: Working file: res_init.c revision 1.6 date: 1996/08/27 03:32:53; author: deraadt; state: Exp; lines: +4 -1 use strncpy correctly revision 1.5 date: 1996/08/25 10:11:02; author: deraadt; state: Exp; lines: +5 -4 use issetugid() ---------------------------------------- libc/nls/msgcat.c >From OpenBSD: Working file: catopen.c revision 1.6 date: 1996/08/26 00:17:20; author: deraadt; state: Exp; lines: +6 -8 use issetugid() to protect against bad getenv ---------------------------------------- libc/stdio/mktemp.c libc/stdio/tempnam.c libc/stdio/tmpnam.c >From OpenBSD: 95% of common uses of these are incorrect and insecure. correct use is incredibly rare. Time for some education! Also, 1.3 and 1.4 from OpenBSD's tempname have been merged in. These only cause TMPDIR env to be fetched when we aren't running setuid or setgid. ---------------------------------------- libc/stdtime/localtime.c >From OpenBSD: Working file: localtime.c revision 1.10 date: 1997/04/02 03:57:30; author: deraadt; state: Exp; lines: +6 -2 correctly code the classes of permitted TZ specifications for the issetugid() case. thanks bitblt and tholo revision 1.6 date: 1996/09/05 12:28:23; author: deraadt; state: Exp; lines: +2 -2 1 char oflow revision 1.5 date: 1996/08/25 10:11:11; author: deraadt; state: Exp; lines: +2 -2 use issetugid() And some unknown witespace change that shouldn't be there.... ---------------------------------------- libedit/el.c >From OpenBSD: Working file: el.c revision 1.2 date: 1996/08/26 00:17:22; author: deraadt; state: Exp; lines: +2 -2 use issetugid() to protect against bad getenv ---------------------------------------- libftpio/ftpio.c >From Julian A: Limit the size of h_length used to the actual size of the sin_addr buffer. ---------------------------------------- libskey/skeyaccess.c >From Julian A: Limit the size of h_length used to the actual size of the sin_addr buffer. Also, fix 1 character overflow (which may also be in OpenBSD). Index: libc/db/btree/bt_open.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/db/btree/bt_open.c,v retrieving revision 1.4 diff -u -r1.4 bt_open.c --- bt_open.c 1996/07/12 18:53:13 1.4 +++ bt_open.c 1997/04/28 20:00:02 @@ -388,10 +388,11 @@ { sigset_t set, oset; int fd; - char *envtmp; + char *envtmp = NULL; char path[MAXPATHLEN]; - envtmp = getenv("TMPDIR"); + if (issetugid() == 0) + envtmp = getenv("TMPDIR"); (void)snprintf(path, sizeof(path), "%s/bt.XXXXXX", envtmp ? envtmp : "/tmp"); Index: libc/gen/glob.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/gen/glob.c,v retrieving revision 1.8 diff -u -r1.8 glob.c --- glob.c 1997/04/04 19:16:08 1.8 +++ glob.c 1997/04/28 20:05:28 @@ -361,7 +361,7 @@ * handle a plain ~ or ~/ by expanding $HOME * first and then trying the password file */ - if ((h = getenv("HOME")) == NULL) { + if (issetugid() != 0 || (h = getenv("HOME")) == NULL) { if ((pwd = getpwuid(getuid())) == NULL) return pattern; else Index: libc/net/res_comp.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/net/res_comp.c,v retrieving revision 1.10 diff -u -r1.10 res_comp.c --- res_comp.c 1997/02/22 15:00:29 1.10 +++ res_comp.c 1997/03/28 17:17:55 @@ -95,7 +95,7 @@ dn = exp_dn; cp = comp_dn; - eom = exp_dn + length; + eom = exp_dn + (length > MAXDNAME ? MAXDNAME : length); /* * fetch next label in domain name */ Index: libc/net/res_init.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/net/res_init.c,v retrieving revision 1.12 diff -u -r1.12 res_init.c --- res_init.c 1997/02/22 15:00:32 1.12 +++ res_init.c 1997/04/28 20:15:17 @@ -177,8 +177,9 @@ _res.pfcode = 0; /* Allow user to override the local domain definition */ - if ((cp = getenv("LOCALDOMAIN")) != NULL) { + if (issetugid() == 0 && (cp = getenv("LOCALDOMAIN")) != NULL) { (void)strncpy(_res.defdname, cp, sizeof(_res.defdname) - 1); + _res.defdname[sizeof(_res.defdname) - 1] = '\0'; haveenv++; /* @@ -231,6 +232,7 @@ if ((*cp == '\0') || (*cp == '\n')) continue; strncpy(_res.defdname, cp, sizeof(_res.defdname) - 1); + _res.defdname[sizeof(_res.defdname) - 1] = '\0'; if ((cp = strpbrk(_res.defdname, " \t\n")) != NULL) *cp = '\0'; havesearch = 0; @@ -246,6 +248,7 @@ if ((*cp == '\0') || (*cp == '\n')) continue; strncpy(_res.defdname, cp, sizeof(_res.defdname) - 1); + _res.defdname[sizeof(_res.defdname) - 1] = '\0'; if ((cp = strchr(_res.defdname, '\n')) != NULL) *cp = '\0'; /* @@ -379,7 +382,9 @@ #endif /* !RFC1535 */ } - if ((cp = getenv("RES_OPTIONS")) != NULL) + if (issetugid()) + _res.options |= RES_NOALIASES; + else if ((cp = getenv("RES_OPTIONS")) != NULL) res_setoptions(cp, "env"); _res.options |= RES_INIT; return (0); Index: libc/nls/msgcat.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/nls/msgcat.c,v retrieving revision 1.9 diff -u -r1.9 msgcat.c --- msgcat.c 1997/03/25 05:36:37 1.9 +++ msgcat.c 1997/04/28 20:02:30 @@ -101,9 +101,8 @@ } else { if ((lang = (char *) getenv("LANG")) == NULL) lang = "C"; - /* XXX Should really be issetguid(), but we don't have that */ if ((nlspath = (char *) getenv("NLSPATH")) == NULL || - getuid() != geteuid() || getgid() != getegid()) { + issetguid() != 0) { nlspath = "/usr/share/nls/%L/%N.cat:/usr/share/nls/%N/%L:/usr/local/share/nls/%L/%N.cat:/usr/local/share/nls/%N/%L"; } Index: libc/stdio/mktemp.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/stdio/mktemp.c,v retrieving revision 1.7 diff -u -r1.7 mktemp.c --- mktemp.c 1997/04/07 18:01:10 1.7 +++ mktemp.c 1997/04/26 18:49:42 @@ -59,10 +59,20 @@ } char * -mktemp(path) +_mktemp(path) char *path; { return(_gettemp(path, (int *)NULL) ? path : (char *)NULL); +} + +__warn_references(mktemp, + "warning: mktemp() possibly used unsafely; consider using mkstemp()"); + +char * +mktemp(path) + char *path; +{ + return(_mktemp(path)); } static int Index: libc/stdio/tempnam.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/stdio/tempnam.c,v retrieving revision 1.5 diff -u -r1.5 tempnam.c --- tempnam.c 1997/02/22 15:02:37 1.5 +++ tempnam.c 1997/04/28 19:51:28 @@ -47,6 +47,11 @@ #include #include +__warn_references(tempnam, + "warning: tempnam() possibly used unsafely; consider using mkstemp()"); + +extern char *_mktemp __P((char *)); + char * tempnam(dir, pfx) const char *dir, *pfx; @@ -60,7 +65,7 @@ if (!pfx) pfx = "tmp."; - if ((f = getenv("TMPDIR"))) { + if (issetugid() == 0 && (f = getenv("TMPDIR"))) { (void)snprintf(name, MAXPATHLEN, "%s%s%sXXXXXX", f, *(f + strlen(f) - 1) == '/'? "": "/", pfx); if ((f = mktemp(name))) Index: libc/stdio/tmpnam.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/stdio/tmpnam.c,v retrieving revision 1.1.1.1 diff -u -r1.1.1.1 tmpnam.c --- tmpnam.c 1994/05/27 04:57:32 1.1.1.1 +++ tmpnam.c 1997/04/28 19:52:25 @@ -43,6 +43,11 @@ #include #include +__warn_references(tmpnam, + "warning: tmpnam() possibly used unsafely; consider using mkstemp()"); + +extern char *_mktemp __P((char *)); + char * tmpnam(s) char *s; Index: libc/stdtime/localtime.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/stdtime/localtime.c,v retrieving revision 1.15 diff -u -r1.15 localtime.c --- localtime.c 1997/03/25 05:34:31 1.15 +++ localtime.c 1997/04/29 14:35:28 @@ -273,6 +273,11 @@ register int i; register int fid; + /* XXX The following is from OpenBSD, and I'm not sure it is correct */ + if (name != NULL && issetugid() != 0) + if ((name[0] == ':' && name[1] == '/') || + name[0] == '/' || strchr(name, '.')) + name = NULL; if (name == NULL && (name = TZDEFAULT) == NULL) return -1; { @@ -293,7 +298,7 @@ if (!doaccess) { if ((p = TZDIR) == NULL) return -1; - if ((strlen(p) + strlen(name) + 1) >= sizeof fullname) + if ((strlen(p) + 1 + strlen(name) + 1) >= sizeof fullname) return -1; (void) strcpy(fullname, p); (void) strcat(fullname, "/"); @@ -306,7 +311,7 @@ name = fullname; } if (doaccess && access(name, R_OK) != 0) - return -1; + return -1; if ((fid = open(name, OPEN_MODE)) == -1) return -1; if ((fstat(fid, &stab) < 0) || !S_ISREG(stab.st_mode)) Index: libedit/el.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libedit/el.c,v retrieving revision 1.3 diff -u -r1.3 el.c --- el.c 1997/03/23 23:17:22 1.3 +++ el.c 1997/04/28 20:23:05 @@ -77,7 +77,7 @@ el->el_prog = strdup(prog); #ifdef DEBUG - if ((tty = getenv("DEBUGTTY")) != NULL) { + if (issetugid() == 0 && (tty = getenv("DEBUGTTY")) != NULL) { el->el_errfile = fopen(tty, "w"); if (el->el_errfile == NULL) { extern errno; @@ -291,7 +291,7 @@ if (fname == NULL) { fname = &elpath[1]; if ((fp = fopen(fname, "r")) == NULL) { - if ((ptr = getenv("HOME")) == NULL) + if (issetugid() != 0 || (ptr = getenv("HOME")) == NULL) return -1; (void)snprintf(path, sizeof(path), "%s%s", ptr, elpath); fname = path; Index: libftpio/ftpio.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libftpio/ftpio.c,v retrieving revision 1.25 diff -u -r1.25 ftpio.c --- ftpio.c 1997/02/22 15:06:50 1.25 +++ ftpio.c 1997/03/22 05:07:29 @@ -35,6 +35,7 @@ #include #include #include +#include #define SUCCESS 0 #define FAILURE -1 @@ -701,7 +702,7 @@ return FAILURE; } ftp->addrtype = sin.sin_family = he->h_addrtype; - bcopy(he->h_addr, (char *)&sin.sin_addr, he->h_length); + bcopy(he->h_addr, (char *)&sin.sin_addr, MIN(he->h_length,sizeof(sin.sin_addr))); } sin.sin_port = htons(port); Index: libskey/skeyaccess.c =================================================================== RCS file: /home/imp/FreeBSD/CVS/src/lib/libskey/skeyaccess.c,v retrieving revision 1.7 diff -u -r1.7 skeyaccess.c --- skeyaccess.c 1996/10/17 21:49:34 1.7 +++ skeyaccess.c 1997/03/22 05:08:11 @@ -408,12 +408,11 @@ for (i = 0; i < MAX_ADDR && hp->h_addr_list[i]; i++) memcpy((char *) &list[i], - hp->h_addr_list[i], hp->h_length); + hp->h_addr_list[i], (length=MIN(hp->h_length, sizeof(struct in_addr)))); list[i].s_addr = 0; strncpy(buf, hp->h_name, MAXHOSTNAMELEN); - buf[MAXHOSTNAMELEN] = 0; - length = hp->h_length; + buf[MAXHOSTNAMELEN - 1] = 0; /* * Wipe addresses that appear to belong to someone else. We will get From eivind@freebsd.org Tue Apr 29 17:09:09 1997 Return-Path: Received: from nic.follonett.no (nic.follonett.no [194.198.43.10]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id RAA19160 for ; Tue, 29 Apr 1997 17:09:08 -0700 (PDT) Received: (from uucp@localhost) by nic.follonett.no (8.8.5/8.8.3) with UUCP id CAA22802; Wed, 30 Apr 1997 02:06:57 +0200 (MET DST) Received: from oo7 (pc136.dimaga.com [192.0.0.136]) by dimaga.com (8.7.5/8.7.2) with SMTP id CAA10268; Wed, 30 Apr 1997 02:05:58 +0200 (MET DST) Message-Id: <3.0.32.19970430010820.012946a0@dimaga.com> X-Sender: eivind@dimaga.com (Unverified) X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Wed, 30 Apr 1997 01:08:22 +0200 To: Warner Losh From: Eivind Eklund Subject: Re: First hunk of diffs Cc: auditors@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 04:51 PM 4/29/97 -0600, Warner Losh wrote: > >I've been running with these diffs now for some time. These seem >sane. > >Comments? Some. What I'm not commenting looked OK. >Index: libc/stdtime/localtime.c >=================================================================== >RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/stdtime/localtime.c,v >retrieving revision 1.15 >diff -u -r1.15 localtime.c >--- localtime.c 1997/03/25 05:34:31 1.15 >+++ localtime.c 1997/04/29 14:35:28 >@@ -273,6 +273,11 @@ > register int i; > register int fid; > >+ /* XXX The following is from OpenBSD, and I'm not sure it is correct */ >+ if (name != NULL && issetugid() != 0) >+ if ((name[0] == ':' && name[1] == '/') || >+ name[0] == '/' || strchr(name, '.')) >+ name = NULL; > if (name == NULL && (name = TZDEFAULT) == NULL) > return -1; > { Why should we deny . in names? Shouldn't that probably be a check for .. (or even better ../)? >@@ -293,7 +298,7 @@ > if (!doaccess) { > if ((p = TZDIR) == NULL) > return -1; >- if ((strlen(p) + strlen(name) + 1) >= sizeof fullname) >+ if ((strlen(p) + 1 + strlen(name) + 1) >= sizeof fullname) > return -1; > (void) strcpy(fullname, p); > (void) strcat(fullname, "/"); This change is bogus. It is OK to copy the exact number of bytes into a buffer, which is a case which is blocked by the change. (Yeah, the original is in a strange style.) >@@ -306,7 +311,7 @@ > name = fullname; > } > if (doaccess && access(name, R_OK) != 0) >- return -1; >+ return -1; > if ((fid = open(name, OPEN_MODE)) == -1) > return -1; > if ((fstat(fid, &stab) < 0) || !S_ISREG(stab.st_mode)) Whitespace change. Please don't commit. >Index: libftpio/ftpio.c >=================================================================== >RCS file: /home/imp/FreeBSD/CVS/src/lib/libftpio/ftpio.c,v >retrieving revision 1.25 >diff -u -r1.25 ftpio.c >--- ftpio.c 1997/02/22 15:06:50 1.25 >+++ ftpio.c 1997/03/22 05:07:29 >@@ -701,7 +702,7 @@ > return FAILURE; > } > ftp->addrtype = sin.sin_family = he->h_addrtype; >- bcopy(he->h_addr, (char *)&sin.sin_addr, he->h_length); >+ bcopy(he->h_addr, (char *)&sin.sin_addr, MIN(he->h_length,sizeof(sin.sin_addr))); > } > > sin.sin_port = htons(port); When you're in there, why not change to memmove? No reason not to go with the standard when it is functionally equvalent. >Index: libskey/skeyaccess.c >=================================================================== >RCS file: /home/imp/FreeBSD/CVS/src/lib/libskey/skeyaccess.c,v >retrieving revision 1.7 >diff -u -r1.7 skeyaccess.c >--- skeyaccess.c 1996/10/17 21:49:34 1.7 >+++ skeyaccess.c 1997/03/22 05:08:11 >@@ -408,12 +408,11 @@ > > for (i = 0; i < MAX_ADDR && hp->h_addr_list[i]; i++) > memcpy((char *) &list[i], >- hp->h_addr_list[i], hp->h_length); >+ hp->h_addr_list[i], (length=MIN(hp->h_length, sizeof(struct in_addr)))); > list[i].s_addr = 0; > > strncpy(buf, hp->h_name, MAXHOSTNAMELEN); >- buf[MAXHOSTNAMELEN] = 0; >- length = hp->h_length; >+ buf[MAXHOSTNAMELEN - 1] = 0; > > /* > * Wipe addresses that appear to belong to someone else. We will get Please move the assignment out of the loop. Having it in the loop will likely slow down the loop for almost all compilers; it is _not_ candidate for compiler code motion, and will fource a read of hp->h_length. (There are non-obvious reasons for this; suffice to say it is nescessary to stay ISO C compliant. If GCC optimize it now, it is broken.) Eivind Eklund perhaps@yes.no http://maybe.yes.no/perhaps/ eivind@freebsd.org From imp@village.org Tue Apr 29 17:17:29 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (8.8.5/8.8.5) with SMTP id RAA19572; Tue, 29 Apr 1997 17:17:19 -0700 (PDT) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 1.60 #1) id 0wMN54-0002La-00; Tue, 29 Apr 1997 18:17:10 -0600 To: Eivind Eklund Subject: Re: First hunk of diffs Cc: auditors@freebsd.org In-reply-to: Your message of "Wed, 30 Apr 1997 01:08:22 +0200." <3.0.32.19970430010820.012946a0@dimaga.com> References: <3.0.32.19970430010820.012946a0@dimaga.com> Date: Tue, 29 Apr 1997 18:17:09 -0600 From: Warner Losh Message-Id: In message <3.0.32.19970430010820.012946a0@dimaga.com> Eivind Eklund writes: : >+ /* XXX The following is from OpenBSD, and I'm not sure it is correct */ : : Why should we deny . in names? Shouldn't that probably be a check for .. : (or even better ../)? I think this was a quick hack to see if it contained '..'. That's why I had the comment on. : >@@ -293,7 +298,7 @@ : > if (!doaccess) { : > if ((p = TZDIR) == NULL) : > return -1; : >- if ((strlen(p) + strlen(name) + 1) >= sizeof fullname) : >+ if ((strlen(p) + 1 + strlen(name) + 1) >= sizeof fullname) : > return -1; : > (void) strcpy(fullname, p); : > (void) strcat(fullname, "/"); : : This change is bogus. It is OK to copy the exact number of bytes into a : buffer, which is a case which is blocked by the change. (Yeah, the : original is in a strange style.) OK. That's easy enough to not do :-). : >@@ -306,7 +311,7 @@ : > name = fullname; : > } : > if (doaccess && access(name, R_OK) != 0) : >- return -1; : >+ return -1; : > if ((fid = open(name, OPEN_MODE)) == -1) : > return -1; : > if ((fstat(fid, &stab) < 0) || !S_ISREG(stab.st_mode)) : : Whitespace change. Please don't commit. OK. : >Index: libftpio/ftpio.c : >=================================================================== : >RCS file: /home/imp/FreeBSD/CVS/src/lib/libftpio/ftpio.c,v : >retrieving revision 1.25 : >diff -u -r1.25 ftpio.c : >--- ftpio.c 1997/02/22 15:06:50 1.25 : >+++ ftpio.c 1997/03/22 05:07:29 : >@@ -701,7 +702,7 @@ : > return FAILURE; : > } : > ftp->addrtype = sin.sin_family = he->h_addrtype; : >- bcopy(he->h_addr, (char *)&sin.sin_addr, he->h_length); : >+ bcopy(he->h_addr, (char *)&sin.sin_addr, : MIN(he->h_length,sizeof(sin.sin_addr))); : > } : > : > sin.sin_port = htons(port); : : When you're in there, why not change to memmove? No reason not to go with : the standard when it is functionally equvalent. I was wanting to make a minimal change. memcpy would be the right routine. : >Index: libskey/skeyaccess.c : >=================================================================== : >RCS file: /home/imp/FreeBSD/CVS/src/lib/libskey/skeyaccess.c,v : >retrieving revision 1.7 : >diff -u -r1.7 skeyaccess.c : >--- skeyaccess.c 1996/10/17 21:49:34 1.7 : >+++ skeyaccess.c 1997/03/22 05:08:11 : >@@ -408,12 +408,11 @@ : > : > for (i = 0; i < MAX_ADDR && hp->h_addr_list[i]; i++) : > memcpy((char *) &list[i], : >- hp->h_addr_list[i], hp->h_length); : >+ hp->h_addr_list[i], (length=MIN(hp->h_length, sizeof(struct : in_addr)))); : > list[i].s_addr = 0; : > : > strncpy(buf, hp->h_name, MAXHOSTNAMELEN); : >- buf[MAXHOSTNAMELEN] = 0; : >- length = hp->h_length; : >+ buf[MAXHOSTNAMELEN - 1] = 0; : > : > /* : > * Wipe addresses that appear to belong to someone else. We will get : : Please move the assignment out of the loop. Having it in the loop will : likely slow down the loop for almost all compilers; it is _not_ candidate : for compiler code motion, and will fource a read of hp->h_length. (There : are non-obvious reasons for this; suffice to say it is nescessary to stay : ISO C compliant. If GCC optimize it now, it is broken.) OK. That's easy enough. Warner From guido@gvr.win.tue.nl Mon May 5 12:07:47 1997 Return-Path: Received: from gvr.win.tue.nl (root@gvr.win.tue.nl [131.155.210.19]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id MAA12775 for ; Mon, 5 May 1997 12:07:40 -0700 (PDT) Received: (from guido@localhost) by gvr.win.tue.nl (8.8.5/8.8.2) id VAA01385; Mon, 5 May 1997 21:07:19 +0200 (MET DST) From: Guido van Rooij Message-Id: <199705051907.VAA01385@gvr.win.tue.nl> Subject: Re: First hunk of diffs In-Reply-To: from Warner Losh at "Apr 29, 97 04:51:28 pm" To: imp@village.org (Warner Losh) Date: Mon, 5 May 1997 21:07:18 +0200 (MET DST) Cc: auditors@freebsd.org X-Mailer: ELM [version 2.4ME+ PL28 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit > Index: libc/net/res_init.c > =================================================================== > RCS file: /home/imp/FreeBSD/CVS/src/lib/libc/net/res_init.c,v > retrieving revision 1.12 > diff -u -r1.12 res_init.c > --- res_init.c 1997/02/22 15:00:32 1.12 > +++ res_init.c 1997/04/28 20:15:17 > @@ -177,8 +177,9 @@ > _res.pfcode = 0; > > /* Allow user to override the local domain definition */ > - if ((cp = getenv("LOCALDOMAIN")) != NULL) { > + if (issetugid() == 0 && (cp = getenv("LOCALDOMAIN")) != NULL) { > (void)strncpy(_res.defdname, cp, sizeof(_res.defdname) - 1); > + _res.defdname[sizeof(_res.defdname) - 1] = '\0'; > haveenv++; > > /* > @@ -231,6 +232,7 @@ > if ((*cp == '\0') || (*cp == '\n')) > continue; > strncpy(_res.defdname, cp, sizeof(_res.defdname) - 1); > + _res.defdname[sizeof(_res.defdname) - 1] = '\0'; > if ((cp = strpbrk(_res.defdname, " \t\n")) != NULL) > *cp = '\0'; > havesearch = 0; > @@ -246,6 +248,7 @@ > if ((*cp == '\0') || (*cp == '\n')) > continue; > strncpy(_res.defdname, cp, sizeof(_res.defdname) - 1); > + _res.defdname[sizeof(_res.defdname) - 1] = '\0'; > if ((cp = strchr(_res.defdname, '\n')) != NULL) > *cp = '\0'; > /* > @@ -379,7 +382,9 @@ > #endif /* !RFC1535 */ > } > > - if ((cp = getenv("RES_OPTIONS")) != NULL) > + if (issetugid()) > + _res.options |= RES_NOALIASES; > + else if ((cp = getenv("RES_OPTIONS")) != NULL) > res_setoptions(cp, "env"); > _res.options |= RES_INIT; > return (0); Probably, the OpenBSD folks already did this, but it is wise to submit resolver stuf back to Paul Vixie (or more correct: the ISC). -Guido From imp@village.org Mon May 5 12:34:34 1997 Return-Path: Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (8.8.5/8.8.5) with SMTP id MAA14334 for ; Mon, 5 May 1997 12:34:27 -0700 (PDT) Received: from rover.village.org [127.0.0.1] by rover.village.org with esmtp (Exim 1.60 #1) id 0wOTWC-0002Zb-00; Mon, 5 May 1997 13:33:52 -0600 To: Guido van Rooij Subject: Re: First hunk of diffs Cc: auditors@freebsd.org In-reply-to: Your message of "Mon, 05 May 1997 21:07:18 +0200." <199705051907.VAA01385@gvr.win.tue.nl> References: <199705051907.VAA01385@gvr.win.tue.nl> Date: Mon, 05 May 1997 13:33:52 -0600 From: Warner Losh Message-Id: In message <199705051907.VAA01385@gvr.win.tue.nl> Guido van Rooij writes: : Probably, the OpenBSD folks already did this, but it is wise to : submit resolver stuf back to Paul Vixie (or more correct: the ISC). OK. Ill do that when I put them into the tree. Any other comments? Warner From mazal@netvision.net.il Tue Sep 1 13:38:58 1998 Return-Path: Received: from P1 ([192.117.232.9]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id NAA19571 for ; Tue, 1 Sep 1998 13:38:51 -0700 (PDT) (envelope-from mazal@netvision.net.il) From: mazal@netvision.net.il Received: from default - 192.117.233.58 by isdn.net.il with Microsoft SMTPSVC; Tue, 1 Sep 1998 23:22:51 +0300 Date: Tue, 01 Sep 1998 23:25:23 +0300 Sender: mazal@netvision.net.il Subject: US$ 120.000.000 de Dolares To: auditors@freebsd.org X-Mailer: Message-ID: <0a8eb5122200198P1@isdn.net.il> Estimado navegante: US$ 120.000.000 lo esperan en: http://209.75.79.87/mazal y un tentador regalo para el espiritu en: http://209.75.79.87/tierra Nunca su bolsillo y su espiritu estuvieron tan cerca de enriquecerse. Gracias por su atencion.- R.M. Srul Mazal Israel Ltd mazal7@aquanet.co.il From print2live@angelfire.com Sat Mar 20 19:03:36 1999 Delivered-To: auditors@freebsd.org Received: from server2525 (ip157.moorestown4.nj.pub-ip.psi.net [38.26.85.157]) by hub.freebsd.org (Postfix) with SMTP id 427BA15246 for ; Sat, 20 Mar 1999 19:03:27 -0800 (PST) (envelope-from print2live@angelfire.com) To: From: Subject: AD:Family Reunion T Shirts & More Date: Sat, 20 Mar 1999 18:28:17 Return-Path: Message-Id: <19990321030329.427BA15246@hub.freebsd.org> Message sent by: Kuppler Graphics, 32 West Main Street, Maple Shade, New Jersey, 08052, 1-800-810-4330. This list will NOT be sold. All addresses are automatically added to our remove list. Hello. My name is Bill from Kuppler Graphics. We do screenprinting on T Shirts, Sweatshirts, Jackets, Hats, Tote Bags and more! Do you or someone you know have a Family Reunion coming up? Kuppler Graphics would like to provide you with some great looking T Shirts for your Reunion. Kuppler Graphics can also provide you with custom T's and promotional items such as imprinted magnets, keychains, pens, mugs, hats, etc. for your business or any fundraising activity (church, school, business etc.) We also can provide you with quality embroidery. We are a family owned company with over 15 years of experience. All work is done at this location. No middle man. Our prices are great! Click reply to email us or call 1-800-810-4330 for more info Bill Kuppler Graphics From remailer@replay.com Sun Mar 28 16:58:30 1999 Return-Path: Delivered-To: auditors@freebsd.org Received: from unknown (1Cust90.tnt3.atl2.da.uu.net [153.36.17.90]) by hub.freebsd.org (Postfix) with SMTP id 4120115588; Sun, 28 Mar 1999 16:58:24 -0800 (PST) (envelope-from remailer@replay.com) From: Subject: toner sales advertising Date: Mon, 29 Mar 1999 04:33:14 Message-Id: <19990329005825.4120115588@hub.freebsd.org> BENCHMARK PRINT SUPPLY LASER PRINTER CARTRIDGES JUST FOR YOU AS WELL AS FAX AND COPIER TONER CHECK OUT THESE NEW PRINT CARTRIDGE PRICES : APPLE LASER WRITER PRO 600 OR 16/600 $69 LASER WRITER SELECT 300,310.360 $64 LASER WRITER 300, 320, 360 $54 LASER WRITER LS,NT,2NTX,2F,2G & 2SC $54 LASER WRITER 12/640 $79 HEWLETT PACKARD LASERJET SERIES 2,3 & 3D $49 LASERJET SERIES 2P AND 3P $54 LASERJET SERIES 3SI AND 4SI $75 LASERJET SERIES 4L AND 4P $49 LASERJET SERIES 4, 4M, 5, 5M $59 LASERJET SERIES 4000 HIGH YIELD $99 LASERJET SERIES 4V $95 LASERJET SERIES 5SI , 8000 $95 LASERJET SERIES 5L AND 6L $49 LASERJET SERIES 5P, 5MP, 6P, 6MP $59 LASERJET SERIES 5000 $89 HP LASERFAX LASERFAX 500, 700, FX1, $59 LASERFAX 5000, 7000, FX2, $59 LASERFAX FX3 $69 LASERFAX FX4 $79 LEXMARK OPTRA 4019, 4029 HIGH YIELD $135 OPTRA R, 4039, 4049 HIGH YIELD $135 OPTRA S 4059 HIGH YIELD $135 OPTRA E $59 OPTRA N $115 EPSON EPL-70000, 8000 $105 EPL-1000, 1500 $105 CANON LBP-430 $49 LBP-460, 465 $59 LBP-8 II $54 LBP-LX $54 LBP-MX $95 LBP-AX $49 LBP-EX $59 LBP-SX $49 LBP-BX $95 LBP-PX $49 LBP-WX $95 LBP-VX $59 CANON FAX L700 THRU L790 FX1 $59 CANONFAX L5000 L70000 FX2 $59 CANON COPIERS PC 20, 25 ETC.... $89 PC 3, 6RE, 7, 11 (A30) $69 PC 320 THRU 700 (E40) $89 NEC SERIES 2 LASER MODEL 95 $105 PLACE YOUR ORDER BY : PHONE 770-399-0953 FAX: 770-698-9700 E-MAIL: BENCHMARK1@USA.NET WITH SUBJECTLINE: ORDER MAIL: 1091 REDSTONE LANE, ATLANTA GA 30338 MAKE SURE YOU INCLUDE THE FOLLOWING INFORMATION IN YOUR ORDER: 1) PHONE NUMBER 2) COMPANY NAME 3) SHIPPING ADDRESS 4) YOUR NAME 5) ITEMS NEEDED WITH QUANTITIES 6) METHOD OF PAYMENT. (COD OR CREDIT CARD) 7) CREDIT CARD NUMBER WITH EXPIRATION DATE WE SHIP UPS GROUND STANDARD. ADD $4.5 FOR SHIPPING AND HANDLING. FOR ORDER QUESTIONS CALL 770-399-0953 FOR CUSTOMER SERVICE 770-399-5505 FOR E-MAIL REMOVAL USE: BENCHMARK2@USA.NET OR CALL 770-399-5614 From workshop@npcollege-edu.net Sun Jul 4 16:01:42 1999 Return-Path: Delivered-To: auditors@freebsd.org Received: from npcollege-edu.net (mail.npcollege-edu.net [209.232.226.194]) by hub.freebsd.org (Postfix) with SMTP id 28FA7151AC; Sun, 4 Jul 1999 16:01:37 -0700 (PDT) (envelope-from workshop@npcollege-edu.net) From: Subject: Re: Deleting Your Address. Date: Sun, 4 Jul 1999 12:36:36 Message-Id: <631.208714.538392@npcollege-edu.net> From:The SFSE(Scientific Facts Search Engine). When you send an email to EDU,R&D,or Job sites,your email might be forwarded via the SFSE to find the info you are looking for. The NU(NewAmerica University)has received a copy of your email,but the date is Feb/25/99.Do you still need this info? To refresh your memory you can go again to the NU website and look for these topics: "The Redeemat has solved all environmental problems"-"The 9% Producer Fee would eliminate crime & taxes within 3 years"-"The free NHRI (National Health & Retirement Insurance")- "The job guarantee system(JIC/Job Insurance Corporation)"and "The list of 22nd Centurys' products & businesses available right now". Or,request for the records of the SFSE search by putting in the subject REQUESTING INFO and click REPLY. Or,in the future send "REQUEST FOR SEARCH" (on any topic) directly to SFSE, and they will do the search for you,put in the subject SFSE. Or,please allow us to DELETE your address permanently by simply clicking REPLY. B.Morrison workshop@npcollege-edu.net From info@resi.net Mon Jul 12 08:54:04 1999 Return-Path: Delivered-To: auditors@freebsd.org Received: from phnxpop3.phnx.uswest.net (phnxpop3.phnx.uswest.net [206.80.192.3]) by hub.freebsd.org (Postfix) with SMTP id DE58515227 for ; Mon, 12 Jul 1999 08:53:59 -0700 (PDT) (envelope-from info@resi.net) Received: (qmail 6678 invoked by alias); 12 Jul 1999 15:52:39 -0000 Delivered-To: fixup-auditors@freebsd.org@fixme Received: (qmail 6640 invoked by uid 0); 12 Jul 1999 15:52:38 -0000 Received: from jdialup120.phnx.uswest.net (HELO mail.phnx.uswest.net) (209.180.135.120) by phnxpop3.phnx.uswest.net with SMTP; 12 Jul 1999 15:52:38 -0000 From: info@resi.net Reply-To: info@resi.net To: auditors@freebsd.org Subject: Richardson Cost Estimating Books, Seminars & Software Message-Id: <19990712155359.DE58515227@hub.freebsd.org> Date: Mon, 12 Jul 1999 08:53:59 -0700 (PDT) Hi, Since 1960 Richardson Engineering has published construction cost estimating books with labor hours, material prices and subcontractor costs. Our books are now available on CD-ROM and have thousands of illustrations, modifiers, technical information and manufacturer specifications. Enter our 40th Birthday drawing and get a FREE (we even pay shipping charges!) evaluation copy CD-ROM of our books and software. Prices start at $187.00 plus shipping. Interested in looking at our estimating books and software? Go to www.resi.net for downloads and ordering information. Thanks, Richardson Engineering Tel: 480.497.2062 Fax: 480.497.5529 www.resi.net sales@resi.net info@resi.net If this message reached you by error we sincerely apologize. If you wish to be removed from future mailings, please reply with the subject "Remove" and this software will automatically block you from future mailings. From searches@freeola.com Fri Oct 15 12:37:09 1999 Return-Path: Delivered-To: auditors@freebsd.org Received: from mailhub5.libertysurf.fr (mailhub5.libertysurf.fr [195.154.209.25]) by hub.freebsd.org (Postfix) with ESMTP id 1897214D7F for ; Fri, 15 Oct 1999 12:37:07 -0700 (PDT) (envelope-from searches@freeola.com) Received: from mail.libertysurf.fr (ppp170-toulon.libertysurf.fr [213.36.56.170]) by mailhub5.libertysurf.fr (8.9.3/8.9.3) with SMTP id VAA20719; Fri, 15 Oct 1999 21:34:11 +0200 (CEST) From: searches@freeola.com Message-Id: <199910151934.VAA20719@mailhub5.libertysurf.fr> To: Friend@public.com Date: Fri, 15 Oct 99 20:52:42 EST Subject: Your Web Page Increase your Web Site visiblity ! Try out : *** Proven Web Page Optimisation Software *** >> Audited results 55% Number 1 90% top 5 << *** Macro Media Web Page Design Software **** >>> Make your Web Pages sparkle <<<< **** Accept checks from your Web Site ***** >>>> Improve your margins & Cashflow <<<< Go to : http://www.freebizlinks.net