Make your own free website on Tripod.com
 
 

The Solaris Security FAQ

Answer all your Solaris Security questions here


 

 


Abstract
The job of tracking all the security information surrounding Solaris 2 is a difficult one. There's general information about securing Solaris, patches to know about, tools to use, many sources of security information, and specific needs if you're trying to secure a Solaris web server. Well your job has just been made much easier. The Solaris Security FAQ has all this and more.

The following is a list of questions that are frequently asked about Solaris 2.x Security. You can help make it an even better-quality FAQ by writing a a contribution or update and sending it BY EMAIL ONLY to security-faq@sunworld.com.

Changes to this document will be indicated in the index by a "+" for new entries and a "*" for changed entries.

1. GENERAL

  • 1.1) How secure is solaris 2?
  • 1.2) What version of Solaris should I run?
  • 1.3) Can I just install a machine and ignore it?
  • 2. How can I configure Solaris to make it more secure?

  • 2.1) What file permissions should I change?
  • 2.2) How should I change root user configuration?
  • 2.3) How should I change startup files?
  • 2.4) How can I disable network root logins?
  • 2.5) How do I disable rlogin/rsh access?
  • 2.6) What accounts are unnecessary?
  • 2.7) How do I protect devices?
  • 2.8) What permissions should I change in /etc?
  • 2.9) Why do Solaris machines act as routers?
  • 2.10) How do I disable automounter?
  • 2.11) How to I disable NFS service?
  • 2.12) Do I need to worry about cron jobs?
  • 2.13) Are there any risks to using dynamic routes?
  • 2.14) When and how should I use static ARP?
  • 2.15) Is it unsecure to run rpcbind?
  • 2.16) What permission bits should be set on /etc/utmp?
  • 2.17) What programs can be un-suid'ed?
  • 2.18) What system facilities can I disable?
  • 2.19) Should I run in.fingerd?
  • 2.20) Can syslog be made to be more effective?
  • 2.21) How can the EEPROM make a system more secure?
  • 2.22) Is my machine being "promiscuous"?
  • 2.23) If I need to use NFS, how can I make it more secure ?
  • 2.24) How can I secure sendmail
  • 2.25) Is NIS secure, and how can it be made more secure?
  • 2.26) What is needed for secure anonymous ftp service?
  • 2.27) How can X be made more secure?
  • 2.28) How do I turn on SUN-DES-1 authentication?
  • 2.29) What patches should I install?
  • 3. What programs should I replace or add?

  • 3.1) inetd?
  • 3.2) ifstatus?
  • 3.3) xntp
  • 3.4) sendmail?
  • 3.5) rpcbind?
  • 3.6) Password checking programs?
  • 3.7) crack?
  • 3.8) ftp?
  • 3.9) fix_modes?
  • 3.10) noshell?
  • 3.11) bind?
  • 3.12) netcat?
  • 3.13) tiger?
  • 3.14) SSH?
  • + 3.15) stelnet?
  • 4. What other useful resources should I know about?

  • 4.1) Sun mailing-lists?
  • 4.2) Sun patches?
  • 4.3) Other Solaris FAQs?
  • 4.4) Useful newsgroups?
  • 4.5) Useful mailing-lists?
  • 4.6) Useful columns?
  • * 4.7) Useful web sites?
  • 4.8) Useful RFCs?
  • 5. How can I make my Solaris web server more secure?

  • 5.0) Overview
  • 5.1) Step 0 - Web server security checklist
  • 5.1) Step 1 - Hardware Setup
  • 5.2) Step 2 - Install the OS
  • 5.3) Step 3 - Strip down the OS
  • 5.4) Step 4 - Install third-party software
  • 5.5) Step 5 - Limit network access to the system
  • 5.6) Step 6 - Configure S/Key
  • 5.7) Step 7 - Configure wu-ftp
  • 5.8) Step 8 - Limit access to files and file systems
  • 5.9) Step 9 - Test the configuration
  • 5.10) Step 10 - Other suggestions
  • + 6. How can I detect a break-in?

  • 6.1) Caveats
  • 6.2) Other resources
  • 6.3) Determine current system state
  • 6.4) Other resources
  • 6.5) Patch Failure
  • 6.6) Binary modifications
  • 6.7) Enlist the users
  • 6.8) What about the applications?
  • 6.9) Use the web, Luke
  • 7. ACKNOWLEDGEMENTS

    1) General

    1.1) How secure is Solaris 2?

    Solaris 2 is relatively secure, considering that it is a general-purpose, time-sharing, multi-user operating system. Such systems are inherently full of compromises. Solaris 2 is a version of Unix, which was not designed for security. However, there are few known security holes in Solaris 2.5.1, and Sun is active in patching bugs found in the system. Additionally, there are facilities that can increase the security of Solaris (see section 3).

    1.2) What version of Solaris should I run?

    Where security is concerned? Each subsequent release of Solaris has been an improvement over its predecessor. Solaris 2.5.1 is currently the latest release, and also the most secure.

    1.3) Can I just install a machine and ignore it?

    Most installed machines suffer from entropy: They lack a current OS release, and up-to-date patches and tools. It's important to install the latest patches, at the least, to be sure that all known security holes are filled.

    2) How can I configure Solaris to make it more secure?

    2.1) What file permissions should I change?

    The program fix-modes runs on Solaris 2.4 and 2.5 and changes system file and directory permissions. The new permissions make it harder for non-root users to become root, and for non-root users to modify system files.

    2.2) How should I change root user configuration?

    Be sure root has a umask setting of 077 or 027.

    Be sure root has a safe search path, as in /usr/bin:/sbin:/usr/sbin

    2.3) How should I change startup files?

    Generally, examine all "S" files in /etc/rc2.d and /etc/rc3.d. Any files that start unneeded facilities should be renamed (be sure the new names don't start with "S"). Test all boot files changes by rebooting, examining /var/adm/messages, and checking for extraneous processes in ps -elf output.

    2.4) How can I disable network root logins?

    Make sure the to enable the "CONSOLE" line in /etc/default/login. To disable use of ftp by root, add "root" to /etc/ftpusers.

    2.5) How do I disable rlogin/rsh access?

    Remove /etc/hosts.equiv, /.rhosts, and all of the "r" commands from /etc/inetd.conf Do a kill -HUP of the inetd process.

    2.6) What accounts are unnecessary?

    Remove, lock, or comment out unnecessary accounts, including "sys", "uucp", "nuucp", and "listen". The cleanest way to shut them down is to put "NP" in the password field of the /etc/shadow file. Also consider using the noshell program to log attempts to use secured accounts.

    2.7) How to I protect devices?

    The file /etc/logindevperm contains configuration information to tell the system the permissions to set on devices associated with login (console, keyboard, etc). Check the values in this file and modify them to give different permissions.

    2.8) What permissions should I change in /etc?

    No file in /etc needs to be group writeable. Remove group write permission via the command chmod -R g-w /etc

    2.9) Why do Solaris machines act as routers?

    By default, if a Solaris machine has more than one network interface, Solaris will route packets between the multiple interfaces. This behavior is controlled by /etc/init.d/inetinit. To turn of routing on a Solaris 2.4 (or lesser) machine, add ndd -set /dev/ip ip_forwarding 0 at the end of /etc/init.d/inetinit. For Solaris 2.5, simply touch /etc/notrouter. Be aware that there is a small window of vulnerability during startup when the machine may route, before the routing is turned off.

    2.10) How do I disable automounter?

    Automounter is controlled by the /etc/auto_* configuration files. To disable automounter, remove those files, and/or disable the /etc/rc2.d/S74autofs.

    2.11) How to I disable NFS service?

    NFS exports are controlled by the /etc/dfs/dfstab file. Remove this file. To disable the NFS server daemon, rename /etc/rc3.d/S15nfs.server. To prevent a machine from being an NFS client, rename /etc/rc2.d/S73nfs.client. When renaming startup files, be sure to name them with a starting letter other than "S".

    2.12) Do I need to worry about cron jobs?

    Review all the cron jobs by reading the cron file of every system account in /var/spool/cron/crontabs. Consider logging all cron activities by setting "CRONLOG=yes" in /etc/default/cron.

    2.13) Are there any risks to using dynamic routes?

    Machines using a dynamic route-receiving daemon like in.routed and in.rdisc are vulnerable to receiving incorrect routes. These routes can disable some or all connectivity to other networks. When possible, use static routes (routes added via the route commands in startup files, rather than the routing daemons.

    2.14) When and how should I use static ARP?

    ARP is the protocol used to associate IP and Ethernet addresses. Machines that share a wire (and have no routers between them) know each others ARP addresses. If one machine is replaced with another, the ARP addresses are usually different. By default, Solaris machines dynamically determine ARP addresses. The arp command can be used to statically set ARP table entries and flush all other entries. This facility is best used when there are few, unchanging systems on a network and the machines need to be assured of each other's identities.

    2.15) Is it unsecure to run rpcbind?

    rpcbind is the program that allows rpc callers and rpc service provides to find each other. Unfortunately, standard rpc is unsecure. It uses "AUTH_UNIX" authentication, which means it depends on the remote system's IP address and the remote user's UID for identification. Both of these forms of identification can be easily forged or changed. Generally-purpose systems usually need rpc running to keep users happy. Special purpose systems (web servers, ftp servers, mail servers, etc) can usually have rpc disabled. Be sure to test all the facilities that you depend on to be sure they aren't affected if you turn off rpc. To disable rpc, rename /etc/rc2.d/S71RPC.

    2.16) What permission bits should be set on /etc/utmp?

    /etc/utmp can be set to mode 644 without disrupting any service.

    2.17) What programs can be un-suid'ed?

    Many of the setuid and setgid programs on Solaris are used only by root, or by the user or group-id to which they are set. They can have setuid and setgid removed without diminishing user's abilities to get their work done. Consider each of these programs individually as to their use on your system. Should they be run by someone other than root, their owner, or a user (running with that user's UID)?

    The command to find all set-uid programs in the system is:

    # find / -perm -4000

    /usr/bin/tip                 /usr/bin/ct           /usr/bin/cu     
    /usr/bin/uuglist             /usr/bin/uuname       /usr/bin/uustat
    /usr/lib/exrecover           /usr/bin/uux          /usr/lib/accton 
    /usr/lib/fs/ufs/ufsrestore   /usr/bin/uucp         /usr/lib/news/inews
    /usr/lib/fs/ufs/ufsdump      /usr/lib/uucp/uuxqt   /usr/lib/uucp/uucico  
    /usr/lib/uucp/remote.unknown /usr/lib/uucp/uusched /usr/sbin/allocate
    Likewise, obtain a list of set-gid files on your system via the command:

    find / -perm -2000 -print

    and remove the set-gid bit from appropriate files, including some of these:

    /usr/bin/mailx             /usr/bin/netstat          /usr/bin/nfsstat
    /usr/bin/write             /usr/bin/ipcs             /usr/lib/fs/ufs/ufsdump  
    /usr/sbin/arp              /usr/sbin/prtconf         /usr/bin/swap   
    /usr/sbin/sysdef           /usr/sbin/wall            /usr/sbin/dmesg
    /usr/openwin/bin/wsinfo    /usr/openwin/bin/ff.core  /usr/kvm/crash   
    /usr/openwin/bin/mailtool  /usr/openwin/bin/xload    /usr/kvm/eeprom 
    /usr/vmsys/bin/chkperm
    Create a master list of the remaining setuid/setgid programs on your system and check that the list remains static over time.

    2.18) What system facilities can I disable?

    Every network on the system should be inspected to determine if the facility that it provides is appropriate for your environment. If not, disable the facility. Some of these facilities are in the system startup files, as discussed in section 2. Other are started in /etc/inetd.conf. Comment out the unneeded facilities and kill -HUP the inetd daemon. Some common facilities are:

    tftp         systat         rexd        ypupdated       netstat
    rstatd       rusersd        sprayd      walld           exec
    comsat       rquotad        name        uucp
    For a very secure system, replace the standard inetd.conf with one that just includes telnet and ftp (if you need those facilities).

    2.19) Should I run in.fingerd?

    in.fingerd has had some security problems in the past. If you want to provide the finger facility, run it as "nobody", not as "root".

    2.20) Can syslog be made to be more effective?

    By default, syslog provides minimal system logging. Modify the /etc/syslog.conf file to have syslog log more information, and separate to where the information is logged by importance. Anything related to security should be sent to a file that gets encrypted. Unfortunately, syslog must be restarted for it to read the new configuration file.

    2.21) How can the EEPROM make a system more secure?

    Set the EEPROM to "security=command" password-protect all EEPROM commands except "boot" and continue". Set the EEPROM's password so no one else can change its modes. Unfortunately, this doesn't truly secure the machine. If someone has physical access to the machine, they can open the machine and replace its EEPROM. Replacing the machine's EEPROM also changes its hostid. Recording all the hostids of your machines and checking this list against the machines occasionally to verify that no EEPROMs have been replaced.

    2.22) Is my machine being "promiscuous"?

    Under Solaris, there is no way to determine if a machine's network interfaces are in "promiscuous" mode. Promiscuous mode allows the machine to see all network packets, rather than just those packets destined for the machine. This allows the machine to snoop the network and monitor all traffic. An interface should only be in promiscuous mode if the snoop program, or another network monitor program, is being run. If you aren't running such a program, and your machine's interface is in promiscuous mode, then it's likely that a hacker is monitoring your network. The public domain ifstatus command returns a machine's promiscuous state. (See section 3.)

    2.23) If I need to use NFS, how can I make it more secure ?

    2.24) How can I secure sendmail

    With Solaris 2.5, Sun is shipping a much more modern sendmail. Still, there are new bugs reported monthly. How can sendmail be made more secure?

    2.25) Is NIS secure, and how can it be made more secure?

    NIS is not a secure distributed name service. NIS+ is more secure when configured properly. NIS will give away all the information in its tables if its domain name is guessable. To close this hole, put trusted host/net addresses to /var/yp/securenets. Also consider using secure RPC or NIS+. Finally, don't include root and other system account information in NIS tables.

    2.26) What is needed for secure anonymous ftp service?

    Solaris 2.5 ftpd(1M) contains a good set of configuration directions, with the following exceptions:

    2.27) How can X be made more secure? 2.28) How do I turn on SUN-DES-1 authentication? Now, to give user "foo" permission to access host "node": 2.29) What patches should I install?

    Use showrev -p to list patches installed on the system. Check Sun's patch list (section 4) for current security-related patches for the version you are running. Download and install all pertinent security patches. Recheck the patch list frequently. Not all security patches need be installed on every machine. But protect machines, or those with public access, should be kept up-to-date.

    3) What programs should I replace or add?

    3.1) inetd?

    inetd can be replaced with ftp://qiclab.scn.rain.com/pub/security/xinetd* to add logging facilities.

    3.2) ifstatus?

    ifstatus can determine if your network interfaces are in promiscuous mode.

    3.3) xntp?

    xntp is a more secure version of ntp, the network time protocol.

    3.4) sendmail?

    The most recent (and usually most secure) version of sendmail is always available from Berkeley. Included in the sendmail package is smrsh, the "sendmail restricted shell" which can be used to control any programs invoked by sendmail.

    3.5) rpcbind?

    rpcbind can be used to replace the standard rpcbind on Solaris machines. This version includes tcpwrapper-like functionality and disables access to NFS through rpcbind.

    3.6) Password checking programs?

    Unfortunately, passwd+ and npasswd are not yet released on Solaris. They are replacements for passwd that disallow "stupid" passwords from being used on Unix systems.

    3.7) crack?

    The crack program can be used to break "guessable" passwords in your /etc/shadow file. It uses a lot of compute cycles, but will generally tell you the passwords of 10% of your accounts the first time it is run.

    3.8) ftp?

    wu-ftp is a replacement for the standard ftpd daemon. It provides extensive logging and access control.

    3.9) fix_modes?

    fix-modes is a shell script that makes extensive changes to the file and directory permissions on standard Solaris machines.

    3.10) noshell?

    noshell is a program that can be used as the shell on accounts that are never supposed to be logged into. It logs the event (and prevents the login).

    3.11) bind?

    The standard bind on Solaris has known security problems (See CERT in section 4). bind fixes those problems. Also be sure to get patch1. There are also patches available from Sun to bring your system to 4.9.3. Version 4.9.4pl fixes even more security problems and should be included with Solaris 2.6.

    3.12) netcat?

    NetCat is a tool that is useful for security managers and crackers alike. It is a flexible network connection creator, allowing for probes of arbitrary ports between arbitrary systems.

    3.13) tiger?

    tiger is a tool that scans an individual machine for security problems including: bad root paths, bad file and directory permissions, bad daemons, and other misconfiguration issues.

    3.14) SSH?

    SSH (Secure Shell) is an enhance versions of rlogin, rsh and rcp that provides RSA authentication and encryption of communications as well as many other security improvements. Previous versions have had security holes, so use with caution. This program has export restrictions for U.S., France, Russia and possibly other countries.

    3.15) stelnet?

    If you are interested in secure telnet sessions, take a look at stelnet. It's similar to ssh and SSLTelnet, but claims to be based on a more recent telnet code base. It uses SSL to create an authenticated, encrypted telnet channel.

    4) What other useful resources should I know about?

    4.1) Sun mailing-list?

    Sun provides a security bulletin service via an email list to subscribe send email to "security-alert@sun.com" with a subject of “subscribe CWS your-mail-address”. Old bulletins are available at ftp://ftp.uu.net/systems/sun/sun-dist. Phone: (415) 688-9081

    4.2) Sun patches?

    Sun security patches are available in two locations: http://sunsolve1.sun.com (for people with Sun contracts) ftp:// ftp.uu.net/systems/sun/sun-dist (for people without Sun contracts) Or for an http interface use http://sunsolve.Sun.COM/pub-cgi/patchpage.pl

    4.3) Other Solaris FAQs?

    The excellent Solaris FAQ: http://www.wins.uva.nl/pub/solaris/solaris2/

    4.4) Useful newsgroups?

    All of the USENET newsgroups with "sun" in their name :-)

    comp.security.announce

    4.5) Useful mailing-lists?

    If you want to be on one security mailing list, consider the best-of-security mailing list. Send subscription requests to best-of-security-request@suburbia.net, or for a digest form to best-of-security-d-request@suburbia.net.

    Many security problems are due to bugs in the operating system or applications. To keep up-to-date on bug happenings, subscribe to the Bugtraq mailing list. Bugtraq is a full-disclosure UNIX security mailing list, started by Scott Chasin . To subscribe to bugtraq, send mail to listserv@netspace.org containing the message body "subscribe bugtraq". An archive of the mailing list is available at http://www.geek-girl.com/bugtraq/index.html.

     

    The Florida SunFlash is a "closed" mailing list for Sun owners. It contains mostly press releases from Sun and third-party vendors. To find out about a mail point in your area, or for other information send mail to infosunflash@Sun.COM Subscription requests should be sent to sunflash-request@Sun.COM Archives are on solar.nova.edu, ftp.uu.net, sunsite.unc.edu, src.doc.ic.ac.uk and ftp.adelaide.edu.au

    The Sun Managers list is an unmoderated mailing list for emergency-only requests. Subscribe and listen for a while, and read the regularly-posted Policy statement BEFORE sending mail to it. Write to sun-managers-request@eecs.nwu.edu

    The "Sneakers" mailing list is for discussion of LEGAL evaluations and experiments in testing various Internet "firewalls" and other TCP/IP network security products.

    You can join CIAC's mailing lists by filling out a form at the CIAC Web site.

    The firewall mailing list is obviously for discussion of firewall-related issues.

    For more mailing list information, check out the Security Mailing Lists FAQ at ISS. Here you'll find an annotated list of many security-related mailing lists and directions on how to join them. Exercise prudence, however, or your mailbox will be more full than President Clinton's hair.

    4.6) Useful columns?

    Peter Galvin's Solaris security column (plug :-) at http://www.sunworld.com/security.html

    4.7) Useful web sites?

    Perhaps the best single source of information, programs, and pointers to other security sites is found at COAST.

    + For a view of what's important in security from one of the experts, check out Spaf's Hotlist

    ISS sells security scanning software, and also provides useful information on their web site. Especially useful is the Security FAQs web page.

    The Computer Security Institute publishes surveys which track usage of current technology and products by security professionals.

    The Qualix Group publishes a useful Firewall-1 FAQ

    Memco sells software to increase the security of individual and groups of machines.

    The University of Houston has its Information Security Manual online. It makes for interesting reading, especially if you're writing a policy manual for your site.

    If you're interested in web site security, you must read the WWW Security FAQ. Also of interest is the Security Issues in WWW page.

    Bellcore sells several security programs.

    Trusted Systems does security consulting and training.

    V-One sells products that provide two-way authenticated, encrypted communication.

    Here's A plethora of PGP information.

    The best all-around information on security bugs and their patches is available at CERT. You can also ftp patches.

    The NIH provides a very useful Computer Security Information site, with an especially useful list of security programs.

    Raptor sells a Proxy Gateway firewall for Unix and NT, and has some whitepapers on general firewall technology.

    The Unix Guru Universe is an all-around useful site, and has a selection of security pointers.

    CIAC is the U.S. Department of Energy's Computer Incident Advisory Capability. Established to provide computer security services to employees and contractors of the United States Department of Energy. It's also useful for the rest of us.

    RSA maintains a very useful cryptography FAQ that explains many of the confusing aspects of cryptography.

     Learn from the hackers and crackers by reading the Phrack Newsletter

     Another useful cracker site is http://www.geocities.com/Area51/5537/. It's an archive of information useful to crackers.

     A nice firewall-comparison checklist is available in free and commercial form.

    For some basic security information, term definitions, and security company ads, check out http://www.securityinfo.com/

     There are many personally-managed lists of security resources on the Internet. Patrick Oonk has put together a set of useful links that may not already be in your bookmark list.

    Bill Wall has put together some very complete links to Web security and computer resources. Check them out at http://www.txdirect.net/~wall/solution.htm and http://www.txdirect.net/~wall/compsec.htm

    4.8) Useful RFCs?

    There are several RFCs that pertain to Sun security, current and future. RFC 1597 is included in this list because sites using firewalls can and do use private Internets, and this RFC explains which IP addresses can be used to build them. All of these RFCs are available at http://mainland.ab.ca/rfc-index.html

    1244 Site Security Handbook. J.P. Holbrook, J.K. Reynolds.
         Jul-01-1991. (Format: TXT=259129 bytes) (Also FYI0008)
    
    1597 Address Allocation for Private Internets. Y. Rekhter, B.
         Moskowitz, D. Karrenberg & G. de Groot. March 1994. (Format:
         TXT=17430 bytes) (Obsoleted by BCP0005, RFC1918)
    
    1675 Security Concerns for IPng. S. Bellovin. August 1994. (Format:
         TXT=8290 bytes)
    
    1825 Security Architecture for the Internet Protocol. R. Atkinson.
         August 1995. (Format: TXT=56772 bytes)
    
    2065 Domain Name System Security Extensions. D. Eastlake, 3rd, C.
         Kaufman. January 1997. (Format: TXT=97718 bytes) (Obsoletes RFC1034,
         RFC1035)
    5) How do I make my Solaris web server more secure?

    There are several steps to installing a "secure" Web server. But what is meant by "secure"? Any machine that is publically accessible is necessarily unsecure. By putting it directly on the Internet you've already opened the front door and allowed anyone to come in and have a look. It is possible to guard the door and be sure the visitors hang out in only one room though. By installing the operating system carefully, adding tools, and adding monitoring software, your Web server can be much safer than one that uses a default installation.

    A previous version of this information was published as a two-part column by Peter Galvin and Hal Pomeranz in SunWorld Online April 1996 and SunWorld Online May 1996

    How secure does a web server have to be? The answer is, "very secure." There have been several embarrassing incidents of graffitti on very public web sites, and worse. Any number of toolkits exist that allow total amateurs to become holy terrors. The good news is that if you can beat the popular intrusion toolkits, 90 percent of the bad guys will go bother somebody else who's less secure.

    While the following techniques can make a Solaris-based system very secure, they have to be combined with a strong network architecture utilizing firewalls or filtering routers (there's no point in controlling access by IP address if crackers can send you packets with spoofed addresses). Such a strong architecture can only truly exist when combined with a comprehensive security policy covering your entire organization.

    Furthermore, no system is ever perfectly secure. The extra services (Web servers and anonymous FTP) that you run on the machine will always impact the overall security of the system. Don't be tempted to install other software not listed in these instructions. This machine is supposed to be secure, not convenient to administer.

    5.0) Web server security checklist
    Use this checklist to be sure your server is installed as securely as possible. Feel free to add to the checklist for site-specific steps, or to share your additions with us if they are general-purpose.

    Even if you only implement some of these security steps, your Web server will be much more secure than a standard server installed with a standard Unix configuration. But to be safe, go whole-hog and lock that server down with all of these tools and techniques.

    5.1) Hardware setup
    Don't connect the system to a network before it is installed and secured. Theoretically, some cracker could get in and drop back doors onto your system while you are in the process of securing it. There are documented cases of people trying to break into machines within five minutes of the machine being connected to the Internet.

    You will need a CD-ROM drive to do the install and a tape or floppy drive for loading binaries and other files you created on other machines. These instructions are appropriate for Solaris 2.5. If you use a different OS version, the security of the resulting system may vary widely.

    5.2) Install the OS
    Start with the latest, solid version of Solaris 2.X. Each version of Solaris has been more secure than its predecessor. Solaris is quite flexible and contains many conveniences that make your work easier. Unfortunately, all of this extra functionality also makes it easier for potential crackers to gain access to your system. When attempting to create a secure system, the best plan is to start with the simplest OS you can and then add packages one-by-one on an "as-needed" basis. A limited OS configuration also boots and runs faster and is less prone to crashing than a feature-rich version.

    Under the Solaris install program, the most limited version of the OS you can select is the Core SPARC installation cluster. In fact, even this cluster has too many features. This cluster is what you should first install on your secure server. After you have selected Core SPARC in the Sun installation screens, you will want to select Customize and add the Terminal Information cluster which gives you support for commonly used terminal emulators (such as xterm and others).

    One of the advantages of the Core SPARC cluster is that it requires much less disk space than most types of Solaris installs. The following partition table is appropriate for machines loaded with the Core SPARC cluster:

    s0:     /               256 megabytes
    s1:     swap            256 megabytes
    s2:     overlap
    s3:
    s4:     
    s5:     
    s6:     /local          ??? megabytes (rest of the drive)
    s7:
    /var is large to allow for keeping extra logging and auditing information. Size swap as appropriate for your hardware, but extra swap helps prevent "denial of service" type attacks.

    Now, using some other machine, ftp to sunsolve.sun.com:/pub/patches and download the latest "recommended" (read: mandatory) patch cluster for Solaris 2.X (2.X_Recommended.tar.Z). Put this tarfile on a tape, move it over to your secure server, and install the patches. Some patches will not be installed because the software they fix is not included in the Core SPARC cluster.

    5.3) Strip down the OS
    Under Solaris, you control what processes are started at boot time by adding and removing files in /etc/rc[S0-3].d (the files in /etc/rc[S0-3].d being hard links to files in /etc/init.d). Many of these startup scripts run processes that you absolutely do not want running on your secure server: NFS is a prime example.

    Consider removing all extraneous startup files from /etc/rc2.d. Our recommendation is to leave the following and remove the rest:

    K15rrcd         S05RMTMPFILES   K15solved       S20sysetup
    S72inetsvc      S99audit        S21perf         
    S99dtlogin      K25snmpd        S30sysid.net    S99netconfig
    K50pop3         S74syslog       S75cron         S92rtvc-config 
    K60nfs.server   K65nfs.client   S69inet                     
    K92volmgt       README          S95SUNWmd.sync
    S01MOUNTFSYS    S71sysid.sys    S88utmpd        S95rrcd
    This list will be larger or smaller depending on if you have a graphics card in the machine, are using Solstice DiskSuite, and so on. Remove the files in /etc/rc3.d.

    For Solaris 2.4, edit /etc/init.d/inetinit and add the following lines near the end of the file:

    ndd -set /dev/ip ip_forward_directed_broadcasts 0
    ndd -set /dev/ip ip_forward_src_routed 0
    ndd -set /dev/ip ip_forwarding 0
    These lines turn off a feature called IP forwarding. Nearly any machine that uses IP-based networking is capable of being a router, which means the bad guys could route packets through your machine to machines on your internal network or other secure machines that might trust the machine you are building. Turning off IP forwarding disables this feature. ```

    Also consider setting the "ip_strict_dst_multihoming" kernel variable via ndd -set /dev/ip ip_strict_dst_multihoming 1

    The solaris machine will then drop packets coming in through one interface that are destined for another interface. This can prevent host spoofing.

    Under Solaris 2.5, creating a file called /etc/notrouter will turn off IP forwarding. To allow routing again, simply remove /etc/notrouter and reboot. It's important to note that there is a small time window between when this file is created and when routing is disabled, theoretically allowing some routing to take place.

    Under Solaris 2.4, add a new script called /etc/init.d/tmpfix:

    #!/bin/sh
    #ident  "@(#)tmpfix 1.0    95/08/14"
     
    if [ -d /tmp ]
    then
            /usr/bin/chmod 1777 /tmp
            /usr/bin/chgrp sys /tmp
            /usr/bin/chown root /tmp
    and then ln /etc/init.d/tmpfix /etc/rc2.d/S79tmpfix so the script is invoked at boot time. This script prevents an attack that would allow a system cracker to get superuser access on your machine. It is not necessary under Solaris 2.5 (See "Re-tooling" SunWorld Online October 1995.)

    Some good advice from Casper Dik. Make sure that all of the startup routes run with the proper umask, so the files they create and not group or world writeable. Here's an easy method to insure this:

         umask 022  # make sure umask.sh gets created with the proper mode
         echo "umask 022" > /etc/init.d/umask.sh
         for d in /etc/rc?.d
         do
             ln /etc/init.d/umask.sh $d/S00umask.sh
         done
    Note: the trailing ".sh" of the scriptname is important, if you don't specify it, the script will will be executed in a sub-shell, not in the main shell that executes all other scripts.

    Remove /etc/auto_*. Removing /etc/init.d/autofs should have prevented the automounter from starting up, but there's no reason to keep these files around either.

    Remove /etc/dfs/dfstab. Again, purging /etc/init.d should prevent the machine from ever becoming an NFS server, but best to get rid of this file anyway.

    Remove crontab files. You should remove all files from /var/spool/cron/crontabs except the root file.

    Use static routing. Create an /etc/defaultrouter file instead of relying on information from routed (which can be spoofed). If you need to route through different gateways, consider adding /usr/bin/route commands to /etc/init.d/inetinit instead of running routed.

    When you are done, reboot your machine. Always test boot file changes thoroughly. When the machine comes back up, the output of ps -ef should look like this:

         UID   PID  PPID  C    STIME TTY      TIME COMD
        root     0     0 55   Mar 04 ?        0:01 sched
        root     1     0 80   Mar 04 ?       22:44 /etc/init -
        root     2     0 80   Mar 04 ?        0:01 pageout
        root     3     0 80   Mar 04 ?       33:18 fsflush
        root  9104     1 17   Mar 13 console  0:00 /usr/lib/saf/ttymon -g -h -p myhost console login:  -T sun -d /dev/console -l co
        root    92     1 80   Mar 04 ?        5:15 /usr/sbin/inetd -s
        root   104     1 80   Mar 04 ?       21:53 /usr/sbin/syslogd
        root   114     1 80   Mar 04 ?        0:11 /usr/sbin/cron
        root   134     1 80   Mar 04 ?        0:01 /usr/lib/utmpd
        root   198     1 25   Mar 04 ?        0:00 /usr/lib/saf/sac -t 300
        root   201   198 33   Mar 04 ?        0:00 /usr/lib/saf/ttymon
        root  6915  6844  8 13:03:32 console  0:00 ps -ef
        root  6844  6842 39 13:02:04 console  0:00 -sh
    You will note that the /usr/lib/sendmail daemon is not running on the system: this is a feature. Processes that need to send mail off the system can and will invoke sendmail directly (possibly via some other mail user agent such as mailx), but you do not have to run a sendmail daemon that listens on port 25 and processes the mail queue immediately. You should add the following to root's crontab file:
    0 * * * * /usr/lib/sendmail -q
    This entry will invoke sendmail every hour to process any queued mail.

    5.4) Install third-party software
    You will need three pieces of software to help secure your system and allow you to do safe administration of the machine over the network. Because there should be no compiler on your secure server, you will have to build these packages on some other machine and bring them onto the new machine you are building via a tape or floppy disk.

    The first package is the TCP Wrappers package written by Wietse Venema. Wietse's source code produces a small binary called tcpd that can be used to control access to services (such as telnet and ftp) that are started out of /etc/inetd.conf. Access control can be performed on IP address, domain name, and a raft of other parameters, and tcpd can raise alarms via syslog or e-mail if unauthorized connection attempts are made.

    Next, configure S/Key for remote secure access. See Q5.6 for S/Key configuration details.

    If you plan to allow ftp access to your secure server (whether anonymous or for administrative access), you need to get a copy of the WU-Archive ftp daemon. You must get version 2.4 or later since previous versions have major security holes. If you are going to grant administrative ftp access (as opposed to only anonymous ftp), you must hack S/Key support into the ftp daemon binary. In the Crimelabs S/Key distribution you will find an skey/misc/ftpd.c file that demonstrates how to make a previous version of the WU-Archive ftp daemon support S/Key. You will have to make the analogous changes to the v2.4 wu-ftpd source. You may also want to read the wu-ftp FAQ

    Compile and install the binaries (tcpd, wu-ftpd, and keyinit, keysu, and keysh from the S/Key software) in /usr/local/bin. When compiling wu-ftpd you will have to specify configuration and logging directories: we recommend you put the configuration directory somewhere under /etc and the logfiles under /var (so they have plenty of room to grow). See Q5.7 for detailed wu-ftp configuration. Use /noshell for all non-login accounts. Make /noshell be the login shell for all users except root and access. This shell disallows the login, while logging that a login attempt occurred. Crackers will never be able to get access through these accounts.

    5.5) Limit network access to the system
    When your secure machine eventually gets into position on the network, you will probably want to be able to use telnet and ftp to access the machine. Note that you do not have to enable these services on your machine, making your machine that much more secure, but that means you will always have to log in on the console and move files on and off the machine using tape or floppy disk.

    The telnet and ftp daemons are started by the inetd process. inetd's configuration file, /etc/inet/inetd.conf, contains many other services besides telnet and ftp, so it is best to simply remove this file and create a new one containing only the following lines:

    ftp stream tcp nowait root /usr/local/bin/tcpd /usr/local/bin/wu-ftpd
    telnet stream tcp nowait root /usr/local/bin/tcpd /usr/sbin/in.telnetd
    Note that we are using tcpd to control access to both of these services and are using the wu-ftpd binary instead of the ftp server that comes with Solaris. If you do not want to allow anybody to telnet or ftp to you system, then simply remove the appropriate line from inetd.conf, or remove the file altogether and inetd will not even be started at boot time.

    Access control for tcpd is configured using the /etc/hosts.allow and /etc/hosts.deny files. tcpd looks at hosts.allow first, so you can permit a few machines to have telnet or ftp access and then deny access to everybody else in hosts.deny (this is often called a "default deny" policy). Here is a sample hosts.allow file:

    ALL: 172.16.3.0/255.255.255.0
    This would allow any user on any host on the 172.16.3.0 network to have access to all of the services (telnet and ftp) on your machine. That user will still have to supply the appropriate password or S/Key response (see below). Always use IP addresses in the hosts.allow file, because hostname information can be spoofed (if you are using the Internet Domain Name Service, or some other name service such as NIS).

    Now we want to disallow access for everybody else. Put the following line into /etc/hosts.deny:

    ALL: ALL: /usr/bin/mailx -s "%d: connection attempt from %c" root@mydomain.com
    Not only does this deny access to all services from all hosts, it causes tcpd to send an alarm via e-mail that somebody is trying to get access to the machine. Substitute the e-mail address of somebody who reads e-mail regularly for root@mydomain.com in the above line.

    Now you want tcpd to log all accesses via syslog. Put the following line in /etc/syslog.conf:

    auth.auth.notice;auth.info           /var/log/authlog
    The white-space on the line above must be tabs, or syslog will be unable to properly parse the file and no logging will happen. Note that the logging facility is a configurable parameter when building tcpd, but we recommend using AUTH, as opposed to any LOCAL* facilities.
  • Other files in /etc/init.d. Casper reiterates that the choice of which scripts to take out of /etc/init.d is highly dependent upon which version of Solaris you are using. The list of files we presented in our article was based on a Solaris 2.4 machine, so make sure you know what you're doing if you try out the directions on a Solaris 2.5 machine.
  • Sendmail. Instead of invoking sendmail from cron you can also run the daemon without the -bd option. This will cause the daemon to be a queue-watcher only. We prefer not to have the daemon running (one less process to worry about on the system), but your mileage may vary.
  • /etc/syslog.conf. There was a typo and an error in our instructions for configuring /etc/syslog.conf for tcpd. The correct line is
  • auth.info               /var/log/authlog
    which will log all auth events with severity info and higher to /var/log/authlog. Don't forget that the whitespace in this entry must be tabs. 5.6) Configure S/Key
    You will need the S/Key package for remote, secure access. (Part of the logdaemon toolset). S/Key is a one-time-only password mechanism. Instead of typing in your password over the network, the S/Key software will send you a challenge string, and you will compute a response on your local machine using the challenge and a secret password that you have memorized. If you send the proper response back, the S/Key software gives you access to the machine but the response you gave will never again be valid for getting access to the machine. This means that if somebody is using a packet sniffer, they cannot capture the response and use it to break into your machine. The S/Key software also comes with a version of the su command that uses S/Key challenge/response to grant superuser access safely over the network.

    To begin using S/Key, create an account that uses /usr/local/bin/keysh as its login shell. You will login as this user with some reusable password of your choosing and then keysh will force you to respond to an S/Key challenge before giving you a shell prompt.

    You should put

    access:x:100:100:Access Account:/tmp:/usr/local/bin/keysh
    at the end of /etc/passwd and
    access:NP:6445::::::
    at the end of /etc/shadow. Then use the passwd access command to set the password for user access. The password you choose here doesn't have to be a very good password, since you will be relying on keysh to provide the security for this account.

    Since /usr/local/bin/keysh is not a standard shell, you have to create an /etc/shells file with these lines in it:

    /sbin/sh
    /usr/local/bin/keysh
    Only users whose login shell is one of these two shells are allowed to access the machine.

    Now create an codepty /etc/skeykeys file and make it mode 600 and owned by root:

    touch /etc/skeykeys
    chmod 600 /etc/skeykeys
    chown root /etc/skeykeys
    chgrp root /etc/skeykeys
    Use the keyinit access command to initialize the S/Key secret for user access. You have completed the S/Key setup for user access.

    You now want to allow the user access to use the keysu command to become superuser. First change the entry for group root in /etc/group:

    root::0:root,access
    Only users listed in this entry are allowed to become superuser using keysu. Now you have to use the keyinit root command to initialize the S/Key secret for the superuser. I recommend using a different secret word than you used for user access.

    At this point you may be tcodepted to simply remove /bin/su so that users are forced to use keysu. Unfortunately, many scripts use /bin/su to start processes that do not require full superuser access. You should, however, chmod 500 /bin/su so that only the superuser can run this program.

    Note the implicit assumption that access is a group account shared by several administrators. You may feel more comfortable giving each administrator a separate account with a different S/Key secret so that you have a better audit trail. The downside to this approach is it creates more accounts that can have bad passwords on them and allow crackers to gain access. Decide what makes sense for your organization.

    5.7) Configure wu-ftp

    Configuring wu-ftpd is tricky, even for experts. When you compiled wu-ftpd, you specified a directory where the daemon will look for its configuration files. This configuration directory will contain a pid directory and three files. The ftpconversions file should be empty but it must exist. The ftpusers file should contain a list of all users in the machine's password file who should not be allowed to ftp into the system. This file should contain every user in the password file: we will add a user later who is allowed to ftp into the system. Most especially, root should always be in the ftpusers file.

    The last file in the configuration directory is the ftpaccess file. Rather than go into full detail, here is a file that you can use if you do not want to allow anonymous ftp into your system, but do want some users to be able to upload files to the machine:

    class users real 172.16.3.*
     
    log commands real
    log transfers real inbound,outbound
    This allows users to ftp from any host on network 172.16.3.0 and from nowhere else (which is redundant if you are already using tcpd, but security in depth is a Good Thing). All commands and file transfers will be logged to the log file you specified when you compiled the daemon (which is why we recommended you put it in /var where it has plenty of space to grow).

    Configuring anonymous ftp is beyond the scope of this FAQ, but be warned to be very careful, because it is easy to make a mistake and give away access to your system. Review the documentation that comes with wu-ftpd for information and pointers to other documents on anonymous ftp.

    5.8) Limit access to files and file systems
    Download and use fix-modes. The programs in this package clean up unsecure group permissions on some system files and directories. This script has recently been updated to support Solaris 2.5 systems.

    One way to thwart potential system crackers is to prevent them from running setuid programs on your machine. The steps you have already taken make it unlikely that anybody is going to get an unauthorized setuid program onto your machine, but a little paranoia never hurt anybody in the security business.

    Use the nosuid option in /etc/vfstab to prevent setuid programs from being executed on any of your UFS filesystems:

    /proc                   -       /proc      proc    -   no   -
    fd                      -       /dev/fd    fd      -   no   -
    swap                    -       /tmp       tmpfs   -   yes  -
    /dev/dsk/c0t3d0s1       -       -          swap    -   no   -
     
    /dev/dsk/c0t3d0s0 /dev/rdsk/c0t3d0s0  /       ufs  1   no   remount,nosuid
    /dev/dsk/c0t3d0s4 /dev/rdsk/c0t3d0s4  /usr    ufs  1   no   ro
    /dev/dsk/c0t3d0s5 /dev/rdsk/c0t3d0s5  /var    ufs  1   no   nosuid
    /dev/dsk/c0t3d0s6 /dev/rdsk/c0t3d0s6  /local  ufs  2   yes  nosuid
    Note that /usr contains some setuid executables (not the least of which are the S/Key binaries in /usr/local/bin), so we mount it read-only instead of nosuid. The root filesystem is mounted by the boot PROM, so it has to be remounted for the nosuid directive to take effect.

    You must not apply the nosuid directive to the special filesystems. Note that TMPFS type filesystems (such as /tmp) are automatically nosuid in recent releases of Solaris.

    5.9) Test the configuration
    Reboot your system one last time and plug it into the network. Confirm that the following are true:

    Congratulations! You have just created a very secure machine.

    5.10) Step 10: Other suggestions
    There are a number of freely available replacements for common system programs that are significantly more secure than the versions provided with Solaris. Consider replacing sendmail with the latest version. Replace the resolver library with the latest version of bind. Replace syslog and crontab (check your local Archie server).

    Your logging information is only as good as the timestamps in your files. Consider running XNTP to keep your system clock in synch with accurate clock sources from the Internet and with other machines on your networks. If somebody breaks in to your network, you may have to correlate logs between dozens of systems, so their clocks had better be in synch.

     Before your system goes live on the network, get a copy of tripwire and make a database of MD5 checksums for all files on your secure server. Put this database on removable media (tape or floppy disk) so that somebody who breaks into your system can't modify the data. Run tripwire at random (but frequent) intervals to make sure nobody is tampering with your files.

    Consider running process accounting so that you have a record of every command executed on your system. You will pay a performance penalty (10 to 20%) if you turn on process accounting. See man acct for more information.

    Change your S/Key secrets regularly (every 30 days if you can stand it) and choose good secret words (use non-alphanumeric characters and long secret words). Change the secret words on the system console or use keyinit -s to change the secret over the network. Do not use the same secret word for both the access and root accounts.

    6) Break-in detection

    6.1) Caveats

    While snooping around the system, avoid leaping to conclusions. Try to perform all the main tests and roughly examine all aspects of the system. Only then should you start drawing conclusions and using the fine-toothed comb in likely areas.

    Remember that there are many ways for a cracker to compromise your site's security, with the main non-technical means being physical penetration and social engineering. Evaluate these potentials as well as the operating system and applications.

    Also remember that the cracker could be anyone. If you find crack tools in a trusted employee's directory, then you must consider the possibility that you've found the cause of your problems. In cases such as this, the next step could be setting traps for the suspects. For instance, replace these tools with instrumented sets that will trace what they are being used for and alert you during their use. This is the type of evidence you will need when confronting a company employee.

    If there is evidence of a break-in, you need to determine a proper course of action. Do you "clean" the system and then further secure it? Do you try to detect and trap the cracker? These decision can involve corporate policies that may dictate security incident response. A lot will depend on the role of the machine in question. For instance, if the machine is not a production server, or does not contain sensitive information, then there may be leeway for long-term break-in detection instrumentation.

    While exploring your site, remember that crackers can try to gain access as legitimate users and then as root, or directly as root. Protecting user accounts can be effective at keeping crackers off the systems and preventing them from growing their access rights.

    6.2) Other resources
    The CERT intruder detection checklist is an invaluable guide to system evaluation. It could be used in conjunction with the information here for a thorough system examination.

    6.3) Determine current system state
    Check current logins and watch for suspicious activity, including long logins and logins at odd times. Other signs that a cracker is using your system include uncharacteristic tool usage (compiler use by an administrative assistant), heavy system use, dial-out modem use, and odd network connections. If the cracker is currently on the system, your activities could be monitored and detection prevented. Depending on what you determine below, the tools you normally use to detect problems may have been compromised and may not be reliable. It would be best to transfer a known good copy of the programs that you use to the system under suspicion and only use those tools. Useful tools include ps, ifconfig, find, df, and ls.

    Other interesting system state information includes what systems have nfs-mounted your disks. Solaris provides the dfmounts command to show current remote mounts. For instance, this output is problematic because l0pht might be used by crackers:

    # dfmounts
    RESOURCE     SERVER PATHNAME    CLIENTS
      -           server /cdrom/    alliant.l0pht.com
    Also, check the network status of your systems. Perhaps run snoop to watch network traffic and try to detect unexpected connections or odd packets. netstat -a shows all network daemons running on the machine. Unfortunately, general-purpose servers provide so many facilities that it is difficult to separate the wheat from the chaff via netstat.

    Determine if your machine is in "promiscuous" mode via the ifstatus program.

    Other system state is recorded in the system's configuration files. Careful examination of these files, including their access rights and the last modification date, can be revealing. Files to examine include /etc/inetd.conf, /.rhosts, /etc/hosts.equiv, /etc/passwd, /etc/group, /home/*/.rhosts, /etc/dfs/dfstab, and /etc/ftpusers.

    Check the e-mail of root and other system directories. Some tools automatically send mail there in case of problems, and those problems can be a sign of an attack or break-in.

    Scan the system for problem files. These files can include programs that are set-uid or set-gid and should not be. They can also include "trick" file and directory names such as "...", ".. ", and names that include odd characters. A trustworthy find is the best bet for these scans. Bad directory permissions, such as group or world-writable /etc can give away the shop, so all system directory permissions should be checked.

    Check the cron and at queues to make sure that all your work is not being undone by automated scripts that re-open security holes.

    6.4) Examine system logs
    Logs can be useful not only for detecting penetration but also for determining the scope and timeframe of the break-in. However, logs can be (and are) modified by crackers to hide their tracks, so a lack of activity in the logs is not an indication that the system is secure. Useful Solaris logs include:

    If syslog is redirecting logs to another host, be sure to check that host's logs as well.

    The logs can indicate events that occur normally but can also be a sign of problems. For instance, a system reboot can be a bad sign. Errors messages from daemons like sendmail and finger can be caused by break-in attempts. Crackers can change the system clock to hide their trails.

    6.5) Patch failure
    Are the operating system patches up-to-date? If not, determine which patches should have been applied and how the associated security hole could have been used. For instance, if sendmail is not up-to-date, then check the cracker sites for exploit scripts. By examining the scripts, you may be able to find tell-tales that would be left behind if that script had been used against your site.

    6.6) Binary modification
    Once crackers penetrate a system, they can leave tools to help gather information from your site, or tools to allow them to regain access to the machine. Searching for these tools can be time consuming, but generally worth the effort. Several specific tools are described below, but a useful approach is to execute ls -ltr in important directories. The last files listed are the most recently changed. Either they were changed by administrators at your site (via installations or patches) or they were changed by a cracker. Typical system binaries that crackers attempt to replace include ps, ifconfig, find, login, df, and ls.

    6.7) Enlist the users
    Users can be either friend or foe. They can ignore your password selection recommendations, give account access to friends, login remotely with clear-text connections, and generally make the life of a security administrator a living hell. Or your users can help secure your systems. User education is one of the best tools in your arsenal. Teach users proper security methodology, and why it is important to secure the machines. If they do not feel the need for security, they will not help provide it. Users are your first and last defense against social engineering attacks: they can either grant a pushy cracker access, or they can alert you to the attack. Users can help keep their accounts secure by understanding permissions and paying attention to "last login" date information. Really helpful users can examine their last output to verify that no one else is using their accounts.

    6.8) What about the applications?
    Many security holes are introduced by application programs, such as Web and database servers. These applications need to be checked for configuration problems, interesting event logs, and access rights. Also, patches should be installed -- the lack of up-to-date patches can give you a place to start in your cracker hunt.

    6.9) Use the Web, Luke
    Because crackers use the Web to communicate vulnerability information and to brag about system conquests, it can also be used by security administrators and auditors to check for this information. For instance, search on the known cracker sites, or the Internet as a whole, for your site and machine names and the words "crack break secure" and "hack." The results can be alarming.

    Useful sites include:

    7) Acknowledgements?

    The following people contributed to the contents of this FAQ: