Make your own free website on Tripod.com

SunService Tip Sheet: TCP/IP

 

INFODOC ID: 12618

SYNOPSIS: TCP/IP PSD/FAQ
DETAIL DESCRIPTION:


SunService Tip Sheet for Sun TCP/IP
Includes Notes on ndd variables and netstat
 
Revision: 1.9
Date: August 26, 1996
 
1.0: About TCP/IP
2.0: Debugging TCP/IP
  2.1: netstat
  2.2: etherfind and snoop
3.0: Common How Tos
  3.1: How to Tune Network Variables in SunOS
  3.2: How to Tune Network Variables in Solaris
4.0: Some Frequently Asked Questions
  4.1: Miscellaneous Questions
  4.2: Questions on Ports
  4.3: netstat Questions
  4.4: Web Server TCP/IP Optimization
5.0: Patches
  5.1: SunOS TCP/IP Patches
  5.2: Solaris TCP/IP Patches
6.0: Known Bugs and RFEs
  6.1: RFEs
7.0: References
  7.1: Important Man Pages
  7.2: Sunsolve Documents
  7.3: Sun Educational Services
  7.4: Solaris Documentation
  7.5: Third Party Documentation
  7.6: RFCs
8.0: Supportability
9.0: Additional Support
1.0: About TCP/IP
=================
 
This Tip Sheet documents a wide variety of information concerning
TCP/IP. It is mainly concerned with information on the tuning options
available in Sun TCP/IP, as well as a few of the most common
questions. Unless you are doing really low level work with TCP/IP,
this Tip Sheet probably will not be of too much use to you. However,
if you are doing low level work, this Tip Sheet will give you some
ideas on what you can do with Sun's TCP.
 
A lot of the topics in this Tip Sheet fall under the category of
Performance Tuning. This is an area that is normally not covered by
SunService. Hopefully, this Tip Sheet will get you started tuning
things on your own. If it is insufficient, you should consult Sections
8.0 and 9.0 for how to get help from Sun Consulting Services.
2.0 Debugging TCP-IP

2.1: netstat
------------
 
The netstat program may be used to examine a wide variety of network
information. In its most basic form, when run without arguments,
netstat will show currently active connections:
 
  # netstat
 
  TCP
     Local Address        Remote Address    Swind Send-Q Rwind Recv-Q  State
  -------------------- -------------------- ----- ------ ----- ------ -------
  localhost.32791      localhost.32796       8192      0  8148      0 CLOSE_WAIT  
psi.3139             sockem.listen         8760      0  8760      0 ESTABLISHE  
psi.login            sun-soft.1020         4096      0  9112      0 ESTABLI
 
The most important bits of information here are the local address
(local name + local port), the remote address (remote name + remote
port) and the State. In many cases, the port number is replaced by a
port name, defined by the /etc/services file.
 
The possible states are all defined in the netstat man page. Of
particular note is the TIME_WAIT state  connections will always wait
in that state for a designated amount of time (called the 2MSL time)
before the connection is totally shut down. This is to prevent new
programs from reusing a port during a period in which packets destined
for an older application might arrive there.
 
netstat -a also shows information as above, but includes all server
processes, which are usually waiting in a LISTEN state (netstat
without options only lists active processess, not passively LISTENing
processes).
 
netstat is commonly used in debugging to examine network performance
problems which may relate to programs getting stuck in certain states.
This is done by examining the states noted in the netstat output
above.
 
There are a number of other uses for netstat: netstat -i will show
interface information, netstat -p will show arp information and
netstat -r will show routing information.
2.2: etherfind and snoop
------------------------
 
You can use etherfind (SunOS) and snoop (Solaris) to examine every
single packet which goes across the network. They can be useful if you
are looking for retransmissions, duplicate ACKs, or other similar
problems. The etherfind and snoop man pages explain how to use those
utilities to examine the network. SunService also has a seperate Tip
Sheet (the Miscellaneous Programs Tip Sheet) which outlines some basic
uses of etherfind and snoop.
3.0 Common How Tos

3.1: How to Tune Network Variables in SunOS
-------------------------------------------
 
3.1.1: config Variables
-----------------------
 
Four variables may be modified in the config file for your kernel (eg
/sys/sun4c/conf/GENERIC). Each of these may be changed by adding a
line as follows:
 
  options OPTIONNAME =value
 
For example:
 
  options IPFORWARDING =-1 
 
The legal options are:
 
DIRECTED_BROADCAST
 
  Determines whether to forward broadcasts directed to a specific net or
  subnet, if that net or subnet is directly connected to the machine.
  The default value is 1 (True).
 
IPFORWARDING
 
  Determines if the workstation will route packets. This variable can
  be one of three selections:
  
    -1    Never forward
    0     Only forward when two or more interfaces are up
    1     Always forward
 
  The default value is 0.
 
IPSENDREDIRECTS
 
  Determines whether host should send ICMP redirects when it routes
  packets. The default value is 1 (True).
 
SUBNETSARELOCAL
 
  Determines if remote subnets on the same network should be
  considered local. The default value is 1 (True).
 
After making any of these changes, config the kernel, and remake it.
 
3.1.2: in_proto.c Variables
---------------------------
 
Other Network variables may be modified under SunOS by modifying the
file /sys/netinet/in_proto.c. These are each standard C declarations.
After modifying them, the kernel must be rebuilt.
 
tcp_default_mss
 
  Determines the Maximum Segment Size for nonlocal packets (ie,
  packets to be sent to different networks or subnets). The Definition
  of local may differ, depending on how SUBNETSARELOCAL is set.
  The default value for tcp_default_mss is 536 bytes.
 
tcp_keepidle
 
  Determines how frequently to test if an idle connection is still
  alive. The default value is 2 hours.
 
tcp_keepintvl
 
  Determines how frequently to check an idle connection, if the first
  check has failed. The default value is 75 seconds.
 
tcp_keeplen
 
  Compatibility variable that must be set to 1 for BSD4.2
  compatibility. The default value is 1.
 
tcp_nodelack
 
  Determines whether ACKs can be delayed. Default is 0 (Allow ACKs
  to be delayed). A value of 1 will turn off delayed ACKs.
 
tcp_recvspace
 
  Determines the maximum value of the TCP receive window. The default
  value is 4096 (4k).
 
tcp_sendspace
 
  Determines the maximum value of the TCP transmit window. The default
  value is 4096 (4k).
 
tcp_ttl
 
  Determines the default Time To Live for TCP packets. The default
  value is 60.
 
udp_cksum
 
  Determines whether to checksum UDP packets. The default value is
  0 (False). This should usually be changed to 1 (True) if you are
  having any troubles with UDP services (ie, NFS, NIS)
 
udp_ttl
 
  Determines the default Time To Live for UDP packets. The default
  value is 60.
 
udp_recvspace
 
  Determines the maximum value of the UDP receive window. The default
  value is 18000 bytes.
 
udp_sendspace
 
  Determines the maximum value of the UDP transmit window. The default
  value is 9000 bytes.
 
3.2: How to Tune Network Variables in Solaris
---------------------------------------------
 
Most TCP, IP, UDP and ARP tuning in Solaris is done via the ndd
command. You can list all of the appropriate variables by supplying
ndd with the driver name and a ?, ie:
 
  % ndd /dev/arp \?
  % ndd /dev/ip \?
  % ndd /dev/tcp \?
  % ndd /dev/udp \?
 
Note: Under Solaris 2.5 or higher, you will need to be root to display
these variables.
 
You can change ndd variables by supplying the -set option, the
variable name, and the value, ie:
 
  % ndd -set /dev/ip ip_forwarding 0
 
If you wish a ndd variables to get set upon every boot, you should put
a line for it in the file /etc/rc2.d/S69inet:
 
  % cat /etc/rc2.d/S69inet
  ...
  #
  # Set configurable parameters.
  #
  ndd -set /dev/tcp tcp_old_urp_interpretation 1
 
Many variables have two options: 1 (True) and 0 (False).
 
Note: in all of the following examples, the default values for 2.4 are
listed. Other OSes may differ, and can always be discovered by
typing:
 
   # ndd device variable
 
Ie:
 
  # ndd /dev/ip ip_debug
  0
 
One word of warning concerning the ndd variables: They should be
considered quite volatile, and might totally change from one release
of Solaris to another. Only the variables that are the most useful, or
the least likely to change, are listed below, but even these could be
potentially modified in new releases of the OS. Unlisted variables,
which can be viewed with the ndd command, should be considered even
less static.
 
3.2.1: IP Variables
-------------------
 
ip_def_ttl
 
  Determines the default Time To Live value for standard IP packets.
  The default value is 255.
 
ip_forwarding
 
  Determines if the workstation will route packets. This variable can be
  one of three selections:
  
    0     Never forward
    1     Always forward
    2     Only forward when two or more interfaces are up
 
  The default value is 2.
 
ip_send_redirects
 
  Determines whether the host should send ICMP redirects when it routes
  packets. The default value is 1 (True).
 
3.2.2: TCP Variables
--------------------
 
tcp_close_wait_interval
 
  Despite the misleading name, this variable actually sets the time
  wait interval (the 2MSL value). The default value is 240000 ms (4
  minutes).
 
tcp_ip_ttl
  
  Determines the default Time To Live for TCP/IP packets. The default
  value is 255.
 
tcp_keepalive_interval
 
  Determines how frequently to test if an idle connection is still
  alive. The default value is 7200000 ms (2 hours).
 
tcp_mss_def
 
  Determines the Maximum Segment Size for nonlocal packets (ie,
  packets to be sent to different networks or subnets). The default
  value is 536 bytes.
 
tcp_recv_hiwat
 
  Determines the maximum value of the TCP receive window. The default
  value is 8192 (8k).
 
tcp_smallest_anon_port
 
  Determines the smallest TCP port number that may be used for an
  anonymous connection. The default value is 32768.
 
tcp_xmit_hiwat
 
  Determines the maximum value of the TCP transmit window. The default
  value is 8192 (8k).
 
tcp_xmit_lowat
 
  Determines the minimum value of the TCP transmit window. The default
  value is 2048 (2k).
 
3.2.3: UDP Variables
--------------------
 
udp_def_ttl    
 
  Determines the default Time To Live for UDP packets. The default
  value is 255.
 
udp_do_checksum
 
  Determines whether checksums should be done for UDP packets. The
  default value is 1 (True).
 
udp_recv_hiwat
 
  Determines the maximum value of the UDP receive window. The default
  value is 8192 (8k).
 
udp_smallest_anon_port
 
  Determines the smallest UDP port number that may be used for an
  anonymous connection. The default value is 32768.
 
udp_xmit_hiwat
 
  Determines the maximum value of the UDP transmit window. The default
  value is 8192 (8k).
 
udp_xmit_lowat
 
  Determines the minimum value of the UDP transmit window. The default
  value is 1024 (1k).
 
4.0 Some Frequently Asked Questions

4.1: Miscellaneous Questions
----------------------------
 
Q: What RFCs does Sun TCP/IP follow?
 
A: Sun follows RFC 793 for TCP, RFC 768 for UDP, RFC 791 for IP, and
RFC 1122 overall. RFC 1323 is notably not implemented. See section 6.1
for more information.
 
Q: Will TCP/IP kill idle connections?
 
A: No. If a connection goes idle for an extended amount of time,
probes will be sent out to verify that the remote machine or
connection has not gone down. However, provided that everything is
still running, TCP/IP connections should stay live forever.
4.2: Questions on Ports
-----------------------
 
Q: Why do ports get stuck in the TIME_WAIT state?
 
A: This is due to the 2MSL timeout value, which is a part of the TCP
specification. After a connection has been closed by both the client
and the server, the port becomes unavailable for a certain amount of
time, so that a new program does not inadvertantly get packets that
were intended for the old program. On Solaris machines, the 2MSL value
may be modified by adjust the /dev/tcp tcp_close_wait_interval ndd
variable. See Section 3.2.2.
 
Q: Why do I get "address already in use" when I try and reuse a port
which a previous program had used.
 
A1: It could be that the port is still in the TIME_WAIT state (see
above). You may confirm this by running netstat, and examining the
port in question. If this is the case, you must simply wait the couple
of minutes it will take for the port to get out of the TIME_WAIT
state.
 
A2: It could be that the program you are using does not correctly list
the port as one that may be reused. You should verify that your
program is setting the SO_REUSEADDR socket option.
4.3: netstat Questions
----------------------
 
Q: Why does netstat hang when I run it?
 
A: This is usually due to an error in name resolution. Run netstat
with the -n option to verify that this is the case:
 
  % netstat -n
 
  % netstat -rn
 
If you find that netstat does not hang when you run with the -n
option, you should investigate your naming services (DNS, NIS, NIS+)
for further problems.
4.4: Web Server TCP/IP Optimization
-----------------------------------
 
Q: How do I increase the listen backlog described in my HTTPD
documentation?
 
A: This may be modified only on Solaris machines. On these machines,
you will want to adjust the /dev/tcp tcp_conn_req_max variable.
Section 3.2 explains how to set ndd variables temporarily or
permanently.
 
On Solaris 2.4 or lower, you can use ndd normally to set the listen
backlog up to 32, from the default of 5:
 
  ndd -set /dev/tcp tcp_conn_req_max 32
 
By modifying your kernel, you can increase the backlog even higher.
 
  echo "tcp_param_arr+14/W 0t128"   adb -kw /dev/ksyms /dev/mem
  ndd -set /dev/tcp tcp_conn_req_max 128
 
You must make sure that you have at least patch 101945-36 installed
before you try this.  Values higher than 128 are not suggested. These
lines should both go into /etc/rc2.d/S69inet to make the change
permanent.
 
On Solaris 2.5 or higher, you can set the listen backlog up to 1024
normally:
 
  ndd -set /dev/tcp tcp_conn_req_max 1024
 
Q: What else can I do to increase Web Server Performance?
 
A: You should adjust the listen backlog up, as is suggested in your
Web Server documentation. You will probably also want to increase the
size of the send, receive and congestion windows:
 
  ndd -set /dev/tcp tcp_cwnd_max 65535    
  ndd -set /dev/tcp tcp_xmit_hiwat 65536
  ndd -set /dev/tcp tcp_recv_hiwat 65536
 
Note that under Solaris 2.5 and higher, tcp_cwnd_max is already set to
65535 by default. Also note that these ndd settings only apply to
Solaris.
5.0 Patches

5.0: Patches

The following is the list of all of the TCP/IP related patches for
4.1.3, 4.1.3_u1, 4.1.4, 5.3 and 5.4. If you are having TCP/IP
problems, installing the patches is a good place to start, especially
if you recognize the general symptoms noted below.
 
In order for a machine to be stable, all of the recommended patches
should be installed as well. The list of recommended patches for your
operating system is available from sunsolve1.sun.com.
5.1: SunOS TCP/IP Patches
-------------------------
 
atch-ID#  SunOS 4.1.1,4.1.2,4.1.3: TCP socket and reset problems  
101790-01 SunOS 4.1.3_U1: TCP socket and reset problems  
 
  Fix some problems regarding TCP resets, and the usage of the
  getsockopt function.
 
100584-08 SunOS 4.1.3: TCP Interface Jumbo Patch  
102010-02 SunOS 4.1.3_U1: TCP interface Jumbo Patch.  
 
  Fix a wide variety of problems having to do with TCP connections.
5.2: Solaris TCP/IP Patches
---------------------------
 
101318-81 SunOS 5.3: Jumbo patch for kernel (includes libc, lockd)  
101945-42 SunOS 5.4: jumbo patch for kernel  
 
  These kernel patches fix a large number of TCP/IP and other
  network problems. They should be installed on every networked
  machines.
6.0 Known Bugs And RFEs

6.0: Known Bugs and RFEs

The following bugs and rfes represent known open issues regarding TCP/IP
6.1: RFEs
---------
 
1179428 Request for implementation of RFC 1323 for tcpip performance
 
  This RFE requests that RFC 1323 be implemented under Solaris, so
  that performance on satellite or delayed links can be improved.
7.0 References

7.1: Important Man Pages
------------------------
 
arp
etherfind
ip
ndd             (Solaris Only)
netstat
snoop
tcp
udp
7.2: Sunsolve Documents
-----------------------
 
The following SunSolve documents may contain some additional
information, which is not covered in this Tip Sheet.
 
7.2.1: FAQs
-----------
 
1048    IP interface questions and answers
1202    TCP/IP applications have slow response over asynchronous commu
 
7.2.2: Infodocs
---------------
 
2194    Internet Protocol Overview (IP and ICMP)
2195    Transmission Control Protocol (TCP)
 
7.2.3: SRDBs
------------
 
5819    how to set socket keep alive timeout length
6729    UDP checksums for NFS
11416   what are the ARP values set by ndd?
 
7.3: Sun Educational Services
-----------------------------
 
There are no Sun Educational Classes which cover TCP/IP tuning to this
level.
7.4: Solaris Documentation
--------------------------
 
There is no Solaris Documentation which covers TCP/IP tuning to this
level.
7.5: Third Party Documentation
------------------------------
 
_TCP/IP Illustrated, Volume 1_, by W Richard Stevens, published by
Addison-Wesley, ISBN 0-201-63346-9
 
  This is a superb book covering TCP/IP. It explains the majority
  of the terms which are outlined in the above document, and gives
  a really good overview of networking as a whole.
7.6: RFCs
---------
 
There are a huge number of RFCs which cover TCP/IP and other
networking protocols.
 
RFCs are the internet-written documents that define the specifications
of many common networking programs. RFCs can be retrieved from
nic.ddn.mil, in the /rfc directory.
8.0: Supportability
===================
 
SunService is not responsible for the initial configuration of your
TCP/IP environment. In addition, SunService can not diagnose your
TCP/IP performance problems, nor suggest TCP/IP tuning guidelines.
 
We can help resolve problems where TCP/IP is not behaving correctly,
but in such cases the contact must be a system administrator who has a
good understanding of TCP/IP.
9.0: Additional Support
=======================
 
For initial configuration, or TCP/IP performance tuning guidelines,
please contact your local SunService office for possible consulting
offerings. Sun's Customer Relations organization can put you in touch
with your local SunIntegration or Sales office. You can reach Customer
Relations at 800-821-4643.

PRODUCT AREA: Gen. Network
PRODUCT: TCP/IP
SUNOS RELEASE: any
HARDWARE: any