Make your own free website on Tripod.com

SunService Tip Sheet: Sun Routing

 

INFODOC ID: 11992

SYNOPSIS: ROUTING PSD/FAQ
DETAIL DESCRIPTION:

Product Support Document (PSD) Sun Routing

Including route, in.routed, in.rdisc, netmasks, traceroute and arp

Revision: 1.51
Date: May 2, 1996

TABLE OF CONTENTS
=================

1.0: About Routing
  1.1: Routing Definitions
  1.2: How Routing Starts Up
  1.3: IP addresses, Netmasks and Subnets
2.0: Debugging Routing
  2.1: ifconfig
  2.2: netstat
  2.3: ping
  2.4: traceroute
3.0: Common How Tos
  3.1: How to Set a defaultrouter
  3.2: How to Add Static Routes
  3.3: How to Use in.rdisc
  3.4: How to Use in.routed
  3.5: How to Force a Machine to Be a Router
  3.6: How to Force a SunOS Machine Not to Be a Router
  3.7: How to Force a Solaris Machine Not to Be a Router
  3.8: How to Set a Netmask under SunOS
  3.9: How to Set a Netmask under Solaris
  3.10: How to Subnet by Bit
  3.11: How to Protect a Gateway Machine
  3.12: How to Proxy ARP
  3.13: How to Delete ARP Entries
4.0: Some Frequently Asked Questions
  4.1: Miscellaneous Questions
  4.2: Basic Routing Problems
  4.3: in.routed Errors
  4.4: netmask Problems
  4.5: traceroute Problems
  4.6: arp Problems
5.0: Patches
  5.1: Routing Patches for SunOS
  5.2: Routing Patches for Solaris
  5.3: ARP Patches for SunOS
6.0: Known Bugs & RFEs
  6.1: Bugs
  6.2: RFEs
7.0: References
  7.1: Important man Pages
  7.2: Sunsolve Documents
  7.3: Sun Educational Services
  7.4: Solaris Documentation
  7.5: Third Party Documentation
  7.6: RFCs
8.0: Supportability
9.0: Additional Support

1.0 About Routing

 1.0: About Routing

This PSD documents a wide variety of information concerning routing as
implemented in the SunOS and Solaris operating systems. It is intended
as both an introduction to routing and as a guide to the most common
problems. Routing can be a very complex subject and this PSD can
really only skim the surface of it. If you are setting up a complex
routing environment, you will probably also want to refer to the more
complete references listed in Section 7.0.

For this discussion of routing, an understanding is needed of various
aspects of ROUTING, as well as NETMASKS. These are both covered below.

 1.1: Routing Definitions

The ROUTING TABLE is a table maintained in the kernel that determines
how packets are routed to other systems. A number of programs can add
or delete routes from the routing tables, including route, ifconfig,
in.routed and in.rdisc. The routing table consists of three types of
routes:

HOST ROUTES are checked first and define a route to just one host.

NETWORK ROUTES are checked second and define a route to all the hosts
on one network.

DEFAULT ROUTES are used as a catch-all, when no host or network routes
are found to a destination. They usually send to a more knowledgeable
routing machine, which has a better chance of being able to find a
host or network route to the destination. (see Section 3.1 for how to
add a default static route  as explained in Sections 3.3 and 3.4,
default routes can also be used with in.routed and in.rdisc.)

For a packet to get from your machine to its destination, it usually
goes through a number of routers. Each of these routers decides what
to do with the packet based on the set of rules listed above.

Systems are configured to handle routing in one of two ways:

With STATIC ROUTING, the tables in the kernel are defined when the
system boots and do not change. The ROUTE command can also be used to
manually add and remove static routes. (see Section 3.1 for how to add
an automatic default route at boot time  see Section 3.2 for how to
use route to add static routes.)

With DYNAMIC ROUTING, the tables in the kernel are updated by a
ROUTING DAEMON as the network changes. SunOS and Solaris machines both
provide IN.ROUTED to exchange routing information between hosts.
in.routed depends upon RIP PACKETS to exchange this info. Solaris also
provides IN.RDISC, which just manipulates default routes. in.rdisc
depends upon ROUTER DISCOVERY PACKETS to gather this information. (see
Sections 3.3 and 3.4 for how to run in.routed and in.rdisc,
respectively.)

In addition, IFCONFIG will always add routes when interfaces are
brought up. These routes will be to directly connected networks for
most interfaces, though they will be to directly connected hosts for
point-to-point interfaces.

With regard to routing, machines are divided into two classes, routers
and hosts:

A machine with more than one interface is, by default, defined to be a
ROUTER. If in.rdisc or in.routed are running, they will advertise
routing information. In addition, the machine will automatically
forward IP packets. (see Section 3.6 and 3.7 for how to force a
multiple-interface machine not to be a router.)

A machine with only one interface is a HOST. If in.rdisc or in.routed
are running, the host will listen for routing information. Hosts do
not advertise routing information.

1.1.1: A Note on ARP

This document also contains some information on ARP, the ADDRESS
RESOLUTION PROTOCOL. ARP is the protocol which converts the IP address
to an address appropriate to the hardware type (i.e., ethernet addresses
on an ether network). After normal routing protocols have done their
work, ARP springs in to action. It's what actually gets packets from
one machine to another.

Usually, you should not have to worry about ARP at all. Except in
special setups, it should work totally transparently.

 1.2: How Routing Starts Up

Solaris and SunOS each have a specific order which they use to start
up routing:

If an /etc/defaultrouter file exists (see Section 3.1), only static
routing will be used and a static default route will be created to
the machine listed in the defaultrouter file.

If there is no /etc/defaultrouter file and the machine is a router,
both in.routed and in.rdisc will be started and they will advertise
routing information. SunOS machines will only start up in.routed, as
in.rdisc is not available under SunOS.

If there is no /etc/defaultrouter file and the machine is a host,
in.rdisc is initially started. If any machines on the same network
advertise via Routing Discovery, in.rdisc will be used as the dynamic
routing program  if there are no machines advertising via Routing
Discovery, in.rdisc will quickly quit. SunOS skips this step.

Finally, if there is no /etc/defaultrouter file and either the
machine is running SunOS or in.rdisc has failed, then in.routed is
started. It will listen for other in.routed programs advertising via
RIP packets. This is the default manner in which most out-of-the-box
Suns will operate.

 1.3: IP addresses, Netmasks and Subnets

All IP ADDRESSES are divided into two portions: the NETWORK ADDRESS,
and the HOST ADDRESS. The network portion of the IP address defines
the level at which routing is done  this means that all machines with
the same network address are on the same network. For example, if we
had the IP address a.b.c.d, we might learn that the network portion of
that address was a.b and the host portion of that address was c.d. In
that case, all machines with the IP address a.b.*.* would be on the
same network. We would typically see a route to the network a.b.0.0.

By default, Internet IP addresses are divided into 3 Classes. Each of
these ranges has a different portion of the IP address reserved as the
network and host addresses.

  Class         IP Range                        Network         Host
  -----         --------                        -------         ----
  Class A       1.*.*.* to 127.*.*.*            a               b.c.d
  Class B       128.*.*.* to 191.*.*.*          a.b             c.d
  Class C       192.*.*.* to 223.*.*.*          a.b.c           d

Given the standard classes, a few typical network routes would be:

  16.0.0.0
  150.101.0.0
  207.48.123.0

NETMASKS are a shorthand way to define what portion of an IP address
is reserved for the network address. The netmask is a set of 1 and 0
bits set to 1 define the network portion of an IP address, while
bits set to 0 define the host portion of an IP address. We can look at
the example of a Class B network address. In the address a.b.c.d, the
network portion is a.b and the host portion is c.d. This is equivalent
to the netmask 255.255.0.0:

  255.255.0.0 = 11111111.11111111.00000000.00000000

This means that the three classes of networks have the following three
default netmasks:

  Class         IP Range                        Netmask         Nodes/Network
  -----         --------                        -------         -------------
  Class A       1.*.*.* to 127.*.*.*            255.0.0.0       16,777,215
  Class B       128.*.*.* to 191.*.*.*          255.255.0.0     65,535
  Class C       192.*.*.* to 223.*.*.*          255.255.255.0   255

Thus, packets sent to 100.1.2.3 are routed at the level 100.0.0.0 (and
all 16 million machines in the 100.*.*.* range should form one
distinct network), while packets sent to 200.1.2.3 are routed at the
level 200.1.2.0 (and all 255 machines in the 200.1.2.* range should
form one distinct network).

Often, the default netmasks are not sufficient. You can further divide
a network by defining a custom netmask. This is termed SUBNETTING. A
custom netmask changes the definition of the network size and
changes the level at which routing is done. For example, many people
on Class A or B networks implement a netmask of 255.255.255.0, so that
all routing is done to networks of 255 or less machines.  Sections 3.8
and 3.9 explain how to set a custom netmask for SunOS and Solaris.

A few examples of custom netmasks:

  IP Address            Netmask                 Network Address
  ----------            -------                 ---------------
  16.1.2.3              255.255.0.0             16.1.0.0
  32.7.3.7              255.255.255.0           32.7.3.0
  150.101.16.28         255.255.255.0           150.101.16.0

This netmasking is really done down at the bit level. These means that
you can divide a network even further. An example of this is given
down in Section 3.10.

[Note: references to the network 150.101.*.* in this document should
be understood to be wholly fictitious and are only used for reasons of
example.]

2.0 Debugging Routing

 2.0: Debugging Routing

Routing can be somewhat tricky because problems could exist anywhere
between you and your destination machine. The following commands are
helpful, because they can be used to isolate which machine is the one
with incorrect information.

The following understanding of routing is necessary to be able to
fully debug your routing tables:

For communication from host A to host B to work, there must be a route
from A to B and there must be a route from B to A.

If there is an intermediate machine, machine C, then there should be a
route from A to B through C, and C must have a route to B. The reverse
is true as well. There must be a route from B to A through C and
a route from C to A.

Also, C must be configured to pass packets from one interface to
another to allow routing. This is done via the ip_forwarding variable
described in sections 3.6 and 3.7.

When debugging routing, you must also know whether you are working
with a host or a router, as they have very different characteristics.

 2.1: ifconfig

It is always best to start investigating routing problems by making
sure that your interfaces are set up correctly. By running ifconfig,
you can see your current interface setup:

  # ifconfig -a
  lo0: flags=849<UP,LOOPBACK,RUNNING,MULTICAST> mtu 8232
          inet 127.0.0.1 netmask ff000000
  le0: flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST> mtu 1500
          inet 150.101.16.28 netmask ffffff00 broadcast 150.101.16.255
          ether 8:0:20:10:78:d

When an interface is brought up, it automatically creates a route.
This route will be to the local network for multipoint interfaces
(ether, FDDI, TR, etc) or to a host for point-to-point interfaces
(PPP).

If your main interface (le0 in the above example) is not UP and
RUNNING, clearly no routing will be occurring through it. If the
interface is UP, you will also want to examine the inet and netmask
entries of your main interface and make sure they are set correctly.
If the inet is set wrong, check your /etc/hostname.interfacename file
(e.g., hostname.le0) and your /etc/hosts file. If the netmask is set
wrong, check your /etc/netmasks file (see Section 3.8 and Section 3.9
for correct netmask setup).

 2.2: netstat

You can examine your current routing tables by running the command
netstat, as follows:

  %%%% netstat -rn

  Routing Table:
    Destination           Gateway           Flags  Ref   Use   Interface
  -------------------- -------------------- ----- ----- ------ ---------
  127.0.0.1            127.0.0.1             UH       0    581  lo0
  150.101.16.0         150.101.16.28         U        3    935  le0
  224.0.0.0            150.101.16.28         U        3      0  le0
  default              150.101.16.3          UG       0    802
  default              150.101.16.12         UG       0    854

By examining this table and correlating it with your problems, you
can sometimes figure out what is going on. For example, if some of my
non-local packets were randomly getting lost, I might suspect that one
of my two default routes was dropping packets.

Some problems are even more obvious. For example, if the "default"
entries were not present in the Routing Table above, I would probably
find that I could route nowhere but 127.0.0.1, 150.101.16.0 and
224.0.0.0. (In actuality, 224.0.0.0 is a special multicast route,
while 127.0.0.1 is a special localhost route  they can both be ignored
for basic discussions of routing, which would mean that I could really
route only to the network 150.101.16.0.)

 2.3: ping

If your problem is not obvious from examining 'ifconfig -a' and
'netstat -rn', the ping command can be very useful in diagnosing it
further. Try running 'ping -s machine-with-no-route'. There are two
possibilities.

If you get some type of 'network unreachable' or 'host unreachable'
error, this typically means that there is no route on your machine to
the machine you are trying to ping. You will probably need to define a
static route for that machine's network or possibly a default route.
Following are a few examples.

Under SunOS, you'll just get the basic '... unreachable' errors:

  %%%% ping -s lab-manta
  PING lab-manta: 56 data bytes
  sendto: Network is unreachable
  ping: wrote 150.101.89.28 64 chars, ret=-1

At this point, return to looking at your 'netstat -r'
table. You should find that there is no entry for the network you are
trying to ping.

Under Solaris, you will get some additional information. In the
following example, I am again pinging lab-manta from psi:

  %%%% ping -s lab-manta
  PING lab-manta: 56 data bytes
  ICMP Host Unreachable from gateway psi (150.101.16.28) for icmp from psi
  (150.101.16.28) to lab-manta (150.101.89.28)

The above message confirms very clearly that it is the gateway 'psi'
that did not have the route, when I tried to ping from 'psi' to
'lab-manta'. Again, I would have to examine psi's routing tables at
this point.

In addition, Solaris can sometimes report back errors that remote
machines are having:

  %%%% ping -s lab-manta
  PING lab-manta: 56 data bytes
  ICMP Host Unreachable from gateway softmtv716 (150.101.16.3) for icmp from
  psi (150.101.16.28) to lab-manta (150.101.89.28)

In this case, it is the gateway 'softmtv716', which is one of my
default routers, which does not have a path to the remote machines. If
I investigate the routing tables on that machine, I will probably find
that there is no route to the 150.101.89.0 subnet.

The other possibility is that ping just reports packets being dropped:

  %%%% ping -s lab-manta
  PING lab-manta: 56 data bytes
  ^C
  ----lab-manta PING Statistics----
  7 packets transmitted, 0 packets received, 100%%%% packet loss

Verify that the remote machine is up. If it is, you
likely have a routing problem. Usually, this means that routes do exist in 
routing tables, but something along the path just drops the packets. This 
usually occurs because a router is set up wrong or because a machine is 
listed as a gateway, but is not configured to forward IP packets.

The only way to diagnose this is to examine a map of your network,
find all the IP addresses in between you and the machine that is not
responding, and then ping them one at a time. For example, I might
discover that to get to lab-manta, a packet first goes out of the
ethernet interface of psi, then into router-le0, then out of
router-le1, then into lab-manta. In this case, I would ping first psi,
then router-le0, then router-le1 then lab-manta. If router-le0
responded, but router-le1 did not, I would suspect that router was not
passing packets correctly.

(If you have such a problem, before examining the router, you should
check the netmask on your local machine with the ifconfig -a command.
An incorrectly set netmask can cause the symptoms described above.)

If you discover that a remote router or non-Sun machine is not passing
packets, SunService is unable to provide further assistance in this
routing matter. Sections 8.0 and 9.0 explain your alternatives.

If you discover that a Sun machine is not passing packets, you should
make sure that the procedures explained in Sections 3.6 and 3.7 have
not been implemented on the remote machine, to prevent it from
routing.

Packets might also be dropped if there are not paths back to your
machine. In the above example, packets might go from psi, to router,
to lab-manta fine, and then run into problems on the return trip. If
nothing else has proved fruitful, you could then examine lab-manta,
and make sure it had a path to psi, through router, and then examine
router and make sure that it also had a path to psi.

2.3.1: ping -sRv

ping can record routes for you. However, this is only of
limited use because:
- You will not get any results if packets are not being returned 
- There is a limit of 9 IP addresses
- This method depends upon source routing, which might be turned off
  on some machines, for security reasons.

Still, if you find that routes are being done incorrectly, you might
find it of some use. Here is an example of usage:

  %%%% ping -sRv lab-classic
  PING lab-classic: 56 data bytes
  64 bytes from lab-classic (150.101.89.54): icmp_seq=0. time=7. ms
  IP options:  <record route> softmtv703-bb (150.101.15.60), four-oh (150.101.89
.4), lab-classic (150.101.89.54), install-7 (150.101.89.25), four-oh-mrn (150.10
1.15.11), softmtv716 (150.101.16.3), psi (150.101.16.28),  (End of record)
^C
  ----lab-classic PING Statistics----
  1 packets transmitted, 1 packets received, 0%%%% packet loss
  round-trip (ms)  min/avg/max = 7/7/7

This was done on this network:

  16.28---(16.0)---16.3/15.60---(15.0)---15.11/89.4---(89.0)---89.54
   psi               softmtv               four-oh          lab-classic

Two things are of note here. First, the route we get returned is a
round trip, to the machine and back. Second, whenever a packet hits a
machine, ping -sRv records only the OUTGOING interface, not the
incoming interface. That is, the first time a packet goes through
softmtv, we see the address 15.60, not 16.3.

It's also interesting to see that the route was not symmetric. The
packet bounced off install-7 on the way back, which is a machine that
it did not touch on the way in. Examining that type of things can
point you towards possible errors in your routing.

If ping -sRv gives you massively incomplete info or no info, you
should ignore it, as you're probably hitting one of the limitations
described at the start of this section.

 2.4: traceroute

If you can't solve your routing problems by looking at the routing
tables and interfaces, the most useful program for solving the problem
is probably traceroute. Unfortunately, this program is not shipped
with SunOS or Solaris, and as a result is not supported by SunService.
However, you can easily find a copy on the internet.

The ftp site ftp.dfn.de contains traceroute under the /pub/tools
directory. If you consult 'archie' or other searching programs, it is
likely that you will be able to find a more local copy.

Traceroute works by sending out a set of 3 UDP packets. These packets
are initially given a Time-To-Live of 1 hop, which is then increased
to 2 hops, then to 3, etc.... until the packets reach the remote
machine. This has the effect of stepping through the routers leading
towards a destination machine, one at a time. However, it also causes
you to only see a unidirectional route, from your client to the
destination machines. It's quite possible for routes to be different
going from and to a machine, and traceroute will not standardly show
this. For example, if packets followed the following path:

    --> C -->
  A           B
    < D <

A traceroute from A to B would show: A, C, B, while a traceroute from
B to A would show: B, D, A. Only by doing both traceroutes could you
be sure of understanding the full route to and from a machine.

Once you have a copy of traceroute, you can run it to request the path
to a machine as follows:

  # traceroute lab-classic
  traceroute to lab-classic (150.101.89.54), 30 hops max, 40 byte packets
   1  rainbow-16 (150.101.16.12)  6 ms *  3 ms
   2  four-oh-mrn (150.101.15.11)  3 ms *  3 ms
   3  lab-classic (150.101.89.54)  10 ms  3 ms  3 ms

Unlike ping, traceroute lists the INCOMING port of each machine.

We can get a round trip path by running traceroute with the following
options:

  # traceroute -g remote-machine local-machine

For example, if I were logged in to psi and wanted to know the round
trip path to lab-classic, I would type:

  # traceroute -g lab-classic psi
  traceroute to psi (150.101.16.28), 30 hops max, 40 byte packets
   1  softmtv716 (150.101.16.3)  13 ms *  2 ms
   2  four-oh-mrn (150.101.15.11)  3 ms *  3 ms
   3  lab-classic (150.101.89.54)  4 ms  4 ms  3 ms
   4  install-7 (150.101.89.25)  3 ms *  3 ms
   5  four-oh-mrn (150.101.15.11)  3 ms *  3 ms
   6  softmtv716 (150.101.16.3)  3 ms *  6 ms
   7  psi (150.101.16.28)  10 ms  9 ms  3 ms

Again, we see that when the packets are headed to lab-classic, we just
get the INCOMING port listed. Oddly enough, traceroute reports the
same port on the way back. This is to avoid confusion.

However, this round trip method again uses source routing, which could
be a problem if some intervening machines disable source routing. If a
round trip invocation of traceroute seems to give bizarre results, you
should try running a traceroute in each direction, to be sure that you
are seeing the correct results.

Traceroute is more useful than ping -sRv because it will show us
partial paths, even if our destination machine is unreachable. In
addition, it will show more than 30 hops, while ping -sRv is limited
to 9.

For example, if I knew that packets were not getting to lab-manta, I
could run traceroute:

  # traceroute lab-manta
  traceroute to lab-manta (150.101.89.28), 30 hops max, 40 byte packets
   1  rainbow-16 (150.101.16.12)  6 ms *  3 ms
   2  four-oh-mrn (150.101.15.11)  3 ms *  3 ms
   3  * * *
   4  * * *
  ...

This would tell me that either four-oh is not passing packets or the
router beyond four-oh is not answering at all. I could then examine
those machines for problems.

3.0 Common How Tos

 3.1: How to Set a defaultrouter

If you want to define a default router, so that you can send all of
your non-local packets through one standard machine, simply create the
file /etc/defaultrouter. It should have as its sole contents, the name
of your router:

  %%%% cat /etc/defaultrouter
  router-1

Note that any machine listed as a defaultrouter _must_ be on the same
subnet as your machine.

On Solaris 2.4 and later, you can actually list multiple routers in
the /etc/defaultrouter file, one default router per line:

  %%%% cat /etc/defaultrouter
  router-1
  router-2
  router-3

You also need to make sure that every name which appears in
/etc/defaultrouter has an entry in your /etc/hosts file:

  %%%% cat /etc/hosts
  ...
  150.101.16.1  router-1
  150.101.16.2  router-2
  150.101.16.3  router-3

After you have created /etc/defaultrouter and made sure that any
names listed in that file are also listed in /etc/hosts, you can
reboot your machine. When it comes back up, it will automatically be
routing through the defaultrouter that you have defined.

If you have a simple network or if you are just trying to get routing
to work after your Sun machine is first installed, you probably want
to set up a defaultrouter, as is described here. Section 4.1 gives
some further advice on what type of routing to use.

 3.2: How to Add Static Routes

At any time, you can add static routes to your machine via the route
command. The standard syntax for adding a network route is:

  route add net remote-network-ip gateway-ip hop-count

remote-network-ip is the network you are adding a route to, gateway-ip
is the ip address of the gateway that leads to that remote-network.
hop-count is an integer which identifies the number of gateways that
you must go through to get to the remote-network. A network that is
directly connected to your machine would have a hop count of 0 (e.g.,
the 150.101.16.0 net in the example below). A remote-network that is
directly connected to a gateway-machine that is, in turn, on the same
network as your machine would have a hop count of 1 (e.g., the
150.101.17.0 net in the example below). Static routes are rarely added
for remote-networks with a hop count greater than 1.

For example, on the following network:

  YOUR-MACHINE---(150.101.16.0 NET)---ROUTER-1---(150.101.17.0 NET)

You could execute the following command from your machine, in order to
create a static route to the 150.101.17.0 net:

  route add net 150.101.17.0 150.101.16.1 1

(150.101.16.1 is the IP address of router-1)

The route command can also be used to add routes to remote machines.
This is typically only done when using some type of point-to-point
interface. The syntax is:

  route add host remote-machine-ip gateway-ip hop-count

i.e.:

  route add host 150.101.17.100 150.101.16.1 1

Static routes are usually only used in conjunction with a
/etc/defaultrouter. You might do so if you had a couple of networks
which you did not want to go through your defaultrouter. In that case,
you will probably want to make the routes permanent, by adding the
route commands to the /etc/init.d/inetinit file.

You can not use the above method to add static routes if you are
running in.routed, as in.routed will automatically remove them after
90 seconds. If you are using in.routed, but want to include a few
static routes, you can do this with the /etc/gateways file, explained
in Section 3.4 below.

 3.3: How to Use in.rdisc

The in.rdisc daemon is only available on Solaris machine  it depends
on multicasting, which is not present on older SunOS machines.

If you do not have an /etc/defaultrouter file on a Solaris machine,
any routing host will automatically bring up in.rdisc at boot time. As
long as there is at least one router on the same network which
advertises Router Discovery messages, in.rdisc will continue running.
If nothing responds to the initial Router Discovery queries, in.rdisc
on a routing host will automatically quit and the rc files will fall
through to in.routed.

Solaris routers will also automatically bring up in.rdisc at boot
time, if there is no /etc/defaultrouter file. They will not ever exit,
but rather will advertise via Router Discovery messages.

If you want in.rdisc to run, you should never have to do anything,
other than delete the /etc/defaultrouter file, if it exists, and make
sure that at least one Router Discovery compliant router exists on the
same subnet as your machine.

If you want to disable in.rdisc, simply rename it:

  %%%% mv /usr/sbin/in.rdisc /usr/sbin/rdisc.save

In general, in.rdisc should only be run on routing hosts if your local
network is relatively complex.

 3.4: How to Use in.routed

On a SunOS machine, in.routed will automatically be started if there
is no /etc/defaultrouter file.

On a Solaris machine, in.routed will automatically be started on
routing hosts if there is no /etc/defaultrouter file and in.rdisc
failed to find any Router Discovery compliant routers on the local
network. in.routed will automatically start on routers if there is no
/etc/defaultrouter file, even if in.rdisc is running.

You should not need to do anything, other than possibly deleting an
/etc/defaultrouter file, in order to get in.routed to run.

You can force in.routed to bring up standard network routes by putting
entries for them into the file /etc/gateways. The standard format for
that file is:

  net <remote-network-ip> gateway <gateway-ip> metric <hop-count> passive

or:

  net <remote-network-ip> gateway <gateway-ip> metric <hop-count> active

remote-network-ip, gateway-ip and hop-count are all described in
Section 3.2. The keyword 'passive' should be used if you want the
entry to be permanent. The keyword 'active' should be used if you want
in.routed to occasionally query the gateway host and delete the route
if it becomes inactive.

An example of a gateway file follows:

  %%%% cat /etc/gateways
  net 150.101.17.0 gateway 150.101.16.1 metric 1 passive
  net 0.0.0.0 gateway 150.101.16.2 metric 1 passive

The first line of this example is identical to the 'route add' command
given in Section 3.2. It creates a route for the network 150.101.17.0
through the gateway machine 150.101.16.1.

In the gateways file, the number 0.0.0.0 is a special number which
means 'default route'. Thus, the second line creates a default route
through the gateway machine 150.101.16.2.

The gateways file can also be used to add routes to specific hosts,
with the following syntax:

  host <remote-machine-ip> gateway <gateway-ip> metric <hop-count> passive

Or:

  host <remote-machine-ip> gateway <gateway-ip> metric <hop-count> active

Finally, the following directives can also be put in a gateways file:

  norip <interface>
  noripin <interface>
  noripout <interface>

These prevent RIP (in.routed) packets from either going in or out of
the specified interface (norip prevents packets from going either in
or out). For example:

  %%%% cat /etc/gateways
  norip le1

The above would prevent RIP packets from going in or out of the le1
interface. This could be exceptionally helpful if you did not want
your RIP info to be broadcast out of your le1 interface for security
reasons.

In general, routed should only be used on routing hosts if your local
network is relatively complex. In addition, there must be machines on
the local network which will correctly send out RIP packets  if there
are not, routed will fail to work correctly on a routing host.

If you wish to disable in.routed, the best way to do this is by
creating a defaultrouter file, as is described in Section 3.1.

 3.5: How To Force a Machine to Be a Router

If your machine has at least two interfaces in addition to lo0 or if
your machine has at least one point-to-point interface, it will
automatically be set up as a router.

On a SunOS machine, you will want to adjust the /etc/rc.local file,
to change the following line:

  in.routed        echo 'running routing daemon.'

To read:

  in.routed -s        echo 'running routing daemon.'

In order to insure that routing information is being advertised.

This is done for you automatically on Solaris machines.

 3.6: How to Force a SunOS Machine Not to Be a Router

If you do not want your multiple-interface SunOS machine to route
packets, you will need to rebuild the kernel. In the file
/sys/netinet/in_proto.c, change the following line:

  #define IPFORWARDING 0

to read:

  #define IPFORWARDING -1

Afterwards, config the kernel, remake it, and then install it.

It might be useful to note, that on SunOS machines, IPFORWARDING can
have three values:

  -1    Never forward IP packets
   0    Don't forward, but change IPFORWARDING to 1 if there are two
        or more interfaces
   1    Always forward IP packets

 3.7: How to Force a Solaris Machine Not to Be a Router

If you do not want your multiple-interface machine, running Solaris
2.4 or lower, to route packets, you just need to make a small change
to the file /etc/init.d/inetinit. Find the following two lines:

  numifs=`ifconfig -au   grep inet    wc -l`
  numptptifs=`ifconfig -au    grep inet    egrep -e '-->    wc -l`

And immediately after them, insert the following:

  numifs=2
  numptptifs=0

After the machine next reboots, it will not act as a router.

If you are using Solaris 2.5 or greater, it's even easier to make your
machine not route packets. Simply create an /etc/notrouter file:

  %% touch /etc/notrouter

Afterwards, reboot the machine.

It might be useful to note, that on Solaris machines, ip_forwarding can
have three values:

  0     Never Forward
  1     Always Forward
  2     Only forward when two or more interfaces are up

The above steps result in the ndd /dev/ip ip_forwarding variable being
changed from "1" to "0", which prevents routing.

 3.8: How to Set a Netmask under SunOS

In order to include a permanent netmask on your SunOS machine, you
must make an entry in the /etc/netmasks file, in the following format:

  network-address-without-zeroes        netmask

For example:

  %%%% cat /etc/netmasks
  150.101       255.255.255.0

The above would subnet the class B network, 150.101.0.0, into 254
subnets, from 150.101.1.0 to 150.101.254.0.

It is important to note that the entry in the left hand column must be
the original base network number (i.e. # for a Class A, #.# for a Class
B and #.#.# for a Class C), not the subnet.

 3.9: How to Set a Netmask under Solaris

In order to include a permanent netmask on your Solaris machine, you
must make an entry in the /etc/netmasks file, in the following format:

  network-address       netmask

For example:

  %%%% cat /etc/netmasks
  150.101.0.0   255.255.255.0

The above would subnet the class B network, 150.101.0.0, into 254
subnets, from 150.101.1.0 to 150.101.254.0.

It is important to note that the entry in the left hand column must be
the original base network number (i.e. #.0.0.0 for a Class A, #.#.0.0
for a Class B and #.#.#.0 for a Class C), not the subnet.

 3.10: How to Subnet by Bit

As was noted in Section 1.3, subnetting is actually done at the bit
level. That is, all the bits that form the IP address are masked by
the bits that form the Netmask, in order to generate the Network
Address. Here's some examples of what a netmask looks like down at
the bit level:

  255.255.255.0   = 11111111.11111111.11111111.00000000
  255.255.255.192 = 11111111.11111111.11111111.11000000

And here are a few examples of using the '192' netmask:

  IP Address            Netmask                 Network Address
  ----------            -------                 ---------------
  150.101.16.90         255.255.255.192         150.101.16.64
  150.101.16.150        255.255.255.192         150.101.16.128

Expanding out the first example should make this a bit more clear:

  150.101.16.90   = 10010110.01100101.00010000.01011010
  255.255.255.192 = 11111111.11111111.11111111.11000000
  Network Address = 10010110.01100101.00010000.01000000 = 150.101.16.64

The only caveat is that RFC 917 states that it is illegal to have a
subnet address that is all 0s or all 1s. This means that when you
split your class C, you will lose the bottom set of IP addresses and
also the top set of IP addresses.

Putting this all together, we can take a look at two examples. These
examples both assume a Class C network (that is a network between
192.*.*.* and 223.*.*.*), which you wish to subnet.

  IP ADDRESS = any Class C
  NETMASK = 255.255.255.128 = 11111111.11111111.11111111.10000000

  Subnet 1 = '0'    Illegal because subnet is all 0s
  Subnet 2 = '1'    Illegal because subnet is all 1s

  Clearly, using the 255.255.255.128 mask to subnet a Class C is useless.

  IP ADDRESS = any Class C
  NETMASK = 255.255.255.192 = 11111111.11111111.11111111.11000000

  Subnet 1 = '00'    Illegal because subnet is all 0s
                  addresses x.x.x.0 to x.x.x.63 are wasted

  Subnet 2 = '01'    network x.x.x.64, broadcast x.x.x.127
                  machines range from x.x.x.65 to x.x.x.126

  Subnet 3 = '10'    network x.x.x.128, broadcast x.x.x.191
                  machines range from x.x.x.129 to x.x.x.190

  Subnet 4 = '11'    Illegal because subnet is all 1s
                  addresses x.x.x.192 to x.x.x.255 are wasted

Thus, if you had a Class C that you wanted to divide into two subnets,
you would use the 192 netmask described above and end up with two
ranges of 63 machines each.

Now, the above all assumes that you own a Class C network, that is a
network with an network address between 192.*.*.* and 223.*.*.*.

Another common situation is the case where you are on a Class B
network (i.e. 150.101.0.0), but you actually control a network
equivalent in size to a Class C (i.e. 150.101.16.0) and you want to
subnet further. Things work about the same as above, but you do not
have to waste the top and bottom IP ranges. This is because the entire
subnet, which is nine or more bits long, is not set to all 0s or all 1s
in those cases  only the last two digits are. So, if we were subnetting
150.101.16.0 further, using the 192 subnet, our table would look like
this:

  IP ADDRESS = 150.101.16.0
  (though any class B, further subnetted will work the same)

  NETMASK = 255.255.255.192 = 11111111.11111111.11111111.11000000

  Subnet 1 = '00010000.00'    network 150.101.16.0, broadcast 150.101.16.63
                           machines range from 150.101.16.1 to 150.101.16.62

  Subnet 2 = '00010000.01'    network 150.101.16.64, broadcast 150.101.16.127
                           machines range from 150.101.16.65 to 150.101.16.126

  Subnet 3 = '00010000.10'    network 150.101.16.128, broadcast 150.101.16.191
                           machines range from 150.101.16.129 to 150.101.16.190

  Subnet 4 = '00010000.11'    network 150.101.16.192, broadcast 150.101.16.255
                           machines range from 150.101.16.193 to 150.101.16.254

Subnetting by the bit can be a pretty complex process and SunService
can not provide further assistance on initial configuration of this
type. However, if you're trying to subnet at this small of a level,
and don't totally understand how to do so, consult Section 8.0 and 9.0
for where you can get further assistance from within Sun.

 3.11: How to Protect a Gateway Machine

You might find that you wish to keep a gateway machine from sending
routing information via certain interfaces. This is usually done on a
machine that acts as a firewall of some type. Assuming that you have a
multiple interface machine that is advertising routing information
(via in.routed and in.rdisc), do two things.

First, totally turn off in.rdisc:

  %%%% mv /usr/sbin/in.rdisc /usr/sbin/rdisc.save

Second, add norip lines to the /etc/gateways file for all interfaces
that you do not want routing information to be sent through. For
example, if le1 led to an external net, you would probably add the
following entry:

  %%%% cat /etc/gateways
  norip le1

This type of setup should only be done on a firewall machine where you
are trying to decrease your net connectivity.

 3.12: How to Proxy Arp

Proxy ARPing involves one machine answering ARP requests for another
machine. This is most frequently done when a point-to-point interface
(i.e. PPP) is being used. For example, imagine the following setup:

     ppp-machine                        psi
  150.101.16.200---PPP LINK---150.101.16.28---150.101.16.0 NETWORK
   8:0:20:1:2:70               8:0:20:1:2:3

ppp-machine would have problems arping, because it is not directly on
the 150.101.16.0 network and thus it can not respond.

The following command, run on psi, would allow that machine to proxy
arp for ppp-machine:

  # arp -s ppp-machine 8:0:20:1:2:3 pub

Note that psi supplies its own ethernet address, not ppp-machine's.
This is because ARP should cause packets to go to psi, which is the
only machine which actually can pass on packets to ppp-machine.

(This command should also be put in the rc files if you wish it to be
permanent.)

 3.13: How to Remove ARP Entries

You might determine that you need to remove arp entries. This could be
done if a machine has changed ether addresses and you must have
access to it at once, or if a proxy arp has changed. You can delete a
single ARP entry with the following command:

  # arp -d machine-name

4.0 Some Frequently Asked Questions

 4.1: Miscellaneous Questions

Q: Does Sun support gated?

A: No. The public domain version of gated is available via ftp from
gated.cornell.edu, in the directory /pub/gated. However, SunService
can provide no assistance with gated.

Q: What type of routing should I use?

A: This depends on your overall network setup and your goals.

If your machine is on a simple unchanging network, that only has a few
routers connecting it, it is best to just install one static default
router. In addition, you _must_ use a static default router if you do
not (or can not) propagate either Router Discovery or RIP messages
within your network. The only deficit with this method is that you
don't have a fallback if the primary router fails. This is not a
problem in Solaris 2.4 and above, which allow multiple router entries
in the defaultrouter file. Section 3.1 explains how to set up a
default router.

If you have a complex internal network, a network that frequently
changes, or you want to make sure that your machine has fallback
opportunities if your primary router goes down, use one of
the dynamic routing programs. If you have multiple routers which all
support Router Discovery and are on a Solaris system, you will want to
run in.rdisc. Otherwise, you'll want to run in.routed.

Note that dynamic routing programs will not work unless you have
already configured the machines on your network to send out either
Router Discovery or RIP messages. Also, be aware that the
dynamic routing protocols will introduce some limited level of
additional traffic to your network.

Sections 3.3 and 3.4 explain how to set up in.routed and in.rdisc.

Q: What routes are automatically set up?

A: A few routes automatically get set up every time your machine is
booted.

First, all machines will always have a localhost route back to
themselves:

  Destination           Gateway           Flags  Ref   Use   Interface
  -------------------- -------------------- ----- ----- ------ ---------
  127.0.0.1            127.0.0.1             UH       0    591  lo0

This is standard and increases the efficiency of packets that a
machine sends to itself.

Second, Solaris machines have a special multicast route:

  224.0.0.0            150.101.16.28         U        3      0  le0

Multicast routes are used by certain programs that broadcast to a
group of machines rather than to the subnet in general.

Finally, all machines will always set up one route for each of their
interfaces. The route will be a simple one which says that all packets
which go to the interface's network should be routed through the
interface.

For example, I have one interface, le0, with an IP address of
150.101.16.28. Since I use a netmask of 255.255.255.0, this means that
my local network is 150.101.16.0 and the route looks like this:

  150.101.16.0         150.101.16.28         U        3    951  le0

(It should be noted that machines which use point-to-point interfaces,
such as PPP, will have slightly different routes  consult the PPP Tip
Sheet for more info on these.)

 4.2: Basic Routing Problems

This sections outlines a very common routing problems. The techniques
described in Section 2.0 will be needed to resolve more difficult
routing problems.

Q: Why can I only ping machines on my subnet?

A1: You are using in.routed (this is how a machine ships by default),
but are not propagating RIP packets on your network. Define
a default router, as described in Section 3.1.

A2: Your netmask is set incorrectly on the machine which can not ping.
Sections 3.8 and 3.9 describe how to define a permanent custom
netmask.

A3: You have a default router defined, but that machine is not
forwarding the IP packets. If that router is a Sun machine, you should
make sure that it has _not_ been modified as described in Section 3.6
or 3.7. If it is a non-Sun machine, SunService can not provide further
assistance in resolving why it might not be forwarding IP packets.

 4.3: in.routed Errors

Q: Why does in.routed constantly generate the following error:

  "packet from unknown router, x.x.x.x"

A: This occurs because your machine is receiving broadcast packets
from a router that is on a different subnet. Since broadcast packets
will not typically cross subnet boundaries, this usually means that
you have machines from two subnets on the same physical wire.

Often, this is a mistake  if you see this error and do not expect to
have multiple networks on the same wire, track down the
source machine (x.x.x.x) and fix its IP address.

However, there are cases where this setup might be intentional, as
outlined in RFC 1597. This would imply that you had several networks
all using the same physical wire. Unfortunately, this is not currently
supported correctly, as is described in Section 6.1.

If you are on a SunOS machine, just install patch 100283 and this
problem will go away. More information on this is contained in Section
5.1.

Under Solaris, a fix for this issue is still pending. If the errors
are too much, revert to using a default router, as
described in Section 3.1, until the problem is corrected.

Q: Why does in.routed keep bringing my PPP interface up?

A: in.routed automatically sends out RIP packets every 30 seconds.
This will keep your PPP interface up. To prevent this, you can put the
following entry in your /etc/gateways file:

  %%%% cat /etc/gateways
  norip ipdptp0

This will allow you to continue running in.routed, without have it
keep your PPP interface constantly up. Another Tip Sheet exists for
PPP which explains many routing problems associated with a PPP
interface.

 4.4: netmask Problems

Q: why is my netmasks entry being ignored?

A1: The network address listed in the netmasks file is a subnet
instead of the base network. Remember that the network address should
just be # if you are on a class A, #.# if you are on a class B, and
#.#.# if you are on a class C. Section 1.3 has a listing of which IP
addresses belong to which classes.

A2: Although you have added the netmasks to /etc/netmasks, you have
not put it in NIS. After NIS is brought up, your /etc/netmasks file is
no longer consulted. Make sure to duplicate any netmasks entries in
your NIS maps.

Q: Why do I get segmentation faults on boot after changing my netmasks
file?

A: This is a known bug in some Solaris systems. It is mentioned in
Section 6.1. If you examine your /etc/netmasks file, you will find
that it contains blank lines. Remove them, reboot, and your system
will come up clean.

Q: Does Sun support variable-length subnetting, per RFC 1219?

A: No, though there is an RFE requesting this enhancement on future
versions of Solaris. See Section 6.2.

Q1: Why can't I add a netmask for a remote subnet?
Q2: If I add a netmask for a subnet not directly connected to my
    machine, it is ignored. Why?

A: Sun systems only support netmasks for networks that are directly
connected to a machine. This means that if you try and add a netmask
for a network that is not directly connected to your machine, it will
fail. If you want to route to a remote network which is subnetted, the
correct method is to route to a machine that is directly connected to
the subnetted network and then allow that machine to route to the
appropriate subnetwork.

When RFC 1219 is implemented on Suns, this functionality might become
available.

 4.5: traceroute Problems

As was noted above, SunService does not support the third-party
traceroute package. The below questions simply note some problems that
we have run into with traceroute and the simple steps that can be
taken to correct them. Further problems or errors with traceroute
should be directed to the traceroute author.

Q: Why does traceroute fail on every other packet, under Solaris:

  # traceroute psi
  traceroute to psi (150.101.16.28), 30 hops max, 40 byte packets
   1  psi (150.101.16.28)  6 ms *  2 ms

A: Solaris introduced a variable called 'ip_icmp_err_interval' which
enforces a minimum time in between ICMP error messages. traceroute
depends upon ICMP error messages and tends to send them very fast. As
a result, if ip_icmp_err_interval is set to the default value on a
Solaris machine, traceroute's second packet will always get dropped,
as shown above. You can disable this Solaris feature by adjusting the
ndd variable:

  # ndd -set /dev/ip ip_icmp_err_interval 0

If you want this change to be permanent, add it to the file
/etc/rc2.d/S69inet.

However, since every single Solaris machine between you and your
destination must change this variable, it is often more reasonable
just to let every second packet get dropped and not worry about it.

 4.6: arp Problems

Q: Why does arp fail with the following message:

  # arp -s system 0:8:20:1:2:3 pub
  "system: No such device or address"

A: This occurs if you try and use arp to add a machine that is not on
a local network. You can only arp entries for hosts on subnets that
are directly connected to your machine.

5.0 Patches

 5.0: Patches

The following is the list of all of the routing related patches for
4.1.3, 4.1.3_u1, 4.1.4, 5.3 and 5.4. If you are having routing
problems, installing the patches is a good place to start, especially
if you recognize the general symptoms noted below.

In order for a machine to be stable, all of the recommended patches
should be installed as well. The list of recommended patches for your
operating system is available from sunsolve1.sun.com.

 5.1: Routing Patches for SunOS

100283-03 SunOS 4.1.1 4.1.2 4.1.3: in.routed mishandles gateways, multipl

  Fixes problems in the way that routed broadcasts some routes and
  corrects an error in routing to machines with multiple interfaces on
  the same physical wire.

 5.2: Routing Patches for Solaris

101318-81 SunOS 5.3: Jumbo patch for kernel (includes libc, lockd)
101945-42 SunOS 5.4: jumbo patch for kernel

  Resolve a large number of problems related to in.routed and the
  kernel routing tables. Should be installed on all 5.3/5.4 machines,
  but are an absolute necessity for routers.

 5.3: ARP Patches for SunOS

100361-04 SunOS 4.1.1 4.1.2 4.1.3: server not responding due to limits of arp

  Corrects problems caused by the arp table being limited to 512 entries.
  Should be installed if you are hitting this limit.

6.0 Known Bugs And RFEs

 6.0: Known Bugs & RFEs

The following bugs & rfes reflect currently open issues with the
routing software.

 6.1: Bugs

1182386 in.routed finds packet from unknown router

  Sun's in.routed complains if it gets a broadcast from a router on a
  different network. This is in violation of RFC 1597. This has been
  corrected under SunOS with patch 100293. However, a fix is still
  pending for Solaris.

3002202 ifconfig core dumps when /etc/netmasks ends with blank line

  On Solaris 2.4, if you inadvertently end your netmasks file with a
  blank line, ifconfig will coredump on the next boot  you will need
  to bring your system down to single user mode in order to correct
  this problem. No patch will be made available for this in 2.4, but the
  problem has been corrected in 2.5

 6.2: RFEs

1061398 allow subnet specification in /etc/netmasks for OSPF2, RFC 1219

  This RFE requests that the Sun routing programs be brought into
  accordance with RFC 1219, to allow variable length subnetting. It is
  currently being evaluated.

7.0 References

 7.1: Important Man Pages

  arp
  in.rdisc      (Solaris only)
  netmasks
  route
  routed
  routing

 7.2: Sunsolve Documents

There are a number of Sunsolve documents concerning routing,
subnetting and netmasks. The ones listed below are simply those which
contain some amount of information which is not already in this
document.

7.2.1: Infodocs

2176    Subnetting definition
2180    Routing and delivery of datagrams

7.2.2: SRDBs

4713    ARP Thrashing in Large Bridged Networks
5823    booting diskless client fails with ARP/RARP timeout
5986    Subnetting

 7.3: Sun Educational Services

[pending]

 7.4: Solaris Documentation

[pending]

 7.5: Third Party Documentation

_TCP/IP Illustrated, Volume 1_, by W. Richard Stevens, published by
Addison-Wesley Publishing Company, ISBN 0-201-63345-9

  An excellent text explaining how the various TCP/IP protocols work.
  The chapters on arp, ping, traceroute, IP routing, dynamic routing,
  broadcasting and multicasting are all very useful in gaining an
  understanding of the way that the routing protocols work.

_Internetworking With TCP/IP Volume 1_, by Douglas Comer

  Another book with good information on networking.

 7.6: RFCs

RFCs are the internet-written documents that define the specifications
of many common networking programs. RFCs can be retrieved from
nic.ddn.mil, in the /rfc directory.

A very large number of RFCs describe routing over the internet.
Included below are just some of the RFCs which most directly cover the
topics described in this document.

917   Internet subnets

  Original specs for subnets and netmasks.

950   Internet Standard Subnetting Procedure

  Additional notes on subnetting.

1219  On the Assignment of Subnet Numbers

  Suggestions for additions to the subnet specs, which have not yet
  been incorporated by Sun.

1256  ICMP router discovery messages

  Specs for the Router discovery protocol, as implemented in in.rdisc.

 8.0: Supportability

SunService is not responsible for the initial configuration of your
network routing. In addition, we can not help resolve routing problems
caused by your routers, bridges or other machines.

SunService can help resolve problems where your Sun is not sending
packets correctly to its next-hop router or where one of your Sun's
routing daemons is not responding correctly, but in such cases, the
contact must be a system administrator who has a good understanding of
the network layout and its routing design.

 9.0: Additional Support

For overall routing configuration, please contact your local
SunService office for possible consulting offerings. Sun's Customer
Relations organization can put you in touch with your local
SunIntegration or Sales office. You can reach Customer Relations at
800-821-4643.

PATCH ID: n/a
PRODUCT AREA: Gen. Network
PRODUCT: Routing
SUNOS RELEASE: any
UNBUNDLED RELEASE: n/a
HARDWARE: any