Network Nightmare Manual - v1.0.9

Introduction

This device enables you to simulate different network conditions, via bandwidth limits, link latency, and packet loss.

Enter settings by selecting the letter of the configuration item (i.e. B) or by using the arrow keys, then press Enter to select. Changes will not be saved persistently except through the configuration menu.

Networking in General

The leftmost and center interfaces (net2 and net1) form a bridge over which the selected network conditions will be effected. The rightmost interface is optional and for management over the network (see Z to configure passwords and IP addresses). You may also configure the interfaces to route packets (in this case, no 2 networks may be on the same subnets).

To wire the device, treat it like a PC, i.e. if connecting to a switch then use a straight through cable, but if connecting to a PC then use a crossover cable. For the serial connection use a null modem to connect to a PC. By default the interfaces are set to autonegotiate for speed and duplex. In that mode they should self-configure to talk to other autonegotiating devices (even gigE, though the communication will be at 100 Mbps full duplex). However if the other devices are set to fixed speed/dupex, a duplex mismatch will likely result, and you will want to use the configuration system to change into a corresponding fixed mode on the Network Nightmare.

Main Menu

B - set the bandwidth limit to effect between networks 1 and 2. This is set in kilobits/second. For Megabits, multiply by 1000. The Network Nightmare should function correctly at rates up to 25 Mbps. And it will attempt to handle rates above that. This setting reflects the lack of bandwidth on a wide area connection. To model a complex network (512 Kbps DSL to the Internet to a T1), it is best to use the speed of the lowest link. Asymmetrical rates are supported through the Advanced mode, select this from the Configure System page to setup.

R - Enforce a delay between nets 1 and 2. This delay ("latency" or "Round Trip Time (RTT)") represents the effects of distance in wide area networks. You can test what your real-life latency is by using "ping" across the network, the average time it reports should be used as the Round Trip Time (or the max it reports, for worst case).

L - Loss occurs in some networks, this setting enables you to have the Network Nightmare lose a certain fraction of packets at random. This can represent collisions in shared media physical connections, satellite networks, or many other environments. This fraction is the loss for each direction, i.e. 0.01 means 1% loss from 1 to 2 and 1% loss back from 2 to 1.

T - Traffic monitor (for experts). This will run the tcpdump command on the bridged interface (network 1). This will print out packets as they go by on the network. You can configure the options for tcpdump on the Configure System menu.

If saving tcpdumps to /tmp, note that there is no disk on the Network Nightmare, so you will likely want to limit your tcpdump to just the important traffic for your test, this is available in the configuration options. Alternately, map or mount a remote drive. When doing this, be sure that the route to the remote drive does not pass through network 1's port! (Or else you may end up recording your own monitoring output.)

N - Network monitor. This will print information about the network on network 1 (in any of the modes). The information printed is:

time CONNECTIONS open: [number of currently open] total: [total ever open connection]  max-sim: [peak # open connections so far]
time BYTES tcp: [num tcp bytes]/[num tcp packets] udp: [num udp bytes]/[num udp packets] other: [num other bytes]/[num other packets]
At the end of the run (hit control-c) summary information is printed. Files recording information about your traffic will be placed in /tmp. These include "top talkers," "Port usage," "SMB signing report," and some other summary files. For more information press ? in the menu with the output file selections.

Note that in very lightly trafficked networks, this display can fall behind the actual network traffic due to buffering within the NN. Once events are displayed their timestamps will be correct. (The timezone can be set in the Configuration, the NN will use rdate to set its time, if it has access to external networks with timeservers, otherwise the date will be somewhat arbitrary.)

! - Shell access (for experts). There are always things that people want to do that the interface will not support. This option provides access to the underlying FreeBSD system. If you break the unit though this access, you will likely need to return it for support. Though try a reboot first!

I - Interface watch - this watches net1 and shows the number of

   packets  errs      bytes    packets  errs      bytes colls drops
that are input and output on net1

W - Watch bandwidth on net1 - this shows the instantaneous datarate, in and out of net1 along with the average (since the Watch process was started) and peak.

Z - configure system - This enables you to set passwords, network settings, tcpdump options and upgrade the unit. See Help within the menu.

X - Log out (useful for network access to control the simulator, not needed on serial console access) S - Shutdown (not normally required, you can just power down the unit as long as it is not in the middle of an upgrade or parameter-save).

Sample settings: These may be representative of your network, they may not.

Use                    Bandwidth    RTT     Loss 
Office-Home(DSL)       512          35      0.01    
Office-Data center(T1) 1500         45      0.003
Office-Office(cross US) 768         100     0.008
Satellite link         2000         650     0.03
NY-Tokyo Internet      1000         225     0.02
Detroit-new Deli E1    2000         285     0.01
SF-London Internet     1500         210     0.02
Data Center-Data Center disaster recovery DS3
                       45000        50      0.00

Configuration Menu

N - Go to the Networking/IP submenu

P - Set the root password. Often used to enable a scp to the Network Nightmare of a software upgrade, e.g. scp nn109.img.gz root@192.168.1.101:/var/tmp/ If you ssh to the Network Nightmare as root, you will login with a shell prompt.

W - Set the wansimuser password. Often used to enable network control of WAN simulator parameters. Login with "ssh wansimuser@NN-IP" to reach the menu system. There is a default password (1.0.9) of NN2wansim (certain models have a ! at the end of the password)

M - Mount/Map a drive (goes to submenu)

T - Configure TCPdump options. There are very many options, a common use is "host 192.168.1.11" which would record only the traffic to/from this particular host. To see all the options, enter the shell and type: tcpdumpman or select T, then CANCEL instead of selecting OK.

D - Set the timezone - menu driven selection of the device timezone.

L - Load Wansim Settings - Several files of typical network conditions have been provided (all are for the non-advanced mode). These will set bandwidth/RTT/loss rates.

F - Save Wansim settings - create a new template file to reload later.

S - All settings last only until power is removed/the Network Nightmare is rebooted. Use this option to save setting across power cycling. Be sure the settings are correct first!

A - Advanced Mode - With 1.0.9 the NN supports two new modes: asymmetric settings and 3way mode. In Asymmetric mode you can work with either bridged interfaces (net1/net2) or routed, but you can emulate different properties in the different directions, such as 1.5 Mbps from net1->net2, but only 386 kbps in the reverse direction. 3way mode works only for routed mode, but enables arbitrary settings in all 6 directions through the device.

U - Used to upgrade to a new release of the Network Nightmare firmware. There are two ways to upgrade, copying an image down or mounting it. To mount, see the mount menu, and mount/map a drive with the new image onto the NetworkNightmare (note an image in /tmp will take priority over one in /tmp/mnt). To copy, make sure the Network Nightmare has an IP address and password, then copy to the Network Nightmare (i.e. scp nn109.img.gz root@192.168.1.101:/tmp/)

Alternate ways to copy are to: - Just set an IP, then drop to the shell, cd /tmp, and scp from another machine to the NN - Use a terminal program supporting zmodem, drop to the shell, cd to /tmp, execute rz (or zmrx), then send with the terminal program (use binary mode, and it's slow!)

After the image has been completely copied, use option U to upgrade. Note that a copied-but-not installed image will be lost in a reboot. A partial copy of an image or a partial install will likely corrupt your device and require you to return it for servicing - possibly at additional cost if you have no Support contract. For safety, an Upgrade will delete all saved settings (IP, password, emulation settings, etc)

X - Return to the main menu.

Configuration: Networking

I - This enables you to change the management IP address/netmask and gateway. This is the network connection next to power. Note that to use this connection, you must also set a password for the root or wansimuser users. The interface is defined as: IP-address/subnet-mask-length The subnet mask length is how many bits to set, i.e. CIDR notation where 255.255.255.0 is /24. You can then set the default gateway, which is needed if you wish to reach the Network Nightmare other than from the LAN. Finally, you will be prompted for the IP of the nameserver. The defaults are: 192.168.1.101/24, 192.168.1.1, and 127.0.0.1 (none).

D - Set the default gateway (also availble under I). Note that if you employ routed mode, the gateway may be off of any interface, not just the management interface.

B - The default Network Nightmare setting is to Bridge ports 1 and 2. This enables "in path" insertion for simple tests. If you have switched to Routing, then this option will revert to bridging. Use this setting if using port mirroring for tcpdump/network mon.

R - Switch to routing mode. To use this mode, you will need to assign IP addresses to ports 1 and 2. The format is as above for (I). Be aware that other routers will also need to know how to get back to the Network Nightmare! No routing protocols are supported, only static routes. You may NOT use the same LAN subnet on any of the 3 interfaces, if you attempt to do so, you will receive an error.

A - Set Autonegotiate vs Fixed speed/duplex. By default the Network Nightmare will autonegotiate for speed and duplex settings on its interfaces. If you need to change these to fixed, select this item.

Q - Query for autnegotiation statis - shows the output of ifconfig. For any interfaces set to autonegotiate, this will reveal what setting they negotiated to.

X - Return to the main menu.

Configuration: Speed and Duplex

Select which setting to use
  Auto - use the auto-negotiation protocol to learn which setting
  10H - 10 Mbps, half duplex (old hub)
  10F - 10 Mbps, full duplex (old switch)          
  100H: 100 Mbps, half duplex (cheap router)
  100F: 100 Mbps, full duplex (modern switch/router)

Gigabit Ethernet is not supported (it will normally auto-negotiate to 100F).

Configuration: Mapping/mounting a remote drive

Mapping/mounting a drive allows one to more easily: save tcpdump information, copy off monitoring statistics files, or upgrade the firmware. (Other options are to drop to the shell to use scp or zmtx, or to remotely scp from /tmp.)

Be warned that if mapping a drive, it should be off the network served on network 0 (mgmt), otherwise the network traffic to the drive may interfere with your WAN emulation/WAN monitoring.

U - Unmount whatever is mounted

N - Mount a Network File System - such as a Linux or Solaris drive which is exported.

W - Map a Windows network "share" drive.

S - Change the default directory (for tcpdump and network monitor) to use /tmp/mnt (or /tmp).

L - List the mounted drive.

X - Return to the main menu.

Advanced Mode

The Network Nightmare supports 2 advanced modes - in one case there will be different network characteristics from net1->net2 compared to net2->net1, in the other the third interface is also usable for network emulation. Note that this later configuration (obviously) requires th euse of routing mode.

N - normal mode - revert to the original way the Network Nightmare operated with just one setting for bidirectional traffic between net1 and net2

A - Asymmetric mode you can work with either bridged interfaces (net1/net2) or routed, but you can emulate different properties in the different directions, such as 1.5 Mbps from net1->net2, but only 386 kbps in the reverse direction.

3 - 3way mode works only for routed mode, but enables arbitrary settings in all 6 directions through the device. i.e. traffic from net1 to net2, net1 to mgmt, etc.

I - Infinite queues (dont drop due to delayed packets) default

Q - Set fixed queue size in packets

B - Use bandwidth * delay to determine queue size

X - Return to the main menu.

Statistics (Monitoring)

Only on the PLUS model

Note that the monitoring will not begin until a significant amount of traffic has been seen, so it is not suitable for extremely low traffic evaluation. While running the monitor will print its status every 10 seconds, the output looks like:

1100650592.535386  CONNECTIONS open: 2 (2)      total: 2 (2)    max-sim: 2 (2)
1100650592.535386  BYTES tcp: 1580404/57        udp: 0/0        other: 0/0

The first number (1100650592.535386) is a timestamp (seconds.msecs) If you have not used rdate with external access or otherwise set the time, this will be from an arbitrary starting point, but will be useful for relative timing in any case.

The Connections line describes the TCP connections currently extant:

The Traffic line gives the number of BYTES sent of each type, followed by the number of packets, e.g. bytes/packets. TCP is a reliable protocol, which is most common on networks. UDP is a lossy protocol used for some applications needing low delay, and other comprises all other traffic types, such as GRE or certain VPN protocols.

The monitoring process will produce files in /tmp (or /tmp/mnt) to explore different aspects of the traffic it saw. You can browse the files from this menu, or copy them off of the Network Nightmare. In addition it will produce several .csv files which are useful to load into other tools, such as Microsoft Excel(TM).

S - Summary of Traffic (traffic.dat) - provides summary statistics for the monitored period. These include the length of trace (secs), number of bytes sent, average speed, average RTT, a breakdown into the number of retransmitted bytes and duplicate acks, #connections, SMB signing statistics, byte and packet counts broken down into TCP/UDP/other, and the peak number of simultaneous connections.

P - Traffic Summary by port (traffic_byport.dat) - How much data was sent on each TCP port. Note that BOTH ports of a connection are summarized. i.e. if a client connects to a web-server on port 80, then that traffic counts towards port 80, but it ALSO counts toward the ephemeral port the client received in order to open this connection. Thus each byte sent is represented twice in this summary.

T - Top Talkers (toptalkers.dat) - The connections seen, sorted by which sent the most traffic (top 20), plus the rate at which they sent the traffic.

A - Top Talking Applications (topapps.dat) - similar to traffic_byport, but just the byte counts and ports.

G - Most talking machines (mosttalking.dat) - Identifies the IPs which sent the most data during the session.

Y - CIFS/SMB Signing status (smb_conns.dat) - SMB signing is an obscure CIFS setting of interest to WAFS vendors who need to know if traffic has had cryptographic checksums added to it by Microsoft.

M - Monitor output (monitor.out) - The recapitulates the output from during the monitoring session.

E - Open connections at monitor end (rt_open.dat) - this identifies the connections which had NOT been closed at the end of the monitor run.

U - Every connection (all_connections.dat) - displays stats (bytes, time, and rate) for each connection seen.

X - Return to the main menu.