These notes look at a system's network configuration with a focus on exploration and understanding. Some of the tools used to examine the configuration also offer the means to alter the configuration, but these notes do not consider changes beyond a few coincident examples.
This suite obsoletes arp (with ip neighbor), ifconfig (with ip address), netstat (with ip route), ipmaddr (with ip maddr), and ip tunnel (with ip tunnel). The obsolete commands come from package net-tools.
You can use Tcpdump (package tcpdump) or Wireshark (packages wireshark-gnome and wireshark) to examine network traffic at the packet level. You describe what packets you wish to capture with expressions intelligible to libpcap (eponymous package); the man page is pcap-filter.
Use lshw to list your system's network devices. For example, this computer has a single Gigabit Ethernet NIC named enp3s0 using driver r8169.
=> lshw -quiet -class network *-network description: Ethernet interface product: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller vendor: Realtek Semiconductor Co., Ltd. physical id: 0 bus info: [email protected]:03:00.0 logical name: enp3s0 version: 0c serial: 50:a4:4c:ce:36:6f size: 1Gbit/s capacity: 1Gbit/s width: 64 bits clock: 33MHz capabilities: pm msi pciexpress msix vpd bus_master cap_list ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd 1000bt 1000bt-fd autonegotiation configuration: autonegotiation=on broadcast=yes driver=r8169 driverversion=2.3LK-NAPI duplex=full firmware=rtl8168g-2_0.0.1 02/06/13 ip=188.8.131.52 latency=0 link=yes multicast=yes port=MII speed=1Gbit/s resources: irq:27 ioport:e000(size=256) memory:f7d00000-f7d00fff memory:f0000000-f0003fff ⋮
You can also use ethtool or ip to look-up your NIC's hardware address:
-> ethtool --show-permaddr enp3s0 Permanent address: 50:a4:4c:ce:36:6f -> ethtool -P enp3s0 Permanent address: 50:a4:4c:ce:36:6f -> ip link show enp3s0 2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 … link/ether 50:a4:4c:ce:36:6f brd ff:ff:ff:ff:ff:ff
You can learn more about your interface's features and capabilities by running ethtool with option --show-features or --driver:
-> ethtool --show-features enp3s0 | wc --lines 57 -> ethtool --driver enp3s0 driver: r8169 version: 2.3LK-NAPI firmware-version: rtl8168g-2_0.0.1 02/06/13 ⋮
Two network devices directly connected on an Ethernet link can automatically determine what transmission rate and duplex mode to use. This Ethernet autonegotiation chooses the greatest common capabilities supported by the pair of devices. For example, when a desktop's NIC capable of 1000 Mb/s at full duplex connects to a switch topping out at 100 Mb/s full duplex, they agree to transmit at 100 Mb/s and communicate over full duplex. They also agree to the type of pause frame to use for flow control. Gigabit devices negotiate additional terms of engagement. Autonegotiation happens transparently on connection provided that both devices offer it.
A device with autonegotiation enabled can infer the hard-set line speed (10 Mb/s or 100 Mb/s) of a non-negotiating partner, a process called parallel detection, but it cannot infer the partner's duplex mode. Instead, it uses half duplex, which is Ethernet's default mode, and hopes its partner does as well. User discontent may ensue in the case of a mismatch.
Autonegotiation operates in the physical layer (i.e., PHY). Each device emits Fast Link Pulse (FLP) bursts that encode its capabilities in a link code word (LCW) or page of 16 bits. Use ethtool to see the agreement of autonegotiation or inference of parallel detection. Packet sniffers are anosmic to negotiation pulses.
Autonegotiation is optional in 10Base-T and 100Base-TX interfaces. Gigabit Ethernet devices must support autonegotiation.
Use ethtool (package ethtool) on an Ethernet interface to review link status, autonegotiation status, speed, duplex mode, and more. Here's an example for a desktop whose interface enp3s0 is connected to a Verizon Fios partner:
=> ethtool enp3s0 Settings for enp3s0: Supported ports: [ TP MII ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Supported FEC modes: Not reported Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised pause frame use: Symmetric Receive-only Advertised auto-negotiation: Yes Advertised FEC modes: Not reported Link partner advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Link partner advertised pause frame use: Transmit-only Link partner advertised auto-negotiation: Yes Link partner advertised FEC modes: Not reported Speed: 1000Mb/s Duplex: Full Port: MII PHYAD: 0 Transceiver: internal Auto-negotiation: on Supports Wake-on: pumbg Wake-on: d Current message level: 0x00000033 (51) drv probe ifdown ifup Link detected: yes
Note that the report reflects both nodes on this link. The desktop and Fios partner use Ethernet autonegotiation to determine their mutually-best interchange capability of 1000 Mb/s and full duplex.
Here's an example for a desktop whose Gigabit Ethernet (1000 Mb/s) interface p2p1 links to a Fast Ethernet (100 Mb/s) switch:
=> ethtool p2p1 Settings for p2p1: Supported ports: [ TP MII ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised pause frame use: Symmetric Receive-only Advertised auto-negotiation: Yes Link partner advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full Link partner advertised pause frame use: Symmetric Link partner advertised auto-negotiation: Yes Speed: 100Mb/s Duplex: Full Port: MII PHYAD: 0 Transceiver: internal Auto-negotiation: on Supports Wake-on: pumbg Wake-on: g Current message level: 0x00000033 (51) drv probe ifdown ifup Link detected: yes
The desktop and switch used autonegotiation to determine their mutually-best interchange capability of 100 Mb/s and full duplex. The negotiated speed falls short of the desktop's top speed of 1000 Mb/s because the partner switch peaks at 100 Mb/s.
ethtool provides a basic sanity check in diagnosing network problems since it reveals the connection status of the first link from a host machine to its LAN. It can also report packet statistics, including errors and collisions:
=> ethtool --statistics enp3s0 NIC statistics: tx_packets: 549783 rx_packets: 894180 tx_errors: 0 rx_errors: 0 rx_missed: 0 align_errors: 0 tx_single_collisions: 0 tx_multi_collisions: 0 unicast: 894180 broadcast: 0 multicast: 0 tx_aborted: 0 tx_underrun: 0
You can also use mii-diag to see basic information about your Ethernet interface and its link partner:
=> mii-diag enp3s0 Basic registers of MII PHY #32: 1000 79ed 001c c800 0de1 c9e1 006d 2801. The autonegotiated capability is 01e0. The autonegotiated media type is 100baseTx-FD. Basic mode control register 0x1000: Auto-negotiation enabled. You have link beat, and everything is working OK. Your link partner advertised c9e1: 100baseTx-FD 100baseTx 10baseT-FD 10baseT. End of basic transceiver information.
You can also use mii-tool (package net-tools) on a wired NIC to get a concise summary of link status:
=> mii-tool enp3s0 enp3s0: negotiated 1000baseT-FD flow-control, link ok
Its man page lists this command as obsolete and defers to ethtool, however.
If your computer's switch normally broadcasts LLDP packets, you can observe these as a second diagnostic of connectivity between your computer and its switch. Their presence at the expected interval indicates a functioning segment. Their absence may indicate a problem with the switch.
Ethernet devices periodically emit electrical pulses when they are not otherwise sending or receiving data frames. When a device detects pulses during traffic dearth, it is reassured that its link partner is still alive. Depending on the context, a pulse may be called a Link Integrity Test (LIT) pulse, a Normal Link Pulse (NLP), or a Fast Link Pulse (FLP) burst. If a device does not receive a frame or detect a pulse in a specified wait period, it enters a fail state until a connection is reestablished. Tcpdump and Wireshark and suchlike cannot sniff these pulses.
The Ethernet Configuration Test Protocol (ECTP) provides link-layer pinging and network discovery. It is also known as the LOOP or Loopback protocol (unrelated with the loopback interface, lo). ECTP is not an IEEE 802.x specification, however. It looks like only Cisco and DEC implemented ECTP, and its support in Linux is not clear (to me).
To list your system's network interfaces and to see their properties:
-> ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 60:a4:4c:ce:63:8f brd ff:ff:ff:ff:ff:ff
This host has a single Ethernet interface p2p1 (link/ether). p2p1 is enabled and connected to a cable or radio (flags UP and LOWER_UP), which means that p2p1 is willing and able to both transmit and receive packets (state UP). It also accepts broadcast and multicast packets from the LAN (flags BROADCAST and MULTICAST). The kernel fragments transmissions over p2p1 into packets with a maximum size of 1,500 bytes (mtu), and it queues outgoing packets using a first-in/first-out algorithm and a queue length of 1,000 packets (qdisc pfifo_fast, qlen, "qdisc" for "queue discipline"). p2p1 accepts unicast packets addressed to 60:a4:4c:ce:63:8f, its link-layer or MAC address, and it also accepts broadcast packets addressed to ff:ff:ff:ff:ff:ff.
The host also has the usual loopback device lo, which is operating (UP, LOWER_UP). Loopback does not queue packets (qdisc noqueue). Link-layer addresses do not apply nor does broadcasting (00:00:00:00:00:00).
The state reports the operational status of the interface, rather than the administrative setting reported with the interface's flags enclosed in angle brackets. Status UP indicates that the interface can operationally exchange packets with another node: The interface's transceiver is enabled, and it is connected to a live carrier. When an interface is disabled administratively, flags UP and LOWER_UP do not appear in the angle brackets:
-> ip link set p2p1 down -> ip link show p2p1 2: p2p1: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether 60:a4:4c:ce:63:8f brd ff:ff:ff:ff:ff:ff -> ip link set p2p1 up
The flag LOWER_UP connotes a layer in the network stack that is lower than the link layer; e.g. the carrier is up. When an interface is enabled but otherwise disconnected from its carrier, flag NO-CARRIER replaces flag LOWER_UP:
-> ip link show p2p1 2: p2p1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether 60:a4:4c:ce:63:8f brd ff:ff:ff:ff:ff:ff
More operational states are possible: dormant, testing, unknown, and not present. The possible administrative flags are up, down, and testing. For details, read about ifOperStatus and ifAdminStatus in RFC 1213 and RFC 2863.
You can see an interface's packet statistics by specifying option -stat:
-> ip -stat link show p2p1 2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 60:a4:4c:ce:63:8f brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 8780168 16407 0 0 0 0 TX: bytes packets errors dropped carrier collsns 1965052 17761 0 0 0 0
If there are errors, add a second -stat to see details.
To see the internet addresses assigned to your system's interfaces:
-> ip -family inet address show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 inet 192.168.1.4/24 brd 192.168.1.255 scope global dynamic p2p1 valid_lft 74853sec preferred_lft 74853sec
The report shows the IPv4 unicast and broadcast (if applicable) addresses associated with each interface (inet, brd). The scope of address 127.0.0.0 is limited to the host system because of the parochial nature of loopback (scope host). In contrast, address 192.168.1.4 on NIC p2p1 travels the LAN (scope global)
The dynamic flag and finite lifetimes (valid_lft, preferred_lft) on 192.168.1.4 reflect the address's assignment through DHCP. Compare this dynamic address with a static assignment:
-> ip address add dev p2p1 192.168.2.4/24 brd + -> ip -family inet address show dev p2p1 2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 inet 192.168.1.4/24 brd 192.168.1.255 scope global dynamic p2p1 valid_lft 69900sec preferred_lft 69900sec inet 192.168.2.4/24 brd 192.168.2.255 scope global p2p1 valid_lft forever preferred_lft forever -> ip address del dev p2p1 192.168.2.4/24
After the preferred lifetime lapses but before the valid lifetime lapses, a dynamic address becomes deprecated: Established connections continue to work, but new connections on the address are denied. Once the valid lifetime passes, the address expires for any use.
You can get IPv6 info, too:
-> ip -family inet6 address 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000 inet6 fe80::62a4:4cff:fece:638f/64 scope link valid_lft forever preferred_lft forever
You can use abbreviation -4 for "-family inet4" and abbreviation -6 for "-family inet6".
Use ipcalculator (package ipcalculator) to expound IP notation:
-> ipcalculator --nocolor 184.108.40.206/24 Address: 220.127.116.11 10111111.10101000.00000001. 00000100 Netmask: 255.255.255.0 = 24 11111111.11111111.11111111. 00000000 Wildcard: 0.0.0.255 00000000.00000000.00000000. 11111111 => Network: 18.104.22.168/24 10111111.10101000.00000001. 00000000 HostMin: 22.214.171.124 10111111.10101000.00000001. 00000001 HostMax: 126.96.36.199 10111111.10101000.00000001. 11111110 Broadcast: 188.8.131.52 10111111.10101000.00000001. 11111111 Hosts/Net: 254 Class B
Or, use ipcalc (package initscripts):
-> ipcalc -hnmpb 192.168.1.4/24 NETMASK=255.255.255.0 PREFIX=24 BROADCAST=192.168.1.255 NETWORK=192.168.1.0 HOSTNAME=desktop.home
Use command ip route for examing and modifying a host's routing tables. This command replaces route, although the man page for route is informative nonetheless.
Here's the main routing table of a desktop, named Avacado, with a single NIC, an Ethernet interface, assigned to IP address 192.168.1.4:
-> ip route show table main default via 192.168.1.1 dev p2p1 proto static metric 1024 192.168.1.0/24 dev p2p1 proto kernel scope link src 192.168.1.4
The first line describes the default route that Avacado uses to send packets to a host that is not a neighbor in Avacado's LAN. This default punts the packet to the LAN's gateway at 192.168.1.1 and lets the gateway figure out where next to route the packet. Avacado can reach the gateway because both computers are neighbors on the same LAN; that's where the second route above comes in.
The second route indicates that the subnet 192.168.1.0/24 is directly reachable via the LAN that Avacado itself belongs to. When Avacado wants to send an IP packet to Banana at 192.168.1.3, say, it encloses the IP packet in an Ethernet packet addressed to Banana's MAC address and then drops the packet on the LAN segment. Since Banana monitors the LAN's traffic, it notices and takes ownership of any such packet.
These two routes suffice to send every outgoing packet on its way.
Link-layer (e.g., Ethernet) packets incorporate the MAC addresses of both the source and destination stations. For example, when Avacado wants to send a packet to Banana, it finds the destination MAC address by first broadcasting an ARP request over the LAN. The ARP message asks the owner of IP address 192.168.1.3 to introduce itself. Its destination address is the special link-layer broadcast address (ff:ff:ff:ff:ff:ff) that all stations monitor, and its source address is Avacado's own MAC address. Banana heeds the request and remits its MAC address directly to Avacado. Now in possession of Banana's MAC address, Avacado can get down to business. Here's what that sequence looks like when Avacado (60:…:8f) pings Banana (c4:…:74):
-> tcpdump -q -e -n -t -i p2p1 'arp or icmp' ⋮ 60:a4:4c:ce:63:8f > Broadcast, ARP, length 42: Request who-has 192.168.1.3 tell 192.168.1.4, length 28 c4:17:fe:75:f7:74 > 60:a4:4c:ce:63:8f, ARP, length 60: Reply 192.168.1.3 is-at c4:17:fe:75:f7:74, length 46 60:a4:4c:ce:63:8f > c4:17:fe:75:f7:74, IPv4, length 98: 192.168.1.4 > 192.168.1.3: ICMP echo request, id 2925, seq 1, length 64 c4:17:fe:75:f7:74 > 60:a4:4c:ce:63:8f, IPv4, length 98: 192.168.1.3 > 192.168.1.4: ICMP echo reply, id 2925, seq 1, length 64
Avacado can skip the ARP request if it already knows Banana's addresses:
-> ip neigh list 192.168.1.3 192.168.1.3 dev p2p1 lladdr c4:17:fe:75:f7:74 REACHABLE
The kernel maintains several routing tables, given in file /etc/iproute2/rt_tables:
-> cat /etc/iproute2/rt_tables | grep -v '#' 255 local 254 main 253 default 0 unspec
The man page remarks that the kernel normally uses only the main table, and this is the table that show displays by default. There is also a table for local and broadcast addresses, but show displays this table only by request.
You can tell ip to show the contents of a specific table by name or index. To see the hidden local table, for example:
-> ip -4 route show table local broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1 local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1 local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1 broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1 broadcast 192.168.1.0 dev enp3s0 proto kernel scope link src 192.168.1.24 local 192.168.1.24 dev enp3s0 proto kernel scope host src 192.168.1.24 broadcast 192.168.1.255 dev enp3s0 proto kernel scope link src 192.168.1
You can also say 'all' to see the whole enchilada at once:
-> ip -4 route show table main | wc --lines 3 -> ip -4 route show table local | wc --lines 10 -> ip -4 route show table all | wc --lines 13
-> ip route show table all src 192.168.122.0/24 192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown broadcast 192.168.122.0 dev virbr0 table local proto kernel scope link src 192.168.122.1 linkdown local 192.168.122.1 dev virbr0 table local proto kernel scope host src 192.168.122.1 broadcast 192.168.122.255 dev virbr0 table local proto kernel scope link src 192.168.122.1 linkdown
The ip-route man page notes that the Linux kernel (from version 3.8) no longer caches routes. If asked to show the cache, ip will complete and return without comment.
To see what DNS servers your system specifies:
-> cat /etc/resolv.conf # Generated by NetworkManager nameserver 184.108.40.206 nameserver 220.127.116.11
To see what server is active:
-> dig | grep SERVER ;; SERVER: 18.104.22.168#53(22.214.171.124)
Or, you can use tcpdump
-> tcpdump -q -n -t -i p2p1 port 53 ⋮ IP 192.168.1.4.37112 > 126.96.36.199.domain: UDP, length 28 IP 188.8.131.52.domain > 192.168.1.4.37112: UDP, length 239
A LAN's gateway may be configured to cache host names once resolved rather than to send each query anew to an upstream name server. You can see the effect of this with dig by looking up a name you've not looked up before (or recently) and then repeating the command. The reported query times tell the tale:
-> dig fox.com | grep -P 'Query time|SERVER' ;; Query time: 29 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) -> dig fox.com | grep -P 'Query time|SERVER' ;; Query time: 2 msec ;; SERVER: 192.168.1.1#53(192.168.1.1)
In this example, the supplicant computer consults the name server on its gateway at 192.168.1.1 (port 53). On the first request, the gateway asked a full-featured name server to resolve the name into an IP address. But on the next request, the gateway simply remitted the cached answer.
Tomato firmware for residential gateways uses Dnsmasq for DHCP and DNS.
Dnsmasq stores configuration settings in /etc/dnsmasq.conf:
pid-file=/var/run/dnsmasq.pid interface=br0 resolv-file=/etc/resolv.dnsmasq addn-hosts=/etc/hosts.dnsmasq expand-hosts min-port=4096 dhcp-range=192.168.1.100,192.168.1.149,255.255.255.0,1440m dhcp-option=3,192.168.1.1 dhcp-lease-max=255 dhcp-authoritative dhcp-host=00:0F:1F:73:94:F2,192.168.1.2,1440m dhcp-host=60:A4:4C:CE:63:8F,192.168.1.4,1440m dhcp-host=00:26:B9:A0:A2:CC,C4:17:FE:75:F7:74,192.168.1.3,1440m
Note the static assignments recorded by the multiple dhcp-host settings.
As a name server, Dnsmasq maintains a cache of familiar names and forwards unrecognized names to an upstream DNS server:
-> cat /etc/resolv.dnsmasq nameserver 184.108.40.206 nameserver 220.127.116.11 nameserver 18.104.22.168
It also keeps its own list of local-host names for static DHCP assignments:
-> cat /etc/hosts.dnsmasq 192.168.1.1 gateway 192.168.1.2 old-desktop.home 192.168.1.4 desktop.home 192.168.1.3 gogo.home
Active DHCP leases live in /var/lib/misc/dnsmasq.leases; for example:
cat /var/lib/misc/dnsmasq.leases 86400 c4:17:fe:75:f7:74 192.168.1.3 gogo 01:c4:17:fe:75:f7:74 85332 60:a4:4c:ce:63:8f 192.168.1.4 desktop 01:60:a4:4c:ce:63:8f
Well-known mappings between services and ports go in /etc/services:
-> grep -P 'boot(ps|pc)' /etc/services bootps 67/tcp # BOOTP server bootps 67/udp bootpc 68/tcp dhcpc # BOOTP client bootpc 68/udp dhcpc
The man pages is services.
A MAC address's first three bytes constitute an Organizationally Unique Identifier (OUI) assigned to the NIC's vendor. Here's an example from an Asus motherboard with a built-in NIC labeled enp3s0:
=> ethtool --show-permaddr enp3s0 Permanent address: 60:a4:4c:ce:63:8f
Wireshark's list of OUIs links the first three bytes, 60:a4:4c, to vendor "ASUSTek COMPUTER INC". A vendor may have multiple OUIs.