netbsd 5 and high speed internet

[Michael D. Spence]

I have been using an old 1ghz 1 or 2Mb PC to run netbsd 5 and act as a
firewall for a number of years. It is not used for anything but this and
has been working great, until yesterday when I upgraded my cable modem to
100 Mbs. If I run a speed test on a windows machine inside the firewall,
things are usually really slow (although I have seen 35 Mbs) whereas if I
plug the windows machine directly into the cable modem, I get speeds in the
mid 80s to mid 90s.

Is there any tuning I should try or a place I can read about this? Here are
the NICs I have:


ex0: flags=8a63<UP,BROADCAST,NOTRAILERS,RUNNING,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1500
address: 00:60:08:9a:53:6c
media: Ethernet 100baseTX full-duplex
status: active
inet 192.168.0.4 netmask 0xffffff00 broadcast 192.168.0.255
inet6 fe80::260:8ff:fe9a:536c%ex0 prefixlen 64 scopeid 0x2
vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
address: 00:24:01:08:93:22
media: Ethernet 100baseTX full-duplex
status: active
inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255
inet6 fe80::224:1ff:fe08:9322%vr0 prefixlen 64 scopeid 0x3

ex0 faces the work, vr0 is on my internal network.

[Thor Lancelot Simon]

Post by Michael D. Spence
I have been using an old 1ghz 1 or 2Mb PC to run netbsd 5 and act as a
firewall for a number of years.

This isn't a very precise description of your machine but I also think it
cannot be accurate. I don't think you ever could get a machine with a 1GHz
CPU and only 2MB of RAM.

Can you perhaps paste us /var/run/dmesg.boot from this system?

Thanks!

Thor

[Michael D. Spence]

As you said, not accurate at all, sorry. I bought it a long time ago. Here
is dmesg.boot:

bash-4.2# cat /var/run/dmesg.boot
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
2006, 2007, 2008
The NetBSD Foundation, Inc. All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.

NetBSD 5.0 (GENERIC) #0: Sun Apr 26 18:50:08 UTC 2009

***@b6.netbsd.org:/home/builds/ab/netbsd-5-0-RELEASE/i386/20090426022
9Z-obj/home/builds/ab/netbsd-5-0-RELEASE/src/sys/arch/i386/compile/GENERIC
total memory = 255 MB
avail memory = 239 MB
timecounter: Timecounters tick every 10.000 msec
timecounter: Timecounter "i8254" frequency 1193182 Hz quality 100
HP Pavilion 04 P5269A-ABA 7955 (70000TG101AMAZO)
mainbus0 (root)
cpu0 at mainbus0 apid 0: Intel 686-class, 1494MHz, id 0xf12
ioapic0 at mainbus0 apid 2: pa 0xfec00000, version 20, 24 pins
acpi0 at mainbus0: Intel ACPICA 20080321
acpi0: X/RSDT: OemId <ASUS ,P4B_LA ,42302e31>, AslId <MSFT,31313031>
acpi0: SCI interrupting at int 9
acpi0: fixed-feature power button present
timecounter: Timecounter "ACPI-Fast" frequency 3579545 Hz quality 1000
ACPI-Fast 24-bit timer
acpibut0 at acpi0 (PWRB, PNP0C0C): ACPI Power Button
attimer1 at acpi0 (TMR, PNP0100): AT Timer
attimer1: io 0x40-0x43 irq 0
pcppi1 at acpi0 (SPKR, PNP0800)
pcppi1: io 0x61
midi0 at pcppi1: PC speaker (CPU-intensive output)
sysbeep0 at pcppi1
npx1 at acpi0 (COPR, PNP0C04)
npx1: io 0xf0-0xff irq 13
npx1: reported by CPUID; using exception 16
FDC0 (PNP0700) at acpi0 not configured
ECP (PNP0401) at acpi0 not configured
UAR1 (PNP0501) at acpi0 not configured
UAR2 (PNP0501) at acpi0 not configured
pckbc1 at acpi0 (PS2K, PNP0303): kbd port
pckbc1: io 0x60,0x64 irq 1
mpu0 at acpi0 (MIDI, PNPB006)
mpu0: io 0x330-0x333 irq 10
midi1 at mpu0: Roland MPU-401 MIDI UART (CPU-intensive output)
joy0 at acpi0 (GAME, PNPB02F)
joy0: io 0x200-0x20f
joy0: joystick not connected
apm0 at acpi0: Power Management spec V1.2
attimer1: attached to pcppi1
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
pchb0 at pci0 dev 0 function 0
pchb0: vendor 0x8086 product 0x1a30 (rev. 0x03)
agp0 at pchb0: aperture at 0xf8000000, size 0x4000000
ppb0 at pci0 dev 1 function 0: vendor 0x8086 product 0x1a31 (rev. 0x03)
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled
vga1 at pci1 dev 0 function 0: vendor 0x10de product 0x002d (rev. 0x15)
wsdisplay0 at vga1 kbdmux 1: console (80x25, vt100 emulation)
wsmux1: connecting to wsdisplay0
drm at vga1 not configured
ppb1 at pci0 dev 30 function 0: vendor 0x8086 product 0x244e (rev. 0x12)
pci2 at ppb1 bus 2
pci2: i/o space, memory space enabled
fxp0 at pci2 dev 8 function 0: Intel i82562 Ethernet, rev 3
fxp0: interrupting at ioapic0 pin 20
fxp0: Ethernet address 00:e0:18:4a:51:e4
inphy0 at fxp0 phy 1: i82562ET 10/100 media interface, rev. 0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
ex0 at pci2 dev 9 function 0: 3Com 3c905-TX 10/100 Ethernet (rev. 0x0)
ex0: interrupting at ioapic0 pin 21
ex0: MAC address 00:60:08:9a:53:6c
nsphy0 at ex0 phy 24: DP83840 10/100 media interface, rev. 1
nsphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vr0 at pci2 dev 10 function 0: VIA VT6105 (Rhine III) 10/100 Ethernet
vr0: interrupting at ioapic0 pin 22
vr0: Ethernet address: 00:24:01:08:93:22
ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface
ukphy0: OUI 0x0002c6, model 0x0034, rev. 4
ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vendor 0x11c1 product 0x044e (miscellaneous communications, revision 0x02)
at pc
i2 dev 11 function 0 not configured
ichlpcib0 at pci0 dev 31 function 0
ichlpcib0: vendor 0x8086 product 0x2440 (rev. 0x12)
timecounter: Timecounter "ichlpcib0" frequency 3579545 Hz quality 1000
ichlpcib0: 24-bit timer
ichlpcib0: TCO (watchdog) timer configured.
piixide0 at pci0 dev 31 function 1
piixide0: Intel 82801BA IDE Controller (ICH2) (rev. 0x12)
piixide0: bus-master DMA support present
piixide0: primary channel wired to compatibility mode
piixide0: primary channel interrupting at ioapic0 pin 14
atabus0 at piixide0 channel 0
piixide0: secondary channel wired to compatibility mode
piixide0: secondary channel interrupting at ioapic0 pin 15
atabus1 at piixide0 channel 1
uhci0 at pci0 dev 31 function 2: vendor 0x8086 product 0x2442 (rev. 0x12)
uhci0: interrupting at ioapic0 pin 19
usb0 at uhci0: USB revision 1.0
uhci1 at pci0 dev 31 function 4: vendor 0x8086 product 0x2444 (rev. 0x12)
uhci1: interrupting at ioapic0 pin 23
usb1 at uhci1: USB revision 1.0
auich0 at pci0 dev 31 function 5: i82801BA (ICH2) AC-97 Audio
auich0: interrupting at ioapic0 pin 17
auich0: ac97: Analog Devices AD1885 codec; headphone, Analog Devices Phat
Stereo
auich0: ac97: ext id 1<VRA>
isa0 at ichlpcib0
lpt0 at isa0 port 0x378-0x37b irq 7
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
auich0: measured ac97 link rate at 47307 Hz, will use 48000 Hz
audio0 at auich0: full duplex, mmap, independent
fd0 at fdc0 drive 0: 1.44MB, 80 cyl, 2 head, 18 sec
uhub0 at usb0: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhub1 at usb1: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
wd0 at atabus0 drive 0: <WDC WD400EB-00CPF0>
wd0: drive supports 16-sector PIO transfers, LBA addressing
wd0: 38166 MB, 77545 cyl, 16 head, 63 sec, 512 bytes/sect x 78165360 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd0(piixide0:0:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using
DMA)
atapibus0 at atabus1: 2 targets
cd0 at atapibus0 drive 1: <TOSHIBA DVD-ROM SD-M1502, Y100300348, 1816> cdrom
rem
ovable
cd0: 32-bit data port
cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33)
cd0(piixide0:1:1): using PIO mode 4, Ultra-DMA mode 2 (Ultra/33) (using DMA)
Kernelized RAIDframe activated
pad0: outputs: 44100Hz, 16-bit, stereo
audio1 at pad0: half duplex
boot device: wd0
root on wd0a dumps on wd0b
root file system type: ffs
wsdisplay0: screen 1 added (80x25, vt100 emulation)
wsdisplay0: screen 2 added (80x25, vt100 emulation)
wsdisplay0: screen 3 added (80x25, vt100 emulation)
wsdisplay0: screen 4 added (80x25, vt100 emulation)
bash-4.2#

-----Original Message-----
Sent: Sunday, April 20, 2014 7:43 PM
To: Michael D. Spence
Cc: NetBSD
Subject: Re: netbsd 5 and high speed internet

Post by Michael D. Spence
I have been using an old 1ghz 1 or 2Mb PC to run netbsd 5 and act as a
firewall for a number of years.

This isn't a very precise description of your machine but I also think it

cannot

be accurate. I don't think you ever could get a machine with a 1GHz CPU

and

only 2MB of RAM.
Can you perhaps paste us /var/run/dmesg.boot from this system?
Thanks!
Thor

[Thor Lancelot Simon]

Post by Michael D. Spence
As you said, not accurate at all, sorry. I bought it a long time ago. Here
fxp0 at pci2 dev 8 function 0: Intel i82562 Ethernet, rev 3
fxp0: interrupting at ioapic0 pin 20
fxp0: Ethernet address 00:e0:18:4a:51:e4

It looks like you're not curerntly using this particular Ethernet
interface -- can you try moving the connection that is currently
on "vr0" to it?

Thor

[Michael D. Spence]

I will try this tonight after work.

-----Original Message-----
Sent: Sunday, April 20, 2014 8:24 PM
To: Michael D. Spence
Cc: 'NetBSD'
Subject: Re: netbsd 5 and high speed internet

Post by Michael D. Spence
As you said, not accurate at all, sorry. I bought it a long time ago.
fxp0 at pci2 dev 8 function 0: Intel i82562 Ethernet, rev 3
fxp0: interrupting at ioapic0 pin 20
fxp0: Ethernet address 00:e0:18:4a:51:e4

It looks like you're not curerntly using this particular Ethernet

interface -- can

you try moving the connection that is currently on "vr0" to it?
Thor

[Greg Troxel]

I have been using an old 1ghz 1 or 2Mb PC to run netbsd 5 and act as a
firewall for a number of years. It is not used for anything but this and

Probably your CPU is adequate, and either there's something wrong
unexpected, or those interfaces are too crufty for 100 Mbps. If you
have PCI slots and can try some other kinds, especially if you can find
an intel card that is fxp(4), or DEC that is tlp(4), it might help.

Run 'netstat -s' on the gw before and after the speed test, and diff the
results. Try running wget or something on the gateway itself to
download a large file.

It's remarkable that you are getting 80-90 Mbps on a cable modem.

[Michael D. Spence]

Well, it is much faster, but I have only their speed tester gadget to tell.

Here is the diff:

bash-4.2# diff before.txt after.txt
27c27
< 60485 total packets received
---

81267 total packets received

42c42
< 3220 packets for this host
---

3422 packets for this host

44c44
< 56039 packets forwarded (0 packets fast forwarded)
---

76594 packets forwarded (0 packets fast forwarded)

48c48
< 2889 packets sent from this host
---

3083 packets sent from this host

57,58c57,58
< 1612 packets sent
< 1557 data packets (136653 bytes)
---

1634 packets sent
1579 data packets (137813 bytes)

60c60
< 44 ack-only packets (614 delayed)
---

44 ack-only packets (635 delayed)

66,67c66,67
< 1346 packets received
< 1024 acks (for 136618 bytes)
---

1376 packets received
1045 acks (for 137778 bytes)

70c70
< 630 packets (38956 bytes) received in-sequence
---

651 packets (40048 bytes) received in-sequence

88c88
< 888 segments updated rtt (of 890 attempts)
---

909 segments updated rtt (of 911 attempts)

96c96
< 307 correct data packet header predictions
---

316 correct data packet header predictions

121c121
< 1853 datagrams received
---

2025 datagrams received

128,130c128,130
< 1813 delivered
< 788 PCB hash misses
< 1164 datagrams output
---

1985 delivered
860 PCB hash misses
1336 datagrams output

132c132
< 1165 total packets received
---

1202 total packets received

146c146
< 30 packets not forwardable
---

32 packets not forwardable

148c148
< 64 packets sent from this host
---

66 packets sent from this host

151c151
< 17 output packets discarded due to no route
---

18 output packets discarded due to no route

156c156
< 30 multicast packets which we don't join
---

32 multicast packets which we don't join

158,159c158,159
< hop by hop: 26
< ICMP6: 1139
---

hop by hop: 27
ICMP6: 1175

162c162
< 1165 one ext mbufs
---

1202 one ext mbufs

175c175
< multicast listener report: 62
---

multicast listener report: 64

182,183c182,183
< multicast listener query: 26
< router advertisement: 1109
---

multicast listener query: 27
router advertisement: 1143

208,209c208,209
< 1612 packets sent
< 1557 data packets (136653 bytes)
---

1634 packets sent
1579 data packets (137813 bytes)

211c211
< 44 ack-only packets (614 delayed)
---

44 ack-only packets (635 delayed)

217,218c217,218
< 1346 packets received
< 1024 acks (for 136618 bytes)
---

1376 packets received
1045 acks (for 137778 bytes)

221c221
< 630 packets (38956 bytes) received in-sequence
---

651 packets (40048 bytes) received in-sequence

239c239
< 888 segments updated rtt (of 890 attempts)
---

909 segments updated rtt (of 911 attempts)

247c247
< 307 correct data packet header predictions
---

316 correct data packet header predictions

300,301c300,301
< 137 packets sent
< 133 reply packets
---

140 packets sent
136 reply packets

303c303
< 164 packets received
---

169 packets received

305,306c305,306
< 156 valid request packets
< 41 broadcast/multicast packets
---

161 valid request packets

============================================================================
===============
Here is a diff of netstat -s after downloading a 22MB tar file:

bash-4.2# diff after.txt bigfile.txt
27c27
< 81267 total packets received
---

99116 total packets received

42c42
< 3422 packets for this host
---

19680 packets for this host

44,45c44,45
< 76594 packets forwarded (0 packets fast forwarded)
< 264 packets not forwardable
---

78065 packets forwarded (0 packets fast forwarded)
266 packets not forwardable

48c48
< 3083 packets sent from this host
---

14248 packets sent from this host

57,58c57,58
< 1634 packets sent
< 1579 data packets (137813 bytes)
---

12747 packets sent
2204 data packets (181041 bytes)

60c60
< 44 ack-only packets (635 delayed)
---

5840 ack-only packets (10829 delayed)

63,64c63,64
< 0 window update packets
< 8 control packets
---

4684 window update packets
16 control packets

66,67c66,67
< 1376 packets received
< 1045 acks (for 137778 bytes)
---

17561 packets received
1565 acks (for 181010 bytes)

70,71c70,71
< 651 packets (40048 bytes) received in-sequence
< 6 completely duplicate packets (0 bytes)
---

16509 packets (22609416 bytes) received in-sequence
10 completely duplicate packets (0 bytes)

74c74
< 2 out-of-order packets (0 bytes)
---

40 out-of-order packets (49232 bytes)

82c82
< 0 connection requests
---

4 connection requests

84,85c84,85
< 9 connections established (including accepts)
< 34 connections closed (including 0 drops)
---

13 connections established (including accepts)
39 connections closed (including 0 drops)

88c88
< 909 segments updated rtt (of 911 attempts)
---

1401 segments updated rtt (of 1402 attempts)

96c96
< 316 correct data packet header predictions
---

15723 correct data packet header predictions

121c121
< 2025 datagrams received
---

2098 datagrams received

128,130c128,130
< 1985 delivered
< 860 PCB hash misses
< 1336 datagrams output
---

2058 delivered
905 PCB hash misses
1388 datagrams output

132c132
< 1202 total packets received
---

1357 total packets received

146c146
< 32 packets not forwardable
---

39 packets not forwardable

148c148
< 66 packets sent from this host
---

72 packets sent from this host

151c151
< 18 output packets discarded due to no route
---

19 output packets discarded due to no route

156c156
< 32 multicast packets which we don't join
---

39 multicast packets which we don't join

158,159c158,159
< hop by hop: 27
< ICMP6: 1175
---

hop by hop: 30
ICMP6: 1327

162c162
< 1202 one ext mbufs
---

1357 one ext mbufs

175c175
< multicast listener report: 64
---

multicast listener report: 70

182,183c182,183
< multicast listener query: 27
< router advertisement: 1143
---

multicast listener query: 30
router advertisement: 1288

208,209c208,209
< 1634 packets sent
< 1579 data packets (137813 bytes)
---

12747 packets sent
2204 data packets (181041 bytes)

211c211
< 44 ack-only packets (635 delayed)
---

5840 ack-only packets (10829 delayed)

214,215c214,215
< 0 window update packets
< 8 control packets
---

4684 window update packets
16 control packets

217,218c217,218
< 1376 packets received
< 1045 acks (for 137778 bytes)
---

17561 packets received
1565 acks (for 181010 bytes)

221,222c221,222
< 651 packets (40048 bytes) received in-sequence
< 6 completely duplicate packets (0 bytes)
---

16509 packets (22609416 bytes) received in-sequence
10 completely duplicate packets (0 bytes)

225c225
< 2 out-of-order packets (0 bytes)
---

40 out-of-order packets (49232 bytes)

233c233
< 0 connection requests
---

4 connection requests

235,236c235,236
< 9 connections established (including accepts)
< 34 connections closed (including 0 drops)
---

13 connections established (including accepts)
39 connections closed (including 0 drops)

239c239
< 909 segments updated rtt (of 911 attempts)
---

1401 segments updated rtt (of 1402 attempts)

247c247
< 316 correct data packet header predictions
---

15723 correct data packet header predictions

300,301c300,301
< 140 packets sent
< 136 reply packets
---

159 packets sent
155 reply packets

303c303
< 169 packets received
---

194 packets received

305,306c305,306
< 161 valid request packets
< 43 broadcast/multicast packets
---

186 valid request packets
50 broadcast/multicast packets

bash-4.2#

-----Original Message-----
Sent: Sunday, April 20, 2014 7:49 PM
To: Michael D. Spence
Cc: NetBSD
Subject: Re: netbsd 5 and high speed internet
I have been using an old 1ghz 1 or 2Mb PC to run netbsd 5 and act as a
firewall for a number of years. It is not used for anything but this and
Probably your CPU is adequate, and either there's something wrong
unexpected, or those interfaces are too crufty for 100 Mbps. If you have

PCI

slots and can try some other kinds, especially if you can find an intel

card that

is fxp(4), or DEC that is tlp(4), it might help.
Run 'netstat -s' on the gw before and after the speed test, and diff the
results. Try running wget or something on the gateway itself to
download a large file.
It's remarkable that you are getting 80-90 Mbps on a cable modem.

[Greg Troxel]

[netstat -s diff]

That looks mostly ok. I ask for netstat -s diffs because there are a
lot of useful counters for error cases, and if some of those are
incrementing it really helps to notice. In your case the only value
that jumped out at me was that there were 30 more out-of-order packets
received. That's not a huge number out of receiving 15000 packets, and
not really cause for concern.

One thing to figure out is the reason for the lack of full speed. TCP
is highly sensitive to loss. So while this may end up being too
complicated for you, the best path to really understanding what's going
on is to first find a way to download a large file so that you don't
rely on a speed test that you can't understand. Using the built-in
ftp(1) and some free software mirror that's near you (use traceroute) is
a good plan, to avoid problems farther out in the internet. Then, use
'tcpdump -w' to save a trace from the WAN-side interface on your router
to a file. Then, install pkgsrc/math/xplot and read the documentation
(in the source more than the installed package) for tcpdump2xplot.
Basically, this takes the tcpdump of the TCP session and turns it into a
graph, from which you can infer missed packets, out-of-order packets,
and retransmissions. You can then see if a rate below what you expect
is steady or bursty.

Also, I concur Thor's advice to use the fxp0 interface instead of vr0.
You might try all three on the WAN side, suing ftp(1) to measure each
one. Keep careful notes, and refrain from assuming there is only one
issue.

Finally, if you're getting 30 Mbps on a TCP stream, that's likely all
you'll get to someplace you actually want to talk to on the Internet
anyway, because of congestion in the core.

[Michael D. Spence]

Thanks guys, using fxp0 did it. Speed test now reports 90+ Mb.

-----Original Message-----
Sent: Monday, April 21, 2014 10:31 AM
To: Michael D. Spence
Cc: 'NetBSD'
Subject: Re: netbsd 5 and high speed internet
[netstat -s diff]
That looks mostly ok. I ask for netstat -s diffs because there are a lot

of

useful counters for error cases, and if some of those are incrementing it
really helps to notice. In your case the only value that jumped out at me

was

that there were 30 more out-of-order packets received. That's not a huge
number out of receiving 15000 packets, and not really cause for concern.
One thing to figure out is the reason for the lack of full speed. TCP is

highly

sensitive to loss. So while this may end up being too complicated for

you, the

best path to really understanding what's going on is to first find a way

to

download a large file so that you don't rely on a speed test that you

can't

understand. Using the built-in
ftp(1) and some free software mirror that's near you (use traceroute) is a
good plan, to avoid problems farther out in the internet. Then, use

'tcpdump

-w' to save a trace from the WAN-side interface on your router to a file.
Then, install pkgsrc/math/xplot and read the documentation (in the source
more than the installed package) for tcpdump2xplot.
Basically, this takes the tcpdump of the TCP session and turns it into a

graph,

from which you can infer missed packets, out-of-order packets, and
retransmissions. You can then see if a rate below what you expect is

steady

or bursty.
Also, I concur Thor's advice to use the fxp0 interface instead of vr0.
You might try all three on the WAN side, suing ftp(1) to measure each one.
Keep careful notes, and refrain from assuming there is only one issue.
Finally, if you're getting 30 Mbps on a TCP stream, that's likely all

you'll get to

someplace you actually want to talk to on the Internet anyway, because of
congestion in the core.