Once again about 3TB

[Dima Veselov]

Hello!

Several times I was trying to settle 3TB on recent stable version.
Now it works and I just wanted to drop something helpful for people
not to waste time.

Now I have production system working on two different 3Tb drives,
handled by NetBSD RAIDframe.

Few advices/notes:

1. Despite of some messages found - RAIDframe can't be built
over raw devices (at least on NetBSD 6.1.4/amd64) - it always
will be limited to 2Tb. Somewhere I saw "kernel know disk size,
but disklabel don't" - thats not true. Everything you try with 'd'
device will be limited to 2Tb.

2. Booting from GPT works well, but it is a real mess to settle.
There is no good documentation about 'gpt biosboot' command - some
says 'gpt biosboot wd0', some says 'gpt biosboot dk0'.
I don't understand much about offsets working there, and have a
sandwitch [wd0 [dk0 [raid0a]]] and somehow this worked out as a
magic. However few things should be fixed (or better documented):

gpt biosboot wd0 says no partition can be found
if disk wasn't zeroed - installboot complain 'not enough space
before FAT'. Worked after zeroing and dk re-creation.
documentation says to create FAT partition and boot it, this is
outdated and don't work at all.

3. dk-raid-dk sandwitch is working! Magic :)
Don't know if it is good or not - I created many wedges for both
disks, most of them work like wd0 -> dk -> raidXa and most big one
work as wd-dk-raid-dk

/dev/raid0a 241M 53M 175M 23% /
/dev/raid4a 7,8G 644M 6,7G 8% /var
/dev/raid5a 16G 2,3G 12G 15% /usr
/dev/dk10 2,7T 824G 1,7T 31% /export

4. dkctl can be "tired". If you play with it much (create/delete
wedges) it always do well, but suddenly all new wedges won't work.
Any try to touch it fail and only reboot will help. This is a bug,
but I don't know how to repeat it.

And few questions I have - where dkctl store wedge information?

[Martin Husemann]

Post by Dima Veselov
And few questions I have - where dkctl store wedge information?

Nowhere - that is what gpt is used for. A wedge (dk*) is a pure runtime
construct with *NO* on-disk format. They are generated "on the fly"
from gpt partitions when booting (or from other on-disk formats if your
kernel is configured that way).

Martin

[Dima Veselov]

So I assume gpt make it and dkctl label it and it finally rises.
Understood, new partition don't rise because kernel isn't sure partition
would be really used.

Yes, and you can also do "temporary" wedges at runtime (e.g. use unallocated
parts of your disk, but never write it to gpt).

Thanks for answer. One more question - what is this (after dk10 init):
opendisk: can't open dev dk0 (16)
opendisk: can't open dev dk1 (16)
opendisk: can't open dev dk2 (16)
opendisk: can't open dev dk3 (16)
opendisk: can't open dev dk4 (16)
opendisk: can't open dev dk5 (16)
opendisk: can't open dev dk6 (16)
opendisk: can't open dev dk7 (16)
opendisk: can't open dev dk8 (16)
opendisk: can't open dev dk9 (16)
opendisk: can't open dev wd0 (16)
opendisk: can't open dev wd1 (16)