Can I use a native Linux installation as DomU?

[Mayuresh]

I am running NetBSD 6.1 i386, XEN3 DOM0, xentools 41.

I have a Fedora 17 installation on a separate partition. Currently I can
dual boot into either of the OSes.

I am trying to run Fedora 17 as DomU on NetBSD Dom0.

Firstly, is it a terrible idea - particularly if I need to continue to use
the Fedora 17 installation natively as well?

I read (lost the reference) that Fedora 17 kernels are Xen DomU compliant.
Also "xm create" does not give any error on the kernel. So hopefully the
kernel is fine.

I have copied the kernel to NetBSD filesystem and set:

kernel = "/vmlinuz-3.9.10-100.fc17.i686.PAE"

I have given the disk layout, by carefully mapping disklabel output on
NetBSD with fdisk -l on Linux :

It has 3 partitions, boot, swap and root.

disk = [ 'phy:/dev/wd0g,sda3,w','phy:/dev/wd0i,sda5,w','phy:/dev/wd0j,sda6,w' ]

Have set the root device as is on Fedora:

root = "/dev/sda3"

When booting Fedora natively, whichever parameters I pass, I have set the
same in extra, with addition of xencons:

extra = "xencons=tty1 root=UUID=<uuid> ro rd.md=0 rd.lvm=0 rd.dm=0
SYSFONT=True KEYTABLE=us rd.luks=0 LANG=en_US.UTF-8 rhgb quiet"


After xm create, after a long time, I get this error:
Device 0 (vif) could not be connected. Hotplug scripts not working.

(bridge0 is created.)

The instance shows in "paused" state and closes on its own after some
time.


Would appreciate help to analyze this further.

Mayuresh

[Justin Cormack]

Post by Mayuresh
I am running NetBSD 6.1 i386, XEN3 DOM0, xentools 41.
I have a Fedora 17 installation on a separate partition. Currently I can
dual boot into either of the OSes.
I am trying to run Fedora 17 as DomU on NetBSD Dom0.
Firstly, is it a terrible idea - particularly if I need to continue to use
the Fedora 17 installation natively as well?
I read (lost the reference) that Fedora 17 kernels are Xen DomU compliant.
Also "xm create" does not give any error on the kernel. So hopefully the
kernel is fine.
kernel = "/vmlinuz-3.9.10-100.fc17.i686.PAE"
I have given the disk layout, by carefully mapping disklabel output on
It has 3 partitions, boot, swap and root.
disk = [ 'phy:/dev/wd0g,sda3,w','phy:/dev/wd0i,sda5,w','phy:/dev/wd0j,sda6,w' ]
root = "/dev/sda3"
When booting Fedora natively, whichever parameters I pass, I have set the
extra = "xencons=tty1 root=UUID=<uuid> ro rd.md=0 rd.lvm=0 rd.dm=0
SYSFONT=True KEYTABLE=us rd.luks=0 LANG=en_US.UTF-8 rhgb quiet"
Device 0 (vif) could not be connected. Hotplug scripts not working.

I think that should work. I would turn off the "rhgb quiet" part, you
might get better diagnostics - are you getting console output from the
kernel?

[Mayuresh]

Post by Justin Cormack

Post by Mayuresh
Device 0 (vif) could not be connected. Hotplug scripts not working.

ifconfig bridge0 shows only this much. Hope that is fine:

bridge0: flags=41<UP,RUNNING> mtu 1500

Post by Justin Cormack
I think that should work. I would turn off the "rhgb quiet" part, you
might get better diagnostics - are you getting console output from the
kernel?

I am not getting any console output. (Tried using xm console.)

Also tried -c option of xm create.

Also the instance goes into paused state on running create and comes out
with error mentioned at the top of this mail after a minute or two.

I can see some logs:

# tail xenbackendd.log
xenstore-read: couldn't read path /local/domain/0/backend/vif/1/0/bridge
Failed to read /local/domain/0/backend/console/1/0/script (No such file or
directory)
Failed to read /local/domain/0/backend/console/1/0/script (No such file or
directory)
xenstore-read: couldn't read path /local/domain/0/backend/vif/2/0/bridge
Failed to read /local/domain/0/backend/console/2/0/script (No such file or
directory)
Failed to read /local/domain/0/backend/console/2/0/script (No such file or
directory)

# tail xend.log
[2014-08-24 19:59:46 320] DEBUG (XendDomainInfo:1276)
XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0
[2014-08-24 19:59:46 320] DEBUG (XendDomainInfo:2414) Removing console/0
[2014-08-24 19:59:46 320] DEBUG (XendDomainInfo:1276)
XendDomainInfo.destroyDevice: deviceClass = console, device = console/0
[2014-08-24 19:59:46 320] DEBUG (XendDomainInfo:2414) Removing vbd/2051
[2014-08-24 19:59:46 320] DEBUG (XendDomainInfo:1276)
XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/2051
[2014-08-24 19:59:46 320] DEBUG (XendDomainInfo:2414) Removing vbd/2053
[2014-08-24 19:59:46 320] DEBUG (XendDomainInfo:1276)
XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/2053
[2014-08-24 19:59:46 320] DEBUG (XendDomainInfo:2414) Removing vbd/2054
[2014-08-24 19:59:46 320] DEBUG (XendDomainInfo:1276)
XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/2054
[2014-08-24 19:59:47 320] DEBUG (DevController:628) hotplugStatusCallback
/local/domain/0/backend/vif/2/0/hotplug-status.

Mayuresh.

[Mayuresh]

Post by Mayuresh
Device 0 (vif) could not be connected. Hotplug scripts not working.

I have compiled NetBSD Xen kernel using XEN3_DOM0 conf that shipped with
NetBSD 6.1 i386. That has the following line commented.

Shall I be uncommenting that?

#xennet* at hypervisor? # Xen virtual network interface

Mayuresh.

[Mayuresh]

Post by Mayuresh

Post by Mayuresh
Device 0 (vif) could not be connected. Hotplug scripts not working.

I have compiled NetBSD Xen kernel using XEN3_DOM0 conf that shipped with
NetBSD 6.1 i386. That has the following line commented.
Shall I be uncommenting that?
#xennet* at hypervisor? # Xen virtual network interface

BTW if I do that config gives an error:

# config XEN3PAE_DOM0
../../../../arch/i386/conf/XEN3_DOM0:223: `xennet' cannot attach to
`hypervisor'
*** Stop.

Mayuresh.

[Mayuresh]

Post by Mayuresh
Device 0 (vif) could not be connected. Hotplug scripts not working.

Sorry. My bad. I had commented out bridge0 in vif, while doing some trial
and error. Now I am able to get the console messages after correcting vif.

Now I am able to see console messages, that indicate it is not finding the
root fs.

Started domain domu-linux (id=5)
[ 1.169852] drivers/rtc/hctosys.c:
unable to open rtc device (rtc0)
[ 1.170077] Kernel panic - not syncing: VFS: Unable to mount root fs on
unknown-block(0,0)
[ 1.170085] Pid: 1, comm: swapper/0 Not tainted
3.9.10-100.fc17.i686.PAE #1
[ 1.170090] Call Trace:
[ 1.170097] [<c0998919>] panic+0x87/0x181
[ 1.170103] [<c0c6aede>] mount_block_root+0x1bb/0x24f
[ 1.170109] [<c056591d>] ? sys_mknod+0x2d/0x30
[ 1.170114] [<c0c6afd0>] mount_root+0x5e/0x64
[ 1.170118] [<c0c6b0f5>] prepare_namespace+0x11f/0x163
[ 1.170124] [<c0556e85>] ? sys_access+0x25/0x30
[ 1.170129] [<c0c6ac4f>] kernel_init_freeable+0x1a9/0x1b6
[ 1.170134] [<c0c6a50d>] ? do_early_param+0x7a/0x7a
[ 1.170139] [<c0988c50>] kernel_init+0x10/0xd0
[ 1.170144] [<c09a81f7>] ret_from_kernel_thread+0x1b/0x28
[ 1.170149] [<c0988c40>] ? rest_init+0x60/0x60

Mayuresh.

[Mayuresh]

Post by Mayuresh
I am running NetBSD 6.1 i386, XEN3 DOM0, xentools 41.
I have a Fedora 17 installation on a separate partition. Currently I can
dual boot into either of the OSes.
I am trying to run Fedora 17 as DomU on NetBSD Dom0.

Ok, all done. I am able to boot the native Fedora installation.

However during boot it does not find my root partition /dev/sda6 and drops
me to dracut prompt.

Later, changing root=sda6 to xvda6 solved this problem, too.

Mayuresh.

[Gerard Lally]

Post by Mayuresh
I am running NetBSD 6.1 i386, XEN3 DOM0, xentools 41.
I have a Fedora 17 installation on a separate partition. Currently I can
dual boot into either of the OSes.
I am trying to run Fedora 17 as DomU on NetBSD Dom0.
Firstly, is it a terrible idea - particularly if I need to continue to use
the Fedora 17 installation natively as well?

No, not a terrible idea at all. I did this some years ago with Slackware.
Quite straightforward, even though I was new to Slackware and NetBSD at
the time. When you are in bare-metal Fedora make sure disk mounts are
specified with UUID rather than /dev, so that the mounts are correct for
both bare-metal and domU.

Just run blkid to get a mapping of UUID to /dev

Then replace /dev/sdaXXX with the corresponding UUID in /etc/fstab.

For example, replace this:

/dev/sda1 / ext3 defaults 1 2

with this:

UUID=41c22818-fbad-4da6-8196-c816df0b7aa8 / ext3 defaults 1 2

Perhaps Fedora already specifies mounts this way; Slackware doesn't.

I also found it easier to set up if I used LVM. Another thing you should
do while in bare-metal Fedora is set up SSH, VNC and/or XDMCP, so that
you will be able to log into Fedora when it's running as a domU.

[Mayuresh]

Post by Gerard Lally
Perhaps Fedora already specifies mounts this way; Slackware doesn't.

Yes, they already use UUID in Fedora.

Post by Gerard Lally
I also found it easier to set up if I used LVM. Another thing you should
do while in bare-metal Fedora is set up SSH, VNC and/or XDMCP, so that
you will be able to log into Fedora when it's running as a domU.

I am stuck on networking. My Dom0 uses wifi, which is bridged with
bridge0.

By setting static IP on DomU I was able to ssh/network with Dom0 but not
able to connect with the internet. May be I have to do packet forwarding
explicitly on Dom0. But that sounds a little roundabout.

Perhaps my situation is same as described here:
http://wiki.xen.org/wiki/Xen_in_WiFi_networks

Will investigate.

Mayuresh.

[Brad Spencer]

Post by Gerard Lally
Perhaps Fedora already specifies mounts this way; Slackware doesn't.

Yes, they already use UUID in Fedora.

Post by Gerard Lally
I also found it easier to set up if I used LVM. Another thing you should
do while in bare-metal Fedora is set up SSH, VNC and/or XDMCP, so that
you will be able to log into Fedora when it's running as a domU.

I am stuck on networking. My Dom0 uses wifi, which is bridged with
bridge0.

By setting static IP on DomU I was able to ssh/network with Dom0 but not
able to connect with the internet. May be I have to do packet forwarding
explicitly on Dom0. But that sounds a little roundabout.

Perhaps my situation is same as described here:
http://wiki.xen.org/wiki/Xen_in_WiFi_networks

Will investigate.

Mayuresh.



Yes, you are almost certainly running into that issue, as I did. There
does not appear to exist, in NetBSD, anything like Linux ebtables, which
basically can do a sort of Layer 2 NAT using the MAC address. Virtualbox,
apparently, does this too in its own way, when it detects that you are
bridging to a wireless interface. The quick solution I ended up using was
to put NAT on the DOM0 hypervisor, and fix it all up at Layer 3. You may
be aided in this by creating a private internal network for the guests
with their own IP address space. You can use a tap device on the
hypervisor to get it to participate in that network, if you need it to.
You then simply turn on route forwarding and use ipnat.

I looked a bit at the userland bridge pkgsrc/net/bridged, which basically
is a BPF packet copier and thought about wedging a MAC rewritter inside of
it, but a lack of time on my part kept me from pursuing this any further.
I wasn't sure if the in kernel bridge code had access to everything
needed, like the frame headers, although the presence of IPF in the bridge
code might suggest that it does. Just no time to look...

It would certainly be better for own selfish purposes if this could all
work at Layer 2 when NetBSD is a DOM0 maybe via ipf, or some such.

I use a Lenovo T530 running NetBSD 6.1.4 DOM0 with Xen 4.2. Some of my
Xen guests could use the wireless card and some could use the hardwire
when they are both available, which would be handy for my purposes.

[Niels Dettenbach]

Post by Mayuresh
I am stuck on networking. My Dom0 uses wifi, which is bridged with
bridge0.

This is typical and depends from the wifi network interface implementation.
Some other drivers are not allowing to add the wifi interface to a linux
bridge ("Operation not supported").

On a notebook with alternating wired and wireless LAN (Broadcom 43224)
connectivity i've bridged the DomU vif with the eth Ethernet interface and use
some routing when on WIFI.

If this is not acceptable for you you might fiddle out / rewrite the MAC of
your VIFs interface to your WIFIs MAC by ebtables or similiar - this gives you
a "complete" Level 2 bridge feeling with such WIFI adaptors. However, it is
not clear how far this doesn't breaks rulez / standards. As long as it works
it might be a suitable workaround for you.

hth
cheerioh,


Niels.

[Mayuresh]

Post by Mayuresh
I am stuck on networking. My Dom0 uses wifi, which is bridged with
bridge0.
By setting static IP on DomU I was able to ssh/network with Dom0 but not
able to connect with the internet. May be I have to do packet forwarding
explicitly on Dom0. But that sounds a little roundabout.
http://wiki.xen.org/wiki/Xen_in_WiFi_networks
Yes, you are almost certainly running into that issue, as I did. There

On DomU all I need is a browser to work. I thought SOCKS proxy via Dom0
would be a quicker solution, though it's application layer. It is working
also - if I provide ip address of a website. However I do not know how to
make DNS requests go via ssh proxy. If I could do that I can browse from
DomU using SOCKS.

Mayuresh

[Mayuresh]

Post by Mayuresh
also - if I provide ip address of a website. However I do not know how to
make DNS requests go via ssh proxy. If I could do that I can browse from
DomU using SOCKS.

The answer would be browser specific. For firefox it was setting flag
network.proxy.socks_remote_dns.

So it is kind of working now.

But definitely wish ip forwarding worked over wifi ...

Mayuresh

[Brad Spencer]

Post by Mayuresh
I am stuck on networking. My Dom0 uses wifi, which is bridged with
bridge0.
By setting static IP on DomU I was able to ssh/network with Dom0 but not
able to connect with the internet. May be I have to do packet forwarding
explicitly on Dom0. But that sounds a little roundabout.
http://wiki.xen.org/wiki/Xen_in_WiFi_networks
Yes, you are almost certainly running into that issue, as I did. There

On DomU all I need is a browser to work. I thought SOCKS proxy via Dom0
would be a quicker solution, though it's application layer. It is working
also - if I provide ip address of a website. However I do not know how to
make DNS requests go via ssh proxy. If I could do that I can browse from
DomU using SOCKS.

Mayuresh



Ya, I messed a bit with that notion, except I used tinyproxy, and found
that the lack of DNS made it simpler to use NAT on DOM0. I suppose if you
installed a DNS cache server on DOM0 and pointed the guests there it
something like SOCKS or tinyproxy would function. I didn't have time to
mess around with that, and NAT ended up being simpler.

[Niels Dettenbach]

Post by Mayuresh
able to connect with the internet. May be I have to do packet
forwarding explicitly on Dom0. But that sounds a little roundabout.

Packet forwarding usually shoud be much easier to realize - especially with
NAT. Just take a look at the NAT docs of NetBSD for that - i.e. a short howto:
https://gmplib.org/~tege/xen.html


hth a little
cheerioh,


Niels.

[Mayuresh]

Post by Niels Dettenbach

Post by Mayuresh
able to connect with the internet. May be I have to do packet
forwarding explicitly on Dom0. But that sounds a little roundabout.

Packet forwarding usually shoud be much easier to realize - especially with
https://gmplib.org/~tege/xen.html

Thanks. That is a nice guide.

I was trying this guide so far:
http://www.netbsd.org/docs/guide/en/chap-net-practice.html#chap-net-practice-ipnat

Did everything right as per that, but it did not work for me and I thought
it had something to do with wifi.

I am missing the role of tap0 here. Could you throw some light on it?

Mayuresh.