Reliable way to run emulated Linux under NetBSD

[Kamil Rytarowski]

Hello,
 
I'm looking for a reliable and flawless way to run emulated Linux (GNU/Linux distribution under some tool). I'm using i386 and amd64.
 
I was trying to use qemu (from pkgsrc) for this purpose... but I can't progress farther than grub boot menu. When a system is supposed to boot all I get is a black screen and 100% CPU usage. I was trying to use a few of popular desktop / livecd Linux distributions (from downloaded .iso).
 
I don't want to dual-boot or get a dedicated physical machine.

Last time I checked XEN, I had fatal issues with X Window under dom0, but it's another issue.

Do you run GNU/Linux under your NetBSD? If so how? What's your .iso? What's your qemu command?

Running GNU/Linux will ease me to port pieces of software to NetBSD.

Thanks

[Benny Siegert]

The problems with X in the Dom0 have been fixed since NetBSD 6.0. I
would say go for Xen, given that it will give you "native" performance
in the Linux guests.

On Wed, Aug 13, 2014 at 10:22 AM, Kamil Rytarowski <***@gmx.com> wrote:
> Hello,
>
> I'm looking for a reliable and flawless way to run emulated Linux (GNU/Linux distribution under some tool). I'm using i386 and amd64.
>
> I was trying to use qemu (from pkgsrc) for this purpose... but I can't progress farther than grub boot menu. When a system is supposed to boot all I get is a black screen and 100% CPU usage. I was trying to use a few of popular desktop / livecd Linux distributions (from downloaded .iso).
>
> I don't want to dual-boot or get a dedicated physical machine.
>
> Last time I checked XEN, I had fatal issues with X Window under dom0, but it's another issue.
>
> Do you run GNU/Linux under your NetBSD? If so how? What's your .iso? What's your qemu command?
>
> Running GNU/Linux will ease me to port pieces of software to NetBSD.
>
> Thanks



--
The first essential in chemistry is that you should perform practical
work and conduct experiments, for he who performs not practical work
nor makes experiments will never attain the least degree of mastery.
-- Abu Musa Jabir ibn Hayyan (721-815)

[Matthias Scheler]

On Wed, Aug 13, 2014 at 10:22:16AM +0200, Kamil Rytarowski wrote:
> I'm looking for a reliable and flawless way to run emulated Linux
> (GNU/Linux distribution under some tool). I'm using i386 and amd64.

A Xen domU under a NetBSD dom0 would do that job.

Kind regards

--
Matthias Scheler https://zhadum.org.uk/

[Malcolm Herbert]

On Thu, Aug 14, 2014 at 08:33:03AM +0100, Matthias Scheler wrote:
|On Wed, Aug 13, 2014 at 10:22:16AM +0200, Kamil Rytarowski wrote:
|> I'm looking for a reliable and flawless way to run emulated Linux
|> (GNU/Linux distribution under some tool). I'm using i386 and amd64.
|
|A Xen domU under a NetBSD dom0 would do that job.

just curious - has anyone thought of creating a Linux Docker API
work-alike layer for NetBSD to act as either a guest or host? I
understand that Xen is probably a preferred solution for the moment, but
I would be curious as to whether that might be useful as an approach

it is definitely flavour of the month in DevOps circles now but might
be worth a look ... I'm not a dev, so I have no idea how much work that
would entail behind the scenes to implement

as far as I understand it, in order to get any sort of widespread
adoption the Docker API would need to be fairly stable and consistent
among the various Linux flavours and kernel versions out there, which
might make it a good target to work toward from the *BSD side ...

Regards,
Malcolm

--
Malcolm Herbert
***@mjch.net

[Kamil Rytarowski]

Hello,

So nobody actually uses qemu?
I will give a try to XEN.

Thanks,

[Gerard Lally]

At date and time Thu, 14 Aug 2014 19:16:15 +0200, Kamil Rytarowski wrote:

> Hello,
>
> So nobody actually uses qemu?
> I will give a try to XEN.
>
> Thanks,

I think Xen is a better choice myself. I tested it a couple of years
ago and found it stable and fast. I had the following domUs running
under a NetBSD dom0:

NetBSD (x3) -- paravirtualised
Slackware (x1) -- paravirtualised
Windows 2008 (x1) -- HVM
Windows XP (x1) -- HVM

The Slackware domU was very interesting. I'll try to explain from
memory, but it's been a while so I don't have all the details. I do have
notes but they're not accessible to me right now. If you get stuck i'll
dig them out and see if they help.

What I did was to install NetBSD first, using just 5 or 10 GB of the
disk, and then set up Xen. I then installed Slackware on the remainder
of the disk, making sure to install Lilo to /boot, not MBR. I installed
Slackware on Logical Volumes (LVM) - lvm-root, lvm-home, etc. I compiled
a new kernel and initrd in Slackware with Xen support (leaving out all
hardware and other options not relevant to a Xen domU), and I copied
this kernel and the initrd to NetBSD's / . I did not delete the old
kernel, but left it as the first option for Lilo. I aslo configured the
NetBSD bootloader to boot into Slackware, using the fdisk -b option.

I then made sure to configure /etc/fstab in Slackware with blkid
references, to ensure Slackware would be able to boot as a domU and as
a physical machine. Another thing I did was to set up Slackware with a
VNC server and also XDMCP. I also set up NFS to share files between the
dom0 and domU.

After all this I was able to return to NetBSD and configure a domU for
Slackware, running in paravirtualised mode (PV is lightning fast). To
access the Slackware domU from the dom0 I used either VNC or XDMCP.
Both of these were fast; the only thing I couldn't configure at the time
was audio. But the icing on the cake for me was also being able to boot
into the same Slackware as a physical machine, by using the first kernel.
So I had a physical Linux setup dual-booting with NetBSD which also
served as a domU inside NetBSD's Xen! To my mind this was incredible.

I fully intend to replicate this setup as my day-to-day working setup
when NetBSD 7 is released. I've learnt a lot about NetBSD since that I
didn't know then. I recommend NetBSD Xen highly: you have all the
benefits of NetBSD and with a paravirtualised Linux domU you also have
the benefit of a very fast Linux setup. And if you do what I did, you
can also dual-boot into this Linux setup whenver you want, if there is
something you find you can't do while it is running under Xen as a domU.

Hope this helps.

--
Gerard Lally

[David Brownlee]

On 14 August 2014 18:16, Kamil Rytarowski <***@gmx.com> wrote:
> Hello,
>
> So nobody actually uses qemu?
> I will give a try to XEN.

I use qemu on my laptop to boot occasional WindowsXP VMs for IE6/7/8
testing (thankfully much less now than I ever did).

I have a 3Ghz i7 K875 NetBSD/amd64 6.x box running 9 CentOS VMs for QA
purposes (running a selection of Jboss java apps and one Jira). Memory
is a little tight, and running something on the dom0 which pages
heavily does quite awful things to the I/O throughput on the DOMUs
(ahem, "Then Dont Do That"), but I've been extremely happy with it.

In the three years or so its been running its saved its costs many
many times over compared to running those VMs on a cloud hoster :)

I did start with the free version of VMware Vsphere to run the Linux
VMs, but.... oh god was the I/O lousy. With four VMs up, running 'yum
update' in one VM would render the others unusable, plus the interface
was... quirky at best.

[Kamil Rytarowski]

Hello,

Thank you for all your comments. I was still trying to use qemu and I've succeed to install Slackware64 14.1.

In general the Linux kernel has to be booted with extra parameters 'noacpi noapic' to not panic.

I'm trying to redirect stdio to the serial interface, use tap and bridge for networking and assign USB devices. I will write a short tutorial for people after me.

Thanks,

[Niels Dettenbach (Syndicat IT&Internet)]

On 17. August 2014 13:08:42 MESZ, Kamil Rytarowski <***@gmx.com> wrote:
>Hello,
>
>Thank you for all your comments. I was still trying to use qemu and
>I've succeed to install Slackware64 14.1.

hiho,

i'm ne to this thread, so i'm hopee i'm not producing doublettes here.


a very reliable and performant way to run linux "under" NetBSD is xen. With NetBSD XEN it is possible to paravirtualize Linux what is in most cases much more efficient and performant then full hardware virt with qemu (what xen even provides) or other full hardware virtualizers.

Xen even allows to provide full access to a PCI device from a "guest" (domU) if the host hardware provides virtualisation features.


hth a little
good luck,

Niels.
--
Niels Dettenbach
Syndicat IT&Internet
http://www.syndicat.com

[Kamil Rytarowski]

Hello,

Thank you everybody for your private and public feedback.

To dot my I's I'm going to give the solution for people looking for a solution how to run Linux in qemu / NetBSD and have it usable, so in my understanding with networking between guest and host and at serial connection.

To spell it out, finish the topic and don't produce off-topic over here (neither to go into non-generic details, so I will skip USB ports pass-through configuration-details), the formula is as follows:

1. Get ingredients:
- qemu from pkgsrc (tested with 2.0.0, from pkgsrc-2014Q2)
- get your favorite distro (Slackware64 14.1 confirmed to work)

2. Create image with for a hard disk [1]

qemu-img create -f qcow2 slack_hd.img 20G

NB. Full Slackware installation requires at least 8-10GB. 20GB is sufficient for basic hacking.

3. Install your favorite distro

NB. You may need X Window environment here.

Run installator inside your qemu machine:

qemu-system-x86_64 -cdrom image.iso -hda slack_hd.img -boot d

At the boot menu you must use (or the guest will crash down) additional kernel parameters 'noapic noacpi'. After system being loaded, perform standard installation.

4. Setup serial connection in your guest

Execute:
qemu-system-x86_64 -hda slack_hd.img

And now follow standard steps relevant for your solution, eg. http://docs.slackware.com/howtos:general_admin:serial_console for Slackware.

5. Setup tap-networking and bring the system up

In your NetBSD host:
ifconfig bridge0 create # Create bridge interface to link tap virtual devices
ifconfig tap0 create # Dedicated as a device piped with qemu's quest
ifconfig tap1 create # Dedicated as a virtual device for host
brconfig bridge0 add tap0 # Bind tap0 to bridge0
brconfig bridge0 add tap1 # Bind tap1 to bridge0
ifconfig tap1 192.168.0.2/24 # Setup host's interface address
ifconfig tap0 up # Bring it up
ifconfig tap1 up # Bring it up
ifconfig bridge0 up # Bring it up

Now run qemu as follows:
emu-system-x86_64 -hda slack_hd.img -net nic -net tap,ifname=tap0,script=no,downscript=no -monitor stdio -serial telnet:127.0.0.1:3000,server -nographic

This will run emulated x86_64 (amd64) system, with a network device over tap0, without up/down network interface scripts (we have done it manually) [2], qemu's monitor over stdout (it eases control over guest), serial connection set to telnet at localhost port 3000.

Now in your guest machine:
ip link set eth0 up # Bring it up
ip addr add 192.168.0.1/24 dev eth0 # Setup addressing

And try to ping host from guest or the other way around :-)

Cheers and EOT! Happy hacking,

[1] http://en.wikibooks.org/wiki/QEMU/Images
[2] http://en.wikibooks.org/wiki/QEMU/Networking