Installing Mythbuntu backend on Virtualbox virtual machine running on Mac OS X

This is just a rough outline of steps, not a how to!

Install Sun Virtualbox on Mac.

Download ISO of Mythbuntu

Create new Linux virtual machine in Virtualbox. I used 8 GB as the disk size (figuring I would store recordings on an external drive) and 500 MB RAM (no particular reason.

Change settings on the virtual machine's Details list:

• Click on Storage then under the CD-ROM entry browse to and select the Mythbuntu ISO.
• Click on Network and then select bridged as the network type instead of NAT (I actually did this after the Mythbuntu install was finished so I am not sure if there would be problems selecting it intitally).

Boot up the virtual machine and go through the Mythbuntu basic install process. I chose to install both a frontend and a backend even though I am not planning to watch recordings on the virtual machine; I did this because I believe some settings are only accessible through the frontend. Other than that I think I just accepted all the defaults.

When it got to the part of the process when the Mythbuntu backend setup program ran I escaped out of it to get to the desktop because I wanted to make some changes to the system before I set up the backend.

Set the virtual machine to use a static IP address:

• Right click on the network manager icon on the right side of the virtual machine menu bar and selecting edit connections.
• Select Auto eth0 under Wired
• Click Edit
• Select IPv4 Settings
• Enter values in all fields; Apply button will grey out unless all required fields are completed.
• Click Apply

Set up the virtual machine to use an external hard drive (for storing recordings).

• Shut down virtual machine.
• Go to Details list for VM and click on Shared Folders
• Browse to external drive and designate it as a shared folder.
• Install VirtualBox Guest additions on the VM by:
• Restart VM
• Click on the Window for the VM to make it active, and then on the Mac menubar click on Devices and then select Install Guest Additions (if the VirtualBox program window is the active window you can't see the Devices menu). This causes a virtual CD to be mounted to the VM.
• Follow the instructions How to install VirtualBox Guest Additions in Ubuntu with the following changes: run the installer by using "sudo ./VBoxLinuxAdditions-x86.run" otherwise you will get command not found.
• Reboot VM.
• Test mounting external hard drive by making mount point and then running this:
• sudo mount -t vboxsf SharedFolderName \MountPointPath where SharedFolderName is the name you gave to the shared folder in VirtualBox and \MountPointPath is the path to the mount point you created.
• However, if you mount the shared folder this way it will be mounted as owner root, group root and read only.
• Dismounting the shared folder and setting the ownership and permissions on the mount point doesn't work to change this because the next time you mount the shared folder the ownership, group, and permissions will revert.
• This is the mount command that finally made it possible to write to the shared folder: sudo mount -t vboxsf -o uid=andy,gid=users SharedFolderName /MountPointPath/
• Once I got that working I added it to the /etc/fstab file so that the shared folder would mount on boot by using sudo nano /etc/fstab to open the fstab file in a text editor and then adding this to the bottom of the file SharedFolderName /MountPointPath vboxsf gid=105,uid=103
• The uid and gid are for the user mythtv and the group mythtv. I tried mounting it with uid=1000 and gid=1000 but mythbackend was not able to write recording files to it, so I just made mythtv the owner of the mount.

Once I got the shared folder working properly I finally ran Myth Backend Setup (Applications -> System -> Myth Backend Setup). Here are the fields that I set differently than the default:

• General-Host Address Backup Setup-Local Backend-IP Address = static IP address of VM
• General-Host Address Backup Setup-Local Backend-Security Pin = 0000
• General-Host Address Backend Setup-Master Backend-IP Address = same
• General-Locale Settings-Channel frequency table = us-cable (selected)
• Capture Card Setup-Card type = HDHomeRun DTV tuner box (selected)
• Capture Card Setup-Available Devices = 1019A9DB-0 (selected)
• On Input Connections I scanned channels using Cable High.
• After the channel scan was done and I added all the ATSC channels I used the Channel Editor to manually fill in the XMLTV ID field for each high def channel. For some reason SchedulesDirect.org doesn't link up to the high def channels correctly, so I just fill in the XMLid values for the broadcast versions of the channels, which I looked up on SchedulesDirect.org
• Then I went to the Storage Groups section, clicked on default, and then filled in the mount point of my shared folder as the default storage location.

Unfortunately, the shared folder approach was too slow, and when I tried to play back recordings stored on the shared folder the video keep halting and stuttering. So it was on to plan b.

I shut down the VM and then in VirtualBox I disconnected the shared folder from the VM by clicking on Details - Shared Folders and then removing the shared folder.

Next in VirtualBox I created a new virtual disk on my external hard drive and made it as big as I could. I picked expandable, instead of fixed, because when I tried fixed it said it would take 20 hours to format the virtual disk!

Then in VirtualBox I attached the new virtual disk to my VM and then started the VM up. From there I used regular linux commands to partition and format the new virtual disk, and then I added it to the /etc/fstab file so it would mount on boot to the same mount point as I had previously used for the shared folder (and obviously I removed the old /etc/fstab entry for the shared folder while I was in there.

Then I did a sudo mount -a to mount the new virtual drive and tested it by recording a show in MythTV, and everything worked great.

mjpg-streamer documentation

The documentation for mjpg-streamer is kind of scattered about.  I have dumped it all here for my own convenience.

Usage introduction

This example shows how to invoke mjpg-streamer from the command line

export LD_LIBRARY_PATH="$(pwd)"
./mjpg_streamer -o "output_http.so -w ./www"

pwd echos the current path you are working at, the backticks open a subshell to execute the command pwd first the exported variable name configures ldopen() to search a certain folder for *.so modules

export LD_LIBRARY_PATH=`pwd`

This is the minimum command line to start mjpg-streamer with webpages for the input-plugin default parameters are used

./mjpg_streamer -o "output_http.so -w `pwd`/www"

To query help for the core:

./mjpg_streamer --help

To query help for the input-plugin "input_uvc.so":

./mjpg_streamer --input "input_uvc.so --help"

To query help for the output-plugin "output_file.so":

./mjpg_streamer --output "output_file.so --help"

To query help for the output-plugin "output_http.so":

./mjpg_streamer --output "output_http.so --help"

To specify a certain device, framerage and resolution for the input plugin:

 ./mjpg_streamer -i "input_uvc.so -d /dev/video2 -r 320x240 -f 10"

To start both, the http-output-plugin and write to files every 15 second:

mkdir pics
./mjpg_streamer -o "output_http.so -w `pwd`/www" -o "output_file.so -f pics -d 15000"

To protect the webserver with a username and password (!! can easily get sniffed and decoded, it is just base64 encoded !!)

./mjpg-streamer -o "output_http.so -w ./www -c UsErNaMe:SeCrEt"

If you want to track down errors, use this simple testpicture plugin as input source. To use the testpicture input plugin instead of a webcam or folder:

./mjpg_streamer -i "./input_testpicture.so -r 320x240 -d 500" -o "./output_http.so -w www"

Usage: mjpg_streamer

mjpg_streamer
  -i | input "<inputplugin.so> [parameters]"
  -o | output "<outputplugin.so> [parameters]"
 [-h | help ]........: display this help
 [-v | version ].....: display version information
 [-b | background]...: fork to the background, daemon mode

Note: If you start mjpg-streamer in the background use this to stop it:

kill -9 `pidof mjpg_streamer`

Example #1:
 To open an UVC webcam "/dev/video1" and stream it via HTTP:
  mjpg_streamer -i "input_uvc.so d /dev/video1" -o "output_http.so"

Example #2:
 To open an UVC webcam and stream via HTTP port 8090:

  mjpg_streamer -i "input_uvc.so" -o "output_http.so -p 8090"

Example #3:
 To get help for a certain input plugin:

mjpg_streamer -i "input_uvc.so help"

In case the modules (=plugins) can not be found:
 * Set the default search path for the modules with:
   export LD_LIBRARY_PATH=/path/to/plugins,
 * or put the plugins into the "/lib/" or "/usr/lib" folder,
 * or instead of just providing the plugin file name, use a complete
   path and filename:
   mjpg_streamer i "/path/to/modules/input_uvc.so"

Parameters for input_uvc.so

This is the output from:
  #mjpg_streamer --input "input_uvc.so --help"

The following parameters can be passed to this plugin:

 [-d | --device ].......: video device to open (your camera)
 [-r | --resolution ]...: the resolution of the video device,
                          can be one of the following strings:
                          QSIF QCIF CGA QVGA CIF VGA
                          SVGA XGA SXGA
                          or a custom value like the following
                          example: 640x480
 [-f | --fps ]..........: frames per second
 [-y | --yuv ]..........: enable YUYV format and disable MJPEG mode
 [-q | --quality ]......: JPEG compression quality in percent
                          (activates YUYV format, disables MJPEG)
 [-m | --minimum_size ].: drop frames smaller then this limit, useful
                          if the webcam produces small-sized garbage frames
                          may happen under low light conditions
 [-n | --no_dynctrl ]...: do not initalize dynctrls of Linux-UVC driver
 [-l | --led ]..........: switch the LED "on", "off", let it "blink" or leave
                          it up to the driver using the value "auto

NSLU2 + Debian Lenny + wireless + webcam

My webcam is a HP KQ246AA 8.0MP Deluxe Webcam
First I installed Debian on my NSLU2 following this guide:

http://www.cyrius.com/debian/nslu2/install.html

Then I connected my HP Webcam (purchased because it is supposed to be UVC compatible) and ran dmesg | tail and it looked like it was recognized.

I tried installing mjpg-streamer using aptitude, but it said there was no package by the name in the repositories.

I then downloaded the tar.gz file from the mjpp-streamer sourceforge site:

http://sourceforge.net/projects/mjpg-streamer/files/

I had no idea what to do with it, so I double-clicked on it on my Mac OS X desktop, which uncompressed it to a directory.  Inside that directory was a file called README which had the following instructions:

To compile and start the tool:

tar xzvf mjpg-streamer.tgz
cd mjpg-streamer
make clean all
export LD_LIBRARY_PATH=.
./mjpg_streamer -o "output_http.so -w ./www"

So I used Macfusion to connect my Macbook to the NSLU2 as root and copied the tar.gz file over to the NSLU2.  Then I ran the tar xzvf command, which created a directory called mjpg-streamer-r63.  I went into that directory and ran make clean all, but that gave me a command not found error.

So I used aptitude to install the make package, and tried again.  This time it ran for a bit and then gave me an error that gcc wasn't found.  So I used aptitude to install the gcc package, and tried again.

This time I got a bunch of errors related to jpeg_utils. Looking at the README file I saw this:

"In case of error: the input plugin "input_uvc.so" depends on libjpeg, make sure it is installed."

So I tried aptitude install libjpeg, but that gave me an error saying there was no such package, but listed the names of some packages with similar names.  I took a chance and installed libjpeg-dev, figuring that sounded the most promising. Aptitude then said it was installing a different package with 62 in the name.

After the 62 variation of libjpg was installed I ran "make clean all" again and it seemed to finish without any fatal errors, but lots of warnings.

However when I tried to run it I got this error:

./mjpg_streamer -o "output_http.so -w ./www"
MJPG Streamer Version.: 2.0
 i: Using V4L2 device.: /dev/video0
 i: Desired Resolution: 640 x 480
 i: Frames Per Second.: 5
 i: Format............: MJPEG
 o: www-folder-path...: ./www/
 o: HTTP TCP port.....: 8080
 o: username:password.: disabled
 o: commands..........: enabled
Unable to start capture: Cannot allocate memory
 i: Error grabbing frames


I figured this meant there wasn't enough memory so I went to this web page I had found about Debian on the NSLU2:

http://www.sunspot.co.uk/Projects/SWEEX/slug/notes/Debian_notes.html

And tried his recommendation:

"If you are not using IPv6, you can prevent the module from being automatically loaded by adding the line
blacklist ipv6
to /etc/modprobe.d/blacklist."

And then rebooted using "shutdown -r now"

Then by running the export command again, and going into the mjpg-streamer directory I was able to get it to work; pointing firefox at port 8080 of the IP address of the NSLU2 worked.

Then I decided to try and install mjpg-streamer in the system itself so I wouldn't have to run it from the folder where I made it, and have to run the export library thing every time.  But I had no idea where to put the files.  So I downloaded the Ubuntu deb package from the mjpg-streamer Sourceforge site, and ran dpkg-deb -c (I think) to get a list of the tree structure of the files, and then I copied the files from my install folder to those folders on my system (I had to make one new folder /usr/www).

Then I was able to run the program from any prompt just using mjpeg-streamer without any ./ etc. but it kept giving me unable to start capture: Cannot allocate memory errors and exiting.  Interestingly, I was still able to run it from the install folder putting ./ before the name, at least once, but then even that didn't work. So back to trying to open up more memory.

I went through the ideas listed here:

http://www.cyrius.com/debian/nslu2/reducing-memory.html

I couldn't find any libdevmapper1.02 in /etc/init.d so I couldn't do that idea.

Then I started reviewing kernel modules shown by lsmod.  The first one, evdev, appeared to just be used for managing input devices for X.org, so I went to the blacklist file:

nano /etc/modprobe.d/blacklist

and added "blacklist evdev"  (no quotes) to the end of the file and rebooted. evdev no longer appears when I run lsmod.

Next I tried blacklisting both usbhid and hid modules since they seem to be for mouses. Slug still boots.

It seems that the ixp4xx_beeper module is for making the NSLU2 beep. I guess I won't blacklist it yet.

Then I moved on to reducing the services (daemons?) loaded when the system starts.  This page mentioned a package called rcconf to change the services loaded.  It is apparently a utility that shows you what is loaded and allows you to specify what gets loaded and what doesn't.  So I installed it using aptitude.

Using rcconf I turned off the following:

• exim4 (apparently for handling email)
• mountnfs.sh (I am guessing its for the NFS file system)
• nfs-common (same guess)

After all that, now I am able to run mjpg-streamer from any command prompt:

mjpg_streamer -o "output_http.so -w /usr/www"


Note that all the mjpg_streamer commands below are for the way I manually installed it on my system.  To get them to work on a machine where mjpg_streamer wasn't manually installed you will have to modify the command to match the syntax in the original instructions.

Next issue: the streamer stops working if you end the terminal session where you launched it.  So time to try this idea from this web page:

"there is an option in mjpg-streamer to launch it in background, use it as a daemon:
[-b | --background]...: fork to the background, daemon mode"
So this lets me keep using the ttyS0 terminal -
mjpg_streamer -i "input_uvc.so -r 960x720" -b -o "output_http.so -p 8080 -w /webcam_www"
Note:- to make mjpg-streamer exit cleanly, use -
kill -9 `pidof mjpg_streamer`"

It works! After I launched mjpg-streamer with the -b option it continues running after closing the SSH terminal session.

kill -9 `pidof mjpg_streamer`

Seems to shut it down successfully.

Now to explore options. Here is the output from mjpg_streamer --help

Usage: mjpg_streamer
  -i | --input " [parameters]"
  -o | --output " [parameters]"
 [-h | --help ]........: display this help
 [-v | --version ].....: display version information
 [-b | --background]...: fork to the background, daemon mode
-----------------------------------------------------------------------
Example #1:
 To open an UVC webcam "/dev/video1" and stream it via HTTP:
  mjpg_streamer -i "input_uvc.so -d /dev/video1" -o "output_http.so"
-----------------------------------------------------------------------
Example #2:
 To open an UVC webcam and stream via HTTP port 8090:
  mjpg_streamer -i "input_uvc.so" -o "output_http.so -p 8090"
-----------------------------------------------------------------------
Example #3:
 To get help for a certain input plugin:
  mjpg_streamer -i "input_uvc.so --help"
-----------------------------------------------------------------------
In case the modules (=plugins) can not be found:
 * Set the default search path for the modules with:
   export LD_LIBRARY_PATH=/path/to/plugins,
 * or put the plugins into the "/lib/" or "/usr/lib" folder,
 * or instead of just providing the plugin file name, use a complete
   path and filename:
   mjpg_streamer -i "/path/to/modules/input_uvc.so"

I am having a heck of a time finding any documentation on the syntax.  I found this forum post:

mjpg_streamer -i "input_uvc.so -r 320x240 -f 6" -o "output_http.so -p 8080"  -b

Which I modified to:

mjpg_streamer -i "input_uvc.so -r 320x240 -f 6" -o "output_http.so -w /usr/www" -b

That seemed to work to reduce the resolution, but how to increase it?

After looking at the webcam specs I tried:

mjpg_streamer -i "input_uvc.so -r 800x600 -f 20" -o "output_http.so -w /usr/www" -b

But it still gave me a 640x480 image. I am giving up in trying to get a higher resolution for now.

On to wireless.  I have a Belkin USB wifi adapter that uses the rt73usb driver which is built into Debian Lenny.

I followed the directions here on how to set up a rt73 wireless device on Debian Lenny:


http://wiki.debian.org/WiFi/rt73

Everything seemed to go smoothly.

I had a lot of troubles getting the wifi to connect to my network, but I think most of it was because I was trying to use the wrong wpa2 password (i.e. psk).  After I used the right wpa psk it worked using the following /etc/network/interfaces file:

auto wlan0
iface wlan0 inet dhcp
        wpa-ssid Chacala
        wpa-psk mywirelesspassword

That is, I was able to get it working while the ethernet cable was also plugged in.   I could connect SSH to the IP address assigned to the wireless connection while the ethernet cable was plugged in, but once I unplugged ethernet the wireless connection dropped.  I did some googling and discovered this hint on this page:

"If everything looks like it's working, don't just unplug your ethernet. use "ifconfig eth0 down" beforehand, otherwise you'll loose both connections."

Once I followed that hint the wireless link stayed alive after I disconnected the ethernet.

So next I tried rebooting with ethernet unplugged and just the wifi adapter.  The Slug appeared to boot, and the LED on the wireless was flashing, but I couldn't ping or SSH to the Slug.  So I connected the ethernet cable again, but I still wasn't able to ping or SSH to the Slug.  So I rebooted with both the wifi adapter and ethernet connected, and after reboot I was able to ping and ssh to both wifi and ethernet IP addresses.

Next I tried a hint from this page:

"Actually there's a strange behavior of networkingon NSLU, when i boot without the ethernet cable plugged, in the wlan interface is also not working.
When i replug the ethernet cable, the wireless network also comes up. When i ifconfig eth0 down and unplug the ethernet cable the wlan does still work.
Strange, after some meditation on the interfaces file I noticed the commented line # The primary network interface and had the idea to just change the sections for the interfaces, so lets wlan0 be my primary interface and eth0 an additional one."

i.e. he swapped the order of eth0 and wlan0 in /etc/network/interfaces to have wlan0 come first.  I changed the order in my interfaces file and rebooted.

It worked! After reboot I was able to connect to the Slug through the wireless connection with no ethernet cable attached.  Here is the final /etc/network/interfaces file that worked:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto wlan0
iface wlan0 inet dhcp
        wpa-ssid mywirelessnetworkname
        wpa-psk mywirelesspassword

allow-hotplug eth0
iface eth0 inet dhcp

Now onto having the camera and the USB wireless adapter connected at the same time using a hub.

I shut down the Slug, and then connected both the webcam and the wireless adapter using a cheap USB hub I had purchased from Walmart years ago on a trip.

After reboot I was able to connect to the Slug via wireless

Next I tried running the camera:

mjpg_streamer -o "output_http.so -w /usr/www

It works! I now have a wireless webcam.

Poking around looking for something else, I discovered that the following command will list all of the supported modes of the webcam:

lsusb -v

Looking at this it seems that the only streaming mode higher than 640 x 480 is 1280 x 1024, which I haven't tried before, so I tried it:

mjpg_streamer -i "input_uvc.so -f 1 -r 1280x1024" -o "output_http.so -w /usr/www"

 And here is what I got:

MJPG Streamer Version.: 2.0
 i: Using V4L2 device.: /dev/video0
 i: Desired Resolution: 1280 x 1024
 i: Frames Per Second.: 1
 i: Format............: MJPEG
 o: www-folder-path...: /usr/www/
 o: HTTP TCP port.....: 8080
 o: username:password.: disabled
 o: commands..........: enabled
Unable to start capture: Cannot allocate memory
 i: Error grabbing frames

The good news is that it didn't reject the resolution saying it wasn't found.  The bad news is the memory error. On to troubleshooting the memory error.  dmesg shows a log of error messages, starting off with:
mjpg_streamer: page allocation failure

[42951766.100000] mjpg_streamer: page allocation failure. order:5, mode:0x1

After a lot of googling it seems that this is some flavor of running out of memory, and one possible solution is to set overcommit to 2. Apparently you do that by running this command:

echo 2 > /proc/sys/vm/overcommit_memory

So I did. But I still got the Cannot allocate memory error when I tried to launch the streamer! Even after rebooting.

Then I decided to try blacklisting the audio modules from the kernel:

nano /etc/modprobe.d/blacklist

And then added the following line to the bottom of the file

blacklist snd_usb_audio

After I rebooted I checked lsmod and it looked like all the sound modules that had previously been loading were no longer loading, and free showed about 6 MB free memory with no use of swap. But still get a memory error trying to run mjpg-streamer!

Then I had a brainstorm.  Free showed no swap being used the first time I ran it after boot, but it should more swap being used, and more memory free, after I tried and failed to run mjpg-streamer, so maybe it would run now that some programs had been moved to swap?  So I tried again, and success! The Slug is now streaming 1280 x 1024 at the rate of 1 frame per second!  Here is the command I used to get the high resolution streaming:

mjpg_streamer -i "input_uvc.so -f 1 -r 1280x1024" -o "output_http.so -w /usr/www"

 The new USB 2.0 hub I ordered (Belkin USB 2.0 4-Port Ultra Mini Hub F5U407  Belkin USB 2.0 4-Port Ultra Mini Hub F5U407) arrived from Amazon.  I bought this because it was under $10 and had good ratings on Amazon.  I plugged it in, hooked the webcam and the wifi dongle up to it and rebooted.  I hung on the first reboot, so I rebooted with nothing connected and then added the new USB hub by itself, then the webcam, and then the wifi dongle, and things worked that time.

The picture quality from the webcam seems much better with the USB 2.0 hub.  I suspect the webcam automatically detects when it is hooked up to a 1.1 hub earlier and jacks up the compression on the video stream to compensate.

Next up: reduce wear on flash drive. I bookmarked some pages about this in the Firefox NSLU2 folder.