OpenSUSE Hackweek VII - Hacking USB Joysticks in YaST

Introduction

This hackweek I spent playing with joysticks in YaST and hwinfo (libhd).
YaST already has a module for configuring joysticks, but I only supported Gameport joysticks which are quite obsoleted these days.

AFAIK all recent mainboards do not have gamport connectors (just a pin header) or the gameport is completely missing. And if you want to buy a joystick you will find only USB models anyway.

There was a note in the YaST module that it only supports Gameport joysticks but some users find it confusing.

So I decided to change the situation and do something interesting during my Hackweek project - to add USB joystick support to YaST.

Hacking

The support in YaST actually has three parts. YaST uses libhd (from hwinfo package) for hardware detection, than there is yast2-hardware-detection package which is a libhd wrapper converting C functions and data to YCP (the main YaST language) and finally there is yast2-sound package which contains the joystick configuration module.

The first step was to add USB joystick support to hwinfo so it could be used for joystick detection. The problem was that hwinfo found the USB joystick device but it didn't know that it's a joystick and reported it as an unclassified generic USB device and also the joystick result was empty:

# hwinfo --usb
[...removed not relevant info...]
12: USB 00.0: 0000 Unclassified device
  [Created at usb.122]
  Unique ID: JPTW.MyFI+3nAFw7
  Parent ID: FKGF.4Nx_qoDfSd7
  SysFS ID: /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.3/2-1.3:1.0
  SysFS BusID: 2-1.3:1.0
  Hardware Class: unknown
  Model: "KYE Flight2000 F-23 Joystick"
  Hotplug: USB
  Vendor: usb 0x0458 "KYE Systems"
  Device: usb 0x1004 "Flight2000 F-23 Joystick"
  Revision: "1.02"
  Driver: "usbhid"
  Driver Modules: "usbhid"
  Device File: /dev/input/event5
  Device Files: /dev/input/event5, /dev/input/by-id/usb-0458_4-axis_8-button_joystick-event-joystick, /dev/input/by-path/pci-0000:00:1d.0-usb-0:1.3:1.0-event-joystick
  Device Number: char 13:69
  Speed: 1.5 Mbps
  Module Alias: "usb:v0458p1004d0102dc00dsc00dp00ic03isc00ip00"
  Config Status: cfg=no, avail=yes, need=no, active=unknown
  Attached to: #8 (Hub)
# hwinfo --joystick
#

So the detection code was there, it just needed small improvement. This was rather easy.
Then I checked also hwinfo --joystick output, it was fine.

Then I put an old low-end sound (Sound Blaster 128 PCI) with gameport into my PC and connected an old analog joystick. The old YaST joystick module worked fine I and I configured it.

Then I checked  hwinfo --joystick output again and the analog joystick was not found. It turned out that the hwinfo code actually could never detect any joystick. So I also added Gameport support and fixed Gameport joystick detection.

So at this point I had working joystick detection, but I wanted to improve it a little bit. I wanted detect and report some joystick properties, like number of buttons and axes it has. This turned out to be similar to mouse button detection which already worked fine. So again not a big problem.

The detection part was ready, then I just added  passing joystick details into the yast2-hardware-detection layer so it could be used from YaST.

The most complicated part was the YaST joystick module itself because it supported only gameport joystick and USB joysticks could not be simply added to the UI and also the internal structure had to be completely rewritten.


The Outcome

So what has been changed?

The  hwinfo --joystick output will contain all detected joysticks (both USB and Gameport):

# hwinfo --joystick 
24: USB 00.0: 10d00 Joystick
  [Created at usb.122]
  Unique ID: o2Ga.iGyiCvRqXrA
  Parent ID: FKGF.4Nx_qoDfSd7
  SysFS ID: /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.7/2-1.7:1.0
  SysFS BusID: 2-1.7:1.0
  Hardware Class: joystick
  Model: "KYE Flight2000 F-23 Joystick"
  Hotplug: USB
  Vendor: usb 0x0458 "KYE Systems"
  Device: usb 0x1004 "Flight2000 F-23 Joystick"
  Revision: "1.02"
  Driver: "usbhid"
  Driver Modules: "usbhid"
  Device File: /dev/input/event6 (/dev/input/js1)
  Device Files: /dev/input/event6, /dev/input/js1, /dev/input/by-id/usb-0458_4-axis_8-button_joystick-event-joystick, /dev/input/by-path/pci-0000:00:1d.0-usb-0:1.7:1.0-event-joystick, /dev/input/by-id/usb-0458_4-axis_8-button_joystick-joystick, /dev/input/by-path/pci-0000:00:1d.0-usb-0:1.7:1.0-joystick
  Device Number: char 13:70
  Speed: 1.5 Mbps
  Module Alias: "usb:v0458p1004d0102dc00dsc00dp00ic03isc00ip00"
  Buttons: 8
  Axes: 6
  Config Status: cfg=no, avail=yes, need=no, active=unknown
  Attached to: #20 (Hub)

25: Gameport 00.0: 10d00 Joystick
  [Created at input.316]
  Unique ID: pJ6W.6RgETWGhHg1
  Parent ID: i_aO.F2uWXyTiEa3
  SysFS ID: /pci0000:00/0000:00:1e.0/0000:04:02.0/gameport0/input/input58
  Hardware Class: joystick
  Model: "Analog 4-axis 4-button joystick"
  Vendor: 0x0001 
  Device: 0x000f "Analog 4-axis 4-button joystick"
  Device File: /dev/input/event5 (/dev/input/js0)
  Device Files: /dev/input/event5, /dev/input/js0, /dev/input/by-path/pci-0000:04:02.0-event-joystick, /dev/input/by-path/pci-0000:04:02.0-joystick
  Device Number: char 13:69
  Buttons: 4
  Axes: 4
  Config Status: cfg=no, avail=yes, need=no, active=unknown
  Attached to: #16 (Multimedia audio controller)

The changes are highlighted: it correctly finds the joystick, reports number of axes and buttons, /dev/input/jsX device name is reported, game port joystick is found and for game port joysticks it reports also the sound card to which it is attached to.

The changes in the YaST module are best described by the following screen shots. Let's start with the old YaST module:

The old YaST module with one configured Gameport joystick

When the computer doesn't have any game port the old module displayed just this message although an USB joystick could be attached.

Old joystick test dialog, notice the wrong number of buttons and axes, also using progress bars is not appropriate for this dialog

Here is the new updated YaST module:

Both USB and Gameport joysticks are displayed with more details

The testing dialog has been improved - button and axes detection is correct, sliders are used instead of progress bars and also the joystick name is displayed (usefull when there are more joysticks attached).

 This is quite a nice change, isn't it?

As a bonus I have added hotplug handling into the joystick module - if you plug or unplug any USB joystick the table get refreshed. AFAIK that is unique feature among all YaST modules...

All these changes have been submitted to  Factory/12.1 and should be available in 12.1-RC1 when released.

Improved Disk Usage Estimation in YaST Installer

Disk Usage

YaST installer estimates used disk space in the final installed system. This is possible thanks to the disk usage metadata in installation repository (located in file suse/setup/descr/packages.DU.gz).

This file contains disk usage statistics for every package, so YaST needs just to sum them for all selected packages. (Actually it's slightly more complicated, the disk usage is stored per directory, so YaST/libzypp has to sum it up according to the proposed disk partitioning. Imagine you have separate /usr or /boot partition...)

The Problem

This works well, but the problem is that the final sum does not match the real value you find after installation is completed.

For example in openSUSE-12.1-M3 in default KDE installation the installer estimates this disk usage:

Free: 3.3GB
Used: 3.1GB
Total: 6.4GB

But if you install the system and check the real disk usage you will find these numbers:

# df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       6,4G  3,7G  2,5G  61% /

So the difference between estimated and real available free space is about 800MB! That's quite a lot!

In this case it was not a problem, but if the target partition was smaller, YaST could tell that there would be e.g. 500MB free space but the installation would actually fail because of not enough free space. This problem was reported several times in Novell bugzilla, e.g. bnc#263275, bnc#495251 and more.

Improvement

So where is the extra space spent? Why it doesn't match? There are basically these reasons:

• Files not owned by any package - the repository metadata contains just disk usage for files owned by packages (the files which are in RPMs), it doesn't and cannot know anything about files which are created at runtime outside of RPM package files.
  
  The files are mainly (with usual size in default installation)
  
  • RPM database (contains info about the installed packages): 42MB
  • Zypp cache (contains info about the configured software repositories): 38MB
  • System logs: 14MB
  • Config backup files: 10MB
  • Kernel initrd: 10MB
  • ... and many more small files, but they almost don't affect the overall size so they can be ignored
• File system fragmentation - space is allocated in blocks, so small files effectively take more space than their total size. (ReiserFS is an exception, it supports tail joining.)
• File system's journal - modern file systems use journaling so the file system is in consistent state after system crash or power failure. On the other hand the journal takes some space, for example default Ext3/Ext4 journal size is 128MB (but it can be smaller, depending on partition size and block size). The journal decreases the usable free space.
• Reserved space for root user - Ext file systems by default reserve 5% space for root user. RPM explicitly checks for non-root space (even if it is running as root and could actually use the space), the reserved space also decreases the usable free space.

Current State

I have added all the above mentioned extra space into the YaST installer (except the FS fragmentation, that is hard to implement and would not significantly change the estimation.)

The most difficult task was to figure out the default journal size depending the partition size. I had to look to the respective mkfs utility sources. Some options (like reserved space) might be also set in the YaST partitioner so the space calculation has to read and check the partitioning options.

If you are interested in the implementation details the relevant commit can be found in http://lists.opensuse.org/yast-commit/2011-08/msg00377.html

Result

After patching the installer the estimated sizes (for the same installation) are:

Free: 2.6GB
Used: 3.8GB
Total: 6.4GB

This pretty matches the real disk usage mentioned above. It's not exactly the same, but it's a big improvement. And it actually will never be perfect as it will be always a guess (just more or less good).

The improvement has been submitted to Factory and will be available in openSUSE-12.1-Beta.

To Do

The only thing which is missing is Btrfs support.I'll have to check the mkfs.btrfs utility or Btrfs documentation. So far I have found out that after formatting 8GB partition using Btrfs df reports about 7.2GB free space. I have to found out how the final free space corresponds to the partition size. Moreover Btrfs supports transparent file compression, snapshots, etc. which also make the estimation more complicated...

If you have any note on this feature just add a comment to this blog post. Thank you!

Installing latest Intel graphics driver to openSUSE 11.4

(Edit: The bug has been fixed in openSUSE update repository, you do not need to compile the driver manually anymore, just install the latest update for xorg-x11-driver-video package.)


I have upgraded my home PC to Intel i5-2500K CPU (Sandy Bridge family). The CPU has a new integrated graphics core (Intel HD 3000) and it works out-of-box in openSUSE-11.4 including 3D and composition. The only problem I noticed are broken popup menus and buttons in title bars in KDE. Esp. broken popups are very annoying as they are hardly usable, see e.g. https://bugs.freedesktop.org/attachment.cgi?id=45061.

Fortunately Intel has released updated X driver version 2.15 which fixes this problem. Here is a step by step how to install the updated driver in openSUSE 11.4.

1. Install xorg-x11-server-sdk package
   

   sudo zypper in xorg-x11-server-sdk

2. Download http://xorg.freedesktop.org/archive/individual/driver/xf86-video-intel-2.15.0.tar.bz2
3. Unpack the archive
   

   tar xfjv xf86-video-intel-2.15.0.tar.bz2

4. Now compile the driver:
   

   cd xf86-video-intel-2.15.0
   ./configure --prefix=/usr --libdir=/usr/lib64
   make

   (If you have installed 32-bit system then use /usr/lib path in the second command.)
5. Install the driver (will overwrite the files from RPM package)
   

   sudo make install

6. Restart the X server (simply relogin to a new session)
   

Voila, now your system should use the new driver and the artifacts in the KDE popups should be gone!

Maybe someone can pack the driver into a RPM package in the openSUSE build service, but for me this solution is sufficient...