I decided to work on WebYaST home page and finish the WebYaST demo appliance.

Some time ago we had a workshop where we decided to create cool web pages for our projects. We started with WebYaST. The goal is to create a nice looking web presentation.

And we want to have some WebYaST demo so users could easily try it without any setup or installation. So we decided to create a WebYaST demo appliance in SUSE Studio which can be used as a LiveCD or USB stick or even directly in Studio testdrive in a web browser (no need to download anything!).

Including WebYaST in an openSUSE-12.1 SUSE Studio Appliance

This is really easy as Studio has WebYaST support built-in, just go into Configuration -> Appliance tab and check Enable WebYaST check box at the very bottom of the page. And that's it!

Studio will add all needed WebYaST packages, opens port at firewall (54984) and autostarts WebYaST at boot.

But if we want to have a really nice demo we still need to do some improvements...

Appliance Fine-tuning

Originally I started with KDE desktop but WebYaST basically does not depend on any desktop environment so I switched to LXDE which takes less space and should run faster than KDE (especially on slower machines).

SUSE Studio supports autologin configuration (Configuration -> Desktop) so users do not have to enter any password to start graphical session, that's nice.

Another nice feature is automatic application start, so we can easily start Firefox. The only problem was that in testdrive WebYaST was sometimes started later than Firefox which obviously displayed error page. This is solved by 5 second delay before starting Firefox.

Importing WebYaST Certificate into Firefox

This was the hard part and it's quite tricky. When you first time connect to a running WebYaST instance in Firefox you'll see a certificate warning. The problem is that WebYaST generates a self-signed certificate (when there is no existing certificate yet) which is not trusted so Firefox displays that warning. And this might be scary for beginner users, we want our users to try WebYaST without any doubts...

Then I found certutil tool available in mozilla-nss-tools package in openSUSE. This can be used to import a certificate to Firefox from command line. So for WebYaST this means running this command for the default user:

# certutil -A -n "Webyast certificate" -t "C,," -d /home/tux/.mozilla/firefox/*.default \
-i /etc/lighttpd/certs/webyast.pem

(See certutil --help for more details.)

The only problem is that Firefox uses profiles (named configurations) stored at random generated directory which is created at first start. And that directory must exist before executing the certutil command.

This is solved by overlay files in Studio, there is a prepared directory with the default Firefox profile.

Server Name in the Generated certificate

During tests I found out that there is a problem with certificate server name and the URL. Firefox displayed a certificate error when opening https://localhost:54984 with message saying that the certificate is valid for linux-foobar server only (with foobar replaced by some random characters).

The problem is that the random hostname is automatically generated and we cannot easily change that in the URL for Firefox.

The trick is to use IP address directly instead of a host name. The WebYaST certificate in the demo appliance is generated for host 127.0.0.1 and URL for Firefox is set to https://127.0.0.1:54984.

See yastwc overlay file for all certificate related changes.

The Final WebYaST Demo Appliance

The final WebYaST demo appliance is available in SUSE Studio Gallery. You can easily try WebYaST as a LiveCD/LiveUSB stick or directly in your browser in Studio testdrive.

And how to use it? That's really simple! Just boot the image, wait until Firefox with WebYaST login dialog opens. Then use root user name with linux password to log into WebYaST.

Have a lot of fun!

From Gettext to FastGettext

In SLMS we use Gettext for i18n support. Unfortunately it doesn't work with new Rails 3. But we found out that there is FastGettext Ruby gem which does work with Rails 3 and we decided to switch to this different implementation.

In this blogpost I'll describe the needed steps when switching from Gettext to FastGettext. And here also are solutions for some problems we found during the transition.

Using the new Ruby gems

You will need these new Ruby gems:

fast_gettext: https://github.com/grosser/fast_gettext
gettext_i18n_rails: https://github.com/grosser/gettext_i18n_rails (FastGettext Rails integration)
rails-i18n: https://github.com/svenfuchs/rails-i18n (Default Rails translations)

The first step is to remove the old Gettext gems and replace them by FastGettext gems.
So replace these gems in your Gemfile:

gem 'locale'
gem 'locale_rails'
gem 'gettext'
gem 'gettext_activerecord'
gem 'gettext_rails'

by:

gem 'fast_gettext'

# 0.4.3 contains fixes in
#'rake gettext:store_model_attributes' task
gem 'gettext_i18n_rails', '>= 0.4.3'

# rails-i18n provides translations for ActiveRecord
# validation error messages
gem 'rails-i18n'

# needed to collect translatable strings
# not needed at production
group :development do
  # needed for HAML support (optional)
  gem 'ruby_parser'

  # no need to load the gem via require
  # we only need the rake tasks
  gem 'gettext', '>= 1.9.3', :require => false
end

Then you need to initialize FastGettext, create config/initializers/fast_gettext.rb file:

# define your text domain
FastGettext.add_text_domain 'foo', :path => File.join(File.dirname(__FILE__), '..', '..', 'locale')


# set the default textdomain
FastGettext.default_text_domain = 'foo'

# set available locales
# (note: the first one is used as a fallback if you try to set an unavailable locale)
FastGettext.default_available_locales = ["en_US","ar","cs","de","es",...]

Replace foo with your textdomain. Now you need to add FastGettext initialization in your application controller:

class ApplicationController < ActionController::Base
  # replace these old Gettext calls:
  #   init_gettext "your_domain"
  #   GetText.textdomain("your_domain")
  # by this:
  include FastGettext::Translation

  before_filter :set_users_locale 

  def set_users_locale
    I18n.locale = FastGettext.set_locale(params[:locale] || cookies[:locale] ||
      request.env['HTTP_ACCEPT_LANGUAGE'] || 'en_US')
    cookies[:locale] = I18n.locale if cookies[:locale] != I18n.locale.to_s
  end
end

The set_users_locale before filter handles setting the correct locale for every request. The locale is set via a cookie and can be changed using ?locale=locale URL option. It is possible to use different solution for switching the locale, e.g. as path prefix aor domain name - see the Rails guide

Note: The application needs to be restarted after any change in the translations.

Solved Problems

Automatic detection of available locales

Using fixed list in the available locales list might not be nice, especially if you want to dynamically add new translations later. In this case you need to find the available locales dynamically at start. The solution si to put this code to config/initializers/fast_gettext.rb file:

# put 'en_US' as first, the first item is used as a fallback
# when requested locale (via ?locale= URL parameter) is not found
FastGettext.default_available_locales = ["en_US"]

# get available locales automatically
Dir[File.join(File.dirname(__FILE__), '..', '..', 'locale', "/*/LC_MESSAGES/*.mo")].each do |l|
  if l.match(/\/([^\/]+)\/LC_MESSAGES\/.*\.mo$/) && !FastGettext.default_available_locales.include?($1)
    FastGettext.default_available_locales << $1
  end
end

Language and Country Separator in locale name

Rails native localization support uses I18n module for translation support. The problem is that it uses dash (-) separator between langugage and country code in locale names.

This makes a problem when using with standard gettext locale schema which uses underscore (_) as the separator. For example translations from rails-i18n gem will not be found when the current locale in en_US, it expects en-US locale.

The problem can be solved by defining locale fallbacks like this (put this to config/initializers/fast_gettext.rb file):

# enable fallback handling
I18n::Backend::Simple.include(I18n::Backend::Fallbacks)

# set some locale fallbacks needed for ActiveRecord translations
# located in rails_i18n gem (e.g. there is en-US.yml translation)
I18n.fallbacks[:"en_US"] = [:"en-US", :en]
I18n.fallbacks[:"en_GB"] = [:"en-GB", :en]
I18n.fallbacks[:"pt_BR"] = [:"pt-BR", :pt]
I18n.fallbacks[:"zh_CN"] = [:"zh-CN"]
I18n.fallbacks[:"zh_TW"] = [:"zh-TW"]
I18n.fallbacks[:"sv"] = [:"sv-SE"]

This means that if for example a translation for en_US locale is not found then en-US will be tried and then en locale.

Including source file name and line number is the final POT file

By default when you run 'rake gettext:find' task to collect the translatable string the output will not contain the source file name and the line number. It's very useful if you get a feedback from translator (like a typo in the original message) then you don't have to scan all file but you immediately know where to fix the problem.

If you want to change this behavior and include the line numbers add this configuration to config/initializers/fast_gettext.rb file:

# configure default msgmerge parameters (the default contains "--no-location" option
# which removes code lines from the final POT file)
Rails.application.config.gettext_i18n_rails.msgmerge = ["--sort-output", "--no-wrap"]

Sorting messsages in the final POT file

The 'rake gettext:find' task sorts the messages in the final POT file alphabetically. The advantage is that if you add a new string and regenerate the file then the files will be similar and the diff will be small.

The problem is that the sorting is done at the merge step, when merging the new found translation wit the old ones. At the very first run (when the final POT file does not exist yet) the merge step is skipped and thus the messages are not sorted. This can be fixed by starting the task once more (the second run will find existing messages and do the merge with sorting).

But the problem is that you can easily forget to run the task for the second run. The workaround is to create an empty target POT file when the it doesn't exist yet. Unfortunately simple touch command is not sufficient (msgmerge failed for me with some strange UTF-8 error), we have to create valid POT but without any messages.

The workaround it to put this code to lib/tasks/gettext.rake file:

# 'gettext:find' sorts the messages alphabetically only when it is merging existing messages
# copying empty pot file from the template forces sorting even at the first run
namespace :gettext do
  task :create_pot_template do
    FileUtils.cp("locale/template.pot", "locale/textdomain.pot") unless File.exists?("locale/textdomain.pot")
  end
end

# add task dependency
task :'gettext:find' => :'gettext:create_pot_template'

The locale/textdomain.pot template should look like this:

# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <email@address>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: version 0.0.1\n"
"POT-Creation-Date: 2012-01-16 17:56+0100\n"
"PO-Revision-Date: 2012-01-16 17:56+0100\n"
"Last-Translator: FULL NAME <email@address>\n"
"Language-Team: LANGUAGE <ll@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\n"

Automatic translation in HAML files

It is possible to extend HAML parser to automatically translate all plain text strings. The advantage is than you don't have to explicitly use _() function and you cannot forget to mark a text to translation.

This can be done using this code snippet. Save it to a file, remove the require calls at the beginning (they are obsoleted and do not work with new gettext) and require it in your ApplicationController.

Then you need to add support to 'rake gettext:find' task. Save this code snippet to lib/haml_parser.rb file. You need to replace require 'gettext/parser/ruby' by require 'gettext/tools/parser/ruby' so it works with newer gettext gem.

Then put this to lib/tasks/gettext.rake file:

# extend the HAML parser to extract plain text messages
# to support automatic translations (without need to mark the text with _())
namespace :gettext do
  task :haml_parser do
    require 'haml_parser'
  end
end

# extend the HAML parser before collecting the translatable texts
task :'gettext:find' => :'gettext:haml_parser'

Ladislav's blog

Tuesday, July 31, 2012

OpenSUSE Hackweek VIII - New WebYaST home page

OpenSUSE Hackweek VIII - New WebYaST Demo Appliance