Wiki

This pages summarises how this wiki might be made multi-lingual, and the steps required to achieve that.

When creating a new page, please also create an associated Discussion page based on the one for this page.

Summary

The Restart Project has an aspiration to make this wiki available in as many different lanuages as possible. Mediawiki, the software on which it's based, has extensions to facilitate multi-lingual support and a translation workflow. These are examined in this page, both from a user's and an implementor's point of view.

For the moment, this page comprises a repository of knowledge and implementation suggestions or proposals, but once multilingual support is implemented it will be changed into or replaced by a multilingual and translation guide for users, and hopefully the first page to be translated into any language.

Safety

There are no safety issues as such, except the risk that a safety warning might be mis-translated, altering its meaning or lessening its force.

Mediawiki Extensions

The MediaWiki Language Extension Bundle (MLEB) comprises 6 individual extensions covering various aspects of multi-lingual support:

UniversalLanguageSelector is foundational to the other extensions comprising MLEB, and adds a language selector to wiki pages.
Babel provides a set of categories by which a user can declare his/her language competencies, and a template to declare them.
cldr gives language and currency names against ISO country codes.
CleanChanges hides less important information in change histories by offering filtering by user and laguage code suffix.
LocalisationUpdate allows for updating of localisations for MediaWiki messages at any time, without needing to upgrade the MediaWiki software.
Translate provides the framework and workflow for the translation of content.

The MLEB page gives installation and basic configuration information. Additional changes to LocalSettings.php are required to configure permissions to our requirements.

A further extension, TranslationNotifications, provides a means by which translators may sign up for notifications about translations within their competance, sent by a Translation Administrator.

In addition to MLEB, ContentTranslation provides further facilities over and above Translate for the translation of content, typically for an entire page. (Further detail required.)

Yet another extension, TranslateWiki, allows entire pages to be automatically translated by Google TRanslate then manually reviewed and corrected.

All extensions are installed by the standard method, i.e. by expanding the extension's tarball into the extensions directory then enabling it in LocalSettings.php. A few require extra directives in LocalSettings.php, documented in the extension's Mediawiki page. In addition, translation permissions need to be defined in LocalSettings.php. All the additional settings that have been added during the translation evaluation execise are listed at the end of this page.

For most extensions there is nothing to add, but those with significant user-visible interfaces are described in the following sections.

Babel

This extension allows you to declare your language proficiencies by including a template in you user page. The template has two effects: it defines a nicely formatted box to display to declare your proficiencies, and it makes you a member of appropriate language groups.

The template takes the form:

 {{#babel: babelcode1 | babelcode2 | ... }}

Each babelcode declares a language proficiency in the form xx-n, where xx is a language code and n delclares a level of proficience as follows:

xx-0	If you don't understand the language at all.
xx-1	Basic ability - enough to understand written material or simple questions in this language.
xx-2	Intermediate ability - enough for editing or discussions.
xx-3	Advanced level - though you can write in this language with no problem, some small errors might occur.
xx-4	"Near-native" level - although it's not your first language from birth, your ability is something like that of a native speaker.
xx-5	Professional proficiency.
xx or xx-N	Native speakers who use a language every day and have a thorough grasp of it, including colloquialisms and idioms.

For example:

{{#babel: en | fr-2 | de-0 }}

declares native English, intermediate French, and zero German. (Declaring a zero competence in not often expected to be useful, but could be for a person living in a largely bi- or multi-lingual community.)

TranslationNotfications

This is additional to MLEB. It allows translators to sign up for messages about content newly requiring translation into languages within the user's competance.

Installation instructions are at TranslationNotifications.

This is dependant on the MassMessage extension, which leaves messages on users' Talk pages. The wiki installation needs to be able to send emails.

ContentTranslation

This, too, is additional to the MLEB.

Installation instructions are at ContentTranslation.

This is dependant on the VisualEditor extension, which in turn depends on Parsoid. These are bundled with mediawiki 1.35, but installation appears to be complicated on earlier versions, and there are hints it's problematic on Debian (which includes Ubuntu).

It seems that this would be a very desirable extension to have as it would reduce or eliminate the need for translators (and editors in general) to be conversant with wikimedia markup. However, since the potential installation problems are likely to be eliminated by Mediawiki 1.35 it would seem best to wait until then before exploring this further.

TranslateWiki

Yet another additional extension facilitating automatic translation by Google Translate would seem to be useful, but in the context of technical content it's likely that it would struggle. It has not been investigated further.

Translate

Translation lifecycle

A page follows thefollowing lifecycle:

The page is created.
Over a number of edits and possibly an extended timescale, the page settles to a fairly steady state with relatively few and infrequent changes.
A Translation Administrator will prepare the page for translation (Special Pages under the Translation heading). This results in the entire page being wrapped in tags and breaking it into individual translation units, generally corresponding to single paragraphs and headings. This is initially done automatically, but the Translation Administrator should perform a sanity check. If it appears ok, s/he will click Publish.
Under Special Pages - Page translation, the page will now appear under Pages proposed for translation. Against it will be a Mark for Translation link, which a Translation Administrator can click. The page is displayed, showing how it's been broken up into "translation units". If these seem reasonable the Translation Administrator can click Mark this version for Translation at the bottom of the page.
Any user can then click Translate (Special Pages under Translation). First select a target language. By default, at the top of the page it will say Message Group: All, and will display all the translation units that need translating. Click on All to select a particular page instead. NB on Mediawiki, "Translate this page" appears at the top of pages marked for translation. How do we get that?
Where translations were made by a user, a Translator can go into the same translation page and mark translations as reviewed.
Changes may be made to the original source page. This will result in translations being marked as requiring update and a return to the previous step but one.
At all stages, statistics are available showing the state, including % completion, of all translations.

Permissions

Objectives

Just as all logged in users can edit pages, subject to review, and in the same way, all users should be able to contribute and edit transations, again, subject to review.

The Editor group has special permission to approve edits, members of which need a good command of English and should be familiar with and able to identify with the conventions, philosopy and style of the wiki.

In the same way, the translator group members will need a professional or native command of the target language. They will approve translations, ensuring that they are to a high standard.

Translation Administrators will decide which pages are ready for translation, and will prepare and mark them for translation. They need an overall view of the wiki and will guide translations at a strategic level. Typically, they will already be Editors. It's probably desirable (though not strictly necessary) that they have intermediate level proficiency in a language other than their own so as to understand the problems and pitfalls of translating technical text whilst keeping it accessible.

Detailed permissions

Logged out: Can view pages and page source. (Seems to allow Translation interface! Probably not a good idea. Permission definitions need a little extra work.)

Autoconfirmed: Special:ImportTranslations enabled. Special:Translation enabled.

Has permissions translate, translate-import.
Can edit pages, and translate pages marked as for translation.

Editor:

... and can approve edits by autoconfirmed users.

Translator:

... and can review and approve translations by autoconfirmed users.

Translate-admin

... and can prepare and mark pages for translation.

Admin:

Can do all the above.

Bureaucrat

... and can assign permissions.

The required directives to achieve these permissions are included in the LocalSettings.php extract at the end of this page.

Pages in non-source languages

Pages are created in English by default. Occasionally it will be necessary to create a page in a different language. A prime example is the Glossary, which will be structured differently in different languages since the alphabetisation may differ (e.g. the French for AC and DC are CA and CD). Potentially, a page created in another language could be translated into English or other languages, for example a page created in French on croissant-baking machines could be translated into English, however, such examples are likely to be rare.

To enable the language of a page to be changed, in LocalSettings.php it is necessary to set:
$wgPageLanguageUseDB = true

Additionally, users need the permission pagelang in order to change a page's language. Users with this privilege will see Change Page Language under Page Tools in Special Pages. Whether this should be given to al users able to create new pages or reserved for Translators or Translation Administrators is not yet clear.

Worked example

The following users are set up, all with the same password:

Norman: Autoconfirmed User
Graham: Translator, Editor
Henry: Translation Admin, Translator, Editor

Outline This example, worked in detail below, takes us through the following steps:

An Editor creates a new page.(This could have been created by an autoconfirmed user then approved by an Editor, and normally it would be given time to settle down to a stable state.)
A Translation Admin Prepares the page for translation (making technical changes to the source) then Marks it for translation, releasing it into the translation process.
- If the page has previously been translated on another wiki, you can import the translation at this point, using a .po file exported from another wiki.
An Autoconfirmed user translates some or all of the page.
A Translator reviews, corrects (if necessary) and approves the translation.
To a user not logged in, the page now appears Protected.
An Autoconfirmed user nevertheless can edit the source language page, and makes changes.
A Translation Admin (having satisfied himself no more edits are immediately forthcoming) once again Marks the page for translation.
A Translator notes Outdated translations and updates them.

Orientation

Login as Norman. In the search box, search for Style Guide.
Notice at the top of the page there is a box Other Languages, indicating which languages this page is available in. Against each is a symbol ranging from a small grey square meaning untranslated through to a 2x2 square in green meaning fully transated. Click francais to see the French version.
Notice also at the top of the page against your logged-in user name it will say English. Click this to select another language for navigation and messages.

Translation Example in detail

Login as Graham (Editor). In the search box, type "just some stuff" and go to the page of that name. Click on Edit at the top of the page, select all the text and copy it to the clipboard.
Search for page "stuff-pierre", replacing pierre with your name. Take the option to create such a page, paste the clipboard into it and save it. You now have your own page which you can try translating.
Logout, and login as Henry (Translate Admin).
Go to Special Pages, and under Translation at the bottom, select Prepare page for translation. Enter Stuff-pierre and click Prepare.
You will see that a
Other languages:
- English
tag has been inserted at the start, and everything that needs to be translated is enclosed in ... tags. Check that this looks sensible, i.e. everything that needs to be translated is so enclosed and anything that shouldn't be (such as urls and wiki links) is excluded. Click the Publish button at the top.
Look at the page history. You will see a change "Prepared the page for translation". If you wanted to, you could undo that change.
Return to Special Pages, and under Translation, select Page Translation. You will be shown your page under Pages proposed for translation, and probably some others under Pages in translation. Under your page, click Mark for translation.
You will see that each heading and paragraph is shown as a separate "translation unit", allowing it to be translated independently of other translation units. Click Mark this version for translation, at the bottom of the page.
Under Special Pages, and under Translation, select Page Translation. You will now see your page under Pages in Translation
If you change your mind, you have the option to Remove the page from Translation, but if you change your mind back it seems to get in a muddle.
Logout and login as Norman (an autoconfirmed user).
Go to Special Pages, and under Translation, select Translate. Initially, you will be shown a list of all translation units in all pages. At the top, where it says Message group All, click All and select stuff-pierre.
At the top of the page, where it says Translate to, select your target language.
Each translation unit will be presented in its source language, with a box below in which to type your translation. You can go through them in sequence, or select any of your choice by clicking Edit to the right of the translation unit. After translating each translation unit, click Publish. You will be moved to the next. For some, it may give you a suggestion if the same text has been translated before.
Continue translating just a few translation units, but not all.
Return to Stuff-Pierre. If you select your target language version you will not see your translations as they have not yet been approved.
Logout and login as Graham (Editor/Translator). Under Special Pages, select Language Statstics. You can select Stuff-Pierre here or under Special Pages select Translate. (It's useful to see both methods.)
Ensure that the target language is correct. At the bottom of the page you will see List - Page - Review. Click Review. Return to the top of the page, and from All - Outdated - Translated - Unreviewed select Unreviewed.
Now review each translation unit. For each, you can either click the tick mark to mark it as reviewed, or you can click the slash to edit it and then mark as reviewed.
Logout. You will now see Stuff-Pierre partially translated. If you hover over View Source you will see that the page is marked as Protected, and cannot be edited.
Login as Norman (Autoconfirmed user). Select the target language version of Stuff-Pierre. At the head of the page where you normally see Edit, you will see Translate. You can select this to continue translating, but to save having to review the translations as a separate step, login as Graham (Translator/Editor) and complete the translation.
Logged in as Graham (Editor/Translator), select the English version of Stuff-Pierre. You can edit it, but get a warning that you are editing a translatable page. Make one or more minor changes.
Under Special - Page Translation, Stuff-Pierre will show as having changes not marked for translation.
Logout and login as Henry (Translation Admin). Under Special - Page Translation, mark the page for translation.
Logged in as Graham (Translator/Editor), ender Special - Translation, outdated translation units will have a clock icon against them. Update the translations.
Return to the target language version and confirm that your latest changes have been translated.

LocalSettings.php

The following additions have been made to LocalSettings.php:

# Now for my translation extensions
wfLoadExtension( 'Babel' );

wfLoadExtension( 'cldr' );

wfLoadExtension( 'CleanChanges' );
$wgCCTrailerFilter = true;
$wgCCUserFilter = false;
$wgDefaultUserOptions['usenewrc'] = 1;

wfLoadExtension( 'LocalisationUpdate' );
$wgLocalisationUpdateDirectory = "$IP/cache";

wfLoadExtension( 'Translate' );
$wgGroupPermissions['*']['translate'] = false;
$wgGroupPermissions['*']['translate-manage'] = false;
$wgGroupPermissions['*']['translate-import'] = false;
$wgGroupPermissions['*']['pagetranslation'] = false;
$wgGroupPermissions['*']['translate-messagereview'] = false;
$wgGroupPermissions['*']['translate-groupreview'] = false;
$wgGroupPermissions['user']['translate'] = true;
$wgGroupPermissions['user']['translate-import'] = true;
$wgGroupPermissions['translator'] = $wgGroupPermissions['user'];
$wgGroupPermissions['translator']['translate-messagereview'] = true;
$wgGroupPermissions['translator']['skipcaptcha'] = true; // Bug 34182: needed with ConfirmEdit
$wgGroupPermissions['translation-admin'] = $wgGroupPermissions['translator'];
$wgGroupPermissions['translation-admin']['translate-manage'] = true;
$wgGroupPermissions['translation-admin']['pagetranslation'] = true;
$wgGroupPermissions['translation-admin']['translate-groupreview'] = true;
$wgGroupPermissions['sysop']['translate'] = true;
$wgGroupPermissions['sysop']['translate-manage'] = true;
$wgGroupPermissions['sysop']['translate-import'] = true;
$wgGroupPermissions['sysop']['pagetranslation'] = true;
$wgGroupPermissions['sysop']['translate-messagereview'] = true;
$wgGroupPermissions['sysop']['translate-groupreview'] = true;

$wgTranslateDocumentationLanguageCode = 'qqq';
$wgExtraLanguageNames['qqq'] = 'Message documentation'; # No linguistic content. Used for documenting messages

# Page translation feature
$wgEnablePageTranslation = true;
$wgPageTranslationNamespace = 1198;

wfLoadExtension( 'UniversalLanguageSelector' );

# Cause the language selector to select content language
# as well as navigation and messages
$wgTranslatePageTranslationULS = true;

wfLoadExtension( 'TranslationNotifications' );
$wgTranslationNotificationsContactMethods = [
        'email' => true,
        'talkpage' => true,
        'talkpage-elsewhere' => false,
];
# TranslationNotifications requires the following cron job to be set up (change times as req'd)
# 0 10 * * 7 /usr/bin/php /var/www/html/mediawiki/wiki/extensions/TranslationNotifications/scripts/DigestEmailer.php 2>&1 > /var/log/digestEmail.log

# MassMessage required by TranslationNotifications
wfLoadExtension( 'MassMessage' );
$wgMassMessageAccountUsername = 'MessengerBot';
$wgNamespacesToConvert = [ NS_USER => NS_USER_TALK ];

#wfLoadExtension( 'ContentTranslation' );