Translating Between Different Encodings

When creating a distributed application, it may happen that some components and the server use different encodings. In this case, it is necessary to translate between the different encodings.

Note: This translation only refers to translating from one character encoding to another, not to translating the strings from one language to another.

The Rogue Wave Server communication protocol provides a mechanism that registers translators. This mechanism is called automatically when necessary.

The method IlsTranslationMng::DeclareTranslation is provided to declare a translation between the current locale and another locale or between the current locale and an external (and usually standard) encoding format.

The following instruction declares a translation between the UTF-8 external format and the current locale, passing the associated encoding and decoding functions as parameters:

A translation is defined by a name, an encoding function and a decoding function.

There is no constraint as to the name used to declare a translation, except that:

When the name is a system-dependent locale name, Rogue Wave Server automatically associates a standard locale name with the translation.

The encoding function is used each time a string is inserted into an Rogue Wave Server message (see class IlsMvMessage).

Both functions take a constant array of bytes as input and return an array of bytes.

The dimension of the input array is passed by reference as the second argument to these functions. It does not include the terminating null character.

The return array must be assigned the dimension of the resulting array. It should not include the terminating null character.

Rogue Wave Server is responsible for freeing the memory allocated to both the input and the output arrays.

During the connection phase between a component and a server, Rogue Wave Server detects the possible incompatibility between the locales used by the two processes. In this case, Rogue Wave Server tries to find a translation that allows the two processes to interpret the strings exchanged.

Let us assume a connection from a component to a server. The component uses a locale whose system name is CL. The server uses a locale whose system name is SL. CL and SL are assumed to be different and associated with different standard locale names, respectively CSL and SSL.

In this example, the algorithm used to select the translation works as follows:

1. This algorithm first looks for a translation named SL, or named SSL, among the translations declared by the component.

2. If step 1 fails, the algorithm looks for a translation named CL, or CSL, if any, among the translations declared by the server.

3. Finally, if step 2 fails, the algorithm looks for the first translations with the same names among the translations declared by the two processes.

If no translation is selected during this process, the string is exchanged between the two processes without any translation. In this case, you can also decide to refuse the connection. You must then declare a callback used in case of failure of the previous algorithm. See the static member function IlsTranslationMng::SetFailCb for details.

Rogue Wave Server provides a mechanism to allow you to specify in a configuration file that certain locales are compatible. If two differing locales are declared compatible, Rogue Wave Server will not try to translate from one encoding to the other. This mechanism is provided as a fallback method to prevent Rogue Wave Server from printing out messages warning that no compatible translation was found.

To provide a list of compatible locales, you need to create a file called locale.equiv and install it in the directory data/locale/<ll_TT.encoding>. If necessary, you can also create a file called equiv.all which will be read for all locales. For example, if you work in an ISO 8859-1 French environment, the following files will be loaded if they exist:

The format of the files equiv.loc and equiv.all is as shown below:

Note: If you are using Rogue Wave Server trace mode, you will see informational messages saying that these files were (or were not) read. These messages are there to help debug in the case where your locale equivalence files do not work as you expect.

This example shows how to add an equivalent locale: all you have to do is add a new line that contains the name of the equivalent locale. As can be seen from the example above, blank lines are accepted and comment lines are lines that start with the # character. On Windows® (NOT Unix®), it is possible to convert a locale to UTF-8 using a built-in converter. This converter is expensive, so it should be used with care. Adding a line that contains just UTF-8 would imply that no conversion will take place between UTF-8 and the current locale.

Note: The encoding used to write those locale equivalence files must be compatible with the current locale that will be reading the files. This is normally the case, as the file equiv.loc is only read when the current locale matches the locale where the file is stored. The file equiv.all should contain only ASCII so as to be compatible with all possible locales.